[Core ML] セグメンテーションモデルの比較

2023年8月28日 19:41

フォトグラメトリ用の空マスクを生成するために、セグメンテーション系のCore MLモデルをいろいろと試したメモ。

こちらのツイート（のスレッドにぶら下がっているツイート）によると、

手持ちカメラで広域フォトグラメトリしたモデルの後処理を極力手作業なしでやるフローをちょっと模索したので結果共有。映像は、未処理モデル -> 建物領域だけを抜き出すマスクテクスチャ -> アルファカットで建物領域のみを表示したモデル、です pic.twitter.com/fjDHNi4Cm1
— ノーベルチョコ (@nobelchoco) June 8, 2019

PSPNetのkeras実装、ADE20Kによる学習済みモデルを使用

とのことで、データセットとして ADE20K 、もしくは CityScapes を利用しているセグメンテーションモデルを比較検討してみることにした。

（CityScapesもラベルに "sky" を含んでいたので対象とした）

CityScapesのラベル一覧：

["road", "sidewalk", "building", "wall", "fence", "pole", "traffic light", "traffic sign", "vegetation", "terrain", "sky", "person", "rider", "car", "truck", "bus", "train", "motorcycle", "bicycle"]

ADE20Kのラベル一覧：

["wall", "building", "sky", "floor", "tree", "ceiling", "road", "bed ", "windowpane", "grass", "cabinet", "sidewalk", "person", "earth", "door", "table", "mountain", "plant", "curtain", "chair", "car", "water", "painting", "sofa", "shelf", "house", "sea", "mirror", "rug", "field", "armchair", "seat", "fence", "desk", "rock", "wardrobe", "lamp", "bathtub", "railing", "cushion", "base", "box", "column", "signboard", "chest of drawers", "counter", "sand", "sink", "skyscraper", "fireplace", "refrigerator", "grandstand", "path", "stairs", "runway", "case", "pool table", "pillow", "screen door", "stairway", "river", "bridge", "bookcase", "blind", "coffee table", "toilet", "flower", "book", "hill", "bench", "countertop", "stove", "palm", "kitchen island", "computer", "swivel chair", "boat", "bar", "arcade machine", "hovel", "bus", "towel", "light", "truck", "tower", "chandelier", "awning", "streetlight", "booth", "television receiver", "airplane", "dirt track", "apparel", "pole", "land", "bannister", "escalator", "ottoman", "bottle", "buffet", "poster", "stage", "van", "ship", "fountain", "conveyer belt", "canopy", "washer", "plaything", "swimming pool", "stool", "barrel", "basket", "waterfall", "tent", "bag", "minibike", "cradle", "oven", "ball", "food", "step", "tank", "trade name", "microwave", "pot", "animal", "bicycle", "lake", "dishwasher", "screen", "blanket", "sculpture", "hood", "sconce", "vase", "traffic light", "tray", "ashcan", "fan", "pier", "crt screen", "plate", "monitor", "bulletin board", "shower", "radiator", "glass", "clock", "flag"]

なおCore MLモデルはだいすけ氏の CoreML-Models リポジトリにあるものを利用。