Skip to content

Latest commit

 

History

History
122 lines (95 loc) · 4.4 KB

README.md

File metadata and controls

122 lines (95 loc) · 4.4 KB
Raw

Prepare Datasets for Training RbA

General Overview

In order to train a model on known data from scratch, we use the same setup as followed by Mask2Former. A dataset can be used by accessing DatasetCatalog for its data, or MetadataCatalog for its metadata (class names, etc). This document explains how to setup the builtin datasets so they can be used by the above APIs. Use Custom Datasets gives a deeper dive on how to use DatasetCatalog and MetadataCatalog, and how to add new datasets to them.

MaskFormer has builtin support for a few datasets. The datasets are assumed to exist in a directory specified by the environment variable DETECTRON2_DATASETS. Under this directory, detectron2 will look for datasets in the structure described below, if needed.

$DETECTRON2_DATASETS/
  cityscapes/
  mapillary_vistas/
  coco/

You can set the location for builtin datasets by export DETECTRON2_DATASETS=/path/to/datasets. If left unset, the default is ./datasets relative to your current working directory.

Expected dataset structure for cityscapes:

cityscapes/
  gtFine/
    train/
      aachen/
        color.png, instanceIds.png, labelIds.png, polygons.json,
        labelTrainIds.png
      ...
    val/
    test/
  leftImg8bit/
    train/
    val/
    test/

Install cityscapes scripts by:

pip install git+https://github.com/mcordts/cityscapesScripts.git

Note: to create labelTrainIds.png, first prepare the above structure, then run cityscapesescript with:

CITYSCAPES_DATASET=/path/to/abovementioned/cityscapes python cityscapesscripts/preparation/createTrainIdLabelImgs.py

Expected dataset structure for Mapillary Vistas:

mapillary_vistas/
  training/
    images/
    labels/
  validation/
    images/
    labels/

No preprocessing is needed for Mapillary Vistas on semantic and panoptic segmentation.

Expected dataset structure for COCO open-set-panoptic:

coco/
  annotations/
    panoptic_{train,val}2017.json
    ood_seg_train2017/
  {train,val}2017/
    # image files that are mentioned in the corresponding json
  panoptic_{train,val}2017/  # png annotations
  panoptic_semseg_{train,val}2017/  # generated by the script mentioned below

Install panopticapi by:

pip install git+https://github.com/cocodataset/panopticapi.git

Then, run python datasets/prepare_coco_semantic_annos_from_panoptic_annos.py, to extract semantic annotations from panoptic annotations (only used for evaluation).

Preparing coco_seg_train2017 for Outlier Supervision

We following Meta-OoD and use their script to generate the binary segmentation masks for potential OoD objects from COCO dataset to be pasted on the driving scenes. The chosen objects are such that they do not semantically overlap with the known classes in Cityscapes.

Prepare OoD Segmentation Datasets for Evaluation

Road Anomaly

In order to use the custom torch dataset defined in datasets/road_anomaly.py the dataset is assumed to follow the following structure:

RoadAnomaly/
  RoadAnomaly_jpg/
    frame_list.json
    frames/
      image_name/
        labels_semantic.png
      image_name.jpg  # `image_name` here is a placeholder for any image name
      ...

The custom dataset is used for evaluation and the root containing the RoadAnomaly/ folder can be passed during evaluation to the script.

Fishyscapes LaF

In order to use the custom torch dataset defined in datasets/fishyscapes.py the dataset is assumed to have the following structure:

Fishyscapes/
  fishyscapes_lostandfound/ # contains labels
  laf_images/ # contains images

The laf_images folder can be created by matching the label names in fishyscapes_lost_andfound with the images from the LostAndFound dataset.

The root that contains Fishyscapes/ can be passed dynamically to the evaluation script.