We use [WebDataset](https://github.com/webdataset/webdataset) to process all datasets for training purposes, as illustrated below. We assume $DATA_PATH is where the datasets are stored and $SAVE_PATH is where the processed wds shards to be saved.

## Expected dataset structure for [COCO Panoptic](https://cocodataset.org/#home):

```
coco/
  annotations/
    panoptic_{train,val}2017.json
  {train,val}2017/ # image files that are mentioned in the corresponding json
  panoptic_{train,val}2017/  # png annotations
```

Run the following commands to convert COCO Panoptic dataset to wds format. Note that we filter some images (15k) from COCO train set that overlaps with LVIS val set.
```bash
python3 convert_coco_panoptic_to_wds.py \
    --output_dir $SAVE_PATH/coco_pan_wds_exclude_lvisval \
    --ann_dir $DATA_PATH/coco/panoptic_train2017 \
    --image_dir $DATA_PATH/coco/train2017 \
    --json_file $DATA_PATH/coco/annotations/panoptic_train2017.json \
    --filter_lvis_val True
```

## Expected dataset structure for [ADE20K Panoptic](http://sceneparsing.csail.mit.edu/):

Firstly, please follow [scripts here](https://github.com/bytedance/fc-clip/tree/main/datasets#expected-dataset-structure-for-ade20k-a150) to obtain merged panoptic annotations for ADE20K datasets.

```
ADEChallengeData2016/
  images/
  # below are generated by prepare_ade20k_pan_seg.py
  ade20k_panoptic_{train,val}.json
  ade20k_panoptic_{train,val}/
```

Then, run the following commands to convert ADE20K Panoptic dataset to wds format.

```bash
python3 convert_coco_panoptic_to_wds.py \
    --output_dir $SAVE_PATH/ade20k_pan_wds \
    --ann_dir $DATA_PATH/ADEChallengeData2016/ade20k_panoptic_train \
    --image_dir $DATA_PATH/ADEChallengeData2016/images/training \
    --json_file $DATA_PATH/ADEChallengeData2016/ade20k_panoptic_train.json
```

## Expected dataset structure for [Cityscapes Panoptic](https://www.cityscapes-dataset.com/downloads/):

Firstly, please follow [scripts here](https://github.com/bytedance/fc-clip/blob/main/datasets/README.md#expected-dataset-structure-for-cityscapes) to obtain panoptic annotations for Cityscapes datasets.

```
cityscapes/
  gtFine/
    train/
      aachen/
        color.png, instanceIds.png, labelIds.png, polygons.json,
        labelTrainIds.png
      ...
    val/
    test/
    # below are generated Cityscapes panoptic annotation
    cityscapes_panoptic_train.json
    cityscapes_panoptic_train/
    cityscapes_panoptic_val.json
    cityscapes_panoptic_val/
    cityscapes_panoptic_test.json
    cityscapes_panoptic_test/
  leftImg8bit/
    train/
    val/
    test/
```

Then, run the following commands to convert Cityscapes Panoptic dataset to wds format.

```bash
python3 convert_cityscapes_panoptic_to_wds.py \
    --output_dir $SAVE_PATH/cityscapes_pan_wds \
    --ann_dir $DATA_PATH/cityscapes/gtFine/cityscapes_panoptic_train \
    --image_dir $DATA_PATH/cityscapes/leftImg8bit/train \
    --json_file $DATA_PATH/cityscapes/gtFine/cityscapes_panoptic_train.json
```

## Expected dataset structure for [LVIS Instance](https://www.lvisdataset.org/):

```
lvis/
  {train,val}2017/ # image files are the same as COCO, you can soft link COCO files here to save disk space
  lvis_v1_train.json
  lvis_v1_val.json
```

Run the following commands to convert LVIS Instance dataset to wds format.
```bash
python3 convert_lvis_to_wds.py \
    --output_dir $SAVE_PATH/lvis_wds \
    --image_dir $DATA_PATH/lvis \
    --json_file $DATA_PATH/lvis/lvis_v1_train.json
```


## Expected dataset structure for [A-847 Semantic](https://groups.csail.mit.edu/vision/datasets/ADE20K/):

Firstly, please follow [scripts here](https://github.com/bytedance/fc-clip/tree/main/datasets#expected-dataset-structure-for-ade20k-full-a-847) to obtain semantic annotations in detectron2 format for A-847 datasets.

```
ADE20K_2021_17_01/
  images/
  index_ade20k.pkl
  objects.txt
  # generated by prepare_ade20k_full_sem_seg.py
  images_detectron2/
  annotations_detectron2/
```

Run the following commands to convert A-847 Semantic dataset to wds format.
```bash
python3 convert_ade_847_to_wds.py \
    --output_dir $SAVE_PATH/a847_wds \
    --image_dir $DATA_PATH/ADE20K_2021_17_01/images_detectron2/training  \
    --ann_dir $DATA_PATH/ADE20K_2021_17_01/annotations_detectron2/training
```

## Expected dataset structure for [PC-459 Semantic](https://www.cs.stanford.edu/~roozbeh/pascal-context/):

Firstly, please follow [scripts here](https://github.com/bytedance/fc-clip/tree/main/datasets#expected-dataset-structure-for-pascal-context-pc-59-pascal-context-full-pc-459-and-pascal-voc-pas-21) to obtain semantic annotations in detectron2 format for PC-459 datasets.

```
VOCdevkit/
  pascal_ctx_d2/
    images/ # generated by prepare_pascal_ctx_full_sem_seg.py
    annotations_ctx459/
```

Run the following commands to convert PC-459 Semantic dataset to wds format.
```bash
python3 convert_ade_847_to_wds.py \
    --output_dir $SAVE_PATH/pc459_wds \
    --image_dir $DATA_PATH/pascal_ctx_d2/images/training  \
    --ann_dir $DATA_PATH/pascal_ctx_d2/annotations_ctx459/training
```

## Expected dataset structure for [Part-ImageNet Semantic](https://github.com/TACJu/PartImageNet):

```
PartImageNet/
  images/
  annotations/
```

Run the following commands to convert Part-ImageNet Semantic dataset to wds format.
```bash
python3 convert_partimagenet_to_wds.py \
    --output_dir $SAVE_PATH/part_imagenet_wds \
    --image_dir $DATA_PATH/PartImageNet/images
```

## Expected dataset structure for [Pascal-Part Semantic](https://roozbehm.info/pascal-parts/pascal-parts.html):

```
pascal_part/
  VOCdevkit/
  Annotations_Part/
```

Run the following commands to convert Pascal-Part Semantic dataset to wds format.
```bash
python3 convert_pascalpart_to_wds.py \
    --output_dir $SAVE_PATH/pascal_part_wds \
    --image_dir $DATA_PATH/pascal_part/VOCdevkit/VOC2010/JPEGImages \
    --ann_dir $DATA_PATH/pascal_part/Annotations_Part
```

## Expected dataset structure for [V3Det Detection](https://github.com/V3Det/V3Det):

```
V3Det/
  images/
  annotations/
  records.json
```

Run the following commands to convert Part-ImageNet Semantic dataset to wds format.
```bash
python3 convert_v3det_to_wds.py \
    --output_dir $SAVE_PATH/v3det_wds \
    --image_dir $DATA_PATH/V3Det \
    --json_file $DATA_PATH/V3Det/annotations/v3det_2023_v1_train.json
```