Part-aware Panoptic Segmentation

v2.0 Release Candidate

[CVPR 2021 Paper] [Datasets Technical Report] [Documentation]

This repository contains code and tools for reading, processing, evaluating on, and visualizing Panoptic Parts datasets. Moreover, it contains code for reproducing our CVPR 2021 paper results.

Datasets

Cityscapes-Panoptic-Parts and PASCAL-Panoptic-Parts are created by extending two established datasets for image scene understanding, namely Cityscapes and PASCAL datasets. Detailed description of the datasets and various statistics are presented in our technical report in arxiv. The datasets can be downloaded from:

• Cityscapes Panoptic Parts
• PASCAL Panoptic Parts (alternative link, code: i7ap)

Examples

More examples here.

Installation and usage

The code can be installed from the PyPI and requires at least Python 3.7. It is recommended to install it in a Python virtual environment.

pip install panoptic_parts

Some functionality requires extra packages to be installed, e.g. evaluation scripts (tqdm) or Pytorch/Tensorflow (torch/tensorflow). These can be installed separately or by downloading the optional.txt file from this repo and running the following command in the virtual environment:

pip install -r optional.txt

After installation you can use the package as:

import panoptic_parts as pp

print(pp.VERSION)

There are three scripts defined as entry points by the package:

pp_merge_to_panoptic <args>
pp_merge_to_pps <args>
pp_visualize_label_with_legend <args>

API and code reference

We provide a public, stable API, and various code utilities that are documented here.

Reproducing CVPR 2021 paper

The part-aware panoptic segmentation results from the paper can be reproduced using this guide.

Evaluation metrics

We provide two metrics for evaluating performance on Panoptic Parts datasets.

• Part-aware Panoptic Quality (PartPQ): here.
• Intersection over Union (IoU): TBA

Citations

Please cite us if you find our work useful or you use it in your research:

@inproceedings{degeus2021panopticparts,
    title = {Part-aware Panoptic Segmentation},
    author = {Daan de Geus and Panagiotis Meletis and Chenyang Lu and Xiaoxiao Wen and Gijs Dubbelman},
    booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year = {2021}
}

@article{meletis2020panopticparts,
    title = {Cityscapes-Panoptic-Parts and PASCAL-Panoptic-Parts datasets for Scene Understanding},
    author = {Panagiotis Meletis and Xiaoxiao Wen and Chenyang Lu and Daan de Geus and Gijs Dubbelman},
    type = {Technical report},
    institution = {Eindhoven University of Technology},
    date = {16/04/2020},
    url = {https://github.com/tue-mps/panoptic_parts},
    eprint={2004.07944},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Contact

Please feel free to contact us for any suggestions or questions.

[email protected]

The Panoptic Parts datasets team

Correspondence: Panagiotis Meletis, Vincent (Xiaoxiao) Wen