SynthDet: An end-to-end object detection pipeline using synthetic data

Overview

SynthDet is an open source project that demonstrates an end-to-end object detection pipeline using synthetic image data. The project includes all the code and assets for generating a synthetic dataset in Unity. Using recent research, SynthDet utilizes Unity's Perception package to generate highly randomized images of 64 common grocery products (example: cereal boxes and candy) and export them along with appropriate labels and annotations (2D bounding boxes). The synthetic dataset generated can then be used to train a deep learning based object detection model. This project is geared towards ML practitioners and enthusiasts who are actively exploring synthetic data or just looking to get started.

Getting started with SynthDet

Contents

• SynthDet - Sample computer vision data generation project using Unity's Perception package
• 3D Assets - High quality models of 64 commonly found grocery products
• Unity's Perception package
• Unity Dataset Insights Python package

Inspiration

SynthDet was inspired by the following research paper from Google Cloud AI:

Hinterstoisser, S., Pauly, O., Heibel, H., Marek, M., & Bokeloh, M. (2019). An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection.

Support

For general questions or concerns please contact the Unity Computer Vision team at [email protected].

For feedback, bugs, or other issues please file a GitHub issue and the Unity Computer Vision team will investigate the issue as soon as possible.

Citation

If you find this package useful, consider citing it using:

@misc{synthdet2020,
    title={Training a performant object detection {ML} model on synthetic data using {U}nity {P}erception tools},
    author={You-Cyuan Jhang and Adam Palmar and Bowen Li and Saurav Dhakad and Sanjay Kumar Vishwakarma and Jonathan Hogins and Adam Crespi and Chris Kerr and Sharmila Chockalingam and Cesar Romero and Alex Thaman and Sujoy Ganguly},
    howpublished = {\url{https://blogs.unity3d.com/2020/09/17/training-a-performant-object-detection-ml-model-on-synthetic-data-using-unity-computer-vision-tools/}},
    journal={Unity Techonologies Blog},
    publisher={Unity Technologies},
    year={2020},
    month={Sep}
}

Additional Resources

GTC 2020: Synthetic Data: An efficient mechanism to train Perception Systems

Synthetic data: Simulating myriad possibilities to train robust machine learning models

Use Unity’s computer vision tools to generate and analyze synthetic data at scale to train your ML models

Training a performant object detection ML model on synthetic data using Unity computer vision tools

License

Apache License 2.0