Nvidia Dataset Utilities (NVDU)

This project is a collection of Python scripts to help work with datasets for deep learning. For example, visualizing annotation data associated with captured sensor images generated by NVIDIA Deep learning Dataset Synthesizer (NDDS) https://github.com/NVIDIA/Dataset_Synthesizer.

This module depends on OpenCV-python which currently doesn't work with Python 3.7; this module requires either Python 3.5 or 3.6.

Example of a dataset frame visualized using NVDU, showing axes and 3D cuboids for annotated objects.

Table of Contents

• Nvidia Dataset Utilities (NVDU)
  • Table of Contents
• Install
  • Install from pip:
  • Install from source code git repo:
• nvdu_ycb
  • Usage
• nvdu_viz
  • Usage
  • Examples
    • Visualize a dataset generated by NDDS:
    • Visualize a set of images using different annotation data:
  • Controls
    • Visualization options:
    • Other:

Install

Install from pip:

pip install nvdu

Install from source code git repo:

Clone the repo

Using ssh path:

git clone ssh://[email protected]:12051/NVIDIA/Dataset_Utilities.git

Using https path:

git clone https://github.com/NVIDIA/Dataset_Utilities.git

Go inside the cloned repo's directory

cd Dataset_Utilities

Install locally

pip install -e .

nvdu_ycb

nvdu_ycb command help download, extract and align the YCB 3d models (which are used in the FAT dataset: http://research.nvidia.com/publication/2018-06_Falling-Things).

Usage

usage: nvdu_ycb [-h] [-s] [-l] [ycb_object_name]

NVDU YCB models Support

positional arguments:
  ycb_object_name  Name of the YCB object to check.

optional arguments:
  -h, --help       show this help message and exit
  -s, --setup      Setup the YCB models for the FAT dataset
  -l, --list       List all the supported YCB objects

NOTE: If you don't run the nvdu_ycb --setup before trying to use nvdu_viz, the visualizer will not be able to find the 3d models of the YCB object to overlay.

nvdu_viz

nvdu_viz command visualizes the annotated datasets using the NDDS format.

Usage

nvdu_viz [-h] [-a DATA_ANNOT_DIR] [-s SIZE SIZE]
                [-o OBJECT_SETTINGS_PATH] [-c CAMERA_SETTINGS_PATH]
                [-m MODEL_DIR] [-n [NAME_FILTERS [NAME_FILTERS ...]]]
                [--fps FPS] [--auto_change] [-e EXPORT_DIR] [--auto_export]
                [--ignore_fixed_transform]
                [dataset_dir]

NVDU Data Visualiser

positional arguments:
  dataset_dir           Dataset directory. This is where all the images
                        (required) and annotation info (optional) are.
                        Defaults to the current directory.

optional arguments:
  -h, --help            show this help message and exit.
  -a DATA_ANNOT_DIR, --data_annot_dir DATA_ANNOT_DIR
                        Directory path - where to find the annotation data.
                        Defaults to be the same directory as the dataset
                        directory.
  -s SIZE SIZE, --size SIZE SIZE
                        Window's size: [width, height]. If not specified then
                        the window is sized to fit the resolution of the camera.
  -o OBJECT_SETTINGS_PATH, --object_settings_path OBJECT_SETTINGS_PATH
                        Object settings file path.
  -c CAMERA_SETTINGS_PATH, --camera_settings_path CAMERA_SETTINGS_PATH
                        Camera settings file path.
  -n [NAME_FILTERS [NAME_FILTERS ...]], --name_filters [NAME_FILTERS [NAME_FILTERS ...]]
                        The name filter of each frame. e.g: *.png.
  --fps FPS             How fast to automatically change frame.
  --auto_change         When using this flag, the visualizer will automatically
                        change the frame.
  -e EXPORT_DIR, --export_dir EXPORT_DIR
                        Directory path - where to store the visualized images.
                        If this is set, the script will automatically export
                        the visualized image to the export directory.
  --auto_export         When using this flag, the visualizer will automatically
                        export the visualized frame to an image file in the
                        `export_dir` directory.
  --ignore_fixed_transform
                        When using this flag, the visualizer will not use the
                        fixed transform matrix for the 3d model.

NOTE: The nvdu_viz script can work from any directory

Examples

Visualize a dataset generated by NDDS:

1. Visualize the current directory:

nvdu_viz

2. Visualize a relative path:

nvdu_viz ../a_dataset

3. Visualize an absolute path:

nvdu_viz ~/data/dataset

4. Visualize different aspect of a frame using a filter:

nvdu_viz dataset_path --name_filters *.left.png *.right.png

Visualize a set of images using different annotation data:

1. The camera and object settings files are in the image directory:

nvdu_viz image_directory_here -a annotation_directory_here

2. The camera and object settings files are NOT in the image directory:

nvdu_viz image_directory_here -a annotation_directory_here -c camera_setting_path_here -o object_setting_path_here

Controls

Visualization options:

F3 - Toggle the 2d cuboid
F4 - Toggle the 3d cuboid
F5 - Toggle the 3d models
F6 - Toggle the axes
F7 - Toggle the overlay frame name
1 - Render the 3d models normally
2 - Render the 3d models using only the edge lines
3 - Render the 3d models as point clouds

Other:

ESC - Quit the visualizer
Right - Go to the next frame
Left - Go to the previous frame
Up - Go to the next 100 frame
Down - Go to the previous 100 frame
Space - Toggle frame auto-changing
F12 - Toggle exporting the visualized frame to file