3D Bounding Box Estimation for Autonomous Drinving

This project fully implemented paper "3D Bounding Box Estimation Using Deep Learning and Geometry" based on previous work by image-to-3d-bbox(https://github.com/experiencor/image-to-3d-bbox).

Depandency:

• Python 3.6
• Tensorflow 1.12.0

Modifications and Improvements:

1. No prior knowledge of the object location is needed. Instead of reducing configuration numbers to 64, the location of each object is solved analytically based on local orientation and 2D location.

2. Add soft constraints to improve the stability of 3D bounding box at certain locations.

3. MobileNetV2 backend is used to significantly reduce parameter numbers and make the model Fully Convolutional.

4. The orientation loss is changed to the correct form.

5. Bird-eye view visualization is added.

Results on KITTI raw data:

MobilenetV2 with ground truth 2D bounding box.

Video: https://www.youtube.com/watch?v=IIReDnbLQAE

Train and Evaluate:

First prepare your KITTI dataset in the following format:

kitti_dateset/
├── 2011_09_26
│   └── 2011_09_26_drive_0084_sync
│           ├── box_3d       <- predicted data
│           ├── calib_02
│           ├── calib_cam_to_cam.txt
│           ├── calib_velo_to_cam.txt
│           ├── image_02
│           ├── label_02
│           └── tracklet_labels.xml
│
└── training
    ├── box_3d    <- predicted data
    ├── calib
    ├── image_2
    └── label_2

To train:

1. Specify parameters in config.py.
2. run train.py to train the model:

python3 train.py

To predict:

1. Change dir in read_dir.py to your prediction folder.
2. run prediction.py to predict 3D bounding boxes. Change -d to your dataset directory, -a to specify which type of dataset(train/val split or raw), -w to specify the training weights.

To visualize 3D bounding box:

1. run visualization3Dbox.py. Specify -s to if save figures or view the plot , specify -p to your output image folders.

Performance: