Learning Single-View 3D Reconstruction with Limited Pose Supervision
TensorFlow implementation for the paper:
Learning Single-View 3D Reconstruction with Limited Pose Supervision
Guandao Yang, Yin Cui, Serge Belongie, Bharath Hariharan
Dependency
- TensorFlow（>=1.4)
- OpenCV
- Matplotlib
The recommended way to install the dependency is
pip install -r requirements.txtPreparation
Please use the following Google Drive link to download the datasets: [drive]. There are two files : data.tar.gz and ShapeNetVox32.tar.gz. Please download both of them and uncompressed it into the project root directory:
tar -xvf ShapeNetVox32.tar.gz
tar -xvf data.tar.gz
rm ShapeNetVox32.tar.gz
rm data.tar.gzTraining
In order to train a model, please use train.py script. The default hyper-parameter are stored in config.py, and all the training settings are stored in training_settings.py. For example, to train single category chairs with 50% pose annotations, we could use:
python train.py --data chairs_pose0.5where chairs_pose0.5 refers to an entry in the training_settings.py file.
The scripts folder contains command line arguments for running different experiments. Note that each script contains multiple training commands.
For example, in order to run all the single category training with 50% of pose annotations, please use:
./scripts/single_category.shTo train all multi-category models or to pre-train all models for few-shot transfer learning, please use:
./scripts/muti.shEvaluation
File inference.py contains codes to load a trained model and evaluate on specific data split or categories. scripts/get_score.py helps to organize the evaluation scores into .csv files. For Detail usage, please refer to scripts/eval_*.sh.
For example, if you want to evaluate all the single category training experiments, try running
./sripts/eval_single_category.shResults
Following results are reported from the testing set. Please compare to these results during reproduction.
Single Category Experiments
| Setting | MaxIoU | AP | IoU(t=0.4) | Iou(t=0.5) |
|---|---|---|---|---|
| airplanes_pose0.5 | 0.484 | 0.6377 | 0.4396 | 0.4235 |
| beches_pose0.5 | 0.368 | 0.4822 | 0.3421 | 0.3379 |
| cars_pose0.5 | 0.7421 | 0.8698 | 0.6976 | 0.6777 |
| chairs_pose0.5 | 0.4459 | 0.5703 | 0.4113 | 0.3896 |
| sofas_pose0.5 | 0.5523 | 0.6954 | 0.5276 | 0.5224 |
| tables_pose0.5 | 0.4171 | 0.5541 | 0.355 | 0.3402 |
Multiple Category (AP)
| Category | Pose 100% | Pose 50% | Pose 10% | Pose 1% |
|---|---|---|---|---|
| airplanes | 0.7103 | 0.7062 | 0.6457 | 0.5316 |
| cars | 0.9223 | 0.9139 | 0.8888 | 0.797 |
| chairs | 0.5922 | 0.5738 | 0.5325 | 0.4079 |
| displays | 0.6025 | 0.5917 | 0.4694 | 0.2857 |
| phones | 0.8294 | 0.813 | 0.662 | 0.498 |
| speakers | 0.7035 | 0.6869 | 0.6336 | 0.5481 |
| tables | 0.5603 | 0.5486 | 0.4827 | 0.3948 |
| Mean | 0.7029 | 0.6906 | 0.6164 | 0.4947 |
Out of category (AP)
| Category | Pose 100% | Pose 50% | Pose 10% | Pose 1% |
|---|---|---|---|---|
| benches | 0.4243 | 0.4044 | 0.3391 | 0.2485 |
| cabinets | 0.6313 | 0.6123 | 0.5667 | 0.5321 |
| vessels | 0.6109 | 0.6063 | 0.5581 | 0.5325 |
TODO
We will release the codes for few-shot experiments soon.
Citation
If you find this our works helpful for your research, please cite:
@InProceedings{Yang_2018_ECCV,
author = {Yang, Guandao and Cui, Yin and Belongie, Serge and Hariharan, Bharath},
title = {Learning Single-View 3D Reconstruction with Limited Pose Supervision},
booktitle = {The European Conference on Computer Vision (ECCV)},
month = {September},
year = {2018}
}