Holistically-Attracted Wireframe Parsing (CVPR 2020)
This is the official implementation of our CVPR paper.
[News] We experimentally provided an easy-to-install version for inference-only usage of HAWP, please checkout the inference branch for the details.
Highlights
- We propose a fast and parsimonious parsing method HAWP to accurately and robustly detect a vectorized wireframe in an input image with a single forward pass.
- The proposed HAWP is fully end-to-end.
- The proposed HAWP does not require the squeeze module.
- State-of-the-art performance on the Wireframe dataset and YorkUrban dataset.
- The proposed HAWP achieves 29.5 FPS on a GPU (Tesla V100) for 1-batch inference.
Quantitative Results
Wireframe Dataset
| Method | Wireframe Dataset | FPS | ||||||
| sAP5 | sAP10 | sAP15 | msAP | mAPJ | APH | FH | ||
| LSD | / | / | / | / | / | 55.2 | 62.5 | 49.6 |
| AFM | 18.5 | 24.4 | 27.5 | 23.5 | 23.3 | 69.2 | 77.2 | 13.5 |
| DWP | 3.7 | 5.1 | 5.9 | 4.9 | 40.9 | 67.8 | 72.2 | 2.24 |
| L-CNN | 58.9 | 62.9 | 64.9 | 62.2 | 59.3 | 80.3 | 76.9 | 15.6 |
| 82.8† | 81.3† | |||||||
| L-CNN (re-trained) | 59.7 | 63.6 | 65.3 | 62.9 | 60.2 | 81.6 | 77.9 | 15.6 |
| 83.7† | 81.7† | |||||||
| HAWP (Ours) | 62.5 | 66.5 | 68.2 | 65.7 | 60.2 | 84.5 | 80.3 | 29.5 |
| 86.1† | 83.1† | |||||||
YorkUrban Dataset
| Method | YorkUrban Dataset | FPS | ||||||
| sAP5 | sAP10 | sAP15 | msAP | mAPJ | APH | FH | ||
| LSD | / | / | / | / | / | 50.9 | 60.1 | 49.6 |
| AFM | 7.3 | 9.4 | 11.1 | 9.3 | 12.4 | 48.2 | 63.3 | 13.5 |
| DWP | 1.5 | 2.1 | 2.6 | 2.1 | 13.4 | 51.0 | 61.6 | 2.24 |
| L-CNN | 24.3 | 26.4 | 27.5 | 26.1 | 30.4 | 58.5 | 61.8 | 15.6 |
| 59.6† | 65.3† | |||||||
| L-CNN (re-trained) | 25.0 | 27.1 | 28.3 | 26.8 | 31.5 | 58.3 | 62.2 | 15.6 |
| 59.3† | 65.2† | |||||||
| HAWP (Ours) | 26.1 | 28.5 | 29.7 | 28.1 | 31.6 | 60.6 | 64.8 | 29.5 |
| 61.2† | 66.3† | |||||||
Installation (tested on Ubuntu-18.04, CUDA 10.0, GCC 7.4.0)
conda create -n hawp python=3.6
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
cd hawp
conda develop .
pip install -r requirement.txt
python setup.py build_ext --inplaceQuickstart with the pretrained model
python scripts/predict.py --img figures/example.pngTraining & Testing
Data Preparation
- Download the Wireframe dataset and the YorkUrban dataset from their project pages.
- Download the JSON-format annotations (Google Drive).
- Place the images to "hawp/data/wireframe/images/" and "hawp/data/york/images/".
- Unzip the json-format annotations to "hawp/data/wireframe" and "hawp/data/york".
The structure of the data folder should be
data/
wireframe/images/*.png
wireframe/train.json
wireframe/test.json
------------------------
york/images/*.png
york/test.jsonTraining
CUDA_VISIBLE_DEVICES=0, python scripts/train.py --config-file config-files/hawp.yamlThe best model is manually selected from the model files after 25 epochs.
Testing
CUDA_VISIBLE_DEVICES=0, python scripts/test.py --config-file config-files/hawp.yaml [optional] --displayThe output results will be saved to OUTPUT_DIR/$dataset_name.json. The dataset_name should be wireframe_test or york_test.
Structural-AP Evaluation
- Run scripts/test.py to get the wireframe parsing results.
- Run scripts/eval_sap.py to get the sAP results
# example on the Wireframe dataset
python scripts/eval_sap.py --path outputs/hawp/wireframe_test.json --threshold 10Citations
If you find our work useful in your research, please consider citing:
@inproceedings{HAWP,
title = "Holistically-Attracted Wireframe Parsing",
author = "Nan Xue and Tianfu Wu and Song Bai and Fu-Dong Wang and Gui-Song Xia and Liangpei Zhang and Philip H.S. Torr
",
booktitle = "IEEE Conference on Computer Vision and Pattern Recognition (CVPR)",
year = {2020},
}
Acknowledgment
We acknowledge the effort from the authors of the Wireframe dataset and the YorkUrban dataset. These datasets make accurate line segment detection and wireframe parsing possible.