All Projects → zhou13 → Lcnn

zhou13 / Lcnn

Licence: mit
LCNN: End-to-End Wireframe Parsing

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Lcnn

Cfsrcnn
Coarse-to-Fine CNN for Image Super-Resolution (IEEE Transactions on Multimedia,2020)
Stars: ✭ 84 (-64.1%)
Mutual labels:  deep-neural-networks, cnn
Pyconv
Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition (https://arxiv.org/pdf/2006.11538.pdf)
Stars: ✭ 231 (-1.28%)
Mutual labels:  deep-neural-networks, cnn
Pytorch Learners Tutorial
PyTorch tutorial for learners
Stars: ✭ 97 (-58.55%)
Mutual labels:  deep-neural-networks, cnn
Gtsrb
Convolutional Neural Network for German Traffic Sign Recognition Benchmark
Stars: ✭ 65 (-72.22%)
Mutual labels:  deep-neural-networks, cnn
Adnet
Attention-guided CNN for image denoising(Neural Networks,2020)
Stars: ✭ 135 (-42.31%)
Mutual labels:  deep-neural-networks, cnn
Cnn Paper2
🎨 🎨 深度学习 卷积神经网络教程 :图像识别,目标检测,语义分割,实例分割,人脸识别,神经风格转换,GAN等🎨🎨 https://dataxujing.github.io/CNN-paper2/
Stars: ✭ 77 (-67.09%)
Mutual labels:  deep-neural-networks, cnn
Robust Lane Detection
Stars: ✭ 110 (-52.99%)
Mutual labels:  deep-neural-networks, cnn
Quickdraw
Implementation of Quickdraw - an online game developed by Google
Stars: ✭ 805 (+244.02%)
Mutual labels:  deep-neural-networks, cnn
Pytorch convlstm
convolutional lstm implementation in pytorch
Stars: ✭ 126 (-46.15%)
Mutual labels:  deep-neural-networks, cnn
Hyperdensenet
This repository contains the code of HyperDenseNet, a hyper-densely connected CNN to segment medical images in multi-modal image scenarios.
Stars: ✭ 124 (-47.01%)
Mutual labels:  deep-neural-networks, cnn
Deepseqslam
The Official Deep Learning Framework for Route-based Place Recognition
Stars: ✭ 49 (-79.06%)
Mutual labels:  deep-neural-networks, cnn
Tf Adnet Tracking
Deep Object Tracking Implementation in Tensorflow for 'Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning(CVPR 2017)'
Stars: ✭ 162 (-30.77%)
Mutual labels:  deep-neural-networks, cnn
Dl Colab Notebooks
Try out deep learning models online on Google Colab
Stars: ✭ 969 (+314.1%)
Mutual labels:  deep-neural-networks, cnn
Dltk
Deep Learning Toolkit for Medical Image Analysis
Stars: ✭ 1,249 (+433.76%)
Mutual labels:  deep-neural-networks, cnn
Servenet
Service Classification based on Service Description
Stars: ✭ 21 (-91.03%)
Mutual labels:  deep-neural-networks, cnn
Models
DLTK Model Zoo
Stars: ✭ 101 (-56.84%)
Mutual labels:  deep-neural-networks, cnn
Deepface
Deep Learning Models for Face Detection/Recognition/Alignments, implemented in Tensorflow
Stars: ✭ 409 (+74.79%)
Mutual labels:  deep-neural-networks, cnn
Cnn Facial Landmark
Training code for facial landmark detection based on deep convolutional neural network.
Stars: ✭ 516 (+120.51%)
Mutual labels:  deep-neural-networks, cnn
Lenet 5
PyTorch implementation of LeNet-5 with live visualization
Stars: ✭ 122 (-47.86%)
Mutual labels:  deep-neural-networks, cnn
Livianet
This repository contains the code of LiviaNET, a 3D fully convolutional neural network that was employed in our work: "3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study"
Stars: ✭ 143 (-38.89%)
Mutual labels:  deep-neural-networks, cnn

L-CNN — End-to-End Wireframe Parsing

This repository contains the official PyTorch implementation of the paper: Yichao Zhou, Haozhi Qi, Yi Ma. "End-to-End Wireframe Parsing." ICCV 2019.

Introduction

L-CNN is a conceptually simple yet effective neural network for detecting the wireframe from a given image. It outperforms the previous state-of-the-art wireframe and line detectors by a large margin. We hope that this repository serves as an easily reproducible baseline for future researches in this area.

Main Results

Qualitative Measures

LSD AFM Wireframe L-CNN Ground Truth

More random sampled results can be found in the supplementary material of the paper.

Quantitative Measures

The following table reports the performance metrics of several wireframe and line detectors on the ShanghaiTech dataset.

ShanghaiTech (sAP10) ShanghaiTech (APH) ShanghaiTech (FH) ShanghaiTech (mAPJ)
LSD / 52.0 61.0 /
AFM 24.4 69.5 77.2 23.3
Wireframe 5.1 67.8 72.6 40.9
L-CNN 62.9 82.8 81.2 59.3

Precision-Recall Curves

Code Structure

Below is a quick overview of the function of each file.

########################### Data ###########################
figs/
data/                           # default folder for placing the data
    wireframe/                  # folder for ShanghaiTech dataset (Huang et al.)
logs/                           # default folder for storing the output during training
########################### Code ###########################
config/                         # neural network hyper-parameters and configurations
    wireframe.yaml              # default parameter for ShanghaiTech dataset
dataset/                        # all scripts related to data generation
    wireframe.py                # script for pre-processing the ShanghaiTech dataset to npz
misc/                           # misc scripts that are not important
    draw-wireframe.py           # script for generating figure grids
    lsd.py                      # script for generating npz files for LSD
    plot-sAP.py                 # script for plotting sAP10 for all algorithms
lcnn/                           # lcnn module so you can "import lcnn" in other scripts
    models/                     # neural network structure
        hourglass_pose.py       # backbone network (stacked hourglass)
        line_vectorizer.py      # sampler and line verification network
        multitask_learner.py    # network for multi-task learning
    datasets.py                 # reading the training data
    metrics.py                  # functions for evaluation metrics
    trainer.py                  # trainer
    config.py                   # global variables for configuration
    utils.py                    # misc functions
demo.py                         # script for detecting wireframes for an image
eval-sAP.py                     # script for sAP evaluation
eval-APH.py                     # script for APH evaluation
eval-mAPJ.py                    # script for mAPJ evaluation
train.py                        # script for training the neural network
post.py                         # script for post-processing
process.py                      # script for processing a dataset from a checkpoint

Reproducing Results

Installation

For the ease of reproducibility, you are suggested to install miniconda (or anaconda if you prefer) before following executing the following commands.

git clone https://github.com/zhou13/lcnn
cd lcnn
conda create -y -n lcnn
source activate lcnn
# Replace cudatoolkit=10.1 with your CUDA version: https://pytorch.org/
conda install -y pytorch cudatoolkit=10.1 -c pytorch
conda install -y tensorboardx -c conda-forge
conda install -y pyyaml docopt matplotlib scikit-image opencv
mkdir data logs post

Pre-trained Models

You can download our reference pre-trained models from Google Drive. Those models were trained with config/wireframe.yaml for 312k iterations. Use demo.py, process.py, and eval-*.py to evaluate the pre-trained models.

Detect Wireframes for Your Own Images

To test LCNN on your own images, you need download the pre-trained models and execute

python ./demo.py -d 0 config/wireframe.yaml <path-to-pretrained-pth> <path-to-image>

Here, -d 0 is specifying the GPU ID used for evaluation, and you can specify -d "" to force CPU inference.

Downloading the Processed Dataset

Make sure curl is installed on your system and execute

cd data
../misc/gdrive-download.sh 1T4_6Nb5r4yAXre3lf-zpmp3RbmyP1t9q wireframe.tar.xz
tar xf wireframe.tar.xz
rm wireframe.tar.xz
cd ..

If gdrive-download.sh does not work for you, you can download the pre-processed dataset wireframe.tar.xz manually from Google Drive and proceed accordingly.

Processing the Dataset

Optionally, you can pre-process (e.g., generate heat maps, do data augmentation) the dataset from scratch rather than downloading the processed one. Skip this section if you just want to use the pre-processed dataset wireframe.tar.xz.

cd data
../misc/gdrive-download.sh 1BRkqyi5CKPQF6IYzj_dQxZFQl0OwbzOf wireframe_raw.tar.xz
tar xf wireframe_raw.tar.xz
rm wireframe_raw.tar.xz
cd ..
dataset/wireframe.py data/wireframe_raw data/wireframe

Training

The default batch size assumes your have a graphics card with 12GB video memory, e.g., GTX 1080Ti or RTX 2080Ti. You may reduce the batch size if you have less video memory.

To train the neural network on GPU 0 (specified by -d 0) with the default parameters, execute

python ./train.py -d 0 --identifier baseline config/wireframe.yaml

Testing Pretrained Models

To generate wireframes on the validation dataset with the pretrained model, execute

./process.py config/wireframe.yaml <path-to-checkpoint.pth> data/wireframe logs/pretrained-model/npz/000312000

Post Processing

To post process the outputs from neural network (only necessary if you are going to evaluate APH), execute

python ./post.py --plot --thresholds="0.010,0.015" logs/RUN/npz/ITERATION post/RUN-ITERATION

where --plot is an optional argument to control whether the program should also generate images for visualization in addition to the npz files that contain the line information, and --thresholds controls how aggressive the post processing is. Multiple values in --thresholds is convenient for hyper-parameter search. You should replace RUN and ITERATION to the desired value of your training instance.

Evaluation

To evaluate the sAP (recommended) of all your checkpoints under logs/, execute

python eval-sAP.py logs/*/npz/*

To evaluate the mAPJ, execute

python eval-mAPJ.py logs/*/npz/*

To evaluate APH, you first need to post process your result (see the previous section). In addition, MATLAB is required for APH evaluation and matlab should be under your $PATH. The parallel computing toolbox is highly suggested due to the usage of parfor. After post processing, execute

python eval-APH.py post/RUN-ITERATION/0_010 post/RUN-ITERATION/0_010-APH

to get the plot, where 0_010 is the threshold used in the post processing, and post/RUN-ITERATION-APH is the temporary directory storing intermediate files. Due to the usage of pixel-wise matching, the evaluation of APH may take up to an hour depending on your CPUs.

See the source code of eval-sAP.py, eval-mAPJ.py, eval-APH.py, and misc/*.py for more details on evaluation.

Citing End-to-End Wireframe Parsing

If you find L-CNN useful in your research, please consider citing:

@inproceedings{zhou2019end,
 author={Zhou, Yichao and Qi, Haozhi and Ma, Yi},
 title={End-to-End Wireframe Parsing},
 booktitle={ICCV 2019},
 year={2019}
}
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].