All Projects → mehtadushy → Selecsls Pytorch

mehtadushy / Selecsls Pytorch

Reference ImageNet implementation of SelecSLS CNN architecture proposed in the SIGGRAPH 2020 paper "XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera". The repository also includes code for pruning the model based on implicit sparsity emerging from adaptive gradient descent methods, as detailed in the CVPR 2019 paper "On implicit filter level sparsity in Convolutional Neural Networks".

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Selecsls Pytorch

Espnetv2
A light-weight, power efficient, and general purpose convolutional neural network
Stars: ✭ 377 (+50.2%)
Mutual labels:  cnn, imagenet, efficient
Computer Vision Leaderboard
Comparison of famous convolutional neural network models
Stars: ✭ 299 (+19.12%)
Mutual labels:  cnn, imagenet
Benchmarking Keras Pytorch
🔥 Reproducibly benchmarking Keras and PyTorch models
Stars: ✭ 346 (+37.85%)
Mutual labels:  cnn, imagenet
Pyconv
Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition (https://arxiv.org/pdf/2006.11538.pdf)
Stars: ✭ 231 (-7.97%)
Mutual labels:  cnn, imagenet
One Pixel Attack Keras
Keras implementation of "One pixel attack for fooling deep neural networks" using differential evolution on Cifar10 and ImageNet
Stars: ✭ 1,097 (+337.05%)
Mutual labels:  cnn, imagenet
Efficientnet
Implementation of EfficientNet model. Keras and TensorFlow Keras.
Stars: ✭ 1,920 (+664.94%)
Mutual labels:  imagenet, efficient
Tf Mobilenet V2
Mobilenet V2(Inverted Residual) Implementation & Trained Weights Using Tensorflow
Stars: ✭ 85 (-66.14%)
Mutual labels:  cnn, imagenet
Iresnet
Improved Residual Networks (https://arxiv.org/pdf/2004.04989.pdf)
Stars: ✭ 163 (-35.06%)
Mutual labels:  cnn, imagenet
Pyramidnet Pytorch
A PyTorch implementation for PyramidNets (Deep Pyramidal Residual Networks, https://arxiv.org/abs/1610.02915)
Stars: ✭ 234 (-6.77%)
Mutual labels:  imagenet
Mobilenetv2 Pytorch
Impementation of MobileNetV2 in pytorch
Stars: ✭ 241 (-3.98%)
Mutual labels:  cnn
Ai Toolbox
Algorithm Engineer Toolbox, for the purpose of quickly iterating new ideas
Stars: ✭ 230 (-8.37%)
Mutual labels:  cnn
Text classification ai100
AI100文本分类竞赛代码。从传统机器学习到深度学习方法的测试
Stars: ✭ 235 (-6.37%)
Mutual labels:  cnn
Caption generator
A modular library built on top of Keras and TensorFlow to generate a caption in natural language for any input image.
Stars: ✭ 243 (-3.19%)
Mutual labels:  cnn
Lcnn
LCNN: End-to-End Wireframe Parsing
Stars: ✭ 234 (-6.77%)
Mutual labels:  cnn
Cnnforandroid
The Convolutional Neural Network(CNN) for Android
Stars: ✭ 245 (-2.39%)
Mutual labels:  cnn
Skimcaffe
Caffe for Sparse Convolutional Neural Network
Stars: ✭ 230 (-8.37%)
Mutual labels:  pruning
Crnn Pytorch
Pytorch implementation of CRNN (CNN + RNN + CTCLoss) for all language OCR.
Stars: ✭ 248 (-1.2%)
Mutual labels:  cnn
Dfl Cnn
This is a pytorch re-implementation of Learning a Discriminative Filter Bank Within a CNN for Fine-Grained Recognition
Stars: ✭ 245 (-2.39%)
Mutual labels:  cnn
Pytorch Sentiment Analysis
Tutorials on getting started with PyTorch and TorchText for sentiment analysis.
Stars: ✭ 3,209 (+1178.49%)
Mutual labels:  cnn
Hierarchical Attention Networks Pytorch
Hierarchical Attention Networks for document classification
Stars: ✭ 239 (-4.78%)
Mutual labels:  cnn

SelecSLS Convolutional Net Pytorch Implementation

Reference ImageNet implementation of SelecSLS Convolutional Neural Network architecture proposed in XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera (SIGGRAPH 2020).

The network architecture is 1.3-1.5x faster than ResNet-50, particularly for larger image sizes, with the same level of accuracy on different tasks! Further, it takes substantially less memory while training, so it can be trained with larger batch sizes!

Update (28 Dec 2019)

Better and more accurate models / snapshots are now available. See the additional ImageNet table below.

Update (14 Oct 2019)

Code for pruning the model based on Implicit Filter Level Sparsity is also a part of the SelecSLS model now. The sparsity is a natural consequence of training with adaptive gradient descent approaches and L2 regularization. It gives a further speedup of 10-30% on the pretrained models with no loss in accuracy. See usage and results below.

ImageNet results

The inference time for the models in the table below is measured on a TITAN X GPU using the accompanying scripts. The accuracy results for ResNet-50 are from torchvision, and the accuracy results for VoVNet-39 are from VoVNet.

* (P) indicates that the model has batch norm fusion and pruning applied
Forward Pass Time (ms)
for different image resolutions
ImageNet
Error
512x512 400x400 224x224 Top-1 Top-5
Batch Size 1 16 1 16 1 16
ResNet-50 15.0 175.0 11.0 114.0 7.2 39.0 23.9 7.1
VoVNet-39 13.0 197.0 10.8 130.0 6 41.0 23.2 6.6
SelecSLS-60 11.0 115.0 9.5 85.0 7.3 29.0 23.8 7.0
SelecSLS-60 (P) 10.2 102.0 8.2 71.0 6.1 25.0 23.8 7.0
SelecSLS-84 16.1 175.0 13.7 124.0 9.9 42.3 23.3 6.9
SelecSLS-84 (P) 11.9 119.0 10.1 82.0 7.6 28.6 23.3 6.9

The following models are trained using Cosine LR, Random Erasing, EMA, Bicubic Interpolation, and Color Jitter using rwightman/pytorch-image-models. The inference time for models here is measured on a TITAN Xp GPU using the accompanying scripts. The script for evaluating ImageNet performance uses Bilinear interpolation, hence the results reported here are marginally worse than they would be with Bicubic interpolation at inference.

Forward Pass Time (ms)
for different image resolutions
ImageNet
Error
512x512 400x400 224x224 Top-1 Top-5
Batch Size 1 16 1 16 1 16
SelecSLS-42_B 6.4 60.8 5.8 42.1 5.7 14.7 22.9 6.6
SelecSLS-60 7.4 69.4 7.3 47.6 7.1 16.8 22.1 6.1
SelecSLS-60_B 7.5 70.5 7.3 49.3 7.2 17.0 21.6 5.8

SelecSLS (Selective Short and Long Range Skip Connections)

The key feature of the proposed architecture is that unlike the full dense connectivity in DenseNets, SelecSLS uses a much sparser skip connectivity pattern that uses both long and short-range concatenative-skip connections. Additionally, the network architecture is more amenable to filter/channel pruning than ResNets. You can find more details about the architecture in the following paper, and details about implicit pruning in the CVPR 2019 paper.

Another recent paper proposed the VoVNet architecture, which shares some design similarities with our architecture. However, as shown in the above table, our architecture is significantly faster than both VoVNet-39 and ResNet-50 for larger batch sizes as well as larger image sizes.

Usage

This repo provides the model definition in Pytorch, trained weights for ImageNet, and code for evaluating the forward pass time and the accuracy of the trained model on ImageNet validation set. In the paper, the model has been used for the task of human pose estimation, and can also be applied to a myriad of other problems as a drop in replacement for ResNet-50.

wget http://gvv.mpi-inf.mpg.de/projects/XNectDemoV2/content/SelecSLS60_statedict.pth -O ./weights/SelecSLS60_statedict.pth
python evaluate_timing.py --num_iter 100 --model_class selecsls --model_config SelecSLS60 --model_weights ./weights/SelecSLS60_statedict.pth --input_size 512 --gpu_id <id>
python evaluate_imagenet.py --model_class selecsls --model_config SelecSLS60 --model_weights ./weights/SelecSLS60_statedict.pth --gpu_id <id> --imagenet_base_path <path_to_imagenet_dataset>

#For pruning the model, and evaluating the pruned model (Using SelecSLS60 or other pretrained models)
python evaluate_timing.py --num_iter 100 --model_class selecsls --model_config SelecSLS84 --model_weights ./weights/SelecSLS84_statedict.pth --input_size 512 --pruned_and_fused True --gamma_thresh 0.001 --gpu_id <id>
python evaluate_imagenet.py --model_class selecsls --model_config SelecSLS84 --model_weights ./weights/SelecSLS84_statedict.pth --pruned_and_fused True --gamma_thresh 0.001 --gpu_id <id> --imagenet_base_path <path_to_imagenet_dataset>

Older Pretrained Models

Newer Pretrained Models (More Accurate)

Requirements

  • Python 3.5
  • Pytorch >= 1.1

License

The contents of this repository, and the pretrained models are made available under CC BY 4.0. Please read the license terms.

Citing

If you use the model or the implicit sparisty based pruning in your work, please cite:

@inproceedings{XNect_SIGGRAPH2020,
 author = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Elgharib, Mohamed and Fua, Pascal and Seidel, Hans-Peter and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian},
 title = {{XNect}: Real-time Multi-Person {3D} Motion Capture with a Single {RGB} Camera},
 journal = {ACM Transactions on Graphics},
 url = {http://gvv.mpi-inf.mpg.de/projects/XNect/},
 numpages = {17},
 volume={39},
 number={4},
 month = July,
 year = {2020},
 doi={10.1145/3386569.3392410}
} 

@InProceedings{Mehta_2019_CVPR,
author = {Mehta, Dushyant and Kim, Kwang In and Theobalt, Christian},
title = {On Implicit Filter Level Sparsity in Convolutional Neural Networks},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
} 
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].