All Projects → zlmzju → Fusenet

zlmzju / Fusenet

Deep fusion project of deeply-fused nets, and the study on the connection to ensembling

Projects that are alternatives of or similar to Fusenet

Aws Machine Learning University Accelerated Nlp
Machine Learning University: Accelerated Natural Language Processing Class
Stars: ✭ 1,695 (+636.96%)
Mutual labels:  jupyter-notebook, mxnet
Octconv.pytorch
PyTorch implementation of Octave Convolution with pre-trained Oct-ResNet and Oct-MobileNet models
Stars: ✭ 229 (-0.43%)
Mutual labels:  resnet, imagenet
Imagenet
This implements training of popular model architectures, such as AlexNet, ResNet and VGG on the ImageNet dataset(Now we supported alexnet, vgg, resnet, squeezenet, densenet)
Stars: ✭ 126 (-45.22%)
Mutual labels:  resnet, imagenet
Imagenetv2
A new test set for ImageNet
Stars: ✭ 109 (-52.61%)
Mutual labels:  jupyter-notebook, imagenet
Senet Caffe
A Caffe Re-Implementation of SENet
Stars: ✭ 169 (-26.52%)
Mutual labels:  resnet, imagenet
Ir Net
This project is the PyTorch implementation of our accepted CVPR 2020 paper : forward and backward information retention for accurate binary neural networks.
Stars: ✭ 119 (-48.26%)
Mutual labels:  resnet, imagenet
Pytorch Imagenet Cifar Coco Voc Training
Training examples and results for ImageNet(ILSVRC2012)/CIFAR100/COCO2017/VOC2007+VOC2012 datasets.Image Classification/Object Detection.Include ResNet/EfficientNet/VovNet/DarkNet/RegNet/RetinaNet/FCOS/CenterNet/YOLOv3.
Stars: ✭ 130 (-43.48%)
Mutual labels:  resnet, imagenet
Mxnet Finetuner
An all-in-one Deep Learning toolkit for image classification to fine-tuning pretrained models using MXNet.
Stars: ✭ 100 (-56.52%)
Mutual labels:  jupyter-notebook, mxnet
Iresnet
Improved Residual Networks (https://arxiv.org/pdf/2004.04989.pdf)
Stars: ✭ 163 (-29.13%)
Mutual labels:  resnet, imagenet
Shufflenet V2 Tensorflow
A lightweight convolutional neural network
Stars: ✭ 145 (-36.96%)
Mutual labels:  jupyter-notebook, imagenet
Shot Type Classifier
Detecting cinema shot types using a ResNet-50
Stars: ✭ 109 (-52.61%)
Mutual labels:  jupyter-notebook, resnet
Deep Learning With Python
Deep learning codes and projects using Python
Stars: ✭ 195 (-15.22%)
Mutual labels:  jupyter-notebook, resnet
Resnet Imagenet Caffe
train resnet on imagenet from scratch with caffe
Stars: ✭ 105 (-54.35%)
Mutual labels:  resnet, imagenet
Pyramidnet
Torch implementation of the paper "Deep Pyramidal Residual Networks" (https://arxiv.org/abs/1610.02915).
Stars: ✭ 121 (-47.39%)
Mutual labels:  resnet, imagenet
Tensorflow2.0 Examples
🙄 Difficult algorithm, Simple code.
Stars: ✭ 1,397 (+507.39%)
Mutual labels:  jupyter-notebook, resnet
Insightface Just Works
Insightface face detection and recognition model that just works out of the box.
Stars: ✭ 127 (-44.78%)
Mutual labels:  jupyter-notebook, mxnet
Caffe Model
Caffe models (including classification, detection and segmentation) and deploy files for famouse networks
Stars: ✭ 1,258 (+446.96%)
Mutual labels:  resnet, imagenet
Pytorch Classification
Classification with PyTorch.
Stars: ✭ 1,268 (+451.3%)
Mutual labels:  resnet, imagenet
Aognet
Code for CVPR 2019 paper: " Learning Deep Compositional Grammatical Architectures for Visual Recognition"
Stars: ✭ 132 (-42.61%)
Mutual labels:  mxnet, imagenet
Imgclsmob
Sandbox for training deep learning networks
Stars: ✭ 2,405 (+945.65%)
Mutual labels:  mxnet, imagenet

Deep Fusion Project

This project contains the models used for the experiments in the paper:

Liming Zhao, Jingdong Wang, Xi Li, Zhuowen Tu, and Wenjun Zeng. "On the Connection of Deep Fusion to Ensembling." arXiv preprint arXiv:1611.07718 (2016).

Contact: Liming Zhao ([email protected])

Introduction

In this work, we provide a systematic study to the prevailing ResNet architecture by showing a connection from a general deeply-fused net view to ensembling.

ensemble

Our empirical results uncover that the deepest network among the ensemble components does not contribute the most significantly to the overall performance and instead it provides a manner to introduce many layers and thus guarantee the ensemble size.

Guided by the above study and observation, we develop a new deeply-fused network that combines two networks in a merge-and-run fusion manner.

blocks

Our approach demonstrates consistent improvements over the ResNet with the comparable setup on CIFAR-10, CIFAR-100, SVHN, and ImageNet.

Results

  • Test error (%) on CIFAR (flip/translation augmentation) and SVHN (no augmentation):
Method Depth #Params CIFAR-10 CIFAR-100 SVHN
DFN-MR1 56 1.7M 4.94 24.46 1.66
DFN-MR2 32 14.9M 3.94 19.25 1.51
DFN-MR3 50 24.8M 3.57 19.00 1.55
  • Training and validation error (%) on ImageNet:
Method Depth #Params Top-1 train Top-5 train Top-1 val Top-5 val
ResNet 98 45.0M 15.09 3.25 23.38 6.79
DFN-MR 50 46.4M 14.46 3.16 23.16 6.61

Requirements

  • Install MXNet on a machine (Windows, Linux, and Mac OS) with CUDA GPU and optional cuDNN.

  • Apply my modified data processing patch on the latest MXNet by merging the pull request:

    git pull origin pull/3936/head master
    
  • (Recommended) If you fail to apply the above patch, you can simply use my MXNet repository:

    git clone --recursive -b fusenet https://github.com/zlmzju/mxnet.git
    

How to Train

Step by step tutorial with jupyter notebook is now available, please check the file tutorial.ipynb.

dataset

You can prepare the *.rec file by yourself, or simply download the Cifar dataset from data.dmlc.ml or my google drive (recommended), which includes both Cifar and SVHN datasets. For ImageNet dataset, follow the mxnet official document to prepare.

training

Current code supports training different deeply-fused nets on Cifar-10, Cifar-100, SVHN and ImageNet, such as plain network, resnet, cross (dfn-mr),half (dfn-il), side (dfn-il without identities), fuse3 (three fusions), fuse6 (six fusions), and ensemble (with sharing weights, training code will come later). All the networks are contained in the network folder.

For example, running the following command can train the DFN-MR network (we call it cross in the coding stage) on Cifar-10.

python train_model.py --dataset=cifar10 --network=cross --depth=56 --gpus=0,1 --dataset=<dataset location>

To train DFN-MR network on ImageNet, run

python train_imagenet.py --network=symbol_cross --gpus=0,1,2,3 --data-dir=<dataset location>

Other usages

visualization

If you want to show the network architecture, run the following command to visualize the network.

python show_model.py --network=half --depth=26 --widen-factor=1

You will obtain a picture half_d26.png in the visualize folder, and more examples can be found there.

Note that you may need to install graphviz for visualization.

show results

The training log are saved to snapshot folder, and you can use get_results.py to obtain the final result of multiple runs in the format of median (mean +/- std, best).

Citation

Please cite our papers on deep fusion in your publications if it helps your research:

@article{WangWZZ16,
  author    = {Jingdong Wang and
               Zhen Wei and
               Ting Zhang and
               Wenjun Zeng},
  title     = {Deeply-Fused Nets},
  journal   = {CoRR},
  volume    = {abs/1605.07716},
  year      = {2016},
  url       = {http://arxiv.org/abs/1605.07716},
}
@article{ZhaoWLTZ16,
  author    = {Liming Zhao and
               Jingdong Wang and
               Xi Li and
               Zhuowen Tu and
               Wenjun Zeng},
  title     = {On the Connection of Deep Fusion to Ensembling},
  journal   = {CoRR},
  volume    = {abs/1611.07718},
  year      = {2016},
  url       = {http://arxiv.org/abs/1611.07718},
}
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].