GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → yoshitomo-matsubara → head-network-distillation

yoshitomo-matsubara / head-network-distillation

Licence: other
[IEEE Access] "Head Network Distillation: Splitting Distilled Deep Neural Networks for Resource-constrained Edge Computing Systems" and [ACM MobiCom HotEdgeVideo 2019] "Distilled Split Deep Neural Networks for Edge-assisted Real-time Systems"

Programming Languages

Jupyter Notebook
11667 projects
python
139335 projects - #7 most used programming language

Labels

pytorchimagenetimage-classificationknowledge-distillationsplit-computing

Projects that are alternatives of or similar to head-network-distillation

Imagenet
Trial on kaggle imagenet object localization by yolo v3 in google cloud
Stars: ✭ 56 (+107.41%)
Mutual labels:  imagenet, image-classification
Regnet
Pytorch implementation of network design paradigm described in the paper "Designing Network Design Spaces"
Stars: ✭ 129 (+377.78%)
Mutual labels:  imagenet, image-classification
Tf Mobilenet V2
Mobilenet V2(Inverted Residual) Implementation & Trained Weights Using Tensorflow
Stars: ✭ 85 (+214.81%)
Mutual labels:  imagenet, image-classification
Hrnet Image Classification
Train the HRNet model on ImageNet
Stars: ✭ 633 (+2244.44%)
Mutual labels:  imagenet, image-classification
Iresnet
Improved Residual Networks (https://arxiv.org/pdf/2004.04989.pdf)
Stars: ✭ 163 (+503.7%)
Mutual labels:  imagenet, image-classification
Orange3 Imageanalytics
🍊 🎑 Orange3 add-on for dealing with image related tasks
Stars: ✭ 24 (-11.11%)
Mutual labels:  imagenet, image-classification
Labelbox
Labelbox is the fastest way to annotate data to build and ship computer vision applications.
Stars: ✭ 1,588 (+5781.48%)
Mutual labels:  imagenet, image-classification
Neural Backed Decision Trees
Making decision trees competitive with neural networks on CIFAR10, CIFAR100, TinyImagenet200, Imagenet
Stars: ✭ 411 (+1422.22%)
Mutual labels:  imagenet, image-classification
Alexnet
implement AlexNet with C / convolutional nerual network / machine learning / computer vision
Stars: ✭ 147 (+444.44%)
Mutual labels:  imagenet, image-classification
Efficientnet
Implementation of EfficientNet model. Keras and TensorFlow Keras.
Stars: ✭ 1,920 (+7011.11%)
Mutual labels:  imagenet, image-classification
Cvat
Powerful and efficient Computer Vision Annotation Tool (CVAT)
Stars: ✭ 6,557 (+24185.19%)
Mutual labels:  imagenet, image-classification
Torchdistill
PyTorch-based modular, configuration-driven framework for knowledge distillation. 🏆18 methods including SOTA are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy.
Stars: ✭ 177 (+555.56%)
Mutual labels:  imagenet, image-classification
Mmclassification
OpenMMLab Image Classification Toolbox and Benchmark
Stars: ✭ 532 (+1870.37%)
Mutual labels:  imagenet, image-classification
Divide And Co Training
[Paper 2020] Towards Better Accuracy-efficiency Trade-offs: Divide and Co-training. Plus, an image classification toolbox includes ResNet, Wide-ResNet, ResNeXt, ResNeSt, ResNeXSt, SENet, Shake-Shake, DenseNet, PyramidNet, and EfficientNet.
Stars: ✭ 54 (+100%)
Mutual labels:  imagenet, image-classification
Fishnet
Implementation code of the paper: FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction, NeurIPS 2018
Stars: ✭ 533 (+1874.07%)
Mutual labels:  imagenet, image-classification
Petridishnn
Code for the neural architecture search methods contained in the paper Efficient Forward Neural Architecture Search
Stars: ✭ 112 (+314.81%)
Mutual labels:  imagenet, image-classification
Rectlabel Support
RectLabel - An image annotation tool to label images for bounding box object detection and segmentation.
Stars: ✭ 338 (+1151.85%)
Mutual labels:  imagenet, image-classification
Computer Vision
Programming Assignments and Lectures for Stanford's CS 231: Convolutional Neural Networks for Visual Recognition
Stars: ✭ 408 (+1411.11%)
Mutual labels:  imagenet, image-classification
Aognet
Code for CVPR 2019 paper: " Learning Deep Compositional Grammatical Architectures for Visual Recognition"
Stars: ✭ 132 (+388.89%)
Mutual labels:  imagenet, image-classification
Imgclsmob
Sandbox for training deep learning networks
Stars: ✭ 2,405 (+8807.41%)
Mutual labels:  imagenet, image-classification
View All Similar Projects ➔

Head Network Distillation for Split Computing

The official implementations of Head Network Distillation (HND) studies for image classification tasks:

Part of this repository is used as baselines in the following paper:

  • "BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing," IEEE WoWMoM '22
    [Preprint]

Looking for object detection tasks? Take a look at https://github.com/yoshitomo-matsubara/hnd-ghnd-object-detectors as well

HND for Split Computing

Citations

@article{matsubara2020head,
  author={Matsubara, Yoshitomo and Callegaro, Davide and Baidya, Sabur and Levorato, Marco and Singh, Sameer},
  journal={IEEE Access}, 
  title={Head Network Distillation: Splitting Distilled Deep Neural Networks for Resource-Constrained Edge Computing Systems}, 
  year={2020},
  volume={8},
  number={},
  pages={212177-212193},
  doi={10.1109/ACCESS.2020.3039714}
}

@inproceedings{matsubara2019distilled,
  title={Distilled Split Deep Neural Networks for Edge-assisted Real-time Systems},
  author={Matsubara, Yoshitomo and Baidya, Sabur and Callegaro, Davide and Levorato, Marco and Singh, Sameer},
  booktitle={Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges},
  pages={21--26},
  year={2019}
}

Requirements

How to clone

git clone https://github.com/yoshitomo-matsubara/head-network-distillation.git
cd head-network-distillation/
git submodule init
git submodule update --recursive --remote
pipenv install

Download datasets

As the terms of use do not allow to distribute the URLs, you will have to create an account here to get the URLs, and replace ${TRAIN_DATASET_URL} and ${VAL_DATASET_URL} with them.

wget ${TRAIN_DATASET_URL} ./
wget ${VAL_DATASET_URL} ./

ILSVRC 2012 (ImageNet) dataset

# Go to home directory
mkdir ~/dataset/ilsvrc2012/{train,val} -p
mv ILSVRC2012_img_train.tar ~/dataset/ilsvrc2012/train/
mv ILSVRC2012_img_val.tar ~/dataset/ilsvrc2012/val/
cd ~/dataset/ilsvrc2012/train/
tar -xvf ILSVRC2012_img_train.tar
for f in *.tar; do
  d=`basename $f .tar`
  mkdir $d
  (cd $d && tar xf ../$f)
done
rm -r *.tar

wget https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh
mv valprep.sh ~/dataset/ilsvrc2012/val/
cd ~/dataset/ilsvrc2012/val/
tar -xvf ILSVRC2012_img_val.tar
sh valprep.sh

# Go to the root of the repository
pipenv run python dataset_converter.py --input ~/dataset/ilsvrc2012/ --dataset imagenet --output ./resource/data/ILSVRC2012/

Trained models

We publish bottleneck-injected DenseNet-169, DenseNet-201, ResNet-152 and Inception-v3 trained on ILSVRC 2012 (a.k.a. ImageNet) dataset in the following three methods:

Unzip the downloaded zip files under ./resource/ckpt/, then there will be ./resource/ckpt/{naive,kd,hnd,ae}/.

Test trained models

Naive

# DenseNet-169
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet169_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet169_head_mimic-ver2b-6ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet169_head_mimic-ver2b-9ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet169_head_mimic-ver2b-12ch.yaml -test_only

# DenseNet-201
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet201_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet201_head_mimic-ver2b-6ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet201_head_mimic-ver2b-9ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/densenet201_head_mimic-ver2b-12ch.yaml -test_only

# ResNet-152
pipenv run python src/model_runner.py --config config/official/imagenet/naive/resnet152_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/resnet152_head_mimic-ver2b-6ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/resnet152_head_mimic-ver2b-9ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/resnet152_head_mimic-ver2b-12ch.yaml -test_only

# Inception-v3
pipenv run python src/model_runner.py --config config/official/imagenet/naive/inception_v3_head_mimic-ver1b-3ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/inception_v3_head_mimic-ver1b-6ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/inception_v3_head_mimic-ver1b-9ch.yaml -test_only
pipenv run python src/model_runner.py --config config/official/imagenet/naive/inception_v3_head_mimic-ver1b-12ch.yaml -test_only

KD

# DenseNet-169
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet169_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet169_head_mimic-ver2b-6ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet169_head_mimic-ver2b-9ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet169_head_mimic-ver2b-12ch.yaml -test_only -student_only

# DenseNet-201
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet201_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet201_head_mimic-ver2b-6ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet201_head_mimic-ver2b-9ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/densenet201_head_mimic-ver2b-12ch.yaml -test_only -student_only

# ResNet-152
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/resnet152_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/resnet152_head_mimic-ver2b-6ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/resnet152_head_mimic-ver2b-9ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/resnet152_head_mimic-ver2b-12ch.yaml -test_only -student_only

# Inception-v3
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/inception_v3_head_mimic-ver1b-3ch.yaml -test_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/inception_v3_head_mimic-ver1b-6ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/inception_v3_head_mimic-ver1b-9ch.yaml -test_only -student_only
pipenv run python src/model_distiller.py --config config/official/imagenet/kd/inception_v3_head_mimic-ver1b-12ch.yaml -test_only -student_only

HND

# DenseNet-169
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet169_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet169_head_mimic-ver2b-6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet169_head_mimic-ver2b-9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet169_head_mimic-ver2b-12ch.yaml -test_only -student_only

# DenseNet-201
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet201_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet201_head_mimic-ver2b-6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet201_head_mimic-ver2b-9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/densenet201_head_mimic-ver2b-12ch.yaml -test_only -student_only

# ResNet-152
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/resnet152_head_mimic-ver2b-3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/resnet152_head_mimic-ver2b-6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/resnet152_head_mimic-ver2b-9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/resnet152_head_mimic-ver2b-12ch.yaml -test_only -student_only

# Inception-v3
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/inception_v3_head_mimic-ver1b-3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/inception_v3_head_mimic-ver1b-6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/inception_v3_head_mimic-ver1b-9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/hnd/inception_v3_head_mimic-ver1b-12ch.yaml -test_only -student_only

Autoencoder

# DenseNet-169
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet169-b3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet169-b6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet169-b9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet169-b12ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet169-b15ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet169-b18ch.yaml -test_only -student_only

# DenseNet-201
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet201-b3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet201-b6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet201-b9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet201-b12ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet201-b15ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4densenet201-b18ch.yaml -test_only -student_only

# ResNet-152
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4resnet152-b3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4resnet152-b6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4resnet152-b9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4resnet152-b12ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4resnet152-b15ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4resnet152-b18ch.yaml -test_only -student_only

# Inception-v3
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4inception_v3-b3ch.yaml -test_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4inception_v3-b6ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4inception_v3-b9ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4inception_v3-b12ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4inception_v3-b15ch.yaml -test_only -student_only
pipenv run python src/mimic_runner.py --config config/official/imagenet/ae/middle_ae4inception_v3-b18ch.yaml -test_only -student_only

Train models

If you would like to train models, you should exclude -test_only from the above commands, and set new file paths for student model in the yaml files.
To enable the distributed training mode, you should use pipenv run python -m torch.distributed.launch --nproc_per_node=${NUM_GPUS} --use_env ONE_OF_THE_ABOVE_THREE.py ... --world_size ${NUM_GPUS}

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].