SE-Net Incorporates with ResNet and WideResnet on CIFAR-10/100 Dataset
This is a SE-Net implementation based on "Squeeze-and-Excitation Networks" [3] on CVPR 2017 "Beyond Imagenet" workshop.
We combine SE Module with ResNet-164 and WideResnet28-10 to construct SeResNet-164 and SeWideResNet28-10 respectively. For details of ResNet-164 and WideResNet28-10, please refers to the original papers [1] and [2].
We evaluate SeResNet-164 and SeWideResNet28-10 on cifar-10 and cifar-100 datasets.
For details of the hyperparameters and training processes, please refer to the /scripts folder.
SeResNet-164 VS ResNet-164 on cifar-10:
Accuracy: 95.12 vs 94.92 (94.54 reported by [1])
SeResNet-164 VS ResNet-164 on cifar-100:
Accuracy: 78.09 vs 76.53 (75.67 reported by [1])
SeWideResNet28-10 VS WideResNet28-10 on cifar-10:
Accuracy: both around 96.10 (96.00 reported by [2])
SeWideResNet28-10 VS WideResNet28-10 on cifar-100:
Accuracy: both around 81.2 (80.75 reported by [2])
Coarse Conclusion:
SE Module seems to work better with thin networks than wide networks on CIFAR-10 and CIFAR-100 datasets.
To-Do:
More networks with SE Module.
Welcome to make contributions!
Pre-requisites:
pytorch http://pytorch.org/
tensorboard https://www.tensorflow.org/get_started/summaries_and_tensorboard
tensorboard-pytorch https://github.com/lanpa/tensorboard-pytorch
How to Run:
# cd to the /scripts folder.
cd /path-to-this-repository/scripts
# run the shells.
sh resnet164.sh
References:
[1] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Identity
mappings in deep residual networks." In European Conference on
Computer Vision, pp. 630-645. Springer International Publishing, 2016.
[2] Zagoruyko, Sergey, and Nikos Komodakis. "Wide residual networks." arXiv
preprint arXiv:1605.07146 (2016).
[3] Hu, Jie, Li Shen, and Gang Sun. "Squeeze-and-Excitation Networks." arXiv preprint arXiv:1709.01507 (2017).