SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data (AAAI 2021)
PyTorch implementation of SnapMix | paper
Method Overview
Cite
@inproceedings{huang2021snapmix,
title={SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data},
author={Shaoli Huang, Xinchao Wang, and Dacheng Tao},
year={2021},
booktitle={AAAI Conference on Artificial Intelligence},
}
Setup
Install Package Dependencies
torch
torchvision
PyYAML
easydict
tqdm
scikit-learn
efficientnet_pytorch
pandas
opencv
Datasets
create a soft link to the dataset directory
CUB dataset
ln -s /your-path-to/CUB-dataset data/cub
Car dataset
ln -s /your-path-to/Car-dataset data/car
Aircraft dataset
ln -s /your-path-to/Aircraft-dataset data/aircraft
Training
Training with Imagenet pre-trained weights
1. Baseline and Baseline+
To train a model on CUB dataset using the Resnet-50 backbone,
python main.py # baseline
python main.py --midlevel # baseline+
To train model on other datasets using other network backbones, you can specify the following arguments:
--netname: name of network architectures (support 4 network families: ResNet,DenseNet,InceptionV3,EfficientNet)
--dataset: dataset name
For example,
python main.py --netname resnet18 --dataset cub # using the Resnet-18 backbone on CUB dataset
python main.py --netname efficientnet-b0 --dataset cub # using the EfficientNet-b0 backbone on CUB dataset
python main.py --netname inceptoinV3 --dataset aircraft # using the inceptionV3 backbone on Aircraft dataset
2. Training with mixing augmentation
Applying SnapMix in training ( we used the hyperparameter values (prob=1., beta=5) for SnapMix in most of the experiments.):
python main.py --mixmethod snapmix --beta 5 --netname resnet50 --dataset cub # baseline
python main.py --mixmethod snapmix --beta 5 --netname resnet50 --dataset cub --midlevel # baseline+
Applying other augmentation methods (currently support cutmix,cutout,and mixup) in training:
python main.py --mixmethod cutmix --beta 3 --netname resnet50 --dataset cub # training with CutMix
python main.py --mixmethod mixup --prob 0.5 --netname resnet50 --dataset cub # training with MixUp
3. Results
ResNet architecture.
| Backbone | Method | CUB | Car | Aircraft |
|---|---|---|---|---|
| Resnet-18 | Baseline | 82.35% | 91.15% | 87.80% |
| Resnet-18 | Baseline + SnapMix | 84.29% | 93.12% | 90.17% |
| Resnet-34 | Baseline | 84.98% | 92.02% | 89.92% |
| Resnet-34 | Baseline + SnapMix | 87.06% | 93.95% | 92.36% |
| Resnet-50 | Baseline | 85.49% | 93.04% | 91.07% |
| Resnet-50 | Baseline + SnapMix | 87.75% | 94.30% | 92.08% |
| Resnet-101 | Baseline | 85.62% | 93.09% | 91.59% |
| Resnet-101 | Baseline + SnapMix | 88.45% | 94.44% | 93.74% |
| Resnet-50 | Baseline+ | 87.13% | 93.80% | 91.68% |
| Resnet-50 | Baseline+ + SnapMix | 88.70% | 95.00% | 93.24% |
| Resnet-101 | Baseline+ | 87.81% | 93.94% | 91.85% |
| Resnet-101 | Baseline+ + SnapMix | 89.32% | 94.84% | 94.05% |
InceptionV3 architecture.
| Backbone | Method | CUB |
|---|---|---|
| InceptionV3 | Baseline | 82.22% |
| InceptionV3 | Baseline + SnapMix | 85.54% |
DenseNet architecture.
| Backbone | Method | CUB |
|---|---|---|
| DenseNet121 | Baseline | 84.23% |
| DenseNet121 | Baseline + SnapMix | 87.42% |
Training from scratch
To train a model without using ImageNet pretrained weights:
python main.py --mixmethod snapmix --prob 0.5 --netname resnet18 --dataset cub --pretrained 0 # resnet-18 backbone
python main.py --mixmethod snapmix --prob 0.5 --netname resnet50 --dataset cub --pretrained 0 # resnet-50 backbone
2. Results
| Backbone | Method | CUB |
|---|---|---|
| Resnet-18 | Baseline | 64.98% |
| Resnet-18 | Baseline + SnapMix | 70.31% |
| Resnet-50 | Baseline | 66.92% |
| Resnet-50 | Baseline + SnapMix | 72.17% |