On the Robustness of Semantic Segmentation Models to Adversarial Attacks
This code performs the experiments presented in our paper, On the Robustness of Semantic Segmentation Models to Adversarial Attacks. The code has been refactored from the version used to perform the experiments in the paper, but should reproduce the results.
If you find this work useful, please cite
@inproceedings{arnab_cvpr_2018,
title={On the Robustness of Semantic Segmentation Models to Adversarial Attacks},
author={Arnab, Anurag and Miksik, Ondrej and Torr, Philip H. S.},
booktitle={CVPR},
year={2018}
}
Installation
The adversarial attacks are implemented in the Caffe framework.
To run the code, Caffe must be installed and pycaffe should be present in the $PYTHONPATH environment variable.
Adversarial attacks
Variants of the FGSM attack, which produces adversarial perturbations with bounded l_{infinity} norm, are implemented in attacks.py.
The code does not depend on the version of Caffe being used.
Therefore, any fork of Caffe can be used (as long as it is in $PYTHONPATH), allowing many different types of models to be attacked.
Demos
Segmentation on Cityscapes
Run ./examples/demo_cityscapes.sh. You should see the following in the output directory which will be created.
 |
 |
 |
|---|---|---|
| Input image | Original prediction | Adversarial prediction |
Segmentation on Pascal VOC
Run ./examples/demo_voc.sh. You should see the following in the output directory which will be created.
 |
 |
 |
|---|---|---|
| Input image | Original prediction | Adversarial prediction |
ImageNet classification
Run ../examples/demo_imagenet.sh. You should see
 |
 |
|---|---|
| Original image | Adversarial image |
Prediction of original image
[281 282 285 277 287]
n02123045 tabby, tabby cat 0.312
n02123159 tiger cat 0.238
n02124075 Egyptian cat 0.124
n02119022 red fox, Vulpes vulpes 0.101
n02127052 lynx, catamount 0.071
Prediction of adversarial image
[274 272 277 273 278]
n02115913 dhole, Cuon alpinus 1.000
n02114855 coyote, prairie wolf, brush wolf, Canis latrans 0.000
n02119022 red fox, Vulpes vulpes 0.000
n02115641 dingo, warrigal, warragal, Canis dingo 0.000
n02119789 kit fox, Vulpes macrotis 0.000
Running on your own images
You can modify the demo script, or refer to opts.py for the command line arguments that need to be passed to the main.py.
Note the keys which are expected to be in the model definition protoxt, as described next
Model definitions
The model defintion prototxt should set force_backward: true. This is necessary to compute gradients.
The code assumes that there are nodes in the graph named data, label and output.
These are the RGB image, the label and logits (not softmax-normalised) predicted by the network.
An example of this is the here
Contact
For any queries, contact [email protected]. Pull requests are also welcome.