This repo is out of maintenance. Browse the new leaderboard from here! New repo link: https://github.com/sokcertifiedrobustness/sokcertifiedrobustness.github.io.

Provable Training and Verification Approaches Towards Robust Neural Networks

Recently, provable (i.e. certified) adversarial robustness training and verification methods have demonstrated their effectiveness against adversarial attacks. In contrast to empirical robustness and empirical adversarial attacks, the provable robustness verification provides rigorous lower bound of robustness for a given neural network, such that no existing or future attacks will attack further.

Note that the training methods towards robust networks are usually connected with the corresponding verification approach. For instance, after training, the robustness bound is often measure on the test set in terms of "robust accuracy"(RACC). One data sample is considered to be provable robust if and only if we can prove that there is no adversarial samples exist in the neighborhood, i.e., the model always outputs the current prediction label in the neighborhood. The neighborhood is usually defined by L-norm distance.

Tighter provable robustness bound can be achieved by better robust training approaches, and tighter robustness verification approaches, or jointly.

Updates:(Jun 2022) The accompanying SoK paper is accepted by IEEE S&P (Oakland) 2023!

If you find this repo helpful, please consider cite our paper:

@inproceedings{li2023sok,
    title={SoK: Certified Robustness for Deep Neural Networks},
    author={Linyi Li and Tao Xie and Bo Li},
    booktitle={44th {IEEE} Symposium on Security and Privacy, {SP} 2023, San Francisco, CA, USA, 22-26 May 2023},
    publisher={IEEE},
    year={2023}
}

News:

• We are happy to announce the FIRST large-scale study of representative certifiably robust defenses with interesting insights @ https://arxiv.org/abs/2009.04131! (It is also a useful paper list for certified robustness of DNNs)
• We also release a unified toolbox VeriGauge for implementing robustness verification approaches conveniently with PyTorch: https://github.com/AI-secure/VeriGauge. Feel free to try it and give us feedback!
• We include a taxnomy tree of representative approaches in this field (adapted from our large-scale study paper) at the bottom and here.

Table of Contents

• Scope of the Repo

• Contact & Updates

• Main Leaderboard
  • ImageNet
    • L2
      • eps = 0.2
      • eps = 0.5
      • eps = 1.0
      • eps = 2.0
      • eps = 3.0
    • L-Infty
      • eps=1/255
      • eps=1.785/255
  • CIFAR-10
    • L2
      • eps=0.14
      • eps=0.25
      • eps=0.5
      • eps=1.0
      • eps=1.5
    • L Infty
      • eps=2/255
      • eps=8/255
  • MNIST
    • L2
      • eps=1.58
    • L-Infty
      • eps=0.1
      • eps=0.3
      • eps=0.4
  • SVHN
    • L2
      • eps=0.1
      • eps=0.2
    • L-Infty
      • eps=0.01
      • eps=8/255
  • Fashion-MNIST
    • L-Infty
      • eps=0.1
• Reference: Empirical Robustness
  • CIFAR-10
    • L-Infty
      • eps=8/255
  • MNIST
    • L-Infty
      • eps=0.3
• Taxonomy Tree

Scope of the Repo

Current works mainly focus on image classification tasks with datasets MNIST, CIFAR10, ImageNet, FashionMNIST, and SVHN.

We focus on perturbation measured by L-2 and L-infty norms.

This repo mainly records recent progress of above settings, while advances in other settings are recorded in the attached paperlist.

We only consider single model robustness.

Contact & Updates

We are trying to keep track of all important advances of provable robustness approaches, but may still miss some.

Please feel free to contact us (Linyi([email protected]) @ UIUC Secure Learning Lab & Illinois ASE Group) or commit your updates :)

Main Leaderboard

ImageNet

All input images contain three channels; each pixel is in range [0, 255].

L2

eps = 0.2

eps = 0.5

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

eps = 1.0

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

eps = 2.0

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

eps = 3.0

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

L-Infty

eps=1/255

eps=1.785/255

In above table, the dataset is ImageNet-200 rather than ImageNet-1000 in other tables.

CIFAR-10

All input images have three channels; 32 x 32 x 3 size; each pixel is in range [0, 255].

L2

eps=0.14

eps=0.25

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

eps=0.5

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

eps=1.0

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

eps=1.5

All above approaches use Randomized Smoothing (Cohen et al) to derive certification, with wrong probability at 0.1%.

L Infty

eps=2/255

eps=8/255

MNIST

All input images are grayscale; 28 x 28 size; each pixel is in range [0, 1].

L2

eps=1.58

eps=1.58 is transformed from L-infty eps=0.1.

L-Infty

eps=0.1

eps=0.3

eps=0.4

SVHN

The image size is 32 x 32 x 3 (3-channel in color). Pixel colors in [0, 255]. When calculating eps, these values are rescaled to [0, 1].

L2

eps=0.1

eps=0.2

L-Infty

eps=0.01

eps=8/255

Fashion-MNIST

This is a MNIST-like dataset. Images are 28 x 28 and grayscale. Values are in [0, 1].

L-Infty

eps=0.1

*. Within one dataset, L-2 and L-Infty balls are mutually transformable. After transformation, a corresponding tight bound may exist but not listed.

Notes:

1. Some papers use rarely-used epsilon to report their results, which may increase comparison difficulty. Some papers use epsilon after regularization instead of raw one, which may also induce confusion.

   We would suggest to adapt common evaluation epsilon values and settings.

2. Instead of evaluating on above benchmarks and reporting the robust accuracy, some papers tend to report average robust radius. We will add comparison table for such metric later.

3. Besides the on-the-board results, all these papers have their own unique takeaways. For interested reader and stackholders, we recommend to not only value the approach with higher numbers, but also dig into their technical meat.

Reference: Empirical Robustness

For comparison, here we cite numbers from MadryLab repositories for MNIST challenge and CIFAR-10 challenge, which records the best attacks towarding their robust model with secret weights.

CIFAR-10

L-Infty

eps=8/255

Block-Box

White-Box

MNIST

L-Infty

eps=0.3

Black-Box

White-Box

Taxonomy Tree

Full introduction of these approaches is available at https://arxiv.org/abs/2009.04131.

Maintained by Linyi.

Last updated: Sept 24, 2020