anatome 
Ἀνατομή is a PyTorch library to analyze internal representation of neural networks
This project is under active development and the codebase is subject to change.
v0.0.5 introduces significant changes to distance.
Installation
anatome requires
Python>=3.9.0
PyTorch>=1.10
torchvision>=0.11
After the installation of PyTorch, install anatome as follows:
pip install -U git+https://github.com/moskomule/anatome
Available Tools
Representation Similarity
To measure the similarity of learned representation, anatome.SimilarityHook is a useful tool. Currently, the following
methods are implemented.
- Raghu et al. NIPS2017 SVCCA
- Marcos et al. NeurIPS2018 PWCCA
- Kornblith et al. ICML2019 Linear CKA
- Ding et al. arXiv Orthogonal Procrustes distance
import torch
from torchvision.models import resnet18
from anatome import Distance
random_model = resnet18()
learned_model = resnet18(pretrained=True)
distance = Distance(random_model, learned_model, method='pwcca')
with torch.no_grad():
distance.forward(torch.randn(256, 3, 224, 224))
# resize if necessary by specifying `size`
distance.between("layer3.0.conv1", "layer3.0.conv1", size=8)Loss Landscape Visualization
from torch.nn import functional as F
from torchvision.models import resnet18
from anatome import landscape2d
x, y, z = landscape2d(resnet18(),
data,
F.cross_entropy,
x_range=(-1, 1),
y_range=(-1, 1),
step_size=0.1)
imshow(z)
Fourier Analysis
- Yin et al. NeurIPS 2019 etc.,
from torch.nn import functional as F
from torchvision.models import resnet18
from anatome import fourier_map
map = fourier_map(resnet18(),
data,
F.cross_entropy,
norm=4)
imshow(map)
Citation
If you use this implementation in your research, please cite as:
@software{hataya2020anatome,
author={Ryuichiro Hataya},
title={anatome, a PyTorch library to analyze internal representation of neural networks},
url={https://github.com/moskomule/anatome},
year={2020}
}