The official repository for our paper "Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks". We develop a method for analyzing emerging functional modularity in neural networks based on differentiable weight masks and use it to point out important issues in current-day neural networks.

✭ 25

python machine-learning deep-learning paper transformers pytorch rnn modularity generalization weight-sharing

6. transformer generalization

The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We significantly improve the systematic generalization of transformer models on a variety of datasets using simple tricks and careful considerations.

✭ 58

python shell machine-learning deep-learning paper transformers pytorch generalization systematic-generalization

1-6 of 6 user projects