Continual Learning

This is a PyTorch implementation of the continual learning experiments described in the following papers:

• Three scenarios for continual learning (link)
• Generative replay with feedback connections as a general strategy for continual learning (link)

Requirements

The current version of the code has been tested with:

• pytorch 1.1.0
• torchvision 0.2.2

Running the experiments

Individual experiments can be run with main.py. Main options are:

• --experiment: which task protocol? (splitMNIST|permMNIST)
• --scenario: according to which scenario? (task|domain|class)
• --tasks: how many tasks?

To run specific methods, use the following:

• Context-dependent-Gating (XdG): ./main.py --xdg=0.8
• Elastic Weight Consolidation (EWC): ./main.py --ewc --lambda=5000
• Online EWC: ./main.py --ewc --online --lambda=5000 --gamma=1
• Synaptic Intelligence (SI): ./main.py --si --c=0.1
• Learning without Forgetting (LwF): ./main.py --replay=current --distill
• Generative Replay (GR): ./main.py --replay=generative
• GR with distillation: ./main.py --replay=generative --distill
• Replay-trough-Feedback (RtF): ./main.py --replay=generative --distill --feedback
• Experience Replay (ER): ./main.py --replay=exemplars --budget=2000
• Averaged Gradient Episodic Memory (A-GEM): ./main.py --replay=exemplars --agem --budget=2000
• iCaRL: ./main.py --icarl --budget=2000

For information on further options: ./main.py -h.

The code in this repository only supports MNIST-based experiments. An extension to more challenging problems (e.g., with natural images as inputs) can be found here: https://github.com/GMvandeVen/brain-inspired-replay.

Running comparisons from the papers

"Three CL scenarios"-paper

This paper describes three scenarios for continual learning (Task-IL, Domain-IL & Class-IL) and provides an extensive comparion of recently proposed continual learning methods. It uses the permuted and split MNIST task protocols, with both performed according to all three scenarios.

A comparison of all methods included in this paper can be run with compare_all.py (this script includes extra methods and reports additional metrics compared to the paper). The comparison in Appendix B can be run with compare_taskID.py, and Figure C.1 can be recreated with compare_replay.py.

"Replay-through-Feedback"-paper

The three continual learning scenarios were actually first identified in this paper, after which this paper introduces the Replay-through-Feedback framework as a more efficent implementation of generative replay.

A comparison of all methods included in this paper can be run with compare_time.py. This includes a comparison of the time these methods take to train (Figures 4 and 5).

Note that the results reported in this paper were obtained with this earlier version of the code.

On-the-fly plots during training

With this code it is possible to track progress during training with on-the-fly plots. This feature requires visdom, which can be installed as follows:

pip install visdom

Before running the experiments, the visdom server should be started from the command line:

python -m visdom.server

The visdom server is now alive and can be accessed at http://localhost:8097 in your browser (the plots will appear there). The flag --visdom should then be added when calling ./main.py to run the experiments with on-the-fly plots.

For more information on visdom see https://github.com/facebookresearch/visdom.

Citation

Please consider citing our papers if you use this code in your research:

@article{vandeven2019three,
  title={Three scenarios for continual learning},
  author={van de Ven, Gido M and Tolias, Andreas S},
  journal={arXiv preprint arXiv:1904.07734},
  year={2019}
}

@article{vandeven2018generative,
  title={Generative replay with feedback connections as a general strategy for continual learning},
  author={van de Ven, Gido M and Tolias, Andreas S},
  journal={arXiv preprint arXiv:1809.10635},
  year={2018}
}

Acknowledgments

The research projects from which this code originated have been supported by an IBRO-ISN Research Fellowship, by the Lifelong Learning Machines (L2M) program of the Defence Advanced Research Projects Agency (DARPA) via contract number HR0011-18-2-0025 and by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/Interior Business Center (DoI/IBC) contract number D16PC00003. Disclaimer: views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA, IARPA, DoI/IBC, or the U.S. Government.