INFO320: Neural Networks and Bayesian Learning

An elective course at the Master o Informatics (MIN) of the Universidad Austral de Chile

Professor: Pablo Huijse H, phuijse at inf dot uach dot cl

Abstract

In this course we will study probabilistic programming techniques that scale to massive datasets (Variational Inference), starting from the fundamentals and also reviewing existing implementations with emphasis on training deep neural network models that have a Bayesian interpretation. The objective is to present the student with the state of the art that lays at the intersection between the fields of Bayesian models and Deep Learning through lectures, paper reviews and practical exercises in Python (Pytorch plus Pyro)

Contents

• Unit 1: Fundamentals
• Unit 2: Bayesian Neural Networks
• Unit 3: Advanced/recent topics

References

Mandatory:

• Barber, D. (2012). Bayesian reasoning and machine learning. Cambridge University Press.
• MacKay, D. J. (2003). Information theory, inference and learning algorithms. Cambridge university press.

Suggested:

• Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518), 859-877.
• Jospin, L. V., Buntine, W., Boussaid, F., Laga, H., & Bennamoun, M. (2020). Hands-on Bayesian Neural Networks--a Tutorial for Deep Learning Users. arXiv preprint arXiv:2007.06823.
• Deep Bayes Moscow 2019

Complementary

• Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT press.
• Theodoridis, S. (2015). Machine learning: a Bayesian and optimization perspective. Academic press.

Software and Libraries

• Programming language: Python3
• Development environment: IPython and Jupyter
• Libraries: PyTorch and Pyro

Recommended installation

To run these notebooks I recommend creating a conda environment, installing pytorch from the pytorch conda channel and installing pyro using pip

If you have a GPU you can copy and paste the following

conda create -n "info320"
conda install -c pyviz holoviews bokeh
conda install -c pytorch pytorch cudatoolkit ignite
conda install scipy scikit-learn selenium 
pip install pyro-ppl