An Updating Survey for Bayesian Deep Learning (BDL)

This is an updating survey for Bayesian Deep Learning (BDL), an constantly updated and extended version for the manuscript, 'A Survey on Bayesian Deep Learning', published in ACM Computing Surveys 2020.

Contents

• Survey
• BDL and Recommender Systems
• BDL and Domain Adaptation
• BDL and Healthcare
• BDL and Natural Language Processing (NLP)
• BDL and Computer Vision (CV)
• BDL and Control
• BDL and Graphs (Link Prediction, Graph Neural Networks, etc.)
• BDL and Topic Modeling
• BDL and Speech Recognition/Synthesis
• BDL and Forecasting (Time Series Analysis)
• BDL as a Framework (Miscellaneous)
• Bayesian/Probabilistic Neural Networks as Building Blocks of BDL

Survey

A Survey on Bayesian Deep Learning
by Wang et al., ACM Computing Surveys (CSUR) 2020
[PDF] [Blog] [BDL Framework in 2016]

BDL and Recommender Systems

Collaborative Deep Learning for Recommender Systems
by Wang et al., KDD 2015
[PDF] [Project Page] [2014 Arxiv Version] [Code] [MXNet Code] [TensorFlow Code] [Dataset A] [Dataset B] [Jupyter Notebook] [Slides] [Slides (Long)]

Collaborative Recurrent Autoencoder: Recommend while Learning to Fill in the Blanks
by Wang et al., NIPS 2016
[PDF]

Collaborative Knowledge Base Embedding for Recommender Systems
by Zhang et al., KDD 2016
[PDF]

Collaborative Deep Ranking: A Hybrid Pair-Wise Recommendation Algorithm with Implicit Feedback
by Ying et al., PAKDD 2016
[PDF]

Collaborative Variational Autoencoder for Recommender Systems
by Li et al., KDD 2017
[PDF]

Variational Autoencoders for Collaborative Filtering
by Liang et al., WWW 2018
[PDF]

Probabilistic Metric Learning with Adaptive Margin for Top-K Recommendation
by Ma et al., KDD 2020
[PDF]

BDL and Domain Adaptation

Continuously Indexed Domain Adaptation
by Wang et al., ICML 2020
[PDF]

BDL and Healthcare

Electronic Health Record Analysis via Deep Poisson Factor Models
by Henao et al., JMLR 2016
[PDF]

Structured Inference Networks for Nonlinear State Space Models
by Krishnan et al., AAAI 2017
[PDF]

Black Box FDR
by Tansey et al., ICML 2018
[PDF]

Bidirectional Inference Networks: A Class of Deep Bayesian Networks for Health Profiling
by Wang et al., AAAI 2019
[PDF]

Sampling-free Uncertainty Estimation in Gated Recurrent Units with Applications to Normative Modeling in Neuroimaging
by Hwang et al., UAI 2019
[PDF]

Neural Jump Stochastic Differential Equations
by Jia et al., NIPS 2019
[PDF]

Towards Interpretable Clinical Diagnosis with Bayesian Network Ensembles Stacked on Entity-Aware CNNs
by Chen et al., ACL 2020
[PDF]

Continuously Indexed Domain Adaptation
by Wang et al., ICML 2020
[PDF] [Cross Referenced in BDL and Domain Adaptation]

BDL and NLP

Sequence to Better Sequence: Continuous Revision of Combinatorial Structures
by Mueller et al., ICML 2017
[PDF]

QuaSE: Sequence Editing under Quantifiable Guidance
by Liao et al., EMNLP 2018
[PDF]

Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation
by Shi et al., ICML 2020
[PDF]

Towards Interpretable Clinical Diagnosis with Bayesian Network Ensembles Stacked on Entity-Aware CNNs
by Chen et al., ACL 2020
[PDF] [Cross Referenced in BDL and Healthcare]

BDL and Computer Vision

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
by Eslami et al., NIPS 2016
[PDF]

Efficient Inference in Occlusion-aware Generative Models of Images
by Huang et al., ICLR 2016
[PDF]

Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
by Kosiorek et al., NIPS 2018
[PDF]

Gaussian Process Prior Variational Autoencoders
by Casale et al., NIPS 2018
[PDF]

Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks
by Crawford et al., AAAI 2019
[PDF]

Faster Attend-Infer-Repeat with Tractable Probabilistic Models
by Stelzner et al., ICML 2019
[PDF]

Asynchronous Temporal Fields for Action Recognition
by Sigurdsson et al., CVPR 2017
[PDF]

Generalizing Eye Tracking with Bayesian Adversarial Learning
by Wang et al., CVPR 2019
[PDF]

Sequential Neural Processes
by Singh et al., NIPS 2019
[PDF]

Being Bayesian about Categorical Probability
by Joo et al., ICML 2020
[PDF]

Generative Neurosymbolic Machines
by Jiang et al., NIPS 2020
[PDF]

BDL and Control

Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
by Watter et al., NIPS 2015
[PDF]

Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
by Karl et al., ICLR 2017
[PDF]

Probabilistic Recurrent State-Space Models
by Doerr et al., ICML 2018
[PDF]

Learning Latent Dynamics for Planning from Pixels
by Hafner et al., ICML 2019
[PDF]

BDL and Graphs (Link Prediction, Graph Neural Networks, Knowledge Graphs, etc.)

Relational Deep Learning: A Deep Latent Variable Model for Link Prediction
by Wang et al., AAAI 2017
[PDF]

Graphite: Iterative Generative Modeling of Graphs
by Grover et al., ICML 2019
[PDF]

Relational Variational Autoencoder for Link Prediction with Multimedia Data
by Li et al., ACM MM 2017
[PDF]

Stochastic Blockmodels meet Graph Neural Networks
by Mehta et al., ICML 2019
[PDF]

Scalable Deep Generative Modeling for Sparse Graphs
by Dai et al., ICML 2020
[PDF]

PGM-Explainer: Probabilistic Graphical Model Explanations for Graph Neural Networks
by Vu et al., NIPS 2020
[PDF]

Dirichlet Graph Variational Autoencoder
by Li et al., NIPS 2020
[PDF]

Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs
by Ren et al., NIPS 2020
[PDF]

BDL and Topic Modeling

Relational Stacked Denoising Autoencoder for Tag Recommendation
by Wang et al., AAAI 2015
[PDF]

Scalable Deep Poisson Factor Analysis for Topic Modeling
by Gan et al., ICML 2015
[PDF]

Deep Latent Dirichlet Allocation with Topic-layer-adaptive Stochastic Gradient Riemannian MCMC
by Cong et al., ICML 2017
[PDF]

Deep Unfolding for Topic Models
by Chien et al., TPAMI 2017
[PDF]

Neural Relational Topic Models for Scientific Article Analysis
by Bai et al., CIKM 2018
[PDF]

Dirichlet Belief Networks for Topic Structure Learning
by Zhao et al., NIPS 2018
[PDF]

Deep Relational Topic Modeling via Graph Poisson Gamma Belief Network
by Wang et al., NIPS 2020
[PDF]

BDL and Speech Recognition/Synthesis

Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
by Hsu et al., NIPS 2017
[PDF]

Scalable Factorized Hierarchical Variational Autoencoder Training
by Hsu et al., Interspeech 2018
[PDF]

Hierarchical Generative Modeling for Controllable Speech Synthesis
by Hsu et al., ICLR 2019
[PDF]

Recurrent Poisson Process Unit for Speech Recognition
by Huang et al., AAAI 2019
[PDF]

Deep Graph Random Process for Relational-thinking-based Speech Recognition
by Huang et al., ICML 2020
[PDF]

BDL and Forecasting (Time Series Analysis)

DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks
by Salinas et al., 2017
[PDF]

Deep State Space Models for Time Series Forecasting
by Rangapuram et al., NIPS 2018
[PDF]

Deep Factors for Forecasting
by Wang et al., ICML 2019
[PDF]

Probabilistic Forecasting with Spline Quantile Function RNNs
by Gasthaus et al., AISTATS 2019
[PDF]

Adversarial Attacks on Probabilistic Autoregressive Forecasting Models
by Dang-Nhu et al., ICML 2020
[PDF]

BDL as a Framework (Miscellaneous)

Towards Bayesian Deep Learning: A Framework and Some Existing Methods
by Wang et al., TKDE 2016
[PDF]

Composing Graphical Models with Neural Networks for Structured Representations and Fast Inference
by Johnson et al., NIPS 2016
[PDF]

Bayesian/Probabilistic Neural Networks as Building Blocks of BDL

Learning Stochastic Feedforward Networks
by Neal et al., Technical Report 1990
[PDF]

A Practical Bayesian Framework for Backprop Networks
by MacKay et al., Neural Computation 1992
[PDF]

Keeping Neural Networks Simple by Minimizing the Description Length of the Weights
by Hinton et al., COLT 1993
[PDF]

Practical Variational Inference for Neural Networks
by Alex Graves, NIPS 2011
[PDF]

Auto-Encoding Variational Bayes
by Kingma et al., ArXiv 2014
[PDF]

Deep Exponential Families
by Ranganath et al., AISTATS 2015
[PDF]

Weight Uncertainty in Neural Networks
by Blundell et al., ICML 2015
[PDF]

Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
by Hernandez-Lobato et al., ICML 2015
[PDF]

Variational Dropout and the Local Reparameterization Trick
by Kingma et al., NIPS 2015
[PDF]

The Poisson Gamma Belief Network
by Zhou et al., NIPS 2015
[PDF]

Deep Poisson Factor Modeling
by Henao et al., NIPS 2015
[PDF]

Natural-Parameter Networks: A Class of Probabilistic Neural Networks
by Wang et al., NIPS 2016
[PDF] [Project Page] [Code]

Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
by Mescheder et al., ICML 2017
[PDF]

Stick-Breaking Variational Autoencoders
by Nalisnick et al., ICLR 2017
[PDF]

Bayesian GAN
by Saatchi et al, NIPS 2017
[PDF]

Neural Expectation Maximization
by Greff et al., NIPS 2017
[PDF]

Lightweight Probabilistic Deep Networks
by Gast et al., CVPR 2018
[PDF]

Feed-forward Propagation in Probabilistic Neural Networks with Categorical and Max Layers
by Shekhovtsov et al., ICLR 2018
[PDF]

Glow: Generative Flow with Invertible 1x1 Convolutions
by Kingma et al., NIPS 2018
[PDF]

ProbGAN: Towards Probabilistic GAN with Theoretical Guarantees
by He et al., ICLR 2019
[PDF] [Project Page]

Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagation
by Postels et al., ICCV 2019
[PDF]

Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
by Dusenberry et al., ICML 2020
[PDF]

Neural Clustering Processes
by Pakman et al., ICML 2020
[PDF]

Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
by Kristiadi et al., ICML 2020
[PDF]

Activation-level Uncertainty in Deep Neural Networks
by Morales-Alvarez et al., ICLR 2021
[PDF]