danfeiX / Model Based Papers

A list of papers on model-based control that I have read so far. The ones that I particularly liked are marked with ⭐️.

Model Learning and Model-predictive Control (MPC)

• Learning model-based planning from scratch, R. Pascanu and Y.Li et al., Arxiv 2017
• Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models, K. Chua et al., NIPS 2018
• SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning, M. Zhang et al., arXiv 2018
• Interaction Networks for Learning about Objects, Relations and Physics, P. Battaglia et al., NIPS 2016 ⭐️
• Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids, Y. Li et al., arXiv 2018 ⭐️
• Propagation Networks for Model-Based Control Under Partial Observation, Y. Li et al., arXiv 2018
• Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation, D. Corneil et al., ICML 2018 ⭐️
• A Compositional Object-Based Approach to Learning Physical Dynamics, M. Chang et al., ICLR 2017 ⭐️
• SPNets: Differentiable Fluid Dynamics for Deep Neural Networks, C. Schenck et al., CoRL 2018
• Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing, A. Ajay et al., IROS 2018
• Graph networks as learnable physics engines for inference and control, A. Sanchez-Gonzalez et al., arXiv 2018 ⭐️
• Learning Latent Dynamics for Planning from Pixels, D. Hafner et al., arXiv 2018

Pixel to Control

• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images, M. Watter and J. Springenberg et al., NIPS 2015 ⭐️
• Robust Locally-Linear Controllable Embedding, E. Banijamali et al., AISTATS 2018
• End-to-End Training of Deep Visuomotor Policies, S. Levine et al., JMLR 2016
• Unsupervised Learning for Physical Interaction through Video Prediction, C. Finn et al., NIPS 2016
• Deep Spatial Autoencoders for Visuomotor Learning, C. Finn et al., ICRA 2016
• Deep Visual Foresight for Planning Robot Motion, C. Finn et al., ICRA 2017
• Learning Plannable Representations with Causal InfoGAN, T. Kurutach et al., NIPS 2018 ⭐️
• Learning Latent Dynamics for Planning from Pixels, D. Hafner et al., arXiv 2018
• SE3-Pose-Nets: Structured Deep Dynamics Models for Visuomotor Planning and Control, A. Byravan et al., ICRA 2018 ⭐️

Model-based + Model-free

• MBMF: Model-Based Priors for Model-Free Reinforcement Learning, S. Bansal et al., CoRL 2017
• Continuous deep q-learning with model-based acceleration, S. Gu et al., ICML 2016
• Recurrent World Models Facilitate Policy Evolution, D. Ha et al., NIPS 2018

Learned Optimal Control

• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images, M. Watter and J. Springenberg et al., NIPS 2015 ⭐️
• Robust Locally-Linear Controllable Embedding, E. Banijamali et al., AISTATS 2018
• Differentiable MPC for End-to-end Planning and Control, B. Amos et al., NIPS 2018 ⭐️
• Path Integral Networks: End-to-End Differentiable Optimal Control, Okada et al., NIPS 2017
• Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning, M. Deisenroth et al., RSS 2011
• SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning, M. Zhang et al., arXiv 2018

State Estimation

• Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data, M. Carl et al., ICLR 2017 ⭐️
• Deep Kalman Filters R. G. Krishnan et al., arXiv 2015
• Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors, R. Jonschkowski et al., RSS 2018 ⭐️
• QMDP-Net: Deep Learning for Planning under Partial Observability, P. Karkus et al., NIPS 2017
• Generative Temporal Models with Spatial Memory for Partially Observed Environments, M. Fraccaro et al., ICML 2018 ⭐️

Survey

• Learning Physical Dynamical Systems for Prediction and Control: A Survey, J. LaChance, 2018

Koopman Theory

• Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition, N Takeishi et al., NIPS 2017 ⭐️
• Deep Dynamical Modeling and Control of Unsteady Fluid Flows, J. Morton et al., NIPS 2018
• Deep learning for universal linear embeddings of nonlinear dynamics, B Lusch et al., Nature Communications 2018
• Data-driven discovery of Koopman eigenfunctions for control, E. Kaiser et al., arXiv 2017

Optimal Control

• Control-Limited Differential Dynamic Programming, Y. Tassa et al., ICRA 2014

Other Resources

• Learning Dynamical System Models from Data, Sergey Levine, CS 294-112: Deep Reinforcement Learning
• EE263: Introduction to Linear Dynamical Systems
• CS 287: Advanced Robotics
• Control Bootcamp, Steve Brunton, 2017
• STUDYWOLF blog, Travis DeWolf