Awesome Online Machine Learning

Online machine learning is a subset of machine learning where data arrives sequentially. In contrast to the more traditional batch learning, online learning methods update themselves incrementally with one data point at a time.

Courses and books

• Introduction to Online Learning
• Machine Learning the Feature - Gives some insights into the inner workings of Vowpal Wabbit, especially the slides on online linear learning.
• Machine learning for data streams with practical examples in MOA
• Online Methods in Machine Learning (MIT)
• Streaming 101: The world beyond batch
• Prediction, Learning, and Games
• Introduction to Online Convex Optimization

Blog posts

• Machine learning is going real-time (Chip Huyen, 2020)
• What is online machine learning? (Max Pagels, 2018)
• Adaptive Real Time Machine Learning (Sundeep Pothula)
• What Is It and Who Needs It (Data Science Central, 2015)

Software

• River - A Python library for general purpose online machine learning.
• dask
• Jubatus
• LIBFFM - A Library for Field-aware Factorization Machines
• LIBLINEAR - A Library for Large Linear Classification
• LIBOL - A collection of online linear models trained with first and second order gradient descent methods. Not maintained.
• MOA
• scikit-learn - Some of scikit-learn's estimators can handle incremental updates, although this is usually intended for mini-batch learning. See also the "Computing with scikit-learn" page.
• Spark Streaming - Doesn't do online learning per say, but instead mini-batches the data into fixed intervals of time.
• SofiaML
• StreamDM - A machine learning library on top of Spark Streaming.
• Tornado
• VFML
• Vowpal Wabbit

Papers

Linear models

• Field-aware Factorization Machines for CTR Prediction (2016)
• Practical Lessons from Predicting Clicks on Ads at Facebook (2014)
• Ad Click Prediction: a View from the Trenches (2013)
• Normalized online learning (2013)
• Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent (2011)
• Dual Averaging Methods for Regularized Stochastic Learning andOnline Optimization (2010)
• Adaptive Regularization of Weight Vectors (2009)
• Stochastic Gradient Descent Training forL1-regularized Log-linear Models with Cumulative Penalty (2009)
• Confidence-Weighted Linear Classification (2008)
• Exact Convex Confidence-Weighted Learning (2008)
• Online Passive-Aggressive Algorithms (2006)
• Logarithmic Regret Algorithms forOnline Convex Optimization (2007)
• A Second-Order Perceptron Algorithm (2005)
• Online Learning with Kernels (2004)
• Solving Large Scale Linear Prediction Problems Using Stochastic Gradient Descent Algorithms (2004)

Support vector machines

• Pegasos: Primal Estimated sub-GrAdient SOlver for SVM (2007)
• A New Approximate Maximal Margin Classification Algorithm (2001)
• The Relaxed Online Maximum Margin Algorithm (2000)

Neural networks

• Three scenarios for continual learning (2019)

Decision trees

• AMF: Aggregated Mondrian Forests for Online Learning (2019)
• Mondrian Forests: Efficient Online Random Forests (2014)
• Mining High-Speed Data Streams (2000)

Unsupervised learning

• Online hierarchical clustering approximations (2019)
• DeepWalk: Online Learning of Social Representations (2014)
• Online Learning with Random Representations (2014)
• Online Latent Dirichlet Allocation with Infinite Vocabulary (2013)
• Web-Scale K-Means Clustering (2010)
• Online Dictionary Learning For Sparse Coding (2009)
• Density-Based Clustering over an Evolving Data Stream with Noise (2006)
• Knowledge Acquisition Via Incremental Conceptual Clustering (2004)
• Online and Batch Learning of Pseudo-Metrics (2004)
• BIRCH: an efficient data clustering method for very large databases (1996)

Time series

• Online Learning for Time Series Prediction (2013)

Drift detection

• A Survey on Concept Drift Adaptation (2014)

Anomaly detection

• Interpretable Anomaly Detection with Mondrian Pólya Forests on Data Streams (2020)
• Fast Anomaly Detection for Streaming Data (2011)

Metric learning

• Online Metric Learning and Fast Similarity Search (2009)
• Information-Theoretic Metric Learning (2007)
• Online and Batch Learning of Pseudo-Metrics (2004)

Graph theory

• DeepWalk: Online Learning of Social Representations (2014)

Ensemble models

• Optimal and Adaptive Algorithms for Online Boosting (2015) - An implementation is available here
• Online Bagging and Boosting (2001)
• A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting (1997)

Expert learning

• On the optimality of the Hedge algorithm in the stochastic regime

Miscellaneous

• Multi-Output Chain Models and their Application in Data Streams (2019)
• A Complete Recipe for Stochastic Gradient MCMC (2015)
• Online EM Algorithm for Latent Data Models (2007) - Source code is available here

Surveys

• Online Learning: A Comprehensive Survey (2018)
• Online Machine Learning in Big Data Streams (2018)
• Incremental learning algorithms and applications (2016)
• Batch-Incremental versus Instance-Incremental Learning in Dynamic and Evolving Data
• Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey (2011)
• Online Learning and Stochastic Approximations (1998)

General-purpose algorithms

• Maintaining Sliding Window Skylines on Data Streams (2006)
• The Sliding DFT (2003) - An online variant of the Fourier Transform, a concise explanation is available here
• Sketching Algorithms for Big Data