sl3
: Super Machine Learning with Pipelines
R/
A flexible implementation of the Super Learner ensemble machine learning system
Authors: Jeremy Coyle, Nima Hejazi, Ivana Malenica, Rachael Phillips, and Oleg Sofrygin
sl3
?
Whatβs sl3
is an implementation of the Super Learner ensemble machine
learning algorithm of van der Laan, Polley, and Hubbard (2007). The
Super Learner algorithm performs ensemble learning in one of two
fashions:
- The discrete Super Learner can be used to select the best
prediction algorithm from among a supplied library of machine
learning algorithms (βlearnersβ in the
sl3
nomenclature) β that is, the discrete Super Learner is the single learning algorithm that minimizes the cross-validated risk. - The ensemble Super Learner can be used to assign weights to a set of specified learning algorithms (from a user-supplied library of such algorithms) so as to create a combination of these learners that minimizes the cross-validated risk. This notion of weighted combinations has also been referred to as stacked regression (Breiman 1996) and stacked generalization (Wolpert 1992).
Looking for long-form documentation or a walkthrough of the sl3
package? Donβt worry! Just browse the chapter in our
book.
Installation
Install the most recent version from the master
branch on GitHub via
remotes
:
remotes::install_github("tlverse/sl3")
Past stable releases may be located via the releases page on GitHub and may be installed by including the appropriate major version tag. For example,
remotes::install_github("tlverse/[email protected]")
To contribute, check out the devel
branch and consider submitting a
pull request.
Issues
If you encounter any bugs or have any specific feature requests, please file an issue.
Examples
sl3
makes the process of applying screening algorithms, learning
algorithms, combining both types of algorithms into a stacked regression
model, and cross-validating this whole process essentially trivial. The
best way to understand this is to see the sl3
package in action:
set.seed(49753)
library(tidyverse)
library(data.table)
library(SuperLearner)
library(origami)
library(sl3)
# load example data set
data(cpp)
cpp <- cpp %>%
dplyr::filter(!is.na(haz)) %>%
mutate_all(~ replace(., is.na(.), 0))
# use covariates of intest and the outcome to build a task object
covars <- c("apgar1", "apgar5", "parity", "gagebrth", "mage", "meducyrs",
"sexn")
task <- sl3_Task$new(
data = cpp,
covariates = covars,
outcome = "haz"
)
# set up screeners and learners via built-in functions and pipelines
slscreener <- Lrnr_pkg_SuperLearner_screener$new("screen.glmnet")
glm_learner <- Lrnr_glm$new()
screen_and_glm <- Pipeline$new(slscreener, glm_learner)
SL.glmnet_learner <- Lrnr_pkg_SuperLearner$new(SL_wrapper = "SL.glmnet")
# stack learners into a model (including screeners and pipelines)
learner_stack <- Stack$new(SL.glmnet_learner, glm_learner, screen_and_glm)
stack_fit <- learner_stack$train(task)
preds <- stack_fit$predict()
head(preds)
#> Lrnr_pkg_SuperLearner_SL.glmnet Lrnr_glm_TRUE
#> 1: 0.3525946 0.36298498
#> 2: 0.3525946 0.36298498
#> 3: 0.2442593 0.25993072
#> 4: 0.2442593 0.25993072
#> 5: 0.2442593 0.25993072
#> 6: 0.0269504 0.05680264
#> Pipeline(Lrnr_pkg_SuperLearner_screener_screen.glmnet->Lrnr_glm_TRUE)
#> 1: 0.36228209
#> 2: 0.36228209
#> 3: 0.25870995
#> 4: 0.25870995
#> 5: 0.25870995
#> 6: 0.05600958
future
s
Parallelization with While itβs straightforward to fit a stack of learners (as above), itβs
easy to take advantage of sl3
βs built-in parallelization support too.
To do this, you can simply choose a plan()
from the future
ecosystem.
# let's load the future package and set 4 cores for parallelization
library(future)
plan(multicore, workers = 4L)
# now, let's re-train our Stack in parallel
stack_fit <- learner_stack$train(task)
preds <- stack_fit$predict()
Controlling the number of CV folds
In the above examples, we fit stacks of learners, but didnβt create a Super Learner ensemble, which uses cross-validation (CV) to build the ensemble model. For the sake of computational expedience, we may be interested in lowering the number of CV folds (from 10). Letβs take a look at how to do both below.
# first, let's instantiate some more learners and create a Super Learner
mean_learner <- Lrnr_mean$new()
rf_learner <- Lrnr_ranger$new()
sl <- Lrnr_sl$new(mean_learner, glm_learner, rf_learner)
# CV folds are controlled in the sl3_Task object; we can lower the number of
# folds simply by specifying this in creating the Task
task <- sl3_Task$new(
data = cpp,
covariates = covars,
outcome = "haz",
folds = 5L
)
# now, let's fit the Super Learner with just 5-fold CV, then get predictions
sl_fit <- sl$train(task)
sl_preds <- sl_fit$predict()
The folds
argument to sl3_Task
supports both integers (for V-fold
CV) and all of the CV schemes supported in the origami
package. To see the full
list, query ?fold_funs
from within R
or take a look at origami
βs
online documentation.
Learner Properties
Properties supported by sl3
learners are presented in the following
table:
binomial |
categorical |
continuous |
cv |
density |
h2o |
ids |
importance |
offset |
preprocessing |
sampling |
screener |
timeseries |
weights |
wrapper |
|
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Lrnr_arima |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_bartMachine |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_bayesglm |
β |
x |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_bilstm |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_bound |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
β |
Lrnr_caret |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
Lrnr_cv |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
Lrnr_cv_selector |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
β |
Lrnr_dbarts |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_define_interactions |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
Lrnr_density_discretize |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_density_hse |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_density_semiparametric |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
Lrnr_earth |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_expSmooth |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_gam |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_gbm |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_glm |
β |
x |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_glm_fast |
β |
x |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_glmnet |
β |
β |
β |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_grf |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_gru_keras |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_gts |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_h2o_glm |
β |
β |
β |
x |
x |
β |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_h2o_grid |
β |
β |
β |
x |
x |
β |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_hal9001 |
β |
x |
β |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_haldensify |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_HarmonicReg |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_hts |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_independent_binomial |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_lightgbm |
β |
β |
β |
x |
x |
x |
x |
β |
β |
x |
x |
x |
x |
β |
x |
Lrnr_lstm_keras |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_mean |
β |
β |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_multiple_ts |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_multivariate |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_nnet |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_nnls |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_optim |
β |
β |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_pca |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
Lrnr_pkg_SuperLearner |
β |
x |
β |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
β |
β |
Lrnr_pkg_SuperLearner_method |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
β |
Lrnr_pkg_SuperLearner_screener |
β |
x |
β |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
β |
β |
Lrnr_polspline |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_pooled_hazards |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_randomForest |
β |
β |
β |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
Lrnr_ranger |
β |
β |
β |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
β |
x |
Lrnr_revere_task |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
Lrnr_rpart |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
Lrnr_rugarch |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_screener_augment |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
x |
Lrnr_screener_coefs |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
x |
Lrnr_screener_correlation |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
x |
Lrnr_screener_importance |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
x |
Lrnr_sl |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
Lrnr_solnp |
β |
β |
β |
x |
x |
x |
x |
x |
β |
x |
x |
x |
x |
β |
x |
Lrnr_solnp_density |
x |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_stratified |
β |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
Lrnr_subset_covariates |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_svm |
β |
β |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
Lrnr_ts_weights |
x |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
Lrnr_tsDyn |
x |
x |
β |
x |
x |
x |
x |
x |
x |
x |
x |
x |
β |
x |
x |
Lrnr_xgboost |
β |
β |
β |
x |
x |
x |
x |
β |
β |
x |
x |
x |
x |
β |
x |
Contributions
Contributions are very welcome. Interested contributors should consult our contribution guidelines prior to submitting a pull request.
Citation
After using the sl3
R package, please cite the following:
@software{coyle2021sl3-rpkg,
author = {Coyle, Jeremy R and Hejazi, Nima S and Malenica, Ivana and
Phillips, Rachael V and Sofrygin, Oleg},
title = {{sl3}: Modern Pipelines for Machine Learning and {Super
Learning}},
year = {2021},
howpublished = {\url{https://github.com/tlverse/sl3}},
note = {{R} package version 1.4.2},
url = {https://doi.org/10.5281/zenodo.1342293},
doi = {10.5281/zenodo.1342293}
}
License
Β© 2017-2021 Jeremy R. Coyle, Nima S. Hejazi, Ivana Malenica, Rachael V. Phillips, Oleg Sofrygin
The contents of this repository are distributed under the GPL-3 license.
See file LICENSE
for details.
References
Breiman, Leo. 1996. βStacked Regressions.β Machine Learning 24 (1): 49β64.
van der Laan, Mark J, Eric C Polley, and Alan E Hubbard. 2007. βSuper Learner.β Statistical Applications in Genetics and Molecular Biology 6 (1).
Wolpert, David H. 1992. βStacked Generalization.β Neural Networks 5 (2): 241β59.