SCAR - Serverless Container-aware ARchitectures

SCAR is a framework to transparently execute containers out of Docker images in AWS Lambda, in order to run applications (see examples for ImageMagick, FFmpeg and AWS CLI, as well as deep learning frameworks such as Theano and Darknet) and code in virtually any programming language (see examples for Ruby, R, Erlang and Elixir) on AWS Lambda.

SCAR provides the benefits of AWS Lambda with the execution environment you decide, provided as a Docker image available in Docker Hub. It is probably the easiest, most convenient approach to run generic applications on AWS Lambda, as well as code in your favourite programming language, not only in those languages supported by AWS Lambda.

SCAR also supports a High Throughput Computing Programming Model to create highly-parallel event-driven file-processing serverless applications that execute on customized runtime environments provided by Docker containers run on AWS Lambda. The development of SCAR has been published in the Future Generation Computer Systems scientific journal.

SCAR is integrated with API Gateway in order to expose an application via a highly-available HTTP-based REST API that supports both synchronous and asynchronous invocations. It is also integrated with AWS Batch. This way, AWS Lambda can be used to acommodate the execution of large bursts of short requests while long-running executions are delegated to AWS Batch.

SCAR allows to create serverless workflows by combining functions that run on either AWS Batch or AWS Lambda which produce output files that trigger the execution of functions that, again, run on either AWS Batch or AWS Lambda, using the very same Docker images, thus effectively creating highly-scalable cross-services serverless workflows.

Related resources:
Scientific Paper (pre-print).

Update 3.0.0

Since version 3.0.0 SCAR creates a lambda layer called 'faas-supervisor' that includes the core functionality for the lambda containers. This layer allows to deploy new functions faster. The layer is created once (the first time that a function is created or after a layer update) and then it's linked to all the other functions.

If a new version of the supervisor is released (e.g. when a new feature is added or a bug is found) the functions can be updated with the command scar update -a -sl.

To check the supervisor layer version that your function is using you only have to do an ls like scar ls

Documentation

SCAR documentation can be found in readthedocs.

Also the examples have extra information that is usefull to execute them.

Licensing

SCAR is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Further information

There is further information on the architecture of SCAR and use cases in the scientific publication "Serverless computing for container-based architectures" (pre-print available here), included in the Future Generation Computer Systems journal. Please acknowledge the use of SCAR by including the following cite:

A. Pérez, G. Moltó, M. Caballer, and A. Calatrava, “Serverless computing for container-based architectures,” Futur. Gener. Comput. Syst., vol. 83, pp. 50–59, Jun. 2018.

Acknowledgements

• udocker