Chaos Injection for AWS Lambda - chaos_lambda

chaos_lambda is a small library injecting chaos into AWS Lambda. It offers simple python decorators to do delay, exception and statusCode injection for your AWS Lambda function. This allows to conduct small chaos engineering experiments for your serverless application in the AWS Cloud.

• Support for Latency injection using fault_type: latency
• Support for Exception injection using fault_type: exception
• Support for HTTP Error status code injection using fault_type: status_code
• Using for SSM Parameter Store to control the experiment using is_enabled: true
• Support for adding rate of failure using rate. (Default rate = 1)
• Per Lambda function injection control using Environment variable (CHAOS_PARAM)

Install

pip install chaos-lambda

Example

# function.py

import os
from chaos_lambda import inject_fault

# this should be set as a Lambda environment variable
os.environ['CHAOS_PARAM'] = 'chaoslambda.config'

@inject_fault
def handler(event, context):
    return {
        'statusCode': 200,
        'body': 'Hello from Lambda!'
    }

Considering a configuration as follows:

{
    "fault_type": "exception",
    "delay": 400,
    "is_enabled": true,
    "error_code": 404,
    "exception_msg": "This is chaos",
    "rate": 1
}

When excecuted, the Lambda function, e.g handler('foo', 'bar'), will produce the following result:

exception_msg from config chaos with a rate of 1
corrupting now
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/.../chaos_lambda.py", line 199, in wrapper
    raise Exception(exception_msg)
Exception: This is chaos

Configuration

The configuration for the failure injection is stored in the AWS SSM Parameter Store and accessed at runtime by the get_config() function:

{
    "fault_type": "exception",
    "delay": 400,
    "is_enabled": true,
    "error_code": 404,
    "exception_msg": "This is chaos",
    "rate": 1
}

To store the above configuration into SSM using the AWS CLI do the following:

aws ssm put-parameter --name chaoslambda.config --type String --overwrite --value "{ "delay": 400, "is_enabled": true, "error_code": 404, "exception_msg": "This is chaos", "rate": 1, "fault_type": "exception"}" --region eu-west-1

AWS Lambda will need to have IAM access to SSM.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ssm:DescribeParameters"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ssm:GetParameters",
                "ssm:GetParameter"
            ],
            "Resource": "arn:aws:ssm:eu-north-1:12345678910:parameter/chaoslambda.config"
        }
    ]
}

Supported Faults:

chaos_lambda currently supports the following faults:

• latency - Add latency in the AWS Lambda execution
• exception - Raise an exception during the AWS Lambda execution
• status_code - force AWS Lambda to return a specific HTTP error code

More information:

Full Documentation