dream11 / kong-circuit-breaker

Overview

kong-circuit-breaker is a Kong plugin that provides circuit-breaker functionality at the route level. It uses lua-circuit-breaker library internally to wrap proxy calls around a circuit-breaker pattern. The functionality provided by this plugin is similar to libraries like resilience4j in Java.

Usecase

In high throughput use cases, if an API of an upstream service results in timeouts/failures, the following will happen:

1. It will bring a cascading failure effect to Kong and reduce its performance
2. Continued calls to upstream service (which is facing downtime) will prevent the upstream service from recovering Thus, it is essential for proxy calls made from Kong to fail fast using an intelligent configurable mechanism, leading to improved resiliency and fault tolerance.

Behaviour

The circuit breaker works like an electric circuit breaker only as it has three states:

1. Open: The CB will not allow any requests to this route, and it will fail fast.
2. Half-open: The CB will allow few requests to this route based on the configuration to check if it fails or passes.
3. Closed: All requests will work as usual.

How does it work?

Internally, the plugin uses lua-circuit-breaker library to wrap proxy calls made by Kong with a circuit-breaker.

1. To decide whether a route is in a healthy/unhealthy state, success % and failure % are calculated in a time window of window_time seconds.
2. For any calculation to happen in step 1, the total number of requests in the time window should >= min_calls_in_window.
3. If failure % calculated crosses failure_percent_threshold circuit is opened. This prevents any more calls to this route until wait_duration_in_open_state seconds have elapsed. After this, the circuit transitions to the half-open state automatically
4. In the half-open state, when total_requests >= half_open_min_calls_in_window, failure % is calculated to resolve circuit-breaker into the open or the closed state.
5. If the circuit-breaker cannot resolve the state in the wait_duration_in_half_open_state seconds, it automatically transitions into the closed state.

Installation

luarocks

luarocks install kong-circuit-breaker

You will also need to enable this plugin by adding it to the list of enabled plugins using KONG_PLUGINS environment variable or the plugins key in kong.conf

export KONG_PLUGINS=circuit-breaker

OR

plugins=circuit-breaker

source

Clone this repo and run:

luarocks make

Usage

conf = {
	window_time = 15,
	min_calls_in_window = 20,
	api_call_timeout_ms = 500,
	failure_percent_threshold = 51,
	wait_duration_in_open_state = 15,
	wait_duration_in_half_open_state = 180,
	error_status_code = 599
}	

You can add this plugin on a global / service / route level in Kong API Gateway.

• Lets say you add this plugin at a global-level with conf, this will create a CB object for each route.
• If you want to exclude some routes from being wrapped with CB then use conf.excluded_apis.
• If you want to override the configuration of global-level CB for a route (say GET /test), then enable this plugin for GET /test route also with a different conf.

Parameters

Caveats

1. Circuit breaker uses time window to count failures, successes, and total_requests. These windows are not sliding, i.e., if you create a window of 10 seconds, it will create windows like:

    window_1 (  0s - 10s ),
    window_2 ( 10s - 20s ),
    window_3 ( 20s - 30s ) ...

2. Circuit-breaker object is created for each route in each nginx worker. The state of CB object (like counters) is never shared among workers. While setting the configuration, carefully set parameters like min_calls_in_window taking total nginx workers into account.
3. Circuit breaker uses failure % to figure out if a route is healthy or not. Always set min_calls_in_window to start calculations; else, you may open the circuit when total_requests are relatively low.
4. Set half_open_max_calls_in_window to prevent allowing too many requests to the route in the half-open state.
5. set_logger_metrics_in_ctx sets circuit_breaker_name, upstream_service_host and circuit_breaker_state in kong.ctx.shared.logger_metrics.circuit_breaker. You can later use this data within context of a request to log these events.

Inspired by

• lua-circuit-breaker
• resilience4j