scheduled-pod-autoscaler

Custom Kubernetes controller for GitOps native scheduled scaling.

Autoscaling with HorizontalPodAutoscaler is difficult to use for spike access etc. ScheduledPodAutoscaler can for more flexible autoscaling by controlling min/max replicas at specific times.

Overview

ScheduledPodAutoscaler is made up of two custom resources.

The parent-child relationship can look like this:

$ kubectl tree scheduledpodautoscaler nginx
NAMESPACE  NAME                             READY  REASON  AGE
default    ScheduledPodAutoscaler/nginx     -              6m5s
default    ├─HorizontalPodAutoscaler/nginx  -              6m5s
default    ├─Schedule/test-1                -              6m4s
default    ├─Schedule/test-2                -              6m4s
default    └─Schedule/test-3                -              6m4s

ScheduledPodAutoscaler

ScheduledPodAutoscaler is a custom resource that wraps HorizontalPodAutoscaler. The ScheduledPodAutoscaler Controller generates a HorizontalPodAutoscaler from this resource.

The specs of the HorizontalPodAutoscaler defined here will be used when no scheduled scaling is taking place.

for example:

apiVersion: autoscaling.d-kuro.github.io/v1
kind: ScheduledPodAutoscaler
metadata:
  name: nginx
spec:
  horizontalPodAutoscalerSpec:
    scaleTargetRef:
      apiVersion: apps/v1
      kind: Deployment
      name: nginx
    minReplicas: 3
    maxReplicas: 10
    metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

$ kubectl get spa # You can use spa as a short name of scheduledpodautoscaler.
NAME    MINPODS   MAXPODS   STATUS      AGE
nginx   3         10        Available   6m52s

Schedule

Schedule is a custom resource for defining scheduled scaling. You can define multiple children's Schedule for the parent ScheduledPodAutoscaler.

The ScheduledPodAutoscaler controller refers to the Schedule and rewrites HorizontalPodAutoscaler created by ScheduledPodAutoscaler when it is time for scheduled scaling. HorizontalPodAutoscaler is not managed in Git, so there are no diffs in GitOps.

📝 Note: A case of schedule conflicts

In case of a schedule conflict, using the maximum value of min/max replicas.

📝 Note: Warm-up time

The ScheduledPodAutoscaler controller only changes the min/max replica of HorizontalPodAutoscaler. Launching the Pod will take some time. Be sure to set a generous amount of time for scheduled scaling.

$ kubectl get schedule -o wide
NAME     REFERENCE   TYPE      STARTTIME          ENDTIME            STARTDAYOFWEEK   ENDDAYOFWEEK   MINPODS   MAXPODS   STATUS      AGE
test-1   nginx       Weekly    20:10              20:15              Saturday         Saturday       1         1         Available   4m49s
test-2   nginx       Daily     20:20              20:25                                              2         2         Available   4m49s
test-3   nginx       OneShot   2020-10-31T20:30   2020-10-31T20:35                                   4         4         Completed   4m49s

Schedule supports 3 different schedule types.

type: Weekly

Write the time in the format of HH:mm and specify the day of the week.

apiVersion: autoscaling.d-kuro.github.io/v1
kind: Schedule
metadata:
  name: nginx-push-notification
spec:
  scaleTargetRef:
    apiVersion: autoscaling.d-kuro.github.io/v1
    kind: ScheduledPodAutoscaler
    name: nginx
  minReplicas: 10
  maxReplicas: 20
  type: Weekly
  startDayOfWeek: Monday
  startTime: "11:50"
  endDayOfWeek: Wednesday
  endTime: "13:00"
  timeZone: Asia/Tokyo

type: Daily

Write the time in the format of HH:mm.

apiVersion: autoscaling.d-kuro.github.io/v1
kind: Schedule
metadata:
  name: nginx-push-notification
spec:
  scaleTargetRef:
    apiVersion: autoscaling.d-kuro.github.io/v1
    kind: ScheduledPodAutoscaler
    name: nginx
  minReplicas: 10
  maxReplicas: 20
  type: Daily
  startTime: "11:50"
  endTime: "13:00"
  timeZone: Asia/Tokyo

type: OneShot

Write the time in the format of yyyy-MM-ddTHH:mm.

apiVersion: autoscaling.d-kuro.github.io/v1
kind: Schedule
metadata:
  name: nginx-push-notification
spec:
  scaleTargetRef:
    apiVersion: autoscaling.d-kuro.github.io/v1
    kind: ScheduledPodAutoscaler
    name: nginx
  minReplicas: 10
  maxReplicas: 20
  type: OneShot
  startTime: "2020-09-01T10:00"
  endTime: "2020-09-10T19:00"
  timeZone: Asia/Tokyo

Install

All resources (the CRDs, Deployment and RBAC) are included in a single YAML manifest file.

By default, it is installed in the kube-system namespace.

# Kubernetes v1.16+
$ kubectl apply -f https://raw.githubusercontent.com/d-kuro/scheduled-pod-autoscaler/v0.0.3/manifests/install/install.yaml

# Kubernetes < v1.16
$ kubectl apply -f https://raw.githubusercontent.com/d-kuro/scheduled-pod-autoscaler/v0.0.3/manifests/install/legacy/install.yaml

Kustomize

Support for kustomize remote base:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  # Kubernetes v1.16+
  - github.com/d-kuro/scheduled-pod-autoscaler/manifests/install?ref=v0.0.3

  # Kubernetes < v1.16
  # - github.com/d-kuro/scheduled-pod-autoscaler/manifests/install/legacy?ref=v0.0.3

Spec

ScheduledPodAutoscaler

Schedule

Metrics

scheduled-pod-autoscaler exports metrics in OpenMetrics format.

You can get the metrics by access to http://localhost:8080/metrics. The port of the metrics endpoint can be changed using the metrics-addr option.

Custom Metrics

Controller Options