K8S RMQ Autoscaler
K8S Autoscaler is a Pod that will run in your k8s cluster and automatically:
- watch for your deployments that match k8s-rmq-autoscaler annotations
- watch rabbitmq for messages in queues and consumers
- choose to scale up / down the deployment
K8s Configuration
Create the namespace
kubectl create namespace k8s-rmq-autoscaler
Create a secret with the RMQ Password
kubectl create secret generic rmq-credentials --from-literal=RMQ_PASSWORD=test -n k8s-rmq-autoscaler
Edit k8s-rmq-autoscaler.yml with your RMQ information and then you can apply the k8s configuration
See bellow for ENV configuration
kubectl apply -f k8s-rmq-autoscaler.yml
You can then watch the logs
kubectl logs -f pod/k8s-rmq-autoscaler -n k8s-rmq-autoscaler
Now we add annotations to a deployment
kubectl annotate deployment/your-deployment -n namespace \
k8s-rmq-autoscaler/enable=true \
k8s-rmq-autoscaler/max-workers=20 \
k8s-rmq-autoscaler/min-workers=4 \
k8s-rmq-autoscaler/queue=worker-queue \
k8s-rmq-autoscaler/vhost=vhost
Now your deployment is watched by the autoscaler
Annotations
| Config | Mandatory | Description |
|---|---|---|
enable |
true |
enable the autoscaling on this deployment |
max-workers |
true |
the maximum amount of worker to scale up |
min-workers |
true |
the minimum amount of worker to scale down |
queue |
true |
RMQ queue to watch |
vhost |
true |
RMQ vhost where the queue can be found |
messages-per-worker |
false |
Default: 1, set the number of message per worker |
cooldown-delay |
false |
Default: 0s, How long the autoscaler has to wait before another downscale operation can be performed after the current one has completed. (Duration: 5m0s) |
steps |
false |
Default: 1, How many workers will be scale up/down if needed |
offset |
false |
Default: 0, The offset will be added if you always want more workers than message in queue. For example, if you set 1 on offset, you will always have 1 worker more than messages |
override |
false |
Default: false, Authorize the user to scale more than the max/min limits manually |
safe-unscale |
false |
Default: true, Forbid the scaler to scale down when you still have message in queue. Used to avoid to unscale a worker that is processing a message |
Environnement config
| Config | Description |
|---|---|
RMQ_USER |
RMQ Username used for authentication with the RabbitMQ API |
RMQ_PASSWORD |
RMQ Password used for authentication with the RabbitMQ API |
RMQ_URL |
RMQ URL with scheme (Ex. https://rmq:15772) |
IN_CLUSTER |
Boolean that indicate if your are inside the cluster or not (default true) |
NAMESPACES |
namespaces to watch separated by commas, (default, watching all namespaces) |
TICK |
Seconds between checks for autoscaling process (default 10) |