All Projects → zalando-incubator → Cluster Lifecycle Manager

zalando-incubator / Cluster Lifecycle Manager

Licence: mit
Cluster Lifecycle Manager (CLM) to provision and update multiple Kubernetes clusters

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to Cluster Lifecycle Manager

Udacity Data Engineering Projects
Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
Stars: ✭ 458 (+129%)
Mutual labels:  aws, cloudformation, cluster
Serverlessbydesign
A visual approach to serverless development. Think. Build. Repeat.
Stars: ✭ 254 (+27%)
Mutual labels:  aws, cloud, cloudformation
Cfn Sphere
AWS CloudFormation stack management tool
Stars: ✭ 76 (-62%)
Mutual labels:  aws, cloud, cloudformation
Arvados
An open source platform for managing and analyzing biomedical big data
Stars: ✭ 274 (+37%)
Mutual labels:  aws, cloud, cluster
Docker Swarm
🐳🐳🐳 This repository is part of a blog series on Docker Swarm example using VirtualBox, OVH Openstack, Azure and Amazon Web Services AWS
Stars: ✭ 43 (-78.5%)
Mutual labels:  aws, cloud, cluster
Sceptre
Build better AWS infrastructure
Stars: ✭ 1,160 (+480%)
Mutual labels:  aws, cloud, cloudformation
Awesome Aws
A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.
Stars: ✭ 9,895 (+4847.5%)
Mutual labels:  aws, cloud, cloudformation
Aws Labs
step by step guide for aws mini labs. Currently maintained on : https://github.com/Cloud-Yeti/aws-labs Youtube playlist for labs:
Stars: ✭ 153 (-23.5%)
Mutual labels:  aws, cloudformation
Kafka Connect Storage Cloud
Kafka Connect suite of connectors for Cloud storage (Amazon S3)
Stars: ✭ 153 (-23.5%)
Mutual labels:  aws, cloud
Firecamp
Serverless Platform for the stateful services
Stars: ✭ 194 (-3%)
Mutual labels:  aws, cloud
Diagram Maker
A library to display an interactive editor for any graph-like data.
Stars: ✭ 2,086 (+943%)
Mutual labels:  aws, cloud
Checkov
Prevent cloud misconfigurations during build-time for Terraform, Cloudformation, Kubernetes, Serverless framework and other infrastructure-as-code-languages with Checkov by Bridgecrew.
Stars: ✭ 3,572 (+1686%)
Mutual labels:  aws, cloudformation
Stratosphere
Haskell EDSL and type-checker for AWS CloudFormation templates
Stars: ✭ 151 (-24.5%)
Mutual labels:  aws, cloudformation
Awscloudformation Samples
Sample AWS CloudFormation templates
Stars: ✭ 153 (-23.5%)
Mutual labels:  aws, cloudformation
Gardener
Kubernetes-native system managing the full lifecycle of conformant Kubernetes clusters as a service on Alicloud, AWS, Azure, GCP, OpenStack, EquinixMetal, vSphere, MetalStack, and Kubevirt with minimal TCO.
Stars: ✭ 2,093 (+946.5%)
Mutual labels:  aws, cluster
Autospotting
Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups. Installs in minutes using CloudFormation or Terraform. Convenient to deploy at scale using StackSets. Uses tagging to avoid launch configuration changes. Automated spot termination handling. Reliable fallback to on-demand instances.
Stars: ✭ 2,014 (+907%)
Mutual labels:  aws, cloudformation
Cloudformation Cli
The CloudFormation Provider Development Toolkit allows you to author your own resource providers and modules that can be used by CloudFormation.
Stars: ✭ 149 (-25.5%)
Mutual labels:  aws, cloudformation
Aws Sdk Perl
A community AWS SDK for Perl Programmers
Stars: ✭ 153 (-23.5%)
Mutual labels:  aws, cloudformation
Deploykit
A toolkit for creating and managing declarative, self-healing infrastructure.
Stars: ✭ 2,237 (+1018.5%)
Mutual labels:  cloud, cluster
Externalsecret Operator
An operator to fetch secrets from cloud services and inject them in Kubernetes
Stars: ✭ 177 (-11.5%)
Mutual labels:  aws, cloud

Cluster Lifecycle Manager (CLM)

Build Status Coverage Status Go Report Card

The Cluster Lifecycle Manager (CLM) is a component responsible for operating (create, update, delete) Kubernetes clusters. It interacts with a Cluster Registry and a configuration source from which it reads information about the clusters and keep them up to date with the latest configuration.

clm

The CLM is designed to run either as a CLI tool for launching clusters directly from your development machine, or as a controller running as a single instance operating many clusters.

It is designed in a reentrant way meaning it can be killed at any point in time and it will just continue any cluster updates from where it left off. All state is stored in the Cluster Registry and the git configuration repository.

For a better understanding on how we use the CLM within Zalando, see the 2018 KubeCon EU talk:

Current state

The CLM has been developed internally at Zalando since January 2017. It's currently used to operate 80+ clusters on AWS where the oldest clusters has been continuously updated all the way from Kubernetes v1.4 to Kubernetes v1.9 by the CLM.

It is currently tightly coupled with our production cluster configuration, but by making it Open Source and developing it in the open going forward we aim to make the CLM useful as a generic solution for operating Kubernetes clusters at scale.

Features

  • Automatically trigger cluster updates based on changes to a Cluster Registry defined either as an HTTP REST API or a yaml file.
  • Automatically trigger cluster updates based on configuration changes, where configuration is stored in a remote git repository or a local directory.
  • Perform Non-disruptive Rolling Updates of nodes in a cluster especially with respect to stateful applications.
  • Declarative deletion of decommissioned cluster resources.

How to build it

This project uses Go modules as introduced in Go 1.11 therefore you need Go >=1.11 installed in order to build. If using Go 1.11 you also need to activate Module support.

Assuming Go has been setup with module support it can be built simply by running:

export GO111MODULE=on # needed if the project is checked out in your $GOPATH.
$ make

How to run it

To run CLM you need to provide at least the following information:

  • URI to a registry --registry either a file path or a url to a cluster registry.
  • A $TOKEN used for authenticating with the target Kubernetes cluster once it has been provisioned (the $TOKEN is an assumption of the Zalando setup, we should support a generic kubeconfig in the future).
  • URL to repository containing the configuration --git-repository-url or, in alternative, a directory --directory

Run CLM locally

To run CLM locally you can use the following command. This assumes valid AWS credentials on your machine e.g. in ~/.aws/credentials.

$ ./build/clm provision \
  --registry=clusters.yaml \
  --token=$TOKEN \
  --directory=/path/to/configuration-folder \
  --debug

The provision command does a cluster create or update depending on whether the cluster already exists. The other command is decommission which terminates the cluster.

The clusters.yaml is of the following format:

clusters:
- id: cluster-id
  alias: alias-for-cluster-id # human readable alias
  local_id: local-cluster-id  # used for separating clusters in the same AWS account
  api_server_url: https://kube-api.example.org
  config_items:
    custom_config_item: value # custom key/value config items
  criticality_level: 1
  environment: test
  infrastructure_account: "aws:12345678910" # AWS account ID
  region: eu-central-1
  provider: zalando-aws
  node_pools:
  - name: master-default
    profile: master-default
    min_size: 2
    max_size: 2
    instance_type: m5.large
    discount_strategy: none
  - name: worker-default
    profile: worker-default
    min_size: 3
    max_size: 20
    instance_type: m5.large
    discount_strategy: none

Deletions

By default the Cluster Lifecycle Manager will just apply any manifest defined in the manifests folder. In order to support deletion of deprecated resources the CLM will read a deletions.yaml file of the following format:

pre_apply: # everything defined under here will be deleted before applying the manifests
- name: mate
  namespace: kube-system
  kind: deployment
post_apply: # everything defined under here will be deleted after applying the manifests
- namespace: kube-system
  kind: deployment
  labels:
    application: external-dns
    version: "v1.0"

Whatever is defined in this file will be deleted pre/post applying the other manifest files, if the resource exists. If the resource has already been deleted previously it's treated as a no-op.

A resource can be identified either by name or labels if both are defined the name will be used. If none of them are defined, it's an error.

namespace can be left out, in which case it will default to kube-system.

kind must be one of the kinds defined in kubectl get.

Configuration defaults

CLM will look for a config-defaults.yaml file in the cluster configuration directory. If the file exists, it will be evaluated as a Go template with all the usual CLM variables and functions available, and the resulting output will be parsed as a simple key-value map. CLM will use the contents of the file to populate the cluster's configuration items, taking care not to overwrite the existing ones.

For example, you can use the defaults file to have different settings for production and test clusters, while keeping the manifests readable:

  • config-defaults.yaml:

    {{ if eq .Environment "production"}}
    autoscaling_buffer_pods: "3"
    {{else}}
    autoscaling_buffer_pods: "0"
    {{end}}
    
  • manifests/example/example.yaml:

    
    spec:
      replicas: {{.ConfigItems.autoscaling_buffer_pods}}
    
    

Non-disruptive rolling updates

One of the main features of the CLM is the update strategy implemented which is designed to do rolling node updates which are non-disruptive for workloads running in the target cluster. Special care is taken to support stateful applications.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].