Tanzu Kubernetes Grid GitOps Flow Sample

Last Run Against TKG 1.1.3

This is a sample showing the power of Cluster API and Tanzu Kubernetes Grid in a GitOps work model.

You start with the Management Cluster, do a typical install of TKG following the directions here

1. Fork this repository, rename references in the code pointing to https://github.com/voor/cluster-api-gitops to your fork. Push those changes.
2. (Optional for deploying workloads into a new cluster upon creation) Go into management-cluster/deploy/workload-secrets and change the example file accordingly.
   1. This assumes you are using CloudFlare to manage your domain specified in the domain secret, if you are not you'll need to change the letsencrypt and external-dns programs in the workload folder accordingly.
   2. You have a GitHub OAuth2.0 application setup, or you'll need to change dex-config.yaml to point elsewhere.
3. Populate the management cluster, go into the management-cluster folder and run:

   ./populate-management.sh
   

4. Create your first workload cluster:

   ./create-cluster.sh hello-world -c 1 -p dev -w 1
   

5. Commit the ${CLUSTER_NAME}.yaml file that was added into workload/manifests into git and push it. (If you skipped the optional step you will need to remove the ${CLUSTER_NAME}-apps App CR)
6. Wait a few minutes.
7. (Optional for deploying workloads into a new cluster upon creation) Visit the URL https://kuard.apps.${DOMAIN} where ${DOMAIN} is the url you specified in the workload-secrets yaml.
8. Or if you didn't do the optional step, use tkg get credentials to get your new clusters credentials.

Scaling Clusters

Workers

Go into the yaml for the workload cluster and change the number of MachineDeployment replicas.

Masters

Go into the yaml for the workload cluster and change the number of KubeadmControlPlane replicas.

Upgrading Clusters

Modify the 00-cluster-values.yaml to roll the machines to a new AMI. Modify the Kubernetes versions to roll the system containers.

Changing Workloads

Everything for workload clusters is found in the workload folder in this repository, this sample takes a few commonly used applications that are both Helm, Jsonnet, and pure manifest based.

Helm

Helm applications are committed directly as manifests to remove any surprises and make this more air-gapped friendly. kapp-controller supports helm charts directly, so see there for more documentation on how to modify this to pull directly from a helm repository.

Pure Manifests

Everything is run through ytt, so even if you are checking out manifests directly from a release (and if you are, you can always just fetch over URL), you can apply overlays or other modifications without committing them directly to the release files.