OKD-LAB: Controlled Environment for OKD4 experiments

You plan a dedicated machine to install and experiment with the Community Distribution of Kubernetes, maybe even on an rented root server in the wild wild world?

You want to manage your OKD cluster and applications the GitOps way?

It is probably worth the time to read a little further....

Naturally when we do some experiments we can destroy our cluster and bring it in a state we can't fix or recover. From this point of view we should try to keep complex things simple and repeatable. This is what this lab wants to address to.

You can expect a fully virtualized small IT center with everything you need to install a User Provisioned Infrastructure (UPI) of OKD4 based on KVM.

Additionally you get mostly all you need for a development environment including git, artifact management, private container registry, centralized user registry..... everything pre-configured and tightly integrated.

OKD-LAB: Overview

Prerequisites

• Dedicated root server (recommended)
• Internet access*
• Git client
• SSH / VNC client
• Visual Studio Code (optional but highly recommended!)

This project is being developed on a Hetzner machine with the following specs:

• AMD Ryzen 9 3900 12-Core
• 128 GB DDR4 ECC
• 2 x 1,92 TB NVMe SSD

You can do it with less but than you have to tweak some settings and/or strip off some optional services.

*Please Note! NO proxy support in this version! Following soon.

Installation

95% of the installation process is copy&paste. No deep Linux or OKD4/Kubernetes skills needed!*

• Install CentOS 8.4
• Setup - lab.okd.example.com
• Provision infrastructure
• Install OKD4

*The missing 5% is a guided CentOS 8.4 Linux installation and using a Firefox to create some tokens.

What's in the box?

Watch a animated gif at dropbox and open pandorra's box.

Operatiing system and virtualization:

• Centos 8.4
• KVM / QEMU

Automation and provisioning:

• Terraform
• Packer
• Ansible

Bastion (KVM):

• Centos 8.4
• OKD4 - UPI installation environment:
  • OKD4 Registry Mirror
  • Fedora CoreOS Mirror
  • NTP
  • DNS
  • DHCP
  • TFTP
• Project Quay with Clair
• Podman, Skopeo, Buildah (no Docker!)
• 389 Directory
• GitLab
• Artifactory

Load Balancer (KVM):

• Centos 8.4
• HAProxy

OKD4 (KVMs):

• Bootstrap
• 3x Master
• 3x Worker

What do you get from the OKD / Kubernetes world?

Terraform/Ansible managed:

• 3x Master and 3x Worker
• Chrony time services configured on all master and worker nodes
• Trusted private Project Quay container registry
• Trusted custom Certificate Authority and SSL certificates for Web console, Router, API, LDAP, Project Quay, Podman etc.
• LDAP(s) authorization provider with:
  • Administrators: admin, lab in the cluster-admin role
  • Team Members: awesome-admin, awesome-developer
• Enabled Image Pruner and disabled Samples Operator

Argo CD (GitOps) managed:

• Operators and instances:
  • OpenShift Logging
  • OpenShift Elastic Search
  • Quay Container Security
  • Grafana Operator
• Storage:
  • Rook Ceph
• Argo Project:
  • ArgoCD
  • Argo ApplicationSet
  • Argo Workflows
  • Argo Events
  • Argo Rollouts
• Tekton:
  • Tekton Pipelines
  • Tekton Triggers
  • Tekton Dashboard
  • OpenShift TektonCD Catalog
• Secret Management:
  • Sealed Secrets
• Container Build:
  • Shipwright
• Security:
  • Kyverno
  • Policy Reporter

Security

Especially with servers available in the wild wild world some kind of security makes sense!

For this reason:

• A Firewall is running on this lab and only SSH (port 53) is allowed on the external interface.
• Only SSH PubkeyAuthentication is allowed.
• Only necessary services are enabled.
• Except SSH all network services are bound to localhost.
• Virtual network is not directly reachable from the wild world.
• Visual Studio Code and VNC is only available via SSH tunnel.

If you go the Hetzner path additional security is possible and recommended.

A few words

This guide is not about installing and maintaining Linux at the highest possible levels. It's not about being the best of class automation expert and it's a controlled environment with intentionally 99% static settings. But if you know what you do, you can change and expand everything with ease and apply it to your needs. Have fun!

Thanks to all in the Open Source Community and especially to @cgruver for inspiration and help!

License

OKD-LAB is released under the Apache 2.0 license. See the LICENSE file for details. Some components may be licensed differently - consult individual vendors and repositories for more.