Public metrics
Public metrics are hosted at: https://grafana.prod.discovery.etcd.io/d/uiLwPyPWk/discoveryserver?orgId=2
discovery.etcd.io Kubernetes Configurations
This repo contains the code to provision the infrastructure and the Kubernetes configurations to operate the public discovery.etcd.io service.
Requirements
- Helm ~> v3.0.0 - Install latest version for your OS.
- Terraform ~> v0.12.15 Please download the proper package for your operating system and architecture.
Building the infrastructure
The infrastructure is built using cloudkite terraform modules, which are used to provision infrastructure in Google Cloud Platform. The following modules have been used:
-
vpc: The vpc module contains Terraform code to provision a GCP Virtual Private Cloud. See VPC docs.
-
gke: The folder contains Terraform code to deploy a GKE Private Cluster.
Provisioning a VPC and deploying a GKE cluster per environment
The infrastructure main code is created per environment, and there are two environments:
Choose an environment - that is, move to the dev or to prod folder in order to run Terraform commands.
Next step is to apply Terraform for the chosen environment. To ensure that it is configured correctly, apply it and get the expected output, go to the project's terraform folder and follow the README instructions.
Manual Step
Once dev and prod infrastructure is built, it is required to update IAM policies of artifacts.<dev-project>.appspot.com GCS bucket adding both dev and prod gke_service_accounts
as members with the role roles/storage.objectViewer. Only after it, clusters in both environments will be able to pull images from gcr.
Gsutil command to update the IAM policy:
gsutil iam ch serviceAccount:[SERVICE_ACCOUNT_EMAIL]:roles/storage.objectViewer gs://artifacts.<dev-project>.appspot.com
Note: if you get an error that the artifacts.<dev-project>.appspot.com does not exit, push an image and it will be created.
After applying terraform, a GKE cluster will be up and running in the VPC created. Now the cluster is ready to get deployments.
Install Releases with Helm
To get the public discovery service running, the following releases have to be installed:
- Nginx Ingress Controller: Used for routing traffic from beyond the cluster to internal Kubernetes Services. To install follow instructions in README.
- certmanager: Is the TLS/SSL certificate management controller, and to get it deployed follow the README.
- prometheus-operator: Used to create/configure/manage Prometheus clusters atop Kubernetes. To install this operator use the instructions here.
- etcd-operator: Is used to configure and manage etcd clusters. This is a pre-requisite to have configured properly the discoveryserver release. To install it follow instructions in README.
- discoveryserver: Is a service that bootstrap new etcd clusters using an existing one. This service helps when the IPs of your cluster peers are not known ahead of time. To install the release follow instructions in README.
Debugging
Hit the discovery service via kubectl proxy
kubectl proxy
curl http://localhost:8001/api/v1/namespaces/default/services/discoveryserver/proxy/new
Execute etcdctl on the cluster
kubectl exec -it $(kubectl get pods -l app=etcd -o jsonpath='{.items[0].metadata.name}') -- /usr/local/bin/etcdctl watch '' --prefix
Access prometheus
Ensure prometheus's externalURL is set to the right path
kubectl edit prometheus prometheus-operator-prometheus
kubectl proxy
Visit http://localhost:8001/api/v1/namespaces/default/services/prometheus-operated:web/proxy