The Encyclopedia of Kubernetes clusters
Clusterpedia
This name Clusterpedia is inspired by Wikipedia. It is an encyclopedia of multi-cluster to synchronize, search for, and simply control multi-cluster resources.
Clusterpedia can synchronize resources with multiple clusters and provide more powerful search features on the basis of compatibility with Kubernetes OpenAPI to help you effectively get any multi-cluster resource that you are looking for in a quick and easy way.
The capability of Clusterpedia is not only to search for and view but also simply control resources in the future, just like Wikipedia that supports for editing entries.
Architecture
- Clusterpedia APIServer: Register to
Kubernetes APIServerby the means of Aggregated API and provide services through a unified entrance - ClusterSynchro Manager: Manage the cluster synchro that is used to synchronize cluster resources
- Storage Layer: Connect with a specific storage component and then register to Clusterpedia APIServer and ClusterSynchro Manager via a storage layer interface
- Storage Component: A specific storage facility such as MySQL, PostgreSQL, Redis or other Graph Databases
In addition, Clusterpedia will use the Custom Resource - PediaCluster to implement cluster authentication and configure resources for synchronization.
Clusterpedia also provides a Default Storage Layer that can connect with MySQL and PostgreSQL.
Clusterpedia does not care about the specific storage components used by users, you can choose or implement the storage layer according to your own needs, and then register the storage layer in Clusterpedia as a plug-in
Features
- Support for complex search, filters, sorting, paging, and more
- Support for requesting relevant resources when you query resources
- Unify the search entry for master clusters and multi-cluster resources
- Compatible with kubernetes OpenAPI, where you can directly use kubectl for multi-cluster search without any third-party plug-ins or tools
- Compatible with synchronizing different versions of cluster resources, not restricted by the version of master cluster
- High performance and low memory consumption for resource synchronization
- Automatically start/stop resource synchronization according to the current health status of the cluster
- Support for plug-in storage layer. You can use other storage components to customize the storage layer according to your needs.
- High availability
The above unimplemented features are already in the Roadmap
Installation | Import Clusters | Sync Cluster Resources
Usage Samples
We can search for resources configured in PediaCluster, Clusterpedia supports two types of resource search:
- Search for resources that are compatible with Kubernetes OpenAPI
- Search for
Collection Resource
$ kubectl api-resources | grep clusterpedia.io
collectionresources clusterpedia.io/v1beta1 false CollectionResource
resources clusterpedia.io/v1beta1 false ResourcesSearch Label and URL Query
| Role | search label key | url query |
|---|---|---|
| Filter cluster names | search.clusterpedia.io/clusters |
clusters |
| Filter namespaces | search.clusterpedia.io/namespaces |
namespaces |
| Filter resource names | search.clusterpedia.io/names |
names |
| Specified Owner UID | internalstorage.clusterpedia.io/owner-uid |
- |
| Specified Owner Seniority | internalstorage.clusterpedia.io/owner-seniority |
ownerSeniority |
| Order by fields | search.clusterpedia.io/orderby |
orderby |
| Set page size | search.clusterpedia.io/size |
limit |
| Set page offset | search.clusterpedia.io/offset |
continue |
| Response include Continue | search.clusterpedia.io/with-continue |
withContinue |
| Response include remaining count | search.clusterpedia.io/with-remaining-count |
withRemainingCount |
Both Search Labels and URL Query support same operators as Label Selector:
exist,not exist=,==,!=in,notin
More information about Search Conditions, about Label Selector and Field Selector
Search in Kubernetes OpenAPI Compatible Way
Although we can use URL to search resources, if we want to use kubectl to query more conveniently,
we need to configure the cluster shortcut for kubectl.
We can use kubectl --cluster <cluster name> to specify the cluster, if <cluster name> is clusterpedia,
it meas it is a multi-cluster search operation.
First check which resources are synchronized. We cannot find a resource until it is properly synchronized:
$ kubectl --cluster clusterpedia api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
configmaps cm v1 true ConfigMap
events ev v1 true Event
namespaces ns v1 false Namespace
nodes no v1 false Node
pods po v1 true Pod
services svc v1 true Service
daemonsets ds apps/v1 true DaemonSet
deployments deploy apps/v1 true Deployment
replicasets rs apps/v1 true ReplicaSet
statefulsets sts apps/v1 true StatefulSet
cronjobs cj batch/v1 true CronJob
jobs batch/v1 true Job
clusters cluster.kpanda.io/v1alpha1 false Cluster
ingressclasses networking.k8s.io/v1 false IngressClass
ingresses ing networking.k8s.io/v1 true Ingress
clusterrolebindings rbac.authorization.k8s.io/v1 false ClusterRoleBinding
clusterroles rbac.authorization.k8s.io/v1 false ClusterRole
roles rbac.authorization.k8s.io/v1 true Role
$ kubectl --cluster cluster-1 api-resources
...Search in Multiple Clusters
Usage of multi-cluster search in documents
Get deployments in the kube-system namespace of all clusters:
$ kubectl --cluster clusterpedia get deployments -n kube-system
CLUSTER NAME READY UP-TO-DATE AVAILABLE AGE
cluster-1 coredns 2/2 2 2 68d
cluster-2 calico-kube-controllers 1/1 1 1 64d
cluster-2 coredns 2/2 2 2 64dGet deployments in the two namespaces kube-system and default of all clusters:
$ kubectl --cluster clusterpedia get deployments -A -l "search.clusterpedia.io/namespaces in (kube-system, default)"
NAMESPACE CLUSTER NAME READY UP-TO-DATE AVAILABLE AGE
kube-system cluster-1 coredns 2/2 2 2 68d
kube-system cluster-2 calico-kube-controllers 1/1 1 1 64d
kube-system cluster-2 coredns 2/2 2 2 64d
default cluster-2 dd-airflow-scheduler 0/1 1 0 54d
default cluster-2 dd-airflow-web 0/1 1 0 54d
default cluster-2 hello-world-server 1/1 1 1 27d
default cluster-2 openldap 1/1 1 1 41d
default cluster-2 phpldapadmin 1/1 1 1 41dGet deployments in the kube-system and default namespaces in cluster-1 and cluster-2:
$ kubectl --cluster clusterpedia get deployments -A -l "search.clusterpedia.io/clusters in (cluster-1, cluster-2),\
search.clusterpedia.io/namespaces in (kube-system,default)"
NAMESPACE CLUSTER NAME READY UP-TO-DATE AVAILABLE AGE
kube-system cluster-1 coredns 2/2 2 2 68d
kube-system cluster-2 calico-kube-controllers 1/1 1 1 64d
kube-system cluster-2 coredns 2/2 2 2 64d
default cluster-2 dd-airflow-scheduler 0/1 1 0 54d
default cluster-2 dd-airflow-web 0/1 1 0 54d
default cluster-2 hello-world-server 1/1 1 1 27d
default cluster-2 openldap 1/1 1 1 41d
default cluster-2 phpldapadmin 1/1 1 1 41dGet deployments in the kube-system and default namespaces in cluster-1 and cluster-2:
$ kubectl --cluster clusterpedia get deployments -A -l "search.clusterpedia.io/clusters in (cluster-1, cluster-2),\
search.clusterpedia.io/namespaces in (kube-system,default),\
search.clusterpedia.io/orderby=name"
NAMESPACE CLUSTER NAME READY UP-TO-DATE AVAILABLE AGE
kube-system cluster-2 calico-kube-controllers 1/1 1 1 64d
kube-system cluster-1 coredns 2/2 2 2 68d
kube-system cluster-2 coredns 2/2 2 2 64d
default cluster-2 dd-airflow-scheduler 0/1 1 0 54d
default cluster-2 dd-airflow-web 0/1 1 0 54d
default cluster-2 hello-world-server 1/1 1 1 27d
default cluster-2 openldap 1/1 1 1 41d
default cluster-2 phpldapadmin 1/1 1 1 41dSearch a specific cluster
Usage of specified cluster search in documents
If you want to search a specific cluster for any resource therein, you can add --cluster to specify the cluster name:
$ kubectl --cluster cluster-1 get deployments -A
NAMESPACE CLUSTER NAME READY UP-TO-DATE AVAILABLE AGE
calico-apiserver cluster-1 calico-apiserver 1/1 1 1 68d
calico-system cluster-1 calico-kube-controllers 1/1 1 1 68d
calico-system cluster-1 calico-typha 1/1 1 1 68d
capi-system cluster-1 capi-controller-manager 1/1 1 1 42d
capi-kubeadm-bootstrap-system cluster-1 capi-kubeadm-bootstrap-controller-manager 1/1 1 1 42d
capi-kubeadm-control-plane-system cluster-1 capi-kubeadm-control-plane-controller-manager 1/1 1 1 42d
capv-system cluster-1 capv-controller-manager 1/1 1 1 42d
cert-manager cluster-1 cert-manager 1/1 1 1 42d
cert-manager cluster-1 cert-manager-cainjector 1/1 1 1 42d
cert-manager cluster-1 cert-manager-webhook 1/1 1 1 42d
clusterpedia-system cluster-1 clusterpedia-apiserver 1/1 1 1 27m
clusterpedia-system cluster-1 clusterpedia-clustersynchro-manager 1/1 1 1 27m
clusterpedia-system cluster-1 clusterpedia-internalstorage-mysql 1/1 1 1 29m
kube-system cluster-1 coredns 2/2 2 2 68d
tigera-operator cluster-1 tigera-operator 1/1 1 1 68dExcept for search.clusterpedia.io/clusters, the support for other complex queries is same as that for multi-cluster search.
If you want to learn about the details of a resource, you need to specify which cluster it is:
$ kubectl --cluster cluster-1 -n kube-system get deployments coredns -o wide
CLUSTER NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
cluster-1 coredns 2/2 2 2 68d coredns registry.aliyuncs.com/google_containers/coredns:v1.8.4 k8s-app=kube-dnsSearch for Collection Resource
Clusterpedia can also perform more advanced aggregation of resources. For example, you can use Collection Resource to get a set of different resources at once.
Let's first check which Collection Resource currently Clusterpedia supports:
$ kubectl get collectionresources
NAME RESOURCES
workloads deployments.apps,daemonsets.apps,statefulsets.appsBy getting workloads, you can get a set of resources aggregated by deployments, daemonsets, and statefulsets, and Collection Resource also supports for all complex queries.
kubectl get collectionresources workloads will get the corresponding resources of all namespaces in all clusters by default:
$ kubectl get collectionresources workloads
CLUSTER GROUP VERSION KIND NAMESPACE NAME AGE
cluster-1 apps v1 DaemonSet kube-system vsphere-cloud-controller-manager 63d
cluster-2 apps v1 Deployment kube-system calico-kube-controllers 109d
cluster-2 apps v1 Deployment kube-system coredns-coredns 109dAdd the collection of Daemonset in cluster-1 and some of the above output is cut out
Due to the limitation of kubectl, you cannot use complex queries in kubectl and can only be queried by URL Query.
Proposals
Automatic discovery and sync cluster resource
The resource used to represent the cluster in Clusterpedia is called PediaCluster, not a simple Cluster.
This is because Clusterpedia was originally designed to build on the existing multi-cluster management platform.
In order to keep the original intention, the first issue is that Clusterpedia should not conflict with the resources in the existing multi-cluster platform. Cluster is a very common resource name that represents a cluster.
In addition, in order to better connect with the existing multi-cluster platform and enable the connected clusters automatically complete resource synchronization, we need a new mechanism to discover clusters. This discovery mechanism needs to solve the following issues:
- Get the authentication info to access the cluster
- Configure conditions that trigger the lifecycle of PediaCluster
- Set the default policy and prefix name for resource synchronization
This feature will be discussed and implemented in detail in Q1 or Q2 2022.
Perform more complex control over resources
In addition to resource search, similar to Wikipedia, Clusterpedia should also have simple capability of resource control, such as watch, create, delete, update, and more.
In fact, a write action is implemented by double write + warning response.
We will discuss this feature and decide whether we should implement it according to the community needs
Notes
Multi-cluster network connectivity
Clusterpedia does not actually solve the problem of network connectivity in a multi-cluster environment. You can use tools such as tower to connect and access sub-clusters, or use submariner or skupper to solve cross-cluster network problems.
Contact
If you have any question, feel free to reach out to us in the following ways:
Contributors
Made with contrib.rocks.
License
Copyright 2022 the Clusterpedia Authors. All rights reserved.
Licensed under the Apache License, Version 2.0.