cert-sync
cert-sync is a Istio add-on to automate the synchronization between Kubernetes TLS secrets and istio-ingressgateway certificates, without having to re-deploy the istio-ingressgateway pods.
For any user-specified Kubernetes TLS secret, it will ensure there's a pair of corresponding public certificate (.crt) and private key (.key) exist in a shared volume path, which will later be loaded by istio-ingressgateway. If users update/delete those secrets, their corresponding keys and certs will be updated/deleted, too.
Deployment
As already menthioned above, the idea of how cert-sync collaborates with istio-ingressgateway is that they share a volume path (directory), where certificates and private keys in TLS secrets will be placed into.
Generally, users may choose one of the following methods to deploy cert-sync.
Sidecar
The sidecar way is straightforward. We simply add a cert-sync container into istio-ingressgateway's pod spec, and mount a shared volume to both of them. For example,
...
...
...
containers:
- name: ingressgateway
image: "gcr.io/istio/proxyv2:1.0.0"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
- containerPort: 443
- containerPort: 31400
args:
- proxy
- router
- -v
- "2"
- --discoveryRefreshDelay
- '1s' #discoveryRefreshDelay
- --drainDuration
- '45s' #drainDuration
- --parentShutdownDuration
- '1m0s' #parentShutdownDuration
- --connectTimeout
- '10s' #connectTimeout
- --serviceCluster
- istio-ingressgateway
- --zipkinAddress
- zipkin:9411
- --statsdUdpAddress
- istio-statsd-prom-bridge:9125
- --proxyAdminPort
- "15000"
- --controlPlaneAuthPolicy
- NONE
- --discoveryAddress
- istio-pilot.istio-system:8080
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: INSTANCE_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: ISTIO_META_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
volumeMounts:
- name: certdir
mountPath: "/etc/istio/ingressgateway-certs"
- name: cert-sync
image: "dunjut/cert-sync:0.1.0"
imagePullPolicy: IfNotPresent
args:
- --certDir
- /etc/istio/ingressgateway-certs
volumeMounts:
- name: certdir
mountPath: "/etc/istio/ingressgateway-certs"
volumes:
- name: certdir
emptyDir: {}
...
Note a shared volume is a requirement, but its type doesn't have to be emptyDir. Users may choose other volume types like hostPath and cephfs as long as these are preferred under their Kubernetes environments.
To get cert-sync working correctly, users also have to add secrets accessibility to the
istio-ingressgateway-istio-system ClusterRole. Here is what will be look like:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
labels:
app: gateways
chart: gateways-1.0.0
heritage: Tiller
release: istio
name: istio-ingressgateway-istio-system
rules:
- apiGroups: ["extensions"]
resources: ["thirdpartyresources", "virtualservices", "destinationrules", "gateways"]
verbs: ["get", "watch", "list", "update"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
Now the final step is to re-apply the modified istio-ingressgateway configurations.
DaemonSet
Some users may run Kubernetes in bare-metal environments and do not have LoadBalancer service supported. Under this circumstance, cluster operators may deploy istio- ingressgateway in edge nodes using DaemonSet, with hostNetwork and a node selector.
To make sure each istio-ingressgateway instance has a cert-sync co-located in the same machine, DaemonSet and same node selector should be used. The shared volume may be hostPath or other types as long as it suits your environment. As cert-sync is deployed separated from istio-ingressgateway, serviceaccount with appropriate permissions must be created.
The example installation yaml file can be found here.
How to use
Upload TLS secrets
Users should store their TLS certificates and private keys in Kubernetes Secrets.
These secrets must be kubernetes.io/tls type and have the following annotation exists.
certsync.istio.io/autosync: "true"
(or, some users may want to use a pair of custom annotation key and value, one of such
use cases is to work with cert-manager generated secrets. In such scenario, the
commandline flags --annotationKey and --annotationValue could let you achieve this.)
This annotation tells cert-sync to watch its updates and synchronize it. Here is an example:
apiVersion: v1
kind: Secret
metadata:
annotations:
certsync.istio.io/autosync: "true"
name: example-com
namespace: foo
type: kubernetes.io/tls
data:
tls.crt: <Base64EncodedCertificateData>
tls.key: <Base64EncodedPrivateKeyData>
Use TLS secrets in Istio Gateway
cert-sync will place your TLS data files in this kind of file path:
<certdir>/<namespace>/<name>.key
<certdir>/<namespace>/<name>.crt
For example, we've specified /etc/istio/ingressgateway-certs as our certificate
direcotry, and we've also created a secret example-com in namespace foo. So the TLS
data files would be:
/etc/istio/ingressgateway-certs/foo/example-com.key
/etc/istio/ingressgateway-certs/foo/example-com.crt
Then use them in Istio Gateway.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: example-com-gateway
namespace: foo
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
serverCertificate: /etc/istio/ingressgateway-certs/foo/example-com.key
privateKey: /etc/istio/ingressgateway-certs/foo/example-com.crt
hosts:
- "example.com"