kubernetes-cookbook

Google Kubernetes installer for Ubuntu

Supported Platforms

Ubuntu

Attributes

default

Key	Type	Description	Default
`['kubernetes']['container_runtime']`	String	type of engine	`docker`
`['kubernetes']['roles']['master']`	String	role name for master servers	`kubernetes_master`
`['kubernetes']['roles']['node']`	String	role name for minions	`kubernetes_node`
`['kubernetes']['install_via']`	String	type of installation	`systemd`
`['kubernetes']['databag']`	String	default chef data_bag	`kubernetes`
`['kubernetes']['version']`	String	kubernetes version	`v1.20.4`
`['kubernetes']['keep_versions']`	Int		`3`
`['kubernetes']['image']`	String	hyperkube image name	`gcr.io/google_containers/hyperkube`
`['kubernetes']['interface']`	String	default interface	`eth1`
`['kubernetes']['enable_firewall']`	Boolean	Enable firewall	`true`
`['kubernetes']['register_as']`	String		`ip`
`['kubernetes']['proxy_mode']`	String	Which proxy mode to use: iptables or ipvs.	`iptables`
`['kubernetes']['use_sdn']`	Boolean	Use sdn	`true`
`['kubernetes']['sdn']`	String	Type of sdn	`weave`
`['kubernetes']['master']`	String	k8s master address	`127.0.0.1`
`['kubernetes']['cluster_name']`	String	cluster name	`kubernetes`
`['kubernetes']['cluster_dns']`	Array	cluster dns	`10.222.222.222`
`['kubernetes']['cluster_domain']`	String	cluster dns name	`kubernetes.local`
`['kubernetes']['cluster_cidr']`	String	cidr	`192.168.0.0/16`
`['kubernetes']['node_cidr_mask_size']`	Int	cidr mask size	`24`
`['kubernetes']['use_cluster_dns_systemwide']`	Boolean	dns systemwide	`false`
`['kubernetes']['ssl']['keypairs']`	Array	ssl keypairs	`['apiserver', 'ca']`
`['kubernetes']['ssl']['ca']['public_key']`	String	ca public_key path	`/etc/kubernetes/ssl/ca.pem`
`['kubernetes']['ssl']['ca']['private_key']`	String	ca private_key path	`/etc/kubernetes/ssl/ca-key.pem`
`['kubernetes']['ssl']['apiserver']['public_key']`	String	apiserver public_key path	`/etc/kubernetes/ssl/apiserver.pem`
`['kubernetes']['ssl']['apiserver']['private_key']`	String	apiserver private_key path	`/etc/kubernetes/ssl/apiserver-key.pem`
`['kubernetes']['kubeconfig']`	String	kubeconfig path	`/etc/kubernetes/kubeconfig.yaml`
`['kubernetes']['tls_cert_file']`	String	tls_cert_file path	`/etc/kubernetes/ssl/apiserver.pem`
`['kubernetes']['tls_private_key_file']`	String	tls private key file	`/etc/kubernetes/ssl/apiserver-key.pem`
`['kubernetes']['client_ca_file']`	String	client_ca_file path	`/etc/kubernetes/ssl/ca.pem`
`['kubernetes']['requestheader_client_ca_file']`	String	Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers	`/etc/kubernetes/ssl/ca.pem`
`['kubernetes']['cluster_signing_cert_file']`	String	cluster_signing_cert_file path	`/etc/kubernetes/ssl/ca.pem`
`['kubernetes']['cluster_signing_key_file']`	String		`/etc/kubernetes/ssl/ca-key.pem`
`['kubernetes']['token_auth']`	Boolean	token auth	`false`
`['kubernetes']['token_auth_file']`	String	tokens file	`/etc/kubernetes/known_tokens.csv`
`['kubernetes']['docker']`	String	path to docker socket	`unix:///var/run/docker.sock`
`['kubernetes'][cgroupdriver']`	String	Driver that the kubelet uses to manipulate cgroups on the host.	`systemd`
`['kubernetes']['feature_gates']`	Hash	feature gates	`'APIServerIdentity' => true, 'CronJobControllerV2' => true, 'CSIStorageCapacity' => true, 'CustomCPUCFSQuotaPeriod' => true, EphemeralContainers => true, 'GenericEphemeralVolume' => true, 'GracefulNodeShutdown' => true, 'ServiceTopology' => true, 'TTLAfterFinished' => true`
`['kubernetes']['audit']['enabled']`	Boolean	enable audit	`true`
`['kubernetes']['audit']['policy_file']`	String	Path to the file that defines the audit policy configuration	`/etc/kubernetes/audit-policy.yaml`
`['kubernetes']['audit']['log_path']`	String	If set, all requests coming to the apiserver will be logged to this file	`/var/log/kubernetes/audit.log`
`['kubernetes']['audit']['log_format']`	String	Format of saved audits. "legacy" indicates 1-line text format for each event. "json" indicates structured json format	`json`
`['kubernetes']['audit']['log_mode']`	String	Strategy for sending audit events. Blocking indicates sending events should block server responses. Batch causes the backend to buffer and write events asynchronously	`blocking`
`['kubernetes']['audit']['log_maxbackup']`	Int	The maximum number of old audit log files to retain	`3`
`['kubernetes']['audit']['log_maxsize']`	Int	The maximum size in megabytes of the audit log file before it gets rotated	`10`
`['kubernetes']['audit_webhook']['enabled']`	Boolean	enable [audit webhook backend](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/#webhook-backend)	`false`
`['kubernetes']['audit_webhook']['config_file']`	String	Path to a kubeconfig formatted file that defines the audit webhook configuration.	`/etc/kubernetes/audit-webhook.yaml`
`['kubernetes']['audit_webhook']['initial_backoff']`	String	The amount of time to wait before retrying the first failed request.	`10s`
`['kubernetes']['audit_webhook']['version']`	String	API group and version used for serializing audit events written to webhook.	`audit.k8s.io/v1`
`['kubernetes']['audit_webhook']['mode']`	String	Strategy for sending audit events. Blocking indicates sending events should block server responses. Batch causes the backend to buffer and write events asynchronously. Known modes are batch,blocking,blocking-strict.	`batch`
`['kubernetes']['audit_webhook_config']['server']`	String	Audit server URL.	`''`
`['kubernetes']['packages']['storage_url']`	String	packages storage	`https://storage.googleapis.com/kubernetes-release/release/#{node['kubernetes']['version']}/bin/linux/amd64/`
`['kubernetes']['checksums']['apiserver']`	String	checksum	`1852bfe86cfa96959ece2db5c70847c4e6b993caf0799ecc0d11c788ed366a56`
`['kubernetes']['checksums']['controller-manager']`	String	checksum	`114e7d1b6ff44bab03ecc84959b76455372445b703661863a9f222bf710e35f0`
`['kubernetes']['checksums']['proxy']`	String	checksum	`7670939861baeeca598bdfcbebc8f7e48f1c6fa73983c4d3f549e894757d2d2f`
`['kubernetes']['checksums']['scheduler']`	String	checksum	`ad44f1c248ce0b6c35b7c7c66567d6e8085f785a130a6a26fd238411088fab5b`
`['kubernetes']['checksums']['kubectl']`	String	checksum	`1bb4d3793fb0f9e1cfee86599e0f43ae5f15578a01b61011fe7c9488e114a00b`
`['kubernetes']['checksums']['kubelet']`	String	checksum	`688d1167c5a8b37bb5f10e330ba43c15092f1d35dcc25929e84484c41a20319d`
`['kubernetes']['addon_manager']['version']`	String	addon_manager version	`v9.1.3`
`['kubernetes']['multimaster']['access_via']`	String	type of access	`haproxy`
`['kubernetes']['multimaster']['haproxy_url']`	String	haproxy url	`127.0.0.1`
`['kubernetes']['multimaster']['haproxy_port']`	Int	haproxy port	`6443`
`['kubernetes']['multimaster']['dns_name']`	String	multimaster dns_name
`['kubernetes']['cni']['plugins']`	Hash	cni plugins	`See attributes/default.rb for this big hash`
`['kubernetes']['cni']['plugins_version']`	String	cni plugins version	`0.9.1`
`['kubernetes']['encryption']`	String	encryption	`aescbc`
`['kubernetes']['node']['packages']`	Hash	default node packages	`See attributes/default.rb for more information`

kubelet

Key	Type	Description	Default
`['kubernetes']['kubelet']['daemon_flags']['config']`	String	kubelet init config	`/etc/kubernetes/kubeletconfig.yaml`
`['kubernetes']['kubelet']['daemon_flags']['bootstrap_kubeconfig']`	String	bootstrap config	`/etc/kubernetes/kubeconfig-bootstrap.yaml`
`['kubernetes']['kubelet']['daemon_flags']['cert_dir']`	String	cert dir	`/etc/kubernetes/ssl`
`['kubernetes']['kubelet']['daemon_flags']['kubeconfig']`	String	kubeconfig	`/etc/kubernetes/kubelet.yaml`
`['kubernetes']['kubelet']['daemon_flags']['allow_privileged']`	Boolean	allow run privileged pods	`true`
`['kubernetes']['kubelet']['daemon_flags']['v']`	Integer	log veribosity	`2`
`['kubernetes']['kubelet']['daemon_flags']['network_plugin']`	String	network plugin	`cni`
`['kubernetes']['kubelet']['daemon_flags']['register_node']`	Boolean	register node	`true`
`['kubernetes']['kubelet']['daemon_flags']['cni_cache_dir']`	String	The full path of the directory in which CNI should store cache files.	`/var/lib/cni/cache`
`['kubernetes']['kubelet']['config']['staticPodPath']`	String	pod manifests	`/etc/kubernetes/manifests`
`['kubernetes']['kubelet']['config']['authentication']['x509']['clientCAFile']`	String	client ca file	`/etc/kubernetes/ssl/ca.pem`
`['kubernetes']['kubelet']['config']['authentication']['webhook']['enabled']`	Boolean	enable webhook	`true`
`['kubernetes']['kubelet']['config']['authentication']['webhook']['cacheTTL']`	String	webhook cacheTTL	`2m0s`
`['kubernetes']['kubelet']['config']['authentication']['anonymous']['enabled']`	Boolean	anonymous auth	`fase`
`['kubernetes']['kubelet']['config']['authorization']['mode']`	String	auth mode	`Webhook`
`['kubernetes']['kubelet']['config']['clusterDNS']`	Array	array of cluster dns ips	`node['kubernetes']['cluster_dns']`
`['kubernetes']['kubelet']['config']['featureGates']`	Hash	hash of feature gates	`node['kubernetes']['feature_gates']`
`['kubernetes']['kubelet']['config']['NodeStatusUpdateFrequency']`	String	NodeStatusUpdateFrequency	`4s`
`['kubernetes']['kubelet']['config']['clusterDomain']`	String	cluster domain	`node['kubernetes']['cluster_domain']`
`['kubernetes']['kubelet']['config']['imageGCLowThresholdPercent']`	Integer	imageGCLowThresholdPercent	`70`
`['kubernetes']['kubelet']['config']['imageGCHighThresholdPercent']`	Integer	imageGCHighThresholdPercent	`80`
`['kubernetes']['kubelet']['config']['failSwapOn']`	Boolean	failSwapOn	`false`
`['kubernetes']['kubelet']['config']['ReadOnlyPort']`	Integer	ReadOnlyPort	`10255`
`['kubernetes']['kubelet']['config']['serverTLSBootstrap]`	Boolean	Server certificate bootstrap	`true`
`['kubernetes']['kubelet']['config']['rotateCertificates']`	Boolean	Auto rotate the kubelet client certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches	`true`
`['kubernetes']['kubelet']['config']['topologyManagerScope']`	String	Scope to which topology hints applied. Topology Manager collects hints from Hint Providers and applies them to defined scope to ensure the pod admission. Possible values: 'container', 'pod'.	`container`

crio

Key	Type	Description	Default
`['kubernetes']['crio']['version']`	String	CRIO binary version	`1.15.2`
`['kubernetes']['crio']['endpoint']`	String	Path to UNIX socket for crio daemon to listen	`/var/run/crio/crio.sock`
`['kubernetes']['crio']['config']['runtime']`	String	OCI compatible runtime used for trusted container workloads.	`/usr/local/bin/runc`
`['kubernetes']['crio']['config']['untrusted_runtime']`	String	OCI compatible runtime used for untrusted container workloads.	`/usr/local/bin/runsc`
`['kubernetes']['crio']['config']['conmon']`	String	Path to conmon binary	`/usr/local/bin/conmon`
`['kubernetes']['crio']['config']['storage_driver']`	String	Storage driver	`aufs`
`['kubernetes']['crio']['config']['stream_port']`	Fixnum	Port on which the stream server will listen	`10010`
`['kubernetes']['crio']['config']['runroot']`	String	Path to the "run directory". CRIO stores all of its state in this directory.	`/var/run/containers/storage`
`['kubernetes']['crio']['config']['root']`	String	Path to the "root directory". CRIO stores all of its data, including container images, in this directory.	`/var/lib/containers/storage`
`['kubernetes']['crio']['config']['log_level']`	String	Log messages above specified level: debug, info, warn, error, fatal or panic	`info`
`['kubernetes']['crio']['daemon_flags']['log_format']`	String	Format used by logs	`text`
`['kubernetes']['crio']['daemon_flags']['profile']`	Boolean	Enable pprof remote profiler on localhost:6060	`false`
`['kubernetes']['crio']['daemon_flags']['enable_metrics']`	Boolean	Enable prometheus-compatible metrics endpoint for the server	`true`
`['kubernetes']['crio']['daemon_flags']['metrics_port']`	Fixnum	Port for the metrics endpoint	`9090`

addons

Key	Type	Description	Default
`['kubernetes']['addons']['dns']['controller']`	String	dns controller	`coredns`
`['kubernetes']['addons']['dns']['antiaffinity_type']`	String	antiaffinity type	`preferredDuringSchedulingIgnoredDuringExecution`
`['kubernetes']['addons']['dns']['antiaffinity_weight']`	Int	antiaffinity weight	`100`
`['kubernetes']['addons']['kubedns']['dns_forward_max']`	Int	dns forward max	`150`
`['kubernetes']['addons']['kubedns']['version']`	String	kubedns version	`1.14.10`
`['kubernetes']['addons']['kubedns']['limits']['cpu']`	String	kubedns cpu limits	`100m`
`['kubernetes']['addons']['kubedns']['limits']['memory']`	String	kubedns memory limits	`170Mi`
`['kubernetes']['addons']['kubedns']['requests']['cpu']`	String	kubedns requests cpu	`100m`
`['kubernetes']['addons']['kubedns']['requests']['memory']`	String	kubedns requests memory	`70Mi`
`['kubernetes']['addons']['coredns']['version']`	String	coredns version	`'1.8.0'`
`['kubernetes']['addons']['coredns']['limits']['cpu']`	String	coredns cpu limits	`100m`
`['kubernetes']['addons']['coredns']['limits']['memory']`	String	coredns memory limits	`256Mi`
`['kubernetes']['addons']['coredns']['requests']['cpu']`	String	coredns cpu requests	`100m`
`['kubernetes']['addons']['coredns']['requests']['memory']`	String	coredns memory requests	`256Mi`
`['kubernetes']['addons']['coredns']['log']`	Boolean	enable coredns log	`false`
`['kubernetes']['addons']['coredns']['hosts']`	Array	Enable CoreDNS `hosts` pluging and add array elements as inline host entries	`[]`
`['kubernetes']['addons']['npd']['enabled']`	Boolean	enable node problem detector addon	`false`
`['kubernetes']['addons']['npd']['version']`	String	node problem detector version	`0.8.7`
`['kubernetes']['addons']['npd']['address']`	String	address to bind the node problem detector server	`0.0.0.0`
`['kubernetes']['addons']['npd']['port']`	Fixnum	port to bind the node problem detector server	`20256`
`['kubernetes']['addons']['npd']['log_level']`	Fixnum	log level for V logs	`0`
`['kubernetes']['addons']['npd']['system_log_monitors']`	Array	List of paths to system log monitor config files	`['/config/kernel-monitor.json', '/config/kernel-monitor-filelog.json', '/config/docker-monitor.json', '/config/docker-monitor-filelog.json']`

authorization

Key	Type	Description	Default
`['kubernetes']['authorization']['admin_groups']`	Array	admin groups	`['admins']`
`['kubernetes']['authorization']['mode']`	String	authorization mode	`None,RBAC`
`['kubernetes']['authorization']['policies']`	Array	auth policies	`See attributes/authorization.rb`

docker

Key	Type	Description	Default
`['docker']['built-in']`	Boolean	enable built-in docker installation	`true`
`['docker']['version']`	String	default daemon version	`19.03.12~3-0`
`['docker']['deb_version']`	String	Debian package version number format	`5`
`['docker']['settings']['storage-driver']`	String	defalt storage driver	`aufs`
`['docker']['settings']['live-restore']`	Boolean	live restore	`true`
`['docker']['settings']['iptables']`	Boolean	iptables	`false`
`['docker']['settings']['ip-masq']`	Boolean	ip masq	`false`

etcd

Key	Type	Description	Default
`['etcd']['version']`	String	version	`v3.4.14`
`['etcd']['image']`	String	image	`quay.io/coreos/etcd`
`['etcd']['trusted_ca_file']`	String	trusted_ca_file	`/etc/kubernetes/ssl/ca.pem`
`['etcd']['client_cert_auth']`	String	client_cert_auth	`true`
`['etcd']['key_file']`	String	key file	`/etc/kubernetes/ssl/apiserver-key.pem`
`['etcd']['cert_file']`	String	cert file	`/etc/kubernetes/ssl/apiserver.pem`
`['etcd']['peer_trusted_ca_file']`	String	trusted ca	`/etc/kubernetes/ssl/ca.pem`
`['etcd']['peer_client_cert_auth']`	String	cert auth	`true`
`['etcd']['peer_key_file']`	String	key file	`/etc/kubernetes/ssl/apiserver-key.pem`
`['etcd']['peer_cert_file']`	String	cert file	`/etc/kubernetes/ssl/apiserver.pem`
`['etcd']['server_port']`	Int	server port	`2380`
`['etcd']['client_port']`	Int	client port	`2379`
`['etcd']['interface']`	String	eth1	`default etcd interface`
`['etcd']['data_dir']`	String	data dir	`/var/lib/etcd`
`['etcd']['wal_dir']`	String	wal_dir	`/var/lib/etcd/member/wal`
`['etcd']['proto']`	String	proto	`http`
`['etcd']['binary']`	String	binary	`/usr/local/bin/etcd`
`['etcd']['user']`	String	etcd user	`etcd`
`['etcd']['group']`	String	etcd group	`etcd`
`['etcd']['initial_cluster_token']`	String	initial cluster token	`etcd-cluster`
`['etcd']['initial_cluster_state']`	String	initial cluster state	`new`
`['etcd']['role']`	String	role name	`etcd`
`['etcd']['default_service_name']`	Boolean	Set default service name like etcd.service	`true`

firewall

Key	Type	Description	Default
`['firewall']['allow_ssh']`	Boolean	allow_ssh	`true`
`['firewall']['allow_loopback']`	Boolean	allow loopback	`true`
`['firewall']['allow_icmp']`	Boolean	allow icmp	`true`
`['firewall']['ubuntu_iptables']`	Boolean	ubuntu iptables	`false`
`['firewall']['allow_established']`	Boolean	allow established	`true`
`['firewall']['ipv6_enabled']`	Boolean	ipv6_enabled	`true`

weave

Key	Type	Description	Default
`['kubernetes']['weave']['version']`	String	version	`2.8.1`
`['kubernetes']['weave']['interface']`	String	interfave	`weave`
`['kubernetes']['weave']['use_scope']`	Boolean	use_scope	`true`
`['kubernetes']['weave']['use_portmap']`	Boolean	use_portmap	`true`
`['kubernetes']['weave'][no_masq_local]`	Boolean	preserve the client source IP address when accessing Services	`true`
`['kubernetes']['weave']['update_strategy']['type']`	String	update_strategy	`RollingUpdate`
`['kubernetes']['weave']['npc_enabled']`	Boolean	toggle weave-npc container	`true`
`['kubernetes']['weavescope']['version']`	String	weavespoce version	`0.17.1`
`['kubernetes']['weavescope']['port']`	String	weavescope port	`4040`

k8s_apiserver

Key	Type	Description	Default
`['kubernetes']['api']['bind_address']`	String	bind_address	`0.0.0.0`
`['kubernetes']['api']['secure_port']`	Integer	secure_port	`8443`
`['kubernetes']['api']['service_cluster_ip_range']`	String		`10.222.0.0/16`
`['kubernetes']['api']['storage_backend']`	String	storage_backend	`etcd3`
`['kubernetes']['api']['storage_media_type']`	String	storage_media_type	`application/vnd.kubernetes.protobuf`
`['kubernetes']['api']['kubelet_https']`	Boolean	kubelet_https	`true`
`['kubernetes']['api']['kubelet_certificate_authority']`	String	kubelet_certificate_authority	`/etc/kubernetes/ssl/ca.pem`
`['kubernetes']['api'][encryption_provider_config']`	String	The file containing configuration for encryption providers to be used for storing secrets in etcd	`/etc/kubernetes/encryption-config.yaml`
`['kubernetes']['api']['kubelet_client_certificate']`	String	kubelet_client_certificate	`/etc/kubernetes/ssl/apiserver.pem`
`['kubernetes']['api']['kubelet_client_key']`	String	kubelet_client_key	`/etc/kubernetes/ssl/apiserver-key.pem`
`['kubernetes']['api']['kubelet_preferred_address_types']`	String	List of the preferred NodeAddressTypes to use for kubelet connections.	`InternalIP,ExternalIP,InternalDNS,ExternalDNS,Hostname`
`['kubernetes']['api']['endpoint_reconciler_type']`	String	endpoint_reconciler_type	`lease`
`['kubernetes']['api']['etcd_certfile']`	String	etcd_certfile	`node['etcd']['cert_file']`
`['kubernetes']['api']['etcd_keyfile']`	String	etcd_keyfile	`node['etcd']['key_file']`
`['kubernetes']['api']['etcd_cafile']`	String	etcd_cafile	`node['etcd']['trusted_ca_file']`
`['kubernetes']['api']['etcd_healthcheck_timeout']`	Duration	The timeout to use when checking etcd health.	`2s`
`['kubernetes']['api']['allow_privileged']`	Boolean	allow privileged containers	`true`
`['kubernetes']['api']['authorization_mode']`	String	authorization_mode	`node['kubernetes']['authorization']['mode']`
`['kubernetes']['api']['enable_bootstrap_token_auth']`		default nit, because option without params	`nil`
`['kubernetes']['api']['tls_cert_file']`	String	tls_cert_file	`node['kubernetes']['tls_cert_file']`
`['kubernetes']['api']['tls_private_key_file']`	String	tls_private_key_file	`node['kubernetes']['tls_private_key_file']`
`['kubernetes']['api']['client_ca_file']`	String	client_ca_file	`node['kubernetes']['client_ca_file']`
`['kubernetes']['api']['service_account_key_file']`	String	service_account_key_file	`node['kubernetes']['service_account_key_file']`
`['kubernetes']['api']['service_account_signing_key_file']`	String	Path to the file that contains the current private key of the service account token issuer. The issuer will sign issued ID tokens with this private key.	`node['kubernetes']['service_account_key_file']`
`['kubernetes']['api']['api_audiences']`	String	Identifiers of the API. The service account token authenticator will validate that tokens used against the API are bound to at least one of these audiences. If the --service-account-issuer flag is configured and this flag is not, this field defaults to a single element list containing the issuer URL.	`api`
`['kubernetes']['api']['service_account_extend_token_expiration']`	Boolean	Turns on projected service account expiration extension during token generation, which helps safe transition from legacy token to bound service account token feature. If this flag is enabled, admission injected tokens would be extended up to 1 year to prevent unexpected failure during transition, ignoring value of service-account-max-token-expiration.	`true`
`['kubernetes']['api'][service_account_issuer]`	String	Identifier of the service account token issuer. The issuer will assert this identifier in "iss" claim of issued tokens. This value is a string or URI. If this option is not a valid URI per the OpenID Discovery 1.0 spec, the ServiceAccountIssuerDiscovery feature will remain disabled, even if the feature gate is set to true. It is highly recommended that this value comply with the OpenID spec: https://openid.net/specs/openid-connect-discovery-1_0.html. In practice, this means that service-account-issuer must be an https URL. It is also highly recommended that this URL be capable of serving OpenID discovery documents at {service-account-issuer}/.well-known/openid-configuration.	`kubernetes/serviceaccount`
`['kubernetes']['api']['log_dir']`	String	log_dir	`/var/log/kubernetes`
`['kubernetes']['api']['audit_log_compress']`	Boolean	If set, the rotated log files will be compressed using gzip.	`true`
`['kubernetes']['api']['feature_gates']`	String	feature_gates	`node['kubernetes']['feature_gates']`
`['kubernetes']['api']['enable_admission_plugins']`	String	plugins separated by comma	`DefaultStorageClass, DefaultTolerationSeconds, LimitRanger, MutatingAdmissionWebhook, NamespaceLifecycle, NodeRestriction, PersistentVolumeClaimResize, Priority, ResourceQuota, ServiceAccount, TaintNodesByCondition, ValidatingAdmissionWebhook`

k8s_controller

Key	Type	Description	Default
`['kubernetes']['controller_manager']['secure_port']`	Fixnum	The port on which to serve HTTPS with authentication and authorization.If 0, don't serve HTTPS at all.	`10257`
`['kubernetes']['controller_manager']['leader_elect']`	Boolean	leader_elect	`true`
`['kubernetes']['controller_manager']['cluster_cidr']`	String	cluster cird	`node['kubernetes']['cluster_cidr']`
`['kubernetes']['controller_manager']['cluster_name']`	String	cluster name	`node['kubernetes']['cluster_name']`
`['kubernetes']['controller_manager']['service_account_private_key_file']`	String	service_account_key_file	`node['kubernetes']['service_account_key_file']`
`['kubernetes']['controller_manager']['cluster_signing_cert_file']`	String	cluster_signing_cert_file	`node['kubernetes']['cluster_signing_cert_file']`
`['kubernetes']['controller_manager']['cluster_signing_key_file']`	String	cluster_signing_key_file	`node['kubernetes']['cluster_signing_key_file']`
`['kubernetes']['controller_manager']['root_ca_file']`	String	root_ca_file	`node['kubernetes']['client_ca_file']`
`['kubernetes']['controller_manager']['master']`	String	master	`http://127.0.0.1:#{node['kubernetes']['api']['insecure_port']}`
`['kubernetes']['controller_manager']['feature_gates']`	String	feature_gates	`node['kubernetes']['feature_gates']`
`['kubernetes']['controller_manager']['node_monitor_period']`	String	node_monitor_period	`2s`
`['kubernetes']['controller_manager']['node_monitor_grace_period']`	String	node_monitor_grace_period	`16s`
`['kubernetes']['controller_manager']['pod_eviction_timeout']`	String	pod_eviction_timeout	`30s`
`['kubernetes']['controller_manager']['horizontal_pod_autoscaler_sync_period']`	String	The period for syncing the number of pods in horizontal pod autoscaler	`30s`
`['kubernetes']['controller_manager']['horizontal_pod_autoscaler_tolerance']`	Float	The minimum change (from 1.0) in the desired-to-actual metrics ratio for the horizontal pod autoscaler to consider scaling	`0.1`

k8s_proxy

Key	Type	Description	Default
`['kubernetes']['proxy']['kubeconfig']`	String	path to config	`/etc/kubernetes/system:kube-proxy_config.yaml`
`['kubernetes']['proxy']['feature_gates']`	Hash	hash of feature gates	`node['kubernetes']['feature_gates']`
`['kubernetes']['proxy']['global']['metrics_port']`	Fixnum	The port to bind the metrics server. Use 0 to disable	`10249`
`['kubernetes']['proxy']['global']['detect_local_mode']`	String	Mode to use to detect local traffic	`10249`

scheduler

Key	Type	Description	Default
`['kubernetes']['scheduler']['secure_port']`	Fixnum	The port on which to serve HTTPS with authentication and authorization.If 0, don't serve HTTPS at all.	`10259`
`['kubernetes']['scheduler']['leader_elect']`	Boolean	leader_elect	`true`
`['kubernetes']['scheduler']['feature_gates']`	String	feature_gates	`node['kubernetes']['feature_gates']`
`['kubernetes']['scheduler']['master']`	String	master	`http://127.0.0.1:#{node['kubernetes']['api']['insecure_port']}`

Usage

Certificates

Create ssl certificates for k8s.

cd ./lib/tasks/ssl
cp config_example.yaml config.yaml
bundler
rake ca:generate
rake apiserver:generate

All keys will be generated at ./ssl folder.

After cluster installation weave pods can contain error about:

FATA: 2018/03/15 19:51:39.168435 [kube-peers] Could not get peers: Get https://192.168.128.1:443/api/v1/nodes:
x509: certificate is valid for 127.0.0.1, 10.222.0.1, not 192.168.128.1

Add 192.168.128.1 to ssl/tasks/config.yaml and recreate and upload new apiserver-key.pem and apiserver.pem

Prepare your data_bag

You need to create kubernetes data_bag in chef server.

Then add next files:

apiserver_ssl
ca_ssl
encryption_keys
users

Structure:

apiserver_ssl

{
  "id": "apiserver_ssl",
  "private_key": "PUT apiserver-key.pem HERE",
  "public_key": "PUT apiserver.pem HERE"
}

ca_ssl

{
  "id": "ca_ssl",
  "private_key": "PUT ca-key.pem HERE",
  "public_key": "PUT ca.pem HERE"
}

encryption_keys

{
  "id": "encryption_keys",
  "aescbc": [
    {
      "name": "key1",
      "secret": "baiBu8ais4bu3uRohqu6och5yai4wai8"
    }
  ]
}

users

{
  "id": "users",
  "users": [
    {
      "name": "exampleuser",
      "token": "aenup6io4ciath7yaxu0vie6guaSie6goi3ahri0eemui3Ieghu4tuhaa3kisohv",
      "uid": "10001",
      "groups": [
        "admins"
      ]
    },
    {
      "name": "kubelet-bootstrap",
      "token": "nieJi3ooGh1ohy8sheowee7ohghei3Xaebeeve8Ooch3omex4cho2xuexuuzeeva",
      "uid": "10100",
      "groups": [
        "system:bootstrappers"
      ]
    },
    {
      "name": "kubelet",
      "token": "ieT5Oogecah6geengaeyai3ohNg6Fiecha6iemaifithah2ui3oChaixeThi5Shi",
      "uid": "10101",
      "groups": [
        "kubelet",
        "system:nodes"
      ]
    },
    {
      "name": "system:kube-proxy",
      "token": "ka2thaijaek0oophoothahbahyaiphe6ahteegieyae8il9XohveeJahn3Aizohy",
      "uid": "10102",
      "groups": [
        "system:node-proxier"
      ]
    },
    {
      "name": "system:kube-scheduler",
      "token": "MoN7ohz2Aebeep2eeneGhie5Hikop9iroSahyezohchuthi8Iu1iVaetae5xaj3W",
      "uid": "10103",
      "groups": [
        "system:kube-scheduler"
      ]
    },
    {
      "name": "system:kube-controller-manager",
      "token": "waiKahbeegh3ooco0oa2oodi7mei5Sahboomahdaedu2ieha2queen0Aiwera7ui",
      "uid": "10104",
      "groups": [
        "system:kube-controller-manager"
      ]
    },
    {
      "name": "evlms:addon-manager",
      "token": "heiyais8Dolee8ma5toh8meetee8Ooyaecixoobai3quoo0phu2iife5ahkoo0ei",
      "uid": "10105",
      "groups": [
        "system:masters"
      ]
    }
  ]
}

kubernetes::etcd

Run kubernetes::etcd recipe or role on your nodes. Run it twice for normal chef search.

Or you can add role without kubernetes::etcd for first servers registration in chef.

name 'etcd'
description 'Etcd cluster node'
override_attributes(
  'etcd' => {
    initial_cluster_state: 'new',
    initial_cluster_token: 'etcd-test-cluster',
    wal_dir: '/var/lib/etcd/member/wal'
  }
)
run_list 'recipe[kubernetes::etcd]'

kubernetes::master

Include kubernetes::master in your master node's run_list:

{
  "run_list": [
    "recipe[kubernetes::master]"
  ]
}

Or role:

name 'kubernetes_master'
description 'Kubernetes master node'
run_list 'recipe[kubernetes::master]'
override_attributes(
  docker: {
    build_in_enable: false
  },
  kubernetes: {
    cluster_name: 'evilms',
    cluster_dns: ['192.168.222.222'],
    cluster_cidr: '192.168.0.0/17',
    api: {
      'service_cluster_ip_range' => '192.168.128.0/17'
    },
    dns: { deploy_via: 'deployment' },
    token_auth: true,
    addons: {
      kubedns: {
        node_selector: 'evl.ms/role=system'
      },
      coredns: {
        node_selector: 'evl.ms/role=system',
        requests: {
          cpu: '200m'
        },
        limits: {
          cpu: '200m'
        }
      },
      dns: {
        controller: 'coredns',
        antiaffinity_type: 'requiredDuringSchedulingIgnoredDuringExecution'
      }
    }
  }
)

If you use master nodes without minions on them add kubernetes::packages to you run_list.

And add master node to role kube_master. This is obligatory in multinode configuration - minions uses role to find master.

kubernetes::default

Include kubernetes::default in your minion node's run_list:

{
  "run_list": [
    "recipe[kubernetes]"
  ]
}

Or role:

name 'kubernetes_node'
description 'kubernetes node'
#run_list 'recipe[kubernetes]'
run_list 'recipe[kubernetes]'
override_attributes(
  kubernetes: {
    cluster_name: 'evilms',
    cluster_dns: ['192.168.222.222'],
    token_auth: true,
    api:   { 'service_cluster_ip_range' => '192.168.128.0/17' },
    weave: {
      network: '192.168.0.0/17',
      use_scope: false
    }
  }
)

If you use custom docker installation you can disable built-in docker installation

docker: {
  'built-in' => false
}

Also you can use CRIO as a container runtime interface:

kubernetes: {
  'container_runtime': 'crio'
}

Don't forget to run docker rm -f `docker ps -aq` after successful CRIO installation.

Dashboard

Starting from release 1.11.0 we are no more ships kubernetes-dashboard with cookbook. From now on we recommends to use helm and install kubernetes-dashboard from official chart.

License and Authors

License:: http://bregor.mit-license.org

Author:: Maxim Filatov ([email protected])

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

evilmartians / chef-kubernetes

Programming Languages

Labels

Projects that are alternatives of or similar to chef-kubernetes