Container Orchestration Benchmark
The purpose of the container orchestration benchmark (cnbm-co for short) is to provide a vendor-neutral, extendable benchmark for container orchestration systems. The current focus is on stateless workloads and we're implementing it for the following container orchestration systems (targets):
- DC/OS
- Kubernetes
If you want to contribute, simply fork this repo, add your implementation in pkg/ and send in a PR.
Contents:
Using
Launching
In general:
$ ./cnbm-co launch -h
Launches the CNBM container orchestration benchmark
Usage:
cnbm-co launch [flags]
Flags:
-h, --help help for launch
-p, --params string Comma separated key-value pair list of target-specific configuration parameters. For example: k1=v1,k2=v2
-r, --runtype string The benchmark run type. Allowed values: [scaling distribution apicalls servicediscovery recovery]
-t, --target string The target container orchestration system to benchmark. Allowed values: [dcos kubernetes]
Global Flags:
--config string config file (default is $HOME/.cnbm.yaml)
DC/OS
$ ./cnbm-co launch --runtype scaling --target dcos -p dcosurl=http://example.com,dcosacstoken=123
INFO[0000] Setting up DC/OS scaling benchmark
INFO[0000] Executing DC/OS scaling benchmark
INFO[0000] Deploying a new application
INFO[0000] RESULT:
Target: DC/OS
Output: {}
Elapsed time: 0s
Kubernetes
To benchmark a Kubernetes cluster, use --target kubernetes, for example, to launch the scaling run type, do:
$ ./cnbm-co launch --runtype scaling --target kubernetes --params kubeconfig=/Users/mhausenblas/.kube/config,ns=cnbm,numpods=3,cpu=50m,mem=300Mi
INFO[0000] Setting up Kubernetes scaling benchmark
INFO[0000] Executing Kubernetes scaling benchmark
INFO[0006] Tearing down Kubernetes scaling benchmark
INFO[0008] RESULT:
Target: Kubernetes
Output: {benchmark succeeded}
Elapsed time: 6.752147169s
Note the following params that are required, depending on the --runtype:
scalingkubeconfig… the Kubernetes config to use, for example/Users/mhausenblas/.kube/configns… the namespace to use, for examplecnbmnumpods… the number of pods to launch, for example,10cpu… the CPU seconds (at least40m) per podmem… the memory (at least300Mi) per pod
servicediscoverykubeconfig… the Kubernetes config to use, for example/Users/mhausenblas/.kube/configns… the namespace to use, for examplecnbm
Availability matrix
The following matrix shows the availability of benchmark run types per target:
| benchmark run type | DC/OS | Kubernetes |
|---|---|---|
scaling |
Y | Y |
distribution |
N | N |
distribution |
N | N |
apicalls |
N | N |
servicediscovery |
N | N |
Developing
Building
$ make
Building the CNBM CO CLI
go build -ldflags "-X github.com/cnbm/container-orchestration/cli/cmd.releaseVersion=0.1.0" -o ./cnbm-co cli/main.go
Vendoring
We are using Go dep for dependency management. If you don't have dep installed yet, do go get -u github.com/golang/dep/cmd/dep now and then:
$ dep ensure
Testing
For unit tests we use the go test command, for example:
$ go test -v -short -run Test* .