azure-terraform-cluster
Scripts to create a minimal mesos or nomad cluster on Azure using terraform.
Table of Contents
Overview
This creates 5
master and 10
agents in a mesos or nomad cluster.
You can change the number of masters with MASTER_COUNT
and the number of
agents with AGENT_COUNT
.
It also creates a "jumpbox" or "bastion host" since all the masters and agents are not publicly accessible.
If you want to ssh into the internal nodes you must first go through the bastion on node.
The username on the nodes is vmuser
.
The base image for all the virtual machines is CoreOS Container Linux.
The cloud-config.yml files defines the servers running on each of the hosts. The hosts are designed to be super minimal. This is done via the CoreOS Cloud Configuration.
All internal IPs are in the block 10.x.0.x.
These are multi-region so westus2
is 10.1.0.x and so on and so forth for the
other regions.
So the first 5 in the block are the masters: 10.x.0.5-9
. And the agents
follow after starting at 10.x.0.10
.
The IPs get populated programmatically in the terraform files. But they are
also generated and substituted into the cloud-config*
files via the Makefile.
Nomad
On each server there are helpers in the bashrc
so that you can run nomad
and consul
commands from the host. This just uses a container under the hood
and you can see the alias with type nomad
.
Both consul and nomad are setup with TLS which is generated by the Makefile when the cluster is created.
On the bastion server we run:
- Nomad Dashboard: This is opened on port 8080 by default so you will want to change that if you want your cluster to be secure. This is only done so it is easy to demo.
On the masters we run:
- Consul
- Nomad Server
On the agents we run:
- Nomad Agent
Mesos
On the bastion server we run:
- Mesos Marathon: This is opened on port 8080 by default so you will want to change that if you want your cluster to be secure. This is only done so it is easy to demo.
On the masters we run:
- Mesos Master
- Zookeeper
On the agents we run:
- Mesos Agent
Makefile
Using the You will need to set the following environment variables:
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET
AZURE_TENANT_ID
AZURE_SUBSCRIPTION_ID
See creating a service principal on how to get these values.
$ make help
mesos-apply Run terraform apply for mesos.
mesos-destroy Run terraform destroy for mesos.
nomad-apply Run terraform apply for nomad.
nomad-destroy Run terraform destroy for nomad.
shellcheck Run shellcheck on all scripts in the repository.
test Runs all the tests.
update-terraform Update terraform binary locally from the docker container.
update Run all update targets.
Spinning up a cluster
This is as simple as:
$ AZURE_CLIENT_ID=0000 AZURE_CLIENT_SECRET=0000 AZURE_TENANT_ID=0000 AZURE_SUBSCRIPTION_ID=0000 \
make az-apply
Azure credentials setup
You need a service principal in order to use the Makefile
.
Creating a service principal
$ az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/SUBSCRIPTION_ID"
The command will output the following:
{
"appId": "00000000-0000-0000-0000-000000000000",
"displayName": "azure-cli-2017-06-05-10-41-15",
"name": "http://azure-cli-2017-06-05-10-41-15",
"password": "0000-0000-0000-0000-000000000000",
"tenant": "00000000-0000-0000-0000-000000000000"
}
These values map to the Makefile
variables like so:
appId
is theAZURE_CLIENT_ID
defined abovepassword
is theAZURE_CLIENT_SECRET
defined abovetenant
is theAZURE_TENANT_ID
defined above
Reference docs:
terraform
docs on setting up authentication: here.