Consul AWS Module

This repo contains a set of modules in the modules folder for deploying a Consul cluster on AWS using Terraform. Consul is a distributed, highly-available tool that you can use for service discovery and key/value storage. A Consul cluster typically includes a small number of server nodes, which are responsible for being part of the consensus quorum, and a larger number of client nodes, which you typically run alongside your apps:

How to use this Module

This repo has the following folder structure:

• modules: This folder contains several standalone, reusable, production-grade modules that you can use to deploy Consul.
• examples: This folder shows examples of different ways to combine the modules in the modules folder to deploy Consul.
• test: Automated tests for the modules and examples.
• root folder: The root folder is an example of how to use the consul-cluster module module to deploy a Consul cluster in AWS. The Terraform Registry requires the root of every repo to contain Terraform code, so we've put one of the examples there. This example is great for learning and experimenting, but for production use, please use the underlying modules in the modules folder directly.

To deploy Consul servers for production using this repo:

1. Create a Consul AMI using a Packer template that references the install-consul module. Here is an example Packer template.

   If you are just experimenting with this Module, you may find it more convenient to use one of our official public AMIs. Check out the aws_ami data source usage in main.tf for how to auto-discover this AMI.

   WARNING! Do NOT use these AMIs in your production setup. In production, you should build your own AMIs in your own AWS account.

2. Deploy that AMI across an Auto Scaling Group using the Terraform consul-cluster module and execute the run-consul script with the --server flag during boot on each Instance in the Auto Scaling Group to form the Consul cluster. Here is an example Terraform configuration to provision a Consul cluster.

To deploy Consul clients for production using this repo:

1. Use the install-consul module to install Consul alongside your application code.
2. Before booting your app, execute the run-consul script with --client flag.
3. Your app can now use the local Consul agent for service discovery and key/value storage.
4. Optionally, you can use the install-dnsmasq module for Ubuntu 16.04 and Amazon Linux 2 or setup-systemd-resolved for Ubuntu 18.04 to configure Consul as the DNS for a specific domain (e.g. .consul) so that URLs such as foo.service.consul resolve automatically to the IP address(es) for a service foo registered in Consul (all other domain names will be continue to resolve using the default resolver on the OS).

What's a Module?

A Module is a canonical, reusable, best-practices definition for how to run a single piece of infrastructure, such as a database or server cluster. Each Module is created using Terraform, and includes automated tests, examples, and documentation. It is maintained both by the open source community and companies that provide commercial support.

Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code that has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage the work of the Module community to pick up infrastructure improvements through a version number bump.

Who maintains this Module?

This Module is maintained by Gruntwork. If you're looking for help or commercial support, send an email to [email protected]. Gruntwork can help with:

• Setup, customization, and support for this Module.
• Modules for other types of infrastructure, such as VPCs, Docker clusters, databases, and continuous integration.
• Modules that meet compliance requirements, such as HIPAA.
• Consulting & Training on AWS, Terraform, and DevOps.

Code included in this Module:

• install-consul: This module installs Consul using a Packer template to create a Consul Amazon Machine Image (AMI).

• consul-cluster: The module includes Terraform code to deploy a Consul AMI across an Auto Scaling Group.

• run-consul: This module includes the scripts to configure and run Consul. It is used by the above Packer module at build-time to set configurations, and by the Terraform module at runtime with User Data to create the cluster.

• install-dnsmasq module: Install Dnsmasq for Ubuntu 16.04 and Amazon Linux 2 and configure it to forward requests for a specific domain to Consul. This allows you to use Consul as a DNS server for URLs such as foo.service.consul.

• setup-systemd-resolved module: Setup systemd-resolved for ubuntu 18.04 and configure it to forward requests for a specific domain to Consul. This allows you to use Consul as a DNS server for URLs such as foo.service.consul.

• consul-iam-policies: Defines the IAM policies necessary for a Consul cluster.

• consul-security-group-rules: Defines the security group rules used by a Consul cluster to control the traffic that is allowed to go in and out of the cluster.

• consul-client-security-group-rules: Defines the security group rules used by a Consul agent to control the traffic that is allowed to go in and out.

How do I contribute to this Module?

Contributions are very welcome! Check out the Contribution Guidelines for instructions.

How is this Module versioned?

This Module follows the principles of Semantic Versioning. You can find each new release, along with the changelog, in the Releases Page.

During initial development, the major version will be 0 (e.g., 0.x.y), which indicates the code does not yet have a stable API. Once we hit 1.0.0, we will make every effort to maintain a backwards compatible API and use the MAJOR, MINOR, and PATCH versions on each release to indicate any incompatibilities.

License

This code is released under the Apache 2.0 License. Please see LICENSE and NOTICE for more details.

Copyright © 2017 Gruntwork, Inc.