tf_aws_elasticsearch

Terraform module for deploying and managing Amazon Elasticsearch Service.

This module has two options for creating an Elasticsearch domain:

1. Create an Elasticsearch domain with a public endpoint. Access policy is then based on the intersection of the following two criteria

   • source IP address
   • client IAM role

   See this Stack Overflow post for further discussion of access policies for Elasticsearch.

2. Create an Elasticsearch domain and join it to a VPC. Access policy is then based on the intersection of the following two criteria:

   • security groups applied to Elasticsearch domain
   • client IAM role

If vpc_options option is set, Elasticsearch domain is created within a VPC. If not, Elasticsearch domain is created with a public endpoint

NOTE: You can either launch your domain within a VPC or use a public endpoint, but you can't do both. Considering this, adding or removing vpc_options will force DESTRUCTION of the old Elasticsearch domain and CREATION of a new one. More INFO - VPC support

Several options affect the resilience and scalability of your Elasticsearch domain. For a production deployment:

• set instance_count to an even number (default: 6) greater than or equal to the dedicated_master_threshold (default: 10)
• choose an instance_type that is not in the T2 family
• set es_zone_awareness to true.

This will result in a cluster with three dedicated master nodes, balanced across two availability zones.

For a production deployment it may also make sense to use EBS volumes rather that instance storage; to do so, set ebs_volume_size greater than 0 and optionally specify a value for ebs_volume_type (right now the only supported values are gp2 and magnetic).

Terraform versions

Terraform 0.12. Pin module version to ~> v1.0. Submit pull-requests to master branch.

Terraform 0.11. Pin module version to ~> v0.0. Submit pull-requests to terraform011 branch.

Usage

Create Elasticsearch domain with public endpoint

module "es" {
  source  = "git::https://github.com/terraform-community-modules/tf_aws_elasticsearch.git?ref=v1.1.0"

  domain_name                    = "my-elasticsearch-domain"
  management_public_ip_addresses = ["34.203.XXX.YYY"]
  instance_count                 = 16
  instance_type                  = "m4.2xlarge.elasticsearch"
  dedicated_master_type          = "m4.large.elasticsearch"
  es_zone_awareness              = true
  ebs_volume_size                = 100
}

Create Elasticsearch domain within a VPC and CloudWatch logs

module "es" {
  source  = "git::https://github.com/terraform-community-modules/tf_aws_elasticsearch.git?ref=v1.1.0"

  domain_name                    = "my-elasticsearch-domain"
  vpc_options                    = {
    security_group_ids = ["sg-XXXXXXXX"]
    subnet_ids         = ["subnet-YYYYYYYY"]
  }
  instance_count                 = 1
  instance_type                  = "t2.medium.elasticsearch"
  dedicated_master_type          = "t2.medium.elasticsearch"
  es_zone_awareness              = false
  ebs_volume_size                = 35
  
  advanced_options = {
    "rest.action.multi.allow_explicit_index" = "true"   # double quotes are required here
  }

  log_publishing_options = [
    {
      cloudwatch_log_group_arn = "arn:aws:logs:eu-central-1:604506250243:log-group:es:*"
      log_type                 = "INDEX_SLOW_LOGS"
      enabled                  = true
    },
    {
      cloudwatch_log_group_arn = "arn:aws:logs:eu-central-1:604506250243:log-group:es:*"
      log_type                 = "SEARCH_SLOW_LOGS"
      enabled                  = true
    },
    {
      cloudwatch_log_group_arn = "arn:aws:logs:eu-central-1:604506250243:log-group:es:*"
      log_type                 = "ES_APPLICATION_LOGS"
      enabled                  = true
    }
  ]
}

Create small (4-node) Elasticsearch domain in a VPC with dedicated master nodes

module "es" {
  source  = "git::https://github.com/terraform-community-modules/tf_aws_elasticsearch.git?ref=v1.1.0"

  domain_name                    = "my-elasticsearch-domain"
  vpc_options                    = {
    security_group_ids = ["sg-XXXXXXXX"]
    subnet_ids         = ["subnet-YYYYYYYY"]
  }
  instance_count                 = 4
  instance_type                  = "m4.2xlarge.elasticsearch"
  dedicated_master_threshold     = 4
  dedicated_master_type          = "m4.large.elasticsearch"
  es_zone_awareness              = true
  ebs_volume_size                = 100
}

Inputs

Outputs

Authors

Originally created by Steve Huff, Alexander Gramovich and these awesome contributors.

Module managed by Anton Babenko.

License

MIT licensed. See LICENSE.md for full details.