terraform-aws-elasticsearch
Terraform module to create Amazon Elasticsearch Service clusters, following the Well-Architected Framework and best AWS practices.
Amazon Elasticsearch Service is a fully managed service that makes it easy to deploy, operate, and scale Elasticsearch clusters in the AWS Cloud. Elasticsearch is a popular open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis. With Amazon ES, you get direct access to the Elasticsearch APIs; existing code and applications work seamlessly with the service.
Examples
Check the examples folder where you can see how to configure a public ES cluster, and another example showing how to set it with VPC options.
Usage
You can use this module to create your Amazon ES cluster by defining each parameters blocks as follows:
module "aws_es" {
source = "lgallard/elasticsearch/aws"
domain_name = "elasticsearch_public"
elasticsearch_version = "7.1"
cluster_config = {
dedicated_master_enabled = true
instance_count = 3
instance_type = "r5.large.elasticsearch"
zone_awareness_enabled = true
availability_zone_count = 3
}
ebs_options = {
ebs_enabled = "true"
volume_size = "25"
}
encrypt_at_rest = {
enabled = true
kms_key_id = "arn:aws:kms:us-east-1:123456789101:key/cccc103b-4ba3-5993-6fc7-b7e538b25fd8"
}
log_publishing_options = {
index_slow_logs = {
enabled = true
cloudwatch_log_group_arn = "arn:aws:logs:us-east-1:123456789101:log-group:/aws/elasticsearch/index_slow_logs:*"
log_publishing_options_retention = 90
}
search_slow_logs = {
enabled = true
cloudwatch_log_group_arn = "arn:aws:logs:us-east-1:123456789101:log-group:/aws/elasticsearch/search_slow_logs:*"
}
es_application_logs = {
enabled = true
cloudwatch_log_group_name = "es_application_logs_dev"
}
audit_logs = {
enabled = false
cloudwatch_log_group_name = "audit_logs_dev"
}
}
advanced_options = {
"rest.action.multi.allow_explicit_index" = true
}
domain_endpoint_options = {
enforce_https = true
custom_endpoint_enabled = true
custom_endpoint = "lgallardo.com"
custom_endpoint_certificate_arn = "arn:aws:acm:us-east-1:123456789101:certificate/abcd1234-ef11-abcd-1234-abcd1234efef"
}
node_to_node_encryption_enabled = true
snapshot_options_automated_snapshot_start_hour = 23
tags = {
Owner = "sysops"
env = "dev"
}
Note: You can also define the above ElasticSearch cluster using just the module variables. Instead of defining a cluster_config block (list of map), you can set each of the cluster_config_* variables, as shown below:
module "aws_es" {
source = "lgallard/elasticsearch/aws"
domain_name = "elasticsearch_public"
elasticsearch_version = "7.1"
cluster_config_dedicated_master_enabled = true
cluster_config_instance_count = 3
cluster_config_instance_type = "r5.large.elasticsearch"
cluster_config_zone_awareness_enabled = "true"
cluster_config_availability_zone_count = 3
ebs_options_ebs_enabled = true
ebs_options_volume_size = 25
encrypt_at_rest_enabled = true
encrypt_at_rest_kms_key_id = "alias/aws/es"
log_publishing_options_enabled = true
log_publishing_options_log_type = "INDEX_SLOW_LOGS"
advanced_options = {
"rest.action.multi.allow_explicit_index" = true
}
node_to_node_encryption_enabled = true
snapshot_options_automated_snapshot_start_hour = 23
tags = {
Owner = "sysops"
env = "dev"
}
Requirements
| Name | Version |
|---|---|
| terraform | >= 0.12.9 |
| aws | >= 3.35.0 |
| random | >=3.1.2 |
Providers
| Name | Version |
|---|---|
| aws | 4.18.0 |
| random | 3.3.1 |
Modules
No modules.
Resources
| Name | Type |
|---|---|
| aws_cloudwatch_log_group.es_cloudwatch_log_group | resource |
| aws_cloudwatch_log_resource_policy.es_aws_cloudwatch_log_resource_policy | resource |
| aws_elasticsearch_domain.es_domain | resource |
| aws_iam_service_linked_role.es | resource |
| random_password.master_password | resource |
| aws_kms_key.aws_es | data source |
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| access_policies | IAM policy document specifying the access policies for the domain | string |
"" |
no |
| advanced_options | Key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your Elasticsearch domain on every apply | map(string) |
{} |
no |
| advanced_security_options | Options for fine-grained access control | any |
{} |
no |
| advanced_security_options_create_random_master_password | Whether to create random master password for Elasticsearch master user | bool |
false |
no |
| advanced_security_options_enabled | Whether advanced security is enabled (Forces new resource) | bool |
false |
no |
| advanced_security_options_internal_user_database_enabled | Whether the internal user database is enabled. If not set, defaults to false by the AWS API. | bool |
false |
no |
| advanced_security_options_master_user_arn | ARN for the master user. Only specify if internal_user_database_enabled is not set or set to false) |
string |
null |
no |
| advanced_security_options_master_user_password | The master user's password, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true. |
string |
null |
no |
| advanced_security_options_master_user_username | The master user's username, which is stored in the Amazon Elasticsearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true. |
string |
null |
no |
| advanced_security_options_random_master_password_length | Length of random master password to create | number |
16 |
no |
| cloudwatch_log_enabled | Change to false to avoid deploying any Cloudwatch Logs resources | bool |
true |
no |
| cluster_config | Cluster configuration of the domain | any |
{} |
no |
| cluster_config_availability_zone_count | Number of Availability Zones for the domain to use with | number |
3 |
no |
| cluster_config_cold_storage_options_enabled | Indicates whether to enable cold storage for an Elasticsearch domain | bool |
false |
no |
| cluster_config_dedicated_master_count | Number of dedicated master nodes in the cluster | number |
3 |
no |
| cluster_config_dedicated_master_enabled | Indicates whether dedicated master nodes are enabled for the cluster | bool |
true |
no |
| cluster_config_dedicated_master_type | Instance type of the dedicated master nodes in the cluster | string |
"r5.large.elasticsearch" |
no |
| cluster_config_instance_count | Number of instances in the cluster | number |
3 |
no |
| cluster_config_instance_type | Instance type of data nodes in the cluster | string |
"r5.large.elasticsearch" |
no |
| cluster_config_warm_count | The number of warm nodes in the cluster | number |
null |
no |
| cluster_config_warm_enabled | Indicates whether to enable warm storage | bool |
false |
no |
| cluster_config_warm_type | The instance type for the Elasticsearch cluster's warm nodes | string |
null |
no |
| cluster_config_zone_awareness_enabled | Indicates whether zone awareness is enabled. To enable awareness with three Availability Zones | bool |
false |
no |
| cognito_options | Options for Amazon Cognito Authentication for Kibana | any |
{} |
no |
| cognito_options_enabled | Specifies whether Amazon Cognito authentication with Kibana is enabled or not | bool |
false |
no |
| cognito_options_identity_pool_id | ID of the Cognito Identity Pool to use | string |
"" |
no |
| cognito_options_role_arn | ARN of the IAM role that has the AmazonESCognitoAccess policy attached | string |
"" |
no |
| cognito_options_user_pool_id | ID of the Cognito User Pool to use | string |
"" |
no |
| create_service_link_role | Create service link role for AWS Elasticsearch Service | bool |
true |
no |
| domain_endpoint_options | Domain endpoint HTTP(S) related options. | any |
{} |
no |
| domain_endpoint_options_custom_endpoint | Fully qualified domain for your custom endpoint | string |
null |
no |
| domain_endpoint_options_custom_endpoint_certificate_arn | ACM certificate ARN for your custom endpoint | string |
null |
no |
| domain_endpoint_options_custom_endpoint_enabled | Whether to enable custom endpoint for the Elasticsearch domain | bool |
false |
no |
| domain_endpoint_options_enforce_https | Whether or not to require HTTPS | bool |
false |
no |
| domain_endpoint_options_tls_security_policy | The name of the TLS security policy that needs to be applied to the HTTPS endpoint. Valid values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07 |
string |
"Policy-Min-TLS-1-2-2019-07" |
no |
| domain_name | Name of the domain | string |
n/a | yes |
| ebs_enabled | Whether EBS volumes are attached to data nodes in the domain | bool |
true |
no |
| ebs_options | EBS related options, may be required based on chosen instance size | any |
{} |
no |
| ebs_options_iops | The baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the Provisioned IOPS EBS volume type | number |
0 |
no |
| ebs_options_volume_size | The size of EBS volumes attached to data nodes (in GB). Required if ebs_enabled is set to true | number |
10 |
no |
| ebs_options_volume_type | The type of EBS volumes attached to data nodes | string |
"gp2" |
no |
| elasticsearch_version | The version of Elasticsearch to deploy. | string |
"7.1" |
no |
| enabled | Change to false to avoid deploying any AWS ElasticSearch resources | bool |
true |
no |
| encrypt_at_rest | Encrypt at rest options. Only available for certain instance types | any |
{} |
no |
| encrypt_at_rest_enabled | Whether to enable encryption at rest | bool |
true |
no |
| encrypt_at_rest_kms_key_id | The KMS key id to encrypt the Elasticsearch domain with. If not specified then it defaults to using the aws/es service KMS key | string |
"alias/aws/es" |
no |
| log_publishing_options | Options for publishing slow logs to CloudWatch Logs | any |
{} |
no |
| log_publishing_options_retention | Retention in days for the created Cloudwatch log group | number |
90 |
no |
| node_to_node_encryption | Node-to-node encryption options | any |
{} |
no |
| node_to_node_encryption_enabled | Whether to enable node-to-node encryption | bool |
true |
no |
| snapshot_options | Snapshot related options | any |
{} |
no |
| snapshot_options_automated_snapshot_start_hour | Hour during which the service takes an automated daily snapshot of the indices in the domain | number |
0 |
no |
| tags | A mapping of tags to assign to the resource | map(any) |
{} |
no |
| timeouts | Timeouts map. | map(any) |
{} |
no |
| timeouts_update | How long to wait for updates. | string |
null |
no |
| vpc_options | VPC related options, see below. Adding or removing this configuration forces a new resource | any |
{} |
no |
| vpc_options_security_group_ids | List of VPC Security Group IDs to be applied to the Elasticsearch domain endpoints. If omitted, the default Security Group for the VPC will be used | list(any) |
[] |
no |
| vpc_options_subnet_ids | List of VPC Subnet IDs for the Elasticsearch domain endpoints to be created in | list(any) |
[] |
no |
Outputs
| Name | Description |
|---|---|
| arn | Amazon Resource Name (ARN) of the domain |
| domain_id | Unique identifier for the domain |
| endpoint | Domain-specific endpoint used to submit index, search, and data upload requests |
| kibana_endpoint | Domain-specific endpoint for kibana without https scheme |
| master_password | Master password |
| master_username | Master username |
| vpc_options_availability_zones | If the domain was created inside a VPC, the names of the availability zones the configured subnet_ids were created inside |
| vpc_options_vpc_id | If the domain was created inside a VPC, the ID of the VPC |