CloudQuery AWS Module
This folder contains a Terraform module to deploy a CloudQuery cluster in AWS on top of EKS.
Usage
Examples are included in the example folder, but simple usage is as follows:
module "cloudquery" {
source = "cloudquery/cloudquery/aws"
version = "~> 0.5"
name = "cloudquery"
cidr = "10.20.0.0/16"
azs = ["us-east-1a", "us-east-1b", "us-east-1c"]
public_subnets = ["10.20.1.0/24", "10.20.2.0/24", "10.20.3.0/24"]
private_subnets = ["10.20.101.0/24", "10.20.102.0/24", "10.20.103.0/24"]
database_subnets = ["10.10.21.0/24", "10.10.22.0/24"]
# path to your cloudquery config
config_file = "config.hcl"
}Existing VPC
This way allows integration with your existing AWS resources - VPC, public and private subnets. Specify the following arguments (see methods described above):
If vpc_id is specified it will take precedence over cidr and existing VPC will be used.
Make sure that both private and public subnets were created in the same set of availability zones.
Run Helm Seperately
Examples
Requirements
| Name | Version |
|---|---|
| terraform | >= 0.15 |
| aws | >= 4.0 |
| helm | >= 2.5 |
Providers
| Name | Version |
|---|---|
| aws | >= 4.0 |
| helm | >= 2.5 |
Modules
| Name | Source | Version |
|---|---|---|
| cluster_irsa | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | ~> 4.18 |
| eks | terraform-aws-modules/eks/aws | ~> 18.17.0 |
| iam_policy | terraform-aws-modules/iam/aws//modules/iam-policy | ~> 4 |
| rds | terraform-aws-modules/rds/aws | ~> 4.2.0 |
| security_group | terraform-aws-modules/security-group/aws | ~> 4.2 |
| vpc | terraform-aws-modules/vpc/aws | ~> 3.0 |
Resources
| Name | Type |
|---|---|
| aws_secretsmanager_secret.cloudquery_secret | resource |
| aws_secretsmanager_secret_version.cloudquery_secret_version | resource |
| helm_release.cloudquery | resource |
| aws_availability_zones.available | data source |
| aws_eks_cluster_auth.cluster | data source |
| aws_vpc.cq_vpc | data source |
Inputs
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| chart_values | Variables to pass to the helm chart | string |
"" |
no |
| chart_version | The version of CloudQuery helm chart | string |
"0.1.10" |
no |
| config_file | Path to the CloudQuery config.hcl | string |
"" |
no |
| database_subnet_group | If vpc_id is specified, path the subnet_group name where the RDS should reside | string |
"" |
no |
| install_helm_chart | Enable/Disable helm chart installation | bool |
true |
no |
| name | Name to use on all resources created (VPC, RDS, etc) | string |
"cloudquery" |
no |
| postgres_engine_version | Version of Amazon RDS Postgres engine to use | string |
"14.2" |
no |
| postgres_family | Family of Amazon RDS Postgres engine to use | string |
"postgres14" |
no |
| postgres_instance_class | Postgresql Instance Class | string |
"db.t4g.large" |
no |
| postgres_major_engine_version | Major version of Amazon RDS Postgres engine to use | string |
"14" |
no |
| public_subnet_ids | A list of IDs of existing public subnets inside the VPC | list(string) |
[] |
no |
| role_policy_arns | Policies for the role to use for the EKS service account | list(string) |
[ |
no |
| tags | A map of tags to use on all resources | map(string) |
{} |
no |
| vpc_id | ID of an existing VPC where resources will be created | string |
null |
no |
Outputs
| Name | Description |
|---|---|
| cq_dsn | CQ_DSN variable for CloudQuery CLI |
| irsa_arn | ARN of IRSA - (IAM Role for service account) |
| irsa_name | Name of IRSA - (IAM Role for service account) |
| rds_cluster_master_password | Master password for cloudquery rds database |
| vpc_id | ID of the VPC that was created or passed in |
Troubleshooting
If helm installtion is stuck in some hanging state you can run the following commands:
# check if helm is installed in cloudquery namespace
helm ls -n cloudquery
# If yes uninstall with the your release name
helm uninstall YOUR_RELEASE_NAME -n cloudqueryAuthors
Module is maintained by Anton Babenko and CloudQuery Team.
License
Apache 2 Licensed. See LICENSE for full details.