AWS Sync Routes

Synchronizes the specified route from the main/default route table to all custom route tables in the VPC.

The primary use case is for VMware Cloud on AWS (VMC) software-defined datacenter (SDDC) managed routes, but this could also be used as-is for any scenario where syncing AWS VPC routes to custom route tables is desired.

This solution should only cost a few dollars per month to operate- depending on the number of routes managed and the number of API calls. Please see the pricing guides for further details: AWS Lambda, AWS API Gateway, Amazon SNS, Amazon S3, Amazon CloudWatch Logs, & CloudFormation.

This is also an infrastructure as code solution, meaning that it should only require a few commands to deploy once the prerequisites are installed & configured, and was designed so that it should not require much attention thereafter.

Once deployed, the endpoint generally takes 1-3 seconds to execute when called, and is idempotent, so changes will only be implemented once when the specified route either does not exist in one or more custom route tables or the next hop value changes. Routes will not be programmatically removed by this solution.

One customer success story to date is that this solution was used in us-east-1 to synchronize 26 production routes across 4 custom route tables (104 concurrent route synchronizations) with a 5 second polling interval. In testing, all route targets updated successfully within 4 seconds. This met the customer's requirement of completing all route synchronizations prior to timeout of a mission critical application at 15 seconds, and allowed them to complete a critical maintenance window.

Please test thoroughly.

Architecture

Architecture diagram

Infrastructure as code diagram

Prerequisites

• NodeJS
• AWS CLI
  • Configured AWS CLI profile
• AWS Amplify CLI toolchain
  • NOTE: Interoperability between the AWS Amplify CLI and AWS CLI Multi-Factor Authentication (MFA), as well as assume role with MFA, have been validated in testing per request. Please also see this AWS blog post for a few great examples.

For Windows users

• Windows Subsystem for Linux

Getting Started

• Run amplify env add, which will:

  • Map your AWS CLI profile to a new AWS Amplify environment

  • NOTE: The default AWS region specified for your AWS CLI profile will be used, such as us-east-1. Update the value (temporarily or otherwise) if you want the resources deployed elsewhere.

  • Deploy the root CloudFormation stack, which includes the following default resources for an AWS Amplify CLI managed project:

    • S3 deployment bucket
    • CloudFormation root stack
    • IAM AuthRole
    • IAM UnauthRole

  • Create:

    • ./amplify/team-provider-info.json:
      • Specifies the names and ARNs of the resources in the root stack for your environment.
      • IMPORTANT: Do not submit pull requests with your ./amplify/team-provider-info.json file. Again, it contains information about your environment's resources.
      • This file should be committed in your private repo and was intentionally excluded from ./.gitignore.
      • If you updated the default region for your AWS CLI profile and want to change it back, you can do so once this file exists because all of the other resources will be deployed to the region specified in this file.
    • ./amplify/backend/amplify-meta.json:
      • Compiled from ./amplify/backend/backend-config.json and ./amplify/team-provider-info.json.
      • Specifies:
        • The managed backend categories
        • Their dependencies
        • Deployment information
    • ./amplify/#current-cloud-backend/: Directory containing the compiled version of the environment.
    • ./src/aws-exports.js: Covered in the next step.

  • The example below specifies an environment name of dev, an editor of None, and the default AWS CLI profile. Adjust these for your use case.

    $ amplify env add
    Note: It is recommended to run this command from the root of your app directory
    ? Enter a name for the environment dev
    ? Choose your default editor:
      Sublime Text
      Visual Studio Code
      Atom Editor
      IDEA 14 CE
      Vim (via Terminal, Mac OS only)
      Emacs (via Terminal, Mac OS only)
    ❯ None
    Using default provider  awscloudformation
    
    For more information on AWS Profiles, see:
    https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
    
    ? Do you want to use an AWS profile? (Y/n)
    ? Please choose the profile you want to use (Use arrow keys)
    ❯ default
    ⠼ Initializing project in the cloud...
    

• Review the API request throttling parameters: burstLimit & rateLimit, in ./amplify/backend/api/awssyncroutes/parameters.json and update if necessary.

• Run amplify push to deploy the rest of the resources.

  $ amplify push
  
  Current Environment: dev
  
  | Category | Resource name | Operation | Provider plugin   |
  | -------- | ------------- | --------- | ----------------- |
  | Function | awssyncroutes | Create    | awscloudformation |
  | Api      | awssyncroutes | Create    | awscloudformation |
  ? Are you sure you want to continue? (Y/n)
  

  • This will create:
    • ./src/aws-exports.js: One place where you can find the root URL of the new API Gateway.
    • ./amplify/backend/awscloudformation/nested-cloudformation-stack.yml: Nested CloudFormation stack specification.
  • Also, the S3Bucket will be automatically populated in ./amplify/backend/function/awssyncroutes/awssyncroutes-cloudformation-template.json.
    • IMPORTANT: Do not submit pull requests with your S3 deployment bucket.

• Once complete, retrieve the API key.

  • One way to do this:
    • aws apigateway get-api-keys: Copy the id value for the next command.
    • aws apigateway get-api-key --include-value --api-key <id>: Copy the value value.

• Then subscribe to the SNS topic to opt-in for change notifications.

• Now, you are ready to test the API endpoint.

  • One way to do this:

    curl --data '{"destination-cidr-block":"<destination cidr block>", "dry-run": true}' --header "X-API-Key: <api key>" --header "Content-Type: application/json" --request PATCH https://<api gateway id>.execute-api.<region>.amazonaws.com/<stage name>/vpcs/<vpc id>/route-tables/<route table id>

Request requirements

Requests will only be accepted if the specified destination CIDR block:

• Is not the default local route.
• Is not a propagated route.
• Has an ENI target.
• Is in an active state in the main route table.

Usage

There is only one API endpoint with two path parameters: /vpcs/<vpc id>/route-tables/<route table id> with a HTTP PATCH request method.

NOTE: This implementation does not follow all of the RFC5789 specifications for a HTTP PATCH method- it does not use a PATCH document and it is idempotent.

The request body schema has one required property: destination-cidr-block, and one optional property: dry-run.

Example

cURL

curl --data '{"destination-cidr-block":"<destination cidr block>", "dry-run": true}' --header "X-API-Key: <api key>" --header "Content-Type: application/json" --request PATCH https://<api gateway id>.execute-api.<region>.amazonaws.com/<stage name>/vpcs/<vpc id>/route-tables/<route table id>

PowerShell Invoke-RestMethod cmdlet

Invoke-RestMethod -Method 'Patch' -Uri 'https://<api gateway id>.execute-api.<region>.amazonaws.com/<stage name>/vpcs/<vpc id>/route-tables/<route table id>' -Headers @{ 'Content-Type'='application/json'; 'X-API-Key'='<api key>'} -Body '{"destination-cidr-block":"172.30.0.0/16", "dry-run": false}'

Client script

A bash script has been added, which can be used to call the API endpoint asynchronously for a comma-delimited list of destination CIDR blocks in an loop. One way to implement this would be to call the script in a screen session so that the script can run without maintaining a SSH session and administrators can disconnect/reconnect as needed.

./scripts/aws-sync-routes-client.sh -i $api_gateway_id -k $api_key -r 'us-east-1' -c '172.30.0.0/16, 172.31.0.0/16' -s 5 -t rtb-01234567 -v vpc-01234567

./scripts/aws-sync-routes-client.sh --help

Troubleshooting

Too Many Requests

If you start receiving Too Many Requests error messages, this means that the configured rate & burst limits for your API Gateway instance are set too low for the frequency in which you are polling. Adjust the burstLimit & rateLimit values in parameters.json, then run amplify push to deploy the changes, and try again.

Of note, the rate & burst limits are only configured in the API token usage plan as described in the API Gateway CloudFormation template, not the API Gateway deployment stage. Throttling could be configured in both locations, but configuring this in both locations is unnecessary and the current configuration makes it easy to use additional API tokens if so desired.

Request Limit Exceeded

If you are syncing a batch of routes and start receiving Request Limit Exceeded error messages, this means that the requests are being throttled due to the number of requests in this region in this account within a set period of time.

First try decreasing the frequency of API calls. If that is unteneble, please open an AWS Support ticket explaining the business case and request an increase to the limits for the following:

• EC2 describe route tables
  • 1 call per request (sustained)
• EC2 create route
  • 1 call per custom route table per request where the route exists in the main route table, but not in the custom route table (burst)
  • In dry-run mode, this action will be called for every request since changes will not be executed, resulting in a higher likelihood of throttling
• EC2 replace route
  • 1 call per custom route table per request where the route exists in both the main & custom route table, but have different next hop values (burst)
  • In dry-run mode, this action will be called for every request since changes will not be executed, resulting in a higher likelihood of throttling

Project tree

.
├── .github/
│   └── PULL_REQUEST_TEMPLATE.md
├── .vscode/
│   └── settings.json
├── amplify/
│   ├── #current-cloud-backend/
│   ├── .config/
│   │   ├── local-aws-info.json
│   │   ├── local-env-info.json
│   │   └── project-config.json
│   ├── backend/
│   │   ├── api/
│   │   │   └── awssyncroutes/
│   │   │       ├── api-params.json
│   │   │       ├── awssyncroutes-cloudformation-template.json
│   │   │       └── parameters.json
│   │   ├── awscloudformation/
│   │   │   └── nested-cloudformation-stack.yml
│   │   ├── function/
│   │   │   └── awssyncroutes/
│   │   │       ├── dist/
│   │   │       ├── src/
│   │   │       │   ├── node_modules/
│   │   │       │   ├── app.js
│   │   │       │   ├── index.js
│   │   │       │   ├── package-lock.json
│   │   │       │   └── package.json
│   │   │       ├── amplify.state
│   │   │       ├── awssyncroutes-cloudformation-template.json
│   │   │       └── parameters.json
│   │   ├── amplify-meta.json
│   │   └── backend-config.json
│   └── team-provider-info.json
│   └── README.md
├── docs/
│   ├── about/
│   │   ├── contributing.md -> ../../CONTRIBUTING.md
│   │   └── license.md -> ../../LICENSE
│   ├── adr/
│   │   ├── 0001-record-architecture-decisions.md
│   │   ├── 0002-aws-amplify-cli-toolchain.md
│   │   ├── 0003-http-patch-method.md
│   │   ├── 0004-api-key.md
│   │   ├── 0005-uri.md
│   │   └── 0006-specificity.md
│   ├── stylesheets/
│   │   └── extra.css
│   ├── Architecture.png -> ../Architecture.png
│   ├── InfrastructureAsCode.png -> ../InfrastructureAsCode.png
│   └── index.md -> ../README.md*
├── material.bak/
│   ├── assets/
│   │   └── stylesheets/
│   │       ├── application-palette.css
│   │       └── application.css
│   ├── partials/
│   │   └── palette.html
│   └── main.html
├── scripts/
│   └── aws-sync-routes-client.sh*
├── src/
│   └── aws-exports.js
├── .adr-dir
├── .editorconfig
├── .eslintrc.js
├── .gitignore
├── .markdownlint.yml
├── Architecture.png
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── InfrastructureAsCode.png
├── LICENSE
├── NOTICE
├── README.md*
└── mkdocs.yml

The project tree was generated with the following command:

tree -aFL 6 --dirsfirst --noreport -I ".git|site|*-latest-build.zip"

License

This library is licensed under the Apache 2.0 License.