shinesolutions / aem-aws-stack-builder

AEM AWS Stack Builder

A set of Ansible playbooks for building Adobe Experience Manager (AEM) architectures on AWS using CloudFormation stacks.

Stack Builder has been designed with a focus on modularity, allowing the separation between network and application components, while also providing a flexible way to support multiple architectures that run a combination of the following components across multiple AEM versions:

• author-primary - contains AEM Author running in primary mode
• author-standby - contains AEM Author running in standby mode
• publish - contains AEM Publish
• author-dispatcher - contains AEM Dispatcher with author-dispatcher configuration, sitting in front of author component
• publish-dispatcher - contains AEM Dispatcher with publish-dispatcher configuration, sitting in front of publish component
• orchestrator - contains AEM Orchestrator
• chaos-monkey - contains Chaos Monkey
• author-publish-dispatcher - contains AEM Author, AEM Publish, and AEM Dispatcher

Stack Builder currently supports the following AEM architectures:

• Full Set (diagram) - includes AEM Author, Publish, and Dispatcher on separate EC2 instances with blue-green deployment, auto-recovery, auto-scaling, backup, and compaction support, suitable for all types (e.g. production, staging, testing, and development) of environments
• Consolidated (diagram) - includes AEM Author, Publish, and Dispatcher on a single EC2 instance with backup and compaction support and a much lower AWS cost footprint, suitable for testing and development environments

Other than the above AEM architectures, Stack Builder also provides the following utilities:

• Stack Manager(diagram) - includes AEM Stack Manager, set of AWS Lambda functions that will execute AEM functionalities via an SSM agent
• Network - includes CloudFormation templates for creating VPC, subnets, and some sample NAT Gateway and Bastion set up

Learn more about AEM AWS Stack Builder:

• Installation
• Configuration
• Usage
  • Network
  • AEM Full-Set Architecture
  • AEM Consolidated Architecture
  • AEM Stack Manager
• Testing
• AWS Resources
• AWS System Tags
• Customisation Points
• Recovery
  • Recovery Steps
  • Recovery Scenarios
• Descriptors
  • Deployment Descriptor Definition
  • Package Backup Descriptor Definition
  • Content Health Check Descriptor Definition
• Logs
• Communication Flow
• Permission Types
• Snapshot Types
• Frequently Asked Questions
• Troubleshooting Guide
• Upgrade Guide
• Stacks Structure
• AWS Services
• Presentations

AEM AWS Stack Builder is part of AEM OpenCloud platform.

Installation

• Either clone AEM AWS Stack Builder git clone https://github.com/shinesolutions/aem-aws-stack-builder.git or download one of the released versions

• Install the following required tools:

  • Ruby version 2.3.0 or later
  • Python version 2.7.x
  • GNU Make
    

  Alternatively, you can use AEM Platform BuildEnv Docker container to run AEM AWS Stack Builder build targets.

• Resolve the Python packages dependencies by running make deps

Usage

• Set up the required AWS resources
• Create configuration file
• Set up the configuration files by running make config config_path=<path/to/config/dir>
• Download open source library artifacts and upload them to S3 by running make library config_path=<path/to/config/dir>

Network

Ensure configuration file for network has been set up.

From the above base configuration, generate template network configuration file:

make generate-network-config stack_prefix=<network_stack_prefix> config_path=<path/to/config/dir>

The above is a once off action which will create a bootstrap network configuration file at <path/to/config/dir>/network.yaml, this will save you the trouble of manually creating a long configuration file. Alternatively, you can create the network configuration file manually.

Create VPC stack:

make create-vpc stack_prefix=<network_stack_prefix> config_path=<path/to/config/dir>

Create network resources stack:

make create-network stack_prefix=<network_stack_prefix> config_path=<path/to/config/dir>

Alternatively, if you don't have the permission to create VPC and/or network resources, you can create a network-exports stack that contains the details of your subnets:

make create-network-exports stack_prefix=<network_stack_prefix> config_path=<path/to/config/dir>

AEM Stack Manager

Ensure configuration file for AEM Stack Manager has been set up.

Create AEM Stack Manager stacks:

make create-stack-manager stack_prefix=<stack_manager_stack_prefix> config_path=<path/to/config/dir>

AEM Full-Set Architecture

Ensure configuration file for AEM Full-Set architecture has been set up.

The simplest way to create this AEM architecture is by standing up both full set prerequisites and main stacks in one go:

make create-full-set stack_prefix=<fullset_stack_prefix> config_path=<path/to/config/dir>

However, it is also possible to separate the prerequisites from the main stacks. A use case scenario for this set up is when you want to keep the prerequisites stack around while creating/deleting the main stack within an environment, this allows you to cut some cost and to speed up environment standing up time from the second time onward.

Create prerequisites stack which contains the instance profiles, security groups, and messaging SNS SQS resources:

make create-full-set-prerequisites stack_prefix=<fullset_prerequisites_stack_prefix> config_path=<path/to/config/dir>

Create main stack which contains EC2 and Route53 resources:

make create-full-set-main stack_prefix=<fullset_main_stack_prefix> prerequisites_stack_prefix=<fullset_prerequisites_stack_prefix> config_path=<path/to/config/dir>

AEM Consolidated Architecture

Ensure configuration file for AEM Consolidated architecture has been set up.

The simplest way to create this AEM architecture is by standing up both full set prerequisites and main stacks in one go:

make create-consolidated stack_prefix=<consolidated_stack_prefix> config_path=<path/to/config/dir>

It is also possible to separate the prerequisites from the main stacks. A use case scenario for this set up is when you want to reuse the same prerequisites stack for multiple main stacks. Please note that having a one to many mapping between prerequisites stack to multiple main stacks is only applicable for development environments, and not for production.

Create prerequisites stack which contains the instance profiles and security groups:

make create-consolidated-prerequisites stack_prefix=<consolidated_prerequisites_stack_prefix> config_path=<path/to/config/dir>

Create main stack which contains EC2 and Route53 resources:

make create-consolidated-main stack_prefix=<consolidated_main_stack_prefix> prerequisites_stack_prefix=<consolidated_prerequisites_stack_prefix> config_path=<path/to/config/dir>

Switch DNS

You can point a DNS record to an AEM Full-Set or AEM Consolidated architecture:

make switch-dns-full-set stack_prefix=<full_set_stack_prefix> config_path=<path/to/config/dir> publish_dispatcher_hosted_zone=<publish_dispatcher_hosted_zone_name> publish_dispatcher_record_set=<publish_dispatcher_record_name> author_dispatcher_hosted_zone=<author_dispatcher_hosted_zone_name>  author_dispatcher_record_set=<author_dispatcher_record_name>

make switch-dns-consolidated stack_prefix=<consolidated_stack_prefix> config_path=<path/to/config/dir> author_publish_dispatcher_hosted_zone=<author_publish_dispatcher_hosted_zone> author_publish_dispatcher_record_set=<author_publish_dispatcher_record_set>

Testing

Testing with remote dependencies

You can run integration test for creating, testing, and deleting the AEM Stack Manager, AEM Consolidated, AEM Full-Set environments using the command make test-integration-<aem_version>-<os_type> test_id=<sometestid>, which downloads the dependencies from the Internet.

Testing with local dependencies

If you're working on the dependencies of AEM AWS Stack Builder and would like to test them as part of environment creation before pushing the changes upstream, you need to:

• Clone the dependency repos AEM AWS Stack Provisioner, Puppet AEM Resources, Puppet AEM Curator, Puppet AEM Orchestrator, Puppet SimianArmy, AEM Hello World Custom Stack Provisioner, AEM Hello World Config at the same directory level as AEM AWS Stack Builder
• Make your code changes against those dependency repos
• Run make test-integration-local-<aem_version>-<os_type> test_id=<sometestid> for integration testing using local dependencies, which copies those local dependency repos to your local AEM AWS Stack Provisioner, packages it and versioned with your test_id, uploads to S3, and uses them as part of the test

Presentations

• AEM OpenCloud
• AEM OpenCloud - What's New Since 2.0.0
• AEM Open Cloud - The First Two Years
• Open Source AEM Platform: A Short Intro