All Projects → juxt → Roll

juxt / Roll

AWS Blue/Green deployment using Clojure flavoured devops

Programming Languages

clojure
4091 projects
clojurescript
191 projects

Projects that are alternatives of or similar to Roll

Catapult
💥 Catapult is a DevOps website management platform for development teams.
Stars: ✭ 64 (-3.03%)
Mutual labels:  paas, devops
K8s Digitalocean Terraform
Deploy latest Kubernetes cluster on DigitalOcean using Terraform
Stars: ✭ 33 (-50%)
Mutual labels:  terraform, devops
Atlantis
Atlantis is now being maintained at https://github.com/runatlantis/atlantis
Stars: ✭ 628 (+851.52%)
Mutual labels:  terraform, devops
Terraform Modules
Reusable Terraform modules
Stars: ✭ 63 (-4.55%)
Mutual labels:  terraform, devops
Origin
Conformance test suite for OpenShift
Stars: ✭ 8,046 (+12090.91%)
Mutual labels:  paas, devops
Terratest
Terratest is a Go library that makes it easier to write automated tests for your infrastructure code.
Stars: ✭ 5,797 (+8683.33%)
Mutual labels:  terraform, devops
Ebs bckup
Stars: ✭ 32 (-51.52%)
Mutual labels:  terraform, devops
Porter
Kubernetes powered PaaS that runs in your own cloud.
Stars: ✭ 498 (+654.55%)
Mutual labels:  paas, devops
Lambda The Terraform Way
AWS Lambda using Terraform., an Introductory Cookbook
Stars: ✭ 1,056 (+1500%)
Mutual labels:  terraform, devops
Para
Para - community plugin manager and a "swiss army knife" for Terraform/Terragrunt - just 1 tool to facilitate all your workflows.
Stars: ✭ 47 (-28.79%)
Mutual labels:  terraform, devops
Guide
Kubernetes clusters for the hobbyist.
Stars: ✭ 5,150 (+7703.03%)
Mutual labels:  terraform, devops
Terraform Azure Reference
This repository helps you to implement Infrastructure as Code best practices using Terraform and Microsoft Azure.
Stars: ✭ 51 (-22.73%)
Mutual labels:  terraform, devops
Intro To Terraform
Sample code for the blog post series "A Comprehensive Guide to Terraform."
Stars: ✭ 550 (+733.33%)
Mutual labels:  terraform, devops
Terraform Aws Secure Baseline
Terraform module to set up your AWS account with the secure baseline configuration based on CIS Amazon Web Services Foundations and AWS Foundational Security Best Practices.
Stars: ✭ 596 (+803.03%)
Mutual labels:  terraform, devops
Terragrunt
Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules.
Stars: ✭ 5,446 (+8151.52%)
Mutual labels:  terraform, devops
Terraform Modules
Terraform Modules
Stars: ✭ 25 (-62.12%)
Mutual labels:  terraform, devops
Lain
LAIN's main repo
Stars: ✭ 455 (+589.39%)
Mutual labels:  paas, devops
Terracognita
Reads from existing Cloud Providers (reverse Terraform) and generates your infrastructure as code on Terraform configuration
Stars: ✭ 452 (+584.85%)
Mutual labels:  terraform, devops
Doact
A Terraform module for hosting your own runner for CI/CD on Digital Ocean to run jobs in your GitHub Actions workflows. 🚀
Stars: ✭ 42 (-36.36%)
Mutual labels:  terraform, devops
Devops Exercises
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
Stars: ✭ 20,905 (+31574.24%)
Mutual labels:  terraform, devops

= Roll - ClojureScript flavoured DevOps.

Roll will build you a Terraform template that will provision you an AWS application environment.

Roll is about the deployment of application services, providing an opinionated description of what services will commonly need in a given environment. Roll will provision you a secure PaaS (platform as a service) like AWS rolling blue/green infrastructure using Terraform, ClojureScript and Lumo.

Roll operates on a per environment basis, so you can have a dedicated Roll configuration file per environment. Roll also just works against Clojure maps, so you have all the flexibility of how you derive the configuration before passing it to Roll and ultimately on to Terraform.

Roll assumes that deployable application versions are accessible in S3, and asks you to provide the start-up instructions. Therefore it's orthogonal to whether you want to use Uberjars, Docker containers, or whatever else.

WARNING: Roll is in active development, everything about it - including this document - is being modified on a regular basis. Currently it is not recommended for general usage.

See our https://github.com/juxt/edge/blob/master/Machfile.edn[Edge] project for an example usage of Mach and Roll.

== Requirements

Above terraform 0.10.8.

== Features

=== Blue/Green deploy using AutoScaling groups

For each deployed version of an application, Roll uses a different AutoScaling group. These ASGs sit behind a load-balancer (Application Load Balancer) that we use to determine which ASG is 'live' at any given point.

The approach has significant advantages in that multiple application versions can be kept running at any given point, providing you flexibility on when you want to make a specific application version live, and to provide the ability to roll back quickly.

=== Route 53

Roll builds up a Terraform configuration so that you can easily point a Route 53 alias towards the load-balancer, that sits in front of the ASGs.

You need to manually configure the hosted zone (i.e. xxx.cloud), but with Roll you then can automatically provision the sub-aliases (i.e. yyy.xxx.cloud) and route them accordingly.

Roll works well when each service has it's own alias sitting inside the hosted zone.

=== Security and Key Management

Roll is opinionated in that each environment should have it's own KMS Encryption Key. Each provisioned service is then automatically configured to have decryption access to the key.

Using KMS Encryption keys is a great way of handling sensitive application configuration secrets (encrypted secrets can be kept in plain text, and only decrypted by the running application service).

=== It's just data

Roll takes in a Clojure map as the main argument and will give you a clojure map back, see the sample (for an example of this).

Roll has three stages, allowing you to hook in and manipulate the input/output data-structures for your own needs.

==== 1) Pre-process the Roll config.

Roll will preprocess the Roll configuration you give it. An example of pre-processing Roll deriving the most recent application version based on the contents of the S3 releases bucket, so you don't have to hardcode the application version manually. After the pre-processing step you can hook and it manipulate the data-structure as you wish.

==== 2 Create the Terraform data structure.

Before handing you Terraform JSON, you can operate on the Terraform Clojure data structure directly, to add additional Terraform configuration if you wish or to simple manipulate what Roll has come up with.

==== 3) Create final TF JSON output

This the final stage, producing Terraform JSON that is ready for Terraform to work with, to provision the infrastructure.

=== Compatible with Aero

We recommend that https://github.com/juxt/aero[Aero] be used to house the Roll configuration, to give you benefits such as be able to have references across the configuration, to pull out the pieces of commonality (to have for example, a reusable load-balancer configuration).

== Usage

=== With Lumo

You can use Roll with Lumo to generate a Terraform JSON file, which will be picked up when you subsequently execute Terraform commands in the same directory.

npm install juxt/roll
lumo -c node_modules/@juxt/roll/src -m roll.tf <roll-edn-file>

Will generate you a file called deployment.tf.json. If you name your Roll file roll.edn then the last argument can be ommitted.

=== With Mach

https://github.com/juxt/mach[Mach] is your Make.

Here is a sample Machfile configured with Roll, Terraform and some useful AWS targets:

{tfjson {product "deployment.tf.json"
         novelty (mach.core/modified-since product ["Machfile.edn" "roll.edn"])
         produce (let [config (aero.core/read-config "roll.edn" {})]
                   (-> config
                       (roll.core/preprocess)
                       (roll.core/deployment->tf)
                       (roll.core/->tf-json)))}

 mach/m2 [[aero "1.1.2"]
          [roll "0.0.7"]]
}

From the command-line

mach tfjson
mach plan
mach apply

Enjoy!

See our https://github.com/juxt/edge/blob/master/Machfile.edn[Edge] project for an example usage of Mach and Roll.

== Configuration

See the sample Roll configuration file link:sample-config.edn[sample-config.edn].

See also the link:sample.tf.json[sample output Terraform file] to get a feel for what is generated.

== Justification

Why not just use a PaaS, and why would you bring all these technologies together?

Let's justify the various pieces of Roll individually:

=== To PaaS or not?

Using a PaaS (Platform as a Service) such as AWS Beanstalk is a great way to get started and https://juxt.pro/blog/posts/beanstalk.html[we've blogged about deploying Clojure apps to Beanstalk using Docker].

Sometimes though a PaaS will not do enough for you, and you will want finer grained control. Unfortunately as soon as you step outside the safe and comfortable confines of the PaaS, you will end up battling low level cloud constructs such as AutoScaling Groups, Launch Configurations, Security Groups etc.

Almost straight away you will want to avoid the anti-pattern of configuring everything via point and click in the cloud provider UI, and you will seek to move towards Infrastructure as Code. Infrastructure as code means the infrastructure can be version controlled, and you won't get into the messy situation of forgetting who provisioned what, when and why.

=== Terraform

So let's consider the 'Infrastructure as Code' options. In AWS we have a couple of choices, and the ones we've investigated heavily are CloudFormation and Terraform. There is https://www.terraform.io/intro/vs/cloudformation.html[lots of resources out there detailing why Terraform is superior to CloudFormation], but for us Terraform offers a friendlier DSL with comments (comments!), and where you can review pre-execution plans prior to Terraform making the declared written infrastructure plans a reality.

TL;DR; Terraform rocks.

=== ClojureScipt and Mach and Lumo

So why doesn't the story stop there? Why friends, must we continue into the lands of ClojureScript tooling running on the server?

Alas, Terraform by itself is not enough. For example we want blue/green deployments - which in our use case involves firing up new AutoScaling groups when we want a new version deployed and to leave the old one running. When we're happy with our candidate deployment we want to effect a load-balancer change that will redirect where the Route53 domain entry is pointing to, thus making the new AutoScaling group the one that is live. We may want to leave the previously 'live' AutoScaling group hanging around - in case we need to revert back to it.

Roll achieves this by rebuilding the Terraform configuration to match a higher level declarative input which you can readily change (i.e. to deploy newer versions of an application). Roll is a DSL that sits above Terraform, providing an opinionated description of infrastructure that Terraform is used to make happen.

Roll aims to give you the full Beanstalk like PaaS experience except there's no PaaS and you're completely in control. It also has extras such as setting up KMS for each 'service'.

== Road map

Roll is in active development, come back soon for the full readme experience.

== Developing Roll

Add these lines to your Machfile before Roll is invoked

#cp "{path-to-roll}/src"
#cp "{path-to-roll}/tf"

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].