All Projects → AutoSpotting → Autospotting

AutoSpotting / Autospotting

Licence: other
Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups. Installs in minutes using CloudFormation or Terraform. Convenient to deploy at scale using StackSets. Uses tagging to avoid launch configuration changes. Automated spot termination handling. Reliable fallback to on-demand instances.

Programming Languages

go
31211 projects - #10 most used programming language
Makefile
30231 projects
Dockerfile
14818 projects

Projects that are alternatives of or similar to Autospotting

AutoSpotting
Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups. Installs in minutes using CloudFormation or Terraform. Convenient to deploy at scale using StackSets. Uses tagging to avoid launch configuration changes. Automated spot termination handling. Reliable fallback to on-demand instances.
Stars: ✭ 2,058 (+2.18%)
Mutual labels:  infrastructure, cloudformation, ec2, autoscaling-groups, spot-instances, cost, ec2-instance, amazon-web-services, autoscaling, aws-autoscaling, golang-application, ec2-spot, terraform-module, cheaper-spot-instances
Aws Sdk Perl
A community AWS SDK for Perl Programmers
Stars: ✭ 153 (-92.4%)
Mutual labels:  aws, cloudformation, amazon-web-services, ec2, autoscaling
Aws Auto Terminate Idle Emr
AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.
Stars: ✭ 21 (-98.96%)
Mutual labels:  automation, aws, aws-lambda, cloudformation, amazon-web-services
Aws Toolkit Eclipse
AWS Toolkit for Eclipse – an open-source plugin for developing, deploying, and managing AWS applications.
Stars: ✭ 252 (-87.49%)
Mutual labels:  aws, aws-lambda, cloudformation, ec2
py-spotme
A CLI tool that creates AWS spot instances on the fly
Stars: ✭ 16 (-99.21%)
Mutual labels:  ec2, spot-instances, ec2-instance, ec2-spot
Aws Cf Templates
A cloudonaut.io project. Engineered by widdix.
Stars: ✭ 2,399 (+19.12%)
Mutual labels:  aws, cloudformation, amazon-web-services, infrastructure
Aws Scalable Big Blue Button Example
Demonstration of how to deploy a scalable video conference solution based on Big Blue Button
Stars: ✭ 29 (-98.56%)
Mutual labels:  aws, cloudformation, ec2, autoscaling
Cfn Create Or Update
Create or update CloudFormation stack also if no updates are to be performed.
Stars: ✭ 59 (-97.07%)
Mutual labels:  aws, cloudformation, amazon-web-services
Cloud Security Audit
A command line security audit tool for Amazon Web Services
Stars: ✭ 68 (-96.62%)
Mutual labels:  aws, amazon-web-services, ec2
Aws Automation
AWS automation scripts and lambda functions
Stars: ✭ 81 (-95.98%)
Mutual labels:  automation, aws, ec2
Content Lambda Boto3
Automating AWS with Lambda, Python, and Boto3
Stars: ✭ 91 (-95.48%)
Mutual labels:  aws, aws-lambda, ec2
Aws Power Tuner Ui
AWS Lambda Power Tuner UI is an open source project creating a deployable easy to use website built on a layered technology stack allowing you to optimize your Lambda functions for cost and/or performance in a data-driven way via an easy to use UI.
Stars: ✭ 52 (-97.42%)
Mutual labels:  aws, aws-lambda, cost
Serverless Es Logs
A Serverless plugin to transport logs to ElasticSearch
Stars: ✭ 51 (-97.47%)
Mutual labels:  aws, aws-lambda, amazon-web-services
Sceptre
Build better AWS infrastructure
Stars: ✭ 1,160 (-42.4%)
Mutual labels:  aws, cloudformation, infrastructure
Infra Personal
Terraform for setting up my personal infrastructure
Stars: ✭ 45 (-97.77%)
Mutual labels:  automation, aws, infrastructure
Awesome Aws
A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.
Stars: ✭ 9,895 (+391.31%)
Mutual labels:  aws, cloudformation, ec2
Docker Swarm
🐳🐳🐳 This repository is part of a blog series on Docker Swarm example using VirtualBox, OVH Openstack, Azure and Amazon Web Services AWS
Stars: ✭ 43 (-97.86%)
Mutual labels:  automation, aws, amazon-web-services
Perun
A command-line validation tool for AWS Cloud Formation that allows to conquer the cloud faster!
Stars: ✭ 82 (-95.93%)
Mutual labels:  aws, cloudformation, amazon-web-services
Aws Workflows On Github
Workflows for automation of AWS services setup from Github CI/CD
Stars: ✭ 95 (-95.28%)
Mutual labels:  automation, aws, ec2
Ecs Formation
Tool to build Docker cluster composition for Amazon EC2 Container Service(ECS)
Stars: ✭ 114 (-94.34%)
Mutual labels:  aws, ec2, autoscaling

AutoSpotting

BuildStatus GoReportCard CoverageStatus CodeClimate IssueCount ChatOnGitter Open Source Helpers Patreon

AutoSpotting is the leading open source spot market automation tool, optimized towards quick/easy/frictionless adoption of the EC2 spot market at any scale.

It's being used by thousands of users around the world, from companies of all shapes and sizes, in aggregate probably saving them in the tens of millions of dollars monthly as per our current estimations.

It is usually set up to monitor existing long-running AutoScaling groups, replacing their instances with Spot instances with minimal configuration changes.

Often all it needs is just tagging them with spot-enabled=true, (in some cases even that can be avoided), yielding the usual 70%-90% Spot cost savings but in a better integrated and easier to adopt way than other alternative tools and solutions.

It is particularly useful if you have a large footprint that you want to migrate to Spot quickly due to management pressure but with minimal effort and configuration changes.

Guiding principles

  • Customer-focused, designed to maximize user benefits and reduce adoption friction
  • Safe and secure, hosted in your AWS account and with minimal required set of IAM permissions
  • Auditable OSS code base developed in the open
  • Inexpensive, easy to install and supported builds offered through the AWS Marketplace
  • Simple, minimalist implementation

How does it work?

Once installed and enabled to run against existing on-demand AutoScaling groups, AutoSpotting gradually replaces their on-demand instances with cheaper spot instances that are at least as large and identically configured to the group's members, without changing the group launch configuration in any way. You can also keep running a configurable number of on-demand instances given as percentage or absolute number and it automatically fails over to on-demand in case of spot instance terminations.

Going forward, as well as on any new ASGs that match the expected tags, any new on-demand instances above the amount configured to be kept running will be immediately replaced with spot clones within seconds of being launched.

If this fails temporarily due to insufficient spot capacity, AutoSpotting will continuously attempt to replace them every few minutes until successful after spot capacity becomes available again.

When launching Spot instances, the compatible instance types are chosen by default using a the capacity-optimized-prioritized allocation strategy, which is given a list of instance types sorted by price. This configuration offers a good tradeoff between low cost and significantly reduced interruption rates. The lowest-price allocation strategy is still available as a configuration option.

This process can partly be seen in action below, you can click to expand the animation:

Workflow

A single installation can handle all enabled groups from an entire AWS account in parallel across all available AWS regions, but it can be restricted to fewer regions if desired in certain situations.

Your groups will then monitor and use these Spot instances just like they would do with your on-demand instances. They will automatically join their respective load balancer and start receiving traffic once passing the health checks, and the traffic would automatically be drained on termination.

What savings can I expect?

The savings it generates are in the 60-90% range usually seen when using spot instances, but they may vary depending on region and instance type.

What's under the hood?

The entire logic described above is implemented in a set of Lambda functions deployed using CloudFormation or Terraform stacks that can be installed and configured in just a few minutes.

The stack uses the minimal set of IAM permissions required for them to work and requires no admin-like cross-account permissions. The entire code base can be audited to see how these permissions are being used and even locked down further if your audit discovers any issues. This is not a SaaS, there's no component that calls home or reveals any details about your infrastructure.

The main Lambda function is written in the Go programming language and the code is compiled as a static binary. As of August 2021 this has been included in a Docker image used by the Lambda function.

The stack also consists of a few CloudWatch event triggers, that run the Lambda function periodically and whenever it needs to take action against the enabled groups. Between runs your group is entirely managed by AutoScaling (including any scaling policies you may have) and load balancer health checks, that can trigger instance launches or replacements using the original on-demand launch configuration.

Read here for more information and implementation details.

FAQs

Frequently asked questions about the project are answered in the FAQ, please read this first before asking for support.

If you have additional questions not covered there, they can be easily added to the source of the FAQ by editing in the browser and creating a pull request, and we'll answer them while reviewing the pull request.

Getting Started

Just like in the above animation, it's as easy as launching a CloudFormation (or Terraform) stack and setting the (configurable) spot-enabled tag on the AutoScaling groups where you want it enabled to true.

When installed from the AWS marketplace, all the required infrastructure and configuration will be created automatically, so you can get started as fast as possible. Otherwise you'll need to build it yourself as per the instructions available here.

For more detailed information on how to get started you can also read this document

Support

Marketplace subscribers can get support from here and any feature requests or issues raised via this communication channel will be prioritized.

Community support is available to OSS users on the gitter chat room, where the main authors and other users are likely to help you solve issues. This is offered on a best effort basis and under certain conditions, such as using the latest version of the software available on the main Github branch, without any code customizations and using the default configuration options.

If you need help for a large scale rollout or migrating from alternative tools/solutions get in touch on gitter.

Contributing

AutoSpotting is fully open source and developed in the open by a vibrant community of dozens of contributors.

If it helps you save any significant amount of money, it would be greatly appreciated if you could support further development on Github Sponsors or would purchase it from the AWS Marketplace.

Note: Non-trivial code should be submitted according to the contribution guidelines. Individuals and companies supporting the development of the open source code get free of charge support in getting their code merged upstream.

Official binaries

The source code is and will always be open source, so you can build and run it yourself, see how it works and even enhance it if you want.

As of August 2021, fully functional, stable official binaries are only offered through Docker images available on the AWS Marketplace.

Evaluation binaries built from trunk after each commit and meant to help the development process are also available from the Docker Hub but they expire after a month since compilation.

Proven referrals towards a subscription to the Marketplace will be compensated over Paypal with 50% of the first charge of the new subscriber. You can request them on gitter.

Subscriptions

A free low traffic mailing list is available on https://autospotting.io, where you can sign up for occasional emails related to the project, mainly related to major changes in the open source code, savings tips or announcements about other tools I've been working on.

Announcements on new Marketplace releases, including comprehensive release notes, upgrade instructions and tips to get the most out of AutoSpotting will be communicated in private to Patreon subscribers.

A Github sponsors subscription is also available for people interested in the ongoing development of AutoSpotting, with tiers covering anything from a non-strings attached donation, prioritization of feature requests, all the way to custom features development and maintenance of private customized forks.

Please get in touch on gitter if you have any questions about these offerings or if you have any other ideas on how I could provide additional value to my community.

Compiling and Installing

It is recommended to use the evaluation or stable Docker images available on the AWS marketplace, which are easy to install, support further development of the software and allow you to get some support.

But if you have some special needs that require some customizations or you don't want to rely on the author's infrastructure or contribute anything for longer term use of the software, you can always build and run your customized binaries that you maintain on your own, just keep in mind that those won't be supported in any way.

More details are available here

License

This software is distributed under the terms of the OSL-3.0 license.

The evaluation images available on Docker Hub are licensed under this proprietary license.

The AWS Marketplace offering is made available under the standard AWS Marketplace EULA.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].