All Projects → fujiwara → Stretcher

fujiwara / Stretcher

Licence: mit
Deployment tool with consul/serf event notification.

Programming Languages

go
31211 projects - #10 most used programming language
golang
3204 projects

Projects that are alternatives of or similar to Stretcher

tuggle
Distributed file mirroring proxy in Consul cluster
Stars: ✭ 16 (-93.1%)
Mutual labels:  deployment, consul
Cqtdeployer
This project is used to deploy applications written using QML, qt or other С / С++ frameworks.
Stars: ✭ 225 (-3.02%)
Mutual labels:  deployment
Obevo
Obevo is a database deployment tool that handles enterprise scale schemas and complexity
Stars: ✭ 192 (-17.24%)
Mutual labels:  deployment
Springcloudexamples
Spring Cloud 学习教程
Stars: ✭ 208 (-10.34%)
Mutual labels:  consul
Kubefate
Manage federated learning workload using cloud native technologies.
Stars: ✭ 195 (-15.95%)
Mutual labels:  deployment
Dbtester
Distributed database benchmark tester
Stars: ✭ 214 (-7.76%)
Mutual labels:  consul
Panteras
PanteraS - PaaS - Platform as a Service in a box
Stars: ✭ 189 (-18.53%)
Mutual labels:  consul
Rollback
Ansible role to rollback scripting applications like PHP, Python, Ruby, etc. in a capistrano style
Stars: ✭ 230 (-0.86%)
Mutual labels:  deployment
Spring Dubbo Service
微服务 spring dubbo项目:dubbo rpc;druid数据源连接池;mybatis配置集成,多数据源;jmx监控MBean;定时任务;aop;ftp;测试;Metrics监控;参数验证;跨域处理;shiro权限控制;consul服务注册,发现;redis分布式锁;SPI服务机制;cat监控;netty服务代理;websocket;disconf;mongodb集成;rest;docker;fescar
Stars: ✭ 224 (-3.45%)
Mutual labels:  consul
Concourse Up
Deprecated - used Control Tower instead
Stars: ✭ 207 (-10.78%)
Mutual labels:  deployment
Consul Backinator
Command line Consul backup and restore utility supporting KVs, ACLs and Queries
Stars: ✭ 206 (-11.21%)
Mutual labels:  consul
Ansible Rails
Ruby on Rails deployment using Ansible - with Lets Encrypt, Sidekiq, PostgreSQL, nginx & puma
Stars: ✭ 199 (-14.22%)
Mutual labels:  deployment
Rocket
Automated software delivery as fast and easy as possible 🚀
Stars: ✭ 217 (-6.47%)
Mutual labels:  deployment
Firecamp
Serverless Platform for the stateful services
Stars: ✭ 194 (-16.38%)
Mutual labels:  consul
Deep Learning In Production
Develop production ready deep learning code, deploy it and scale it
Stars: ✭ 216 (-6.9%)
Mutual labels:  deployment
Seldon Core
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
Stars: ✭ 2,815 (+1113.36%)
Mutual labels:  deployment
Remco
remco is a lightweight configuration management tool
Stars: ✭ 200 (-13.79%)
Mutual labels:  consul
Core Grpc
C# Grpc驱动封装,基于Consul实现服务注册服务发现,支持dotnetcore / framework,可快速实现基于Grpc的微服务,内部有完整案例,包含服务端Server 客户端 Client,core+grpc, netcore+grpc, dotnetcore+grpc
Stars: ✭ 209 (-9.91%)
Mutual labels:  consul
Mason
Cross platform package manager for C/C++ apps
Stars: ✭ 230 (-0.86%)
Mutual labels:  deployment
K8s Diagrams
A collection of kubernetes-related diagrams
Stars: ✭ 227 (-2.16%)
Mutual labels:  deployment

Stretcher

A deployment tool with Consul / Serf event.

Example manifest

src: s3://example.com/app.tar.gz
checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93
dest: /home/stretcher/app
commands:
  pre:
    - echo 'staring deploy'
  post:
    - echo 'deploy done'
  success:
    - echo 'deploy success'
  failure:
    - echo 'deploy failed!!'
    - cat >> /path/to/failure.log
excludes:
  - "*.pid"
  - "*.socket"

Run

stretcher agent

$ stretcher -h
Usage of stretcher:
  -max-bandwidth string
        max bandwidth for download src archives (Bytes/sec)
  -random-delay float
        sleep [0,random-delay) sec on start
  -retry int
        retry count for download src archives
  -retry-wait int
        wait for retry download src archives (sec) (default 3)
  -rsync-verbose string
        rsync verbose option (default -v)
  -timeout int
        timeout for download src archives (sec)
  -v    show version
  -version
        show version

with Consul

A stretcher agent is designed as running under "consul watch" and will be kicked by Consul event.

$ consul watch -type event -name deploy /path/to/stretcher
  • -name: your deployment identity name.

with Serf

A stretcher agent can be running as Serf event handler.

$ serf agent -event-handler="user:deploy=/path/to/stretcher >> /path/to/stretcher.log 2>&1"

Load AWS credentials

When you specify a S3 URL in the manifest, requires AWS regions and credentials configuration as the same of aws-sdk-go.

for regions,

  1. AWS_REGION or AWS_DEFAULT_REGION environment variable.

for credentials,

  1. Environment variables.
  2. Shared credentials file.
  3. If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.

See also Configuring the AWS SDK for Go.

Load GCP credentials

When you specify a GS(Google Cloud Storage) URL in the manifest, requires a GCP credential setting one of below.

  • ServiceAccount
    • requires GOOGLE_APPLICATION_CREDENTIALS=[PATH] environment variable.
    • Replace [PATH] with the file path of the JSON file that contains your service account key.
  • DefaultAccount
    • If the environment variable isn't set, load the default service account that Compute Engine provide, for applications that run on those services.

Deployment process

Preparing

This process is not included in a stretcher agent.

  1. Create a tar(or tar.gz) archive for deployment.
  2. Upload the archive file to remote server (S3 or HTTP(S)).
  3. Create a manifest file (YAML) and upload it to remote server.

Executing with Consul

Create a consul event to kick stretcher agents.

$ consul event -name [event_name] [manifest_url]
$ consul event -name deploy s3://example.com/deploy-20141117-112233.yml
  • -name: consul event name (specified by consul watch -name)

Executing with Serf

Create a serf user event to kick stretcher agents.

$ serf event [event_name] [manifest_url]
$ serf event deploy s3://example.com/deploy-20141117-112233.yml
  • event_name: user event name (specified by serf event handler).

Executing as command

Stretcher can read a manifest URL from stdin simply.

$ echo s3://example.com/deploy-20141117-112233.yml | stretcher

You can execute stretcher via ssh or any other methods.

Deployment process

A stretcher agent executes a following process.

  1. Receive a manifest URL as Consul/Serf event's payload.
  2. Get a manifest.
  3. Get src URL and store it to a temporary file, and Check checksum.
  4. Invoke pre commands.
  5. Extract src archive to a temporary directory.
  6. Sync files from extracted archive to dest directory.
  • use rsync -a --delete or mv
  • sync strategy is switched by sync_strategy
  1. Invoke post commands.
  • Invoke success commands when the deployment process succeeded.
  • Invoke failure commands when the deployment process failed.

Manifest spec

src

Source archive URL.

  • URL schema: 's3', 'http', 'file'
  • Format: 'tar', 'tar.gz'
src: http://example.com/src/archive.tar.gz

checksum

Checksum of source archive.

  • Type: 'md5', 'sha1', 'sha256', 'sha512'
checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93

dest

Destination directory.

dest: /home/stretcher/app

dest_mode

Destination directory mode. Default: 0755

dest_mode: 0711

Destination directory mode will be set as...

  1. src archive includes . => same as . in the archive.
  2. src archive does not include . => dest_mode

commands

  • pre: Commands which will be invoked at before src archive extracted.
  • post: Commands which will be invoked at after dest directory synced.
  • success: Commands which will be invoked at deployment process is succeeded.
  • failure: Commands which will be invoked at deployment process is failed.
commands:
  pre:
    - echo 'staring deploy'
  post:
    - echo 'deploy done'
  success:
    - echo 'deploy success'
  failure:
    - echo 'deploy failed!!'
    - cat >> /path/to/failure.log

stretcher agent logs will be passed to STDIN of success and failure commands.

excludes

Pass to rsync --exclude arguments.

excludes:
  - "*.pid"
  - "*.socket"

exclude_from

Pass to rsync --exclude-from arguments. The file must be included in src archive.

exclude_from: exclude.list

sync_strategy

A strategy for syncing src extracted directory to dest directory.

  • rsync: Default
    • Use rsync(1) command with option -av --delete
  • mv
    • Use os.Rename() of Golang.
    • Deployment will be failed if dest directory is already exists.

Requirements

  • tar
  • rsync

tar and rsync must be exist in PATH environment.

If you use stretcher under systemd, You can see unfinished stdout with journald. You should add RateLimitBurst=0 into /etc/systemd/journald.conf for getting stdout completely.

Commands execution only mode

If src is not defined in a manifest, Stretcher runs pre/post and success/failure commands simply.

LICENSE

The MIT License (MIT)

Copyright (c) 2014 FUJIWARA Shunichiro / (c) 2014 KAYAC Inc.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].