All Projects → cpg1111 → maestro

cpg1111 / maestro

Licence: Apache-2.0 license
Faster CI/CD for multi-artifact projects

Programming Languages

go
31211 projects - #10 most used programming language
Makefile
30231 projects
shell
77523 projects

Projects that are alternatives of or similar to maestro

Dyn365 Ce Vsts Tasks
VSTS Extension for Dynamics 365 Customer Engagement
Stars: ✭ 94 (+623.08%)
Mutual labels:  continuous-integration, continuous-delivery, build-automation, continuous-deployment
Drone
Drone is a Container-Native, Continuous Delivery Platform
Stars: ✭ 24,287 (+186723.08%)
Mutual labels:  continuous-integration, continuous-delivery, build-pipelines, build-automation
Dyn365 Ce Devops
DevOps for Dynamics 365 Customer Engagement (CE) is becoming a popular topic. The goal of this project is to help Dynamics 365 CE solution builders understand and accelerate their implementation of DevOps practices with Dynamics CE and VSTS.
Stars: ✭ 82 (+530.77%)
Mutual labels:  continuous-integration, continuous-delivery, build-automation, continuous-deployment
Flubucore
A cross platform build and deployment automation system for building projects and executing deployment scripts using C# code.
Stars: ✭ 695 (+5246.15%)
Mutual labels:  builder, continuous-integration, build-automation, continuous-deployment
Xrm Ci Framework
xRM CI Framework provides you with the tools automate the build and deployment of your CRM Solution. Using the framework to implement a fully automated DevOps pipeline will allow you to deploy more frequently with added consistency and quality.
Stars: ✭ 172 (+1223.08%)
Mutual labels:  continuous-integration, continuous-delivery, build-automation, continuous-deployment
Rok8s Scripts
Opinionated scripts for managing application deployment lifecycle in Kubernetes
Stars: ✭ 248 (+1807.69%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Bbrun
Run Bitbucket Pipelines locally
Stars: ✭ 127 (+876.92%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Sshdeploy
A command-line tool that enables quick build and run deployments over SSH.
Stars: ✭ 131 (+907.69%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Cdeasy
Continuous Delivery made Easy ;)
Stars: ✭ 143 (+1000%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Build
Netlify Build runs the build command, Build Plugins and bundles Netlify Functions.
Stars: ✭ 135 (+938.46%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Rocket
Automated software delivery as fast and easy as possible 🚀
Stars: ✭ 217 (+1569.23%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Pipelines
Build pipelines for automation, deployment, testing...
Stars: ✭ 105 (+707.69%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Orkestra
Functional DevOps with Scala and Kubernetes
Stars: ✭ 102 (+684.62%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
badwolf
Docker based continuous integration, continuous deployment and code lint review system for BitBucket
Stars: ✭ 88 (+576.92%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Github Slug Action
GitHub Action to expose slug value of GitHub environment variables inside your GitHub workflow
Stars: ✭ 96 (+638.46%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Nevergreen
🐤 A build monitor with attitude
Stars: ✭ 170 (+1207.69%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Syncd
syncd是一款开源的代码部署工具,它具有简单、高效、易用等特点,可以提高团队的工作效率.
Stars: ✭ 2,065 (+15784.62%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
generator-nullfactory-xrm
Yeoman generator for Dynamics 365 Solutions. It generates a project structure that facilitates the quick creation builds and automated release strategies with minimal effort.
Stars: ✭ 15 (+15.38%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Docker For All
Docker applied in development, devops, testing, product management etc.
Stars: ✭ 88 (+576.92%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment
Ecs Nginx Proxy
Reverse proxy for AWS ECS. Lets you address your docker containers by sub domain.
Stars: ✭ 93 (+615.38%)
Mutual labels:  continuous-integration, continuous-delivery, continuous-deployment

maestro

Deploy only what's changed for your multiple services in mono-repos

Go Report Card Code Climate

How it Works

Maestro pulls a given repository then builds a dependency graph based on a given config file. Once the dependency graph is created, Maestro diffs against a given previous commit, with the pathspec being the root directory for each artifact. Maestro then flags only the changed artifacts for the pipeline, which are then ran concurrently per teir of dependencies, therefore siblings in the graph will build, test and deploy concurrently, but parents and children dependencies will always be built in the correct order.

Used with Maestrod you can have a build manager that integrates with Github webhooks and runs on Kubernetes or a single-host Docker setup.

For more details see this talk: https://www.youtube.com/watch?v=dGM8mYj8nz4&feature=youtu.be

Install

    git clone [email protected]:cpg1111/maestro
    cd maestro
    make docker

or

    docker pull cpg1111/maestro:<release>

Test

    go test ./...

Run

    Usage of maestro:
      --branch string
            Git branch to checkout for project (default "master")
      --clone-path string
            Local path to clone repo to defaults to PWD (default "./")
      --config string
            Path to the config for maestro to use (default "./conf.toml")
      --deploy
            Whether or not to deploy this build                             # defaults to false
      --prev-commit string
            Previous commit to compare to                                   # required
    maestro --branch <git branch to build> --conf <project config> --prev-commit <commit to compare to> --deploy <whether to deploy build or not> --clone-path <tmp path to clone repo into>

or

    docker run -v <path of conf>:<target> -v <path to ssh credentials if using ssh for git>:<target> maestro --branch <git branch to build> --conf <project config> --prev-commit <commit to compare to> --deploy <whether to deploy build or not> --clone-path <tmp path to clone repo into>

Example Config

    [Environment] # Environment will run before anything else, ExecSync will execute commands in the array synchronously, while exec will execute them concurrently
    Env=["node_env:test", "docker_tls_verify:1"] # set environment variables all lowercase keys and keys are separated from values with ':'
    ExecSync=["apt-get install -y docker node go"]
    Exec=["docker pull someOrg/logger", "docker pull someOrg/models", "docker pull someOrg/auth", "docker pull someOrg/client"]

    [Project]
    RepoURL="[email protected]:someOrg/someRepo.git"
    CloneCMD="git clone"
    AuthType="SSH"
    SSHPrivKeyPath="~/.ssh/id_rsa"
    SSHPubKeyPath="~/.ssh/id_rsa.pub"
    Username="git" # github's ssh user is git, but this can vary
    Password=""
    PromptForPWD=false # when requiring a password, you prompt for a password

    [[Services]] # Services are either actual services or libraries / packages / separately compiled objects
    Name="logger"
    Tag="0.1.0"
    TagType="git"
    Path="./src/logger"
    BuildCMD=["docker build -t logger ."] # '.' is relative to the given path field of the service
    TestCMD=["go test ./..."]
    CheckCMD=["bash -c 'docker images -a | grep logger'"]
    CreateCMD=[
        "docker tag logger <org>/logger:{{.Curr}}",
        "docker push <org>/logger:{{.Curr}}"
    ]
    UpdateCMD=[
        "docker tag logger <org>/logger:{{.Curr}}",  # {{.Curr}} will template the current commit hash into the command
        "docker push <org>/logger:{{.Curr}}"
    ]
    DependsOn=[]

    [[Services]]
    Name="models"
    Tag="0.1.0"
    TagType="git"
    Path="./src/models"
    BuildCMD=["docker build -t models ."]
    TestCMD=["go test ./..."]
    CheckCMD=["bash -c 'docker images -a | grep models'"]
    CreateCMD=[
        "docker tag models <org>/models:{{.Curr}}",
        "docker push <org>/models:{{.Curr}}"
    ]
    UpdateCMD=[
        "docker tag models <org>/models:{{.Curr}}",
        "docker push <org>/models:{{.Curr}}"
    ]
    DependsOn=["logger"] # Assume Dockerfile contains FROM logger

    [[Services]]
    Name="auth"
    Tag="0.1.0"
    TagType="git"
    Path="./src/auth"
    BuildCMD=["docker build -t auth ."]
    TestCMD=["go test ./..."]
    CheckCMD=["bash -c 'docker ps -a | grep auth'"]
    CreateCMD=[
        "docker tag auth <org>/auth:{{.Curr}}",
        "docker tag auth <org>/auth:{{.Curr}}",
        "docker run --rm -d <org>/auth:{{.Curr}}"
    ]
    UpdateCMD=[
        "docker tag auth <org>/auth:{{.Curr}}",
        "docker tag auth <org>/auth:{{.Curr}}",
        "docker run --rm -d <org>/auth:{{.Curr}}"
    ]
    DependsOn=["models"] # Assume Dockerfile contains FROM models

    [[Services]]
    Name="client"
    Tag="0.1.0"
    TagType="git"
    Path="./src/client"
    BuildCMD=["docker build -t client ."]
    TestCMD=["npm test"]
    CheckCMD=["bash -c 'docker ps -a | grep client'"]
    CreateCMD=[
        "docker tag client <org>/client:{{.Curr}}",
        "docker push <org>/client:{{.Curr}}",
        "docker run --rm -d <org>/client:{{.Curr}}"
    ]
    UpdateCMD=[
        "docker tag client <org>/client:{{.Curr}}",
        "docker push <org>/client:{{.Curr}}",
        "docker run --rm -d <org>/client:{{.Curr}}"
    ]
    DependsOn=[]

    [CleanUp]
    AdditionalCMDs=["docker inspect auth", "docker export -o ./dist/auth.tgz auth"] # Will execute synchronously
    InDaemon=false # COMING SOON for maestrod
        [[CleanUp.Artifacts]] # Artifacts are saved concurrently
        RuntimeFilePath="./dist/auth.tgz"
        SaveFilePath="/opt/data/auth.tgz"

Roadmap

  • Allow larger log buffers
  • More possible dependency structures
  • Encrypted Environment variable values
  • Log Versbosity control
  • Debug with bash session

Daemon

See this (https://github.com/cpg1111/maestrod) for a manager daemon for handling git push hooks and multiple concurrent builds and repos.

Warning

Some dependency structures are not supported yet. Best to refer to immediate dependencies only and avoid circlular references.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].