All Projects → getpopper → Popper

getpopper / Popper

Licence: mit
Container-native task automation engine.

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Popper

Terrahub
Terraform Automation and Orchestration Tool (Open Source)
Stars: ✭ 148 (-31.48%)
Mutual labels:  devops, containers
Docker Workshop
Docker workshop
Stars: ✭ 174 (-19.44%)
Mutual labels:  devops, containers
Knctl
Knative CLI
Stars: ✭ 163 (-24.54%)
Mutual labels:  cli, containers
Ops
Tools&scripts for daily devops.
Stars: ✭ 128 (-40.74%)
Mutual labels:  devops, containers
Netns
Runc hook (OCI compatible) for setting up default bridge networking for containers.
Stars: ✭ 197 (-8.8%)
Mutual labels:  cli, containers
Arkade
Open Source Kubernetes Marketplace
Stars: ✭ 2,343 (+984.72%)
Mutual labels:  cli, devops
Choerodon
Open Source Multi-Cloud Integrated Platform
Stars: ✭ 2,149 (+894.91%)
Mutual labels:  devops, containers
Dksnap
Docker Snapshots for Development and Test Data
Stars: ✭ 122 (-43.52%)
Mutual labels:  cli, containers
Minishift
Run OpenShift 3.x locally
Stars: ✭ 2,246 (+939.81%)
Mutual labels:  devops, containers
Mbt
The most flexible build tool for monorepo
Stars: ✭ 184 (-14.81%)
Mutual labels:  cli, devops
Linuxdeploy Cli
Linux Deploy CLI
Stars: ✭ 127 (-41.2%)
Mutual labels:  cli, containers
Ahoy
Create self-documenting cli programs from YAML files. Easily wrap bash, grunt, npm, docker, (anything) to standardize your processes and make the lives of the people working on your project better.
Stars: ✭ 201 (-6.94%)
Mutual labels:  cli, devops
Ckube
A cli to simplify working with kubectl for some common workflows
Stars: ✭ 127 (-41.2%)
Mutual labels:  cli, containers
Operos
Linux-based operating system that brings hyperscaler-grade infrastructure automation to organizations of all sizes
Stars: ✭ 144 (-33.33%)
Mutual labels:  devops, containers
Training
Container, Monitoring & Logging, Cloud & DevOps Tutorials and Labs
Stars: ✭ 121 (-43.98%)
Mutual labels:  devops, containers
Devspace
DevSpace - The Fastest Developer Tool for Kubernetes ⚡ Automate your deployment workflow with DevSpace and develop software directly inside Kubernetes.
Stars: ✭ 2,559 (+1084.72%)
Mutual labels:  cli, devops
Carvel Kwt
Kubernetes Workstation Tools CLI
Stars: ✭ 119 (-44.91%)
Mutual labels:  cli, devops
Pipedream
Connect APIs, remarkably fast. Free for developers.
Stars: ✭ 2,068 (+857.41%)
Mutual labels:  cli, workflows
Mcw Cloud Native Applications
MCW Cloud-native applications
Stars: ✭ 184 (-14.81%)
Mutual labels:  devops, containers
Iamy
A cli tool for importing and exporting AWS IAM configuration to YAML files
Stars: ✭ 200 (-7.41%)
Mutual labels:  cli, devops

Popper Popper

Downloads Build Status codecov Join the chat at https://gitter.im/systemslab/popper slack CROSS

Popper is a tool for defining and executing container-native testing workflows in Docker. With Popper, you define a workflow in a YAML file, and then execute it with a single command. A workflow file looks like this:

steps:
- id: dev-init
  uses: docker://rikorose/gcc-cmake:gcc-9
  runs: [sh, -uexc]
  args:
  - |
    rm -rf build/
    cmake -DCMAKE_BUILD_TYPE=Release -S . -B build

- id: build
  uses: docker://rikorose/gcc-cmake:gcc-9
  runs: [cmake, --build, build/, --parallel, '4']

- id: test
  uses: docker://rikorose/gcc-cmake:gcc-9
  dir: /workspace/build/
  runs: [ctest]

Assuming the above is stored in a ci.yml file in the root of your project folder, this entire workflow gets executed by running:

popper run -f ci.yml

Running a single step:

popper run -f ci.yml build

Starting a shell inside the build step container:

popper run -f ci.yml build

Running on another engine (Podman):

popper run -f ci.yml -e podman build

See the examples/ folder for examples for tests for other languages, as well as other types of tests (integration, regresssion, etc.).

Installation

To install or upgrade Popper, run the following in your terminal:

curl -sSfL https://raw.githubusercontent.com/getpopper/popper/master/install.sh | sh

Docker is required to run Popper and the installer will abort if the docker command cannot be invoked from your shell. For other installation options, including installing for use with the other supported engines (Singularity and Podman), or for setting up a developing environment for Popper, read the complete installation instructions.

Once installed, you can get an overview and list of available commands:

popper help

Read the Quickstart Guide to learn the basics of how to use Popper. Or browse the Official documentation.

Features

  • Lightweight workflow and task automation syntax. Defining a list of steps is as simple as writing file in a lightweight YAML syntax and invoking popper run (see demo above). If you're familiar with Docker Compose, you can think of Popper as Compose but for end-to-end tasks instead of services.

  • An abstraction over container runtimes. In addition to Docker, Popper can seamlessly execute workflows in other runtimes by interacting with distinct container engines. Popper currently supports Docker, Singularity and Podman.

  • An abstraction over CI services. Define a pipeline once and then instruct Popper to generate configuration files for distinct CI services, allowing users to run the exact same workflows they run locally on Travis, Jenkins, Gitlab, Circle and others. See the examples/ folder for examples on how to automate CI tasks for multiple projects (Go, C++, Node, etc.).

  • An abstraction over resource managers. Popper can also execute workflows on a variety of resource managers and schedulers such as Kubernetes and SLURM, without requiring any modifications to a workflow YAML file. We currently support SLURM and are working on adding support for Kubernetes.

  • Aid in workflow development. Aid in the implementation and debugging of workflows, and provide with an extensive list of example workflows that can serve as a starting point.

What Problem Does Popper Solve?

Popper is a container-native workflow execution and task automation engine. In practice, when we work following the container-native paradigm, we end up interactively executing multiple Docker commands (docker pull, docker build, docker run, etc.) so that we can build containers, compile code, test applications, deploy software, among others. Keeping track of which docker commands we have executed, in which order, and which flags were passed to each, can quickly become unmanageable, difficult to document (think of outdated README instructions) and error prone.

On top of this, having the same workflow work in other environments (CI, K8S, etc.) is time-consuming and defeats the purpose of using containers (portability). The goal of Popper is to bring order to this chaotic scenario by providing a framework for clearly and explicitly defining container-native tasks. You can think of Popper as tool for wrapping all these manual tasks in a lightweight, machine-readable, self-documented format (YAML).

While this sounds simple at first, it has significant implications: results in time-savings, improves communication and in general unifies development, testing and deployment workflows. As a developer or user of "Popperized" container-native projects, you only need to learn one tool, and leave the execution details to Popper, whether is to build and tests applications locally, on a remote CI server or a Kubernetes cluster.

Contributing

Anyone is welcome to contribute to Popper! To get started, take a look at our contributing guidelines, then dive in with our list of good first issues.

Participation Guidelines

Popper adheres to the code of conduct posted in this repository. By participating or contributing to Popper, you're expected to uphold this code. If you encounter unacceptable behavior, please immediately email us.

How to Cite Popper

Ivo Jimenez, Michael Sevilla, Noah Watkins, Carlos Maltzahn, Jay Lofstead, Kathryn Mohror, Andrea Arpaci-Dusseau and Remzi Arpaci-Dusseau. The Popper Convention: Making Reproducible Systems Evaluation Practical. In 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 1561–70, 2017. (https://doi.org/10.1109/IPDPSW.2017.157)

PDF for a pre-print version available here. For BibTeX, click here.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].