ovh / Celery Director
Licence: bsd-3-clause
Simple and rapid framework to build workflows with Celery
Stars: ✭ 263
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Celery Director
change
A simple tool that automates generating and updating a changelog
Stars: ✭ 47 (-82.13%)
Mutual labels: workflow
shrek-springboot-vue
springboot+vue+shiro 项目,包括权限,代码生成,,工作流等 demo:地址
Stars: ✭ 49 (-81.37%)
Mutual labels: workflow
tukio
Tukio is an event based workflow generator library
Stars: ✭ 27 (-89.73%)
Mutual labels: workflow
cwlexec
A new open source tool to run CWL workflows on LSF
Stars: ✭ 34 (-87.07%)
Mutual labels: workflow
snorkeling
Extracting biomedical relationships from literature with Snorkel 🏊
Stars: ✭ 56 (-78.71%)
Mutual labels: workflow
Workflow Kotlin
A Swift and Kotlin library for making composable state machines, and UIs driven by those state machines.
Stars: ✭ 255 (-3.04%)
Mutual labels: workflow
cli-property-manager
Use this Property Manager CLI to automate Akamai property changes and deployments across many environments.
Stars: ✭ 22 (-91.63%)
Mutual labels: workflow
prosto
Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby
Stars: ✭ 54 (-79.47%)
Mutual labels: workflow
ansible-roles
Library of Ansible plugins and roles for deploying various services.
Stars: ✭ 14 (-94.68%)
Mutual labels: celery
full-stack-flask-couchdb
Full stack, modern web application generator. Using Flask, CouchDB as database, Docker, Swagger, automatic HTTPS and more.
Stars: ✭ 28 (-89.35%)
Mutual labels: celery
uchuva
A scientific web portal that allows users to create and submit workflows to HTCondor (Dagman), Slurm, OpenLava (LSF), Torque (PBS)
Stars: ✭ 17 (-93.54%)
Mutual labels: workflow
genesis-sample-task-runner
Gulp workflow for Auto prefixing, Sass compiling, CSS minification, automatic browser reloading and more
Stars: ✭ 23 (-91.25%)
Mutual labels: workflow
celery-priority-tasking
This is a prototype to schedule jobs in the backend based on some priority using Rabbitmq and Celery.
Stars: ✭ 28 (-89.35%)
Mutual labels: celery
sapporo
A standard implementation conforming to the Global Alliance for Genomics and Health (GA4GH) Workflow Execution Service (WES) API specification and a web application for managing and executing those WES services.
Stars: ✭ 17 (-93.54%)
Mutual labels: workflow
Polyaxon
Machine Learning Platform for Kubernetes (MLOps tools for experimentation and automation)
Stars: ✭ 2,966 (+1027.76%)
Mutual labels: workflow
Docker Camunda Bpm Platform
Docker images for the camunda BPM platform
Stars: ✭ 259 (-1.52%)
Mutual labels: workflow
ejyy
「e家宜业」是一整套开源智慧物业解决方案,基于nodejs、typescript、koa、vue开发,包含web中台、业主小程序、员工小程序、公众号、物联网应用等,涵盖业主服务、物业运营、智能物联、数据统计等主要业务。
Stars: ✭ 561 (+113.31%)
Mutual labels: workflow
Director is a simple and rapid framework used to manage tasks and build workflows using Celery.
The objective is to make Celery easier to use by providing :
- a WebUI to track the tasks states,
- an API and a CLI to manage and execute the workflows,
- a YAML syntax used to combine tasks into workflows,
- the ability to periodically launch a whole workflow,
- and many others.
See how to use Director with the quickstart and guides in the documentation.
Installation
Install the latest version of Director with pip (requires Python 3.6
at least):
pip install celery-director
Usage
Write your code in Python
# tasks/orders.py
from director import task
from .utils import Order, Mail
@task(name="ORDER_PRODUCT")
def order_product(*args, **kwargs):
order = Order(
user=kwargs["payload"]["user"],
product=kwargs["payload"]["product"]
).save()
return {"id": order.id}
@task(name="SEND_MAIL")
def send_mail(*args, **kwargs):
order_id = args[0]["id"]
mail = Mail(
title=f"Your order #{order_id} has been received",
user=kwargs["payload"]["user"]
)
mail.send()
Build your workflows in YAML
# workflows.yml
product.ORDER:
tasks:
- ORDER_PRODUCT
- SEND_MAIL
Run it
You can simply test your workflow in local :
$ director workflow run product.ORDER '{"user": 1234, "product": 1000}'
And run it in production using the director API :
$ curl --header "Content-Type: application/json" \
--request POST \
--data '{"project": "product", "name": "ORDER", "payload": {"user": 1234, "product": 1000}}' \
http://localhost:8000/api/workflows
Read the documentation to try the quickstart and see advanced usages of Celery Director.
Project layout
.env # The configuration file.
workflows.yml # The workflows definition.
tasks/
example.py # A file containing some tasks.
... # Other files containing other tasks.
Commands
-
director init [path]
- Create a new project. -
director celery [worker|beat|flower]
- Start Celery daemons. -
director webserver
- Start the webserver. -
director workflow [list|show|run]
- Manage your project workflows.
License
See https://github.com/ovh/celery-director/blob/master/LICENSE
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].