All Projects → rq → Rq

rq / Rq

Licence: other
Simple job queues for Python

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Rq

rqmonitor
Flask based more dynamic and actionable frontend dashboard for monitoring Redis Queue 👩🏿‍💻 http://python-rq.org
Stars: ✭ 152 (-98.12%)
Mutual labels:  workers, job-queue, task-queue, rq
Django Rq
A simple app that provides django integration for RQ (Redis Queue)
Stars: ✭ 1,361 (-83.12%)
Mutual labels:  background-jobs, job-queue, redis, task-queue
Qutee
PHP Background Jobs (Tasks) Manager
Stars: ✭ 63 (-99.22%)
Mutual labels:  workers, background-jobs, job-queue, redis
celery.node
Celery task queue client/worker for nodejs
Stars: ✭ 164 (-97.97%)
Mutual labels:  workers, background-jobs, job-queue, task-queue
Flower
Real-time monitor and web admin for Celery distributed task queue
Stars: ✭ 5,036 (-37.56%)
Mutual labels:  workers, redis, task-queue
Node Celery
Celery client for Node.js
Stars: ✭ 648 (-91.97%)
Mutual labels:  task, background-jobs, redis
Fennel
A task queue library for Python and Redis
Stars: ✭ 24 (-99.7%)
Mutual labels:  async, task, redis
Kiq
📮 Robust job queue powered by GenStage and Redis
Stars: ✭ 49 (-99.39%)
Mutual labels:  workers, background-jobs, redis
Asynq
Asynq: simple, reliable, and efficient distributed task queue in Go
Stars: ✭ 934 (-88.42%)
Mutual labels:  background-jobs, redis, task-queue
Que
Simple Job Processing in Elixir with Mnesia ⚡️
Stars: ✭ 612 (-92.41%)
Mutual labels:  background-jobs, job-queue
Verk
A job processing system that just verks! 🧛‍
Stars: ✭ 666 (-91.74%)
Mutual labels:  workers, redis
Asyncawaitbestpractices
Extensions for System.Threading.Tasks.Task and System.Threading.Tasks.ValueTask
Stars: ✭ 693 (-91.41%)
Mutual labels:  async, task
Machinery
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Stars: ✭ 5,821 (-27.82%)
Mutual labels:  task, redis
Honeydew
Job Queue for Elixir. Clustered or Local. Straight BEAM. Optional Ecto. 💪🍈
Stars: ✭ 670 (-91.69%)
Mutual labels:  workers, job-queue
Taskq
Golang asynchronous task/job queue with Redis, SQS, IronMQ, and in-memory backends
Stars: ✭ 555 (-93.12%)
Mutual labels:  redis, task-queue
Arq
Fast job queuing and RPC in python with asyncio and redis.
Stars: ✭ 695 (-91.38%)
Mutual labels:  async, redis
Gush
Fast and distributed workflow runner using ActiveJob and Redis
Stars: ✭ 894 (-88.92%)
Mutual labels:  workers, redis
Then
🎬 Tame async code with battle-tested promises
Stars: ✭ 908 (-88.74%)
Mutual labels:  async, task
Sidekiq Job Php
Push and schedule jobs to Sidekiq from PHP
Stars: ✭ 34 (-99.58%)
Mutual labels:  workers, redis
Yii2 Queue
Yii2 Queue Extension. Supports DB, Redis, RabbitMQ, Beanstalk and Gearman
Stars: ✭ 977 (-87.89%)
Mutual labels:  async, redis

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

RQ requires Redis >= 3.0.0.

Build status PyPI Coverage

Full documentation can be found here.

Support RQ

If you find RQ useful, please consider supporting this project via Tidelift.

Getting started

First, run a Redis server, of course:

$ redis-server

To put jobs on queues, you don't have to do anything special, just define your typically lengthy or blocking function:

import requests

def count_words_at_url(url):
    """Just an example function that's called async."""
    resp = requests.get(url)
    return len(resp.text.split())

You do use the excellent requests package, don't you?

Then, create an RQ queue:

from redis import Redis
from rq import Queue

queue = Queue(connection=Redis())

And enqueue the function call:

from my_module import count_words_at_url
job = queue.enqueue(count_words_at_url, 'http://nvie.com')

Scheduling jobs are also similarly easy:

# Schedule job to run at 9:15, October 10th
job = queue.enqueue_at(datetime(2019, 10, 10, 9, 15), say_hello)

# Schedule job to run in 10 seconds
job = queue.enqueue_in(timedelta(seconds=10), say_hello)

Retrying failed jobs is also supported:

from rq import Retry

# Retry up to 3 times, failed job will be requeued immediately
queue.enqueue(say_hello, retry=Retry(max=3))

# Retry up to 3 times, with configurable intervals between retries
queue.enqueue(say_hello, retry=Retry(max=3, interval=[10, 30, 60]))

For a more complete example, refer to the docs. But this is the essence.

The worker

To start executing enqueued function calls in the background, start a worker from your project's directory:

$ rq worker --with-scheduler
*** Listening for work on default
Got count_words_at_url('http://nvie.com') from default
Job result = 818
*** Listening for work on default

That's about it.

Installation

Simply use the following command to install the latest released version:

pip install rq

If you want the cutting edge version (that may well be broken), use this:

pip install git+https://github.com/rq/rq.git@master#egg=rq

Related Projects

Check out these below repos which might be useful in your rq based project.

Project history

This project has been inspired by the good parts of Celery, Resque and this snippet, and has been created as a lightweight alternative to the heaviness of Celery or other AMQP-based queueing implementations.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].