All Projects → olivere → jobqueue

olivere / jobqueue

Licence: MIT license
Jobqueue manages running and scheduling jobs (think Sidekiq or Resque for Go).

Programming Languages

go
31211 projects - #10 most used programming language
javascript
184084 projects - #8 most used programming language
HTML
75241 projects

Projects that are alternatives of or similar to jobqueue

taskinator
A simple orchestration library for running complex processes or workflows in Ruby
Stars: ✭ 25 (-32.43%)
Mutual labels:  sidekiq, workers, resque
Workq
Job server in Go
Stars: ✭ 1,546 (+4078.38%)
Mutual labels:  workers, jobqueue
rocketjob
Ruby's missing background and batch processing system
Stars: ✭ 281 (+659.46%)
Mutual labels:  sidekiq, resque
rails async migrations
Asynchronous support for ActiveRecord::Migration
Stars: ✭ 56 (+51.35%)
Mutual labels:  sidekiq, workers
Sidekiq Job Php
Push and schedule jobs to Sidekiq from PHP
Stars: ✭ 34 (-8.11%)
Mutual labels:  sidekiq, workers
Verk
A job processing system that just verks! 🧛‍
Stars: ✭ 666 (+1700%)
Mutual labels:  sidekiq, workers
Gush
Fast and distributed workflow runner using ActiveJob and Redis
Stars: ✭ 894 (+2316.22%)
Mutual labels:  sidekiq, workers
Activejob Scheduler
A background job scheduler for any queue backend
Stars: ✭ 24 (-35.14%)
Mutual labels:  sidekiq, scheduled-jobs
La gear
What do you get when you glue sneakers and sidekiq together? la_gear! Pump it up!
Stars: ✭ 8 (-78.38%)
Mutual labels:  sidekiq, workers
Sidekiq Cron
Scheduler / Cron for Sidekiq jobs
Stars: ✭ 1,383 (+3637.84%)
Mutual labels:  sidekiq, scheduled-jobs
Activejob Retry
Automatic retries for ActiveJob
Stars: ✭ 138 (+272.97%)
Mutual labels:  sidekiq
Sidekiq status
Extension to Sidekiq to pass job execution metadata such as status and result back to the client
Stars: ✭ 155 (+318.92%)
Mutual labels:  sidekiq
cloudflare-worker-router
A super lightweight router (1.3K) with middleware support and ZERO dependencies for CloudFlare Workers.
Stars: ✭ 144 (+289.19%)
Mutual labels:  workers
Sidekiq Prometheus Exporter
All the basic metrics of Sidekiq with pluggable contribs prepared for Prometheus
Stars: ✭ 129 (+248.65%)
Mutual labels:  sidekiq
Sidekiq
Simple, efficient background processing for Ruby
Stars: ✭ 11,450 (+30845.95%)
Mutual labels:  sidekiq
meteor-cluster
worker pool for meteor using node js native `cluster` module
Stars: ✭ 18 (-51.35%)
Mutual labels:  jobqueue
worker-template-postgres
Reference demo and modified PostgreSQL driver to connect Cloudflare Workers to a relational database.
Stars: ✭ 75 (+102.7%)
Mutual labels:  workers
Simple scheduler
An enhancement for Heroku Scheduler + Sidekiq for scheduling jobs at specific times.
Stars: ✭ 127 (+243.24%)
Mutual labels:  sidekiq
sinatra-bootstrap
My opinionated Sinatra base application
Stars: ✭ 14 (-62.16%)
Mutual labels:  sidekiq
terraform-aws-eks-workers
Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers
Stars: ✭ 82 (+121.62%)
Mutual labels:  workers

Jobqueue

Jobqueue manages running and scheduling jobs (think Sidekiq or Resque).

Test Docs License

Prerequisites

You can choose between MySQL and MongoDB as a backend for persistent storage.

Getting started

Get the repository with go get github.com/olivere/jobqueue.

Example:

import (
	"github.com/olivere/jobqueue"
	"github.com/olivere/jobqueue/mysql"
)

// Create a MySQL-based persistent backend.
store, err := mysql.NewStore("root@tcp(127.0.0.1:3306)/jobqueue_e2e?loc=UTC&parseTime=true")
if err != nil {
	panic(err)
}

// Create a manager with the MySQL store and 10 concurrent workers.
m := jobqueue.New(
	jobqueue.SetStore(store),
	jobqueue.SetConcurrency(10),
)

// Register one or more topics and their processor
m.Register("clicks", func(args ...interface{}) error {
	// Handle "clicks" topic
})

// Start the manager
err := m.Start()
if err != nil {
	panic(err)
}

// Add a job: It'll be added to the store and processed eventually.
err = m.Add(&jobqueue.Add{Topic: "clicks", Args: []interface{}{640, 480}})
if err != nil {
	panic(err)
}

...

// Stop the manager, either via Stop/Close (which stops after all workers
// are finished) or CloseWithTimeout (which gracefully waits for a specified
// time span)
err = m.CloseWithTimeout(15 * time.Second) // wait for 15 seconds before forced stop
if err != nil {
	panic(err)
}

See the tests for more details on using jobqueue.

Tests and Web UI

Ensure the tests succeed with go test. You may have to install dependencies.

You can run a simulation of a real worker like so:

cd e2e
go run main.go

Play with the options: go run e2e/main.go -h.

Then open a second console and watch the worker doing its job:

cd ui
go run main.go

Then open your web browser at http://127.0.0.1:12345.

Screenshot

License

MIT License. See LICENSE file for details.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].