wang502 / Gores
Licence: mit
👷 Redis-backed library for creating background jobs in Go. Placing jobs in multiple queues, and process them later asynchronously.
Stars: ✭ 137
Programming Languages
go
31211 projects - #10 most used programming language
Projects that are alternatives of or similar to Gores
Relieve
Ease the implementation of multi processing accross your microservices
Stars: ✭ 47 (-65.69%)
Mutual labels: microservice, workers
Web Development Interview With Java
Java 开发相关技术栈(大中厂)高频面试问题收录。
Stars: ✭ 69 (-49.64%)
Mutual labels: microservice, redis
Kiq
📮 Robust job queue powered by GenStage and Redis
Stars: ✭ 49 (-64.23%)
Mutual labels: workers, redis
Toro
Multithreaded message processing on Postgres
Stars: ✭ 39 (-71.53%)
Mutual labels: workers, message-queue
Carmine
Redis client and message queue for Clojure
Stars: ✭ 999 (+629.2%)
Mutual labels: redis, message-queue
Goodskill
🐂基于springcloud +dubbo构建的模拟秒杀项目,模块化设计,集成了分库分表、elasticsearch🔍、gateway、mybatis-plus、spring-session等常用开源组件
Stars: ✭ 786 (+473.72%)
Mutual labels: microservice, redis
Netpro
🌈An enhanced version of asp.netcore,Support for netcore3.1
Stars: ✭ 112 (-18.25%)
Mutual labels: microservice, redis
Butterfly
🔥 蝴蝶--【简单】【稳定】【好用】的 Python web 框架🦋 除 Python 2.7,无其他依赖; 🦋 butterfly 是一个 RPC 风格 web 框架,同时也是微服务框架,自带消息队列通信机制实现分布式
Stars: ✭ 82 (-40.15%)
Mutual labels: microservice, redis
Sidekiq Job Php
Push and schedule jobs to Sidekiq from PHP
Stars: ✭ 34 (-75.18%)
Mutual labels: workers, redis
Simpleue
PHP queue worker and consumer - Ready for AWS SQS, Redis, Beanstalkd and others.
Stars: ✭ 124 (-9.49%)
Mutual labels: workers, redis
Gush
Fast and distributed workflow runner using ActiveJob and Redis
Stars: ✭ 894 (+552.55%)
Mutual labels: workers, redis
Huststore
High-performance Distributed Storage
Stars: ✭ 806 (+488.32%)
Mutual labels: redis, message-queue
Verk
A job processing system that just verks! 🧛
Stars: ✭ 666 (+386.13%)
Mutual labels: workers, redis
Rmq
Message queue system written in Go and backed by Redis
Stars: ✭ 722 (+427.01%)
Mutual labels: redis, message-queue
Rqueue
Rqueue aka Redis Queue [Task Queue, Message Broker] for Spring framework
Stars: ✭ 76 (-44.53%)
Mutual labels: workers, redis
Dtcqueuebundle
Symfony2/3/4/5 Queue Bundle (for background jobs) supporting Mongo (Doctrine ODM), Mysql (and any Doctrine ORM), RabbitMQ, Beanstalkd, Redis, and ... {write your own}
Stars: ✭ 115 (-16.06%)
Mutual labels: workers, redis
Gores
An asynchronous job execution system based on Redis
Installation
Get the package
$ go get github.com/wang502/gores/gores
Import the package
import "github.com/wang502/gores/gores"
Usage
Start local Redis server
$ git clone [email protected]:antirez/redis.git
$ cd redis
$ ./src/redis-server
Configuration
Add a config.json in your project folder
{
"RedisURL": "127.0.0.1:6379",
"RedisPassword": "mypassword",
"BlpopMaxBlockTime" : 1,
"MaxWorkers": 2,
"Queues": ["queue1", "queue2"],
"DispatcherTimeout": 5,
"WorkerTimeout": 5
}
-
RedisURL: Redis server address. If you run in a local Redis, the dafault host is
127.0.0.1:6379
- RedisPassword: Redis password. If the password is not set, then password can be any string.
- BlpopMaxBlockTime: Blocking time when calling BLPOP command in Redis.
- MaxWorkers: Maximum number of concurrent workers, each worker is a separate goroutine that execute specific task on the fetched item.
- Queues: Array of queue names on Redis message broker.
- DispatcherTimeout: Duration dispatcher will wait to dispatch new job before quitting.
- WorkerTimeout: Duration worker will wait to process new job before quitting.
Initialize config
configPath := flag.String("c", "config.json", "path to configuration file")
flag.Parse()
config, err := gores.InitConfig(*configPath)
Enqueue job to Redis queue
A job is a Go map. It is required to have several keys:
- Name: name of the item to enqueue, items with different names are mapped to different tasks.
- Queue: name of the queue you want to put the item in.
- Args: the required arguments that you need in order for the workers to execute those tasks.
- Enqueue_timestamp: the Unix timestamp of when the item is enqueued.
gores := gores.NewGores(config)
job := map[string]interface{}{
"Name": "Rectangle",
"Queue": "TestJob",
"Args": map[string]interface{}{
"Length": 10,
"Width": 10,
},
"Enqueue_timestamp": time.Now().Unix(),
}
err = gores.Enqueue(job)
if err != nil {
log.Fatalf("ERROR Enqueue item to Gores")
}
$ go run main.go -c ./config.json -o produce
Define tasks
package tasks
// task for item with 'Name' = 'Rectangle'
// calculating the area of an rectangle by multiplying Length with Width
func CalculateArea(args map[string]interface{}) error {
var err error
length := args["Length"]
width := args["Width"]
if length == nil || width == nil {
err = errors.New("Map has no required attributes")
return err
}
fmt.Printf("The area is %d\n", int(length.(float64)) * int(width.(float64)))
return err
}
Launch workers to consume items and execute tasks
tasks := map[string]interface{}{
"Item": tasks.PrintItem,
"Rectangle": tasks.CalculateArea,
}
gores.Launch(config, &tasks)
$ go run main.go -c ./config.json -o consume
The output will be:
The rectangle area is 100
Info about processed/failed job
gores := gores.NewGores(config)
if gores == nil {
log.Fatalf("gores is nil")
}
info, err := gores.Info()
if err != nil {
log.Fatal(err)
}
for k, v := range info {
switch v.(type) {
case string:
fmt.Printf("%s : %s\n", k, v)
case int:
fmt.Printf("%s : %d\n", k, v)
case int64:
fmt.Printf("%s : %d\n", k, v)
}
}
The output will be:
Gores Info:
queues : 2
workers : 0
failed : 0
host : 127.0.0.1:6379
pending : 0
processed : 1
Contribution
Please feel free to suggest new features. Also open to pull request!
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].