atelierdisko / Cute_php

    .

,-. . . |- ,-. | | | | |-' -'-^ '-' -- PHP version of the beanstalkd-backed job queuing system.

Synopsis

First of all is cute a concept of a coherent and feature complete job system. Its small API surface and flat class hierarchy allows to easily understand and reimplement it in other languages.

In many ways it can be thought of a hybrid between resque[0] and gearman[1] while using beanstalkd[2] as its backend. Resque inspired how worker processes behave, the overall feature set and finally the concept of creating an implementation that through its simplicity becomes specification-like. The way how handlers are registered centrally has been borrowed from gearman.

While one of the goals for cute is to keep it as slim and lean as possible, it should provide a feature set that fulfills all common requirements of web applications. Cute expects jobs to fail[3] and provides extensive logging throughout its codebase.

[0] https://github.com/resque/resque [1] http://gearman.org [2] https://kr.github.io/beanstalkd [3] http://xph.us/2010/05/02/how-to-handle-job-failures.html

=== This is an early release and not yet ready for production. === Some features are still being implemented (delayed and scheduled jobs). === You may still try it, anybody who's testing helps making cute better.

Features

• Administer and debug by using standard beanstalkd tooling. No special "cute admin" tooling needed.
• Extensive logging to ease debugging, builtin process title support for workers without the need of additional extensions.
• Native job persistence.
• Native job priorities and TTR.
• Native delayed jobs. (not yet implemented)
• Blocking and non-blocking jobs.
• Fallback to inline processing if job server goes offline.
• Resilient workers.
• Back off strategies builtin.
• Namespaced connection operations allow for sharing a single job server instance with multiple apps.

Installation

The preferred installation method is via composer. You can add the library as a dependency via:

$ composer require atelierdisko/cute

Handlers

First we create a handlers.php file in which we'll register handlers. In this example we register a handler that resizes images. Handlers must be available to both workers and producers so its a good idea to keep them in a dedicated file.

Handlers are identified by a name and have one handler function. This function does the actual work when processing a job.

fit(100, 150); // Returning `false` will fail and bury the job, `true` // to signal success. return $image->store($data['target']); }); ?>

Regular Jobs

run('thumbnail', [ 'from' => '/path/to/cat.png', 'to' => '/tmp/cat_thumb.png' ]); ?>

Delayed Jobs

in('+2 minutes', 'sendFollowUpMail', '[email protected]'); ?>

Workers

Workers can be started using the following command. Workers must have access to registered handlers.

$ bin/cute-worker --queue=image --require=./handlers.php

Cute workers are very similar to resque workers and can be controlled through signals in the exact same way:

QUIT - Wait for child to finish processing then exit. TERM / INT - Immediately kill child then exit. USR2 - Don't start to process any new jobs. CONT - Start to process new jobs again after a USR2.

Job Options

You may - as a last parameter - pass a number of options to the handlers and enqueuing methods. Options passed on an enqueue call take precendence over the ones defined on handlers.

run(/* handler, data, options */); $job->in(/* delay, handler, data, options */); // The following options are available. $options = [ // Prioritize job. _HIGH, _NORMAL, _LOW are available; defaults // to _NORMAL. Alternatively an integer between 0 (highest) and // 4294967295 (lowest). 'priority' => Job::PRIORITY_NORMAL, // Makes the call block until worker has processed the job. // Defaults to `false`. 'wait' => false, // If we can't access the job server, should we process inline // instead? Note that inline operations will block. Defaults to // `false`. 'fallback' => false, // Time the job is allowed to run in seconds; by default // 10 minutes. 'ttr' => 600, // The name of the queue to put the job in. Defaults to `default`. 'queue' => 'default' ]; ?>

Design Choices

Why not Resque?

• Resque is great. Especially that its simple "specification" spawned many implementations in different languages.

  However one of the original reasons why resque was created, was that beanstalkd didn't provide persistence. That doesn't hold true anymore.

  We also wanted a simpler way to perform jobs. We didn't want to create a class for each "Performance" and prefer simple anonymous functions instead.

Why not Redis as a Backend?

• We've been very tempted to use redis as the job server backend as it's already part of ours (and probably many others) infrastructure and could simply be reused for this purpose.

  But we also had lots of very good experience with beanstalkd and enjoy its semantics and minimalism. We also think we can benefit from beanstalkd being specialized as a work queue and its feature set. Using redis would've meant reimplementing a poor beanstalkd instead.

Why not Gearman?

• Gearman is great and provides lots of valuable features. We didn't like the semantics (do vs doBackound vs doTask) and the sheer size of it. We wanted to define our own minimal semantics.

Copyright & License

Cute is Copyright (c) 2014-2015 Atelier Disko if not otherwise stated. The code is distributed under the terms of the BSD 3-clause License. For the full license text see the LICENSE file.