Rocket Job

Ruby's missing batch system

Checkout https://rocketjob.io/

Documentation

• Guide
• API Reference

Support

• Questions? Join the chat room on Gitter for rocketjob support
• Report bugs

Rocket Job v5

• Support for Ruby v3 and Rails 6.
• Multiple output file support through extended output_categories capability.
  • File output formats for each category. For example: CSV, PSV, JSON, etc.
• Support for AWS DocumentDB as the data store.
• Removed use of Symbols to meet Symbol deprecation in MongoDB and Mongoid.

The following plugins have been deprecated and will be removed in Rocket Job v5.1

• RocketJob::Batch::Tabular::Input
• RocketJob::Batch::Tabular::Output

Rocket Job v4

Rocket Job Pro is now open source and included in Rocket Job.

The RocketJob::Batch plugin now adds batch processing capabilities to break up a single task into many concurrent workers processing slices of the entire job at the same time.

Example:

class MyJob < RocketJob::Job
  include RocketJob::Batch
  
  self.description         = "Reverse names"
  self.destroy_on_complete = false
  self.collect_output      = true

  # Method to call by all available workers at the same time.
  # Reverse the characters for each line: 
  def perform(line)
    line.reverse
  end
end

Upload a file for processing, for example names.csv which could contain:

jack
jane
bill
john
blake
chris
dave
marc

To queue the above job for processing:

job = MyJob.new
job.upload('names.csv')
job.save!

Once the job has completed, download the results into a file:

job.download('names_reversed.csv')

Contributing to the documentation

To contribute to the documentation it is as easy as forking the repository and then editing the markdown pages directly via the github web interface.

For more complex documentation changes checkout the source code locally.

Local checkout

• Fork the repository in github.
• Checkout your fork of the source code locally.
• Install Jekyll

    cd docs
    bundle update

• Run Jekyll web server:

    jekyll serve

• Open a web browser to view the local documentation: http://127.0.0.1:4000
• Edit the files in the /docs folder.
• Refresh the page to see the changes.

Once the changes are complete, submit a github pull request.

Upgrading to V3

V3 replaces MongoMapper with Mongoid which supports the latest MongoDB Ruby client driver.

Upgrading Mongo Config file

Replace mongo.yml with mongoid.yml.

Start with the sample mongoid.yml.

For more information on the new Mongoid config file.

Note: The rocketjob and rocketjob_slices clients in the above mongoid.yml file are required.

Other changes

• Arguments are no longer supported, use fields for defining all named arguments for a job.

• Replace usages of rocket_job do to set default values:

  rocket_job do |job|
    job.priority = 25
  end

With:

  self.priority = 25

• Replace key with field when adding attributes to a job:

  key :inquiry_defaults, Hash

With:

  field :inquiry_defaults, type: Hash, default: {}

• Replace usage of public_rocket_job_properties with the user_editable option:

field :priority, type: Integer, default: 50, user_editable: true

Ruby Support

Rocket Job is tested and supported on the following Ruby platforms:

• Ruby 2.1, 2.2, 2.3, 2.4, and above
• JRuby 9.0.5 and above

Dependencies

• MongoDB
  • Persists job information.
  • Version 2.7 or greater.
• Semantic Logger
  • Highly concurrent scalable logging.

Versioning

This project uses Semantic Versioning.

Author

Reid Morrison

Contributors

Contributors