Top 51 batch-processing open source projects

Simplify HPC and Batch workloads on Azure

A Free Krita plugin to batch export layers and groups with maximum flexibility. Scale, trim, export multiple copies of your layers...

*Decompile All the Things* - IDA Batch Decompile plugin and script for Hex-Ray's IDA Pro that adds the ability to batch decompile multiple files and their imports with additional annotations (xref, stack var size) to the pseudocode .c file

Simply does the tedious, repetitive operations for all rows of excel files step by step and reports after the job is done. It can download files from URL(s) in a column of Excel files. If a new filename is provided at column B it will rename the file before saving. It will even create sub folders if column C is full with a valid folder name.

面向 OLTP、OLAP、批处理、流处理场景的大一统 SQL 引擎

Parallel SSH Tools

Elastic data processing with Apache Pulsar and Apache Flink

Fully automated batch processing of Revit files with your own Python or Dynamo task scripts!

Asakusa Framework

Python Data Processing library

Serverless batch computing made simple.

Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning

Multi-device OpenCL kernel load balancer and pipeliner API for C#. Uses shared-distributed memory model to keep GPUs updated fast while using same kernel on all devices(for simplicity).

A Mass Image Processing tool for Windows

When Apache Pulsar meets Apache Spark

This library for Android will take any set of events and batch them up before sending it to the server. It also supports persisting the events on disk so that no event gets lost because of an app crash. Typically used for developing any in-house analytics sdk where you have to make a single api call to push events to the server but you want to optimize the calls so that the api call happens only once per x events, or say once per x minutes. It also supports exponential backoff in case of network failures

Batch-processing of Revit models using the Dynamo visual programming environment

Windows Batch script to automate batch processing using mkvtoolnix.

Unofficial alternative launcher for QuantConnect's LEAN allowing for parallel execution and looping/batching with customizable parameters and ranges.

Distributed Stream and Batch Processing

Distributed, fault tolerant batch processing for Natural Language Applications and Search, using remote partitioning

Simple API, Complex Emails (JavaMail smtp wrapper)

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes

BIMP. Batch Image Manipulation Plugin for GIMP.

The simple, stupid batch framework for Java

Short lived microservices with Spring Batch

Ruby's missing background and batch processing system

Object Oriented Modular Abstract Calibration Toolbox

Batch Guetzli compressions in a manageable fashion

A simple utility for executing multiple sequential or multi-threaded applications in a single multi-node batch job

Online batch converter of SVG images to Android vector drawable XML resource files

Ruby's missing background and batch processing system

Batch processing library for Golang.

Tensorflow 2 Faster-RCNN implementation from scratch supporting to the batch processing with MobileNetV2 and VGG16 backbones

Generic file search & replace tool, written in Python 3

Detailed documentation is available here: http://ifiscripts.readthedocs.io/en/latest/index.html

Average and morph faces online http://webmorph.org/

Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution

Shell script to run OpenRefine in batch mode (import, transform, export). It orchestrates OpenRefine (server) and a python client that communicates with the OpenRefine API.

The Proxima platform.

Elasticsearch tool for easily collecting and batch inserting Python data and pandas DataFrames

Recursive Neural Networks for PyTorch

”Building a concurrent and distributed system for batch processing which is fault tolerant and can scale up or scale out using Akka.NET (based on actor model)”.

🚚 Batch requests for Rack apps (works with Rails, Sinatra, etc)

It is an efficient, flexible, and lightweight goroutine pool. It provides an easy way to deal with concurrent tasks with limited resource.

MarkLogic tool for processing and reporting on content, enhanced from the original CoRB

A tiny framework for building batch applications as a collection of tasks in a workflow.

Example use of AWS batch

REST API for Spring Batch using Spring Boot 2.2

A blazing fast job processing system backed by GenStage & Redis.