363 Open source projects that are alternatives of or similar to machine-learning-data-pipeline

Parallel algorithms and data structures for tree-based AMR with arbitrary element shapes.

A fast, concurrency-focused task automation tool.

Yet Another Concurrency Library

Run async mocha specs in parallel

Parallel Computing Library for Linux and macOS & NVIDIA CUDA Wrapper

Extract video features from raw videos using multiple GPUs. We support RAFT and PWC flow frames as well as S3D, I3D, R(2+1)D, VGGish, CLIP, ResNet features.

Singularity: Application containers for Linux

Distributed parallelization of 3D volume mesh adaptation

A parallel neural net microframework

Parallel NDJSON Reader for Python

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Build and deploy a serverless data pipeline on AWS with no effort.

Asynchronous Programming Library for JavaScript & Node.js

Functional collections extension functions for Python

Ray tracer with phong lighting, reflections, refractions, normal mapping, procedural textures, super sampling, and depth of field.

WAND-PIC

LargeScale Multiphysics Scientific Simulation Environment-OneFLOW CFD

a tiny function that "parallelizes" work in NodeJS

A pytest plugin for parallel and concurrent testing

28Kb, small memory footprint, single binary that run list of commands in parallel and waits for their termination

The versatile ocean simulator, in pure Python, powered by Bohrium.

A collection of various resources, examples, and executables for the general NREL HPC user community's benefit. Use the following website for accessing documentation.

Sort files and pack them into partitions

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

parallel finite element unstructured meshes

Remote Sensing and GIS Software Library; python module tools for processing spatial data.

Parallel event emitter built on futures-rs

Compose your async functions with elegance.

Bulk Stash is a docker rclone service to sync, or copy, files between different storage services. For example, you can copy files either to or from a remote storage services like Amazon S3 to Google Cloud Storage, or locally from your laptop to a remote storage.

CB-Geo High-Performance Material Point Method

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

A simple and efficient tool to parallelize Pandas operations on all available CPUs

List of inspiring articles, blogs, tutorials and books. Tech stuff.

Linear optimization software

A Clojure machine learning library

A R package that allows users to submit parallel workloads in Azure

Galaxy is an asynchronous parallel visualization ray tracer for performant rendering in distributed computing environments. Galaxy builds upon Intel OSPRay and Intel Embree, including ray queueing and sending logic inspired by TACC GraviT.

The official repository for ROOT: analyzing, storing and visualizing big data, scientifically

In-place Parallel Super Scalar Radix Sort (IPS²Ra)

Scroll - making scrolling through buffers fun since 2016

✨🐌 🐎✨ fastest npm scripts runner

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

Hierarchical bit set container

R package providing access and examples to TRNG C++ library

Parallel RSpec test runner

An interactive parallel ssh client featuring autocomplete and asynchronous execution.

Fast Concurrent / Parallel Sorting in Go

A Parallel Graphlet Decomposition Library for Large Graphs

Async utilities for Golang.

🐺 Create browser tests 10x faster

A T-Digest implementation in golang

Lightweight parallel Java tasks

🔥Service level control flow for Node.js

Companion code in JAX for the paper Parallel Iterated Extended and Sigma-Point Kalman Smoothers.

C# Framework to automate tests using Selenium WebDriver

Analyze Data with Pandas-based Networks. Documentation:

turbo.js - perform massive parallel computations in your browser with GPGPU.

SST Structural Simulation Toolkit Parallel Discrete Event Core and Services

sciblox - Easier Data Science and Machine Learning