363 Open source projects that are alternatives of or similar to machine-learning-data-pipeline

Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby

Using Arrows to model parallel processes/computations.

GWAS summary statistics files QC tool

A lightweight data processing framework for Apache Spark

Browser computing unleashed!

Model complex data transformation pipelines easily

Parallel Computing Library for Linux and macOS & NVIDIA CUDA Wrapper

Flow-based data pre-processing for deep learning

Generalised and highly customisable, hybrid-parallelism, database based, deep learning framework.

LightJason - AgentSpeak(L++) for Java

Goroutine analogue for Node.js, spreads I/O-bound routine calls to utilize thread pool (worker_threads) using balancer with event loop utilization. 🌱

A simple and lightweight JavaScript data processing tool. Live demo:

Heavy Peer To Peer: a MPI based benchmark for network diagnostic

Functional library for Javascript

Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

一个基于浏览器环境的数据采集SDK

My name is Fang Biao. I'm currently pursuing my Master degree with the college of Computer Science and Engineering, Si Chuan University, Cheng Du, China. For more informantion about me and my research, you can go to [my homepage](https://github.com/hisrg). One of my research interests is architecture design for deep learning and neuromorphic com…

Blog post on ETL pipelines with Airflow

Python surface process framework on highly scalable unstructured meshes

Scientific and statistical computing with Rust.

Implementation of PCI drivers, kprobe, sysfs, devfs, sensor driver, miscdevices, synchronization

Easy massive SNMP-agent simulation with the use of simple YAML files

Cross-platform test runner written for Android and iOS projects

This MATLAB class provides a smart progress bar like tqdm in the command window and is optimized for progress information in simple iterations or large frameworks with full support of parallel parfor loops provided by the MATLAB Parallel Computing Toolbox.

🚀 Ping IP addresses and domains in parallel to find the accessible and inaccessible ones.

⭐️ ⭐️⭐️ Migrated to Salesforce Open Source Platform - https://github.com/salesforce/codeceptjs-bdd

A framework for running appium tests in parallel across devices and also on desktop browser... U like it STAR it !!

Cross-platform test runner written for Android and iOS projects

Parallel algorithms and data structures for tree-based AMR with arbitrary element shapes.

Yet Another Concurrency Library

Extract video features from raw videos using multiple GPUs. We support RAFT and PWC flow frames as well as S3D, I3D, R(2+1)D, VGGish, CLIP, ResNet features.

Distributed parallelization of 3D volume mesh adaptation

Parallel NDJSON Reader for Python

Build and deploy a serverless data pipeline on AWS with no effort.

Functional collections extension functions for Python

WAND-PIC

a tiny function that "parallelizes" work in NodeJS

28Kb, small memory footprint, single binary that run list of commands in parallel and waits for their termination

A collection of various resources, examples, and executables for the general NREL HPC user community's benefit. Use the following website for accessing documentation.

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Remote Sensing and GIS Software Library; python module tools for processing spatial data.

Parallel event emitter built on futures-rs

Bulk Stash is a docker rclone service to sync, or copy, files between different storage services. For example, you can copy files either to or from a remote storage services like Amazon S3 to Google Cloud Storage, or locally from your laptop to a remote storage.

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

List of inspiring articles, blogs, tutorials and books. Tech stuff.

A Clojure machine learning library

Galaxy is an asynchronous parallel visualization ray tracer for performant rendering in distributed computing environments. Galaxy builds upon Intel OSPRay and Intel Embree, including ray queueing and sending logic inspired by TACC GraviT.

In-place Parallel Super Scalar Radix Sort (IPS²Ra)

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

R package providing access and examples to TRNG C++ library

A Parallel Graphlet Decomposition Library for Large Graphs

An interactive parallel ssh client featuring autocomplete and asynchronous execution.

Fast, flexible and easy to use probabilistic modelling in Python.

🐺 Create browser tests 10x faster

Lightweight parallel Java tasks

A keyphrase extractor for Persian

Reinforcement Learning Algorithms Based on TensorFlow 2.x

turbo.js - perform massive parallel computations in your browser with GPGPU.

Companion code in JAX for the paper Parallel Iterated Extended and Sigma-Point Kalman Smoothers.

SST Structural Simulation Toolkit Parallel Discrete Event Core and Services