525 Open source projects that are alternatives of or similar to Thrill

Fundamentals of Spark with Python (using PySpark), code examples

Open-source distributed computation and storage platform

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

An advanced distributed task flow management on top of Celery

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

A simplified, lightweight ETL Framework based on Apache Spark

Bare bone examples of machine learning in TensorFlow

A Clojure dataframe library that runs on Spark

The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.

A distributed block-based data storage and compute engine

Stream computing platform for bigdata

Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin

d3 library to build circular graphs

Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Mirror of Apache Bigtop

Server for the ListenBrainz project

DevOps methodology & roadmap for a devops developer in 2019. Interesting books to learn new technologies.

Stroom is a highly scalable data storage, processing and analysis platform.

Source code for the CERN Open Data portal

Scalable, redundant, and distributed object store for Apache Hadoop

Distributed training framework with parameter server

Single-pass algorithms for statistics

distributed dataflows with functional list operations for data processing with C++14

MockNeat is a Java 8+ library that facilitates the generation of arbitrary data for your applications.

Golang implementation of the PlatON protocol

Datafuse is a free Cloud-Native Analytics DBMS(Inspired by ClickHouse) implemented in Rust

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Apache Tez

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

🏅State-of-the-art learned data structure that enables fast lookup, predecessor, range searches and updates in arrays of billions of items using orders of magnitude less space than traditional indexes

A HTTP Ruby API for Consul

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

The open big data serving engine. https://vespa.ai

Awesome list of distributed systems resources

Mirror of Apache Apex core

Alex Castrounis' curated set of resources for artificial intelligence (AI), machine learning, data science, internet of things (IoT), and more.

Apache Parquet

Fast Fourier Transform-accelerated Interpolation-based t-SNE (FIt-SNE)

🦘 The Grouparoo Monorepo - open source customer data sync framework

CogComp's Natural Language Processing libraries and Demos:

Best Practices and Style Guides in BEEVA

Arkime (formerly Moloch) is an open source, large scale, full packet capturing, indexing, and database system.

A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

A SDK/library for decentralized web and distributing computing projects

Distributed Stockfish analysis for lichess.org

Redis in a python module.

📚 👓 A collection of research papers, codes, tutorials and blogs on Federated Computing/Learning.

Kafka Connect HDFS connector

ROOT I/O in pure Python and NumPy.

Geo Spatial Data Analytics on Spark

An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads.

Apache ORC - the smallest, fastest columnar storage for Hadoop workloads

Serverless proxy for Spark cluster

Fluid, elastic data abstraction and acceleration for BigData/AI applications in cloud

Apache Ignite

Mirror of Apache Helix

Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.

Quiz & Assignment of Coursera

Building Large-Scale AI Applications for Distributed Big Data