536 Open source projects that are alternatives of or similar to pyspark-cheatsheet

repo for code published on pythondata.com

An admission control policy that safeguards against accidental duplicate claiming of Hosts/Domains.

🐬 Apache Parquet for modern .Net

Dremio - the missing link in modern data

Social Network Facebook Analysis (Python, Networkx)

Apache Accumulo

This project aim is to provide free software to fetch data from plcs (Siemens S7-300/400/1200/1500) and store it. Used stack is completly opensource. I used InfluDB as data storage, so application principle is following Big Data paradigm.

Google Cloud Dataflow provides a simple, powerful model for building both batch and streaming parallel data processing pipelines.

PredictionIO Ruby SDK

A distributed key-value storage that's tolerant to Byzantine fault.

A distributed block-based data storage and compute engine

The Proxima platform.

Mirror of Apache Ambari

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

A connector for Apache Spark to access Exasol

Mirror of Apache Sqoop

Code for my presentation: Using PySpark to Process Boat Loads of Data

Titanoboa makes complex workflows easy. It is a low-code workflow orchestration platform for JVM - distributed, highly scalable and fault tolerant.

Microsoft's contributions for Spark with Apache Accumulo

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

The most widely used Python to C compiler

Mirror of Apache Samza

Apache Spark Data Source for ROOT File Format

Intel® Scalable Dataframe Compiler for Pandas*

A connector for Apache Spark and PySpark to Dgraph databases.

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Apache Pulsar Adapters

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

Koalas: pandas API on Apache Spark

Efficient video analysis at scale

A distributed, fast open-source graph database featuring horizontal scalability and high availability

Workflows and interfaces for neuroimaging packages

An easy to use, self-service open BI reporting and BI dashboard platform.

"Byteslice: Pushing the envelop of main memory data processing with a new storage layout" (SIGMOD'15)

Distributed Big Data Orchestration Service

Apache Beam is a unified programming model for Batch and Streaming

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.

Geo Spatial Data Analytics on Spark

DevOps pipeline for Real Time Social/Web Mining

Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Apache Trafodion

Redis in a python module.

Mirror of Apache Falcon

Quiz & Assignment of Coursera

An advanced distributed task flow management on top of Celery

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Mirror of Apache Airavata Django Portal

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

整理一些书籍 ,包含 C&C++ 、git 、Java、Keras 、Linux 、NLP 、Python 、Scala 、TensorFlow 、大数据 、推荐系统、数据库、数据挖掘 、机器学习 、深度学习 、算法等。

Alex Castrounis' curated set of resources for artificial intelligence (AI), machine learning, data science, internet of things (IoT), and more.

Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

A curated list of big data engineering tools, resources and communities.

Two-day workshop that covers how to use R to interact databases and Spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/

Service for automatically managing and cleaning up unreferenced data

R on Apache Spark (SparkR) tutorials for Big Data analysis and Machine Learning as IPython / Jupyter notebooks

PredictionIO Java SDK

Ultimate Tennis Statistics and Tennis Crystal Ball - Tennis Big Data Analysis and Prediction

High-scalability analytics database built on immutable merkle-trees

MySQL performance monitoring and analysis.