6372 Open source projects that are alternatives of or similar to Pyspark Learning

A simple Spark TDD example

Fundamentals of Spark with Python (using PySpark), code examples

Apache Spark (PySpark) Practice on Real Data

Word2Vec models with Twitter data using Spark. Blog:

🌐 Interactive Workshop on GeoAnalysis using PySpark

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

HandySpark - bringing pandas-like capabilities to Spark dataframes

Jupyter magics and kernels for working with remote Spark clusters

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

Apache Spark Connector for Azure Cosmos DB

Code examples on Apache Spark using python

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

An open source framework for building data analytic applications.

Example project implementing best practices for PySpark ETL jobs and applications.

Real Time Analytics and Data Pipelines based on Spark Streaming

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

Demo of PySpark and Jupyter Notebook with the Jupyter Docker Stacks

C# and F# language binding and extensions to Apache Spark

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

A Flexible and Powerful Parameter Server for large-scale machine learning

Apache Spark - Turbofan Engine Degradation Simulation Data Set example in Apache Spark

Python Helper library for Jupyter Notebooks

scala、spark使用过程中，各种测试用例以及相关资料整理

This is a repo documenting the best practices in PySpark.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Scala examples for learning to use Spark

酷玩 Spark: Spark 源代码解析、Spark 类库等

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

A tutorial on the most important features and idioms of Scala that you need to use Spark's Scala APIs.

Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

A free tutorial for Apache Spark.

Training models with Apache Spark, PySpark for Titanic Kaggle competition

零基础学习spark，大数据学习

Custom state store providers for Apache Spark

The Hunting ELK

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

Models and Pipelines for the Spark NLP library

Big Data for Data Engineers Coursera Specialization from Yandex

Spark 2.0 Python Machine Learning examples

PySpark Code for Hands-on Learners

[NOT MAINTAINED] Predicting Bit coin price using Time series analysis and sentiment analysis of tweets on bitcoin

Udacity Data Engineering Nano Degree (DEND)

Relation Extraction using Deep learning(CNN)

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

A Scala kernel for Jupyter

🎓 Jupyter notebooks from UFC data science course

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

Data science and Big Data with Python

Kinesis Connector for Structured Streaming

pyspark🍒🥭 is delicious，just eat it!😋😋

Production Ready Data Integration Product, documentation：

Apache Spark and Apache Kafka integration example

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser