6338 Open source projects that are alternatives of or similar to Spark Practice

🌐 Interactive Workshop on GeoAnalysis using PySpark

Word2Vec models with Twitter data using Spark. Blog:

Jupyter magics and kernels for working with remote Spark clusters

Updated repository

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

A simple Spark TDD example

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

HandySpark - bringing pandas-like capabilities to Spark dataframes

Fundamentals of Spark with Python (using PySpark), code examples

Apache Spark Connector for Azure Cosmos DB

This is a repo documenting the best practices in PySpark.

The Hunting ELK

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Example project implementing best practices for PySpark ETL jobs and applications.

Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

Demo of PySpark and Jupyter Notebook with the Jupyter Docker Stacks

Training models with Apache Spark, PySpark for Titanic Kaggle competition

Apache Spark - Turbofan Engine Degradation Simulation Data Set example in Apache Spark

An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

Python Helper library for Jupyter Notebooks

Code examples on Apache Spark using python

🎓 Jupyter notebooks from UFC data science course

[NOT MAINTAINED] Predicting Bit coin price using Time series analysis and sentiment analysis of tweets on bitcoin

PySpark Code for Hands-on Learners

Relation Extraction using Deep learning(CNN)

A guide on how to set up Jupyter with Pyspark painlessly on AWS EC2 clusters, with S3 I/O support

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

A tutorial on the most important features and idioms of Scala that you need to use Spark's Scala APIs.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

A library that provides useful extensions to Apache Spark and PySpark.

A free tutorial for Apache Spark.

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Data science and Big Data with Python

A Scala kernel for Jupyter

pyspark🍒🥭 is delicious，just eat it!😋😋

BERT, AWS RDS, AWS Forecast, EMR Spark Cluster, Hive, Serverless, Google Assistant + Raspberry Pi, Infrared, Google Cloud Platform Natural Language, Anomaly detection, Tensorflow, Mathematics

Udacity Data Engineering Nano Degree (DEND)

Models and Pipelines for the Spark NLP library

Spark 2.0 Python Machine Learning examples

🐍 Quick reference guide to common patterns & functions in PySpark.

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

LearningApacheSpark

Content for architecting a data science platform for products using Luigi, Spark & Flask.

Big Data Ecosystem Docker

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Process Common Crawl data with Python and Spark

🎼 Docker compose files for various kafka stacks

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Big Data for Data Engineers Coursera Specialization from Yandex

A repository to keep track of all the code that I end up writing for my blog posts.

Isolation Forest on Spark

State of the Art Natural Language Processing