536 Open source projects that are alternatives of or similar to pyspark-cheatsheet

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Fundamentals of Spark with Python (using PySpark), code examples

Simple and Distributed Machine Learning

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Structural Bioinformatics Training Workshop & Hackathon 2018

Simple and Distributed Machine Learning

curated list of awesome tools and libraries for specific domains

Asynchronous actions for PySpark

A boilerplate for writing PySpark Jobs

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

[NOT MAINTAINED] Predicting Bit coin price using Time series analysis and sentiment analysis of tweets on bitcoin

Demo of PySpark and Jupyter Notebook with the Jupyter Docker Stacks

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.

Griffon Data Science Virtual Machine

Apache Spark Course Material

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

pyspark methods to enhance developer productivity 📣 👯 🎉

A curated list of awesome Apache Spark packages and resources.

Serverless proxy for Spark cluster

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Simple windows desktop application for viewing & querying Apache Parquet files

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, …

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

Apache Spark for data engineers

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

No description or website provided.

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

Apache Spark on AWS Lambda

Spark Gotchas. A subjective compilation of the Apache Spark tips and tricks

JupyterLab extension that enables monitoring launched Apache Spark jobs from within a notebook

Apache Spark Connector for Azure Cosmos DB

Apache (Py)Spark type annotations (stub files).

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

Powerful rapid automatic EDA and feature engineering library with a very easy to use API 🌟

Data validation library for PySpark 3.0.0

Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.

🏐 Apache Parquet for modern .NET

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

A visual ETL development and debugging tool for big data

Project for James' Apache Spark with Scala course

Real-world Spark pipelines examples

Bulletproof Apache Spark jobs with fast root cause analysis of failures.

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

🔥 One of the most comprehensive open-source data annotation platform.

Spark in Action, 2nd edition - chapter 1 - Introduction

subsemble R package for ensemble learning on subsets of data

Basic framework utilities to quickly start writing production ready Apache Spark applications

This repo contains commands that data engineers use in day to day work.

PredictionIO PHP SDK

an online learning algorithm library for Storm

Base types for OnlineStats.

🐬 Apache Parquet for modern .Net

A distributed key-value storage that's tolerant to Byzantine fault.

可视化大屏解决方案, 提供一套可视化编辑引擎, 助力个人或企业轻松定制自己的可视化大屏应用.

The Internals of Spark SQL

An Apache Spark standalone application using the Spark API in Scala. The application uses Simple Build Tool(SBT) for building the project.