456 Open source projects that are alternatives of or similar to spark-extension

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

A simple Spark TDD example

Example project implementing best practices for PySpark ETL jobs and applications.

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

LearningApacheSpark

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

🎼 Docker compose files for various kafka stacks

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Relation Extraction using Deep learning(CNN)

Process Common Crawl data with Python and Spark

Apache Spark Connector for Azure Cosmos DB

Some class materials for a data processing course using PySpark

pyspark🍒🥭 is delicious，just eat it!😋😋

🌐 Interactive Workshop on GeoAnalysis using PySpark

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Isolation Forest on Spark

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

Word2Vec models with Twitter data using Spark. Blog:

Spark 2.0 Python Machine Learning examples

Updated repository

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

State of the Art Natural Language Processing

Apache Spark (PySpark) Practice on Real Data

🐍 Quick reference guide to common patterns & functions in PySpark.

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Fundamentals of Spark with Python (using PySpark), code examples

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

Jupyter magics and kernels for working with remote Spark clusters

Training models with Apache Spark, PySpark for Titanic Kaggle competition

HandySpark - bringing pandas-like capabilities to Spark dataframes

Simple and Distributed Machine Learning

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.

Apache Spark Sentry Integration

An Azure Databricks workshop leveraging the New York Taxi and Limousine Commission Trip Records dataset

TPC-H queries in Apache Spark SQL using native DataFrames API

人生苦短 我用Python

Open-source distribute workflow schedule tools, also support streaming task.

An extensive non-reactive Typescript framework that eases the development experience in Spark AR

ACID Data Source for Apache Spark based on Hive ACID

Resources for Spark AR

HL7 Apache Spark Datasource

Code examples for my blog posts

Visualize column-level data lineage in Spark SQL

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Implementation of Fellegi-Sunter's canonical model of record linkage in Apache Spark, including EM algorithm to estimate parameters

A parallel implementation of word2vec based on Spark

Spark functions to run popular phonetic and string matching algorithms

A Jupyter notebook using some standard techniques for data science and data engineering to analyze data for the 2017 flooding in Houston, TX.

Framework of vectorized and distributed data analytics

spark on kubernetes

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

Autoscaling for Google Cloud Dataproc

Distributed least squares approximation (dlsa) implemented with Apache Spark

Machine Learning Course @ Santa Clara University

A compiler for automatically re-targeting sequential Java code to Apache Spark.