633 Open source projects that are alternatives of or similar to SparkProgrammingInScala

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Serverless proxy for Spark cluster

Ppts, códigos y videos de las meetups, data science days, videollamadas y workshops. Data Science Research es una organización sin fines de lucro que busca difundir, descentralizar y difundir los conocimientos en Ciencia de Datos e Inteligencia Artificial en el Perú, dando oportunidades a nuevos talentos mediante MeetUps, Workshops y Semilleros …

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

Drools processor for Apache NiFi

🏐 Apache Parquet for modern .NET

Structural Bioinformatics Training Workshop & Hackathon 2018

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

Countly Product Analytics SDK for Cordova, Icenium and Phonegap

ROOT I/O in pure Python and NumPy.

Ultimate Tennis Statistics and Tennis Crystal Ball - Tennis Big Data Analysis and Prediction

ROOT I/O in pure Python and NumPy.

R on Apache Spark (SparkR) tutorials for Big Data analysis and Machine Learning as IPython / Jupyter notebooks

Project for James' Apache Spark with Scala course

Simple and Distributed Machine Learning

Fundamentals of Spark with Python (using PySpark), code examples

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

No description or website provided.

A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.

Full featured TypeScript Twitter archive reader and browser

Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

可视化大屏解决方案, 提供一套可视化编辑引擎, 助力个人或企业轻松定制自己的可视化大屏应用.

Simple and Distributed Machine Learning

Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

d3 library to build circular graphs

Big Data for Data Engineers Coursera Specialization from Yandex

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

大数据入门指南 ⭐

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Distributed Big Data Orchestration Service

SQL Parsers for BigData, built with antlr4.

Simple windows desktop application for viewing & querying Apache Parquet files

A visual ETL development and debugging tool for big data

🐳 big data study

编程路上加油站！------【持续更新中...欢迎star,欢迎常回来看看......】【内容：编程/学习/阅读资源，开源项目,面试题,网站,书,博客,教程等等】

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

bigdatatutorial

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.

Apache Spark on AWS Lambda

bring sf to spark in production

curated list of awesome tools and libraries for specific domains

An Apache Spark standalone application using the Spark API in Scala. The application uses Simple Build Tool(SBT) for building the project.

Plot live-stats as graph from ApacheSpark application using Lightning-viz

Splash, a flexible Spark shuffle manager that supports user-defined storage backends for shuffle data storage and exchange

Griffon Data Science Virtual Machine

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Bulletproof Apache Spark jobs with fast root cause analysis of failures.

The Internals of Spark SQL