683 Open source projects that are alternatives of or similar to Dpark

Content for architecting a data science platform for products using Luigi, Spark & Flask.

Apache Spark - A unified analytics engine for large-scale data processing

📊 Cube — Open-Source Analytics API for Building Data Apps

AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.

The Apache Flink SQL Cookbook is a curated collection of examples, patterns, and use cases of Apache Flink SQL. Many of the recipes are completely self-contained and can be run in Ververica Platform as is.

Plot live-stats as graph from ApacheSpark application using Lightning-viz

Spark RDD with Lucene's query and entity linkage capabilities

A Time Series Library for Apache Spark

A Clojure dataframe library that runs on Spark

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

Training models with Apache Spark, PySpark for Titanic Kaggle competition

Building event-driven applications the easy way in Go.

liteflow是一个基于任务版本来实现的分布式任务流调度系统

A heterogeneous Apache Spark framework.

MATLAB library for non-negative matrix factorization (NMF): Version 1.8.1

Distributed Big Data Orchestration Service

Distributed Stream and Batch Processing

A scalable nearest neighbor search library in Apache Spark

50+ DockerHub public images for Docker & Kubernetes - Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak, TeamCity and DevOps tools built on the major Linux distros: Alpine, CentOS, Debian, Fedora, Ubuntu

Distributed Deep learning with Keras & Spark

最全的Java技术知识点，以及Java源码分析。为开源贡献自己的一份力。

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

List of Data Science Cheatsheets to rule the world

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

Demo for Redis Streams

A Spark library for Amazon SageMaker.

Java Convolutional Neural Network example for Hand Writing Digit Recognition

java 进阶之路：面试高频算法、akka、多线程、NIO、Netty、SpringBoot、Spark&&Flink 等

10-weeks of technology exploration

Julia binding for Apache Spark

▁▂▄▆▇█▇▆▄▂▁

An end-to-end machine learning and data mining framework on Hadoop

Log Anomaly Detection - Machine learning to detect abnormal events logs

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

技术书籍等

Version 1 of Technical Best Practices of Azure Databricks based on real world Customer and Technical SME inputs

深圳地铁大数据客流分析系统🚇🚄🌟

基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法

Live-coding the cluster!

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Spark SQL index for Parquet tables

Simple-IT-English: smart wordbook from community for community

Golang Kafka consumer and producer example

Optimize and improve the Label propagation algorithm

Monitor twitter stream

Stream Processing and Complex Event Processing Engine

CTR prediction model based on spark(LR, GBDT, DNN)

🐍 Quick reference guide to common patterns & functions in PySpark.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

A better compressed bitset in Java

"Sentiment analysis" on a live Twitter feed with Apache Spark and Apache Bahir

Spark Structured Streaming / Kafka / Cassandra / Elastic

Samsara is a real-time analytics platform

Research on distributed system

Next generation tool for decentralized exchange and transformation of semi-structured data

A luigi powered analytics / warehouse stack

Spark, Spark Streaming and Spark SQL unit testing strategies