Top 58 spark-streaming open source projects

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Spark, Spark Streaming and Spark SQL unit testing strategies

Schema Registry

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Simple yet powerful live data computation framework

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

StreamLine - Streaming Analytics

Updated repository

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Apache Spark and Apache Kafka integration example

Kinesis Connector for Structured Streaming

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

Production Ready Data Integration Product, documentation：

Custom state store providers for Apache Spark

Code examples on Apache Spark using python

scala、spark使用过程中，各种测试用例以及相关资料整理

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

零基础学习spark，大数据学习

Repository for all parts of the Fortis architecture

Plot live-stats as graph from ApacheSpark application using Lightning-viz

Wormhole is a SPaaS (Stream Processing as a Service) Platform

C# and F# language binding and extensions to Apache Spark

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

A Flexible and Powerful Parameter Server for large-scale machine learning

Streaming System 相关的论文读物

Real Time Analytics and Data Pipelines based on Spark Streaming

An open source framework for building data analytic applications.

Scala examples for learning to use Spark

Stream computing platform for bigdata

酷玩 Spark: Spark 源代码解析、Spark 类库等

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Live Twitter sentiment analysis using Python, Apache Spark Streaming, Kafka, NLTK, SocketIO

Final Project for IoT: Big Data Processing and Analytics class. Analyzing U.S nationwide temperature from IoT sensors in real-time

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

Design/Implement stream/batch architecture on NYC taxi data | #DE

四川大学拓思艾诺广告流量实时分析项目

Basic framework utilities to quickly start writing production ready Apache Spark applications

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

code, labs and lectures for the course

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

Bitnami Docker Image for Apache Spark

大数据生态圈学习

a one-stop repo to lookup for code snippets of core java concepts, sql, data structures as well as big data. It also consists of interview questions asked in real-life.

seatunnel plugin developing examples.

This repository contains the code base for the Open Stream Processing Benchmark.

Real Time Twitter Sentiment Analysis Product

A real time analytics dashboard to analyze the trending hashtags and @ mentions at any location using kafka and spark streaming.

Counting Tweets Per User in Real-Time

基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

🐧基于spark streaming+flume+kafka+hbase的实时日志处理分析系统(分为控制台版本和基于springboot、Echarts等的Web UI可视化版本)

An umbrella project for multiple implementations of model serving

A general purpose metrics monitor implemented with Apache Spark. Kafka source, Elastic sink, aggregate metrics, different analysis, notifications, actions, live configuration update, missing metrics, ...

bigdatatutorial