1352 Open source projects that are alternatives of or similar to Real Time Stream Processing Engine

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Apache Spark and Apache Kafka integration example

深圳地铁大数据客流分析系统🚇🚄🌟

flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

酷玩 Spark: Spark 源代码解析、Spark 类库等

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

Real Time Analytics and Data Pipelines based on Spark Streaming

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Custom state store providers for Apache Spark

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

C# and F# language binding and extensions to Apache Spark

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

A guide on how to set up Jupyter with Pyspark painlessly on AWS EC2 clusters, with S3 I/O support

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

基于Gradle构建，使用SpringBoot在各个场景的应用，包括集成消息中间件、前后端分离、数据库、缓存、分布式锁、分布式事务等

Web tool for Kafka Connect |

Plot live-stats as graph from ApacheSpark application using Lightning-viz

Haskell on Apache Spark.

Scala examples for learning to use Spark

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Interactive and Reactive Data Science using Scala and Spark.

The Hunting ELK

用java实现一下Logstash的几个常用input/filter/output, 希望能有效率上面的大提升. 现在我们迁移到golang了 https://github.com/childe/gohangout

The Internals of Spark Structured Streaming

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

Test your code without writing mocks with ephemeral Docker containers 📦 Setup popular services with just a couple lines of code ⏱️ No bash, no yaml, only code 💻

Spark structured streaming examples with using of version 3.0.0

A list about Apache Kafka

🎉🎉🎉JAVA高级架构师技术栈==任何技能通过 “刻意练习” 都可以达到融会贯通的境界，就像烹饪一样，这里有一份JAVA开发技术手册，只需要增加自己练习的次数。🏃🏃🏃

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

零基础学习spark，大数据学习

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Web UI for Trino, Presto, Hive, Elasticsearch, SparkSQL

【咕泡学院实战项目】-基于SpringBoot+Dubbo构建的电商平台-微服务架构、商城、电商、微服务、高并发、kafka、Elasticsearch

大数据生态解决方案数据平台：基于大数据、数据平台、微服务、机器学习、商城、自动化运维、DevOps、容器部署平台、数据平台采集、数据平台存储、数据平台计算、数据平台开发、数据平台应用搭建的大数据解决方案。

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

golang版本的hangout, 希望能省些内存. 使用了自己写的Kafka lib .. 虚. 不过我们在生产环境已经使用近1年, kafka 版本从0.9.0.1到2.0都在使用, 目前情况稳定. 吞吐量在每天2000亿条以上.

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Streaming System 相关的论文读物

Kafka Connect Elasticsearch connector

Easy CPU Profiling for Apache Spark applications

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

程序员进阶书籍（视频），持续更新（Programmer Books）

A cohesive & pragmatic framework of FP centric Scala libraries

Streaming reference architecture for ETL with Kafka and Kafka-Connect. You can find more on http://lenses.io on how we provide a unified solution to manage your connectors, most advanced SQL engine for Kafka and Kafka Streams, cluster monitoring and alerting, and more.

A Flexible and Powerful Parameter Server for large-scale machine learning

🐂基于springcloud +dubbo构建的模拟秒杀项目，模块化设计，集成了分库分表、elasticsearch🔍、gateway、mybatis-plus、spring-session等常用开源组件

✍️ Java 工程师必备架构体系知识总结：涵盖分布式、微服务、RPC等互联网公司常用架构，以及数据存储、缓存、搜索等必备技能

Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

R interface for Apache Spark

Simple learning project pushing CSV data into Kafka then indexing the data in ElasticSearch

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料