2813 Open source projects that are alternatives of or similar to Sparta

Kinesis Connector for Structured Streaming

☁️ Cloud-scale telemetry ingestion from any stream of data with Azure Event Hubs

Spark, Spark Streaming and Spark SQL unit testing strategies

A lightweight stream processing library for Go

📊 Cube — Open-Source Analytics API for Building Data Apps

大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

Streaming reference architecture for ETL with Kafka and Kafka-Connect. You can find more on http://lenses.io on how we provide a unified solution to manage your connectors, most advanced SQL engine for Kafka and Kafka Streams, cluster monitoring and alerting, and more.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Track website visitors with Serverless Analytics using Kinesis, Lambda, and TypeScript.

Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline

Apache Spark on AWS Lambda

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Hyperconverged cloud-edge native database

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

Mirror of Linkedin's Camus

Anomaly Detection on Time-Evolving Streams in Real-time. Detecting intrusions (DoS and DDoS attacks), frauds, fake rating anomalies.

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

In this repository, I have developed the entire server-side principal architecture for real-time stock market prediction with Machine Learning. I have used Tensorflow.js for constructing ml model architecture, and Kafka for real-time data streaming and pipelining.

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Massively real-time traffic streaming application

❄️ 🐳 Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

A social and synchronized streaming experience

Beneath is a serverless real-time data platform ⚡️

汇总java生态圈常用技术框架、开源中间件，系统架构、数据库、大公司架构案例、常用三方类库、项目管理、线上问题排查、个人成长、思考等知识

Fancy stream processing made operationally mundane

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of data in real-time.

An orchestration platform for the development, production, and observation of data assets.

♒︎ [WIP] An experimental ~distributed~ commit-log

Production Ready Data Integration Product, documentation：

Java生态研究(Spring Boot + Redis + Dubbo + RocketMQ + Elasticsearch)🔥🔥🔥🔥🔥

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Scala examples for learning to use Spark

Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.

Haskell on Apache Spark.

Streaming reactive and dataflow graphs in Python

Explore and visualize analytical datasets

Massively real-time city transit streaming application

DuckDB is an in-process SQL OLAP Database Management System

Fast & numerically stable statistical analysis

ClickHouse Native Protocol JDBC implementation

Flexible and efficient data processing engine and an evolution of the popular Scramjet Framework based on node.js. Our Transform Hub was designed specifically for data processing and has its own unique algorithms included.

A complete (distributed) BigData stack, running in containers

Materialize lets you ask questions of your live data, which it answers and then maintains for you as your data continue to change. The moment you need a refreshed answer, you can get it in milliseconds. Materialize is designed to help you interactively explore your streaming data, perform data warehousing analytics against live relational data, or just increase the freshness and reduce the load of your dashboard and monitoring tasks.

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

A data visualization and analytics component, especially well-suited for large and/or streaming datasets.

Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)

酷玩 Spark: Spark 源代码解析、Spark 类库等

An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads.

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

Web tool for Kafka Connect |

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Custom state store providers for Apache Spark

Minikube for big data with Scala and Spark