998 Open source projects that are alternatives of or similar to kafka-compose

Word2Vec models with Twitter data using Spark. Blog:

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

🌐 Interactive Workshop on GeoAnalysis using PySpark

Apache Spark (PySpark) Practice on Real Data

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

A simple Spark TDD example

Updated repository

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Isolation Forest on Spark

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

A library that provides useful extensions to Apache Spark and PySpark.

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

Example project implementing best practices for PySpark ETL jobs and applications.

Jupyter magics and kernels for working with remote Spark clusters

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

🐍 Quick reference guide to common patterns & functions in PySpark.

pyspark🍒🥭 is delicious，just eat it!😋😋

HandySpark - bringing pandas-like capabilities to Spark dataframes

Process Common Crawl data with Python and Spark

Web tool for Kafka Connect |

"Sentiment analysis" on a live Twitter feed with Apache Spark and Apache Bahir

Some class materials for a data processing course using PySpark

Apache Spark Connector for Azure Cosmos DB

Fundamentals of Spark with Python (using PySpark), code examples

Simple and Distributed Machine Learning

LearningApacheSpark

MongoDB Kafka Connector

**Unofficial / Community** Kafka Connect MongoDB Sink Connector - Find the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

Relation Extraction using Deep learning(CNN)

Iceberg is a table format for large, slow-moving tabular data

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Training models with Apache Spark, PySpark for Titanic Kaggle competition

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

Avro SerDe for Apache Spark structured APIs.

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Spark 2.0 Python Machine Learning examples

State of the Art Natural Language Processing

Kafka Connect connector to stream data in real time from Twitter.

An avro converter for Kafka Connect without a Schema Registry

A convenient Elixir library to work with Avro schemas and Confluent® Schema Registry

🐦 Put your tweets/likes in an EPUB and delete them like a boss

❄️ 🐳 Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.

ACID Data Source for Apache Spark based on Hive ACID

CakePHP link generator for sharing content on social networks

An Azure Databricks workshop leveraging the New York Taxi and Limousine Commission Trip Records dataset

A client UI to inspect Kafka topics, consume, produce and much more

Final Project for IoT: Big Data Processing and Analytics class. Analyzing U.S nationwide temperature from IoT sensors in real-time

A Twitter clone written in Python + Flask with extended features and a focus on inclusivity.

PHP client library for Twitter API v2 endpoints.

Apache Spark Sentry Integration