1957 Open source projects that are alternatives of or similar to Wedatasphere

Open-source IoT Platform - Device management, data collection, processing and visualization.

Mirror of Linkedin's Camus

MySQL to NoSQL real time dataflow

Real Time Analytics and Data Pipelines based on Spark Streaming

Create clusters of VMs on the cloud and configure them with Ansible.

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

Connect Spark to HBase for reading and writing data with ease

equivalent to kafka-connect 🔧 for nodejs ✨🐢🚀✨

The purpose of this tiny project is to put things together with the know how that i learned from the course big data expert from formacionhadoop.com The idea is to show how to play with apache spark streaming, kafka,mongo, spark machine learning algorithms.

Streaming data changes to a Data Lake with Debezium and Delta Lake pipeline

💥 A simple multi-purpose connection pool client (Kafka & Hbase & Redis & RMDB & Socket & Http)

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

Avro SerDe for Apache Spark structured APIs.

Apache Spark and Apache Kafka integration example

Spark Structured Streaming / Kafka / Cassandra / Elastic

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Write your Spark data to Kafka seamlessly

A simplified, lightweight ETL Framework based on Apache Spark

大数据分析可视化实践

An orchestration platform for the development, production, and observation of data assets.

Machine Learning Platform and Recommendation Engine built on Kubernetes

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

推荐项目（实时推荐和离线推荐）

550+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Kafka, Docker, APIs, Hadoop, SQL, PostgreSQL, MySQL, Hive, Impala, Travis CI, Jenkins, Concourse, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, .tmux.conf, .psqlrc ...

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

A daily digest of the articles or videos I've found interesting, that I want to share with you.

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Spring Boot 2.x code examples

Apache Phoenix / Hbase Spring Boot Microservices

Thain is a distributed flow schedule platform.

Gathers scalable tensorflow and infrastructure deployment

Smart Automation Tool for building modern Data Lakes and Data Pipelines

大数据学习笔记，学习路线，技术案例整理。

同步Hive数据仓库数据到Elasticsearch的小工具

大数据采集,抽取平台

Hive Storage Handler for interoperability between BigQuery and Apache Hive

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

Fancy stream processing made operationally mundane

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

基于hadoop+hbase+springboot实现分布式网盘系统

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Liquibase extension to add Impala Database support

An alternative to the "hive standalone" jar for connecting Java applications to Apache Hive via JDBC

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

Avro Schema Evolution made easy

Spark RDD to read, write and delete from HBase

tools for bigData

Manage Apache Atlas and Ranger configuration for your Hadoop environment.

Management and automation platform for Stateful Distributed Systems

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

A set of commands for managing CDH clusters using Cloudera Manager REST API.

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Today I Learned