1018 Open source projects that are alternatives of or similar to Aliyun Emapreduce Datasources

Building Large-Scale AI Applications for Distributed Big Data

A cohesive & pragmatic framework of FP centric Scala libraries

A Spark cluster setup running on Docker containers

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

A big data platform for analyzing web access logs

Avro SerDe for Apache Spark structured APIs.

零基础学习spark，大数据学习

🚢 Docker image for Apache Spark

Java开发者或者大数据开发者面试知识点整理

Cookbook to install Hadoop 2.0+ using Chef

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

汇总java生态圈常用技术框架、开源中间件，系统架构、数据库、大公司架构案例、常用三方类库、项目管理、线上问题排查、个人成长、思考等知识

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Production Ready Data Integration Product, documentation：

Big Data Toolkit for the JVM

推荐项目（实时推荐和离线推荐）

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

low-level helpers for Apache Spark libraries and tests

Open-source distribute workflow schedule tools, also support streaming task.

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

A complete (distributed) BigData stack, running in containers

Gathers scalable tensorflow and infrastructure deployment

MapReduce, Spark, Java, and Scala for Data Algorithms Book

Mirror of Linkedin's Camus

Asakusa Framework

[https://www.sofineday.com], golang项目开发脚手架,集成最佳实践(gin+gorm+go-redis+mongo+cors+jwt+json日志库zap(支持日志收集到kafka或mongo)+消息队列kafka+微信支付宝支付gopay+api加密+api反向代理+go modules依赖管理+headless爬虫chromedp+makefile+二进制压缩+livereload热加载)

Spark RDD with Lucene's query and entity linkage capabilities

Support TensorFlow on YARN

This package contains a generic implementation of greedy Information Theoretic Feature Selection (FS) methods. The implementation is based on the common theoretic framework presented by Gavin Brown. Implementations of mRMR, InfoGain, JMI and other commonly used FS filters are provided.

Spring-Shiro-Spark是Spring-Boot Hibernate Spark Spark-SQL Shiro iView VueJs... ...的集成尝试

Distributed Java game server, including login, gateway, game demo

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.

Mirror of Apache Calcite - Avatica

Ephemeral Hadoop clusters using Google Compute Platform

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

Data science and Big Data with Python

PG数据同步工具（Java实现）

This repo contains my notes on working with Go and computer systems.

支持mysql、MongoDB数据变更订阅分发

Griffon Data Science Virtual Machine

Production-ready, stable Kafka client for PHP

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

Distributed Deep learning with Keras & Spark

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

Reliable messages resolve distributed transactions

Applying Lambda Architecture with Spark, Kafka, and Cassandra.

Metronome is a distributed and fault-tolerant event scheduler

An encrypted data analytics platform

A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [PRE-ALPHA]

Kafka Zookeeper RESTful API to perform topic/consumer group administration/metric(offset\lag\message) collection and monitor