666 Open source projects that are alternatives of or similar to Airflow Pipeline

https://www.researchgate.net/profile/Rajah_Iyer

A tool for monitoring and tuning Spark jobs for efficiency.

Distributed Deep learning with Keras & Spark

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

This application comes as Spark2.1-as-Service-Provider using an embedded, Reactive-Streams-based, fully asynchronous HTTP server

Example source code accompanying O'Reilly's "Hadoop: The Definitive Guide" by Tom White

Run spark calculations from Ammonite

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

Griffon Data Science Virtual Machine

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

Asakusa Framework

Spark Terasort

A light-weight distributed stream computing framework for Golang

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

Airflow basics tutorial

Spark examples

A curated list of awesome resources related to the Ada and SPARK programming language

Spark 2.0 Python Machine Learning examples

Apache Spark Connector for Azure Kusto

Connect Spark to HBase for reading and writing data with ease

Fast iterative local development and testing of Apache Airflow workflows

Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.

Python Helper library for Jupyter Notebooks

Sparkline BI Accelerator provides fast ad-hoc query capability over Logical Cubes. This has been folded into our SNAP Platform(http://bit.ly/2oBJSpP) an Integrated BI platform on Apache Spark.

Google Authenticator support for Laravel Spark

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

Spark Streaming监控平台，支持任务部署与告警、自启动

ETL Library for Machine Learning - data pipelines, data munging and wrangling

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

The Hunting ELK

Webex Bot SDK for Node.js (deprecated in favor of https://github.com/webex/webex-bot-node-framework)

🐘 Source code for assignments of Udacity course "Introduction to Hadoop and MapReduce"

Code snippets for solving common big data problems in various platforms. Inspired by Rosetta Code

A Java NIO file system provider for HDFS

Profile Photo Camera support for Laravel Spark

Airflow training for the crunch conf

本项目收藏这些年来看过或者听过的一些不错的书籍，在整理文件时看见这些，发现删掉有点可惜，放着又太浪费空间，本着分享的原则，就把它们共享出来，一方面给需要的读者提供这些书籍，另一方面也是一种像知识库的积累吧

Continuously synchronize directories from remote object store to local filesystem

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Open-source IoT Platform - Device management, data collection, processing and visualization.

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

An easy way to deploy your Akka services to a distributed environment.

dllib is a distributed deep learning library running on Apache Spark

DataFusion has now been donated to the Apache Arrow project

phData Pulse application log aggregation and monitoring

Jupyter magics and kernels for working with remote Spark clusters

No description or website provided.

java 进阶之路：面试高频算法、akka、多线程、NIO、Netty、SpringBoot、Spark&&Flink 等

FFM (Field-Awared Factorization Machine) on Spark

RSparkling: Use H2O Sparkling Water from R (Spark + R + Machine Learning)

Apache DolphinScheduler is a distributed and extensible workflow scheduler platform with powerful DAG visual interfaces, dedicated to solving complex job dependencies in the data pipeline and providing various types of jobs available out of box.

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

Google BigQuery support for Spark, Structured Streaming, SQL, and DataFrames with easy Databricks integration.

The MongoDB Spark Connector

Spark RDD with Lucene's query and entity linkage capabilities

Jumbune, an open source BigData APM & Data Quality Management Platform for Data Clouds. Enterprise feature offering is available at http://jumbune.com. More details of open source offering are at,

定期更新Hadoop生态圈中常用大数据组件文档 重心依次为: Flink Solr Sparksql ES Scala Kafka Hbase/phoenix Redis Kerberos (项目包含hadoop思维导图 印象笔记 Scala版本简单demo 常用工具类 去敏后的train code 持续更新!!!)