1887 Open source projects that are alternatives of or similar to Repository

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

tools for bigData

网站点击流离线日志分析

Code and presentation for Strata Model Serving tutorial

A pandas-like deferred expression system, with first-class SQL support

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

Mapreduce project by Hadoop, Nutch, AWS EMR, Pig, Tez, Hive

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Connect Spark to HBase for reading and writing data with ease

Surging is a micro-service engine that provides a lightweight, high-performance, modular RPC request pipeline. The service engine supports http, TCP, WS,Grpc, Thrift,Mqtt, UDP, and DNS protocols. It uses ZooKeeper and Consul as a registry, and integrates it. Hash, random, polling, Fair Polling as a load balancing algorithm, built-in service gove…

Create clusters of VMs on the cloud and configure them with Ansible.

Data Lineage Tracking And Visualization Solution

Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.

Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark

Cascading is a feature rich API for defining and executing complex and fault tolerant data processing flows locally or on a cluster. See https://github.com/Cascading/cascading for the release repository.

Gathers scalable tensorflow and infrastructure deployment

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

Zenko is the open source multi-cloud data controller: own and keep control of your data on any cloud.

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

📚 📖 📚CSE GATE Resources for GATE and CSE Aspirants 😎 😁 . Show your ❤️ by ⭐️⭐️

Ytk-learn is a distributed machine learning library which implements most of popular machine learning algorithms(GBDT, GBRT, Mixture Logistic Regression, Gradient Boosting Soft Tree, Factorization Machines, Field-aware Factorization Machines, Logistic Regression, Softmax).

🚢 Docker image for Apache Spark

C++ implementations of well-known (and some rare) algorithms, while following good software development practices

Apache Hive

Generic Graph Data Structures and Algorithms for .NET

Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark

Building Large-Scale AI Applications for Distributed Big Data

Web tool for Kafka Connect |

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.

全栈工程师手册

Iceberg is a table format for large, slow-moving tabular data

一个大数据架构师应该掌握的技能

【咕泡学院实战项目】-基于SpringBoot+Dubbo构建的电商平台-微服务架构、商城、电商、微服务、高并发、kafka、Elasticsearch

Moonbox is a DVtaaS (Data Virtualization as a Service) Platform

A Scala feature transformation library for data science and machine learning

Web UI for Trino, Presto, Hive, Elasticsearch, SparkSQL

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

🎉🎉🎉JAVA高级架构师技术栈==任何技能通过 “刻意练习” 都可以达到融会贯通的境界，就像烹饪一样，这里有一份JAVA开发技术手册，只需要增加自己练习的次数。🏃🏃🏃

Java源码分析 【源码笔记】专注于Java后端系列框架的源码分析，每周持续推出Java后端系列框架的源码分析文章。

🔥 Crack you JS interviews ⚡ Collection of most common JS Interview questions with Unit Tests 🚀

180+ Algorithm & Data Structure Problems using C++

Generic Data Ingestion & Dispersal Library for Hadoop

Yet Another UserAgent Analyzer

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

An open source framework for building data analytic applications.

✍️ Java 工程师必备架构体系知识总结：涵盖分布式、微服务、RPC等互联网公司常用架构，以及数据存储、缓存、搜索等必备技能

程序员进阶书籍（视频），持续更新（Programmer Books）

Algorithms and Data Structures in Java

An efficient implementation of tree data structure in python 2/3.

Alluxio, data orchestration for analytics and machine learning in the cloud

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

A cohesive & pragmatic framework of FP centric Scala libraries

Spark RDD to read, write and delete from HBase

📚 Java 技术体系面试指南 , 旨在锻炼学习方法论的技术指南 🚀 数学，算法，基础框架，原理剖析，职业感悟，技术面试

📌 📚 Solution of competitive programming problems, code templates, Data Structures and Algorithms, hackathons, interviews and much more.

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

电商用户行为分析大数据平台

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.