713 Open source projects that are alternatives of or similar to Waterdrop

Apache Spark and Apache Kafka integration example

Kinesis Connector for Structured Streaming

🌟 ✨ Analyze and visualize Twitter Sentiment on a world map using Spark MLlib

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

extremely distributed machine learning

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Java开发者或者大数据开发者面试知识点整理

Big Data Ecosystem Docker

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Big Data Processing Framework - Unified Data API or SQL on Any Storage

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

大数据学习笔记，学习路线，技术案例整理。

🚢 Docker image for Apache Spark

大数据生态圈学习

大数据相关内容汇总，包括分布式存储引擎、分布式计算引擎、数仓建设等。关键词：Hadoop、HBase、ES、Kudu、Hive、Presto、Spark、Flink、Kylin、ClickHouse

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

An umbrella project for multiple implementations of model serving

A Flexible and Powerful Parameter Server for large-scale machine learning

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Different ways to process data into Cassandra in realtime with technologies such as Kafka, Spark, Akka, Flink

DataFibers Data Service

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

C# and F# language binding and extensions to Apache Spark

OLAP and ETL of Big Data

基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统

Code examples on Apache Spark using python

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

Data Lineage Tracking And Visualization Solution

A complete (distributed) BigData stack, running in containers

Code and presentation for Strata Model Serving tutorial

Ytk-learn is a distributed machine learning library which implements most of popular machine learning algorithms(GBDT, GBRT, Mixture Logistic Regression, Gradient Boosting Soft Tree, Factorization Machines, Field-aware Factorization Machines, Logistic Regression, Softmax).

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

A Docker container with a full Hadoop cluster setup with Spark and Zeppelin

一个大数据架构师应该掌握的技能

Generic Data Ingestion & Dispersal Library for Hadoop

Iceberg is a table format for large, slow-moving tabular data

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Alluxio, data orchestration for analytics and machine learning in the cloud

50+ DockerHub public images for Docker & Kubernetes - Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak, TeamCity and DevOps tools built on the major Linux distros: Alpine, CentOS, Debian, Fedora, Ubuntu

按知识领域整理面试题，包括C++、Java、Hadoop、机器学习等

An open source framework for building data analytic applications.

基于Spark2.2新闻网大数据实时系统项目

电商用户行为分析大数据平台

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

A big data platform for analyzing web access logs

This is an example of real time stream processing using Spark Streaming, Kafka & Elasticsearch.

Schema Registry

A pandas-like deferred expression system, with first-class SQL support

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Streaming System 相关的论文读物

MapReduce, Spark, Java, and Scala for Data Algorithms Book

java 进阶之路：面试高频算法、akka、多线程、NIO、Netty、SpringBoot、Spark&&Flink 等