651 Open source projects that are alternatives of or similar to Big Whale

▁▂▄▆▇█▇▆▄▂▁

Apache Spark™ and Scala Workshops

Ruby wrapper for Apache Spark

A Spark library for Amazon SageMaker.

A Spark plugin for reading Excel files via Apache POI

Big Data Processing Framework - Unified Data API or SQL on Any Storage

MLOps Platform

Spark, Spark Streaming and Spark SQL unit testing strategies

k-Nearest Neighbors algorithm on Spark

Simple and Distributed Machine Learning

Apache Spark (PySpark) Practice on Real Data

Distributed compute platform implemented in Rust, and powered by Apache Arrow.

A scalable nearest neighbor search library in Apache Spark

Realtime calculation distributed system. AKA distributed lodash

Version 1 of Technical Best Practices of Azure Databricks based on real world Customer and Technical SME inputs

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

A better compressed bitset in Java

Self-contained examples of Apache Spark streaming integrated with Apache Kafka.

Unified SQL Analytics Engine Based on SparkSQL

Write your Spark data to Kafka seamlessly

A RPC framework leveraging Spark RPC module

Firely's open source FHIR server

State of the Art Natural Language Processing

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

Spark Structured Streaming / Kafka / Cassandra / Elastic

Isolation Forest on Spark

GeoTrellis for PySpark

Apache Spark Connector for Azure Cosmos DB

550+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Kafka, Docker, APIs, Hadoop, SQL, PostgreSQL, MySQL, Hive, Impala, Travis CI, Jenkins, Concourse, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, .tmux.conf, .psqlrc ...

A collection of pentest tools and resources targeting Hadoop environments

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

Apache Calcite

Facebook's Hive UDFs

An end-to-end machine learning and data mining framework on Hadoop

推荐项目（实时推荐和离线推荐）

实践源码库：https://github.com/jast90/bigdata 。 微信搜索Jast关注公众号，获取最新技术分享😯。

Apache Nutch is an extensible and scalable web crawler

Hive JDBC "uber" or "standalone" jar based on the latest Apache Hive version

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Apache Flink 中文文档

懒松鼠Flink-Boot 脚手架让Flink全面拥抱Spring生态体系，使得开发者可以以Java WEB开发模式开发出分布式运行的流处理程序，懒松鼠让跨界变得更加简单。懒松鼠旨在让开发者以更底上手成本（不需要理解分布式计算的理论知识和Flink框架的细节）便可以快速编写业务代码实现。为了进一步提升开发者使用懒松鼠脚手架开发大型项目的敏捷的度，该脚手架默认集成Spring框架进行Bean管理，同时将微服务以及WEB开发领域中经常用到的框架集成进来，进一步提升开发速度。比如集成Mybatis ORM框架，Hibernate Validator校验框架,Spring Retry重试框架等，具体见下面的脚手架特性。

🚁🚀基于Flink实现的商品实时推荐系统。flink统计商品热度，放入redis缓存，分析日志信息，将画像标签和实时记录放入Hbase。在用户发起推荐请求后，根据用户画像重排序热度榜，并结合协同过滤和标签两个推荐模块为新生成的榜单的每一个产品添加关联产品，最后返回新的用户列表。

The Apache Flink SQL Cookbook is a curated collection of examples, patterns, and use cases of Apache Flink SQL. Many of the recipes are completely self-contained and can be run in Ververica Platform as is.

Framework for Apache Flink unit tests

Schema Registry

Process authoring tool for Apache Flink

🐳基于 Flink 的商品实时推荐系统。使用了 redis 缓存热点数据。当用户产生评分行为时，数据由 kafka 发送到 flink，根据用户历史评分行为进行实时和离线推荐。实时推荐包括：基于行为和实时热门，离线推荐包括：历史热门、历史优质商品和 itemcf 。

Based on Apache Flink. support data synchronization/integration and streaming SQL computation.

Flink sink for Clickhouse

Alink is the Machine Learning algorithm platform based on Flink, developed by the PAI team of Alibaba computing platform.