443 Open source projects that are alternatives of or similar to Scanns

Spark Structured Streaming / Kafka / Cassandra / Elastic

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

Julia binding for Apache Spark

🌿 基于springboot的快速学习示例,整合自己遇到的开源框架,如：rabbitmq(延迟队列)、Kafka、jpa、redies、oauth2、swagger、jsp、docker、spring-batch、异常处理、日志输出、多模块开发、多环境打包、缓存cache、爬虫、jwt、GraphQL、dubbo、zookeeper和Async等等📌

Unified SQL Analytics Engine Based on SparkSQL

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.

Distributed t-SNE via Apache Spark

Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive

GeoTrellis for PySpark

There are pieces of scala code that explain Scala syntax and related things - like what you can do with all this

A minimal benchmark for scalability, speed and accuracy of commonly used open source implementations (R packages, Python scikit-learn, H2O, xgboost, Spark MLlib etc.) of the top machine learning algorithms for binary classification (random forests, gradient boosted trees, deep neural networks etc.).

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

A better compressed bitset in Java

Boiler plate framework to use Spark and ZIO together.

Fundamentals of Spark with Python (using PySpark), code examples

Apache Spark and Apache Kafka integration example

Spark、Flink等离线任务的调度以及实时任务的监控

Kinesis Connector for Structured Streaming

Updated repository

A pandas-like deferred expression system, with first-class SQL support

A RPC framework leveraging Spark RPC module

Spark RDD with Lucene's query and entity linkage capabilities

DataStax Spark Cassandra Connector

Nearest neighbor search for Rails and Postgres

Big Data Ecosystem Docker

extremely distributed machine learning

Pure python implementation of product quantization for nearest neighbor search

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

Version 1 of Technical Best Practices of Azure Databricks based on real world Customer and Technical SME inputs

Applying Lambda Architecture with Spark, Kafka, and Cassandra.

A Spark SQL extension which provides SQL Standard Authorization for Apache Spark

java 进阶之路：面试高频算法、akka、多线程、NIO、Netty、SpringBoot、Spark&&Flink 等

Simple and beautiful card component with an elegant spark line, for VueJS.

Spark SQL index for Parquet tables

A repository to keep track of all the code that I end up writing for my blog posts.

🐍 Quick reference guide to common patterns & functions in PySpark.

State of the Art Natural Language Processing

flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

A Linux based software package to control led's on Logitech G910, G810, G610 and G410.

An open-source toolkit for large-scale genomic analysis

Source code for James Lee's Aparch Spark with Java course

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.

Spark Terasort

A Java library implementing practical nearest neighbour search algorithm for multidimensional vectors that operates in sublinear time. It implements Locality-sensitive Hashing (LSH) and multi index hashing for hamming space.

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

A Scala kernel for Jupyter

HandySpark - bringing pandas-like capabilities to Spark dataframes

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Node.js bindings for Apache Spark DataFrame APIs

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Avro SerDe for Apache Spark structured APIs.

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

Realtime calculation distributed system. AKA distributed lodash

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

💥 🚀 封装sparkstreaming动态调节batch time(有数据就执行计算)；🚀 支持运行过程中增删topic；🚀 封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

LearningApacheSpark