488 Open source projects that are alternatives of or similar to Spark Fast Tests

Run spark calculations from Ammonite

Tools for handling firmwares of DJI products, with focus on quadcopters.

A RPC framework leveraging Spark RPC module

A Scala feature transformation library for data science and machine learning

Spark 2.0 Python Machine Learning examples

Server for the ListenBrainz project

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

A framework for writing Spark 2.x applications in a pretty way

Generic Data Ingestion & Dispersal Library for Hadoop

Python clone of Spark, a MapReduce alike framework in Python

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops

一个大数据架构师应该掌握的技能

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

Bach Testing Framework

Cookbook to install Hadoop 2.0+ using Chef

Learn and understand Docker technologies, with real DevOps practice!

State of the Art Natural Language Processing

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

MLeap: Deploy ML Pipelines to Production

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

A Spark/Scala implementation of the isolation forest unsupervised outlier detection algorithm.

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

Kernel Test Framework - a unit test framework for the Linux kernel

Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

A simple Spark-powered ETL framework that just works 🍺

A modern Python test framework designed to help you find and fix flaws faster.

Apache Spark on AWS Lambda

Qubole Sparklens tool for performance tuning Apache Spark

ApacheCN 开源组织：公告、介绍、成员、活动、交流方式

An ad hoc query service based on the spark sql engine.(基于spark sql引擎的即席查询服务)

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Wirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.

type-class based data cleansing library for Apache Spark SQL

🧪 single header unit testing framework for C and C++

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

ClickHouse Native Protocol JDBC implementation

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

The Internals of Spark SQL

An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads.

Optimize and improve the Label propagation algorithm

Data Lineage Tracking And Visualization Solution

✔️ tf is a microframework for parameterized testing of functions and HTTP in Go.

A curated list of awesome resources related to the Ada and SPARK programming language

Research on distributed system

Connect Spark to HBase for reading and writing data with ease

Spark Structured Streaming / Kafka / Cassandra / Elastic

A luigi powered analytics / warehouse stack

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs

recheck for web apps – change comparison tool with local Golden Masters, Git-like ignore syntax and "Unbreakable Selenium" tests.

Version 1 of Technical Best Practices of Azure Databricks based on real world Customer and Technical SME inputs

Distributed t-SNE via Apache Spark

Two-day workshop that covers how to use R to interact databases and Spark

A free tutorial for Apache Spark.