536 Open source projects that are alternatives of or similar to pyspark-cheatsheet

A converter for the OSM PBFs to Parquet files

Apache Drill is a distributed MPP query layer for self describing data

No description or website provided.

CMAK is a tool for managing Apache Kafka clusters

Spark cloud integration: tests, cloud committers and more

Old repo for R interface for GraphFrames

Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR)

An ORC file format reader and writer for Go.

The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.

An open-source, real-time Security Information & Event Management tool based on big data technologies, providing a scalable, advanced security analytics framework.

Asakusa Framework

Tree diagrams with JavaScript 🌲 📈

Apache Phoenix Query Server

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

📊 📋 Dashboards using YAML or JSON files

Convert large amounts of data array to tree fastly

Repository for integration with Apache Kafka

Example end to end data engineering project.

Mirror of Apache Airavata PHP Gateway

ROOT I/O in pure Python and NumPy.

repo for code published on pythondata.com

A simple Spark-powered ETL framework that just works 🍺

Spark in Action, 2nd edition - chapter 1 - Introduction

Apache Spark Website

Big Data Management and Analysis Final Project

Apache Bookkeeper

A boilerplate project for Azure Big Data PaaS services

Big Data for Data Engineers Coursera Specialization from Yandex

Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes

Countly Product Analytics SDK for Cordova, Icenium and Phonegap

Basic framework utilities to quickly start writing production ready Apache Spark applications

Hazelcast IMDG C++ Client

A distributed block-based data storage and compute engine

The Proxima platform.

Mirror of Apache Ambari

RSparkling: Use H2O Sparkling Water from R (Spark + R + Machine Learning)

Apache Beam Site

Apache Spark 官方文档中文版

C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.

This repo contains commands that data engineers use in day to day work.

Mirror of Apache Lens

PredictionIO Python SDK

A Spark cluster setup running on Docker containers

PySpark phonetic and string matching algorithms

A native Node.js producer and consumer library for Amazon Kinesis Data Streams

Astronomy Broker based on Apache Spark

Mirror of Apache OODT

Query engine for TrailDB

Bullet is a streaming query engine that can be plugged into any singular data stream using a Stream Processing framework like Apache Storm, Spark or Flink.

MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)

Toolkit for highly memory efficient analysis of single-cell RNA-Seq, scATAC-Seq and CITE-Seq data. Analyze atlas scale datasets with millions of cells on laptop.

Distributed Big Data Orchestration Service

A curated list of big data engineering tools, resources and communities.

Two-day workshop that covers how to use R to interact databases and Spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/

Service for automatically managing and cleaning up unreferenced data

R on Apache Spark (SparkR) tutorials for Big Data analysis and Machine Learning as IPython / Jupyter notebooks

PredictionIO Java SDK

Ultimate Tennis Statistics and Tennis Crystal Ball - Tennis Big Data Analysis and Prediction

High-scalability analytics database built on immutable merkle-trees