1872 Open source projects that are alternatives of or similar to Datafusion

Fundamentals of Spark with Python (using PySpark), code examples

A javascript library providing a new data structure for datascientists and developpers

Distributed compute platform implemented in Rust, and powered by Apache Arrow.

Light-weight Python OLAP framework for multi-dimensional data analysis

SQL-based streaming analytics platform at scale

DeveelDB is a complete SQL database system, primarly developed for .NET/Mono frameworks

Clean APIs for data cleaning. Python implementation of R package Janitor

A converter that generates a bash one-liner from an SQL Select query (no DB necessary)

Apache Spark examples exclusively in Java

A Clojure dataframe library that runs on Spark

Read data from, write data to, and modify the formatting of Google Sheets

Apache Spark - A unified analytics engine for large-scale data processing

opensource distributed database with base JPA implementation and event processing support

Apache Spark Website

Next generation tool for decentralized exchange and transformation of semi-structured data

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Unified SQL Analytics Engine Based on SparkSQL

C# and F# language binding and extensions to Apache Spark

Spark SQL index for Parquet tables

Fast Data Cluster (Apache Cassandra, Kafka, Spark, Flink, YARN and HDFS with Vagrant and VirtualBox)

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

数据可视化分析平台，使用Java语言开发，采用浏览器/服务器架构，支持SQL、CSV、Excel、HTTP接口、JSON等多种数据源

Create clusters of VMs on the cloud and configure them with Ansible.

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Ahead of Time Data Repositories

KeyPathKit is a library that provides the standard functions to manipulate data along with a call-syntax that relies on typed keypaths to make the call sites as short and clean as possible.

A facebook for data

Parquet file generator

PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

Essential Spark extensions and helper methods ✨😲

locopy: Loading/Unloading to Redshift and Snowflake using Python.

Multiple node presto cluster on docker container

Pandas and Spark DataFrame comparison for humans

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Use Join Monster to fetch your data with Apollo Server.

Splitgraph command line client and python library

BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.

A connector for Spark that allows reading and writing to/from Redis cluster

Modin: Speed up your Pandas workflows by changing a single line of code

A curated list of amazingly awesome Cybersecurity datasets

Easy pipelines for pandas DataFrames.

A repository to keep track of all the code that I end up writing for my blog posts.

Fast multi-threaded DataFrame library in Rust | Python | Node.js

Apache Arrow DataFusion SQL Query Engine

Create full-fledged APIs for static datasets without writing a single line of code.

Go data analysis / manipulation library built on top of Apache Arrow

CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of data in real-time.

Lightweight, simple structured NoSQL database for Android

Qubole Sparklens tool for performance tuning Apache Spark

Koalas: pandas API on Apache Spark

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

Query processing and transformation of array-backed data tables.

Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark

Jupyter magics and kernels for working with remote Spark clusters

🐍 Quick reference guide to common patterns & functions in PySpark.

A Scala feature transformation library for data science and machine learning

A simplified, lightweight ETL Framework based on Apache Spark

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.