1712 Open source projects that are alternatives of or similar to Metorikku

Fundamentals of Spark with Python (using PySpark), code examples

A simple Spark-powered ETL framework that just works 🍺

Apache Spark - A unified analytics engine for large-scale data processing

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Apache Spark Website

A visual ETL development and debugging tool for big data

A Clojure dataframe library that runs on Spark

A framework for rapid reporting API development; with out of the box support for high cardinality dimension lookups with druid.

An example mini data warehouse for python project stats, template for new projects

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

ClickHouse® is a free analytics DBMS for big data

locopy: Loading/Unloading to Redshift and Snowflake using Python.

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources

Spark SQL index for Parquet tables

A Presto client for the Go programming language.

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

www.vixtract.ru

The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.

Stream computing platform for bigdata

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

☁️ nodejs kafka connect connector for Google BigQuery

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

❄️ 🐳 Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

Next generation tool for decentralized exchange and transformation of semi-structured data

Actively curated list of awesome BI tools. PRs welcome!

A python library to push ethereum blockchain data into an sql database.

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

ACID Data Source for Apache Spark based on Hive ACID

Framework of vectorized and distributed data analytics

Mirror of Apache Calcite - Avatica

The official home of the Presto distributed SQL query engine for big data

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

Unified SQL Analytics Engine Based on SparkSQL

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

Apache Calcite

Linq to database provider.

Datafuse is a free Cloud-Native Analytics DBMS(Inspired by ClickHouse) implemented in Rust

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

A distributed block-based data storage and compute engine

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

Real-Time Event Streaming & Change Data Capture

Configurable data bridge for permanent ETL jobs

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

A SQL-like language for performing ETL transformations.

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

A complete (distributed) BigData stack, running in containers

ETL Library for Machine Learning - data pipelines, data munging and wrangling

Enabling queries on compressed data.

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

An extensible Java framework for building XML and non-XML streaming applications

Parquet file generator

Mirror of Apache Phoenix

BETL. Meta data driven ETL generation using T-SQL