1012 Open source projects that are alternatives of or similar to redis-connect-dist

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

www.vixtract.ru

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

python ETL framework

A SQL-like language for performing ETL transformations.

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

BETL. Meta data driven ETL generation using T-SQL

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

Kafka Web UI

Fancy stream processing made operationally mundane

Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.

A simplified, lightweight ETL Framework based on Apache Spark

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

EventMesh is a dynamic event-driven application runtime used to decouple the application and backend middleware layer, which supports a wide range of use cases that encompass complex multi-cloud, widely distributed topologies using diverse technology stacks.

Bender - Serverless ETL Framework

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

Configurable data bridge for permanent ETL jobs

dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.

Configurable Extract, Transform, and Load

Postgresql To Kinesis For Java

♻ Replicates SharePoint Lists

A Kafka source connector capturing Scylla CDC changes

Production Ready Data Integration Product, documentation：

A Kafka Connect source connector that generates data for tests

A scalable Netflix DBLog implementation for PostgreSQL

This repository is a getting started guide to Singer.

Extract, Transform, Load: Any SQL Database in 4 lines of Code.

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

Mass processing data with a complete ETL for .net developers

mito ETL tool

A visual ETL development and debugging tool for big data

MySQL/MariaDB binlog replication client for .NET

A tool for building feature stores.

Logical Replication extension for PostgreSQL 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.

Applying RLS to PostgreSQL WAL

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

Blog post on ETL pipelines with Airflow

The CSV to database command line loader

ETL with Python - Taught at DWH course 2017 (TAU)

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

ETL flow framework based on Yaml configs in Python

seatunnel plugin developing examples.

A python library to push ethereum blockchain data into an sql database.

Build event-driven and event streaming applications with ease

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

A lib to transform data from jdbc,csv,json to ecah other.

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

Treats CSV and JSON files as SQL tables, and exports SQL SELECTs back to CSV or JSON.

☁️ nodejs kafka connect connector for Google BigQuery

an open-source data management platform for knowledge workers (https://github.com/dswarm/dswarm-documentation/wiki)

Simplified clustering module for PostgreSQL

Flight ticket booking system implemented with CQRS and ES.

Generates html and pdf invoices using html template files, CSV databases for products, clients, and transactions

Scraps historical stock market data from Yahoo Finance (https://finance.yahoo.com/)

A schema-aware Scala library for data transformation