217 Open source projects that are alternatives of or similar to link-move

A visual ETL development and debugging tool for big data

A SQL-like language for performing ETL transformations.

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

mito ETL tool

A tool for building feature stores.

This repository is a getting started guide to Singer.

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

Bender - Serverless ETL Framework

python ETL framework

A simplified, lightweight ETL Framework based on Apache Spark

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Real-Time Event Streaming & Change Data Capture

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

Extract, Transform, Load: Any SQL Database in 4 lines of Code.

Configurable Extract, Transform, and Load

BETL. Meta data driven ETL generation using T-SQL

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Configurable data bridge for permanent ETL jobs

Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.

www.vixtract.ru

Thain is a distributed flow schedule platform.

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

Linq to database provider.

📘 Making Data, the DataMade Way

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

Server application to serve U.S. federal spending data via a RESTful API

A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.

Python based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data enrichment (annotation) pipelines & ingestor to Solr or Elastic search index & linked data graph database

数据可视化, 数据挖掘, 数据处理 ETL

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

A minimalist (geospatial) ETL

Example DAGs using hooks and operators from Airflow Plugins

An example mini data warehouse for python project stats, template for new projects

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

A client for connecting and running DDLs on hive metastore.

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Big Data Toolkit for the JVM

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Utility that converts an .etl file containing a Windows network packet capture into .pcapng format.

基于spring boot通过java代码调用kette

Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more

Python based Wikidata framework for easy dataframe extraction

📜 A CLI toolkit for extracting and working with your digital history

NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile y…

R client for the Elasticsearch HTTP API

Mass processing data with a complete ETL for .net developers

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

Blog post on ETL pipelines with Airflow

Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

A Python stream processing engine modeled after Yahoo! Pipes

Categorical Query Language IDE

Data processing & ETL framework for Ruby