571 Open source projects that are alternatives of or similar to Metl

Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.

⚙️ Schedule notebooks, run them like APIs, expose securely your assets: Jupyter as a viable ⚡️ Production environment

A visual ETL development and debugging tool for big data

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

A SQL-like language for performing ETL transformations.

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

www.vixtract.ru

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

Real-Time Event Streaming & Change Data Capture

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Extract, Transform, Load: Any SQL Database in 4 lines of Code.

Generate beautiful documentation for your data pipelines in markdown format

🇨🇳 MyDataHarbor是一个致力于解决任意数据源到任意数据源的分布式、高扩展性、高性能、事务级的数据同步中间件。帮助用户可靠、快速、稳定的对海量数据进行准实时增量同步或者定时全量同步，主要定位是为实时交易系统服务，亦可用于大数据的数据同步（ETL领域）。

Bender - Serverless ETL Framework

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

BETL. Meta data driven ETL generation using T-SQL

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

A tool for building feature stores.

Configurable data bridge for permanent ETL jobs

Configurable Extract, Transform, and Load

A lightweight data processing framework for Apache Spark

This repository is a getting started guide to Singer.

A lightweight stream processing library for Go

python ETL framework

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

M-Lab ingestion pipeline

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

A simplified, lightweight ETL Framework based on Apache Spark

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

ETL Library for Machine Learning - data pipelines, data munging and wrangling

A simple Spark-powered ETL framework that just works 🍺

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

Let's get the ball rolling on some Container-driven CI & CD

Data processing & ETL framework for Ruby

Turn Java 8 Streams into Unix like pipelines

Deploy machine learning projects developed in Python, to Kubernetes. Accelerated MLOps 🚀

基于spring boot通过java代码调用kette

A SingleCell RNASeq pre-processing snakemake workflow

Lightweight, Python library for fast and reproducible experimentation 🔬

Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more

Xenomorph Crawler, a Concise, Declarative and Observable Distributed Crawler(Node / Go / Java / Rust) For Web, RDB, OS, also can act as a Monitor(with Prometheus) or ETL for Infrastructure 💫 多语言执行器，分布式爬虫

c++ LINQ -like library of higher-order functions for data manipulation

A Julia package for piping a value through a series of transformation expressions using a more convenient syntax than Julia's native piping functionality.

Modern WebGL state wrapper for PEX: allocate GPU resources (textures, buffers), setup state pipelines and passes, and combine them into commands.

System for containerized apps management. From build to scaling.

Mass processing data with a complete ETL for .net developers

DataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server

Puppet Container Registry

Demo of Jenkins Configuration-As-Code with Docker and Groovy Hook Scripts

all kinds of scrapy demo

The MarkLogic Data Hub: documentation ==>