997 Open source projects that are alternatives of or similar to Stetl

mito ETL tool

ETL Library for Machine Learning - data pipelines, data munging and wrangling

Extract, Transform, Load: Any SQL Database in 4 lines of Code.

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

⚙️ Schedule notebooks, run them like APIs, expose securely your assets: Jupyter as a viable ⚡️ Production environment

A SQL-like language for performing ETL transformations.

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

A simple Spark-powered ETL framework that just works 🍺

python ETL framework

Real-Time Event Streaming & Change Data Capture

Generate beautiful documentation for your data pipelines in markdown format

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

BETL. Meta data driven ETL generation using T-SQL

Configurable data bridge for permanent ETL jobs

A tool for building feature stores.

A visual ETL development and debugging tool for big data

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

🇨🇳 MyDataHarbor是一个致力于解决任意数据源到任意数据源的分布式、高扩展性、高性能、事务级的数据同步中间件。帮助用户可靠、快速、稳定的对海量数据进行准实时增量同步或者定时全量同步，主要定位是为实时交易系统服务，亦可用于大数据的数据同步（ETL领域）。

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

A simplified, lightweight ETL Framework based on Apache Spark

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

🔮 Transform, query, and download geospatial data on the web.

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

www.vixtract.ru

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

A lightweight data processing framework for Apache Spark

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

This repository is a getting started guide to Singer.

Configurable Extract, Transform, and Load

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

Bender - Serverless ETL Framework

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

M-Lab ingestion pipeline

A lightweight stream processing library for Go

For extensive instructor led learning

Service Status and QoS Checker for OGC Web Services

EnsEMBL Hive - a system for creating and running pipelines on a distributed compute resource

基于ArcGIS API for JavaScript封装的专题图制图类库

🌐 R package to query openrouteservice.org

Morphing mod for Minecraft 1.12.2

Groovy pipeline jobs that build and test Container Linux with Jenkins

contains handy groovy workflow-libs scripts

A Geographic Information System (GIS) used by Ministério Público do Estado do Rio de Janeiro to show social, institutional and administrative data , based on React and Leaflet, interacting with a GeoServer back-end.

A python library to push ethereum blockchain data into an sql database.

Parsing .shp files in Julia

Command-line runner for Google Earth Engine Playground scripts

Example workflows for the drake R package

Specification for streaming massive heterogeneous 3D geospatial datasets 🌎

A realtime ETL engine

Data Stream Processing API for GO

🌅 Dawn is a lightweight task management and build tool for front-end and nodejs.

Introduction to GIS and mapping in R with the sf package

stream splicer with labels