redis-connect-distReal-Time Event Streaming & Change Data Capture
Stars: ✭ 21 (+5%)
Mutual labels: etl, etl-framework, etl-pipeline, etl-automation
vixtractwww.vixtract.ru
Stars: ✭ 40 (+100%)
Mutual labels: etl, etl-framework, etl-pipeline, etl-automation
csvpluscsvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.
Stars: ✭ 67 (+235%)
Mutual labels: etl, etl-framework, etl-pipeline
Data-Warehouse-Automation-Metadata-SchemaGeneric interface exchange format for Data Warehouse Automation and ETL generation.
Stars: ✭ 26 (+30%)
Mutual labels: datawarehouse, etl-automation, datawarehouseautomation
BETL-oldBETL. Meta data driven ETL generation using T-SQL
Stars: ✭ 17 (-15%)
Mutual labels: etl, etl-framework, etl-automation
hamiltonA scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.
Stars: ✭ 612 (+2960%)
Mutual labels: etl, etl-framework, etl-pipeline
etlflowEtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.
Stars: ✭ 38 (+90%)
Mutual labels: etl, etl-framework, etl-pipeline
datalake-etl-pipelineSimplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
Stars: ✭ 39 (+95%)
Mutual labels: etl, etl-framework, etl-pipeline
DaFlowApache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (+20%)
Mutual labels: etl, etl-framework, etl-pipeline
EtlboxA lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.
Stars: ✭ 203 (+915%)
Mutual labels: etl, etl-framework
ChoetlETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Stars: ✭ 372 (+1760%)
Mutual labels: etl, etl-framework
AirflowETLBlog post on ETL pipelines with Airflow
Stars: ✭ 20 (+0%)
Mutual labels: etl, etl-pipeline
MetorikkuA simplified, lightweight ETL Framework based on Apache Spark
Stars: ✭ 361 (+1705%)
Mutual labels: etl, etl-framework
qweryA SQL-like language for performing ETL transformations.
Stars: ✭ 28 (+40%)
Mutual labels: etl, etl-framework
EtlalchemyExtract, Transform, Load: Any SQL Database in 4 lines of Code.
Stars: ✭ 460 (+2200%)
Mutual labels: etl, etl-framework
TEAMThe Taxonomy for ETL Automation Metadata (TEAM) is a metadata management tool for data warehouse automation. It is part of the ecosystem for data warehouse automation, alongside the Virtual Data Warehouse pattern manager and the generic schema for Data Warehouse Automation.
Stars: ✭ 27 (+35%)
Mutual labels: etl, datawarehouseautomation
Pyetlpython ETL framework
Stars: ✭ 33 (+65%)
Mutual labels: etl, etl-framework
StetlStetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.
Stars: ✭ 64 (+220%)
Mutual labels: etl, etl-framework
kafka-connect-datagenA Kafka Connect source connector that generates data for tests
Stars: ✭ 27 (+35%)
Mutual labels: etl, etl-pipeline
Getting StartedThis repository is a getting started guide to Singer.
Stars: ✭ 734 (+3570%)
Mutual labels: etl, etl-framework
