281 Open source projects that are alternatives of or similar to arthur-redshift-etl

Data transformation framework for LinkML data models

an open-source data management platform for knowledge workers (https://github.com/dswarm/dswarm-documentation/wiki)

A collection of online resources to help you on your Tech journey.

Multi-core processing of 'Gameday' data from Major League Baseball Advanced Media. Additional tools to parallelize large data sets and write them to a database.

A data transformation package for deep learning with Autonomio, Keras and TensorFlow.

数据采集平台zdh,etl 处理服务

Flock: A Low-Cost Streaming Query Engine on FaaS Platforms

网站点击流离线日志分析

Enterprise Grade Single-Step Streaming Data Infrastructure Setup. (Under Development)

Starlake is a Spark Based On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing

GAMECHANGER aspires to be the Department’s trusted solution for evidence-based, data-driven decision-making across the universe of DoD requirements

ETL flow framework based on Yaml configs in Python

The practical use-cases of how to make your Machine Learning Pipelines robust and reliable using Apache Airflow.

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

ETL for the IEX Stocks API

proc(ess)-that - easy extendable ETL tool for Node.js. Written in TypeScript.

🇨🇳 MyDataHarbor是一个致力于解决任意数据源到任意数据源的分布式、高扩展性、高性能、事务级的数据同步中间件。帮助用户可靠、快速、稳定的对海量数据进行准实时增量同步或者定时全量同步，主要定位是为实时交易系统服务，亦可用于大数据的数据同步（ETL领域）。

Datart is a next generation Data Visualization Open Platform

Generate beautiful documentation for your data pipelines in markdown format

大数据采集,抽取平台

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

The OpenRefine Python Client from Paul Makepeace provides a library for communicating with an OpenRefine server. This fork extends the command line interface (CLI) and is distributed as a convenient one-file-executable (Windows, Linux, Mac). It is also available via Docker Hub, PyPI and Binder.

Python client for using Prefect Cloud with Saturn Cloud

bqloader is a simple ETL framework to load data from Cloud Storage into BigQuery.

⚙️ Schedule notebooks, run them like APIs, expose securely your assets: Jupyter as a viable ⚡️ Production environment

Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data sc…

Live import all your Google Sheets to your data warehouse

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

Dummy variable generation with fit/transform capabilities

Virtual Machine for Desktop Search with Open Semantic Search

Easy way to define, execute and store quality rules for your data.

A Pachyderm deep learning tutorial for conference workshops

DEPRECATED: Stream data processing micro-framework

A lightweight data processing framework for Apache Spark

Shell script to run OpenRefine in batch mode (import, transform, export). It orchestrates OpenRefine (server) and a python client that communicates with the OpenRefine API.

The Move to Islandora Kit is an extensible PHP command-line tool for converting source content and metadata into packages suitable for importing into Islandora (or other digital repository and preservations systems).

A curated list of big data engineering tools, resources and communities.

The Taxonomy for ETL Automation Metadata (TEAM) is a metadata management tool for data warehouse automation. It is part of the ecosystem for data warehouse automation, alongside the Virtual Data Warehouse pattern manager and the generic schema for Data Warehouse Automation.

A curated list of awesome system integration software and resources.

☁️ nodejs kafka connect connector for Google BigQuery

JDBC driver for Neo4j

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

A Kafka Connect source connector that generates data for tests

code, labs and lectures for the course

Less-Resilient MapReduce framework for Go

Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes

Python3 lib and cli for operating a Cardano Passive Node and using the API's. (PRE-ALPHA)

Summary of my projects on kaggle

A CLI and library to run Singer Taps and Targets

MySQL / MariaDB import for DTD feeds (fares, timetable and routeing)

A lib to transform data from jdbc,csv,json to ecah other.

CSVs sliced, diced & analyzed.

🔄 Simple tool for ETL. From SQL to Redis.

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

Personal Data Engineering Projects

📜 A CLI toolkit for extracting and working with your digital history

🗺 GIS plugins for Pentaho Data Integration

A dbt adapter for Databricks.

数据质量控制系统

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.