371 Open source projects that are alternatives of or similar to csvplus

www.vixtract.ru

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

Real-Time Event Streaming & Change Data Capture

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

BETL. Meta data driven ETL generation using T-SQL

A lightweight stream processing library for Go

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

Production Ready Data Integration Product, documentation：

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

A SQL-like language for performing ETL transformations.

An extensible Java framework for building XML and non-XML streaming applications

🐟 A streaming ETL for fish

A tool for building feature stores.

Configurable Extract, Transform, and Load

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

Configurable data bridge for permanent ETL jobs

A visual ETL development and debugging tool for big data

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Extract, Transform, Load: Any SQL Database in 4 lines of Code.

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes

python ETL framework

This repository is a getting started guide to Singer.

Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.

A Python stream processing engine modeled after Yahoo! Pipes

mito ETL tool

Bender - Serverless ETL Framework

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

A simplified, lightweight ETL Framework based on Apache Spark

Fancy stream processing made operationally mundane

A Kafka Connect source connector that generates data for tests

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

Blog post on ETL pipelines with Airflow

seatunnel plugin developing examples.

Yet Another ETL in PHP

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

An Express Swift Client of Apache Kafka 0.8, the Stream Processing Platform

Flock: A Low-Cost Streaming Query Engine on FaaS Platforms

This PHP script converts IP2Location CSV database into IP range or CIDR format.

This repository contains the code base for the Open Stream Processing Benchmark.

proc(ess)-that - easy extendable ETL tool for Node.js. Written in TypeScript.

Simple examle for Spark Streaming over Kafka topic

A library to compute histograms on distributed environments, on streaming data

CSV Parser and Writer as VBA functions

An open source, cloud-native streaming data integration and analytics product optimized for agile digital businesses

The purpose of this project is to apply mediapipe to more AI chips.

数据采集平台zdh,etl 处理服务

A high-performance observability data pipeline.

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data.

Shell script to run OpenRefine in batch mode (import, transform, export). It orchestrates OpenRefine (server) and a python client that communicates with the OpenRefine API.

Collection of stream processing / multiplexing / networking libs in Go

大数据采集,抽取平台

ETL flow framework based on Yaml configs in Python

SwiftBuilder is a fast way to assign new value to the property of the object.

Stream Processing Made Easy

Open Source Phasor Data Concentrator