287 Open source projects that are alternatives of or similar to Etl2pcapng

A luigi powered analytics / warehouse stack

Bender - Serverless ETL Framework

Sync data between persistence engines, like ETL only not stodgy

Mass processing data with a complete ETL for .net developers

Actively curated list of awesome BI tools. PRs welcome!

Categorical Query Language IDE

Stetl, Streaming ETL, is a lightweight geospatial processing and ETL framework written in Python.

A tool for building feature stores.

Kiba enhancement for Ruby ETL.

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

A python library to push ethereum blockchain data into an sql database.

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

Public, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores

python ETL framework

A Python stream processing engine modeled after Yahoo! Pipes

🔬 An R package to work with PCAPs

Python based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data enrichment (annotation) pipelines & ingestor to Solr or Elastic search index & linked data graph database

Extract your data from the Yunmai weighing scales cloud API so you can use it elsewhere

the TCPdump network dissector

Detect threats with log data and improve cloud security posture

A cross-platform command line tool for parallelised content extraction and analysis.

🐟 A streaming ETL for fish

DataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server

An example mini data warehouse for python project stats, template for new projects

Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Run Airflow in AWS ECS(Elastic Container Service) using Fargate tasks

WindowsSpyBlocker 🛡️ is an application written in Go and delivered as a single executable to block spying and tracking on Windows systems.

This repository is a getting started guide to Singer.

为 Wireshark 能使用纯真网络 IP 数据库(QQwry)而提供的格式转换工具

React components to build CSV files on the fly basing on Array/literal object of data

mito ETL tool

Example project implementing best practices for PySpark ETL jobs and applications.

The CSV to database command line loader

A lightweight stream processing library for Go

Metl is a simple, web-based integration platform that allows for several different styles of data integration including messaging, file based Extract/Transform/Load (ETL), and remote procedure invocation via Web Services. Read more at www.jumpmind.com/products/metl/overview

A hackable data integration & analysis tool to enable non technical users to edit data processing jobs and visualise data on demand.

Česká otevřená data

Ostinato - Packet/Traffic Generator and Analyzer

A visual ETL development and debugging tool for big data

🔮 Transform, query, and download geospatial data on the web.

Use Pentaho's open source data integration tool (Kettle) to create Extract-Transform-Load (ETL) processes to update a Socrata open data portal. Documentation is available at http://open-data-etl-utility-kit.readthedocs.io/en/stable

SmartCode = IDataSource -> IBuildTask -> IOutput => Build Everything!!!

A lightweight ETL (extract, transform, load) library and data integration toolbox for .NET.

Logical Replication extension for PostgreSQL 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.

Daggy - Data Aggregation Utility. Open source, free, cross-platform, server-less, useful utility for remote or local data aggregation and streaming

The premier open source Data Quality solution

A highly configurable multi-threaded packet sniffer and parser build in rust-lang.

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

A simple Spark-powered ETL framework that just works 🍺

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

Linked Data & RDF Manufacturing Tools in Clojure

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

R client for the Elasticsearch HTTP API

Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.

Plug-and-play bash script for sniffing 802.11 probes requests 👃

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ