1382 Open source projects that are alternatives of or similar to qwery

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

python ETL framework

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Real-Time Event Streaming & Change Data Capture

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

Zero Allocation Writer/Reader Parser for .NET Core

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.

Data Curator - share usable open data

🧚 pxi (pixie) is a small, fast, and magical command-line data processor similar to jq, mlr, and awk.

One interface to read and write the data in various excel formats, import the data into and export the data from databases

Faster CSV on Python 3

CSV validator, highlighter and formatter plugin for JetBrains Intellij IDEA, PyCharm, WebStorm, ...

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.

A minimalist (geospatial) ETL

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

Standalone alternatives to Kafka Connect Connectors

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

BETL. Meta data driven ETL generation using T-SQL

A SQLite driver for S3 and Amazon Athena 😳

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

A command line tool that draw plots on the terminal.

A lib to transform data from jdbc,csv,json to ecah other.

What's in your data? Extract schema, statistics and entities from datasets

A list of command line tools for manipulating structured text data

A CLI tool to convert CSV / Excel / HTML / JSON / Jupyter Notebook / LDJSON / LTSV / Markdown / SQLite / SSV / TSV / Google-Sheets to a SQLite database file.

q - Run SQL directly on CSV or TSV files

A cross-platform, efficient and practical CSV/TSV toolkit in Golang

WinMerge is an Open Source differencing and merging tool for Windows. WinMerge can compare both folders and files, presenting differences in a visual text format that is easy to understand and handle.

🦜RBQL - Rainbow Query Language: SQL-like language for (not only) CSV file processing. Supports SQL queries with Python and JavaScript expressions

dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.

CSV parser for Swift

A client for connecting and running DDLs on hive metastore.

🌀 .NET Event Processing library

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

www.vixtract.ru

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

A fully-featured AWS Athena database driver (+ athenareader https://github.com/uber/athenadriver/tree/master/athenareader)

JupyterLab spreadsheet editor for tabular data (e.g. csv, tsv)

Fast reading of delimited files

Kafka Examples focusing on Producer, Consumer, KStreams, KTable, Global KTable using Spring, Kafka Cluster Setup & Monitoring. Implementing Event Sourcing and CQRS Design Pattern using Kafka

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

Detects tabular data (spreadsheets, dsv or json, 20+ different formats) and emits normalized objects.

Service for automatically managing and cleaning up unreferenced data

Configurable data bridge for permanent ETL jobs

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

Scraps historical stock market data from Yahoo Finance (https://finance.yahoo.com/)

Code snippets that demonstrate how to leverage the new Kafka 0.11 APIs

Collection of scripts for use with Carbon Black Cb Response API

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

an open-source data management platform for knowledge workers (https://github.com/dswarm/dswarm-documentation/wiki)

📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.

Kafka End to End Encryption

Mapreduce project by Hadoop, Nutch, AWS EMR, Pig, Tez, Hive

Today I Learned

A client UI to inspect Kafka topics, consume, produce and much more

A terminal spreadsheet multitool for discovering and arranging data

pytablewriter is a Python library to write a table in various formats: CSV / Elasticsearch / HTML / JavaScript / JSON / LaTeX / LDJSON / LTSV / Markdown / MediaWiki / NumPy / Excel / Pandas / Python / reStructuredText / SQLite / TOML / TSV.