polygon-etlETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub
Stars: ✭ 53 (-20.9%)
ip2location-csv-converterThis PHP script converts IP2Location CSV database into IP range or CIDR format.
Stars: ✭ 26 (-61.19%)
openPDCOpen Source Phasor Data Concentrator
Stars: ✭ 109 (+62.69%)
Bitcoin EtlETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ
Stars: ✭ 174 (+159.7%)
xlstreamTurns XLSX into a readable stream.
Stars: ✭ 148 (+120.9%)
proc-thatproc(ess)-that - easy extendable ETL tool for Node.js. Written in TypeScript.
Stars: ✭ 25 (-62.69%)
distogramA library to compute histograms on distributed environments, on streaming data
Stars: ✭ 19 (-71.64%)
Example Airflow DagsExample DAGs using hooks and operators from Airflow Plugins
Stars: ✭ 243 (+262.69%)
VBA-CSVCSV Parser and Writer as VBA functions
Stars: ✭ 26 (-61.19%)
StoragetapperStorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service
Stars: ✭ 232 (+246.27%)
vectorA high-performance observability data pipeline.
Stars: ✭ 12,138 (+18016.42%)
ElasticR client for the Elasticsearch HTTP API
Stars: ✭ 227 (+238.81%)
zinggScalable identity resolution, entity resolution, data mastering and deduplication using ML
Stars: ✭ 655 (+877.61%)
Linq2dbLinq to database provider.
Stars: ✭ 2,211 (+3200%)
openrefine-batchShell script to run OpenRefine in batch mode (import, transform, export). It orchestrates OpenRefine (server) and a python client that communicates with the OpenRefine API.
Stars: ✭ 76 (+13.43%)
ExtractA cross-platform command line tool for parallelised content extraction and analysis.
Stars: ✭ 188 (+180.6%)
go-riversCollection of stream processing / multiplexing / networking libs in Go
Stars: ✭ 35 (-47.76%)
MetlMetl is a simple, web-based integration platform that allows for several different styles of data integration including messaging, file based Extract/Transform/Load (ETL), and remote procedure invocation via Web Services. Read more at www.jumpmind.com/products/metl/overview
Stars: ✭ 185 (+176.12%)
SwiftBuilderSwiftBuilder is a fast way to assign new value to the property of the object.
Stars: ✭ 26 (-61.19%)
GrafterLinked Data & RDF Manufacturing Tools in Clojure
Stars: ✭ 174 (+159.7%)
makinageStream Processing Made Easy
Stars: ✭ 31 (-53.73%)
CVparserCVparser is software for parsing or extracting data out of CV/resumes.
Stars: ✭ 28 (-58.21%)
sync-engine-exampleSynchronization Algorithm Exploration: Techniques to synchronize a SQL database with external destinations.
Stars: ✭ 17 (-74.63%)
csv-cruncherTreats CSV and JSON files as SQL tables, and exports SQL SELECTs back to CSV or JSON.
Stars: ✭ 32 (-52.24%)
awesome-integrationA curated list of awesome system integration software and resources.
Stars: ✭ 117 (+74.63%)
Usaspending ApiServer application to serve U.S. federal spending data via a RESTful API
Stars: ✭ 166 (+147.76%)
iex-stocksETL for the IEX Stocks API
Stars: ✭ 19 (-71.64%)
AirbyteAirbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.
Stars: ✭ 4,919 (+7241.79%)
fluentcheckFluent assertions for Python
Stars: ✭ 79 (+17.91%)
Open Semantic EtlPython based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data enrichment (annotation) pipelines & ingestor to Solr or Elastic search index & linked data graph database
Stars: ✭ 165 (+146.27%)
neo4j-jdbcJDBC driver for Neo4j
Stars: ✭ 110 (+64.18%)
Etl unicorn数据可视化, 数据挖掘, 数据处理 ETL
Stars: ✭ 156 (+132.84%)
django-data-migrationData migration framework for Django that migrates legacy data into your new django app
Stars: ✭ 18 (-73.13%)
streamsx.kafkaRepository for integration with Apache Kafka
Stars: ✭ 13 (-80.6%)
Mara Example Project 2An example mini data warehouse for python project stats, template for new projects
Stars: ✭ 154 (+129.85%)
beepbeep-3An event stream processor anyone can use
Stars: ✭ 20 (-70.15%)
Omniparseromniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
Stars: ✭ 148 (+120.9%)
Eel SdkBig Data Toolkit for the JVM
Stars: ✭ 140 (+108.96%)
morph-kgcPowerful RDF Knowledge Graph Generation with [R2]RML Mappings
Stars: ✭ 77 (+14.93%)
football-eventsEvent-Driven microservices with Kafka Streams
Stars: ✭ 57 (-14.93%)
Mara PipelinesA lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
Stars: ✭ 1,841 (+2647.76%)
Kettle Web基于spring boot通过java代码调用kette
Stars: ✭ 128 (+91.04%)
Reddit DetectivePlay detective on Reddit: Discover political disinformation campaigns, secret influencers and more
Stars: ✭ 129 (+92.54%)
filterCSVTools to manipulate CSV files in a format suitable for importing into various mindmapping programs - such as iThoughts, Freemind, and MindNode.
Stars: ✭ 29 (-56.72%)
dtd2mysqlMySQL / MariaDB import for DTD feeds (fares, timetable and routeing)
Stars: ✭ 25 (-62.69%)
Etl.netMass processing data with a complete ETL for .net developers
Stars: ✭ 129 (+92.54%)
uptasticsearchAn Elasticsearch client tailored to data science workflows.
Stars: ✭ 47 (-29.85%)
starlakeStarlake is a Spark Based On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing
Stars: ✭ 16 (-76.12%)
naas⚙️ Schedule notebooks, run them like APIs, expose securely your assets: Jupyter as a viable ⚡️ Production environment
Stars: ✭ 219 (+226.87%)
spStream Processors on Kafka in Golang
Stars: ✭ 29 (-56.72%)
Aws Data WranglerPandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Stars: ✭ 2,385 (+3459.7%)
KibaData processing & ETL framework for Ruby
Stars: ✭ 1,618 (+2314.93%)
wikirepoPython based Wikidata framework for easy dataframe extraction
Stars: ✭ 33 (-50.75%)
Sentinel CrawlerXenomorph Crawler, a Concise, Declarative and Observable Distributed Crawler(Node / Go / Java / Rust) For Web, RDB, OS, also can act as a Monitor(with Prometheus) or ETL for Infrastructure 💫 多语言执行器,分布式爬虫
Stars: ✭ 118 (+76.12%)
DataxDataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server
Stars: ✭ 116 (+73.13%)