Kafka Connect Datagen Connector
kafka-connect-datagen is a Kafka Connect plugin that generates data for integration and performance tests.
Documentation
- User guide: http://kafka-connect-datagen.rtfd.io
xushiyan / kafka-connect-datagen
Licence: Apache-2.0 license
A Kafka Connect source connector that generates data for tests
Stars: ✭ 27
Programming Languages
java
68154 projects - #9 most used programming language
Projects that are alternatives of or similar to kafka-connect-datagen
DIRECT
DIRECT, the Data Integration Run-time Execution Control Tool, is a data logistics framework that can be used to monitor, log, audit and control data integration / ETL processes.
Stars: ✭ 20 (-25.93%)
Mutual labels: etl, etl-pipeline
Kafka Connect
equivalent to kafka-connect 🔧 for nodejs ✨🐢🚀✨
Stars: ✭ 102 (+277.78%)
Mutual labels: etl, kafka-connect
redis-connect-dist
Real-Time Event Streaming & Change Data Capture
Stars: ✭ 21 (-22.22%)
Mutual labels: etl, etl-pipeline
maxwell-sink
consume maxwell generated message from kafka,export it to another mysql.
Stars: ✭ 16 (-40.74%)
Mutual labels: etl, kafka-connect
datalake-etl-pipeline
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
Stars: ✭ 39 (+44.44%)
Mutual labels: etl, etl-pipeline
AirflowETL
Blog post on ETL pipelines with Airflow
Stars: ✭ 20 (-25.93%)
Mutual labels: etl, etl-pipeline
etlflow
EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.
Stars: ✭ 38 (+40.74%)
Mutual labels: etl, etl-pipeline
hamilton
A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.
Stars: ✭ 612 (+2166.67%)
Mutual labels: etl, etl-pipeline
csvplus
csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.
Stars: ✭ 67 (+148.15%)
Mutual labels: etl, etl-pipeline
bigquery-kafka-connect
☁️ nodejs kafka connect connector for Google BigQuery
Stars: ✭ 17 (-37.04%)
Mutual labels: etl, kafka-connect
DaFlow
Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (-11.11%)
Mutual labels: etl, etl-pipeline
registryless-avro-converter
An avro converter for Kafka Connect without a Schema Registry
Stars: ✭ 45 (+66.67%)
Mutual labels: kafka-connect
versatile-data-kit
Versatile Data Kit (VDK) is an open source framework that enables anybody with basic SQL or Python knowledge to create their own data pipelines.
Stars: ✭ 144 (+433.33%)
Mutual labels: etl
mydataharbor
🇨🇳 MyDataHarbor是一个致力于解决任意数据源到任意数据源的分布式、高扩展性、高性能、事务级的数据同步中间件。帮助用户可靠、快速、稳定的对海量数据进行准实时增量同步或者定时全量同步，主要定位是为实时交易系统服务，亦可用于大数据的数据同步（ETL领域）。
Stars: ✭ 28 (+3.7%)
Mutual labels: etl
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].