GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → songxin1990 → maxwell-sink

songxin1990 / maxwell-sink

Licence: MIT license
consume maxwell generated message from kafka,export it to another mysql.

Programming Languages

java
68154 projects - #9 most used programming language

Labels

mysqletlkafka-connectmaxwelldata-syncmaxwell-sink

Projects that are alternatives of or similar to maxwell-sink

kafka-connect-datagen
A Kafka Connect source connector that generates data for tests
Stars: ✭ 27 (+68.75%)
Mutual labels:  etl, kafka-connect
DataBridge.NET
Configurable data bridge for permanent ETL jobs
Stars: ✭ 16 (+0%)
Mutual labels:  etl, data-sync
Kafka Connect
equivalent to kafka-connect 🔧 for nodejs ✨🐢🚀✨
Stars: ✭ 102 (+537.5%)
Mutual labels:  etl, kafka-connect
mydataharbor
🇨🇳 MyDataHarbor是一个致力于解决任意数据源到任意数据源的分布式、高扩展性、高性能、事务级的数据同步中间件。帮助用户可靠、快速、稳定的对海量数据进行准实时增量同步或者定时全量同步，主要定位是为实时交易系统服务，亦可用于大数据的数据同步（ETL领域）。
Stars: ✭ 28 (+75%)
Mutual labels:  etl, data-sync
bigquery-kafka-connect
☁️ nodejs kafka connect connector for Google BigQuery
Stars: ✭ 17 (+6.25%)
Mutual labels:  etl, kafka-connect
go-bqloader
bqloader is a simple ETL framework to load data from Cloud Storage into BigQuery.
Stars: ✭ 16 (+0%)
Mutual labels:  etl
web-click-flow
网站点击流离线日志分析
Stars: ✭ 14 (-12.5%)
Mutual labels:  etl
cubetl
CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)
Stars: ✭ 21 (+31.25%)
Mutual labels:  etl
wrangle
A data transformation package for deep learning with Autonomio, Keras and TensorFlow.
Stars: ✭ 15 (-6.25%)
Mutual labels:  etl
redis-connect-dist
Real-Time Event Streaming & Change Data Capture
Stars: ✭ 21 (+31.25%)
Mutual labels:  etl
dswarm
an open-source data management platform for knowledge workers (https://github.com/dswarm/dswarm-documentation/wiki)
Stars: ✭ 57 (+256.25%)
Mutual labels:  etl
dflib
In-memory Java DataFrame library
Stars: ✭ 50 (+212.5%)
Mutual labels:  etl
connor
A commandline tool for resetting Kafka Connect source connector offsets.
Stars: ✭ 17 (+6.25%)
Mutual labels:  kafka-connect
cobrix
A COBOL parser and Mainframe/EBCDIC data source for Apache Spark
Stars: ✭ 109 (+581.25%)
Mutual labels:  etl
persistity
A persistence framework for game developers
Stars: ✭ 34 (+112.5%)
Mutual labels:  etl
DaFlow
Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (+50%)
Mutual labels:  etl
gallia-core
A schema-aware Scala library for data transformation
Stars: ✭ 44 (+175%)
Mutual labels:  etl
versatile-data-kit
Versatile Data Kit (VDK) is an open source framework that enables anybody with basic SQL or Python knowledge to create their own data pipelines.
Stars: ✭ 144 (+800%)
Mutual labels:  etl
PDAP-Scrapers
Code relating to scraping public police data.
Stars: ✭ 72 (+350%)
Mutual labels:  etl
oic-options-chains
ETL for OIC Options Chains
Stars: ✭ 22 (+37.5%)
Mutual labels:  etl
View All Similar Projects ➔

What is maxwell-sink

Maxwell-sink is an application based on Kafka Connect that consumes JSON messages produced by Maxwell.
Filter and then Converter the message to SQL to be executed.
It's used for synchronizing the data change between two or more MySQL clusters.

What is maxwell

Maxwell is an application that reads MySQL binlogs and writes row updates to Kafka as JSON.
For more details,please visit Maxwell's daemon

How to use maxwell-sink

Just read the maxwell-sink部署文档.docx and modify the maxwell-sink.json to fit your needs.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].