All Projects → gitstliu → MysqlToAll

gitstliu / MysqlToAll

Licence: other
A Mysql data to file/elasticsearch/kafka/redis/redis-cluster/mongodb(audit and normal)/TiDB/postgresql with binlog tool.

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to MysqlToAll

Redisc
A Go redis cluster client built on top of redigo.
Stars: ✭ 183 (+117.86%)
Mutual labels:  redis-cluster
replicator
MySQL Replicator. Replicates MySQL tables to Kafka and HBase, keeping the data changes history in HBase.
Stars: ✭ 41 (-51.19%)
Mutual labels:  binlog
deflek
index and API RBAC for Elasticsearch and Kibana via reverse proxy. DEPRECATED
Stars: ✭ 13 (-84.52%)
Mutual labels:  audit-log
Blog
MyBlog
Stars: ✭ 197 (+134.52%)
Mutual labels:  redis-cluster
auditor
auditor, the missing audit log library
Stars: ✭ 105 (+25%)
Mutual labels:  audit-log
mysqlbinlog-rs
A MySQL binlog file (row format) parser in Rust
Stars: ✭ 20 (-76.19%)
Mutual labels:  binlog
Redex
Cloud-native Redis server implemented in Elixir
Stars: ✭ 160 (+90.48%)
Mutual labels:  redis-cluster
rancher-redis
A containerized redis master/slave configuration with sentinels for use in Rancher
Stars: ✭ 13 (-84.52%)
Mutual labels:  redis-cluster
ghec-audit-log-cli
Query the GitHub Audit Log for your organization to send it over to other services like elastic, splunk or sentinel for visualization and security
Stars: ✭ 54 (-35.71%)
Mutual labels:  audit-log
think-redisd
thinkphp 5 redis读写分离驱动
Stars: ✭ 24 (-71.43%)
Mutual labels:  redis-cluster
Redis Manager
Redis 一站式管理平台,支持集群的监控、安装、管理、告警以及基本的数据操作
Stars: ✭ 2,646 (+3050%)
Mutual labels:  redis-cluster
Redis Cluster
Redis Cluster setup running on Kubernetes
Stars: ✭ 230 (+173.81%)
Mutual labels:  redis-cluster
DBProxy
redis proxy、ssdb proxy(使用脚本控制sharding)
Stars: ✭ 1 (-98.81%)
Mutual labels:  redis-cluster
Anotherredisdesktopmanager
🚀🚀🚀A faster, better and more stable redis desktop manager [GUI client], compatible with Linux, Windows, Mac. What's more, it won't crash when loading massive keys.
Stars: ✭ 17,704 (+20976.19%)
Mutual labels:  redis-cluster
asyncmy
A fast asyncio MySQL/MariaDB driver with replication protocol support
Stars: ✭ 126 (+50%)
Mutual labels:  binlog
Undermoon
Mordern Redis Cluster solution for easy operation.
Stars: ✭ 166 (+97.62%)
Mutual labels:  redis-cluster
docker-redis-haproxy-cluster
A Redis Replication Cluster accessible through HAProxy running across a Docker Composed-Swarm with Supervisor and Sentinel
Stars: ✭ 44 (-47.62%)
Mutual labels:  redis-cluster
td-redis-operator
一款强大的云原生redis-operator,经过大规模生产级运行考验,支持分布式集群、支持主备切换等缓存集群解决方案…The powerful cloud-native redis-operator, which has passed the test of large-scale production-level operation, supports distributed clusters and active/standby switching ...
Stars: ✭ 327 (+289.29%)
Mutual labels:  redis-cluster
sherlock
Sherlock is an anomaly detection service built on top of Druid
Stars: ✭ 137 (+63.1%)
Mutual labels:  redis-cluster
kunlun-storage
Kunlun-storage is the storage component for KunlunBase. It's developed based on percona-mysql-8.0.x and contains exclusive features used by KunlunBase, performance enhancements and XA transaction crash safety enhancements without which MySQL would not be able to execute XA transactions reliably under error conditions such as power outage, proces…
Stars: ✭ 2 (-97.62%)
Mutual labels:  binlog

中文

MysqlToAll

A Mysql data to file/elasticsearch/kafka/redis/redis-cluster with binlog tool.

High Performance

How to config?

canalconfigs

xxx.toml     (canal config file)
xxx.pos      (binlog pos file)

config

config.toml (Output source config file)

具体用法

QQ群

691944436

The answer is zero

Performance Test Data

Test Machine : 4C 8G

Scene1:Mixed Channels:Tow mysql master binlog write to redis、redis-cluster、kafka、es、local file in one time.

Total Binlog Raw Count:1 Million
Bulksize:              1000
Resource utilization
CPU: 250%
Mem:3%
Duration:178s

Scene2:Single Channel:One mysql master binlog write to redis.

Total Binlog Raw Count:1 Million
Bulksize:              1000
Resource utilization
CPU: 230%
Mem:0.5%
Duration:27s

Scene3:Single Channel:One mysql master binlog write to redis-cluster.

Total Binlog Raw Count:1 Million
Bulksize:              1000
Resource utilization
CPU: 230%
Mem:0.5%
Duration:21s

Scene4:Single Channel:One mysql master binlog write to elasticsearch.

Total Binlog Raw Count:1 Million
Bulksize:              1000
Resource utilization
CPU: 90%
Mem:0.5%
Duration:62s

Scene5:Single Channel:One mysql master binlog write to kafka.

Total Binlog Raw Count:1 Million
Bulksize:              1000
Resource utilization
CPU: 230%
Mem:1%
Duration:24s

Scene6:Single Channel:One mysql master binlog write to local file.

Total Binlog Raw Count:1 Million
Bulksize:              1000
Resource utilization
CPU: 280%
Mem:0.5%
Duration:38s
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].