GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → noplay → Python Mysql Replication

noplay / Python Mysql Replication

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL

Programming Languages

python
139335 projects - #7 most used programming language
shell
77523 projects

Labels

mysqlmysql-binlogmysql-replication-protocol

Projects that are alternatives of or similar to Python Mysql Replication

laravel-binlog
Add mysql binlog event listening for Laravel ( 为Laravel框架添加Mysql Binlog事件监听 )
Stars: ✭ 19 (-99.02%)
Mutual labels:  mysql-binlog, mysql-replication-protocol
Pychat
webchat via WebSockets/WebRTC that allows messaging/video call/screen sharing
Stars: ✭ 152 (-92.2%)
Mutual labels:  mysql
Querybuilder
SQL query builder, written in c#, helps you build complex queries easily, supports SqlServer, MySql, PostgreSql, Oracle, Sqlite and Firebird
Stars: ✭ 2,111 (+8.37%)
Mutual labels:  mysql
Serendipity
A PHP blog software
Stars: ✭ 151 (-92.25%)
Mutual labels:  mysql
Carbon Forum
A high performance open-source forum software written in PHP. Discussions Tags based with Quora/StackOverflow style.
Stars: ✭ 1,814 (-6.88%)
Mutual labels:  mysql
Amy
A C++11 compliant header-only asynchronous MySQL client library based on Asio. Enables you to work with MySQL in both asynchronous and blocking ways.
Stars: ✭ 152 (-92.2%)
Mutual labels:  mysql
Meetingfilm
基于微服务架构的在线电影购票平台
Stars: ✭ 149 (-92.35%)
Mutual labels:  mysql
Goa
基于Beego开发的问答系统
Stars: ✭ 154 (-92.09%)
Mutual labels:  mysql
Myproxy
A sharding proxy for MYSQL databases
Stars: ✭ 153 (-92.15%)
Mutual labels:  mysql
Slimdump
A tool for creating configurable dumps of large MySQL-databases.
Stars: ✭ 151 (-92.25%)
Mutual labels:  mysql
Grimoire
Database access layer for golang
Stars: ✭ 151 (-92.25%)
Mutual labels:  mysql
Mysqlbackup.net
A tool to backup and restore MySQL database in C#/VB.NET/ASP.NET.
Stars: ✭ 150 (-92.3%)
Mutual labels:  mysql
Tortoise Orm
Familiar asyncio ORM for python, built with relations in mind
Stars: ✭ 2,558 (+31.31%)
Mutual labels:  mysql
Xupdateservice
Use Spring Boot easy build, Gradle build, and provide update service for XUpdate.(使用Spring Boot简易搭建，Gradle构建，为XUpdate提供更新服务)
Stars: ✭ 149 (-92.35%)
Mutual labels:  mysql
Quill
Compile-time Language Integrated Queries for Scala
Stars: ✭ 1,998 (+2.57%)
Mutual labels:  mysql
Algernon
🎩 Small self-contained pure-Go web server with Lua, Markdown, HTTP/2, QUIC, Redis and PostgreSQL support
Stars: ✭ 1,880 (-3.49%)
Mutual labels:  mysql
Travianz Legacy
Join our Discord Server: https://discordapp.com/invite/9fbJKP9 | New repo: https://github.com/iopietro/Travianz
Stars: ✭ 150 (-92.3%)
Mutual labels:  mysql
Zhihuquestionsspider
😊😊😊 知乎问题爬虫
Stars: ✭ 152 (-92.2%)
Mutual labels:  mysql
So Sql Injections
SQL injection vulnerabilities in Stack Overflow PHP questions
Stars: ✭ 154 (-92.09%)
Mutual labels:  mysql
Movie recommend
基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统
Stars: ✭ 2,092 (+7.39%)
Mutual labels:  mysql
View All Similar Projects ➔

python-mysql-replication

 

Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL. This allow you to receive event like insert, update, delete with their datas and raw SQL queries.

Use cases

  • MySQL to NoSQL database replication
  • MySQL to search engine replication
  • Invalidate cache when something change in database
  • Audit
  • Real time analytics

Documentation

A work in progress documentation is available here: https://python-mysql-replication.readthedocs.org/en/latest/

Instruction about building documentation is available here: https://python-mysql-replication.readthedocs.org/en/latest/developement.html

Installation

pip install mysql-replication

Mailing List

You can get support and discuss about new features on: https://groups.google.com/d/forum/python-mysql-replication

Project status

The project is test with:

  • MySQL 5.5, 5.6 and 5.7
  • Python >= 2.7
  • Python 3.3, 3.4, 3.5 and 3.6 (3.2 is not supported)
  • PyPy (really faster than the standard Python interpreter)

The project is used in production for critical stuff in some medium internet corporations. But all use case as not been perfectly test in the real world.

Limitations

https://python-mysql-replication.readthedocs.org/en/latest/limitations.html

Projects using this library

MySQL server settings

In your MySQL server configuration file you need to enable replication:

[mysqld]
server-id		 = 1
log_bin			 = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size  = 100M
binlog-format    = row #Very important if you want to receive write, update and delete row events

Examples

All examples are available in the examples directory

This example will dump all replication events to the console:

from pymysqlreplication import BinLogStreamReader

mysql_settings = {'host': '127.0.0.1', 'port': 3306, 'user': 'root', 'passwd': ''}

stream = BinLogStreamReader(connection_settings = mysql_settings, server_id=100)

for binlogevent in stream:
    binlogevent.dump()

stream.close()

For this SQL sessions:

CREATE DATABASE test;
use test;
CREATE TABLE test4 (id int NOT NULL AUTO_INCREMENT, data VARCHAR(255), data2 VARCHAR(255), PRIMARY KEY(id));
INSERT INTO test4 (data,data2) VALUES ("Hello", "World");
UPDATE test4 SET data = "World", data2="Hello" WHERE id = 1;
DELETE FROM test4 WHERE id = 1;

Output will be:

=== RotateEvent ===
Date: 1970-01-01T01:00:00
Event size: 24
Read bytes: 0

=== FormatDescriptionEvent ===
Date: 2012-10-07T15:03:06
Event size: 84
Read bytes: 0

=== QueryEvent ===
Date: 2012-10-07T15:03:16
Event size: 64
Read bytes: 64
Schema: test
Execution time: 0
Query: CREATE DATABASE test

=== QueryEvent ===
Date: 2012-10-07T15:03:16
Event size: 151
Read bytes: 151
Schema: test
Execution time: 0
Query: CREATE TABLE test4 (id int NOT NULL AUTO_INCREMENT, data VARCHAR(255), data2 VARCHAR(255), PRIMARY KEY(id))

=== QueryEvent ===
Date: 2012-10-07T15:03:16
Event size: 49
Read bytes: 49
Schema: test
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2012-10-07T15:03:16
Event size: 31
Read bytes: 30
Table id: 781
Schema: test
Table: test4
Columns: 3

=== WriteRowsEvent ===
Date: 2012-10-07T15:03:16
Event size: 27
Read bytes: 10
Table: test.test4
Affected columns: 3
Changed rows: 1
Values:
--
* data : Hello
* id : 1
* data2 : World

=== XidEvent ===
Date: 2012-10-07T15:03:16
Event size: 8
Read bytes: 8
Transaction ID: 14097

=== QueryEvent ===
Date: 2012-10-07T15:03:17
Event size: 49
Read bytes: 49
Schema: test
Execution time: 0
Query: BEGIN

=== TableMapEvent ===
Date: 2012-10-07T15:03:17
Event size: 31
Read bytes: 30
Table id: 781
Schema: test
Table: test4
Columns: 3

=== UpdateRowsEvent ===
Date: 2012-10-07T15:03:17
Event size: 45
Read bytes: 11
Table: test.test4
Affected columns: 3
Changed rows: 1
Affected columns: 3
Values:
--
* data : Hello => World
* id : 1 => 1
* data2 : World => Hello

=== XidEvent ===
Date: 2012-10-07T15:03:17
Event size: 8
Read bytes: 8
Transaction ID: 14098

=== QueryEvent ===
Date: 2012-10-07T15:03:17
Event size: 49
Read bytes: 49
Schema: test
Execution time: 1
Query: BEGIN

=== TableMapEvent ===
Date: 2012-10-07T15:03:17
Event size: 31
Read bytes: 30
Table id: 781
Schema: test
Table: test4
Columns: 3

=== DeleteRowsEvent ===
Date: 2012-10-07T15:03:17
Event size: 27
Read bytes: 10
Table: test.test4
Affected columns: 3
Changed rows: 1
Values:
--
* data : World
* id : 1
* data2 : Hello

=== XidEvent ===
Date: 2012-10-07T15:03:17
Event size: 8
Read bytes: 8
Transaction ID: 14099

Tests

When it's possible we have a unit test.

More information is available here: https://python-mysql-replication.readthedocs.org/en/latest/developement.html

Changelog

https://github.com/noplay/python-mysql-replication/blob/master/CHANGELOG

Similar projects

Special thanks

Contributors

Major contributor:

Other contributors:

Thanks to GetResponse for their support

Licence

Copyright 2012-2017 Julien Duponchelle

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].