All Projects → github → Gh Ost

github / Gh Ost

Licence: mit
GitHub's Online Schema Migrations for MySQL

Programming Languages

go
31211 projects - #10 most used programming language
shell
77523 projects

Projects that are alternatives of or similar to Gh Ost

Wtcms
基于thinkphp的内容管理系统,可快速搭建个人博客、公司学校官网、新闻类站点。
Stars: ✭ 75 (-99.21%)
Mutual labels:  mysql
Shorty
🔗 A URL shortening service built using Flask and MySQL
Stars: ✭ 78 (-99.18%)
Mutual labels:  mysql
Sql
MySQL & PostgreSQL pipe
Stars: ✭ 81 (-99.15%)
Mutual labels:  mysql
Bige
游戏服务器框架。
Stars: ✭ 76 (-99.2%)
Mutual labels:  mysql
Ycsocket
基于swoole的socket框架,支持协程版MySQL、Redis连接池,已用于大型RPG游戏服务端
Stars: ✭ 77 (-99.19%)
Mutual labels:  mysql
Sqlite3 To Mysql
Transfer data from SQLite to MySQL
Stars: ✭ 79 (-99.17%)
Mutual labels:  mysql
Snail
🎈 🎈 这里有各领域丰富且有用的知识! 🚀 http://note.isnico.com/
Stars: ✭ 74 (-99.22%)
Mutual labels:  mysql
Butterfly
🔥 蝴蝶--【简单】【稳定】【好用】的 Python web 框架🦋 除 Python 2.7,无其他依赖; 🦋 butterfly 是一个 RPC 风格 web 框架,同时也是微服务框架,自带消息队列通信机制实现分布式
Stars: ✭ 82 (-99.14%)
Mutual labels:  mysql
Bbs node
node后端服务,node+express+mysql, 搭建token-权限-管理完整的web服务, 对应页面: https://www.lyh.red/admin \ https://www.lyh.red/bbs \ 接口地址https://www.lyh.red/apidoc/index.html
Stars: ✭ 78 (-99.18%)
Mutual labels:  mysql
Student Homework Management System
使用SSM+Shiro开发的学生作业管理系统。支持批量打包下载,QQ登陆等功能 生产版:
Stars: ✭ 79 (-99.17%)
Mutual labels:  mysql
Laravel Log To Db
Custom Laravel and Lumen 5.6+ Log channel handler that can store log events to SQL or MongoDB databases. Uses Laravel/Monolog native logging functionality.
Stars: ✭ 76 (-99.2%)
Mutual labels:  mysql
Db Seeder
A database seeder app for MySQL
Stars: ✭ 77 (-99.19%)
Mutual labels:  mysql
Agent
The best way to backup and restore your database
Stars: ✭ 80 (-99.16%)
Mutual labels:  mysql
Cetus
Cetus is a high performance middleware that provides transparent routing between your application and any backend MySQL Servers.
Stars: ✭ 1,199 (-87.41%)
Mutual labels:  mysql
Mysqltuner
MySQL Tuner for Windows
Stars: ✭ 81 (-99.15%)
Mutual labels:  mysql
Sql to sqlalchemy
本教程是为了展现 sql 原始语句转换为 sqlalchemy 语句的各个实例。
Stars: ✭ 75 (-99.21%)
Mutual labels:  mysql
Olog
📔 Online Note-Taking Experience | 即时灵感笔记记录平台
Stars: ✭ 79 (-99.17%)
Mutual labels:  mysql
Chloe
A lightweight and high-performance Object/Relational Mapping(ORM) library for .NET --C#
Stars: ✭ 1,248 (-86.89%)
Mutual labels:  mysql
Archer
基于inception的自动化SQL操作平台,支持SQL执行、LDAP认证、发邮件、OSC、SQL查询、SQL优化建议、权限管理等功能,支持docker镜像
Stars: ✭ 1,239 (-86.99%)
Mutual labels:  mysql
Dble Docs Cn
Documents for dble in Chinese
Stars: ✭ 80 (-99.16%)
Mutual labels:  mysql

gh-ost

ci replica-tests downloads release

GitHub's online schema migration for MySQL

gh-ost is a triggerless online schema migration solution for MySQL. It is testable and provides pausability, dynamic control/reconfiguration, auditing, and many operational perks.

gh-ost produces a light workload on the master throughout the migration, decoupled from the existing workload on the migrated table.

It has been designed based on years of experience with existing solutions, and changes the paradigm of table migrations.

How?

All existing online-schema-change tools operate in similar manner: they create a ghost table in the likeness of your original table, migrate that table while empty, slowly and incrementally copy data from your original table to the ghost table, meanwhile propagating ongoing changes (any INSERT, DELETE, UPDATE applied to your table) to the ghost table. Finally, at the right time, they replace your original table with the ghost table.

gh-ost uses the same pattern. However it differs from all existing tools by not using triggers. We have recognized the triggers to be the source of many limitations and risks.

Instead, gh-ost uses the binary log stream to capture table changes, and asynchronously applies them onto the ghost table. gh-ost takes upon itself some tasks that other tools leave for the database to perform. As result, gh-ost has greater control over the migration process; can truly suspend it; can truly decouple the migration's write load from the master's workload.

In addition, it offers many operational perks that make it safer, trustworthy and fun to use.

gh-ost general flow

Highlights

  • Build your trust in gh-ost by testing it on replicas. gh-ost will issue same flow as it would have on the master, to migrate a table on a replica, without actually replacing the original table, leaving the replica with two tables you can then compare and satisfy yourself that the tool operates correctly. This is how we continuously test gh-ost in production.
  • True pause: when gh-ost throttles, it truly ceases writes on master: no row copies and no ongoing events processing. By throttling, you return your master to its original workload
  • Dynamic control: you can interactively reconfigure gh-ost, even as migration still runs. You may forcibly initiate throttling.
  • Auditing: you may query gh-ost for status. gh-ost listens on unix socket or TCP.
  • Control over cut-over phase: gh-ost can be instructed to postpone what is probably the most critical step: the swap of tables, until such time that you're comfortably available. No need to worry about ETA being outside office hours.
  • External hooks can couple gh-ost with your particular environment.

Please refer to the docs for more information. No, really, read the docs.

Usage

The cheatsheet has it all. You may be interested in invoking gh-ost in various modes:

  • a noop migration (merely testing that the migration is valid and good to go)
  • a real migration, utilizing a replica (the migration runs on the master; gh-ost figures out identities of servers involved. Required mode if your master uses Statement Based Replication)
  • a real migration, run directly on the master (but gh-ost prefers the former)
  • a real migration on a replica (master untouched)
  • a test migration on a replica, the way for you to build trust with gh-ost's operation.

Our tips:

  • Testing above all, try out --test-on-replica first few times. Better yet, make it continuous. We have multiple replicas where we iterate our entire fleet of production tables, migrating them one by one, checksumming the results, verifying migration is good.
  • For each master migration, first issue a noop
  • Then issue the real thing via --execute.

More tips:

  • Use --exact-rowcount for accurate progress indication
  • Use --postpone-cut-over-flag-file to gain control over cut-over timing
  • Get familiar with the interactive commands

Also see:

What's in a name?

Originally this was named gh-osc: GitHub Online Schema Change, in the likes of Facebook online schema change and pt-online-schema-change.

But then a rare genetic mutation happened, and the c transformed into t. And that sent us down the path of trying to figure out a new acronym. gh-ost (pronounce: Ghost), stands for GitHub's Online Schema Transmogrifier/Translator/Transformer/Transfigurator

License

gh-ost is licensed under the MIT license

gh-ost uses 3rd party libraries, each with their own license. These are found here.

Community

gh-ost is released at a stable state, but with mileage to go. We are open to pull requests. Please first discuss your intentions via Issues.

We develop gh-ost at GitHub and for the community. We may have different priorities than others. From time to time we may suggest a contribution that is not on our immediate roadmap but which may appeal to others.

Please see Coding gh-ost for a guide to getting started developing with gh-ost.

Download/binaries/source

gh-ost is now GA and stable.

gh-ost is available in binary format for Linux and Mac OS/X

Download latest release here

gh-ost is a Go project; it is built with Go 1.15 and above. To build on your own, use either:

  • script/build - this is the same build script used by CI hence the authoritative; artifact is ./bin/gh-ost binary.
  • build.sh for building tar.gz artifacts in /tmp/gh-ost

Generally speaking, master branch is stable, but only releases are to be used in production.

Authors

gh-ost is designed, authored, reviewed and tested by the database infrastructure team at GitHub:

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].