All Projects → jackc → pglogrepl

jackc / pglogrepl

Licence: MIT license
PostgreSQL logical replication library for Go.

Programming Languages

go
31211 projects - #10 most used programming language
shell
77523 projects

Projects that are alternatives of or similar to pglogrepl

Lapidus
Stream your PostgreSQL, MySQL or MongoDB databases anywhere, fast.
Stars: ✭ 145 (-3.33%)
Mutual labels:  replication
Vasto
A distributed key-value store. On Disk. Able to grow or shrink without service interruption.
Stars: ✭ 206 (+37.33%)
Mutual labels:  replication
Flopnite Ue4
A remake of the popular battle royale game, Fortnite, made in Unreal Engine 4 and integrated with Amazon GameLift
Stars: ✭ 250 (+66.67%)
Mutual labels:  replication
Gobblin
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
Stars: ✭ 2,006 (+1237.33%)
Mutual labels:  replication
Amazonriver
amazonriver 是一个将postgresql的实时数据同步到es或kafka的服务
Stars: ✭ 198 (+32%)
Mutual labels:  replication
Seamless database pool
Add support for master/slave database clusters in ActiveRecord to improve performance.
Stars: ✭ 222 (+48%)
Mutual labels:  replication
Sos
Simple Object Storage (I wish I could call it Steve's Simple Storage, or S3 ;)
Stars: ✭ 129 (-14%)
Mutual labels:  replication
vrrm
rough code for running consensus
Stars: ✭ 18 (-88%)
Mutual labels:  replication
Synch
Sync data from the other DB to ClickHouse(cluster)
Stars: ✭ 200 (+33.33%)
Mutual labels:  replication
Acid State
Add ACID guarantees to any serializable Haskell data structure
Stars: ✭ 246 (+64%)
Mutual labels:  replication
Mars
Asynchronous Block-Level Storage Replication
Stars: ✭ 168 (+12%)
Mutual labels:  replication
Herddb
A JVM-embeddable Distributed Database
Stars: ✭ 192 (+28%)
Mutual labels:  replication
Litestream
Streaming replication for SQLite.
Stars: ✭ 3,795 (+2430%)
Mutual labels:  replication
Seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
Stars: ✭ 13,380 (+8820%)
Mutual labels:  replication
ldb
Replication of CRDTs
Stars: ✭ 37 (-75.33%)
Mutual labels:  replication
Sanoid
Policy-driven snapshot management and replication tools. Using ZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.) Primarily intended for Linux, but BSD use is supported and reasonably frequently tested.
Stars: ✭ 1,991 (+1227.33%)
Mutual labels:  replication
Php Mysql Replication
Pure PHP Implementation of MySQL replication protocol. This allow you to receive event like insert, update, delete with their data and raw SQL queries.
Stars: ✭ 213 (+42%)
Mutual labels:  replication
radiusd
Distributed Radius-server to do authentication+accounting.
Stars: ✭ 50 (-66.67%)
Mutual labels:  replication
itc.lua
A Lua implementation of Interval Tree Clocks
Stars: ✭ 21 (-86%)
Mutual labels:  replication
Reopenldap
Production-ready replacement for OpenLDAP with robust multi-master replication
Stars: ✭ 242 (+61.33%)
Mutual labels:  replication

Build Status

pglogrepl

pglogrepl is a Go package for PostgreSQL logical replication.

pglogrepl uses package github.com/jackc/pgx/v5/pgconn as its underlying PostgreSQL connection.

Proper use of this package requires understanding the underlying PostgreSQL concepts. See https://www.postgresql.org/docs/current/protocol-replication.html.

Example

In example/pglogrepl_demo, there is an example demo program that connects to a database and logs all messages sent over logical replication. In example/pgphysrepl_demo, there is an example demo program that connects to a database and logs all messages sent over physical replication.

Testing

Testing requires a user with replication permission, a database to replicate, access allowed in pg_hba.conf, and logical replication enabled in postgresql.conf.

Create a database:

create database pglogrepl;

Create a user:

create user pglogrepl with replication password 'secret';

If you're using PostgreSQL 15 or newer grant access to the public schema, just for these tests:

grant all on schema public to pglogrepl;

Add a replication line to your pg_hba.conf:

host replication pglogrepl 127.0.0.1/32 md5

Change the following settings in your postgresql.conf:

wal_level=logical
max_wal_senders=5
max_replication_slots=5

To run the tests set PGLOGREPL_TEST_CONN_STRING environment variable with a replication connection string (URL or DSN).

Since the base backup would request postgres to create a backup tar and stream it, this test cn be disabled with

PGLOGREPL_SKIP_BASE_BACKUP=true

Example:

PGLOGREPL_TEST_CONN_STRING=postgres://pglogrepl:[email protected]/pglogrepl?replication=database go test
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].