watchdb
A tool for easily replicating a SQLite database across a network.
Overview
watchdb is a tool that enables quick setup of master-slave synchronization for SQLite databases across a network.
Synchronization is one-way only (changes made on the master will overwrite changes made on slaves). Slave databases are kept read-only to prevent accidental writes from application code.
watchdb replication is eventually consistent by design, so it's AP in CAP. If you need strong consistency (at the expense of performance and required changes to application code), take a look at rqlite.
Features
- Works on Linux, OS X, Windows
- No dependencies, just one binary
- Replicates sqlite DB changes quickly across any number of nodes
- No changes to application code required
- Optional authentication and encryption supported
In the future:
- Incremental (partial) DB syncing
Installation
Precompiled binaries for supported operating systems are available.
Alternatively, run go get github.com/dkulchenko/watchdb. You'll need a working sqlite3
binary in $PATH if you go this route. The precompiled binaries embed a copy of sqlite.
Usage
Watch a SQLite database:
watchdb watch mydb.sqlite
On a slave server, sync:
watchdb sync 127.0.0.1:8144 mydbcopy.sqlite
Easy as that. Any changes made to mydb.sqlite will quickly show up in mydbcopy.sqlite. Try it out!
Options
You can specify any option on the command line, or provide a configuration file (an example config is available at conf/example.yml):
watchdb watch --config-file=config.yml mydb.sqlite
Set a custom bind address/port:
watchdb watch --bind-addr=127.0.0.1 --bind-port=1234 mydb.sqlite
Authentication
Require an auth key to be sent before syncing is allowed (similar to Redis AUTH):
watchdb watch --auth-key 927bc430fc2195fa2f0caaf35d115c mydb.sqlite
and on the slave:
watchdb sync --auth-key 927bc430fc2195fa2f0caaf35d115c 127.0.0.1:8144 mydbcopy.sqlite
Or specify it in the configuration file using the auth_key parameter.
Encryption
watchdb supports SSL for encrypted syncing between nodes.
watchdb watch --ssl --ssl-key-file=key.pem --ssl-cert-file=key.crt mydb.sqlite
and on the client:
watchdb watch --ssl 127.0.0.1:8144 mydbcopy.sqlite
You may omit the ssl key file/cert file and a self-signed one will be generated for you
at startup. Note that you'll need to provide the --ssl-skip-verify option on the client
for this to work.
Why?
sqlite3 is an excellent database, and by far the easiest way to embed SQL into an app where using a larger DB like MySQL or PostgreSQL might not be possible. The downside, of course, is the single-node nature of the DB, which means that if the primary goes down, the data goes down with it.
I wanted to build a simple way to replicate a sqlite database across a network without requiring any special configuration or changes to application code. Hence, watchdb.
Contribute
- Fork repository
- Create a feature or bugfix branch
- Open a new pull request
License
The MIT License (MIT)
Copyright (c) 2015 Daniil Kulchenko [email protected]