Interval rsync
Source: https://github.com/txn2/irsync
Command line utility and Docker container for running rsync on an interval. rsync, synchronizes (one way) files between two sources, irsync wrapps rsync and runs it on a specified interval.
As a basic command-line utility, the only requirement is rsync. As a Docker container the conly requirement is Docker. The docker container size with all the requirements is only around 5 megs.
Background
I wrote up a little background on the motivation behind irsync on my imti.co blog, rSync Files on Interval.
Demo
Setup a quick demo using Dockers docker-compose command. Included with this project is a docker-compose.yaml with a simple client/server setup. In this composer configuration irsync is set to check the server every 30 seconds (after sync is complete. The server mounts the ./data/source directory and the client mounts the ./data/dest directory. Drop files in ./data/source and see them appear in ./data/dest.
Watch a 2 minute youtube video demoing the commands below
Setup and run demo (requires Docker):
# create a source and dest directories (mounted from the docker-compose)
mkdir -p ./data/source
mkdir -p ./data/dest
# make a couple of sample files
touch ./data/source/test1.txt
touch ./data/source/test2.txt
# get the docker-compose.yml
curl https://raw.githubusercontent.com/txn2/irsync/master/docker-compose.yml >docker-compose.yml
# run docker-compose in the background (-d flag)
docker-compose up -d
# view logs
docker-compose logs -f
# drop some more files in the ./data/source directory
# irsync is configured to check every 30 seconds in this demo.
#### Cleanup
# stop containers
docker-compose stop
# remove containers
docker-compose rm
Run on MacOS
Install
brew tap txn2/homebrew-tap
brew install irsyncUpdate (when new versions are released)
brew upgrade irsyncRun Prints the rsync version every 5 seconds.
irsync --irsync-interval-seconds=5 --version Adjust the interval to your requirement and add rsync flags as you normally would.
Run From Source
Example #1 Local with interval defaults
go run ./irsync.go -pvrt --exclude='custom' --exclude='special.txt' --delete ./data/source/ ./data/dest/Run From Container
Example #1 Local
docker run --rm -v "$(pwd)"/data:/data txn2/irsync \
-pvrt --delete /data/source/ /data/dest/Example #2 Local Every 10 Seconds
docker run --rm -v "$(pwd)"/data:/data txn2/irsync \
--irsync-interval-seconds=10 \
-pvrt --delete /data/source/ /data/dest/Example #3 Server to Local - Restart on Failure
docker run -d --name irsync-demo --restart on-failure \
-v "$(pwd)"/data:/data txn2/irsync \
--irsync-interval-seconds=10 \
-pvrt --delete /data/source/ /data/dest/Example#4: Custom container
Create Dockerfile:
FROM txn2/irsync:2.0.0
LABEL vendor="mk.imti.co"
LABEL co.imti.mk.source="https://github.com/txn2/irsync"
# if the rsync server requires a password
ENV RSYNC_PASSWORD=password
# exampe: keep local synchronized with server
# interval default: --irsync-interval-seconds=120
# activity timout default: --irsync-timeout-seconds=7200
CMD ["-pvrt", "--modify-window=30", "--delete", "--exclude='fun'", "rsync://[email protected]:873/data/", "/media"]Build:
docker build -t custom-sync .Run:
docker run -d --name custom-sync --restart on-failure \
-v "$(pwd)"/data:/data custom-syncLogs:
docker logs -f custom-syncEnvironment Configuration
RSYNC_PASSWORD=passwordUse environment variableRSYNC_PASSWORDto keep from being prompted for a password for servers requiring authentication.
Development
Building and Releasing
Interval rsync uses GORELEASER to build binaries and Docker containers.
Test Release Steps
Install GORELEASER with brew (MacOS):
brew install goreleaser/tap/goreleaserBuild without releasing:
goreleaser --skip-publish --rm-dist --skip-validateRelease Steps
- Commit latest changes
- Tag a version
git tag -a v2.0 -m "Version 2.0" - Push tag
git push origin v2.0 - Run:
GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist
