ustcmirror-images

Table Of Content

• ustcmirror-images
  • Table Of Content
  • Introduction
  • Quick Start
  • Configuration
    • Volumes
    • Common Configuration Parameters(AKA environment variables)
    • aptsync
    • apt-sync
    • archvsync
    • debian-cd
    • fedora
    • freebsd-pkg
    • freebsd-ports
    • github-release
    • gitsync
    • gsutil-rsync
    • hackage
    • homebrew-bottles
    • julia
    • julia-storage
    • lftpsync
    • nix-channels
    • nodesource
    • pypi
    • rclone
    • rsync
    • rubygems
    • stackage
    • yum-sync
  • License
  • Contributing

Introduction

These images are designed for mirroring remote directories/repositories in a consistent and portable way. They are used by ustcmirror (yuki).

Quick Start

docker run --rm \
    -e LOG_ROTATE_CYCLE='5' \
    -e RSYNC_HOST='rsync.alpinelinux.org' \
    -e RSYNC_PATH='alpine/' \
    -e RSYNC_MAXDELETE='10000' \
    -v /var/repos/alpine:/data \
    -v /var/sync-logs/alpine:/log \
    ustcmirror/rsync:latest

Configuration

Volumes

• /data: The mount point of the repository on the host. You can refer to it as environment variable TO in your program.
• /log: The mount point of the host directory that save logs. You can refer to it as environment variable LOG in your program.

Common Configuration Parameters(AKA environment variables)

aptsync

Notes: The following mirror.list:

deb-i386 https://apt.dockerproject.org/repo debian-jessie main
deb-amd64 https://apt.dockerproject.org/repo debian-jessie main
deb-armhf https://apt.dockerproject.org/repo raspbian-jessie main testing

is equivalent to the following parameters:

APTSYNC_URL='https://apt.dockerproject.org/repo'
APTSYNC_DISTS='debian-jessie|main|i386 amd64:raspbian-jessie|main testing|armhf'

apt-sync

It is almost the same as aptsync. Except that APTSYNC_DISTS accepts 4 parameters (rather than 3) for every item.

Consider use apt-sync when the upstream replaces packages in-place, as aptsync will simply ignore if there're any changes in existed packages.

archvsync

A.K.A. ftpsync

archvsync respects the env vars used in ftpsync

debian-cd

debian-cd accepts the same parameters specified in debian-cd-mirror.conf

fedora

fedora-quick-mirror

See dist conf for parameters meaning.

Note: This image is not in use now, as quick-fedora-mirror has some mysterious bugs when being used.

freebsd-pkg

freebsd-ports

Notice: BIND_ADDRESS is only added for curl in freebsd-ports. Make sure that github.com is accessible under default network settings.

github-release

To specified the repo list to sync, you can:

• Read-only bind mount a YAML file into the container at /repos.yaml. See the example.
• Pass the YAML-format repo list string as REPOS env.

gitsync

gsutil-rsync

hackage

homebrew-bottles

julia

Sync from official site. No parameters needed. (Deprecated)

julia-storage

A new solution to sync Julia general registry (using StorageMirrorServer.jl). No parameters needed.

lftpsync

nix-channels

nodesource

Sync from official site. No parameter needed.

pypi

rclone

ref:

• rclone environment variables
• rclone-sync manual

rsync

rubygems

stackage

Stackage doesn't need to specify upstream, but this mirror use cabal to install necessary Haskell packages. Replacing default mirror of cabal with faster one will speed up building process.

Read the user guide before writing preferred mirror to config

yum-sync

yum-sync tries to imitate the parameters of aptsync, and it supports the following substitution rule for YUMSYNC_URL and <reponame> & <download_dir> in YUMSYNC_DISTS:

• @{arch}: Architecture (x86_64, armhf, ...)
• @{os_ver}: OS version (6-8, ...)
• @{comp}: The <component> in YUMSYNC_DISTS

yum-sync.py is modified to get the same directory structure as upstream when syncing. And <reponame> should be named the same as the directory containing repodata dir.

Notes:

The following repo configuration:

baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el$releasever-$basearch

translates to:

YUMSYNC_URL='https://packages.cloud.google.com/yum/repos/[email protected]{os_ver}[email protected]{arch}'
YUMSYNC_DISTS='6-7|kubernetes|x86_64,aarch64,armhfp,ppc64le,s390x|[email protected]{os_ver}[email protected]{arch}|/yum/repos/[email protected]{os_ver}[email protected]{arch}'

And the following:

[mysql80-community]
name=MySQL 8.0 Community Server
baseurl=http://repo.mysql.com/yum/mysql-8.0-community/el/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

[mysql57-community]
name=MySQL 5.7 Community Server
baseurl=http://repo.mysql.com/yum/mysql-5.7-community/el/$releasever/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

translates to:

YUMSYNC_URL='https://repo.mysql.com/yum/@{comp}/el/@{os_ver}/@{arch}/'
YUMSYNC_DISTS='6-8|mysql-8.0-community,mysql-5.7-community|aarch64,i386,x86_64|@{arch}|/yum/@{comp}/el/@{os_ver}/@{arch}/'

License

Specially, contents of folder apt-sync, yum-sync, github-release and the generated container image from them are under GPLv3 license, as it uses code from https://github.com/tuna/tunasync-scripts.

Other contents are under MIT license.

Contributing

See CONTRIBUTING.md.