Chaotic AUR

Disclaimer

I receive questions like "Why didn't you write it in language X? Why didn't you use sudoers instead of setuid? Why don't you guarantee reproducible builds? Why don't you use submodules and review every PKGBUILD change?"

Because I can't, I am the only person, taking care of all the steps required for updating 3800 packages non-stop, treating all the differences between those PKGBUILDs and their sources, in an infra that runs in donated VMs which are not similar.

If at some point you see something that could be better, then please open a PR. And keep it simple, the more complex the codebase becomes, the more complicated will be in the future for one man alone to maintain it.

CLI

• chaotic pr{,epare} ${INPUTDIR} $@

  It generates a building script to be later run in a containerized environment. $INPUTDIR is the name of the directory in "$PWD" which contains a PKGBUILD.

• chaotic {lw,lowerstrap}

  It generates a lowerdir for later chrooting.

• chaotic {mk,makepkg} ${INPUTDIR} $@

  Builds the package in a container using systed-nspawn. $INPUTDIR is the result of a prepare

• chaotic {mkd,makepwd} [${PACKAGES[@]}]

  Prepare and build all packages in the current directory. If PACKAGES are not provided then it will try to build all sub-directories.

• chaotic {si,interfere-sync}

  Sync packages' interference repo.

• chaotic {bump,interfere-bump} [${PACKAGES[@]}]

  Generate and push a PKGREL_BUMPS file for use in interfere. Old entries will automatically be removed. Uses repoctl.

• chaotic {sp,package-lists-sync}

  Sync package list repo.

• chaotic {dp,deploy} ${INPUTDIR}

  Sign the package and send to primary node.

• chaotic {dbb,db-bump}

  Add recently deployed packages to the database, while moving replaced packages to the archive. Uses repoctl.

• chaotic db-rebuild Completely rebuild the database and files database from scratch, keeping the original DB around until the rebuild is finished.

• chaotic {rm,remove} ${PACKAGES[@]}

  Remove and archive all listed packages. Uses repoctl.

• chaotic {get,aur-download} [-r] ${PACKAGES[@]}

  Download listed packages' sources from AUR. Uses repoctl.

• chaotic cl{,eanup} ${INPUTDIR}

  Safely deletes old package sources.

• chaotic help {syncthing,rsync}

  Instructions to the mirroring services. RSync is one-way (primary->cluster) only, and Syncthing is both ways.

• chaotic routine {hourly,morning,afternoon,nightly,midnight}

  Run the specified routine.

• chaotic routine clean-archive

  When on a primary node, clean up the archive folder.

• chaotic {clean-duplicates,dedup}

  Moves packages which have been replaced by newer (modify time) packages to the archive folder.

• chaotic {clg,clean-logs}

  After a chaotic makepwd, remove successfull and "already built" logs.

• chaotic {cls,clean-srccache} ${PACKAGE}

  Removes cached sources from a specific package.

• chaotic clean-sigs {,-q}

  Moves package files or archive files without their corresponding signature or archive file to the package archive. Files have to be at least 1 hour old to be considered.

• chaotic {srt,sort-logs}

  Unclutters and sorts the raw log directory into easy to read subcategories

• chaotic find-discarded

  Find and print discarded packages. All packages that are in the database, but do not exist in the package list repo, are considered discarded, but have to be deleted manually.

Involved directories

• /var/cache/chaotic/sources/${PACKAGETAG}

  Per-package SRCDEST.

• /var/cache/chaotic/lower/{latest,$DATESTAMP}

  Lowerdirs.

• /var/cache/chaotic/cc/{PACKAGETAG}

  Per-package ~/.ccache.

• /var/cache/chaotic/packages

  Container-shared pacman's cache.

• /var/lib/chaotic/interfere

  Cloned version of interfere repository

Dependencies

pacman -S --needed base-devel git arch-install-scripts repoctl fuse-overlayfs rsync python-telegram-send openssh

One needs an active mirror or a setting (in /etc/chaotic.conf) like this:

export CAUR_URL='https://builds.garudalinux.org/repos/chaotic-aur/x86_64'
export REPOCTL_CONFIG='/etc/chaotic/repoctl.conf'
export CAUR_REPOCTL_DB_URL="${CAUR_URL}/chaotic-aur.db.tar.zst"
export CAUR_REPOCTL_DB_FILE="/tmp/chaotic/db.tar.zst"

To create a gpg key for the root user refer to this ArchWiki article for more information. If you find problems when using "sudo", read the "su" subsection. Then generate a ssh keypair for the root user.

sudo ssh-keygen

The ssh public key (cat /root/.ssh/id_rsa.pub) then needs to be added to the primary servers' root authorized keys (/root/.ssh/authorized_keys). After that follow these instructions to export the gpg public key. This key will have to be uploaded to keyserver.ubuntu.com in order for the key to be verified. Then, configure it as follows in /etc/chaotic.conf, like this:

export CAUR_DEPLOY_PKGS="/var/www/chaotic-aur/x86_64"
export CAUR_URL="http://localhost:8080/chaotic-aur/x86_64"
export CAUR_SIGN_KEY='8A9E14A07010F7E3'
export CAUR_TYPE='cluster'
export REPOCTL_CONFIG='/etc/chaotic/repoctl.toml'

You'll find more options in src/chaotic first lines.

Supported type values are primary, cluster, and dev.

Furthermore, a valid .gitconfig needs to be supplied in /root/.gitconfig to allow bumping packages via chaotic bump with a meaningful email and name supplied:

[user]
email = "[email protected]"
name = "Chaotic Dragon (Node) ☭"

To allow pushing to the interfere repo, a fitting ssh key also needs to be added to authenticate the node.

To have clean logs & less bandwidth usage /etc/pacman.conf settings need to be adjusted:

• Enable NoProgressBar

• Use Server = file:///path-to-local-repo as repo link if a local mirror is available

• Don't use ILoveCandy

To deploy faster replace openssh with openssh-hpn on all nodes (adds performance-related patches). It should be noted that openssh-hpn recently changed its configuration and behavior and therefore doesn't provide a 1:1 replacement for openssh anymore.

Installation

Install dependencies, then:

sudo groupadd chaotic_op
sudo usermod -aG chaotic_op $(whoami)

make build && sudo make install

Lint

pacman -S --needed yarn shellcheck
yarn install
yarn run lint