All Projects → solus-project → ferryd

solus-project / ferryd

Licence: Apache-2.0 License
Fast, safe and reliable transit for the delivery of software updates to users.

Programming Languages

go
31211 projects - #10 most used programming language
Makefile
30231 projects

Projects that are alternatives of or similar to ferryd

Universe
The Mesosphere Universe package repository.
Stars: ✭ 308 (+616.28%)
Mutual labels:  package, repository
Patchman
Patchman is a Linux Patch Status Monitoring System
Stars: ✭ 163 (+279.07%)
Mutual labels:  package, repository
Repology Updater
Repology backend service to update repository and package data
Stars: ✭ 348 (+709.3%)
Mutual labels:  package, repository
Auto Cpufreq
Automatic CPU speed & power optimizer for Linux
Stars: ✭ 843 (+1860.47%)
Mutual labels:  monitor, daemon
repology-rules
Package normalization ruleset for Repology
Stars: ✭ 67 (+55.81%)
Mutual labels:  package, repository
repogen
Easy-to-use signed APT repository generator with a web-based package browser.
Stars: ✭ 34 (-20.93%)
Mutual labels:  package, repository
Xmake Repo
📦 An official xmake package repository
Stars: ✭ 130 (+202.33%)
Mutual labels:  package, repository
Dcrm
Darwin Cydia Repo Manager - v4 redesigned in Django.
Stars: ✭ 188 (+337.21%)
Mutual labels:  repository, repository-management
get-bin-path
Get the current package's binary path
Stars: ✭ 25 (-41.86%)
Mutual labels:  package, binary
gpkg
🌎 A global Node binary manager written in Rust
Stars: ✭ 53 (+23.26%)
Mutual labels:  package, binary
Appimaged
appimaged is a daemon that monitors the system and integrates AppImages.
Stars: ✭ 262 (+509.3%)
Mutual labels:  monitor, daemon
macpack
Makes a macOS binary redistributable by searching the dependency tree and copying/patching non-system libraries.
Stars: ✭ 20 (-53.49%)
Mutual labels:  package, binary
Nexus Oss
Sonatype Nexus OSS
Stars: ✭ 240 (+458.14%)
Mutual labels:  repository, repository-management
Xcframeworks
Demonstration of creating and integrating xcframeworks and their co-op with static libraries and Swift packages
Stars: ✭ 272 (+532.56%)
Mutual labels:  package, binary
Projj
Manage repository easily.
Stars: ✭ 206 (+379.07%)
Mutual labels:  repository, repository-management
Lambda Packages
Various popular python libraries, pre-compiled to be compatible with AWS Lambda
Stars: ✭ 713 (+1558.14%)
Mutual labels:  package, binary
Git Repo Watcher
A simple bash script to watch a git repository and pull upstream changes if needed.
Stars: ✭ 73 (+69.77%)
Mutual labels:  repository, daemon
Antergos Packages
PKGBUILD's for antergos packages
Stars: ✭ 119 (+176.74%)
Mutual labels:  repository, linux-distribution
grepo
GKISS - A fork of KISS Linux that uses the GNU C library, mirror of https://codeberg.org/kiss-community/grepo
Stars: ✭ 51 (+18.6%)
Mutual labels:  repository, linux-distribution
devliver
Your private self hosted composer repository with user management
Stars: ✭ 50 (+16.28%)
Mutual labels:  package, repository

ferryd

Fast, safe and reliable transit for the delivery of software updates to users.

Report License

ferryd is a Solus project.

logo

ferryd is the binary repository manager for Solus. In addition to providing basic management for repositories, it is also an asynchronous job-based daemon, processing incoming package uploads from authorised builder machines. ferryd attempts to optimise all operations ahead of time, by caching all metadata required for repository indexes.

The primary goal for ferryd is to provide a daemon that constantly monitors new uploads, and processes them as fast as possible. This ensures new packages are available almost immediately. Complex, long running operations, are run in the background within a dedicated worker pool. This allows new packages to turn up in batch, and the delta packages to be produced lazily. Once those delta packages are available, they're inserted into the main repository (and will appear in the index.)

The design of ferryd allows us to blit a repository index from the database to disk very fast (around 2-3s for a large repository). Special care is taken to only perform atomic updates to the index - meaning no connectivity issues for clients with corrupt or partial indexes. The repository index should always be available, and all published packages should permanently be present.

ferryd takes special care to cache wherever possible, and uses a reference-counted package pool. All package files within each repository are hard-linked from the pool tree, allowing to save disk space through enforced deduplication. As such, a package's ID (the basename of the file) must be unique to a ferryd instance. Putting it all together, this allows us to simply "ref" a package into a repository from the pool, which is used for very rapid clone and pull operations.

ferryd is the replacement for the aging binman.py script previously used by Solus, and is designed to combat the design mistakes of that implementation. Emphasis is placed on speed, scaling, and having packages immediately and permanently available. Less delays for developers, and rapid updates and sync deployment to users.

Lastly, ferryd aims to provide very simple sync abilities to help control deployment of packages to other repositories. An explicit design goal is to enable "Pulling" a repository into an existing repository, which in turn publishes one channel to another. This is used in Solus to control sync-windows from unstable to stable, and is done as a single atomic operation.

Disclaimer regarding the name: Solus has this weird obsession with all things nautical. Oh and birds.

Usage (basic)

Start ferryd to monitor ferryd.sock:

./bin/ferryd -d myRepoBase -s ./ferryd.sock

Create a repo:

./bin/ferryctl -s ./ferryd.sock create-repo testing

Add packages:

./bin/ferryctl -s ./ferryd.sock import testing path/to/eopkgs

License

Copyright © 2016-2017 Solus Project, ferryd Developers (See AUTHORS file)

ferryd is available under the terms of the Apache-2.0 license

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].