skiffos / Skiffos

Introduction

Skiff compiles a lightweight operating system for any Linux-compatible computer, ranging from RPi, Odroid, NVIDIA Jetson, to Desktop PCs, Laptops (i.e. Apple MacBook), Phones (PinePhone), Containers, or Cloud VMs. It is:

• Adoptable: any userspace can be imported/exported to/from container images.
• Familiar: uses simple Makefile and KConfig language for configuration.
• Flexible: supports all major OS distributions inside containers.
• Portable: containers can be moved between machines of similar CPU type.
• Reliable: changes inside user environments cannot break the host boot-up.
• Reproducible: a given Skiff Git tree will always produce identical output.

Uses Buildroot to produce a minimal "single-file" host OS as a standardized base cross-platform operating system "shim" for hosting containers. Most Linux platforms have widely varying requirements for kernel, firmware, and additional hardware support packages. The immutable SkiffOS host system contains everything needed to support the hardware, cleanly separated from the applications.

Getting started

The SKIFF_CONFIG comma-separated environment variable selects which configuration layers should be merged together to configure the build.

$ make                             # lists all available layers
$ export SKIFF_CONFIG=pi/4,skiff/core
$ make configure                   # configure the system
$ make compile                     # build the system

After you run make configure Skiff will remember what you selected in SKIFF_CONFIG. The compile command instructs Skiff to build the system.

Adjustments can be made to configuration layers, and make compile can be called again to re-pack the system image without re-building everything.

You can add your SSH public key to the target image by adding it to overrides/root_overlay/etc/skiff/authorized_keys/my-key.pub, or by adding it to your own custom configuration package.

The example above uses pi/4, which can be replaced with any of the hardware support packages listed in the Supported Systems table.

Once the build is complete, it's time to flash the system to a SD card. You will need to switch to sudo bash for this on most systems.

$ sudo bash             # switch to root
$ blkid                 # look for your SD card's device file
$ export PI_SD=/dev/sdz # make sure this is right!
$ make cmd/pi/common/format  # tell skiff to format the device
$ make cmd/pi/common/install # tell skiff to install the os

The device needs to be formatted only one time, after which, the install command can be used to update the SkiffOS images without clearing the persistent state. The persist partition is not touched in this step, so anything you save there, including Docker state and system configuration, will not be modified.

There are many other utility commands made available by Buildroot, which can be listed using make br/help, some examples:

$ make br/menuconfig # optionally explore Buildroot config
$ make br/sdk        # build relocatable SDK for target
$ make br/graph-size # graph the target packages sizes

Connect using SSH to [email protected] to access the SkiffOS system, and connect to [email protected] to access the "Core" system container. The mapping between users and containers can be edited in the /mnt/persist/skiff/core/config.yaml file.

The system can then be upgraded over-the-air (OTA) using the rsync script:

$ ./scripts/push_image.bash [email protected]

The SkiffOS upgrade (or downgrade) will take effect on next reboot.

Supported Systems

SkiffOS is based on Buildroot, which can compile operating systems for virtually any machine. Therefore, SkiffOS also works on nearly any architecture or board.

Here are the boards/systems currently supported by Skiff:

All targets marked "tested" use automated end-to-end testing on real hardware.

Adding support for a board involves creating a Skiff configuration package for the board, as described above. If you have a device that is not yet supported by SkiffOS, please open an issue.

Skiff Core

View Demo!

The Skiff Core subsystem, enabled with the skiff/core layer or by selecting any of the core environment packages, automatically configures mappings between users and containerized environments. It maps incoming SSH sessions accordingly:

• Configured using a YAML configuration file skiff-core.yaml.
• The container image is either pulled or built from scratch.
• systemd and/or other init systems operate as PID 1 inside the container.

This allows virtually any workflow to be migrated to Skiff. The config file structure is flexible, and allows for any number of containers, users, and images to be defined and built.

Supported Environments

Any existing GNU/Linux system with compatibility with the running kernel version can be loaded as a Docker image with the docker import command.

All core configurations work with all target platforms:

The default configuration creates a user named "core" mapped into a container, but this can be easily configured in the skiff-core.yaml configuration file:

containers:
  core:
    image: skiffos/skiff-core-gentoo:latest
    [...]
users:
  core:
    container: core
    containerUser: core
    [...]

To customize the core configuration after booting into SkiffOS, edit the file at /mnt/persist/skiff/core/config.yaml.

Release Channels

There are three release channels: next, master, and stable.

Skiff can be upgraded or downgraded (rolled back) independently from the persistent storage partition. This allows for easy OTA, and significant improvements in confidence when upgrading system components.

Configuration Packages/Layers

Skiff supports modular configuration packages. A configuration directory contains kernel configs, buildroot configs, system overlays, etc.

These packages are denoted as namespace/name. For example, an ODROID XU4 configuration would be odroid/xu.

Configuration package directories should have a depth of 2, where the first directory is the category name and the second is the package name.

Package Layout

A configuration package is laid out into the following directories:

├── cflags:         compiler flags in files
├── buildroot:      buildroot configuration fragments
├── buildroot_ext:  buildroot extensions (extra packages)
├── buildroot_patches: extra Buildroot global patches
│   ├── <packagename>: patch files for Buildroot <packagename>
│   └── <packagename>/<version>: patches for package version
├── extensions:     extra commands to add to the build system
│   └── Makefile
├── hooks:          scripts hooking pre/post build steps
│   ├── post.sh
│   └── pre.sh
├── kernel:         kernel configuration fragments
├── kernel_patches: kernel .patch files
├── root_overlay:   root overlay files
├── metadata:       metadata files
│   ├── commands
│   ├── dependencies
│   ├── description
│   └── unlisted
├── resources:     files used by the configuration package
├── scripts:       any scripts used by the extensions
├── uboot:         u-boot configuration fragments
└── uboot_patches: u-boot .patch files

All files are optional.

Out-of-tree configuration packages

You can set the following env variables to control this process:

• SKIFF_CONFIG_PATH_ODROID_XU: Set the path for the ODROID_XU config package. You can set this to add new packages or override old ones.
• SKIFF_EXTRA_CONFIGS_PATH: Colon separated list of paths to look for config packages.
• SKIFF_CONFIG: Name of skiff config to use, or comma separated list to overlay, with the later options taking precedence

These packages will be available in the Skiff system.

Local Overrides

It's often useful to be able to adjust the buildroot, kernel, or other configurations locally during development without actually creating a new configuration layer. This can be easily done with the overrides system.

The overrides directory, as well as the overrides/workspaces/$SKIFF_WORKSPACE directory, are automatically used as additional Skiff configuration packages. You can follow the Skiff configuration package format as defined below to override any of the settings in Buildroot or the Linux kernel, add extra Buildroot packages, add build hooks, etc.

Workspaces

Workspaces allow you to configure and compile multiple systems at a time.

Set SKIFF_WORKSPACE to the name of the workspace you want to use. The Buildroot setup will be constructed in workspaces/$SKIFF_WORKSPACE. You can also place configuration files in overrides/workspaces/$SKIFF_WORKSPACE/ to override settings for that particular workspace locally.

Virtualization

The virt/ packages are designed for running Skiff in various virtualized environments.

Qemu

Here is a minimal working example of running Skiff in Qemu:

$ SKIFF_CONFIG=virt/qemu make configure compile
$ make cmd/virt/qemu/run

Docker

Here is a minimal working example of running Skiff in Docker:

$ SKIFF_CONFIG=virt/docker,skiff/core make configure compile
$ make cmd/virt/docker/buildimage
$ make cmd/virt/docker/run

# inside container
$ su - core

The build command compiles the image, and run executes it.

You can execute a shell inside the container with:

$ make cmd/virt/docker/exec
# alternatively
$ docker exec -it skiff sh

Alternatively, run the latest demo release on Docker Hub:

docker run -t -d --name=skiff \
  --privileged \
  --cap-add=NET_ADMIN \
  --security-opt seccomp=unconfined \
  --stop-signal=SIGRTMIN+3 \
  -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
  -v $(pwd)/skiff-persist:/mnt/persist \
  skiffos/skiffos:latest

Configuration

SkiffOS includes a systemd-based configuration and a standard partition layout, with boot files separated from the persistent data, on default. This can be disabled, overridden, and/or customized by other configuration packages.

NetworkManager

Skiff uses NetworkManager to manage network connections.

Network configurations are loaded from /etc/NetworkManager/system-connections and from skiff/connections on the persist partition.

The configuration file format for these connections is documented here with examples.

You can use nmcli on the device to manage NetworkManager, and any connection definitions written by nmcli device wifi connect or similar will automatically be written to the persist partition and persisted to future boots.

Hostname

You can set the hostname by placing the desired hostname in the skiff/hostname file on the persist partition. You could also set this in one of your config packages by writing the desired hostname to /etc/hostname.

SSH Keys

The system on boot will generate the authorized_keys file for root.

It takes SSH public key files (*.pub) from these locations:

• /etc/skiff/authorized_keys from inside the image
• skiff/keys from inside the persist partition

Build in Docker

You can build Skiff inside Docker if you encounter any incompatibility with your build host operating system.

Support

If you encounter issues or questions at any point when using Skiff, please file a GitHub issue and/or Join Discord.