Containers

This package is a simple implementation of containers for Linux, making secure containers as easy to create and use as a traditional chroot. It comprises three utilities, contain, inject and pseudo, which use the kernel support for user namespaces merged in Linux 3.8.

Demonstration

With the utilities already installed, the demo begins in an unprivileged user's shell:

$ echo $$ $UID 21260 1000

To create a simple test container, copy /bin and /lib* from the host into a temporary directory with the default UID/GID mappings applied:

$ cd $(mktemp -d) $ tar -c -f - -C / bin lib lib32 lib64 | pseudo tar -x -f -

It is very straightforward to launch a container with this newly-created root filesystem:

$ contain . /bin/bash

The new shell has PID 1 within the container, and cannot see other processes on the host:

echo $$ $UID

1 0

ps ax

PID TTY      STAT   TIME COMMAND
  1 console  Ss     0:00 /bin/bash
  2 console  R+     0:00 ps ax

The container root user is able to manipulate ownerships and permissions within its filesystem:

ls -l /dev/console

crw--w---- 1 0 5 136, 9 Jul 1 14:00 /dev/console

chown 12:34 /dev/console

chmod a+rw /dev/console

ls -l /dev/console

crw-rw-rw- 1 12 34 136, 9 Jul 1 14:00 /dev/console

and can also make other privileged changes such as setting the hostname:

echo -n "hostname $(hostname) -> " && hostname brian && hostname

hostname alice -> brian

or configuring the network stack:

ip link show

1: lo: mtu 65536 qdisc noop state DOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

ping -w 1 1.2.3.4 &>/dev/null && echo up || echo down

down

ip addr add 1.2.3.4/32 dev lo && ip link set lo up

ping -w 1 1.2.3.4 &>/dev/null && echo up || echo down

up

ip link add type veth && ip link show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether 3a:0c:96:36:2d:ff brd ff:ff:ff:ff:ff:ff 3: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether a2:86:1a:92:58:cb brd ff:ff:ff:ff:ff:ff

In all cases, these changes affect the container but not the host as a whole. Processes in the container live in different resource namespaces isolated from the host, and the container root user is unable to do anything that would require elevated capabilities or root privilege on the host itself.

contain

The contain utility is invoked as

contain [OPTIONS] DIR [CMD [ARG]...]

with options

-c disable console emulation in the container -g MAP set the container-to-host GID map -i CMD run a helper child inside the new namespaces -n share the host network unprivileged in the container -o CMD run a helper child outside the new namespaces -u MAP set the container-to-host UID map

and creates a new container with DIR recursively bound as its root filesystem, running CMD as PID 1 within that container. If unspecified, CMD defaults to /bin/sh to start a shell, so to fully boot a distribution, specify CMD as /bin/init or /sbin/init.

The container init process is isolated in new user, cgroup, mount, IPC, UTS, time and PID namespaces. A synthetic /dev with device nodes bound from the host /dev is automatically mounted within the new mount namespace, together with standard /dev/pts, /proc and /sys filesystems.

Because it runs in its own user namespace, users and groups seen inside a container are not the same as the underlying credentials visible for the same processes and files on the host. Sensible default container-to-host UID and GID mappings are provided and described below, but the -u and -g options can be used to override the defaults.

The container console is a host pseudo-terminal bound at /dev/console in the new /dev filesystem: stdin and stdout are copied to/from this, and it serves as stdin, stdout and stderr for the container init process. This console emulation can be disabled using the -c option: if -c is used, init is run directly with the stdin, stdout and stderr of the contain command.

Containers are usually isolated in their own network namespace, with a distinct set of network interfaces from the host. By specifying the -n option, it is possible to safely share the host network stack instead. If you do this, user networking within the container will work normally, but the container has no privileges with respect to its network namespace so it isn't possible to (re)configure interfaces or routes, and setuid utilities like ping which use a raw socket will fail.

Two different kinds of helper program can be used to help set up a container. A program specified with -i is run inside the new namespaces with the new root filesystem as its working directory, just before pivoting into it. Typically this type of helper is used to bind-mount additional parts of the host filesystem inside the container.

A helper specified with -o is run outside the namespaces but as a direct child of the supervisor process which is running within them. This type of helper can be used to move host network interfaces (such as a macvtap interface or one half of a veth pair) into the container's network namespace.

The environment of the container init process includes "container=contain" so that distributions can identify when they are running under contain.

inject

The inject utility is invoked as

inject PID [CMD [ARG]...]

where PID is the process ID of a running container supervisor, and runs a command or shell inside the existing container. The environment, stdin, stdout and stderr of inject are all inherited by the command to be run.

The container supervisor PID (i.e. that of contain itself) should be given to inject, not the PID of the descendant init process. The inject utility will only work if the process specified has a child with "container=contain" in its environment, which it assumes to be the container init.

Linux allows an unprivileged user to join the user namespace of any process he can dump or ptrace, so inject need not be installed setuid even if contain and pseudo are setuid root. It will refuse to run if it detects setuid/setgid operation.

pseudo

The pseudo utility is invoked as

pseudo [OPTIONS] [CMD [ARG]...]

with options

-g MAP set the user namespace GID map -u MAP set the user namespace UID map

and runs a command or shell as root in a new user namespace, by analogy with sudo which runs a command as root in the host user namespace.

Unlike contain, pseudo does not unshare other namespaces or attempt to isolate the new process from the rest of the host. It has identical default UID/GID mappings, -u and -g options, and support for /etc/subuid and /etc/subgid when installed setuid root, but no other contain options are supported.

One use for pseudo is as a more capable replacement for fakeroot, useful for testing, when building software packages or for constructing system images. Unlike the traditional fakeroot approach based on LD_PRELOAD, static binaries and chroot jails are both handled correctly.

It is also invaluable for running host software to access the same filesystem as a container, replicating the user and group file ownerships that the container would see. For example, in the demo above, the system image is untarred under pseudo so that files are written into the filesystem with UIDs and GIDs mapped for the container rather than unmapped as on the host.

User and group mappings

By default, when run as root, contain and pseudo will map container UID/GID 0 onto the highest available host UID/GID (4294967294 unless nested), and all other UIDs/GIDs are mapped onto themselves apart from the top container UID and GID which must be left unmapped.

The default mappings avoid host UID and GID 0 as the host root user is still granted a variety of privileges even after dropping all capabilities in the host user namespace. For example, /proc and /sys files typically have (host) root:root ownership, and allowing the container access unfiltered access to things like /proc/sys is dangerous.

Run as an unprivileged user, container UID/GID 0 is mapped onto the unprivileged user's UID/GID, then container UIDs/GIDs 1, 2, etc. are successively mapped onto any ranges delegated to that user in /etc/subuid and /etc/subgid.

The -u and -g options can be used to specify custom mappings, in the format START:LOWER:COUNT[,START:LOWER:COUNT]... where START is the first UID/GID in a container range, LOWER is the first UID/GID in the corresponding range in the host, and COUNT is the length of these ranges.

For example, -u 0:1000:1,1:4000:2000 will map container UID 0 onto host UID 1000 and container UIDs 1...2000 onto host UIDs 4000...5999.

It is not possible to map more than one container ID onto a given host ID, nor to list the same container ID twice in a map specification. When invoked by an unprivileged user, all host ranges are checked against /etc/subuid and /etc/subgid.

Unmapped users and groups are mapped by the kernel onto the overflow UID and GID set in /proc/sys/kernel/overflowuid and /proc/sys/kernel/overflowgid. By default the kernel sets both these values to 65534.

Unprivileged operation, /etc/subuid and /etc/subgid

When a non-root user runs contain or pseudo unprivileged, these tools can only map container UID/GIDs onto the host UID/GID of that user. The resulting container is not very useful as it has just a single user and group available. (Typically only root is mapped in the container.)

However, contain and pseudo can also be installed setuid root, and in this case, unprivileged users can also map onto ranges of UIDs/GIDs that have been delegated for their use in /etc/subuid and /etc/subgid.

The format of these files is similar to /etc/passwd, /etc/group and /etc/shadow. Each line specifies an additional range of UIDs/GIDs allocated to a particular user, and there can be zero, one, or multiple lines for any given user. There are three colon-delimited fields: the user's login name, the first UID/GID in the range, and the number of UIDs/GIDs in the range. For example, an /etc/subuid containing the lines

chris:100000:10000 chris:120000:10000

allocates UID ranges 100000-109999 and 120000-129999 to my user 'chris' in addition to my normal login UID.

The kernel user namespace author Eric Biederman [email protected] has proposed patches against the standard GNU/Linux Shadow package which add support for creating and updating these files in this format; they are likely to become a standard way to delegate sub-users and sub-groups.

Linux 3.19 and later do not allow unprivileged processes to write a GID map unless the setgroups() call has been permanently disabled by writing "deny" to /proc/PID/setgroups. This is a fix for CVE-2014-8989 which applied to strangely-configured systems where group membership implies more restricted permissions rather than supplementary permissions.

As a result, when run non-setuid by an unprivileged user, contain and pseudo must disable setgroups() in the container. Conversely, when installed setuid root, they will use their privilege to bypass this kernel restriction, resulting in fully-functional containers which still support setgroups(). However, this also means that they can be used to bypass restrictions implemented by group membership.

Building and installing

Unpack the source tar.gz file and change to the unpacked directory.

Run 'make', then 'make install' as root to install both binaries setuid root in /bin. Alternatively, you can set DESTDIR and/or BINDIR to install in a different location, or strip and copy the compiled binaries into the correct place manually.

Note that setuid contain and pseudo effectively enable unprivileged users to to drop supplementary group memberships using setgroups(). Consequently, they should NOT be installed setuid root on systems where group membership implies more restricted permissions rather than supplementary permissions.

These utilities were developed on GNU/Linux and are not portable to other platforms as they rely on Linux-specific facilities such as namespaces. Please report any problems or bugs to Chris Webb [email protected].

Copying

This software was written by Chris Webb [email protected] and is distributed as Free Software under the terms of the MIT license in COPYING.