xonsh Quickstart

xonsh is an interactive system shell that is a Python superset. It is very handy. I'm not going to give you the commercial for it here. If you know Python and you use Bash ever, you already know why you want more Python in your shell.

This guide exists because, while the xonsh documentation is well-written and complete, it involves a lot of reading to be productive with xonsh. The goal of this guide is to allow you to become productive as quickly as possible in the interactive prompt, which is accomplished by omitting a lot of details. Refer to the official documentation for more information if you need it.

I have another tutorial on administrative scripting with Python which is complementary to this one in some ways, in that it shows how to do a lot of common shell tasks in native Python.

This page will not teach any Python. If you want to use Xonsh and don't know Python, you're going to need that. Try the official tutorial. I may at some point write a Python quickstart guide, but I wouldn't hold your breath. This guide also assumes some familiarity with Bash or other POSIX shell.

A Tale of Two Modes

Shell syntax is optimized for launching processes. Python is not. The xonsh solution is to have two modes.

• xonsh has Python mode and subprocess mode which have different syntax.
• Switching between modes is usually implicit, but can be forced.
• Python mode is just Python with a few extras.
• Subprocess mode looks a bit like Bash [1], but it also has big differences.
• xonsh provides constructs which work across both modes. Understanding these constructs is key to effective use of the shell.

Subprocess mode requires a little description, which you will find in the next section. Following that, there is a description of the mechanisms one can use to move data back and forth between these two modes, and finally some configuration tips.

Subprocess Mode

Similarities to POSIX

Subprocess mode is automatic when a line begins with a name that doesn't exist in the current scope. In this mode Syntax is superficially similar to any POSIX shell:

• A list of whitespace-separated arguments is handed to the executable as strings.
• these arguments can be quoted to escape special characters and whitespace.
• Basic globbing also works, and ** may be used in Python 3.5+ for recursive globbing as in some other popular shells.
• && and || work as in a POSIX shell, but the creators encourage the use of and and or instead.
• Backgrounding processes with & works. See job control for more.
• Pipes and I/O redirection also works in a similar way to the shell. | > >> < 2>, etc.

Heredocs are not included. xonsh provides additional, more explicit redirection syntax, but the standard POSIX forms work just fine. I'm not sure to what extent more advanced use of file descriptors is supported, but 2>&1 works (I believe it may be special-cased).

$ ls -l
total 40
-rw-r--r-- 1 ninjaaron ninjaaron 26872 Oct  3 23:01 out.html
-rw-r--r-- 1 ninjaaron ninjaaron 11313 Oct  4 21:32 README.rst
$ touch 'filename with spaces'
$ ls -l 'filename with spaces'
-rw-r--r-- 1 ninjaaron ninjaaron 0 Oct  3 21:15 'filename with spaces'
$ # here's a glob
$ echo /usr/*
/usr/bin /usr/include /usr/lib /usr/lib32 /usr/lib64 /usr/local /usr/sbin /usr/share /usr/src
$ sudo apt-get update && sudo apt-get dist-upgrade
[...]
$ # alternative: sudo apt-get update and sudo apt-get dist-upgrade
$ firefox &
$ # firefox is running along on its merry way.
$ echo /usr/l* | tr a-z A-Z
/USR/LIB /USR/LIB32 /USR/LIB64 /USR/LOCAL

Differences from POSIX

• Strings are Python 3 strings in every way.
• Backslash escapes in arguments are not allowed.
• Strings literals are the only way to escape special shell characters. (... excluding so-called subprocess macros...)

$ rm filename\ with\ spaces
/usr/bin/rm: cannot remove 'filename\': No such file or directory
/usr/bin/rm: cannot remove 'with\': No such file or directory
/usr/bin/rm: cannot remove 'spaces': No such file or directory
$ rm 'filename with spaces'
$

• No brace expansion yet (iterables can be expanded. see: Python Substitution)
• quoting part of a string with special characters and leaving another part unquoted (perhaps for the use of a glob character or brace expansion) is not permitted. The creators of xonsh find this behavior to be "insane".

$ touch "filename with spaces"
$ ls -l "filename with"*
/usr/bin/ls: cannot access '"filename with"*': No such file or directory
$ # ^ someone else's idea of sanity.
$ # xonsh has additional globbing mechanisms to compensate for this
$ # lack, which are covered in the next section.

• Command substitution in subprocess mode only works with $(). Backticks mean something else in xonsh. Both of these features will be covered in more detail in the following section.

That about covers it for the quickstart to subprocesses mode. The next section deals with passing data between the two modes.

Going between the Modes

There are several special xonsh constructs that work both in subprocess mode and in Python mode which can be useful for carting data around, though the first feature we'll cover will be globbing, which isn't exactly a way to move data between the modes.

Globs

Aside from the unquoted globbing behavior in subprocess mode, xonsh supports regex globbing everywhere with backticks. This feels overkill most of the time, but is extremely useful when you need it. It is also somewhat necessitated by the omission of brace expansion.

$ echo `/usr/l.*`
/usr/lib /usr/lib32 /usr/lib64 /usr/local
$ # in a folder containing folders with dates as names...
$ ls -d `18\.0[5-6].*`
18.05.13  18.05.20  18.06.03  18.06.22  18.06.24
18.05.19  18.05.27  18.06.17  18.06.23
$ # in Bash this would be `ls -d 18.0{5..6}*`

Likewise, xonsh supports normal globbing syntax everywhere through the use of g-strings. These are created with backticks and a g prefix.

$ ls -ld g`/usr/l*`
drwxr-xr-x 1 root root 137232 Sep 24 20:50 /usr/lib
drwxr-xr-x 1 root root  38424 Sep 24 20:38 /usr/lib32
lrwxrwxrwx 1 root root      3 Aug 21 16:21 /usr/lib64 -> lib
drwxr-xr-x 1 root root     72 Mar 26  2017 /usr/local

This is once again useful for recursive globbing with ** in Python 3.5+.

One very useful feature glob literals in xonsh is that they can be used to return pathlib.Path instances, which are a very pleasant way of dealing with paths if I do say so myself. This is done by prefixing either type of glob string with a p

>>> for p in p`/etc/.*`:
...     if p.is_dir():
...         print(p)
...
/etc/ImageMagick-6
/etc/ImageMagick-7
/etc/NetworkManager
/etc/UPower
/etc/X11
/etc/asciidoc
/etc/audisp
/etc/audit
[...]

Environment Variables

In xonsh, "environment variables" are prefixed with a $, as in Bash. xonsh's notion of environment variables includes things like $HOME and $PATH, but also includes the assignment of arbitrary values to arbitrary names beginning with $, which only exist for the lifetime of the current shell. These values are global, and they work in both subprocess mode and Python mode. In subprocess mode, this is how they are converted into arguments:

• certain built-in environment variables have predefined conversion functions, which will create a sensible string representation.
• if a variable doesn't have such a function registered (e.g. any variable you create yourself), it will call str() on the object.

An example of the first kind of variable is $PATH which is a wrapper on a list internally, but will print as colon-separated values (as a $PATH would in Bash).

Environment variables work like any other variable in Python mode. Like Bash, these variables can be interpolated freely into strings. Unlike Bash, they don't require quoting for safety.

>>> for $p in p`/etc/.*`:
...     if $p.is_dir():
...         echo '$p is a directory'
...
/etc/ImageMagick-6 is a directory
/etc/ImageMagick-7 is a directory
/etc/NetworkManager is a directory
/etc/UPower is a directory
[...]

Substitutions

Python Substitution

One problem with user-created environment variables is that they just call str() when they are used in subprocess mode. That means:

$ $dirs = ['/usr', '/bin', '/etc']
$ ls -ld $dirs
/usr/bin/ls: cannot access '['\''/usr'\'', '\''/bin'\'', '\''/etc'\'']': No such file or directory

The way to get this to do the right thing is with Python substitution. Python substitution allows embedding the value of arbitrary Python expressions into commands. If the Python value is an iterable, it will be split into separate arguments. Python substitution is marked with @().

$ dirs = ['/usr', '/bin', '/etc']
$ ls -ld @(dirs)
lrwxrwxrwx 1 root root    7 Aug 21 16:21 /bin -> usr/bin
drwxr-xr-x 1 root root 3068 Sep 25 22:47 /etc
drwxr-xr-x 1 root root   80 Sep 25 19:43 /usr
$ echo hello-@('foo    bar     baz'.split())
hello-foo hello-bar hello-baz
$ # Cartesian products can also be produced
$ echo @(list('abc')):@(list('def'))
a:d a:e a:f b:d b:e b:f c:d c:e c:f

Python substitution only works in subprocess mode (because it is redundant in Python mode).

Command Substitution(s)

xonsh has two forms of command substitution. The first is similar to that of Bash, using $() syntax.

$ ls -l $(which vi)
lrwxrwxrwx 1 root root 4 Feb 27  2018 /usr/bin/vi -> nvim
$ # why are permissions on this alias set to 777 instead of 755?
$ # Oh well...

If this form of substitution is used in Python mode, it returns a string.

$ print(repr($(which vi)))
'/usr/bin/vi'

The other form of command substitution only works in Python mode, where it returns a CommandPipeline object, which among other things, implements an iterator that lazily yields lines as they become available from the process. Trailing newlines are not stripped.

>>> for line in !(ls):
...     print(line.split())
...
['total', '40']
['-rw-r--r--', '1', 'ninjaaron', 'ninjaaron', '26872', 'Oct', '3', '23:01', 'out.html']
['-rw-r--r--', '1', 'ninjaaron', 'ninjaaron', '10726', 'Oct', '3', '23:20', 'README.rst']

This object has other interesting properties as well, such as boolean coercion based on the exit code of the process. Look at the documentation for further details. This form of substitution is probably what you generally want in Python mode.

You can also force subprocess mode without capturing output using $[] and ![]. ![] returns a (weirdly unprintable) object with information about the process. $[] always returns None, and I don't know why anyone would ever use it over ![].

Basic Configuration, etc.

Information on configuration is spread out all over the official docs, which was the most frustrating thing for me when I was trying it out the first time. I've tried to collect them here.

• Run Control File tells about ~/.xonshrc and has some things you might want to stick in it.
• Customizing xonsh also shows how to do some interesting things like set the color scheme, but also some bad things like how to set xonsh as your default shell. Believe it or not, some 3rd-party programs do rely on the default shell setting, and setting your shell with chsh to a non-POSIX shell can break such programs. My advice is to instead configure your terminal to launch with xonsh running, rather than to change your default shell.
• Customizing the Prompt
• Environment Variables is a complete list of environment variables, many of which can be used for settings.
• Aliases work differently in xonsh than in other shells.

Personally, because I use several shells (zsh, fish and xonsh), I try to avoid more complex functions in my shell configuration files. Instead, I keep simple aliases in one place and parse out the correct code for different shells from there. This is how I do it in xonsh.

from hashlib import md5
# import shell aliases
al_cache = p'$HOME/.cache/ali_cache.xsh'  # home of xonsh aliases.
al_path = p'$HOME/.aliases'               # home of POSIX aliases.
al_hash = p'$HOME/.cache/ali_hash'        # place where a hash of the
                                          # POSIX aliases live.

# see if the file has changed. Probably should just do this with timestamps.
with al_path.open('rb') as af, al_hash.open('rb') as ah:
    old_hash = ah.read()
    shell_aliases = af.read()
    new_hash = md5(shell_aliases).digest()

if old_hash != new_hash:
    # find lines containing aliases and reformate them to xonsh aliases.
    ali = '\n'.join(i for i in shell_aliases.decode().splitlines()
                    if i.startswith('alias '))
    ali = re.sub(r'^alias ([\w-]*)=(.*?)$', r"aliases['\1'] = \2",
                 ali, flags=re.M)
    with al_hash.open('wb') as ah, al_cache.open('w') as ac:
        ah.write(new_hash)
        ac.write(ali)

    exec(ali)

else:
    source @(al_cache)

For things that cannot be expressed as simple aliases, I try to just write scripts so I don't have to worry about portability between my various exotic shells.

My whole xonshrc is here. It contains some strange and possibly bad ideas. It comes with no warranty.

Working with Python Virtual Environments

I don't know if this is exactly part of configuration, but, as you might expect, the standard virtualenv tools don't work with xonsh. This may not exactly be a configuration thing, but it's something Python developers need to know, and the info about it is here: https://xon.sh/python_virtual_environments.html

Using xonsh over SSH

If you enjoy using xonsh and wish to use it on remote systems without having to install it everywhere you go, you may want to check out xxh, a program for executing xonsh interactively on remote systems.