shallow-backup

shallow-backup lets you easily create lightweight backups of installed packages, applications, fonts and dotfiles, and automatically push them to a remote Git repository.

Contents

• Why?
• Installation
• Usage
• Git Integration
• What can I back up?
• Configuration
• Output Structure
• Reinstalling Dotfiles
• Want to contribute?

Why?

I wanted a tool that allows you to:

• Back up dotfiles from where they live on the system.
• Back up files from any path on the system, not just $HOME.
• Reinstall them from the backup directory idempotently.
• Backup and reinstall files conditionally, so you can easily manage dotfiles across multiple systems.
• Copy files on installation and backup, as opposed to symlinking them.
• Backup package installations in a highly compressed manner

And is incredibly fault tolerant and user-protective.

shallow-backup is the only tool that checks all of those boxes.

Installation

Do not install this as root.

1. Install with pip3

   • $ pip3 install shallow-backup
   • $ shallow-backup

2. Download the shallow-backup binary from Releases tab.

Usage

To start the interactive program, simply run $ shallow-backup.

shallow-backup was built with scripting in mind. Every feature that's supported in the interactive program is supported with command line arguments.

Usage: shallow-backup [OPTIONS]

  Easily back up installed packages, dotfiles, and more.
  You can edit which files are backed up in ~/.shallow-backup.

  Written by Aaron Lichtman (@alichtman).

Options:
  --add-dot TEXT              Add a dotfile or dotfolder to config by path.
  -backup-all                 Full back up.
  -backup-configs             Back up app config files.
  -backup-dots                Back up dotfiles.
  -backup-packages            Back up package libraries.
  -delete-config              Delete config file.
  -destroy-backup             Delete backup directory.
  -dry-run                    Don't backup or reinstall any files, just give
                              verbose output.

  -backup-fonts               Back up installed fonts.
  --new-path TEXT             Input a new back up directory path.
  -no-new-backup-path-prompt  Skip setting new back up directory path prompt.
  -no-splash                  Don't display splash screen.
  -reinstall-all              Full reinstallation.
  -reinstall-configs          Reinstall configs.
  -reinstall-dots             Reinstall dotfiles and dotfolders.
  -reinstall-fonts            Reinstall fonts.
  -reinstall-packages         Reinstall packages.
  --remote TEXT               Set remote URL for the git repo.
  -separate-dotfiles-repo     Use if you are trying to maintain a separate
                              dotfiles repo and running into issue #229.

  -show                       Display config file.
  -v, --version               Display version and author info.
  -h, -help, --help           Show this message and exit.

Git Integration

A Word of Caution

This backup tool is git-integrated, meaning that you can easily store your backups remotely (on GitHub, for example.) Dotfiles and configuration files may contain sensitive information like API keys and ssh keys, and you don't want to make those public. To make sure no sensitive files are uploaded accidentally, shallow-backup creates a .gitignore file if it can't find one in the directory. It excludes .ssh/ and .pypirc by default. It's safe to remove these restrictions if you're pushing to a remote private repository, or you're only backing up locally. To do this, you should clear the .gitignore file without deleting it.

If you choose to back up to a public repository, look at every file you're backing up to make sure you want it to be public.

What if I'd like to maintain a separate repo for my dotfiles?

shallow-backup makes this easy! After making your first backup, cd into the dotfiles/ directory and run $ git init. Create a .gitignore and a new repo on your favorite version control platform. This repo will be maintained independently (manually) of the base shallow-backup repo. Note that you may need to use the -separate_dotfiles_repo flag to get this to work, and it may break some other functionality of the tool. It's ok for my use case, though.

Here's a bash script that I wrote to automate my dotfile backup workflow. You can use this by placing it in your $PATH, making it executable, and running it.

What can I back up?

By default, shallow-backup backs these up.

1. Dotfiles and dotfolders

   • .bashrc
   • .bash_profile
   • .gitconfig
   • .pypirc
   • .config/shallow-backup.conf
   • .ssh/
   • .vim/
   • .zshrc

2. App Config Files

   • Atom
   • VSCode
   • Sublime Text 2/3
   • Terminal.app

3. Installed Packages

   • apm
   • brew and cask
   • cargo
   • gem
   • pip
   • pip3
   • npm
   • macports
   • VSCode Extensions
   • Sublime Text 2/3 Packages
   • System Applications

4. User installed fonts.

Configuration

If you'd like to modify which files are backed up, you have to edit the JSON config file, located at ~/.config/shallow-backup.conf. There are two ways to do this.

1. Select the appropriate option in the CLI and follow the prompts.
2. Open the file in a text editor and make your changes.

Editing the file in a text editor will give you more control and be faster.

Conditional Backup and Reinstallation

Every dotfile has two subkeys: backup_condition and reinstall_condition. Both of these accept expressions that will be evaluated in bash. An empty string ("") is the default value, and is considered to be True. If the return value of the expression is 0, this is considered True. Otherwise, it is False. This lets you do simple things like preventing backup with:

// Because `$ false` returns 1
"backup_condition": "false"

And also more complicated things like only backing up certain files if an environment variable is set:

"backup_condition": "[[ -n \"$ENV_VAR\" ]]"

My config (as of v5.0.0a) looks like this, and is used to back up my dotfiles:

{
	"backup_path": "~/shallow-backup",
	"lowest_supported_version": "5.0.0a",
	"dotfiles": {
		".config/agignore": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/crontab": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/fzf-notes": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/git/config": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/git/gitignore_global": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/jrnl/jrnl.yaml": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/kitty": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/nvim": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/pycodestyle": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/pylintrc": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/python": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/radare2/radare2rc": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/ranger": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/shallow-backup.conf": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/starship.toml": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/tmux": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/tuir/tuir.cfg": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/zathura/zathurarc": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".config/zsh": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".ctags": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".ghc/ghci.conf": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".pypirc": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".ssh": {
			"reinstall_condition": "",
			"backup_condition": ""
		},
		".zshenv": {
			"reinstall_condition": "",
			"backup_condition": ""
		}
	},
	"root-gitignore": [
		".DS_Store",
		"dotfiles/.config/nvim/.netrwhist",
		"dotfiles/.config/nvim/spell/en.utf-8.add",
		"dotfiles/.config/ranger/plugins/ranger_devicons",
		"dotfiles/.config/zsh/.zcompdump*",
		"dotfiles/.pypirc",
		"dotfiles/.ssh"
	],
	"dotfiles-gitignore": [
		".DS_Store",
		".config/nvim/.netrwhist",
		".config/nvim/spell/en.utf-8.add*",
		".config/ranger/plugins/*",
		".config/zsh/.zcompdump*",
		".config/zsh/.zinit",
		".config/tmux/plugins",
		".config/tmux/resurrect",
		".pypirc",
		".ssh/*"
	],
	"config_mapping": {
		"/Users/alichtman/Library/Application Support/Sublime Text 2": "sublime2",
		"/Users/alichtman/Library/Application Support/Sublime Text 3": "sublime3",
		"/Users/alichtman/Library/Application Support/Code/User/settings.json": "vscode/settings",
		"/Users/alichtman/Library/Application Support/Code/User/Snippets": "vscode/Snippets",
		"/Users/alichtman/Library/Application Support/Code/User/keybindings.json": "vscode/keybindings",
		"/Users/alichtman/.atom": "atom",
		"/Users/alichtman/Library/Preferences/com.apple.Terminal.plist": "terminal_plist"
	}
}

.gitignore

As of v4.0, any .gitignore changes should be made in the shallow-backup config file. .gitignore changes that are meant to apply to all directories should be under the root-gitignore key. Dotfile specific gitignores should be placed under the dotfiles-gitignore key. The original default-gitignore key in the config is still supported for backwards compatibility, however, converting to the new config format is strongly encouraged.

Output Structure

backup_dir/
├── configs
│   ├── plist
│   │   └── com.apple.Terminal.plist
│   ├── sublime_2
│   │   └── ...
│   └── sublime_3
│       └── ...
├── dotfiles
│   ├── .bash_profile
│   ├── .bashrc
│   ├── .gitconfig
│   ├── .pypirc
│   ├── ...
│   ├── .shallow-backup
│   ├── .ssh/
│   │   └── known_hosts
│   ├── .vim/
│   └── .zshrc
├── fonts
│   ├── AllerDisplay.ttf
│   ├── Aller_Bd.ttf
│   ├── Aller_BdIt.ttf
│   ├── Aller_It.ttf
│   ├── ...
│   ├── Ubuntu Mono derivative Powerline Bold Italic.ttf
│   ├── Ubuntu Mono derivative Powerline Bold.ttf
│   ├── Ubuntu Mono derivative Powerline Italic.ttf
│   └── Ubuntu Mono derivative Powerline.ttf
└── packages
    ├── apm_list.txt
    ├── brew-cask_list.txt
    ├── brew_list.txt
    ├── cargo_list.txt
    ├── gem_list.txt
    ├── installed_apps_list.txt
    ├── npm_list.txt
    ├── macports_list.txt
    ├── pip_list.txt
    └── sublime3_list.txt

Reinstalling Dotfiles

To reinstall your dotfiles, clone your dotfiles repo and make sure your shallow-backup config path can be found at either ~/.config/shallow-backup.conf or $XDG_CONFIG_HOME/.shallow_backup.conf. Set the backup-path key in the config to the path of your cloned dotfiles. Then run $ shallow-backup -reinstall-dots.

When reinstalling your dotfiles, the top level .git/, .gitignore, img/ and README.md files and directories are ignored.

Want to Contribute?

Check out CONTRIBUTING.md and the docs directory.