Utopia

Utopia is a comprehensive modelling framework for complex and evolving systems. It aims to provide the tools to conveniently implement a model, configure and perform simulation runs, and evaluate the resulting data.

Utopia is free software and licensed under the GNU Lesser General Public License Version 3 or any later version. For the copyright notice and the list of copyright holders, see COPYING.md.

Overview

Complex and evolving systems are investigated with a wide array of different models types. Among the most popular are cellular automata (CA), agent-based models (ABMs), and network models. Utopia provides a library to build these types of models in modern C++. Its core template library contains functions for implementing model dynamics and scaffoldings for building new models. The Utopia dataIO library is capable of storing hierarchical simulation data efficiently in the H5 file format. Within Utopia, and when using Utopia as a dependency, models are integrated into a simulation infrastructure for conveniently executing and analyzing them.

The utopya Python package constitutes Utopia's frontend. It configures and performs the simulation runs and embeds the model into a data processing pipeline, such that the simulation data can directly be analyzed and visualized. All parts of the frontend make use of a hierarchic, recursively-updated YAML-based configuration structure. Using the paramspace package, this allows to easily define parameter sweeps, which can then be carried out as simultaneous simulations on massively parallel, distributed machines. The dantro-based data processing pipeline automates visualization, thereby coupling the model implementation and its analysis closer together.

Several models are readily included in the framework, among them Conway's Game of Life, one CA- and one agent-based contagious disease model, and a network-based model of social opinion dynamics. Investigating these models by performing simulation runs with a few varying parameters, or sensitivity analysis over a large parameter space, requires little to no programming skills.

For introductory guides, feature lists, FAQs, and API references refer to the online user manual at docs.utopia-project.org

Utopia development happens mainly on our self-hosted GitLab instance at ts-gitlab.iup.uni-heidelberg.de. Additionally, you can retrieve the code and get in contact via mirrored repositories on GitLab.com and GitHub.

How to cite Utopia

Utopia was reviewed and published in the Journal of Open Source Software (JOSS). Please cite at least the following publication if you use Utopia (or a modified version thereof) for your own work:

Lukas Riedel, Benjamin Herdeanu, Harald Mack, Yunus Sevinchan, and Julian Weninger. 2020. “Utopia: A Comprehensive and Collaborative Modeling Framework for Complex and Evolving Systems.” Journal of Open Source Software 5 (53): 2165. DOI: 10.21105/joss.02165.

The CITATION.cff file in this repository follows the citation file format and contains additional metadata to reference this software, its authors, and associated publications. See the user manual for more information and BibTeX data.

Contents of this README

• How to install
  • On your machine
  • Alternative: docker image
• Quickstart
• Documentation and Guides
• Information for Developers
• Dependencies
• Troubleshooting

How to install

The following instructions will install Utopia into a development environment on your machine. If you simply want to run Utopia, you can do so via a ready-to-use docker image; see below for more information.

Step-by-step Instructions

These instructions are intended for 'clean' macOS (both Intel and Apple Silicon) or Ubuntu (20.04 or newer) setups. Since Windows supports the installation of Ubuntu via Windows Subsystem for Linux, Utopia can also be used on Windows. Follow the WSL Installation Guide to install Ubuntu, then follow the instructions for Ubuntu in this README.

Note: Utopia is always tested against a recent Ubuntu release. However, you may also use Utopia with an earlier release, as long as the dependencies can be fulfilled.

⚠️ If you encounter difficulties, have a look at the troubleshooting section. If this does not resolve your installation problems, please file an issue in the GitLab project.

1 — Clone Utopia

First, create a Utopia directory at a place of your choice. This is where the Utopia repository will be cloned to. When working with or developing for Utopia, auxiliary data will have a place there as well.

In your terminal, enter the Utopia directory you just created.

Now, get a clone URL via the Clone button in the top right-hand corner of the Utopia project page.
If you are a developer, you should get an SSH key registered with the GitLab and use the SSH address. Otherwise, use the HTTPS address.

To clone the repository, use the following command and add the chosen clone URL at the indicated place:

git clone UTOPIA-CLONE-URL

After cloning, there will be a new utopia directory (mirroring this repository) inside your top-level Utopia directory.

2 — Install dependencies

Install the third-party dependencies using a package manager.

Note: If you have Anaconda installed, you already have a working Python installation on your system, and you can omit installing the python packages below. However, notice that there might be issues during the configuration step. Have a look at the troubleshooting section to see how to address them.

On Ubuntu (20.04)

apt update
apt install cmake gcc g++ gfortran git libarmadillo-dev libboost-all-dev \
            libhdf5-dev libspdlog-dev libyaml-cpp-dev pkg-config \
            python3-dev python3-pip python3-venv

Further, we recommend installing the following optional packages:

apt update
apt install ffmpeg graphviz doxygen

You will probably need administrator rights. (sudo, anyone?)

Note: For simplicity, we suggest installing the meta-package libboost-all-dev which includes the whole Boost library. If you want a minimal installation (only the strictly required components), use the following packages instead: libboost-dev libboost-test-dev libboost-graph-dev libboost-regex-dev.

On macOS

First, install the Apple Command Line Tools:

xcode-select --install

There are two popular package managers on macOS, Homebrew and MacPorts. We recommend you use Homebrew. Here are the installation instructions for both:

• Homebrew:

  Install the required packages:

  brew update
  brew install armadillo boost cmake hdf5 pkg-config python3 spdlog yaml-cpp

  Further, we recommend installing the following optional packages:

  brew update
  brew install ffmpeg graphviz doxygen

• MacPorts:

  Please be aware that port commands typically require administrator rights (sudo).

  Install the required packages:

  port install armadillo boost cmake hdf5 python37 py37-pip spdlog yaml-cpp

  Select the installed versions of Python and Pip:

  port select --set python python37
  port select --set python3 python37
  port select --set pip pip37
  port select --set pip3 pip37

  Further, we recommend installing the following optional packages:

  port install doxygen ffmpeg graphviz

3 — Configure and build

Enter the repository and create your desired build directory:

cd utopia
mkdir build

Now, enter the build directory and invoke CMake (and mind the caveats below):

cd build
cmake ..

Note: If you are using MacPorts, append the location of your Python installation to the CMake command (this is only required when calling CMake on a clean build directory):

cmake -DPython_ROOT_DIR=/opt/local ..

The terminal output will show the configuration steps, which includes the installation of further Python dependencies and the creation of a virtual environment.

After this, you can build a specific or all Utopia models using:

make dummy     # builds only the dummy model
make -j4 all   # builds all models, using 4 CPUs

4 — Run a model 🎉

You should now be able to run a Utopia model. Being in the build directory, call:

source ./activate

to enter the virtual environment, where the Utopia Command Line Interface (CLI) is available. (If you later want to exit the virtual environment, call the deactivate command.)

Note: If you are using csh or fish shells, use the respective activate scripts located in build/utopia-env/bin/ (see below).

The utopia command is now available and gives you control over running and evaluating model simulations:

utopia run dummy

The model output will be written into ~/utopia_output/dummy/<timestamp>. For more information on how to use the command line interface, see the information for users below and the documentation.

5 — Make sure everything works

This step is optional, but recommended.

To make sure that Utopia works as expected on your machine, build and carry out all tests.

Optional Installation Steps

The following instructions will enable additional, optional features of Utopia.

Enable Multithreading in Utopia Models

1. Install the optional dependencies for multithreading.

   • On Ubuntu, we recommend using the GCC compiler with Intel TBB:

     apt update && apt install libtbb-dev

     Alternatively, one may install the Intel oneAPI base toolkit following these installation instructions. The only required package is intel-basekit. It includes the oneDPL library we use for parallelization.

   • On macOS, with Homebrew (please mind the note below!):

     brew update && brew install onedpl

   • On macOS with MacPorts, we are currently unsure whether multithreading is workable.

2. Enter the Utopia build directory, and call CMake again. This time, enable the use of multithreading with the MULTITHREADING option:

   cd build
   cmake -DMULTITHREADING=On ..

   At the end of its output, CMake should now report that the "Multithreading" feature has been enabled. If the requirements for multithreading are not met, however, the build will fail.

3. Re-compile the models. Inside the build directory, call

   make all

4. Depending on the algorithms used inside the respective model, it will automatically exploit the multithreading capabilities of your system when executed! 🎉

Utopia via Docker

Docker is a free OS-level virtualization software. It allows running any application in a closed environment container.

The Utopia docker image is a way to run Utopia models and evaluate them without having to go through the installation procedure. It is also suitable for framework and model development.

The image and instructions on how to create a container from it can be found on the ccees/utopia docker hub page.

Quickstart

This section gives a glimpse into working with Utopia. It's not more than a glimpse; after playing around with this, consult the documentation to gain access to more information material, and especially: the Utopia Tutorial.

How to run a model?

The Utopia command line interface (CLI) is, by default, only available in a Python virtual environment, in which utopya (the Utopia frontend) and its dependencies are installed. To conveniently work with the frontend, you should thus enter the virtual environment. Execute one of the commands below, depending on which type of shell you're using:

source ./build/activate                  # For bash, zsh, or similar
source ./build/utopia-env/activate.csh   # For csh
source ./build/utopia-env/activate.fish  # For fish

Now, your shell should be prefixed with (utopia-env). (To exit the virtual environment, simply call deactivate.) All the following should take place inside this virtual environment.

As you have already done with the dummy model, the basic command to run a model named SomeModel is:

utopia run SomeModel

where SomeModel needs to be replaced with a valid model name. To get a list of available models, run the utopia models ls command. Alternatively, have a look at the src/utopia/models directory, where they are implemented.

The utopia run command carries out a pre-configured simulation for that model, loads the data, and performs automated plots.

Note:

• The script will always run through, even if there were errors in the individual parts. Thus, you should check the terminal output for errors and warnings (red or yellow, respectively). If you want the CLI to fail on errors, you can also set the --debug flag.
• By default, you can find the simulation output in ~/utopia_output/SomeModel/<timestamp>. It contains the written data, all the used configuration files, and the default plots.

For further information, e.g. on how you can pass a custom configuration, check the CLI help:

utopia --help
utopia run --help

The best place to continue from here is the tutorial.

Utopia Documentation

This Readme only covers the very basics. For all further documentation, tutorials, guides, model descriptions, frequently asked questions and more, you should consult the actual Utopia documentation.

The latest build (corresponding to the latest commit to master) is available online.

See below on how to build the documentation locally.

Information for Developers

SSH Repository Access

To work comfortably with Utopia, access to the GitLab needs to be easy. The best way to achieve that is by registering a so-called SSH key with your GitLab account.

To do that, follow the linked instructions to generate a key pair and to add a key to your GitLab account.

New to Utopia? How do I implement a model?

Aside from exploring the already existing models, you should check out the guides in the documentation which will guide you through the process of implementing your very own Utopia model.

Building the documentation locally

To build the documentation locally, navigate to the build directory and run

make doc

The Sphinx-built user documentation will then be located at build/doc/html/. The C++ doxygen-documentation can be found at build/doc/doxygen/html/. Open the respective index.html files to browse the documentation.

Choosing a different compiler

CMake will inspect your system paths and use the default compiler. You can use the CC and CXX environment variables to select a specific C and C++ compiler, respectively. As compiler paths are cached by CMake, changing the compiler requires you to delete the build directory and re-run CMake.

Ubuntu 20.04 LTS (Focal Fossa), for example, provides GCC 9 and GCC 10. To use GCC 10, first install it via APT:

apt update && apt install gcc-10 g++-10

Then create the build directory, enter it, and set the CC and CXX environment variables when calling CMake:

mkdir build && cd build
CC=/usr/bin/gcc-10 CXX=/usr/bin/g++-10 cmake ..

Alternatively, you can export these variables in separate bash commands before executing CMake.

Build Types

If you followed the instructions above, you have a Release build which is optimized for maximum performance. If you need to debug, you should reconfigure the entire project with CMake by navigating to the build folder and calling

cmake -DCMAKE_BUILD_TYPE=Debug ..

Afterwards, call

make <something>

to rebuild executables. CMake handles the required compiler flags automatically.

The build type (as most other CMake flags) persists until it is explicitly changed by the user. To build optimized executables again, reconfigure with

cmake -DCMAKE_BUILD_TYPE=Release ..

Testing

Utopia contains unit tests to ensure consistency by checking whether class members and functions are working correctly. This is done both for the C++ and the Python code. The tests are integrated into the GitLab Continuous Integration pipeline, meaning that tests are run upon every push to the project and failing tests can be easily detected. Tests can also be executed locally, to test a (possibly altered) version of Utopia before committing and pushing changes to the GitLab.

See the user manual for more details on how to run tests.

Setting up a separate repository for models

Working inside a clone or a fork of this repository is generally not a good idea: it makes updating harder, prohibits efficient version control on the models, and makes it more difficult to include additional dependencies or code.

To make setting up such a project repository as easy as possible, we provide a template repository, which can be used to start a new Utopia project! Follow the instructions in the models_template project for more information.

Dependencies

Utopia aims to allow rapid development, and is thus being tested only against the more recent releases of its dependencies. Currently, Utopia is tested against the packages provided by Ubuntu 20.04. However the above version requirements (i.e., those enforced by the build system) can be fulfilled also with Ubuntu 19.10.

To get Utopia running on a system with an earlier Ubuntu version, the above dependencies still need to be fulfilled. You can use the Ubuntu Package Search to find the versions available on your system. If a required version is not available, private package repositories may help to install a more recent version of a dependency.

If you encounter difficulties with the installation procedure for any of these dependencies, please file an issue in the GitLab project.

Python

Utopia's frontend, utopya, uses some additional python packages. These packages and their dependencies are automatically installed into a virtual environment when the cmake .. command is carried out during the configuration step of the installation.

Recommended

The following depencies are recommended to be installed, but are not strictly required by Utopia:

Troubleshooting

• If you have a previous installation and the build fails inexplicably, removing the build directory completely and starting anew from the configuration step should help.
  In cases where the installation used to work but at some point stopped working, this should be a general remedy. If the problem does not seem to be related to the Python environment, deleting only build/CMakeCache.txt may already suffice and save some configuration time.

• In cases where you encounter errors with the model registry, it helps to remove the registry entries of all models and regenerate them:

  utopia models rm --all
  cd build
  cmake ..

• Anaconda ships its own version of the HDF5 library which is not compatible with Utopia. To tell CMake where to find the correct version of the library, add the following argument (without the comments!) to the cmake .. command during configuration:

  -DHDF5_ROOT=/usr/           # on Ubuntu
  -DHDF5_ROOT=/usr/local/     # on macOS (Homebrew)

  For Ubuntu, the full command would then be:

  cd build
  cmake -DHDF5_ROOT=/usr/ ..