ga-benchmark

This repository is a joint effort to define standards and methodologies for benchmarking Geometric Algebra libraries, library generators, and code optimizers.

The goal of this project is to help physicists, chemists, engineers, and computer scientists to choose the Geometric Algebra solution that best suits their practical needs, as well as to push further the improvement of the compared solutions and to motivate the development of new tools.

The current version compares seven C++ solutions:

• Gaalet (https://sourceforge.net/projects/gaalet)
• Gaalop (http://www.gaalop.de)
• Garamon (https://github.com/vincentnozick/garamon)
• GATL (https://github.com/laffernandes/gatl)
• GluCat (https://github.com/penguian/glucat)
• TbGAL (https://github.com/Prograf-UFF/tbgal)
• Versor (http://versor.mat.ucsb.edu)

But it is easy to include new ones!

The solutions are compared considering four models of geometry:

• Conformal (2D and 3D)
• Euclidean (2D, 3D, 4D, and 5D)
• Homogeneous (2D, 3D, and 4D)
• Minkowski (2D and 3D)

ten unary operations:

• Clifford conjugation
• Dualization
• Grade Involution
• Inversion
• Normalization
• Reversion
• Squared reverse norm
• Unary minus
• Unary plus
• Undualization

twelve binary operations:

• Addition
• Commutator product
• Dot product
• Geometric product
• Hestenes inner product
• Inverse geometric product
• Left contraction
• Outer product
• Regressive product
• Right contraction
• Scalar product
• Subtraction

and one algorithm:

• Inverse kinematics assuming the conformal model of 3D Euclidean space

The project documentation is structured as follows:

1. Requirements
2. Building and Running
3. Ploting Results
4. Latest Results
5. Further Knowledge
   • Gaalet
   • Gaalop
   • Garamon
   • GATL
   • GluCat
   • TbGAL
   • Versor
   • How ga-benchmark Recognizes a Solution
6. License

1. Requirements

Make sure that you have all the following tools, libraries, library generators, and code optimizers installed and working before attempting to use ga-benchmark.

Required tools:

• Your favorite C++17 compiler
• CMake
• Python 3 interpreter
• Jupyter Notebook

Required C++ libraries:

• Google Benchmark
• Threads (pthread library on Linux, shlwapi library on Windows, and kstat library on Solaris)

Required Python modules:

• argparse
• ipywidgets
• json
• scipy
• shutil
• tqdm
• typing

Expected Geometric Algebra libraries, library generators, and code optimizers:

• Gaalet: Geometric Algebra ALgorithms Expression Templates
• Gaalop: Geometic Algebra Algorithms Optimizer
• Garamon: Geometric Algebra Recursive and Adaptive Monster
• GATL: Geometric Algebra Template Library
• GluCat: Clifford Algebra Templates
• TbGAL: Tensor-based Geometric Algebra Library
• Versor

The Further Knowledge section describes how to install the expected libraries, library generators, and code optimizers, and how to make ga-benchmark recognize the existence of a solution to be compared, including custom ones not listed above. Notice that Gaalop and Garamon must be instrumented.

2. Building and Running

Use the git clone command to download the project:

git clone https://github.com/ga-developers/ga-benchmark.git ga-benchmark
cd ga-benchmark

The basic steps for configuring and building ga-benchmark look like this:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --parallel 8

Call the test target to run the benchmark:

cmake --build . --target test

Building and running ga-benchmark may take several hours. Please be patient and keep the computer allocated exclusively for the execution of the tests.

3. Ploting Results

After running ga-benchmark, use the Jupyter Notebook ga-benchmark/gabm_results.ipynb to see tables and charts that compare solutions. Those tables and charts are produced by functions defined in the ga-benchmark/gabm.py file.

Alternatively, you can generate the tables and charts from the command line. Assuming that the current folder is ga-benchmark/build and you want to save resulting charts and tables to the ga-benchmark/results folder, call:

cd ..
python -m gabm -i ./build -o ./results

If everything is alright, then results are in the ga-benchmark/results folder. This is the folder structure expected by the repository https://github.com/ga-developers/ga-benchmark-runs, since it also includes a copy of the input JSON files.

4. Latest Results

The latest results are available here: https://github.com/ga-developers/ga-benchmark-runs. Please, help us to keep it updated.

5. Further Knowledge

If your system does not include the expected libraries, library generators, and code optimizers listed above, then you must install them following the instructions provided by the developers.

The commands below summarize the installation process of each of the expected solutions. However, one should note that the process may have changed with the release of new versions.

Here, we assume that ga-benchmark is the current folder and Linux operating system. The installation procedures will place the expected solutions inside the ga-benchmark/libs folder.

Gaalet

svn checkout https://svn.code.sf.net/p/gaalet/code/trunk libs/Gaalet/repository
mkdir -p libs/Gaalet/install/include/gaalet
cp libs/Gaalet/repository/include/cpp0x/* libs/Gaalet/install/include/gaalet

Gaalop

sudo apt install xmaxima
sudo apt install default-jre
mkdir -p libs/Gaalop/download
wget -O libs/Gaalop/download/GaalopPrecompiler.tar.gz "http://www.gaalop.de/wp-content/uploads/x86-64/GaalopPrecompiler-2.0.1447-Linux.tar.gz"
tar xf libs/Gaalop/download/GaalopPrecompiler.tar.gz -C libs/Gaalop/download
mkdir -p libs/Gaalop/install/usr/local/bin/GaalopPrecompiler
mv libs/Gaalop/download/GaalopPrecompiler-2.0.1447-Linux/* libs/Gaalop/install/usr/local/bin/GaalopPrecompiler
rmdir libs/Gaalop/download/GaalopPrecompiler-2.0.1447-Linux

See Gaalop's download page for a list of prebuilt binaries of the precompiler for Linux and Windows.

Garamon

sudo apt install libeigen3-dev
git clone https://github.com/vincentnozick/garamon.git libs/Garamon/repository
mkdir libs/Garamon/repository/build
mkdir libs/Garamon/install
cd libs/Garamon/repository/build
cmake ..
make
for conf in ../../../../source/Garamon/algebras/*.conf
do
  ./garamon_generator $conf
  filename=$(basename -- "$conf")
  cd output/garamon_"${filename%.*}"
  mkdir build
  cd build
  cmake -DCMAKE_BUILD_TYPE=Release ..
  make
  make DESTDIR=../../../../../install install
  cd ../../..
done
cd ../../../..

GATL

git clone https://github.com/laffernandes/gatl.git libs/GATL/repository
mkdir libs/GATL/repository/cpp/build
mkdir libs/GATL/install
cd libs/GATL/repository/cpp/build
cmake -DCMAKE_INSTALL_PREFIX="$(cd ../../../install; pwd -P)" ..
cd ../../../../..

GluCat

sudo apt install libeigen3-dev
sudo apt install libboost-all-dev
git clone https://github.com/penguian/glucat.git libs/GluCat/repository
cd libs/GluCat/repository
make -f admin/Makefile.common cvs
./configure --disable-pyclical
make
make DESTDIR=$(realpath ../install) install
cd ../../..

TbGAL

sudo apt install libeigen3-dev
git clone https://github.com/Prograf-UFF/TbGAL.git libs/TbGAL/repository
mkdir libs/TbGAL/repository/build
mkdir libs/TbGAL/install
cd libs/TbGAL/repository/build
cmake -DCMAKE_INSTALL_PREFIX="$(cd ../../install; pwd -P)" ..
cd ../../../..

Versor

git clone https://github.com/wolftype/versor.git libs/Versor/repository
mkdir libs/Versor/repository/build
mkdir libs/Versor/install
cd libs/Versor/repository/build
cmake -DCMAKE_BUILD_TYPE=Release ..
make
make DESTDIR=../../install install
cd ../../../..

How ga-benchmark Recognizes a Solution

A given libraries, library generators, and code optimizers will be recognized by ga-benchmark if and only if the ga-benchmark/source folder includes a subfolder with the benchmark code of the given solution. The ga-benchmark/source/README.md file presents detailed instructions about how to prepare such subfolder.

6. License

This software is licensed under the GNU General Public License v3.0. See the LICENSE file for details.