oooooooooooooooooooooooooo
          O                 __       O
          O   \   /  /\    /   |     O
          O    \ /  /  \  /___ |     O
          O     |  /----\ \    |     O
          O     | /      \ \__ |___  O
          O                          O
           oooooooooooooooooooooooooo 

What is it?

This is a library for performing efficient basic operations, in particular kmeans and exhaustive nearest neighbor search function. It offers three interfaces: C, python and matlab

Only a few function are exported in matlab, as most of the function provided in the library are native matlab functions.

Documentation

Online: http://yael.gforge.inria.fr/

Legal & Contact

This library has been written and is maintained by Herve Jegou and Matthijs Douze. Contact: [email protected] and [email protected] Copyright (C) INRIA 2009-2011 Last change to this documentation: September 2011.

This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. See http://www.cecill.info/licences.en.html

INRIA may provide a licence for industrial applications. Do not hesitate to contact us.

Prerequisites

1- Blas and Lapack.

Any implementation should work as it is wrapped with the Fortran calling conventions.

You might need to adjust the location of these libraries in the makefile.inc file (generated by configure.sh) The LD_LIBRARY_PATH environment variable should be set accordingly

2- Python and its development libraries (python-dev). If only python is installed, not python-dev, you will get an error saying that Python.h can not be found

3- swig (swig)

These two last pre-requisites are required by the python interface and are not strictly mandatory. If you are not interested by this python interface and that you don't have python-dev and swig installed, you should remove the target _yael.so target in the Makefile of the yael core directory.

The library is provided with a pre-compiled documentation (in doc subdirectory). The reference documentation is generated by doxygen while the reference manual is generated using sphinx.

Installation

1- ./configure.sh in the yael root directory. For most configurations, nothing has to be done. However you might need to adjust the variables defined in makefile.inc to fit your local config.

Moreover, we recommand to use --msse4 flags if your processor supports it, in particular for better performance with Hamming distance computation: ./configure.sh --msse4

2- make

3- Try to compile the programs in progs and test

At this point, only the core C and python library are compiled. If you need the matlab interface, you have to compile the mex files:

4- cd matlab ; matlab -r Make -nodisplay -nojvm

The mex executable should be in the PATH.

It is also possible to compile an version that works with Octave (see the Makefile and octave_mex.sh)

Remarks

The performance of the kmeans and of the most computationally intensive functions heavily depend on the version of Lapack and Blas. Timings might be scaled in the range from 1 to 3, depending on your choice.

In order to obtain the best performance, you should consider testing the different version installed on your machine (e.g., in /usr/lib and /usr/lib/atlas).

Troubleshooting

LLVM compiler on the Mac

In recent version of MacOS X (Maverick update), there is a compilation issue trouble with the LLVM compiler when multi-threading is activated. These occur both in all interface. While this problem is not related directly to Yael, the problem may be solved by using gcc-4.9, for instance as follows:

1. download the compiler, $ brew install homebrew/versions/gcc49

For this to work, you need brew package installer. It can be obtained with following command line:

$ ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go/install)"

Ensure that gcc-4.9 is in the path

2. Modify makefile.inc: replace CC=gcc by CC=gcc-4.9

Alternately, you might consider deactivating multi-threading if you do not it. This can be done in makefile.inc by replacing USEOPENMP=yes by USEOPENMP=no

This remark does not apply to Matlab, which is only compatible with LLVM, at least for the common user.

32- and 64- bit integers in Fortran code

BLAS and Lapack are called with the fortran interface, which is most standard. Some implementations are compiled with 32-bit integers (eg. Atlas on most Linux distributions) and some with 64-bit integers (eg. Matlab's -lmwblas and -lmwlapack libraries). The compilation flags -DFINTEGER=long or -DFINTEGER=int switches between the two calling conventions.

For example, if you have a license for Intel's MKL, you can use:

LAPACKLDFLAGS=-L MY_MKL_INSTALL_PATH -Wl,-rpath,MY_MKL_INSTALL_PATH
-lmkl_intel_ilp64 -lmkl_sequential -lmkl_lapack -lmkl_core -lpthread -lm LAPACKCFLAGS=-DFINTEGER=long

This selects the non-threaded version with 64-bit integers. Multithreading will usually be performed by Yael.

For the multithreaded version (tested with MKL from composer_xe_2013):

LAPACKLDFLAGS=
-L /home/clear/lear/intel/mkl/lib/intel64 -Wl,-rpath,/home/clear/lear/intel/mkl/lib/intel64
-L /home/clear/lear/intel/lib/intel64 -Wl,-rpath,/home/clear/lear/intel/lib/intel64
-lmkl_intel_ilp64 -lmkl_gnu_thread -lmkl_core -liomp5 -lpthread

NB that the MKL flags are a bit like black magic. A useful starting point

https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

The computation of subsets of eigenvectors or singular values depends on the ARPACK package, enabled with ./configure.sh --enable-arpack (from http://www.caam.rice.edu/software/ARPACK/). Arpack should be compiled with the same integer size as the blas/lapack implementation. To compile ARPACK with gfortran and 64-bit integers, set

%-----------------------------------------%

| Various compilation programs and flags. |

| You need to make sure these are correct |

| for your system. |

%-----------------------------------------%

FC = gfortran FFLAGS = -g -O3 -fdefault-integer-8 -fPIC

in ARPACK's ARmake.inc