mfp / ld.ocaml

Dynamic linker/loader for OCaml.

ld.ocaml does for OCaml native code plugins (.cmxs files) what ld.so does for
shared libraries.  Given a list of cmxs objects (typically only one,
corresponding to a program to execute), ld_ocaml scans the filesystem to find
the available (.cmxs) libraries, infers which are needed in order to load the
objects, and then proceeds to load them all in order.

Requirements
------------
* OCaml >= 3.11 with native Dynlink (.cmxs) support
* GNU BFD (available as binutils-dev in Debian)
* OMake for building

Building
--------

  $ omake

Installing
----------
Copy the  ld_ocaml  executable to some directory in your PATH.
Don't rename the file (adding an extension is OK): ld_ocaml examines ARGV[0]
at runtime in order to decide how to rewrite ARGV and which cmxs files to load
(see below).

Usage
-----

  ld_ocaml foo.cmxs -other --options --are --ignored

looks for .cmxs libraries and loads those required by foo.cmxs. Sys.argv is
modified so that foo.cmxs becomes Sys.argv.(0).
You can also specify multiple .cmxs files to be loaded in order. The last one
will become Sys.argv.(0).

Alternatively, if only one cmxs object (plus its dependencies) is to be
loaded, you can create a symlink to ld_ocaml in the same directory as the
.cmxs, with the same name as the .cmxs *without the extension* (it's OK to add
a different extension). For example, if you want to run hello.cmxs directly,
you can do

  $ cd /dir/where/hello.cmxs/is
  $ ln -s /usr/local/bin/ld_ocaml hello    # hello.exe or any other extension OK
...
  $ hello -other -options                  # assuming it's in the path
  $ /dir/where/hello.cmxs/is/hello         # otherwise

ld_ocaml honors these environment variables:

 LD_OCAML_VERBOSE

    set to 0 for no output (default), 1 for information about the libs that
    will be loaded, 2 for debug info

 LD_OCAML_CACHE

    name of the cache file (default: $HOME/.ld.ocaml.cache)

 LD_OCAML_LIBRARY_PATH

    list of extra paths (separated by :) where .cmxs libraries can be found,
    in addition to the standard directories. The information about these libs
    will not be saved in the cache.

 LD_OCAML_SYS_LIBRARY_PATH

    list of paths (separated by :) where system .cmxs libraries can be
    found. The information about these libs will be saved in the cache.
    (defaults to (/usr/lib/ocaml, /usr/local/lib/ocaml, and their subdirs))
    (Note: you should use absolute paths.)

 LD_OCAML_EXTRA_SYS_LIBRARY_PATH

    list of extra paths (separated by :) where system .cmxs libraries can be
    found. The information about these libs will be saved in the cache.
    (Note: you should use absolute paths.)


ld_ocaml caches the system DLL catalog in the file specified in LD_OCAML_CACHE
($HOME/.ld.ocaml.cache by default).  Delete that file to force a system DLL
catalog rebuild when new libraries are installed.

Creating the .cmxs libraries
----------------------------
Given a .cmxa file with PIC code, you can create the cmxs easily:

  ocamlopt -shared -o foo.cmxs foo.cmxa -linkall

For instance, if you want to run a program that depends on AAA batteries,
itself dependent on camomile.cmxs and nums.cmxs, you can build the required
.cmxs libs as follows (other libs, including Thread, Dynlink, Unix and Str,
are already included in ld_ocaml's runtime):

  ocamlfind ocamlopt -package camomile -shared -o camomile.cmxs \
      camomile.cmxa -linkall
  ocamlfind ocamlopt -package num -shared -o nums.cmxs nums.cmxa -linkall
  ocamlfind ocamlopt -package aaa -shared -o aaa.cmxs aaa.cmxa -linkall

Debian is already starting to distribute .cmxs files in OCaml library
packages. Presumably, all packages in Debian and other distributions will
include .cmxs files in the future, making manual cmxs generation unnecessary.

If you use OMake, you can use these rules to generate the .cmxs files for all
the OCAMLPACKS at once under libs/:

  # the target used to build all the .cmxs
  .PHONY: libs

  # get all the packages required recursively
  ALL_PACKS = \
      $(shell ocamlfind query -predicates "mt,native,mt_posix" -r \
          -p-format $(OCAMLPACKS) | sort -u)

  # for each foo.cmxs, define a    libs: libs/foo.cmxs  dependency
  # and a  libs/foo.cmxs  target
  foreach(pack, $(ALL_PACKS))
      ARCH = $(shell ocamlfind query -predicates "mt,native,mt_posix" \
                 -a-format $(pack))
      DST = libs/$(pack).cmxs
      libs: $(DST)
      $(DST):
          $(OCAMLFIND) ocamlopt -shared $(OCAMLFLAGS) -o $@ \
            -package $(pack) $(ARCH) -linkall

Program compilation
-------------------
Let's suppose you want to compile a program that links against AAA Bateries.
You'd normally do

   ocamlfind ocamlopt -package aaa -o hello hello.ml -linkpkg

which would link statically against AAA's libs and all its dependencies.

In order to create a cmxs that can be run with

  ld_ocaml hello.cmxs

you simply need to do

  ocamlfind ocamlopt -package aaa -shared -o hello.cmxs hello.ml

The difference is that we don't link against the AAA package anymore, and the
dynamic libs will be loaded as required by ld_ocaml (you'll have to make sure
the required .cmxs libs are available, though --- see the above section).

How it works
------------
.cmxs files hold information about the interfaces they import ("imports_cmi")
and the implementations they are tied to because of inlining ("imports_cmx").
This information is an OCaml string associated to the "caml_plugin_header"
symbol which corresponds to a Marshal serialization of the header (see
ld_header.ml). ld_ocaml looks for .cmxs files and builds a DLL catalog
which is later used to resolve dependencies as needed.