Home page > OCaml > Dynlink as dlopen..
Dynlink as dlopen..
Saturday 14 May 2011, by
Dear Lazyweb,
One big issue we constantly face with liquidsoap is that most of our users have to recompile the whole software just to have support for mp3 encoding using liblame and ocaml-lame.
We have been devising two different approaches with Samuel to address this issue. I would like to expose mine here, both for documentation purposes — finding documentation and examples of how to use the dynlink module is not easy — and to maybe get some feedback..
The whole issue with dynlink is that it is capable of loading a module at run-time but cannot expose its API to the running program. Therefore, a shared module has to be defined so that it registers some functionalities when loaded.
In our case, we would like to be able to activate mp3 encoding by loading the Lame module at run-time if detected. Our requirements are the following:
- We should be able to compile liquidsoap on a system which does not have ocaml-lame installed
- We would like to expose the whole ocaml-lame API once loaded dynamically
The first requirement is important because we would like to distribute a canonical build of liquidsoap through debian for instance, so that if a user wants to dynamically load Lame, she only has to compile ocaml-lame. On the one hand, this reduces our workload by only having to work on one package and on the other hand, this the only case which makes sense.
Indeed, the other solution would be to compile a liquidsoap plugin against a compiled Lame module. Practically, this means that we would have to compile a liquidsoap package on a system which has lame installed. But in this case, we should simply build the sotware with lame included and forget about the shared module..
The second requirement is a natural consequence of the first one: if the shared module is built independently of liquidsoap, then it has to be provided by ocaml-lame. Then in this case, it should be more general than just exposing the functionalities needed by liquidsoap..
So, after much poking around, here is the solution I came to. I will explain it with a simple example.
Imagine that you have a module Foo which has the following interface.
(** An abstract type for the encoder. *)
type encoder
(** A function to instanciate an encoder. *)
val create : unit -> encoder
(** A function to encode. *)
val encode : encoder -> float array array -> stringThe idea here is that we want to expose the whole API and make it possible for any software to be compiled without having the module Foo installed and still be able to dynalically load it at run-time. Therefore, we define the Foo_dynlink interface:
type encoder
type encoder_handler =
{ create : unit -> encoder;
encode : encoder -> float array array -> string }
type handler = { mutable encoder_handler : encoder_handler option }
val handler : handlerBut we do not implement the corresponding .ml module. This mli specifies what the program that will load dynamically Foo will have to implement.
Now, we implement a loader for Foo, called Foo_loader:
open Foo_dynlink
let () =
let create () =
Obj.magic (Foo.create ())
in
let encode enc data =
Foo.encode (Obj.magic enc) data
in
handler.encoder_handler <-
Some { create = create;
encode = encode }Here, we need to use Obj.magic as the two abstract types Foo.encoder and Foo_dynlink.encoder cannot be unified: Foo_dynlink will be implemented by the calling program, which will not have Foo available..
Finally, we can now write a program that dynamically loads Foo. It consists of at least two modules, Foo_dynlink, which implements the corresponding interface and another module that uses it. Let’s see foo_dynlink.ml first:
type encoder
type encoder_handler =
{ create : unit -> encoder;
encode : encoder -> float array array -> string }
type handler = { mutable encoder_handler : encoder_handler option }
let handler = { encoder_handler = None }Now main.ml
let foo_dyn = "/path/to/foo.cmxs"
let foo_loader = "/path/to/foo_loader.cmxs"
open Foo_dynlink
let () =
(** First load Foo. *)
Dynlink.loadfile foo_dyn;
(** Now execute the loader. *)
Dynlink.loadfile foo_loader;
(** At this point, Foo_dynlink.handler should be populated. *)
match handler.encoder_handler with
| None -> Printf.printf "Dynamic loading failed..\n"
| Some _ -> Printf.printf "Dynamic loading succeeded!!\n"Let’s try it now!
First. make a seperate directory for Foo:
mkdir /tmp/foo
cd /tmp/fooThen compile the module and its loader:
ocamlopt -c foo.mli
ocamlopt -shared foo.ml -o foo.cmxs
ocamlopt -c foo_dynlink.mli
ocamlopt -shared foo_loader.ml -o foo_loader.cmxs
# We do not need the compile interface for foo_dynlink at this point..
rm foo_dynlink.cmiNow, let’s prepare another directory for the main program:
mkdir /tmp/bar
cd /tmp/bar
cp /tmp/foo/foo_dynlink.mli .And compile it:
ocamlopt -c foo_dynlink.mli
ocamlopt -c foo_dynlink.ml
ocamlopt -c main.ml
ocamlfind ocamlopt -linkpkg -package dynlink foo_dynlink.cmx main.cmx -o barOf course, the paths to foo.cmxs and foo_loader.cmxs in main.ml have been changed to their actual location..
At this point, bar has been compiled without any reference to Foo. The only requirement is that it implements a module that has the interface required by foo_loader.cmxs. Now is the time to try it!
./bar
Dynamic loading succeeded!!:-)
What do you think? Ugly or interesting?
RSS 2.0
4 Forum messages