In my CVX <http://cvxr.com/cvx> package I distribute precompiled MEX
files for 32- and 64- bit versions of Mac, Windows, and Linux MATLAB. In
the process of determining how to do the same for Octave, I found a
couple of different issues that complicate matters.
1.
The 'mkoctfile' routine links the compiled code to every dependency
that the current installation of Octave has, even if those libraries
are not needed by the code itself. What happens if a particular
user's Octave build does not link to the same set of dependencies? I
don't have a complete understanding about how dynamic library
dependency resolution is performed---whether it occurs on load, or
is lazily done as calls are made to those dependencies. If it's the
former, then MEX/OCT files must be linked to the least common
denominator of dependencies if they have any chance of being binary
portable.
It should be noted that MATLAB takes the minimal approach to
linking; even LAPACK and BLAS must be explicitly requested (although
you can link to the MATLAB image of BLAS/LAPACK). It's also
important to point out that not linking to the other dependencies
does *not* prevent them from being used---they just have to be
called through the interfaces provided by lib octave or liboctinterp.
2.
On the Mac, dynamic library path resolution is a bit tricky. For a
package like Octave, the best way to go is to use @rpath constructs
in the dynamic libraries (liboctave/libinterp), and then give the
executable an rpath (likely using an @executable_path construct)
that points all dynamically loaded objects to the library install
directory. Unfortunately, GNU libtool provides no way to insert
these constructs into the library and/or executable. (I've submitted
a patch, but I'm not optimistic.) I don't think Linux really has an
issue here; I don't yet know about Windows.
The Mac Ports folks use the @rpath approach when building their app
bundles; you can see the hoops they jump through here
<http://wiki.octave.org/Create_a_MacOS_X_App_Bundle_Using_MacPorts>.
JWE pointed me to this link. Of course, the Mac app builders have a
bigger challenge, because they need to include every non-system
Octave dependency in the package. So they have to employ these
techniques not just for liboctave/liboctinterp, but every non-system
library.
Can these issues be at least partially solved in the build itself? I
have a couple of proposals that at least partially address the problem:
* For #1, I contributed Path 8308
<http://savannah.gnu.org/patch/?8308> to the patch tracker. It
implements a command-line flag for mkoctfile called
--minimal-dependencies. As its name implies, it links with one of
the Octave dependencies except liboctave, liboctinterp, and system
libraries. If you wish to link to other libraries, you have to
explicitly list them. I've been using this patched version of
mkoctfile to compile tens of MEX files that I use regularly.
* For #2, I manually inserted some commands into the Makefiles to
implement the RPATH approach for the Mac. For instance, in {lib
octave,libinterp}/Makefile, I changed the final link steps to this:
|liboctave.la: $(liboctave_la_OBJECTS) $(liboctave_la_DEPENDENCIES)
$(EXTRA_liboctave_la_DEPENDENCIES)
$(AM_V_CXXLD)$(liboctave_la_LINK) -rpath $(octlibdir)
$(liboctave_la_OBJECTS) $(liboctave_la_LIBADD) $(LIBS)
install_name_tool -id @rpath/liboctave.2.dylib
.libs/liboctave.2.dylib
liboctinterp.la: $(liboctinterp_la_OBJECTS) $(liboctinterp_la_DEPENDENCIES)
$(EXTRA_liboctinterp_la_DEPENDENCIES)
$(AM_V_CXXLD)$(liboctinterp_la_LINK) -rpath $(octlibdir)
$(liboctinterp_la_OBJECTS) $(liboctinterp_la_LIBADD) $(LIBS)
install_name_tool -id @rpath/liboctinterp.2.dylib
.libs/liboctinterp.2.dylib|
I do something similar for libinterp/Makefile. For the executables,
I added this in src/Makefile:
|octave-cli$(EXEEXT): $(octave_cli_OBJECTS) $(octave_cli_DEPENDENCIES)
$(EXTRA_octave_cli_DEPENDENCIES)
@rm -f octave-cli$(EXEEXT)
$(AM_V_CXXLD)$(octave_cli_LINK) $(octave_cli_OBJECTS)
$(octave_cli_LDADD) $(LIBS)
install_name_tool -add_rpath @executable_path/../lib/octave/3.9.0+
.libs/octave-cli
octave-gui$(EXEEXT): $(octave_gui_OBJECTS) $(octave_gui_DEPENDENCIES)
$(EXTRA_octave_gui_DEPENDENCIES)
@rm -f octave-gui$(EXEEXT)
$(AM_V_CXXLD)$(octave_gui_LINK) $(octave_gui_OBJECTS)
$(octave_gui_LDADD) $(LIBS)
install_name_tool -add_rpath
@executable_path/../../../../../lib/octave/3.9.0+ .libs/octave-gui
|
Together these accomplish exactly what I'm looking for at the moment. If
I look at the dependencies for one of my Mex files, I see this:
|>> system('otool -L cvx_bcompress_mex.mex')
cvx_bcompress_mex.mex:
@rpath/liboctinterp.2.dylib (compatibility version 3.0.0, current
version 3.0.0)
@rpath/liboctave.2.dylib (compatibility version 3.0.0, current version
3.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
120.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1197.1.1)
|
This is perfect: there are no extra libraries linked, and the path to
the Octave libraries can be picked up at load time, thanks to the @rpath
constructs. This message has gone long enough. What are your
thoughts---especially those who work with the Mac, but anyone who might
be affected by these kinds of improvements.
* Is it reasonable to insert these post-build commands into the
auto-generated Makefiles (obviously, using the macros wisely to
ensure that the library paths are created properly)? Obviously we
can do it just for the Mac.
* And is my proposed mkoctfile improvement something that is worth
considering?
* Any improvements? One remaining concern I have is potential
conflicts between the BLAS/LAPACK Octave links to and the one the
MEX/OCT files link to. This could be addressed by employing a
similar @rpath approach *specifically* for BLAS/LAPACK.
Thanks for your time, everyone!