Re: Implementing LIBTOOL on OpenVMS

[Ralf Wildenhues]

Hi John,

* John E. Malmberg wrote on Fri, Nov 18, 2005 at 09:26:14PM CET:
> I am encountering libtool scripts in programs that I am building on OpenVMS.
> 
> Currently the scripts are building static libraries based on what they 
> can determine from the configure scripts.  What I would like to do is to 
> get them to build shared libraries for better efficiency.

OK.  This should be possible, I believe.

> Because of differences between *NIX and OpenVMS in this area, it looks 
> like it may not be possible to make generating shared images on OpenVMS 
> totally transparent to a build procedure with out some cooperation with 
> the maintainers of the package.

I'm not sure yet.  For a large subset of packages, it may be possible.

> The documentation suggests a post to this mailing list before I do anything.

Yes, and it's a good idea you do so.

> There are some issues that I need to deal with.
> 
> 1. All symbols must be resolved at the time a shared library is built on 
> OpenVMS, either internal to the resulting shared library or by a shared 
> library it is built against.  If I do not have this information, only a 
> 'static' library can be built.  Symbol preemption or definition at load 
> time is not supported.

OK.  There are other systems with this requirement -- notably some w32
ones (e.g., mingw); so most infrastructure for this is set.  You may
notice some software packages aren't all set for this -- well, many
people never develop with such systems.

> 2. I need a list of public symbols to feed into the linker.  It appears 
> that the method used for .DLL files can be used to identify them, or the 
> .sym file.

OK.  This should be doable easily as well.  libtool allows passing a
manually created symbol file.  If the system has a means to create the
list of public symbols (e.g. with a program like nm, see
global_symbol_pipe and its associated macros in libtool.m4), it may be
possible to automate this in most cases.  For example, AIX without
runtimelinking has always_export_symbols=yes and does this always.

> 3. For dot-n releases, in order to provide upward compatibility so that 
> new shared libraries can be used by older programs, the list of public 
> symbols must always be in the same order, with new symbols being added 
> to the end.

Ouch.  This is a new requirement.  So in general the author of the
library has to provide the list of symbols, it can't be generated.
Right?

This probably means that we should have a new version_type for OpenVMS.

> Note that this does not apply if the shared image is only 
> loaded by dlload().  On OpenVMS, programs are normally linked against 
> shared images, and the dlload() method is only used for things like 
> "plug-ins".

OK.  This is similar to dlopen() on some ELF systems.  libltdl may be
extended to support dlload, I suppose.

> Special handing is needed if routine foo(x,y,z) changes to be foo(p,d,q) 
> in a newer dot-n version.  The old version of foo would get a new name, 
> but retain it's position in the symbol table, and it needs to still 
> return something, if only an ENOSYS error code if it was really bad to 
> continue to use it.  The new version of foo() would go to the end of the 
> table.
> 
> So for OpenVMS, some history needs to be maintained for the full 
> advantages of shared images to be used.

Hmm.  This feature does not map on features commonly found in other
shared library systems (altough of course there are systems with
versioned symbols).  So, probably the first support for OpenVMS in
Libtool should disallow and need not support symbols with changed
interface, since for portable programs, you cannot use them either.
We should then strive to add support for this together with adding
support for versioned symbols on other systems.

> Other issues:
> 
> PIC mode.  OpenVMS compilers generate PIC code unless explicitly tell 
> them not to.

That's fine.

> Normally the eventual location of the shared library will reside is not 
> used at build time.  A construct known as a logical name is used to find 
> it if the shared image is not in the /SYS$SHARE directory.  For shared 
> libraries not supplied by the operating system, normally a logical name 
> is used.
> 
> A logical name works like a symbolic link, except that to the file 
> system it looks like it is always present in the in either the current 
> working directory or in the root directory when it is explicitly 
> referenced.  A logical name can supersede access to a file of the same name.
> 
> When an application is built against a test shared library instead of a 
> production one, it makes a local logical name for this.
> 
> There is a naming convention for the logical names and program names 
> where prefixes can be reserved to prevent conflicts so that multiple 
> libraries with similar names can coexist.  These prefixes are not 
> directory paths.

OK.  I think this might need a bit of code, but is doable.
Note Libtool (in default mode) wants the user to be able to use the
uninstalled libraries, and has means to relink libraries and/or programs
upon either execution or installation.

> On the 64 bit OpenVMS platforms, the detailed debugging symbols can be 
> put in separate files that only need to be installed on the target 
> system if there is a problem.
> 
> The install / deinstall steps for shared images on OpenVMS is probably 
> not practical to be implemented through libtool because of significant 
> differences in the way that task is done on OpenVMS.  The closest 
> equivalent might be moving the images to a staging directory.

Well, Libtool has postinstall_cmds and finish_cmds and all that, and you
can stick pretty much arbitrary code in there.

> Because public data items in a shared image must never move or change 
> size for upward compatibility, it is best not to have any.  Instead the 
> data is best accessed through procedures that return a pointer to their 
> internal location.  Macros can be used to hide this from the calling 
> programs in the header files to the library.

This is good advice in general, not only for OpenVMS, as data exports
pose issues on other systems, too.

> Incremental linking does not exist on OpenVMS.  I can fake it, but all 
> that does is use more disk space.

Well, it's needed mostly to cope with too large command line lengths.
Does OpenVMS have a high limit there?

Another question for you: I remember a VMS version for Autoconf being
announced on its mailing list some time ago.  Do you use this in order
to get working configure scripts on OpenVMS?  Is GNU Autoconf portable 
to OpenVMS these days?  I also wonder how much of the shell scripting
of Libtool will just break on VMS..

A couple of notes, before you start hacking: we generally like patches
against the CVS HEAD version of Libtool.  Unintrusive patches may be
backported to branch-1-5.  Especially for libltdl, working with CVS HEAD
should be easier.

For large contributions, the FSF requires either a copyright assignment
or disclaimer; the assignment paperwork takes a while, so it may be
helpful to start early with this (and to check whether you may have any
issues with this).  Other than that, patch discussions are best handled
on the libtool-patches mailing list.

Feel free to ask questions about anything you don't understand, show
whatever changes you're working with -- we may be able to give useful
hints.

Cheers,
Ralf