Re: Implementing LIBTOOL on OpenVMS

[John E. Malmberg]

Ralf Wildenhues wrote:
> Hi John,
> Sorry for the delay.

No problem, I am fighting with pkg-config right now.  I have to fix the 
CC under bash emulation to ignore some ANSI issues that GCC 3.x 
apparently does by default, and to include a header file that makes the 
equivalent of some of the GCC builtins visible.  But only do that when 
the source module is a .c or a .cxx module, because it breaks things 
when other types of files are run through the pre-processor only.

> I had to read a bit about OpenVMS first.

I assume that you found the documentation link at 
http://www.hp.com/go/openvms .

There are a few public access OpenVMS systems on the wild wild web,
http://encompasserve.org, http://deathrow.vistech.net/, and of course
http://www.testdrive.hp.com/.  And of course
http://www.openvmshobbyist.org/ for non-commercial home use.


> * John E. Malmberg wrote on Tue, Nov 29, 2005 at 12:25:56AM CET:
>
> > Ralf Wildenhues wrote:
> >
> > > * John E. Malmberg wrote on Fri, Nov 18, 2005 at 09:26:14PM CET:

> [ changing public interface ]

> (which I think is what you denoted in your previous post as a change
> in a newer dot-n version) is an _incompatible_ change.  For the
> libtool versioning model, that means you have to increase CURRENT,
> set REVISION and AGE to 0, and logically it means that programs
> linked against one version of the library won't work with the other
> version.

> I'm still not quite sure how libtool should deal with this
> information on OpenVMS.  But generally, whether the changes are
> compatible or not *should* be specified by the library author with
> the version triple, as explained in info Libtool Versioning and
> following chapters.  It's not exactly trivial, but it's quite 
> important that it be followed closely by both library authors and 
> libtool implementations on specific systems.  Otherwise software 
> packaging and use will be very frustrating.

See GSMATCH explanation below.  It looks like a similar method.

> > The current hack around this it to set the flags so that a relink
> > is always required for a new version.  This removes one of the
> > benefits of using a shared image.  Especially if it is something
> > that only one copy is expected to be run at a time.
>
> I don't quite understand this paragraph.

See GSMATCH explanation below.  The load checks are set to be exact 
instead of equal or better.

What is happening is that some programmers came up with a hack to scan 
an object library or set of object modules for the public symbols and 
then build a shared image exporting all of them.  Of course nothing 
could be predicted about what order they would show up in.

This did allow building a shared image automatically from an OpenSource 
project, but everything must either have a private version of the image 
instead of it being shared, or all applications must be relinked if the 
common shared image is rebuilt.

It sort of defeats the purposes of having a shared library.

> Does OpenVMS only allow to load one shared library of a certain name
> at a time? In which context: per-process, per-user, or globally 
> system-wide?

System wide there can be multiple libraries sharing a name, however an
executing program will generally only see one of those libraries.

And a shared image is only really shared if it is "Installed".
Otherwise a private copy is loaded.

A logical name can be used to cause a private copy to be loaded instead
of the existing system wide installed one or to cause an alternate 
system wide library to be used.

> > With OpenVMS, no relink is needed, just change what the logical
> > name is pointing to.  It is like if you could have a symbolic link
> > mean something different to a process tree than it does to the rest
> > of the system.
>
> Hmm.  I don't quite understand logical names yet, but I am reading
> and learning about OpenVMS.

Think of having multiple file systems overlayed over "/" or ".".  And
that one is looked at first for a file and then the others in sequence
until it is found or you run out of file systems.

And then think of several of those file systems only containing symbolic
links, which are always checked before the file systems that are on disks.

And that every process can have their own set of file systems overlayed
so if they did not like how the directory tree looked from "/", they
could make up their own.

So I can have my /usr/lib/foo.so be used before the system wide
/usr/lib/foo.so, unless the image requesting the load is setuid root, in
which case it ignores my private settings and gets the correct one.

Now with OpenVMS, the convention is not to request a load of
/usr/lib/foo/bar.so but to just request a load of "foo_bar".  The system
looks first for a logical name of "foo_bar" and if found, treats it like
a symbolic link to find the real file.  Then it looks for
/sys$share/foo_bar.exe to find it.

> > This is close to the issue that brought me to this conference, the
> >  generated libtool script of one of the packages I am trying to
> > build is looking for the destination library path to be set up, and
> > it does not have any meaning on OpenVMS, so the caller of libtool
> > did not set it up.
> >
> > The build script on OpenVMS sets a logical name that points to the
> >  uninstalled libraries, and the install script sets it to point to
> > the installed libraries.  The linked image does not change.
>
>
> OK.  Would it be reasonable if libtool took over those steps during 
> libtool --mode=link ... and libtool --mode=install respectively?

Possibly.  I would have to do some experimentation.

> > > > Incremental linking does not exist on OpenVMS.  I can fake it,
> > > > but all that does is use more disk space.
> > >
> > > Well, it's needed mostly to cope with too large command line
> > > lengths. Does OpenVMS have a high limit there?
> >
> > Current versions support 4096 characters.  Older only 1024.
>
>
> Ouch.  I know libraries that contain 1000 objects. Does the linker
> have something similar as GNU ld's --whole-archive flag?

Actually the equivalent flag is set by the librarian, if not set, the
linker grabs the entire file.  As a shared library needs public symbols,
which in OpenVMS terms are universal symbols, resolving those usually
pulls in all the library modules for a shared image.

However, the AR emulation on OpenVMS does not always create a library, 
sometimes it just concatenates the object files.

> > > Another question for you: I remember a VMS version for Autoconf
> > > being announced on its mailing list some time ago.  Do you use
> > > this in order to get working configure scripts on OpenVMS?  Is
> > > GNU Autoconf portable to OpenVMS these days?  I also wonder how
> > > much of the shell scripting of Libtool will just break on VMS..
> >
> > I tried to run a very recent version of Autoconf.  It failed
> > because it seems to require enhancements to Perl on OpenVMS.
> > Currently Perl on OpenVMS assumes a DCL shell environment.  I am
> > working on that, but it will take some time.

> I was talking about 
> http://lists.gnu.org/archive/html/autoconf/2005-03/msg00067.html 
> which is not GNU Autoconf, but a special version for VMS.  You might 
> have more luck with it.

It looks promising.  Currently I run the autoconf step on a LINUX system
for the one project that I need it for.

I (or someone) stills need to fix Perl on OpenVMS where it does not work 
for Autoconf though.

> > > Feel free to ask questions about anything you don't understand,
> > > show whatever changes you're working with -- we may be able to
> > > give useful hints.
> >
> > This is a fragment of the option file that I manually generated for
> > zlib 1.2.3.  An image linked against it will continue to work with
> > newer versions as long as the second number on the GSMATCH is equal
> > to or greater than the value present, and the first number is the
> > same.
> >
> > I encoded the minor version numbers in that section, but it might
> > be more appropriate to use the commit numbers from a central change
> > control library.  Using the major version number as the first
> > number may not be a good idea, and I may change that again before I
> > submit that change to the ZLIB maintainers.
>
>
> Hmm.  What does the GSMATCH number convey exactly?  What are its 
> semantics for VMS?

A GSmajor and GSminor "version" type number.  The GSmajor number changes 
with incompatible changes to the public interface.  The GSminor number 
generally changes with upward compatible changes to the public interface.

A shared image can be set so that both numbers must match, or just that 
the major number matches and the minor number of the shared image is 
equal or higher than the one that the calling image was linked against.

Encoding is up to the programmer.  When an image is activated through 
running a program, these checks are done, and if they do not pass, the 
image does not get activated.  With a dload(), these checks are not done 
on that image, but the loaded image may trigger the attempt to load 
other shared libraries.

These numbers may not want to track a triple of x.y.z, because bar-2.0.0 
may be upward compatible with bar-1.9.9, so the GSmajor would stay at 1, 
and if new public interfaces were added, the GSminor would get bumped by 
at least 1.

--------

For my first passes at things, I plan to just do what I can to get 
libtool to generate static libraries like it is attempting to do, and 
then I will then add my own link procedure to generate a shared library 
if that is needed.

Thanks,
-John
address@hidden
Personal Opinion Only