[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Implementing LIBTOOL on OpenVMS

From: John E. Malmberg
Subject: Re: Implementing LIBTOOL on OpenVMS
Date: Mon, 28 Nov 2005 18:25:56 -0500
User-agent: Mozilla/5.0 (X11; U; OpenVMS AlphaServer_DS10_466_MHz; en-US; rv:1.7) Gecko/20040621

Ralf Wildenhues wrote:
* John E. Malmberg wrote on Fri, Nov 18, 2005 at 09:26:14PM CET:

Because of differences between *NIX and OpenVMS in this area, it looks like it may not be possible to make generating shared images on OpenVMS totally transparent to a build procedure with out some cooperation with the maintainers of the package.

I'm not sure yet.  For a large subset of packages, it may be possible.

Item 3 below from my first post is the sticky point for shared images.

3. For dot-n releases, in order to provide upward compatibility so that new shared libraries can be used by older programs, the list of public symbols must always be in the same order, with new symbols being added to the end.

Ouch.  This is a new requirement.  So in general the author of the
library has to provide the list of symbols, it can't be generated.

Mostly right. A generated approach can be taken as long as the generating program has the previous generation of the file as a base and the generated output is distributed with the source, and understands the rules for generating the output.

It does cause problems if the project spins off a new branch that needs backwards compatibility, as the list of symbols must be maintained by the head branch.

For full compatibility, Microsoft Windows DLL files may have the same requirement. Microsoft Windows DLL routines can be invoked either by ordinal or by name. Change the order of the symbols to be exported, and it changes the ordinal value, thus breaking any routine that calls by ordinal.

And on a Microsoft based system, it appears that call by ordinal would be faster than call by symbol name.

I do not know if the Microsoft Linker sets up images to load their DLLs by ordinal or by name.

Effectively the OpenVMS linker sets up calling routines by ordinal, not by name.

This probably means that we should have a new version_type for OpenVMS.

Special handing is needed if routine foo(x,y,z) changes to be foo(p,d,q) in a newer dot-n version. The old version of foo would get a new name, but retain it's position in the symbol table, and it needs to still return something, if only an ENOSYS error code if it was really bad to continue to use it. The new version of foo() would go to the end of the table.

So for OpenVMS, some history needs to be maintained for the full advantages of shared images to be used.

Hmm.  This feature does not map on features commonly found in other
shared library systems (altough of course there are systems with
versioned symbols).  So, probably the first support for OpenVMS in
Libtool should disallow and need not support symbols with changed
interface, since for portable programs, you cannot use them either.
We should then strive to add support for this together with adding
support for versioned symbols on other systems.

It is pretty hard to "disallow" this feature because with out a history, it is impossible for the build procedure to automatically know if it is building a shared image that is incompatible with the previous version.

The current hack around this it to set the flags so that a relink is always required for a new version. This removes one of the benefits of using a shared image. Especially if it is something that only one copy is expected to be run at a time.

Supplying a old/new version of a public routine is something that a programmer needs to do. I do not think that most change control systems will flag if someone makes an incompatible change to a program's parameters.

Also, public symbols on OpenVMS are traditionally case insensitive and stored in uppercase and limited to 31 characters. They can also be case sensitive. I have the ability to export symbols in both exact case and uppercase.

The 31 character limit is a special case that I am trying to get some tools to make easier. What happens is that if a public symbol is more than 31 characters, the C compiler shortens to be much smaller and adds a CRC type value to the end to keep it unique. That is the symbol name that I have to put in the symbol table file.

Of course the current C standard states that only the first 31 characters of a public symbol are significant, but that the restriction is planned to be removed in a future version of the standard.

Other issues:

Normally the eventual location of the shared library will reside is not used at build time. A construct known as a logical name is used to find it if the shared image is not in the /SYS$SHARE directory. For shared libraries not supplied by the operating system, normally a logical name is used.

A logical name works like a symbolic link, except that to the file system it looks like it is always present in the in either the current working directory or in the root directory when it is explicitly referenced. A logical name can supersede access to a file of the same name.

When an application is built against a test shared library instead of a production one, it makes a local logical name for this.

There is a naming convention for the logical names and program names where prefixes can be reserved to prevent conflicts so that multiple libraries with similar names can coexist. These prefixes are not directory paths.

OK.  I think this might need a bit of code, but is doable.
Note Libtool (in default mode) wants the user to be able to use the
uninstalled libraries, and has means to relink libraries and/or programs
upon either execution or installation.

With OpenVMS, no relink is needed, just change what the logical name is pointing to. It is like if you could have a symbolic link mean something different to a process tree than it does to the rest of the system.

This is close to the issue that brought me to this conference, the generated libtool script of one of the packages I am trying to build is looking for the destination library path to be set up, and it does not have any meaning on OpenVMS, so the caller of libtool did not set it up.

The build script on OpenVMS sets a logical name that points to the uninstalled libraries, and the install script sets it to point to the installed libraries. The linked image does not change.

A linker option file contains the shared libraries that will be referenced by the image being built.

$ type svn_client.opt;3

During the build, LIBSVN_SHARE points to image that was just built. At run-time, a system wide definition is installed to point to the production version of the library.

Essentially sometime before the LIBSVN_SHARE is used to build something, the build script must do something like "DCL DEFINE/JOB LIBSVN_SHARE LCL_ROOT:[LIB]LIBSVN_SHARE.EXE". This definition must be cleaned up after the build is complete because it will persist for the life of the job tree containing the build script. And I also need the name to be in VMS syntax, which is hard to do from a bash shell script at this time.

In a traditional OpenVMS build environment, all the building takes place in a single process, so a temporary logical name is used that automatically self destructs when the script ends.

Also in this case the prefix "APACHE$" is reserved to to files that are part of the Apache web server. The "$" in the name indicates the name has been registered.

By changing APACHE$APR_SHR definition, I can change either the test or production version of svn_client to use a compatible version of that library.

Well, Libtool has postinstall_cmds and finish_cmds and all that, and you
can stick pretty much arbitrary code in there.

Of course that would require a programmer familiar with VMS to put in
such support for each project.

Incremental linking does not exist on OpenVMS. I can fake it, but all that does is use more disk space.
Well, it's needed mostly to cope with too large command line lengths.
Does OpenVMS have a high limit there?

Current versions support 4096 characters.  Older only 1024.

What has been done in the past for large number of files is either to populate an object library, or to put the filenames in a linker option file. Simply a list of files, usually one per line for readability. I would have to check if commas are required.

Another question for you: I remember a VMS version for Autoconf being
announced on its mailing list some time ago.  Do you use this in order
to get working configure scripts on OpenVMS? Is GNU Autoconf portable to OpenVMS these days? I also wonder how much of the shell scripting
of Libtool will just break on VMS..

I tried to run a very recent version of Autoconf. It failed because it seems to require enhancements to Perl on OpenVMS. Currently Perl on OpenVMS assumes a DCL shell environment. I am working on that, but it will take some time.

Currently for the project I need to run Autoconf on, I have an NFS client on a LINUX system that runs it for me after refreshing the changes in that directory from the previous day.

A couple of notes, before you start hacking: we generally like patches
against the CVS HEAD version of Libtool.  Unintrusive patches may be
backported to branch-1-5.  Especially for libltdl, working with CVS HEAD
should be easier.

A CVS client has been ported to OpenVMS.  I will see if I can find it.
Otherwise is there ftp access to the unpacked head files?

For large contributions, the FSF requires either a copyright assignment
or disclaimer; the assignment paperwork takes a while, so it may be
helpful to start early with this (and to check whether you may have any
issues with this).  Other than that, patch discussions are best handled
on the libtool-patches mailing list.

For large contributions to open source projects, I have to go through a central corporate office.

Feel free to ask questions about anything you don't understand, show
whatever changes you're working with -- we may be able to give useful

This is a fragment of the option file that I manually generated for zlib 1.2.3. An image linked against it will continue to work with newer versions as long as the second number on the GSMATCH is equal to or greater than the value present, and the first number is the same.

I encoded the minor version numbers in that section, but it might be more appropriate to use the commit numbers from a central change control library. Using the major version number as the first number may not be a good idea, and I may change that again before I submit that change to the ZLIB maintainers.

The symbols are exported both in exact case and in upper case so that the users of the library do not need to change their build procedures to require exact case, but the library can be called by modules that need to be built using exact case. The uppercase version of the symbol is the alias assigned by the linker.

[start of fragment]



I am not all that skilled with bash shell scripting. I have been on vacation for the past week, so I have to get back to the specific build issues that I was seeing to be more specific. I also am having to create/fix the VMS versions of some UNIX utilities.


Personal Opinion Only

reply via email to

[Prev in Thread] Current Thread [Next in Thread]