Re: ld --auto-import for cygwin and libtool

[Charles Wilson]

Robert Collins wrote:

> I've cc'd this for wide exposure.


As have I.  However, the appropriate place for this discussion is the 
binutils mailing list; please restrict to that list when replying.

I'm trying to restart this thread after ~6 weeks, to lobby for the 
inclusion of Robert's version of Paul Sokolovsky's auto-import patch.

Robert's version also includes Paul's auto-EXport filtering patch which 
in its final form appeared here:
[patch] ld: PE: comprehensive auto-filtering for DLL-exportable symbols
http://sources.redhat.com/ml/binutils/2001-04/msg00373.html
http://sources.redhat.com/ml/binutils/2001-05/msg00019.html
It first appeared in an early form here:
[patch] ld [pei]: Filter off unnecessary symbols from auto-export
http://sources.redhat.com/ml/binutils/2000-11/msg00131.html

While I can't seem to find Paul's original auto-import patch, it made 
its appearance on Nov 4, 2000 as part of the mingw binutils port. 
Robert's version is here:
http://users.bigpond.net.au/robertc/paul-ld-auto-import-with-cygwin.patch

Ralf Habacker reported a bug in Paul's auto-import, which the patch at 
the URL below fixes(?). However, Robert's version does not use this 
patch; I've been testing without this patch, and do not see the buggy 
behavior Ralf reported.  I mention it here only for completeness.
http://sources.redhat.com/ml/binutils/2001-05/msg00088.html

Robert wrote:
> Paul a while back made a rather neat hack to pe to allow automatic
> importing of symbols without needing decoration. This allows libraries
> to be built with out any decoration in the headers - ie in standard UNIX
> format. AFAIK the patch has languished due to no one reviewing it.
>
> So... I've spent some time putting it through it's paces, and it seems
> to work without any issues at all. There were some teething issue, due
> to the patch not being developed for Cygwin, but for pw32. Thats been
> resolved (see below).
>
> I'm very happy with the way the patch performs, it builds all the
> existing decorated code I could find without trouble, and quite happily
> builds fully non-decorated code. Ralf Habacker has built KDE 1 with it,
> with a modified libtool.


I've also been experimenting recently with binutils as modified by 
Robert/Paul.  I've built the ld, and  then used that ld to rebuild 
another one.  The bootstrapped ld works fine.  I've built cygwin1.dll 
and newlib using this modified binutils.  I'm currently running my 
cygwin system with this cygwin1.dll without problems.

--> So, it seems the autoimport patch has no deleterious effects on 
previously-buildable source code. (At least in the case of cygwin1.dll, 
and a few others as listed below.)

I've built a dynamically linked version of the gettext package with both 
the old ld and the new patched ld.  In both cases, each version passed 
all of the 'make check' tests. (However, that package had previously 
been hacked to build on cygwin as a dll in the "old-fashioned" way -- 
__declspec()'s everywhere, .def files, nasty #ifdef this and #ifndef 
that.  I did NOT revert the cgywin-gettext back to the pristine "GNU" 
state and try to use libtool on it -- that'll come later <g>.  In this 
paragraph, I'm just concerned with regression testing -- so, I took the 
gettext source code EXACTLY as it is currently distributed in the cygwin 
distro, and used THAT.)

Anuway, with these two versions of gettext, I played musical chairs: I 
ran 'make check' in the old-ld gettext-buildtree but first replaced the 
oldld-cygintl.dll with the newld-cygintl.dll.  And vice versa.  Again, 
each passed all the 'make check' tests.  Thus, I've proven that 
executables and dll's built using the patched ld can interoperate with 
those produced by the old ld (as long as these target dll's and exe's 
are built from the same source, which has been appropriately decorated 
according to the 'old-style' -- __declspec's and .def's)

Okay, so then, moving on:

After installing the patched ld, I built:
  autoconf-2.52 (current cygwin distro uses 2.13)
  automake-1.4i (from latest CVS -- contains improvements necessary for:
  libtool-hacked (Robert Collin's port, but autoreconf'ed and built 
from his source.  This version uses the auto-imports capability to 
enable almost seamless dll-building on cygwin.  Probably mingw, too)

Using these tools I was able to build a very nice cygiconv-2.dll and 
dynamically-linked iconv.exe that passed all of its internal tests.  I 
only had to make fairly minimal changes to the libiconv source(*) and 
run autoreconf and (re)-libtoolize.

(*) These were mostly *.in changes and *.m4 updates since the libiconv 
maintainer ships his own hacked version of autoconf inside the tarball 
-- I wanted to use my locally-installed version so that autoreconf and 
libtoolize would update things correctly.

All of these packages and their source code (except libiconv) are 
available at 
http://www.neuro.gatech.edu/users/cwilson/cygutils/robert-collins/ in 
the following tree:

/setup.ini
   partial, so that setup.exe can be used to install the packages if 
you download and replicate this archive on your local hard drive.  The 
advantage here is that then, you can also use setup.exe to restore the 
official packages.
/latest/cygwin/
/latest/autoconf/
/latest/automake/
/contrib/libtool/
/buildscripts/
  the build scripts I used to create the above packages
/other
  gettext-0.10.38-newld.tar.bz2
  gettext-0.10.38-oldld.tar.bz2
  (so you can play round-robin-dll's too...)

The automake fails three tests (currently under investigation), as does 
libtool.  But, those are topics for other lists -- and the proof is in 
the pudding: libiconv built nice and smoothly, and WORKS!

Also, AFAIK, this auto-import patch has been in continuous use in the 
mingw project since last Novemeber.

Below is Robert's description of his version of Paul's patch(es):


> My patch also differs from Paul's original in that I've
> left disabled the auto-image-base option, which can cause issues with
> cygwin, due to a issue previously discussed. That option isn't part of
> the auto-import logic and thus shouldn't be part of this discussion IMO.


Can we PLEEEAAASSSEEE approve this patch for inclusion in binutils-CVS ? 
  Are there any objections remaining that are keeping it out? 
(Previously, I think the problem was lack of review/testing.  Hopefully 
Robert and I have demonstrated its utility and safety...)

--Chuck