Re: gettext cvs, woe32 dlls

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Forwarding this very useful discussion of library builds on cygwin on
bug-gnu-utils.  There are some suggestions in there for what libtool
should be doing to ease library builds.

Eric Blake

Date: Wed, 10 May 2006 23:25:46 -0400
From: Charles Wilson
To: Bruno Haible
CC: bug-gnu-gettext

Bruno Haible wrote:
> Hello Charles,
> 
> I have a question about the WOE32GCC_SHLIBS conditional:

> When a library libfoo contains a public, exported variable 'int data;',
> and the library is installed with both shared and static libs (.dll, .dll.a
> and .a), then what should the include file libfoo.h contain?
> 
>     extern __declspec(dllimport) int data;
> is the right thing when a program links to the shared variant (so that the
> compiler knows to use _imp__data instead of data itself).
> 
>     extern int data;
> is the right thing when a program links to the static variant.
> 
> But how can the header file know whether the program will link statically
> or not?

It can't, not without help, unfortunately.

> This matters for libintl, libasprintf and libgettextpo.

Yes, in general you are right: we have four states (per library)
   building the library as shared : declspec(dllexport)
   building the library as static : <no decorator> (*)
   building a client of the library, intending to link shared :
      extern declspec(dllimport)
   building a client of the library, intending to link static :
      extern (*)

(*) Technically, you could ALWAYS declare variables extern in the header
files, and just ensure that one .c file in the library actually defines
the variable.  This collapses your state space to just three: in-library
shared, out-library shared, and static(in or out).  But...

This means we need two binary macros to fully span the state space.
With the code in its present state, only libintl of those three libs has
the full complement (and even that is cheating).  With my patch,
libintl, libgettextsrc and libgettextlib have the full complement (but
are still cheating):

BUILDING_LIBINTL + DLL_EXPORT

Back before the advent of autoimport, I supported DLL builds of various
libraries using the mechanism below. However, it doesn't really mesh
well with libtool, unless you "overload" the meaning of libtool's
DLL_EXPORT macro (which is what I did in the patches I posted).

(This lack of meshing well with libtool is why most other packages --
and libtool -- signed on to the auto-import bandwagon)

==== old mechanism ====
(1) First, assert that there is one "normal" way and one "special" way
to build the library and to link against the library (e.g. there has to
be some default when you don't #define any of your state macros).

I decided that "default" was 'I'm building an application that will link
dynamically against the library'.  I felt this placed the least burden
on "normal" users: use DLL, no need to #define anything.  use .a
...well, then you have to take special steps.  Building the dll or .a --
more special steps, but those were under my control.  Plus, in the
presence of .la with both old and shared libraries, that's libtool's
default; and if both .a and .dll.a exist, it is gcc/ld's default as
well: link against shared when possible.

(2) One state variable is "BUILDING_LIBRARY_FOO" -- I added this to the
Makefile.am/ la_CFLAGS for the internal gettext libraries, but not for
libasprintf and libgettextpo (libintl already had it).  Why did I skip
those two?  'Cause they seemed to work okay in my minimal tests, but I
admit I did not attempt use the .a's.

(3) The other state variable is "STATIC_LIBRARY_FOO".  This differs
(sortof) from the way I implemented it in the patch.  [[[In the patch, I
overloaded DLL_EXPORT to serve almost the same purpose, except that
ideally you'd have one macro for each library FOO, not a single macro
for all libraries.  The latter situation complicates matters when you're
building apps that depend on libs that depend on libs: your app SHALL
link against the shared version of both libs, or the static version of
both libs, not dll one and static other.]]]

Anyway, then I had a stanza in some header library file that looked like
this:

#if BUILDING_LIBRARY_FOO
# if STATIC_LIBRARY_FOO
#  define LIBRARY_FOO_EXTERN
#  define LIBRARY_FOO_API
# else
#  define LIBRARY_FOO_EXTERN
#  define LIBRARY_FOO_API declspec(dllexport)
# endif
#else
# if STATIC_LIBRARY_FOO
#  define LIBRARY_FOO_EXTERN extern
#  define LIBRARY_FOO_API
# else
#  define LIBRARY_FOO_EXTERN extern
#  define LIBRARY_FOO_API declspec(dllimport)
# endif

So you see, the "default" case is group 4: 'I'm linking against the foo
DLL.'  Since that's the default behavior of libtool and gcc when both
static and dynamic libraries are available, I thought it made sense.

However, it means that clients who want to link statically have to
behave differently during both compile and link phase: they must #define
STATIC_LIBRARY_FOO when compiling as well as link with -static (and
don't get me started on the libtool disaster that is -all-static and its
newer cousin.)

I don't know how to coerce libtool into managing that ('when libtool
mode=link is called, the link will find an import library, so don't
define anything during this libtool mode=compile.  However, if in that
future libtool mode=link, the link will instead find only a static
library, then I'd better define the appropriate LIB_STATIC flat now'

When you figure out how to make libtool precognitive, let me know.

And it gets worse if you have a series of unrelated libraries:
- -DJPEG_STATIC -DTIFF_STATIC -DPNG_STATIC -DZLIB_STATIC... ick.
Expecially if there IS, for instance, a zlib dll but NOT a png dll.  So,
your precognitive libtool would -DPNG_STATIC but not ZLIB_STATIC -- and
again, the link would fail!  Because libpng.a has some undefined symbols
satisfied only by libz.a NOT libz.dll.a!  So, the libtool precognition,
when building/linking an exe, needs to take into account the libdeps and
propagate PNG_STATIC to ZLIB_STATIC even if it "knows" that there is a
.dll.a (and it better also use -static/-all-static/-whatever-static-cousin).

So, I eventually came up with a shortcut: all such stanzas would support
a new macro ALL_STATIC, and 'if STATIC_LIBRARY_FOO' became 'if
STATIC_LIBRARY_FOO || ALL_STATIC'. Which was somewhat easier, but not as
flexible -- and I still don't know how to get libtool to handle it for
me automatically.  But at least only ONE symbol is needed (assuming all
deps honor it) and the static-obsessed user building some client can at
least do:

  CFLAGS=-DALL_STATIC ./configure --disable-shared

which oughta work.
=====

So, having explained that, what I did in the patch I posted was to
overload the meaning of libtool's DLL_EXPORT to mean NONE_STATIC -- with
the restriction that only-like-may-link-to-like -- which is basically
true with libtool: on cygwin/mingw, you can't make a shared lib with
static dependencies (except for certain system libs).  So, this is not
TOO restrictive when dealing with libtoolized libraries, because it adds
no NEW restrictions.  The downside is that the "default case" -- with
neither state variable defined -- is now "I'm compiling an object that
will eventually be linked against a *static* library".

This was acceptable for the internal libs, where I (or my patch, at
least) controlled the compile-time variables.  It wouldn't be so great
for external clients of, say, libgettextpo, because now the "default"
compile setting does not match the default (from gcc and ld's point of
view) link setting.

Anyway, with those definitions, the libgettextsrc macro stanza becomes
something like the following (note the reversed order):

#if BUILDING_LIBRARY_FOO
# if DLL_EXPORT
#  define LIBRARY_FOO_EXTERN
#  define LIBRARY_FOO_API declspec(dllexport)
# else
#  define LIBRARY_FOO_EXTERN
#  define LIBRARY_FOO_API
# endif
#else
# if DLL_EXPORT
#  define LIBRARY_FOO_EXTERN extern
#  define LIBRARY_FOO_API declspec(dllimport)
# else
#  define LIBRARY_FOO_EXTERN extern
#  define LIBRARY_FOO_API
# endif

and I forced the apps to ALSO define DLL_EXPORT (because it now also
means 'I'm linking against a DLL'). Happily, since libtool already
defined DLL_EXPORT when building client *dlls* like libgettextpo, The
Right Thing happened when building the objects of libgettextpo that
depended upon the libgettextsrc headers.  But that was only possible
because you had library-specific BUILDING_ macros.

One further simplification: since I didn't really worry about 'extern'
(maybe I should have?) I could collapse two of the four states (and,
with a little rearranging):

#if DLL_EXPORT
# if BUILDING_LIBRARY_FOO
#  define LIBRARY_FOO_API declspec(dllexport)
# else
#  define LIBRARY_FOO_API declspec(dllimport)
# endif
#else
# define LIBRARY_FOO_API
#endif

and that's basically what's in libgettextsrc.

If the 'overloading' of DLL_EXPORT makes sense, then you could obviously
 define new macros BUILDING_LIBGETTEXTPO and BUILDING_LIBASPRINTF in
the appropriate Makefile.am/la_CFLAGS, and reuse this same machinery.
But it imposes a burdon on all clients: they must choose

  (1) either link using -static
OR
  (2) compile defining DLL_EXPORT

(note that external clients using libtool to build client ***DLLs***
will do this correctly: when in "pic" mode they WILL define DLL_EXPORT,
otherwise not)

If the client does something else (like neither (1) nor (2)) then they
will have link errors.  This is bad, and doesn't meet user expectation.
 That's why I was happy that I could restrict my manipulations to the
gettext-private libraries, where "I" was the the only client (e.g. my
patch controlled the client's compile and link flags).

=====

perhaps the "right" thing to do is for libtool on cygwin to -DALL_STATIC
(or something less likely to cause namespace collision) in every case
that it does NOT define DLL_EXPORT.  Then, we could gradually begin
using that flag instead, and reassert "correct" default behavior over time.

Sorry for the rambling, but I don't really have a good answer.  I
suspect there isn't one that will satisfy all needs.  When auto-import
was invented, I dropped the mess above like a bad habit, and never
wanted to look back...

- --
Chuck


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEYyW684KuGfSFAYARAjpPAJ9QhbMtWKFd7nqrFOc6uVfkVLoAkwCcD/8V
36+8A7T1JGND9QwhCUZoE5g=
=cyDS
-----END PGP SIGNATURE-----