[cygwin|mingw] dlpreopen and disable-static

[Charles Wilson]

When attempting to extract symbols from a library, libtool fails when 
there is no static library available.  This is demonstrated by (old) 
testsuite failures 'demo-exec.test' after 'demo-shared.test', which are 
caused by missing symbol errors when running helldl.  That is, the 
lt__PROGRAM__LTX_preloaded_symbols[] is being populated incorrectly in 
that case, because $global_symbols_pipe is not correct for dlls.

NOTE: when both static lib and dll are available, we actually link 
against the import lib NOT the static lib, even for dlpreopen.  We just 
use the static lib to obtain the symbols names. Not sure if /that/ is 
the right thing to do either...

The key passage in libtool:

          # Prefer using a static library (so that no silly _DYNAMIC 
symbols
          # are required to link).
          if test -n "$old_library"; then
            newdlprefiles="$newdlprefiles $dir/$old_library"
            # Keep a list of preopened convenience libraries to check
            # that they are being used correctly in the link pass.
            test -z "$libdir" && \
                dlpreconveniencelibs="$dlpreconveniencelibs 
$dir/$old_library"
          # Otherwise, use the dlname, so that lt_dlopen finds it.
          elif test -n "$dlname"; then
            newdlprefiles="$newdlprefiles $dir/$dlname"
          else
            newdlprefiles="$newdlprefiles $dir/$linklib"
          fi
        fi # $pass = dlpreopen

So, the normal case (where static libraries are available) dlpreopen 
works fine -- the correct symbol names are obtained from the static lib 
(even though the eventual link is against the dll thanks to cygwin ld's 
-l<name> search order, and exploits auto-import).

By manually running the import library, dll, and static library through 
the various steps, I generated three different versions of 
helldl.exeS.c, attached.

What is interesting is that the current transformation steps are 
incorrect if used with EITHER the import lib or the DLL; the file 
generated using the static library is appropriate for both static and 
dynamic linking, assuming --enable-auto-import (and possibly 
--enable-runtime-pseudo-reloc).

In the import lib case, the cyghello_2_dll_iname and 
_head_cyghello_2_dll symbols are inappropriate; these could be solved 
using a tag variable exclude_expsyms (why do cygwin|mingw libtools 
exclude "_GLOBAL_OFFSET_TABLE_|_GLOBAL__F[ID]_.*"? Those are not PE 
symbols...)


The __imp_* symbols for functions are also imappropriate (ASIDE: unless 
we know we are doing a link to the DLL, and want to transform the symbol 
lookup as:
   extern char _imp__foo; -->
     extern int _imp__foo();
   ...
  {"_imp__foo", (void *) &_imp__foo}, -->
    {"foo", (void *) &_imp__foo},
(END ASIDE)

but these do no harm if left as-is.  That leaves symbols related to the 
data imports like 'nothing':

extern char _imp__nothing;
extern char _nm__nothing;
...
  {"_imp__nothing", (void *) &_imp__nothing},
  {"_nm__nothing", (void *) &_nm__nothing},

We could exclude all "^_imp__*" symbols -- or all _imp__ symbols with 
matching _nm_s, with the understanding that a simple $global_symbol_pipe 
can't do that -- and transform "^_nm__*" symbols into s/^_nm__//.

But that's only if we're using the link library, and not the DLL itself, 
to generate the symbol list. But if we change the "key passage" above to 
prefer the link library -- even if only for cygwin|mingw -- then the 
wrong library name will be encoded as the first "import" in the 
lt__PROGRAM__LTX_preloaded_symbols[] array:

  {"libhello.dll.a", (void *) 0},

instead of

  {"cyghello-2.dll", (void *) 0},

which means that lt_dlopen won't work properly. [*]  So, we really ought 
to figure out how to extract *the correct* symbols from the dll, or say 
dlpreopen doesn't work on win32 with --disable-static.

[*] not sure why it works properly now, when both static lib and DLL are 
present. We encode {"libhello.a", (void *) 0}, in the symbol array, but 
ld's default -l<name> search order means we actually link against 
libhello.dll.a and our runtime symbols are satisfied by cyghello-0.dll 
-- not "libhello.a".  Weird.


The import list currently generated from the DLL is a real mess. Just 
running 'nm' on cyghello-0.dll gives a few interesting results:

6c641020 T _foo
6c6413c0 T _free

_free is from cygwin1.dll, not cyghello-0.dll.  But _foo is from 
cyghello-0.dll.  How do you distinguish between them? (In this case, 
_free has a matching _imp__free, but _foo does not. Is that always the 
case? And besides, a simple pipe can't do that sort of elimination: 
"don't include entries for X if _imp__X exists elsewhere in the input")

6c642004 D __data_end__
6c642000 D _nothing

Again, how do you distinguish between the first, a book-keeping symbol 
createed by the linker, and the second, an actual data export of 
cyghello-0.dll.  (Now, __data_end__ is a "well known symbol" for the 
cygwin linker, but see next paragraph).

We could specify a very long and detailed exclude_expsyms for this case, 
but that requires close tracking of binutils and platform-runtime 
changes. For instance, the new binutils released *yesterday* for cygwin 
adds new book-keeping export(s):

6c640000 A __image_base__
6c640000 A ___ImageBase

I believe this stuff used to work, even when --disable-static.  At one 
point, I seem to remember that some $filter included some 
transformations with _nm_ and _imp_, but I can no longer locate that in 
the archives or online.

--
Chuck

/* helldl.exeS.c - symbol resolution table for `helldl.exe' dlsym emulation. */
/* Generated by ltmain.sh (GNU libtool 1.2983 2008-05-20) 2.2.5a */

#ifdef __cplusplus
extern "C" {
#endif

/* External symbol declarations for the compiler. */
extern int _CTOR_LIST__();
extern int _DTOR_LIST__();
extern char _RUNTIME_PSEUDO_RELOC_LIST_END__;
extern char _RUNTIME_PSEUDO_RELOC_LIST__;
extern int __CTOR_LIST__();
extern int __DTOR_LIST__();
extern char __ImageBase;
extern char __RUNTIME_PSEUDO_RELOC_LIST_END__;
extern char __RUNTIME_PSEUDO_RELOC_LIST__;
extern char _bss_end__;
extern char _bss_start__;
extern char _data_end__;
extern char _data_start__;
extern char _dll__;
extern char _file_alignment__;
extern char _fmode;
extern char _head_cygwin1_dll;
extern char _head_libkernel32_a;
extern char _image_base__;
extern char _imp___impure_ptr;
extern char _imp__calloc;
extern char _imp__cygwin_detach_dll;
extern char _imp__cygwin_internal;
extern char _imp__dll_dllcrt0;
extern char _imp__free;
extern char _imp__malloc;
extern char _imp__printf;
extern char _imp__puts;
extern char _imp__realloc;
extern char _libkernel32_a_iname;
extern char _loader_flags__;
extern char _major_image_version__;
extern char _major_os_version__;
extern char _major_subsystem_version__;
extern char _minor_image_version__;
extern char _minor_os_version__;
extern char _minor_subsystem_version__;
extern char _nm___impure_ptr;
extern int _pei386_runtime_relocator();
extern char _section_alignment__;
extern char _size_of_heap_commit__;
extern char _size_of_heap_reserve__;
extern char _size_of_stack_commit__;
extern char _size_of_stack_reserve__;
extern char _subsystem__;
extern int calloc();
extern char cygwin1_dll_iname;
extern int cygwin_attach_dll();
extern int cygwin_detach_dll();
extern int cygwin_internal();
extern int cygwin_premain0();
extern int cygwin_premain1();
extern int cygwin_premain2();
extern int cygwin_premain3();
extern int dll_dllcrt0();
extern int do_pseudo_reloc();
extern char environ;
extern int foo();
extern int free();
extern int hello();
extern int malloc();
extern char nothing;
extern int printf();
extern int puts();
extern int realloc();

/* The mapping between symbol names and symbols.  */
typedef struct {
  const char *name;
  void *address;
} lt_dlsymlist;

/* DATA imports from DLLs on WIN32 con't be const, because
   runtime relocations are performed -- see ld's documentation
   on pseudo-relocs.  */
extern  lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[];
 lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[] =
{  { "@PROGRAM@", (void *) 0 },
  {"cyghello-2.dll", (void *) 0},
  {"_CTOR_LIST__", (void *) &_CTOR_LIST__},
  {"_DTOR_LIST__", (void *) &_DTOR_LIST__},
  {"_RUNTIME_PSEUDO_RELOC_LIST_END__", (void *) 
&_RUNTIME_PSEUDO_RELOC_LIST_END__},
  {"_RUNTIME_PSEUDO_RELOC_LIST__", (void *) &_RUNTIME_PSEUDO_RELOC_LIST__},
  {"__CTOR_LIST__", (void *) &__CTOR_LIST__},
  {"__DTOR_LIST__", (void *) &__DTOR_LIST__},
  {"__ImageBase", (void *) &__ImageBase},
  {"__RUNTIME_PSEUDO_RELOC_LIST_END__", (void *) 
&__RUNTIME_PSEUDO_RELOC_LIST_END__},
  {"__RUNTIME_PSEUDO_RELOC_LIST__", (void *) &__RUNTIME_PSEUDO_RELOC_LIST__},
  {"_bss_end__", (void *) &_bss_end__},
  {"_bss_start__", (void *) &_bss_start__},
  {"_data_end__", (void *) &_data_end__},
  {"_data_start__", (void *) &_data_start__},
  {"_dll__", (void *) &_dll__},
  {"_file_alignment__", (void *) &_file_alignment__},
  {"_fmode", (void *) &_fmode},
  {"_head_cygwin1_dll", (void *) &_head_cygwin1_dll},
  {"_head_libkernel32_a", (void *) &_head_libkernel32_a},
  {"_image_base__", (void *) &_image_base__},
  {"_imp___impure_ptr", (void *) &_imp___impure_ptr},
  {"_imp__calloc", (void *) &_imp__calloc},
  {"_imp__cygwin_detach_dll", (void *) &_imp__cygwin_detach_dll},
  {"_imp__cygwin_internal", (void *) &_imp__cygwin_internal},
  {"_imp__dll_dllcrt0", (void *) &_imp__dll_dllcrt0},
  {"_imp__free", (void *) &_imp__free},
  {"_imp__malloc", (void *) &_imp__malloc},
  {"_imp__printf", (void *) &_imp__printf},
  {"_imp__puts", (void *) &_imp__puts},
  {"_imp__realloc", (void *) &_imp__realloc},
  {"_libkernel32_a_iname", (void *) &_libkernel32_a_iname},
  {"_loader_flags__", (void *) &_loader_flags__},
  {"_major_image_version__", (void *) &_major_image_version__},
  {"_major_os_version__", (void *) &_major_os_version__},
  {"_major_subsystem_version__", (void *) &_major_subsystem_version__},
  {"_minor_image_version__", (void *) &_minor_image_version__},
  {"_minor_os_version__", (void *) &_minor_os_version__},
  {"_minor_subsystem_version__", (void *) &_minor_subsystem_version__},
  {"_nm___impure_ptr", (void *) &_nm___impure_ptr},
  {"_pei386_runtime_relocator", (void *) &_pei386_runtime_relocator},
  {"_section_alignment__", (void *) &_section_alignment__},
  {"_size_of_heap_commit__", (void *) &_size_of_heap_commit__},
  {"_size_of_heap_reserve__", (void *) &_size_of_heap_reserve__},
  {"_size_of_stack_commit__", (void *) &_size_of_stack_commit__},
  {"_size_of_stack_reserve__", (void *) &_size_of_stack_reserve__},
  {"_subsystem__", (void *) &_subsystem__},
  {"calloc", (void *) &calloc},
  {"cygwin1_dll_iname", (void *) &cygwin1_dll_iname},
  {"cygwin_attach_dll", (void *) &cygwin_attach_dll},
  {"cygwin_detach_dll", (void *) &cygwin_detach_dll},
  {"cygwin_internal", (void *) &cygwin_internal},
  {"cygwin_premain0", (void *) &cygwin_premain0},
  {"cygwin_premain1", (void *) &cygwin_premain1},
  {"cygwin_premain2", (void *) &cygwin_premain2},
  {"cygwin_premain3", (void *) &cygwin_premain3},
  {"dll_dllcrt0", (void *) &dll_dllcrt0},
  {"do_pseudo_reloc", (void *) &do_pseudo_reloc},
  {"environ", (void *) &environ},
  {"foo", (void *) &foo},
  {"free", (void *) &free},
  {"hello", (void *) &hello},
  {"malloc", (void *) &malloc},
  {"nothing", (void *) &nothing},
  {"printf", (void *) &printf},
  {"puts", (void *) &puts},
  {"realloc", (void *) &realloc},
  {0, (void *) 0}
};

/* This works around a problem in FreeBSD linker */
#ifdef FREEBSD_WORKAROUND
static const void *lt_preloaded_setup() {
  return lt__PROGRAM__LTX_preloaded_symbols;
}
#endif

#ifdef __cplusplus
}
#endif

/* helldl.exeS.c - symbol resolution table for `helldl.exe' dlsym emulation. */
/* Generated by ltmain.sh (GNU libtool 1.2983 2008-05-20) 2.2.5a */

#ifdef __cplusplus
extern "C" {
#endif

/* External symbol declarations for the compiler. */
extern char _head_cyghello_2_dll;
extern char _imp__foo;
extern char _imp__hello;
extern char _imp__nothing;
extern char _nm__nothing;
extern char cyghello_2_dll_iname;
extern int foo();
extern int hello();

/* The mapping between symbol names and symbols.  */
typedef struct {
  const char *name;
  void *address;
} lt_dlsymlist;

/* DATA imports from DLLs on WIN32 con't be const, because
   runtime relocations are performed -- see ld's documentation
   on pseudo-relocs.  */
extern  lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[];
 lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[] =
{  { "@PROGRAM@", (void *) 0 },
  {"libhello.dll.a", (void *) 0},
  {"_head_cyghello_2_dll", (void *) &_head_cyghello_2_dll},
  {"_imp__foo", (void *) &_imp__foo},
  {"_imp__hello", (void *) &_imp__hello},
  {"_imp__nothing", (void *) &_imp__nothing},
  {"_nm__nothing", (void *) &_nm__nothing},
  {"cyghello_2_dll_iname", (void *) &cyghello_2_dll_iname},
  {"foo", (void *) &foo},
  {"hello", (void *) &hello},
  {0, (void *) 0}
};

/* This works around a problem in FreeBSD linker */
#ifdef FREEBSD_WORKAROUND
static const void *lt_preloaded_setup() {
  return lt__PROGRAM__LTX_preloaded_symbols;
}
#endif

#ifdef __cplusplus
}
#endif

/* helldl.exeS.c - symbol resolution table for `helldl.exe' dlsym emulation. */
/* Generated by ltmain.sh (GNU libtool 1.2983 2008-05-20) 2.2.5a */

#ifdef __cplusplus
extern "C" {
#endif

/* External symbol declarations for the compiler. */
extern int foo();
extern int hello();
extern char nothing;

/* The mapping between symbol names and symbols.  */
typedef struct {
  const char *name;
  void *address;
} lt_dlsymlist;

/* DATA imports from DLLs on WIN32 con't be const, because
   runtime relocations are performed -- see ld's documentation
   on pseudo-relocs.  */
extern  lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[];
 lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[] =
{  { "@PROGRAM@", (void *) 0 },
  {"libhello.a", (void *) 0},
  {"hello", (void *) &hello},
  {"foo", (void *) &foo},
  {"nothing", (void *) &nothing},
  {0, (void *) 0}
};

/* This works around a problem in FreeBSD linker */
#ifdef FREEBSD_WORKAROUND
static const void *lt_preloaded_setup() {
  return lt__PROGRAM__LTX_preloaded_symbols;
}
#endif

#ifdef __cplusplus
}
#endif