Re: libltdl doesn't work on Cygwin, HP-UX, & Tru64 UNIX

[Albert Chin]

On Fri, Jun 21, 2002 at 09:16:14PM -0500, Bob Friesenhahn wrote:
> I thought you might want to know that CVS libltdl is still throwing an
> assertion under Cygwin.  It has now been reported to me (by Albert
> Chin-A-Young) that this happens under HP-UX & Tru64 UNIX as well:
> 
> Assertion failed: (!*pargz && !*pargz_len && !before) || ((*pargz <=
> before) && (before < (*pargz + *pargz_len))), file ltdl.c, line 547
> 
> We discovered that commenting out the assertion allowed loading
> modules to work for Cygwin.
> 
> Since it works on 1/2 the platforms that we have access to, the
> problem could be something like use of uninitialized data.

I did some research on this yesterday. Under platforms where
LTDL_DLOPEN_DEPLIBS is *not* set, libltdl never works (tested on HP-UX
10.20, Tru64 UNIX 4.0d, 5.1, AIX 4.3.2). The assertion above is
raised. The patch below unsets LTDL_DLOPEN_DEPLIBS for HP-UX, and
unsets it correctly for AIX/IRIX (autoconf bug).

However, we still have the problem where load_deplibs fails when
LTDL_DLOPEN_DEPLIBS is *not* set. Here's a backtrace on HP-UX 10.20:
(gdb) bt
#0  0xc0140310 in kill () from /usr/lib/libc.1
#1  0xc00a980c in raise () from /usr/lib/libc.1
#2  0xc008b2d8 in _sigaddset () from /usr/lib/libc.1
#3  0xc0085dbc in abort () from /usr/lib/libc.1
#4  0xc008b97c in _assert () from /usr/lib/libc.1
#5  0xc0ae31f4 in rpl_argz_insert (pargz=0x7b041ee8, pargz_len=0x7b041eec, 
    before=0x0, entry=0x4000fe06 "/opt/TWWfsw/libttf20/lib") at ltdl.c:547
#6  0xc0ae9424 in lt_argz_insert (pargz=0x7b041ee8, pargz_len=0x7b041eec, 
    before=0x0, entry=0x4000fe06 "/opt/TWWfsw/libttf20/lib") at ltdl.c:3037
#7  0xc0aeb234 in lt_dlpath_insertdir (ppath=0x7afc3340, before=0x0, 
    dir=0x4000fe06 "/opt/TWWfsw/libttf20/lib") at ltdl.c:3494
#8  0xc0aeb414 in lt_dladdsearchdir (
    search_dir=0x4000fe06 "/opt/TWWfsw/libttf20/lib") at ltdl.c:3519
#9  0xc0ae6bd0 in load_deplibs (handle=0x4000f9c0, 
    deplibs=0x4000fde8 " -L/opt/TWWfsw/libxml23/lib 
-L/opt/TWWfsw/libttf20/lib") at ltdl.c:2353
#10 0xc0ae85e0 in try_dlopen (phandle=0x7b041ce8, 
    filename=0x400e1d48 
"/opt/TWWfsw/imagick/lib/ImageMagick/modules/coders/gif.la") at ltdl.c:2818
#11 0xc0ae8c74 in lt_dlopen (
    filename=0x400e1d48 
"/opt/TWWfsw/imagick/lib/ImageMagick/modules/coders/gif.la") at ltdl.c:2917
#12 0xc0a613c4 in OpenModule (module=0x4001421c "GIF", exception=0x7b040420)
    at module.c:667
#13 0xc0a5be68 in GetMagickInfo (name=0x4001421c "GIF", exception=0x40017418)
    at magick.c:411
#14 0xc09adc6c in OpenBlob (image_info=0x40014168, image=0x40017268, 
    type=0x7af7eac8 "rb", exception=0x7b03a870) at blob.c:1126
#15 0xc09dcc64 in SetImageInfo (image_info=0x40014168, rectify=0, 
    exception=0x7b03a870) at image.c:6265
#16 0xc09bfe6c in ReadImage (image_info=0x40012090, exception=0x7b03a870)
    at constitute.c:2022
#17 0x5434 in main (argc=2, argv=0x40005e18) at display.c:424

Once lt_dladdsearchdir is entered, it calls lt_dlpath_insertdir as
follows:
  if (search_dir && *search_dir)
    {
      LT_DLMUTEX_LOCK ();
      if (lt_dlpath_insertdir (&user_search_path, 0, search_dir) != 0)
        ++errors;
      LT_DLMUTEX_UNLOCK ();
    }

This sets before = NULL as before is the second argument to
lt_dlpath_insertdir.

Once lt_dlpath_insertdir is entered, local vars argz, argz_len, and
before are NULL. The call to:
  if (argzize_path (*ppath, &argz, &argz_len) != 0)
    {
      ++errors;
      goto cleanup;
    }

sets argz and argz_len to non-NULL values. Then a call is made to:
  if (lt_argz_insert (&argz, &argz_len, before, dir) != 0)
    {
      ++errors;
      goto cleanup;
    }

which then calls the following from lt_argz_insert():
  if ((error = argz_insert (pargz, pargz_len, before, entry)))
    {
      switch (error)

We then encounter the assert that fails:
  /* Either PARGZ/PARGZ_LEN is empty and BEFORE is NULL,
     or BEFORE points into an address within the ARGZ vector.  */
  assert ((!*pargz && !*pargz_len && !before)
          || ((*pargz <= before) && (before < (*pargz + *pargz_len))));

So, at this point, pargz != NULL, pargz_len != NULL, and before ==
NULL. How is this assert *ever* going to succeed?

-- 
albert chin (address@hidden)

-- snip snip
2002-06-22  Albert Chin-A-Young <address@hidden>

        * ltdl.m4: fix invalid use of [] to quote. set
          libltdl_cv_sys_dlopen_deplibs=yes for HP-UX 10.x, 11.x.

Index: ltdl.m4
===================================================================
RCS file: /cvsroot/libtool/libtool/ltdl.m4,v
retrieving revision 1.40
diff -u -3 -p -r1.40 ltdl.m4
--- ltdl.m4     17 May 2002 16:07:25 -0000      1.40
+++ ltdl.m4     22 Jun 2002 14:45:44 -0000
@@ -133,13 +133,16 @@ AC_CACHE_CHECK([whether deplibs are load
     # we want this `case' here to explicitly catch those versions.
     libltdl_cv_sys_dlopen_deplibs=unknown
     ;;
-  aix[45]*)
+  aix[[45]]*)
     libltdl_cv_sys_dlopen_deplibs=yes
     ;;
   gnu*)
     libltdl_cv_sys_dlopen_deplibs=yes
     ;;
-  irix[12345]*|irix6.[01]*)
+  hpux10*|hpux11*)
+    libltdl_cv_sys_dlopen_deplibs=yes
+    ;;
+  irix[[12345]]*|irix6.[[01]]*)
     # Catch all versions of IRIX before 6.2, and indicate that we don't
     # know how it worked for any of those versions.
     libltdl_cv_sys_dlopen_deplibs=unknown
@@ -158,7 +161,7 @@ AC_CACHE_CHECK([whether deplibs are load
   openbsd*)
     libltdl_cv_sys_dlopen_deplibs=yes
     ;;
-  osf[1234]*)
+  osf[[1234]]*)
     # dlopen did load deplibs (at least at 4.x), but until the 5.x series,
     # it did *not* use an RPATH in a shared library to find objects the
     # library depends on, so we explictly say `no'.