Re: Autoconf tests, libtool symlist files, undefined behavior, and LTO

[Richard Guenther]

On Tue, Mar 30, 2010 at 8:52 PM, Ralf Wildenhues <address@hidden> wrote:
> Hello gcc and libtool lists,
>
> Summary: both Autoconf-generated configure tests as well as some Libtool
> construct invoke undefined behavior.  Question is how to deal with it,
> and whether GCC, as QoI, may want to define behavior in these cases.
>
>
> 1) Autoconf-generated configure tests often fake the prototype of some
> function; e.g., AC_CHECK_FUNC([func]) uses
>  char func();
>
> and tries to link that.  Using this is undefined according to C99, if
> func has a different actual prototype, and when all system libraries are
> LTO'ed, gcc -flto may even detect this kind of inconsistency and could
> act accordingly (nasal demons and such).

I suppose autoconf cannot do this for C++ functions then, because
of mangling issues?

Note that the only thing GCC with LTO might do here is to issue
a diagnostic (which of course might confuse the configure script),
but we cannot really reject such programs (as such errors are
unfortunately very common) and thus defer any problems to
link- and/or runtime.

> 2) libtool has a feature that makes it extract symbol lists from
> objects and turn them into fake declarations and function/object
> pointers: fake static preloaded modules.
>
> It currently works by running nm or a similar tool over the object, then
> converting the output with a couple of sed script or so
> (global_symbol_pipe, global_symbol_to_cdecl, and a couple more) to a
> synthesized extra source file that then contains code like this:
>
> extern int func();
> extern char variable;
>
> typedef struct {
>  const char *name;
>  void *address;
> } lt_dlsymlist;
>
> extern const lt_dlsymlist
> lt__PROGRAM__LTX_preloaded_symbols[];
> const lt_dlsymlist
> lt__PROGRAM__LTX_preloaded_symbols[] =
> {  { "@PROGRAM@", (void *) 0 },
>  {"func", (void *) &func},
>  {"variable", (void *) &variable},
>  {0, (void *) 0}
> };
>
> This is invoking undefined behavior in a couple of respects:
>
> a) Pointers to functions are stored in pointer-to-void variables.
> This could be fixed with an incompatible API change to using a union of
> object and function pointer, I guess.
>
> b) The symbols 'func' and 'variable' likely have the wrong prototypes,
> i.e., elsewhere, they might be declared as
>
>  void func(int, double);
>  double variable[42];
>
> instead.  I haven't come across any actual issues with this yet, except
> now LTO may rightfully complain about it.

Same issue as above.  We try to handle it - there might be bugs
in the current implementation of LTO though.

> Question is, what can we do about this?  We could ensure to never pass
> -flto or -fwhopr to the compilation of the libtool symfile object, and
> remove it from some or all link tests done in configure.  That's ugly.
> Would that even be sufficient though?  Conversely, would GCC developers
> be willing to agree that, when GCC detects such inconsistencies, it
> wouldn't take adverse advantage of it (e.g., by turning off LTO in this
> case, or similar)?
>
> Other possibilities for Autoconf would be to work toward a new set of
> checking macros (or extensions of current one) where the configure.ac
> author passes a full prototype for each function to check (Autoconf
> could keep a list of known prototypes for often-checked functions).
> I'm not sure how to fix the libtool symfile in a C99-conforming way.

I'd say wait and see.  What would be nice to have is a few testcases
that cover the autoconf cases in the GCC testsuite (feel free to
just file them into bugzilla).  We really do not want to break
working setups with LTO - any fancy ODR violation diagnostics
should IMHO be optional, only things that LTO does not get
correct are currently diagnosed IIRC.

Thanks,
Richard.

> Thanks for reading this far.
>
> Cheers,
> Ralf
>