Re: Stack direction check fails with optimizations

[Ralf Wildenhues]

Hi Peter,

I'm far from an expert in this area, so take this with a grain of salt.

* Peter O'Gorman wrote on Tue, Dec 12, 2006 at 01:26:42PM CET:
> 
> There is a test to check the stack direction when building the
> replacement alloca. This test returns bad results with optimizations:
[...]
>   exit (find_stack_direction () < 0);

Side note: a declaration for exit was not in place.  Autoconf uses
return from main, now that it does not provide for such a declaration
any more (since 2.60).

> It also fails with -xO2 on HPUX11.23/IA64. I realize that both these
> platforms have alloca so the autoconf test will never be run, but am
> wondering if any of the gurus on this list can come up with any ideas
> for a working test with optimizations. The problem is, of course, that
> the compiler inlines everything when optimizing so the results are
> not valid.

I think the ISO C answer is that a guaranteed working test is not
possible: you may not compare pointers to distinct objects.

In practice, you can make optimization harder by adding a second
translation unit which has an intermediate function that is called
by and calls itself find_stack_direction.  But that's nowhere near
bulletproof, with compilers that do link-time optimization.

For non-statically linked executables obeying ELF symbol resolution
rules you can preclude inlining by putting both functions in a shared
library; doing this portably inside a configure script is fun in itself
(e.g., avoid -Bsymbolic).

Since from an ISO C standpoint, there is no answer to the question in
general, it's probably best to reformulate the question: what do you
need to know the stack direction for, if not to emulate alloca?  The
answer should point you to some piece of code that should be able to
detect the difference (at least in some cases).

Hope that helps.

Cheers,
Ralf