Stack direction check fails with optimizations

[Peter O'Gorman]

Hi,

Really I discovered this issue when building mysql, but they swiped a
test from autoconf to check the stack direction, so I'll ask here.

There is a test to check the stack direction when building the
replacement alloca. This test returns bad results with optimizations:

Solaris 10/x86
% cat stack.c
int
find_stack_direction ()
{
  static char *addr = 0;
  auto char dummy;
  if (addr == 0)
    {
      addr = &dummy;
      return find_stack_direction ();
    }
  else
    return (&dummy > addr) ? 1 : -1;
}

int
main ()
{
  exit (find_stack_direction () < 0);
}
% cc -o stack stack.c
% ./stack
% echo $?
1
% cc -o  stack -xO4 stack.c
% ./stack
% echo $?
0

It also fails with -xO2 on HPUX11.23/IA64. I realize that both these
platforms have alloca so the autoconf test will never be run, but am
wondering if any of the gurus on this list can come up with any ideas
for a working test with optimizations. The problem is, of course, that
the compiler inlines everything when optimizing so the results are
not valid.

Peter