[bug #18396] stack size setrlimit call interacts badly with Solaris/x86 kernel bug

[Paul D. Smith]

Update of bug #18396 (project make):

              Item Group:                     Bug => Enhancement            

    _______________________________________________________

Follow-up Comment #1:

> If 'make' needs to allocate a large amount (megabytes) of data,
> it would be better to do so on the heap, both from a
> portability standpoint (the stack size) and from a performance
> standpoint (it messes up the normally good locality of stack
> access).

> Alternatively, if it must allocate on the stack, then detecting
> and complaining about a too-low limit would be better in my
> opinion than silently changing it. It's easy to uncap the stack
> size explicitly in build scripts and whatnot when truly
> necessary.

Unfortunately, none of the above is true.  Make needs extra stack space
because it makes extensive use of the alloca() function.  It does not
allocate huge chunks of memory on the stack: if large amounts of memory are
needed they are allocated on the stack.  alloca() is used for modest-sized
memory chunks, to hold filenames and smaller strings (make is, at heart, a
string manipulation program).  Using heap, which requires a system call to
get more memory, for all of these small allocations would be much less
efficient than using the stack.  Not to mention the overhead of having to
allocate these memory segments to be used for a short time, then freeing them
again, over and over.

However, because make is also very recursive, even though no single alloca()
call is very large it's quite possible for the entirety of the alloca()
invocations for a complex makefile to use quite a bit of stack.

Of course, there's no way to determine how much stack will be used a priori,
since it depends entirely on the construction of the makefile, exactly which
functions are invoked in which order, and even the current state of the build
(whether various targets need to be rebuilt or not).  Further, there is no
portable way that I'm aware of to determine how much stack is left before the
program runs out.

Finally, there is no way to detect an out of stack error and exit gracefully
with a warning as you suggest: the behavior of alloca() is undefined if you
run out of stack space (it doesn't just return NULL as malloc() etc. do).

So.  In order to avoid the need for extra stack in GNU make all the
invocations of alloca() would need to be rewritten to use xmalloc(), and
those functions would need to be changed to be careful to free all that
memory whenever the function exited to avoid memory leaks... with what must
certainly be a decrease in performance (although of how much we can't be
sure).

On the other hand, as you point out, the problem on Solaris is a bug in the
kernel which you can hardly blame GNU make for.

However, your point about programs invoked by make inheriting the setrlimit()
is definitely something that seems problematic.  Perhaps GNU make could change
the stack limit back to what it was after it forks but before it execs its
child.  I wonder what happens if you change a limit to something too small
for the current processes resources?

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?18396>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/