[PATCH,HURD] fix muntrace with mmap-less libio

[Pino Toscano]

Hi,

I was debugging on Hurd a misbehaviour of muntrace, which would just 
spin taking 100% CPU: using a simple test like:
--vvvvv--
#include <mcheck.h>
int main()
{ mtrace(); muntrace(); return 0; }
--^^^^^--
when run as `MALLOC_TRACE=out test`, you get a backtrace like the 
attached trace.log (that one has been produced with debian's eglibc 
2.13, but it does the same with current glibc).

What happens is the following:
a) muntrace gets called
b) in muntrace, "= End\n" is written to the file, and fclose on it is
   called
c) in fclose, free is called to free the FILE*, triggering the free hook
   (which is still set)
d) in tr_freehook, lock_and_info is called which locks the lock, and
   then tr_where is called
e) in tr_where, fprintf in called, which at some point calls
   _IO_file_doallocate
and at this point there is the behaviour difference between Linux and 
Hurd: in Linux EXEC_PAGESIZE is provided (by sys/param.h, coming from 
linux/param.h), thus in libio/libioP.h _G_HAVE_MMAP is kept, so 
ALLOC_BUF and FREE_BUF (used in _IO_file_doallocate) use mmap/munmap. On 
the Hurd, however, EXEC_PAGESIZE is provided nowhere (and this causes 
build issues also in two files under elf/), so the two _BUF macros use 
malloc/free... which during the muntrace execution in turn calls the 
malloc hook (which is still set), and then lock_and_info tries to lock 
the lock -> deadlock.

Ignoring the fact that in libio mmap is not used on Hurd (it will need a 
different fix), it seems to me this whole hook triggering during 
muntrace seems more harmful than useful (on Linux it is attempted to 
output the log for the free of mallstream, which always fails since that 
stream is closed at that point), so my attached proposal is to first 
unset mallstream and the hooks, and only after that close the file.

Thanks,
-- 
Pino Toscano

trace.log
Description: Text Data

fix-muntrace.diff
Description: Text Data

signature.asc
Description: This is a digitally signed message part.