Re: possible bug in Cray port -- "virtual memory exhausted"

[Ted Stern]

On Tue, 17 Dec 2002, Paul D. Smith wrote:
> Hm.  Have you applied the patch contained in Bug 1517?
>
>   https://savannah.gnu.org/bugs/?func=detailbug&bug_id=1517&group_id=71

Hi Paul,

Just completed a test of my build using a rebuilt 3.80 with the bug 1517
patch.  [Note to WendyP -- this was using the rebuilt GNU make in the open34
module.]

The build now manages to include 4321 makefiles instead of 4305.  So perhaps a
tiny amount of memory was saved by this patch, but not enough to fix the bug.

I'll try a test in which I try to include 10,000 empty files and see if that
triggers the bug. ... yes, I'm able to reproduce it.  Here is a simple script
to generate 10K files:

      #!/bin/ksh
      i=0
      while [[ $i -lt 10 ]] ; do
         j=0
         while [[ $j -lt 10 ]] ; do
            k=0
            while [[ $k -lt 10 ]] ; do
               mkdir -p AAAA_$i/BBBB_$j/CCCC_$k
               m=0
               while [[ $m -lt 10 ]] ; do
                  touch AAAA_$i/BBBB_$j/CCCC_$k/DDDD_$m.mk
                  m=$((m+1))
               done
               k=$((k+1))
            done
            j=$((j+1))
         done
         i=$((i+1))
      done

and here is a simple makefile:

      ALL_INCLUDES := $(shell find `/bin/pwd`/AAAA* -name '*.mk' -print)
      
      include $(ALL_INCLUDES)
      
      .PHONY: all
      
      all: ;    echo "completed successfully"

BTW, the point at which the memory runs out is dependent on the length of the
path, which is why I use relatively long pathnames for the test.

The result of "make -R -k -d all" on an SV1 Cray UNICOS system within an NQS
batch job requesting 128 megawords (1GB) is

      GNU Make 3.80
      Copyright (C) 2002  Free Software Foundation, Inc.
      This is free software; see the source for copying conditions.
      There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
      PARTICULAR PURPOSE.
      Reading makefiles...
      Reading makefile `Makefile'...
      Reading makefile `/ptmp/stern/testmake/AAAA_0/BBBB_0/CCCC_0/DDDD_0.mk' 
(search path) (no ~ expansion)...
      Reading makefile `/ptmp/stern/testmake/AAAA_0/BBBB_0/CCCC_0/DDDD_1.mk' 
(search path) (no ~ expansion)...
      Reading makefile `/ptmp/stern/testmake/AAAA_0/BBBB_0/CCCC_0/DDDD_2.mk' 
(search path) (no ~ expansion)...
      
        <snip>
      
      Reading makefile `/ptmp/stern/testmake/AAAA_4/BBBB_5/CCCC_9/DDDD_6.mk' 
(search path) (no ~ expansion)...
      Reading makefile `/ptmp/stern/testmake/AAAA_4/BBBB_5/CCCC_9/DDDD_7.mk' 
(search path) (no ~ expansion)...
      Reading makefile `/ptmp/stern/testmake/AAAA_4/BBBB_5/CCCC_9/DDDD_8.mk' 
(search path) (no ~ expansion)...
      Reading makefile `/ptmp/stern/testmake/AAAA_4/BBBB_5/CCCC_9/DDDD_9.mk' 
(search path) (no ~ expansion)...
      make: *** virtual memory exhausted.  Stop.

As you can see, the memory is exhausted when 5000 files have been included.

Perhaps this will help you isolate the problem.

Ted
--
 Ted Stern                           Engineering Applications
 Cray Inc.                               office: 206-701-2182
 411 First Avenue South, Suite 600         cell: 206-383-1049
 Seattle, WA 98104-2860                     FAX: 206-701-2500
 Debuggers' motto:  Frango ut patefaciam -- I break in order to reveal