Re: make 3.81 + i386-pc-interix* parallelism problem, and "write error"

[Markus Duft]

On Wed, 2009-05-06 at 09:02 +0100, Tim Murphy wrote:
> Hi,
> 
> I'm not a maintainer or anything but I have hacked around in the bit
> of code that implements the job-server.  It's complicated and in my
> case I realised that I actually didn't want it at all so I compiled
> make without it.

hey, so than even more thanks for replying :)

> 
> Do you really need it? Do you use sub-makes (i.e. do you have
> makefiles that call make again on other makefiles)?

hmm. we're using automake, so i guess yes, every subdir is managed by a
sub-make. does building parallel inside one dir work even without
jobserver, or is there no capability for this anymore as soon as i
disable it? (i thought so until now, but what you're saying sounds
somewhat like beeing able to keep building parallel, and just loosing
the capability to spread down in subdirs in parallel).

> I might be proposing a huge effort for you and you might not be
> compiling code that you can change at will, but if not then you could
> try using include to compose a big makefile rather than using
> sub-makes.

hehe. you're right, thats a big effort, and i guess not an option as
long as automake doesn't have propper support for that...

> 
> This brings its own complexities but it is pretty cool because it
> represents the whole dependency tree properly and it sort of allows
> more parallelism.

yeah that's right, but again: does this work without jobserver?

> 
> Anyhow the alternative is to debug job handling for interix and work
> out what's happening.  It could be some kind of issue with make
> noticing that some job has finished, or something to do with the job
> server blocking or just a bit of odd logic for interix.  It's not easy
> to debug this without having the os and a debugger in front of you.

mhm... and bad enought gdb on interix is badly broken, so the better
debugger is printf :-/ ...

Thanks anyway, Cheers, Markus

> 
> Regards,
> 
> Tim
> 
> 
> 
> 2009/5/6 Markus Duft <address@hidden>:
> > On Tue, 2009-03-24 at 07:06 +0100, Markus Duft wrote:
> >> On Tue, 2009-03-17 at 17:56 +0100, Markus Duft wrote:
> >> > Hi!
> >>
> >> ping...
> >
> > ping ping.... at least a comment on this would be cool... don't leave me
> > in the dark! i think my report is quite detailed, and not one of the
> > "make is broken - fix it" ...
> >
> >>
> >> >
> >> > I'm running make on x86 interix (SFU/SUA). All works like a charm, except
> >> > for occasional problems with multi-core machines (I'm not sure if the 
> >> > same
> >> > problems would occur on single cores too...).
> >> >
> >> > I'm seeing two different problems, the "bader" one being a jobserver 
> >> > token
> >> > leak. Sometimes make exits with messages like this:
> >> >
> >> >     make: INTERNAL: Exiting with 3 jobserver tokens available; should be
> >> > 2!
> >> >
> >> > But much worse is the case where make simply hangs, and seems to be out 
> >> > of
> >> > jobserver tokens! Is this a known problem? I tried the patch from [1], 
> >> > but
> >> > that didn't make any difference.
> >> >
> >> > Also I found a patch against 3.80, adding the jobserver_tokens global
> >> > variable, so I assume that one has been incorporated into 3.81, since 
> >> > this
> >> > variable already exists... (?).
> >> >
> >> > I attached the output of this command and the corresponding .diff file:
> >> >     while ( cd tests && perl ./run_make_tests.pl features/parallelism
> >> > -make ../make -debug -verbose ); do echo ...; done > xx 2>&1 &
> >> >
> >> > the test-1 log is seemingly another problem, which occurred while I 
> >> > tried to
> >> > faster provocate the error I'm reporting here by putting my 2 core AMD 
> >> > CPU
> >> > under heavy load. Ugh... another one :(
> >> >
> >> > FYI I attached the config.log too, so you can identify potential issues
> >> > caused by assumptions which may be wrong for interix.
> >> >
> >> > [1] http://savannah.gnu.org/bugs/?18124
> >> >
> >> > ----
> >> >
> >> > Now for the second problem, which is far easier to work around: Again 
> >> > only
> >> > occasionally I'm getting "make[N]: write error". After
> >> > R[eading]T[he]F[ine]S[source] I found out that ferror(stdout) triggers 
> >> > this
> >> > message. Strange... seems like there was an error writing to stdout..? or
> >> > could this point to an error writing to the jobserver fds?
> >> >
> >> > I worked around the problem for now by simply not exiting on this error,
> >> > since it's not too critical for me if a line in the output is missing or
> >> > something like that. Also since it's only occasionally, I can simple do 
> >> > make
> >> > again, If I really need all the output.
> >> >
> >> > BTW I never noticed missing output when I got "write error", but output
> >> > tends to be kilometers long, so I could have overseen that :)
> >> >
> >> > Any ideas regarding those two?
> >> >
> >> > P.S.: I just noticed that the --enable-case-insensitive-file-system 
> >> > (interix
> >> > is insensitive, as it uses window's fs') is win32 only as-is, since it 
> >> > uses
> >> > strcmpi. Attached is a patch fixing this for interix. Maybe this fixes 
> >> > for
> >> > this feature for mac osx too (is there support for max osx yet?).
> >> >
> >> > Thanks in advance, Cheers,
> >> > Markus Duft
> >> >
> >> > _______________________________________________
> >> > Bug-make mailing list
> >> > address@hidden
> >> > http://lists.gnu.org/mailman/listinfo/bug-make
> >>
> >>
> >>
> >> _______________________________________________
> >> Bug-make mailing list
> >> address@hidden
> >> http://lists.gnu.org/mailman/listinfo/bug-make
> >
> >
> >
> > _______________________________________________
> > Bug-make mailing list
> > address@hidden
> > http://lists.gnu.org/mailman/listinfo/bug-make
> >
> 
> 
>