Re: patch to support output synchronization under -j

[Eli Zaretskii]

> Date: Fri, 15 Apr 2011 10:37:13 -0400
> From: David Boyce <address@hidden>
> Cc: address@hidden, address@hidden
> 
> > Finally, wouldn't it be a potential problem top inherit so many
> > handles to subordinate processes (2 for each running job)?  We could
> > run out of available handles in deeply recursive jobs, couldn't we?
> 
> Just to be completely clear, we're not talking about a change to
> default behavior - it's merely a new option.

But this new option uses up 2 additional files per job, doesn't it?

> But yes, any feature
> which consumes additional resources increases the risk of running out
> of that resource. I don't know that recursion comes into it though,

Each sub-Make inherits all the file descriptors of all its parents,
grandparents, etc.  If a sub-Make was launched when 4 other jobs were
running in parallel, the sub-Make will inherit 8 file descriptors that
it won't use.  The deeper the recursion, the more inherited
descriptors.

> it's more a question of how many parallel jobs a given make process is
> managing since limits on file handles/descriptors are per process.

What about people who use "make -j" without limits?

> The strictest file limit I know of is on old Unixy platforms. The
> tmpfile() API returns a FILE *, not a file descriptor, and older
> Unices have a limit of 256 of these. But even in that corner case, how
> many people are going to run make -j200+? How many 64-CPU processors
> are running really old kernels?

It's not like having a 256-core machine is a fantasy that will not
happen in a few years.  On my day job, we are already using 12 cores
and don't consider that "living on the edge".

> Parenthetically, I only use FILE * because that's what tmpfile hands
> back. I believe it would be possible to dup the file descriptor out of
> the FILE *, close it right away, and work with an integral file
> descriptor from there. That would avoid the above limit.

I think the real kernel limitation, if there is one, is on file
descriptors, not FILE objects.  The latter live in the application.

Anyway, if no one is worried about the inherited handles, just ignore
me on that.