Re: patch to support output synchronization under -j

[David Boyce]

On Fri, Apr 15, 2011 at 11:09 AM, Eli Zaretskii <address@hidden> wrote:
> But this new option uses up 2 additional files per job, doesn't it?

One or two, as discussed elsewhere in this thread.

> Each sub-Make inherits all the file descriptors of all its parents,
> grandparents, etc.  If a sub-Make was launched when 4 other jobs were
> running in parallel, the sub-Make will inherit 8 file descriptors that
> it won't use.  The deeper the recursion, the more inherited
> descriptors.

You have a good point. I should probably turn on FD_CLOEXEC for these.

> What about people who use "make -j" without limits?

As noted, those people are at risk of exhausting many resources.

> It's not like having a 256-core machine is a fantasy that will not
> happen in a few years.  On my day job, we are already using 12 cores
> and don't consider that "living on the edge".

Right, but my point was the limits of which I speak are old. I don't
think a 256-core processor will ever run on a kernel from the 1980's.
Or in other words, part of scaling an OS to many cores involves
removing or at least upping artificial resource constraints. The
natural trend toward 64-bit systems is part of that of course.

> I think the real kernel limitation, if there is one, is on file
> descriptors, not FILE objects.  The latter live in the application.

On that you are wrong, at least WRT POSIX machines. The old limit on
streams was a function of an 8-bit field in the FILE structure,
whereas every system I've ever used can allocate at least 1024
descriptors.

David