[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug #22923] option to prevent "interspersed" output in parallel bui

From: Howard Chu
Subject: Re: [bug #22923] option to prevent "interspersed" output in parallel builds
Date: Thu, 26 Mar 2009 15:30:06 -0700
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; rv:1.9.1b3pre) Gecko/20090317 SeaMonkey/2.0a1pre Firefox/3.0.3

Brian Kustel wrote:

Follow-up Comment #3, bug #22923 (project make):

Thanks ipjrest and psmith for both replies. They are very appreciated.

Responding to ipjrest's comments:
I agree some commands are long-running and that if "watching" progress is
important, the proposed behavior may not be desireable. This is one reason I
suggested the proposed new behavior be optional.

1) Line-buffering: While it would help in some cases, not intermixing lines
from different processes is only partially helpful. The fact that a "single
line" from process A might be interrupted by a line from process B is my
concern here. For example, instead of seeing
   Process A Fragment 1 was emitted and then fragment 2 was emitted.
   Process B Message
one might see
   A: Process A Fragment 1 was emitted
   B: Process B Message
   A: and then fragment 2 was emitted.
if the line for process A is emitted in 2 separate write operations.
2) The VS line prefixing would be OK. In fact I like this idea. But it still
has the problem that single lines can be broken if the tool being run
sometimes emits lines "in pieces". I have noticed that gcc has this problem in
some cases (at least the gcc version I am currently using, 4.1.0).

Responding to psmith's comments:
Thanks for pointing out that make is not multithreaded. I guess I was
assuming it was. I agree this complicates matters somewhat. Also, I see your
point about my original stdout/stderr "marking" idea. In both cases I assumed
make would have either an auxilliary thread or its main thread doing these
tasks but I now realize that this is not how make currently works.

But I think something along the lines of your "special variable" idea could
be employed in a generic fashion. Perhaps this is what you intended your
second paragraph below to imply? Specifically, the proposed option could cause
all "commands" to have their output piped to a standard tool which stores
(its) stdin to a temp file and then writes it synchronously to stdout after
stdin is closed. Also, I think it could be reasonable to allow for an option
to either combine stdout/stderr here or to pipe them to separate processes (so
that stdout/stderr could be atomically emitted to stdout/stderr

While I have nothing against the "special variable" idea, I'd very much like
to see a way to accomplish this via a make command-line option and the
variable idea sounded as if it might require a makefile modification.

Just my 2 cents, the Alliant parallel make (which inspired my design of the make -j jobserver) also redirected all of the spawned processes' stdio to a set of pipes, so that all of their output was prefixed with |jobnum|. It was quite nice, but I never felt it was important enough to duplicate that behavior for gmake.

  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/

reply via email to

[Prev in Thread] Current Thread [Next in Thread]