[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [bug #22923] option to prevent "interspersed" output in parallel bui
Re: [bug #22923] option to prevent "interspersed" output in parallel builds
Thu, 26 Mar 2009 15:30:06 -0700
Mozilla/5.0 (X11; U; Linux x86_64; rv:1.9.1b3pre) Gecko/20090317 SeaMonkey/2.0a1pre Firefox/3.0.3
Brian Kustel wrote:
Follow-up Comment #3, bug #22923 (project make):
Thanks ipjrest and psmith for both replies. They are very appreciated.
Responding to ipjrest's comments:
I agree some commands are long-running and that if "watching" progress is
important, the proposed behavior may not be desireable. This is one reason I
suggested the proposed new behavior be optional.
1) Line-buffering: While it would help in some cases, not intermixing lines
from different processes is only partially helpful. The fact that a "single
line" from process A might be interrupted by a line from process B is my
concern here. For example, instead of seeing
Process A Fragment 1 was emitted and then fragment 2 was emitted.
Process B Message
one might see
A: Process A Fragment 1 was emitted
B: Process B Message
A: and then fragment 2 was emitted.
if the line for process A is emitted in 2 separate write operations.
2) The VS line prefixing would be OK. In fact I like this idea. But it still
has the problem that single lines can be broken if the tool being run
sometimes emits lines "in pieces". I have noticed that gcc has this problem in
some cases (at least the gcc version I am currently using, 4.1.0).
Responding to psmith's comments:
Thanks for pointing out that make is not multithreaded. I guess I was
assuming it was. I agree this complicates matters somewhat. Also, I see your
point about my original stdout/stderr "marking" idea. In both cases I assumed
make would have either an auxilliary thread or its main thread doing these
tasks but I now realize that this is not how make currently works.
But I think something along the lines of your "special variable" idea could
be employed in a generic fashion. Perhaps this is what you intended your
second paragraph below to imply? Specifically, the proposed option could cause
all "commands" to have their output piped to a standard tool which stores
(its) stdin to a temp file and then writes it synchronously to stdout after
stdin is closed. Also, I think it could be reasonable to allow for an option
to either combine stdout/stderr here or to pipe them to separate processes (so
that stdout/stderr could be atomically emitted to stdout/stderr
While I have nothing against the "special variable" idea, I'd very much like
to see a way to accomplish this via a make command-line option and the
variable idea sounded as if it might require a makefile modification.
Just my 2 cents, the Alliant parallel make (which inspired my design of the
make -j jobserver) also redirected all of the spawned processes' stdio to a
set of pipes, so that all of their output was prefixed with |jobnum|. It was
quite nice, but I never felt it was important enough to duplicate that
behavior for gmake.
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/