Re: [PATCH] docs: other issues with parallel BSD make (was: Re: bug#9245: FreeBSD make in concurrent mode report spurious success in automake-generated tests harness)

[Stefano Lattarini]

On Tuesday 16 August 2011, Bob Friesenhahn wrote:
> On Tue, 16 Aug 2011, Stefano Lattarini wrote:
> >>
> >> What is useful information today may become 'lore' in a few years so
> >> it would be good to add additional data so that the reader (and
> >> documentation maintainer) knows the vintage of the information.
> >>
> > That's a good point.  Do you think it would be OK to put such information
> > only in Texinfo comments for the moment, and then, as a second and later
> > step, devise a way to report it in the final manual too?  This second step
> > wouldn't be trivial IMHO, since we would need to present such version
> > information in a way that is at the same time clear, non-obtrusive and
> > complete (mabe we could take a look at how Gnulib does this?).  Finally,
> > as a third step, we might try to revisit the existing examples of bugs
> > and portability pitfalls, and try to pin-point them to a precise version
> > of the system/tools used (in case this version is not already reported).
> 
> Perhaps specifying the OS release version the issue was noticed on 
> (not all-inclusive) or the date that the issue was noticed would be 
> sufficient?  For example "FreeBSD 8.2" rather than "FreeBSD"? or 
> "FreeBSD (2011)" rather than "FreeBSD"?  The drawback of this might be 
> that someone may assume that the issues are particular to that release 
> only.
> 
> Hardly anyone reads the raw Texinfo so comments in the Texinfo do not 
> seem very useful for the common reader.  If the reader knows that the 
> current version of FreeBSD is 12.8, then he might want to check to see 
> if claims made about 8.2 (totally defunct by then) are still 
> applicable. Comments in the raw Texinfo from the authors regarding 
> versions when the issue was checked (or rechecked) would be useful.
> 
> Bob
> 
OK, given your considerations, I've updated my patch with the attached
squash-in.  The amended patch is attached too.  Let me know if it is
good to apply now.

Regards,
  Stefano

diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index b2b0a59..a0f7a55 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -19838,7 +19838,8 @@ all:
 @end example
 
 @noindent
-may output the following with @code{make -j1}:
+may output the following with @code{make -j1}, at least on NetBSD up to
+5.1 and FreeBSD up to 8.2:
 
 @example
 /
@@ -19877,17 +19878,18 @@ read:
 @c $$ @c restore font-lock
 $ @kbd{echo foo | make read}
 LINE: foo
-$ @kbd{echo foo | make -j1 read}
+$ @kbd{echo foo | make -j1 read} # NetBSD 5.1 and FreeBSD 8.2
 LINE:
 @end example
 
 @noindent
-Moreover, when FreeBSD @command{make} is run in parallel mode, it
-implements the @code{@@} and @code{-} ``recipe modifiers'' by dynamically
-modifying the active shell flags.  This behaviour has the effects of
-potentially clobbering the exit status of recipes silenced with the
address@hidden@@} modifier if they also unset the @option{errexit} shell flag,
-and of mangling the output in unexpected ways:
+Moreover, when FreeBSD @command{make} (up at least to 8.2) is run in
+parallel mode, it implements the @code{@@} and @code{-} ``recipe
+modifiers'' by dynamically modifying the active shell flags.  This
+behaviour has the effects of potentially clobbering the exit status
+of recipes silenced with the @code{@@} modifier if they also unset
+the @option{errexit} shell flag, and of mangling the output in
+unexpected ways:
 
 @example
 $ @kbd{cat Makefile}
@@ -19928,12 +19930,12 @@ standard output is connected to a tty, the recipe 
command will not be:
 $ @kbd{cat Makefile}
 all:
         @@test -t 1 && echo "Is a tty" || echo "Is not a tty"
-$ @kbd{make -j 2} # FreeBSD make
+$ @kbd{make -j 2} # FreeBSD 8.2 make
 Is not a tty
-$ @kbd{make -j 2} # NetBSD make
+$ @kbd{make -j 2} # NetBSD 5.1 make
 --- all ---
 Is not a tty
-$ @kbd{dmake -j 2} # Solaris dmake
+$ @kbd{dmake -j 2} # Solaris 10 dmake
 @var{hostname} --> 1 job
 @var{hostname} --> Job output
 Is not a tty

From a621f6fb03ee2ce45bafd882b17dafc9a8d0bdeb Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Tue, 16 Aug 2011 18:31:42 +0200
Subject: [PATCH] docs: other issues with parallel BSD make

Motivated by automake bug#9245:
 <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9245>
and FreeBSD PR bin/159730:
 <http://www.freebsd.org/cgi/query-pr.cgi?pr=159730>

* doc/autoconf.texi (Parallel Make): Document other BSD make
incompatibilities.  Reorganize the existing related documentation
accordingly.
---
 ChangeLog         |   11 ++++
 doc/autoconf.texi |  141 +++++++++++++++++++++++++++++++++++-----------------
 2 files changed, 106 insertions(+), 46 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 6ca51fd..6e36455 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2011-08-16  Stefano Lattarini  <address@hidden>
+
+       docs: other issues with parallel BSD make
+       Motivated by automake bug#9245:
+        <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9245>
+       and FreeBSD PR bin/159730:
+        <http://www.freebsd.org/cgi/query-pr.cgi?pr=159730>
+       * doc/autoconf.texi (Parallel Make): Document other BSD make
+       incompatibilities.  Reorganize the existing related documentation
+       accordingly.
+
 2011-08-08  Stefano Lattarini  <address@hidden>
 
        docs: fix minor typos
diff --git a/doc/autoconf.texi b/doc/autoconf.texi
index 7c1818d..a0f7a55 100644
--- a/doc/autoconf.texi
+++ b/doc/autoconf.texi
@@ -19826,45 +19826,10 @@ sh
 Support for parallel execution in @command{make} implementation varies.
 Generally, using GNU make is your best bet.
 
-Some make implementations (among them, FreeBSD @command{make}, NetBSD
address@hidden, and Solaris @command{dmake}), when invoked with a
address@hidden@var{N}} option, connect the standard output and standard
-error of all their child processes to pipes or temporary regular
-files.  This can lead to subtly different semantics in the behaviour
-of the spawned processes.  For example, even if the @command{make}
-standard output is connected to a tty, the recipe command will not be:
-
address@hidden
-$ @kbd{cat Makefile}
-all:
-        @@test -t 1 && echo "Is a tty" || echo "Is not a tty"
-$ @kbd{make -j 2} # FreeBSD make
-Is not a tty
-$ @kbd{make -j 2} # NetBSD make
---- all ---
-Is not a tty
-$ @kbd{dmake -j 2} # Solaris dmake
address@hidden --> 1 job
address@hidden --> Job output
-Is not a tty
address@hidden example
-
address@hidden
-On the other hand:
-
address@hidden
-$ @kbd{make -j 2} # GNU make, Heirloom make
-Is a tty
address@hidden example
-
-Furthermore, parallel runs of those @command{make} implementations will
-route standard error from commands that they spawn into their own
-standard output, and may remove leading whitespace from output lines.
-
 When NetBSD or FreeBSD @command{make} are run in parallel mode, they will
 reuse the same shell for multiple commands within one recipe.  This can
-have unexpected consequences. For example, change of directories or
-variables persist between commands:
+have various unexpected consequences.  For example, changes of directories
+or variables persist between recipes, so that:
 
 @example
 all:
@@ -19873,10 +19838,10 @@ all:
 @end example
 
 @noindent
-may output the following with @code{make -j1}:
+may output the following with @code{make -j1}, at least on NetBSD up to
+5.1 and FreeBSD up to 8.2:
 
 @example
---- all ---
 /
 value
 32235
@@ -19897,19 +19862,103 @@ value
 32239
 @end example
 
-Another consequence of this is that, if one command in a recipe uses
address@hidden 0} to indicate a successful exit, the shell will be gone
-and the remaining commands of this recipe will not be executed.
address@hidden
+Another consequence is that, if one command in a recipe uses @code{exit 0}
+to indicate a successful exit, the shell will be gone and the remaining
+commands of this recipe will not be executed.
+
+The BSD @command{make} implementations, when run in parallel mode,
+will also pass the @command{Makefile} recipes to the shell through
+its standard input, thus making it unusable from the recipes:
+
address@hidden
+$ @kbd{cat Makefile}
+read:
+        @@read line; echo LINE: $$line
address@hidden $$ @c restore font-lock
+$ @kbd{echo foo | make read}
+LINE: foo
+$ @kbd{echo foo | make -j1 read} # NetBSD 5.1 and FreeBSD 8.2
+LINE:
address@hidden example
 
-The above example also shows additional status output produced by NetBSD
address@hidden (but @emph{not} by FreeBSD @command{make}) in parallel
-mode for targets being updated.
address@hidden
+Moreover, when FreeBSD @command{make} (up at least to 8.2) is run in
+parallel mode, it implements the @code{@@} and @code{-} ``recipe
+modifiers'' by dynamically modifying the active shell flags.  This
+behaviour has the effects of potentially clobbering the exit status
+of recipes silenced with the @code{@@} modifier if they also unset
+the @option{errexit} shell flag, and of mangling the output in
+unexpected ways:
 
-You can avoid these issues by using the @option{-B} option to enable
address@hidden
+$ @kbd{cat Makefile}
+a:
+        @@echo $$-; set +e; false
+b:
+        -echo $$-; false; echo set -
+$ @kbd{make a; echo status: $?}
+ehBc
+*** Error code 1
+status: 1
+$ @kbd{make -j1 a; echo status: $?}
+ehB
+status: 0
+$ @kbd{make b}
+echo $-; echo set -
+hBc
+set -
+$ @kbd{make -j1 b}
+echo $-; echo hvB
address@hidden example
+
address@hidden
+You can avoid all these issues by using the @option{-B} option to enable
 compatibility semantics.  However, that will effectively also disable
 all parallelism as that will cause prerequisites to be updated in the
 order they are listed in a rule.
 
+Some make implementations (among them, FreeBSD @command{make}, NetBSD
address@hidden, and Solaris @command{dmake}), when invoked with a
address@hidden@var{N}} option, connect the standard output and standard
+error of all their child processes to pipes or temporary regular
+files.  This can lead to subtly different semantics in the behaviour
+of the spawned processes.  For example, even if the @command{make}
+standard output is connected to a tty, the recipe command will not be:
+
address@hidden
+$ @kbd{cat Makefile}
+all:
+        @@test -t 1 && echo "Is a tty" || echo "Is not a tty"
+$ @kbd{make -j 2} # FreeBSD 8.2 make
+Is not a tty
+$ @kbd{make -j 2} # NetBSD 5.1 make
+--- all ---
+Is not a tty
+$ @kbd{dmake -j 2} # Solaris 10 dmake
address@hidden --> 1 job
address@hidden --> Job output
+Is not a tty
address@hidden example
+
address@hidden
+On the other hand:
+
address@hidden
+$ @kbd{make -j 2} # GNU make, Heirloom make
+Is a tty
address@hidden example
+
address@hidden
+The above examples also show additional status output produced in parallel
+mode for targets being updated by Solaris @command{dmake} and NetBSD
address@hidden (but @emph{not} by FreeBSD @command{make}).
+
+Furthermore, parallel runs of those @command{make} implementations will
+route standard error from commands that they spawn into their own
+standard output, and may remove leading whitespace from output lines.
+
+
 @node Comments in Make Rules
 @section Comments in Make Rules
 @cindex Comments in @file{Makefile} rules
-- 
1.7.2.3