Make, MAKE_TERMOUT, color escape sequences, TTYs, and PTYs

[Josh Triplett]

Thank you very much for the new "Integrating make" chapter of the GNU
Make manual; I really appreciate the documentation for the jobserver.

The new "Terminal Output" section
(https://www.gnu.org/software/make/manual/make.html#Terminal-Output)
documents variables "MAKE_TERMOUT" and "MAKE_TERMERR", which indicate
that programs should assume output to the corresponding file descriptor
goes to a terminal even though isatty returns false on that fd.

However, if programs start observing those variables, that seems highly
likely to lead to potential breakage in makefiles, for one key reason:
those variables remain in the environment even for programs run with
stdout/stderr redirected or piped.

Suppose a program that defaulted to --color=auto started checking
MAKE_TERMOUT, and when set, the autodetection would assume it can safely
print color even if output doesn't go to a TTY.  Then, deep inside a
makefile (or inside a script invoked from that makefile), some command
includes a pipeline based on that program, passing its output through a
filter like sed or awk, or writing it to a file used as part of the
build proces.  The program would then print color escape sequences to
the pipeline or built file, almost certainly not what the caller
expected.

I don't see any way a program checking MAKE_TERMOUT could avoid this
scenario.  If its stdout refers to a pipe, the program has no way of
knowing if that pipe matches the one make provided, or a shell pipeline.

I do have a proposal for an alternative that would address this problem,
and avoid the need for programs to override isatty when autodetecting if
they can use terminal escape sequences.

What if make had an option to collect output via PTYs, rather than
pipes?  This option would only work on systems that support PTYs, but
almost every modern system provides such support (as terminal emulators
typically require it).  (And on a system that doesn't have such support,
users would just lose out on color output and similar.)

If make's own stdout/stderr refers to a PTY, make could create PTYs in
place of pipes, collect output that way, and synchronize it to its own
stdout/stderr as it does now.  The programs run this way could call
isatty as normal, and would find out that their output does indeed refer
to a TTY.  And running a command with its output redirected or piped
would then make its output *not* a TTY, so that case would work
automatically as well.

Does this sound like a plausible solution?

- Josh Triplett