[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[bug #47392] should document --jobserver-fds and the jobserver pipe

From: Manoj Srivastava
Subject: [bug #47392] should document --jobserver-fds and the jobserver pipe
Date: Fri, 11 Mar 2016 18:13:35 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36


                 Summary: should document --jobserver-fds and the jobserver
                 Project: make
            Submitted by: srivasta
            Submitted on: Fri 11 Mar 2016 12:13:34 PM CST
                Severity: 3 - Normal
              Item Group: Documentation
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
       Component Version: SCM
        Operating System: None
           Fixed Release: None
           Triage Status: None




   This enhancement request was filed by a Debian user. Please retain a CC to
address@hidden to ensure that the Debian BTS retains a copy
of your response.

trying to build a program that, like gcc's LTO linker, can
integrate with make's jobserver.  gcc's LTO linker can run multiple 
threads in parallel, and can use make's jobserver pipe to manage how
many threads it runs at a time.  If you pass -flto=jobserv to gcc, and
tell make to treat gcc as a sub-make, gcc will respect make's jobserver
pipe when spawning its threads.

That seems like a really good idea for any program used as part of a
build process and capable of running multiple parallel threads for
performance.  And conversely, any other build system capable of spawning
off other programs and running multiple jobs in parallel could follow
the same convention, to interoperate with make.

So, I wanted to learn how to build a program that could interoperate
with make like GCC did.  I figured out most of the details, but the
interface has some subtleties.  For instance, when make spawns a
sub-make (or any other program launched with '+' and acting like a
sub-make), the sub-make can always run one "free" job in addition to one
job per byte read from the jobserver pipe, and should not write back a
byte when that "free" job completes, because the parent make (or some
ancestor make) will have consumed a byte (or used its free job) to run
the sub-make.

This kind of subtlety makes the interface worth documenting.  Some
third-party articles document the jobserver, but make's own manpage
ought to do so as well.

http://make.mad-scientist.net/papers/jobserver-implementation/ provides
full documentation.

For the manpage, I'd suggest a few additions.

First, the documentation for -j should add a sentence "When make invokes
a sub-make, all instances of make will coordinate to run the specified
number of jobs at a time; see PARALLEL MAKE AND THE JOBSERVER for

Second, in the options section right after -j, add a single entry
"--jobserver-fds R,W" "Internal option make uses to pass the jobserver
pipe read and write file descriptor numbers to sub-makes; see PARALLEL

Finally, the manpage should include that referenced section "PARALLEL
MAKE AND THE JOBSERVER", giving a couple of paragraphs explaining the
jobserver pipe (and interaction with '+', $(MAKE), and $(MAKEFLAGS)).



Reply to this item at:


  Message sent via/by Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]