[Qemu-commits] [qemu/qemu] e0f372: Makefile: Ensure we don't run Sphinx in parallel f...

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: e0f3728d819001f4e2ae0dd6a77ca29acfdc04d4
      
https://github.com/qemu/qemu/commit/e0f3728d819001f4e2ae0dd6a77ca29acfdc04d4
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M Makefile
    M rules.mak

  Log Message:
  -----------
  Makefile: Ensure we don't run Sphinx in parallel for manpages

Sphinx will corrupt its doctree cache if we run two copies
of it in parallel. In commit 6bda415c10d966c8d3 we worked
around this by having separate doctrees for 'html' vs 'manpage'
runs. However now that we have more than one manpage produced
from a single manual we can run into this again when trying
to produce the two manpages.

Use the trick described in 'Atomic Rules in GNU Make'
https://www.cmcrossroads.com/article/atomic-rules-gnu-make
to ensure that we only run the Sphinx manpage builder once
for each manual, even if we're producing several manpages.
This fixes doctree corruption in parallel builds and also
avoids pointlessly running Sphinx more often than we need to.

(In GNU Make 4.3 there is builtin support for this, via
the "&:" syntax, but we can't wait for that to be available
in all the distros we support...)

The generic "one invocation for multiple output files"
machinery is provided as a macro named 'atomic' in rules.mak;
we then wrap this in a more specific macro for defining
the rule and dependencies for the manpages in a Sphinx
manual, to avoid excessive repetition.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Tested-by: Alex Bennée <address@hidden>
Message-id: address@hidden


  Commit: b0cecc0d65ec7c4c06cfd987e79eab1a0afee39b
      
https://github.com/qemu/qemu/commit/b0cecc0d65ec7c4c06cfd987e79eab1a0afee39b
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M scripts/hxtool

  Log Message:
  -----------
  hxtool: Support SRST/ERST directives

We want to add support for including rST document fragments
in our .hx files, in the same way we currently have texinfo
fragments. These will be delimited by SRST and ERST directives,
in the same way the texinfo is delimited by STEXI/ETEXI.
The rST fragments will not be extracted by the hxtool
script, but by a different mechanism, so all we need to
do in hxtool is have it ignore all the text inside a
SRST/ERST section, with suitable error-checking for
mismatched rST-vs-texi fragment delimiters.

The resulting effective state machine has only three states:
 * flag = 0, rstflag = 0 : reading section for C output
 * flag = 1, rstflag = 0 : reading texi fragment
 * flag = 0, rstflag = 1 : reading rST fragment
and flag = 1, rstflag = 1 is not possible. Using two
variables makes the parallel between the rST handling and
the texi handling clearer; in any case all this code will
be deleted once we've converted entirely to rST.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Richard Henderson <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Message-id: address@hidden


  Commit: 6803d6e9a7d951cd65bbdd88bfe6f8a921c95f54
      
https://github.com/qemu/qemu/commit/6803d6e9a7d951cd65bbdd88bfe6f8a921c95f54
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M docs/conf.py
    A docs/sphinx/hxtool.py

  Log Message:
  -----------
  docs/sphinx: Add new hxtool Sphinx extension

Some of our documentation includes sections which are created
by assembling fragments of texinfo from a .hx source file into
a .texi file, which is then included from qemu-doc.texi or
qemu-img.texi.

For Sphinx, rather than creating a file to include, the most natural
way to handle this is to have a small custom Sphinx extension which
reads the .hx file and process it.  So instead of:
 * makefile produces foo.texi from foo.hx
 * qemu-doc.texi says '@include foo.texi'
we have:
 * qemu-doc.rst says 'hxtool-doc:: foo.hx'
 * the Sphinx extension for hxtool has code that runs to handle that
   Sphinx directive which reads the .hx file and emits the appropriate
   documentation contents

This is pretty much the same way the kerneldoc extension works right
now. It also has the advantage that it should work for third-party
services like readthedocs that expect to build the docs directly with
sphinx rather than by invoking our makefiles.

In this commit we implement the hxtool extension.

Note that syntax errors in the rST fragments will be correctly
reported to the user with the filename and line number within the
hx file.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Tested-by: Alex Bennée <address@hidden>
Message-id: address@hidden


  Commit: 195b51094cd95247b75248414e6aee38cf6fb0f6
      
https://github.com/qemu/qemu/commit/195b51094cd95247b75248414e6aee38cf6fb0f6
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M qemu-img-cmds.hx

  Log Message:
  -----------
  qemu-img-cmds.hx: Add rST documentation fragments

Add the rST versions of the documentation fragments.
Once we've converted qemu-img.texi to rST we can delete
the texi fragments; for the moment we leave them in place.

(Commit created with the aid of emacs query-replace-regexp
from "@var{\([^}]*\)}" to "\,(upcase \1))".)

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Richard Henderson <address@hidden>
Message-id: address@hidden


  Commit: e13c59fa4414215500e66c2604292caa6e80d57e
      
https://github.com/qemu/qemu/commit/e13c59fa4414215500e66c2604292caa6e80d57e
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile
    M docs/interop/conf.py
    M docs/interop/index.rst
    A docs/interop/qemu-img.rst
    M qemu-doc.texi
    R qemu-img.texi

  Log Message:
  -----------
  qemu-img: Convert invocation documentation to rST

The qemu-img documentation is currently in qemu-nbd.texi in Texinfo
format, which we present to the user as:
 * a qemu-img manpage
 * a section of the main qemu-doc HTML documentation

Convert the documentation to rST format, and present it to the user as:
 * a qemu-img manpage
 * part of the interop/ Sphinx manual

The qemu-img rST document uses the new hxtool extension
to handle pulling rST fragments out of qemu-img-cmds.hx.

The documentation of the various options and commands is rather
muddled, with some options being described inside the relevant
command description and some in a more general section near the start
of the manual.  All the command synopses are replicated in the .hx
file and then again in the manual.  A lot of text is also duplicated
in the qemu-img.c code for the help text.  I have not attempted to
deal with any of this, but have simply transposed the existing
structure into rST.

As usual, there are some minor formatting changes but no
textual changes, except that as with one or two other conversions
I have dropped the 'see also' section since it's not very
informative and looks odd in the HTML.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Tested-by: Alex Bennée <address@hidden>
Message-id: address@hidden


  Commit: 96ddefbc871895ed14b422d3571d61985165cf2a
      
https://github.com/qemu/qemu/commit/96ddefbc871895ed14b422d3571d61985165cf2a
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M qemu-img-cmds.hx

  Log Message:
  -----------
  qemu-img-cmds.hx: Remove texinfo document fragments

Now the qemu-img documentation has been converted to rST, we can
remove the texinfo document fragments from qemu-img-cmds.hx, as
they are no longer used.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Richard Henderson <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Message-id: address@hidden


  Commit: 605ffebb2e206d9dec746ceed0de7be561b7354b
      
https://github.com/qemu/qemu/commit/605ffebb2e206d9dec746ceed0de7be561b7354b
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile
    M docs/interop/conf.py
    M docs/interop/index.rst
    A docs/interop/qemu-trace-stap.rst
    R scripts/qemu-trace-stap.texi

  Log Message:
  -----------
  scripts/qemu-trace-stap: Convert documentation to rST

The qemu-trace-stap documentation is currently in
scripts/qemu-trace-stap.texi in Texinfo format, which we
present to the user as:
 * a qemu-trace-stap manpage
 * but not (unusually for QEMU) part of the HTML docs

Convert the documentation to rST format that lives in
the docs/ subdirectory, and present it to the user as:
 * a qemu-trace-stap manpage
 * part of the interop/ Sphinx manual

There are minor formatting changes to suit Sphinx, but no
content changes.

Signed-off-by: Peter Maydell <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Tested-by: Alex Bennée <address@hidden>
Message-id: address@hidden


  Commit: 78813586b04e89639754cfdcef23802dc9f54ff4
      
https://github.com/qemu/qemu/commit/78813586b04e89639754cfdcef23802dc9f54ff4
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile
    M docs/interop/conf.py
    M docs/interop/index.rst
    A docs/interop/virtfs-proxy-helper.rst
    R fsdev/virtfs-proxy-helper.texi

  Log Message:
  -----------
  virtfs-proxy-helper: Convert documentation to rST

The virtfs-proxy-helper documentation is currently in
fsdev/qemu-trace-stap.texi in Texinfo format, which we
present to the user as:
 * a virtfs-proxy-helper manpage
 * but not (unusually for QEMU) part of the HTML docs

Convert the documentation to rST format that lives in
the docs/ subdirectory, and present it to the user as:
 * a virtfs-proxy-helper manpage
 * part of the interop/ Sphinx manual

There are minor formatting changes to suit Sphinx, but no
content changes. In particular I've split the -u and -g
options into each having their own description text.

Signed-off-by: Peter Maydell <address@hidden>
Acked-by: Greg Kurz <address@hidden>
Message-id: address@hidden


  Commit: f31160c7d1b89cfb4dd4001a23575b42141cb0ec
      
https://github.com/qemu/qemu/commit/f31160c7d1b89cfb4dd4001a23575b42141cb0ec
  Author: Peter Maydell <address@hidden>
  Date:   2020-02-03 (Mon, 03 Feb 2020)

  Changed paths:
    M MAINTAINERS
    M Makefile
    M docs/conf.py
    M docs/interop/conf.py
    M docs/interop/index.rst
    A docs/interop/qemu-img.rst
    A docs/interop/qemu-trace-stap.rst
    A docs/interop/virtfs-proxy-helper.rst
    A docs/sphinx/hxtool.py
    R fsdev/virtfs-proxy-helper.texi
    M qemu-doc.texi
    M qemu-img-cmds.hx
    R qemu-img.texi
    M rules.mak
    M scripts/hxtool
    R scripts/qemu-trace-stap.texi

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-docs-20200203' into 
staging

docs:
 * Fix Makefile concurrency bug where we could run Sphinx twice
   in parallel on the same manual (which makes it crash)
 * Support handling hxtool doc fragments for rST manuals
 * Convert qemu-img docs to rST
 * Convert qemu-trace-stap docs to rST
 * Convert virtfs-proxy-helper docs to rST

# gpg: Signature made Mon 03 Feb 2020 11:11:44 GMT
# gpg:                using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg:                issuer "address@hidden"
# gpg: Good signature from "Peter Maydell <address@hidden>" [ultimate]
# gpg:                 aka "Peter Maydell <address@hidden>" [ultimate]
# gpg:                 aka "Peter Maydell <address@hidden>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-docs-20200203:
  virtfs-proxy-helper: Convert documentation to rST
  scripts/qemu-trace-stap: Convert documentation to rST
  qemu-img-cmds.hx: Remove texinfo document fragments
  qemu-img: Convert invocation documentation to rST
  qemu-img-cmds.hx: Add rST documentation fragments
  docs/sphinx: Add new hxtool Sphinx extension
  hxtool: Support SRST/ERST directives
  Makefile: Ensure we don't run Sphinx in parallel for manpages

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/035b21977ce1...f31160c7d1b8