Re: [RFC v3 13/32] rust: use vendored-sources

[Peter Maydell]

On Tue, 7 Sept 2021 at 13:32, <marcandre.lureau@redhat.com> wrote:
>
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
>
> Most likely, QEMU will want tighter control over the sources, rather
> than relying on crates.io downloading, use a git submodule with all the
> dependencies. However, cargo --offline was added in 1.36.
>
> "cargo vendor" helps gathering and updating the dependencies.
>
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>  configure                 | 8 ++++++++
>  meson.build               | 7 ++++++-
>  .cargo/config.toml.in     | 5 +++++
>  .cargo/meson.build        | 5 +++++
>  .gitmodules               | 4 ++++
>  rust/vendored             | 1 +
>  scripts/archive-source.sh | 2 +-
>  scripts/cargo_wrapper.py  | 1 +
>  8 files changed, 31 insertions(+), 2 deletions(-)
>  create mode 100644 .cargo/config.toml.in
>  create mode 100644 .cargo/meson.build
>  create mode 160000 rust/vendored

So, this is a lot of extra code in a submodule. Historically we've
found that submodules are a colossal pain, and so I think we should
think about whether we really want to have all our rust dependencies
in a submodule forever.

I am definitely only at the beginner stage with Rust, but I think
we should have a discussion about what the different alternative
options are here, and what we want to achieve, so that we know
why we're doing this and what we're gaining from the pain...

For instance, could we instead commit Cargo.lock in git and
use that to nail down specific versions of the dependencies ?

FWIW, the "why submodules" for the C dependencies we ship
like that is basically
 * C doesn't have a package manager, so if we need a dependency that
   distros don't ship then we need to wrap it up and provide it ourselves
 * where we ship binary blobs (guest BIOS etc) we want to also ship
   the source code for those blobs
I think for Rust dependencies those don't really apply.

Overall, I think that to the extent that we can look like a "normal"
user of Rust, that's a good plan. Distros may well want to be able
to do "build against our packaged rust stuff rather than downloading
from crates.io" but I imagine they have machinery for that already;
if we act like most other Rust programs we have better chances of
not breaking that machinery.

We do already effectively do "download code when QEMU is built" --
the makefile invokes scripts/git-submodule-update which pulls
down submodule code. (Thanks to Ian for pointing out this framing
of the question.)

(I'm not personally a fan of the "download everything from crates.io"
Rust ecosystem, but it is what it is, and wishing the Rust world
worked more like a trad Linux-distro-provides-all-your-dependencies
isn't, alas, going to make it so :-))

thanks
-- PMM