[bug#52555] [RFC PATCH 0/3] Decentralized substitute distribution with ERIS

[Ludovic Courtès]

Hello!

pukkamustard <pukkamustard@posteo.net> skribis:

> I'm very happy to present a V3 of a proposal towards decentralizing substitute
> distribution with ERIS. An initial version [1] and a V2 [2] are now almost a
> years old!

Woohoo! 👍

> The idea is to use ERIS (Encoding for Robust Immutable Storage) [3] to allow
> more decentralized substitute distribution. ERIS defines an encoding of
> content into uniformly sized, encrypted and content-addressed blocks
> (32KiB). The content can be decoded from the blocks given a short identifier
> called the read capability. It allows a network-optimized form of
> content-addressing.
>
> Blocks can be transported over many different transport protocols, such as
> HTTP, CoAP, GNUnet, IPFS or a SD card sent via pigeons. Only the read
> capability must be transmited securely. Given an authentic read capability,
> the content can be decoded correctly from blocks that might have been
> transported over unreliable (and untrusted) protocols.

Neat!  (Do we have an implementation yet for SD-cards-over-pigeons?)

> For Guix, substitutes (Nar files) are encoded using ERIS and the read
> capability is added to the signed Narinfos. The read capability published in
> the Narinfo can then be used to decode the Nar from blocks that are fetched
> from many peers over many different protocols.
>
> This version of the patches allows blocks to be fetched over HTTP and IPFS.

Nice.

> A summary of this patch series:
>
> - Use the stable version of the ERIS encoding (version 1.0.0)
> - Add two fields to Narinfos (ERIS and ERISFormat)
> - Store blocks of published substitutes in a local block store (in
>   `/var/guix/eris`)
> - Add an endpoint for resolving ERIS blocks over HTTP (a la RFC 2169)
> - Use ERIS when fetching substitutes
> - Use IPFS for de-referencing ERIS blocks
>
> Testing procedure is a bit tedious, but described in the V2:
> https://issues.guix.gnu.org/52555#8-lineno16

OK.  There are still I guess a number of unknowns, including the cost of
ERIS support for publishers (how much disk space ‘guix publish’ will use
for blocks, how much CPU is needed to compute those blocks, bandwidth
usage over IFPS/HTTP) and for consumers (performance of substitution
over HTTP+ERIS or IPFS+ERIS CPU-wise and bandwidth-wise).

To address that, we’ll need to make it easy to test, and to make it easy
to disable it if things don’t work as expected.

(Perhaps this is already the case, I’m thinking out loud.)

> Thanks for making it so far! :) I'd be very happy for your thoughs and
> ideas. There might also be opportunity to discuss these ideas at Guix Days in
> February.

Definitely!  I think it would be great if you could present what you’re
up to and maybe make a quick demo.

Thanks for the great news to begin the year!

Ludo’.