Re: [GSoC 23] distributed substitutes, cost of storage

[Attila Lendvai]

> Also I didn't really think about the point about having to pay for
> the p2p services at some point of time.


a quick note here: i forgot to mention that e.g. the Swarm Foundation has 
programs for supporting opensource projects. so, chances are high that the 
storage needs for Guix would be paid for by the foundation.


> In this case we will have to pay for the storage of substitutes both
> on the p2p storage backend as well as for storage in the substitute
> server am I right?


well, the substitute servers are currently already operated (and paid for) by 
the Guix team. i don't think that p2p storage solutions have reached a point of 
maturity that we could rely on them alone. there should definitely be some time 
where both infrastructures are running in parallel. somewhere down the road a 
choice could be made to stop running the current substitute servers, but we are 
far away from that.

also, running swarm nodes that serve the network can earn money. so, the cost 
of running enough swarm nodes to pay for the storage needs of Guix on the swarm 
network should be in the same ballpark of the costs of running the current 
substitute servers. the storage price will be market based (this part is just 
being rolled out on the live network), so it's reasonable to expect that people 
will fire up nodes if the storage price go well above the VPS costs.

and not all p2p storage networks are made equal. e.g. IPFS is only a registry 
of who is serving what. if you want to keep your data alive on IPFS, then you 
need to run some nodes and make sure that they are serving the content that you 
care about... and bear the costs of running these nodes. i.e. the DoS attack 
surface of IPFS is much smaller. (IPFS stores only the metadata in the DHT 
(i.e. where is what), while Swarm stores there the data itself -- they are 
different architectures with different features)

(i need to learn more about GNUnet)


> So ideally we will want to eliminate the usage of these substitute
> servers and shift totally to p2p services and in this case we will
> have to shift the responsibility of re-uploading the blocks to the
> clients itself.


yep, that's my way of thinking, too.

note that 'client' here has two meanings:

 1) some part of the codebase

 2) a program that is running on the computers of the Guix users

i was using it in the first sense.

without a functional Web of Trust solution, the Guix team will have to run 
nodes that compile packages, sign them with their PGP keys, and make them 
available somewhere. currently it's published through a HTTP based service that 
we call 'substitute servers'. this GSoC project is about adding more storage 
backends.

but those backends don't solve the problem that the Guix users need to trust 
someone with a private key who compiles and signs packages, regadless of the 
transport mechanism that gets the packages to the clients.

i can dream about a future where there's a social network that is based on 
digital signatures and encryption, and my Guix client authorizes compiled 
binaries based on some weighted transitive closure of signatures of my trusted 
peers... but we are not there yet. for now it's either trusting the Guix team's 
signature, or setting up your own substitute servers and build workers (or 
trusting someone else, but i'm not aware of any third party offering 
substitutes).


> Also if we dont keep the re-uploading blocks option as default for
> the users, won't users usually not choose to enable it? Maybe we can
> keep it on as default and resource conscious users can choose to
> turn it off? Please let me know your thoughts on these points and I
> will change the implementation point of my proposal accordingly.


first, it's a philosophical question: i value consensual relationships, and 
that implies that the other party is well informed. and clogging someone's 
network bandwidth is not an expected behavior from installing a linux 
distribution.

there's also a technical issue: these p2p backends will need to be configured. 
i have my doubts that we could ship a default config with Guix with which these 
p2p backends could just work out of the box, but... let's hope that i'm wrong!

and then there's the issue of payments: it's not obvious that a random client 
can just upload binaries into a p2p storage network. on some p2p networks 
someone needs to pay for that, and i don't yet understand well enough how the 
data is authorized through payments (they are called postage stamps in swarm).

a good, high level comparison of p2p storage solutions would be useful.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“There is something in the human spirit that will survive and prevail, there is 
a tiny and brilliant light burning in the heart of man that will not go out no 
matter how dark the world becomes.”
        — Leo Tolstoy (1828–1910)