[GSoC 23] distributed substitutes, cost of storage

[Attila Lendvai]

welcome on board Anand!


> In case a user requests for a substitute and there is a missing
> block in the decoding process, a HTTP request for block would sent
> to the substitute server and the server will encode the
> corresponding block in real time and push it back into the
> network. The block will be searched again and retrieved.


something to consider here: whose responsibility should it be that a block, 
that is missing from a p2p network, is (re-)uploaded there? the clients? or the 
current substitute server?

my gut instinct says that it's better if the clients do the (re-)upload of the 
blocks.

in this architecture the substitute server is just another storage mechanism 
along the other storage backends (although with a different reliability 
characteristics), and it's the clients that are doing the 
mirroring/spreading/distribution of the blocks among the various backends. the 
clients of course will/should keep the current substitute servers at the bottom 
of their list of backends in their configuration.

this way the load is distributed, and we don't need to add (too much) extra 
complexity to the substitute server codebase, and the actors are less tightly 
coupled.

it's another question whether this mirroring should be enabled by default in 
the clients. probably it shouldn't, and the project infrastructure should be 
running clients where it is turned on. altruistic third parties could also 
enable this mirroring feature, and donate their bandwidth/resources.

there's an issue with this, though:

some p2p storage backends will require some form of payment/credentials to use 
their resources. arguably, all p2p storage networks that will survive into the 
future will have some mechanism to limit the infinite abuse of their resources. 
it is to be researched how these payment mechanisms work on the various p2p 
networks, and whether it is possible that the Guix project pays for the storage 
globally, and then the random clients will have the necessary credentials to 
(re-)upload the missing blocks.

this architecture shouldn't be impossible, because the content is authenticated 
by its hash, and if the payment/authorization mechanism is based on the hashes 
of the blocks (probably), then any client could (re-)upload a missing block 
that was already paid for.

i'll look into this, especially in the context of Swarm.

meta: i think such specific discussions should be kept off-list, but the 
financing of the storage fees is probably something that should be known about 
more widely.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
Every lie is a debt to the truth.