Re: When substitute download + decompression is CPU-bound

[Guillaume Le Vaillant]

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Hi Ludo!
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> I suppose a possible agenda would be:
>>
>>   1. Start providing zstd susbstitutes anytime.  However, most clients
>>      will keep choosing lzip because it usually compresses better.
>>
>>   2. After the next release, stop providing lzip substitutes and provide
>>      only gzip + zstd-19.
>>
>> This option has the advantage that it wouldn’t break any installation.
>
> But why would we keep gzip since it offers no benefits compared to zstd?
> It feels like continuing to carry a (huge) burden forever...
>
> Besides, dropping Lzip seems like a step backward in my opinion.  Users
> with lower bandwidth (or simply further away from Berlin) will be
> impacted a lot.
>
> I would opt for dropping gzip instead, only to keep zstd-19 and lzip-9
> (possibly plzip-9 if we update the bindings).
>
>> It’s not as nice as the ability to choose a download strategy, as we
>> discussed earlier, but implementing that download strategy sounds
>> tricky.
>
> If the user can choose their favourite substitute compression, I believe
> it's usually enough since they are the best judge of their bandwidth /
> hardware requirements.
>
> Wouldn't this simple enough?

Here are a few numbers for the installation time in seconds (download
time + decompression time) when fetching 580 MB of substitutes for
download speeds between 0.5 MB/s and 20 MB/s.

| Download speed | gzip -9 | lzip -9 | zstd -19 |
|----------------+---------+---------+----------|
|            0.5 |     287 |     151 |      181 |
|            1.0 |     144 |      78 |       91 |
|            1.5 |      97 |      54 |       61 |
|            2.0 |      73 |      42 |       46 |
|            2.5 |      59 |      35 |       37 |
|            3.0 |      49 |      30 |       31 |
|            3.5 |      42 |      27 |       26 |
|            4.0 |      37 |      24 |       23 |
|            4.5 |      33 |      22 |       21 |
|            5.0 |      30 |      21 |       19 |
|            5.5 |      28 |      19 |       17 |
|            6.0 |      25 |      18 |       16 |
|            6.5 |      24 |      17 |       14 |
|            7.0 |      22 |      17 |       14 |
|            7.5 |      21 |      16 |       13 |
|            8.0 |      20 |      15 |       12 |
|            8.5 |      18 |      15 |       11 |
|            9.0 |      18 |      14 |       11 |
|            9.5 |      17 |      14 |       10 |
|           10.0 |      16 |      13 |       10 |
|           11.0 |      15 |      13 |        9 |
|           12.0 |      14 |      12 |        8 |
|           13.0 |      13 |      12 |        8 |
|           14.0 |      12 |      11 |        7 |
|           15.0 |      11 |      11 |        7 |
|           16.0 |      11 |      11 |        6 |
|           17.0 |      10 |      10 |        6 |
|           18.0 |      10 |      10 |        6 |
|           19.0 |       9 |      10 |        5 |
|           20.0 |       9 |      10 |        5 |

When the download speed is lower than 3.5 MB/s, Lzip is better, and
above that speed Zstd is better.

As Gzip is never the best choice, it would make sense to drop it, even
if we have to wait a little until everyone has updated their Guix daemon
to a version with at least Lzip support.

I think there are many people (like me) with a download speed slower
than 3 MB/s, so like Pierre I would prefer keeping "lzip -9" and
"zstd -19".

signature.asc
Description: PGP signature