[bug#58035] sync-before-registering is false, possibly the cause of empty files in the store

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> On 04-10-2022 09:52, Ludovic Courtès wrote:
>> Hi,
>> [...]
>>> However, currently sync-before-registering is set to 'false' AFAICT.
>>> I think this might be the cause of bugs like
>>> <https://issues.guix.gnu.org/58013> (‘Can't use "guix pull"’), and
>>> maybe <https://issues.guix.gnu.org/57838> (‘failing to boot, probably
>>> due to guix gc’).
>> It might be a factor, combined with the fact that the file system
>> was
>> not properly unmounted (power outage or similar).
>> However, calling sync(2) for each store item is going to be
>> expensive.
>> Recursive fsync/fdatasync calls are also likely to be too expensive (see
>> <https://issues.guix.gnu.org/55707> for a concrete example of the cost
>> on a spinning disk).
>> Thoughts?
>
> Debian uses fsync (going by https://wiki.debian.org/Teams/Dpkg/FAQ),
> and even though that according to that FAQ dpkg can be slow, in my
> experience it wasn't too bad.  Also, having to investigate store
> corruption and how to fix it is a form of slowness, especially when it
> fails or you don't have the technical expertise and consequentially
> you need to reinstall (losing old non-back-upped work).

Having not experience corruption issues as in the bug you mention, I’m
tempted to minimize the problem, but it’s true that we get such reports
from time to time.

> 'sync' seems relatively inexpensive to me, compared to the time
> required for building a package and even more inexpensive compared to
> the cost of debugging store corruption:

That’s not a fair comparison.  :-) Imagine, you run reconfigure/upgrade;
that downloads tens to hundreds of store items.  Calling sync(2) after
each item may be hardly noticeably on an SSD, but I bet it’s going to be
super expensive on an HDD.  (In the syslogd case, each fsync(2) call—not
even sync(2)—would cause pauses of several 100s of ms.)

Maybe a good test would be to run a daemon on an “average” HDD with
sync-before-registering = true and to run ‘perf timechart record’ while
it’s fetching a large number of substitutes.  That way we’d have
concrete data to talk about.

Any takers?  :-)

Ludo’.