Re: Avoiding the garbage collector for the Guix Data Service

[Christopher Baines]

Ludovic Courtès <address@hidden> writes:

> Hi Christopher,
>
> Christopher Baines <address@hidden> skribis:
>
>> The Guix Data Service has Guix compute lots of derivations, and then it
>> loads the data in to a database. I'm having some issues with the garbage
>> collector though, as sometimes it removes data from the disk before I've
>> had a chance to read it (at least I think this is what's happening, I
>> think this job failed for this reason [1]).
>>
>> 1: http://data.guix.gnu.org/job/10638
>>
>> I'm not quite sure what the best strategy is to combat this? I had a
>> look, and I couldn't see how to acquire a lock to prevent the garbage
>> collector from running. I thought about registering thousands of gc
>> roots, but that seems to just move the race a bit, as there's still a
>> period between generating the derivation and registering the root where
>> it can be deleted.
>>
>> Any ideas?
>
> At the Guix Days we discussed using ‘add-temp-root’ and similar to
> ensure that .drv files are at least GC-protected for the duration of the
> session (connection to the daemon).
>
> Does that address your problem?

I think so, it at least seems to greatly reduce the chance the
derivations get removed. I made the Guix Data Service change here [1],
and I haven't seen the issue since.

1: 
https://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=f59354ed23a9c50e43b7aaaeaddb5365feda4e29

Thanks,

Chris

signature.asc
Description: PGP signature