[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GC strategy on clusters
From: |
Ludovic Courtès |
Subject: |
Re: GC strategy on clusters |
Date: |
Thu, 01 Apr 2021 15:43:45 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
>> It depends. A practical use case I have in mind: you run experiments,
>> you submit a paper including its results, you get initial reviews months
>> later, and even later it’s published and you get to present it. At that
>> point, you want to answer questions and to reproduce it. 4–6 months is
>> not a lot in that context.
>
> To add another data point. Even it is hard to have a good overview, I
> would say the average is 2-4 years for a typical project. It is hard
> because currently all is not done with the same tools for the same
> task. For instance, some data is cleaned with some tools, then an
> partial analysis is done, months later another data is added so
> another partial analysis with probably different tools, then months
> later a full toolsuite as Bioconductor (or whatever) is updated and
> some analysis are re-done. The final publication is a mix of all over
> the 2-4 years project with details at various level.
Right.
>> Good points. Hopefully “sources go missing” can soon be considered
>> addressed. Really, failing TLS tests is the most worrisome issue to me
>> because we don’t have any idea on how to address it systematically.
>
> By "soon", you mean the bricks are there and it is missing to glue
> them together. From my opinion, some details need to be addressed to
> have a full end-to-end sources fallback, since evil is hidden inside
> the details. ;-)
True! I’m really hopeful about the Disarchive/SWH combination, together
with <https://guix.gnu.org/sources.json>. Of course we’ll have to
monitor that, and we can expect bumps on the road :-), but at least we
have a plan.
> About the TLS, you proposed to setup a machine ahead of clock. Maybe
> it is worth to try.
Oh sure, that trick definitely works. But it’s a terrible hack, and it
means that, by default, people will just fail to build the package.
> Well, when we discussed the '--list-profiles', it was initially for my
> personal purposes. Then, I have tried to use it to monitor the few
> users that I have. Well, in Biology they have the concept of "-80
> fridge". It is a big and very cold fridge where you keep samples,
> potentially for a long time. Everybody put in until it is full and
> once it is full, there is endless discussion on what to throw... until
> the fridge is broken because shutdown or unexpected failures and then
> it is obvious to everybody what needs to be taken or thrown. I am
> using such analogy to explain the hygiene to have on shared machines
> Hum, once written I do not know if it relevant. ;-)
It surely is. :-) I mean, I didn’t even think about it until GC
couldn’t make any progress.
> From my point of view, having a channel+manifest "backup" for old
> profiles seems something to try. It cost nothing with
> ''--export-manifest" and "--export-channels".
Yes, though it’s an approximation; so it should only be used when we
know that it’s 100% faithful, as is the case for ‘guix pull’ profiles.
Ludo’.