Re: [RFC PATCH v3 0/1] kern/dl: Add module version check

[Pete Batard]

On 2022.12.23 07:18, Zhang Boyang wrote:
> This feature is implemented in kern/dl.c in core.img, which is UNDER 
> YOUR CONTROL.
>
> Let me analyze the worst case from your perspective:
>
> 1) Every distro is enforcing version check, even in BIOS mode.
>
> 2) Because this check is in kern/dl.c, there is no such code in any 
> module (.mod) files. However, this modules are little fatter because 
> they have version information (.modver section), which is harmless.
>
> 3) You can still build your own copy of core.img with this check 
> disabled (or just revert this patch completely), then ship that core.img 
> with Rufus.
>
> 4) Since there is no check in your core.img, it will load any module 
> happily, regardless whether it has mismatched version information.
>
> Therefore, there is no problem at all. The only thing you need to do is 
> build your core.img with this patch reverted in the future. Then things 
> will "work" as usual.

I need to properly test this, but I think that should work, yes.

I should have looked a bit more closely at your proposal, because I 
agree that it should address my concerns, though I do have a small worry 
about the possibility of transitive checks in the future (i.e. modules 
that perform their own loading of submodules for whatever reason, with a 
version check being introduced there).

I'm not planning to try to hold your patch on that, but I sure hope that 
transitive version checks never become a thing.
> Let me analyze why I want to add version check to BIOS mode:
>
> 1) Because you can always remove this patch from your build for Rufus, 
> there is no harm in having this patch in vanilla GRUB.

As long as the version check remains localised inside core.img, yes.

> 2) This patch can provide benefits to BIOS users, because it can 
> diagnose improper installations. For example, user runs `grub-install 
> /dev/sdb` to update GRUB but machine boot from `/dev/sda`.
>
> So there are benefits without any harm, why not have it in BIOS mode?
>
> By the way, I highly recommend you to test latest git GRUB (as a preview 
> of upcoming 2.12 release) with Rufus and it will almost certainly break 
> your existing Rufus build. e.g. use Rufus (with GRUB2.06) to process 
> ISOs (with GRUB2.12); and the reverse, use Rufus (with GRUB2.12) to 
> process ISOs (with GRUB2.06). You probably need to ship another bunch of 
> core.img with Rufus.

That's already been the case for previous releases, e.g. trying to use 
core.img from 2.0(x) with 2.0(x-1) is usually asking for trouble, and I 
do have a mechanism in place to address just that, where users don't 
even need to update Rufus as we can always download a new 'core.img' 
server from our repository. This is also the mechanism that helps us 
address the rare case where distro maintainers have applied breaking 
pacthes onto a dot release that affect core.img.

So, once distros start using GRUB 2.12 (because I need to be able to 
validate the core.img I build against an actual image), I'll have a 
core.img for GRUB 2.12 that users can automatically use in Rufus.

As I pointed out before, the main issue is that even when using the same 
base version, and because GRUB releases are far and in-between, distro 
maintainers cherry pick patches to apply on top of the dot release, and 
(usually, which is the proper thing to do) suffix their version id, 
which of course, would make a version check fail.

But we're still working with the same dot release base in Rufus always, 
which is why the core.img replacement (barring a few exceptions) always 
works.

> Also, I'd also like to advice you to create a patchset/proposal, which 
> make it easy to convert the bootcode in ISO to the bootcode in USB.

I considered it a long time ago, but the main issue is that it would 
only help with future GRUB based images, and not the many many current 
ones I needed to support.

Thus, all it would accomplish then, is force me to have two different 
methods of solving a problem, where one can most certainly do, and hope 
that, in 10 years time (at best, because you bet some people will still 
want to use 10-years old ISOs then, especially if they are booting old 
BIOS-based computers) I *might* be able to remove one of them.

I hope you can appreciate how this approach makes very little sense then 
when one method can accomplish all we need.

Plus, trying to reuse the files from El-Torito would be a major 
challenge, as, even if we were to somehow force include part_msdos, 
biosdisk, fat and ntfs always (which are completely unneeded for optical 
boot) and recover them, we'd still need to alter the prefix directory, 
and that resides in a part that is usually compressed.

So, the only viable alternative, would be to force optical media 
creators to generate a core.img that can work for USB boot (which means 
silent invocation of grub-mkimage behind the scenes), and have that 
somehow find its way on the resulting ISO media.

All in all, these are fairly heavy alterations to the GRUB optical boot 
generation process, especially again for people who (unlike what is the 
case for our UEFI File System Transposition proposal), are not 
guaranteed to care about ISOHybrid in the first place and will be 
wondering why they are suddenly forced to cater for it (which I wouldn't 
personally mind imposing on them, but I don't think it's the proper way 
to go about it).

Thus, even today, the better solution in my view is to just have a 
mechanism where we recompile and use our own core.img, with the modules 
and prefix we need, especially as experience shows that, if using the 
same dot release, this is seldom an issue.

> I 
> think you are expert in this area. If that patch finally got merged 
> (e.g. before 2.12 release window), you life will be definitely easier

Again, that would only potentially make my life easier in 10 years time, 
at best (while still probably leaving users who are trying to use a 10 
yo ISO image stranded then, if this is when I choose to remove the old 
way of using our own core.img).

As with anything GRUB (and this is no criticism of the project), it 
usually takes years for any new feature to make its way to the end 
users, and I will need to support GRUB based images that have been 
generated with older versions of GRUB for a long long time in Rufus.

In short, please be mindful not to apply the perspective of a Linux 
distro maintainer (i.e. "We can just update to latest GRUB when we 
release our new distro and use the new feature!") to a problem that must 
encompass older GRUB versions along with newer ones.

> because you are free from shipping another bunch of core.img. However, 
> since I'm not a expert of this area, please point out if I said 
> something wrong.

I hope that what I've clarified above makes sense.

I'll be testing your proposal as soon as I get a chance, to validate 
that there are no hidden gotchas for my usage scenario, but it will 
probably be a few days. I'll report tot this list when I have done so.

Regards,

/Pete