Re: GRUB 2.12 release - update

[Pete Batard]

Hi Daniel,

Thanks for keeping us updated on your the release progress and release 
plans.

On 2022.10.26 15:52, Daniel Kiper wrote:
> We are getting closer to the 2.12 release. Sadly we still do not have
> many of important patch sets in the tree. So, I am going to spend more
> time on reviews in the following weeks. Below you can find my list of
> key patch sets which should land in the release:
>    - Dynamic allocation of memory regions and IBM vTPM v2,
>    - Unify ARM64 & RISC-V Linux Loader,
>    - Add support for LoongArch,
>    - plainmount: Support plain encryption mode,
>    - Glenn's tests fixes.
>
> Of course all patches which got my RB or are under review will be merged too.
>
> There is also "nice to have" list but I do not consider lack of this
> patch sets as release blockers:
>    - Add support for EFI file system transposition,

I'm afraid I will have to strongly disagree about this being a mere 
"nice to have".

Lack of support for EFI file system transposition needs to be considered 
as blocking as if grub-mkrescue was producing ISOs that *broke* the 
ability to write them in dd mode. In other words, if someone came to 
this list and told you that grub-mkrescue in its current state was 
unable to produce functional ISOHybrid images that can be copied in dd 
mode, I am pretty sure that it would be treated as a showstopper.

Well, it is exactly that. Except the showstopping happens with the 
alternative to dd mode.

It is that simple.

And it is exceedingly disheartening for folks who are ultimately 
advocating for the end-users' welfare (rather than the distro 
maintainer's welfare, because these two do not always entirely overlap) 
to see that, yet again, the ability to create ISOHybrid media using EFI 
file system transposition is being treated as a simple "nice to have" 
feature, instead of the "SHOULD have equal footing as dd" that it should 
arguably be viewed as.

In my submission from 2022-06-06, I have provided more than a dozen 
direct examples of how having a casual approach to EFI File system 
transposition is very negatively impacting end users, and I could easily 
pick a dozen more examples that intervened since I submitted the series. 
So I will kindly remind folks that we're not talking about an alleged or 
estimated negative impact here. The negative impact of not supporting 
EFI file system transposition is happening. It is affecting thousands of 
ISOHybrid end users. And the failure of grub-mkrescue to produce 
ISOHybrids that can be used in file system transposition mode is going 
to continue to be a major contributor to it.

So, let me go over the main points yet again, so that, maybe, it'll 
start to make sense why this issue needs to be treated as as the 
showstopper it actually is.

1. ISOHybrid should not be seen as nothing more than a "glorified dd 
image that can also burned to optical" (And I could, again, go over the 
may reasons that make ISOHybrid as dd less than the panacea that many 
proponents think it is, such as, for instance, the fact that it is 
factually impossible to create a *valid* GPT based media from an 
ISOhybrid using dd, on account that the backup GPT will never ever be 
written properly [*MUST* reside in the last 23 sectors of the disk], 
unless you use a media that is the exact same size as the ISO, which is 
never the case). Instead, users SHOULD be given the *choice* to create 
bootable media from ISOHybrid using either dd or EFI File System 
transposition with no nudging by the people who mastered the ISO for one 
or the other. But current grub-mkrescue is removing that *choice* from 
end-users, which is showstopper behaviour.

2. One of whole points of UEFI was to do away with the need to write 
boot media at the block level, to instead write it at the file system 
level (i.e. EFI file system transposition) because it was identified 
that writing anything at the block system level was (rightfully) 
ultimately problematic for end users. Yet what we are seeing is that 
Linux maintainers (and by extension GRUB maintainers) have done a 
complete 180 on that promise, by embracing the idea that users 
everywhere, including ones on platforms where it's super inconvenient 
(read Windows, but not only) should just use dd to write their boot 
media, thereby sticking to the old problematic write boot media at the 
block level, with little or no concern for file system transposition.

3. With GRUB releases being spaced about one year at best, if this is 
treated as "nice to have" and delayed, we're not going to see file 
system transposition being supported downstream for at least another 
year. But the problem is that we are seeing Linux distro maintainers, 
who were using a mix of Syslinux (for BIOS) and GRUB (for UEFI) [which 
would usually support or be salvageable when it comes to file system 
transposition] in the process of switching to GRUB exclusively, and 
switching to using a grub-mkrescue based process to do just that. This 
means that, if we don't get file system transposition included in the 
next release, we're going to get a full year or more of distros 
switching to unsalvageable grub-mkrescue and completely preventing 
end-users from using file system transposition to create their boot 
media, resulting in even more pain than what we're currently seeing. So 
please be very mindful that we are in an active transition phase (which 
is how I came to realise that we had a major problem on our hand, as it 
took a while for the grub-mkrescue limitation to percolate downstream up 
to the end-user,  otherwise I would have sent a patch much sooner) and 
that any delay we add in fixing this issue will result in more literal 
years of pain for end users. In other words, if we are actively looking 
up at what's coming up ahead, it should be clear that time to fix this 
is now rather than later (because, even if this gets integrated shortly 
after the next release, a lot of distros tend to stick with dot release 
with a few patches rather than rolling GRUB git, and they are unlikely 
to pick this, especially when GRUB is actively cementing the idea that 
EFI file system transposition is simply "nice to have" instead of the 
fundamental foundation of creating boot media that is should actually be 
seen as).

4. While full file system transposition support does also require the 
involvement of Linux distro maintainers (in its current state, a fixed 
grub-mkrescue alone is not enough as the ISO masterer must also make 
sure that the content of the efi.img is dusplicated onto the ISO file 
system), this last matter is something that can be easily salvaged by 
forward thinking applications (e.g. Rufus is smart enough to extract the 
content of efi.img for the user in case it wasn't duplicated on the 
ISO). But this cannot be salvaged at all, if grub-mkrescue remains in 
it's current state, which is why the grub-mkrescue issue needs to be 
addressed as broken behaviour that requires fixing for the next release.

So please please please!!, for the sake of the end-users who are --and 
will be-- creating GRUB based bootable media everywhere, do make sure 
that you consider these patches (or any updated version -- I'll be there 
to work with you on that if needed!) for integration in the next formal 
release. And do not dismiss EFI file system translation as "nice to 
have" when it is both a fundamental keystone of the boot media creation 
process, and GRUB should be seen at the very forefront of promoting it.

Thank you,

/Pete

>    - term/serial: Add support for PCI serial devices,
>    - Add MMIO serial ports support and SPCR detection,
>    - Glenn's gdb patch set.
>
> I hope I will be able to review and merge all patch sets from first list
> during November. Then if everything goes well we will make code freeze
> in December and release at the beginning of next year, preferably in January.
>
> I am considering to not block merges for tests and documentation during
> code freeze.
>
> If you have any comments on that plan please drop me a line.
>
> Daniel
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel