Re: Handling large allocations (bypassing mm?)

[Ard Biesheuvel]

On Thu, 15 Dec 2022 at 15:29, Julian Andres Klode
<julian.klode@canonical.com> wrote:
>
> On Wed, Dec 14, 2022 at 04:11:18PM +0100, Daniel Kiper wrote:
> > Adding a few folks who may be interested in this discussion too...
> >
> > On Wed, Dec 14, 2022 at 02:21:49PM +0100, Julian Andres Klode wrote:
> > > Hi,
> > >
> > > so I want to bring this discussion here that I had mostly with myself
> > > in the past days on IRC.
> > >
> > > As some of you know, we had a couple issues with large initrds in
> > > Ubuntu, Jeremy posted a patch series earlier about mmunlimited.
> >
> > I skimmed through it but at first sight I do not like it. I will comment
> > the patches more extensively in the following days.
> >
> > > I wanted to propose a more fine-grained approach, as well as a
> > > more generic approach to handling large allocations.
> > >
> > > The first issue one experiences when opening large initrds is
> > > that grub_file_open() calls grub_verifier_open() which simply
> > > grub_malloc()s a buffer for the size of the file.
> > >
> > > Later, for initrd, we have to allocate it a second time, in
> > > the upstream tree that happens via relocator, in the rhboot
> > > tree it allocates directly from EFI.
> > >
> > > Now my basic proposal is quite simple: We make grub_malloc()
> > > and that relocator allocation code bypass the grub memory
> > > management altogether and just do raw EFI page allocations
> > > (provide two function pointers grub_mm_allocate_pages and
> > > grub_mm_free_pages, and just call them if allocation size
> > > is large[1]). e.g. at the start of grub_malloc:
> >
> > In general I think we should stop using legacy Linux loader on UEFI
> > platforms and migrate to LoadFile2 protocol. Relevant patches for ARM64
> > are in master now. We are waiting for patches from Atish which will
> > merge ARM64 implementation into RISC-V (you can find earlier version
> > of the patch set here [1]). Then it could be extended into other
> > architectures including AMD64. Of course this will not solve problem
> > with extra allocation in verifiers. However, at least we should not
> > have problems with large initrds and we will not deal with relocator
> > on EFI platforms.
> >
> > WRT verifiers extra allocation, as you said, this is a problem but it can
> > be solved later. Now at least we should be sure it is possible to load
> > big initrds with LoadFile2 protocol and there are no obstacles in the
> > GRUB code for this thing.
>
> To my knowledge, that is incorrect and LoadFile2 suppoort does nothing
> to address the issue of large initrds:
>

It does.

You should only need 1x the space for the initrd with LoadFile2(), and
given that it is the EFI stub itself that performs the allocation,
whether or not it is above 4 GB or meets some other arch-specific
requirement is completely immaterial to GRUB.

The initrd data will only be loaded once, when the EFI stub asks for
it, and if it doesn't, GRUB will never access the file (or more
accurately, never call grub_file_read() on it).

For verification and other purposes, GRUB is free to authenticate,
decompress or do other things to the payload after loading it into the
buffer but before returning from the callback, and if needed, it can
wipe the buffer again and return an error code to the EFI stub, which
will cause it to give up and return back to GRUB (provided that you
are not using the terminally broken EFI handover protocol)

As for supporing kernels from 2012: I don't see why upstream GRUB
should care about that. If your distro fork supports those today, you
will simply need to carry those patches out of tree a bit longer.