Re: Handling large allocations (bypassing mm?)

[Daniel Kiper]

Adding a few folks who may be interested in this discussion too...

On Wed, Dec 14, 2022 at 02:21:49PM +0100, Julian Andres Klode wrote:
> Hi,
>
> so I want to bring this discussion here that I had mostly with myself
> in the past days on IRC.
>
> As some of you know, we had a couple issues with large initrds in
> Ubuntu, Jeremy posted a patch series earlier about mmunlimited.

I skimmed through it but at first sight I do not like it. I will comment
the patches more extensively in the following days.

> I wanted to propose a more fine-grained approach, as well as a
> more generic approach to handling large allocations.
>
> The first issue one experiences when opening large initrds is
> that grub_file_open() calls grub_verifier_open() which simply
> grub_malloc()s a buffer for the size of the file.
>
> Later, for initrd, we have to allocate it a second time, in
> the upstream tree that happens via relocator, in the rhboot
> tree it allocates directly from EFI.
>
> Now my basic proposal is quite simple: We make grub_malloc()
> and that relocator allocation code bypass the grub memory
> management altogether and just do raw EFI page allocations
> (provide two function pointers grub_mm_allocate_pages and
> grub_mm_free_pages, and just call them if allocation size
> is large[1]). e.g. at the start of grub_malloc:

In general I think we should stop using legacy Linux loader on UEFI
platforms and migrate to LoadFile2 protocol. Relevant patches for ARM64
are in master now. We are waiting for patches from Atish which will
merge ARM64 implementation into RISC-V (you can find earlier version
of the patch set here [1]). Then it could be extended into other
architectures including AMD64. Of course this will not solve problem
with extra allocation in verifiers. However, at least we should not
have problems with large initrds and we will not deal with relocator
on EFI platforms.

WRT verifiers extra allocation, as you said, this is a problem but it can
be solved later. Now at least we should be sure it is possible to load
big initrds with LoadFile2 protocol and there are no obstacles in the
GRUB code for this thing.

Daniel

[1] https://lists.gnu.org/archive/html/grub-devel/2022-11/msg00018.html