RE: grub causing NVDIMMs to be treated as normal memory

[Elliott, Robert (Persistent Memory)]

> -----Original Message-----
> From: Andrei Borzenkov [mailto:address@hidden
...
> From the build directory
> 
> pkgdatadir=$PWD ./grub-install --bootloader-id testgrub -d grub-core
> 
> This should install grub in \EFI\testgrub on ESP and add EFI menu for
> it. You can add --no-nvram to skip EFI menu update and load it
> manually then. It will install modules in /boot/grub (instead of
> /boot/grub2), so you should probably copy /boot/grub2/grub.cfg there.

grub-install reported one complaint, which I ignored:
        Installing for x86_64-efi platform.
        ./grub-install: warning: cannot open directory 
`/usr/local/share/locale': No such file or directory.
        Installation finished. No error reported.

I ran this to create a grub.cfg there like the one I had been using:
        grub2-mkconfig /boot/grub 

The system rebooted and ran the code, giving me a menu on the serial
port:
        GNU GRUB  version 2.02~beta2 menu 

Booting failed.  I had to change these with 'e':
* linuxefi to linux
* initrdefi to initrd

Hitting Ctrl-x to boot after that resulted in a successful boot with
these prints:
  Booting a command list

Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
Unknown memory type 14, considering reserved
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.0-rc2+ (address@hidden) (gcc version 5.1.1 
20150618 (Red Hat 5.1.1-4) (GCC) ) #87 SMP Mon Nov 23 16:12:10 CST 2015
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.4.0-rc2+ 
root=/dev/mapper/fedora_pmem03-root ro rd.lvm.lv=fedora_pmem03/swap 
rd.lvm.lv=fedora_pmem03/root rhgb quiet efi=debug libnvdimm.dyndbg nfit.dyndbg 
nd_pmem.dyndbg nd_btt.dyndbg ignore_loglevel console=tty0 console=ttyS0,115200
...


Next, I ran grub-mkconfig from the build directory:
        grub-mkconfig -o /boot/grub/grub.cfg
and it created a grub.cfg that used linux and initrd, but 
lacked all the linux command line options from /etc/default/grub.

Based on comments in the top of grub.cfg file, I copied
/etc/default/grub to /usr/local/etc/default/grub
and reran.  This preserved the linux command line options.

I also added this to /boot/grub/grub.cfg:
        set debug=mmap,linux

That results in a different looking boot menu with two top
level entries:
        *Fedora GNU/Linux
         Advanced options for Fedora GNU/Linux

I let the timeout expire and it booted with these prints:
  Booting `Fedora GNU/Linux'

Loading Linux 4.4.0-rc2+ ...
mmap/efi/mmap.c:66: EFI memory region 0x0-0x93000: 7
...[skipping rest of the entries]...
mmap/efi/mmap.c:66: EFI memory region 0x880000000-0xc80000000: 14
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x1480000000-0x1a80000000: 14
Unknown memory type 14, considering reserved
loader/i386/linux.c:253: prot_mode_mem = 0x2000000, prot_mode_target = 2000000,
prot_size = 12e9000
loader/i386/linux.c:877: bzImage, setup=0x4200, size=0x12e9000
Loading initial ramdisk ...
loader/i386/linux.c:1123: Initrd, addr=0x35dfd000, size=0x21f1ab1
mmap/efi/mmap.c:66: EFI memory region 0x0-0x93000: 7
...
mmap/efi/mmap.c:66: EFI memory region 0x880000000-0xc80000000: 14
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x1480000000-0x1a80000000: 14
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x0-0x93000: 7
...
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x1480000000-0x1a80000000: 14
Unknown memory type 14, considering reserved
loader/i386/linux.c:569: real_size = 6000, mmap_size = 2000
mmap/efi/mmap.c:66: EFI memory region 0x0-0x93000: 7
loader/i386/linux.c:423: addr = 10000, size = 80000, need_size = a000
mmap/efi/mmap.c:66: EFI memory region 0x93000-0x94000: 0
...
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x1480000000-0x1a80000000: 14
Unknown memory type 14, considering reserved
loader/i386/linux.c:581: real_mode_target = 86000, real_size = 6000,
efi_mmap_size = 4000
loader/i386/linux.c:598: real_mode_mem = 0x77926000
loader/i386/linux.c:608: code32_start = 2000000
mmap/efi/mmap.c:66: EFI memory region 0x0-0x93000: 7
...
mmap/efi/mmap.c:66: EFI memory region 0x880000000-0xc80000000: 14
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x1480000000-0x1a80000000: 14
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x0-0x93000: 7
...
mmap/efi/mmap.c:66: EFI memory region 0x880000000-0xc80000000: 14
Unknown memory type 14, considering reserved
mmap/efi/mmap.c:66: EFI memory region 0x1480000000-0x1a80000000: 14
Unknown memory type 14, considering reserved



This is booting a new kernel with the EFI boot stub, so I can't
confirm that it fixes the issue of exposing the address range as
normal, but based on the print it's probably working.  I could
add prints in grub-core/loader/i386/linux.c grub_e820_add_region
to confirm the e820 contents at exit.


---
Robert Elliott, HPE Persistent Memory