Re: [PATCH v3 0/3] Cryptomount detached headers

[Glenn Washburn]

On Tue,  2 Aug 2022 22:49:27 +0200 (CEST)
brutser--- via Grub-devel <grub-devel@gnu.org> wrote:

> 
> Van: Glenn Washburn <development@efficientek.com>
> Aan: brutser--- via Grub-devel <grub-devel@gnu.org>
> Onderwerp: Re: [PATCH v3 0/3] Cryptomount detached headers
> Datum: 02/08/2022 20:58:17 Europe/Paris
> Cc: brutser@perso.be;
>    dkiper@net-space.pl;
>    ps@pks.im
> 
> Hi Bruster,
> 
> Are you able to add your responses inline like I have been doing in my
> replies? And not top-post, which is posting at the top.
> 
> I will reply here, though my mail client is not really doing a great job to 
> get this inline response done, sorry for that.

Ok, I figured that. However, you've not responded to much of my
comments that I wrote inline. Perhaps you only read the first part of
my response and do not realize that I wrote more response further down.
Please look at the full email until you see my name, which indicates
that I am finished responding. Please respond to all of my questions,
otherwise it makes it difficult to help you and if you don't take the
time to respond to me fully I am less inclined to help. I would like to
get this issue sorted out, but I need you to help me debug it right now.

Also, I have never built GRUB with coreboot, so I don't have a way to
precisely reproduce your setup anyway. And it currently does work for
me.

Glenn

> 
> Let me be clear on my testing environment, I was purely testing latest grub 
> as payload for coreboot, so even on qemu, I am using coreboot for BIOS with 
> my compiled grub as one of the payloads.
> I am testing on a basic Thinkpad laptop with x86_64 architecture.
> 
> "I'm not sure I understand why you're saying this.", I was basically just 
> expressing that I was testing the functionality purely for my own setup, so 
> if you wanted me to test more general (for example without coreboot), then 
> you could say me.
> Not that it shouldn't work with coreboot obviously, but if other testing 
> could give more accurate results and debug logs for example, I am willing to 
> do that, that was all.
> 
> On Tue, 2 Aug 2022 02:26:58 +0200 (CEST)
> brutser--- via Grub-devel <grub-devel@gnu.org> wrote:
> 
> > 
> > 
> > Glenn,
> > 
> > 
> > 
> > But my testing is very limited, i only create grub payload for coreboot and 
> > then i create the encrypted sda2 from the debian expert installer etc.
> 
> I'm not sure I understand why you're saying this. Did I ask you to do
> something that you think you can't do because your "testing is very
> limited"? You've proven capable of modifying the source code to add
> debug log messages and successfully reproducing the issue in QEMU. That
> makes your testing ability not very limited in my opinion. What am I
> missing that you're wanting to convey here?
> 
> Since you've reproduced the issue in QEMU and I'm assuming that you're
> not running coreboot in QEMU, then I conclude that coreboot is not a
> relevant factor here. What exactly was the QEMU commandline you used in
> getting the output you previously created? Did you ever try to get the
> serial output with the QEMU options I suggested previously?
> 
> > 
> > If you got suggestions with this setup, how i can do another way of 
> > testing, let me know and I will do it.
> 
> Why do you need another way of testing? As I mentioned above you can
> modify the source code of GRUB and run that modified GRUB in QEMU. I
> don't think we need another way. Were you having problems adding the
> debug message to cryptodisk_read_hook I suggested in my last response?
> I'm starting to think there is an issue in the hook mechanism, but I'd
> like your help in confirming that.
> 
> > 
> > Van: brutser--- via Grub-devel <grub-devel@gnu.org>
> > Aan: grub-devel@gnu.org
> > Onderwerp: Re: [PATCH v3 0/3] Cryptomount detached headers
> > Datum: 02/08/2022 01:47:57 Europe/Paris
> > Cc: brutser@perso.be;
> >    dkiper@net-space.pl;
> >    ps@pks.im
> > 
> > 
> > 
> > Debian 11.4 for all the testing.
> 
> Thanks for this info, but that wasn't what I was asking for. I asked
> for the architecture and endianness. For example, are you running on
> x86_64 or i386 (64-bit vs 32-bit) x86 architecture? These are always
> little endian. But you could be running on a MIPS architecture that can
> be either big or little endian. I want this confirmed so I can get the
> full picture. I'm doubting now that this is important, but you never
> know.
> 
> From your response below, I believe that the host and the target are
> the same machine, but are you building GRUB on that machine as well?
> Are you running QEMU for testing on that machine as well?
> 
> I haven't tried to reproduce this issue locally yet due to time
> constraints and it may be a while before I can get to it. But we're
> getting close to the point where it may require me doing that.
> 
> Glenn
> 
> > 
> > as i said, i execute shell during installation, then simply enter the 
> > commands I wrote earlier:
> > 
> > 
> > 
> > cryptsetup luksFormat --type luks2 -q -h sha512 -s 512 --pbkdf pbkdf2 
> > --header /root/header.bin --luks2-metadata-size=16k 
> > --luks2-keyslots-size=512k /dev/sda2
> > 
> > cryptsetup luksOpen --header /root/header.bin /dev/sda2 sda2crypt
> > 
> > pvcreate /dev/mapper/sda2crypt
> > 
> > vgcreate testvg /dev/mapper/sda2crypt
> > 
> > lvcreate -L 2G -n root testvg
> > 
> > 
> > 
> > - continue install debian 11.4
> > 
> > - chroot into system
> > 
> > - copy header
> > 
> > - populate crypttab etc.
> > 
> > 
> > 
> > this whole process works 100% fine with grub 2.04 and luks1 as i said 
> > before...
> > 
> > 
> > 
> > 
> > 
> > Van: Glenn Washburn <development@efficientek.com>
> > Aan: brutser--- via Grub-devel <grub-devel@gnu.org>
> > Onderwerp: Re: [PATCH v3 0/3] Cryptomount detached headers
> > Datum: 02/08/2022 01:24:47 Europe/Paris
> > Cc: brutser@perso.be;
> >    dkiper@net-space.pl;
> >    ps@pks.im
> > 
> > On Tue, 2 Aug 2022 00:21:09 +0200 (CEST)
> > brutser--- via Grub-devel <grub-devel@gnu.org> wrote:
> > 
> > > Glenn,
> > > 
> > > 
> > > 
> > > Still resorted to screenshots for the debug (with the added dprintf):
> > > 
> > > 
> > > 
> > > https://imgur.com/a/YkVMdBe
> > 
> > Ok, that confirms that the luks2 module is loaded and that the scan is
> > happening. Based on the output I think luks2_read_header must be
> > failing. That means that either disk reads are failing, which doesn't
> > seem like the case, the disk read hook is failing or the LUKS2 magic
> > bytes are not what they should be.
> > 
> > Have you verified that after creating the volume and header file that
> > cryptsetup/dm can open the volume successfully?
> > 
> > What architecture and endianness is the machine you're running
> > cryptsetup on and what is it for the one GRUB is running on?
> > 
> > To test the read hook, add 'grub_dprintf("luks2", "read hook
> > successed");' just before the last return statement in the function
> > cryptodisk_read_hook in grub-core/disk/cryptodisk.c.
> > 
> > Glenn
> > 
> > > 
> > > 
> > > 
> > > Van: Glenn Washburn <development@efficientek.com>
> > > Aan: brutser--- via Grub-devel <grub-devel@gnu.org>
> > > Onderwerp: Re: [PATCH v3 0/3] Cryptomount detached headers
> > > Datum: 01/08/2022 22:50:27 Europe/Paris
> > > Cc: brutser@perso.be;
> > >    dkiper@net-space.pl;
> > >    ps@pks.im
> > > 
> > > On Sat, 30 Jul 2022 11:54:32 +0200 (CEST)
> > > brutser--- via Grub-devel <grub-devel@gnu.org> wrote:
> > > 
> > > > Glenn,
> > > > 
> > > > 
> > > > 
> > > > As I had no idea how to get the debug logs from qemu, I made 
> > > > screenshots, find them attached. As this is probably something I am 
> > > > doing wrong, I hope it shows from the logs.
> > > > 
> > > > https://imgur.com/a/rAlfZ77
> > > 
> > > Getting the output to go to serial depends on the target. For i386
> > > using seabios, use "-fw_cfg name=etc/sercon-port,string=0 -serial
> > > stdio".
> > > 
> > > Unfortunately, I'm now seeing that there are no debug log messages
> > > in the luks2 module that would be shown in this case. How about putting
> > > the line 'grub_dprintf("entering luks_scan");' at the start of the
> > > function luks2_scan in grub-core/disk/luks2.c and then recompiling and
> > > getting the output?
> > > 
> > > Glenn
> > > 
> > > 
> > > > 
> > > > Van: Glenn Washburn <development@efficientek.com>
> > > > Aan: brutser@perso.be
> > > > Onderwerp: Re: [PATCH v3 0/3] Cryptomount detached headers
> > > > Datum: 29/07/2022 21:27:48 Europe/Paris
> > > > Cc: grub-devel@gnu.org;
> > > >    dkiper@net-space.pl;
> > > >    ps@pks.im
> > > > 
> > > > On Fri, 29 Jul 2022 20:56:18 +0200 (CEST)
> > > > brutser@perso.be wrote:
> > > > 
> > > > > 
> > > > > testing detached header failed:
> > > > > 
> > > > > 
> > > > > 
> > > > > 1. built grub payload with following modules: ahci usb_keyboard 
> > > > > part_msdos part_gpt at_keyboard cbfs cryptodisk luks2 lvm 
> > > > > gcry_rijndael gcry_sha1 gcry_sha256 gcry_sha512
> > > > > 
> > > > > 2. encrypt a partition: cryptsetup luksFormat --type luks2 -q -h 
> > > > > sha512 -s 512 --pbkdf pbkdf2 --header /path/to/header 
> > > > > --luks2-metadata-size=16k --luks2-keyslots-size=512k /dev/sda1
> > > > > 
> > > > > (where --luks2-metadata-size=16k --luks2-keyslots-size=512k is 
> > > > > optional, this is just to minimize header size, but I also tested 
> > > > > without).
> > > > > 
> > > > > 3. from the grub cmd, i try to decrypt this partition using: 
> > > > > cryptomount -H /path/to/header (ahci0,msdos1)
> > > > > 
> > > > > 
> > > > > 
> > > > > 4. I also tried luks1 encryption with detached header.
> > > > > 
> > > > > 
> > > > > 
> > > > > whatever I try, I always get the same error:
> > > > > 
> > > > > "no cryptodisk module can handle this device"
> > > > > 
> > > > > 
> > > > > 
> > > > > Is this feature not 100% implemented yet, I saw people already 
> > > > > verifying the patches and would expect this to be working, so if yes, 
> > > > > this seems like a bug.
> > > > 
> > > > This feature should be working in all cases, and if not there may be a
> > > > bug. I responded to your off-list email before seeing this one. I'll
> > > > repeat what I said there and let's continue this discussion on the list.
> > > > 
> > > > I see nothing obviously wrong with what you're doing, given the
> > > > information above. To further debug this, would you be able to send a
> > > > log of the serial output when the GRUB envvar debug is set to "all"
> > > > while running the cryptomount command? If so, please send compressed in
> > > > a reply to this email on the list.
> > > > 
> > > > If you can't because of hardware issues, would you be able to replicate
> > > > this in QEMU and grab the serial output from there? If you can boot the
> > > > system via other means, you should be able to use the raw disks (the
> > > > one with the LUKS volume and the other with the filesystem containing
> > > > the header file).
> > > > 
> > > > Glenn
> > > > 
> > > > 
> > > > _______________________________________________
> > > > Grub-devel mailing list
> > > > Grub-devel@gnu.org
> > > > https://lists.gnu.org/mailman/listinfo/grub-devel
> > > > 
> > > 
> > > _______________________________________________
> > > Grub-devel mailing list
> > > Grub-devel@gnu.org
> > > https://lists.gnu.org/mailman/listinfo/grub-devel
> > > 
> > 
> > _______________________________________________
> > Grub-devel mailing list
> > Grub-devel@gnu.org
> > https://lists.gnu.org/mailman/listinfo/grub-devel
> > _______________________________________________
> > Grub-devel mailing list
> > Grub-devel@gnu.org
> > https://lists.gnu.org/mailman/listinfo/grub-devel
> > 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>