Re: Improving grub-mkstandalone for reproducible build

[Daniel Kiper]

On Wed, Dec 06, 2023 at 11:42:02AM +0800, Michael Chang via Grub-devel wrote:
> Enclosed is the description from openSUSE bugzilla entry:
>
> While working on reproducible builds for openSUSE, I found that our
> grub2 package's /usr/share/grub2/x86_64-xen/grub.xen varies across
> builds.
>
> I identified 2 issues: The tar contains changing (octal) mtime values.
> The tar stores files in random filesystem readdir order.
>
> The first issue produces such diffs:
> --- old /usr/share/grub2/x86_64-xen/grub.xen (objdump)
> +++ new /usr/share/grub2/x86_64-xen/grub.xen (objdump)
> @@ -4896,7 +4896,7 @@
>   01319f 00000000 30303030 36303000 30303031  ....0000600.0001
>   0131af 37353000 30303031 37353000 30303030  750.0001750.0000
>   0131bf 30303033 34353000 31343533 31323035  0003450.14531205
> - 0131cf 35303300 30303135 30313320 30000000  503.0015013 0...
> + 0131cf 36313200 30303135 30313420 30000000  612.0015014 0...
>   0131df 00000000 00000000 00000000 00000000  ................
>   0131ef 00000000 00000000 00000000 00000000  ................
>
>
> and the second issue produced this diff:
>   01311f 00000000 00000000 01000000 00000000  ................
>   01312f 00000000 00000000 01000000 08b42600  ..............&.
>   01313f 626f6f74 2f677275 622f7838 365f3634  boot/grub/x86_64
> - 01314f 2d78656e 2f646973 6b2e6d6f 64000000  -xen/disk.mod...
> + 01314f 2d78656e 2f6c7378 656e2e6d 6f640000  -xen/lsxen.mod..
>   01315f 00000000 00000000 00000000 00000000  ................
>
> The second issue probably comes from grub_util_fd_readdir that would
> need to collect+sort entries before further processing.
>
> Furthermore, both patches were developed to address the aforementioned
> issues individually. We hope to contribute them to upstream if the
> enhancement sounds appealing to others as well.
>
> Thanks.
>
> Michael Chang (2):
>   mkstandalone: ensure stable timestamps for generated images
>   mkstandalone: ensure deterministic tar file creation by sorting
>     contents

For both patches Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>...

Daniel