bug#47064: [racket-users] bytevector-uncompress: internal error uncompressing

[Mark H Weaver]

Hi Philip,  [removed 'racket-users' from the recipient list]

Philip McGrath <philip@philipmcgrath.com> writes:

> My guess is that Racket CS is compressing string literals in compiled 
> code. Currently, Guix patches Racket source files to include the 
> absolute paths to foreign libraries in the store as string literals. 
> There are a bunch of grafts for GTK and such: if I'm right, Guix somehow 
> mangles the compiled code while attempting to apply the grafts.

I think I know what happened here.

Recall that the grafting code performs a set of substitutions, replacing
store item names (i.e. file names in /gnu/store) with replacement store
items of the same length, with rules like:
"fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10" =>
"rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12".

The grafting code currently only checks the first 33 bytes, consisting
of the nix-base32 hash and the "-".  It *assumes* that the remainder of
the associated store item name immediately follows, and blindly writes
the replacement string over whatever is there.

In this case, I suspect that within a *.zo file, a Guix store item name
was split into pieces, with the hash and "-" together in one piece but
split somewhere between the "-" and the last byte of the store item.
This results in corruption of the bytes following that piece.

I've recently observed the splitting of store item names in *.zo files
(see <https://bugs.gnu.org/47614>), but in that case the "-" was
separated from the hash, and as a result the reference was _invisible_
to the grafter.

For the record, when I originally wrote this fast(er) grafting code
(commit 5a1add373ab427a3b336981d857252e703a9f8d1), by design it only
rewrote the hashes, and so naturally it had the following desirable
property: it never overwrote any byte without first checking it against
an expected value.  Later, starting in commit
57bdd79e485801ccf405ca7389bd099809fe5d67, the grafting code was modified
to allow rewriting the entire store item name (notably including the
version number).  Unfortunately, although the set of overwritten bytes
was extended past the "-", the set of bytes *checked* was left
unchanged, and thus the aforementioned desirable property was lost.

I think we ought to restore that property.  I'm already working on some
other changed to the grafting code (supporting UTF-16 and UTF-32 encoded
references), so I'll try to find the time to fix this problem as well.

    Regards,
      Mark