[bug#57050] [PATCH v2 05/13] gnu: racket: Update to 8.6.

[Philip McGrath]

Hi,

On Thu, Aug 11, 2022, at 7:44 AM, Liliana Marie Prikler wrote:
> Am Donnerstag, dem 11.08.2022 um 07:08 -0400 schrieb Philip McGrath:
>> Also, update 'chez-scheme-for-racket' to 9.5.9.2.
> As with Zuo, can this be split into a separate patch?  If not, why does
> it get mentioned here as something noteworthy rather than just being
> part of the ChangeLog?
>

This absolutely should not be split into more patches. While I split out Zuo as 
you requested, I continue to believe that it was better as a single patch.

More broadly, I hope to put together a 'racket-build-system' before the 8.7 
release. Whenever that happens, it will mean turning the inputs of the 'racket' 
and 'racket-minimal' packages into, last I counted, 203 additional packages. I 
think it would be bad for everyone—patch writers, patch reviewers, users, and 
people reading the history later—to break Racket releases into a deluge of 
little patches, many of which would leave the Racket universe as a whole in 
states no one had ever built or tested, because Racket has combinations of 
technical and social mechanisms to make certain potentially problematic states 
difficult to fall into.

A Racket release is an update to a family of packages that are developed and 
released together. Like a TeX Live update (cf. 
ee25e3fcab9d2e24c2826b771b52d797c152193b) or the KDE Frameworks libraries (cf. 
9d3965edca29f80667374da45214cc6f22a85be4), the contents of a Racket release 
should be updated together. I wouldn't insist on one commit in all imaginable 
circumstances, but I think it should be the norm. That's all the more true of 
Racket components that share a canonical Git repository, for which Racket tools 
take steps to warn you if not fail if you have mismatched versions.

I also want to emphasize that splitting up patches is not free. 
Splitting/rebasing v2 of this series took hours of work. Even specifically in 
splitting off Zuo, I made a (trivial) error in my first version of this 
particular patch and had to amend the commit and rebuild. Splitting this any 
further would get farther away both from how I actually wrote and tested this 
patch series over the past four months and from the way Racket is developed and 
released. It would take a non-negligible amount of effort, all to produce a 
result that I believe would be worse.

I mention the version number for 'chez-scheme-for-racket' because it shows up 
relatively prominently in Guix tooling and even e.g. in the path of the package 
documentation. I don't know why it would be a problem to do so, but I would 
vastly prefer to remove the mention than to split the patch, if it really has 
to be a choice. The Racket release notes don't mention the corresponding 
version of Chez Scheme.

>> [...]
>> @@ -448,18 +456,52 @@ (define-public chez-scheme-for-racket
>>         (delete "libx11" "util-linux:lib")))
>>      (native-inputs
>>       (modify-inputs (package-native-inputs chez-scheme)
>> +       (append zuo)
>>         (replace "chez-scheme-bootstrap-bootfiles"
>>           chez-scheme-for-racket-bootstrap-bootfiles)))
> Prefer prepend over append.
>

Why?

>>      (arguments
>>       (substitute-keyword-arguments (package-arguments chez-scheme)
>> +       ((#:out-of-source? _ #f)
>> +        #t)
>> +       ((#:tests? _ #t)
>> +        ;; FIXME: There have been some flaky test failures. Some have
>> been
>> +        ;; fixed upstream post-release but have proven non-trivial to
>> +        ;; backport; at least one issue remains. Re-enable tests once
>> +        ;; https://github.com/racket/racket/issues/4359 is fixed.
>> +        #f)
> Rather than skipping tests altogether, skip just the flaky ones.
>

If I knew how to do that, I absolutely would. Well, short of making the 
`process` function silently fail to run `/bin/sh` again, maybe. If you look at 
the linked issue, you'll see that I've been chasing down failures for the last 
month. For a while I had cherry-picked 
153ff9acb7ad63717a50bb26cd5aaa053870c666, which fixed the only issue with the 
implementation (a race condition in one mode of the garbage collector), but it 
didn't seem worth carrying the patch when things were still failing. The other 
issues seem to be problems with running the test suite. Probably they appeared 
now because we hadn't actually been running as much of it as we seemed to be. 
So far, I haven't had failures building with current master, but there aren't 
any commits touching anything obviously related, and Unicode 14 and grapheme 
cluster support have landed since the release branch, so it wouldn't be 
reasonable to just sweep in all the changes indiscriminately. Further, it's at 
least possible that I may just have been winning the race recently and that the 
actual problem might still be there. The Chez test suite takes an hour to run 
(maybe more), and I haven't been able to reproduce the failures outside of 
Guix, so it's not exactly rapid iteration. All in all, I don't know any change 
to make other than turning off tests that I can feel confident will work 
reliably for Guix users.

For what it's worth, difficulty running the Chez tests is not unique to Guix. 
In the course of adding Zuo, Matthew Flatt commented, "After this conversion, I 
was able to run the Chez Scheme test suite on Windows for the first time; it 
must have been possible before, but I never quite got the right tools installed 
in the right way to make it work." [1] AFAICT Debian skips the Chez test suite 
unless it is specifically requested [2]: I don't know why, maybe just because 
it takes so long.

[1]: https://github.com/racket/racket/pull/4179#issuecomment-1094137092
[2]: 
https://salsa.debian.org/scheme-team/chezscheme/-/blob/fbfa9c35db184f5b22ac6c0058d754e1a45f5f68/debian/rules#L66-69

>> @@ -130,12 +132,12 @@ (define-module (gnu packages racket)
>>  ;; output. The function 'racket-vm-for-system' returns the
>> recomended Racket
>>  ;; VM package for a given system.
>>  ;;
>> -;; The file 'racket.scm' builds on these packages to define 'racket-
>> minimal'
>> -;; and 'racket' packages. These use Racket's support for ``layered
>> -;; installations'', which allow an immutable base layer to be
>> extended with
>> -;; additional packages. They use the layer configuration directly
>> provide
>> -;; ready-to-install FHS-like trees, rather than relying on the built
>> in
>> -;; ``Unix-style install'' mechanism.
>> +;; We then define the packages 'racket-minimal' and
>> +;; 'racket'. These use Racket's support for ``layered
>> installations'', which
>> +;; allow an immutable base layer to be extended with additional
>> packages.
>> +;; They use the layer configuration directly provide ready-to-
>> install FHS-like
>> +;; trees, rather than relying on the built in ``Unix-style install''
>> +;; mechanism.
>>  ;;
>>  ;; Bootstrapping Racket:
>>  ;; ---------------------
> This is a leftover from 8.6, but do the "FHS-like" installations
> actually adhere to the FHS or just to the bit that says "opt means we
> can do whatever we want 😎️"?
>

The 'racket' and 'racket-minimal' packages follow FHS. (Pedantically, because 
compiled Racket code may or may not be architecture specific, and used to 
always be architecture-independent, some configuration modes put things in 
"share" that belong in "lib", but the current upstream "Unix-style" defaults 
and out packages do not do that.)

Here's a comparison of the hierarchy of our non-VM Racket packages vs. an 
in-place Racket installation, with the bracketed numbers indicating how the two 
match up:

/gnu/store/xyz-racket-8.5/
├── bin/ [1]
├── etc/
│   └── racket/
│       └── config.rktd [2]
├── lib/
│   └── racket/ [3]
│       ├── links.rktd
│       ├── mans.rktd
│       └── pkgs/ [4]
└── share/
    ├── applications/ [5]
    │       └── drracket.desktop
    ├── doc/
    │       └── racket/ [6]
    │           └── index.html
    ├── man/ [7]
    │       └── man1/
    │           └── racket.1.gz
    └── racket/ [8]
             └── infocache.rktd

/gnu/store/xyz-racket-vm-cs-8.5/opt/racket-vm/
├── bin/ [1]
├── collects/ [would be share/racket/collects, but not duplicated in layers]
├── doc/ [6]
├── etc/
│   └── config.rktd [2]
├── include/
│   ├── chezscheme.h
│   ├── racketcsboot.h
│   └── racketcs.h
├── lib/ [3]
│   ├── libracketcs.a
│   ├── petite.boot
│   ├── racket.boot
│   └── scheme.boot
├── man/ [7]
│   └── man1/
└── share/ [8]
    ├── applications/ [5]
    └── pkgs/ [4]


>>  ;;
>> -;; Code:
>> +;; Zuo is notably *not* a problem for bootstrapping. The
>> implementation is a
>> +;; single hand-written C file designed to build with just `cc -o zuo
>> zuo.c`,
>> +;; even with very old or limited compilers. (We use the Autoconf
>> support for
>> +;; convienience.)
>> +;;
>> +;; CODE:
> Is that something that needs be mentioned?
>

Some people have asked whether the Zuo implementation does or should share 
things with Racket and/or Chez Scheme. It seemed worth noting that it 
intentionally does not, for the sake of bootstrapping.

-Philip