Re: [PATCH] RFC: add --shuffle[=seed] argument support

[Sergei Trofimovich]

On Sun, 6 Feb 2022 09:34:22 +0000
Sergei Trofimovich <slyich@gmail.com> wrote:

> On Sat, 05 Feb 2022 18:39:41 -0500
> Paul Smith <psmith@gnu.org> wrote:
> 
> > Nice work!
> > 
> > On Sat, 2022-02-05 at 22:04 +0000, Sergei Trofimovich wrote:  
> > > Some high level TODOs:    
> > 
> > For this amount of change it's likely that you'll need to provide
> > copyright assignment paperwork.  Let me know if you'd like more details
> > about this.

Somehow managed to completely miss this part last time. Google (my
employer) has the assignment agreement with FSF. Do I also need to sign
personal agreement? I am UK citizen if that afects the decision.

Sent v2 of the patch with @google.com address as an author and attempted
to address almost everything you mentioned. The only thing I left in place
is goal reordering.

> > I recommend you try your version of GNU make on a bunch of different
> > real codebases and make sure it still works (and of course, create the
> > above-mentioned regression tests).  
> 
> Will do. I tried on 100 simple packages, but most of them are resursive
> makefiles (which I missed): 'toenv = 0' effectively disabled the option
> very early. Will try a few crafted inputs and handpick projects with
> handwritten build systems.

Tested '--shuffle=random' on ~2000 packages as part of NixOS distribution
bootstrap. It builds all dependencies from binutils, gcc and kernel up to
xorg. Build found a few real bugs on: ion-3, cramfsswap, dev86, libf2c,
groff, source-highlight, aspell, bind, pth, slang, gpm and subversion.

Having inspected failures manually most of the failures are real bugs of
missing dependencies and they came up before when 'make -j' was tested on
them.

cramfsswap is the shortest example:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996964

-- 

  Sergei