Re: gnulib-tool: Ensure that long-running tests are executed last

[Bruno Haible]

Hi Eric, Jim,

> >> This ensures that long-running tests are executed last. So that the user
> >> gets the maximum of information as quickly as possible.
> >
> > That goes counter to the idea of parallel execution, where you want the
> > long-running tests front-loaded so that the shorter tests can fill in
> > the gaps.  If the long-running tests are last, then the entire parallel
> > operation is stuck waiting for the last test.
> 
> I have deliberately reordered some tests to put long-running
> ones early enough so that a parallel (often -j5 or more) build
> completes as quickly as possible.

Hmm, it seems we are trying to optimize for different execution models and
different goals:
  - I was assuming a sequential execution (since this is the default with
    "make") and the goal of maximizing the information that is available
    within a short time after starting the test.
  - You are assuming a parallel execution with several processors, and
    the only goal being to minimize the turnaround time, that is, assuming
    that the user wants to execute all tests and is only interested in the
    final summary result.

I agree that in the scenario of parallel execution and the only goal being
to minimize the turnaround time, the long-running tasks need to be started
first.

But we don't know what the user's goals will be.

> > Is there any way this can
> > be made conditional, because there are good arguments for both orderings?

In theory, we could build up two variables TESTS1 and TESTS2, instead of just
TESTS, and decide at "make" time whether to use
  TESTS = $(TESTS1) $(TESTS2)
or
  TESTS = $(TESTS2) $(TESTS1).
Maybe even by inspecting the MAKEFLAGS.

But this sounds like a big hack: Basically, we would be adding a context
dependent choice of a scheduler algorithm in a particular Makefile. It would
be better if we could some way declare the expected running time to GNU make,
and have GNU make choose its scheduler algorithm itself.

Bruno