random numbers in tests

[John W. Eaton]

After renaming the DLD-FUNCTIONS directory to dldfcn, two new test
failures popped up.  The failing tests were the last two from
splinefit.m:

  %!shared xb, yb, x
  %! xb = 0:2:10;
  %! yb = randn (size (xb));
  %! x = 0:0.1:10;

  %!test
  %! y = interp1 (xb, yb, x, "linear");
  %! assert (ppval (splinefit (x, y, xb, "order", 1), x), y, 10 * eps ());
  %!test
  %! y = interp1 (xb, yb, x, "spline");
  %! assert (ppval (splinefit (x, y, xb, "order", 3), x), y, 10 * eps ());
  %!test
  %! y = interp1 (xb, yb, x, "spline");
  %! assert (ppval (splinefit (x, y, xb), x), y, 10 * eps ());

The new test failure was just that the compute value did not match the
expected value to the requested tolerance.  To the digits displayed in
the log file, everything looked OK.

Since all I had done was rename some files, I couldn't understand what
could have caused the problem.  After determining that the changeset
that renamed the files was definitely the one that resulted in the 
failed tests, and noting that running the tests from the command line
worked, I was really puzzled.  Only after all of that did I finally
notice that the tests use random data.

It seems the reason the change reliably affected "make check" was that
by renaming the DLD-FUNCTION directory to dldfcn, the tests were run
in a different order.  Previously, the tests from files in the
DLD-FUNCTION directory were executed first.  Now they were done later,
after many other tests, some of which have random values, and some
that may set the random number generator state.

Is this sort of thing also what caused the recent problem with the
svds test failure?

Should we always set the random number generator state for tests so
that they can be reproducible?  If so, should this be done
automatically by the testing functions, or left to each individual
test?

jwe