Re: svds test failure

[Rik]

On 07/31/2012 11:26 AM, Daniel J Sebald wrote:
> On 07/31/2012 12:47 PM, Rik wrote:
>> On 07/31/2012 07:02 AM, address@hidden wrote:
>>>
>>> No, you're not. There issue is that svds has a test that is
>>> nondeterministic and sometimes the error is higher than tolerance. One
>>> the machine that failed, try running "test svds" a few times and see
>>> how often that fails.
>>>
>>> I don't know if we can make this test nondeterministic because the
>>> randomness comes from ARPACK, not from Octave, so I don't know if we
>>> can seed it. We could just increase the tolerance.
>> 7/31/12
>>
>> Melvin,
>>
>> This issue comes up again and again.  The issue is that ARPACK is an
>> iterative solution solver and the success or failure can depend on the
>> initial starting point relative to the solution.  In order to stop
>> variability in tests, we initialize Octave's random number generators and
>> then create an initial seed location (which is now always the same) to pass
>> to ARPACK.  However, even this hasn't fully stopped all failures.  It is
>> possible that this is due to small machine-dependent round-off errors which
>> render even the initial seed location slightly different between machines.
>
> I agree with your assessment.  However, there are six tests in svds.m
> and only two of them use the 'opts' input options.  The tests' fail/pass
> shouldn't be dependent upon having a specific initial state for the code.
>
> There are some other tests here that look somewhat suspect as well.  For
> example:
>
> %!testif HAVE_ARPACK
> %! s = svds (speye (10));
> %! assert (s, ones (6, 1), 2*eps);
>
> To expect a numerical routine to be accurate within two times the
> machine precision is asking for quite a bit.  The tests which have
> 'opts' passed in are asking for accuracy within 1e-10, which seems more
> reasonable.
The special case of the identity matrix and a matrix of all zeros are just
that -- special cases.  ARPACK should be able in these instances to arrive
at a nearly exact solution.
>
> I realize one would expect the result to be deterministic and not fail
> in one case and pass in the other, but who knows how ARPACK is set up?
>From what I can tell from the API, ARPACK chooses a random initial seed
ONLY if a seed is not provided to them.  Other than this, the algorithm is
deterministic.
>
> Melvin, is there a way you could make some simple mod to the svds.m
> tests to indicate exactly which test(s) is failing.
The test which fails is recorded in the file test/fntests.log.  Melvin
could check this file and report back which test is showing an issue.  I
assumed it was the same ones we had trouble with before which depend on
ARPACK and UMFPACK.  If it is the test involving speye() then I agree that
just bumping up the tolerance just enough to allow it to pass is a fine
solution.

--Rik