Re: Test harness on VMS, running 512 tests, 68 tests failing.

[John E. Malmberg]

On 3/10/2014 11:00 PM, John E. Malmberg wrote:

Currently the status is 61 failures in 30 categories out of 511 test log
files found. I do not know why the category count changed from
just skipping one test in a file.

It also appears the some tests result counts are affected by the -keep 
qualifier since it leaves files around that other tests expect not to be 
there.  So there may be a few more or less failures.

With out -keep, 64 Tests in 30 Categories Failed, as an example, but 
then I do not get a count of all the log files.

5 archive tests fail, because that code is not completely implemented on 
VMS.  Ticket 41761.

1 override test fails because multi-line defines not correctly 
implemented on VMS.  Ticket 41813

Of the 9 parallelism tests, 4 work, 4 fail, and one is not current valid 
to run on VMS.
   1. Make recursion appears not to be working,
      at least with parallelism enabled.
   2. Parallelism is not working the same on VMS as UNIX.
      Needs more investigation.

The feature patternrules test 4 after being converted to use VMS syntax 
is exiting with a status of 5584176 or 0x03538110, which is an invalid 
exit code.  Need to find out where that is coming from.  It may be the 
result of the new vms_exit() function getting a VMS exit code instead of 
the expected UNIX exit code.

I am temporarily having the test harness interpret that as a 
MAKE_FAILURE since that is what the test expects until that issue is 
resolved.

Also need to document the VMS syntax for makefiles.  I can not find it 
in the readme.vms file or anywhere else in the checkout of master.  I am 
having to figure it out based on the logs from the tests and some trial 
and error.

Regards,
-John