Re: Test harness patch for VMS.

[John E. Malmberg]

On 3/31/2014 12:22 PM, h.becker wrote:
> On 03/31/2014 01:53 AM, John E. Malmberg wrote:
> > This is the first cut at the patches for running the test harness on
> > VMS.  The test_make.com sets up the environment and runs the test.
> > As it makes persistent changes to the logged in session, I recommend
> > dedicating a logged in session to running tests.
> >
> > The "-help" option to test_make.com explains the usage for VMS.
> >
> > The updates to the individual test scripts will follow as separate patches.
> >
> > This has some temporary hacks in it to get around some known issues with
> > VMS that affects multiple tests that will need to be removed as the
> > issues are resolved.
> >
> > The test harness on VMS requires:
> >
> >    Perl 5.18 or later.
> >    PCSI kits available from
> >       http://sourceforge.net/projects/vmsperlkit/files/
> >
> >    GNV 2.1.3 or later.  GNV 3.0.1 has not tested with this script.
> >    Bash 4.2.47 or later.
> >    Coreutils 8.21 or later.
> >      http://sourceforge.net/projects/gnv/files/
> >    Read before installing:
> >       http://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/
>
> I appreciate your work on the tests with/for/on VMS. This wasn't done
> before and the tests obviously exhibit problems, features and
> limitations. (To me they also indicate that there isn't much or heavy
> usage of GNU make on VMS.)

I do not see much active use of it for the VMS mode.  There is quite a 
bit of work being done with the GNV fork, but the users are running into 
problems with bugs, and rather than trying to fix that older forked 
port, it seems like the thing to do go back to the master, especially 
since there seems to be some significant VMS maintenance being done.

> I tried the test_make.com but it failed. As
> far as I can see it checks whether the required components are present.
> And they seem to be available. Maybe I missed something, but it fails with:
>
> $   perl run_make_tests.pl -verbose
> %DCL-W-IVKEYW, unrecognized keyword - check validity and spelling
>   \-C\
> %DCL-W-IVKEYW, unrecognized keyword - check validity and spelling
>   \-C\
> %DCL-W-IVKEYW, unrecognized keyword - check validity and spelling
>   \-C\

For some reason perl is only able to run foreign commands that are 
global DCL symbols, not local DCL symbols.

In test_make.com this change is needed.

$!
$ testname = ""
$ make :== $bin:make.exe"
$!
$ i = 0
$param_loop:
$

The bin logical name is not visible outside of the test_make.com 
procedure, so to run the test harness outside of the test harness, the 
full path is needed.

The original intent was only to set local symbols for foreign commands 
so that the test harness did not leave them behind.  Unfortunately Perl 
does not seem to be passing these symbols to its child processes.

If it is desired to have the DCL procedure just set up for running the 
Perl suite, some logical names that it defines just before it runs perl 
should also be defined.

The error message was being generated because the make global symbol 
foreign command was missing.  I missed removing that definition before 
re-running the test_make.com

> ------------------------------------------------------------------------------
>                        Running tests for GNU make on VMS
>                                 GNU Make 4.0.90
> ------------------------------------------------------------------------------
>
> Also, with all the dependencies on GNV I wonder whether it makes sense
> to run this in/from DCL at all. With GNV and Bash etc. required,
> wouldn't it be better (and require less VMS specifc perl code?) to run
> it from bash? And, because it depends on all of this, I think the test
> driver should be named with a prefix "gnv".

Perl changes mode when run under Bash because it detects the bash shell. 
 It becomes more Unix like.  That still is a work in progress.  The 
next step is to get it to run child programs under the Bash shell 
instead of DCL.  Once that step is done, it may be possible to run 
automake on VMS.

The test harness patches, which I have not yet posted most of them, only 
expect the VMS behavior when not run under bash.

When the GNV for of GNU make is merged back in, the same test suite will 
be run with Perl under Bash to verify that behavior.  So the tests will 
end up being run twice.

The whole purpose of running the Perl under DCL for this round of tests 
is to fully document the existing VMS behavior and to make sure that 
there are no regressions when the GNV code is merged in.

But it looks like there are a number of features that should be common 
to both modes that need to be fixed first.

We will probably need to add some VMS specific tests to the suite.

I did the same regression tests with the Perl port to add in ODS-5 
support, while still allowing the old ODS-2 behavior.

Regards,
-John