bug#47382: runtest doesn't work with Solaris 10 /bin/sh

[Jacob Bachmeyer]

Rainer Orth wrote:
> Jacob Bachmeyer <jcb62281@gmail.com> writes:
>
>   
> > > However, there are more errors still:
> > >
> > >                 === launcher Summary ===
> > >
> > > # of expected passes            5
> > > # of unexpected failures        45
> > > # of unsupported tests          2
> > >   
> > >       
> > Can you post the launcher.log file?  The dejagnu script is fairly 
> > simple, and I suspect that I may be able to deduce the causes of those 
> > failures, especially if they are like the "report-card" failure that 
> > follows.
> >     
>
> After the fix for the expr foo : - issue, I've made some progress in
> identifying what's still going wrong:
>   

Many thanks for your efforts.

> I now see
>
> Running "env EXPECT=true TCLSH=true 
> /vol/gcc/obj/dejagnu/dejagnu-1.6.3-branch/10/testsuite/launcher.all/command/bin/dejagnu 
> foo -v -v" ...
> Verbose level is 2
> Running launcher from 
> /vol/gcc/obj/dejagnu/dejagnu-1.6.3-branch/10/testsuite/launcher.all/command/bin
> Probing directory 
> /vol/gcc/obj/dejagnu/dejagnu-1.6.3-branch/10/testsuite/launcher.all/command/share/dejagnu/commands
> Looking for commands in 
> /vol/gcc/obj/dejagnu/dejagnu-1.6.3-branch/10/testsuite/launcher.all/command/share/dejagnu/commands
> ERROR: could not resolve command dejagnu-foo
> child process exited abnormally
> FAIL: dejagnu foo as Tcl
>
> Comparing sh -x output between Solaris 10 and 11, I found that the
> difference starts in dejagnu at this point:
>
> # Remove any leading autoconf platform prefix and the "dejagnu" prefix.
> command=`basename "$0" | sed -e 's/^.*-\?dejagnu-\?//'`
>
> * S10 or 11 with /usr/bin/sed:
>
>   command=dejagnu
>
> * S11.3 with /usr/gnu/bin/sed:
>
>   command=
>
> The issue is the Solaris sed vs. GNU sed difference: /usr/bin/sed
> behaves identically between Solaris 10 and 11, however GNU sed isn't
> bundled with Solaris 10.
>
> Solaris sed doesn't support ? in REs (cf. regexp(7)).  The Autoconf
> manual documents
>
> Portable @command{sed} regular expressions should use @samp{\} only to escape
> characters in the string @samp{$()*.0123456789[\^n@{@}}.  For example,
> alternation, @samp{\|}, is common but Posix does not require its
> support, so it should be avoided in portable scripts.  Solaris
> @command{sed} does not support alternation; e.g., @samp{sed '/a\|b/d'}
> deletes only lines that contain the literal string @samp{a|b}.
> Similarly, @samp{\+} and @samp{\?} should be avoided.
>   

At the time I wrote that script, (my first foray into an attempt at 
portable shell) I was using an old Autoconf manual that did not mention 
these issues; I was also trying to avoid using Awk, but DejaGnu 
configure now fails with an error if Awk is not available, which was not 
the case then.

> One can use \{0,1\} instead, but I cannot tell for certain how portable
> that is:
>
> diff --git a/dejagnu b/dejagnu
> --- a/dejagnu
> +++ b/dejagnu
> @@ -147,7 +147,7 @@ if $want_version ; then
>  fi
>  
>  # Remove any leading autoconf platform prefix and the "dejagnu" prefix.
> -command=`basename "$0" | sed -e 's/^.*-\?dejagnu-\?//'`
> +command=`basename "$0" | sed -e 's/^.*-\{0,1\}dejagnu-\{0,1\}//'`
>  
>  while expr $# \> 0 > /dev/null
>  do
>   

The current Autoconf manual also mentions that basename is not portable; 
I will find a solution, probably using Awk.

> With that fixed I ran into:
>
> Found subcommand foo with variants:  tcl sh
> grep: illegal option -- q
> Usage: grep -hblcnsviw pattern file . . .
> Selected variant tcl
>
> grep -q isn't portable, too, also documented in the Autoconf manual:
>
> Some of the options required by Posix are not portable in practice.
> Don't use @samp{grep -q} to suppress output, because traditional @command{grep}
> implementations (e.g., Solaris) do not support @option{-q}.
>   

Another improvement not in the older copy I used when writing that script.

> Using I/O redirection instead got me way further:
>
> diff --git a/dejagnu b/dejagnu
> --- a/dejagnu
> +++ b/dejagnu
> @@ -235,7 +235,7 @@ if $have_gawk ; then
>  fi
>  # is "awk" actually GNU Awk?
>  if $have_awk ; then
> -    if "$awkbin" --version | sed 1q | grep -qi 'GNU Awk' ; then
> +    if "$awkbin" --version | sed 1q | grep -i 'GNU Awk' > /dev/null; then
>         have_gawk_as_awk=true
>      else
>         have_gawk_as_awk=false
> @@ -406,8 +406,8 @@ if $want_help ; then
>         echo ERROR: file "'$help_file'" is not readable
>         exit 2
>      fi
> -    if grep -q '#help' "$help_file" \
> -       && grep -q '#end' "$help_file"; then : ; else
> +    if grep '#help' "$help_file" > /dev/null \
> +       && grep '#end' "$help_file" > /dev/null; then : ; else
>         echo ERROR: file "'$help_file'" does not contain a help message
>         exit 2
>      fi
>   

The help logic will probably get rewritten to use Awk, and the test for 
GNU Awk is another simple string match.  I think I can improve these a 
bit...

> Now I'm down to
>
> Running 
> /vol/src/gnu/dejagnu/dejagnu-1.6.3-branch/local/testsuite/launcher.all/command.exp
>  ...
> FAIL: dejagnu --help works
> FAIL: dejagnu foo --help works if shell variant selected
>
> This only occurs when running runtest --tool launcher manually, but
> works with make check.
>
> Running 
> /vol/src/gnu/dejagnu/dejagnu-1.6.3-branch/local/testsuite/launcher.all/help.exp 
> ...
> ERROR: The 'man' command in Solaris does not work in the source tree.
> Running 
> /vol/src/gnu/dejagnu/dejagnu-1.6.3-branch/local/testsuite/launcher.all/interp.exp
>  ...
> FAIL: have no Awk
> FAIL: have no GNU Awk
> FAIL: have no Tcl
> FAIL: have no Expect
>
> Running "env AWK=bogus GAWK=bogus 
> /vol/src/gnu/dejagnu/dejagnu-1.6.3-branch/local/dejagnu --DGTimpl awk" ...
> child process exited abnormally
> FAIL: have no Awk
>
> Those FAILs happen because interp.exp expects exitcode 1 while we get
> 255 instead.  The autoconf manual documents this, too:
>
> Don't expect @command{false} to exit with status 1: in native
> Solaris @file{/bin/false} exits with status 255.
>   

That one is in the older Autoconf manual; I just completely overlooked 
it and have fixed it in commit 
9539a1e2ffe8506b92bfcb4363c767e4bc6a0700.  The dejagnu launcher now 
reliably returns code 1 instead of whatever false(1) produces.

> > >                 === report-card Summary ===
> > >
> > > # of unresolved testcases       2
> > >
> > > Running /vol/src/gnu/dejagnu/dejagnu-1.6.3-rc2/testsuite/report-card.all/onetest
> > > .exp ...
> > > spawn /bin/sh -c cd /vol/gcc/obj/dejagnu/dejagnu-1.6.3-rc2/testsuite/report-card
> > > .all/onetest && exec /vol/src/gnu/dejagnu/dejagnu-1.6.3-rc2/dejagnu report-card^
> > > M
> > > expr: syntax error
> > > ERROR: could not resolve command dejagnu-report-card
> > >
> > > There are obviously more hardcoded uses of /bin/sh here and elsewhere,
> > > but I haven't looked for those yet.
> > >       
> > That hardcoded use of /bin/sh only changes directory and execs 
> > $LAUNCHER, which should run with /bin/ksh if you have patched its #! line.
> >
> > The "dejagnu" script is run, but fails to locate the report-card 
> > subcommand.  I would be interested in the output of "/bin/ksh -x 
> > ./dejagnu report-card" in the source directory, assuming that "-x" 
> > produces an execution trace from ksh as it does from bash (... and 
> > likewise for Solaris 10 /bin/sh after applying the patch above to change 
> > the use of the readonly command).  There are two likely candidates I see 
> > for this error:  one is an `expr :` match to detect a leading "-" and 
> > the other is a possibility that `expr $# \> 0` in a while loop test is 
> > somehow being executed as `expr \> 0` under some condition.  Both of 
> > these hypotheses imply unexpected behavior if not outright bugs in 
> > Solaris 10.
> >
> > There is a possibility here that dejagnu may be tickling a bug in 
> > Solaris 10 /bin/ksh and may work with /bin/sh, or may not work with 
> > either of them.
> >     
>
> When I retried this now, the report-card results are good:
>
>                 === report-card Summary ===
>
> # of expected passes            245

These results are very reassuring:  the report-card tool is written in Awk.


-- Jacob