Re: What is a 'program' in Section 15.3.3.1?

[Bob Friesenhahn]

On Wed, 27 May 2015, Arthur Schwarz wrote:

> In looking at tap-driver.sh there doesn't appear to be a place where a
> 'program' is accepted on the input command line. It appears that after all
> options are read if the input command line '$#' is not zero then an error is
> declared. So, is the TAP interface different from other Custom Test Driver
> interfaces?

The TAP interface is very different.  It depends on subordinate 
scripts printing 'ok' or 'not ok' (plus some optional extra diagnostic 
info) rather than return codes.  The subordinate scripts take full 
responsibility for the set of tests to be performed, what "programs" 
are invoked, and the options supplied to those programs.

For GraphicsMagick, I created a script which is sourced by actual test 
scripts (".tap" scripts), which provides generic functions used by 
those scripts.  The functions support pass/fail detection based on 
program exit status, but also support 'xfail' by being passed a set of 
feature id strings which are matched which the actual available 
features.  If the test fails, but a necessary feature is missing, then 
an extra diagnosis is added which Automake's TAP script will turn into 
an informative XFAIL message:

   not ok # SKIP requires foo support

See the implementation at

http://hg.code.sf.net/p/graphicsmagick/code/file/b84df92f710a/scripts/tap-functions.shi

and

http://hg.code.sf.net/p/graphicsmagick/code/file/b84df92f710a/tests/common.shi

and

http://hg.code.sf.net/p/graphicsmagick/code/file/b84df92f710a/tests/rwfile.tap

Bob
--
Bob Friesenhahn
address@hidden, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/