[Axiom-developer] running a single test case

[daly]

In response to an offline user query....



Axiom includes a test suite in src/input which is a set of pamphlet files.

You can run an individual test suite. There are two lisp functions built
into the lisp image, "tangle" and "regress". These will eventually be
top-level Axiom commands. The Axiom ")read" command will also be able
to directly handle latex input so these steps will be even easier.

======================================================================
RUNNING A TEST
======================================================================

To run a test, type

  axiom 
  -> )lisp (tangle "/path/foo.input.pamphlet" "*" "foo.input")
  -> )read foo

This will "tangle" the test in "/path/foo.input.pamphlet", extracting the
chunk named "*", and put the tangled output into "foo.input".
Now you have an ordinary Axiom input file which you can read.

The test "foo.input" will create a file "foo.output"

Note well, the tests all have
  )lisp (bye)
as their last statement so Axiom will exit at the end of the test.
Remove this from the input file if you don't want this behavior.

======================================================================
REGRESSION TESTING THE RESULT
======================================================================

Once a test is run you can "regression test" it with the lines
 
  axiom
  -> )lisp (regress "foo.output")

This will check that every test worked properly and that every
test was run. A summary line is printed.

======================================================================
MORE DETAILS ON TEST STRUCTURE
======================================================================

Each pamphlet file is a latex document using the \begin{chunk} \end{chunk}
environment that is defined in axiom.sty.

Each test file has the structure:

  latex header
  \begin{chunk}{*}
    axiom setup
    numbered code blocks
    axiom shutdown
  \end{chunk}
  latex footer

The latex header gives some structure to the document, including author
names,  and sometimes includes an abstract.

The code is usually in one large chunk named "*" but the extraction
routine will concatenate chunks with the same name so it is possible
to insert latex between examples, as in:
    \begin{chunk}{*}
      code block a
    \end{chunk}
      latex stuff
    \begin{chunk}{*}
      code block b
    \end{chunk}
    ...
which will generate 
       code block a
       code block b
in the tangle output.

Each test is numbered sequentially using Axiom comment lines.
Each test is structured as:

--S n of M
some-axiom-input
--R
--R the expected axiom output
--E n

Notice that everything but the 'some-axiom-input' is an Axiom comment.
The Axiom comments have a letter as the first character.

--S n of M      (start test n, of M tests)
--R             (expect exactly this output text)
--I             (ignore this line, if the compare fails pass anyway)
--E n           (end test n)

======================================================================
CREATING YOUR OWN TEST FILES
======================================================================

These files are trivial to create. You just 
  )spool foo.output
and then run a series of Axiom commands. When you are finished
  )spool
will close the output file. Now edit the "foo.output" by wrapping
each test with the "--S n of M" and the "--R" on the output lines and
the "--E n" on the end of each test, insert the latex wrappings by
copying and modifying something like src/input/algagger.input.pamphlet
and you now have a valid test file.

You can add your test file to the test suite by modifying
src/input/Makefile.pamphlet
Just find every "algaggr" and follow the pattern to add your
test file to the appropriate tables.

Tim