[Monotone-devel] Re: Summer of Code kick-off

[Timothy Brownawell]

So, um, I'm tbrownaw on IRC and I've been around here a while now. I'm
also a student at UIUC [http://www.ece.uiuc.edu/], graduating this
december (yay!).

For SoC I'm working on rewriting our testsuite in lua, in
n.v.m.lua-testsuite . So far I've got the first 27 tests, which now take
~50sec instead of the ~150sec they take with the old testsuite. Started
about a week ago, will end when it's done (hopefully by late August).

I've had to add some platform-specific code, for setting env vars and
redirecting the standard filedescriptors. These are currently in
tester.cc, would they be useful enough in monotone proper (say, for our
lua hooks there) that we'd want them as part of libplatform.a?

Right now, I'm trying to get it to compile in VC8. I've been adding
functionality as needed by individual tests; I think all that's left is
background process support (for netsync). That's next, then back to
translating more tests.

Cool new features so far:
    * each test is a directory, not a single file. There's a command to
      copy supporting files into the scratch directory.
      (getfile("name-in-test-dir" [, "name-in-scratch-dir"]))
    * no "make testsuite" each time you tweak a test.
    * xfail applies to a specific command, not the entire test. If an
      xfailed test fails in the wrong place, it'll tell you.
    * you can specify the stdin for a command the same way you specify
      the expected stdout/stderr.
    * tests can be specified by regex (against the test name) or number.
      Negative numbers count backwards. (./tester -1 is good for
      checking a new test you've just added)
    * seems to be about 3x faster on Linux. There's probably a bigger
      improvement on Windows, since autotest uses pretty excessive
      fork()ing.
    * failed commands log a backtrace, showing exactly what it was
      doing.

I've also tried to keep nice things from autotest like individual
scratch dirs, -d to not clean up, and detailed logging.

Tim