Re: Tests, Documentation

[graydon hoare]

Dalibor Topic <address@hidden> writes:

> >     - tests can be added inline with a source file, or "right next
> >       door" in a similarly-named class or package.
> 
> only if you want mauve to have GNU Classpath specific features.

I guess my point is I'd like to lower, as far as possible, the cost of
writing such a class, and putting it inline with the classpath code
lowers the cost further (no need to do a separate "cvs add", open a
new file, copy the legal boilerplate, etc.)

my gut feeling (I don't mean to be insulting or dismissive, just
observing) is that few people -- and progressively fewer as time
passes -- will have a use for mauve outside testing classpath. if
they're a proprietary vendor, they'll buy a sun compatibility kit;
everyone else seems, from my limited perspective, to be slowly coming
around to the benefits of just throwing their weight behind classpath.

> >     - it uses simple introspection conventions rather than scanning
> >       text in a source tree and writing makefiles for itself
> 
> java.lang.reflect? I think that one's out of question for tests for
> java 1.0. But I agree that reflection could make writing some tests
> easier. I was thinking about using reflection for generating tests for
> getter/setter pairs, for example.

is 1.0 testing -- or rather, running on a non-reflect-able JVM --
really an issue? I appreciate the air of openness and compatibility
suggested here, but some aspects of introspection are pretty handy,
and can help automate a lot.

> >     - comparability, equality, hashing, toString, etc.
> 
> I'm dealing with equality, hashing & toString atm, and hope to have
> something to discuss on mauve-discuss based on some tests I'm writing
> for FieldPosition.

cool!

> >     - all pairs / all n-tuples test reduction
> 
> test reduction? could you elaborate?

http://fit.c2.com/wiki.cgi?AllPairs
http://burtleburtle.net/bob/math/jenny.html

> >     - noise-tolerant boundary / range checking
> 
> again, culd you elaborate some more on that?

oh, this is actually a mental fudge; I'm really describing two things
here, neither of which is terribly high-tech:

  - utility functions which know how to explore boundary conditions
    for various data (maximum and minimum integral sizes, successive
    powers of two, edges of floating point representations, etc.) and
    iterate tests "near" boundary conditions (+1 and -1, >>1 and <<1,
    +epsilon, -epsilon, etc.)

  - utility functions for checking results "near" ideal values, within
    tolerance ranges, which you need for many things using floating
    point, anything which tests speed or memory use, or anything 
    with nondeterministic results (say, related to threading).

> >     - coverage and profiling analysis
> 
> yes! when I write a mauve test, I'd like to know what code paths I've
> missed. That should be possible to automatize.

indeed it is, there's a JUnit extension which does decision coverage
(not quite as powerful as complete path coverage, but tractable) using
BCEL, seems quite friendly: http://hansel.sourceforge.net/

> Yep. And it is not so much fun to write tests in, if you have to do
> much of the above by hand for every other test you write.

that's the thing. test-writing has to be made to feel at least as
engaging as code-writing, if it's to be successful.

> That's only the case if a class is badly designed and has a ton of
> methods, fields & constructors. Of course, the JDK API is full of such
> things. ;) But that doesn't mean that our implementations need to be
> equally messy.

well, they do if they conform to the public API. I guess I was just
responding to sascha's demand for each and every method to get a
javadoc comment; personally I feel this is a waste of our limited
time, and am anyways much happier as a user when I open a class to see
a multi-paragraph block at the beginning, than one with lots of little
fiddly document fragments that are uninformative, or repetitive.

> Example: A closed stream can not be read from. That's easy to see from
> the code, usually. But try to see what some methods in
> GregorianCalender are supposed to do just by reading the code. ;) I
> assume that understanding the Java2D code, if you're not proficient in
> 2D graphics, would not be trivial either.

exactly my point: I think the user should get a short education in 2d
graphics programming when they look up Graphics2D, not a list of
method-by-method details. I'm happy to write the former, less so the
latter.

-graydon