[Gnu-arch-users] Re: Testing approach

[Stephen J. Turnbull]

>>>>> "Misha" == Misha Dorman <address@hidden> writes:

    Misha> Samuel wrote:

    >> On Wednesday 26 November 2003 03:21 am, Andrew Suffield wrote:

    >> > So, now you know that somewhere in that 500 line chunk lies a
    >> > bug. Are you seriously suggesting that you can often spot it
    >> > by inspection?

    >> Not only am I suggesting it, I've done it ... hundreds of
    >> times.

    Misha> With only enough info to narrow it down to 500 lines? I'm
    Misha> surprised.

That depends on exactly what you mean by 500 lines.  Typically in that
500 line _contiguous_ chunk there are half a dozen 10-line chunks, all
similar in form, that you really need to look at.

As usual in this kind of thread, people are talking past each other.
I'm surprised you're doing it here, because most of the rest of what
you said makes an awful lot of sense.  Especially as a corrective to
Samuel's naivete.  :-)

    Misha> Not because it isn't possible -- I've done it myself
    Misha> (probably tens or hundreds of times, though that is over
    Misha> the last ~15 years) and usually on code written by other
    Misha> people -- but rather because I got the impression that your
    Misha> tests would be granular enough to narrow it down more than
    Misha> that. Typically, I would expect testing/DbC granularity
    Misha> sufficient to narrow bugs down to 20-50 lines or so at most
    Misha> (approximately, a method).

Samuel made that point already, actually.

    Misha> Just because you _can_ find bugs by inspection in 500-line
    Misha> stretches of code -- and perhaps even enjoy the challenge
    Misha> -- doesn't mean it is a good idea :-)

That's really Samuel's point.  Andrew says he does most of his
debugging in gdb.  To me, that suggests (1) his implementations are
extraordinarily accurate realizations of the design, or (2) he
programs mostly in a special domain (such as device drivers for new
and poorly specified hardware) where testing is hard to do accurately,
or (3) he does way too little testing.  I think (3) is most likely,
and that's really Samuel's point.

And of course sometimes inspection is a good idea simply because it's
really the _only_ idea.



-- 
Institute of Policy and Planning Sciences     http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.