Re: I volunteer

[Jacob Bachmeyer]

Rob Savoye wrote:
> On 5/18/20 2:16 PM, Jacob Bachmeyer wrote:
>   
> > My work on DejaGnu is one of my hobbies.  However, DejaGnu is a
> > relatively stable project, so I do not expect it to require much time.
> >     
>
>   To adequately test DejaGnu, you need to be able to build and test the
> GNU toolchain, both native and cross. That's necessary to test patches,
> or reproduce bugs. Once all that is setup, then yes testing patches
> isn't too time consuming. Building several toolchains and setting up
> cross testing is what sucks up most of the time. You don't need to be a
> toolchain developer, but you do need to be able to work in that environment.
>   
What is the minimal set needed for sufficient testing?  (Which 
packages?  GCC, GDB, ...?  How many versions?  Latest release/Past three 
releases/All supported releases/Something else?  I have/have access to 
AMD64 (with 32-bit compatibility available); otherwise I will need to 
ask for an account on the GCC Compile Farm Andrew Pinski mentioned.)

How to run the testsuites with a testing version of DejaGnu?  Is it as 
simple as specifying 
"RUNTEST=/path/to/testing/version/of/dejagnu/runtest" as an argument to 
"make check"?  Is running diff on the *.log (or *.sum) files after 
running "make check" with both "old" and "new" DejaGnu sufficient to 
verify a proposed release?

>   Luckily you can use my ABE tool (also GPLv3), which I built for
> Linaro, and it fully automates the process of building and testing
> native or cross toolchains.
It seems to have disappeared from the Linaro wiki... is it currently 
posted anywhere?

>  I agree you've been deeper into the DejaGnu
> code than anybody else these days, but unless you run the toolchain
> testsuites in a cross build from git, you can never really be sure if
> something doesn't break in obscure ways. Maybe others disagree with me
> on this, but testing the toolchain is the primary purpose of DejaGnu.
>   
Are released versions sufficient or do the tests also need to be run 
with Git checkouts of the toolchain packages?

>   I'm open to an additional co-maintainer if you can survive getting
> fully setup. ABE makes it easy, you just need a ton of disk space. While
> it is possible to add bug fixes or features without much toolchain
> testing, it's impossible to do a stable release without it. With
> DejaGnu, all releases are long-term support... and we will  need to do a
> release after the patch backlog is caught up.
>   
Part of the reason I am now involved with writing tests for DejaGnu 
itself is to reduce this burden.  Right now, the only way to know if a 
change causes problems is to run numerous testsuites for other (large) 
packages and "see if it breaks".


-- Jacob