Re: automatic daily checker

[Graham Percival]

On Fri, Sep 23, 2011 at 01:30:47AM +0200, Reinhold Kainhofer wrote:
> Am Friday, 23. September 2011, 01:12:13 schrieben Sie:
> > - you're not building them from scratch
> 
> I am. I nuke the build dir before each build.

oh, good!  I didn't realize that.  :)

> > - you don't keep a record of which commits could build
> 
> Nope.

This key, IMO.  We have an average of 2-10 commits per day; when
master breaks next time, I wouldn't be surprised if we don't even
need to guess.  And if it's not obvious, then hey; it's a couple
of git bisects to figure out why.

> > - AFAIK you don't check the logs daily -- at least, I haven't seen
> 
> If a build fails, I get a mail from the cronjob (which contains also the ~200 
> last output lines from make). However, since the error message is usually 
> buried in thousands of lines of make commands and debug output, I usually 
> don't bother to even try to find the error message.

True enough.  That's what I'm trying to change.

The first step is to automatically identify a narrow range of
commits.  The next step (in the auto-detection direction, rather
than dev/staging) is to automatically extract the info from a
failing build (outside of make doc).  The final step is to improve
the doc build system so that we can get information from that.

> > This script is also the first step towards an automatic
> > dev/staging branch, but I want the basics tested before I move
> > ahead to work on that stuff.
> 
> I haven't looked at it, but Jenkins (http://jenkins-ci.org/) is about 
> continuous integration (i.e. automated builds to detect build failures or 
> regtest failures). It might be overkill for us, but still, there are 
> solutions 
> for exactly this task.

Hmm, Jenkins is java-based.  :)

I agree that there's stuff out there.  Maybe I'm suffering from
NIH syndrome yet again, but I think that a few 100-200-line python
scripts written by ourselves, specifically for our use case, will
end up being better for us.


Don't get me wrong; if somebody wants to pop in and say "hey, my
day job uses XYZ.  Give me half an hour and I'll set up exactly
what you want", great!  But plans for this type of system have
been floating around for months, and nobody's stepped forward with
such an offer.  Writing a set of python scripts like this is a
nice way for me to unwind, and I'm only expecting to spend a total
of 10-20 hours on this, so I'm happy with the potential NIH-ness
of this.

Cheers,
- Graham