Re: Build Processes

[James Youngman]

Bil Joga <address@hidden> writes:

> I am interested in  hearing how companies use CVS as it relates to
> their build process. 

Well, obviously CVS is orthogonal to the build process.  But it may
have a relationship to the _release_ process. 

> For example, a company may choose to take a snapshot (tag) of the
> code, and then build based upon that tag. Another may choose to
> build off of the trunk, and then tag the code whenever an error-free
> build is achieved. Even though these are simplistic, both are not
> without drawbacks, but I'm interested in hearing how you do it, and
> learning about the "process" that is involved in order to achieve
> maximum results for your company. 

For us ...

Two shell scripts, 

make-source-reelase module version
make-binary-release module version 

There are more, but they don't add much to this exposition.


make-source-reelase does something like this :-
  
  unset t &&
  co co $module && cd $module && co tag $version && cd .. && 
  rm -rf $module &&
  t=$(mktemp build.XXXXXX) && mkdir $t && 
  cd $t &&
  cvs export -r$version $module &&
  tarf=/var/releases/$module/$version/$module-$version.tar &&
  tar cf $tarf $module &&
  gzip -9 $tarf
  test x"$t" = x || rm -rf $t
        
        
make-binary-release is a bit like this :-

  t=$(mktemp build.XXXXXX) && mkdir $t && 
  tarf=/var/releases/$module/$version/$module-$version.tar &&
  cd $t && gunzip < $tarf | tar xfB - &&
  cd $module && 
  ( test -f transform.ksh && ksh transform.ksh || true ) &&
  ( test -f configure && ./configure           || true ) &&
  make &&
  ( if test -d tests; then make check; else true; fi ) &&
  ti=$(mktemp /tmp/install-dest.XXXX) &&
  make install INSTALL_PREFIX=$ti &&
  bd=/var/releases/$module/$version/$(uname -s) &&
  bf=$bd/$module-$version.tar
  ( cd $ti && tar cf $bf . && gzip -9 $bf )
  test x"$ti" = x || rm -rf $ti


The real scritpts do extra magic (and are more readable), for example
generating md5dums of all files in the release, and producing reports
of what files changed between releases (using cvs rdiff -s for source
releases and a seprarate script with join and diff of the md5 lists
for binary releases).
    
The downside is that currently make-binary-release has to be run
manually on each build platform, and that doesn't scale well beyong
one supported architecture.  

Also, not all the files resulting from make-binary-release actually
qualify as external (i.e. final) releases.  Some are just internal
snapshots which get pushed into the later stages of the testing
process. 

> Some do's and dont's would be nice to hear. An explanation of your
> use of branching and merging, and the coordination between
> developers and the build master would be helpful. 

We tend to hack on branches and fix on the trunk.  Committing onto the
trunk is a controlled activity (but no support from CVS for automating
that, we just use a big stick).  Maintenance releases come fromt the
trunk.  Enhancement releases at the moment are the result of
test-on-branch-merge-onto-trunk-snifftest-commit-regression-test-on-trunk. 

Namespace control is applied to tags -- certain forms of tag name are
reserved for certain purposes.  We also tend to merge from trunk to
branch during extended enhancement activities, to place a cap on the
amount of effort involved in merge-onto-trunk-and-release.  This
requires care.

The principal release engineer is also responsible for approving
bugfixes at the design and commit stages, and is responsible for QA
(but not usually the actual testing) of the releases.  This is bound
to be an efficiency win, if your proect is small enough to make this
practical, and the release engineer has sufficient seniority and
expersie to make good decisions and have them stick.

Some of the steps in our release process are application specific; for
example with databases we also produce update scripts and we have a
separate testing activity that compares the result of a fresh build of
the old code followed by applying the upgrade script with a fresh
build of the new code.  If the result of these is not identical (and
we have automated this test) the software is not released.

-- 
James Youngman
Manchester, UK.  +44 161 226 7339
PGP (GPG) key ID for <address@hidden> is 64A95EE5 (F1B83152).