Package installability and testing (was: Re: GSoC application deadline passed)

[Michael Banck]

(CCing debian-hurd as this is really Debian-specific now)

On Tue, Mar 18, 2008 at 01:15:27PM +0100, Michal Suchanek wrote:
> On 18/03/2008, Samuel Thibault <samuel.thibault@ens-lyon.org> wrote:
> > Michal Suchanek, le Mon 17 Mar 2008 16:34:42 +0100, a écrit :
> >
> > > On 17/03/2008, Samuel Thibault <samuel.thibault@ens-lyon.org> wrote:
> >  > > Arne Babenhauserheide, le Mon 17 Mar 2008 12:26:30 +0100, a écrit :
> >  > >
> >  > > > > As for automatically building live CDs and/or qemu images, this 
> > would be
> >  > >  > > very useful -- maybe that part is indeed an appropriate task for 
> > GSoC.
> >  > >  > > But as others pointed out, there are often issues with building a
> >  > >  > > working system that require manual intervention; so it's 
> > questionable
> >  > >  > > how far this process can really be automated... I think this 
> > needs some
> >  > >  > > more consideration.
> >  > >  >
> >  > >  > Maybe some people with more background knowledge could add their 
> > feedback
> >  > >  > there.
> >  > >  >
> >  > >  > Would it be possible to simplify the process _a lot_ with the right 
> > tools?
> >  > >
> >  > > The problem is that there is no easy automatic process: missing
> >  > >  dependencies have to be found in the archive, etc.
> >  > >
> >  > What kind of archive? Shouldn't Debian just keep the packages until
> >  > new ones are built?
> >
> > Debian doesn't wait for non-official architectures to catch up.
> 
> They do delete Hurd packages when there are no new ones to replace
> them? I can usually see different versions of packages for different
> architectures.

There are several cases to consider here.

1. hurd-i386 binary packages which are outdated with respect to the
source package in unstable, but otherwise installable.  Those are not a
problem to hurd-i386 users, but are a problem for the Debia project and
its mirrors, as they possibly keep outdated source packages from being
purged from the archive (unless the same source version is in testing
or equally outdated on another architecture) and thus consume valuable
mirror space.

2. hurd-i386 binary packages which are outdated with respect to the
source package in unstable, and are uninstallable.  Usually, this is due
to a binary-independent binary package being built by the source
packages which has a strong dependency on one of the binary-dependent
packages (e.g. some -dev packages just ship headers these days and are
Arch: all, but depend (by policy) on the exact version of the Arch: any
library package).  If the hurd-i386 autobuilder is lagging behind, or if
there is a regression in the package so it no longer builds, those
packages become uninstallable, which is a problem for hurd-i386 users.

3. Library transitions changing package names.  The old library package
currently gets semi-automatically purged from unstable for all
architectures.  If hurd-i386 has not yet built the new library and its
reverse-dependencies, those reverse-dependencies will be uninstallable
for hurd-i386 users.

> > The problem is to determine automatically what has to be kept.
> 
> Everything. 

First off, for case 2 above, it is not a problem of keeping hurd-i386
binaries around.  They /are/ around, just not installable.

> There aren't that many packages for the Hurd. Plus keep all versions
> of packages on some "hurd-core" list until somebody manually marks a
> newer version as verified working.

The solution to the above problems is setting up a testing distribution
for hurd-i386.  This is not realistic to have on ftp.debian.org right
now, so somebody would have to invest time and energy into this.  It
would very likely be no problem to use ftp.debian-ports.org for this,
but it is unclear whether you can just reuse the .debs from
ftp.debian.org then, or need a full hurd-i386 mirror there.

And then somebody has to constantly monitor the packages and transition
them to testing, a process which happens only semi-automatically.
Shadowing Debian's official testing might make things easier, and
collaborating with other unofficial architectures (m68k, kfreebsd-*,
etc.) would probably as well, but it will still be a time-consuming
task.

If we have a testing distribution, mastering CD snapshots from it should
be much easier than what we do currently.


Michael