[Gnu-arch-users] Re: in-tree pristines fatally wounded (merge-fest etc)

[Tom Lord]

    > From: Pau Aliagas <address@hidden>

    [Various things]


Rather than a point-by-point reply, I think it will be clearer just to
speak directly to the points you've raised.

A good goal, I think, might be this:

* goal

  Make it trivial for a new user to create an environment in which:

        ~ none of their project trees ever contain pristine copies
        ~ they have a library which operates sparsely
        ~ --hard-links works for them and its easy to extend their
          default environment to account for working on multiple 
          devices
        ~ they don't have to write or install any hook functions
        ~ if their library or libraries start getting too large,
          they can issue a single easy command to prune them back down
        ~ it's trivial to make hardlink project trees their default
        ~ all of this is done upwards compatibly

  I think we can do that:


* library paths

  Yes, I do plan to either merge/extend or simply rewrite library path
  support.

  In fact, as I've tried to write up these ideas for this message, it
  occurs to me that we can actually do without the hooks quite nicely!
  (I'll still keep the hooks -- it's just that I think with a
  sufficienly good path mechanism, the default behavior can eliminate
  99% of the need to ever actually use the hooks!)

  I think we, in fact, want _two_ library paths:  one for search-only
  and one for add-precedence.

  The total path when just searching for a revision is the
  concatenation of the search-only path with the add-precedence path
  (in that order).

  The total path when automatically adding a revision is the
  concatenation of the add-precedence path with the search-only
  path (in that order).

  (Many users will be happy with just one of those two paths but
  in some situations it is useful to say, in effect, "i prefer
  to add to low-priority libraries but take from high-priority 
  libraries.")

  When searching, if searching for --hard-links purposes, return
  the first revision found that is on the appropriate device;  
  otherwise, just return the first revision found.

  When auto-adding, add to the first of the following libraries
  found (on the appropriate device if for --hard-links purposes):

  ~ the first library on the path that contains the revision's 
    version

  ~ the first library on the path that contains the revision's
    branch

  ~ the first library on the path that contains the revision's
    category

  ~ the first library on the path that contains the revision's
    archive

  ~ the first library on the path

  I also think we therefor need some new commands:

        tla library-add-archive
        tla library-add-category
        tla library-add-branch
        tla library-add-version

  (and the symmetric `delete' commands).

  And how about a per-library or per-library-part flag that says
  whether --sparse is the default, along with some other parameters:

        tla library-policy [--every <N> | --decay] \
                           [--greedy] \
                           [--sparse=yes|no] \
                           <library>  \
                           [<archive>|<fqcat>|<fqbranch>|...]

        tla library-lock [--unlock] <revision> [<library>]

        tla library-prune [<library> [<archive>|<fqcat>|...]]

  where library-prune attempts to reduce the number of 
  revisions in (optionally: some part of) a library following
  the rules:

        ~ keep all locked revisions
        ~ keep all revisions whose link count indicates that
          there are hardlinks from other library versions or
          from project trees
        ~ within those constraints, get as close as possible
          to (depending on the policy) every Nth revision
          or an exponential decay from most recent to 
          oldest

  With those low-level mechanisms in place, we can imagine a 
  high-level convenience command.   Something like:

        tla project-mirror [params] <name> <project-spec>
  
  where the <project-spec> is a list of archives and/or
  fully qualified categories, branches, or versions and
  the params specify:

        ~ if and where to create local archive mirrors
        ~ if and where to add the archives, categories, etc.
          to particular libraries
        ~ what library caching policies (if any) to apply 
          to the archives, categories, etc.

  Perhaps then:

        tla sync-project <name>

  to issues all the necessary push-mirror commands and

        tla prune-project <name>

  to issue appropriate library-prune commands.

  Finally, about the --greedy option to library-policy: if some part
  of a library is greedy, that means that any demand at all for a
  pristine copy of some revision that can go in that library or
  library-part is satisfied by auto-adding to the library.  For
  example, if you have a greedy library, `what-changed' won't
  every build and then delete a "scratch copy" of some revision -- 
  it'll just auto-add it to the greedy library.

  So a default "new user" set-up can create a single library,
  make is sparse if that's the consensus, set it up for 
  exponential-decay caching-policy, and make it greedy.

  The net effect is that such new users will never have an
  in-tree pristine and `tla library-prune' will have a good
  default behavior for them.


* make --hard-links the default

  I don't think that that should be the default default but it should
  be easy for a user to make that a personal default:

        tla want-hard-links [never | sometimes | always]

  Never means nevery make a hard-link tree except if --hard-links is
  explicitly provided.

  Sometimes means to make hard-links if there is a handy library 
  revision on the same device, but otherwise just copy.

  Always means to always make hard-links.  If there is no library
  on the appropriate device, issue an error message advising the user
  to create one (and suggesting how).


* making --sparse the default

  Nah.  That's a coin-toss issue.  Neither sparse or non-sparse
  is the right default.   But I'm willing (see above) to make
  sparse the default for a particular library or part of a library.

  (Other than that, though, non-sparse wins as the default because
  it is upwards compatible.)


* how many library hooks?

  I don't care whether it is one or three but the hooks should
  take additional parameters which we can work out as we go along.


* "returning" a path from the library hooks?

  That just strikes me as awkward.   I think the library-path 
  search algorithm described above is simpler.


    > I completely agree that now we are ready to use these huge trees and be 
    > superior to other non-free alternatives: less space and more speed, what 
    > else can you ask for?

:-)

-t