[Gnu-arch-users] Re: designing how to kill pristine trees

[Pau Aliagas]

(long post: take a breath, get a coffee and a pencil to write down side
notes -or buffers for annotations for those emacs fans-).

After reading all the weekend posts, it looks clear that there are as many
opinions as people :), so I'm summarising what I understand are the pros
and cons of pristine trees and revision libraries so that we can discuss 
technical merits and possible goals:

comparaison
-----------
                        rev libs        pristine trees
operation efficiency
           creation     +               -
           get          +               -
           diff         =               =
           commit       +               -
space efficiency        +               -
space management        -               +


operation efficiency
--------------------
Four main operations have to be considered: creation, get (co), commit 
(ci) and diffs.

> creation
  --------
  Revision libraries are created faster than pristine trees because they
  only have to copy a fraction of the files, sharing usually an important
  part via hardlinks with the ancestor.

* possible improvements
  -for revision libraries:
   · (1) sparse revisions
  -for pristine trees:
   · using hardlinks to avoid the copy of all the files
     (2) a root directory to store the pristine trees (i.e. .arch-cache)

> get
  ---
  Gets are faster with revision libaries as they don't involve creation of 
  a library. Even when they do (get --links and no existing revision), 
  creation speed still applies.

* possible improvements
  -for revision libraries:
   · (1) sparse revisions
  -for pristine trees:
   · using hardlinks to avoid the copy of all the files
     (2) a root directory to store the pristine trees (i.e. .arch-cache)

> diff
  ----
  Once you have either a revision library or a pristine tree created, diff 
  operations that involve a working dir and one revision or one
  pristine tree, should run at the same speed, unless the working dir was 
  hardlinked to one of them.

* possible improvements:
  -for revision libraries and pristine trees:
   · (3) get --link

> commit
  ------
  Commit operations should be the same as they should involve only the 
  repository, but they are slower in pristine trees. Why? Because a commit 
  also updates the pristine, making double work. A commit does not ass a 
  revision to the library.

* possible improvements:
  -for pristine trees:
   · (4) eliminate the update of pristine tree and do it in a hook

space efficiency
----------------
Revision libraries are more space efficient than pristine trees (even more
with the sparse revisions patch) because they share, via hardlinks, the
unmodified inodes with all their ancestors, whilst pristine trees belong
to the working directory and don't share anything.
 
* possible improvements:
  -for revision libraries: none.
  -for pristine trees:
   · hardlinking them all as with revision libraries.
     (2) a root directory to store the pristine trees (i.e. .arch-cache)


space management
----------------
One general perception is that revision libraries are more "long-lasting"  
than pristine trees, because they stay "there", unnoticed, whilst pristine
trees appear and disappear with the working dirs. So it seems that
deleting and adding pristine trees comes almost for free.

The objective fact is that, functionally, they have the same goal: speed 
up things.

* possible improvements
  -for revision libraries and pristine trees:
   · (5) easier pruning of revision libraries and pristine trees


Possible improvements
---------------------
4 points have come out as possible improvements.


(1) sparse revisions
    ----------------
There's a patch available that implements the sparse revisions. What it 
does is to avoid the creation of all the ancestor revisions when one is 
added.


(2) a root directory to store the pristine trees (i.e. .arch-cache)
    ---------------------------------------------------------------
There's a consensus that pristine trees should share inodes among them.  
Now there's a manual optimisation in the code that looks for pristine
trees in the slibing dirs before creating them, but this is a
side-help-hack more than a definate solution.

Tom proposed a .arch-cache for this function.

* possible implementations
  (a) independent of revision libraries
      -the pristine trees wold share no links with revision libraries
  (b) links to revision libraries
      -the prisitn trees would be hardlinked to revision libraries
      -revision libraries would be created on demand
  (c) revision libraries
      -instead of using pristine trees, use revision libraries
      -revision libraries would be created on demand

* my opinion

  My opinion is that we should eliminate pristine trees and move to 
  revision libraries excusively.

  If we need temporary libraries and long-lasting libraries we could 
  consider adding multiple paths for them, to classify them by its a 
  priori endurance, but I think this would be over-engineering as it can
  be done now just removing them.


(3) get --link
    ----------
There's a patch available that implements get --link. What it does is to 
create a revision library, if it does not exist, and hardlink the 
extracted tree to it.

If the working dir is hardlinked to either the revision or the pristine 
tree, all the diff operation that involve identic inodes don't need to 
read the file.


(4) eliminate the update of pristine tree and do it in a hook
    ---------------------------------------------------------
This would be a speed-up, but maybe it would turn pristine trees useless. 
It can alwasys be done in a hook.


(5) easier pruning of revision libraries and pristine trees
    -------------------------------------------------------
The current implementation implies that pristine trees management is 
almost not necessary because they appear and disappear with working rdirs.

But if we move prsitine-trees to a root dir, as it seems it should happen, 
we have the same issues than with revision libraries.


We could add commands to prune old revisions easyly.

* possible implementations
  (a) add commands prune-libraries/prune prsitin-trees
  (b) offer scripts for hooks that offer the same funcionality

* my opinion:
  
  Let's go for the scripts: more flexible, less code to mantain and no 
  lose of funtionality.


That's all folks!
Pau