[Gnu-arch-users] an hack.. one night long

[wave++]

Quoting the WIKI, from 'About "archive conversion"':

  Second, it's not entirely clear that there is a great advantage in
  having your history "duplicated" into an arch archive. The history is
  still there, you just might have to use cvs to get at it.

At some point.. I needed that history back in arch. There are several
reasons for this, but mostly, its for convenience. So I decided to do a
"converter". It took one night long: I managed to stay awake for 26
hours: cvs2arch wasn't ready yet but I've then fallen aslept on the
keyboard. Today I fixed a couple of things and decided to share this
hack (requires cvsps >= 2.0 and tla):

  http://www.yuv.info/~wavexx/hacks/cvs2arch

BEFORE trying to convert your cvs repositories, here are some
assumptions you should consider:

 - cvs2arch won't convert branches. branches in the cvs sense are too
   complicated to convert in arch sense. The HEAD branch is followed.

 - cvs2arch will use cvsps (see freshmeat) to reconstruct atomic commits
   made on the cvs repository. atomic commits will be pushed in arch as
   single patchsets.

 - If you moved manually files in the cvs repository to "handle"
   renames, the conversion will fail. Delete+Adds will be kept.

 - .cvsignore files will disappear on arch history because of naming
   conventions. For the same reason, all files in the CVS repository
   MUST adhere to naming conventions (ie: no spaces in files).

   I've a couple of ideas about transforming these during the
   conversions instead of bailing out. I'll try to do something
   tomorrow..

= Conversion example:

  # extract your cvs module in a new directory
  cvs co hello-cvs
  # convert hello as hello--base--0.1 (using current default-archive)
  cvs2arch hello-cvs hello--base--0.1 hello-arch

cvs2arch will convert hello into hello-arch. If you're lucky enough,
hello-arch will contain an arch working copy of the project.

= Conversion examples:

To do some basic tests, I'm using my own cvs repository for wmnd. Here's
how "arch looks like" after the conversion:

  $ tla abrowse --desc wmnd--base--1.0
  address@hidden
    wmnd
      wmnd--base
        wmnd--base--1.0
  
          base-0    (initial import)
            Sat Aug 23 06:40:32 CEST 2003     wave++ <address@hidden>
            initial import
  
          patch-1    (simple changeset)
            Mon Jan 20 10:19:38 GMT 2003      wavexx <-- here starts the cvs
            Initial revision
  
          patch-2    (simple changeset)
            Mon Jan 20 10:23:59 GMT 2003      wavexx
            removing some autogenerated files
  
          patch-3    (simple changeset)
            Mon Jan 20 10:26:12 GMT 2003      wavexx
            more regenerated files
  
          patch-4    (simple changeset)
            Mon Jan 20 10:30:31 GMT 2003      wavexx
            even more regenerated files, adding ignores
  <cut>

I reconstruct the "summary" using the cvs log. If it's short enough, the entire
log is on the summary line. Otherwise is cut to 70 characters and put in the
log (avaible through cat-archive-log). Dates/Owners are restored. Tags will be
(read futher).

= How it works:

It's quite simple. First, I use the great cvsps tool to reconstruct patchset
informations out of cvs. This >considerably< reduces the number of commits made
to a cvs repository, as cvsps handles atomic commits.  I then "replay" the
informations contained in the cvsps output. I extract all files using the
versions contained in the cvsps patchset. I reconstruct a tree containing all
these files with the correct versions and tla-commit. I then reiterate for all
patchsets, adding and/or removing files in the tree as needed. This way, cvsps
and tla do the "hard" work.

= Notes:

To restore cvs file creators and date I put in the make-log file some
addictional fields like "Creator, Date, etc". As tla uses the last item found
in the log, this will work great.

Actually, I'd also like to restore tag names but I don't really know how to
tag the arch tree. Actually I was thinking about this:

  Cvs tag name: burp
  Arch project: project--name--0.1

  tla tag project--name--0.1--patch-XXX project--name-cvs-burp--0.1
                                                      ^^^^^^^^

This seems acceptable (what do you think?), but my version of tla (1.1pre5)
seems to be buggy:

  unable to open directory 
"/home/wavexx/archives/2003-acid/wmnd/wmnd--base-cvs-burp/wmnd--base-cvs-burp--1.0"
 (No such file or directory)

Note that the directory seems to be wrong anyway: it's missing the
category-brach block. Has this been fixed recently?

I also need to determine the lastest patch available.
Is there anything better than "tla revisions | tail -1"?

= Considerations:

Now that I have a converter I can make some comparisons. The first one is
about repository size. I've tried to convert a small repository that I have
under control:

  cvs repository: 2260k (565 commits)
  arch repository: 2556k (270 patchsets)

I'd say... a cvs repository isn't even compressed (duh). When doing a lot of
little commits, arch has more redundant data. I don't think compression does
make really any difference here.

Another point is speed. While continuosly committing, tla slows down
considerably. I know you can archive-cache, but this may be a problem (even
space-wise).

As I sayd before, I work in a team on work projects with around 40 people. If
we decide to migrate to arch, we'll certainly use a shared archive (would be
just stupid to use separate archives).  Under certain projects, we can manage
to do from a minimum of 10 of commits per day, and up to 100. In a couple of
days it will be very likely to get around patch-500.  arch @ patch-100 is
already two or three times slower (I haven't done any measures, but I've looked
at it during the conversion). And we can't cache revisions frequently because
of space.

Apart that.... comments? Ideas? Suggestions?
Considering that you convert once, please don't argue about speed :). Also,
DEEP-NIGHT-HACK warning applies.

-- 
'(wave++ "Yuri D'Elia" "http://www.yuv.info/";)