Re: Attic headaches.

[Kaz Kylheku]

In article <address@hidden>, Greg A. Woods wrote:
>[ On Thursday, August 23, 2001 at 22:08:01 (GMT), Kaz Kylheku wrote: ]
>There's also the very real issue of tagging.  From sanity.sh:
>
>          # Test 5 reveals a problem with having symlinks in the
>          # repository.  CVS will try to tag both of the files
>          # separately.  After processing one, it will do the same
>          # operation to the other, which is actually the same file,
>          # so the tag will already be there.  FIXME: do we bother
>          # changing operations to notice cases like this?  This
>          # strikes me as a difficult problem.  -Noel

I've seen this happen all the time; it works just fine. What happens is
simply that CVS sees that the tag exists already and will report ``not
moving the tag to <wherever>''.  The sanity.sh comment isn't clear
on what the actual problem is. It's normally okay to try to create a tag
which is already there. Under what exact circumstances does it not work?

Note that the only symlinks I have go across directories; I don't have
any symlinks within the same directory. Maybe there is a problem within
the same directory?

I've been doing symlinks in a large repository for a year and a half
without difficulties. Lots of developers, lots of branching going on
and so on.

>Symlinking files though, especially between directories, is a certain
>recipe for eventual disaster.

You're too pessimistic. I have daily backups going back three months. ;)

>> Suppose you are tracking some outside sources that are arranged
>> in one way, but in your repository, the files are moved around a bit.
>
>I use CVS import.  Have since CVS-1.4alpha.  It's improved a lot
>recently w.r.t. supporting file death and resurrection on the vendor
>branch, but I've NEVER ever even contemplated trying to use symlinks to
>do such a thing!  UGH!  YUCK!  What ever possesed you to try such a
>twisted idea?

A twisted mind, clearly. Handling death and resurrection isn't good enough.
If the external project has a file called foo/bar.c  but in your version
it's moved to xyzzy/bar.c, you still want the vendor patches that are done to
foo/bar.c to continue to be applied to xyzzy/bar.c. So, in goes the symlink,
creating a handy little ``wormhole'' for the patches to be funneled
to the new home.  :)

One day, you decide to stop accepting the changes from the external
source, You go in, break the symlink so there are two distinct copies
of bar.c,v in the repository. (Of course, you do this with manual locks
created in both directories).  Once the link is broken, You do a
cvs remove on ``foo/bar.c''.  It's now dead, existing only in its new
location, yet you can recover the past tagged releases. ``xyzzy/bar.c''
lives on.

This was basically done just to ease a migration pain of some projects
into CVS. For a while, some people were still working on an old version
of the project outside of CVS, but the project in CVS was rearranged:
like some source files migrating out of individual directories and
going into a library.

Without the technique, CVS would not know that xyzzy/bar.c is
the same object as foo/bar.c; the vendor imports would not successfully
apply the changes for you. So then you would be looking at error-prone
manual patching. Manual patching has its risks too! 

>> So what you do is keep a directory tree that has the original shape,
>> and then connect things with symlinks. So you can keep doing 
>> imports of the outside source, and the patches to the moved
>> files will go to the right place thanks to the redirection of the
>> symlinks. When you no longer need the outside source, you can break the
>> links, by duplicating the files.
>
>That's so inside-out, upside-down, and backwards that I wouldn't be

It's so twisted that it works like a charm! Basically, it handles 95%
of the cases where you'd otherwise need directory structure versioning
like Clearcase. I'd do the same thing again if I had to.

>surprised if you've now tied your repository into so many twisty little
>knots that you'll never be able to straighten out and do things the
>right way without starting over.

Nope; it's perfectly clean, in good health. I just have a small handful
of symlinks left, according to ``find . -type l''. I didn't allow just
anyone to make symlinks; they are closely monitored. If someone wanted
a new symlink in the repository, they have to pay me fifty bucks and
provide a very good reason.

None of the few remaining symlinks are related to this technique at all,
they are just very rare instances of braindamaged file sharing. I'll
get rid of them in due time.

>You've done the wrong things and gotten yourself into the above scenario
>for all the wrong reasons.  You'll not be able to manage things properly
>until you undo the mess you've created.

The only scenario I'm in is that of a satisfied CVS user.

What disturbs me somewhat that you don't trust the symlink handling code
which places a lock in the right place.

Note that I did tests on it with 1.10.7 and I found that it does work
to a reasonable degree of confidence. (Which is all we can have about
any feature of any software, without formal proof methods, right?)

I created various repository scenarios, and I performed operations
on it. One approach was to capture the strace output, to analyze what
filesystem accesses cvs was performing. I satisfied myself thus that the
lock files related to symlinked files were being created in the right
places, etc.

Another testing strategy I followed was to make ``fake'' locks by hand
in the directories which house the files that are linked to, and then do
some operation in the directory which has the symbolic links. I verified
that the operation is blocked, waiting for the lock to be released in the
target directory where the symlinked file lives.

Lastly, I tested the branching and tagging too and all that, to make sure that
making multiple tagging passes on the same file does the right thing.

So as you can see, I did not enter into the use of symlinks with my
eyes closed. There was reasonable preparation, testing and planning.
The only issue I ran into with symlinks was the Attic move.

I don't like symlinks in the repository, but they have been a kind
of necessary evil.  Anything that requires manual intervention in the
repository is a bad thing because the general user population cannot
(and must not be allowed to) maintain it. So they are not part of some
long-term nefarious plan to twist the repository into knots and loops
that can never be unravelled again!