RE: tar backup - inconsistent snapshot of repository

[Geoffrey Hird]

Mark,

One last (I think) question below.

> -----Original Message-----
> From: address@hidden [mailto:address@hidden On Behalf Of 
> Mark D. Baushke
> Sent: Tuesday, April 18, 2006 1:24 AM
> To: Geoffrey Hird
> Cc: address@hidden
> Subject: Re: tar backup - inconsistent snapshot of repository 
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Geoffrey Hird <address@hidden> writes:
> 
> > Thanks Mark.  That's good information.  A couple of questions.
> > 
> > > You may wish to exclude ,*, files from the backup
> > > tape as well sa the various kinds of CVS lock files.
> > 
> > What do you mean by ,*, files?  I haven't seen either ,*, or ,foo, .
> 
> A file foo,v when it is being committed has the new version 
> of the file
> written to ,foo, and then renamed as foo,v when the file has been
> written and closed without any errors. They should only exist for a
> brief moment before they are gone. However, a foo,v file that is more
> than a few hundred megabyte will often take a few seconds during which
> time doing a tar operation can have problems.
> 
> > > Or, you might consider using a LockDir= in the
> > > CVSROOT/config to put the locks into a separate
> > > directory. All of the ,v files should be
> > > self-consistent in this case.
> > 
> > Why would moving the lock files make a difference?
> 
> If a multi-directory commit is in progress when you are doing the tar
> and you later need to restore, the locks that were copied into the tar
> file will have been carried forward into the restored file with no
> process around to clean up the lock file. Thus, you will have a lock
> that will hang around until you manually clean it out.

I take it that the point of this is that if all lock files are in the
same place, it is easier to clean them out manually (or by a script)
than it would be if you had to search through the whole source tree.
Am I missing something?

gh  

> 
>       -- Mark
> 
> > Thanks again,
> > 
> > gh
> > 
> > > -----Original Message-----
> > > From: address@hidden [mailto:address@hidden On Behalf Of 
> > > Mark D. Baushke
> > > Sent: Thursday, April 13, 2006 12:48 PM
> > > To: Geoffrey Hird
> > > Cc: address@hidden
> > > Subject: Re: tar backup - inconsistent snapshot of repository 
> > > 
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > > 
> > > Geoffrey Hird <address@hidden> writes:
> > > 
> > > > Suppose:
> > > > 
> > > > 1. We are making a tar of the CVS repository as
> > > > 1. a backup.
> > > 
> > > You may wish to exclude ,*, files from the backup
> > > tape as well sa the various kinds of CVS lock files. 
> > > 
> > > Or, you might consider using a LockDir= in the
> > > CVSROOT/config to put the locks into a separate
> > > directory. All of the ,v files should be
> > > self-consistent in this case.
> > > 
> > > > 2. We do not lock users out from writing to CVS,
> > > > as the process takes 3 hours and there are users
> > > > in multiple time zones.
> > > 
> > > Sure. We actually use CVSup to create a read-only
> > > clone of the repository that is able to be backed
> > > up and find this to be a superior mechanism for
> > > large repositories... it also gives us a
> > > hot-standby copy in case the primary repository
> > > location has a hardware failure.
> > > 
> > > CVSup creates a read-lock for each directory in
> > > turn as it copies the deltas across, so each
> > > directory is self-consistent which is a good
> > > thing. It takes about as a much times as a 'cvs
> > > checkout' or 'cvs update' might take to perform,
> > > so it is fairly low overhead to your server.
> > > 
> > > > 3. A user commits a file during the tar so that
> > > > the CVS bookkeeping is inconsistent with the
> > > > committed contents.
> > > 
> > > Well, the CVSROOT/history file may not reflect the
> > > contents of the ,v files, but that is not too big
> > > a problem.
> > > 
> > > > 
> > > > I'd like to know what are the consequences from
> > > > the following point of view. Suppose our CVS
> > > > server dies (irrecoverable disk failure), and we
> > > > switch to a backup machine, and restore the
> > > > repository from the tarball of the previous day.
> > > 
> > > You will have lost the writes to the ,v files for
> > > new revisions, new tags, modified tags, deleted
> > > tags, deleted revisions, that sort of thing.
> > > 
> > > > Obviously we've lost a day's work, but that's
> > > > not the end of the world. If the snapshot fails
> > > > badly, it *is* the end of the world.
> > > 
> > > Yup.
> > > 
> > > > 
> > > > How bad can things get? 
> > > 
> > > Well, if you are backing up the ,*, files, then
> > > your users will not be able to commit to those
> > > files as they will be considered 'locked' by the
> > > restored tree. Likewise, if you have the various
> > > cvs.lock directories or the various transient
> > > locks in the tarball, you will need to clean those
> > > out before you can get back to a repository
> > > suitable for commits.
> > > 
> > > There could be the odd case of a file living in
> > > both the parent directory and being present in the
> > > Attic in the case of a 'cvs rm' having occured.
> > > So, you will need to look for that sort of thing.
> > > 
> > > > Can this make CVS crash?
> > > 
> > > No, it should not be a problem. Of course, if the
> > > ,v file itself was corrupted due to the root-cause
> > > that is crashing your machine this may not be
> > > true. In that case, you might need to restore that
> > > particular ,v file off of a previous backup.
> > > 
> > > > Will CVS continue to function, 
> > > 
> > > Yes.
> > > 
> > > > but give error messages about just the directory
> > > > in which the file in question was committed?
> > > 
> > > Probably not even that, depending on the nature of
> > > the operation that was in progress during the
> > > commit.
> > > 
> > > > Will CVS function with no error reporting, but
> > > > simply show the repository incorrectly from
> > > > users' perspectives?
> > > 
> > > Yes, this is possible. A 'cvs update' could see
> > > the most 'recent' revision of the users tree being
> > > lost.
> > > 
> > > > 
> > > > I'd appreciate any help.
> > > > 
> > > > gh
> > > 
> > >   Good luck,
> > >   -- Mark
> > > -----BEGIN PGP SIGNATURE-----
> > > Version: GnuPG v1.4.3 (FreeBSD)
> > > 
> > > iD8DBQFEPqr7Cg7APGsDnFERAs/aAJ9li34hjolnX6+A6gJ/5s3KSdVo4wCg6MBg
> > > piDmVOonLGpIeswCxa81Mbk=
> > > =tn8H
> > > -----END PGP SIGNATURE-----
> > > 
> > 
> > 
> > _______________________________________________
> > info-cvs mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/info-cvs
> > 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (FreeBSD)
> 
> iD8DBQFERKIPCg7APGsDnFERAqInAJ4vzz8El9lOXPcru9AJrjxE0ia7dgCgnUjc
> MSNdoln7uRuL+kXOOv6TVXg=
> =rKGh
> -----END PGP SIGNATURE-----
>