Next Steps: CVS Import Bug - Solution Determination

[Oproescu Bogdan (KTXA 3)]

   Hi Lawrence and all, 

   Thanks for your message below. For your information, I tried parallel  
   CVS Tags on the SAME Module and Repository pairs on our CVS Central Server, 
   and that works correctly. I can reproduce File Corruption on CVS in only 
   the following cases: 

   1. Parallel "cvs import" and "cvs import" of SAME Module & Repository, and: 

   2. Parallel "cvs import" and "cvs tag" on SAME Module & Repository. 

   From our discussion so far, it seems the culprit is "cvs import", 
   because it's currently non-locking, and if this were fixed, I am 
   quite sure that the parallel "cvs import" and "cvs tag" would 
   also work correctly. 

   So, maybe your idea that you don't need to lock the entire directory tree 
   for "cvs import" like you do for "cvs commit" is a good one. Maybe then the 
   directory-at-a-time locking like for "cvs tag" will be sufficient to ensure 
   the File Corruption Issue doesn't repeat itself. In this case, can Derek's 
   solution below be implemented in just one pass? ie. lock directory, import 
   directory, release lock on directory. But then directories should be locked 
   top-first, and lower levels later, left-to-right tree traversal, and 
unlocked 
   in exactly the reverse order, because you should not release a lock on a 
directory 
   if any part of its sub-tree is still being imported, right? Otherwise 
another parallel 
   CVS Process may hop on and ask to import the top-level directory, and think 
it's 
   unlocked, and therefore try to import, while at the same time some subtree 
   in that directory is still busy being imported, which may cause further 
chaos 
   and File Corruption in CVS.  

   Cheers & Regards, Bogdan 

   Bogdan Oproescu   

   CREDIT SUISSE FINANCIAL SERVICES
   Technology & Services
   Advanced Middleware &
   Development Environments KTXA 3 
   Postfach 600
   CH-8070 Zürich
   Tel.: +41 1 334 6846
   Fax.:+41 1 332 8024
   E-Mail: bogdan.oproescu@credit-suisse.ch
   Internet: http://www.credit-suisse.ch/de/index.html

-----Original Message-----
From: lawrence.jones@ugs.com [mailto:lawrence.jones@ugs.com] 
Sent: Thursday, August 11, 2005 10:53 PM
To: derek@ximbiot.com
Cc: Oproescu Bogdan (KTXA 3); bug-cvs@nongnu.org; berndj@prism.co.za
Subject: Re: Next Steps: CVS Import Bug - Please Respond

Derek Price writes:
> 
> It shouldn't be too hard to fix.  In the import function in import.c,
> the import_descend and import_descend_dir need to be split into two
> passes.  The first pass should construct the directory strucure in the
> repository and save a list of the directory names.  It's mostly going to
> fit right into what those functions currently do in client mode, though
> the global repository variable may need to be deglobalized.  The list of
> files and directories to be imported could be cached at the same time,
> avoiding some of the overhead of the extra pass, but that's not necessary.
> 
> Then, the directory list needs to be used as an argument to Write_Lock
> (or lock_list_promotably for feature) when not on the client, just
> before the import data is traversed for the second time and actually
> imported.
> 
> During the second pass, as each directory is visited, the write lock
> will need to be grabbed on feature.  On both feature and stable the
> write lock will need to be released when each directory is finished with.

I don't know that there's any need to lock the entire directory tree for
import like we do for commit, directory-at-a-time locking like we do for
tag should suffice and be simpler to implement.  (The main reason commit
needs a tree lock is that it reads all the RCS files and caches the data
before starting to do any writes, so you have to have locks in place for
the entire operation.)

-Larry Jones

Life's a lot more fun when you're not responsible for your actions. -- Calvin