Re: Import large amount of files on Windows (but cvsnt has issues)

[Todd Denniston]

Wurdock, Tom wrote, On 09/22/2008 11:55 PM:
> Hello all,
>
> I have 450,000 files in a folder that need to be imported into cvs.  The
> server is linux.  We're migrating from another source control system.
>
> The files were developed on Windows and have windows eol's.  When I try
> to import anything "cvs import" I get a bad error:
>
> cvs [import aborted]: received abort signal
> cvs: /usr/local/src/cvs-1.11.21-v3/src/import.c:610:
> process_import_file: Assertion `entdata->options[0] == '-' &&
> entdata->options[1] == 'k'' failed.

We'll have to wait on one of the developers to comment on the above.

BTW have you asked about this on CVSNT's list too, someone there may have 
already seen this?

Following list borrowed from Arthur Barrett
CVSNT newsgroup:
http://www.cvsnt.org/cgi-bin/mailman/listinfo/cvsnt
or
news://news.cvsnt.org/support.cvsnt


BTW2 what make & version of Linux server OS and what does `cvs --version` 
report on both the server and client?  The above error looks to me like 
someone has made three variants off of the official cvs-1.11.21 tree, and that 
is what you are running on the sever. Who made those mods?

> That's not even the 450,000 files.  Googling has not explained that
> error.
>
> I zipped up a smaller directory and copied it onto a linux box.  I ran
> the import there and it worked fine, but I ended up with eol issues when
> I checked back out onto windows.  I assumed tht would happen but I
> thought just maybe an import would just stick the files in there and
> we'd be on our way.  I still have to understand tht issue, but IIRC it's
> that the windows checkout ends up with \n's replaced with \r\n (so we
> have \r\r\n instead of \r\n)???

If you only have source (text) files[1], you could unzip on Unix with
unzip -a myfiles.zip
or
unzip -aa myfiles.zip
to auto convert to Unix line endings.

Still I think the error you got needs ran to ground.

> So to summarize the windows client blows and I can't just use linux.
> How can I check in half-a-million files into cvs?  Scripting a recursive
> add-and-commit is also giving me trouble.

On Unix (assuming you were sane enough to keep spaces out of file and 
directory names):
find path -type d |grep -v CVS| xargs --max-chars=1023 cvs add
find path -type f |grep -v CVS| xargs --max-chars=1023 cvs add
cd path; cvs commit -m "man this is huge :)"

We have to be sure the command line in no case is over ~1023 chars long IIRC.


Use with caution, may eat small (or large) repositories and creatures.

[1] i.e., no word docs, pdf, xls, bmp, jpg
--
Todd Denniston
Crane Division, Naval Surface Warfare Center (NSWC Crane)
Harnessing the Power of Technology for the Warfighter