Re: Have I got a corrupt repository?

[Mark D. Baushke]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andy Jones <address@hidden> writes:

> >  a) #cvs.lock  - a directory (used when creating #cvs.[rw]fl.* locks)
> >  b) #cvs.rfl.*.<pid> or #cvs.wfl.*.<pid>
> >  c) ,<file>,
> 
> a's and b's but no c's.

Okay, if cvs core dumps you expect stale locks to be found in the
repository. If it core dumps while writing the new rcs ,v files, then
that gets more tricky.

> This continues to be the case - lock files are being left across the
> repository at odd moments - most likely whenever CVS starts displaying
> greek, but I'm getting ahead of myself.
> 
> >A cvs commit will only print 
> >
> >    Up-to-date check failed for `<filename>'
> >
> >where <filename> is the name of a <filename>,v that corresponds to the a
> >file that exists in the repository.
> >
> >The other odd characters are not normally part of cvs output.
> 
> The odd characters seem to be associated with a particular file.
> Unfortunately it is not a file that "ls" can see, only "cvs status".

And you are sure that there are no 'odd' files present in your
repository? (ls -alR) I think Redhat Linux allows UTF-8 characters in
filenames...

> The sequence seems to be (at least, as I observe it):
> 
> 1) I do a large commit across mutiple directories that results in a
>    segmentation fault & core dump. Smaller commits are okay. This is
>    in local mode, now, not pserver.

Define 'large commit' please. Many files? Many LARGE files? A few LARGE
files? It almost sounds like mmap() is broken on your system.

You might try downloading cvs 1.11.11 (just to avoid other bugs that
have been fixed since your 1.11.10 version), configure it and then go
into the config.h file and change the

        #define HAVE_MMAP 1
to
        #undef HAVE_MMAP

and then 'make' to build it and install it and see if it fixes your
problem.

> 2) Lock files need to be cleared from the repository (naturally enough).

Yup.

> 3) cvs status will now show a new file with a "greek" file name
>    somewhere in the sandbox, although ls shows nothing and fsck shows
>    the partition has no errors.  

What do you see in the CVS/Entries file for the corresponding directory
that is showing those 'greek' filenames?

> 4) Any further cvs operation across this "file" will fail and require
>    a further clearing of lock files from the repository. 

Sure, if locks have been created you will have to clear them.

> This sequence has happened once before I restored the old repository,
> and once since.

Okay.

> >You should check that you have enough swap space on your system
> >so that you can have a process that is ~2.6 times the size of your
> >largest file being committed. Your /tmp filesystem also needs to
> >hold a copy of the tree in /tmp/cvs-serv* that is big enough to hold
> >multiple checked out trees of your repository at the same time.
> 
> The biggest file in the repository looks to be 716k.
> /tmp is 1Gb.

So, /tmp is probably not a swap (aka tempfs aka mfs) file system. Okay.

> Swap is 500Mb, although for some reason it doesn't show in a df -a. I
> don't know if this is an artifact of RH7.1 or means that the swap
> partition is not loading ... although that 716k file hasn't grown much
> in the last three months, so I'm inclined to think that this is a red
> herring, too...

Use 'top' to see what memory and sap you have free on your system.

The first line will have the number of processes, the second the CPU
states, the third the memory being used and the fourth the swap.

Something like this:

97 processes: 95 sleeping, 1 running, 1 zombie, 0 stopped
CPU states:  4.6% user,  0.3% system,  0.1% nice,  0.2% idle
Mem:   513648K av,  499192K used,   14456K free,       0K shrd,  198720K buff
Swap: 1052216K av,   25844K used, 1026372K free                  201544K cached

after that will come a list of running processes. If you don't have
'top' installed, then the 'vmstat' command along with 'cat /proc/meminfo'
should help you determine what is on your system.

For example:

% head /proc/meminfo
        total:    used:    free:  shared: buffers:  cached:
Mem:  525975552 513347584 12627968        0 203735040 220934144
Swap: 1077469184 26464256 1051004928
MemTotal:       513648 kB
MemFree:         12332 kB
MemShared:           0 kB
Buffers:        198960 kB
Cached:         202224 kB
SwapCached:      13532 kB
Active:         315240 kB
% 

> >If you have any core dumps, feel free to use gdb to get a backtrace of
> >where things went wrong and send it here to address@hidden 
> 
> You mean like this? (speaking as a complete gdb novice who's just read
> the man page):

Yes, like that.

> =========================
> (gdb) bt

Crashing in a malloc() to obtain 39 bytes is a bad sign. You really need
to see how much free memory and swap you have on your system.

> #0  0x40176401 in malloc () from /lib/libc.so.6
> #1  0x401761fa in malloc () from /lib/libc.so.6
> #2  0x080884f5 in xmalloc (bytes=39) at subr.c:41
> #3  0x0808865f in xstrdup (str=0x8229628 
> "/tapestry/cvsSupport/tapestry.checklog") at subr.c:148
> #4  0x0807fb5e in run_setup (prog=0x8229628 
> "/tapestry/cvsSupport/tapestry.checklog") at run.c:61
> #5  0x0806c055 in do_verify (messagep=0x80cea58,
>     repository=0x81b4a38 "/usr/local/cvs_repository/tapestry/clcb") at 
> logmsg.c:471
> #6  0x08058e7a in commit_direntproc (callerdat=0x0, dir=0x82a2f38 "clcb",
>     repos=0x820c4e8 "/usr/local/cvs_repository/tapestry/clcb", 
> update_dir=0x80f8c50 "clcb", entries=0x80fe958)
>     at commit.c:1532
> #7  0x0807db59 in do_dir_proc (p=0x8260700, closure=0xbffff648) at 
> recurse.c:1073
> #8  0x0806213a in walklist (list=0x8253720, proc=0x807d77c <do_dir_proc>, 
> closure=0xbffff648) at hash.c:370
> #9  0x0807d61c in do_recursion (frame=0xbffff6f0) at recurse.c:783
> #10 0x0807dcdb in do_dir_proc (p=0x80d3598, closure=0xbffff798) at 
> recurse.c:1134
> #11 0x0806213a in walklist (list=0x80d3f10, proc=0x807d77c <do_dir_proc>, 
> closure=0xbffff798) at hash.c:370
> #12 0x0807d61c in do_recursion (frame=0xbffff830) at recurse.c:783
> #13 0x0807d0b6 in start_recursion (fileproc=0x80587e0 <commit_fileproc>,
>     filesdoneproc=0x8058c58 <commit_filesdoneproc>, direntproc=0x8058dd4 
> <commit_direntproc>,
>     dirleaveproc=0x8058e90 <commit_dirleaveproc>, callerdat=0x0, argc=0, 
> argv=0x82ec5c0, local=0, which=1,
>     aflag=0, locktype=0, update_preload=0x0, dosrcs=1, repository_in=0x0) at 
> recurse.c:366
> #14 0x08057ed1 in commit (argc=1, argv=0x80d31ac) at commit.c:679
> #15 0x0806d82d in main (argc=1, argv=0x80d31a8) at main.c:994
> #16 0x40113237 in __libc_start_main () from /lib/libc.so.6
> =============================
> 
> >You probably want to send the information in a way that is NOT going to be
> >mangled with word-wraps by your e-mail client. Possibly, putting the log
> >file up on a web site and sending a URL of it to this list would make
> >more sense.
> 
> Regretably, I don't have that option, athough I could always mail it
> to someone directly in an attachment if they wanted. Likewise, I'm
> happy to run cleverer gdb commands if asked/shown.
> 
> OTOH I can/have turn(ed) off word wrapping in my client - which may or
> may not help you.

It worked fine.

        Good luck,
        -- Mark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (FreeBSD)

iD8DBQFABB/w3x41pRYZE/gRAhcaAJ930OiL36IWPm5PnFkuOg+YhyfUtACgjyZk
iNzE/YD85mLXZDUPAkJdh38=
=H22W
-----END PGP SIGNATURE-----