Re: cvs as a heartbeat client (questions)

[Todd Denniston]

Derek Robert Price wrote:
> 
> 
> Todd Denniston wrote:
> 
> > If I am somewhat more patient, after ~30 seconds from the kill
> > command, 
> >cvs [any command] will terminate.  It looks like the client processes
> >another 30-50 files after the server is given the SIGTERM, which seems
> >like a lot of files to me.
> >
> >I am going to change my plan a little and issue the SIGTERM a couple
> > of times, sleep 30 seconds and then issue `killall -9 cvs`, so 
> >if there is some kind of emergency we are able to go down in 
> >under a minute.
>
> This really shouldn't be the case and I cannot reproduce it.  I would
> really love to see a reproducible case if you can provide a short script
> or the like that creates one.

Attached text file describes how I replicated the situation (with releasable
data).

When I run the killall line during checkout I get cvs _fully_ passing on in a
little under 25 seconds from the kill. [if you let it get several(2 or 3) 
directories into the checkout before issuing the kill it can be more like
30-35 seconds]
22644 ?        S      0:00 /usr/bin/cvs server
22646 ?        S      0:00 /usr/bin/cvs server
22648 pts/0    S      0:00 grep cvs
 ##0
22644 ?        S      0:00 /usr/bin/cvs server
22646 ?        Z      0:00 [cvs <defunct>]
22652 pts/0    S      0:00 grep cvs
 ##1
22644 ?        S      0:00 /usr/bin/cvs server
22646 ?        Z      0:00 [cvs <defunct>]
22659 pts/0    S      0:00 grep cvs
 ##2
22662 pts/0    S      0:00 grep cvs
 ##3
22665 pts/0    S      0:00 grep cvs
 ##4
22668 pts/0    S      0:00 grep cvs
 ##5
22671 pts/0    S      0:00 grep cvs

connections involving pserver seem to be a little quicker, on the order of a
10 to 20 second delay.

> 
> Are you sure that new servers are not starting up between when you issue
> the killall and look for new processes?  Try taking inetd down before
> issuing the killall.

Yes.

> 
> You do mention 30 seconds - that's how long CVS waits for locks by
> default, but I don't think this should be a problem.  I think SIGTERM
> should wake your processes from their sleep to exit.
I picked 30 seconds because that was 1) a little longer than most of the death
times I noted, and 2) under a minute.  After further testing I may push it to
more like 40 seconds because it will probably take longer under a load.

I don't think it is locks, because I am the only one with access to the (test)
repo machine, and I am only doing one operation at a time with it.
BTW repo machine is:
dell 650 
1GB ram 
27GB free for /tmp
145GB free in /cm
218GB free in ~/ (exported over nfs to the client machine)
also the /cm and ~/ drives are on a 5 disk raid 5 hardware box, so they are no
slouches (extremely fast when operating directly on the repo host).

-- 
Todd Denniston
Crane Division, Naval Surface Warfare Center (NSWC Crane) 
Harnessing the Power of Technology for the Warfighter

#######On Repository machine.
export CVSROOT=/cm/junk2
cvs init
mkdir /cm/junk2/test
mkdir cvstest
cd cvstest/
cvs checkout test
cd test
 for j in `seq 0 4`; do for n in `seq 0 4`; do for o in `seq 0 4`; do for i in 
`seq 0 40`; do mkdir -vp td$j/tsd$n/tssd$o/; date >> 
td$j/tsd$n/tssd$o/tf$i.txt;done ; done; done; done

cvs add -m "some message" *

 for j in `seq 0 4`; do for n in `seq 0 4`; do for o in `seq 0 4`; do for i in 
`seq 0 40`; do cvs add -m "some message" td$j/*; cvs add -m "some message" 
td$j/tsd$n/*; cvs add -m "some message" td$j/tsd$n/tssd$o/*; done ; done; done; 
done

cvs commit -m "some commit add message"
cvs tag AfirstTag

#######On Client machine
mkdir test2
cd test2
#using rsh for connect.
export CVSROOT=:ext:repomachine:/cm/junk2/
cvs checkout test

##while the above checkout is occurring, run the following line (as root) on
##the repo machine
ps -ax |grep cvs;killall -q -SIGTERM cvs;sleep 10;echo " ##0";ps -ax |grep 
cvs;sleep 10;echo " ##1";ps -ax |grep cvs;sleep 5;echo " ##2";ps -ax |grep 
cvs;sleep 5;echo " ##3";ps -ax |grep cvs;sleep 5;echo " ##4";ps -ax |grep 
cvs;sleep 5;echo " ##5";ps -ax |grep cvs


#if the checkout completes before the kill happens you can update 
#the data and catch it on check in too:
cd test
 for j in `seq 0 4`; do for n in `seq 0 4`; do for o in `seq 0 4`; do for i in 
`seq 0 40`; do  date >> td$j/tsd$n/tssd$o/tf$i.txt;done ; done; done; done

cvs commit -m "some commit mod all"  

#run kill line while committing.