bug-bash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: problems with output redirection


From: Bob Proulx
Subject: Re: problems with output redirection
Date: Thu, 3 Apr 2008 00:18:59 -0600
User-agent: Mutt/1.5.13 (2006-08-11)

Verena Alishahi wrote:
> last week I've updated my workstation cluster client (amd 64 bit) from
> SuSE 10.0 to openSUSE 10.3. My bash-version is 3.2.25(1)-release.
> ...
> Both the bash-scripts and the output file are stored on an
> NFS-Filesystem (on the workstation cluster server), so every client can
> access them.

I believe the use of NFS to be the root cause of your problem.  The
NFS synchronization here will be related to the NFS version in the
kernel.  Changing that has caused this to change behavior.  I don't
think this is related to changes in bash's version.  Effectively echo
is the same echo as before.

> | less local.sh
> #!/bin/bash
> OUTPUT=$HOME/bashtest/output.txt
> INPUT_HOSTS=$HOME/bashtest/hosts.inp
> REMOTE_SKRIPT=$HOME/bashtest/remote.sh
> echo "created from $HOME/bashtest/local.sh" > $OUTPUT
> for HOST in `cat $INPUT_HOSTS`
> do
> ~  echo $HOST >> $OUTPUT
> ~  ssh $HOST $REMOTE_SKRIPT
> ~  echo "" >> $OUTPUT
> done

Why are there '~' chars in the above lines?

But the real problem is that you are going to be toggling back and
forth between different nfs client hosts writing to the same file.
The writes will look like this in very close succession:

  local    >> $OUTPUT
  client1  >> $OUTPUT
  local    >> $OUTPUT
  client2  >> $OUTPUT
  local    >> $OUTPUT
  client3  >> $OUTPUT
  local    >> $OUTPUT
  client4  >> $OUTPUT
  local    >> $OUTPUT

I am confident this will stress the nfs client subsystem to failure.
The problem is one of distributed nfs cache coherency in the buffer
cache.  It is not trivial to solve this problem.  You might be able to
get more expert discussion of the problem on the nfs mailing lists.
You might be able to convince yourself of this through brute force by
adding sleep statements to slow things down enough to give enough time
to flush the cache between writes.  I am not advocating it as a
solution but only as a debugging data point.

Google for "useless use of cat".

In different contexts such as removing and recreating the file you
would normally see "stale nfs file handle" errors.

I don't think there is anything that bash can do to affect this since
it is just doing normal output.  The reason you tripped into this
wasn't the bash version update but the system kernel upgrade which
brought in a new nfs subsystem.  You might be able to affect things
favorably by changing the mount options.

Bob




reply via email to

[Prev in Thread] Current Thread [Next in Thread]