Re: bash3 on FreeBSD weirdness

[Bob Proulx]

Miek Gieben wrote:
> http://www.miek.nl/projects/rdup/svn/trunk/sh-tools/mirror.sh.in
> > Chet Ramey wrote:
> > > The root cause is probably that something is consuming all of the
> > > input from stdin (the pipe) on FreeBSD and not on Linux.  I suspect
> > > the call to `head'.  The two implementations probably read and buffer
> > > differently.  This happens all the time, mostly with ssh.

I did not see a 'head' call anywhere in the script though.

> > With ssh for batch mode scripts that are not expected to read stdin it
> > is important to include the -n option.  Otherwise ssh itself consumes
> > stdin.
> 
> thanks, I might need this in the near future :-)

Also 'rsh' has the same issue and the same option.  'ssh' got it from
'rsh' after all.

Reviewing things I see this pattern emerging:

> With this, you get the output of a bin/ directory:
> % ./rdup /dev/null bin
> +16877 1000 20 11 0 /home/miekg
> +16877 1000 20 15 0 /home/miekg/svn
> +16877 1000 20 20 0 /home/miekg/svn/rdup
> ....
> etc

And the script processes it this way:

> % ./rdup -c /dev/null bin | ./sh-tools/mirror.sh -c -b /tmp/storage

> local_mirror() {
>        declare -a path # catch spacing in the path
>        while read -r mode uid gid psize fsize path
> ...
> remote_mirror() {
>         while read -r mode uid gid psize fsize

And I see Chet quoted this from some other message:
> > Basicly the function local_mirror() (line 167) works, and
> > remote_mirror() (line 257) doesn't (on FreeBSD that is).

Here are my observations.  Does your script handle filenames with
embedded newlines and other odd characters?

It is possible that you have a data dependent failure.  You say you
see differences between FreeBSD and GNU/Linux systems?  Is the input
data to the script identical?  If not then I suspect a data dependent
failure.  I suspect that the two data sets are different between the
systems because you are walking filesystems and the filesystems will
have unique data.  If you were to capture the input to a file and then
feed the same file to both scripts on each platform do they then
behave the same or different with identical input data?

You say the local_mirror works and the remote_mirror fails but the
read code for each is different from each other.  It is not
immediately clear to me why the code is different.  But doesn't that
point to a problem here:

> +16877 1000 20 11 0 /home/miekg
>         while read -r mode uid gid psize fsize

Won't fsize be "0 /home/miekg"?  Or am I missing something?

Good luck!
Bob