Re: address@hidden: Re: CVS commit: pkgsrc/devel/m4]

[ebb9]

[Arrgh - your reply arrived after I left my primary computer,
and I'm using nabble to respond which does not understand
reply-to-all; feel free to repost this reply to the full cc chain]

Joerg Sonnenberger wrote:
> The missing part above might clarify what I meant. Normally a stdio
> implementation is allowed to keep its read buffer even across fseeko.

But not if the fseeko immediately followed fflush.

> I don't think the rpl_fseeko changes that as it only modifies the
> internal position accordingly.

No, gnulib's replacement rpl_fseeko works in tandem with the
rpl_fflush to guarantee that there is no read buffer.  By itself,
you are correct that rpl_fseeko tries to keep the read buffer;
it is rpl_fflush that guarantees that there is no read buffer
(in part by using fpurge under the hood), so that the following
fseeko goes to the correct location.  And the gnulib tests
(distributed in the tests subdir of m4-1.4.10) check this
behavior.  According to
http://www.nabble.com/forum/ViewPost.jtp?post=11700211&framed=y,
the fact that gnulib's self-tests did not fail for fflush or fseeko,
(only for frexpl), the gnulib implementation correctly worked
around NetBSD's current deficiencies.

> So the example is m4 is called with a
> regular file on stdin and the subprogram modifies this file after the
> current position. Is m4 expected to see such changes?

Yes.

> 
> There's also the indenpend question whether lseek is the best approach
> to find seekability. There were a number of issues with real seeks on
> character devices like tapes and I'm not sure how the interactions with
> zero seeks are. So an explicit test for a regular file might be the
> better idea.

The standard is clear that the fflush(stdin) coordination between
handles should work for ALL seekable input, not just regular files.
You are correct that the standard is clear that lseek on an
unseekable device is implementation-defined (note that
implementation-defined is different from unspecified - in other
words, I should be able to browse your system specs and
find a statement of what would happen in that case).  However,
since the standard also states that there is no obligation for
unseekable fd's to coordinate positions between processes,
there is no compliance bug even if the implementation-defined
behavior of the lseek on the unseekable fd changes its state,
because you were already in unspecified territory for using
an unseekable fd to coordinate between processes.

I also admit that it would be possible on a fully-compliant
system to remove the lseek filter altogether; the problem
is that the gnulib rpl_fflush is only designed to work on
seekable input (ie. gnulib can't guarantee a sane implementation
of what rpl_fflush will do on unseekable input, because
it depends on knowing the implementation-defined effects
of ALL of the platforms that gnulib runs on top of).  So,
the only reason I have the initial lseek is not because of
POSIX, but because of gnulib.

> 
>> On a system where m4 and sed both follow POSIX rules, this is an example
>> of the above rules

I botched my transcribing of the example (in converting from
autotest back to shell), and left out a redirection. Corrected,
(and tested this time), it should read:

$ cat <<\EOF > in.m4
define(`foo', `FOO')m4 foo
syscmd(`sed -e "s/foo/bar/;q"')sed foo
m4 foo
EOF 
$ m4 < in.m4
m4 FOO
sed bar
m4 FOO
$

>> Nope.  It may work on some systems, but the POSIX specification of
>> setvbuf
>> states that it may only safely be used prior to any I/O on the stream. 

> 
> I don't see such a requirement anywhere, at least not in SUSv3.

http://www.opengroup.org/onlinepubs/009695399/functions/setvbuf.html

"The setvbuf() function may be used after the stream pointed to
by stream is associated with an open file but before any other
operation (other than an unsuccessful call to setvbuf()) is
performed on the stream."

Yes, SUSv3 and POSIX 2001 are not quite the same documents,
but in many cases, SUSv3 defers to POSIX, which I just quoted.

> The BSD
> man page correctly annotates that calling setvbuf might flush unread
> input, but that is kind of the desired affect here. It leaves the
> question whether buffering more than one line at a time is not
> conflicting with the file positioning issues.

I see no conflict (and neither does Sun, newlib, or glibc's
implementation of fflush/fseek).  In particular, newlib
took a BSD-based stdio library and implemented these
POSIX rules only within the past year, in part because of my
efforts to make m4 comply with these POSIX rules on
the cygwin platform (a newlib client).

-- 
Eric Blake