Re: Updating FreeBSD port

[Eric Blake]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

[I'm replying to several mails in this chain, sorry for any confusion it
causes]

According to Mikhail Teterin on 9/20/2006 8:59 AM:
> On Wednesday 20 September 2006 09:28, Eric Blake wrote:
> = >   . Although FreeBSD's getopt.h is detected as sufficiently capable
> = >     by configure, the getopt.c and getopt1.c are still compiled and
> = >     linked into m4, instead of relying on the libc's versions
> = 
> = No, the FreeBSD's getopt is NOT sufficiently capable.  It has subtly
> = different semantics than GNU which break the option parsing done by m4.
> = Cygwin is another platform that has the BSD-flavored getopt_long but must
> = use gnulib's getopt for the testsuite to pass.  Read the comments in
> = getopt.m4.
> 
> I see -- would you have a test-case, that detects this difference? 
> Getopt_long 
> was introduced into *BSD for the sake of compatibility with GNU software -- 
> any incompatibilities in semantics are a bug, which should be fixed. It can 
> not be fixed, however, if it is not reported...

Yes.  This program demonstrates why the m4 testsuite fails when compiled
with a BSD-flavored getopt_long:

#include <stdio.h>
#include "getopt.h"

int main(int argc, char **argv) {

  int c = 3;
  char *v[3] = {"test", "-r", "foo"};
  getopt(c, v, "r::");

  if (optarg == NULL) {
    printf("GNU getopt\n");
  } else {
    printf("OpenBSD getopt\n");
  }
}

This is because the m4 testsuite uses the -d flag, which takes an optional
argument, but with BSD semantics, m4 can't tell that no argument was
supplied, and fails when trying to parse a filename as debug flags to use.

> = >   . mkstemp_safer appears to be redundant -- how can mkstemp possibly
> = >     return the STDIN_FILENO?
> = 
> = When you invoke "m4 file <&-".  Trust me, I tested it, which is why I had
> = to use mkstemp_safer from gnulib.
> 
> Sorry, I don't quite understand this... But I will trust you :-)

OK, the longer answer:  <&- is shell terminology for starting a program
with stdin closed, leaving fd 0 available for the next operation that
opens a file, per POSIX rules on open().  mkstemp happens to be such a
method, so mkstemp_safer is gnulib's answer that keeps fd 0 closed if it
started that way so that you don't leak a bogus stdin to child processes.
 (Actually, in m4 1.4.6, the only uses of mkstemp are the maketemp macro,
where m4 immediately closes the fd before it would have leaked, and in a
naive replacement for platforms lacking tmpfile, although I doubt BSD
falls in that camp.  So you are right that mkstemp_safer is overkill in
this particular case, but I'd rather be safe than sorry.)

> Does running `gmake check' after building gm4 and getting all tests to
> succeed, constitute a comprehensive test? Are all aspects of m4 being
> covered?

Unfortunately, no.  In fact, poor testsuite coverage is one of the reasons
that a bug introduced in 1.3 (more than 10 years ago) in the format macro,
which allowed arbitrary code execution due to a buffer overrun in the
format macro on platforms lacking ecvt, went undetected for so long (it
was finally patched in 1.4.5).

1.4.6 is better than 1.4.4, by a long shot, but is still limited by the
fact that you can't parse tests out of texinfo and still do all these
sorts of tests (such as "m4 file <&-").  I will be releasing 1.4.7 in the
near future, with yet more tests.  And M4 2.0 will be better yet, where
the testsuite moved to autotest and can do much more powerful tests, but I
continue to find bugs and add tests.  But even in the 2.0 release, unless
I figure out how to hook in something like expect, I can't test some of
the corner cases involving ptys as input streams, even though I have found
and fixed bugs in that area as documented in the m4 mailing lists.  But
you are always welcome to help submit testcases.

> My concern was for m4 only :-) It was Eric, who added gnulib to the
> list of CC. If *gm4* is not going to trip over any getopt_long
> differences, then it is safe for it to not use its own implementation.

Everything in m4's libm4 comes from gnulib.  So an issue here affects more
than just m4, which is why I cc'd gnulib.

> :-( Are there regex-patches currently waiting to be merged into GNU's
> libc? FreeBSD may be able to incorporate them faster...

Yes - Paul already pointed you to the gnulib version of regex.h, which is
based on the glibc version but heavily patched.  See also:
http://cvs.savannah.gnu.org/viewcvs/gnulib/gnulib/config/srclist.txt?rev=1.139&view=markup
Which lists a number of bugzilla id's that Paul has already filed against
glibc (although that list is incomplete; he's made more bug fixes than
what he has had time to report upstream).

> . Although *snprintf code is compiled and linked into libm4, none of
> >       these functions make it into the m4 executable -- are the files
> >       obsolete?

> I'll take a look at that.  It may be that it was only used by the code
> that I deleted when chopping out the usage of ecvt from the
> implementation of the format builtin.

Found it.  The xvasprintf gnulib module provides both xvasprintf and
xvasnprintf in the same .c, pulling in the *nprintf versions even though
m4 doesn't use them.  Oskar, is this worth splitting the module into two
pieces?

Finally, Paul's answer has merit:

> You can just do a diff between glibc and regex to find the list of
> patches.  They aren't marshaled into a coherent set of small patches,
> which is the bottleneck.  However, you _can_ just use the gnulib
> version, which is a popular option.

Which is exactly what gm4 does by using gnulib :)

- --
Life is short - so eat dessert first!

Eric Blake             address@hidden
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFEem684KuGfSFAYARAp2fAJ43DlU5V8P6Wq/uk0ZE+3l6NEgXYgCcCzL4
H0KPqegkr7EEDyJbQ3ku0Ew=
=20At
-----END PGP SIGNATURE-----