Re: non portable sed scripts

[Ralf Wildenhues]

[ let's limit this to autoconf-patches ]

Hi Stepan,

* Stepan Kasal wrote on Mon, May 22, 2006 at 10:59:06AM CEST:
> > Ralf Wildenhues <address@hidden> writes:
> > > So IMHO I wouldn't mind if only the bugfix and documentation parts of
> > > the patch below were committed.
> 
> Ralf, I'm not sure what you mean by ``the bugfix'' but I guess it
> might be this:
>         (_AC_FEATURE_CHECK_LENGTH): Do not remove `conftest.*', but only
>         the files that this macro actually generates, to keep the file
> and I agree that it is harmless and should be committed.
> (BTW, I do not see the change in your patch.)

Sorry about that, I forgot to include it somehow.

> I share your opinion that committing all the AC_PROG_SED & Co. would
> be dangerous and would require a lot of testing.

With "the bugfix parts" I meant everything except munging with
config.status, see the proposed patch below; so that /usr/ucb/sed was
properly eliminated from the list of good ones.  But I was going to ask
again anyway, because I'm not so sure whether AC_PROG_SED should be
changed.  I think it's a good idea though (and I did do some testing
with some system seds).

> I think we should lower ac_max_sed_lines artificially to workaround the
> problem which started this thread.
> 
> I'm attaching a patch, which uses the value of 60, which usually leads to
> sed scripts of size < 5000.  This should be reasonably safe.
> Or should we go to the old value of 38?
> 
> Since most packages use only one config header, we don't have to be
> sad that its creation has been slowed down.

Agreed.  But the problem is not only the config header generation.  The
config files generation suffers from the size limitation, too.  Luckily,
it seems we are generally still below the limit on that.

> What do you think about this hack?

Sounds ok to me.

Cheers,
Ralf

2006-05-21  Paul Eggert  <address@hidden>
        and Ralf Wildenhues  <address@hidden>

        * lib/autoconf/programs.m4 (AC_PROG_SED): Catch script length
        limits in Solaris 8 /usr/ucb/sed by testing a long script.
        (_AC_FEATURE_CHECK_LENGTH): Do not remove `conftest.*', but only
        the files that this macro actually generates, to keep the file
        `conftest.sed' created by AC_PROG_SED.
        (_AC_OUTPUT_FILE): Fix typo in comment.
        * doc/autoconf.texi (Particular Programs): Update description of
        AC_PROG_SED.
        (Limitations of Usual Tools) <sed>: Mention script length
        limitations with some of the sed implementations on Solaris.

2006-05-21  Paul Eggert  <address@hidden>

        Undo this change, and add comment about not counting labels:

        * lib/autoconf/status.m4 (_AC_OUTPUT_HEADER): Fix off-by-one bug
        that caused config.status to generate 100-command sed scripts;
        the portable limit is 99.

Index: doc/autoconf.texi
===================================================================
RCS file: /cvsroot/autoconf/autoconf/doc/autoconf.texi,v
retrieving revision 1.1018
diff -u -r1.1018 autoconf.texi
--- doc/autoconf.texi   20 May 2006 05:39:03 -0000      1.1018
+++ doc/autoconf.texi   22 May 2006 03:45:47 -0000
@@ -3632,9 +3632,10 @@
 @defmac AC_PROG_SED
 @acindex{PROG_SED}
 @ovindex SED
-Set output variable @code{SED} to a Sed implementation on @env{PATH} that
-truncates as few characters as possible.  If @sc{gnu} Sed is found,
-use that instead.
+Set output variable @code{SED} to a Sed implementation on
address@hidden that conforms to Posix without arbitrary length limits,
+if possible.  If @sc{gnu} Sed is found, use that instead.  If no
+acceptable Sed is found, an error is reported.
 @end defmac
 
 @defmac AC_PROG_YACC
@@ -13207,9 +13208,14 @@
 Unicos 9 @command{sed} loops endlessly on patterns like @samp{.*\n.*}.
 
 Sed scripts should not use branch labels longer than 8 characters and
-should not contain comments.  HP-UX sed has a limit of 99 commands and
+should not contain comments.  HP-UX sed has a limit of 99 commands
+(not counting @samp{:} commands) and
 48 labels, which can not be circumvented by using more than one script
 file.  It can execute up to 19 reads with the @samp{r} command per cycle.
+Solaris @command{/usr/ucb/sed} has a limit of about 6000 bytes for the
+internal representation of the sed script, which can not be circumvented
+using more than one script file.  It fails with longer scripts, and
+and Solaris @command{/usr/xpg4/bin/sed} dumps core with long scripts.
 
 Avoid redundant @samp{;}, as some @command{sed} implementations, such as
 address@hidden 1.4.2's, incorrectly try to interpret the second
Index: lib/autoconf/programs.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/autoconf/programs.m4,v
retrieving revision 1.54
diff -u -r1.54 programs.m4
--- lib/autoconf/programs.m4    19 May 2006 08:11:27 -0000      1.54
+++ lib/autoconf/programs.m4    22 May 2006 09:18:30 -0000
@@ -505,7 +505,7 @@
     # 10*(2^10) chars as input seems more than enough
     test $ac_count -gt 10 && break
   done
-  rm -f conftest.*])
+  rm -f conftest.in conftest.tmp conftest.nl conftest.out])
 ])
 
 
@@ -812,11 +812,20 @@
 # as few characters as possible.  Prefer GNU sed if found.
 AC_DEFUN([AC_PROG_SED],
 [AC_CACHE_CHECK([for a sed that does not truncate output], ac_cv_path_SED,
-    [_AC_PATH_PROG_FEATURE_CHECK(SED, [sed gsed],
+    [dnl ac_script should not contain more than 99 commands (for HP-UX sed),
+     dnl but more than about 7000 bytes, to catch a limit in Solaris 8 
/usr/ucb/sed.
+     
ac_script=s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb/
+     for ac_i in 1 2 3 4 5 6 7; do
+       ac_script="$ac_script$as_nl$ac_script"
+     done
+     echo "$ac_script" | sed 99q >conftest.sed
+     $as_unset ac_script || ac_script= 
+     _AC_PATH_PROG_FEATURE_CHECK(SED, [sed gsed],
        [_AC_FEATURE_CHECK_LENGTH([ac_path_SED], [ac_cv_path_SED],
-               ["$ac_path_SED" -e 's/a$//'])])])
+               ["$ac_path_SED" -f conftest.sed])])])
  SED="$ac_cv_path_SED"
- AC_SUBST([SED])
+ AC_SUBST([SED])dnl
+ rm -f conftest.sed
 ])# AC_PROG_SED
 
 
Index: lib/autoconf/status.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/autoconf/status.m4,v
retrieving revision 1.103
diff -u -r1.103 status.m4
--- lib/autoconf/status.m4      19 May 2006 21:02:10 -0000      1.103
+++ lib/autoconf/status.m4      21 May 2006 11:46:41 -0000
@@ -303,7 +303,7 @@
 # _AC_SED_CMD_LIMIT
 # -----------------
 # Evaluate to an m4 number equal to the maximum number of commands to put
-# in any single sed program.
+# in any single sed program, not counting ":" commands.
 #
 # Some seds have small command number limits, like on Digital OSF/1 and HP-UX.
 m4_define([_AC_SED_CMD_LIMIT],
@@ -480,7 +480,7 @@
 # Do the variable substitutions to create the Makefiles or whatever.
 #
 # This macro is expanded inside a here document.  If the here document is
-# closed, it has to be reopen with "cat >>$CONFIG_STATUS <<\_ACEOF".
+# closed, it has to be reopened with "cat >>$CONFIG_STATUS <<\_ACEOF".
 #
 m4_define([_AC_OUTPUT_FILE],
 [
@@ -660,7 +660,7 @@
 [s,^[   #]*u.*,/* & */,]' >>conftest.defines
 
 # Break up conftest.defines:
-ac_max_sed_lines=m4_eval(_AC_SED_CMD_LIMIT - 4)
+ac_max_sed_lines=m4_eval(_AC_SED_CMD_LIMIT - 3)
 
 # First sed command is:         sed -f defines.sed $ac_file_inputs >"$tmp/out1"
 # Second one is:        sed -f defines.sed "$tmp/out1" >"$tmp/out2"