bug#16662: Libtool execute mode is DOSing my system

[Nick Bowler]

Hi,

I was surprised to discover that in one of my test cases, running the
program via libtool --mode=execute is consuming huge amounts of memory,
apparently because of a runaway sed program it runs on my program's
input files.  This is a problem because these input files are not plain
text files, are several gigabytes in size and GNU sed balloons out to
consume all available memory.

It does eventually complete and the test passes, but not before spending
several minutes pushing the entirety of system memory out to disk.

This can be easily reproduced by running something like the following:

  % truncate -s 16G /tmp/bigfile
  % libtool --mode=execute /usr/bin/wc -c /tmp/bigfile

Running sh -x libtool, we see...

  [...]
  + for file in '"$@"'
  + case $file in
  + func_ltwrapper_script_p /tmp/bigfile
  + func_lalib_p /tmp/bigfile
  + test -f /tmp/bigfile
  + /bin/grep '^# Generated by .*libtool'
  + /bin/sed -e 4q /tmp/bigfile
  [thrashing begins here]

and sure enough, running just that sed command by itself exhibits the
same behaviour.

I guess I can see why libtool's execute mode has to search to see
which input files are libtool wrapper scripts, but that sed program
is obviously a poor choice for large binary files.

Frustratingly, putting "--" before the input files does not appear to
affect the behaviour of libtool at all in this matter.

So it would be nice if libtool didn't consume all available memory
before it even starts running my program :)

For reference, I'm running:

  libtool 2.4.2
  GNU sed 4.2.1

Cheers,
  Nick