Re: execvp: /bin/sh: Argument list too long

[Ralf Wildenhues]

Hello Pippijn,

* Pippijn van Steenhoven wrote on Mon, Nov 08, 2010 at 04:55:45PM CET:
> I looked through the mailing list archives and found several questions
> and proposed solutions related to this problem:
> 
>   - Split up file lists so several rules are generated:
>   http://lists.gnu.org/archive/html/automake/2004-11/msg00106.html
> 
>   - "What is your env's size?"
>   http://lists.gnu.org/archive/html/automake/2009-08/msg00027.html
> 
>   - It will fail. In other words "Sorry, can't fix"?
>   http://lists.gnu.org/archive/html/automake/2007-02/msg00013.html

Also, we wrote a short section in the manual about this (in recent
versions):
  info Automake "Length Limitations"

> My problem is that I can not make dist in one project and not make check
> in another project. Both have a large number of tests. The one where make
> check fails has a few more tests, but make dist will fail there, as well.

One workaround is to put tests in more than one directory, with more
than one Makefile.am.  Yes, it is ugly, I know.  Another is to require
non-VPATH builds on those systems with the most severe restrictions,
such as MSYS and IRIX (if you can't convince root to increase the
limit).

> What are the plans on fixing this?

We'll fix what we can fix.  Without GNU make-specifics, that is hard.
One thing I'd definitely like to see done is allowing multiple
parallel-tests test suites in one Makefile.am, so you can still have a
nonrecursive setup.  That may mean that 'make dist' still fails, but
my hope is that is not needed to work on every platform.

Would that be sufficient for your needs?

> Are there any? Autoconf already checks
> for the maximum argument length,

It's a configure check from a Libtool macro that does that, actually.

> but since by that time, Makefile.in is
> already generated, it can't help. Maybe storing the list of dist files
> and check files in a different file and reading this with xargs < $file
> would help.

One problem is that with many constructs, automake would like to know
the exact set of files it deals with.  We can try to deviate from that
in some places but my guess is that will always look like a hack that
avoids some but not all issues.

One other strategy we maybe could implement is alternative GNU
make-specific rules that avoid limitations by dealing with files with
make functions.  That is usually doable, but doing this consistently
will be quite some work and cause quite some duplication in the
Makefile.in.  I don't have current plans to pursue this; you can see
e.g., the write_entries_to_file macro in the gcc/Makefile.in file in the
GCC tree for how this can be done in principle.

Cheers,
Ralf