Re: filter-out, very slow?

[Ted Stern]

On 30 Jan 2004 at 09:55 PST, Daniel Shane wrote:
> I see,
>
> In that case, could we add a new text function that would work like
> filter-out, but simply faster? We could call it comm. The reason is quite
> simple, when strings become overwhelming, it is impossible to pass them to a
> shell script because of command line size limit. Therefore, they are not
> easy alternative to get the functionnality of filter-out at a reasonable
> speed.
>
> Daniel Shane

Hi Daniel,

If that is your problem, there is a work-around solution (suggested by Paul to
me 4 years back).

You can use make functions to echo (or rather printf) the elements of your
variable into a file.  Then you use xargs to process those values in sections
that will fit in your command line -- that is precisely the intended use of
xargs.

Make 3.80 allows you to call MAKE functions recursively.  This can be used to
loop through the list in chunks until you've extracted everything.

The full implementation of MAKE_LONG_LIST is attached below.

I needed this functionality to archive a library containing some 9000 objects.
I use it as follows:

AR_LIST := libfoo.list

include MakeLongList.mk

libfoo.a: $(AR_OBJS)
        $(call MAKE_LONG_LIST,$(AR_LIST),$(AR_OBJS))
        $(XARGS) $(AR) $(ARFLAGS) $@ < $(AR_LIST)

This is pretty fast, incidentally.

Ted        
-- 
 Ted Stern                                 Applications Group
 Cray Inc.                               office: 206-701-2182
 411 First Avenue South, Suite 600         cell: 206-383-1049
 Seattle, WA 98104-2860                     FAX: 206-701-2500

 Frango ut patefaciam -- I break that I may reveal
 (The Paleontological Society motto, equally apropos for debugging)

MakeLongList.mk
Description: MakeLongList.mk