Re: filter-out, very slow?

[Ted Stern]

On 30 Jan 2004 at 11:03 PST, Ted Stern wrote:
>
> On 30 Jan 2004 at 09:55 PST, Daniel Shane wrote:
>> I see,
>>
>> In that case, could we add a new text function that would work like
>> filter-out, but simply faster? We could call it comm. The reason is quite
>> simple, when strings become overwhelming, it is impossible to pass them to
>> a shell script because of command line size limit. Therefore, they are not
>> easy alternative to get the functionnality of filter-out at a reasonable
>> speed.
>>
>> Daniel Shane
>
> Hi Daniel,
>
> If that is your problem, there is a work-around solution (suggested by Paul
> to me 4 years back).
>
> You can use make functions to echo (or rather printf) the elements of your
> variable into a file.  Then you use xargs to process those values in
> sections that will fit in your command line -- that is precisely the
> intended use of xargs.

Another thought -- once your variable is in a file, with one word per line,
you can then use all manner of unix commands to process it.  For instance,

       fgrep -v -f filter-out.list variable.list

Ted
-- 
 Ted Stern                                 Applications Group
 Cray Inc.                               office: 206-701-2182
 411 First Avenue South, Suite 600         cell: 206-383-1049
 Seattle, WA 98104-2860                     FAX: 206-701-2500

 Frango ut patefaciam -- I break that I may reveal
 (The Paleontological Society motto, equally apropos for debugging)