[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 8/8] fts: do not exhaust memory when processing million-entry
From: |
Jim Meyering |
Subject: |
Re: [PATCH 8/8] fts: do not exhaust memory when processing million-entry directories |
Date: |
Thu, 18 Aug 2011 21:36:29 +0200 |
Paul Eggert wrote:
> Thanks for all that work to make fts better! A couple of minor things
> about comments:
>
> On 08/18/2011 06:53 AM, Jim Meyering wrote:
>
>> + into memory at once. However, When an fts_compar function
>
> The "However," can be removed (there are too many Buts etc. in the
> neighborhood already ...).
>
>> + The other conditionals ensure
>> + that we are using the *at functions (FTS_CWDFD) and that we
>> + are not in no-chdir mode (induced by use of FTS_LOGICAL). */
>
> Are these other conditionals independent of whether we want to avoid
> putting too many entries in RAM? If so, perhaps we should remove these
> other conditionals; if not, it'd help for the comment to explain why not.
Initially I didn't even write that "The other conditionals..." sentence.
I agree that it doesn't say enough. The main motivation was to solve
the problems that were most common, and to avoid getting bogged down in
the less-common use cases. Already, this change is defending against
something very unusual: applying common tools to directories with
millions of entries. Worrying about applying "du -L" (FTS_LOGICAL)
on such a directory is far lower priority.
I'll add a FIXME saying that we should try to remove those conditionals,
but that doing so will require careful testing of those less-common
use cases. Actually, as I write this, I confess I'm thinking that the
marginal gain is not worth the risk.
- Re: [PATCH 1/8] maint: fts.c: remove #if-0'd FTS_WHITEOUT code, (continued)
[PATCH 3/8] maint: fts.c: correct off-by-one indentation, Jim Meyering, 2011/08/18
[PATCH 2/8] maint: fts.c: move __opendir2 #define "up" out of function body, Jim Meyering, 2011/08/18
[PATCH 7/8] fts: move decl of "dp" into while loop; split long line, Jim Meyering, 2011/08/18
[PATCH 5/8] maint: fts: give __opendir2 a new parameter, Jim Meyering, 2011/08/18
[PATCH 6/8] fts: add/use new struct member, fts_dirp, Jim Meyering, 2011/08/18
[PATCH 8/8] fts: do not exhaust memory when processing million-entry directories, Jim Meyering, 2011/08/18
Re: fts: do not exhaust memory when processing million-entry directory, Pádraig Brady, 2011/08/18