Re: [PATCH 1/2] fts: introduce the FTS_NOLEAF flag

[Pádraig Brady]

On 09/12/15 10:35, Pádraig Brady wrote:
> On 09/12/15 06:34, Kamil Dudka wrote:
>> The flag is needed to implement the -noleaf option of find.
>> * lib/fts.c (link_count_optimize_ok): Implement the FTS_NOLEAF flag.
>> * lib/fts_.h (FTS_NOLEAF): New macro, shifted conflicting constants.
> 
> Is this exposed to fix issues with certain file systems,
> or just in case there may be issues, or support easily
> testing find without the leaf optimization?
> 
> I see Jim said the current FTS implementation
> would make -noleaf a no-op there:
> https://lists.gnu.org/archive/html/bug-gnulib/2008-12/msg00280.html
> 
> cheers,
> Pádraig.
> 
> p.s. I see that find does a stat per file on XFS,
> while d_type can be used to distinguish dirs there.
> On XFS DT_DIR is set for dirs and DT_UNKNOWN otherwise.
> I wonder is there some optimization we could do for that case.

I did a quick check on XFS which suggests the leaf optimization
based on st_nlink is valid:
  test $(($(find . -maxdepth 1 -type d | wc -l) + 1)) = $(stat -c %h .) && echo 
leaf_ok

Applying this diff:
@@ -717,6 +718,7 @@ leaf_optimization_applies (int dir_fd)
     {
       /* List here the file system types that lack usable dirent.d_type
          info, yet for which the optimization does apply.  */
+    case S_MAGIC_XFS:
     case S_MAGIC_REISERFS:
       return true;

Gives this significant speedup:
  $ time find/find-before /usr/share >/dev/null
  real    0m0.410s
  user    0m0.145s
  sys     0m0.266s

  $ time find/find-after /usr/share >/dev/null
  real    0m0.278s
  user    0m0.147s
  sys     0m0.131s

I also noticed a lot of fcntl calls on XFS
(basically one per file), which I need to look further into:
  $ strace -c find/find /usr/share >/dev/null
  % time     seconds  usecs/call     calls    errors syscall
  ------ ----------- ----------- --------- --------- ----------------
   40.03    0.147809           1    151710           fcntl
   17.62    0.065069           1     63154           close
   14.52    0.053608           1     40071           newfstatat
   13.15    0.048547           1     35400           getdents

cheers,
Pádraig