[PATCH] Add 2-pass mode to gtags

[Hideki IWAMOTO]

Hi.

To improve the performance of gtags, this patch adds 2-pass mode. 

The following three points are improved by moving use of function defined()
from gtags-parser to gtags. 
 1. Decreases the frequency of parsing source file from three times to two 
times.
 2. Decreases reading of GTAGS by keeping cache of GTAGS
    beyond the duration of parser process. 
 3. Increases parallelism of gtags-parser and gtags on multiprocessor system. 


[ 3-pass mode (current implementation) ]
  Pass 1:
                    +--------------+            +-------+
   source files ===>| gtags-parser |===========>| gtags |===> GATGS
                    +--------------+            +-------+
  Pass 2:
                    +--------------+            +-------+
   source files ===>| gtags-parser |===========>| gtags |===> GRTAGS
          GTAGS ===>|              |            +-------+
                    +--------------+            
  pass 3:
                    +--------------+            +-------+
   source files ===>| gtags-parser |===========>| gtags |===> GSYMS
          GTAGS ===>|              |            +-------+
                    +--------------+            
 
[ 2-pass mode ]
  Pass 1:
                    +--------------+            +-------+
   source files ===>| gtags-parser |===========>| gtags |===> GATGS
                    +--------------+            +-------+
  Pass 2:                 
                    +--------------+            +-------+ 
   source files ===>| gtags-parser |===========>| gtags |===> GRTAGS
                    +--------------+  GTAGS ===>|       |===> GSYMS
                                                +-------+


Performance improvement:
  2-pass mode is 20-30% faster than 3-pass mode on 1-CPU system,
  40-50% faster on 2-CPU system.

$ foreach cpumask ( 1 3 )
foreach? foreach label ( gtags gtags-2pass )
foreach? echo ==== cpumask:$cpumask label:$label ====
foreach? foreach dir ( linux-2.0.40 linux-2.2.26 linux-2.4.37 linux-2.6.31 )
foreach? rm -fr $dir; tar xfj ~/download/linux/$dir.tar.bz2; sync
foreach? (cd $dir; time env GTAGSLABEL=$label taskset $cpumask gtags)
foreach? end
foreach? end
foreach? end
==== cpumask:1 label:gtags ====
8.360u 0.716s 0:09.83 92.2%     0+0k 0+0io 1pf+0w
23.297u 2.448s 0:28.14 91.4%    0+0k 0+0io 0pf+0w
50.083u 6.136s 0:59.39 94.6%    0+0k 0+0io 0pf+0w
116.895u 16.377s 2:34.86 86.0%  0+0k 0+0io 4pf+0w
==== cpumask:1 label:gtags-2pass ====
6.532u 0.600s 0:07.75 92.0%     0+0k 0+0io 1pf+0w
18.221u 1.836s 0:22.01 91.0%    0+0k 0+0io 0pf+0w
39.414u 4.344s 0:46.94 93.2%    0+0k 0+0io 0pf+0w
92.857u 11.216s 1:58.56 87.7%   0+0k 0+0io 1pf+0w
==== cpumask:3 label:gtags ====
8.092u 0.848s 0:08.42 106.0%    0+0k 0+0io 1pf+0w
23.105u 2.268s 0:24.24 104.6%   0+0k 0+0io 0pf+0w
50.067u 5.856s 0:58.19 96.0%    0+0k 0+0io 0pf+0w
114.619u 16.581s 2:16.91 95.8%  0+0k 0+0io 0pf+0w
==== cpumask:3 label:gtags-2pass ====
6.372u 0.608s 0:05.55 125.5%    0+0k 0+0io 0pf+0w
18.189u 1.636s 0:15.46 128.1%   0+0k 0+0io 0pf+0w
38.886u 4.148s 0:35.89 119.8%   0+0k 0+0io 0pf+0w
90.541u 11.100s 1:38.27 103.4%  0+0k 0+0io 1pf+0w


Diffstat:
 global/global.c          |   24 ++++++++++++
 gtags-parser/C.c         |   82 +++++++++++++++++++++++++++-----------------
 gtags-parser/Cpp.c       |   87 ++++++++++++++++++++++++++++-------------------
 gtags-parser/asm_parse.y |   32 +++++++++--------
 gtags-parser/asm_scan.l  |    7 ++-
 gtags-parser/gctags.c    |   10 ++---
 gtags-parser/gctags.h    |    4 +-
 gtags-parser/java.c      |   15 +++-----
 gtags-parser/php.l       |   13 ++++---
 gtags.conf.in            |    8 ++++
 gtags/gtags.c            |   65 ++++++++++++++++++++++++++++-------
 11 files changed, 231 insertions(+), 116 deletions(-)

----
Hideki IWAMOTO  address@hidden

20091020-gtags-2pass.patch
Description: Binary data