Re: Problem running gtags on Drupal

[Hideki IWAMOTO]

Hi.

> To Iwamoto san:
> 
> It seems that we can change the limit using the -D option of
> the latest flex like this:
> 
>       flex -DYY_BUF_SIZE=65536 ...
> 
> I would like to change the limit using this option.
> How about changing the required version of flex to 2.5.6 or later?

I think that the limit of flex which caused the problem is that
using yylineno is regarded as using REJECT. 

flex-2.5.4:
        if ( do_yylineno )
                /* This should really be "maintain_backup_tables = true" */
                reject_really_used = true;

Since this code is eliminated in flex-2.5.11 and later, option -D may be
unnecessary if required version is changed to "2.5.11 or later". 

http://flex.cvs.sourceforge.net/viewvc/flex/flex/main.c?r1=2.94&r2=2.95


I could not find flex-2.5.11. 
Therefore, I checked the output of flex-2.5.4 and flex-2.3.35. 
========================================================================
$ flex --version
flex version 2.5.4
$ grep REJECT libparser/*.c
libparser/asm_scan.c:#define YY_USES_REJECT
libparser/asm_scan.c:#define REJECT \
libparser/asm_scan.c:#ifdef YY_USES_REJECT
libparser/asm_scan.c:"input buffer overflow, can't enlarge buffer because 
scanner uses REJECT" );
libparser/php.c:#define YY_USES_REJECT
libparser/php.c:#define REJECT \
libparser/php.c:#ifdef YY_USES_REJECT
libparser/php.c:"input buffer overflow, can't enlarge buffer because scanner 
uses REJECT" );
========================================================================
$ flex --version
flex 2.5.35
$ grep REJECT libparser/*.c
libparser/asm_scan.c: * any uses of REJECT which flex missed.
libparser/asm_scan.c:#define REJECT reject_used_but_not_detected
libparser/php.c: * any uses of REJECT which flex missed.
libparser/php.c:#define REJECT reject_used_but_not_detected


On Thu, 31 May 2012 09:20:59 +0900
"Shigio YAMAGUCHI" <address@hidden> wrote:

> Hi,
> 
> It seems a limit of flex(1). The php parser of gtags(1) uses flex(1).
> 
> Please try the following to extend the limit.
> 
> [in GLOBAL project]
> 
> $ vi libparser/php.c
> 
> [libparser/php.c]
> +-------------------------------
> |#define YY_BUF_SIZE 16384
>       |
>       v
> +-------------------------------
> |#define YY_BUF_SIZE 65536
> 
> $ sudo make install
> 
> 
> To Iwamoto san:
> 
> It seems that we can change the limit using the -D option of
> the latest flex like this:
> 
>       flex -DYY_BUF_SIZE=65536 ...
> 
> I would like to change the limit using this option.
> How about changing the required version of flex to 2.5.6 or later?
> 
> > Hi,
> > 
> > When I'm trying to generate tag files for Drupal I run into this error
> > message (and exit status 2):
> > 
> >  input buffer overflow, can't enlarge buffer because scanner uses REJECT
> > 
> > 
> > I narrowed the problem down to one file in Drupal:
> > 
> >  
> > http://drupalcode.org/project/drupal.git/blob_plain/30d1e719aa5e9a9ad66514078ca3b0975ddadc9c:/mo
> dules/system/system.api.php
> > 
> > 
> > And inside that file I narrowed the problem down to a large block
> > comment just before the hook_menu() function.
> > 
> > The block comment (lines 905-1209) is 305 lines long and removing any
> > ~18 lines from the comment eliminates the problem.
> > 
> > Now, I don't have commit access to Drupal so I can't shorten the comment
> > by 18 lines myself and I'm not sure that would be the right solution
> > anyway :-)
> > 
> > Is this problem a known limit in gtags or something? Or is there some
> > way to get around the problem (besides using
> > :skip=modules/system/system.api.php)?
> > 
> > I'm using GNU GLOBAL 6.2 (and also tried 6.2.2 and 6.2.3) on Mac OS X
> > using Homebrew. I have a coworker running Ubuntu who experiences the
> > same problem (don't known his version though).
> > 
> > Best regards,
> > Arne
> --
> Shigio YAMAGUCHI <address@hidden>
> PGP fingerprint: D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3

-- 
岩本 秀樹 <address@hidden>