[Bug-librejs] JS Parsing bugs prevent @license and @license-end from working

[Micah Cowan]

(Please CC me, I'm not subscribed.)

I've been working on a project called StarfishJS, which is release
under GPLv3+, and uses minified jQuery/jQuery-UI. Since it's all Free
Software, I figured I should play nice with LibreJS, so I put tags at
the various appropriate places.

Playing nice with LibreJS turned out to be more challenging than
expected: I slapped the appropriate @license / @license-end around the
minified jQuery and jQuery-UI sources, but LibreJS was still reporting
them as "NONTRIVIAL". I figured I must have done something wrong, but
I managed to figure out how to get the Firefox Add-On debugger going
(first time for me), and stepped through code to figure out where it
was going wrong, under the assumption that some small discrepency was
tripping me up (such as the required "the following/preceding" text
for @licstart/@licend had previously done briefly). It took me some
time to figure out what was going on, partly because of LibreJS's
caching which I wasn't aware of (but makes sense), so I eventually
triggered my breakpoints by adding nonsense characters to comments in
the minified JS sources.

After plenty of banging around, I finally found out what was going on
(but not why): within js_checker/free_checker.js's matchMagnet method,
it was recognizing and accepting the @license tag, but when it
searched the current node queue for the ending @license-end comment
(which is indeed present at the end of the file), no nods with that
comment block were found.

That's as far as I got: I've spent enough time debugging this issue
and don't really want to spend more time on it. Now that I know that
it's apparently a parsing issue within jQuery/Narcissus and not my
incantation, I'm not really motivated to dig it the rest of the way
out.

This was with LibreJS 6.0.8 running in Firefox 41.0.1. LibreJS 6.0.9
is completely broken for me (gives that "Error parsing: Reference
Error: Type is not defined" message for every page it finds). I tried
building latest git sources, and successfully built an xpi (another
first for me), but that result appeared to have no effect for me in
Firefox (no LibreJS icon in the toolbar, no notifications during
analysis, despite my Preferences setting to do so). (I'd like to note
here that inconsistently-working packages is liable to discourage
developers from making the effort to ensure their Free JavaScript
programs play nicely with LibreJS.)

A version of my project that still exhibits this problem is at

  http://micah.cowan.name/starfishjs-librejs/

This version differs ever-so-slightly from the one I was actually
testing against, in that I added @source tags to jquery-2.1.4.min.js
and jquery-ui.min.js, whereas during testing I had omitted those (to
prevent LibreJS going out and searching the source for additional
license information when it failed to find it in the minified
version). I'm assuming that's not significant for debugging purposes.
but if you have trouble reproducing the problem as I described it, you
might try removing the @source tags to see if it makes a difference.
My debugging was focused on jquery-2.1.4.min.js, not jquery-ui.min.js,
in case that also makes a difference.

I've worked around the issue in the official project by using the
license-table mechanism instead, you can see the version that LibreJS
6.0.8 currently accepts for me, at

  http://micah.cowan.name/starfishjs/

...If I might make a suggestion, I think that it might ultimately be
simpler to get correct functionality by parsing the comments outside
of the tree, at the lexer level (or perhaps as a separate pass, but
that's liable to slow down an already slowish process). If a comment
is encountered before any non-root-level, non-comment nodes have
started, it could be scanned for @license. Then, further comments are
checked for @license-end, and if the last lexical item seen was such a
comment (optionally followed by further comments or whitespace), it
marks the whole tree with the information it found (license checking
wouldn't happen at this stage, just capturing).

.

Also: I was dismayed to find upon downloading the source tarball from
the LibreJS site at gnu.org, that it unpacks in the current directory
rather than a subdirectory. I've come to expect better from GNU
packages in particular (if you're using "git archive", don't forget to
use "--prefix").

.

Thanks for your attention!

-mjc