Re: Is intellisense features integration in Emacs technically possible?

[Eric M. Ludlam]

On 01/22/2014 12:29 PM, Phillip Lord wrote:
> Eli Zaretskii<address@hidden>  writes:
> > > A better way is to build on the hard work of other and interface
> > > emacs with an external tool.
> >
> > Personally, I think implementing such features via external programs
> > is a terrible design.  It will never be smooth and responsive enough,
> > and on top of that you'd need to track development of those other
> > tools.  And what if they become abandoned some day?
>
>
> I think that it depends on the language. Introspecting over, for
> example, Java would require an awful of elisp, which would be difficult
> to write. Getting Java to do this work is quite a lot less effort.
> Hence, the JDEEs use of Java for this (via bsh). Likewise, Clojure and
> Scala both of which use their own language to do much of the work. Or
> for that matter, common lisp with slime/swank. Or even, for that matter,
> English with aspell. I didn't have a problem with responsiveness with
> any of these.

I don't think it is a good idea to consider the problem of doing smart 
completion in Emacs as "Emacs Lisp VS External library".  It is much 
more advantageous to figure out what each is best at and do that instead.

There are certainly examples on both extremes.  CEDET and Semantic by 
themselves can parse surprisingly large C++ code bases and provide smart 
completion quite accurately, but as the code base grows, it gets slower. 
 It also took a really long time to implement in my spare time in spite 
of the vast amount of help I've gotten from its users.

There are also completion you can implement by passing entire buffers 
into an external parser as explained by Jorgen for Python, and I think 
clang could do something similar.  These were coded more quickly, and 
can provide rapid results.

I think it would be better to have a strong mix between the two.  I am 
of course a bit biased since I've put support for this in CEDET quite a 
while ago.  The premise for those who haven't studied how the smart 
completion engine in CEDET works is pretty simple.  Emacs is in charge 
of basic buffer parsing.  As David explained, it is a simple tagging 
parser, but unlike etags/ctags, it collects much more detailed 
information.   It also has a local context parser, allowing it to find 
local variables, and identify the kind of command the cursor is sitting 
on, such as an assignment, struct dereferences, etc.

Once that basic part is done, the smart completion engine starts looking 
up symbols.  It does this via a series of 'semanticdb' EIEIO objects 
that are associated with the buffer.  With no external program support, 
it uses an all Emacs solution to lookup header files or whatever that 
language uses.  Alternately, there are external programs wrapped up in 
this EIEIO class that can provide the same services.   Thus for Java 
there is a 'javap' service that can find symbols in .jar files to 
provide the answer.

There are many external programs such as GNU Global, idutils, cscope, 
and javap that are already supported, and each provides different levels 
of support based on what they can do.  They all make CEDET faster or 
more accurate when in use.

It is tempting to go with an entirely external solution.  They can 
usually be wrapped up by Emacs in pretty quickly.  There are additional 
advantages aside from smart completion to adding parsing support for a 
language directly to Emacs however.  What CEDET does is place overlays 
in the buffer identifying the tags and their nesting.  Simple queries 
can tell you exactly where you are structurally in a program with no 
regexp calls, searching, or other time-consuming activities.  You can 
use this for breadcrumbs, decorations, named bookmarks, navigation, and 
any kind of function that needs to know where you are by name. 
Semantic also can provide access to the current buffer tag-list at next 
to 0 cost when asked (assuming the idle timers have had a chance to run) 
making think like imenu, which-func, speedbar, and ECB very fast.  It 
seems unlikely an external tool could do all that without quite a bit of 
effort.

Lastly, most parts of CEDET/Semantic's parsing and completion engines 
can be replaced per-mode at almost any level, not just at the database 
level.  As such, if someone really wants to use an external tool to 
parse a buffer or provide completion (2 different operations), it will 
let you do it, and all the existing CEDET tools will then work just fine 
on top of the result.

Eric