Re: Learning the Octave interpreter code to implement Java class dot-referencing

[Andrew Janke]

On 7/8/20 1:28 PM, John W. Eaton wrote:
> On 7/8/20 2:27 AM, Andrew Janke wrote:
>> Hi, Octave maintainers,
>>
>> A while ago, I took a stab at adding support for dot-reference syntax
>> for Java classes. (https://savannah.gnu.org/bugs/index.php?41239) I
>> totally failed, because I don't understand the Octave interpreter code
>> well enough, and don't have bison/yacc skills.
>>
>> Can anyone point me at resources for learning about the Octave
>> interpreter code, or bison/yacc generically, that would help me get good
>> enough to write a patch for this?
> 
> Unless you need new syntax for this feature,  I don't think you'll need
> to know too much about Bison.
> 
> Do you want to make expressions like
> 
>   java.lang.Double (42)
> 
> ?  Is this similar to invoking static methods for classdef classes?  If
> so, maybe we can do the same kind of thing that we do for those?   Or,
> maybe there is a better way?

Exactly. I want to support the

  java.lang.Double(42)

constructor invocation form, and the

  java.lang.Double.parseDouble("42")

static method invocation form. It is completely analagous to invoking
constructors or static methods on M-code classdef classes.

> Here is what happens for classdef: an expression like
> 
>   myclass.method (args)
> 
> is evaluated in tree_index_expression::evaluate_n in pt-idx.cc.
> 
> When evaluating the first component of the expression ("myclass"),
> Octave will find and load the constructor for the myclass object.  This
> step happens in the fcn_info::fcn_info_rep::xfind function (around line
> 740 of fcn-info.cc).  But since we don't know at that point whether we
> are looking up a constructor call or the name will be used to invoke a
> static method, we get a classdef_meta object (a builtin function object)
> instead of the constructor function itself.
> 
> Then, back in tree_index_expression::evaluate_n, we see that we are
> indexing an object (classdef_meta) and then gather up the remaining
> arguments to pass to the classdef_meta subsref method.
> 
> So, to handle "java" similarly, we would need some kind of java_meta
> object with an appropriate subsref method.  That object would be
> inserted in the function table when the octave_java type is installed.
> Then fcn_info::xfind can return that object when it sees the "java"
> symbol.  Or, if we only have to handle the single word "java", maybe it
> could just be a special case at the appropriate place infcn_info::xfind
> to get the precedence right.

Yep, a similar mechanism for Java sounds right to me.

(It's not just the single word "java"; Java packages can start with any
identifier, just like Octave packages, and most of the interesting ones
don't start with "java.*", so we'd have to handle all identifiers.)

The one complication is in resolving package-qualified names. Consider:

  x = foo.bar.baz.Qux(42);

The namespace "foo.bar.baz" could be either a Java package or an Octave
classdef package. And in fact, it could be both in the same codebase: in
Matlab at least, M-code and Java namespaces do not mask each other, and
identifier resolution is done at the fully-qualified class level. So in
order to determine whether this expression should dispatch to a Java
constructor or an M-code classdef constructor, the interpreter needs to
consider the full expression "foo.bar.baz.Qux", and not consider "foo"
on its own and resolve that independently.

That is where I got stuck when I first tried this. Based on my reading
of the libinterp code, the Octave interpreter takes the components
"foo", "bar", "baz", and "Qux" one at a time, and ends up eagerly
resolving "foo" to an Octave namespace, and then proceeds to resolve
"bar" relative to that. Doing it this way I think means that package
prefixes in Java and M-code would need to mask each other, and do so at
the prefix/component level.

And in the Java world, packages names are not hierarchical: the
existence of the "foo.bar.baz" package does not imply the existence of
or have any relationship to the "foo" package. The dots are just plain
characters that are part of the package name, and not special; arranging
those packages in a hierarchy based on prefixes is just something done
by humans and Java IDEs as a convenience for developers. So, given a
Java class "foo.bar.baz.Qux" loaded in to your JVM, if you queried the
JVM for the existence of the "foo" package, it would return false
(unless some other class separately defined some "foo.Whatever" class).

In practice, maybe this isn't an issue: M-code developers don't use
package names the way Java developers do, so it seems unlikely to me
that there would actually be a namespace conflict. So it would probably
be fine if Octave did allow M-code and Java packages to mask each other.
And Octave could, I think, just get a list of the Java packages from the
JVM, and then build the fake subpackage hierarchy internally.

A java_meta object sounds like a good approach to me.

Cheers,
Andrew

> I'm not 100% certain, but I think the classdef_meta object is derived
> from octave_function instead of just being a value so that it can't be
> wiped out by a variable definition.  But if there is a better way, then
> maybe we can simplify the way the whole classdef_meta thing works at the
> same time we implement support for java static methods.

I'll have a look at classdef_meta.

> 
> jwe
> 
>