Re: Opaque objects and Emacs documentation

[Alan Mackenzie]

Hello, Dmitry.

On Tue, Jul 21, 2020 at 04:09:18 +0300, Dmitry Gutov wrote:
> On 20.07.2020 05:44, Richard Stallman wrote:
> > [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> > [[[ whether defending the US Constitution against all enemies,     ]]]
> > [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

> >    > Consider the docstring of said constructor function. What should it say
> >    > its return value is?

> > One natural possibility:

> >    Return an empty project object.  You need to store values in it with ...

> > Another:

> >    Return a project object with name MAME and containing files FILES.

> > I am guessing here.  I don't know what these projects do.
> > I don't know what data a project object should have.

> Okay, see, neither option is going to work.

> We have a "factory" function (called 'project-current') which delegates 
> project instance creation to a hook. Which is what makes it extensible. 
> The only argument is DIRECTORY, meaning the current directory. The 
> functions in the hook get called in turn, and the first one that returns 
> non-nil is going to be used. Its return value must be a "project 
> instance" which satisfies a certain protocol defined by several 
> cl-generic methods (think "interface" in Java, or "typeclass" in 
> Haskell, or "protocol" in Clojure, or multimethods elsewhere). One 
> method is mandatory to be defined for all project implementations, the 
> rest have default implementations.

> Thanks to the cl-generic infrastructure, the datatype representing the 
> current project can be almost anything, and the structure of the fields 
> inside can be arbitrary.

You seem to be writing that as though it were a good thing.  From the
perspective of somebody who's likely to want to debug things like this in
the future, it sounds like a Bad Thing indeed.  What precisely forces you
to use such difficult and arcane facilities?  What does it gain you that
you couldn't get out of plain Emacs Lisp (without things like
cl-generic)?

> This is sorted out by each individual backend which knows how to access
> fields inside "project instances" belonging to it, and which implements
> the aforementioned protocol for that instance type.

Trouble is, when I'm trying to debug a problem involving such code, that
style of coding adds an order of magnitude onto the time and effort
needed.

> The two built-in backends use very simple structures (right now: conses 
> like (TYPE-TAG . PROJECT-ROOT)), which is very obvious from their 
> implementations. The core of the dispute is whether project-current's 
> docstring should document the return value in detail (giving an example 
> of built-in project instances's structure). Eli is insisting on it 
> because (AFAICT) that suits his thought process and he thinks that will 
> help users/clients to make sense of how things work.

if it is not documented in detail, it will add another order of magnitude
onto the time to debug it; anybody needing to know the structure of a
return value in detail, possibly to check it's valid, would first have to
read and understand the intricate details of the source code, and
effectively recreate in her private notes the details missing from the
doc string.

That isn't fun.  Not so long ago I gave up on an Edebug bug because the
proliferation of inadequately documented cl- functions in edebug.el just
made it too tedious to continue.

> I am adamantly against this because it's against the design (clients 
> should program against the public interface, not against the 
> implementation), and is likely to encourage incorrect programs.

That's known as "security by obscurity", I think.  I think your fear is
misfounded.  For a start, if the public interface is adequate, people
will use it in preference to dissecting internal structures.  When gaps
are found in that interface, people will use the internal details to
craft a workaround whether the documentation is there or not.  (I have
done this at least once in the past.)

> Of course, we can add this information elsewhere, e.g. in the comments 
> of the "backend" functions (which are the elements of the hook I 
> mentioned above). I haven't done so because, well, the implementations 
> are fairly transparent and the structures are trivial. But I wouldn't 
> object against somebody doing that.

I think that is the job of the package's author/maintainer.

>  > It is important to document the meaning of each slot -- that may not
>  > be obvious.  Which values are valid?

> The space of possible structures of valid project instances is 
> theoretically infinite.

It remains important to document that infinite space.

-- 
Alan Mackenzie (Nuremberg, Germany).