Re: scratch/accurate-warning-pos: next steps.

[Alan Mackenzie]

Hello, Paul.

On Tue, Dec 11, 2018 at 10:05:49 -0800, Paul Eggert wrote:
> On 12/11/18 3:34 AM, Alan Mackenzie wrote:

>  > The list of known functions can be reliably generated by objdump 
>  > (from binutils).

> Would objdump be run on every build that compiles a .c file that goes 
> into the Emacs executable?

I don't know, at this stage.  Probably not.

> If so, aren't we limiting builds to platforms that have binutils,
> which would be a new restriction?

Well, we use ld, which also belongs to binutils, and that doesn't seem
to restrict the platforms.  Other platforms surely have equivalents to
both objdump and ld, and they are/would be used appropriately.

> And if not, wouldn't the objdump use be a hand-done process that would
> need to be redone on a binutils platform (with output committed to
> Git) whenever a significant-enough change is made to src/*? Either
> way, this sounds like a hassle.

globals.h seems to manage.  The objdump output could be generated
analogously.  Somehow.  Probably.

>  >> .... and Emacs already uses the C preprocessor aggressively.
>  >> Instead, why not use the C preprocessor itself, rather than
>  >> writing another preprocessor for C? In other words, compile each
>  >> file twice, once with one -D option and once with another.

>  > Because the two interpreters will need to share file static data,
>  > of which there must be only one copy. So the two versions of each
>  > function need to be in the same "source" file.

> It should be easy enough to move shared file-static data into another 
> file, that would be compiled only once. We don't have so much shared 
> file-static data that this would be a major obstacle.

Possibly not.  The same would have to be done with file global data,
too.  But doing it that way would involve a great deal of change to the
source code (testing for the -D option) which would not be popular.

>  > The form Stefan suggested is MUCH bigger than the plain form, having,
>  > perhaps four times the number of conses (I haven't counted them).

> This overhead would occur only when byte-compiling the form, which 
> shouldn't be much of a problem in practice.

It would likely slow down the compilation by a very great deal.

>  > This preprocessor would be tedious rather than difficult to write.

> ...

>  > A large part of the compiler would need to be amended to cope with
>  > the new format, even supposing it could work with macros (which I
>  > don't think it could). This amendment would be uninspiring and
>  > tedious in the extreme.

> I agree that either approach would be tedious. :-)

No.  You're conflating "tedious" with "tedious in the extreme".  They're
different.  The first is several days of work.  The second is many weeks
of work.  I tried an approach two years ago which involved amending most
of the compiler.  Although spending a fair amount of time on it, I
didn't get very far, and gave up.  Were the reader to produce "Stefan's
form", the work to amend the compiler would be more even than what I
gave up on.

> However, a tedious approach that is limited to reading and
> byte-compilation is better than a tedious approach that affects all of
> execution.

My proposed approach would affect only byte compilation (and the build
process, of course).  That's the whole point.  Besides, my approach will
work.  The competing half-baked ideas are not even fully formulated, and
likely wouldn't work.

>  >> for more info, see Gemini's followup 
>  >> <https://lists.gnu.org/r/emacs-devel/2018-12/msg00043.html>.

>  > I've read this several times. It suffers the same drawbacks as
>  > Stefan's idea. In particular it doesn't give any idea how the
>  > compiler would operate on the proposed forms.

> As I understand it, Gemini and Stefan are thinking of essentially the 
> same thing: have the reader optionally generate symbols-with-positions, 
> have the compiler deal with symbols-with-positions, and have the 
> compiler strip positions before passing forms to macro arguments that 
> are not annotated to accept positions. Although (as you mention) this 
> would require amending a large part of the compiler in a tedious way, it 
> should solve the problem for macro arguments that accept positions.

This would place onerous restrictions on what macros were allowed to do,
and likely be incompatible with a vast proportion of existing macros.

>  >> <https://lists.gnu.org/r/emacs-devel/2018-12/msg00023.html>

>  > I don't think it would work, either. That idea is for macros' uses
>  > of eq to be replaced by BC-eq inside the macro. The trouble is,
>  > many uses of eq are actually expansions of EQ in the C code (e.g.
>  > in Fequal, Fassq, ....) and they would all need modifying too, and
>  > we're back in the same situation of having an alternative
>  > interpreter.

> No, the idea is that the onus of doing comparisons correctly is on the 
> writer of any macro annotated to understand symbols-with-positions. That 
> is, it's the macro's responsibility to use appropriate comparison 
> operations, and this responsibility extends to comparison operations 
> like EQ that are executed in C code.

I think that is an unacceptable change in Emacs.  Macros are already
difficult enough to write.  Such restrictions could make macros
impossible, except for "experts".  Besides, we want to maintain
compatibility with the vast body of existing macros.

> For example, I suggested that 'equal' should ignore symbol positions, as 
> this would let these macros use 'equal' instead of 'eq', 'assoc' instead 
> of 'assq', etc. 
> <https://lists.gnu.org/r/emacs-devel/2018-12/msg00033.html>. Although 
> 'assoc' is written in C and its source code uses EQ, the source code 
> would not need to be changed nor would it need to be compiled twice, as 
> 'assoc' defers to 'equal' (i.e., Fequal in C) to do the tricky work and 
> 'equal' would do the right thing.

The trouble is, macros DO use eq.  And why not?

The contortions you're envisaging contrast horribly with the simplicity
of scratch/accurate-warning-pos, which simply works.  It works because
it has not changed any of the basic types or interactions between them.

-- 
Alan Mackenzie (Nuremberg, Germany).