Roughly weekly, we (I and Rocky Bernstein) will be sending status updates in this thread on my GSoC project to add source-code positions to Elisp tracebacks.
Right now this project has two components,
- reporting bytecode offsets when there is an error, and
- being able to map those offset into source locations.
With respect to reporting bytecode offsets, right now there are three branches on Savannah that implement this: feature/zach-soc-bytecode-in-traceback, feature/soc-bytecode-in-traceback-reduced and feature/zach-soc-funcall-from-bytecode.
The first tracks offsets as part of the NEXT macro (advance PC). It has a
performance impact of ~7%. So far people have expressed concern over the
degradation in performance, though I am not sure if anyone has used this and
noticed a slowdown in use.
The second branch saves the offset only before a call. Therefore, the traceback on all of the functions other than the current one are accurate, but the current one is not accurate if the error happens in a byte op.
A number of people have made suggestions for how to speed up the code on these branches. We have spent a lot of time trying to implement the more isolated changes that were suggested with little effect.
The third branch bypasses invoking Ffuncall from within exec_byte_code, and
instead does essentially the same thing that Ffuncall does, right in the Bcall
ops. This means the offset can be stored in the backtrace frame without the need
to make it globally accessible in the thread_state. Additionally, lexically
scoped compiled functions are passed directly to a recursive call of
exec_byte_code. The speed gained from that outweighs the overhead of tracking
the offset. But some changes are necessary to make it viable.
All of them print the offset next to function names in the backtrace like this:
Debugger entered--Lisp error: (wrong-type-argument stringp t)
string-match(t t nil)
13 test-condition-case()
load("/home/zach/.repos/bench-compare.el/test/test-debug...")
78 byte-recompile-file("/home/zach/.repos/bench-compare.el/test/test-debug..." nil 0 t)
35 emacs-lisp-byte-compile-and-load()
funcall-interactively(emacs-lisp-byte-compile-and-load)
call-interactively(emacs-lisp-byte-compile-and-load record nil)
101 command-execute(emacs-lisp-byte-compile-and-load record)
(with varying degrees of accuracy, as described above). You can then M-x disassemble
the function to find the index and see which instruction made things go wrong.
If there are others who want to improve the code we have or follow through implementing any suggestion, we’d be grateful. Feel free to branch off the code in Savannah and continue.
If we go with the second solution which just updates the bytecode offsets before
funcalls, which has negligible overhead, then we can probably get this into
master sooner. A comparison of the performance using Andrea Corallo’s
elisp-benchmarks.el can be found in this file.
Getting something of this nature into the sources takes time and careful consideration. The C changes for the reduced offset tracking are pretty small, but there is still a noticeable improvement in traceback information. Those of us who care about better traceback location agree this is not ideal, but it is better than the no-location situation that is there now.
So right now we are proposing just location information on calls, and then we will gauge what people think after trying this, and if they would like more we will pursue that further.
With respect to reporting offsets, using code from edebug we have a
Lisp-_expression_ reader that will track source-code locations and store the
information in a source-map-_expression_ cl-struct. The code in progress is here.
Information currently saved is:
- The _expression_ itself
- The exact string that was read
- Begin and end
points of the sexp in the buffer source-map-_expression_children (for conses and vectors)
which can be generated for a whole lisp file with the function source-map-file.
We are testing this on lots of files such as the lisp files in the GNU Emacs
distribution. After this is done we will try hooking this into the compilation
process.
Feel free to contribute to all the links in this message and extend in the areas
mentioned. The sooner this gets done, the sooner we can move onto next steps,
toward the ultimate goal of line-and-column code locations in *Backtrace*.
- Zach and Rocky