A new Guile debugging interface ------------------------------- 1. Requirement/Motivation To me, the business of developing and debugging Guile Scheme code is not yet very neat. Scheme is theoretically a language that lends itself to interactive development, but as yet there is no UI or IDE for Guile that makes interactive development easy in practice. A particular example is debugging. Guile is pretty good about backtraces which allow you to explore the context of an error, and the CVS version of Guile has some support for breakpoints. But all this currently works only through a klunky command line language, and only for applications that have a command line in the first place - which rules out interesting graphical apps like Lilypond and Gnucash. What we need is a new interface that makes debugging a more pleasant experience, and which works equally for text-based and graphical apps. 2. Description of Features I envisage Guile applications communicating through some channel with an Emacs frontend. The debugging channel is out of band with respect to whatever other mechanisms the application uses for interaction; in practice it will probably be a TCP connection over sockets. Specifically, for a command line application, including the standard Guile REPL, the debugging channel is completely separate from the application's standard input and output. (This avoids the problem inherent in a comint-based approach of determining when the application is in a good state to receive new instructions from the IDE, e.g. to set a breakpoint or to reevaluate some code). For a graphical application, the lack of any standard input or output is obviously no longer a problem, as the debugging channel does not use these. Library code in the application uses the debug channel to: - pause the application's processing until the frontend allows it to continue - send information about an error that has occurred or a breakpoint that has been hit, and their backtraces - send information about application context, such as what modules have been loaded - send tracing information (without pausing the application). The Emacs frontend uses the debug channel to: - set breakpoints - send code to the application to be evaluated - tell the application to continue - obtain help for symbols known to the application. To show all this working together, I envisage a typical debugging session, for a graphical Guile-based application, proceeding as follows. - The application is started with with some application-defined option that tells it to support debugging. Because this option is present, the application at some point during its startup calls `(debug)'. - `(debug)' establishes the debug channel to the Emacs frontend, announces the presence of the new application, and blocks its thread (=> the whole application, if single-threaded) until the frontend tells it to continue. - If the intent is only to debug errors, the frontend (under the control of the Emacs user, of course) tells it to continue immediately. - Otherwise the frontend user may want to set breakpoints. To help with this, there may be exchanges between the application and the frontend to discover what modules are loaded, what procedures are defined by each module, what their documentation is, etc. - Once all desired breakpoints have been set, the frontend tells the application to continue. - When the application hits an error or a breakpoint, it sends information about it down the debugging channel, including full stack trace, and again blocks except for servicing requests from the frontend until it is told to continue. The frontend user can now: - explore the stack, with Emacs showing the corresponding source code for each frame - send test expressions for evaluation in a specified frame - modify the application source code and send it to the application for reevaluation, to affect operation next time around - set new breakpoints, or delete or modify existing ones - when done, tell the application to continue. - And so on to the next error or breakpoint ... In this scenario the application only accepts instruction from the frontend at particular times: at startup, or when stopped at an error or a breakpoint. Some applications may be able to accept instructions more permissively, more or less whenever the frontend choses to send them. For such applications the debugger library code makes it easy for the application to check for and process such instructions whenever it likes. 3. High Level Design The high level design has three components: - debugger library code running in the application (written in Scheme) - an independently running debug server process (written in Scheme) - Emacs frontend code (written in Elisp). The Emacs frontend starts the debug server as an Emacs subprocess and communicates with it through a pipe. The debug server listens for connections from Guile applications on a well known port. Once connections are established, the debug server receives debug forms from the applications that it forwards via its pipe to the frontend code, and receives debug forms from the frontend, tagged with an application ID, that it forwards to the intended application. The debugger library code implements easy-to-use functions like `(debug)' by establishing a connection to the debug server process and managing the exchange of debug forms through that connection. 4. Low Level Design Very much in progress, but here's what I have so far ... 4.1. Debugger library code Main details to be added here. :-) One extra component that may prove useful here is a soft port that selects on the debugging channel as well as on its usual input. This would allow a simple command line application to support debugging just by using that port. 4.2. Debug server This is simple enough that it hardly needs further design, and I've already written it anyway. 4.3. Emacs frontend code Lots of complexity here. Many interesting ideas come from Mikael Djurfeldt's guileint interface, but the end result will look quite different from that, I think. The following subsections describe some of the ideas that I have in mind. 4.3.1. Tracking the relationship between Scheme source buffers and the application they are relevant to [This idea comes from guileint.] Suppose a debugged application hits an error, and Emacs pops up a buffer displaying the source code for the error location. If the user edits and then wants to reevaluate part of that source code, the frontend should know that the application to send the evaluation to is the one that just hit the error. Then there is the case where the edit is in a file not yet known by the frontend to be associated with the application ... Getting the associated application is ultimately a heuristic effort. Therefore the goal is not to implement something provably correct, but to guess helpfully what the user wants 90% of the time, provide useful cues as to what the frontend is guessing, and make it equally easy for the user to go with the guess and to override it. 4.3.2. Enhanced Scheme buffer editing [This idea comes from guileint.] Enhanced editing helps the user by keeping track of which top level forms have been edited since they were last evaluated by the associated application, and provides a convenient `reevaluate all edited forms' command. It also remembers, for each top level form, what the result of the last evaluation was. It also detects when a top level form is syntactically broken - i.e. its parentheses don't balance. Broken forms are excluded from the `reevaluate all edited forms' command. Finally, it visually indicates the current state of each top level form, which is one of `up to date', `edited' or `broken'. 4.3.3. Application and module browsing This is a tree-like display of: Applications Modules (used by each application) Bindings (of each module) It can be used to: - set and configure breakpoints - see the current value of a binding - see the documentation for a binding. There are several possible methods for getting the list of modules for an application, and the list of bindings for a module, from static scanning to actually loading or running the code and seeing what results. The desirability and safety of each method depends on the code itself and is difficult to determine heuristically, so the frontend simply provides all the available options and leaves the choice up to the user. (Perhaps defaulting to a safe static scan.) I think it would be good to implement this as a specialized kind of Speedbar, to avoid completely reinventing the wheel. 4.3.4. Error location and stack display [This idea comes from guileint.] When an error or breakpoint is hit, the frontend displays, in 3 windows: - the relevant source code - the stack - the error or breakpoint details. At such times, the frontend provides convenience commands for exploring the stack, evaluating variables, single stepping, continuing execution and whatever else is useful. 4.3.5. Debug session recording and replay One disadvantage of a graphical interface is that it can be difficult to keep a log of actions performed and to repeat actions if necessary. The Emacs frontend remedies this by keeping a record of actions performed, on a per-application basis, and making it possible to replay them. 4.4. Debugging channel protocol (name NAME) First form sent by library code to indicate application's name. Lots more detail to be pinned down here! :-) 5. Pie in the Sky One day, I'd like the `Emacs frontend' to be another Guile process, using Elisp translation to interpret the Elisp code and displaying buffer contents (and so debugging information) in Gtk windows.