2003 Grants - Fourth Quarter

Mark Jason Dominus - Debugger

AMOUNT: $3500

DURATION: 3 months

DISCUSSION:

What is wrong with the existing debugger?

  • The current debugger is 2600 almost-unreadable lines of magic and special cases. Its structure is a giant if-else tree that looks at the command that was entered and then executes some code depending on the command. The only way to define a new command or to alter the behavior of an existing command is to hack on this giant if-else tree. Bugs in the debugger are hard to repair because the debugger is not written in a modular style.
  • The debugger is full of weird code that nobody understands. Every time some bright person got a clever idea for a clever feature, they hacked it into the core code for the debugger. Some of these features are incompletely implemented; some just don't work. For example, there is supposed to be a feature in the debugger that detects when the target program calls fork(), and will start up a fresh terminal to follow the child process. There is code for this feature in the debugger, but as far as I know nobody knows how to use it. It is not documented.
  • The debugger has no test suite. It has been repeatedly broken by changes that could not be adequately tested. Because of the existing architecture, it would be very difficult to add a test suite.
  • The debugger is not adequately configurable. Every time some anal-retentive pinhead decides to rationalize the command set, or to extend a command, or add a new command, they do it in the core, and everyone else has to live with the changed behavior.
  • The debugger promotes bad programming practices. People often write this code:
        my $I = mkdir(...);
        if ($I) {
          ...
        }

This could be written more briefly and clearly as:

        if (mkdir(...)) {
          ...
        }

When I suggest this in my classes, a common reply is that the long version is better because the user will be able to examine the return value of 'mkdir' in the debugger before the branch is taken. This is a good answer, and it points directly at a defect in the debugger. There should be an option to place a breakpoint after the condition is evaluated, but before the branch is taken. Perl can support this, but the debugger won't do it. Similarly, one might like to place a breakpoint in the middle of an expression like this:

        $result = complicated_subexpression_1 + complicated_subexpression_2;

and have the debugger stop automatically, display the value of complicated_subexpression_1, and offer an opportunity to change it before evaluation of the expression continues.

  • I have been studying the way people use debuggers. Most debuggers, including the perl debugger, are awful. As far as I can tell, debugger technology has not advanced a single step since 1962. Perl can do better. I have at least half a dozen ideas for useful debugger features that people will wonder how they ever got along without.

Here is a simple and obvious example. Everyone has the experience of discovering that they have stepped three steps past the place where they wanted to stop. Then they sigh, kill the debugger, and start over.
A debugger that can be stepped backwards is probably impossible. But there's no reason why the debugger can't remember the sequence of commands you used to get it to where it is, and then accept a command that says "Start over, and do everything just the same, but stop three lines sooner." This is easy and obvious, but the Perl debugger doesn't do it.

  • The debugger will not debug regexes. I wrote a regex debugger back-end several years ago. I put new features into the core to support better regex debugging. Then neither I nor anyone else followed up. If I rewrite the debugger, I can integrate regex debugging support into it.

DELIVERABLES:

  1. A reusable library of essential debugger functions. This library would abstract out the weird debugger guts manipulation that is necessary to display source code, set break points, and so on. Debugger applications could be written atop this library.

The library should be designed to be easily subclassed so that people can extend it. Useful extensions would eventually be merged back into the core library. The subclassing interface should be well-documented.
If the underlying implementation of Perl's built-in debugger features changes, only the library would need to be rewritten. It's easy to imagine that the debugger facilities in Perl 6 or even Ponie will be very different from the one we have now. If this is true, the current debugger will have to be thrown out anyway. Any work done on the old debugger from now on will be wasted.
The library should support plugins for sub-debuggers that take control at certain points in the debugging process. The regex debugger would be one of these plugins.
I may also deliver some small core patches if they enable comparatively large improvements in debugger function. For example, consider a graphical debugger in which one can point to a variable on the screen and immediately see its current value. This is much easier for debuggers to do if the Perl lexer provides a little support for it.

  1. A replacement debugger application. The new debugger application will be backward compatible with the current debugger, but the architecture will be different. Instead of being a giant monolithic application, it will be a thin front-end to the debugger library. Building new debuggers will be easy to do by writing new thin front-ends.

If some anal-retentive pinhead wants to rationalize the command set, it should be possible to tell them to do it in their own copy of the debugger. If we can't stop them from touching the Perl distribution, we should at least be able to have their changes in as an option which can be enabled or disabled.

  1. A working regex debugger. The regex debugger should be integrated with the debugger front-end and easily usable in conjunction with it.
  2. A test suite for the library and for the standard debugger application. The architecture of the front-end application must be designed to allow easy regression testing.
  3. Complete documentation. The library and front end should be well-documented, to encourage development of alternative and experimental debuggers.
  4. Six months after the delivery of items 1-5, I will deliver a report on the mistakes made in the project and lessons learned.

COMMENTS:

I think this project needs to be done as soon as possible. Ponie and Perl 6 are coming along. When that happens, we will need to redo the debugger from scratch anyway.

Fred Brooks said "Build one to throw away; you will anyway." My debugger will be the one we can throw away, if we want to. Of course, I will do it absolutely as well as I can so that we don't have to throw it away. If we don't, that is excellent. If we do throw it away, it will be because my project taught us a lot of things about what mistakes we shouldn't make. If I don't do this project first, we will make those same mistakes when it really matters, in the Perl 6 debugger.