Troubleshooting Yourself in the Foot


What makes some programmers better than others? I’m sure you can find a couple of popular software industry bloggers that have strong opinions about this, but I’m more interested in specific empirical answers to the broad question.

I once found some answers in a humble-looking library book, titled _Empirical Studies of Programmers: Papers_ by Elliot Soloway.  

“Ermahgerd, berks!”

When I was a computer science undergrad, I spent a lot of time in my university library. My friends were used to it; if I had disappeared for an entire afternoon, they could probably find me with my head tilted sideways, browsing stacks in the CS or math shelves.

That’s how I stumbled upon this book, which is actually a collection of papers from the First Workshop on Empirical Studies of Programmers, held in 1986. When I found Soloway’s book in 2001, I was definitely in the “novice programmer” category. Ten years of experience later, it’s useful to reflect on the wisdom of that research in my modern context.

One paper that caught my attention was _Comprehension Differences in Debugging by Skilled and Novice Programmers_ by L. Gugerty and G. Olson, both from the University of Michigan. The experiments involved LOGO, which was the first language I learned, circa 1992). The authors asked groups composed of novice and expert programmers to debug programs written in LOGO, and observed the results.

One of the important results from the paper — and my topic here — is that novice programmers tend to make a specific kind of mistake more often than expert programmers:

Novices often added their own bugs. We defined adding bugs as testing an incorrect hypothesis which changed the program’s output and not undoing this change before testing the next hypothesis. […] For novices, adding a bug was correlated with being unsuccsessful at debugging. When novices failed to get the program to work in the allotted time, 92% of the time they had added at least one bug of their own. […] In contrast to this novice behavior, only once did an expert ever add a new bug to the program.

This was a big eye-opener for me as a novice! Looking back, I definitely remember this kind of thing holding me back in early work. It also helped me mentor other programmers since then. I was glad of the foreknowledge to look for this kind of mistake and help learners overcome it. Fortunately, the paper suggested some ways their experts overcame this bad habit…

Understand what you are troubleshooting, and don’t make new bugs while fixing others.

Here’s an excerpt from the end of that paper:

In conclusion, these experiments suggest that expert-novice performance differences in debugging are due, at least in part, to the different program-comprehension abilities of these two groups. [We] also suggested that experts and novices may differ in the use of remembered symptom-bug associations (symptomatic search) and of strategies such as backtracking (topographic search). At least in the LOGO experiment, the expert-novice differences in these experiments could be due to experts using topographic search strategies more effectively.

The take-away is that experts were utilizing their mental modelling skills to better understand the problem they were debugging. Furthermore, they used specific methodical techniques (symptomatic and topographic search) more effectively to prevent creating new bugs by guessing or forgetting to remove previous breaking changes.

I’d love to hear stories of this kind of novice-expert interaction, or other lessons learned about not shooting your own foot while trying to fix code. Has this happened to you?