Just Point Your Defects Already!

We have been using agile workflows on our teams at Atomic since we were founded back in 2001. User stories have always required points, although there has long been a debate about whether or not a team should point defects. Usually, pointing defects is harshly discouraged, yet the argument has come up time and time again.

What is the Point of Points?

For starters, I think we can all agree that story points are a metric. A metric, by definition, is used to measure something. A user story is a description of a feature that provides value to a customer. So then story points must be a measure of the value delivered to the customer, right? WRONG!

Points are a measure of the cost to implement a given user story. Undeniably, the time it takes to implement a feature costs time, and this cost is very important to the planning of a project. The cost of a given story is very useful when determining whether or not a given feature is worth implementing. Over a period of time, points allow us to estimate project completion, since again, they are a measure of time. My coworker Micah covers this topic very well in this post.

So, when all points are implemented, the product is ready to ship, right? Wrong again!

Why? Defects!

What are Defects?

Defects are inevitable. No matter how hard we try, defects still happen. What’s worse is that they slow us down. Why? Because they take time. That’s right, time.

No matter how you look at it, spending time on defects decreases the time available for implementing features. Furthermore, the time to fix a given defect can vary.

Luckily, we usually make a decent guess at how hard it will be to track down the cause and develop a fix for a defect. Therefore, we should be able to assign some scalar to a defect to represent the cost, or time, necessary to fix it.

How about…points!

Mike Cohn explains this very well:

My usual recommendation is to assign points to bug fixing the agile defects. This really achieves the best of both worlds. We are able to see how much work the team is really able to accomplish, but also able to look at the historical data and see how much went into the bug-fixing story each sprint.

That’s right. We can’t analyze something if we don’t measure it. Also, not giving points to defects is very much like brushing them under the rug.

Now What?

Well, now we can have a cost associated with each of our defects, just as we do with our user stories. And now that we use the same scale, we can weigh defects and stories against each other. Better yet, we can use the sum of the cost of both to much better estimate when we will be done.

Mike’s example shows how a product owner can assess the effects of not only fixing bugs, but not fixing them in order to get features completed:

Knowing this [the cost of bugs] can be helpful to a team and its product owner. For example, imagine a situation in which the product owner is considering suspending bug fixing for the next six sprints in order to add a valuable different feature into a release. If we know that the team’s full historical average velocity was 25 but that 5 points went to bug fixing each sprint, we know that suspending bug fixing for the next six sprints will result in 30 (6*5) more points of new functionality.

But My Gantt Chart Just Exploded!

Well, you have likely just taken a huge step closer to reality. Better to find this out sooner rather than later. Defects, just like stories, take time! Being armed with how much each story and defect will cost will help you decide how to best spend your time.

As much as we would like to get away from the constraints of cost and time to market, a business needs to deliver solid products that provide value, and time to market is driven by competition and trade show dates. Pointing both stories and defects gives the stakeholders extremely valuable information on what all the pieces are and how much they will cost, so that tough decisions can be made. A product is judged not only by its features, but by its performance, robustness and user experience.

Conversation
  • Thoughtful post! This topic has been hot in the agile community from day one. Defects in particular are troublesome, and various authorities differ. I usually agree with Mike Cohn but not on this subject.

    As an example of the special problems with defects, I have heard it argued that you can’t know in advance how many points to give a defect until you’ve found the root cause, and finding the cause is usually the bulk of the work. I’ve also heard of successful teams assigning all defects one story point, to accommodate that uncertainty.

    I tried to resolve it by asking Jim Shore (another authority, jamesshore.com) about it in a private email. Here is his response. The first sentence is key…

    “Points and velocity are a prediction mechanism, not a productivity measure. I think this is where many teams get tripped up. They want to “take credit” for everything, so they look better from a productivity perspective, and in so doing, they lose the ability to make predictions.

    So no, I don’t ‘point’ unpredicted overhead [BL: i.e. defects]. (…) This allows me to make more useful predictions.”

    I’ve found that only in what I would call a high trust organizational environment are teams willing to follow his advice. In less favorable circumstances, teams seem to fear being judged (many times rightly so) and want to get “credit”… in other words velocity devolves to a productivity measure – the less valuable alternative. In those situations, improving trust usually resolves their concerns. How to do that is another conversation

    The situation is less clear when there are contractual relationships. Although those can be very high trust, the nature of a contract tends to drive a team towards wanting to get “credit”, because that is perceived as the best proof that money is being spent well. And that perception comes because, in hybrid teams (employees + consultants), the contracts are typically governed as if they were pay-for-effort rather than for-results. That is true regardless of what the contract language actually says, unless project financial governance is unusually enlightened.

    Teams whose velocity fluctuates widely (as more and less technical debt is tackled) do not inspire confidence among the less enlightened governors. And we’re all familiar with the result: prodding the team or its leaders to “increase velocity”. When velocity = productivity, that uncomfortable conversation is less likely to occur. But it becomes harder to predict delivery from a product backlog (that by nature identifies no unexpected work).

    For teams where either trust issues or contractual relationships are involved, this question will always be challenging and controversial. And the answer will depend on what factor is deemed most important to the business… justifying cost or predicting delivery.

  • Comments are closed.