“Who Tested That?” Probably Someone Like You

I’m sure many of you have come across errors like the one above when you are surfing the internet or using an app. Your first reaction might be, “Who tested that?” You know your site and app would never have an error like this in production!

Or would it? Here are a few reasons why this bug might have made into it production.

1. Found But Not Fixed

The bug was found during testing, but someone decided not to fix it. This could have been due to time constraints, technical difficulty, or a use case that was too narrow to be worth fixing.

2. Found But Badly Reported

It was discovered during testing, but the report of the bug never got the attention of the team. Maybe they had so much work in progress that the report got lost in the noise. Or the report was not well-worded so it was ignored. Or the steps to reproduce it were not detailed enough so they could not be replicated by the person responsible for the fix.

3. Environment Issue

The issue might not be due to a problem in the test environment. It could be that some difference between test and production made the issue appear.

4. A Very Edgy Edge Case

You just happened to be using some combination of data and environment that testing had missed. Maybe you were the first person to try pasting 800 cat emojis into a message. Or maybe you were using the app on the day of a switch to Daylight Savings Time.

 

There are a few reasons why a bug might be “missed.” That leads us to the next problem: If you had a missed bug, would you know about it?

In the example error I used at the top, there was no way for the user to report the bug. It’s also very unlikely for users to report a bug unless they really, really, REALLY have to use the app and need the problem resolved.

You might have logs that would catch the error, but are you monitoring them to see if there are problems?

Next time you find a bug in the wild, instead of being smug and thinking it would never happen to you, ask yourself if it could.