Exploratory Testing: Looking for Patterns

When I took my first steps into learning about testing, I joined the online test community and heard about The Dice Game. A couple of years later, I went to my first test conference and was able to join in the infamous game. In essence, the game is simple. Someone rolls dice, and the person in charge of the game announces a number. The task is to work out what the formula is to make the number. You can roll all the dice again, just one, change the dice on the table — anything to work out the pattern.

It can even be totally unrelated to what the dice are showing. Instead, it could be down to how many seconds past the minute the last roll or touch of the dice was.

Looking for patterns, especially non-obvious ones, has been a useful skill I still use a lot. Here are a few examples.

Crashes for Mobile Users When Opening the App

It was only happening for iOS users but not all of them. The ones it happened to were on the latest iOS release, but not all users on that release were experiencing the crash. The first report was in September, then more were reported in October.

Aha! Could it be that users were upgrading their iOS version, and so new permissions from the latest iOS were not being applied properly? Yes, it was. The date was the non-obvious influence, with September being the clue.

Window Width

There seemed to be a display problem that was only happening to iPad users. However, nothing in our code should have made things behave differently. Then a couple of desktop users reported the same problem. What could an iPad and a desktop have in common? Window size of course. The display problem was only happening in landscape mode for iPad, while some desktop users had resized their windows, and the size was one we had missed.

Input Combinations

An app we were building allows writers to create an online book. A few books had a bug where the dialogue would show but not the character. As it was more than one book, I could compare their inputs to others with no issue. Then, I was able to zero in on a specific combination of elements that caused the problem.

Knowing what the pattern was, I was able to develop a workaround to break the pattern until we could fix the code. That way, I was able to let the writers and editors know what pattern to look out for before the book was published.

User Data

Some users were experiencing an issue accessing part of the site, while other users were fine. What was the difference?

The users with the issue were heavy users of the system and had a lot of data for the system to fetch. A timeout threshold was set fairly low, and these users were getting caught with it. We increased the timeout, and we wrote a small enhancement to catch the timeout error and make obvious what had happened.

Being able to spot patterns has proved invaluable, so go grab yourself some dice and practice with your co-workers. Or try a few games of SET.

 
Conversation

Join the conversation

Your email address will not be published. Required fields are marked *