Using Game Titles to Estimate Testing Scope

At Atomic, we have multiple projects happening at once, with team sizes ranging from a solo maker to teams of six and up. The teams are also working on different technologies (web, mobile, IOT devices) with different project risks—it might be a quick and dirty MVP, an app for a high-profile event used by tens of thousands, or a complete rewrite of a legacy system.

When projects differ so much in size and scope, how can I estimate the way I help out with my testing expertise and experience? I sat down with one of the managing partners at Atomic to analyze past projects. Our goal was to detect any patterns and figure out how we could use the data to help teams estimate how much time to schedule for me.

We came up with a number of different sized “buckets.” To help explain the concept to the teams, I based the buckets on different video games.

Pong

This really simple game corresponds to a one- to two-person project working on a single environment. I go in and help catch any edge or use cases they might have missed from their own testing, and I will need a few hours a week.

Pac-Man

A more complex game, Pac-Man corresponds to a two- to four-person team working on more complex environments, e.g. web and mobile. There’s more functionality to be tested across more devices, so more hours are needed.

Super Mario World

The games are now getting more complex. There are different characters and interactions, worlds to explore, and equipment to track. This game corresponds to a large team working on multiple devices and data sources. There’s plenty of testing to be done across devices, data types, and use cases.

Doom

Run around and shoot everything! Sometimes, we take over a project and have to quickly find out how well or badly it performs. In these cases, I’m brought in to help test by going through the paths and options in an intensive blitz to get an overall picture of the health of the system. This is usually a one- to two-week session with 10-20 hours of testing.

It’s Alive!

This bucket didn’t fall into a game category. Once a project has gone live, a bug might be found and the team might need help reproducing or narrowing down the problem. I’ll be brought in to help for a few hours.

We’ve found that this concept helps the project leads get a rough idea of how much testing a project might need. There’s more detailed discussion on what testing deliverables are required and what types of testing (accessibility, performance, etc.) will be used.

Thinking of the different “buckets” as games also helps to establish a framework and time estimate. The project lead has an idea of the testing hours and scope for the project, and I can work out my schedule more accurately, having a rough idea of how many hours each project is going to need.