One of the many tests that I often do is a consistency test. It’s something that’s really hard to automate, and testing for it can unearth different sorts of problems, six of which I’ll detail in this post.
1. Consistency Between Screens
An app can evolve over time as new screens and forms are added. Do these new screens use the same terminology, layout, and colors as the existing ones?
2. Consistency Between App and Portal
It’s common for an app to also have a portal. Is the terminology consistent between them? Is the handling of data consistent–does the app allow values that the portal does not (and vice-versa)? Are there fields in the app that don’t appear in the portal (and vice-versa)?
3. Consistency Between App and Documentation
Are there are any user manuals, online help, or marketing materials? If so, do their descriptions and screen shots match the app? Often they are written only once and fail to get updated if any changes are made to the app.
4. Consistency with Standards, Laws, and Regulations
Does your app have to conform to any standards? Are there are any laws it has to follow? Are there any regulations that it has to meet?
5. Consistency with Older Versions
Can the latest version of the app handle file formats from previous versions? Has the terminology changed? Is the workflow different? Is button placement different? If there are differences, were these intentional?
6. Consistency with Environment
Is the app consistent with other apps or equipment in the same environment? If it is different, is this a problem or a feature?
A Worthy Process
As well as checking for any consistency errors, going through the app like this can also unearth other problems and give you ideas for more tests. It’s an exercise well worth doing.
Are there other types of problems you’ve discovered through consistency checks?
Looking for more ways to test? Read some of the other posts in this series:
- 9 Situations to Test the Strength of Your Mobile App
- Learning from Fairy Tales
- Bugs Don’t Take Vacations
- Adding Software Tools to My Toolbelt
- Travel Bugs
- Testing Error Conditions
- Using Tea/Coffee Breaks in your Mobile Testing
- Alternating Phone Models Per Test Cycle
- Peripherals? I’d Forgotten about Those…
- A Tester’s Consistency Checklist
- “Alarming” Problems You Should be Preventing
- Quick Tests for Your Web App
- Testing for App Consistency
- Is it a Good Story?
- Quick Attacks on CRUD Apps
- Lights, Camera, Action, Bugs!
- Follow the Data
These consistency issues I think are checks that should be done diligently. However, the developers and content creators et al ought to be engaged with consistency as part of the evolution. For instance, each story will have tasks associated with it during iteration planning. Included here should be a “scan for consistency” task when it is anticipated that a change or addition may have an impact. At a minimum this should evoke some up front thinking and in the long run improve the user experience. I know I always enjoy apps that are evolving and where I see change, there is new or updated help information.