One of the many tests that I often do is a consistency test. It’s something that’s really hard to automate, and testing for it can unearth different sorts of problems, six of which I’ll detail in this post.
1. Consistency Between Screens
An app can evolve over time as new screens and forms are added. Do these new screens use the same terminology, layout, and colors as the existing ones?
2. Consistency Between App and Portal
It’s common for an app to also have a portal. Is the terminology consistent between them? Is the handling of data consistent–does the app allow values that the portal does not (and vice-versa)? Are there fields in the app that don’t appear in the portal (and vice-versa)?
3. Consistency Between App and Documentation
Are there are any user manuals, online help, or marketing materials? If so, do their descriptions and screen shots match the app? Often they are written only once and fail to get updated if any changes are made to the app.
4. Consistency with Standards, Laws, and Regulations
Does your app have to conform to any standards? Are there are any laws it has to follow? Are there any regulations that it has to meet?
5. Consistency with Older Versions
Can the latest version of the app handle file formats from previous versions? Has the terminology changed? Is the workflow different? Is button placement different? If there are differences, were these intentional?
6. Consistency with Environment
Is the app consistent with other apps or equipment in the same environment? If it is different, is this a problem or a feature?
A Worthy Process
As well as checking for any consistency errors, going through the app like this can also unearth other problems and give you ideas for more tests. It’s an exercise well worth doing.
Are there other types of problems you’ve discovered through consistency checks?
Looking for more ways to test? Read some of the other posts in this series:
- Lights, Camera, Action, Bugs!
- Follow the Data
- Quick Attacks on CRUD Apps
- Is it a Good Story?
- Testing for App Consistency
- Quick Tests for Your Web App
- “Alarming” Problems You Should be Preventing
- A Tester’s Consistency Checklist
- Peripherals? I’d Forgotten about Those…
- Alternating Phone Models Per Test Cycle
- Using Tea/Coffee Breaks in your Mobile Testing
- Testing Error Conditions