Phone simulators are great for helping development move quickly, but they are only simulators. Testing on a real physical device should be part of your workflow.
On my desk, I have a large selection of different Android and iPhone models covering a range of screen sizes and OS versions. I can quickly test out the new functionality on each one.
Of course, running a regression test on all of these every week would be time-consuming and wasteful–the developers have automated tests running, so there is high confidence in the app.
Nevertheless, occasional errors can creep in, so we need to test OS upgrades to see if they cause problems. I may also get new ideas about what to test after gaining a deeper understanding of a particular app or testing other apps. Having a more in-depth test session allows me to test out all of these ideas to see if there are any issues.
To cover all the possibilities, my approach is to alternate the phones on which I do in-depth testing. The cycle might look something like this:
- Week 1: Run in-depth tests on iPhone6+ and Android Pixel
- Week 2: Run in-depth tests on iPhoneSE and Nexus 5
- Week 3: Run in-depth tests on Galaxy S5 and Note 5 as an Android OS update came out
- Week 4: Run in-depth tests on iPhoneX and Galaxy J5
This approach is also useful for testing the app’s ability to switch accounts. Eventually, users get new phones and move their apps (and accounts and data) onto these devices. By using different phones every week, we can test out this functionality.
What is your strategy for dealing with all the variety and combinations of phones? Let me know in the comments.
Looking for more ways to test? Read some of the other posts in this series:
- 9 Situations to Test the Strength of Your Mobile App
- Learning from Fairy Tales
- Bugs Don’t Take Vacations
- Adding Software Tools to My Toolbelt
- Travel Bugs
- Testing Error Conditions
- Using Tea/Coffee Breaks in your Mobile Testing
- Alternating Phone Models Per Test Cycle
- Peripherals? I’d Forgotten about Those…
- A Tester’s Consistency Checklist
- “Alarming” Problems You Should be Preventing
- Quick Tests for Your Web App
- Testing for App Consistency
- Is it a Good Story?
- Quick Attacks on CRUD Apps
- Lights, Camera, Action, Bugs!
- Follow the Data