I recently gave a talk to the new graduates who had joined Atomic as part of our Accelerator program. I told them about my background, how I ended up Atomic and what I do here as an exploratory tester. During Q&A time, I got some good questions that I continued to think about afterward.
What’s your favorite browser?
For my normal browsing, I don’t really have any preference — Chrome or Safari is fine. However, when it comes to testing, it used to be Internet Explorer 6 (IE6), since that was where I’d find problems with website applications. When IE6 became history, IE10 was the browser I would try out first.
Nowadays, browser compatibility is not the big issue that it used to be, and my preferred torture device is now an old low-spec Android phone. Testing should be risk based, and you should be testing the places at the highest risk. The risk of something not working on a particular browser is now being replaced by Bluetooth problems as the app on your Android tries to connect to the device you want to communicate with.
Are there any evergreen tests that always find problems?
Finding ways to bypass any validation of inputs can still be fruitful. Most Atoms are now familiar with my trick of simply entering a space to get around the “field must not be empty” check. But adding leading/trailing spaces can still turn up the occasional issue with validation — especially when there’s a check for duplicate entries.
Speaking of duplicates, mixing the case up is a way to get around that, as is editing an existing entry to match another one, as is testing that the edit validation matches the validation when creating one.
Then there’s double-clicking of buttons, going back along paths already taken and taking a new one, lots of “quick attacks” I can do on any app without having any domain knowledge, and things that I do whenever I start to learn how an app works.
This was a really great question as it also made me think:
- Should these evergreen techniques still work?
- Am I getting stale and testing the same way, or am I learning new ways?
What testing problems do you see in the future?
Will we ever get to where testing is not needed — to zero-defect code? After being a tester for fifteen years, I suspect not. I do think, though, that one part of my role is to make myself redundant by mentoring and coaching the developers so they know and understand what I do. Already some of them have “The Phil Test” for their project, which could either be a rapid click of entries, putting spaces everywhere, or going off to make a cup of tea to test session timeouts.
In a more general view, testing is moving both earlier and later in the process. It’s happening earlier by testing stories and assumptions as they are being written and talked about before a line of code has been written. And it’s happening later in the process by monitoring logs and analytics to find out how customers are actually using the product (since are often use it in ways you never thought of).
In addition, apps are now using “The Cloud” and AWS Lambdas and Docker and Terraform and many other third party services, so there’s a whole area of configuration issues and services being unavailable that I am starting to learn about so I can understand where and how they might go wrong.
What did you learn about testing at college?
This was actually my question back to them, and the general answer was “very little,” which has got me thinking about whether I should be giving them a quick lesson or two on how I learned and what I’ve learned.
So if you have new hires, talk to them. Explaining what you do, how you do it, why you do it, and how you learned to do it can be useful to them and to yourself.