I recently found myself making updates to an application that had a bad system test suite. It wasn’t bad because it was incomplete. It wasn’t bad because it was unreliable. It wasn’t slow, or ugly any of those things – it was just brittle. If one simple rule was followed when the suite was written originally, it would have been much better.
That rule is: Never say “click”.
In fact, in your cucumber features you probably shouldn’t say “follow”, or “press”. Don’t ever type “check”, “uncheck”, “highlight”, “drag” or anything like that either. The problem with writing such very specific steps is that they have a tendency to fail for the wrong reasons. A good test will fail because the feature is broken. A bad test will fail for other reasons.
Writing general tests
Here’s a concrete example:
Suppose I want to test logging in to an application. I could write:
Scenario: Viewing a report Given that I am on the home page And I follow "Reports..." And I select "Time report" from "Report type" And I select "Trailing 12 months" from "Report range" And I press "Generate" Then I should see "Time report for 2010/06 - 2011/05" And I should see time ....
Tests like the above are equivalent to telling someone how to spell your name by describing the movements of their pen. It’ll break as soon as they decide to type it.
Maintaining tests like this is a nightmare because if I change the “reports” link to be a drop-down menu, or perhaps I change the “Report type” select field to be a text field with completion, then the test will break. While we’d prefer our tests don’t break for these sorts of changes, that isn’t always possible. However, we can at least write a test suite where a code update will require a minimal amount of test updates.
Here’s a better test for the above:
Scenario: Viewing a report Given that I am on the home page When I view the "Time report" for the range "Trailing 12 months" Then I should see "Time report for 2010/06 - 2011/05" And I should see time ....
These steps are 1) shorter, 2) closer to the business domain, and 3) robust in the face of workflow changes. If the workflow for generating a report in this fictional application changes, then we will only need to update one step definition.
If you always write your tests in your domain language and avoid terms like “click”, “link”, “follow”, “select” and the like, you will be well on your way to a maintainable test suite.
Edit: A coworker just pointed me towards this post, also discussing good cucumber test practices. Interestingly enough, Brandon is arguing precisely the opposite side that I am. He cites two reasons for using the explicit “press”, “follow”, etc style:
- It communicates exactly how the feature works to an end user.
- It makes use of very reusable steps.
Personally, I don’t feel like the first is valuable. My tests are there to validate code, and not to be a user’s manual.
The second is interesting, as it is the same motivation I felt. It doesn’t make much of a difference if it’s a step that’ll only be used once, but for any piece of important functionality which either has multiple test cases or is a part of many workflows, I think using custom steps is worthwhile for the reasons I cite above. An extreme example of this benefit is writing steps for logging in to the application: if I change the form, I don’t want to go back and edit every single test I have.Which side of the fence do you all fall into? Leave a comment and tell us about your experiences.