Article summary
One of the biggest challenges with writing quality end-to-end (E2E) tests in Cypress is avoiding flaky tests. This means tests that fail unexpectedly, in spite of accurate code. Of course, you can’t completely avoid all flakiness in true E2E tests. When you’re relying on actual API calls and data, there’s bound to be times when your tests fail, and it has nothing to do with the quality of your code. However, there are tips and best practices you can utilize to reduce the brittle-ness of your Cypress E2E tests. Here are a few “Do” and “Don’t” rules to guide you through writing (more) stable E2E tests in Cypress.
Cypress E2E Tests: Dos and Don’ts
Don’t use CSS attributes such as class or id in your Cypress selectors.
CSS class and id attributes from production code are non-specific and prone to development changes that will cause your E2E tests to fail. As an alternative, use data-* attributes in your HTML elements and query objects with them using your Cypress selectors. Furthermore, Cypress will prioritize elements in the DOM with data-* attributes over basic class and id attributes. Here’s an example in the Cypress documentation that explains how to select based on the data-cy attribute.
Don’t use cy.wait() with a raw amount of time to make your tests pass.
In my experience, the urge to do this is often a symptom of a larger problem. The times I’ve been tempted to use cy.wait(2000) were times when I thought my test was just flaky when in reality it was exposing a bug in our workflow we hadn’t noticed before, or when I was frustrated with aliased API calls that weren’t behaving properly (another sign to step back and look more closely at the failing test).
Don’t write your tests so that their outcomes are dependent on each other.
A test should not require the previous test to run in order to have the proper set up. For example, if I have a test that creates a new user for an application and a test that deletes a user, those shouldn’t refer to the creation of the same user. If something breaks the creation test, then your deletion test will also fail. Instead, set up the creation of a user in a beforeEach hook and then perform tests on it.
Do mock when you must.
Mocking is a balancing act. Cypress makes mocking API calls very easy with cy.intercept(), but mocking out API calls does stray from the true nature of E2E testing. However, it is sometimes the lesser of two evils for consistently unreliable API calls that are creating flakiness in your tests.
Do use Cypress ‘retries’.
I’ve found that for particularly flaky endpoints in my E2E tests, sometimes running them again is the only thing that gets my tests to pass. Using the retries property in your Cypress configuration file enables your test suite to retry failed tests automatically any number of times. With that said, I usually find that a flaky test only fails once. When the same test fails multiple times consecutively, I find that is indicative of an actually failing test. A ‘retries’ value of 1 or 2 should suffice in most cases.
Do use the { timeout: x } parameter in your Cypress calls like cy.get() or cy.wait() – when waiting on previously defined aliases.
The default wait time that Cypress allots for finding an element in the DOM or hooking onto an API call that’s been made is only 4 seconds. For some applications, this simply isn’t enough time. Modify the timeout option to give your application more time. Note: timeout is specified in milliseconds.
Do use a specific E2E user for authentication & permissions in your tests.
Typical users (such a dev or demo environment user) utilized by developers can have permissions and other aspects changed frequently, which can break your tests. It’s best to isolate your E2E testing as much as you can from everyday environments. Having a specific user with specific authentication set up is one way to do this.
Do consider adding cleanup in your beforeEach hooks.
This might be a bit controversial, but it’s a practice that has worked well for my team’s specific use case. While it’s good practice to include final cleanup steps in afterEach or afterAll hooks – especially if your tests create data that could interfere with other developers or subsequent tests. I’ve also found it helpful to do cleanup at the beginning of each test. This is because in Cypress, if something fails in a beforeEach hook, the entire describe block is skipped, and any afterEach hooks won’t run.
This means your intended cleanup might never happen, leaving behind a state that causes future test runs to flake or fail. To prevent this, I start each test by ensuring a clean slate. For tests that create an object (from the user’s perspective), I use custom Cypress commands to check for the object’s existence and delete it if it’s already there. This proactive cleanup has made my tests significantly more reliable and less flaky.
Cypress E2E Tests That Aren’t Flaky
Writing reliable E2E tests in Cypress requires strategy and thoughtfulness. Certainly, every project has its own peculiarities, and no testing strategy fits every use case. However, I hope these tips will help you write Cypress tests that are more maintainable, less flaky, and less frustrating to debug.
Acknowledgements
In addition to my own experiences with writing Cypress E2E tests, some of the tips in this post were inspired/influenced by a talk given by Alain Chautard. Some of his ideas are summarized in this post. If you are in search of more resources to learn about Cypress, I recommend checking his work out.