How I Test CSS

At Atomic, we practice Test-Driven Development for all the code we can, from single functions to entire stacks. But there’s one kind of code we’ve long neglected testing: CSS. We rarely have coverage of it, and we often discover bugs and style regressions long after they were introduced. We’re not alone. Most software developers don’t do CSS testing. It’s tough to assert that a website looks the way you want.

I’ve wrestled with the problem of CSS testing for a long time, and I have two suggestions for catching style problems early.

1. Use a Living Style Guide

When developing a new UI component, I do it outside the application, in a living style guide. In a style guide, I can see many different states of a component at once. Rather than manually exercising the different ways a component can look, I can see and compare their various styles side-by-side.

Besides being a much better way to work on the code, that style guide sticks around and can be reviewed at any time. It provides a single place where a developer or designer can check the app for style problems.

When I find a bug in the CSS, I add a scenario in the style guide to reproduce it. Once it’s fixed, the style guide looks right. Then any time there’s a CSS regression, the style guide will show it.

2. Fuzz Test the UI

Fuzz testing the UI is another good approach to CSS testing. To fuzz test a program, simply give it unexpected or invalid inputs and see how it behaves. Fuzz testing the UI might simulate random user inputs to forms or click random places on the screen, but there’s another, hidden source of input in a web app: the data that comes from the server.

Lately, I’ve been working on a data dashboard that displays complex information in a variety of layouts. We didn’t have access to the data provider and real-world information, so we created some simple placeholder data to work with instead. It worked well for getting the app up and running, but the example data was hand-coded and remarkably uniform.

To expose potential layout problems, I fuzz tested it, replacing the app’s API with a simple Clojure server that returned JSON the way our app expected. Instead of returning hand-coded values, I used Clojure’s test.check generators to create random data. Feeding this non-uniform data into our app quickly showed where our styles were breaking down. In some places, long content created misalignments. Elsewhere, short or missing content revealed that certain widgets weren’t appearing full-width the way we meant them to be.

A few quick fixes later, the app looked much better in a broader variety of situations. Without fuzz testing, we wouldn’t have discovered these style problems until after the app was launched.

Conclusion

As helpful as living style guides and fuzz testing are, however, they still require manual inspection. I’d be happy to hear of more automated ways of testing CSS. If you have any more tips, let me know!

Conversation
  • hinok says:

    Eric, do you have any plans for incorporating visual regression testing to your workflow? I’m thinking everyday about some kind of automatic tests for styleguide which holds components, but creating that kind of system is a bit overwhelming at the beginning. From the other side simple styleguide with different states for each component is much easier to do, but again all work has to be done manually.

    I really liked the concept of recreating bugs in styleguide and then fixing them ;)

    • Eric Shull says:

      I’d definitely like more automation in my workflow, and I’ve done a little with visual regression testing. Unfortunately, as you say, it’s much more complicated: you have to keep a set of screenshots to test against, you have to specify when the new screenshots should be considered canonical, you need a server that always supplies the same data, and you have to filter out visual changes that aren’t regressions. And even after all that work, visual regression testing still overlooks some key considerations, such as how widgets look when hovered or focused.

      I’d love a better solution. If you figure one out, let me know!

  • Bernd says:

    my company had a good experience with

    https://garris.github.io/BackstopJS/

  • Vlad says:

    I’m wondering if you have considered or using something like https://github.com/jamesshore/quixote? 8-)

    • Eric Shull says:

      Very interesting. That looks useful. Have you used it? Are you able to make meaningful assertions of the style?

  • Comments are closed.