Property-based testing is a powerful technique that’s been widely and successfully applied to functional-style codebases for a long time. As functional programming continues to get more and more popular among JavaScript developers, the value of this style of testing is becoming more obvious to a wider audience.
There are several popular libraries that bring property-based testing to JavaScript. The one I will be using for the examples in this post is JSVerify.
What’s Different About Property-Based Testing
Property-based testing isn’t a replacement for any of the other types of testing you might know. It just lets us test things in a way that’s impractical with other styles of testing.
With a traditional unit test, you supply a fixed set of inputs and assert that a function call returns a fixed value for those inputs. Each time you run the test suite, it uses the same inputs and the same outputs.
With property-based testing, you first define a property. Then the testing library will generate many random inputs and verify that your property holds for all of them. If you aren’t used to thinking about your functions in terms of properties, then it might not be clear what this actually means.
Identifying a Property to Test
If you think back to when you took math, functions were often defined in terms of properties. A function that describes addition over a field is required to be associative and commutative. A function that’s invertible must be surjective (onto) and injective (one-to-one). These are all good candidates for a property-based test.
Another simple property that you might want to test is idempotency. That means for a function f, f(f(x)) = f(x) for all values of x. One example of an idempotent function is the absolute value function.
If you were writing a unit test for this, you might pick some examples that seem exhaustive: a large negative, a small negative, zero, a small positive, and a large positive. You might stop there, or you might also write some tests to make sure it doesn’t blow up with any of JavaScript’s numeric constants: Infinity, -Infinity, NaN. It starts to get pretty verbose, and there’s the mental overhead associated with making sure you’re covering all the right edge cases.
What’s Wrong with a Bunch of Examples?
Even after going through all that trouble, you aren’t getting as much as you think you might be out of an exhaustive list of examples. Why? Because someone can always replace the function under test with a switch statement that does no computation, but merely returns the expected value for each set of expected inputs without breaking the test.
That’s not something that’s likely to happen in real life, but here’s something that might. Say you solve a problem with a naive and slow but correct algorithm in a low-traffic area of your application where performance isn’t critical. Over time, the low-traffic area gains traffic until your algorithm becomes a bottlenenck.
Along comes a well-intentioned developer with a knack for optimization who is tasked with fixing your slow algorithm. After a little bit of research, the developer comes across a blazingly fast approach that will only work on a restricted domain. The function is then changed to check for and handle special cases at the top, restricting the domain. Once those have been ruled out, the general code runs.
This is a very common pattern with highly optimized code. But if the well-intentioned developer forgets to handle one of the special cases, then your function will no longer be correct. Unless the developer adds a spec for that case, the tests won’t catch it, either.
Property-Based Testing to the Rescue
With a property-based test, you are able to express the idea that a function should satisfy some property. The testing library will automatically handle generating random examples and running them through your function.
Since it automatically finds the edge cases, it will make your tests less verbose and easier to write. When a test does fail, the library will go a little bit further to try to find the minimal failing example. This makes it easier to track down the problem without getting bogged down with a bunch of potentially unimportant details.
Have you tried property-based testing? I’d like to hear about your experiences.