An Introduction to Property-Based Testing with JavaScript

Property-based testing is a powerful technique that’s been widely and successfully applied to functional-style codebases for a long time. As functional programming continues to get more and more popular among JavaScript developers, the value of this style of testing is becoming more obvious to a wider audience. Read more on An Introduction to Property-Based Testing with JavaScript…

Evaluating Property-Based Testing Through a Random Walk

Lately, I’ve been interested in property-based testing. It’s a sort of “Monte Carlo”-esque approach where you execute your application randomly (rather than according to strict scripts) and test that it never reaches an invalid state.

It has proven usefulness in lower-level software (such as implementations of data structures), but I’ve been wondering if it could be applied at a higher level. I’ve been wanting to apply it to a web app to test its domain objects and possibly also run it at a higher level, such as through a REST API.
Read more on Evaluating Property-Based Testing Through a Random Walk…

Fuzz Testing with afl-fuzz (American Fuzzy Lop)

Last year’s wave of major network security vulnerabilities has kept adversarial testing on my mind. Security auditing tools can discover bugs which are missed during more general testing. In particular, my interest was piqued by American fuzzy lop, a fuzzer released by the Google security team, and I’ve been waiting for the right project to try it out. Read more on Fuzz Testing with afl-fuzz (American Fuzzy Lop)…

QuickCheck in Ruby

Scott recently posted about the theft property-based testing library in C. Theft is very similar to Haskell’s QuickCheck. Theft tends to have a little less magic than Quick Check for generating random input and for failure case reduction. Theft makes them more manual, but also gives you better control of how they work. The theft ruby gem is a direct port to allow the same kind of testing in Ruby. Let’s take a look at the value in property-based testing and walk through a contrived example in Ruby.

Testing complicated algorithms can be, well, complicated. Typically, a developer will try to reason about their code and come up with a representative set of test cases to exercise the normal flow of the algorithm as well as all of the edge cases. This approach can leave holes in test coverage and brittle tests to maintain. That leads us to property-based testing. Basically, generate valid, randomly generated input and validate that a particular property is true.

Scott’s post does a good job of showing a real world example on the heatshrink compression library. I’ll use a contrived example to show how the theft gem can be used in Ruby. Read more on QuickCheck in Ruby…

theft: Property-Based Testing for C

Recently, I discovered a bug in my heatshrink data compression library, a result of a hidden assumption — I expected that input to uncompress would be padded with ‘0’ bits (like its compressed output), but if given trailing ‘1’ bits, it could get stuck: it detected that processing was incomplete, but polling for more output made no further progress.

Read more on theft: Property-Based Testing for C…

Property-Based Testing – Testing Assumptions You Don’t Know You’re Making

Finding good test input can be tricky. Even with loads of unit tests, bugs still get through.

Consider a wear-leveling algorithm for flash memory — it takes a series of write operations and spreads them over the flash, because individual blocks can only be written and erased so many times before they wear out. If any write sequence leads to writes concentrating in specific blocks, something isn’t working.

Read more on Property-Based Testing – Testing Assumptions You Don’t Know You’re Making…