2 Comments

Bye-Bye, Sinon – Hello, testdouble

UPDATE: Justin Searls, the author of testdouble.js sent me an email with some notes on this post. I’ve added some his comments below to provide some additional context.

I’ve been working in JavaScript-land for the last little while, writing lots of Node.js code. Since I practice TDD, I’m always trying to keep my eye on the best new ways to test JavaScript code.

In unit testing, mock objects are a key tool for isolating your tests to single components. For a long time, the status quo for JavaScript test mocking has been Sinon.JS. It is ubiquitous, full-featured, and easy to use. It is not perfect, however. The API is confusing enough that even after using it for around three years, I still routinely look up how to do common things in the documentation. It also does not support some workflows, such as injecting entire mock objects into tests, very well.

I recently attended a talk presenting a new alternative mocking library for JavaScript: testdouble.js. I really liked the API and design decisions presented in the talk, so I have started integrating testdouble.js into my current project (which is already using Sinon.JS). After a few weeks of using both tools side-by-side, I thought I would share my thoughts and observations.

testdouble.js is OO, while sinon.js is function-based

The first thing that struck me about testdouble.js is that it was clearly designed to fit into an object-oriented JavaScript codebase. The td.object call makes it trivial to generate entire mock objects from a constructor or object literal. In contrast, Sinon.JS is entirely focused on individual functions, and it requires you to mock out each function on an object individually.

Justin Searls: This isn’t inaccurate, but just a heads up that it wasn’t my intention to give the impression td.js favors objecty units over functiony units, one reason you might think so is because the object stuff is (inherently) more complex and feature-laden and so if you were to document both features, the td.object API takes 3 times as much ink to cover. FWIW, the fast majority of modules I write w/ td.js export a single non-prototypal function.

One consequence of this is that testdouble.js explicitly doesn’t support partial mocks. This is an interesting contrast to Sinon.JS, which only supports partially mocking objects. In general this isn’t a problem, and I agree with the justifications for avoiding them. However, sometimes you need them, particularly when adding unit tests to a codebase that didn’t have them before (and where the refactoring necessary to avoid them isn’t feasible in the short term). When I’ve run into these situations in my current codebase, I’ve just continued to use Sinon.JS, but I’d like to find a better long-term solution.

Justin Searls: For what it’s worth, while we don’t go out of our way to support faking singular functions on objects, you still totally can using the td.replace() API. If you have an object named `dog` with 5 functions on it, you can replace one named `woof()` with `td.replace(dog, ‘woof’)`, which works just as well as Sinon for the same task. Moreover, and better than Sinon, it’ll automatically restore itself on the global `td.reset()` you should have in an afterEach helper, whereas you’d have to manually restore it (unless using the separately kloogy Sinon sandbox feature).

Our docs on that API are here

testdouble.js has a nicer API

This is one area where testdouble.js wins hands-down. Sinon.JS has an API that builds up mocks, spies, and stubs via method chaining. For example:

sinon.stub(myObject, 'myMethod').withArgs("a", 3).returns("aaa")

On the other hand, testdouble.js has a much more natural API where you simply call mocked functions just like you expect them to be in your test:

td.when(myObject.myMethod("a", 3)).thenReturn("aaa");

testdouble.js can inject dependencies via require

When writing tests for Node.js, testdouble.js also provides the td.replace API. This lets you inject test doubles as dependencies directly through the Node.js API. I personally prefer to use constructors to inject dependencies manually in JavaScript–a method which eliminates the need for this most of the time. However, it is a great tool for working with code that wasn’t designed that way, allowing you to inject mock objects without having to drastically refactor your system.

Justin Searls: Just a personal style comment here, not a critique, because I applaud you for sticking this out, but I’ve given up on manual dep injection in my Node.js for no other reason other than how counter-idiomatic it is compared to most Node code. It made sense for a long time in Java because of how common bean-constructing containers were, but I’d encourage you to give simple replace/require a try more often on your next project. The symmetry is really nice, and if it’s an internal API, the extra step of manual injection rarely buys me anything (but for the sense that I was future-proofing).

The other great thing is that since the td.object API accepts constructors, testdouble.js also makes it much easier to do my preferred constructor dependency injection. I simply wrap all of my dependencies in a td.object call and pass them in as complete mock objects that I can control.

I have, unfortunately, had some weird dependency resolution issues when using the td.replace API. While I have not root-caused it yet, it generally seems to come up when using instanceof calls in my tests. I have been able to work around it by moving the troublesome import to before my first call to td.replace.

Justin Searls: Damn, I totally missed this one when I designed the feature, and I’m not sure how to improve it. If you look at the implementation of the constructor wrapper, it’s its own class and won’t match the real constructor: github.com.

I have not run into this b/c I tend not to do a lot of `instanceof` checks on internal code, so I don’t have a good answer for this but I’m sorry you ran into it and I am open to feedback about how we can maybe use prototype chaining to pass a check without making a huge mess on the original object.

I followed up on this on GitHub

testdouble.js can’t replace some Sinon.JS things.

In particular, Sinon.JS has some incredibly useful utilities for mocking out async timers and AJAX calls, allowing you to test those things totally syncronously. testdouble.js has no replacement. Fortunately, there is absolutly no issue with using just those parts of Sinon.JS alongside testdouble.js. The Sinon.JS maintainers have even been nice enough to split just those two pieces into their own separate modules which you can get without the rest of the library.

Justin Searls:
@cjno started Sinon in another era before small modules were a popular notion, but I think from a cohesion perspective, both of these things would be better served as separate concerns.

First, on clocks, in unit tests I always wrap stuff I don’t own, so I’d much more likely have a dependency called `dueDateFor(thing)` that returned a fixed date as opposed to trying to fake Date itself. (See: “don’t mock what you don’t own” from GOOS). If someone really needs to freeze or fix the clock, then I’d strongly urge them to only do so in an integration (e.g. no mocking) test and for clarity sake to use a tool focused for that task like Timecop

Second, on AJAX, I really find Sinon’s fake server stuff to also be at a bad level of abstraction; In my opinion isolated unit tests shouldn’t need to fake out the host environment directly, but rather wrap their use and fake the wrapper (or, if it’s sufficiently broken down, just test the functionality without any test doubles at all). For this purpose, I much prefer modules like supertest, pretender, covet, because they’re focused and fit in well with integration test suites.

The tl;dr on the above is clocks & XHRs are integration concerns and by featuring them in a test double library, you’re encouraging users to leak integration concerns into isolated unit tests. I _only_ use test doubles in isolated unit tests of how my subject code should invoke subordinate code that I’m designing, and I take pains to keep them out of integration-concerned tests because it’s incredibly hard to keep track of what is real, what is fake, and what the value of the test is when the line is blurry

Conclusion

Sinon.JS is great and has been the status quo for a long time for a reason. However, testdouble.js feels more polished and natural, and it fits way better into an OO design and workflow. Going forward, I plan to use testdouble.js as much as possible.