Article summary
It happened again the other day: A team member was using a mocking library for unit testing and setting up the mocked members they knew they needed. The test didn’t fail, exactly, but it was behaving in a very strange way.
When they asked me for help, I had a feeling about what might be going on. Indeed, they hadn’t mocked every method that could be needed. But the mocking library wasn’t helping them figure that out.
Nearly every mocking library I’ve used has made the conscious design decision that you should be able to call mocks in an unexpected way.
Many are wired to return null or some conveniently-shaped object that just barely satisfies the function signature instead of complaining. Others do nothing at all.
Null Pointers Tell No Tales
I initially ran into this problem five years ago, during my first project as an Atom. We were using Mockito to do mocking in Java. On the surface, it looked easy:
Foo mockedFoo = mock(Foo.class);
mockedFoo.bar("baz");
verify(mockedFoo).bar("baz");The problem with this approach is that the type checker won’t let you call methods that don’t exist on the mocked class. So, while the mocking library will let you verify the interactions, the type check may not be enough to satisfy code that’s using the mock.
Here’s a simplification of a real-world example in which this can present a problem. Let’s say we are mocking a network connection creator. It might look something like this:
NetConnector mockedConnector = mock(NetConnector.class);
NetConnection mockedConnection = mock(NetConnection.class);
when(mockedConnector.connect("server.example.com"))
.thenReturn(mockedConnection);This test setup works so well for connecting to server.example.com that we can put it in our shared test setup. It may be far from where we typically see it as we’re writing tests, but it works well, right up until someone writes a test that connects to server.exmaple.com.
When the code under test calls connect, it won’t get the mocked NetConnection object, but null instead. Maybe you’ll get lucky and find this out right away—especially if the connection is immediately used when the null pointer exception is thrown. But maybe you won’t.
When I dealt with this situation, it wasn’t this simple. The failure was far away from the connect call, and we were left scratching our heads. Why were we dealing with a null connection object in a part of the code that we didn’t expect to be failing?
This unexpected behavior usually means that we need to hunt down the problem with a debugger—even if we know that it’s likely a missing expectation. A mocking library that makes the design assumption that it should try to fake out things we haven’t explicitly specified will make us hunt for the thing we’re missing, instead of telling us what we’ve missed.
Strictly Speaking
The good news is that today, many mocking frameworks support some kind of “strict” behavior where undefined behavior blows up at the source, instead of in some undefined spot later in the code.
In this case, Mockito 3’s new default STRICT_STUBS behavior (optional in 2.3) will help you catch some of these problems right at the source. You should definitely be using these.
As for the problem I mentioned at the start of this post, we were using Moq, which thankfully had MockBehavior.Strict.
With strict checking in operation, we can get back to scaffolding useful tests that don’t trip over peripheral issues such as mocks that don’t quite have every last expectation set up. Our tools will tell us exactly what’s wrong. Then we can fix it quickly and get back to work.