Convenient Trick for Tolerance-based Test Assertions Using #ish

Sometimes it’s hard to just say what you mean.

When writing automated system and integration tests, I occasionally find it tricky to express the precise expected value of test output. For example, I expect the timestamp on a database record to be within a second or two of “now,” but I can’t predict the exact millisecond it will be created by the web server I’m posting to. And sometimes a floating point result is off by a millionth or trillionth, because the numeric representations used in my tests are not necessarily the binary equivalent of what the code under test is using.

In the cases where I know the differences are circumstantial (and irrelevant to my test), I find myself wanting to write…

assert_equal record.created_at, Time.now.ish
assert_equal bullet.velocity, 5.25.ish

…and have it “just work.” I know what I mean, and so (likely) will anyone reading the test after me.

I know, I know… most test tools already provide some sort of tolerance-based assertion helpers or value matchers. (MiniTest has #assert_in_delta and RSpec has #be_close.) But they can damage readability and obscure intent, and sometimes they simply don’t fit the situation.

In the following example, I want to compare a Hash of attribute values for my space ship after ticking the game engine one quantum into the future:

@game_clock.tick
@space_ship.attrs.should == { x: 120, y: 0, rotation: 0 }

This looks simple enough, but if I happen to be employing a physics simulator, my results may be off by a very minuscule amount, and I won’t be able to predict that difference. Worse, I can’t employ #be_close because it won’t know what I mean by feeding it a Hash. Better:

@space_ship.attrs.should == { x: 120.ish, y: 0.ish, rotation: 0.ish }

Here’s the code that enables #ish for Numeric values:

class Numeric
  def ish(acceptable_delta=0.001)
    ApproximateValue.new self, acceptable_delta
  end
end
class ApproximateValue
  def initialize(me, acceptable_delta)
    @me = me
    @acceptable_delta = acceptable_delta
  end

  def ==(other)
    (other - @me).abs < @acceptable_delta
  end

  def to_s
    "within #{@acceptable_delta} of #{@me}"
  end
end

Here's how you can do something similar with Time:

class Time
  def ish
    ApproxTime.new(self)
  end
end
class ApproxTime
  def initialize(t); @time = t; end
  def ==(o)
    return false if @time.nil? or o.nil?
    (@time - o).abs < 5
  end
end

Note that it's easy to provide for optional tolerance tuning (eg, to be a little looser, you could do something like 3.14.ish(0.01)).

I generally call this function #ish because it's short, unlikely to collide with any other method names, and it's fun to say "five-ish" than "assert in delta five comma oh-point-oh-one. It also implies a semantic dependence on the object we're invoking the #ish method on. It's easy to deduce that 9.ish could mean something quite different from (3.days.from_now).ish.

I tend to implement the #ish helper on a per-project basis, in order to localize my assumptions for reasonable default tolerances. Is plus-or-minus 5 seconds a reasonable tolerance for all people on all projects? No. But for any given project, it's usually easy to just pick something and move on.

If monkey-patching Ruby's built-in value types gives you heartburn, you might instead implement a helper function #about or roughly that accepts any object and determines, internally, how best to inject tolerance into equality tests.

Anyway, I just wanted to share another of our sneaky little testing tricks. Anybody interested in seeing this become a Rubygem? Better yet, have you got any of your own tricks you'd like to share with us?