System tests are a crucial piece of testing any application. I’m a big believer in isolated unit tests as well, but if I had to pick just one kind of test to use, it would be system tests. There is no substitute for actually exercising the full code base from top to bottom.
That said, I don’t know if any one thing about software development has caused me more frustration than system tests. Read more on Automating Artifact Generation on Capybara(-Webkit) Failures with Autopsy…
While the process of writing software remains a challenging, sometimes frustrating endeavor, I was reminded recently how simple some tasks have become. The data available on the internet (along with the tools we have to manipulate that data) make answering some difficult questions a rather enjoyable undertaking.
Today’s post is the first part in a series on using these tools to solve a rather frivolous problem — getting a leg up on my competition in EVE. I’ll be using MySQL, ruby, RSpec, and ActiveRecord to read in the static data needed to answer my questions.
For the majority of you who have never heard of EVE, it’s an online game with a player-generated economy. That means when you buy a ship or some other item in the game, another player has mined the materials needed, researched the blueprints, and coordinated the manufacturing jobs to build that item and put it on the market. Some items are overbuilt, and manufacturing these items is a money-losing proposition. Others are in low supply and can turn a pretty profit.
Read more on Hacking EVE, Part 1 – Reading SDD Data…
There are plenty of articles that talk about how you should name your css, what the files should be called, etc. (Here are some at Snook.ca, Smashing Magazine, and phpied, for example.) The topic has been covered and for the most part is agreed upon.
A couple of weeks back, I was working on some RSpec tests that required a different setup and tear-down than other tests in the app. We had database cleaner providing cleanup for us to prevent test interaction, but certain tests we created needed a heavier cleanup than others. We had a simple
after(:each) with the setup and tear-down, but that had to change to accommodate certain types of tests.
Read more on Using the RSpec Around Hooks…
This is one of those Rubygems I wish I’d known about a long time ago: capybara-screenshot.
As the name suggests, when a capybara test fails, the gem will automatically take a screenshot of what the browser rendered. I don’t normally need this level of information, as the error is usually fairly obvious. But a handful of times now, the extra information has helped to work through those times where I’m scratching my head and can’t figure out why a test is failing.
I did change the stock behavior of the gem a bit — I don’t have it automatically take a screenshot. Since I don’t normally need that, I didn’t want it cluttering up my filesystem. Instead, I tag scenarios with a little piece of metadata when I want a screenshot. Here’s how I did that:
Read more on Easier Debugging with capybara-screenshot…
The title of this post says it all. I’m super excited that I now have RSpec configured to run my tests in random order. Test interactions are gone. Assumptions have been eliminated. The test suite is improved.
In early 2012, RSpec added the option to run tests in a random order. Last week (early Agust 2012) I had a good opportunity to try it out with my current project’s test suite. Random ordering revealed three flaws in the test suite, which I’ve since eliminated:
- Our poor usage of the rspec-set library.
- Our less-than-optimal usage for Fabrication’s sequence number feature.
- The assumptions in our system tests’ navigation helpers that are not always true.
Read more on RSpec: Thank You for Running My Tests in Random Order…
I recently spent some time cleaning up a large test suite that had fallen into a bit of disrepair. The project was a Ruby on Rails web application being tested with RSpec and Cucumber. The biggest problem was that various tests would fail intermittently. Some tests would pass consistently when run by themselves, but would frequently fail when the whole suite was run. Re-running the suite once or twice would often result in all tests passing, so the non-deterministic tests were largely ignored eventually becoming broken windows.
Read more on These Intermittent Test Failures Will Not Stand, Man…
Ever come back to your terminal after a 15-minute test run and see a lone failure you know would have passed if your tests had been just a little more, um, patient?
This happens to me all the time when I’m writing acceptance tests for web or desktop apps. If I were watching the app with my own eyes, I wouldn’t declare Total Failure simply because an in-page update took an extra few seconds to appear.
So, I gave my tests the gift of patience, letting me write code like this:
page.should have_content("Task Table")
page.should have_selector(".row:nth-child(1)", text: "Ride to work")
page.should have_selector(".row:nth-child(2)", text: "Write some code")
page.should have_selector(".row:nth-child(3)", text: "Go to The Meanwhile")
…even if every row in the table is being lazy-loaded, one at a time, from a server struggling under load.
I’ve added this capability to our FirePoll RubyGem (see GitHub for source and docs).
gem install fire_poll.
Read more on Be #patiently – FirePoll 1.2.0 Released…
Since this code was meant for a demo, it was originally written to be throw-away. Moreover, since it was leveraging a large codebase which did not have any establisted testing framework – or tests of any kind – I made the decision to just get the code working and forget about writing test suites. And then, of course, the situation changed. Suddenly the code needed to be solid enough to pass off to someone else and have them run with it. In my four months at Atomic Object, I’ve already absorbed enough of Atomic’s philosophy that the thought of my code persisting in someone else’s hands without any tests was cringe-inducing. Having done nearly all of my testing in RSpec (with Mocha) and a little Jasmine, I wasn’t optimistic that I could find something that easy for C++. I wanted something that would provide straightforward mocking capabilities so I could perform true unit tests.
Read more on Comparing googlemock to Mocha…
I’ve just released the minilab 2.0.0 Ruby gem.
By far, the biggest change in the 2.0 version is Ruby 1.9 support. Ruby 1.8 isn’t supported anymore. This is primarily because the version of the ffi gem I’m using only runs on 1.9. That said, the public interface of the
Minilab class hasn’t changed, so from a user’s point of view, there’s little difference in the library.
Read more on Minilab 2.0.0…