This style is great for testing because it clearly establishes intent for each test. This point is invaluable when another developer either inherits your code, or you revisit it at some point in the future. The main problem with this style is the need to create stubs for all the calls in the describe's before block, or resort to using stub_everything for all your mock objects. These are both fine solutions for simple code, but as soon as the logic encounters a condition you may be setting yourself up for potential testing bugs, and difficulty when you add new logic.

One way to avoid this problem is to leverage the power of rspec, and use nested describe blocks for each condition statement. This helps limit the before block for each describe to only define the stubs for that condition block. This is helpful because it prevents the tester from accidentally defining a stubbed expectation for something that should not happen. It also creates a nice placeholder for adding logic into the code in the future.

The following example shows how nesting rspec describes works. It is important to note that the branch that contains the most logic should be encapsulated in the sub-describe.

Example Source Code:

1
2
3
4
5
6
7
8
9
10
11
12
13

class Example 
  def run(director, runner)
    runner.prepare
    if director.go?
      runner.run
      if runner.complete?
        director.stop
      end
      runner.stop
    end
    runner.finish
  end
end

Example Test Code with nesting describes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

describe Example do
  before do
    @target = Example.new
  end
  
  describe '#run' do
    before do
      @director = mock
      @runner = mock
      
      @runner.stubs(:prepare)
      @director.stubs(:go?).returns false
      @runner.stubs(:finish)
    end
    
    it "should prepare the runner" do
      @runner.expects(:prepare)
      @target.run @director, @runner
    end
    
    it "should check the director" do
      @director.expects(:go?).returns false
      @target.run @director, @runner
    end
    
    describe 'director returns true for go' do
      before do
        @director.stubs(:go?).returns true # redefine
        @runner.stubs(:run) 
        @runner.stubs(:complete?).returns false
        @runner.stubs(:stop)
      end
      
      it "should run the runner" do
        @runner.expects(:run)
        @target.run @director, @runner
      end
      
      it "should check if the runner is complete" do
        @runner.expects(:complete?).returns false
        @target.run @director, @runner
      end
      
      describe 'runner returns true for complete' do
        before do
          @runner.stubs(:complete?).returns true # redefine
        end
        
        it "should stop the director" do
          @director.expects(:stop)
          @target.run @director, @runner
        end
      end
      
      it "should stop the runner" do
        @runner.expects(:stop)
        @target.run @director, @runner
      end
    end
    
    it "should finish the runner" do
      @runner.expects(:finish)
      @target.run @director, @runner
    end
    
  end
end

I realize the test code gets a bit long, but it is very easy to understand and extend. Also, nesting on conditional breaks creates an easy framework to follow when writing test code.

Creating a perfectly functional web application isn't too helpful if you forget to renew the domain name, and DNS requests start going to a placeholder for-sale page. It doesn't matter to your customers if they were the ones responsible for managing registrar information. All they see is a broken website.

Any time you're working on a web project for a customer, you have many more things to track besides the software itself. You have accounts for web hosting, accounts for DNS, domain registration, SSL Certificates to keep renewed. All these things will have their own expiration dates, and they all have the potential to cause your customer's project to fail.

What we needed was a system that would track a variety of details about a project, specifically focusing on expiration dates.

In addition to just warning on expiration dates, the system does a few additional things:

• Automatically grabs domain expiration from whois records
• Automatically grabs SSL certificate expiration by connecting to the site and checking the actual certificate used.
• Checks that the SSL certificate is valid
• Checks that it can connect to a URL for the domain name and get a valid response.
• Iterates through all the nameservers listed in the whois record, and checks that those servers return an A record for the domain.

A cron job is run hourly to update all the expiration dates and server statuses. The results are displayed in our continuous integration server, so everyone is aware of the results.

The system is implemented using Rails, although Rails is really only being used as a means to provide an easy configuration editor and an object model to represent the data. Streamlined is being used to provide an interface for adding and deleting resources. Streamlined is an advanced scaffolding plugin for Rails.

Here are some screenshots and samples of output from the system:

This is the configuration screen for the application, to define the resource that should be tracked:

This is the continuous integration server status screen, showing that there is a problem with some of the customer's domains.

This is the error output from the system that is displayed by the CI system.

********************************************************************************
* Projects have expiring or broken resources!!!
*
* To update the resources connect to http://dci:7171/
*
* There should be a process running that automatically updates expiration dates
* by running rake expirables:update
********************************************************************************
Project   : Smithee
--------------------------------------------------------------------------------
Developer : AO
Customer  : Tom Tim
SCM URL   : 

Domain Name: xyzzy.com
  Expires: 2012-07-08
  Broken : YES
  Reason : Nameserver [NS1.T324.COM] did not provide an A record: 
  DNS result has no information for xyzzy.com
Nameserver [NS3.T324.COM] did not provide an A record: 
  DNS result has no information for xyzzy.com


Project   : Foo Bar
--------------------------------------------------------------------------------
Developer : AO
Customer  : Xyzzy Myxlplic
SCM URL   : 

SSL Certificate: app.gooblegabble.com
  Expires: 2009-07-16
  Broken : NO
  Reason : 

Project   : Anon site
--------------------------------------------------------------------------------
Developer : AO
Customer  : XYZZY MYXLPLIC
SCM URL   : 

Domain Name: XYZZY.org
  Expires: 2011-01-31
  Broken : YES
  Reason : Nameserver [NS2.DREAMHOST.COM] did not provide an A record: 
  DNS result has no information for XYZZY.org


rake aborted!
Test failures

Jeff Patton and Lane Halley ran an Agile UX workshop for Atomic Object recently. One part of the workshop was the practice of user and stakeholder research through interviewing. For the exercise, we used a speculative development project that we’ve been researching for six months. We interviewed our office manager, Mary O’Neill, as the basis for one of the main personas for the project.

I didn’t expect to learn much from this interview about the project itself. After all, I’ve been thinking about this project, researching the opportunity, discussing it with many people, pitching investors, collaborating with our partners, and generally immersing myself in it for the last six months. On top of that, our office manager is also my wife, so I live with the interviewee and know very well her habits, goals and objectives concerning the subject of the project. How much more could I learn from a 30 minute interview? Lots, it turned out.

Interviewing Mary to craft a persona representing a user of the application we’re building turned out to be a rich vein of insights, surprises, new ideas, concerns and hidden opportunities. This experience brought home the value of the pair interviewing technique Lane taught. It also furthered my resolve to help our clients understand the value of this research. Learning so much, with so little effort, on something I already knew so well – how can this not be part of every project we do?

Carl Erickson recently wrote about the rich life of stories at Atomic Object. In addition to our vanilla user stories, we also create “Epic” stories.

We use Epics as large placeholders for some number of user stories. Epics are written at a very high level, and do not contain much detail. We want to record a general understanding of what the customer is looking for, without having to dig into all of the details.

Epics will typically have very large estimates. One Epic could potentially be estimated to take many iterations to implement. By using Epics to put rough estimates on all of the customer’s desired functionality we can provide them with a better idea of the size of the project, without having to go into great detail in up-front requirements work.

When the customer prioritizes an Epic towards the top of the backlog, it is broken down into smaller user stories. We make no attempt to restrict that the total number of points from the resulting stories adds up to the previous estimate of the Epic. This allows more flexibility in granular story estimates and ultimately better accuracy in the final backlog.

Using Epics in this way at the beginning of a project we can quickly provide our customers with the information they need to budget, schedule, and prioritize. We defer spending the time necessary to gain a deeper understanding of the requested functionality until it is more likely that the functionality is actually going to be needed.

Over the last 18 months, Atomic has made significant growth in our product development services. We now kickoff most projects with an initial discovery and design phase. In this phase we grow our understanding of the client's domain and their vision of the software they want to build. We create artifacts like financial models, personas, wireframes, visual mockups, prototypes and release plans. Eventually, we create a backlog of estimated stories that the development team can start implementing in the development phase.

A few developers at Atomic have shifted into a UX role. We've been growing the company's awareness of UX and product development practices. Last Thursday and Friday Atomic held an in-house UX workshop led by Lane Halley and Jeff Patton. The workshop exercises focused on a speculative development opportunity Atomic has been considering.

Thursday morning started by Lane and Jeff asking what we wanted to get out of the the workshop. They built a backlog of objectives for the workshop and included what we wanted to learn.

Once the schedule was set, we started by talking to the project's stakeholders about their general ideas for the project. Lane and Jeff gave a sample stakeholder interview with one of the stakeholders. We were taught some interview techniques and had the opportunity to practice interviewing with partners. We had some interview candidates lined up for the afternoon so we put together an outline of interview questions.

After Thursday's lunch we slimmed down to a smaller group. This group refined and organized the interview outlines. We conducted four interviews as pairs while the rest of the group observed. We had a retrospective after each interview where we compared notes and critiqued the interviewers.

On Friday the whole company came back together and the interviewers shared their notes. The notes were taken on cards. The cards were laid out and grouped together based on observational similarity. We constructed provisional personas (obviously real personas would require more research) from our notes. Each persona had a name, attributes, objectives and values. We broke out into teams and constructed context scenarios for each persona.

After Friday's lunch, Jeff had our product stakeholders define their high level business goals. We discussed what would constitute success. The discussion gave us insight into what kind of metrics the software may need to provide.

Then Jeff asked how we define a user story. This was a rich discussion that gave us insight into how much we have internalized user stories and how the simple definition of a story has changed over time

After our user story discussion, we extracted activities from our context scenarios. Jeff led us through an exercise where we created a story map. We organized our activities and derived stories from the activities. We looked to the project stakeholders for story validation. We then engaged the stakeholders in release planning using the story map. The exercise showed a great way to prioritize high level stories before fully defining the details required of a finer grained story to be entered into a development backlog.

The workshop ended with a retrospective where the entire company asked questions and shared thoughts.

I'm very excited to see Atomic continue to grow in our UX and product development practices. We've stayed at the forefront of agile development and management for some time. While there, we saw that our clients needed help in formalizing what their project was, who it was for and what business value it would provide. That need and our drive for excellence has pushed us forward to better help our clients. We are enjoying the ability to provide the necessary up front services that ensure we are developing successful products.

Web entrepreneurs, at least the ones we’re experienced with helping, have limited budgets. They’re also inherently optimistic people, believe deeply in their idea, and want confidence that their development team will build something that handles the heavy traffic that success will bring. The answer they want to hear is usually something like, “we design scaleability in right from the start”. Our experience tells us there’s a better way.

Atomic Object has been organizing work around user stories since the very beginning of our company in 2001. Our understanding of stories, their uses, how we write and track them, customer involvement, and level and relationship to testing has evolved considerably over time. A recent experience watching Jeff Patton pull this out of Atomic Object developers made me realize how rich and powerful the story concept has become.

1. Identified
2. Named
3. Structure
4. Estimated
5. Prioritized
6. Selected
7. Disambiguated
8. Implemented
9. Accepted
10. Counted

Identified

Stories are born through a conversation with the customer, or an email, or a planning session. Stories are born generally bereft of detail. Some never make it through the full life cycle.

Named

Stories are given a short name so we can talk about and track them. This happens when they are entered into our iteration planning tool. Some of our customers enter stories directly themselves.

Structured

    As [a type of user]
    I want to [do something]
    So that [valuable achievement X happens]

Standardizing the language makes it easier for customers to write stories themselves, makes describing stories more efficient by reducing the range of the specification language, and ensures we get the context of the story.

Estimated

We estimate stories so customers can plan the development of their product, and so we have a way of tracking our velocity and projecting the completion date. When stories are given big estimates it often results in conversation between the customer and the developers to break a story apart, better understand complexity and dependencies, and consider alternatives.

We estimate stories in relative complexity points.

Prioritized

Customers place stories into the backlog. Our convention is to keep the backlog loosely prioritized with higher priority stories towards the top.

Selected

The customer selects stories from the backlog to create the next iteration. The customer knows how many stories to place in the iteration by matching the sum of the estimates of the selected stories to the average development velocity.

Disambiguated

Stories typically have plenty of ambiguity, subtlety, hidden crannies, inconsistencies and depth at this point. The structured language we use helps, but doesn’t clarify all aspects of a story—that’s not the intent. Rather than write lengthy requirements in English, or force our customers to write them, we apply the classic, XP definition of a story (“a placeholder for a future conversation with the customer”) and have that conversation. You can think of this as just-in-time requirement specification.

Disambiguating (or the better alternative of “understanding”, as Jeff good-naturedly pointed out) stories now, when you know you’re going to be working on them immediately, saves you from a lot of potential wasted work (stories that are never selected don’t need to have this work done), and makes sure the understanding is gained close in time to when it is needed.

The understanding we gain from the customer is partially captured in structured acceptance tests. These often, but not always, get turned into automated system tests. Whether automated or not, they serve as a means for the customer to know that the story has been understood by the developers, and whether or not it is completed and correct.

    Given [some context]
    When [the user does something]
    Then [the user sees a result]

Implemented

Stories are implemented following our test-driven development practice.

Accepted

At the close of the iteration, a new build of the tool is deployed for customers to evaluate. Customers can refer to the acceptance tests to know the story is complete and correct.

Counted

When a story is accepted by the customer, the developers get credit for having completed it. The number of points this story was estimated at are added to the burndown chart. The total number of points achieved at the close of the iteration contributes to the average development velocity.

If a story isn’t completed, or the customer doesn’t accept it, the developers don’t get credit for the story in this iteration. That in turn lowers the total points achieved, and possibly the average development velocity.

Ten distinct phases in the life cycle of a story? I was struck by the richness of this simple concept when Jeff pulled it out of us and put it on the wall. We’ve so deeply internalized the story as a means for organizing development that we don’t have it written out anywhere. Jeff helped me step back and think about the many phases a story goes through, and all the goodness this simple idea brings to our development process.

You can check it out on iTunes.

There were some interesting experiences with iPhone development that we'll cover in future blog post. One useful tip to know is that the UIImage class will try to manage its own memory if you construct it using the initWithContentsOfFile: method. This means it will unload and reload the image from the file as needed, which can kill performance if you're dealing with a lot of offscreen images that need to be shown on screen quickly. Using initWithData: using an NSData will let you manage the memory yourself.

Lane Halley at Cooper recently sent me an article from the Harvard Business Review on why small companies have an edge in this economy. The author argues that the employment situation is so uncertain in large companies that customers of those companies won’t be well-served. The invisibility of working for a large company means that your efforts on behalf of clients are easily missed. That in turn makes it easy for you to be laid off, even if you’re doing a stellar job. Big companies erode the trust of their employees, which in turn erodes the trust of their clients.

We occasionally interview someone who asks about the risks of working for a small company like Atomic. That’s usually a sign that this person is a “big company” type person, and not a good match for us. While security isn’t the only difference between small and big companies, I think in our case it actually cuts the other way. Atomic has never laid anyone off in our eight years of operations. By contrast, every single one of our customers that employs developers or engineers has had to do that at some point. These are great companies with successful products that employ smart people they care about. But they’ve all had layoffs.

We only employ 23 people, so it’s an apple/orange comparison with our large customers, but for those 23 people, it seems to me the security argument favors Atomic. (Knock on wood.)

Part of a series on Making better estimates. Estimates require making assumptions. Assumptions violated are like risks realized. Both can be accounted for with some estimate buffering.