Many times during the sales process we are asked to give a cost estimation based on our early understanding of an application’s core features. In order to responsibly and efficiently estimate, we conduct a 50/90 range analysis. We use this technique because we want to establish a responsible middle ground between our optimistic and pessimistic estimates. Our estimate is appropriately buffered for the risk we see in the defined tasks.

This spreadsheet is the tool that we use to conduct our analysis. Tasks can be estimated in any time duration that you choose (i.e. hours, days, weeks). At the start of a large project we generally estimate tasks in days or weeks.

AO devs Dustin Tinney and Drew Colthrop participated in Grand Rapids GiveCamp this weekend, a volunteer event in which developers lend their skills to area nonprofits. This year 125 volunteers from all over the Midwest came to the Downtown YMCA to work with 23 nonprofits. A combination of local and national businesses sponsored the event, including AO’s neighbors – Brick Road Pizza and Sandmanns.

Dustin worked with C-snip, a low-cost veterinary outfit specializing in spaying and neutering pets. C-snip currently receives thousands of calls a day from area pet owners seeking their services; each call can take up to fifteen minutes because of a detailed questionnaire. Dustin’s team created a web application that would streamline this questionnaire process and free up valuable resources for C-snip. You can see the page they created here.

Drew worked for Friends of Grand Rapids Parks, a grass-roots movement to preserve and enhance area parks and other public spaces. He added park-specific pages to their website that will feature photos from their Flickr pools and convenient Google maps of the parks.

In order to develop deliverable software, the volunteer teams needed to take on projects of the appropriate scope. For Drew and Dustin, their experience of working in short iterations and delivering the most essential features first came in handy. Dustin and Drew commented that the project management was very similar to what we do at AO: “We’re used to working on small stories, working closely with the clients, and receiving constant feedback.”

Some clients prefer vendors to offer fixed bids for their projects. The perceived benefit of a fixed bid is that it appears to remove uncertainty from the cost, timeline and scope of the project. The contract controls three of the four variables outlined in Kent Beck’s Extreme Programming Explained. Clients are happy to have control over these aspects of the project, unaware that in so doing they may be relinquishing control over the fourth variable: quality.

What does it take to produce a high-quality product? First, we find that the best products are the result of the collaborative efforts of clients and vendors – business and development – because you need the expertise of both parties. Second, a project needs to be able to adapt to the inevitable changes that it will encounter along the way. All features don’t have to be fully defined at the beginning of the project. Both clients and vendors are in a better position to work out the details of a feature or set of features as time moves on.

A fixed-bid contract works against these principles of quality. It does not easily adapt to changes because the contract determines the scope of the project and straying from that contract involves time-consuming renegotiation, which detracts from the working relationship of clients and vendors. Second, the contract does not align the incentives of clients and vendors. The focus is not on producing high-quality software, but on fulfilling the said contractual agreements. The development question is no longer “what will be best” but “what does the contract stipulate.”

The picture described here is one of competing interests on opposite sides of an inflexible document (i.e. wall) between clients and vendors that prevents effective communication. In the long run, software produced on this model is more costly, and we’re not just talking about cost of maintenance. Even adequately functioning products that fit the contractual requirements can have additional opportunity costs: a software product may miss its intended market, or an internal system may fail to address inevitable inefficiencies.

How AO Works

At Atomic Object, we bill for time and materials at regular intervals throughout the lifecycle of the project. But this doesn’t mean we simply dive in headfirst with no concern for budget. We are very budget sensitive, and recognize the long-term cost savings of owning well-built software. Our project estimate, while accurate, does adapt over time to fit the needs of the project. The following three practices work together to create an increasingly accurate budget and satisfying project.

• Prioritizing Features
• Tracking Velocity
• Working in Weekly Iterations

Prioritizing Features. While in the planning phase, we help our clients produce a list of all the features they desire of their product, ranking them in order of importance. The developers then use this list to structure development. (Clients can change the priority of any feature during iteration meetings.) Once established, this list can be used to estimate how long the project will take. Rather than assign an arbitrary amount of time to the feature, we measure the tasks according to their relative complexity. If adding a feature has a complexity level of one point, another that is twice as hard will be allotted two points in the schedule.

Velocity Tracking. Once each feature is estimated using a relative scale, we can make an iteration budget (e.g. ten points this iteration), and then measure our progress. If ten points takes us one week, we now can estimate that a one-hundred point project will take ten weeks. By tracking our velocity, we are able to produce an increasingly accurate estimate of the project’s completion time. Clients are part of this process—they know exactly what our velocity is, how it compares historically, and how it affects the future of their project.

Working in Weekly Iterations. We release tested, functional software to clients in weekly iterations, offering them a good sense of where the team is and assuring them that the released features match their expectations. The weekly iterations provide clients with a platform to change their minds without needing to renegotiate a contract. Rather than dropping off a contract and returning three months later for the finished product, clients can see the progress as it happens, mitigating the risk that the product isn’t what they had in mind. Moreover, these weekly checkpoints assure clients that their developers are using their time as efficiently as possible. Because clients collaborate with the developers on a weekly basis, the progress is completely transparent. Tracking our velocity and breaking up projects into smaller tasks incentivizes developers to work more thoroughly and efficiently, quelling any concerns that billing for time discourages developers to use their time wisely.

The advantage of billing for time and materials over fixed-bids is that it removes the wall that a contract creates, promoting communication and collaboration between clients and developers. Everyone shares the responsibility for the project’s success when communication is open—and that means higher quality software.

Carl Erickson recently wrote about the rich life of stories at Atomic Object. In addition to our vanilla user stories, we also create “Epic” stories.

We use Epics as large placeholders for some number of user stories. Epics are written at a very high level, and do not contain much detail. We want to record a general understanding of what the customer is looking for, without having to dig into all of the details.

Epics will typically have very large estimates. One Epic could potentially be estimated to take many iterations to implement. By using Epics to put rough estimates on all of the customer’s desired functionality we can provide them with a better idea of the size of the project, without having to go into great detail in up-front requirements work.

When the customer prioritizes an Epic towards the top of the backlog, it is broken down into smaller user stories. We make no attempt to restrict that the total number of points from the resulting stories adds up to the previous estimate of the Epic. This allows more flexibility in granular story estimates and ultimately better accuracy in the final backlog.

Using Epics in this way at the beginning of a project we can quickly provide our customers with the information they need to budget, schedule, and prioritize. We defer spending the time necessary to gain a deeper understanding of the requested functionality until it is more likely that the functionality is actually going to be needed.

Over the last 18 months, Atomic has made significant growth in our product development services. We now kickoff most projects with an initial discovery and design phase. In this phase we grow our understanding of the client's domain and their vision of the software they want to build. We create artifacts like financial models, personas, wireframes, visual mockups, prototypes and release plans. Eventually, we create a backlog of estimated stories that the development team can start implementing in the development phase.

A few developers at Atomic have shifted into a UX role. We've been growing the company's awareness of UX and product development practices. Last Thursday and Friday Atomic held an in-house UX workshop led by Lane Halley and Jeff Patton. The workshop exercises focused on a speculative development opportunity Atomic has been considering.

Thursday morning started by Lane and Jeff asking what we wanted to get out of the the workshop. They built a backlog of objectives for the workshop and included what we wanted to learn.

Once the schedule was set, we started by talking to the project's stakeholders about their general ideas for the project. Lane and Jeff gave a sample stakeholder interview with one of the stakeholders. We were taught some interview techniques and had the opportunity to practice interviewing with partners. We had some interview candidates lined up for the afternoon so we put together an outline of interview questions.

After Thursday's lunch we slimmed down to a smaller group. This group refined and organized the interview outlines. We conducted four interviews as pairs while the rest of the group observed. We had a retrospective after each interview where we compared notes and critiqued the interviewers.

On Friday the whole company came back together and the interviewers shared their notes. The notes were taken on cards. The cards were laid out and grouped together based on observational similarity. We constructed provisional personas (obviously real personas would require more research) from our notes. Each persona had a name, attributes, objectives and values. We broke out into teams and constructed context scenarios for each persona.

After Friday's lunch, Jeff had our product stakeholders define their high level business goals. We discussed what would constitute success. The discussion gave us insight into what kind of metrics the software may need to provide.

Then Jeff asked how we define a user story. This was a rich discussion that gave us insight into how much we have internalized user stories and how the simple definition of a story has changed over time

After our user story discussion, we extracted activities from our context scenarios. Jeff led us through an exercise where we created a story map. We organized our activities and derived stories from the activities. We looked to the project stakeholders for story validation. We then engaged the stakeholders in release planning using the story map. The exercise showed a great way to prioritize high level stories before fully defining the details required of a finer grained story to be entered into a development backlog.

The workshop ended with a retrospective where the entire company asked questions and shared thoughts.

I'm very excited to see Atomic continue to grow in our UX and product development practices. We've stayed at the forefront of agile development and management for some time. While there, we saw that our clients needed help in formalizing what their project was, who it was for and what business value it would provide. That need and our drive for excellence has pushed us forward to better help our clients. We are enjoying the ability to provide the necessary up front services that ensure we are developing successful products.

Atomic Object has been organizing work around user stories since the very beginning of our company in 2001. Our understanding of stories, their uses, how we write and track them, customer involvement, and level and relationship to testing has evolved considerably over time. A recent experience watching Jeff Patton pull this out of Atomic Object developers made me realize how rich and powerful the story concept has become.

Jeff talked about the life cycle of a story. As a group, we came up with the following phases of this life cycle:

1. Identified
2. Named
3. Structure
4. Estimated
5. Prioritized
6. Selected
7. Disambiguated
8. Implemented
9. Accepted
10. Counted

Identified

Stories are born through a conversation with the customer, or an email, or a planning session. Stories are born generally bereft of detail. Some never make it through the full life cycle.

Named

Stories are given a short name so we can talk about and track them. This happens when they are entered into our iteration planning tool. Some of our customers enter stories directly themselves.

Structured

On most projects we find it helpful to impose a simple structure on the description of a story. We use a template Dan North at Thoughtworks came up with:

    As [a type of user]
    I want to [do something]
    So that [valuable achievement X happens]

Standardizing the language makes it easier for customers to write stories themselves, makes describing stories more efficient by reducing the range of the specification language, and ensures we get the context of the story.

Estimated

We estimate stories so customers can plan the development of their product, and so we have a way of tracking our velocity and projecting the completion date. When stories are given big estimates it often results in conversation between the customer and the developers to break a story apart, better understand complexity and dependencies, and consider alternatives.

We estimate stories in relative complexity points.

Prioritized

Customers place stories into the backlog. Our convention is to keep the backlog loosely prioritized with higher priority stories towards the top.

Selected

The customer selects stories from the backlog to create the next iteration. The customer knows how many stories to place in the iteration by matching the sum of the estimates of the selected stories to the average development velocity.

Disambiguated

Stories typically have plenty of ambiguity, subtlety, hidden crannies, inconsistencies and depth at this point. The structured language we use helps, but doesn’t clarify all aspects of a story—that’s not the intent. Rather than write lengthy requirements in English, or force our customers to write them, we apply the classic, XP definition of a story (“a placeholder for a future conversation with the customer”) and have that conversation. You can think of this as just-in-time requirement specification.

Disambiguating (or the better alternative of “understanding”, as Jeff good-naturedly pointed out) stories now, when you know you’re going to be working on them immediately, saves you from a lot of potential wasted work (stories that are never selected don’t need to have this work done), and makes sure the understanding is gained close in time to when it is needed.

The understanding we gain from the customer is partially captured in structured acceptance tests. These often, but not always, get turned into automated system tests. Whether automated or not, they serve as a means for the customer to know that the story has been understood by the developers, and whether or not it is completed and correct.

We use the following structure for acceptance tests:

    Given [some context]
    When [the user does something]
    Then [the user sees a result]

Implemented

Stories are implemented following our test-driven development practice.

Accepted

At the close of the iteration, a new build of the tool is deployed for customers to evaluate. Customers can refer to the acceptance tests to know the story is complete and correct.

Counted

When a story is accepted by the customer, the developers get credit for having completed it. The number of points this story was estimated at are added to the burndown chart. The total number of points achieved at the close of the iteration contributes to the average development velocity.

If a story isn’t completed, or the customer doesn’t accept it, the developers don’t get credit for the story in this iteration. That in turn lowers the total points achieved, and possibly the average development velocity.

Ten distinct phases in the life cycle of a story? I was struck by the richness of this simple concept when Jeff pulled it out of us and put it on the wall. We’ve so deeply internalized the story as a means for organizing development that we don’t have it written out anywhere. Jeff helped me step back and think about the many phases a story goes through, and all the goodness this simple idea brings to our development process.

Part of a series on Making better estimates. Estimates require making assumptions. Assumptions violated are like risks realized. Both can be accounted for with some estimate buffering.

Read the rest of this entry

Part of a series on Making better estimates. Probably the simplest way of improving your project estimates is to explicitly de-couple duration from calendar. It’s easy to become so involved with estimating the duration of tasks that you forget to account for the actual work calendar.

Date vs duration

On a small time scale, estimating in relative complexity and measuring velocity helps with this de-coupling. If your company culture is such that you can expect to spend 10 hours per week in meetings, then your velocity will automatically reflect that – you don’t need to remember to use 30 work hours per week when making calendar projections from duration.

Even with points and velocity, you still need to keep your eye on the large time scale picture. If your estimate for a project indicates you need 30, one-week iterations to complete it, don’t forget to lay those 30 iterations against a calendar of vacations, holidays, conferences, plant shutdowns, or whatever else takes people away from work in order to predict a completion date.

---

Previous posting: Range estimates

Next posting: Assumptions and Risks

Part of a series on Making better estimates. Single point estimates don’t accurately represent the natural variation in a task. A range estimate of low and high is a good first step to improve accuracy. With a little definition and some simple math, range estimates can be used to much more accurately and responsibly estimate a project.

Read the rest of this entry

Atomic has strong opinions about how to do projects, and we have practices that are so deeply a part of our culture that they are taken for granted internally. There’s a common way we approach all the projects we do, whether it’s embedded, desktop or web, in any of the many domains in which we work. I’ve known this for a while, but I’ve never had a good word for it. Watching one of our younger developers on a recent project gave me that word.

People at Atomic have a strong projectstance.

Read the rest of this entry

Part of a series on Making better estimates. Estimation is frustrating: fuzzy, difficult, inexact. You can simplify the process, reduce the effort, and maybe even improve overall accuracy by trying to be less accurate at the detail level.

Read the rest of this entry

Part of a series on Making better estimates. Relative estimation in arbitrary units beats absolute estimates in actual time units.

Read the rest of this entry

Part of a series on Making better estimates. Experiment trumps speculation when it comes to improving estimation accuracy.

Read the rest of this entry

Part of a series on Making better estimates. Mining data from prior projects, especially if time was formally tracked, can be an invaluable source of data for estimation efforts.

Read the rest of this entry

Part of a series on Making better estimates. The team needs to own the estimate, so the team needs to make the estimate.

Read the rest of this entry