I always feel a tension between the idea of building more foundation code upfront and waiting until it’s needed. If we spend all our time building framework and foundation, then we’ll never get around to actually building the product. But of course, if we plow ahead without planning, we can end up with an unmaintainable mess.
I’d like to share some of the heuristics I use to help make these decisions on my products.
The Problem
On the one hand, we have YAGNI, which tells us we don’t know the future well enough to get the right foundation built. Therefore, we should just build what we need today.
This is a very appealing and valuable philosophy. Code is only valuable if it brings value to the user, so don’t spend time building frameworks that may (at some foggy future date) be useful. Instead, just build that framework later — when you need it.
But honestly, I’ve never been able to do that very well.
The problem is that “when you need it” isn’t especially concrete. I can always justify the “fact” that when I need it will be tomorrow, not today. We just need a little bit more right now…
But now, looking back on my projects with 20/20 hindsight, I can see a pattern and infer a heuristic that will help with this sort of decision.
Sidebar: Iterating vs. Incrementing
But before I get there, let me introduce some background. There’s an article by Jeff Patton that I’ve been forwarding to people for better than 10 years. Read the whole thing, it’s that good! But let me highlight his insight that “iterating and incrementing are separate ideas.”
The best way to explain this is to present the example Jeff provided (under a creative commons license.) Here’s his illustration of incrementing:
In this mode, we build each piece one after another, and we finish each piece completely. It’s simply chunking work. This works well when you have a clear vision of what you want to build.
And here’s iterating:
We start small and cheap, expecting to change and throw away code. As we iterate further, the picture gets clearer, and we get closer to what we want.
A Foundation Heuristic
Here’s my heuristic for when to invest in foundation code. When you move from iterating (to discover the shape of the solution) to incrementing (to implement a now-known solution), that is the time to invest.
I like this heuristic because in practice it becomes much more concrete when you ask questions like this:
- Is this story the first effort in a new area of functionality where we don’t have a clear vision of how it works? This is the YAGNI case. You clearly can’t understand the needs well enough to develop a robust foundation.
- Is this an expansion of newly-created are of functionality? One that will take it from bare demonstration closer to being a complete workflow? In that case, as you expand your initial implementation, you should now have a much clearer view of what the needs are, and you are now well-positioned to build all the foundation pieces you need.
I find it’s usually quite easy to take an upcoming user story and identify which of those buckets it should be in. Most of the time, we start our project with a backlog of a few large epic stories, so the first step is to iterate enough on each to understand the remaining work. And after that’s completed, we incrementally work through the now-identified chunks.
At the Project Level
This is also the philosophy with which I attack a project release.
We start with some idea of the shape of the solution to our problem. We may try a few technical spikes to understand what we can and can’t accomplish. We get some prototypes in front of users (anything ranging from paper prototypes to a technical spike that only covers the happy-day cases.)
Soon after that, we start to implement a real version of the features — usually not a deep, feature-complete implementation, but rather a first cut to de-risk all the technology. Now we have an app that doesn’t do much, but it does a little bit from each core area of functionality.
That’s all iterating, and typically it’s the first third of a release cycle. I’ve found that this piece of a project usually can’t sustain very many developers without them stepping on each other’s toes. Because everything is entirely new, there’s much more churn at this stage. For a while, anyway.
Then you shift into an incremental mode. You start to pick up speed (since you’ve now accumulated all that domain knowledge by iterating earlier). And you add onto the existing foundation with new functionality or handle for new edge cases.
Then, as you approach your release date, the project tends to move towards iterating again: small refinements and polish.
The Important Thing Is to Know
Both modes of operating are necessary! It’s perfectly okay to plow forward, screaming, “YAGNI!” as you first integrate some external service. As long as you know you’re in that mode.
The important thing is to be conscious of which mode you’re in. If you find yourself iterating on many unknowns while you’re late in your release cycle, that’s a red flag — you’re in trouble. And on the other hand, if you’re working incrementally very early on, you should ask yourself if you really understand the problem space as well as you think you do.
The world is messy, and these things are rarely clear. Hopefully, having this additional lens through which to look at your project will help provide new insights.