Tooling for an Iterative Infrastructure

Iteration is valuable to more than just application development; design also greatly benefits. But the benefits don’t stop there — infrastructure can gain as well.

A big portion of the value in an iterative approach comes from the empirical evidence gained through the experience of using the software. By exposing your product to the light of reality, all of your theories are put to the test. This feedback loop ensures that each cycle of work stays tightly focused on the things that really matter — even when they change and especially when they turn out to be things you might not have anticipated.

The way this happens is by shipping a complete product as early as possible. It might not be refined or have every feature implemented, but the core functionality should be present. As soon as this is ready, it should be delivered to the customer (and ideally, the end user). This is where operations comes in.

Oftentimes deployment is left as an after-thought, a last step to check off a list when a project wraps up. This is a mistake. Just as teams take design considerations into account along the way, they should also consider how they will deploy the application.

This doesn’t mean that a team needs to know exactly how their application will be deployed in production for the first iteration, but the tooling to support a production deployment should be put in place as soon as possible. Then, as more is learned about the production environment in each iteration, it can be incorporated into the deployment configuration.

One pattern that that makes a lot of sense for web applications is to use a combination of Puppet or Chef, Vagrant, and Capistrano.

Puppet and Chef

The benefits of using configuration management tools like Puppet and Chef are clear to anyone who’s had to manage a collection of servers over an extended period of time. Documenting server configurations and preventing drift are a couple of the most obvious advantages, but Puppet and Chef also provide a ton of value when it comes to setting up new servers or migrating from one environment (say, AWS) to another (say, your clients’ own internal servers). Which one you use is mostly a matter of preference, but after working with both, Chef seems a little more accessible and faster to set up (though I suspect that Puppet’s declarative DSL may have some long-term benefits for larger or longer-running projects).


Vagrant is extremely popular now, but I still find that I have to make a case for it at times. Many developers working on Macs are used to OS X being UNIX-y enough to port their apps to Linux servers without too much hassle. They often only spin up VMs to work in Microsoft Windows and generally use VMWare Fusion or Parallels for this, not VirtualBox. Why install another virtualization platform and set up another VM?

Automation and Phoenix Servers

Setting up a new VM isn’t all that painful when you have tools like Puppet and Chef (and Vagrant) to automate the process for you. In fact once you have these things in place, you can completely destroy your VM and recreate it with very little effort. This leads to the adoption of a hygienic practice of destroying and rebuilding your environment from code, a pattern Martin Fowler has called the Phoenix Server.


By capturing everything essential about your VM’s configuration in code (Puppet/Chef, Vagrant config), you’ve also made this environment easily shareable to the entire team (and beyond). Not only is everyone on the project working with an identical environment (saving countless headaches of “works on my machine”), but it also creates a description of exactly what will be needed in your production environment. Making your configuration manifests and scripts portable across a Vagrant VM and one or more production servers is often only a matter of minor refactoring.

Known Base State

One of the other things that makes Vagrant so useful is the fact that Vagrant VMs are typically created from an existing “base box.” These base boxes are VM images that serve as reusable templates or jumping-off points for individual projects. Generally minimal installs of the most common Linux distros, they are often no more than a few hundred megabytes in size so they don’t take long to download or share, but since they’re reusable, if you have several projects working with the same base box, you’ll only need to install it once. By using generic base boxes, you start with a known state and can share knowledge about that particular platform with others without worrying that you may have made different choices when you first installed the OS because the only differences between your running VM and the common base box will be captured in code. I like the base boxes that Opscode provides along with the Bento project (for creating custom base boxes).


Despite its warts, Capistrano really is a great tool for deploying Ruby on Rails web applications. Capistrano can also serve as an easy way to organize code to be executed either locally or remotely in conjunction with either server configuration tasks or application deployment (separate, but related concerns). Much like Rake within a Ruby app, Capistrano can serve as the glue to tie all of your operational code together.


These tools make it easy to create a minimum viable product for your first iteration that includes not only application code, but code for managing your infrastructure requirements and deployment needs as well. There is no longer any reason to put off operational concerns until your project is over. Even the smallest web app should be able to deploy to a Vagrant VM with its first iteration. Otherwise, you’re just incrementing.