At Atomic Object, we like to keep things simple. For Justin and I, this often means using Puppet Standalone or Chef Solo to configure servers that don’t yet (and may never) require the coordination of a Puppet Master or Chef Server. Justin recently blogged about the pattern we use with Chef Solo. I’m going to share a new tool we’ve started using to keep these repositories simple, even as the requirements grow.
LEGO My Eggo
As a server configuration repository grows beyond a single monolithic recipe, we refactor it to use several cookbooks like Lego building blocks. Some of these are “weird blocks” specific to the individual needs of a particular server, but some of them are “base blocks” serving a more generic role.
Oftentimes these base blocks have even been implemented already by Opscode or by others in the Chef community. If we re-use the same base blocks over and over for multiple servers, it doesn’t make sense to manually copy those files into each repo every time. There must be a better way.
Ermahgerd! Berkshelf!
Ermahgerd! There is! Cookbooks are Chef’s analog to Ruby’s gems, and Berkshelf lets you treat your cookberks — err… cookbooks the way you treat gems in a modern Ruby project. Just put all of your fravrit — ahem, favorite cookbooks in your Berksfile, add gem "berkshelf" to your Gemfile, bundle install, berks install! It gives me gersberms.
Base Blocks on Your Berkshelf
Seriously though, Berkshelf really does make managing cookbooks easy. Once you’ve installed the berkshelf gem (via your Gemfile with its counterpart, bundler, or with a simple gem install berkshelf), you can run berks init to generate an empty Berksfile (and .gitignore, Gemfile, and Vagrantfile) to get you started.
Adding cookbooks to your Berksfile follows the same pattern as gems in a Gemfile: e.g. cookbook "database", "1.3.6". Whereas with bundler your Gemfile probably has source :rubygems at the top, your Berksfile will have source :opscode.
When you run berks install, Berkshelf installs the cookbooks specified in your Berksfile to your “Berkshelf” usually in ~/.berkshelf/cookbooks/. This works well if you’re working with a Chef Server, but if you’re using Chef Solo the way we do (tarring up all of your server config and scp-ing it to the server using Capistrano), you’ll be happy to know that Berkshelf, like bundler, has an option to “vendorize” the cookbooks in your Berksfile. Simply run berks install --path chef/cookbooks or wherever you’d like to put them.
This works great for Opscode-provided cookbooks, but what about other cookbooks you find on github? No problem — just provide a path to the git repository and Berkshelf will take care of things.
cookbook "mysql", git: "https://github.com/opscode-cookbooks/mysql.git"
You can even specify a branch or tag:
cookbook "mysql", git: "https://github.com/opscode-cookbooks/mysql.git", branch: "1.0.1"
You could also use the new shorthand for repositories hosted on github:
cookbook "artifact", github: "RiotGames/artifact-cookbook", ref: "0.9.8"
Weird Blocks on Your Berkshelf?
All of this makes it really easy to get those base blocks in place. But what about our weird blocks? Do they just get mixed in with all the other ones in chef/cookbooks that Berkshelf is managing for us?
We could leave them in chef/cookbooks, but if we move them somewhere else (say, chef/custom_cookbooks) we can .gitignore chef/cookbooks. We can have Berkshelf copy our weird blocks in by specifying them with a path in our Berksfile like this: cookbook 'radar-console', path: 'chef/custom_cookbooks'.
Adding Berkshelf to Our Chef Solo Pattern
I’ve forked Justin’s repo and added Berkshelf so you can see what it looks like in action.
One thing you’ll notice is that not every cookbook is listed in the Berksfile. That’s because Berkshelf will resolve dependencies specified in a cookbook’s metadata.rb file.
I also added a berks:install task to Capistrano and made sure that it gets executed as the first step of the chef task so that our “vendorized” cookbooks will be copied to the server we’re provisioning. You can see the diff for that here.
Finally, you’ll notice a fair amount of churn in Gemfile.lock. I had to run a bundle update to get Berkshelf to install. It happens to be one of the gems trying to juggle some rather complex dependencies which include the notorious json gem. Be aware that you may also have to bundle update or manually juggle a few dependencies in order to add Berkshelf to your project.