Standalone Puppet with Capistrano
Mike English and I frequently make use of Puppet to provision servers and manage server configurations. It is very convenient to be able to setup a server (while simultaneously documenting its configuration) by writing Puppet modules and manifests and then simply running the Puppet agent. For larger setups, we have used the Puppet agent/master model (client/server), but for smaller setups (such as for a single server), this doesn’t make much sense — especially if it is a standalone server for a client running privately in the cloud.
However, it feels rather “un-Puppet-like” to SSH into a server directly, make Puppet manifest changes, and then run the Puppet agent in order to make configuration changes to a server; it is not very efficient.
The Solution: Capistrano
We felt it would be more effective to have a single point from which we could modify manifests, and then run commands to make the necessary configuration changes on a server, while not actually needing to run a Puppet master.
To achieve this, we’ve started using Capistrano to bootstrap Puppet, upload manifests, and run the Puppet agent in standalone mode. For a while now, we’ve used Capistrano to deploy applications to servers. We’ve found it incredibly convenient to have both application deployment information and server configuration information (Puppet manifests) in the same place — allowing us to readily provision and configure a server, deploy an application, and document the whole setup in one source control repository.
At a high level, once we have an application that needs to be deployed, we:
- Add Capistrano to the source code repository, if not already in use.
- Add Puppet manifests and modules to the source code repository.
- Configure Capistrano to bootstrap Puppet, upload Puppet manifests and modules, and run Puppet.
- Run Puppet via Capistrano to provision and configure the servers.
- Configure Capistrano to deploy the application, making use of Capistrano multi-stage if we need to handle multiple environments or servers.
- Run Capistrano to deploy and manage the application.
In practice, we write Puppet manifests and modules incrementally — writing some Puppet code, running the Puppet agent to verify it produces expected results, and repeating the process. After we have finished writing the Puppet manifests and modules, we do a full test run on a clean system to verify that our code behaves correctly from bootstrapping to final configuration implementation.
After we have the Puppet code finalized, we use a similar process on the Capistrano recipes to verify that deployment completes correctly. Often we will complete this process on our local machines, making use of virtual machines that we can snapshot and revert as needed to facilitate the code development process.
The remainder of the post gives some specific examples and code snippets which show how we configure Capistrano to bootstrap and run puppet, how we bootstrap RVM and Puppet, and how we tie it all together.
For a generic Rails application, our source control repository may look something like this (emphasis on directories used by Capistrano and Puppet):
Capistrano Modifications for Puppet
Note: We presume that the given user has sudo privileges on the system already. The “bootstrap” namespace involves bootstrapping Puppet, which includes installing RVM. The “puppet” namespace involves running puppet after bootstrapping.
Note: This particular shell script is written for Ubuntu-based systems. It may also work for other Debian-based systems. It shouldn’t be hard to tweak for Redhat/Fedora-based systems. We install Ruby
1.9.3-p125 to manage Puppet. This may or may not be the version of Ruby needed for the application that will be deployed. That can be specified elsewhere in Capistrano, in the individual stage files as
Making it All Work
Now that we have all of our Puppet manifests and modules in one repository with our application code and Capistrano tying everything together, we can setup our server and get our application running:
- cap bootstrap
- cap puppet
- cap deploy:setup
- cap deploy
And if we had stages setup for separate servers with Capistrano multi-stage, we could easily setup our server and application on each stage:
- cap [stage] bootstrap
- cap [stage] puppet
- cap [stage] deploy:setup
- cap [stage] deploy