Docker for Mac: Overcoming Slow Mounted Volumes

For about two years, I’ve wanted to use Docker for local development. Hypothetically, it offers all the benefits of virtualized development environments like Vagrant (stable, re-creatable, isolated, etc.) but requires fewer resources.

Working at a consultancy, sometimes I need to switch back and forth between multiple projects in a day. Spinning a full VM up and down can take a while. Alternatively, running two or more virtual machines at once can eat up all of my computer’s resources.

While I’d been interested in Docker for a while, I hadn’t had the time and energy to really dive into it. Then I went to DockerCon this April, which finally gave me enough momentum to figure out how to integrate it into my development workflow.

Sharing is Caring (About File IO Speed)

I suspected—correctly—that Docker would fall prey to some of the same shortcomings as Vagrant. Specifically, I am thinking of the speed of shared volumes. Generally speaking, sharing files between MacOS and a virtual OS on a hypervisor breaks down when too many reads or writes are required in a short amount of time. This may be due to running asset pipelines like those of Rails or Ember that generate tons of temporary files.

Most sharing systems (VirtualBox, NFS) do not support ignoring subdirectories. That wouldn’t be so bad, except that many framework tools do not let you configure the location of dependencies and temp directories. I’m looking at you, Ember.

What’s a Developer to Do?

Generally, this problem leads you to one of the following solutions:

1. Doing the reads/writes on the host and pushing the finished output into the container/guest
2. Using SSHFS to share files out of the container/guest back to the host
3. Using rsync or similar utilities that have to run separately but allow for ignoring subdirectories
4. Editing the code in the container via SSH plus Vim or Emacs

Why I Wasn’t Satisfied

Each of the solutions mentioned above has drawbacks:

1. Building on the host: You risk drift between your host machine and the development environment, and you have to install all the tooling on your host for all versions used by your projects.
2. SSHFS: Personally, I’ve found this very flaky, especially if your computer goes to sleep. Searching the shared files from the host can be slow, to say nothing of editor tooling with compiled languages like TypeScript or C#. Additionally, having the code “live” in something as ephemeral as a Docker container seems like a bad idea.
3. rsync: I think it’s annoying to have to run another process. Plus, rsync is one way, so changes on the container will not be reflected on the host (e.g. output from Rails generators). Of course, there are other options in this space such as Unison, that are bi-directional.
4. Editing in the container: I miss some of the features/layout of editors like VS Code or Atom. And you’ve got to find a way to get all your favorite Vim or Emacs configs into the container. This solution also suffers from the same drawbacks as SSHFS with regard to code living the container.

Of course, there’s one option for Docker that sidesteps this whole issue. You could develop in a windowed Linux environment (native or virtual) that functions as the Docker machine. File sharing between the Docker machine and containers is extremely fast. But if you want to stay in MacOS and use Docker, I have a few tips to make your life better.

Docker-sync and Upcoming Changes

Docker-sync is a very handy Ruby gem that makes it easy to use rsync or unison file sharing with Docker. Rsync and unison allow you to exclude subdirectories, so you can ignore ./tmp, ./node_modules, ./dist, and so on. This gem even takes file sharing a step farther, using Docker volumes in conjunction with rsync/unison for optimum performance.

For instance, in my Rails project’s Dockerfile, I ADD just enough files to run bundle install, then I mount in my source directory via docker-compose. It’s important to remember that this wipes whatever you ADDed during the build. So, for example, if you add a gem to your Gemfile and bundle install, you’ll eventually need to rebuild your base image.

Generally, you should just run docker-compose up --build to make sure your image doesn’t get too out of date. Also, if you are using rsync, you’ll need to docker cp the automatically updated Gemfile.lock back out of the container to the host.

Docker is working on some improvements to Docker for Mac that may significantly improve the speed of reads and writes–provided you are okay with eventual consistency. The improvements for reading speed are coming soon, but last I checked, the improvements for writing speed (the more common problem in my experience) are still a ways out.

Slow IO±It’s More than Just Volume Mounting

Another disk IO problem you might run into using Docker for Mac is slow database speed. I noticed this when our Rails database migrations took around 10 times longer to run on Docker for Mac versus native.

After a bit of searching, I found this script on a GitHub issue. Running the script brought performance back to approximately the same as native. Hopefully, this issue will be fixed soon, and the script will be unnecessary.

I Still Want to Docker All the Things

Developing in Docker isn’t perfect, but the workflow is improving rapidly. So is the whole Docker ecosystem.

Multi-stage builds are going to streamline the process of creating development and production images. The Moby project will help developers pick and choose what parts of the tools they want to use.

I have faith that in a year or two, the file sharing/disk IO speed problems will be a distant memory. The container revolution is just getting started.

If you’re at all on the fence, I encourage you to try out Docker. For me, it wasn’t until I ported an existing project into Docker that I began to understand the big picture.