Most of today’s popular CI services support running jobs in arbitrary Docker containers. In this post, I’ll describe why Docker is such a great fit for CI and how it can be made even better with custom images.
## Why Docker
Docker is great for CI for several reasons:
– **Speed:** Containers spin up very quickly.
– **Consistency**: Docker images provide a consistent environment that changes on _your_ schedule. If you need it, you can still spin up something old like `ubuntu:14.04`, even as the cloud host evolves its infrastructure to follow the market.
– **Tailor-made:** Images can be targeted to your specific needs (e.g. `node:12` or `mcr.microsoft.com/dotnet/core/sdk:3.1` or `circleci/android:api-29`)
This is a big improvement over running CI on general-purpose VMs or natively on machines you manage, but it gets even better.
## Why _Custom_ Docker
The idea behind a custom image is that you can take an existing image meant for your language _ecosystem_, and then apply specific changes for your _project_.
If your CI job configuration has accumulated a handful of setup steps, especially if they take a while to run and don’t change often, then you’re a good candidate for a custom image. Perhaps you need to install a particular version of a compiler or SDK, download a proprietary CLI tool, or precompile a big slow native dependency.
If you’ve never built a custom image before, follow along! It’s easier than I expected.
A Docker _container_ is a running instance of a Docker _image_. A Docker _image_ is defined in a [Dockerfile]. Let’s write one!
- Begin by installing and starting Docker Desktop if you don’t already have it.
- Choose an upstream image. If your CI job is already running in Docker, you probably want to start with its existing image. Otherwise, you can search Docker Hub for your language, tool, or Linux distribution of choice.
- In an empty directory, create a text file called
Dockerfile, and put your upstream image at the top behind a FROM instruction:
- Build it with
docker build . -t my-image:
> docker build . -t my-image Sending build context to Docker daemon 2.048kB Step 1/1 : FROM ubuntu:latest ---> 775349758637 Successfully built 775349758637 Successfully tagged my-image:latest
- Shell into it with
docker run -it my-image:
> docker run -it my-image root@a27df4742348:/# cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
This interactive shell is an ephemeral sandbox, so it’s a great place to try out the setup steps that you’d like to bake into your CI image. Once you know what commands you need, add them to the Dockerfile behind RUN instructions:
FROM ubuntu:latest RUN apt-get update && apt-get install -y clang
Rebuild it with the same docker build command until you’re satisfied with the results.
- Push your new image to a registry. For Docker Hub, start with
docker login, then tag your image according to your user or organization name:
docker tag my-image:latest my-username/my-image:latest
And then push it with:
docker push my-username/my-image:latest
(You can also use alternative registries like Amazon’s ECR or Microsoft’s Azure Container Registry.)
- Finally, update your CI config to reference your new image, kick off a new build, and enjoy the improvement.
This approach works with [CircleCI][circle_docker], [GitHub Actions][github_actions_container], [GitLab CI][gitlab_docker], [BitBucket Pipelines][bitbucket_pipelines_docker], [Azure Pipelines][azdo_container_jobs], and probably others.
For concrete examples, I’ve had good outcomes using it to [add Mono to a ruby-node image](https://github.com/jrr/ruby-node-mono/blob/master/Dockerfile) and to build an embedded project with [Espressif’s Xtensa GCC](https://github.com/jrr/esp32-ci/blob/master/Dockerfile), shaving minutes off project build times.
Could your project benefit from a custom image? Let me know in the comments!