Most of today’s popular CI services support running jobs in arbitrary Docker containers. In this post, I’ll describe why Docker is such a great fit for CI and how it can be made even better with custom images.
Docker is great for CI for several reasons:
- Speed: Containers spin up very quickly.
- Consistency: Docker images provide a consistent environment that changes on your schedule. If you need it, you can still spin up something old like
ubuntu:14.04, even as the cloud host evolves its infrastructure to follow the market.
- Tailor-made: Images can be targeted to your specific needs (e.g.
This is a big improvement over running CI on general-purpose VMs or natively on machines you manage, but it gets even better.
Why Custom Docker
The idea behind a custom image is that you can take an existing image meant for your language ecosystem, and then apply specific changes for your project.
If your CI job configuration has accumulated a handful of setup steps, especially if they take a while to run and don’t change often, then you’re a good candidate for a custom image. Perhaps you need to install a particular version of a compiler or SDK, download a proprietary CLI tool, or precompile a big slow native dependency.
If you’ve never built a custom image before, follow along! It’s easier than I expected.
A Docker container is a running instance of a Docker image. A Docker image is defined in a Dockerfile. Let’s write one!
- Begin by installing and starting Docker Desktop if you don’t already have it.
- Choose an upstream image. If your CI job is already running in Docker, you probably want to start with its existing image. Otherwise, you can search Docker Hub for your language, tool, or Linux distribution of choice.
- In an empty directory, create a text file called
Dockerfile, and put your upstream image at the top behind a FROM instruction:
- Build it with
docker build . -t my-image:
> docker build . -t my-image Sending build context to Docker daemon 2.048kB Step 1/1 : FROM ubuntu:latest ---> 775349758637 Successfully built 775349758637 Successfully tagged my-image:latest
- Shell into it with
docker run -it my-image:
> docker run -it my-image root@a27df4742348:/# cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=18.04 DISTRIB_CODENAME=bionic DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"This interactive shell is an ephemeral sandbox, so it’s a great place to try out the setup steps that you’d like to bake into your CI image. Once you know what commands you need, add them to the Dockerfile behind RUN instructions:
Rebuild it with the same docker build command until you’re satisfied with the results.
FROM ubuntu:latest RUN apt-get update && apt-get install -y clang
- Push your new image to a registry. For Docker Hub, start with
docker login, then tag your image according to your user or organization name:
docker tag my-image:latest my-username/my-image:latestAnd then push it with:
docker push my-username/my-image:latest(You can also use alternative registries like Amazon’s ECR or Microsoft’s Azure Container Registry.)
- Finally, update your CI config to reference your new image, kick off a new build, and enjoy the improvement.
Could your project benefit from a custom image? Let me know in the comments!