Article summary
Continuous Integration/Continuous Deployment (CI/CD) is an important process that creates a reliable and consistent way for developers to push and deploy tested code. A slow CI/CD process can create roadblocks and unnecessary strain on developers. This lengthens the delivery time an organization can deliver a product to its customers.
Here, I’ll discuss what I’ve learned from my project to speed up our GitLab CI pipelines. Also, I want to note that these tips may not apply to every project, but you can use them as starter ideas you can adjust to your needs.
Caching
Caching is one of the most effective and practical ways to speed up the Gitlab CI/CD process. A cache is a stored list of files and directories where your runner is installed. It is commonly used to download and save dependencies (for example, packages) you pull from the internet.
This includes but is not limited to PHP packages, Ruby Gems, Python packages, and Maven dependencies. In a recent project, we decided to cache our Yarn node modules. By caching those node modules, we no longer have to install them in later stages. This erases repetitive behaviors in later stages reducing job runtime.
Artifacts
Similar to caching, artifacts are efficient in reducing pipeline runtime. However, artifacts are different when compared to caches. Artifacts are stored as an archive consisting of files and directories in GitLab. They are used to pass build results between stages, so it’s common to store build files as artifacts. They can also be downloaded in case you want to access them yourself.
Here’s a simplified example of how we set up our artifacts in a project of mine.
.build-artifacts:
stage: build
script:
yarn workspace components build:storybook
yarn workspace api build:lambda_zips
artifacts:
paths:
- ./api/dist
- ./api/prisma
- ./components/dist
- ./components/storybook-static
- ./components/src/graphql/graphql-types.gen.ts
Above, we created a job called build-artifacts that runs in the build stage. We then run some build scripts for our components and API workspaces. Then we specify the paths we want to store as artifacts. Now, we can use these build files in later stages without rebuilding them.
This was all we needed to set up our artifacts, but your situation may be different. Luckily, GitLab allows us to customize our artifacts further. You can find the documentation here.
Using Smaller Docker Images
If you’re building a Docker image in your CI pipelines, it can be beneficial to use a smaller Docker image. You might be using a larger image than you need. Larger images can contain unnecessary tools, libraries, and other files. A smaller Docker image will shorten the time your jobs need to pull and prepare. For example, In my current project, we were using the image node:16.14.0 based on buildpack-deps that contain many common Debian packages. While it was sufficient, we knew we could improve by switching to node:16.14.0-alpine. We switched to an Alpine-Linux-based node image because it’s a popular minimal Docker image that can be prepared faster than our larger node image.
Speeding up your GitLab CI
GitLab CI/CD provides us with amazing tools to improve the speed we can test, build, and deploy our software. To get the most out of GitLab’s CI/CD, we need to configure their tools in a way that provides us with the most value. I hope these tips help you on your journey to improve your CI pipelines.