Using a custom Docker image to build locally and in Cloud Build
Going over the afterthoughts from yesterday’s post I figured out I could try one in a minute and refresh some Docker knowledge to use the same build environment locally and in Cloud Build. In this post, I will describe how I improved the CI/CD speed and unified my local and cloud enviroments.
Let’s start with the easy ones. I wanted to verify if the environment variable was picked up, for that, I decided to configure Google Analytics and Disqus for this site. I won’t detail how I did that because it will depend on the theme you use. After adding the required configuration I could see events in GA and the Disqus box loading at the bottom of the posts. I also wanted to see if using -c
while syncing the files in the storage bucket would make a difference. After trying it locally and seeing that it worked fine, I added it to my build configurations.
For the core topic of this post, I had to, one more time, spend some time checking Docker documentation. I looked at the image I used yesterday (‘jekyll/jekyll`), but this image needs to be very flexible and secure, which makes the configuration and the required scripts quite complicated. For that reason, I decided to create my own image for this project.
Creating the image
First, a warning. I built this for my personal project, I’m not an expert with Ruby and its Gems, and I know the basics of Docker. If you read this, and you can help me to improve what I did, drop a message at the box at the bottom of the page. This is how the final Dockerfile looked like:
FROM ruby:2.7.1-alpine3.12
COPY Gemfile* ./
RUN apk upgrade --update
RUN apk add git build-base
RUN gem install --no-document bundler
RUN bundle install
WORKDIR /workspace
VOLUME /workspace
ENTRYPOINT [ "bundle", "exec", "jekyll" ]
EXPOSE 4000
I added the Dockerfile to my project repository’s root to keep it together with the project it is built for. I was using bundler
to manage the project dependencies (Jekyll is made in Ruby). Basically, I needed an image with Ruby, install bundler, and use my Gemfile to install my dependencies. Let’s go over the lines in the Dockerfile.
FROM ruby:2.7.1-alpine3.12
-> I looked at my local Ruby version (2.7.1) and looked in Dockerhub for an official Ruby image for that version. I picked the latest alpine version possible for that Ruby version. In case Alpine is new to you, it is a Linux distribution widely use with Docker images.COPY Gemfile* ./
-> This copies in the image my Gemfile and Gemfile.lock, which will be later used to install Jekyll and my theme.RUN apk upgrade --update
-> This is to upgrade all installed packages in the base image and update the local Alpine package repository.RUN apk add git build-base
-> Installs 2 packages needed to build my environment, more information on how I knew the dependencies, below.RUN gem install --no-document bundler
-> This install bundler without documentation.RUN bundle install
-> This is used to install my project dependencies. I will talk about them below.WORKDIR /workspace
-> This to set the working directory in the image.VOLUME /workspace
-> This creates a volume in that path (in the working directory) to mount an external path when the image is run.ENTRYPOINT [ "bundle", "exec", "jekyll" ]
-> I will use this image just for Jekyll commands, setting the entry point this way simplifies running commands later.EXPOSE 4000
-> This is merely informative but good practice. It is the port to bind with the host.
The Docker file is not specially complicated, but it took me a while to get right a couple of instructions. For example, my Gemfile is:
source "https://rubygems.org"
gem "jekyll", "~> 4.1.1"
gem "minima", :github => 'jekyll/minima', :ref => 'a98a8fe'
I tried to RUN bundle install
without any of the apk
lines in the first trial. That failed because I use a specific commit for one of my dependencies, and that required git
. Knowing that, I added the first apk
line and the second only with the git package. After those changes, the installation still failed with a more cryptic message. Everything pointed out was related to building some C extensions, and a couple of searches helped find that build-base
is the very minimum package needed in Alpine to build tools. Luckily for me, that did it.
I also didn’t choose workspace
as the name of the working directory and volume, but that is a requirement I will explain in one of the following sections.
Using the image locally
Building the image is simple: docker build -t jekyll-builder .
. I already had a little bash script for the usual commands, and to deploy the site manually, all I had to do was to replace the bundle commands with docker commands:
bundle exec jekyll serve
->docker run -p 4000:4000 -v "$PWD:/workspace" jekyll-builder serve
JEKYLL_ENV=production bundle exec jekyll build
->docker run -e JEKYLL_ENV=production -v "$PWD:/workspace" jekyll-builder build
If you are a little familiar with Docker, both commands should be easy to understand. In both, we need to bind-mount the current directory to the workspace volume. When “serving” you need to bind the image and host ports to see the page, and when building, you need to pass the environment variable to the image.
To get serving the site from the image, I had to change the serving IP from 127.0.0.1
to 0.0.0.0
. It is as simple as adding host: 0.0.0.0
to your Jenkins configuration file (_config.yml
).
Using the image in Cloud Build
Once I had the image working locally, the only missing piece was using its Google Cloud build. To avoid problems by having a Dockerfile and a cloudbuild.yaml
file in the repository, I went to my trigger configuration and selected build configuration file type to be Could Build. Another issue was that I needed to make the image available in the cloud. I already had a couple of images in Dockerhub, and uploaded the new one there.
This was the final cloudbuild.yaml
file:
steps:
- id: Build Jekyll Site
name: 'nestorlafon/jekyll-builder'
args: ['build']
env: ['JEKYLL_ENV=production']
- id: Deploy to Cloud Storage
name: 'gcr.io/cloud-builders/gsutil'
args: ['rsync', '-c', '-d', '-r', './_site', 'gs://notes.nestorlafon.com']
timeout: '10m'
Compared with the previous one, the first step of updating file permissions is gone. That’s possible because my image is run as root. For the second step, I simply had to update the name and the args to match my image requirements.
How does the repository code get into the image? That’s why you need to name your volume workspace
. Could Build tries to bind the build workspace to a volume with that name by default, otherwise, you need to add more configuration in your cloudbuild file.
My CI/CD pipeline when from a minute to 30 seconds with these changes
Afterthoughts
I’m pleased with my current setup, but there are 2 things that bug me. First is that the image gets downloaded in Code Build every time; I couldn’t find how to cache it. The second has to do with the fact that I probably could stop using bundler and add the gems in my Gemfile to the image itself. The problem here is that I didn’t find a simple way to install a gem pointing to a commit without a Gemfile.