Jekyll2021-01-22T10:12:28+00:00http://notes.nestorlafon.com/feed.xmlNestor’s NotesNestor's notes about programming, tech, about programming, technology, sofware engineering, and engineering management.
{"name"=>"", "email"=>""}Using a custom Docker image to build locally and in Cloud Build2020-12-30T00:00:00+00:002020-12-30T00:00:00+00:00http://notes.nestorlafon.com/tech/2020/12/30/could-build-jekyll-custom-build<p>Going over the afterthoughts from <a href="/tech/2020/12/29/jekyll-site-ci-cd-in-gcp.html">yesterday’s post</a> I figured out I could try one in a minute and refresh some Docker knowledge to use the same build environment locally and in Cloud Build. In this post, I will describe how I improved the CI/CD speed and unified my local and cloud enviroments.
<!--more--></p>
<p>Let’s start with the easy ones. I wanted to verify if the environment variable was picked up, for that, I decided to configure Google Analytics and Disqus for this site. I won’t detail how I did that because it will depend on the theme you use. After adding the required configuration I could see events in GA and the Disqus box loading at the bottom of the posts. I also wanted to see if using <code class="language-plaintext highlighter-rouge">-c</code> while syncing the files in the storage bucket would make a difference. After trying it locally and seeing that it worked fine, I added it to my build configurations.</p>
<p>For the core topic of this post, I had to, one more time, spend some time checking Docker documentation. I looked at the <a href="https://hub.docker.com/r/jekyll/jekyll">image I used yesterday</a> (‘jekyll/jekyll`), but this image needs to be very flexible and secure, which makes the configuration and the required scripts quite complicated. For that reason, I decided to create my own image for this project.</p>
<h3 id="creating-the-image">Creating the image</h3>
<p>First, a warning. I built this for my personal project, I’m not an expert with Ruby and its Gems, and I know the basics of Docker. If you read this, and you can help me to improve what I did, drop a message at the box at the bottom of the page. This is how the final Dockerfile looked like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>FROM ruby:2.7.1-alpine3.12
COPY Gemfile* ./
RUN apk upgrade --update
RUN apk add git build-base
RUN gem install --no-document bundler
RUN bundle install
WORKDIR /workspace
VOLUME /workspace
ENTRYPOINT [ "bundle", "exec", "jekyll" ]
EXPOSE 4000
</code></pre></div></div>
<p>I added the Dockerfile to my project repository’s root to keep it together with the project it is built for. I was using <code class="language-plaintext highlighter-rouge">bundler</code> to manage the project dependencies (Jekyll is made in Ruby). Basically, I needed an image with Ruby, install bundler, and use my Gemfile to install my dependencies. Let’s go over the lines in the Dockerfile.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">FROM ruby:2.7.1-alpine3.12</code> -> I looked at my local Ruby version (2.7.1) and looked in Dockerhub for an official Ruby image for that version. I picked the latest alpine version possible for that Ruby version. In case Alpine is new to you, it is a Linux distribution widely use with Docker images.</li>
<li><code class="language-plaintext highlighter-rouge">COPY Gemfile* ./</code> -> This copies in the image my Gemfile and Gemfile.lock, which will be later used to install Jekyll and my theme.</li>
<li><code class="language-plaintext highlighter-rouge">RUN apk upgrade --update</code> -> This is to upgrade all installed packages in the base image and update the local Alpine package repository.</li>
<li><code class="language-plaintext highlighter-rouge">RUN apk add git build-base</code> -> Installs 2 packages needed to build my environment, more information on how I knew the dependencies, below.</li>
<li><code class="language-plaintext highlighter-rouge">RUN gem install --no-document bundler</code> -> This install bundler without documentation.</li>
<li><code class="language-plaintext highlighter-rouge">RUN bundle install</code> -> This is used to install my project dependencies. I will talk about them below.</li>
<li><code class="language-plaintext highlighter-rouge">WORKDIR /workspace</code> -> This to set the working directory in the image.</li>
<li><code class="language-plaintext highlighter-rouge">VOLUME /workspace</code> -> This creates a volume in that path (in the working directory) to mount an external path when the image is run.</li>
<li><code class="language-plaintext highlighter-rouge">ENTRYPOINT [ "bundle", "exec", "jekyll" ]</code> -> I will use this image just for Jekyll commands, setting the entry point this way simplifies running commands later.</li>
<li><code class="language-plaintext highlighter-rouge">EXPOSE 4000</code> -> This is merely informative but good practice. It is the port to bind with the host.</li>
</ul>
<p>The Docker file is not specially complicated, but it took me a while to get right a couple of instructions. For example, my Gemfile is:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>source "https://rubygems.org"
gem "jekyll", "~> 4.1.1"
gem "minima", :github => 'jekyll/minima', :ref => 'a98a8fe'
</code></pre></div></div>
<p>I tried to <code class="language-plaintext highlighter-rouge">RUN bundle install</code> without any of the <code class="language-plaintext highlighter-rouge">apk</code> lines in the first trial. That failed because I use a specific commit for one of my dependencies, and that required <code class="language-plaintext highlighter-rouge">git</code>. Knowing that, I added the first <code class="language-plaintext highlighter-rouge">apk</code> line and the second only with the git package. After those changes, the installation still failed with a more cryptic message. Everything pointed out was related to building some C extensions, and a couple of searches helped find that <code class="language-plaintext highlighter-rouge">build-base</code> is the very minimum package needed in Alpine to build tools. Luckily for me, that did it.</p>
<p>I also didn’t choose <code class="language-plaintext highlighter-rouge">workspace</code> as the name of the working directory and volume, but that is a requirement I will explain in one of the following sections.</p>
<h3 id="using-the-image-locally">Using the image locally</h3>
<p>Building the image is simple: <code class="language-plaintext highlighter-rouge">docker build -t jekyll-builder .</code>. I already had a little bash script for the usual commands, and to deploy the site manually, all I had to do was to replace the bundle commands with docker commands:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">bundle exec jekyll serve</code> -> <code class="language-plaintext highlighter-rouge">docker run -p 4000:4000 -v "$PWD:/workspace" jekyll-builder serve</code></li>
<li><code class="language-plaintext highlighter-rouge">JEKYLL_ENV=production bundle exec jekyll build</code> -> <code class="language-plaintext highlighter-rouge">docker run -e JEKYLL_ENV=production -v "$PWD:/workspace" jekyll-builder build</code></li>
</ul>
<p>If you are a little familiar with Docker, both commands should be easy to understand. In both, we need to bind-mount the current directory to the workspace volume. When “serving” you need to bind the image and host ports to see the page, and when building, you need to pass the environment variable to the image.</p>
<p>To get serving the site from the image, I had to change the serving IP from <code class="language-plaintext highlighter-rouge">127.0.0.1</code> to <code class="language-plaintext highlighter-rouge">0.0.0.0</code>. It is as simple as adding <code class="language-plaintext highlighter-rouge">host: 0.0.0.0</code> to your Jenkins configuration file (<code class="language-plaintext highlighter-rouge">_config.yml</code>).</p>
<h3 id="using-the-image-in-cloud-build">Using the image in Cloud Build</h3>
<p>Once I had the image working locally, the only missing piece was using its Google Cloud build. To avoid problems by having a Dockerfile and a <code class="language-plaintext highlighter-rouge">cloudbuild.yaml</code> file in the repository, I went to my trigger configuration and selected build configuration file type to be Could Build. Another issue was that I needed to make the image available in the cloud. I already had a couple of images in Dockerhub, and <a href="https://docs.docker.com/docker-hub/repos/">uploaded the new one there</a>.</p>
<p>This was the final <code class="language-plaintext highlighter-rouge">cloudbuild.yaml</code> file:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>steps:
- id: Build Jekyll Site
name: 'nestorlafon/jekyll-builder'
args: ['build']
env: ['JEKYLL_ENV=production']
- id: Deploy to Cloud Storage
name: 'gcr.io/cloud-builders/gsutil'
args: ['rsync', '-c', '-d', '-r', './_site', 'gs://notes.nestorlafon.com']
timeout: '10m'
</code></pre></div></div>
<p>Compared with the previous one, the first step of updating file permissions is gone. That’s possible because my image is run as root. For the second step, I simply had to update the name and the args to match my image requirements.</p>
<p>How does the repository code get into the image? That’s why you need to name your volume <code class="language-plaintext highlighter-rouge">workspace</code>. Could Build tries to bind the build workspace to a volume with that name by default, otherwise, you need to add more configuration in your cloudbuild file.</p>
<p>My CI/CD pipeline when from a minute to 30 seconds with these changes</p>
<h3 id="afterthoughts">Afterthoughts</h3>
<p>I’m pleased with my current setup, but there are 2 things that bug me. First is that the image gets downloaded in Code Build every time; I couldn’t find how to cache it. The second has to do with the fact that I probably could stop using bundler and add the gems in my Gemfile to the image itself. The problem here is that I didn’t find a simple way to install a gem pointing to a commit without a Gemfile.</p>
<div>
<center>
<script type="text/javascript" src="https://cdnjs.buymeacoffee.com/1.0.0/button.prod.min.js" data-name="bmc-button" data-slug="nestorlafon" data-color="#FF5F5F" data-emoji="" data-font="Bree" data-text="Like it? You can invite me to coffee ;) " data-outline-color="#000000" data-font-color="#ffffff" data-coffee-color="#FFDD00"></script>
</center>
</div>{"name"=>"", "email"=>""}Going over the afterthoughts from yesterday’s post I figured out I could try one in a minute and refresh some Docker knowledge to use the same build environment locally and in Cloud Build. In this post, I will describe how I improved the CI/CD speed and unified my local and cloud enviroments.Using Google Cloud Build to publish your Jekyll site in GCP2020-12-29T00:00:00+00:002020-12-29T00:00:00+00:00http://notes.nestorlafon.com/tech/2020/12/29/jekyll-site-ci-cd-in-gcp<p>After <a href="/tech/2020/12/28/free-static-site-in-gcp.html">yesterday’s post</a>, this site was hosted in Google Cloud Storage without incurring any cost. Every good project needs its own CI/CD pipeline, and I decided to find a way to add one to build and publish the site automatically. And I got it to work, using Google Could Build.
<!--more--></p>
<p>As with every little project, everything starts with research. I found dozens of promising leads, but nothing that was exactly what I was looking for. Besides GCP official docs, these two are the pages I used to create this post:</p>
<ul>
<li><a href="https://cloud.google.com/community/tutorials/automated-publishing-cloud-build">Automated static website publishing with Cloud Build</a>
<ul>
<li>From this little tutorial, I learned how to create a trigger and send files from the build workspace to the storage bucket</li>
</ul>
</li>
<li><a href="https://queensidecastle.com/guides/jekyll-on-app-engine-with-ci-cd">Jekyll on App Engine with CI/CD</a>
<ul>
<li>I thought this page was what I was looking for, but besides being a bit outdated, I don’t want to use App Engine for this project at all, and it is the most complicated part of the article</li>
<li>Nevertheless, the section about <a href="https://queensidecastle.com/guides/jekyll-on-app-engine-with-ci-cd#add-the-cloudbuildyaml">adding the cloudbuild.yaml</a> was pure gold.</li>
</ul>
</li>
</ul>
<h3 id="connecting-github-and-google-cloud-build">Connecting GitHub and Google Cloud Build</h3>
<p>Let’s get our hands dirty. The first thing was creating a trigger every time there was a new push in the repository. For that, you need to use Cloud Build product and make sure the project is the right one in the GCP console. In my case, my code is hosted in GitHub and not in GCP, which requires an extra step: connect your GitHub repository and Google Cloud Build. There are multiple steps, but the user flow is quite detailed:</p>
<ol>
<li>Click in the Triggers section in Could Build</li>
<li>Click in “Connect to repository” in the center of the page</li>
<li>Add your GitHub repository URL (the HTTPS link works fine)</li>
<li>Authorize Could Build in your GitHub account</li>
<li>Install Google Could Build app at least in the repository you are going to use</li>
<li>Read and acknowledge the “risks” of what you just did ;)</li>
</ol>
<h3 id="creating-the-trigger-and-configuring-the-build">Creating the trigger and configuring the build</h3>
<p>Next is creating the trigger, either right after the connection is done or from the Triggers section. It is relatively straightforward, and primarily your job is to find a decent unique name. Add the name, the description, pick the repository, make sure “push to a branch” and the branch regexp is correct, and click on create.</p>
<p>Before you continue, don’t skip one of the steps mentioned in one of the pages I listed above. You will need to enable <code class="language-plaintext highlighter-rouge">Could Build API</code> to be able to use Cloud Build. To do so, you can go to Setting in Cloud Build, follow the link and enable the API. There you can see the free tier, 120 minutes per day, more than enough if you are only running a little static site.</p>
<p>Next step is to create and add the build configuration file. I used what I learned in from the pages references above to compose mine:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>steps:
- id: Update Permissions
name: 'gcr.io/cloud-builders/git'
entrypoint: 'chmod'
args: ['-v','-R', 'a+rw','.']
- id: Build Jekyll Site
name: 'jekyll/jekyll'
args: ['jekyll','build']
env: ['JEKYLL_ENV=production']
- id: Deploy to Cloud Storage
name: 'gcr.io/cloud-builders/gsutil'
args: ['rsync', '-d', '-r', './_site', 'gs://notes.nestorlafon.com']
timeout: '10m'
</code></pre></div></div>
<p>There are 3 build steps: Making all files in the build workspace accessible to all users, build the static site, upload the files to the right storage bucket. To better understand the file syntax and what happens, the best is to read Could Build documentation and check the logs once the first build runs. The first step is needed because the second step runs using a non-root user, and files won’t be readable otherwise.</p>
<p>Add your configuration to <code class="language-plaintext highlighter-rouge">couldbuild.yaml</code> in the root of your repository, commit, and push. That should trigger your first build. Go to the History section, click on the build id, and enjoy it ;)</p>
<h3 id="afterthoughts">Afterthoughts</h3>
<p>I don’t use docker locally to build my Jekyll site. I tried using it, and it works, but it makes things a bit slower. Regardless of that little inconvenience, it is a good way to be “environment independent”. This is the command I used locally:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>docker run --rm \
-p 4000:4000 \
--volume="$PWD:/srv/jekyll" \
-it jekyll/jekyll:4 \
jekyll serve
</code></pre></div></div>
<p>I want to see if I can modify or create a similar image that doesn’t need to update/install gems in each run, but it is nice that the default image uses the Gemfile in your folder. In Could Build, I see that having a more “precise” image will help build time (the build itself is a second, but the step takes close to a minute).</p>
<p>In the current state, I found 2 minor issues: It looks like the png asset in the first post always gets uploaded to the bucket even if nothing has changed, and I’m unsure the environment is being picked up in the build step. I will see if adding <code class="language-plaintext highlighter-rouge">-c</code> to use checksums instead of timestamps to the last step helps with the first issue, and I will add some production-only part in the site to verify the latter.</p>
<div>
<center>
<script type="text/javascript" src="https://cdnjs.buymeacoffee.com/1.0.0/button.prod.min.js" data-name="bmc-button" data-slug="nestorlafon" data-color="#FF5F5F" data-emoji="" data-font="Bree" data-text="Like it? You can invite me to coffee ;) " data-outline-color="#000000" data-font-color="#ffffff" data-coffee-color="#FFDD00"></script>
</center>
</div>{"name"=>"", "email"=>""}After yesterday’s post, this site was hosted in Google Cloud Storage without incurring any cost. Every good project needs its own CI/CD pipeline, and I decided to find a way to add one to build and publish the site automatically. And I got it to work, using Google Could Build.Eliminating costs of running a basic static site in GCP2020-12-28T00:00:00+00:002020-12-28T00:00:00+00:00http://notes.nestorlafon.com/tech/2020/12/28/free-static-site-in-gcp<p>In <a href="/tech/2020/12/27/static-site-in-gcp.html">yesterday’s post</a>, I went over how to run a little static site (the one where you are reading this) in CGP following their guide.</p>
<p>After finishing the setup, I had the feeling that at least the load balancer (LB) was going to have some cost. I was right. Checking the billing today, I could see how the LB will incur daily charges even with very low traffic. Calculating the cost of the LB is indeed tricky. The pricing is quite complicated, and you need to take into account multiple factors. I will show ways to eliminate the cost of it in this post.
<!--more--></p>
<p>Not only that. Looking at the <a href="https://cloud.google.com/free/docs/gcp-free-tier#free-tier-usage-limits">quite-limiting free tier in GCP</a>, I realized the Could Storage free tier only applies to US region buckets. Below I will describe how I moved the bucket I created in <code class="language-plaintext highlighter-rouge">europe-west3</code> to <code class="language-plaintext highlighter-rouge">us-east1</code> to make sure the little storage I need will be included in the free tier and remove the Load Balancer.</p>
<h3 id="moving-a-bucket-to-a-different-region">Moving a bucket to a different region</h3>
<p>If your bucket is empty, the best is to delete it and then create a new one in the new region. If, like my case, you already have data in the bucket, you will need to create the new bucket in the new region and transfer the data before deleting the current bucket. Bucket names are unique; if for any reason, you need to reuse the name, which again is my situation, you will need to use an intermediate bucket for your data.</p>
<p>In my case, as mentioned above, I want to have the same bucket name but in a different region. Luckily, I don’t need to move the data in the bucket because I can reupload the site as easily as I update it. That saves me the hassle of creating an intermediate bucket.</p>
<p>First, I removed the bucket using the console. It was the first time deleting a bucket in GCP for me, and the console is more forgiving if you are still learning. Yesterday, I created the bucket using the console. This time, I decided to use the CLI. Running <code class="language-plaintext highlighter-rouge">gsutil mb -l us-east1 -c standard -b on -p notes-nestorlafon-com gs://notes.nestorlafon.com</code> creates the bucket with the configuration needed. <em>Careful</em>, the guide doesn’t have the same parameters as my command; some of them are the default value, but others, like <code class="language-plaintext highlighter-rouge">-p</code> are needed unless you have a configuration file for <code class="language-plaintext highlighter-rouge">gsutil</code>. Once the bucket is created, you want to make it public. The CLI command for that is simple <code class="language-plaintext highlighter-rouge">gsutil iam ch allUsers:objectViewer gs://notes.nestorlafon.com</code>.</p>
<p>Next is uploading the site again. For that I decided to give a try to the <code class="language-plaintext highlighter-rouge">rsync</code> option with the command <code class="language-plaintext highlighter-rouge">gsutil rsync -d -r _site gs://notes.nestorlafon.com</code>. It spits some warnings, but it seems to do the expected job. The <code class="language-plaintext highlighter-rouge">-d</code> option is to delete files not present in the source.</p>
<p>One last step before verifying that everything is working as expected is to set up the bucket for acting as a website. For that, I used the command <code class="language-plaintext highlighter-rouge">gsutil web set -m index.html -e 404.html gs://notes.nestorlafon.com</code>.</p>
<p>After that, <code class="language-plaintext highlighter-rouge">notes.nestorlafon.com</code> was working again as expected from a bucket in a different region.</p>
<h3 id="removing-the-load-balancer">Removing the Load Balancer</h3>
<p>To be able to remove the load balancer, as I explain below, your site needs to be served from a subdomain, in my case <code class="language-plaintext highlighter-rouge">notes</code> in `nestorlafon.com, and you should be ok use insecure HTTP for it. If you plan to run any serious website, it is better to stick to pay for the LB and an SSL certificate for your HTTPS connection.</p>
<p>In the console, if you go to your bucket, and click on one file, you will see that the public URL look like <code class="language-plaintext highlighter-rouge">https://storage.googleapis.com/notes.nestorlafon.com/index.html</code>. You can rewrite it as <code class="language-plaintext highlighter-rouge">http://notes.nestorlafon.com.storage.googleapis.com/index.html</code> and it will still work. Note that the second link uses HTTP and has the bucket’s name as part of the domain. That’s what makes possible the next step.</p>
<p>Now you can go to your domain registrant and change the DNS configuration. Delete the old <code class="language-plaintext highlighter-rouge">A</code> entry for your subdomain and create a new <code class="language-plaintext highlighter-rouge">CNAME</code> entry pointing to your bucket domain. In my case, <code class="language-plaintext highlighter-rouge">notes</code> pointing to <code class="language-plaintext highlighter-rouge">notes.nestorlafon.com.storage.googleapis.com</code>.</p>
<p>Now you can remove the LB. I went to the console, selected the LB, and press delete. It will prompt you to remove the backend bucket. Tick that box; deleting that backend bucket won’t affect your storage bucket, as it is only part of the LB configuration.</p>
<p>There you go. Your static site is running for free in GCP.</p>
<h3 id="amendment-a-few-days-later">Amendment a few days later</h3>
<p>Accidentally, I found Google was still billing me some cents per day. Looking at the billing details, this was due to <code class="language-plaintext highlighter-rouge">Static IP Charge</code> in <code class="language-plaintext highlighter-rouge">Compute Engine</code>. But looking in the console section for <code class="language-plaintext highlighter-rouge">Compute Engine</code> showed nothing. After a bit of reading, I found that I could use <code class="language-plaintext highlighter-rouge">gcloud compute addresses list</code> to see all addresses used in that project, and in the same document, I found that external static IPs are reserved in the <code class="language-plaintext highlighter-rouge">VPC Network</code>. Once there in the console, deleting the IP is a couple of clicks away. I didn’t request that IP, most probably it was needed for the Load Balancer. It wasn’t gone when I removed the <code class="language-plaintext highlighter-rouge">Load Balancer</code> as it is not part of it. I wish Google was smarter and notify you that you deleted the last resource using that IP reservation.</p>
<div>
<center>
<script type="text/javascript" src="https://cdnjs.buymeacoffee.com/1.0.0/button.prod.min.js" data-name="bmc-button" data-slug="nestorlafon" data-color="#FF5F5F" data-emoji="" data-font="Bree" data-text="Like it? You can invite me to coffee ;) " data-outline-color="#000000" data-font-color="#ffffff" data-coffee-color="#FFDD00"></script>
</center>
</div>{"name"=>"", "email"=>""}In yesterday’s post, I went over how to run a little static site (the one where you are reading this) in CGP following their guide. After finishing the setup, I had the feeling that at least the load balancer (LB) was going to have some cost. I was right. Checking the billing today, I could see how the LB will incur daily charges even with very low traffic. Calculating the cost of the LB is indeed tricky. The pricing is quite complicated, and you need to take into account multiple factors. I will show ways to eliminate the cost of it in this post.Setting up a static blog with Jekyll and Google Cloud Storage2020-12-27T00:00:00+00:002020-12-27T00:00:00+00:00http://notes.nestorlafon.com/tech/2020/12/27/static-site-in-gcp<p><strong>UPDATE</strong><em>: You can save time and money if you don’t need HTTPS and you want to use a subdomain. In that case, you don’t need a Load Balancer. See <a href="/tech/2020/12/28/free-static-site-in-gcp.html">the next note</a> for more information.</em></p>
<p>In my <a href="/tech/2020/11/01/starting-this-site.html">previous post</a> I wrote how I created this site and uploaded to AWS S3. For different reasons, I’ve decided to invest my time in getting familiar with Google Could Platform and besides some other little basic experiments, I wanted to move the page to Google Cloud Storage. The process has been quite simple as I’m finding Google documentation quite more detailed than AWS one.</p>
<!--more-->
<p>All I did was follow the instructions on <a href="https://cloud.google.com/storage/docs/hosting-static-website">how to host a static site in GCP</a>. I will highlight some points in the process below.</p>
<p>If you want to use your domain, you need to verify you “own” it. It is pretty simple, and if you follow the guide, it shouldn’t be a problem. Basically, you need to add a TXT entry in your DNS configuration. If your registrant (in my case GoDaddy) asks for a host for that entry, use <code class="language-plaintext highlighter-rouge">@</code> for the domain itself.</p>
<p>Although I had the CLI tools for GCP installed already (after fixing my Python environment setup for <em>one more time</em>), I did some of the steps in the console. I find the console easier to use if you are learning, and many errors are better there and have follow up links.</p>
<p>Following the guide’s steps, I created the bucket in the console and uploaded the files using the CLI <code class="language-plaintext highlighter-rouge">gsutil cp -r _site/* gs://notes.nestorlafon.com</code>. My site is very simple and I didn’t looked at the <code class="language-plaintext highlighter-rouge">rsync</code> option this time.</p>
<p>Compared to AWS, setting up the permissions for the bucket is simpler, the guide and console are quite explicit. The same goes for setting the main and error pages. Where it becomes more complicated is when you want to link your domain to the bucket itself. For that, the guide tells you to create a load balancer.</p>
<p>Creating the load balancer is easy following the nice documentation. In my case I didn’t want to use HTTPS, and using HTTP makes some of the steps (related to certificates) irrelevant. Once you have your external IP, go to your DNS settings and create an <code class="language-plaintext highlighter-rouge">A</code> entry for your domain (o subdomain in my case).</p>
<p>Running the page in AWS was free, I’m unsure what would be the cost in CGP as their free tier is more restrictive, but nevertheless, the traffic and size of the page won’t make it an issue straight away.</p>
<p>For the following posts, I will see how to improve the files’ uploading to be just the changed ones and later how to use CI/CD to automate updating the site.</p>
<div>
<center>
<script type="text/javascript" src="https://cdnjs.buymeacoffee.com/1.0.0/button.prod.min.js" data-name="bmc-button" data-slug="nestorlafon" data-color="#FF5F5F" data-emoji="" data-font="Bree" data-text="Like it? You can invite me to coffee ;) " data-outline-color="#000000" data-font-color="#ffffff" data-coffee-color="#FFDD00"></script>
</center>
</div>{"name"=>"", "email"=>""}UPDATE: You can save time and money if you don’t need HTTPS and you want to use a subdomain. In that case, you don’t need a Load Balancer. See the next note for more information. In my previous post I wrote how I created this site and uploaded to AWS S3. For different reasons, I’ve decided to invest my time in getting familiar with Google Could Platform and besides some other little basic experiments, I wanted to move the page to Google Cloud Storage. The process has been quite simple as I’m finding Google documentation quite more detailed than AWS one.Setting up a static blog with Jekyll and AWS S32020-11-01T00:00:00+00:002020-11-01T00:00:00+00:00http://notes.nestorlafon.com/tech/2020/11/01/starting-this-site<p>Today I decided I wanted a blog for myself, a place to put notes I can look at later, share in social networks, or write down for closure. I had many other blogs before, all on different platforms, some of them not existing anymore. I wanted to have something I could keep forever and opted to set up a markdown based static blog using Jekyll, git, and AWS S3. This is how I did it.</p>
<p><img src="/assets/images/2020/11/jekyll.png" alt="jekyll logo" /></p>
<p>I could have taken the easy path and just use <a href="https://pages.github.com/">Github pages</a>, but “no pain, no glory”. Also, I wanted to have private storage for the files and remove all magic possible in the process. I still wanted to use git for managing my “project”. I used a private repository in GitHub.</p>
<p>I spent some time reading the documentation for <a href="https://jekyllrb.com/">Jekyll</a>, an excellent and simple static site generator. I considered <a href="https://www.gatsbyjs.com/">Gatsbyjs</a>, but it is a bigger tool than I needed for this. For Jekyll, I had to fix my Ruby environment, but following the instructions on their site and closing/opening my terminal almost after each line made it work.</p>
<p>After that, I followed the “Quickstart” instructions to set up the local project, and everything worked fine. The next step was pushing this to the remote git repo. This is not different than pushing text files, and basic git knowledge is enough.</p>
<p>Now was time to upload it to <em>the Internet</em>. I want to get more familiar with AWS products, and I decided to use S3 to serve the static side. I create the bucket, configured it for Static Web Hosting (I intentionally don’t link a guide here. AWS changes so fast that most of the examples are outdated). I uploaded an index.html, verified it was working fine, then uploaded manually the static site to check it worked. Like a charm!. But that was a too manual process, and I decided to at least use some CLI to do it. That part took more than I wanted.</p>
<p>I installed the AWSCLI, which is just a “next, next, done” installer. Then created a user without permissions in AWS AIM, saved the key and secret, and added 2 more <em>Statements</em> for the new user. The policy looks like (<code class="language-plaintext highlighter-rouge"><<<redacted>>></code>):</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<<<my_bucket_name>>/*"
},
{
"Sid": "JekyllObjectPolicy",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<<<new_user_ARN>>"
},
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::<<<my_bucket_name>>/*"
},
{
"Sid": "JekyllBucketPolicy",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<<<new_user_ARN>>"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<<<my_bucket_name>>"
}
]
}
</code></pre></div></div>
<p>After some reading in the AWS CLI documents about S3, I was able to make it work by setting the new user credentials.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aws configure set aws_access_key_id <<<AWS_ACCESS_KEY_ID>>>
aws configure set aws_secret_access_key <<<AWS_SECRET_ACCESS_KEY>>>
</code></pre></div></div>
<p>and the using this command to sync the files</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aws s3 sync _site/ "s3://<<<my_bucket_name>>" --size-only --exclude "*" --include "*.*" --delete
</code></pre></div></div>
<p>After that, all needed to do was tunning the theme and the site configuration and write this post!</p>
<p>The last part was getting it under the right domain, for that, I just added a CNAME entry in my domain configuration to point the subdomain <code class="language-plaintext highlighter-rouge">notes</code> to the long AWS URL.</p>
<p>If you are reading this, it is because all of the above worked ;) .</p>
<div>
<center>
<script type="text/javascript" src="https://cdnjs.buymeacoffee.com/1.0.0/button.prod.min.js" data-name="bmc-button" data-slug="nestorlafon" data-color="#FF5F5F" data-emoji="" data-font="Bree" data-text="Like it? You can invite me to coffee ;) " data-outline-color="#000000" data-font-color="#ffffff" data-coffee-color="#FFDD00"></script>
</center>
</div>{"name"=>"", "email"=>""}Today I decided I wanted a blog for myself, a place to put notes I can look at later, share in social networks, or write down for closure. I had many other blogs before, all on different platforms, some of them not existing anymore. I wanted to have something I could keep forever and opted to set up a markdown based static blog using Jekyll, git, and AWS S3. This is how I did it.