11 minutes
Gatsby.js, Netlify & Azure Devops - Part One

Since this article was written I’ve migrated to Hugo. The pipeline and deployment processes still apply, but I’m no longer using Gatsby.
I decided recently that it was about time I started a blog, if for nothing else other than document my own experiments so that when I forget what I did and how I did it I will at least have some semblence of a record.
This post is all about how to get started with Gatsby, host using Netlify and have a build and deployment pipeline using Azure DevOps. I use Azure DevOps extensively in my day job, but for my own purposes I want to try things out when there isn’t an urgent need to develop a solution immediately.
Development Environment
I am one of those annoying people that can’t decide between Windows and Mac, so I use both. I will try to highlight any platform differences throughout the post, but on the whole the experience is pretty much the same regardless of operating system.
Getting started
To begin with, you need to install the following:
- Nodejs (I discovered some issues with the Lovell/Sharp package on the latest version, so install the LTS version (10.16.3 at the time of writing)
- Yarn - I used version 1.17.3
- Visual Studio Code
- git
Now that we’ve got the relevant packages and tools installed, we need to get going with Gatsby. I used a starter to site to get going. You can view and choose various starter sites online. I chose the awesome ‘Hello Starter’ kit by Panr.
Install Gatsby
Installing Gatsby is very straightforward. The following command installs the gatsby-cli
as a global tool (using the -g
flag):
npm install -g gatsby-cli
Now create a directory to hold your source code:
cd {base_dir}
mkdir web
For example, if my base directory is ~/dev/projects/my-site
then the command would be:
cd ~/dev/projects/my-site
mkdir web
Tip: For Windows users, replace
~/dev/
withc:\Dev
or equivalent
Assuming you’re using the ‘Hello’ starter kit, run the following:
gatsby new {directory} https://github.com/panr/gatsby-starter-hello-friend
For example, I want my directory structure to be:
- root/
- pipeline/ (build pipeline config)
- web/ (source code)
- readme.md (readme for the repo)
- .gitignore (exclude files/directories from my repo)
- azure-pipelines.yml (Azure DevOps pipeline definition)
- GitVersion.yml (more on this later)
As I want all the source code (i.e. the Gatsby site) to live in the web
directory, I run:
gatsby new web https://github.com/panr/gatsby-starter-hello-friend
This will clone the repo into the web
directory and the starter kit is ready to go.
Install site dependencies
Now we’re ready to install all the dependencies that come with the starter kit. From the web
directory, run the following command:
yarn install
This will download all relevant dependencies.
Run the site locally
Now we’re ready to run the site locally. Once the dependencies have been installed, run the following from the web
directory:
yarn run dev
This should complete successfully and your site will now be running locally at http://localhost:8000.
Tip: If you run into any problems at this stage, make sure you’re using the LTS version of node - I experienced issues with the latest node version. You can check the latest version with
node --version
.
At this point, your site should be running locally and we’re ready to start thinking about the build pipeline.
I’ll split this into several stages - the first thing we’ll look at is setting up a new pipeline, then we’ll discuss setting up a Netlify account, then we’ll add a new deployment stage to the pipeline to get the code deployed.
Azure DevOps
It seems to be a rare thing to not be a GitHub user these days, especially since free private repositories are now available.
However, I’m an Azure guy at heart and I use Azure DevOps extensively in my day job - I love the way it works, and I’m really into the fact that I can declare my pipeline in code and keep it with the source code. Of course this is possible in other platforms, but I like how it works in Azure DevOps. Other tools are available 😉
The pricing page for Azure DevOps provides the most up-to-date information, but you should be able to get a free account for a single private repository with up to 1,800 free build agent minutes per month using hosted build agents. You can add your own machine as a build agent if this isn’t enough, or if it isn’t suitable. I’ll cover this in a later post.
Once you’ve signed up and created your organisation, create a new repository to store your code.
Note: Your URL will be
https://dev.azure.com/{username}/{repository_name}
Initialize git repo
Now that you’ve created an empty repository, it’s time to initialise the git repo locally and get your changes pushed in.
From the root directory of your project, e.g. ~/dev/my-project
:
git init
This will create a new empty git repository. Before committing any of your changes to the repo, add a .gitignore
file so that you don’t commit unnecessary files to the repository. Again from the root directory:
touch .gitignore
This will create a new .gitignore
file.
If you’re on Windows, you can use PowerShell:
New-Item -Path .gitignore -ItemType File
Now we need to add some entries to the file to prevent us from checking in things like the node_modules
directory:
node_modules
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Runtime data
pids
*.pid
*.seed
*.pid.lock
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# nyc test coverage
.nyc_output
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# Bower dependency directory (https://bower.io/)
bower_components
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directories
node_modules/
jspm_packages/
# Typescript v1 declaration files
typings/
# Optional npm cache directory
.npm
# Optional eslint cache
.eslintcache
# Optional REPL history
.node_repl_history
# Output of 'npm pack'
*.tgz
# dotenv environment variables file
.env
# gatsby files
.cache/
public
# Mac files
.DS_Store
# Yarn
yarn-error.log
.pnp/
.pnp.js
# Yarn Integrity file
.yarn-integrity
Adding these entries will prevent items from being checked in.
Now make your initial commit to the repository:
git add .
git commit -m "Initial Commit"
These two lines will add all files to the repository (except those excluded by .gitignore
) and will then make an initial commit with the message “Initial Commit” (I’m not Mr. Imagination).
Add Azure DevOps git remote
Now we’re going to add the remote git repository details so that push and pull operations interact with our Azure DevOps account.
Note: You’ll need a valid SSH key to run these commands. Details of this can be found here: https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops.
git remote add origin [email protected]:v3/{organisation}/{project}
Note: To confirm your remote URL, check your repo in Azure DevOps and click on the ‘Clone’ link in the top right-hand corner. Switch to the
ssh
tab and copy the URL.
Now you can push to the remote:
git push -u origin --all
If you now check in the Azure DevOps portal you should be able to see your files.
Add pipeline details
Now we’re going to start defining the build pipeline.
Note: I have my own “funny” ways of doing things (as my friends and colleagues would happily tell you) so don’t assume I’m telling you the gospel method of defining pipelines. This is how I like to do it, because I think it keeps things nicely separated.
Create a new directory in the root of your project called pipeline
. This is going to hold individual .yaml
files that make up the pipeline:
mkdir pipeline
In the root of the project, create a new azure-pipelines.yml
file:
touch azure-pipelines.yml
Or, for Windows/PowerShell:
New-Item azure-pipelines.yml -ItemType File
Now we’re going to switch to VS Code and start building the pipeline. In the first stage of the pipeline we’re only going to produce the deployable artifacts; we’ll get to deploying to Netlify later.
Start with the following:
name: 'Site'
trigger:
branches:
include:
- master
- feature/*
exclude:
- develop/*
Here we’re specifying the name of the pipeline as it will appear in the Azure DevOps portal (‘Site’).
Then we are defining the triggers for the pipeline. The configuration I have is that only the master
branch and any feature/*
e.g. feature/my-post
branches will trigger the pipeline. I like to develop locally whilst still pushing to the remote repo (so I can easily switch devices and continue elsewhere). However, when I’m in the middle of something like writing a new post, I don’t want the pipeline to be triggered. Any incomplete work I will do on a branch prefixed with develop/
. It just helps me to keep organised.
Next, we’re going to add some variables. These allow us to define a value once and then re-use it elsewhere in the pipeline. There are other, more advanced uses for variables, but in this example I’m simply using them to keep the pipeline organised and simpler to update:
variables:
- name: node-version-spec
value: '11.x'
- name: yarn-version-spec
value: '1.x'
- name: vm-image
value: 'ubuntu-latest'
You’ll see these values in use shortly.
Next, we going to start building the first stage of our pipeline.
First pipeline stage
As I experiment more and more with Azure DevOps / Azure Pipelines, I find myself preferring the multi-stage pipelines to the previous incarnation of build pipelines and releases. I’m expecting new functionality to appear in the future, but for now the multi-stage pipelines provide exactly what we need.
The first stage will build our static site, and the second stage will deploy it. Read more about multi-stage pipelines at https://docs.microsoft.com/en-us/azure/devops/pipelines/process/stages?view=azure-devops&tabs=yaml.
The first stage is defined as follows:
stages:
- stage: Build
jobs:
- job: Prepare
pool:
vmImage: $(vm-image)
steps:
- template: pipeline/git-version.yml
- template: pipeline/node.yml
- template: pipeline/yarn.yml
- template: pipeline/copy-files.yml
- template: pipeline/publish.yml
That’s the entire stage defined, but note I am using templates so that I can keep the azure-pipelines.yml
file clean. Each of the - template: pipeline/{template}.yml
files contains the real details, and we’ll step through those now.
git-version.yml
I have my repository set up to be tagged with the build version when the build succeeds. This helps me to correlate a specific branch with a specific build. However, this is something I would normally use when I’m working in a team and multiple people are working in the same repository, or when I need an audit trail for what was deployed when. As this is just a demo for a blog, I’m going to skip over that stage for now. I may blog about this in the future.
node.yml
The node.yml
file has the following contents:
steps:
- task: NodeTool@0
displayName: 'Installing Node.js $(node-version-spec)'
inputs:
versionSpec: $(node-version-spec)
This command is part of the default set of tasks available in Azure DevOps and will ensure that node js is available on the build agent. The $(node-version-spec)
syntax references the variable that we defined at the start of the build definition.
yarn.yml
The yarn.yml
step involves a custom task that I installed into my Azure DevOps organisation. You can find it here: https://marketplace.visualstudio.com/items?itemName=geeklearningio.gl-vsts-tasks-yarn. Add it to your organisation, and then enter the following:
steps:
- task: YarnInstaller@3
displayName: 'Install Yarn $(yarn-version-spec)'
inputs:
versionSpec: $(yarn-version-spec)
checkLatest: true
- task: Yarn@3
displayName: 'Install dependencies'
inputs:
projectDirectory: 'web'
arguments: 'install'
productionMode: false
- task: Yarn@3
displayName: 'Build Site'
inputs:
projectDirectory: 'web'
arguments: 'run build'
productionMode: false
These steps will install the Yarn tool, install your site’s dependencies using Yarn, then build the site using Yarn. At the end of these stages, you should have a public
folder containing the static assets ready for deployment.
You can test these steps locally by simply running the following commands from the web
directory of your project:
yarn install
yarn run build
copy-files.yml
Now that we’ve ‘built’ the site, we want to ensure that the contents of the public
directory are copied into the artifacts staging directory, so that we can publish them in the next stage.
The contents of copy-files.yml
is as follows:
steps:
- task: CopyFiles@2
displayName: 'Copy Static Files'
inputs:
sourceFolder: '$(System.DefaultWorkingDirectory)'
contents: |
web/public/**/*
targetFolder: '$(Build.ArtifactStagingDirectory)'
overwrite: true
This task will copy all files from the web/public
directory (the globbing pattern **/*
basically means all files from all sub-directories) to the artifact staging directory. Anything in the staging directory will be published in the following step.
publish.yml
The publish.yml
file contains the steps to publish the contents of the web/public
folder so that they are available to subsequent stages.
steps:
- task: PublishBuildArtifacts@1
displayName: Publish
inputs:
pathtoPublish: '$(Build.ArtifactStagingDirectory)'
artifactName: 'drop'
This config will result in a compressed archive drop
being published. The pathtoPublish
(casing is correct according to the Azure DevOps task - I’d have preferred pathToPublish
but I need to pick my battles…).
Stage One Summary
I appreciate that we’ve covered a lot up to this point, so let’s summarise what we’ve done so far:
- Created a new git repo in Azure DevOps
- Pushed our default/starter site up to the remote repo
- Defined the following pipeline elements:
- Pipeline named
Site
- Included only
master
andfeature/*
branches - Defined some re-usable variables
- Installed node on the build agent
- Installed Yarn on the build agent
- Run
yarn install
to restore dependencies - Run
yarn run build
to build the Gatsby site - Copied the
/web/public
folder contents to the artifact staging directory - Published the artifact staging directory as a build artifact called
drop
- Pipeline named
Now we’re ready to get this site deployed. To do that, you’re going to need a Netlify account.
We’ll cover that process in part 2 (still to come!).