The aim of this blog is to give a high level introduction of Gitlab CI/CD
that help people get started in 30 minutes without spending much time on gitlab documents. This tutorial is geared toward beginners who wish to tinker with CI/CD tools like GitLab CI/CD. In this tutorial, I will briefly go over what is CI/CD, why I decided to go with GitLab's tool and a walkthrough on how to create a .gitlab-ci.yaml with an example application.
Some Background:
Successful DevOps adaption is more of a cultural changes than setting up a bunch of tools. The boundaries between Development and Operations are blurring day by day. Teams today are shipping code more frequently than before. This practice has also led to functional teams to be responsible for what happens after the code is shipped, hence they need to automate as much of the workflow as possible. All of this emphasizes the need for proper CI/CD setup.
Earlier, the developer would commit their code to SCM and the operations team would spend time on building the code and deploy it in different environments, at various levels - module, system integration, pre-prod, production to name a few. At each stage QA would need to test and would say, “ Hey, I have tested the build and is ready to be deployed to the next environment”. So basically for traditional development cycle with fixed release plans to a customer and in return get their feedback. But for a more agile team, this will be seen as a bottleneck. This gave rise to ‘DevOps’ and one of the key worklflow changes is CI/CD.
What is CI / CD?
Continuous integration and continuous delivery are two approaches to software development that are designed to improve code quality and enables rapid delivery and deployment of code. They are usually deployed together (CI/CD) to ensure the rapid overall delivery of new software features and fixes.
CI/CD is an approach which goes by many names but covers the same basic ideas. In short, they are two practices to be followed in order to write the code more quickly and safely to reach your users to ultimately get the value.
[caption id="attachment_9254" align="aligncenter" width="702"] CI/CD Work Flow[/caption]
The Differences:
CI is pretty straightforward and it stands for Continuous Integration, a practice that targets on making a release easier. When Developer merges their code from their working branch to the main branch, it is validated by creating a build and running automated tests on top of it.
Continuous Delivery is an extension of continuous integration to make sure that you can release changes in a substantial way. This means that on top of having an automated build and test, you can also have the release process automated and deploy the application at any point of time by clicking on a button.
Continuous Development is a process which is one step ahead of Continuous Delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There is no human intervention and only a failed test will prevent a new change to be deployed to production.
How do they relate to each other?
Continuous Integration is a part of both - Continuous deployment and Continuous Delivery. Continuous development is like Continuous delivery, except that releases happen without any human intervention.
Why I use Gitlab CI/CD?
I use Gitlab CI/CD for several reasons: First and foremost, we were already using Gitlab for SCM, and CI/CD is in the same place. I can create tickets, merge requests, push code, avoid creating webhooks and run pipelines without another application integrated to gitlab. I can use gitlab for the entire development cycle, it’s fast, provides monitoring etc.
To get started with Gitlab CI/CD, all you need is an application codebase hosted in a Git repository, and for your build, test, and deployment scripts to be specified in a file called .gitlab-ci.yaml, located in the root path of your repository.
Gitlab uses Runner to execute pipelines. GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. You need to install this before you get started with .gitlab-ci.yaml.
How do you install a Runner?
You can install the runner in your integration server, different environments or even in your local for understanding the concepts. The following are the steps to install a Runner.
Step 1: Simply download one of the binaries for your system:
sudo wget -O /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
Step 2: Give it permissions to execute:
sudo chmod +x /usr/local/bin/gitlab-runner
Step 3: Create a GitLab CI user:
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
Step 4: Install and run as service:
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start
Registering the Runner
Step 1: Run the following command:
sudo gitlab-runner register
Step 2: Enter your GitLab instance URL:
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com )
https://gitlab.company.com
Step 3: Enter the token you obtained to register the Runner:
Please enter the gitlab-ci token for this runner
Xxx (go to your project repository -> CI/CD settings -> Runner to get the token)
Step 4: Enter a description for the Runner, you can change this later in GitLab's UI:
Please enter the gitlab-ci description for this runner
[hostame] my-runner
Step 5: Enter the tags associated with the Runner, you can change this later in GitLab's UI:
Please enter the gitlab-ci tags for this runner (comma separated):
My-tag,another-tag ( this will be used to specify the runner in jobs)
Step 6: Enter the Runner executor:
Please enter the executor: ssh, docker+machine, docker-ssh+machine, kubernetes, docker, parallels, virtualbox, docker-ssh, shell:
Xxx ( I’ll be using shell for this tutorial)
References: https://docs.gitlab.com/runner/executors/
Step 7 (optional): Add the following into $visudo (only for shell executor)
gitlab-runner ALL=(ALL) NOPASSWD: ALL
Here’s a simple example of how to set up a pipeline for your angular application.
image: node:10
cache:
key: modules
untracked: true
paths:
- node_modules/
before_script:
- echo “Pipeline triggered”
after_script:
- service nginx restart
cache directive is used to specify a list of files and directories which should be cached between jobs. You can only use paths that are within the project workspace. node_modules from setup stage is cached so that the build stage can download the same.As I mentioned earlier, GitLab CI/CD uses Runners to execute pipelines. We can define which operating system and predefined libraries we would want our Runner to be based off by using the image directive. This is one of the features I like about GitLab CI/CD because you don’t need to install the libraries in environments.
before_script directive is used to define the command that should be run before all jobs, including deploy jobs, but after the restoration of artifacts(explained below). This can be an array or a multi-line string. You can run any commands that need to be run prior to your stages.
after_script directive is used to define the command that will be run after for all jobs, including failed ones. This has to be an array or a multi-line string. I will restart my web server using service nginx restart command after all the jobs are run.
stages:
- setup
- build
- deploy
stages directive is used to define stages that can be used by jobs and is defined globally. Here, the setup stage is used to install the npm packages, build stage to create the build and deploy stage to deploy the build in the designated environment.
setup_code:
stage: setup
script:
- rm ./package-lock.json
- npm install
artifacts:
untracked: true
paths:
- node_modules/
expire_in: 1 hour
tags:
- staging-runner
- production-runner
Now let’s start our job dedicated to the “setup” stage. We will call this job setup_code. In this job, In this job we want it to remove the package-lock.json and install dependencies. We can start this off with using the script directive. The script directive is a shell script that gets executed within the Runner. Then we are going to assign this job to the "setup" stage. To assign a job to a stage, use the stage directive. The artifact directive is used to specify the node_modules directory which should be attached to the job after success.
build_code:
stage: build
script:
- ng build --prod
dependencies:
- setup_code
tags:
- staging-runner
- production-runner
Now we have a job associated to the “setup” stage, we’ll now move to our next stage: “build”. stage directive is used to assign the job to build stage. Our build job is going to be called build_code and we are going to use angular-cli to run the ng build --prod command. The dependencies directive is used to download the artifact node_modules from the “setup_code” job.
deploy_staging:
stage: deploy
script:
- cp -r dist /var/www/html
only:
- staging
tags:
- staging-runner
deploy_production:
stage: deploy
script:
- cp -r dist /var/www/html
only:
- production
tags:
- production-runner
In our deploy_staging, the Runner will only execute if there was a change to the staging branch and for deploy_production the production branch. Here is a screenshot below, if the changes were made to production branch.Now finally we are going to add a job for our “deploy” stage: deploy_staging and deploy_production. In this instance, we are going to have two different jobs for deployment (staging and production). These jobs will reflect the same layout as our previous jobs but with a small change. Currently, all of our jobs are automatically set to be triggered on any code push or branch. We do not want to have that for when we deploy our code to staging and production. To prevent that from happening we use the only directive. The only directive defines the names of branches and tags for which the job will run.
From this image, as you can see, all three stages and jobs are triggered except the deploy_staging job since the code was pushed to production branch. Now, that’s interesting, ain’t it? Below is the final version of the .gitlab-ci.yaml file.
image: node:10
cache:
key: modules
untracked: true
paths:
- node_modules/
before_script:
- echo “Pipeline triggered”
after_script:
- service nginx restart
stages:
- setup
- build
- deploy
setup_code:
stage: setup
script:
- rm ./package-lock.json
- npm install
artifacts:
untracked: true
paths:
- node_modules/
expire_in: 1 hour
tags:
- staging-runner
- production-runner
build_code:
stage: build
script:
- ng build --prod
dependencies:
- setup_code
tags:
- staging-runner
- production-runner
deploy_staging:
stage: deploy
script:
- cp -r dist /var/www/html
only:
- staging
tags:
- staging-runner
deploy_production:
stage: deploy
script:
- cp -r dist /var/www/html
only:
- production
tags:
- production-runner
Conclusion:
The things covered above is an introduction to the features of GitLab CI/CD can offer. GitLab CI/CD has the ability to have a more in-depth control of the automation of codebases by building and publishing Docker images to integrating with third-party tools. You can find all the GitLab CI/CD yaml directives in this link.
In the next couple of articles, I’ll be discussing on how to get started with Jenkins pipelines and multibranch pipelines, netlify, AWS code pipeline.
Author - Shibily Shukoor (DevOps Engineer at Pacewisdom)
Made with Superblog