Day 28–101 Days of DevOps — Terraform Pipeline using Jenkins

Welcome to Day 28 of 101 Days of DevOps. The topic for today is building Terraform Pipeline using Jenkins.

On Day 13, you have learned how to automate the process of GitHub repository creation using Python

On Day24 you have learned about Jenkinsfile

On Day 25 and Day 26, you have learned the basics of terraform.

To view the complete course, please check the below url.

For more info, register via the below link

YouTube Channel link

Let’s combine all these concepts and build a terraform Jenkins pipeline that looks like this

What is the Pipeline?

A pipeline is typically referred to as a part of Continuous Integration, and it’s mostly used to merge developer changes into code mainline.

Why do we need a pipeline?

  1. Test changes before pushing to Production
  2. Another pair of eyes in terms of approval
  3. Logging to see who and when someone pushed the changes
  4. Separate workspace/tfstate for Development and Production Environment

How Jenkins Terraform Pipeline Works?

  • User pushes their code changes to GitHub.
  • Code change trigger a Git Webhooks, which triggers the terraform pipeline

Setting up Jenkins

  • Install Java
# yum -y install java-1.8.0-openjdk.x86_64
  • Download the latest version of Jenkins rpm
# wget
  • Install the Jenkins rpm
# rpm -ivh jenkins-2.164.1-1.1.noarch.rpmwarning: jenkins-2.164.1-1.1.noarch.rpm: Header V4 DSA/SHA1 Signature, key ID d50582e6: NOKEYPreparing...                          ################################# [100%]Updating / installing...1:jenkins-2.164.1-1.1              ################################# [100%]
  • Start Jenkins
# systemctl status jenkins jenkins.service - LSB: Jenkins Automation ServerLoaded: loaded (/etc/rc.d/init.d/jenkins; bad; vendor preset: disabled)Active: active (running) since Sat 2019-03-16 18:59:54 UTC; 5s agoDocs: man:systemd-sysv-generator(8)Process: 4241 ExecStart=/etc/rc.d/init.d/jenkins start (code=exited, status=0/SUCCESS)CGroup: /system.slice/jenkins.service└─4260 /etc/alternatives/java -Dcom.sun.akuma.Daemon=daemonized -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war --logfile=/var/log/jenkins/jenkins....Mar 16 18:59:53 systemd[1]: Starting LSB: Jenkins Automation Server...Mar 16 18:59:53 runuser[4246]: pam_unix(runuser:session): session opened for user jenkins by (uid=0)Mar 16 18:59:54 jenkins[4241]: Starting Jenkins [  OK  ]Mar 16 18:59:54 systemd[1]: Started LSB: Jenkins Automation Server.
  • Browse your Jenkins URL on port 8080; you will see something like this.
  • Copy and Paste the password(from below location)and then Select the installed Plugins
# cat /var/lib/jenkins/secrets/initialAdminPassword
  • Once Jenkins is up and running, click on Manage Jenkins and then Manage Plugins.
  • Search for Terraform under the Available tab, Select the Plugin and then Click on Install without restart
  • Go to Manage Jenkins but this Global Tool Configuration
* Name: This version must be consistent with the version of Jenkins we are going to refer in the Jenkins file
* Version: Use the same version
  • Again go back to the main Jenkins console, click on Credentials, and then global.
  • Add Credentials and then Secret text.
  • Here we want to add AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in Secret Text so that when Jenkins job update doesn’t show AWS Key and Secret in plain text.
  • This is how Jenkinsfile looks like
pipeline {
agent any
tools {
"org.jenkinsci.plugins.terraform.TerraformInstallation" "terraform"
parameters {
string(name: 'WORKSPACE', defaultValue: 'development', description:'setting up workspace for terraform')
environment {
TF_HOME = tool('terraform')
stages {
steps {
sh "terraform init -input=false"
sh "echo \$PWD"
sh "whoami"

steps {
sh "terraform fmt -list=true -write=false -diff=true -check=true"

steps {
sh "terraform validate"

steps {
script {
try {
sh "terraform workspace new ${params.WORKSPACE}"
} catch (err) {
sh "terraform workspace select ${params.WORKSPACE}"
sh "terraform plan -var 'access_key=$ACCESS_KEY' -var 'secret_key=$SECRET_KEY' \
-out terraform.tfplan;echo \$? > status"
stash name: "terraform-plan", includes: "terraform.tfplan"

steps {
def apply = false
try {
input message: 'Can you please confirm the apply', ok: 'Ready to Apply the Config'
apply = true
} catch (err) {
apply = false
currentBuild.result = 'UNSTABLE'
unstash "terraform-plan"
sh 'terraform apply terraform.tfplan'
* On the top of the pipeline, I am defining any agent that is available to run the pipeline
* Next step I am defining tools that is required to run the configuration and in this case terraform with version number that we setup during the global configuration.
* Next step we need some parameter that is required to run the pipeline, in this case workspace, Right now the only thing you need to know about workspace is that each workspace maintain seperate tfstate file, in this case I am putting development as default workspace
* Next step is to setting up some environment variable 
1: Where to find terraform
2: Setting up TF_IN_AUTOMATION = "true" will make terraform don't spit too much information
3: Access and Secret key to interact with AWS
  • Various Stages of Terraform Pipeline

Terraform init

The first step is to check in the code from the GitHub repo

* second step is setting input=false which means we if terraform need input and its not present fail the command
* some extra command for informational purpose

Terraform Format

  • In this case, I am checking the file checked in to the git is properly formatted if not simply failed at that point.

Terraform Validate

  • Check if the syntax of the .tf file is proper or not

Terraform Plan

  • Try to create a new workspace, and if it's already present, select it(default workspace is development)
  • Then I am running a simple plan command, providing access and secret key and storing the output in terraform.tfplan
  • The next step is I am stashing the output generated by the Previous command so that we can use it later

Terraform Apply

  • Here I am giving a choice to the user whether he wants to apply this change; if he chooses Ready to Apply the config, then terraform will run with the plan created earlier.


Creating Jenkins Pipeline

GitHub End

GitHub link:

Please check this video if you want a step-by-step explanation of the various concepts discussed above.

If you like to dig deeper into the AWS concept, please feel free to check my book.

Looking forward to you guys joining this journey and spend a minimum of an hour every day for the next 101 days on DevOps work and post your progress using any of the below mediums.



AWS Community Builder, Ex-Redhat, Author, Blogger, YouTuber, RHCA, RHCDS, RHCE, Docker Certified,4XAWS, CCNA, MCP, Certified Jenkins, Terraform Certified, 1XGCP

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Prashant Lakhera

AWS Community Builder, Ex-Redhat, Author, Blogger, YouTuber, RHCA, RHCDS, RHCE, Docker Certified,4XAWS, CCNA, MCP, Certified Jenkins, Terraform Certified, 1XGCP