terraform plan – Infrastructure as Code (IaC) with Terraform

To run a Terraform plan, use the following command:

$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions:

  • azurerm_resource_group.rg will be created + resource “azurerm_resource_group” “rg” {
 +id= (known after apply)
 +location= “westeurope”
 + name= “terraform-exercise”
}   
Plan:1to add, 0 to change, 0 to destroy.

Note: You didn’t use the -out option to save this plan, so Terraform can’t guarantee to take exactly these actions if you run terraform apply now.

The plan output tells us that if we run terraform apply immediately, it will create a single terraform_exercise resource group. It also outputs a note that since we did not save this plan, the subsequent application is not guaranteed to result in the same action. Meanwhile, things might have changed; therefore, Terraform will rerun plan and prompt us for yes when applying. Thus, you should save the plan to a file if you don’t want surprises.

Tip

Always save terraform plan output to a file and use the file to apply the changes. This is to avoid any last-minute surprises with things that might have changed in the background and apply not doing what it is intended to do, especially when your plan is reviewed as a part of your process.

So, let’s go ahead and save the plan to a file first using the following command:

$ terraform plan -out rg_terraform_exercise.tfplan

This time, the plan is saved to a file calledrg_terraform_exercise.tfplan. We can use this file to apply the changes subsequently.

terraform apply

To apply the changes using the plan file, run the following command:

$ terraform apply “rg_terraform_exercise.tfplan”

azurerm_resource_group.rg: Creating…

azurerm_resource_group.rg: Creation complete after 2s [id=/subscriptions/id/ resourceGroups/terraform-exercise]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

And that’s it! Terraform has applied the configuration. Let’s use the Azure CLI to verify whether the resource group is created.

Run the following command to list all resource groups within your subscription:

$ az group list

…

“id”: “/subscriptions/id/resourceGroups/terraform-exercise”,

“location”: “westeurope”,

“name”: “terraform-exercise”,

…

We see that our resource group is created and within the list.

There might be instances when apply is partially successful. In that case, Terraform will automatically taint resources it believes weren’t created successfully. Such resources will be recreated automatically in the next run. If you want to taint a resource for recreation manually, you can use the terraform taint command:

$ terraform taint <resource>

Suppose we want to destroy the resource group as we no longer need it. We can use terraform destroy for that.

2024-08-31 by Building the Python Flask app Cloud Certifications Deleting an ECS service Exams of Cloud Knative architecture 0

terraform init – Infrastructure as Code (IaC) with Terraform

To initialize a Terraform workspace, run the following command:

$ terraform init
Initializing the backend…
Initializing provider plugins…

  • Finding hashicorp/azurerm versions matching “3.63.0”…
  • Installing hashicorp/azurerm v3.63.0…
  • Installed hashicorp/azurerm v3.63.0 (signed by HashiCorp)

Terraform has created a lock file, .terraform.lock.hcl, to record the provider selections it made previously. Include this file in your version control repository so that Terraform can guarantee to make the same selections by default when you run terraform init in the future.

Terraform has been successfully initialized!

As the Terraform workspace has been initialized, we can create an Azure resource group to start working with the cloud.

Creating the first resource – Azure resource group

We must use the azurerm_resource_group resource within the main.tf file to create an

Azure resource group. Add the following to your main.tf file to do so:

resource “azurerm_resource_group” “rg” {
name         = var.rg_name
location = var.rg_location
}

As we’ve used two variables, we’ve got to declare those, so add the following to the vars.tf file:

variable “rg_name” {
type              = string
description = “The resource group name”
}
variable “rg_location” {
type              = string
description = “The resource group location”
}

Then, we need to add the resource group name and location to the terraform.tfvars file.

Therefore, add the following to the terraform.tfvars file:

rg_name=terraform-exercise
rg_location=”West Europe”

So, now we’re ready to run a plan, but before we do so, let’s use terraform fmt to format our files into the canonical standard.

terraform fmt

The terraform fmt command formats the .tf files into a canonical standard. Use the following command to format your files:

$ terraform fmt
terraform.tfvars
vars.tf

The command lists the files that it formatted. The next step is to validate your configuration.

terraform validate

The terraform validate command validates the current configuration and checks whether there are any syntax errors. To validate your configuration, run the following:

$ terraform validate
Success! The configuration is valid.

The success output denotes that our configuration is valid. If there were any errors, it would have highlighted them in the validated output.

Tip

Always run fmt and validate before every Terraform plan. It saves you a ton of planning time and helps you keep your configuration in good shape.

As the configuration is valid, we are ready to run a plan.

2024-07-02 by Cloud Certifications Deleting an ECS service Exams of Cloud Knative architecture Load testing your app on Knative 0

Providing variable values – Infrastructure as Code (IaC) with Terraform

There are a few ways to provide variable values within Terraform:

  • Via the console using -var flags: You can use multiple -var flags with the variable_ name=variable_value string to supply the values.
  • Via a variable definition file (the .tfvars file): You can use a file containing the list of variables and their values ending with an extension of .tfvars (if you prefer HCL) or .tfvars. json (if you prefer JSON) via the command line with the -var-file flag.
  • Via default variable definition files: If you don’t want to supply the variable definition file via the command line, you can create a file with the name terraform.tfvars or end it with an extension of .auto.tfvars within the Terraform workspace. Terraform will automatically scan these files and take the values from there.
  • Environment variables: If you don’t want to use a file or pass the information via the command line, you can use environment variables to supply it. You must create environment variables with the TF_VAR_<var-name> structure containing the variable value.
  • Default: When you run a Terraform plan without providing values to variables in any other way, the Terraform CLI will prompt for the values, and you must manually enter them.

If multiple methods are used to provide the same variable’s value, the first method in the preceding list has the highest precedence for a specific variable. It overrides anything that is defined in the methods listed later.

We will use the terraform.tfvars file for this activity and provide the values for the variables.

Add the following data to the terraform.tfvars file:

subscription_id = “<SUBSCRIPTION_ID>”

app_id= “<SERVICE_PRINCIPAL_APP_ID>”
password=“<SERVICE_PRINCIPAL_PASSWORD>”
tenant=“<TENANT_ID>”

If you are checking the Terraform configuration into source control, add the file to the ignore list to avoid accidentally checking it in.

If you use Git, adding the following to the .gitignore file will suffice:

*.tfvars

.terraform*

Now, let’s go ahead and look at the Terraform workflow to progress further.

Terraform workflow

The Terraform workflow typically consists of the following:

  • init: Initializes the Terraform workspace and backend (more on them later) and downloads all required providers. You can run the init command multiple times during your build, as it does not change your workspace or state.
  • plan: It runs a speculative plan on the requested resources. This command typically connects with the cloud provider and then checks whether the objects managed by Terraform exist within the cloud provider and whether they have the same configuration as defined in the Terraform template. It then shows the delta in the plan output that an admin can review and change the

configuration if unsatisfied. If satisfied, they can apply the plan to commit the changes to the cloud platform. The plan command does not make any changes to the current infrastructure.

  • apply: This applies the delta configuration to the cloud platform. When you useapply by itself, it runs the plan command first and asks for confirmation. If you supply a plan to it, it applies the plan directly. You can also use apply without running the plan using the -auto-approve flag.
  • destroy: The destroy command destroys the entire infrastructure Terraform manages. It is, therefore, not a very popular command and is rarely used in a production environment. That does not mean that the destroy command is not helpful. Suppose you are spinning up a development infrastructure for temporary purposes and don’t need it later. In that case, destroying everything you created using this command takes a few minutes.

To access the resources for this section, cd into the following:

$ cd ~/modern-devops/ch8/terraform-exercise Now, let’s look at these in detail with hands-on exercises.

2024-06-03 by Deleting an ECS service Exams of Cloud Knative architecture Load testing your app on Knative Providing variable values 0

Terraform variables – Infrastructure as Code (IaC) with Terraform

To declare variables, we will need to create a vars.tf file with the following data:

variable “subscription_id” {
type              = string
description = “The azure subscription id”
}
variable “app_id” {
type              = string
description = “The azure service principal appId”
}
variable “password” {
type              = string
description = “The azure service principal password”
sensitive     = true
}
variable “tenant” {
type              = string
description = “The azure tenant id”
}

So, we’ve defined four variables here using variable blocks. Variable blocks typically have a type and a description. The type attribute defines the data type of the variable we declare and defaults to the string data type. It can be a primitive data type such as string , number, or bool, or a complex data structure such as list, set, map, object, or tuple. We will look at types in detail when we use them later in the exercises. The description attribute provides more information regarding the variable so users can refer to it for better understanding.

Tip

Always set the description attribute right from the beginning, as it is user-friendly and promotes the reuse of your template.

The client_secret variable also contains a third attribute called sensitive, a Boolean attribute set to true. When the sensitive attribute is true, the Terraform CLI does not display it in the screen’s output. This attribute is highly recommended for sensitive variables such as passwords and secrets.

Tip

Always declare a sensitive variable as sensitive. This is because if you use Terraform within your CI/CD pipelines, unprivileged users might access sensitive information by looking at the logs.

Apart from the other three, an attribute called default will help you specify default variable values. The default values help you provide the best possible value for a variable, which your users can override if necessary.

Tip

Always use default values where possible, as they allow you to provide users with soft guidance about your enterprise standard and save them time.

The next step would be to provide variable values. Let’s have a look at that.

2024-05-08 by Exams of Cloud Knative architecture Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Using the Azure Terraform provider – Infrastructure as Code (IaC) with Terraform

Before we define the Azure Terraform provider, let’s understand what makes a Terraform root module. The Terraform root module is just a working directory within your filesystem containing one or more .tf files that help you define your configuration and are where you would typically run your Terraform commands.

Terraform scans all your .tf files, combines them, and processes them internally as one. Therefore, you can have one or more .tf files that you can split according to your needs. While there are no defined standards for naming .tf files, most conventions use main.tf as the main Terraform file where they define resources, a vars.tf file for defining variables, and outputs.tf for defining outputs.

For this discussion, let’s create a main.tf file within our working directory and add a provider configuration like the following:

terraform {
required_providers {
azurerm = {
source  = “azurerm”
version = “=3.55.0”
}
}
}
provider “azurerm” {
subscription_id = var.subscription_id
client_id            = var.client_id
client_secret     = var.client_secret
tenant_id            = var.tenant_id
features {}
}

The preceding file contains two blocks. The terraform block contains the required_providers block, which declares the version constraint for the azurerm provider. The provider block

declares an azurerm provider, which requires four parameters.

Tip

Always constrain the provider version, as providers are released without notice, and if you don’t include the version number, something that works on your machine might not work on someone else’s machine or the CI/CD pipeline. Using a version constraint avoids breaking changes and keeps you in control.

You might have noticed that we have declared several variables within the preceding file instead of

directly inputting the values. There are two main reasons for that – we want to make our template as generic as possible to promote reuse. So, suppose we want to apply a similar configuration in another subscription or use another service principal; we should be able to change it by changing the variable values. Secondly, we don’t want to check client_id and client_secret in source control. It

is a bad practice as we expose our service principal to users beyond those who need to know about it.

Tip

Never store sensitive data in source control. Instead, use a tfvars file to manage sensitive information and keep it in a secret management system such as HashiCorp’s Vault.

Okay, so as we’ve defined the provider resource and the attribute values are sourced from variables, the next step would be to declare variables. Let’s have a look at that now.

2024-04-03 by Building the Python Flask app Exams of Cloud Knative architecture Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Authentication and authorization with Azure – Infrastructure as Code (IaC) with Terraform

The simplest way to authenticate and authorize with Azure is to log in to your account using the Azure CLI. When you use the Azure provider within your Terraform file, it will automatically act as your account and do whatever it needs to. Now, this sounds dangerous. Admins generally have a lot of access, and having a tool that acts as an admin might not be a great idea. What if you want to plug Terraform into your CI/CD pipelines? Well, there is another way to do it – by using Azure service principals. Azure service principals allow you to access the required features without using a nameduser account. You can then apply the principle of least privilege to the service principal and provide only the necessary access.

Before configuring the service principal, let’s install the Azure CLI on our machine. To do so, run the following command:

$ curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

The preceding command will download a shell script and execute it using bash. The script will then automatically download and configure the Azure CLI. To confirm whether the Azure CLI is installed successfully, run the following command:

$ az –version 
azure-cli2.49.0

We see that the Azure CLI is correctly installed on the system. Now, let’s go ahead and configure the service principal.

To configure the Azure service principal, follow these steps.

Log in to Azure using the following command and follow all the steps the command prompts. You must browse to a specified URL and enter the given code. Once you’ve logged in, you will get a JSON response that will include some details, something like the following:

$ az login

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter

the code XXXXXXXXX to authenticate:

[

{

“id”: “00000000-0000-0000-0000-0000000000000”,

…

}

]

Make a note of the id attribute, which is the subscription ID, and if you have more than one subscription, you can use the following to set it to the correct one:

$ export SUBSCRIPTION_ID=”<SUBSCRIPTION_ID>”

$ az account set –subscription=”$SUBSCRIPTION_ID”

Use the following command to create a service principal with the contributor role to allow Terraform to manage the subscription’s infrastructure.

Tip

Follow the principle of least privilege while granting access to the service principal. Do not give privileges thinking you might need them in the future. If any future access is required, you can grant it later.

We use contributor access for simplicity, but finer-grained access is possible and should be used:

$ az ad sp create-for-rbac –role=”Contributor” \

–scopes=”/subscriptions/$SUBSCRIPTION_ID”

Creating ‘Contributor’ role assignment under scope ‘/subscriptions/<SUBSCRIPTION_ID>’ The output includes credentials that you must protect. Ensure you do not include these credentials in your code or check the credentials into your source control (for more information, see https://aka.ms/azadsp-cli): {

“appId”: “00000000-0000-0000-0000-0000000000000”,

“displayName”: “azure-cli-2023-07-02-09-13-40”,

“password”: “00000000000.xx-00000000000000000”,

“tenant”: “00000000-0000-0000-0000-0000000000000”

}

We’ve successfully created the service principal. The response JSON consists of appId, password, and tenant. We will need these to configure Terraform to use the service principal. In the next section, let’s define the Azure Terraform provider with the details.

2024-03-16 by Building the Python Flask app Cloud Certifications Exams of Cloud Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Installing Terraform – Infrastructure as Code (IaC) with Terraform

Installing Terraform is simple; go to https://www.terraform.io/downloads.html and follow the instructions for your platform. Most of it will require you to download a binary and move it to your system path.

Since we’ve been using Ubuntu throughout this book, I will show the installation on Ubuntu. Use the following commands to use the apt package manager to install Terraform:

$ wget -O- https://apt.releases.hashicorp.com/gpg | \

sudo gpg –dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

$ echo “deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \ https://apt.releases.hashicorp.com $(lsb_release -cs) main” | \ sudo tee /etc/apt/sources.list.d/hashicorp.list

$ sudo apt update && sudo apt install terraform

Check whether Terraform has been installed successfully with the following command:

$ terraform version

Terraform v1.5.2

It shows that Terraform has been installed successfully. Terraform uses Terraform providers to interact with cloud providers, so let’s look at those in the next section.

Terraform providers

Terraform has a decentralized architecture. While the Terraform CLI contains Terraform’s core functionality and provides all functionalities not related to any specific cloud provider, Terraform providers provide the interface between the Terraform CLI and the cloud providers themselves. This decentralized approach has allowed public cloud vendors to offer their Terraform providers so that their customers can use Terraform to manage infrastructure in their cloud. Such is Terraform’s popularity that it has now become an essential requirement for every public cloud provider to offer a Terraform provider.

We will interact with Azure for this chapter’s entirety and use the Azure Terraform provider for our activity.

To access the resources for this section, cd into the following:

$ cd ~/modern-devops/ch8/terraform-exercise/

Before we go ahead and configure the provider, we need to understand how Terraform needs to authenticate and authorize with the Azure APIs.

2024-02-20 by Building the Python Flask app Cloud Certifications Deleting an ECS service Exams of Cloud Providing variable values Using the Azure Terraform provider 0

Introduction to IaC – Infrastructure as Code (IaC) with Terraform

IaC is the concept of using code to define infrastructure. While most people can visualize infrastructure as something tangible, virtual infrastructure is already commonplace and has existed for around two decades. Cloud providers provide a web-based console through which you can manage your infrastructure intuitively. But the process is not repeatable or recorded.

If you spin up a set of infrastructure components using the console in one environment and want to replicate it in another, it is a duplication of effort. To solve this problem, cloud platforms provide APIs to manipulate resources within the cloud and some command-line tools that can help trigger the APIs. You can start writing scripts using commands to create the infrastructure and parameterize them to use the same scripts in another environment. Well, that solves the problem, right?

Not really! Writing scripts is an imperative way of managing infrastructure. Though you can still call it IaC, its problem is that it does not effectively manage infrastructure changes. Let me give you a few examples:

  • What would happen if you needed to modify something already in the script? Changing the script somewhere in the middle and rerunning the entire thing may create havoc with your infrastructure. Imperative management of infrastructure is not idempotent. So, managing changes becomes a problem.
  • What if someone manually changes the script-managed infrastructure using the console? Will your script be able to detect it correctly? What if you want to change the same thing using a script? It will soon start to get messy.
  • With the advent of hybrid cloud architecture, most organizations use multiple cloud platforms for their needs. When you are in such a situation, managing multiple clouds with imperative scripts soon becomes a problem. Different clouds have different ways of interacting with their APIs and have distinct command-line tools.

The solution to all these problems is a declarative IaC solution such as Terraform. HashiCorp’s Terraform is the most popular IaC tool available on the market. It helps you automate and manage your infrastructure using code and can run on various platforms. As it is declarative, you just need to define what you need (the desired end state) instead of describing how to achieve it. It has the following features:

  • It supports multiple cloud platforms via providers and exposes a single declarative HashiCorp Configuration Language (HCL)-based interface to interact with it. Therefore, it allows you to manage various cloud platforms using a similar language and syntax. So, having a few Terraform experts within your team can handle all your IaC needs.
  • It tracks the state of the resources it manages using state files and supports local and remote backends to store and manage them. That helps in making the Terraform configuration idempotent. So, if someone manually changes a Terraform-managed resource, Terraform can detect the difference in the next run and prompt corrective action to bring it to the defined configuration. The admin can then absorb the change or resolve any conflicts before applying it.
  • It enables GitOps in infrastructure management. With Terraform, you can have the infrastructure configuration alongside application code, making versioning, managing, and releasing infrastructure the same as managing code. You can also include code scanning and gating using pull requests so that someone can review and approve the changes to higher environments before you apply them. A great power indeed!

Terraform has multiple offerings – open source, cloud , and enterprise. The open source offering is a simple command- line interface (CLI)-based tool that you can download on any supported operating system (OS) and use. The cloud and enterprise offerings are more of a wrapper on top of the open source one. They provide a web-based GUI and advanced features such as policy as code with Sentinel, cost analysis, private modules, GitOps, and CI/CD pipelines.

This chapter will discuss the open source offering and its core functions.

Terraform open source is divided into two main parts – Terraform Core and Terraform providers, as seen in the following diagram:

Figure 8.1 – Terraform architecture

Let’s look at the functions of both components:

  • Terraform Core is the CLI that we will use to interact with Terraform. It takes two main inputs – your Terraform configuration files and the existing state. It then takes the difference in configuration and applies it.
  • Terraform providers are plugins that Terraform uses to interact with cloud providers. The providers translate the Terraform configuration into the respective cloud’s REST API calls so that Terraform can manage its associated infrastructure. For example, if you want Terraform to manage AWS infrastructure, you must use the Terraform AWS provider.

Now let’s see how we can install open source Terraform.

2024-01-16 by Building the Python Flask app Cloud Certifications Deleting an ECS service Exams of Cloud Using the Azure Terraform provider 0

Technical requirements – Infrastructure as Code (IaC) with Terraform

Cloud computing is one of the primary factors of DevOps enablement today. The initial apprehensions about the cloud are a thing of the past. With an army of security and compliance experts manning cloud platforms 24×7, organizations are now trusting the public cloud like never before. Along with cloud computing, another buzzword has taken the industry by storm – Infrastructure as Code (IaC). This chapter will focus on IaC withTerraform, and by the end of this chapter, you will understand the concept and have enough hands-on experience with Terraform to get you started on your journey.

In this chapter, we’re going to cover the following main topics:

  • Introduction to IaC
  • Setting up Terraform and Azure providers
  • Understanding Terraform workflows and creating your first resource using Terraform
  • Terraform modules
  • Terraform state and backends
  • Terraform workspaces
  • Terraform outputs, state, console, and graphs

Technical requirements

For this chapter, you can use any machine to run Terraform. Terraform supports many platforms, including Windows, Linux, and macOS.

You will need an active Azure subscription to follow the exercises. Currently, Azure is offering a free trial for 30 days with $200 worth of free credits; you can sign up at https://azure.microsoft. com/en-in/free.

You will also need to clone the following GitHub repository for some of the exercises: https://github.com/PacktPublishing/Modern-DevOps-Practices-2e

Run the following command to clone the repository into your home directory, and cd into the ch8 directory to access the required resources:

$ git clone https://github.com/PacktPublishing/Modern-DevOps-Practices-2e.git \ modern-devops

$ cd modern-devops/ch8

So, let’s get started!

2023-12-21 by Building the Python Flask app Cloud Certifications Exams of Cloud Knative architecture 0

Load testing your app on Knative – Containers as a Service (CaaS) and Serverless Computing for Containers

We will use the hey utility to perform load testing. Since your application has already been deployed, run the following command to do the load test:

$ hey -z 30s -c 500 http://py-time.default.35.226.198.46.sslip.io

Once the command has executed, run the following command to get the currently running instances of the py-time pods:

$ kubectl get pod    
NAMEREADY STATUS RESTARTSAGE
py-time-00001-deployment-52vjv 2/2Running044s
py-time-00001-deployment-bhhvm 2/2Running044s
py-time-00001-deployment-h6qr5 2/2Running042s
py-time-00001-deployment-h92jp 2/2Running040s
py-time-00001-deployment-p27gl 2/2Running088s
py-time-00001-deployment-tdwrh 2/2Running038s
py-time-00001-deployment-zsgcg 2/2Running042s

As we can see, Knative has created seven instances of the py-time pod. This is horizontal autoscaling in action.

Now, let’s look at the cluster nodes by running the following command:

$ kubectl get nodes

NAME STATUS AGE gke-cluster-1-default-pool-353b3ed4-js71 Ready 3m17s gke-cluster-1-default-pool-353b3ed4-mx83 Ready 106m gke-cluster-1-default-pool-353b3ed4-vf7q Ready 106m

As we can see, GKE has created another node in the node pool because of the extra burst of traffic it received. This is phenomenal, as we have the Kubernetes API to do what we want. We have automatically horizontally autoscaled our pods. We have also automatically horizontally autoscaled our cluster worker nodes. This means we have a fully automated solution for runningcontainers without worrying about the management nuances! That is open source serverless in action for you!

Summary

This chapter covered CaaS and serverless CaaS services. These help us manage container applications with ease without worrying about the underlying infrastructure and managing them. We used Amazon’s ECS as an example and deep-dived into it. Then, we briefly discussed other solutions that are available on the market.

Finally, we looked at Knative, an open source serverless solution for containers that run on top of Kubernetes and use many other open source CNCF projects.

In the next chapter, we will delve into IaC with Terraform.

2023-11-05 by Deleting an ECS service Exams of Cloud Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0