Using the Azure Terraform provider

Terraform variables – Infrastructure as Code (IaC) with Terraform

To declare variables, we will need to create a vars.tf file with the following data:

variable “subscription_id” {
type              = string
description = “The azure subscription id”
}
variable “app_id” {
type              = string
description = “The azure service principal appId”
}
variable “password” {
type              = string
description = “The azure service principal password”
sensitive     = true
}
variable “tenant” {
type              = string
description = “The azure tenant id”
}

So, we’ve defined four variables here using variable blocks. Variable blocks typically have a type and a description. The type attribute defines the data type of the variable we declare and defaults to the string data type. It can be a primitive data type such as string , number, or bool, or a complex data structure such as list, set, map, object, or tuple. We will look at types in detail when we use them later in the exercises. The description attribute provides more information regarding the variable so users can refer to it for better understanding.

Tip

Always set the description attribute right from the beginning, as it is user-friendly and promotes the reuse of your template.

The client_secret variable also contains a third attribute called sensitive, a Boolean attribute set to true. When the sensitive attribute is true, the Terraform CLI does not display it in the screen’s output. This attribute is highly recommended for sensitive variables such as passwords and secrets.

Tip

Always declare a sensitive variable as sensitive. This is because if you use Terraform within your CI/CD pipelines, unprivileged users might access sensitive information by looking at the logs.

Apart from the other three, an attribute called default will help you specify default variable values. The default values help you provide the best possible value for a variable, which your users can override if necessary.

Tip

Always use default values where possible, as they allow you to provide users with soft guidance about your enterprise standard and save them time.

The next step would be to provide variable values. Let’s have a look at that.

2024-05-08 by Exams of Cloud Knative architecture Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Using the Azure Terraform provider – Infrastructure as Code (IaC) with Terraform

Before we define the Azure Terraform provider, let’s understand what makes a Terraform root module. The Terraform root module is just a working directory within your filesystem containing one or more .tf files that help you define your configuration and are where you would typically run your Terraform commands.

Terraform scans all your .tf files, combines them, and processes them internally as one. Therefore, you can have one or more .tf files that you can split according to your needs. While there are no defined standards for naming .tf files, most conventions use main.tf as the main Terraform file where they define resources, a vars.tf file for defining variables, and outputs.tf for defining outputs.

For this discussion, let’s create a main.tf file within our working directory and add a provider configuration like the following:

terraform {
required_providers {
azurerm = {
source  = “azurerm”
version = “=3.55.0”
}
}
}
provider “azurerm” {
subscription_id = var.subscription_id
client_id            = var.client_id
client_secret     = var.client_secret
tenant_id            = var.tenant_id
features {}
}

The preceding file contains two blocks. The terraform block contains the required_providers block, which declares the version constraint for the azurerm provider. The provider block

declares an azurerm provider, which requires four parameters.

Tip

Always constrain the provider version, as providers are released without notice, and if you don’t include the version number, something that works on your machine might not work on someone else’s machine or the CI/CD pipeline. Using a version constraint avoids breaking changes and keeps you in control.

You might have noticed that we have declared several variables within the preceding file instead of

directly inputting the values. There are two main reasons for that – we want to make our template as generic as possible to promote reuse. So, suppose we want to apply a similar configuration in another subscription or use another service principal; we should be able to change it by changing the variable values. Secondly, we don’t want to check client_id and client_secret in source control. It

is a bad practice as we expose our service principal to users beyond those who need to know about it.

Tip

Never store sensitive data in source control. Instead, use a tfvars file to manage sensitive information and keep it in a secret management system such as HashiCorp’s Vault.

Okay, so as we’ve defined the provider resource and the attribute values are sourced from variables, the next step would be to declare variables. Let’s have a look at that now.

2024-04-03 by Building the Python Flask app Exams of Cloud Knative architecture Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Authentication and authorization with Azure – Infrastructure as Code (IaC) with Terraform

The simplest way to authenticate and authorize with Azure is to log in to your account using the Azure CLI. When you use the Azure provider within your Terraform file, it will automatically act as your account and do whatever it needs to. Now, this sounds dangerous. Admins generally have a lot of access, and having a tool that acts as an admin might not be a great idea. What if you want to plug Terraform into your CI/CD pipelines? Well, there is another way to do it – by using Azure service principals. Azure service principals allow you to access the required features without using a nameduser account. You can then apply the principle of least privilege to the service principal and provide only the necessary access.

Before configuring the service principal, let’s install the Azure CLI on our machine. To do so, run the following command:

$ curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

The preceding command will download a shell script and execute it using bash. The script will then automatically download and configure the Azure CLI. To confirm whether the Azure CLI is installed successfully, run the following command:

$ az –version 
azure-cli2.49.0

We see that the Azure CLI is correctly installed on the system. Now, let’s go ahead and configure the service principal.

To configure the Azure service principal, follow these steps.

Log in to Azure using the following command and follow all the steps the command prompts. You must browse to a specified URL and enter the given code. Once you’ve logged in, you will get a JSON response that will include some details, something like the following:

$ az login

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter

the code XXXXXXXXX to authenticate:

[

{

“id”: “00000000-0000-0000-0000-0000000000000”,

…

}

]

Make a note of the id attribute, which is the subscription ID, and if you have more than one subscription, you can use the following to set it to the correct one:

$ export SUBSCRIPTION_ID=”<SUBSCRIPTION_ID>”

$ az account set –subscription=”$SUBSCRIPTION_ID”

Use the following command to create a service principal with the contributor role to allow Terraform to manage the subscription’s infrastructure.

Tip

Follow the principle of least privilege while granting access to the service principal. Do not give privileges thinking you might need them in the future. If any future access is required, you can grant it later.

We use contributor access for simplicity, but finer-grained access is possible and should be used:

$ az ad sp create-for-rbac –role=”Contributor” \

–scopes=”/subscriptions/$SUBSCRIPTION_ID”

Creating ‘Contributor’ role assignment under scope ‘/subscriptions/<SUBSCRIPTION_ID>’ The output includes credentials that you must protect. Ensure you do not include these credentials in your code or check the credentials into your source control (for more information, see https://aka.ms/azadsp-cli): {

“appId”: “00000000-0000-0000-0000-0000000000000”,

“displayName”: “azure-cli-2023-07-02-09-13-40”,

“password”: “00000000000.xx-00000000000000000”,

“tenant”: “00000000-0000-0000-0000-0000000000000”

}

We’ve successfully created the service principal. The response JSON consists of appId, password, and tenant. We will need these to configure Terraform to use the service principal. In the next section, let’s define the Azure Terraform provider with the details.

2024-03-16 by Building the Python Flask app Cloud Certifications Exams of Cloud Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Installing Terraform – Infrastructure as Code (IaC) with Terraform

Installing Terraform is simple; go to https://www.terraform.io/downloads.html and follow the instructions for your platform. Most of it will require you to download a binary and move it to your system path.

Since we’ve been using Ubuntu throughout this book, I will show the installation on Ubuntu. Use the following commands to use the apt package manager to install Terraform:

$ wget -O- https://apt.releases.hashicorp.com/gpg | \

sudo gpg –dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

$ echo “deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \ https://apt.releases.hashicorp.com $(lsb_release -cs) main” | \ sudo tee /etc/apt/sources.list.d/hashicorp.list

$ sudo apt update && sudo apt install terraform

Check whether Terraform has been installed successfully with the following command:

$ terraform version

Terraform v1.5.2

It shows that Terraform has been installed successfully. Terraform uses Terraform providers to interact with cloud providers, so let’s look at those in the next section.

Terraform providers

Terraform has a decentralized architecture. While the Terraform CLI contains Terraform’s core functionality and provides all functionalities not related to any specific cloud provider, Terraform providers provide the interface between the Terraform CLI and the cloud providers themselves. This decentralized approach has allowed public cloud vendors to offer their Terraform providers so that their customers can use Terraform to manage infrastructure in their cloud. Such is Terraform’s popularity that it has now become an essential requirement for every public cloud provider to offer a Terraform provider.

We will interact with Azure for this chapter’s entirety and use the Azure Terraform provider for our activity.

To access the resources for this section, cd into the following:

$ cd ~/modern-devops/ch8/terraform-exercise/

Before we go ahead and configure the provider, we need to understand how Terraform needs to authenticate and authorize with the Azure APIs.

2024-02-20 by Building the Python Flask app Cloud Certifications Deleting an ECS service Exams of Cloud Providing variable values Using the Azure Terraform provider 0

Introduction to IaC – Infrastructure as Code (IaC) with Terraform

IaC is the concept of using code to define infrastructure. While most people can visualize infrastructure as something tangible, virtual infrastructure is already commonplace and has existed for around two decades. Cloud providers provide a web-based console through which you can manage your infrastructure intuitively. But the process is not repeatable or recorded.

If you spin up a set of infrastructure components using the console in one environment and want to replicate it in another, it is a duplication of effort. To solve this problem, cloud platforms provide APIs to manipulate resources within the cloud and some command-line tools that can help trigger the APIs. You can start writing scripts using commands to create the infrastructure and parameterize them to use the same scripts in another environment. Well, that solves the problem, right?

Not really! Writing scripts is an imperative way of managing infrastructure. Though you can still call it IaC, its problem is that it does not effectively manage infrastructure changes. Let me give you a few examples:

  • What would happen if you needed to modify something already in the script? Changing the script somewhere in the middle and rerunning the entire thing may create havoc with your infrastructure. Imperative management of infrastructure is not idempotent. So, managing changes becomes a problem.
  • What if someone manually changes the script-managed infrastructure using the console? Will your script be able to detect it correctly? What if you want to change the same thing using a script? It will soon start to get messy.
  • With the advent of hybrid cloud architecture, most organizations use multiple cloud platforms for their needs. When you are in such a situation, managing multiple clouds with imperative scripts soon becomes a problem. Different clouds have different ways of interacting with their APIs and have distinct command-line tools.

The solution to all these problems is a declarative IaC solution such as Terraform. HashiCorp’s Terraform is the most popular IaC tool available on the market. It helps you automate and manage your infrastructure using code and can run on various platforms. As it is declarative, you just need to define what you need (the desired end state) instead of describing how to achieve it. It has the following features:

  • It supports multiple cloud platforms via providers and exposes a single declarative HashiCorp Configuration Language (HCL)-based interface to interact with it. Therefore, it allows you to manage various cloud platforms using a similar language and syntax. So, having a few Terraform experts within your team can handle all your IaC needs.
  • It tracks the state of the resources it manages using state files and supports local and remote backends to store and manage them. That helps in making the Terraform configuration idempotent. So, if someone manually changes a Terraform-managed resource, Terraform can detect the difference in the next run and prompt corrective action to bring it to the defined configuration. The admin can then absorb the change or resolve any conflicts before applying it.
  • It enables GitOps in infrastructure management. With Terraform, you can have the infrastructure configuration alongside application code, making versioning, managing, and releasing infrastructure the same as managing code. You can also include code scanning and gating using pull requests so that someone can review and approve the changes to higher environments before you apply them. A great power indeed!

Terraform has multiple offerings – open source, cloud , and enterprise. The open source offering is a simple command- line interface (CLI)-based tool that you can download on any supported operating system (OS) and use. The cloud and enterprise offerings are more of a wrapper on top of the open source one. They provide a web-based GUI and advanced features such as policy as code with Sentinel, cost analysis, private modules, GitOps, and CI/CD pipelines.

This chapter will discuss the open source offering and its core functions.

Terraform open source is divided into two main parts – Terraform Core and Terraform providers, as seen in the following diagram:

Figure 8.1 – Terraform architecture

Let’s look at the functions of both components:

  • Terraform Core is the CLI that we will use to interact with Terraform. It takes two main inputs – your Terraform configuration files and the existing state. It then takes the difference in configuration and applies it.
  • Terraform providers are plugins that Terraform uses to interact with cloud providers. The providers translate the Terraform configuration into the respective cloud’s REST API calls so that Terraform can manage its associated infrastructure. For example, if you want Terraform to manage AWS infrastructure, you must use the Terraform AWS provider.

Now let’s see how we can install open source Terraform.

2024-01-16 by Building the Python Flask app Cloud Certifications Deleting an ECS service Exams of Cloud Using the Azure Terraform provider 0

Load testing your app on Knative – Containers as a Service (CaaS) and Serverless Computing for Containers

We will use the hey utility to perform load testing. Since your application has already been deployed, run the following command to do the load test:

$ hey -z 30s -c 500 http://py-time.default.35.226.198.46.sslip.io

Once the command has executed, run the following command to get the currently running instances of the py-time pods:

$ kubectl get pod    
NAMEREADY STATUS RESTARTSAGE
py-time-00001-deployment-52vjv 2/2Running044s
py-time-00001-deployment-bhhvm 2/2Running044s
py-time-00001-deployment-h6qr5 2/2Running042s
py-time-00001-deployment-h92jp 2/2Running040s
py-time-00001-deployment-p27gl 2/2Running088s
py-time-00001-deployment-tdwrh 2/2Running038s
py-time-00001-deployment-zsgcg 2/2Running042s

As we can see, Knative has created seven instances of the py-time pod. This is horizontal autoscaling in action.

Now, let’s look at the cluster nodes by running the following command:

$ kubectl get nodes

NAME STATUS AGE gke-cluster-1-default-pool-353b3ed4-js71 Ready 3m17s gke-cluster-1-default-pool-353b3ed4-mx83 Ready 106m gke-cluster-1-default-pool-353b3ed4-vf7q Ready 106m

As we can see, GKE has created another node in the node pool because of the extra burst of traffic it received. This is phenomenal, as we have the Kubernetes API to do what we want. We have automatically horizontally autoscaled our pods. We have also automatically horizontally autoscaled our cluster worker nodes. This means we have a fully automated solution for runningcontainers without worrying about the management nuances! That is open source serverless in action for you!

Summary

This chapter covered CaaS and serverless CaaS services. These help us manage container applications with ease without worrying about the underlying infrastructure and managing them. We used Amazon’s ECS as an example and deep-dived into it. Then, we briefly discussed other solutions that are available on the market.

Finally, we looked at Knative, an open source serverless solution for containers that run on top of Kubernetes and use many other open source CNCF projects.

In the next chapter, we will delve into IaC with Terraform.

2023-11-05 by Deleting an ECS service Exams of Cloud Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Building the Python Flask app – Containers as a Service (CaaS) and Serverless Computing for Containers

Deploying a Python Flask application on Knative

To understand Knative, let’s try to build and deploy a Flask application that outputs the current timestamp in the response. Let’s start by building the app.

Building the Python Flask app

We will have to create a few files to build such an app.

The app.py file looks like this:

import os

import datetime

from flask import Flask

app = Flask(__name__)

@app.route(‘/’)

def current_time():

ct = datetime.datetime.now()

return ‘The current time is : {}!\n’.format(ct)

if __name__ == “__main__”:

app.run(debug=True,host=’0.0.0.0′)

We will need the following Dockerfile to build this application:

FROM python:3.7-slim

ENV PYTHONUNBUFFERED True

ENV APP_HOME /app

WORKDIR $APP_HOME

COPY . ./

RUN pip install Flask gunicorn

CMD exec gunicorn –bind :$PORT –workers 1 –threads 8 –timeout 0 app:app

Now, let’s go ahead and build the Docker container using the following command:

$ docker build -t <your_dockerhub_user>/py-time .

Now that the image is ready, let’s push it to Docker Hub by using the following command:

$ docker push <your_dockerhub_user>/py-time

As we’ve successfully pushed the image, we can run this on Knative.

Deploying the Python Flask app on Knative

We can use the kn command line or create a manifest file to deploy the app. Use the following command to deploy the application:

$ kn service create py-time –image <your_dockerhub_user>/py-time

Creating service ‘py-time’ in namespace ‘default’:

9.412s Configuration “py-time” is waiting for a Revision to become ready.

9.652s Ingress has not yet been reconciled.

9.847s Ready to serve.

Service ‘py-time’ created to latest revision ‘py-time-00001’ is available at URL:

http://py-time.default.35.226.198.46.sslip.io

As we can see, Knative has deployed the app and provided a custom endpoint. Let’s curl the endpoint to see what we get:

$ curl http://py-time.default.35.226.198.46.sslip.io The current time is : 2023-07-03 13:30:20.804790!

We get the current time in the response. As we already know, Knative should detect whether there is no traffic coming into the pod and delete it. Let’s watch the pods for some time and see what happens:

$ kubectl get pod -w    
NAMEREADYSTATUSRESTARTSAGE
py-time-00001-deployment-jqrbk2/2Running05s
py-time-00001-deployment-jqrbk2/2Terminating064s

As we can see, just after 1 minute of inactivity, Knative starts terminating the pod. Now, that’s what we mean by scaling from zero.

To delete the service permanently, we can use the following command:

$ kn service delete py-time

We’ve just looked at the imperative way of deploying and managing the application. But what if we want to declare the configuration as we did previously? We can create a CRD manifest with the Service resource provided by apiVersion—serving.knative.dev/v1.

We will create the following manifest file, called py-time-deploy.yaml, for this:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: py-time
spec:
template:
spec:
containers:
image: /py-time

As we’ve created this file, we will use the kubectl CLI to apply it. It makes deployment consistent with Kubernetes.

Note

Though it is a service resource, don’t confuse this with the typical Kubernetes Service resource. It is a custom resource provided by apiVersion serving.knative.dev/ v1. That is why apiVersion is very important.

Let’s go ahead and run the following command to do so:

$ kubectl apply -f py-time-deploy.yaml

service.serving.knative.dev/py-time created

With that, the service has been created. To get the service’s endpoint, we will have to query the ksvc resource using kubectl. Run the following command to do so:

$ kubectl get ksvc py-time

NAME          URL

py-time     http://py-time.default.35.226.198.46.sslip.io

The URL is the custom endpoint we have to target. Let’s curl the custom endpoint using the following command:

$ curl http://py-time.default.35.226.198.46.sslip.io The current time is : 2023-07-03 13:30:23.345223!

We get the same response this time as well! So, if you want to keep using kubectl for managing Knative, you can easily do so.

Knative helps scale applications based on the load it receives—automatic horizontal scaling. Let’s run load testing on our application to see that in action.

2023-10-19 by Deleting an ECS service Exams of Cloud Providing variable values Using the Azure Terraform provider 0

Scheduling services on ECS – Containers as a Service (CaaS) and Serverless Computing for Containers

ECS services are similar to Kubernetes ReplicaSets. They ensure that a certain number of tasks are always running at a particular time. To schedule a service, we can use the ecs-cli command line.

Tip

Always use services for applications that are long-running, such as web servers. For batch jobs, always use tasks, as we don’t want to recreate the job after it ends.

To run the nginx web server as a service, we can use the following command:

$ ecs-cli compose service up –create-log-groups \

–cluster cluster-1 –launch-type FARGATE

INFO[0001] Using ECS task definition TaskDefinition=”FARGATE:1″

INFO[0002] Auto-enabling ECS Managed Tags

INFO[0013] (service FARGATE) has started 1 tasks: (task 9b48084d).   timestamp=”2023-07-03

11:24:42 +0000 UTC”

INFO[0029] Service status desiredCount=1 runningCount=1 serviceName=FARGATE

INFO[0029] (service FARGATE) has reached a steady state. timestamp=”2023-07-03 11:25:00 +0000 UTC”

INFO[0029] (service FARGATE) (deployment ecs-svc/94284856) deployment completed. timestamp=”2023-07-03 11:25:00 UTC”

INFO[0029] ECS Service has reached a stable state desiredCount=1 runningCount=1 serviceName=FARGATE

INFO[0029] Created an ECS service service=FARGATE taskDefinition=”FARGATE:1″

Looking at the logs, we can see that the service is trying to ensure that the task’s desired count matches the task’s running count. If your task is deleted, ECS will replace it with a new one.

Let’s list the tasks and see what we get by using the following command:

$ ecs-cli ps –cluster cluster-1 Name State Ports cluster-1/9b48084d/web RUNNING 18.234.123.71:80

TaskDefinition

FARGATE:1

As we can see, the service has created a new task that is running on 18.234.123.71:80. Let’s try to access this URL using the following command:

$ curl 18.234.123.71

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

…

</html>

We get the default nginx home page in the response. Now, let’s try to browse the logs of the task.

Browsing container logs using the ECS CLI

Apart from using Amazon CloudWatch, you can also use the friendly ECS CLI to do this, irrespective of where your logs are stored. This helps us see everything from a single pane of glass.

Run the following command to do so:

$ ecs-cli logs –task-id 9b48084d –cluster cluster-1 /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration

/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/ default.conf

/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh

/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh

/docker-entrypoint.sh: Configuration complete; ready for start up

2023/07/03 11:24:57 [notice] 1#1: nginx/1.25.1

2023/07/03 11:24:57 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14)

2023/07/03 11:24:57 [notice] 1#1: OS: Linux 5.10.184-175.731.amzn2.x86_64

2023/07/03 11:24:57 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 65535:65535

2023/07/03 11:24:57 [notice] 1#1: start worker processes

2023/07/03 11:24:57 [notice] 1#1: start worker process 29

2023/07/03 11:24:57 [notice] 1#1: start worker process 30

13.232.8.130 – – [03/Jul/2023:11:30:38 +0000] “GET / HTTP/1.1” 200 615 “-” “curl/7.81.0”

“-“

As we can see, we can browse the logs for the particular task this service is running. Now, let’s go ahead and delete the service.

2023-02-04 by Exams of Cloud Knative architecture Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Scheduling Fargate tasks on ECS – Containers as a Service (CaaS) and Serverless Computing for Containers

Scheduling tasks on Fargate is very similar to EC2. Here, we need to specify the launch-type value as FARGATE.

To schedule the same task on Fargate, run the following command:

$ ecs-cli compose up –create-log-groups –cluster cluster-1 –launch-type FARGATE FATA[0001] ClientException: Fargate only supports network mode ‘awsvpc’.

Oops! We have a problem! Well, it’s complaining about the network type. For a Fargate task, we must supply the network type as awsvpc instead of the default bridge network. The awsvpc network is an overlay network that implements the Container Network Interface (CNI). To understand more about Docker networking, please refer to Chapter 1, The Modern Way of DevOps. For now, let’s go ahead and configure the awsvpc network type. But before that, the Fargate task requires a few configurations.

To access the resources for this section, cd into the following directory:

$ cd ~/modern-devops/ch7/ECS/tasks/FARGATE/

First, we’ll have to assume a task execution role for the ECS agent to authenticate with the AWS API and interact with Fargate.

To do so, create the following task-execution-assume-role.json file:

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Sid”: “”,
“Effect”: “Allow”,
“Principal”: {
“Service”: “ecs-tasks.amazonaws.com”
},
“Action”: “sts:AssumeRole”
}
]
}

Then, use the following command to assume the task execution role:

$ aws iam –region us-east-1 create-role –role-name ecsTaskExecutionRole \ –assume-role-policy-document file://task-execution-assume-role.json

ECS provides a default role policy called AmazonECSTaskExecutionRolePolicy, which contains various permissions that help you interact with CloudWatch and Elastic Container Registry (ECR). The following JSON code outlines the permission that the policy has:

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“ecr:GetAuthorizationToken”,
“ecr:BatchCheckLayerAvailability”,
“ecr:GetDownloadUrlForLayer”,
“ecr:BatchGetImage”,
“logs:CreateLogStream”,
“logs:PutLogEvents”
],
“Resource”: “*”
}
]
}

We have to assign this role policy to the ecsTaskExecution role we assumed previously by using the following command:

$ aws iam attach-role-policy \

–policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy \ –role-name ecsTaskExecutionRole

Once we’ve assigned the policy to the ecsTaskExecution role, we need to source the ID of both subnets and the security group of the ECS cluster when we created it. You can find those details in the command-line output from when we created the cluster. We will use these details in the following ecs-params.yml file:

version: 1
task_definition:
task_execution_role: ecsTaskExecutionRole
ecs_network_mode: awsvpc
task_size:
mem_limit: 0.5GB
cpu_limit: 256
run_params:
network_configuration:
awsvpc_configuration:
subnets:
“subnet-088b52c91a6f40fd7”
“subnet-032cd63290da67271” security_groups:
“sg-097206175813aa7e7” assign_public_ip: ENABLED

The ecs-params.yml file consists of task_execution_role, which we created, and ecs_ network_mode set to awsvpc, as Fargate requires. We’ve defined task_size to have 0.5GB of memory and 256 millicores of CPU. So, since Fargate is a serverless solution, we only pay for the CPU cores and memory we consume. The run_params section consists of network_configuration, which contains awsvpc_configuration. Here, we specify both subnets created when we created the ECS cluster. We must also specify security_groups, which we created with the ECS cluster.

Note

Use the subnets and security groups of your ECS cluster instead of copying the ones in this example.

Now that we’re ready to fire the task on Fargate, let’s run the following command:

$ ecs-cli compose up –create-log-groups –cluster cluster-1 –launch-type FARGATE

Now, let’s check whether the task is running successfully by using the following command:

$ ecs-cli ps –cluster cluster-1 Name State cluster-1/8717a149/web RUNNING
Ports                   TaskDefinition
3.80.173.230:80 FARGATE:1

As we can see, the task is running on 3.80.173.230:80 as a Fargate task. Let’s curl this URL to see whether we get a response by using the following command:

$ curl 3.80.173.230:80
Welcome to nginx! …

As we can see, we get the default nginx home page.

Now, let’s go ahead and delete the task we created by using the following command:

$ ecs-cli compose down –cluster cluster-1

As we already know, tasks have a set life cycle, and once they stop, they stop. You cannot start the same task again. Therefore, we must create a service to ensure that a certain number of tasks are always running. We’ll create a service in the next section.

2023-01-18 by Building the Python Flask app Exams of Cloud Load testing your app on Knative Providing variable values Using the Azure Terraform provider 0

Scheduling EC2 tasks on ECS – Containers as a Service (CaaS) and Serverless Computing for Containers

Let’s use ecs-cli to apply the configuration and schedule our task using the following command:

$ ecs-cli compose up –create-log-groups –cluster cluster-1 –launch-type EC2

Now that the task has been scheduled and the container is running, let’s list all the tasks to get the container’s details and find out where it is running. To do so, run the following command:

$ ecs-cli ps –cluster cluster-1

Name                                   State       Ports                            TaskDefinition

cluster-1/fee1cf28/web  RUNNING  34.237.218.7:80->80  EC2:1

As we can see, the web container is running on cluster-1 on 34.237.218.7:80. Now, use the following command to curl this endpoint to see what we get:

$ curl 34.237.218.7:80

<html>

<head>

<title>Welcome to nginx!</title>

…

</html>

Here, we get the default nginx home page! We’ve successfully scheduled a container on ECS using the EC2 launch type. You might want to duplicate this task to handle more traffic. This is known as horizontal scaling. We’ll see how in the next section.

Scaling tasks

We can easily scale tasks using ecs-cli. Use the following command to scale the tasks to 2:

$ ecs-cli compose scale 2 –cluster cluster-1 –launch-type EC2

Now, use the following command to check whether two containers are running on the cluster:

$ ecs-cli ps –cluster cluster-1  
NameStatePortsTaskDefinition
cluster-1/b43bdec7/webRUNNING54.90.208.183:80->80EC2:1
cluster-1/fee1cf28/webRUNNING34.237.218.7:80->80EC2:1

As we can see, two containers are running on the cluster. Now, let’s query CloudWatch to get the logs of the containers.

Querying container logs from CloudWatch

To query logs from CloudWatch, we must list the log streams using the following command:

$ aws logs describe-log-streams –log-group-name /aws/webserver \ –log-stream-name-prefix ecs | grep logStreamName

“logStreamName”: “ecs/web/b43bdec7”,

“logStreamName”: “ecs/web/fee1cf28”,

As we can see, there are two log streams for this—one for each task. logStreamName follows the convention <log_stream_prefix>/<task_name>/<task_id>. So, to get the logs for ecs/ b43bdec7/web, run the following command:

$ aws logs get-log-events –log-group-name/aws/webserver \ –log-stream ecs/web/b43bdec7

Here, you will see a stream of logs in JSON format in the response. Now, let’s look at how we can stop running tasks.

Stopping tasks

ecs-cli uses the friendly docker-compose syntax for everything. Use the following command to stop the tasks in the cluster:

$ ecs-cli compose down –cluster cluster-1

Let’s list the containers to see whether the tasks have stopped by using the following command:

$ ecs-cli ps –cluster cluster-1

INFO[0001] Stopping container… container=cluster-1/b43bdec7/web INFO[0001] Stopping container… container=cluster-1/fee1cf28/web INFO[0008] Stopped container… container=cluster-1/b43bdec7/web desiredStatus=STOPPED lastStatus=STOPPED taskDefinition=”EC2:1″ INFO[0008] Stopped container… container=cluster-1/fee1cf28/web desiredStatus=STOPPED lastStatus=STOPPED taskDefinition=”EC2:1″

As we can see, both containers have stopped.

Running tasks on EC2 is not a serverless way of doing things. You still have to provision and manage the EC2 instances, and although ECS manages workloads on the cluster, you still have to pay for the amount of resources you’ve provisioned in the form of EC2 instances. AWS offers Fargate as a serverless solution where you pay per resource consumption. Let’s look at how we can create the same task as a Fargate task.

2022-12-21 by Building the Python Flask app Cloud Certifications Exams of Cloud Providing variable values Using the Azure Terraform provider 0