How to stop/start EC2 instance on a scheduled basis to save cost by using Boto3 and Lambda

6 min readJun 20, 2021

To read the complete post

How to stop/start EC2 instance on a scheduled basis to save cost by using Boto3 and Lambda

This is one of the ask I came across in Dev env in terms of saving cost where I need to shut down all the EC2 instance…

101daysofdevops.com

This is one of the ask I came across in Dev env in terms of saving cost where I need to shut down all the EC2 instance at 6 pm and bring it back the next day at 9 am.

Problem: Shutdown all EC2 instances in the AWS DEV account at 6 pm and bring them back the next day at 9 am (Monday to Friday).

Solution: Using Lambda function in a combination with CloudWatch events.

One of the major challenge in implementing this what would be the case if Developer is working late and he wants his instance to be run beyond 6 pm or if there is an urgent patch he needs to implement and need to work on the weekend?
One common solution we come out with is manually specifying the list of an instance in Python Code(Lambda Function) and in case of exception it goes through a change management process where we need to remove developer instance manually(Agree not an ideal solution but so far works great)

Step1: Create IAM Role so that Lambda can interact with CloudWatch Events

Go to IAM Console https://console.aws.amazon.com/iam/home?region=us-west-2#/home --> Roles --> Create role

In the next screen, select Create Policy and then JSON tab and then copy the below IAM Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:StartInstances",
        "ec2:StopInstances"
      ],
      "Resource": "*"
    }
  ]
}

Skip the tag section and click Next
Give some meaningful name to your policy for e.g. ec2_stop_start and then click on Create policy

Go back to the IAM console and Create a role

Go to IAM Console https://console.aws.amazon.com/iam/home?region=us-west-2#/home --> Roles --> Create role

But this time select the policy(ec2_stop_start) we have created above. Click Next:Tags and again skip the Tagging part

Give some meaningful name to the role(for e.g. ec2_stop_start_role)and click on Create role

In case if you don’t want to create an IAM role manually, you can automate the entire process using awscli
The below IAM Policy has the permission to create and push logs to AWS CloudWatch. It also has the permission to stop/start/describe the EC2 instance and Regions.

cat iam_policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:StartInstances",
        "ec2:StopInstances"
      ],
      "Resource": "*"
    }
  ]
}

This time let’s create IAM policy using awscli

aws iam create-policy --policy-name iam-policy-stop-start-ec2 --policy-document file://iam_policy.json
{
    "Policy": {
        "PolicyName": "iam-policy-stop-start-ec2",
        "PolicyId": "ANPA47TQRQ43C3PXNTFLU",
        "Arn": "arn:aws:iam::XXXXXXXXX:policy/iam-policy-stop-start-ec2",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "CreateDate": "2021-06-18T22:20:36Z",
        "UpdateDate": "2021-06-18T22:20:36Z"
    }
}

NOTE: Please note down the arn as we need it in the later step while creating the IAM role.

The next step is to create an IAM Role and attach the trust policy to it

cat Role-Trust-Policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

NOTE: Trust policy defines which principals can assume the role and under which conditions.

Now execute the aws iam create-role command to create the role

aws iam create-role --role-name ec2-stop-start --assume-role-policy-document file://Role-Trust-Policy.json
{
    "Role": {
        "Path": "/",
        "RoleName": "ec2-stop-start",
        "RoleId": "AROA47TQRQ43EZG6DFGOO",
        "Arn": "arn:aws:iam::XXXXXXXXX:role/ec2-stop-start",
        "CreateDate": "2021-06-18T22:32:17Z",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": "ec2.amazonaws.com"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}

Finally, create an IAM Role by attaching the arn of the IAM Policy create in the earlier step.

aws iam attach-role-policy --policy-arn arn:aws:iam::XXXXXXX:policy/iam-policy-stop-start-ec2 --role-name ec2-stop-start

Step2: Create Lambda function

Go to Lambda https://us-west-2.console.aws.amazon.com/lambda/home?region=us-west-2#/home
Select Create Function

* Select Author from scratch
* Name: Give your Lambda function any name(for e.g. EC2_stop)
* Runtime: Select Python3.7 as runtime
* Role: Choose the role we create in first step(ec2_stop_start_role)
* Click on Create function

In this scenario, we need to create Function one to stop an instance and others to start an instance.

Approach 1

import boto3

ec2 = boto3.resource("ec2")

for instance in ec2.instances.all():
    instance.stop()

Approach 2.1

import boto3

ec2 = boto3.resource("ec2")

ec2_instance = {"Name": "instance-type", "Values": ["t2.micro"]}

for instance in ec2.instances.filter(Filters=[ec2_instance]):
    instance.stop()

Approach 2.2

import boto3

ec2 = boto3.resource("ec2")

ec2_instance = {"Name": "instance-type", "Values": ["t2.micro"]}
ec2_tag = {"Name": "tag:Name", "Values": ["boto3-prod"]}

for instance in ec2.instances.filter(Filters=[ec2_tag]):
    instance.stop()

Approach 3

import boto3

ec2 = boto3.resource("ec2")

regions = []
for region in ec2.meta.client.describe_regions()['Regions']:
    regions.append(region['RegionName'])


for region in regions:
    ec2 = boto3.resource("ec2", region_name=region)

    print(“EC2 region is:", region)

    ec2_instance = {"Name": "instance-state-name", "Values": ["running"]}

    instances = ec2.instances.filter(Filters=[ec2_instance])

    for instance in instances:
        instance.stop()
        print("The following EC2 instances is now in stopped state", instance.id)

Once the function is created copy-paste the code from Approach 3(or depend upon your requirement)to the Lambda console and click on Deploy. Keep all the other settings as default

If you have a large set of EC2 instances, you might need to increase the Timeout value from the default value of 3 sec. In order to do that click on the Configuration tab and then Edit.

Step3: Create the CloudWatch event to trigger this Lambda function

Open the Amazon CloudWatch console.
Choose Events, and then choose Create rule.
Choose Schedule under Event Source.

* Under Cron expression choose * 18 * * ? * (If you want to shutdown your instance at 6pm everyday)
* Choose Add target, and then choose Lambda function(EC2_stop)that you created earlier to stop the instance

The same steps need to be repeated for Starting the instance

* Only difference is different time schedule
* Under target different Lambda function(to start the instance)

Please join me with my journey by following any of the below links

Website: https://101daysofdevops.com/
Twitter: @100daysofdevops OR @lakhera2015
Facebook: https://www.facebook.com/groups/795382630808645/
Medium: https://medium.com/@devopslearning
GitHub: https://github.com/100daysofdevops/100daysofdevops
Slack: https://join.slack.com/t/100daysofdevops/shared_invite/enQtNzg1MjUzMzQzMzgxLWM4Yjk0ZWJiMjY4ZWE3ODBjZjgyYTllZmUxNzFkNTgxZjQ4NDlmZjkzODAwNDczOTYwOTM2MzlhZDNkM2FkMDA
YouTube Channel: https://www.youtube.com/user/laprashant/videos?view_as=subscriber

How to stop/start EC2 instance on a scheduled basis to save cost by using Boto3 and Lambda

How to stop/start EC2 instance on a scheduled basis to save cost by using Boto3 and Lambda

This is one of the ask I came across in Dev env in terms of saving cost where I need to shut down all the EC2 instance…

Approach 1

Approach 2.1

Approach 2.2

Approach 3

Written by Prashant Lakhera

Responses (2)