Day 23–101 Days of DevOps — How to stop/start EC2 instances on a scheduled basis to save cost using AWS Lambda and CloudWatch

Welcome to Day 23 of 101 Days of DevOps. The topic for today is how to stop/start EC2 instances on a scheduled basis to save cost using AWS Lambda and CloudWatch.

To view the complete course, please check the below url.

101 Days of DevOps

For more info, register via the below link

Create an Account

YouTube Channel link

Prashant Lakhera

On Day 17, you have learned how to stop/start instances using Boto3. https://www.101daysofdevops.com/courses/101-days-of-devops/lessons/day-17/. On Day 22, you have learned AWS Lambda https://www.101daysofdevops.com/courses/101-days-of-devops/lessons/day-22/. Let’s tie both of these concepts together and automate the process of stop/start instances on a scheduled basis.

This is one of the ask I came across in Dev env in terms of saving money where I need to shut down all the EC2 instance at 6 pm and bring it back the next day at 9 am.

Problem: Shut down all EC2 instances in the AWS DEV account at 6 pm and bring them back the next day at 9 am (Monday to Friday).

Solution: Using Lambda function in combination with CloudWatch events.

• One of the major challenges in implementing this will be if the Developer is working late and wants his instance to be run beyond 6 pm or an urgent patch he needs to implement and need to work on the weekend?
• One common solution we come out with is manually specifying the list of an instance in Python Code(Lambda Function), and in case of exception it goes through a change management process where we need to remove developer instance manually(Agree is not an ideal solution but so far works great)

Step1: Create IAM Role so that Lambda can interact with CloudWatch Events

Go to IAM Console https://console.aws.amazon.com/iam/home?region=us-west-2#/home --> Roles --> Create role

• In the next screen, select Create Policy; IAM Policy will look like this

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeRegions",
        "ec2:StartInstances",
        "ec2:StopInstances"
      ],
      "Resource": "*"
    }
  ]
}

• Add this newly created policy to the role

Step2: Create Lambda function

• Go to Lambda https://us-west-2.console.aws.amazon.com/lambda/home?region=us-west-2#/home.
• Select Create Function

* Select Author from scratch
* Name: Give your Lambda function any name
* Runtime: Select Python3.8 as runtime
* Role: Choose the role we create in first step
* Click on Create function

• In this scenario, we need to create Function one to stop an instance and others to start an instance.
• To stop an instance, the code will look like this. We already discussed this code on Day 17.

import boto3

def lambda_handler(event, context):
    ec2 = boto3.resource("ec2")

    regions = []
    for region in ec2.meta.client.describe_regions()['Regions']:
        regions.append(region['RegionName'])

    for region in regions:
        ec2 = boto3.resource("ec2", region_name=region)

        print("EC2 region is: ", region)

        ec2_instance = {"Name": "instance-state-name", "Values": ["running"]}

        instances = ec2.instances.filter(Filters=[ec2_instance])

        for instance in instances:
            instance.stop()
            print("The following EC2 instances is now in stopped state", instance.id)

Step3: Create the CloudWatch event to trigger this Lambda function

• Open the Amazon CloudWatch console.
• Choose Events, and then choose Create rule.
• Choose Schedule under Event Source.

* Under Cron expression choose * 18 * * ? * (If you want to shutdown your instance at 6pm everyday)
* Choose Add target, and then choose Lambda function that you created earlier to stop the instance

• The same steps need to be repeated for Starting the instance(rather than using instance.stop() you need to use instance.start(), and during filtering, the value should be stopped.

* Only difference is different time schedule
* Under target different Lambda function(to start the instance)

NOTE: One important point to note is that all scheduled events are in the UTC timezone, so you need to customize it based on your timezone.

• Once the event is triggered, go to your

Lambda function --> Monitoring --> View logs in CloudWatch

• For stopping the instance, you will see something like this.

• For starting the instance

• The simple automation system is ready to stop/start the instance and to save some company money.

GitHub link:

https://github.com/100daysofdevops/100daysofdevops/tree/master/aws-lambda/ec2_stop_start_instances_updated

For more information, please check the below video.

If you like to dig deeper into the AWS concept, please feel free to check my book.

AWS for System Administrators: Build, automate, and manage your infrastructure on the most popular…

Looking forward to you guys joining this journey and spend a minimum of an hour every day for the next 101 days on DevOps work and post your progress using any of the below mediums.

• Website: https://101daysofdevops.com/
• Linkedin: https://www.linkedin.com/in/prashant-lakhera-696119b/
• Twitter: @100daysofdevops OR @lakhera2015
• Facebook: https://www.facebook.com/groups/795382630808645/
• Medium: https://medium.com/@devopslearning
• GitHub: https://github.com/100daysofdevops/100daysofdevops
• YouTube Channel: https://www.youtube.com/user/laprashant/videos
• Slack: https://join.slack.com/t/100daysofdevops/shared_invite/zt-au03logz-YfDUp_FJF4rAUeDEbgWmsg
• Reddit: r/101DaysofDevops
• Meetup: https://www.meetup.com/100daysofdevops/