100 Days of AWS — Day 9— Using CloudWatch Logs(Metric Filters) to create a Simple Monitoring system that alerts for any unauthorized access

To view the complete course, please enroll it using the below link(it’s free)

https://www.101daysofdevops.com/courses/100-days-of-aws/

Welcome to Day 9 of 100 Days of AWS. The topic for today is Using CloudWatch Logs(Metric Filters) to create a Simple Monitoring system that alerts for any unauthorized access

This can be achieved using CloudWatch Metric Filter in combination with SNS.

Step1

• Install CloudWatch Agent(Make sure you are pushing /var/log/messages and /var/log/secure logs from your instance to CloudWatch log group). Please check the Day8 blog how to install cloudwatch agent and push logs to CloudWatch logs https://www.101daysofdevops.com/courses/100-days-of-aws/lessons/day-8-pushing-system-logs-and-custom-metrics-using-cloudwatch-agent/
• Go to CloudWatch Logs https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/secure/log-events/i-025d42f856ff0da4c$3FfilterPattern$3Dinvalid+user and search for invalid user string. As you can see there is already lots of invalid user attempt going on with my ec2 instance.

• Go back to CloudWatch Log groups, select the Log group(for example secure, as it contain all the invalid login attempt) and under Actions drop down select Create metric filter

• In the next screen, under Filter pattern enter invalid user

* Filter Pattern : Type Invalid user
* Select Log Data to Test: Select the right instance

• Click on Test pattern to see the result and click on Next
• Enter the details like Filter name(for example invalid user), Create new Metric namespace(for example: invaliduser) or use the existing one and unique metric name in the namespace. For Metric value, if your metric filter is counting occurrences of the keywords in the filter, enter 1. This increments the metric by 1 for each log event that includes one of the keywords.Alternatively, enter a token such as $size. This increments the metric by the value of the number in the size field for every log event that contains a size field. Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CreateMetricFilterProcedure.html

• Click on Create at the bottom of the screen.
• Select the metrics we have created and click on Create alarm

• Keep all the parameters default except for this demo I am putting Period as 1 minute and threshold value as 1. The alarm goes to ALARM state when the metric breaches the threshold for a specified number of evaluation periods

• Under Configure action, select Create new topic and enter the email address under Email endpoints and click on Create topic.

• Give your alarm some name, click on Next. In the next screen, click on Create alarm.

• Once the threshold is reached, you will receive an email like this

• Your simple notification system against un-authorized user is up and running.