SYMPLIFY LEARNING

Menu

Using CloudWatch Log Insights

by

Cloudwatch Insights allows you to search and analyze your log data that was sent to Amazon CloudWatch.

CloudWatch Log Insights automatically discovers fields in logs from AWS services such as Amazon Route 53, AWS Lambda, AWS CloudTrail and Amazon VPC

When you navigate to Logs Insights in the AWS Console and select the log group you want to analyze logs in, CloudWatch logs insights automatically detects the data fields in the group.

For example, in the screenshot below, after selecting a log group, the fields identified are shown below:

Notice the percentage amount stated next to each field. That percentage value tells you the percentage of log events listed that contain those fields. For example, it means the @message field can be found in 100% of the log events.

Using the data in those log event fields, you can find specific information. For example, I used the following query to find log events where the query_name included symplify.

fields @timestamp, @message, query_name
| sort @timestamp desc
| limit 50
| filter query_name like "symplify"

Each item in the query is described as follows:

  • Fields: This shows specific fields in the query result. Each item I specify under ‘fields’ is shown as a separate column after I run my query (unless I specifically use the ‘display’ command to choose which fields to display). As a result, my query will show the timestamp, message, and query_name columns.
  • Sort: Here I use ‘sort’ to display my log events in descending order based on the timestamp. This will display the latest entry at the top
  • Limit: I use the limit command to specify the number of log events that I want the query to return
  • Filter: Using the ‘filter’ command, I’m able to query only the log events/entries where the query name contains “symplify”. By specifying the ‘like’ operator, I’m able to find all events that include “symplify” instead of having to match the exact DNS query

This returned the following log entry:

This could be useful in a scenario where I’m trying to verify that DNS queries from an EC2 instance in my environment to symplifylearning.com are successfully resolved.

In the above screenshot, I see the DNS query for symplifylearning.com was successfully resolved to 198.54.115.236.

AWS Documentation showing the query syntax for using CloudWatch log insights to further analyse your data can be found here.