Amazon CloudWatch | Domain 4 — AWS Dev Associate

🌐 What is Amazon CloudWatch? — The Big Picture

Imagine you have a car. You have a dashboard showing speed, fuel level, engine temperature, and warning lights. Amazon CloudWatch is that dashboard for your AWS applications.

When you run apps on AWS (Lambda, EC2, API Gateway, RDS, etc.), those services automatically send data to CloudWatch. You can also send your own custom data. CloudWatch stores it, lets you visualize it, and alerts you when something goes wrong.

CloudWatch Big Picture — Everything flows into it

λLambdaInvocations, Errors, Duration

🖥️EC2CPU, Network, Disk

🚦API GatewayLatency, 4XX, 5XX

🗄️RDS/DynamoDBConnections, Read/Write

📦Your App CodeCustom Metrics

↓ Metrics & Logs ↓

☁️ Amazon CloudWatch The Central Hub

↓ Capabilities ↓

📊MetricsNumbers over time

📝LogsText records

📈DashboardsVisual charts

🔔AlarmsAlert & Act

🔍Logs InsightsQuery logs

🤖SyntheticsTest endpoints

🎯 Exam Tip

CloudWatch is the #1 topic in Domain 4. The exam will test every sub-feature. Don't just know "what CloudWatch is" — know the difference between Metrics, Logs, Logs Insights, Alarms, Synthetics, and EMF. These are different things and the exam tests which one to use when.

📊 CloudWatch Metrics — Numbers Over Time

A metric is a number that changes over time. It's the most basic unit of monitoring. Examples:

Lambda error count over the last hour: 15 errors
EC2 CPU utilization: 73%
API Gateway request latency: 230ms average
RDS database connections: 150 active connections

AWS services automatically publish metrics to CloudWatch. You never have to do anything — they just appear.

🗂️ Key Metric Concepts You MUST Know

Namespace

A container (like a folder) that groups related metrics. AWS services use their own namespaces: AWS/Lambda, AWS/EC2, AWS/ApiGateway, AWS/RDS. Your custom metrics go in a namespace you name, like MyApp/OrderService. Namespaces keep metrics organized and prevent naming collisions.

Dimension

A key-value pair that provides extra context (metadata) for a metric. Like a filter. Example: For Lambda metrics, a dimension might be FunctionName=my-order-function. This lets you view metrics for one specific Lambda function instead of all Lambda functions. Without dimensions, you'd only see aggregate data for the entire service.

Period

The length of time over which one data point is collected and aggregated. Example: A 1-minute period means CloudWatch calculates one data point every minute (average of all values in that minute). Common periods: 1 sec, 10 sec, 30 sec, 1 min, 5 min.

Statistics

How to aggregate multiple values within a period into a single number. Options: Average (typical value), Sum (total), Minimum, Maximum, SampleCount (how many data points), p90/p95/p99 (percentiles — 99% of requests were faster than this). Percentiles are very useful for latency!

Resolution

How granular (detailed) the metric is. Standard Resolution = minimum 1-minute granularity (default for most AWS services). High Resolution = down to 1-second granularity (for custom metrics only, using the StorageResolution: 1 parameter).

🕐 Metric Retention — How Long Does CloudWatch Keep Data?

Resolution	Retention Period	Example Use
1-second (High Resolution)	3 hours	Very short-lived detailed view
1-minute (Standard)	15 days	Day-to-day monitoring
5-minute	63 days	Month-long trend analysis
1-hour	15 months	Long-term capacity planning

After a period passes, CloudWatch automatically rolls up data to lower resolution. Your 1-minute data from 20 days ago is summarized into 5-minute data.

✏️ Custom Metrics — Sending Your Own Data

AWS services send their metrics automatically. But your application code doesn't — you need to send custom metrics yourself. This is how you track business KPIs (Key Performance Indicators) like "orders placed per minute" or "payment failures".

Three Ways to Send Custom Metrics:

1. AWS SDK — PutMetricData

Call the CloudWatch API directly from your code. Works anywhere (Lambda, EC2, on-prem). Simple but requires an extra API call during execution.

2. Unified CloudWatch Agent

Install an agent on EC2 or on-prem servers. Automatically collects system-level metrics (memory, disk, process count) that AWS doesn't collect by default.

3. Embedded Metric Format (EMF)

Write structured JSON logs — CloudWatch automatically extracts metrics. Best for Lambda (no extra API calls). Explained in detail later.

        Python SDK
# Sending a custom metric with AWS SDK (Boto3)
import boto3

cloudwatch = boto3.client('cloudwatch')

# Send a metric: "OrdersPlaced" in namespace "MyApp"
cloudwatch.put_metric_data(
    Namespace='MyApp/OrderService',
    MetricData=[{
        'MetricName': 'OrdersPlaced',
        'Dimensions': [
            {'Name': 'Environment', 'Value': 'Production'},
            {'Name': 'Region',      'Value': 'us-east-1'}
        ],
        'Value': 1,                  # value to record
        'Unit': 'Count',             # None, Count, Bytes, Seconds, etc.
        'StorageResolution': 60      # 1 = high-res (1 second), 60 = standard
    }]
)
      

📐 Metric Math — Calculations Between Metrics

Sometimes one metric isn't enough. You need to calculate something from multiple metrics. For example, error rate = (errors / total requests) × 100. Metric Math lets you do this directly in CloudWatch without writing code.

📌 Example

Error Rate Alarm: You want to alarm when Lambda's error rate exceeds 5%.
Metric Math: (m1/m2)*100 where m1 = Lambda/Errors and m2 = Lambda/Invocations.
Create an alarm on this calculated metric. If > 5%, trigger the alarm.

📝 CloudWatch Logs — Storing and Searching Text Output

Logs are text records of what your application did. Every time your Lambda function runs, it can print messages. Those messages go to CloudWatch Logs. Without logs, when something breaks you have no idea what happened.

📌 Real Example

Your order processing Lambda fails. Without logs, you know it failed but not why. With logs, you see: "ERROR: Payment service returned 503 at 14:32:07 for order #12345". Now you know exactly what broke and when.

🗂️ Log Structure — From Groups to Events

CloudWatch Logs Hierarchy

📁Log GroupOne per application or service

→ contains

📄Log StreamOne per instance / invocation session

→ contains

📌Log EventsIndividual log lines with timestamps

Log Group

A container for logs from the same source. Think of it as a project folder. AWS automatically creates log groups for services like Lambda (e.g., /aws/lambda/my-function) and API Gateway. You set the retention policy at the log group level (how long to keep logs). Default: never expire (you pay forever). Best practice: set retention to 7, 30, 90 days etc. to control costs.

Log Stream

A sequence of log events from one specific source instance. For Lambda: each Lambda execution environment gets its own log stream (named like 2024/01/15/[$LATEST]abc123def456). Multiple invocations in the same environment share the same stream. For EC2: each instance gets its own stream.

Log Event

A single log entry — one line of text with a timestamp. This is the most granular unit. Example: 2024-01-15T14:32:07Z ERROR Payment service unavailable.

🔍 Metric Filters — Extract Numbers from Log Text

Metric Filters let you search log text for patterns and convert matches into CloudWatch Metrics. This is powerful because many applications log error counts, latency values, etc. as text — and you want to alarm on them.

📌 Example

Your application logs lines like: ERROR: payment failed.
Create a Metric Filter with pattern ERROR on the log group. Every time CloudWatch sees this pattern, it increments a counter metric called PaymentErrors.
Now create a CloudWatch Alarm on PaymentErrors > 10 in 5 minutes.
Result: You get alerted when payment errors spike, even though you never called PutMetricData.

📦 Log Retention — Control Costs

By default, CloudWatch Logs never expire. This can become expensive. Always set retention policies:

7 Days

Dev/test environments

30 Days

Most production apps

90 Days

Security/compliance

1-7 Years

Regulatory requirements

For long-term archival (cheaper), export logs to S3 and use Athena to query them.

🔍 CloudWatch Logs Insights — Query Your Logs Like a Database

CloudWatch Logs Insights is a query tool. Instead of scrolling through thousands of log lines manually, you write a query and it finds what you need in seconds. Think of it like SQL for your logs.

ℹ️ When to Use

Use Logs Insights when you need to: find all error messages in the last 24 hours, count how many times an event happened, find slow requests, identify which user caused the most errors, or aggregate log data over time.

📝 Query Syntax — Logs Insights Language

The query language uses pipe-separated commands (similar to Unix pipes). Each command filters or transforms the results:

        CloudWatch Logs Insights
# Basic structure: each line is a command, piped together
fields @timestamp, @message, @logStream
| filter @message like /ERROR/        # find lines containing ERROR
| sort @timestamp desc                # newest first
| limit 50                              # show first 50 results

# -------------------------------------------------------
# Count errors per minute (useful for trending)
filter @message like /ERROR/
| stats count() as errorCount by bin(1m)

# -------------------------------------------------------
# Find the slowest requests (latency > 1000ms)
filter duration > 1000
| stats avg(duration), max(duration), count() by requestPath
| sort avg(duration) desc

# -------------------------------------------------------
# Top 10 users by request count
stats count() as requestCount by userId
| sort requestCount desc
| limit 10

# -------------------------------------------------------
# Lambda function — find cold starts
filter @message like /Init Duration/
| stats avg(@initDuration) by @logStream
      

📋 Key Logs Insights Commands

Command	What it does	Example
fields	Select which fields to show	`fields @timestamp, @message`
filter	Keep only matching lines	`filter level = "ERROR"`
stats	Aggregate (count, avg, sum, min, max)	`stats count() by errorType`
sort	Order results	`sort @timestamp desc`
limit	Limit number of results	`limit 100`
parse	Extract values from log text	`parse @message "user=* latency=*" as userId, latency`
like /regex/	Regex pattern matching	`filter @message like /timeout/`

🎯 Exam Tip

Logs Insights is the answer when the exam asks about querying logs, log analytics, or searching through application logs. It's different from CloudWatch Metrics (which are numbers, not text). The exam tests this distinction.

📈 CloudWatch Dashboards — Visual Monitoring

Dashboards are visual displays of your CloudWatch data. Think of a mission control room with multiple screens. You can put any metrics or alarms on a dashboard and share it with your team.

📊 What Can You Put on a Dashboard?

📉 Line Charts

Show metric trends over time. E.g., CPU usage over the last 24 hours.

📊 Bar Charts

Compare values across dimensions. E.g., error count per Lambda function.

🔢 Number Widgets

Show the current value of a metric. E.g., "Active DB Connections: 127".

🔔 Alarm Status

Show whether alarms are OK, ALARM, or INSUFFICIENT_DATA.

📝 Text Widgets

Add notes, links, or documentation to the dashboard.

🔍 Logs Insights Query

Embed a live Logs Insights query result directly on the dashboard.

🔗 Sharing Dashboards

CloudWatch Dashboards can be shared outside your AWS account:

Shared Dashboards: Make a dashboard publicly accessible with a URL. Anyone with the link can view (read-only) without an AWS account. Great for sharing metrics with managers or stakeholders.
Account-level sharing: Share with specific AWS accounts in your organization.
Cross-account/Cross-region: View metrics from multiple accounts and regions in one dashboard.

🎯 Exam Tip

If a question asks "how do you show CloudWatch metrics to someone without an AWS account" → answer is Shared Dashboard (public sharing). Not IAM user, not cross-account role.

🔔 CloudWatch Alarms — Automated Alerting and Actions

An Alarm monitors a single metric and takes action when that metric crosses a threshold you define. This is how you get notified when something breaks — and even auto-fix it.

🚦 The Three Alarm States

✅ OK

The metric is within the threshold you defined. Everything is fine.

Example: CPU is 45%, alarm threshold is > 80%

🚨 ALARM

The metric has exceeded the threshold. Time to act!

Example: CPU has been > 80% for 5 consecutive minutes

❓ INSUFFICIENT_DATA

Not enough data points to evaluate. Common for new alarms or metrics that don't fire often.

Example: Alarm just created, no data yet

⚙️ Alarm Configuration — Key Settings

Threshold

The value that triggers the alarm. Example: "CPU > 80%" or "Error count > 100". You choose greater than, less than, or equal to.

Evaluation Period

How many consecutive data points must cross the threshold before the alarm triggers. Example: "3 out of 3 data points" = the threshold must be exceeded 3 times in a row (prevents false alarms from momentary spikes). "1 out of 1" = alarm triggers immediately on first breach.

Period

How long each data point represents. Example: 5-minute period means each data point is the average over 5 minutes. With 3 evaluation periods, the alarm only fires if the threshold is exceeded for 15 continuous minutes.

🎬 Alarm Actions — What Happens When Alarm Triggers?

When an alarm goes to ALARM state (or back to OK state), you can automatically:

📢 Send SNS Notification

SNS (Simple Notification Service) then delivers the alert to:

Email addresses
SMS messages
Lambda functions (for automated remediation)
SQS queues
HTTP endpoints (Slack, PagerDuty, etc.)

📐 Auto Scaling Action

Trigger EC2 Auto Scaling to add or remove instances. Example: CPU > 70% → scale out (add more EC2 instances). CPU < 30% → scale in (remove instances). This is how AWS auto-scaling works behind the scenes.

🖥️ EC2 Action

Directly act on the EC2 instance: Stop, Terminate, Reboot, or Recover. Example: "If status check fails twice in a row → reboot the instance".

🔧 Systems Manager Action

Create an OpsItem in Systems Manager OpsCenter for incident tracking and automated runbooks.

🔀 Composite Alarms — AND/OR Logic

A regular alarm watches one metric. A Composite Alarm combines multiple alarms using AND/OR logic. This reduces "alarm noise" (getting paged when only one of many metrics spikes).

📌 Real-World Example

Problem: Your app team gets paged at 3am every time CPU spikes briefly, even if the app is fine.
Solution: Create a Composite Alarm:
Alert ONLY IF (CPU > 80% AND Error Rate > 5% AND Response Time > 2 seconds).
Now you only get paged when the application is actually in trouble, not just during a temporary CPU spike from a batch job.

🎯 Exam Tip

When a scenario asks about "reducing alarm noise" or "only alert when multiple conditions are true simultaneously" → answer is Composite Alarm.

🔢 Embedded Metric Format (EMF) — Metrics From Logs

EMF is a way to create CloudWatch Metrics by writing structured JSON logs. You don't call the CloudWatch API directly — instead you write a specially formatted JSON log, and CloudWatch automatically extracts and creates metrics from it.

🤔 Why Use EMF Instead of PutMetricData?

📞 Regular PutMetricData API

Makes a separate HTTP API call to CloudWatch
Adds latency to your Lambda function execution
Two separate systems: logs AND metrics
Logging and metric publishing are separate operations

📝 Embedded Metric Format (EMF)

Just write a JSON log line — no extra API call!
CloudWatch processes it asynchronously in background
One operation = both a log entry AND a metric
Same data is searchable as logs AND graphable as metric

📋 EMF JSON Format

        JSON (EMF)
{
    // "_aws" is the required EMF metadata block
    "_aws": {
        "Timestamp": 1574109600000,
        "CloudWatchMetrics": [{
            "Namespace": "MyApp/OrderService",
            "Dimensions": [["ServiceName", "Environment"]],
            "Metrics": [
                { "Name": "ProcessingTime", "Unit": "Milliseconds" },
                { "Name": "OrdersProcessed", "Unit": "Count" }
            ]
        }]
    },
    // Dimension values
    "ServiceName":   "OrderProcessor",
    "Environment":    "Production",
    // Metric values
    "ProcessingTime": 234,
    "OrdersProcessed": 1,
    // Any extra data (searchable in logs but not a metric)
    "orderId": "ORD-12345",
    "userId":  "user-789"
}
      

When CloudWatch receives this log, it:

Stores the entire JSON as a log event (searchable with Logs Insights)
Automatically extracts ProcessingTime and OrdersProcessed as CloudWatch Metrics in namespace MyApp/OrderService

Lambda Powertools — EMF Made Easy

Writing EMF JSON manually is tedious. AWS Lambda Powertools is a library that makes structured logging and EMF very easy with just a few lines of code:

        Python — Lambda Powertools
from aws_lambda_powertools import Metrics, Logger
from aws_lambda_powertools.metrics import MetricUnit

metrics = Metrics(namespace="MyApp", service="OrderService")
logger  = Logger(service="OrderService")

@metrics.log_metrics           # auto-flushes EMF at end
def lambda_handler(event, context):
    metrics.add_metric(
        name="OrdersProcessed",
        unit=MetricUnit.Count,
        value=1
    )
    logger.info("Order processed", orderId=event["orderId"])
      

🎯 Exam Tip

EMF is the answer when: "Lambda needs to publish metrics WITHOUT making extra API calls" or "combine logging and metrics in one operation". The key benefit is no additional API calls from Lambda = lower latency + simpler code.

🤖 CloudWatch Synthetics — Test Your APIs 24/7

CloudWatch Synthetics lets you create canaries — automated scripts that run on a schedule and mimic what a real user would do. They test your APIs, websites, and workflows even when no one is actually using them.

Canary (in CloudWatch Synthetics)

A script (written in Node.js or Python) that runs on a schedule (every 1 min, 5 min, etc.) and checks if your endpoint is working. Named after the "canary in a coal mine" — it detects problems before your users do. If the canary script fails, CloudWatch raises an alarm.

🎯 What Can Canaries Test?

🔗 API Endpoint Monitoring

Make HTTP GET/POST requests to your API. Check the response code (expect 200), check the response body contains expected data, check the response time is below a threshold.

🖥️ Website/UI Monitoring

Use headless Chrome to load a web page, click buttons, fill forms (like a login or checkout flow). Check that each step works correctly and within time limits.

📡 Availability Monitoring

Continuously verify that an endpoint is reachable and responding. Alert the team before users even notice a problem.

⏱️ Performance Baselines

Track how long your API takes to respond over time. Detect performance regressions after deployments.

📊 What Canaries Produce

CloudWatch Metrics: Success rate, response time, failure count
CloudWatch Logs: Detailed execution logs
Screenshots: Visual screenshots of the page state when a test fails
Alarms: Can trigger SNS notifications when canary fails

🎯 Exam Tip

Synthetics is the answer for: "continuously test an API endpoint", "detect issues before users do", "synthetic monitoring", "scheduled API health checks". Don't confuse with CloudWatch Logs Insights (which queries existing logs) or CloudWatch Alarms (which react to metrics).

🔭 CloudWatch ServiceLens — End-to-End Observability

ServiceLens is a CloudWatch feature that combines CloudWatch metrics/logs/alarms with AWS X-Ray traces. It gives you a complete picture of your application's health in one place.

🔗 What ServiceLens Connects

📊CloudWatch MetricsPerformance numbers

+

📝CloudWatch LogsApplication text output

+

🔍X-Ray TracesRequest flows

↓ All in one place ↓

🔭 ServiceLens End-to-end observability

💡 Key ServiceLens Capabilities

Service Map: Visual graph of your application services with health indicators
Drill down: Click on an alarm → see the X-Ray trace that caused it
Correlated data: For any time period, see related metrics, logs, AND traces together
Anomaly detection: See which services have unusual patterns

🎯 Exam Tip

ServiceLens = "end-to-end observability" = CloudWatch + X-Ray integrated. It's the answer for "unified view of application health" or "correlate metrics with traces".

🚦 API Gateway Metrics — What to Monitor

API Gateway automatically sends important metrics to CloudWatch. These are tested frequently in the exam.

⏱️

Latency

Total time from request received by API Gateway to response sent back to client. Includes all processing (auth, backend call, response transform).

🔗

IntegrationLatency

Time spent waiting for the backend (Lambda, HTTP, etc.) to respond. Does NOT include API Gateway overhead. If Latency is high but IntegrationLatency is low → bottleneck is inside API Gateway.

✅

CacheHitCount

Number of requests served from API Gateway's cache (no backend call made). High hit count = cache is working well = low cost, low latency.

❌

CacheMissCount

Number of requests that went to the backend because the response wasn't in cache. High miss count = cache is not helping, check TTL settings.

⚠️

4XXError

Client-side errors (bad request, unauthorized, not found). Spike in 4XX = clients sending wrong requests or authentication issues.

🔴

5XXError

Server-side errors (Lambda errors, timeout, backend failure). Spike in 5XX = backend is broken. Check Lambda logs.

🎯 Critical Exam Pattern

Latency is high but IntegrationLatency is low → The problem is INSIDE API Gateway (not in Lambda/backend). Could be: SSL handshake, request/response transformation, WAF, authorizer latency.

Latency is high AND IntegrationLatency is high → The backend (Lambda, HTTP) is slow. Focus optimization there.

📋 Monitoring KPIs — What to Track in Practice

KPI stands for Key Performance Indicator — a specific, measurable number that tells you if your application is meeting its goals. The exam may ask what to monitor for various use cases.

Service	Key Metrics to Monitor	What to Alarm On
Lambda	Errors, Duration, Throttles, ConcurrentExecutions, IteratorAge (for streams)	Errors > 0, Duration approaching timeout, Throttles > 0
API Gateway	Latency, IntegrationLatency, 5XXError, 4XXError, CacheHitCount	5XX rate > 1%, Latency p99 > 5 seconds
DynamoDB	ConsumedReadCapacityUnits, ConsumedWriteCapacityUnits, ThrottledRequests, SuccessfulRequestLatency	ThrottledRequests > 0, Latency spike
SQS	ApproximateNumberOfMessagesVisible (queue depth), ApproximateAgeOfOldestMessage	Queue depth growing, messages too old
RDS	DatabaseConnections, CPUUtilization, FreeableMemory, ReadIOPS, WriteIOPS	Connections near max, CPU > 80%
EC2	CPUUtilization, NetworkIn/Out, StatusCheckFailed	CPU > 80%, Status check failed

💡 Best Practice

For SQS-based architectures, always monitor ApproximateAgeOfOldestMessage. If messages are sitting in the queue for hours, your consumers are too slow. Trigger alarms and scale out your consumer Lambda/EC2 fleet when this metric spikes.