⭐⭐⭐⭐ Strong Emphasis in Optimization
📨 SNS + SQS Optimization
SNS (Simple Notification Service) and SQS (Simple Queue Service) are the backbone of event-driven architectures in AWS. Understanding their combined patterns — especially the fanout pattern and message filtering — is critical for optimization.
🔑 Covers: SNS Basics · SQS Basics · Fanout Pattern · Subscription Filter Policies · Event RoutingSNS — Simple Notification Service
Amazon SNS (Simple Notification Service)
A pub/sub (publish/subscribe) messaging service. Publishers send messages to a Topic. All subscribers to that topic receive a copy of the message. It's a one-to-many messaging system — one message, many recipients. Think of it like an email mailing list: you send one email, everyone on the list gets it.
📋 SNS Key Concepts
📌 Topic
A named endpoint that publishers send messages to. Think of it as a "channel". Example: "OrderEvents" topic receives all order-related events.
📡 Publisher
Anyone who sends a message to a topic. Could be: your Lambda function, API Gateway, S3 (on object upload), CloudWatch Alarm, or your application code.
👥 Subscriber
Anyone who wants to receive messages from a topic. Types: SQS queue, Lambda function, HTTP/HTTPS endpoint, email, SMS, mobile push. Each gets a COPY of every message.
SQS — Simple Queue Service
Amazon SQS (Simple Queue Service)
A message queue service. Producers send messages to a Queue. Consumers poll the queue to receive and process messages one at a time. Unlike SNS, a message is processed by ONLY ONE consumer. It's a one-to-one system for reliable, ordered (or unordered) message processing. Think of it like a ticket queue — one person takes one ticket.
⚙️ Key SQS Configuration
| Setting | Purpose | Default |
|---|---|---|
| Visibility Timeout | How long a message is hidden from other consumers after one consumer picks it up. If the consumer doesn't delete it before timeout, the message reappears. | 30 seconds |
| Message Retention | How long messages stay in the queue if not processed. | 4 days (max 14 days) |
| Dead Letter Queue (DLQ) | Where messages go after failing processing N times. Lets you investigate failed messages. | Not configured |
| Standard Queue | At-least-once delivery, best-effort ordering, unlimited throughput. | Default type |
| FIFO Queue | Exactly-once processing, strict ordering, limited to 3000 msg/sec. | Optional |
The SNS + SQS Fanout Pattern — Core Architecture
The fanout pattern is when one SNS message is "fanned out" to multiple SQS queues. Each queue has its own consumer processing the message independently. This decouples the publisher from each consumer.
SNS + SQS Fanout Pattern — One Event, Multiple Processors
Order ServicePublishes "OrderPlaced" event
↓ one publish call ↓
SNS Topic: OrderEvents
↓ copies sent to all subscribers ↓
SQS: Inventory QueueUpdate stock levels
SQS: Shipping QueueCreate shipment
SQS: Analytics QueueRecord for reporting
Email/SMSNotify customer
✅ Why Fanout is Better Than Direct Calls
❌ Without Fanout (Direct Calls)
- Order service calls inventory service directly
- Order service calls shipping service directly
- Order service calls analytics service directly
- If any downstream service is slow → order service is slow
- Adding a new subscriber = change order service code
- Tight coupling between all services
✅ With SNS + SQS Fanout
- Order service just publishes to SNS — one call
- SNS delivers to all queues asynchronously
- If inventory service is slow, order service is unaffected
- Adding new subscriber = just subscribe a new SQS queue to SNS
- Loose coupling — services are independent
SNS Subscription Filter Policies — Message Routing
By default, ALL subscribers to an SNS topic receive ALL messages. But what if you only want some subscribers to receive certain types of messages? Subscription filter policies solve this.
Subscription Filter Policy
A JSON policy attached to an SNS subscription that defines which messages the subscriber wants to receive. Messages that don't match the filter are NOT delivered to that subscriber — they're silently dropped. Each subscriber can have its own different filter.
📌 Example
You have an SNS topic that receives all order events. You have 3 SQS queues:• HighValueOrders queue: Only wants orders where
orderValue > 1000• NewCustomerOrders queue: Only wants orders where
customerType = "new"• UrgentOrders queue: Only wants orders where
priority = "urgent"Without filters, all 3 queues would receive ALL orders and have to filter them in code. With filter policies, SNS does the filtering — each queue only gets what it cares about.
JSON — SNS Filter Policy Examples
// Filter: Only receive messages where orderType = "PREMIUM" or "VIP"
{
"orderType": ["PREMIUM", "VIP"]
}
// Filter: Only receive messages where orderValue is >= 500
{
"orderValue": [{"numeric": [">=", 500]}]
}
// Filter: Only receive if region = "US" AND status = "COMPLETED"
{
"region": ["US"],
"status": ["COMPLETED"]
}
// In the message, these must be MESSAGE ATTRIBUTES (not in body)
// Publisher must set them:
{
"MessageAttributes": {
"orderType": { "DataType": "String", "StringValue": "PREMIUM" },
"orderValue": { "DataType": "Number", "StringValue": "750" }
}
}
🎯 Exam Tip
Filter policies filter based on Message Attributes (NOT the message body). The publisher must add attributes to their message. The subscriber sets the filter policy in their SNS subscription. When a scenario says "different consumers need different subsets of messages from the same SNS topic" → answer is Subscription Filter Policies.