π¨ SQS & SNS
Master SQS message queuing, SNS pub/sub, and fan-out patterns.
Module: AWS Data Engineering β’ Topic 49 of 65 β’ Premium Content
SQS & SNS Architecture
Architecture Diagram
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β SQS & SNS ARCHITECTURE β
β β
β SQS (Queue): Producer β Queue β Consumer (one message = one consumer) β
β SNS (Topic): Publisher β Topic β Multiple Subscribers β
β β
β Fan-out Pattern: β
β Publisher β SNS Topic β SQS Queue 1 β Consumer A β
β β SQS Queue 2 β Consumer B β
β β SQS Queue 3 β Consumer C β
β β
β Dead Letter Queue (DLQ): β
β Failed messages β DLQ for debugging and reprocessing β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Interview Q&A
Q1: SQS vs SNS?
Answer: SQS: point-to-point queue, one consumer per message. SNS: pub/sub topic, multiple subscribers. Use together for fan-out.
Q2: What is a Dead Letter Queue?
Answer: A queue that receives messages that fail processing after max retries. Enables debugging and reprocessing.
Q3: What are FIFO queues?
Answer: SQS FIFO queues provide exactly-once processing and ordering. Use for transactional data processing.
Summary
- SQS: Point-to-point queue, at-least-once delivery
- SNS: Pub/sub topic, multiple subscribers
- Fan-out: SNS + SQS for parallel processing
- DLQ: Capture failed messages for debugging
- FIFO: Exactly-once processing and ordering