Amazon SQS for system design interviews

13 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

Amazon SQS is a fully managed queue behind an HTTPS API. You SendMessage, consumers ReceiveMessage, and a message stays in the queue until someone calls DeleteMessage with the receipt handle from their receive.
The visibility timeout is the whole delivery model: a received message is hidden, not removed, and reappears if it is not deleted in time. That gives at-least-once delivery with no consumer registration at all.
Standard queues scale almost without limit but may deliver twice and out of order. FIFO queues give ordering per message group and deduplication, at lower throughput.
There are no servers to size. You pay per request (a 64 KB chunk counts as one request), and the operational work moves to choosing timeouts, batching, dead-letter queues and idempotent consumers.
Choose it for decoupling AWS services and smoothing load spikes. Avoid it when you need replay, fan-out to many consumers (pair it with SNS or EventBridge), or sub-10 ms latency.

How it works

The message lifecycle

A producer sends a message of up to 1 MiB, raised from 256 KiB on 4 August 2025 (AWS announcement). SQS stores it redundantly before returning a message ID. A consumer calls ReceiveMessage for up to 10 messages. Each returned message carries a receipt handle, which is tied to that particular receive. The message becomes invisible for the queue's visibility timeout (default 30 seconds, maximum 12 hours) (SQS message quotas). If the consumer finishes, it calls DeleteMessage with the receipt handle. About 120,000 messages can be in flight per queue; past that, short polls get OverLimit and long polls simply return nothing (queue quotas). If a consumer crashes or runs long, the timeout expires and the message becomes visible again for another consumer, with its ApproximateReceiveCount incremented. A consumer that needs more time calls ChangeMessageVisibility to extend the timer.

Polling

Consumers pull. Short polling (WaitTimeSeconds=0) queries a subset of SQS's storage hosts and returns immediately, so it can return nothing even when messages exist. Long polling (up to 20 seconds) waits until a message arrives or the wait expires and queries all hosts, which cuts empty responses and cost (polling docs). Long polling should be the default.

Failure handling

A redrive policy names a dead-letter queue and a maxReceiveCount. When a message has been received that many times without being deleted, SQS moves it to the DLQ rather than delivering it again, so one poison message cannot consume your fleet forever. Messages expire after the retention period (1 minute to 14 days, default 4 days). Delay queues and per-message timers hide new messages for up to 15 minutes. To recover, a redrive task (StartMessageMoveTask) moves DLQ messages back to the source or another queue of the same type, optionally capped at up to 500 messages per second; moved messages get new IDs and a fresh retention clock, and nothing can be filtered or edited on the way (DLQ redrive).

FIFO queues

A FIFO queue requires a MessageGroupId. Messages in the same group are delivered in order, and while any message of a group is in flight, no later message of that group is handed out, which is how ordering survives multiple consumers. A MessageDeduplicationId (or content-based hashing) suppresses duplicates sent within a 5-minute window. Different groups proceed in parallel, so the group ID is your unit of both ordering and parallelism.

Fair queues

Since July 2025 a MessageGroupId also means something on a standard queue: it names a tenant, not an ordering group. SQS watches how in-flight messages split across tenants, and when one tenant holds a disproportionate share and builds a backlog, it serves quiet tenants' messages first. The noisy tenant is not rate-limited, consumers need no changes, and throughput stays standard-queue unlimited (fair queues docs, AWS Compute blog). This is the noisy-neighbour shape, solved inside the queue instead of with one queue per tenant.

Down the stack

Layer 1The contract

Standard queues guarantee at-least-once delivery and best-effort ordering: an acknowledged send is delivered one or more times until deleted or expired, possibly out of order and occasionally duplicated even when consumers behave perfectly, because a storage server holding one copy can be unreachable when you delete (at-least-once delivery). FIFO queues drop sends that repeat a deduplication ID within 5 minutes and keep strict order within a message group, but a consumer that overruns its timeout still causes redelivery. The visibility timeout is a lease, not a lock. Standard throughput is nearly unlimited; FIFO allows 300 calls per second per action per partition (3,000 messages with batches of 10). High-throughput mode raises that to 70,000 calls per second per action in N. Virginia, Oregon and Ireland (700,000 messages batched), but only 2,400 in many smaller regions (SQS message quotas).

Layer 2The mechanism

AWS publishes little about SQS internals, so treat this as a model consistent with public material, not a spec. A front-end fleet terminates HTTPS, verifies the SigV4 signature, checks quotas and routes to back-end storage hosts that hold partitions of the queue. A message is written to several hosts across availability zones before the send returns. Standard queues spread messages over many partitions with no global order, which is why throughput scales and why a receive can see a later message first, or a delete racing a host failure can leave a copy behind.

Each message is visible, in flight with an invisible_until timestamp, or deleted. A receive picks visible messages, sets invisible_until = now + timeout, increments the receive count and mints a fresh receipt handle. DeleteMessage and ChangeMessageVisibility must present the latest handle; an older one names a superseded lease. The redrive check runs at receive time: if the count already equals maxReceiveCount, the message moves to the DLQ.

FIFO queues partition by a hash of the group ID. Only the head of each group is deliverable, and the group stays locked while it has a message in flight. High-throughput FIFO spreads groups over more partitions, so it only helps with many distinct group IDs.

Since November 2023 the SDKs can speak the AWS JSON protocol instead of query/XML, which AWS measured at up to 23% lower end-to-end latency for 5 KB messages (AWS announcement). Responses carry MD5OfMessageBody for verification. Every 64 KB of payload is billed as one request, so a 1 MiB message costs 16 (SQS pricing).

Layer 3The machine

Every call is a TLS-protected HTTP request, so cost per message depends on batching and connection reuse; the SDK connection pool often caps throughput before SQS does. With 15 ms per receive and 10 messages per call, one polling thread moves about 650 messages per second, so 20,000 per second needs about 30 pollers plus batched deletes.

Long polling holds the connection for up to 20 seconds, so client read timeouts must exceed the wait or the client aborts healthy polls and retries. Bodies are text, so binary payloads are base64-encoded, inflating them by a third; payloads above the limit go to S3 with a pointer in the message. At roughly $0.40 per million requests (hedge: check current pricing), a tight short-polling loop on an empty queue can cost more than the real traffic.

Why it's built this way

AWS wanted a queue that never needs capacity planning, so messages are spread across many partitions with no global coordination.
Without global coordination there is no cheap global order or exactly-once delivery, so standard queues offer best effort and push idempotency to consumers.
Consumers fail in unknown ways, so instead of tracking consumers SQS uses a lease (visibility timeout) that expires on its own; no heartbeats or sessions are required.
Some workloads need order, so FIFO restricts ordering to a group and locks the group during processing, trading throughput for a guarantee scoped small enough to partition.
HTTPS calls cost milliseconds and money, so the API batches 10 messages per call and long polling replaces busy loops.

Diagram

stateDiagram-v2
    [*] --> Visible: SendMessage
    Visible --> InFlight: ReceiveMessage (lease starts)
    InFlight --> Visible: timeout expires
    InFlight --> InFlight: ChangeMessageVisibility
    InFlight --> Deleted: DeleteMessage (latest receipt)
    Visible --> DLQ: receive count = maxReceiveCount
    Visible --> Expired: retention period passes
    Deleted --> [*]

Worked example

A video platform enqueues a transcoding job for each upload: 2 million uploads per day, peaking at 120 per second. Each job takes 30 seconds to 8 minutes.

A 10-minute visibility timeout would cover the worst job, but a crashed worker's job would then wait 10 minutes for retry. Instead the timeout is 2 minutes and each worker extends it by 2 minutes every 60 seconds while working, so a crash delays retry by at most 2 minutes.

Workers long-poll with MaxNumberOfMessages=1, because receiving 10 long jobs would hide 9 behind one busy worker. At 3 minutes average and 120 jobs per second, peak needs about 120 x 180 = 21,600 concurrent workers, or a backlog that drains later; queue depth drives autoscaling. Each job costs about 6 requests (send, receive, three extensions, delete), so 12 million requests a day, around $5. maxReceiveCount=3 sends corrupt uploads to a DLQ, and output keys derive from the upload ID, so a duplicate delivery overwrites the same file.

In the wild

AWS, 2015: the queue survives, its metadata cache does not. On 20 September a DynamoDB metadata overload in US-East disabled DynamoDB traffic at 5:45 a.m. PDT. SQS keeps queue metadata in an internal DynamoDB table; sends and receives did not read it directly, but they relied on frequently refreshed caches of it, so error rates rose sharply until DynamoDB returned at 7:10 a.m. No queued data was lost, and AWS said it would adjust SQS metadata caching so sends and receives keep working without the table (AWS summary).
AWS, 2025: the pollers, not the queue, stall. During the 19 to 20 October US-East outage, triggered by an empty DNS record for DynamoDB's regional endpoint, Lambda's internal subsystem that polls SQS queues failed and did not recover on its own. It was restored at 4:40 a.m. PDT and backlogs cleared by 6:00, then Lambda throttled event source mappings to protect synchronous calls and only drained everything by 2:15 p.m. Queue depth and oldest-message age were the signals that mattered (AWS summary).
Dropbox, 2020: SQS as the buffer in a task framework. Dropbox's Async Task Framework stores each task with its trigger time, polls for due tasks every two seconds, and pushes them onto one SQS queue per (callback, priority) pair. It served about 9,000 tasks per second for 28 teams with at-least-once execution, and 95% of tasks started within 5 seconds of schedule (Dropbox Tech).

Trade-offs

Option	Strengths	Weaknesses	Choose when
SQS standard	Effectively unlimited throughput, zero ops	Duplicates, no ordering, no replay	Background jobs and buffering on AWS
SQS FIFO	Ordered per group, dedup window	Lower throughput, group head-of-line blocking	Per-entity ordered commands
SNS + SQS	Fan-out to many queues	Two services to configure, still no replay	One event, many consumers
Kinesis or Kafka (MSK)	Replay, ordered shards, high throughput	Capacity planning, consumers manage offsets	Event streams, analytics
RabbitMQ (Amazon MQ)	Routing, push delivery, low latency	You size and operate brokers	Complex routing, protocol compatibility

Pitfalls in production

Timeout shorter than processing. Jobs run twice and the queue never drains. Set it above p99 or extend with a heartbeat.
No DLQ. A poison message retries until retention expires. Attach a redrive policy and alarm on DLQ depth.
Assuming FIFO means exactly-once side effects. Deduplication only covers sends. Make handlers idempotent.
Hot message group. One group ID serialises the whole FIFO queue. Use entity-level group IDs.
Lambda pollers flooding a database. Cap maximum concurrency on the event source mapping.

Leadership lens

Ask "what if this message arrives twice, or an hour late?" for every queue; the answer defines idempotency and timeouts.
Make DLQs and a redrive runbook, with an owner, part of the definition of done.
Prefer standard queues plus idempotency unless ordering is a real requirement.
Alarm on age of oldest message, which maps to user-visible delay, not just depth.

Check yourself

Why can a standard queue deliver a message twice even when every consumer deletes within the timeout?
Show answer
Messages are stored on several hosts without global coordination, so a delete can miss a copy or a receive can race it during failures; at-least-once is the contract.
A consumer deletes using a receipt handle from its first receive, after the message has been received again elsewhere. What should you expect?
Show answer
The handle refers to a superseded lease, so the delete may not take effect and the second consumer will process the message too.
Why does a FIFO queue with one message group not benefit from more consumers?
Show answer
Only one message from a group can be in flight at a time, so the group is processed serially regardless of consumer count.
Why set MaxNumberOfMessages=1 for long-running jobs?
Show answer
Receiving 10 hides all 10 behind one worker's lease while other workers sit idle, and their timeouts may expire before the worker reaches them.

Sources and further study

Primary

ReceiveMessage API reference: the exact parameters (wait time, visibility, attempt IDs for FIFO) behind the lease model.
StartMessageMoveTask API reference: the redrive call and its rate limit.

Systems docs

SQS fair queues: noisy-neighbour detection on standard queues and its CloudWatch metrics.
SQS message quotas: current size, retention, timer and FIFO throughput limits by region.
Using Lambda with Amazon SQS: how the event source mapping polls, batches, scales and reports partial failures.

Books

Understanding Distributed Systems, 2nd ed., Vitillo, Ch 23 "Messaging" (at-least-once delivery, idempotent consumers, dead-letter channels).
Designing Distributed Systems, 2nd ed., Burns, Ch 11 "Work Queue Systems" (a generic work queue and dynamic scaling of workers).

Other curricula

Azure Architecture Center: Queue-Based Load Leveling: frames a queue as a buffer that lets producers and a slow service run at different rates.
AWS Builders' Library: Timeouts, retries and backoff with jitter: why retries, including queue redeliveries, need idempotency and backoff.
system-design-primer: places SQS-style queues under asynchronism, with back pressure as the safeguard.

Watch

AWS re:Invent 2019: Scalable serverless event-driven applications using Amazon SQS & Lambda (AWS Events): visibility timeouts, batching, DLQs and how the Lambda poller scales against a queue.
AWS re:Invent 2021: Building modern cloud applications? Think integration (AWS Events): Gregor Hohpe on where queues fit among integration patterns.

Free resources

Amazon SQS visibility timeout: the lease model, in-flight limits and how to extend it.
Short and long polling: why short polls sample servers and when they return empty.
FIFO queues: message groups, deduplication and ordering rules.
High throughput for FIFO queues: how partitioning by group ID raises FIFO limits.
Dead-letter queues: redrive policy and redriving messages back.
Avoiding insurmountable queue backlogs (Amazon Builders' Library): how AWS teams design queue-based systems that recover from backlogs.

Open in the interactive app: practice, quiz, mock interview →