Designing clean REST APIs

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

REST models a system as resources named by URLs and manipulated through a small set of HTTP methods with defined semantics.
Method semantics (safe, idempotent) are a contract that caches, proxies and retry logic rely on, so honour them.
Good APIs are boring and consistent: plural nouns, predictable status codes, one error shape, one pagination style.
The hard parts are evolution, pagination over changing data, retries of non-idempotent writes, and long-running operations.
An API outlives the code behind it; design for the client three versions from now.

How it works

Resources and methods

A resource is a noun with an identity: /orders/8812, /orders/8812/items. Collections take GET (list) and POST (create). Members take GET, PUT (replace), PATCH (partial update) and DELETE. GET, HEAD, OPTIONS and TRACE are safe: the client asks for no state change, so crawlers and prefetchers can call them. PUT, DELETE and the safe methods are idempotent: repeating them has the same intended effect as doing them once, so clients may retry after a connection drops (RFC 9110). POST is neither, which is why creating things needs extra care. PATCH is neither by default either (RFC 5789), and its body format is a choice you must document: JSON Merge Patch (RFC 7396) sends just the changed fields and uses null to delete, which breaks if null is a real value; JSON Patch (RFC 6902) sends an ordered list of operations (Azure guide).

Actions that do not fit CRUD ("cancel an order") are modelled either as a state change (PATCH /orders/8812 {"status":"cancelled"}) or as a sub-resource (POST /orders/8812/cancellation). Pick one convention and use it everywhere.

Searches with large filter objects used to be forced into POST, which gave up caching and safe retries. RFC 10008 (June 2026) adds QUERY: the filter goes in the body, the request is safe and idempotent, and caches may store the response under a key that includes the body (RFC 10008).

Status codes and errors

Use the code classes properly: 2xx success, 4xx the client must change something, 5xx the server failed and a retry may help. The distinctions that matter operationally: 400 vs 422 (malformed vs semantically invalid; 422 began in WebDAV and is now part of core HTTP as "Unprocessable Content" in RFC 9110), 401 vs 403 (not authenticated vs not allowed), 409 (conflict with current state), 429 (rate limited, with Retry-After), 503 (temporarily unavailable). Return a single error body shape everywhere, such as RFC 9457 problem details with a machine-readable type, so clients can branch on it.

Idempotency for POST

A client creating a payment times out. Did it succeed? Without help, retrying may charge twice. The standard fix is an Idempotency-Key header: the server stores the key with a fingerprint of the request and the response, and replays the stored response for a repeat. The IETF draft that standardises the header says a missing key gets 400, a reused key with a different body gets 422, and a retry that arrives while the first attempt is still running gets 409; the draft (revision 07, October 2025) expired without becoming an RFC, so treat it as convention (IETF draft). Stripe prunes keys after 24 hours and saves a result only once the endpoint starts executing, so a request that failed validation can simply be retried (Stripe docs). AWS adds a subtle rule: the replay should be semantically the same as the first success, not an "already exists" error that sends the caller down a different code path (AWS Builders' Library).

Pagination

Offset pagination (?offset=10000&limit=50) is simple but the database still walks the skipped rows, and inserts shift pages so clients see duplicates or gaps. Cursor pagination returns an opaque token encoding the last seen sort key (created_at, id), and the next query uses WHERE (created_at, id) > (...), which uses an index and is stable under inserts. Google's API rules add two details worth copying: tokens must be opaque (base64 of readable JSON does not count), and an oversized page size is quietly capped to the maximum instead of rejected (AIP-158).

Concurrency and caching

ETag plus If-None-Match lets clients revalidate cheaply (304 Not Modified). If-Match on writes gives optimistic concurrency: the update succeeds only if the resource has not changed, otherwise 412 Precondition Failed. This prevents lost updates without locks.

Long-running operations

Some writes take minutes, such as an export or a bulk refund, and holding the request open hits proxy timeouts. The usual shape is to validate, then answer 202 Accepted with a Location header pointing at a status resource and a Retry-After hint. The status resource returns 200 with a state such as running or failed while work continues, and 303 See Other to the result when it is done, which forces the client's next request to be a GET. A DELETE on the status resource cancels (Azure pattern). Combine it with an idempotency key so a retried submit returns the existing operation rather than starting a second one.

Versioning

Additive changes (new optional fields, new endpoints) are non-breaking if clients ignore unknown fields. Removing or renaming fields, changing types, or tightening validation are breaking. Version in the URL (/v2/), a query string, a header or a media type; URL and query-string versions are the cache-friendly ones because the same URL always means the same representation (Azure guide). Support old versions with a published deprecation window.

Down the stack

Layer 1The contract

HTTP semantics (RFC 9110) are the contract every client, proxy and cache relies on. GET and HEAD are safe; PUT and DELETE are idempotent; a 2xx means the method's effect happened; a 5xx or a timeout means the outcome is unknown. Conditional requests are precise: a write with If-Match: "v7" either applies to version 7 or fails with 412 and changes nothing. An idempotency key extends the contract to POST: for 24 hours, the same key and the same body produce the same result exactly once, and a different body under that key is rejected.

Layer 2The mechanism

A request is a method, a target, header fields and an optional body. The server derives an ETag from a version column or a content hash (strong, or weak with a W/ prefix when only semantic equality is promised). Optimistic concurrency becomes one SQL statement, UPDATE orders SET ... , version = version + 1 WHERE id = $1 AND version = $2; zero rows updated means 412. The idempotency store is a table with a unique key column, a request fingerprint, a state (in_progress or complete), the stored status and body, and a lock timestamp. A second request arriving while the first is in progress gets 409, so the client backs off rather than racing. Cursors are a base64url encoding of the last row's sort key with an HMAC, and the next page runs WHERE (created_at, id) > ($1, $2) ORDER BY created_at, id LIMIT 100 against a composite index.

Layer 3The machine

A new HTTPS connection costs a TCP handshake plus a TLS 1.3 handshake, two round trips before the first byte of the request, so keep-alive and connection pools matter more than payload tuning on high-latency links. HTTP/1.1 headers often run 500 to 800 bytes per request; HTTP/2's HPACK replaces repeated headers with table indexes of a byte or two. On the database side, a Postgres B-tree leaf page is 8 KB and holds a few hundred index entries, so OFFSET 1500000 walks around 5,000 leaf pages, each a possible 100-microsecond SSD read if not cached. Keyset pagination descends three or four levels of the tree and reads one or two leaf pages, whatever the depth.

Why it's built this way

Networks drop responses, so clients cannot tell failure from success, and retries must be made safe by method semantics or idempotency keys.
Caches and proxies sit between clients and servers and know nothing about your app, so a uniform interface with standard meanings lets them help.
HTTP requests are stateless and long locks across them are impossible, so concurrency control uses validators like ETags instead.
Databases read fixed-size pages and cannot skip to row N, so deep offsets cost linear I/O and cursors use the index directly.
Clients upgrade slowly (mobile apps live for years), so APIs evolve additively and version only when they must break.

Diagram

sequenceDiagram
    participant C as Client
    participant A as API
    participant S as "Idempotency store"
    participant D as Database
    C->>A: POST /payments, Idempotency-Key k1
    A->>S: lookup k1
    S-->>A: not found, reserve k1
    A->>D: insert payment
    A->>S: save response for k1
    A--xC: 201 lost in network
    C->>A: retry POST, Idempotency-Key k1
    A->>S: lookup k1
    S-->>A: stored 201
    A-->>C: 201 replayed, no second charge

Worked example

A marketplace exposes GET /orders?seller_id=... to 3,000 sellers, some with 2 million orders.

With offset pagination, a seller exporting all orders at 100 per page makes 20,000 requests; page 15,000 makes the database scan and discard 1.5 million index entries, so late pages take 800 ms instead of 5 ms and the export takes hours while hammering the primary. Switching to a cursor on (created_at, id) makes each page an index range scan of 100 rows, around 3 ms regardless of depth. The cursor is base64 of the last key, signed so clients cannot forge it.

For writes, the team adds idempotency keys to POST /payments. At 400 payments per second and a 24-hour key retention, the store holds about 35 million keys; at roughly 300 bytes each (key, fingerprint, response) that is about 10 GB, comfortably in a Redis cluster or a database table with a TTL index.

In the wild

T-Mobile, 2023: one API, 37 million accounts. On 5 January T-Mobile found that an attacker had been pulling data through a single API without authorization since about 25 November 2022. Names, billing addresses, emails, phone numbers, dates of birth, account numbers and plan details for about 37 million accounts were taken; the activity was stopped within a day of detection (SEC 8-K). Six weeks of bulk reads went unnoticed, which is an argument for per-caller rate and volume alerts on every endpoint.
USPS, 2018: any logged-in user could read any account. An API behind usps.com let any authenticated user query other users' account details, about 60 million in all, and many calls accepted wildcard parameters that returned whole data sets. A researcher's report went unanswered for over a year; USPS fixed the API on 20 November 2018 once a journalist contacted it (KrebsOnSecurity). Authentication without per-object authorization is the classic REST failure.
Stripe, 2017: idempotency keys as a design write-up. Stripe explains why a charge must happen once even when the network loses the response, and how keys plus exponential backoff with jitter make retries safe; its client libraries generate keys automatically (Stripe).

Trade-offs

Option	Strengths	Weaknesses	Choose when
Offset pagination	Simple, jump to page N	Slow deep pages, unstable under writes	Small admin lists
Cursor pagination	Fast at any depth, stable	No random page access	Feeds, exports, large tables
URL versioning	Visible, easy to route and cache	Duplicate surface per version	Public APIs with many clients
Header versioning	Clean URLs, per-request choice	Harder to test and cache	Internal APIs with capable clients
PATCH partial update	Small payloads, fewer conflicts	Merge semantics need defining	Large resources edited piecemeal

Pitfalls in production

200 with an error body. Monitoring and clients treat it as success. Use proper status codes.
Unbounded list endpoints. One call returns 500,000 rows and takes down the service. Enforce a maximum page size.
Retrying POST blindly. Duplicate orders and charges. Require idempotency keys on side-effecting creates.
Missing object-level authorization. The endpoint checks that the caller is logged in but not that order 8812 belongs to them, so changing the ID in the URL reads someone else's data. It is number one on the OWASP API Security Top 10 (2023) (OWASP). Check ownership on every object access and test it with two accounts.
Leaking internal models. Database column names in responses freeze your schema. Map to explicit API types.
Silent breaking changes. Tightened validation rejects requests that worked yesterday. Run contract tests against recorded client traffic.

Leadership lens

Publish an API style guide (naming, errors, pagination, versioning) and lint OpenAPI specs in CI.
Ask in review: is this endpoint safe to retry, and what does the client do on a timeout?
Decide a deprecation policy up front, with usage metrics per version and client.
Treat the API spec as the contract and generate clients from it.

Check yourself

Why is PUT /users/7 safe to retry but POST /users not?
Show answer
PUT sets the resource to a given state, so repeating it yields the same state. POST creates a new resource each time unless the server deduplicates with an idempotency key.
Why does cursor pagination avoid duplicates when new rows are inserted?
Show answer
The next page starts strictly after the last seen key, so inserts elsewhere do not shift which rows come next.
What does If-Match prevent?
Show answer
Lost updates: a write based on a stale version fails with 412 instead of overwriting someone else's change.

Sources and further study

Primary

RFC 10008: The HTTP QUERY Method: the new safe, cacheable method for queries with a body.
IETF draft: The Idempotency-Key HTTP Header Field: the status codes for missing, mismatched and concurrent keys.
RFC 5789: PATCH Method for HTTP: why PATCH is not idempotent unless you make it so.

Systems docs

Stripe: Idempotent requests: key length, retention and exactly which results are replayed.
Google AIP-158: Pagination: page tokens, page-size caps and consistency rules.
OWASP API Security Top 10 (2023): the most common ways real APIs are broken, led by object-level authorization.

Books

Understanding Distributed Systems (2nd edition), Roberto Vitillo, chapter "APIs": HTTP, REST and idempotency from a distributed-systems view.
Designing Distributed Systems (2nd edition), Brendan Burns, chapter 2 "Important Distributed System Concepts": APIs and RPCs, latency, reliability and idempotency.

Other curricula

AWS Builders' Library: Making retries safe with idempotent APIs: how EC2 uses client tokens and why the replay must look like the original success.
Azure: Asynchronous Request-Reply pattern: the 202, status-resource and 303 flow for long operations.
donnemartin/system-design-primer: a table of methods with their safe, idempotent and cacheable properties, plus REST versus RPC examples.

Watch

REST API Pros and Cons (Hussein Nasser): REST constraints and their practical consequences.
Top 10 REST API Design Pitfalls (Spring I/O): conference talk on common design mistakes.

Free resources

RFC 9110: HTTP Semantics: the authoritative definitions of methods, status codes and conditional requests.
Google API Design Guide: resource-oriented design at large scale.
Azure: Web API design best practices: practical guidance on naming, paging and versioning.
RFC 9457: Problem Details for HTTP APIs: a standard error body.
Designing robust APIs with idempotency (Stripe): how a payments company uses idempotency keys.
Fielding's dissertation, chapter 5: where REST was defined.

Open in the interactive app: practice, quiz, mock interview →