Engineering Atlas › System Design Fundamentals

The client-server model

14 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Two roles, one direction of initiative

A client initiates; a server listens. A browser fetching /orders, a mobile app posting a payment, a Go service calling Postgres, and an AI agent calling a model API are all clients. The server publishes an address and a contract (HTTP routes, gRPC methods, the Postgres wire protocol) and answers whoever connects. The asymmetry matters more than the hardware: the same process can be a server to its callers and a client to its database.

This is the trust but verify problem shape. Anything the client sends, including prices and role flags, is a claim the server re-derives from its own state. Games show it in a second domain: a client reporting "I hit the target" invites cheating, so authoritative game servers simulate the shot themselves.

Thin and thick clients, two and three tiers

A thin client (a server-rendered page) mostly displays; a thick client (a mobile app, a single-page app) caches data, works offline and syncs later. Two-tier systems had desktop clients talking straight to a database; three-tier systems put an application server in between so business rules live in one place. Modern systems are N-tier, and each tier is a client of the next.

Stateless servers

If a server keeps a user's session in its own memory, that user is stuck to that instance, and a restart logs them out. If the request carries everything needed (a token, the resource ID) and durable state lives in a database or cache, any instance can serve any request. That is what lets a load balancer spread traffic freely and lets you deploy by replacing instances. "Stateless" means no conversation state between requests, not no state at all. Fielding's REST dissertation states the strict version: each request carries everything needed to understand it, and session state lives on the client, which buys visibility, easier recovery from partial failure and scalability (Fielding, section 5.1.3). Most production systems relax this by keeping sessions in a shared store, which keeps the instances interchangeable but makes that store a dependency of every request.

The connection is the hard part

Requests ride a TCP (or QUIC) connection wrapped in TLS, and opening one costs round trips, so clients reuse connections. HTTP/1.1 allowed pipelining several requests back to back, but responses had to return in order, so one slow response blocked the rest and no modern browser turns it on by default; they open about six connections per host instead (MDN). HTTP/2 tags each request with a stream ID so many requests share one connection and responses return in any order. gRPC runs on that. TCP still delivers bytes in order, so one lost packet stalls every stream on the connection. HTTP/3 moves the streams onto QUIC, where a stream that loses a packet does not block the others (RFC 9114), and QUIC combines the transport and cryptographic handshakes to cut connection setup (RFC 9000).

When things go wrong

A waiting client gets a response, an error, or silence. Silence is the dangerous one: every request needs a deadline, and when it passes the client does not know whether the server acted. That is why idempotency keys exist (see the Idempotency lesson), and why HTTP defines GET, PUT and DELETE as idempotent and POST as not: they tell a client which requests it may resend blindly (RFC 9110). Amazon sets client timeouts from the server's measured latency, for example its p99.9 if a 0.1% false-timeout rate is acceptable, and retries at one layer only (AWS Builders' Library).

Down the stack

Layer 1The contract

The server promises at most one of three outcomes per request: a response that reflects a committed effect, an error that means no effect (a 4xx, or a 503 sent before work began), or nothing. The client promises to give up at its deadline and to resend only requests that are idempotent or carry an idempotency key. On HTTP/2 there is one extra, precise guarantee: when a server sends GOAWAY with a last-stream-ID, every stream with a higher ID was not processed and can be retried safely on a new connection, even a POST. Streams at or below that ID may or may not have been processed (RFC 9113, section 6.8). A new TLS 1.3 connection costs two round trips (TCP, then TLS) before the first request byte leaves.

Layer 2The mechanism

The server calls socket, bind and listen, then accepts connections as epoll reports the listening socket readable; each connection becomes a file descriptor in the event loop. HTTP/2 (RFC 9113) frames everything: a HEADERS frame opens a stream, DATA frames carry the body, RST_STREAM cancels one stream without touching the others, and SETTINGS_MAX_CONCURRENT_STREAMS caps how many streams a client may have open at once; the RFC recommends no less than 100. Clients use odd stream IDs (1, 3, 5) and servers even ones, so both can open streams without clashing (RFC 9113). The cap counts only open streams, which the 2023 Rapid Reset attack exploited: the client sends HEADERS and then RST_STREAM straight away, so it never hits the limit while the server keeps starting work. Servers now count cancellations per connection and close abusive connections outright (Google Cloud). When the cap is reached, a well-behaved client queues further requests locally or opens another connection. gRPC adds a grpc-timeout header (for example grpc-timeout: 1S) that carries the remaining deadline to the server (gRPC over HTTP/2), which can propagate it to its own downstream calls so work stops everywhere once nobody is waiting.

Layer 3The machine

When a SYN arrives, the kernel puts the half-open connection in the SYN queue; once the handshake completes, it moves to the accept queue, whose length is capped by the backlog argument to listen and by net.core.somaxconn (4096 by default since Linux 5.4, 128 before) (Linux ip-sysctl docs). If the application is too slow to call accept, the queue fills and new connections are dropped, which clients see as connect timeouts of 1 second or more as the SYN is retransmitted. The side that closes first enters TIME_WAIT for 60 seconds on Linux, holding its 4-tuple, so a client that opens and closes connections to one server address at high rates runs out of its roughly 28,000 ephemeral ports (32768 to 60999). A new TCP connection also starts with a congestion window of 10 segments (about 14 KB) (RFC 6928), so the first large response takes extra round trips. Reused connections skip all of that.

Why it's built this way

  1. Data and rules must have one authoritative home, so a server owns them and clients ask.
  2. Round trips cost tens to hundreds of milliseconds across the internet, so clients reuse connections instead of paying the handshake every time.
  3. Reusing one connection for many requests causes head-of-line blocking if responses must be in order, so HTTP/2 tags requests with stream IDs and caps concurrency per connection.
  4. The network can lose a reply after the server acted, so clients need deadlines, and protocols need a way (GOAWAY, idempotency) to say which requests are safe to repeat.

Diagram

sequenceDiagram
  participant C as Client
  participant S as Server
  C->>S: TCP SYN
  S-->>C: SYN-ACK
  C->>S: TLS ClientHello
  S-->>C: ServerHello, Finished
  C->>S: HEADERS stream 1 (GET /a)
  C->>S: HEADERS stream 3 (POST /b)
  S-->>C: response stream 3
  S-->>C: response stream 1
  C->>S: HEADERS stream 5 (GET /c)
  S-->>C: GOAWAY last-stream-id=3
  Note over C: stream 5 not processed, retry on new connection

Worked example

A mobile banking app sees a 120 ms median round trip. A cold start needs DNS (40 ms), TCP (120 ms) and TLS 1.3 (120 ms), then the request itself (120 ms plus 30 ms of server time): 430 ms before the first screen has data. The next request on the same connection costs 150 ms. Keeping connections warm, plus TLS resumption, beats any server-side tuning here.

Now the failure case. The app sets a 3-second deadline on "transfer money". The server commits the transfer at 2.9 seconds, but the response is delayed in a congested cell network and arrives at 3.4 seconds. The client has already given up. If it blindly resends, the user pays twice. The fix is an idempotency key generated on the phone and stored with the transfer, so the resend returns the original result. Had the server instead sent GOAWAY with last-stream-ID 7 while the transfer was on stream 9, the client would know it was never processed.

On the server side, 400,000 idle keep-alive connections at roughly 10 KB each cost about 4 GB across the fleet, cheap for the latency saved. The risk is a deploy: if 50,000 phones reconnect within a second against a backlog of 128, most SYNs are dropped and retried a second later, so raise somaxconn and add jitter to reconnects.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Thin clientOne place to change logic, little to secure on the deviceEvery interaction is a round trip, no offline useInternal tools, content sites, low-powered devices
Thick clientFast UI, offline work, less server loadLogic duplicated, old versions in the wild for yearsMobile apps, editors, collaborative tools
Stateful server (sticky sessions)Fast access to in-memory stateUneven load, lost state on restart, harder deploysGame sessions, WebSocket rooms with a clear owner
Stateless serverAny instance serves any request, easy scaling and deploysEvery request reads shared stateDefault for APIs
HTTP/2 multiplexed connectionOne connection, many concurrent requestsTCP-level head-of-line blocking on loss, stream capsBrowsers and service-to-service calls

Pitfalls in production

Leadership lens

Check yourself

  1. Why did HTTP/1.1 pipelining fail in practice while HTTP/2 multiplexing succeeded?

    Show answer

    Pipelined responses must return in request order, so one slow response blocks all those behind it on that connection. HTTP/2 tags each request with a stream ID, so responses can be interleaved and returned in any order.

  2. A client's POST times out. Under what conditions can it resend safely?

    Show answer

    If the request carries an idempotency key the server dedupes on, or if the server's GOAWAY shows the stream was above the last processed stream ID. Otherwise the server may already have acted.

  3. Clients see 1-second connect delays during a traffic spike, but server CPU is at 40%. What is a likely cause?

    Show answer

    The accept queue is full because the application calls accept too slowly or the backlog is small, so the kernel drops SYNs and clients wait for the 1-second SYN retransmit.

  4. What does "stateless server" actually require?

    Show answer

    That no conversation state lives only in one instance's memory between requests. Durable and session state lives in a shared store or in a signed token the client sends back.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →