Engineering Atlas › System Design Fundamentals

Forward proxies and reverse proxies

15 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

Same mechanism, different principal

Mechanically both accept a connection, read a request, open or reuse a connection elsewhere, and relay. What differs is who configured it and whom it serves. A forward proxy sits near clients (a corporate gateway, a scraper's proxy pool, your services' egress proxy) and the destination sees the proxy's address. A reverse proxy sits near servers and owns the public hostname; the client cannot tell how many servers, regions or versions are behind it.

Forward proxies and egress control

An explicit forward proxy is configured in the client (HTTPS_PROXY). For HTTPS the client sends CONNECT api.partner.com:443, the proxy opens a TCP connection and then blindly relays bytes, so it sees the destination host but not the content, unless it intercepts TLS with its own trusted CA, which corporate gateways do. A transparent proxy intercepts traffic without client configuration, usually by routing rules.

In backend systems, the important use is an egress proxy. Any feature that fetches a user-supplied URL (webhooks, link previews, an AI agent's browse tool) can be tricked into calling http://169.254.169.254/ and leaking cloud credentials. On AWS, requiring IMDSv2 blunts the simplest version, because a session token must first be fetched with a PUT that most URL fetchers never send; the setting can be made the account default and enforced for every new instance (AWS). It does not help against other internal services, so the egress proxy is still needed. An egress proxy such as Stripe's Smokescreen resolves the hostname itself, refuses private and link-local addresses, applies a per-service allow-list, and logs every call. Resolving inside the proxy is what defeats DNS rebinding, where a name resolves to a public address at check time and a private one at connect time. This is the "trust but verify" shape: the same control applies to an agent calling tools and to a payment service calling a bank.

What a reverse proxy does

Restoring client identity

The backend sees the proxy's address. HTTP proxies append the address they received the connection from to X-Forwarded-For, so after two hops it reads client, edge and the backend's peer is the second proxy. X-Forwarded-Proto and X-Forwarded-Host carry the original scheme and host, and RFC 7239 standardises all of this as Forwarded: for=...;proto=... (RFC 7239). Layer 4 proxies, which do not parse HTTP, use the PROXY protocol: a short header at the very start of the TCP stream carrying the original addresses and ports, in a text form (v1) or a binary form with optional TLV fields (v2) (HAProxy spec).

Any client can send its own X-Forwarded-For. The safe rule is to walk the list from the right, skipping addresses of proxies you trust, and take the first untrusted one. The leftmost entry is whatever the client chose to write.

Layer 4 versus layer 7

A layer 4 proxy forwards TCP or UDP byte streams, chooses a backend per connection and can pass TLS through untouched. It is cheap and protocol-agnostic but blind to requests. A layer 7 proxy parses HTTP, so it can route per request, retry, cache and rewrite, at the cost of terminating TLS and spending CPU on parsing.

Down the stack

Layer 1The contract

A reverse proxy promises the client the origin's HTTP semantics. It must strip hop-by-hop headers and may add Via. RFC 9110, which replaced RFC 7230 in 2022, requires removing Connection and every header it names, and lists Proxy-Connection, Keep-Alive, TE, Transfer-Encoding and Upgrade as fields to drop (RFC 9110); most proxies also strip the proxy authentication headers and Trailer, as the older RFC 2616 list did. Its own failures have distinct codes: 502 when the upstream sent an invalid response or the connection failed, 503 when no healthy upstream exists, 504 when the upstream did not answer in time. Retries are safe only when the method is idempotent or the request was never sent; NGINX's proxy_next_upstream has not retried POST, LOCK or PATCH once sent since 1.9.13 unless told to with non_idempotent (NGINX proxy module).

Layer 2The mechanism

NGINX runs a master process and one single-threaded worker per core. Each worker runs an epoll event loop over thousands of connections and keeps its own pool of idle upstream connections (keepalive 32 in an upstream block). For most of NGINX's history upstream keep-alive had to be switched on by hand, with proxy_http_version 1.1 and an empty Connection header, because the default upstream protocol was HTTP/1.0 with a new connection per request. That changed in 1.29.7 (March 2026, now in the 1.30 stable line): upstream keep-alive is on by default (keepalive 32 local), the default proxy protocol is HTTP/1.1, and the Connection header is no longer sent (NGINX changes). Older configs and older builds still behave the old way, so check the version before trusting the default.

Envoy organises everything as a pipeline (Envoy terminology). A listener accepts connections; listener filters such as the TLS inspector peek at SNI and ALPN; a filter chain is selected; the HTTP connection manager network filter parses HTTP/1.1, 2 or 3 and runs HTTP filters (authentication, rate limiting, and finally the router); the router picks a cluster, a load-balancing policy chooses an endpoint, and a per-worker connection pool supplies the connection. A main thread receives configuration from a control plane over xDS (listeners, routes, clusters and endpoints) and publishes read-only snapshots to workers through thread-local storage, so the request path takes no locks (Envoy threading model).

Layer 3The machine

Each proxied request holds two sockets and, with TLS on both sides, two sessions, so a proxy serving 100,000 concurrent clients needs file-descriptor limits above 200,000. Every byte crosses the kernel twice; a layer 4 proxy can use splice to move data between sockets without copying into user space, while a layer 7 proxy must copy to parse. NGINX's default response buffers are 8 x 4 or 8 KB per connection, and larger responses spill to temporary files on disk. Inside one data centre a proxy hop adds roughly 0.1 to 0.5 ms. Without upstream keep-alive, one proxy talking to one backend address can open at most about 28,000 connections (Linux's default ephemeral range is 32768 to 60999 (kernel docs)) per 60-second TIME_WAIT period, roughly 470 new connections per second, before it runs out of ephemeral ports. With SO_REUSEPORT, the kernel spreads new connections across workers' listening sockets instead of waking every worker for each one.

Why it's built this way

  1. Clients should not know the server topology, so one stable entry point hides scaling, deploys and failures.
  2. That entry point sees every request, so cross-cutting work (TLS, auth, limits, logs) is cheapest there.
  3. Application workers are expensive and slow clients hold them hostage, so the proxy buffers and multiplexes.
  4. Terminating connections loses the client's identity, so forwarding headers and the PROXY protocol exist, with trust rules.
  5. Fleet configuration changes constantly, so Envoy takes it over an API (xDS) rather than from files and reloads.

Diagram

flowchart LR
  U["Client 198.51.100.7"] --> E["CDN edge (reverse proxy)"]
  E -->|"XFF: 198.51.100.7"| L["Envoy ingress (L7)"]
  L -->|"XFF: 198.51.100.7, 10.0.1.5"| S["Orders service"]
  S --> G["Egress proxy (forward)"]
  G -->|"allow-listed"| P["api.partner.com"]
  G -.->|"blocked"| M["169.254.169.254"]

Worked example

A media API returns 500 KB responses to mobile clients averaging 50 KB/s, with 20,000 downloads in flight. Each app instance has 200 worker threads.

Without proxy buffering, a worker writes until the client's TCP window fills and then waits, so each request occupies a worker for about 10 seconds. Serving 20,000 concurrent downloads needs 100 instances doing almost nothing. With buffering, the app writes 500 KB to the proxy over the data-centre network in about 5 ms and moves on, and the proxy streams to the phone. The app needs a handful of instances; the proxy needs memory for buffered bodies, so you size buffers for typical responses (here 64 KB in memory, the rest spilling to disk) rather than holding 20,000 x 500 KB = 10 GB in RAM.

Second issue: the proxy forwards 5,000 requests per second to one backend address with upstream keep-alive off. It opens 5,000 connections per second, each leaving a TIME_WAIT entry for 60 seconds, and exhausts its 28,000 ephemeral ports in under 6 seconds; requests fail with EADDRNOTAVAIL and the proxy returns 502. Enabling keepalive 64 with HTTP/1.1 upstream drops new connections to a few per second. On NGINX 1.29.7 and later this is the default behaviour, but the default pool of 32 idle connections per worker may still be too small for 5,000 requests per second, so size it from the request rate and latency.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Explicit forward proxyCentral egress allow-lists and audit, SSRF defenceEvery client must be configured; a bypass path defeats itOutbound calls to partners, webhooks, agent tools
Transparent forward proxyNo client changesTLS interception breaks pinned clients and raises privacy issuesCorporate networks, compliance gateways
L4 reverse proxyCheap, any protocol, TLS passthroughPer-connection balancing only, no HTTP featuresDatabases, raw TCP, very high throughput
L7 reverse proxy (NGINX, HAProxy)Rich routing, caching, buffering, matureFile-based config and reloadsEdge and ingress for a modest number of services
Envoy with a control planeDynamic config over xDS, deep observability, HTTP/2 and gRPCOperating a control plane, steeper learning curveLarge microservice fleets and service meshes
Managed load balancer or CDNNo servers to run, global edge, DDoS absorptionLess control, per-request cost, vendor limitsPublic entry points for most teams

Pitfalls in production

Leadership lens

Check yourself

  1. A request reaches your service with X-Forwarded-For: 6.6.6.6, 203.0.113.9 from your ingress at 10.0.0.4, and only 10.0.0.0/8 is trusted. What is the client address?

    Show answer

    203.0.113.9. Walking from the right, 10.0.0.4 is trusted, 203.0.113.9 is not, so it is the client; 6.6.6.6 was written by the client and is ignored.

  2. Why does proxy response buffering let an application serve far more slow clients with the same workers?

    Show answer

    The application hands the full response to the proxy at data-centre speed and frees the worker, and the proxy's event loop holds thousands of slow connections cheaply.

  3. Why can't a forward proxy using CONNECT cache HTTPS responses?

    Show answer

    After CONNECT it relays an encrypted byte stream and never sees the requests or responses, only the destination host and port.

  4. When would you pick a layer 4 proxy over a layer 7 one?

    Show answer

    For non-HTTP protocols, when TLS must pass through to the backend, or when per-connection balancing is enough and you want the lowest cost per byte.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →