Engineering Atlas › System Design Interviews

Docker for system design interviews

14 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

How it works

From docker run to a process

The docker CLI talks to dockerd over a Unix socket. dockerd handles images, networks and volumes, and delegates container lifecycle to containerd over gRPC. For each container, containerd starts a small containerd-shim-runc-v2 process, which calls runc. runc reads an OCI runtime bundle (a config.json plus a root filesystem), creates namespaces and cgroups, mounts the root filesystem, drops privileges and execs your entrypoint. Then runc exits; the shim stays as parent so dockerd can restart without killing containers. Since Docker Engine 29 (November 2025), fresh installs also hand image storage to containerd: layers are unpacked by containerd snapshotters instead of Docker's own overlay2 graph driver, which keeps the compressed blobs as well and so uses more disk (Docker docs).

Images and layers

Each Dockerfile instruction that changes the filesystem (RUN, COPY, ADD) produces a layer: a tar of the files it added, changed or deleted. An image manifest, defined by the OCI image spec, lists the layers by SHA-256 digest along with a config blob holding the entrypoint, environment and the uncompressed diff_ids. Because layers are named by their content, two images built on the same base share those layers on disk and in the registry, and a pull downloads only layers the node lacks. A multi-architecture image adds an index pointing to one manifest per platform (OCI image spec).

Build cache

BuildKit, the default builder on Linux since Docker Engine 23.0 (release notes), reuses a layer if its parent and instruction are unchanged and, for COPY, the copied files' checksums match. So install dependencies before copying source code, and a code change invalidates only the last small layer.

Networking and storage

By default a container gets its own network namespace, joined to the host by a veth pair attached to the docker0 bridge, with outbound traffic masqueraded via iptables NAT and published ports forwarded with DNAT rules. State that must survive the container goes in volumes or bind mounts, which bypass the overlay filesystem.

Down the stack

Layer 1The contract

  • Filesystem. A container sees the merged image layers as its root. Writes go to a private writable layer that disappears with the container unless committed. Image layers are immutable and identified by digest, so a given digest always yields the same bytes.
  • Isolation. A container cannot see other processes, mounts, hostnames or network interfaces outside its namespaces, and cannot exceed its cgroup limits. It shares the host kernel, so a kernel exploit escapes the boundary.
  • Resources. --memory is a hard limit: exceeding it triggers the kernel OOM killer inside the cgroup. --cpus is a bandwidth quota, not a reservation: the container may use up to N CPUs of time per period and is throttled beyond that.
  • Performance. CPU and memory access run at native speed because there is no hypervisor. Startup is typically 100 ms to 1 s when the image is already on the node.

Layer 2The mechanism

Namespaces. runc calls clone or unshare with flags such as CLONE_NEWPID, CLONE_NEWNS, CLONE_NEWNET, CLONE_NEWUTS, CLONE_NEWIPC, CLONE_NEWCGROUP and optionally CLONE_NEWUSER. In a new PID namespace the entrypoint is PID 1, which must reap zombies and handle SIGTERM (use --init or tini if it does not). The mount namespace plus pivot_root swaps in the container root. docker exec joins the namespaces listed under /proc/<pid>/ns/.

cgroups v2. Each container gets a directory in the unified hierarchy under /sys/fs/cgroup. cpu.max holds "quota period", for example 150000 100000 for 1.5 CPUs. memory.max is the hard limit and memory.high a throttling threshold. pids.max caps fork bombs, and io.max limits block device bandwidth (cgroup v2 docs). Docker 29 deprecates cgroup v1, with support promised only until at least May 2029 (Engine 29 release notes). Counters such as cpu.stat (nr_throttled, throttled_usec) and memory.events (oom_kill) show when limits bite.

overlayfs. The overlay2 driver mounts each container with lowerdir (the image layers, colon-separated, top first), upperdir (the writable layer), workdir (scratch space for atomic operations) and the merged view (overlayfs docs). A lookup checks the upper directory, then each lower one in order. The first write to a lower-layer file triggers copy-up: the whole file is copied into the upper directory before the write proceeds. Deleting a lower file creates a whiteout (a 0/0 character device) in the upper directory, and replacing a directory marks it opaque with the trusted.overlay.opaque xattr. In image tar layers the same ideas are encoded as .wh.<name> files and a .wh..wh..opq marker, which the driver converts when it unpacks a layer.

Seccomp and capabilities. The default seccomp profile is an allowlist that leaves out around 44 of the 300-plus syscalls, such as kexec_load, bpf and io_uring_setup (Docker docs), and the default capability set drops CAP_SYS_ADMIN and most other root powers.

Layer 3The machine

CPU throttling. The bandwidth controller refills each cgroup's quota every period, 100 ms by default (kernel docs). It is still called CFS bandwidth control, although Linux 6.6 replaced CFS's task picking with EEVDF (kernel docs). A service limited to 2 CPUs that runs 16 busy threads consumes 200 ms of CPU time in 12.5 ms of wall time, then sleeps for the remaining 87.5 ms of the period. Average utilisation looks fine while p99 jumps by up to 90 ms. Watch nr_throttled and size thread pools to the quota, not to the host core count. cpu.max.burst lets a cgroup bank unused quota for short spikes, at the cost of more interference with neighbours.

Memory. The memory cgroup counts anonymous memory and page cache. When usage hits memory.max, the kernel first reclaims cache in that cgroup, then OOM-kills a process inside it, usually PID 1, and the container exits with code 137. Language runtimes that size heaps from host RAM (older JVMs, some Node settings) overshoot the limit, so set heap sizes explicitly or rely on container-aware defaults.

Disk and page cache. Lower layers are shared files, so 50 containers from one image read the same inodes and share one copy in page cache. Copy-up is the cost: modifying one byte of a 2 GB file in a lower layer first writes 2 GB to the upper directory. Databases and logs belong in volumes for that reason.

Network. Bridge networking costs a veth hop, a bridge forward and conntrack NAT, adding a few microseconds per packet and a conntrack entry per flow. A host handling hundreds of thousands of short connections can fill nf_conntrack_max and silently drop new flows. --network host skips all of it at the cost of port conflicts.

Why it's built this way

  1. Booting a guest kernel per application costs seconds and hundreds of MB, so containers reuse the host kernel and isolate with kernel features that already existed.
  2. Shipping full filesystem images to thousands of hosts is expensive, so images are split into content-addressed layers that deduplicate and cache.
  3. Layers must stay immutable to be shared, so writes go to a per-container upper layer through copy-up, and deletion needs whiteouts.
  4. Copying whole files on write is slow for large mutable data, so state lives in volumes outside the layered filesystem.
  5. Kernels share resources by default, so cgroups add accounting and limits, and their period-based CPU quotas trade tail latency for simple, enforceable fairness.

Diagram

flowchart TD
    CLI["docker CLI"] --> D["dockerd"]
    D --> CT["containerd"]
    CT --> SH["containerd-shim"]
    SH --> R["runc"]
    R --> NS["namespaces: pid, mnt, net, uts, ipc"]
    R --> CG["cgroup: cpu.max, memory.max"]
    R --> OV["overlay mount"]
    OV --> UP["upperdir (writable)"]
    OV --> L3["layer 3: app code"]
    OV --> L2["layer 2: dependencies"]
    OV --> L1["layer 1: base OS"]
    R --> P["exec entrypoint as PID 1"]

Worked example

A Python API image has three layers: a slim base (80 MB compressed), dependencies (300 MB) and application code (5 MB). It deploys to 200 nodes ten times a day.

With the Dockerfile ordered COPY requirements.txt, RUN pip install, COPY . ., a code change rebuilds only the 5 MB layer. Each deploy pulls 200 × 5 MB = 1 GB from the registry. If someone writes COPY . . before pip install, every commit invalidates the dependency layer too: 200 × 305 MB = 61 GB per deploy. At an effective 10 Gbit/s of registry egress, that is about 50 s of pure transfer versus under 1 s, plus decompression on each node, and over 600 GB of transfer per day.

The same service runs with --cpus 2 and a thread pool of 32. Under load its CPU graphs show 70 percent of quota, but nr_throttled climbs and p99 rises from 40 ms to 120 ms. Shrinking the pool to 4 workers, or raising the quota, removes the throttling.

In the wild

Trade-offs

OptionStrengthsWeaknessesChoose when
Docker / OCI containersFast start, dense packing, standard image formatShared kernel, noisy-neighbour effectsDefault for services you trust
Virtual machinesStrong isolation, any kernelSeconds to boot, GB of overheadUntrusted tenants, kernel-specific needs
MicroVMs (Firecracker, Kata)VM isolation with container-like start (around 100 to 200 ms)More complex ops, fewer featuresMulti-tenant serverless, CI runners
gVisorUser-space kernel limits syscall surfaceSyscall-heavy workloads slow downUntrusted code that is mostly compute
Bare process plus systemdNo layers, no overheadNo packaging story, dependency driftSingle-purpose hosts, appliances

Pitfalls in production

Leadership lens

Check yourself

  1. Two containers from the same 1 GB image run on one host. Roughly how much disk and page cache do they use, and why?

    Show answer

    About 1 GB plus each container's small writable layer, because both mount the same read-only lower layers and share their page cache.

  2. A RUN rm -rf /tmp/cache in a later Dockerfile line does not shrink the image. Why?

    Show answer

    Earlier layers are immutable; the delete only adds whiteouts in a new layer, so the bytes are still shipped. Remove files in the same RUN that created them.

  3. Why can a container at 60 percent average CPU of its quota still see 80 ms latency spikes?

    Show answer

    Quota is enforced per period; bursts across many threads exhaust it early in the period and the cgroup is throttled until the next refill.

  4. Why is a container a weaker security boundary than a VM?

    Show answer

    All containers share the host kernel, so a kernel bug reachable through any allowed syscall can escape, whereas a VM exposes only the much smaller hypervisor interface.

Sources and further study

Primary

Systems docs

Books

Other curricula

Watch

Free resources

Open in the interactive app: practice, quiz, mock interview →