Hiding complexity with abstraction

12 min read · last reviewed 2026-10-17

Open in the interactive app: practice, quiz, mock interview →

In one minute

An abstraction is a simplified model that lets you use something without knowing how it works: a file instead of disk blocks, a Buffer instead of an array with a gap. It is a choice of what to show and what to hide.
Good abstractions are deep: a small interface over a lot of useful work. Shallow ones add a layer that hides nothing and just renames calls.
Every abstraction has three parts: the operations, the cost model (how fast each one is) and the failure model (what can go wrong). Most abstractions only document the first, and the other two leak.
Abstract too early and you get the wrong abstraction: a shared base class with flags for every caller's special case. Duplicating code a little is cheaper than a model that does not fit.
Agents generate abstractions fluently (base classes, managers, generic helpers) because they pattern-match on structure. Ask what each layer hides and what it would cost to delete.

How it works

The problem: everything at one level

A text editor's command handler, written quickly:

def on_key(editor, ch):
    s = editor.chars                  # a Python list of characters
    s.insert(editor.cursor, ch)       # shifts everything after the cursor
    editor.cursor += 1
    editor.dirty_lines.add(editor.line_of(editor.cursor))
    if len(s) > editor.autosave_limit:
        write_file(editor.path, "".join(s))

Key handling, storage, line tracking and saving are mixed together. Changing the storage, because inserting into a 50 MB list per keystroke is slow, means touching every command that reaches into editor.chars. The code handles the how of storage when it only cares about the what: "put this text here".

Separate what from how

An abstraction names the what:

class Buffer(Protocol):
    def insert(self, pos: int, text: str) -> None: ...
    def delete(self, pos: int, n: int) -> None: ...
    def length(self) -> int: ...
    def text(self) -> str: ...

Behind Buffer, storage can be a plain string for tests, a gap buffer (an array with a movable hole at the cursor, as in Emacs), a rope (a balanced tree of chunks) or a piece table (VS Code's choice). Commands do not change when storage does: a decision that may change is hidden behind one that will not.

The Interfaces lesson covers how languages dispatch such a contract, and the Encapsulation lesson covers keeping the hidden part safe. Abstraction is the step before both: deciding which concept to expose.

Deep and shallow

John Ousterhout's measure is depth: the functionality a module provides divided by the size of its interface. Unix file I/O is deep: five calls (open, read, write, lseek, close) hide block allocation, caching, permissions, device drivers and dozens of filesystems. A UserManager whose methods each forward to one UserRepository method is shallow: it adds a file, a name and a stack frame, and hides nothing.

A useful test: inline the layer in your head. If callers would be just as clear calling the thing underneath, the layer is shallow and costs reading time for nothing.

Abstract classes

An abstract class is a partial implementation with holes that subclasses fill. It fits when implementations share real code that runs in a fixed order (the Template Method lesson). It is the wrong tool for sharing helpers, because it ties every subclass to the base forever, and in Java or C# spends the single base class; a helper module or composition does that job without the coupling.

The wrong abstraction

Two report exporters share 60 percent of their code, so someone extracts a BaseExporter. A third exporter needs one step skipped, so the base gets a skip_header flag. A fourth needs a different order, so the base gets a hook. After a year the base class has nine flags, and every change risks one of six subclasses. Sandi Metz's advice applies: duplication is far cheaper than the wrong abstraction. When an abstraction needs flags to serve its callers, inline it back into them and look for the real shared concept again. A common rule is to wait until the third instance before extracting.

Leaky abstractions

Joel Spolsky's law: all non-trivial abstractions leak. TCP presents a reliable byte stream, but a pulled cable still surfaces as a timeout minutes later. An ORM presents objects, but a loop that touches order.customer issues one query per order (the N+1 problem). A List presents indexing, but a linked list makes get(i) linear. The leak is usually in the cost model or the failure model, which the interface does not mention. The honest response is to document them as part of the contract, as Down the stack shows.

Reviewing agent-written code

Agents produce abstraction-shaped code quickly: an AbstractBaseProcessor, a ProcessorFactory, a ProcessorManager, generic type parameters nobody instantiates twice. It looks designed and passes tests. In review, ask for each layer: what decision does it hide, and how many implementations exist today? Push back on base classes that only share helpers, on managers that only forward, and on generic code with one concrete use. Ask for the cost and failure notes on any abstraction others will build on.

The same shape elsewhere

An abstraction boundary is a contract between levels, the same shape as a storage engine API or a cloud service. Amazon S3's original model hid replication behind "put then get", and for years list-after-write was only eventually consistent, a leak that broke data pipelines until S3 became strongly consistent in 2020. Kubernetes presents "desired state" but leaks scheduling and eviction to anyone running stateful workloads.

Down the stack

Layer 1The contract

A complete abstraction states three things: operations (inputs and outputs), costs (which operations are cheap, which are linear, which do I/O) and failures (what errors exist, and what state is left after one). A gap buffer's contract is: insert and delete at or near the previous edit are O(1) amortised; an edit far away costs O(distance) to move the gap; text() is O(n). Callers can then write code that is fast on purpose, not by luck.

Layer 2The mechanism

Abstract types are enforced differently in each language. Python's abc.ABC records abstract methods at class creation and refuses to instantiate a class that leaves any unimplemented; a Protocol describes the shape for type checkers only. Java marks the class ACC_ABSTRACT, so the JVM throws InstantiationError on new, and an abstract method has no bytecode; a subclass compiled against an older base that missed a new abstract method throws AbstractMethodError at the call. TypeScript abstract is checked by tsc and erased: the emitted class is ordinary JavaScript. Go has no abstract classes: an interface plus a struct you embed gives the same split between required methods and shared code.

Leaks come from mechanisms below the abstraction. A write() on Linux returns once bytes are copied into the page cache; durability needs fsync on the file and, after a rename or create, on the directory, because the "file" abstraction does not promise either. An array presents uniform O(1) access, but memory is fetched in 64-byte cache lines and 4 KB pages, so access order changes speed. A Python list presents "insert anywhere", implemented as a memmove of every element after the position.

Layer 3The machine

The numbers are what make the leaks matter. Writing to the page cache takes microseconds; fsync on an NVMe SSD takes tens of microseconds to a few milliseconds, and on a network disk several milliseconds, so a "save" that calls fsync per keystroke is 1,000 times slower than one that does not. Summing a 4096 x 4096 matrix of 4-byte integers row by row reads each 64-byte cache line once; column by column touches a new line, and often a new page, on every access, and runs several times slower for the same O(n) work.

The editor example shows the same at small scale. Inserting one character at the start of a 10 MB document held as a Python list of characters moves 10 million 8-byte pointers, about 80 MB of memory traffic at roughly 10 GB/s, so 8 ms per keystroke, enough to drop frames. A gap buffer at the cursor moves nothing; jumping 1 MB away moves 1 MB once, about 100 us, after which typing is free again. The abstraction is identical; the cost model is not, and the cost model is what the user feels.

Why it's built this way

Programs are too large to hold in one head, so each part must be usable without knowing its inside.
Hiding the inside lets the implementation change, so abstractions are drawn around decisions likely to change (Parnas's information hiding).
Hardware is not uniform (caches, pages, disks, networks), so any simple model of it is approximate and some cost leaks.
Leaks cannot be removed without giving up the simplicity, so good abstractions document their cost and failure models instead.

Diagram

flowchart TD
  CMD["editor commands"] --> BUF["Buffer: insert, delete, length, text"]
  BUF --> GAP["GapBuffer"]
  BUF --> STR["StringBuffer (tests)"]
  BUF --> ROPE["Rope (large files)"]
  GAP --> ARR["array: text | gap | text"]
  ARR --> MOVE{"edit at gap?"}
  MOVE -- "yes" --> FAST["O(1) write into gap"]
  MOVE -- "no" --> SHIFT["memmove chars across gap"]

Worked example

A notes app stored each document as a single string and rebuilt it on every keystroke: doc = doc[:pos] + ch + doc[pos:]. For a 20 KB note that is a 20 KB copy, about 2 us, invisible. Then customers pasted log files into notes. At 5 MB a keystroke cost about 1 ms to copy, plus allocating a 5 MB string that the garbage collector had to reclaim; typing stuttered at roughly 30 to 60 ms per key once GC pauses landed.

Because every edit already went through a Buffer interface with insert, delete and text, the team swapped in a gap buffer in one pull request of about 150 lines. Typing cost fell to under 1 us per key regardless of document size. The leak they then found was text(): the autosave and the word counter each called it on every keystroke, materialising the whole 5 MB document twice per key. Adding length() and an incremental word count to the abstraction, and debouncing autosave to once per second, removed it. The interface grew by two operations, both motivated by the cost model.

Trade-offs

Option	Strengths	Weaknesses	Choose when
No abstraction, direct use	Nothing to learn; no indirection	Callers change when the detail changes	One caller; detail unlikely to change
Deep module (small interface, much hidden)	Simplifies every caller; implementation can change	Harder to design; cost model must be documented	Many callers; volatile implementation
Shallow wrapper	Looks tidy	Extra layer that hides nothing	Rarely; only to adapt a foreign API
Abstract base class	Shares code and fixed-order steps	Couples subclasses to the base	Real shared algorithm, few variants
Duplication until the third use	Avoids the wrong abstraction	Copies can drift	The shared concept is not clear yet

Pitfalls in production

Abstractions that hide cost, such as a property that runs a database query, called in a loop. Fix: name expensive operations as operations (load_customer()), and document costs.
Base classes grown flags for every subclass. Fix: inline the abstraction back into callers and re-extract what is truly shared.
Hiding failures, such as a storage interface that swallows fsync errors. Fix: make the failure model part of the contract and surface errors.
Shallow layers added "for future flexibility". Fix: delete them; add the layer when the second implementation exists.
Tests that only use the simple fake behind an abstraction, never the real implementation's cost or failure behaviour. Fix: run the contract tests against the real one too, including slow paths.

Leadership lens

In design review, ask what each new layer hides and what decision it lets you change later. "Nothing yet" means it can wait.
Require abstractions shared across teams to document cost and failure models, not just method signatures. That is where integrations break.
Budget time to remove wrong abstractions. Inlining a bad base class is real work that rarely gets scheduled unless someone owns it.
Treat a growing list of flags or special cases on a shared abstraction as a signal to revisit the model, not as normal growth.

Check yourself

Why is a class whose every method forwards to one method of another class considered shallow?
Show answer
Its interface is as large as what it hides, so callers gain nothing: they learn a new name and still deal with the same concepts. Depth is a lot of functionality behind a small interface.
Your storage interface has save(doc). A user loses a document after a power cut although save returned. What leaked?
Show answer
The failure and durability model: save wrote into the OS page cache but did not fsync the file or its directory, so the data was not on disk. The contract must say whether save means durable.
When should you not extract an abstraction from two similar functions?
Show answer
When you cannot yet name the shared concept or the differences are still changing. Waiting for a third instance usually reveals the real abstraction, and duplication is cheaper than undoing a wrong one.
Why does a gap buffer make typing cheap but jumping across a large file expensive?
Show answer
Edits at the gap write straight into free space, while moving the edit point shifts every character between the old and new positions across the gap.

Watch

A Philosophy of Software Design (John Ousterhout, Talks at Google): deep versus shallow modules, and why interface size is the cost of an abstraction.
All the Little Things (Sandi Metz, RailsConf 2014): refactoring tangled conditionals into small objects, and when duplication beats the wrong abstraction.

Free resources

The Law of Leaky Abstractions (Joel Spolsky): the original essay with TCP, SQL and string examples.
The Wrong Abstraction (Sandi Metz): why flag-laden shared code should be inlined back before it is re-extracted.
On the Criteria To Be Used in Decomposing Systems into Modules (David Parnas, 1972): drawing module boundaries around decisions likely to change.
The Humble Programmer (Edsger Dijkstra, EWD340): abstraction as the tool for creating a level where one can be absolutely precise.
Files are hard (Dan Luu): how the file abstraction leaks durability and ordering failures.
What Every Programmer Should Know About Memory (Ulrich Drepper): the cache and page behaviour hidden behind "memory is an array".

Open in the interactive app: practice, quiz, mock interview →