Code a spreadsheet engine with formulas
Open in the interactive app: practice, quiz, mock interview →In one minute
- A formula is a small program, so parse it once into a tree of nodes (Interpreter plus Composite) and evaluate the tree. Never re-scan the text when a value changes.
- The tree also tells you which cells the formula reads. Those reads are the edges of a dependency graph, and that graph is what makes recalculation both correct and cheap.
- On every edit, mark only the edited cell and its transitive dependents as dirty, then evaluate that subgraph in topological order, so each cell is computed once, after everything it reads.
- A cycle is not a crash. Cells that can never become ready in the topological sort are the cycle and everything downstream of it. Give them a
#CYCLE!value and keep the engine alive. - Errors are values, not exceptions.
#DIV/0!,#NAME?and friends flow through formulas like numbers, and Observer subscribers hear only about cells whose value really changed.
Requirements
Functional:
set(cell, raw)accepts a number, an empty string (clear the cell) or a formula that starts with=.get(cell)returns the current value.- Formulas support numbers, cell references such as
B2,+ - * /, unary minus, parentheses, andSUMover any mix of ranges (A1:A9) and expressions. - Changing a cell recalculates every cell that depends on it, directly or through other formulas, and only those.
- Cycles are detected and reported as an error value. Bad formulas and bad arithmetic produce error values too:
#PARSE!,#NAME?,#DIV/0!,#VALUE!. - Listeners can subscribe and learn which cells changed value, and what the new value is.
Non-functional:
- An edit costs time proportional to the affected subgraph, not to the size of the sheet.
- Deterministic: the same edits give the same values and the same notification order.
- A formula cannot crash the engine, however malformed or deep.
Out of scope: text cells, dates, INDIRECT-style dynamic references, cross-sheet references, undo, file formats, multi-user editing and threads. Extensibility covers the first two.
Entities and relationships
- Sheet owns everything: the formulas by cell, the current values, the two adjacency maps and the list of observers. It is the only class with mutable state.
- Cell address is a plain string such as
C7, normalised to upper case. It splits into a row number and a column number, and the order(row, column)is used to break ties. - Node is the abstract piece of a parsed formula.
Num,Ref,Range,Neg,BinandCallare its subclasses. Each node can list the cells it reads (refs) and compute its value from a values table (eval). - Value is a number or an error code. Empty cells read as 0, but are skipped by ranges.
- Parser turns formula text into a node tree. It is stateless between formulas.
- Observer is a callback
(cell, value)registered on the sheet.
Two maps hold the graph: precedents[c] (what c reads) and dependents[p] (who reads p). They are mirror images, kept in sync by set. Edges point to cells that do not exist yet, so a formula can refer to a cell the user fills in later.
Class diagram
classDiagram
class Sheet {
-ast: Map~Cell, Node~
-values: Map~Cell, Value~
-precedents: Map~Cell, Set~
-dependents: Map~Cell, Set~
+set(cell, raw) Cell[]
+get(cell) Value
+subscribe(fn)
}
class Node {
<<abstract>>
+refs(out)
+eval(values) Value
}
class Num
class Ref
class Range
class Neg
class Bin
class Call
class Parser {
+parse(text) Node
}
class Observer {
<<interface>>
+onChange(cell, value)
}
Sheet "1" o-- "many" Node
Sheet --> Parser
Sheet "1" o-- "many" Observer
Num --|> Node
Ref --|> Node
Range --|> Node
Neg --|> Node
Bin --|> Node
Call --|> Node
Bin "1" *-- "2" Node
Call "1" *-- "many" NodeKey flows
sequenceDiagram
participant U as Caller
participant S as Sheet
participant P as Parser
participant N as Node tree
participant O as Observer
U->>S: set("B1", "=A1*2")
S->>P: parse("A1*2")
P-->>S: Bin(*, Ref A1, Num 2)
S->>S: rewire precedents and dependents
S->>S: collect dirty set (B1 and dependents)
S->>S: Kahn order over the dirty subgraph
loop each ready cell
S->>N: eval(values)
N-->>S: value
end
S->>S: unreachable cells get #CYCLE!
S->>O: onChange(cell, value) for changed cells
S-->>U: changed cells in evaluation orderThe write path has six steps. Parse the raw text. Remove the cell's old edges and add the new ones. Walk dependents breadth first from the edited cell to get the dirty set. Count, for each dirty cell, how many of its precedents are also dirty. Start from the cells with a count of zero and repeatedly evaluate one, then decrement its dependents. Finally compare new values with old and tell the observers about the differences.
The read path is a single map lookup. get never recalculates, because every set leaves the sheet fully consistent. That keeps reads fast and makes the sheet safe to read from a renderer.
Design patterns used and why
- Interpreter. The formula grammar is tiny, so each grammar rule becomes a class with an
evalmethod. This is the textbook case for the pattern (Interpreter pattern). It stops being the right tool when the language grows to hundreds of functions, loops or user-defined code. Then you compile to bytecode or closures. - Composite.
BinandCallhold childNodes and are themselvesNodes, so callers treat a single number and a deep expression identically.refsandevalrecurse through the tree with no type checks. - Observer. The sheet publishes
(cell, value)events after a recalculation finishes. The UI, a chart and an audit log subscribe without the sheet knowing they exist. Publishing after the pass, not during it, means a listener that reads other cells sees a consistent sheet. - Recursive-descent parser with precedence layers. One method per precedence level (
expr,term,unary,atom) gives the right order of operations without a parser generator. Pratt parsers are the next step when you have many operators (matklad). - Errors as values. A
Valueis a number or an error code, and every operator checks its operands first. This is the same idea as a null object: the failure is part of the normal data flow, so one bad cell cannot unwind the whole recalculation.
Concurrency and edge cases
- Diamonds.
D1 = B1 + C1with both readingA1. A naive recursive "recompute dependents" visitsD1twice, once per path, and the first visit sees a staleC1(a glitch). Ordering the dirty cells topologically evaluates each cell exactly once, after all its inputs. - Self reference and cycles.
A1 = A1 + 1has an edge fromA1to itself, so its count never reaches zero. The same holds for a longer loop. Detection falls out of Kahn's algorithm for free: whatever is left unevaluated is the cycle plus everything fed by it. Excel treats this as a user error and warns, unless iterative calculation is enabled (Microsoft Learn). - Breaking a cycle. Editing a cell that belongs to a cycle puts the whole loop in the dirty set (its members are dependents of each other), so they recover as soon as the loop is cut.
- References to empty cells. Reading an empty cell gives 0, and a range skips it. The edge still exists, so filling the cell later triggers the dependents.
- Unchanged results.
=A1*0stays 0 whateverA1is. The cell is re-evaluated, since you cannot know in advance, but observers are told only when the value differs. Skipping the evaluation of its own dependents when a value did not change is an optimisation covered in Extensibility. - Floating point. Values are IEEE 754 doubles, so
0.1 + 0.2is0.30000000000000004. Display rounding belongs to the view, not the engine. - Concurrency. The practice engine is single-threaded: one writer at a time, and
setis atomic from the caller's point of view. In a shared editor, put one lock or one event queue aroundsetand let readers use immutable snapshots of the values map. - Hostile input. Deeply nested parentheses can overflow a recursive parser. Real engines cap nesting (Excel allows 64 nested function levels and 8,192 formula characters, per its specifications and limits). Do the same.
In the wild
- Public Health England, 2020: cases lost to a file format. Test results were collected into Excel templates saved in the old XLS format, which holds about 65,000 rows, and each result used several rows, so a template fitted roughly 1,400 cases. Cases past the limit were silently dropped: 15,841 positive results from 25 September to 2 October 2020 reached contact tracers late. The lesson for an engine is to fail loudly at a capacity limit, with an error value or a rejected edit, and never to truncate (BBC News).
- Genomics literature, 2016: auto-conversion corrupts data. Excel converts gene symbols such as SEPT2 and MARCH1 to the dates 2-Sep and 1-Mar by default. A screen of 35,175 supplementary Excel files found gene-name errors in 987 files from 704 published papers, 19.6% of the articles in the selected journals. The failure sits in input coercion, not in recalculation, which is why a strict, documented input grammar matters (Ziemann, Eren and El-Osta, Genome Biology).
Down the stack
Layer 1The contract
After set returns, every cell's value equals what you would get by evaluating all formulas from scratch in some valid dependency order, as if the sheet were rebuilt. The tests pin this down as four checkable promises. A cell is evaluated only after every cell it reads. A cell outside the dirty set keeps its old value. A value never depends on the order of earlier edits. Cells on or below a cycle read #CYCLE!. set returns the changed cells in evaluation order and notifies observers in that same order.
This is the same contract as an incremental build system. The "Build systems à la carte" paper calls Excel a build system in disguise: formulas are the task descriptions, cell values are the build products, and an edit is a changed input (Mokhov, Mitchell, Peyton Jones).
Layer 2The mechanism
Excel's documented model has three stages: build a dependency tree, build a calculation chain (the formula cells in the order they must be computed), then recalculate the dirty cells in chain order (Microsoft Learn). When a cell changes, all direct and indirect dependents are marked dirty. Volatile functions such as NOW are re-evaluated on every recalculation, together with their dependents.
Our engine builds the order per edit instead of keeping a chain. The steps are Kahn's algorithm on the dirty subgraph (topological sorting): compute in-degrees restricted to dirty cells, queue the zero-degree cells, pop one, evaluate it, decrement its dependents. The queue is a min-heap on (row, column) so output is deterministic. Cost is O(V + E) in the dirty subgraph plus the heap's log factor.
Excel's real chain differs in one respect that matters. Because INDIRECT makes dependencies depend on values, Excel starts from the previous chain, and if a formula depends on a cell that has not been calculated yet, the cell being calculated and its dependents are moved down the chain. A static graph like ours is the simpler special case, and stays valid for any formula whose references can be read off the text.
Range nodes matter for scale. Expanding SUM(A1:A100000) into 100,000 edges is fine for a lesson. Production engines store a range as one node with interval bookkeeping, so a column total costs one edge, not one hundred thousand.
Layer 3The machine
A worksheet is up to 1,048,576 rows by 16,384 columns, about 17 billion addresses (Microsoft support), so storage must be sparse: hash maps keyed by address, not a dense grid. A used cell costs a hash lookup (tens of nanoseconds when warm) plus pointer chasing through the node tree. A tree walk is a chain of dependent loads and virtual or interface calls, so a large sheet of tiny formulas is bound by memory latency rather than arithmetic, and an interpreter language pays a further constant factor on every node visit.
Numbers are 8-byte IEEE 754 doubles with a 53-bit mantissa, which is why integers are exact only up to 2^53 and why 0.1 + 0.2 is not 0.3. The parser's recursion uses the call stack, one frame per nesting level, so a formula of 100,000 open parentheses needs a depth cap. The evaluation order, however, is computed with an explicit queue and not recursion, so a chain of a million cells (A2 = A1 + 1, A3 = A2 + 1, ...) cannot overflow the stack. A recursive "evaluate my precedents first" design would.
Why it's built this way
- Typing a formula is rare and cheap; recalculating is constant. So parse once into a tree and keep the tree.
- The tree's references are exactly the dependencies, so the graph comes free from parsing and never needs a manual declaration.
- Most edits touch a small part of a large sheet, so recalculating everything is wasteful. Dirty marking restricts the work to the affected subgraph.
- Evaluating a dependent before its input yields a wrong value, so the order must be topological. Doing that with an explicit queue also detects cycles and avoids deep recursion.
- Users type nonsense all day, so failures become values that propagate. The sheet stays alive and shows where the problem is.
Extensibility
- Skip unchanged dependents. Today every dirty cell is re-evaluated. With a per-cell "an input actually changed" flag, a cell is evaluated only if one of its precedents produced a different value. In the
=A1*0case that stops the wave early. The cost is a flag per cell. - More functions. Add
MIN,MAX,AVERAGEandIFas a function table mapping names to implementations, soCallstays closed to modification.IFneeds lazy evaluation of its branches, which changes which references are really needed. - Text, booleans, dates.
Valuebecomes a tagged union. Coercion rules (is"3"a number?) become the hardest part, and the spreadsheet's well-known surprises live there. A study of 35,175 supplementary Excel files found gene-name errors, caused by gene symbols being auto-converted to dates, in 704 papers (Genome Biology, 2016). - Dynamic references (
INDIRECT,OFFSET). Dependencies now depend on values, so edges must be recomputed during evaluation, which is the calc-chain reordering described above. - Undo and history. Record each
setas a command with the old raw text. Undo is anotherset. Because recalculation is a pure function of the raw inputs, you need not store computed values. - Large sheets. Range nodes, a persistent calc order, and batch edits that mark many cells dirty and run a single recalculation (so a paste of 10,000 cells does not recalculate 10,000 times).
Leadership lens
- Decide what correctness means before performance: "same answer as a full recalculation" is a testable property. Property-test the incremental path against a naive evaluate-everything oracle on random edit sequences.
- Ask what the spreadsheet is to the business. Models that drive money decisions need an audit trail, and the biggest risk is usually a wrong formula or an input pipeline limit, not the engine (see In the wild).
- Keep the grammar small and versioned. Every new function is a compatibility promise, and users will paste formulas from other products and expect identical results.
- Budget for the long tail: error codes, locale decimal separators, and precision rules take more time than the graph algorithm. Staff for them.
- Make failures visible. A cycle or an error cell must be easy to find, with a "trace precedents" tool, or users will quietly trust wrong totals.
Check yourself
A sheet has
A1 = 1,B1 = A1 + 1,C1 = A1 + 2,D1 = B1 + C1. If you changeA1and recalculate recursively through dependents, how many times canD1be computed, and what is the fix?Show answer
Twice, once per path, and the first time with a stale
B1orC1. Evaluate the dirty cells in topological order soD1runs once, after both inputs are final.In Kahn's algorithm over the dirty subgraph, how do you know which cells are in a cycle, and why are some non-cycle cells also reported?
Show answer
Cells never reaching in-degree zero are never evaluated. They are the cycle members and every cell that depends on them, because those wait forever for an input that never completes.
#CYCLE!on all of them is consistent with error propagation.Why must
setremove a cell's old edges before adding the new ones?Show answer
Otherwise a formula changed from
=A1to=B1would still be recalculated whenA1changes and could look cyclic through stale edges. The graph must always equal the references of the current formulas.Why does the engine publish observer events after the whole recalculation instead of inside the loop?
Show answer
A listener that reads other cells mid-pass would see some cells new and some old. Publishing after the pass guarantees the sheet is consistent, and the changed list lets listeners redraw only what moved.
Sources and further study
Primary
- Build systems à la carte (Mokhov, Mitchell, Peyton Jones): treats Excel as a build system with a calc chain and dynamic dependencies, the clearest account of the theory behind this design.
- Ziemann, Eren and El-Osta, Genome Biology 2016: a measured study of spreadsheet auto-conversion damage in published data.
Systems docs
- Excel recalculation (Microsoft Learn): the dependency tree, calculation chain, dirty cells, volatile functions and circular references.
- Excel specifications and limits (Microsoft): real size and nesting limits to copy as caps against hostile input.
- OpenDocument 1.2 part 2: formula (OASIS): an open specification of spreadsheet formula syntax and functions.
- Topological sorting (Wikipedia): Kahn's algorithm and the depth-first alternative, with complexity.
Books
- Design Patterns: Elements of Reusable Object-Oriented Software, Gamma, Helm, Johnson and Vlissides: chapter 5 (Behavioral), for Interpreter and Observer, and chapter 4 (Structural), for Composite.
- Crafting Interpreters, Robert Nystrom, the chapter on parsing expressions: recursive descent and precedence, free to read online.
Other curricula
- Refactoring.Guru: Composite: presents the tree-of-uniform-nodes structure that the expression tree uses.
- Refactoring.Guru: Observer: the publish and subscribe structure behind change notifications, including its ordering caveats.
Watch
- Observer Pattern, Design Patterns ep 2 (Christopher Okhravi): a careful walkthrough of the pattern behind cell change notifications.
Free resources
- Crafting Interpreters: parsing expressions: how to turn text into an expression tree with correct precedence.
- Simple but powerful Pratt parsing (matklad): the compact alternative to layered recursive descent.
- Interpreter pattern (Wikipedia): the pattern's structure, uses and limits.
- Directed acyclic graph (Wikipedia): background on the structure that makes ordered recalculation possible.