Iterator pattern

In one minute

• An iterator gives sequential access to the elements of something (a list, a tree, a file, a database result, a paginated API) without exposing how that something is stored. The caller sees "give me the next one" and "there are no more".
• The pattern's real value is laziness: elements are produced on demand, so memory stays constant, work stops when the caller stops, and the consumer sets the pace. That last property is backpressure.
• Every mainstream language has an iterator protocol built in: Python's __iter__ and __next__, JavaScript's Symbol.iterator and {value, done}, Java's hasNext and next, and Go's Next/Err types plus, since Go 1.23, range-over-func iterators.
• The hard parts are state and lifetime: what happens if the collection changes during iteration, who closes the underlying resource when the caller stops early, and how an iterator resumes after an error or across processes (pagination tokens, cursors, offsets).
• Agents often write rows = list(fetch_all()) and loop over it, or pagination loops with no exit guard. In review, ask what the peak memory is, what happens if the caller breaks early, and what stops the loop if the server returns the same token twice.

How it works

The problem

A billing team exports invoices to a data warehouse. The first version calls repo.all_invoices(), which returns a list. At 200,000 invoices it took 400 MB and 20 seconds before the first row was written. At 30 million it ran out of memory. A second consumer, a fraud check, needs only the first 1,000 invoices that match a rule, and still waits for all 30 million.

The iterator answer: repo.invoices() returns an iterator that fetches 5,000 rows at a time from a server-side cursor. The export writes as it reads in constant memory, and the fraud check stops after 1,000 matches, so no more batches are fetched.

External and internal iterators

An external (pull) iterator is an object the caller advances: next(it), it.next(), rows.Next(). The caller controls the loop, so it can stop, interleave two iterators (merging two sorted streams) or hand the iterator to another function. An internal (push) iterator is a function that takes a callback and calls it per element: list.forEach(fn), Go's func(yield func(T) bool). The producer controls the loop, which makes producers easy to write, because their state lives in ordinary local variables and the call stack, but makes merging two of them awkward.

Generators bridge the two. A Python or JavaScript generator is written like a push iterator (a loop with yield) and consumed like a pull iterator, because the runtime suspends the function at each yield and resumes it on the next next(). Go 1.23's iter.Pull converts a push iterator into a pull one by running it as a coroutine.

Protocols in four languages

Python calls __iter__(), then __next__() until it raises StopIteration. JavaScript calls x[Symbol.iterator](), then next() until {done: true}; for...of calls the optional return() on early exit so the iterator can release resources. Java has hasNext() and next(). Go's standard library uses Next() bool with accessors and an Err() checked after the loop (bufio.Scanner, sql.Rows), and Go 1.23 added iter.Seq[V], a function type for range loops over directly.

Mutation during iteration

What should happen if the list changes while someone iterates it? There are three common answers. Fail-fast: Java's ArrayList iterator throws ConcurrentModificationException, and Python raises RuntimeError: dictionary changed size during iteration. Snapshot: Java's CopyOnWriteArrayList iterates the array as it was when the iterator was created; RocksDB iterators read a consistent snapshot even while compactions rewrite files (see LSM Trees). Weakly consistent: Java's ConcurrentHashMap iterators never throw and may or may not see concurrent changes. Pick one deliberately and document it; "undefined" is the answer that produces skipped or duplicated elements in production.

Iterators that outlive a process

A pagination token, a cursor name and a Kafka consumer offset are iterator state serialised so iteration can resume in another request or after a crash. The token must capture position, not page number: offset pages skip or repeat rows when earlier rows change, while "after id 81723" stays correct. Clients must also guard against a buggy server returning the same token forever.

Reviewing agent-written code

Agents reach for list(...) and .all() because they make the next line simple; ask for peak memory at production volume. Pagination loops need an exit on an empty token, a repeated-token guard and a page cap. Iterators over files, sql.Rows or HTTP streams must close on early exit: defer rows.Close(), with or try/finally, or return() in JavaScript.

The same shape elsewhere

A pull iterator solves the overload problem shape by letting the consumer set the rate: nothing is produced until it is asked for. TCP flow control does the same between machines, where the receiver's advertised window limits how much the sender may transmit, and Reactive Streams makes it explicit with request(n), where a subscriber asks for the next n elements.

Down the stack

Diagram

sequenceDiagram
  participant C as Consumer
  participant P as Paginator
  participant A as "List API"
  C->>P: next()
  P->>A: fetch(token "")
  A-->>P: [a, b], next "t1"
  P-->>C: a
  C->>P: next()
  P-->>C: b (buffered)
  C->>P: next()
  P->>A: fetch("t1")
  A-->>P: [c], next ""
  P-->>C: c
  C->>P: next()
  P-->>C: done

Worked example

The invoice export reads 30 million rows from PostgreSQL and writes Parquet files. The first version used the ORM's .all(): 3.6 GB of Python objects, a 14-minute delay before the first write, and an out-of-memory kill on the 8 GB worker when another job shared the node.

The rewrite used a named server-side cursor with itersize = 5000, wrapped in a generator that yields rows and closes the cursor in finally. Peak memory dropped to 180 MB, mostly the Parquet writer's row group buffer, and the first file appeared in 2 seconds. Total time fell from 31 to 22 minutes, because writing now overlapped reading.

Two bugs surfaced. The fraud check broke out of the loop after 1,000 matches, but the generator was never closed, so the cursor and its transaction stayed open until the connection pool recycled the connection, holding back vacuum on the invoices table. Wrapping the iteration in with closing(...) fixed it. The second came from a partner API used for enrichment: during an incident it returned the same next_page_token repeatedly, and the paginator fetched the same page 400,000 times in an hour. A guard that remembers seen tokens and fails on a repeat, plus a page cap, stopped it.

Trade-offs

Pitfalls in production

• Loading whole result sets by default (.all(), JDBC without a fetch size). Fix: stream with server-side cursors and batch sizes, and test with production-sized data.
• Early exit leaking resources: open cursors, transactions or file handles from abandoned generators. Fix: defer rows.Close(), context managers, and finally in generators.
• Mutating a collection while iterating it, producing exceptions or skipped elements. Fix: iterate a copy, collect changes and apply them after, or use a snapshot iterator.
• Offset pagination over changing data, skipping or repeating rows. Fix: keyset tokens that encode the last seen sort key.
• Paginators that trust the server forever. Fix: detect repeated tokens and cap pages per run.
• Reusing an exhausted iterator and silently getting nothing, for example passing a generator to two functions. Fix: pass the iterable, not the iterator, or materialise deliberately.

Leadership lens

• Ask for peak memory and time-to-first-result at production volume in design review for any batch or export job; iterators change both by orders of magnitude.
• Treat pagination tokens as a public API: opaque, versioned, stable under inserts and deletes. Changing their meaning breaks clients mid-iteration.
• Decide and document the mutation semantics of shared collections and caches. "It depends on timing" is a defect report waiting to happen.
• Make streaming the default in shared data-access libraries, so teams opt in to materialising, not into streaming.

Check yourself

1. Why can a pull iterator merge two sorted streams easily while a push iterator cannot?

   Show answer

   With pull, the consumer advances whichever stream has the smaller head; with push, each producer drives its own loop, so interleaving needs threads, coroutines or buffering.

2. What does Java's fail-fast iterator actually guarantee?

   Show answer

   Nothing under concurrency; it compares a modification counter on each step and throws on a detected mismatch, which catches most single-threaded mistakes but is explicitly best-effort.

3. Why do keyset pagination tokens survive inserts while offset pages do not?

   Show answer

   A keyset token says "after this sort key", which still identifies the same position after rows are inserted earlier, while an offset counts rows and shifts when earlier rows appear or disappear.

4. What leaks when a consumer breaks out of a loop over a database-backed generator, and how is it prevented?

   Show answer

   The cursor and its transaction stay open until the generator is collected or closed; close it explicitly with a context manager, finally, or defer rows.Close().

Watch

• Iterator Pattern, Design Patterns ep 16 (Christopher Okhravi): the Gang of Four structure and the separation of traversal from collection.
• Iterator Design Pattern (Derek Banas): a code walkthrough of iterating different collection types through one interface.

Free resources

• PEP 234: Iterators: the design of Python's iterator protocol and why it uses StopIteration.
• PEP 255: Simple Generators: generators as resumable functions, with the rationale for yield.
• Iteration protocols (MDN): JavaScript's iterable and iterator protocols, including return().
• Range over function types (Go blog): Go 1.23 push iterators, iter.Pull, and the design reasoning.
• Iterator (RocksDB wiki): snapshot semantics, pinned resources and the cost of long-lived iterators in a real storage engine.
• Iterator (Refactoring Guru): the classic structure and its relation to Composite and Factory Method.