Splitting concerns cleanly

In one minute

• A concern is one thing a piece of software has to get right: the business rule, the storage format, the user interface, authentication, logging, retries. Separating concerns means each one lives in its own place, so you can think about, change and test it without holding the others in your head.
• Dijkstra coined the phrase in 1974 to describe a way of thinking: study one aspect of a problem in depth while knowing, but temporarily ignoring, the others. Code structure is how that way of thinking survives after the author has moved on.
• The most useful split in everyday code is between deciding and doing: a pure core that decides what should happen, and a thin shell that talks to disks, networks and clocks. The core is tested with plain values; the shell is tested with a few fakes.
• Concerns come in three shapes: horizontal layers (presentation, domain, data), vertical slices (features), and cross-cutting concerns (auth, logging, transactions) that touch everything and belong in middleware or wrappers.
• Agents happily write one function that parses input, applies policy, calls the database and formats output. Review by asking which lines would change if the policy changed, and which if the storage changed. If the answer is "the same function", the concerns are tangled.

How it works

The problem

A nightly job keeps a build-artifact cache under its disk quota. The first version is one 80-line function: it walks the directory, reads each file's modification time, decides whether the file is too old, deletes it, keeps a running byte total, and prints a summary. It works. Then three requests arrive in one month: the platform team wants pinned artifacts to survive cleanup, finance wants the quota lowered from 500 GB to 300 GB with oldest files removed first, and the on-call team wants a dry-run mode that shows what would be deleted.

Every request touches the same function. Dry-run needs the decision without the deletion, which is impossible because both happen on the same line inside the loop, and testing anything requires a real directory with timestamps set by os.utime. The function has three concerns (policy, execution, reporting) and no seams between them.

Separate deciding from doing

The fix is to split the job along its concerns:

def plan_cleanup(files, now, max_age, max_bytes):   # policy: pure
    ...returns [(path, size, reason), ...]

def apply_plan(plan, fs):                             # execution: I/O through a port
    ...returns [(path, status), ...]

def render_report(plan, results):                     # presentation: pure
    ...returns a string

plan_cleanup takes plain data and returns plain data. It never touches the disk and never reads the clock; now is a parameter. Dry-run is now print(render_report(plan, [])). Pinning and quota ordering are changes to one pure function, tested with lists of tuples in microseconds. apply_plan is ten lines that loop and call fs.remove, and its tests need one fake that fails on a chosen path. Gary Bernhardt named this shape "functional core, imperative shell": push decisions into code with no side effects, and keep the code with side effects so simple it barely needs tests.

Three shapes of concern

Horizontal layers. Presentation, domain and data access are the classic split, described well in Fowler's note on presentation-domain-data layering. Each layer knows only the one beneath it. A web handler parses the request and renders the response; the domain decides; the repository loads and stores. The rule that matters is direction: the domain must not import the web framework or the SQL driver.

Vertical slices. Within a large application, features are also concerns. Billing, search and notifications change for different reasons and are owned by different people. Grouping code by feature (billing/handler.py, billing/rules.py, billing/repo.py) keeps a billing change inside one folder, where grouping by layer (handlers/, services/, repos/) spreads it across three. Most mature codebases do both: slices at the top level, layers inside each slice.

Cross-cutting concerns. Authentication, request IDs, logging, metrics, timeouts and transactions apply to almost every operation. Pasting them into each handler tangles them with business logic and guarantees that one handler forgets. They belong in a layer that wraps every handler: HTTP middleware in Go and Express, interceptors in gRPC, decorators in Python, aspects in Spring. The handler then contains only its own concern.

Separation of concerns is not separation of technologies

Splitting by file type is not splitting by concern. When React put markup and behaviour for one component in one file, critics called it a violation of separation of concerns. Pete Hunt's answer at JSConf EU 2013 was that HTML, CSS and JavaScript files separated technologies, not concerns; the component is the concern. Things that change together should live together, and things that change for different reasons should not.

Relation to SRP and coupling

The Single Responsibility Principle is separation of concerns applied to one class, with "concern" defined as "who asks for changes". Coupling and Cohesion gives you the measurements. Hexagonal architecture is separation of concerns at service scale: the domain in the centre, technology at the edges. Separation of concerns is the umbrella idea; the others are specific ways to apply it.

When not to separate

A 40-line script that runs once does not need a core, a shell and a presenter. Separation costs names, files and hops. A good trigger is the second change request that touches two concerns in one function, or the first test you cannot write without a real disk or network.

Reviewing agent-written code

Ask an agent to "add dry-run to the cleanup job" and the usual result is a dry_run flag threaded through the loop with if not dry_run: around each side effect. It passes the test it wrote, and it leaves the policy untestable without a filesystem. Ask instead for a planner that returns actions and an executor that performs them, then check three things in review: the planner has no imports of I/O modules and no calls to the clock, the executor has no business rules (no if size > ...), and the report is built from the plan and results rather than from print statements inside the loop.

The same shape elsewhere

Deciding separately from doing is the trust but verify shape: the plan is data, so a human, a test or a policy engine can inspect it before anything irreversible happens. Terraform's plan and apply, Kubernetes controllers that compute a diff before calling the API, database query planners that produce an EXPLAIN tree before execution, and agent frameworks that ask for approval of a tool call before running it all use the same split.

Down the stack

Diagram

flowchart LR
  FS0["disk listing (shell)"] --> D["files as data"]
  CLK["clock (shell)"] --> N["now"]
  D --> P["plan_cleanup (pure policy)"]
  N --> P
  P --> PL["plan: path, size, reason"]
  PL --> A["apply_plan (shell)"]
  A --> FS1["fs.remove"]
  A --> R["results"]
  PL --> V["render_report (pure)"]
  R --> V
  V --> OUT["summary text"]

Worked example

A CI platform keeps 420 GB of build artifacts on a 500 GB volume across 1.8 million files. The original cleanup function mixed policy, deletion and logging. It had no unit tests, only an integration test that created 200 files in a temp directory and took 9 seconds. Two incidents in six months came from policy bugs: one run deleted pinned release artifacts because the pin check was inside an else branch that the quota path skipped.

The refactor split the job three ways. The planner takes a list of (path, size, mtime, pinned) tuples and returns actions. Its test suite grew to 64 cases, including a randomised property test that asserts pinned files never appear in a plan, and runs in 40 ms. The executor is 14 lines; three tests with a fake filesystem cover success, a permission error mid-plan, and a file that vanished before removal. The report is built from the plan and results.

Dry-run became a flag in main that skips apply_plan. Before the quota drop to 300 GB, ops reviewed a dry-run plan of 1.1 million files and 210 GB and caught that a team's nightly snapshot directory was unpinned. Planning 1.8 million entries takes about 1.5 seconds and a few hundred MB for the tuples; the unlink calls take roughly 25 minutes, so the separation cost nothing that mattered.

Trade-offs

Pitfalls in production

• A dry_run flag threaded through side-effecting code, where one forgotten if deletes data during a dry run. Fix: compute a plan and decide at the top whether to execute it.
• Hidden inputs in the pure core (datetime.now(), environment variables, a global config). Fix: pass them as parameters; ban the imports in the core with a lint rule.
• Business rules creeping into the shell ("skip files over 1 GB" added inside the executor loop). Fix: keep the executor free of conditions on domain data, and put every rule in the planner.
• Cross-cutting code copied into handlers, so the one new endpoint lacks auth. Fix: apply it in middleware registered once, and test that every route passes through it.
• Layers that only forward calls and copy fields, adding files without separating anything. Fix: merge layers that never change independently.

Leadership lens

• The useful review question is "which concerns does this change touch?". A one-line rule change that edits five files signals tangled concerns; a feature that touches one folder signals a healthy split.
• Enforce the important boundaries mechanically (Go internal/, import-linter, ArchUnit) so they survive staff turnover and high volumes of agent-written code.
• Ask for plans before actions in anything destructive: cleanup, migrations, bulk updates, infrastructure changes. A reviewable plan is cheaper than a postmortem.
• Do not split into services to get separation you could have had with modules. The network boundary adds latency, partial failure and on-call load that the team will carry for years.

Check yourself

1. Why does moving the clock read out of the cleanup policy make dry-run easier as well as testing?

   Show answer

   Once the policy is a pure function of its inputs, it can be called on its own to produce a plan without performing it, and the same call in a test gives the same answer every time.

2. Your team groups code into controllers/, services/ and repositories/, and every feature change touches all three folders. Which kind of separation is missing?

   Show answer

   Vertical separation by feature. Group by feature first and keep the layers inside each feature folder, so a change to one feature stays in one place.

3. Why is logging better applied as middleware than written into each handler?

   Show answer

   It is a cross-cutting concern with the same rules for every request, so applying it once guarantees coverage and keeps each handler about its own concern.

4. When is it wrong to separate a concern into its own service?

   Show answer

   When it does not need its own deployment, scaling or team. You pay network latency and partial failure for a separation that a module boundary would give you for free.

Watch

• Boundaries (Gary Bernhardt, RubyConf 12): the functional core and imperative shell, and why isolating decisions from effects removes most mocks.
• React: Rethinking best practices (Pete Hunt, JSConf EU): the argument that splitting by file type separates technologies, not concerns.

Free resources

• On the role of scientific thought (Dijkstra, EWD447): the 1974 essay where the phrase "separation of concerns" appears.
• On the criteria to be used in decomposing systems into modules (Parnas): the classic paper on splitting modules by the decisions they hide.
• Presentation Domain Data Layering (Martin Fowler): the standard horizontal split and why it is often applied inside feature modules.
• Functional Core, Imperative Shell (Destroy All Software): the short screencast that named the deciding versus doing split.
• Proxying mechanisms (Spring docs): how cross-cutting aspects are applied with proxies, including the self-invocation limitation.
• Using middleware (Express docs): cross-cutting concerns as an ordered chain of functions.