Code a logging framework

In one minute

• Split the framework into four roles with one job each: a Logger that application code calls, a LogRecord that carries one event, a Formatter that turns a record into text, and a Sink (appender, handler) that sends text somewhere.
• Configuration attaches to dotted names, not to logger objects. app.db.pool inherits its level from app.db, then app, then the root, so one line of config can silence or open up a whole subsystem.
• Records travel up the name hierarchy to every sink on the way, like a chain of responsibility that does not stop at the first handler. A propagate flag cuts the chain.
• The disabled path must be almost free. Check the level before you allocate, format or read the clock, because most log calls in production are debug lines that nobody will read.
• Logging must never take the application down. A failing sink is counted and skipped, and fields are data that is never interpreted (Log4Shell is what happens when a formatter interprets them).

Requirements

Functional:

• Log a message at a level (DEBUG, INFO, WARN, ERROR) with optional key-value fields.
• Get a logger by name. Set a level on any name; names without a level inherit from the nearest ancestor that has one, ending at the root.
• Attach any number of sinks to any name, each with its own minimum level and its own formatter (plain text, JSON).
• Bind context fields once (request_id, tenant) and have them appear on every line from that logger.
• Stop propagation at a chosen name, so a noisy subsystem can write to its own sink only.

Non-functional:

• A disabled log call costs a few nanoseconds and allocates nothing.
• Thread-safe: many threads log at once, and configuration can change at runtime.
• A broken sink (full disk, closed socket) does not throw into application code and does not stop other sinks.
• Output is deterministic and machine-parseable, with stable key order, so tests and log pipelines can rely on it.

Out of scope: log shipping, indexing and search (that is a distributed logging system), configuration file parsing, and log rotation policies, which are a sink's internal concern.

Entities and relationships

LogManager owns the registry: a map from logger name to its configuration (level, sinks, propagate flag) and the injected clock. There is one manager per process in practice, but making it an ordinary object, not a global, keeps tests isolated.

Logger is a lightweight handle: a name, a reference to the manager and a set of bound fields. Two handles with the same name share configuration because the configuration lives in the manager, keyed by name. That is why bind() can return a new Logger cheaply: it copies the field map, not the configuration.

LogRecord is an immutable value: timestamp, level, logger name, message and fields. It is created once per enabled call and handed to every sink, so sinks must not mutate it.

Formatter is a strategy with one method, format(record) -> string. Sink is the output port with one method, emit(record). A sink owns a formatter and a minimum level. Concrete sinks include an in-memory sink for tests, a file sink, a stdout sink and an async sink that wraps another sink with a queue.

Relationships: the manager has many configurations; a configuration has many sinks; a sink has one formatter; loggers create records; records flow from a logger's name up through its ancestors.

Class diagram

classDiagram
  class LogManager {
    +clock
    +getLogger(name) Logger
    +effectiveLevel(name) int
    +dispatch(record)
    +errorCount() int
  }
  class LoggerConfig {
    +level
    +sinks
    +propagate
  }
  class Logger {
    +name
    +fields
    +log(level, msg, fields) bool
    +bind(fields) Logger
    +setLevel(level)
    +addSink(sink)
  }
  class LogRecord {
    +ts
    +level
    +logger
    +msg
    +fields
  }
  class Sink {
    <<interface>>
    +emit(record)
  }
  class Formatter {
    <<interface>>
    +format(record) string
  }
  LogManager "1" *-- "many" LoggerConfig : keyed by name
  Logger --> LogManager
  Logger ..> LogRecord : creates
  LoggerConfig o-- Sink
  Sink <|.. MemorySink
  Sink <|.. FileSink
  Sink <|.. AsyncSink
  AsyncSink --> Sink : wraps
  MemorySink --> Formatter
  FileSink --> Formatter
  Formatter <|.. TextFormatter
  Formatter <|.. JsonFormatter

Key flows

The main flow is one error() call on app.db, where app has a sink and the root has a sink that is currently failing.

sequenceDiagram
  participant App
  participant L as Logger app.db
  participant M as LogManager
  participant S1 as Sink on app
  participant S2 as Sink on root
  App->>L: error("timeout", ms=900)
  L->>M: effectiveLevel("app.db")
  M-->>L: INFO from app
  L->>L: merge bound fields, read clock
  L->>M: dispatch(record)
  M->>S1: emit(record)
  S1->>S1: level check, format, write
  M->>M: app.propagate is true
  M->>S2: emit(record)
  S2-->>M: raises disk full
  M->>M: errorCount += 1
  M-->>L: done
  L-->>App: true

The level lookup walks app.db, app, root and stops at the first name that has a level set. Note that application is not a child of app: the hierarchy is split on dots, never on string prefixes. If the call is disabled, the flow ends after the first arrow pair and nothing else happens. That early exit is the most important line in the framework.

Design patterns used and why

• Strategy (Formatter). Text for humans, JSON for machines and logfmt for the team that likes it are three algorithms behind one interface. A sink holds one, so the same record can be text on the console and JSON in the file.
• Chain of responsibility (hierarchy walk). Each name in the chain gets a chance to handle the record, then passes it to its parent. Unlike the textbook version, handling does not stop the chain; only propagate = false does. Log4j and Logback call this flag additivity; Python's logging calls it propagate.
• Observer (sinks). Many sinks subscribe to one logger name and are notified of each record. Adding a sink is registering an observer; nobody else changes.
• Decorator (AsyncSink, FilteringSink). An async sink implements Sink and wraps another sink, adding a queue and a background writer without the inner sink knowing.
• Flyweight-like handles. Loggers are cheap views over shared configuration. You can create one per request with bound fields without copying sinks or levels.
• Avoid Singleton as the only access path. A process-wide default manager is convenient, but code that can only reach a global is hard to test. Pass a manager in, and keep the global as a thin convenience.

Concurrency and edge cases

• Concurrent logging and reconfiguration. Writes to the configuration (set level, add sink) are rare; reads happen on every call. Use a read-write lock or copy-on-write: build a new sink list and swap the reference atomically, so the hot path reads without contention.
• Do not call sinks under the manager lock. Snapshot the list of sinks along the chain under the lock, release it, then emit. A sink that blocks on a slow disk would otherwise block every thread that wants to log, and a sink that logs from inside emit would deadlock.
• Each sink protects itself. Two threads writing to one file must not interleave half lines. The sink holds its own mutex around format-and-write, or hands the record to a single writer through a queue.
• Recursion. A sink that fails and logs the failure through the same framework can loop forever. Count sink errors in a counter or write them to stderr directly.
• Async sinks drop or block. A bounded queue in front of a slow sink must choose: block the caller (backpressure, latency spikes) or drop records (count drops and log the count later). Make the choice explicit per sink; audit logs block, debug logs drop.
• Field collisions. A user field named msg or level must not overwrite the record's own values. Reserved keys win.
• Never interpret data. Messages and fields are printed, never evaluated. Log4Shell (CVE-2021-44228) came from Log4j expanding ${jndi:...} lookups inside logged strings, so an attacker-controlled user agent became remote code execution.
• Shutdown. Buffered and async sinks must flush on exit, or the last lines before a crash, the ones you need, are lost.

Down the stack

Extensibility

• New output, such as Kafka or a syslog socket. Add a Sink. Nothing else changes, which is the test of the design.
• New format, such as logfmt or OpenTelemetry log records. Add a Formatter.
• Sampling. Wrap a sink in a SamplingSink that passes 1 in N debug records, or the first K per message per second. Sampling by message keeps rare messages visible.
• Context propagation. Read bound fields from a request context (Go's context.Context, Python's contextvars) so trace and span IDs appear without passing a logger through every function.
• Dynamic levels. Expose an admin endpoint that calls setLevel on a name for ten minutes, then reverts. Because levels are keyed by name, this works without restarting.
• Redaction. A RedactingFormatter decorator masks fields named password or card_number. Put it in the framework so every team gets it for free.

Leadership lens

• Standardise the record shape (field names for request ID, tenant, error, duration) across services before choosing a vendor. Inconsistent field names cost more in incident time than any logging library choice.
• Ask in design review: what happens to the application when the log destination is down or slow? The answer should be "we drop and count", or "we block, and here is why that is acceptable".
• Logging volume is a real cost line. Ingest pricing is per GB, so debug logs left on in a busy service can cost more than the service. Budget by service and alert on volume jumps.
• Treat logs as untrusted data and as personal data. Redaction rules and retention periods are compliance decisions, not developer preferences.
• Build first: levels with name inheritance, a JSON formatter, a stdout sink and bound fields. Async sinks and sampling come when measurement shows a need.

Check yourself

1. Why is the level stored per name in the manager instead of on each Logger object?

   Show answer

   So every handle for a name, including bound copies and handles created before the change, sees the same level, and so a parent's change is inherited by children that never set their own.

2. A child logger app.db has an ERROR sink and propagates to a root sink at INFO. app.db has level WARN. Which sinks get a WARN record?

   Show answer

   Only the root sink. The call is enabled by the logger level, the app.db sink filters it out at its own ERROR threshold, and the root sink accepts it.

3. Why must the level check happen before reading the clock and merging fields?

   Show answer

   Because those steps allocate and cost time on every call, and most calls are disabled. Doing them first makes the disabled path as expensive as the enabled one.

4. What is the risk of holding the manager's lock while calling emit?

   Show answer

   A slow sink blocks every logging thread, and a sink that logs from inside emit deadlocks. Snapshot the sinks under the lock, then emit without it.

Watch

• Modern Python logging (mCoding): how Python's logger hierarchy, handlers, formatters and queue handlers fit together, and how to configure them well.
• Software Architecture in Golang: Structured Logging using slog (Mario Carrion): Go's slog handlers, attributes and levels in practice.

Free resources

• Logging HOWTO (Python docs): the logger hierarchy, propagation and the flow of a record through handlers, with a diagram.
• Architecture (Apache Log4j 2 manual): LoggerConfig, additivity, appenders and layouts in a mature Java framework.
• Logback architecture (Logback manual): level inheritance tables and appender additivity, explained with worked examples.
• Structured Logging with slog (Go blog): why Go added structured logging and how the Handler interface separates front end from back end.
• Logs (The Twelve-Factor App): the case for treating logs as an event stream written to stdout.