Python's logging Module Is Already a Singleton

The Setup

You’re running a Python service. It has a Kafka consumer pulling messages, uvicorn serving HTTP requests, SQLAlchemy talking to Postgres, and your own application code doing the actual work. Each of these libraries has its own logger. Kafka logs heartbeats every second. Uvicorn logs every health check. SQLAlchemy logs every query. Your application logs are buried in the noise.

How do you control all of them? How do you silence Kafka’s heartbeat spam without touching kafka-python’s source code? How do you format uvicorn’s access logs as JSON for your log aggregator? How do you add a request_id to every log line — even from third-party libraries — so you can trace a single request across your entire system?

The Wrong Answer: Build a Registry

In my first production ML system, I built this:

import threading
import logging

class LoggerRegistry:
    _instance = None
    _lock = threading.Lock()

    def __new__(cls):
        with cls._lock:
            if cls._instance is None:
                cls._instance = super().__new__(cls)
                cls._instance._loggers = {}
                cls._instance._initialized = False
            return cls._instance

    def get_logger(self, name):
        if name not in self._loggers:
            logger = logging.getLogger(name)
            # ... configure with custom formatter
            # ... add custom handler
            # ... set level from YAML config
            self._loggers[name] = logger
        return self._loggers[name]

This was part of a 400-line logging package spread across 7 files: a thread-safe singleton registry, factory methods for formatters and handlers, Pydantic models for config validation, a custom YAML loader, a TensorRT logging adapter, and a custom exception hierarchy with ConfigNotFoundError, ConfigParseError, and ConfigValidationError.

It worked. It was also entirely unnecessary.

The Revelation: logging Is Already a Singleton

Python’s logging module maintains a global Manager object that stores every logger ever created in a single dictionary. When you call logging.getLogger("kafka"), it checks this dictionary. If a logger named "kafka" exists, it returns the same object. If not, it creates one, stores it, and returns it.

# These two calls return the EXACT SAME object
logger_a = logging.getLogger("kafka")
logger_b = logging.getLogger("kafka")
assert logger_a is logger_b  # True — same object in memory

This isn’t an implementation detail. It’s a design guarantee. The logging module is a singleton by design. Every call to getLogger with the same name returns the same logger instance, process-wide.

That LoggerRegistry I built? logging already has one. It’s called logging.Manager. The thread-safe __new__ with threading.Lock? logging.getLogger() is already thread-safe. The _loggers dictionary? logging.Manager already maintains one.

The Power Move: dictConfig Controls Everything

Here’s where it gets powerful. Every Python library that uses logging calls logging.getLogger(__name__) internally. Kafka uses logging.getLogger("kafka"). SQLAlchemy uses logging.getLogger("sqlalchemy.engine"). Uvicorn uses logging.getLogger("uvicorn.access").

Because they all share the same global logger tree, you can configure all of them from a single dictConfig YAML:

# logging.yaml
logging:
  version: 1
  loggers:
    kafka:
      level: WARNING # Silence heartbeat spam
    sqlalchemy.engine:
      level: WARNING # Silence query logging
    uvicorn.access:
      handlers: [json_file] # JSON format for log aggregation
      level: INFO
    myapp:
      level: DEBUG # Full debug for your code
  root:
    level: INFO
    handlers: [console]

You don’t need to import kafka’s logger. You don’t need to wrap uvicorn. You don’t need to subclass anything. You just name the logger in your YAML and configure it. The library doesn’t know or care.

Adding Context: ContextVar for Request Correlation

Now the real trick. You want every log line — from your code, from Kafka, from SQLAlchemy — to include a request_id so you can trace a single request across your entire system. This is 8 lines of code:

import contextvars
import logging

request_id_var = contextvars.ContextVar('request_id', default='-')

class RequestIdFilter(logging.Filter):
    def filter(self, record):
        record.request_id = request_id_var.get()
        return True

Add this filter to your root handler in dictConfig, set the ContextVar in your middleware, and every log line in the process gets a request_id — even from libraries you didn’t write. This works because all loggers inherit from the root logger, and the filter runs on every log record that passes through the handler.

When structlog Earns Its Place

stdlib logging + dictConfig + ContextVar covers most production needs. But structlog adds genuine value when you need:

• Typed context binding: logger.bind(user_id=123) attaches context that persists across calls without ContextVar boilerplate
• Processor pipelines: Chain transformations (add timestamp, add caller info, format as JSON) as composable functions
• Dual-format output: Pretty colored logs for development, JSON for production, switched by one config flag
• Automatic caller injection: structlog.dev.ConsoleRenderer adds filename and line number without manual formatting

structlog wraps stdlib logging — it doesn’t replace it. dictConfig still controls third-party library loggers. structlog just gives you a nicer API for your own code. The ~80 lines of structlog configuration in our latest codebase pays for itself through cleaner context management and dual-format output.

The Evolution

This isn’t a story about “don’t over-engineer.” That’s too generic to be useful. The actual learning is specific:

Python’s logging module is already a singleton. getLogger() is the registry. dictConfig is the configuration system. Filter is the extension point. The standard library solved this problem in 2002.

Each codebase iteration removed code that duplicated what stdlib already provides:

• v1 (ml-tagger): Custom singleton, custom registry, custom YAML loader, custom exception hierarchy — ~400 lines rebuilding logging.Manager
• v2 (ml-apis): dictConfig YAML + ContextVar filter — ~100 lines using what stdlib provides
• v3 (Neo-Factryze): structlog processor pipeline + ContextVar — ~80 lines delegating to a library that wraps stdlib

Practical Checklist

1. Start with dictConfig YAML — configure ALL loggers (yours and third-party) from one file
2. Use __name__ for logger identity — logger = logging.getLogger(__name__) maps to your module hierarchy
3. Add ContextVar + Filter for request correlation — 8 lines, works across all loggers
4. Add structlog only if you need typed context binding or processor pipelines — it wraps stdlib, doesn’t replace it
5. Never build a custom singleton — logging is already one

Common Mistakes

Building a Custom Logger Registry

logging.getLogger(name) returns the same object every time. logging.Manager is the registry. If you’re storing loggers in a dictionary with a threading.Lock, you’re duplicating stdlib.

Importing Third-Party Loggers to Configure Them

You don’t need from kafka.client import logger. Just put kafka in your dictConfig YAML. The logger name is the configuration key.

Using print() for Debugging in Production

print() bypasses the entire logging infrastructure — no levels, no formatting, no routing, no correlation IDs. Use logger.debug() and control verbosity through configuration.

Forgetting to Configure Third-Party Loggers

If your logs are noisy with Kafka heartbeats or SQLAlchemy query dumps, you haven’t configured those loggers. Add them to your dictConfig YAML and set appropriate levels.

Related Articles

• The Registry Pattern: Another pattern where Python’s dynamic nature eliminates boilerplate
• CPython Internals: How Python’s module system creates the singleton behavior that logging relies on