PROTOCOLS · ADAPTERS · FACADES · DECORATORS · DESCRIPTORS · MIXINS

Design patterns in Python, part 2:
how objects fit together

Structural patterns decide how objects are put together: which one wraps, holds, stands in for or inherits from another, so pieces written separately work as one. Each chapter takes one pattern, shows the problem it solves in a library you know, and ends with the Python you would write yourself.

Every chapter has the same parts. A pattern card sums it up. Then come the problem, the pattern’s shape in plain words, the same picture with a real project’s classes and an excerpt of its source, one call followed through it, and the code you would write. The patterns used most in Python come first, and you can read the chapters in any order.

You need to know Python classes, functions and imports. Samples run on Python 3.15.2. Excerpts come from requests 2.34.2, Werkzeug 3.1.9, Flask 3.1.3, tenacity 9.2.1, Django 6.1.3, Click 8.5.1 and the standard library, trimmed, with comments marked ← and a link to the lines they came from. Tap any figure to open it full screen. Dotted terms are tappable. This is part 2 of 3: part 1 covers how objects get made, and part 3 how they share the work.

Protocols: an interface Python checks by calling it

A protocol is the set of methods a caller relies on, and in Python it is the real interface: code calls a method and any object that has it works. You have used this every time json.load read a file, a string in memory or a network response. None of those classes knows about the others, and none inherits from a base class json defines. How does one function accept objects it has never heard of?

THE PATTERN

you’ll write it often

Accept any object that has the methods you call, instead of requiring one base class.

ROLES
  • callercalls a method by name and trusts the object to have it
  • protocolthe method names the caller uses, such as read() or __len__
  • any objecthas those methods, whatever its class
REACH FOR IT WHEN
a function should work with types you don’t own, including ones written later.
SKIP IT WHEN
you need to share code, not just method names: use a base class or a mixin (chapter 09).
HOW TO SPOT IT
a “file-like” parameter, dunder methods such as __len__, Protocol classes, abc checks.
IN PYTHON
Built in: every len(), for loop and with statement calls a protocol method.
SEEN IN
  • CPython 3.15 · Lib/json/__init__.pyjson.load calls fp.read() and nothing else, so any object with read() works.
  • CPython 3.15 · Lib/_collections_abc.pyisinstance(x, Sized) looks for __len__ on the class; nothing has to inherit or register.
  • CPython 3.15 · Lib/typing.pytyping.Protocol lets a type checker verify the method names before the code runs.
STEP 01 / 05

A BASE CLASS TURNS AWAY TYPES YOU DON’T OWN

Fig 1.1The same four objects in both panels; only the contract changes, and the left one rejects three.

Picture json.load written the way some languages push you to: it accepts only subclasses of one base class, JSONSource, and checks with isinstance first. A file from open(), an io.StringIO and an HTTP response from urlopen() all have a read() method, and all three are refused.

Each of those classes would have to inherit from JSONSource to pass, and their authors have never heard of it. The check protects nothing: all load needs is the one method it is about to call.

A BASE-CLASS CONTRACT TURNS AWAY STRINGIO
import io
from abc import ABC, abstractmethod


class JSONSource(ABC):  # a design that names one base class
    @abstractmethod
    def read(self) -> str: ...


def load(src: JSONSource) -> str:
    if not isinstance(src, JSONSource):
        raise TypeError(f"expected a JSONSource, got {type(src).__name__}")
    return src.read()


load(io.StringIO('{"id": 7}'))  # → TypeError: expected a JSONSource, got StringIO
STEP 02 / 05

THE CONTRACT IS A METHOD NAME, NOT A CLASS

Fig 1.2There is no dashed “is a” arrow anywhere: the objects share a method name, not a parent class.

A protocol is the set of method names a caller relies on. Python’s name for using one is duck typing: the caller calls the method, and any object that has it works. Nothing is declared and nothing is registered.

Every pattern in this part builds on that rule. An adapter, a decorator or a proxy only has to offer the right method names to stand where the original object stood, because nobody checks its class.

STEP 03 / 05

JSON.LOAD CALLS READ() AND NOTHING ELSE

Fig 1.3Three unrelated classes on the right, and json.load accepts each one because each has read().

CPython’s json.load does one thing before parsing: it calls fp.read() and passes the result to json.loads. Its docstring states the contract in words: a file-like object that supports read().

So one function reads a file on disk, a string in memory, a gzip stream or a response body from the network, and none of those classes was written with json in mind. Since json.loads accepts bytes as well as str, the HTTP response needs no decoding first.

CPYTHON 3.15.2 · JSON.LOAD
# Lib/json/__init__.py · CPython 3.15.2 (trimmed)
def load(fp, *, cls=None, object_hook=None, parse_float=None,
         parse_int=None, parse_constant=None, object_pairs_hook=None,
         array_hook=None, **kw):
    """Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
    a JSON document) to a Python object.
    """
    return loads(fp.read(),  # ← the whole protocol: one method called read()
        cls=cls, object_hook=object_hook,
        parse_float=parse_float, parse_int=parse_int,
        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook,
        array_hook=array_hook, **kw)

Source: Lib/json/__init__.py at v3.15.2

STEP 04 / 05

ISINSTANCE CAN CHECK FOR A METHOD INSTEAD OF A PARENT

Fig 1.4Follow 1 to 6: nobody registered Deck, and the hook still answers True from Deck’s own __dict__.

Sometimes code has to branch on what an object can do. The collections.abc module holds abstract base classes such as Sized and Iterable, and isinstance(deck, Sized) is true for a Deck class that never inherits from Sized.

The answer comes from __subclasshook__. Before it looks at base classes, ABCMeta asks the ABC itself, and Sized walks the class’s method resolution order looking for __len__. A class that sets __len__ = None opts out.

CPYTHON 3.15.2 · SIZED AND _CHECK_METHODS
# Lib/_collections_abc.py · CPython 3.15.2 (trimmed)
def _check_methods(C, *methods):
    mro = C.__mro__
    for method in methods:
        for B in mro:  # ← the class, then each of its bases
            if method in B.__dict__:
                if B.__dict__[method] is None:  # ← __len__ = None opts a class out
                    return NotImplemented
                break
        else:
            return NotImplemented
    return True


class Sized(metaclass=ABCMeta):
    __slots__ = ()

    @classmethod
    def __subclasshook__(cls, C):  # ← ABCMeta asks this before it looks at bases
        if cls is Sized:
            return _check_methods(C, "__len__")
        return NotImplemented

Source: Lib/_collections_abc.py at v3.15.2

A CLASS WITH __LEN__ IS SIZED WITHOUT INHERITING
from collections.abc import Iterable, Sized


class Deck:
    def __init__(self) -> None:
        self.cards = ["A", "K", "7"]

    def __len__(self) -> int:
        return len(self.cards)


isinstance(Deck(), Sized)  # → True
Sized in Deck.__mro__  # → False  (no base class, no register())
isinstance(Deck(), Iterable)  # → False  (no __iter__)
STEP 05 / 05

TYPING.PROTOCOL GIVES DUCK TYPING A TYPE CHECKER

Fig 1.5The accent column is the one to reach for in new code: it is checked before the program runs.

In your own code, write the protocol down with typing.Protocol. A parameter annotated src: SupportsRead accepts any class with a matching read method, with no inheritance, and a type checker such as mypy rejects a wrong argument before the program runs.

Add @runtime_checkable only when code must also test at run time. That test looks at names alone: a class whose read is a string passes, as the last line of the sample shows.

A PROTOCOL OF YOUR OWN
import io
from typing import Protocol, runtime_checkable


@runtime_checkable
class SupportsRead(Protocol):
    def read(self, size: int = -1, /) -> str: ...


def first_line(src: SupportsRead) -> str:  # any object with read() type-checks
    return src.read().partition("\n")[0]


class Sign:
    read = "a string, not a method"


first_line(io.StringIO("total 42\nrest"))  # → 'total 42'
isinstance(io.StringIO(), SupportsRead)  # → True
isinstance(Sign(), SupportsRead)  # → True  (only the name is checked)

COMMON QUESTIONS

Is duck typing the same as having no types?
No. The type is the set of methods the code calls. typing.Protocol writes that set down so a type checker can verify it, and the program runs the same with or without the annotation.
When should I still inherit from an abstract base class?
When you want the methods it adds. Inheriting collections.abc.Sequence gives you index, count and in for two methods of your own (chapter 09). For a contract alone, a Protocol is enough.

WHAT YOU NOW KNOW

  • json.load calls fp.read() and nothing else, so a file, a StringIO and an HTTP response all work without a shared base class.
  • isinstance(deck, Sized) is True because Sized.__subclasshook__ finds __len__ on Deck, with no registration.
  • typing.Protocol writes the method contract down; mypy checks it before the code runs, and runtime_checkable checks only names.
DEEP DIVE: asking forgiveness, special methods and the type, history, look-alikes

Ask forgiveness, not permission

Python style calls the method and handles the failure (try, then except AttributeError or TypeError) rather than testing first with hasattr. One lookup replaces two, and nothing can change between the check and the use. The style has a name in the Python glossary: EAFP, easier to ask for forgiveness than permission.

Special methods are looked up on the type

len(x) calls the __len__ defined on type(x), never one stored on the instance. The same holds for iteration, with, == and every other operator. That rule is why a proxy that forwards attributes with __getattr__ still fails len() (chapter 06).

History

PEP 3119 added abstract base classes and __subclasshook__ in Python 2.6 and 3.0. PEP 544 added typing.Protocol in Python 3.8, in 2019, so type checkers could follow the duck typing Python code already used.

LOOKS LIKE

Adapter
An adapter makes an object fit a protocol it lacks; the protocol is the shape it fits.
A wrapper class whose methods translate calls to another object.
Mixins
A mixin shares code; a protocol shares only method names.
A base class whose name ends in Mixin and that has method bodies.
Abstract base class
An ABC can force subclasses to implement methods and can add helpers.
class X(ABC) with @abstractmethod, and subclasses that inherit it.

Adapter: make one interface fit another

An adapter is a class that wraps an object and translates calls from the interface your code uses into the one that object has. The requests library is the everyday example: it sends every request through urllib3, yet your code never imports urllib3, catches its errors or reads its response type. How does requests put urllib3 to work behind an interface urllib3 doesn’t have?

THE PATTERN

you’ll write it often

Wrap an object that does the job, and translate each call from the interface your code uses to the one it has.

ROLES
  • clientcalls the interface it knows: Session calls send()
  • target interfacewhat the client expects: BaseAdapter.send()
  • adaptertranslates each call in and each result out: HTTPAdapter
  • adapteedoes the work behind its own interface: urllib3’s connection pool
REACH FOR IT WHEN
a library does the job, but its calls, arguments, errors or return types don’t match your code.
SKIP IT WHEN
you own both sides; change one of them instead of adding a layer between them.
HOW TO SPOT IT
a class named *Adapter or *Wrapper whose methods convert arguments, call one other object and convert the result.
IN PYTHON
A small class, or a function or lambda when the interface is a single call.
SEEN IN
  • requests 2.34 · requests/adapters.pyHTTPAdapter.send() turns a PreparedRequest into a urllib3 urlopen() call and builds a requests Response.
  • asgiref 3.13 · asgiref/wsgi.pyWsgiToAsgi lets a WSGI application, such as a Flask app, run under an ASGI server.
  • CPython 3.15 · Lib/socket.pysocket.makefile() wraps a socket in a file object, so code that reads files can read the network.
STEP 01 / 05

WITHOUT AN ADAPTER, URLLIB3 LEAKS INTO EVERY CALLER

Fig 2.1Read the bottom row of each panel: the right-hand design keeps a urllib3 change inside one class.

urllib3 does the hard part of HTTP: connection pools, TLS, retries, chunked bodies. Its interface is its own, though: a urlopen() call with a dozen keyword arguments, its own exception classes and its own response type.

If every function that fetches a URL called urllib3 directly, every one of them would depend on those details. A urllib3 release that renames an argument or an exception would mean edits across the whole codebase.

STEP 02 / 05

THE ADAPTER IS THE ONLY CLASS THAT KNOWS BOTH SIDES

Fig 2.2Only the accent box has arrows to both sides; the client never touches the existing class.

An adapter sits between a client and an existing class. It offers the interface the client expects, called the target, and implements each method by calling the existing class, called the adaptee, through the adaptee’s own interface.

Translation runs both ways. Arguments go in reshaped, and results and errors come back reshaped, so nothing of the adaptee’s interface reaches the client.

STEP 03 / 05

REQUESTS NEVER OPENS A SOCKET ITSELF

Fig 2.3HTTPAdapter, the accent box, is the one class in requests that calls urllib3.

A Session keeps a table of adapters by URL prefix and mounts an HTTPAdapter for https:// and http:// when it is created. Every request goes through adapter.send(request), the one method in BaseAdapter’s interface that does the work.

Inside send, HTTPAdapter takes a pool, passes the prepared request’s method, URL, body and headers to urlopen(), and turns urllib3’s errors into requests’ own: a ProtocolError becomes ConnectionError. Redirects stay off here because the Session follows them. build_response then copies status, headers and body into a requests Response.

REQUESTS 2.34.2 · HTTPADAPTER.SEND
# src/requests/adapters.py · requests 2.34.2 (trimmed; type hints dropped)
class HTTPAdapter(BaseAdapter):  # ← the adapter; BaseAdapter.send() is the interface Session calls
    def send(self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None):
        conn = self.get_connection_with_tls_context(  # ← the adaptee: a urllib3 connection pool
            request, verify, proxies=proxies, cert=cert
        )
        url = self.request_url(request, proxies)
        ...
        try:
            resp = conn.urlopen(  # ← the same request, spelled the way urllib3 takes it
                method=request.method,
                url=url,
                body=request.body,
                headers=request.headers,
                redirect=False,  # ← redirects stay in requests’ Session
                retries=self.max_retries,
                timeout=resolved_timeout,
            )
        except (ProtocolError, OSError) as err:
            raise ConnectionError(err, request=request)  # ← urllib3’s errors become requests’ errors
        ...
        return self.build_response(request, resp)  # ← urllib3’s response becomes a requests Response

Source: src/requests/adapters.py at v2.34.2

STEP 04 / 05

ONE GET CROSSES THE ADAPTER TWICE

Fig 2.4Follow 1 to 6: urllib3’s response type stops at HTTPAdapter on the way back.

The adapter is crossed twice per request: on the way in it reshapes the call, and on the way out it reshapes the result. Your code sees requests types at both ends.

Because the Session picks the adapter by URL prefix, you can mount your own for one host or scheme. The usual reason is retries: HTTPAdapter(max_retries=Retry(…)) hands a urllib3 Retry policy down to the pool, and no code that calls session.get changes.

MOUNTING AN ADAPTER WITH RETRIES (REQUESTS API, NOT RUN HERE)
import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

session = requests.Session()  # mounts HTTPAdapter() for "https://" and "http://"
session.mount("https://", HTTPAdapter(max_retries=Retry(total=3, backoff_factor=0.5)))

response = session.get("https://api.example.com/rates", timeout=5)
response.status_code  # a requests Response: no urllib3 type reaches this code
STEP 05 / 05

WHEN THE INTERFACE IS ONE CALL, A FUNCTION IS THE ADAPTER

Fig 2.5Both columns satisfy the same one-call interface; the accent one is the shorter answer.

When your code depends on a small interface and a class you don’t own offers the job under other names, write a class with the names your code calls. Here alert() needs something it can call with a text, and MailerAdapter.send turns that text into a subject and body for LegacyMailer.deliver.

If the interface is a single call, a function or a lambda is a complete adapter, and the class adapter’s bound method is passed the same way. Reach for the class when the adapter must hold state, such as the team list here.

AN ADAPTER OF YOUR OWN, AND THE FUNCTION FORM
from collections.abc import Callable


class LegacyMailer:  # a class you don't own
    def deliver(self, to: list[str], subject: str, body: str) -> int:
        print(f"mail to {', '.join(to)}: {subject}")
        return 250


class MailerAdapter:  # gives LegacyMailer the send(text) interface your code calls
    def __init__(self, mailer: LegacyMailer, team: list[str]) -> None:
        self.mailer, self.team = mailer, team

    def send(self, text: str) -> None:
        subject, _, body = text.partition("\n")
        self.mailer.deliver(self.team, subject, body)


def alert(send: Callable[[str], None]) -> None:  # the interface is one call
    send("disk 91% full\nhost db-2")


alert(MailerAdapter(LegacyMailer(), ["ops@example.com"]).send)  # → mail to ops@example.com: disk 91% full
alert(lambda text: print("sms:", text.splitlines()[0]))  # → sms: disk 91% full

COMMON QUESTIONS

What is the difference between an adapter and a facade?
An adapter keeps the job and changes the interface to one a client already expects. A facade invents a simpler interface over several objects (chapter 03).
Why does requests mount adapters by URL prefix?
So one Session can use different transports: a retrying adapter for one API, a test adapter that returns canned responses, or a third-party adapter for another protocol.

WHAT YOU NOW KNOW

  • requests calls urllib3 in one class: HTTPAdapter.send turns a PreparedRequest into urlopen() and builds a requests Response.
  • Errors cross the adapter too: urllib3’s ProtocolError reaches your code as requests.ConnectionError.
  • For a one-call interface, a function or lambda is a complete adapter; use a class when it holds state.
DEEP DIVE: also in asgiref, object and class adapters, testing through an adapter, look-alikes

Also in asgiref: WsgiToAsgi

WSGI is the synchronous interface Flask and older Django apps speak. ASGI is the asynchronous one that servers such as Uvicorn speak. asgiref’s WsgiToAsgi wraps a WSGI application so an ASGI server can run it: each request’s WSGI call runs in a thread, and its status, headers and body are translated into ASGI messages.

Source: asgiref/wsgi.py at 3.13.0

Object adapters and class adapters

The adapters in this chapter hold the adaptee, which makes them object adapters. A class adapter inherits from both the target and the adaptee instead. Python allows it, but every public method of the adaptee then leaks into the adapter’s interface, which is what the pattern was meant to prevent.

Testing through the adapter

A test can mount its own BaseAdapter subclass whose send returns a prepared Response. The code under test still calls session.get, and no socket is ever opened. The requests-mock library is built on this mount point.

LOOKS LIKE

Facade
A facade gives several objects a new, smaller interface; an adapter fits one object to an existing one.
One function making many calls, against one class wrapping one object.
Decorator
A decorator keeps the interface and adds behaviour; an adapter changes the interface.
The wrapper’s methods have the same names as the wrapped object’s.
Proxy
A proxy keeps the interface and controls access to the real object.
Every attribute is forwarded unchanged, often through __getattr__.

Facade: one simple call over a busy subsystem

A facade is one function or class that runs the common job of a subsystem in a single call. Python’s subprocess module has a full subsystem, Popen, which handles pipes, timeouts, signals and exit codes, and most code needs one sentence of it: run this, give me the output, fail if it failed. What does subprocess.run do for you that you would otherwise write around Popen every time?

THE PATTERN

you’ll write it often

Give the common case of a complex subsystem one simple entry point, and leave the full subsystem available.

ROLES
  • clientwants the usual result, not the details
  • facadeone call that drives the subsystem in the right order: subprocess.run
  • subsystemthe full set of classes and calls: Popen, communicate, kill, poll
REACH FOR IT WHEN
callers keep repeating the same sequence of calls on a library, with the same cleanup.
SKIP IT WHEN
each caller needs a different sequence; a facade would grow a flag for every case.
HOW TO SPOT IT
a module-level function that creates objects, calls them in order, cleans up and returns plain data.
IN PYTHON
A function. Modules are often facades over their own classes.
SEEN IN
  • CPython 3.15 · Lib/subprocess.pyrun() wraps Popen, communicate, kill and poll in one call and returns a CompletedProcess.
  • requests 2.34 · requests/api.pyrequests.get() opens a Session, sends one request and closes the Session again.
  • CPython 3.15 · Lib/logging/__init__.pybasicConfig() creates a handler and a formatter and attaches them to the root logger.
STEP 01 / 05

RUNNING ONE COMMAND WITH POPEN TAKES FIVE STEPS

Fig 3.1The accent row is the step hand-written Popen code forgets most often.

Popen can do anything with a child process, and that is the trouble: doing the usual thing correctly takes five steps in the right order. Reading stdout and then stderr one after the other can deadlock when the child fills the other pipe, so communicate() reads both together.

A timeout raises in your process but leaves the child running, so you must kill it and collect it. A non-zero exit code is just a number until you check it.

ONE COMMAND, RUN CORRECTLY WITH POPEN
import subprocess
import sys

cmd = [sys.executable, "-c", "print('ok')"]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
try:
    out, err = proc.communicate(timeout=5)  # read both pipes, or a full one blocks the child
except subprocess.TimeoutExpired:
    proc.kill()  # never leave the child running
    proc.communicate()
    raise
if proc.returncode != 0:
    raise subprocess.CalledProcessError(proc.returncode, cmd, out, err)
print(out.strip())  # → ok
STEP 02 / 05

THE FACADE HIDES THE ORDER, NOT THE PARTS

Fig 3.2The client has one arrow, to the facade; the subsystem boxes stay open to anyone who needs them.

A facade is an extra entry point, not a wall. It knows the order the subsystem’s parts must be called in and the cleanup the common case needs, and it returns a plain result.

Callers with unusual needs still use the subsystem directly. The facade only removes the boilerplate from the case most callers have.

STEP 03 / 05

SUBPROCESS.RUN IS POPEN WITH THE USUAL CASE FILLED IN

Fig 3.3Every box on the right is a call you would otherwise write yourself; run makes all of them.

subprocess.run is about seventy lines, and most of them are the steps from the table. It opens Popen in a with block so the pipes always close, calls communicate with your timeout, kills the child on any exception, then reads the exit code with poll().

It returns a CompletedProcess, a plain record of the arguments, the exit code and the output. With check=True, a non-zero exit raises CalledProcessError instead.

CPYTHON 3.15.2 · SUBPROCESS.RUN
# Lib/subprocess.py · CPython 3.15.2 (trimmed)
def run(*popenargs,
        input=None, capture_output=False, timeout=None, check=False, **kwargs):
    ...
    if capture_output:
        ...
        kwargs['stdout'] = PIPE
        kwargs['stderr'] = PIPE
    with Popen(*popenargs, **kwargs) as process:  # ← the subsystem, opened and closed for you
        try:
            stdout, stderr = process.communicate(input, timeout=timeout)
        except TimeoutExpired as exc:
            process.kill()  # ← the cleanup callers forget
            ...
            raise
        except:  # Including KeyboardInterrupt, communicate handled that.
            process.kill()
            raise
        retcode = process.poll()
        if check and retcode:
            raise CalledProcessError(retcode, process.args,
                                     output=stdout, stderr=stderr)
    return CompletedProcess(process.args, retcode, stdout, stderr)  # ← plain data back

Source: Lib/subprocess.py at v3.15.2

STEP 04 / 05

ONE RUN CALL STARTS, WAITS, CHECKS AND CLOSES

Fig 3.4Follow 1 to 7: the facade makes several calls on the subsystem for the one you made.

One call to run turns into the whole conversation with Popen. The child here is the running Python interpreter itself, so the sample gives the same output on any machine.

The last line passes check=True to a command that exits with status 3. run raises CalledProcessError, carrying the exit code and any captured output, instead of handing back a result your code might forget to test.

THE SAME COMMAND THROUGH THE FACADE
import subprocess
import sys

done = subprocess.run(
    [sys.executable, "-c", "print('ok')"],
    capture_output=True, text=True, timeout=5, check=True,
)
done.stdout  # → 'ok\n'
done.returncode  # → 0
subprocess.run([sys.executable, "-c", "raise SystemExit(3)"]).returncode  # → 3
subprocess.run([sys.executable, "-c", "raise SystemExit(3)"], check=True)  # → CalledProcessError
STEP 05 / 05

LOGGING.BASICCONFIG IS A FACADE YOU ALREADY CALL

Fig 3.5Five calls on the left, one on the right; basicConfig is the standard library’s own version.

Write a facade when your own code repeats a setup sequence. Configuring a logger takes a logger, a handler, a formatter, a level and a propagation flag, in that order; get_logger() does it once and returns the logger.

The standard library ships the same idea: logging.basicConfig() creates a handler and a formatter for the root logger in one call. The guard on logger.handlers matters: without it, a second call adds a second handler, and every line prints twice.

A FACADE OVER YOUR OWN LOGGING SETUP
import logging
import sys


def get_logger(name: str, level: int = logging.INFO) -> logging.Logger:
    """The setup every service repeats, in one call."""
    logger = logging.getLogger(name)
    if not logger.handlers:  # a second call must not print every line twice
        handler = logging.StreamHandler(sys.stdout)
        handler.setFormatter(logging.Formatter("%(levelname)s %(name)s: %(message)s"))
        logger.addHandler(handler)
        logger.setLevel(level)
        logger.propagate = False
    return logger


log = get_logger("billing")
log.info("invoice %s sent", 42)  # → INFO billing: invoice 42 sent
get_logger("billing") is log  # → True

COMMON QUESTIONS

Does a facade stop me from using Popen?
No. Popen is still public, and run passes extra keyword arguments straight to it. Use Popen when you need to stream output or talk to the child while it runs.
What is the difference between a facade and an adapter?
An adapter fits one object to an interface a client already expects (chapter 02). A facade defines a new, smaller interface of its own over several objects.

WHAT YOU NOW KNOW

  • subprocess.run wraps Popen, communicate, kill and poll in one call and returns a plain CompletedProcess.
  • run kills the child on a timeout or any other exception, the cleanup hand-written Popen code most often skips.
  • A facade of your own is a function over a setup you repeat; logging.basicConfig is the standard library’s.
DEEP DIVE: also in requests, modules as facades, testing at the facade, look-alikes

Also in requests: a convenient call with a cost

requests.get(url) is requests.request("GET", url), and that function opens a Session, sends one request and closes the Session in a with block. Each call therefore builds a new connection pool, so ten calls to one host open ten connections. Code that calls a host repeatedly should create one Session and reuse it.

Source: src/requests/api.py at v2.34.2

Modules as facades

shutil.copytree walks a directory with os.scandir and copies each file; json.dumps builds or reuses a JSONEncoder and calls it. A module-level function over a module’s own classes is the most common facade in Python, and the standard library is full of them.

Testing at the facade

Patch at the facade’s boundary. A test that replaces subprocess.run with a function returning a CompletedProcess is a few lines; faking Popen’s pipes, timeouts and exit codes is far more.

LOOKS LIKE

Adapter
An adapter fits one object to an existing interface; a facade defines a new one over many.
Count the objects behind it: one for an adapter, several for a facade.
Mediator (part 3)
A mediator sits between objects that talk to each other; a facade is called by clients only.
In a mediator, the parts hold a reference to the coordinator.

Decorator: wrap an object to add one behaviour

An object decorator wraps another object, offers the same methods and adds one behaviour around them. When you open a text file, Python doesn’t hand you one object: it hands you three, each wrapped around the next, and each layer adds one job. Why does open() build three objects instead of one, and which of them is a decorator?

THE PATTERN

you’ll write it sometimes

Wrap an object in another with the same methods, which adds one behaviour and passes each call on.

ROLES
  • componentthe base object that does the job: FileIO
  • decoratorholds a component, adds one behaviour, forwards the call: BufferedReader
  • interfacethe methods both share: read() and readinto()
REACH FOR IT WHEN
behaviours combine freely (buffering, compression, counting) and a subclass per combination would multiply.
SKIP IT WHEN
there is only ever one combination; one class is easier to read and debug.
HOW TO SPOT IT
a constructor that takes an object of the same kind: BufferedReader(raw), GzipFile(fileobj=f), Middleware(app).
IN PYTHON
A class with the same methods. For functions, Python has @ syntax instead (chapter 05).
SEEN IN
  • CPython 3.15 · Lib/_pyio.pyopen() wraps a FileIO in a BufferedReader, then the buffer in a TextIOWrapper.
  • CPython 3.15 · Lib/gzip.pyGzipFile(fileobj=f) decompresses any binary file object as you read from it.
  • Werkzeug 3.1 · werkzeug/middleware/proxy_fix.pyProxyFix(app) is itself a WSGI app: it fixes the client address, then calls the app it wraps.
STEP 01 / 04

A SUBCLASS PER COMBINATION MULTIPLIES

Fig 4.1Two bars on one scale: 24 classes against 6.

Bytes can come from a file, a socket or memory, and each source may need buffering, decompression and decoding to text, in any mix. With one subclass per combination, 3 sources and 3 optional behaviours make 24 classes, and a fourth behaviour doubles that.

With decorators, each behaviour is one class that wraps any source, and you stack the ones you need at run time: 3 source classes plus 3 wrappers, 6 in all.

STEP 02 / 04

A DECORATOR HAS THE SAME METHODS AS WHAT IT WRAPS

Fig 4.2Both dashed arrows reach the same interface: to the client, the wrapper and the wrapped look alike.

A decorator holds a component and has the component’s methods. Each method does its one extra job and calls the same method on the component, so to the client a decorated object and a plain one are interchangeable.

Because a decorator accepts any component with those methods, decorators stack: a decorator can wrap another decorator. The order of the stack is the order the behaviours run in.

STEP 03 / 04

OPEN() HANDS YOU THREE OBJECTS, ONE INSIDE ANOTHER

Fig 4.3Call 2 happens only when the buffer runs out; most reads stop at the accent box.

open() builds the stack in three lines. FileIO reads bytes from the operating system. BufferedReader wraps it with the same read methods plus a buffer of at least 131,072 bytes in Python 3.15, so a thousand small reads cost a handful of system calls.

The top layer, TextIOWrapper, is not a decorator: it takes bytes and gives back str, a different interface, which makes it an adapter (chapter 02). f.buffer and f.buffer.raw reach the layers underneath, as the second sample shows.

CPYTHON 3.15.2 · _PYIO.OPEN, THE PURE-PYTHON TWIN OF IO.OPEN
# Lib/_pyio.py · CPython 3.15.2, the pure-Python twin of io.open (trimmed)
def open(file, mode="r", buffering=-1, encoding=None, errors=None,
         newline=None, closefd=True, opener=None):
    ...
    raw = FileIO(file,
                 (creating and "x" or "") +
                 (reading and "r" or "") +
                 (writing and "w" or "") +
                 (appending and "a" or "") +
                 (updating and "+" or ""),
                 closefd, opener=opener)  # ← the base object: bytes from the OS
    ...
    if updating:
        buffer = BufferedRandom(raw, buffering)
    elif creating or writing or appending:
        buffer = BufferedWriter(raw, buffering)
    elif reading:
        buffer = BufferedReader(raw, buffering)  # ← the decorator: the same read(), plus a buffer
    ...
    text = TextIOWrapper(buffer, encoding, errors, newline, line_buffering)  # ← bytes in, str out
    ...
    return result

Source: Lib/_pyio.py at v3.15.2

THE THREE LAYERS OF AN OPEN TEXT FILE
import io
import tempfile

with tempfile.NamedTemporaryFile("w", suffix=".txt", delete_on_close=False) as tmp:
    tmp.write("first line\n")
    tmp.close()
    with open(tmp.name) as f:
        layers = [type(f).__name__, type(f.buffer).__name__, type(f.buffer.raw).__name__]
        line = f.readline()

layers  # → ['TextIOWrapper', 'BufferedReader', 'FileIO']
line  # → 'first line\n'
io.DEFAULT_BUFFER_SIZE  # → 131072
STEP 04 / 04

OVERRIDE WHAT YOU CHANGE, FORWARD THE REST

Fig 4.4The red row is the gap __getattr__ leaves: special methods need their own definitions.

To write a decorator, define the methods whose behaviour you change and forward the rest. CountingReader counts bytes in read(), and __getattr__, which Python calls only for names the wrapper lacks, passes tell(), seek() and close() to the wrapped object.

Special methods are the exception. Python looks them up on the type, never through __getattr__, so a with statement or a for loop over the wrapper fails until you define __enter__, __exit__ and __iter__ yourself.

A DECORATOR THAT COUNTS BYTES
import io
from typing import BinaryIO


class CountingReader:
    """Wraps any binary file object and counts the bytes read through it."""

    def __init__(self, raw: BinaryIO) -> None:
        self._raw = raw
        self.bytes_read = 0

    def read(self, size: int = -1) -> bytes:
        data = self._raw.read(size)
        self.bytes_read += len(data)  # the one added behaviour
        return data

    def __getattr__(self, name: str) -> object:  # everything else goes to the wrapped object
        return getattr(self._raw, name)


src = CountingReader(io.BytesIO(b"GIF89a" + bytes(250)))
src.read(6)  # → b'GIF89a'
len(src.read())  # → 250
src.bytes_read  # → 256
src.tell()  # → 256  (forwarded to BytesIO)

COMMON QUESTIONS

Is this the same as a @decorator?
No. The @ syntax wraps a function in another function (chapter 05). The idea is the same, one behaviour added around a call, but this pattern wraps objects and their methods.
Why not inherit from BytesIO instead?
Then CountingReader would only work with BytesIO. Wrapping accepts any binary file object: a real file, a socket’s file or a GzipFile.

WHAT YOU NOW KNOW

  • open() returns TextIOWrapper over BufferedReader over FileIO; the buffer is a decorator, and the text layer is an adapter.
  • One decorator per behaviour stacks at run time: 6 classes cover what 24 subclasses would.
  • Write your own by overriding what changes and forwarding the rest with __getattr__; special methods need explicit definitions.
DEEP DIVE: WSGI middleware, gzip, what each layer costs, look-alikes

WSGI middleware is a stack of decorators

A WSGI application is a callable that takes environ and start_response. Middleware such as Werkzeug’s ProxyFix takes an application and is one: its __call__ fixes the client address from proxy headers, then calls the application it wraps. Stacking middleware is stacking decorators.

Source: src/werkzeug/middleware/proxy_fix.py at 3.1.9

gzip.GzipFile

GzipFile(fileobj=f) reads compressed bytes from any binary file object and returns the decompressed bytes from the same read() method. It can sit on a file, a socket’s file or a BytesIO, and a TextIOWrapper can sit on it.

Each layer costs a call

A stack of five wrappers turns one read into five method calls, and a traceback into five frames. For a loop of tiny reads, a buffer near the bottom of the stack pays for the layers above it, which is why open() puts BufferedReader right on top of FileIO.

LOOKS LIKE

Function decorators
The @ syntax wraps one function; this pattern wraps an object and its methods.
A def inside a def, against a class whose constructor takes the wrapped object.
Proxy
A proxy controls how the real object is reached and adds nothing to the result.
A proxy often looks its subject up on each use instead of holding it.
Chain of Responsibility (part 3)
In a chain, any handler may stop the request; every decorator in a stack runs.
A handler that returns without calling the next one.

Function decorators: wrap a function, keep its name

A function decorator is a callable that takes a function and returns a new function that runs code around it. The @ line above a def is shorthand for one assignment, and libraries use it to add retries, caching, routing and command-line parsing in one line. What does @retry(stop=stop_after_attempt(3)) turn your function into, and why does the result still look like your function?

THE PATTERN

you’ll write it often

Take a function and return a new one that runs extra code before, after or around every call.

ROLES
  • decoratora callable that takes a function and returns a wrapper: retry
  • wrapperthe function callers now hold, which calls the original: wrapped_f
  • originalyour function, still reachable as __wrapped__
REACH FOR IT WHEN
the same code before and after a call (retry, cache, timing, permission checks) wraps many functions.
SKIP IT WHEN
only one function needs it; write the code inside that function.
HOW TO SPOT IT
@name or @name(...) above a def, and functools.wraps inside the decorator.
IN PYTHON
Built-in syntax since Python 2.4 (PEP 318); functools.wraps copies the name and docstring.
SEEN IN
  • tenacity 9.2 · tenacity/__init__.pyRetrying.wraps returns a wrapper that reruns your function under a retry policy.
  • CPython 3.15 · Lib/functools.pylru_cache wraps a function in a C-level cache, then copies its name with update_wrapper.
  • Click 8.5 · click/decorators.py@click.command() turns your function into a Command object, which chapter 08 nests into trees.
STEP 01 / 05

THE SAME RETRY LOOP, COPIED INTO EVERY FUNCTION

Fig 5.1The red row is how copies drift: one of three loops lost its wait between tries.

Code that calls a network needs retries, and a retry loop is about eight lines: a for loop, a try, an except for the right exception, a wait. Copy it into three functions and the copies drift apart: one forgets the wait, another retries on the wrong error.

The loop has nothing to do with what each function fetches. It belongs around the function, not inside it.

STEP 02 / 05

@ IS ONE ASSIGNMENT: F = DECORATOR(F)

Fig 5.2The caller’s arrow lands on the wrapper; the original runs only when the wrapper calls it.

Writing @logged above def fetch runs fetch = logged(fetch) once, when the def statement executes. The decorator returns a new function, usually a closure that remembers the original, and the name fetch now points to that wrapper.

Every later call goes to the wrapper. It can run code before the original, after it, instead of it, or several times, as a retry does.

STEP 03 / 05

TENACITY’S DECORATOR IS A FACTORY THAT RETURNS A WRAPPER

Fig 5.3The accent box is what the name fetch_rates points to once the @ line has run.

With arguments, @retry(stop=…) is a decorator factory: the call returns the real decorator, and Python applies that to your function. tenacity’s retry() also accepts the bare @retry form by checking whether its only argument is callable.

The wrapper comes from Retrying.wraps. It copies your function’s name, docstring and defaults with functools.wraps, makes a fresh copy of the retry state for each call, and attaches the policy as fetch_rates.retry so tests can inspect or change it.

TENACITY 9.2.1 · RETRY AND RETRYING.WRAPS
# tenacity/__init__.py · tenacity 9.2.1 (trimmed; type hints dropped)
class BaseRetrying(ABC):
    def wraps(self, f):  # ← takes your function, returns the one callers will hold
        @functools.wraps(  # ← copy f’s name, docstring and defaults onto the wrapper
            f, functools.WRAPPER_ASSIGNMENTS + ("__defaults__", "__kwdefaults__")
        )
        def wrapped_f(*args, **kw):
            ...
            copy = self.copy()  # ← fresh retry state for this one call
            ...
            return copy(f, *args, **kw)  # ← run f under the retry policy

        wrapped_f.retry = self  # ← the policy stays reachable from the function
        ...
        return wrapped_f


def retry(*dargs, **dkw):
    if len(dargs) == 1 and callable(dargs[0]):  # ← bare @retry, with no parentheses
        return retry()(dargs[0])

    def wrap(f):  # ← @retry(...) returns this, and Python calls it with your function
        ...
        r = Retrying(*dargs, **dkw)
        return r.wraps(f)

    return wrap

Source: tenacity/__init__.py at 9.2.1

A RETRY POLICY ON A FLAKY CALL (TENACITY API, NOT RUN HERE)
import requests
from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_exponential


@retry(
    stop=stop_after_attempt(3),  # give up after the third try
    wait=wait_exponential(multiplier=0.5),  # wait 0.5 s, then 1 s, between tries
    retry=retry_if_exception_type(requests.ConnectionError),
)
def fetch_rates(base: str) -> dict:
    return requests.get(f"https://api.example.com/rates/{base}", timeout=5).json()


fetch_rates.__name__  # 'fetch_rates': functools.wraps kept it
fetch_rates.retry.stop  # the policy, reachable from the function
STEP 04 / 05

ONE CALL, TWO ATTEMPTS, ONE RESULT

Fig 5.4Follow 1 to 8: the failed attempt never reaches your code, only the second one’s result.

From the caller’s side nothing changed: it called fetch_rates and got rates back. In between, the wrapper caught a ConnectionError, waited, and called the original again.

The wait comes from wait_exponential(multiplier=0.5): half a second after the first failure, one second after the second. With stop_after_attempt(3), a third failure raises RetryError wrapping the last exception, so the caller still learns that every attempt failed.

STEP 05 / 05

WITHOUT FUNCTOOLS.WRAPS, EVERY FUNCTION IS CALLED WRAPPER

Fig 5.5One line apart: the accent panel is what debuggers, docs and logs show for your function.

A decorator of your own takes three parts: an outer function that receives the original, an inner wrapper, and functools.wraps on the wrapper. That line copies the name, docstring and module, and sets __wrapped__, which inspect.signature follows to report the real parameters.

Typing it with ParamSpec, the [**P, R] in the sample, tells a type checker that the wrapper takes exactly the original’s arguments and returns its result type.

A TYPED DECORATOR, WITH AND WITHOUT FUNCTOOLS.WRAPS
import functools
import inspect
from collections.abc import Callable


def logged[**P, R](func: Callable[P, R]) -> Callable[P, R]:
    @functools.wraps(func)  # without this line, fetch.__name__ would be 'wrapper'
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        print("calling", func.__name__, args)
        return func(*args, **kwargs)

    return wrapper


@logged  # the same as: fetch = logged(fetch)
def fetch(url: str, timeout: float = 5.0) -> int:
    """Return the HTTP status code."""
    return 200


def bare(func):  # the same decorator without functools.wraps
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)

    return wrapper


fetch("https://example.com")  # → 200
fetch.__name__, bare(fetch.__wrapped__).__name__  # → ('fetch', 'wrapper')
str(inspect.signature(fetch))  # → '(url: str, timeout: float = 5.0) -> int'
str(inspect.signature(bare(fetch.__wrapped__)))  # → '(*args, **kwargs)'
fetch.__doc__  # → 'Return the HTTP status code.'

COMMON QUESTIONS

When does the decorator itself run?
Once, when the def statement executes, usually at import time. A decorator that opens a connection or reads a file there does it at import, not on the first call.
In what order do stacked decorators apply?
Bottom up. The decorator nearest the def wraps it first and the top one wraps the result, so the top one’s code runs first on each call.

WHAT YOU NOW KNOW

  • @retry(stop=…) is a decorator factory: the call returns a decorator, and that returns wrapped_f, which the name then points to.
  • tenacity’s wrapper reruns the original on a ConnectionError and waits between attempts; the caller sees only the final result.
  • Put functools.wraps on every wrapper, and type it with ParamSpec so the signature survives.
DEEP DIVE: also in functools, decorators that register, class decorators, look-alikes

Also in functools: lru_cache

lru_cache(maxsize=128) returns decorating_function, which builds a C-level caching wrapper and passes it through update_wrapper, the function behind functools.wraps. cache_info() and cache_clear() are attributes set on that wrapper, the same trick as tenacity’s .retry.

Source: Lib/functools.py at v3.15.2

Decorators that register

Flask’s @app.route(…) records the function in the app’s URL map and returns it unchanged. That is a registry using decorator syntax, not a wrapper: calling the function afterwards runs your code and nothing else.

Class decorators

@dataclass takes a class and returns it with generated __init__, __repr__ and __eq__ methods. The syntax and the timing are the same as for functions: it runs once, when the class statement finishes.

LOOKS LIKE

Decorator (objects)
The object pattern wraps an object and all its methods; @ wraps one function.
A class whose constructor takes the wrapped object, against a def inside a def.
Proxy
A proxy stands in for an object and decides how to reach it; a wrapper runs code around a call.
A proxy forwards attribute access; a wrapper calls one function.
Registry (part 1)
A registering decorator records the function and returns it unchanged.
No inner def: the decorator ends with return f.

Proxy: stand in for an object you can’t hold directly

A proxy is an object with the same interface as another object, which stands in for it and decides how to reach it. Flask lets you write from flask import request at the top of a module and use request inside any view, even while the server handles many requests at once. How can one global name mean a different request in every thread?

THE PATTERN

you’ll write it sometimes

Stand in for an object with its interface, and decide on each use how to reach it: find, delay, check or call it remotely.

ROLES
  • clientuses the proxy as if it were the real object
  • proxyhas the same attributes and finds the subject on each use: LocalProxy
  • subjectthe real object: the Request for the current request
REACH FOR IT WHEN
the real object is per request, expensive, remote or guarded, and callers should not have to care.
SKIP IT WHEN
you can pass the real object as an argument; that is simpler to read and to test.
HOW TO SPOT IT
__getattr__ that forwards to _get_current_object() or _wrapped, weakref.proxy, names like *Proxy or Lazy*.
IN PYTHON
__getattr__ plus each special method you need, since Python looks those up on the type.
SEEN IN
  • Werkzeug 3.1 · werkzeug/local.pyLocalProxy looks the current object up in a ContextVar on every attribute access.
  • Flask 3.1 · flask/globals.pyrequest, session, current_app and g are all LocalProxy objects.
  • CPython 3.15 · Lib/multiprocessing/managers.pyBaseProxy sends each method call to an object that lives in another process.
STEP 01 / 04

A PLAIN GLOBAL HOLDS ONE REQUEST FOR EVERY THREAD

Fig 6.1Hop 4 is the bug: thread A reads a value thread B wrote a moment earlier.

A web server handles many requests at once, on threads or async tasks. If the framework kept the current request in a plain module-level variable, each new request would overwrite it, and code running for one user would read another user’s request.

Passing the request to every function that needs it avoids the problem, and Django does exactly that. Flask chose another design: one importable name that finds the right request each time it is used.

STEP 02 / 04

THE PROXY FINDS THE REAL OBJECT ON EVERY USE

Fig 6.2The accent box never holds the subject; it asks the lookup above it every time.

A proxy has the same interface as its subject, so callers use it without knowing. What it adds is control over how the subject is reached: found per thread, created on first use, checked for permission, or called in another process.

The kinds of proxy differ only in that decision. The client code is identical in every case, which is the point of the pattern.

STEP 03 / 04

FLASK’S REQUEST IS A PROXY OVER A CONTEXTVAR

Fig 6.3Calls 1 to 3 run on every attribute read; call 2 gives each thread or task its own answer.

Flask creates request once, at import, as LocalProxy(_cv_request, "request"). The first argument is a ContextVar, a variable with one value per thread and per async task. When Flask starts handling a request, it sets that variable for the current context.

Every attribute read on the proxy runs _get_current_object(): it reads the ContextVar, takes .request from the context it finds and forwards the read. Werkzeug defines each special method, such as == and bool(), the same way through _ProxyLookup, which is itself a descriptor.

FLASK 3.1.3 · REQUEST
# src/flask/globals.py · Flask 3.1.3 (trimmed)
_cv_request: ContextVar[RequestContext] = ContextVar("flask.request_ctx")
request: Request = LocalProxy(  # ← one importable name, a different Request per request
    _cv_request, "request", unbound_message=_no_req_msg
)

Source: src/flask/globals.py at 3.1.3

WERKZEUG 3.1.9 · LOCALPROXY
# src/werkzeug/local.py · Werkzeug 3.1.9 (trimmed; type hints dropped)
class LocalProxy(t.Generic[T]):
    def __init__(self, local, name=None, *, unbound_message=None):
        if name is None:
            get_name = _identity
        else:
            get_name = attrgetter(name)  # ← "request": the attribute to take from the context
        ...
        if isinstance(local, ContextVar):
            def _get_current_object():
                try:
                    obj = local.get()  # ← this thread’s, or this task’s, RequestContext
                except LookupError:
                    raise RuntimeError(unbound_message) from None
                return get_name(obj)
        ...
        object.__setattr__(self, "_get_current_object", _get_current_object)

    __getattr__ = _ProxyLookup(getattr)  # ← every attribute read goes through the lookup
    __eq__ = _ProxyLookup(operator.eq)  # ← and every special method, declared one by one

Source: src/werkzeug/local.py at 3.1.9

STEP 04 / 04

__GETATTR__ FORWARDS NAMES, NOT LEN() OR ==

Fig 6.4Everything in the red panel needs its own method on the proxy class, as Werkzeug writes them.

A small proxy needs one method. LazyProxy.__getattr__ runs only for names the proxy itself lacks; it builds the real object on first use, then forwards. That makes it a lazy proxy, the kind Django uses for its settings object (part 1, lazy initialization).

The sample’s last two lines show the limit. len() and == look for special methods on the class, so they never reach __getattr__: len raises TypeError, and == compares the proxy object itself. A complete proxy defines each special method it supports, which is why Werkzeug’s LocalProxy lists dozens.

A LAZY PROXY, AND WHAT __GETATTR__ MISSES
from collections.abc import Callable
from typing import Any


class LazyProxy:
    """Builds the real object on first use, then forwards to it."""

    def __init__(self, factory: Callable[[], Any]) -> None:
        self._factory = factory
        self._target: Any = None

    def __getattr__(self, name: str) -> Any:  # runs only for names the proxy lacks
        if self._target is None:
            print("connecting…")
            self._target = self._factory()
        return getattr(self._target, name)


settings = LazyProxy(lambda: {"region": "eu-west-1"})
settings.get("region")  # → 'eu-west-1'
settings.keys()  # → dict_keys(['region'])
settings == {"region": "eu-west-1"}  # → False  (== compares the proxy itself)
len(settings)  # → TypeError: object of type 'LazyProxy' has no len()

COMMON QUESTIONS

Is a ContextVar the same as threading.local?
Close, and more general. A ContextVar has one value per thread and also per asyncio task, so it works in async views; threading.local only separates threads.
Does isinstance(request, Request) work on the proxy?
Yes. LocalProxy forwards __class__ to the real object, so isinstance sees a Request. type(request) still shows LocalProxy.

WHAT YOU NOW KNOW

  • Flask’s request is a LocalProxy over a ContextVar; each attribute read finds the Request for the current thread or task.
  • Werkzeug declares every special method on LocalProxy, because Python looks those up on the type, never through __getattr__.
  • A lazy proxy of your own is a __getattr__ that builds the subject once; len() and == still need their own methods.
DEEP DIVE: remote proxies, weak references, testing code that reads request, errors, look-alikes

Remote proxies in multiprocessing

A manager’s proxy forwards each method call to an object in the manager’s process: BaseProxy._callmethod sends the method name and its arguments over a connection and waits for the result. The call reads like a local one and costs a round trip between processes.

Source: Lib/multiprocessing/managers.py at v3.15.2

Weak references

weakref.proxy(obj) behaves like obj without keeping it alive. When the garbage collector frees the object, any use of the proxy raises ReferenceError. Caches and parent pointers in trees use it to avoid reference cycles.

Testing code that reads request

Flask’s app.test_request_context() pushes a request context, so request resolves inside a with block in a test, with no server running and no thread to manage.

ERRORS THIS CHAPTER EXPLAINS

RuntimeError: Working outside of request context.
The proxy was used where no request is active: at import, in a thread you started, or in a test.
Use test_request_context() in tests; pass plain values to threads.

LOOKS LIKE

Decorator
A decorator adds behaviour to each call; a proxy decides how the subject is reached.
A proxy usually looks its subject up instead of holding it.
Adapter
An adapter changes the interface; a proxy keeps it.
An adapter’s methods have different names from the adaptee’s.
Lazy initialization (part 1)
Lazy initialization delays creating a value; a lazy proxy also stands in for it.
A cached_property on the owner, against a wrapper object in its place.

Descriptors: run code when an attribute is read

A descriptor is an object stored on a class whose __get__ runs when an instance attribute of that name is read, and whose __set__, if it has one, runs when it is set. Properties, methods and Django model fields all work this way. Django can load a book without its long summary column, and book.summary still returns the text. What runs when you read it?

THE PATTERN

you’ll write it often

Put an object on a class that runs your code whenever an attribute of that name is read or set on an instance.

ROLES
  • owner classholds the descriptor as a class attribute: Book
  • descriptordefines __get__, and __set__ for writes: DeferredAttribute
  • instancewhose attribute access triggers it: book, with values in its __dict__
REACH FOR IT WHEN
many attributes need the same rule on access: validate, convert, compute once or load lazily.
SKIP IT WHEN
one attribute on one class needs it; a property is a descriptor you don’t have to write.
HOW TO SPOT IT
a class attribute set to an object (title = CharField()), classes defining __get__ or __set_name__.
IN PYTHON
Built in: functions, property, classmethod, staticmethod, __slots__ and cached_property are all descriptors.
SEEN IN
  • Django 6.1 · django/db/models/query_utils.pyDeferredAttribute.__get__ queries the database the first time a deferred field is read.
  • CPython 3.15 · Lib/functools.pycached_property computes once and stores the value in the instance’s __dict__.
  • SQLAlchemy 2.1 · sqlalchemy/orm/attributes.pyInstrumentedAttribute records each change so the session knows what to write on flush.
STEP 01 / 05

A PLAIN ATTRIBUTE CAN’T RUN CODE WHEN IT IS READ

Fig 7.1Only the accent column gets both: plain attribute syntax and code that runs on demand.

Book.objects.defer("summary") loads books without one large column, a deferred field. Later a template reads book.summary. A plain attribute can’t notice it was never loaded; a get_summary() method could, but every template, view and test would have to call it.

What the ORM needs is attribute syntax that still runs code: look for the value, and fetch it the first time it is missing.

STEP 02 / 05

THE ATTRIBUTE LIVES ON THE CLASS, THE VALUE ON THE INSTANCE

Fig 7.2The descriptor sits on the class once; each instance keeps its own value in its own dict.

When you read obj.attr, Python looks for attr on the class. If it finds an object with a __get__ method, a descriptor, it calls __get__(obj, type(obj)) and returns whatever that returns.

One descriptor serves every instance, so it keeps per-instance values per instance, usually in the instance’s __dict__ under the same name or a private one.

STEP 03 / 05

DJANGO LOADS A DEFERRED FIELD THE FIRST TIME YOU READ IT

Fig 7.3The accent box is what Book.summary really is: a DeferredAttribute, not a TextField.

When a model class is created, each field puts a descriptor on it with setattr(cls, self.attname, self.descriptor_class(self)). For most fields that is a DeferredAttribute. Loading a row writes the column values straight into the instance’s __dict__.

__get__ checks that dict first. If the field was deferred, the value is missing, and Django 6.1 asks the instance’s fetch mode to load it. The default, FETCH_ONE, runs refresh_from_db(fields=["summary"]): one query for one column.

DJANGO 6.1.3 · DEFERREDATTRIBUTE
# django/db/models/query_utils.py · Django 6.1.3 (trimmed)
# Field.contribute_to_class puts one on the model class per field:
#     setattr(cls, self.attname, self.descriptor_class(self))
class DeferredAttribute:
    """
    A wrapper for a deferred-loading field. When the value is read from this
    object the first time, the query is executed.
    """

    def __init__(self, field):
        self.field = field

    def __get__(self, instance, cls=None):
        if instance is None:  # ← Book.summary, read on the class, returns the descriptor
            return self
        data = instance.__dict__
        field_name = self.field.attname
        if field_name not in data:  # ← only when the row was loaded without this column
            ...
            instance._state.fetch_mode.fetch(self, instance)  # ← one more query for this field
            ...
        return data[field_name]  # ← the value now sits in the instance’s __dict__

Source: django/db/models/query_utils.py at 6.1.3

STEP 04 / 05

THE SECOND READ NEVER REACHES THE DESCRIPTOR

Fig 7.4Calls 1 to 5 hit the database once; calls 6 and 7 never reach the descriptor.

DeferredAttribute defines __get__ but not __set__, which makes it a non-data descriptor. For those, Python checks the instance’s __dict__ first. Once __get__ has stored the value there, later reads find it directly, and the descriptor is never called again.

The standard library’s functools.cached_property uses the same rule, and the sample runs it: the first read prints and stores the total, and the second comes straight from vars(report).

CACHED_PROPERTY: THE SAME TRICK, RUNNABLE
from functools import cached_property


class Report:
    def __init__(self, rows: list[int]) -> None:
        self.rows = rows

    @cached_property
    def total(self) -> int:  # runs once, then the value lives in vars(report)
        print("summing")
        return sum(self.rows)


report = Report([120, 80, 40])
"total" in vars(report)  # → False
report.total  # → 240
"total" in vars(report)  # → True
report.total  # → 240  (read from vars(report): no "summing" this time)
STEP 05 / 05

A DESCRIPTOR WITH __SET__ BEATS THE INSTANCE DICT

Fig 7.5Row 1 is why a validating descriptor sees every read and write; row 2 is why caching works.

To check every assignment, write a data descriptor, one with __set__. Data descriptors come before the instance’s __dict__ in the lookup order, so every read and write goes through them. __set_name__ tells the descriptor the name it was assigned to, so one class serves quantity and price_cents alike.

Store each value on the instance, here under a leading underscore. A value stored on the descriptor itself would be shared by every OrderLine.

A VALIDATING DESCRIPTOR OF YOUR OWN
class Positive:
    """A data descriptor: __set__ checks every assignment."""

    def __set_name__(self, owner: type, name: str) -> None:
        self.attr = "_" + name  # store per instance, never on the descriptor

    def __get__(self, obj: object | None, objtype: type | None = None) -> object:
        return self if obj is None else getattr(obj, self.attr)

    def __set__(self, obj: object, value: int) -> None:
        if value <= 0:
            raise ValueError(f"{self.attr[1:]} must be positive, got {value}")
        setattr(obj, self.attr, value)


class OrderLine:
    quantity = Positive()
    price_cents = Positive()

    def __init__(self, quantity: int, price_cents: int) -> None:
        self.quantity = quantity  # goes through Positive.__set__
        self.price_cents = price_cents


line = OrderLine(3, 1250)
line.quantity  # → 3
vars(line)  # → {'_quantity': 3, '_price_cents': 1250}
line.quantity = 0  # → ValueError: quantity must be positive, got 0

COMMON QUESTIONS

Why does a method get self automatically?
Functions are non-data descriptors. Reading obj.method calls the function’s __get__, which returns a bound method with obj filled in as the first argument.
Should I write a descriptor or a property?
A property for one attribute on one class. A descriptor class when the same rule applies to many attributes or across classes, as Django fields do.

WHAT YOU NOW KNOW

  • Every Django field puts a DeferredAttribute on the model class; reading a deferred field runs one query and stores the value in book.__dict__.
  • A descriptor without __set__ comes after the instance dict, so once the value is stored the descriptor stops running.
  • A descriptor with __set__ comes first and sees every assignment: use one, with __set_name__, to validate many fields.
DEEP DIVE: methods are descriptors, SQLAlchemy’s attributes, fetch modes, history, look-alikes

Methods are descriptors

A function defines __get__. Reading line.total on an instance returns a bound method; reading OrderLine.total on the class returns the plain function. classmethod and staticmethod are descriptors that change what __get__ binds: the class, or nothing.

SQLAlchemy’s InstrumentedAttribute

SQLAlchemy puts a data descriptor on every mapped attribute. Its __set__ records the old and new value in the instance’s state, which is how session.flush() knows which columns changed and writes only those.

Source: lib/sqlalchemy/orm/attributes.py at 2.1.9

Fetch modes in Django 6.1

FETCH_ONE loads a missing field for one instance. FETCH_PEERS loads it for every instance from the same queryset in one query, which removes the one-query-per-row pattern. FETCH_RAISE raises FieldFetchBlocked instead, which makes stray queries fail loudly in tests. django/db/models/fetch_modes.py holds all three.

History

Descriptors arrived with new-style classes in Python 2.2 (PEP 252). __set_name__ came in Python 3.6 (PEP 487), which let field-style descriptors learn their own names without a metaclass.

LOOKS LIKE

Proxy
A proxy stands in for a whole object; a descriptor manages one attribute.
A class attribute with __get__, not a wrapper object.
Lazy initialization (part 1)
cached_property is lazy initialization built on a descriptor.
The value appears in vars(obj) after the first read.
property
property is a ready-made data descriptor for a single attribute.
@property above a def in the class body.

Composite: one interface for a thing and a tree of things

A composite lets a container and the items it holds share one interface, so code that handles one item handles a whole tree. Click builds command-line tools such as tool db migrate --dry-run, where tool and db are groups and only migrate does any work. How does Click run that line when two of its three words name groups?

THE PATTERN

you’ll write it sometimes

Make containers and single items share one interface, so code that handles one item handles a whole tree.

ROLES
  • componentthe shared interface: Command.invoke(ctx)
  • leafa single item that does the work: a Command with a callback
  • compositeholds children and passes the call down: Group
REACH FOR IT WHEN
objects nest to any depth: commands, menus, file trees, syntax trees, widgets.
SKIP IT WHEN
the nesting is always one level deep; a list is enough.
HOW TO SPOT IT
a class holding a list or dict of objects of its own base type, looping or dispatching over them.
IN PYTHON
A class with a children list or dict; Python’s own ast nodes and ElementTree elements are composites.
SEEN IN
  • Click 8.5 · click/core.pyGroup is a Command that holds commands, and invoke() passes the rest of the arguments to the chosen one.
  • CPython 3.15 · Lib/ast.pyevery node holds child nodes, and ast.walk visits a whole tree with one loop.
  • CPython 3.15 · Lib/xml/etree/ElementTree.pyan Element holds child elements, so iter() searches a whole document at any depth.
STEP 01 / 04

CHECKING WHAT YOU HOLD NEEDS ONE LOOP PER LEVEL

Fig 8.1The left design grows a loop per level; the right one is the same three lines at any depth.

A command-line tool with subcommands is a tree: the program, its groups, their commands. Code that treats groups and commands as different kinds of object must test which one it holds at every level, and loop over the children wherever it finds a group.

Add a level, such as tool db replica promote, and that code needs another branch. The depth of the tree leaks into every function that walks it.

STEP 02 / 04

THE CONTAINER IS ONE OF THE THINGS IT CONTAINS

Fig 8.2Both dashed arrows reach the same interface, so a container can hold other containers.

A composite is a container that is also a subclass of the component interface it holds. Leaves implement the method by doing the work; containers implement it by calling the same method on one child or on all of them.

Because a container counts as a component, containers nest, and the client calls one method on the root without knowing how deep the tree goes.

STEP 03 / 04

CLICK’S GROUP IS A COMMAND THAT HOLDS COMMANDS

Fig 8.3Call 2 is the same method as call 1, one level down: the group only picks which child gets it.

In Click, Group is a subclass of Command and keeps its children in self.commands, a dict from name to Command. The decorators @cli.group() and @db.command() add entries to it, so the sample builds a tree three levels deep.

Group.invoke runs its own callback, takes the next word on the command line, finds that child with resolve_command, makes a child context and calls invoke on it. When the child is itself a group, the same code runs again one level down.

CLICK 8.5.1 · GROUP
# src/click/core.py · Click 8.5.1 (trimmed)
class Group(Command):  # ← the composite is a Command, so a group fits wherever a command does
    commands: cabc.MutableMapping[str, Command]  # ← its children: commands or other groups

    def add_command(self, cmd: Command, name: str | None = None) -> None:
        name = name or cmd.name
        ...
        self.commands[name] = cmd

    def get_command(self, ctx: Context, cmd_name: str) -> Command | None:
        return self.commands.get(cmd_name)

    def invoke(self, ctx: Context) -> t.Any:
        ...
        if not self.chain:
            with ctx:
                cmd_name, cmd, args = self.resolve_command(ctx, args)  # ← the child named next
                ctx.invoked_subcommand = cmd_name
                super().invoke(ctx)  # ← the group’s own callback runs first
                sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
                with sub_ctx:
                    return _process_result(sub_ctx.command.invoke(sub_ctx))  # ← the same call, one level down

Source: src/click/core.py at 8.5.1

A COMMAND TREE IN CLICK (CLICK API, NOT RUN HERE)
import click


@click.group()
def cli() -> None:
    """Project tools."""


@cli.group()  # a group inside a group
def db() -> None:
    """Database commands."""


@db.command()
@click.option("--dry-run", is_flag=True)
def migrate(dry_run: bool) -> None:
    click.echo(f"migrating, dry run: {dry_run}")


# $ tool db migrate --dry-run  prints  migrating, dry run: True
STEP 04 / 04

A FOLDER’S SIZE IS THE SUM OF ITS CHILDREN’S

Fig 8.4Each accent folder’s number is the sum of the boxes directly under it.

A composite of your own needs one method on both classes. File.size() returns its bytes; Folder.size() returns the sum of size() over its children, whatever they are. The type hint list[File | Folder] says that children may be either.

Calling size() on the root walks the whole tree: 100,352 bytes for site/, of which 83,968 are in img/. No code anywhere checks whether a child is a folder.

A FOLDER TREE AS A COMPOSITE
from dataclasses import dataclass, field


@dataclass
class File:
    name: str
    size_bytes: int

    def size(self) -> int:
        return self.size_bytes


@dataclass
class Folder:
    name: str
    children: list[File | Folder] = field(default_factory=list)

    def size(self) -> int:  # the same method; a folder asks each child
        return sum(child.size() for child in self.children)


root = Folder("site", [
    File("index.html", 4096),
    Folder("img", [File("logo.svg", 2048), File("hero.png", 81920)]),
    File("app.js", 12288),
])
root.size()  # → 100352
root.children[1].size()  # → 83968

COMMON QUESTIONS

Why not just nest dicts and lists?
You can, but then every function that walks the tree tests for dict, list or value. Classes put the walking code next to the data, once.
How is this different from a decorator?
A decorator wraps exactly one object and adds behaviour (chapter 04). A composite holds any number of children and combines their results.

WHAT YOU NOW KNOW

  • Click’s Group is a subclass of Command, keeps children in self.commands and passes invoke() to the one named next.
  • Because the container is a component, groups nest, and one method runs a command any depth down.
  • Your own composite needs one method on both classes; the container’s version loops over its children.
DEEP DIVE: also in ast and ElementTree, chained groups, typing a tree, look-alikes

Also in ast and ElementTree

Every ast node holds child nodes in its fields, and ast.walk yields every node of a tree with one loop. ElementTree’s Element holds child elements, and element.iter("a") finds every link in a document, however deep it sits.

Chained groups

Click’s Group(chain=True) runs several subcommands from one command line, one after another, and collects their results in a list. The excerpt shows the default branch, one subcommand per group.

Typing a tree

A recursive annotation, children: list[File | Folder], lets a type checker follow the tree. Since Python 3.12, a type statement can name it once: type Node = File | Folder, and the alias may refer to itself.

LOOKS LIKE

Decorator
A decorator wraps one object; a composite holds many.
A single wrapped attribute, against a list or dict of children.
Visitor (part 3)
A visitor adds operations over a composite tree without changing its classes.
Methods named visit_ plus a node class name.
Chain of Responsibility (part 3)
A chain passes a request along a line until one handler takes it.
Each handler holds one next handler, not many children.

Mixins: add one slice of behaviour through inheritance

A mixin is a small class that adds one behaviour to any class that lists it as a base, and calls super() so the class after it still runs. Django protects a page with one change to a class statement: add LoginRequiredMixin before the view’s base. How does a base class you list stop a view from running, without touching the view’s code?

THE PATTERN

you’ll write it often

Package one behaviour as a small base class that hooks a method other classes already have, then hands on with super().

ROLES
  • mixinadds one behaviour, then calls super() to continue: LoginRequiredMixin
  • base classthe real class the mixin sits in front of: View
  • combined classlists mixins first and the base last: ReportView
REACH FOR IT WHEN
unrelated classes need the same slice of behaviour, attached to a method they all share.
SKIP IT WHEN
the behaviour has its own state and lifecycle; hold an object instead of inheriting one.
HOW TO SPOT IT
class names ending in Mixin, several bases with the real one last, methods that call super().same_name().
IN PYTHON
Multiple inheritance plus super(), which follows the method resolution order (MRO).
SEEN IN
  • Django 6.1 · django/contrib/auth/mixins.pyLoginRequiredMixin.dispatch() redirects anonymous users before the view runs.
  • CPython 3.15 · Lib/socketserver.pyThreadingMixIn.process_request() starts a thread; ThreadingTCPServer is TCPServer plus that mixin.
  • CPython 3.15 · Lib/_collections_abc.pySequence adds in, iteration, reversed(), index() and count() to any class with __getitem__ and __len__.
STEP 01 / 05

A LOGIN CHECK COPIED INTO EVERY VIEW FAILS OPEN

Fig 9.1The red row is the risk of copies: one view of three left its export open.

Every page that shows private data needs the same lines before anything else: if the user isn’t signed in, redirect to the login page. Copied into each view’s get method, the check is easy to forget in the next view, and a forgotten copy fails open.

The views share no other code: a report page, an invoice page, an export. They do share one method, dispatch, that every request passes through.

STEP 02 / 05

SUPER() CALLS THE NEXT CLASS IN THE MRO, NOT THE PARENT

Fig 9.2Read the solid arrows left to right: that is the order one method call travels.

Python lines up a class and all its bases in one list, the method resolution order. For class C(Mixin, Base) it is C, Mixin, Base, then their own bases. super() inside Mixin means the class after Mixin in that list, here Base, even though Mixin doesn’t inherit from Base.

So a mixin can define the same method as the base, do its part and call super() to let the base run. Listed first, it runs first.

STEP 03 / 05

DJANGO’S LOGIN CHECK IS ONE DISPATCH() IN FRONT OF THE VIEW’S

Fig 9.3The accent class is a few lines; everything it needs to redirect lives in AccessMixin above it.

Every Django class-based view handles a request through View.dispatch, which picks get, post or another method by the HTTP verb. LoginRequiredMixin defines dispatch too. If the user isn’t authenticated it returns handle_no_permission(), a redirect to the login page; otherwise it calls super().dispatch.

The redirect logic sits in AccessMixin, shared with PermissionRequiredMixin and UserPassesTestMixin. It sends the browser to the LOGIN_URL setting, /accounts/login/ by default, with the current path in a next parameter.

DJANGO 6.1.3 · LOGINREQUIREDMIXIN
# django/contrib/auth/mixins.py · Django 6.1.3 (trimmed)
class AccessMixin:
    redirect_field_name = REDIRECT_FIELD_NAME  # ← "next"

    def handle_no_permission(self):
        if self.raise_exception or self.request.user.is_authenticated:
            raise PermissionDenied(self.get_permission_denied_message())
        path = self.request.build_absolute_uri()
        resolved_login_url = resolve_url(self.get_login_url())  # ← LOGIN_URL, "/accounts/login/"
        ...
        return redirect_to_login(path, resolved_login_url, self.get_redirect_field_name())


class LoginRequiredMixin(AccessMixin):
    """Verify that the current user is authenticated."""

    def dispatch(self, request, *args, **kwargs):  # ← the same method name View has
        if not request.user.is_authenticated:
            return self.handle_no_permission()  # ← stop here: the view never runs
        return super().dispatch(request, *args, **kwargs)  # ← the next class in the MRO

Source: django/contrib/auth/mixins.py at 6.1.3

A PROTECTED VIEW (DJANGO API, NOT RUN HERE)
from django.contrib.auth.mixins import LoginRequiredMixin
from django.views.generic import TemplateView


class ReportView(LoginRequiredMixin, TemplateView):  # the mixin first, the real view last
    template_name = "reports/index.html"


# ReportView.__mro__: ReportView, LoginRequiredMixin, AccessMixin,
#     TemplateView, TemplateResponseMixin, ContextMixin, View, object
STEP 04 / 05

AN ANONYMOUS REQUEST NEVER REACHES GET()

Fig 9.4Calls 1 to 3 stop in LoginRequiredMixin; calls 4 to 7 pass through it to View.

Both requests enter the same way: as_view() builds a function that creates the view object and calls self.dispatch(request). The method resolution order sends that call to LoginRequiredMixin first.

For the anonymous request the mixin answers with a redirect, and View.dispatch never runs, so neither does get. For the signed-in request the mixin calls super().dispatch, and the view runs as if the mixin were not there.

STEP 05 / 05

TWO METHODS IN, FIVE METHODS OUT

Fig 9.5Everything in the accent panel is built from the two methods on the left.

The standard library’s collections.abc.Sequence is a mixin for your own containers. Define __getitem__ and __len__, inherit Sequence, and the class gains in, iteration, reversed(), index() and count(), each written in terms of your two methods.

The same works for mixins of your own: keep each one small, give it no __init__, and have it rely only on methods the classes it mixes into already have.

SEQUENCE AS A MIXIN FOR YOUR OWN CLASS
from collections.abc import Sequence


class Playlist(Sequence[str]):
    def __init__(self, *songs: str) -> None:
        self._songs = list(songs)

    def __getitem__(self, i: int) -> str:  # the two methods you write
        return self._songs[i]

    def __len__(self) -> int:
        return len(self._songs)


p = Playlist("Intro", "Blue", "Outro", "Blue")
"Blue" in p  # → True
p.index("Outro")  # → 2
p.count("Blue")  # → 2
list(reversed(p))  # → ['Blue', 'Outro', 'Blue', 'Intro']
[c.__name__ for c in Playlist.__mro__][:3]  # → ['Playlist', 'Sequence', 'Reversible']

COMMON QUESTIONS

Can a mixin have an __init__?
Avoid it. If it must, accept **kwargs and call super().__init__(**kwargs), so every class in the MRO still receives its arguments.
How is a mixin different from an abstract base class?
An ABC states what subclasses must implement; a mixin supplies finished code. Sequence is both: it requires __getitem__ and __len__ and supplies the rest (chapter 01).

WHAT YOU NOW KNOW

  • LoginRequiredMixin defines dispatch, the method View already has, and calls super().dispatch only for signed-in users.
  • super() follows the MRO, so the mixin must come before the real base in the class statement.
  • collections.abc.Sequence is a mixin: define __getitem__ and __len__, and in, iteration, index and count come with it.
DEEP DIVE: ThreadingMixIn, how the MRO is computed, typing a mixin, look-alikes

Also in socketserver: ThreadingMixIn

ThreadingMixIn overrides one method, process_request, to start a thread per request, and the standard library defines ThreadingTCPServer as ThreadingMixIn plus TCPServer. ForkingMixIn does the same with a child process per request.

Source: Lib/socketserver.py at v3.15.2

How the MRO is computed

Python uses the C3 linearization: every class comes before its bases, and the order of bases in each class statement is kept. A hierarchy that can’t satisfy both raises TypeError when the class is defined. ReportView.__mro__ shows the result.

Typing a mixin

A mixin calls things it doesn’t define, such as self.request. Declare those attributes on the mixin with annotations, or give self a Protocol type in each method, so a type checker can follow it.

LOOKS LIKE

Decorator
A decorator wraps at run time and stays a separate object; a mixin is fixed when the class is defined.
A wrapper object, against an extra base class in the class statement.
Template Method (part 3)
In a template method the base calls hooks a subclass fills in; a mixin adds a step around an existing method.
Methods such as setUp or handle that a base class calls.
Bridge
A bridge holds another object instead of inheriting code from it.
An attribute set to an object of a second class hierarchy.

Bridge: two hierarchies joined by one reference

A bridge splits a class that varies in two directions into two classes joined by a reference, so each side can grow on its own. The logging module has many destinations for a record and many ways to format one, and any destination works with any format. How does logging offer every pair without a class for each?

THE PATTERN

you’ll mostly read it

Split a class that varies in two ways into two classes joined by a reference, so each side changes alone.

ROLES
  • abstractionthe side callers use, holding the reference: Handler, where a record goes
  • implementorthe side it hands part of the job to: Formatter, how a record reads
  • refined classessubclasses on either side: StreamHandler, FileHandler, your JsonFormatter
REACH FOR IT WHEN
a class hierarchy grows in two directions, and subclass names start combining both, like JsonFileHandler.
SKIP IT WHEN
one of the two directions has a single option; one class is enough.
HOW TO SPOT IT
an object that holds an object from another hierarchy and calls it for part of its job: handler.setFormatter(fmt).
IN PYTHON
Two classes and an attribute; often the implementor can be a plain function.
SEEN IN
  • CPython 3.15 · Lib/logging/__init__.pyevery Handler calls self.format(record), which hands the record to the Formatter it holds.
  • SQLAlchemy 2.1 · sqlalchemy/engine/default.pyan Engine speaks SQL through a Dialect, and the Dialect drives whichever database driver you installed.
  • CPython 3.15 · Lib/email/policy.pyan email message holds a policy object that decides line endings and how long headers fold.
STEP 01 / 03

3 DESTINATIONS × 4 FORMATS = 12 CLASSES

Fig 10.1Two bars on one scale: 12 classes for every pair, 7 for a class per side.

If each logging handler formatted its own records, a JSON file handler and a JSON console handler would be separate classes, and so would every other pairing. With 3 destinations and 4 formats that is 12 classes, and each new format adds 3 more.

Split in two, the destinations become 3 classes, the formats 4, and any handler can hold any formatter: 7 classes, and a new format adds one.

STEP 02 / 03

A LOGGING HANDLER HOLDS A FORMATTER IT KNOWS NOTHING ABOUT

Fig 10.2The accent arrow is the whole pattern: one attribute joining the two hierarchies.

A logging.Handler decides where a record goes: a stream, a file, syslog, an HTTP endpoint. It doesn’t decide how the record reads. Handler.format(record) hands that to self.formatter, set with setFormatter, or to a default Formatter when none is set.

Each handler subclass implements only emit. StreamHandler.emit calls self.format and writes the result to its stream, so it works with any formatter, including ones written long after it.

CPYTHON 3.15.2 · LOGGING.HANDLER
# Lib/logging/__init__.py · CPython 3.15.2 (trimmed)
class Handler(Filterer):  # ← the abstraction: where a record goes
    def setFormatter(self, fmt):
        self.formatter = fmt  # ← the bridge: one reference to the other hierarchy

    def format(self, record):
        if self.formatter:
            fmt = self.formatter
        else:
            fmt = _defaultFormatter
        return fmt.format(record)  # ← how the record reads is the formatter’s job


class StreamHandler(Handler):  # ← one destination of many
    def emit(self, record):
        try:
            msg = self.format(record)
            stream = self.stream
            stream.write(msg + self.terminator)  # ← where it goes is the handler’s job
            self.flush()
        except RecursionError:  # See issue 36272
            raise
        except Exception:
            self.handleError(record)

Source: Lib/logging/__init__.py at v3.15.2

STEP 03 / 03

SWAP THE FORMATTER AND THE SAME HANDLER PRINTS A NEW FORMAT

Fig 10.3The handler is the same object in both panels; only the formatter it holds changed.

To add a format, subclass logging.Formatter and override format. JsonFormatter turns a record into one JSON line, and every handler, existing or future, can use it through setFormatter.

The sample swaps the formatter on a live handler between two calls, and the output changes with no change to the handler. In your own code, a bridge is that one attribute: the object you hand the second dimension to.

ONE HANDLER, TWO FORMATTERS
import json
import logging
import sys


class JsonFormatter(logging.Formatter):  # a new "how", usable by every "where"
    def format(self, record: logging.LogRecord) -> str:
        return json.dumps({"level": record.levelname, "msg": record.getMessage()})


handler = logging.StreamHandler(sys.stdout)  # one "where" of many
handler.setFormatter(JsonFormatter())
log = logging.getLogger("checkout")
log.addHandler(handler)
log.propagate = False
log.warning("card declined for order %s", 1042)  # → {"level": "WARNING", "msg": "card declined for order 1042"}
handler.setFormatter(logging.Formatter("%(levelname)s %(message)s"))
log.warning("retrying order %s", 1042)  # → WARNING retrying order 1042

COMMON QUESTIONS

Isn’t a bridge just composition?
Yes, composition with a purpose: two hierarchies that vary independently. The name helps you spot the moment a class hierarchy should be split in two.
How is this different from Strategy?
A strategy is an algorithm a caller picks, often per call (part 3). A bridge’s implementor is part of the object’s design, set once and kept.

WHAT YOU NOW KNOW

  • Each logging Handler holds one Formatter and calls it for every record; handlers choose where, formatters choose how.
  • A class per pair needs 12 classes for 3 destinations and 4 formats; a bridge needs 7.
  • A new format is one Formatter subclass, and every handler can hold it.
DEEP DIVE: SQLAlchemy’s dialects, an implementor that is a function, look-alikes

Also in SQLAlchemy

An Engine runs SQL through a Dialect, the side that knows one database’s SQL and its driver. Code written against the Engine works with PostgreSQL, MySQL or SQLite, and each dialect works with everything built on top of the Engine.

Source: lib/sqlalchemy/engine/default.py at 2.1.9

When the implementor is a function

If the second dimension needs a single method, hold a callable instead of an object. logging does this for filters: since Python 3.2, addFilter accepts a plain function as well as a Filter object.

LOOKS LIKE

Strategy (part 3)
A strategy is chosen by the caller, often per call; a bridge’s implementor is set once.
A function passed as an argument, against an attribute set at construction.
Adapter
An adapter fits an existing class after the fact; a bridge is designed in from the start.
The adapter wraps a class it can’t change.
Mixins
A mixin shares code by inheritance; a bridge holds an object.
An extra base class, against an attribute.

Flyweight: share one object instead of making many

A flyweight is an immutable object shared by everyone who needs that value, handed out by a factory that returns the existing object instead of building a new one. Python does this more often than you might guess. Why is Color("red") is Color.RED true, and when do two equal strings share one object?

THE PATTERN

you’ll mostly read it

Share one immutable object per distinct value, handed out by a factory, instead of creating a copy for every use.

ROLES
  • factoryreturns the shared object for a value, creating it at most once: EnumType.__call__, sys.intern
  • flyweightthe shared, immutable object: Color.RED
  • contextthe per-use data kept outside the flyweight: each row
REACH FOR IT WHEN
millions of objects repeat a few values, and those values never change.
SKIP IT WHEN
values are mutable or mostly unique; sharing then saves nothing and adds a lookup.
HOW TO SPOT IT
a dict from value to instance inside a constructor or factory, is comparisons, sys.intern, __slots__.
IN PYTHON
CPython shares small ints and many strings for you; Enum and functools.cache give you the factory.
SEEN IN
  • CPython 3.15 · Lib/enum.pyColor("red") looks the value up in _value2member_map_ and returns the member made with the class.
  • CPython 3.15 · Include/internal/pycore_runtime_structs.hevery int from -5 to 1,024 is created once at startup and shared.
  • pandas 3.1 · pandas/core/arrays/categorical.pya Categorical stores each distinct value once, plus a small integer code per row.
STEP 01 / 03

100,000 ROWS, 4 DISTINCT VALUES

Fig 11.1The accent bar is 4 objects, too short to see next to 100,000.

Reading a CSV of orders, the csv module creates a new str for every field it parses. A status column that only ever says pending, paid, shipped or refunded still becomes 100,000 separate string objects for 100,000 rows, 4,750,000 bytes in all.

Those strings are equal and immutable, so nothing would change if every row pointed at one object per value. The copies cost memory and buy nothing.

STEP 02 / 03

COLOR("RED") RETURNS THE MEMBER MADE WITH THE CLASS

Fig 11.2Every call lands on the same accent object; the rows hold references to it, not copies.

Calling an enum class with a value never creates an object. EnumType.__call__ sends the value to Enum.__new__, which looks it up in _value2member_map_ and returns the member made when the class was defined. Every Color("red") in the program is the same object, so is works.

One difference from the textbook pattern: enum builds every member up front, while a classic flyweight factory creates each object on its first request. sys.intern works the classic way.

CPYTHON 3.15.2 · ENUM VALUE LOOKUP
# Lib/enum.py · CPython 3.15.2 (trimmed)
class EnumType(type):
    def __call__(cls, value, names=_not_given, *values, module=None, qualname=None, type=None, start=1, boundary=None):
        if cls._member_map_:
            # simple value lookup if members exist
            if names is not _not_given:
                value = (value, names) + values
            return cls.__new__(cls, value)  # ← Color("red") never builds a new object
        ...


class Enum(metaclass=EnumType):
    def __new__(cls, value):
        # all enum instances are actually created during class construction
        # without calling this method; this method is called by the metaclass'
        # __call__ (i.e. Color(3) ), and by pickle
        if type(value) is cls:
            return value
        try:
            return cls._value2member_map_[value]  # ← the pool: each value maps to one shared member
        except KeyError:
            pass
        ...

Source: Lib/enum.py at v3.15.2

STEP 03 / 03

INTERN REPEATED STRINGS WHEN YOU KEEP MILLIONS OF THEM

Fig 11.3The left panel happens without you; reach for the right one when profiling shows repeated values.

For strings, sys.intern(s) is the factory: it returns one shared object per distinct value, creating it the first time. Interning the status column takes it from 100,000 objects to 4, and from 4,750,000 bytes to 190.

CPython also shares some values on its own. In 3.15 every int from -5 to 1,024 is one preallocated object, up from 256 in 3.14, which is why int("1024") is int("1024") is True in the sample and the same test with 1025 is not.

INTERNING A CSV COLUMN, AND THE SMALL-INT CACHE
import csv
import io
import sys

statuses = ("pending", "paid", "shipped", "refunded")
text = "".join(f"{i},{statuses[i % 4]}\n" for i in range(100_000))
orders = list(csv.reader(io.StringIO(text)))  # csv makes a new str for every field
rows = len(orders)
before = len({id(row[1]) for row in orders})
before_bytes = sum(sys.getsizeof(row[1]) for row in orders)
for row in orders:
    row[1] = sys.intern(row[1])  # the factory: one shared object per distinct value
after = len({id(row[1]) for row in orders})
after_bytes = sum(sys.getsizeof(s) for s in {row[1] for row in orders})
print(rows, before, after)  # → 100000 100000 4
print(before_bytes, after_bytes)  # → 4750000 190
int("1024") is int("1024")  # → True  (CPython 3.15 shares every int from -5 to 1024)
int("1025") is int("1025")  # → False

COMMON QUESTIONS

Is a flyweight the same as a singleton?
No. A singleton has one instance in all (part 1); a flyweight factory has one instance per distinct value.
When is interning worth it?
When many long-lived objects hold repeated strings, as in parsed files or columns of data. For a few thousand strings, the saving is too small to measure.

WHAT YOU NOW KNOW

  • Color("red") returns the member made with the class, looked up in _value2member_map_; no enum call builds an object.
  • sys.intern took 100,000 status strings from 100,000 objects to 4.
  • CPython 3.15 shares every int from -5 to 1,024, so is works there by accident; compare values with ==.
DEEP DIVE: pandas categoricals, __slots__, look-alikes

Also in pandas: categoricals

A Categorical column stores each distinct value once, in its categories, and one small integer code per row. A million rows of four statuses become four strings and a million one-byte codes.

Source: pandas/core/arrays/categorical.py at v3.1.1

__slots__ for many small objects

When objects can’t be shared, make each one smaller. A class with __slots__ has no per-instance __dict__, which saves memory across millions of instances. Each slot is a data descriptor on the class (chapter 07).

LOOKS LIKE

Singleton (part 1)
A singleton has one instance in all; a flyweight factory has one per value.
A single module-level instance, against a dict from value to instance.
Object pool (part 1)
A pool lends mutable objects to one user at a time; flyweights are immutable and shared by all.
acquire and release calls, against a lookup that only returns.
Prototype (part 1)
A prototype copies an object to make a new one; a flyweight avoids the copy.
copy.copy or replace() calls.

The patterns in one table

Every pattern in this part, the problem it answers and where to see it at work. Pattern names link to their chapters, and project names to the source lines each chapter quotes.

  1. problem
    code should accept types it has never heard of
    move
    call methods by name; any object that has them fits
    seen in
    json.load, collections.abc
    use when
    a function should work with types you don’t own
    Python
    built in; typing.Protocol to check it
  2. problem
    an object does the job behind the wrong interface
    move
    a wrapper translates calls in and results out
    seen in
    requests, asgiref
    use when
    a library’s interface doesn’t match your code
    Python
    a small class, or a function
  3. problem
    the common case of a subsystem takes many calls
    move
    one function makes the calls in order and cleans up
    seen in
    subprocess.run, requests.get
    use when
    callers repeat the same sequence and cleanup
    Python
    a function
  4. problem
    behaviours combine and subclasses would multiply
    move
    wrap an object in one with the same methods plus one job
    seen in
    io (open), Werkzeug ProxyFix
    use when
    behaviours stack in any order
    Python
    a class that forwards with __getattr__
  5. problem
    the same code runs around many functions
    move
    replace the function with a wrapper that calls it
    seen in
    tenacity, functools.lru_cache
    use when
    retry, cache, time or check many functions
    Python
    @ syntax and functools.wraps
  6. problem
    the real object is per request, lazy or remote
    move
    a stand-in with its interface finds the real one on each use
    seen in
    Flask request, multiprocessing
    use when
    callers shouldn’t care where the object comes from
    Python
    __getattr__ plus the special methods
  7. problem
    an attribute must run code when read or set
    move
    an object on the class with __get__ and __set__
    seen in
    Django fields, cached_property
    use when
    many attributes share one rule
    Python
    property, or a class with __get__
  8. problem
    items nest to any depth
    move
    containers and items share one method
    seen in
    Click Group, ast
    use when
    trees of commands, nodes or widgets
    Python
    a class with a children list
  9. problem
    unrelated classes need the same slice of behaviour
    move
    a small base class hooks a shared method and calls super()
    seen in
    Django auth mixins, socketserver
    use when
    one behaviour attaches to a method many classes have
    Python
    multiple inheritance plus super()
  10. problem
    a hierarchy varies in two directions
    move
    two hierarchies joined by one reference
    seen in
    logging, SQLAlchemy dialects
    use when
    subclass names start combining two ideas
    Python
    an attribute holding the other object
  11. problem
    many objects repeat a few immutable values
    move
    a factory returns one shared object per value
    seen in
    enum, pandas Categorical
    use when
    memory goes to millions of equal objects
    Python
    sys.intern, Enum, functools.cache

Telling them apart

Several of these patterns look alike in code, because most of them are one object holding another. The difference is what the outer object does with the inner one. Start from chapter 01 if the rows below blur together: every wrapper here works because Python checks for methods, not classes.

TELLING THEM APART

Adapter, decorator, proxy
All three wrap one object. An adapter changes the interface, a decorator keeps it and adds behaviour, and a proxy keeps it and controls access.
Compare the wrapper’s method names with the wrapped object’s, then ask what each method adds.
Facade and adapter
A facade invents a smaller interface over many objects; an adapter fits one object to an interface that already exists.
Count the objects behind it: several for a facade, one for an adapter.
Decorator and function decorator
The object pattern wraps an object and its methods; the @ syntax wraps one function.
A class that takes the wrapped object, against a def inside a def.
Composite and decorator
A composite holds many children of its own type; a decorator wraps exactly one.
A list or dict of children, against a single wrapped attribute.
Bridge and Strategy
A bridge’s implementor is part of the design and set once; a strategy is picked by the caller, often per call.
An attribute set at construction, against a function passed as an argument.
Flyweight and Singleton
A singleton has one instance in all; a flyweight factory has one per distinct value.
A single module-level instance, against a dict from value to instance.
Mixin and Template Method
A mixin adds a step around a method the class already has; a template method’s base calls hooks that subclasses fill in.
super() calls in a *Mixin class, against hook methods a base class calls.

THREE THINGS TO KEEP

  • Structural patterns decide which object wraps, holds or stands in for which, so separate pieces work as one.
  • Python checks for methods, not classes, so most of these patterns are a small wrapper or a function.
  • Every wrapper is one more step to read and debug. Add one only when its callers gain something.

The next time a library call surprises you, open its source: the answer is usually a small wrapper.