FACTORIES · SINGLETONS · REGISTRIES · LAZY OBJECTS · INJECTION

Design patterns in Python, part 1:
how objects get made

Creational patterns decide which object gets made, when it gets made, and which code makes that choice. This part covers ten of them, each traced through the source of a project you have probably installed.

Five of these come from the Gang of Four book; the other five are what Python code uses at least as often. Every chapter has the same shape: a pattern card, the problem a real project had, the pattern’s shape, the same shape with that project’s own classes, and the code you’d write yourself. Read them in order or jump to the one you need.

You need to know Python classes, inheritance and imports. Samples run on Python 3.15.2. Project excerpts are trimmed, with our comments marked ←, and link to the source at the release they quote: Django 6.1.3, SQLAlchemy 2.1.9, requests 2.34.2, urllib3 2.8.0, pytest 9.2.0 and NumPy 2.5.5, read on 22 November 2026. Tap any figure to open it full screen and zoom. Dotted terms are tappable. This is part 1 of 3; part 2 covers how objects fit together, and part 3 how they share the work.

Factory functions: more than one way to build an object

An alternative constructor is a classmethod that builds an instance from input the plain constructor doesn’t take. Your data rarely arrives as the exact fields a class stores. A date comes as text from JSON, as a Unix timestamp from a log, or as three integers from a form, and one __init__ can’t sensibly take all three. How does CPython’s date class offer several ways in, and why do they keep working for subclasses?

THE PATTERN

you’ll write it often

Give a class extra named constructors, each taking a different kind of input and returning a finished instance.

ROLES
  • the classkeeps one plain constructor that takes the fields it stores
  • alternative constructora classmethod named after its input; it parses, then calls cls(…)
  • callerpicks the constructor that matches the data it has
REACH FOR IT WHEN
Your data arrives in several shapes, such as text, a timestamp or a dict, and each needs parsing first.
SKIP IT WHEN
There is only one way in. Keyword arguments on the constructor are clearer then.
HOW TO SPOT IT
Classmethods named from_something, from_dict or fromisoformat that end in return cls(…).
IN PYTHON
Built in: @classmethod passes the class it was called on, so cls(…) builds the subclass.
SEEN IN
  • CPython 3.15 · Lib/_pydatetime.pydate.fromisoformat() and fromtimestamp() parse their input, then call cls(year, month, day).
  • pydantic 2.14 · pydantic/main.pyBaseModel.model_validate() builds a model from a dict, and model_validate_json() from JSON text.
  • pandas 3.1 · pandas/core/frame.pyDataFrame.from_dict() and from_records() build a frame from dictionaries or record lists.
  • CPython 3.15 · Lib/ipaddress.pyip_address() is a plain function that returns an IPv4Address or an IPv6Address.
STEP 01 / 05

ONE CONSTRUCTOR TAKES FIELDS, NOT TEXT

Fig 1.1Only the bottom arrow arrives: the two red inputs need parsing before the constructor can take them.

date(year, month, day) builds a date from three integers, which is exactly what a date stores. Hand it the text "2026-11-22" and Python raises a TypeError at once: the constructor never looks inside a string.

You could teach __init__ to accept a string, a number or three integers and guess which one it got. Every call then pays for the guess, and a typo in a date string turns into a strange error from deep inside the constructor.

THE CONSTRUCTOR TAKES THREE INTEGERS
from datetime import date

date(2026, 11, 22)  # → datetime.date(2026, 11, 22)
date("2026-11-22")  # → TypeError
STEP 02 / 05

EVERY ALTERNATIVE CONSTRUCTOR ENDS IN THE SAME CALL

Fig 1.2Both middle boxes end at the same constructor; only the parsing in front of it differs.

An alternative constructor is a classmethod named after its input: from_text, from_number, from_dict. Python passes it the class it was called on as cls. It turns its input into fields and returns cls(…), so the real constructor runs once, with clean values.

The caller picks by name, and the name documents what it has: Thing.from_text(s) says “this is text” where Thing(s) would hide it. A plain function at module level, a factory function, does the same job when no single class should own it.

STEP 03 / 05

DATE HAS ONE __NEW__ AND SIX CLASSMETHODS THAT FEED IT

Fig 1.3The same boxes with the standard library’s names: two of date’s six classmethods.

CPython’s date keeps __new__(cls, year, month, day) as its only real constructor. fromisoformat checks the string, parses it with a private helper and returns cls(*fields). fromtimestamp asks the operating system for the local calendar date and returns cls(y, m, d). The other four, today, fromordinal, fromisocalendar and strptime, have the same shape.

The datetime module you import runs a C implementation. The excerpt is _pydatetime.py, the pure-Python version that the C code mirrors and that Python falls back to when the C module isn’t there. It shows the pattern without the C.

CPYTHON’S DATE: ONE CONSTRUCTOR, CLASSMETHODS IN FRONT OF IT
# Lib/_pydatetime.py · CPython 3.15.2 (trimmed)
class date:
    def __new__(cls, year, month=None, day=None):  # ← the one core constructor
        ...
        year, month, day = _check_date_fields(year, month, day)
        self = object.__new__(cls)
        ...
        return self

    @classmethod
    def fromtimestamp(cls, t):  # ← alternative constructor: a number in
        ...
        y, m, d, hh, mm, ss, weekday, jday, dst = _time.localtime(t)
        return cls(y, m, d)  # ← cls, not date: a subclass gets its own type

    @classmethod
    def fromisoformat(cls, date_string):  # ← alternative constructor: text in
        ...
        try:
            return cls(*_parse_isoformat_date(date_string))
        except Exception:
            raise ValueError(f'Invalid isoformat string: {date_string!r}')
STEP 04 / 05

A SUBCLASS GETS ITS OWN TYPE BACK

Fig 1.4Follow 1 to 5: cls travels with the call, so call 4 builds the subclass.

Deadline.fromisoformat("2026-12-01") runs code written in date, yet it returns a Deadline. The classmethod received cls=Deadline, and cls(2026, 12, 1) built one, so the new days_left method is there on the result.

The C implementation behaves the same way, as the sample shows when you run it. Any subclass of date you write gets all six constructors without writing one of them.

THE SUBCLASS COMES BACK FROM A CONSTRUCTOR IT NEVER WROTE
from datetime import date


class Deadline(date):
    def days_left(self, today: date) -> int:
        return (self - today).days


due = Deadline.fromisoformat("2026-12-01")  # cls is Deadline here
type(due).__name__  # → 'Deadline'
due.days_left(date(2026, 11, 22))  # → 9
STEP 05 / 05

USE A CLASSMETHOD WHEN SUBCLASSES MATTER, A FUNCTION WHEN THEY DON’T

Fig 1.5The highlighted row settles most cases: what should a subclass get back?

In your own code, give the class plain fields, with a dataclass or __init__, and add one classmethod per input format. Annotate the return as Self from typing, so a type checker knows that from_str called on a subclass returns that subclass.

A module-level function is right when the result isn’t tied to one class. ipaddress.ip_address() returns an IPv4Address or an IPv6Address depending on the text, so neither class could own it.

MONEY, WITH TWO WAYS IN
from dataclasses import dataclass
from decimal import Decimal
from typing import Self


@dataclass(frozen=True)
class Money:
    amount: Decimal
    currency: str

    @classmethod
    def from_str(cls, text: str) -> Self:  # "12.50 EUR"
        amount, currency = text.split()
        return cls(Decimal(amount), currency)

    @classmethod
    def from_cents(cls, cents: int, currency: str) -> Self:
        return cls(Decimal(cents) / 100, currency)


Money.from_str("12.50 EUR")  # → Money(amount=Decimal('12.50'), currency='EUR')
Money.from_cents(1250, "EUR") == Money.from_str("12.50 EUR")  # → True

COMMON QUESTIONS

Why @classmethod and not @staticmethod?
A staticmethod gets no class, so it has to name one, and every subclass gets that class back. A classmethod gets the class it was called on.
Should __init__ do the parsing instead?
Keep it to fields. dataclasses.replace() and your tests build objects through it with the fields already in hand, and they have no text to parse.
Is this the Gang of Four’s Factory Method?
No. A factory method is an overridable method that a base class calls for you; an alternative constructor is one you call. Chapter 02 covers the first.

WHAT YOU NOW KNOW

  • date keeps one constructor that takes year, month and day; fromisoformat and fromtimestamp parse their input, then call it.
  • A classmethod receives the class it was called on, so return cls(…) builds a Deadline when called on Deadline.
  • Name each constructor after its input, annotate it with Self, and use a module function when the result’s class varies.
DEEP DIVE: also in pydantic, functions that pick the class, typing it, testing it, look-alikes

Also in pydantic

pydantic’s BaseModel offers model_validate() for a dict or an object, and model_validate_json() for JSON text. Both are classmethods that run the model’s validator and return an instance of cls, so a subclass of your model validates into the subclass. model_construct() is a third way in that skips validation, for data you already trust.

Functions that pick the class

ipaddress.ip_address() tries IPv4Address first, then IPv6Address, and returns whichever accepts the text. pathlib’s Path() builds a PosixPath on Linux and macOS and a WindowsPath on Windows, because Path.__new__ swaps the class before the object exists. When the input decides the class, one of these two shapes is the honest one.

ONE FUNCTION, TWO POSSIBLE CLASSES
import ipaddress

type(ipaddress.ip_address("192.0.2.7")).__name__  # → 'IPv4Address'
type(ipaddress.ip_address("2001:db8::7")).__name__  # → 'IPv6Address'

Typing it

Annotate the return as Self (typing, Python 3.11). Writing the class name instead tells a type checker that a subclass’s from_str() returns the base class, which is wrong at runtime. For a factory function that returns one of several classes, annotate the union or the shared base class.

Testing it

Alternative constructors make test data readable: Money.from_str("12.50 EUR") says what it holds. Test each parser on its own, with bad input too, because the error a caller sees comes from the parser, not from the constructor.

LOOKS LIKE

Factory Method, chapter 02
a base class calls an overridable method for you; you never call it yourself
get_form_class() or make_something() overridden in a subclass, not a from_ classmethod
Builder, chapter 04
collects the parts over several calls, then builds once
a mutable object with add or set methods and a final build() or prepare()
Registry, chapter 07
picks a class by looking a name up in a dict
a dict of classes and a lookup function, not one class with several parsers

Factory Method: let a subclass pick what gets made

A factory method is a method that a base class calls whenever it needs a new object, so a subclass can override it and change which class gets made. Every form page in a Django project runs the same steps: show a blank form, take the POST, validate it, then redirect or show the errors. Only the form class differs from page to page. How does Django write those steps once and still let each view pick its own form?

THE PATTERN

you’ll write it sometimes

A base class runs a fixed procedure and calls one overridable method whenever that procedure needs a new object.

ROLES
  • the procedurethe base class’s code that needs an object, such as post()
  • factory methodreturns the class, or the object, to use; subclasses override it
  • productwhat gets made: many classes, one way of using them
REACH FOR IT WHEN
A framework or base class runs the same steps for everyone, and each user needs a different object inside them.
SKIP IT WHEN
The caller could pass the class or a callable in. In Python, a parameter usually beats a subclass.
HOW TO SPOT IT
Methods named get_something_class(), create_something() or make_something() that subclasses override.
IN PYTHON
Often just a class attribute (form_class = ContactForm); the method is for a choice that needs code.
SEEN IN
  • Django 6.1 · django/views/generic/edit.pyFormMixin.get_form() calls get_form_class(); ModelFormMixin overrides it to build a form from the model.
  • CPython 3.15 · Lib/socketserver.pyBaseServer.finish_request() makes one RequestHandlerClass object per request, whatever class you passed.
  • CPython 3.15 · Lib/logging/__init__.pyThe logger manager makes loggers with loggerClass, which logging.setLoggerClass() replaces.
STEP 01 / 04

TWO FORM VIEWS DIFFER IN ONE LINE

Fig 2.1Three rows are identical; only the highlighted row names a different class.

Written by hand, a contact page and a signup page are the same view twice. Each handles GET and POST, binds the posted data to its form, calls is_valid(), and redirects or renders the errors. The only line that differs is the one that creates the form.

Copy the view for a third form and you copy the whole procedure for the sake of one class name. Fix a bug in one copy, and the other two still have it.

STEP 02 / 04

THE PROCEDURE ASKS A METHOD WHICH CLASS TO MAKE

Fig 2.2The accent box is the only one a subclass writes; everything else stays in the base class.

The base class keeps the whole procedure. Wherever the procedure needs an object, it calls a method instead of naming a class: get_product_class() returns the class, and get_product() builds an instance from it.

A subclass overrides that one method and inherits everything else. The procedure never learns which product it got, so it only uses what every product shares, here is_valid().

STEP 03 / 04

DJANGO’S POST() NEVER NAMES A FORM CLASS

Fig 2.3Follow 1 to 4: post() delegates, and only call 3 knows that the form is a BookForm.

In Django, ProcessFormView.post() calls self.get_form(), which calls self.get_form_class() and instantiates the result with the posted data. FormMixin returns the form_class attribute. ModelFormMixin, behind CreateView and UpdateView, overrides it to build a form class from the model with modelform_factory().

So CreateView with model = Book and fields set gets a working form page with no form class written at all. A view that sets form_class instead gets that class. The procedure in post() is the same in both.

DJANGO’S FORM VIEWS: THE PROCEDURE AND THE OVERRIDABLE CHOICE
# django/views/generic/edit.py · Django 6.1.3 (trimmed)
class FormMixin(ContextMixin):
    def get_form_class(self):  # ← the factory method: which class to make
        return self.form_class

    def get_form(self, form_class=None):  # ← makes the product
        if form_class is None:
            form_class = self.get_form_class()
        return form_class(**self.get_form_kwargs())


class ModelFormMixin(FormMixin, SingleObjectMixin):
    def get_form_class(self):  # ← a subclass changes the choice
        ...
        return model_forms.modelform_factory(model, fields=self.fields)


class ProcessFormView(View):
    def post(self, request, *args, **kwargs):  # ← never names a form class
        form = self.get_form()
        if form.is_valid():
            return self.form_valid(form)
        else:
            return self.form_invalid(form)
STEP 04 / 04

START WITH A CLASS ATTRIBUTE; OVERRIDE A METHOD WHEN THE CHOICE NEEDS CODE

Fig 2.4Read the highlighted row: how much information each option has when it picks a class.

In your own classes, start with a class attribute that holds the class, like encoder_class in the sample. A subclass changes it in one line. Add a factory method such as make_encoder() when the choice needs code: different arguments, or a decision based on self.

When the caller already knows the class, accept it as a parameter instead, since classes are first-class objects in Python. Django’s as_view(form_class=ContactForm) does exactly that: it sets the attribute for one URL without a subclass.

A CLASS ATTRIBUTE FOR THE USUAL CASE, A METHOD FOR THE REST
import json
from typing import Any


class Report:
    encoder_class: type[json.JSONEncoder] = json.JSONEncoder

    def render(self, data: dict[str, Any]) -> str:  # the fixed procedure
        return self.make_encoder().encode(data)

    def make_encoder(self) -> json.JSONEncoder:  # the factory method
        return self.encoder_class(sort_keys=True)


class CompactReport(Report):
    def make_encoder(self) -> json.JSONEncoder:  # only the choice changes
        return self.encoder_class(separators=(",", ":"), sort_keys=True)


Report().render({"b": 1, "a": [1, 2]})  # → '{"a": [1, 2], "b": 1}'
CompactReport().render({"b": 1, "a": [1, 2]})  # → '{"a":[1,2],"b":1}'

COMMON QUESTIONS

Is get_form() or get_form_class() the factory method?
Both are hooks. get_form_class() picks the class and get_form() builds the object, so overriding either one changes the product.
Why not pass the form class to the view’s constructor?
Django does: as_view(form_class=…) sets the attribute for one URL. The method exists for choices that depend on the request or the object.

WHAT YOU NOW KNOW

  • Django’s post() calls get_form(), which asks get_form_class() which class to build, so the procedure is written once.
  • ModelFormMixin overrides get_form_class() to build a form from the model, which is how CreateView works without a form class.
  • In your own code, start with a class attribute, then a factory method, or take the class as a parameter.
DEEP DIVE: also in socketserver, the other get_ hooks in Django, testing it, look-alikes

Also in socketserver

socketserver.TCPServer takes a RequestHandlerClass when you create it and stores it. For every connection, finish_request() calls that class with the socket, so one handler object lives per request. Here the factory is an attribute set by a parameter, the Pythonic end of the range, and the server never subclasses anything to change it.

THE PRODUCT CLASS, PASSED IN
# Lib/socketserver.py · CPython 3.15.2 (trimmed)
class BaseServer:
    def __init__(self, server_address, RequestHandlerClass):
        ...
        self.RequestHandlerClass = RequestHandlerClass  # ← the product class, passed in

    def finish_request(self, request, client_address):
        """Finish one request by instantiating RequestHandlerClass."""
        self.RequestHandlerClass(request, client_address, self)  # ← one object per request

The other get_ hooks in Django

Django’s generic views are full of factory methods with the same two layers: an attribute for the usual case and a get_ method for the rest. ListView has paginator_class and get_paginator(), and every view with a template has template_name and get_template_names(). Look for the pair when you need to change what a view creates.

Testing it

A factory method gives a test a seam: subclass the view in the test and return a stub form, or set the attribute on a test subclass. When the class is a parameter instead, the test passes the stub in and needs no subclass at all.

LOOKS LIKE

Alternative constructors, chapter 01
you call them on the class; nothing overrides them to change a procedure
a from_ classmethod ending in return cls(…)
Abstract Factory, chapter 03
several matching products come from one object instead of one product from one method
a class with several class attributes that each name a related class
Template Method, part 3
any step of the procedure can be a hook; a factory method is a hook whose job is to create
a base method calling several overridable steps such as setup(), handle() and finish()

Abstract Factory: swap a whole family of objects at once

An abstract factory is one object that hands out a matching set of related objects, so swapping that object swaps the whole set at once. SQLAlchemy turns the same Python query into SQL for PostgreSQL, SQLite, MySQL and others. Each database needs its own SQL text, its own column type names, its own rules for quoting names and its own execution details, and all of them must come from the same database. How does SQLAlchemy keep that family together?

THE PATTERN

you’ll mostly read it

One object supplies every related class a job needs, so swapping that object swaps the whole set and the parts always match.

ROLES
  • abstract factoryone object or class that names a whole family of classes
  • productsthe related classes, one per job, that must come from the same family
  • clientasks the factory for a product and never names a product class
REACH FOR IT WHEN
Several objects must agree with each other, such as the same database or platform, and you support more than one family.
SKIP IT WHEN
There is one family, or the parts don’t depend on each other. Separate parameters are simpler.
HOW TO SPOT IT
A class with several attributes that each hold a class, overridden together in each subclass.
IN PYTHON
A class whose attributes are classes. No interfaces needed; the attributes are the factory.
SEEN IN
  • SQLAlchemy 2.1 · dialects/postgresql/base.pyPGDialect names its compiler, DDL compiler, type compiler, quoting and execution-context classes.
  • SQLAlchemy 2.1 · dialects/sqlite/base.pySQLiteDialect sets the same five attributes to its SQLite classes.
  • Django 6.1 · db/backends/postgresql/base.pyDatabaseWrapper names its client, creation, features, introspection and operations classes.
STEP 01 / 03

FIVE CLASSES PER DATABASE MUST COME FROM THE SAME ONE

Fig 3.1Read one column top to bottom: that is one family, and its five parts must match.

To run one query, SQLAlchemy needs a statement compiler that writes the SQL text, a type compiler that names column types, a preparer that quotes names, a DDL compiler for CREATE TABLE and an execution context that runs the statement. Each database has its own version of all five.

Mixing them produces SQL that no database accepts: PostgreSQL’s compiler with SQLite’s type names, say. If every part of the code picked its own class, one missed spot would mix families.

STEP 02 / 03

THE DIALECT IS THE ONLY CLASS THAT NAMES POSTGRESQL’S COMPILER

Fig 3.2The statement on the left talks to one box; only the dialect knows the five classes on the right.

In SQLAlchemy the factory is the dialect. PGDialect sets five class attributes to PostgreSQL’s classes, and SQLiteDialect sets the same five names to SQLite’s. A statement never names a compiler: _compiler() calls dialect.statement_compiler(dialect, self), and the base dialect’s constructor builds its preparer and type compiler the same way.

Switching databases means switching one object, and the five parts change together. A third-party database ships a dialect subclass that overrides some of the five and inherits the rest. Each attribute is a factory method in the sense of chapter 02; the abstract factory is the class that groups them.

SQLALCHEMY’S POSTGRESQL DIALECT: ONE CLASS, FIVE PRODUCT CLASSES
# lib/sqlalchemy/dialects/postgresql/base.py · SQLAlchemy 2.1.9 (trimmed)
class PGDialect(default._BackendsMultiReflection, default.DefaultDialect):
    name = "postgresql"
    ...
    statement_compiler = PGCompiler  # ← SELECT, INSERT, UPDATE as SQL text
    ddl_compiler = PGDDLCompiler  # ← CREATE TABLE and the rest of DDL
    type_compiler_cls = PGTypeCompiler  # ← column types: JSONB, UUID, …
    preparer = PGIdentifierPreparer  # ← quoting of names
    execution_ctx_cls = PGExecutionContext  # ← one statement’s run


# lib/sqlalchemy/sql/elements.py · SQLAlchemy 2.1.9 (trimmed)
class CompilerElement(Visitable):
    def _compiler(self, dialect: Dialect, **kw: Any) -> Compiled:
        return dialect.statement_compiler(dialect, self, **kw)  # ← asks; never names PGCompiler
STEP 03 / 03

IN PYTHON, A FAMILY CAN BE A CLASS OF CLASSES, OR A FROZEN DATACLASS

Fig 3.3The highlighted row is the difference: a family as a subclass, or a family as a value.

SQLAlchemy’s products are big classes with state, so its families are subclasses. When the products are small functions, a family can be a frozen dataclass of callables, as in the sample: MARKDOWN and HTML are two values of one Markup type.

Either way the code that uses the family takes one argument and names no format. Adding a third family is a new subclass or a new value, and no if statement anywhere has to change.

TWO FAMILIES OF ONE MARKUP TYPE
from collections.abc import Callable
from dataclasses import dataclass


@dataclass(frozen=True)
class Markup:  # one family: the parts that must match
    heading: Callable[[str], str]
    item: Callable[[str], str]


MARKDOWN = Markup(heading=lambda t: "# " + t, item=lambda t: "- " + t)
HTML = Markup(heading=lambda t: f"<h1>{t}</h1>", item=lambda t: f"<li>{t}</li>")


def render(kit: Markup, title: str, items: list[str]) -> list[str]:
    return [kit.heading(title)] + [kit.item(i) for i in items]  # names no format


render(MARKDOWN, "Menu", ["tea"])  # → ['# Menu', '- tea']
render(HTML, "Menu", ["tea"])  # → ['<h1>Menu</h1>', '<li>tea</li>']

COMMON QUESTIONS

How is this different from a factory method?
A factory method makes one product. An abstract factory groups several factory methods whose products must match, and you swap them as a set.
How does SQLAlchemy pick the dialect from a URL?
Through a registry keyed by the URL’s name, such as postgresql+psycopg. Chapter 07 follows that lookup.

WHAT YOU NOW KNOW

  • A query needs five matching classes per database: statement, DDL and type compilers, name quoting and an execution context.
  • PGDialect names all five as class attributes, and statements ask the dialect, so one object picks the whole family.
  • In your code a family can be a subclass with class attributes or a frozen dataclass of callables.
DEEP DIVE: also in Django’s database backends, extending a family, look-alikes

Also in Django’s database backends

Each Django database backend has a DatabaseWrapper class that names five companion classes: the client that runs the database shell, the creation code for test databases, the feature flags, the introspection that reads tables back, and the operations that write database-specific SQL snippets. The base wrapper’s constructor builds all five, so one ENGINE setting chooses the whole family.

DJANGO: ONE WRAPPER, FIVE COMPANION CLASSES
# django/db/backends/postgresql/base.py · Django 6.1.3 (trimmed)
class DatabaseWrapper(BaseDatabaseWrapper):
    ...
    client_class = DatabaseClient  # ← runs psql for dbshell
    creation_class = DatabaseCreation  # ← creates the test database
    features_class = DatabaseFeatures  # ← what this database supports
    introspection_class = DatabaseIntrospection  # ← reads tables back
    ops_class = DatabaseOperations  # ← SQL snippets for this database


# django/db/backends/base/base.py · Django 6.1.3 (trimmed)
class BaseDatabaseWrapper:
    def __init__(self, settings_dict, alias=DEFAULT_DB_ALIAS):
        ...
        self.client = self.client_class(self)  # ← the base class builds the family
        self.creation = self.creation_class(self)
        self.features = self.features_class(self)
        self.introspection = self.introspection_class(self)
        self.ops = self.ops_class(self)

Extending a family

Because the family is a class, a new database usually subclasses the closest existing dialect and overrides only the products that differ. A PostgreSQL-compatible database can reuse PostgreSQL’s type compiler and replace its statement compiler. Inheritance gives the rest of the family for free.

LOOKS LIKE

Factory Method, chapter 02
one overridable method makes one product
a single get_something_class() hook, not a set of matching class attributes
Registry, chapter 07
finds which family to use from a name; it doesn’t make the products
a dict from names to factories, consulted once to pick the family
Bridge, part 2
splits one abstraction from its implementation so both can vary; nothing is created as a set
an object holding a reference to an implementation object it calls into

Builder: assemble an object step by step

A builder collects the parts of an object first and assembles the finished object in a separate, final step. An HTTP request has a method, a URL with query parameters, headers, cookies, a body that may be a form, files or JSON, authentication and hooks. Some parts depend on others: the body decides two headers, and a signature may cover the finished URL and body. How does requests turn your keyword arguments into the exact bytes it sends?

THE PATTERN

you’ll write it sometimes

Gather an object’s parts first, then assemble it in a fixed order in one last step.

ROLES
  • the specholds the parts as given, still easy to read and change
  • the builderruns the assembly steps in the order the parts depend on each other
  • productthe finished object, ready to use
REACH FOR IT WHEN
An object has many optional parts, some parts depend on others, or you want to inspect it before it is final.
SKIP IT WHEN
Keyword arguments with defaults cover it. Python’s named arguments remove most reasons other languages have builders.
HOW TO SPOT IT
A spec object plus build(), prepare() or compile(), or chained calls that each return an object.
IN PYTHON
Keyword arguments replace the simple builder; a builder pays off when steps have an order or parts repeat.
SEEN IN
  • requests 2.34 · src/requests/models.pyRequest holds your arguments; prepare() runs seven prepare_ steps, in a fixed order, to build a PreparedRequest.
  • SQLAlchemy 2.1 · sql/selectable.pyselect().where().order_by() adds one part per call; compiling the statement builds the SQL.
  • CPython 3.15 · Lib/argparse.pyadd_argument() calls collect a parser’s spec; parse_args() builds the result from it.
STEP 01 / 04

SOME PARTS OF A REQUEST DEPEND ON OTHERS

Fig 4.1Follow the arrows: auth needs the URL and the body, so it can’t run first.

You pass params= and json=, but the request that goes out needs a URL with the query string joined in, a byte body, and headers that describe that body. Authentication comes after those: a scheme that signs the request has to sign the finished URL and body.

A constructor that did all of this in one go would hide the order inside one long method. A caller also couldn’t look at the result, or add a header, before it is sent.

STEP 02 / 04

COLLECT THE PARTS, THEN BUILD IN ONE PLACE

Fig 4.2The accent box is the only place that knows the order of the steps.

The pattern separates two moments. While parts arrive, they sit in a plain spec object that is easy to read and change. When the object is needed, one builder runs every step in order and produces the finished product.

Callers never touch the steps. They fill in the spec, ask for the product, and get something complete. Fixing the order, or adding a step, happens in one method.

STEP 03 / 04

REQUESTS BUILDS EVERY REQUEST IN SEVEN STEPS, AUTH NEXT TO LAST

Fig 4.3Follow 1 to 4; the seven lines in the accent box run in exactly that order.

requests.get() creates a Request that holds your arguments as given. Session.prepare_request() then makes an empty PreparedRequest, merges in the session’s headers and cookies, and runs the seven prepare_ steps. prepare_body sets Content-Type and the length; prepare_auth runs after it and recomputes the length in case the auth handler changed the body.

The result is the prepared request: the exact bytes that Session.send() puts on the wire. requests lets you build one yourself, change a header, and send it, which is the builder’s second payoff.

REQUESTS: A SPEC, THEN SEVEN STEPS IN A FIXED ORDER
# src/requests/models.py · requests 2.34.2 (trimmed, parameters shortened)
class Request(RequestHooksMixin):  # ← the spec: your arguments, as given
    def prepare(self) -> PreparedRequest:
        p = PreparedRequest()  # ← the product, still empty
        p.prepare(method=self.method, url=self.url, headers=self.headers, files=self.files,
                  data=self.data, json=self.json, params=self.params, auth=self.auth,
                  cookies=self.cookies, hooks=self.hooks)  # ← copies every part across
        return p


class PreparedRequest(RequestEncodingMixin, RequestHooksMixin):
    def prepare(self, method=None, url=None, headers=None, files=None, data=None,
                params=None, auth=None, cookies=None, hooks=None, json=None) -> None:
        self.prepare_method(method)  # ← step 1 of 7, in a fixed order
        self.prepare_url(url, params)  # ← query parameters join the URL
        self.prepare_headers(headers)
        self.prepare_cookies(cookies)
        self.prepare_body(data, files, json)  # ← sets Content-Type and Content-Length
        self.prepare_auth(auth, url)  # ← after the body: a signature may cover it
        self.prepare_hooks(hooks)  # ← last: an auth handler may add a hook
STEP 04 / 04

KEYWORD ARGUMENTS COVER MOST BUILDERS; KEEP ONE FOR ORDERED STEPS

Fig 4.4The highlighted row decides: do the parts arrive together, or over time?

In Java a builder often exists because the language has no named arguments. In Python, a dataclass with defaults gives you Email(to, subject=…) with any subset of fields, already readable, so you don’t need a builder for that.

Write a builder when parts arrive over time or repeat, as with a query’s where() clauses, or when assembly has an order or a final check. Return self from each step so calls chain, and do the assembling in one build().

MOST OF THE TIME: KEYWORD ARGUMENTS
from dataclasses import dataclass, field


@dataclass(frozen=True)
class Email:
    to: str
    subject: str = ""
    cc: tuple[str, ...] = ()
    headers: dict[str, str] = field(default_factory=dict)


Email("ana@example.com", subject="Invoice")  # → Email(to='ana@example.com', subject='Invoice', cc=(), headers={})
WHEN PARTS REPEAT: A SMALL BUILDER
from typing import Self


class Query:
    def __init__(self, table: str) -> None:
        self.table = table
        self.wheres: list[str] = []
        self.params: list[object] = []

    def where(self, clause: str, value: object) -> Self:
        self.wheres.append(clause)
        self.params.append(value)
        return self  # so calls chain

    def build(self) -> tuple[str, list[object]]:  # assembles, in one place
        sql = "SELECT * FROM " + self.table
        if self.wheres:
            sql += " WHERE " + " AND ".join(self.wheres)
        return sql, self.params


q = Query("books").where("author = ?", "Le Guin").where("year < ?", 1980)
q.build()  # → ('SELECT * FROM books WHERE author = ? AND year < ?', ['Le Guin', 1980])

COMMON QUESTIONS

Can I use requests’ builder directly?
Yes. Build a Request, call prepare() or session.prepare_request(), change the PreparedRequest, then session.send() it. The requests docs call these prepared requests.
Is select().where() a builder?
Yes, a generative one: each call returns a new Select with one more part, and compiling it builds the SQL. Chapter 05 shows the copying behind it.

WHAT YOU NOW KNOW

  • A request’s parts depend on each other: params join the URL, the body sets headers, and auth needs both.
  • requests keeps your arguments in a Request, and prepare() builds a PreparedRequest in seven steps, auth next to last.
  • In your code, keyword arguments cover most cases; write a builder when parts repeat or must be assembled in order.
DEEP DIVE: also in SQLAlchemy, fluent and generative builders, testing it, look-alikes

Also in SQLAlchemy

select(User).where(User.name == 'ana') is a builder you use every day. Each where() call adds one criterion to a tuple, and nothing becomes SQL until the statement is compiled for a dialect. The @_generative decorator on where() makes every call return a copy, so a half-built query can be shared and extended in two directions.

SQLALCHEMY: ONE PART PER CALL
# lib/sqlalchemy/sql/selectable.py · SQLAlchemy 2.1.9 (trimmed)
class Select(GenerativeSelect):  # (seven more base classes trimmed)
    @_generative  # ← copy the Select first (chapter 05), then run the method on the copy
    def where(self, *whereclause: _ColumnExpressionArgument[bool]) -> Self:
        assert isinstance(self._where_criteria, tuple)

        for criterion in whereclause:
            where_criteria: ColumnElement[Any] = coercions.expect(
                roles.WhereHavingRole, criterion, apply_propagate_attrs=self
            )
            self._where_criteria += (where_criteria,)  # ← one more part collected
        return self

Fluent and generative builders

A fluent builder changes itself and returns self, like the Query sample. A generative builder returns a changed copy each time, like SQLAlchemy’s Select and Django’s QuerySet. Fluent is cheaper; generative is safe to share, because no caller can change a query that someone else is still holding.

Testing it

A builder’s output can be checked without side effects. requests’ prepared request can be inspected without any network: assert on prep.url, prep.headers and prep.body. Test the builder’s final check too, with parts missing or out of order.

LOOKS LIKE

Alternative constructors, chapter 01
build from one input in one call
a from_ classmethod that takes the whole input at once
Prototype, chapter 05
copies an existing object; a generative builder does this on every call
_clone(), copy.copy() or __replace__ inside a method that returns a changed copy
Template Method, part 3
a fixed order of steps with hooks; prepare() looks like one, but its job is to build
a base method calling overridable steps, without a separate product at the end

Prototype: make a new object by copying one you have

A prototype is an existing object that you copy to make a new one, changing only what differs. In Django you write published = Book.objects.filter(published=True), then build two narrower queries from it: one by author, one sorted by title. Other code still holds published and expects it unchanged. How does a QuerySet let you branch a query without touching the original?

THE PATTERN

you’ll write it sometimes

Make new objects by copying one that is already set up, then changing the copy.

ROLES
  • prototypethe object you have, already configured
  • clone methodmakes a new object of the same class with the same state
  • clientcalls a method and gets a changed copy; the original never changes
REACH FOR IT WHEN
Objects take setup to configure, and you need many variations that share most of it, or others hold the original.
SKIP IT WHEN
The object is cheap to build from scratch, or nothing else holds it. Then changing it in place is fine.
HOW TO SPOT IT
Methods that return self.__class__(…) or copy.copy(self) and change the copy; __copy__ or __replace__.
IN PYTHON
The copy module does the copying; copy.replace() (3.13) calls your class’s __replace__.
SEEN IN
  • Django 6.1 · django/db/models/query.pyEvery filter() calls _chain(), which clones the QuerySet, so the original stays as it was.
  • SQLAlchemy 2.1 · sql/base.py@_generative copies a Select with _generate() before where() or order_by() changes it.
  • CPython 3.15 · Lib/copy.pycopy.replace(obj, **changes) asks the object’s __replace__ for a changed copy.
STEP 01 / 04

CHANGING A SHARED QUERY CHANGES IT FOR EVERYONE

Fig 5.1Both names point at one red object, so the filter added for by_author lands on published as well.

If filter() changed the query in place, by_author = published.filter(author=3) would change published too: both names point at the same object. The home page would show only one author’s books, and nothing would raise an error.

Building every query from scratch avoids that, but repeats the shared part each time. You want the shared part once and every narrower query as a separate object.

STEP 02 / 04

CLONE FIRST, THEN CHANGE THE CLONE

Fig 5.2The accent box makes a copy before anything changes, and only the copy goes back.

The prototype’s methods never change the object they’re called on. Each one first makes a copy of the same class with the same state, changes the copy, and returns it.

The caller sees a method that returns a new object, so the original can be held, shared and extended in several directions at once.

STEP 03 / 04

DJANGO CLONES THE QUERYSET ON EVERY FILTER()

Fig 5.3Follow 1 to 3: the WHERE clause is added to the copy on the right, never to qs.

In Django, filter() and exclude() both go through _filter_or_exclude(), which starts with clone = self._chain(). _clone() calls self.__class__(…), so a custom QuerySet subclass survives, and passes self.query.chain(), a copy of the SQL description. Only then is the new condition added, to the clone.

The copy is deliberate and shallow. It copies the lists it will change, like _prefetch_related_lookups[:], and leaves out the cached results, since the new query hasn’t run. Its docstring calls it “a lightweight alternative to deepcopy()”.

DJANGO: COPY FIRST, THEN CHANGE THE COPY
# django/db/models/query.py · Django 6.1.3 (trimmed)
class QuerySet(AltersData):
    def _filter_or_exclude(self, negate, args, kwargs):  # ← behind filter() and exclude()
        ...
        clone = self._chain()  # ← copy first
        if self._defer_next_filter:
            ...
        else:
            clone._filter_or_exclude_inplace(negate, args, kwargs)  # ← then change the copy
        return clone

    def _chain(self):  # ← returns self._clone(), unless cloning is switched off
        ...

    def _clone(self):  # ← the prototype’s copy method
        c = self.__class__(  # ← same class, so your QuerySet subclass survives
            model=self.model,
            query=self.query.chain(),  # ← the SQL description is copied too
            using=self._db,
            hints=self._hints,
        )
        c._prefetch_related_lookups = self._prefetch_related_lookups[:]  # ← a list: copy it
        ...
        return c
STEP 04 / 04

COPY.REPLACE() MAKES THE CHANGED COPY; WATCH WHAT IT SHARES

Fig 5.4Two objects, one list: a shallow copy shares every mutable attribute it didn’t replace.

copy.replace(obj, **changes), new in Python 3.13, returns a copy with some fields replaced. Dataclasses and named tuples support it out of the box, and your own class can by defining __replace__. It is the standard library’s prototype call.

The copy is shallow: attributes you didn’t replace point at the same objects. Appending to base.tags shows up in page2.tags. Use tuples for fields that copies share, or copy the mutable ones yourself, as Django does with its lists.

A CHANGED COPY THAT STILL SHARES A LIST
import copy
from dataclasses import dataclass, field


@dataclass(frozen=True)
class Search:
    text: str
    page: int = 1
    tags: list[str] = field(default_factory=list)


base = Search("python", tags=["books"])
page2 = copy.replace(base, page=2)  # a changed copy; base is untouched
page2  # → Search(text='python', page=2, tags=['books'])
base.page  # → 1
page2.tags is base.tags  # → True
base.tags.append("sale")  # the list is shared, so the copy sees it
page2.tags  # → ['books', 'sale']

COMMON QUESTIONS

Doesn’t cloning on every call cost a lot?
It copies a description of the query, not rows. No SQL runs until you iterate, and the clone is shallow on purpose.
Why not copy.deepcopy()?
deepcopy follows every reference, including model instances and cached results. Django’s own _clone() docstring calls itself the lightweight alternative.

WHAT YOU NOW KNOW

  • Changing a shared query in place changes it for every name that points at it.
  • Django’s filter() calls _chain() and _clone() first, then adds the condition to the copy, so qs never changes.
  • copy.replace() makes a changed copy of your own objects; it is shallow, so share only immutable fields.
DEEP DIVE: also in SQLAlchemy, shallow and deep copies, typing it, look-alikes

Also in SQLAlchemy

SQLAlchemy’s Select, Insert and Update objects clone themselves the same way. The _generative decorator wraps where(), order_by() and friends: it calls _generate(), which makes a new object with cls.__new__ and a copy of the old __dict__, runs the method on the copy and returns it. The method itself is written as if it changed self.

SQLALCHEMY: THE COPY HAPPENS IN A DECORATOR
# lib/sqlalchemy/sql/base.py · SQLAlchemy 2.1.9 (trimmed)
def _generative(fn: _Fn) -> _Fn:
    @util.decorator
    def _generative(fn: _Fn, self: _SelfGenerativeType, *args: Any, **kw: Any) -> _SelfGenerativeType:
        self = self._generate()  # ← the copy
        x = fn(self, *args, **kw)  # ← where(), order_by()… change the copy
        assert x is self, "generative methods must return self"
        return self
    ...


class Generative(HasMemoized):
    def _generate(self) -> Self:
        skip = self._memoized_keys
        cls = self.__class__
        s = cls.__new__(cls)  # ← a new object, no __init__
        if skip:
            ...
        else:
            s.__dict__ = self.__dict__.copy()  # ← a shallow copy of every attribute
        return s

Shallow and deep copies

copy.copy() and copy.replace() are shallow: the new object’s attributes point at the same objects as the old one’s. copy.deepcopy() copies recursively and keeps a memo dict so shared and circular references stay shared within the copy. A class can control both through __copy__ and __deepcopy__.

Typing it

Annotate clone methods and __replace__ as returning Self, so a subclass’s copy is typed as the subclass. Frozen dataclasses make the pattern safer still: a copy can’t be changed by accident, so the only way to vary it is another replace().

LOOKS LIKE

Builder, chapter 04
assembles a new object from parts; a generative builder clones on every step
build() or prepare() at the end, rather than a copy method on the object itself
Memento, part 3
copies state to restore later, not to make a variation
getstate() and setstate(), or a snapshot object handed back to the same object
Flyweight, part 2
shares one object among many users instead of copying it
a cache that returns the same instance for the same key

Singleton: one shared instance, and the module that already is one

A singleton is an object that exists exactly once and is shared by everyone who asks for it. You call random.choice() in one module and random.seed(42) in a test, and the seed controls the choice, so both must reach the same generator. Yet nobody wrote Random.instance(). Where does that one generator live, how does Python make it a single object without a singleton class, and when does sharing it hurt?

THE PATTERN

you’ll write it often

Keep exactly one instance of something and give every caller that same instance.

ROLES
  • the instancethe one shared object, made once
  • access pointhow callers reach it: a module attribute, functions bound to it, or a getter
  • callersevery module that imports it, all sharing one state
REACH FOR IT WHEN
Exactly one should exist per process: a default generator, a registry, the settings, a pool of connections.
SKIP IT WHEN
You only want to avoid passing an argument. Pass the object in (chapter 10) and tests stay independent.
HOW TO SPOT IT
A module-level instance such as _inst = Random(), an instance() classmethod, or __new__ returning a cached object.
IN PYTHON
Built in: a module runs once and is cached, so a module-level object already exists once per process.
SEEN IN
  • CPython 3.15 · Lib/random.pyMakes one Random() at import and exports its bound methods as random.random, seed and choice.
  • CPython 3.15 · Lib/logging/__init__.pyOne root logger per process; getLogger(name) returns the same Logger for the same name.
  • CPython 3.15 · Lib/importlib/_bootstrap.pyEvery import checks sys.modules first, which is why a module’s code runs only once.
STEP 01 / 05

TWO GENERATORS MEAN A SEED CONTROLS ONLY ONE

Fig 6.1The red generator was never seeded, so the test can’t make shop.py’s numbers repeat.

If shop.py made its own Random(), a test that seeds its own generator would control nothing: the code under test draws from a different object. Every module that wants reproducible numbers would need a way to reach the test’s generator.

Some things should exist once per process, like the default random generator, the root logger or the settings. The pattern answers two questions: where that one object lives, and how every caller reaches it.

STEP 02 / 05

ONE INSTANCE, ONE WAY TO REACH IT

Fig 6.2Two callers, one access point, one accent object at the end.

A singleton has two parts: the one instance and an access point that always returns it. The classic version hides the constructor and adds a get_instance() method that creates the object on first use.

Python already has an object that is created once and shared by every importer: the module. So in Python the access point is usually just a module attribute.

STEP 03 / 05

RANDOM’S FUNCTIONS ARE METHODS OF ONE HIDDEN INSTANCE

Fig 6.3The middle box is the trick: module functions that are really methods of the accent object.

The last lines of random.py create _inst = Random() and then assign seed = _inst.seed, choice = _inst.choice and so on for 24 functions. Each one is a bound method: a method that already carries its self, here the one hidden generator.

The comment above those lines says it plainly: the functions share state across all uses, in your code and in the libraries you call. That is the singleton, with no class written for it.

CPYTHON’S RANDOM: ONE INSTANCE, 24 BOUND METHODS
# Lib/random.py · CPython 3.15.2 (trimmed)
# Create one instance, seeded from current time, and export its methods
# as module-level functions.  The functions share state across all uses
# (both in the user's code and in the Python libraries), but that's fine
# for most programs and is easier for the casual user than making them
# instantiate their own Random() instance.

_inst = Random()  # ← the one instance, made when the module first runs
seed = _inst.seed  # ← each module function is a bound method of it
random = _inst.random
randint = _inst.randint
choice = _inst.choice
...
shuffle = _inst.shuffle
getstate = _inst.getstate
setstate = _inst.setstate
STEP 04 / 05

IMPORT RUNS A MODULE ONCE, SO EVERY FILE GETS THE SAME OBJECT

Fig 6.4Follow 1 to 7: call 3 finds the module already loaded, so calls 5 and 6 reach one object.

The first import random runs the module and stores it in sys.modules. Every later import, in any file, finds it there and gets the same module object, so the module-level code, and Random(), runs exactly once.

The sample checks it: random.choice and random.random share one __self__, and seeding twice with 42 repeats the sequence. The import lock also makes that first run safe when several threads import at once.

ONE MODULE, ONE GENERATOR, SHARED BY EVERY CALLER
import random
import sys

random.choice.__self__ is random.random.__self__  # → True
type(random.random.__self__).__name__  # → 'Random'
sys.modules["random"] is random  # → True

random.seed(42)
first = random.random()
random.seed(42)  # any code that seeds the module resets everyone's sequence
random.random() == first  # → True
STEP 05 / 05

PREFER A MODULE-LEVEL OBJECT OR A CACHED GETTER TO A SINGLETON CLASS

Fig 6.5The highlighted row is what a test needs: an easy way to start over.

For your own shared object, create it at module level and import it. When creating it is slow, or needs configuration that isn’t ready at import time, wrap it in a function decorated with functools.cache: the first call builds it, and later calls return the same object.

A class whose __new__ returns a cached instance works too, but subclassing it, resetting it in tests and reading it later are all harder. Before you share an object at all, ask whether passing it in would be simpler.

A CACHED GETTER, AND A GENERATOR NOBODY ELSE CAN SEED
import functools
import random
from dataclasses import dataclass


@dataclass
class Settings:
    debug: bool = False


@functools.cache  # the first call builds it; later calls return that object
def get_settings() -> Settings:
    return Settings(debug=True)


get_settings() is get_settings()  # → True
get_settings.cache_clear()  # a test can start over
rng = random.Random(7)  # a private generator: seeding it touches nobody else
rng.random() == random.Random(7).random()  # → True

COMMON QUESTIONS

Is a module really a singleton?
Once per name per interpreter. A file run as a script is __main__, and importing it again by its own name makes a second module object.
What about getLogger(name)?
A registry of singletons: one Logger per name, kept in a dict, and the same object every time you ask for that name.

WHAT YOU NOW KNOW

  • random.py creates one Random() at import and exports its bound methods, so random.seed() and random.choice() share it.
  • Python runs a module once and keeps it in sys.modules, so a module-level object is already a singleton.
  • Prefer a module attribute or a functools.cache getter, and pass the object in when tests need another one.
DEEP DIVE: also in logging, shared-state variants, testing it, what sharing costs, look-alikes

Also in logging

logging keeps one root logger, created when the module loads, and a manager with a dict of named loggers. getLogger("app.db") creates that logger the first time and returns the same object after that, which is why configuring it in one module affects every other module that asks for the same name.

LOGGING: ONE ROOT, ONE LOGGER PER NAME
# Lib/logging/__init__.py · CPython 3.15.2 (trimmed)
root = RootLogger(WARNING)  # ← one root logger per process
Logger.root = root
Logger.manager = Manager(Logger.root)  # ← keeps a dict: name → Logger


def getLogger(name=None):
    """
    Return a logger with the specified name, creating it if necessary.

    If no name is specified, return the root logger.
    """
    if not name or isinstance(name, str) and name == root.name:
        return root
    return Logger.manager.getLogger(name)  # ← the same name returns the same object

Shared-state variants

The Borg variant lets you create many instances that share one __dict__, so they behave as one object while each has its own identity. A multiton keeps one instance per key, as getLogger(name) does. Both are rare in modern code; a module attribute or a cached function covers most needs.

Testing it

A shared object carries state from one test into the next. pytest’s monkeypatch can swap a module attribute for one test and put it back afterwards, and a functools.cache getter resets with cache_clear(). Code that receives the object as a parameter needs neither.

What sharing costs

Any function that reaches a module-level object depends on it without saying so in its signature. That hidden coupling is the usual complaint about singletons. It is fine for a logger or a default generator, and costly for a database connection that tests want to replace.

LOOKS LIKE

Flyweight, part 2
many shared instances, one per distinct value, to save memory
a cache keyed by value, such as sys.intern() or small-integer reuse
Registry, chapter 07
one shared dict that maps names to many objects or classes
register() and lookup functions over a module-level dict
Dependency injection, chapter 10
the object is passed in, so each caller could get a different one
the dependency is a parameter, not a module-level name

Registry: find the right class from a name

A registry is a mapping from names to classes or factories, filled in by code that registers itself and read by code that needs to create something by name. create_engine("postgresql+psycopg://…") gets a string, and somewhere that string must become a dialect class, including dialects SQLAlchemy has never heard of, such as DuckDB’s, installed as a separate package. How does a name in a URL become a class that SQLAlchemy didn’t ship?

THE PATTERN

you’ll write it often

Map names to classes or factories, so code can create the right object from a string, and new entries need no edit to the lookup.

ROLES
  • registrya dict from name to class, factory or loader
  • registrantscode that adds entries: a register() call, a decorator, an entry point
  • lookupturns a name into a class, and fails clearly for an unknown one
REACH FOR IT WHEN
Names come from configuration, URLs, files or the command line, and plugins must be able to add new kinds.
SKIP IT WHEN
The set is small and fixed. A dict literal or a match statement is easier to read.
HOW TO SPOT IT
A module-level dict of classes, register() or a registering decorator, __init_subclass__ adding cls to a dict, or entry_points(group=…).
IN PYTHON
A dict plus a decorator or __init_subclass__; entry points extend it across installed packages.
SEEN IN
  • SQLAlchemy 2.1 · util/langhelpers.pyPluginLoader maps dialect names to loaders and searches built-ins, then entry points.
  • Flask 3.1 · src/flask/sansio/app.py@app.route stores each view function in app.view_functions under its endpoint name.
  • CPython 3.15 · Lib/encodings/__init__.pycodecs.register() adds a search function; this one finds a codec module by its name.
  • Django 6.1 · django/apps/registry.pyEvery model class registers itself with the app registry when its class statement runs.
STEP 01 / 05

AN IF CHAIN CAN’T NAME A CLASS A PLUGIN ADDS

Fig 7.1The red arrow has no branch to land on, and the faint package on the right is never reached.

The obvious lookup is an if chain or a dict literal inside the library: if the name is "postgresql", use this class; if it is "sqlite", use that one. It works for exactly the names the library’s authors wrote down.

A database that ships its own dialect as a separate package can’t edit that chain. Neither can your application. Every new kind needs a release of the library.

STEP 02 / 05

REGISTER BY NAME, LOOK UP BY NAME

Fig 7.2Arrows into the accent box come from both sides: registrants on top, lookups below.

A registry splits the lookup in two. Registration adds a name and a class, or a loader that imports the class when needed. Lookup reads the name and returns what was registered, or raises an error that lists the known names.

Neither side knows the other. A plugin registers without touching the library, and the library looks up names it has never seen, as long as somebody registered them first.

STEP 03 / 05

SQLALCHEMY LOOKS IN THREE PLACES, IN ORDER

Fig 7.3Same layout, SQLAlchemy’s names: the accent box is PluginLoader, and the top two boxes are its sources.

SQLAlchemy’s registry is PluginLoader("sqlalchemy.dialects", auto_fn=_auto_fn). Its load(name) tries three places in order: the impls dict, a function that imports SQLAlchemy’s own dialect modules, and then the installed packages’ entry points in that group. Whatever it finds is stored in impls, so the search runs once per name.

There is also register(name, modulepath, objname) for adding a dialect by hand, in a test or an application. The lookup raises NoSuchModuleError with the group and the name when all three places miss.

SQLALCHEMY’S DIALECT REGISTRY: A DICT AND THREE SOURCES
# lib/sqlalchemy/util/langhelpers.py · SQLAlchemy 2.1.9 (trimmed)
class PluginLoader:
    def __init__(self, group: str, auto_fn: Optional[Callable[..., Any]] = None):
        self.group = group  # ← the entry-point group: "sqlalchemy.dialects"
        self.impls: Dict[str, Any] = {}  # ← the registry: name → loader
        self.auto_fn = auto_fn

    def load(self, name: str) -> Any:
        if name in self.impls:  # ← 1: a name it has seen or been told about
            return self.impls[name]()

        if self.auto_fn:  # ← 2: a dialect that ships inside SQLAlchemy
            loader = self.auto_fn(name)
            if loader:
                self.impls[name] = loader
                return loader()

        for impl in compat.importlib_metadata_get(self.group):  # ← 3: installed packages
            if impl.name == name:
                self.impls[name] = impl.load
                return impl.load()

        raise exc.NoSuchModuleError("Can't load plugin: %s:%s" % (self.group, name))

    def register(self, name: str, modulepath: str, objname: str) -> None:
        ...  # ← adds a loader to impls by hand
STEP 04 / 05

A NAME SQLALCHEMY DOESN’T KNOW FALLS THROUGH TO ENTRY POINTS

Fig 7.4Follow 1 to 8: the built-in search fails at 4, and the entry point answers at 6.

The URL turns postgresql+psycopg into the key postgresql.psycopg, and a bare duckdb stays as it is. For duckdb, the built-in search tries to import sqlalchemy.dialects.duckdb, fails, and returns None.

The duckdb_engine package declares an entry point named duckdb in the sqlalchemy.dialects group, pointing at duckdb_engine:Dialect. Installing the package is the registration: SQLAlchemy finds it in the installed metadata without importing anything first.

THE URL’S DRIVER NAME BECOMES THE REGISTRY KEY
# lib/sqlalchemy/engine/url.py · SQLAlchemy 2.1.9 (trimmed)
class URL(NamedTuple):
    def _get_entrypoint(self) -> Type[Dialect]:
        if "+" not in self.drivername:
            name = self.drivername  # ← "duckdb"
        else:
            name = self.drivername.replace("+", ".")  # ← "postgresql+psycopg" → "postgresql.psycopg"
        cls = registry.load(name)  # ← registry = PluginLoader("sqlalchemy.dialects", …)
        ...
STEP 05 / 05

__INIT_SUBCLASS__ REGISTERS A CLASS THE MOMENT IT IS DEFINED

Fig 7.5The highlighted row decides between them: does the registry need classes from other packages?

Inside one codebase, __init_subclass__ is the shortest registry. The base class defines it, and Python calls it every time a subclass is defined, with any keywords from the class line, so class CsvExporter(Exporter, name="csv") registers itself.

A decorator does the same for functions or classes that don’t share a base, as Flask’s @app.route does. Both register only when their module is imported. For plugins in other packages, declare an entry point and read it with importlib.metadata.entry_points(group=…).

A REGISTRY FILLED BY THE CLASS STATEMENT ITSELF
from typing import ClassVar


class Exporter:
    registry: ClassVar[dict[str, type["Exporter"]]] = {}

    def __init_subclass__(cls, /, name: str, **kwargs: object) -> None:
        super().__init_subclass__(**kwargs)
        cls.registry[name] = cls  # runs when the subclass is defined

    @classmethod
    def for_name(cls, name: str) -> "Exporter":
        try:
            return cls.registry[name]()
        except KeyError:
            raise ValueError(f"no exporter {name!r}; known: {sorted(cls.registry)}") from None


class CsvExporter(Exporter, name="csv"): ...
class JsonExporter(Exporter, name="json"): ...


sorted(Exporter.registry)  # → ['csv', 'json']
type(Exporter.for_name("csv")).__name__  # → 'CsvExporter'
Exporter.for_name("xml")  # → ValueError

COMMON QUESTIONS

Why does my registered class not show up?
Its module was never imported, so the class statement never ran. Import it somewhere at startup, or register it through an entry point.
Can two plugins claim the same name?
Flask raises an AssertionError when an endpoint name is reused for another function. SQLAlchemy takes the first matching entry point. Decide which your registry does.

WHAT YOU NOW KNOW

  • An if chain inside the library can only name the dialects its authors knew about.
  • PluginLoader.load() checks its dict, then SQLAlchemy’s own dialects, then the sqlalchemy.dialects entry points, and caches what it finds.
  • Use __init_subclass__ or a decorator inside one codebase, and entry points for plugins from other packages.
DEEP DIVE: also in Flask, codecs and encodings, testing a registry, look-alikes

Also in Flask

@app.route("/books") looks like routing, but its decorator only registers. It calls add_url_rule(), which adds a URL rule to the map and stores your function in app.view_functions under its endpoint name. When a request arrives, Flask matches the URL to an endpoint and looks the function up in that dict.

FLASK: THE ROUTE DECORATOR IS A REGISTRATION
# src/flask/sansio/scaffold.py · Flask 3.1.3 (trimmed)
class Scaffold:
    def route(self, rule: str, **options: t.Any) -> t.Callable[[T_route], T_route]:
        def decorator(f: T_route) -> T_route:
            endpoint = options.pop("endpoint", None)
            self.add_url_rule(rule, endpoint, f, **options)  # ← registers the view
            return f  # ← and hands the function back unchanged

        return decorator


# src/flask/sansio/app.py · Flask 3.1.3 (trimmed)
class App(Scaffold):
    def add_url_rule(self, rule, endpoint=None, view_func=None, **options):
        ...
        if view_func is not None:
            old_func = self.view_functions.get(endpoint)
            if old_func is not None and old_func != view_func:
                raise AssertionError(
                    "View function mapping is overwriting an existing"
                    f" endpoint function: {endpoint}"
                )
            self.view_functions[endpoint] = view_func  # ← endpoint name → function

Codecs and encodings

codecs.lookup("utf-8") asks every registered search function in turn. The one the encodings package registers at import normalises the name, imports the matching module and caches the result, so an encoding name in a file header reaches the right codec. codecs.register() lets your code add its own search function.

Testing a registry

A registry is global state, so a test that registers something leaves it behind. Clear or copy the dict around the test, or use the registry’s own tools: SQLAlchemy’s PluginLoader has register(), deregister() and clear() for exactly this.

LOOKS LIKE

Plugin hooks, part 3
calls every registered implementation at a named point instead of picking one by name
hook.something(…) that returns a list of results, one per plugin
Factory functions, chapter 01
one function decides the class from its input in code
if or match on the input inside the function, with no dict to register into
Singleton, chapter 06
the registry itself is usually one shared object; the pattern is the name lookup
register() and lookup functions, not just a module-level instance

Lazy initialization: build it the first time someone asks

Lazy initialization delays creating an object until the first time code asks for it, then keeps the result. import numpy happens at the top of countless scripts, and most of them never touch numpy.testing, which pulls in unittest and more. Yet np.testing works without an import of its own. How does NumPy make a submodule available without importing it up front?

THE PATTERN

you’ll write it often

Create an expensive object on first use, store it, and hand back the stored one after that.

ROLES
  • holderthe object or module the value belongs to
  • triggerthe hook that runs when the value is missing: __getattr__, a property, a descriptor
  • cachewhere the value is stored, so the trigger never runs again
REACH FOR IT WHEN
Creating it is slow or needs resources, and many runs never use it: large imports, connections, parsed files.
SKIP IT WHEN
It is cheap, or every run needs it. Laziness moves the cost and the errors to a less predictable moment.
HOW TO SPOT IT
A module-level def __getattr__(name), @cached_property, a property that checks for None first, or a LazyObject.
IN PYTHON
Built in: functools.cached_property for attributes, module __getattr__ (PEP 562), lazy import in 3.15.
SEEN IN
  • NumPy 2.5 · numpy/__init__.pyA module-level __getattr__ imports submodules such as numpy.testing the first time you read them.
  • CPython 3.15 · Lib/functools.pycached_property runs the method once and stores the value in the instance’s __dict__.
  • Django 6.1 · django/conf/__init__.pysettings imports your settings module on the first attribute read, not at import.
STEP 01 / 05

IMPORTING EVERYTHING UP FRONT MAKES EVERY START PAY

Fig 8.1The highlighted row is the saving: work at import time that most programs never needed.

A package that imports all its submodules in __init__.py makes every program pay for all of them, at every start. For a test helper like numpy.testing, that is time and memory spent by thousands of scripts that never call it.

The alternative is to build the object the first time someone asks for it. The last row is the price: if the import fails, it fails later, at the line that first reads the attribute.

STEP 02 / 05

A HOOK RUNS FOR A MISSING VALUE, BUILDS IT AND STORES IT

Fig 8.2The accent box runs once; after that the read stops at the cache in the top right.

Lazy initialization needs three things: somewhere the value will live, a trigger that runs when it is missing, and a store. The trigger builds the object, stores it, and returns it.

The store is what makes it lazy rather than slow: every later read finds the value and never reaches the trigger. Without the store you have a property that rebuilds the object on every read.

STEP 03 / 05

NUMPY’S __GETATTR__ RUNS ONLY FOR NAMES THE MODULE DOESN’T HAVE

Fig 8.3Same boxes, NumPy’s names: here the cache is filled by the import system, not by the hook.

PEP 562 lets a module define __getattr__(name). Python calls it only when normal lookup fails, so it never slows down attributes that exist. NumPy’s version has one branch per lazy submodule, such as import numpy.testing as testing, and raises AttributeError for anything else.

There is no explicit cache in that function. Importing a submodule binds it as an attribute of its parent package, so after the first np.testing the name exists and the hook never runs for it again. NumPy also defines __dir__, so the lazy names still show up in autocomplete.

NUMPY: A MODULE-LEVEL __GETATTR__ FOR ITS SUBMODULES
# numpy/__init__.py · NumPy 2.5.5 (trimmed, dedented)
def __getattr__(attr):  # ← Python calls it only for names numpy doesn't have (PEP 562)
    # Warn for expired attributes
    import warnings

    if attr == "linalg":
        import numpy.linalg as linalg  # ← the expensive part, done on first use
        return linalg
    elif attr == "fft":
        import numpy.fft as fft
        return fft
    # ← … six more: dtypes, random, polynomial, ma, ctypeslib, exceptions …
    elif attr == "testing":
        import numpy.testing as testing  # ← pulls in unittest; most programs never need it
        return testing
    # ← … seven more submodules, then messages for removed names …
    raise AttributeError(f"module {__name__!r} has no attribute {attr!r}")
STEP 04 / 05

THE SECOND READ NEVER REACHES THE HOOK

Fig 8.4Follow 1 to 7: calls 2 to 5 happen once; 6 and 7 are every read after that.

The first read misses, calls the hook, imports the submodule, and returns it. By the time the hook returns, the import system has already set the attribute, so the second read is an ordinary module attribute lookup.

The sample shows the same mechanism on a module made at runtime: the hook stores the value with setattr, the second read returns it directly, and the call log holds one entry.

A MODULE __GETATTR__ THAT RUNS ONCE
import types

calls: list[str] = []
lib = types.ModuleType("lib")


def __getattr__(name: str) -> object:
    calls.append(name)
    if name == "heavy":
        value = {"loaded": True}  # stands in for an expensive import
        setattr(lib, name, value)  # store it where normal lookup finds it
        return value
    raise AttributeError(f"module 'lib' has no attribute {name!r}")


lib.__getattr__ = __getattr__  # PEP 562: the module's own __getattr__
lib.heavy  # → {'loaded': True}
lib.heavy  # → {'loaded': True}
calls  # → ['heavy']
STEP 05 / 05

CACHED_PROPERTY STORES THE VALUE IN THE INSTANCE’S __DICT__

Fig 8.5The highlighted row is the whole difference: where, if anywhere, the result is kept.

For an attribute of your own objects, use functools.cached_property. It is a descriptor that runs your method on the first read and writes the result into the instance’s __dict__ under the same name. Python checks that dict before the class, so later reads never reach the method.

To recompute, del obj.rows. For modules, write a module __getattr__. For a whole import, Python 3.15 adds lazy import, shown in the deep dive.

AN ATTRIBUTE COMPUTED ON THE FIRST READ ONLY
from functools import cached_property


class Report:
    def __init__(self, path: str) -> None:
        self.path = path
        self.loads = 0

    @cached_property
    def rows(self) -> list[str]:  # runs on the first read only
        self.loads += 1
        return ["row 1", "row 2"]  # stands in for reading a large file


r = Report("sales.csv")
"rows" in vars(r)  # → False
(len(r.rows), len(r.rows), r.loads)  # → (2, 2, 1)
"rows" in vars(r)  # → True
del r.rows  # forget the value; the next read computes it again

COMMON QUESTIONS

Is cached_property thread-safe?
It has no lock since Python 3.12. Two threads reading it first at the same moment may both compute it; the last write wins.
Does cached_property work with __slots__?
Only if the class still has a __dict__. Without one, the first read raises a TypeError, because there is nowhere to store the value.

WHAT YOU NOW KNOW

  • Importing every submodule up front makes every program pay for code most of them never use.
  • NumPy’s module __getattr__ imports numpy.testing on the first read, and the import binds it, so later reads skip the hook.
  • Use cached_property for attributes, module __getattr__ for modules, and lazy import for whole imports on 3.15.
DEEP DIVE: also in Django’s settings, lazy import in Python 3.15, what laziness costs, errors, look-alikes

Also in Django’s settings

django.conf.settings is a LazySettings object, created when the module is imported but empty. The first attribute read calls _setup(), which reads DJANGO_SETTINGS_MODULE and imports your settings file. Each value read is also copied into the object’s own __dict__, so the next read of that setting skips __getattr__ entirely. That is why a script can call settings.configure() before anything reads a setting.

DJANGO: SETTINGS THAT LOAD ON THE FIRST READ
# django/conf/__init__.py · Django 6.1.3 (trimmed)
class LazySettings(LazyObject):
    def _setup(self, name=None):  # ← builds the real Settings, once
        settings_module = os.environ.get(ENVIRONMENT_VARIABLE)
        ...
        self._wrapped = Settings(settings_module)  # ← imports your settings.py

    def __getattr__(self, name):  # ← every settings.DEBUG read starts here
        if (_wrapped := self._wrapped) is empty:
            self._setup(name)
            _wrapped = self._wrapped
        val = getattr(_wrapped, name)
        ...
        self.__dict__[name] = val  # ← cached: the next read skips __getattr__
        return val


settings = LazySettings()  # ← nothing is imported yet

Lazy import in Python 3.15

PEP 810 adds a lazy import statement in Python 3.15. lazy import json binds the name at once, and the module loads the first time the name is used. It does for a whole import what NumPy’s __getattr__ does by hand for its submodules.

PYTHON 3.15: AN IMPORT THAT WAITS FOR ITS FIRST USE
lazy import json  # bound now, imported on first use
import sys

"json" in sys.modules  # → False
json.dumps([1, 2])  # → '[1, 2]'
"json" in sys.modules  # → True

What laziness costs

The first request that touches a lazy object is slower than the others, which shows up as a slow first call after every deploy. Errors move too: a missing dependency fails at the first read, not at startup. Warm up the objects you know you need when the process starts.

ERRORS THIS CHAPTER EXPLAINS

TypeError: No '__dict__' attribute on 'S' instance to cache 'x' property.
cached_property on a class with __slots__ and no __dict__
add __dict__ to __slots__, or compute it in __init__
AttributeError: module 'numpy' has no attribute '…'
the name is in no branch of the module’s __getattr__
check the spelling; the hook covers only the names it lists

LOOKS LIKE

Proxy, part 2
stands in for another object on every access; Django’s LazyObject is both a proxy and lazy
a wrapper class that forwards __getattr__ to a _wrapped object
Singleton, chapter 06
one shared object; it may be created lazily, but sharing is the point
a module attribute or cached getter that every caller reaches
Object pool, chapter 09
keeps several finished objects to lend out, not one value per holder
acquire and release methods around a queue of objects

Object pool: reuse objects that are slow to make

An object pool keeps finished objects that are expensive to create and lends them out again instead of making new ones. Every HTTPS request to a server needs an open connection, and opening one costs a TCP handshake and a TLS handshake before the first byte of the request. A client that calls the same API a thousand times shouldn’t pay that a thousand times. How does urllib3, the library under requests, reuse connections, and what happens when all of them are busy?

THE PATTERN

you’ll write it sometimes

Keep a set of ready objects, lend one out for each job, and take it back afterwards for the next job.

ROLES
  • poolholds idle objects and a limit on how many it keeps
  • check out and give backlend an idle object, or make one; take it back when the job is done
  • pooled objectslow to create and safe to reuse: an open connection, a thread
REACH FOR IT WHEN
Creating the object costs a network handshake, a process or a thread, and the program creates many over its life.
SKIP IT WHEN
The objects are cheap, like most Python objects. A pool then adds state and bugs for nothing.
HOW TO SPOT IT
A queue or list of idle objects, get and put or acquire and release methods, a maxsize and a block flag.
IN PYTHON
A class around queue.LifoQueue; requests sessions, database engines and executors ship one ready-made.
SEEN IN
  • urllib3 2.8 · src/urllib3/connectionpool.pyHTTPConnectionPool keeps idle connections in a LifoQueue; _get_conn() reuses one and _put_conn() returns it.
  • requests 2.34 · src/requests/adapters.pyHTTPAdapter asks urllib3 for 10 pools of up to 10 connections each by default.
  • SQLAlchemy 2.1 · pool/impl.pyQueuePool, the default for most engines, keeps 5 database connections and allows 10 more.
  • CPython 3.15 · Lib/concurrent/futures/thread.pyThreadPoolExecutor runs submitted calls on a reused set of worker threads.
STEP 01 / 04

A NEW CONNECTION COSTS TWO ROUND TRIPS BEFORE THE REQUEST

Fig 9.1Calls 1 to 4 are two round trips of handshake; a reused connection starts at call 5.

A fresh HTTPS connection exchanges a TCP handshake and a TLS 1.3 handshake, two round trips, before the request can leave. Across an ocean, a round trip is easily tens of milliseconds, so the handshake can cost more than the request itself.

HTTP/1.1 keeps a connection open after a response by default, a keep-alive connection. The next request to the same host can reuse it and skip both handshakes, but only if the client kept the connection somewhere.

STEP 02 / 04

CHECK OUT, USE, GIVE BACK

Fig 9.2Follow 1 to 4: the accent pool only creates an object at 3, when the queue had none to give.

A pool lends objects instead of creating them. Checking out takes an idle object if there is one and creates a new one otherwise. Giving back puts the object in the idle set for the next borrower.

Two settings shape it: how many idle objects to keep, and what to do when every object is lent out. It can make one more and drop it later, or make the caller wait.

STEP 03 / 04

URLLIB3’S POOL STARTS FULL OF NONE

Fig 9.3Look at the queue in the top right: a None is a slot that has no connection yet.

urllib3 fills each pool’s LifoQueue with maxsize copies of None when it is created. _get_conn() takes whatever is on top: a connection to reuse, or None, which means “open a new one”, through conn or self._new_conn(). A connection the server dropped is closed first, and it reconnects when it is next used.

_put_conn() puts the connection back without waiting. If the queue is already full, because more connections were opened than maxsize, it closes the extra one and logs a warning.

URLLIB3: CHECKING OUT A CONNECTION
# src/urllib3/connectionpool.py · urllib3 2.8.0 (trimmed, parameters shortened)
class HTTPConnectionPool(ConnectionPool, RequestMethods):
    def __init__(self, host, port=None, maxsize=1, block=False, **kwargs):
        ...
        self.pool = self._new_pool_queue(maxsize)  # ← a LifoQueue: last in, first out
        self.block = block
        for _ in range(maxsize):
            self.pool.put(None)  # ← starts full of None: "no connection yet"

    def _get_conn(self, timeout=None):  # ← check out
        conn = None
        ...
        try:
            conn = self.pool.get(block=self.block, timeout=timeout)
        except queue.Empty:
            if self.block:
                raise EmptyPoolError(self, "Pool is empty and a new connection can't be opened due to blocking mode.") from None
            pass  # Oh well, we'll create a new connection then
        ...
        return conn or self._new_conn()  # ← None means: open a new one
URLLIB3: GIVING IT BACK
# src/urllib3/connectionpool.py · urllib3 2.8.0 (trimmed)
class HTTPConnectionPool(ConnectionPool, RequestMethods):
    def _put_conn(self, conn):  # ← give back
        if self.pool is not None:
            try:
                self.pool.put(conn, block=False)  # ← back on top of the stack
                return  # Everything is dandy, done.
            except queue.Full:
                if conn:
                    conn.close()  # ← one more than maxsize: close it
                ...
                log.warning(
                    "Connection pool is full, discarding connection: %s. Connection pool size: %s",
                    self.host,
                    self.pool.qsize(),
                )
        ...
STEP 04 / 04

LEND OBJECTS INSIDE A WITH BLOCK SO THEY ALWAYS COME BACK

Fig 9.4The highlighted row is the choice every pool makes: go over the limit, or wait.

Most of the time you use a pool you already have: one requests.Session per host you call, one SQLAlchemy engine per database. Create it once and reuse it; a new Session per request throws the pool away each time.

If you write one, lend objects through a context manager, so an exception can’t leak one. The sample’s lease() gives the object back in a finally, and reuses the most recently returned object first.

A SMALL POOL THAT ALWAYS GETS ITS OBJECTS BACK
import queue
from collections.abc import Callable, Iterator
from contextlib import contextmanager


class Pool[T]:
    def __init__(self, make: Callable[[], T]) -> None:
        self.make = make
        self.idle: queue.LifoQueue[T] = queue.LifoQueue()
        self.created = 0

    @contextmanager
    def lease(self) -> Iterator[T]:
        try:
            obj = self.idle.get_nowait()  # the most recently returned one
        except queue.Empty:
            obj = self.make()
            self.created += 1
        try:
            yield obj
        finally:
            self.idle.put(obj)  # back in the pool, even if the block raised


pool = Pool(list)
for _ in range(3):
    with pool.lease() as conn:
        pass
pool.created  # → 1
with pool.lease() as a, pool.lease() as b:
    pair = (a, b)
pool.created  # → 2

COMMON QUESTIONS

Why a LIFO queue and not FIFO?
It hands out the connection that was idle for the shortest time. That one is the least likely to have been closed by the server.
Do I ever need to write my own pool?
Rarely. requests sessions, SQLAlchemy engines and the executors in concurrent.futures already pool what is expensive.

WHAT YOU NOW KNOW

  • A new HTTPS connection costs a TCP and a TLS handshake, two round trips, before the request can leave.
  • urllib3’s pool is a LifoQueue started with maxsize Nones: _get_conn() reuses a connection or opens one for a None, and _put_conn() returns it.
  • Reuse one Session or engine, and lend your own pooled objects inside a with block.
DEEP DIVE: also in SQLAlchemy, pools of pools, stale objects, look-alikes

Also in SQLAlchemy

An SQLAlchemy Engine owns a QueuePool, the default for every engine except an in-memory SQLite database. It keeps up to pool_size connections, 5 by default, and opens up to max_overflow more, 10 by default, when all are checked out. Beyond that, a caller waits up to pool_timeout, 30 seconds by default, and then gets an error.

Pools of pools

urllib3’s PoolManager keeps one HTTPConnectionPool per scheme, host and port, and by default up to 10 of them, dropping the least recently used. requests’ HTTPAdapter sets both numbers through pool_connections and pool_maxsize. A client that talks to many hosts needs the first raised; a client with many threads talking to one host needs the second.

Stale objects

A pooled object can go bad while it waits: the server closes an idle connection, or a database restarts. urllib3 checks each connection as it is checked out and closes a dropped one, which reconnects on its next request. SQLAlchemy offers pool_pre_ping, which tests a connection before handing it out, and pool_recycle, which retires connections after a set age.

LOOKS LIKE

Flyweight, part 2
shares objects among many users at once; a pooled object has one user at a time
a cache keyed by value, with no check out and give back
Singleton, chapter 06
one shared object for everyone, never lent out
a module attribute or cached getter, no idle queue
Lazy initialization, chapter 08
creates one value on first use and keeps it; a pool also creates on demand, then lends
@cached_property or __getattr__, not acquire and release

Dependency injection: hand objects their collaborators

Dependency injection means a piece of code receives the objects it depends on from outside instead of creating them itself. A checkout function that opens its own database connection and reads the clock can only be tested against a real database at the real time. A test wants to hand it a temporary database and a fixed clock. How does pytest decide what to pass to def test_checkout(db, clock) when you never call the test yourself?

THE PATTERN

you’ll write it often

Code asks for what it needs as parameters; something outside builds those objects and passes them in.

ROLES
  • clientdeclares what it needs, usually as parameters
  • providerknows how to build one dependency: a fixture, a factory function
  • injectormatches needs to providers, builds them and calls the client
REACH FOR IT WHEN
Code depends on things you swap in tests or between environments: databases, clocks, HTTP clients, settings.
SKIP IT WHEN
The dependency is a pure function or a constant that never changes. Import it; injecting it adds noise.
HOW TO SPOT IT
Parameters a framework fills in, such as pytest fixtures or FastAPI’s Depends(), or constructors that take clients.
IN PYTHON
A parameter with a default is the simplest form; frameworks read inspect.signature() to do it for you.
SEEN IN
  • pytest 9.2 · src/_pytest/python.pypytest_pyfunc_call() builds the test’s arguments from fixture values and calls the test with them.
  • pytest 9.2 · src/_pytest/compat.pygetfuncargnames() reads the test’s parameter names with inspect.signature().
  • FastAPI 0.165 · fastapi/dependencies/utils.pysolve_dependencies() calls each Depends() provider and passes the results to the endpoint.
STEP 01 / 05

A FUNCTION THAT BUILDS ITS OWN DATABASE CAN ONLY BE TESTED WITH ONE

Fig 10.1The red arrow is the problem: the test can reach the function, but not what it builds inside.

checkout() creates its database and reads the clock itself. A test that calls it talks to the production database, and any assertion about the time has to race the real clock.

The fix is not a smarter test. Move the decision about which database and which clock out of the function, so the caller can make it.

EVERYTHING IT NEEDS, BUILT INSIDE
from datetime import datetime


class Database:
    def __init__(self, url: str) -> None:
        self.url = url  # imagine a real connection here


def checkout(cart_id: int) -> str:
    db = Database("postgresql://prod/shop")  # built inside: a test can't swap it
    now = datetime.now()  # read inside: a test can't fix the time
    return f"order for cart {cart_id} at {now:%H:%M} in {db.url}"
STEP 02 / 05

THE CLIENT ASKS; SOMETHING ELSE BUILDS AND PASSES

Fig 10.2Every arrow starts at the accent box: the client never calls a provider itself.

With injection, checkout(db, clock) only declares what it needs. Providers know how to build each thing. An injector matches the declared names or types to providers, builds the values and makes the call.

In plain Python the injector is often just the caller: whoever calls checkout() passes a database. Frameworks add the matching step, so you declare needs and they build the objects for you.

STEP 03 / 05

PYTEST MATCHES PARAMETERS TO FIXTURES BY NAME

Fig 10.3Same boxes, pytest’s names: the two providers on the right are your fixture functions.

pytest reads each test’s parameter names and treats them as fixture names. During setup, _fillfixtures() calls getfixturevalue(name) for each one, which finds the fixture function with that name and runs it.

At call time, pytest_pyfunc_call() builds a dict from those names to the values and calls testfunction(**testargs). The test never imports a fixture; the parameter name is the whole request.

PYTEST: SETUP FILLS THE VALUES, THE CALL INJECTS THEM
# src/_pytest/python.py · pytest 9.2.0 (trimmed)
@hookimpl(trylast=True)
def pytest_pyfunc_call(pyfuncitem: Function) -> object | None:
    testfunction = pyfuncitem.obj
    ...
    funcargs = pyfuncitem.funcargs  # ← fixture values, built during setup
    testargs = {arg: funcargs[arg] for arg in pyfuncitem._fixtureinfo.argnames}
    result = testfunction(**testargs)  # ← the injection: test(db=…, clock=…)
    ...
    return True


# src/_pytest/fixtures.py · pytest 9.2.0 (trimmed)
class TopRequest(FixtureRequest):
    def _fillfixtures(self) -> None:  # ← runs at setup, before the test
        item = self._pyfuncitem
        for argname in item.fixturenames:
            if argname not in item.funcargs:
                item.funcargs[argname] = self.getfixturevalue(argname)  # ← find the provider by name
STEP 04 / 05

A FIXTURE RUNS ONCE PER SCOPE, AND THE TEST GETS THE CACHED VALUE

Fig 10.4Follow 1 to 7: call 3 runs once per session, however many tests ask for db.

The names come from getfuncargnames(), which reads inspect.signature() and keeps parameters without defaults. A fixture with scope="session" runs once for the whole run; every test that names it gets the same cached value. The default scope runs it again for each test.

A fixture that uses yield is setup and teardown in one function: pytest runs it to the yield, injects the value, and resumes it when the scope ends.

A TEST AND ITS FIXTURES, AS YOU WRITE THEM
from datetime import datetime

import pytest


@pytest.fixture(scope="session")
def db():  # built once for the whole run
    database = Database("sqlite:///:memory:")
    yield database  # every test that asks for db runs here
    database.close()  # teardown, after the last one


@pytest.fixture
def clock():  # built again for every test
    return lambda: datetime(2026, 11, 22, 9, 30)


def test_checkout(db, clock):  # names, not imports: pytest matches them to fixtures
    assert checkout(7, db=db, now=clock).startswith("order for cart 7")
PYTEST: WHERE THE FIXTURE NAMES COME FROM
# src/_pytest/compat.py · pytest 9.2.0 (trimmed, annotations removed)
def getfuncargnames(function, *, name="", cls=None):
    ...
    try:
        parameters = signature(function).parameters.values()  # ← inspect.signature
    except (ValueError, TypeError) as e:
        ...
    arg_names = tuple(
        p.name
        for p in parameters
        if (
            p.kind is Parameter.POSITIONAL_OR_KEYWORD
            or p.kind is Parameter.KEYWORD_ONLY
        )
        and p.default is Parameter.empty  # ← a parameter with a default isn't a fixture
    )
    ...
    return arg_names
STEP 05 / 05

A DEFAULT ARGUMENT IS ALREADY DEPENDENCY INJECTION

Fig 10.5The highlighted row is the reason to inject: the dependency is visible where the function is defined.

In your own code, start with parameters. checkout(cart_id, now=datetime.now) runs on the real clock and takes a fixed one in tests, with no framework. For classes, take collaborators in __init__ rather than building them there.

The sample also has a ten-line injector that reads inspect.signature() and fills parameters from a dict of providers. That is the mechanism pytest and FastAPI build on, with scopes, caching and error messages added.

INJECTION WITH A DEFAULT, AND A TINY INJECTOR
import inspect
from collections.abc import Callable
from datetime import datetime


def checkout(cart_id: int, now: Callable[[], datetime] = datetime.now) -> str:
    return f"cart {cart_id} at {now():%H:%M}"  # the default is the real clock


checkout(7, now=lambda: datetime(2026, 11, 22, 9, 30))  # → 'cart 7 at 09:30'

PROVIDERS = {"now": lambda: (lambda: datetime(2026, 1, 1, 8, 0))}


def call_with_deps(fn: Callable[..., str], **given: object) -> str:  # a tiny injector
    params = inspect.signature(fn).parameters
    extra = {n: PROVIDERS[n]() for n in params if n in PROVIDERS and n not in given}
    return fn(**given, **extra)


call_with_deps(checkout, cart_id=8)  # → 'cart 8 at 08:00'

COMMON QUESTIONS

Do I need a dependency injection library in Python?
Rarely. Parameters, defaults and factory functions cover most code, and pytest and FastAPI already inject where it pays off.
How is FastAPI’s Depends() different?
You name the provider explicitly, Depends(get_db), instead of by parameter name. By default a provider runs once per request even if several dependencies ask for it.

WHAT YOU NOW KNOW

  • A function that builds its own database and reads the clock can only be tested against the real ones.
  • pytest reads a test’s parameter names, builds the fixtures with those names once per scope, and calls the test with them.
  • In your code, take collaborators as parameters with sensible defaults; that alone is dependency injection.
DEEP DIVE: also in FastAPI, kinds of injection, service locators, errors, look-alikes

Also in FastAPI

FastAPI reads an endpoint’s signature too. A parameter declared as db: Session = Depends(get_db) names its provider, and solve_dependencies() calls get_db, including any dependencies get_db declares itself, before it calls the endpoint. A provider that yields runs its cleanup after the endpoint, before or after the response depending on its scope argument, the same shape as a pytest fixture.

Kinds of injection

Constructor injection passes collaborators to __init__ and keeps them on the instance. Parameter injection passes them to each call, as checkout(now=…) does. Setter injection assigns them after creation, which leaves a window where the object is incomplete; Python code rarely needs it.

Service locators

A service locator is a registry that code asks for its dependencies at runtime:get_service('db'). It removes the hard-coded constructor but hides the dependency again, because the signature no longer shows it. Injection keeps the need in the signature, where readers and tests can see it.

ERRORS THIS CHAPTER EXPLAINS

fixture 'dbb' not found
a parameter names no fixture visible to the test
fix the spelling, or move the fixture into a conftest.py the test can see
ScopeMismatch
a session-scoped fixture asks for a function-scoped one
give the inner fixture the wider scope, or narrow the outer one

LOOKS LIKE

Singleton, chapter 06
one shared object that code reaches by name, invisible in its signature
a module-level name used inside the function, not passed in
Factory Method, chapter 02
a subclass overrides how a collaborator is made; injection passes it in
an overridden get_something() instead of a parameter
Strategy, part 3
the injected object is an algorithm the client calls, such as a key function
a callable parameter such as key= or a hasher, used inside the function

The patterns in one table

Every pattern from the ten chapters: the problem it solves, the move it makes, where you have already met it, and what Python gives you instead. The pattern names link back to their chapters, and the projects to their source.

  1. problem
    one class, built from several kinds of input
    move
    a from_ classmethod parses the input, then calls cls(…)
    seen in
    CPython datetime, pydantic, pandas
    use when
    data arrives as text, numbers or dicts
    Python
    built in: @classmethod passes the subclass
  2. problem
    a procedure in a base class needs an object a subclass should choose
    move
    the procedure calls an overridable method that returns the class
    seen in
    Django form views, socketserver
    use when
    a framework runs the same steps for every user
    Python
    often a class attribute, or a parameter
  3. problem
    several classes must always come from the same family
    move
    one class names every product class; swap the class, swap the set
    seen in
    SQLAlchemy dialects, Django database backends
    use when
    you support several databases or platforms
    Python
    a class of classes, or a dataclass of callables
  4. problem
    parts depend on each other or arrive over time
    move
    collect the parts in a spec, then build in a fixed order once
    seen in
    requests, SQLAlchemy select()
    use when
    order matters or parts repeat
    Python
    keyword arguments cover the simple case
  5. problem
    many variations of a configured object, while others hold the original
    move
    every changing method clones first and changes the clone
    seen in
    Django QuerySet, SQLAlchemy, copy.replace()
    use when
    objects are shared and extended in several directions
    Python
    copy.replace() and __replace__
  6. problem
    everyone must share exactly one object
    move
    create it once and give every caller that instance
    seen in
    CPython random, logging
    use when
    one default generator, logger or settings object
    Python
    a module attribute: modules run once
  7. problem
    a name from config or a URL must become a class, plugins included
    move
    registrants add names to a dict; a lookup reads it
    seen in
    SQLAlchemy plugins, Flask routes, codecs
    use when
    kinds are chosen by name and plugins add more
    Python
    __init_subclass__, a decorator, or entry points
  8. problem
    an expensive object that many runs never use
    move
    build it on the first read and store it where the next read finds it
    seen in
    NumPy, functools, Django settings
    use when
    imports, connections or values that are slow to make
    Python
    cached_property, module __getattr__, lazy import
  9. problem
    each new object costs a handshake, a process or a thread
    move
    lend idle objects, make one only when none is waiting
    seen in
    urllib3, SQLAlchemy QueuePool, ThreadPoolExecutor
    use when
    connections or workers you create many times
    Python
    use the one inside your Session or engine
  10. problem
    code that builds its own collaborators can’t be tested or reconfigured
    move
    take collaborators as parameters; something outside builds them
    seen in
    pytest fixtures, FastAPI Depends
    use when
    databases, clocks and clients that tests replace
    Python
    a parameter with a default

TELLING THEM APART

Alternative constructor or Factory Method
you call an alternative constructor; a base class calls a factory method for you
from_ classmethod on the class, or get_something_class() overridden in a subclass
Factory Method or Abstract Factory
one product from one method, or a matching set from one class
a single hook, or several class attributes that change together
Builder or alternative constructor
parts over several calls and a final build, or everything in one call
set, add or where() methods and build(), or one from_ call
Prototype or Builder
copying a finished object, or assembling a new one; generative builders do both
_clone() or copy.replace(), or build() and prepare()
Singleton or dependency injection
code reaches one shared object by name, or receives an object it is given
a module-level name inside the function, or a parameter
Registry or plugin hooks (part 3)
look up one entry by name, or call every registered implementation
registry.load(name), or hook.something(…) returning a list
Lazy initialization or Proxy (part 2)
build a value once on first use, or stand in for an object on every access
@cached_property, or a wrapper forwarding __getattr__ to a wrapped object
Object pool or Flyweight (part 2)
one user at a time borrows an object, or many users share one at once
acquire and release, or a cache returning the same instance per key

THREE THINGS TO KEEP

  • Creational patterns decide which object gets made, when it gets made, and which code makes that choice.
  • In Python a class is an object you can pass and store, so most of these patterns shrink.
  • A shared or hidden object is convenient until a test needs a different one. Pass dependencies in where you can.

Next time a library hands you an object, ask who decided to make it, and when.