100 LINKS · 1 THREAD · 24.6 SECONDS DOWN TO 5.2

Async Python:
how one thread checks 100 links

Async Python lets one program wait for many slow things at once, such as web pages or database replies, on a single thread. While one request waits for the network, the same thread works on another.

WHAT THIS TUTORIAL COUNTS

One script checks the 100 links on a documentation site, at most 10 at a time. The tutorial counts every switch: one time the thread is handed to one of the script’s own functions, to start it or to continue it where it paused.

switches in one 5.2-second run494
  • waiting for the network203 switches
    a page’s bytes arrived, so the function that asked for them continues
  • starting the checks101 switches
    each link gets its own job, which starts once; the main function continues when all 100 are done
  • waiting for a parsed page96 switches
    counting a page’s links runs on another core, and the check continues when it is done
  • waiting for a free slot90 switches
    only 10 links are fetched at once, so 90 checks start by waiting in line
  • the rest4 switches
    the main function starting, one timeout, one refused connection, one saved report
AND WHAT THE ONE THREAD DID FOR 5.3 SECONDS
  • waiting for the operating system5.17 s
    asleep inside one system call until a socket has data or a timer is due
  • running Python0.15 s
    the 494 switches and everything between them
Python 3.15.2 · one run on a 12-core laptop · test server on the same machine

A documentation site links to 100 pages, and a script checks that each one still answers. Written the ordinary way, it asks for one page, waits, then asks for the next: 24.6 seconds, of which the processor works for 0.117. Rewritten with asyncio, the same script on the same single thread finishes in 5.2 seconds. The tutorial follows that run and counts every : each time the thread is handed to one of the script’s functions, to start it or to continue it after a pause. There are 494. Between them the thread waits, and arranging that waiting is the whole technique.

You need to know Python functions, exceptions and with blocks, and what an HTTP request is; no async experience is assumed. Code samples run on Python 3.15, and each feature is marked with the release that added it. The scenes are static diagrams except three that play themselves. are tappable. Numbers come from one run on a 12-core laptop against a test server on the same machine, so the network is fast and repeatable; on the real internet every wait is longer, and the case for async gets stronger.

100 links, one at a time

You have the 100 links from a documentation site in links.txt and a short script that checks each one still answers. It works. It takes 24.6 seconds, and for 24.5 of them the program is doing nothing at all. The switch count starts at 0 here: this version has no function that can pause, so there is nothing to switch to.

STEP 01 / 04

24.6 SECONDS, 0.117 OF THEM WORK

the program’s one thread24.59 s
  • waiting for servers24.47 sthe dim bar: no CPU used
  • working0.117 s · 0.5%the thin ticks: time.process_time() for the whole run
Fig 0.1The strip is the program’s one thread, drawn to scale. Only the thin accent ticks are the processor working.

The plain version reads links.txt and calls urllib.request.urlopen on each link in turn, with a 5-second timeout. Each call is a : it does not return until the server has answered, and the program’s one thread can do nothing else in the meantime.

Timed, the loop takes 24.59 seconds. time.process_time(), which counts only the time the processor spent on this program, says 0.117 seconds. The program worked for 0.5% of the run; for the rest it was waiting for another program, the server, to answer.

CHECK_LINKS.PY, VERSION 0: ONE LINK AFTER ANOTHER
import time
import urllib.error
import urllib.request

urls = open("links.txt").read().split()              # 100 links
start = time.perf_counter()
for url in urls:
    try:
        with urllib.request.urlopen(url, timeout=5) as resp:
            resp.read()                              # the thread waits here
    except urllib.error.HTTPError:
        pass                                         # 404 and 500 still answered
    except (urllib.error.URLError, TimeoutError):
        pass                                         # refused, or silent for 5 s
print(f"{time.perf_counter() - start:.2f} s")        # 24.59 s, 0.117 s of it CPU
STEP 02 / 04

THE RUN IS THE SUM OF EVERY WAIT

  • one link, waiting for its server
page-17 · no answer · 5 s from 3.63 s
page-00page-99
Fig 0.2Each row is one link, and each starts where the one above ended. The accent row is page-17’s 5-second timeout.

Done one after another, the run time is the sum of every link’s wait. The test server answers the 98 working pages after delays between 0.039 and 0.718 seconds, a median of 0.1565, and those delays add up to 19.19 seconds.

page-17 never answers, so its request waits for the full 5-second timeout. Connecting and reading take another 0.4 seconds. Together that is the 24.59 on the clock, and the one dead link alone costs a fifth of the run.

STEP 03 / 04

WHERE A WAITING THREAD WAITS

  1. check_links.pyresp.read() for page-00
  2. urllib.requestreads the HTTP response from the socket
  3. socket.recv()a system call: a request to the kernel
  4. kernel: thread parked until data arrivesno CPU used; runnable again when bytes land in the socket buffer
  5. network cardhere, the loopback interface to the test server
Fig 0.3Follow the call down the layers to the kernel. The accent layer is where the thread is parked.

resp.read() ends in recv(), a system call: Python asks the operating system’s kernel for bytes from the socket, the kernel’s handle on one network connection. If no bytes have arrived, the kernel parks the thread, and makes it runnable again when data lands in the socket’s buffer.

A parked thread uses no CPU, which is why the processor sat idle for 24.5 seconds. The cost is the thread itself. This program has one, it is inside recv() for one link, and the other 99 cannot even be started until that call returns.

STEP 04 / 04

TWO WAYS TO WAIT FOR 100 THINGS

100 threadseach thread parked in recv() on its own socket100 system calls waiting at once
1 thread and a selector
1 threadkqueue
one call watches all 100 sockets and returns the ready onesfilled squares: sockets with data now
Fig 0.4Left, 100 threads each parked on one socket. Right, one thread asking about all 100 sockets at once.

There are two ways to wait for 100 links at once. One is to give every link its own thread, so 100 threads sit parked in 100 recv() calls; the 100-thread version of this script takes 5.13 seconds. The other is to keep one thread and ask the kernel a different question: which of these 100 sockets has data now?

macOS answers that question with kqueue and Linux with epoll. Python wraps both in its selectors module as a , and asyncio is built on it. Threads work fine at 100; the concurrency tutorial follows what they cost at 10,000.

WHAT YOU NOW KNOW

  • The blocking script spends 24.59 s waiting and 0.117 s working, because each call holds the only thread until its server answers.
  • Done one after another, the run is the sum of every wait: 19.2 s of server delays plus page-17’s 5-second timeout.
  • To wait for 100 links, park one thread per link, or ask the kernel which sockets are ready with kqueue. asyncio does the second.
deep dive: what urllib does per link, the 100-thread version, and why the kernel can watch 100 sockets

What urllib does per link. It connects (no DNS lookup, because the host is an IP address), sends the request line and headers, reads the status line and headers, reads the body until Content-Length bytes have arrived, and closes. timeout=5 applies to each blocking socket operation, not to the whole request: a server that sends one byte every 4 seconds never trips it. asyncio.timeout(), in chapter 04, bounds a whole block instead.

The 100-thread version. ThreadPoolExecutor(100) running the same function finishes in 5.13 seconds, as fast as asyncio here. At 100 links threads are fine. The difference shows up in memory and scheduling at thousands of connections, which the concurrency tutorial measures; this one does not repeat it.

Why the kernel can do this at all. It already keeps a buffer for every socket. select scans the whole list of sockets on every call, while kqueue (FreeBSD, 2000) and epoll (Linux, 2002) keep the watched set inside the kernel and return only the sockets that are ready.

The numbers are local. The test server runs on the same machine, so there is no real network delay; each wait is the server’s own seeded pause. On the public internet every wait is longer, which makes the blocking version worse and leaves the async version bounded by its slowest link.

A function that can pause

The blocking script’s problem is that a function waiting for a server cannot step aside and let another one run. Python’s answer is a function that can stop in the middle, keep its place, and be continued later: a , written with async def. In the final script the first switch is main() starting, so the switch count goes from 0 to 1.

STEP 01 / 05

CALLING IT RUNS NOTHING

the callcoro = check(url)returns at once with<coroutine object check>sys.getsizeof(coro)208 bytes
check('…/page-00.html')CORO_CREATED▶ on the def line: no line of the body has run yet
  • url'http://127.0.0.1:8765/docs/page-00.html'
if nothing awaits it, Python prints this when the object is garbage-collected:RuntimeWarning: coroutine 'check' was never awaited
Fig 1.1The card is what the call returned. Its state chip says CORO_CREATED, and the pointer is still on the def line.

Calling check(url) runs none of its body. It returns a coroutine object: a frame with the arguments already bound, its pointer still on the def line, and no request sent. sys.getsizeof puts that object at 208 bytes.

If nothing ever awaits the object, the body never runs at all. When it is garbage-collected, Python prints RuntimeWarning: coroutine 'check' was never awaited, the usual sign of a missing await.

STEP 02 / 05

AWAIT IS WHERE IT STOPS

  1. answer()CORO_CREATEDon the def line: nothing has run
    • cr_awaitNone
  2. answer()CORO_SUSPENDEDpaused at line 4: await asyncio.sleep(0)
    • cr_await<coroutine object sleep>
  3. answer()CORO_CLOSEDraised StopIteration(42): stop.value is 42
    • cr_frameNone: the frame is gone
Fig 1.2Read the three cards in order: created, paused at the await, closed with 42.

pauses the coroutine and hands control back to whatever is running it. The frame stays in memory with its local variables and the line it stopped at, so the next run continues from that line instead of starting over.

You can be the thing that runs it. coro.send(None) runs answer() up to its first pause and returns. The second send runs it from the await to the end, and the return value, 42, arrives on a StopIteration exception.

asyncio drives every coroutine through the same two calls: send() to continue it, and throw() to continue it with an exception raised at the paused line.

DRIVING A COROUTINE BY HAND
import asyncio

async def answer():
    await asyncio.sleep(0)       # pause once
    return 42

coro = answer()                  # runs nothing: a 200-byte coroutine object
print(coro.send(None))           # runs to the await, prints None
try:
    coro.send(None)              # runs from the await to the end
except StopIteration as stop:
    print(stop.value)            # 42
STEP 03 / 05

A GENERATOR WITH NEW KEYWORDS

  1. Python 2.52006generators get send()
  2. Python 3.32012yield from
  3. Python 3.42014asyncio, built on yield from
  4. Python 3.52015async and await
  5. Python 3.72018asyncio.run()
await x
  1. GET_AWAITABLE≠ prepares the object
  2. SEND
  3. YIELD_VALUE
  4. RESUME
  5. JUMP_BACKWARD_NO_INTERRUPT
  6. END_SEND
yield from x
  1. GET_ITER≠ prepares the object
  2. SEND
  3. YIELD_VALUE
  4. RESUME
  5. JUMP_BACKWARD_NO_INTERRUPT
  6. END_SEND
Fig 1.3Five releases on one line. The accent mark is Python 3.5, where async and await became keywords.

Coroutines grew out of generators, the functions that yield values one at a time. Generators gained send() in Python 2.5 (PEP 342, 2006-09-19) and yield from in 3.3 (PEP 380, 2012-09-29). asyncio shipped in 3.4 on 2014-03-16, written with yield from (PEP 3156, code name Tulip).

Python 3.5 made async and await keywords on 2015-09-13 (PEP 492, by Yury Selivanov). On 3.15 both forms still compile to the same loop of instructions: SEND, YIELD_VALUE, RESUME, then a jump back. What differs is the instruction that prepares the object: GET_AWAITABLE for await, GET_ITER for yield from.

STEP 04 / 05

AWAIT ALONE STILL WAITS IN LINE

urllib, one link after another24.59 s
version 0: blocking calls
async, one at a time24.49 s
version 1: async def and await, each link awaited before the next
0.1 s apart: the one thread still waits for each server in turn
Fig 1.4Two strips drawn to the same scale: the blocking script and the async one. They end 0.1 seconds apart.

The next version swaps urllib for aiohttp, makes check() a coroutine, and awaits each link before starting the next. asyncio.run(main()) creates the event loop that runs it, and main() starting is the first switch in the switch count.

It takes 24.49 seconds, 0.1 faster than the blocking version. Each await now pauses the coroutine while the server works, but nothing else runs during the pause: the script has only one coroutine in progress, so the thread waits exactly as before.

VERSION 1: ASYNC AND AWAIT, ONE LINK AT A TIME
import asyncio
import aiohttp

async def check(session, url):
    try:
        async with asyncio.timeout(5):
            async with session.get(url) as resp:
                await resp.read()
                return resp.status
    except (TimeoutError, aiohttp.ClientError):
        return None

async def main():
    urls = open("links.txt").read().split()
    async with aiohttp.ClientSession() as session:
        for url in urls:
            await check(session, url)    # this link finishes before the next starts

asyncio.run(main())                      # 24.49 s
STEP 05 / 05

ASYNC SPREADS UPWARD

  1. python check_links.pyordinary code: the interpreter starts here
  2. asyncio.run(main())the bridge: an ordinary call that runs a coroutine
  3. async def main()awaits check() for each link
  4. async def check(url)awaits the two network calls
  5. session.get(url)aiohttp: async
  6. resp.text()aiohttp: async
  7. requests.get(url)ordinary and blocking: it holds the loop’s thread (chapter 06)
Fig 1.5Accent boxes are async functions. Every path from them up to ordinary code passes through the asyncio.run bridge.

await is only legal inside async def; anywhere else it is a SyntaxError. So a function that awaits must itself be async, and so must every caller that waits for its result, up to one bridge from ordinary code: asyncio.run(). Libraries split the same way: requests blocks, aiohttp awaits.

Bob Nystrom named this split in 2015 in “What Color is Your Function?”, and async functions are often called since. The asyncio REPL, python -m asyncio (since 3.8), is another bridge: it runs a loop for you, so a top-level await works for experiments.

WHAT YOU NOW KNOW

  • Calling an async def function returns a 208-byte coroutine object and runs nothing; await is where it pauses and later continues.
  • Coroutines are generators with keywords: await compiles to the same SEND and YIELD_VALUE loop as yield from.
  • Awaiting each link in turn still took 24.49 s: pausing only helps when something else runs during the pause.
deep dive: coroutine states, awaitables, what asyncio.run does, and what await compiles to

Four states. inspect.getcoroutinestate() returns CORO_CREATED, CORO_RUNNING, CORO_SUSPENDED or CORO_CLOSED. coro.cr_frame.f_lineno is the paused line and coro.cr_await the object it is waiting on, which is how chapter 02 reads a paused stack.

What you can await. Coroutines, Task and Future objects, and any object with an __await__ method that returns an iterator. asyncio.sleep(0) is the simplest awaitable that pauses, and chapter 02 opens it up.

What asyncio.run does. It creates a new event loop and runs the coroutine to completion. Then it cancels any tasks still running, finishes async generators with shutdown_asyncgens(), and waits up to 300 seconds for the default thread pool to shut down (THREAD_JOIN_TIMEOUT in asyncio/constants.py). Since 3.14, asyncio.get_event_loop() with no loop running raises RuntimeError: There is no current event loop in thread 'MainThread'. instead of creating one; use asyncio.run or asyncio.Runner (3.11).

What await compiles to. RETURN_GENERATOR is the first instruction of every coroutine, and it is why calling one returns an object instead of running the body. For .pyc files, specialization and the JIT, see the interpreted-versus-compiled tutorial; this one needs only the instructions in step 3.

The loop that continues them

A coroutine can now pause at await, but pausing is half the job: something has to notice when the server has answered and run the coroutine again from that line. In asyncio that something is the , created by asyncio.run(). In the final run, continuing a coroutine after a network wait is the largest single cause of switches, and the switch count goes from 1 to 204.

STEP 01 / 05

ONE TURN OF THE LOOP

  1. timer heapsorted by due time; one 5 s timeout per link in a slot
    1. page-04 timeout · 5.00 s
    2. page-05 timeout · 5.00 s
    3. + 7 more
    4. page-17 timeout · 5.21 s
  2. select(timeout)kqueue, watching 10 sockets (page numbers)
    • 04
    • 05
    • 06
    • 09
    • 11
    • 13
    • 14
    • 15
    • 16
    • 17
  3. ready queuecallbacks waiting to run, oldest on the left
  4. run each callback oncehandle._run() for each, then back to the timer heap
  5. ↑ back to the timer heap: the next turn
nothing is ready, so select() may block until the earliest timer is due
Fig 2.1Follow the pointer around the four stations. One full lap is one turn; the final run made 677 of them.

The event loop is a method in plain Python, BaseEventLoop._run_once in Lib/asyncio/base_events.py (lines 1990–2074 on 3.15.2), called over and over. Each call is one turn: work out how long the selector may block, ask it which sockets are ready, move their callbacks and any due timers onto the ready queue, then run each ready callback once.

The event loop is a loop that runs whichever paused function has something to do, then asks the operating system which ones are ready next. It works like a chess master playing 40 boards at once: one player walks the room and moves only at the boards where the opponent has answered. The final run made 677 turns.

LIB/ASYNCIO/BASE_EVENTS.PY, TRIMMED
# BaseEventLoop._run_once, Python 3.15.2. Trimmed: cancelled-timer cleanup,
# the timeout cap and debug-mode timing are left out.
def _run_once(self):
    timeout = None
    if self._ready or self._stopping:
        timeout = 0                                     # work is waiting: no sleep
    elif self._scheduled:
        timeout = self._scheduled[0]._when - self.time()   # sleep until the next timer
    event_list = self._selector.select(timeout)         # kqueue here, epoll on Linux
    self._process_events(event_list)                    # ready sockets -> self._ready
    end_time = self.time() + self._clock_resolution
    while self._scheduled and self._scheduled[0]._when < end_time:
        handle = heapq.heappop(self._scheduled)         # due timers -> self._ready
        self._ready.append(handle)
    for _ in range(len(self._ready)):
        handle = self._ready.popleft()
        if not handle._cancelled:
            handle._run()                               # 677 turns in the final run
STEP 02 / 05

97% OF THE RUN IS ONE SYSTEM CALL

the loop’s one thread5.32 s
ticks sit where the run recorded a slot taken, a reply finished or a page parsed
  • waiting in kqueue5.17 s · 97%the dim bar: inside selector.select(), no CPU used
  • running Python0.15 s · 3%677 loop turns; the longest stretch between two waits was 4.4 ms
Fig 2.2The same thread strip as chapter 00, now 5.3 seconds long. The accent ticks are Python running.

Of the final run’s 5.32 seconds, the loop spent 5.169 inside selector.select(), blocked in kqueue until a socket had data or a timer was due. It ran Python for 0.149 seconds across 677 turns, and the longest stretch of Python between two waits was 4.4 ms.

A blocked kqueue call costs no CPU, exactly like the parked thread in chapter 00. The difference is what it waits on: every open socket at once, so whichever reply arrives first gets the next turn.

STEP 03 / 05

A FUTURE HOLDS A RESULT THAT ISN’T THERE YET

  1. socket readablemore of a page reaches the socket’s kernel buffer
  2. kqueue reports itselector.select() returns the socket to _run_once
  3. transport reads bytesits read callback calls sock.recv()
  4. protocol.data_received(data)aiohttp adds the bytes to its buffer
  5. future.set_result()the future aiohttp was awaiting goes from PENDING to FINISHED
  6. call_soon(task wake-up)the task’s callback joins the ready queue
  7. check() continues at line 27next turn: one send() resumes the coroutine
hops 3 to 6 run inside one callback in the same turn; the last waits for the next turn
Fig 2.3Follow the dot from the socket to the coroutine: six hops, and the loop runs every one of them.

When aiohttp needs the next bytes of a page, it creates a with loop.create_future() and awaits it. A future is an object with a state (PENDING, FINISHED or CANCELLED), a slot for the eventual result, and a list of callbacks. The task awaiting it, the object that runs one coroutine for the loop and chapter 03’s subject, pauses.

When the socket becomes readable, the loop runs the transport’s read callback, which reads the bytes and passes them to aiohttp’s data_received. aiohttp stores them and calls set_result() on the waiting future, which schedules the task’s wake-up callback with call_soon. On the next turn that callback continues the coroutine.

STEP 04 / 05

ONE RESUME, SIX FRAMES

  1. check()check_links.py:26async with session.get(url) as resp
  2. _BaseRequestContextManager.__aenter__()aiohttp/client.py:1756
  3. ClientSession._request()aiohttp/client.py:900
  4. _connect_and_send_request()aiohttp/client.py:878
  5. ClientResponse.start()aiohttp/client_reqrep.py:745
  6. DataQueue.read()aiohttp/streams.py:716await self._waiter: a future, still PENDING, waiting for the status line
Task-19 · page-17at 1.0 s into the run
awaited byTask-1 · main()in TaskGroup.__aexit__
58 of 100tasks still running at 1.0 s
Fig 2.4Read the stack top to bottom. The accent frame at the bottom is where page-17 is actually waiting.

At 1.0 s into the run, 58 of the 100 tasks were still running. asyncio.print_call_graph() (since 3.14) printed page-17’s task, Task-19, paused six frames deep. A frame is one function call in progress, with its locals and current line: check() on top, then five aiohttp frames down to DataQueue.read(), waiting for a status line.

Each of those frames is a coroutine awaiting the one below it. When the future at the bottom is set, the task makes one send() call into check(), and Python passes it down the chain to DataQueue.read(). Each frame then continues in turn as results come back up.

Network waits account for 203 switches: 118 at session.get, 84 at resp.text(), and 1 when the session closed its connections.

STEP 05 / 05

SLEEP(0) IS A BARE YIELD

await asyncio.sleep(0.1)a future and a timer, due 0.1 s later
  1. loop.create_future()the task pauses on it
  2. timer heapcall_later(0.1): due in 0.1 s
  3. ready queuethe timer fires and sets the future
  4. task continueson the turn after
await asyncio.sleep(0)no future and no timer · 7.14 µs a round trip
  1. __sleep0()one bare yield
  2. timer heapskipped
  3. ready queuethe task’s next step goes to the back
  4. task continuesafter everything already queued
Fig 2.5Two paths through the loop. sleep(0.1) goes via the timer heap; the accent sleep(0) goes straight back to the ready queue.

await asyncio.sleep(0.1) creates a future and puts a timer in the heap, due 0.1 s later; when the timer fires it sets the future, and the task continues. await asyncio.sleep(0) is special-cased in tasks.py: it awaits __sleep0(), a generator that runs one bare yield, with no future and no timer.

The bare yield hands control straight back to the task, which puts its own next step at the back of the ready queue, so every callback already queued runs first. It is the standard way to let other tasks run inside a long loop, and one round trip cost 7.14 µs in isolation.

WHAT YOU NOW KNOW

  • The event loop is _run_once in base_events.py, run 677 times: wait in the selector, queue what is ready, run each callback once.
  • A future holds a result that is not there yet; set_result schedules the waiting task, which continues its coroutine on the next turn.
  • Network waits made 203 of the switches, and the thread spent 97% of the run waiting in kqueue.
deep dive: selectors on every OS, streams, uvloop, the policy system going away, and what is written in C

Selectors on each OS. selectors.DefaultSelector tries kqueue, then epoll, devpoll, poll and select. On macOS it is KqueueSelector, on Linux epoll. Windows has used ProactorEventLoop by default since 3.8, built on I/O completion ports: the OS reports finished reads instead of ready sockets, which is why some socket APIs behave differently there.

Transports, protocols and streams. The loop works in callbacks such as data_received. aiohttp and asyncio.open_connection wrap those callbacks in streams you can await. The streams API (reader.read(), writer.drain()) is what most code should use.

uvloop. A drop-in event loop built on libuv, version 0.23.0 with wheels for 3.15 and 3.15t. You install it with asyncio.run(main(), loop_factory=uvloop.new_event_loop) (loop_factory since 3.12). The project claims asyncio code runs 2–4 times faster on it; that is its claim, not a measurement here.

Policies are going away. Event loop policies were deprecated in 3.14 and are scheduled for removal in 3.16, and loop_factory replaces set_event_loop_policy. asyncio.iscoroutinefunction goes in 3.16 too; use inspect.iscoroutinefunction.

What is written in C. Task, Future and the current-task bookkeeping live in the C module _asyncio; the loop itself is Python. Python 3.14 moved task tracking to a per-thread linked list, which made the standard asyncio benchmarks 10–20% faster and uses less memory.

How this tutorial counts switches. sys.monitoring (since 3.12) reports PY_START, PY_RESUME and PY_THROW events for the script’s own coroutines, and each one is a switch. The finale shows the lines that do it.

100 checks in one group

The loop can continue any paused coroutine whose data has arrived, but the script still hands it one link and waits for the answer. To overlap the waits, each link has to become its own job that the loop runs independently: a task. Made into 100 tasks, the same checks take 5.01 seconds instead of 24.49, and the switch count goes from 204 to 305.

STEP 01 / 05

CREATE_TASK STARTS THE CLOCK

  • one link’s task, waiting for its server
page-17 · no answer · runs to 5.01 s
100 tasks, started together
page-00page-99
last good link · 0.733 s
chapter 00, one at a time · 24.59 s
this run ends · 5.01 s
Fig 3.1Same 100 rows as the staircase in chapter 00, now starting together. The accent row is page-17 again.

asyncio.create_task(coro) wraps a coroutine in a and puts its first step on the ready queue. create_task returns at once, and the coroutine starts on the loop’s next turn. From then on the loop continues the task each time what it awaits is ready, and the code that created it carries on without waiting.

With all 100 links made into tasks, the 100 waits overlap: 100 requests are in flight at the peak, and the last good link is done at 0.733 seconds. The run takes 5.01 seconds instead of 24.49. The remaining 4.3 seconds are page-17 waiting out its 5-second timeout.

STEP 02 / 05

THE BLOCK DOESN’T END UNTIL THEY DO

async def main():
async with asyncio.TaskGroup() as tg:tg.create_task(check(session, url)) × 100
one chip per task · the hollow chip is page-17, the last to finish, at 5.01 s
end of the async with blockwaits for all 100 tasks · then main continues
print(…, "ok") · runs only after the block
  • each task starts once100 switches
  • main continues after the block1 switch
Fig 3.2The group’s box sits inside main’s box. The accent line is the block’s exit, where main waits for all 100.

Since Python 3.11, async with asyncio.TaskGroup() as tg: gives tasks an owner. A holds every task made with tg.create_task. Reaching the end of the block awaits all of them, so the function that opened the block cannot continue past it while any of its tasks is running.

This is : the indentation of the code shows how long every task can run. The idea comes from the Trio library’s “nursery” (Trio 0.1.0, 2017-03-11) and Nathaniel J. Smith’s essay “Notes on structured concurrency, or: Go statement considered harmful” (2018-04-25). PEP 654 credits Trio, and asyncio’s class was adapted from EdgeDB.

The 100 task starts add 100 switches, one per link. main continuing after the block adds 1, at the moment the last task finishes.

VERSION 2: 100 TASKS IN A GROUP · 5.01 S
import asyncio
import aiohttp

async def main():
    urls = open("links.txt").read().split()
    async with aiohttp.ClientSession() as session:
        async with asyncio.TaskGroup() as tg:
            tasks = [tg.create_task(check(session, url)) for url in urls]
        # the block above ends only when all 100 tasks are done
    print(sum(t.result() == 200 for t in tasks), "ok")    # 96 ok, 5.01 s

asyncio.run(main())
STEP 03 / 05

A TASK NOBODY HOLDS CAN DISAPPEAR

event loopdashed: a weak reference, which does not stop the garbage collector freeing the task
TaskGroup tgsolid: a strong reference, which keeps the task alive until it is done
  • check(page-00)made with tg.create_task()
    • event loopweak reference
    • tgstrong reference
  • check(page-01)made with tg.create_task()
    • event loopweak reference
    • tgstrong reference
  • send_report()made with asyncio.create_task(), result not saved
    • event loopweak reference
    may be garbage-collected before it finishes
Fig 3.3Solid arrows keep a task alive; the dashed arrow from the loop does not. The accent task has only the dashed one.

The event loop keeps only a weak reference to each task: a reference that does not stop Python’s garbage collector from freeing the object. The create_task documentation says so directly: “Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks.”

So a fire-and-forget asyncio.create_task(send_report()), with no variable holding the result, can be freed before its coroutine reaches the end, and asyncio logs Task was destroyed but it is pending!. A TaskGroup keeps a strong reference to every task until it is done, which is one more reason to create tasks through one.

STEP 04 / 05

A WAITING TASK COSTS ABOUT A KILOBYTE

  • one coroutine object208 B
  • one waiting task, with its timer and future1,143 B
  • this script’s 100 tasks114 KB
  • 100,000 waiting tasks114.4 MB
log scale: each tick is 100 times the one before
Fig 3.4Bars drawn to scale in bytes. The accent bar is this script’s 100 tasks; the long bar is 100,000.

A coroutine object is 208 bytes. 100,000 tasks, each paused in asyncio.sleep(10), took 114.4 MB by tracemalloc, which counts every allocation Python makes. That is about 1,143 bytes per task, with its timer and the future it waits on.

Creating a task that returns at once and running it to the end took 2.73 µs. The 100 tasks in this script cost about 114 KB, which is why one thread can keep tens of thousands of tasks waiting at the same time.

STEP 05 / 05

EACH TASK CARRIES ITS OWN CONTEXT

main() · Task-1link.get() returns '-'
  • link in its context'-', the default
↓ context copied at each create_task
  • check(0) · Task-2link.set('page-00')
    • link'page-00'
  • check(1) · Task-3link.set('page-01')
    • link'page-01'
  • check(2) · Task-4link.set('page-02')
    • link'page-02'
Fig 3.5Three task cards, one value each. The accent card is main, which still sees the default.

A (contextvars.ContextVar, PEP 567, since 3.7) is the async replacement for thread-local storage. Each task runs in a copy of the context taken when it was created, so a value one task sets, such as a request ID or the current link, is invisible to the other tasks and to main.

In the sample, three tasks each set link and read it back after pauses that finish them in reverse order. They print ['page-00', 'page-01', 'page-02'], and main still sees the default, '-'. Logging and tracing libraries use this to label every line with the task that wrote it.

ONE VALUE PER TASK
import asyncio
from contextvars import ContextVar

link = ContextVar("link", default="-")

async def check(n):
    link.set(f"page-{n:02d}")            # visible only inside this task
    await asyncio.sleep(0.01 * (3 - n))  # finish in reverse order
    return link.get()

async def main():
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(check(n)) for n in range(3)]
    print([t.result() for t in tasks], link.get())
    # ['page-00', 'page-01', 'page-02'] -

asyncio.run(main())

WHAT YOU NOW KNOW

  • A task wraps a coroutine and starts it on the next turn; 100 tasks overlap their waits, cutting the run from 24.49 s to 5.01 s.
  • A TaskGroup’s async with block ends only when every task in it has finished, and it holds a strong reference to each task until then.
  • A waiting task costs about 1.1 KB, so 100,000 fit in 114.4 MB, and each task runs in its own copy of the context variables.
deep dive: gather and as_completed, eager tasks, task names, and inspecting a live program

gather and as_completed. asyncio.gather(*coros) is the older way to run several coroutines and is still fine. It returns the results in the order the coroutines were passed, but it does not cancel the others when one fails. With return_exceptions=True it returns exceptions as values instead of raising the first one. asyncio.as_completed() yields results in the order they finish and, since 3.13, also works with async for.

Eager tasks (3.12). loop.set_task_factory(asyncio.eager_task_factory) runs a new task’s coroutine immediately, inside the call to create_task, until its first real pause. A task that never pauses is already done when create_task returns: task.done() is True with the eager factory and False without it. That saves a loop turn when many calls are answered from a cache. It is not the default in 3.15.

Names and arguments. Tasks are named Task-1, Task-2 and so on by default, which is why page-17’s task is Task-19. Pass name= to make logs readable. Since 3.14, create_task passes extra keyword arguments on to a custom task factory.

Looking at a live program (3.14). asyncio.capture_call_graph() and asyncio.print_call_graph() work inside the program with no special rights. From outside, python -m asyncio ps PID prints a table of every task and what it awaits, and python -m asyncio pstree PID prints the tree of awaits. Both read another process’s memory, so macOS asks for root and Linux needs ptrace permission; that is why this tutorial shows no output from them.

One refused, one silent

Four of the 100 links are broken. page-42 returns 404 and page-88 returns 500, which are ordinary answers. page-63 refuses the connection, which raises an exception, and page-17 accepts the request and never answers at all. Inside a TaskGroup the last two are the dangerous ones. Handled inside each check, they add 2 switches, one exception thrown into a paused coroutine each, and the switch count goes from 305 to 307.

STEP 01 / 05

ONE REFUSED CONNECTION CANCELS 99 CHECKS

  • one link’s task, waiting until it is cancelled
page-63 · ClientConnectorError at 14 ms
page-00page-99
99 tasks cancelled at 14 ms
ExceptionGroup( 'unhandled errors in a TaskGroup', [ClientConnectorError(…)])100 tasks: 1 failed · 99 cancelled · 0 finished
Fig 4.1All 100 rows stop at the same moment, 14 ms in. The accent row is page-63, the one that failed.

If check lets exceptions escape, page-63’s refused connection raises ClientConnectorError 14 ms into the run. A TaskGroup treats the first exception other than CancelledError as fatal to the group: it cancels the other 99 tasks, waits for them to finish, and raises an at the end of its async with block.

The group reaches main as ExceptionGroup('unhandled errors in a TaskGroup', [ClientConnectorError(…)]), and 0 links were checked. except* (PEP 654, since 3.11) catches by type inside the group, so one clause can handle the connection errors while any other kind keeps propagating.

LETTING ERRORS ESCAPE · 14 MS
import asyncio
import aiohttp

async def check(session, url):
    async with session.get(url) as resp:     # page-63: ClientConnectorError
        resp.raise_for_status()
        await resp.read()

async def main():
    urls = open("links.txt").read().split()
    try:
        async with aiohttp.ClientSession() as session:
            async with asyncio.TaskGroup() as tg:
                tasks = [tg.create_task(check(session, url)) for url in urls]
    except* aiohttp.ClientError as group:
        print(group)        # unhandled errors in a TaskGroup (1 sub-exception)
        print(sum(t.cancelled() for t in tasks), "cancelled")   # 99, after 14 ms

asyncio.run(main())
STEP 02 / 05

CANCELLING MEANS THROWING AN EXCEPTION IN

check(session, limit, pool, 'page-17')CORO_SUSPENDEDpaused at line 26: async with session.get(url) as resp: ← CancelledError thrown in here
  • url…/page-17.html
  • thrown bythe 5-second timeout
on the way out, innermost first
  1. 1async with asyncio.timeout(5):its exit turns the CancelledError into TimeoutError
  2. 2except TimeoutError:return url, "timeout", 0
  3. 3async with limit:its exit releases the slot, so the next link can start
a cancel() from outside passes through step 1 as CancelledError, and every exit above still runs
Fig 4.2The exception enters at the accent line, where the coroutine paused, and leaves through every async with above it.

is an exception thrown into a paused coroutine. task.cancel() makes the task continue its coroutine one more time, with CancelledError raised at the await where it paused. The exception then travels up like any other, so finally: blocks and async with exits run on the way out.

Since 3.8 CancelledError is a BaseException, so except Exception: does not catch it. Catching it without re-raising breaks TaskGroup and timeouts, which both need it to reach them. A failed await arrives the same way: the task throws the future’s exception in. Both of the final run’s 2 thrown-in switches happened at session.get.

STEP 03 / 05

CATCH IT WHERE IT HAPPENS

errors escape the checks
  • 0links checked
  • 99checks cancelled
  • 14 msuntil the group raised
failures caught inside each check
  • 96links ok
  • 5.23 sfor the whole run
  • timeoutpage-17
  • 404page-42
  • ClientConnectorErrorpage-63
  • 500page-88
4 broken links, reported as results
Fig 4.3Left, the group that stopped after 14 ms. Right, the same 100 links with failures caught inside each check.

The fix is to let each check handle the failures it expects and return a result instead. In the final check, TimeoutError becomes "timeout", any aiohttp.ClientError becomes the exception’s class name, and a status other than 200 is reported, not raised. The group then sees only real bugs.

The final run reports 96 good links and 4 broken ones in 5.23 seconds. The only cancellation left is page-17’s, and its own timeout catches it inside the check.

THE FINAL CHECK()
async def check(session, limit, pool, url):
    async with limit:                            # one of 10 slots
        try:
            async with asyncio.timeout(5):       # counts only time inside the slot
                async with session.get(url) as resp:
                    body = await resp.text()
        except TimeoutError:
            return url, "timeout", 0             # page-17
        except aiohttp.ClientError as exc:
            return url, type(exc).__name__, 0    # page-63: ClientConnectorError
    if resp.status != 200:
        return url, str(resp.status), 0          # page-42 and page-88
    loop = asyncio.get_running_loop()
    links = await loop.run_in_executor(pool, count_links, body)
    return url, "200", links
STEP 04 / 05

AIOHTTP WOULD HAVE WAITED FIVE MINUTES

  • one link in a slot, waiting for its server
last good link done · 2.291 s
page-17 · slot 3 · from 0.209 s to its timeout at 5.212 s
  1. slot 1
  2. slot 2
  3. slot 3
  4. slot 4
  5. slot 5
  6. slot 6
  7. slot 7
  8. slot 8
  9. slot 9
  10. slot 10
2.9 s waiting for one link
Fig 4.4Every row ends by 2.3 seconds except the accent row, page-17, which runs until its timeout at 5.21 s.

page-17 accepts the connection and never sends a status line. aiohttp’s default ClientTimeout allows 300 seconds for the whole request, while httpx defaults to 5 seconds. So the script sets its own with async with asyncio.timeout(5) (since 3.11), placed inside the slot so that time spent queueing does not count.

When 5 seconds pass, the timeout cancels the task. When the block exits, it turns that CancelledError into TimeoutError, using Task.uncancel() to tell its own cancellation from anyone else’s.

page-17 took its slot at 0.209 s and timed out at 5.212 s. All 96 good links had finished by 2.291 s, so for the final 2.9 seconds the loop has nothing to run until page-17’s timer is due.

STEP 05 / 05

TASKGROUP.CANCEL(), NEW IN 3.15

async def main():
async with asyncio.TaskGroup() as tg:tg.create_task(check(session, url)) × 100
the sample’s 100 tasks · filled: 17 finished · hollow: 83 cancelled
an exception escapes a task↓ the group cancels the rest↓ ExceptionGroup raised in mainas in step 1: 99 of 100 cancelled in 14 ms
tg.cancel()↓ the group cancels the rest↓ the block exits, nothing raised83 of 100 cancelled
Fig 4.5Two ways out of the same group. The accent exit is tg.cancel(): no exception reaches main.

Sometimes a program stops the group on purpose, for example after the third broken link. Python 3.15 adds TaskGroup.cancel(): it cancels every task still in the group, and the async with block exits without raising. In the sample the third broken link arrives at 0.17 s, and 83 of the 100 tasks are cancelled. Before 3.15 this took a custom exception, raised inside the group and caught outside the block.

Ctrl-C works the same way since 3.11. The first press cancels the main task, so the group cancels its tasks and every finally: block runs. A second press raises KeyboardInterrupt at once.

TASKGROUP.CANCEL(), PYTHON 3.15
import asyncio

broken = 0

async def check(tg, n):
    global broken
    await asyncio.sleep(0.01 * n)
    if n % 7 == 3:                   # links 3, 10 and 17 are broken
        broken += 1
        if broken == 3:
            tg.cancel()              # Python 3.15: stop the group, raise nothing

async def main():
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(check(tg, n)) for n in range(100)]
    print(sum(t.cancelled() for t in tasks), "of 100 cancelled")   # 83 of 100

asyncio.run(main())

WHAT YOU NOW KNOW

  • In a TaskGroup the first escaping exception cancels everything else: one refused connection stopped all 100 checks after 14 ms.
  • Cancelling continues a coroutine with CancelledError thrown in at its paused line, so cleanup runs; never swallow it.
  • asyncio.timeout(5) turned page-17’s silence into TimeoutError; the run still waits 2.9 s for it after every other link is done.
deep dive: uncancel and shield, where to put a timeout, wait_for, except* rules, and what asyncio.run does at exit

Who cancelled this task? Task.cancelling() counts the pending cancellation requests, and Task.uncancel() removes one. That is how asyncio.timeout() and TaskGroup tell their own cancellations from one that came from outside. asyncio.shield(aw) protects an inner awaitable from an outer cancellation: the inner one keeps running, while the outer await still raises CancelledError.

Where to put the timeout. In check_links.py the timeout sits inside async with limit, so time spent waiting for a slot does not count against the 5 seconds. Put it outside, and a link that queued for 2 seconds would get only 3 of its own. asyncio.timeout_at() takes an absolute deadline instead, and the context manager’s reschedule() moves the deadline while the block runs.

wait_for. asyncio.wait_for(aw, 5) is the older form of the same limit. Since 3.12 it is implemented with asyncio.timeout(), so the two behave the same way on cancellation.

except* rules. Each except* clause takes the matching sub-exceptions out of the group, and anything left over is re-raised as a smaller group. KeyboardInterrupt and SystemExit escaping a task still cancel the group, but they are re-raised as themselves rather than inside a group.

At exit. asyncio.run cancels every task still running and gathers them with return_exceptions=True, so a forgotten background task receives CancelledError at shutdown.

Ten at a time

The script now starts all 100 checks in the same millisecond, and nothing stops them from reaching the server together. Without a limit it opens 99 TCP connections to one server in its first few milliseconds, which a real documentation site might rate-limit or block. The fix is to let at most 10 links be in flight at once. 90 of the 100 tasks then start by waiting for a slot, and the switch count goes from 307 to 397.

STEP 01 / 05

A SEMAPHORE IS A COUNTER WITH A QUEUE

Semaphore(10) · 10 slotsfree slots · 0
0 ms · the first 10 tasks hold every slotin the queue · 90 tasks
Fig 5.1When a slot frees up, the queue’s oldest task moves into it. The accent chip is the next to move.

asyncio.Semaphore(10) is a : a counter that starts at 10, plus a queue of waiting tasks. async with limit: takes one from the counter on the way in and gives it back on the way out. At 0, the task pauses on a future in the semaphore’s queue, and each release continues the oldest waiter.

In the final run the first 10 tasks got a slot at once. The 11th got one at 78 ms, when page-00’s request finished. 90 of the 100 tasks waited, the longest for 2.03 seconds, and each wait ended in one switch when the semaphore continued its task: 90 switches.

STEP 02 / 05

THE LIMIT COST 0.2 SECONDS

  • one link holding the slot until its request ends
  • page-17, waiting for its 5 s timeout
Fig 5.2Ten rows, one per slot, drawn to scale. The accent segment is page-17, holding its slot for 5 of the 5.2 seconds.

With 10 slots the run takes 5.23 seconds instead of 5.01. The good links take longer: the last one finishes at 2.291 seconds instead of 0.733, because each slot now serves about ten links one after another.

The total barely moves because page-17 sets it. It gets a slot at 0.209 seconds and holds it until its 5-second timeout fires at 5.212. The other nine slots finish every remaining link long before that, and then sit empty.

STEP 03 / 05

AIOHTTP ALREADY HAD A LIMIT OF 100

  • no limit99 TCP connections
    peak 100 requests in flight; page-63 was refused, so it never opened one
  • Semaphore(10)10 TCP connections
    served all 96 good pages: each connection was reused
Fig 5.3Two counts of TCP connections opened. The accent bar is the limited run: 10 connections for 96 pages.

aiohttp keeps a connection pool: the open TCP connections a session can reuse. Its TCPConnector defaults to limit=100 connections per session and limit_per_host=0, which means no limit per host. Without the semaphore, the run peaked at exactly 100 requests in flight and opened 99 TCP connections; page-63’s was refused, so it never opened.

With the semaphore, 10 connections served all 96 good links. When a response has been read, aiohttp puts its connection back in the pool instead of closing it, and the next request to the same host reuses it. That is HTTP keep-alive, and aiohttp keeps an idle connection open for 15 seconds by default.

STEP 04 / 05

A QUEUE MAKES THE LIMIT PART OF THE DESIGN

  1. 100 linksmain() calls await queue.put(url) for each
  2. Queue · maxsize 2020 of 20 fullput() pauses here when full
  3. 10 workersat most 10 links in flighteach loops: get(), check the link, task_done()
  4. results100 checkedqueue.shutdown() then ends all 10 workers
Fig 5.4Read left to right: links into a queue of 20, out to 10 workers. The accent box is the full queue.

The other standard shape for a limit is a pool of workers: 10 worker tasks take links from an asyncio.Queue(maxsize=20). Each worker loops on await queue.get(), checks the link and asks for the next. The number of workers is the limit, so no semaphore is needed.

put() pauses while the queue already holds 20 links. That is backpressure: the code adding work can go only as fast as the workers take it, so memory stays bounded however long the input is. Since 3.13, queue.shutdown() makes a get() on the empty queue raise QueueShutDown, so each worker returns. The sample checks all 100.

TEN WORKERS AND A QUEUE, PYTHON 3.13+
import asyncio

async def worker(queue, done):
    while True:
        try:
            url = await queue.get()          # pauses while the queue is empty
        except asyncio.QueueShutDown:        # Python 3.13+
            return
        await check_one(url)
        done.append(url)
        queue.task_done()

async def main(urls):
    queue, done = asyncio.Queue(maxsize=20), []
    async with asyncio.TaskGroup() as tg:
        for _ in range(10):
            tg.create_task(worker(queue, done))
        for url in urls:
            await queue.put(url)             # pauses while 20 are waiting
        await queue.join()
        queue.shutdown()
    print(len(done), "checked")              # 100 checked
STEP 05 / 05

ASYNC WITH CAN PAUSE TWICE

what you write
  1. async with limit:
  2. # the request, inside the slot
one line, no visible await
what Python runs
  1. await limit.__aenter__()pauses here while all 10 slots are taken
  2. # the request, inside the slot
  3. await limit.__aexit__(None, None, None)can pause too; a semaphore’s release never does
trimmed: on an error, __aexit__ gets the exception instead of None
Fig 5.5The two columns say the same thing. The accent lines are the two places the left column can pause.

async with limit: runs await limit.__aenter__() on the way in and await limit.__aexit__(None, None, None) on the way out. Both are awaits, so the line itself can pause the coroutine: the 90 slot waits happened inside __aenter__. Any object with those two coroutine methods is an async context manager, and works in an async with.

Loops get the same treatment. An async iterator is an object whose __anext__() is a coroutine, so async for chunk in resp.content.iter_chunked(65536) can pause between items while the next chunk arrives. An async generator, a function with both async def and yield (PEP 525, since 3.6), is the short way to write one.

WHAT YOU NOW KNOW

  • Semaphore(10) let 10 checks run at once; 90 waited for a slot, the longest for 2.03 s, adding 90 switches.
  • The good links finished by 2.291 s instead of 0.733 s; page-17’s timeout still ends the run, at 5.23 s instead of 5.01.
  • aiohttp reused 10 TCP connections for all 96 good pages; a queue with 10 workers is the other way to build the same limit.
deep dive: Lock, Event, Condition and Barrier, rate limits over time, per-host limits, and fairness

The other primitives. asyncio.Lock, Event, Condition, BoundedSemaphore and Barrier (3.11) mirror the ones in threading, but they are not thread-safe: use them only from coroutines on one loop. Since 3.10 none of them takes a loop argument, so creating one before the loop starts is fine.

A limit on concurrency is not a rate limit. A semaphore caps how many requests run at once. A server that allows 5 requests per second needs a limit over time, such as the leaky bucket in the aiolimiter package: AsyncLimiter(5, 1) lets 5 through per second.

Per host. For a crawler that visits many sites, aiohttp.TCPConnector(limit_per_host=4) caps the connections to each host without a semaphore per host.

Fairness. The semaphore continues its waiters oldest first, which is why the slot rows in step 2 fill in link order: page-10 got the first free slot, page-11 the second.

One call that stops the loop

The checker now waits for every page with an await, and that is the promise everything so far relies on: a coroutine pauses at an await while it waits. Ordinary libraries such as urllib, requests, sqlite3 and time.sleep make no such promise, and one call to any of them holds the loop’s only thread until it returns. The final script has one such call, its SQLite save, and moves it to a worker thread. That adds 1 switch, and the switch count goes from 397 to 398.

STEP 01 / 05

24.6 SECONDS AGAIN

async def + urllib, 100 tasks · 24.61 spage-17 · Task-19 · 5.002 s in one step
  • one task’s first step, holding the thread until its page is read
page-00page-99
Fig 6.1The staircase from chapter 00 is back, inside a TaskGroup. The accent row is the step debug mode will flag.

Put urllib.request.urlopen inside async def check, start 100 tasks in a TaskGroup, and the run takes 24.61 seconds, as slow as the plain script. urlopen is a : it holds the thread until its page has arrived, and nothing inside it is an await.

So each task’s first step runs to the end without pausing. The first task holds the thread until its page is read, then the second does, and so on. The 100 tasks run one after another, and the loop never gets back to select() in between to look at the other sockets.

STEP 02 / 05

DEBUG MODE TIMES EVERY STEP

  1. Executing <Task finished name='Task-3' coro=<check() done, defined at check_links_blocking.py:11> result=('http://127.0.../page-01.html', 200) …> took 0.177 seconds
  2. Executing <Task finished name='Task-5' coro=<check() done, defined at check_links_blocking.py:11> result=('http://127.0.../page-03.html', 200) …> took 0.206 seconds
  3. Executing <Task finished name='Task-7' coro=<check() done, defined at check_links_blocking.py:11> result=('http://127.0.../page-05.html', 200) …> took 0.449 seconds
  4. Executing <Task finished name='Task-19' coro=<check() done, defined at check_links_blocking.py:11> result=('http://127.0.../page-17.html', 'TimeoutError') …> took 5.002 seconds
  5. … 82 such lines in all, 4 shown
Fig 6.2Real log lines from the blocking run. The accent line is page-17, holding the loop for 5 seconds.

Debug mode is asyncio’s own checking mode, turned on by asyncio.run(main(), debug=True), PYTHONASYNCIODEBUG=1 or python -X dev. It times every callback the loop runs and logs any that takes 100 ms or more (the loop’s slow_callback_duration is 0.1 seconds), as a line ending in took 0.177 seconds.

The blocking version logs 82 such lines. The worst says took 5.002 seconds, and its task is Task-19, page-17. Debug mode also reports coroutines that were never awaited, with the line that created them, and loop methods called from the wrong thread.

DEBUG MODE ON THE BLOCKING VERSION
PYTHONASYNCIODEBUG=1 python check_links_blocking.py
# Executing <Task finished name='Task-3' coro=<check() done, defined at
#   check_links_blocking.py:11> result=('http://127.0.../page-01.html', 200)
#   created at asyncio/taskgroups.py:220> took 0.177 seconds
# and 81 more such lines; the worst is page-17:
#   <Task finished name='Task-19' ...> took 5.002 seconds
STEP 03 / 05

TO_THREAD HANDS THE WAIT TO A WORKER

loop thread · MainThreadin select(), free to run other tasks
↓ to_thread(save, results)↑ result: the await returns
worker thread · asyncio_01 of 16 in the default pool
5.212 s · page-17 times out, the group ends5.23 s · the report prints
the run’s last 18 ms; the save was not timed on its own, so the block’s position and length are not to scale
Fig 6.3Two strips: the loop thread and one worker. The accent block is the SQLite save, on the worker.

asyncio.to_thread(fn, *args), since 3.9, runs a blocking function in the loop’s default and returns an awaitable. The coroutine pauses on it like on any other await. The loop thread goes back to running other tasks while a worker thread sits in the blocking call.

The pool has min(32, os.process_cpu_count() + 4) threads, the formula since 3.13: 16 on this 12-core machine, named with the prefix asyncio. The final script saves its 100 results to SQLite this way, which is 1 switch. Reading links.txt before any task starts needs no thread, because nothing else is waiting yet.

SAVING THE REPORT WITHOUT BLOCKING THE LOOP
import asyncio
import sqlite3

def save(results):                       # ordinary, blocking code
    with sqlite3.connect("report.db") as db:
        db.execute("DROP TABLE IF EXISTS links")
        db.execute("CREATE TABLE links (url TEXT, status TEXT, links INTEGER)")
        db.executemany("INSERT INTO links VALUES (?, ?, ?)", results)

async def finish(results):
    await asyncio.to_thread(save, results)   # runs on a pool thread; the loop keeps going
STEP 04 / 05

ASYNCIO USES THREADS FOR DNS

  1. session.get(url)aiohttp needs a connection to the URL’s host
  2. loop.create_connection(host, port)asyncio opens the TCP socket
  3. loop.getaddrinfo(host, port)skipped when the host is already an IP address, as 127.0.0.1 is here
  4. the loop’s default thread poolthe lookup leaves the loop thread; the coroutine awaits a future
  5. socket.getaddrinfo(host, port)the C library’s lookup, which blocks its worker thread until DNS answers
Fig 6.4Follow the connect call down: the accent step leaves the loop thread for a worker.

Turning a hostname into an IP address is a DNS lookup, and the C library offers no non-blocking way to do one. So loop.getaddrinfo() runs the blocking socket.getaddrinfo in the same default thread pool, as base_events.py shows, and the connecting coroutine awaits the result.

This run never takes that path: every link points at 127.0.0.1, an IP address that needs no lookup. A crawler of real sites takes it for each new host. aiohttp caches each answer for 10 seconds by default (ttl_dns_cache=10), so repeat requests to one host within that window skip the thread.

STEP 05 / 05

LOOKING INSIDE A RUNNING PROGRAM

  • debug mode, asyncio.print_call_graph()call graph since 3.14inside the programno extra permission
  • python -m asyncio ps / pstree PIDsince 3.14from another processroot on macOS, ptrace on Linux
  • python -m profiling.sampling run --async-awaresince 3.15, PEP 799from another processroot on macOS, ptrace on Linux
Fig 6.5Three tools in rows. The accent row runs inside the program and needs no extra permission.

Three tools show what a running asyncio program is doing, sorted by how much access they need. Debug mode and asyncio.print_call_graph() (3.14) run inside the program itself. python -m asyncio ps PID and pstree PID (3.14) attach from outside and print every task and what it awaits.

Python 3.15 adds a sampling profiler, python -m profiling.sampling run --async-aware (PEP 799, nicknamed Tachyon), which rebuilds stacks per task and can take up to 1,000,000 samples a second. The last two read another process’s memory, so on macOS they need root, and on Linux they need ptrace permission.

LOOKING AT A RUNNING CHECKER
# inside the program, no extra permission
PYTHONASYNCIODEBUG=1 python check_links.py
# another process (Python 3.14+): every task and what it awaits
sudo python -m asyncio ps 48213
sudo python -m asyncio pstree 48213
# the Python 3.15 sampling profiler, stacks rebuilt per task
sudo python -m profiling.sampling run --async-aware --flamegraph -o checker.html check_links.py

WHAT YOU NOW KNOW

  • One blocking call inside a coroutine holds the loop’s only thread: 100 such tasks took 24.61 s, as slow as no async at all.
  • Debug mode flags every step of 100 ms or more; it logged 82 of them, the worst 5.002 s.
  • asyncio.to_thread moves a blocking call to one of 16 worker threads; asyncio itself does that for every DNS lookup.
deep dive: async libraries and the threads inside them, calling the loop from other threads, and run_in_executor

Async libraries and their threads. HTTP has aiohttp and httpx (httpx is still at 0.28.1 from December 2024, with 1.0 in development and a Pydantic-maintained fork, httpx2, since May 2026). PostgreSQL has asyncpg and psycopg 3, and Redis has redis.asyncio. aiosqlite and aiofiles run the blocking calls in threads for you, because operating systems offer no readiness events for files on disk the way they do for sockets.

From other threads into the loop. asyncio.run_coroutine_threadsafe(coro, loop) submits a coroutine from another thread and returns a concurrent.futures.Future. loop.call_soon_threadsafe(fn) schedules a plain callback. Calling loop.call_soon from another thread is a bug, and debug mode reports it.

run_in_executor. loop.run_in_executor(pool, fn, *args) is the general form of to_thread: it takes any executor, including the interpreter pool of chapter 07. to_thread also copies the current context variables into the thread; run_in_executor does not.

Why not use threads everywhere? For 100 links, threads work: 5.13 seconds in the chapter 00 deep dive. The concurrency tutorial measures where they stop working, at 10,000 clients.

Work for more than one core

The checker now also counts the links on every page that answers, ready to crawl them next. That is CPU-bound work: the processor is busy the whole time and nothing is waited for. It covers 96 pages, 22.7 MB of HTML and 76,124 links, and takes 0.48 seconds on one core. Run on the loop thread, each parse sits between two awaits and keeps the thread until it returns. Handed to an interpreter pool instead, each count comes back through an await, and the switch count goes from 398 to 494.

STEP 01 / 05

ONE PAGE HOLDS THE LOOP FOR 128 MS

the loop thread, PARSE=loop runfirst 600 ms

dim ticks: the other pages’ parses, 3.4 ms each at the median; the ones due during page-07 run after it

Fig 7.1The accent block is page-07’s parse on the loop thread. It runs past the dashed tick, where debug mode warns.

count_links(body) is CPU-bound: it keeps the processor busy from its first line to its last and never awaits. Called on the loop thread, it runs from start to finish inside one step of one task, and no other task runs until it returns. A median page of 160 KB parses in 3.4 ms.

page-07 is the whole manual on one page: 6.0 MB of HTML with 20,000 links, which take 126 ms to count. In the run with PARSE=loop, it held the loop for 128.0 ms in one stretch, and debug mode logged Executing <Task finished name='Task-9' …> took 0.129 seconds. For those milliseconds, no other link’s bytes were read.

STEP 02 / 05

THREADS DON’T SPEED IT UP, BUT THE LOOP KEEPS MOVING

worker thread: count_links(page-07)holds the GIL 5 ms at a time
loop threadlongest stretch · 21.8 ms
  • GIL handed over every 5 ms: sys.getswitchinterval() returns 0.005 (schematic: slices every 5 ms)
  • longest loop stretch measured in the PARSE=threads run, against 128.0 ms with the parse on the loop
Fig 7.2Two strips sharing one GIL. The loop thread’s accent slices keep arriving while the worker parses page-07.

On the default build, the lets only one thread run Python bytecode at a time, however many cores the machine has. The concurrency tutorial covers where it came from. Here, 8 threads parse the 96 pages in 0.482 s, no faster than one thread at 0.477 s.

Threads still help the loop. asyncio.to_thread(count_links, body) runs the parse in a worker thread, and the GIL changes hands every 5 ms: sys.getswitchinterval() returns 0.005. In the PARSE=threads run, the loop’s longest stretch fell from 128.0 ms to 21.8 ms.

STEP 03 / 05

PROCESSES ARE FAST ONCE THEY ARE RUNNING

  • one core0.477 s
    every page, one after another
  • 8 threads, default build0.482 s
    one GIL: one thread runs Python at a time
  • 8 processes, warm0.133 s
    workers already started; each page pickled through a pipe
  • 8 processes, cold1.017 s
    includes starting 8 interpreter processes with spawn

seconds to count the links in 96 pages · 22.7 MB of HTML

Fig 7.3Bars to scale in seconds. Compare the two process bars: the same work, warm and cold.

ProcessPoolExecutor runs each worker in its own process, with its own interpreter and its own GIL. Each page is pickled (turned into bytes by Python’s pickle module) and sent to a worker through a pipe, and the count comes back the same way. Warm, 8 workers parse all 96 pages in 0.133 s, 3.6 times faster than one core and close to the 126 ms page-07 takes on its own.

Cold, the same job takes 1.017 s, twice as long as one core, because the time includes starting 8 interpreter processes. macOS starts them with spawn: each worker is a fresh Python that imports the script’s modules. In the PARSE=processes run, the loop’s longest stretch was 44.4 ms.

STEP 04 / 05

SUBINTERPRETERS START IN 0.17 SECONDS

  • 8 processes, cold1.017 s
    the last step’s slow bar: 8 processes started with spawn
  • 8 interpreters, cold0.169 s
    includes starting 8 interpreters, one per thread, in the same process
  • 8 interpreters, warm0.135 s
    workers already started
longest loop stretch, final script4.4 ms

seconds to count the links in 96 pages, 8 workers per pool

Fig 7.4The accent bar is the interpreter pool, cold. It sits next to the cold process pool from the last step.

concurrent.futures.InterpreterPoolExecutor (since 3.14) runs each worker as a in its own thread: a separate interpreter with its own modules and its own GIL, inside the same process. Warm, it parsed the 96 pages in 0.135 s. Cold, start included, it took 0.169 s, 6 times faster than a cold process pool.

The final script hands each page to the pool with loop.run_in_executor(pool, count_links, body). In that run the loop’s longest stretch was 4.4 ms. When a count comes back, the task continues past that await, so each parsed page adds one switch, 96 in all, which takes the switch count to 494.

PARSING IN AN INTERPRETER POOL
import asyncio
from concurrent.futures import InterpreterPoolExecutor

from pages import count_links           # importable by name in a fresh interpreter

async def parse_all(bodies):
    loop = asyncio.get_running_loop()
    with InterpreterPoolExecutor() as pool:   # one interpreter, with its own GIL, per worker
        counts = await asyncio.gather(*(
            loop.run_in_executor(pool, count_links, body) for body in bodies
        ))
    return sum(counts)                  # 76,124 links in 96 pages
STEP 05 / 05

NO GIL AT ALL: PYTHON 3.15T

  • 1 thread
    with GIL
    0.356 s
    no GIL
    0.316 s
  • 2 threads
    with GIL
    0.356 s
    no GIL
    0.192 s
  • 4 threads
    with GIL
    0.356 s
    no GIL
    0.134 s
  • 8 threads
    with GIL
    0.355 s
    no GIL
    0.131 s

with GIL: the default build · no GIL: python3.15t, the free-threaded build · seconds for the 95 pages without page-07, median of 3

Fig 7.5Grouped bars over thread counts. The accent bars are the free-threaded build; the flat ones are the default.

The , python3.15t, has no GIL, so threads run Python in parallel. It has been officially supported but optional since 3.14 (PEP 779), and the 3.15 macOS installer installs it by default. aiohttp 3.14.5 runs on it with the GIL off.

One call to count_links runs on one thread, so the test leaves page-07 out. The other 95 pages take 0.316 s on 1 thread, 0.192 s on 2, 0.134 s on 4 and 0.131 s on 8: 2.4 times faster at four threads, then flat in this run. The default build stays at 0.356 s at every thread count.

Removing the GIL costs some speed on a single thread: the docs put it at about 1% on macOS (Arm) to 8% on Linux (x86-64).

THE FREE-THREADED BUILD
uv python install 3.15t
python3.15t -c "import sys; print(sys._is_gil_enabled())"
# False
PARSE=threads python3.15t check_links.py
# 100 links, 96 ok, 5.22 s

WHAT YOU NOW KNOW

  • CPU work never awaits, so on the loop it blocks: page-07’s 126 ms parse held the loop for 128 ms in one stretch.
  • Threads speed up CPU work only on the free-threaded build (2.4 times at four threads here); on the default build they keep the loop responsive.
  • An interpreter pool did the 0.48 s of parsing in 0.17 s from a cold start, against 1.02 s for a cold process pool.
deep dive: what crosses into a worker, start methods, extensions that bring the GIL back, and loops in many threads

What crosses into a worker. Process and interpreter pools pickle the function reference and its arguments, so the function must be importable by name. That is why count_links lives in pages.py and not in __main__. A worker in an interpreter pool imports the function by name in a fresh interpreter, so a module reachable only through a sys.path change made at runtime fails there with NotShareableError: args not shareable. The lower-level module is concurrent.interpreters (PEP 734, since 3.14), with interpreter queues for passing data.

Start methods. macOS and Windows start pool processes with spawn: a fresh interpreter that imports your module. On Linux the default changed from fork to forkserver in 3.14. Since 3.15, when a pool process dies, the traceback includes the dead worker’s PID and exit code.

Extensions can bring the GIL back. On a free-threaded build, importing a C extension that does not declare support for running without the GIL turns the GIL back on for the whole process, with a warning. Setting PYTHON_GIL=0 keeps it off. sys._is_gil_enabled() reports the state at runtime, and sysconfig.get_config_var("Py_GIL_DISABLED") reports the build. 3.15 adds abi3t, a stable ABI for free-threaded builds (PEP 803).

Loops in many threads. Since 3.14, asyncio supports free-threaded builds: several event loops, one per thread, run in parallel, and the docs say they scale linearly with the number of threads. The docs publish no figure, and this tutorial measured only parsing, not several loops.

Why 2.4 times and not 8. Not investigated in this run. Four threads took 0.134 s and eight took 0.131 s, and this tutorial does not guess at the cause. With page-07 back in, 8 free-threaded threads take 0.218 s for all 96 pages; one page’s 126 ms parse cannot be split across threads.

All 494 switches

One run of the checker, 100 links in 5.23 seconds on one thread. Each row below is the switches one chapter added, with the line of the script they happened at. The largest row is the network: 203 times a page’s bytes arrived and its check continued. The smallest row that matters is chapter 04’s: 2 switches, one of them page-17’s timeout, which decides when the run ends.

switches in one run of check_links.py494
  • 01main() starts1
    async def main()
  • 02waiting for the network203
    async with session.get(url) · await resp.text()
  • 03tasks starting, the group finishing101
    tg.create_task(…) · async with asyncio.TaskGroup()
  • 04exceptions thrown in2
    a timeout and a refused connection, at session.get
  • 05waiting for a free slot90
    async with limit
  • 06the SQLite save in a thread1
    await asyncio.to_thread(save, results)
  • 07pages parsed in another interpreter96
    await loop.run_in_executor(pool, …)
one run of check_links.py · Python 3.15.2 · totals of 484–505 across five runs, because page reads arrive in different chunks
the one thread, start to exit5.32 s
  • waiting for the OS5.17 s · 97.2%
  • running Python0.15 s
  • loop turns677 · one switch every 10.8 ms
ticks at the moments the run’s timeline recorded Python running; the dim bar is time inside kqueue
WHICH TOOL FOR WHICH WAIT
  • waiting for the network→ coroutines and tasks24.59 s became 5.23 s on one thread
  • a library that blocks→ asyncio.to_thread16 worker threads on this 12-core machine
  • CPU work→ an interpreter pool, a process pool, or threads on 3.15t0.17 s, 1.02 s and 0.22 s for 0.48 s of parsing on one core
HOW THIS TUTORIAL COUNTED SWITCHES
import inspect
import sys

mon, events = sys.monitoring, sys.monitoring.events
switches = 0

def count(code, offset, *exc):          # PY_THROW also passes the exception
    global switches
    ours = code.co_filename.endswith("check_links.py")
    if not (ours and code.co_flags & inspect.CO_COROUTINE):
        return mon.DISABLE              # other code: stop reporting it
    switches += 1                       # 469 to 505 per run, in the runs measured

mon.use_tool_id(3, "switches")
for event in (events.PY_START, events.PY_RESUME, events.PY_THROW):
    mon.register_callback(3, event, count)
mon.set_events(3, events.PY_START | events.PY_RESUME | events.PY_THROW)

sys.monitoring (Python 3.12) reports every start, resume and thrown-in exception of the script’s own coroutines; library frames are switched off after their first event, so counting costs almost nothing.

Read next. Concurrency: how Java, Node, Go & Python hold 10,000 clients for what threads and event loops cost at a hundred times this scale, and Interpreted vs compiled: how one line reaches the CPU for the bytecode underneath every coroutine.
$ python check_links.py
100 links, 96 ok, 5.23 s
$