Lesson 34 — Asyncio: Cooperative Multitasking and Event Loop Architecture¶
Module: 4 — Network & Concurrent Systems
Level: intermediate
Language: English
Contents¶
- Why asyncio?
- Key concepts
- Coroutine Lifecycle
- Event Loop Architecture
- Cooperative vs Preemptive Multitasking
- asyncio API Reference
- Blocking vs Non-Blocking
- FastAPI & Backend Integration
- Threading vs Multiprocessing vs Asyncio
- Common mistakes and anti-patterns
- Production Patterns
- Debugging Guide
1. Why asyncio?¶
The problem with synchronous servers¶
In a traditional WSGI architecture, every connection requires a separate OS thread. A thread takes ~8 MB of RAM and is blocked for the entire time it waits for I/O. With 10,000 concurrent users, the server spends ~80 GB of RAM on threads alone — even if they spend 99% of their time just waiting.
Time scale: why I/O is so slow¶
| Operation | Real time | "Human" equivalent |
|---|---|---|
| CPU cache (L1) | 1 ns | 3 seconds |
| RAM | 100 ns | 5 minutes |
| SSD read | 100 µs | 3.5 days |
| TCP round trip | 100 ms | ~9.5 years |
| External API | 500 ms | ~48 years |
Asyncio solves this with cooperative multitasking: instead of keeping a thread blocked for ~48 "human years", await frees the Event Loop to serve thousands of other connections.
2. Key concepts¶
Coroutine¶
A special function defined with async def that:
- When called, returns a coroutine object (executes nothing)
- When awaited with await, runs until its first internal await
- On await asyncio.sleep(), suspends itself, saves its state and hands control to the Event Loop
async def my_coroutine(): # Definition
...
obj = my_coroutine() # Call → coroutine object (FROZEN)
result = await my_coroutine() # Execute and wait for the result
Task¶
asyncio.Task is a wrapper around a coroutine that schedules its execution on the Event Loop.
task = asyncio.create_task(my_coroutine()) # Schedules NOW, does not wait
result = await task # Explicitly wait for completion
Task states:
Event Loop¶
The central scheduler of asyncio. How it works:
while True:
1. Run all READY tasks up to their first await
2. Ask the OS: which sockets/timers are ready? (epoll/kqueue)
3. Move completed tasks to the READY queue
4. Repeat
3. Coroutine Lifecycle¶
Call async def func()
│
▼
┌─────────────────┐
│ coroutine object│ ← STATE: GEN_CREATED
│ code: lines... │ Nothing is executed!
│ locals: {} │
└────────┬────────┘
│ await func() or asyncio.run(func())
▼
┌─────────────────┐
│ RUNNING │ ← Executes line by line
│ │
│ ... code ... │
│ await <awaitable> ← PAUSE
└────────┬────────┘
│ await
▼
┌─────────────────┐
│ SUSPENDED │ ← State saved (locals, position)
│ waiting for I/O │ The Event Loop is free!
└────────┬────────┘
│ I/O completed
▼
┌─────────────────┐
│ RUNNING │ ← Resumes from the await point
└────────┬────────┘
│ return
▼
┌─────────────────┐
│ CLOSED │ ← GEN_CLOSED, result available
└─────────────────┘
4. Event Loop Architecture¶
┌────────────────────────────────────────────────────────────┐
│ EVENT LOOP (single thread) │
│ │
│ ┌──────────────────┐ ┌────────────────────────────────┐ │
│ │ Ready Queue │ │ I/O Selector (epoll/kqueue) │ │
│ │ │ │ │ │
│ │ [Task A] ──────► │ │ Socket A: HTTP ── WAITING │ │
│ │ [Task C] │ │ Socket B: DB ── WAITING │ │
│ │ │ │ Socket C: API ── READY! ◄──── │ │
│ └────────┬─────────┘ └──────────────┬─────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ Run Task Wake Task B │
│ until await (move to Queue) │
└────────────────────────────────────────────────────────────┘
Where does the "waiting" actually happen?¶
When Python executes await asyncio.sleep(1):
1. The Event Loop registers a timer with the OS
2. The coroutine moves to SUSPENDED
3. The OS uses epoll/kqueue — a hardware-backed notification mechanism
4. The Python thread sleeps, using ~0% CPU
5. After 1 second the OS wakes Python up, and the Event Loop resumes the coroutine
5. Cooperative vs Preemptive Multitasking¶
| Preemptive (Threading) | Cooperative (Asyncio) | |
|---|---|---|
| Who interrupts | The OS, forcibly, at any moment | The coroutine, voluntarily, via await |
| Race conditions | Yes — you need Lock/Semaphore | Less often — switches happen only at await, but state can change between awaits (asyncio.Lock) |
| Stack | ~8 MB/thread | ~2 KB/coroutine |
| Max concurrent | ~500–1000 | ~100,000+ |
| Blocking call | Stops 1 thread | Stops the entire Event Loop |
The key danger of asyncio: a single blocking call (time.sleep, requests.get) stops EVERYTHING — not just one task.
6. asyncio API Reference¶
Core functions¶
# Program entry point
asyncio.run(main())
asyncio.run(main(), debug=True)
# Scheduling tasks
task = asyncio.create_task(coro())
tasks = asyncio.gather(coro1(), coro2(), coro3())
results = await asyncio.gather(*coros, return_exceptions=True)
# Waiting
await asyncio.sleep(seconds) # Releases the Event Loop
await asyncio.wait_for(coro, timeout=5.0) # With a timeout
# Checking status
task.done() # True if finished
task.cancelled() # True if cancelled
task.result() # Returns the result or raises the exception
task.cancel() # Sends CancelledError into the task
# Current Event Loop
loop = asyncio.get_event_loop()
# Delegating blocking code
result = await loop.run_in_executor(executor, sync_func, *args)
async with and async for¶
# async context manager
async with aiofiles.open("file.txt") as f:
data = await f.read()
# async generator / iterator
async for item in async_generator():
process(item)
7. Blocking vs Non-Blocking¶
Replacement table¶
| Synchronous (BLOCKING) | Asynchronous (CORRECT) |
|---|---|
time.sleep(n) |
await asyncio.sleep(n) |
requests.get(url) |
await session.get(url) (aiohttp) |
open(path).read() |
await aio_file.read() (aiofiles) |
psycopg2.connect(...) |
await asyncpg.connect(...) |
subprocess.run(cmd) |
await asyncio.create_subprocess_exec(...) |
| CPU-bound loop | await loop.run_in_executor(process_pool, func) |
run_in_executor — the bridge between worlds¶
import asyncio
import concurrent.futures
process_pool = concurrent.futures.ProcessPoolExecutor()
thread_pool = concurrent.futures.ThreadPoolExecutor()
async def handle_request():
loop = asyncio.get_event_loop()
# CPU-bound → ProcessPoolExecutor (bypasses the GIL)
result = await loop.run_in_executor(process_pool, heavy_computation)
# Legacy sync I/O → ThreadPoolExecutor
data = await loop.run_in_executor(thread_pool, legacy_db_query)
return result, data
8. FastAPI & Backend Integration¶
Three kinds of endpoints in FastAPI¶
from fastapi import FastAPI
import asyncio, time
app = FastAPI()
# ✅ async def + await: best for async I/O
@app.get("/users")
async def get_users(pool):
async with pool.acquire() as conn:
return await conn.fetch("SELECT * FROM users")
# ❌ async def + blocking: the worst (blocks the entire Event Loop!)
@app.get("/broken")
async def broken_endpoint():
time.sleep(2) # Stops the WHOLE server!
return {"status": "bad"}
# ⚠️ def (not async): FastAPI automatically runs it in a threadpool
@app.get("/legacy")
def legacy_endpoint():
time.sleep(2) # OK — a separate thread, the loop is free
return {"status": "ok but limited"}
The FastAPI rule:¶
async def+ onlyawaitoperations → Event Loop (optimal)async def+ a blocking call → disasterdef(sync) → automatic threadpool (safe, but limited)
WebSocket scale¶
Synchronous WSGI (Gunicorn threads):
10,000 WebSocket connections → 10,000 threads → ~80 GB RAM
(practically impossible)
Asyncio ASGI (uvicorn):
10,000 WebSocket connections → 10,000 coroutine objects → ~20 MB RAM
(easy — only memory for the state)
9. Threading vs Multiprocessing vs Asyncio¶
| Threading | Multiprocessing | Asyncio | |
|---|---|---|---|
| Model | N threads, 1 process | N processes | 1 thread, 1 process |
| GIL | One shared by all | Each has its own | One (not a limitation) |
| Scheduling | Preemptive (OS) | Preemptive (OS) | Cooperative (await) |
| Memory/unit | ~8 MB/thread | ~50 MB/process | ~2 KB/coroutine |
| Race conditions | Yes | No | Less often (only at await) |
| CPU-bound | ❌ GIL | ✅ True parallelism | ❌ Blocks the loop |
| I/O-bound | ✅ OK | ✅ OK | ✅ Best |
| Max concurrent | ~500 | ~CPU cores | ~100,000+ |
| Ideal for | Legacy sync libraries | ML, image, math | Web, API, WebSocket |
A solution for combined workloads (FastAPI production)¶
HTTP request → FastAPI async endpoint
│
├── DB query → asyncpg (async) → await
├── HTTP call → aiohttp (async) → await
└── CPU work → ProcessPoolExecutor → run_in_executor → await
10. Common mistakes and anti-patterns¶
Anti-pattern 1: Blocking in async (the most frequent mistake)¶
# DANGEROUS
async def endpoint():
time.sleep(2) # Stops the whole server!
requests.get(url) # Blocks the Event Loop!
psycopg2.execute(...) # Stops all connections!
# CORRECT
async def endpoint():
await asyncio.sleep(2)
async with session.get(url) as r: ...
async with pool.acquire() as conn: await conn.fetch(...)
Anti-pattern 2: A forgotten await¶
# DANGEROUS — RuntimeWarning, nothing is executed
result = fetch_data() # Returns a coroutine object!
# CORRECT
result = await fetch_data() # Executes and waits
Anti-pattern 3: A Task without a reference (the exception disappears)¶
# DANGEROUS
asyncio.create_task(risky_op()) # The exception will be swallowed!
# CORRECT
task = asyncio.create_task(risky_op())
try:
await task
except Exception as e:
logger.error(f"Task failed: {e}")
Anti-pattern 4: CPU-bound in async (freezes the loop)¶
# DANGEROUS
async def process():
result = sum(i*i for i in range(10**8)) # Blocks the loop for seconds!
# CORRECT
async def process():
loop = asyncio.get_event_loop()
result = await loop.run_in_executor(process_pool, heavy_sync_func)
Anti-pattern 5: Main finishes before the Task¶
# DANGEROUS
async def main():
asyncio.create_task(background_work())
# The Event Loop stops → the Task is destroyed!
# CORRECT
async def main():
task = asyncio.create_task(background_work())
await asyncio.sleep(0) # Let the Task start
await task # Explicitly wait
11. Production Patterns¶
Pattern 1: Connection Pool with a Semaphore¶
class ConnectionPool:
def __init__(self, max_size: int):
self._semaphore = asyncio.Semaphore(max_size)
async def __aenter__(self):
await self._semaphore.acquire()
return await self._create_connection()
async def __aexit__(self, *args):
self._semaphore.release()
Pattern 2: Retry with Exponential Backoff¶
async def with_retry(coro_factory, max_retries=3, base_delay=1.0):
for attempt in range(max_retries):
try:
return await coro_factory()
except Exception as e:
if attempt == max_retries - 1:
raise
delay = base_delay * (2 ** attempt)
await asyncio.sleep(delay)
Pattern 3: Producer-Consumer with asyncio.Queue¶
async def producer(queue: asyncio.Queue, items):
for item in items:
await queue.put(item)
await queue.put(None) # sentinel
async def consumer(queue: asyncio.Queue):
while True:
item = await queue.get()
if item is None:
break
await process(item)
queue.task_done()
Pattern 4: Rate Limiter¶
class AsyncRateLimiter:
def __init__(self, max_calls: int, period: float = 1.0):
self._semaphore = asyncio.Semaphore(max_calls)
self._period = period
async def __aenter__(self):
await self._semaphore.acquire()
asyncio.create_task(self._release_after_period())
return self
async def _release_after_period(self):
await asyncio.sleep(self._period)
self._semaphore.release()
async def __aexit__(self, *args):
pass
12. Debugging Guide¶
Debugging tools¶
# 1. Debug mode via an environment variable
PYTHONASYNCIODEBUG=1 python app.py
# 2. Verbose asyncio logging
PYTHONASYNCIODEBUG=1 python -W always app.py
# 3. Debug mode in code
asyncio.run(main(), debug=True)
# 4. A custom exception handler
def handle_exception(loop, context):
logger.error(f"Unhandled exception: {context}")
loop = asyncio.get_event_loop()
loop.set_exception_handler(handle_exception)
What debug mode detects¶
| Problem | What debug mode reports |
|---|---|
| A coroutine was never awaited | RuntimeWarning: coroutine was never awaited |
| A blocking function > 100ms | Executing took X.XX seconds |
| An exception in a Task without await | A log message at GC time |
| Slow callbacks | Executing callback took X seconds |
Analyzing a typical stack trace¶
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<risky_op() at app.py:15>
exception=ValueError('...')>
Traceback (most recent call last):
File "app.py", line 17, in risky_op ← Where the exception occurred
raise ValueError(...)
ValueError: ...
Rule: if you see "Task exception was never retrieved", find the place where the Task was created and add a try/except around await task.
Connection to previous lessons¶
| Lesson | Topic | Relation to Asyncio |
|---|---|---|
| 32 | Threading | Preemptive vs Cooperative multitasking |
| 33 | Multiprocessing | ProcessPoolExecutor for run_in_executor |
| 34 | Asyncio | This lesson |
| — | FastAPI | Asyncio = the engine for ASGI web servers |