Skip to content

Part IX. Async and Real-Time

HTTP polling for a chat means 100 requests/sec for 100 clients. WebSocket gives you one long-lived connection. Django Channels implements WebSocket on top of ASGI, and Redis handles broadcast between processes.

Prerequisites: Parts II, III, VIII. Level: Advanced.


Conceptual bridge: asyncio → ASGI → Channels

Three concepts form a single stack. Each one answers a specific question:

asyncio          — How do you hold thousands of connections without thousands of threads?
    ↓               one event loop; await instead of blocking
ASGI             — How does Django receive async requests from the server?
    ↓               an async interface between Daphne/Uvicorn and Django; replaces WSGI
Django Channels  — How does Django route WebSocket alongside HTTP?
    ↓               ProtocolTypeRouter → "http" → views, "websocket" → Consumer
Redis            — How does Worker 1 send messages to Worker 2's clients?
                    channel layer: pub/sub between processes

Dependencies between the levels:

Without What becomes impossible
asyncio ASGI is impossible — there is no event loop for async I/O
ASGI Channels cannot be plugged into Django — there is no async entry point
Channels No WebSocket routing and no Consumer API
Redis WebSocket messages will not reach clients of another worker process

WSGI vs ASGI — why the transition is needed:

# WSGI — synchronous: one request = one thread, the thread blocks until the response
def application(environ, start_response):
    data = requests.get(url)   # blocks the thread — nobody else can use it
    ...

# ASGI — asynchronous: one thread = thousands of coroutines, await frees the thread
async def application(scope, receive, send):
    async with httpx.AsyncClient() as client:
        data = await client.get(url)   # the thread is free — it handles other connections
    ...

Django Channels is the Django implementation of ASGI that adds a WebSocket Consumer API on top of the standard. ProtocolTypeRouter is the first routing point: HTTP goes to Django views, WebSocket goes to a Consumer.


Chapters

Document What we will learn
Sync vs Async blocking vs non-blocking I/O, GIL, thread pool, event loop
asyncio coroutine, await, Task, gather, asyncio.run()
asyncio: Event Loop event loop architecture, callbacks, futures, libuv
ASGI ASGI spec, ProtocolTypeRouter, lifecycle, scope
Django Async Views async def view(), limitations, when it is worth it
Async ORM aadd, acreate, aget, async for, differences from sync
sync_to_async database_sync_to_async, sync_to_async, thread safety
Async HTTP Clients httpx, aiohttp, connection pooling
Benchmarking comparing sync vs async, locust, results
Async Use Cases when async is justified, anti-patterns
WebSocket and Channels Django Channels, AsyncWebsocketConsumer, channel layer, Redis

Key concepts

Coroutine and await:

import asyncio

async def fetch_data():
    await asyncio.sleep(1)   # does not block the event loop
    return "data"

async def main():
    result = await fetch_data()
    print(result)

asyncio.run(main())

ASGI entrypoint:

# notes_project/asgi.py
# IMPORT ORDER IS CRITICAL!
import os
from django.core.asgi import get_asgi_application
from django.contrib.staticfiles.handlers import ASGIStaticFilesHandler

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'notes_project.settings')
django_asgi_app = get_asgi_application()  # ← BEFORE any notes_app import

from channels.routing import ProtocolTypeRouter, URLRouter
from channels.auth import AuthMiddlewareStack
from notes_project.routing import websocket_urlpatterns  # ← AFTER (it imports GroupChatConsumer)

application = ProtocolTypeRouter({
    "http":      ASGIStaticFilesHandler(django_asgi_app),  # static files — development only
    "websocket": AuthMiddlewareStack(
        URLRouter(websocket_urlpatterns)  # r"^ws/groups/(?P<group_pk>\d+)/chat/$"
    ),
})

GroupChatConsumer — lifecycle:

class GroupChatConsumer(AsyncWebsocketConsumer):
    async def connect(self):
        user     = self.scope['user']
        group_pk = self.scope['url_route']['kwargs']['group_pk']

        if not user.is_authenticated:
            await self.close(); return

        is_member = await database_sync_to_async(self.check_membership)(group_pk, user)
        if not is_member:
            await self.close(); return

        self.room_group_name = f"chat_group_{group_pk}"
        await self.channel_layer.group_add(self.room_group_name, self.channel_name)
        await self.accept()

        # send the 50 most recent messages
        history = await database_sync_to_async(self.load_history)(group_pk)
        for msg in reversed(history):
            await self.send(text_data=json.dumps({'type': 'history', **msg}))

    async def disconnect(self, close_code):
        await self.channel_layer.group_discard(self.room_group_name, self.channel_name)

    async def receive(self, text_data):
        data    = json.loads(text_data)
        content = data.get('content', '').strip()
        if not content:
            return
        user = self.scope['user']
        await database_sync_to_async(self.save_message)(
            int(self.scope['url_route']['kwargs']['group_pk']), user, content
        )
        await self.channel_layer.group_send(
            self.room_group_name,
            {'type': 'chat_message', 'content': content, 'author': user.username}
        )

    async def chat_message(self, event):
        await self.send(text_data=json.dumps({'type': 'chat_message', **event}))

database_sync_to_async + materialization:

# ✅ CORRECT: list() materializes the QuerySet in the worker thread, before returning
def load_history(self, group_pk):
    return list(
        ChatMessage.objects.filter(group_id=group_pk)
        .select_related('author')
        .order_by('-timestamp')[:50]
        .values('content', 'author__username', 'timestamp')
    )

# ❌ WRONG: returns a lazy QuerySet → SynchronousOnlyOperation
def load_history_bad(self, group_pk):
    return ChatMessage.objects.filter(group_id=group_pk)  # lazy!

Redis channel layer:

# settings.py
CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels_redis.core.RedisChannelLayer",
        "CONFIG": {
            "hosts": [REDIS_URL],
            "socket_timeout": None,    # ← None = no timeout for idle WS
        },
    }
}

socket_timeout=None is critical: without it, the Redis connection is closed when idle → TimeoutError on the next message.

Polling vs WebSocket:

HTTP Polling (setInterval 1s):
  100 clients × 1 req/s = 100 HTTP requests/sec
  Each request: TCP handshake + HTTP headers + response
  Latency: up to 1000ms

WebSocket:
  100 clients = 100 open connections (persistent)
  Messages: < 50ms
  Load: only on real events

Where this lives in Notes Chat App

File What it shows
notes_project/asgi.py ProtocolTypeRouter, import order
notes_project/routing.py re_path(r"^ws/groups/.../chat/$", ...)
notes_app/consumers.py GroupChatConsumer — connect, disconnect, receive, chat_message
notes_app/static/.../group_chat.js JS client: new WebSocket(), onmessage, send()
notes_app/tests/test_consumers.py WebsocketCommunicator, TransactionTestCase
notes_project/settings.py CHANNEL_LAYERS with socket_timeout: None

Pedagogical connections

Part IX: ASGI / Channels
  ← Part II: WSGI vs ASGI (async entrypoint)
  ← Part III: ORM (database_sync_to_async wrapping)
  ← Part VII: Auth (scope['user'], check_membership)
  ← Part VIII: TransactionTestCase (consumer tests)

Zero to Hero: Step 7 — WebSocket chat: Channels, Redis, GroupChatConsumer, consumer tests.

Lab: Async Lab — write a simple consumer, a test, and a channel layer broadcast.


Pitfalls

Situation Error Solution
get_asgi_application() after from notes_app... AppRegistryNotReady move get_asgi_application() BEFORE any notes_app import
Lazy QuerySet in database_sync_to_async SynchronousOnlyOperation list() in the sync helper before returning
socket_timeout is not None TimeoutError on idle WS "socket_timeout": None in CHANNEL_LAYERS CONFIG
TestCase for consumer tests setUp objects disappear TransactionTestCase

Review questions

  • What is a coroutine? What is the difference between async def and def?
  • What is an event loop? How many event loops can a process have?
  • Why do we need database_sync_to_async? What happens without it?
  • Why is list() inside a sync helper critical when returning into an async context?
  • What is a channel layer? Why use Redis if there is an in-memory layer?
  • Why is socket_timeout=None important for idle WebSocket connections?
  • When should you use async views, and when sync views, in Django?

Next → Part X. Linux, DevOps, and Deployment — how to run it in production.