python-asyncio-internals
Table of Contents
Python asyncio internals
Python asyncio internals refer to the event loop and task scheduling that power async/await. The event loop repeatedly polls I/O operations and runs ready coroutines, allowing thousands of concurrent connections with minimal threads. Understanding the event loop helps debug async behavior and write efficient concurrent code.
Use asyncio for network I/O, concurrent file operations, or coordinating many independent tasks without threading overhead.
Example
This example shows the event loop, tasks, and concurrent execution.
# run: python3 asyncio_internals.py # description: asyncio event loop and task scheduling import asyncio import time async def fetch(url, delay): """Simulated async fetch.""" print(f"Starting fetch from {url}") await asyncio.sleep(delay) # yield control to event loop print(f"Finished fetch from {url}") return f"Data from {url}" async def main(): # Create tasks (scheduled coroutines) start = time.time() # Sequential: slow # result1 = await fetch("url1", 1) # result2 = await fetch("url2", 1) # Concurrent: fast - both run at once task1 = asyncio.create_task(fetch("url1", 1)) task2 = asyncio.create_task(fetch("url2", 1)) result1 = await task1 result2 = await task2 elapsed = time.time() - start print(f"Completed in {elapsed:.2f}s (concurrent, not sequential)") print(f"Results: {result1}, {result2}") # Get or create event loop and run asyncio.run(main()) # Event loop internals async def show_event_loop(): loop = asyncio.get_event_loop() print(f"\nEvent loop type: {type(loop)}") print(f"Running: {loop.is_running()}") # Create multiple tasks tasks = [ asyncio.create_task(asyncio.sleep(0.1 * i, result=f"Task {i}")) for i in range(1, 4) ] # Wait for all tasks results = await asyncio.gather(*tasks) print(f"All tasks done: {results}") asyncio.run(show_event_loop()) # Timeouts and cancellation async def with_timeout(): try: await asyncio.wait_for(asyncio.sleep(5), timeout=1) except asyncio.TimeoutError: print("Task timed out") # Cancellation task = asyncio.create_task(asyncio.sleep(10)) await asyncio.sleep(0.1) task.cancel() try: await task except asyncio.CancelledError: print("Task was cancelled") asyncio.run(with_timeout())
Common patterns
Event loop:
- Single-threaded loop that alternates between:
- Running ready coroutines
- Polling I/O (waiting for network, disk, timers)
- Running callbacks
asyncio.run()creates, runs, and closes loopasyncio.get_event_loop()gets current loop (inside async function)
Tasks:
asyncio.create_task(coro): schedule coroutine on event loop- Returns immediately with Task object
- Task runs when event loop runs
await taskwaits for completion
Concurrent execution:
- Multiple
await asyncio.sleep()at same point are concurrent - Event loop interleaves them
- No threads needed; single thread with efficient switching
Synchronization:
await asyncio.gather(*tasks): wait for allawait asyncio.wait(tasks): low-level, returns done/pendingasync with asyncio.Lock():: mutual exclusionasync with asyncio.Semaphore(5):: limit concurrent access
Error handling:
- Exceptions in tasks stored; raise when awaited
await asyncio.gather(..., return_exceptions=True): collect errorstask.exception(): get exception without re-raising
Cancellation:
task.cancel(): request cancellation- Raises
asyncio.CancelledErrorinside task - Task should handle and clean up
CancelledErroris subclass ofBaseException(Python 3.8+)
Timeouts:
await asyncio.wait_for(coro, timeout=5): raises TimeoutError- Cancels coroutine if it times out
Callbacks and futures (low-level):
loop.call_soon(func): schedule function to run ASAPasyncio.Future(): low-level promise object- Most code uses async/await, not callbacks/futures
python-asyncio-internals.md · Last modified: by 127.0.0.1
