Coroutines
A coroutine is a lightweight, user-space concurrency unit whose scheduling is controlled by the program itself (not the operating system). Python 3.5+ natively supports coroutines through async/await syntax, and the standard library asyncio provides a complete event loop and toolkit.
This article covers the foundational concepts of coroutines. For more complete asyncio usage (
gather, TaskGroup, timeout, async context managers, etc.), see Async Programming and asyncio.Coroutines vs Threads vs Processes
| Dimension | Coroutine | Thread | Process |
|---|---|---|---|
| Scheduler | User program | Operating system | Operating system |
| Memory overhead | Very small (a few KB) | Larger (MB range) | Largest (independent address space) |
| Switching cost | Very low (function call level) | Higher (kernel mode switch) | Highest |
| Parallel capability | None (single-threaded) | Limited by GIL | True parallelism (multi-core) |
| Best fit | I/O-bound, massive connections | I/O-bound, legacy synchronous code | CPU-bound |
async/await Basics
import asyncio
async def greet(name: str, delay: float) -> str:
"""async def defines a coroutine function; calling it returns a coroutine object (does not execute immediately)."""
print(f"{name} starting to wait {delay}s")
await asyncio.sleep(delay) # await suspends the current coroutine and yields control to the event loop
return f"Hello, {name}!"
async def main() -> None:
# Sequential execution: total time 1+2 = 3s
r1 = await greet("Alice", 1.0)
r2 = await greet("Bob", 2.0)
print(r1, r2)
asyncio.run(main()) # Python 3.7+: creates an event loop and runs itConcurrent Execution
await is inherently sequential; to achieve concurrency, use gather or create_task:
import asyncio
import time
async def fetch(name: str, delay: float) -> str:
await asyncio.sleep(delay)
return f"{name} done"
async def main() -> None:
start = time.perf_counter()
# gather: runs concurrently, total time ≈ max(1, 2, 1.5) ≈ 2s
results = await asyncio.gather(
fetch("Task A", 1.0),
fetch("Task B", 2.0),
fetch("Task C", 1.5),
)
print(results)
print(f"Total time: {time.perf_counter() - start:.2f}s")
asyncio.run(main())Tasks and Cancellation
import asyncio
async def long_task(n: int) -> int:
await asyncio.sleep(n)
return n
async def main() -> None:
# create_task immediately submits the coroutine to the event loop
task = asyncio.create_task(long_task(5))
await asyncio.sleep(2)
task.cancel() # Cancel the task
try:
await task
except asyncio.CancelledError:
print("Task was cancelled")
asyncio.run(main())Pros and Cons of Coroutines
Pros
- Switching overhead is extremely small (user-space switching, no kernel involvement).
- Achieves high concurrency within a single thread without needing locks (shared state lives in the same thread;
awaitpoints are the only switching opportunities). - Can support tens of thousands or even hundreds of thousands of concurrent connections (e.g., a web server).
Cons
- Fundamentally single-threaded; cannot utilize multiple CPU cores (use
ProcessPoolExecutorfor CPU-bound tasks). - If any coroutine executes a blocking synchronous I/O call (e.g.,
time.sleep), it will block the entire event loop. - The entire call chain must use
async/await; mixing in synchronous blocking calls requiresloop.run_in_executor().
Calling Blocking Functions from a Coroutine
import asyncio
from concurrent.futures import ThreadPoolExecutor
def blocking_read(path: str) -> str:
"""Synchronous blocking file read (legacy code)."""
with open(path, encoding="utf-8") as f:
return f.read()
async def main() -> None:
loop = asyncio.get_event_loop()
# run_in_executor runs the blocking call in a thread pool without blocking the event loop
with ThreadPoolExecutor() as pool:
content = await loop.run_in_executor(pool, blocking_read, "README.md")
print(content[:100])
asyncio.run(main())Historical Background (Reference)
Python coroutines have evolved through several stages:
| Stage | Technology | Characteristics |
|---|---|---|
| Python 2 | generator (yield) | Manual switching, no standard scheduler |
| Python 2/3 | greenlet (third-party) | Explicit switch(), manual scheduling required |
| Python 2/3 | gevent (third-party) | Based on greenlet + monkey patch for automatic switching |
| Python 3.4 | asyncio (standard library) | Based on yield from, standardized event loop |
| Python 3.5+ | async/await (current standard) | Clear syntax, complete ecosystem |
New projects should use async/await + asyncio directly. greenlet and gevent only appear in legacy codebases.
Last updated on