Skip to content

Coroutines

A coroutine is a lightweight, user-space concurrency unit whose scheduling is controlled by the program itself (not the operating system). Python 3.5+ natively supports coroutines through async/await syntax, and the standard library asyncio provides a complete event loop and toolkit.

This article covers the foundational concepts of coroutines. For more complete asyncio usage (gather, TaskGroup, timeout, async context managers, etc.), see Async Programming and asyncio.

Coroutines vs Threads vs Processes

DimensionCoroutineThreadProcess
SchedulerUser programOperating systemOperating system
Memory overheadVery small (a few KB)Larger (MB range)Largest (independent address space)
Switching costVery low (function call level)Higher (kernel mode switch)Highest
Parallel capabilityNone (single-threaded)Limited by GILTrue parallelism (multi-core)
Best fitI/O-bound, massive connectionsI/O-bound, legacy synchronous codeCPU-bound

async/await Basics

import asyncio

async def greet(name: str, delay: float) -> str:
    """async def defines a coroutine function; calling it returns a coroutine object (does not execute immediately)."""
    print(f"{name} starting to wait {delay}s")
    await asyncio.sleep(delay)   # await suspends the current coroutine and yields control to the event loop
    return f"Hello, {name}!"

async def main() -> None:
    # Sequential execution: total time 1+2 = 3s
    r1 = await greet("Alice", 1.0)
    r2 = await greet("Bob", 2.0)
    print(r1, r2)

asyncio.run(main())   # Python 3.7+: creates an event loop and runs it

Concurrent Execution

await is inherently sequential; to achieve concurrency, use gather or create_task:

import asyncio
import time

async def fetch(name: str, delay: float) -> str:
    await asyncio.sleep(delay)
    return f"{name} done"

async def main() -> None:
    start = time.perf_counter()

    # gather: runs concurrently, total time ≈ max(1, 2, 1.5) ≈ 2s
    results = await asyncio.gather(
        fetch("Task A", 1.0),
        fetch("Task B", 2.0),
        fetch("Task C", 1.5),
    )

    print(results)
    print(f"Total time: {time.perf_counter() - start:.2f}s")

asyncio.run(main())

Tasks and Cancellation

import asyncio

async def long_task(n: int) -> int:
    await asyncio.sleep(n)
    return n

async def main() -> None:
    # create_task immediately submits the coroutine to the event loop
    task = asyncio.create_task(long_task(5))

    await asyncio.sleep(2)
    task.cancel()   # Cancel the task

    try:
        await task
    except asyncio.CancelledError:
        print("Task was cancelled")

asyncio.run(main())

Pros and Cons of Coroutines

Pros

  • Switching overhead is extremely small (user-space switching, no kernel involvement).
  • Achieves high concurrency within a single thread without needing locks (shared state lives in the same thread; await points are the only switching opportunities).
  • Can support tens of thousands or even hundreds of thousands of concurrent connections (e.g., a web server).

Cons

  • Fundamentally single-threaded; cannot utilize multiple CPU cores (use ProcessPoolExecutor for CPU-bound tasks).
  • If any coroutine executes a blocking synchronous I/O call (e.g., time.sleep), it will block the entire event loop.
  • The entire call chain must use async/await; mixing in synchronous blocking calls requires loop.run_in_executor().

Calling Blocking Functions from a Coroutine

import asyncio
from concurrent.futures import ThreadPoolExecutor

def blocking_read(path: str) -> str:
    """Synchronous blocking file read (legacy code)."""
    with open(path, encoding="utf-8") as f:
        return f.read()

async def main() -> None:
    loop = asyncio.get_event_loop()
    # run_in_executor runs the blocking call in a thread pool without blocking the event loop
    with ThreadPoolExecutor() as pool:
        content = await loop.run_in_executor(pool, blocking_read, "README.md")
    print(content[:100])

asyncio.run(main())

Historical Background (Reference)

Python coroutines have evolved through several stages:

StageTechnologyCharacteristics
Python 2generator (yield)Manual switching, no standard scheduler
Python 2/3greenlet (third-party)Explicit switch(), manual scheduling required
Python 2/3gevent (third-party)Based on greenlet + monkey patch for automatic switching
Python 3.4asyncio (standard library)Based on yield from, standardized event loop
Python 3.5+async/await (current standard)Clear syntax, complete ecosystem

New projects should use async/await + asyncio directly. greenlet and gevent only appear in legacy codebases.

Last updated on