await asyncio.gather

Concurrency & Async

Meaning

Runs multiple awaitable coroutines concurrently and collects all their results into a list in submission order. Addresses the pain point of sequential await calls that waste wall-clock time waiting for independent I/O operations one after another. Reached for whenever two or more independent async tasks must complete before proceeding.

Primary Function

Concurrency

Communicative Purpose

Enables parallel execution of independent async tasks to avoid sequential waiting on I/O-bound operations.

Pattern

await asyncio.gather(*coroutines)

Core Structure

await asyncio.gather(*...)

Função primária

Concurrency

Propósito comunicativo

Enables parallel execution of independent async tasks to avoid sequential waiting on I/O-bound operations.

Situações de gatilho

Network programming: fetching multiple API endpoints simultaneously without blocking; Service health checks: probing several hosts concurrently within a timeout; Data pipelines: running independent async transformations in parallel before merging results

Contextos

asyncio applications, concurrent HTTP clients, async test suites, data ingestion pipelines

Padrão

await asyncio.gather(*coroutines)

Estrutura central

await asyncio.gather(*...)

Slots de substituição

coroutines: iterable of awaitable objects (coroutines, Tasks, or Futures)

Colocados típicos

asyncio.create_task
asyncio.wait
asyncio.as_completed
async def
await
return_exceptions

Substituições comuns

asyncio.wait: lower-level alternative offering fine-grained completion control (FIRST_COMPLETED
ALL_COMPLETED) but returns done/pending sets instead of ordered results
sequential awaits: simpler but loses all concurrency
asyncio.TaskGroup (3.11+): structured concurrency with automatic cancellation on child failure

Erros comuns

Forgetting the * unpacking operator — passing a list directly causes gather to treat the list itself as a single awaitable and raises TypeError; Awaiting each coroutine individually before gather — this runs them sequentially, completely defeating the concurrency; Not handling exceptions — one failing coroutine cancels all remaining tasks by default unless return_exceptions=True; Passing non-awaitable values like plain function calls instead of coroutine objects — raises TypeError at runtime

Similar / contraste

asyncio.wait: returns done/pending sets with fine-grained control, no result ordering; asyncio.as_completed: yields futures as they complete, useful for progress reporting; asyncio.TaskGroup: structured concurrency with automatic cancellation on any child failure (3.11+)

Interferências

Coming from JavaScript: may expect Promise.all to reject immediately on first rejection — asyncio.gather also cancels remaining tasks on first exception by default, but the exception propagation model and cancellation behavior differ; Coming from Go: may expect goroutine-style fire-and-forget — gather still requires explicit await and does not provide true parallelism due to the GIL

Família do chunk

asyncio.gather
asyncio.wait
asyncio.as_completed
asyncio.create_task
asyncio.TaskGroup

Nuance

Do not use gather when tasks have data dependencies or ordering requirements — sequential awaits are clearer and safer. gather schedules all coroutines immediately but executes them cooperatively on one thread, so CPU-bound work gains no speedup. If one coroutine raises and return_exceptions is False, all other still-running tasks are cancelled before the exception propagates upward.

Efeito pragmático

Dramatically reduces wall-clock time for I/O-bound workloads by overlapping wait times, turning N sequential round-trips into roughly one round-trip's duration.

Dica de memória

Like a restaurant host seating multiple tables at once — all orders go to the kitchen simultaneously, but each dish comes back labeled with its original table number.

Nota

gather preserves result order matching submission order regardless of actual completion order — the first list element always corresponds to the first coroutine argument.

Upgrade path

asyncio.TaskGroup for structured concurrency with automatic error propagation and cancellation (Python 3.11+)

Frequência: HighFormulaicidade: Semi-fixedTipo de construção: syntactic_patternPrioridade de aquisição: Active recallPrioridade de output: OutputTag de espaçamento: Medium-term

Examples

import asyncio async def fetch_data(source_id, delay): await asyncio.sleep(delay) return f"data_from_{source_id}" async def main(): sources = [1, 2, 3] coroutines = [fetch_data(sid, 0.5) for sid in sources] results = await asyncio.gather(*coroutines) print(results) asyncio.run(main())

import asyncio async def check_service(host, port): try: reader, writer = await asyncio.wait_for( asyncio.open_connection(host, port), timeout=2.0 ) writer.close() await writer.wait_closed() return (host, port, True) except (asyncio.TimeoutError, ConnectionRefusedError): return (host, port, False) async def health_check(services): coroutines = [check_service(host, port) for host, port in services] results = await asyncio.gather(*coroutines) for host, port, alive in results: status = "UP" if alive else "DOWN" print(f"{host}:{port} - {status}") async def main(): services = [("example.com", 80), ("example.com", 443), ("example.com", 8080)] await health_check(services) asyncio.run(main())

import asyncio async def fetch_with_retry(url, retries=3): for attempt in range(retries): try: await asyncio.sleep(0.1) if "fail" in url: raise ConnectionError(f"Failed: {url}") return f"ok:{url}" except ConnectionError: if attempt == retries - 1: raise async def main(): urls = [ "https://api.example.com/users", "https://api.example.com/fail/data", "https://api.example.com/posts", ] coroutines = [fetch_with_retry(url) for url in urls] results = await asyncio.gather(*coroutines, return_exceptions=True) for url, result in zip(urls, results): if isinstance(result, Exception): print(f"{url} -> ERROR: {result}") else: print(f"{url} -> {result}") asyncio.run(main())