request collapsing

Resilience Patterns

Meaning

Request collapsing merges multiple identical in‑flight requests into a single backend call and distributes the single response to all original callers. It addresses the problem of redundant network traffic and overload on downstream services when many clients request the same resource at the same time. The technique is triggered whenever concurrent code initiates the same request before a prior one has completed.

Primary Function

Request deduplication

Communicative Purpose

Prevents redundant network calls by merging identical in‑flight requests into one and sharing the result.

Pattern

detect duplicate in‑flight requests → combine them into a single request → share the resulting response with all callers

Função primária

Request deduplication

Propósito comunicativo

Prevents redundant network calls by merging identical in‑flight requests into one and sharing the result.

Situações de gatilho

Web APIs: multiple clients request the same endpoint within milliseconds Microservices: repeated calls to a downstream service for the same key in a short window Client‑side data fetching: UI components request identical data concurrently

Contextos

high‑throughput REST services, client‑side data fetching libraries, distributed caching layers, serverless functions handling burst traffic

Padrão

detect duplicate in‑flight requests → combine them into a single request → share the resulting response with all callers

Colocados típicos

cache
deduplication
thundering herd
batch processing
semaphore

Substituições comuns

request coalescing (focuses on transport‑layer merging)
batch request (groups different keys together) – trade‑off: coalescing reduces latency but may increase response size

Erros comuns

Treating unrelated requests as identical and returning wrong data – caused by using only the URL string without query parameters; Forgetting to clear the pending‑request map on error – leads to stale promises and deadlocks; Collapsing requests that have side‑effects – duplicates may suppress required state changes.

Similar / contraste

caching (stores completed responses, not in‑flight ones); throttling (limits request rate but does not merge them); debouncing (delays execution until silence, unsuitable for immediate responses).

Interferências

Coming from JavaScript: assuming Promise.all automatically deduplicates calls → in Python you must implement explicit request collapsing logic.

Família do chunk

request deduplication
cache stampede mitigation
thundering herd protection
batch processing
request coalescing

Nuance

Do not use when requests have side‑effects because collapsing would suppress needed actions; Collapsing adds a small coordination overhead and a lock, which can affect latency under low load; It works best when the request is idempotent and the response size is comparable to a single request.

Efeito pragmático

Reduces bandwidth consumption and downstream load, preventing thundering‑herd failures and improving overall system stability under burst traffic.

Dica de memória

Think of a coffee shop barista who groups identical orders placed at the same moment into one preparation batch, then serves each customer the same cup.

Nota

Request collapsing is most effective for read‑only, idempotent operations; for mutable operations consider other concurrency controls.

Upgrade path

Implement distributed request collapsing using a shared cache and consistent hashing to coordinate across multiple service instances.

Frequência: HighFormulaicidade: Semi-fixedTipo de construção: conceptPrioridade de aquisição: Active recallPrioridade de output: BothTag de espaçamento: Medium-term

Examples

import asyncio _pending = {} async def fetch(url): if url in _pending: return await _pending[url] fut = asyncio.Future() _pending[url] = fut try: # simulate network call await asyncio.sleep(1) result = f"data for {url}" fut.set_result(result) return result finally: _pending.pop(url, None) async def main(): tasks = [fetch('https://example.com') for _ in range(5)] results = await asyncio.gather(*tasks) print(results) asyncio.run(main())

import threading _pending = {} _lock = threading.Lock() def fetch(url): with _lock: if url in _pending: event, result = _pending[url] else: event = threading.Event() result = None _pending[url] = (event, result) if not event.is_set(): # simulate I/O import time; time.sleep(1) result = f"data for {url}" with _lock: _pending[url] = (event, result) event.set() else: event.wait() with _lock: _, result = _pending[url] with _lock: _pending.pop(url, None) return result threads = [threading.Thread(target=lambda: print(fetch('https://example.com'))) for _ in range(5)] for t in threads: t.start() for t in threads: t.join()

from functools import wraps from collections import defaultdict _pending = defaultdict(list) def collapse(func): @wraps(func) def wrapper(key, *args, **kwargs): if _pending[key]: # register a callback placeholder fut = [] _pending[key].append(fut) return fut else: _pending[key].append(None) # marker for the leader result = func(key, *args, **kwargs) # notify all waiting callers for waiter in _pending[key][1:]: if waiter is not None: waiter.append(result) _pending.pop(key, None) return result return wrapper @collapse def get_data(key): import time; time.sleep(1) return f"value for {key}" # Simulate concurrent calls print(get_data('alpha')) print(get_data('alpha')) # returns the same result without a second sleep