Distributed tracing

Enables end-to-end tracing of a single request across multiple services by propagating a shared identifier through logs, headers, and RPC calls.

Microservices: debugging a failed user request that traverses 5+ services with no shared log context Distributed systems: correlating logs from multiple services for a single transaction during incident response API gateways: stitching together upstream and downstream request logs when an external partner reports an error

Microservices, distributed systems, observability platforms, API gateways, serverless architectures, event-driven pipelines

generate unique ID → attach to request context → propagate via headers/logs → aggregate in observability backend

• request headers
• structured logging
• OpenTelemetry
• trace context
• log aggregation
• distributed tracing
• span IDs
• W3C Trace Context
• context propagation

• Trace ID: broader scope
• includes timing spans across the full trace Request ID: simpler
• typically scoped to a single service boundary Session ID: identifies a user session rather than a single request Transaction ID: database-flavored
• often used inside a single service

Generating a new correlation ID at each service hop instead of propagating the original — breaks the trace chain and makes cross-service log stitching impossible Logging the correlation ID only on the happy path — error responses lose the ID so clients cannot reference it in bug reports Using non-UUID formats that collide under high throughput — log entries from unrelated requests merge incorrectly Storing correlation IDs in thread-local only — fails when work is handed off to async tasks, thread pools, or message queues Not propagating to downstream async or background work — orphaned tasks become untraceable in incident reviews

Trace ID: spans the entire distributed trace including timing and causality Request ID: typically scoped to a single service boundary Span ID: identifies a single operation within a trace Session ID: identifies a user session, not an individual request

Coming from monolithic apps: may rely on thread-local request context — in async or multi-threaded services, must use context-aware propagation (e.g., contextvars in Python, AsyncLocalStorage in Node.js) Coming from synchronous RPC: may assume the correlation ID travels in function arguments — in event-driven systems, must embed it in message headers or envelope metadata

• Request IDs
• Trace IDs
• Span IDs
• OpenTelemetry context propagation
• structured logging

When NOT to use: single-service applications where logs are already centralized and request-scoped, since the overhead of propagation adds no value. Performance: minimal overhead when the ID is just a UUID string in headers; can become expensive if every log line is enriched with the full trace context tree. Boundary conditions: regenerate the correlation ID at trust boundaries (external API entry points) to prevent log injection and to scope internal traces per external caller.

Reduces mean time to resolution (MTTR) for cross-service incidents from hours to minutes by enabling single-query log aggregation across the entire request path.

Like a passport stamped at every border crossing — each service checks the same ID so you can reconstruct the full journey from scattered log entries.

Distributed tracing with OpenTelemetry (W3C Trace Context with span IDs and parent-child relationships)