An incomplete distributed trace is worse than no trace at all. The span for a slow database call exists, the API handler that triggered it exists, but they are not connected. The trace view shows two separate traces where one should be. The on-call has to manually join them by timestamp, which is exactly the work the tracing system was supposed to remove.
Disconnected spans are almost always a context propagation problem. Tracing libraries do most of the work — they propagate context across HTTP calls and database drivers automatically — but specific boundaries break the chain. Queues. Scheduled jobs. Outbound HTTP clients that bypass the library. Async runtimes with broken context awareness.
This guide is about finding and fixing these boundaries.
What Context Actually Is
A trace context is two values: a trace ID (constant across the whole trace) and a span ID (the parent for the next span). Together they let a child span attach itself to the right parent in the trace tree.
The W3C Trace Context specification defines the wire format:
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
Four fields: version, trace ID, span ID, flags. Tracing libraries inject this header into outbound HTTP requests and extract it from inbound ones automatically.
The propagation works when:
- An incoming request carries
traceparent. The library extracts it and creates a child span. - An outgoing request is started while a span is active. The library injects the current
traceparentinto the headers. - Both ends use a compatible propagator (W3C, or the legacy Jaeger / B3 formats with a configured propagator).
It fails when one of these steps is missing.
The Common Boundaries Where It Breaks
Outbound HTTP without the instrumented client
OTel's auto-instrumentation hooks into specific HTTP clients — Guzzle in PHP, axios in Node, requests in Python. Code that uses a different client bypasses the instrumentation, and the outbound request goes out without a traceparent header.
Fix: explicitly inject the header in the unconventional client, or switch to the instrumented one.
from opentelemetry import propagators
headers = {}
propagators.inject(headers)
response = my_special_client.get(url, headers=headers)
Background jobs and queues
This is the most common failure mode. A request handler enqueues a job; a worker picks it up later. The worker is a separate process with no inherent connection to the enqueuing context.
The pattern is to serialize the context into the job payload and restore it in the worker:
// Producer
class GenerateInvoiceJob implements ShouldQueue
{
public function __construct(
public int $clientId,
public array $traceContext = [],
) {
$context = Context::getCurrent();
TraceContextPropagator::getInstance()->inject($this->traceContext, $context);
}
}
// Consumer
public function handle(): void
{
$context = TraceContextPropagator::getInstance()->extract($this->traceContext);
$scope = Context::storage()->attach($context);
try {
// do work; spans attach to the producer's trace
} finally {
$scope->detach();
}
}
The job carries the trace context as data. The worker reconstitutes it before doing work.
This pattern needs to be repeated for every job class — there is no clean way to make it automatic without overriding the queue's serialization. Wrap it in a base class or trait so every new job class inherits the behavior.
Scheduled jobs and cron
A scheduled job has no parent span — it was not triggered by a request. The cleanest pattern is to start a new root span for the scheduled run and let everything inside it inherit:
with tracer.start_as_current_span("daily_invoice_export") as span:
span.set_attribute("scheduled_at", datetime.now().isoformat())
run_export()
Without this, every span inside the scheduled job starts as a separate root. The trace UI shows hundreds of orphan traces instead of one.
Async runtimes
Languages with async runtimes — Python's asyncio, Node's promises, Go's goroutines — handle context differently from synchronous code. Context that lives in thread-local storage in a sync codebase needs explicit handling in async code.
OTel's language SDKs each have async-aware variants:
- Python:
opentelemetry.context.copy_context()andattach_to_task() - Node:
AsyncLocalStorage-based propagation in modern OTel JS - Go: pass
context.Contextthrough every call
The gotcha is when code mixes sync and async. A library that uses asyncio.create_task() without copying context loses the trace. Auditing for these cases is part of getting async propagation right.
Outbound calls across mesh / proxy boundaries
A service mesh (Istio, Linkerd) often re-emits requests as the proxy. If the proxy strips or doesn't forward the traceparent, the receiving service sees a request with no context.
Fix: configure the mesh to propagate W3C trace headers. Most do this by default but require explicit configuration to forward additional baggage. Verify in production by looking at actual request headers, not the docs.
Webhooks and external callbacks
When your service makes an outbound request to a third-party API, you include the traceparent. When that API later calls back into your webhook, the context is whatever the third party chose to send — usually nothing. The webhook handler starts a fresh root span.
The practical fix: log enough context (your internal correlation ID) when you initiate the outbound call so you can manually join the traces in queries. Tracing across third-party boundaries is rarely worth solving with infrastructure; it is a query-time problem.
How to Find the Broken Boundaries
A few diagnostic patterns that surface propagation issues:
"Orphan trace" queries. Most tracing backends let you query for traces with no parent. A large number of orphans usually means a scheduler or queue boundary is missing propagation.
Per-service trace length distribution. Plot the count of spans per trace, segmented by service. A service that only ever appears as the root of one-span traces is probably starting fresh contexts where it should be inheriting.
Manual sampling of broken traces. In an incident, follow one specific user action through the system. Note where the trace ends. That boundary is where propagation broke.
Flame graphs and dependency views in the tracing UI are surprisingly useful for this. A service that should appear as a child but never does is a sign of a missing propagator.
Verify in Production, Not in Tests
Unit tests rarely catch propagation problems. The trace context is set up correctly in tests because the test runs synchronously, in one process, with no real queue or scheduler.
The verification has to happen with production-shape traffic. Send a real request through the system, then check the trace in the backend. Look for:
- All the services you expect, all in the same trace.
- Spans for background jobs that were triggered by the request.
- Database calls under the appropriate service span.
- No orphans for any work that should have been a child.
Do this verification once per service deployment for the first few weeks. After context propagation is solid, occasional audits catch regressions.
A Propagation Checklist
For any service you maintain:
- All outbound HTTP uses an instrumented client, or explicitly injects context.
- Every queue job class serializes and restores trace context.
- Scheduled jobs start their own root span.
- Async code paths use the language's context-aware primitives.
- Service mesh forwards W3C headers.
- Webhooks log a correlation ID for cross-system joins.
- Production traces show the expected service topology end-to-end.
Most teams have two or three items unchecked. The cost is real — incomplete traces erode trust in the tool and add minutes to every investigation. The fixes are mechanical once the boundaries are identified.
Traces showing up disconnected in production? We help teams find and fix the propagation boundaries breaking their tracing pipeline. scopeforged.com