Distributed Tracing

Reviewed & published by Brayan K

When one request flows through many services, a single log file can't tell you where the time went. Distributed tracing stitches the whole journey together so you can follow one request — and find the slow span — across every service it touches.

Part of the free Node.js course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

In this lesson you'll learn the core ideas of observability — traces, spans, and context propagation — and set up the @opentelemetry/sdk-node SDK with auto-instrumentation, exporting to a backend like Jaeger or Zipkin.

What You'll Learn in This Lesson

• What observability and distributed tracing are
• Traces, spans, and the parent-child span tree
• Setting up @opentelemetry/sdk-node
• Auto-instrumenting http, express, and pg
• Context propagation across service boundaries
• Exporting spans to Jaeger or Zipkin

1️⃣ Set Up the OpenTelemetry SDK

Tracing starts with the SDK. You configure NodeSDK with a service name, an exporter (where spans go), and a set of instrumentations. The catch: this file must load before the rest of your app, because auto-instrumentation patches libraries at the moment they're required.

// install: npm install @opentelemetry/sdk-node \
//   @opentelemetry/auto-instrumentations-node \
//   @opentelemetry/exporter-trace-otlp-http
//
// tracing.js — MUST be loaded FIRST, before any other module, so the
// auto-instrumentation can patch http, express, pg, etc. at require time.
// Start it with: node --require ./tracing.js server.js
import { NodeSDK } from "@opentelemetry/sdk-node";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";

const sdk = new NodeSDK({
  serviceName: "orders-service",
  // An EXPORTER ships finished spans to a backend (Jaeger, Zipkin, etc.)
  traceExporter: new OTLPTraceExporter({
    url: "http://localhost:4318/v1/traces", // OTLP/HTTP endpoint
  }),
  // Auto-instrument popular libraries with zero manual span code.
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start(); // begin collecting traces
console.log("OpenTelemetry tracing started");

getNodeAutoInstrumentations() is the magic: it patches common libraries — http, express, pg, and more — so every request and query produces a span without you writing any tracing code. Load it with node --require ./tracing.js server.js.

2️⃣ Traces and Spans

A span is one timed unit of work; a trace is the tree of spans that share a trace ID. Auto-instrumentation gives you spans for free, but you can also create your own to time a specific piece of business logic. A span started inside another becomes its child, building the tree you see in the UI.

// Automatic spans are great, but you can also create your own to time
// a specific chunk of business logic. A TRACE is a tree of SPANS that
// share one trace ID; nested spans become children of the active span.
import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("orders-service");

async function placeOrder(cart) {
  // Start a span for the whole operation.
  return await tracer.startActiveSpan("placeOrder", async (span) => {
    span.setAttribute("cart.items", cart.length); // searchable metadata

    // A nested span automatically records placeOrder as its PARENT,
    // building the parent -> child tree you see in the trace UI.
    await tracer.startActiveSpan("chargePayment", async (child) => {
      await chargeCard(cart);
      child.end(); // always end every span you start
    });

    span.end();
  });
}

Two rules matter: set attributes so spans are searchable (by user, cart size, status), and always end every span you start. An unended span never records its duration and can leak.

• • Span — a single timed operation with attributes and a parent.
• • Trace — all spans sharing one trace ID, forming a tree.
• • startActiveSpan — start a span and make it the current parent.
• • span.end() — close the span and record its duration.

3️⃣ Context Propagation Across Services

What makes tracing distributed is context propagation. When one service calls another, OpenTelemetry injects the current trace and span IDs into a traceparent header. The receiving service reads it and continues the same trace, so a request that crosses four services shows up as one connected trace, not four disconnected ones.

// CONTEXT PROPAGATION is what makes a trace "distributed". When this
// service calls another over HTTP, OpenTelemetry injects the current
// trace + span IDs into a W3C "traceparent" header. The auto HTTP
// instrumentation does this for you automatically:
//
//   traceparent: 00-<trace-id>-<span-id>-01
//
// The RECEIVING service reads that header, continues the SAME trace,
// and links its spans as children. So one user request that flows
// gateway -> orders -> payments -> email shows up as ONE connected
// trace, not four disconnected ones.

import { trace } from "@opentelemetry/api";

async function callPayments(order) {
  // Because http is auto-instrumented, this fetch already carries the
  // traceparent header — the payments service joins the same trace.
  const res = await fetch("http://payments/charge", {
    method: "POST",
    body: JSON.stringify(order),
  });
  return res.json();
}

Because http is auto-instrumented, the traceparent header rides along automatically — you don't set it by hand. This correlation is what lets you open one slow request and see, span by span, exactly which service ate the time.

Your turn. Fill in the ___ blank so the span is properly closed.

// 🎯 YOUR TURN — fill in the ___ blanks to create and finish a span.
import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("orders-service");

async function loadUser(id) {
  return await tracer.startActiveSpan("loadUser", async (span) => {
    span.setAttribute("user.id", id);

    const user = await db.findUser(id);

    span.___();   // 👉 every span you start must be closed — which method?
    return user;
  });
}

// ✅ The blank is end. A span that is never ended never shows its duration
//    and can leak, so always call span.end() when the work is done.

Mini-Challenge: Find the Bottleneck

Add a custom span around a deliberately slow function, for example one that awaits a 500ms timer, and give it an attribute like slow.reason. Trigger the request, then open the trace in Jaeger or Zipkin and find your span in the waterfall. Confirm its duration stands out against the fast spans around it. That's the everyday workflow: a request feels slow, you open its trace, and the longest span tells you exactly where to look.

Common Errors (and the fix)

• Starting the SDK too late — If your app requires http or pg before the SDK starts, those libraries are never patched and produce no spans. Load tracing first.
• Forgetting to span.end() — An unended span has no duration and can pin memory. Every started span must be ended.
• No exporter configured — Spans are collected but never sent, so the backend stays empty. Configure an exporter pointing at Jaeger, Zipkin, or a collector.
• Breaking propagation with custom HTTP — A bespoke client that strips headers drops the traceparent and splits the trace. Use instrumented clients or inject context manually.
• Over-instrumenting — A span for every tiny function floods the backend and costs money. Trace meaningful units of work and sample in production.

Pro Tips

• 💡 Export through a Collector in production so your app sends telemetry to one OTLP endpoint and the collector fans it out to Jaeger, Zipkin, or your vendor.
• 💡 Add rich attributes (user id, route, status code) to spans so you can search and group traces, not just stare at one at a time.
• 💡 Sample under load: tracing every request at high volume is expensive. Use head- or tail-based sampling to keep the interesting traces while controlling cost.

📋 Quick Reference — OpenTelemetry in Node

🎉 Lesson Complete!

• ✅ Tracing follows one request across many services
• ✅ A span is one timed operation; a trace is the tree of spans
• ✅ @opentelemetry/sdk-node bootstraps tracing — load it first
• ✅ Auto-instrumentation patches http, express, and pg for free
• ✅ Context propagation via traceparent connects services
• ✅ Exporters ship spans to Jaeger or Zipkin to find bottlenecks

Practice quiz

Continue this course

• Previous: gRPC Services
• Next: Database Transactions & ACID — Keep data consistent with BEGIN/COMMIT/ROLLBACK, isolation levels, and deadlock handling
• Quick reference: Node.js cheat sheet

Frequently asked questions

What is the difference between a trace and a span?

A span is the basic building block: it represents a single named unit of work, such as handling an HTTP request, running a database query, or calling another service. Every span has a start time, a duration, a set of attributes, and a parent. A trace is the whole picture: it is the collection of all spans that share a single trace ID, arranged as a tree that shows how one request flowed through your system. So when a user clicks buy, the top-level span might be the incoming request, with child spans for the database read, the payment call, and the email send. The trace is that entire tree; each box in it is a span.

What is context propagation and why does it matter?

Context propagation is how a trace stays connected as a request crosses from one service to another. When service A calls service B over HTTP, it injects the current trace ID and span ID into a header, almost always the W3C standard traceparent header. Service B extracts that context and continues the same trace, recording its work as child spans rather than starting a brand-new, disconnected trace. Without propagation you would get four separate traces for a request that touched four services and no way to see the end-to-end path. With it, you see one unified trace that follows the request everywhere it goes, which is the entire point of distributed tracing.

Why must the OpenTelemetry SDK start before the rest of my app?

Auto-instrumentation works by monkey-patching libraries at the moment they are required, wrapping functions like http.request or a pg query so they emit spans. If your application code requires http or express or pg before the SDK has had a chance to patch them, those original, unpatched versions are already in memory and will never produce spans. That is why the recommended setup is to put the SDK initialization in its own file and load it first, for example with node --require ./tracing.js server.js or as the very first import. Starting it early guarantees every library is wrapped before your code uses it.

Where do traces actually go, and how do I view them?

Your application collects spans in memory and then hands them to an exporter, which ships them to a tracing backend. Common backends are Jaeger and Zipkin, both of which give you a web UI where you can search traces and see the waterfall of spans with their durations. In production the usual pattern is to export over OTLP to an OpenTelemetry Collector, a separate process that receives, batches, and forwards telemetry to one or more backends, so your app does not need to know the final destination. Either way, once a trace lands in the backend you can open one slow request and immediately see which span, and therefore which service or call, ate the time.