What are traces and spans in LLM observability?

beginner

← All AI Evals & Observability questions

Re-explain

A trace is the full record of one request through your app, start to finish. A span is one timed step inside that trace, such as a single LLM call or a retrieval query.

Spans nest. Each one has a parent, so a trace forms a tree with one root span. Every span carries a name, start and end times, and a status. Its attributes hold the model, the prompt and response, and token counts with cost.

trace: chat_request (2.4s)  user=u_912  session=s_44
├── span: retrieve_docs     (310ms)  k=5, index=faq_v3
├── span: llm_call          (1.9s)   model=..., in=1200 tok, out=180 tok
└── span: guardrail_check   (60ms)   verdict=pass

This is what makes debugging possible. You see which step was slow, and what the model actually received. You can also attach eval scores to one span or to the whole trace.

Rewriting in plainer words…

This answer doesn't lend itself to a diagram - it reads best . No credits were charged.

Why there's no diagram: “”

The interactive diagram is below the answer - jump to diagram ↓ · Below it, the related concept . Jump to it ↓

The diagram below the answer is the concept . Jump to it ↓

Tailored explanation · switch back to · ·

What should the new diagram focus on?

Guided practice for AI Evals & Observability One question at a time. Answer out loud, get graded, see what you missed.

How well did you know this?

AI:

Saved in this browser - sign in to keep your review list.

How should your speech become text?

Listening… your words appear above as you speak - tap Stop when you're done.

Recording · cr - tap Stop & transcribe when you're done.

Transcribing with AI…

Voice:

Keep going - a few more words and AI can grade it.

Interview lens How interviewers actually use this question

Likely follow-ups

How would you use traces to debug a user-reported bad answer you can't reproduce? Find the trace by user or session ID, read the exact prompt and retrieved docs, replay that input, then add the case to the eval set.
How would you lay out spans for an agent that loops through tool calls and retries? One root span per run, a child span per LLM call and tool call, retries as separate spans with their own status, so the tree shows the path taken.
Full traces with prompts get expensive at scale. How do you decide which ones to keep? Name head vs tail sampling. Keep errors, slow and low-scored traces, sample the rest, and keep cheap metrics for all traffic.
Spans hold raw prompts and responses. What do you do about user PII in them? Redact or mask before export, restrict access and shorten retention for raw text, and keep IDs, token counts and scores so debugging still works.
Why instrument with OpenTelemetry GenAI conventions instead of a vendor SDK's own format? Shared attribute names for model, tokens and operation let any backend read the spans, so you can switch tools without re-instrumenting.

What you can say

A trace is the full record of one request through your app, and a span is one timed step inside it, like a single LLM call or a retrieval query.
Spans nest, so each one has a parent and the trace forms a tree with one root, say chat_request, with retrieval, the model call and a guardrail check under it.
Every span has a name, start and end times and a status, and its attributes hold the model, the prompt and response, and token counts with cost.
That's what makes debugging possible, because I can see which step was slow and exactly what the model received, not what I assumed it received.
You can also attach eval scores to one span or to the whole trace, so a bad score points at the step that produced it.
The catch is that spans full of prompts and responses are sensitive and costly to store, so at scale you need redaction and some sampling.

Weak answers to avoid

Treats traces as plain logs Flat log lines lose timing and parent links. Say a trace is a tree of timed spans, so you can see which step was slow and what fed into it.
Uses trace and span as synonyms They are different levels. A trace is one whole request and a span is one step in it. Mixing them up suggests you have never read a trace view.
Only talks about latency Timing is half of it. For LLMs the value is in the attributes: the exact prompt, response, model and token cost, plus eval scores on the span.
Stops at the definition Interviewers want to know why you'd use it. Say how a trace shows the slow step, or the bad retrieved context behind a wrong answer.

Interview lens

Likely follow-ups, what you can say, and the weak answers to avoid.

Want a quick review of the fundamentals? See the AI Evals & Observability cheatsheet.

← Back to all AI Evals & Observability questions

Pro · $10/mo

50 of 57 AI Evals & Observability answers are in Pro.

Full answers, code samples, and AI explanations that go simpler or deeper. Cancel anytime.

Full answers + code
AI explanations, simpler or deeper
1,000 AI credits / month
Cancel anytime

Get Pro → See plans & pricing →