The first instinct when wiring observability into an LLM system is to treat the LLM call as just another HTTP outbound call. Within a week the team realises conventional observability is missing everything that matters — the span tells you the call took 4.2 seconds and returned 200 OK but does not tell you token counts, cost in dollars, model variant, prompt template version, eval scores, tool calls, or whether the response was a hallucination versus a useful answer. Conventional observability tells you the system is up; AI observability needs to tell you whether the system is producing useful output at acceptable cost. This article covers the OpenTelemetry GenAI semantic conventions stabilising in late 2025/early 2026, what to instrument and how, the agent trace hierarchy that makes multi-step debugging tractable, how to attribute token cost across a multi-tenant deployment, how to surface eval scores as first-class telemetry, and the dashboards every production AI team should have on day one.
