h2. The Ask

Three fixes to the playground's RAG answer path, found while investigating why RAG spend exceeded ingestion spend.

# *Tracing.* The playground's answer, selection, and planner agents ran with no LangSmith run, so RAG cost was invisible next to document processing. Each agent call now opens a LangSmith run with the same contract as the extraction agents (prompt as messages in, assistant message and usage out, document key in metadata).
# *Selection agent never ran on the hosted task.* {{selection.py}} demanded {{ANTHROPIC_AUTH_TOKEN}} where every other agent accepts {{ANTHROPIC_API_KEY}}. The hosted task only has the key, so every hosted answer since launch came from fused similarity order alone, with a warning log line as the only sign (283 such lines in CloudWatch on 2026-09-08). The selection agent now builds its model through the same credential resolver as the answer model. Measured effect on 558 sweep questions across six KREF filings: 49 questions went from unanswerable to answered, 8 the other way, with the cited source in the first slot 90% of the time versus 40% without selection.
# *The answer model is told which document it is reading.* A chart on a supplement slide carries no date; handed the chart, the answer model refused to call it a quarter's figure (21 such declines measured). The routes now prepend a document block (Title, Issuer, Form, Period, from the documents row) to the user message on every answer path via a {{DocumentIdentity}} built by the route. The system prompt is unchanged. Measured at full context: +6 chart answers on the Q1 2026 supplement, no attributable losses.

h2. Acceptance Criteria

h3. AC1: Playground agent calls (answer, selection, planner) land in LangSmith with usage and cost, filterable by document.

h3. AC2: With only {{ANTHROPIC_API_KEY}} set, retrieval runs the selection agent (no "selection agent failed" fallback).

h3. AC3: Every answer path (fixed, declared scalar and prose, streaming, grounded) opens the user prompt with the document identity block, and works unchanged when no document is given.

h2. Out of scope

Recorded for follow-up:
* Making the selection fallback visible in the response rather than a log line.
* Reducing the answer context (picks-only contexts were measured and cost 4 wrong figures and 10 missed figures per 558 questions at 60% of input).

h2. Current State

In Review. All three fixes on branch {{feature/QUE-366-playground-rag-langsmith-tracing}}, PR #209.

h2. Links

* Epic: QUE-92 (Pipeline & Web UI)
* PR: https://github.com/xmandeng/quber/pull/209
* Related: QUE-329 (hosted playground on AWS), QUE-279 (RAG playground), QUE-316 (playground in the package)
* Standards: see docs/ENGINEERING_STANDARDS.md