h2. The Ask

Three fixes to the playground's RAG answer path, found while investigating why RAG spend exceeded ingestion spend.

# *Tracing.* The playground's answer, selection, and planner agents ran with no LangSmith run, so RAG cost was invisible next to document processing. Each agent call now opens a LangSmith run with the same contract as the extraction agents (prompt as messages in, assistant message and usage out, document key in metadata).
# *Selection agent never ran on the hosted task.* {{selection.py}} demanded {{ANTHROPIC_AUTH_TOKEN}} where every other agent accepts {{ANTHROPIC_API_KEY}}. The hosted task only has the key, so every hosted answer since launch came from fused similarity order alone, with a warning log line as the only sign (283 such lines in CloudWatch on 2026-09-08). The selection agent now builds its model through the same credential resolver as the answer model. Measured effect on 558 sweep questions across six KREF filings: 49 questions went from unanswerable to answered, 8 the other way, with the cited source in the first slot 90% of the time versus 40% without selection.
# *The answer model is told which document it is reading.* A chart on a supplement slide carries no date; handed the chart, the answer model refused to call it a quarter's figure (21 such declines measured). The routes now prepend a document block (Title, Issuer, Form, Period, from the documents row) to the user message on every answer path via a {{DocumentIdentity}} built by the route. The system prompt is unchanged. Measured at full context: +6 chart answers on the Q1 2026 supplement, no attributable losses.

h2. Acceptance Criteria

h3. AC1: Playground agent calls (answer, selection, planner) land in LangSmith with usage and cost, filterable by document.

h3. AC2: With only {{ANTHROPIC_API_KEY}} set, retrieval runs the selection agent (no "selection agent failed" fallback).

h3. AC3: Every answer path (fixed, declared scalar and prose, streaming, grounded) opens the user prompt with the document identity block, and works unchanged when no document is given.

h3. AC4: The playground transcript no longer displays the list of retrieved chunks under an answer. The list stays in the chat payload as data; only the rendering is commented out, so it is one import away for a developer.

h2. Out of scope

Recorded for follow-up:
* Making the selection fallback visible in the response rather than a log line.
* Reducing the answer context (picks-only contexts were measured and cost 4 wrong figures and 10 missed figures per 558 questions at 60% of input).

h2. Current State

In Review. All three fixes on branch {{feature/QUE-366-playground-rag-langsmith-tracing}}, PR #209.

Commit 349b9ce adds AC4 (retrieved-chunks panel hidden in the UI).

h2. Links

* Epic: QUE-92 (Pipeline & Web UI)
* PR: https://github.com/xmandeng/quber/pull/209
* Related: QUE-329 (hosted playground on AWS), QUE-279 (RAG playground), QUE-316 (playground in the package)
* Standards: see docs/ENGINEERING_STANDARDS.md