Files
coder/docs/reference/api
Thomas Kosiewski 5393f9d5f4 feat: stream advisor reasoning in tool cards (#29742)
Show the advisor’s live reasoning under **Thinking**, separately from
**Advice**, in one tool card with the original question and title. Keep
nested reasoning out of the parent assistant’s reasoning and saved chat
history.

Clear both live channels on retry and remove transient reasoning when
the call finishes. Cover streaming, retry, error, and interrupted-output
behavior with regression tests, and regenerate the SDK and API schemas.

This also changes a shared seam for every tool, not only the advisor:
live output for a tool call that is already saved now completes that
call's card, and the separate live-row card for it is removed.

Remote UAT passed on `a39bbe4c21`: deterministic provider tests covered
streamed reasoning, retry, errors, and refresh; a real-provider run
confirmed advice streaming and card association. The real model did not
emit reasoning. The intermittent, unattributed React console warning is
tracked in #29741.

<details>
<summary>Remote UAT evidence (deterministic provider)</summary>

![One advisor card showing its question, Thinking, and
Advice](https://github.com/user-attachments/assets/edd05bf2-59b7-48c2-a608-2fd5a4fabcc8)

Streaming, completion, and refresh:


https://github.com/user-attachments/assets/dd1f3a24-a365-40c8-a1d9-b78cd17ef4c5

</details>

---

<details>
<summary>📋 Implementation Plan</summary>

# Plan: Stream advisor reasoning in Coder Agents

## Goal and delivery boundary

Show nested advisor reasoning as live **Thinking** in its advisor tool
card. Keep it separate from live **Advice**, normal parent-assistant
reasoning, and the final persisted advisor result.

Deliver one focused PR: optional SDK field, nested publisher, transient
frontend state, existing card rendering, generated API artifacts, and
regression tests. Do not add provider features, persistence, new stream
event types, UI redesigns, or unrelated cleanup.

The implementation already exists and has been reconciled with current
main. The implement-flow implementation phase is an acceptance audit,
not a rewrite. Preserve correct behavior and current architecture.
Change only demonstrated gaps. A no-op implementation result is valid;
do not manufacture an empty commit.

## Current integration points

1. `codersdk/chats.go`: add optional `ReasoningDelta string` with
`json:"reasoning_delta,omitempty" variants:"tool-result?"`. Preserve it
in `StripInternal` alongside other live fields. Register the same
NUL-encoding policy as `ResultDelta` in `chatprompt` so the SDK field
coverage and round-trip tests remain complete. Regenerate SDK types and
API documentation with `make gen`.
2. `coderd/x/chatd/chatadvisor/runner.go`: route assistant text and
reasoning through separate optional callbacks on `RunAdvisorOptions`.
Install the publisher if either callback exists. Preserve empty-delta
and role guards. Use current `GenerateAssistant` and `chatretry.Retry`.
Rename the reset callback to `OnLiveOutputReset` because it clears
advice and reasoning. Keep `extractAdvisorText` text-only and existing
budget/refund behavior.
3. `coderd/x/chatd/chatadvisor/tool.go`: use the current
context-injected `MessagePartPublisherFromContext`; do not reintroduce
the old ToolOptions publishing callbacks or chatd.go wiring. Preserve
the missing-publisher error and the nonempty tool-call-ID guard. Publish
reasoning as role `tool`, part type `tool-result`, tool name `advisor`,
matching `tool_call_id`, and `reasoning_delta`. Use existing
`result_reset` to reset both live channels on retry.
4.
`site/src/pages/AgentsPage/components/ChatConversation/streamState.ts`
and `types.ts`: retain a single transient reasoning string. Append
deltas exactly as plain text, never JSON-parse or trim them during
accumulation. Reasoning makes the tool streaming. An empty
`result_delta` must not swallow a nonempty `reasoning_delta`. Reset
removes the entry. A final result or `is_error` clears reasoning and
streaming state. Pass reasoning through `buildStreamTools` into
`MergedTool`.
5. `MessageBlocks.tsx` and `ChatElements/tools/Tool.tsx`: forward
reasoning at both current Tool instantiation paths and through
AdvisorRenderer. Do not restore the old ConversationTimeline rendering
implementation.
6. `AdvisorTool.tsx`: use current `ToolCall.Root`, `ScrollArea`,
`Response`, and semantic theme tokens. Display Thinking only when
running and trimmed reasoning is nonempty. Display live advice
separately when present. Keep completed advice, empty/loading, limit,
and error behavior unchanged. Reasoning-only running cards must not show
the empty-advice fallback. Do not restore removed MUI dependencies.
7. `messageParsing.ts`, `streamState.ts`, and `ChatPageContent.tsx`:
associate live tool results with matching durable tool calls. Preserve
the invocation's question, intent, and metadata. A durable final result
remains authoritative, including explicit null results. Remove only the
overlaid result and its block from the separate live row; keep unrelated
results and live call/result pairs unchanged. This closes the
durable-call/live-result split found during integration testing.

## Acceptance criteria

1. Nested reasoning reaches the dedicated runner callback and the client
as tool-scoped live output for the same advisor call as advice.
2. Advisor reasoning is not parent assistant reasoning, final advice,
input to the parent's continuation, or persisted chat history.
3. Retry clears stale reasoning and advice before fresh output. Final
success and bare error clear transient reasoning.
4. Raw whitespace and JSON-looking reasoning survive concatenation
unchanged. Empty advice deltas do not hide reasoning.
5. Reasoning-only and reasoning-plus-advice states render in exactly one
existing advisor card per invocation, retaining its question and intent
title when the call is already durable. Completed cards never display
transient thinking, even if handed a stale prop. Other tool and normal
assistant reasoning behavior is unchanged.
6. Generated artifacts match source. Final candidate passes relevant Go
tests, frontend tests, stories, typecheck, lint, and pre-commit checks,
or has explicit verified blockers. Old evidence is not final-candidate
evidence.

## Phases and quality gates

1. Reconcile existing work onto current main while retaining a recovery
snapshot. Remove obsolete workarounds, not current upstream behavior.
Gate: no conflicts, focused diff, preserved original work.
2. Audit regressions at the existing runner, tool publisher, parent-chat
stream, and frontend state boundaries. Add one missing behavior at a
time. Reproduce failures before fixes. Gate: failing tests demonstrate
the defect and pass after the smallest fix.
3. Simplify only within this feature. Use one reasoning string and
existing publishers/components. Gate: tests remain green, generated diff
is scoped, FE1 through FE10 pass.
4. Commit the candidate with repository hooks enabled, then run the
packaged implement-flow audit, validation, remote UAT, and polish
stages. Gate: readyForPr and verified remote evidence for the exact
final candidate. If polish changes HEAD, refresh the UAT evidence for
that SHA.
5. Create a reviewable PR and complete green CI plus bounded Codex
review on its exact head. Get one fresh independent readiness
recommendation. Gate: no unresolved review threads, required evidence
recorded, and a justified readiness decision. Do not merge.

## Regression coverage

- Runner: reasoning and advice callbacks remain separate; final advice
and usage count stay correct. Retain existing retry/refund tests.
- Tool: all reasoning/advice events carry role tool, type tool-result,
advisor name, and the invocation ID. Without an ID, emit no live output.
Retry emits stale reasoning/advice, reset, then fresh reasoning/advice.
- Chatd: extend `TestAdvisorHappyPath_RootChat` with deterministic
`openai-compat` reasoning chunks. Hold the nested response until the
live subscriber captures expected reasoning and advice. Observe the
complete relevant event set, assert no parent-assistant reasoning, and
check continuation and persisted content contain no advisor reasoning.
Extend only the test fixture needed to emit `reasoning_content`; do not
change production adapters.
- Frontend: extend streamState tests for exact accumulation, whitespace,
JSON-looking text, reasoning-only streaming, empty advice delta, reset,
final result, and bare error.
- Durable/live integration: replay a durable assistant tool call
followed by live reasoning and advice results. Assert the durable card
receives both channels and the live row no longer owns the same result
or block. Preserve unrelated IDs and live calls. Clear the overlay on
reset; prefer durable final results, including null. Cover the complete
display path with a real-store ChatPageContent story.
- Visual stories: running reasoning-only, running reasoning-plus-advice,
and completed with stale reasoning. Keep existing limit/error stories.
Stories set up visual states without assertions, following current FE1.

## Validation

Run targeted checks while iterating, then the broader gates:

```sh
umask 022
go test ./coderd/x/chatd/chatadvisor ./coderd/x/chatd ./codersdk -run 'Advisor|ChatMessagePart|ChatStream' -count=1
pnpm -C site exec vitest run --project=unit src/pages/AgentsPage/components/ChatConversation/streamState.test.ts
pnpm -C site exec vitest run --project=storybook src/pages/AgentsPage/components/ChatElements/tools/AdvisorTool.stories.tsx
make gen
make fmt
make test RUN='Advisor|ChatMessagePart|ChatStream'
pnpm -C site lint:types
pnpm -C site check
pnpm -C site lint
make lint
make pre-commit
```

Keep native exit codes and logs. Do not hide a failed check, bypass
hooks, or fix unrelated baseline problems in this PR. Stage intentional
generated/formatting changes before pre-commit's clean-diff checks.
Report exact blockers and risks.

## Remote dogfood UAT

Use the coder-agents-uat skill and an independent critical Coder Agent,
not a local-only substitute. Push only the named feature branch. Test
its exact SHA in an isolated workspace/deployment without changing
shared model credentials or configuration. Use an isolated user, chat,
and model config.

Setup:

1. Read the branch diff and UAT scope. Record the branch and SHA.
2. Start Coder with the repository-supported `./scripts/develop.sh
--port-offset` (or `CODER_DEV_PORT_OFFSET=true`). Do not replace it with
a manual server launch. Record effective API/UI URLs and wait for
readiness.
3. Prefer a deterministic mock OpenAI-compatible provider that emits
advisor reasoning and advice separately, and a retry after stale output.
Use only isolated configuration. Never print or capture secrets.
4. Start a named agent-browser session. Attempt `agent-browser skills
get core`; record the installed version/fallback if unavailable. Record
video before the scenario, then install any stream capture since
recording may recreate browser context.

Scenario:

1. Create a normal advisor-enabled root chat. Do not use Plan Mode:
current product behavior excludes the advisor there.
2. Capture the initial state. Prompt the parent to call advisor.
3. While the tool is running, capture reasoning-only and
reasoning-plus-advice states. Verify exactly one advisor card per
tool-call ID, with its original question and intent title. Verify
Thinking and Advice stay separate inside that card and the parent
thinking area does not receive nested reasoning.
4. Capture the relevant WebSocket event set. Verify role tool, type
tool-result, advisor name, matching tool-call ID, and reasoning_delta.
An advisor-filtered subset is insufficient to prove absence of parent
reasoning.
5. Let the advisor finish. Verify final advice remains, Thinking
disappears, and refresh does not restore transient reasoning.
6. Exercise retry with the deterministic provider if feasible. Verify
stale reasoning/advice clears before fresh output. If not exercised,
record the omission and remaining test evidence; do not claim it was
dogfooded.
7. Capture console errors before and after. Inspect the screenshots and
semantic video frames, not only successful media decoding.

Required evidence: exact tested SHA; remote chat URL; setup command and
effective URLs; non-secret provider/model names;
before/thinking/thinking-plus-advice/completed screenshots; a short
streaming video; complete relevant stream evidence; refresh and retry
observations; console results; explicit omissions; and owned-resource
cleanup receipts. Keep evidence under `.mux-uat/round-<n>/`, excluded
from source commits. When publishing evidence to GitHub, use native `gh
--attach`, not cookie-based uploads or gh-image.

Cleanup: stop only owned develop/provider processes and named browser
sessions. Do not use unscoped process sweeps, shared database resets, or
`agent-browser close --all`.

</details>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`coder:openai/gpt-6-astra` • Thinking: `xhigh`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-09-25 07:58:22 +02:00
..