mirror of
https://github.com/coder/coder.git
synced 2026-11-22 03:18:31 +08:00
feea5ab959f59cd2135443ed2a0c19bf7643db8f
98
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5cbf086517 |
feat: send tool result media to providers like user files (#31383)
Part of [CODAGT-1430](https://linear.app/codercom/issue/CODAGT-1430). This is the first of four stacked PRs. Today a PDF in an MCP tool result reaches no provider. On Anthropic we strip it in chatd. On OpenAI Chat Completions fantasy swaps it for a "cannot be displayed" note, and on Responses it sends a placeholder that points at a user message it never sends. coder/fantasy#73 and coder/fantasy#74 make fantasy convert tool result media with the converter it already uses for user files: an Anthropic document inside `tool_result`, an `input_file` inside `function_call_output` on Responses, and a user file part straight after the tool message on Chat Completions. Both have merged, and this PR pins `charm.land/fantasy` to the coder_2_33 tip that has them. Chat Completions file parts now refuse text files, as Responses already did, so `AcceptsFilePartMediaType` follows. The converters take a filename, so an MCP embedded resource blob is now named after the last element of its URI path, and we store the name with the tool result so later turns send the same one. OpenAI picks the file type from the extension, so we append the right one when it's missing or wrong. We drop names with control characters, as Postgres `jsonb` rejects NUL, and we redact the name like the rest of the tool response, since a URI like `https://host/sse/<token>` would otherwise put the token in the chat history. There's also a test proving aibridge's Chat Completions re-encode keeps a user file part, as that's where these PDFs end up. Anthropic still strips tool result PDFs until the next PR in the stack. |
||
|
|
082668fbc1 |
feat: run computer use on OpenAI-compatible AI providers (#31895)
Computer use only ran on Anthropic and OpenAI AI providers. Chatd derives the tool flavor from the AI provider type, and `openai-compat` had no flavor, so every self-hosted model was marked unavailable for `type=computer_use`, including the model a single-provider deployment already uses for chat. Hosted computer tools are not an option on those servers: an OpenAI-compatible endpoint accepts a `computer_20250124` tool over `/v1/messages` and a `computer-use-preview` tool over `/v1/responses` and then drops both, so the model never sees a way to act. Computer use is now declared as an ordinary function tool for those providers. The model sends one action per call and every call answers with a fresh screenshot, which is the shape these models already handle through the shared action runner. The tool omits `zoom`, which the agent desktop cannot perform. The desktop clamps a point outside the screen to its edge, so the tool rejects a coordinate outside the declared screenshot with an error that states the valid range, and a model that aims off the screen learns that it missed. Media tool results reach the model because the pinned fantasy fork promotes them to user-role messages, so no dependency changed. Coder can't verify an arbitrary OpenAI-compatible model, so those models keep the unverified warning icon in the admin UI. The virtual desktop docs, the Coder Agents index, the chatd architecture doc, and the organization model settings copy now list OpenAI-compatible providers as supported, and the docs say what to check before using one. Three shared behaviors needed fixing to make this usable. A coordinate of `[0,0]` is the top-left pixel and decoded as an omitted field, so the desktop rejected those clicks with a missing-coordinate error; coordinates now decode by key presence, which also keeps an absent coordinate absent. A `scroll_amount` of zero or less, or a key `repeat` of zero, was also forwarded as omitted, so the desktop scrolled its default of three clicks or pressed the key once; the runner now rejects both, and the function tool declares a minimum of 1 for each. And because a chat completions model sends one action per call, it plans every call of a step against one screen, so a failed action now halts the later calls of that step, as it already did for Anthropic. The hosted OpenAI flavor still differs here on purpose: it delivers an array of actions in one call. Halting would strand a mouse press whose release was planned for a later call, so the function tool offers `left_click_drag` instead of separate press and release actions, and rejects any action it does not declare. Provider types that reuse the chat completions client without being OpenAI-compatible (Azure, Google, OpenRouter, Vercel, Copilot) stay unsupported until each is verified. > [!NOTE] > `coderd/apidoc/docs.go`, `coderd/apidoc/swagger.json`, `docs/reference/api/schemas.md` and `site/src/api/typesGenerated.ts` are hand-edited to match the new enum value, because the environment that prepared this change had no pnpm to re-run the generators. The same environment had no node, so `pnpm run typecheck`, biome and prettier did not run locally; CI is the validator for the `site/` change and the generated files. > [!NOTE] > Prepared by Xum, Mike's coding agent, from the measurement that showed an OpenAI-compatible server dropping both hosted computer tools. --------- Co-authored-by: Ethan Dickson <ethanndickson@gmail.com> |
||
|
|
b6dc7158b4 |
feat: use the Anthropic computer use toolset for Claude 4.8 and later (#31521)
> Xum acted for Mike: this PR was written and opened by an AI agent on
Mike's behalf.
Computer use subagents on Claude 5.5 models (`claude-opus-5-5`,
`claude-sonnet-5-5`) failed on their first request: these models reject
`computer_20251124` and accept only the GA toolset
`computer_toolset_20260801`.
chatd now picks the Anthropic computer use tool version from the chat's
model with `ComputerUseToolVersionForModel` from coder/fantasy: the
toolset for Claude 4.8 and later, `computer_20251124` for Opus 4.5 to
4.7, Sonnet 4.6 and model IDs it cannot parse, and `computer_20250124`
for Haiku and Sonnet 4.5. Claude Platform on AWS and Amazon Bedrock
offer computer use only through the earlier versions, so an Anthropic
provider configured for either declares `computer_20251124` where the
model would take the toolset. For the toolset, chatd answers like
Anthropic's SDK tool runner: actions answer with text and only
`screenshot` returns an image, `cursor_position` returns the position as
text, `key` honors `repeat` (1 to 100), and zoom is declared disabled
because the agent desktop has no zoom action. The earlier versions still
answer every action with a screenshot.
A step's computer calls run one at a time in call order. For every
Anthropic version, the calls after one that failed, or that was rejected
before it ran (malformed input or a pre_tool_use hook denial), return
`Not executed: an earlier computer action in this turn failed.` without
running, which is the SDK runner's text. The chat loop enforces this
halt, so the computer tool keeps no state between calls. `wait` and
`hold_key` durations are read as seconds (capped at 300).
The frontend marks Claude Opus 5.5 and Sonnet 5.5 as verified for
computer use, and the computer tool row now labels each action ("Left
click", "Type", "Press key") instead of "Screenshot", shows text output,
and shows the error of failed or halted calls inline. The error block is
shared with the write_file and edit_files rows as
`ToolCall.ErrorDetail`.
Uses coder/fantasy#80 and coder/fantasy#82, both merged. `go.mod` pins
the fork to `7882f906a89a`.
<details>
<summary>UAT</summary>
TL;DR: it works. A remote Coder Agents UAT on dogfood passed on
`00143d5b12`, with all model traffic logged through a proxy:
- `claude-opus-5-5`, `claude-sonnet-5-5` and `claude-opus-4-8` declared
`computer_toolset_20260801` with zoom disabled and no beta header.
`claude-opus-4-6` and an unknown model ID declared `computer_20251124`
with the `computer-use-2025-11-24` beta, `claude-haiku-4-5` declared
`computer_20250124`, and a Claude Platform on AWS provider declared
`computer_20251124`. Opus 5.5, Sonnet 5.5, Opus 4.6 and Haiku 4.5 each
completed a desktop task.
- Toolset actions answered with text and only `screenshot` returned an
image. `repeat` 100 ran, 150 failed, and `wait` 400 stopped at 300
seconds.
- A failed call (`repeat` 150 on the toolset, a desktop error on 4.6)
and a pre_tool_use hook denial each halted the rest of the step without
touching the desktop. The next step ran normally.
- The computer tool rows showed action labels, text output and the halt
error. Opus 5.5 and Sonnet 5.5 showed no warning in the computer use
model selector. Switching a chat between 4.6 and 5.5 worked in both
directions.
Not tested: the tool type on Bedrock, and a halt after malformed model
input. Known, unchanged behavior: the desktop reports success for
unknown key names and off-screen coordinates, the model can still emit
`zoom`, which fails, and `scroll` up fails in `agent/x/agentdesktop`
because the negative amount is read as a flag. Both failures halt the
rest of their batch.
</details>
---------
Co-authored-by: Ethan Dickson <ethanndickson@gmail.com>
|
||
|
|
c0f7663e95 |
feat: support editing queued chat messages (#29215)
The owner of a chat can edit a queued message through
`PATCH /api/v2/chats/{chat}/queue/{queuedMessage}`. chatd never sends
a row under edit (`editing_since`): rows ahead of it still send in
order, and a turn that ends at it puts the chat in the new `paused`
status. Ending the head's edit, deleting it, or promoting a queued row
continues the chat. Interrupting sends still run ahead of the queue.
The chat state machine classifies a queue whose head is under edit as
its own sub-state (R1P, I1P, A1P, E1P, XE1P, P), so every path to
`paused` is in the transition matrix. A chat whose family has a paused
member cannot be archived, and auto-archive skips that family. Queue
edit and delete are owner-only, because both can send the next message
with the owner's credentials.
`ChatQueuedMessage` gains `editing_since` and `reasoning_effort`, and
`ChatStatus` gains `paused`.
Closes CODAGT-1028
Refs CODAGT-971
|
||
|
|
78fa9a8613 |
refactor(coderd/x/chatd)!: retire the close_agent tool alias (#31005)
This PR breaks compatibility with chat histories that contain `close_agent` tool calls, which chats from v2.34 and earlier have. #26673 (v2.35) renamed `close_agent` to `interrupt_agent` and kept a hidden alias, so a model that copied the old name from such a history still interrupted the child. This PR deletes the alias: `subagentToolNameAliases`, `ExecuteLocalToolsOptions.ToolNameAliases` with `(*toolBatch).resolve` and its five call sites in `chatloop`, the alias step in builtin input validation, and the `close_agent` entries in the unbilled tool set, the plan policy, and the chatprompt lifecycle list. Coder Agents is experimental (`coderd/x`), and our rule for experimental persisted data is to drop the legacy shape outright and delete the code that existed only to support it, instead of carrying it forward. What happens to an old chat: - Replay does not change, because the replay path does not look up tool names: the model still sees the historical `close_agent` call and its stored result, and the UI renders them as before. The write path reads a tool name only to parse `execute` commands and to keep structured error results from subagent lifecycle tools. Dropping `close_agent` from that lifecycle list changes nothing, because its calls are now rejected with a plain-text error. - If the model calls `close_agent` again, nothing runs and the child keeps running. The tool result is the error `Tool not active in this turn: close_agent`, and the model can retry with `interrupt_agent`. **Why it is safe** - Deleted alias-only tests: `TestExecuteSingleTool_ResolvesToolNameAlias`, the `AliasNamesClassifyAsCalled` billing row, the alias subtest of `TestPartitionAmbiguousToolCallsGatesOnBuiltins`, the alias loop in `TestUnbilledSubagentToolNamesMatchCatalog`, and the aliased calls that #30939 added to `TestExecuteToolsSerialToolCallOrder` and `TestExecuteLocalTools_ProviderToolRunner`; both tests are back to their #30938 form. - `TestExecuteSingleTool_UnknownAliasFallsThrough` is now `TestExecuteSingleTool_RejectsUnknownTool` and asserts the exact error above. The observer tests call the tool by its real name. - Mutants of the active tool check, step observer notification, serial dispatch, the unbilled set, the plan policy and builtin input validation each fail a remaining test. Only the deleted alias test caught an empty name passed to the tool handler; no chatd tool reads that name (MCP tools use their original name). - `git grep` finds no use of the removed identifiers and no backend code that resolves `close_agent`. The rejection test still uses that name as its input. The frontend keeps its `close_agent` display mapping, which old histories still use. **Review focus** - A rejected `close_agent` call is now billed like any rejected tool call; the rejection is immediate. - `ARCHITECTURE.md`, "Generation goroutine": this PR deletes the sub-bullet that called `close_agent` a deprecated alias of `interrupt_agent`. - The `resolved_tool_name` debug log field is gone. It differed from `tool_name` only for aliased calls, and the alias is gone. **Validation** - `go test -count=1 ./coderd/x/chatd/ ./coderd/x/chatd/chatloop/ ./coderd/x/chatd/chatprompt/`: ok - `go vet` and `golangci-lint run --fix=false ./coderd/x/chatd/...`: clean - `paralleltestctx -custom-funcs="testutil.Context,chatdTestContext" ./coderd/x/chatd/...`: clean - pre-commit hook: passed **Stack** This stack retires chatd compatibility paths for chat data that older Coder versions stored. This PR replaces #31003 and carries its change, adapted to the `toolBatch` code that #30938 and #30939 merged. Mike chose to keep the fallbacks that #31001 and #31002 removed, so those two layers and #31003 left the stack. 1. #30996 stamp chat message fixtures with the current content version 2. #30997 drop tests of the legacy content codec 3. #30998 retire the V0 chat message content decoder 4. #31000 inline the model config provider identity 5. #31005 retire the `close_agent` tool alias (this PR) > Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description. |
||
|
|
2255794d65 | feat: add response_size_bytes metric to chatd (#29816) | ||
|
|
2cb10b5cec |
feat: show web searches and the pages they found in chats (#29861)
Provider-executed web searches were invisible in chats. The UI skipped provider-executed parts, and the only trace was one source row per message that mixed OpenAI citations with Anthropic search results. The fantasy pin moves to coder/fantasy#62, which tags each found page with the search that found it and makes OpenAI request and report its found pages and search actions. chatd stores the tag and marks a search failed on an Anthropic error_code or an OpenAI failed status. Each search renders as one row with its queries or opened page, a running or unfinished state, and the pages it found. Cited pages stay in the answer's source row, which now follows the answer text. Closes CODAGT-795 |
||
|
|
1aff810443 |
chore: sync fantasy with upstream v0.45.5 (#31288)
Moves the fantasy pin to the tip of `coder_2_33` (`85ffbbe`). That commit is coder/fantasy#72, which merges 87 upstream charmbracelet/fantasy commits, v0.40.0 to v0.45.5. Between the two pins 117 files change. Upstream's own v0.40.0 to v0.45.5 change, without tests, testdata, examples, `go.sum` and the kronk provider, is 43 files, +2271/−928. ## What reaches chatd chatd builds only the `anthropic`, `openai` and `openaicompat` fantasy clients (`chatprovider.ModelFromConfig`); other vendors reach it through the AI Gateway on one of those. So the upstream changes to the Google, Vercel, OpenRouter, Bedrock, Azure and kronk providers, and to the `fantasy.Agent` loop and its retries, do not run in chatd. These do: - **Chat Completions and openaicompat streams** keep reasoning on batched and DeepSeek-shaped chunks, keep `finish_reason: length` when tool calls are truncated, and stop sending empty reasoning fields. openaicompat drops assistant turns that hold only reasoning. - **Token usage.** When an OpenAI-shaped backend reports reasoning tokens apart from completion tokens (reasoning larger than output), output tokens now include them, so that backend's reported output tokens and cost rise. OpenAI's own shape, where reasoning is a subset, is unchanged. - **Responses.** A content-filtered turn no longer reads as a tool-call turn. - **Context overflow.** Baseten, Fireworks and Parasail prompt-length errors set `ContextTooLargeErr`, which compaction acts on. - **Anthropic.** Tool calls whose content block never closes are reported. For Claude 5 models, thinking defaults to `display: "summarized"` when the chat config sets none; a configured display still wins. `TestThinkingDropBlockModel_RetriesWithEffort` now expects the field in the retry body. - **Anthropic keepalives.** Fantasy forwards `message_start`, `message_delta` and unknown events as content-free `keepalive` parts. The first one arrives before any model output, so this PR makes `ttft_seconds` and the `time_to_first_token` stage skip them, as they skip warnings. They still reset the stream silence guard. `processStepStream` ignores them, and the debug recorder counts them in its part count but records no content. - **Tool-result media** now goes through upstream's shared `ToolRunBuffer`, which replaces coder/fantasy#64 and does the same thing. #72 keeps coder's anthropic-sdk-go and openai-go forks, Go 1.26 and kronk v1.31.5, so upstream's SDK and Go moves do not come along. `go mod tidy` raises the modules fantasy v0.45.5 requires, among them otel exporters v1.47, the AWS SDK, `google.golang.org/api` v0.307 and go-openapi v1. All are upgrades. The `go.mod` comment now names #72 as the upstream sync. coder/coder#29861 stacks on this PR to pin coder/fantasy#62. <details> <summary>Verification</summary> - `go build ./...`. - `go test` passes on every package that imports fantasy (`coderd/x/chatd/...`), and `go test ./coderd -run 'TestChat|TestExpChat|TestMCP|TestAIBridge'` passes. - `TestGuardedStreamTTFTStage/WarningsAndKeepalivesThenContentObservesOnce` fails without the TTFT change (TTFT observed 0s instead of 1s) and passes with it. - `make pre-commit` passes. The generated Prometheus docs pick up the metric help change. Not verified against live backends in this PR: the stream fixes, the token usage change and the context-overflow patterns have no coder test against a real provider. coder/fantasy#72 reports a local chatd run against the AI Gateway on Anthropic, Responses, OpenAI Chat and openaicompat. </details> > 🤖 This PR was created with the help of Coder Agents, and _will be_ reviewed by a human. 🏂🏻 |
||
|
|
a33af01605 |
refactor(coderd/x/chatd/chatloop): create the compaction debug run through StartRun (#31087)
Compaction was the last debug-run caller that did by hand what `chatdebug.Service.StartRun` does: insert the run on a detached context with a 5 second limit, rebuild the run context, and finalize it with `ClassifyError`. This PR moves `startCompactionDebugRun` onto `StartRun` and deletes `compactionDebugCreateRunTimeout`. #30934 moved the other callers; compaction waited for #30766, which has merged. **Why it is safe:** - The first commit changes only the test, renamed from `TestStartCompactionDebugRun_DoesNotReportDebugErrors` to `TestStartCompactionDebugRun`. It now asserts the whole inserted row: kind `compaction`, status `in_progress`, the parent run's root chat, parent chat and trigger message, the summary model's provider, model, model config and history tip when the caller sets them and the parent run's otherwise, and no summary. It also asserts the returned run context and the `completed` and `interrupted` finalize statuses. It passes on the old code and on the new code. - On the old code, 16 mutants (wrong kind or status, a dropped parent field, an ignored override or fallback, a seeded summary, a wrong run context field, a wrong finalize status) all fail the new test. Before this PR, the chatloop tests caught 1 of them and `TestActiveServer_CompactionModelOverride` 3 more (kind, model and model config); the other 12 passed both. On the new code, all 13 mutants of the `StartRun` call fail the test. - `StartRun` keeps the rest: a failed insert returns the caller's context and a no-op finalizer, the insert ignores the caller's cancellation, and the run still finalizes through `FinalizeRun` with its default 5 second limit. `TestService_StartRun` pins the first two. A nil seed summary still stores no summary, because `nullJSON` maps a nil map to SQL NULL. **Review focus:** log output. The old code discarded create and finalize errors. `StartRun` logs them at Warn as `failed to create debug run` and `failed to finalize debug run`, like the other callers. A debug failure still never fails compaction. Validation: - `go test -count=1 ./coderd/x/chatd/chatloop/...` passes, and so does `go test -race` on the compaction debug tests. - `go test -count=1 -p 2 -parallel 4 ./coderd/x/chatd/chatdebug/...` passes. - `go test -count=1 -parallel 4 -run 'Debug|Compact' ./coderd/x/chatd/`: the same 53 tests pass before and after. - `go vet` is clean on both packages, and the pre-commit hook, which runs `paralleltestctx`, passed on both commits. > Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description. |
||
|
|
e75885a37e |
feat: add ACP subagent waiting and transcript streaming (#30560)
Add `acp_wait_agent` to stream ACP transcripts, including reasoning and nested tool activity, to users while returning the subagent's text response to the parent model. Reconnect dropped streams from their transcript cursor, preserve running sessions on wait timeout or cancellation, and persist separate user-visible and model-visible results. Depends on #31022. Generated with Codex, carefully reviewed by Hugo. |
||
|
|
3c36790c86 |
fix(coderd/x/chatd): coalesce deltas when persisting interrupted turns (#29661)
An interrupted turn was stored with one ChatMessagePart per streamed delta, while a completed turn gets one part per block. One 28,539 character interrupted response became 3,987 parts and 1 MB of JSON, and a /messages page with such turns reached 11.7 MB. Whitespace-only deltas were separate parts too, and prompt replay skips whitespace-only parts, so words and paragraphs ran together in the next turn's prompt. Adjacent deltas of the same type are now joined into one part. A text run ends at another part type, a tool result or a narration change. A reasoning run ends where CreatedAt, the block's start time, changes. Adjacent text blocks still merge because text deltas have no block identity. Rows already stored are not rewritten. |
||
|
|
a86012f82b |
refactor(coderd/x/chatd/chatloop): resolve tool name aliases in one toolBatch method (#30939)
Adds `(*toolBatch).resolve` for the `ToolNameAliases` lookup that `runOne`, `notifyStepToolCallObservers`, `notifyStepToolResultObservers`, and `isSerialToolCall` each repeated. The last three become `toolBatch` methods instead of taking the tool map and the alias map. Where they looked up a tool and then type-asserted it, they now assert on the map value: a missing name gives a nil interface, so the assertion fails as the missing-key check did. It also fixes a latent bug: `runOne` looked up result provider metadata by the name the model sent, while its other lookups use the resolved name, so a runner called by an alias lost its metadata. This cannot happen in production today: the only alias is `close_agent` to `interrupt_agent` (`subagent.go:45`), and only `computer_use.go:112` sets `ResultProviderMetadata`. ## Why it is safe Each mutant below puts the raw name back at one call site, and each fails a test: | Call site | Failing test | |---|---| | `runOne` tool lookup | `TestExecuteSingleTool_ResolvesToolNameAlias` and 5 more | | `runOne` metadata lookup | `TestExecuteLocalTools_ProviderToolRunner` | | `notifyStepToolCallObservers` | `TestExecuteToolsNotifiesStepToolCallObservers` | | `notifyStepToolResultObservers` (either loop) | `TestExecuteToolsNotifiesStepToolResultObservers` | | `isSerialToolCall` | `TestExecuteToolsSerialToolCallOrder` | No test called a serial tool or a provider runner by an alias, so this PR adds one aliased call to `TestExecuteToolsSerialToolCallOrder` and one to `TestExecuteLocalTools_ProviderToolRunner`. With the `chatloop.go` of #30938, the first test still passes, and the second fails because the aliased result has no metadata. ## Validation - `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes on #30938 and here. - `go test -count=1 -p 4 -parallel 4 ./coderd/x/chatd/...` passes here (20 packages). - `golangci-lint run ./coderd/x/chatd/chatloop/...` reports 0 issues, and the pre-commit hook passed. ## Stack This stack replaces the 20-parameter `executeTools` and the 15-parameter `executeSingleTool` in `chatloop` with methods on one `toolBatch` type, and resolves tool name aliases in one place. Production behavior does not change. 1. #30937 Cover provider tool runners in `ExecuteLocalTools` 2. #30938 Replace `executeTools` and `executeSingleTool` with `toolBatch` 3. #30939 Resolve tool name aliases in one `toolBatch` method > Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description. |
||
|
|
63c116fddd |
refactor(coderd/x/chatd): delete the CompactionOptions and compactionOutcome mirrors (#30984)
Deletes two structs that each repeat another struct field for field. `chatloop.CompactionOptions` repeated 21 fields of `GenerateCompactionOptions`, and `normalizedCompactionGenerateConfig` copied them one by one, so each new option had to be added in three places, as #30766 did for `Logger`. `compactionOutcome` in `chatd` repeated the 11 fields of `chatloop.CompactionResult` only so that `generateCompaction` could convert one into the other. The PR also deletes `TestGenerateCompactionSummaryUsesToolDefinitions`, which repeats `TestGenerateCompactionSummaryPreservesCallOptions`. ## Why it is safe - `publishCompactionError`, `startCompactionDebugRun`, and `generateCompactionSummary` now take `GenerateCompactionOptions`. Every read of the normalized config, in `GenerateCompaction` and in those helpers, uses one of the 21 fields that the copier copied without change. None reads the 7 fields that only `GenerateCompactionOptions` has. - `normalizedCompactionGenerateConfig` applies the same four defaults and the same threshold gate to its parameter, which is already a copy. So `GenerateCompaction` still reports the requested `ThresholdPercent` and `ContextLimit` from its own `opts`. If it read `config.ThresholdPercent` instead, `TestGenerateCompaction_RecordsRequestedTrigger` would fail. - Go allows the `compactionOutcome(outcome)` conversion only because the two structs have identical fields, so `buildCompactionMessagesInput` can hold the `chatloop.CompactionResult` itself. The four `TestBuildCompactionMessages_*` tests that build it keep their assertions. - Six mutants of the summary call each fail the deleted test on #30766: no tools, no call options, no summary prompt, one tool dropped, the caller's history changed, and tool descriptions stripped. The kept test failed only the first five, because its tool had no description. With that description moved into it, the kept test fails all six. - `git grep -w CompactionOptions` and `git grep compactionOutcome` find no matches. ## Review focus `generateCompactionSummary` keeps its `model` and `messages` parameters, so its 15 direct test calls change only the type name. It now also receives `Model` and `Messages` fields with the same values, and it does not read them. ## Validation - `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes. - `go test -count=1 -p 4 -parallel 4 ./coderd/x/chatd/...` passes (20 packages). - `golangci-lint run ./coderd/x/chatd/...` reports 0 issues, and the pre-commit hook passed. ## Stack #30766 has merged, so this PR now targets `main` by itself. > Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description. |
||
|
|
3df3c03288 |
refactor(coderd/x/chatd): replace executeTools and executeSingleTool with toolBatch (#30938)
Replaces the 20-parameter `executeTools` and the 15-parameter `executeSingleTool` in `chatloop` with methods on an unexported `toolBatch`. `newToolBatch` builds it once from `ExecuteLocalToolsOptions`, `run` replaces `executeTools`, and `runOne` replaces `executeSingleTool`. The 12 internal test calls set only the options they use instead of up to 20 positional arguments. It also deletes two copies: - `executeTools` filtered out provider-executed calls that `ExecuteLocalTools` had already removed. Only the `ExecuteLocalTools` filter remains. - The `runOne` metric defer repeated the body of `recordToolResultMetrics`. It now calls that function. ## Why it is safe - `newToolBatch` builds the tool, runner, and metadata maps with the lines `executeTools` used, and the byte budget as `ExecuteLocalTools` did. `run` drops the empty-batch returns: `ExecuteLocalTools` returns before it when no local call is left. - `recordToolResultMetrics` labels by `result.ToolName`, which `runOne` sets to `tc.ToolName` and never changes. It skips a nil `Metrics`. Production never passes one, because `ExecuteLocalTools` sets `NopMetrics()` first; only the direct test calls, which omit `Metrics`, use that path. - With the duplicate filter gone, removing the remaining `ProviderExecuted` filter fails `TestExecuteLocalTools_ProviderToolRunner` (#30937). The other four mutants from #30937 still fail it here. - `git grep 'executeTools\|executeSingleTool'` finds one comment, in `chatd_test.go`, which this PR updates. ## Review focus The metric defer stays a closure so that it reads `result` when `runOne` returns. ## Validation - `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes on #30937 and here, including the 16 tests that reach this code: `TestExecuteTools*` (5), `TestExecuteSingleTool_*` (5), `TestExecuteLocalTools*` (6). - `TestActiveServer_ToolErrorRecordsMetric`, `TestActiveServer_ToolExecutionAndPolicy`, and `TestProcessChatPanicRecovery` in `./coderd/x/chatd/` pass on #30937 and here. - `golangci-lint run ./coderd/x/chatd/chatloop/...` reports 0 issues, and the pre-commit hook passed. ## Stack This stack replaces the 20-parameter `executeTools` and the 15-parameter `executeSingleTool` in `chatloop` with methods on one `toolBatch` type, and resolves tool name aliases in one place. Production behavior does not change. 1. #30937 Cover provider tool runners in `ExecuteLocalTools` 2. #30938 Replace `executeTools` and `executeSingleTool` with `toolBatch` 3. #30939 Resolve tool name aliases in one `toolBatch` method > Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description. |
||
|
|
f004c72749 |
test(coderd/x/chatd/chatloop): cover provider tool runners in ExecuteLocalTools (#30937)
Adds `TestExecuteLocalTools_ProviderToolRunner`. No test ran a `ProviderTool` runner (the local half of a provider-defined tool, such as OpenAI computer use) through `ExecuteLocalTools`, so nothing checked runner dispatch, the active-tool exemption for runners, result metadata, or the skip of provider-executed calls. #30938 rewrites that code, so this PR pins the behavior first. The test sends a runner call and a provider-executed `web_search` call. The runner is in neither `Tools` nor `ActiveTools`. The test checks that only the runner call gets a result, and that the result has the runner's output and its `ResultProviderMetadata`. ## Why it can fail Each mutant is `chatloop.go` with one change, run with `go test -overlay`. Each one fails the new test, and all other `chatloop` tests pass with it. | Mutant | New test fails with | |---|---| | Runners are not added to the tool map | `Tool not found: computer` | | Runners are not exempt from the active-tool check | `Tool not active in this turn: computer` | | `ResultProviderMetadata` is not attached | nil metadata | | Provider tools without a runner are not skipped | nil pointer panic | | Both `ProviderExecuted` filters are removed | 2 results | Removing only one of the two `ProviderExecuted` filters, in `ExecuteLocalTools` or in `executeTools`, fails no test, because the other filter still runs. #30938 deletes the one in `executeTools`. `go test ./coderd/x/chatd/` also passes with runners left out of the tool map and the metadata attachment removed. Its subtest `TestActiveServer_ToolExecutionAndPolicy/provider_runner_executes_and_preserves_metadata` replays a stored computer-use call and result; it does not run the runner. ## Validation - `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes. - `golangci-lint run ./coderd/x/chatd/chatloop/...` reports 0 issues, and the pre-commit hook passed. ## Stack This stack replaces the 20-parameter `executeTools` and the 15-parameter `executeSingleTool` in `chatloop` with methods on one `toolBatch` type, and resolves tool name aliases in one place. Production behavior does not change. 1. #30937 Cover provider tool runners in `ExecuteLocalTools` 2. #30938 Replace `executeTools` and `executeSingleTool` with `toolBatch` 3. #30939 Resolve tool name aliases in one `toolBatch` method > Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description. |
||
|
|
8245457c8a |
fix(coderd/x/chatd): make compaction fit an overflowing context (#30766)
Compaction is the recovery path for a chat near its context window, but its summary request was larger than the turn that overflowed: the whole history plus the tool definitions. Its only fallback, a retry without tools, also missed context-window errors that arrive inside an HTTP 200 stream (OpenAI Responses API, Bedrock Mantle). In the incident behind this PR, one MCP tool result of about 1 MB moved a 1M-token chat from 67% to 92% of its window in one step. After that, every turn and every `/compact` failed with "Your input exceeds the context window of this model." This PR does not try to predict whether the summary request fits. Each time the model rejects it for its size, the next request is thinner: 1. The full request: tool definitions and the whole history. 2. The same history without tool definitions (as before). 3. Tool results longer than 32 KiB cut to 32 KiB, keeping their head and tail. 4. Tool results longer than 4 KiB cut to 4 KiB. A step that would cut nothing is skipped. Only locally executed text and error tool results are cut. User messages, attachments, provider-executed results, and media stay whole. The cuts apply only to the summary request, not to the stored history. Other changes: - **In-stream rejections match.** Fantasy stream errors without a status code and Responses `error` and `response.failed` events now count as size rejections. A Responses error code other than `context_length_exceeded` never matches, so a rate limit stays retryable. Anthropic's "input length and `max_tokens` exceed context limit" also matches. - **A conversation that still does not fit** fails as a non-retryable config error that tells the user how to recover. - **Each rejected summary request is logged at Warn** with its tool count, tool result cap, and the bytes it cut. Not in this PR: counting tool results that arrive after the measured step in the compaction trigger, estimating the summary request before sending it, and a tighter per-result cap for large-window models. Earlier revisions of this branch did these and were dropped for the simpler path above. ## Testing - `go test ./coderd/x/chatd/...` (20 packages), `golangci-lint` on `chatd` and `chatloop`, and the pre-commit hook pass. - The real OpenAI Responses client runs against `httptest` for each in-stream error shape and must retry without tools. - Each of these breaks a test: removing the cut steps, sending a cut that cuts nothing, turning error results into text, cutting provider-executed results, and dropping the in-stream match. > Xum opened this PR on behalf of @ibetitsmike. |
||
|
|
7517301ffa |
chore: add golangci new-code quality gates (#30622)
Stack context: second layer of the Go lint gate stack, built on the exclusion fix below. Why: adds commented-out code detection and a `log/slog` ban to the main config, plus a new-code-only pass requiring reasons on lint suppressions, flagging ignored errors (including `_, _ = w.Write(...)`), and banning `time.Sleep` in tests. Fixing the hits surfaced a real bug: GitLab warnings went to the wrong logger. The new-code pass resolves symlinked checkout paths so it does not silently report nothing. Validation: both configs report 0 issues; each seeded violation is caught; `make pre-commit` passed. > Xum acted for Mike to open this PR. |
||
|
|
41f5caafec | fix: fold web searches, answer reasoning, and OpenAI narration into working blocks (#30529) | ||
|
|
c28ae0dc9b |
feat: trigger compaction independently for the chat and compaction models (#28568)
> [!NOTE]
> Xum (an AI agent) authored this pull request on Mike's behalf.
## Summary
Automatic compaction now fires at whichever of two independent triggers
is reached first, and the UI shows when the organization override's
trigger applies.
- **Chat trigger**: the chat owner's personal threshold (or the chat
model's) against the chat model's window. A threshold of 100 turns off
only this trigger.
- **Organization trigger**: a fixed 80% of the organization compaction
override model's window. The override model's own compression threshold
is ignored. The other 20% is a target margin for the summary prompt and
the summary, not a guarantee, because the trigger reads the previous
step's prompt size.
On main, the chat threshold was applied to the smaller of the two
windows, and nothing in the UI showed it. Example: with a 20K chat model
at 70% and an 8K override, main compacts at 5,600 tokens; this PR
compacts at 6,400 (80% of 8K, shown as 32%).
## Changes
Backend (`coderd/x/chatd`):
- The trigger with the lower token point binds (ties go to the chat
trigger). It drives the automatic compaction gate and the
still-over-limit check, and every summary, automatic or manual, records
it.
- An override that is unset, deleted, disabled, on a disabled provider,
or without credentials for the chat owner is treated as absent: the chat
model summarizes and only the chat trigger applies.
- The summary's output cap is bounded by the summarizer's own window,
not the trigger's.
- The still-over-limit error names what can clear it: the compaction
threshold (chat trigger), an override model with a larger window
(organization trigger), or both when usage is over both points.
- The still-over-limit check compacts once more when the latest summary
was recorded under a different threshold or trigger window. The
`chat_summarized` result gains `trigger_context_limit_tokens`, and
`context_limit_tokens` now records the chat model's window (main
recorded the smaller window). On main, lowering a threshold after a
compaction ended every later turn with the error.
- The still-over-limit check skips a 0% threshold ("always compact"),
which on main ended the run with the error after every compaction.
- `ARCHITECTURE.md` documents the two triggers, the summary cap, and the
still-over-limit rules.
Frontend (`site`), through a shared `compactionTriggers` helper that
mirrors the backend rules (no new API):
- Coder Agents settings: the compaction override row warns when the
organization trigger undercuts chat models' default triggers ("the
organization override compacts chats at 80% of {model}'s {N}-token
window").
- Personal thresholds page: an Effective column shows the trigger that
applies, with a popover when the organization override binds. "Compacts
at" follows the same trigger, and a point the chat model's window cannot
reach shows as Off. The header's 100% sentence, and the row warning when
a member types 100, say an organization override may still compact chats
when one is set or its organization's overrides failed to load (the
header also while they load).
- Chat context gauge: the compact-at percent follows the binding trigger
and is annotated when the organization override binds. If the overrides
request fails, it shows the chat-only value with "(organization override
not loaded)", or "Compaction off (organization override not loaded)"
when the chat trigger is off.
- An override whose provider the user cannot use is ignored on the gauge
and the personal page, matching chatd.
- The shared `formatContextLimit` prints counts below 1,000 exactly (an
800-token compaction point showed "~1K") and 999,500 to 999,999 as
"1.0M" instead of "1000K". It also labels the model pickers; their other
values are unchanged.
- `HelpPopoverIconTrigger` lets callers set their own `aria-label`; only
the Effective column's popover does.
Known v1 limits: the Coder Agents settings alert is organization-scoped,
so it can warn about an override that some members fall back from for
lack of provider credentials. The gauge checks the viewer's provider
access, so on a shared chat its label can differ from the owner's
runtime trigger (Codex P2, declined). Compaction itself is unaffected.
## Screenshots
From remote UAT round 13 on
|
||
|
|
bb9f31d94b |
fix: apply Anthropic prompt caching through OpenRouter and OpenAI-compatible gateways (#29563)
Anthropic models configured through an `openrouter` provider (and, by
the same code path, `vercel` and generic `openai-compat` providers)
never got prompt caching. Two independent gaps: chatd only emitted
`cache_control` breakpoints when the fantasy client was `anthropic`, and
even when a client sent them, the AI Gateway chat-completions
interceptor rebuilt the upstream body from typed
`openai.ChatCompletionNewParams` and silently dropped the unknown key.
Dogfood UAT on main @
|
||
|
|
fc0af7fbd8 |
feat: open a chat_turn span per prompt (#29771)
**TL;DR:** Wraps each prompt in a `chat_turn` root span with every lifecycle stage parented under it. The span records one outcome per turn (`completed`, `interrupted`, `error`, `abandoned`), and the new `coderd_chatd_turn_outcomes_total` counter counts those outcomes. Turns stay correct when tasks are canceled, replaced, or retried. Each prompt runs inside a `chat_turn` root span owned by the runner (`runnerTurnSpan`). The generation step, prepare, MCP connect, stream, TTFT, provider attempt, commit, thinking, tool call, compaction, and retry backoff stages are parented under it, activating the stages from #29770. `chat_turn` spans are standalone roots: the turn runs on a worker, often on another replica, and no trace context is persisted. **Lifecycle** - **Start.** Each generation loop iteration calls `Ensure`, which opens the turn if none is open and returns a `turnToken`. The span is backdated to the trigger: the latest of the last user prompt, a dynamic tool's result message after it, and a pending compaction request. An `acquisition` stage covers trigger to pickup. - **Stale trigger / takeover.** A trigger at or before the previous anchor opens the turn at now with no `acquisition` and counts `stale_anchor`. A runner that took the chat over from an owner with a stale heartbeat (`TakenOver`, from acquisition) opens its first turn at now with no `acquisition`, so the dead owner's run is not reported as pickup delay. - **Newer prompt.** `Ensure` on the current turn with a newer trigger closes it and opens a new one. The closed turn is `abandoned` unless the task that ran it records an outcome before releasing it. - **Finish.** After `FinishTurn` or `EnterRequiresAction` commits, the step calls `Complete`; `Settle` closes the span after the step's own stage ends, so the finishing step is inside the turn. A chat waiting in `requires_action` is outside any turn; the generation resumed by its tool results opens a new one. - **Emission.** Closing a turn fixes its end time; its span is emitted once every task that joined it (`Ensure` takes the task ID) has called `Release`, which `runTask` does after the task returns. A finishing task whose `FinishTurn` promoted a queued prompt therefore still counts its turn as `completed` even when the promoted task's `Ensure` runs before its `Complete`. `End` at runner exit emits whatever is left. - **Failure.** A step that commits `FinishError`, or fails closed, invalidates the turn as `error` after the commit. Errors the task runner retries, including task timeouts, keep the same turn open; an expected, non-retryable exit records no outcome and leaves the turn open until the interrupt task, a newer prompt, or runner exit closes it. A canceled generation task records no outcome and cannot obtain a turn token, so a replaced task never touches its successor's turn. - **Interrupt.** The interrupt task marks the open turn `interrupted` before `FinishInterruption` commits and closes it after. Only a stop (by the user or a parent agent) is `interrupted`; it ends with unset span status, since error status is reserved for `error` turns. runner shutdown, loss of ownership, and message edits close the turn as `abandoned` with unset status. - **Promotion.** Every promoting transition (`FinishTurn`, `FinishInterruption`, a send to an errored chat, `PromoteQueued`) returns the promoted message's queued time in its result, and one `recordQueueWait` helper records a standalone `queue_wait` to the promotion. The promoted turn opens at the promotion time. `chat_turn` is observed on `coderd_chatd_stage_duration_seconds` only for completed turns, so the histogram is completed-turn latency; every turn still emits its span. `coderd_chatd_turn_outcomes_total{outcome, chat_kind}` counts every closed turn exactly once (`completed`, `interrupted`, `error`, `abandoned`), matching the span's `turn_outcome`, under the `chat-stage-metrics` experiment. Turn time accounting is in draft #29772. **Stages and identity** - `tool_call` is a live stage around each local tool call in `chatloop`; the tool runs on its context, and it ends errored only when the tool fails to execute. Calls that never run (inactive, unknown, or rejected by policy) record no stage. - `mcp_connect` carries connected and failed server counts and ends errored only when servers were configured and none connected. - The compaction stage carries the model that runs the summary. - `withStageIdentity` puts scope and chat kind on the turn context. Detached background work runs in `background` scope as its own trace roots. - The `advisor` tool's nested model call appears only as its enclosing `tool_call`. ARCHITECTURE.md gains a "Lifecycle tracing" section. <details> <summary>Review decisions</summary> - Turn identity stays timestamp-based with targeted fixes (clamped acquisition, rotation on a newer trigger, invalidate after commit, fail-closed invalidation, settle closing invalidated turns) plus a takeover flag; keying turns on message IDs would not detect takeover and cannot express interrupted/abandoned outcomes. - Outcome classification: the interrupt task owns `interrupted`, and cancellation or a fence mismatch alone never records an outcome, so shutdown, ownership loss, and edits are `abandoned`. Only `error` turns set error span status. Remaining ordering races between a step and an interrupt are documented as known limitations in ARCHITECTURE.md. - Turns are emitted after the tasks that ran them return, not when they close, so an outcome a replaced task records while unwinding reaches its own turn; the successor task is not delayed. Chosen over an after-commit, pre-publish hook on `ChatMachine.Update`, which would fix only the `FinishTurn` ordering. - `requires_action` ends the turn rather than pausing it, so `chat_turn` measures chatd time; a prompt with client tools counts as several turns. - Promoted turns anchor at promotion time and `queue_wait` is always standalone, so `chat_turn` never includes time spent queued behind another turn. - The former `capacity_wait` stage (#29805) was dropped: it was a per-worker lower bound, excluded from accounting, and needed its own SQL and bookkeeping. #29805's remaining helpers are folded into this PR. </details> Supersedes #28972, #28973, and #29805. 🤖 This PR was generated by Coder Agents on behalf of @jscottmiller. |
||
|
|
e988ddb11d |
feat: trace chatd provider stages and count only output parts as first token (#29770)
**TL;DR:** Traces the model provider call: a `stream` stage, a nested `time_to_first_token` stage, and one `provider_attempt` per HTTP round trip, labelled with provider type and model. Behavior change: `coderd_chatd_ttft_seconds` now counts only a real output part as the first token, so error, warning, and finish parts no longer count. Instruments the provider call. **Stream and time to first token.** `GenerateAssistant` records a `stream` stage from stream open to close and a nested `time_to_first_token` stage. The TTFT window closes once, on the first streamed output part: any part other than warnings, finish, or error, including start markers such as `text_start`. A closed window observes the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same elapsed time. A failed open, an error part, or a release without an output part ends the TTFT span with the error (a silence timeout is classified as such) and observes nothing. The `stream` span ends once, with the error `GenerateAssistant` returns, so content-filter and no-output attempts end errored. Failed `stream` windows are observed like successful ones. The attempt is released before the stream span ends, so the TTFT window always falls inside its stream. The compaction summary stream passes a nil tracer, so it records neither. Without #29771, production emits only `provider_attempt`; #29771 passes the tracer to `GenerateAssistant`. **Provider attempts.** The provider HTTP transport is wrapped in `stageSpanRoundTripper`, which emits one `provider_attempt` stage per round trip, so retries each get a span. The span closes on response headers; a status >= 400 marks it errored without changing what the client sees. `resolveModelCall` resolves the `StageModel` (wire provider, configured `provider_type`, model, effective reasoning effort) and threads it to the transport, so span attributes and labels match the call. `provider_type` is read from `ai_providers.type`, so types outside the fantasy set (copilot) keep their configured name. `chatd.New` builds the tracer with the server clock. **Behavior change to `coderd_chatd_ttft_seconds`:** only an output part counts as a first token. Error parts no longer close the window as a success, and warnings and finish parts are skipped; the help text says so. `StageSpan.End` returns the span's elapsed window, which `ttft_seconds` observes. Callers pass `Stages` and `StageModel` through `GenerateAssistantOptions`; a nil tracer discards (TTFT still records `ttft_seconds`). `Server` gains the `stages` tracer and a `TracerProvider` option, wired from `coderd`. <details> <summary>Review decisions</summary> - Start markers close the TTFT window: they are the first sign the model is producing output, and waiting for a delta would fold hidden reasoning time into TTFT. - Failed `stream` and `provider_attempt` windows are observed: that time was spent. TTFT is the exception because a first token that never arrived has no duration. - `http_host` was dropped from `provider_attempt`; every request goes through the local AI Gateway transport, so it was constant. - The advisor tool's nested model call is intentionally not instrumented as `stream`/TTFT. </details> Supersedes #28970 and #28971. 🤖 This PR was generated by Coder Agents on behalf of @jscottmiller. |
||
|
|
7a9442014e |
feat: add chatd lifecycle stage tracer (#29769)
**TL;DR:** Adds a stage tracer that turns each chat lifecycle stage into
both an OpenTelemetry span and a Prometheus histogram observation, so
traces and metrics always agree. The metrics are only registered with
the `chat-stage-metrics` experiment. Nothing calls the tracer until the
next PRs.
Adds `chatloop.StageTracer`, which emits each chat lifecycle stage as an
OpenTelemetry span and a `coderd_chatd_stage_duration_seconds{stage,
scope, chat_kind}` observation from one `End` call, so trace and metric
durations cannot disagree. The provider-bound stages (`stream`,
`provider_attempt`) are also observed on
`coderd_chatd_model_stage_duration_seconds{stage, provider_type,
chat_kind, model}`, the only family carrying `provider_type` and
`model`, and only for turn-scoped stages. Time to first token per model
stays on the existing `coderd_chatd_ttft_seconds`. Nothing calls the
tracer yet; the next PRs instrument the provider call and the runner.
- The families are registered only when the `chat-stage-metrics`
experiment is enabled (safe and known, so
`--experiments=chat-stage-metrics` or `--experiments=*` turns it on).
Spans are emitted regardless. `NewMetrics` leaves the families off;
`chatd.New` opts in from the experiment, and a test asserts the
experiment decides whether they reach the server's registry.
- `Stage`, `Scope`, `ChatKind`, and `StageAnomaly` are typed string
constants. `scope` separates latency attributable to a prompt (`turn`)
from detached `background` work; `chat_kind` is `root` or `subagent`,
empty without a known chat.
- `StageModel` carries the wire `provider`, `provider_type`, `model`,
and `reasoning_effort`. All are span attributes; only `provider_type`
and `model` are metric labels, on the model family. No family carries an
organization label.
- Observed stages: `chat_turn`, `queue_wait`, `acquisition`,
`mcp_connect`, `stream`, `time_to_first_token`, `provider_attempt`,
`tool_call`, `commit`, `retry_backoff`. `generation_step`, `prepare`,
`thinking`, and `compaction` are span-only. A test keeps the stage sets
and help text consistent.
- API: `Start` / `StartRootAt` for live spans (span timestamps come from
the tracer clock), `Record` for stages reconstructed from timestamps
(the scope comes from the context). `End` is idempotent;
`EndWithoutObservation` ends the span without a sample. A nil tracer,
provider, or metrics discards.
- `coderd_chatd_stage_anomalies_total{reason}` counts observations
dropped, adjusted, or inconsistent (`negative_elapsed`,
`inverted_window`, `missing_timestamp`, `future_start`, `stale_anchor`).
Span-only stages never feed it.
- Buckets are 12 round edges (`0.1 0.25 0.5 1 2.5 5 10 30 60 300 1800
3600`).
- Moves the body of the `for` loop in `StartGeneration` into
`runGenerationStep` so the step has a function boundary for a deferred
stage end. Control flow and error wrapping are unchanged.
<details>
<summary>Series cost (approximate, whole stack)</summary>
A histogram label set is 12 edges + `+Inf` + `_sum` + `_count`: 15
series. Per replica, each `chat_kind` costs about 150 stage series and 4
turn outcome counters; each `(chat_kind, model)` costs 30 (2 stages x
15). Both chat kinds on `M` models: about `308 + 60*M` per replica, plus
background-scope stage series.
Dogfood `main`, 3 replicas: ~3.4k `coderd_*` series per replica (~10.1k
total).
| Models used per replica | Added series (3 replicas) | vs. baseline |
|---:|---:|---:|
| 1 | ~1.1k | +11% |
| 5 | ~1.8k | +18% |
| 10 | ~2.7k | +27% |
| 35 | ~7.2k | +72% |
Most deployments run four or fewer models, so a typical replica adds
roughly 370 to 550 series. The experimental turn accounting in #29772
would add 88 more per replica (11 categories x 4 outcomes x 2 chat
kinds). Series appear lazily and do not grow with organizations.
</details>
<details>
<summary>Review decisions</summary>
- Experiment gate only; no deployment flag or metric levels. If the
experiment graduates, the families become always on like the other chatd
families; a flag is added then only if dogfood shows a cardinality
problem.
- Organization is on neither metrics nor spans, and `effort` is
span-only; `provider_type` (configured type), not the wire `provider`,
is the model-family label.
- `time_to_first_token` is not on the model family; per-model TTFT is
`ttft_seconds`, which observes the same measured window.
- The model family records only turn-scoped stages.
- Nil guards are kept so a nil or zero tracer is always safe to call.
- HTTP attribute keys stay snake_case like the rest of the chatd span
attributes.
</details>
Supersedes #28968 and #28969.
🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
|
||
|
|
fa5bac6e77 |
feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt recorded the same generic result whether a call had finished, was still running, or never started. chatd now derives a tool call ID from the chat, the committed assistant message, and the provider tool call ID, and sends it with execute starts, edit_files, and write_file. Every replica and retry computes the same ID, so the agent returns the first result instead of running the tool again, and a retried foreground execute waits only for the rest of its timeout. On interrupt, chatd cancels the unresolved calls of those tools on the agent before committing, within 30 seconds, and records what happened: "tool call was canceled before it ran; no changes were made", the output of a command canceled while running, or the real result. The UI shows a canceled command with the existing killed marker. If the cancel fails, including an old agent's 404, the result stays the generic interrupted text. Calls whose provider tool call ID is empty or repeats in the message get no ID and behave as before. Fixes CODAGT-757 |
||
|
|
5a997c471a |
fix(coderd/x/chatd): tell the model when output hit the token limit mid tool call (#30120)
When an Anthropic response hits the output token limit while the model
is writing a tool call's input, chatd gets `FinishReasonLength` with a
cut-off call. It handled both shapes of that call badly:
- If the provider closed the `tool_use` block, the call carried
truncated JSON and got the generic malformed-input result, which invites
the model to resend the same oversized input and hit the limit again.
- If the block never closed, fantasy emitted no tool call, chatloop
discarded the dangling tool input, and the turn ended with only
reasoning and no explanation.
On a length finish, `GenerateAssistant` now gives each cut-off local
call an error result: "This tool call was not executed because the
response reached the output token limit (N tokens) while writing its
input. Split the content into smaller tool calls." The token count is
included when the call has a configured output limit. A tool input that
started streaming but never became a call first gets a stand-in call
with empty input. The call persists with nil args and replays as `{}`,
so the provider receives a valid `tool_use` and `tool_result` pair, and
generation continues with another step.
With lifecycle hooks enabled, `pre_tool_use` admission skips calls the
step already resolved, so the hook is not dispatched for them and no
second result is added. The duplicate tool ID check still runs on the
full batch, because a resolved call still occupies its ID.
Independent of #30119, which stops the AI Gateway from failing these
streams before chatd sees them.
## Testing
-
`TestGenerateAssistant_SilentNoOutputFinish/LengthFinishResolvesTruncatedToolCalls`
(truncated tool call, dangling tool input): no error, reasoning kept,
and each call paired with the output-limit result.
- `TestPreToolUseHookOutputLimitTruncatedToolCallResolvedOnce` with
hooks enabled, for truncated and unfinished input: no hook dispatch, one
stored error result per call, a second model request, and a replayed
request that pairs `tool_use` input `{}` with its `tool_result`.
`DuplicateIDFailsClosed` checks that a reused ID still fails the chat
with `duplicate tool use ID`.
- Red-green: with `chatloop.go` and `generation.go` restored to the base
code, all four truncation cases fail. `DuplicateIDFailsClosed` passes on
the base code by design and fails on `d02bf4aa89b`, which dropped
resolved calls before the duplicate check.
- `go test ./coderd/x/chatd/... -count=1` passes, and lint on the
touched packages reports no issues. Local pre-push checks were skipped
per standing preference; PR CI covers them.
- Remote UAT: not run yet.
## Review record
- Parent review (Xum) of `d02bf4aa89b`: skipping resolved calls inside
`chathooks.PendingToolCalls` hid them from the duplicate tool ID check
in `admitStepToolCalls`. Fixed in `745f722165b`.
- Codex: not requested yet.
> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.
|
||
|
|
fd5b37576a |
feat(coderd): record provider response IDs on chat messages (#29763)
Chat messages could only be linked to AI Gateway records per chat, so nothing recorded which model response produced a given message. fantasy surfaces the response ID the model endpoint returns on the stream's finish part: the Anthropic Messages ID as the part ID, and the OpenAI Responses ID in provider metadata. chatd now stores that ID in the existing `chat_messages.provider_response_id` column for assistant steps and the compaction summary row. No migration and no aibridge changes. What the stored value is depends on the route: | Route | Stored value | Joins to | |---|---|---| | Anthropic Messages API through the AI Gateway (Anthropic and Bedrock Claude) | The gateway's interception ID. The gateway replaces the message ID because injected MCP tool loops can fold several upstream messages into one response. | `aibridge_interceptions.id` | | OpenAI Responses API through the AI Gateway | The upstream `resp_` ID, passed through unchanged | `provider_response_id` on `aibridge_token_usages`, `aibridge_user_prompts`, and `aibridge_tool_usages` | | Direct to the provider | The provider's own ID (`msg_`, `resp_`) | Nothing in the gateway tables | To resolve messages to interceptions, then read usage by `interception_id`: ```sql SELECT cm.id AS chat_message_id, i.id AS interception_id FROM chat_messages cm JOIN aibridge_interceptions i ON i.id::text = cm.provider_response_id UNION SELECT cm.id, tu.interception_id FROM chat_messages cm JOIN aibridge_token_usages tu ON tu.provider_response_id = cm.provider_response_id; ``` Not covered, and left NULL: - Partial rows saved on interrupt, because fantasy only reports the ID in the finish part. - Chat Completions providers (OpenAI-compatible, OpenRouter, Vercel), because fantasy does not surface the chunk ID there. - User, tool, and system rows, and all existing messages. <details> <summary>Delivery record</summary> - `96a896c` replaced the earlier interception-link design (`59fa58f`, `7f93c18`, `9d04ccd`) in place: the aibridge hook, the `aibridge_interception_id` column and migration, the message part buffer tracking, and the `ARCHITECTURE.md` section are all removed. The net diff against the base touches only `coderd/database` and `coderd/x/chatd`. - Local validation on `96a896c`: pre-commit hook green (gen, fmt, lint, build); `go test ./coderd/x/chatd/... ./coderd/database/` green. Red-green: removing the finish-part capture fails both new end-to-end tests (`TestActiveServer_PersistsAnthropicMessageID`, `TestOpenAIResponsesPersistsProviderResponseID`). CI on `96a896c`: green. - Remote UAT round 2 on `96a896c` ([chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)): **FAIL** against this description's earlier claim that every stored ID joins the gateway usage rows directly. Anthropic rows routed through the gateway stored the interception ID, so 0 of 18 joined usage rows directly, while every checked Anthropic row equaled `aibridge_interceptions.id`. Both OpenAI Responses rows joined directly. Interrupted partial rows, Chat Completions rows, and user and tool rows were NULL, with no empty strings or duplicate IDs. Multi-step turns, manual compaction, edit and regenerate, a provider switch, and a retried 429 behaved as expected. One unattributed anomaly: a single duplicate user row after interrupt and continue, which did not reproduce and is outside this diff. - `2e889ba` corrects the doc comment on `PersistedStep.ProviderResponseID` and this description to match; behavior is unchanged. Pre-commit hook and CI green. - Remote UAT round 3 on `2e889ba` against the corrected contract ([same chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)): **PASS**. The query above resolved all 40 stamped rows to exactly one interception from the same chat, with none unresolved. Reading usage by interception recovered every usage row for 38 of them; the last two resolved but their usage rows were not counted. Fresh Anthropic and Responses calls, a seven-step tool turn, manual compaction, an Anthropic to Responses to Anthropic switch in one chat, and a UI reload behaved as described, and the NULL cases held. Three interrupt and continue attempts did not reproduce the round-2 duplicate, so it stays unattributed. Not tested: gateway-injected MCP tool loops (unreachable from chatd, whose delegated requests skip the gateway's MCP proxy), direct Anthropic `msg_` IDs, Bedrock, and threshold-triggered compaction. Both rounds had a verified isolation proof and a clean dogfood audit. - Codex rounds 1 to 3 reviewed the old design. Codex round 4 on `2e889ba` ([request](https://github.com/coder/coder/pull/29763#issuecomment-5796193371), 13:58Z): no major issues ([result](https://github.com/coder/coder/pull/29763#issuecomment-5796267231), reviewed commit `2e889ba05c`). No unresolved review threads. Review passes so far: 4 of 6. - Final head `2e889ba` is the exact commit remote UAT round 3 and Codex round 4 covered; required checks green. </details> > Xum updated this PR on behalf of @ibetitsmike. |
||
|
|
e35cbe2a7f |
fix(coderd/x/chatd): stream compaction summaries (#28886)
Chats could get permanently stuck once their context crossed the compaction threshold on adaptive-thinking Claude models: every compaction attempt failed with `compaction produced an empty summary`, and both automatic and manual compaction take the same path, so the chat could never recover. ## Problem `compactionSummaryCall` deliberately set `MaxOutputTokens = nil` because the summary request was non-streaming and the Anthropic SDK rejects non-streaming requests whose `max_tokens` implies a completion longer than ten minutes. But a nil cap makes fantasy's Anthropic provider silently default to 4096 output tokens, and adaptive-thinking Claude models can spend that entire budget on a thinking block, leaving zero summary text. ## Fix - Stream the compaction summary request. Streaming lifts the Anthropic SDK's non-streaming duration restriction, which was the only reason the cap was nil'd. - Keep the resolved output cap (32k default) on the summary call instead of nil'ing it. - When the summary comes back empty with truncation evidence (length finish reason, or reasoning with no text), fail with `compaction summary was truncated at the output token cap` instead of the generic empty-summary error, so the next occurrence is attributable at a glance. Unit tests cover text-block joining and the truncation classification; the compaction e2e fixtures now serve streamed summaries and assert the summary request streams with `max_tokens: 32000`. Dogfood UAT on a deployment built from this branch verified manual compaction with Claude Opus 5 (adaptive thinking, twice, with reload persistence) and an OpenAI regression chat. > 🤖 Opened by [Xum](https://mux.coder.com) on behalf of @ibetitsmike. |
||
|
|
19e2b0415a |
fix(coderd/x/chatd): preserve tool definitions during compaction (#29436)
Supersedes #28883 by @pfrybar. His commit is carried with authorship intact (rebased onto main as |
||
|
|
77479408b4 |
feat: make the Coder Agents stream silence timeout configurable (#29394)
The `chatloop` stream silence guard cancels and retries a model attempt after `10m` of silence, and it was hard-coded. It's armed before the request opens, so it also bounds time to first token, and slow local models (e.g. `llama-server` on a large prompt) can spend longer than that on prompt processing alone, meaning every attempt just gets cancelled and retried. This PR adds `--chat-stream-silence-timeout` / `CODER_CHAT_STREAM_SILENCE_TIMEOUT` (default `10m`) and passes it to both the main loop and the advisor. Setting it to `0` disables the guard entirely, and anything above `24h` is rejected, which also keeps the watchdog arithmetic below from overflowing. Whilst doing this I ran into the runner's `15m` task watchdog, which kept counting during an open stream. That silently cancelled any healthy stream longer than `15m`, and would've stopped a silence timeout above `15m` from ever firing. To fix, the silence guard now kicks the watchdog out to `max(15m, silence + 5m)` whenever it arms or sees a part. Silent streams still fail through the guard, healthy streams can run for as long as they like, and hang detection is unchanged everywhere else. With the guard disabled there's nothing to kick against, so arming it just stops the watchdog for the rest of the attempt. That does mean the DB write after the stream isn't bounded either when `0` is set; I think that's fine for an opt-in setting, and a user can still stop the chat from the UI if it wedges there. I did consider removing the guard entirely, but the provider HTTP clients have no timeout, so a provider that holds the connection open would only ever be caught as a generic task hang, which is what #25782 was fixing. |
||
|
|
26e4429362 |
test: fold near-identical test bodies into tables across eight packages (#29463)
Folds groups of near-identical test bodies (same parent, same parallel flag, differing only in inputs and expected values) into table-driven tests in eight packages: `coderd/templatebuilder`, `scripts/develop`, `agent/agentfiles`, `coderd/x/chatd/chatloop`, `coderd/x/chatd/chatdebug`, `enterprise/aibridgeproxyd`, `cli`, and `coderd/database/db2sdk`. One commit per package. Every case keeps its previous subtest name, `t.Parallel()` placement is unchanged, and the set of `=== RUN` names is identical except where standalone `TestX_Y` funcs became `TestX/Y` subtests (chatloop: 9 funcs into `TestExecuteLocalTools_BatchRuntime`; chatdebug: 15 funcs into two tables). Members whose bodies differ in more than data stayed as they were (listed per package in the commit messages). In `agent/agentfiles` the four `TestFuzzyReplace_*` funcs were already tables, so the shared request plumbing was hoisted into one helper instead of merging the funcs and losing their per-rule docs. Net -1035 lines across 9 test files; runtime is unchanged (all sub-100ms unit tests). | Package | Diff | Funcs removed | Subtests | RUN lines | |---|---|---|---|---| | coderd/templatebuilder | +237 / -291 | 0 | 31 -> 31 | 33 -> 33 | | scripts/develop | +194 / -246 | 0 | 27 -> 27 | 28 -> 28 | | agent/agentfiles | +117 / -260 | 0 | unchanged | 58 -> 58 | | coderd/x/chatd/chatloop | +224 / -306 | 9 | 3 -> 12 | 14 -> 15 | | coderd/x/chatd/chatdebug | +251 / -537 | 13 | 0 -> 15 | 42 -> 44 | | enterprise/aibridgeproxyd | +287 / -419 | 0 | 31 -> 31 | 32 -> 32 | | cli | +143 / -326 | 0 | 15 -> 15 | 16 -> 16 | | coderd/database/db2sdk | +130 / -233 | 0 | 9 -> 9 | 10 -> 10 | ## Validation - Per package: `go test -count=1 -tags=testsmallbatch -parallel=8 ./<pkg>/ -run <regex> -v` before and after, with the `=== RUN` name sets diffed - `go test -race -count=2 -shuffle=on` on each converted test: pass - `golangci-lint run ./<pkg>/...` with a fresh cache: 0 issues in all eight packages - Repository pre-commit hook (including lint/go) passed on every commit Part of the test-suite speedup series (see #29457); independent of the other PRs. ## Remote validation Remote run on a dogfood workspace (separate compute), chat https://dogfood.cdr.dev/agents/5bbe73d0-866a-4841-9cc3-c5e5b4d97e44, tested head `d7c55d6b0e8` against base `9d973372774`: all eight packages pass on the head and under `-race -count=3 -shuffle=on`; package times are sub-second on both sides (unchanged). `=== RUN` name sets: chatloop 15 to 16 lines with all nine `TestExecuteLocalTools_<X>` leaves surviving as `TestExecuteLocalTools_BatchRuntime/<X>`; chatdebug 42 to 44; `aibridgeproxyd` `TestNew` 32 to 32, byte-identical sorted list. One correction to the text above: seven chatdebug leaves are renamed, not preserved verbatim (`TestRecordingTransport_*UnknownLength*` became `TestRecordingTransport_UnknownLengthClose/<name without "UnknownLength">`), so `-run` filters keyed on those old names need updating. Endorsed verdict: PASS. > Xum (an AI agent) authored this PR on behalf of @ibetitsmike. |
||
|
|
8397680687 |
fix: keep text and image together in MCP tool results (#29332)
## Problem MCP tool results that mix text and an image (Playwright's default `browser_take_screenshot` returns `[text, image]`) were reduced to text only in both MCP conversion paths, so screenshots never reached the model. The generic tool renderer also showed media results as a JSON dump of base64. ## Fix - `mcpclient` and `chattool`: keep all text plus the first usable binary item. Error flags survive empty and MIME-less results. Only one binary payload is represented; additional images are still dropped. - `chatloop`: at execution time, cap tool media at the 10 MiB attachment limit and correct or reject declared image types by byte signature (sniffing, not decoding). - `chatd` prompt build and `chatprovider`: at generation and compaction, replace media the destination provider cannot accept with the text plus an omission note (Anthropic/Bedrock: JPEG, PNG, GIF, WebP under the inline cap; OpenAI/Azure and Google: their image allowlists, applying the configured vendor's rules behind `openai-compat`). Persisted history is unchanged, so the UI keeps the payload and provider switches keep working. - `site`: results flagged `is_media` render through `ToolResultMedia` (image with the existing lightbox, or a short notice for non-image media, plus the text). Unflagged media-shaped JSON stays in the JSON viewer. - `go.mod`: pin [coder/fantasy `9a3598480a71`](https://github.com/coder/fantasy/commit/9a3598480a71e45901bfcfbf566ffa42c6bbdd63), main's `a511353f83eb` plus the batch-order fix `460d4fdd113b` (Chat Completions and compatible transports emit every tool result before synthetic user images). The fix lives on `mike/mcp-tool-media-batch-order`; no upstream PR yet. ## Validation Round 3 dogfood UAT on this head, `522bd76a58a`: **PASS** ([chat](https://dogfood.cdr.dev/agents/5e617636-c6f2-46fc-96e5-97ae35ed822e), real Playwright MCP, Claude Haiku 4.5 and GPT-4o mini through the AI gateway). Verified in the UI and on the wire: image plus text render together, the lightbox opens and closes, both survive a reload; audio shows a notice instead of base64; media-shaped JSON stays in the JSON viewer; BMP on OpenAI Responses and Chat Completions reaches the model as text plus an omission note; SVG and PNG histories switch from Anthropic to OpenAI; cross-provider compaction through the organization override works; a result over 10 MiB persists only text plus a note, and a 5.5 MiB PNG is substituted for Anthropic yet still renders in the UI. Google is not routable on the dogfood gateway (404), so it was not exercised. The remote agent's own FAIL was a probe artifact (the code viewer renders in a shadow root) and was overturned on its screenshots by the independent runner. Rounds 1 and 2 (`65db84a`, `5001de7`) covered the same API surface earlier. Known limits: signature sniffing does not catch a truncated PNG with a valid header (providers return 400 for it), and only the first binary item of a result is kept. Pre-existing and out of scope: chatd opens a new MCP session per generation action, so Playwright's default per-session browser loses page state between steps (UAT used `--cdp-endpoint`). Reviews: | Review | Head | Outcome | |---|---|---| | Codex 1 to 5 | `472914f` to `bcf4c72` | 16 findings (empty binary promoted over text, no provider caps, SVG on Anthropic, provider-switch gap, Bedrock labels, unbounded media, unvalidated bytes, renderer gating, lightbox test), all fixed | | Codex 6, 7 | `8a9a9e0`, `5001de7` | no major issues; one Responses audio concern [refuted with a live probe](https://github.com/coder/coder/pull/29332#discussion_r4018514197) | | [Codex](https://github.com/coder/coder/pull/29332#pullrequestreview-5226106293) | `83e1a10` | two P2s refuted with serializer captures ([MIME](https://github.com/coder/coder/pull/29332#discussion_r4029188332), [advisor](https://github.com/coder/coder/pull/29332#discussion_r4029160781)); [reassessment](https://github.com/coder/coder/pull/29332#issuecomment-5702295886) clean | | Codex | `522bd76` | [no major issues](https://github.com/coder/coder/pull/29332#issuecomment-5713663081) on the rewritten tests | | @DanielleMaywood | `472914f`, `5001de7` | renderer restructure and explicit audio mapping done; lazy slice cloning removed; two threads left open for her re-check | Local pre-push hooks were skipped per Mike's instruction; CI is the gate. > Xum acted on Mike's (@ibetitsmike) behalf to create this PR. |
||
|
|
0362e5386c |
fix(coderd/x/chatd/chatloop): tell the model to gather context before retrying an exclusive tool (#29281)
When the model batches `advisor` (an exclusive tool) with other tool calls, every call in the batch is rejected and the `advisor` result only says to retry with `advisor` alone. That nudges the model toward re-calling `advisor` immediately, even when the sibling calls were there to gather context for it. Extend the message so the model is told to run the other tools first when it still needs their output, then retry with only the exclusive tool call. Generated by Coder Agents on behalf of @ibetitsmike. |
||
|
|
46c904d181 |
fix: show estimated context usage after compaction (#28984)
After compaction, the context gauge showed the unused-chat placeholder until the next response. It now shows a clearly labeled estimate based on the compacted summary, keeps that estimate after reload, and uses the selected chat model’s context limit. The next response replaces it with measured usage. The estimate stays separate from provider-reported token accounting. Remote dogfood UAT passed the core workflow on cb18cbdac596231acf821cd612c5a8c5d1db1c7b. It has not been repeated on the current head, |
||
|
|
680b2f59cd |
refactor: remove obsolete chatd code and tests (#29228)
Chatd still carries an unreachable interruption flush path, unused state and options, an uncalled workspace MCP cache invalidation callback, and a stream-buffer-drop counter that production never increments. Remove these paths and their dedicated assertions, and update the generated metrics inventory and documentation. Consolidate duplicate publication and rollback tests and move fixture writers and test-only inspection helpers into test files. Retain the interruption persistence, state-machine matrix, legacy message format, and provider compatibility coverage. |
||
|
|
c12355d6e5 |
fix(coderd/x/chatd): carry unanswered user messages through compaction verbatim (#28645)
Fixes [CODAGT-737](https://linear.app/codercom/issue/CODAGT-737/compaction-can-treat-the-last-user-message-as-an-assistant-message) ## Problem Auto-compaction can trigger on a turn whose last history message is a fresh, unanswered **user** message. Two things go wrong: 1. The summary instruction is appended as another user message. Providers that merge consecutive same-role messages (verified against fantasy's Anthropic conversion) fuse it with the user's message, and the summarizer misattributes the user's instruction as assistant intent. 2. After compaction the original user row falls outside the prompt window, so the instruction survives only through summary fidelity. ## Fix Carry unanswered trailing user messages through compaction verbatim instead of summarizing them. All logic lives in `chatd`; `chatloop` is unchanged apart from prompt wording. - **Prepare** (`generation_preparer.go`): `pendingUserSegmentStart` locates the trailing user rows on the persisted rows, before provider-switch sanitization (which can drop the assistant row separating them from an already-answered message). Head and tail are converted separately so the summarizer input excludes the tail exactly. Falls back to summarizing everything when no usable segment exists. - **Commit** (`buildCompactionMessages`): replays the tail after the boundary trio as uncompressed, model-only user rows. The existing prompt window query includes them; no SQL, UI, or turn-guard changes. Replay rows carry no usage, so compaction cannot immediately re-fire. - **Prompt** (`chatloop/compaction.go`): the summary prefix no longer claims the assistant "was actively working", which is false in this case. The summarizer prompt is unchanged. - `ARCHITECTURE.md` compaction section updated. ## Testing `go test ./coderd/x/chatd/...` green. New: `TestBuildCompactionMessages_ReplaysPendingUserMessages`, `TestPendingUserSegmentStart`, `TestPendingUserSegmentConversionEquivalence`, `TestPendingUserSegmentDerivedBeforeProviderSwitchSanitization`, `TestDecisionGeneratesAfterCompactionWithReplayedPendingUser`. <details> <summary>Root-cause repro</summary> Replaying the reported scenario through fantasy's Anthropic provider against a capture server: ``` === fantasy prompt (roles as chatd builds them) === [2] role=user "You'll hit compaction now, we'll need to go through some fin..." [3] role=user "You are performing a context compaction. Summarize the conve..." === wire request to Anthropic === [2] role=user, 2 content block(s) block[0]: "You'll hit compaction now, we'll need to go through some final checks..." block[1]: "You are performing a context compaction. Summarize the conversation so..." ``` </details> --- 🤖 This PR was generated by Coder Agents on behalf of @f0ssel. |
||
|
|
ac824cd235 |
chore(coderd): remove dead code from chatd (#29008)
Removes production code in `coderd/x/chatd` that nothing reaches, the tests that only existed to exercise it, and option/config fields that callers filled in but chatd never read. No runtime behaviour changes; the only edits outside `coderd/x/chatd` drop the now-nonexistent fields at their three call sites (`coderd/coderd.go`, `coderd/exp_chats.go`, `coderd/coderdtest`). Net: 42 files, +147 / -1065. Verification method for every item: `golang.org/x/tools/cmd/deadcode` (with and without `-test`) rooted at `cmd/coder`, `enterprise/cmd/coder` and every package that imports chatd; a `go/packages` pass counting `TypesInfo.Uses` per declared type, const, var, field and method across the same roots, split into test and non-test use sites, plus a read-vs-write split for struct fields; `rg` over the whole repo including `site/src` for json tag names; then `go build ./...`, `go vet ./coderd/x/chatd/...`, `golangci-lint run ./coderd/x/chatd/... ./coderd/ ./coderd/coderdtest/`, and `go test ./coderd/x/chatd/...` (all 20 packages) locally. <details> <summary>Evidence per deletion</summary> | Removed | What it was | Why unused | |---|---|---| | `subagent.go` `resolveConfiguredModelOverride`, `parseModelOverride`, `parsedModelOverride` (~135 lines) + `TestResolveConfiguredModelOverride_*`, `TestParseModelOverride` | raw-string model override resolver | Only callers were its two tests; production resolves overrides through `resolveOrganizationModelOverride` (`title_override.go`, `compaction_override.go`, `subagent.go`). | | `chatd.go` `removeSkillIndexMessages`, `isSkillIndexMessage` + `TestSkillIndexRefreshReplacesStaleAliases` | skill-index refresh helpers | Zero production references; the test simulated a refresh path no production code performs. | | `chatd.go` `chatMessage`, `newChatMessage`, `withCreatedBy`, `appendMessageFields`, `BuildSingleChatMessageInsertParams` (~90 lines) | insert-params builder | Exported, but every caller was a `chatd_test` file (6 sites). Replaced by a 25-line `singleChatMessageInsertParams` helper in `chatd_helpers_test.go`. | | `chatd.go` `Server.closeInflightAdmission` | one-line wrapper around `inflightClosed.Store(true)` | Single call site (`Close`); inlined. | | `chatd.go` `Server.workerID`, `instructionLookupTimeout`, `pendingChatAcquireInterval`, `maxChatsPerAcquire`, `chatHeartbeatInterval` | Server struct fields | Assigned in `New` and never read anywhere (the values reach the worker through `chatWorkerOptions`, not these fields). | | `Config.InstructionLookupTimeout`, `homeInstructionLookupTimeout`, `coderd.Options.ChatdInstructionLookupTimeout`, `coderdtest.Options.ChatdInstructionLookupTimeout` | timeout plumbing | The value only landed in the dead `Server.instructionLookupTimeout`; no CLI flag sets it and nothing reads it. | | `PromoteQueuedOptions.CreatedBy` | option field | Set by `coderd/exp_chats.go` and three tests, never read in chatd (promotion keeps the queued row's `created_by`). | | `SubmitToolResultsOptions.DynamicTools` | option field | Set by `coderd/exp_chats.go` (which re-read `chat.DynamicTools` just to pass it) and three tests, never read in chatd. | | `attachments.go` `buildAssistantPartsForPersist` (whole file) | assistant part builder | Stale copy left by #26270; production uses `buildAssistantParts` in `message_conversion.go`. Its six tests were retargeted to the live function (now `message_conversion_internal_test.go`) and a dead-copy assertion in `subagent_internal_test.go` was dropped. | | `chatloop.RunOptions` (94 lines) | old loop options struct | Only two of 23 fields were ever populated, for one internal call to `prepareMessagesForRequest`, which now takes `model, logger` directly and no longer returns the ignored canonical slice. `GenerateAssistantOptions` / `ExecuteLocalToolsOptions` are the live types. | | `chatloop.ErrDynamicToolCall`, `ErrStopAfterTool`, `PendingToolCall`, `PersistedStep.PendingDynamicToolCalls` | dynamic-tool exit protocol | Zero references outside their declarations (one stale comment in `chatdebug/service.go` updated). | | `chatloop.CompactionOptions.Persist`, `.OnError` | callbacks | `Persist` had zero references; `OnError` was set only by two tests that asserted it was never invoked. | | `chatloop` `stepResult.shouldContinue` + its computation | flag | Written once, never read. | | `chatloop.StateWaiting` | metric label const | Zero references. | | `generation.go` `generationOutcome.InsertedMessages`, `runnerActionMessage` | outcome field | Built on every commit, never read. | | `chatstate` `ExecutionState.IsArchived`, `AllOwnershipStates`, `OwnershipState`/`StateU`/`StateO`, `TransitionAcquire` | state-machine leftovers | Zero references (the ownership consts became dead once `AllOwnershipStates` went). | | `chatstate` `ExecutionState.QueueNonEmpty` | predicate | Only referenced by the test asserting its own table; the sibling `IsRunnable` check in that test is kept. | | `chatdebug.Attempt.RetryClassification`, `.RetryDelayMs` | json fields | Never assigned anywhere, `omitempty`, and `retry_classification`/`retry_delay_ms` do not appear in `site/src` or `typesGenerated.ts`. | | `chatdebug.RunContext.Kind`, `StepContext.HistoryTipMessageID`, `RecorderOptions.Provider`/`.Model`, `stepHandle.opts`/`.usage`/`.err` | context/handle fields | Written (from `chatd.go`, `chatloop/compaction.go`, `modelcall.go`, `recorder.go`) and never read. The DB run row keeps its own `kind` column. | | `chatprovider.MergeProviderAPIKeys` + `TestMergeProviderAPIKeys_PreservesProviderRegions` | key merge | Superseded by `ResolveUserProviderKeys`, whose region test already covers the same behaviour. | | `chatprovider` `providerCreationError` | one-line `xerrors.Errorf` wrapper | Single call site; inlined with identical message. | | `chatretry.IsRetryable` | classifier wrapper | Zero references. | | `computeruse.IsTool` + `TestComputerUseTool` | one-line forwarder to `fantasyopenai.IsComputerUseTool` | No production caller; two test assertions now call the fantasy function directly, and the dropped test only asserted the library's own constructor. | | `chatprompt.ToolResultToPart` | one-line forwarder to `codersdk.ChatMessageToolResult` | Single call site; inlined. | | `subagent_catalog.go` `formatSubagentDefinitions` | forwarder with `nil` overrides | Single call site; inlined. | | `worker.go` `chatWorker.chatWorkerID` | getter | Only tests called it; they read `opts.WorkerID` directly now. | | `chattest.AnthropicErrorResponse`, `OpenAIServerErrorResponse`, `ChatMessageWithParts` (`messages.go` removed) | test fixtures | Zero references. | | `chattest` `AnthropicRequest.Options`, `OpenAIRequest.Options` | `json:",inline"` map fields | `encoding/json` has no inline tag (the TODO on each said so); never populated, never read. | </details> <details> <summary>Considered and kept</summary> - `chatstate.CreateChat` (test-only thin wrapper over `insertChat`, production uses `CreateChatWithID`): documented package API with 25 test call sites; not worth the churn. - `chatprompt.MarshalContent` / `MarshalToolResult`: test-only, but they deliberately produce legacy-format DB rows so the read path's legacy handling stays covered. - `chatstate.AllowedExecutionTransitionsFrom` / `AllowedInputStates` / `AllowedExecutionTransitionOutputs`, `chatdebug.StepFromContext`, `WithClock`, `WithStaleThreshold`, `WaitUntilIdleForTest`, `PublishBuffer.BufferedChannels`, `chattool.NewWorkspaceMCPTool`: test seams that tests actively use to reach otherwise unexported state. - `AgentCapacityLimiter` interface (one production implementation): tests inject fakes. - `applyHookResultMessages` / `appendHookResultMessages`: a named pair over one placement enum, both used. - Nil-receiver guards on `*Metrics`, `*Result`, `*streamSyncPoller`, `*runnerDebugTurn`: nil is a supported "disabled" value on those paths. - Tests flagged by heuristics as assertion-light (`TestNopMetrics_DoesNotPanic`, `TestStepHandleFinish_NilHandle`, `TestRetry_OnRetryNilDoesNotPanic`, the `chatstate` synthetic-cancellation tests that assert through helpers): they cover real nil-safety or delegate assertions to helpers; kept. </details> <details> <summary>Out-of-scope findings (not changed here)</summary> - `chatdebug.Service.FinalizeStale` has had no production caller since #26270 removed the background sweep loop; only its tests call it, and comments in `chatd_debug.go` and `database/querier.go` still describe the loop. Deleting it would erase the implementation a fix needs, so it is left for a decision. - `coderd_chatd_chats` (gauge) and `coderd_chatd_stream_buffer_dropped_total` (counter) are registered and documented in `docs/admin/integrations/prometheus.md` but never set or incremented in production; `Metrics.RecordStreamBufferDropped` is test-only. Removing them changes the exported metric set and the docs, so they are left alone. </details> ## Declined review findings None yet. > Xum (AI agent) opened this PR on behalf of @ibetitsmike. |
||
|
|
b428c8feb6 |
feat(coderd): bill completed local tool batches in agent runtime (#28360)
**Stack**: #28359 → **#28360 (this PR)** → #28361 → #28362 ## Summary Completed local tool batches now count toward Coder Agent runtime (`chat_messages.runtime_ms`, the source of truth for `hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid double counting, and without multiplying runtime for parallel tool calls. Second of a four-PR stack splitting #28211. Stacked on #28359. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime). ## Problem Agent runtime previously measured only model invocation wall clock (stream open to fully consumed). Time spent executing local tools between steps, including file operations, terminal commands, workspace provisioning, and MCP tools, was deliberately excluded, which undercounts the product definition of "actively processing a task". Naive inclusion has two hazards: a batch of parallel tool calls would bill N windows for one wall-clock wait, and `wait_agent` would re-bill child agents that already bill their own model and tool time. ## Fix Each completed local tool batch bills one window: the union of the billed tools' execution intervals, persisted as `runtime_ms` on a dedicated usage record appended after the batch's tool-result rows. The record is a tool-role message with `visibility='model'`, so it never reaches the API, SSE, or clients, and prompt replay drops it because its single internal `tool-batch-usage` part converts to no provider content. Its content carries an audit payload (`billed_ms`, `billed_calls`) so the billed window is inspectable after the fact; real tool-result rows never carry batch-level runtime. Concurrent calls all start at batch start, so 5 parallel 10s reads bill 10s, not 50s; serial calls (`SerialToolCalls`) count only from their own launch, so unbilled waits before them do not count. Sub-agent orchestration tools (`spawn_agent`, `wait_agent`, `message_agent`, `interrupt_agent`, `list_agents`, `list_subagent_models`, plus the deprecated `close_agent` alias) never extend the window: every chat, including children, applies the same rules to its own runtime, so a parent's `wait_agent` window would double count. A lone `wait_agent` bills 0 and appends no usage record; `execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins the unbilled set to the registered sub-agent tool catalog so they cannot drift. `GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms` role-agnostically, so the usage record is picked up with no schema, query, or cron changes. Client-executed dynamic tools, external agents, parked/idle time, and retry backoff remain unbilled. Interrupted batches still bill only the model window; the rest of the stack adds partial-window billing via the same usage record. `ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder` (`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order execution stamps. This PR wires no recorder; the next PR in the stack connects it to the message part buffer for interrupt billing. **NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from deploy forward. There is no backfill and no feature flag. |
||
|
|
d44aae37de |
refactor(coderd/x/chatd): return PersistedStep directly from ExecuteLocalTools (#28359)
**Stack**: **#28359 (this PR)** → #28360 → #28361 → #28362 `chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so callers had to reach through `outcome.Step` for every field. `ExecuteLocalTools` now returns the `PersistedStep` directly and the wrapper type is deleted. No behavior change. First of a four-PR stack that makes local tool execution count toward Coder Agent runtime (splitting #28211). This refactor keeps the rename noise out of the feature PRs. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime). |
||
|
|
a48aedd924 |
fix(coderd/x): surface Gemini malformed-function-call stream deaths as retryable errors (#28470)
## Problem
When Gemini's OpenAI-compatible endpoint rejects a model-generated
function call server-side, it ends the SSE stream cleanly with the
nonstandard finish reason `function_call_filter:
MALFORMED_FUNCTION_CALL` after streaming only thought summaries; the
rejected call never reaches the wire. chatd treated this as a normal
completion: fantasy maps the unrecognized finish reason to `unknown`,
the reasoning block never closes (the only non-thought delta is the
`</thought>` marker, which the transport seam strips to an empty string,
and the openaicompat hook only ends reasoning on a non-empty content
delta), so the step accumulates no content and the generation loop
finishes the turn as complete. The user sees the model think for ~40
seconds and then nothing: no assistant message, no `last_error`, chat
status `waiting`. Observed twice in a row in production on
gemini-3.7-flash, with "Resume" reproducing it identically.
## Fix
Two independent layers:
- `coderd/x/googleopenai`: the stream rewrite now converts any chunk
whose `finish_reason` starts with `function_call_filter` into an
OpenAI-style SSE `{"error": ...}` event embedding the raw reason.
openai-go turns error-bearing events into stream errors, so the failure
rides the existing stream-error path instead of ending the stream
cleanly.
- `coderd/x/chatd/chaterror`: classifies that injected error as
retryable (kind `generic`, provider `google`) with a clear user-facing
message, so the existing generation retry machinery re-runs the step and
persists a `last_error` if retries exhaust.
- `coderd/x/chatd/chatloop`: provider-agnostic guard: a step that
produced no user-visible content and no tool calls under a finish reason
of `unknown`, `error`, `other`, or `tool-calls` (a tool-calls finish
that delivered zero calls) now returns a retryable error instead of
silently completing the turn. `stop` and `length` finishes keep their
existing semantics.
Tests cover the seam rewrite (live-capture SSE shape plus standard
finish reason passthrough), the new classification, and the chatloop
guard (error cases plus preserved stop, length, text, and tool-call
behavior). Each layer was red-green verified independently.
Remote dogfood UAT ran against this exact commit: normal reasoning and
tool-call chats on a real model complete cleanly with no spurious guard
errors and no retry loops. The Google-side failure itself is not
deterministically triggerable against live Gemini and is owned by the
unit tests.
> Xum acted on Mike's (@ibetitsmike) behalf.
<!-- xum-attribution: model=claude-opus, thinking=enabled -->
|
||
|
|
9168dbf4d3 |
refactor(coderd/x/chatd): unify LLM model-call construction behind a resolver (#28145)
Centralizes every LLM client build in chatd behind one resolver and makes all model calls behave the same, instead of preserving per-flow quirks. ## Why `chatd` built LLM clients in five places (standard turn, compaction override, quickgen, title override, advisor), each repeating config parsing, route resolution, client construction, and debug wrapping with slightly different behavior. An earlier iteration of this PR centralized construction but preserved every historical quirk behind purpose-specific spec constructors and policy flags. Review concluded the quirks themselves were the problem. ## What changed - `Server.resolveModelCall` (`modelcall.go`) is the single pipeline from a `modelCallSpec` to a ready client plus call metadata. Callers build spec literals inline; the only spec fields are data: model source (chat config, explicit config row, or fixed provider/model pair for computer use), requested reasoning effort, chatd-scoped routing, and build options. - Uniform behavior for every call: - Provider options are always derived. Whole-chat summaries, turn status labels, and chat-model compaction summaries previously sent none; they now follow the config like everything else. - `MaxOutputTokens` always defaults to 32k when the config leaves it unset. The compaction summary call nulls the cap after copying the template because it is non-streaming and the Anthropic SDK rejects non-streaming requests whose cap implies a completion over ten minutes. - Debug recording always wraps the client when chat debug is enabled, labeled with the resolved provider/model. The debug-rebuild paths (`manualTitleDebugSpec`, `quickgenDebugSpec`) are deleted; debug helpers now only create and finalize run records, since run association flows through the generate context, not the model. - Clients are always constructed with the configured model string and unmutated route, so AI Gateway validation sees the name exactly as configured. The resolved identity feeds metadata, logs, and debug labels only. - `resolvedModelCall.newCall` / `newObjectCall` are the only production constructors of `fantasy.Call` / `fantasy.ObjectCall`; object calls now carry the config's sampling parameters. - Net: -240 lines. ## Validation - Full `./coderd/x/chatd/...` suite green (PostgreSQL-backed). - Red-green toggles verified for the three key unifications (option derivation, token default, summary cap removal). - ARCHITECTURE.md: prose the refactor made stale is deleted; no new resolver documentation is added. > Mux acted on Mike's behalf for this PR. |
||
|
|
5480299cef |
fix: serialize empty MCP tool properties as {} not null for OpenAI (#28380)
## Summary
Fixed a bug where MCP tools with an empty `properties` schema had their
tool definitions sent to OpenAI with `"properties": null`, causing
OpenAI to reject the request.
## Problem
When an MCP server reports a tool with an empty input schema, e.g.:
```json
{
"type": "object",
"properties": {},
"required": []
}
```
Coder was forwarding the tool definition to OpenAI as:
```json
{
"type": "object",
"properties": null
}
```
OpenAI's function-calling API rejects `properties: null`, making these
tools unusable with OpenAI models.
## Root cause
The empty `properties` object collapsed to Go `nil` at two layers:
1. `agent/x/agentmcp/manager.go` `toolInputSchemaMap` stripped the
present-but-empty `"properties": {}` (guarded by `len(properties) > 0`),
so the wire copy lost it entirely.
2. `coderd/x/chatd/chatloop/chatloop.go` `buildToolDefinitions` embedded
`info.Parameters` directly into the input schema. When nil,
`encoding/json` marshals it as `null`.
## Fix
- `buildToolDefinitions` now substitutes an empty object when
`Parameters` is nil, mirroring the existing `required` guard (the sink,
protects all tool sources).
- `toolInputSchemaMap` now preserves an empty `properties` object
instead of dropping it (the upstream producer).
## Tests
Added regression tests (failing before, passing after) covering both
layers and asserting the schema serializes to
`{"type":"object","properties":{}}`:
- `TestBuildToolDefinitionsNilPropertiesBecomesEmptyObject`
- `TestToolInputSchemaMapPreservesEmptyProperties`
-
`TestWorkspaceMCPToolInfosFromResources/EmptyPropertiesYieldsEmptySchema`
---
*This PR was generated by Coder Agents.*
|
||
|
|
7724ee281a |
feat: defer MCP tool schemas behind a find_tools search (#28225)
## Summary When the `mcp-tool-search` experiment is enabled, chatd stops inlining connected MCP tool schemas into every generation. It instead exposes a built-in `find_tools` tool whose description carries a compact catalog of the deferred tools, and only ships full JSON schemas for tools the model has activated by searching or by calling them directly. Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760). ## Problem Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling MCP servers) inline over 100k tokens of tool schema definitions into every generation. Initial uncached requests reached ~216k tokens with time-to-first-token close to nine minutes, while the model typically invokes only a handful of tools per turn. ## How it works - `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP tools whenever the experiment is enabled. Native, dynamic, provider, skill, and transport tools are never deferred. - `find_tools` embeds a server-grouped catalog in its tool description (degrading to names-only, then counts-only, then a constant-size summary past a context-scaled size cap) and scores keyword matches across tool names, descriptions, parameter schemas, and server metadata. Queries can scope to one server with a `server:` prefix, and exact `names` arguments always activate. - Activation state is ephemeral: it is re-derived each generation from surviving chat history (`find_tools` results and direct calls to deferred tools), so activations naturally lapse when compaction summarizes them away. Aggregate activated schema weight is capped at 10% of the context window, shedding the least recently activated schemas first; `find_tools` shares that budget across parallel calls in one step. No new persistence. - Deferred tools stay registered for execution, so the model can call a cataloged tool directly without searching first; the schema is activated for subsequent steps. - Fail-open: the experiment being disabled, an empty candidate set, or an MCP tool named `find_tools` all disable deferral, leaving today's behavior byte-identical on the wire. - Prometheus counters/histograms track `find_tools` calls, matches, activations, and deferred token weight. - The conversation timeline renders `find_tools` calls with a collapsed search summary and expandable match list, falling back to the generic renderer on malformed payloads. ## Validation - Unit tests for the catalog, matcher, experiment-gated decision, and activation derivation; end-to-end chatd generation tests covering search-then-call, direct-call activation, experiment-off wire parity, compaction lapse, and subagent tool gating. - Storybook interaction tests for the timeline rendering and malformed-payload fallback. - Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP server and Anthropic model, direct calls without prior search, activation persistence across turns, experiment-off parity, and clean UI/console. > Disclosure: Mux (AI agent) authored this PR on Mike's behalf. |
||
|
|
4b7494be72 |
feat: harden chat generation runtime instrumentation for billing (#27451)
Closes CODAGT-835 ## Summary `chat_messages.runtime_ms` becomes the billing source of truth for Coder Agents runtime (summed hourly by #27312), but it was built for debugging: the June refactor (#26270) silently stopped recording tool-step runtime, compaction was never measured, and interrupted turns lost their partial runtime entirely. This PR defines the billable metric, closes the paths that dropped it, and documents the definition where the data lives. ## The billable definition **`runtime_ms` is the wall-clock duration of the model invocation that produced the persisted message content**, measured from just before the provider stream opens until it is fully consumed. What counts: - Assistant generation steps, in top-level and sub-agent chats (sub-agents are ordinary chats on the same generation path). - Compaction summarization calls, persisted on the compaction assistant message (**new**). - Interrupted attempts: the message-part episode's lifetime is persisted on the partial assistant message committed by `FinishInterruption`, so partial generation time survives interruption (**new**; measured via a new `Buffer.EpisodeDuration`, which works even though the generation goroutine and the interrupt task are different tasks). What deliberately does not count (each is documented in code and docs): - **Local tool execution.** Tool wall time includes idle waits, most importantly `wait_agent` polling a sub-agent chat that already bills its own model invocations; billing the batch would double count, and excluding one tool from a concurrent batch's wall time is ill-defined. Pre-refactor instrumentation did include tool time; this makes the exclusion an explicit product definition instead of a silent regression. - **Failed model calls whose output is discarded** (retried attempts, terminal errors, content-filter refusals). They persist no content, so they bill nothing; billing errs toward undercounting. Notably a stream-silence timeout can burn 10 idle minutes before a retry, which should not be billable "active generation". If product later wants failed attempts billed, that needs a place to persist runtime on error turns (`FinishError` inserts no rows today) and is a deliberate follow-up, not instrumentation drift. - **Ancillary calls that produce no chat messages** (title generation, advisor, turn summaries) and all idle/parked time (`requires_action`, queueing). The definition is documented as `COMMENT ON COLUMN chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc comment on `ChatMessage.RuntimeMs`), on `chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in the Spend Management docs page. ## Index for the hourly scan None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an hour-wide `created_at` range, which the existing `idx_chat_messages_created_at` b-tree already serves; the residual `runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial index would add permanent write amplification for a query that runs once an hour. > [!NOTE] > Migration 000551 is also claimed by #27312; whichever merges second renumbers via `fix_migration_numbers.sh`. ## Tests - End-to-end: the existing full-server generation test now asserts `RuntimeMs.Valid` on the committed assistant row (it previously read `.Int64` without checking `.Valid`, so it passed on NULL). - Interrupted turn: full task-level test (real DB, mock clock) asserting the partial assistant message persists the attempt's runtime. - Errored stream: asserts a failed invocation yields no step and no runtime. - Tool-using turn: asserts runtime lands on the assistant row only and tool rows stay NULL. - Compaction: asserts the summarization call duration is recorded and lands on the compaction assistant message only. - `messagepartbuffer.EpisodeDuration` unit coverage. Blocks: CODAGT-843 (B3), CODAGT-838 (D8). --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Hugo Dutka <hugo@coder.com> |
||
|
|
c17bed25e0 |
feat: wire chat lifecycle hooks into chatd (#27429)
Wires chat lifecycle hooks into chatd, gated by the `agent-lifecycle-hooks` experiment. Part of the lifecycle hooks stack (#27401, #27428, #27430). See `docs/admin/setup/chat-lifecycle-hooks.md` for the consumer-facing contract. ## Summary When a hook URL is configured, chatd dispatches `session_start`, `user_prompt_submit`, `pre_tool_use`, `post_tool_use`, `pre_compact`, `post_compact`, and `stop` events to the consumer and applies its responses. ## Design - **Stateless**: Coder stores no hook dispatch or decision state. Delivery is at least once; consumers deduplicate on stable payload identifiers (chat ID, event type, tool-use ID) and answer duplicates with the same decision. - **Admission-time prompt effects**: `user_prompt_submit` dispatches exactly once per submission (create, send, queue, edit, subagent spawn) and folds its effects into the stored prompt as typed message parts: original-or-overridden user parts, then model-only `hook-context`, then a user-visible `hook-notice`. Hook context is stripped from every client-facing conversion; hook notices are excluded from model prompts. The server rejects hook parts in client-submitted content. - **Tool gating**: `pre_tool_use` allow can override tool input; deny becomes a synthetic denied tool result, with any returned model context persisted as a model-only transcript row so it never reaches clients. The denial text identifies an external policy (the deployment's lifecycle hook) as the source and marks the decision as persistent, so the model explains the denial instead of retrying it or misreporting it as an infrastructure failure. - **Fail closed**: a dispatch failure rejects the triggering request or moves the chat to the error state in the same transaction as the affected step, so a runnable state is never published with unapproved content. - **Admission before persistence**: `pre_tool_use` is dispatched for the calls the model produced, before the assistant message is stored. See "Staged tool admission" below. - **Fresh dispatch per tool call**: every non-provider-executed tool call is decided by its own `pre_tool_use` dispatch; Coder never reuses an earlier decision on the consumer's behalf. Retries re-dispatch the same logical event. ## Structure All hook dispatch flows through one seam: entry points build a `chathooks.Chat` (chat identity) and a `chathooks.Message` (event details) and call `Trigger.Trigger`, the only component that talks to the dispatcher. The integration lives in the `coderd/x/chatd/chathooks` subpackage, split by responsibility: - `trigger.go`: the trigger seam; builds the wire envelope per event, normalizes deny into a typed error, and holds the package's single enabled-check. - `effects.go`: pure conversion of hook results into transcript rows and prompt parts. - `errors.go`: failure classification (dispatch error messages, denial mapping, tool-result dispatch-failure scanning). - `tooluse.go`: the tool-call gate (`pre_tool_use` preflight, `post_tool_use` payloads, applying admitted input to the step). Server-bound glue stays in `coderd/x/chatd/hook_server.go`: the chat-parking dispatch error handlers, the step-commit row insertion wrappers, and the dynamic post-tool-use state loader, which depends on chatd validation types. This PR adopts the `codersdk/x/agenthooks` and `coderd/x/agenthooks/dispatch` import paths introduced at the tip of #27401; intermediate commits still reference the pre-move paths and are not individually buildable. ## Staged tool admission `pre_tool_use` originally ran at tool execution time, which is after the assistant message carrying the tool call was already committed. An `input_override` therefore had to rewrite stored message content in place. @hugodutka pointed out that chatd treats message content as immutable, and that the rewrite was a shortcut rather than a requirement. It was also a correctness problem in its own right: the rewrite only updated the database, so the transcript could show one input while a different one had executed. The hook now runs before the step is persisted: ```text provider stream ends (tool calls complete, in memory) -> pre_tool_use dispatch per call -> ONE transaction: assistant row with admitted inputs, synthetic denials, hook rows -> execute ``` The step is inserted once, carrying the input the tool runs with. `UpdateChatMessageContentByID` and `Tx.UpdateMessageContent` are deleted from #27428, so message content stays immutable. Two consequences, both intentional: - **Clients converge rather than wait.** Tool-call parts still stream live, so a rewritten call briefly shows the model's proposed input before the committed message replaces it. The chat store already clears stream state when an assistant message arrives, so the stored input wins with no frontend change and no added latency before tool cards appear. - **A call already in history was already admitted.** Execution consumes the stored input instead of dispatching a second decision, which keeps one dispatch and one set of hook effects per call. A consumer policy change between admission and execution applies to later calls, not to calls already admitted. The per-chat debug endpoint still records the provider's original tool input. Its purpose is to report provider behavior, and it requires an explicit per-chat debug flag; the invariant here covers the transcript. ## Configuration Adds `chat-hook-url`, `chat-hook-secret`, `chat-hook-timeout`, and `chat-hook-enabled` deployment options with startup validation. The flags are hidden from `coder server --help` while the feature is experimental; the setup guide documents them. ## Tool input validation Built-in tool arguments reach a consumer as raw JSON with key spelling preserved, but the tools decode those bytes with Go, which matches struct fields case-insensitively and keeps the last match. A policy reading `path` could therefore authorize one value while the tool executed another, and a lone case variant such as `{"PATH":"/secret"}` was invisible to a policy checking for `path`. Coder now rejects a built-in tool call whose input repeats a key or spells a schema property with different capitalization, before the `pre_tool_use` dispatch, so a consumer is never asked to authorize bytes whose meaning depends on the reader. Rejected calls produce an error result the model can retry; unambiguous calls in the same batch still run. A consumer-authored `input_override` is rechecked after the dispatch and fails the turn closed, because the model cannot correct it. Dynamic and MCP inputs are excluded because the client and the workspace agent execute those calls rather than coderd. Two paths needed more than a schema check. Execution resolves a deprecated tool name to its canonical tool, so validation resolves aliases first. The `edit_files` decoder also reads `search` and `replace`, which its schema does not advertise, so those aliases are now matched exactly and their case variants ignored. A hook denial now returns a structured 403 carrying `kind: "hook_denied"`, mirroring the dispatch-failure response that already carries its own kind. Without it a client cannot tell a policy decision apart from a generic failure, and the chat UI titled a denial "Request failed". Adding a kind needs no migration: `ChatErrorKind` is persisted only inside the JSONB `chats.last_error` column, whose decoder accepts unknown kinds. The hook docs also correct the tool-input convergence window. A batch dispatches sequentially before the assistant row commits, so the original input stays visible for a span that scales with the number of tool calls in the step rather than a single hook timeout. > This PR was written by Mux, an AI coding agent, on Mike's behalf. |
||
|
|
e9951b07d4 |
fix(coderd/x/chatd/chatloop): surface reasoning-only content-filter refusals as terminal errors (#27476)
## Problem Anthropic can end a stream with `stop_reason: "refusal"` after reasoning content has already streamed. The content-filter guard in `chatloop.GenerateAssistant` only fired when the step content was completely empty, so a reasoning-only refusal bypassed it: the turn finished as `status=waiting` with `last_error=null`, and the user saw the chat silently stop mid-turn with no explanation. This looked like a Coder fault when the provider had rejected the response. Observed twice in dogfood on 2026-07-23 (chat `c72f99fc`, debug steps show `finish_reason=content-filter` with reasoning-only content). ## Change Treat a content-filter finish as terminal whenever the step produced no user-visible output. A new `hasUserVisibleContent` helper counts any non-reasoning part (text, tool call, tool result) as user-visible; reasoning-only or empty steps now return the existing `contentFilterError`, which flows through the established pipeline: classified `ChatErrorKindContentFilter` (non-retryable, refusal category/detail when provided), persisted `chats.last_error`, streamed error event, and the "Response blocked" callout in the chat UI. Behavior for steps with visible text or tool calls is unchanged, and the frontend needs no changes. ## Testing - New regression subtest `ReasoningOnlyContentSurfacesTerminalError` (reasoning stream then content-filter finish) beside the existing empty-content and partial-content subtests, which are unchanged. - `go test ./coderd/x/chatd/...` and lint pass. - Dogfood UAT against a local dev instance with a mock Anthropic upstream passed all three scenarios: reasoning-only refusal shows the "Response blocked" callout with `last_error.kind=content_filter` and no retry affordance; text-then-refusal still completes normally; empty refusal still errors. > This PR was created by Mux acting on Mike's behalf. |
||
|
|
3227cac217 |
feat: add manual chat compaction via /compact (#27081)
Adds a user-triggered `/compact` action for Coder Agents chats: typing
`/compact` in the composer (or picking it from the `/` trigger menu)
summarizes the conversation so far to free up context window space.
## How it works
- New `POST /api/experimental/chats/{chat}/compact` endpoint
(owner-only, RBAC `ActionUpdate`, excluded from the public API reference
via `x-apidocgen skip`). It marks the chat with a durable one-shot
`chats.compaction_requested_at` signal and moves it `waiting -> running`
via a new `RequestCompaction` state transition; no message row is
inserted. AI Gateway attribution needs no per-request key: generation
preparation resolves the owner's synthetic API key (#27170) like any
other turn.
- `RequestCompaction` hands off chat ownership (clears
`worker_id`/`runner_id`) so a worker acquisition hint is published;
since the transition changes no history, the previous runner could
otherwise miss the request under reordered pubsub delivery.
- The background chat worker picks the chat up like any other turn. A
pending manual request takes precedence over turn completion in the
generation decision, and forces compaction even below the automatic
threshold (and when compaction is disabled via threshold=100). The
commit step consumes the request marker in the same transaction; any
transition that ends the turn clears stale markers.
- The summary triplet reuses the automatic-compaction path, now tagged
with a `source` (`automatic` | `manual`) that is plumbed through
streamed progress parts, persisted tool JSON, and the UI label
("Summarized (manual)").
- Validation order: busy chats reject with 409 (state-machine conflict),
empty/already-compacted chats with 409 "nothing to compact", archived
chats with 400; the owner usage-limit check runs last so no-op requests
surface the specific conflict instead of a limit error.
- Web UI: the `/` trigger menu now has a built-in "Commands" group
listing `/compact`; submit intercepts exactly `/compact` and calls the
endpoint instead of sending a message. A personal or workspace skill
named `compact` takes precedence over the built-in command; while skill
collisions are still resolving, an exact `/compact` submission is
blocked with a retryable hint instead of leaking as message text.
History and queued-message edits are never intercepted. After
compaction, the context usage indicator resets to its unknown state
until the next assistant response reports fresh usage, instead of
showing the stale pre-compaction number.
- codersdk: `ExperimentalClient.CompactChat`.
Worker-path execution (rather than compacting synchronously in the
handler) reuses the existing lock fencing, live "Summarizing..."
streaming, retry accounting, restart resilience, and debug-run
observability. Rationale documented in `coderd/x/chatd/ARCHITECTURE.md`.
## Testing
- State machine: transition-matrix coverage for `RequestCompaction`,
marker lifecycle tests (carried by lease renewals/queue appends, cleared
by terminal transitions, consumed by commit), ownership handoff +
acquisition hint assertions.
- Worker: decision-ordering and forced-compaction unit tests;
active-server end-to-end test (manual compact below threshold produces a
`source=manual` summary, returns to `waiting`, no assistant follow-up;
busy chat rejected).
- API: success, archived, non-owner, RBAC-denied, empty-chat, no-daemon
cases; usage-limit ordering (at-limit owners still get
state/nothing-to-compact conflicts for no-op requests, with marker
rollback).
- Frontend: Storybook play tests for the Commands menu group, submit
intercept, skill-name collision, queued-edit passthrough, and
manual/automatic tool rendering; unit tests for command availability
resolution and the post-compaction context usage reset.
> This PR was created by Mux, an AI coding agent, working on Mike's
behalf.
|
||
|
|
f997afa220 |
feat(coderd/x/chatd/chatloop): retain user constraints in compaction summaries (#27230)
Compaction summaries drop or soften user-stated constraints, corrections, and prohibitions, so post-compaction assistants repeat behavior the user already corrected. Add a summary prompt bullet that instructs the summarizer to quote them, treat them as standing until revoked, and attribute rules to their true source instead of defaulting to the user. Validated offline on unseen human chats: holdout P1 delta +0.175 (arbitrated), 13/18 cases improve. This improves per-compaction retention only; it does not address deep-chain correction loss. |
||
|
|
cc11c8a536 |
feat: surface model content-filter refusals as a blocked chat error (#27118)
Blocked turns from a provider's content filter (Anthropic's `refusal` stop reason with empty content) previously ended silently on the "Thinking" spinner. They now end as a terminal `content_filter` error that renders as a "Response blocked" message with the provider's category and explanation. <img width="888" height="335" alt="image" src="https://github.com/user-attachments/assets/cef85a59-4091-4e62-9d45-1eb06748db48" /> Closes CODAGT-611 Follow-ups will involve implementing fallbacks, but this alone is pretty important |
||
|
|
a567f6a89f | feat: allow admins to override the chat compaction model (#27151) | ||
|
|
0ae4554de9 | fix(coderd/x/chatd/chatloop): remove compaction timeout (#27007) |