Files
coder/coderd/x/chatd/message_conversion.go
T
Michael Suchacz c28ae0dc9b feat: trigger compaction independently for the chat and compaction models (#28568)
> [!NOTE]
> Xum (an AI agent) authored this pull request on Mike's behalf.

## Summary

Automatic compaction now fires at whichever of two independent triggers
is reached first, and the UI shows when the organization override's
trigger applies.

- **Chat trigger**: the chat owner's personal threshold (or the chat
model's) against the chat model's window. A threshold of 100 turns off
only this trigger.
- **Organization trigger**: a fixed 80% of the organization compaction
override model's window. The override model's own compression threshold
is ignored. The other 20% is a target margin for the summary prompt and
the summary, not a guarantee, because the trigger reads the previous
step's prompt size.

On main, the chat threshold was applied to the smaller of the two
windows, and nothing in the UI showed it. Example: with a 20K chat model
at 70% and an 8K override, main compacts at 5,600 tokens; this PR
compacts at 6,400 (80% of 8K, shown as 32%).

## Changes

Backend (`coderd/x/chatd`):

- The trigger with the lower token point binds (ties go to the chat
trigger). It drives the automatic compaction gate and the
still-over-limit check, and every summary, automatic or manual, records
it.
- An override that is unset, deleted, disabled, on a disabled provider,
or without credentials for the chat owner is treated as absent: the chat
model summarizes and only the chat trigger applies.
- The summary's output cap is bounded by the summarizer's own window,
not the trigger's.
- The still-over-limit error names what can clear it: the compaction
threshold (chat trigger), an override model with a larger window
(organization trigger), or both when usage is over both points.
- The still-over-limit check compacts once more when the latest summary
was recorded under a different threshold or trigger window. The
`chat_summarized` result gains `trigger_context_limit_tokens`, and
`context_limit_tokens` now records the chat model's window (main
recorded the smaller window). On main, lowering a threshold after a
compaction ended every later turn with the error.
- The still-over-limit check skips a 0% threshold ("always compact"),
which on main ended the run with the error after every compaction.
- `ARCHITECTURE.md` documents the two triggers, the summary cap, and the
still-over-limit rules.

Frontend (`site`), through a shared `compactionTriggers` helper that
mirrors the backend rules (no new API):

- Coder Agents settings: the compaction override row warns when the
organization trigger undercuts chat models' default triggers ("the
organization override compacts chats at 80% of {model}'s {N}-token
window").
- Personal thresholds page: an Effective column shows the trigger that
applies, with a popover when the organization override binds. "Compacts
at" follows the same trigger, and a point the chat model's window cannot
reach shows as Off. The header's 100% sentence, and the row warning when
a member types 100, say an organization override may still compact chats
when one is set or its organization's overrides failed to load (the
header also while they load).
- Chat context gauge: the compact-at percent follows the binding trigger
and is annotated when the organization override binds. If the overrides
request fails, it shows the chat-only value with "(organization override
not loaded)", or "Compaction off (organization override not loaded)"
when the chat trigger is off.
- An override whose provider the user cannot use is ignored on the gauge
and the personal page, matching chatd.
- The shared `formatContextLimit` prints counts below 1,000 exactly (an
800-token compaction point showed "~1K") and 999,500 to 999,999 as
"1.0M" instead of "1000K". It also labels the model pickers; their other
values are unchanged.
- `HelpPopoverIconTrigger` lets callers set their own `aria-label`; only
the Effective column's popover does.

Known v1 limits: the Coder Agents settings alert is organization-scoped,
so it can warn about an override that some members fall back from for
lack of provider credentials. The gauge checks the viewer's provider
access, so on a shared chat its label can differ from the owner's
runtime trigger (Codex P2, declined). Compaction itself is unaffected.

## Screenshots

From remote UAT round 13 on 093076f402: an isolated test deployment
with a local fake LLM. The models are test fixtures named for their
context windows: Chat model 20K (own threshold 70%), Chat model 5K
(40%), and Chat model 8K (50%). Chat model 8K is also the organization
compaction override, so the organization trigger is 80% of 8,000 = 6,400
tokens, which is 32% of Chat model 20K's window. Chat model 8K's own 50%
applies only to chats that use it as the chat model; changing it to 10%
or 100% left every value below unchanged.

**Coder Agents settings:** the organization trigger undercuts Chat model
20K's 70% default.

![Override warning: 80% of Chat model 8K's window undercuts Chat model
20K](https://github.com/user-attachments/assets/188c92e6-faaa-4e9a-a563-567cbf98af85)

**Personal thresholds:** the organization trigger binds Chat model 20K
at 32%. Chat model 5K (40%) and Chat model 8K (50%) keep their own chat
triggers.

![Effective column: 32% from the organization trigger on Chat model
20K](https://github.com/user-attachments/assets/9949be4e-9417-4cfc-a683-f1367a1dd77a)
![Organization override popover: 80% of Chat model 8K's window, about
32% of this model's
window](https://github.com/user-attachments/assets/4bc3e002-0be1-46fd-91f0-aa27503034af)

**Personal thresholds, organization settings request failed** (the first
red line is the injected error). Rows fall back to chat-only values.

![Organization settings load failure above usable chat-only
rows](https://github.com/user-attachments/assets/dcf48546-aa47-4f82-a6eb-85f106b4bf45)

**Chat context gauge:** organization trigger binds (Chat model 20K),
chat trigger binds (Chat model 5K), overrides request failed (injected).

![Gauge: Compacts at 32% (organization
override)](https://github.com/user-attachments/assets/cd5402d9-9849-4254-a7a7-6623c4006a2a)
![Gauge: Compacts at
40%](https://github.com/user-attachments/assets/f372c9e9-72ef-41c8-950c-e77d74965f65)
![Gauge: Compacts at 70% (organization override not
loaded)](https://github.com/user-attachments/assets/ab829746-2ab5-4743-848f-dce4e806de3a)

**Still-over-limit errors from real runs:** organization trigger, chat
trigger, both.

![Organization-only over-limit
error](https://github.com/user-attachments/assets/f210807a-a2cc-47a0-807c-f1404373bb28)
![Chat-only over-limit
error](https://github.com/user-attachments/assets/eecec38e-97de-49b6-8670-b32090103bbb)
![Combined over-limit
error](https://github.com/user-attachments/assets/459a6d08-7215-492d-93f5-96a4f767a09e)

## Testing

- Go: `TestBindingCompactionTriggerSource`,
`TestCompactionStillOverLimitError`, `TestCompactionStatusFromHistory`,
and `TestActiveServer_CompactionModelOverride` cover the 80% point
(fails if the override's own threshold is read again), user threshold
100 leaving the organization trigger on, the summary cap for automatic
and manual compaction, the over-limit messages, and recompaction after a
trigger change. The site binding tests mirror the Go table row for row.
- Site: `compactionTriggers` unit tests and Storybook stories for the
three surfaces, including the overrides fetch failure.
- On c6f8750479: `go vet`, the `chatd` and `chatloop` tests, and the
AgentsPage, AISettingsPage, and HelpPopover unit tests (2,274 passed, 2
skipped) pass, and CI is green. On 987cefe399, tsc, the five affected
story files (109 passed), and `make pre-commit` also passed. 093076f402
only changes `ARCHITECTURE.md`; the pre-commit hook's markdown format
and lint checks pass on it.
- On 3cd5b08f1f: tsc, the new `formatContextLimit` test (fails on the
old helper for 800 and 999,999), the `compactionTriggers` and `aiModels`
unit tests (224 passed), and the affected story files pass, and the
pre-commit hook passed on all three commits.

## Review record

- Earlier heads: 12 Codex passes and 3 human reviews, all threads
answered. Rebased onto main several times, most recently ff3fa22f68
(kept #30274, #29892, and #29890 in conflicted files).
- coder-agents-review rounds 1 to 18: findings fixed; approved since
round 15. The round 1 `ARCHITECTURE.md` finding was left for the
maintainer, who asked for the update in the last commit.
- coder-frontend-review (eight roles): all pass. Splitting the PR (RO6)
was declined because it is reviewed as one unit; the
`organizationTriggerFor` story helper stays because its three fields
must match the model.
- Design change after UAT round 11: the organization trigger is a fixed
80% of the override window instead of the override model's own threshold
(171d5f568e, 21d2ce7fce, 987cefe399).
- coder-agents-review round 19: copy, naming, and an unreachable
zero-window gate fixed (6e8efb9054, c6f8750479). A summary-time fit
check with chat-model fallback (CRF-156) was declined: the previous-step
lag applies to every trigger, including main's. Round 20 approved
c6f8750479 with no findings.
- Codex: one P2 on 987cefe399 declined (see known limits); no major
issues on c6f8750479.
- Remote UAT on dogfood (fake LLM, isolated test deployment): rounds 8
to 13 PASS. Round 12 on c6f8750479 first failed one criterion that was
written wrong (the admin alert correctly lists an override model that is
also a chat model at 100%); the reviewer endorsed PASS on the same
evidence after the correction. Round 13 on 093076f402 retook the
screenshots with neutral model names and confirmed that the override
model's own threshold (10%, 50%, 100%) does not move the organization
trigger.
- UAT round 13 notes: the header after a failed or pending overrides
load and the sub-1K labels are fixed in 4e172ab5ff and d5888ce15dc;
coder-agents-review round 21 found the same gap in the row warning
(CRF-167, fixed in 3cd5b08f1f). Left as on main: a summary's
`usage_percent` is measured against the binding trigger's window, so it
can exceed 100 (nothing reads it), and the 100% help text shows only for
an unsaved edit.
- Codex: no major issues on d5888ce15d and 3cd5b08f1f.
coder-agents-review round 22 approved 3cd5b08f1f with no open findings.
- coder-frontend-review on those commits: all roles pass after the
fixes. Declined: moving the formatter fix to its own PR (RO5; the owner
asked for it here), adding "tokens" to sub-1K labels (TN6; "~800" reads
like "~16K"), and splitting the component (RP6).
2026-10-11 12:10:59 +02:00

1119 lines
37 KiB
Go

package chatd
import (
"cmp"
"context"
"database/sql"
"encoding/json"
"slices"
"strings"
"time"
"charm.land/fantasy"
"github.com/google/uuid"
"github.com/sqlc-dev/pqtype"
"golang.org/x/xerrors"
"cdr.dev/slog/v3"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/x/chatd/chatloop"
"github.com/coder/coder/v2/coderd/x/chatd/chatprompt"
"github.com/coder/coder/v2/coderd/x/chatd/chatstate"
"github.com/coder/coder/v2/coderd/x/chatd/chattool"
"github.com/coder/coder/v2/coderd/x/chatd/messagepartbuffer"
"github.com/coder/coder/v2/codersdk"
)
const interruptedToolResultErrorMessage = "tool call was interrupted before it produced a result"
type buildCommitStepMessagesInput struct {
modelConfigID uuid.UUID
step stepData
toolNameToConfigID map[string]uuid.UUID
logger slog.Logger
contentVersion int16
hookRewrittenToolCalls map[string]json.RawMessage
}
type stepMessagesForCommit struct {
Messages []chatstate.Message
// ConsumeCompactionRequest clears the manual compaction marker
// atomically with the commit. Set on compaction commits.
ConsumeCompactionRequest bool
}
func buildCommitStepMessages(input buildCommitStepMessagesInput) (stepMessagesForCommit, error) {
contentVersion := input.contentVersion
if contentVersion == 0 {
contentVersion = chatprompt.CurrentContentVersion
}
assistantBlocks, toolResults := splitStepContent(input.step.Content)
assistantParts := buildAssistantParts(input.logger, assistantBlocks, toolResults, input.step, input.toolNameToConfigID, input.hookRewrittenToolCalls)
messages := make([]chatstate.Message, 0, 1+len(toolResults))
if len(assistantParts) > 0 {
assistantContent, err := chatprompt.MarshalParts(assistantParts)
if err != nil {
return stepMessagesForCommit{}, xerrors.Errorf("marshal assistant content: %w", err)
}
messages = append(messages, assistantMessage(input.modelConfigID, contentVersion, assistantContent, input.step))
}
for _, toolResult := range toolResults {
part := chatprompt.PartFromContentWithLogger(context.Background(), input.logger, toolResult)
applyToolMetadata(&part, input.toolNameToConfigID)
if part.ToolCallID != "" && input.step.ToolResultCreatedAt != nil {
if ts, ok := input.step.ToolResultCreatedAt[part.ToolCallID]; ok {
part.CreatedAt = &ts
}
}
content, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{part})
if err != nil {
return stepMessagesForCommit{}, xerrors.Errorf("marshal tool result: %w", err)
}
messages = append(messages, baseMessage(database.ChatMessageRoleTool, database.ChatMessageVisibilityBoth, input.modelConfigID, contentVersion, content))
}
// Usage sums runtime_ms across rows, so the batch window is billed
// once on a dedicated record instead of an arbitrary member row.
stamp, ok, err := batchUsageMessage(input.modelConfigID, contentVersion, input.step.BatchRuntime, input.step.BatchBilledCalls)
if err != nil {
return stepMessagesForCommit{}, err
}
if ok {
messages = append(messages, stamp)
}
return stepMessagesForCommit{
Messages: messages,
}, nil
}
func splitStepContent(content []fantasy.Content) ([]fantasy.Content, []fantasy.ToolResultContent) {
assistantBlocks := make([]fantasy.Content, 0, len(content))
toolResults := make([]fantasy.ToolResultContent, 0)
for _, block := range content {
if tr, ok := asToolResultContent(block); ok && !tr.ProviderExecuted {
toolResults = append(toolResults, tr)
continue
}
assistantBlocks = append(assistantBlocks, block)
}
return assistantBlocks, toolResults
}
func asToolResultContent(block fantasy.Content) (fantasy.ToolResultContent, bool) {
if tr, ok := fantasy.AsContentType[fantasy.ToolResultContent](block); ok {
return tr, true
}
if tr, ok := fantasy.AsContentType[*fantasy.ToolResultContent](block); ok && tr != nil {
return *tr, true
}
return fantasy.ToolResultContent{}, false
}
func buildAssistantParts(
logger slog.Logger,
assistantBlocks []fantasy.Content,
toolResults []fantasy.ToolResultContent,
step stepData,
toolNameToConfigID map[string]uuid.UUID,
hookRewrittenToolCalls map[string]json.RawMessage,
) []codersdk.ChatMessagePart {
parts := make([]codersdk.ChatMessagePart, 0, len(assistantBlocks)+len(toolResults))
reasoningIdx := 0
for _, block := range assistantBlocks {
part := chatprompt.PartFromContentWithLogger(context.Background(), logger, block)
applyToolMetadata(&part, toolNameToConfigID)
switch part.Type {
case codersdk.ChatMessagePartTypeToolCall:
if part.ToolCallID != "" && step.ToolCallCreatedAt != nil {
if ts, ok := step.ToolCallCreatedAt[part.ToolCallID]; ok {
part.CreatedAt = &ts
}
}
// Hooks never see provider-executed calls, so such a call must not
// inherit attribution from an ordinary call that reused its ID.
if part.ToolCallID != "" && !part.ProviderExecuted {
_, part.HookRewritten = hookRewrittenToolCalls[part.ToolCallID]
}
case codersdk.ChatMessagePartTypeToolResult:
if part.ToolCallID != "" && step.ToolResultCreatedAt != nil {
if ts, ok := step.ToolResultCreatedAt[part.ToolCallID]; ok {
part.CreatedAt = &ts
}
}
case codersdk.ChatMessagePartTypeReasoning:
if reasoningIdx < len(step.ReasoningStartedAt) {
if ts := step.ReasoningStartedAt[reasoningIdx]; !ts.IsZero() {
part.CreatedAt = &ts
}
}
if reasoningIdx < len(step.ReasoningCompletedAt) {
if ts := step.ReasoningCompletedAt[reasoningIdx]; !ts.IsZero() {
part.CompletedAt = &ts
}
}
reasoningIdx++
}
if part.Type != "" {
parts = append(parts, part)
}
}
for _, tr := range toolResults {
attachments, err := chattool.AttachmentsFromMetadata(tr.ClientMetadata)
if err != nil {
logger.Warn(context.Background(), "skipping malformed tool attachment metadata",
slog.F("tool_name", tr.ToolName),
slog.F("tool_call_id", tr.ToolCallID),
slog.Error(err),
)
continue
}
for _, attachment := range attachments {
parts = append(parts, codersdk.ChatMessageFile(attachment.FileID, attachment.MediaType, attachment.Name))
}
}
return parts
}
func applyToolMetadata(part *codersdk.ChatMessagePart, toolNameToConfigID map[string]uuid.UUID) {
if part.ToolName == "" || len(toolNameToConfigID) == 0 {
return
}
if configID, ok := toolNameToConfigID[part.ToolName]; ok {
part.MCPServerConfigID = uuid.NullUUID{UUID: configID, Valid: true}
}
}
func assistantMessage(
modelConfigID uuid.UUID,
contentVersion int16,
content pqtype.NullRawMessage,
step stepData,
) chatstate.Message {
msg := baseMessage(database.ChatMessageRoleAssistant, database.ChatMessageVisibilityBoth, modelConfigID, contentVersion, content)
if step.Usage != (fantasy.Usage{}) {
msg.InputTokens = nullInt64IfNonZero(step.Usage.InputTokens)
msg.OutputTokens = nullInt64IfNonZero(step.Usage.OutputTokens)
msg.TotalTokens = nullInt64IfNonZero(step.Usage.TotalTokens)
msg.ReasoningTokens = nullInt64IfNonZero(step.Usage.ReasoningTokens)
msg.CacheCreationTokens = nullInt64IfNonZero(step.Usage.CacheCreationTokens)
msg.CacheReadTokens = nullInt64IfNonZero(step.Usage.CacheReadTokens)
}
msg.ContextLimit = step.ContextLimit
// InsertChatMessages maps a zero runtime to NULL, so a model
// invocation shorter than a millisecond persists the same way an
// unmeasured one does.
msg.RuntimeMs = nullInt64IfNonZero(step.Runtime.Milliseconds())
msg.ProviderResponseID = sql.NullString{String: step.ProviderResponseID, Valid: step.ProviderResponseID != ""}
return msg
}
func baseMessage(
role database.ChatMessageRole,
visibility database.ChatMessageVisibility,
modelConfigID uuid.UUID,
contentVersion int16,
content pqtype.NullRawMessage,
) chatstate.Message {
return chatstate.Message{
Role: role,
Content: content,
Visibility: visibility,
ModelConfigID: uuid.NullUUID{UUID: modelConfigID, Valid: modelConfigID != uuid.Nil},
ContentVersion: contentVersion,
}
}
func nullInt64IfNonZero(value int64) sql.NullInt64 {
if value == 0 {
return sql.NullInt64{}
}
return sql.NullInt64{Int64: value, Valid: true}
}
// toolBatchUsagePartType marks the dedicated billing record for a local
// tool batch. Internal to chatd: the row is persisted with model
// visibility so it never reaches the API or SSE, and prompt replay drops
// it because the part converts to no provider content.
const toolBatchUsagePartType codersdk.ChatMessagePartType = "tool-batch-usage"
// toolBatchUsagePayload is the audit payload stored on the usage record.
// It duplicates the row's runtime_ms so the billed window survives in
// content for debugging, alongside how many call intervals produced it.
type toolBatchUsagePayload struct {
BilledMs int64 `json:"billed_ms"`
BilledCalls int `json:"billed_calls"`
}
// batchUsageMessage builds the single model-invisible row that carries a
// local tool batch's billed runtime. Usage sums runtime_ms across rows,
// so a dedicated record keeps real tool results free of batch-level
// runtime. Completed and interrupted batches share this helper. Returns
// false when the batch bills no whole millisecond.
func batchUsageMessage(
modelConfigID uuid.UUID,
contentVersion int16,
runtime time.Duration,
billedCalls int,
) (chatstate.Message, bool, error) {
runtimeMs := runtime.Milliseconds()
if runtimeMs <= 0 {
return chatstate.Message{}, false, nil
}
payload, err := json.Marshal(toolBatchUsagePayload{
BilledMs: runtimeMs,
BilledCalls: billedCalls,
})
if err != nil {
return chatstate.Message{}, false, xerrors.Errorf("marshal tool batch usage payload: %w", err)
}
content, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{{
Type: toolBatchUsagePartType,
Result: payload,
}})
if err != nil {
return chatstate.Message{}, false, xerrors.Errorf("marshal tool batch usage part: %w", err)
}
msg := baseMessage(database.ChatMessageRoleTool, database.ChatMessageVisibilityModel, modelConfigID, contentVersion, content)
msg.RuntimeMs = sql.NullInt64{Int64: runtimeMs, Valid: true}
return msg, true, nil
}
func textFromParts(parts []codersdk.ChatMessagePart) string {
var builder strings.Builder
for _, part := range parts {
if part.Type == codersdk.ChatMessagePartTypeText {
_, _ = builder.WriteString(part.Text)
}
}
return builder.String()
}
type buildCompactionMessagesInput struct {
modelConfigID uuid.UUID
toolCallID string
toolName string
compaction compactionOutcome
contentVersion int16
pendingUserMessages []database.ChatMessage
}
type compactionMessagesForCommit struct {
Messages []chatstate.Message
}
func buildCompactionMessages(input buildCompactionMessagesInput) (compactionMessagesForCommit, error) {
contentVersion := input.contentVersion
if contentVersion == 0 {
contentVersion = chatprompt.CurrentContentVersion
}
toolName := input.toolName
if toolName == "" {
toolName = "chat_summarized"
}
systemContent, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{codersdk.ChatMessageText(input.compaction.SystemSummary)})
if err != nil {
return compactionMessagesForCommit{}, xerrors.Errorf("marshal compaction system summary: %w", err)
}
source := input.compaction.Source
if source == "" {
source = chatloop.CompactionSourceAutomatic
}
args, err := json.Marshal(map[string]any{
"source": source,
"threshold_percent": input.compaction.ThresholdPercent,
})
if err != nil {
return compactionMessagesForCommit{}, xerrors.Errorf("marshal compaction args: %w", err)
}
assistantContent, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
codersdk.ChatMessageToolCall(input.toolCallID, toolName, args),
})
if err != nil {
return compactionMessagesForCommit{}, xerrors.Errorf("marshal compaction tool call: %w", err)
}
summaryResult, err := json.Marshal(chatloop.CompactionToolResult{
Summary: input.compaction.SummaryReport,
Source: source,
ThresholdPercent: input.compaction.ThresholdPercent,
UsagePercent: input.compaction.UsagePercent,
ContextTokens: input.compaction.ContextTokens,
ContextLimitTokens: input.compaction.ChatContextLimit,
EstimatedContextTokens: input.compaction.EstimatedContextTokens,
TriggerContextLimitTokens: input.compaction.TriggerContextLimit,
})
if err != nil {
return compactionMessagesForCommit{}, xerrors.Errorf("marshal compaction result: %w", err)
}
toolContent, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
codersdk.ChatMessageToolResult(input.toolCallID, toolName, summaryResult, false, false),
})
if err != nil {
return compactionMessagesForCommit{}, xerrors.Errorf("marshal compaction tool result: %w", err)
}
assistantMsg := baseMessage(database.ChatMessageRoleAssistant, database.ChatMessageVisibilityUser, input.modelConfigID, contentVersion, assistantContent)
assistantMsg.RuntimeMs = nullInt64IfNonZero(input.compaction.Runtime.Milliseconds())
assistantMsg.ProviderResponseID = sql.NullString{String: input.compaction.ProviderResponseID, Valid: input.compaction.ProviderResponseID != ""}
messages := []chatstate.Message{
{
Role: database.ChatMessageRoleUser,
Content: systemContent,
Visibility: database.ChatMessageVisibilityModel,
ModelConfigID: uuid.NullUUID{UUID: input.modelConfigID, Valid: input.modelConfigID != uuid.Nil},
ContentVersion: contentVersion,
},
assistantMsg,
baseMessage(database.ChatMessageRoleTool, database.ChatMessageVisibilityBoth, input.modelConfigID, contentVersion, toolContent),
}
for i := range messages {
messages[i].Compressed = true
}
for _, row := range input.pendingUserMessages {
messages = append(messages, chatstate.Message{
Role: database.ChatMessageRoleUser,
Content: row.Content,
Visibility: database.ChatMessageVisibilityModel,
ModelConfigID: uuid.NullUUID{UUID: input.modelConfigID, Valid: input.modelConfigID != uuid.Nil},
ContentVersion: row.ContentVersion,
})
}
return compactionMessagesForCommit{Messages: messages}, nil
}
type buildClearMessagesInput struct {
modelConfigID uuid.UUID
toolCallID string
contentVersion int16
}
// buildClearMessages produces the manual context-clear boundary
// triplet, mirroring the compaction triplet shape: a hidden
// model-only user-role row (the boundary anchor the prompt query keys
// on), a user-visible synthetic chat_cleared tool call, and its tool
// result. The hidden row carries a short sentinel rather than empty
// content so the next prompt never sends an empty user message.
func buildClearMessages(input buildClearMessagesInput) ([]chatstate.Message, error) {
contentVersion := input.contentVersion
if contentVersion == 0 {
contentVersion = chatprompt.CurrentContentVersion
}
sentinelContent, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
codersdk.ChatMessageText("Previous conversation context was cleared by the user."),
})
if err != nil {
return nil, xerrors.Errorf("marshal clear sentinel: %w", err)
}
payload := json.RawMessage(`{"source":"manual"}`)
assistantContent, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
codersdk.ChatMessageToolCall(input.toolCallID, "chat_cleared", payload),
})
if err != nil {
return nil, xerrors.Errorf("marshal clear tool call: %w", err)
}
toolContent, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{
codersdk.ChatMessageToolResult(input.toolCallID, "chat_cleared", payload, false, false),
})
if err != nil {
return nil, xerrors.Errorf("marshal clear tool result: %w", err)
}
messages := []chatstate.Message{
{
Role: database.ChatMessageRoleUser,
Content: sentinelContent,
Visibility: database.ChatMessageVisibilityModel,
ModelConfigID: uuid.NullUUID{UUID: input.modelConfigID, Valid: input.modelConfigID != uuid.Nil},
ContentVersion: contentVersion,
},
baseMessage(database.ChatMessageRoleAssistant, database.ChatMessageVisibilityUser, input.modelConfigID, contentVersion, assistantContent),
baseMessage(database.ChatMessageRoleTool, database.ChatMessageVisibilityBoth, input.modelConfigID, contentVersion, toolContent),
}
for i := range messages {
messages[i].Compressed = true
}
return messages, nil
}
// hasClearableMessageAfter reports whether any active, uncompressed
// model-visible conversation message follows the boundary index.
// System prompts and user-only rows do not make a chat clearable.
func hasClearableMessageAfter(messages []database.ChatMessage, index int) bool {
for i := index + 1; i < len(messages); i++ {
msg := messages[i]
if msg.Deleted || msg.Compressed {
continue
}
if msg.Role == database.ChatMessageRoleSystem {
continue
}
if msg.Visibility == database.ChatMessageVisibilityModel || msg.Visibility == database.ChatMessageVisibilityBoth {
return true
}
}
return false
}
// Hook model-context messages use the user role but must not reset
// per-turn guards.
func lastUserPromptIndex(messages []database.ChatMessage) int {
index := -1
for i, msg := range messages {
if msg.Deleted || msg.Compressed {
continue
}
if msg.Role == database.ChatMessageRoleUser && msg.Visibility != database.ChatMessageVisibilityModel {
index = i
}
}
return index
}
func currentTurnStartIndex(messages []database.ChatMessage) int {
return lastUserPromptIndex(messages) + 1
}
func currentTurnStepCount(messages []database.ChatMessage) int {
count := 0
for _, msg := range messages[currentTurnStartIndex(messages):] {
if msg.Deleted || msg.Compressed {
continue
}
if msg.Role == database.ChatMessageRoleAssistant {
count++
}
}
return count
}
type compactionRequirement int
const (
compactionRequirementNotNeeded compactionRequirement = iota
compactionRequirementNeeded
)
func compactionStatusFromHistory(
messages []database.ChatMessage,
requirement compactionRequirement,
thresholdPercent int32,
triggerContextLimit int64,
) compactionStatus {
boundaryIndex := latestContextBoundaryIndex(messages)
if requirement == compactionRequirementNeeded {
if boundaryIndex == -1 {
return compactionStatusNeeded
}
// The first assistant response after the previously compacted summary.
// Messages with role ChatMessageRoleAssistant carry context usage.
// Looking at ChatMessageRoleAssistant is enough - ChatMessageRoleTool
// does not carry context usage, and is always preceded by an assistant
// message.
if assistant, ok := firstUncompressedAssistantAfter(messages, boundaryIndex); ok &&
postCompactionAssistantOverLimit(assistant, thresholdPercent, triggerContextLimit) &&
!summaryRecordedUnderOtherTrigger(messages[boundaryIndex], thresholdPercent, triggerContextLimit) {
return compactionStatusStillOverLimit
}
if hasUncompressedMessageAfter(messages, boundaryIndex) {
return compactionStatusNeeded
}
return compactionStatusAfterCompaction
}
if boundaryIndex != -1 && !hasUncompressedMessageAfter(messages, boundaryIndex) {
return compactionStatusAfterCompaction
}
return compactionStatusNotNeeded
}
// summaryRecordedUnderOtherTrigger reports whether a chat_summarized
// boundary recorded a trigger other than the current one. A summary made
// under an older trigger is not a failed compaction under the current one.
func summaryRecordedUnderOtherTrigger(boundary database.ChatMessage, thresholdPercent int32, triggerContextLimit int64) bool {
parts, err := chatprompt.ParseContent(boundary)
if err != nil {
return false
}
for _, part := range parts {
if part.ToolName != "chat_summarized" || part.Type != codersdk.ChatMessagePartTypeToolResult {
continue
}
// Results written before trigger_context_limit_tokens stored the
// trigger's window in context_limit_tokens. A result without these
// keys compares equal to the current trigger.
const noTriggerContextLimit = -1
recorded := chatloop.CompactionToolResult{
ThresholdPercent: thresholdPercent,
ContextLimitTokens: triggerContextLimit,
TriggerContextLimitTokens: noTriggerContextLimit,
}
if err := json.Unmarshal(part.Result, &recorded); err != nil {
return false
}
recordedLimit := recorded.TriggerContextLimitTokens
if recordedLimit == noTriggerContextLimit {
recordedLimit = recorded.ContextLimitTokens
}
return recorded.ThresholdPercent != thresholdPercent || recordedLimit != triggerContextLimit
}
return false
}
// latestContextBoundaryIndex finds the latest compressed
// chat_summarized or chat_cleared boundary. Compaction and clear
// eligibility both stop here so neither reaches across the other.
func latestContextBoundaryIndex(messages []database.ChatMessage) int {
for i := len(messages) - 1; i >= 0; i-- {
if isContextBoundaryMessage(messages[i]) {
return i
}
}
return -1
}
func isContextBoundaryMessage(msg database.ChatMessage) bool {
if msg.Deleted || !msg.Compressed {
return false
}
parts, err := chatprompt.ParseContent(msg)
if err != nil {
return false
}
for _, part := range parts {
if (part.ToolName == "chat_summarized" || part.ToolName == "chat_cleared") &&
(part.Type == codersdk.ChatMessagePartTypeToolCall || part.Type == codersdk.ChatMessagePartTypeToolResult) {
return true
}
}
return false
}
// pendingUserSegmentStart returns the index of the first row of the trailing run of unanswered user-role rows (len(promptRows) when there is none or when no assistant row precedes it); scanning persisted rows means assistant rows terminate the run even when prompt conversion or sanitization drops them.
func pendingUserSegmentStart(promptRows []database.ChatMessage) int {
start := len(promptRows)
for start > 0 {
row := promptRows[start-1]
if row.Deleted || row.Compressed || row.Role != database.ChatMessageRoleUser {
break
}
start--
}
for _, row := range promptRows[:start] {
if !row.Deleted && row.Role == database.ChatMessageRoleAssistant {
return start
}
}
return len(promptRows)
}
func firstUncompressedAssistantAfter(messages []database.ChatMessage, index int) (database.ChatMessage, bool) {
for i := index + 1; i < len(messages); i++ {
msg := messages[i]
if msg.Deleted || msg.Compressed {
continue
}
if msg.Role == database.ChatMessageRoleAssistant {
return msg, true
}
}
return database.ChatMessage{}, false
}
func hasUncompressedMessageAfter(messages []database.ChatMessage, index int) bool {
for i := index + 1; i < len(messages); i++ {
msg := messages[i]
if !msg.Deleted && !msg.Compressed {
return true
}
}
return false
}
// postCompactionOverLimit reports whether the first assistant usage after the
// latest boundary reaches trigger's point. A disabled trigger never does.
func postCompactionOverLimit(messages []database.ChatMessage, trigger compactionTrigger) bool {
assistant, ok := firstUncompressedAssistantAfter(messages, latestContextBoundaryIndex(messages))
return ok && postCompactionAssistantOverLimit(assistant, trigger.thresholdPercent, trigger.contextLimit)
}
func postCompactionAssistantOverLimit(msg database.ChatMessage, thresholdPercent int32, triggerContextLimit int64) bool {
// Zero means "always compact." Excluding it from this check prevents a
// terminal over-limit error after every compaction.
if thresholdPercent <= 0 {
return false
}
return shouldCompactPromptUsage(usageFromMessage(msg), triggerContextLimit, thresholdPercent)
}
func usageFromMessage(msg database.ChatMessage) fantasy.Usage {
var usage fantasy.Usage
if msg.InputTokens.Valid {
usage.InputTokens = msg.InputTokens.Int64
}
if msg.OutputTokens.Valid {
usage.OutputTokens = msg.OutputTokens.Int64
}
if msg.TotalTokens.Valid {
usage.TotalTokens = msg.TotalTokens.Int64
}
if msg.ReasoningTokens.Valid {
usage.ReasoningTokens = msg.ReasoningTokens.Int64
}
if msg.CacheCreationTokens.Valid {
usage.CacheCreationTokens = msg.CacheCreationTokens.Int64
}
if msg.CacheReadTokens.Valid {
usage.CacheReadTokens = msg.CacheReadTokens.Int64
}
return usage
}
func historyHasStopAfterToolResult(messages []database.ChatMessage, stopAfterTools map[string]struct{}) (bool, error) {
if len(stopAfterTools) == 0 {
return false, nil
}
for _, msg := range messages[currentTurnStartIndex(messages):] {
if msg.Deleted || msg.Compressed || msg.Role != database.ChatMessageRoleTool {
continue
}
parts, err := chatprompt.ParseContent(msg)
if err != nil {
return false, xerrors.Errorf("parse tool message: %w", err)
}
for _, part := range parts {
if part.Type != codersdk.ChatMessagePartTypeToolResult || part.IsError {
continue
}
if _, ok := stopAfterTools[part.ToolName]; ok {
return true, nil
}
}
}
return false, nil
}
func currentHistoryComplete(messages []database.ChatMessage) (bool, error) {
idx := lastMessageIndex(messages, func(database.ChatMessage) bool { return true })
if idx == -1 || messages[idx].Role != database.ChatMessageRoleAssistant {
return false, nil
}
parts, err := chatprompt.ParseContent(messages[idx])
if err != nil {
return false, xerrors.Errorf("parse latest assistant message: %w", err)
}
for _, part := range parts {
if part.Type == codersdk.ChatMessagePartTypeToolCall && !part.ProviderExecuted {
return false, nil
}
}
return true, nil
}
func lastMessageIndex(messages []database.ChatMessage, accept func(database.ChatMessage) bool) int {
for i := len(messages) - 1; i >= 0; i-- {
if messages[i].Deleted || messages[i].Compressed {
continue
}
if accept(messages[i]) {
return i
}
}
return -1
}
func handledToolCallIDs(messages []database.ChatMessage) (map[string]bool, error) {
handled := make(map[string]bool)
for _, msg := range messages {
if msg.Deleted || msg.Compressed || msg.Role != database.ChatMessageRoleTool {
continue
}
parts, err := chatprompt.ParseContent(msg)
if err != nil {
return nil, xerrors.Errorf("parse tool message: %w", err)
}
for _, part := range parts {
if part.Type == codersdk.ChatMessagePartTypeToolResult && part.ToolCallID != "" {
handled[part.ToolCallID] = true
}
}
}
return handled, nil
}
type bufferedPartsToPartialMessagesInput struct {
parts []messagepartbuffer.Part
modelConfigID uuid.UUID
contentVersion int16
logger slog.Logger
interruptedAt time.Time
// attemptRuntime is the interrupted attempt's billable model
// invocation window: the span from the provider stream opening to
// the interrupt closing its buffer episode. It is persisted as
// runtime_ms on the first partial assistant message when the
// attempt streamed model-generated assistant content.
attemptRuntime time.Duration
}
type partialToolCall struct {
part codersdk.ChatMessagePart
index int
argsDelta strings.Builder
valid bool
durable bool
}
type partialToolResult struct {
part codersdk.ChatMessagePart
// streamed records that live result or reasoning deltas arrived
// without a durable result; they are stream-only and never persisted.
streamed bool
completed bool
}
func bufferedPartsToPartialMessages(input bufferedPartsToPartialMessagesInput) ([]chatstate.Message, error) {
contentVersion := input.contentVersion
if contentVersion == 0 {
contentVersion = chatprompt.CurrentContentVersion
}
parts := slices.Clone(input.parts)
slices.SortFunc(parts, func(a, b messagepartbuffer.Part) int {
return cmp.Compare(a.Seq, b.Seq)
})
state := partialMessageConversionState{
input: input,
contentVersion: contentVersion,
toolCalls: make(map[string]*partialToolCall),
toolResults: make(map[string]*partialToolResult),
answered: make(map[string]bool),
}
for _, buffered := range parts {
if err := state.consume(buffered); err != nil {
return nil, err
}
}
if err := state.finalizeToolCallPlaceholders(); err != nil {
return nil, err
}
if err := state.flushAssistant(); err != nil {
return nil, err
}
if err := state.flushAccumulatedToolResults(); err != nil {
return nil, err
}
if err := state.appendSyntheticInterruptionResults(); err != nil {
return nil, err
}
if input.attemptRuntime > 0 && state.modelStreamedAssistant {
// Usage reporting sums runtime_ms across rows, so placing the
// whole span on the first assistant message is sufficient.
for i := range state.messages {
if state.messages[i].Role != database.ChatMessageRoleAssistant {
continue
}
state.messages[i].RuntimeMs = nullInt64IfNonZero(input.attemptRuntime.Milliseconds())
break
}
}
return state.messages, nil
}
type partialMessageConversionState struct {
input bufferedPartsToPartialMessagesInput
contentVersion int16
messages []chatstate.Message
assistantParts []codersdk.ChatMessagePart
toolCalls map[string]*partialToolCall
toolCallOrder []string
toolResults map[string]*partialToolResult
toolResultOrder []string
answered map[string]bool
// modelStreamedAssistant distinguishes streamed content from tool
// attachment parts, which must not carry model runtime.
modelStreamedAssistant bool
}
func (s *partialMessageConversionState) consume(buffered messagepartbuffer.Part) error {
switch buffered.Role {
case codersdk.ChatMessageRoleAssistant:
s.consumeAssistantPart(buffered)
case codersdk.ChatMessageRoleTool:
return s.consumeToolPart(buffered)
default:
s.logSkippedPart(buffered, "unsupported buffered part role")
}
return nil
}
func (s *partialMessageConversionState) consumeAssistantPart(buffered messagepartbuffer.Part) {
part := buffered.MessagePart
if part.Type == "" {
s.logSkippedPart(buffered, "empty buffered assistant part type")
return
}
if part.Type != codersdk.ChatMessagePartTypeFile {
s.modelStreamedAssistant = true
}
if part.Type != codersdk.ChatMessagePartTypeToolCall {
if part.Type == codersdk.ChatMessagePartTypeReasoning &&
!s.input.interruptedAt.IsZero() {
interruptedAt := s.input.interruptedAt
if part.CreatedAt == nil {
part.CreatedAt = &interruptedAt
}
if part.CompletedAt == nil {
part.CompletedAt = &interruptedAt
}
}
s.assistantParts = append(s.assistantParts, part)
return
}
if part.ToolCallID == "" {
s.logSkippedPart(buffered, "tool call part missing tool call ID")
return
}
call := s.toolCall(part.ToolCallID)
call.part.Type = codersdk.ChatMessagePartTypeToolCall
call.part.ToolCallID = part.ToolCallID
if part.ToolName != "" {
call.part.ToolName = part.ToolName
}
if part.MCPServerConfigID.Valid {
call.part.MCPServerConfigID = part.MCPServerConfigID
}
if part.CreatedAt != nil {
call.part.CreatedAt = part.CreatedAt
}
call.part.ProviderExecuted = call.part.ProviderExecuted || part.ProviderExecuted
if part.ArgsDelta != "" {
if call.durable {
s.logSkippedPart(buffered, "tool call args delta arrived after full tool call")
return
}
_, _ = call.argsDelta.WriteString(part.ArgsDelta)
return
}
durable := part
durable.ArgsDelta = ""
if len(durable.Args) > 0 && !json.Valid(durable.Args) {
call.valid = false
s.assistantParts[call.index] = codersdk.ChatMessagePart{}
s.logSkippedPart(buffered, "tool call part has invalid durable args")
return
}
if call.durable {
s.logSkippedPart(buffered, "duplicate durable tool call part")
}
call.part = durable
call.valid = true
call.durable = true
s.assistantParts[call.index] = durable
}
func (s *partialMessageConversionState) consumeToolPart(buffered messagepartbuffer.Part) error {
part := buffered.MessagePart
if part.Type != codersdk.ChatMessagePartTypeToolResult {
s.logSkippedPart(buffered, "non tool-result part with tool role")
return nil
}
if part.ToolCallID == "" {
s.logSkippedPart(buffered, "tool result part missing tool call ID")
return nil
}
if part.ResultReset {
result := s.toolResult(part.ToolCallID)
result.part.ToolCallID = part.ToolCallID
result.part.ToolName = part.ToolName
result.streamed = false
s.logSkippedPart(buffered, "streaming tool result reset is not durable")
return nil
}
if part.ResultDelta != "" || part.ReasoningDelta != "" {
result := s.toolResult(part.ToolCallID)
result.part.ToolCallID = part.ToolCallID
if part.ToolName != "" {
result.part.ToolName = part.ToolName
}
if part.MCPServerConfigID.Valid {
result.part.MCPServerConfigID = part.MCPServerConfigID
}
if part.CreatedAt != nil {
result.part.CreatedAt = part.CreatedAt
}
result.part.ProviderExecuted = result.part.ProviderExecuted || part.ProviderExecuted
result.streamed = true
return nil
}
if err := s.finalizeToolCallPlaceholders(); err != nil {
return err
}
if !s.toolCallDurable(part.ToolCallID) {
s.logSkippedPart(buffered, "tool result has no matching durable tool call")
return nil
}
if len(part.Result) == 0 || !json.Valid(part.Result) {
s.logSkippedPart(buffered, "tool result part has invalid durable result")
return nil
}
if s.answered[part.ToolCallID] {
s.logSkippedPart(buffered, "duplicate durable tool result part")
return nil
}
part.ResultDelta = ""
part.ReasoningDelta = ""
part.ResultReset = false
if err := s.flushAssistant(); err != nil {
return err
}
if err := s.appendToolResult(part); err != nil {
return err
}
s.answered[part.ToolCallID] = true
return nil
}
func (s *partialMessageConversionState) toolCall(id string) *partialToolCall {
call := s.toolCalls[id]
if call != nil {
return call
}
call = &partialToolCall{index: len(s.assistantParts), valid: true}
s.toolCalls[id] = call
s.toolCallOrder = append(s.toolCallOrder, id)
s.assistantParts = append(s.assistantParts, codersdk.ChatMessagePart{})
return call
}
func (s *partialMessageConversionState) toolResult(id string) *partialToolResult {
result := s.toolResults[id]
if result != nil {
return result
}
result = &partialToolResult{}
s.toolResults[id] = result
s.toolResultOrder = append(s.toolResultOrder, id)
return result
}
func (s *partialMessageConversionState) finalizeToolCallPlaceholders() error {
for _, id := range s.toolCallOrder {
call := s.toolCalls[id]
if call == nil || call.durable || !call.valid {
continue
}
args := json.RawMessage(call.argsDelta.String())
if len(args) == 0 || !json.Valid(args) {
s.assistantParts[call.index] = codersdk.ChatMessagePart{}
call.valid = false
s.logSkippedPart(messagepartbuffer.Part{
Role: codersdk.ChatMessageRoleAssistant,
MessagePart: call.part,
}, "tool call args delta did not form durable JSON")
continue
}
call.part.Args = args
call.part.ArgsDelta = ""
call.durable = true
s.assistantParts[call.index] = call.part
}
return nil
}
func (s *partialMessageConversionState) flushAssistant() error {
if len(s.assistantParts) == 0 {
return nil
}
durable := make([]codersdk.ChatMessagePart, 0, len(s.assistantParts))
for _, part := range s.assistantParts {
if part.Type == "" {
continue
}
part.ArgsDelta = ""
part.ResultDelta = ""
part.ReasoningDelta = ""
part.ResultReset = false
durable = append(durable, part)
}
s.assistantParts = nil
if len(durable) == 0 {
return nil
}
content, err := chatprompt.MarshalParts(durable)
if err != nil {
return xerrors.Errorf("marshal partial assistant: %w", err)
}
s.messages = append(s.messages, baseMessage(database.ChatMessageRoleAssistant, database.ChatMessageVisibilityBoth, s.input.modelConfigID, s.contentVersion, content))
return nil
}
func (s *partialMessageConversionState) flushAccumulatedToolResults() error {
for _, id := range s.toolResultOrder {
if s.answered[id] {
continue
}
result := s.toolResults[id]
if result == nil || result.completed || !result.streamed {
continue
}
s.logSkippedPart(messagepartbuffer.Part{Role: codersdk.ChatMessageRoleTool, MessagePart: result.part}, "streaming tool result delta is not durable")
}
return nil
}
func (s *partialMessageConversionState) appendToolResult(part codersdk.ChatMessagePart) error {
content, err := chatprompt.MarshalParts([]codersdk.ChatMessagePart{part})
if err != nil {
return xerrors.Errorf("marshal partial tool result: %w", err)
}
s.messages = append(s.messages, baseMessage(database.ChatMessageRoleTool, database.ChatMessageVisibilityBoth, s.input.modelConfigID, s.contentVersion, content))
return nil
}
func (s *partialMessageConversionState) appendSyntheticInterruptionResults() error {
for _, id := range s.toolCallOrder {
if s.answered[id] {
continue
}
call := s.toolCalls[id]
if call == nil || !call.valid || !call.durable || call.part.ProviderExecuted {
continue
}
result, err := json.Marshal(map[string]string{"error": interruptedToolResultErrorMessage})
if err != nil {
return xerrors.Errorf("marshal synthetic interruption result: %w", err)
}
part := codersdk.ChatMessageToolResult(call.part.ToolCallID, call.part.ToolName, result, true, false)
part.MCPServerConfigID = call.part.MCPServerConfigID
if !s.input.interruptedAt.IsZero() {
part.CreatedAt = &s.input.interruptedAt
}
if err := s.appendToolResult(part); err != nil {
return xerrors.Errorf("marshal synthetic interruption message: %w", err)
}
s.answered[id] = true
}
return nil
}
func (s *partialMessageConversionState) toolCallDurable(id string) bool {
call := s.toolCalls[id]
return call != nil && call.valid && call.durable
}
func (s *partialMessageConversionState) logSkippedPart(buffered messagepartbuffer.Part, reason string) {
s.input.logger.Warn(context.Background(), "skipping buffered chat message part",
slog.F("reason", reason),
slog.F("role", buffered.Role),
slog.F("part_type", buffered.MessagePart.Type),
slog.F("tool_call_id", buffered.MessagePart.ToolCallID),
slog.F("tool_name", buffered.MessagePart.ToolName),
)
}