Files
coder/coderd/x/chatd/chatloop/tooltruncate.go
T
Mathias Fredriksson fa5bac6e77 feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt
recorded the same generic result whether a call had finished, was
still running, or never started.

chatd now derives a tool call ID from the chat, the committed
assistant message, and the provider tool call ID, and sends it with
execute starts, edit_files, and write_file. Every replica and retry
computes the same ID, so the agent returns the first result instead
of running the tool again, and a retried foreground execute waits
only for the rest of its timeout.

On interrupt, chatd cancels the unresolved calls of those tools on
the agent before committing, within 30 seconds, and records what
happened: "tool call was canceled before it ran; no changes were
made", the output of a command canceled while running, or the real
result. The UI shows a canceled command with the existing killed
marker. If the cancel fails, including an old agent's 404, the result
stays the generic interrupted text.

Calls whose provider tool call ID is empty or repeats in the message
get no ID and behave as before.

Fixes CODAGT-757
2026-09-30 14:31:52 +03:00

108 lines
4.2 KiB
Go

package chatloop
import (
"fmt"
"strings"
)
const (
// toolResultContextDivisor bounds how much of a model's context
// window a single tool result may occupy: at most 1/3 of the
// window. This caps a single oversized result (most often a large
// MCP response) so it cannot overflow the prompt on its own, while
// still letting a useful amount of output through. The divisor is
// deliberately larger than 2 so the absolute cap stays sane on
// large-context (e.g. 1M-token) models, where a more generous
// fraction would still admit multi-megabyte results. Cumulative
// growth across many results is handled separately by context
// compaction.
toolResultContextDivisor = 3
// bytesPerTokenEstimate converts a token budget into a byte budget.
// Tool output is capped before tokenization, so this is a coarse,
// provider-agnostic estimate. It is deliberately conservative at ~3
// bytes per token: dense payloads (JSON, logs, code, non-ASCII) run
// well below 4 bytes per token, so the lower estimate yields a
// smaller byte budget that is less likely to underestimate the true
// token cost.
bytesPerTokenEstimate = 3
// minToolResultBytes is the floor for the per-result byte budget so
// small or unknown context windows still let useful output through.
minToolResultBytes = 16 << 10 // 16KB
// defaultToolResultBytes is the budget used when the model's context
// window is unknown (context limit <= 0).
defaultToolResultBytes = 64 << 10 // 64KB
// truncationMarkerReserve is the number of bytes reserved for the
// truncation marker when splitting output into a head and tail. It
// is comfortably larger than the longest marker
// TruncateToolResultText can produce, so the assembled result never
// exceeds the budget.
truncationMarkerReserve = 256
)
// maxIntValue is the largest value of the platform int type.
const maxIntValue = int(^uint(0) >> 1)
// ToolResultByteBudget converts a model context-window size (in
// tokens) into the maximum number of bytes a single tool result may
// contribute to the prompt. A context limit <= 0 means the window is
// unknown and the default budget is used. The result is never below
// minToolResultBytes.
func ToolResultByteBudget(contextLimitTokens int64) int {
if contextLimitTokens <= 0 {
return defaultToolResultBytes
}
budgetBytes := contextLimitTokens / toolResultContextDivisor * bytesPerTokenEstimate
if budgetBytes < minToolResultBytes {
return minToolResultBytes
}
// Clamp to the int range for 32-bit safety on very large windows.
if budgetBytes > int64(maxIntValue) {
return maxIntValue
}
return int(budgetBytes)
}
// TruncateToolResultText caps text to at most maxBytes using a
// head-and-tail strategy: it keeps the start and end of the output and
// replaces the middle with a marker noting how many bytes were
// removed. This preserves the most useful context (a tool's leading
// summary and trailing status) while bounding size. The returned
// string is always valid UTF-8 and never exceeds maxBytes. It returns
// (text, false) unchanged when maxBytes <= 0 or the text already fits.
func TruncateToolResultText(text string, maxBytes int) (string, bool) {
if maxBytes <= 0 || len(text) <= maxBytes {
return text, false
}
// When the budget is too small to fit the marker plus a meaningful
// head and tail, hard-cut the head and drop any partial trailing
// rune.
if maxBytes <= truncationMarkerReserve*2 {
return strings.ToValidUTF8(text[:maxBytes], ""), true
}
avail := maxBytes - truncationMarkerReserve
head := avail * 2 / 3
tail := avail - head
// ToValidUTF8 with an empty replacement drops invalid byte runs, so
// a cut that lands in the middle of a multi-byte rune is discarded
// rather than corrupting the output. This can only shrink the
// slices, so the assembled result stays within budget.
headStr := strings.ToValidUTF8(text[:head], "")
tailStr := strings.ToValidUTF8(text[len(text)-tail:], "")
removed := len(text) - len(headStr) - len(tailStr)
marker := fmt.Sprintf(
"\n\n[... Coder truncated %d bytes to fit the model context; "+
"narrow the query or read a specific file or range for the "+
"full output ...]\n\n",
removed,
)
return headStr + marker + tailStr, true
}