Removes production code in `coderd/x/chatd` that nothing reaches, the tests that only existed to exercise it, and option/config fields that callers filled in but chatd never read. No runtime behaviour changes; the only edits outside `coderd/x/chatd` drop the now-nonexistent fields at their three call sites (`coderd/coderd.go`, `coderd/exp_chats.go`, `coderd/coderdtest`). Net: 42 files, +147 / -1065. Verification method for every item: `golang.org/x/tools/cmd/deadcode` (with and without `-test`) rooted at `cmd/coder`, `enterprise/cmd/coder` and every package that imports chatd; a `go/packages` pass counting `TypesInfo.Uses` per declared type, const, var, field and method across the same roots, split into test and non-test use sites, plus a read-vs-write split for struct fields; `rg` over the whole repo including `site/src` for json tag names; then `go build ./...`, `go vet ./coderd/x/chatd/...`, `golangci-lint run ./coderd/x/chatd/... ./coderd/ ./coderd/coderdtest/`, and `go test ./coderd/x/chatd/...` (all 20 packages) locally. <details> <summary>Evidence per deletion</summary> | Removed | What it was | Why unused | |---|---|---| | `subagent.go` `resolveConfiguredModelOverride`, `parseModelOverride`, `parsedModelOverride` (~135 lines) + `TestResolveConfiguredModelOverride_*`, `TestParseModelOverride` | raw-string model override resolver | Only callers were its two tests; production resolves overrides through `resolveOrganizationModelOverride` (`title_override.go`, `compaction_override.go`, `subagent.go`). | | `chatd.go` `removeSkillIndexMessages`, `isSkillIndexMessage` + `TestSkillIndexRefreshReplacesStaleAliases` | skill-index refresh helpers | Zero production references; the test simulated a refresh path no production code performs. | | `chatd.go` `chatMessage`, `newChatMessage`, `withCreatedBy`, `appendMessageFields`, `BuildSingleChatMessageInsertParams` (~90 lines) | insert-params builder | Exported, but every caller was a `chatd_test` file (6 sites). Replaced by a 25-line `singleChatMessageInsertParams` helper in `chatd_helpers_test.go`. | | `chatd.go` `Server.closeInflightAdmission` | one-line wrapper around `inflightClosed.Store(true)` | Single call site (`Close`); inlined. | | `chatd.go` `Server.workerID`, `instructionLookupTimeout`, `pendingChatAcquireInterval`, `maxChatsPerAcquire`, `chatHeartbeatInterval` | Server struct fields | Assigned in `New` and never read anywhere (the values reach the worker through `chatWorkerOptions`, not these fields). | | `Config.InstructionLookupTimeout`, `homeInstructionLookupTimeout`, `coderd.Options.ChatdInstructionLookupTimeout`, `coderdtest.Options.ChatdInstructionLookupTimeout` | timeout plumbing | The value only landed in the dead `Server.instructionLookupTimeout`; no CLI flag sets it and nothing reads it. | | `PromoteQueuedOptions.CreatedBy` | option field | Set by `coderd/exp_chats.go` and three tests, never read in chatd (promotion keeps the queued row's `created_by`). | | `SubmitToolResultsOptions.DynamicTools` | option field | Set by `coderd/exp_chats.go` (which re-read `chat.DynamicTools` just to pass it) and three tests, never read in chatd. | | `attachments.go` `buildAssistantPartsForPersist` (whole file) | assistant part builder | Stale copy left by #26270; production uses `buildAssistantParts` in `message_conversion.go`. Its six tests were retargeted to the live function (now `message_conversion_internal_test.go`) and a dead-copy assertion in `subagent_internal_test.go` was dropped. | | `chatloop.RunOptions` (94 lines) | old loop options struct | Only two of 23 fields were ever populated, for one internal call to `prepareMessagesForRequest`, which now takes `model, logger` directly and no longer returns the ignored canonical slice. `GenerateAssistantOptions` / `ExecuteLocalToolsOptions` are the live types. | | `chatloop.ErrDynamicToolCall`, `ErrStopAfterTool`, `PendingToolCall`, `PersistedStep.PendingDynamicToolCalls` | dynamic-tool exit protocol | Zero references outside their declarations (one stale comment in `chatdebug/service.go` updated). | | `chatloop.CompactionOptions.Persist`, `.OnError` | callbacks | `Persist` had zero references; `OnError` was set only by two tests that asserted it was never invoked. | | `chatloop` `stepResult.shouldContinue` + its computation | flag | Written once, never read. | | `chatloop.StateWaiting` | metric label const | Zero references. | | `generation.go` `generationOutcome.InsertedMessages`, `runnerActionMessage` | outcome field | Built on every commit, never read. | | `chatstate` `ExecutionState.IsArchived`, `AllOwnershipStates`, `OwnershipState`/`StateU`/`StateO`, `TransitionAcquire` | state-machine leftovers | Zero references (the ownership consts became dead once `AllOwnershipStates` went). | | `chatstate` `ExecutionState.QueueNonEmpty` | predicate | Only referenced by the test asserting its own table; the sibling `IsRunnable` check in that test is kept. | | `chatdebug.Attempt.RetryClassification`, `.RetryDelayMs` | json fields | Never assigned anywhere, `omitempty`, and `retry_classification`/`retry_delay_ms` do not appear in `site/src` or `typesGenerated.ts`. | | `chatdebug.RunContext.Kind`, `StepContext.HistoryTipMessageID`, `RecorderOptions.Provider`/`.Model`, `stepHandle.opts`/`.usage`/`.err` | context/handle fields | Written (from `chatd.go`, `chatloop/compaction.go`, `modelcall.go`, `recorder.go`) and never read. The DB run row keeps its own `kind` column. | | `chatprovider.MergeProviderAPIKeys` + `TestMergeProviderAPIKeys_PreservesProviderRegions` | key merge | Superseded by `ResolveUserProviderKeys`, whose region test already covers the same behaviour. | | `chatprovider` `providerCreationError` | one-line `xerrors.Errorf` wrapper | Single call site; inlined with identical message. | | `chatretry.IsRetryable` | classifier wrapper | Zero references. | | `computeruse.IsTool` + `TestComputerUseTool` | one-line forwarder to `fantasyopenai.IsComputerUseTool` | No production caller; two test assertions now call the fantasy function directly, and the dropped test only asserted the library's own constructor. | | `chatprompt.ToolResultToPart` | one-line forwarder to `codersdk.ChatMessageToolResult` | Single call site; inlined. | | `subagent_catalog.go` `formatSubagentDefinitions` | forwarder with `nil` overrides | Single call site; inlined. | | `worker.go` `chatWorker.chatWorkerID` | getter | Only tests called it; they read `opts.WorkerID` directly now. | | `chattest.AnthropicErrorResponse`, `OpenAIServerErrorResponse`, `ChatMessageWithParts` (`messages.go` removed) | test fixtures | Zero references. | | `chattest` `AnthropicRequest.Options`, `OpenAIRequest.Options` | `json:",inline"` map fields | `encoding/json` has no inline tag (the TODO on each said so); never populated, never read. | </details> <details> <summary>Considered and kept</summary> - `chatstate.CreateChat` (test-only thin wrapper over `insertChat`, production uses `CreateChatWithID`): documented package API with 25 test call sites; not worth the churn. - `chatprompt.MarshalContent` / `MarshalToolResult`: test-only, but they deliberately produce legacy-format DB rows so the read path's legacy handling stays covered. - `chatstate.AllowedExecutionTransitionsFrom` / `AllowedInputStates` / `AllowedExecutionTransitionOutputs`, `chatdebug.StepFromContext`, `WithClock`, `WithStaleThreshold`, `WaitUntilIdleForTest`, `PublishBuffer.BufferedChannels`, `chattool.NewWorkspaceMCPTool`: test seams that tests actively use to reach otherwise unexported state. - `AgentCapacityLimiter` interface (one production implementation): tests inject fakes. - `applyHookResultMessages` / `appendHookResultMessages`: a named pair over one placement enum, both used. - Nil-receiver guards on `*Metrics`, `*Result`, `*streamSyncPoller`, `*runnerDebugTurn`: nil is a supported "disabled" value on those paths. - Tests flagged by heuristics as assertion-light (`TestNopMetrics_DoesNotPanic`, `TestStepHandleFinish_NilHandle`, `TestRetry_OnRetryNilDoesNotPanic`, the `chatstate` synthetic-cancellation tests that assert through helpers): they cover real nil-safety or delegate assertions to helpers; kept. </details> <details> <summary>Out-of-scope findings (not changed here)</summary> - `chatdebug.Service.FinalizeStale` has had no production caller since #26270 removed the background sweep loop; only its tests call it, and comments in `chatd_debug.go` and `database/querier.go` still describe the loop. Deleting it would erase the implementation a fix needs, so it is left for a decision. - `coderd_chatd_chats` (gauge) and `coderd_chatd_stream_buffer_dropped_total` (counter) are registered and documented in `docs/admin/integrations/prometheus.md` but never set or incremented in production; `Metrics.RecordStreamBufferDropped` is test-only. Removing them changes the exported metric set and the docs, so they are left alone. </details> ## Declined review findings None yet. > Xum (AI agent) opened this PR on behalf of @ibetitsmike.
346 lines
8.9 KiB
Go
346 lines
8.9 KiB
Go
package chatd
|
|
|
|
import (
|
|
"context"
|
|
"database/sql"
|
|
"errors"
|
|
"sync"
|
|
"time"
|
|
|
|
"github.com/google/uuid"
|
|
"golang.org/x/xerrors"
|
|
|
|
"github.com/coder/coder/v2/coderd/database"
|
|
coderdpubsub "github.com/coder/coder/v2/coderd/pubsub"
|
|
"github.com/coder/coder/v2/coderd/x/chatd/chatstate"
|
|
)
|
|
|
|
// chatWorker owns chat acquisition and runner lifecycle for one process.
|
|
type chatWorker struct {
|
|
server *Server
|
|
opts chatWorkerOptions
|
|
|
|
mu sync.Mutex
|
|
started bool
|
|
ctx context.Context
|
|
cancel context.CancelFunc
|
|
manager *runnerManager
|
|
unsubscribe func()
|
|
wakeCh chan struct{}
|
|
wg sync.WaitGroup
|
|
}
|
|
|
|
// newChatWorker constructs a chat worker. The worker is idle until Start is
|
|
// called.
|
|
func newChatWorker(server *Server, opts chatWorkerOptions) (*chatWorker, error) {
|
|
if server == nil {
|
|
return nil, xerrors.New("chatworker: server is required")
|
|
}
|
|
withDefaults, err := opts.withDefaults()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &chatWorker{server: server, opts: withDefaults}, nil
|
|
}
|
|
|
|
// Start starts the acquisition and runner manager loops.
|
|
func (w *chatWorker) Start(ctx context.Context) error {
|
|
w.mu.Lock()
|
|
defer w.mu.Unlock()
|
|
if w.started {
|
|
return xerrors.New("chatworker: worker already started")
|
|
}
|
|
workerID := w.opts.WorkerID
|
|
workerCtx, cancel := context.WithCancel(ctx)
|
|
manager := newRunnerManager(workerCtx, w.server, w.opts)
|
|
if manager.opts.TaskStarter == nil {
|
|
starter, err := newTaskStarter(manager.server, manager.opts, manager.RouteStateHint, manager.requestCleanup)
|
|
if err != nil {
|
|
cancel()
|
|
return err
|
|
}
|
|
manager.opts.TaskStarter = starter
|
|
}
|
|
wakeCh := make(chan struct{}, w.opts.AcquisitionWakeChannelSize)
|
|
|
|
unsubscribe, err := w.opts.Pubsub.SubscribeWithErr(
|
|
coderdpubsub.ChatStateOwnershipChannel,
|
|
coderdpubsub.HandleChatStateOwnership(func(ctx context.Context, _ coderdpubsub.ChatStateOwnershipMessage, err error) {
|
|
if err != nil {
|
|
w.opts.Logger.Warn(ctx, "chatworker ownership hint decode failed", slogError(err))
|
|
return
|
|
}
|
|
wake(wakeCh)
|
|
}),
|
|
)
|
|
if err != nil {
|
|
cancel()
|
|
return xerrors.Errorf("subscribe ownership hints: %w", err)
|
|
}
|
|
|
|
w.started = true
|
|
w.ctx = workerCtx
|
|
w.cancel = cancel
|
|
w.manager = manager
|
|
w.unsubscribe = unsubscribe
|
|
w.wakeCh = wakeCh
|
|
|
|
manager.start()
|
|
w.wg.Go(func() {
|
|
w.acquisitionLoop(workerCtx, workerID, manager, wakeCh)
|
|
})
|
|
w.wg.Go(func() {
|
|
w.archiveLoop(workerCtx)
|
|
})
|
|
if w.opts.CapacityMetrics != nil {
|
|
w.wg.Go(func() {
|
|
w.capacityMetricsLoop(workerCtx)
|
|
})
|
|
}
|
|
wake(wakeCh)
|
|
return nil
|
|
}
|
|
|
|
// Wake requests an immediate acquisition pass.
|
|
func (w *chatWorker) Wake() {
|
|
w.mu.Lock()
|
|
wakeCh := w.wakeCh
|
|
w.mu.Unlock()
|
|
if wakeCh != nil {
|
|
wake(wakeCh)
|
|
}
|
|
}
|
|
|
|
// WaitIdle waits until the worker has no active or cleaning runners.
|
|
func (w *chatWorker) WaitIdle(ctx context.Context) error {
|
|
for {
|
|
w.mu.Lock()
|
|
manager := w.manager
|
|
w.mu.Unlock()
|
|
if manager == nil || manager.idle() {
|
|
return nil
|
|
}
|
|
timer := w.opts.Clock.NewTimer(10*time.Millisecond, "chatworker", "wait-idle")
|
|
select {
|
|
case <-timer.C:
|
|
case <-ctx.Done():
|
|
timer.Stop()
|
|
return ctx.Err()
|
|
}
|
|
timer.Stop()
|
|
}
|
|
}
|
|
|
|
// Close stops the worker and waits for its loops to exit.
|
|
func (w *chatWorker) Close() error {
|
|
w.mu.Lock()
|
|
if !w.started {
|
|
w.mu.Unlock()
|
|
return nil
|
|
}
|
|
cancel := w.cancel
|
|
unsubscribe := w.unsubscribe
|
|
manager := w.manager
|
|
w.started = false
|
|
w.cancel = nil
|
|
w.unsubscribe = nil
|
|
w.manager = nil
|
|
w.wakeCh = nil
|
|
w.mu.Unlock()
|
|
|
|
if unsubscribe != nil {
|
|
unsubscribe()
|
|
}
|
|
cancel()
|
|
w.wg.Wait()
|
|
if manager != nil {
|
|
manager.wait()
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func wake(ch chan<- struct{}) {
|
|
select {
|
|
case ch <- struct{}{}:
|
|
default:
|
|
}
|
|
}
|
|
|
|
func (w *chatWorker) acquisitionLoop(
|
|
ctx context.Context,
|
|
workerID uuid.UUID,
|
|
manager *runnerManager,
|
|
wakeCh <-chan struct{},
|
|
) {
|
|
ticker := w.opts.Clock.NewTicker(w.opts.AcquisitionInterval, "chatworker", "acquisition")
|
|
defer ticker.Stop()
|
|
for {
|
|
select {
|
|
case <-wakeCh:
|
|
w.acquireOnce(ctx, workerID, manager)
|
|
case <-ticker.C:
|
|
w.acquireOnce(ctx, workerID, manager)
|
|
case <-ctx.Done():
|
|
return
|
|
}
|
|
}
|
|
}
|
|
|
|
func (w *chatWorker) acquireOnce(ctx context.Context, workerID uuid.UUID, manager *runnerManager) {
|
|
// Fetch twice the budget so one full pool cannot hide candidates in the other.
|
|
rows, err := w.opts.Store.GetChatWorkerAcquisitionCandidates(ctx, database.GetChatWorkerAcquisitionCandidatesParams{
|
|
StaleSeconds: w.opts.HeartbeatStaleSeconds,
|
|
LimitCount: w.opts.AcquisitionBatchSize * 2,
|
|
})
|
|
if err != nil {
|
|
if ctx.Err() == nil {
|
|
w.opts.Logger.Warn(ctx, "chatworker acquisition query failed", slogError(err))
|
|
}
|
|
return
|
|
}
|
|
|
|
acquired := int32(0)
|
|
rootPoolRefused := false
|
|
subagentPoolRefused := false
|
|
for _, row := range rows {
|
|
if acquired >= w.opts.AcquisitionBatchSize {
|
|
return
|
|
}
|
|
// Interrupting and requires-action chats bypass capacity so their runners
|
|
// can finish work or enforce the action deadline.
|
|
isSubagent := row.ParentChatID.Valid
|
|
if row.Status == database.ChatStatusRunning &&
|
|
((isSubagent && subagentPoolRefused) || (!isSubagent && rootPoolRefused)) {
|
|
continue
|
|
}
|
|
candidateAcquired, err := w.acquireCandidateSafely(ctx, workerID, manager, row.ID)
|
|
if errors.Is(err, errCapacityRefused) {
|
|
if isSubagent {
|
|
subagentPoolRefused = true
|
|
} else {
|
|
rootPoolRefused = true
|
|
}
|
|
continue
|
|
}
|
|
if err != nil {
|
|
if ctx.Err() != nil {
|
|
return
|
|
}
|
|
w.opts.Logger.Warn(ctx, "chatworker acquisition candidate failed", slogError(err))
|
|
continue
|
|
}
|
|
if candidateAcquired {
|
|
acquired++
|
|
}
|
|
}
|
|
}
|
|
|
|
var (
|
|
errSkipAcquire = xerrors.New("skip acquire")
|
|
errCapacityRefused = xerrors.New("capacity refused")
|
|
)
|
|
|
|
func (w *chatWorker) acquireCandidateSafely(
|
|
ctx context.Context,
|
|
workerID uuid.UUID,
|
|
manager *runnerManager,
|
|
chatID uuid.UUID,
|
|
) (acquired bool, err error) {
|
|
defer func() {
|
|
if recovered := recover(); recovered != nil {
|
|
err = xerrors.Errorf("chatworker acquisition panic: %v", recovered)
|
|
}
|
|
}()
|
|
return w.acquireCandidate(ctx, workerID, manager, chatID)
|
|
}
|
|
|
|
func (w *chatWorker) acquireCandidate(
|
|
ctx context.Context,
|
|
workerID uuid.UUID,
|
|
manager *runnerManager,
|
|
chatID uuid.UUID,
|
|
) (bool, error) {
|
|
runnerID := uuid.New()
|
|
machine := chatstate.NewChatMachine(w.opts.Store, w.opts.Pubsub, chatID)
|
|
err := machine.Update(ctx, func(tx *chatstate.Tx, store database.Store) error {
|
|
chat, err := store.GetChatByID(ctx, chatID)
|
|
if errors.Is(err, sql.ErrNoRows) {
|
|
return errSkipAcquire
|
|
}
|
|
if err != nil {
|
|
return xerrors.Errorf("load chat: %w", err)
|
|
}
|
|
queueCount, err := store.CountChatQueuedMessages(ctx, chatID)
|
|
if err != nil {
|
|
return xerrors.Errorf("count queue: %w", err)
|
|
}
|
|
if !chatstate.ClassifyExecutionState(chat, queueCount > 0, true).IsRunnable() || chat.Archived {
|
|
return errSkipAcquire
|
|
}
|
|
if chat.WorkerID.Valid && chat.RunnerID.Valid {
|
|
stale, err := store.IsChatHeartbeatStale(ctx, database.IsChatHeartbeatStaleParams{
|
|
ChatID: chat.ID,
|
|
RunnerID: chat.RunnerID.UUID,
|
|
StaleSeconds: w.opts.HeartbeatStaleSeconds,
|
|
})
|
|
if err != nil {
|
|
return xerrors.Errorf("check heartbeat stale: %w", err)
|
|
}
|
|
if !stale {
|
|
return errSkipAcquire
|
|
}
|
|
}
|
|
admitted, err := w.opts.AgentCapacityLimiter.Admit(ctx, store, chat)
|
|
if err != nil {
|
|
return xerrors.Errorf("agent admission: %w", err)
|
|
}
|
|
if !admitted {
|
|
// Roll back to suppress the ownership hint, which would wake every
|
|
// worker into an immediate retry of this unowned chat.
|
|
return errCapacityRefused
|
|
}
|
|
_, err = tx.Acquire(chatstate.AcquireInput{WorkerID: workerID, RunnerID: runnerID})
|
|
return err
|
|
})
|
|
if errors.Is(err, errCapacityRefused) {
|
|
return false, errCapacityRefused
|
|
}
|
|
if errors.Is(err, errSkipAcquire) || errors.Is(err, chatstate.ErrChatNotFound) {
|
|
return false, nil
|
|
}
|
|
if err != nil {
|
|
return false, err
|
|
}
|
|
if err := manager.Spawn(ctx, spawnRunnerRequest{ChatID: chatID, WorkerID: workerID, RunnerID: runnerID}); err != nil {
|
|
if errAbandon := w.abandonAcquiredChat(ctx, workerID, runnerID, chatID); errAbandon != nil {
|
|
return false, errors.Join(err, errAbandon)
|
|
}
|
|
return false, err
|
|
}
|
|
return true, nil
|
|
}
|
|
|
|
func (w *chatWorker) abandonAcquiredChat(ctx context.Context, workerID uuid.UUID, runnerID uuid.UUID, chatID uuid.UUID) error {
|
|
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), shutdownCleanupTimeout)
|
|
defer cancel()
|
|
machine := chatstate.NewChatMachine(w.opts.Store, w.opts.Pubsub, chatID)
|
|
err := machine.Update(cleanupCtx, func(tx *chatstate.Tx, store database.Store) error {
|
|
chat, err := store.GetChatByID(cleanupCtx, chatID)
|
|
if errors.Is(err, sql.ErrNoRows) {
|
|
return errSkipAcquire
|
|
}
|
|
if err != nil {
|
|
return xerrors.Errorf("load chat: %w", err)
|
|
}
|
|
if !chat.WorkerID.Valid || chat.WorkerID.UUID != workerID || !chat.RunnerID.Valid || chat.RunnerID.UUID != runnerID {
|
|
return errSkipAcquire
|
|
}
|
|
_, err = tx.Abandon(chatstate.AbandonInput{})
|
|
return err
|
|
})
|
|
if errors.Is(err, errSkipAcquire) || errors.Is(err, chatstate.ErrChatNotFound) {
|
|
return nil
|
|
}
|
|
return err
|
|
}
|