Commit Graph
42 Commits
Author SHA1 Message Date
Thomas Kosiewski d11809f095 fix(coderd): enforce chatd concurrent-agent capacity on every path into running (#30243)
## Problem

The concurrent-agent cap (5 root chats and 10 subagent chats per
deployment by default) could be exceeded. Three paths put more chats in
`running` with a fresh lease than the pool allows:

1. **Interrupting a waiting chat.** A user sends a message with
`busy_behavior=interrupt` to a chat that is waiting for capacity. The
chat becomes `interrupting` while unowned, a worker acquires it without
admission, and `FinishInterruption` promotes the queued message into a
running turn under that ownership.
2. **Taking over an orphaned `requires_action` chat.** A replica dies
while its chat is in `requires_action`, its lease goes stale and another
chat takes the slot. A worker acquires the `requires_action` chat
without admission, and a tool result, an interrupt or the action
deadline returns it to `running` under that ownership.
3. **Heartbeat reviving a stale lease.** A replica loses the database
for longer than the 30 second stale threshold and another replica admits
a second chat into the slot. When the first replica comes back, its
heartbeat upsert renews the stale lease unconditionally, and its runner
keeps generating until the turn ends.

## Root cause

`agentCapacityLimiter.Admit` returned `true` without taking the
admission lock or counting for every status other than `running`. The
count query already treats every owned chat with a fresh lease as
holding a slot, so the shortcut let chats acquire a slot uncounted and
later move back to `running` under that ownership. Separately,
`BatchUpsertChatHeartbeats` had no freshness or ownership condition.

## Fix

- **Admission for every runnable status.** `Admit` now counts for
`running`, `interrupting` and `requires_action`, and a refused pool
skips the rest of its candidates in the pass regardless of status. An
owned chat holds its slot until it releases ownership, so every later
transition back to `running` stays within capacity.
- **Fenced heartbeat renewal.** `BatchUpsertChatHeartbeats` is replaced
by `RenewChatHeartbeats`, which only extends rows that are still fresh
and still own their chat, using `statement_timestamp()` read after the
capacity admission lock is granted. The renewal runs under that lock
with a `lock_timeout` (`HeartbeatLockTimeout`) and a per-tick timeout
(`HeartbeatRenewalTimeout`, a third of the stale threshold). Admission
also waits at most 5 seconds for the lock.
- **Stop runners that lost their lease.** `heartbeatOnce` requests
cleanup of every runner whose lease was not renewed, which cancels its
generation. The chat stays owned by the stale lease until an admitted
worker takes it over. A tick that fails or times out cleans up no
runners and retries on the next tick.
- **Instant Stop on a queued chat.** Requiring admission for
`interrupting` would have made Stop on a chat waiting for capacity hang
until a slot frees. When `Interrupt`, or `SendMessage` with
`busy_behavior=interrupt`, runs on a running chat with no owner, the
transition now applies `FinishInterruption` in the same transaction and
cancels orphaned tool calls. An empty queue lands in `waiting`;
otherwise the queue head is promoted into `running`, still unowned, so
it goes through admission.
- **Queued metric.** `CountChatCapacityQueuedByPool` (behind
`coderd_chatd_agents_queued_for_capacity`) now also counts
`interrupting` and `requires_action` chats without a fresh lease, since
they also wait for admission.

## How it was found

A TLA+ model of capacity admission found a counterexample for each path.
Go tests reproduced all three against the previous code (each failed
with `"2" is not less than or equal to "1"` for a root pool of capacity
1), and the model with the fix enabled passes the safety invariant and
the liveness properties. The model itself is not part of this PR.

## Tests

The three reproductions are kept as regression tests in
`coderd/x/chatd/agentcapacity_regression_internal_test.go`:
`TestAdmission_InterruptWithMessageOnWaitingChatStaysWithinCapacity`,
`TestAdmission_RequiresActionTakeoverStaysWithinCapacity` and
`TestRunnerManager_HeartbeatDoesNotReviveStaleLease`. The original
reproductions failed on the base commit. Restoring the admission
shortcut makes the `requires_action` test fail, and restoring the
unconditional heartbeat upsert, or dropping the runner cleanup, makes
the heartbeat test fail. New coverage also includes
`TestAdmission_StopWaitingChatNeedsNoCapacity`,
`TestRenewChatHeartbeats_OnlyRenewsFreshOwnedLeases`,
`TestRunnerManager_FailedHeartbeatTickKeepsRunners`, and chatstate tests
for the unowned interrupt paths. Existing tests that encoded the old
behavior (`requires_action` chats holding no slot, interrupting chats
always acquirable, unconditional heartbeat renewal) were updated.

## Behavior / API change

- `POST /api/v2/chats/{chat}/messages` with `busy_behavior=interrupt` on
a chat that no worker owns (for example one waiting for capacity) now
returns the promoted message with `queued=false` instead of a queued
message, and the chat stays in `running` instead of `interrupting`. If
the chat already had queued messages, the queue head is promoted and the
new message stays queued.
- `POST /api/v2/chats/{chat}/interrupt` on such a chat lands it in
`waiting` immediately (or in `running` with the queue head promoted)
instead of `interrupting`.
- An unowned `interrupting` or `requires_action` chat (for example after
its replica died) now waits for a free slot before its runner starts,
which also delays enforcing its action deadline while the pool is full.
- New internal chat worker settings `HeartbeatRenewalTimeout` and
`HeartbeatLockTimeout`, defaulting to a third of the stale threshold and
half of that.
- Heartbeat ticks now take the capacity admission advisory lock briefly
on each replica.

## Out of scope / follow-ups

- Interrupting a chat still owned by a dead runner (stale lease,
`worker_id` set) still waits for a slot.
- An admission lock timeout should end the acquisition pass instead of
logging a warning per candidate.
- The per-chat `queued_for_capacity` flag in the chat API still reports
only `running` chats.
- Alert on failed heartbeat ticks.

`coderd/x/chatd/ARCHITECTURE.md` gains TODO lines for the affected
sections; the prose is left for the owner.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-01 16:42:51 +02:00
Jaayden Halko bc44cdda75 feat: rank chat workspace templates (#25037)
closes CODAGT-203

## Summary

`list_templates` now returns a ranked shortlist with a recommendation,
so the chat agent can pick the right template the way a colleague would:
prefer what matches the request, what the user already uses, and what
the rest of the organization uses. Instead of teaching the model an enum
protocol in prompts, every result carries a fixed `next_step`
instruction telling the agent what to do.

## How list_templates works

1. **Fetch**: active, non-deprecated templates in the chat's
organization, filtered by the admin template allowlist, authorized as
the chat owner (no system escalation).
2. **Query relevance** (optional `query` argument): each template
receives the highest tier any of its fields matches, and a higher tier
always outranks a lower one regardless of usage:

   | Tier | Match |
   |------|-------|
   | 4 | name or display name equals the query |
   | 3 | name or display name starts with the query |
   | 2 | name or display name contains the query |
| 1 | description contains the query (checked only when no name field
matched) |
   | 0 | no match; the template is excluded |

Matching is case-insensitive and ignores spaces/hyphens/underscores
(`python gpu` matches `python-gpu`).
3. **Usage signals**: a new `GetTemplateRankingSignalsByOwnerID` query
returns, per template, the owner's active and recently-deleted workspace
counts within a 60-day window, the last in-window usage, and the count
of distinct developers with an active workspace (unclaimed prebuilds
excluded).
4. **Affinity score** (computed in Go, per template, from that
template's signals only):

   ```text
affinity = 10 x (active + 0.5 x deleted) x 0.5^(days_since_last_use /
14)
            + ln(1 + active_developers)
   ```

`active`/`deleted` are the owner's in-window workspace counts,
`days_since_last_use` is measured from the most recent in-window usage
(the personal term is zero without in-window usage), and
`active_developers` is the org-wide count. Personal usage carries 10x
the weight of org popularity; the confidence floor is the score of two
active developers (`ln 3`) and the required lead over the runner-up is
`ln 3 - ln 2`.
5. **Rank**: query tier first (when a query is present), then affinity
score, then name/ID for determinism. Results paginate 10 per page with
`next_page` present only when more exist.

## Recommendation contract

The result tells the agent what to do next instead of describing
confidence levels:

- `recommended_template_id` is present only when the top template is a
clear winner: the only available template, a decisive query match, or an
affinity score that clears a floor and leads the runner-up by a derived
margin.
- `next_step` is always present and is one of four fixed sentences: use
the recommendation, ask the user to choose, retry a query that matched
nothing, or report that no templates are available.

Per-template items carry raw evidence (`active_developers`,
`your_workspace_count`, `last_used_by_you`) rather than derived labels.
When signals fail to load, the tool logs and degrades to asking the user
unless the query alone is decisive.

Prompts and the `create_workspace`/`read_template` descriptions
reference the field through the `chattool.NextStepField` constant, so
the instruction lives in one place and cannot drift. `create_workspace`
remains idempotent and allowlist-enforced.

## Authorization

The signals query runs with the chat owner's permissions: reading the
owner's own workspaces plus a template-metadata read for the cross-user
popularity count. dbauthz rejects the call if any requested template is
not readable by the owner (covered by allow and deny method tests).

## Docs

Adds `docs/ai-coder/agents/tools/` explaining how agent tool calls work,
with `list_templates` ranking and the `next_step` contract as the first
documented tools.
2026-06-18 06:41:47 +01:00
Cian Johnston b5a625549e feat: migrate agents-access to org-scoped system role for proper chat RBAC (#24438)
The agents-access role previously granted chat permissions at user
scope, but chats are org-scoped objects. Rego skips user-level perms
when org_owner is set, making the grants invisible. Handler-level
band-aids used synthetic non-org-scoped objects as a workaround.

  - Migrates agents-access from users.rbac_roles (site-level) to
    organization_members.roles (org-scoped) via DB migration
  - Redefines agents-access as a predefined org-scoped builtin role
    alongside organization-admin, organization-auditor, etc., with
    Member permissions granting chat create/read/update
  - Excludes ResourceChat from OrgMemberPermissions so org membership
    alone no longer grants chat access
  - Fixes handler Authorize checks to use org-scoped objects with
semantically correct actions (ActionUpdate for message/tool operations)
  - Grants org admins the ability to assign agents-access

Closes #24250
Fixes CODAGT-174

Note: this does not update the "Usage" endpoints. Tracked by CODAGT-161.
> 🤖
2026-04-23 17:59:42 +01:00
Cian Johnston fd1e2f0dd9 fix(coderd/database/dbauthz): skip Accounting check when sub-test filtering (#23281)
- Detect `-testify.m` sub-test filtering in `SetupSuite` and skip the `Accounting` check.

> 🤖 This PR was created with the help of Coder Agents, and was reviewed by my human. 🧑‍💻
2026-03-24 14:58:04 +00:00
Mathias Fredriksson 147df5c971 refactor: replace sort.Strings with slices.Sort (#23457)
The slices package provides type-safe generic replacements for the
old typed sort convenience functions. The codebase already uses
slices.Sort in 43 call sites; this finishes the migration for the
remaining 29.

- sort.Strings(x)          -> slices.Sort(x)
- sort.Float64s(x)         -> slices.Sort(x)
- sort.StringsAreSorted(x) -> slices.IsSorted(x)
2026-03-23 23:19:23 +02:00
91ec0f1484 feat: add service_accounts workspace sharing mode (#23093)
Introduce a three-way workspace sharing setting (none, everyone,
service_accounts) replacing the boolean workspace_sharing_disabled.
In service_accounts mode, only service account-owned workspaces can be
shared while regular members' share permissions are removed. Adds a
new organization-service-account system role with per-org permissions
reconciled alongside the existing organization-member system role.

Related to:
https://linear.app/codercom/issue/PLAT-28/feat-service-accounts-sharing-mode-and-rbac-role

---------

Co-authored-by: Steven Masley <Emyrk@users.noreply.github.com>
Co-authored-by: Kayla はな <mckayla@hey.com>
2026-03-17 12:16:43 -07:00
George K cc2efe9e1f feat(coderd/rbac): make organization-member a per-org system custom role (#21359)
Migrated the built-in organization-member role to DB storage so it can be customized per org.

Closes https://github.com/coder/internal/issues/1073 (part 1)
2026-01-12 18:19:19 -08:00
Spike Curtis bddb808b25 chore: arrange imports in a standard way (#21452)
Fixes all our Go file imports to match the preferred spec that we've _mostly_ been using. For example:

```
import (
	"context"
	"time"

	"github.com/prometheus/client_golang/prometheus"
	"golang.org/x/xerrors"
	"gopkg.in/natefinch/lumberjack.v2"

	"cdr.dev/slog/v3"
	"github.com/coder/coder/v2/codersdk/agentsdk"
	"github.com/coder/serpent"
)
```

3 groups: standard library, 3rd partly libs, Coder libs.

This PR makes the change across the codebase. The PR in the stack above modifies our formatting to maintain this state of affairs, and is a separate PR so it's possible to review that one in detail.
2026-01-08 15:24:11 +04:00
Spike Curtis 49b34a716a fix: fix slog to always use array of Fields (#21426)
Upgrades to slog v3 which includes a small, but backward incompatible API change to the acceptible call arguments when logging. This change allows us to verify via compile time type checking that arguments are correct and won't cause a panic, as was possible in slog v1, which this replaces (v2 was tagged but never used in coder/coder).

It also updates dependencies that also use slog and were updated.

I've left the `aibridge` dependency as a commit SHA, under the assumption that the team there (cc @pawbana @dannykopping ) will tag and update the dependency soon and on their own schedule.

Other dependencies, I pushed new tags.
2026-01-08 10:29:41 +04:00
Hugo Dutka e62c5db678 chore: remove references to dbtestutil.WillUsePostgres (#20436)
Addresses https://github.com/coder/internal/issues/758.

This PR only cleans up dead code, it makes no changes to test logic.
2025-10-23 14:24:54 +02:00
Thomas Kosiewski ed90ecf00e feat: add allow_list to resource-scoped API tokens (#19964)
# Add API key allow_list for resource-scoped tokens

This PR adds support for API key allow lists, enabling tokens to be scoped to specific resources. The implementation:

1. Adds a new `allow_list` field to the `CreateTokenRequest` struct, allowing clients to specify resource-specific scopes when creating API tokens
2. Implements `APIAllowListTarget` type to represent resource targets in the format `<type>:<id>` with support for wildcards
3. Adds validation and normalization logic for allow lists to handle wildcards and deduplication
4. Integrates with RBAC by creating an `APIKeyEffectiveScope` that merges API key scopes with allow list restrictions
5. Updates API documentation and TypeScript types to reflect the new functionality

This feature enables creating tokens that are limited to specific resources (like workspaces or templates) by ID, making it possible to create more granular API tokens with limited access.
2025-10-09 14:53:08 +02:00
Steven Masley ce935657f6 test: start migrating dbauthz tests to mocked db (#19257)
This PR adds a framework to move to a mocked db. And therefore massively speed up these tests.
2025-08-08 13:46:24 -05:00
Cian Johnston 198d50dbc2 chore: replace original GetPrebuiltWorkspaces with optimized version (#18832)
Fixes https://github.com/coder/internal/issues/715

Follow-up from https://github.com/coder/coder/pull/18717

Now that we've determined the updated query is safe, remove the duplication.
2025-07-21 15:31:11 +01:00
Ethan 7c077d39c5 chore: populate connectionlog count using a separate query (#18629)
This is the third PR for moving connection events out of the audit log.

This PR populates `count` on `ConnectionLogResponse` using a separate query, to preemptively mitigate the issue described in #17689. It's structurally identical to a portion of https://github.com/coder/coder/pull/18600, but for the connection log instead of the audit log.
       
Future PRs:
- Implement a table in the Web UI for viewing connection logs.
- Write a query to delete old events from the audit log, call it from dbpurge.
- Write documentation for the endpoint / feature
2025-07-15 15:03:30 +10:00
Cian Johnston 0367dbac43 chore: optimize GetPrebuiltWorkspaces query (#18717)
* Adds GetRunningPrebuiltWorkspacesOptimized query
* Runs both original and updated query side-by-side and logs diffs
2025-07-09 11:30:42 +01:00
Kacper Sawicki 695de6e0c0 chore(coderd/database): optimize AuditLogs queries (#18600)
Closes #17689

This PR optimizes the audit logs query performance by extracting the
count operation into a separate query and replacing the OR-based
workspace_builds with conditional joins.

## Query changes
* Extracted count query to separate one
* Replaced single `workspace_builds` join with OR conditions with
separate conditional joins
* Added conditional joins
* `wb_build` for workspace_build audit logs (which is a direct lookup)
    * `wb_workspace` for workspace create audit logs (via workspace)

Optimized AuditLogsOffset query:
https://explain.dalibo.com/plan/4g1hbedg4a564bg8

New CountAuditLogs query:
https://explain.dalibo.com/plan/ga2fbcecb9efbce3
2025-07-01 07:31:14 +02:00
ケイラ fae30a00fd chore: remove unnecessary redeclarations in for loops (#18440) 2025-06-20 13:16:55 -06:00
Jon AyersandClaude 17ddee05e5 chore: update golang to 1.24.1 (#17035)
- Update go.mod to use Go 1.24.1
- Update GitHub Actions setup-go action to use Go 1.24.1
- Fix linting issues with golangci-lint by:
  - Updating to golangci-lint v1.57.1 (more compatible with Go 1.24.1)

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <claude@anthropic.com>
2025-03-26 01:56:39 -05:00
brettkolodny 8c0350e20c feat: add a paginated organization members endpoint (#16835)
Closes
[coder/internal#460](https://github.com/coder/internal/issues/460)
2025-03-10 14:42:07 -04:00
Mathias Fredriksson 0e8ef09415 test(coderd/database/dbauthz): compare outputs with cmp (#16161) 2025-01-20 11:12:04 +02:00
Hugo Dutka 106b1cd3bc chore: convert dbauthz tests to also run with Postgres (#15862)
Another PR to address https://github.com/coder/coder/issues/15109.

- adds the DisableForeignKeysAndTriggers utility, which simplifies
converting tests from in-mem to postgres
- converts the dbauthz test suite to pass on both the in-mem db and
Postgres
2025-01-08 16:22:51 +01:00
Steven Masley 854044e811 chore: fix concurrent CommitQuota transactions for unrelated users/orgs (#15261)
The failure condition being fixed is `w1` and `w2` could belong
to different users, organizations, and templates and still cause a
serializable failure if run concurrently. This is because the old query 
did a `seq scan` on the `workspace_builds` table. Since that is the 
table being updated, we really want to prevent that.

So before this would fail for any 2 workspaces. Now it only fails if
`w1` and `w2` are owned by the same user and organization.
2024-11-01 11:05:49 -04:00
Steven Masley 7b09d98238 chore: add /groups endpoint to filter by organization and/or member (#14260)
* chore: merge get groups sql queries into 1

* Add endpoint for fetching groups with filters
* remove 2 ways to customizing a fake authorizer
2024-08-15 13:40:15 -05:00
Steven Masley de9e6889bb chore: merge organization member db queries (#13542)
Merge members queries into 1 that also joins in the user table for username.
Required to list organization members on UI/cli
2024-06-12 09:23:48 -10:00
Steven Masley 5ccf5084e8 chore: create type for unique role names (#13506)
* chore: create type for unique role names

Using `string` was confusing when something should be combined with
org context, and when not to. Naming this new name, "RoleIdentifier"
2024-06-11 08:55:28 -05:00
Garrett Delfosse 5789ea5397 chore: move stat reporting into workspacestats package (#13386) 2024-05-29 11:49:08 -04:00
Steven Masley cf91eff7cf chore: implement databased backend for custom roles (#13295)
Includes db schema and dbauthz layer for upserting custom roles. Unit test in `customroles_test.go` verify against escalating permissions through this feature.
2024-05-16 13:11:26 -05:00
Steven Masley cb6b5e8fbd chore: push rbac actions to policy package (#13274)
Just moved `rbac.Action` -> `policy.Action`. This is for the stacked PR to not have circular dependencies when doing autogen. Without this, the autogen can produce broken golang code, which prevents the autogen from compiling.

So just avoiding circular dependencies. Doing this in it's own PR to reduce LoC diffs in the primary PR, since this has 0 functional changes.
2024-05-15 09:46:35 -05:00
Steven Masley dd05a6b13a chore: mockgen archived, moved to new location (#11415)
* chore: mockgen archived, moved to new location
2024-01-04 18:35:56 -06:00
Cian Johnston 213b768785 feat(coderd): insert provisioner daemons (#11207)
* Adds UpdateProvisionerDaemonLastSeenAt
* Adds heartbeat to provisioner daemons
* Inserts provisioner daemons to database upon start
* Ensures TagOwner is an empty string and not nil
* Adds COALESCE() in idx_provisioner_daemons_name_owner_key
2023-12-18 16:44:52 +00:00
Steven MasleyandCian Johnston e63de9a259 chore: enforcement of dbauthz tests was broken (#11218)
* chore: enforcement of dbauthz tests was broken

Implemented missing tests to catch back up

---------

Co-authored-by: Cian Johnston <cian@coder.com>
2023-12-15 18:30:21 +00:00
Steven Masley dba0dfa859 chore: correct 500 -> 404 on workspace agent mw (#11129)
* chore: correct 500 -> 404
2023-12-12 15:14:32 -06:00
Kyle Carberry 5abfe5afd0 chore: rename dbfake to dbmem (#10432) 2023-10-30 17:42:20 +00:00
Jon Ayers 997493d4ae feat: add template setting to require active template version (#10277) 2023-10-18 17:07:21 -05:00
Kyle Carberry 22e781eced chore: add /v2 to import module path (#9072)
* chore: add /v2 to import module path

go mod requires semantic versioning with versions greater than 1.x

This was a mechanical update by running:
```
go install github.com/marwan-at-work/mod/cmd/mod@latest
mod upgrade
```

Migrate generated files to import /v2

* Fix gen
2023-08-18 18:55:43 +00:00
Cian Johnston 784696dfa5 feat: add prometheus metrics to database.Store (#7713)
* Adds dbmetrics package and wraps database.Store with a Prometheus HistogramVec of timings.
* Adds Wrappers method to database.Store to avoid double-wrapping interfaces
* Fixes test flake in TestLicensesListFake
2023-05-31 14:55:57 +01:00
Steven Masley 38e5b9679b chore: Rbac errors should be returned, and not hidden behind 404 (#7122)
* chore: Rbac errors should be returned, and not hidden behind 404

SqlErrNoRows was hiding actual errors
* Replace sql.ErrNoRow checks
* Remove sql err no rows check from dbauthz test
* Fix to use dbauthz system user
2023-04-13 13:06:16 -05:00
Ammar Bandukwala 3b73321a6c feat: refactor deployment config (#6347) 2023-03-07 15:10:01 -06:00
Ammar Bandukwala f05609b4da chore: format Go more aggressively 2023-02-18 18:32:09 -06:00
Cian Johnston f0f39b4892 chore: break down dbauthz.System into smaller roles (#6218)
- rbac: export rbac.Permissions
- dbauthz: move GetDeploymentDAUs, GetTemplateDAUs,
  GetTemplateAverageBuildTime from querier.go to system.go
  and removes auth checks
- dbauthz: remove AsSystem(), add individual roles for
  autostart, provisionerd, add restricted system role for 
  everything else
2023-02-15 16:14:37 +00:00
Steven Masley 80bde1e2c9 chore: Ensure cancelled errors return proper (#6200)
The authz library returns a 404 if the authorization fails. If the
context is cancelled, then a 404 message is inaccurate.
Add a unit test to ensure context cancelled errors are raised
properly
2023-02-14 10:57:19 -06:00
Steven Masley 6fb8aff6d0 feat: Add initial AuthzQuerier implementation (#5919)
feat: Add initial AuthzQuerier implementation
- Adds package database/dbauthz that adds a database.Store implementation where each method goes through AuthZ checks
- Implements all database.Store methods on AuthzQuerier
- Updates and fixes unit tests where required
- Updates coderd initialization to use AuthzQuerier if codersdk.ExperimentAuthzQuerier is enabled
2023-02-14 14:27:06 +00:00