Commit Graph
440 Commits
Author SHA1 Message Date
Molly Fraser 7f245d8296 chore!: remove legacy AI Gateway metric aliases (#32381)
Removes the deprecated `coder_aibridged_*` and `coder_aibridgeproxyd_*`
Prometheus metric aliases, which were scheduled for removal in v2.37.
The embedded AI Gateway and AI Gateway Proxy now emit only
`coder_ai_gateway_*` and `coder_ai_gateway_proxy_*`.

- Replace `NewMetricAliasRegisterer` with
`prometheus.WrapRegistererWithPrefix` in `cli/server.go` and
`enterprise/cli/aibridgeproxyd.go`.
- Delete `coderd/prometheusmetrics/metricalias.go` and its test (no
remaining callers).
- Drop alias support from the metrics docs scanner.
- Update the AI Gateway monitoring and rebranding migration docs: legacy
names are removed as of v2.40.

**Breaking:** dashboards and alerts that still use the legacy names stop
matching. The monitoring docs show a temporary `metric_relabel_configs`
bridge.

Closes AIGOV-447.

<details>
<summary>Implementation plan</summary>

| File | Change |
|---|---|
| `cli/server.go` | Remove the `TODO(deprecation)` comment; register
embedded gateway metrics with
`prometheus.WrapRegistererWithPrefix(aibridgemetrics.PrometheusMetricPrefix,
...)`. |
| `enterprise/cli/aibridgeproxyd.go` | Same swap with the
`coder_ai_gateway_proxy_` prefix; drop the unused `prometheusmetrics`
import. |
| `coderd/prometheusmetrics/metricalias.go`, `metricalias_test.go` |
Delete; the two call sites above were the only callers. |
| `scripts/metricsdocgen/scanner/prefix.go` | Remove
`NewMetricAliasRegisterer` from `prefixFuncs`. |
| `scripts/metricsdocgen/scanner/scanner_test.go` | Remove the alias
fixture and the legacy-alias assertion; the remaining literal-prefix
case already covers plain `WrapRegistererWithPrefix`. |
| `scripts/metricsdocgen/README.md` | Drop the
`NewMetricAliasRegisterer` mention. |
| `docs/ai-governance/ai-gateway/monitoring.md` | State that legacy
names are no longer emitted as of v2.40; reword the relabel snippet as a
temporary bridge and explain its cost. |
| `docs/ai-governance/ai-gateway/rebranding-migration.md` | Mark metrics
as "No (removed in v2.40)"; rewrite the Metrics section; remove the
now-obsolete "dropping the old names" section. |

Release note: the `!` title applies `release/breaking`, so the change is
listed under breaking changes in the v2.40 release notes.

Verification: `make gen` (no metrics reference changes), `make lint`,
`pnpm run format-docs`, `pnpm run lint-docs`, doc-check and test-audit
self-review, and targeted tests for `enterprise/cli` (AI Gateway /
proxy), `cli` (AI Gateway), `scripts/metricsdocgen`, and
`coderd/prometheusmetrics`.

</details>

Code written by @mollymaefraser, with Coder Agents used as a guide.
2026-11-19 13:24:00 +00:00
Ehab Younes 1126533c31 feat: export workspace last used time to Prometheus (#31346)
Add an opt-in coderd_workspace_last_used_timestamp_seconds gauge that
reports workspaces.last_used_at, the value the dormancy threshold
measures inactivity from. To enable it, set
--prometheus-collect-workspace-stats or
CODER_PROMETHEUS_COLLECT_WORKSPACE_STATS.

A separate WorkspaceStats collector refreshes the gauge once a minute
in the background from a new GetWorkspaceLastUsedForMetrics query. Each
series has a workspace_id label, so a workspace that reuses a deleted
workspace's name starts a new series. Workspaces owned by deleted users
are excluded.

Closes CLIENT-1065
2026-11-11 15:02:18 +00:00
Molly Fraser 0a7ef88af9 feat: add option to disable the embedded AI Gateway (#31967)
Adds `--ai-gateway-embedded-enabled` /
`CODER_AI_GATEWAY_EMBEDDED_ENABLED` (default `true`) so standalone-only
deployments can stop `coderd` from starting the in-memory AI Gateway.

`CODER_AI_GATEWAY_ENABLED` stays the global switch: the AI Gateway
entitlement, `/api/v2/ai-gateway/serve`, and cost-control metrics still
depend only on it. The embedded daemon now starts only when both options
are `true`.

| `ENABLED` | `EMBEDDED_ENABLED` | Result |
|---|---|---|
| `true` | `true` | Unchanged |
| `true` | `false` | No embedded gateway; standalone gateways keep
working |
| `false` | any | No AI Gateway |

Refs AIGOV-514

## Open questions for reviewers

- **Coder Agents does not work with the embedded gateway disabled.**
`chatd` still starts, but model calls fail with "AI Gateway transport
factory is not configured" because there is no in-process transport.
AIGOV-597 (routing Agents through standalone gateways) was canceled. The
docs and option help state that Agents requires the embedded gateway. Is
that acceptable, or should `chatd` also be turned off in this mode?
- **Option name:** the ticket proposed
`CODER_AI_GATEWAY_EMBEDDED_ENABLED` tentatively. OK to keep?

## Verification

```
$ go test ./cli/ -run 'TestServer_AIGateway' -count=1
ok      github.com/coder/coder/v2/cli   0.518s
$ go test ./enterprise/coderd/ -run 'TestAIGatewayServe' -count=1
ok      github.com/coder/coder/v2/enterprise/coderd     0.755s
```

`TestServer_AIGatewayEmbedded` fails when the new check is removed.

<details>
<summary>Implementation notes</summary>

- `codersdk/deployment.go`: new option and
`AIBridgeConfig.EmbeddedEnabled` field; updated `--ai-gateway-enabled`
description.
- `cli/server.go`: outer `if Enabled` sets up cost-control metrics and
the blocked-users collector (the standalone `/serve` endpoint uses these
too). Inner `if EmbeddedEnabled` creates the daemon, registers the
in-memory handler, and adds its cleanup.
- `cli/server_test.go`: `TestServer_AIGatewayEmbedded` covers the three
modes by checking for the embedded daemon's startup log.
- `enterprise/coderd/aibridgeserve_test.go`:
`TestAIGatewayServeEmbeddedDisabled` checks the entitlement and `/serve`
still work with embedded disabled.
- `docs/ai-governance/ai-gateway/standalone.md`: new "Disable the
embedded gateway" section.
- Remaining files are generated by `make gen`.
- Found during review: keeping metrics inside the embedded check would
have dropped cost-control metrics for standalone gateways.

</details>

---

Generated with Coder Agents; reviewed, revised, and verified by
@mollymaefraser.
2026-11-11 11:50:16 +00:00
Michael Suchacz 7c39819665 fix(cli): close coderd on server startup errors (#30280)
`coder server` closes the coderd API only in its graceful shutdown
sequence. Every startup error returned after `newAPI` (creating
aibridged, registering Prometheus metrics, writing the config URL,
creating provisioner daemons, notifying systemd) skipped that close and
leaked the whole coderd instance: tailnet and wireguard-go, gvisor,
metricscache, dbrollup, cryptokeys, chatd, and pubsub queues.
`TestServer/Logging` hits this when test cleanup cancels startup while
aibridged is being created. The test helper ignores the resulting
`context canceled` error, so only the package-level goleak check in
`cli` fails.

The fix closes the API in a deferred `sync.OnceValue` registered right
after `newAPI`, and the shutdown sequence calls the same function. The
guard is needed because the closer is not safe to call twice: the AGPL
API returns "API already closed", and the enterprise closer closes each
component again. The normal shutdown path is unchanged.

## Testing

The new `TestServer/StartupErrorNoLeak` fails startup after coderd is
created by putting a directory at the config URL path.

- Red (test only): `go test ./cli -run
'^TestServer$/^StartupErrorNoLeak$' -count=1` exited 1 in 3 of 3 runs
with `goleak: Errors on successful test run: found unexpected
goroutines` (coderd frames) while the subtest passed.
- Green (with the fix): the same command passed in 3 of 3 runs.
- `TestServer/Logging` alone, 20 runs from one test binary: 20 of 20
passed with no goleak failure, including 3 runs that hit the `create
aibridged` startup error.
- `go test ./cli -count=1` passed in 4 of 4 runs, and `go vet ./cli` is
clean.

Found in the CI of #30272 (`test-go-pg (ubuntu-latest)`).

> Xum, an AI coding agent, implemented, tested, and opened this PR on
behalf of @ibetitsmike.
2026-10-07 13:39:31 +02:00
Callum Styan 15616728d5 refactor: move coderd/x/nats to coderd/nats 2026-10-05 16:52:36 -07:00
Sas Swart 23fb35d185 feat: let the AI Gateway emit the structured interception records (#30030)
coderd emitted the structured interception records as they arrived over
DRPC, so
a record the gateway never sends cannot be exported.

- Add `--ai-gateway-structured-logging-source` (`coderd`, `gateway`,
`both`) and
teach the gateway's `LogRecorder` to emit the same format where the
records
originate. Gateway lines omit `thread_parent_id` and `thread_root_id`,
which
  are resolved by a database lookup only coderd can perform.
- Reaching the serving pool required repairing the pool options: they
were
derived at construction sites and discarded, so the server always built
its
pool with the defaults. They are now derived from the deployment
configuration
in one place and passed to the pool that serves requests. The server
reports
those options, and its ownership of the `key_pool_state` collector, so a
second
  registration cannot silently reappear and panic startup.
- The standalone gateway inherits both structured logging options, since
it is
  now one of the processes that can emit.

This is the keystone of the stack: the flag, the mechanism that carries
it and
the behavior it selects land together, so
`TestAIGatewayStart_InheritedOptions`
and the gateway help output stay truthful at every commit.

<details>
<summary>Why the stack is ordered this way</summary>

The final design is a chain of recorder decorators:

    +---------------------+
    |    AsyncRecorder    |  all records except the interception start
    +---------------------+
    |     LogRecorder     |  emit the structured record
    +---------------------+
    | ValidatingRecorder  |  refuse malformed records
    +---------------------+
    |  caller middleware  |  record policy, e.g. drop content
    +---------------------+
    |    TraceRecorder    |
    +---------------------+
    |    DRPCRecorder     |  acquire client, send to coderd
    +---------------------+

Ordering constraints:

- The record type constants (1) are the vocabulary both emitters and the
  validator use, so they come first.
- Validation (5) rejects unset timestamps, so records must already be
stamped
where they are created (3). Stamping at creation leaves
`WrappedRecorder`
  doing nothing but client acquisition, which is why it is folded into
  `DRPCRecorder` (4) and not before.
- coderd's server-time fallback (2) is what keeps a gateway of an older
version
  from persisting zero times once the gateway owns stamping (3).
- PRs 1 to 5 change no user-visible surface. Each deployment option then
lands
with the code that reads it: the log source with gateway-side emission
(6),
content recording with the filter that drops the records (7). Pool
options are
plumbed to the serving pool in 6, where the first option that needs them
  arrives, rather than as an unused hook beforehand.

</details>

---
🤖 Raised by [Coder Agents](https://coder.com) on behalf of @SasSwart.
2026-09-29 11:48:52 +02:00
Callum Styan 83bef8a12d fix: consolidate pg and nats pubsub metrics generation into one (#26843) 2026-09-22 14:00:44 -07:00
Ehab Younes 9974294ca9 feat: report arbitrary workspace agent session types (#28338)
Report session counts as a dynamic app-name map from the agent, the SSH CLI, and the workspace usage API, so a new IDE flows end to end without a schema, proto, or server change.

- Report through `Stats.session_counts` on Agent API v2.11. Older agents keep reporting through the deprecated count fields.
- Accept arbitrary names from `coder ssh --usage-app` and the workspace usage API. A name the registry does not recognize keeps its identity and is attributed to the `unknown` family.
- Add reconnecting PTY counts to the map without overwriting the counts SSH reported.
- Normalize and merge names before applying the deterministic 64-name cap, ordered by known apps, then count, then name. Overflow sums into `unknown`, and non-positive counts cannot cancel valid activity.
- Strip control characters, treat whitespace-only HTTP app names as absent, and require the exact environment variable prefix when you extract an SSH session type.
- Count overflow entries in `coderd_agentstats_session_counts_overflow_total` and keep raw payloads out of logs.
- Change the usage API's `app_name` to a string, and update the SDK types and API reference.

closes #27410
2026-09-19 00:32:51 +02:00
Spike Curtis aae145bdc9 feat: make NATS pubsub opt-out via no_nats_pubsub experiment (#29283)
Makes the embedded NATS pubsub an opt-out feature instead of opt-in. The
`nats_pubsub` experiment is removed and replaced with a `no_nats_pubsub`
experiment, so NATS pubsub is now the default and can be disabled to
fall back to PostgreSQL pubsub.

NATS clustering requires this replica's routable address, resolved into
`clusterHost` from either `--cluster-host` (`CODER_CLUSTER_HOST`) or the
DERP relay URL. When neither is set (allowed when DERP is disabled on
the primary), NATS cannot form a cluster, so startup logs a big error
and falls back to PostgreSQL pubsub. The NATS cluster CA is only
minted/rotated when NATS clustering is actually active (experiment not
set and a cluster host is present).

<details>
<summary>Implementation notes</summary>

- `codersdk/deployment.go`: removed `ExperimentNATSPubsub`
(`nats_pubsub`), added `ExperimentNoNATSPubsub` (`no_nats_pubsub`,
display name "No NATS Pubsub").
- `cli/server.go`: NATS is used by default unless `no_nats_pubsub` is
enabled; if `clusterHost` is empty, log an error and keep the PostgreSQL
pubsub.
- `coderd/coderd.go`: gate NATS CA minting/rotation on
`!ExperimentNoNATSPubsub && ClusterHost != ""` (i.e. only when NATS
clustering is active).
- `enterprise/coderd/licenses.go`: updated a stale comment.
- `enterprise/coderd/coderd_test.go`: `TestMultiReplica_NATSPubsubPeers`
no longer sets the removed experiment.
- Regenerated `coderd/apidoc/docs.go`, `coderd/apidoc/swagger.json`,
`docs/reference/api/schemas.md`, `site/src/api/typesGenerated.ts`.

</details>

---
*This PR was generated by Coder Agents on behalf of @spikecurtis.*
2026-09-18 12:58:06 +00:00
Michael Suchacz d2e94e299a chore: delete dead functions and skip-only or duplicated tests (#29461) 2026-09-17 18:14:08 +02:00
Bobby Ho 7bb049f24c feat: log the OAuth2 provider state and registered apps at startup (#29267)
PLAT-492: Log OAuth2 provider state at startup

Log whether the OAuth2 provider is enabled and name CODER_OAUTH2_PROVIDER_ENABLE.
When disabled, warn if OAuth2 applications are still registered and tell admins how to re-enable the provider.
Startup continues if the application lookup fails.
Add tests for enabled, disabled, registered-app, and query-error cases.
Manually verified upgrade behavior, route gating, token preservation, dashboard visibility, and all supported flag forms.
2026-09-14 18:31:48 -07:00
Paweł Banaszewski 8472a6884f chore!: remove env-based AI provider configuration (#29053)
> AI tools were used to create this PR

Removes deprecated AI provider configuration through environment
variables, CLI flags, and YAML.
Deletion includes indexed providers and legacy OpenAI, Anthropic, and
Bedrock options under both AI Gateway and AI Bridge branding, eg.
`CODER_AI_GATEWAY_PROVIDER_<N>_*` or `CODER_AI_GATEWAY_ANTHROPIC_KEY`.
Removes the associated startup-seeding code.
Updates tests and documentation.

**Breaking change:** Remove [obsolete AI provider CLI flags and YAML
keys](https://coder.com/docs/ai-coder/ai-gateway/providers#database-management-of-providers)
before upgrading, otherwise Coder will fail to start with an `unknown
flag` or `unknown option` error. Provider environment variables are
ignored and no longer seed the database. Existing database-stored
provider configuration is preserved.
2026-09-09 10:16:24 +02:00
Michael Suchacz d6b67e8e31 feat: enforce SSRF protection for MCP config-directed traffic (#28242)
Routes all MCP config-directed traffic from coderd and chatd through a
shared SSRF-protected HTTP client, now that organization admins, not
only deployment admins, control MCP server URLs (#27942 and its stack
below).

## Summary

- Uses the [`coder/safedial`](https://github.com/coder/safedial)
library: it blocks private and special-purpose destinations at dial time
(validating resolved addresses at connection time so DNS rebinding
cannot bypass the check) and rejects cross-origin redirects.
- Covers the complete traffic surface: OAuth2 discovery, dynamic client
registration, code exchange, token refresh, revocation, and runtime MCP
connections from chatd.
- Deployments that intentionally host internal MCP servers opt in via
the new `--mcp-allowed-private-cidrs`
(`CODER_MCP_ALLOWED_PRIVATE_CIDRS`) option.
- Includes the deployment configuration surface, generated docs, CLI
goldens, TypeScript types, and regression coverage for each traffic
path.

## Merge window

This protection originally lived inside #27942 and was split out to keep
that diff reviewable. Until this PR lands, the stack below ships with
only main's existing discovery IP-range guard
(`CODER_MCP_OAUTH2_DISCOVERY_ALLOWED_IP_RANGES`), while org admins can
already point MCP configs at arbitrary URLs. This PR should merge
promptly after the stack below it.

Top of the MCP org-separation stack (CODAGT-711 -> CODAGT-717 audit ->
CODAGT-712 ACLs -> CODAGT-806 token RBAC -> CODAGT-714 org picker ->
this PR).

> Mux (AI agent) authored this PR on Mike's behalf.

<!-- mux-attribution: model=claude-fable-5 thinking=high -->
2026-08-30 13:09:17 +02:00
Yevhenii Shcherbina 0bc2858b6e feat: report unpriced AI models to owners (#28419)
Closes
[AIGOV-568](https://linear.app/codercom/issue/AIGOV-568/notify-admins-about-unpriced-ai-models).

AI Bridge records token usage for a model with no price at a NULL cost,
so its spend is neither reported nor enforced against any budget. Until
now that only reached admins through an info log and a Prometheus
metric.

Owners now receive a weekly report listing models used without a price
in the past week. The report links to documentation explaining how Coder
calculates spend from the default price book and how owners can
configure missing prices with `coder exp ai-model-prices`.

## Screenshots

<img width="726" height="531" alt="image"
src="https://github.com/user-attachments/assets/168ae378-60cb-4c96-8f60-36c6edc0b268"
/>
<img width="470" height="417" alt="image"
src="https://github.com/user-attachments/assets/7dc71a1e-0c57-44f1-86f7-df800603c865"
/>



## Design

**Derived, not tracked.** The unpriced set is computed at report time
from interceptions that recorded token usage, joined against the current
`ai_model_prices` table. Requests that produced no token usage are
excluded.

**No new loop or table.** `reportUnpricedAIModels` is a sibling of
`reportFailedWorkspaceBuilds` in the existing report generator, reusing
its ticker and `notification_report_generator_logs`. Each report runs in
its own transaction under a separate advisory lock, so failures and lock
contention do not couple otherwise independent reports. The weekly
frequency is enforced by the persisted timestamp rather than by the
ticker, which restarts with the process and runs on a different phase in
each replica.

## Behaviour

| Situation | Outcome |
|---|---|
| Model used without a price | Listed in the next weekly report |
| Price set | Disappears from the next report |
| Still unpriced and still in use | Reported again each week |
| Model stops being used | Drops out of the report |
| Nothing unpriced | No notification; window still advances |
| More than 100 unpriced models | Top 100 by usage, with the total
reported alongside |
| openai-compat models | Excluded, they cannot be priced |
| Many replicas | Exactly one report per week |
| First ever run | Reports models used in the preceding week immediately
|

## Design choices

1. **openai-compat** is excluded because it cannot be priced and would
produce permanent, unactionable notifications.
2. **Token volume** orders the list but is not shown. The 100-model cap
therefore prioritizes models responsible for the most unpriced usage.
3. **Placement** remains in the generic report generator rather than
`aibridgedserver`. This keeps one periodic-report lifecycle at the cost
of coupling the notifications package to an AI cost-control query.
4. **No license gate.** The reporter runs even when AI Gateway is
disabled. With no recorded token usage the query is empty and no
notification is sent.

---

Created by Coder Agents on behalf of @evgeniy-scherbina.
2026-08-27 21:04:13 +00:00
McKayla はな 2a8e390f0c refactor: replace most usage of ptr.Ref with new (#28282) 2026-08-26 11:57:56 -06:00
McKayla はな 5ca0143992 refactor: upgrade golangci-lint to v2.0.2 (#28416) 2026-08-21 23:46:27 -06:00
Cian Johnston 6079c514ee fix: follow-up fixes for conditional VCS requests (#27711)
Follow-ups from #27627 

- Memoizes `Config.Git()` with a mutex so the provider's ETag response
cache survives across calls. Only successful construction is cached;
errors are retried.
- Moves the HTTP client onto `Config.HTTPClient`, wired through
`ConvertConfig`, so `Git()` no longer takes a per-call argument that
would be silently ignored after memoization.
- `newGitHub` and `newGitLab` now return `(Provider, error)`,
eliminating the typed-nil-interface class in `gitprovider.New` rather
than the single instance.
- Gates the 304 branch on a `haveCached` flag instead of a nil body
check.
- Only caches bodies that decode successfully, preventing poisoned
entries.
- Keys the response cache on the full token digest rather than a
truncated prefix.
- Tests added: `TestConfigGitMemoizesProvider`,
`TestConfigGitRetriesOnConstructorError`,
`TestGitLabConstructorErrorReturnsNilInterface`,
`TestResponseCacheStore`,
`TestConditionalRequestReuse/MalformedResponseNotCached`;
`TestConvertYAML/CustomScopesAndEndpoint` now asserts
`Config.HTTPClient` wiring.

Follow-ups tracked in #28139, #28140, #28141, #28142.

> 🤖 Generated by Coder Agents on behalf of @johnstcn.
2026-08-18 09:00:20 +01:00
Asher b5d18bb9c9 feat: add redirect URL override for external auth (#28082) 2026-08-17 14:09:23 -08:00
J. Scott Miller 66b065323b feat: log rate-limited external auth token validation (#26754)
When `ValidateToken` keeps a token because the external auth validation
endpoint was rate-limited (a `403` with rate-limit headers or a `429`),
it returns `valid=true` without provider confirmation. Previously this
happened silently, so operators couldn't tell a provider-confirmed token
from one kept optimistically during a rate limit.

This adds a `Logger` to `externalauth.Config` and emits a `Warn` (with
`provider_id`, `provider_type`, `status_code`, and `reason`) on those
rate-limit branches. It also adds a
`coderd_oauth2_external_requests_rate_limited_total{name, source,
status_code}` counter, incremented in the instrumented round tripper
whenever a provider returns a rate-limited response. The rate-limit
detection is the shared `xhttp.IsRateLimited` (in `coderd/util/xhttp`),
used by both the tripper and `ValidateToken` so the metric and the
validation decision share one definition; no extra wiring is needed
since `ValidateToken` already routes through the instrumented client
with `source="ValidateToken"`.

One deliberate behavioral change rides along: rate-limit detection now
also recognizes the unprefixed `RateLimit-Remaining` header (GitLab, and
the IETF draft rate-limit headers), so a `403` with
`RateLimit-Remaining: 0` is treated as optimistically valid where it was
previously treated as revoked. All other valid/invalid decisions are
unchanged. `TestValidateToken` asserts the warning's fields on the
rate-limited cases and no warning for revocations, `401`, and confirmed
responses; `promoauth` and `xhttp` tests cover the detector and the new
counter.

<details>
<summary>Manual testing</summary>

The signals fire on the external-auth status check (`GET
/api/v2/external-auth/{id}`), which calls `ValidateToken`. To force a
rate-limited response, point a provider's `validate_url` at a mock that
returns the rate-limit shape:

1. Run a mock returning `429` on one path and `403` +
`X-RateLimit-Remaining: 0` on another.
2. Start `coder server` with `--prometheus-enable` and external auth
providers whose `validate_url` point at those mock paths (e.g.
`CODER_EXTERNAL_AUTH_0_VALIDATE_URL=http://127.0.0.1:5599/429`).
3. Create a stored link, either complete the OAuth flow, or insert a row
into `external_auth_links` with a future `oauth_expiry` (token contents
are irrelevant; the mock rejects regardless).
4. `curl` the status endpoint with a session token, then check:
- coderd logs for the `Warn` (`reason=status_code` for `429`,
`reason=rate_limit_headers` for `403`),
- the metrics endpoint for
`coderd_oauth2_external_requests_rate_limited_total{...,status_code="429"|"403"}`.

Notes: `scripts/testidp -429` only rate-limits `/oauth2/userinfo`, not
the `/external-auth-validate/...` path, so it does not exercise this;
use a mock `validate_url`. The default Prometheus port `2112` may
already be taken on dogfood workspaces, set `CODER_PROMETHEUS_ADDRESS`
to a free port.

</details>

🤖 Generated with the help of Coder Agents on behalf of @jscottmiller.
2026-08-10 14:43:16 -05:00
Susana Ferreira c351280a37 feat: add Prometheus metrics for AI Governance cost control (#27490)
## Description

Adds Prometheus metrics for AI budget cost control, emitted by the
aibridged server under the `cost_control` subsystem (full names are
prefixed `coder_ai_gateway_`).

- `blocked_requests_total` (counter) — labels: `group_id`
- `blocked_users` (gauge) — labels: `group_id`
- `unpriced_requests_total` (counter) — labels: `provider`, `model`
- `enforcement_duration_seconds` (histogram) — labels: `outcome`

## Changes

- Add `GetOverBudgetUsersPerGroup` query (plus dbauthz/dbmetrics/dbmock
wiring) to count over-budget users per effective group.
- Add a background collector that refreshes the `blocked_users` gauge on
an interval, started only when Prometheus is enabled.
- Wire `Metrics` through the aibridged server, coderd API,
`cli/server.go`, and the enterprise AI gateway handler; recording is
nil-safe when metrics are unset.

Closes
https://linear.app/codercom/issue/AIGOV-296/add-prometheus-metrics-for-cost-control

> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
2026-07-28 09:22:58 +01:00
Paweł Banaszewski 5770085435 fix: add prefix to standalone metrics (#27526)
Adds `coder_ai_gateway_` to standalone Gateway metics to match embedded
case.
2026-07-27 13:02:49 +00:00
Atif Ali 2b2a5c963a Revert "fix(coderd): explain default GitHub app org visibility on login rejection" (#27388) 2026-07-21 16:17:27 +00:00
Atif Ali 48e9bb3391 fix(coderd): explain default GitHub app org visibility on login rejection (#27374)
## Problem

On a fresh deployment with no custom GitHub OAuth app, Coder falls back
to the default Coder-managed GitHub app. That app can only see
organization memberships in organizations where it has been installed.
If `CODER_OAUTH2_GITHUB_ALLOWED_ORGS` is set but the app isn't installed
in the allowed organizations, the membership list comes back empty and
every login, including the first admin login, is rejected with a bare
"You aren't a member of the authorized Github organizations!" with no
hint about the actual cause. This leaves fresh deployments in an
apparently broken state.

## Fix

* Append a remediation hint to the login rejection when the default
provider is configured, pointing at the [app installation
page](<https://github.com/apps/coder/installations/select_target>) and
at configuring a custom GitHub OAuth app.
* Log a startup warning when the default provider is combined with
`CODER_OAUTH2_GITHUB_ALLOWED_ORGS`, listing the allowed orgs and the
install URL.
* Document the installation requirement next to the
`CODER_OAUTH2_GITHUB_ALLOWED_ORGS` step in the GitHub auth docs.

Access-control behavior is unchanged; the org check still rejects logins
as before, it just explains why and how to fix it.

## Testing

* New `TestUserOAuth2Github/NotInAllowedOrganizationDefaultProvider`
asserts the hint appears when `DefaultProviderConfigured` is set; the
existing `NotInAllowedOrganization` subtest asserts it does not leak
into the custom-app path.

Fixes coder/coder#17752
2026-07-21 20:43:10 +05:00
Paweł Banaszewski 3126306598 feat: add --aigateway-proxy-target flag (#27122)
Adds `--aigateway-proxy-target` option to
`deploymentGroupAIGatewayProxy` that defines URL to which intercepted
requests should be forwarded to.
Forward URL used to be hardcoded to `coderAPI.AccessURL` pointing to
embedded Gateway. With addition of standalone AI Gateway this needs to
be configurable.

Renamed `aibridgeproxyd.Server.coderAccessURL` and `coderAccessPort` ->
`gatewayURL` and `gatewayPort` + option to better reflect reality.
2026-07-14 16:40:22 +00:00
Callum Styan ad29777cb2 feat: NATS mTLS pubsub implementation (#26902) 2026-07-13 11:00:02 -07:00
Paweł Banaszewski bab8ce9d41 feat: setup logging, tracing and metrics in standalone AI Gateway (#27068)
Adds logging, tracing and metrics setup to standalone AI Gateway.
Existing options are re-used when possible.
2026-07-09 14:02:18 +00:00
Ethan 8b60d1d877 ci: cache embedded postgres binaries in flake checks (#26986)
flake-go keeps failing every `TestServer` subtest that boots `coder
server` with built-in PostgreSQL ([example
run](https://github.com/coder/coder/actions/runs/28685288282/job/85134336803))
with `no version found matching 13.21.0`, which is embedded-postgres's
error for any non-200 while downloading the Postgres binary archive from
Maven. The archive gets cached under the server's config root, which is
a fresh temp dir in every test, so with `test-count: 35` one flake run
downloads it dozens of times and Maven rate-limits the runner. Using an
external Postgres (like coder/terraform-provider-coderd#370 did) would
defeat the point, since these subtests exist to exercise the built-in
Postgres path.

Instead, `startBuiltinPostgres` now honors `EMBEDDED_PG_CACHE_DIR` as
the archive cache path (test runs only, data/runtime dirs stay
per-test), and flake-go.yaml wires in the existing `embedded-pg-cache`
actions the same way the Windows/macOS lanes in ci.yaml already do. All
iterations then share a single download, and usually zero once the
actions cache is warm. The upload step only saves on `main`.
2026-07-07 11:44:46 +10:00
Itay Dafna d7ad85f7f6 feat: support multiple OIDC redirect URIs (#25408)
This PR adds a new opt-in setting, `CODER_OIDC_REDIRECT_ALLOWED_HOSTS`,
that lets a single Coder deployment complete OIDC login on more than one
hostname. When the allowlist is non-empty, Coder picks the OIDC
`redirect_uri` based on the incoming request's Host header (validated
against the list) instead of always using the static URL derived from
`CODER_ACCESS_URL`. When unset, the (default) behavior is identical to
today.

The motivation is that a single Coder deployment is frequently reachable
via multiple hostnames - for example, an internal hostname for users on
a corporate VPN and a different hostname routed through a zero-trust
gateway for users off-VPN - but OIDC login today only works on whichever
single hostname `CODER_ACCESS_URL` points to, because the `redirect_uri`
sent to the IdP is fixed at server startup. Users who reach the
deployment on any other valid hostname can see the login page but fail
the OIDC callback, since the IdP redirects them back to a hostname they
can't reach (or whose cookies they don't have).
2026-07-05 06:36:33 +02:00
Mathias Fredriksson 047c47495b refactor: drop chat_model_configs provider column (#26877)
The provider type already lives authoritatively in ai_providers.type,
reachable on every active row through ai_provider_id, which the
chat_model_configs_ai_provider_required_when_active CHECK makes
mandatory. The stored provider string was a denormalized copy the system
kept in sync with a startup backfill and no longer needs.

Every surface now derives provider type from the linked ai_providers
row. Telemetry is the one exception: it keeps emitting provider, now
sourced from ai_providers.type via a JOIN, so the BigQuery column and the
Nexus dashboards that read it are unaffected. The experimental HTTP/SDK
response drops provider and makes ai_provider_id required, since those
endpoints return only active configs; consumers resolve provider type
from ai_provider_id and the AI providers listing.

This ships in a single release with no compatibility window: production
reads the table via SELECT *, so a pre-drop binary fails config reads the
moment the column is gone. Operators must scale to zero before upgrading,
and there is no rollback.

Closes CODAGT-599
2026-07-01 15:59:55 +03:00
Danny Kopping ce94d42e19 feat: fetch providers over DRPC (#26650)
Closes [AIGOV-455](https://linear.app/codercom/issue/AIGOV-455/extend-drpc-with-buildproviders).

## Why

The AI Gateway (`aibridged`) is being split into a standalone process that must not touch the database. `coderd` stays the source of truth and seeds the `ai_providers` / `ai_provider_keys` tables from the environment. This PR adds a DRPC call so the gateway fetches provider config from `coderd` instead of reading the DB, for both the embedded and standalone daemons.

## What

- **Proto:** new `ProviderConfigurator` service with a unary `GetAIProviders` RPC, plus `AIProvider` / `AIProviderBedrock` messages. `CurrentMinor` bumped to 1 (additive).
- **Server (`coderd/aibridgedserver`):** `GetAIProviders` runs a read-only `InTx` under `LockIDAIProvidersEnvSeed` so it never returns a mid-seed snapshot, reads providers (incl. disabled) plus keys for enabled ones, and maps to proto under `dbauthz.AsAIBridged`. Unmappable rows are skipped and logged; plaintext keys and Bedrock secrets are never logged.
- **Client:** `DRPCProviderConfiguratorClient` wired into the client union, `dialer.go`, and `CreateInMemoryAIBridgeServer`.
- **cli:** `BuildProvidersFromProto` maps the response through the existing DB-neutral `buildProvider`. A shared `poolRPCReloader` does the fetch/build/replace for both daemons: the embedded daemon reloads on every `ai_providers` change and fails startup if it cannot subscribe; the standalone gateway drives the same reloader once at startup, retrying until success and staying interruptible.
- **Dead code removed:** `BuildProvidersFromConfig`, `ProvidersFromConfig`, `AIProviderFromConfig`, and the DB-read `BuildProviders` path.
2026-06-29 13:34:58 +02:00
Steven Masley ad355aeaa9 feat: add INSECURE oidc email fallback flag for IdP brokers (#26751)
<!-- Authored by Coder Agents on behalf of @Emyrk. -->

Adds an opt-in `CODER_DANGEROUS_OIDC_EMAIL_FALLBACK` flag (alias
`--dangerous-oidc-email-fallback`) for IdP brokers that do not issue a
stable `sub` for the same user across connections.
2026-06-26 11:23:42 -05:00
Steven Masley 84350e4e7c feat: report SCIM configuration on Deployment (#26628)
Adds two nullable booleans to `telemetry.Deployment`:

- `SCIMEnabled`: `true` when `CODER_SCIM_AUTH_HEADER` is set.
- `SCIMUseLegacy`: `true` when `CODER_SCIM_USE_LEGACY` is set.

Both mirror `Deployment.IDPOrgSync`: nullable for backward
compatibility, and report configuration state rather than license
entitlement (#16323).

Lives on `Deployment` rather than `Snapshot` so the existing
`bqDeployment` table on `coder/coder-telemetry-server` gets two columns
instead of a new table.

`SCIMAPIKey` is annotated as a secret and is scrubbed by
`WithoutSecrets` before the config reaches telemetry, so
`DeploymentConfig.SCIMAPIKey` is always empty in production. The
booleans are pre-computed from the pre-scrub `DeploymentValues` in
`cli/server.go` and passed in via `telemetry.Options.SCIMEnabled` /
`SCIMUseLegacy`.

Pairs with
[coder/coder-telemetry-server#43](https://github.com/coder/coder-telemetry-server/pull/43),
which adds the matching `bqDeployment` columns and the manual BigQuery
`ALTER TABLE` step.

---

Generated by Coder Agents on behalf of @Emyrk.
2026-06-25 08:54:37 -05:00
Jon Ayers 6da322d59f feat: add Prometheus metrics to NATS pubsub for parity with PGPubsub (#26441) 2026-06-23 11:59:48 -05:00
Susana Ferreira 970bd73691 feat: add /api/v2/ai-gateway API route aliases (#26475)
## Description

Registers `/api/v2/ai-gateway/*` as the new API path for AI Gateway, replacing `/api/v2/aibridge/*`. Both prefixes share the same route builder (`aiBridgeRoutes`) backed by a single in-memory handler, so existing `/aibridge` endpoints continue to work. New endpoints must be registered on the enterprise API handler under `/api/v2/ai-gateway` only.

Swagger annotations now point to `/api/v2/ai-gateway` paths with a backward-compatibility note referencing `/aibridge`. The legacy `/aibridge` routes are skipped in the swagger documentation test.

## Changes

- Store one raw handler (`aiGatewayHandler`) instead of two prefix-stripped handlers
- Register `/ai-gateway` and `/ai-gateway/proxy` route aliases alongside legacy `/aibridge` routes
- Move `/aibridge/keys` to `/ai-gateway/keys`
- Update in-process transport to use `/api/v2/ai-gateway` prefix
- Update SDK client URLs and proxy forwarding URL
- Swap `@Router` and `@Tags` annotations from `aibridge`/`AI Bridge` to `ai-gateway`/`AI Gateway`
- Rename user-facing error messages from "AI Bridge" to "AI Gateway"
- Define consts for route prefixes (`AIGatewayRootPath`, `AIBridgeRootPath`)
- Update tests and comments to use new paths

Note: the following will be addressed in follow-up PRs:
- Frontend API URLs
- Frontend routes and redirects
- Dogfood main.tf updates
- Hand-written documentation URL updates
- aibridge internal comments and nits
- Scale tests path updates

Refs https://linear.app/coder/issue/AIGOV-230

> Generated with the assistance of Coder Agents (@ssncferreira)
2026-06-23 12:15:10 +01:00
Cian Johnston d5ec26beac chore: replace testing.Testing with flag lookup (#26552)
In our codebase we have an existing convention of using
`flag.Lookup("test.v")` instead of `testing.Testing()`. This avoids
pulling in the entire `testing` package. Another consequence: some of
our custom linters trigger upon import of the `testing` package which
can lead to unexpected linter errors.
2026-06-19 19:59:54 +01:00
Paweł Banaszewski f1ce1013c4 chore: export AI Gateway metrics under new branding + keep old as alias (#26413)
> AI Tools where used in this request.

Registers `coder_aibridged_*` and `coder_aibridgeproxyd_*` metrics under
new prefixes: `coder_ai_gateway_*` and `coder_ai_gateway_proxy_*`.
Old prefix is still exported. Will be removed in later release.

Also updated the `metricsdocgen` static fixture. Added 4
previously-undocumented metrics `key_pool_state`,
`key_pool_state_transitions_total`, `key_pool_exhaustions_total`,
`key_pool_failover_attempts` added the `client` label to the existing
interception, prompt, and token counter samples.

Updated AI Gateway documentation.
2026-06-17 13:10:53 +02:00
Steven Masley 0e45ded0ed feat: deployment flag to auto handle changed oidc providers (#26419)
An opt-out flag exists as an escape hatch

closes https://linear.app/codercom/issue/PLAT-343/automatically-reset-user-link-for-affected-users-when-idp-provider
2026-06-16 13:26:04 -07:00
Steven Masley 1d03e63f4f feat: implement package and cli tool for repairing oidc links (#26418) 2026-06-16 12:46:10 -07:00
Hugo Dutka 62288782fc chore: clean up dbpurge after the chatd refactor (#26344)
Addresses
https://github.com/coder/coder/pull/26109#discussion_r3379072397 and
https://github.com/coder/coder/pull/26109#discussion_r3379093655.
2026-06-16 14:01:31 +02:00
Danny Kopping a1330e3a8c refactor: rename Ai* database identifiers to AI* (AIGOV-369) (#26327)
Adds `ai` to sqlc's `gen.go.initialisms` in `coderd/database/sqlc.yaml`
so the generated DB code follows Go's initialism convention. Adds the
matching `ai` -> `AI` case to the dbgen PascalCase helper
(`scripts/dbgen/main.go`) so the corresponding `dbmem` / mock
identifiers stay in sync. `make gen` regenerates the rest; hand-written
call sites that consume DB-generated identifiers
(`enterprise/audit/table.go`, `coderd/database/modelmethods.go`,
`enterprise/coderd/aigatewaykeys.go`, `coderd/database/dbauthz/*`, etc.)
are updated to match.

Scope is deliberately limited to the database layer:

- `coderd/rbac/*` (resource and scope generators) is untouched —
`ResourceAi*` / `ScopeAi*` constants stay on main's casing.
- `codersdk/*` (Go SDK) is untouched — `codersdk.ResourceAi*` /
`codersdk.APIKeyScopeAi*` constants stay on main's casing, so external
Go SDK consumers see no source-level break.
- `Aibridge*` identifiers (one SQL token `aibridge`, not `ai_bridge`)
are out of scope.

On-the-wire values are unchanged: enum strings, RBAC resource type
strings, API key scope strings, and JSON tags all stay the same. The
HTTP/JSON surface is unaffected.

Refs:
[AIGOV-369](https://linear.app/codercom/issue/AIGOV-369/change-ai-references-in-coderddatabasemodelsgo-to-ai)

🤖 Generated with [Coder Agents](https://coder.com)
2026-06-16 09:01:43 +00:00
Hugo Dutka 4debd23cbb fix: chatd refactor (#26270)
Implements the chatd stabilization RFC.

Combines:
- https://github.com/coder/coder/pull/25908
- https://github.com/coder/coder/pull/25923
- https://github.com/coder/coder/pull/26109
- https://github.com/coder/coder/pull/26110
- https://github.com/coder/coder/pull/26111
- https://github.com/coder/coder/pull/26112
2026-06-12 13:33:12 +02:00
Cian Johnston a4c867f11b fix: backfill legacy Bedrock AI provider rows and stale model config strings (#26155)
Fixes CODAGT-548

Adds two idempotent startup backfills run after `newAPI():

- `BackfillBedrockProviderType`: promotes `ai_providers` rows from
`type=anthropic` with Bedrock settings to `type=bedrock`.
- `BackfillChatModelConfigProviderStrings`: fixes stale
`chat_model_configs.provider = "anthropic"` strings on rows whose linked
provider was just promoted.
- `UpdateAIProvider` query now also writes the `type` column, so the
fix persists on any subsequent PATCH.


> 🤖 Generated by Claude with oversight from a human.
2026-06-11 15:31:31 +01:00
Danny Kopping 78a6ec293e revert: "fix: avoid an errant license warning banner on new deployments that d…" (#26240)
Reverts coder/coder#26239

We cannot disable a feature which was previously enabled; this is a BC
break.
This is also using `AIGatewayRoutingEnabled` which will be removed in
the next release.
2026-06-11 08:07:32 +00:00
Sas Swart d0e9c5eda5 fix: avoid an errant license warning banner on new deployments that d… (#26239)
Problem: CODER_AI_GATEWAY_ENABLED defaulted to true, which both started
the in-memory gateway and enabled the licensed FeatureAIBridge. As a
result, deployments that never configured AI Gateway saw a spurious "AI
Governance add-on is required" warning whenever they had an older
(non-add-on) Premium license, since the feature was enabled-and-entitled
by default.

Fix: Decouple "external AI Gateway API enabled" from "in-memory daemon
running," so the external/licensed surface is off by default while Coder
Agents retain access by default.
2026-06-11 09:17:26 +02:00
Kyle Carberry dab1d3c81e fix(cli): sort external auth env vars by numeric index (#26230) 2026-06-10 14:21:54 -07:00
Cian JohnstonandCopilot Autofix powered by AI a26c46a3bf fix!: validate HostnameSuffix and SSHConfigOptions' (#26154)
- Adds server-side and client-side validation for
CODER_CONFIGSSH_HOSTNAME_SUFFIX and CODER_SSH_CONFIG_OPTIONS.
- **Server-side breaking change:** invalid values for either of these will cause `coderd` to exit with an error.
- Client-side: `coder config-ssh` will exit with an error if it detects invalid config.
- Adds tests for the above

Local smoke-testing: ran `develop.sh --env-file <path to an env file
containing badness>`. Validated that server startup failed as expected.

> 🤖 Generated by Coder Agents with supervision from a human.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-10 15:48:02 +01:00
Susana Ferreira 01ec5e4577 feat: add key pool failover metrics to aibridge (#25901)
## Description

This PR adds Prometheus metrics for aibridge's API-key failover, giving visibility into key pool health and failover behavior per provider.

The following metrics are introduced:

- **`key_pool_state`** (gauge): number of keys currently in each state (`valid`, `temporary`, `permanent`) per provider, sampled at scrape time.
- **`key_pool_state_transitions_total`** (counter): key state transitions during failover, labeled by `reason` (`rate_limited`, `unauthorized`, `forbidden`).
- **`key_pool_exhaustions_total`** (counter): times a pool ran out of usable keys, labeled by `outcome` (`rate_limited`, `auth_failed`).
- **`key_pool_failover_attempts`** (histogram): keys attempted before success or exhaustion (per interception for bridged requests, per request for passthrough).

## Changes

- Moves `MarkKeyOnStatus` and key-pool error handling onto `*keypool.Pool`.
- Attaches metrics to each provider's key pool at install time, on construction and on provider reload.
- Adds a scrape-time state collector and a `KeyPools()` accessor on the bridge pool to feed it.
- Tracks per-request key attempts in the bridged and passthrough failover paths.
- Adds test coverage for the new metrics across the keypool unit tests, the bridged intercept failover tests, and the passthrough failover test.

Closes https://github.com/coder/internal/issues/1447
Closes https://linear.app/codercom/issue/AIGOV-198/aibridge-key-failover-observability

> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
2026-06-09 10:49:47 +01:00
Jon Ayers 167ac7b879 feat: add nats experiment (#25703) 2026-06-03 15:37:19 -05:00
Danny Kopping c8555e2163 fix: deprecate ai provider seeding env config (#25854)
Environment variables used to configure AI Gateway providers are now deprecated, and we need to reflect this as such.
2026-06-01 15:15:47 +02:00
Danny Kopping 12520ee964 feat: add ai provider status and reload freshness metrics (#25770)
Add metrics for `aibridged` and `aibridgeproxyd`'s provider statuses. AI providers can be modified, and possibly misconfigured, at runtime. These metrics help operators understand the state of these provider definitions in case unexpected behaviour is observed.
2026-05-28 14:57:33 +02:00