Files
coder/coderd/telemetry/agentrelay.go
Steven Masley bc5dd20a05 feat: report Agent Relay builds in telemetry (#31407)
Telemetry builds now report whether Agent Relay created them. Before
this, relay workspaces looked like any other build.

A start build counts as a relay build when it has the
`agent_relay_session_id` parameter. Stop and delete builds never get the
field. For those builds, the new `agent_relay` field on `WorkspaceBuild`
reports:

- `provider`: `claude_code`, `cursor`, `ambiguous`, or `unknown`
- `client_platform`: Claude Code only. Lowercased, characters outside
`[a-z0-9_]` become `_`, and cut to 15 characters
- `attempt`: omitted when missing or not an integer, so `0` is sent
- `session_id_hash` and `pool_hash`: SHA-256 hashes

No other parameter values are sent. Other builds leave the field out.

The snapshot fetches these parameters with one new batch query. The
workspace-create event reads the stored parameters with one extra query,
only when telemetry is enabled, so it reports the same data as the
snapshot. `docs/admin/setup/telemetry.md` lists the parameters that are
read.

The collector side is
[coder/coder-telemetry-server#56](https://github.com/coder/coder-telemetry-server/pull/56).
Merge this PR first.

Refs
[RELAY-9](https://linear.app/codercom/issue/RELAY-9/telemetry-report-agent-relay-usage-from-codercoder)

<details>
<summary>Decision log</summary>

- **Detect relay builds from parameters.** This needs no relay changes.
Relay registration or
[CORE-426](https://linear.app/codercom/issue/CORE-426) provenance
metadata can come later.
- **Builds only, not workspaces.** `GetWorkspaces` doesn't return the
latest build's parameters, so a workspace field would cost another
query. The collector can join on `workspace_id`.
- **Provider comes from parameter names only.**
`agent_relay_claude_code_*` means Claude Code and `agent_relay_cursor_*`
means Cursor. A build with both reports `ambiguous`; a build with
neither reports `unknown`. coderd stores every declared parameter on
every build (ephemeral ones with their default), so a Claude Code start
after a stop still reports `claude_code`.
- **Start builds only.** Stop and delete builds of a relay workspace
carry the same parameters but are not dispatches, so the snapshot skips
them before querying parameters.
- **Hash session ID and pool.** Pool names are user content, and
sessions only need counting.
- **Known gaps.** Cursor has no client platform. Relay version and
health would need relay registration.
- **Parameter names are shared with coder/agent-relay.** If agent-relay
renames one, telemetry silently stops reading it.

</details>

<details>
<summary>Manual test on <code>develop.sh</code></summary>

Ran on `8ad7cd491f` with `CODER_TELEMETRY_ENABLE=true
CODER_TELEMETRY_URL=http://127.0.0.1:8081` and `go run
./scripts/telemetry-server --port 8081` as the collector.

Pushed three templates with no infrastructure, each declaring the same
parameters as the real modules:

- `relay-claude`: `agent_relay_session_id`, `_delivery_id`, `_pool`,
`_client_platform`, plus ephemeral `agent_relay_attempt` and
`agent_relay_claude_code_lock_to_account`
- `relay-cursor`: `agent_relay_session_id`, `_delivery_id`, `_pool`,
`agent_relay_cursor_pool_name`, `_idle_release_timeout`, `_repo_url`
- `relay-plain`: `region`

Created workspaces with `POST /api/v2/users/me/workspaces` and
`rich_parameter_values`, the same call the relay makes. Then ran stop
and start builds on the relay workspaces, and restarted coderd to get a
periodic snapshot.

| Workspace | Build 1 (create event and snapshot) | Builds 2 and 3
(stop, start) |
|---|---|---|
| Claude (`claude_in_slack`, attempt 1) | `claude_code`, platform,
attempt 1 | `claude_code`, platform, no attempt |
| Cursor | `cursor` | `cursor` |
| Plain | field omitted | n/a |
| Claude template, no session ID | field omitted | n/a |

- Stop and start builds keep `agent_relay_claude_code_lock_to_account`,
with an empty value, so Claude Code rebuilds still report `claude_code`.
- No session ID, pool name, or `region` value appears in the captured
telemetry. The hashes match `sha256` of the inputs.
- Fed the captured snapshot through the conversion code in
coder/coder-telemetry-server#56: all 8 builds converted, 6 of them with
`agent_relay` set.
- No telemetry errors or panics in the coderd logs.

This run predates the start-only change: build 2 (stop) carried the
field then. Stop and delete builds are now omitted, which
`TestTelemetry_WorkspaceBuildAgentRelay` covers.

</details>


---

*Generated by Coder Agents on behalf of @Emyrk.*
2026-11-03 15:48:10 -06:00

170 lines
6.1 KiB
Go

package telemetry
import (
"context"
"crypto/sha256"
"fmt"
"strconv"
"strings"
"github.com/google/uuid"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/coderd/database"
)
// Rich parameter names stamped by Agent Relay (github.com/coder/agent-relay)
// on every build it dispatches. The names are a contract owned by
// agent-relay's template contract and the agent-relay registry modules.
// Renaming one there silently drops it from telemetry here.
const (
agentRelayParameterSessionID = "agent_relay_session_id"
agentRelayParameterPool = "agent_relay_pool"
agentRelayParameterAttempt = "agent_relay_attempt"
agentRelayParameterClientPlatform = "agent_relay_client_platform"
agentRelayParameterClaudeCodePrefix = "agent_relay_claude_code_"
agentRelayParameterCursorPrefix = "agent_relay_cursor_"
)
// agentRelayClientPlatformMaxLength caps the reported client platform. The
// value is user-settable, so it is normalized before it leaves the deployment.
const agentRelayClientPlatformMaxLength = 15
// AgentRelayProvider identifies whose provider-specific parameters a build
// carries.
type AgentRelayProvider string
const (
AgentRelayProviderClaudeCode AgentRelayProvider = "claude_code"
AgentRelayProviderCursor AgentRelayProvider = "cursor"
// AgentRelayProviderUnknown is reported when a build carries no
// provider-specific parameters.
AgentRelayProviderUnknown AgentRelayProvider = "unknown"
// AgentRelayProviderAmbiguous is reported when a build carries
// parameters of more than one provider.
AgentRelayProviderAmbiguous AgentRelayProvider = "ambiguous"
)
// WorkspaceBuildAgentRelay is derived from the agent_relay_* parameters.
type WorkspaceBuildAgentRelay struct {
Provider AgentRelayProvider `json:"provider"`
// ClientPlatform is the surface that started the session, such as
// claude_in_slack. Only the Claude Code provider reports it. It is
// lowercased, characters outside [a-z0-9_] become underscores, and it is
// cut to 15 characters.
ClientPlatform string `json:"client_platform,omitempty"`
// SessionIDHash is the SHA-256 of the relay session ID. Builds of the
// same session share it.
SessionIDHash string `json:"session_id_hash"`
// PoolHash is the unsalted SHA-256 of the relay pool name. It is for
// counting distinct pools; it does not hide common names.
PoolHash string `json:"pool_hash,omitempty"`
// Attempt is the dispatch attempt number, nil when the parameter is
// missing or not an integer.
Attempt *int `json:"attempt,omitempty"`
}
// agentRelayBuildParameters only looks up start builds: stop and delete
// builds of a relay workspace carry the same parameters, but they are not
// dispatches and get no AgentRelay field. Only the parameters the converter
// reads are loaded, so other agent_relay_* values, such as credentials,
// never reach the reporter.
func agentRelayBuildParameters(ctx context.Context, db database.Store, builds []database.WorkspaceBuild) (map[uuid.UUID]map[string]string, error) {
paramsByBuild := make(map[uuid.UUID]map[string]string)
buildIDs := make([]uuid.UUID, 0, len(builds))
for _, build := range builds {
if build.Transition != database.WorkspaceTransitionStart {
continue
}
buildIDs = append(buildIDs, build.ID)
}
if len(buildIDs) == 0 {
return paramsByBuild, nil
}
params, err := db.GetWorkspaceBuildParametersByBuildIDsAndNames(ctx, database.GetWorkspaceBuildParametersByBuildIDsAndNamesParams{
WorkspaceBuildIds: buildIDs,
Names: []string{
agentRelayParameterSessionID,
agentRelayParameterPool,
agentRelayParameterAttempt,
agentRelayParameterClientPlatform,
},
NamePrefixes: []string{
agentRelayParameterClaudeCodePrefix,
agentRelayParameterCursorPrefix,
},
})
if err != nil {
return nil, xerrors.Errorf("get agent relay workspace build parameters: %w", err)
}
for _, param := range params {
if paramsByBuild[param.WorkspaceBuildID] == nil {
paramsByBuild[param.WorkspaceBuildID] = make(map[string]string)
}
paramsByBuild[param.WorkspaceBuildID][param.Name] = param.Value
}
return paramsByBuild, nil
}
// convertWorkspaceBuildAgentRelay returns nil unless the build has a
// non-empty relay session ID. The pool is hashed because it is user content,
// and the session ID because it only needs to be counted.
func convertWorkspaceBuildAgentRelay(parameters map[string]string) *WorkspaceBuildAgentRelay {
sessionID, ok := parameters[agentRelayParameterSessionID]
if !ok || sessionID == "" {
return nil
}
relay := &WorkspaceBuildAgentRelay{
Provider: agentRelayProvider(parameters),
ClientPlatform: normalizeAgentRelayClientPlatform(parameters[agentRelayParameterClientPlatform]),
SessionIDHash: fmt.Sprintf("%x", sha256.Sum256([]byte(sessionID))),
}
if pool := parameters[agentRelayParameterPool]; pool != "" {
relay.PoolHash = fmt.Sprintf("%x", sha256.Sum256([]byte(pool)))
}
if attempt, err := strconv.Atoi(parameters[agentRelayParameterAttempt]); err == nil {
relay.Attempt = &attempt
}
return relay
}
func normalizeAgentRelayClientPlatform(platform string) string {
platform = strings.Map(func(r rune) rune {
switch {
case r >= 'a' && r <= 'z', r >= '0' && r <= '9', r == '_':
return r
default:
return '_'
}
}, strings.ToLower(platform))
if len(platform) > agentRelayClientPlatformMaxLength {
platform = platform[:agentRelayClientPlatformMaxLength]
}
return platform
}
// agentRelayProvider only looks at names: coderd stores a row for every
// parameter the template declares, so ephemeral provider parameters stay
// present, with their default value, on later builds.
func agentRelayProvider(parameters map[string]string) AgentRelayProvider {
var claudeCode, cursor bool
for name := range parameters {
switch {
case strings.HasPrefix(name, agentRelayParameterClaudeCodePrefix):
claudeCode = true
case strings.HasPrefix(name, agentRelayParameterCursorPrefix):
cursor = true
}
}
switch {
case claudeCode && cursor:
return AgentRelayProviderAmbiguous
case claudeCode:
return AgentRelayProviderClaudeCode
case cursor:
return AgentRelayProviderCursor
default:
return AgentRelayProviderUnknown
}
}