Files
J. Scott MillerandGarrett Delfosse 38e74c4506 feat: add agent-plugins experiment and Agent API v2.17 plugin messages (#29320)
Linear:
[CODAGT-1101](https://linear.app/codercom/issue/CODAGT-1101/agent-plugins-read-only-support-behind-the-agent-plugins-experiment)

**tl;dr:** Adds the `agent-plugins` experiment and the Agent API v2.17
messages that carry Agent Plugins data from the agent to coderd. First
of eight stacked PRs adding read-only [Agent
Plugins](https://agent-plugins.org/) v1.0.0 support.

A plugin is a directory with a `plugin.json` manifest, skills under
`skills/`, and an optional `mcp.json`. Across the stack, the agent
discovers plugins in `plugins/` and `.agents/plugins/` under its scan
roots (for example `~/.coder/plugins/<name>`), chatd lists their skills
with plugin attribution, the agent runs the MCP servers of plugins in
the workspace working directory, and the Agents page shows plugin
attribution in the chat context. Hooks, commands, agents, install flows,
and org-managed plugins are out of scope. Background:
[research/agent-plugins-standard.md](https://github.com/coder/scott-misc/blob/main/research/agent-plugins-standard.md)
(internal).

This PR adds `codersdk.ExperimentAgentPlugins` and a `PluginBody` oneof
variant on `ContextResource` (tag 18) that carries the validated
manifest fields and nests the plugin's skills and MCP servers.
`Manifest` gains `agent_plugins_enabled` (field 23), which coderd sets
when the experiment is enabled; the agent discovers plugins only then.
coderd stores each nested resource as its own row with the plugin's name
in a new `plugin_name` column (migration 000642), which the hydrate, MCP
sync, and re-pin queries copy. With the experiment off,
`PushContextState` drops plugin resources and everything nested in them.
Otherwise it rejects pushes with plugin names that fail
`workspacesdk.ValidatePluginName` (the spec's name grammar, shared by
agent and coderd), OK plugins without a name or with a duplicate name, a
non-OK plugin that nests resources, nested entries of the wrong kind,
plugin skill sources outside `<plugin source>/skills/`, and plugin MCP
servers with an empty `server_name` or a source other than `<plugin
name>/<server_name>`. Nested resources count toward the 1000-resource
cap.

Agents now dial v2.17, so a new agent needs a coderd that supports it,
as with every Agent API minor. Turning the experiment off does not
remove plugin data already pinned to existing chats.

<!-- pr-map:start -->
<details><summary>PR map</summary>

`main..4b0cfc9657`

| Category | Files | + | - |
|---|---:|---:|---:|
| prod (backend) | 15 | 342 | 78 |
| test (backend) | 9 | 534 | 3 |
| migrations | 2 | 14 | 0 |
| generated (backend) | 7 | 1216 | 1027 |
| generated (frontend) | 1 | 2 | 0 |
| **Total** | 34 | 2108 | 1108 |

1. `coderd/agentapi/context.go` (+211/-51, API): drops plugin resources
when the experiment is off; otherwise validates plugin names and
nesting, and stores each nested skill and MCP server as its own row with
the plugin's name.
2. `agent/proto/agent.proto` (+39/-9, API): `PluginBody` with nested
`skills` and `mcp_servers`, and `Manifest.agent_plugins_enabled`.
3. `codersdk/workspacesdk/frontmatter.go` (+31/-0): `ValidatePluginName`
and `MaxPluginNameLength` (manifest name grammar).
4. `coderd/database/queries/chats.sql`, `workspaceagentcontext.sql`,
migration 000642 (+28/-7, DB): the `plugin_name` column on both context
resource tables, copied by the hydrate, MCP sync, and re-pin queries.
5. `codersdk/agentsdk/agentsdk.go`, `agent/agent.go` (+18/-4, API):
`Manifest.AgentPluginsEnabled` and `ConnectRPC217WithRole`; the agent
dials v2.17.
6. `coderd/agentapi/api.go`, `manifest.go` (+4): the manifest flag
follows the experiment.
7. `tailnet/proto/version.go` (+8/-1, API): Agent API v2.17.
8. `codersdk/deployment.go` (+4, API): the `agent-plugins` experiment.

</details>
<!-- pr-map:end -->

---

This PR was generated by Coder Agents on behalf of @jscottmiller.

Co-authored-by: Garrett Delfosse <delfossegarrett@gmail.com>
2026-11-13 10:39:16 -05:00

412 lines
14 KiB
Go

package agentapi
import (
"context"
"io"
"net"
"net/url"
"sync"
"sync/atomic"
"time"
"github.com/google/uuid"
"golang.org/x/xerrors"
"storj.io/drpc/drpcmux"
"storj.io/drpc/drpcserver"
"tailscale.com/tailcfg"
"cdr.dev/slog/v3"
agentproto "github.com/coder/coder/v2/agent/proto"
"github.com/coder/coder/v2/coderd/agentapi/metadatabatcher"
"github.com/coder/coder/v2/coderd/agentapi/resourcesmonitor"
"github.com/coder/coder/v2/coderd/appearance"
"github.com/coder/coder/v2/coderd/boundaryusage"
"github.com/coder/coder/v2/coderd/connectionlog"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/pubsub"
"github.com/coder/coder/v2/coderd/externalauth"
"github.com/coder/coder/v2/coderd/notifications"
"github.com/coder/coder/v2/coderd/portsharing"
"github.com/coder/coder/v2/coderd/prometheusmetrics"
"github.com/coder/coder/v2/coderd/tracing"
"github.com/coder/coder/v2/coderd/workspacestats"
"github.com/coder/coder/v2/coderd/wspubsub"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/coder/v2/codersdk/drpcsdk"
"github.com/coder/coder/v2/tailnet"
tailnetproto "github.com/coder/coder/v2/tailnet/proto"
"github.com/coder/quartz"
)
const workspaceCacheRefreshInterval = 5 * time.Minute
// API implements the DRPC agent API interface from agent/proto. This struct is
// instantiated once per agent connection and kept alive for the duration of the
// session.
type API struct {
opts Options
*ManifestAPI
*AnnouncementBannerAPI
*StatsAPI
*LifecycleAPI
*AppsAPI
*MetadataAPI
*ResourcesMonitoringAPI
*LogsAPI
*ScriptsAPI
*ConnLogAPI
*SubAgentAPI
*BoundaryLogsAPI
*ContextAPI
tailnetService *tailnet.DRPCService
cachedWorkspaceFields *CachedWorkspaceFields
mu sync.Mutex
}
var _ agentproto.DRPCAgentServer = &API{}
// agentTailnetService exposes only Tailnet RPCs intended for workspace agents.
// Other current and future RPCs remain unavailable until explicitly forwarded.
type agentTailnetService struct {
tailnetproto.DRPCTailnetUnimplementedServer
service *tailnet.DRPCService
}
func (s *agentTailnetService) PostTelemetry(ctx context.Context, req *tailnetproto.TelemetryRequest) (*tailnetproto.TelemetryResponse, error) {
return s.service.PostTelemetry(ctx, req)
}
func (s *agentTailnetService) StreamDERPMaps(req *tailnetproto.StreamDERPMapsRequest, stream tailnetproto.DRPCTailnet_StreamDERPMapsStream) error {
return s.service.StreamDERPMaps(req, stream)
}
func (s *agentTailnetService) Coordinate(stream tailnetproto.DRPCTailnet_CoordinateStream) error {
return s.service.Coordinate(stream)
}
var _ tailnetproto.DRPCTailnetServer = (*agentTailnetService)(nil)
type Options struct {
AgentID uuid.UUID
OwnerID uuid.UUID
WorkspaceID uuid.UUID
OrganizationID uuid.UUID
TemplateVersionID uuid.UUID
// WorkspaceBuildID is the build that created the agent, resolved by the
// agent token middleware.
WorkspaceBuildID uuid.UUID
AuthenticatedCtx context.Context
Log slog.Logger
Clock quartz.Clock
Database database.Store
NotificationsEnqueuer notifications.Enqueuer
Pubsub pubsub.Pubsub
// ContextDirtyMarker is the chatd-backed hydrate/dirty fan-out invoked
// from PushContextState. Nil when chatd is disabled.
ContextDirtyMarker ContextDirtyMarker
// ContextSyncDisabled makes PushContextState reject pushes with a dRPC
// Unimplemented code so agents stop sending context snapshots.
ContextSyncDisabled bool
ConnectionLogger *atomic.Pointer[connectionlog.ConnectionLogger]
DerpMapFn func() *tailcfg.DERPMap
TailnetCoordinator *atomic.Pointer[tailnet.Coordinator]
StatsReporter *workspacestats.Reporter
MetadataBatcher *metadatabatcher.Batcher
AppearanceFetcher *atomic.Pointer[appearance.Fetcher]
PublishWorkspaceUpdateFn func(ctx context.Context, userID uuid.UUID, event wspubsub.WorkspaceEvent)
PublishWorkspaceAgentLogsUpdateFn func(ctx context.Context, workspaceAgentID uuid.UUID, msg agentsdk.LogsNotifyMessage)
NetworkTelemetryHandler func(batch []*tailnetproto.TelemetryEvent)
BoundaryUsageTracker *boundaryusage.Tracker
LifecycleMetrics *LifecycleMetrics
PortSharer *atomic.Pointer[portsharing.PortSharer]
AccessURL *url.URL
AppHostname string
AgentStatsRefreshInterval time.Duration
DisableDirectConnections bool
DerpForceWebSockets bool
DisableUserSecretFilePath bool
DerpMapUpdateFrequency time.Duration
ExternalAuthConfigs []*externalauth.Config
Experiments codersdk.Experiments
UpdateAgentMetricsFn func(ctx context.Context, labels prometheusmetrics.AgentMetricLabels, metrics []*agentproto.Stats_Metric)
}
func New(opts Options, workspace database.Workspace, agent database.WorkspaceAgent) *API {
if opts.Clock == nil {
opts.Clock = quartz.NewReal()
}
api := &API{
opts: opts,
mu: sync.Mutex{},
}
api.ManifestAPI = &ManifestAPI{
AccessURL: opts.AccessURL,
AppHostname: opts.AppHostname,
ExternalAuthConfigs: opts.ExternalAuthConfigs,
DisableDirectConnections: opts.DisableDirectConnections,
DerpForceWebSockets: opts.DerpForceWebSockets,
DisableUserSecretFilePath: opts.DisableUserSecretFilePath,
AgentPluginsEnabled: opts.Experiments.Enabled(codersdk.ExperimentAgentPlugins),
AgentFn: api.agent,
Database: opts.Database,
DerpMapFn: opts.DerpMapFn,
WorkspaceID: opts.WorkspaceID,
WorkspaceBuildID: opts.WorkspaceBuildID,
}
// Don't cache details for prebuilds, though the cached fields will eventually be updated
// by the refresh routine once the prebuild workspace is claimed.
api.cachedWorkspaceFields = &CachedWorkspaceFields{}
if !workspace.IsPrebuild() {
api.cachedWorkspaceFields.UpdateValues(workspace)
}
api.AnnouncementBannerAPI = &AnnouncementBannerAPI{
appearanceFetcher: opts.AppearanceFetcher,
}
api.ResourcesMonitoringAPI = &ResourcesMonitoringAPI{
AgentID: opts.AgentID,
WorkspaceID: opts.WorkspaceID,
Clock: opts.Clock,
Database: opts.Database,
NotificationsEnqueuer: opts.NotificationsEnqueuer,
Debounce: 30 * time.Minute,
Config: resourcesmonitor.Config{
NumDatapoints: 20,
CollectionInterval: 10 * time.Second,
Alert: resourcesmonitor.AlertConfig{
MinimumNOKsPercent: 20,
ConsecutiveNOKsPercent: 50,
},
},
}
api.StatsAPI = &StatsAPI{
AgentID: agent.ID,
AgentName: agent.Name,
Workspace: api.cachedWorkspaceFields,
Database: opts.Database,
Log: opts.Log,
StatsReporter: opts.StatsReporter,
AgentStatsRefreshInterval: opts.AgentStatsRefreshInterval,
Experiments: opts.Experiments,
}
api.LifecycleAPI = &LifecycleAPI{
AgentFn: api.agent,
WorkspaceID: opts.WorkspaceID,
Database: opts.Database,
Log: opts.Log,
PublishWorkspaceUpdateFn: api.publishWorkspaceUpdate,
Metrics: opts.LifecycleMetrics,
}
api.AppsAPI = &AppsAPI{
AgentID: agent.ID,
AgentFn: api.agent,
Database: opts.Database,
Log: opts.Log,
Workspace: api.cachedWorkspaceFields,
PublishWorkspaceUpdateFn: api.publishWorkspaceUpdate,
Clock: opts.Clock,
NotificationsEnqueuer: opts.NotificationsEnqueuer,
}
api.MetadataAPI = &MetadataAPI{
AgentID: agent.ID,
Workspace: api.cachedWorkspaceFields,
Database: opts.Database,
Log: opts.Log,
Batcher: opts.MetadataBatcher,
}
api.LogsAPI = &LogsAPI{
AgentFn: api.agent,
Database: opts.Database,
Log: opts.Log,
PublishWorkspaceUpdateFn: api.publishWorkspaceUpdate,
PublishWorkspaceAgentLogsUpdateFn: opts.PublishWorkspaceAgentLogsUpdateFn,
}
api.ScriptsAPI = &ScriptsAPI{
Database: opts.Database,
}
api.ConnLogAPI = &ConnLogAPI{
AgentID: agent.ID,
AgentName: agent.Name,
WorkspaceBuildID: opts.WorkspaceBuildID,
ConnectionLogger: opts.ConnectionLogger,
Database: opts.Database,
Workspace: api.cachedWorkspaceFields,
Log: opts.Log,
}
api.tailnetService = &tailnet.DRPCService{
CoordPtr: opts.TailnetCoordinator,
Logger: opts.Log,
DerpMapUpdateFrequency: opts.DerpMapUpdateFrequency,
DerpMapFn: opts.DerpMapFn,
NetworkTelemetryHandler: opts.NetworkTelemetryHandler,
}
api.SubAgentAPI = &SubAgentAPI{
OwnerID: opts.OwnerID,
OrganizationID: opts.OrganizationID,
AgentFn: api.agent,
Log: opts.Log,
Clock: opts.Clock,
Database: opts.Database,
PortSharer: opts.PortSharer,
}
api.BoundaryLogsAPI = &BoundaryLogsAPI{
Log: opts.Log,
Database: opts.Database,
AgentID: opts.AgentID,
WorkspaceID: opts.WorkspaceID,
OwnerID: opts.OwnerID,
TemplateID: workspace.TemplateID,
TemplateVersionID: opts.TemplateVersionID,
BoundaryUsageTracker: opts.BoundaryUsageTracker,
}
api.ContextAPI = &ContextAPI{
AgentID: agent.ID,
Workspace: api.cachedWorkspaceFields,
Log: opts.Log,
Clock: opts.Clock,
Database: opts.Database,
DirtyMarker: opts.ContextDirtyMarker,
Disabled: opts.ContextSyncDisabled,
Experiments: opts.Experiments,
}
// Start background cache refresh loop to handle workspace changes
// like prebuild claims where owner_id and other fields may be modified in the DB.
go api.startCacheRefreshLoop(opts.AuthenticatedCtx)
return api
}
func (a *API) Server(ctx context.Context) (*drpcserver.Server, error) {
mux := drpcmux.New()
err := agentproto.DRPCRegisterAgent(mux, a)
if err != nil {
return nil, xerrors.Errorf("register agent API protocol in DRPC mux: %w", err)
}
err = tailnetproto.DRPCRegisterTailnet(mux, &agentTailnetService{
service: a.tailnetService,
})
if err != nil {
return nil, xerrors.Errorf("register tailnet API protocol in DRPC mux: %w", err)
}
return drpcsdk.NewServer(a.opts.Log, &tracing.DRPCHandler{Handler: mux},
drpcserver.Options{
Manager: drpcsdk.DefaultDRPCOptions(nil),
Log: func(err error) {
if xerrors.Is(err, io.EOF) {
return
}
a.opts.Log.Debug(ctx, "drpc server error", slog.Error(err))
},
},
), nil
}
func (a *API) Serve(ctx context.Context, l net.Listener) error {
server, err := a.Server(ctx)
if err != nil {
return xerrors.Errorf("create agent API server: %w", err)
}
if err := a.InitMonitors(ctx); err != nil {
return xerrors.Errorf("initialize resource monitoring: %w", err)
}
return server.Serve(ctx, l)
}
func (a *API) agent(ctx context.Context) (database.WorkspaceAgent, error) {
agent, err := a.opts.Database.GetWorkspaceAgentByID(ctx, a.opts.AgentID)
if err != nil {
return database.WorkspaceAgent{}, xerrors.Errorf("get workspace agent by id %q: %w", a.opts.AgentID, err)
}
return agent, nil
}
// refreshCachedWorkspace periodically updates the cached workspace fields.
// This ensures that changes like prebuild claims (which modify owner_id, name, etc.)
// are eventually reflected in the cache without requiring agent reconnection.
func (a *API) refreshCachedWorkspace(ctx context.Context) {
ws, err := a.opts.Database.GetWorkspaceByID(ctx, a.opts.WorkspaceID)
if err != nil {
// Do not clear the cache on transient DB errors. Stale data is
// preferable to no data, which forces callers to fall back to
// expensive queries like GetWorkspaceByAgentID.
a.opts.Log.Warn(ctx, "failed to refresh cached workspace fields", slog.Error(err))
return
}
if ws.IsPrebuild() {
return
}
// If we still have the same values, skip the update and logging calls.
if a.cachedWorkspaceFields.identity.Equal(database.WorkspaceIdentityFromWorkspace(ws)) {
return
}
// Update fields that can change during workspace lifecycle (e.g., AutostartSchedule)
a.cachedWorkspaceFields.UpdateValues(ws)
a.opts.Log.Debug(ctx, "refreshed cached workspace fields",
slog.F("workspace_id", ws.ID),
slog.F("owner_id", ws.OwnerID),
slog.F("name", ws.Name))
}
// startCacheRefreshLoop runs a background goroutine that periodically refreshes
// the cached workspace fields. This is primarily needed to handle prebuild claims
// where the owner_id and other fields change while the agent connection persists.
func (a *API) startCacheRefreshLoop(ctx context.Context) {
// Refresh every 5 minutes. This provides a reasonable balance between:
// - Keeping cache fresh for prebuild claims and other workspace updates
// - Minimizing unnecessary database queries
ticker := a.opts.Clock.TickerFunc(ctx, workspaceCacheRefreshInterval, func() error {
a.refreshCachedWorkspace(ctx)
return nil
}, "cache_refresh")
// We need to wait on the ticker exiting.
_ = ticker.Wait()
a.opts.Log.Debug(ctx, "cache refresh loop exited, invalidating the workspace cache on agent API",
slog.F("workspace_id", a.cachedWorkspaceFields.identity.ID),
slog.F("owner_id", a.cachedWorkspaceFields.identity.OwnerUsername),
slog.F("name", a.cachedWorkspaceFields.identity.Name))
a.cachedWorkspaceFields.Clear()
}
func (a *API) publishWorkspaceUpdate(ctx context.Context, agentID uuid.UUID, kind wspubsub.WorkspaceEventKind) error {
a.opts.PublishWorkspaceUpdateFn(ctx, a.opts.OwnerID, wspubsub.WorkspaceEvent{
Kind: kind,
WorkspaceID: a.opts.WorkspaceID,
AgentID: &agentID,
})
return nil
}