Files
coder/coderd/x/chatd/skill_invocation.go
T
Michael Suchacz d1f308d9f3 feat: let organization members turn off organization skills in chats (#31134)
## Stack

Organization skills load into every member's Coder Agents chats in their
organization, with no way for a member to opt out. This stack adds a
per-organization setting that lets members turn them off for their own
chats, bottom to top:

1. **#31134** data model, API, and chatd (this PR)
2. #31135 admin and member controls in the dashboard, the `/` menu, and
docs

## What changes

Organizations get `members_can_disable_skills`, off by default. Users
who can update organization skills change it with `PUT
/api/experimental/organizations/{organization}/skill-settings`, which
writes an organization audit entry. Anyone who can read the organization
can `GET` it.

While the setting is on, each member saves their own choices with `PATCH
/api/experimental/organizations/{organization}/members/{user}/skill-preferences`:
`all_skills_disabled` (a master switch that also covers skills created
later) and `skills`, a sparse map from skill ID to on or off. `GET`
returns the setting, the master switch, and `disabled_skill_ids`. The
routes follow the personal-settings rules: without `read_personal` or
`update_personal` on that user the caller gets 404, a PATCH while the
setting is off gets 403, and a key that is not an organization skill
that member can read gets a 400 that names the key. The check runs as
the member, not the caller, so an owner or user admin editing someone
else's choices is held to that member's skills. Turning the setting off
keeps the choices but ignores them.

Migration 000636 adds the column and two tables keyed by organization
membership. Both cascade when the membership goes away, and the
per-skill table also cascades when the skill is deleted, so a recreated
skill loads by default.

chatd reads the preferences as the chat owner when it builds the skill
index and when it loads a body for `read_skill` or for a structured
skill invocation (#30861). A skill the member turned off is left out of
the index, and both loads report it as not found, including later in a
running chat. If the preferences cannot be read, turn preparation fails,
as it already does when the skill metadata cannot be read.

Older migration tests created organizations with `dbgen.Organization`
and `GetDefaultOrganization`, which now select the new column before it
exists, so they insert organizations with raw SQL instead.

## Review focus

- Authorization in `coderd/organizationskillpreferences.go` and the five
new dbauthz methods (`read_personal` and `update_personal` on the user,
`organization_skill` update for the setting).
- `GetOrganizationSkillMemberPreferencesRow.Excludes` is the one rule
chatd applies; #31135 mirrors it in `activeOrganizationSkills`.

## Testing

- Go tests cover the queries and cascades, dbauthz, both routes
(including the audit diff and each 400 case), and chatd (index,
`read_skill`, skill invocations, master switch, a change during a chat).
Each new test fails with its code path removed.
- Remote dogfood UAT (10 scenarios, real model) passed on the pre-split
branch head `186c28d81ab`, which has the same code except the fix below.
It found one issue: two spellings of one skill ID with opposite values
stored a random result. The PATCH now rejects every aliased key with a
400. A second run on the stack's top head at the time, `718dd6d1bd1`,
passed: every aliased form returns 400 and stores nothing, and chats
leave out a skill the member turned off, including in `read_skill`.
- Polish and the test audit stayed inside the implementation diff. No
new dependencies.
- Rebased on main as one commit. The migration is now 000636, after
main's 000634 and 000635. Every organization skill load passes the chat
owner, including main's skill invocation path, so `/skill` invocations
honor the opt-out too. The model's skill index leaves out a skill that
is not model-invocable or that the member turned off. Apart from that,
the diff is unchanged.

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-29 12:53:50 +00:00

175 lines
6.0 KiB
Go

package chatd
import (
"context"
"errors"
"fmt"
"slices"
"sync"
"github.com/google/uuid"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbauthz"
"github.com/coder/coder/v2/coderd/rbac"
"github.com/coder/coder/v2/coderd/x/chatd/chattool"
skillspkg "github.com/coder/coder/v2/coderd/x/skills"
"github.com/coder/coder/v2/codersdk"
)
// MaxSkillInvocationsPerMessage bounds the skill-invocation parts in one
// message, and with them the skill bodies a message stores.
const MaxSkillInvocationsPerMessage = 8
// ErrTooManySkillInvocations reports a message with more than
// MaxSkillInvocationsPerMessage skill-invocation parts.
var ErrTooManySkillInvocations = xerrors.Errorf("a message can invoke at most %d skills", MaxSkillInvocationsPerMessage)
// SkillInvocationUnavailableError reports a skill-invocation part whose
// skill the chat owner cannot use: it is missing, disabled, or not readable.
type SkillInvocationUnavailableError struct {
Source codersdk.ChatSkillSource
Name string
}
func (e *SkillInvocationUnavailableError) Error() string {
return fmt.Sprintf("Skill %s/%s is not available.", e.Source, e.Name)
}
// resolveChatSkillInvocations loads the chat and resolves the
// skill-invocation parts of a message sent to it. The chat is only read
// when the message invokes a skill.
func (p *Server) resolveChatSkillInvocations(
ctx context.Context,
chatID uuid.UUID,
parts []codersdk.ChatMessagePart,
) ([]codersdk.ChatMessagePart, error) {
if !slices.ContainsFunc(parts, isSkillInvocation) {
return parts, nil
}
chat, err := p.db.GetChatByID(ctx, chatID)
if err != nil {
return nil, xerrors.Errorf("load chat for skill invocations: %w", err)
}
return p.resolveSkillInvocations(ctx, chat, parts)
}
// resolveSkillInvocations stores the current body of every invoked skill on
// its part, read with the chat owner's permissions through the same loaders
// as read_skill. Skills load by source and name, so an invocation does not
// depend on the skill being listed in the turn's skill index, and skills
// that are not model-invocable still load. Workspace
// skills come from the chat's pinned context snapshot, so a chat without
// one cannot invoke them. An invoked workspace skill also stores its
// directory: read_skill refuses a skill hidden from the model, so the
// invocation is the only place the model is told where its scripts are.
// The directory is stored with the chat's workspace, because the path
// means nothing once the chat is bound to another workspace.
func (p *Server) resolveSkillInvocations(
ctx context.Context,
chat database.Chat,
parts []codersdk.ChatMessagePart,
) ([]codersdk.ChatMessagePart, error) {
invocations := countSkillInvocations(parts)
if invocations == 0 {
return parts, nil
}
if invocations > MaxSkillInvocationsPerMessage {
return nil, ErrTooManySkillInvocations
}
loadOwner := sync.OnceValues(func() (rbac.Subject, error) {
return p.chatOwnerSubject(ctx, chat.OwnerID)
})
loadWorkspaceSkills := sync.OnceValues(func() ([]chattool.SkillMeta, error) {
_, skills, err := p.resolveTurnWorkspaceContext(ctx, chat, database.WorkspaceAgent{})
return skills, err
})
type invokedSkill struct {
body string
dir string
workspaceID uuid.UUID
}
loadSkill := func(part codersdk.ChatMessagePart) (invokedSkill, error) {
switch part.SkillSource {
case codersdk.ChatSkillSourcePersonal:
parsed, err := p.loadPersonalSkillBody(ctx, chat.OwnerID, part.SkillName, skillLoaderUser)
return invokedSkill{body: parsed.Body}, err
case codersdk.ChatSkillSourceOrganization:
owner, err := loadOwner()
if err != nil {
return invokedSkill{}, err
}
parsed, err := p.loadOrganizationSkillBody(ctx, owner, chat.OwnerID, chat.OrganizationID, part.SkillName, skillLoaderUser)
return invokedSkill{body: parsed.Body}, err
case codersdk.ChatSkillSourceProject:
// Read as chatd behind the chat owner's project usability check,
// like project skill loading for turns.
//nolint:gocritic // Access is gated on ChatProjectUsableBy for the chat owner.
chatdCtx := dbauthz.AsChatd(ctx)
projectID := p.resolveUsableChatProjectID(chatdCtx, p.logger, chat)
if !projectID.Valid {
return invokedSkill{}, skillspkg.ErrSkillNotFound
}
parsed, err := p.loadProjectSkillBody(chatdCtx, projectID.UUID, part.SkillName, skillLoaderUser)
return invokedSkill{body: parsed.Body}, err
case codersdk.ChatSkillSourceWorkspace:
skills, err := loadWorkspaceSkills()
if err != nil {
return invokedSkill{}, err
}
for _, skill := range skills {
if skill.Name == part.SkillName {
body, err := chattool.PinnedWorkspaceSkillBody(skill)
return invokedSkill{body: body, dir: skill.Dir, workspaceID: chat.WorkspaceID.UUID}, err
}
}
}
return invokedSkill{}, skillspkg.ErrSkillNotFound
}
// A message that repeats a skill loads it once.
type skillKey struct {
source codersdk.ChatSkillSource
name string
}
loaded := make(map[skillKey]invokedSkill, invocations)
resolved := slices.Clone(parts)
for i, part := range resolved {
if !isSkillInvocation(part) {
continue
}
key := skillKey{source: part.SkillSource, name: part.SkillName}
skill, ok := loaded[key]
if !ok {
var err error
skill, err = loadSkill(part)
if errors.Is(err, skillspkg.ErrSkillNotFound) {
return nil, &SkillInvocationUnavailableError{Source: part.SkillSource, Name: part.SkillName}
}
if err != nil {
return nil, xerrors.Errorf("load invoked skill %s/%s: %w", part.SkillSource, part.SkillName, err)
}
loaded[key] = skill
}
resolved[i].SkillBody = skill.body
resolved[i].SkillDir = skill.dir
resolved[i].SkillWorkspaceID = skill.workspaceID
}
return resolved, nil
}
func countSkillInvocations(parts []codersdk.ChatMessagePart) int {
count := 0
for _, part := range parts {
if isSkillInvocation(part) {
count++
}
}
return count
}
func isSkillInvocation(part codersdk.ChatMessagePart) bool {
return part.Type == codersdk.ChatMessagePartTypeSkillInvocation
}