Files
coder/coderd/x/chatd/chattool/skill.go
T
Michael Suchacz 0a71846385 refactor(coderd/x/chatd/chattool): drop the read_skill alias fallback (#31123)
When `ReadSkillOptions.ResolveAlias` is nil, `read_skill` and
`read_skill_file` build a workspace skill from `GetSkills` and serve it.
Only tests leave the resolver nil. The one production caller,
`skillOpts` in `turn_environment.go`, always sets it to
`skillspkg.Lookup` over the merged personal, organization, project, and
workspace skills.

This PR deletes `resolveSkillAlias` and that fallback. Without a
resolver, both tools now return `skill alias resolver is not
configured`, as they already do for the other unset options. The tests
resolve names with the real `skillspkg.MergeSkills` and
`skillspkg.Lookup` instead of hand-written fakes. `read_skill_file`
still serves a workspace skill hidden from the model (#31147): it checks
for a nil resolver before `resolveSkillFileTarget`, which now calls
`ResolveAlias` directly. The PR adds 89 lines and removes 160.

## Why it is safe

- `git grep 'ReadSkillOptions{'` finds one non-test caller, and it sets
`ResolveAlias`. Behavior with a resolver does not change.
- `GetSkills` and `findSkill` stay. They still supply the pinned body,
the directory, and the supporting files of a resolved workspace skill.
- `TestReadSkillTool/ModelHiddenSkill` is deleted. It checked that the
fallback hid a skill that disables model invocation. With the real
resolver, `TestModelHiddenWorkspaceSkillsLeftOutOfModelSkills` checks
that, and it fails when the model-invocable filter in
`workspaceSkillsForResolution` is removed.
`ListedSkillServesFileThatDisablesModelInvocation` now sets a resolver.

Mutants of `skill.go` (`go test -overlay`, `-run TestReadSkill`), on
`main` at 27db7ace68 and on this branch:

| Mutant | Fails on `main` | Fails on this branch |
|---|---|---|
| Fallback restored (`skill.go` from `main`) | n/a |
`TestReadSkillToolsRequireAliasResolver` |
| No nil check in `ReadSkill`, or in `ReadSkillFile` | n/a |
`TestReadSkillToolsRequireAliasResolver` (nil function call) |
| Response `name` is `resolved.Alias` | nothing |
`PersonalQualifiedAliasPreservesAlias`, `WorkspaceQualifiedAlias` |
| Pinned body empty | `PinnedBodyFromMeta`,
`PinnedBodyServedWhenWorkspaceUnreachable`, `WorkspaceQualifiedAlias` |
the same, and `CollisionAliasRoundTrip` |
| 10 others: no `dir`, no supporting files, empty file content,
ambiguity reported as not found, resolver or loader error text returned,
deleted stored skill reported as a load failure, file read allowed for a
personal skill, hidden skill not matched, hidden skill meta file served
| at least one test each | the same tests |

A copy of `skill.go` with a syntax error fails to build in both runs, so
the overlay applies.

## Review focus

- The qualified alias tests are stronger. The fakes returned the alias
they got, so on `main` a tool that echoed `resolved.Alias` passed. The
real lookup resolves `personal/my-skill` to the bare alias `my-skill`,
so the tests now check that the tool returns the requested name.
- `CollisionAliasRoundTrip` no longer calls the resolver itself. It
checks each response body instead. The fake said that a bare `deploy`
was not found; the real lookup says it is ambiguous, which
`AmbiguousLookupSurfacesAliases` covers.
- `ResolveAliasErrorIsSanitized` keeps a fake, because the real lookup
returns only not-found and ambiguous errors.
- #29323 (plugin skills) replaces `ResolveAlias` and rewrites
`resolveSkillAlias`, so it conflicts with this PR. Whichever merges
second must reconcile.

## Validation

- `go test ./coderd/x/chatd/chattool/ -count=1`: pass.
- `go test ./coderd/x/chatd/chattool/ -race -count=3 -run Skill`: pass.
- `go test ./coderd/x/chatd/ -run Skill`: pass.
- `go vet ./coderd/x/chatd/chattool/ ./coderd/x/chatd/`: no issues.
- Mutants: each fails as listed.
- `make pre-commit` with `CODER_HOOK_RUN_ALL=1` passed.

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-30 21:52:43 +01:00

606 lines
18 KiB
Go

package chattool
import (
"context"
"errors"
"fmt"
"io"
"path"
"strings"
"unicode/utf8"
"charm.land/fantasy"
"golang.org/x/xerrors"
skillspkg "github.com/coder/coder/v2/coderd/x/skills"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/codersdk/workspacesdk"
)
const (
maxSkillMetaBytes = workspacesdk.MaxSkillMetaBytes
maxSkillFileBytes = 512 * 1024
// AvailableSkillsOpenTag is the XML start tag for the skill index block.
AvailableSkillsOpenTag = "<available-skills>"
// AvailableSkillsCloseTag is the XML end tag for the skill index block.
AvailableSkillsCloseTag = "</available-skills>"
// maxSkillIndexDescriptionRunes is the Agent Skills specification limit
// for a description. Longer stored descriptions are only cut in the index
// so it stays bounded; read_skill still returns the full skill.
maxSkillIndexDescriptionRunes = 1024
)
// SkillMeta is the frontmatter from a skill meta file discovered in a
// workspace. It carries just enough information to list the skill
// in the prompt index without reading the full body.
type SkillMeta struct {
Name string
Description string
// Dir is the absolute path to the skill directory inside
// the workspace filesystem.
Dir string
// Meta is the verbatim skill meta file (SKILL.md) content the
// agent pushed in the workspace context snapshot: front-matter
// plus body. When present, read_skill serves the body from it
// instead of reading the file back over the workspace
// connection, so a pinned chat keeps returning the same
// instructions even when the workspace is unreachable. It is
// empty on the legacy per-turn discovery path, where the body is
// read live.
Meta []byte
}
// ModelInvocable reports whether agents may list and load the skill on
// their own. A skill whose pinned meta file sets
// disable-model-invocation: true stays hidden from them but still loads
// when a person invokes it. A skill without a pinned meta file stays
// model-invocable.
func (s SkillMeta) ModelInvocable() bool {
return !workspacesdk.SkillDisablesModelInvocation(s.pinnedMeta())
}
func (s SkillMeta) pinnedMeta() string {
raw := s.Meta
if int64(len(raw)) > maxSkillMetaBytes {
raw = raw[:maxSkillMetaBytes]
}
return string(raw)
}
// SkillContent is the full body of a skill, loaded on demand
// when the model calls read_skill.
type SkillContent struct {
SkillMeta
// Body is the markdown content after the frontmatter
// delimiters have been stripped.
Body string
// Files lists relative paths of supporting files in the
// skill directory (everything except the skill meta file).
Files []string
}
// FormatResolvedSkillIndex renders an XML block listing all source-aware
// skills. Aliases are the names the model should pass to the skill tools.
func FormatResolvedSkillIndex(resolved []skillspkg.ResolvedSkill) string {
if len(resolved) == 0 {
return ""
}
entries := make([]skillIndexEntry, 0, len(resolved))
hasQualifiedAlias := false
hasWorkspaceSkill := false
for _, s := range resolved {
entries = append(entries, skillIndexEntry{
Alias: s.Alias,
Description: truncateSkillIndexDescription(s.Description),
})
if s.Source == skillspkg.SourceWorkspace {
hasWorkspaceSkill = true
}
if s.Alias == skillspkg.QualifiedAlias(s.Source, s.Name) {
hasQualifiedAlias = true
}
}
return renderSkillIndex(entries, skillIndexFormatOptions{
includeQualifiedAliasInstruction: hasQualifiedAlias,
includeReadSkillFileInstruction: hasWorkspaceSkill,
})
}
func truncateSkillIndexDescription(description string) string {
if utf8.RuneCountInString(description) <= maxSkillIndexDescriptionRunes {
return description
}
runes := 0
for i := range description {
// Reserve one rune for the ellipsis so the result stays within the limit.
if runes == maxSkillIndexDescriptionRunes-1 {
return description[:i] + "…"
}
runes++
}
return description
}
type skillIndexEntry struct {
Alias string
Description string
}
type skillIndexFormatOptions struct {
includeQualifiedAliasInstruction bool
includeReadSkillFileInstruction bool
}
func renderSkillIndex(entries []skillIndexEntry, opts skillIndexFormatOptions) string {
if len(entries) == 0 {
return ""
}
var b strings.Builder
_, _ = b.WriteString(AvailableSkillsOpenTag + "\n")
_, _ = b.WriteString(
"Use read_skill to load a skill's full instructions " +
"before following them.\n",
)
if opts.includeReadSkillFileInstruction {
_, _ = b.WriteString(
"Use read_skill_file to read supporting files " +
"referenced by a workspace skill.\n",
)
}
if opts.includeQualifiedAliasInstruction {
_, _ = b.WriteString(
"When a skill is listed with a source prefix, such as " +
"personal/name, pass that qualified alias to read_skill.\n",
)
}
_, _ = b.WriteString("\n")
for _, s := range entries {
_, _ = b.WriteString("- ")
_, _ = b.WriteString(s.Alias)
if s.Description != "" {
_, _ = b.WriteString(": ")
_, _ = b.WriteString(s.Description)
}
_, _ = b.WriteString("\n")
}
_, _ = b.WriteString(AvailableSkillsCloseTag)
return b.String()
}
// listSkillFiles lists the supporting files in a skill directory,
// excluding the skill meta file itself. Directory entries are
// suffixed with "/" so the model can tell them apart from files.
func listSkillFiles(
ctx context.Context,
conn workspacesdk.AgentConn,
dir, metaFile string,
) ([]string, error) {
lsResp, err := conn.LS(ctx, "", workspacesdk.LSRequest{
Path: []string{dir},
Relativity: workspacesdk.LSRelativityRoot,
})
if err != nil {
return nil, xerrors.Errorf("list skill directory: %w", err)
}
var files []string
for _, entry := range lsResp.Contents {
if entry.Name == metaFile {
continue
}
name := entry.Name
if entry.IsDir {
name += "/"
}
files = append(files, name)
}
return files, nil
}
// loadPinnedWorkspaceSkillContent builds skill content from the
// SKILL.md the agent pushed in the workspace context snapshot. The
// body is parsed from the pinned bytes without dialing the workspace,
// so it keeps working when the workspace is unreachable. The
// supporting-file list is a best-effort live lookup, because the
// snapshot carries only the meta file (per the agent push contract):
// a missing connection or LS failure yields an empty file list rather
// than failing the read.
func loadPinnedWorkspaceSkillContent(
ctx context.Context,
options ReadSkillOptions,
skill SkillMeta,
) (SkillContent, error) {
body, err := PinnedWorkspaceSkillBody(skill)
if err != nil {
return SkillContent{}, err
}
return SkillContent{
SkillMeta: skill,
Body: body,
Files: bestEffortSkillFiles(ctx, options, skill),
}, nil
}
// PinnedWorkspaceSkillBody parses the body of a workspace skill from the
// SKILL.md pinned in the workspace context snapshot.
func PinnedWorkspaceSkillBody(skill SkillMeta) (string, error) {
_, _, body, err := workspacesdk.ParseSkillFrontmatter(skill.pinnedMeta())
if err != nil {
return "", xerrors.Errorf("parse skill frontmatter: %w", err)
}
return body, nil
}
// bestEffortSkillFiles lists a pinned skill's supporting files over the
// workspace connection, returning nil when the connection or listing
// fails so an unreachable workspace never blocks read_skill from
// returning the pinned body.
func bestEffortSkillFiles(
ctx context.Context,
options ReadSkillOptions,
skill SkillMeta,
) []string {
if options.GetWorkspaceConn == nil {
return nil
}
conn, err := options.GetWorkspaceConn(ctx)
if err != nil {
return nil
}
files, err := listSkillFiles(ctx, conn, skill.Dir, DefaultSkillMetaFile)
if err != nil {
return nil
}
return files
}
// LoadSkillFile reads a supporting file from a skill's directory.
// The relativePath is validated to prevent directory traversal and
// access to hidden files, and the workspace agent refuses a path that
// leaves the skill directory through a symlink.
func LoadSkillFile(
ctx context.Context,
conn workspacesdk.AgentConn,
skill SkillMeta,
relativePath string,
) (string, error) {
if err := validateSkillFilePath(relativePath); err != nil {
return "", err
}
reader, _, err := conn.ReadFileInRoot(
ctx, skill.Dir, relativePath, 0, maxSkillFileBytes+1,
)
if err != nil {
return "", skillFileReadError(err)
}
raw, err := io.ReadAll(io.LimitReader(reader, maxSkillFileBytes+1))
reader.Close()
if err != nil {
return "", xerrors.Errorf(
"read skill file bytes: %w", err,
)
}
if int64(len(raw)) > maxSkillFileBytes {
raw = raw[:maxSkillFileBytes]
}
return string(raw), nil
}
// skillFileReadError rewrites an agent error for the model. It omits
// the request URL that codersdk.Error includes and explains the
// validation error that agents without the root parameter return.
func skillFileReadError(err error) error {
var sdkErr *codersdk.Error
if !errors.As(err, &sdkErr) {
return xerrors.Errorf("read skill file: %w", err)
}
for _, v := range sdkErr.Validations {
if v.Field == "root" {
return xerrors.New("read skill file: the workspace agent is outdated; restart the workspace to read skill files")
}
}
return xerrors.Errorf("read skill file: %s", sdkErr.Message)
}
// validateSkillFilePath rejects paths that could escape the skill
// directory or access hidden files. Only forward-relative,
// non-hidden paths are allowed.
func validateSkillFilePath(p string) error {
if p == "" {
return xerrors.New("path is required")
}
if strings.HasPrefix(p, "/") {
return xerrors.New(
"absolute paths are not allowed",
)
}
for _, component := range strings.Split(p, "/") {
if component == ".." {
return xerrors.New(
"path traversal is not allowed",
)
}
if strings.HasPrefix(component, ".") {
return xerrors.New(
"hidden file components are not allowed",
)
}
}
return nil
}
// DefaultSkillMetaFile is the fallback skill meta file name used
// when loading skill bodies on demand from older agents.
const DefaultSkillMetaFile = "SKILL.md"
// ReadSkillOptions configures the read_skill and read_skill_file
// tools.
type ReadSkillOptions struct {
GetWorkspaceConn func(context.Context) (workspacesdk.AgentConn, error)
GetSkills func() []SkillMeta
ResolveAlias func(string) (skillspkg.ResolvedSkill, error)
// LoadStoredSkillBody loads the current body of a personal,
// organization, or project skill. It returns skillspkg.ErrSkillNotFound
// when the skill is gone, disabled, or no longer readable by the chat
// owner.
LoadStoredSkillBody func(context.Context, skillspkg.Skill) (skillspkg.ParsedSkill, error)
}
// ReadSkillArgs are the parameters accepted by read_skill.
type ReadSkillArgs struct {
Name string `json:"name" description:"The name or qualified alias of the skill to read."`
}
// ReadSkill returns an AgentTool that reads the full instructions
// for a skill by name. The model should call this before
// following any skill's instructions.
func ReadSkill(options ReadSkillOptions) fantasy.AgentTool {
return fantasy.NewAgentTool(
"read_skill",
"Read the full instructions for a skill by name. "+
"Returns the skill meta file body and a list of "+
"supporting files. Use read_skill before "+
"following a skill's instructions.",
func(ctx context.Context, args ReadSkillArgs, _ fantasy.ToolCall) (fantasy.ToolResponse, error) {
if args.Name == "" {
return fantasy.NewTextErrorResponse(
"name is required",
), nil
}
if options.ResolveAlias == nil {
return fantasy.NewTextErrorResponse(
"skill alias resolver is not configured",
), nil
}
resolved, err := options.ResolveAlias(args.Name)
if err != nil {
return skillResolveErrorResponse(args.Name, err), nil
}
switch resolved.Source {
case skillspkg.SourcePersonal, skillspkg.SourceOrganization, skillspkg.SourceProject:
if options.LoadStoredSkillBody == nil {
return fantasy.NewTextErrorResponse(
"stored skill loader is not configured",
), nil
}
content, err := options.LoadStoredSkillBody(ctx, resolved.Skill)
if err != nil {
if xerrors.Is(err, skillspkg.ErrSkillNotFound) {
return skillNotFoundResponse(args.Name), nil
}
return fantasy.NewTextErrorResponse(
fmt.Sprintf("failed to load %s skill %q", resolved.Source, args.Name),
), nil
}
return toolResponse(map[string]any{
"name": args.Name,
"body": content.Body,
"files": []string{},
}), nil
case skillspkg.SourceWorkspace:
content, response, ok := readWorkspaceSkillBody(ctx, options, args.Name, resolved.Name)
if ok {
return response, nil
}
// Include the absolute skill directory so the agent can
// reach supporting files with read_file and execute.
return toolResponse(map[string]any{
"name": args.Name,
"dir": content.Dir,
"body": content.Body,
"files": nonNilFiles(content.Files),
}), nil
default:
return skillNotFoundResponse(args.Name), nil
}
},
)
}
// ReadSkillFileArgs are the parameters accepted by
// read_skill_file.
type ReadSkillFileArgs struct {
Name string `json:"name" description:"The name or qualified alias of the skill to read."`
Path string `json:"path" description:"Relative path to a file in the skill directory (e.g. roles/security-reviewer.md)."`
}
// ReadSkillFile returns an AgentTool that reads a supporting file
// from a skill's directory.
func ReadSkillFile(options ReadSkillOptions) fantasy.AgentTool {
return fantasy.NewAgentTool(
"read_skill_file",
"Read a supporting file from a skill's directory "+
"(e.g. roles/security-reviewer.md).",
func(ctx context.Context, args ReadSkillFileArgs, _ fantasy.ToolCall) (fantasy.ToolResponse, error) {
if args.Name == "" {
return fantasy.NewTextErrorResponse(
"name is required",
), nil
}
if args.Path == "" {
return fantasy.NewTextErrorResponse(
"path is required",
), nil
}
if options.ResolveAlias == nil {
return fantasy.NewTextErrorResponse(
"skill alias resolver is not configured",
), nil
}
skill, response, ok := resolveSkillFileTarget(options, args.Name)
if !ok {
return response, nil
}
// Only a person's invocation loads the body of a skill hidden
// from the model, so its meta file is not served here.
if !skill.ModelInvocable() && strings.EqualFold(path.Clean(args.Path), DefaultSkillMetaFile) {
return hiddenSkillMetaFileResponse(args.Name, args.Path), nil
}
// Validate the path early so we reject bad
// inputs before dialing the workspace agent.
if err := validateSkillFilePath(args.Path); err != nil {
return fantasy.NewTextErrorResponse(
err.Error(),
), nil
}
if options.GetWorkspaceConn == nil {
return fantasy.NewTextErrorResponse(
"workspace connection resolver is not configured",
), nil
}
conn, err := options.GetWorkspaceConn(ctx)
if err != nil {
return fantasy.NewTextErrorResponse(
err.Error(),
), nil
}
content, err := LoadSkillFile(
ctx, conn, skill, args.Path,
)
if err != nil {
return fantasy.NewTextErrorResponse(
err.Error(),
), nil
}
return toolResponse(map[string]any{
"content": content,
}), nil
},
)
}
func hiddenSkillMetaFileResponse(name, filePath string) fantasy.ToolResponse {
return fantasy.NewTextErrorResponse(fmt.Sprintf(
"%s of skill %q is the meta file of a skill hidden from agents and cannot be read with read_skill_file",
filePath, name,
))
}
// resolveSkillFileTarget finds the workspace skill whose supporting files
// read_skill_file serves. Besides the skills listed for the model, it
// accepts a workspace skill hidden from the model by bare or qualified
// name: a person can invoke such a skill, and its body may reference
// supporting files.
func resolveSkillFileTarget(options ReadSkillOptions, name string) (SkillMeta, fantasy.ToolResponse, bool) {
// Stored skills have no supporting files, so a name that names a
// hidden workspace skill means that skill, even when the bare name
// also names one or more listed stored skills.
bareName := strings.TrimPrefix(name, skillspkg.QualifiedAlias(skillspkg.SourceWorkspace, ""))
if skill, ok := findSkill(options.GetSkills, bareName); ok && !skill.ModelInvocable() {
return skill, fantasy.ToolResponse{}, true
}
resolved, err := options.ResolveAlias(name)
if err != nil {
return SkillMeta{}, skillResolveErrorResponse(name, err), false
}
if resolved.Source != skillspkg.SourceWorkspace {
return SkillMeta{}, fantasy.NewTextErrorResponse(fmt.Sprintf(
"read_skill_file is not supported for %s skills (no supporting files)",
resolved.Source,
)), false
}
skill, ok := findSkill(options.GetSkills, resolved.Name)
if !ok {
return SkillMeta{}, skillNotFoundResponse(name), false
}
return skill, fantasy.ToolResponse{}, true
}
func readWorkspaceSkillBody(
ctx context.Context,
options ReadSkillOptions,
requestedName string,
canonicalName string,
) (SkillContent, fantasy.ToolResponse, bool) {
skill, ok := findSkill(options.GetSkills, canonicalName)
if !ok {
return SkillContent{}, skillNotFoundResponse(requestedName), true
}
// The SKILL.md body travels in the workspace context snapshot, so it
// is served from the pin without dialing the workspace. The supporting
// file list is still a best-effort live lookup; see
// loadPinnedWorkspaceSkillContent.
content, err := loadPinnedWorkspaceSkillContent(ctx, options, skill)
if err != nil {
return SkillContent{}, fantasy.NewTextErrorResponse(err.Error()), true
}
return content, fantasy.ToolResponse{}, false
}
func skillResolveErrorResponse(name string, err error) fantasy.ToolResponse {
if xerrors.Is(err, skillspkg.ErrSkillNotFound) {
return skillNotFoundResponse(name)
}
if xerrors.Is(err, skillspkg.ErrSkillAmbiguous) {
return fantasy.NewTextErrorResponse(err.Error())
}
return fantasy.NewTextErrorResponse(
fmt.Sprintf("failed to resolve skill %q", name),
)
}
func skillNotFoundResponse(name string) fantasy.ToolResponse {
return fantasy.NewTextErrorResponse(
fmt.Sprintf("skill %q not found", name),
)
}
func nonNilFiles(files []string) []string {
if files == nil {
return []string{}
}
return files
}
// findSkill looks up a skill by name in the current skill list.
func findSkill(
getSkills func() []SkillMeta,
name string,
) (SkillMeta, bool) {
if getSkills == nil {
return SkillMeta{}, false
}
for _, s := range getSkills() {
if s.Name == name {
return s, true
}
}
return SkillMeta{}, false
}