Files
Garrett DelfosseandClaude Opus 5.5 a08ba6a70c feat: add shared project memory API for chats in a project (#29301)
- Experiment, stacked on #29242. UI is in #29583.
- Adds **project memory**: `chat_project_memories` rows with a name
(lowercase slug, unique per project), a one-line description (150
characters) and a body (8KB). Up to 200 per project, shared by every
root chat in the project. Chats outside a project and subagents have no
memory.
- Memories are immutable: they are created and deleted, never edited.
Agent tools are `read_memory`, `save_memory`, `delete_memory` and
`consolidate_memory`. Saving an existing name fails and points the agent
at `consolidate_memory` to replace it.
- **Consolidation is agent-driven, in the turn**, modeled on how Claude
Code compacts its memory index when warned it is near its limit. From
160 memories (80%), save results ask the agent to get under 140 (70%)
with `consolidate_memory`, choosing what to merge and drop. That tool
deletes and saves memories in one transaction under the per-project
advisory lock, and rolls back entirely if a delete is missing, a name is
taken, or the result exceeds 200. There is no background extraction or
consolidation.
- **The memory index reaches the model as model-only conversation
messages**, not in a tool description or the system prompt. At turn
start the chat gets a full `<project-memory-index>` snapshot when its
prompt has none (first turn, or after compaction), otherwise a
`<project-memory-index-update>` listing only what changed. Tool
definitions and the system prompt carry no memory state, so memory
writes never invalidate the provider's cached prompt prefix. Writes by
other chats, or by this chat mid-turn, reach the model at its next turn.
- REST API (create, list, get, delete) under
`/api/experimental/organizations/{organization}/chats/projects/{project}/memories`.
Memory permissions follow projects onto the `agents-access` role.
- **Every memory change is audited** as `ChatProjectMemory` create or
delete. REST changes are attributed to the caller. Tool changes
(`save_memory`, `delete_memory`, `consolidate_memory`) are attributed to
the chat owner, carry the `chat_id`, and record one entry per memory
after the change commits, so a rolled-back consolidation records
nothing.

### Examples

**What the model sees.** The system prompt gets a fixed guidance block
with no memory state:

```text
<memory>
Memory is durable context shared by every chat in the project "platform".
Save facts that will matter in future chats: … The memory index is in the conversation: a <project-memory-index> message lists every saved memory, and <project-memory-index-update> messages at the start of later turns list what changed. Changes made during a turn, including your own, appear in the next turn's update. … Memories cannot be edited: replace one that no longer holds with consolidate_memory, or delete it.
</memory>
```

On the first turn, after the user's message, the chat gets a model-only
row with the full index (hidden in the UI):

```text
<project-memory-index>
- deploy-day: Deploys happen on Tuesdays
- release-owner: Alice owns releases; ping her before tagging
</project-memory-index>
```

On a later turn where memories changed, it gets only the difference:

```text
<project-memory-index-update>
Added or changed:
- deploy-day: Deploys happen on Wednesdays (changed 2026-09-30)
- staging-url: Staging lives at staging.example.com
Removed:
- old-ci-notes
</project-memory-index-update>
```

Resulting request layout, with the provider's cached prefix unchanged by
memory writes:

```text
[tools: fixed]  [system: fixed <memory> guidance]  [turn 1 user] [index snapshot] [assistant/tools …]  [turn 2 user] [index update] [assistant …]
```

**Tool calls and results** (produced by the tools in this PR):

```text
save_memory {"name":"staging-url","description":"Staging lives at staging.example.com","body":"Staging is https://staging.example.com; deploys from main."}
→ {"count":3,"saved":"staging-url"}

save_memory {"name":"deploy-day", …}   // name already exists
→ error: a memory named "deploy-day" already exists; read it, then replace it with consolidate_memory by deleting "deploy-day" and saving the new version in one call

consolidate_memory {"delete":["deploy-day"],"save":[{"name":"deploy-day","description":"Deploys happen on Wednesdays (changed 2026-09-30)","body":"The team deploys every Wednesday since 2026-09-30."}]}
→ {"count":3,"deleted":["deploy-day"],"saved":["deploy-day"]}

read_memory {"name":"deploy-day"}
→ {"name":"deploy-day","description":"…","body":"…","created_at":"2026-09-30T15:04:05Z","created_by":"alice"}
```

Near and at the cap:

```text
save_memory … → {"count":160,"saved":"one-more","warning":"memory is 160/200, approaching the limit. Consolidate it to under 140 memories now with consolidate_memory: merge overlapping memories into one, and drop stale, wrong, or superseded ones."}

save_memory … → error: memory is full (200/200); use consolidate_memory to merge overlapping memories and drop stale ones, and include this memory in its save list
```

**REST memory object** (`GET .../chats/projects/{project}/memories`):

```json
{
  "id": "59500000-0000-4000-8000-000000000001",
  "project_id": "7c0e4a3e-5b1f-4f7d-9a51-2f3c8f6d2b10",
  "organization_id": "bb640d07-ca8a-4869-b6bc-ae61ebb2fda1",
  "name": "deploy-day",
  "description": "Deploys happen on Wednesdays (changed 2026-09-30)",
  "body": "The team deploys every Wednesday since 2026-09-30.",
  "created_by": "0ed9befc-4911-4ccf-a8e2-559bf72daa94",
  "created_by_username": "alice",
  "created_at": "2026-09-30T15:04:05Z"
}
```

Create takes `{"name", "description", "body"}` and answers 409 on a
duplicate name or at the cap.

<details>
<summary>Screenshots (predate the immutable-memory rework; tool output
shape has since changed)</summary>


![save_memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-01-chat-saves-memory-raw.png)

![read_memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-02-chat-reads-memory-raw.png)
![Agent reads every
memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-agent-lists-memories-raw.png)

</details>

---
Generated with Coder Agents on behalf of @f0ssel.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:52:10 -04:00

364 lines
14 KiB
Go

package chattool_test
import (
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"charm.land/fantasy"
"github.com/google/uuid"
"github.com/stretchr/testify/require"
"golang.org/x/xerrors"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbgen"
"github.com/coder/coder/v2/coderd/database/dbtestutil"
"github.com/coder/coder/v2/coderd/x/chatd/chattool"
"github.com/coder/coder/v2/testutil"
)
// memoryStore is an in-memory MemoryStore with the same all-or-nothing and
// cap semantics as the project store.
type memoryStore struct {
memories map[string]chattool.Memory
// getErr, when set, is returned by Get.
getErr error
}
func newMemoryStore(n int) *memoryStore {
store := &memoryStore{memories: make(map[string]chattool.Memory, n)}
for i := range n {
name := fmt.Sprintf("seeded-%03d", i)
store.memories[name] = chattool.Memory{Name: name}
}
return store
}
func (s *memoryStore) Get(_ context.Context, name string) (chattool.Memory, error) {
if s.getErr != nil {
return chattool.Memory{}, s.getErr
}
memory, ok := s.memories[name]
if !ok {
return chattool.Memory{}, chattool.ErrMemoryNotFound
}
return memory, nil
}
func (*memoryStore) List(context.Context) ([]chattool.MemoryIndexEntry, error) { return nil, nil }
func (s *memoryStore) Insert(ctx context.Context, input chattool.MemoryInput) (int64, error) {
return s.Consolidate(ctx, nil, []chattool.MemoryInput{input})
}
func (s *memoryStore) Delete(ctx context.Context, name string) error {
_, err := s.Consolidate(ctx, []string{name}, nil)
return err
}
func (s *memoryStore) Consolidate(_ context.Context, deleteNames []string, saves []chattool.MemoryInput) (int64, error) {
next := make(map[string]chattool.Memory, len(s.memories))
for name, memory := range s.memories {
next[name] = memory
}
for _, name := range deleteNames {
if _, ok := next[name]; !ok {
return 0, chattool.ErrMemoryNotFound
}
delete(next, name)
}
for _, input := range saves {
if _, ok := next[input.Name]; ok {
return 0, chattool.ErrMemoryExists
}
next[input.Name] = chattool.Memory{Name: input.Name, Description: input.Description, Body: input.Body}
}
if len(next) > chattool.MaxMemories {
return 0, chattool.ErrMemoryLimit
}
s.memories = next
return int64(len(next)), nil
}
func runMemoryTool(t *testing.T, tool fantasy.AgentTool, input string) (fantasy.ToolResponse, map[string]any) {
t.Helper()
response, err := tool.Run(context.Background(), fantasy.ToolCall{Input: input})
require.NoError(t, err)
var result map[string]any
if !response.IsError {
require.NoError(t, json.Unmarshal([]byte(response.Content), &result))
}
return response, result
}
func TestNormalizeMemoryInput(t *testing.T) {
t.Parallel()
for _, name := range []string{"Release Notes", "-starts-with-dash", ""} {
_, err := chattool.NormalizeMemoryName(name)
require.Error(t, err, name)
}
name, err := chattool.NormalizeMemoryName(" Release_Notes-2026 ")
require.NoError(t, err)
require.Equal(t, "release_notes-2026", name)
input, err := chattool.NormalizeMemoryInput("fact", "first line\n- forged: index entry\u200b", "body\u200b\r\n")
require.NoError(t, err)
require.Equal(t, "first line - forged: index entry", input.Description, "descriptions are one index line")
require.Equal(t, "body", input.Body)
_, err = chattool.NormalizeMemoryInput("fact", strings.Repeat("d", chattool.MaxMemoryDescriptionChars+1), "body")
require.ErrorContains(t, err, "description must be at most")
_, err = chattool.NormalizeMemoryInput("fact", "description", strings.Repeat("b", chattool.MaxMemoryBodyBytes+1))
require.ErrorContains(t, err, "body must be at most")
_, err = chattool.NormalizeMemoryInput("fact", "\u200b", "body")
require.ErrorContains(t, err, "description is required")
}
func TestFormatMemoryGuidance(t *testing.T) {
t.Parallel()
guidance := chattool.FormatMemoryGuidance("platform")
require.Contains(t, guidance, "<memory>")
require.Contains(t, guidance, `project "platform"`)
require.Contains(t, guidance, "people on this project")
require.Contains(t, guidance, "<project-memory-index>")
require.Contains(t, guidance, chattool.ConsolidateMemoryToolName)
}
func TestReadMemoryDescriptionIsFixed(t *testing.T) {
t.Parallel()
// Tool definitions lead the provider's cached prefix, so memory state
// must never reach them.
a := chattool.ReadMemory(newMemoryStore(0)).Info().Description
b := chattool.ReadMemory(newMemoryStore(chattool.MaxMemories)).Info().Description
require.Equal(t, a, b)
}
func TestReadMemory(t *testing.T) {
t.Parallel()
store := newMemoryStore(1)
_, result := runMemoryTool(t, chattool.ReadMemory(store), `{"name": "seeded-000"}`)
require.Equal(t, "seeded-000", result["name"])
response, _ := runMemoryTool(t, chattool.ReadMemory(store), `{"name": "missing"}`)
require.True(t, response.IsError)
require.Equal(t, "memory was not found", response.Content)
// Other failures must not tell the model the memory is gone.
store.getErr = xerrors.New("connection reset")
response, _ = runMemoryTool(t, chattool.ReadMemory(store), `{"name": "seeded-000"}`)
require.True(t, response.IsError)
require.Equal(t, "failed to read memory", response.Content)
}
func TestMemoryIndexReplay(t *testing.T) {
t.Parallel()
entry := func(name, description string) chattool.MemoryIndexEntry {
return chattool.MemoryIndexEntry{Name: name, Description: description}
}
_, ok := chattool.ReplayMemoryIndex([]string{"unrelated", chattool.FormatMemoryIndexUpdate([]chattool.MemoryIndexEntry{entry("a", "A")}, nil)})
require.False(t, ok, "an update without a snapshot has no base")
snapshot := chattool.FormatMemoryIndexSnapshot([]chattool.MemoryIndexEntry{entry("alpha", "First: with a colon"), entry("beta", "Second")})
seen, ok := chattool.ReplayMemoryIndex([]string{snapshot})
require.True(t, ok)
require.Equal(t, map[string]string{"alpha": "First: with a colon", "beta": "Second"}, seen)
current := []chattool.MemoryIndexEntry{entry("alpha", "First, corrected"), entry("gamma", "Third")}
changed, removed := chattool.DiffMemoryIndex(seen, current)
require.Equal(t, []chattool.MemoryIndexEntry{entry("alpha", "First, corrected"), entry("gamma", "Third")}, changed)
require.Equal(t, []string{"beta"}, removed)
// Replaying the snapshot and the update reproduces the current index,
// so the next diff is empty.
seen, ok = chattool.ReplayMemoryIndex([]string{snapshot, "unrelated", chattool.FormatMemoryIndexUpdate(changed, removed)})
require.True(t, ok)
changed, removed = chattool.DiffMemoryIndex(seen, current)
require.Empty(t, changed)
require.Empty(t, removed)
// A later snapshot replaces everything before it.
seen, ok = chattool.ReplayMemoryIndex([]string{snapshot, chattool.FormatMemoryIndexSnapshot(nil)})
require.True(t, ok)
require.Empty(t, seen)
}
func TestSaveMemory(t *testing.T) {
t.Parallel()
const input = `{"name":"durable-fact","description":"Durable fact","body":"Body"}`
t.Run("Saves", func(t *testing.T) {
t.Parallel()
store := newMemoryStore(0)
response, result := runMemoryTool(t, chattool.SaveMemory(store, "platform"), `{"name":"DURABLE-fact","description":"Durable","body":"Body"}`)
require.False(t, response.IsError, response.Content)
require.Equal(t, "durable-fact", result["saved"])
require.EqualValues(t, 1, result["count"])
require.NotContains(t, result, "warning")
require.Contains(t, store.memories, "durable-fact")
})
t.Run("ExistingNameIsNotOverwritten", func(t *testing.T) {
t.Parallel()
store := newMemoryStore(0)
store.memories["durable-fact"] = chattool.Memory{Name: "durable-fact", Body: "original"}
response, _ := runMemoryTool(t, chattool.SaveMemory(store, "platform"), input)
require.True(t, response.IsError)
require.Contains(t, response.Content, "already exists")
require.Contains(t, response.Content, chattool.ConsolidateMemoryToolName)
require.Equal(t, "original", store.memories["durable-fact"].Body)
})
t.Run("NearCapAsksToConsolidate", func(t *testing.T) {
t.Parallel()
store := newMemoryStore(chattool.MemoryConsolidateThreshold - 1)
response, result := runMemoryTool(t, chattool.SaveMemory(store, "platform"), input)
require.False(t, response.IsError, response.Content)
require.EqualValues(t, chattool.MemoryConsolidateThreshold, result["count"])
require.Equal(t, fmt.Sprintf(
"memory is %d/%d, approaching the limit. Consolidate it to under %d memories now with %s: merge overlapping memories into one, and drop stale, wrong, or superseded ones.",
chattool.MemoryConsolidateThreshold, chattool.MaxMemories, chattool.MemoryConsolidateTarget, chattool.ConsolidateMemoryToolName,
), result["warning"])
})
t.Run("FullPointsAtConsolidate", func(t *testing.T) {
t.Parallel()
store := newMemoryStore(chattool.MaxMemories)
response, _ := runMemoryTool(t, chattool.SaveMemory(store, "platform"), input)
require.True(t, response.IsError)
require.Contains(t, response.Content, "memory is full (200/200)")
require.Contains(t, response.Content, chattool.ConsolidateMemoryToolName)
require.Len(t, store.memories, chattool.MaxMemories)
})
}
func TestConsolidateMemory(t *testing.T) {
t.Parallel()
t.Run("MergesAndMakesRoom", func(t *testing.T) {
t.Parallel()
store := newMemoryStore(chattool.MaxMemories)
response, result := runMemoryTool(t, chattool.ConsolidateMemory(store, "platform"), `{
"delete": ["seeded-000", "seeded-001", "seeded-002"],
"save": [
{"name": "merged", "description": "Merged fact", "body": "Merged body"},
{"name": "new-fact", "description": "New fact", "body": "New body"}
]
}`)
require.False(t, response.IsError, response.Content)
require.EqualValues(t, chattool.MaxMemories-1, result["count"])
require.Contains(t, result, "warning", "still over the threshold")
require.NotContains(t, store.memories, "seeded-000")
require.Contains(t, store.memories, "merged")
require.Contains(t, store.memories, "new-fact")
})
t.Run("ReplacesByName", func(t *testing.T) {
t.Parallel()
store := newMemoryStore(1)
response, result := runMemoryTool(t, chattool.ConsolidateMemory(store, "platform"), `{
"delete": ["seeded-000"],
"save": [{"name": "seeded-000", "description": "Corrected", "body": "Corrected body"}]
}`)
require.False(t, response.IsError, response.Content)
require.EqualValues(t, 1, result["count"])
require.NotContains(t, result, "warning")
require.Equal(t, "Corrected body", store.memories["seeded-000"].Body)
})
t.Run("InvalidInputAppliesNothing", func(t *testing.T) {
t.Parallel()
for _, input := range []string{
`{}`,
`{"delete": ["Not Valid"]}`,
`{"delete": ["seeded-000"], "save": [{"name": "ok", "description": "", "body": "body"}]}`,
} {
store := newMemoryStore(1)
response, _ := runMemoryTool(t, chattool.ConsolidateMemory(store, "platform"), input)
require.True(t, response.IsError, input)
require.Contains(t, store.memories, "seeded-000", input)
}
})
}
// TestProjectMemoryStoreConsolidateIsAtomic runs against Postgres because the
// guarantee under test is the transaction rollback. It also checks that only
// committed changes are audited, one entry per memory created or deleted.
func TestProjectMemoryStoreConsolidateIsAtomic(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
db, _ := dbtestutil.NewDB(t)
org := dbgen.Organization(t, db, database.Organization{})
user := dbgen.User(t, db, database.User{})
project := dbgen.ChatProject(t, db, database.ChatProject{OrganizationID: org.ID, OwnerID: user.ID})
var audited []string
store := chattool.NewProjectMemoryStore(db, project.ID, org.ID, user.ID, func(_ context.Context, action database.AuditAction, oldMemory, newMemory database.ChatProjectMemory) {
switch action {
case database.AuditActionCreate:
require.Equal(t, uuid.Nil, oldMemory.ID)
audited = append(audited, "create "+newMemory.Name)
case database.AuditActionDelete:
require.Equal(t, uuid.Nil, newMemory.ID)
audited = append(audited, "delete "+oldMemory.Name)
default:
t.Fatalf("unexpected audit action %q", action)
}
})
takeAudited := func() []string {
out := audited
audited = nil
return out
}
names := func() []string {
entries, err := store.List(ctx)
require.NoError(t, err)
out := make([]string, len(entries))
for i, entry := range entries {
out[i] = entry.Name
}
return out
}
input := func(name string) chattool.MemoryInput {
return chattool.MemoryInput{Name: name, Description: name + " description", Body: name + " body"}
}
for _, name := range []string{"alpha", "beta"} {
_, err := store.Insert(ctx, input(name))
require.NoError(t, err)
}
_, err := store.Insert(ctx, input("alpha"))
require.ErrorIs(t, err, chattool.ErrMemoryExists)
require.Equal(t, []string{"create alpha", "create beta"}, takeAudited())
// A missing delete rolls back the deletes and saves before it.
_, err = store.Consolidate(ctx, []string{"alpha", "missing"}, []chattool.MemoryInput{input("gamma")})
require.ErrorIs(t, err, chattool.ErrMemoryNotFound)
require.Equal(t, []string{"alpha", "beta"}, names())
// A duplicate save rolls back the deletes before it.
_, err = store.Consolidate(ctx, []string{"alpha"}, []chattool.MemoryInput{input("gamma"), input("gamma")})
require.ErrorIs(t, err, chattool.ErrMemoryExists)
require.Equal(t, []string{"alpha", "beta"}, names())
require.Empty(t, takeAudited(), "rolled-back consolidations are not audited")
// Deletes run first, so a save may reuse a deleted name.
count, err := store.Consolidate(ctx, []string{"alpha", "beta"}, []chattool.MemoryInput{input("alpha"), input("merged")})
require.NoError(t, err)
require.EqualValues(t, 2, count)
require.Equal(t, []string{"alpha", "merged"}, names())
require.Equal(t, []string{"delete alpha", "delete beta", "create alpha", "create merged"}, takeAudited())
// A result over the cap rolls back.
saves := make([]chattool.MemoryInput, chattool.MaxMemories-1)
for i := range saves {
saves[i] = input(fmt.Sprintf("fill-%03d", i))
}
_, err = store.Consolidate(ctx, nil, saves)
require.ErrorIs(t, err, chattool.ErrMemoryLimit)
require.Equal(t, []string{"alpha", "merged"}, names())
require.Empty(t, takeAudited())
count, err = store.Consolidate(ctx, []string{"merged"}, saves)
require.NoError(t, err)
require.EqualValues(t, chattool.MaxMemories, count)
require.Len(t, takeAudited(), 1+len(saves))
require.NoError(t, store.Delete(ctx, "alpha"))
require.ErrorIs(t, store.Delete(ctx, "alpha"), chattool.ErrMemoryNotFound)
require.Equal(t, []string{"delete alpha"}, takeAudited())
}