## Summary **Batch A of Phase 3** of the H1 → front-matter migration (`DOCS-484`; parent `DOCS-477`). Adds a front-matter `title` to every navigable docs page whose title can be migrated **mechanically**, with no editorial judgment. This is the content step that Phase 1 (renderers prefer front-matter title, `DOCS-482`) and Phase 2 (tooling + generators front-matter-aware, `DOCS-483`) unblocked. Both are merged; coder.com #964/#974 are merged and live. **Rendered no-op.** The renderers already resolve the page title from the manifest and hide the leading body H1 (Phase 1), so no page changes visually. This just moves the title into front matter where Fumadocs and the migrated tooling can read it. ## What's in this batch Dry-run on `main` (464 navigable pages) splits into: | category | count | this PR | |----------|-------|---------| | already has front-matter title (Reference, from Phase 2 generators) | 196 | skipped (idempotent) | | **mechanical** — leading body H1 equals the manifest label | 138 | ✅ add front-matter `title`, drop the duplicate H1 | | **no body H1** — renders under the manifest label only | 4 | ✅ add front-matter `title` only | | **mismatch** — body H1 differs from the manifest label | 126 | ⏭️ deferred (needs an editorial decision, see below) | 142 files changed, all under `docs/`. ## Deliberately out of scope: the 126 mismatches Pages where the body H1 is richer than the short sidebar label (e.g. label **Modules** / H1 *Contributing modules*, label **Install** / H1 *Installing Coder*) need a canonical-title decision, not a script. A few even look like the body H1 is the redundant one (`install/cli.md` and `install/index.md` both carry the H1 *Installing Coder*). These will land in follow-up batches **by nav section** once the policy is set, so each gets real review. ## Verification AI was the primary author of this PR (see disclosure below); per the [AI Contribution Guidelines](https://coder.com/docs/about/contributing/AI_CONTRIBUTING) here is the manual verification. - Every added front-matter block parses as YAML and its `title` round-trips to the manifest label (checked programmatically across all 142 files). - Every removed line is a leading `# H1` that equalled the manifest title; no body prose was reflowed. Front matter is correctly hoisted above pre-existing `<!-- markdownlint-disable -->` comments on the two pages that had them. - `pnpm check-docs` (markdownlint-cli2 + table formatter) passes on the changed set: `Summary: 0 error(s)`. `MD041` stays off (re-enabled in Phase 4); `MD025` is not tripped because the duplicate body H1s are removed. ``` $ pnpm exec markdownlint-cli2 $(git diff --name-only origin/main) Linting: 142 file(s) Summary: 0 error(s) ``` Linear: DOCS-484 > This PR was created with AI assistance (Coder Agents).
225 lines
7.8 KiB
Markdown
225 lines
7.8 KiB
Markdown
---
|
|
title: Data Retention
|
|
---
|
|
|
|
Coder supports configurable retention policies that automatically purge old
|
|
Audit Logs, Connection Logs, Workspace Agent Logs, API keys, and AI Gateway
|
|
records. These policies help manage database growth by removing records older
|
|
than a specified duration.
|
|
|
|
## Overview
|
|
|
|
Large deployments can accumulate significant amounts of data over time.
|
|
Retention policies help you:
|
|
|
|
- **Reduce database size**: Automatically remove old records to free disk space.
|
|
- **Improve performance**: Smaller tables mean faster queries and backups.
|
|
- **Meet compliance requirements**: Configure retention periods that align with
|
|
your organization's data retention policies.
|
|
|
|
> [!NOTE]
|
|
> Retention policies are disabled by default (set to `0`) to preserve existing
|
|
> behavior. The exceptions are API keys and workspace agent logs, which default
|
|
> to 7 days.
|
|
|
|
## Configuration
|
|
|
|
You can configure retention policies using CLI flags, environment variables, or
|
|
a YAML configuration file.
|
|
|
|
### Settings
|
|
|
|
| Setting | CLI Flag | Environment Variable | Default | Description |
|
|
|----------------------|------------------------------------|----------------------------------------|----------------|-----------------------------------------|
|
|
| Audit Logs | `--audit-logs-retention` | `CODER_AUDIT_LOGS_RETENTION` | `0` (disabled) | How long to retain Audit Log entries |
|
|
| Connection Logs | `--connection-logs-retention` | `CODER_CONNECTION_LOGS_RETENTION` | `0` (disabled) | How long to retain Connection Logs |
|
|
| API Keys | `--api-keys-retention` | `CODER_API_KEYS_RETENTION` | `7d` | How long to retain expired API keys |
|
|
| Workspace Agent Logs | `--workspace-agent-logs-retention` | `CODER_WORKSPACE_AGENT_LOGS_RETENTION` | `7d` | How long to retain workspace agent logs |
|
|
| AI Gateway | `--ai-gateway-retention` | `CODER_AI_GATEWAY_RETENTION` | `60d` | How long to retain AI Gateway records |
|
|
|
|
> [!NOTE]
|
|
> AI Gateway retention is configured separately from other retention settings.
|
|
> See [AI Gateway Setup](../../ai-coder/ai-gateway/setup.md#data-retention) for
|
|
> detailed configuration options.
|
|
|
|
### Duration Format
|
|
|
|
Retention durations support days (`d`) and weeks (`w`) in addition to standard
|
|
Go duration units (`h`, `m`, `s`):
|
|
|
|
- `7d` - 7 days
|
|
- `2w` - 2 weeks
|
|
- `30d` - 30 days
|
|
- `90d` - 90 days
|
|
- `365d` - 1 year
|
|
|
|
### CLI Example
|
|
|
|
```sh
|
|
coder server \
|
|
--audit-logs-retention=365d \
|
|
--connection-logs-retention=90d \
|
|
--api-keys-retention=7d \
|
|
--workspace-agent-logs-retention=7d \
|
|
--ai-gateway-retention=60d
|
|
```
|
|
|
|
### Environment Variables Example
|
|
|
|
```sh
|
|
export CODER_AUDIT_LOGS_RETENTION=365d
|
|
export CODER_CONNECTION_LOGS_RETENTION=90d
|
|
export CODER_API_KEYS_RETENTION=7d
|
|
export CODER_WORKSPACE_AGENT_LOGS_RETENTION=7d
|
|
export CODER_AI_GATEWAY_RETENTION=60d
|
|
```
|
|
|
|
### YAML Configuration Example
|
|
|
|
```yaml
|
|
retention:
|
|
audit_logs: 365d
|
|
connection_logs: 90d
|
|
api_keys: 7d
|
|
workspace_agent_logs: 7d
|
|
|
|
ai_gateway:
|
|
retention: 60d
|
|
```
|
|
|
|
## How Retention Works
|
|
|
|
### Background Purge Process
|
|
|
|
Coder runs a background process that periodically deletes old records. The
|
|
purge process:
|
|
|
|
1. Runs approximately every 10 minutes.
|
|
2. Processes records in batches to avoid database lock contention.
|
|
3. Deletes records older than the configured retention period.
|
|
4. Logs the number of deleted records for monitoring.
|
|
|
|
### Effective Retention
|
|
|
|
Each retention setting controls its data type independently:
|
|
|
|
- When set to a non-zero duration, records older than that duration are deleted.
|
|
- When set to `0`, retention is disabled and data is kept indefinitely.
|
|
|
|
### API Keys Special Behavior
|
|
|
|
API key retention only affects **expired** keys. A key is deleted only when:
|
|
|
|
1. The key has expired (past its `expires_at` timestamp).
|
|
2. The key has been expired for longer than the retention period.
|
|
|
|
Setting `--api-keys-retention=7d` deletes keys that expired more than 7 days
|
|
ago. Active keys are never deleted by the retention policy.
|
|
|
|
Keeping expired keys for a short period allows Coder to return a more helpful
|
|
error message when users attempt to use an expired key.
|
|
|
|
### Workspace Agent Logs Behavior
|
|
|
|
Workspace agent logs are deleted based on when the agent last connected, not the
|
|
age of the logs themselves. **Logs from the latest build of each workspace are
|
|
always retained** regardless of when the agent last connected. This ensures you
|
|
can always debug issues with active workspaces.
|
|
|
|
For non-latest builds, logs are deleted if the agent hasn't connected within the
|
|
retention period. Setting `--workspace-agent-logs-retention=7d` deletes logs for
|
|
agents that haven't connected in 7 days (excluding those from the latest build).
|
|
|
|
### AI Gateway Data Behavior
|
|
|
|
AI Gateway retention applies to interception records and all related data,
|
|
including token usage, prompts, and tool invocations. The default of 60 days
|
|
provides a reasonable balance between storage costs and the ability to analyze
|
|
usage patterns.
|
|
|
|
For details on what data is retained, see the
|
|
[AI Gateway Data Retention](../../ai-coder/ai-gateway/setup.md#data-retention)
|
|
documentation.
|
|
|
|
## Best Practices
|
|
|
|
### Recommended Starting Configuration
|
|
|
|
For most deployments, we recommend:
|
|
|
|
```yaml
|
|
retention:
|
|
audit_logs: 365d
|
|
connection_logs: 90d
|
|
api_keys: 7d
|
|
workspace_agent_logs: 7d
|
|
|
|
ai_gateway:
|
|
retention: 60d
|
|
```
|
|
|
|
### Compliance Considerations
|
|
|
|
> [!WARNING]
|
|
> Audit Logs provide critical security and compliance information. Purging
|
|
> Audit Logs may impact your organization's ability to investigate security
|
|
> incidents or meet compliance requirements. Consult your security and
|
|
> compliance teams before configuring Audit Log retention.
|
|
|
|
Common compliance frameworks have varying retention requirements:
|
|
|
|
- **SOC 2**: Typically requires 1 year of audit logs.
|
|
- **HIPAA**: Requires 6 years for certain records.
|
|
- **PCI DSS**: Requires 1 year of audit logs, with 3 months immediately
|
|
available.
|
|
- **GDPR**: Requires data minimization but does not specify maximum retention.
|
|
|
|
### External Log Aggregation
|
|
|
|
If you use an external log aggregation system (Splunk, Datadog, etc.), you can
|
|
configure shorter retention periods in Coder since logs are preserved
|
|
externally. See
|
|
[Capturing/Exporting Audit Logs](../security/audit-logs.md#capturingexporting-audit-logs)
|
|
for details on exporting logs.
|
|
|
|
### Database Maintenance
|
|
|
|
After enabling retention policies, you may want to run a `VACUUM` operation on
|
|
your PostgreSQL database to reclaim disk space. See
|
|
[Maintenance Procedures](../security/audit-logs.md#maintenance-procedures-for-the-audit-logs-table)
|
|
for guidance.
|
|
|
|
## Keeping Data Indefinitely
|
|
|
|
To keep data indefinitely for any data type, set its retention value to `0`:
|
|
|
|
```yaml
|
|
retention:
|
|
audit_logs: 0s # Keep audit logs forever
|
|
connection_logs: 0s # Keep connection logs forever
|
|
api_keys: 0s # Keep expired API keys forever
|
|
workspace_agent_logs: 0s # Keep workspace agent logs forever
|
|
|
|
ai_gateway:
|
|
retention: 0s # Keep AI Gateway records forever
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
The purge process logs deletion counts at the `DEBUG` level. To monitor
|
|
retention activity, enable debug logging or search your logs for entries
|
|
containing the table name (e.g., `audit_logs`, `connection_logs`, `api_keys`).
|
|
|
|
## Related Documentation
|
|
|
|
- [Audit Logs](../security/audit-logs.md): Learn about Audit Logs and manual
|
|
purge procedures.
|
|
- [Connection Logs](../monitoring/connection-logs.md): Learn about Connection
|
|
Logs and monitoring.
|
|
- [AI Gateway](../../ai-coder/ai-gateway/index.md): Learn about AI Gateway for
|
|
centralized LLM and MCP proxy management.
|
|
- [AI Gateway Setup](../../ai-coder/ai-gateway/setup.md#data-retention): Configure
|
|
AI Gateway data retention.
|
|
- [AI Gateway Monitoring](../../ai-coder/ai-gateway/monitoring.md): Monitor AI
|
|
Gateway usage and metrics.
|