A review reuses one chat for each pull request, and that chat owns one workspace. The workspace stops after about a day and then goes dormant. When a later push reuses the chat, every tool call fails: ``` get workspace agents in latest build: unauthorized: rbac: forbidden ``` The review reads no files, searches no docs, and posts nothing. This job read only the chat's terminal status, so the pull request got a green check for a review that never ran. Tracked in [CODAGT-1025](https://linear.app/codercom/issue/CODAGT-1025). ## How often Measured on the deployment that runs doc-check, over 30 days: | | | |---|---| | review sessions inspected | 250 | | sessions that hit this error | 54 (21.6%), 54 distinct pull requests | | failure on a reused chat, more than 48 h after the chat was created | 44 | | reused chat, 6 to 48 h later | 9 | | within 15 min of chat creation | 1 | | first seen | 2026-07-23 | The failing turn ran on Opus 4.8 in 49 of 54 cases, so this is not model specific. Right now 116 of 159 review workspaces are dormant, and 20 of those belong to open pull requests. ## What changes A new step reads the workspace state from Coder before the chat starts. If the workspace is dormant, the job fails, and the error and the summary link [CODAGT-1025](https://linear.app/codercom/issue/CODAGT-1025) and say how to start the workspace. If Coder cannot be read, the step warns and continues, so this never blocks a review on its own. The prompt now tells the review what to do when a tool call fails on the workspace, and it matches what the tools can actually do: - stopped workspace: call `start_workspace` - deleted workspace: `list_templates`, then `create_workspace` with the `coder-workflow-bot` template - dormant workspace: it cannot be repaired from inside the chat, so stop, say the review did not happen, and post no comment ## Why the chat cannot repair a dormant workspace A dormant workspace is a different RBAC resource. `WorkspaceTable.RBACObject` returns `ResourceWorkspaceDormant` when `DormantAt` is set. The chat daemon subject holds `workspace` read and update and no `workspace_dormant` permission, so both `start_workspace` and `create_workspace` fail on the same read before they do any work. Full write-up, with the code paths and a suggested fix, in [CODAGT-1025](https://linear.app/codercom/issue/CODAGT-1025). I tested this on this pull request: with its workspace stopped and dormant, the review called `start_workspace` twice, got `load workspace: unauthorized: rbac: forbidden`, then stopped and posted nothing. The new step then failed the job before the chat started, which is what the red check on this pull request is. ## Still open The job reads only the chat's terminal status, so a review that could not run and a review that found nothing both report success. The dormant case is covered here. Other failures are not.
293 lines
13 KiB
YAML
293 lines
13 KiB
YAML
# This workflow checks if a PR requires documentation updates.
|
|
# It creates a Coder Agent chat session that uses AI to analyze the PR
|
|
# changes, search existing docs, and comment with recommendations.
|
|
#
|
|
# Triggers:
|
|
# - New PR opened: Initial documentation review
|
|
# - PR updated (synchronize): Re-review after changes
|
|
# - Label "doc-check" added: Manual trigger for review
|
|
# - PR marked ready for review: Review when draft is promoted
|
|
# - Workflow dispatch: Manual run with PR URL
|
|
#
|
|
# Note: This workflow requires access to secrets and will be skipped for:
|
|
# - Any PR where secrets are not available
|
|
# For these PRs, maintainers can manually trigger via workflow_dispatch.
|
|
|
|
name: AI Documentation Check
|
|
|
|
on:
|
|
pull_request:
|
|
types:
|
|
- opened
|
|
- synchronize
|
|
- labeled
|
|
- ready_for_review
|
|
workflow_dispatch:
|
|
inputs:
|
|
pr_url:
|
|
description: "Pull Request URL to check"
|
|
required: true
|
|
type: string
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
doc-check:
|
|
name: Analyze PR for Documentation Updates Needed
|
|
runs-on: ubuntu-latest
|
|
# Run on: opened, synchronize, labeled (with doc-check label), ready_for_review, or workflow_dispatch
|
|
# Skip draft PRs unless manually triggered
|
|
if: |
|
|
(
|
|
github.event.action == 'opened' ||
|
|
github.event.action == 'synchronize' ||
|
|
github.event.label.name == 'doc-check' ||
|
|
github.event.action == 'ready_for_review' ||
|
|
github.event_name == 'workflow_dispatch'
|
|
) &&
|
|
(github.event.pull_request.draft == false || github.event_name == 'workflow_dispatch') &&
|
|
(github.event_name == 'workflow_dispatch' || github.event.pull_request.head.repo.full_name == github.repository)
|
|
timeout-minutes: 30
|
|
permissions:
|
|
contents: read
|
|
pull-requests: write
|
|
|
|
steps:
|
|
- name: Check if secrets are available
|
|
id: check-secrets
|
|
env:
|
|
CODER_URL: ${{ secrets.DOC_CHECK_CODER_URL }}
|
|
CODER_TOKEN: ${{ secrets.DOC_CHECK_CODER_SESSION_TOKEN }}
|
|
run: |
|
|
if [[ -z "${CODER_URL}" || -z "${CODER_TOKEN}" ]]; then
|
|
echo "skip=true" >> "${GITHUB_OUTPUT}"
|
|
echo "Secrets not available - skipping doc-check."
|
|
echo "This is expected for PRs where secrets are not available."
|
|
echo "Maintainers can manually trigger via workflow_dispatch if needed."
|
|
{
|
|
echo "⚠️ Workflow skipped: Secrets not available"
|
|
echo ""
|
|
echo "This workflow requires secrets that are unavailable for this run."
|
|
echo "Maintainers can manually trigger via workflow_dispatch if needed."
|
|
} >> "${GITHUB_STEP_SUMMARY}"
|
|
else
|
|
echo "skip=false" >> "${GITHUB_OUTPUT}"
|
|
fi
|
|
|
|
- name: Determine PR Context
|
|
if: steps.check-secrets.outputs.skip != 'true'
|
|
id: determine-context
|
|
env:
|
|
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
|
GITHUB_EVENT_ACTION: ${{ github.event.action }}
|
|
GITHUB_EVENT_PR_HTML_URL: ${{ github.event.pull_request.html_url }}
|
|
GITHUB_EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
INPUTS_PR_URL: ${{ inputs.pr_url }}
|
|
run: |
|
|
# Determine trigger type for context
|
|
if [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then
|
|
echo "trigger_type=manual" >> "${GITHUB_OUTPUT}"
|
|
echo "Using PR URL: ${INPUTS_PR_URL}"
|
|
|
|
# Validate PR URL format
|
|
if [[ ! "${INPUTS_PR_URL}" =~ ^https://github\.com/[^/]+/[^/]+/pull/[0-9]+$ ]]; then
|
|
echo "::error::Invalid PR URL format: ${INPUTS_PR_URL}"
|
|
echo "::error::Expected format: https://github.com/owner/repo/pull/NUMBER"
|
|
exit 1
|
|
fi
|
|
|
|
ISSUE_URL="${INPUTS_PR_URL/\/pull\//\/issues\/}"
|
|
echo "pr_url=${ISSUE_URL}" >> "${GITHUB_OUTPUT}"
|
|
PR_NUMBER=$(echo "${INPUTS_PR_URL}" | grep -oP '(?<=pull/)\d+')
|
|
echo "pr_number=${PR_NUMBER}" >> "${GITHUB_OUTPUT}"
|
|
|
|
elif [[ "${GITHUB_EVENT_NAME}" == "pull_request" ]]; then
|
|
echo "Using PR URL: ${GITHUB_EVENT_PR_HTML_URL}"
|
|
ISSUE_URL="${GITHUB_EVENT_PR_HTML_URL/\/pull\//\/issues\/}"
|
|
echo "pr_url=${ISSUE_URL}" >> "${GITHUB_OUTPUT}"
|
|
echo "pr_number=${GITHUB_EVENT_PR_NUMBER}" >> "${GITHUB_OUTPUT}"
|
|
|
|
# Set trigger type based on action
|
|
case "${GITHUB_EVENT_ACTION}" in
|
|
opened)
|
|
echo "trigger_type=new_pr" >> "${GITHUB_OUTPUT}"
|
|
;;
|
|
synchronize)
|
|
echo "trigger_type=pr_updated" >> "${GITHUB_OUTPUT}"
|
|
;;
|
|
labeled)
|
|
echo "trigger_type=label_requested" >> "${GITHUB_OUTPUT}"
|
|
;;
|
|
ready_for_review)
|
|
echo "trigger_type=ready_for_review" >> "${GITHUB_OUTPUT}"
|
|
;;
|
|
*)
|
|
echo "trigger_type=unknown" >> "${GITHUB_OUTPUT}"
|
|
;;
|
|
esac
|
|
|
|
else
|
|
echo "::error::Unsupported event type: ${GITHUB_EVENT_NAME}"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Check the review workspace is usable
|
|
if: steps.check-secrets.outputs.skip != 'true'
|
|
id: check-workspace
|
|
env:
|
|
CODER_URL: ${{ secrets.DOC_CHECK_CODER_URL }}
|
|
CODER_TOKEN: ${{ secrets.DOC_CHECK_CODER_SESSION_TOKEN }}
|
|
PR_NUMBER: ${{ steps.determine-context.outputs.pr_number }}
|
|
run: |
|
|
# A review reuses one chat per PR, and that chat owns one workspace.
|
|
# The workspace stops after about a day and then goes dormant. A chat
|
|
# cannot repair a dormant workspace: start_workspace and
|
|
# create_workspace both fail with
|
|
# "unauthorized: rbac: forbidden". The review then reads
|
|
# nothing and posts nothing, and this job used to still pass. Fail
|
|
# here instead, so the check reports what happened.
|
|
# https://linear.app/codercom/issue/CODAGT-1025
|
|
WS_NAME="doc-check-pr-${PR_NUMBER}"
|
|
RESPONSE=$(curl -sS --fail-with-body -G \
|
|
-H "Coder-Session-Token: ${CODER_TOKEN}" \
|
|
--data-urlencode "q=owner:me name:${WS_NAME}" \
|
|
"${CODER_URL%/}/api/v2/workspaces") || {
|
|
echo "::warning::Could not read the workspace state from Coder. Continuing."
|
|
exit 0
|
|
}
|
|
DORMANT_AT=$(echo "${RESPONSE}" | jq -r '.workspaces[0].dormant_at // empty')
|
|
if [[ -n "${DORMANT_AT}" ]]; then
|
|
echo "::error::${WS_NAME} is dormant since ${DORMANT_AT}. The review cannot run, because a chat cannot start its own dormant workspace: https://linear.app/codercom/issue/CODAGT-1025"
|
|
{
|
|
echo "### Documentation check did not run"
|
|
echo ""
|
|
echo "The workspace \`${WS_NAME}\` is dormant since ${DORMANT_AT}."
|
|
echo ""
|
|
echo "A chat cannot start its own dormant workspace, so the review would read no files and post no comment. See [CODAGT-1025](https://linear.app/codercom/issue/CODAGT-1025)."
|
|
echo ""
|
|
echo "To run the review now, start \`${WS_NAME}\` in Coder, then push again or re-run this job."
|
|
} >> "${GITHUB_STEP_SUMMARY}"
|
|
exit 1
|
|
fi
|
|
echo "Workspace state is usable."
|
|
|
|
- name: Build chat prompt
|
|
if: steps.check-secrets.outputs.skip != 'true'
|
|
id: extract-context
|
|
env:
|
|
PR_NUMBER: ${{ steps.determine-context.outputs.pr_number }}
|
|
TRIGGER_TYPE: ${{ steps.determine-context.outputs.trigger_type }}
|
|
run: |
|
|
echo "Analyzing PR #${PR_NUMBER} (trigger: ${TRIGGER_TYPE})"
|
|
|
|
# Build context based on trigger type
|
|
case "${TRIGGER_TYPE}" in
|
|
new_pr)
|
|
CONTEXT="This is a NEW PR. Perform initial documentation review."
|
|
;;
|
|
pr_updated)
|
|
CONTEXT="This PR was UPDATED with new commits. Check if previous feedback was addressed or if new doc needs arose."
|
|
;;
|
|
label_requested)
|
|
CONTEXT="A documentation review was REQUESTED via label. Perform a thorough review."
|
|
;;
|
|
ready_for_review)
|
|
CONTEXT="This PR was marked READY FOR REVIEW. Perform a thorough review."
|
|
;;
|
|
manual)
|
|
CONTEXT="This is a MANUAL review request. Perform a thorough review."
|
|
;;
|
|
*)
|
|
CONTEXT="Perform a documentation review."
|
|
;;
|
|
esac
|
|
|
|
# Build chat prompt with sticky comment logic
|
|
CHAT_PROMPT="Use the doc-check skill to review PR #${PR_NUMBER} in coder/coder.
|
|
|
|
${CONTEXT}
|
|
|
|
When creating a new workspace, use the \"coder-workflow-bot\" template.
|
|
|
|
If a tool call fails with \`get workspace agents in latest build: unauthorized: rbac: forbidden\`, this chat has no working workspace. That is an infrastructure state, not a mistake in your command, so the same command fails again if you retry it.
|
|
|
|
Call \`start_workspace\` with no arguments to start the workspace this chat owns. If it answers \`workspace was deleted\` or \`chat has no workspace\`, the old one is gone: call \`list_templates\` to get the ID of the \`coder-workflow-bot\` template, then call \`create_workspace\` with that ID. Wait for the workspace to be ready, then run the failed tool call again.
|
|
|
|
If \`start_workspace\` answers \`load workspace: unauthorized: rbac: forbidden\`, the workspace is dormant and you cannot repair it. Stop there.
|
|
|
|
When you cannot repair the workspace, say that you could not review the PR, and name the error you got. Post no doc-check comment. Do not say or imply the docs are fine, because you did not read them.
|
|
|
|
Use \`gh\` to get PR details, diff, and all comments. Look for an existing doc-check comment containing \`<!-- doc-check-sticky -->\` - if one exists, you'll update it instead of creating a new one.
|
|
|
|
Follow the skill's evidence discipline, including its mandatory
|
|
search: list every literal this diff changes, then search the whole
|
|
docs tree for each OLD literal before you decide. Do not review only
|
|
the files in the diff. Report each search you ran and what it
|
|
returned, and post nothing only after every search came back empty
|
|
outside \`docs/reference/\`.
|
|
|
|
**Do not comment if no documentation changes are needed.**
|
|
|
|
If a sticky comment already exists, compare your current findings against it:
|
|
- Check off \`[x]\` items that are now addressed
|
|
- Strikethrough items no longer needed (e.g., code was reverted)
|
|
- Add new unchecked \`[ ]\` items for newly discovered needs
|
|
- If an item is checked but you can't verify the docs were added, add a warning note below it
|
|
- If nothing meaningful changed, don't update the comment at all
|
|
|
|
## Comment format
|
|
|
|
Use this structure (only include relevant sections):
|
|
|
|
\`\`\`
|
|
## Documentation Check
|
|
|
|
### Updates Needed
|
|
- [ ] \`docs/path/file.md\` - What needs to change
|
|
- [x] \`docs/other/file.md\` - This was addressed
|
|
- ~~\`docs/removed.md\` - No longer needed~~ *(reverted in abc123)*
|
|
|
|
### New Documentation Needed
|
|
- [ ] \`docs/suggested/path.md\` - What should be documented
|
|
> ⚠️ *Checked but no corresponding documentation changes found in this PR*
|
|
|
|
---
|
|
*Automated review via [Coder Agents](https://coder.com/docs/ai-coder/agents)*
|
|
<!-- doc-check-sticky -->
|
|
\`\`\`
|
|
|
|
The \`<!-- doc-check-sticky -->\` marker must be at the end so future runs can find and update this comment."
|
|
|
|
# Output the prompt
|
|
{
|
|
echo "chat_prompt<<EOFOUTPUT"
|
|
echo "${CHAT_PROMPT}"
|
|
echo "EOFOUTPUT"
|
|
} >> "${GITHUB_OUTPUT}"
|
|
|
|
- name: Run doc-check via Coder Agent Chat
|
|
if: steps.check-secrets.outputs.skip != 'true'
|
|
uses: coder/agents-chat-action@b3fc81d7dae5006dd124e98ef6fada1a36cdd86e # v0.3.0
|
|
with:
|
|
coder-url: ${{ secrets.DOC_CHECK_CODER_URL }}
|
|
coder-token: ${{ secrets.DOC_CHECK_CODER_SESSION_TOKEN }}
|
|
# The bot account belongs to many organizations on this deployment,
|
|
# where the action's fallback organization choice is not
|
|
# deterministic. Pin it, because a model configuration ID only
|
|
# resolves inside its own organization.
|
|
coder-organization: coder
|
|
# Pins the model that reviews the PR. Set this repository variable to
|
|
# a model configuration ID in the organization above. Unset means the
|
|
# deployment's default model, so clearing the variable is the
|
|
# rollback path and needs no code change.
|
|
model-config-id: ${{ vars.DOC_CHECK_MODEL_CONFIG_ID }}
|
|
chat-prompt: ${{ steps.extract-context.outputs.chat_prompt }}
|
|
github-url: ${{ steps.determine-context.outputs.pr_url }}
|
|
github-token: ${{ github.token }}
|
|
wait: complete
|
|
wait-timeout-seconds: "600"
|
|
# The doc-check agent posts its own sticky comment when there
|
|
# are findings; failures surface in the workflow run log.
|
|
comment-on-issue: "false"
|