Files
coder/.github/workflows/audit-docs-paths.yaml
T
Nick Vigilante ac95e7b743 ci: notify Slack when audit-docs-paths finds docs URL drift (#27924)
## What

Adds Slack notifications to `.github/workflows/audit-docs-paths.yaml` so
the scheduled audit actively pings the docs team both when it finds docs
URL drift and when the run itself fails, on top of the existing tracked
issue + red check + report artifact.

## Why

The dedicated `audit-docs-paths` workflow (#27245) surfaces drift only
through a tracked GitHub issue, a failing check, and an uploaded report.
None of those actively notify anyone, so drift can sit unseen unless
someone is watching repo issues. The embedded `audit-docs-paths` job
that #27245 removed from `weekly-docs.yaml` already posted to Slack, so
this also restores that notification in the new workflow.

This also satisfies the condition on @bpmct's approval of #27245:
"assuming you have a solid solution for getting notified/assigned when a
link is old/stale."

## How

- Two notifications, both reusing the existing
`secrets.DOCS_LINK_SLACK_WEBHOOK` (the same docs webhook
`weekly-docs.yaml` uses), so no new secret or channel:
- **Drift** (`findings > 0`, step id `slack_drift`): runs immediately
after the audit, gated only on the audit outputs, so a later report- or
issue-step failure can't suppress it.
- **Failure** (`failure() && steps.slack_drift.outcome != 'success'`):
the last step in the job, so it fires on any job failure the drift ping
didn't already announce, whether pre-audit (e.g. an expired
`CDRCI_GITHUB_TOKEN` failing the coder.com checkout), the audit itself,
or a post-audit step (issue write, artifact upload). This restores and
generalizes the `failure()` coverage the pre-#27245 embedded job had,
and can't rot as steps are added. It stays silent on a drift run, where
that ping already fired.
- The audit measurement now fails loud rather than open: it asserts both
scan roots exist before running and drops the count fallback, so an
incomplete scan (e.g. a coder.com `src/` restructure the script would
only warn about) or an undetermined finding count fails the step and
triggers the failure ping, instead of reporting a false all-clear and
closing the tracked issue.
- Clean runs stay silent. The message links the run; the report artifact
and tracked issue are reachable from there.
- The webhook is passed via `env` with a `[ -n ]` guard (named error if
unset). `curl` uses `-fsS --max-time 10 --retry 3 --retry-all-errors`
under `set -euo pipefail`, so a revoked or misconfigured webhook fails
the step (red) instead of silently logging success, while a transient
blip is retried.
- Everything stays gated behind the job-level
`vars.AUDIT_DOCS_PATHS_ENABLED`.

## Follow-up (out of scope)

- `weekly-docs.yaml`'s `Send Slack notification` step has the same
bare-`curl` pattern this step was copied from. It predates this PR and
is tracked separately in
[DOCS-705](https://linear.app/codercom/issue/DOCS-705/harden-weekly-docsyaml-slack-curl-with-fail).

## Rebase note

Originally stacked on #27245; since that merged, the branch is rebased
onto `main`. The diff is scoped to
`.github/workflows/audit-docs-paths.yaml`.

## Validation

- `make lint/actions` (actionlint + shellcheck + zizmor): clean.
- `make lint/emdash`, `make lint/typos`: clean.
- Verified empirically: both message payloads are valid JSON; the `[ -n
]` guard fails the step on an unset webhook; `curl -fsS` exits non-zero
on an HTTP 4xx/5xx and `set -e` then fails the step, so a bad webhook
can't log a false success.
- Traced the `failure() && steps.slack_drift.outcome != 'success'` gate
across scenarios: clean run (silent), drift (drift ping only, no
double-ping), and pre-audit / audit / post-audit failures (failure ping
fires).

Linear: https://linear.app/codercom/issue/DOCS-617

> This PR was created with AI assistance (Coder Agents).
2026-08-27 20:37:44 +00:00

246 lines
11 KiB
YAML

name: audit-docs-paths
# Scheduled audit for docs-URL drift.
#
# site/scripts/audit-docs-paths.mjs cross-references every docs("...")
# call, every hardcoded coder.com/docs/... URL, and every ](/docs/...)
# Markdown link in the TS/TSX of BOTH coder/coder (site/src) and
# coder/coder.com (src) against the source side of every /docs/* rule in
# coder/coder.com/redirects.json. Anything that matches a redirect source
# is stale: the reference points at a path that now redirects elsewhere
# and should be updated to the redirect's destination.
#
# This runs the audit on a weekly cron (and on demand) so drift is caught
# automatically instead of only when someone remembers to run the script.
# The suggestion to schedule it came from the audit script's original PR,
# coder/coder#25740.
#
# Why this job is gated (vars.AUDIT_DOCS_PATHS_ENABLED)
# ----------------------------------------------------
# The audit needs coder/coder.com, which is a PRIVATE repo. The default
# GITHUB_TOKEN cannot read another repo, so the coder.com checkout below
# uses the cdrci machine-user token (secrets.CDRCI_GITHUB_TOKEN), the
# same org CI credential already used for cross-repo checkouts in
# release.yaml. The job
# stays dormant behind a repository variable so it can merge without
# running until an operator has validated it once.
#
# To enable
# 1. secrets.CDRCI_GITHUB_TOKEN (the cdrci CI bot, a coder.com
# collaborator) reads coder/coder.com. Issue writes on this repo are
# covered by the default GITHUB_TOKEN below, not by this token.
# 2. Set the repository variable AUDIT_DOCS_PATHS_ENABLED to 'true'.
# 3. Trigger once from the Actions UI (workflow_dispatch) to confirm an
# end-to-end run before relying on the schedule.
#
# What it does with findings
# * Always uploads the dated report as a build artifact and writes it to
# the run summary, so a report exists even when the audit is clean.
# * On findings, opens (or updates) a single deduplicated tracked issue
# with the report and fails the run so the check goes red.
# * On a clean run, closes the tracked issue if one is open.
on:
schedule:
# Weekly, Monday 09:00 UTC - same cadence as the weekly-docs link check.
- cron: "0 9 * * 1"
workflow_dispatch: # manual, for testing and on-demand audits
permissions:
contents: read
concurrency:
group: audit-docs-paths
cancel-in-progress: false
jobs:
audit-docs-paths:
# Dormant until an operator flips the variable. See the header comment.
if: vars.AUDIT_DOCS_PATHS_ENABLED == 'true'
runs-on: ubuntu-latest
permissions:
contents: read # checkout
issues: write # open/update/close the tracked issue via GITHUB_TOKEN
steps:
- name: Harden Runner
uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with:
egress-policy: audit
# coder/coder at the workspace root so the local composite actions
# under ./.github/actions/* resolve normally.
- name: Checkout coder/coder
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
# coder/coder.com beside it, read with the cdrci org CI token
# (the default GITHUB_TOKEN can't read another private repo). The
# audit's report path detection keys off the ".../coder/site/" and
# ".../coder.com/src/" path segments, which the absolute --roots
# below preserve.
- name: Checkout coder/coder.com
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: ${{ github.repository_owner }}/coder.com
path: coder.com
token: ${{ secrets.CDRCI_GITHUB_TOKEN }}
persist-credentials: false
- name: Set up mise tools
uses: ./.github/actions/setup-mise
with:
install-args: "node"
- name: Run docs-paths audit
id: audit
env:
WORKSPACE: ${{ github.workspace }}
run: |
set -euo pipefail
# audit-docs-paths.mjs treats a missing scan root as a warning, not an error, so fail the step here.
for root in "${WORKSPACE}/site/src" "${WORKSPACE}/coder.com/src"; do
[ -d "${root}" ] || { echo "::error::audit scan root not found: ${root}"; exit 1; }
done
# Absolute --roots are required: the audit classifies findings by
# matching the "/coder/site/" and "/coder.com/src/" segments in
# each file's absolute path. Relative roots would leave every
# finding unclassified.
node site/scripts/audit-docs-paths.mjs \
--redirects="${WORKSPACE}/coder.com/redirects.json" \
--roots="${WORKSPACE}/site/src,${WORKSPACE}/coder.com/src" \
--out="${WORKSPACE}/audit-report.md" \
2>&1 | tee "${WORKSPACE}/audit-output.txt"
# No fallback: a missing "Total findings:" line must fail, not read as zero.
count=$(grep -oP 'Total findings: \K\d+' "${WORKSPACE}/audit-output.txt")
echo "count=${count}" >> "$GITHUB_OUTPUT"
echo "Total findings: ${count}"
{
echo "# audit-docs-paths"
echo
echo "Findings: **${count}**"
echo
cat "${WORKSPACE}/audit-report.md"
} >> "$GITHUB_STEP_SUMMARY"
- name: Send Slack notification on drift
id: slack_drift
# Reuses weekly-docs.yaml's docs Slack webhook (same channel, no new secret).
# Runs before the report/issue/fail steps so their failure can't drop the ping.
if: steps.audit.outputs.count != '' && steps.audit.outputs.count != '0'
env:
COUNT: ${{ steps.audit.outputs.count }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
SLACK_WEBHOOK: ${{ secrets.DOCS_LINK_SLACK_WEBHOOK }}
run: |
set -euo pipefail
[ -n "${SLACK_WEBHOOK}" ] || { echo "::error::DOCS_LINK_SLACK_WEBHOOK is not set"; exit 1; }
# -f: a revoked or misconfigured webhook must fail the step, not log success.
curl \
-fsS \
--max-time 10 \
--retry 3 \
--retry-all-errors \
-X POST \
-H 'Content-type: application/json' \
-d '{"text":":warning: *Docs URL drift detected by audit-docs-paths.*\n'"${COUNT}"' stale docs path reference(s) resolve to a redirect source. See the run: '"${RUN_URL}"'"}' \
"${SLACK_WEBHOOK}"
echo "Sent Slack drift notification"
- name: Upload audit report
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: audit-docs-paths-report
path: ${{ github.workspace }}/audit-report.md
if-no-files-found: warn
- name: Open, update, or close tracked issue
if: always() && steps.audit.outcome == 'success'
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
COUNT: ${{ steps.audit.outputs.count }}
REPORT: ${{ github.workspace }}/audit-report.md
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
set -euo pipefail
TITLE="Docs URL drift detected by audit-docs-paths"
MARKER="<!-- audit-docs-paths-tracked-issue -->"
# Find an existing open tracked issue by exact title.
existing=$(gh issue list --repo "$REPO" --state open \
--search "${TITLE} in:title" \
--json number,title \
--jq "map(select(.title == \"${TITLE}\")) | (.[0].number // empty)")
if [ "${COUNT:-0}" -gt 0 ]; then
body_file="$(mktemp)"
{
echo "${MARKER}"
echo
echo "The scheduled \`audit-docs-paths\` workflow found **${COUNT}** stale docs path reference(s) that resolve to a redirect source in \`coder/coder.com/redirects.json\`. Each should be updated to the redirect's destination."
echo
echo "- Latest run: ${RUN_URL}"
echo "- Full report: the \`audit-docs-paths-report\` artifact on that run."
echo
echo "<details><summary>Report</summary>"
echo
# Cap the embedded report to stay under GitHub's ~65k issue
# body limit; the full report is always in the artifact.
head -c 55000 "${REPORT}"
echo
echo "</details>"
} > "${body_file}"
if [ -n "${existing}" ]; then
gh issue edit "${existing}" --repo "${REPO}" --body-file "${body_file}"
echo "Updated tracked issue #${existing}."
else
gh issue create --repo "${REPO}" --title "${TITLE}" --body-file "${body_file}"
echo "Created tracked issue."
fi
else
if [ -n "${existing}" ]; then
gh issue comment "${existing}" --repo "${REPO}" \
--body "Audit is clean as of [this run](${RUN_URL}). Closing."
gh issue close "${existing}" --repo "${REPO}"
echo "Closed tracked issue #${existing} (audit clean)."
else
echo "Audit clean and no tracked issue open; nothing to do."
fi
fi
- name: Fail the run when findings exist
if: steps.audit.outputs.count != '' && steps.audit.outputs.count != '0'
env:
COUNT: ${{ steps.audit.outputs.count }}
run: |
echo "::error::audit-docs-paths found ${COUNT} stale docs path reference(s). See the tracked issue and the audit-docs-paths-report artifact."
exit 1
- name: Send Slack notification on failure
# A failed scheduled run notifies nobody, so ping on any job failure the
# drift step above didn't already announce.
if: failure() && steps.slack_drift.outcome != 'success'
env:
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
SLACK_WEBHOOK: ${{ secrets.DOCS_LINK_SLACK_WEBHOOK }}
run: |
set -euo pipefail
[ -n "${SLACK_WEBHOOK}" ] || { echo "::error::DOCS_LINK_SLACK_WEBHOOK is not set"; exit 1; }
curl \
-fsS \
--max-time 10 \
--retry 3 \
--retry-all-errors \
-X POST \
-H 'Content-type: application/json' \
-d '{"text":":rotating_light: *audit-docs-paths failed; docs URL drift monitoring needs attention.* See the run: '"${RUN_URL}"'"}' \
"${SLACK_WEBHOOK}"
echo "Sent Slack failure notification"