2 Commits
Author SHA1 Message Date
NianJiu 6729e7bf38 fix(ci,hooks): watch swagger specs in app.yml, compile all sandbox test pkgs, and enforce client gofmt in hooks
Why:
1. With docs/docs.go embedding swagger.json via //go:embed instead of codegen,
   editing docs/swagger.{json,yaml} or docs/swag.overrides without touching a
   .go file would bypass .github/workflows/app.yml path filters.
2. Cross-platform localsandbox CI only test-compiled seatbelt on darwin and
   skipped go test -c across the remaining darwin/windows localsandbox packages.
3. Excluding client/** from root pre-push Go checks dropped local gofmt
   enforcement on client/*.go and inherited developer WEKNORA_LOG_FORMAT in
   client/cli hook test runs.

What:
- Add docs/swagger.json, docs/swagger.yaml, and docs/swag.overrides to app.yml
  push and pull_request path triggers.
- Compile test binaries (go test -c) for every ./internal/localsandbox/...
  package on both darwin/arm64 and windows/amd64, and include client/go.sum in
  cli.yml cache-dependency-path.
- Enforce gofmt -l . and WEKNORA_LOG_FORMAT="" in run_client_checks() /
  run_cli_checks(), format staged cli/client Go files in pre-commit, and add
  regression coverage in scripts/test-git-hooks.sh.
2026-11-18 15:53:54 +08:00
MidoriKurageandwizardchen 236d8c098f fix(datasource): perform real sync deletions scoped per data source (#2690)
* test(datasource): cover scoped sync deletions

- deleted connector items remove the matching KB knowledge, scoped per data
  source via metadata datasource_id + external_id
- SyncDeletions=false neither deletes nor counts
- stream handler classifies deleted/failed items like the batch loop

* fix(datasource): perform real sync deletions scoped per data source

- applyFetchedItem now performs the KB deletion instead of only counting
- lookup is scoped to datasource_id + external_id so identical external IDs
  from two data sources cannot collide or overwrite each other
- ingestItem update-path lookup uses the same scoped query

* fix(datasource): note failed deletions retry only on full sync

- SyncResult.DeletionFailed tracks deletion failures (subset of Failed)
- streaming partial message warns that failed deletions only retry on a full
  sync, since the connector cursor is checkpointed past the deleted item

* fix(datasource): humanize deletion errors, keep raw detail in logs

- raw lookup/delete errors go to server logs only, with full context
- UI samples carry a humanized "see server logs" fallback
- add datasource.syncError.* i18n keys

Note: ko-KR / ru-RU syncError translations were generated without a
native-speaker review.

* fix(datasource): hard-delete synced rows and tighten deletion scoping

Sync-internal deletions now call HardDeleteKnowledge after the soft-delete
cascade so tombstones cannot block re-sync; subtree sweeps and URL metadata
attach failures are handled with datasource scoping and explicit errors.

* chore: add git hooks aligned with CI and PR checks

Install via scripts/install-git-hooks.sh (sets core.hooksPath). Pre-commit
runs whitespace/gofmt/golangci-lint on staged files; pre-push mirrors
app/frontend/cli workflows on the diff since origin/main.

* fix: gofmt test file; tighten pre-push to match CI app workflow

Pre-push now gofmts the full PR diff and runs go vet on all app packages
(like app.yml), not only Go files in the latest commit. Previous pushes
skipped checks via --no-verify and because hook-only commits had no .go files.

* fix(hooks): align pre-push diff with GitHub PR base via gh

Use three-dot diff against the open PR baseRefOid (same as CI) instead of
a two-dot merge-base range; run full go vet on every push with Go PR changes.

* style: gofumpt datasource_service_test.go

---------

Co-authored-by: wizardchen <wizardchen@tencent.com>
2026-08-13 19:52:44 +08:00