Compare commits

...
93 Commits
Author SHA1 Message Date
github-actions[bot] 8d3e4ad904 chore(master): release 2.8.5 (#558)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-27 08:09:02 +00:00
vastsa 381a539c92 fix: resolve OSS chunk cleanup and legacy download issues
Use compatible S3 checksum settings, retry failed cleanup without reusing completed uploads, and include legacy attachment filenames. Correct admin-page URLs in both documentation languages.

Refs #556, #531, #502
2026-10-27 16:08:35 +08:00
github-actions[bot] 6f88eae854 chore(master): release 2.8.4 (#555)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-25 15:44:31 +00:00
Lan 5da611c5de Merge pull request #554 from jcbl1/upstream/07-runtime
perf(runtime): enable uvloop/httptools and tune container memory/keep…
2026-10-25 23:44:26 +08:00
Lan a578fc690c Merge pull request #551 from jcbl1/upstream/04-db-indexes
perf(db): index expiry, recency and chunk lookup columns
2026-10-25 23:44:20 +08:00
Lan 2073f9ba4b Merge pull request #550 from jcbl1/upstream/03-sqlite-pragmas
perf(db): tune SQLite connection pragmas for small VPS
2026-10-25 23:44:12 +08:00
Lan 0a1f08fd6e Merge pull request #549 from jcbl1/upstream/01-web-startup
Optimize performance with caching and startup improvements
2026-10-25 23:44:06 +08:00
Lan 9bf03bc427 Merge pull request #522 from vastsa/dependabot/github_actions/docker/setup-qemu-action-4
chore(deps): bump docker/setup-qemu-action from 3 to 4
2026-10-25 23:39:01 +08:00
Lan 264eb4f986 Merge pull request #521 from vastsa/dependabot/github_actions/actions/upload-pages-artifact-5
chore(deps): bump actions/upload-pages-artifact from 3 to 5
2026-10-25 23:38:52 +08:00
Lan c973d28256 Merge pull request #520 from vastsa/dependabot/github_actions/actions/configure-pages-6
chore(deps): bump actions/configure-pages from 4 to 6
2026-10-25 23:38:46 +08:00
Lan e7f761c35f Merge pull request #519 from vastsa/dependabot/github_actions/googleapis/release-please-action-5
chore(deps): bump googleapis/release-please-action from 4 to 5
2026-10-25 23:38:39 +08:00
Lan 8f7a1e1594 Merge pull request #517 from vastsa/dependabot/github_actions/actions/setup-node-7
chore(deps): bump actions/setup-node from 4 to 7
2026-10-25 23:38:32 +08:00
jcbl1 4f141eeb49 perf(startup): drop duplicate Tortoise init, keep ORM exception handlers 2026-10-24 08:22:41 +00:00
jcbl1 496a7139bd perf(rate-limit): bound tracked IPs and use monotonic clock 2026-10-24 08:22:41 +00:00
jcbl1 5b5d70c3b7 perf(web): cache rendered index.html and add theme asset cache headers 2026-10-24 08:22:41 +00:00
jcbl1 ec714dcd6f perf(db): tune SQLite connection pragmas for small VPS
- synchronous=NORMAL: WAL keeps committed data safe against app crashes
  while skipping an fsync per commit.
- temp_store=MEMORY: temp tables and sorts stay in RAM.
- cache_size=-8000: cap page cache at ~8MB.

Tortoise's SqliteClient applies every non-file_path credential as a PRAGMA
on its single connection, so this is the only place that needs changing.
2026-10-24 08:17:06 +00:00
jcbl1 588af7b6c3 perf(runtime): enable uvloop/httptools and tune container memory/keep-alive
- deps: pin uvloop 0.23.0 (non-Windows marker) and httptools 0.9.0; uvicorn's
  --loop/--http auto picks both up. Lockfile regenerated with constraints so
  only these two packages are added (no transitive version drift).
- Dockerfile: expose TIMEOUT_KEEP_ALIVE (default 5, passed to uvicorn) and
  MALLOC_TRIM_THRESHOLD_=65536 so glibc returns freed heap to the OS after
  large transfers. Defaults keep previous behaviour.
- ci: allow `; marker` pins in the lockfile drift check, otherwise uvloop
  would be silently skipped and never verified.
- docs: document both environment variables (zh/en).
- tests: pin the Dockerfile/requirements/lockfile runtime contract.
2026-10-24 08:16:54 +00:00
jcbl1 080fecf653 perf(db): index expiry, recency and chunk lookup columns
Migration 008 adds:
- filecodes(expired_at) for expiry scans and admin expiry stats
- filecodes(created_at) for the default recency ordering and today/yesterday
  counters
- uploadchunk(upload_id, chunk_index) for chunk status checks and merges;
  its prefix also covers upload_id-only deletes (previously unindexed)

Models mirror the same metadata for the generate_schemas test path.
Upgrade benchmark: 50k filecodes + 20k chunks -> 30ms index build; scans
become index searches.

Refs NOTES batch 4.
2026-10-24 08:16:54 +00:00
vastsa 3d2c8ec709 docs: document MCP support and remove BokeBox link 2026-10-12 18:37:48 +08:00
github-actions[bot] f1fee1edff chore(master): release 2.8.3 (#545)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 13:08:55 +00:00
vastsa 65f4929d40 fix: remove HTML preview endpoint 2026-10-10 21:08:32 +08:00
github-actions[bot] 40f22de7f8 chore(master): release 2.8.2 (#544)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 11:04:19 +00:00
vastsa da374b4595 fix: type MCP protocol results 2026-10-10 19:03:51 +08:00
github-actions[bot] d3f9e7902c chore(master): release 2.8.1 (#543)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 10:57:39 +00:00
vastsa 2f5eab18b1 fix: restore lint and Docker theme build 2026-10-10 18:57:15 +08:00
github-actions[bot] 6bb1ef06c1 chore(master): release 2.8.0 (#542)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 10:52:52 +00:00
vastsa 63e72c1455 feat: add MCP quick share and safe HTML preview 2026-10-10 18:52:28 +08:00
github-actions[bot] 648d578d53 chore(master): release 2.7.1 (#536)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-19 18:06:24 +00:00
vastsa 031d300cae fix: keep Docker as root, close s3_hostname SSRF, unbreak 404 tests
Revert gosu/non-root entrypoint so NAS and root-owned volumes keep
working. Reject @/userinfo in s3_hostname. Theme-less checkouts no
longer crash the HTML 404 handler.
2026-09-20 02:05:57 +08:00
Marrrrrrrrry 57c75a7141 全量修复/补测试+UED改进 (#535)
* fix: constant-time comparison for download tokens (security item 5)

Replace the set-membership key check in /share/download with
hmac.compare_digest over the two valid window tokens; add negative-path
tests (wrong key 403, both windows accepted, foreign-code token 403).

* fix: single source for attachment headers + guard tests (security item 2)

All six Content-Disposition construction sites across the five storage
backends now go through build_attachment_headers; the header is what
neutralizes stored XSS on the same-origin download path. Guard tests
assert every get_file_response uses the builder and no hand-built
disposition reappears.

* fix: reject internal-network endpoints for storage config (security item 7)

s3_endpoint_url/s3_hostname/webdav_url are fetched server-side; with
APP_ENV=production the write entry now enforces http(s) schemes and a
loopback/private/link-local/metadata host blacklist. Development env
stays open for local minio/webdav. Already-stored values are never
re-validated, so existing deployments are unaffected.

* fix: JSON 404 for handler-raised 404s; negative-path test batch

Starlette status-code handlers take precedence over the HTTPException
class handler, so registering the theme index as the 404 handler turned
every HTTPException(404) app-wide into a 200 HTML page (found by the new
negative-path tests). The new not_found_handler serves the theme page
only to browsers (Accept: text/html) and JSON 404 to API clients.

Negative-path batch: download count exhaustion, missing chunk session
404s, expired presign session deletion, admin update_file uniqueness/
existence.

* fix: run container as non-root app user with compat volume chown (security item 3)

Entrypoint starts as root only to fix the data-dir ownership (skipped
when nothing needs changing or when the deployer sets an explicit user),
then drops to uid 10001 via gosu. Verified in-container: uvicorn runs as
app (uid 10001), setup and share round-trip work against a root-owned
volume.

* fix: compare download tokens as bytes (non-ASCII key caused 500)

* test: lock changed-only semantics for endpoint config saves

Integration regression for the background lesson: a stored internal
endpoint must not block unrelated settings saves in production mode;
changing to an internal endpoint (URL or bare-hostname form) is 400.

* fix: resolve-based SSRF check — deny IP shorthands, hex/decimal IPs, DNS maps

Probe found the static blacklist bypassed by glibc shorthands (127.1,
10.1), decimal (2130706433) and hex IP forms, *.localhost, and
loopback-mapping DNS services. Both validators now resolve the host and
re-check every resolved address; hostname tier extracts the host part
correctly for [v6]/host:port/bare-v6 forms. 44 tests.

* test: pin 404 handler dual-branch behavior (browser theme page vs API JSON)

* fix: write back normalized endpoint values; hostname dev-gate test

* test: header-injection negative paths for attachment disposition

* fix: S3 merge buffers parts to >=5MiB; missing object 404 upfront

- S3 multipart rejects parts <5MiB (EntityTooSmall) except the last; chunk
  size is client-controlled (commonly 2-4MB) so multi-chunk merges always
  failed with 500. Merge now buffers chunks into >=5MiB parts (memory
  bound: 5MB + one chunk).
- get_file_response: a 404 head_object now raises StorageError(404)
  upfront instead of signing a doomed presigned URL and failing mid-stream
  (aligns with local backend semantics from the M2 behavior unification).

* test: S3 backend coverage via in-process moto server (13 cases)

mock_aws cannot intercept aioboto3's aiohttp stack, so tests run against
a real in-process ThreadedMotoServer endpoint. Covers roundtrips, missing
object 404, merge failure modes (missing chunk / hash mismatch -> abort,
no leftover object), cleanup scoping, presign shapes, proxy dispatch.
Also adds moto[s3,server] to the dev group and CI.

* fix: WebDAV missing object 404 upfront; connection errors map to 503

HEAD 404 was silently swallowed (signed a 200 bad stream that failed
mid-download, same family as the S3 bug), and HEAD connection errors were
also swallowed so the 503 mapping never fired. Both now surface properly;
the aiohttp session is reclaimed on pre-stream exception paths.

* test: WebDAV backend coverage via in-process WebDAV server (9 cases)

Real HTTP (HEAD/GET/PUT/DELETE/MKCOL/PROPFIND) against a minimal aiohttp
WebDAV server over a temp dir: roundtrips, missing object 404, connection
error 503, delete with empty-parent cleanup, merge failure modes, chunk
cleanup scoping.

* fix: OneDrive missing object maps to 404 upfront; OpenDAL behavior pinned

OneDrive get_file_response translated graph itemNotFound into the outer
503 catch-all; now raises StorageError(404) like local/S3/WebDAV.
OpenDAL already mapped missing objects to 404 via its outer catch —
pinned with fake-operator tests (SDK not in runtime deps, instances built
via __new__). Also tightens the S3 presign URL assertion.

* test: admin write-path coverage (batch/single delete, batch update, policy actions)

13 negative-path cases for the previously untested data-modifying admin
endpoints: mixed-id aggregation with duplicates and missing records,
empty-list rejections, clear_expired_at permanence semantics, policy
action boundaries (zero limit 400, unknown action 400, missing 404),
and the DoesNotExist->404 mapping pinned for single delete.

* test: admin read-path and view-preset CRUD coverage (22 cases)

Detail/metadata/preview/admin-download/activities/local-lists+delete/
verify: missing-record 404s, note/tag truncation limits, preview max_chars
truncation, activity filtering and the 80-event clamp, local path
traversal rejection. Presets: update-vs-create by id, name truncation,
filter normalization clamps, 24-preset cap, delete-missing 404.

* refactor: move auth primitives to apps/base/auth, kill base->admin dep

JWT create/verify, bearer extraction and the share-upload gate move to
apps.base.auth (consumed by both surfaces); admin.dependencies keeps
admin session gating and service providers, re-exporting the primitives
for import-path compatibility. base.views no longer imports from
apps.admin — the last cross-app reverse dependency is gone.

* refactor: split admin services god-module (ConfigService/LocalFile out)

apps/admin/services.py (1697 lines) becomes FileService (~1200 lines) +
a compatibility facade re-exporting ConfigService, LocalFileService,
LocalFileClass and keyvalue_write_lock from their new homes
(config_service.py holds the D5 KeyValue lock; one-way imports, no
cycle). test_admin_security patch targets follow ConfigService to its
new module. Behavior unchanged; 247 tests green.

* fix: size-less uploads no longer crash (upstream seek bug)

validate_file_size's size-None branch called UploadFile.seek(0, 2),
which raises TypeError (UploadFile.seek takes one arg); the underlying
SpooledTemporaryFile.seek is sync, so awaiting it also fails. Use the
underlying file object with sync seek(0, SEEK_END). Surface discovered
by the mypy pilot run; regression test covers the size-None branch.

* chore: bump direct deps (patch/minor)

fastapi 0.139.2->0.141.1, pydantic 2.12.5->2.13.5, uvicorn 0.51.0->0.53.0,
aiohttp 3.14.2->3.14.3; lockfile regenerated. tortoise-orm 0.x->1.x major
deliberately deferred pending API-change review.

* ci: gradual mypy adoption via baseline ratchet

scripts/mypy_ratchet.py compares mypy output against
scripts/mypy-baseline.txt (line numbers stripped for edit stability):
new type errors fail CI, fixes are folded back via --regenerate.
Baseline starts at 30 known errors; the seek bug found in the pilot was
fixed before baselining. mypy joins the dev group and CI.

* fix: migrate WebDAV auth off aiohttp.BasicAuth (deprecated in 4.0)

Dependency-upgrade probe on aiohttp 3.14.3 surfaced the BasicAuth
deprecation; switch to aiohttp.encode_basic_auth() headers so the
aiohttp 4.0 upgrade does not break the WebDAV backend.

* chore: drop dead code (WebDAV _instance stub, unused LocalFileClass.write)

* test: file-validation negative paths (13 cases)

Magic-bytes spoofing (text-as-png, exe-as-pdf, mp4/webp box detection),
whitelist rule semantics (star/image-wildcard/content mismatch), chunk-0
header validation parity, and end-to-end UploadFile rejection.

* test: migration runner coverage (5 cases)

Full chain executes 001-007 in filename order and registers; rerun is
idempotent; pre-registered entries are skipped (DDL not re-executed);
the resulting schema matches deployment; a failing migration propagates
and is not registered. Uses a fresh :memory: DB without
generate_schemas so the migrations themselves build the schema (the
real deployment path).

* refactor: split core/storage.py into a package (per-backend modules)

core/storage.py (1400+ lines, six backends) becomes core/storage/ with
_base.py (data contracts, interface, shared header builder), and
local/s3/onedrive/opendal/webdav modules. The package __init__
re-exports the full historical public surface; test patch targets move
to the precise backend modules (core.storage.local.data_root etc.) and
the attachment guard scans the package. Import-only change; 264 tests
green. Storage-layer DI deferred: current tests construct instances
against patched settings, so the added surface isn't justified yet.

* refactor: TypedDict for IPRateLimit records

The rate-limit ledger mixed int and datetime under a loose Union typing,
producing 5 mypy noise errors. A TypedDict gives precise per-key types;
setdefault replaces the get-then-assign pattern. One mypy baseline error
family resolved; 264 tests green.

* chore: tighten mypy baseline after IPRateLimit TypedDict

* refactor: resolve all 26 baseline mypy errors; precise test assertions

Per-error treatment (no bulk script): honest annotations for
StoredDownload (Path/bytes/Callable), KeyValue.value (Any for JSONField),
dict[str, Any] for form/config dicts; signature split in create_token;
str() normalization in config int(); platform/type-ignore only where the
checker cannot follow (msvcrt, FastAPI Request injection). Baseline file
and ratchet compare now redundant at zero errors but kept as the guard
against regressions.

Tests: assertions tightened to status_code == 403 with HTTPException
raises; _create_share explicit parameters replace **extra passthrough;
seek-fix pointer-reset assertion persisted.
2026-09-20 03:01:08 +09:00
github-actions[bot] 82555c08ed chore(master): release 2.7.0 (#533)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-16 10:31:45 +00:00
vastsa 93a30a257c feat: mint extract codes from data/local without re-upload
Admin can browse NAS files under data/local including subfolders and share
them in place. Shares use file_path=local-ref, skip storage quota, and never
delete the original file on expire.
2026-09-16 18:31:00 +08:00
github-actions[bot] 3c135eb29f chore(master): release 2.6.1 (#529)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-14 13:53:56 +00:00
Lan 0d82fbe26a Merge pull request #528 from vastsa/fix/post-514-upgrade-regressions
fix: 存量 background 值不再阻塞后台设置保存(#514 升级回归)
2026-09-14 22:53:32 +09:00
vastsa eb2db8f507 fix: don't let a legacy background value block settings saves
#514 began validating `background` as an http(s) URL against the *merged*
config, so a value stored by an older release (a relative path, or one
containing a space/parenthesis — all legal before) made every subsequent
settings save return 400, including saves that never touched `background`.
Existing deployments could not change any setting at all.

Validate only values that actually change: an untouched legacy value is
kept as-is (it is still html-escaped on render), while any edit must pass
validation. Regression tests cover both directions and fail without this
change.

Also treat OverflowError from hashlib.scrypt as a non-matching hash so a
hand-crafted n/r/p cannot disturb the login path on runtimes that raise it.
2026-09-14 21:52:23 +08:00
dependabot[bot] fda470b076 chore(deps): bump docker/setup-qemu-action from 3 to 4
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-14 13:47:26 +00:00
dependabot[bot] 41e3ce1b9c chore(deps): bump actions/upload-pages-artifact from 3 to 5
Bumps [actions/upload-pages-artifact](https://github.com/actions/upload-pages-artifact) from 3 to 5.
- [Release notes](https://github.com/actions/upload-pages-artifact/releases)
- [Commits](https://github.com/actions/upload-pages-artifact/compare/v3...v5)

---
updated-dependencies:
- dependency-name: actions/upload-pages-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-14 13:47:25 +00:00
dependabot[bot] 31027e6eea chore(deps): bump actions/configure-pages from 4 to 6
Bumps [actions/configure-pages](https://github.com/actions/configure-pages) from 4 to 6.
- [Release notes](https://github.com/actions/configure-pages/releases)
- [Commits](https://github.com/actions/configure-pages/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/configure-pages
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-14 13:47:21 +00:00
dependabot[bot] d0f08a7be5 chore(deps): bump googleapis/release-please-action from 4 to 5
Bumps [googleapis/release-please-action](https://github.com/googleapis/release-please-action) from 4 to 5.
- [Release notes](https://github.com/googleapis/release-please-action/releases)
- [Changelog](https://github.com/googleapis/release-please-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/release-please-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: googleapis/release-please-action
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-14 13:47:18 +00:00
dependabot[bot] 5646f41255 chore(deps): bump actions/setup-node from 4 to 7
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 7.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v4...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-09-14 13:47:13 +00:00
github-actions[bot] 18d344cc9b chore(master): release 2.6.0 (#515)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-14 13:47:00 +00:00
Lan 8215f91edf Merge pull request #514 from Marrrrrrrrry/fix/m0-stop-bleeding
工程化加固、安全与性能修复、API 契约统一 snake_case(含 CI 测试流水线)
2026-09-14 22:46:18 +09:00
Marrrrrrrrry 775f3fc4ce ci: add test pipeline and API contract guard
- ci.yml: ruff + pytest on push/PR, installing from the hashed lockfile
  with --require-hashes, plus a lockfile-vs-requirements consistency
  check (dependabot cannot regenerate the lockfile; drift was silent).
- test_api_contract.py: deep-scans public config, dashboard, admin file
  list and share metadata responses for camelCase keys - the regression
  net for the D7 public-config leak found during review.
2026-09-11 04:24:47 +08:00
Marrrrrrrrry 13f572a86a test: follow public-config snake_case rename in admin-address test 2026-09-11 04:11:47 +08:00
Marrrrrrrrry 800bac0128 fix: snake_case the public config contract (D7 follow-up)
build_public_config/build_public_meta served a mixed camel/snake
contract (allowedFileTypes, meta.features.*, api.legacyConfig,
limits.uploadWindowMinutes). Rename to snake_case; frontend reads
allowed_file_types only.
2026-09-11 04:09:25 +08:00
Marrrrrrrrry 8972344235 docs: rename legacy camelCase config keys to snake_case in guides
Sync zh/en guide docs with migration 007 key names (upload_size,
open_upload, enable_chunk, expire_style, error_count, etc.) so existing
deployments following the docs use the current schema.
2026-09-11 03:38:52 +08:00
Marrrrrrrrry 6e7580b8dc docs: refresh API reference - real version, snake_case fields, EN presign page
API version 2.1.0 -> 2.5.6; response examples use the single message
field; dashboard/list examples and params match current snake_case
contract; drop stale save_path from presign init response; add English
presign-upload page and sidebar entry; uploadSize -> upload_size.
2026-09-11 03:37:24 +08:00
Marrrrrrrrry c7b5eabe7c refactor: single canonical snake_case keys across admin/share APIs (D7)
Drop the message/msg envelope twin, all camel+snake dual-field response
pairs (file list/detail/policy/storage, dashboard counters, batch op
counts, status insights, presets, metadata), and camel request fields
(clearExpiredAt, downloadLimit, sortBy/sortOrder, maxChars). Stored
KeyValue JSON (metadata, presets, activities) keeps legacy-key read
fallbacks so existing deployments normalize transparently.
2026-09-11 03:10:04 +08:00
Marrrrrrrrry bb40f40a18 refactor: hoist all five upload-entry workflows into FileUploadService (2.3)
share_text/share_file/complete_upload/presign proxy/presign confirm now
delegate quota reservation, storage writes, record creation, rollback and
chunk cleanup to service methods; handlers keep auth, request validation,
rate limiting and response wrapping. validate_file_size and
PRESIGN_SESSION_EXPIRES move to services as shared single sources.
2026-09-11 02:56:13 +08:00
Marrrrrrrrry a49727b63d refactor: move admin update_file orchestration into FileService (2.3) 2026-09-11 02:56:02 +08:00
Marrrrrrrrry 0038636a2a fix: serialize KeyValue JSON read-modify-write sections behind a process lock (D5)
settings/activities/view-presets are whole-blob KeyValue rows; concurrent
admin operations could silently drop each other's writes (last-writer-wins).
Single-worker deployment assumption documented; lock is process-local.
2026-09-09 19:09:14 +08:00
Marrrrrrrrry 3c93718d4b fix: FileCodes.expired_count defaults to -1 (time-based) instead of instantly-expired 0 (D4) 2026-09-09 19:05:26 +08:00
Marrrrrrrrry baecebd81e perf: dashboard counters and storage quota use SQL aggregation (D3)
- quota.get_storage_usage (runs on every upload) now aggregates via Sum
  instead of loading every row
- dashboard simple counters (total/size/expired/active/text/chunked/used,
  yesterday/today, top suffixes, recent 8) moved to SQL; the per-row health
  rules engine keeps its single pass (documented)
2026-09-09 19:04:12 +08:00
Marrrrrrrrry bb95017aa3 fix: save_file accepts plain binary streams; repair admin local file share (D2)
- storage contract drops UploadFile in favor of BinaryIO (+content_type param),
  completing the framework decoupling of the storage layer
- LocalFileClass.read() leaked an open fd and its handle was passed where
  save_file expected UploadFile — /admin/local/share 500ed on every call;
  now reads bytes via with-block and shares correctly (regression tests added)
- OneDrive save_file additionally had save_path called as a function — fixed
2026-09-09 18:53:31 +08:00
Marrrrrrrrry 61d72ab6cc perf: cache settings refresh behind a 2s TTL, force refresh on write paths (D6) 2026-09-09 18:47:40 +08:00
Marrrrrrrrry ec424dd909 perf: make is_config_initialized cheap — no scrypt on the per-request probe (D1) 2026-09-09 18:33:27 +08:00
Marrrrrrrrry 5121db6edc feat: rename all config keys to snake_case with DB migration 007
- 20 camelCase keys (uploadSize, expireStyle, openUpload, ...) renamed across
  DEFAULT_CONFIG, settings accessors, setup form fields, and ConfigService
- migrations_007 renames keys inside the existing settings KeyValue row,
  preserving all values for existing deployments
- schema guard meta-test keeps future keys declared
2026-09-09 18:09:19 +08:00
Marrrrrrrrry c5d525df08 feat: declare missing config keys (opendal_scheme, chunk_expire_hours) and add config schema guard
- opendal_scheme was read directly by the OpenDAL backend but never declared
  in DEFAULT_CONFIG: selecting that backend crashed with AttributeError
- chunk_expire_hours existed only as getattr fallbacks in two modules
- refresh_settings now warns about unknown user_config keys
- meta-test scans the codebase so any future settings.X reference without a
  DEFAULT_CONFIG declaration fails the suite
2026-09-09 17:43:28 +08:00
Marrrrrrrrry 61724cbf32 chore: warn at startup when multi-worker deployment weakens in-process rate limiting 2026-09-09 17:37:06 +08:00
Marrrrrrrrry 6587ec86be refactor: extract setup wizard and public pages from main.py (945 -> 148 lines)
- apps/base/setup_wizard.py: form parsing rules, setup options, and the
  two admin-facing HTML pages
- apps/base/pages.py: setup/assets/index/robots/public-config routes and
  theme resolution (404 handler registered on the app in main.py)
- tests updated to import from the new modules
2026-09-09 17:25:42 +08:00
Marrrrrrrrry 832266101e refactor: storage returns StoredDownload data; view layer builds starlette responses 2026-09-09 17:13:34 +08:00
Marrrrrrrrry ad3d319362 refactor: storage raises framework-neutral StorageError rendered identically by app handler 2026-09-09 17:07:43 +08:00
Marrrrrrrrry 892fb24ffc refactor: storage layer accepts plain StoredFile refs and caller-fetched chunk records (core/ is now ORM-free) 2026-09-09 17:05:38 +08:00
Marrrrrrrrry 3396644da3 refactor: move config and tasks modules from core to apps/base (core no longer imports apps) 2026-09-09 17:00:42 +08:00
Marrrrrrrrry 81d33b70de refactor: extract shared chunk-record fetch and hash verification into storage base class 2026-09-09 16:41:29 +08:00
Marrrrrrrrry a75936d85a refactor: use rollback_saved_file helper in presign proxy/confirm workflows 2026-09-09 16:39:42 +08:00
Marrrrrrrrry 357ab2fb6f refactor: extract FileUploadService into apps/base/services.py with shared rollback helper 2026-09-09 16:38:37 +08:00
Marrrrrrrrry 443cfe7472 feat: upgrade password hashing to memory-hard scrypt with transparent rehash
- new hashes use scrypt (n=2^14, r=8, p=1); sha256 and plaintext formats
  keep verifying unchanged, so existing deployments are unaffected
- on successful login a legacy-stored admin password is rehashed to scrypt
  in the settings KeyValue row
- add password_needs_rehash() and extend scheme detection; stdlib only, no
  new dependency
2026-09-09 16:33:07 +08:00
Marrrrrrrrry bde1fee17e docs: document lockfile regeneration flow in dependabot config 2026-09-09 16:28:36 +08:00
Marrrrrrrrry 964cba3636 chore: install dependencies from hashed lockfile in Docker build 2026-09-09 16:28:14 +08:00
Marrrrrrrrry 4904695955 chore: add hashed universal lockfile (uv pip compile --generate-hashes) 2026-09-09 16:27:38 +08:00
Marrrrrrrrry 4cab625d5a fix: validate background config as http(s) URL to close CSS quote-breakout in 2023 theme 2026-09-09 16:21:35 +08:00
Marrrrrrrrry a8c4072497 test: cover presigned upload journey (proxy mode auto-confirm, status, cancel, size limit) 2026-09-09 16:17:41 +08:00
Marrrrrrrrry 2bc278a58e test: cover merge_chunks failure modes and temp-file hygiene 2026-09-09 13:51:36 +08:00
Marrrrrrrrry 3f58e42be1 test: cover the three data-deleting cleanup tasks (expired files, stale chunks, presign sessions) 2026-09-09 13:42:11 +08:00
Marrrrrrrrry 625e50ca95 test: add end-to-end ASGI journey tests for share, chunk, and auth flows 2026-09-09 13:37:33 +08:00
Marrrrrrrrry 177a212c0f test: add shared test helpers and ASGI fixtures, dedupe 10x boilerplate
- tests/helpers.py: SettingsOverrideMixin + init_memory_db()/close_db()
- tests/conftest.py: httpx ASGI client fixtures (db, client, initialized_client)
- enable pytest asyncio_mode=auto
- replace 6 duplicated SettingsOverrideMixin copies and 4 duplicated
  Tortoise.init blocks across the legacy unittest suite
2026-09-08 17:47:21 +08:00
Marrrrrrrrry d701c6e00e chore: engineering hardening - test tooling, lint, security fix, and cleanup
- add pyproject.toml with pytest config and dev dependency group (PEP 735)
- add ruff (core correctness rules) + pre-commit hook
- fix stored XSS: html.escape site config before index template injection,
  with regression tests
- log previously swallowed cleanup exceptions in upload rollback paths
- consolidate three duplicate path generators into build_file_path
  (also aligns chunk/presign date dirs to get_now() instead of local time)
- drop redundant ip_limit sync in load_config (refresh_settings covers all limiters)
- remove dead code (calculate_file_hash, commented-out block), two orphan
  guide docs, and stale .gitignore entries
- add dependabot for pip, docker, and github-actions
- bump starlette 1.3.1 -> 1.6.0 (no breaking API changes; verified with the
  full test suite plus an ASGI smoke test incl. Range downloads)
- skip theme asset tests when themes/ is not built locally
2026-09-07 23:43:50 +08:00
github-actions[bot] 6c76aabe1e chore(master): release 2.5.6 (#509)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-01 16:08:44 +00:00
vastsa e08555a6e8 Merge fix/prod-log-noise-20260901 2026-09-01 23:56:25 +08:00
vastsa 5537f56b88 fix(logging): reduce production log noise 2026-09-01 23:56:08 +08:00
github-actions[bot] 00f1f1d8da chore(master): release 2.5.5 (#507)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-29 04:44:30 +00:00
Lan 5a7195c5c7 Merge pull request #506 from vastsa/fix/pr505-hardened
fix: harden chunk upload filename metadata
2026-08-29 12:44:06 +08:00
vastsa da1dd06655 fix(chunk-upload): persist sanitized filenames 2026-08-29 12:38:46 +08:00
vastsa d510aa7f57 Merge remote-tracking branch 'contributor/pr505' into fix/pr505-hardened 2026-08-29 12:32:00 +08:00
zhou-kkk cebac32145 fix: sanitize chunk upload file_name to prevent path traversal 2026-08-29 12:13:39 +08:00
vastsa f76d1e9841 docs: refresh readme and docs screenshots
Replace legacy PNG assets with new WebP banners and UI captures, and point Chinese/English readme plus docs pages at the updated images.
2026-07-24 16:34:24 +08:00
github-actions[bot] 287b647da1 chore(master): release 2.5.4 (#501)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-23 14:35:42 +00:00
Lan fbcd4a5b2b Merge pull request #500 from vastsa/fix/docker-run-as-root
fix: run Docker as root for volume compatibility
2026-07-23 22:35:20 +08:00
vastsa fc0cbeb4f0 fix: run Docker image as root for data volume compatibility
Revert the non-root container user so existing deployments with
root-owned data volumes keep working without a manual chown step.
2026-07-23 22:35:00 +08:00
github-actions[bot] 24bf6e1a81 chore(master): release 2.5.3 (#499)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-23 14:23:38 +00:00
Lan 0ee2f046dd Merge pull request #498 from vastsa/fix/security-hardening-gaps
fix: magic-byte file checks, admin login rate limit, non-root Docker
2026-07-23 22:23:11 +08:00
vastsa eda0a3127a fix: add magic-byte checks, login rate limits, and non-root Docker
Close the remaining security gaps: validate upload content against
file signatures, rate-limit admin login failures with configurable
thresholds, and run the container as uid 1000 instead of root.
2026-07-23 22:23:04 +08:00
171 changed files with 12441 additions and 4129 deletions

No files matched your search

+18
View File
@@ -0,0 +1,18 @@
version: 2
updates:
# Runtime deps. requirements.txt pins direct deps; requirements.lock.txt is
# the hashed, transitive-resolved lockfile. Dependabot raises version PRs
# against requirements.txt; regenerate the lockfile in the same PR
# (uv pip compile requirements.txt --generate-hashes --universal -o requirements.lock.txt).
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
Binary file not shown.

Before

Width:  |  Height:  |  Size: 571 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 557 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 205 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 679 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 684 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 411 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 431 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 579 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 201 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 254 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 514 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 674 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 678 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 219 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 192 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 233 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 171 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

+67
View File
@@ -0,0 +1,67 @@
name: CI
on:
push:
branches:
- master
- dev
pull_request:
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
test:
name: Lint and test
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
# The hashed lockfile is the deploy artifact; install from it with
# --require-hashes so CI fails on lockfile drift, exactly like the
# Docker build does.
pip install --require-hashes -r requirements.lock.txt
pip install pytest pytest-asyncio httpx 'moto[s3,server]' mypy
- name: Ruff
run: pipx run ruff==0.16.6 check .
- name: Mypy ratchet (new type errors are rejected; fix and run
scripts/mypy_ratchet.py --regenerate to tighten the baseline)
run: python scripts/mypy_ratchet.py
- name: Verify lockfile matches requirements.txt
# Dependabot bumps requirements.txt but cannot regenerate the hashed
# lockfile; without this check a stale lockfile would silently keep
# the Docker build on old versions.
run: |
python - <<'PY'
import re, sys
pins = dict(re.findall(r'^([\w-]+)==([\w.]+)(?:\s*;[^#\n]*)?$', open('requirements.txt').read(), re.M))
lock = open('requirements.lock.txt').read()
stale = []
for name, ver in pins.items():
m = re.search(rf'(?mi)^{re.escape(name)}==([\w.]+)\b', lock)
if m is None or m.group(1) != ver:
stale.append((name, ver, m.group(1) if m else 'ABSENT'))
for name, req_ver, lock_ver in stale:
print(f'STALE LOCKFILE: {name} requirements.txt={req_ver} lock={lock_ver}')
sys.exit(1 if stale else 0)
PY
- name: Run tests
run: pytest -q
+1 -1
View File
@@ -92,7 +92,7 @@ jobs:
fi
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
+3 -3
View File
@@ -37,13 +37,13 @@ jobs:
version: 9
# - uses: oven-sh/setup-bun@v1 # 如果使用 Bun,请取消注释
- name: Setup Node
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: 20
cache: 'pnpm'
cache-dependency-path: 'docs/pnpm-lock.yaml'
- name: Setup Pages
uses: actions/configure-pages@v4
uses: actions/configure-pages@v6
- name: Install dependencies
working-directory: docs
run: pnpm install
@@ -51,7 +51,7 @@ jobs:
working-directory: docs
run: pnpm run docs:build
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
uses: actions/upload-pages-artifact@v5
with:
path: docs/.vitepress/dist
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
steps:
- name: Create or update release
id: release
uses: googleapis/release-please-action@v4
uses: googleapis/release-please-action@v5
with:
config-file: release-please-config.json
manifest-file: .release-please-manifest.json
+2 -7
View File
@@ -80,8 +80,8 @@ docs/_build/
.pybuilder/
target/
*.db
./filecodebox.db-shm
./filecodebox.db-wal
*.db-shm
*.db-wal
# Jupyter Notebook
.ipynb_checkpoints
@@ -147,13 +147,8 @@ cython_debug/
# Project
.vscode
.DS_Store
for_test.py
.html
/evaluate/temp.py
/evaluation/back.json
data/.env
.backup/
/cloc-1.64.exe
# Ignore node_modules
node_modules/
+8
View File
@@ -0,0 +1,8 @@
# First-time setup: pre-commit install
# Behind a proxy: HTTPS_PROXY=http://<proxy-host>:<port> pre-commit run --all-files
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.6
hooks:
- id: ruff-check
args: [--fix]
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "2.5.2"
".": "2.8.5"
}
+11 -5
View File
@@ -54,27 +54,33 @@ RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
echo 'Asia/Shanghai' > /etc/timezone
# 从构建阶段复制编译好的前端主题
COPY --from=frontend-builder /build/fronted-2024/dist ./themes/2024
COPY --from=frontend-builder /build/FileCodeBox/themes/2024 ./themes/2024
COPY --from=frontend-builder /build/fronted-2023/dist ./themes/2023
# 安装系统安全更新 + Python 依赖
# 依赖从带哈希的锁定文件安装(--require-hashes),保证构建可复现、防供应链篡改。
# 清理 apt 缓存,降低镜像噪音与扫描面
RUN apt-get update \
&& apt-get upgrade -y --no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir -r requirements.txt \
&& pip install --no-cache-dir --require-hashes -r requirements.lock.txt \
&& pip cache purge || true
# 环境变量配置
ENV HOST="0.0.0.0" \
PORT=12345 \
WORKERS=1 \
LOG_LEVEL="info" \
FORWARDED_ALLOW_IPS=""
APP_ENV="production" \
LOG_LEVEL="warning" \
ACCESS_LOG="false" \
FORWARDED_ALLOW_IPS="" \
TIMEOUT_KEEP_ALIVE=5 \
MALLOC_TRIM_THRESHOLD_=65536
EXPOSE 12345
# 生产环境启动命令
# FORWARDED_ALLOW_IPS 默认为空:仅信任直连 IP,避免任意客户端伪造 X-Forwarded-*。
# 若前面有反向代理,请显式设置为代理网段,例如 "10.0.0.0/8,172.16.0.0/12"。
CMD ["sh", "-c", "exec uvicorn main:app --host \"$HOST\" --port \"$PORT\" --workers \"$WORKERS\" --log-level \"$LOG_LEVEL\" --proxy-headers --forwarded-allow-ips \"${FORWARDED_ALLOW_IPS:-}\""]
# 容器以 root 运行:兼容存量 root 属主的数据卷,以及 NAS/只读 bind mount。
CMD ["sh", "-c", "access_log_arg=--no-access-log; if [ \"${APP_ENV:-development}\" != \"production\" ] || [ \"${ACCESS_LOG:-false}\" = \"true\" ]; then access_log_arg=--access-log; fi; exec uvicorn main:app --host \"$HOST\" --port \"$PORT\" --workers \"$WORKERS\" --log-level \"$LOG_LEVEL\" \"$access_log_arg\" --timeout-keep-alive \"${TIMEOUT_KEEP_ALIVE:-5}\" --proxy-headers --forwarded-allow-ips \"${FORWARDED_ALLOW_IPS:-}\""]
+1 -1
View File
@@ -1 +1 @@
2.5.2
2.8.5
+160
View File
@@ -0,0 +1,160 @@
"""System config write path (ConfigService) + the KeyValue JSON lock.
keyvalue_write_lock lives here because ConfigService.update_config is its
primary consumer; FileService imports it (one-way, no cycle).
"""
import asyncio
from fastapi import HTTPException
from core.settings import (
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
)
from apps.base.config import refresh_settings
from core.security import (
INTERNAL_CONFIG_KEYS,
OUTBOUND_ENDPOINT_CONFIG_KEYS,
generate_jwt_secret,
validate_outbound_endpoint,
validate_outbound_hostname,
)
from apps.base.models import KeyValue
from core.utils import hash_password, is_password_hashed, validate_background_url
# KeyValue 里的 settings/activities/presets 都是整块 JSON 读-改-写;
# 进程内写锁串行化这三个写路径,避免并发管理操作互相覆盖(last-writer-wins)。
# 多进程部署下锁不跨进程——文档已锁定单 worker 部署。
keyvalue_write_lock = asyncio.Lock()
class ConfigService:
INT_FIELDS = {
"admin_session_expire",
"enable_chunk",
"error_count",
"error_minute",
"login_count",
"login_minute",
"max_save_seconds",
"onedrive_proxy",
"open_upload",
"mcp_require_token",
"port",
"s3_proxy",
"server_port",
"server_workers",
"show_admin_addr",
"storage_limit",
"upload_count",
"upload_minute",
"upload_size",
"webdav_proxy",
}
FLOAT_FIELDS = {"opacity"}
def get_config(self):
config = dict(settings.items())
config["admin_token"] = ""
for key in INTERNAL_CONFIG_KEYS:
config.pop(key, None)
return config
async def update_config(self, data: dict):
current_config = dict(settings.items())
next_config = dict(current_config)
update_data = {
key: value
for key, value in data.items()
if key in settings.default_config and key not in INTERNAL_CONFIG_KEYS
}
admin_token = update_data.get("admin_token")
admin_password_changed = False
if admin_token is None or admin_token == "":
update_data.pop("admin_token", None)
elif not is_password_hashed(admin_token):
update_data["admin_token"] = hash_password(admin_token)
admin_password_changed = True
else:
admin_password_changed = True
for key, value in update_data.items():
if value == "" and key in self.INT_FIELDS | self.FLOAT_FIELDS:
continue
try:
if key in self.INT_FIELDS:
next_config[key] = int(value)
elif key in self.FLOAT_FIELDS:
next_config[key] = float(value)
else:
next_config[key] = value
except (TypeError, ValueError):
raise HTTPException(status_code=400, detail=f"{key} 配置值格式错误")
try:
session_expire = int(str(next_config.get("admin_session_expire")))
except (TypeError, ValueError):
raise HTTPException(
status_code=400,
detail="admin_session_expire 配置值格式错误",
)
if (
not ADMIN_SESSION_EXPIRE_MIN <= session_expire <= ADMIN_SESSION_EXPIRE_MAX
or session_expire % ADMIN_SESSION_EXPIRE_MIN != 0
):
raise HTTPException(
status_code=400,
detail="admin_session_expire 必须是 1 到 365 个整天",
)
next_config["admin_session_expire"] = session_expire
if int(next_config.get("storage_limit", 0)) < 0:
raise HTTPException(
status_code=400,
detail="storage_limit 不能小于 0",
)
# 只校验"发生变化"的值:升级前存入的旧格式 background(相对路径、含空格
# 或括号)在旧版本是合法的,若每次保存都重新校验,存量部署会连无关设置项
# 都保存不了(一律 400)。渲染侧仍然 html 转义,而任何修改都必须通过校验。
current_background = str(settings.background or "")
candidate_background = str(next_config.get("background") or "")
if candidate_background != current_background:
try:
validate_background_url(candidate_background)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
# 只校验"发生变化"的值:企业内网 minio/webdav 是正当场景,存量部署历史
# 合法写入的内网 endpoint 若每次保存都重新校验,会连无关设置都保存不了
# (background 曾有同款回归,上游 #528 修复过——本处沿用同一语义)。
# s3_hostname 是裸主机名(存储层按 https://{hostname} 拼接),单独分档校验。
for endpoint_key in OUTBOUND_ENDPOINT_CONFIG_KEYS:
if endpoint_key not in next_config:
continue
candidate = str(next_config[endpoint_key] or "")
current = str(getattr(settings, endpoint_key, "") or "")
if candidate == current:
continue
validator = (
validate_outbound_hostname
if endpoint_key == "s3_hostname"
else validate_outbound_endpoint
)
try:
# 写回规范化值(validator 去除首尾空白):校验通过但入库脏值
# 会让存储层在连接期才报错,应在校验点归一。
next_config[endpoint_key] = validator(candidate)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
if admin_password_changed:
next_config["jwt_secret"] = generate_jwt_secret()
async with keyvalue_write_lock:
await KeyValue.update_or_create(key="settings", defaults={"value": next_config})
await refresh_settings(force=True)
+32 -139
View File
@@ -2,127 +2,40 @@
# @Author : Lan
# @File : depends.py
# @Software: PyCharm
from fastapi import Header, HTTPException, Depends
from fastapi import Header
from fastapi.requests import Request
import base64
import hmac
import json
import time
from core.settings import (
ADMIN_SESSION_EXPIRE_DEFAULT,
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
from apps.admin.services import ConfigService, FileService, LocalFileService
from apps.base.auth import (
_extract_bearer_token,
_get_jwt_secret,
_require_admin_payload,
create_token,
get_admin_session_expire_seconds,
share_required_login,
verify_token,
)
from apps.admin.services import FileService, ConfigService, LocalFileService
# 认证原语(JWT 创建/校验、Bearer 提取、分享上传门控)已下沉 apps.base.auth
# ——base 分享面与本模块共同消费,留在 base 避免 base→admin 反向依赖。
# 此处 re-export 保持既有 import 路径兼容(tests/admin.views 等)。
def _get_jwt_secret() -> bytes:
secret = getattr(settings, "jwt_secret", "")
if not secret:
raise RuntimeError("JWT签名密钥未初始化")
return secret.encode()
def get_admin_session_expire_seconds() -> int:
try:
expires_in = int(
getattr(settings, "adminSessionExpire", ADMIN_SESSION_EXPIRE_DEFAULT)
)
except (TypeError, ValueError):
return ADMIN_SESSION_EXPIRE_DEFAULT
if (
not ADMIN_SESSION_EXPIRE_MIN <= expires_in <= ADMIN_SESSION_EXPIRE_MAX
or expires_in % ADMIN_SESSION_EXPIRE_MIN != 0
):
return ADMIN_SESSION_EXPIRE_DEFAULT
return expires_in
def create_token(data: dict, expires_in: int | None = None) -> str:
"""
创建JWT token
:param data: 数据负载
:param expires_in: 过期时间(秒)
"""
token_lifetime = (
get_admin_session_expire_seconds() if expires_in is None else expires_in
)
header = base64.urlsafe_b64encode(
json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode()
).decode().rstrip("=")
payload = base64.urlsafe_b64encode(
json.dumps(
{**data, "exp": int(time.time()) + token_lifetime},
separators=(",", ":"),
).encode()
).decode().rstrip("=")
signature = hmac.new(
_get_jwt_secret(), f"{header}.{payload}".encode(), "sha256"
).digest()
signature = base64.urlsafe_b64encode(signature).decode().rstrip("=")
return f"{header}.{payload}.{signature}"
def verify_token(token: str) -> dict:
"""
验证JWT token
:param token: JWT token
:return: 解码后的数据
"""
try:
header_b64, payload_b64, signature_b64 = token.split(".")
# 验证签名
expected_signature = hmac.new(
_get_jwt_secret(),
f"{header_b64}.{payload_b64}".encode(),
"sha256",
).digest()
expected_signature_b64 = (
base64.urlsafe_b64encode(expected_signature).decode().rstrip("=")
)
if not hmac.compare_digest(signature_b64, expected_signature_b64):
raise ValueError("无效的签名")
# 解码payload(兼容历史标准 base64 与 urlsafe base64)
padded = payload_b64 + "=" * (-len(payload_b64) % 4)
try:
payload_bytes = base64.urlsafe_b64decode(padded)
except Exception:
payload_bytes = base64.b64decode(padded)
payload = json.loads(payload_bytes)
# 检查是否过期
if payload.get("exp", 0) < time.time():
raise ValueError("token已过期")
return payload
except Exception as e:
raise ValueError(f"token验证失败: {str(e)}")
def _extract_bearer_token(authorization: str) -> str:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
token = authorization.split(" ", 1)[1].strip()
if not token:
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return token
def _require_admin_payload(authorization: str) -> dict:
token = _extract_bearer_token(authorization)
try:
payload = verify_token(token)
except ValueError as e:
raise HTTPException(status_code=401, detail=str(e))
if not payload.get("is_admin", False):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return payload
__all__ = [
"_extract_bearer_token",
"_get_jwt_secret",
"_require_admin_payload",
"ADMIN_PUBLIC_ENDPOINTS",
"admin_required",
"create_token",
"get_admin_session",
"get_admin_session_expire_seconds",
"get_config_service",
"get_file_service",
"get_local_file_service",
"share_required_login",
"verify_token",
]
def get_admin_session(authorization: str = Header(default=None)) -> dict:
@@ -141,7 +54,8 @@ ADMIN_PUBLIC_ENDPOINTS = {("POST", "/admin/login")}
async def admin_required(
authorization: str = Header(default=None), request: Request = None
authorization: str = Header(default=None),
request: Request = None, # type: ignore[assignment] # FastAPI 运行时注入 Request;可选语义由下方 if request 判断
):
"""
验证管理员权限
@@ -151,27 +65,6 @@ async def admin_required(
return _require_admin_payload(authorization)
async def share_required_login(authorization: str = Header(default=None)):
"""
验证分享上传权限
当settings.openUpload为False时,要求用户必须登录并具有管理员权限
当settings.openUpload为True时,允许游客上传
:param authorization: 认证头信息
:param request: 请求对象
:return: 验证结果
"""
if not settings.openUpload:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=403, detail="本站未开启游客上传,如需上传请先登录后台"
)
_require_admin_payload(authorization)
return True
async def get_file_service():
return FileService()
+98
View File
@@ -0,0 +1,98 @@
"""Admin-side local (NAS) file browsing and deletion."""
from pathlib import Path
from typing import Any
from fastapi import HTTPException
from apps.base.local_share import (
MAX_LIST_ENTRIES,
format_local_ctime,
get_local_root,
normalize_local_relpath,
resolve_under_local,
)
class LocalFileService:
async def list_files(self, path: str = ""):
relpath = normalize_local_relpath(path, allow_empty=True)
directory = resolve_under_local(relpath)
if not directory.exists() or not directory.is_dir():
raise HTTPException(status_code=404, detail="目录不存在")
root = get_local_root()
items: list[dict[str, Any]] = []
try:
children = list(directory.iterdir())
except OSError as exc:
raise HTTPException(status_code=500, detail="无法读取目录") from exc
children.sort(key=lambda p: (not p.is_dir(), p.name.lower()))
truncated = False
for child in children:
if len(items) >= MAX_LIST_ENTRIES:
truncated = True
break
try:
resolved = child.resolve()
resolved.relative_to(root)
except (OSError, ValueError):
continue
if not resolved.is_file() and not resolved.is_dir():
continue
child_rel = child.name if not relpath else f"{relpath}/{child.name}"
is_dir = resolved.is_dir()
items.append(
{
"file": child.name,
"name": child.name,
"path": child_rel,
"type": "dir" if is_dir else "file",
"ctime": format_local_ctime(resolved),
"size": None if is_dir else resolved.stat().st_size,
}
)
parent = ""
if relpath:
parent_path = Path(relpath).parent.as_posix()
parent = "" if parent_path == "." else parent_path
return {
"path": relpath,
"parent": parent,
"truncated": truncated,
"items": items,
}
async def delete_file(self, filename: str):
file = LocalFileClass(filename)
if await file.exists():
await file.delete()
return "删除成功"
raise HTTPException(status_code=404, detail="文件不存在")
class LocalFileClass:
def __init__(self, file):
relpath = normalize_local_relpath(file)
self.file = relpath
self.name = Path(relpath).name
self.path = resolve_under_local(relpath)
if self.path.is_file():
self.ctime = format_local_ctime(self.path)
self.size = self.path.stat().st_size
else:
self.ctime = None
self.size = None
async def read(self) -> bytes:
with open(self.path, "rb") as fh:
return fh.read()
async def delete(self):
if not self.path.is_file():
raise HTTPException(status_code=404, detail="文件不存在")
self.path.unlink()
async def exists(self):
return self.path.is_file()
-3
View File
@@ -39,21 +39,18 @@ class BatchUpdateFileData(BaseModel):
ids: list[int]
expired_at: Optional[Union[datetime.datetime, str]] = None
expired_count: Optional[int] = None
clearExpiredAt: Optional[bool] = None
clear_expired_at: Optional[bool] = None
class FilePolicyActionData(BaseModel):
id: int
action: str
downloadLimit: Optional[int] = None
download_limit: Optional[int] = None
class BatchFilePolicyActionData(BaseModel):
ids: list[int]
action: str
downloadLimit: Optional[int] = None
download_limit: Optional[int] = None
+206 -420
View File
File diff suppressed because it is too large. Load diff
+122 -106
View File
@@ -3,6 +3,7 @@
# @File : views.py
# @Software: PyCharm
import datetime
from typing import Any
from collections import Counter
from typing import Optional
@@ -31,13 +32,18 @@ from apps.admin.schemas import (
)
from core.response import APIResponse
from apps.base.models import FileCodes, KeyValue
from apps.base.quota import owned_storage_queryset
from tortoise.expressions import Q
from tortoise.functions import Count, Sum
from apps.admin.dependencies import (
create_token,
get_admin_session_expire_seconds,
verify_token,
)
from core.logger import logger
from core.settings import settings
from core.utils import get_now, verify_password
from core.utils import get_now, hash_password, password_needs_rehash, verify_password
from apps.base.utils import ip_limit
admin_api = APIRouter(
prefix="/admin", tags=["管理"], dependencies=[Depends(admin_required)]
@@ -53,10 +59,26 @@ def _pick_query_text(*values: Optional[str]) -> Optional[str]:
@admin_api.post("/login")
async def login(data: LoginData):
async def login(data: LoginData, ip: str = Depends(ip_limit["login"])):
# 登录失败计入 IP 频率限制,超过 login_count/login_minute 后暂时锁定
if not verify_password(data.password, settings.admin_token):
ip_limit["login"].add_ip(ip)
raise HTTPException(status_code=401, detail="密码错误")
# 透明重哈希:存量 sha256/明文口令在登录成功(已证明持有口令)时升级为 scrypt
if password_needs_rehash(settings.admin_token):
try:
record = await KeyValue.filter(key="settings").first()
if record:
stored = dict(record.value or {})
stored["admin_token"] = hash_password(data.password)
record.value = stored
await record.save()
settings.admin_token = stored["admin_token"]
logger.info("管理员口令已升级为 scrypt 哈希")
except Exception:
logger.warning("口令哈希升级失败,保持旧格式", exc_info=True)
expires_in = get_admin_session_expire_seconds()
token = create_token({"is_admin": True}, expires_in=expires_in)
return APIResponse(
@@ -90,81 +112,103 @@ async def build_dashboard_recent_file(file_code: FileCodes) -> dict:
"suffix": file_code.suffix,
"size": file_code.size,
"text": file_code.text is not None,
"expiredAt": file_code.expired_at,
"expiredCount": file_code.expired_count,
"usedCount": file_code.used_count,
"createdAt": file_code.created_at,
"isExpired": is_expired,
"expired_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"created_at": file_code.created_at,
"is_expired": is_expired,
}
def _expired_predicate(now: datetime.datetime) -> Q:
"""SQL version of FileCodes.is_expired:
expired_at IS NOT NULL AND ((expired_count < 0 AND expired_at < now) OR expired_count = 0)
"""
return Q(expired_at__isnull=False) & (
Q(expired_count__lt=0, expired_at__lt=now) | Q(expired_count=0)
)
async def _sum_size(queryset) -> int:
rows = await queryset.annotate(total=Sum("size")).values("total")
return rows[0]["total"] or 0
@admin_api.get("/dashboard")
async def dashboard(file_service: FileService = Depends(get_file_service)):
all_codes = await FileCodes.all()
all_size = sum([code.size for code in all_codes])
sys_start = await KeyValue.filter(key="sys_start").first()
now = await get_now()
today_start = now.replace(hour=0, minute=0, second=0, microsecond=0)
yesterday_start = today_start - datetime.timedelta(days=1)
yesterday_end = today_start - datetime.timedelta(microseconds=1)
yesterday_codes = FileCodes.filter(
created_at__gte=yesterday_start, created_at__lte=yesterday_end
)
today_codes = FileCodes.filter(created_at__gte=today_start)
yesterday_file_codes = await yesterday_codes
today_file_codes = await today_codes
expired_count = 0
for file_code in all_codes:
if await file_code.is_expired():
expired_count += 1
health_summary = await file_service.build_file_health_summary(all_codes, now=now)
text_count = sum(1 for file_code in all_codes if file_code.text is not None)
chunked_count = sum(1 for file_code in all_codes if file_code.is_chunked)
used_count = sum([file_code.used_count for file_code in all_codes])
# 简单计数全部走 SQL 聚合,避免整表载入内存(D3)。
# 健康摘要依赖逐行 rules engine(_build_file_status_insights),保留一次遍历。
total_files = await FileCodes.all().count()
expired_count = await FileCodes.filter(_expired_predicate(now)).count()
all_size = await _sum_size(owned_storage_queryset())
used_count = (
await FileCodes.all().annotate(total=Sum("used_count")).values("total")
)[0]["total"] or 0
text_count = await FileCodes.filter(text__isnull=False).count()
chunked_count = await FileCodes.filter(is_chunked=True).count()
yesterday_count = await FileCodes.filter(
created_at__gte=yesterday_start, created_at__lte=yesterday_end
).count()
yesterday_size = await _sum_size(
owned_storage_queryset(
FileCodes.filter(created_at__gte=yesterday_start, created_at__lte=yesterday_end)
)
)
today_count = await FileCodes.filter(created_at__gte=today_start).count()
today_size = await _sum_size(
owned_storage_queryset(FileCodes.filter(created_at__gte=today_start))
)
suffix_rows = (
await FileCodes.filter(text__isnull=True)
.annotate(count=Count("id"))
.values("suffix", "count")
)
suffix_counter = Counter(
"Text" if file_code.text is not None else (file_code.suffix or "file")
for file_code in all_codes
{((row["suffix"] or "file") or "file"): row["count"] for row in suffix_rows}
)
recent_file_codes = await FileCodes.all().order_by("-created_at").limit(8)
health_summary = await file_service.build_file_health_summary(
await FileCodes.all(), now=now
)
recent_file_codes = sorted(
all_codes,
key=lambda file_code: file_code.created_at.timestamp()
if file_code.created_at
else 0,
reverse=True,
)[:8]
recent_activities = await file_service.list_admin_activities(limit=8)
return APIResponse(
detail={
"totalFiles": len(all_codes),
"storageUsed": str(all_size),
"sysUptime": sys_start.value if sys_start else None,
"yesterdayCount": len(yesterday_file_codes),
"yesterdaySize": str(sum([code.size for code in yesterday_file_codes])),
"todayCount": len(today_file_codes),
"todaySize": str(sum([code.size for code in today_file_codes])),
"activeCount": len(all_codes) - expired_count,
"expiredCount": expired_count,
"textCount": text_count,
"fileCount": len(all_codes) - text_count,
"chunkedCount": chunked_count,
"usedCount": used_count,
"storageBackend": settings.file_storage,
"uploadSizeLimit": settings.uploadSize,
"openUpload": settings.openUpload,
"enableChunk": settings.enableChunk,
"maxSaveSeconds": settings.max_save_seconds,
"total_files": total_files,
"storage_used": str(all_size),
"sys_uptime": sys_start.value if sys_start else None,
"yesterday_count": yesterday_count,
"yesterday_size": str(yesterday_size),
"today_count": today_count,
"today_size": str(today_size),
"active_count": total_files - expired_count,
"expired_count": expired_count,
"text_count": text_count,
"file_count": total_files - text_count,
"chunked_count": chunked_count,
"used_count": used_count,
"storage_backend": settings.file_storage,
"upload_size_limit": settings.upload_size,
"open_upload": settings.open_upload,
"enable_chunk": settings.enable_chunk,
"max_save_seconds": settings.max_save_seconds,
**health_summary,
"healthSummary": health_summary,
"topSuffixes": [
"health_summary": health_summary,
"top_suffixes": [
{"suffix": suffix, "count": count}
for suffix, count in suffix_counter.most_common(8)
],
"recentFiles": [
"recent_files": [
await build_dashboard_recent_file(file_code)
for file_code in recent_file_codes
],
"recentActivities": recent_activities["activities"],
"recent_activities": recent_activities["activities"],
}
)
@@ -230,14 +274,15 @@ async def batch_update_files(
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要更新的文件")
update_data = {}
update_data: dict[str, Any] = {}
fields_set = data.model_fields_set
should_clear_expired_at = bool(data.clearExpiredAt or data.clear_expired_at)
should_clear_expired_at = bool(data.clear_expired_at)
if should_clear_expired_at:
update_data["expired_at"] = None
update_data["expired_count"] = -1
elif "expired_at" in fields_set and data.expired_at != "":
# schema 声明 Union[datetime, str]——透传两种形态,tortoise 均可序列化
update_data["expired_at"] = data.expired_at
if (
@@ -274,9 +319,7 @@ async def apply_file_policy_action(
data: FilePolicyActionData,
file_service: FileService,
):
download_limit = data.downloadLimit
if download_limit is None:
download_limit = data.download_limit
download_limit = data.download_limit
detail = await file_service.apply_file_policy_action(
file_id=data.id,
@@ -309,9 +352,7 @@ async def apply_batch_file_policy_action(
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要更新的文件")
download_limit = data.downloadLimit
if download_limit is None:
download_limit = data.download_limit
download_limit = data.download_limit
result = await file_service.apply_files_policy_action(
file_ids=data.ids,
@@ -345,8 +386,8 @@ async def file_list(
status: str = "",
type: str = "",
health: str = "",
sortBy: str = "created_at",
sortOrder: str = "desc",
sort_by: str = "created_at",
sort_order: str = "desc",
file_service: FileService = Depends(get_file_service),
):
page = max(page, 1)
@@ -358,8 +399,8 @@ async def file_list(
status=status,
file_type=type,
health=health,
sort_by=sortBy,
sort_order=sortOrder,
sort_by=sort_by,
sort_order=sort_order,
)
return APIResponse(
detail={
@@ -500,7 +541,7 @@ async def update_config(
config_service: ConfigService = Depends(get_config_service),
file_service: FileService = Depends(get_file_service),
):
data.pop("themesChoices", None)
data.pop("themes_choices", None)
await config_service.update_config(data)
await file_service.record_admin_activity(
action="config.update",
@@ -524,18 +565,19 @@ async def file_download(
@admin_api.get("/file/preview")
async def file_preview(
id: int,
maxChars: int = 4000,
max_chars: int = 4000,
file_service: FileService = Depends(get_file_service),
):
preview = await file_service.preview_file(id, maxChars)
preview = await file_service.preview_file(id, max_chars)
return APIResponse(detail=preview)
@admin_api.get("/local/lists")
async def get_local_lists(
path: str = "",
local_file_service: LocalFileService = Depends(get_local_file_service),
):
files = await local_file_service.list_files()
files = await local_file_service.list_files(path)
return APIResponse(detail=files)
@@ -569,8 +611,8 @@ async def share_local_file(
target_name=item.filename,
count=1,
meta={
"expireValue": item.expire_value,
"expireStyle": item.expire_style,
"expire_value": item.expire_value,
"expire_style": item.expire_style,
},
)
return APIResponse(detail=share_info)
@@ -581,38 +623,12 @@ async def update_file(
data: UpdateFileData,
file_service: FileService = Depends(get_file_service),
):
file_code = await FileCodes.filter(id=data.id).first()
if not file_code:
raise HTTPException(status_code=404, detail="文件不存在")
target_name = file_service._build_file_activity_name(file_code)
update_data = {}
if data.code is not None and data.code != file_code.code:
# 判断code是否存在
if await FileCodes.filter(code=data.code).first():
raise HTTPException(status_code=400, detail="code已存在")
update_data["code"] = data.code
if data.prefix is not None and data.prefix != file_code.prefix:
update_data["prefix"] = data.prefix
if data.suffix is not None and data.suffix != file_code.suffix:
update_data["suffix"] = data.suffix
if (
data.expired_at is not None
and data.expired_at != ""
and data.expired_at != file_code.expired_at
):
update_data["expired_at"] = data.expired_at
if data.expired_count is not None and data.expired_count != file_code.expired_count:
update_data["expired_count"] = data.expired_count
await file_code.update_from_dict(update_data).save()
if update_data:
await file_service.record_admin_activity(
action="file.update",
target_type="file",
target_id=data.id,
target_name=target_name,
count=1,
meta={"fields": sorted(update_data.keys())},
)
await file_service.update_file(
file_id=data.id,
code=data.code,
prefix=data.prefix,
suffix=data.suffix,
expired_at=data.expired_at,
expired_count=data.expired_count,
)
return APIResponse(detail="更新成功")
+150
View File
@@ -0,0 +1,150 @@
"""Cross-cutting auth primitives and the share-upload gate.
Lives in apps.base because BOTH the share surface (base.views) and the
admin surface (admin.views) consume it; keeping it here avoids the
base → admin reverse dependency. Admin-only session gating stays in
apps.admin.dependencies, which re-exports these primitives for
compatibility.
"""
import base64
import hmac
import json
import time
from fastapi import Header, HTTPException
from core.settings import (
ADMIN_SESSION_EXPIRE_DEFAULT,
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
)
def _get_jwt_secret() -> bytes:
secret = getattr(settings, "jwt_secret", "")
if not secret:
raise RuntimeError("JWT签名密钥未初始化")
return secret.encode()
def get_admin_session_expire_seconds() -> int:
try:
expires_in = int(
getattr(settings, "admin_session_expire", ADMIN_SESSION_EXPIRE_DEFAULT)
)
except (TypeError, ValueError):
return ADMIN_SESSION_EXPIRE_DEFAULT
if (
not ADMIN_SESSION_EXPIRE_MIN <= expires_in <= ADMIN_SESSION_EXPIRE_MAX
or expires_in % ADMIN_SESSION_EXPIRE_MIN != 0
):
return ADMIN_SESSION_EXPIRE_DEFAULT
return expires_in
def create_token(data: dict, expires_in: int | None = None) -> str:
"""
创建JWT token
:param data: 数据负载
:param expires_in: 过期时间(秒)
"""
token_lifetime = (
get_admin_session_expire_seconds() if expires_in is None else expires_in
)
header = base64.urlsafe_b64encode(
json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode()
).decode().rstrip("=")
payload = base64.urlsafe_b64encode(
json.dumps(
{**data, "exp": int(time.time()) + token_lifetime},
separators=(",", ":"),
).encode()
).decode().rstrip("=")
signature_digest = hmac.new(
_get_jwt_secret(), f"{header}.{payload}".encode(), "sha256"
).digest()
signature = base64.urlsafe_b64encode(signature_digest).decode().rstrip("=")
return f"{header}.{payload}.{signature}"
def verify_token(token: str) -> dict:
"""
验证JWT token
:param token: JWT token
:return: 解码后的数据
"""
try:
header_b64, payload_b64, signature_b64 = token.split(".")
# 验证签名
expected_signature = hmac.new(
_get_jwt_secret(),
f"{header_b64}.{payload_b64}".encode(),
"sha256",
).digest()
expected_signature_b64 = (
base64.urlsafe_b64encode(expected_signature).decode().rstrip("=")
)
if not hmac.compare_digest(signature_b64, expected_signature_b64):
raise ValueError("无效的签名")
# 解码payload(兼容历史标准 base64 与 urlsafe base64)
padded = payload_b64 + "=" * (-len(payload_b64) % 4)
try:
payload_bytes = base64.urlsafe_b64decode(padded)
except Exception:
payload_bytes = base64.b64decode(padded)
payload = json.loads(payload_bytes)
# 检查是否过期
if payload.get("exp", 0) < time.time():
raise ValueError("token已过期")
return payload
except Exception as e:
raise ValueError(f"token验证失败: {str(e)}")
def _extract_bearer_token(authorization: str) -> str:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
token = authorization.split(" ", 1)[1].strip()
if not token:
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return token
def _require_admin_payload(authorization: str) -> dict:
token = _extract_bearer_token(authorization)
try:
payload = verify_token(token)
except ValueError as e:
raise HTTPException(status_code=401, detail=str(e))
if not payload.get("is_admin", False):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return payload
async def share_required_login(authorization: str = Header(default=None)):
"""
验证分享上传权限
当 settings.open_upload 为False时,要求用户必须登录并具有管理员权限
当 settings.open_upload 为True时,允许游客上传
:param authorization: 认证头信息
:param request: 请求对象
:return: 验证结果
"""
if not settings.open_upload:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=403, detail="本站未开启游客上传,如需上传请先登录后台"
)
_require_admin_payload(authorization)
return True
+42 -18
View File
@@ -16,16 +16,18 @@ MIN_ADMIN_PASSWORD_LENGTH = 8
SETUP_CONFIG_FIELDS = {
"allowed_file_types",
"code_generate_type",
"enableChunk",
"errorCount",
"errorMinute",
"expireStyle",
"enable_chunk",
"error_count",
"error_minute",
"login_count",
"login_minute",
"expire_style",
"max_save_seconds",
"name",
"openUpload",
"uploadCount",
"uploadMinute",
"uploadSize",
"open_upload",
"upload_count",
"upload_minute",
"upload_size",
}
@@ -58,23 +60,45 @@ async def ensure_security_settings() -> None:
logger.info("已将管理员密码迁移为哈希存储")
if security_config.jwt_secret_rotated:
logger.info("已生成独立 JWT 签名密钥")
await refresh_settings()
await refresh_settings(force=True)
def _sync_ip_limits() -> None:
ip_limit["error"].minutes = settings.errorMinute
ip_limit["error"].count = settings.errorCount
ip_limit["metadata"].minutes = settings.errorMinute
ip_limit["metadata"].count = settings.errorCount
ip_limit["upload"].minutes = settings.uploadMinute
ip_limit["upload"].count = settings.uploadCount
ip_limit["error"].minutes = settings.error_minute
ip_limit["error"].count = settings.error_count
ip_limit["metadata"].minutes = settings.error_minute
ip_limit["metadata"].count = settings.error_count
ip_limit["upload"].minutes = settings.upload_minute
ip_limit["upload"].count = settings.upload_count
ip_limit["login"].minutes = settings.login_minute
ip_limit["login"].count = settings.login_count
async def refresh_settings() -> None:
"""从数据库读取最新配置并应用到运行时。"""
# Per-process config cache: the middleware refreshes settings on every request,
# which otherwise means one DB read per request. Writes call refresh_settings(force=True)
# so same-process visibility is immediate; other processes converge within the TTL
# (documented deployment is a single worker).
CONFIG_CACHE_TTL_SECONDS = 2.0
_config_cached_until = 0.0
async def refresh_settings(force: bool = False) -> None:
"""从数据库读取最新配置并应用到运行时。
默认带短 TTL 缓存(避免每请求查库);写配置的代码路径必须传 force=True。
"""
global _config_cached_until
import time as _time
if not force and _time.monotonic() < _config_cached_until:
return
config_record = await KeyValue.filter(key="settings").first()
settings.user_config = config_record.value if config_record and config_record.value else {}
unknown = settings.unknown_keys(settings.user_config)
if unknown:
logger.warning("配置中存在未登记的键(将被忽略其类型语义,仅透传): %s", unknown)
_sync_ip_limits()
_config_cached_until = _time.monotonic() + CONFIG_CACHE_TTL_SECONDS
def is_runtime_initialized() -> bool:
@@ -113,4 +137,4 @@ async def initialize_system(
next_config["jwt_secret"] = generate_jwt_secret()
await KeyValue.update_or_create(key="settings", defaults={"value": next_config})
await refresh_settings()
await refresh_settings(force=True)
+56 -23
View File
@@ -1,13 +1,14 @@
import time
from collections import OrderedDict
from ipaddress import ip_address, ip_network
from typing import Dict, Iterable, Union
from datetime import datetime, timedelta
from typing import Iterable, TypedDict
from fastapi import HTTPException, Request
from core.settings import settings
def _iter_trusted_proxies() -> Iterable[str]:
trusted_proxies = getattr(settings, "trustedProxies", [])
trusted_proxies = getattr(settings, "trusted_proxies", [])
if isinstance(trusted_proxies, str):
trusted_proxies = [item.strip() for item in trusted_proxies.split(",")]
return [item for item in trusted_proxies if item]
@@ -65,36 +66,68 @@ def get_client_ip(request: Request) -> str:
return client_host
class _IPRecord(TypedDict):
count: int
last_seen: float
# Upper bound for tracked source IPs per limiter. Without it a scan using
# spoofed/rotating addresses grows the dict forever on a small VPS.
DEFAULT_MAX_TRACKED_IPS = 10_000
class IPRateLimit:
def __init__(self, count: int, minutes: int):
self.ips: Dict[str, Dict[str, Union[int, datetime]]] = {}
def __init__(
self,
count: int,
minutes: int,
max_ips: int = DEFAULT_MAX_TRACKED_IPS,
):
# Ordered by last activity (oldest first) so capacity eviction and the
# periodic cleanup are both O(1)/O(expired) instead of a full rebuild.
self.ips: OrderedDict[str, _IPRecord] = OrderedDict()
self.count = count
self.minutes = minutes
self.max_ips = max(1, max_ips)
def _window_seconds(self) -> float:
# minutes is mutable at runtime via apps.base.config._sync_ip_limits.
return float(self.minutes) * 60
def check_ip(self, ip: str) -> bool:
if ip in self.ips:
ip_info = self.ips[ip]
if ip_info["count"] >= self.count:
if ip_info["time"] + timedelta(minutes=self.minutes) > datetime.now():
return False
self.ips.pop(ip)
record = self.ips.get(ip)
if record is None:
return True
if record["count"] >= self.count:
if record["last_seen"] + self._window_seconds() > time.monotonic():
return False
self.ips.pop(ip, None)
return True
def add_ip(self, ip: str) -> int:
ip_info = self.ips.get(ip, {"count": 0, "time": datetime.now()})
ip_info["count"] += 1
ip_info["time"] = datetime.now()
self.ips[ip] = ip_info
return ip_info["count"]
now = time.monotonic()
record = self.ips.get(ip)
if record is None:
record = {"count": 0, "last_seen": now}
self.ips[ip] = record
else:
self.ips.move_to_end(ip)
record["count"] += 1
record["last_seen"] = now
while len(self.ips) > self.max_ips:
# Evict the least recently active IP; fail-open for the evicted
# address is preferable to unbounded memory growth.
self.ips.popitem(last=False)
return record["count"]
async def remove_expired_ip(self) -> None:
now = datetime.now()
expiration = timedelta(minutes=self.minutes)
self.ips = {
ip: info
for ip, info in self.ips.items()
if info["time"] + expiration >= now
}
now = time.monotonic()
window = self._window_seconds()
while self.ips:
_, record = next(iter(self.ips.items()))
if record["last_seen"] + window >= now:
break
self.ips.popitem(last=False)
def __call__(self, request: Request) -> str:
ip = get_client_ip(request)
+196
View File
@@ -0,0 +1,196 @@
"""文件类型校验:扩展名 / MIME 白名单 + magic bytes 防伪造。"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
from fastapi import HTTPException, UploadFile
from fnmatch import fnmatchcase
from core.settings import settings
@dataclass(frozen=True)
class FileKind:
name: str
extensions: tuple[str, ...]
mimes: tuple[str, ...]
signatures: tuple[bytes, ...]
FILE_KINDS: tuple[FileKind, ...] = (
FileKind("png", (".png",), ("image/png",), (b"\x89PNG\r\n\x1a\n",)),
FileKind("jpg", (".jpg", ".jpeg"), ("image/jpeg",), (b"\xff\xd8\xff",)),
FileKind("gif", (".gif",), ("image/gif",), (b"GIF87a", b"GIF89a")),
FileKind("webp", (".webp",), ("image/webp",), ()),
FileKind("bmp", (".bmp",), ("image/bmp", "image/x-ms-bmp"), (b"BM",)),
FileKind("pdf", (".pdf",), ("application/pdf",), (b"%PDF",)),
FileKind(
"zip",
(".zip", ".docx", ".xlsx", ".pptx", ".apk", ".jar"),
(
"application/zip",
"application/x-zip-compressed",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/java-archive",
"application/vnd.android.package-archive",
),
(b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08"),
),
FileKind(
"rar",
(".rar",),
("application/x-rar-compressed", "application/vnd.rar"),
(b"Rar!\x1a\x07\x00", b"Rar!\x1a\x07\x01\x00"),
),
FileKind("7z", (".7z",), ("application/x-7z-compressed",), (b"7z\xbc\xaf'\x1c",)),
FileKind("gz", (".gz", ".tgz"), ("application/gzip", "application/x-gzip"), (b"\x1f\x8b",)),
FileKind(
"mp3",
(".mp3",),
("audio/mpeg",),
(b"ID3", b"\xff\xfb", b"\xff\xf3", b"\xff\xf2"),
),
FileKind("mp4", (".mp4", ".m4a", ".mov"), ("video/mp4", "audio/mp4", "video/quicktime"), ()),
FileKind(
"exe",
(".exe", ".dll", ".sys"),
("application/x-msdownload", "application/x-dosexec"),
(b"MZ",),
),
FileKind("elf", (".elf", ".so", ".o"), ("application/x-executable",), (b"\x7fELF",)),
)
KNOWN_EXTENSIONS = {ext for kind in FILE_KINDS for ext in kind.extensions}
def normalize_allowed_file_types() -> list[str]:
raw_value = settings.allowed_file_types
if isinstance(raw_value, str):
values = [item.strip() for item in raw_value.split(",")]
elif isinstance(raw_value, (list, tuple, set)):
values = [str(item).strip() for item in raw_value]
else:
values = []
normalized = [item.lower() for item in values if item]
return normalized or ["*"]
def _extension_of(file_name: str) -> str:
return Path(file_name or "").suffix.lower()
def _match_allow_rule(rule: str, file_name: str, content_type: str) -> bool:
if rule in {"*", "*/*"}:
return True
if "/" in rule:
return fnmatchcase(content_type, rule)
extension = rule if rule.startswith(".") else f".{rule}"
return file_name.endswith(extension)
def is_type_allowed(file_name: str, content_type: Optional[str] = None) -> bool:
allowed = normalize_allowed_file_types()
if any(rule in {"*", "*/*"} for rule in allowed):
return True
normalized_name = (file_name or "").strip().lower()
normalized_content_type = (content_type or "").strip().lower()
return any(
_match_allow_rule(rule, normalized_name, normalized_content_type)
for rule in allowed
)
def detect_file_kind(header: bytes) -> Optional[FileKind]:
if not header:
return None
if len(header) >= 12 and header.startswith(b"RIFF") and header[8:12] == b"WEBP":
return next(kind for kind in FILE_KINDS if kind.name == "webp")
if len(header) >= 12 and header[4:8] == b"ftyp":
return next(kind for kind in FILE_KINDS if kind.name == "mp4")
candidates: list[tuple[int, FileKind]] = []
for kind in FILE_KINDS:
for signature in kind.signatures:
if signature and header.startswith(signature):
candidates.append((len(signature), kind))
if not candidates:
return None
candidates.sort(key=lambda item: item[0], reverse=True)
return candidates[0][1]
def validate_file_type(file_name: str, content_type: Optional[str] = None) -> None:
if not is_type_allowed(file_name, content_type):
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
def _kind_matches_ext(kind: FileKind, extension: str) -> bool:
return extension in kind.extensions
def _kind_matches_mime(kind: FileKind, mime: str) -> bool:
return mime in kind.mimes
def validate_file_magic(
file_name: str,
content_type: Optional[str],
header: Optional[bytes],
) -> None:
"""白名单通过后,用 magic bytes 防止扩展名 / MIME 伪造。"""
validate_file_type(file_name, content_type)
if not header:
return
claimed_ext = _extension_of(file_name)
claimed_mime = (content_type or "").strip().lower()
detected = detect_file_kind(header)
if claimed_ext in KNOWN_EXTENSIONS:
if detected is None or not _kind_matches_ext(detected, claimed_ext):
raise HTTPException(
status_code=403,
detail="文件内容与扩展名不匹配,疑似伪造类型",
)
if claimed_mime and any(claimed_mime in kind.mimes for kind in FILE_KINDS):
if detected is None or not _kind_matches_mime(detected, claimed_mime):
raise HTTPException(
status_code=403,
detail="文件内容与 Content-Type 不匹配,疑似伪造类型",
)
allowed = normalize_allowed_file_types()
if detected and not any(rule in {"*", "*/*"} for rule in allowed):
allowed_by_detected = any(
is_type_allowed(f"file{ext}", mime)
for ext in detected.extensions
for mime in (detected.mimes or (None,))
)
if not allowed_by_detected:
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
async def read_upload_header(file: UploadFile, size: int = 64) -> bytes:
await file.seek(0)
header = await file.read(size)
await file.seek(0)
return header or b""
async def validate_upload_file(file: UploadFile) -> None:
header = await read_upload_header(file)
validate_file_magic(file.filename or "", file.content_type, header)
def validate_header_bytes(
file_name: str, content_type: Optional[str], header: bytes
) -> None:
validate_file_magic(file_name, content_type, header)
+104
View File
@@ -0,0 +1,104 @@
"""Local NAS-style share helpers.
Files under ``data/local`` can be browsed by admin and turned into extract
codes without copying them into ``data/share``. The original file is never
deleted when a share expires or is removed from the file list.
``FileCodes.file_path`` is the sentinel ``local-ref``; ``uuid_file_name``
stores the posix relative path under ``data/local``. Accidental storage
deletes therefore look at ``data/local-ref/...`` and cannot touch the
original NAS file.
"""
from __future__ import annotations
import time
from pathlib import Path
from fastapi import HTTPException
from core.errors import StorageError
from core.storage import StoredDownload, build_attachment_headers
LOCAL_REF_MARKER = "local-ref"
MAX_RELPATH_LEN = 255
MAX_LIST_ENTRIES = 500
def is_local_ref(file_code) -> bool:
return (getattr(file_code, "file_path", None) or "") == LOCAL_REF_MARKER
def should_skip_storage_delete(file_code) -> bool:
return is_local_ref(file_code) or getattr(file_code, "text", None) is not None
def get_local_root() -> Path:
from core.settings import data_root
root = (Path(data_root) / "local").resolve()
root.mkdir(parents=True, exist_ok=True)
return root
def normalize_local_relpath(raw: str, *, allow_empty: bool = False) -> str:
text = str(raw or "").replace("\\", "/").strip()
if not text:
if allow_empty:
return ""
raise HTTPException(status_code=400, detail="非法文件名")
if (
text in {".", ".."}
or text.startswith("/")
or text.startswith("~")
or "\x00" in text
or ":" in text.split("/", 1)[0]
):
raise HTTPException(status_code=400, detail="非法文件名")
parts = [part for part in text.split("/") if part not in {"", "."}]
if not parts or any(part == ".." for part in parts):
raise HTTPException(status_code=400, detail="非法文件名")
relpath = "/".join(parts)
if len(relpath) > MAX_RELPATH_LEN:
raise HTTPException(status_code=400, detail="路径过长")
return relpath
def resolve_under_local(relpath: str) -> Path:
root = get_local_root()
if not relpath:
return root
candidate = (root / relpath).resolve()
try:
candidate.relative_to(root)
except ValueError as exc:
raise HTTPException(status_code=400, detail="非法文件路径") from exc
return candidate
def format_local_ctime(path: Path) -> str | None:
try:
return time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(path.stat().st_ctime))
except OSError:
return None
def build_local_ref_download(file_code) -> StoredDownload:
try:
relpath = normalize_local_relpath(getattr(file_code, "uuid_file_name", None) or "")
path = resolve_under_local(relpath)
except HTTPException as exc:
raise StorageError(status_code=404, detail="文件已过期删除") from exc
if not path.is_file():
raise StorageError(status_code=404, detail="文件已过期删除")
filename = f"{getattr(file_code, 'prefix', '') or ''}{getattr(file_code, 'suffix', '') or ''}"
if not filename:
filename = path.name
headers = build_attachment_headers(filename)
try:
headers["Content-Length"] = str(path.stat().st_size)
except OSError:
pass
return StoredDownload(filename=filename, headers=headers, path=path)
+231
View File
@@ -0,0 +1,231 @@
"""Stateless MCP Streamable HTTP endpoint for quick share uploads."""
import base64
import binascii
import html
import io
import json
from typing import Any
from urllib.parse import quote, urlsplit
from fastapi import APIRouter, HTTPException, Request, UploadFile
from starlette.datastructures import Headers
from starlette.responses import JSONResponse, Response
from apps.base.auth import _require_admin_payload
from apps.base.file_validation import validate_file_type, validate_upload_file
from apps.base.services import FileUploadService, validate_file_size
from apps.base.utils import ip_limit, validate_expire_style
from core.settings import settings
from core.utils import sanitize_filename
router = APIRouter(tags=["MCP"])
MCP_PROTOCOL_VERSION = "2025-03-26"
TEXT_SHARE_MAX_BYTES = 222 * 1024
MAX_MCP_BODY_BYTES = 64 * 1024 * 1024
def _jsonrpc_error(request_id, code: int, message: str) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "error": {"code": code, "message": message}}
def _jsonrpc_result(request_id, result: dict) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "result": result}
def _validate_origin(request: Request) -> None:
"""Reject browser cross-site POSTs before parsing any upload tool."""
origin = request.headers.get("origin")
fetch_site = request.headers.get("sec-fetch-site", "").lower()
if fetch_site == "cross-site":
raise HTTPException(status_code=403, detail="跨站请求已拒绝")
if not origin:
return # Native MCP clients do not send Origin and use no ambient cookies.
try:
parsed = urlsplit(origin)
host = request.headers.get("host", "").lower()
if (
parsed.scheme not in {"http", "https"}
or not parsed.netloc
or parsed.netloc.lower() != host
or parsed.username is not None
or parsed.password is not None
or parsed.path not in {"", "/"}
or parsed.query
or parsed.fragment
):
raise ValueError("origin mismatch")
except (ValueError, TypeError):
raise HTTPException(status_code=403, detail="跨站请求已拒绝")
async def _read_limited_body(request: Request) -> bytes:
max_size = min(
MAX_MCP_BODY_BYTES,
max(1024 * 1024, int(settings.upload_size * 4 / 3) + 256 * 1024),
)
declared_size = request.headers.get("content-length")
if declared_size:
try:
if int(declared_size) > max_size:
raise HTTPException(status_code=413, detail="MCP 请求体过大")
except ValueError:
raise HTTPException(status_code=400, detail="Content-Length 无效")
body = bytearray()
async for chunk in request.stream():
body.extend(chunk)
if len(body) > max_size:
raise HTTPException(status_code=413, detail="MCP 请求体过大")
return bytes(body)
def _tool_definitions() -> list[dict]:
return [
{
"name": "share_text",
"description": "将文本快速上传到 FileCodeBox 并创建分享链接。",
"inputSchema": {
"type": "object",
"properties": {
"text": {"type": "string", "description": "要分享的文本"},
"expire_value": {"type": "integer", "minimum": 1, "default": 1},
"expire_style": {"type": "string", "enum": ["day", "hour", "minute", "forever", "count"], "default": "day"},
},
"required": ["text"],
},
},
{
"name": "share_file",
"description": "将 Base64 编码的文件上传到 FileCodeBox 并创建分享链接。",
"inputSchema": {
"type": "object",
"properties": {
"filename": {"type": "string", "description": "文件名"},
"content_base64": {"type": "string", "description": "文件内容的标准 Base64 编码"},
"content_type": {"type": "string", "default": "application/octet-stream"},
"expire_value": {"type": "integer", "minimum": 1, "default": 1},
"expire_style": {"type": "string", "enum": ["day", "hour", "minute", "forever", "count"], "default": "day"},
},
"required": ["filename", "content_base64"],
},
},
]
def _share_result(request: Request, code: str, name: str) -> dict:
base_url = str(request.base_url).rstrip("/")
share_url = f"{base_url}/#/?code={quote(code, safe='')}"
markdown_name = html.escape(name).replace("[", "\\[").replace("]", "\\]")
return {
"code": code,
"name": name,
"url": share_url,
"markdown": f"[{markdown_name}]({share_url})",
}
async def _call_tool(request: Request, tool_name: str, arguments: dict) -> dict:
authorization = request.headers.get("authorization", "")
if settings.mcp_require_token or not settings.open_upload:
_require_admin_payload(authorization)
ip = ip_limit["upload"](request)
if tool_name == "share_text":
text = arguments.get("text")
if not isinstance(text, str):
raise ValueError("text 必须是字符串")
if len(text.encode("utf-8")) > TEXT_SHARE_MAX_BYTES:
raise ValueError("内容过多,请改用文件分享")
expire_value = int(arguments.get("expire_value", 1))
expire_style = str(arguments.get("expire_style", "day"))
if expire_value < 1:
raise ValueError("expire_value 必须大于 0")
validate_expire_style(expire_style)
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return _share_result(request, code, "Text")
if tool_name == "share_file":
filename = arguments.get("filename")
encoded_content = arguments.get("content_base64")
content_type = str(arguments.get("content_type", "application/octet-stream"))
if not isinstance(filename, str) or not filename.strip():
raise ValueError("filename 不能为空")
if not isinstance(encoded_content, str):
raise ValueError("content_base64 必须是字符串")
try:
content = base64.b64decode(encoded_content, validate=True)
except (ValueError, binascii.Error):
raise ValueError("content_base64 不是有效的标准 Base64")
safe_name = await sanitize_filename(filename.rsplit("/", 1)[-1].rsplit("\\", 1)[-1])
validate_file_type(safe_name, content_type)
upload = UploadFile(filename=safe_name, file=io.BytesIO(content), size=len(content), headers=Headers({"content-type": content_type}))
size = await validate_file_size(upload, settings.upload_size)
await validate_upload_file(upload)
expire_value = int(arguments.get("expire_value", 1))
expire_style = str(arguments.get("expire_style", "day"))
if expire_value < 1:
raise ValueError("expire_value 必须大于 0")
validate_expire_style(expire_style)
share = await FileUploadService.create_file_share(upload, size=size, expire_value=expire_value, expire_style=expire_style)
ip_limit["upload"].add_ip(ip)
return _share_result(request, share["code"], share["name"])
raise LookupError(f"未知工具:{tool_name}")
@router.post("/mcp", include_in_schema=False)
async def mcp_endpoint(request: Request):
_validate_origin(request)
media_type = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
if media_type != "application/json":
raise HTTPException(status_code=415, detail="MCP 请求必须使用 application/json")
try:
message = json.loads(await _read_limited_body(request))
except (json.JSONDecodeError, UnicodeDecodeError):
return JSONResponse(_jsonrpc_error(None, -32700, "Parse error"), status_code=400)
if not isinstance(message, dict) or message.get("jsonrpc") != "2.0" or not isinstance(message.get("method"), str):
return JSONResponse(_jsonrpc_error(message.get("id") if isinstance(message, dict) else None, -32600, "Invalid Request"), status_code=400)
request_id = message.get("id")
method = message["method"]
if request_id is None and method.startswith("notifications/"):
return Response(status_code=202)
result: dict[str, Any] = {}
try:
if method == "initialize":
result = {
"protocolVersion": MCP_PROTOCOL_VERSION,
"capabilities": {"tools": {"listChanged": False}},
"serverInfo": {"name": "filecodebox", "version": "1.0.0"},
}
elif method == "ping":
result = {}
elif method == "tools/list":
result = {"tools": _tool_definitions()}
elif method == "tools/call":
params = message.get("params")
if not isinstance(params, dict) or not isinstance(params.get("name"), str):
return JSONResponse(_jsonrpc_error(request_id, -32602, "Invalid params"))
arguments = params.get("arguments", {})
if not isinstance(arguments, dict):
return JSONResponse(_jsonrpc_error(request_id, -32602, "Invalid params"))
try:
tool_value = await _call_tool(request, params["name"], arguments)
result = {"content": [{"type": "text", "text": json.dumps(tool_value, ensure_ascii=False)}], "structuredContent": tool_value}
except LookupError as exc:
return JSONResponse(_jsonrpc_error(request_id, -32602, str(exc)))
except HTTPException as exc:
if exc.status_code in {401, 403}:
raise
result = {"content": [{"type": "text", "text": str(exc.detail)}], "isError": True}
except (TypeError, ValueError) as exc:
result = {"content": [{"type": "text", "text": str(exc)}], "isError": True}
else:
return JSONResponse(_jsonrpc_error(request_id, -32601, "Method not found"))
return JSONResponse(_jsonrpc_result(request_id, result), headers={"MCP-Protocol-Version": MCP_PROTOCOL_VERSION})
except HTTPException as exc:
return JSONResponse(
_jsonrpc_error(request_id, -32001 if exc.status_code == 401 else -32003, str(exc.detail)),
status_code=exc.status_code,
)
+51
View File
@@ -0,0 +1,51 @@
import json
from tortoise import connections
# 配置键驼峰 -> snake_case 统一(2.5)。settings 存于 KeyValue 表的 JSON 值中,
# 迁移在 Python 层读取、改写键名后写回,保证存量部署升级后键名与代码一致。
KEY_RENAMES = {
"adminSessionExpire": "admin_session_expire",
"enableChunk": "enable_chunk",
"errorCount": "error_count",
"errorMinute": "error_minute",
"expireStyle": "expire_style",
"loginCount": "login_count",
"loginMinute": "login_minute",
"openUpload": "open_upload",
"robotsText": "robots_text",
"serverHost": "server_host",
"serverPort": "server_port",
"serverWorkers": "server_workers",
"showAdminAddr": "show_admin_addr",
"storageLimit": "storage_limit",
"themesChoices": "themes_choices",
"themesSelect": "themes_select",
"trustedProxies": "trusted_proxies",
"uploadCount": "upload_count",
"uploadMinute": "upload_minute",
"uploadSize": "upload_size",
}
async def migrate():
conn = connections.get("default")
rows = await conn.execute_query_dict(
"SELECT id, value FROM keyvalue WHERE key = 'settings'"
)
for row in rows:
value = row["value"]
if isinstance(value, str):
try:
value = json.loads(value)
except ValueError:
continue
if not isinstance(value, dict):
continue
renamed = {KEY_RENAMES.get(k, k): v for k, v in value.items()}
if renamed != value:
await conn.execute_query(
"UPDATE keyvalue SET value = ? WHERE id = ?",
[json.dumps(renamed, ensure_ascii=False), row["id"]],
)
+26
View File
@@ -0,0 +1,26 @@
from tortoise import connections
# 高频查询字段索引。真实部署的 schema 全部由迁移建立
# (init_db 不调用 generate_schemas),因此索引必须写在迁移里;
# apps/base/models.py 中的 index/Meta 仅用于测试的 generate_schemas 路径。
#
# - filecodes(expired_at):过期扫描(delete_expire_files)与后台过期统计。
# - filecodes(created_at):后台默认倒序列表、首页最近记录、今日/昨日统计。
# - uploadchunk(upload_id, chunk_index):分片状态检查与合并;其前缀同时
# 覆盖仅按 upload_id 的删除/清理查询(原先没有对应索引)。
# 已知缺口:clean_incomplete_uploads 的 chunk_index=-1 AND created_at<? 用不上
# 该复合索引,待清理任务改为分批查询后再评估。
async def migrate():
conn = connections.get("default")
await conn.execute_script(
"""
CREATE INDEX IF NOT EXISTS idx_filecodes_expired_at
ON filecodes (expired_at);
CREATE INDEX IF NOT EXISTS idx_filecodes_created_at
ON filecodes (created_at);
CREATE INDEX IF NOT EXISTS idx_uploadchunk_upload_chunk
ON uploadchunk (upload_id, chunk_index);
"""
)
+22 -14
View File
@@ -2,13 +2,13 @@
# @Author : Lan
# @File : models.py
# @Software: PyCharm
from typing import Optional
from typing import Any
from tortoise.models import Model
from tortoise.contrib.pydantic import pydantic_model_creator
from tortoise import fields, models
from datetime import datetime
from core.utils import get_now
@@ -21,10 +21,14 @@ class FileCodes(models.Model):
file_path = fields.CharField(max_length=255, null=True)
size = fields.BigIntField(default=0)
text = fields.TextField(null=True)
expired_at = fields.DatetimeField(null=True)
expired_count = fields.IntField(default=0)
# 过期扫描与后台过期统计的高频过滤列
expired_at = fields.DatetimeField(null=True, index=True)
# -1 = 时间式(看 expired_at),<=0 的 0 = 次数耗尽。默认 -1 而非 0:
# 0 语义是"次数已用完即过期",忘传该字段的创建路径会得到立即过期的记录。
expired_count = fields.IntField(default=-1)
used_count = fields.IntField(default=0)
created_at = fields.DatetimeField(auto_now_add=True)
# 后台默认按创建时间倒序、今日/昨日统计均落在该列
created_at = fields.DatetimeField(auto_now_add=True, index=True)
file_hash = fields.CharField(max_length=64, null=True)
is_chunked = fields.BooleanField(default=False)
upload_id = fields.CharField(max_length=36, null=True)
@@ -42,7 +46,9 @@ class FileCodes(models.Model):
class UploadChunk(models.Model):
id = fields.IntField(pk=True)
upload_id = fields.CharField(max_length=36, index=True)
# 上传会话/分片查询始终同时带 upload_id;复合索引前缀已覆盖仅按 upload_id
# 的删除与清理查询,单列索引会造成重复。
upload_id = fields.CharField(max_length=36)
chunk_index = fields.IntField()
chunk_hash = fields.CharField(max_length=64)
total_chunks = fields.IntField()
@@ -53,16 +59,18 @@ class UploadChunk(models.Model):
created_at = fields.DatetimeField(auto_now_add=True)
completed = fields.BooleanField(default=False)
class Meta:
indexes = [("upload_id", "chunk_index")]
class KeyValue(Model):
id: Optional[int] = fields.IntField(pk=True)
key: Optional[str] = fields.CharField(
max_length=255, description="键", index=True, unique=True
)
value: Optional[str] = fields.JSONField(description="值", null=True)
created_at: Optional[datetime] = fields.DatetimeField(
auto_now_add=True, description="创建时间"
)
# 与 FileCodes 保持同一风格:类属性不写值注解(等号右侧的 Field 描述符
# 才是类级真身;实例级由 tortoise 运行时给出对应 Python 值)
id = fields.IntField(pk=True)
key = fields.CharField(max_length=255, description="键", index=True, unique=True)
# JSONField 实际承载 dict/list(如 settings 配置)
value: Any = fields.JSONField(description="值", null=True)
created_at = fields.DatetimeField(auto_now_add=True, description="创建时间")
class PresignUploadSession(models.Model):
+223
View File
@@ -0,0 +1,223 @@
"""Public-facing pages and routes: setup wizard, theme assets, index,
robots.txt, and the public config endpoints."""
import html
import re
from functools import lru_cache
from pathlib import Path
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, RedirectResponse
from apps.base.config import initialize_system, is_runtime_initialized
from apps.base.setup_wizard import (
build_public_config,
build_public_meta,
build_setup_page,
build_setup_success_page,
parse_setup_options,
read_setup_payload,
setup_response,
)
from core.response import APIResponse
from core.settings import BASE_DIR, DEFAULT_CONFIG, settings
from core.version import APP_VERSION
router = APIRouter()
@router.get("/setup", include_in_schema=False)
@router.get("/setup/", include_in_schema=False)
async def setup_page():
if is_runtime_initialized():
return RedirectResponse(url="/", status_code=303)
return setup_response(build_setup_page())
@router.post("/setup", include_in_schema=False)
@router.post("/setup/", include_in_schema=False)
async def setup_submit(request: Request):
if is_runtime_initialized():
return RedirectResponse(url="/", status_code=303)
data = await read_setup_payload(request)
admin_password = str(data.get("admin_password") or "")
confirm_password = str(data.get("confirm_password") or "")
site_name = str(data.get("site_name") or "")
if admin_password != confirm_password:
return setup_response(build_setup_page("两次输入的管理员密码不一致", data), 400)
try:
setup_options = parse_setup_options(data)
await initialize_system(
admin_password=admin_password,
site_name=site_name,
setup_options=setup_options,
)
except ValueError as exc:
return setup_response(build_setup_page(str(exc), data), 400)
if "application/json" in request.headers.get("accept", ""):
return APIResponse(detail={"ok": True, "admin": "/#/admin"})
return setup_response(build_setup_success_page())
def resolve_theme_root():
themes_root = (BASE_DIR / "themes").resolve()
theme_root = (BASE_DIR / str(settings.themes_select)).resolve()
try:
theme_root.relative_to(themes_root)
except ValueError:
theme_root = (BASE_DIR / DEFAULT_CONFIG["themes_select"]).resolve()
if not theme_root.exists():
theme_root = (BASE_DIR / DEFAULT_CONFIG["themes_select"]).resolve()
return theme_root
def resolve_theme_file(*parts: str):
theme_root = resolve_theme_root()
file_path = theme_root.joinpath(*parts).resolve()
# 防止通过 /assets/../ 读取主题目录外的文件。
try:
file_path.relative_to(theme_root)
except ValueError:
raise HTTPException(status_code=404, detail="资源不存在")
if not file_path.is_file():
raise HTTPException(status_code=404, detail="资源不存在")
return file_path
# Vite 产物带内容指纹(如 index-BX2s1QqP.js),内容变化时文件名必然变化,
# 可以安全地长缓存;public/ 下的未哈希资源原地升级时名字不变,只给短缓存,
# 过期后靠 FileResponse 自带的 ETag/Last-Modified 走 304。
_HASHED_ASSET_RE = re.compile(
r"-[A-Za-z0-9_-]{8,}\.(?:js|mjs|css|map|woff2?|ttf|otf|eot|svg|png|jpe?g|gif|webp|avif|ico)$",
re.IGNORECASE,
)
_IMMUTABLE_ASSET_CACHE = "public, max-age=31536000, immutable"
_DEFAULT_ASSET_CACHE = "public, max-age=86400"
def _asset_cache_control(asset_path: str) -> str:
if _HASHED_ASSET_RE.search(asset_path):
return _IMMUTABLE_ASSET_CACHE
return _DEFAULT_ASSET_CACHE
@lru_cache(maxsize=8)
def _render_index_html(
template_path: str,
inode: int,
mtime_ns: int,
size: int,
title: str,
description: str,
keywords: str,
opacity: str,
background: str,
) -> str:
"""Render index.html once per (theme file version, site config) pair.
The stat fields keep the cache honest when a theme is replaced in place;
config values are part of the key so admin edits are visible immediately
without any cross-module invalidation hook.
"""
return (
Path(template_path)
.read_text(encoding="utf-8")
.replace("{{title}}", html.escape(title))
.replace("{{description}}", html.escape(description))
.replace("{{keywords}}", html.escape(keywords))
.replace("{{opacity}}", html.escape(opacity))
.replace("{{background}}", html.escape(background))
)
@router.get("/assets/{asset_path:path}", include_in_schema=False)
async def theme_asset(asset_path: str):
return FileResponse(
resolve_theme_file("assets", asset_path),
headers={"Cache-Control": _asset_cache_control(asset_path)},
)
async def not_found_handler(request, exc=None):
"""区分"浏览器导航到未知路径"与"API 处理器抛出的 404"。
Starlette 的 status-code handler 优先于 HTTPException 类 handler,
直接注册 index 会让全应用所有 HTTPException(404) 变成 200 HTML 页
(负路径测试抓到的存量 bug)。浏览器(Accept 含 text/html)仍拿到
主题首页做 SPA 兜底;API 客户端拿到 JSON 404。
"""
json_404 = JSONResponse(
status_code=404,
content={"code": 404, "message": "Not Found", "detail": "资源不存在"},
)
if request is not None and request.method in {"GET", "HEAD"} and "text/html" in (
request.headers.get("accept", "")
):
try:
return await index(request, exc)
except HTTPException:
return json_404
return json_404
@router.get("/")
async def index(request=None, exc=None):
# Site config is admin input (and during the setup window anyone can claim it);
# always escape before injecting into the theme template to prevent stored XSS
# (mirrors the setup page).
template_path = resolve_theme_file("index.html")
try:
stat_result = template_path.stat()
except OSError:
# resolve_theme_file 与 stat 之间文件被删除/替换时仍按 404 处理,
# 不能退回缓存内容。
raise HTTPException(status_code=404, detail="资源不存在")
content = _render_index_html(
str(template_path),
stat_result.st_ino,
stat_result.st_mtime_ns,
stat_result.st_size,
str(settings.name),
str(settings.description),
str(settings.keywords),
str(settings.opacity),
str(settings.background),
)
return HTMLResponse(
content=content,
media_type="text/html",
headers={"Cache-Control": "no-cache"},
)
@router.get("/robots.txt")
async def robots():
return HTMLResponse(content=settings.robots_text, media_type="text/plain")
@router.post("/")
async def get_config():
return APIResponse(detail=build_public_config())
@router.get("/api/v1/config")
async def get_public_config():
return APIResponse(
detail={
"config": build_public_config(),
"meta": build_public_meta(),
}
)
@router.get("/health")
async def health_check():
return APIResponse(
detail={
"status": "ok",
"version": APP_VERSION,
"storage": settings.file_storage,
"theme": settings.themes_select,
}
)
+21 -8
View File
@@ -2,12 +2,24 @@ import datetime
from fastapi import HTTPException
from tortoise import connections
from tortoise.expressions import Q
from tortoise.functions import Sum
from apps.base.models import FileCodes, StorageReservation
from apps.base.local_share import LOCAL_REF_MARKER
from core.settings import settings
from core.utils import get_now
def owned_storage_queryset(queryset=None):
"""FileCodes that occupy FileCodeBox storage (exclude NAS local-ref rows).
SQL ``file_path != 'local-ref'`` drops NULLs, so keep null paths explicitly.
"""
qs = FileCodes.all() if queryset is None else queryset
return qs.filter(Q(file_path__isnull=True) | ~Q(file_path=LOCAL_REF_MARKER))
def _detect_sql_dialect() -> str:
"""识别当前默认连接的 SQL 方言。
@@ -47,20 +59,21 @@ def _sql_placeholders(count: int) -> list[str]:
def get_storage_limit() -> int:
try:
return max(0, int(getattr(settings, "storageLimit", 0)))
return max(0, int(getattr(settings, "storage_limit", 0)))
except (TypeError, ValueError):
return 0
async def get_storage_usage() -> dict[str, int | None]:
now = await get_now()
used = await FileCodes.all().values_list("size", flat=True)
reserved = await StorageReservation.filter(expires_at__gt=now).values_list(
"size", flat=True
)
# SQL 聚合:此函数在每次上传配额检查时调用,禁止全表拉取(D3)
used_rows = await owned_storage_queryset().annotate(total=Sum("size")).values("total")
reserved_rows = await StorageReservation.filter(expires_at__gt=now).annotate(
total=Sum("size")
).values("total")
limit = get_storage_limit()
used_bytes = sum(used)
reserved_bytes = sum(reserved)
used_bytes = used_rows[0]["total"] or 0
reserved_bytes = reserved_rows[0]["total"] or 0
return {
"limit": limit,
"used": used_bytes,
@@ -92,7 +105,7 @@ async def reserve_storage(token: str, size: int, ttl_seconds: int) -> None:
INSERT INTO storagereservation (token, size, expires_at)
SELECT {ph[0]}, {ph[1]}, {ph[2]}
WHERE (
COALESCE((SELECT SUM(size) FROM filecodes), 0)
COALESCE((SELECT SUM(size) FROM filecodes WHERE file_path IS NULL OR file_path != '{LOCAL_REF_MARKER}'), 0)
+ COALESCE((SELECT SUM(size) FROM storagereservation WHERE expires_at > {ph[3]}), 0)
+ {ph[4]}
) <= {ph[5]}
-1
View File
@@ -1,5 +1,4 @@
from pydantic import BaseModel
from typing import Optional
class SelectFileModel(BaseModel):
+435
View File
@@ -0,0 +1,435 @@
"""apps.base service layer: upload orchestration shared by share/chunk/presign.
Handlers in views.py keep HTTP concerns (form parsing, session lookup, status
codes for malformed requests, rate limiting); this module owns the business
workflow: quota reservation, storage writes, share-record creation, and
failure rollback/cleanup.
"""
import os
from fastapi import HTTPException, UploadFile
from fastapi.responses import FileResponse, Response, StreamingResponse
from tortoise.transactions import in_transaction
from core.logger import logger
from core.settings import settings
from core.storage import FileStorageInterface, StoredDownload, StoredFile, storages
from apps.base.file_validation import validate_upload_file
from apps.base.local_share import build_local_ref_download, is_local_ref
from apps.base.models import FileCodes, PresignUploadSession, UploadChunk
from apps.base.quota import release_storage, reserve_storage
from apps.base.utils import build_file_path, get_expire_info
import uuid
# 预签名上传会话有效期(秒)
PRESIGN_SESSION_EXPIRES = 900 # 15分钟
def stored_file_of(code: FileCodes) -> StoredFile:
"""Adapter: project an ORM FileCodes row onto the storage-layer contract."""
return StoredFile(
file_path=code.file_path,
uuid_file_name=code.uuid_file_name,
code=code.code,
prefix=code.prefix,
suffix=code.suffix,
text=code.text or "",
)
async def get_stored_download(file_code, file_storage: FileStorageInterface | None = None) -> StoredDownload:
if is_local_ref(file_code):
return build_local_ref_download(file_code)
storage = file_storage or storages[settings.file_storage]()
return await storage.get_file_response(stored_file_of(file_code))
async def rollback_saved_file(
storage: FileStorageInterface,
file_path: str,
uuid_file_name: str,
*,
context: str,
upload_id: str | None = None,
) -> None:
"""Best-effort delete of a stored file after a failed upload workflow.
Rollback failures are logged (with traceback) and swallowed: the original
error must propagate; the file, if left behind, is cleaned by the expiry
task once its orphaned record ages out.
"""
try:
await storage.delete_file(
StoredFile(file_path=file_path, uuid_file_name=uuid_file_name)
)
except Exception:
logger.warning(
"%s:回滚删除已保存文件失败%s",
context,
f" upload_id={upload_id}" if upload_id else "",
exc_info=True,
)
async def validate_file_size(file: UploadFile, max_size: int) -> int:
"""Return the upload's size, rejecting anything above max_size."""
size = file.size
if size is None:
# 无长度声明的上传(如 chunked 传输):必须走底层文件对象拿末尾偏移。
# 注意两层都不能用:UploadFile.seek 只接受单参数(TypeError);
# 底层 SpooledTemporaryFile.seek 是同步方法(不能 await)。
file.file.seek(0, 2)
size = file.file.tell()
file.file.seek(0)
if size > max_size:
max_size_mb = max_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"大小超过限制,最大为{max_size_mb:.2f} MB"
)
return size
def chunk_reservation_ttl() -> int:
ttl = max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600
return ttl
class FileUploadService:
"""统一的文件上传服务"""
@staticmethod
def _storage() -> FileStorageInterface:
return storages[settings.file_storage]()
@staticmethod
async def generate_file_path(
file_name: str, upload_id: str | None = None
) -> tuple[str, str, str, str, str]:
"""Delegates path generation to apps.base.utils.build_file_path."""
return await build_file_path(file_name, upload_id or uuid.uuid4().hex)
@staticmethod
async def create_file_record(
file_name: str,
file_size: int,
file_path: str,
expire_value: int,
expire_style: str,
**extra_fields,
) -> str:
"""统一创建FileCodes记录,返回code"""
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
prefix, suffix = os.path.splitext(file_name)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=file_name,
file_path=file_path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
**extra_fields,
)
return code
@staticmethod
async def create_text_share(
text: str, expire_value: int, expire_style: str
) -> str:
"""文本分享:配额预留 → 建分享记录 → 释放配额。"""
text_size = len(text.encode("utf-8"))
token = f"text:{uuid.uuid4().hex}"
await reserve_storage(token, text_size, ttl_seconds=300)
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await FileCodes.create(
code=code,
text=text,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
size=text_size,
prefix="Text",
)
finally:
await release_storage(token)
return code
@staticmethod
async def create_file_share(
file: UploadFile, *, size: int, expire_value: int, expire_style: str
) -> dict[str, str]:
"""文件分享:路径生成 → 配额预留 → 存储写入 → 建分享记录,失败回滚已存文件。"""
path, suffix, prefix, uuid_file_name, save_path = (
await FileUploadService.generate_file_path(file.filename or "")
)
token = f"file:{uuid.uuid4().hex}"
await reserve_storage(token, size, ttl_seconds=3600)
storage = FileUploadService._storage()
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await storage.save_file(file.file, save_path, file.content_type)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=uuid_file_name,
file_path=path,
size=size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
)
except Exception:
await rollback_saved_file(
storage, path, uuid_file_name, context="分享上传"
)
raise
finally:
await release_storage(token)
# UploadFile.filename 类型为 str | None;路径生成已用 "" 兜底,
# 响应处保持 str 以匹配 dict[str, str] 契约
return {"code": code, "name": file.filename or ""}
@staticmethod
async def complete_chunked_upload(
upload_id: str, chunk_info: UploadChunk, expire_value: int, expire_style: str
) -> dict[str, str]:
"""分片合并:配额 → 完整性/大小校验 → 合并 → 建分享记录 → 清理分片。
失败路径的配额释放与清理范围与原实现逐一对齐:
完整性校验失败仅抛 400(预留由 TTL 兜底);合并失败清理分片文件后抛 500。
"""
if chunk_info.completed:
raise HTTPException(409, "上传会话已完成,仅待清理")
storage = FileUploadService._storage()
await reserve_storage(
f"chunk:{upload_id}", chunk_info.file_size, ttl_seconds=chunk_reservation_ttl()
)
completed_chunks = await UploadChunk.filter(
upload_id=upload_id, completed=True, chunk_index__gte=0
).all()
if len(completed_chunks) != chunk_info.total_chunks:
raise HTTPException(400, "分片不完整")
# 用分片数 * chunk_size 校验最大可能大小
max_total_size = len(completed_chunks) * chunk_info.chunk_size
if max_total_size > settings.upload_size:
save_path = chunk_info.save_path
cleanup_succeeded = True
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
cleanup_succeeded = False
logger.warning(
"分片超限中止:清理分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
if cleanup_succeeded:
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
403, f"实际上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
save_path = chunk_info.save_path
path = os.path.dirname(save_path) if save_path else ""
safe_file_name = os.path.basename(save_path) if save_path else ""
prefix, suffix = os.path.splitext(safe_file_name)
try:
records = {r.chunk_index: r for r in completed_chunks}
_, file_hash = await storage.merge_chunks(
upload_id,
chunk_info.total_chunks,
chunk_info.chunk_size,
save_path,
records,
)
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
# 发布记录与会话终态一起提交;即使分享过期删除,会话也不能被复用。
async with in_transaction() as connection:
await FileCodes.create(
code=code,
file_hash=file_hash, # 使用合并后计算的哈希
is_chunked=True,
upload_id=upload_id,
size=chunk_info.file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
file_path=path,
uuid_file_name=safe_file_name,
prefix=prefix,
suffix=suffix,
using_db=connection,
)
chunk_info.completed = True
await chunk_info.save(using_db=connection, update_fields=["completed"])
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
# 分享已创建:不把清理故障伪装成合并失败,保留会话供定时任务重试。
logger.warning(
"分片合并完成:清理临时分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
else:
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
return {"code": code, "name": safe_file_name}
except ValueError as e:
raise HTTPException(400, str(e))
except Exception as e:
# 合并失败时清理临时文件
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning(
"分片合并失败:清理临时分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
raise HTTPException(500, f"文件合并失败: {str(e)}")
@staticmethod
async def commit_proxy_upload(
session: PresignUploadSession, file: UploadFile
) -> str:
"""预签名代理上传:配额 → 大小/类型/一致性校验 → 转存 → 建记录 → 会话清理。
校验失败不释放预留(与原实现一致,由 TTL 兜底)。
"""
await reserve_storage(
f"presign:{session.upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
if abs(file_size - session.file_size) > 1024:
raise HTTPException(400, "文件大小与声明不符")
storage = FileUploadService._storage()
try:
await storage.save_file(file.file, session.save_path, file.content_type)
except Exception as e:
raise HTTPException(500, f"文件保存失败: {str(e)}")
try:
code = await FileUploadService.create_file_record(
session.file_name,
file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
await rollback_saved_file(
storage,
os.path.dirname(session.save_path),
os.path.basename(session.save_path),
context="预签名代理上传:记录创建失败",
upload_id=session.upload_id,
)
raise
await session.delete()
await release_storage(f"presign:{session.upload_id}")
return code
@staticmethod
async def confirm_direct_upload(session: PresignUploadSession) -> str:
"""预签名直传确认:配额 → 文件存在性 → 建记录 → 会话清理。
预留失败说明配额已耗尽,此时清理远端临时文件与会话后原样抛出。
"""
try:
await reserve_storage(
f"presign:{session.upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
except HTTPException:
storage = FileUploadService._storage()
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
finally:
await session.delete()
await release_storage(f"presign:{session.upload_id}")
raise
storage = FileUploadService._storage()
if not await storage.file_exists(session.save_path):
raise HTTPException(404, "文件未上传或上传失败")
try:
code = await FileUploadService.create_file_record(
session.file_name,
session.file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
await rollback_saved_file(
storage,
os.path.dirname(session.save_path),
os.path.basename(session.save_path),
context="预签名确认:记录创建失败",
upload_id=session.upload_id,
)
raise
await session.delete()
await release_storage(f"presign:{session.upload_id}")
return code
def response_from_download(download: StoredDownload):
"""Build the starlette Response for a StoredDownload (view-layer duty)."""
if download.path is not None:
return FileResponse(
download.path,
media_type=download.media_type,
headers=download.headers,
filename=download.filename,
)
if download.content is not None:
return Response(
download.content, media_type=download.media_type, headers=download.headers
)
if download.stream_factory is None:
# 数据契约要求三者至少其一;防御性 500 语义
raise ValueError("StoredDownload 缺少可用的下载载荷")
return StreamingResponse(
download.stream_factory(),
media_type=download.media_type,
headers=download.headers,
background=download.background,
)
+694
View File
@@ -0,0 +1,694 @@
"""Setup wizard: form parsing rules and the two admin-facing HTML pages.
Pure presentation/logic — no router here; routes live in apps.base.pages.
"""
import html
from urllib.parse import parse_qs
from typing import Any
from fastapi import Request
from fastapi.responses import HTMLResponse
from core.settings import DEFAULT_CONFIG, settings
from core.version import APP_VERSION
FILE_SIZE_UNITS = {"KB": 1024, "MB": 1024**2, "GB": 1024**3}
SAVE_TIME_UNITS = {"second": 1, "minute": 60, "hour": 3600, "day": 86400}
EXPIRE_STYLE_OPTIONS = [
("day", "按天"),
("hour", "按小时"),
("minute", "按分钟"),
("forever", "永久"),
("count", "按取件次数"),
]
def normalize_public_flag(value) -> int:
if isinstance(value, str):
return int(value.strip().lower() in {"1", "true", "on", "yes"})
return int(bool(value))
def build_public_config() -> dict:
return {
"name": settings.name,
"description": settings.description,
"explain": settings.page_explain,
"upload_size": settings.upload_size,
"allowed_file_types": settings.allowed_file_types,
"expire_style": settings.expire_style,
"enable_chunk": settings.enable_chunk,
"open_upload": settings.open_upload,
"notify_title": settings.notify_title,
"notify_content": settings.notify_content,
"show_admin_address": normalize_public_flag(settings.show_admin_addr),
"max_save_seconds": settings.max_save_seconds,
}
def build_public_meta() -> dict:
return {
"version": APP_VERSION,
"api": {
"legacy_config": "/",
"public_config": "/api/v1/config",
"health": "/health",
},
"features": {
"chunk_upload": bool(settings.enable_chunk),
"guest_upload": bool(settings.open_upload),
"admin_address_visible": bool(normalize_public_flag(settings.show_admin_addr)),
"expiration_modes": settings.expire_style,
},
"limits": {
"upload_size": settings.upload_size,
"allowed_file_types": settings.allowed_file_types,
"max_save_seconds": settings.max_save_seconds,
"upload_window_minutes": settings.upload_minute,
"upload_window_count": settings.upload_count,
},
}
def get_form_value(data: dict[str, Any], key: str, default: str = "") -> str:
value = data.get(key, default)
if isinstance(value, list):
value = value[-1] if value else default
return str(value if value is not None else default)
def get_form_list(data: dict[str, Any], key: str) -> list[str]:
value = data.get(key, [])
if isinstance(value, list):
return [str(item) for item in value if str(item)]
if value:
return [str(value)]
return []
def normalize_bool_field(data: dict, key: str, default: bool) -> bool:
if key not in data:
return default
return get_form_value(data, key).lower() in {"1", "true", "on", "yes"}
def parse_int_field(
data: dict[str, Any],
key: str,
default: int,
label: str,
min_value: int = 0,
max_value: int | None = None,
) -> int:
raw_value = get_form_value(data, key, str(default)).strip()
try:
value = int(raw_value)
except ValueError:
raise ValueError(f"{label} 必须是整数")
if value < min_value:
raise ValueError(f"{label} 不能小于 {min_value}")
if max_value is not None and value > max_value:
raise ValueError(f"{label} 不能大于 {max_value}")
return value
def parse_allowed_file_types(value: str) -> list[str]:
items = [item.strip() for item in value.split(",") if item.strip()]
return items or ["*"]
def parse_setup_options(data: dict) -> dict:
upload_size_unit = get_form_value(data, "upload_size_unit", "MB").upper()
if upload_size_unit not in FILE_SIZE_UNITS:
raise ValueError("文件大小单位不正确")
upload_size_value = parse_int_field(
data, "upload_size_value", 10, "文件大小限制", min_value=1
)
save_time_unit = get_form_value(data, "save_time_unit", "day")
if save_time_unit not in SAVE_TIME_UNITS:
raise ValueError("最长保存时间单位不正确")
save_time_value = parse_int_field(
data, "save_time_value", 0, "最长保存时间", min_value=0
)
expire_styles = get_form_list(data, "expire_style")
valid_expire_styles = {style for style, _label in EXPIRE_STYLE_OPTIONS}
expire_styles = [style for style in expire_styles if style in valid_expire_styles]
if not expire_styles:
raise ValueError("至少需要选择一种过期方式")
code_generate_type = get_form_value(
data, "code_generate_type", DEFAULT_CONFIG["code_generate_type"]
)
if code_generate_type not in {"number", "secret"}:
raise ValueError("提取码类型不正确")
return {
"allowed_file_types": parse_allowed_file_types(
get_form_value(data, "allowed_file_types", "*")
),
"code_generate_type": code_generate_type,
"enable_chunk": int(normalize_bool_field(data, "enable_chunk", False)),
"error_count": parse_int_field(
data, "error_count", DEFAULT_CONFIG["error_count"], "取件错误次数限制", 1
),
"error_minute": parse_int_field(
data, "error_minute", DEFAULT_CONFIG["error_minute"], "取件错误检测窗口", 1
),
"login_count": parse_int_field(
data, "login_count", DEFAULT_CONFIG["login_count"], "登录失败次数限制", 1
),
"login_minute": parse_int_field(
data, "login_minute", DEFAULT_CONFIG["login_minute"], "登录失败检测窗口", 1
),
"expire_style": expire_styles,
"max_save_seconds": save_time_value * SAVE_TIME_UNITS[save_time_unit],
"open_upload": int(normalize_bool_field(data, "open_upload", True)),
"upload_count": parse_int_field(
data, "upload_count", DEFAULT_CONFIG["upload_count"], "上传次数限制", 1
),
"upload_minute": parse_int_field(
data, "upload_minute", DEFAULT_CONFIG["upload_minute"], "上传检测窗口", 1
),
"upload_size": upload_size_value * FILE_SIZE_UNITS[upload_size_unit],
}
def build_expire_style_inputs(selected_styles: list[str]) -> str:
inputs = []
selected = set(selected_styles)
for style, label in EXPIRE_STYLE_OPTIONS:
checked = " checked" if style in selected else ""
inputs.append(
f'<label class="check"><input type="checkbox" name="expire_style" value="{style}"{checked}> {label}</label>'
)
return "\n ".join(inputs)
def build_setup_page(error: str = "", form: dict | None = None) -> str:
form = form or {}
escaped_error = html.escape(error)
escaped_site_name = html.escape(
get_form_value(form, "site_name", DEFAULT_CONFIG["name"])
)
escaped_allowed_types = html.escape(get_form_value(form, "allowed_file_types", "*"))
upload_size_value = html.escape(get_form_value(form, "upload_size_value", "10"))
upload_size_unit = get_form_value(form, "upload_size_unit", "MB").upper()
save_time_value = html.escape(get_form_value(form, "save_time_value", "0"))
save_time_unit = get_form_value(form, "save_time_unit", "day")
upload_minute = html.escape(
get_form_value(form, "upload_minute", str(DEFAULT_CONFIG["upload_minute"]))
)
upload_count = html.escape(
get_form_value(form, "upload_count", str(DEFAULT_CONFIG["upload_count"]))
)
error_minute = html.escape(
get_form_value(form, "error_minute", str(DEFAULT_CONFIG["error_minute"]))
)
error_count = html.escape(
get_form_value(form, "error_count", str(DEFAULT_CONFIG["error_count"]))
)
login_minute = html.escape(
get_form_value(form, "login_minute", str(DEFAULT_CONFIG["login_minute"]))
)
login_count = html.escape(
get_form_value(form, "login_count", str(DEFAULT_CONFIG["login_count"]))
)
open_upload_checked = (
" checked" if normalize_bool_field(form, "open_upload", True) else ""
)
chunk_checked = (
" checked" if normalize_bool_field(form, "enable_chunk", False) else ""
)
code_generate_type = get_form_value(
form, "code_generate_type", DEFAULT_CONFIG["code_generate_type"]
)
selected_expire_styles = get_form_list(form, "expire_style") or list(
DEFAULT_CONFIG["expire_style"]
)
expire_style_inputs = build_expire_style_inputs(selected_expire_styles)
size_unit_options = "\n".join(
f'<option value="{unit}"{" selected" if unit == upload_size_unit else ""}>{unit}</option>'
for unit in FILE_SIZE_UNITS
)
save_time_unit_options = "\n".join(
f'<option value="{unit}"{" selected" if unit == save_time_unit else ""}>{label}</option>'
for unit, label in [
("second", "秒"),
("minute", "分钟"),
("hour", "小时"),
("day", "天"),
]
)
code_type_options = "\n".join(
f'<option value="{value}"{" selected" if value == code_generate_type else ""}>{label}</option>'
for value, label in [("number", "数字"), ("secret", "随机字符")]
)
error_block = (
f'<div class="alert" role="alert">{escaped_error}</div>' if escaped_error else ""
)
return f"""<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>初始化 FileCodeBox</title>
<style>
:root {{
color-scheme: light;
--bg: #f5f5f7;
--panel: rgba(255, 255, 255, .86);
--panel-strong: rgba(255, 255, 255, .96);
--text: #18181b;
--muted: #71717a;
--line: rgba(228, 228, 231, .9);
--line-strong: rgba(212, 212, 216, .95);
--primary: #18181b;
--primary-soft: #f4f4f5;
--danger-bg: #fef2f2;
--danger: #b91c1c;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0;
min-height: 100vh;
display: grid;
place-items: center;
padding: 14px;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: var(--bg);
color: var(--text);
}}
main {{
width: min(100%, 1040px);
padding: 14px;
border: 1px solid rgba(255, 255, 255, .8);
border-radius: 20px;
background: rgba(255, 255, 255, .62);
box-shadow: 0 22px 70px -34px rgba(24, 24, 27, .32);
backdrop-filter: blur(22px);
}}
.setup-header {{
display: flex;
align-items: center;
justify-content: space-between;
gap: 16px;
padding: 10px 10px 12px;
}}
.brand-mark {{
flex: 0 0 auto;
width: 40px;
height: 40px;
display: grid;
place-items: center;
border-radius: 14px;
background: #18181b;
color: #fff;
font-weight: 800;
letter-spacing: 0;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .42);
}}
.title-wrap {{
min-width: 0;
display: flex;
align-items: center;
gap: 12px;
}}
h1 {{
margin: 0;
font-size: 21px;
line-height: 1.25;
letter-spacing: 0;
font-weight: 750;
}}
p {{
margin: 2px 0 0;
color: var(--muted);
line-height: 1.35;
font-size: 13px;
}}
form {{
margin: 0;
padding: 12px;
border: 1px solid var(--line);
border-radius: 16px;
background: var(--panel);
}}
.panel-grid {{
display: grid;
grid-template-columns: 1.05fr 1fr 1fr;
gap: 10px;
}}
.panel {{
min-width: 0;
padding: 10px;
border: 1px solid var(--line);
border-radius: 14px;
background: rgba(255, 255, 255, .68);
}}
.panel-wide {{
grid-column: 1 / -1;
display: grid;
grid-template-columns: minmax(0, 1.3fr) minmax(0, .9fr);
gap: 12px;
align-items: end;
}}
.panel-title {{
margin: 0 0 2px;
color: #18181b;
font-size: 13px;
font-weight: 760;
}}
label {{
display: block;
margin: 8px 0 5px;
font-weight: 650;
font-size: 12px;
color: #3f3f46;
}}
input, select {{
width: 100%;
height: 34px;
border: 1px solid var(--line);
border-radius: 10px;
padding: 0 10px;
font: inherit;
color: var(--text);
outline: none;
background: rgba(255, 255, 255, .84);
transition: border-color .2s ease, box-shadow .2s ease, background-color .2s ease;
}}
input:focus, select:focus {{
border-color: #a1a1aa;
background: #fff;
box-shadow: 0 10px 22px -12px rgba(24, 24, 27, .2), 0 0 0 3px rgba(24, 24, 27, .06);
}}
.grid {{
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 6px 12px;
}}
.full {{ grid-column: 1 / -1; }}
.span-2 {{ grid-column: span 2; }}
.row {{
display: grid;
grid-template-columns: minmax(0, 1fr) 88px;
gap: 8px;
}}
.checks {{
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-top: 5px;
}}
.check {{
display: flex;
align-items: center;
gap: 8px;
min-height: 36px;
margin: 0;
padding: 0 9px;
border: 1px solid var(--line);
border-radius: 10px;
font-weight: 550;
font-size: 13px;
color: var(--text);
white-space: nowrap;
background: rgba(255, 255, 255, .72);
transition: border-color .2s ease, background-color .2s ease, box-shadow .2s ease;
}}
.check:hover {{
border-color: var(--line-strong);
background: #fff;
box-shadow: 0 8px 20px -8px rgba(0, 0, 0, .1);
}}
.check input {{
width: 16px;
height: 16px;
padding: 0;
box-shadow: none;
}}
.section-title {{
display: none;
}}
.help {{
margin: 4px 0 0;
font-size: 12px;
color: var(--muted);
}}
.alert {{
margin-bottom: 8px;
padding: 9px 12px;
border-radius: 12px;
background: var(--danger-bg);
color: var(--danger);
font-size: 13px;
}}
button {{
width: 100%;
height: 38px;
margin-top: 12px;
border: 0;
border-radius: 12px;
background: var(--primary);
color: white;
font: inherit;
font-weight: 700;
cursor: pointer;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .42);
transition: background-color .2s ease, box-shadow .2s ease;
}}
button:hover {{
background: #27272a;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .56);
}}
.hint {{
margin-top: 8px;
margin-bottom: 0;
font-size: 12px;
}}
.compact-help {{
display: inline;
margin-left: 6px;
color: var(--muted);
font-weight: 500;
font-size: 12px;
}}
.topline {{
display: none;
}}
.badge {{
min-height: 26px;
display: inline-flex;
align-items: center;
padding: 0 10px;
border: 1px solid var(--line);
border-radius: 999px;
background: rgba(255, 255, 255, .76);
color: #52525b;
font-size: 12px;
font-weight: 650;
white-space: nowrap;
}}
@media (max-width: 640px) {{
body {{ padding: 10px; }}
main {{ padding: 10px; }}
.setup-header {{ align-items: flex-start; padding: 6px 4px 10px; }}
.title-wrap {{ align-items: flex-start; }}
.panel-grid, .panel-wide {{ grid-template-columns: 1fr; }}
.grid, .checks {{ grid-template-columns: 1fr; }}
.span-2 {{ grid-column: auto; }}
.row {{ grid-template-columns: 1fr; }}
.badge {{ display: none; }}
}}
</style>
</head>
<body>
<main>
<div class="setup-header">
<div class="title-wrap">
<div class="brand-mark">FCB</div>
<div>
<h1>初始化 FileCodeBox</h1>
<p>首次配置管理员密码、上传限制和取件策略,后续可在后台调整。</p>
</div>
</div>
<span class="badge">首次配置向导</span>
</div>
<form method="post" action="/setup" autocomplete="off">
{error_block}
<div class="panel-grid">
<section class="panel">
<div class="panel-title">基础设置</div>
<label for="site_name">站点名称</label>
<input id="site_name" name="site_name" maxlength="80" value="{escaped_site_name}" placeholder="文件快递柜 - FileCodeBox">
<label for="admin_password">管理员密码</label>
<input id="admin_password" name="admin_password" type="password" minlength="8" required autofocus>
<label for="confirm_password">确认管理员密码</label>
<input id="confirm_password" name="confirm_password" type="password" minlength="8" required>
</section>
<section class="panel">
<div class="panel-title">上传设置</div>
<label for="upload_size_value">单文件大小限制</label>
<div class="row">
<input id="upload_size_value" name="upload_size_value" type="number" min="1" value="{upload_size_value}" required>
<select name="upload_size_unit" aria-label="文件大小单位">
{size_unit_options}
</select>
</div>
<label for="upload_count">上传频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="upload_count" name="upload_count" type="number" min="1" value="{upload_count}" required>
<input name="upload_minute" type="number" min="1" value="{upload_minute}" aria-label="上传检测窗口分钟" required>
</div>
<div class="grid">
<input type="hidden" name="open_upload" value="0">
<label class="check"><input type="checkbox" name="open_upload" value="1"{open_upload_checked}> 允许游客上传</label>
<input type="hidden" name="enable_chunk" value="0">
<label class="check"><input type="checkbox" name="enable_chunk" value="1"{chunk_checked}> 启用切片上传</label>
</div>
</section>
<section class="panel">
<div class="panel-title">取件与保存</div>
<label for="error_count">取件错误频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="error_count" name="error_count" type="number" min="1" value="{error_count}" required>
<input name="error_minute" type="number" min="1" value="{error_minute}" aria-label="取件错误检测窗口分钟" required>
</div>
<label for="login_count">管理员登录失败频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="login_count" name="login_count" type="number" min="1" value="{login_count}" required>
<input name="login_minute" type="number" min="1" value="{login_minute}" aria-label="登录失败检测窗口分钟" required>
</div>
<label for="save_time_value">最长保存时间</label>
<div class="row">
<input id="save_time_value" name="save_time_value" type="number" min="0" value="{save_time_value}" required>
<select name="save_time_unit" aria-label="最长保存时间单位">
{save_time_unit_options}
</select>
</div>
<label for="code_generate_type">提取码类型</label>
<select id="code_generate_type" name="code_generate_type">
{code_type_options}
</select>
</section>
<section class="panel panel-wide">
<div>
<div class="panel-title">可用策略</div>
<label>允许的过期方式</label>
<div class="checks">
{expire_style_inputs}
</div>
</div>
<div>
<label for="allowed_file_types">允许文件类型 <span class="compact-help">逗号分隔,* 表示不限制</span></label>
<input id="allowed_file_types" name="allowed_file_types" value="{escaped_allowed_types}" placeholder="* 或 .zip, image/*">
</div>
</section>
</div>
<button type="submit">完成初始化</button>
</form>
</main>
</body>
</html>"""
def build_setup_success_page() -> str:
return """<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="refresh" content="2;url=/#/admin">
<title>初始化完成</title>
<style>
body {
margin: 0;
min-height: 100vh;
display: grid;
place-items: center;
padding: 24px;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: #f6f8fb;
color: #172033;
}
main {
width: min(100%, 420px);
padding: 28px;
border: 1px solid #d9e1ec;
border-radius: 8px;
background: #fff;
text-align: center;
box-shadow: 0 18px 50px rgba(23, 32, 51, .08);
}
h1 { margin: 0 0 10px; font-size: 24px; letter-spacing: 0; }
p { margin: 0 0 22px; color: #60708a; line-height: 1.65; }
a {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 42px;
padding: 0 18px;
border-radius: 6px;
background: #2563eb;
color: white;
text-decoration: none;
font-weight: 700;
}
</style>
</head>
<body>
<main>
<h1>初始化完成</h1>
<p>管理员密码已设置,请使用刚才的密码登录后台。</p>
<a href="/#/admin">进入后台</a>
</main>
</body>
</html>"""
def setup_response(content: str, status_code: int = 200) -> HTMLResponse:
return HTMLResponse(
content=content,
status_code=status_code,
media_type="text/html",
headers={"Cache-Control": "no-store"},
)
def is_setup_path(path: str) -> bool:
return path.rstrip("/") == "/setup"
def wants_html_response(request: Request) -> bool:
if request.method not in {"GET", "HEAD"}:
return False
accept = request.headers.get("accept", "")
return not accept or "text/html" in accept or "*/*" in accept
async def read_setup_payload(request: Request) -> dict:
content_type = request.headers.get("content-type", "")
if "application/json" in content_type:
data = await request.json()
return data if isinstance(data, dict) else {}
body = (await request.body()).decode("utf-8")
return {
key: values if len(values) > 1 else values[-1]
for key, values in parse_qs(body).items()
}
+19 -15
View File
@@ -4,7 +4,6 @@
# @Software: PyCharm
import asyncio
import datetime
import logging
import os
from tortoise.expressions import Q
@@ -16,12 +15,14 @@ from apps.base.models import (
UploadChunk,
)
from apps.base.utils import ip_limit, get_chunk_file_path_name
from core.config import refresh_settings
from apps.base.config import refresh_settings
from core.logger import logger
from core.settings import settings, data_root
from core.storage import FileStorageInterface, storages
from core.storage import FileStorageInterface, StoredFile, storages
from apps.base.services import stored_file_of
from apps.base.local_share import is_local_ref
from core.utils import get_now
async def delete_expire_files():
while True:
try:
@@ -41,15 +42,16 @@ async def delete_expire_files():
).all()
for exp in expire_data:
try:
await file_storage.delete_file(exp)
if not is_local_ref(exp):
await file_storage.delete_file(stored_file_of(exp))
except Exception as e:
logging.error(f"删除过期文件失败 code={exp.code}: {e}")
logger.error(f"删除过期文件失败 code={exp.code}: {e}")
try:
await exp.delete()
except Exception as e:
logging.error(f"删除记录失败 code={exp.code}: {e}")
logger.error(f"删除记录失败 code={exp.code}: {e}")
except Exception as e:
logging.error(e)
logger.error(e)
finally:
await asyncio.sleep(600)
@@ -75,23 +77,25 @@ async def clean_incomplete_uploads():
)
await file_storage.clean_chunks(session.upload_id, save_path)
except Exception as e:
logging.error(
logger.error(
f"清理分片文件失败 upload_id={session.upload_id}: {e}"
)
# 存储清理失败(包括部分删除)时保留记录,下轮继续重试。
continue
try:
await UploadChunk.filter(upload_id=session.upload_id).delete()
await StorageReservation.filter(
token=f"chunk:{session.upload_id}"
).delete()
logging.info(f"已清理过期上传会话 upload_id={session.upload_id}")
logger.info(f"已清理过期上传会话 upload_id={session.upload_id}")
except Exception as e:
logging.error(
logger.error(
f"删除分片记录失败 upload_id={session.upload_id}: {e}"
)
except Exception as e:
logging.error(f"清理未完成上传任务异常: {e}")
logger.error(f"清理未完成上传任务异常: {e}")
finally:
await asyncio.sleep(3600)
@@ -110,13 +114,13 @@ async def clean_expired_presign_sessions():
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
FileCodes(
StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception as e:
logging.error(
logger.error(
"清理过期直传文件失败 "
f"upload_id={session.upload_id}: {e}"
)
@@ -125,6 +129,6 @@ async def clean_expired_presign_sessions():
).delete()
await session.delete()
except Exception as e:
logging.error(f"清理过期预签名会话异常: {e}")
logger.error(f"清理过期预签名会话异常: {e}")
finally:
await asyncio.sleep(900)
+20 -28
View File
@@ -1,5 +1,4 @@
import datetime
import hashlib
import os
import uuid
from urllib.parse import unquote
@@ -21,33 +20,37 @@ from core.utils import (
def validate_expire_style(expire_style: str) -> str:
"""校验过期方式是否在管理员配置的白名单内。"""
if expire_style not in settings.expireStyle:
if expire_style not in settings.expire_style:
raise HTTPException(status_code=400, detail="过期时间类型错误")
return expire_style
async def get_file_path_name(file: UploadFile) -> Tuple[str, str, str, str, str]:
async def build_file_path(
file_name: str, file_uuid: str
) -> Tuple[str, str, str, str, str]:
"""Single source of storage path generation (date dir + UUID), shared by
regular, chunked, and presigned uploads.
Always use get_now() (UTC+8); do not switch to server-local time.
"""
today = await get_now()
storage_path = settings.storage_path.strip("/")
file_uuid = uuid.uuid4().hex
filename = await sanitize_filename(unquote(file.filename or ""))
filename = await sanitize_filename(unquote(file_name or ""))
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{file_uuid}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(filename)
save_path = f"{path}/{filename}"
save_path = f"{path}/{prefix}{suffix}"
return path, suffix, prefix, filename, save_path
async def get_file_path_name(file: UploadFile) -> Tuple[str, str, str, str, str]:
return await build_file_path(file.filename or "", uuid.uuid4().hex)
async def get_chunk_file_path_name(
file_name: str, upload_id: str
) -> Tuple[str, str, str, str, str]:
today = await get_now()
storage_path = settings.storage_path.strip("/")
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{upload_id}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(file_name)
save_path = f"{path}/{prefix}{suffix}"
return path, suffix, prefix, file_name, save_path
return await build_file_path(file_name, upload_id)
async def get_expire_info(
@@ -120,20 +123,9 @@ async def get_random_code(style: str | None = None) -> str:
return str(code)
async def calculate_file_hash(file: UploadFile, chunk_size=1024 * 1024) -> str:
sha = hashlib.sha256()
await file.seek(0)
while True:
chunk = await file.read(chunk_size)
if not chunk:
break
sha.update(chunk)
await file.seek(0)
return sha.hexdigest()
ip_limit = {
"error": IPRateLimit(count=settings.errorCount, minutes=settings.errorMinute),
"metadata": IPRateLimit(count=settings.errorCount, minutes=settings.errorMinute),
"upload": IPRateLimit(count=settings.uploadCount, minutes=settings.uploadMinute),
"error": IPRateLimit(count=settings.error_count, minutes=settings.error_minute),
"metadata": IPRateLimit(count=settings.error_count, minutes=settings.error_minute),
"upload": IPRateLimit(count=settings.upload_count, minutes=settings.upload_minute),
"login": IPRateLimit(count=settings.login_count, minutes=settings.login_minute),
}
+72 -358
View File
@@ -1,10 +1,9 @@
import datetime
from fnmatch import fnmatchcase
import hashlib
import hmac
import os
import uuid
from datetime import timedelta
from urllib.parse import quote, unquote
from urllib.parse import unquote
from typing import Optional, Tuple, Union
@@ -14,22 +13,32 @@ from starlette import status
from starlette.responses import Response
from tortoise.expressions import Case, F, Q, When
from apps.admin.dependencies import share_required_login
from apps.base.auth import share_required_login
from apps.base.models import FileCodes, UploadChunk, PresignUploadSession
from apps.base.quota import release_storage, reserve_storage
from apps.base.services import (
PRESIGN_SESSION_EXPIRES,
FileUploadService,
get_stored_download,
response_from_download,
stored_file_of,
validate_file_size,
)
from core.storage import StoredFile, build_attachment_headers
from core.logger import logger
from apps.base.schemas import (
SelectFileModel,
InitChunkUploadModel,
CompleteUploadModel,
PresignUploadInitRequest,
)
from apps.base.file_validation import validate_file_type, validate_upload_file, validate_header_bytes
from apps.base.utils import (
get_expire_info,
get_file_path_name,
ip_limit,
get_chunk_file_path_name,
validate_expire_style,
)
from apps.base.local_share import is_local_ref
from core.response import APIResponse
from core.settings import settings
from core.storage import storages, FileStorageInterface
@@ -44,102 +53,6 @@ share_api = APIRouter(prefix="/share", tags=["分享"])
# ============ 公共服务层 ============
class FileUploadService:
"""统一的文件上传服务"""
@staticmethod
async def generate_file_path(
file_name: str, upload_id: Optional[str] = None
) -> tuple[str, str, str, str, str]:
"""统一的路径生成"""
today = datetime.datetime.now()
storage_path = settings.storage_path.strip("/")
file_uuid = upload_id or uuid.uuid4().hex
filename = await sanitize_filename(unquote(file_name))
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{file_uuid}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(filename)
save_path = f"{path}/{filename}"
return path, suffix, prefix, filename, save_path
@staticmethod
async def create_file_record(
file_name: str,
file_size: int,
file_path: str,
expire_value: int,
expire_style: str,
**extra_fields,
) -> str:
"""统一创建FileCodes记录,返回code"""
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
prefix, suffix = os.path.splitext(file_name)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=file_name,
file_path=file_path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
**extra_fields,
)
return code
async def validate_file_size(file: UploadFile, max_size: int) -> int:
size = file.size
if size is None:
await file.seek(0, 2) # type: ignore[arg-type]
size = file.file.tell()
await file.seek(0)
if size > max_size:
max_size_mb = max_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"大小超过限制,最大为{max_size_mb:.2f} MB"
)
return size
def normalize_allowed_file_types() -> list[str]:
raw_value = settings.allowed_file_types
if isinstance(raw_value, str):
values = raw_value.replace(";", ",").split(",")
elif isinstance(raw_value, (list, tuple, set)):
values = raw_value
else:
values = []
allowed = [str(item).strip().lower() for item in values if str(item).strip()]
return allowed or ["*"]
def validate_file_type(file_name: str, content_type: Optional[str] = None) -> None:
allowed = normalize_allowed_file_types()
if "*" in allowed or "*/*" in allowed:
return
normalized_name = file_name.strip().lower()
normalized_content_type = (content_type or "").strip().lower()
for rule in allowed:
if "/" in rule and fnmatchcase(normalized_content_type, rule):
return
extension = rule if rule.startswith(".") else f".{rule}"
if normalized_name.endswith(extension):
return
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
async def create_file_code(code, **kwargs):
return await FileCodes.create(code=code, **kwargs)
def normalize_share_code(code: str) -> str:
@@ -159,23 +72,7 @@ async def share_text(
if text_size > max_txt_size:
raise HTTPException(status_code=403, detail="内容过多,建议采用文件形式")
reservation_token = f"text:{uuid.uuid4().hex}"
await reserve_storage(reservation_token, text_size, ttl_seconds=300)
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await create_file_code(
code=code,
text=text,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
size=text_size,
prefix="Text",
)
finally:
await release_storage(reservation_token)
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code})
@@ -187,41 +84,14 @@ async def share_file(
file: UploadFile = File(...),
ip: str = Depends(ip_limit["upload"]),
):
file_size = await validate_file_size(file, settings.uploadSize)
validate_file_type(file.filename or "", file.content_type)
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
validate_expire_style(expire_style)
path, suffix, prefix, uuid_file_name, save_path = await get_file_path_name(file)
reservation_token = f"file:{uuid.uuid4().hex}"
await reserve_storage(reservation_token, file_size, ttl_seconds=3600)
file_storage: FileStorageInterface = storages[settings.file_storage]()
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await file_storage.save_file(file, save_path)
await create_file_code(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=uuid_file_name,
file_path=path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
)
except Exception:
try:
await file_storage.delete_file(
FileCodes(file_path=path, uuid_file_name=uuid_file_name)
)
except Exception:
pass
raise
finally:
await release_storage(reservation_token)
detail = await FileUploadService.create_file_share(
file, size=file_size, expire_value=expire_value, expire_style=expire_style
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": file.filename})
return APIResponse(detail=detail)
async def get_code_file_by_code(
@@ -272,7 +142,6 @@ def build_file_metadata(file_code: FileCodes) -> dict:
"is_text": is_text,
"created_at": file_code.created_at,
"expired_at": file_code.expired_at,
"expires_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"remaining_downloads": remaining_downloads,
@@ -285,11 +154,11 @@ async def build_select_detail(
metadata = build_file_metadata(file_code)
if file_code.text is not None:
download_url = None
elif file_code.expired_count >= 0:
# 有次数限制的文件必须经过下载接口,第三方直链无法阻止重复使用。
elif file_code.expired_count >= 0 or is_local_ref(file_code):
# 次数限制与 NAS 引用都必须走下载接口,避免直链绕过次数或打到错误存储。
download_url = await get_proxy_file_url(file_code.code)
else:
download_url = await file_storage.get_file_url(file_code)
download_url = await file_storage.get_file_url(stored_file_of(file_code))
content = file_code.text if file_code.text is not None else None
return {
**metadata,
@@ -325,6 +194,8 @@ async def post_file_metadata(
return APIResponse(detail=build_file_metadata(file_code))
@share_api.get("/select/")
async def get_code_file(code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
@@ -341,13 +212,9 @@ async def get_code_file(code: str, ip: str = Depends(ip_limit["error"])):
return Response(
content=file_code.text,
media_type="text/plain",
headers={
"Content-Disposition": (
f"attachment; filename*=UTF-8''{quote(filename, safe='')}"
)
},
headers=build_attachment_headers(filename),
)
return await file_storage.get_file_response(file_code)
return response_from_download(await get_stored_download(file_code, file_storage))
@share_api.post("/select/")
@@ -375,12 +242,16 @@ async def select_file(data: SelectFileModel, ip: str = Depends(ip_limit["error"]
async def download_file(key: str, code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
normalized_code = normalize_share_code(code)
# 同时接受当前窗口与上一窗口 token,避免时间窗边界竞态导致偶发 403
valid_keys = {
# 同时接受当前窗口与上一窗口 token,避免时间窗边界竞态导致偶发 403。
# 逐个常量时间比较(hmac.compare_digest),避免 set 成员判断的时序侧信道。
valid_keys = [
await get_select_token(normalized_code, offset=0),
await get_select_token(normalized_code, offset=1),
}
if key not in valid_keys:
]
if not any(
hmac.compare_digest(key.encode(), candidate.encode())
for candidate in valid_keys
):
ip_limit["error"].add_ip(ip)
raise HTTPException(status_code=403, detail="下载鉴权失败")
has, file_code = await get_code_file_by_code(normalized_code)
@@ -392,7 +263,7 @@ async def download_file(key: str, code: str, ip: str = Depends(ip_limit["error"]
return (
APIResponse(detail=file_code.text)
if file_code.text
else await file_storage.get_file_response(file_code)
else response_from_download(await get_stored_download(file_code, file_storage))
)
@@ -423,37 +294,24 @@ async def parse_complete_upload(request: Request) -> CompleteUploadModel:
@chunk_api.post("/upload/init/", dependencies=[Depends(share_required_login)])
async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chunk_upload)):
validate_file_type(data.file_name)
safe_file_name = await sanitize_filename(unquote(data.file_name or ""))
validate_file_type(safe_file_name)
# 服务端校验:根据 total_chunks * chunk_size 计算理论最大上传量
total_chunks = (data.file_size + data.chunk_size - 1) // data.chunk_size
max_possible_size = total_chunks * data.chunk_size
if max_possible_size > settings.uploadSize:
max_size_mb = settings.uploadSize / (1024 * 1024)
if max_possible_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"文件大小超过限制,最大为 {max_size_mb:.2f} MB"
)
# # 秒传检查
# existing = await FileCodes.filter(file_hash=data.file_hash).first()
# if existing:
# if await existing.is_expired():
# file_storage: FileStorageInterface = storages[settings.file_storage](
# )
# await file_storage.delete_file(existing)
# await existing.delete()
# else:
# return APIResponse(detail={
# "code": existing.code,
# "existed": True,
# "name": f'{existing.prefix}{existing.suffix}'
# })
# 断点续传:检查是否存在相同文件的未完成上传会话
existing_session = await UploadChunk.filter(
chunk_hash=data.file_hash,
chunk_index=-1,
completed=False,
file_size=data.file_size,
file_name=data.file_name,
file_name=safe_file_name,
).first()
if existing_session:
@@ -470,7 +328,7 @@ async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chun
* 3600,
)
uploaded_chunks = await UploadChunk.filter(
upload_id=existing_session.upload_id, completed=True
upload_id=existing_session.upload_id, completed=True, chunk_index__gte=0
).values_list("chunk_index", flat=True)
return APIResponse(
detail={
@@ -489,7 +347,9 @@ async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chun
await reserve_storage(
reservation_token, data.file_size, ttl_seconds=chunk_expire_seconds
)
_, _, _, _, save_path = await get_chunk_file_path_name(data.file_name, upload_id)
_, _, _, safe_file_name, save_path = await get_chunk_file_path_name(
data.file_name, upload_id
)
try:
await UploadChunk.create(
upload_id=upload_id,
@@ -498,7 +358,7 @@ async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chun
file_size=data.file_size,
chunk_size=data.chunk_size,
chunk_hash=data.file_hash,
file_name=data.file_name,
file_name=safe_file_name,
save_path=save_path,
)
except Exception:
@@ -528,6 +388,8 @@ async def upload_chunk(
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
if chunk_info.completed:
raise HTTPException(409, "上传会话已完成,仅待清理")
await reserve_storage(
f"chunk:{upload_id}",
chunk_info.file_size,
@@ -549,6 +411,8 @@ async def upload_chunk(
# 读取分片数据并计算哈希
chunk_data = await chunk.read()
if chunk_index == 0:
validate_header_bytes(chunk_info.file_name, None, chunk_data[:64])
chunk_size = len(chunk_data)
# 校验分片大小不超过声明的 chunk_size
@@ -560,12 +424,12 @@ async def upload_chunk(
# 计算已上传分片数,校验累计大小不超限(用分片数 * chunk_size 估算)
uploaded_count = await UploadChunk.filter(
upload_id=upload_id, completed=True
upload_id=upload_id, completed=True, chunk_index__gte=0
).count()
# 已上传分片的最大可能大小 + 当前分片
max_uploaded_size = uploaded_count * chunk_info.chunk_size + chunk_size
if max_uploaded_size > settings.uploadSize:
max_size_mb = settings.uploadSize / (1024 * 1024)
if max_uploaded_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"累计上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
@@ -616,8 +480,9 @@ async def cancel_upload(upload_id: str):
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception as e:
pass
except Exception:
logger.warning("取消分片上传:清理分片文件失败 upload_id=%s", upload_id, exc_info=True)
raise HTTPException(503, "分片清理失败,请稍后重试")
# 清理数据库记录
await UploadChunk.filter(upload_id=upload_id).delete()
@@ -637,7 +502,7 @@ async def get_upload_status(upload_id: str):
# 获取已上传的分片列表
uploaded_chunks = await UploadChunk.filter(
upload_id=upload_id, completed=True
upload_id=upload_id, completed=True, chunk_index__gte=0
).values_list("chunk_index", flat=True)
return APIResponse(
@@ -665,87 +530,19 @@ async def complete_upload(
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
if chunk_info.completed:
raise HTTPException(409, "上传会话已完成,仅待清理")
validate_expire_style(data.expire_style)
await reserve_storage(
f"chunk:{upload_id}",
chunk_info.file_size,
ttl_seconds=max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600,
detail = await FileUploadService.complete_chunked_upload(
upload_id, chunk_info, data.expire_value, data.expire_style
)
storage = storages[settings.file_storage]()
# 验证所有分片
completed_chunks_list = await UploadChunk.filter(
upload_id=upload_id, completed=True
).all()
if len(completed_chunks_list) != chunk_info.total_chunks:
raise HTTPException(status.HTTP_400_BAD_REQUEST, detail="分片不完整")
# 用分片数 * chunk_size 校验最大可能大小
max_total_size = len(completed_chunks_list) * chunk_info.chunk_size
if max_total_size > settings.uploadSize:
save_path = chunk_info.save_path
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
pass
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
max_size_mb = settings.uploadSize / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"实际上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
save_path = chunk_info.save_path
path = os.path.dirname(save_path) if save_path else ""
prefix, suffix = os.path.splitext(chunk_info.file_name)
try:
# 合并文件并计算哈希
_, file_hash = await storage.merge_chunks(upload_id, chunk_info, save_path)
# 创建文件记录
expired_at, expired_count, used_count, code = await get_expire_info(
data.expire_value, data.expire_style
)
await FileCodes.create(
code=code,
file_hash=file_hash, # 使用合并后计算的哈希
is_chunked=True,
upload_id=upload_id,
size=chunk_info.file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
file_path=path,
uuid_file_name=f"{prefix}{suffix}",
prefix=prefix,
suffix=suffix,
)
# 清理临时文件
await storage.clean_chunks(upload_id, save_path)
# 清理数据库中的分片记录
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": chunk_info.file_name})
except ValueError as e:
raise HTTPException(status.HTTP_400_BAD_REQUEST, detail=str(e))
except Exception as e:
# 合并失败时清理临时文件
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
pass
raise HTTPException(
status.HTTP_500_INTERNAL_SERVER_ERROR, detail=f"文件合并失败: {str(e)}"
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail=detail)
# ============ 预签名上传API ============
presign_api = APIRouter(prefix="/presign", tags=["预签名上传"])
PRESIGN_SESSION_EXPIRES = 900 # 15分钟
def build_proxy_upload_urls(upload_id: str) -> dict:
proxy_upload_url = f"/presign/upload/proxy/{upload_id}"
@@ -777,10 +574,10 @@ async def presign_upload_init(
):
"""初始化预签名上传,S3返回直传URL,其他存储返回代理URL"""
validate_file_type(data.file_name)
if data.file_size > settings.uploadSize:
if data.file_size > settings.upload_size:
raise HTTPException(
403,
f"文件大小超过限制,最大为 {settings.uploadSize / (1024 * 1024):.2f} MB",
f"文件大小超过限制,最大为 {settings.upload_size / (1024 * 1024):.2f} MB",
)
validate_expire_style(data.expire_style)
@@ -837,44 +634,7 @@ async def presign_upload_proxy(
):
"""代理模式上传,服务器转存到存储后端"""
session = await _get_valid_session(upload_id, expected_mode="proxy")
await reserve_storage(
f"presign:{upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
file_size = await validate_file_size(file, settings.uploadSize)
if abs(file_size - session.file_size) > 1024:
raise HTTPException(400, "文件大小与声明不符")
storage: FileStorageInterface = storages[settings.file_storage]()
try:
await storage.save_file(file, session.save_path)
except Exception as e:
raise HTTPException(500, f"文件保存失败: {str(e)}")
try:
code = await FileUploadService.create_file_record(
session.file_name,
file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
try:
await storage.delete_file(
FileCodes(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception:
pass
raise
await session.delete()
await release_storage(f"presign:{upload_id}")
code = await FileUploadService.commit_proxy_upload(session, file)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@@ -885,53 +645,7 @@ async def presign_upload_proxy(
async def presign_upload_confirm(upload_id: str, ip: str = Depends(ip_limit["upload"])):
"""直传确认,客户端完成S3直传后调用获取分享码"""
session = await _get_valid_session(upload_id, expected_mode="direct")
try:
await reserve_storage(
f"presign:{upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
except HTTPException:
storage: FileStorageInterface = storages[settings.file_storage]()
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
FileCodes(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
finally:
await session.delete()
await release_storage(f"presign:{upload_id}")
raise
storage: FileStorageInterface = storages[settings.file_storage]()
if not await storage.file_exists(session.save_path):
raise HTTPException(404, "文件未上传或上传失败")
try:
code = await FileUploadService.create_file_record(
session.file_name,
session.file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
try:
await storage.delete_file(
FileCodes(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception:
pass
raise
await session.delete()
await release_storage(f"presign:{upload_id}")
code = await FileUploadService.confirm_direct_upload(session)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@@ -969,13 +683,13 @@ async def presign_upload_cancel(upload_id: str):
storage: FileStorageInterface = storages[settings.file_storage]()
try:
if await storage.file_exists(session.save_path):
temp_file_code = FileCodes(
temp_file_code = StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
await storage.delete_file(temp_file_code)
except Exception:
pass
logger.warning("取消预签名会话:清理临时文件失败 upload_id=%s", upload_id, exc_info=True)
await session.delete()
await release_storage(f"presign:{upload_id}")
+13 -2
View File
@@ -25,6 +25,17 @@ def get_db_config() -> dict:
"journal_mode": "WAL",
"busy_timeout": 10000,
"foreign_keys": "ON",
# 小内存 VPS 下的 IOPS 与内存折中。Tortoise 的 SqliteClient 会把
# credentials 中 file_path 以外的键逐条执行成 PRAGMA,且 SQLite
# 后端只维持单条连接,因此这些设置在进程生命周期内全局生效。
# synchronous=NORMAL:WAL 下应用崩溃不丢已提交事务,
# 仅 OS/断电可能丢最后若干事务,省去每次提交的 fsync。
# temp_store=MEMORY:临时表与排序走内存,小库收益明显;
# 后台大排序的内存峰值需在压测中关注。
# cache_size=-8000:page cache 上限约 8MB(负数单位为 KiB)。
"synchronous": "NORMAL",
"temp_store": "MEMORY",
"cache_size": -8000,
},
}
},
@@ -47,7 +58,7 @@ def _lock_file(file_obj: IO[str]) -> None:
if os.fstat(file_obj.fileno()).st_size == 0:
file_obj.write("0")
file_obj.flush()
msvcrt.locking(file_obj.fileno(), msvcrt.LK_LOCK, 1)
msvcrt.locking(file_obj.fileno(), msvcrt.LK_LOCK, 1) # type: ignore[attr-defined]
else:
import fcntl
@@ -58,7 +69,7 @@ def _unlock_file(file_obj: IO[str]) -> None:
if os.name == "nt":
import msvcrt
msvcrt.locking(file_obj.fileno(), msvcrt.LK_UNLCK, 1)
msvcrt.locking(file_obj.fileno(), msvcrt.LK_UNLCK, 1) # type: ignore[attr-defined]
else:
import fcntl
+13
View File
@@ -0,0 +1,13 @@
"""Framework-neutral errors raised by storage backends.
main.py registers an exception handler that renders these exactly like
FastAPI's HTTPException responses ({"detail": ...} with the same status
code), so callers need no per-site wrapping while core/ stays framework-free.
"""
class StorageError(Exception):
def __init__(self, status_code: int, detail: str):
self.status_code = status_code
self.detail = detail
super().__init__(detail)
+46 -12
View File
@@ -1,26 +1,60 @@
import logging
import os
import sys
PRODUCTION_ENVIRONMENTS = {"production", "prod"}
LOG_LEVELS = {
"debug": logging.DEBUG,
"info": logging.INFO,
"warning": logging.WARNING,
"error": logging.ERROR,
"critical": logging.CRITICAL,
}
def is_production_environment() -> bool:
environment = os.getenv("APP_ENV", "development").strip().lower()
return environment in PRODUCTION_ENVIRONMENTS
def get_log_level_name() -> str:
configured_level = os.getenv("LOG_LEVEL", "").strip().lower()
if configured_level in LOG_LEVELS:
return configured_level
return "warning" if is_production_environment() else "info"
def get_log_level() -> int:
return LOG_LEVELS[get_log_level_name()]
def is_access_log_enabled() -> bool:
configured_value = os.getenv("ACCESS_LOG")
if configured_value is not None:
return configured_value.strip().lower() in {"1", "true", "yes", "on"}
return not is_production_environment()
def setup_logger():
# 创建logger对象
_logger = logging.getLogger('FileCodeBox')
_logger.setLevel(logging.INFO)
_logger = logging.getLogger("FileCodeBox")
level = get_log_level()
_logger.setLevel(level)
_logger.propagate = False
# 创建控制台处理器
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(logging.INFO)
if not _logger.handlers:
console_handler = logging.StreamHandler(sys.stdout)
_logger.addHandler(console_handler)
else:
console_handler = _logger.handlers[0]
console_handler.setLevel(level)
# 设置日志格式
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
"%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
console_handler.setFormatter(formatter)
# 添加处理器到logger
_logger.addHandler(console_handler)
return _logger
-9
View File
@@ -1,7 +1,3 @@
# @Time : 2023/8/14 11:48
# @Author : Lan
# @File : response.py
# @Software: PyCharm
from typing import Generic, Optional, TypeVar
from pydantic import BaseModel
@@ -12,9 +8,4 @@ T = TypeVar("T")
class APIResponse(BaseModel, Generic[T]):
code: int = 200
message: str = "ok"
msg: Optional[str] = None
detail: Optional[T] = None
def model_post_init(self, __context) -> None:
if self.msg is None:
self.msg = self.message
+112
View File
@@ -1,7 +1,11 @@
import copy
import ipaddress
import os
import secrets
import socket
from dataclasses import dataclass
from typing import Any
from urllib.parse import urlsplit
from core.utils import hash_password, is_password_hashed, verify_password
@@ -29,9 +33,18 @@ def is_valid_jwt_secret(secret: Any) -> bool:
def is_config_initialized(config: dict[str, Any]) -> bool:
"""Cheap initialization probe — runs on EVERY request via the middleware.
Must never run a slow hash: a scrypt-stored token cannot be the legacy
default (which only ever existed as plaintext or sha256), so those tokens
are initialized without any verification. Only plaintext/sha256 tokens go
through the legacy-default comparison, which is microseconds.
"""
admin_token = str(config.get("admin_token") or "")
if not admin_token:
return False
if is_password_hashed(admin_token) and admin_token.startswith("scrypt$"):
return True
return not verify_password(LEGACY_DEFAULT_ADMIN_TOKEN, admin_token)
@@ -58,3 +71,102 @@ def prepare_security_config(config: dict[str, Any]) -> SecurityConfigResult:
result.changed = True
return result
# 出站端点白名单/黑名单(SSRF 防护)。
# 这些配置项会被服务器直接当作上游地址访问:被劫持的管理员会话可以把实例
# 变成内网代理跳板,或把数据发往任意地址。校验只在写入入口(update_config),
# 已存值不受影响;APP_ENV 非 production(本地开发,如 minio/webdav)时放行。
OUTBOUND_ENDPOINT_CONFIG_KEYS = ("s3_endpoint_url", "s3_hostname", "webdav_url")
_ENDPOINT_SCHEME_WHITELIST = {"http", "https"}
_ENDPOINT_HOST_DENY_SUFFIXES = (
".local",
".internal",
".lan",
".home.arpa",
".corp",
".localhost",
)
def _endpoint_ip_is_denied(ip_str: str) -> bool:
try:
ip = ipaddress.ip_address(ip_str)
except ValueError:
return False
return (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_reserved
or ip.is_multicast
or ip.is_unspecified
)
def _endpoint_host_is_denied(host: str) -> bool:
host = host.strip("[]").lower()
if host in {"localhost"} or host.endswith(_ENDPOINT_HOST_DENY_SUFFIXES):
return True
# 字面 IP 直接判;非字面形态(速记 "127.1"、十进制 2130706433、十六进制、
# *.localhost、nip.io 类 DNS 映射)交给解析后逐 IP 复判——静态字符串
# 黑名单对它们全部失效,这是探测中实测确认的绕过面。
if _endpoint_ip_is_denied(host):
return True
try:
infos = socket.getaddrinfo(host, None)
except (socket.gaierror, UnicodeError, OSError):
return False
for info in infos:
# getaddrinfo 的 sockaddr 首元素在 IPv4/IPv6 下均为 str(int 仅出现在
# AF_UNIX 等异形族)——显式 str() 让类型检查器与运行时一致。
if _endpoint_ip_is_denied(str(info[4][0])):
return True
return False
def validate_outbound_endpoint(value: Any) -> str:
"""校验完整 URL 形态的出站端点(s3_endpoint_url / webdav_url)。"""
endpoint = str(value or "").strip()
if not endpoint:
return ""
if os.environ.get("APP_ENV", "development") != "production":
return endpoint
parts = urlsplit(endpoint)
if parts.scheme.lower() not in _ENDPOINT_SCHEME_WHITELIST:
raise ValueError(f"端点协议必须是 http(s):{endpoint}")
host = parts.hostname or ""
if not host:
raise ValueError(f"端点缺少主机名:{endpoint}")
if _endpoint_host_is_denied(host):
raise ValueError(f"端点不允许指向内网/保留地址:{endpoint}")
return endpoint
def validate_outbound_hostname(value: Any) -> str:
"""校验裸主机名形态的出站端点(s3_hostname,存储层拼接 https://{hostname})。"""
hostname = str(value or "").strip()
if not hostname:
return ""
if os.environ.get("APP_ENV", "development") != "production":
return hostname
if "://" in hostname or "/" in hostname or "@" in hostname:
raise ValueError(f"s3_hostname 应为裸主机名,不含协议、路径或 userinfo:{hostname}")
# 取主机部分:[v6] 形态取括号内;host:port 取冒号前;裸 IPv6(多个冒号)整体
if "[" in hostname:
host = hostname.split("[", 1)[1].split("]", 1)[0]
rest = hostname.split("]", 1)[1] if "]" in hostname else ""
if rest and not rest.startswith(":"):
raise ValueError(f"s3_hostname 的 IPv6 括号后只允许端口:{hostname}")
elif hostname.count(":") == 1:
host = hostname.split(":")[0]
else:
host = hostname
if not host or _endpoint_host_is_denied(host):
raise ValueError(f"端点不允许指向内网/保留地址:{hostname}")
return hostname
+33 -19
View File
@@ -1,3 +1,4 @@
from typing import Any
# @Time : 2023/8/15 09:51
# @Author : Lan
# @File : settings.py
@@ -14,10 +15,10 @@ data_root = BASE_DIR / "data"
if not data_root.exists():
data_root.mkdir(parents=True, exist_ok=True)
DEFAULT_CONFIG = {
DEFAULT_CONFIG: dict[str, Any] = {
"file_storage": "local",
"storage_path": "",
"storageLimit": 0,
"storage_limit": 0,
"name": "文件快递柜 - FileCodeBox",
"description": "开箱即用的文件快传系统",
"notify_title": "系统通知",
@@ -45,21 +46,22 @@ DEFAULT_CONFIG = {
"webdav_proxy": 0,
"admin_token": "",
"jwt_secret": "",
"adminSessionExpire": ADMIN_SESSION_EXPIRE_DEFAULT,
"openUpload": 1,
"uploadSize": 1024 * 1024 * 10,
"admin_session_expire": ADMIN_SESSION_EXPIRE_DEFAULT,
"open_upload": 1,
"mcp_require_token": 0,
"upload_size": 1024 * 1024 * 10,
"allowed_file_types": ["*"],
"expireStyle": ["day", "hour", "minute", "forever", "count"],
"expire_style": ["day", "hour", "minute", "forever", "count"],
"code_generate_type": "secret",
"uploadMinute": 1,
"enableChunk": 0,
"upload_minute": 1,
"enable_chunk": 0,
"webdav_url": "",
"webdav_password": "",
"webdav_username": "",
"opacity": 0.9,
"background": "",
"uploadCount": 10,
"themesChoices": [
"upload_count": 10,
"themes_choices": [
{
"name": "2023",
"key": "themes/2023",
@@ -73,15 +75,19 @@ DEFAULT_CONFIG = {
"version": "1.0",
},
],
"themesSelect": "themes/2024",
"errorMinute": 1,
"errorCount": 10,
"serverWorkers": 1,
"serverHost": "0.0.0.0",
"serverPort": 12345,
"showAdminAddr": 0,
"robotsText": "User-agent: *\nDisallow: /",
"trustedProxies": [],
"themes_select": "themes/2024",
"error_minute": 1,
"error_count": 10,
"login_count": 5,
"login_minute": 15,
"server_workers": 1,
"server_host": "0.0.0.0",
"server_port": 12345,
"show_admin_addr": 0,
"robots_text": "User-agent: *\nDisallow: /",
"trusted_proxies": [],
"chunk_expire_hours": 24,
"opendal_scheme": "s3",
}
@@ -105,6 +111,14 @@ class Settings:
else:
self.user_config[key] = value
def unknown_keys(self, config: dict) -> list[str]:
"""Keys in `config` that DEFAULT_CONFIG does not define.
Unknown keys silently fall through __getattr__ and crash later at the
usage site; callers (refresh_settings) surface them at load time.
"""
return sorted(k for k in config if k not in self.default_config)
def items(self):
return {**self.default_config, **self.user_config}.items()
-1364
View File
File diff suppressed because it is too large. Load diff
+25
View File
@@ -0,0 +1,25 @@
"""Storage backend package (split from the former 1400+ line storage.py).
Public surface is re-exported here — `from core.storage import X` keeps
working for every historical name.
"""
from core.storage._base import ( # noqa: F401
FileStorageInterface,
S3_MIN_MULTIPART_PART_SIZE,
StoredDownload,
StoredFile,
build_attachment_headers,
)
from core.storage.local import SystemFileStorage # noqa: F401
from core.storage.s3 import S3FileStorage # noqa: F401
from core.storage.onedrive import OneDriveFileStorage # noqa: F401
from core.storage.opendal import OpenDALFileStorage # noqa: F401
from core.storage.webdav import WebDAVFileStorage # noqa: F401
storages = {
"local": SystemFileStorage,
"s3": S3FileStorage,
"onedrive": OneDriveFileStorage,
"opendal": OpenDALFileStorage,
"webdav": WebDAVFileStorage,
}
+189
View File
@@ -0,0 +1,189 @@
# @Time : 2023/8/11 20:06
# @Author : Lan
# @File : storage.py
# @Software: PyCharm
import hashlib
import re
from typing import BinaryIO, Optional
from urllib.parse import quote
from collections.abc import Callable
from typing import Any
from dataclasses import dataclass
from pathlib import Path
@dataclass
class StoredDownload:
"""Framework-free description of a file download.
Backends return this; the view layer builds the starlette Response:
- ``path`` set -> FileResponse (local files, Range support for free)
- ``content`` set -> small full-read Response (legacy OpenDAL fallback)
- ``stream_factory`` set -> StreamingResponse(stream_factory(), ...)
``background`` is an optional response-sent cleanup hook.
"""
filename: str
headers: dict
media_type: str = "application/octet-stream"
path: Path | None = None
content: bytes | None = None
stream_factory: Callable[[], Any] | None = None
background: Any = None # starlette BackgroundTask(core 层不 import starlette)
@dataclass
class StoredFile:
"""Plain, framework- and ORM-free description of a stored file.
Storage backends accept this instead of ORM models so core/ never imports
apps/. Callers (views/tasks/services) build it from their own records.
"""
file_path: str
uuid_file_name: str
code: str = ""
prefix: str = ""
suffix: str = ""
text: str = ""
def get_file_path(self) -> str:
return f"{self.file_path}/{self.uuid_file_name}"
# S3 multipart 除最后一片外每部分最小 5MB(服务端强制,小于即 EntityTooSmall)
S3_MIN_MULTIPART_PART_SIZE = 5 * 1024 * 1024
def build_attachment_headers(filename: str, content_length=None) -> dict:
"""所有存储后端统一的下载响应头。
Content-Disposition: attachment 是防御存储型 XSS 的关键——同源下载路径
(/share/download)因此永不内联渲染 HTML/SVG。此函数是唯一构造点,
新增后端必须复用(tests/test_attachment_guard.py 有源码级 tripwire)。
"""
# Older Android download managers ignore filename*. Keep an ASCII fallback
# with the extension, without allowing quoted-string/header injection.
fallback = re.sub(r"[^A-Za-z0-9._ -]", "_", filename).strip(" .") or "download"
encoded_filename = quote(filename, safe="")
headers = {
"Content-Disposition": (
f'attachment; filename="{fallback}"; filename*=UTF-8\'\'{encoded_filename}'
)
}
if content_length is not None:
headers["Content-Length"] = str(content_length)
return headers
class FileStorageInterface:
@staticmethod
def _get_chunk_record(chunk_records: dict, index: int):
"""Look up the caller-provided record for chunk `index`.
Records are plain objects exposing ``chunk_hash``; fetching them from
the DB is the caller's job (keeps storage ORM-free).
"""
chunk_record = chunk_records.get(index)
if not chunk_record:
raise ValueError(f"分片{index}记录不存在")
return chunk_record
def _verify_and_hash_chunk(
self,
index: int,
chunk_record,
chunk_data: bytes,
file_sha256,
) -> None:
"""Verify a chunk against its recorded hash, then stripe it into the
whole-file digest. Raises the shared ValueError wording on mismatch."""
current_hash = hashlib.sha256(chunk_data).hexdigest()
if current_hash != chunk_record.chunk_hash:
raise ValueError(
f"分片{index}哈希不匹配: 期望 {chunk_record.chunk_hash}, 实际 {current_hash}"
)
file_sha256.update(chunk_data)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""Save a binary stream (caller owns closing the stream)."""
raise NotImplementedError
async def delete_file(self, file_code: StoredFile):
"""
删除文件
"""
raise NotImplementedError
async def get_file_url(self, file_code: StoredFile):
"""
获取文件分享的url
如果服务不支持直接访问文件,可以通过服务器中转下载。
此时,此方法可以调用 utils.py 中的 `get_file_url` 方法,获取服务器中转下载的url
"""
raise NotImplementedError
async def get_file_response(self, file_code: StoredFile):
"""
获取文件响应
如果服务不支持直接访问文件,则需要实现该方法,返回文件响应
其余情况,可以不实现该方法
"""
raise NotImplementedError
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片文件
:param upload_id: 上传会话ID
:param chunk_index: 分片索引
:param chunk_data: 分片数据
:param chunk_hash: 分片哈希值
:param save_path: 文件保存路径
"""
raise NotImplementedError
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并分片文件并返回文件路径和完整哈希值
:param upload_id: 上传会话ID
:param chunk_info: 分片信息
:param save_path: 文件保存路径
:return: (文件路径, 文件哈希值)
"""
raise NotImplementedError
async def generate_presigned_upload_url(self, save_path: str, expires_in: int = 900) -> Optional[str]:
"""
生成预签名上传URL
:param save_path: 文件保存路径
:param expires_in: URL过期时间(秒),默认15分钟
:return: 预签名URL,如果不支持直传则返回None
"""
return None # 默认不支持直传,使用代理模式
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在
:param save_path: 文件路径
:return: 文件是否存在
"""
raise NotImplementedError
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
raise NotImplementedError
+184
View File
@@ -0,0 +1,184 @@
import hashlib
from core.logger import logger
import shutil
from typing import BinaryIO, Optional
import aiofiles
import asyncio
from pathlib import Path
from core.errors import StorageError
from core.settings import data_root
from core.utils import get_file_url, sanitize_filename
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class SystemFileStorage(FileStorageInterface):
def __init__(self):
self.chunk_size = 256 * 1024
self.root_path = data_root
def _resolve_safe_path(self, relative_path: str) -> Path:
"""将相对路径解析到数据根目录内,阻止路径穿越。"""
root = self.root_path.resolve()
raw = str(relative_path or "").replace("\\", "/").lstrip("/")
if any(part == ".." for part in raw.split("/")):
raise ValueError("非法文件路径")
candidate = (root / raw).resolve()
try:
candidate.relative_to(root)
except ValueError as exc:
raise ValueError("非法文件路径") from exc
return candidate
def _save(self, file, save_path):
with open(save_path, "wb") as f:
chunk = file.read(self.chunk_size)
while chunk:
f.write(chunk)
chunk = file.read(self.chunk_size)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
path_obj = Path(str(save_path).replace("\\", "/"))
directory = str(path_obj.parent).replace("\\", "/").lstrip("/")
# 提取原始文件名并进行清理
filename = await sanitize_filename(path_obj.name)
# 构建安全的完整保存路径
safe_save_path = self._resolve_safe_path(f"{directory}/{filename}" if directory not in {"", "."} else filename)
# 确保目录存在
if not safe_save_path.parent.exists():
safe_save_path.parent.mkdir(parents=True)
await asyncio.to_thread(self._save, stream, safe_save_path)
async def delete_file(self, file_code: StoredFile):
save_path = self._resolve_safe_path(file_code.get_file_path())
if save_path.exists():
save_path.unlink()
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
file_path = self._resolve_safe_path(file_code.get_file_path())
if not file_path.exists():
raise StorageError(status_code=404, detail="文件已过期删除")
filename = f"{file_code.prefix}{file_code.suffix}"
try:
headers = build_attachment_headers(filename, file_path.stat().st_size)
except OSError:
# 文件大小不可得时省略 Content-Length
headers = build_attachment_headers(filename)
return StoredDownload(
filename=filename,
headers=headers,
path=file_path,
)
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片文件到本地文件系统
:param upload_id: 上传会话ID
:param chunk_index: 分片索引
:param chunk_data: 分片数据
:param chunk_hash: 分片哈希值
:param save_path: 文件保存路径
"""
# 先校验目标文件路径合法,再将分片落到同级 chunks 目录。
self._resolve_safe_path(save_path)
chunk_path = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
)
if not chunk_path.parent.exists():
chunk_path.parent.mkdir(parents=True, exist_ok=True)
# 使用临时文件写入,确保原子性
temp_path = chunk_path.with_suffix('.tmp')
try:
async with aiofiles.open(temp_path, "wb") as f:
await f.write(chunk_data)
# 原子重命名
temp_path.rename(chunk_path)
except Exception as e:
if temp_path.exists():
temp_path.unlink()
raise e
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并本地文件系统的分片文件并返回文件路径和完整哈希值
:param upload_id: 上传会话ID
:param chunk_info: 分片信息
:param save_path: 文件保存路径
:return: (文件路径, 文件哈希值)
"""
output_path = self._resolve_safe_path(save_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
chunk_base_dir = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id)
)
file_sha256 = hashlib.sha256()
# 使用临时文件写入,确保原子性
temp_output = output_path.with_suffix('.merging')
try:
async with aiofiles.open(temp_output, "wb") as out_file:
for i in range(total_chunks):
# 获取分片记录
chunk_record = self._get_chunk_record(chunk_records, i)
chunk_path = chunk_base_dir / f"{i}.part"
if not chunk_path.exists():
raise ValueError(f"分片{i}文件不存在")
async with aiofiles.open(chunk_path, "rb") as in_file:
chunk_data = await in_file.read()
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
# 原子重命名
temp_output.rename(output_path)
except Exception as e:
if temp_output.exists():
temp_output.unlink()
raise e
return str(output_path), file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理本地文件系统的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id)
)
if chunk_dir.exists():
try:
shutil.rmtree(chunk_dir)
except Exception as e:
logger.warning(f"清理本地分片目录失败: {e}")
# 清理父级 chunks 目录(如果为空)
chunks_parent = chunk_dir.parent
if chunks_parent.exists() and not any(chunks_parent.iterdir()):
try:
chunks_parent.rmdir()
except Exception as e:
logger.warning(f"清理 chunks 父目录失败: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于本地文件系统
:param save_path: 文件路径
:return: 文件是否存在
"""
try:
file_path = self._resolve_safe_path(save_path)
except ValueError:
return False
return file_path.exists()
+339
View File
@@ -0,0 +1,339 @@
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
import aiofiles
import aiohttp
import asyncio
from pathlib import Path
import datetime
import re
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url, sanitize_filename
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class OneDriveFileStorage(FileStorageInterface):
def __init__(self):
try:
import msal
from office365.graph_client import GraphClient
from office365.runtime.client_request_exception import (
ClientRequestException,
)
except ImportError:
raise ImportError("请先安装`msal`和`Office365-REST-Python-Client`")
self.msal = msal
self.domain = settings.onedrive_domain
self.client_id = settings.onedrive_client_id
self.username = settings.onedrive_username
self.password = settings.onedrive_password
self.proxy = settings.onedrive_proxy
self._ClientRequestException = ClientRequestException
try:
client = GraphClient(self.acquire_token_pwd)
self.root_path = (
client.me.drive.root.get_by_path(settings.onedrive_root_path)
.get()
.execute_query()
)
except ClientRequestException as e:
if e.code == "itemNotFound":
client.me.drive.root.create_folder(settings.onedrive_root_path)
self.root_path = (
client.me.drive.root.get_by_path(
settings.onedrive_root_path)
.get()
.execute_query()
)
else:
raise e
except Exception as e:
raise Exception("OneDrive验证失败,请检查配置是否正确\n" + str(e))
def acquire_token_pwd(self):
authority_url = f"https://login.microsoftonline.com/{self.domain}"
app = self.msal.PublicClientApplication(
authority=authority_url, client_id=self.client_id
)
result = app.acquire_token_by_username_password(
username=self.username,
password=self.password,
scopes=["https://graph.microsoft.com/.default"],
)
return result
def _get_path_str(self, path):
if isinstance(path, str):
path = path.replace("\\", "/").replace("//", "/").split("/")
elif isinstance(path, Path):
path = str(path).replace("\\", "/").replace("//", "/").split("/")
else:
raise TypeError("path must be str or Path")
path[-1] = path[-1].split(".")[0]
return "/".join(path)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""保存文件(自动创建目录;修复旧实现把 save_path 字符串当函数调用的崩溃)"""
content = await asyncio.to_thread(stream.read)
normalized = str(save_path).replace("\\", "/")
name = await sanitize_filename(Path(normalized).name)
dir_path = "/".join(normalized.split("/")[:-1])
current_folder = self.root_path
for part in dir_path.split("/"):
if not part:
continue
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
await asyncio.to_thread(
lambda: current_folder.get_by_path(name)
.upload(name, content)
.execute_query()
)
def _delete(self, save_path):
path = self._get_path_str(save_path)
try:
self.root_path.get_by_path(path).delete_object().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
pass
else:
raise e
async def delete_file(self, file_code: StoredFile):
await asyncio.to_thread(self._delete, file_code.get_file_path())
def _convert_link_to_download_link(self, link):
p1 = re.search(r"https://(.+)\.sharepoint\.com", link).group(1)
p2 = re.search(r"personal/(.+)/", link).group(1)
p3 = re.search(rf"{p2}/(.+)", link).group(1)
return f"https://{p1}.sharepoint.com/personal/{p2}/_layouts/52/download.aspx?share={p3}"
def _get_file_url(self, save_path, name):
path = self._get_path_str(save_path)
remote_file = self.root_path.get_by_path(path + "/" + name)
expiration_datetime = datetime.datetime.now(
tz=datetime.timezone.utc
) + datetime.timedelta(hours=1)
expiration_datetime = expiration_datetime.strftime(
"%Y-%m-%dT%H:%M:%SZ")
permission = remote_file.create_link(
"view", "anonymous", expiration_datetime=expiration_datetime
).execute_query()
return self._convert_link_to_download_link(permission.link.webUrl)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
try:
link = await asyncio.to_thread(
self._get_file_url, file_code.get_file_path(), filename
)
except self._ClientRequestException as e:
# 对象不存在时前置 404(与 local/S3/WebDAV 语义对齐),
# 不再落入外层兜底的 503
if str(getattr(e, "code", "")).lower() in {"itemnotfound", "404"}:
raise StorageError(
status_code=404, detail="文件已过期删除"
) from e
raise
content_length = None # 初始化为 None,表示未知大小
# 创建ClientSession并复用
session = aiohttp.ClientSession()
# 尝试发送HEAD请求获取Content-Length
try:
async with session.head(link) as resp:
if resp.status == 200 and 'Content-Length' in resp.headers:
content_length = int(resp.headers['Content-Length'])
except Exception:
# 如果HEAD请求失败,则不提供 Content-Length
pass
async def stream_generator():
try:
async with session.get(link) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"从OneDrive获取文件失败: {resp.status}"
)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except StorageError:
raise
except Exception:
raise StorageError(status_code=503, detail="服务代理下载异常,请稍后再试")
async def get_file_url(self, file_code: StoredFile):
if self.proxy:
return await get_file_url(file_code.code)
else:
return await asyncio.to_thread(
self._get_file_url,
file_code.get_file_path(),
f"{file_code.prefix}{file_code.suffix}",
)
def _save_chunk(self, chunk_path: str, chunk_data: bytes):
"""同步保存分片到 OneDrive"""
path_parts = chunk_path.replace("\\", "/").split("/")
filename = path_parts[-1]
dir_path = "/".join(path_parts[:-1])
# 确保目录存在
current_folder = self.root_path
for part in dir_path.split("/"):
if part:
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
# 上传分片
current_folder.upload(filename, chunk_data).execute_query()
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 OneDrive"""
chunk_path = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
await asyncio.to_thread(self._save_chunk, chunk_path, chunk_data)
def _read_chunk(self, chunk_path: str) -> bytes:
"""同步读取分片"""
path = self._get_path_str(chunk_path)
file_obj = self.root_path.get_by_path(path).get().execute_query()
return file_obj.get_content().execute_query().value
def _upload_merged(self, save_path: str, data: bytes):
"""同步上传合并后的文件"""
path_parts = save_path.replace("\\", "/").split("/")
filename = path_parts[-1]
dir_path = "/".join(path_parts[:-1])
# 确保目录存在
current_folder = self.root_path
for part in dir_path.split("/"):
if part:
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
current_folder.upload(filename, data).execute_query()
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""合并 OneDrive 上的分片文件,使用临时文件避免内存问题"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
chunk_data = await asyncio.to_thread(self._read_chunk, chunk_path)
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 读取临时文件并上传
async with aiofiles.open(temp_path, 'rb') as f:
merged_content = await f.read()
await asyncio.to_thread(self._upload_merged, save_path, merged_content)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
def _delete_chunk_dir(self, chunk_dir: str):
"""同步删除分片目录"""
try:
path = self._get_path_str(chunk_dir)
self.root_path.get_by_path(path).delete_object().execute_query()
except self._ClientRequestException as e:
if e.code != "itemNotFound":
raise e
async def clean_chunks(self, upload_id: str, save_path: str):
"""清理 OneDrive 上的临时分片文件"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
try:
await asyncio.to_thread(self._delete_chunk_dir, chunk_dir)
except Exception as e:
logger.warning(f"清理 OneDrive 分片时出错: {e}")
def _file_exists(self, save_path: str) -> bool:
"""同步检查文件是否存在"""
try:
path = self._get_path_str(save_path)
self.root_path.get_by_path(path).get().execute_query()
return True
except self._ClientRequestException as e:
if e.code == "itemNotFound":
return False
raise e
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于OneDrive
:param save_path: 文件路径
:return: 文件是否存在
"""
return await asyncio.to_thread(self._file_exists, save_path)
+160
View File
@@ -0,0 +1,160 @@
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
import aiofiles
import asyncio
from pathlib import Path
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class OpenDALFileStorage(FileStorageInterface):
def __init__(self):
try:
import opendal
except ImportError:
raise ImportError('请先安装 `opendal`, 例如: "pip install opendal"')
self.service = settings.opendal_scheme
service_settings = {}
for key, value in settings.items():
if key.startswith("opendal_" + self.service):
setting_name = key.split("_", 2)[2]
service_settings[setting_name] = value
self.operator = opendal.AsyncOperator(
settings.opendal_scheme, **service_settings
)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
# 使用 asyncio.to_thread 避免阻塞事件循环
content = await asyncio.to_thread(stream.read)
await self.operator.write(save_path, content)
async def delete_file(self, file_code: StoredFile):
await self.operator.delete(file_code.get_file_path())
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
content_length = None # 初始化为 None,表示未知大小
# 尝试获取文件大小
try:
stat_result = await self.operator.stat(file_code.get_file_path())
if hasattr(stat_result, 'content_length') and stat_result.content_length:
content_length = stat_result.content_length
elif hasattr(stat_result, 'size') and stat_result.size:
content_length = stat_result.size
except Exception:
# 如果获取大小失败,则不提供 Content-Length
pass
# 尝试使用流式读取器
try:
# OpenDAL 可能提供 reader 方法返回一个异步读取器
reader = await self.operator.reader(file_code.get_file_path())
except AttributeError:
# 如果 reader 方法不存在,回退到全量读取(兼容旧版本)
content = await self.operator.read(file_code.get_file_path())
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
content=content,
)
async def stream_generator():
chunk_size = 65536
while True:
chunk = await reader.read(chunk_size)
if not chunk:
break
yield chunk
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
)
except Exception as e:
logger.info(e)
raise StorageError(status_code=404, detail="文件已过期删除")
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 OpenDAL 存储"""
chunk_path = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
await self.operator.write(chunk_path, chunk_data)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""合并 OpenDAL 存储上的分片文件,使用临时文件避免内存问题"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
chunk_data = await self.operator.read(chunk_path)
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 读取临时文件并写入存储
async with aiofiles.open(temp_path, 'rb') as f:
merged_content = await f.read()
await self.operator.write(save_path, merged_content)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""清理 OpenDAL 存储上的临时分片文件"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
try:
# OpenDAL 支持递归删除
await self.operator.remove_all(chunk_dir)
except Exception as e:
logger.warning(f"清理 OpenDAL 分片时出错: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于OpenDAL存储
:param save_path: 文件路径
:return: 文件是否存在
"""
try:
await self.operator.stat(save_path)
return True
except Exception:
return False
+344
View File
@@ -0,0 +1,344 @@
from botocore.exceptions import ClientError
import hashlib
from core.logger import logger
from typing import BinaryIO, Optional
import aiohttp
import asyncio
from pathlib import Path
import aioboto3
from botocore.config import Config
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
S3_MIN_MULTIPART_PART_SIZE,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class S3FileStorage(FileStorageInterface):
def __init__(self):
self.access_key_id = settings.s3_access_key_id
self.secret_access_key = settings.s3_secret_access_key
self.bucket_name = settings.s3_bucket_name
self.s3_hostname = settings.s3_hostname
self.region_name = settings.s3_region_name
self.signature_version = settings.s3_signature_version
self.endpoint_url = settings.s3_endpoint_url or f"https://{self.s3_hostname}"
self.aws_session_token = settings.aws_session_token
self.addressing_style = str(settings.s3_addressing_style or "auto").lower()
self.proxy = settings.s3_proxy
self.session = aioboto3.Session(
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
)
if not settings.s3_endpoint_url:
self.endpoint_url = f"https://{self.s3_hostname}"
else:
# 如果提供了 s3_endpoint_url,则优先使用它
self.endpoint_url = settings.s3_endpoint_url
def _client_config(self) -> Config:
# OSS 等兼容服务不支持默认 flexible checksum 的 aws-chunked trailer。
config = {
"signature_version": self.signature_version,
"request_checksum_calculation": "when_required",
"response_checksum_validation": "when_required",
}
s3_config = {}
if self.addressing_style in {"path", "virtual", "auto"}:
s3_config["addressing_style"] = self.addressing_style
if s3_config:
config["s3"] = s3_config
return Config(**config)
def _client(self):
return self.session.client(
"s3",
endpoint_url=self.endpoint_url,
aws_session_token=self.aws_session_token,
region_name=self.region_name,
config=self._client_config(),
)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
async with self._client() as s3:
# 使用 upload_fileobj 流式上传,避免将整个文件加载到内存
await s3.upload_fileobj(
stream,
self.bucket_name,
save_path,
ExtraArgs={"ContentType": content_type or "application/octet-stream"},
)
async def delete_file(self, file_code: StoredFile):
async with self._client() as s3:
await s3.delete_object(
Bucket=self.bucket_name, Key=file_code.get_file_path()
)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
content_length = None # 初始化为 None,表示未知大小
async with self._client() as s3:
# 尝试获取文件大小(HEAD请求);对象不存在时前置 404——
# 与 local 后端语义一致(M2 行为统一),不能签出 200 的坏流
try:
head_response = await s3.head_object(
Bucket=self.bucket_name,
Key=file_code.get_file_path()
)
# 从HEAD响应中获取Content-Length
if 'ContentLength' in head_response:
content_length = head_response['ContentLength']
elif 'Content-Length' in head_response['ResponseMetadata']['HTTPHeaders']:
content_length = int(head_response['ResponseMetadata']['HTTPHeaders']['Content-Length'])
except ClientError as e:
error_code = e.response.get("Error", {}).get("Code", "")
if error_code in {"404", "NoSuchKey", "NotFound"}:
raise StorageError(
status_code=404, detail="文件已过期删除"
) from e
# 其他 HEAD 错误不阻断:流式下载阶段会给出真实状态
except Exception:
# 如果HEAD请求失败,则不提供 Content-Length
pass
link = await s3.generate_presigned_url(
"get_object",
Params={
"Bucket": self.bucket_name,
"Key": file_code.get_file_path(),
},
ExpiresIn=3600,
)
# 创建ClientSession并传递给生成器复用
session = aiohttp.ClientSession()
async def stream_generator():
try:
async with session.get(link) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"从S3获取文件失败: {resp.status}"
)
# 设置块大小(例如64KB)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except StorageError:
raise
except Exception:
raise StorageError(status_code=503, detail="服务代理下载异常,请稍后再试")
async def get_file_url(self, file_code: StoredFile):
if file_code.prefix == "文本分享":
return file_code.text
if self.proxy:
return await get_file_url(file_code.code)
else:
async with self._client() as s3:
result = await s3.generate_presigned_url(
"get_object",
Params={
"Bucket": self.bucket_name,
"Key": file_code.get_file_path(),
},
ExpiresIn=3600,
)
return result
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片到 S3(使用独立对象存储每个分片)
注意:这里不使用 S3 原生的 multipart upload,而是将每个分片作为独立对象存储
"""
chunk_key = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
async with self._client() as s3:
# 将分片作为独立对象上传
await s3.put_object(
Bucket=self.bucket_name,
Key=chunk_key,
Body=chunk_data,
Metadata={
'chunk-hash': chunk_hash,
'chunk-index': str(chunk_index)
}
)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并 S3 上的分片文件
使用 S3 的 multipart upload API 实现流式合并,避免内存问题
"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
async with self._client() as s3:
# 创建 multipart upload
mpu = await s3.create_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
ContentType='application/octet-stream'
)
mpu_id = mpu['UploadId']
parts = []
try:
# 按顺序读取、验证每个分片;S3 multipart 规范要求除最后一片外
# 每个部分 ≥5MB(EntityTooSmall),而分片大小由客户端决定(常见
# 2-4MB)——因此缓冲到 S3_MIN_MULTIPART_PART_SIZE 再上传 part,
# 内存上界 = 5MB + 单个分片,不破坏流式合并的初衷。
part_buffer = bytearray()
part_number = 0
async def _flush_part():
nonlocal part_number
if not part_buffer:
return
part_number += 1
part_response = await s3.upload_part(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id,
PartNumber=part_number,
Body=bytes(part_buffer),
)
parts.append({
'PartNumber': part_number,
'ETag': part_response['ETag']
})
part_buffer.clear()
for i in range(total_chunks):
chunk_key = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
response = await s3.get_object(
Bucket=self.bucket_name,
Key=chunk_key
)
chunk_data = await response['Body'].read()
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
part_buffer.extend(chunk_data)
if len(part_buffer) >= S3_MIN_MULTIPART_PART_SIZE:
await _flush_part()
# 收尾:剩余缓冲作为最后一个 part(S3 允许最后一片小于 5MB;
# 恰好整除时缓冲为空,跳过)
await _flush_part()
# 完成 multipart upload
await s3.complete_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id,
MultipartUpload={'Parts': parts}
)
except Exception as e:
# 出错时取消 multipart upload
await s3.abort_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id
)
raise e
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理 S3 上的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
# 尾部斜杠防止误删 upload_id 相同前缀的其他会话。
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id) + "/"
async with self._client() as s3:
paginator = s3.get_paginator("list_objects_v2")
async for page in paginator.paginate(Bucket=self.bucket_name, Prefix=chunk_dir):
for obj in page.get("Contents", []):
# OSS 批量删除要求 Content-MD5;单对象删除不依赖该扩展。
# 失败向调用方传播,保留会话供重试,不能当作清理成功。
await s3.delete_object(Bucket=self.bucket_name, Key=obj["Key"])
async def generate_presigned_upload_url(self, save_path: str, expires_in: int = 900) -> Optional[str]:
"""
生成S3预签名上传URL
:param save_path: 文件保存路径
:param expires_in: URL过期时间(秒),默认15分钟
:return: 预签名PUT URL
"""
async with self._client() as s3:
return await s3.generate_presigned_url(
"put_object",
Params={
"Bucket": self.bucket_name,
"Key": save_path,
},
ExpiresIn=expires_in,
)
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于S3
:param save_path: 文件路径
:return: 文件是否存在
"""
async with self._client() as s3:
last_error = None
for attempt in range(3):
try:
await s3.head_object(Bucket=self.bucket_name, Key=save_path)
return True
except Exception as e:
last_error = e
if attempt < 2:
await asyncio.sleep(0.2 * (attempt + 1))
try:
result = await s3.list_objects_v2(
Bucket=self.bucket_name,
Prefix=save_path,
MaxKeys=1,
)
for item in result.get("Contents", []):
if item.get("Key") == save_path:
return True
except Exception as e:
last_error = e
logger.warning(f"S3文件确认失败 key={save_path}: {last_error}")
return False
+354
View File
@@ -0,0 +1,354 @@
import base64
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
from urllib.parse import quote, unquote
import aiofiles
import aiohttp
import asyncio
from pathlib import Path
import re
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url, sanitize_filename
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class WebDAVFileStorage(FileStorageInterface):
def __init__(self):
if not hasattr(self, "_initialized"):
self.base_url = settings.webdav_url.rstrip("/") + "/"
# aiohttp 4.0 移除 BasicAuth(auth=...) 参数——改用编码后的 Authorization 头
self.auth_headers = {
"Authorization": aiohttp.encode_basic_auth(
settings.webdav_username, settings.webdav_password
)
}
self._initialized = True
def _build_url(self, path: str) -> str:
encoded_path = quote(str(path.replace("\\", "/").lstrip("/")).lstrip("/"))
return f"{self.base_url}{encoded_path}"
async def _mkdir_p(self, directory_path: str):
"""递归创建目录(类似mkdir -p)"""
path_obj = Path(unquote(directory_path))
current_path = ""
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 逐级检查目录是否存在
for part in path_obj.parts:
current_path = str(Path(current_path) / part)
url = self._build_url(current_path)
# 检查目录是否存在
async with session.head(url) as resp:
if resp.status == 404:
# 创建目录
async with session.request("MKCOL", url) as mkcol_resp:
if mkcol_resp.status not in (200, 201, 409):
content = await mkcol_resp.text()
raise StorageError(
status_code=mkcol_resp.status,
detail=f"目录创建失败: {content[:200]}",
)
async def _is_dir_empty(self, dir_path: str) -> bool:
"""检查目录是否为空"""
url = self._build_url(dir_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.request("PROPFIND", url, headers={"Depth": "1"}) as resp:
if resp.status != 207: # 207 是 Multi-Status 响应
return False
content = await resp.text()
# 如果只有一个 response(当前目录),说明目录为空
return content.count("<D:response>") <= 1
async def _delete_empty_dirs(self, file_path: str, session: aiohttp.ClientSession):
"""递归删除空目录"""
path_obj = Path(file_path)
current_path = path_obj.parent
while str(current_path) != ".":
if not await self._is_dir_empty(str(current_path)):
break
url = self._build_url(str(current_path))
async with session.delete(url) as resp:
if resp.status not in (200, 204, 404):
break
current_path = current_path.parent
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""保存文件(自动创建目录,流式上传)"""
path_obj = Path(save_path)
directory_path = str(path_obj.parent)
# 提取原始文件名并进行清理
filename = await sanitize_filename(path_obj.name)
# 构建安全的保存路径
safe_save_path = str(Path(directory_path) / filename)
try:
# 先创建目录结构
await self._mkdir_p(directory_path)
# 上传文件(流式)
url = self._build_url(safe_save_path)
async def file_sender():
"""流式读取文件内容"""
chunk_size = 256 * 1024 # 256KB chunks
while True:
chunk = await asyncio.to_thread(stream.read, chunk_size)
if not chunk:
break
yield chunk
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.put(
url,
data=file_sender(),
headers={"Content-Type": content_type or "application/octet-stream"}
) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"文件上传失败: {content[:200]}",
)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def delete_file(self, file_code: StoredFile):
"""删除WebDAV文件及空目录"""
file_path = file_code.get_file_path()
url = self._build_url(file_path)
try:
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 删除文件
async with session.delete(url) as resp:
if resp.status not in (200, 204, 404):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"WebDAV删除失败: {content[:200]}",
)
# 使用同一个 session 删除空目录
await self._delete_empty_dirs(file_path, session)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
"""获取文件响应(代理模式)"""
try:
filename = file_code.prefix + file_code.suffix
url = self._build_url(file_code.get_file_path())
content_length = None # 初始化为 None,表示未知大小
# 创建ClientSession并复用(包含认证头)
session = aiohttp.ClientSession(headers={
"Authorization": f"Basic {base64.b64encode(f'{settings.webdav_username}:{settings.webdav_password}'.encode()).decode()}"
})
# 尝试发送HEAD请求获取Content-Length;对象不存在时前置 404
# (与 local/S3 语义对齐),连接层错误映射 503——两者都不能
# 静默吞掉后签出 200 坏流。异常路径回收 session 防泄漏。
try:
try:
async with session.head(url) as resp:
if resp.status == 404:
raise StorageError(
status_code=404, detail="文件已过期删除"
)
if resp.status == 200 and 'Content-Length' in resp.headers:
content_length = int(resp.headers['Content-Length'])
except StorageError:
raise
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}"
) from e
except Exception:
# 其他 HEAD 异常不阻断:流式下载阶段会给出真实状态
pass
except BaseException:
await session.close()
raise
async def stream_generator():
try:
async with session.get(url) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"文件获取失败{resp.status}: {await resp.text()}",
)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 WebDAV"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
chunk_path = f"{chunk_dir}/{chunk_index}.part"
# 先创建目录结构
await self._mkdir_p(chunk_dir)
chunk_url = self._build_url(chunk_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.put(chunk_url, data=chunk_data) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"分片上传失败: {content[:200]}"
)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并 WebDAV 上的分片文件
使用临时文件避免内存问题
"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据,避免内存问题
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 按顺序读取并验证每个分片,写入临时文件
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_url = self._build_url(chunk_path)
# 获取分片记录
chunk_record = self._get_chunk_record(chunk_records, i)
# 下载分片数据
async with session.get(chunk_url) as resp:
if resp.status != 200:
raise ValueError(f"分片{i}文件不存在或无法访问")
chunk_data = await resp.read()
# 验证哈希
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 确保目标目录存在
output_dir = str(Path(save_path).parent)
await self._mkdir_p(output_dir)
# 流式上传合并后的文件
output_url = self._build_url(save_path)
async def file_sender():
async with aiofiles.open(temp_path, 'rb') as f:
while True:
chunk = await f.read(256 * 1024)
if not chunk:
break
yield chunk
async with session.put(output_url, data=file_sender()) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"合并文件上传失败: {content[:200]}"
)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理 WebDAV 上的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
chunk_dir_url = self._build_url(chunk_dir)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
try:
# 检查分片目录是否存在
async with session.request("PROPFIND", chunk_dir_url, headers={"Depth": "1"}) as resp:
if resp.status == 207: # 207 表示 Multi-Status
# 获取目录下的所有分片文件
xml_data = await resp.text()
file_paths = re.findall(
r'<D:href>(.*?)</D:href>', xml_data)
for file_path in file_paths:
if file_path.endswith(".part"):
# 删除分片文件
file_url = self._build_url(file_path)
async with session.delete(file_url) as delete_resp:
if delete_resp.status not in (200, 204, 404):
logger.warning(f"删除分片文件失败: {file_path}")
# 删除分片目录
async with session.delete(chunk_dir_url) as delete_resp:
if delete_resp.status not in (200, 204, 404):
logger.warning(f"删除分片目录失败: {chunk_dir_url}")
else:
logger.info(f"分片目录不存在: {chunk_dir_url}")
except Exception as e:
logger.warning(f"清理 WebDAV 分片时出错: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于WebDAV
:param save_path: 文件路径
:return: 文件是否存在
"""
url = self._build_url(save_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.head(url) as resp:
return resp.status == 200
+71 -7
View File
@@ -25,6 +25,27 @@ async def get_random_num():
r_s = string.ascii_uppercase + string.digits
def validate_background_url(value) -> str:
"""Validate the site background config before it reaches the theme template.
Themes inject this value into inline CSS ``url('...')`` where html escaping
cannot neutralize a single-quote breakout, so only well-formed http(s) URLs
(or an empty string) are accepted.
"""
value = str(value or "").strip()
if not value:
return ""
from urllib.parse import urlparse
parsed = urlparse(value)
if parsed.scheme not in ("http", "https") or not parsed.netloc:
raise ValueError("background 必须是 http(s) 完整 URL 或留空")
# quote()/whitespace would break out of the CSS url('') quoting context
if any(ch in value for ch in ("'", '"', "(", ")", " ", "\\", ";")):
raise ValueError("background URL 含有不允许的字符")
return value
async def get_random_string():
"""
获取随机字符串
@@ -105,25 +126,61 @@ async def max_save_times_desc(max_save_seconds: int):
return desc_zh, desc_en
# scrypt work factors (OWASP-recommended memory-hard parameters; ~50ms/verify)
SCRYPT_N = 2**14
SCRYPT_R = 8
SCRYPT_P = 1
_SCRYPT_MAXMEM = 64 * 1024 * 1024 # hashlib default cap (32MB) is too low for n=2^14,r=8
PASSWORD_SCHEME = "scrypt" # current scheme for new hashes; sha256/plaintext stay verifiable
def hash_password(password: str) -> str:
"""
使用 SHA256 + salt 哈希密码
返回格式: sha256$<salt>$<hash>
使用 scrypt(memory-hard)哈希密码
返回格式: scrypt$<n>$<r>$<p>$<salt>$<hash>
"""
salt = os.urandom(16).hex()
password_hash = hashlib.sha256(f"{salt}{password}".encode()).hexdigest()
return f"sha256${salt}${password_hash}"
password_hash = hashlib.scrypt(
password.encode(),
salt=salt.encode(),
n=SCRYPT_N,
r=SCRYPT_R,
p=SCRYPT_P,
maxmem=_SCRYPT_MAXMEM,
).hex()
return f"scrypt${SCRYPT_N}${SCRYPT_R}${SCRYPT_P}${salt}${password_hash}"
def verify_password(password: str, hashed: str) -> bool:
"""
验证密码是否匹配
支持新格式 (sha256$salt$hash) 和旧格式 (明文)
支持格式: scrypt$n$r$p$salt$hash(现行)、sha256$salt$hash(历史)、明文(最老)
"""
if not hashed:
return False
# 新格式: sha256$salt$hash
if hashed.startswith("scrypt$"):
parts = hashed.split("$")
if len(parts) != 6:
return False
try:
scheme, n, r, p, salt, stored_hash = parts
password_hash = hashlib.scrypt(
password.encode(),
salt=salt.encode(),
n=int(n),
r=int(r),
p=int(p),
maxmem=_SCRYPT_MAXMEM,
).hex()
# 默认解释器对超范围的 n/r/p 抛 TypeError/ValueError;部分构建会抛
# OverflowError,一并视为校验失败,避免坏掉的存量哈希影响登录路径。
except (ValueError, TypeError, OverflowError):
return False
return hmac.compare_digest(password_hash, stored_hash)
# sha256 格式: sha256$salt$hash(历史数据,验证逻辑保持原样)
if hashed.startswith("sha256$"):
parts = hashed.split("$")
if len(parts) != 3:
@@ -138,11 +195,18 @@ def verify_password(password: str, hashed: str) -> bool:
def is_password_hashed(password: str) -> bool:
"""
检查密码是否已经是哈希格式
检查密码是否已经是哈希格式(现行或历史哈希均视为已哈希)
"""
if password.startswith("scrypt$") and len(password.split("$")) == 6:
return True
return password.startswith("sha256$") and len(password.split("$")) == 3
def password_needs_rehash(hashed: str) -> bool:
"""True when the stored hash uses a legacy scheme (sha256/plaintext)."""
return bool(hashed) and not hashed.startswith(f"{PASSWORD_SCHEME}$")
async def sanitize_filename(filename: str) -> str:
"""
安全处理文件名:
+13 -1
View File
@@ -1,12 +1,24 @@
version: "3"
services:
file-code-box:
image: lanol/filecodebox:2.5.2 # x-release-please-version
image: lanol/filecodebox:2.8.5 # x-release-please-version
volumes:
- fcb-data:/app/data:rw
# NAS 已有文件:把共享目录挂到 /app/data/local,即可在后台「本地分享」里生成提取码,无需再上传
# - /volume1/share:/app/data/local:ro
restart: unless-stopped
ports:
- "12345:12345"
environment:
- APP_ENV=production
- LOG_LEVEL=warning
- ACCESS_LOG=false
- WORKERS=1
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
volumes:
fcb-data:
external: false
+5
View File
@@ -31,6 +31,7 @@ export default defineConfig({
{ text: '上传与分享', link: '/guide/upload' },
{ text: '管理面板', link: '/guide/management' },
{ text: '存储配置', link: '/guide/storage' },
{ text: 'MCP 接入', link: '/guide/mcp' },
],
},
{ text: 'API', link: '/api/' },
@@ -64,6 +65,7 @@ export default defineConfig({
{ text: '文件上传', link: '/guide/upload' },
{ text: '文件分享', link: '/guide/share' },
{ text: '文件管理', link: '/guide/management' },
{ text: 'MCP 接入', link: '/guide/mcp' },
],
},
{
@@ -148,6 +150,7 @@ export default defineConfig({
{ text: 'Getting Started', link: '/en/guide/getting-started' },
{ text: 'Upload & Share', link: '/en/guide/upload' },
{ text: 'Admin Panel', link: '/en/guide/management' },
{ text: 'MCP Integration', link: '/en/guide/mcp' },
{ text: 'Storage', link: '/en/guide/storage' },
],
},
@@ -185,6 +188,7 @@ export default defineConfig({
{ text: 'File Upload', link: '/en/guide/upload' },
{ text: 'File Sharing', link: '/en/guide/share' },
{ text: 'File Management', link: '/en/guide/management' },
{ text: 'MCP Integration', link: '/en/guide/mcp' },
],
},
{
@@ -205,6 +209,7 @@ export default defineConfig({
text: 'API Reference',
items: [
{ text: 'API Overview', link: '/en/api/' },
{ text: 'Presigned Upload', link: '/en/api/presign-upload' },
{ text: 'Share API', link: '/en/api/#share-api' },
{ text: 'Admin API', link: '/en/api/#admin-api' },
{ text: 'Error Response', link: '/en/api/#error-response' },
+17 -15
View File
@@ -1,6 +1,6 @@
# FileCodeBox API 文档
## API 版本: 2.1.0
## API 版本: 2.5.6
## 目录
- [认证](#认证)
@@ -30,7 +30,7 @@ curl -X POST "http://localhost:12345/admin/login" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"token": "xxx.xxx.xxx",
"token_type": "Bearer"
@@ -74,7 +74,7 @@ curl -X POST "http://localhost:12345/share/text/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123"
}
@@ -125,7 +125,7 @@ curl -X POST "http://localhost:12345/share/file/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt"
@@ -157,7 +157,7 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -184,7 +184,7 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -232,15 +232,15 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"totalFiles": 100,
"storageUsed": "1.5GB",
"sysUptime": "10天",
"yesterdayCount": 50,
"yesterdaySize": "500MB",
"todayCount": 30,
"todaySize": "300MB"
"total_files": 100,
"storage_used": "1610612736",
"sys_uptime": 1725489600.0,
"yesterday_count": 50,
"yesterday_size": "524288000",
"today_count": 30,
"today_size": "314572800",
}
}
```
@@ -258,13 +258,15 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
| page | integer | 否 | 1 | 当前页码 |
| size | integer | 否 | 10 | 每页数量 |
| keyword | string | 否 | "" | 搜索关键词 |
| sort_by | string | 否 | "created_at" | 排序字段(created_at/expired_at/name/size/used_count/code) |
| sort_order | string | 否 | "desc" | 排序方向(asc/desc) |
**响应示例:**
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"page": 1,
"size": 10,
+1 -4
View File
@@ -76,7 +76,6 @@ Content-Type: application/json
"upload_id": "a1b2c3d4e5f6...",
"upload_url": "https://bucket.s3.amazonaws.com/path?X-Amz-Signature=...",
"mode": "direct",
"save_path": "share/data/2024/01/01/uuid/document.pdf",
"expires_in": 900
}
}
@@ -89,7 +88,6 @@ Content-Type: application/json
| upload_id | string | 上传会话 ID,后续操作需要 |
| upload_url | string | 上传目标 URL |
| mode | string | 上传模式:`direct`(直传 S3)或 `proxy`(服务器代理) |
| save_path | string | 文件存储路径 |
| expires_in | integer | URL 有效期(秒),默认 900 秒(15 分钟) |
**错误响应**
@@ -308,7 +306,6 @@ interface PresignInitResponse {
upload_id: string
upload_url: string
mode: 'direct' | 'proxy'
save_path: string
expires_in: number
}
@@ -448,7 +445,7 @@ async function upload() {
## 注意事项
1. **会话有效期**: 上传会话默认 15 分钟后过期,请在有效期内完成上传
2. **文件大小限制**: 受系统配置 `uploadSize` 限制
2. **文件大小限制**: 受系统配置 `upload_size` 限制
3. **过期类型**: 支持 `day`、`hour`、`minute`、`forever`、`count`
4. **CORS**: 直传模式下,S3 需要配置正确的 CORS 策略
5. **重试机制**: 建议实现上传失败重试逻辑
+125
View File
@@ -1,5 +1,130 @@
# Changelog
## [2.8.5](https://github.com/vastsa/FileCodeBox/compare/v2.8.4...v2.8.5) (2026-10-27)
### Bug Fixes
* resolve OSS chunk cleanup and legacy download issues ([381a539](https://github.com/vastsa/FileCodeBox/commit/381a539c92ba85681a1e820a4150fff17984147a))
## [2.8.4](https://github.com/vastsa/FileCodeBox/compare/v2.8.3...v2.8.4) (2026-10-25)
### Performance Improvements
* **db:** tune SQLite connection pragmas for small VPS ([2073f9b](https://github.com/vastsa/FileCodeBox/commit/2073f9ba4b6d17e2a9c81832c7d77468e9a15814))
* **db:** tune SQLite connection pragmas for small VPS ([ec714dc](https://github.com/vastsa/FileCodeBox/commit/ec714dcd6fec2dc2097d05a250ea3962d01d66aa))
* **rate-limit:** bound tracked IPs and use monotonic clock ([496a713](https://github.com/vastsa/FileCodeBox/commit/496a7139bdb2cdac7f516986b613715a693fff13))
* **startup:** drop duplicate Tortoise init, keep ORM exception handlers ([4f141ee](https://github.com/vastsa/FileCodeBox/commit/4f141eeb4940311ad6dcfe2bc11650fa505f825b))
* **web:** cache rendered index.html and add theme asset cache headers ([5b5d70c](https://github.com/vastsa/FileCodeBox/commit/5b5d70c3b7eb3b8efb3d77d731c860e88d1e7a92))
## [2.8.3](https://github.com/vastsa/FileCodeBox/compare/v2.8.2...v2.8.3) (2026-10-10)
### Bug Fixes
* remove HTML preview endpoint ([65f4929](https://github.com/vastsa/FileCodeBox/commit/65f4929d405408e1c9e58127e0097f8b78b1819f))
## [2.8.2](https://github.com/vastsa/FileCodeBox/compare/v2.8.1...v2.8.2) (2026-10-10)
### Bug Fixes
* type MCP protocol results ([da374b4](https://github.com/vastsa/FileCodeBox/commit/da374b459537ee079e1d03da121a935c8fb18f0e))
## [2.8.1](https://github.com/vastsa/FileCodeBox/compare/v2.8.0...v2.8.1) (2026-10-10)
### Bug Fixes
* restore lint and Docker theme build ([2f5eab1](https://github.com/vastsa/FileCodeBox/commit/2f5eab18b1ab28fcb4f416248076ce574998d718))
## [2.8.0](https://github.com/vastsa/FileCodeBox/compare/v2.7.1...v2.8.0) (2026-10-10)
### Features
* add MCP quick share and safe HTML preview ([63e72c1](https://github.com/vastsa/FileCodeBox/commit/63e72c145576d169265e4fa8cf373e33531a06a0))
## [2.7.1](https://github.com/vastsa/FileCodeBox/compare/v2.7.0...v2.7.1) (2026-09-19)
### Bug Fixes
* keep Docker as root, close s3_hostname SSRF, unbreak 404 tests ([031d300](https://github.com/vastsa/FileCodeBox/commit/031d300caea99117e82b3bddc3e5d0532cf3efa6))
## [2.7.0](https://github.com/vastsa/FileCodeBox/compare/v2.6.1...v2.7.0) (2026-09-16)
### Features
* mint extract codes from data/local without re-upload ([93a30a2](https://github.com/vastsa/FileCodeBox/commit/93a30a257cc5b0ee23a374521e1e12602c3959da))
## [2.6.1](https://github.com/vastsa/FileCodeBox/compare/v2.6.0...v2.6.1) (2026-09-14)
### Bug Fixes
* don't let a legacy background value block settings saves ([eb2db8f](https://github.com/vastsa/FileCodeBox/commit/eb2db8f507e3198e0ae45b976bdd90eab2f7d1ed))
* 存量 background 值不再阻塞后台设置保存([#514](https://github.com/vastsa/FileCodeBox/issues/514) 升级回归) ([0d82fbe](https://github.com/vastsa/FileCodeBox/commit/0d82fbe26ad06ca77af7b2b5aa05f514e8362295))
## [2.6.0](https://github.com/vastsa/FileCodeBox/compare/v2.5.6...v2.6.0) (2026-09-14)
### Features
* declare missing config keys (opendal_scheme, chunk_expire_hours) and add config schema guard ([c5d525d](https://github.com/vastsa/FileCodeBox/commit/c5d525df08bb9534d1efb3ab5ff9357b52240fe9))
* rename all config keys to snake_case with DB migration 007 ([5121db6](https://github.com/vastsa/FileCodeBox/commit/5121db6edcdac57318d058e28d7249efb9ea4d03))
* upgrade password hashing to memory-hard scrypt with transparent rehash ([443cfe7](https://github.com/vastsa/FileCodeBox/commit/443cfe7472693bb954845f4cc4ac77e3abfcbd23))
### Bug Fixes
* FileCodes.expired_count defaults to -1 (time-based) instead of instantly-expired 0 (D4) ([3c93718](https://github.com/vastsa/FileCodeBox/commit/3c93718d4b2f3a8e93978c7eb7a4a3b2553b0855))
* save_file accepts plain binary streams; repair admin local file share (D2) ([bb95017](https://github.com/vastsa/FileCodeBox/commit/bb95017aa398e80e5eca512aa9a330aaeeceb364))
* serialize KeyValue JSON read-modify-write sections behind a process lock (D5) ([0038636](https://github.com/vastsa/FileCodeBox/commit/0038636a2a5fc8e73f944c38789dce8de0354c3a))
* snake_case the public config contract (D7 follow-up) ([800bac0](https://github.com/vastsa/FileCodeBox/commit/800bac01285dd6eb827e28460996997dbb80e2c3))
* validate background config as http(s) URL to close CSS quote-breakout in 2023 theme ([4cab625](https://github.com/vastsa/FileCodeBox/commit/4cab625d5ab2fe5921f8f621ebe015623afb6fb8))
### Performance Improvements
* cache settings refresh behind a 2s TTL, force refresh on write paths (D6) ([61d72ab](https://github.com/vastsa/FileCodeBox/commit/61d72ab6ccba5df121945ef825de2967c45888b4))
* dashboard counters and storage quota use SQL aggregation (D3) ([baecebd](https://github.com/vastsa/FileCodeBox/commit/baecebd81efc9677acac29d5335e281d6bb5a252))
* make is_config_initialized cheap — no scrypt on the per-request probe (D1) ([ec424dd](https://github.com/vastsa/FileCodeBox/commit/ec424dd9093d94b69ec3a118f321cd1f85c7dd54))
## [2.5.6](https://github.com/vastsa/FileCodeBox/compare/v2.5.5...v2.5.6) (2026-09-01)
### Bug Fixes
* **logging:** reduce production log noise ([5537f56](https://github.com/vastsa/FileCodeBox/commit/5537f56b884a15eda015665c67c3f8469e4e9d89))
## [2.5.5](https://github.com/vastsa/FileCodeBox/compare/v2.5.4...v2.5.5) (2026-08-29)
### Bug Fixes
* **chunk-upload:** persist sanitized filenames ([da1dd06](https://github.com/vastsa/FileCodeBox/commit/da1dd066551cc7cc9a631a0c982b5675bd4b1967))
* harden chunk upload filename metadata ([5a7195c](https://github.com/vastsa/FileCodeBox/commit/5a7195c5c7d1028500b0e363c6b47130d260e11c))
* sanitize chunk upload file_name to prevent path traversal ([cebac32](https://github.com/vastsa/FileCodeBox/commit/cebac32145f948bfca7c44c663a0bcdcd9c56830))
## [2.5.4](https://github.com/vastsa/FileCodeBox/compare/v2.5.3...v2.5.4) (2026-07-23)
### Bug Fixes
* run Docker as root for volume compatibility ([fbcd4a5](https://github.com/vastsa/FileCodeBox/commit/fbcd4a5b2b788cdfc8e86d920f9c5a7299dc839a))
* run Docker image as root for data volume compatibility ([fc0cbeb](https://github.com/vastsa/FileCodeBox/commit/fc0cbeb4f0c7485d2e7abc0501a5fd9b1456cc22))
## [2.5.3](https://github.com/vastsa/FileCodeBox/compare/v2.5.2...v2.5.3) (2026-07-23)
### Bug Fixes
* add magic-byte checks, login rate limits, and non-root Docker ([eda0a31](https://github.com/vastsa/FileCodeBox/commit/eda0a3127ab562b6a2cf828c0df4a93933c44526))
* magic-byte file checks, admin login rate limit, non-root Docker ([0ee2f04](https://github.com/vastsa/FileCodeBox/commit/0ee2f046dd057fecceaee1059d72d67de3038f19))
## [2.5.2](https://github.com/vastsa/FileCodeBox/compare/v2.5.1...v2.5.2) (2026-07-23)
+17 -15
View File
@@ -1,6 +1,6 @@
# FileCodeBox API Documentation
## API Version: 2.1.0
## API Version: 2.5.6
## Table of Contents
- [Authentication](#authentication)
@@ -30,7 +30,7 @@ curl -X POST "http://localhost:12345/admin/login" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"token": "xxx.xxx.xxx",
"token_type": "Bearer"
@@ -74,7 +74,7 @@ curl -X POST "http://localhost:12345/share/text/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123"
}
@@ -125,7 +125,7 @@ curl -X POST "http://localhost:12345/share/file/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt"
@@ -157,7 +157,7 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -184,7 +184,7 @@ Select file by share code.
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -232,15 +232,15 @@ Get system dashboard data.
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"totalFiles": 100,
"storageUsed": "1.5GB",
"sysUptime": "10 days",
"yesterdayCount": 50,
"yesterdaySize": "500MB",
"todayCount": 30,
"todaySize": "300MB"
"total_files": 100,
"storage_used": "1610612736",
"sys_uptime": 1725489600.0,
"yesterday_count": 50,
"yesterday_size": "524288000",
"today_count": 30,
"today_size": "314572800",
}
}
```
@@ -258,13 +258,15 @@ Get system file list.
| page | integer | No | 1 | Current page |
| size | integer | No | 10 | Page size |
| keyword | string | No | "" | Search keyword |
| sort_by | string | No | "created_at" | Sort field(created_at/expired_at/name/size/used_count/code) |
| sort_order | string | No | "desc" | Sort direction(asc/desc) |
**Response Example:**
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"page": 1,
"size": 10,
+389
View File
@@ -0,0 +1,389 @@
# Presigned Upload API
## Overview
The presigned upload feature provides a unified file upload interface that automatically picks the optimal upload method based on the configured storage backend:
- **S3 storage**: returns a presigned URL so the client uploads directly to S3 (reduces server bandwidth usage)
- **Other storage**: returns a proxy upload URL and the file is transferred through the server
## Upload Flow
### Flow Diagram
```
┌─────────┐ 1. Initialize upload ┌─────────┐
│ Client │ ───────────────────────▶ │ Server │
└─────────┘ └─────────┘
│ │
│◀──── upload_url + mode ──────────┤
│ │
│ ┌─────────────────────────────────────────┐
│ │ if mode == "direct" (S3 storage) │
│ │ 2a. PUT file to upload_url (S3) │
│ │ 3a. POST /confirm to finish │
│ │ │
│ │ if mode == "proxy" (other storage) │
│ │ 2b. PUT file to upload_url (server) │
│ │ (share code returned automatically, │
│ │ no confirmation needed) │
│ └─────────────────────────────────────────┘
│
▼
Get share code
```
---
## API Endpoints
### 1. Initialize Upload
Initialize a presigned upload session and get the upload URL and mode.
**Request**
```
POST /presign/upload/init
Content-Type: application/json
```
**Request Body**
| Field | Type | Required | Default | Description |
| ------------ | ------- | -------- | ------- | --------------------------------------- |
| file_name | string | ✅ | - | File name (with extension) |
| file_size | integer | ✅ | - | File size in bytes |
| expire_value | integer | ❌ | 1 | Expiration time value |
| expire_style | string | ❌ | "day" | Expiration type: day/hour/minute/forever/count |
**Request Example**
```json
{
"file_name": "document.pdf",
"file_size": 1048576,
"expire_value": 7,
"expire_style": "day"
}
```
**Response**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"upload_url": "https://bucket.s3.amazonaws.com/path?X-Amz-Signature=...",
"mode": "direct",
"expires_in": 900
}
}
```
**Response Fields**
| Field | Type | Description |
| ---------- | ------- | --------------------------------------------------------------- |
| upload_id | string | Upload session ID, required by all subsequent operations |
| upload_url | string | Target upload URL |
| mode | string | Upload mode: `direct` (direct to S3) or `proxy` (via server) |
| expires_in | integer | URL validity in seconds, default 900 seconds (15 minutes) |
In `proxy` mode the response also includes `proxy_upload_url` (and `legacy_proxy_upload_url`); `upload_url` equals the proxy URL.
**Error Responses**
| Status | Description |
| ------ | -------------------------------------------------- |
| 400 | Invalid expiration type |
| 403 | File size exceeds limit / IP rate limit exceeded |
---
### 2a. Direct Mode - Upload File to S3
When `mode == "direct"`, the client PUTs the file directly to the returned presigned URL.
**Request**
```
PUT {upload_url}
Content-Type: application/octet-stream
[binary file content]
```
**Notes**
- Use the returned `upload_url` as-is, do not modify it
- `application/octet-stream` is the recommended Content-Type
- The request goes directly to S3, it does not pass through the server
**JavaScript Example**
```javascript
const response = await fetch(uploadUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': 'application/octet-stream',
},
})
if (response.ok) {
// Upload succeeded, call the confirm endpoint
}
```
---
### 2b. Proxy Mode - Upload File to Server
When `mode == "proxy"`, the client PUTs the file to the server proxy endpoint.
**Request**
```
PUT /presign/upload/proxy/{upload_id}
Content-Type: multipart/form-data
file: [file]
```
**Path Parameters**
| Parameter | Description |
| --------- | ----------------------------------------------- |
| upload_id | Upload session ID returned by the init endpoint |
**Response**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**Note**: In proxy mode the share code is returned immediately after upload; no confirmation call is needed.
**Error Responses**
| Status | Description |
| ------ | ------------------------------------------------------------------ |
| 400 | File size mismatches declared size / session does not support proxy |
| 404 | Upload session does not exist or has expired |
| 500 | Failed to save file |
---
### 3. Confirm Upload (direct mode only)
In direct mode, call this endpoint after the S3 upload completes to obtain the share code.
**Request**
```
POST /presign/upload/confirm/{upload_id}
Content-Type: application/json
```
**Path Parameters**
| Parameter | Description |
| --------- | ----------------------------------------------- |
| upload_id | Upload session ID returned by the init endpoint |
**Response**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**Error Responses**
| Status | Description |
| ------ | ------------------------------------------------------------------------------- |
| 400 | Session does not support direct confirmation |
| 404 | Upload session does not exist or has expired / file not uploaded or upload failed |
---
### 4. Query Upload Status
Query the current state of an upload session.
**Request**
```
GET /presign/upload/status/{upload_id}
```
**Response**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"file_name": "document.pdf",
"file_size": 1048576,
"mode": "direct",
"created_at": "2024-01-01T12:00:00",
"expires_at": "2024-01-01T12:15:00",
"is_expired": false
}
}
```
**Error Responses**
| Status | Description |
| ------ | ---------------------------------- |
| 404 | Upload session does not exist |
---
### 5. Cancel Upload
Cancel an upload session and clean up related resources.
**Request**
```
DELETE /presign/upload/{upload_id}
```
**Response**
```json
{
"code": 200,
"detail": {
"message": "Upload session cancelled"
}
}
```
**Error Responses**
| Status | Description |
| ------ | ---------------------------------- |
| 404 | Upload session does not exist |
---
## Frontend Integration Example
### Complete Upload Flow (JavaScript/TypeScript)
```typescript
interface PresignInitResponse {
upload_id: string
upload_url: string
mode: 'direct' | 'proxy'
expires_in: number
}
interface UploadResult {
code: string
name: string
}
async function uploadFile(
file: File,
expireValue: number = 1,
expireStyle: string = 'day'
): Promise<UploadResult> {
// 1. Initialize upload
const initResponse = await fetch('/presign/upload/init', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
file_name: file.name,
file_size: file.size,
expire_value: expireValue,
expire_style: expireStyle,
}),
})
const initData = await initResponse.json()
if (initData.code !== 200) {
throw new Error(initData.detail)
}
const { upload_id, upload_url, mode } = initData.detail as PresignInitResponse
// 2. Upload file based on mode
if (mode === 'direct') {
// Direct mode: upload to S3
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: file,
headers: { 'Content-Type': 'application/octet-stream' },
})
if (!uploadResponse.ok) {
throw new Error('S3 upload failed')
}
// 3. Confirm upload
const confirmResponse = await fetch(
`/presign/upload/confirm/${upload_id}`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
}
)
const confirmData = await confirmResponse.json()
if (confirmData.code !== 200) {
throw new Error(confirmData.detail)
}
return confirmData.detail
} else {
// Proxy mode: upload through the server
const formData = new FormData()
formData.append('file', file)
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: formData,
})
const uploadData = await uploadResponse.json()
if (uploadData.code !== 200) {
throw new Error(uploadData.detail)
}
return uploadData.detail
}
}
// Usage example
const file = document.querySelector('input[type="file"]').files[0]
const result = await uploadFile(file, 7, 'day')
console.log('Share code:', result.code)
```
---
## Notes
1. **Session validity**: upload sessions expire after 15 minutes by default; complete the upload within the window
2. **File size limit**: bounded by the `upload_size` system setting
3. **Expiration types**: `day`, `hour`, `minute`, `forever`, `count`
4. **CORS**: in direct mode, S3 must have a proper CORS policy configured
5. **Retry**: implement upload retry logic on the client side for failed uploads
+1 -1
View File
@@ -1,6 +1,6 @@
## [Unreleased]
<!-- Current release: 2.5.2 x-release-please-version -->
<!-- Current release: 2.8.5 x-release-please-version -->
### Added
+38 -38
View File
@@ -6,7 +6,7 @@ FileCodeBox provides rich configuration options. Prefer the admin panel; directl
FileCodeBox supports two configuration methods:
1. **Admin Panel Configuration** (Recommended): Access `/admin` to enter the admin panel and modify settings on the settings page
1. **Admin Panel Configuration** (Recommended): Access `/#/admin` to enter the admin panel and modify settings on the settings page
2. **Database Configuration**: Configuration is stored in the `data/filecodebox.db` database
::: tip Note
@@ -22,9 +22,9 @@ On first startup, the system uses default configuration from `core/settings.py`.
| `name` | string | `文件快递柜 - FileCodeBox` | Site name, displayed in page title and navigation bar |
| `description` | string | `开箱即用的文件快传系统` | Site description, used for SEO |
| `keywords` | string | `FileCodeBox, 文件快递柜...` | Site keywords, used for SEO |
| `serverHost` | string | `0.0.0.0` | Service listening address |
| `serverPort` | int | `12345` | Service listening port |
| `serverWorkers` | int | `1` | Worker count; keep one worker for SQLite deployments |
| `server_host` | string | `0.0.0.0` | Service listening address |
| `server_port` | int | `12345` | Service listening port |
| `server_workers` | int | `1` | Worker count; keep one worker for SQLite deployments |
### Notification Settings
@@ -33,7 +33,7 @@ On first startup, the system uses default configuration from `core/settings.py`.
| `notify_title` | string | `系统通知` | Notification title |
| `notify_content` | string | Welcome message | Notification content, supports HTML |
| `page_explain` | string | Legal disclaimer | Footer explanation text |
| `robotsText` | string | `User-agent: *\nDisallow: /` | robots.txt content |
| `robots_text` | string | `User-agent: *\nDisallow: /` | robots.txt content |
## Upload Settings
@@ -41,21 +41,21 @@ On first startup, the system uses default configuration from `core/settings.py`.
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `openUpload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `uploadSize` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `enableChunk` | int | `0` | Enable chunked upload (1=enabled, 0=disabled) |
| `open_upload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `upload_size` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `enable_chunk` | int | `0` | Enable chunked upload (1=enabled, 0=disabled) |
| `allowed_file_types` | list | `["*"]` | Allowed extensions; `*` allows every file type |
::: warning Note
`uploadSize` is in bytes. 10MB = 10 * 1024 * 1024 = 10485760 bytes
`upload_size` is in bytes. 10MB = 10 * 1024 * 1024 = 10485760 bytes
:::
### Upload Rate Limiting
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `uploadMinute` | int | `1` | Upload limit time window (minutes) |
| `uploadCount` | int | `10` | Maximum uploads allowed within the time window |
| `upload_minute` | int | `1` | Upload limit time window (minutes) |
| `upload_count` | int | `10` | Maximum uploads allowed within the time window |
Example: Default configuration allows up to 10 uploads per minute.
@@ -64,7 +64,7 @@ Example: Default configuration allows up to 10 uploads per minute.
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `max_save_seconds` | int | `0` | Maximum file retention time (seconds), 0 means no limit |
Expiration methods explained:
@@ -80,8 +80,8 @@ Expiration methods explained:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `themesSelect` | string | `themes/2024` | Currently active theme |
| `themesChoices` | list | See below | Available themes list |
| `themes_select` | string | `themes/2024` | Currently active theme |
| `themes_choices` | list | See below | Available themes list |
Default available themes:
```json
@@ -113,8 +113,8 @@ Default available themes:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `admin_token` | string | Set during setup | Admin login password |
| `showAdminAddr` | int | `0` | Show admin panel entry on homepage (1=show, 0=hide) |
| `adminSessionExpire` | int | `2592000` | Admin session lifetime in seconds (30 days) |
| `show_admin_addr` | int | `0` | Show admin panel entry on homepage (1=show, 0=hide) |
| `admin_session_expire` | int | `2592000` | Admin session lifetime in seconds (30 days) |
::: danger Security Warning
The setup page is shown automatically while the system is uninitialized. Complete setup before exposing a production service to the public internet.
@@ -126,9 +126,9 @@ The setup page is shown automatically while the system is uninitialized. Complet
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `errorMinute` | int | `1` | Error limit time window (minutes) |
| `errorCount` | int | `10` | Maximum errors allowed within the time window |
| `trustedProxies` | list | `[]` | Trusted reverse-proxy IPs used to resolve client addresses |
| `error_minute` | int | `1` | Error limit time window (minutes) |
| `error_count` | int | `10` | Maximum errors allowed within the time window |
| `trusted_proxies` | list | `[]` | Trusted reverse-proxy IPs used to resolve client addresses |
This setting prevents brute-force attacks on extraction codes.
@@ -140,7 +140,7 @@ This setting prevents brute-force attacks on extraction codes.
|---------|------|---------|-------------|
| `file_storage` | string | `local` | Storage backend type |
| `storage_path` | string | `""` | Custom storage path |
| `storageLimit` | int | `0` | Total storage quota in bytes; 0 means unlimited |
| `storage_limit` | int | `0` | Total storage quota in bytes; 0 means unlimited |
Supported storage types:
- `local` - Local storage
@@ -160,11 +160,11 @@ Suitable for personal or small team use with relaxed limits:
```python
{
"name": "My File Share",
"uploadSize": 52428800, # 50MB
"uploadMinute": 5, # 5 minutes
"uploadCount": 20, # Max 20 uploads
"expireStyle": ["day", "hour", "forever"],
"showAdminAddr": 1
"upload_size": 52428800, # 50MB
"upload_minute": 5, # 5 minutes
"upload_count": 20, # Max 20 uploads
"expire_style": ["day", "hour", "forever"],
"show_admin_addr": 1
}
```
@@ -175,14 +175,14 @@ Suitable for public services requiring stricter limits:
```python
{
"name": "Public File Box",
"uploadSize": 10485760, # 10MB
"uploadMinute": 1, # 1 minute
"uploadCount": 5, # Max 5 uploads
"errorMinute": 5, # 5 minutes
"errorCount": 3, # Max 3 errors
"expireStyle": ["hour", "minute", "count"],
"upload_size": 10485760, # 10MB
"upload_minute": 1, # 1 minute
"upload_count": 5, # Max 5 uploads
"error_minute": 5, # 5 minutes
"error_count": 3, # Max 3 errors
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # Max retention 1 day
"showAdminAddr": 0
"show_admin_addr": 0
}
```
@@ -193,13 +193,13 @@ Suitable for enterprise internal use with large file and chunked upload support:
```python
{
"name": "Enterprise File Transfer",
"uploadSize": 1073741824, # 1GB
"enableChunk": 1, # Enable chunked upload
"uploadMinute": 10, # 10 minutes
"uploadCount": 100, # Max 100 uploads
"expireStyle": ["day", "forever"],
"upload_size": 1073741824, # 1GB
"enable_chunk": 1, # Enable chunked upload
"upload_minute": 10, # 10 minutes
"upload_count": 100, # Max 100 uploads
"expire_style": ["day", "forever"],
"file_storage": "s3", # Use S3 storage
"showAdminAddr": 1
"show_admin_addr": 1
}
```
+11 -1
View File
@@ -21,6 +21,10 @@ FileCodeBox is a simple and efficient file sharing tool that supports temporary
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
@@ -32,7 +36,13 @@ The named volume persists configuration and uploaded files across container upgr
| `HOST` | `0.0.0.0` | Service listening address |
| `PORT` | `12345` | Service port |
| `WORKERS` | `1` | Worker count; keep one worker for SQLite deployments |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warning`, or `error` |
| `APP_ENV` | `production` (Docker) | Runtime environment; production disables HTTP access logs by default |
| `LOG_LEVEL` | `warning` (Docker) | `debug`, `info`, `warning`, or `error` |
| `ACCESS_LOG` | `false` (production) | Enable HTTP access logs; set to `true` when debugging requests |
| `TIMEOUT_KEEP_ALIVE` | `5` | HTTP Keep-Alive idle timeout in seconds; raise it behind a reverse proxy to reduce reconnects |
| `MALLOC_TRIM_THRESHOLD_` | `65536` | glibc heap trim threshold in bytes; lowers resident memory after large transfers (glibc images only) |
Production deployments output only warning, error, and other key events by default. Docker retains at most three 10 MB log files.
### Manual Deployment
+12 -4
View File
@@ -1,6 +1,6 @@
<div align="center">
<img src="https://fastly.jsdelivr.net/gh/vastsa/FileCodeBox@V1.6/static/banners/img_1.png" alt="FileCodeBox Logo">
<img src="/screenshots/retrieve.webp" alt="FileCodeBox" width="860" />
<p><em>Share text and files anonymously with a passcode, like picking up a package</em></p>
@@ -136,7 +136,15 @@ Command-line download
### Docker Deployment
```bash
docker run -d --restart unless-stopped -p 12345:12345 -v fcb-data:/app/data --name filecodebox lanol/filecodebox:latest
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
### Manual Deployment
@@ -171,7 +179,7 @@ python main.py
3. Download file or view text
### Admin Panel
1. Visit `/admin`
1. Visit `/#/admin`
2. Enter admin password
3. Manage files and settings
@@ -218,7 +226,7 @@ npm run dev
## ❓ FAQ
### Q: How to modify upload size limit?
A: Change `uploadSize` in admin panel
A: Change `upload_size` in admin panel
### Q: How to configure storage engine?
A: Select storage engine and configure parameters in admin panel
+58 -41
View File
@@ -4,12 +4,16 @@ FileCodeBox provides a fully-featured admin panel that allows administrators to
## Accessing the Admin Panel
<p align="center">
<img src="/screenshots/login.webp" alt="Admin login" width="640" />
</p>
### Login Method
The admin panel is located at the `/admin` path. Access method:
The admin panel is located at `/#/admin` (a frontend hash route). Access method:
1. On first run, set the admin password on the setup page
2. Visit `http://your-domain.com/admin` in your browser
2. Visit `http://your-domain.com/#/admin` in your browser
3. Enter the admin password (the value of the `admin_token` configuration)
4. Click the login button
@@ -23,10 +27,10 @@ By default, the admin panel entry is not shown on the homepage. You can control
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `showAdminAddr` | int | `0` | Show admin entry on homepage (1=show, 0=hide) |
| `show_admin_addr` | int | `0` | Show admin entry on homepage (1=show, 0=hide) |
::: warning Security Recommendation
For public services, it's recommended to keep `showAdminAddr` at `0` and access the admin panel directly via the `/admin` path to reduce the risk of malicious scanning.
For public services, keep `show_admin_addr` at `0` and access the admin panel via `/#/admin`. Hiding the link is not access control; a strong password is still required.
:::
### Authentication Mechanism
@@ -39,6 +43,10 @@ The admin panel uses JWT (JSON Web Token) for authentication:
## Dashboard
<p align="center">
<img src="/screenshots/dashboard.webp" alt="Dashboard" width="860" />
</p>
After logging in, you first see the dashboard page, which displays the overall system status.
### Statistics
@@ -67,6 +75,10 @@ Storage usage is displayed in bytes. For example, `10485760` represents approxim
## File Management
<p align="center">
<img src="/screenshots/files.webp" alt="File management" width="860" />
</p>
### File List
The file management page displays all shared files in the system, supporting pagination and search.
@@ -154,52 +166,57 @@ When modifying extraction codes, the system checks if the new code is already in
## Local File Management
In addition to managing shared files, the admin panel also provides local file management functionality for managing files in the `data/local` directory.
In addition to managing shared files, the admin panel has a Local Share page for browsing `data/local` (including subfolders) and minting extract codes.
Mount a NAS share at `/app/data/local` to generate extract codes for files already on disk, without downloading and re-uploading them.
```yaml
volumes:
- fcb-data:/app/data:rw
# Read-only NAS share for admin browsing and extract-code generation
- /volume1/share:/app/data/local:ro
```
### View Local Files
The local file list displays all files in the `data/local` directory:
The Local Share page supports nested folders. List API:
| Information | Description |
|-------------|-------------|
| Filename | Complete filename |
| Creation Time | File creation time |
| File Size | File size (bytes) |
```
GET /admin/local/lists?path=movies
```
`path` is relative to `data/local`; leave it empty for the root. Response:
| Field | Description |
|------|------|
| `path` | Current directory |
| `parent` | Parent directory |
| `truncated` | Whether the listing was truncated |
| `items[].type` | `dir` or `file` |
| `items[].path` | Path relative to `data/local`, used as `filename` when sharing |
### Share Local Files
You can quickly share local files:
1. Select the file to share in the local file list
1. Open the target subfolder and pick a file
2. Set expiration method and value
3. Click the share button
4. System generates extraction code
3. Create an extract code
Sharing is a **zero-copy reference**: the file is not copied into `data/share` and does not count against storage quota. Expiring or deleting the share record **does not delete the original NAS file**.
**Share parameters:**
| Parameter | Description |
|-----------|-------------|
| `filename` | Filename to share |
| `filename` | Path relative to `data/local`, e.g. `movies/a.mp4` |
| `expire_style` | Expiration method (day/hour/minute/forever/count) |
| `expire_value` | Expiration value (days/hours/minutes/download count) |
### Delete Local Files
You can delete files in the `data/local` directory:
1. Find the file to delete in the local file list
2. Click the delete button
3. Confirm deletion
::: tip Use Cases
Local file management is useful for:
- Sharing files after batch uploading to the server
- Managing files uploaded to the server through other means
- Cleaning up unnecessary local files
:::
## System Settings
<p align="center">
<img src="/screenshots/settings.webp" alt="System settings" width="860" />
</p>
### View Configuration
On the system settings page, you can view the current values of all configuration items. Configuration items are grouped by category:
@@ -224,15 +241,15 @@ Administrators can modify most configurations through the admin panel:
| Category | Example Settings |
|----------|------------------|
| Basic Settings | `name`, `description`, `keywords`, `notify_title`, `notify_content` |
| Upload Settings | `uploadSize`, `uploadMinute`, `uploadCount`, `openUpload`, `enableChunk` |
| Expiration Settings | `expireStyle`, `max_save_seconds` |
| Theme Settings | `themesSelect`, `opacity`, `background` |
| Security Settings | `admin_token`, `showAdminAddr`, `errorMinute`, `errorCount` |
| Upload Settings | `upload_size`, `upload_minute`, `upload_count`, `open_upload`, `enable_chunk` |
| Expiration Settings | `expire_style`, `max_save_seconds` |
| Theme Settings | `themes_select`, `opacity`, `background` |
| Security Settings | `admin_token`, `show_admin_addr`, `error_minute`, `error_count` |
| Storage Settings | `file_storage`, `storage_path` and storage backend-specific configurations |
::: warning Note
- `admin_token` (admin password) cannot be set to empty
- `themesChoices` (theme list) cannot be modified through the admin panel
- `themes_choices` (theme list) cannot be modified through the admin panel
- After modifying storage settings, existing files will not be automatically migrated
:::
@@ -322,7 +339,7 @@ Content-Type: application/json
**Get Local File List**
```
GET /admin/local/lists
GET /admin/local/lists?path=movies
Authorization: Bearer <token>
```
@@ -333,7 +350,7 @@ Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "example.txt"
"filename": "movies/example.txt"
}
```
@@ -344,7 +361,7 @@ Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "example.txt",
"filename": "movies/example.txt",
"expire_style": "day",
"expire_value": 7
}
@@ -366,7 +383,7 @@ Content-Type: application/json
{
"admin_token": "new-password",
"uploadSize": 52428800
"upload_size": 52428800
}
```
+24
View File
@@ -0,0 +1,24 @@
# MCP Integration
FileCodeBox provides a **Model Context Protocol (MCP)** endpoint over Streamable HTTP. Compatible AI clients can create text or file shares through it.
## Connection
Add a remote HTTP server in your MCP client:
- **Transport**: Streamable HTTP
- **Server URL**: `http://<FileCodeBox host>:12345/mcp`
- **Authentication**: no token required by default
For a local instance, use `http://localhost:12345/mcp`. If the administrator disables guest uploads or enables MCP token validation in the admin settings, configure `Authorization: Bearer <admin token>` in the client. Never commit the token to public configuration or source code.
## Available tools
- `share_text`: create a text share. Required: `text`; optional: `expire_value`, `expire_style`.
- `share_file`: upload a Base64-encoded file and create a share. Required: `filename`, `content_base64`; optional: `content_type`, `expire_value`, `expire_style`.
`expire_style` accepts `day`, `hour`, `minute`, `forever`, or `count`; `expire_value` must be greater than zero. Successful calls return a share code, URL, and Markdown link.
## Security and limits
MCP uploads follow FileCodeBox's guest-upload switch, admin-token validation, upload-size and file-type limits, and upload rate limits. Expose the endpoint only to trusted AI clients. Use HTTPS for public deployments and keep the admin token secure.
+41 -41
View File
@@ -12,7 +12,7 @@ FileCodeBox does not ship with a default admin password. Open the site in your b
Change the admin password through the admin panel:
1. Access `/admin` to enter the admin panel
1. Access `/#/admin` to enter the admin panel
2. Log in with the current password
3. Go to the "System Settings" page
4. Find the `admin_token` configuration item
@@ -34,14 +34,14 @@ Change the admin password through the admin panel:
### Hide Admin Entry
By default, the admin panel entry is hidden. You can control whether to show the admin entry on the homepage via the `showAdminAddr` configuration:
By default, the admin panel entry is hidden. You can control whether to show the admin entry on the homepage via the `show_admin_addr` configuration:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `showAdminAddr` | int | `0` | Show admin entry (1=show, 0=hide) |
| `show_admin_addr` | int | `0` | Show admin entry (1=show, 0=hide) |
::: tip Recommendation
For public services, it's recommended to keep `showAdminAddr` at `0` and access the admin panel directly via the `/admin` path.
For public services, it's recommended to keep `show_admin_addr` at `0` and access the admin panel directly via `/#/admin`.
:::
## IP Rate Limiting
@@ -54,12 +54,12 @@ Limit the number of uploads from a single IP within a specified time:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `uploadMinute` | int | `1` | Upload limit time window (minutes) |
| `uploadCount` | int | `10` | Maximum uploads allowed within the time window |
| `upload_minute` | int | `1` | Upload limit time window (minutes) |
| `upload_count` | int | `10` | Maximum uploads allowed within the time window |
**How it works:**
- System records upload requests from each IP
- When an IP's upload count reaches `uploadCount` within `uploadMinute` minutes
- When an IP's upload count reaches `upload_count` within `upload_minute` minutes
- Subsequent upload requests from that IP will be rejected with HTTP 423 error
- Counter resets after the time window expires
@@ -68,14 +68,14 @@ Limit the number of uploads from a single IP within a specified time:
```python
# Relaxed configuration: Max 20 uploads in 5 minutes
{
"uploadMinute": 5,
"uploadCount": 20
"upload_minute": 5,
"upload_count": 20
}
# Strict configuration: Max 3 uploads in 1 minute
{
"uploadMinute": 1,
"uploadCount": 3
"upload_minute": 1,
"upload_count": 3
}
```
@@ -86,13 +86,13 @@ Limit the number of error attempts from a single IP to prevent brute-force attac
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `errorMinute` | int | `1` | Error limit time window (minutes) |
| `errorCount` | int | `10` | Maximum errors allowed within the time window |
| `error_minute` | int | `1` | Error limit time window (minutes) |
| `error_count` | int | `10` | Maximum errors allowed within the time window |
**How it works:**
- When a user enters an incorrect extraction code, the system records the error count for that IP
- When error count reaches `errorCount`, that IP will be temporarily locked
- Lock duration is `errorMinute` minutes
- When error count reaches `error_count`, that IP will be temporarily locked
- Lock duration is `error_minute` minutes
- During lockout, all extraction requests from that IP will be rejected
**Configuration example:**
@@ -100,8 +100,8 @@ Limit the number of error attempts from a single IP to prevent brute-force attac
```python
# Anti-brute-force configuration: Max 3 errors in 5 minutes
{
"errorMinute": 5,
"errorCount": 3
"error_minute": 5,
"error_count": 3
}
```
@@ -115,8 +115,8 @@ The default allows up to 10 errors per IP each minute. Public services can tight
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `uploadSize` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `openUpload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `upload_size` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `open_upload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
**Common size conversions:**
- 10MB = 10 * 1024 * 1024 = `10485760`
@@ -130,7 +130,7 @@ Through file expiration mechanisms, you can automatically clean up expired files
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `max_save_seconds` | int | `0` | Maximum file retention time (seconds), 0 means no limit |
**Expiration methods explained:**
@@ -150,7 +150,7 @@ For public services, it's recommended to:
```python
# Recommended configuration for public services
{
"expireStyle": ["hour", "minute", "count"],
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400 # Max retention 1 day
}
```
@@ -161,7 +161,7 @@ In some cases, you may need to temporarily disable upload functionality:
```python
{
"openUpload": 0 # Disable upload functionality
"open_upload": 0 # Disable upload functionality
}
```
@@ -197,7 +197,7 @@ server {
add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# Limit request body size (match uploadSize configuration)
# Limit request body size (match upload_size configuration)
client_max_body_size 100M;
# Pass real IP
@@ -236,7 +236,7 @@ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
**2. Request Body Size Limit**
Nginx's `client_max_body_size` should match or be slightly larger than FileCodeBox's `uploadSize` configuration:
Nginx's `client_max_body_size` should match or be slightly larger than FileCodeBox's `upload_size` configuration:
```nginx
client_max_body_size 100M; # Allow max 100MB uploads
@@ -269,7 +269,7 @@ your-domain.com {
Before deploying FileCodeBox, confirm the following security configurations:
- [ ] Completed first-run setup and set the admin password `admin_token`
- [ ] Hidden admin entry `showAdminAddr: 0`
- [ ] Hidden admin entry `show_admin_addr: 0`
- [ ] Configured appropriate upload rate limiting
- [ ] Configured error rate limiting to prevent brute-force attacks
- [ ] Set reasonable file size limits
@@ -285,15 +285,15 @@ Before deploying FileCodeBox, confirm the following security configurations:
```python
{
"admin_token": "your-very-secure-password",
"showAdminAddr": 0,
"uploadSize": 10485760, # 10MB
"uploadMinute": 1,
"uploadCount": 5,
"errorMinute": 5,
"errorCount": 3,
"expireStyle": ["hour", "minute", "count"],
"show_admin_addr": 0,
"upload_size": 10485760, # 10MB
"upload_minute": 1,
"upload_count": 5,
"error_minute": 5,
"error_count": 3,
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # Max 1 day
"openUpload": 1
"open_upload": 1
}
```
@@ -302,15 +302,15 @@ Before deploying FileCodeBox, confirm the following security configurations:
```python
{
"admin_token": "internal-secure-password",
"showAdminAddr": 1,
"uploadSize": 104857600, # 100MB
"uploadMinute": 5,
"uploadCount": 50,
"errorMinute": 1,
"errorCount": 5,
"expireStyle": ["day", "hour", "forever"],
"show_admin_addr": 1,
"upload_size": 104857600, # 100MB
"upload_minute": 5,
"upload_count": 50,
"error_minute": 1,
"error_count": 5,
"expire_style": ["day", "hour", "forever"],
"max_save_seconds": 0, # No limit
"openUpload": 1
"open_upload": 1
}
```
+12 -4
View File
@@ -4,6 +4,10 @@ FileCodeBox provides simple and easy-to-use file and text sharing functionality.
## Sharing Methods
<p align="center">
<img src="/screenshots/send.webp" alt="Send page" width="720" />
</p>
FileCodeBox supports two sharing methods:
1. **Text Sharing** - Share text content directly, suitable for code snippets, configuration files, etc.
@@ -62,10 +66,10 @@ Response example:
### File Size Limit
The default maximum single file upload size is **10MB**. Administrators can modify this limit via the `uploadSize` configuration.
The default maximum single file upload size is **10MB**. Administrators can modify this limit via the `upload_size` configuration.
::: tip Tip
If you need to upload large files, contact the administrator to enable chunked upload functionality or adjust the `uploadSize` configuration.
If you need to upload large files, contact the administrator to enable chunked upload functionality or adjust the `upload_size` configuration.
:::
### Supported Upload Methods
@@ -112,7 +116,7 @@ FileCodeBox supports multiple flexible expiration methods:
| By Count | `count` | File expires after specified download count |
::: info Note
- Administrators can control available expiration methods via the `expireStyle` configuration
- Administrators can control available expiration methods via the `expire_style` configuration
- Administrators can limit maximum file retention time via the `max_save_seconds` configuration
:::
@@ -137,6 +141,10 @@ expire_value=5, expire_style=count
## Retrieving Files
<p align="center">
<img src="/screenshots/retrieve.webp" alt="Retrieve page" width="720" />
</p>
### How to Use
1. Enter the extraction code in the "Retrieve File" area on the homepage
@@ -200,7 +208,7 @@ This endpoint returns file content directly, suitable for direct browser access.
## Chunked Upload (Large Files)
For large file uploads, FileCodeBox supports chunked upload functionality. This feature requires administrator enablement (`enableChunk=1`).
For large file uploads, FileCodeBox supports chunked upload functionality. This feature requires administrator enablement (`enable_chunk=1`).
### Chunked Upload Flow
+7 -2
View File
@@ -35,6 +35,7 @@ storage_path=
- Files are stored by default in `data/share/data/` directory
- Subdirectories are automatically created by date: `year/month/day/fileID/`
- In production, it's recommended to mount the `data/` directory to persistent storage
- Existing NAS files can be mounted at `data/local` (including subfolders). Admin Local Share references them in place to mint extract codes and does not copy them into `data/share`
## S3-Compatible Storage
@@ -51,6 +52,7 @@ Supports all S3-compatible object storage services, including AWS S3, Aliyun OSS
| `s3_endpoint_url` | string | `""` | S3 endpoint URL |
| `s3_region_name` | string | `auto` | Region name |
| `s3_signature_version` | string | `s3v2` | Signature version (`s3v2` or `s3v4`) |
| `s3_addressing_style` | string | `auto` | Addressing style (`auto`, `path`, or `virtual`); use `virtual` for OSS |
| `s3_hostname` | string | `""` | S3 hostname (alternative) |
| `s3_proxy` | int | `0` | Download through server proxy (1=yes, 0=no) |
| `aws_session_token` | string | `""` | AWS session token (optional) |
@@ -75,13 +77,16 @@ file_storage=s3
s3_access_key_id=YourAccessKeyId
s3_secret_access_key=YourSecretAccessKey
s3_bucket_name=bucket-name
s3_endpoint_url=https://bucket-name.oss-cn-hangzhou.aliyuncs.com
s3_endpoint_url=https://oss-cn-hangzhou.aliyuncs.com
s3_region_name=oss-cn-hangzhou
s3_signature_version=s3v4
s3_addressing_style=virtual
```
::: tip Aliyun OSS Endpoint Format
Endpoint URL format: `https://<bucket-name>.<region>.aliyuncs.com`
Endpoint URL format: `https://oss-<region>.aliyuncs.com` (for example, region `cn-hangzhou`), **without the bucket name**. Set the bucket only in `s3_bucket_name` and select Virtual-host (`virtual`) addressing. The SDK adds the bucket to the hostname; including it in the endpoint can duplicate the bucket prefix or cause DNS failures.
The S3 client sets request and response checksums to `when_required` to avoid OSS's unsupported `STREAMING-UNSIGNED-PAYLOAD-TRAILER`. Chunk cleanup uses paginated, single-object deletion; no additional checksum environment variables are needed. Failed cleanup retains the upload session for retry by the scheduled task.
Common regions:
- Hangzhou: `oss-cn-hangzhou`
+23 -19
View File
@@ -4,6 +4,10 @@ FileCodeBox provides multiple flexible file upload methods, supporting both regu
## Upload Methods
<p align="center">
<img src="/screenshots/send.webp" alt="Upload and send page" width="720" />
</p>
FileCodeBox supports the following upload methods:
### Drag and Drop Upload
@@ -47,7 +51,7 @@ Paste upload only supports image formats, not other file types. Specific support
| Setting | Default | Description |
|---------|---------|-------------|
| `uploadSize` | 10MB | Maximum single file upload size |
| `upload_size` | 10MB | Maximum single file upload size |
### Modify Upload Limits
@@ -55,11 +59,11 @@ Administrators can modify upload size limits through the admin panel or configur
```python
# Set maximum upload size to 100MB
uploadSize = 104857600 # 100 * 1024 * 1024
upload_size = 104857600 # 100 * 1024 * 1024
```
::: info Note
`uploadSize` is in bytes. Common conversions:
`upload_size` is in bytes. Common conversions:
- 10MB = 10485760
- 50MB = 52428800
- 100MB = 104857600
@@ -143,7 +147,7 @@ curl -L "http://localhost:12345/share/select/?code=YOUR_CODE" -o downloaded_file
```
::: tip When Authentication Required
If guest upload is disabled in admin panel (`openUpload=0`), you need to login first:
If guest upload is disabled in admin panel (`open_upload=0`), you need to login first:
```bash
# 1. Login to get token
@@ -171,7 +175,7 @@ curl -X POST "http://localhost:12345/share/text/" \
For large files, FileCodeBox supports chunked upload functionality. Chunked upload splits large files into multiple small chunks for separate uploading, supporting resume capability.
::: warning Prerequisite
Chunked upload functionality requires administrator enablement: `enableChunk=1`
Chunked upload functionality requires administrator enablement: `enable_chunk=1`
:::
### Chunked Upload Flow
@@ -359,7 +363,7 @@ await fetch(`/chunk/upload/complete/${upload_id}`, {
| HTTP Status | Error Message | Cause | Solution |
|-------------|---------------|-------|----------|
| 403 | Size exceeds limit | File exceeds `uploadSize` limit | Reduce file size or contact administrator to adjust limit |
| 403 | Size exceeds limit | File exceeds `upload_size` limit | Reduce file size or contact administrator to adjust limit |
| 403 | Upload rate limit | Exceeded IP upload rate limit | Wait for limit time window before retrying |
| 400 | Invalid expiration type | `expire_style` value not in allowed list | Use a valid expiration method |
| 404 | Upload session not found | `upload_id` invalid or expired | Re-initialize upload |
@@ -372,8 +376,8 @@ The system has rate limits on upload operations to prevent abuse:
| Setting | Default | Description |
|---------|---------|-------------|
| `uploadMinute` | 1 | Limit time window (minutes) |
| `uploadCount` | 10 | Maximum uploads within time window |
| `upload_minute` | 1 | Limit time window (minutes) |
| `upload_count` | 10 | Maximum uploads within time window |
When rate limit is exceeded, you need to wait for the time window to pass before continuing uploads.
@@ -391,26 +395,26 @@ When rate limit is exceeded, you need to wait for the time window to pass before
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `openUpload` | int | 1 | Enable upload (1=enabled, 0=disabled) |
| `uploadSize` | int | 10485760 | Maximum upload size (bytes) |
| `enableChunk` | int | 0 | Enable chunked upload (1=enabled, 0=disabled) |
| `uploadMinute` | int | 1 | Upload rate limit time window (minutes) |
| `uploadCount` | int | 10 | Maximum uploads within time window |
| `expireStyle` | list | ["day","hour","minute","forever","count"] | Allowed expiration methods |
| `open_upload` | int | 1 | Enable upload (1=enabled, 0=disabled) |
| `upload_size` | int | 10485760 | Maximum upload size (bytes) |
| `enable_chunk` | int | 0 | Enable chunked upload (1=enabled, 0=disabled) |
| `upload_minute` | int | 1 | Upload rate limit time window (minutes) |
| `upload_count` | int | 10 | Maximum uploads within time window |
| `expire_style` | list | ["day","hour","minute","forever","count"] | Allowed expiration methods |
### Configuration Example
```python
# Allow 100MB file uploads, enable chunked upload
uploadSize = 104857600
enableChunk = 1
upload_size = 104857600
enable_chunk = 1
# Relax upload rate limit: max 50 uploads per 5 minutes
uploadMinute = 5
uploadCount = 50
upload_minute = 5
upload_count = 50
# Only allow expiration by days and count
expireStyle = ["day", "count"]
expire_style = ["day", "count"]
```
## Next Steps
+25 -1
View File
@@ -37,8 +37,12 @@ features:
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v ./data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:2.5.2 # x-release-please-version
lanol/filecodebox:2.8.5 # x-release-please-version
```
Open `http://localhost:12345` and complete the first-run setup.
@@ -61,6 +65,26 @@ Open `http://localhost:12345` and complete the first-run setup.
</a>
</div>
## Interface preview
<p align="center">
<img src="/screenshots/retrieve.webp" alt="Retrieve files" width="860" />
</p>
<p align="center">
<img src="/screenshots/send.webp" alt="Send files" width="48%" />
&nbsp;
<img src="/screenshots/dashboard.webp" alt="Admin dashboard" width="48%" />
</p>
<table>
<tr>
<td width="33%"><img src="/screenshots/login.webp" alt="Admin login" /></td>
<td width="33%"><img src="/screenshots/files.webp" alt="File management" /></td>
<td width="33%"><img src="/screenshots/settings.webp" alt="System settings" /></td>
</tr>
</table>
::: warning Responsible use
Use FileCodeBox only for lawful file and text sharing. Operators are responsible for deployment security, data compliance, and content moderation.
:::
+38 -38
View File
@@ -6,7 +6,7 @@ FileCodeBox 提供了丰富的配置选项。请优先通过管理面板修改
FileCodeBox 支持两种配置方式:
1. **管理面板配置**(推荐):访问 `/admin` 进入管理面板,在设置页面修改配置
1. **管理面板配置**(推荐):访问 `/#/admin` 进入管理面板,在设置页面修改配置
2. **数据库配置**:配置存储在 `data/filecodebox.db` 数据库中
::: tip 提示
@@ -22,9 +22,9 @@ FileCodeBox 支持两种配置方式:
| `name` | string | `文件快递柜 - FileCodeBox` | 站点名称,显示在页面标题和导航栏 |
| `description` | string | `开箱即用的文件快传系统` | 站点描述,用于 SEO |
| `keywords` | string | `FileCodeBox, 文件快递柜...` | 站点关键词,用于 SEO |
| `serverHost` | string | `0.0.0.0` | 服务监听地址 |
| `serverPort` | int | `12345` | 服务监听端口 |
| `serverWorkers` | int | `1` | 工作进程数;SQLite 部署建议保持单进程 |
| `server_host` | string | `0.0.0.0` | 服务监听地址 |
| `server_port` | int | `12345` | 服务监听端口 |
| `server_workers` | int | `1` | 工作进程数;SQLite 部署建议保持单进程 |
### 通知设置
@@ -33,7 +33,7 @@ FileCodeBox 支持两种配置方式:
| `notify_title` | string | `系统通知` | 通知标题 |
| `notify_content` | string | 欢迎信息 | 通知内容,支持 HTML |
| `page_explain` | string | 法律声明 | 页面底部说明文字 |
| `robotsText` | string | `User-agent: *\nDisallow: /` | robots.txt 内容 |
| `robots_text` | string | `User-agent: *\nDisallow: /` | robots.txt 内容 |
## 上传设置
@@ -41,21 +41,21 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `openUpload` | int | `1` | 是否开启上传功能(1=开启,0=关闭) |
| `uploadSize` | int | `10485760` | 单文件最大上传大小(字节),默认 10MB |
| `enableChunk` | int | `0` | 是否启用分片上传(1=启用,0=禁用) |
| `open_upload` | int | `1` | 是否开启上传功能(1=开启,0=关闭) |
| `upload_size` | int | `10485760` | 单文件最大上传大小(字节),默认 10MB |
| `enable_chunk` | int | `0` | 是否启用分片上传(1=启用,0=禁用) |
| `allowed_file_types` | list | `["*"]` | 允许上传的扩展名;`*` 表示不限制 |
::: warning 注意
`uploadSize` 的单位是字节。10MB = 10 * 1024 * 1024 = 10485760 字节
`upload_size` 的单位是字节。10MB = 10 * 1024 * 1024 = 10485760 字节
:::
### 上传频率限制
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `uploadMinute` | int | `1` | 上传限制的时间窗口(分钟) |
| `uploadCount` | int | `10` | 在时间窗口内允许的最大上传次数 |
| `upload_minute` | int | `1` | 上传限制的时间窗口(分钟) |
| `upload_count` | int | `10` | 在时间窗口内允许的最大上传次数 |
例如:默认配置表示每 1 分钟内最多允许上传 10 次。
@@ -63,7 +63,7 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | 可选的过期方式 |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | 可选的过期方式 |
| `max_save_seconds` | int | `0` | 文件最大保存时间(秒),0 表示不限制 |
过期方式说明:
@@ -79,8 +79,8 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `themesSelect` | string | `themes/2024` | 当前使用的主题 |
| `themesChoices` | list | 见下方 | 可用主题列表 |
| `themes_select` | string | `themes/2024` | 当前使用的主题 |
| `themes_choices` | list | 见下方 | 可用主题列表 |
默认可用主题:
```json
@@ -112,8 +112,8 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `admin_token` | string | 初始化页面设置 | 管理员登录密码(哈希存储) |
| `showAdminAddr` | int | `0` | 是否在首页显示管理入口(1=显示,0=隐藏) |
| `adminSessionExpire` | int | `2592000` | 管理会话有效期(秒),默认 30 天 |
| `show_admin_addr` | int | `0` | 是否在首页显示管理入口(1=显示,0=隐藏) |
| `admin_session_expire` | int | `2592000` | 管理会话有效期(秒),默认 30 天 |
::: danger 安全警告
未初始化时会自动显示初始化页面。生产环境请在服务对外开放前完成初始化。
@@ -125,9 +125,9 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `errorMinute` | int | `1` | 错误限制的时间窗口(分钟) |
| `errorCount` | int | `10` | 在时间窗口内允许的最大错误次数 |
| `trustedProxies` | list | `[]` | 可信反向代理 IP;用于安全解析客户端地址 |
| `error_minute` | int | `1` | 错误限制的时间窗口(分钟) |
| `error_count` | int | `10` | 在时间窗口内允许的最大错误次数 |
| `trusted_proxies` | list | `[]` | 可信反向代理 IP;用于安全解析客户端地址 |
此设置用于防止暴力破解提取码。
@@ -139,7 +139,7 @@ FileCodeBox 支持两种配置方式:
|--------|------|--------|------|
| `file_storage` | string | `local` | 存储后端类型 |
| `storage_path` | string | `""` | 自定义存储路径 |
| `storageLimit` | int | `0` | 总存储配额(字节),0 表示不限制 |
| `storage_limit` | int | `0` | 总存储配额(字节),0 表示不限制 |
支持的存储类型:
- `local` - 本地存储
@@ -159,11 +159,11 @@ FileCodeBox 支持两种配置方式:
```python
{
"name": "我的文件分享",
"uploadSize": 52428800, # 50MB
"uploadMinute": 5, # 5分钟
"uploadCount": 20, # 最多20次
"expireStyle": ["day", "hour", "forever"],
"showAdminAddr": 1
"upload_size": 52428800, # 50MB
"upload_minute": 5, # 5分钟
"upload_count": 20, # 最多20次
"expire_style": ["day", "hour", "forever"],
"show_admin_addr": 1
}
```
@@ -174,14 +174,14 @@ FileCodeBox 支持两种配置方式:
```python
{
"name": "公共文件快递柜",
"uploadSize": 10485760, # 10MB
"uploadMinute": 1, # 1分钟
"uploadCount": 5, # 最多5次
"errorMinute": 5, # 5分钟
"errorCount": 3, # 最多3次错误
"expireStyle": ["hour", "minute", "count"],
"upload_size": 10485760, # 10MB
"upload_minute": 1, # 1分钟
"upload_count": 5, # 最多5次
"error_minute": 5, # 5分钟
"error_count": 3, # 最多3次错误
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # 最长保存1天
"showAdminAddr": 0
"show_admin_addr": 0
}
```
@@ -192,13 +192,13 @@ FileCodeBox 支持两种配置方式:
```python
{
"name": "企业文件中转站",
"uploadSize": 1073741824, # 1GB
"enableChunk": 1, # 启用分片上传
"uploadMinute": 10, # 10分钟
"uploadCount": 100, # 最多100次
"expireStyle": ["day", "forever"],
"upload_size": 1073741824, # 1GB
"enable_chunk": 1, # 启用分片上传
"upload_minute": 10, # 10分钟
"upload_count": 100, # 最多100次
"expire_style": ["day", "forever"],
"file_storage": "s3", # 使用S3存储
"showAdminAddr": 1
"show_admin_addr": 1
}
```
+22 -2
View File
@@ -23,6 +23,10 @@ FileCodeBox 是一个简单高效的文件分享工具,支持文件临时中
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
@@ -40,8 +44,15 @@ services:
ports:
- "12345:12345"
environment:
- APP_ENV=production
- WORKERS=1
- LOG_LEVEL=info
- LOG_LEVEL=warning
- ACCESS_LOG=false
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
volumes:
fcb-data:
external: false
@@ -54,7 +65,13 @@ volumes:
| `HOST` | `0.0.0.0` | 服务监听地址 |
| `PORT` | `12345` | 服务端口 |
| `WORKERS` | `1` | 工作进程数;SQLite 部署建议保持单进程 |
| `LOG_LEVEL` | `info` | 日志级别:debug/info/warning/error |
| `APP_ENV` | `production`(Docker) | 运行环境;生产环境默认关闭 HTTP 访问日志 |
| `LOG_LEVEL` | `warning`(Docker) | 日志级别:debug/info/warning/error |
| `ACCESS_LOG` | `false`(生产环境) | 是否启用 HTTP 访问日志;调试请求时可设为 `true` |
| `TIMEOUT_KEEP_ALIVE` | `5` | HTTP Keep-Alive 空闲超时(秒);前置反向代理后可调大以减少重复握手 |
| `MALLOC_TRIM_THRESHOLD_` | `65536` | glibc 归还空闲堆的阈值(字节);降低大文件请求后的常驻内存(仅 glibc 镜像生效) |
生产环境默认只输出 warning、error 等关键日志,并限制 Docker 日志最多保留 3 个 10 MB 文件。
#### 自定义配置示例
@@ -63,8 +80,11 @@ docker run -d --restart=always \
-p 12345:12345 \
-v fcb-data:/app/data \
-e WORKERS=1 \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--name filecodebox \
--log-opt max-size=10m \
--log-opt max-file=3 \
lanol/filecodebox:latest
```
Loaded 100 of 171 files, more files were not shown because too many files have changed in this diff. Show more