Compare commits

...
65 Commits
Author SHA1 Message Date
github-actions[bot] d3f9e7902c chore(master): release 2.8.1 (#543)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 10:57:39 +00:00
vastsa 2f5eab18b1 fix: restore lint and Docker theme build 2026-10-10 18:57:15 +08:00
github-actions[bot] 6bb1ef06c1 chore(master): release 2.8.0 (#542)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-10 10:52:52 +00:00
vastsa 63e72c1455 feat: add MCP quick share and safe HTML preview 2026-10-10 18:52:28 +08:00
github-actions[bot] 648d578d53 chore(master): release 2.7.1 (#536)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-19 18:06:24 +00:00
vastsa 031d300cae fix: keep Docker as root, close s3_hostname SSRF, unbreak 404 tests
Revert gosu/non-root entrypoint so NAS and root-owned volumes keep
working. Reject @/userinfo in s3_hostname. Theme-less checkouts no
longer crash the HTML 404 handler.
2026-09-20 02:05:57 +08:00
Marrrrrrrrry 57c75a7141 全量修复/补测试+UED改进 (#535)
* fix: constant-time comparison for download tokens (security item 5)

Replace the set-membership key check in /share/download with
hmac.compare_digest over the two valid window tokens; add negative-path
tests (wrong key 403, both windows accepted, foreign-code token 403).

* fix: single source for attachment headers + guard tests (security item 2)

All six Content-Disposition construction sites across the five storage
backends now go through build_attachment_headers; the header is what
neutralizes stored XSS on the same-origin download path. Guard tests
assert every get_file_response uses the builder and no hand-built
disposition reappears.

* fix: reject internal-network endpoints for storage config (security item 7)

s3_endpoint_url/s3_hostname/webdav_url are fetched server-side; with
APP_ENV=production the write entry now enforces http(s) schemes and a
loopback/private/link-local/metadata host blacklist. Development env
stays open for local minio/webdav. Already-stored values are never
re-validated, so existing deployments are unaffected.

* fix: JSON 404 for handler-raised 404s; negative-path test batch

Starlette status-code handlers take precedence over the HTTPException
class handler, so registering the theme index as the 404 handler turned
every HTTPException(404) app-wide into a 200 HTML page (found by the new
negative-path tests). The new not_found_handler serves the theme page
only to browsers (Accept: text/html) and JSON 404 to API clients.

Negative-path batch: download count exhaustion, missing chunk session
404s, expired presign session deletion, admin update_file uniqueness/
existence.

* fix: run container as non-root app user with compat volume chown (security item 3)

Entrypoint starts as root only to fix the data-dir ownership (skipped
when nothing needs changing or when the deployer sets an explicit user),
then drops to uid 10001 via gosu. Verified in-container: uvicorn runs as
app (uid 10001), setup and share round-trip work against a root-owned
volume.

* fix: compare download tokens as bytes (non-ASCII key caused 500)

* test: lock changed-only semantics for endpoint config saves

Integration regression for the background lesson: a stored internal
endpoint must not block unrelated settings saves in production mode;
changing to an internal endpoint (URL or bare-hostname form) is 400.

* fix: resolve-based SSRF check — deny IP shorthands, hex/decimal IPs, DNS maps

Probe found the static blacklist bypassed by glibc shorthands (127.1,
10.1), decimal (2130706433) and hex IP forms, *.localhost, and
loopback-mapping DNS services. Both validators now resolve the host and
re-check every resolved address; hostname tier extracts the host part
correctly for [v6]/host:port/bare-v6 forms. 44 tests.

* test: pin 404 handler dual-branch behavior (browser theme page vs API JSON)

* fix: write back normalized endpoint values; hostname dev-gate test

* test: header-injection negative paths for attachment disposition

* fix: S3 merge buffers parts to >=5MiB; missing object 404 upfront

- S3 multipart rejects parts <5MiB (EntityTooSmall) except the last; chunk
  size is client-controlled (commonly 2-4MB) so multi-chunk merges always
  failed with 500. Merge now buffers chunks into >=5MiB parts (memory
  bound: 5MB + one chunk).
- get_file_response: a 404 head_object now raises StorageError(404)
  upfront instead of signing a doomed presigned URL and failing mid-stream
  (aligns with local backend semantics from the M2 behavior unification).

* test: S3 backend coverage via in-process moto server (13 cases)

mock_aws cannot intercept aioboto3's aiohttp stack, so tests run against
a real in-process ThreadedMotoServer endpoint. Covers roundtrips, missing
object 404, merge failure modes (missing chunk / hash mismatch -> abort,
no leftover object), cleanup scoping, presign shapes, proxy dispatch.
Also adds moto[s3,server] to the dev group and CI.

* fix: WebDAV missing object 404 upfront; connection errors map to 503

HEAD 404 was silently swallowed (signed a 200 bad stream that failed
mid-download, same family as the S3 bug), and HEAD connection errors were
also swallowed so the 503 mapping never fired. Both now surface properly;
the aiohttp session is reclaimed on pre-stream exception paths.

* test: WebDAV backend coverage via in-process WebDAV server (9 cases)

Real HTTP (HEAD/GET/PUT/DELETE/MKCOL/PROPFIND) against a minimal aiohttp
WebDAV server over a temp dir: roundtrips, missing object 404, connection
error 503, delete with empty-parent cleanup, merge failure modes, chunk
cleanup scoping.

* fix: OneDrive missing object maps to 404 upfront; OpenDAL behavior pinned

OneDrive get_file_response translated graph itemNotFound into the outer
503 catch-all; now raises StorageError(404) like local/S3/WebDAV.
OpenDAL already mapped missing objects to 404 via its outer catch —
pinned with fake-operator tests (SDK not in runtime deps, instances built
via __new__). Also tightens the S3 presign URL assertion.

* test: admin write-path coverage (batch/single delete, batch update, policy actions)

13 negative-path cases for the previously untested data-modifying admin
endpoints: mixed-id aggregation with duplicates and missing records,
empty-list rejections, clear_expired_at permanence semantics, policy
action boundaries (zero limit 400, unknown action 400, missing 404),
and the DoesNotExist->404 mapping pinned for single delete.

* test: admin read-path and view-preset CRUD coverage (22 cases)

Detail/metadata/preview/admin-download/activities/local-lists+delete/
verify: missing-record 404s, note/tag truncation limits, preview max_chars
truncation, activity filtering and the 80-event clamp, local path
traversal rejection. Presets: update-vs-create by id, name truncation,
filter normalization clamps, 24-preset cap, delete-missing 404.

* refactor: move auth primitives to apps/base/auth, kill base->admin dep

JWT create/verify, bearer extraction and the share-upload gate move to
apps.base.auth (consumed by both surfaces); admin.dependencies keeps
admin session gating and service providers, re-exporting the primitives
for import-path compatibility. base.views no longer imports from
apps.admin — the last cross-app reverse dependency is gone.

* refactor: split admin services god-module (ConfigService/LocalFile out)

apps/admin/services.py (1697 lines) becomes FileService (~1200 lines) +
a compatibility facade re-exporting ConfigService, LocalFileService,
LocalFileClass and keyvalue_write_lock from their new homes
(config_service.py holds the D5 KeyValue lock; one-way imports, no
cycle). test_admin_security patch targets follow ConfigService to its
new module. Behavior unchanged; 247 tests green.

* fix: size-less uploads no longer crash (upstream seek bug)

validate_file_size's size-None branch called UploadFile.seek(0, 2),
which raises TypeError (UploadFile.seek takes one arg); the underlying
SpooledTemporaryFile.seek is sync, so awaiting it also fails. Use the
underlying file object with sync seek(0, SEEK_END). Surface discovered
by the mypy pilot run; regression test covers the size-None branch.

* chore: bump direct deps (patch/minor)

fastapi 0.139.2->0.141.1, pydantic 2.12.5->2.13.5, uvicorn 0.51.0->0.53.0,
aiohttp 3.14.2->3.14.3; lockfile regenerated. tortoise-orm 0.x->1.x major
deliberately deferred pending API-change review.

* ci: gradual mypy adoption via baseline ratchet

scripts/mypy_ratchet.py compares mypy output against
scripts/mypy-baseline.txt (line numbers stripped for edit stability):
new type errors fail CI, fixes are folded back via --regenerate.
Baseline starts at 30 known errors; the seek bug found in the pilot was
fixed before baselining. mypy joins the dev group and CI.

* fix: migrate WebDAV auth off aiohttp.BasicAuth (deprecated in 4.0)

Dependency-upgrade probe on aiohttp 3.14.3 surfaced the BasicAuth
deprecation; switch to aiohttp.encode_basic_auth() headers so the
aiohttp 4.0 upgrade does not break the WebDAV backend.

* chore: drop dead code (WebDAV _instance stub, unused LocalFileClass.write)

* test: file-validation negative paths (13 cases)

Magic-bytes spoofing (text-as-png, exe-as-pdf, mp4/webp box detection),
whitelist rule semantics (star/image-wildcard/content mismatch), chunk-0
header validation parity, and end-to-end UploadFile rejection.

* test: migration runner coverage (5 cases)

Full chain executes 001-007 in filename order and registers; rerun is
idempotent; pre-registered entries are skipped (DDL not re-executed);
the resulting schema matches deployment; a failing migration propagates
and is not registered. Uses a fresh :memory: DB without
generate_schemas so the migrations themselves build the schema (the
real deployment path).

* refactor: split core/storage.py into a package (per-backend modules)

core/storage.py (1400+ lines, six backends) becomes core/storage/ with
_base.py (data contracts, interface, shared header builder), and
local/s3/onedrive/opendal/webdav modules. The package __init__
re-exports the full historical public surface; test patch targets move
to the precise backend modules (core.storage.local.data_root etc.) and
the attachment guard scans the package. Import-only change; 264 tests
green. Storage-layer DI deferred: current tests construct instances
against patched settings, so the added surface isn't justified yet.

* refactor: TypedDict for IPRateLimit records

The rate-limit ledger mixed int and datetime under a loose Union typing,
producing 5 mypy noise errors. A TypedDict gives precise per-key types;
setdefault replaces the get-then-assign pattern. One mypy baseline error
family resolved; 264 tests green.

* chore: tighten mypy baseline after IPRateLimit TypedDict

* refactor: resolve all 26 baseline mypy errors; precise test assertions

Per-error treatment (no bulk script): honest annotations for
StoredDownload (Path/bytes/Callable), KeyValue.value (Any for JSONField),
dict[str, Any] for form/config dicts; signature split in create_token;
str() normalization in config int(); platform/type-ignore only where the
checker cannot follow (msvcrt, FastAPI Request injection). Baseline file
and ratchet compare now redundant at zero errors but kept as the guard
against regressions.

Tests: assertions tightened to status_code == 403 with HTTPException
raises; _create_share explicit parameters replace **extra passthrough;
seek-fix pointer-reset assertion persisted.
2026-09-20 03:01:08 +09:00
github-actions[bot] 82555c08ed chore(master): release 2.7.0 (#533)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-16 10:31:45 +00:00
vastsa 93a30a257c feat: mint extract codes from data/local without re-upload
Admin can browse NAS files under data/local including subfolders and share
them in place. Shares use file_path=local-ref, skip storage quota, and never
delete the original file on expire.
2026-09-16 18:31:00 +08:00
github-actions[bot] 3c135eb29f chore(master): release 2.6.1 (#529)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-14 13:53:56 +00:00
Lan 0d82fbe26a Merge pull request #528 from vastsa/fix/post-514-upgrade-regressions
fix: 存量 background 值不再阻塞后台设置保存(#514 升级回归)
2026-09-14 22:53:32 +09:00
vastsa eb2db8f507 fix: don't let a legacy background value block settings saves
#514 began validating `background` as an http(s) URL against the *merged*
config, so a value stored by an older release (a relative path, or one
containing a space/parenthesis — all legal before) made every subsequent
settings save return 400, including saves that never touched `background`.
Existing deployments could not change any setting at all.

Validate only values that actually change: an untouched legacy value is
kept as-is (it is still html-escaped on render), while any edit must pass
validation. Regression tests cover both directions and fail without this
change.

Also treat OverflowError from hashlib.scrypt as a non-matching hash so a
hand-crafted n/r/p cannot disturb the login path on runtimes that raise it.
2026-09-14 21:52:23 +08:00
github-actions[bot] 18d344cc9b chore(master): release 2.6.0 (#515)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-14 13:47:00 +00:00
Lan 8215f91edf Merge pull request #514 from Marrrrrrrrry/fix/m0-stop-bleeding
工程化加固、安全与性能修复、API 契约统一 snake_case(含 CI 测试流水线)
2026-09-14 22:46:18 +09:00
Marrrrrrrrry 775f3fc4ce ci: add test pipeline and API contract guard
- ci.yml: ruff + pytest on push/PR, installing from the hashed lockfile
  with --require-hashes, plus a lockfile-vs-requirements consistency
  check (dependabot cannot regenerate the lockfile; drift was silent).
- test_api_contract.py: deep-scans public config, dashboard, admin file
  list and share metadata responses for camelCase keys - the regression
  net for the D7 public-config leak found during review.
2026-09-11 04:24:47 +08:00
Marrrrrrrrry 13f572a86a test: follow public-config snake_case rename in admin-address test 2026-09-11 04:11:47 +08:00
Marrrrrrrrry 800bac0128 fix: snake_case the public config contract (D7 follow-up)
build_public_config/build_public_meta served a mixed camel/snake
contract (allowedFileTypes, meta.features.*, api.legacyConfig,
limits.uploadWindowMinutes). Rename to snake_case; frontend reads
allowed_file_types only.
2026-09-11 04:09:25 +08:00
Marrrrrrrrry 8972344235 docs: rename legacy camelCase config keys to snake_case in guides
Sync zh/en guide docs with migration 007 key names (upload_size,
open_upload, enable_chunk, expire_style, error_count, etc.) so existing
deployments following the docs use the current schema.
2026-09-11 03:38:52 +08:00
Marrrrrrrrry 6e7580b8dc docs: refresh API reference - real version, snake_case fields, EN presign page
API version 2.1.0 -> 2.5.6; response examples use the single message
field; dashboard/list examples and params match current snake_case
contract; drop stale save_path from presign init response; add English
presign-upload page and sidebar entry; uploadSize -> upload_size.
2026-09-11 03:37:24 +08:00
Marrrrrrrrry c7b5eabe7c refactor: single canonical snake_case keys across admin/share APIs (D7)
Drop the message/msg envelope twin, all camel+snake dual-field response
pairs (file list/detail/policy/storage, dashboard counters, batch op
counts, status insights, presets, metadata), and camel request fields
(clearExpiredAt, downloadLimit, sortBy/sortOrder, maxChars). Stored
KeyValue JSON (metadata, presets, activities) keeps legacy-key read
fallbacks so existing deployments normalize transparently.
2026-09-11 03:10:04 +08:00
Marrrrrrrrry bb40f40a18 refactor: hoist all five upload-entry workflows into FileUploadService (2.3)
share_text/share_file/complete_upload/presign proxy/presign confirm now
delegate quota reservation, storage writes, record creation, rollback and
chunk cleanup to service methods; handlers keep auth, request validation,
rate limiting and response wrapping. validate_file_size and
PRESIGN_SESSION_EXPIRES move to services as shared single sources.
2026-09-11 02:56:13 +08:00
Marrrrrrrrry a49727b63d refactor: move admin update_file orchestration into FileService (2.3) 2026-09-11 02:56:02 +08:00
Marrrrrrrrry 0038636a2a fix: serialize KeyValue JSON read-modify-write sections behind a process lock (D5)
settings/activities/view-presets are whole-blob KeyValue rows; concurrent
admin operations could silently drop each other's writes (last-writer-wins).
Single-worker deployment assumption documented; lock is process-local.
2026-09-09 19:09:14 +08:00
Marrrrrrrrry 3c93718d4b fix: FileCodes.expired_count defaults to -1 (time-based) instead of instantly-expired 0 (D4) 2026-09-09 19:05:26 +08:00
Marrrrrrrrry baecebd81e perf: dashboard counters and storage quota use SQL aggregation (D3)
- quota.get_storage_usage (runs on every upload) now aggregates via Sum
  instead of loading every row
- dashboard simple counters (total/size/expired/active/text/chunked/used,
  yesterday/today, top suffixes, recent 8) moved to SQL; the per-row health
  rules engine keeps its single pass (documented)
2026-09-09 19:04:12 +08:00
Marrrrrrrrry bb95017aa3 fix: save_file accepts plain binary streams; repair admin local file share (D2)
- storage contract drops UploadFile in favor of BinaryIO (+content_type param),
  completing the framework decoupling of the storage layer
- LocalFileClass.read() leaked an open fd and its handle was passed where
  save_file expected UploadFile — /admin/local/share 500ed on every call;
  now reads bytes via with-block and shares correctly (regression tests added)
- OneDrive save_file additionally had save_path called as a function — fixed
2026-09-09 18:53:31 +08:00
Marrrrrrrrry 61d72ab6cc perf: cache settings refresh behind a 2s TTL, force refresh on write paths (D6) 2026-09-09 18:47:40 +08:00
Marrrrrrrrry ec424dd909 perf: make is_config_initialized cheap — no scrypt on the per-request probe (D1) 2026-09-09 18:33:27 +08:00
Marrrrrrrrry 5121db6edc feat: rename all config keys to snake_case with DB migration 007
- 20 camelCase keys (uploadSize, expireStyle, openUpload, ...) renamed across
  DEFAULT_CONFIG, settings accessors, setup form fields, and ConfigService
- migrations_007 renames keys inside the existing settings KeyValue row,
  preserving all values for existing deployments
- schema guard meta-test keeps future keys declared
2026-09-09 18:09:19 +08:00
Marrrrrrrrry c5d525df08 feat: declare missing config keys (opendal_scheme, chunk_expire_hours) and add config schema guard
- opendal_scheme was read directly by the OpenDAL backend but never declared
  in DEFAULT_CONFIG: selecting that backend crashed with AttributeError
- chunk_expire_hours existed only as getattr fallbacks in two modules
- refresh_settings now warns about unknown user_config keys
- meta-test scans the codebase so any future settings.X reference without a
  DEFAULT_CONFIG declaration fails the suite
2026-09-09 17:43:28 +08:00
Marrrrrrrrry 61724cbf32 chore: warn at startup when multi-worker deployment weakens in-process rate limiting 2026-09-09 17:37:06 +08:00
Marrrrrrrrry 6587ec86be refactor: extract setup wizard and public pages from main.py (945 -> 148 lines)
- apps/base/setup_wizard.py: form parsing rules, setup options, and the
  two admin-facing HTML pages
- apps/base/pages.py: setup/assets/index/robots/public-config routes and
  theme resolution (404 handler registered on the app in main.py)
- tests updated to import from the new modules
2026-09-09 17:25:42 +08:00
Marrrrrrrrry 832266101e refactor: storage returns StoredDownload data; view layer builds starlette responses 2026-09-09 17:13:34 +08:00
Marrrrrrrrry ad3d319362 refactor: storage raises framework-neutral StorageError rendered identically by app handler 2026-09-09 17:07:43 +08:00
Marrrrrrrrry 892fb24ffc refactor: storage layer accepts plain StoredFile refs and caller-fetched chunk records (core/ is now ORM-free) 2026-09-09 17:05:38 +08:00
Marrrrrrrrry 3396644da3 refactor: move config and tasks modules from core to apps/base (core no longer imports apps) 2026-09-09 17:00:42 +08:00
Marrrrrrrrry 81d33b70de refactor: extract shared chunk-record fetch and hash verification into storage base class 2026-09-09 16:41:29 +08:00
Marrrrrrrrry a75936d85a refactor: use rollback_saved_file helper in presign proxy/confirm workflows 2026-09-09 16:39:42 +08:00
Marrrrrrrrry 357ab2fb6f refactor: extract FileUploadService into apps/base/services.py with shared rollback helper 2026-09-09 16:38:37 +08:00
Marrrrrrrrry 443cfe7472 feat: upgrade password hashing to memory-hard scrypt with transparent rehash
- new hashes use scrypt (n=2^14, r=8, p=1); sha256 and plaintext formats
  keep verifying unchanged, so existing deployments are unaffected
- on successful login a legacy-stored admin password is rehashed to scrypt
  in the settings KeyValue row
- add password_needs_rehash() and extend scheme detection; stdlib only, no
  new dependency
2026-09-09 16:33:07 +08:00
Marrrrrrrrry bde1fee17e docs: document lockfile regeneration flow in dependabot config 2026-09-09 16:28:36 +08:00
Marrrrrrrrry 964cba3636 chore: install dependencies from hashed lockfile in Docker build 2026-09-09 16:28:14 +08:00
Marrrrrrrrry 4904695955 chore: add hashed universal lockfile (uv pip compile --generate-hashes) 2026-09-09 16:27:38 +08:00
Marrrrrrrrry 4cab625d5a fix: validate background config as http(s) URL to close CSS quote-breakout in 2023 theme 2026-09-09 16:21:35 +08:00
Marrrrrrrrry a8c4072497 test: cover presigned upload journey (proxy mode auto-confirm, status, cancel, size limit) 2026-09-09 16:17:41 +08:00
Marrrrrrrrry 2bc278a58e test: cover merge_chunks failure modes and temp-file hygiene 2026-09-09 13:51:36 +08:00
Marrrrrrrrry 3f58e42be1 test: cover the three data-deleting cleanup tasks (expired files, stale chunks, presign sessions) 2026-09-09 13:42:11 +08:00
Marrrrrrrrry 625e50ca95 test: add end-to-end ASGI journey tests for share, chunk, and auth flows 2026-09-09 13:37:33 +08:00
Marrrrrrrrry 177a212c0f test: add shared test helpers and ASGI fixtures, dedupe 10x boilerplate
- tests/helpers.py: SettingsOverrideMixin + init_memory_db()/close_db()
- tests/conftest.py: httpx ASGI client fixtures (db, client, initialized_client)
- enable pytest asyncio_mode=auto
- replace 6 duplicated SettingsOverrideMixin copies and 4 duplicated
  Tortoise.init blocks across the legacy unittest suite
2026-09-08 17:47:21 +08:00
Marrrrrrrrry d701c6e00e chore: engineering hardening - test tooling, lint, security fix, and cleanup
- add pyproject.toml with pytest config and dev dependency group (PEP 735)
- add ruff (core correctness rules) + pre-commit hook
- fix stored XSS: html.escape site config before index template injection,
  with regression tests
- log previously swallowed cleanup exceptions in upload rollback paths
- consolidate three duplicate path generators into build_file_path
  (also aligns chunk/presign date dirs to get_now() instead of local time)
- drop redundant ip_limit sync in load_config (refresh_settings covers all limiters)
- remove dead code (calculate_file_hash, commented-out block), two orphan
  guide docs, and stale .gitignore entries
- add dependabot for pip, docker, and github-actions
- bump starlette 1.3.1 -> 1.6.0 (no breaking API changes; verified with the
  full test suite plus an ASGI smoke test incl. Range downloads)
- skip theme asset tests when themes/ is not built locally
2026-09-07 23:43:50 +08:00
github-actions[bot] 6c76aabe1e chore(master): release 2.5.6 (#509)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-01 16:08:44 +00:00
vastsa e08555a6e8 Merge fix/prod-log-noise-20260901 2026-09-01 23:56:25 +08:00
vastsa 5537f56b88 fix(logging): reduce production log noise 2026-09-01 23:56:08 +08:00
github-actions[bot] 00f1f1d8da chore(master): release 2.5.5 (#507)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-29 04:44:30 +00:00
Lan 5a7195c5c7 Merge pull request #506 from vastsa/fix/pr505-hardened
fix: harden chunk upload filename metadata
2026-08-29 12:44:06 +08:00
vastsa da1dd06655 fix(chunk-upload): persist sanitized filenames 2026-08-29 12:38:46 +08:00
vastsa d510aa7f57 Merge remote-tracking branch 'contributor/pr505' into fix/pr505-hardened 2026-08-29 12:32:00 +08:00
zhou-kkk cebac32145 fix: sanitize chunk upload file_name to prevent path traversal 2026-08-29 12:13:39 +08:00
vastsa f76d1e9841 docs: refresh readme and docs screenshots
Replace legacy PNG assets with new WebP banners and UI captures, and point Chinese/English readme plus docs pages at the updated images.
2026-07-24 16:34:24 +08:00
github-actions[bot] 287b647da1 chore(master): release 2.5.4 (#501)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-23 14:35:42 +00:00
Lan fbcd4a5b2b Merge pull request #500 from vastsa/fix/docker-run-as-root
fix: run Docker as root for volume compatibility
2026-07-23 22:35:20 +08:00
vastsa fc0cbeb4f0 fix: run Docker image as root for data volume compatibility
Revert the non-root container user so existing deployments with
root-owned data volumes keep working without a manual chown step.
2026-07-23 22:35:00 +08:00
github-actions[bot] 24bf6e1a81 chore(master): release 2.5.3 (#499)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-23 14:23:38 +00:00
Lan 0ee2f046dd Merge pull request #498 from vastsa/fix/security-hardening-gaps
fix: magic-byte file checks, admin login rate limit, non-root Docker
2026-07-23 22:23:11 +08:00
vastsa eda0a3127a fix: add magic-byte checks, login rate limits, and non-root Docker
Close the remaining security gaps: validate upload content against
file signatures, rate-limit admin login failures with configurable
thresholds, and run the container as uid 1000 instead of root.
2026-07-23 22:23:04 +08:00
156 changed files with 11207 additions and 4028 deletions

No files matched your search

+18
View File
@@ -0,0 +1,18 @@
version: 2
updates:
# Runtime deps. requirements.txt pins direct deps; requirements.lock.txt is
# the hashed, transitive-resolved lockfile. Dependabot raises version PRs
# against requirements.txt; regenerate the lockfile in the same PR
# (uv pip compile requirements.txt --generate-hashes --universal -o requirements.lock.txt).
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
Binary file not shown.

Before

Width:  |  Height:  |  Size: 571 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 557 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 205 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 679 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 684 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 411 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 431 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 579 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 201 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 254 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 514 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 674 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 678 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 219 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 192 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 233 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 54 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 171 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

+67
View File
@@ -0,0 +1,67 @@
name: CI
on:
push:
branches:
- master
- dev
pull_request:
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
test:
name: Lint and test
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
# The hashed lockfile is the deploy artifact; install from it with
# --require-hashes so CI fails on lockfile drift, exactly like the
# Docker build does.
pip install --require-hashes -r requirements.lock.txt
pip install pytest pytest-asyncio httpx 'moto[s3,server]' mypy
- name: Ruff
run: pipx run ruff==0.16.6 check .
- name: Mypy ratchet (new type errors are rejected; fix and run
scripts/mypy_ratchet.py --regenerate to tighten the baseline)
run: python scripts/mypy_ratchet.py
- name: Verify lockfile matches requirements.txt
# Dependabot bumps requirements.txt but cannot regenerate the hashed
# lockfile; without this check a stale lockfile would silently keep
# the Docker build on old versions.
run: |
python - <<'PY'
import re, sys
pins = dict(re.findall(r'^([\w-]+)==([\w.]+)$', open('requirements.txt').read(), re.M))
lock = open('requirements.lock.txt').read()
stale = []
for name, ver in pins.items():
m = re.search(rf'(?mi)^{re.escape(name)}==([\w.]+)\b', lock)
if m is None or m.group(1) != ver:
stale.append((name, ver, m.group(1) if m else 'ABSENT'))
for name, req_ver, lock_ver in stale:
print(f'STALE LOCKFILE: {name} requirements.txt={req_ver} lock={lock_ver}')
sys.exit(1 if stale else 0)
PY
- name: Run tests
run: pytest -q
+2 -7
View File
@@ -80,8 +80,8 @@ docs/_build/
.pybuilder/
target/
*.db
./filecodebox.db-shm
./filecodebox.db-wal
*.db-shm
*.db-wal
# Jupyter Notebook
.ipynb_checkpoints
@@ -147,13 +147,8 @@ cython_debug/
# Project
.vscode
.DS_Store
for_test.py
.html
/evaluate/temp.py
/evaluation/back.json
data/.env
.backup/
/cloc-1.64.exe
# Ignore node_modules
node_modules/
+8
View File
@@ -0,0 +1,8 @@
# First-time setup: pre-commit install
# Behind a proxy: HTTPS_PROXY=http://<proxy-host>:<port> pre-commit run --all-files
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.6
hooks:
- id: ruff-check
args: [--fix]
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "2.5.2"
".": "2.8.1"
}
+8 -4
View File
@@ -54,22 +54,25 @@ RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
echo 'Asia/Shanghai' > /etc/timezone
# 从构建阶段复制编译好的前端主题
COPY --from=frontend-builder /build/fronted-2024/dist ./themes/2024
COPY --from=frontend-builder /build/FileCodeBox/themes/2024 ./themes/2024
COPY --from=frontend-builder /build/fronted-2023/dist ./themes/2023
# 安装系统安全更新 + Python 依赖
# 依赖从带哈希的锁定文件安装(--require-hashes),保证构建可复现、防供应链篡改。
# 清理 apt 缓存,降低镜像噪音与扫描面
RUN apt-get update \
&& apt-get upgrade -y --no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir -r requirements.txt \
&& pip install --no-cache-dir --require-hashes -r requirements.lock.txt \
&& pip cache purge || true
# 环境变量配置
ENV HOST="0.0.0.0" \
PORT=12345 \
WORKERS=1 \
LOG_LEVEL="info" \
APP_ENV="production" \
LOG_LEVEL="warning" \
ACCESS_LOG="false" \
FORWARDED_ALLOW_IPS=""
EXPOSE 12345
@@ -77,4 +80,5 @@ EXPOSE 12345
# 生产环境启动命令
# FORWARDED_ALLOW_IPS 默认为空:仅信任直连 IP,避免任意客户端伪造 X-Forwarded-*。
# 若前面有反向代理,请显式设置为代理网段,例如 "10.0.0.0/8,172.16.0.0/12"。
CMD ["sh", "-c", "exec uvicorn main:app --host \"$HOST\" --port \"$PORT\" --workers \"$WORKERS\" --log-level \"$LOG_LEVEL\" --proxy-headers --forwarded-allow-ips \"${FORWARDED_ALLOW_IPS:-}\""]
# 容器以 root 运行:兼容存量 root 属主的数据卷,以及 NAS/只读 bind mount。
CMD ["sh", "-c", "access_log_arg=--no-access-log; if [ \"${APP_ENV:-development}\" != \"production\" ] || [ \"${ACCESS_LOG:-false}\" = \"true\" ]; then access_log_arg=--access-log; fi; exec uvicorn main:app --host \"$HOST\" --port \"$PORT\" --workers \"$WORKERS\" --log-level \"$LOG_LEVEL\" \"$access_log_arg\" --proxy-headers --forwarded-allow-ips \"${FORWARDED_ALLOW_IPS:-}\""]
+1 -1
View File
@@ -1 +1 @@
2.5.2
2.8.1
+160
View File
@@ -0,0 +1,160 @@
"""System config write path (ConfigService) + the KeyValue JSON lock.
keyvalue_write_lock lives here because ConfigService.update_config is its
primary consumer; FileService imports it (one-way, no cycle).
"""
import asyncio
from fastapi import HTTPException
from core.settings import (
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
)
from apps.base.config import refresh_settings
from core.security import (
INTERNAL_CONFIG_KEYS,
OUTBOUND_ENDPOINT_CONFIG_KEYS,
generate_jwt_secret,
validate_outbound_endpoint,
validate_outbound_hostname,
)
from apps.base.models import KeyValue
from core.utils import hash_password, is_password_hashed, validate_background_url
# KeyValue 里的 settings/activities/presets 都是整块 JSON 读-改-写;
# 进程内写锁串行化这三个写路径,避免并发管理操作互相覆盖(last-writer-wins)。
# 多进程部署下锁不跨进程——文档已锁定单 worker 部署。
keyvalue_write_lock = asyncio.Lock()
class ConfigService:
INT_FIELDS = {
"admin_session_expire",
"enable_chunk",
"error_count",
"error_minute",
"login_count",
"login_minute",
"max_save_seconds",
"onedrive_proxy",
"open_upload",
"mcp_require_token",
"port",
"s3_proxy",
"server_port",
"server_workers",
"show_admin_addr",
"storage_limit",
"upload_count",
"upload_minute",
"upload_size",
"webdav_proxy",
}
FLOAT_FIELDS = {"opacity"}
def get_config(self):
config = dict(settings.items())
config["admin_token"] = ""
for key in INTERNAL_CONFIG_KEYS:
config.pop(key, None)
return config
async def update_config(self, data: dict):
current_config = dict(settings.items())
next_config = dict(current_config)
update_data = {
key: value
for key, value in data.items()
if key in settings.default_config and key not in INTERNAL_CONFIG_KEYS
}
admin_token = update_data.get("admin_token")
admin_password_changed = False
if admin_token is None or admin_token == "":
update_data.pop("admin_token", None)
elif not is_password_hashed(admin_token):
update_data["admin_token"] = hash_password(admin_token)
admin_password_changed = True
else:
admin_password_changed = True
for key, value in update_data.items():
if value == "" and key in self.INT_FIELDS | self.FLOAT_FIELDS:
continue
try:
if key in self.INT_FIELDS:
next_config[key] = int(value)
elif key in self.FLOAT_FIELDS:
next_config[key] = float(value)
else:
next_config[key] = value
except (TypeError, ValueError):
raise HTTPException(status_code=400, detail=f"{key} 配置值格式错误")
try:
session_expire = int(str(next_config.get("admin_session_expire")))
except (TypeError, ValueError):
raise HTTPException(
status_code=400,
detail="admin_session_expire 配置值格式错误",
)
if (
not ADMIN_SESSION_EXPIRE_MIN <= session_expire <= ADMIN_SESSION_EXPIRE_MAX
or session_expire % ADMIN_SESSION_EXPIRE_MIN != 0
):
raise HTTPException(
status_code=400,
detail="admin_session_expire 必须是 1 到 365 个整天",
)
next_config["admin_session_expire"] = session_expire
if int(next_config.get("storage_limit", 0)) < 0:
raise HTTPException(
status_code=400,
detail="storage_limit 不能小于 0",
)
# 只校验"发生变化"的值:升级前存入的旧格式 background(相对路径、含空格
# 或括号)在旧版本是合法的,若每次保存都重新校验,存量部署会连无关设置项
# 都保存不了(一律 400)。渲染侧仍然 html 转义,而任何修改都必须通过校验。
current_background = str(settings.background or "")
candidate_background = str(next_config.get("background") or "")
if candidate_background != current_background:
try:
validate_background_url(candidate_background)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
# 只校验"发生变化"的值:企业内网 minio/webdav 是正当场景,存量部署历史
# 合法写入的内网 endpoint 若每次保存都重新校验,会连无关设置都保存不了
# (background 曾有同款回归,上游 #528 修复过——本处沿用同一语义)。
# s3_hostname 是裸主机名(存储层按 https://{hostname} 拼接),单独分档校验。
for endpoint_key in OUTBOUND_ENDPOINT_CONFIG_KEYS:
if endpoint_key not in next_config:
continue
candidate = str(next_config[endpoint_key] or "")
current = str(getattr(settings, endpoint_key, "") or "")
if candidate == current:
continue
validator = (
validate_outbound_hostname
if endpoint_key == "s3_hostname"
else validate_outbound_endpoint
)
try:
# 写回规范化值(validator 去除首尾空白):校验通过但入库脏值
# 会让存储层在连接期才报错,应在校验点归一。
next_config[endpoint_key] = validator(candidate)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
if admin_password_changed:
next_config["jwt_secret"] = generate_jwt_secret()
async with keyvalue_write_lock:
await KeyValue.update_or_create(key="settings", defaults={"value": next_config})
await refresh_settings(force=True)
+32 -139
View File
@@ -2,127 +2,40 @@
# @Author : Lan
# @File : depends.py
# @Software: PyCharm
from fastapi import Header, HTTPException, Depends
from fastapi import Header
from fastapi.requests import Request
import base64
import hmac
import json
import time
from core.settings import (
ADMIN_SESSION_EXPIRE_DEFAULT,
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
from apps.admin.services import ConfigService, FileService, LocalFileService
from apps.base.auth import (
_extract_bearer_token,
_get_jwt_secret,
_require_admin_payload,
create_token,
get_admin_session_expire_seconds,
share_required_login,
verify_token,
)
from apps.admin.services import FileService, ConfigService, LocalFileService
# 认证原语(JWT 创建/校验、Bearer 提取、分享上传门控)已下沉 apps.base.auth
# ——base 分享面与本模块共同消费,留在 base 避免 base→admin 反向依赖。
# 此处 re-export 保持既有 import 路径兼容(tests/admin.views 等)。
def _get_jwt_secret() -> bytes:
secret = getattr(settings, "jwt_secret", "")
if not secret:
raise RuntimeError("JWT签名密钥未初始化")
return secret.encode()
def get_admin_session_expire_seconds() -> int:
try:
expires_in = int(
getattr(settings, "adminSessionExpire", ADMIN_SESSION_EXPIRE_DEFAULT)
)
except (TypeError, ValueError):
return ADMIN_SESSION_EXPIRE_DEFAULT
if (
not ADMIN_SESSION_EXPIRE_MIN <= expires_in <= ADMIN_SESSION_EXPIRE_MAX
or expires_in % ADMIN_SESSION_EXPIRE_MIN != 0
):
return ADMIN_SESSION_EXPIRE_DEFAULT
return expires_in
def create_token(data: dict, expires_in: int | None = None) -> str:
"""
创建JWT token
:param data: 数据负载
:param expires_in: 过期时间(秒)
"""
token_lifetime = (
get_admin_session_expire_seconds() if expires_in is None else expires_in
)
header = base64.urlsafe_b64encode(
json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode()
).decode().rstrip("=")
payload = base64.urlsafe_b64encode(
json.dumps(
{**data, "exp": int(time.time()) + token_lifetime},
separators=(",", ":"),
).encode()
).decode().rstrip("=")
signature = hmac.new(
_get_jwt_secret(), f"{header}.{payload}".encode(), "sha256"
).digest()
signature = base64.urlsafe_b64encode(signature).decode().rstrip("=")
return f"{header}.{payload}.{signature}"
def verify_token(token: str) -> dict:
"""
验证JWT token
:param token: JWT token
:return: 解码后的数据
"""
try:
header_b64, payload_b64, signature_b64 = token.split(".")
# 验证签名
expected_signature = hmac.new(
_get_jwt_secret(),
f"{header_b64}.{payload_b64}".encode(),
"sha256",
).digest()
expected_signature_b64 = (
base64.urlsafe_b64encode(expected_signature).decode().rstrip("=")
)
if not hmac.compare_digest(signature_b64, expected_signature_b64):
raise ValueError("无效的签名")
# 解码payload(兼容历史标准 base64 与 urlsafe base64)
padded = payload_b64 + "=" * (-len(payload_b64) % 4)
try:
payload_bytes = base64.urlsafe_b64decode(padded)
except Exception:
payload_bytes = base64.b64decode(padded)
payload = json.loads(payload_bytes)
# 检查是否过期
if payload.get("exp", 0) < time.time():
raise ValueError("token已过期")
return payload
except Exception as e:
raise ValueError(f"token验证失败: {str(e)}")
def _extract_bearer_token(authorization: str) -> str:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
token = authorization.split(" ", 1)[1].strip()
if not token:
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return token
def _require_admin_payload(authorization: str) -> dict:
token = _extract_bearer_token(authorization)
try:
payload = verify_token(token)
except ValueError as e:
raise HTTPException(status_code=401, detail=str(e))
if not payload.get("is_admin", False):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return payload
__all__ = [
"_extract_bearer_token",
"_get_jwt_secret",
"_require_admin_payload",
"ADMIN_PUBLIC_ENDPOINTS",
"admin_required",
"create_token",
"get_admin_session",
"get_admin_session_expire_seconds",
"get_config_service",
"get_file_service",
"get_local_file_service",
"share_required_login",
"verify_token",
]
def get_admin_session(authorization: str = Header(default=None)) -> dict:
@@ -141,7 +54,8 @@ ADMIN_PUBLIC_ENDPOINTS = {("POST", "/admin/login")}
async def admin_required(
authorization: str = Header(default=None), request: Request = None
authorization: str = Header(default=None),
request: Request = None, # type: ignore[assignment] # FastAPI 运行时注入 Request;可选语义由下方 if request 判断
):
"""
验证管理员权限
@@ -151,27 +65,6 @@ async def admin_required(
return _require_admin_payload(authorization)
async def share_required_login(authorization: str = Header(default=None)):
"""
验证分享上传权限
当settings.openUpload为False时,要求用户必须登录并具有管理员权限
当settings.openUpload为True时,允许游客上传
:param authorization: 认证头信息
:param request: 请求对象
:return: 验证结果
"""
if not settings.openUpload:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=403, detail="本站未开启游客上传,如需上传请先登录后台"
)
_require_admin_payload(authorization)
return True
async def get_file_service():
return FileService()
+98
View File
@@ -0,0 +1,98 @@
"""Admin-side local (NAS) file browsing and deletion."""
from pathlib import Path
from typing import Any
from fastapi import HTTPException
from apps.base.local_share import (
MAX_LIST_ENTRIES,
format_local_ctime,
get_local_root,
normalize_local_relpath,
resolve_under_local,
)
class LocalFileService:
async def list_files(self, path: str = ""):
relpath = normalize_local_relpath(path, allow_empty=True)
directory = resolve_under_local(relpath)
if not directory.exists() or not directory.is_dir():
raise HTTPException(status_code=404, detail="目录不存在")
root = get_local_root()
items: list[dict[str, Any]] = []
try:
children = list(directory.iterdir())
except OSError as exc:
raise HTTPException(status_code=500, detail="无法读取目录") from exc
children.sort(key=lambda p: (not p.is_dir(), p.name.lower()))
truncated = False
for child in children:
if len(items) >= MAX_LIST_ENTRIES:
truncated = True
break
try:
resolved = child.resolve()
resolved.relative_to(root)
except (OSError, ValueError):
continue
if not resolved.is_file() and not resolved.is_dir():
continue
child_rel = child.name if not relpath else f"{relpath}/{child.name}"
is_dir = resolved.is_dir()
items.append(
{
"file": child.name,
"name": child.name,
"path": child_rel,
"type": "dir" if is_dir else "file",
"ctime": format_local_ctime(resolved),
"size": None if is_dir else resolved.stat().st_size,
}
)
parent = ""
if relpath:
parent_path = Path(relpath).parent.as_posix()
parent = "" if parent_path == "." else parent_path
return {
"path": relpath,
"parent": parent,
"truncated": truncated,
"items": items,
}
async def delete_file(self, filename: str):
file = LocalFileClass(filename)
if await file.exists():
await file.delete()
return "删除成功"
raise HTTPException(status_code=404, detail="文件不存在")
class LocalFileClass:
def __init__(self, file):
relpath = normalize_local_relpath(file)
self.file = relpath
self.name = Path(relpath).name
self.path = resolve_under_local(relpath)
if self.path.is_file():
self.ctime = format_local_ctime(self.path)
self.size = self.path.stat().st_size
else:
self.ctime = None
self.size = None
async def read(self) -> bytes:
with open(self.path, "rb") as fh:
return fh.read()
async def delete(self):
if not self.path.is_file():
raise HTTPException(status_code=404, detail="文件不存在")
self.path.unlink()
async def exists(self):
return self.path.is_file()
-3
View File
@@ -39,21 +39,18 @@ class BatchUpdateFileData(BaseModel):
ids: list[int]
expired_at: Optional[Union[datetime.datetime, str]] = None
expired_count: Optional[int] = None
clearExpiredAt: Optional[bool] = None
clear_expired_at: Optional[bool] = None
class FilePolicyActionData(BaseModel):
id: int
action: str
downloadLimit: Optional[int] = None
download_limit: Optional[int] = None
class BatchFilePolicyActionData(BaseModel):
ids: list[int]
action: str
downloadLimit: Optional[int] = None
download_limit: Optional[int] = None
+206 -420
View File
File diff suppressed because it is too large. Load diff
+122 -106
View File
@@ -3,6 +3,7 @@
# @File : views.py
# @Software: PyCharm
import datetime
from typing import Any
from collections import Counter
from typing import Optional
@@ -31,13 +32,18 @@ from apps.admin.schemas import (
)
from core.response import APIResponse
from apps.base.models import FileCodes, KeyValue
from apps.base.quota import owned_storage_queryset
from tortoise.expressions import Q
from tortoise.functions import Count, Sum
from apps.admin.dependencies import (
create_token,
get_admin_session_expire_seconds,
verify_token,
)
from core.logger import logger
from core.settings import settings
from core.utils import get_now, verify_password
from core.utils import get_now, hash_password, password_needs_rehash, verify_password
from apps.base.utils import ip_limit
admin_api = APIRouter(
prefix="/admin", tags=["管理"], dependencies=[Depends(admin_required)]
@@ -53,10 +59,26 @@ def _pick_query_text(*values: Optional[str]) -> Optional[str]:
@admin_api.post("/login")
async def login(data: LoginData):
async def login(data: LoginData, ip: str = Depends(ip_limit["login"])):
# 登录失败计入 IP 频率限制,超过 login_count/login_minute 后暂时锁定
if not verify_password(data.password, settings.admin_token):
ip_limit["login"].add_ip(ip)
raise HTTPException(status_code=401, detail="密码错误")
# 透明重哈希:存量 sha256/明文口令在登录成功(已证明持有口令)时升级为 scrypt
if password_needs_rehash(settings.admin_token):
try:
record = await KeyValue.filter(key="settings").first()
if record:
stored = dict(record.value or {})
stored["admin_token"] = hash_password(data.password)
record.value = stored
await record.save()
settings.admin_token = stored["admin_token"]
logger.info("管理员口令已升级为 scrypt 哈希")
except Exception:
logger.warning("口令哈希升级失败,保持旧格式", exc_info=True)
expires_in = get_admin_session_expire_seconds()
token = create_token({"is_admin": True}, expires_in=expires_in)
return APIResponse(
@@ -90,81 +112,103 @@ async def build_dashboard_recent_file(file_code: FileCodes) -> dict:
"suffix": file_code.suffix,
"size": file_code.size,
"text": file_code.text is not None,
"expiredAt": file_code.expired_at,
"expiredCount": file_code.expired_count,
"usedCount": file_code.used_count,
"createdAt": file_code.created_at,
"isExpired": is_expired,
"expired_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"created_at": file_code.created_at,
"is_expired": is_expired,
}
def _expired_predicate(now: datetime.datetime) -> Q:
"""SQL version of FileCodes.is_expired:
expired_at IS NOT NULL AND ((expired_count < 0 AND expired_at < now) OR expired_count = 0)
"""
return Q(expired_at__isnull=False) & (
Q(expired_count__lt=0, expired_at__lt=now) | Q(expired_count=0)
)
async def _sum_size(queryset) -> int:
rows = await queryset.annotate(total=Sum("size")).values("total")
return rows[0]["total"] or 0
@admin_api.get("/dashboard")
async def dashboard(file_service: FileService = Depends(get_file_service)):
all_codes = await FileCodes.all()
all_size = sum([code.size for code in all_codes])
sys_start = await KeyValue.filter(key="sys_start").first()
now = await get_now()
today_start = now.replace(hour=0, minute=0, second=0, microsecond=0)
yesterday_start = today_start - datetime.timedelta(days=1)
yesterday_end = today_start - datetime.timedelta(microseconds=1)
yesterday_codes = FileCodes.filter(
created_at__gte=yesterday_start, created_at__lte=yesterday_end
)
today_codes = FileCodes.filter(created_at__gte=today_start)
yesterday_file_codes = await yesterday_codes
today_file_codes = await today_codes
expired_count = 0
for file_code in all_codes:
if await file_code.is_expired():
expired_count += 1
health_summary = await file_service.build_file_health_summary(all_codes, now=now)
text_count = sum(1 for file_code in all_codes if file_code.text is not None)
chunked_count = sum(1 for file_code in all_codes if file_code.is_chunked)
used_count = sum([file_code.used_count for file_code in all_codes])
# 简单计数全部走 SQL 聚合,避免整表载入内存(D3)。
# 健康摘要依赖逐行 rules engine(_build_file_status_insights),保留一次遍历。
total_files = await FileCodes.all().count()
expired_count = await FileCodes.filter(_expired_predicate(now)).count()
all_size = await _sum_size(owned_storage_queryset())
used_count = (
await FileCodes.all().annotate(total=Sum("used_count")).values("total")
)[0]["total"] or 0
text_count = await FileCodes.filter(text__isnull=False).count()
chunked_count = await FileCodes.filter(is_chunked=True).count()
yesterday_count = await FileCodes.filter(
created_at__gte=yesterday_start, created_at__lte=yesterday_end
).count()
yesterday_size = await _sum_size(
owned_storage_queryset(
FileCodes.filter(created_at__gte=yesterday_start, created_at__lte=yesterday_end)
)
)
today_count = await FileCodes.filter(created_at__gte=today_start).count()
today_size = await _sum_size(
owned_storage_queryset(FileCodes.filter(created_at__gte=today_start))
)
suffix_rows = (
await FileCodes.filter(text__isnull=True)
.annotate(count=Count("id"))
.values("suffix", "count")
)
suffix_counter = Counter(
"Text" if file_code.text is not None else (file_code.suffix or "file")
for file_code in all_codes
{((row["suffix"] or "file") or "file"): row["count"] for row in suffix_rows}
)
recent_file_codes = await FileCodes.all().order_by("-created_at").limit(8)
health_summary = await file_service.build_file_health_summary(
await FileCodes.all(), now=now
)
recent_file_codes = sorted(
all_codes,
key=lambda file_code: file_code.created_at.timestamp()
if file_code.created_at
else 0,
reverse=True,
)[:8]
recent_activities = await file_service.list_admin_activities(limit=8)
return APIResponse(
detail={
"totalFiles": len(all_codes),
"storageUsed": str(all_size),
"sysUptime": sys_start.value if sys_start else None,
"yesterdayCount": len(yesterday_file_codes),
"yesterdaySize": str(sum([code.size for code in yesterday_file_codes])),
"todayCount": len(today_file_codes),
"todaySize": str(sum([code.size for code in today_file_codes])),
"activeCount": len(all_codes) - expired_count,
"expiredCount": expired_count,
"textCount": text_count,
"fileCount": len(all_codes) - text_count,
"chunkedCount": chunked_count,
"usedCount": used_count,
"storageBackend": settings.file_storage,
"uploadSizeLimit": settings.uploadSize,
"openUpload": settings.openUpload,
"enableChunk": settings.enableChunk,
"maxSaveSeconds": settings.max_save_seconds,
"total_files": total_files,
"storage_used": str(all_size),
"sys_uptime": sys_start.value if sys_start else None,
"yesterday_count": yesterday_count,
"yesterday_size": str(yesterday_size),
"today_count": today_count,
"today_size": str(today_size),
"active_count": total_files - expired_count,
"expired_count": expired_count,
"text_count": text_count,
"file_count": total_files - text_count,
"chunked_count": chunked_count,
"used_count": used_count,
"storage_backend": settings.file_storage,
"upload_size_limit": settings.upload_size,
"open_upload": settings.open_upload,
"enable_chunk": settings.enable_chunk,
"max_save_seconds": settings.max_save_seconds,
**health_summary,
"healthSummary": health_summary,
"topSuffixes": [
"health_summary": health_summary,
"top_suffixes": [
{"suffix": suffix, "count": count}
for suffix, count in suffix_counter.most_common(8)
],
"recentFiles": [
"recent_files": [
await build_dashboard_recent_file(file_code)
for file_code in recent_file_codes
],
"recentActivities": recent_activities["activities"],
"recent_activities": recent_activities["activities"],
}
)
@@ -230,14 +274,15 @@ async def batch_update_files(
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要更新的文件")
update_data = {}
update_data: dict[str, Any] = {}
fields_set = data.model_fields_set
should_clear_expired_at = bool(data.clearExpiredAt or data.clear_expired_at)
should_clear_expired_at = bool(data.clear_expired_at)
if should_clear_expired_at:
update_data["expired_at"] = None
update_data["expired_count"] = -1
elif "expired_at" in fields_set and data.expired_at != "":
# schema 声明 Union[datetime, str]——透传两种形态,tortoise 均可序列化
update_data["expired_at"] = data.expired_at
if (
@@ -274,9 +319,7 @@ async def apply_file_policy_action(
data: FilePolicyActionData,
file_service: FileService,
):
download_limit = data.downloadLimit
if download_limit is None:
download_limit = data.download_limit
download_limit = data.download_limit
detail = await file_service.apply_file_policy_action(
file_id=data.id,
@@ -309,9 +352,7 @@ async def apply_batch_file_policy_action(
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要更新的文件")
download_limit = data.downloadLimit
if download_limit is None:
download_limit = data.download_limit
download_limit = data.download_limit
result = await file_service.apply_files_policy_action(
file_ids=data.ids,
@@ -345,8 +386,8 @@ async def file_list(
status: str = "",
type: str = "",
health: str = "",
sortBy: str = "created_at",
sortOrder: str = "desc",
sort_by: str = "created_at",
sort_order: str = "desc",
file_service: FileService = Depends(get_file_service),
):
page = max(page, 1)
@@ -358,8 +399,8 @@ async def file_list(
status=status,
file_type=type,
health=health,
sort_by=sortBy,
sort_order=sortOrder,
sort_by=sort_by,
sort_order=sort_order,
)
return APIResponse(
detail={
@@ -500,7 +541,7 @@ async def update_config(
config_service: ConfigService = Depends(get_config_service),
file_service: FileService = Depends(get_file_service),
):
data.pop("themesChoices", None)
data.pop("themes_choices", None)
await config_service.update_config(data)
await file_service.record_admin_activity(
action="config.update",
@@ -524,18 +565,19 @@ async def file_download(
@admin_api.get("/file/preview")
async def file_preview(
id: int,
maxChars: int = 4000,
max_chars: int = 4000,
file_service: FileService = Depends(get_file_service),
):
preview = await file_service.preview_file(id, maxChars)
preview = await file_service.preview_file(id, max_chars)
return APIResponse(detail=preview)
@admin_api.get("/local/lists")
async def get_local_lists(
path: str = "",
local_file_service: LocalFileService = Depends(get_local_file_service),
):
files = await local_file_service.list_files()
files = await local_file_service.list_files(path)
return APIResponse(detail=files)
@@ -569,8 +611,8 @@ async def share_local_file(
target_name=item.filename,
count=1,
meta={
"expireValue": item.expire_value,
"expireStyle": item.expire_style,
"expire_value": item.expire_value,
"expire_style": item.expire_style,
},
)
return APIResponse(detail=share_info)
@@ -581,38 +623,12 @@ async def update_file(
data: UpdateFileData,
file_service: FileService = Depends(get_file_service),
):
file_code = await FileCodes.filter(id=data.id).first()
if not file_code:
raise HTTPException(status_code=404, detail="文件不存在")
target_name = file_service._build_file_activity_name(file_code)
update_data = {}
if data.code is not None and data.code != file_code.code:
# 判断code是否存在
if await FileCodes.filter(code=data.code).first():
raise HTTPException(status_code=400, detail="code已存在")
update_data["code"] = data.code
if data.prefix is not None and data.prefix != file_code.prefix:
update_data["prefix"] = data.prefix
if data.suffix is not None and data.suffix != file_code.suffix:
update_data["suffix"] = data.suffix
if (
data.expired_at is not None
and data.expired_at != ""
and data.expired_at != file_code.expired_at
):
update_data["expired_at"] = data.expired_at
if data.expired_count is not None and data.expired_count != file_code.expired_count:
update_data["expired_count"] = data.expired_count
await file_code.update_from_dict(update_data).save()
if update_data:
await file_service.record_admin_activity(
action="file.update",
target_type="file",
target_id=data.id,
target_name=target_name,
count=1,
meta={"fields": sorted(update_data.keys())},
)
await file_service.update_file(
file_id=data.id,
code=data.code,
prefix=data.prefix,
suffix=data.suffix,
expired_at=data.expired_at,
expired_count=data.expired_count,
)
return APIResponse(detail="更新成功")
+150
View File
@@ -0,0 +1,150 @@
"""Cross-cutting auth primitives and the share-upload gate.
Lives in apps.base because BOTH the share surface (base.views) and the
admin surface (admin.views) consume it; keeping it here avoids the
base → admin reverse dependency. Admin-only session gating stays in
apps.admin.dependencies, which re-exports these primitives for
compatibility.
"""
import base64
import hmac
import json
import time
from fastapi import Header, HTTPException
from core.settings import (
ADMIN_SESSION_EXPIRE_DEFAULT,
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
)
def _get_jwt_secret() -> bytes:
secret = getattr(settings, "jwt_secret", "")
if not secret:
raise RuntimeError("JWT签名密钥未初始化")
return secret.encode()
def get_admin_session_expire_seconds() -> int:
try:
expires_in = int(
getattr(settings, "admin_session_expire", ADMIN_SESSION_EXPIRE_DEFAULT)
)
except (TypeError, ValueError):
return ADMIN_SESSION_EXPIRE_DEFAULT
if (
not ADMIN_SESSION_EXPIRE_MIN <= expires_in <= ADMIN_SESSION_EXPIRE_MAX
or expires_in % ADMIN_SESSION_EXPIRE_MIN != 0
):
return ADMIN_SESSION_EXPIRE_DEFAULT
return expires_in
def create_token(data: dict, expires_in: int | None = None) -> str:
"""
创建JWT token
:param data: 数据负载
:param expires_in: 过期时间(秒)
"""
token_lifetime = (
get_admin_session_expire_seconds() if expires_in is None else expires_in
)
header = base64.urlsafe_b64encode(
json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode()
).decode().rstrip("=")
payload = base64.urlsafe_b64encode(
json.dumps(
{**data, "exp": int(time.time()) + token_lifetime},
separators=(",", ":"),
).encode()
).decode().rstrip("=")
signature_digest = hmac.new(
_get_jwt_secret(), f"{header}.{payload}".encode(), "sha256"
).digest()
signature = base64.urlsafe_b64encode(signature_digest).decode().rstrip("=")
return f"{header}.{payload}.{signature}"
def verify_token(token: str) -> dict:
"""
验证JWT token
:param token: JWT token
:return: 解码后的数据
"""
try:
header_b64, payload_b64, signature_b64 = token.split(".")
# 验证签名
expected_signature = hmac.new(
_get_jwt_secret(),
f"{header_b64}.{payload_b64}".encode(),
"sha256",
).digest()
expected_signature_b64 = (
base64.urlsafe_b64encode(expected_signature).decode().rstrip("=")
)
if not hmac.compare_digest(signature_b64, expected_signature_b64):
raise ValueError("无效的签名")
# 解码payload(兼容历史标准 base64 与 urlsafe base64)
padded = payload_b64 + "=" * (-len(payload_b64) % 4)
try:
payload_bytes = base64.urlsafe_b64decode(padded)
except Exception:
payload_bytes = base64.b64decode(padded)
payload = json.loads(payload_bytes)
# 检查是否过期
if payload.get("exp", 0) < time.time():
raise ValueError("token已过期")
return payload
except Exception as e:
raise ValueError(f"token验证失败: {str(e)}")
def _extract_bearer_token(authorization: str) -> str:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
token = authorization.split(" ", 1)[1].strip()
if not token:
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return token
def _require_admin_payload(authorization: str) -> dict:
token = _extract_bearer_token(authorization)
try:
payload = verify_token(token)
except ValueError as e:
raise HTTPException(status_code=401, detail=str(e))
if not payload.get("is_admin", False):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return payload
async def share_required_login(authorization: str = Header(default=None)):
"""
验证分享上传权限
当 settings.open_upload 为False时,要求用户必须登录并具有管理员权限
当 settings.open_upload 为True时,允许游客上传
:param authorization: 认证头信息
:param request: 请求对象
:return: 验证结果
"""
if not settings.open_upload:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=403, detail="本站未开启游客上传,如需上传请先登录后台"
)
_require_admin_payload(authorization)
return True
+42 -18
View File
@@ -16,16 +16,18 @@ MIN_ADMIN_PASSWORD_LENGTH = 8
SETUP_CONFIG_FIELDS = {
"allowed_file_types",
"code_generate_type",
"enableChunk",
"errorCount",
"errorMinute",
"expireStyle",
"enable_chunk",
"error_count",
"error_minute",
"login_count",
"login_minute",
"expire_style",
"max_save_seconds",
"name",
"openUpload",
"uploadCount",
"uploadMinute",
"uploadSize",
"open_upload",
"upload_count",
"upload_minute",
"upload_size",
}
@@ -58,23 +60,45 @@ async def ensure_security_settings() -> None:
logger.info("已将管理员密码迁移为哈希存储")
if security_config.jwt_secret_rotated:
logger.info("已生成独立 JWT 签名密钥")
await refresh_settings()
await refresh_settings(force=True)
def _sync_ip_limits() -> None:
ip_limit["error"].minutes = settings.errorMinute
ip_limit["error"].count = settings.errorCount
ip_limit["metadata"].minutes = settings.errorMinute
ip_limit["metadata"].count = settings.errorCount
ip_limit["upload"].minutes = settings.uploadMinute
ip_limit["upload"].count = settings.uploadCount
ip_limit["error"].minutes = settings.error_minute
ip_limit["error"].count = settings.error_count
ip_limit["metadata"].minutes = settings.error_minute
ip_limit["metadata"].count = settings.error_count
ip_limit["upload"].minutes = settings.upload_minute
ip_limit["upload"].count = settings.upload_count
ip_limit["login"].minutes = settings.login_minute
ip_limit["login"].count = settings.login_count
async def refresh_settings() -> None:
"""从数据库读取最新配置并应用到运行时。"""
# Per-process config cache: the middleware refreshes settings on every request,
# which otherwise means one DB read per request. Writes call refresh_settings(force=True)
# so same-process visibility is immediate; other processes converge within the TTL
# (documented deployment is a single worker).
CONFIG_CACHE_TTL_SECONDS = 2.0
_config_cached_until = 0.0
async def refresh_settings(force: bool = False) -> None:
"""从数据库读取最新配置并应用到运行时。
默认带短 TTL 缓存(避免每请求查库);写配置的代码路径必须传 force=True。
"""
global _config_cached_until
import time as _time
if not force and _time.monotonic() < _config_cached_until:
return
config_record = await KeyValue.filter(key="settings").first()
settings.user_config = config_record.value if config_record and config_record.value else {}
unknown = settings.unknown_keys(settings.user_config)
if unknown:
logger.warning("配置中存在未登记的键(将被忽略其类型语义,仅透传): %s", unknown)
_sync_ip_limits()
_config_cached_until = _time.monotonic() + CONFIG_CACHE_TTL_SECONDS
def is_runtime_initialized() -> bool:
@@ -113,4 +137,4 @@ async def initialize_system(
next_config["jwt_secret"] = generate_jwt_secret()
await KeyValue.update_or_create(key="settings", defaults={"value": next_config})
await refresh_settings()
await refresh_settings(force=True)
+11 -4
View File
@@ -1,5 +1,5 @@
from ipaddress import ip_address, ip_network
from typing import Dict, Iterable, Union
from typing import Dict, Iterable, TypedDict
from datetime import datetime, timedelta
from fastapi import HTTPException, Request
@@ -7,7 +7,7 @@ from core.settings import settings
def _iter_trusted_proxies() -> Iterable[str]:
trusted_proxies = getattr(settings, "trustedProxies", [])
trusted_proxies = getattr(settings, "trusted_proxies", [])
if isinstance(trusted_proxies, str):
trusted_proxies = [item.strip() for item in trusted_proxies.split(",")]
return [item for item in trusted_proxies if item]
@@ -65,9 +65,14 @@ def get_client_ip(request: Request) -> str:
return client_host
class _IPRecord(TypedDict):
count: int
time: datetime
class IPRateLimit:
def __init__(self, count: int, minutes: int):
self.ips: Dict[str, Dict[str, Union[int, datetime]]] = {}
self.ips: Dict[str, _IPRecord] = {}
self.count = count
self.minutes = minutes
@@ -81,7 +86,9 @@ class IPRateLimit:
return True
def add_ip(self, ip: str) -> int:
ip_info = self.ips.get(ip, {"count": 0, "time": datetime.now()})
ip_info = self.ips.setdefault(
ip, {"count": 0, "time": datetime.now()}
)
ip_info["count"] += 1
ip_info["time"] = datetime.now()
self.ips[ip] = ip_info
+196
View File
@@ -0,0 +1,196 @@
"""文件类型校验:扩展名 / MIME 白名单 + magic bytes 防伪造。"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
from fastapi import HTTPException, UploadFile
from fnmatch import fnmatchcase
from core.settings import settings
@dataclass(frozen=True)
class FileKind:
name: str
extensions: tuple[str, ...]
mimes: tuple[str, ...]
signatures: tuple[bytes, ...]
FILE_KINDS: tuple[FileKind, ...] = (
FileKind("png", (".png",), ("image/png",), (b"\x89PNG\r\n\x1a\n",)),
FileKind("jpg", (".jpg", ".jpeg"), ("image/jpeg",), (b"\xff\xd8\xff",)),
FileKind("gif", (".gif",), ("image/gif",), (b"GIF87a", b"GIF89a")),
FileKind("webp", (".webp",), ("image/webp",), ()),
FileKind("bmp", (".bmp",), ("image/bmp", "image/x-ms-bmp"), (b"BM",)),
FileKind("pdf", (".pdf",), ("application/pdf",), (b"%PDF",)),
FileKind(
"zip",
(".zip", ".docx", ".xlsx", ".pptx", ".apk", ".jar"),
(
"application/zip",
"application/x-zip-compressed",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/java-archive",
"application/vnd.android.package-archive",
),
(b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08"),
),
FileKind(
"rar",
(".rar",),
("application/x-rar-compressed", "application/vnd.rar"),
(b"Rar!\x1a\x07\x00", b"Rar!\x1a\x07\x01\x00"),
),
FileKind("7z", (".7z",), ("application/x-7z-compressed",), (b"7z\xbc\xaf'\x1c",)),
FileKind("gz", (".gz", ".tgz"), ("application/gzip", "application/x-gzip"), (b"\x1f\x8b",)),
FileKind(
"mp3",
(".mp3",),
("audio/mpeg",),
(b"ID3", b"\xff\xfb", b"\xff\xf3", b"\xff\xf2"),
),
FileKind("mp4", (".mp4", ".m4a", ".mov"), ("video/mp4", "audio/mp4", "video/quicktime"), ()),
FileKind(
"exe",
(".exe", ".dll", ".sys"),
("application/x-msdownload", "application/x-dosexec"),
(b"MZ",),
),
FileKind("elf", (".elf", ".so", ".o"), ("application/x-executable",), (b"\x7fELF",)),
)
KNOWN_EXTENSIONS = {ext for kind in FILE_KINDS for ext in kind.extensions}
def normalize_allowed_file_types() -> list[str]:
raw_value = settings.allowed_file_types
if isinstance(raw_value, str):
values = [item.strip() for item in raw_value.split(",")]
elif isinstance(raw_value, (list, tuple, set)):
values = [str(item).strip() for item in raw_value]
else:
values = []
normalized = [item.lower() for item in values if item]
return normalized or ["*"]
def _extension_of(file_name: str) -> str:
return Path(file_name or "").suffix.lower()
def _match_allow_rule(rule: str, file_name: str, content_type: str) -> bool:
if rule in {"*", "*/*"}:
return True
if "/" in rule:
return fnmatchcase(content_type, rule)
extension = rule if rule.startswith(".") else f".{rule}"
return file_name.endswith(extension)
def is_type_allowed(file_name: str, content_type: Optional[str] = None) -> bool:
allowed = normalize_allowed_file_types()
if any(rule in {"*", "*/*"} for rule in allowed):
return True
normalized_name = (file_name or "").strip().lower()
normalized_content_type = (content_type or "").strip().lower()
return any(
_match_allow_rule(rule, normalized_name, normalized_content_type)
for rule in allowed
)
def detect_file_kind(header: bytes) -> Optional[FileKind]:
if not header:
return None
if len(header) >= 12 and header.startswith(b"RIFF") and header[8:12] == b"WEBP":
return next(kind for kind in FILE_KINDS if kind.name == "webp")
if len(header) >= 12 and header[4:8] == b"ftyp":
return next(kind for kind in FILE_KINDS if kind.name == "mp4")
candidates: list[tuple[int, FileKind]] = []
for kind in FILE_KINDS:
for signature in kind.signatures:
if signature and header.startswith(signature):
candidates.append((len(signature), kind))
if not candidates:
return None
candidates.sort(key=lambda item: item[0], reverse=True)
return candidates[0][1]
def validate_file_type(file_name: str, content_type: Optional[str] = None) -> None:
if not is_type_allowed(file_name, content_type):
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
def _kind_matches_ext(kind: FileKind, extension: str) -> bool:
return extension in kind.extensions
def _kind_matches_mime(kind: FileKind, mime: str) -> bool:
return mime in kind.mimes
def validate_file_magic(
file_name: str,
content_type: Optional[str],
header: Optional[bytes],
) -> None:
"""白名单通过后,用 magic bytes 防止扩展名 / MIME 伪造。"""
validate_file_type(file_name, content_type)
if not header:
return
claimed_ext = _extension_of(file_name)
claimed_mime = (content_type or "").strip().lower()
detected = detect_file_kind(header)
if claimed_ext in KNOWN_EXTENSIONS:
if detected is None or not _kind_matches_ext(detected, claimed_ext):
raise HTTPException(
status_code=403,
detail="文件内容与扩展名不匹配,疑似伪造类型",
)
if claimed_mime and any(claimed_mime in kind.mimes for kind in FILE_KINDS):
if detected is None or not _kind_matches_mime(detected, claimed_mime):
raise HTTPException(
status_code=403,
detail="文件内容与 Content-Type 不匹配,疑似伪造类型",
)
allowed = normalize_allowed_file_types()
if detected and not any(rule in {"*", "*/*"} for rule in allowed):
allowed_by_detected = any(
is_type_allowed(f"file{ext}", mime)
for ext in detected.extensions
for mime in (detected.mimes or (None,))
)
if not allowed_by_detected:
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
async def read_upload_header(file: UploadFile, size: int = 64) -> bytes:
await file.seek(0)
header = await file.read(size)
await file.seek(0)
return header or b""
async def validate_upload_file(file: UploadFile) -> None:
header = await read_upload_header(file)
validate_file_magic(file.filename or "", file.content_type, header)
def validate_header_bytes(
file_name: str, content_type: Optional[str], header: bytes
) -> None:
validate_file_magic(file_name, content_type, header)
+106
View File
@@ -0,0 +1,106 @@
"""Local NAS-style share helpers.
Files under ``data/local`` can be browsed by admin and turned into extract
codes without copying them into ``data/share``. The original file is never
deleted when a share expires or is removed from the file list.
``FileCodes.file_path`` is the sentinel ``local-ref``; ``uuid_file_name``
stores the posix relative path under ``data/local``. Accidental storage
deletes therefore look at ``data/local-ref/...`` and cannot touch the
original NAS file.
"""
from __future__ import annotations
import time
from pathlib import Path
from urllib.parse import quote
from fastapi import HTTPException
from core.errors import StorageError
from core.storage import StoredDownload
LOCAL_REF_MARKER = "local-ref"
MAX_RELPATH_LEN = 255
MAX_LIST_ENTRIES = 500
def is_local_ref(file_code) -> bool:
return (getattr(file_code, "file_path", None) or "") == LOCAL_REF_MARKER
def should_skip_storage_delete(file_code) -> bool:
return is_local_ref(file_code) or getattr(file_code, "text", None) is not None
def get_local_root() -> Path:
from core.settings import data_root
root = (Path(data_root) / "local").resolve()
root.mkdir(parents=True, exist_ok=True)
return root
def normalize_local_relpath(raw: str, *, allow_empty: bool = False) -> str:
text = str(raw or "").replace("\\", "/").strip()
if not text:
if allow_empty:
return ""
raise HTTPException(status_code=400, detail="非法文件名")
if (
text in {".", ".."}
or text.startswith("/")
or text.startswith("~")
or "\x00" in text
or ":" in text.split("/", 1)[0]
):
raise HTTPException(status_code=400, detail="非法文件名")
parts = [part for part in text.split("/") if part not in {"", "."}]
if not parts or any(part == ".." for part in parts):
raise HTTPException(status_code=400, detail="非法文件名")
relpath = "/".join(parts)
if len(relpath) > MAX_RELPATH_LEN:
raise HTTPException(status_code=400, detail="路径过长")
return relpath
def resolve_under_local(relpath: str) -> Path:
root = get_local_root()
if not relpath:
return root
candidate = (root / relpath).resolve()
try:
candidate.relative_to(root)
except ValueError as exc:
raise HTTPException(status_code=400, detail="非法文件路径") from exc
return candidate
def format_local_ctime(path: Path) -> str | None:
try:
return time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(path.stat().st_ctime))
except OSError:
return None
def build_local_ref_download(file_code) -> StoredDownload:
try:
relpath = normalize_local_relpath(getattr(file_code, "uuid_file_name", None) or "")
path = resolve_under_local(relpath)
except HTTPException as exc:
raise StorageError(status_code=404, detail="文件已过期删除") from exc
if not path.is_file():
raise StorageError(status_code=404, detail="文件已过期删除")
filename = f"{getattr(file_code, 'prefix', '') or ''}{getattr(file_code, 'suffix', '') or ''}"
if not filename:
filename = path.name
encoded_filename = quote(filename, safe="")
headers = {"Content-Disposition": f"attachment; filename*=UTF-8''{encoded_filename}"}
try:
headers["Content-Length"] = str(path.stat().st_size)
except OSError:
pass
return StoredDownload(filename=filename, headers=headers, path=path)
+302
View File
@@ -0,0 +1,302 @@
"""Stateless MCP Streamable HTTP endpoint for quick share uploads."""
import base64
import binascii
import html
import io
import json
from html.parser import HTMLParser
from urllib.parse import quote, urlsplit
from fastapi import APIRouter, HTTPException, Request, UploadFile
from starlette.datastructures import Headers
from starlette.responses import JSONResponse, Response
from apps.base.auth import _require_admin_payload
from apps.base.file_validation import validate_file_type, validate_upload_file
from apps.base.services import FileUploadService, validate_file_size
from apps.base.utils import ip_limit, validate_expire_style
from core.settings import settings
from core.utils import sanitize_filename
router = APIRouter(tags=["MCP"])
MCP_PROTOCOL_VERSION = "2025-03-26"
TEXT_SHARE_MAX_BYTES = 222 * 1024
MAX_MCP_BODY_BYTES = 64 * 1024 * 1024
def _jsonrpc_error(request_id, code: int, message: str) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "error": {"code": code, "message": message}}
def _jsonrpc_result(request_id, result: dict) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "result": result}
def _validate_origin(request: Request) -> None:
"""Reject browser cross-site POSTs before parsing any upload tool."""
origin = request.headers.get("origin")
fetch_site = request.headers.get("sec-fetch-site", "").lower()
if fetch_site == "cross-site":
raise HTTPException(status_code=403, detail="跨站请求已拒绝")
if not origin:
return # Native MCP clients do not send Origin and use no ambient cookies.
try:
parsed = urlsplit(origin)
host = request.headers.get("host", "").lower()
if (
parsed.scheme not in {"http", "https"}
or not parsed.netloc
or parsed.netloc.lower() != host
or parsed.username is not None
or parsed.password is not None
or parsed.path not in {"", "/"}
or parsed.query
or parsed.fragment
):
raise ValueError("origin mismatch")
except (ValueError, TypeError):
raise HTTPException(status_code=403, detail="跨站请求已拒绝")
async def _read_limited_body(request: Request) -> bytes:
max_size = min(
MAX_MCP_BODY_BYTES,
max(1024 * 1024, int(settings.upload_size * 4 / 3) + 256 * 1024),
)
declared_size = request.headers.get("content-length")
if declared_size:
try:
if int(declared_size) > max_size:
raise HTTPException(status_code=413, detail="MCP 请求体过大")
except ValueError:
raise HTTPException(status_code=400, detail="Content-Length 无效")
body = bytearray()
async for chunk in request.stream():
body.extend(chunk)
if len(body) > max_size:
raise HTTPException(status_code=413, detail="MCP 请求体过大")
return bytes(body)
def _tool_definitions() -> list[dict]:
return [
{
"name": "share_text",
"description": "将文本快速上传到 FileCodeBox 并创建分享链接。",
"inputSchema": {
"type": "object",
"properties": {
"text": {"type": "string", "description": "要分享的文本"},
"expire_value": {"type": "integer", "minimum": 1, "default": 1},
"expire_style": {"type": "string", "enum": ["day", "hour", "minute", "forever", "count"], "default": "day"},
},
"required": ["text"],
},
},
{
"name": "share_file",
"description": "将 Base64 编码的文件上传到 FileCodeBox 并创建分享链接。",
"inputSchema": {
"type": "object",
"properties": {
"filename": {"type": "string", "description": "文件名"},
"content_base64": {"type": "string", "description": "文件内容的标准 Base64 编码"},
"content_type": {"type": "string", "default": "application/octet-stream"},
"expire_value": {"type": "integer", "minimum": 1, "default": 1},
"expire_style": {"type": "string", "enum": ["day", "hour", "minute", "forever", "count"], "default": "day"},
},
"required": ["filename", "content_base64"],
},
},
]
def _share_result(request: Request, code: str, name: str) -> dict:
base_url = str(request.base_url).rstrip("/")
share_url = f"{base_url}/#/?code={quote(code, safe='')}"
markdown_name = html.escape(name).replace("[", "\\[").replace("]", "\\]")
return {
"code": code,
"name": name,
"url": share_url,
"markdown": f"[{markdown_name}]({share_url})",
}
async def _call_tool(request: Request, tool_name: str, arguments: dict) -> dict:
authorization = request.headers.get("authorization", "")
if settings.mcp_require_token or not settings.open_upload:
_require_admin_payload(authorization)
ip = ip_limit["upload"](request)
if tool_name == "share_text":
text = arguments.get("text")
if not isinstance(text, str):
raise ValueError("text 必须是字符串")
if len(text.encode("utf-8")) > TEXT_SHARE_MAX_BYTES:
raise ValueError("内容过多,请改用文件分享")
expire_value = int(arguments.get("expire_value", 1))
expire_style = str(arguments.get("expire_style", "day"))
if expire_value < 1:
raise ValueError("expire_value 必须大于 0")
validate_expire_style(expire_style)
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return _share_result(request, code, "Text")
if tool_name == "share_file":
filename = arguments.get("filename")
encoded_content = arguments.get("content_base64")
content_type = str(arguments.get("content_type", "application/octet-stream"))
if not isinstance(filename, str) or not filename.strip():
raise ValueError("filename 不能为空")
if not isinstance(encoded_content, str):
raise ValueError("content_base64 必须是字符串")
try:
content = base64.b64decode(encoded_content, validate=True)
except (ValueError, binascii.Error):
raise ValueError("content_base64 不是有效的标准 Base64")
safe_name = await sanitize_filename(filename.rsplit("/", 1)[-1].rsplit("\\", 1)[-1])
validate_file_type(safe_name, content_type)
upload = UploadFile(filename=safe_name, file=io.BytesIO(content), size=len(content), headers=Headers({"content-type": content_type}))
size = await validate_file_size(upload, settings.upload_size)
await validate_upload_file(upload)
expire_value = int(arguments.get("expire_value", 1))
expire_style = str(arguments.get("expire_style", "day"))
if expire_value < 1:
raise ValueError("expire_value 必须大于 0")
validate_expire_style(expire_style)
share = await FileUploadService.create_file_share(upload, size=size, expire_value=expire_value, expire_style=expire_style)
ip_limit["upload"].add_ip(ip)
return _share_result(request, share["code"], share["name"])
raise LookupError(f"未知工具:{tool_name}")
@router.post("/mcp", include_in_schema=False)
async def mcp_endpoint(request: Request):
_validate_origin(request)
media_type = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
if media_type != "application/json":
raise HTTPException(status_code=415, detail="MCP 请求必须使用 application/json")
try:
message = json.loads(await _read_limited_body(request))
except (json.JSONDecodeError, UnicodeDecodeError):
return JSONResponse(_jsonrpc_error(None, -32700, "Parse error"), status_code=400)
if not isinstance(message, dict) or message.get("jsonrpc") != "2.0" or not isinstance(message.get("method"), str):
return JSONResponse(_jsonrpc_error(message.get("id") if isinstance(message, dict) else None, -32600, "Invalid Request"), status_code=400)
request_id = message.get("id")
method = message["method"]
if request_id is None and method.startswith("notifications/"):
return Response(status_code=202)
try:
if method == "initialize":
result = {
"protocolVersion": MCP_PROTOCOL_VERSION,
"capabilities": {"tools": {"listChanged": False}},
"serverInfo": {"name": "filecodebox", "version": "1.0.0"},
}
elif method == "ping":
result = {}
elif method == "tools/list":
result = {"tools": _tool_definitions()}
elif method == "tools/call":
params = message.get("params")
if not isinstance(params, dict) or not isinstance(params.get("name"), str):
return JSONResponse(_jsonrpc_error(request_id, -32602, "Invalid params"))
arguments = params.get("arguments", {})
if not isinstance(arguments, dict):
return JSONResponse(_jsonrpc_error(request_id, -32602, "Invalid params"))
try:
tool_value = await _call_tool(request, params["name"], arguments)
result = {"content": [{"type": "text", "text": json.dumps(tool_value, ensure_ascii=False)}], "structuredContent": tool_value}
except LookupError as exc:
return JSONResponse(_jsonrpc_error(request_id, -32602, str(exc)))
except HTTPException as exc:
if exc.status_code in {401, 403}:
raise
result = {"content": [{"type": "text", "text": str(exc.detail)}], "isError": True}
except (TypeError, ValueError) as exc:
result = {"content": [{"type": "text", "text": str(exc)}], "isError": True}
else:
return JSONResponse(_jsonrpc_error(request_id, -32601, "Method not found"))
return JSONResponse(_jsonrpc_result(request_id, result), headers={"MCP-Protocol-Version": MCP_PROTOCOL_VERSION})
except HTTPException as exc:
return JSONResponse(
_jsonrpc_error(request_id, -32001 if exc.status_code == 401 else -32003, str(exc.detail)),
status_code=exc.status_code,
)
_SAFE_TAGS = {
"a", "abbr", "article", "b", "blockquote", "br", "caption", "code", "dd", "del", "details",
"div", "dl", "dt", "em", "figcaption", "figure", "h1", "h2", "h3", "h4", "h5", "h6",
"hr", "i", "img", "li", "main", "mark", "ol", "p", "pre", "s", "section", "small", "span",
"strong", "sub", "summary", "sup", "table", "tbody", "td", "th", "thead", "tr", "u", "ul",
}
_VOID_TAGS = {"br", "hr", "img"}
_DROP_CONTENT_TAGS = {"script", "style", "iframe", "object", "embed", "svg", "math", "form", "template"}
_SAFE_ATTRS = {"alt", "title", "class", "id", "colspan", "rowspan", "width", "height", "scope", "dir", "lang"}
class _HTMLPreviewSanitizer(HTMLParser):
"""Whitelist-only HTML serializer; scripts, active embeds and URL attrs are removed."""
def __init__(self):
super().__init__(convert_charrefs=True)
self.output: list[str] = []
self.drop_depth = 0
def handle_starttag(self, tag, attrs):
tag = tag.lower()
if self.drop_depth:
if tag in _DROP_CONTENT_TAGS:
self.drop_depth += 1
return
if tag in _DROP_CONTENT_TAGS:
self.drop_depth = 1
return
if tag not in _SAFE_TAGS:
return
safe_attrs = []
for name, value in attrs:
name = name.lower()
if value is None or name.startswith("on") or name not in _SAFE_ATTRS:
continue
safe_attrs.append(f' {name}="{html.escape(value, quote=True)}"')
self.output.append(f"<{tag}{''.join(safe_attrs)}>")
def handle_startendtag(self, tag, attrs):
self.handle_starttag(tag, attrs)
if tag.lower() in _SAFE_TAGS and tag.lower() not in _VOID_TAGS:
self.handle_endtag(tag)
def handle_endtag(self, tag):
tag = tag.lower()
if self.drop_depth:
if tag in _DROP_CONTENT_TAGS:
self.drop_depth -= 1
return
if tag in _SAFE_TAGS and tag not in _VOID_TAGS:
self.output.append(f"</{tag}>")
def handle_data(self, data):
if not self.drop_depth:
self.output.append(html.escape(data))
def handle_entityref(self, name):
if not self.drop_depth:
self.output.append(f"&amp;{html.escape(name)};")
def handle_charref(self, name):
if not self.drop_depth:
self.output.append(f"&amp;#{html.escape(name)};")
def sanitize_html_preview(source: str) -> str:
parser = _HTMLPreviewSanitizer()
parser.feed(source)
parser.close()
return "<!doctype html><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><body>" + "".join(parser.output) + "</body>"
+51
View File
@@ -0,0 +1,51 @@
import json
from tortoise import connections
# 配置键驼峰 -> snake_case 统一(2.5)。settings 存于 KeyValue 表的 JSON 值中,
# 迁移在 Python 层读取、改写键名后写回,保证存量部署升级后键名与代码一致。
KEY_RENAMES = {
"adminSessionExpire": "admin_session_expire",
"enableChunk": "enable_chunk",
"errorCount": "error_count",
"errorMinute": "error_minute",
"expireStyle": "expire_style",
"loginCount": "login_count",
"loginMinute": "login_minute",
"openUpload": "open_upload",
"robotsText": "robots_text",
"serverHost": "server_host",
"serverPort": "server_port",
"serverWorkers": "server_workers",
"showAdminAddr": "show_admin_addr",
"storageLimit": "storage_limit",
"themesChoices": "themes_choices",
"themesSelect": "themes_select",
"trustedProxies": "trusted_proxies",
"uploadCount": "upload_count",
"uploadMinute": "upload_minute",
"uploadSize": "upload_size",
}
async def migrate():
conn = connections.get("default")
rows = await conn.execute_query_dict(
"SELECT id, value FROM keyvalue WHERE key = 'settings'"
)
for row in rows:
value = row["value"]
if isinstance(value, str):
try:
value = json.loads(value)
except ValueError:
continue
if not isinstance(value, dict):
continue
renamed = {KEY_RENAMES.get(k, k): v for k, v in value.items()}
if renamed != value:
await conn.execute_query(
"UPDATE keyvalue SET value = ? WHERE id = ?",
[json.dumps(renamed, ensure_ascii=False), row["id"]],
)
+12 -11
View File
@@ -2,13 +2,13 @@
# @Author : Lan
# @File : models.py
# @Software: PyCharm
from typing import Optional
from typing import Any
from tortoise.models import Model
from tortoise.contrib.pydantic import pydantic_model_creator
from tortoise import fields, models
from datetime import datetime
from core.utils import get_now
@@ -22,7 +22,9 @@ class FileCodes(models.Model):
size = fields.BigIntField(default=0)
text = fields.TextField(null=True)
expired_at = fields.DatetimeField(null=True)
expired_count = fields.IntField(default=0)
# -1 = 时间式(看 expired_at),<=0 的 0 = 次数耗尽。默认 -1 而非 0:
# 0 语义是"次数已用完即过期",忘传该字段的创建路径会得到立即过期的记录。
expired_count = fields.IntField(default=-1)
used_count = fields.IntField(default=0)
created_at = fields.DatetimeField(auto_now_add=True)
file_hash = fields.CharField(max_length=64, null=True)
@@ -55,14 +57,13 @@ class UploadChunk(models.Model):
class KeyValue(Model):
id: Optional[int] = fields.IntField(pk=True)
key: Optional[str] = fields.CharField(
max_length=255, description="键", index=True, unique=True
)
value: Optional[str] = fields.JSONField(description="值", null=True)
created_at: Optional[datetime] = fields.DatetimeField(
auto_now_add=True, description="创建时间"
)
# 与 FileCodes 保持同一风格:类属性不写值注解(等号右侧的 Field 描述符
# 才是类级真身;实例级由 tortoise 运行时给出对应 Python 值)
id = fields.IntField(pk=True)
key = fields.CharField(max_length=255, description="键", index=True, unique=True)
# JSONField 实际承载 dict/list(如 settings 配置)
value: Any = fields.JSONField(description="值", null=True)
created_at = fields.DatetimeField(auto_now_add=True, description="创建时间")
class PresignUploadSession(models.Model):
+159
View File
@@ -0,0 +1,159 @@
"""Public-facing pages and routes: setup wizard, theme assets, index,
robots.txt, and the public config endpoints."""
import html
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, RedirectResponse
from apps.base.config import initialize_system, is_runtime_initialized
from apps.base.setup_wizard import (
build_public_config,
build_public_meta,
build_setup_page,
build_setup_success_page,
parse_setup_options,
read_setup_payload,
setup_response,
)
from core.response import APIResponse
from core.settings import BASE_DIR, DEFAULT_CONFIG, settings
from core.version import APP_VERSION
router = APIRouter()
@router.get("/setup", include_in_schema=False)
@router.get("/setup/", include_in_schema=False)
async def setup_page():
if is_runtime_initialized():
return RedirectResponse(url="/", status_code=303)
return setup_response(build_setup_page())
@router.post("/setup", include_in_schema=False)
@router.post("/setup/", include_in_schema=False)
async def setup_submit(request: Request):
if is_runtime_initialized():
return RedirectResponse(url="/", status_code=303)
data = await read_setup_payload(request)
admin_password = str(data.get("admin_password") or "")
confirm_password = str(data.get("confirm_password") or "")
site_name = str(data.get("site_name") or "")
if admin_password != confirm_password:
return setup_response(build_setup_page("两次输入的管理员密码不一致", data), 400)
try:
setup_options = parse_setup_options(data)
await initialize_system(
admin_password=admin_password,
site_name=site_name,
setup_options=setup_options,
)
except ValueError as exc:
return setup_response(build_setup_page(str(exc), data), 400)
if "application/json" in request.headers.get("accept", ""):
return APIResponse(detail={"ok": True, "admin": "/#/admin"})
return setup_response(build_setup_success_page())
def resolve_theme_root():
themes_root = (BASE_DIR / "themes").resolve()
theme_root = (BASE_DIR / str(settings.themes_select)).resolve()
try:
theme_root.relative_to(themes_root)
except ValueError:
theme_root = (BASE_DIR / DEFAULT_CONFIG["themes_select"]).resolve()
if not theme_root.exists():
theme_root = (BASE_DIR / DEFAULT_CONFIG["themes_select"]).resolve()
return theme_root
def resolve_theme_file(*parts: str):
theme_root = resolve_theme_root()
file_path = theme_root.joinpath(*parts).resolve()
# 防止通过 /assets/../ 读取主题目录外的文件。
try:
file_path.relative_to(theme_root)
except ValueError:
raise HTTPException(status_code=404, detail="资源不存在")
if not file_path.is_file():
raise HTTPException(status_code=404, detail="资源不存在")
return file_path
@router.get("/assets/{asset_path:path}", include_in_schema=False)
async def theme_asset(asset_path: str):
return FileResponse(resolve_theme_file("assets", asset_path))
async def not_found_handler(request, exc=None):
"""区分"浏览器导航到未知路径"与"API 处理器抛出的 404"。
Starlette 的 status-code handler 优先于 HTTPException 类 handler,
直接注册 index 会让全应用所有 HTTPException(404) 变成 200 HTML 页
(负路径测试抓到的存量 bug)。浏览器(Accept 含 text/html)仍拿到
主题首页做 SPA 兜底;API 客户端拿到 JSON 404。
"""
json_404 = JSONResponse(
status_code=404,
content={"code": 404, "message": "Not Found", "detail": "资源不存在"},
)
if request is not None and request.method in {"GET", "HEAD"} and "text/html" in (
request.headers.get("accept", "")
):
try:
return await index(request, exc)
except HTTPException:
return json_404
return json_404
@router.get("/")
async def index(request=None, exc=None):
# Site config is admin input (and during the setup window anyone can claim it);
# always escape before injecting into the theme template to prevent stored XSS
# (mirrors the setup page).
return HTMLResponse(
content=resolve_theme_file("index.html")
.read_text(encoding="utf-8")
.replace("{{title}}", html.escape(str(settings.name)))
.replace("{{description}}", html.escape(str(settings.description)))
.replace("{{keywords}}", html.escape(str(settings.keywords)))
.replace("{{opacity}}", html.escape(str(settings.opacity)))
.replace("{{background}}", html.escape(str(settings.background))),
media_type="text/html",
headers={"Cache-Control": "no-cache"},
)
@router.get("/robots.txt")
async def robots():
return HTMLResponse(content=settings.robots_text, media_type="text/plain")
@router.post("/")
async def get_config():
return APIResponse(detail=build_public_config())
@router.get("/api/v1/config")
async def get_public_config():
return APIResponse(
detail={
"config": build_public_config(),
"meta": build_public_meta(),
}
)
@router.get("/health")
async def health_check():
return APIResponse(
detail={
"status": "ok",
"version": APP_VERSION,
"storage": settings.file_storage,
"theme": settings.themes_select,
}
)
+21 -8
View File
@@ -2,12 +2,24 @@ import datetime
from fastapi import HTTPException
from tortoise import connections
from tortoise.expressions import Q
from tortoise.functions import Sum
from apps.base.models import FileCodes, StorageReservation
from apps.base.local_share import LOCAL_REF_MARKER
from core.settings import settings
from core.utils import get_now
def owned_storage_queryset(queryset=None):
"""FileCodes that occupy FileCodeBox storage (exclude NAS local-ref rows).
SQL ``file_path != 'local-ref'`` drops NULLs, so keep null paths explicitly.
"""
qs = FileCodes.all() if queryset is None else queryset
return qs.filter(Q(file_path__isnull=True) | ~Q(file_path=LOCAL_REF_MARKER))
def _detect_sql_dialect() -> str:
"""识别当前默认连接的 SQL 方言。
@@ -47,20 +59,21 @@ def _sql_placeholders(count: int) -> list[str]:
def get_storage_limit() -> int:
try:
return max(0, int(getattr(settings, "storageLimit", 0)))
return max(0, int(getattr(settings, "storage_limit", 0)))
except (TypeError, ValueError):
return 0
async def get_storage_usage() -> dict[str, int | None]:
now = await get_now()
used = await FileCodes.all().values_list("size", flat=True)
reserved = await StorageReservation.filter(expires_at__gt=now).values_list(
"size", flat=True
)
# SQL 聚合:此函数在每次上传配额检查时调用,禁止全表拉取(D3)
used_rows = await owned_storage_queryset().annotate(total=Sum("size")).values("total")
reserved_rows = await StorageReservation.filter(expires_at__gt=now).annotate(
total=Sum("size")
).values("total")
limit = get_storage_limit()
used_bytes = sum(used)
reserved_bytes = sum(reserved)
used_bytes = used_rows[0]["total"] or 0
reserved_bytes = reserved_rows[0]["total"] or 0
return {
"limit": limit,
"used": used_bytes,
@@ -92,7 +105,7 @@ async def reserve_storage(token: str, size: int, ttl_seconds: int) -> None:
INSERT INTO storagereservation (token, size, expires_at)
SELECT {ph[0]}, {ph[1]}, {ph[2]}
WHERE (
COALESCE((SELECT SUM(size) FROM filecodes), 0)
COALESCE((SELECT SUM(size) FROM filecodes WHERE file_path IS NULL OR file_path != '{LOCAL_REF_MARKER}'), 0)
+ COALESCE((SELECT SUM(size) FROM storagereservation WHERE expires_at > {ph[3]}), 0)
+ {ph[4]}
) <= {ph[5]}
-1
View File
@@ -1,5 +1,4 @@
from pydantic import BaseModel
from typing import Optional
class SelectFileModel(BaseModel):
+415
View File
@@ -0,0 +1,415 @@
"""apps.base service layer: upload orchestration shared by share/chunk/presign.
Handlers in views.py keep HTTP concerns (form parsing, session lookup, status
codes for malformed requests, rate limiting); this module owns the business
workflow: quota reservation, storage writes, share-record creation, and
failure rollback/cleanup.
"""
import os
from fastapi import HTTPException, UploadFile
from fastapi.responses import FileResponse, Response, StreamingResponse
from core.logger import logger
from core.settings import settings
from core.storage import FileStorageInterface, StoredDownload, StoredFile, storages
from apps.base.file_validation import validate_upload_file
from apps.base.local_share import build_local_ref_download, is_local_ref
from apps.base.models import FileCodes, PresignUploadSession, UploadChunk
from apps.base.quota import release_storage, reserve_storage
from apps.base.utils import build_file_path, get_expire_info
import uuid
# 预签名上传会话有效期(秒)
PRESIGN_SESSION_EXPIRES = 900 # 15分钟
def stored_file_of(code: FileCodes) -> StoredFile:
"""Adapter: project an ORM FileCodes row onto the storage-layer contract."""
return StoredFile(
file_path=code.file_path,
uuid_file_name=code.uuid_file_name,
code=code.code,
prefix=code.prefix,
suffix=code.suffix,
text=code.text or "",
)
async def get_stored_download(file_code, file_storage: FileStorageInterface | None = None) -> StoredDownload:
if is_local_ref(file_code):
return build_local_ref_download(file_code)
storage = file_storage or storages[settings.file_storage]()
return await storage.get_file_response(stored_file_of(file_code))
async def rollback_saved_file(
storage: FileStorageInterface,
file_path: str,
uuid_file_name: str,
*,
context: str,
upload_id: str | None = None,
) -> None:
"""Best-effort delete of a stored file after a failed upload workflow.
Rollback failures are logged (with traceback) and swallowed: the original
error must propagate; the file, if left behind, is cleaned by the expiry
task once its orphaned record ages out.
"""
try:
await storage.delete_file(
StoredFile(file_path=file_path, uuid_file_name=uuid_file_name)
)
except Exception:
logger.warning(
"%s:回滚删除已保存文件失败%s",
context,
f" upload_id={upload_id}" if upload_id else "",
exc_info=True,
)
async def validate_file_size(file: UploadFile, max_size: int) -> int:
"""Return the upload's size, rejecting anything above max_size."""
size = file.size
if size is None:
# 无长度声明的上传(如 chunked 传输):必须走底层文件对象拿末尾偏移。
# 注意两层都不能用:UploadFile.seek 只接受单参数(TypeError);
# 底层 SpooledTemporaryFile.seek 是同步方法(不能 await)。
file.file.seek(0, 2)
size = file.file.tell()
file.file.seek(0)
if size > max_size:
max_size_mb = max_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"大小超过限制,最大为{max_size_mb:.2f} MB"
)
return size
def chunk_reservation_ttl() -> int:
ttl = max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600
return ttl
class FileUploadService:
"""统一的文件上传服务"""
@staticmethod
def _storage() -> FileStorageInterface:
return storages[settings.file_storage]()
@staticmethod
async def generate_file_path(
file_name: str, upload_id: str | None = None
) -> tuple[str, str, str, str, str]:
"""Delegates path generation to apps.base.utils.build_file_path."""
return await build_file_path(file_name, upload_id or uuid.uuid4().hex)
@staticmethod
async def create_file_record(
file_name: str,
file_size: int,
file_path: str,
expire_value: int,
expire_style: str,
**extra_fields,
) -> str:
"""统一创建FileCodes记录,返回code"""
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
prefix, suffix = os.path.splitext(file_name)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=file_name,
file_path=file_path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
**extra_fields,
)
return code
@staticmethod
async def create_text_share(
text: str, expire_value: int, expire_style: str
) -> str:
"""文本分享:配额预留 → 建分享记录 → 释放配额。"""
text_size = len(text.encode("utf-8"))
token = f"text:{uuid.uuid4().hex}"
await reserve_storage(token, text_size, ttl_seconds=300)
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await FileCodes.create(
code=code,
text=text,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
size=text_size,
prefix="Text",
)
finally:
await release_storage(token)
return code
@staticmethod
async def create_file_share(
file: UploadFile, *, size: int, expire_value: int, expire_style: str
) -> dict[str, str]:
"""文件分享:路径生成 → 配额预留 → 存储写入 → 建分享记录,失败回滚已存文件。"""
path, suffix, prefix, uuid_file_name, save_path = (
await FileUploadService.generate_file_path(file.filename or "")
)
token = f"file:{uuid.uuid4().hex}"
await reserve_storage(token, size, ttl_seconds=3600)
storage = FileUploadService._storage()
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await storage.save_file(file.file, save_path, file.content_type)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=uuid_file_name,
file_path=path,
size=size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
)
except Exception:
await rollback_saved_file(
storage, path, uuid_file_name, context="分享上传"
)
raise
finally:
await release_storage(token)
# UploadFile.filename 类型为 str | None;路径生成已用 "" 兜底,
# 响应处保持 str 以匹配 dict[str, str] 契约
return {"code": code, "name": file.filename or ""}
@staticmethod
async def complete_chunked_upload(
upload_id: str, chunk_info: UploadChunk, expire_value: int, expire_style: str
) -> dict[str, str]:
"""分片合并:配额 → 完整性/大小校验 → 合并 → 建分享记录 → 清理分片。
失败路径的配额释放与清理范围与原实现逐一对齐:
完整性校验失败仅抛 400(预留由 TTL 兜底);合并失败清理分片文件后抛 500。
"""
storage = FileUploadService._storage()
await reserve_storage(
f"chunk:{upload_id}", chunk_info.file_size, ttl_seconds=chunk_reservation_ttl()
)
completed_chunks = await UploadChunk.filter(
upload_id=upload_id, completed=True
).all()
if len(completed_chunks) != chunk_info.total_chunks:
raise HTTPException(400, "分片不完整")
# 用分片数 * chunk_size 校验最大可能大小
max_total_size = len(completed_chunks) * chunk_info.chunk_size
if max_total_size > settings.upload_size:
save_path = chunk_info.save_path
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning(
"分片超限中止:清理分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
403, f"实际上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
save_path = chunk_info.save_path
path = os.path.dirname(save_path) if save_path else ""
safe_file_name = os.path.basename(save_path) if save_path else ""
prefix, suffix = os.path.splitext(safe_file_name)
try:
records = {r.chunk_index: r for r in completed_chunks}
_, file_hash = await storage.merge_chunks(
upload_id,
chunk_info.total_chunks,
chunk_info.chunk_size,
save_path,
records,
)
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await FileCodes.create(
code=code,
file_hash=file_hash, # 使用合并后计算的哈希
is_chunked=True,
upload_id=upload_id,
size=chunk_info.file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
file_path=path,
uuid_file_name=safe_file_name,
prefix=prefix,
suffix=suffix,
)
await storage.clean_chunks(upload_id, save_path)
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
return {"code": code, "name": safe_file_name}
except ValueError as e:
raise HTTPException(400, str(e))
except Exception as e:
# 合并失败时清理临时文件
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning(
"分片合并失败:清理临时分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
raise HTTPException(500, f"文件合并失败: {str(e)}")
@staticmethod
async def commit_proxy_upload(
session: PresignUploadSession, file: UploadFile
) -> str:
"""预签名代理上传:配额 → 大小/类型/一致性校验 → 转存 → 建记录 → 会话清理。
校验失败不释放预留(与原实现一致,由 TTL 兜底)。
"""
await reserve_storage(
f"presign:{session.upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
if abs(file_size - session.file_size) > 1024:
raise HTTPException(400, "文件大小与声明不符")
storage = FileUploadService._storage()
try:
await storage.save_file(file.file, session.save_path, file.content_type)
except Exception as e:
raise HTTPException(500, f"文件保存失败: {str(e)}")
try:
code = await FileUploadService.create_file_record(
session.file_name,
file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
await rollback_saved_file(
storage,
os.path.dirname(session.save_path),
os.path.basename(session.save_path),
context="预签名代理上传:记录创建失败",
upload_id=session.upload_id,
)
raise
await session.delete()
await release_storage(f"presign:{session.upload_id}")
return code
@staticmethod
async def confirm_direct_upload(session: PresignUploadSession) -> str:
"""预签名直传确认:配额 → 文件存在性 → 建记录 → 会话清理。
预留失败说明配额已耗尽,此时清理远端临时文件与会话后原样抛出。
"""
try:
await reserve_storage(
f"presign:{session.upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
except HTTPException:
storage = FileUploadService._storage()
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
finally:
await session.delete()
await release_storage(f"presign:{session.upload_id}")
raise
storage = FileUploadService._storage()
if not await storage.file_exists(session.save_path):
raise HTTPException(404, "文件未上传或上传失败")
try:
code = await FileUploadService.create_file_record(
session.file_name,
session.file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
await rollback_saved_file(
storage,
os.path.dirname(session.save_path),
os.path.basename(session.save_path),
context="预签名确认:记录创建失败",
upload_id=session.upload_id,
)
raise
await session.delete()
await release_storage(f"presign:{session.upload_id}")
return code
def response_from_download(download: StoredDownload):
"""Build the starlette Response for a StoredDownload (view-layer duty)."""
if download.path is not None:
return FileResponse(
download.path,
media_type=download.media_type,
headers=download.headers,
filename=download.filename,
)
if download.content is not None:
return Response(
download.content, media_type=download.media_type, headers=download.headers
)
if download.stream_factory is None:
# 数据契约要求三者至少其一;防御性 500 语义
raise ValueError("StoredDownload 缺少可用的下载载荷")
return StreamingResponse(
download.stream_factory(),
media_type=download.media_type,
headers=download.headers,
background=download.background,
)
+694
View File
@@ -0,0 +1,694 @@
"""Setup wizard: form parsing rules and the two admin-facing HTML pages.
Pure presentation/logic — no router here; routes live in apps.base.pages.
"""
import html
from urllib.parse import parse_qs
from typing import Any
from fastapi import Request
from fastapi.responses import HTMLResponse
from core.settings import DEFAULT_CONFIG, settings
from core.version import APP_VERSION
FILE_SIZE_UNITS = {"KB": 1024, "MB": 1024**2, "GB": 1024**3}
SAVE_TIME_UNITS = {"second": 1, "minute": 60, "hour": 3600, "day": 86400}
EXPIRE_STYLE_OPTIONS = [
("day", "按天"),
("hour", "按小时"),
("minute", "按分钟"),
("forever", "永久"),
("count", "按取件次数"),
]
def normalize_public_flag(value) -> int:
if isinstance(value, str):
return int(value.strip().lower() in {"1", "true", "on", "yes"})
return int(bool(value))
def build_public_config() -> dict:
return {
"name": settings.name,
"description": settings.description,
"explain": settings.page_explain,
"upload_size": settings.upload_size,
"allowed_file_types": settings.allowed_file_types,
"expire_style": settings.expire_style,
"enable_chunk": settings.enable_chunk,
"open_upload": settings.open_upload,
"notify_title": settings.notify_title,
"notify_content": settings.notify_content,
"show_admin_address": normalize_public_flag(settings.show_admin_addr),
"max_save_seconds": settings.max_save_seconds,
}
def build_public_meta() -> dict:
return {
"version": APP_VERSION,
"api": {
"legacy_config": "/",
"public_config": "/api/v1/config",
"health": "/health",
},
"features": {
"chunk_upload": bool(settings.enable_chunk),
"guest_upload": bool(settings.open_upload),
"admin_address_visible": bool(normalize_public_flag(settings.show_admin_addr)),
"expiration_modes": settings.expire_style,
},
"limits": {
"upload_size": settings.upload_size,
"allowed_file_types": settings.allowed_file_types,
"max_save_seconds": settings.max_save_seconds,
"upload_window_minutes": settings.upload_minute,
"upload_window_count": settings.upload_count,
},
}
def get_form_value(data: dict[str, Any], key: str, default: str = "") -> str:
value = data.get(key, default)
if isinstance(value, list):
value = value[-1] if value else default
return str(value if value is not None else default)
def get_form_list(data: dict[str, Any], key: str) -> list[str]:
value = data.get(key, [])
if isinstance(value, list):
return [str(item) for item in value if str(item)]
if value:
return [str(value)]
return []
def normalize_bool_field(data: dict, key: str, default: bool) -> bool:
if key not in data:
return default
return get_form_value(data, key).lower() in {"1", "true", "on", "yes"}
def parse_int_field(
data: dict[str, Any],
key: str,
default: int,
label: str,
min_value: int = 0,
max_value: int | None = None,
) -> int:
raw_value = get_form_value(data, key, str(default)).strip()
try:
value = int(raw_value)
except ValueError:
raise ValueError(f"{label} 必须是整数")
if value < min_value:
raise ValueError(f"{label} 不能小于 {min_value}")
if max_value is not None and value > max_value:
raise ValueError(f"{label} 不能大于 {max_value}")
return value
def parse_allowed_file_types(value: str) -> list[str]:
items = [item.strip() for item in value.split(",") if item.strip()]
return items or ["*"]
def parse_setup_options(data: dict) -> dict:
upload_size_unit = get_form_value(data, "upload_size_unit", "MB").upper()
if upload_size_unit not in FILE_SIZE_UNITS:
raise ValueError("文件大小单位不正确")
upload_size_value = parse_int_field(
data, "upload_size_value", 10, "文件大小限制", min_value=1
)
save_time_unit = get_form_value(data, "save_time_unit", "day")
if save_time_unit not in SAVE_TIME_UNITS:
raise ValueError("最长保存时间单位不正确")
save_time_value = parse_int_field(
data, "save_time_value", 0, "最长保存时间", min_value=0
)
expire_styles = get_form_list(data, "expire_style")
valid_expire_styles = {style for style, _label in EXPIRE_STYLE_OPTIONS}
expire_styles = [style for style in expire_styles if style in valid_expire_styles]
if not expire_styles:
raise ValueError("至少需要选择一种过期方式")
code_generate_type = get_form_value(
data, "code_generate_type", DEFAULT_CONFIG["code_generate_type"]
)
if code_generate_type not in {"number", "secret"}:
raise ValueError("提取码类型不正确")
return {
"allowed_file_types": parse_allowed_file_types(
get_form_value(data, "allowed_file_types", "*")
),
"code_generate_type": code_generate_type,
"enable_chunk": int(normalize_bool_field(data, "enable_chunk", False)),
"error_count": parse_int_field(
data, "error_count", DEFAULT_CONFIG["error_count"], "取件错误次数限制", 1
),
"error_minute": parse_int_field(
data, "error_minute", DEFAULT_CONFIG["error_minute"], "取件错误检测窗口", 1
),
"login_count": parse_int_field(
data, "login_count", DEFAULT_CONFIG["login_count"], "登录失败次数限制", 1
),
"login_minute": parse_int_field(
data, "login_minute", DEFAULT_CONFIG["login_minute"], "登录失败检测窗口", 1
),
"expire_style": expire_styles,
"max_save_seconds": save_time_value * SAVE_TIME_UNITS[save_time_unit],
"open_upload": int(normalize_bool_field(data, "open_upload", True)),
"upload_count": parse_int_field(
data, "upload_count", DEFAULT_CONFIG["upload_count"], "上传次数限制", 1
),
"upload_minute": parse_int_field(
data, "upload_minute", DEFAULT_CONFIG["upload_minute"], "上传检测窗口", 1
),
"upload_size": upload_size_value * FILE_SIZE_UNITS[upload_size_unit],
}
def build_expire_style_inputs(selected_styles: list[str]) -> str:
inputs = []
selected = set(selected_styles)
for style, label in EXPIRE_STYLE_OPTIONS:
checked = " checked" if style in selected else ""
inputs.append(
f'<label class="check"><input type="checkbox" name="expire_style" value="{style}"{checked}> {label}</label>'
)
return "\n ".join(inputs)
def build_setup_page(error: str = "", form: dict | None = None) -> str:
form = form or {}
escaped_error = html.escape(error)
escaped_site_name = html.escape(
get_form_value(form, "site_name", DEFAULT_CONFIG["name"])
)
escaped_allowed_types = html.escape(get_form_value(form, "allowed_file_types", "*"))
upload_size_value = html.escape(get_form_value(form, "upload_size_value", "10"))
upload_size_unit = get_form_value(form, "upload_size_unit", "MB").upper()
save_time_value = html.escape(get_form_value(form, "save_time_value", "0"))
save_time_unit = get_form_value(form, "save_time_unit", "day")
upload_minute = html.escape(
get_form_value(form, "upload_minute", str(DEFAULT_CONFIG["upload_minute"]))
)
upload_count = html.escape(
get_form_value(form, "upload_count", str(DEFAULT_CONFIG["upload_count"]))
)
error_minute = html.escape(
get_form_value(form, "error_minute", str(DEFAULT_CONFIG["error_minute"]))
)
error_count = html.escape(
get_form_value(form, "error_count", str(DEFAULT_CONFIG["error_count"]))
)
login_minute = html.escape(
get_form_value(form, "login_minute", str(DEFAULT_CONFIG["login_minute"]))
)
login_count = html.escape(
get_form_value(form, "login_count", str(DEFAULT_CONFIG["login_count"]))
)
open_upload_checked = (
" checked" if normalize_bool_field(form, "open_upload", True) else ""
)
chunk_checked = (
" checked" if normalize_bool_field(form, "enable_chunk", False) else ""
)
code_generate_type = get_form_value(
form, "code_generate_type", DEFAULT_CONFIG["code_generate_type"]
)
selected_expire_styles = get_form_list(form, "expire_style") or list(
DEFAULT_CONFIG["expire_style"]
)
expire_style_inputs = build_expire_style_inputs(selected_expire_styles)
size_unit_options = "\n".join(
f'<option value="{unit}"{" selected" if unit == upload_size_unit else ""}>{unit}</option>'
for unit in FILE_SIZE_UNITS
)
save_time_unit_options = "\n".join(
f'<option value="{unit}"{" selected" if unit == save_time_unit else ""}>{label}</option>'
for unit, label in [
("second", "秒"),
("minute", "分钟"),
("hour", "小时"),
("day", "天"),
]
)
code_type_options = "\n".join(
f'<option value="{value}"{" selected" if value == code_generate_type else ""}>{label}</option>'
for value, label in [("number", "数字"), ("secret", "随机字符")]
)
error_block = (
f'<div class="alert" role="alert">{escaped_error}</div>' if escaped_error else ""
)
return f"""<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>初始化 FileCodeBox</title>
<style>
:root {{
color-scheme: light;
--bg: #f5f5f7;
--panel: rgba(255, 255, 255, .86);
--panel-strong: rgba(255, 255, 255, .96);
--text: #18181b;
--muted: #71717a;
--line: rgba(228, 228, 231, .9);
--line-strong: rgba(212, 212, 216, .95);
--primary: #18181b;
--primary-soft: #f4f4f5;
--danger-bg: #fef2f2;
--danger: #b91c1c;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0;
min-height: 100vh;
display: grid;
place-items: center;
padding: 14px;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: var(--bg);
color: var(--text);
}}
main {{
width: min(100%, 1040px);
padding: 14px;
border: 1px solid rgba(255, 255, 255, .8);
border-radius: 20px;
background: rgba(255, 255, 255, .62);
box-shadow: 0 22px 70px -34px rgba(24, 24, 27, .32);
backdrop-filter: blur(22px);
}}
.setup-header {{
display: flex;
align-items: center;
justify-content: space-between;
gap: 16px;
padding: 10px 10px 12px;
}}
.brand-mark {{
flex: 0 0 auto;
width: 40px;
height: 40px;
display: grid;
place-items: center;
border-radius: 14px;
background: #18181b;
color: #fff;
font-weight: 800;
letter-spacing: 0;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .42);
}}
.title-wrap {{
min-width: 0;
display: flex;
align-items: center;
gap: 12px;
}}
h1 {{
margin: 0;
font-size: 21px;
line-height: 1.25;
letter-spacing: 0;
font-weight: 750;
}}
p {{
margin: 2px 0 0;
color: var(--muted);
line-height: 1.35;
font-size: 13px;
}}
form {{
margin: 0;
padding: 12px;
border: 1px solid var(--line);
border-radius: 16px;
background: var(--panel);
}}
.panel-grid {{
display: grid;
grid-template-columns: 1.05fr 1fr 1fr;
gap: 10px;
}}
.panel {{
min-width: 0;
padding: 10px;
border: 1px solid var(--line);
border-radius: 14px;
background: rgba(255, 255, 255, .68);
}}
.panel-wide {{
grid-column: 1 / -1;
display: grid;
grid-template-columns: minmax(0, 1.3fr) minmax(0, .9fr);
gap: 12px;
align-items: end;
}}
.panel-title {{
margin: 0 0 2px;
color: #18181b;
font-size: 13px;
font-weight: 760;
}}
label {{
display: block;
margin: 8px 0 5px;
font-weight: 650;
font-size: 12px;
color: #3f3f46;
}}
input, select {{
width: 100%;
height: 34px;
border: 1px solid var(--line);
border-radius: 10px;
padding: 0 10px;
font: inherit;
color: var(--text);
outline: none;
background: rgba(255, 255, 255, .84);
transition: border-color .2s ease, box-shadow .2s ease, background-color .2s ease;
}}
input:focus, select:focus {{
border-color: #a1a1aa;
background: #fff;
box-shadow: 0 10px 22px -12px rgba(24, 24, 27, .2), 0 0 0 3px rgba(24, 24, 27, .06);
}}
.grid {{
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 6px 12px;
}}
.full {{ grid-column: 1 / -1; }}
.span-2 {{ grid-column: span 2; }}
.row {{
display: grid;
grid-template-columns: minmax(0, 1fr) 88px;
gap: 8px;
}}
.checks {{
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-top: 5px;
}}
.check {{
display: flex;
align-items: center;
gap: 8px;
min-height: 36px;
margin: 0;
padding: 0 9px;
border: 1px solid var(--line);
border-radius: 10px;
font-weight: 550;
font-size: 13px;
color: var(--text);
white-space: nowrap;
background: rgba(255, 255, 255, .72);
transition: border-color .2s ease, background-color .2s ease, box-shadow .2s ease;
}}
.check:hover {{
border-color: var(--line-strong);
background: #fff;
box-shadow: 0 8px 20px -8px rgba(0, 0, 0, .1);
}}
.check input {{
width: 16px;
height: 16px;
padding: 0;
box-shadow: none;
}}
.section-title {{
display: none;
}}
.help {{
margin: 4px 0 0;
font-size: 12px;
color: var(--muted);
}}
.alert {{
margin-bottom: 8px;
padding: 9px 12px;
border-radius: 12px;
background: var(--danger-bg);
color: var(--danger);
font-size: 13px;
}}
button {{
width: 100%;
height: 38px;
margin-top: 12px;
border: 0;
border-radius: 12px;
background: var(--primary);
color: white;
font: inherit;
font-weight: 700;
cursor: pointer;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .42);
transition: background-color .2s ease, box-shadow .2s ease;
}}
button:hover {{
background: #27272a;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .56);
}}
.hint {{
margin-top: 8px;
margin-bottom: 0;
font-size: 12px;
}}
.compact-help {{
display: inline;
margin-left: 6px;
color: var(--muted);
font-weight: 500;
font-size: 12px;
}}
.topline {{
display: none;
}}
.badge {{
min-height: 26px;
display: inline-flex;
align-items: center;
padding: 0 10px;
border: 1px solid var(--line);
border-radius: 999px;
background: rgba(255, 255, 255, .76);
color: #52525b;
font-size: 12px;
font-weight: 650;
white-space: nowrap;
}}
@media (max-width: 640px) {{
body {{ padding: 10px; }}
main {{ padding: 10px; }}
.setup-header {{ align-items: flex-start; padding: 6px 4px 10px; }}
.title-wrap {{ align-items: flex-start; }}
.panel-grid, .panel-wide {{ grid-template-columns: 1fr; }}
.grid, .checks {{ grid-template-columns: 1fr; }}
.span-2 {{ grid-column: auto; }}
.row {{ grid-template-columns: 1fr; }}
.badge {{ display: none; }}
}}
</style>
</head>
<body>
<main>
<div class="setup-header">
<div class="title-wrap">
<div class="brand-mark">FCB</div>
<div>
<h1>初始化 FileCodeBox</h1>
<p>首次配置管理员密码、上传限制和取件策略,后续可在后台调整。</p>
</div>
</div>
<span class="badge">首次配置向导</span>
</div>
<form method="post" action="/setup" autocomplete="off">
{error_block}
<div class="panel-grid">
<section class="panel">
<div class="panel-title">基础设置</div>
<label for="site_name">站点名称</label>
<input id="site_name" name="site_name" maxlength="80" value="{escaped_site_name}" placeholder="文件快递柜 - FileCodeBox">
<label for="admin_password">管理员密码</label>
<input id="admin_password" name="admin_password" type="password" minlength="8" required autofocus>
<label for="confirm_password">确认管理员密码</label>
<input id="confirm_password" name="confirm_password" type="password" minlength="8" required>
</section>
<section class="panel">
<div class="panel-title">上传设置</div>
<label for="upload_size_value">单文件大小限制</label>
<div class="row">
<input id="upload_size_value" name="upload_size_value" type="number" min="1" value="{upload_size_value}" required>
<select name="upload_size_unit" aria-label="文件大小单位">
{size_unit_options}
</select>
</div>
<label for="upload_count">上传频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="upload_count" name="upload_count" type="number" min="1" value="{upload_count}" required>
<input name="upload_minute" type="number" min="1" value="{upload_minute}" aria-label="上传检测窗口分钟" required>
</div>
<div class="grid">
<input type="hidden" name="open_upload" value="0">
<label class="check"><input type="checkbox" name="open_upload" value="1"{open_upload_checked}> 允许游客上传</label>
<input type="hidden" name="enable_chunk" value="0">
<label class="check"><input type="checkbox" name="enable_chunk" value="1"{chunk_checked}> 启用切片上传</label>
</div>
</section>
<section class="panel">
<div class="panel-title">取件与保存</div>
<label for="error_count">取件错误频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="error_count" name="error_count" type="number" min="1" value="{error_count}" required>
<input name="error_minute" type="number" min="1" value="{error_minute}" aria-label="取件错误检测窗口分钟" required>
</div>
<label for="login_count">管理员登录失败频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="login_count" name="login_count" type="number" min="1" value="{login_count}" required>
<input name="login_minute" type="number" min="1" value="{login_minute}" aria-label="登录失败检测窗口分钟" required>
</div>
<label for="save_time_value">最长保存时间</label>
<div class="row">
<input id="save_time_value" name="save_time_value" type="number" min="0" value="{save_time_value}" required>
<select name="save_time_unit" aria-label="最长保存时间单位">
{save_time_unit_options}
</select>
</div>
<label for="code_generate_type">提取码类型</label>
<select id="code_generate_type" name="code_generate_type">
{code_type_options}
</select>
</section>
<section class="panel panel-wide">
<div>
<div class="panel-title">可用策略</div>
<label>允许的过期方式</label>
<div class="checks">
{expire_style_inputs}
</div>
</div>
<div>
<label for="allowed_file_types">允许文件类型 <span class="compact-help">逗号分隔,* 表示不限制</span></label>
<input id="allowed_file_types" name="allowed_file_types" value="{escaped_allowed_types}" placeholder="* 或 .zip, image/*">
</div>
</section>
</div>
<button type="submit">完成初始化</button>
</form>
</main>
</body>
</html>"""
def build_setup_success_page() -> str:
return """<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="refresh" content="2;url=/#/admin">
<title>初始化完成</title>
<style>
body {
margin: 0;
min-height: 100vh;
display: grid;
place-items: center;
padding: 24px;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: #f6f8fb;
color: #172033;
}
main {
width: min(100%, 420px);
padding: 28px;
border: 1px solid #d9e1ec;
border-radius: 8px;
background: #fff;
text-align: center;
box-shadow: 0 18px 50px rgba(23, 32, 51, .08);
}
h1 { margin: 0 0 10px; font-size: 24px; letter-spacing: 0; }
p { margin: 0 0 22px; color: #60708a; line-height: 1.65; }
a {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 42px;
padding: 0 18px;
border-radius: 6px;
background: #2563eb;
color: white;
text-decoration: none;
font-weight: 700;
}
</style>
</head>
<body>
<main>
<h1>初始化完成</h1>
<p>管理员密码已设置,请使用刚才的密码登录后台。</p>
<a href="/#/admin">进入后台</a>
</main>
</body>
</html>"""
def setup_response(content: str, status_code: int = 200) -> HTMLResponse:
return HTMLResponse(
content=content,
status_code=status_code,
media_type="text/html",
headers={"Cache-Control": "no-store"},
)
def is_setup_path(path: str) -> bool:
return path.rstrip("/") == "/setup"
def wants_html_response(request: Request) -> bool:
if request.method not in {"GET", "HEAD"}:
return False
accept = request.headers.get("accept", "")
return not accept or "text/html" in accept or "*/*" in accept
async def read_setup_payload(request: Request) -> dict:
content_type = request.headers.get("content-type", "")
if "application/json" in content_type:
data = await request.json()
return data if isinstance(data, dict) else {}
body = (await request.body()).decode("utf-8")
return {
key: values if len(values) > 1 else values[-1]
for key, values in parse_qs(body).items()
}
+17 -15
View File
@@ -4,7 +4,6 @@
# @Software: PyCharm
import asyncio
import datetime
import logging
import os
from tortoise.expressions import Q
@@ -16,12 +15,14 @@ from apps.base.models import (
UploadChunk,
)
from apps.base.utils import ip_limit, get_chunk_file_path_name
from core.config import refresh_settings
from apps.base.config import refresh_settings
from core.logger import logger
from core.settings import settings, data_root
from core.storage import FileStorageInterface, storages
from core.storage import FileStorageInterface, StoredFile, storages
from apps.base.services import stored_file_of
from apps.base.local_share import is_local_ref
from core.utils import get_now
async def delete_expire_files():
while True:
try:
@@ -41,15 +42,16 @@ async def delete_expire_files():
).all()
for exp in expire_data:
try:
await file_storage.delete_file(exp)
if not is_local_ref(exp):
await file_storage.delete_file(stored_file_of(exp))
except Exception as e:
logging.error(f"删除过期文件失败 code={exp.code}: {e}")
logger.error(f"删除过期文件失败 code={exp.code}: {e}")
try:
await exp.delete()
except Exception as e:
logging.error(f"删除记录失败 code={exp.code}: {e}")
logger.error(f"删除记录失败 code={exp.code}: {e}")
except Exception as e:
logging.error(e)
logger.error(e)
finally:
await asyncio.sleep(600)
@@ -75,7 +77,7 @@ async def clean_incomplete_uploads():
)
await file_storage.clean_chunks(session.upload_id, save_path)
except Exception as e:
logging.error(
logger.error(
f"清理分片文件失败 upload_id={session.upload_id}: {e}"
)
@@ -84,14 +86,14 @@ async def clean_incomplete_uploads():
await StorageReservation.filter(
token=f"chunk:{session.upload_id}"
).delete()
logging.info(f"已清理过期上传会话 upload_id={session.upload_id}")
logger.info(f"已清理过期上传会话 upload_id={session.upload_id}")
except Exception as e:
logging.error(
logger.error(
f"删除分片记录失败 upload_id={session.upload_id}: {e}"
)
except Exception as e:
logging.error(f"清理未完成上传任务异常: {e}")
logger.error(f"清理未完成上传任务异常: {e}")
finally:
await asyncio.sleep(3600)
@@ -110,13 +112,13 @@ async def clean_expired_presign_sessions():
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
FileCodes(
StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception as e:
logging.error(
logger.error(
"清理过期直传文件失败 "
f"upload_id={session.upload_id}: {e}"
)
@@ -125,6 +127,6 @@ async def clean_expired_presign_sessions():
).delete()
await session.delete()
except Exception as e:
logging.error(f"清理过期预签名会话异常: {e}")
logger.error(f"清理过期预签名会话异常: {e}")
finally:
await asyncio.sleep(900)
+20 -28
View File
@@ -1,5 +1,4 @@
import datetime
import hashlib
import os
import uuid
from urllib.parse import unquote
@@ -21,33 +20,37 @@ from core.utils import (
def validate_expire_style(expire_style: str) -> str:
"""校验过期方式是否在管理员配置的白名单内。"""
if expire_style not in settings.expireStyle:
if expire_style not in settings.expire_style:
raise HTTPException(status_code=400, detail="过期时间类型错误")
return expire_style
async def get_file_path_name(file: UploadFile) -> Tuple[str, str, str, str, str]:
async def build_file_path(
file_name: str, file_uuid: str
) -> Tuple[str, str, str, str, str]:
"""Single source of storage path generation (date dir + UUID), shared by
regular, chunked, and presigned uploads.
Always use get_now() (UTC+8); do not switch to server-local time.
"""
today = await get_now()
storage_path = settings.storage_path.strip("/")
file_uuid = uuid.uuid4().hex
filename = await sanitize_filename(unquote(file.filename or ""))
filename = await sanitize_filename(unquote(file_name or ""))
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{file_uuid}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(filename)
save_path = f"{path}/{filename}"
save_path = f"{path}/{prefix}{suffix}"
return path, suffix, prefix, filename, save_path
async def get_file_path_name(file: UploadFile) -> Tuple[str, str, str, str, str]:
return await build_file_path(file.filename or "", uuid.uuid4().hex)
async def get_chunk_file_path_name(
file_name: str, upload_id: str
) -> Tuple[str, str, str, str, str]:
today = await get_now()
storage_path = settings.storage_path.strip("/")
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{upload_id}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(file_name)
save_path = f"{path}/{prefix}{suffix}"
return path, suffix, prefix, file_name, save_path
return await build_file_path(file_name, upload_id)
async def get_expire_info(
@@ -120,20 +123,9 @@ async def get_random_code(style: str | None = None) -> str:
return str(code)
async def calculate_file_hash(file: UploadFile, chunk_size=1024 * 1024) -> str:
sha = hashlib.sha256()
await file.seek(0)
while True:
chunk = await file.read(chunk_size)
if not chunk:
break
sha.update(chunk)
await file.seek(0)
return sha.hexdigest()
ip_limit = {
"error": IPRateLimit(count=settings.errorCount, minutes=settings.errorMinute),
"metadata": IPRateLimit(count=settings.errorCount, minutes=settings.errorMinute),
"upload": IPRateLimit(count=settings.uploadCount, minutes=settings.uploadMinute),
"error": IPRateLimit(count=settings.error_count, minutes=settings.error_minute),
"metadata": IPRateLimit(count=settings.error_count, minutes=settings.error_minute),
"upload": IPRateLimit(count=settings.upload_count, minutes=settings.upload_minute),
"login": IPRateLimit(count=settings.login_count, minutes=settings.login_minute),
}
+107 -350
View File
@@ -1,6 +1,6 @@
import datetime
from fnmatch import fnmatchcase
import hashlib
import asyncio
import hmac
import os
import uuid
from datetime import timedelta
@@ -11,25 +11,36 @@ from typing import Optional, Tuple, Union
from fastapi import APIRouter, Form, Request, UploadFile, File, Depends, HTTPException
from pydantic import BaseModel, ValidationError
from starlette import status
from starlette.responses import Response
from starlette.responses import HTMLResponse, Response
from tortoise.expressions import Case, F, Q, When
from apps.admin.dependencies import share_required_login
from apps.base.auth import share_required_login
from apps.base.models import FileCodes, UploadChunk, PresignUploadSession
from apps.base.quota import release_storage, reserve_storage
from apps.base.services import (
PRESIGN_SESSION_EXPIRES,
FileUploadService,
get_stored_download,
response_from_download,
stored_file_of,
validate_file_size,
)
from core.storage import StoredFile
from core.logger import logger
from apps.base.schemas import (
SelectFileModel,
InitChunkUploadModel,
CompleteUploadModel,
PresignUploadInitRequest,
)
from apps.base.file_validation import validate_file_type, validate_upload_file, validate_header_bytes
from apps.base.utils import (
get_expire_info,
get_file_path_name,
ip_limit,
get_chunk_file_path_name,
validate_expire_style,
)
from apps.base.local_share import is_local_ref
from apps.base.mcp import sanitize_html_preview
from core.response import APIResponse
from core.settings import settings
from core.storage import storages, FileStorageInterface
@@ -44,102 +55,6 @@ share_api = APIRouter(prefix="/share", tags=["分享"])
# ============ 公共服务层 ============
class FileUploadService:
"""统一的文件上传服务"""
@staticmethod
async def generate_file_path(
file_name: str, upload_id: Optional[str] = None
) -> tuple[str, str, str, str, str]:
"""统一的路径生成"""
today = datetime.datetime.now()
storage_path = settings.storage_path.strip("/")
file_uuid = upload_id or uuid.uuid4().hex
filename = await sanitize_filename(unquote(file_name))
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{file_uuid}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(filename)
save_path = f"{path}/{filename}"
return path, suffix, prefix, filename, save_path
@staticmethod
async def create_file_record(
file_name: str,
file_size: int,
file_path: str,
expire_value: int,
expire_style: str,
**extra_fields,
) -> str:
"""统一创建FileCodes记录,返回code"""
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
prefix, suffix = os.path.splitext(file_name)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=file_name,
file_path=file_path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
**extra_fields,
)
return code
async def validate_file_size(file: UploadFile, max_size: int) -> int:
size = file.size
if size is None:
await file.seek(0, 2) # type: ignore[arg-type]
size = file.file.tell()
await file.seek(0)
if size > max_size:
max_size_mb = max_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"大小超过限制,最大为{max_size_mb:.2f} MB"
)
return size
def normalize_allowed_file_types() -> list[str]:
raw_value = settings.allowed_file_types
if isinstance(raw_value, str):
values = raw_value.replace(";", ",").split(",")
elif isinstance(raw_value, (list, tuple, set)):
values = raw_value
else:
values = []
allowed = [str(item).strip().lower() for item in values if str(item).strip()]
return allowed or ["*"]
def validate_file_type(file_name: str, content_type: Optional[str] = None) -> None:
allowed = normalize_allowed_file_types()
if "*" in allowed or "*/*" in allowed:
return
normalized_name = file_name.strip().lower()
normalized_content_type = (content_type or "").strip().lower()
for rule in allowed:
if "/" in rule and fnmatchcase(normalized_content_type, rule):
return
extension = rule if rule.startswith(".") else f".{rule}"
if normalized_name.endswith(extension):
return
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
async def create_file_code(code, **kwargs):
return await FileCodes.create(code=code, **kwargs)
def normalize_share_code(code: str) -> str:
@@ -159,23 +74,7 @@ async def share_text(
if text_size > max_txt_size:
raise HTTPException(status_code=403, detail="内容过多,建议采用文件形式")
reservation_token = f"text:{uuid.uuid4().hex}"
await reserve_storage(reservation_token, text_size, ttl_seconds=300)
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await create_file_code(
code=code,
text=text,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
size=text_size,
prefix="Text",
)
finally:
await release_storage(reservation_token)
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code})
@@ -187,41 +86,14 @@ async def share_file(
file: UploadFile = File(...),
ip: str = Depends(ip_limit["upload"]),
):
file_size = await validate_file_size(file, settings.uploadSize)
validate_file_type(file.filename or "", file.content_type)
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
validate_expire_style(expire_style)
path, suffix, prefix, uuid_file_name, save_path = await get_file_path_name(file)
reservation_token = f"file:{uuid.uuid4().hex}"
await reserve_storage(reservation_token, file_size, ttl_seconds=3600)
file_storage: FileStorageInterface = storages[settings.file_storage]()
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await file_storage.save_file(file, save_path)
await create_file_code(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=uuid_file_name,
file_path=path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
)
except Exception:
try:
await file_storage.delete_file(
FileCodes(file_path=path, uuid_file_name=uuid_file_name)
)
except Exception:
pass
raise
finally:
await release_storage(reservation_token)
detail = await FileUploadService.create_file_share(
file, size=file_size, expire_value=expire_value, expire_style=expire_style
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": file.filename})
return APIResponse(detail=detail)
async def get_code_file_by_code(
@@ -272,7 +144,6 @@ def build_file_metadata(file_code: FileCodes) -> dict:
"is_text": is_text,
"created_at": file_code.created_at,
"expired_at": file_code.expired_at,
"expires_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"remaining_downloads": remaining_downloads,
@@ -285,11 +156,11 @@ async def build_select_detail(
metadata = build_file_metadata(file_code)
if file_code.text is not None:
download_url = None
elif file_code.expired_count >= 0:
# 有次数限制的文件必须经过下载接口,第三方直链无法阻止重复使用。
elif file_code.expired_count >= 0 or is_local_ref(file_code):
# 次数限制与 NAS 引用都必须走下载接口,避免直链绕过次数或打到错误存储。
download_url = await get_proxy_file_url(file_code.code)
else:
download_url = await file_storage.get_file_url(file_code)
download_url = await file_storage.get_file_url(stored_file_of(file_code))
content = file_code.text if file_code.text is not None else None
return {
**metadata,
@@ -325,6 +196,51 @@ async def post_file_metadata(
return APIResponse(detail=build_file_metadata(file_code))
@share_api.get("/preview/html/{code}", include_in_schema=False)
async def preview_html(code: str):
has, file_code = await get_code_file_by_code(code)
if not has or not isinstance(file_code, FileCodes):
raise HTTPException(status_code=404, detail="文件不存在")
if file_code.text is not None or (file_code.suffix or "").lower() not in {".html", ".htm"}:
raise HTTPException(status_code=415, detail="仅支持 HTML 文件预览")
preview_limit = 2 * 1024 * 1024
if file_code.size > preview_limit:
raise HTTPException(status_code=413, detail="HTML 文件过大,无法在线预览")
download = await get_stored_download(file_code)
if download.path is not None:
if download.path.stat().st_size > preview_limit:
raise HTTPException(status_code=413, detail="HTML 文件过大,无法在线预览")
content = await asyncio.to_thread(download.path.read_bytes)
elif download.content is not None:
content = download.content
elif download.stream_factory is not None:
stream = download.stream_factory()
if hasattr(stream, "__aiter__"):
chunks = bytearray()
async for chunk in stream:
chunks.extend(chunk)
if len(chunks) > preview_limit:
raise HTTPException(status_code=413, detail="HTML 文件过大,无法在线预览")
content = bytes(chunks)
else:
raise HTTPException(status_code=500, detail="HTML 预览读取失败")
else:
raise HTTPException(status_code=500, detail="HTML 预览读取失败")
if len(content) > preview_limit:
raise HTTPException(status_code=413, detail="HTML 文件过大,无法在线预览")
safe_html = sanitize_html_preview(content.decode("utf-8-sig", errors="replace"))
return HTMLResponse(
safe_html,
headers={
"Content-Security-Policy": "sandbox; default-src 'none'; img-src data:; style-src 'unsafe-inline'; font-src data:; base-uri 'none'; form-action 'none'",
"X-Content-Type-Options": "nosniff",
"Cache-Control": "private, no-store",
"Content-Disposition": 'inline; filename="preview.html"',
},
background=download.background,
)
@share_api.get("/select/")
async def get_code_file(code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
@@ -347,7 +263,7 @@ async def get_code_file(code: str, ip: str = Depends(ip_limit["error"])):
)
},
)
return await file_storage.get_file_response(file_code)
return response_from_download(await get_stored_download(file_code, file_storage))
@share_api.post("/select/")
@@ -375,12 +291,16 @@ async def select_file(data: SelectFileModel, ip: str = Depends(ip_limit["error"]
async def download_file(key: str, code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
normalized_code = normalize_share_code(code)
# 同时接受当前窗口与上一窗口 token,避免时间窗边界竞态导致偶发 403
valid_keys = {
# 同时接受当前窗口与上一窗口 token,避免时间窗边界竞态导致偶发 403。
# 逐个常量时间比较(hmac.compare_digest),避免 set 成员判断的时序侧信道。
valid_keys = [
await get_select_token(normalized_code, offset=0),
await get_select_token(normalized_code, offset=1),
}
if key not in valid_keys:
]
if not any(
hmac.compare_digest(key.encode(), candidate.encode())
for candidate in valid_keys
):
ip_limit["error"].add_ip(ip)
raise HTTPException(status_code=403, detail="下载鉴权失败")
has, file_code = await get_code_file_by_code(normalized_code)
@@ -392,7 +312,7 @@ async def download_file(key: str, code: str, ip: str = Depends(ip_limit["error"]
return (
APIResponse(detail=file_code.text)
if file_code.text
else await file_storage.get_file_response(file_code)
else response_from_download(await get_stored_download(file_code, file_storage))
)
@@ -423,37 +343,23 @@ async def parse_complete_upload(request: Request) -> CompleteUploadModel:
@chunk_api.post("/upload/init/", dependencies=[Depends(share_required_login)])
async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chunk_upload)):
validate_file_type(data.file_name)
safe_file_name = await sanitize_filename(unquote(data.file_name or ""))
validate_file_type(safe_file_name)
# 服务端校验:根据 total_chunks * chunk_size 计算理论最大上传量
total_chunks = (data.file_size + data.chunk_size - 1) // data.chunk_size
max_possible_size = total_chunks * data.chunk_size
if max_possible_size > settings.uploadSize:
max_size_mb = settings.uploadSize / (1024 * 1024)
if max_possible_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"文件大小超过限制,最大为 {max_size_mb:.2f} MB"
)
# # 秒传检查
# existing = await FileCodes.filter(file_hash=data.file_hash).first()
# if existing:
# if await existing.is_expired():
# file_storage: FileStorageInterface = storages[settings.file_storage](
# )
# await file_storage.delete_file(existing)
# await existing.delete()
# else:
# return APIResponse(detail={
# "code": existing.code,
# "existed": True,
# "name": f'{existing.prefix}{existing.suffix}'
# })
# 断点续传:检查是否存在相同文件的未完成上传会话
existing_session = await UploadChunk.filter(
chunk_hash=data.file_hash,
chunk_index=-1,
file_size=data.file_size,
file_name=data.file_name,
file_name=safe_file_name,
).first()
if existing_session:
@@ -489,7 +395,9 @@ async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chun
await reserve_storage(
reservation_token, data.file_size, ttl_seconds=chunk_expire_seconds
)
_, _, _, _, save_path = await get_chunk_file_path_name(data.file_name, upload_id)
_, _, _, safe_file_name, save_path = await get_chunk_file_path_name(
data.file_name, upload_id
)
try:
await UploadChunk.create(
upload_id=upload_id,
@@ -498,7 +406,7 @@ async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chun
file_size=data.file_size,
chunk_size=data.chunk_size,
chunk_hash=data.file_hash,
file_name=data.file_name,
file_name=safe_file_name,
save_path=save_path,
)
except Exception:
@@ -549,6 +457,8 @@ async def upload_chunk(
# 读取分片数据并计算哈希
chunk_data = await chunk.read()
if chunk_index == 0:
validate_header_bytes(chunk_info.file_name, None, chunk_data[:64])
chunk_size = len(chunk_data)
# 校验分片大小不超过声明的 chunk_size
@@ -564,8 +474,8 @@ async def upload_chunk(
).count()
# 已上传分片的最大可能大小 + 当前分片
max_uploaded_size = uploaded_count * chunk_info.chunk_size + chunk_size
if max_uploaded_size > settings.uploadSize:
max_size_mb = settings.uploadSize / (1024 * 1024)
if max_uploaded_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"累计上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
@@ -616,8 +526,8 @@ async def cancel_upload(upload_id: str):
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception as e:
pass
except Exception:
logger.warning("取消分片上传:清理分片文件失败 upload_id=%s", upload_id, exc_info=True)
# 清理数据库记录
await UploadChunk.filter(upload_id=upload_id).delete()
@@ -666,86 +576,16 @@ async def complete_upload(
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
validate_expire_style(data.expire_style)
await reserve_storage(
f"chunk:{upload_id}",
chunk_info.file_size,
ttl_seconds=max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600,
detail = await FileUploadService.complete_chunked_upload(
upload_id, chunk_info, data.expire_value, data.expire_style
)
storage = storages[settings.file_storage]()
# 验证所有分片
completed_chunks_list = await UploadChunk.filter(
upload_id=upload_id, completed=True
).all()
if len(completed_chunks_list) != chunk_info.total_chunks:
raise HTTPException(status.HTTP_400_BAD_REQUEST, detail="分片不完整")
# 用分片数 * chunk_size 校验最大可能大小
max_total_size = len(completed_chunks_list) * chunk_info.chunk_size
if max_total_size > settings.uploadSize:
save_path = chunk_info.save_path
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
pass
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
max_size_mb = settings.uploadSize / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"实际上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
save_path = chunk_info.save_path
path = os.path.dirname(save_path) if save_path else ""
prefix, suffix = os.path.splitext(chunk_info.file_name)
try:
# 合并文件并计算哈希
_, file_hash = await storage.merge_chunks(upload_id, chunk_info, save_path)
# 创建文件记录
expired_at, expired_count, used_count, code = await get_expire_info(
data.expire_value, data.expire_style
)
await FileCodes.create(
code=code,
file_hash=file_hash, # 使用合并后计算的哈希
is_chunked=True,
upload_id=upload_id,
size=chunk_info.file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
file_path=path,
uuid_file_name=f"{prefix}{suffix}",
prefix=prefix,
suffix=suffix,
)
# 清理临时文件
await storage.clean_chunks(upload_id, save_path)
# 清理数据库中的分片记录
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": chunk_info.file_name})
except ValueError as e:
raise HTTPException(status.HTTP_400_BAD_REQUEST, detail=str(e))
except Exception as e:
# 合并失败时清理临时文件
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
pass
raise HTTPException(
status.HTTP_500_INTERNAL_SERVER_ERROR, detail=f"文件合并失败: {str(e)}"
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail=detail)
# ============ 预签名上传API ============
presign_api = APIRouter(prefix="/presign", tags=["预签名上传"])
PRESIGN_SESSION_EXPIRES = 900 # 15分钟
def build_proxy_upload_urls(upload_id: str) -> dict:
proxy_upload_url = f"/presign/upload/proxy/{upload_id}"
@@ -777,10 +617,10 @@ async def presign_upload_init(
):
"""初始化预签名上传,S3返回直传URL,其他存储返回代理URL"""
validate_file_type(data.file_name)
if data.file_size > settings.uploadSize:
if data.file_size > settings.upload_size:
raise HTTPException(
403,
f"文件大小超过限制,最大为 {settings.uploadSize / (1024 * 1024):.2f} MB",
f"文件大小超过限制,最大为 {settings.upload_size / (1024 * 1024):.2f} MB",
)
validate_expire_style(data.expire_style)
@@ -837,44 +677,7 @@ async def presign_upload_proxy(
):
"""代理模式上传,服务器转存到存储后端"""
session = await _get_valid_session(upload_id, expected_mode="proxy")
await reserve_storage(
f"presign:{upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
file_size = await validate_file_size(file, settings.uploadSize)
if abs(file_size - session.file_size) > 1024:
raise HTTPException(400, "文件大小与声明不符")
storage: FileStorageInterface = storages[settings.file_storage]()
try:
await storage.save_file(file, session.save_path)
except Exception as e:
raise HTTPException(500, f"文件保存失败: {str(e)}")
try:
code = await FileUploadService.create_file_record(
session.file_name,
file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
try:
await storage.delete_file(
FileCodes(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception:
pass
raise
await session.delete()
await release_storage(f"presign:{upload_id}")
code = await FileUploadService.commit_proxy_upload(session, file)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@@ -885,53 +688,7 @@ async def presign_upload_proxy(
async def presign_upload_confirm(upload_id: str, ip: str = Depends(ip_limit["upload"])):
"""直传确认,客户端完成S3直传后调用获取分享码"""
session = await _get_valid_session(upload_id, expected_mode="direct")
try:
await reserve_storage(
f"presign:{upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
except HTTPException:
storage: FileStorageInterface = storages[settings.file_storage]()
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
FileCodes(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
finally:
await session.delete()
await release_storage(f"presign:{upload_id}")
raise
storage: FileStorageInterface = storages[settings.file_storage]()
if not await storage.file_exists(session.save_path):
raise HTTPException(404, "文件未上传或上传失败")
try:
code = await FileUploadService.create_file_record(
session.file_name,
session.file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
try:
await storage.delete_file(
FileCodes(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception:
pass
raise
await session.delete()
await release_storage(f"presign:{upload_id}")
code = await FileUploadService.confirm_direct_upload(session)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@@ -969,13 +726,13 @@ async def presign_upload_cancel(upload_id: str):
storage: FileStorageInterface = storages[settings.file_storage]()
try:
if await storage.file_exists(session.save_path):
temp_file_code = FileCodes(
temp_file_code = StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
await storage.delete_file(temp_file_code)
except Exception:
pass
logger.warning("取消预签名会话:清理临时文件失败 upload_id=%s", upload_id, exc_info=True)
await session.delete()
await release_storage(f"presign:{upload_id}")
+2 -2
View File
@@ -47,7 +47,7 @@ def _lock_file(file_obj: IO[str]) -> None:
if os.fstat(file_obj.fileno()).st_size == 0:
file_obj.write("0")
file_obj.flush()
msvcrt.locking(file_obj.fileno(), msvcrt.LK_LOCK, 1)
msvcrt.locking(file_obj.fileno(), msvcrt.LK_LOCK, 1) # type: ignore[attr-defined]
else:
import fcntl
@@ -58,7 +58,7 @@ def _unlock_file(file_obj: IO[str]) -> None:
if os.name == "nt":
import msvcrt
msvcrt.locking(file_obj.fileno(), msvcrt.LK_UNLCK, 1)
msvcrt.locking(file_obj.fileno(), msvcrt.LK_UNLCK, 1) # type: ignore[attr-defined]
else:
import fcntl
+13
View File
@@ -0,0 +1,13 @@
"""Framework-neutral errors raised by storage backends.
main.py registers an exception handler that renders these exactly like
FastAPI's HTTPException responses ({"detail": ...} with the same status
code), so callers need no per-site wrapping while core/ stays framework-free.
"""
class StorageError(Exception):
def __init__(self, status_code: int, detail: str):
self.status_code = status_code
self.detail = detail
super().__init__(detail)
+46 -12
View File
@@ -1,26 +1,60 @@
import logging
import os
import sys
PRODUCTION_ENVIRONMENTS = {"production", "prod"}
LOG_LEVELS = {
"debug": logging.DEBUG,
"info": logging.INFO,
"warning": logging.WARNING,
"error": logging.ERROR,
"critical": logging.CRITICAL,
}
def is_production_environment() -> bool:
environment = os.getenv("APP_ENV", "development").strip().lower()
return environment in PRODUCTION_ENVIRONMENTS
def get_log_level_name() -> str:
configured_level = os.getenv("LOG_LEVEL", "").strip().lower()
if configured_level in LOG_LEVELS:
return configured_level
return "warning" if is_production_environment() else "info"
def get_log_level() -> int:
return LOG_LEVELS[get_log_level_name()]
def is_access_log_enabled() -> bool:
configured_value = os.getenv("ACCESS_LOG")
if configured_value is not None:
return configured_value.strip().lower() in {"1", "true", "yes", "on"}
return not is_production_environment()
def setup_logger():
# 创建logger对象
_logger = logging.getLogger('FileCodeBox')
_logger.setLevel(logging.INFO)
_logger = logging.getLogger("FileCodeBox")
level = get_log_level()
_logger.setLevel(level)
_logger.propagate = False
# 创建控制台处理器
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(logging.INFO)
if not _logger.handlers:
console_handler = logging.StreamHandler(sys.stdout)
_logger.addHandler(console_handler)
else:
console_handler = _logger.handlers[0]
console_handler.setLevel(level)
# 设置日志格式
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s',
datefmt='%Y-%m-%d %H:%M:%S'
"%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
console_handler.setFormatter(formatter)
# 添加处理器到logger
_logger.addHandler(console_handler)
return _logger
-9
View File
@@ -1,7 +1,3 @@
# @Time : 2023/8/14 11:48
# @Author : Lan
# @File : response.py
# @Software: PyCharm
from typing import Generic, Optional, TypeVar
from pydantic import BaseModel
@@ -12,9 +8,4 @@ T = TypeVar("T")
class APIResponse(BaseModel, Generic[T]):
code: int = 200
message: str = "ok"
msg: Optional[str] = None
detail: Optional[T] = None
def model_post_init(self, __context) -> None:
if self.msg is None:
self.msg = self.message
+112
View File
@@ -1,7 +1,11 @@
import copy
import ipaddress
import os
import secrets
import socket
from dataclasses import dataclass
from typing import Any
from urllib.parse import urlsplit
from core.utils import hash_password, is_password_hashed, verify_password
@@ -29,9 +33,18 @@ def is_valid_jwt_secret(secret: Any) -> bool:
def is_config_initialized(config: dict[str, Any]) -> bool:
"""Cheap initialization probe — runs on EVERY request via the middleware.
Must never run a slow hash: a scrypt-stored token cannot be the legacy
default (which only ever existed as plaintext or sha256), so those tokens
are initialized without any verification. Only plaintext/sha256 tokens go
through the legacy-default comparison, which is microseconds.
"""
admin_token = str(config.get("admin_token") or "")
if not admin_token:
return False
if is_password_hashed(admin_token) and admin_token.startswith("scrypt$"):
return True
return not verify_password(LEGACY_DEFAULT_ADMIN_TOKEN, admin_token)
@@ -58,3 +71,102 @@ def prepare_security_config(config: dict[str, Any]) -> SecurityConfigResult:
result.changed = True
return result
# 出站端点白名单/黑名单(SSRF 防护)。
# 这些配置项会被服务器直接当作上游地址访问:被劫持的管理员会话可以把实例
# 变成内网代理跳板,或把数据发往任意地址。校验只在写入入口(update_config),
# 已存值不受影响;APP_ENV 非 production(本地开发,如 minio/webdav)时放行。
OUTBOUND_ENDPOINT_CONFIG_KEYS = ("s3_endpoint_url", "s3_hostname", "webdav_url")
_ENDPOINT_SCHEME_WHITELIST = {"http", "https"}
_ENDPOINT_HOST_DENY_SUFFIXES = (
".local",
".internal",
".lan",
".home.arpa",
".corp",
".localhost",
)
def _endpoint_ip_is_denied(ip_str: str) -> bool:
try:
ip = ipaddress.ip_address(ip_str)
except ValueError:
return False
return (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_reserved
or ip.is_multicast
or ip.is_unspecified
)
def _endpoint_host_is_denied(host: str) -> bool:
host = host.strip("[]").lower()
if host in {"localhost"} or host.endswith(_ENDPOINT_HOST_DENY_SUFFIXES):
return True
# 字面 IP 直接判;非字面形态(速记 "127.1"、十进制 2130706433、十六进制、
# *.localhost、nip.io 类 DNS 映射)交给解析后逐 IP 复判——静态字符串
# 黑名单对它们全部失效,这是探测中实测确认的绕过面。
if _endpoint_ip_is_denied(host):
return True
try:
infos = socket.getaddrinfo(host, None)
except (socket.gaierror, UnicodeError, OSError):
return False
for info in infos:
# getaddrinfo 的 sockaddr 首元素在 IPv4/IPv6 下均为 str(int 仅出现在
# AF_UNIX 等异形族)——显式 str() 让类型检查器与运行时一致。
if _endpoint_ip_is_denied(str(info[4][0])):
return True
return False
def validate_outbound_endpoint(value: Any) -> str:
"""校验完整 URL 形态的出站端点(s3_endpoint_url / webdav_url)。"""
endpoint = str(value or "").strip()
if not endpoint:
return ""
if os.environ.get("APP_ENV", "development") != "production":
return endpoint
parts = urlsplit(endpoint)
if parts.scheme.lower() not in _ENDPOINT_SCHEME_WHITELIST:
raise ValueError(f"端点协议必须是 http(s):{endpoint}")
host = parts.hostname or ""
if not host:
raise ValueError(f"端点缺少主机名:{endpoint}")
if _endpoint_host_is_denied(host):
raise ValueError(f"端点不允许指向内网/保留地址:{endpoint}")
return endpoint
def validate_outbound_hostname(value: Any) -> str:
"""校验裸主机名形态的出站端点(s3_hostname,存储层拼接 https://{hostname})。"""
hostname = str(value or "").strip()
if not hostname:
return ""
if os.environ.get("APP_ENV", "development") != "production":
return hostname
if "://" in hostname or "/" in hostname or "@" in hostname:
raise ValueError(f"s3_hostname 应为裸主机名,不含协议、路径或 userinfo:{hostname}")
# 取主机部分:[v6] 形态取括号内;host:port 取冒号前;裸 IPv6(多个冒号)整体
if "[" in hostname:
host = hostname.split("[", 1)[1].split("]", 1)[0]
rest = hostname.split("]", 1)[1] if "]" in hostname else ""
if rest and not rest.startswith(":"):
raise ValueError(f"s3_hostname 的 IPv6 括号后只允许端口:{hostname}")
elif hostname.count(":") == 1:
host = hostname.split(":")[0]
else:
host = hostname
if not host or _endpoint_host_is_denied(host):
raise ValueError(f"端点不允许指向内网/保留地址:{hostname}")
return hostname
+33 -19
View File
@@ -1,3 +1,4 @@
from typing import Any
# @Time : 2023/8/15 09:51
# @Author : Lan
# @File : settings.py
@@ -14,10 +15,10 @@ data_root = BASE_DIR / "data"
if not data_root.exists():
data_root.mkdir(parents=True, exist_ok=True)
DEFAULT_CONFIG = {
DEFAULT_CONFIG: dict[str, Any] = {
"file_storage": "local",
"storage_path": "",
"storageLimit": 0,
"storage_limit": 0,
"name": "文件快递柜 - FileCodeBox",
"description": "开箱即用的文件快传系统",
"notify_title": "系统通知",
@@ -45,21 +46,22 @@ DEFAULT_CONFIG = {
"webdav_proxy": 0,
"admin_token": "",
"jwt_secret": "",
"adminSessionExpire": ADMIN_SESSION_EXPIRE_DEFAULT,
"openUpload": 1,
"uploadSize": 1024 * 1024 * 10,
"admin_session_expire": ADMIN_SESSION_EXPIRE_DEFAULT,
"open_upload": 1,
"mcp_require_token": 0,
"upload_size": 1024 * 1024 * 10,
"allowed_file_types": ["*"],
"expireStyle": ["day", "hour", "minute", "forever", "count"],
"expire_style": ["day", "hour", "minute", "forever", "count"],
"code_generate_type": "secret",
"uploadMinute": 1,
"enableChunk": 0,
"upload_minute": 1,
"enable_chunk": 0,
"webdav_url": "",
"webdav_password": "",
"webdav_username": "",
"opacity": 0.9,
"background": "",
"uploadCount": 10,
"themesChoices": [
"upload_count": 10,
"themes_choices": [
{
"name": "2023",
"key": "themes/2023",
@@ -73,15 +75,19 @@ DEFAULT_CONFIG = {
"version": "1.0",
},
],
"themesSelect": "themes/2024",
"errorMinute": 1,
"errorCount": 10,
"serverWorkers": 1,
"serverHost": "0.0.0.0",
"serverPort": 12345,
"showAdminAddr": 0,
"robotsText": "User-agent: *\nDisallow: /",
"trustedProxies": [],
"themes_select": "themes/2024",
"error_minute": 1,
"error_count": 10,
"login_count": 5,
"login_minute": 15,
"server_workers": 1,
"server_host": "0.0.0.0",
"server_port": 12345,
"show_admin_addr": 0,
"robots_text": "User-agent: *\nDisallow: /",
"trusted_proxies": [],
"chunk_expire_hours": 24,
"opendal_scheme": "s3",
}
@@ -105,6 +111,14 @@ class Settings:
else:
self.user_config[key] = value
def unknown_keys(self, config: dict) -> list[str]:
"""Keys in `config` that DEFAULT_CONFIG does not define.
Unknown keys silently fall through __getattr__ and crash later at the
usage site; callers (refresh_settings) surface them at load time.
"""
return sorted(k for k in config if k not in self.default_config)
def items(self):
return {**self.default_config, **self.user_config}.items()
-1364
View File
File diff suppressed because it is too large. Load diff
+25
View File
@@ -0,0 +1,25 @@
"""Storage backend package (split from the former 1400+ line storage.py).
Public surface is re-exported here — `from core.storage import X` keeps
working for every historical name.
"""
from core.storage._base import ( # noqa: F401
FileStorageInterface,
S3_MIN_MULTIPART_PART_SIZE,
StoredDownload,
StoredFile,
build_attachment_headers,
)
from core.storage.local import SystemFileStorage # noqa: F401
from core.storage.s3 import S3FileStorage # noqa: F401
from core.storage.onedrive import OneDriveFileStorage # noqa: F401
from core.storage.opendal import OpenDALFileStorage # noqa: F401
from core.storage.webdav import WebDAVFileStorage # noqa: F401
storages = {
"local": SystemFileStorage,
"s3": S3FileStorage,
"onedrive": OneDriveFileStorage,
"opendal": OpenDALFileStorage,
"webdav": WebDAVFileStorage,
}
+181
View File
@@ -0,0 +1,181 @@
# @Time : 2023/8/11 20:06
# @Author : Lan
# @File : storage.py
# @Software: PyCharm
import hashlib
from typing import BinaryIO, Optional
from urllib.parse import quote
from collections.abc import Callable
from typing import Any
from dataclasses import dataclass
from pathlib import Path
@dataclass
class StoredDownload:
"""Framework-free description of a file download.
Backends return this; the view layer builds the starlette Response:
- ``path`` set -> FileResponse (local files, Range support for free)
- ``content`` set -> small full-read Response (legacy OpenDAL fallback)
- ``stream_factory`` set -> StreamingResponse(stream_factory(), ...)
``background`` is an optional response-sent cleanup hook.
"""
filename: str
headers: dict
media_type: str = "application/octet-stream"
path: Path | None = None
content: bytes | None = None
stream_factory: Callable[[], Any] | None = None
background: Any = None # starlette BackgroundTask(core 层不 import starlette)
@dataclass
class StoredFile:
"""Plain, framework- and ORM-free description of a stored file.
Storage backends accept this instead of ORM models so core/ never imports
apps/. Callers (views/tasks/services) build it from their own records.
"""
file_path: str
uuid_file_name: str
code: str = ""
prefix: str = ""
suffix: str = ""
text: str = ""
def get_file_path(self) -> str:
return f"{self.file_path}/{self.uuid_file_name}"
# S3 multipart 除最后一片外每部分最小 5MB(服务端强制,小于即 EntityTooSmall)
S3_MIN_MULTIPART_PART_SIZE = 5 * 1024 * 1024
def build_attachment_headers(filename: str, content_length=None) -> dict:
"""所有存储后端统一的下载响应头。
Content-Disposition: attachment 是防御存储型 XSS 的关键——同源下载路径
(/share/download)因此永不内联渲染 HTML/SVG。此函数是唯一构造点,
新增后端必须复用(tests/test_attachment_guard.py 有源码级 tripwire)。
"""
encoded_filename = quote(filename, safe="")
headers = {"Content-Disposition": f"attachment; filename*=UTF-8''{encoded_filename}"}
if content_length is not None:
headers["Content-Length"] = str(content_length)
return headers
class FileStorageInterface:
@staticmethod
def _get_chunk_record(chunk_records: dict, index: int):
"""Look up the caller-provided record for chunk `index`.
Records are plain objects exposing ``chunk_hash``; fetching them from
the DB is the caller's job (keeps storage ORM-free).
"""
chunk_record = chunk_records.get(index)
if not chunk_record:
raise ValueError(f"分片{index}记录不存在")
return chunk_record
def _verify_and_hash_chunk(
self,
index: int,
chunk_record,
chunk_data: bytes,
file_sha256,
) -> None:
"""Verify a chunk against its recorded hash, then stripe it into the
whole-file digest. Raises the shared ValueError wording on mismatch."""
current_hash = hashlib.sha256(chunk_data).hexdigest()
if current_hash != chunk_record.chunk_hash:
raise ValueError(
f"分片{index}哈希不匹配: 期望 {chunk_record.chunk_hash}, 实际 {current_hash}"
)
file_sha256.update(chunk_data)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""Save a binary stream (caller owns closing the stream)."""
raise NotImplementedError
async def delete_file(self, file_code: StoredFile):
"""
删除文件
"""
raise NotImplementedError
async def get_file_url(self, file_code: StoredFile):
"""
获取文件分享的url
如果服务不支持直接访问文件,可以通过服务器中转下载。
此时,此方法可以调用 utils.py 中的 `get_file_url` 方法,获取服务器中转下载的url
"""
raise NotImplementedError
async def get_file_response(self, file_code: StoredFile):
"""
获取文件响应
如果服务不支持直接访问文件,则需要实现该方法,返回文件响应
其余情况,可以不实现该方法
"""
raise NotImplementedError
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片文件
:param upload_id: 上传会话ID
:param chunk_index: 分片索引
:param chunk_data: 分片数据
:param chunk_hash: 分片哈希值
:param save_path: 文件保存路径
"""
raise NotImplementedError
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并分片文件并返回文件路径和完整哈希值
:param upload_id: 上传会话ID
:param chunk_info: 分片信息
:param save_path: 文件保存路径
:return: (文件路径, 文件哈希值)
"""
raise NotImplementedError
async def generate_presigned_upload_url(self, save_path: str, expires_in: int = 900) -> Optional[str]:
"""
生成预签名上传URL
:param save_path: 文件保存路径
:param expires_in: URL过期时间(秒),默认15分钟
:return: 预签名URL,如果不支持直传则返回None
"""
return None # 默认不支持直传,使用代理模式
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在
:param save_path: 文件路径
:return: 文件是否存在
"""
raise NotImplementedError
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
raise NotImplementedError
+184
View File
@@ -0,0 +1,184 @@
import hashlib
from core.logger import logger
import shutil
from typing import BinaryIO, Optional
import aiofiles
import asyncio
from pathlib import Path
from core.errors import StorageError
from core.settings import data_root
from core.utils import get_file_url, sanitize_filename
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class SystemFileStorage(FileStorageInterface):
def __init__(self):
self.chunk_size = 256 * 1024
self.root_path = data_root
def _resolve_safe_path(self, relative_path: str) -> Path:
"""将相对路径解析到数据根目录内,阻止路径穿越。"""
root = self.root_path.resolve()
raw = str(relative_path or "").replace("\\", "/").lstrip("/")
if any(part == ".." for part in raw.split("/")):
raise ValueError("非法文件路径")
candidate = (root / raw).resolve()
try:
candidate.relative_to(root)
except ValueError as exc:
raise ValueError("非法文件路径") from exc
return candidate
def _save(self, file, save_path):
with open(save_path, "wb") as f:
chunk = file.read(self.chunk_size)
while chunk:
f.write(chunk)
chunk = file.read(self.chunk_size)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
path_obj = Path(str(save_path).replace("\\", "/"))
directory = str(path_obj.parent).replace("\\", "/").lstrip("/")
# 提取原始文件名并进行清理
filename = await sanitize_filename(path_obj.name)
# 构建安全的完整保存路径
safe_save_path = self._resolve_safe_path(f"{directory}/{filename}" if directory not in {"", "."} else filename)
# 确保目录存在
if not safe_save_path.parent.exists():
safe_save_path.parent.mkdir(parents=True)
await asyncio.to_thread(self._save, stream, safe_save_path)
async def delete_file(self, file_code: StoredFile):
save_path = self._resolve_safe_path(file_code.get_file_path())
if save_path.exists():
save_path.unlink()
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
file_path = self._resolve_safe_path(file_code.get_file_path())
if not file_path.exists():
raise StorageError(status_code=404, detail="文件已过期删除")
filename = f"{file_code.prefix}{file_code.suffix}"
try:
headers = build_attachment_headers(filename, file_path.stat().st_size)
except OSError:
# 文件大小不可得时省略 Content-Length
headers = build_attachment_headers(filename)
return StoredDownload(
filename=filename,
headers=headers,
path=file_path,
)
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片文件到本地文件系统
:param upload_id: 上传会话ID
:param chunk_index: 分片索引
:param chunk_data: 分片数据
:param chunk_hash: 分片哈希值
:param save_path: 文件保存路径
"""
# 先校验目标文件路径合法,再将分片落到同级 chunks 目录。
self._resolve_safe_path(save_path)
chunk_path = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
)
if not chunk_path.parent.exists():
chunk_path.parent.mkdir(parents=True, exist_ok=True)
# 使用临时文件写入,确保原子性
temp_path = chunk_path.with_suffix('.tmp')
try:
async with aiofiles.open(temp_path, "wb") as f:
await f.write(chunk_data)
# 原子重命名
temp_path.rename(chunk_path)
except Exception as e:
if temp_path.exists():
temp_path.unlink()
raise e
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并本地文件系统的分片文件并返回文件路径和完整哈希值
:param upload_id: 上传会话ID
:param chunk_info: 分片信息
:param save_path: 文件保存路径
:return: (文件路径, 文件哈希值)
"""
output_path = self._resolve_safe_path(save_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
chunk_base_dir = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id)
)
file_sha256 = hashlib.sha256()
# 使用临时文件写入,确保原子性
temp_output = output_path.with_suffix('.merging')
try:
async with aiofiles.open(temp_output, "wb") as out_file:
for i in range(total_chunks):
# 获取分片记录
chunk_record = self._get_chunk_record(chunk_records, i)
chunk_path = chunk_base_dir / f"{i}.part"
if not chunk_path.exists():
raise ValueError(f"分片{i}文件不存在")
async with aiofiles.open(chunk_path, "rb") as in_file:
chunk_data = await in_file.read()
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
# 原子重命名
temp_output.rename(output_path)
except Exception as e:
if temp_output.exists():
temp_output.unlink()
raise e
return str(output_path), file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理本地文件系统的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id)
)
if chunk_dir.exists():
try:
shutil.rmtree(chunk_dir)
except Exception as e:
logger.warning(f"清理本地分片目录失败: {e}")
# 清理父级 chunks 目录(如果为空)
chunks_parent = chunk_dir.parent
if chunks_parent.exists() and not any(chunks_parent.iterdir()):
try:
chunks_parent.rmdir()
except Exception as e:
logger.warning(f"清理 chunks 父目录失败: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于本地文件系统
:param save_path: 文件路径
:return: 文件是否存在
"""
try:
file_path = self._resolve_safe_path(save_path)
except ValueError:
return False
return file_path.exists()
+339
View File
@@ -0,0 +1,339 @@
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
import aiofiles
import aiohttp
import asyncio
from pathlib import Path
import datetime
import re
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url, sanitize_filename
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class OneDriveFileStorage(FileStorageInterface):
def __init__(self):
try:
import msal
from office365.graph_client import GraphClient
from office365.runtime.client_request_exception import (
ClientRequestException,
)
except ImportError:
raise ImportError("请先安装`msal`和`Office365-REST-Python-Client`")
self.msal = msal
self.domain = settings.onedrive_domain
self.client_id = settings.onedrive_client_id
self.username = settings.onedrive_username
self.password = settings.onedrive_password
self.proxy = settings.onedrive_proxy
self._ClientRequestException = ClientRequestException
try:
client = GraphClient(self.acquire_token_pwd)
self.root_path = (
client.me.drive.root.get_by_path(settings.onedrive_root_path)
.get()
.execute_query()
)
except ClientRequestException as e:
if e.code == "itemNotFound":
client.me.drive.root.create_folder(settings.onedrive_root_path)
self.root_path = (
client.me.drive.root.get_by_path(
settings.onedrive_root_path)
.get()
.execute_query()
)
else:
raise e
except Exception as e:
raise Exception("OneDrive验证失败,请检查配置是否正确\n" + str(e))
def acquire_token_pwd(self):
authority_url = f"https://login.microsoftonline.com/{self.domain}"
app = self.msal.PublicClientApplication(
authority=authority_url, client_id=self.client_id
)
result = app.acquire_token_by_username_password(
username=self.username,
password=self.password,
scopes=["https://graph.microsoft.com/.default"],
)
return result
def _get_path_str(self, path):
if isinstance(path, str):
path = path.replace("\\", "/").replace("//", "/").split("/")
elif isinstance(path, Path):
path = str(path).replace("\\", "/").replace("//", "/").split("/")
else:
raise TypeError("path must be str or Path")
path[-1] = path[-1].split(".")[0]
return "/".join(path)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""保存文件(自动创建目录;修复旧实现把 save_path 字符串当函数调用的崩溃)"""
content = await asyncio.to_thread(stream.read)
normalized = str(save_path).replace("\\", "/")
name = await sanitize_filename(Path(normalized).name)
dir_path = "/".join(normalized.split("/")[:-1])
current_folder = self.root_path
for part in dir_path.split("/"):
if not part:
continue
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
await asyncio.to_thread(
lambda: current_folder.get_by_path(name)
.upload(name, content)
.execute_query()
)
def _delete(self, save_path):
path = self._get_path_str(save_path)
try:
self.root_path.get_by_path(path).delete_object().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
pass
else:
raise e
async def delete_file(self, file_code: StoredFile):
await asyncio.to_thread(self._delete, file_code.get_file_path())
def _convert_link_to_download_link(self, link):
p1 = re.search(r"https://(.+)\.sharepoint\.com", link).group(1)
p2 = re.search(r"personal/(.+)/", link).group(1)
p3 = re.search(rf"{p2}/(.+)", link).group(1)
return f"https://{p1}.sharepoint.com/personal/{p2}/_layouts/52/download.aspx?share={p3}"
def _get_file_url(self, save_path, name):
path = self._get_path_str(save_path)
remote_file = self.root_path.get_by_path(path + "/" + name)
expiration_datetime = datetime.datetime.now(
tz=datetime.timezone.utc
) + datetime.timedelta(hours=1)
expiration_datetime = expiration_datetime.strftime(
"%Y-%m-%dT%H:%M:%SZ")
permission = remote_file.create_link(
"view", "anonymous", expiration_datetime=expiration_datetime
).execute_query()
return self._convert_link_to_download_link(permission.link.webUrl)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
try:
link = await asyncio.to_thread(
self._get_file_url, file_code.get_file_path(), filename
)
except self._ClientRequestException as e:
# 对象不存在时前置 404(与 local/S3/WebDAV 语义对齐),
# 不再落入外层兜底的 503
if str(getattr(e, "code", "")).lower() in {"itemnotfound", "404"}:
raise StorageError(
status_code=404, detail="文件已过期删除"
) from e
raise
content_length = None # 初始化为 None,表示未知大小
# 创建ClientSession并复用
session = aiohttp.ClientSession()
# 尝试发送HEAD请求获取Content-Length
try:
async with session.head(link) as resp:
if resp.status == 200 and 'Content-Length' in resp.headers:
content_length = int(resp.headers['Content-Length'])
except Exception:
# 如果HEAD请求失败,则不提供 Content-Length
pass
async def stream_generator():
try:
async with session.get(link) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"从OneDrive获取文件失败: {resp.status}"
)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except StorageError:
raise
except Exception:
raise StorageError(status_code=503, detail="服务代理下载异常,请稍后再试")
async def get_file_url(self, file_code: StoredFile):
if self.proxy:
return await get_file_url(file_code.code)
else:
return await asyncio.to_thread(
self._get_file_url,
file_code.get_file_path(),
f"{file_code.prefix}{file_code.suffix}",
)
def _save_chunk(self, chunk_path: str, chunk_data: bytes):
"""同步保存分片到 OneDrive"""
path_parts = chunk_path.replace("\\", "/").split("/")
filename = path_parts[-1]
dir_path = "/".join(path_parts[:-1])
# 确保目录存在
current_folder = self.root_path
for part in dir_path.split("/"):
if part:
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
# 上传分片
current_folder.upload(filename, chunk_data).execute_query()
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 OneDrive"""
chunk_path = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
await asyncio.to_thread(self._save_chunk, chunk_path, chunk_data)
def _read_chunk(self, chunk_path: str) -> bytes:
"""同步读取分片"""
path = self._get_path_str(chunk_path)
file_obj = self.root_path.get_by_path(path).get().execute_query()
return file_obj.get_content().execute_query().value
def _upload_merged(self, save_path: str, data: bytes):
"""同步上传合并后的文件"""
path_parts = save_path.replace("\\", "/").split("/")
filename = path_parts[-1]
dir_path = "/".join(path_parts[:-1])
# 确保目录存在
current_folder = self.root_path
for part in dir_path.split("/"):
if part:
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
current_folder.upload(filename, data).execute_query()
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""合并 OneDrive 上的分片文件,使用临时文件避免内存问题"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
chunk_data = await asyncio.to_thread(self._read_chunk, chunk_path)
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 读取临时文件并上传
async with aiofiles.open(temp_path, 'rb') as f:
merged_content = await f.read()
await asyncio.to_thread(self._upload_merged, save_path, merged_content)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
def _delete_chunk_dir(self, chunk_dir: str):
"""同步删除分片目录"""
try:
path = self._get_path_str(chunk_dir)
self.root_path.get_by_path(path).delete_object().execute_query()
except self._ClientRequestException as e:
if e.code != "itemNotFound":
raise e
async def clean_chunks(self, upload_id: str, save_path: str):
"""清理 OneDrive 上的临时分片文件"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
try:
await asyncio.to_thread(self._delete_chunk_dir, chunk_dir)
except Exception as e:
logger.warning(f"清理 OneDrive 分片时出错: {e}")
def _file_exists(self, save_path: str) -> bool:
"""同步检查文件是否存在"""
try:
path = self._get_path_str(save_path)
self.root_path.get_by_path(path).get().execute_query()
return True
except self._ClientRequestException as e:
if e.code == "itemNotFound":
return False
raise e
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于OneDrive
:param save_path: 文件路径
:return: 文件是否存在
"""
return await asyncio.to_thread(self._file_exists, save_path)
+160
View File
@@ -0,0 +1,160 @@
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
import aiofiles
import asyncio
from pathlib import Path
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class OpenDALFileStorage(FileStorageInterface):
def __init__(self):
try:
import opendal
except ImportError:
raise ImportError('请先安装 `opendal`, 例如: "pip install opendal"')
self.service = settings.opendal_scheme
service_settings = {}
for key, value in settings.items():
if key.startswith("opendal_" + self.service):
setting_name = key.split("_", 2)[2]
service_settings[setting_name] = value
self.operator = opendal.AsyncOperator(
settings.opendal_scheme, **service_settings
)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
# 使用 asyncio.to_thread 避免阻塞事件循环
content = await asyncio.to_thread(stream.read)
await self.operator.write(save_path, content)
async def delete_file(self, file_code: StoredFile):
await self.operator.delete(file_code.get_file_path())
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
content_length = None # 初始化为 None,表示未知大小
# 尝试获取文件大小
try:
stat_result = await self.operator.stat(file_code.get_file_path())
if hasattr(stat_result, 'content_length') and stat_result.content_length:
content_length = stat_result.content_length
elif hasattr(stat_result, 'size') and stat_result.size:
content_length = stat_result.size
except Exception:
# 如果获取大小失败,则不提供 Content-Length
pass
# 尝试使用流式读取器
try:
# OpenDAL 可能提供 reader 方法返回一个异步读取器
reader = await self.operator.reader(file_code.get_file_path())
except AttributeError:
# 如果 reader 方法不存在,回退到全量读取(兼容旧版本)
content = await self.operator.read(file_code.get_file_path())
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
content=content,
)
async def stream_generator():
chunk_size = 65536
while True:
chunk = await reader.read(chunk_size)
if not chunk:
break
yield chunk
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
)
except Exception as e:
logger.info(e)
raise StorageError(status_code=404, detail="文件已过期删除")
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 OpenDAL 存储"""
chunk_path = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
await self.operator.write(chunk_path, chunk_data)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""合并 OpenDAL 存储上的分片文件,使用临时文件避免内存问题"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
chunk_data = await self.operator.read(chunk_path)
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 读取临时文件并写入存储
async with aiofiles.open(temp_path, 'rb') as f:
merged_content = await f.read()
await self.operator.write(save_path, merged_content)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""清理 OpenDAL 存储上的临时分片文件"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
try:
# OpenDAL 支持递归删除
await self.operator.remove_all(chunk_dir)
except Exception as e:
logger.warning(f"清理 OpenDAL 分片时出错: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于OpenDAL存储
:param save_path: 文件路径
:return: 文件是否存在
"""
try:
await self.operator.stat(save_path)
return True
except Exception:
return False
+345
View File
@@ -0,0 +1,345 @@
from botocore.exceptions import ClientError
import hashlib
from core.logger import logger
from typing import BinaryIO, Optional
import aiohttp
import asyncio
from pathlib import Path
import aioboto3
from botocore.config import Config
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
S3_MIN_MULTIPART_PART_SIZE,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class S3FileStorage(FileStorageInterface):
def __init__(self):
self.access_key_id = settings.s3_access_key_id
self.secret_access_key = settings.s3_secret_access_key
self.bucket_name = settings.s3_bucket_name
self.s3_hostname = settings.s3_hostname
self.region_name = settings.s3_region_name
self.signature_version = settings.s3_signature_version
self.endpoint_url = settings.s3_endpoint_url or f"https://{self.s3_hostname}"
self.aws_session_token = settings.aws_session_token
self.addressing_style = str(settings.s3_addressing_style or "auto").lower()
self.proxy = settings.s3_proxy
self.session = aioboto3.Session(
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
)
if not settings.s3_endpoint_url:
self.endpoint_url = f"https://{self.s3_hostname}"
else:
# 如果提供了 s3_endpoint_url,则优先使用它
self.endpoint_url = settings.s3_endpoint_url
def _client_config(self) -> Config:
config = {"signature_version": self.signature_version}
s3_config = {}
if self.addressing_style in {"path", "virtual", "auto"}:
s3_config["addressing_style"] = self.addressing_style
if s3_config:
config["s3"] = s3_config
return Config(**config)
def _client(self):
return self.session.client(
"s3",
endpoint_url=self.endpoint_url,
aws_session_token=self.aws_session_token,
region_name=self.region_name,
config=self._client_config(),
)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
async with self._client() as s3:
# 使用 upload_fileobj 流式上传,避免将整个文件加载到内存
await s3.upload_fileobj(
stream,
self.bucket_name,
save_path,
ExtraArgs={"ContentType": content_type or "application/octet-stream"},
)
async def delete_file(self, file_code: StoredFile):
async with self._client() as s3:
await s3.delete_object(
Bucket=self.bucket_name, Key=file_code.get_file_path()
)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
content_length = None # 初始化为 None,表示未知大小
async with self._client() as s3:
# 尝试获取文件大小(HEAD请求);对象不存在时前置 404——
# 与 local 后端语义一致(M2 行为统一),不能签出 200 的坏流
try:
head_response = await s3.head_object(
Bucket=self.bucket_name,
Key=file_code.get_file_path()
)
# 从HEAD响应中获取Content-Length
if 'ContentLength' in head_response:
content_length = head_response['ContentLength']
elif 'Content-Length' in head_response['ResponseMetadata']['HTTPHeaders']:
content_length = int(head_response['ResponseMetadata']['HTTPHeaders']['Content-Length'])
except ClientError as e:
error_code = e.response.get("Error", {}).get("Code", "")
if error_code in {"404", "NoSuchKey", "NotFound"}:
raise StorageError(
status_code=404, detail="文件已过期删除"
) from e
# 其他 HEAD 错误不阻断:流式下载阶段会给出真实状态
except Exception:
# 如果HEAD请求失败,则不提供 Content-Length
pass
link = await s3.generate_presigned_url(
"get_object",
Params={
"Bucket": self.bucket_name,
"Key": file_code.get_file_path(),
},
ExpiresIn=3600,
)
# 创建ClientSession并传递给生成器复用
session = aiohttp.ClientSession()
async def stream_generator():
try:
async with session.get(link) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"从S3获取文件失败: {resp.status}"
)
# 设置块大小(例如64KB)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except StorageError:
raise
except Exception:
raise StorageError(status_code=503, detail="服务代理下载异常,请稍后再试")
async def get_file_url(self, file_code: StoredFile):
if file_code.prefix == "文本分享":
return file_code.text
if self.proxy:
return await get_file_url(file_code.code)
else:
async with self._client() as s3:
result = await s3.generate_presigned_url(
"get_object",
Params={
"Bucket": self.bucket_name,
"Key": file_code.get_file_path(),
},
ExpiresIn=3600,
)
return result
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片到 S3(使用独立对象存储每个分片)
注意:这里不使用 S3 原生的 multipart upload,而是将每个分片作为独立对象存储
"""
chunk_key = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
async with self._client() as s3:
# 将分片作为独立对象上传
await s3.put_object(
Bucket=self.bucket_name,
Key=chunk_key,
Body=chunk_data,
Metadata={
'chunk-hash': chunk_hash,
'chunk-index': str(chunk_index)
}
)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并 S3 上的分片文件
使用 S3 的 multipart upload API 实现流式合并,避免内存问题
"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
async with self._client() as s3:
# 创建 multipart upload
mpu = await s3.create_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
ContentType='application/octet-stream'
)
mpu_id = mpu['UploadId']
parts = []
try:
# 按顺序读取、验证每个分片;S3 multipart 规范要求除最后一片外
# 每个部分 ≥5MB(EntityTooSmall),而分片大小由客户端决定(常见
# 2-4MB)——因此缓冲到 S3_MIN_MULTIPART_PART_SIZE 再上传 part,
# 内存上界 = 5MB + 单个分片,不破坏流式合并的初衷。
part_buffer = bytearray()
part_number = 0
async def _flush_part():
nonlocal part_number
if not part_buffer:
return
part_number += 1
part_response = await s3.upload_part(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id,
PartNumber=part_number,
Body=bytes(part_buffer),
)
parts.append({
'PartNumber': part_number,
'ETag': part_response['ETag']
})
part_buffer.clear()
for i in range(total_chunks):
chunk_key = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
response = await s3.get_object(
Bucket=self.bucket_name,
Key=chunk_key
)
chunk_data = await response['Body'].read()
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
part_buffer.extend(chunk_data)
if len(part_buffer) >= S3_MIN_MULTIPART_PART_SIZE:
await _flush_part()
# 收尾:剩余缓冲作为最后一个 part(S3 允许最后一片小于 5MB;
# 恰好整除时缓冲为空,跳过)
await _flush_part()
# 完成 multipart upload
await s3.complete_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id,
MultipartUpload={'Parts': parts}
)
except Exception as e:
# 出错时取消 multipart upload
await s3.abort_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id
)
raise e
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理 S3 上的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
async with self._client() as s3:
try:
# 列出并删除所有分片对象
paginator = s3.get_paginator('list_objects_v2')
async for page in paginator.paginate(Bucket=self.bucket_name, Prefix=chunk_dir):
objects = page.get('Contents', [])
if objects:
delete_objects = [{'Key': obj['Key']} for obj in objects]
await s3.delete_objects(
Bucket=self.bucket_name,
Delete={'Objects': delete_objects}
)
except Exception as e:
logger.warning(f"清理 S3 分片数据时出错: {e}")
async def generate_presigned_upload_url(self, save_path: str, expires_in: int = 900) -> Optional[str]:
"""
生成S3预签名上传URL
:param save_path: 文件保存路径
:param expires_in: URL过期时间(秒),默认15分钟
:return: 预签名PUT URL
"""
async with self._client() as s3:
return await s3.generate_presigned_url(
"put_object",
Params={
"Bucket": self.bucket_name,
"Key": save_path,
},
ExpiresIn=expires_in,
)
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于S3
:param save_path: 文件路径
:return: 文件是否存在
"""
async with self._client() as s3:
last_error = None
for attempt in range(3):
try:
await s3.head_object(Bucket=self.bucket_name, Key=save_path)
return True
except Exception as e:
last_error = e
if attempt < 2:
await asyncio.sleep(0.2 * (attempt + 1))
try:
result = await s3.list_objects_v2(
Bucket=self.bucket_name,
Prefix=save_path,
MaxKeys=1,
)
for item in result.get("Contents", []):
if item.get("Key") == save_path:
return True
except Exception as e:
last_error = e
logger.warning(f"S3文件确认失败 key={save_path}: {last_error}")
return False
+354
View File
@@ -0,0 +1,354 @@
import base64
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
from urllib.parse import quote, unquote
import aiofiles
import aiohttp
import asyncio
from pathlib import Path
import re
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url, sanitize_filename
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class WebDAVFileStorage(FileStorageInterface):
def __init__(self):
if not hasattr(self, "_initialized"):
self.base_url = settings.webdav_url.rstrip("/") + "/"
# aiohttp 4.0 移除 BasicAuth(auth=...) 参数——改用编码后的 Authorization 头
self.auth_headers = {
"Authorization": aiohttp.encode_basic_auth(
settings.webdav_username, settings.webdav_password
)
}
self._initialized = True
def _build_url(self, path: str) -> str:
encoded_path = quote(str(path.replace("\\", "/").lstrip("/")).lstrip("/"))
return f"{self.base_url}{encoded_path}"
async def _mkdir_p(self, directory_path: str):
"""递归创建目录(类似mkdir -p)"""
path_obj = Path(unquote(directory_path))
current_path = ""
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 逐级检查目录是否存在
for part in path_obj.parts:
current_path = str(Path(current_path) / part)
url = self._build_url(current_path)
# 检查目录是否存在
async with session.head(url) as resp:
if resp.status == 404:
# 创建目录
async with session.request("MKCOL", url) as mkcol_resp:
if mkcol_resp.status not in (200, 201, 409):
content = await mkcol_resp.text()
raise StorageError(
status_code=mkcol_resp.status,
detail=f"目录创建失败: {content[:200]}",
)
async def _is_dir_empty(self, dir_path: str) -> bool:
"""检查目录是否为空"""
url = self._build_url(dir_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.request("PROPFIND", url, headers={"Depth": "1"}) as resp:
if resp.status != 207: # 207 是 Multi-Status 响应
return False
content = await resp.text()
# 如果只有一个 response(当前目录),说明目录为空
return content.count("<D:response>") <= 1
async def _delete_empty_dirs(self, file_path: str, session: aiohttp.ClientSession):
"""递归删除空目录"""
path_obj = Path(file_path)
current_path = path_obj.parent
while str(current_path) != ".":
if not await self._is_dir_empty(str(current_path)):
break
url = self._build_url(str(current_path))
async with session.delete(url) as resp:
if resp.status not in (200, 204, 404):
break
current_path = current_path.parent
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""保存文件(自动创建目录,流式上传)"""
path_obj = Path(save_path)
directory_path = str(path_obj.parent)
# 提取原始文件名并进行清理
filename = await sanitize_filename(path_obj.name)
# 构建安全的保存路径
safe_save_path = str(Path(directory_path) / filename)
try:
# 先创建目录结构
await self._mkdir_p(directory_path)
# 上传文件(流式)
url = self._build_url(safe_save_path)
async def file_sender():
"""流式读取文件内容"""
chunk_size = 256 * 1024 # 256KB chunks
while True:
chunk = await asyncio.to_thread(stream.read, chunk_size)
if not chunk:
break
yield chunk
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.put(
url,
data=file_sender(),
headers={"Content-Type": content_type or "application/octet-stream"}
) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"文件上传失败: {content[:200]}",
)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def delete_file(self, file_code: StoredFile):
"""删除WebDAV文件及空目录"""
file_path = file_code.get_file_path()
url = self._build_url(file_path)
try:
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 删除文件
async with session.delete(url) as resp:
if resp.status not in (200, 204, 404):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"WebDAV删除失败: {content[:200]}",
)
# 使用同一个 session 删除空目录
await self._delete_empty_dirs(file_path, session)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
"""获取文件响应(代理模式)"""
try:
filename = file_code.prefix + file_code.suffix
url = self._build_url(file_code.get_file_path())
content_length = None # 初始化为 None,表示未知大小
# 创建ClientSession并复用(包含认证头)
session = aiohttp.ClientSession(headers={
"Authorization": f"Basic {base64.b64encode(f'{settings.webdav_username}:{settings.webdav_password}'.encode()).decode()}"
})
# 尝试发送HEAD请求获取Content-Length;对象不存在时前置 404
# (与 local/S3 语义对齐),连接层错误映射 503——两者都不能
# 静默吞掉后签出 200 坏流。异常路径回收 session 防泄漏。
try:
try:
async with session.head(url) as resp:
if resp.status == 404:
raise StorageError(
status_code=404, detail="文件已过期删除"
)
if resp.status == 200 and 'Content-Length' in resp.headers:
content_length = int(resp.headers['Content-Length'])
except StorageError:
raise
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}"
) from e
except Exception:
# 其他 HEAD 异常不阻断:流式下载阶段会给出真实状态
pass
except BaseException:
await session.close()
raise
async def stream_generator():
try:
async with session.get(url) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"文件获取失败{resp.status}: {await resp.text()}",
)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 WebDAV"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
chunk_path = f"{chunk_dir}/{chunk_index}.part"
# 先创建目录结构
await self._mkdir_p(chunk_dir)
chunk_url = self._build_url(chunk_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.put(chunk_url, data=chunk_data) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"分片上传失败: {content[:200]}"
)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并 WebDAV 上的分片文件
使用临时文件避免内存问题
"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据,避免内存问题
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 按顺序读取并验证每个分片,写入临时文件
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_url = self._build_url(chunk_path)
# 获取分片记录
chunk_record = self._get_chunk_record(chunk_records, i)
# 下载分片数据
async with session.get(chunk_url) as resp:
if resp.status != 200:
raise ValueError(f"分片{i}文件不存在或无法访问")
chunk_data = await resp.read()
# 验证哈希
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 确保目标目录存在
output_dir = str(Path(save_path).parent)
await self._mkdir_p(output_dir)
# 流式上传合并后的文件
output_url = self._build_url(save_path)
async def file_sender():
async with aiofiles.open(temp_path, 'rb') as f:
while True:
chunk = await f.read(256 * 1024)
if not chunk:
break
yield chunk
async with session.put(output_url, data=file_sender()) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"合并文件上传失败: {content[:200]}"
)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理 WebDAV 上的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
chunk_dir_url = self._build_url(chunk_dir)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
try:
# 检查分片目录是否存在
async with session.request("PROPFIND", chunk_dir_url, headers={"Depth": "1"}) as resp:
if resp.status == 207: # 207 表示 Multi-Status
# 获取目录下的所有分片文件
xml_data = await resp.text()
file_paths = re.findall(
r'<D:href>(.*?)</D:href>', xml_data)
for file_path in file_paths:
if file_path.endswith(".part"):
# 删除分片文件
file_url = self._build_url(file_path)
async with session.delete(file_url) as delete_resp:
if delete_resp.status not in (200, 204, 404):
logger.warning(f"删除分片文件失败: {file_path}")
# 删除分片目录
async with session.delete(chunk_dir_url) as delete_resp:
if delete_resp.status not in (200, 204, 404):
logger.warning(f"删除分片目录失败: {chunk_dir_url}")
else:
logger.info(f"分片目录不存在: {chunk_dir_url}")
except Exception as e:
logger.warning(f"清理 WebDAV 分片时出错: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于WebDAV
:param save_path: 文件路径
:return: 文件是否存在
"""
url = self._build_url(save_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.head(url) as resp:
return resp.status == 200
+71 -7
View File
@@ -25,6 +25,27 @@ async def get_random_num():
r_s = string.ascii_uppercase + string.digits
def validate_background_url(value) -> str:
"""Validate the site background config before it reaches the theme template.
Themes inject this value into inline CSS ``url('...')`` where html escaping
cannot neutralize a single-quote breakout, so only well-formed http(s) URLs
(or an empty string) are accepted.
"""
value = str(value or "").strip()
if not value:
return ""
from urllib.parse import urlparse
parsed = urlparse(value)
if parsed.scheme not in ("http", "https") or not parsed.netloc:
raise ValueError("background 必须是 http(s) 完整 URL 或留空")
# quote()/whitespace would break out of the CSS url('') quoting context
if any(ch in value for ch in ("'", '"', "(", ")", " ", "\\", ";")):
raise ValueError("background URL 含有不允许的字符")
return value
async def get_random_string():
"""
获取随机字符串
@@ -105,25 +126,61 @@ async def max_save_times_desc(max_save_seconds: int):
return desc_zh, desc_en
# scrypt work factors (OWASP-recommended memory-hard parameters; ~50ms/verify)
SCRYPT_N = 2**14
SCRYPT_R = 8
SCRYPT_P = 1
_SCRYPT_MAXMEM = 64 * 1024 * 1024 # hashlib default cap (32MB) is too low for n=2^14,r=8
PASSWORD_SCHEME = "scrypt" # current scheme for new hashes; sha256/plaintext stay verifiable
def hash_password(password: str) -> str:
"""
使用 SHA256 + salt 哈希密码
返回格式: sha256$<salt>$<hash>
使用 scrypt(memory-hard)哈希密码
返回格式: scrypt$<n>$<r>$<p>$<salt>$<hash>
"""
salt = os.urandom(16).hex()
password_hash = hashlib.sha256(f"{salt}{password}".encode()).hexdigest()
return f"sha256${salt}${password_hash}"
password_hash = hashlib.scrypt(
password.encode(),
salt=salt.encode(),
n=SCRYPT_N,
r=SCRYPT_R,
p=SCRYPT_P,
maxmem=_SCRYPT_MAXMEM,
).hex()
return f"scrypt${SCRYPT_N}${SCRYPT_R}${SCRYPT_P}${salt}${password_hash}"
def verify_password(password: str, hashed: str) -> bool:
"""
验证密码是否匹配
支持新格式 (sha256$salt$hash) 和旧格式 (明文)
支持格式: scrypt$n$r$p$salt$hash(现行)、sha256$salt$hash(历史)、明文(最老)
"""
if not hashed:
return False
# 新格式: sha256$salt$hash
if hashed.startswith("scrypt$"):
parts = hashed.split("$")
if len(parts) != 6:
return False
try:
scheme, n, r, p, salt, stored_hash = parts
password_hash = hashlib.scrypt(
password.encode(),
salt=salt.encode(),
n=int(n),
r=int(r),
p=int(p),
maxmem=_SCRYPT_MAXMEM,
).hex()
# 默认解释器对超范围的 n/r/p 抛 TypeError/ValueError;部分构建会抛
# OverflowError,一并视为校验失败,避免坏掉的存量哈希影响登录路径。
except (ValueError, TypeError, OverflowError):
return False
return hmac.compare_digest(password_hash, stored_hash)
# sha256 格式: sha256$salt$hash(历史数据,验证逻辑保持原样)
if hashed.startswith("sha256$"):
parts = hashed.split("$")
if len(parts) != 3:
@@ -138,11 +195,18 @@ def verify_password(password: str, hashed: str) -> bool:
def is_password_hashed(password: str) -> bool:
"""
检查密码是否已经是哈希格式
检查密码是否已经是哈希格式(现行或历史哈希均视为已哈希)
"""
if password.startswith("scrypt$") and len(password.split("$")) == 6:
return True
return password.startswith("sha256$") and len(password.split("$")) == 3
def password_needs_rehash(hashed: str) -> bool:
"""True when the stored hash uses a legacy scheme (sha256/plaintext)."""
return bool(hashed) and not hashed.startswith(f"{PASSWORD_SCHEME}$")
async def sanitize_filename(filename: str) -> str:
"""
安全处理文件名:
+13 -1
View File
@@ -1,12 +1,24 @@
version: "3"
services:
file-code-box:
image: lanol/filecodebox:2.5.2 # x-release-please-version
image: lanol/filecodebox:2.8.1 # x-release-please-version
volumes:
- fcb-data:/app/data:rw
# NAS 已有文件:把共享目录挂到 /app/data/local,即可在后台「本地分享」里生成提取码,无需再上传
# - /volume1/share:/app/data/local:ro
restart: unless-stopped
ports:
- "12345:12345"
environment:
- APP_ENV=production
- LOG_LEVEL=warning
- ACCESS_LOG=false
- WORKERS=1
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
volumes:
fcb-data:
external: false
+1
View File
@@ -205,6 +205,7 @@ export default defineConfig({
text: 'API Reference',
items: [
{ text: 'API Overview', link: '/en/api/' },
{ text: 'Presigned Upload', link: '/en/api/presign-upload' },
{ text: 'Share API', link: '/en/api/#share-api' },
{ text: 'Admin API', link: '/en/api/#admin-api' },
{ text: 'Error Response', link: '/en/api/#error-response' },
+17 -15
View File
@@ -1,6 +1,6 @@
# FileCodeBox API 文档
## API 版本: 2.1.0
## API 版本: 2.5.6
## 目录
- [认证](#认证)
@@ -30,7 +30,7 @@ curl -X POST "http://localhost:12345/admin/login" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"token": "xxx.xxx.xxx",
"token_type": "Bearer"
@@ -74,7 +74,7 @@ curl -X POST "http://localhost:12345/share/text/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123"
}
@@ -125,7 +125,7 @@ curl -X POST "http://localhost:12345/share/file/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt"
@@ -157,7 +157,7 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -184,7 +184,7 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -232,15 +232,15 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"totalFiles": 100,
"storageUsed": "1.5GB",
"sysUptime": "10天",
"yesterdayCount": 50,
"yesterdaySize": "500MB",
"todayCount": 30,
"todaySize": "300MB"
"total_files": 100,
"storage_used": "1610612736",
"sys_uptime": 1725489600.0,
"yesterday_count": 50,
"yesterday_size": "524288000",
"today_count": 30,
"today_size": "314572800",
}
}
```
@@ -258,13 +258,15 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
| page | integer | 否 | 1 | 当前页码 |
| size | integer | 否 | 10 | 每页数量 |
| keyword | string | 否 | "" | 搜索关键词 |
| sort_by | string | 否 | "created_at" | 排序字段(created_at/expired_at/name/size/used_count/code) |
| sort_order | string | 否 | "desc" | 排序方向(asc/desc) |
**响应示例:**
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"page": 1,
"size": 10,
+1 -4
View File
@@ -76,7 +76,6 @@ Content-Type: application/json
"upload_id": "a1b2c3d4e5f6...",
"upload_url": "https://bucket.s3.amazonaws.com/path?X-Amz-Signature=...",
"mode": "direct",
"save_path": "share/data/2024/01/01/uuid/document.pdf",
"expires_in": 900
}
}
@@ -89,7 +88,6 @@ Content-Type: application/json
| upload_id | string | 上传会话 ID,后续操作需要 |
| upload_url | string | 上传目标 URL |
| mode | string | 上传模式:`direct`(直传 S3)或 `proxy`(服务器代理) |
| save_path | string | 文件存储路径 |
| expires_in | integer | URL 有效期(秒),默认 900 秒(15 分钟) |
**错误响应**
@@ -308,7 +306,6 @@ interface PresignInitResponse {
upload_id: string
upload_url: string
mode: 'direct' | 'proxy'
save_path: string
expires_in: number
}
@@ -448,7 +445,7 @@ async function upload() {
## 注意事项
1. **会话有效期**: 上传会话默认 15 分钟后过期,请在有效期内完成上传
2. **文件大小限制**: 受系统配置 `uploadSize` 限制
2. **文件大小限制**: 受系统配置 `upload_size` 限制
3. **过期类型**: 支持 `day`、`hour`、`minute`、`forever`、`count`
4. **CORS**: 直传模式下,S3 需要配置正确的 CORS 策略
5. **重试机制**: 建议实现上传失败重试逻辑
+93
View File
@@ -1,5 +1,98 @@
# Changelog
## [2.8.1](https://github.com/vastsa/FileCodeBox/compare/v2.8.0...v2.8.1) (2026-10-10)
### Bug Fixes
* restore lint and Docker theme build ([2f5eab1](https://github.com/vastsa/FileCodeBox/commit/2f5eab18b1ab28fcb4f416248076ce574998d718))
## [2.8.0](https://github.com/vastsa/FileCodeBox/compare/v2.7.1...v2.8.0) (2026-10-10)
### Features
* add MCP quick share and safe HTML preview ([63e72c1](https://github.com/vastsa/FileCodeBox/commit/63e72c145576d169265e4fa8cf373e33531a06a0))
## [2.7.1](https://github.com/vastsa/FileCodeBox/compare/v2.7.0...v2.7.1) (2026-09-19)
### Bug Fixes
* keep Docker as root, close s3_hostname SSRF, unbreak 404 tests ([031d300](https://github.com/vastsa/FileCodeBox/commit/031d300caea99117e82b3bddc3e5d0532cf3efa6))
## [2.7.0](https://github.com/vastsa/FileCodeBox/compare/v2.6.1...v2.7.0) (2026-09-16)
### Features
* mint extract codes from data/local without re-upload ([93a30a2](https://github.com/vastsa/FileCodeBox/commit/93a30a257cc5b0ee23a374521e1e12602c3959da))
## [2.6.1](https://github.com/vastsa/FileCodeBox/compare/v2.6.0...v2.6.1) (2026-09-14)
### Bug Fixes
* don't let a legacy background value block settings saves ([eb2db8f](https://github.com/vastsa/FileCodeBox/commit/eb2db8f507e3198e0ae45b976bdd90eab2f7d1ed))
* 存量 background 值不再阻塞后台设置保存([#514](https://github.com/vastsa/FileCodeBox/issues/514) 升级回归) ([0d82fbe](https://github.com/vastsa/FileCodeBox/commit/0d82fbe26ad06ca77af7b2b5aa05f514e8362295))
## [2.6.0](https://github.com/vastsa/FileCodeBox/compare/v2.5.6...v2.6.0) (2026-09-14)
### Features
* declare missing config keys (opendal_scheme, chunk_expire_hours) and add config schema guard ([c5d525d](https://github.com/vastsa/FileCodeBox/commit/c5d525df08bb9534d1efb3ab5ff9357b52240fe9))
* rename all config keys to snake_case with DB migration 007 ([5121db6](https://github.com/vastsa/FileCodeBox/commit/5121db6edcdac57318d058e28d7249efb9ea4d03))
* upgrade password hashing to memory-hard scrypt with transparent rehash ([443cfe7](https://github.com/vastsa/FileCodeBox/commit/443cfe7472693bb954845f4cc4ac77e3abfcbd23))
### Bug Fixes
* FileCodes.expired_count defaults to -1 (time-based) instead of instantly-expired 0 (D4) ([3c93718](https://github.com/vastsa/FileCodeBox/commit/3c93718d4b2f3a8e93978c7eb7a4a3b2553b0855))
* save_file accepts plain binary streams; repair admin local file share (D2) ([bb95017](https://github.com/vastsa/FileCodeBox/commit/bb95017aa398e80e5eca512aa9a330aaeeceb364))
* serialize KeyValue JSON read-modify-write sections behind a process lock (D5) ([0038636](https://github.com/vastsa/FileCodeBox/commit/0038636a2a5fc8e73f944c38789dce8de0354c3a))
* snake_case the public config contract (D7 follow-up) ([800bac0](https://github.com/vastsa/FileCodeBox/commit/800bac01285dd6eb827e28460996997dbb80e2c3))
* validate background config as http(s) URL to close CSS quote-breakout in 2023 theme ([4cab625](https://github.com/vastsa/FileCodeBox/commit/4cab625d5ab2fe5921f8f621ebe015623afb6fb8))
### Performance Improvements
* cache settings refresh behind a 2s TTL, force refresh on write paths (D6) ([61d72ab](https://github.com/vastsa/FileCodeBox/commit/61d72ab6ccba5df121945ef825de2967c45888b4))
* dashboard counters and storage quota use SQL aggregation (D3) ([baecebd](https://github.com/vastsa/FileCodeBox/commit/baecebd81efc9677acac29d5335e281d6bb5a252))
* make is_config_initialized cheap — no scrypt on the per-request probe (D1) ([ec424dd](https://github.com/vastsa/FileCodeBox/commit/ec424dd9093d94b69ec3a118f321cd1f85c7dd54))
## [2.5.6](https://github.com/vastsa/FileCodeBox/compare/v2.5.5...v2.5.6) (2026-09-01)
### Bug Fixes
* **logging:** reduce production log noise ([5537f56](https://github.com/vastsa/FileCodeBox/commit/5537f56b884a15eda015665c67c3f8469e4e9d89))
## [2.5.5](https://github.com/vastsa/FileCodeBox/compare/v2.5.4...v2.5.5) (2026-08-29)
### Bug Fixes
* **chunk-upload:** persist sanitized filenames ([da1dd06](https://github.com/vastsa/FileCodeBox/commit/da1dd066551cc7cc9a631a0c982b5675bd4b1967))
* harden chunk upload filename metadata ([5a7195c](https://github.com/vastsa/FileCodeBox/commit/5a7195c5c7d1028500b0e363c6b47130d260e11c))
* sanitize chunk upload file_name to prevent path traversal ([cebac32](https://github.com/vastsa/FileCodeBox/commit/cebac32145f948bfca7c44c663a0bcdcd9c56830))
## [2.5.4](https://github.com/vastsa/FileCodeBox/compare/v2.5.3...v2.5.4) (2026-07-23)
### Bug Fixes
* run Docker as root for volume compatibility ([fbcd4a5](https://github.com/vastsa/FileCodeBox/commit/fbcd4a5b2b788cdfc8e86d920f9c5a7299dc839a))
* run Docker image as root for data volume compatibility ([fc0cbeb](https://github.com/vastsa/FileCodeBox/commit/fc0cbeb4f0c7485d2e7abc0501a5fd9b1456cc22))
## [2.5.3](https://github.com/vastsa/FileCodeBox/compare/v2.5.2...v2.5.3) (2026-07-23)
### Bug Fixes
* add magic-byte checks, login rate limits, and non-root Docker ([eda0a31](https://github.com/vastsa/FileCodeBox/commit/eda0a3127ab562b6a2cf828c0df4a93933c44526))
* magic-byte file checks, admin login rate limit, non-root Docker ([0ee2f04](https://github.com/vastsa/FileCodeBox/commit/0ee2f046dd057fecceaee1059d72d67de3038f19))
## [2.5.2](https://github.com/vastsa/FileCodeBox/compare/v2.5.1...v2.5.2) (2026-07-23)
+17 -15
View File
@@ -1,6 +1,6 @@
# FileCodeBox API Documentation
## API Version: 2.1.0
## API Version: 2.5.6
## Table of Contents
- [Authentication](#authentication)
@@ -30,7 +30,7 @@ curl -X POST "http://localhost:12345/admin/login" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"token": "xxx.xxx.xxx",
"token_type": "Bearer"
@@ -74,7 +74,7 @@ curl -X POST "http://localhost:12345/share/text/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123"
}
@@ -125,7 +125,7 @@ curl -X POST "http://localhost:12345/share/file/" \
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt"
@@ -157,7 +157,7 @@ curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -184,7 +184,7 @@ Select file by share code.
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
@@ -232,15 +232,15 @@ Get system dashboard data.
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"totalFiles": 100,
"storageUsed": "1.5GB",
"sysUptime": "10 days",
"yesterdayCount": 50,
"yesterdaySize": "500MB",
"todayCount": 30,
"todaySize": "300MB"
"total_files": 100,
"storage_used": "1610612736",
"sys_uptime": 1725489600.0,
"yesterday_count": 50,
"yesterday_size": "524288000",
"today_count": 30,
"today_size": "314572800",
}
}
```
@@ -258,13 +258,15 @@ Get system file list.
| page | integer | No | 1 | Current page |
| size | integer | No | 10 | Page size |
| keyword | string | No | "" | Search keyword |
| sort_by | string | No | "created_at" | Sort field(created_at/expired_at/name/size/used_count/code) |
| sort_order | string | No | "desc" | Sort direction(asc/desc) |
**Response Example:**
```json
{
"code": 200,
"msg": "success",
"message": "success",
"detail": {
"page": 1,
"size": 10,
+389
View File
@@ -0,0 +1,389 @@
# Presigned Upload API
## Overview
The presigned upload feature provides a unified file upload interface that automatically picks the optimal upload method based on the configured storage backend:
- **S3 storage**: returns a presigned URL so the client uploads directly to S3 (reduces server bandwidth usage)
- **Other storage**: returns a proxy upload URL and the file is transferred through the server
## Upload Flow
### Flow Diagram
```
┌─────────┐ 1. Initialize upload ┌─────────┐
│ Client │ ───────────────────────▶ │ Server │
└─────────┘ └─────────┘
│ │
│◀──── upload_url + mode ──────────┤
│ │
│ ┌─────────────────────────────────────────┐
│ │ if mode == "direct" (S3 storage) │
│ │ 2a. PUT file to upload_url (S3) │
│ │ 3a. POST /confirm to finish │
│ │ │
│ │ if mode == "proxy" (other storage) │
│ │ 2b. PUT file to upload_url (server) │
│ │ (share code returned automatically, │
│ │ no confirmation needed) │
│ └─────────────────────────────────────────┘
│
▼
Get share code
```
---
## API Endpoints
### 1. Initialize Upload
Initialize a presigned upload session and get the upload URL and mode.
**Request**
```
POST /presign/upload/init
Content-Type: application/json
```
**Request Body**
| Field | Type | Required | Default | Description |
| ------------ | ------- | -------- | ------- | --------------------------------------- |
| file_name | string | ✅ | - | File name (with extension) |
| file_size | integer | ✅ | - | File size in bytes |
| expire_value | integer | ❌ | 1 | Expiration time value |
| expire_style | string | ❌ | "day" | Expiration type: day/hour/minute/forever/count |
**Request Example**
```json
{
"file_name": "document.pdf",
"file_size": 1048576,
"expire_value": 7,
"expire_style": "day"
}
```
**Response**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"upload_url": "https://bucket.s3.amazonaws.com/path?X-Amz-Signature=...",
"mode": "direct",
"expires_in": 900
}
}
```
**Response Fields**
| Field | Type | Description |
| ---------- | ------- | --------------------------------------------------------------- |
| upload_id | string | Upload session ID, required by all subsequent operations |
| upload_url | string | Target upload URL |
| mode | string | Upload mode: `direct` (direct to S3) or `proxy` (via server) |
| expires_in | integer | URL validity in seconds, default 900 seconds (15 minutes) |
In `proxy` mode the response also includes `proxy_upload_url` (and `legacy_proxy_upload_url`); `upload_url` equals the proxy URL.
**Error Responses**
| Status | Description |
| ------ | -------------------------------------------------- |
| 400 | Invalid expiration type |
| 403 | File size exceeds limit / IP rate limit exceeded |
---
### 2a. Direct Mode - Upload File to S3
When `mode == "direct"`, the client PUTs the file directly to the returned presigned URL.
**Request**
```
PUT {upload_url}
Content-Type: application/octet-stream
[binary file content]
```
**Notes**
- Use the returned `upload_url` as-is, do not modify it
- `application/octet-stream` is the recommended Content-Type
- The request goes directly to S3, it does not pass through the server
**JavaScript Example**
```javascript
const response = await fetch(uploadUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': 'application/octet-stream',
},
})
if (response.ok) {
// Upload succeeded, call the confirm endpoint
}
```
---
### 2b. Proxy Mode - Upload File to Server
When `mode == "proxy"`, the client PUTs the file to the server proxy endpoint.
**Request**
```
PUT /presign/upload/proxy/{upload_id}
Content-Type: multipart/form-data
file: [file]
```
**Path Parameters**
| Parameter | Description |
| --------- | ----------------------------------------------- |
| upload_id | Upload session ID returned by the init endpoint |
**Response**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**Note**: In proxy mode the share code is returned immediately after upload; no confirmation call is needed.
**Error Responses**
| Status | Description |
| ------ | ------------------------------------------------------------------ |
| 400 | File size mismatches declared size / session does not support proxy |
| 404 | Upload session does not exist or has expired |
| 500 | Failed to save file |
---
### 3. Confirm Upload (direct mode only)
In direct mode, call this endpoint after the S3 upload completes to obtain the share code.
**Request**
```
POST /presign/upload/confirm/{upload_id}
Content-Type: application/json
```
**Path Parameters**
| Parameter | Description |
| --------- | ----------------------------------------------- |
| upload_id | Upload session ID returned by the init endpoint |
**Response**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**Error Responses**
| Status | Description |
| ------ | ------------------------------------------------------------------------------- |
| 400 | Session does not support direct confirmation |
| 404 | Upload session does not exist or has expired / file not uploaded or upload failed |
---
### 4. Query Upload Status
Query the current state of an upload session.
**Request**
```
GET /presign/upload/status/{upload_id}
```
**Response**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"file_name": "document.pdf",
"file_size": 1048576,
"mode": "direct",
"created_at": "2024-01-01T12:00:00",
"expires_at": "2024-01-01T12:15:00",
"is_expired": false
}
}
```
**Error Responses**
| Status | Description |
| ------ | ---------------------------------- |
| 404 | Upload session does not exist |
---
### 5. Cancel Upload
Cancel an upload session and clean up related resources.
**Request**
```
DELETE /presign/upload/{upload_id}
```
**Response**
```json
{
"code": 200,
"detail": {
"message": "Upload session cancelled"
}
}
```
**Error Responses**
| Status | Description |
| ------ | ---------------------------------- |
| 404 | Upload session does not exist |
---
## Frontend Integration Example
### Complete Upload Flow (JavaScript/TypeScript)
```typescript
interface PresignInitResponse {
upload_id: string
upload_url: string
mode: 'direct' | 'proxy'
expires_in: number
}
interface UploadResult {
code: string
name: string
}
async function uploadFile(
file: File,
expireValue: number = 1,
expireStyle: string = 'day'
): Promise<UploadResult> {
// 1. Initialize upload
const initResponse = await fetch('/presign/upload/init', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
file_name: file.name,
file_size: file.size,
expire_value: expireValue,
expire_style: expireStyle,
}),
})
const initData = await initResponse.json()
if (initData.code !== 200) {
throw new Error(initData.detail)
}
const { upload_id, upload_url, mode } = initData.detail as PresignInitResponse
// 2. Upload file based on mode
if (mode === 'direct') {
// Direct mode: upload to S3
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: file,
headers: { 'Content-Type': 'application/octet-stream' },
})
if (!uploadResponse.ok) {
throw new Error('S3 upload failed')
}
// 3. Confirm upload
const confirmResponse = await fetch(
`/presign/upload/confirm/${upload_id}`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
}
)
const confirmData = await confirmResponse.json()
if (confirmData.code !== 200) {
throw new Error(confirmData.detail)
}
return confirmData.detail
} else {
// Proxy mode: upload through the server
const formData = new FormData()
formData.append('file', file)
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: formData,
})
const uploadData = await uploadResponse.json()
if (uploadData.code !== 200) {
throw new Error(uploadData.detail)
}
return uploadData.detail
}
}
// Usage example
const file = document.querySelector('input[type="file"]').files[0]
const result = await uploadFile(file, 7, 'day')
console.log('Share code:', result.code)
```
---
## Notes
1. **Session validity**: upload sessions expire after 15 minutes by default; complete the upload within the window
2. **File size limit**: bounded by the `upload_size` system setting
3. **Expiration types**: `day`, `hour`, `minute`, `forever`, `count`
4. **CORS**: in direct mode, S3 must have a proper CORS policy configured
5. **Retry**: implement upload retry logic on the client side for failed uploads
+1 -1
View File
@@ -1,6 +1,6 @@
## [Unreleased]
<!-- Current release: 2.5.2 x-release-please-version -->
<!-- Current release: 2.8.1 x-release-please-version -->
### Added
+37 -37
View File
@@ -22,9 +22,9 @@ On first startup, the system uses default configuration from `core/settings.py`.
| `name` | string | `文件快递柜 - FileCodeBox` | Site name, displayed in page title and navigation bar |
| `description` | string | `开箱即用的文件快传系统` | Site description, used for SEO |
| `keywords` | string | `FileCodeBox, 文件快递柜...` | Site keywords, used for SEO |
| `serverHost` | string | `0.0.0.0` | Service listening address |
| `serverPort` | int | `12345` | Service listening port |
| `serverWorkers` | int | `1` | Worker count; keep one worker for SQLite deployments |
| `server_host` | string | `0.0.0.0` | Service listening address |
| `server_port` | int | `12345` | Service listening port |
| `server_workers` | int | `1` | Worker count; keep one worker for SQLite deployments |
### Notification Settings
@@ -33,7 +33,7 @@ On first startup, the system uses default configuration from `core/settings.py`.
| `notify_title` | string | `系统通知` | Notification title |
| `notify_content` | string | Welcome message | Notification content, supports HTML |
| `page_explain` | string | Legal disclaimer | Footer explanation text |
| `robotsText` | string | `User-agent: *\nDisallow: /` | robots.txt content |
| `robots_text` | string | `User-agent: *\nDisallow: /` | robots.txt content |
## Upload Settings
@@ -41,21 +41,21 @@ On first startup, the system uses default configuration from `core/settings.py`.
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `openUpload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `uploadSize` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `enableChunk` | int | `0` | Enable chunked upload (1=enabled, 0=disabled) |
| `open_upload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `upload_size` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `enable_chunk` | int | `0` | Enable chunked upload (1=enabled, 0=disabled) |
| `allowed_file_types` | list | `["*"]` | Allowed extensions; `*` allows every file type |
::: warning Note
`uploadSize` is in bytes. 10MB = 10 * 1024 * 1024 = 10485760 bytes
`upload_size` is in bytes. 10MB = 10 * 1024 * 1024 = 10485760 bytes
:::
### Upload Rate Limiting
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `uploadMinute` | int | `1` | Upload limit time window (minutes) |
| `uploadCount` | int | `10` | Maximum uploads allowed within the time window |
| `upload_minute` | int | `1` | Upload limit time window (minutes) |
| `upload_count` | int | `10` | Maximum uploads allowed within the time window |
Example: Default configuration allows up to 10 uploads per minute.
@@ -64,7 +64,7 @@ Example: Default configuration allows up to 10 uploads per minute.
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `max_save_seconds` | int | `0` | Maximum file retention time (seconds), 0 means no limit |
Expiration methods explained:
@@ -80,8 +80,8 @@ Expiration methods explained:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `themesSelect` | string | `themes/2024` | Currently active theme |
| `themesChoices` | list | See below | Available themes list |
| `themes_select` | string | `themes/2024` | Currently active theme |
| `themes_choices` | list | See below | Available themes list |
Default available themes:
```json
@@ -113,8 +113,8 @@ Default available themes:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `admin_token` | string | Set during setup | Admin login password |
| `showAdminAddr` | int | `0` | Show admin panel entry on homepage (1=show, 0=hide) |
| `adminSessionExpire` | int | `2592000` | Admin session lifetime in seconds (30 days) |
| `show_admin_addr` | int | `0` | Show admin panel entry on homepage (1=show, 0=hide) |
| `admin_session_expire` | int | `2592000` | Admin session lifetime in seconds (30 days) |
::: danger Security Warning
The setup page is shown automatically while the system is uninitialized. Complete setup before exposing a production service to the public internet.
@@ -126,9 +126,9 @@ The setup page is shown automatically while the system is uninitialized. Complet
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `errorMinute` | int | `1` | Error limit time window (minutes) |
| `errorCount` | int | `10` | Maximum errors allowed within the time window |
| `trustedProxies` | list | `[]` | Trusted reverse-proxy IPs used to resolve client addresses |
| `error_minute` | int | `1` | Error limit time window (minutes) |
| `error_count` | int | `10` | Maximum errors allowed within the time window |
| `trusted_proxies` | list | `[]` | Trusted reverse-proxy IPs used to resolve client addresses |
This setting prevents brute-force attacks on extraction codes.
@@ -140,7 +140,7 @@ This setting prevents brute-force attacks on extraction codes.
|---------|------|---------|-------------|
| `file_storage` | string | `local` | Storage backend type |
| `storage_path` | string | `""` | Custom storage path |
| `storageLimit` | int | `0` | Total storage quota in bytes; 0 means unlimited |
| `storage_limit` | int | `0` | Total storage quota in bytes; 0 means unlimited |
Supported storage types:
- `local` - Local storage
@@ -160,11 +160,11 @@ Suitable for personal or small team use with relaxed limits:
```python
{
"name": "My File Share",
"uploadSize": 52428800, # 50MB
"uploadMinute": 5, # 5 minutes
"uploadCount": 20, # Max 20 uploads
"expireStyle": ["day", "hour", "forever"],
"showAdminAddr": 1
"upload_size": 52428800, # 50MB
"upload_minute": 5, # 5 minutes
"upload_count": 20, # Max 20 uploads
"expire_style": ["day", "hour", "forever"],
"show_admin_addr": 1
}
```
@@ -175,14 +175,14 @@ Suitable for public services requiring stricter limits:
```python
{
"name": "Public File Box",
"uploadSize": 10485760, # 10MB
"uploadMinute": 1, # 1 minute
"uploadCount": 5, # Max 5 uploads
"errorMinute": 5, # 5 minutes
"errorCount": 3, # Max 3 errors
"expireStyle": ["hour", "minute", "count"],
"upload_size": 10485760, # 10MB
"upload_minute": 1, # 1 minute
"upload_count": 5, # Max 5 uploads
"error_minute": 5, # 5 minutes
"error_count": 3, # Max 3 errors
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # Max retention 1 day
"showAdminAddr": 0
"show_admin_addr": 0
}
```
@@ -193,13 +193,13 @@ Suitable for enterprise internal use with large file and chunked upload support:
```python
{
"name": "Enterprise File Transfer",
"uploadSize": 1073741824, # 1GB
"enableChunk": 1, # Enable chunked upload
"uploadMinute": 10, # 10 minutes
"uploadCount": 100, # Max 100 uploads
"expireStyle": ["day", "forever"],
"upload_size": 1073741824, # 1GB
"enable_chunk": 1, # Enable chunked upload
"upload_minute": 10, # 10 minutes
"upload_count": 100, # Max 100 uploads
"expire_style": ["day", "forever"],
"file_storage": "s3", # Use S3 storage
"showAdminAddr": 1
"show_admin_addr": 1
}
```
+9 -1
View File
@@ -21,6 +21,10 @@ FileCodeBox is a simple and efficient file sharing tool that supports temporary
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
@@ -32,7 +36,11 @@ The named volume persists configuration and uploaded files across container upgr
| `HOST` | `0.0.0.0` | Service listening address |
| `PORT` | `12345` | Service port |
| `WORKERS` | `1` | Worker count; keep one worker for SQLite deployments |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warning`, or `error` |
| `APP_ENV` | `production` (Docker) | Runtime environment; production disables HTTP access logs by default |
| `LOG_LEVEL` | `warning` (Docker) | `debug`, `info`, `warning`, or `error` |
| `ACCESS_LOG` | `false` (production) | Enable HTTP access logs; set to `true` when debugging requests |
Production deployments output only warning, error, and other key events by default. Docker retains at most three 10 MB log files.
### Manual Deployment
+11 -3
View File
@@ -1,6 +1,6 @@
<div align="center">
<img src="https://fastly.jsdelivr.net/gh/vastsa/FileCodeBox@V1.6/static/banners/img_1.png" alt="FileCodeBox Logo">
<img src="/screenshots/retrieve.webp" alt="FileCodeBox" width="860" />
<p><em>Share text and files anonymously with a passcode, like picking up a package</em></p>
@@ -136,7 +136,15 @@ Command-line download
### Docker Deployment
```bash
docker run -d --restart unless-stopped -p 12345:12345 -v fcb-data:/app/data --name filecodebox lanol/filecodebox:latest
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
### Manual Deployment
@@ -218,7 +226,7 @@ npm run dev
## ❓ FAQ
### Q: How to modify upload size limit?
A: Change `uploadSize` in admin panel
A: Change `upload_size` in admin panel
### Q: How to configure storage engine?
A: Select storage engine and configure parameters in admin panel
+56 -39
View File
@@ -4,6 +4,10 @@ FileCodeBox provides a fully-featured admin panel that allows administrators to
## Accessing the Admin Panel
<p align="center">
<img src="/screenshots/login.webp" alt="Admin login" width="640" />
</p>
### Login Method
The admin panel is located at the `/admin` path. Access method:
@@ -23,10 +27,10 @@ By default, the admin panel entry is not shown on the homepage. You can control
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `showAdminAddr` | int | `0` | Show admin entry on homepage (1=show, 0=hide) |
| `show_admin_addr` | int | `0` | Show admin entry on homepage (1=show, 0=hide) |
::: warning Security Recommendation
For public services, it's recommended to keep `showAdminAddr` at `0` and access the admin panel directly via the `/admin` path to reduce the risk of malicious scanning.
For public services, it's recommended to keep `show_admin_addr` at `0` and access the admin panel directly via the `/admin` path to reduce the risk of malicious scanning.
:::
### Authentication Mechanism
@@ -39,6 +43,10 @@ The admin panel uses JWT (JSON Web Token) for authentication:
## Dashboard
<p align="center">
<img src="/screenshots/dashboard.webp" alt="Dashboard" width="860" />
</p>
After logging in, you first see the dashboard page, which displays the overall system status.
### Statistics
@@ -67,6 +75,10 @@ Storage usage is displayed in bytes. For example, `10485760` represents approxim
## File Management
<p align="center">
<img src="/screenshots/files.webp" alt="File management" width="860" />
</p>
### File List
The file management page displays all shared files in the system, supporting pagination and search.
@@ -154,52 +166,57 @@ When modifying extraction codes, the system checks if the new code is already in
## Local File Management
In addition to managing shared files, the admin panel also provides local file management functionality for managing files in the `data/local` directory.
In addition to managing shared files, the admin panel has a Local Share page for browsing `data/local` (including subfolders) and minting extract codes.
Mount a NAS share at `/app/data/local` to generate extract codes for files already on disk, without downloading and re-uploading them.
```yaml
volumes:
- fcb-data:/app/data:rw
# Read-only NAS share for admin browsing and extract-code generation
- /volume1/share:/app/data/local:ro
```
### View Local Files
The local file list displays all files in the `data/local` directory:
The Local Share page supports nested folders. List API:
| Information | Description |
|-------------|-------------|
| Filename | Complete filename |
| Creation Time | File creation time |
| File Size | File size (bytes) |
```
GET /admin/local/lists?path=movies
```
`path` is relative to `data/local`; leave it empty for the root. Response:
| Field | Description |
|------|------|
| `path` | Current directory |
| `parent` | Parent directory |
| `truncated` | Whether the listing was truncated |
| `items[].type` | `dir` or `file` |
| `items[].path` | Path relative to `data/local`, used as `filename` when sharing |
### Share Local Files
You can quickly share local files:
1. Select the file to share in the local file list
1. Open the target subfolder and pick a file
2. Set expiration method and value
3. Click the share button
4. System generates extraction code
3. Create an extract code
Sharing is a **zero-copy reference**: the file is not copied into `data/share` and does not count against storage quota. Expiring or deleting the share record **does not delete the original NAS file**.
**Share parameters:**
| Parameter | Description |
|-----------|-------------|
| `filename` | Filename to share |
| `filename` | Path relative to `data/local`, e.g. `movies/a.mp4` |
| `expire_style` | Expiration method (day/hour/minute/forever/count) |
| `expire_value` | Expiration value (days/hours/minutes/download count) |
### Delete Local Files
You can delete files in the `data/local` directory:
1. Find the file to delete in the local file list
2. Click the delete button
3. Confirm deletion
::: tip Use Cases
Local file management is useful for:
- Sharing files after batch uploading to the server
- Managing files uploaded to the server through other means
- Cleaning up unnecessary local files
:::
## System Settings
<p align="center">
<img src="/screenshots/settings.webp" alt="System settings" width="860" />
</p>
### View Configuration
On the system settings page, you can view the current values of all configuration items. Configuration items are grouped by category:
@@ -224,15 +241,15 @@ Administrators can modify most configurations through the admin panel:
| Category | Example Settings |
|----------|------------------|
| Basic Settings | `name`, `description`, `keywords`, `notify_title`, `notify_content` |
| Upload Settings | `uploadSize`, `uploadMinute`, `uploadCount`, `openUpload`, `enableChunk` |
| Expiration Settings | `expireStyle`, `max_save_seconds` |
| Theme Settings | `themesSelect`, `opacity`, `background` |
| Security Settings | `admin_token`, `showAdminAddr`, `errorMinute`, `errorCount` |
| Upload Settings | `upload_size`, `upload_minute`, `upload_count`, `open_upload`, `enable_chunk` |
| Expiration Settings | `expire_style`, `max_save_seconds` |
| Theme Settings | `themes_select`, `opacity`, `background` |
| Security Settings | `admin_token`, `show_admin_addr`, `error_minute`, `error_count` |
| Storage Settings | `file_storage`, `storage_path` and storage backend-specific configurations |
::: warning Note
- `admin_token` (admin password) cannot be set to empty
- `themesChoices` (theme list) cannot be modified through the admin panel
- `themes_choices` (theme list) cannot be modified through the admin panel
- After modifying storage settings, existing files will not be automatically migrated
:::
@@ -322,7 +339,7 @@ Content-Type: application/json
**Get Local File List**
```
GET /admin/local/lists
GET /admin/local/lists?path=movies
Authorization: Bearer <token>
```
@@ -333,7 +350,7 @@ Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "example.txt"
"filename": "movies/example.txt"
}
```
@@ -344,7 +361,7 @@ Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "example.txt",
"filename": "movies/example.txt",
"expire_style": "day",
"expire_value": 7
}
@@ -366,7 +383,7 @@ Content-Type: application/json
{
"admin_token": "new-password",
"uploadSize": 52428800
"upload_size": 52428800
}
```
+40 -40
View File
@@ -34,14 +34,14 @@ Change the admin password through the admin panel:
### Hide Admin Entry
By default, the admin panel entry is hidden. You can control whether to show the admin entry on the homepage via the `showAdminAddr` configuration:
By default, the admin panel entry is hidden. You can control whether to show the admin entry on the homepage via the `show_admin_addr` configuration:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `showAdminAddr` | int | `0` | Show admin entry (1=show, 0=hide) |
| `show_admin_addr` | int | `0` | Show admin entry (1=show, 0=hide) |
::: tip Recommendation
For public services, it's recommended to keep `showAdminAddr` at `0` and access the admin panel directly via the `/admin` path.
For public services, it's recommended to keep `show_admin_addr` at `0` and access the admin panel directly via the `/admin` path.
:::
## IP Rate Limiting
@@ -54,12 +54,12 @@ Limit the number of uploads from a single IP within a specified time:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `uploadMinute` | int | `1` | Upload limit time window (minutes) |
| `uploadCount` | int | `10` | Maximum uploads allowed within the time window |
| `upload_minute` | int | `1` | Upload limit time window (minutes) |
| `upload_count` | int | `10` | Maximum uploads allowed within the time window |
**How it works:**
- System records upload requests from each IP
- When an IP's upload count reaches `uploadCount` within `uploadMinute` minutes
- When an IP's upload count reaches `upload_count` within `upload_minute` minutes
- Subsequent upload requests from that IP will be rejected with HTTP 423 error
- Counter resets after the time window expires
@@ -68,14 +68,14 @@ Limit the number of uploads from a single IP within a specified time:
```python
# Relaxed configuration: Max 20 uploads in 5 minutes
{
"uploadMinute": 5,
"uploadCount": 20
"upload_minute": 5,
"upload_count": 20
}
# Strict configuration: Max 3 uploads in 1 minute
{
"uploadMinute": 1,
"uploadCount": 3
"upload_minute": 1,
"upload_count": 3
}
```
@@ -86,13 +86,13 @@ Limit the number of error attempts from a single IP to prevent brute-force attac
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `errorMinute` | int | `1` | Error limit time window (minutes) |
| `errorCount` | int | `10` | Maximum errors allowed within the time window |
| `error_minute` | int | `1` | Error limit time window (minutes) |
| `error_count` | int | `10` | Maximum errors allowed within the time window |
**How it works:**
- When a user enters an incorrect extraction code, the system records the error count for that IP
- When error count reaches `errorCount`, that IP will be temporarily locked
- Lock duration is `errorMinute` minutes
- When error count reaches `error_count`, that IP will be temporarily locked
- Lock duration is `error_minute` minutes
- During lockout, all extraction requests from that IP will be rejected
**Configuration example:**
@@ -100,8 +100,8 @@ Limit the number of error attempts from a single IP to prevent brute-force attac
```python
# Anti-brute-force configuration: Max 3 errors in 5 minutes
{
"errorMinute": 5,
"errorCount": 3
"error_minute": 5,
"error_count": 3
}
```
@@ -115,8 +115,8 @@ The default allows up to 10 errors per IP each minute. Public services can tight
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `uploadSize` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `openUpload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `upload_size` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `open_upload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
**Common size conversions:**
- 10MB = 10 * 1024 * 1024 = `10485760`
@@ -130,7 +130,7 @@ Through file expiration mechanisms, you can automatically clean up expired files
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `max_save_seconds` | int | `0` | Maximum file retention time (seconds), 0 means no limit |
**Expiration methods explained:**
@@ -150,7 +150,7 @@ For public services, it's recommended to:
```python
# Recommended configuration for public services
{
"expireStyle": ["hour", "minute", "count"],
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400 # Max retention 1 day
}
```
@@ -161,7 +161,7 @@ In some cases, you may need to temporarily disable upload functionality:
```python
{
"openUpload": 0 # Disable upload functionality
"open_upload": 0 # Disable upload functionality
}
```
@@ -197,7 +197,7 @@ server {
add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# Limit request body size (match uploadSize configuration)
# Limit request body size (match upload_size configuration)
client_max_body_size 100M;
# Pass real IP
@@ -236,7 +236,7 @@ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
**2. Request Body Size Limit**
Nginx's `client_max_body_size` should match or be slightly larger than FileCodeBox's `uploadSize` configuration:
Nginx's `client_max_body_size` should match or be slightly larger than FileCodeBox's `upload_size` configuration:
```nginx
client_max_body_size 100M; # Allow max 100MB uploads
@@ -269,7 +269,7 @@ your-domain.com {
Before deploying FileCodeBox, confirm the following security configurations:
- [ ] Completed first-run setup and set the admin password `admin_token`
- [ ] Hidden admin entry `showAdminAddr: 0`
- [ ] Hidden admin entry `show_admin_addr: 0`
- [ ] Configured appropriate upload rate limiting
- [ ] Configured error rate limiting to prevent brute-force attacks
- [ ] Set reasonable file size limits
@@ -285,15 +285,15 @@ Before deploying FileCodeBox, confirm the following security configurations:
```python
{
"admin_token": "your-very-secure-password",
"showAdminAddr": 0,
"uploadSize": 10485760, # 10MB
"uploadMinute": 1,
"uploadCount": 5,
"errorMinute": 5,
"errorCount": 3,
"expireStyle": ["hour", "minute", "count"],
"show_admin_addr": 0,
"upload_size": 10485760, # 10MB
"upload_minute": 1,
"upload_count": 5,
"error_minute": 5,
"error_count": 3,
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # Max 1 day
"openUpload": 1
"open_upload": 1
}
```
@@ -302,15 +302,15 @@ Before deploying FileCodeBox, confirm the following security configurations:
```python
{
"admin_token": "internal-secure-password",
"showAdminAddr": 1,
"uploadSize": 104857600, # 100MB
"uploadMinute": 5,
"uploadCount": 50,
"errorMinute": 1,
"errorCount": 5,
"expireStyle": ["day", "hour", "forever"],
"show_admin_addr": 1,
"upload_size": 104857600, # 100MB
"upload_minute": 5,
"upload_count": 50,
"error_minute": 1,
"error_count": 5,
"expire_style": ["day", "hour", "forever"],
"max_save_seconds": 0, # No limit
"openUpload": 1
"open_upload": 1
}
```
+12 -4
View File
@@ -4,6 +4,10 @@ FileCodeBox provides simple and easy-to-use file and text sharing functionality.
## Sharing Methods
<p align="center">
<img src="/screenshots/send.webp" alt="Send page" width="720" />
</p>
FileCodeBox supports two sharing methods:
1. **Text Sharing** - Share text content directly, suitable for code snippets, configuration files, etc.
@@ -62,10 +66,10 @@ Response example:
### File Size Limit
The default maximum single file upload size is **10MB**. Administrators can modify this limit via the `uploadSize` configuration.
The default maximum single file upload size is **10MB**. Administrators can modify this limit via the `upload_size` configuration.
::: tip Tip
If you need to upload large files, contact the administrator to enable chunked upload functionality or adjust the `uploadSize` configuration.
If you need to upload large files, contact the administrator to enable chunked upload functionality or adjust the `upload_size` configuration.
:::
### Supported Upload Methods
@@ -112,7 +116,7 @@ FileCodeBox supports multiple flexible expiration methods:
| By Count | `count` | File expires after specified download count |
::: info Note
- Administrators can control available expiration methods via the `expireStyle` configuration
- Administrators can control available expiration methods via the `expire_style` configuration
- Administrators can limit maximum file retention time via the `max_save_seconds` configuration
:::
@@ -137,6 +141,10 @@ expire_value=5, expire_style=count
## Retrieving Files
<p align="center">
<img src="/screenshots/retrieve.webp" alt="Retrieve page" width="720" />
</p>
### How to Use
1. Enter the extraction code in the "Retrieve File" area on the homepage
@@ -200,7 +208,7 @@ This endpoint returns file content directly, suitable for direct browser access.
## Chunked Upload (Large Files)
For large file uploads, FileCodeBox supports chunked upload functionality. This feature requires administrator enablement (`enableChunk=1`).
For large file uploads, FileCodeBox supports chunked upload functionality. This feature requires administrator enablement (`enable_chunk=1`).
### Chunked Upload Flow
+1
View File
@@ -35,6 +35,7 @@ storage_path=
- Files are stored by default in `data/share/data/` directory
- Subdirectories are automatically created by date: `year/month/day/fileID/`
- In production, it's recommended to mount the `data/` directory to persistent storage
- Existing NAS files can be mounted at `data/local` (including subfolders). Admin Local Share references them in place to mint extract codes and does not copy them into `data/share`
## S3-Compatible Storage
+23 -19
View File
@@ -4,6 +4,10 @@ FileCodeBox provides multiple flexible file upload methods, supporting both regu
## Upload Methods
<p align="center">
<img src="/screenshots/send.webp" alt="Upload and send page" width="720" />
</p>
FileCodeBox supports the following upload methods:
### Drag and Drop Upload
@@ -47,7 +51,7 @@ Paste upload only supports image formats, not other file types. Specific support
| Setting | Default | Description |
|---------|---------|-------------|
| `uploadSize` | 10MB | Maximum single file upload size |
| `upload_size` | 10MB | Maximum single file upload size |
### Modify Upload Limits
@@ -55,11 +59,11 @@ Administrators can modify upload size limits through the admin panel or configur
```python
# Set maximum upload size to 100MB
uploadSize = 104857600 # 100 * 1024 * 1024
upload_size = 104857600 # 100 * 1024 * 1024
```
::: info Note
`uploadSize` is in bytes. Common conversions:
`upload_size` is in bytes. Common conversions:
- 10MB = 10485760
- 50MB = 52428800
- 100MB = 104857600
@@ -143,7 +147,7 @@ curl -L "http://localhost:12345/share/select/?code=YOUR_CODE" -o downloaded_file
```
::: tip When Authentication Required
If guest upload is disabled in admin panel (`openUpload=0`), you need to login first:
If guest upload is disabled in admin panel (`open_upload=0`), you need to login first:
```bash
# 1. Login to get token
@@ -171,7 +175,7 @@ curl -X POST "http://localhost:12345/share/text/" \
For large files, FileCodeBox supports chunked upload functionality. Chunked upload splits large files into multiple small chunks for separate uploading, supporting resume capability.
::: warning Prerequisite
Chunked upload functionality requires administrator enablement: `enableChunk=1`
Chunked upload functionality requires administrator enablement: `enable_chunk=1`
:::
### Chunked Upload Flow
@@ -359,7 +363,7 @@ await fetch(`/chunk/upload/complete/${upload_id}`, {
| HTTP Status | Error Message | Cause | Solution |
|-------------|---------------|-------|----------|
| 403 | Size exceeds limit | File exceeds `uploadSize` limit | Reduce file size or contact administrator to adjust limit |
| 403 | Size exceeds limit | File exceeds `upload_size` limit | Reduce file size or contact administrator to adjust limit |
| 403 | Upload rate limit | Exceeded IP upload rate limit | Wait for limit time window before retrying |
| 400 | Invalid expiration type | `expire_style` value not in allowed list | Use a valid expiration method |
| 404 | Upload session not found | `upload_id` invalid or expired | Re-initialize upload |
@@ -372,8 +376,8 @@ The system has rate limits on upload operations to prevent abuse:
| Setting | Default | Description |
|---------|---------|-------------|
| `uploadMinute` | 1 | Limit time window (minutes) |
| `uploadCount` | 10 | Maximum uploads within time window |
| `upload_minute` | 1 | Limit time window (minutes) |
| `upload_count` | 10 | Maximum uploads within time window |
When rate limit is exceeded, you need to wait for the time window to pass before continuing uploads.
@@ -391,26 +395,26 @@ When rate limit is exceeded, you need to wait for the time window to pass before
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `openUpload` | int | 1 | Enable upload (1=enabled, 0=disabled) |
| `uploadSize` | int | 10485760 | Maximum upload size (bytes) |
| `enableChunk` | int | 0 | Enable chunked upload (1=enabled, 0=disabled) |
| `uploadMinute` | int | 1 | Upload rate limit time window (minutes) |
| `uploadCount` | int | 10 | Maximum uploads within time window |
| `expireStyle` | list | ["day","hour","minute","forever","count"] | Allowed expiration methods |
| `open_upload` | int | 1 | Enable upload (1=enabled, 0=disabled) |
| `upload_size` | int | 10485760 | Maximum upload size (bytes) |
| `enable_chunk` | int | 0 | Enable chunked upload (1=enabled, 0=disabled) |
| `upload_minute` | int | 1 | Upload rate limit time window (minutes) |
| `upload_count` | int | 10 | Maximum uploads within time window |
| `expire_style` | list | ["day","hour","minute","forever","count"] | Allowed expiration methods |
### Configuration Example
```python
# Allow 100MB file uploads, enable chunked upload
uploadSize = 104857600
enableChunk = 1
upload_size = 104857600
enable_chunk = 1
# Relax upload rate limit: max 50 uploads per 5 minutes
uploadMinute = 5
uploadCount = 50
upload_minute = 5
upload_count = 50
# Only allow expiration by days and count
expireStyle = ["day", "count"]
expire_style = ["day", "count"]
```
## Next Steps
+25 -1
View File
@@ -37,8 +37,12 @@ features:
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v ./data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:2.5.2 # x-release-please-version
lanol/filecodebox:2.8.1 # x-release-please-version
```
Open `http://localhost:12345` and complete the first-run setup.
@@ -61,6 +65,26 @@ Open `http://localhost:12345` and complete the first-run setup.
</a>
</div>
## Interface preview
<p align="center">
<img src="/screenshots/retrieve.webp" alt="Retrieve files" width="860" />
</p>
<p align="center">
<img src="/screenshots/send.webp" alt="Send files" width="48%" />
&nbsp;
<img src="/screenshots/dashboard.webp" alt="Admin dashboard" width="48%" />
</p>
<table>
<tr>
<td width="33%"><img src="/screenshots/login.webp" alt="Admin login" /></td>
<td width="33%"><img src="/screenshots/files.webp" alt="File management" /></td>
<td width="33%"><img src="/screenshots/settings.webp" alt="System settings" /></td>
</tr>
</table>
::: warning Responsible use
Use FileCodeBox only for lawful file and text sharing. Operators are responsible for deployment security, data compliance, and content moderation.
:::
+37 -37
View File
@@ -22,9 +22,9 @@ FileCodeBox 支持两种配置方式:
| `name` | string | `文件快递柜 - FileCodeBox` | 站点名称,显示在页面标题和导航栏 |
| `description` | string | `开箱即用的文件快传系统` | 站点描述,用于 SEO |
| `keywords` | string | `FileCodeBox, 文件快递柜...` | 站点关键词,用于 SEO |
| `serverHost` | string | `0.0.0.0` | 服务监听地址 |
| `serverPort` | int | `12345` | 服务监听端口 |
| `serverWorkers` | int | `1` | 工作进程数;SQLite 部署建议保持单进程 |
| `server_host` | string | `0.0.0.0` | 服务监听地址 |
| `server_port` | int | `12345` | 服务监听端口 |
| `server_workers` | int | `1` | 工作进程数;SQLite 部署建议保持单进程 |
### 通知设置
@@ -33,7 +33,7 @@ FileCodeBox 支持两种配置方式:
| `notify_title` | string | `系统通知` | 通知标题 |
| `notify_content` | string | 欢迎信息 | 通知内容,支持 HTML |
| `page_explain` | string | 法律声明 | 页面底部说明文字 |
| `robotsText` | string | `User-agent: *\nDisallow: /` | robots.txt 内容 |
| `robots_text` | string | `User-agent: *\nDisallow: /` | robots.txt 内容 |
## 上传设置
@@ -41,21 +41,21 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `openUpload` | int | `1` | 是否开启上传功能(1=开启,0=关闭) |
| `uploadSize` | int | `10485760` | 单文件最大上传大小(字节),默认 10MB |
| `enableChunk` | int | `0` | 是否启用分片上传(1=启用,0=禁用) |
| `open_upload` | int | `1` | 是否开启上传功能(1=开启,0=关闭) |
| `upload_size` | int | `10485760` | 单文件最大上传大小(字节),默认 10MB |
| `enable_chunk` | int | `0` | 是否启用分片上传(1=启用,0=禁用) |
| `allowed_file_types` | list | `["*"]` | 允许上传的扩展名;`*` 表示不限制 |
::: warning 注意
`uploadSize` 的单位是字节。10MB = 10 * 1024 * 1024 = 10485760 字节
`upload_size` 的单位是字节。10MB = 10 * 1024 * 1024 = 10485760 字节
:::
### 上传频率限制
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `uploadMinute` | int | `1` | 上传限制的时间窗口(分钟) |
| `uploadCount` | int | `10` | 在时间窗口内允许的最大上传次数 |
| `upload_minute` | int | `1` | 上传限制的时间窗口(分钟) |
| `upload_count` | int | `10` | 在时间窗口内允许的最大上传次数 |
例如:默认配置表示每 1 分钟内最多允许上传 10 次。
@@ -63,7 +63,7 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | 可选的过期方式 |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | 可选的过期方式 |
| `max_save_seconds` | int | `0` | 文件最大保存时间(秒),0 表示不限制 |
过期方式说明:
@@ -79,8 +79,8 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `themesSelect` | string | `themes/2024` | 当前使用的主题 |
| `themesChoices` | list | 见下方 | 可用主题列表 |
| `themes_select` | string | `themes/2024` | 当前使用的主题 |
| `themes_choices` | list | 见下方 | 可用主题列表 |
默认可用主题:
```json
@@ -112,8 +112,8 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `admin_token` | string | 初始化页面设置 | 管理员登录密码(哈希存储) |
| `showAdminAddr` | int | `0` | 是否在首页显示管理入口(1=显示,0=隐藏) |
| `adminSessionExpire` | int | `2592000` | 管理会话有效期(秒),默认 30 天 |
| `show_admin_addr` | int | `0` | 是否在首页显示管理入口(1=显示,0=隐藏) |
| `admin_session_expire` | int | `2592000` | 管理会话有效期(秒),默认 30 天 |
::: danger 安全警告
未初始化时会自动显示初始化页面。生产环境请在服务对外开放前完成初始化。
@@ -125,9 +125,9 @@ FileCodeBox 支持两种配置方式:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `errorMinute` | int | `1` | 错误限制的时间窗口(分钟) |
| `errorCount` | int | `10` | 在时间窗口内允许的最大错误次数 |
| `trustedProxies` | list | `[]` | 可信反向代理 IP;用于安全解析客户端地址 |
| `error_minute` | int | `1` | 错误限制的时间窗口(分钟) |
| `error_count` | int | `10` | 在时间窗口内允许的最大错误次数 |
| `trusted_proxies` | list | `[]` | 可信反向代理 IP;用于安全解析客户端地址 |
此设置用于防止暴力破解提取码。
@@ -139,7 +139,7 @@ FileCodeBox 支持两种配置方式:
|--------|------|--------|------|
| `file_storage` | string | `local` | 存储后端类型 |
| `storage_path` | string | `""` | 自定义存储路径 |
| `storageLimit` | int | `0` | 总存储配额(字节),0 表示不限制 |
| `storage_limit` | int | `0` | 总存储配额(字节),0 表示不限制 |
支持的存储类型:
- `local` - 本地存储
@@ -159,11 +159,11 @@ FileCodeBox 支持两种配置方式:
```python
{
"name": "我的文件分享",
"uploadSize": 52428800, # 50MB
"uploadMinute": 5, # 5分钟
"uploadCount": 20, # 最多20次
"expireStyle": ["day", "hour", "forever"],
"showAdminAddr": 1
"upload_size": 52428800, # 50MB
"upload_minute": 5, # 5分钟
"upload_count": 20, # 最多20次
"expire_style": ["day", "hour", "forever"],
"show_admin_addr": 1
}
```
@@ -174,14 +174,14 @@ FileCodeBox 支持两种配置方式:
```python
{
"name": "公共文件快递柜",
"uploadSize": 10485760, # 10MB
"uploadMinute": 1, # 1分钟
"uploadCount": 5, # 最多5次
"errorMinute": 5, # 5分钟
"errorCount": 3, # 最多3次错误
"expireStyle": ["hour", "minute", "count"],
"upload_size": 10485760, # 10MB
"upload_minute": 1, # 1分钟
"upload_count": 5, # 最多5次
"error_minute": 5, # 5分钟
"error_count": 3, # 最多3次错误
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # 最长保存1天
"showAdminAddr": 0
"show_admin_addr": 0
}
```
@@ -192,13 +192,13 @@ FileCodeBox 支持两种配置方式:
```python
{
"name": "企业文件中转站",
"uploadSize": 1073741824, # 1GB
"enableChunk": 1, # 启用分片上传
"uploadMinute": 10, # 10分钟
"uploadCount": 100, # 最多100次
"expireStyle": ["day", "forever"],
"upload_size": 1073741824, # 1GB
"enable_chunk": 1, # 启用分片上传
"upload_minute": 10, # 10分钟
"upload_count": 100, # 最多100次
"expire_style": ["day", "forever"],
"file_storage": "s3", # 使用S3存储
"showAdminAddr": 1
"show_admin_addr": 1
}
```
+20 -2
View File
@@ -23,6 +23,10 @@ FileCodeBox 是一个简单高效的文件分享工具,支持文件临时中
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
@@ -40,8 +44,15 @@ services:
ports:
- "12345:12345"
environment:
- APP_ENV=production
- WORKERS=1
- LOG_LEVEL=info
- LOG_LEVEL=warning
- ACCESS_LOG=false
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
volumes:
fcb-data:
external: false
@@ -54,7 +65,11 @@ volumes:
| `HOST` | `0.0.0.0` | 服务监听地址 |
| `PORT` | `12345` | 服务端口 |
| `WORKERS` | `1` | 工作进程数;SQLite 部署建议保持单进程 |
| `LOG_LEVEL` | `info` | 日志级别:debug/info/warning/error |
| `APP_ENV` | `production`(Docker) | 运行环境;生产环境默认关闭 HTTP 访问日志 |
| `LOG_LEVEL` | `warning`(Docker) | 日志级别:debug/info/warning/error |
| `ACCESS_LOG` | `false`(生产环境) | 是否启用 HTTP 访问日志;调试请求时可设为 `true` |
生产环境默认只输出 warning、error 等关键日志,并限制 Docker 日志最多保留 3 个 10 MB 文件。
#### 自定义配置示例
@@ -63,8 +78,11 @@ docker run -d --restart=always \
-p 12345:12345 \
-v fcb-data:/app/data \
-e WORKERS=1 \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--name filecodebox \
--log-opt max-size=10m \
--log-opt max-file=3 \
lanol/filecodebox:latest
```
+33 -3
View File
@@ -1,6 +1,6 @@
<div align="center">
<img src="https://fastly.jsdelivr.net/gh/vastsa/FileCodeBox@V1.6/static/banners/img_1.png" alt="FileCodeBox Logo">
<img src="/screenshots/retrieve.webp" alt="FileCodeBox" width="860" />
<p><em>匿名口令分享文本和文件,像拿快递一样取文件</em></p>
@@ -15,6 +15,28 @@
</div>
## 🖼️ 界面预览
<p align="center">
<img src="/screenshots/retrieve.webp" alt="提取文件" width="860" />
</p>
<p align="center">
<img src="/screenshots/send.webp" alt="文件发送" width="48%" />
&nbsp;
<img src="/screenshots/dashboard.webp" alt="管理仪表盘" width="48%" />
</p>
<p align="center">
<img src="/screenshots/files.webp" alt="文件管理" width="48%" />
&nbsp;
<img src="/screenshots/settings.webp" alt="系统设置" width="48%" />
</p>
<p align="center">
<img src="/screenshots/login.webp" alt="管理登录" width="60%" />
</p>
## 🚀 更新计划
- [ ] 切片上传,同文件秒传,断点续传
- [ ] 用户登录重构
@@ -128,7 +150,15 @@ FileCodeBox 是一个基于 FastAPI + Vue3 开发的轻量级文件分享工具
### Docker 部署
```bash
docker run -d --restart unless-stopped -p 12345:12345 -v fcb-data:/app/data --name filecodebox lanol/filecodebox:latest
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
### 手动部署
@@ -210,7 +240,7 @@ npm run dev
## ❓ 常见问题
### Q: 如何修改上传大小限制?
A: 在管理面板中修改配置项 `uploadSize`
A: 在管理面板中修改配置项 `upload_size`
### Q: 如何配置存储引擎?
A: 在管理面板中选择存储引擎并配置相应参数
+55 -38
View File
@@ -4,6 +4,10 @@ FileCodeBox 提供了功能完善的管理面板,让管理员可以方便地
## 访问管理面板
<p align="center">
<img src="/screenshots/login.webp" alt="管理登录" width="640" />
</p>
### 登录方式
管理面板位于 `/admin` 路径。访问方式:
@@ -23,10 +27,10 @@ FileCodeBox 提供了功能完善的管理面板,让管理员可以方便地
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `showAdminAddr` | int | `0` | 是否在首页显示管理入口(1=显示,0=隐藏) |
| `show_admin_addr` | int | `0` | 是否在首页显示管理入口(1=显示,0=隐藏) |
::: warning 安全建议
在公开服务中,建议保持 `showAdminAddr` 为 `0`,通过直接访问 `/admin` 路径进入管理面板,减少被恶意扫描的风险。
在公开服务中,建议保持 `show_admin_addr` 为 `0`,通过直接访问 `/admin` 路径进入管理面板,减少被恶意扫描的风险。
:::
### 认证机制
@@ -39,6 +43,10 @@ FileCodeBox 提供了功能完善的管理面板,让管理员可以方便地
## 仪表盘
<p align="center">
<img src="/screenshots/dashboard.webp" alt="管理仪表盘" width="860" />
</p>
登录后首先看到的是仪表盘页面,展示系统的整体运行状态。
### 统计指标
@@ -67,6 +75,10 @@ FileCodeBox 提供了功能完善的管理面板,让管理员可以方便地
## 文件管理
<p align="center">
<img src="/screenshots/files.webp" alt="文件管理" width="860" />
</p>
### 文件列表
文件管理页面展示系统中所有已分享的文件,支持分页浏览和搜索。
@@ -153,52 +165,57 @@ FileCodeBox 提供了功能完善的管理面板,让管理员可以方便地
## 本地文件管理
除了管理已分享的文件,管理面板还提供了本地文件管理功能,用于管理 `data/local` 目录中的文件。
除了管理已分享的文件,管理面板还提供「本地分享」页面,用于浏览 `data/local`(含子目录)里的文件并生成提取码。
把 NAS 共享目录挂到容器的 `/app/data/local` 后,即可指定内部文件夹中的文件生成提取码,无需再下载后上传。
```yaml
volumes:
- fcb-data:/app/data:rw
# 只读挂载 NAS 分享目录,供后台浏览并生成提取码
- /volume1/share:/app/data/local:ro
```
### 查看本地文件
本地文件列表显示 `data/local` 目录中的所有文件:
后台「本地分享」支持进入子目录。列表接口:
| 信息 | 说明 |
```
GET /admin/local/lists?path=movies
```
`path` 为相对 `data/local` 的目录,留空表示根目录。返回:
| 字段 | 说明 |
|------|------|
| 文件名 | 文件的完整名称 |
| 创建时间 | 文件的创建时间 |
| 文件大小 | 文件大小(字节) |
| `path` | 当前目录 |
| `parent` | 上级目录 |
| `truncated` | 是否因条目过多被截断 |
| `items[].type` | `dir` 或 `file` |
| `items[].path` | 相对 `data/local` 的路径,分享时作为 `filename` |
### 分享本地文件
可以将本地文件快速分享:
1. 在本地文件列表中选择要分享的文件
1. 进入目标子目录,选择文件
2. 设置过期方式和过期值
3. 点击分享按钮
4. 系统生成提取码
3. 点击生成提取码
分享是**零拷贝引用**:不会把文件复制进 `data/share`,也不占用快递柜存储配额。提取码过期或在文件管理中删除记录,**不会删除 NAS 原文件**。
**分享参数:**
| 参数 | 说明 |
|------|------|
| `filename` | 要分享的文件名 |
| `filename` | 相对 `data/local` 的路径,如 `movies/a.mp4` |
| `expire_style` | 过期方式(day/hour/minute/forever/count) |
| `expire_value` | 过期值(天数/小时数/分钟数/下载次数) |
### 删除本地文件
可以删除 `data/local` 目录中的文件:
1. 在本地文件列表中找到要删除的文件
2. 点击删除按钮
3. 确认删除
::: tip 使用场景
本地文件管理功能适用于:
- 批量上传文件到服务器后进行分享
- 管理通过其他方式上传到服务器的文件
- 清理不需要的本地文件
:::
## 系统设置
<p align="center">
<img src="/screenshots/settings.webp" alt="系统设置" width="860" />
</p>
### 查看配置
在系统设置页面可以查看当前所有配置项的值。配置项按类别分组显示:
@@ -223,15 +240,15 @@ FileCodeBox 提供了功能完善的管理面板,让管理员可以方便地
| 类别 | 配置项示例 |
|------|------------|
| 基础设置 | `name`, `description`, `keywords`, `notify_title`, `notify_content` |
| 上传设置 | `uploadSize`, `uploadMinute`, `uploadCount`, `openUpload`, `enableChunk` |
| 过期设置 | `expireStyle`, `max_save_seconds` |
| 主题设置 | `themesSelect`, `opacity`, `background` |
| 安全设置 | `admin_token`, `showAdminAddr`, `errorMinute`, `errorCount` |
| 上传设置 | `upload_size`, `upload_minute`, `upload_count`, `open_upload`, `enable_chunk` |
| 过期设置 | `expire_style`, `max_save_seconds` |
| 主题设置 | `themes_select`, `opacity`, `background` |
| 安全设置 | `admin_token`, `show_admin_addr`, `error_minute`, `error_count` |
| 存储设置 | `file_storage`, `storage_path` 及各存储后端的配置 |
::: warning 注意
- `admin_token`(管理员密码)不能设置为空
- `themesChoices`(主题列表)不可通过管理面板修改
- `themes_choices`(主题列表)不可通过管理面板修改
- 修改存储设置后,已有文件不会自动迁移
:::
@@ -321,7 +338,7 @@ Content-Type: application/json
**获取本地文件列表**
```
GET /admin/local/lists
GET /admin/local/lists?path=movies
Authorization: Bearer <token>
```
@@ -332,7 +349,7 @@ Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "example.txt"
"filename": "movies/example.txt"
}
```
@@ -343,7 +360,7 @@ Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "example.txt",
"filename": "movies/example.txt",
"expire_style": "day",
"expire_value": 7
}
@@ -365,7 +382,7 @@ Content-Type: application/json
{
"admin_token": "new-password",
"uploadSize": 52428800
"upload_size": 52428800
}
```
+40 -40
View File
@@ -34,14 +34,14 @@ FileCodeBox 首次启动时不会生成默认管理员密码。请在浏览器
### 隐藏管理入口
默认情况下,管理面板入口是隐藏的。您可以通过 `showAdminAddr` 配置控制是否在首页显示管理入口:
默认情况下,管理面板入口是隐藏的。您可以通过 `show_admin_addr` 配置控制是否在首页显示管理入口:
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `showAdminAddr` | int | `0` | 是否显示管理入口(1=显示,0=隐藏) |
| `show_admin_addr` | int | `0` | 是否显示管理入口(1=显示,0=隐藏) |
::: tip 建议
在公开服务中,建议保持 `showAdminAddr` 为 `0`,通过直接访问 `/admin` 路径进入管理面板。
在公开服务中,建议保持 `show_admin_addr` 为 `0`,通过直接访问 `/admin` 路径进入管理面板。
:::
## IP 速率限制
@@ -54,12 +54,12 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `uploadMinute` | int | `1` | 上传限制的时间窗口(分钟) |
| `uploadCount` | int | `10` | 在时间窗口内允许的最大上传次数 |
| `upload_minute` | int | `1` | 上传限制的时间窗口(分钟) |
| `upload_count` | int | `10` | 在时间窗口内允许的最大上传次数 |
**工作原理:**
- 系统记录每个 IP 的上传请求
- 当某 IP 在 `uploadMinute` 分钟内的上传次数达到 `uploadCount` 时
- 当某 IP 在 `upload_minute` 分钟内的上传次数达到 `upload_count` 时
- 该 IP 的后续上传请求将被拒绝,返回 HTTP 423 错误
- 等待时间窗口过期后,计数器重置
@@ -68,14 +68,14 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
```python
# 宽松配置:5分钟内最多上传20次
{
"uploadMinute": 5,
"uploadCount": 20
"upload_minute": 5,
"upload_count": 20
}
# 严格配置:1分钟内最多上传3次
{
"uploadMinute": 1,
"uploadCount": 3
"upload_minute": 1,
"upload_count": 3
}
```
@@ -85,13 +85,13 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `errorMinute` | int | `1` | 错误限制的时间窗口(分钟) |
| `errorCount` | int | `10` | 在时间窗口内允许的最大错误次数 |
| `error_minute` | int | `1` | 错误限制的时间窗口(分钟) |
| `error_count` | int | `10` | 在时间窗口内允许的最大错误次数 |
**工作原理:**
- 当用户输入错误的提取码时,系统记录该 IP 的错误次数
- 当错误次数达到 `errorCount` 时,该 IP 将被暂时锁定
- 锁定时间为 `errorMinute` 分钟
- 当错误次数达到 `error_count` 时,该 IP 将被暂时锁定
- 锁定时间为 `error_minute` 分钟
- 锁定期间,该 IP 的所有提取请求都将被拒绝
**配置示例:**
@@ -99,8 +99,8 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
```python
# 防暴力破解配置:5分钟内最多允许3次错误
{
"errorMinute": 5,
"errorCount": 3
"error_minute": 5,
"error_count": 3
}
```
@@ -114,8 +114,8 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `uploadSize` | int | `10485760` | 单文件最大上传大小(字节),默认 10MB |
| `openUpload` | int | `1` | 是否开启上传功能(1=开启,0=关闭) |
| `upload_size` | int | `10485760` | 单文件最大上传大小(字节),默认 10MB |
| `open_upload` | int | `1` | 是否开启上传功能(1=开启,0=关闭) |
**常用大小换算:**
- 10MB = 10 * 1024 * 1024 = `10485760`
@@ -129,7 +129,7 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
| 配置项 | 类型 | 默认值 | 说明 |
|--------|------|--------|------|
| `expireStyle` | list | `["day","hour","minute","forever","count"]` | 可选的过期方式 |
| `expire_style` | list | `["day","hour","minute","forever","count"]` | 可选的过期方式 |
| `max_save_seconds` | int | `0` | 文件最大保存时间(秒),0 表示不限制 |
**过期方式说明:**
@@ -149,7 +149,7 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
```python
# 公开服务推荐配置
{
"expireStyle": ["hour", "minute", "count"],
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400 # 最长保存1天
}
```
@@ -160,7 +160,7 @@ FileCodeBox 内置了基于 IP 的速率限制机制,可以有效防止滥用
```python
{
"openUpload": 0 # 关闭上传功能
"open_upload": 0 # 关闭上传功能
}
```
@@ -196,7 +196,7 @@ server {
add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# 限制请求体大小(与 uploadSize 配置一致)
# 限制请求体大小(与 upload_size 配置一致)
client_max_body_size 100M;
# 传递真实 IP
@@ -235,7 +235,7 @@ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
**2. 请求体大小限制**
Nginx 的 `client_max_body_size` 应该与 FileCodeBox 的 `uploadSize` 配置一致或略大:
Nginx 的 `client_max_body_size` 应该与 FileCodeBox 的 `upload_size` 配置一致或略大:
```nginx
client_max_body_size 100M; # 允许上传最大 100MB
@@ -268,7 +268,7 @@ your-domain.com {
部署 FileCodeBox 前,请确认以下安全配置:
- [ ] 已完成首次初始化并设置管理员密码 `admin_token`
- [ ] 已隐藏管理入口 `showAdminAddr: 0`
- [ ] 已隐藏管理入口 `show_admin_addr: 0`
- [ ] 已配置合适的上传频率限制
- [ ] 已配置错误次数限制防止暴力破解
- [ ] 已设置合理的文件大小限制
@@ -284,15 +284,15 @@ your-domain.com {
```python
{
"admin_token": "your-very-secure-password",
"showAdminAddr": 0,
"uploadSize": 10485760, # 10MB
"uploadMinute": 1,
"uploadCount": 5,
"errorMinute": 5,
"errorCount": 3,
"expireStyle": ["hour", "minute", "count"],
"show_admin_addr": 0,
"upload_size": 10485760, # 10MB
"upload_minute": 1,
"upload_count": 5,
"error_minute": 5,
"error_count": 3,
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # 最长1天
"openUpload": 1
"open_upload": 1
}
```
@@ -301,15 +301,15 @@ your-domain.com {
```python
{
"admin_token": "internal-secure-password",
"showAdminAddr": 1,
"uploadSize": 104857600, # 100MB
"uploadMinute": 5,
"uploadCount": 50,
"errorMinute": 1,
"errorCount": 5,
"expireStyle": ["day", "hour", "forever"],
"show_admin_addr": 1,
"upload_size": 104857600, # 100MB
"upload_minute": 5,
"upload_count": 50,
"error_minute": 1,
"error_count": 5,
"expire_style": ["day", "hour", "forever"],
"max_save_seconds": 0, # 不限制
"openUpload": 1
"open_upload": 1
}
```
+12 -4
View File
@@ -4,6 +4,10 @@ FileCodeBox 提供了简单易用的文件和文本分享功能。用户可以
## 分享方式
<p align="center">
<img src="/screenshots/send.webp" alt="文件发送页面" width="720" />
</p>
FileCodeBox 支持两种分享方式:
1. **文本分享** - 直接分享文本内容,适合代码片段、配置文件等
@@ -62,10 +66,10 @@ FileCodeBox 支持两种分享方式:
### 文件大小限制
默认单文件最大上传大小为 **10MB**。管理员可以通过 `uploadSize` 配置项修改此限制。
默认单文件最大上传大小为 **10MB**。管理员可以通过 `upload_size` 配置项修改此限制。
::: tip 提示
如果需要上传大文件,请联系管理员启用分片上传功能,或调整 `uploadSize` 配置。
如果需要上传大文件,请联系管理员启用分片上传功能,或调整 `upload_size` 配置。
:::
### 支持的上传方式
@@ -111,7 +115,7 @@ FileCodeBox 支持多种灵活的过期方式:
| 按次数过期 | `count` | 文件在被下载指定次数后过期 |
::: info 说明
- 管理员可以通过 `expireStyle` 配置项控制用户可选的过期方式
- 管理员可以通过 `expire_style` 配置项控制用户可选的过期方式
- 管理员可以通过 `max_save_seconds` 配置项限制文件的最长保存时间
:::
@@ -136,6 +140,10 @@ expire_value=5, expire_style=count
## 提取文件
<p align="center">
<img src="/screenshots/retrieve.webp" alt="提取文件页面" width="720" />
</p>
### 使用方法
1. 在首页的「提取文件」区域输入提取码
@@ -199,7 +207,7 @@ expire_value=5, expire_style=count
## 分片上传(大文件)
对于大文件上传,FileCodeBox 支持分片上传功能。此功能需要管理员启用(`enableChunk=1`)。
对于大文件上传,FileCodeBox 支持分片上传功能。此功能需要管理员启用(`enable_chunk=1`)。
### 分片上传流程
-135
View File
@@ -1,135 +0,0 @@
# OneDrive作为存储的配置方法
**仅支持工作或学校账户,并且需要有管理员权限以授权API**
## 1. 需要配置的参数
```
file_storage=onedrive
onedrive_domain=XXXXXX
onedrive_client_id=XXXXXX-XXXXXX-XXXXXX-XXXXXX
onedrive_username=XXXXXX@XXXXXX
onedrive_password=XXXXXX
```
`onedrive_username`和`onedrive_password`是你的账户名(邮箱)和密码,另外两个参数需要在[微软Azure门户](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade)中注册应用后获取。
## 2. 应用注册
1. 登录[https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade),鼠标置于右上角账号处,浮窗将显示的`域`即为`onedrive_domain`的值。
![onedrive_domain](https://api.onedrive.com/v1.0/shares/s!Au-BDzXcM6_VmGCiErO85doq9Tcu/root/content)
2. 点击左上角的`+新注册`,输入名称,
* 受支持的帐户类型:选择任何组织目录(任何 Azure AD 目录 - 多租户)中的帐户和个人 Microsoft 帐户(例如,Skype、Xbox)
* 重定向 URI (可选):选择`Web`,并输入`http://localhost`
3. 完成注册后进入概述页面,在概要中找到`应用程序(客户端)ID`,即为`onedrive_client_id`的值。
![onedrive_client_id](https://api.onedrive.com/v1.0/shares/s!Au-BDzXcM6_VmGHD4CNyJxm_QBb8/root/content)
4. 此时还需要配置允许公共客户端流和API权限
* 在左侧选择`身份验证`,找到`允许的客户端流`,选择`是`,并**点击`保存`**。
![允许的客户端流](https://api.onedrive.com/v1.0/shares/s!Au-BDzXcM6_VmGJQMOlOCb2-L0Lh/root/content)
* 在左侧选择`API权限`,点击`+添加权限`,选择`Microsoft Graph`->`委托的权限`,并勾选下述权限:openid、Files中所有权限、User.Read,如下图所示。最后**点击下方的`添加权限`**。
![添加权限](https://api.onedrive.com/v1.0/shares/s!Au-BDzXcM6_VmGOZzz7sIrdXkD4w/root/content)
* 最后点击`授予管理员同意`,并**点击`是`**,最终状态变为`已授予`。
![授予管理员同意](https://api.onedrive.com/v1.0/shares/s!Au-BDzXcM6_VmGSOAnjnHUlbirbU/root/content)
## 3. 使用下述代码测试是否配置成功
安装依赖:`pip install Office365-REST-Python-Client`
```python
# common.py
import msal
domain = 'XXXXXX'
client_id = 'XXXXXX'
username = 'XXXXXX'
password = 'XXXXXX'
def acquire_token_pwd():
authority_url = f'https://login.microsoftonline.com/{domain}'
app = msal.PublicClientApplication(
authority=authority_url,
client_id=client_id
)
result = app.acquire_token_by_username_password(
username=username,
password=password,
scopes=['https://graph.microsoft.com/.default']
)
return result
```
测试登录,如果成功打印出账户名,说明配置成功。
```python
from common import acquire_token_pwd
from office365.graph_client import GraphClient
try:
client = GraphClient(acquire_token_pwd)
me = client.me.get().execute_query()
print(me.user_principal_name)
except Exception as e:
print(e)
```
测试文件上传
```python
import os
from office365.graph_client import GraphClient
from common import acquire_token_pwd
remote_path = 'tmp'
local_path = '.tmp/1689843925000.png'
def convert_link_to_download_link(link):
import re
p1 = re.search(r'https:\/\/(.+)\.sharepoint\.com', link).group(1)
p2 = re.search(r'personal\/(.+)\/', link).group(1)
p3 = re.search(rf'{p2}\/(.+)', link).group(1)
return f'https://{p1}.sharepoint.com/personal/{p2}/_layouts/52/download.aspx?share={p3}'
client = GraphClient(acquire_token_pwd)
folder = client.me.drive.root.get_by_path(remote_path)
# 1. upload
file = folder.upload_file(local_path).execute_query()
print(f'File {file.web_url} has been uploaded')
# 2. create sharing link
remote_file = folder.get_by_path(os.path.basename(local_path))
permission = remote_file.create_link("view", "anonymous").execute_query()
print(f"sharing link: {convert_link_to_download_link(permission.link.webUrl)}")
```
测试文件下载
```python
import os
from office365.graph_client import GraphClient
from common import acquire_token_pwd
remote_path = 'tmp/1689843925000.png'
local_path = '.tmp'
if not os.path.exists(local_path):
os.makedirs(local_path)
client = GraphClient(acquire_token_pwd)
remote_file = client.me.drive.root.get_by_path(remote_path).get().execute_query()
with open(os.path.join(local_path, os.path.basename(remote_path)), 'wb') as local_file:
remote_file.download(local_file).execute_query()
print(f'{remote_file.name} has been downloaded into {local_file.name}')
```
测试删除文件
```python
from office365.graph_client import GraphClient
from common import acquire_token_pwd
remote_path = 'tmp/1689843925000.png'
client = GraphClient(acquire_token_pwd)
file = client.me.drive.root.get_by_path(remote_path)
file.delete_object().execute_query()
```
Loaded 100 of 156 files, more files were not shown because too many files have changed in this diff. Show more