Files
FileCodeBox/apps/base/views.py
T
Marrrrrrrrry 57c75a7141 全量修复/补测试+UED改进 (#535)
* fix: constant-time comparison for download tokens (security item 5)

Replace the set-membership key check in /share/download with
hmac.compare_digest over the two valid window tokens; add negative-path
tests (wrong key 403, both windows accepted, foreign-code token 403).

* fix: single source for attachment headers + guard tests (security item 2)

All six Content-Disposition construction sites across the five storage
backends now go through build_attachment_headers; the header is what
neutralizes stored XSS on the same-origin download path. Guard tests
assert every get_file_response uses the builder and no hand-built
disposition reappears.

* fix: reject internal-network endpoints for storage config (security item 7)

s3_endpoint_url/s3_hostname/webdav_url are fetched server-side; with
APP_ENV=production the write entry now enforces http(s) schemes and a
loopback/private/link-local/metadata host blacklist. Development env
stays open for local minio/webdav. Already-stored values are never
re-validated, so existing deployments are unaffected.

* fix: JSON 404 for handler-raised 404s; negative-path test batch

Starlette status-code handlers take precedence over the HTTPException
class handler, so registering the theme index as the 404 handler turned
every HTTPException(404) app-wide into a 200 HTML page (found by the new
negative-path tests). The new not_found_handler serves the theme page
only to browsers (Accept: text/html) and JSON 404 to API clients.

Negative-path batch: download count exhaustion, missing chunk session
404s, expired presign session deletion, admin update_file uniqueness/
existence.

* fix: run container as non-root app user with compat volume chown (security item 3)

Entrypoint starts as root only to fix the data-dir ownership (skipped
when nothing needs changing or when the deployer sets an explicit user),
then drops to uid 10001 via gosu. Verified in-container: uvicorn runs as
app (uid 10001), setup and share round-trip work against a root-owned
volume.

* fix: compare download tokens as bytes (non-ASCII key caused 500)

* test: lock changed-only semantics for endpoint config saves

Integration regression for the background lesson: a stored internal
endpoint must not block unrelated settings saves in production mode;
changing to an internal endpoint (URL or bare-hostname form) is 400.

* fix: resolve-based SSRF check — deny IP shorthands, hex/decimal IPs, DNS maps

Probe found the static blacklist bypassed by glibc shorthands (127.1,
10.1), decimal (2130706433) and hex IP forms, *.localhost, and
loopback-mapping DNS services. Both validators now resolve the host and
re-check every resolved address; hostname tier extracts the host part
correctly for [v6]/host:port/bare-v6 forms. 44 tests.

* test: pin 404 handler dual-branch behavior (browser theme page vs API JSON)

* fix: write back normalized endpoint values; hostname dev-gate test

* test: header-injection negative paths for attachment disposition

* fix: S3 merge buffers parts to >=5MiB; missing object 404 upfront

- S3 multipart rejects parts <5MiB (EntityTooSmall) except the last; chunk
  size is client-controlled (commonly 2-4MB) so multi-chunk merges always
  failed with 500. Merge now buffers chunks into >=5MiB parts (memory
  bound: 5MB + one chunk).
- get_file_response: a 404 head_object now raises StorageError(404)
  upfront instead of signing a doomed presigned URL and failing mid-stream
  (aligns with local backend semantics from the M2 behavior unification).

* test: S3 backend coverage via in-process moto server (13 cases)

mock_aws cannot intercept aioboto3's aiohttp stack, so tests run against
a real in-process ThreadedMotoServer endpoint. Covers roundtrips, missing
object 404, merge failure modes (missing chunk / hash mismatch -> abort,
no leftover object), cleanup scoping, presign shapes, proxy dispatch.
Also adds moto[s3,server] to the dev group and CI.

* fix: WebDAV missing object 404 upfront; connection errors map to 503

HEAD 404 was silently swallowed (signed a 200 bad stream that failed
mid-download, same family as the S3 bug), and HEAD connection errors were
also swallowed so the 503 mapping never fired. Both now surface properly;
the aiohttp session is reclaimed on pre-stream exception paths.

* test: WebDAV backend coverage via in-process WebDAV server (9 cases)

Real HTTP (HEAD/GET/PUT/DELETE/MKCOL/PROPFIND) against a minimal aiohttp
WebDAV server over a temp dir: roundtrips, missing object 404, connection
error 503, delete with empty-parent cleanup, merge failure modes, chunk
cleanup scoping.

* fix: OneDrive missing object maps to 404 upfront; OpenDAL behavior pinned

OneDrive get_file_response translated graph itemNotFound into the outer
503 catch-all; now raises StorageError(404) like local/S3/WebDAV.
OpenDAL already mapped missing objects to 404 via its outer catch —
pinned with fake-operator tests (SDK not in runtime deps, instances built
via __new__). Also tightens the S3 presign URL assertion.

* test: admin write-path coverage (batch/single delete, batch update, policy actions)

13 negative-path cases for the previously untested data-modifying admin
endpoints: mixed-id aggregation with duplicates and missing records,
empty-list rejections, clear_expired_at permanence semantics, policy
action boundaries (zero limit 400, unknown action 400, missing 404),
and the DoesNotExist->404 mapping pinned for single delete.

* test: admin read-path and view-preset CRUD coverage (22 cases)

Detail/metadata/preview/admin-download/activities/local-lists+delete/
verify: missing-record 404s, note/tag truncation limits, preview max_chars
truncation, activity filtering and the 80-event clamp, local path
traversal rejection. Presets: update-vs-create by id, name truncation,
filter normalization clamps, 24-preset cap, delete-missing 404.

* refactor: move auth primitives to apps/base/auth, kill base->admin dep

JWT create/verify, bearer extraction and the share-upload gate move to
apps.base.auth (consumed by both surfaces); admin.dependencies keeps
admin session gating and service providers, re-exporting the primitives
for import-path compatibility. base.views no longer imports from
apps.admin — the last cross-app reverse dependency is gone.

* refactor: split admin services god-module (ConfigService/LocalFile out)

apps/admin/services.py (1697 lines) becomes FileService (~1200 lines) +
a compatibility facade re-exporting ConfigService, LocalFileService,
LocalFileClass and keyvalue_write_lock from their new homes
(config_service.py holds the D5 KeyValue lock; one-way imports, no
cycle). test_admin_security patch targets follow ConfigService to its
new module. Behavior unchanged; 247 tests green.

* fix: size-less uploads no longer crash (upstream seek bug)

validate_file_size's size-None branch called UploadFile.seek(0, 2),
which raises TypeError (UploadFile.seek takes one arg); the underlying
SpooledTemporaryFile.seek is sync, so awaiting it also fails. Use the
underlying file object with sync seek(0, SEEK_END). Surface discovered
by the mypy pilot run; regression test covers the size-None branch.

* chore: bump direct deps (patch/minor)

fastapi 0.139.2->0.141.1, pydantic 2.12.5->2.13.5, uvicorn 0.51.0->0.53.0,
aiohttp 3.14.2->3.14.3; lockfile regenerated. tortoise-orm 0.x->1.x major
deliberately deferred pending API-change review.

* ci: gradual mypy adoption via baseline ratchet

scripts/mypy_ratchet.py compares mypy output against
scripts/mypy-baseline.txt (line numbers stripped for edit stability):
new type errors fail CI, fixes are folded back via --regenerate.
Baseline starts at 30 known errors; the seek bug found in the pilot was
fixed before baselining. mypy joins the dev group and CI.

* fix: migrate WebDAV auth off aiohttp.BasicAuth (deprecated in 4.0)

Dependency-upgrade probe on aiohttp 3.14.3 surfaced the BasicAuth
deprecation; switch to aiohttp.encode_basic_auth() headers so the
aiohttp 4.0 upgrade does not break the WebDAV backend.

* chore: drop dead code (WebDAV _instance stub, unused LocalFileClass.write)

* test: file-validation negative paths (13 cases)

Magic-bytes spoofing (text-as-png, exe-as-pdf, mp4/webp box detection),
whitelist rule semantics (star/image-wildcard/content mismatch), chunk-0
header validation parity, and end-to-end UploadFile rejection.

* test: migration runner coverage (5 cases)

Full chain executes 001-007 in filename order and registers; rerun is
idempotent; pre-registered entries are skipped (DDL not re-executed);
the resulting schema matches deployment; a failing migration propagates
and is not registered. Uses a fresh :memory: DB without
generate_schemas so the migrations themselves build the schema (the
real deployment path).

* refactor: split core/storage.py into a package (per-backend modules)

core/storage.py (1400+ lines, six backends) becomes core/storage/ with
_base.py (data contracts, interface, shared header builder), and
local/s3/onedrive/opendal/webdav modules. The package __init__
re-exports the full historical public surface; test patch targets move
to the precise backend modules (core.storage.local.data_root etc.) and
the attachment guard scans the package. Import-only change; 264 tests
green. Storage-layer DI deferred: current tests construct instances
against patched settings, so the added surface isn't justified yet.

* refactor: TypedDict for IPRateLimit records

The rate-limit ledger mixed int and datetime under a loose Union typing,
producing 5 mypy noise errors. A TypedDict gives precise per-key types;
setdefault replaces the get-then-assign pattern. One mypy baseline error
family resolved; 264 tests green.

* chore: tighten mypy baseline after IPRateLimit TypedDict

* refactor: resolve all 26 baseline mypy errors; precise test assertions

Per-error treatment (no bulk script): honest annotations for
StoredDownload (Path/bytes/Callable), KeyValue.value (Any for JSONField),
dict[str, Any] for form/config dicts; signature split in create_token;
str() normalization in config int(); platform/type-ignore only where the
checker cannot follow (msvcrt, FastAPI Request injection). Baseline file
and ratchet compare now redundant at zero errors but kept as the guard
against regressions.

Tests: assertions tightened to status_code == 403 with HTTPException
raises; _create_share explicit parameters replace **extra passthrough;
seek-fix pointer-reset assertion persisted.
2026-09-20 03:01:08 +09:00

693 lines
25 KiB
Python

import hashlib
import hmac
import os
import uuid
from datetime import timedelta
from urllib.parse import quote, unquote
from typing import Optional, Tuple, Union
from fastapi import APIRouter, Form, Request, UploadFile, File, Depends, HTTPException
from pydantic import BaseModel, ValidationError
from starlette import status
from starlette.responses import Response
from tortoise.expressions import Case, F, Q, When
from apps.base.auth import share_required_login
from apps.base.models import FileCodes, UploadChunk, PresignUploadSession
from apps.base.quota import release_storage, reserve_storage
from apps.base.services import (
PRESIGN_SESSION_EXPIRES,
FileUploadService,
get_stored_download,
response_from_download,
stored_file_of,
validate_file_size,
)
from core.storage import StoredFile
from core.logger import logger
from apps.base.schemas import (
SelectFileModel,
InitChunkUploadModel,
CompleteUploadModel,
PresignUploadInitRequest,
)
from apps.base.file_validation import validate_file_type, validate_upload_file, validate_header_bytes
from apps.base.utils import (
ip_limit,
get_chunk_file_path_name,
validate_expire_style,
)
from apps.base.local_share import is_local_ref
from core.response import APIResponse
from core.settings import settings
from core.storage import storages, FileStorageInterface
from core.utils import (
get_file_url as get_proxy_file_url,
get_select_token,
get_now,
sanitize_filename,
)
share_api = APIRouter(prefix="/share", tags=["分享"])
# ============ 公共服务层 ============
def normalize_share_code(code: str) -> str:
return str(code or "").strip()
@share_api.post("/text/", dependencies=[Depends(share_required_login)])
async def share_text(
text: str = Form(...),
expire_value: int = Form(default=1, gt=0),
expire_style: str = Form(default="day"),
ip: str = Depends(ip_limit["upload"]),
):
validate_expire_style(expire_style)
text_size = len(text.encode("utf-8"))
max_txt_size = 222 * 1024
if text_size > max_txt_size:
raise HTTPException(status_code=403, detail="内容过多,建议采用文件形式")
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code})
@share_api.post("/file/", dependencies=[Depends(share_required_login)])
async def share_file(
expire_value: int = Form(default=1, gt=0),
expire_style: str = Form(default="day"),
file: UploadFile = File(...),
ip: str = Depends(ip_limit["upload"]),
):
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
validate_expire_style(expire_style)
detail = await FileUploadService.create_file_share(
file, size=file_size, expire_value=expire_value, expire_style=expire_style
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail=detail)
async def get_code_file_by_code(
code: str, check: bool = True
) -> Tuple[bool, Union[FileCodes, str]]:
normalized_code = normalize_share_code(code)
if not normalized_code:
return False, "文件不存在"
file_code = await FileCodes.filter(code=normalized_code).first()
if not file_code:
return False, "文件不存在"
if await file_code.is_expired() and check:
return False, "文件已过期"
return True, file_code
async def consume_file_usage(file_code: FileCodes) -> bool:
"""原子校验分享状态并记录一次实际领取。"""
now = await get_now()
eligible = (
Q(expired_count__gt=0)
| Q(expired_count__lt=0, expired_at__gt=now)
| Q(expired_count__lt=0, expired_at=None)
)
updated = await FileCodes.filter(Q(id=file_code.id) & eligible).update(
expired_count=Case(
When(expired_count__gt=0, then=F("expired_count") - 1),
default=F("expired_count"),
),
used_count=F("used_count") + 1,
)
if not updated:
return False
await file_code.refresh_from_db()
return True
def build_file_metadata(file_code: FileCodes) -> dict:
is_text = file_code.text is not None
remaining_downloads = (
file_code.expired_count if file_code.expired_count > 0 else None
)
return {
"code": file_code.code,
"name": file_code.prefix + file_code.suffix,
"size": file_code.size,
"type": "text" if is_text else "file",
"is_text": is_text,
"created_at": file_code.created_at,
"expired_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"remaining_downloads": remaining_downloads,
}
async def build_select_detail(
file_code: FileCodes, file_storage: FileStorageInterface
) -> dict:
metadata = build_file_metadata(file_code)
if file_code.text is not None:
download_url = None
elif file_code.expired_count >= 0 or is_local_ref(file_code):
# 次数限制与 NAS 引用都必须走下载接口,避免直链绕过次数或打到错误存储。
download_url = await get_proxy_file_url(file_code.code)
else:
download_url = await file_storage.get_file_url(stored_file_of(file_code))
content = file_code.text if file_code.text is not None else None
return {
**metadata,
"text": content if content is not None else download_url,
"content": content,
"download_url": download_url,
}
@share_api.get("/metadata/")
async def get_file_metadata(code: str, ip: str = Depends(ip_limit["metadata"])):
has, file_code = await get_code_file_by_code(code)
if not has:
ip_limit["metadata"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
ip_limit["metadata"].add_ip(ip)
return APIResponse(detail=build_file_metadata(file_code))
@share_api.post("/metadata/")
async def post_file_metadata(
data: SelectFileModel, ip: str = Depends(ip_limit["metadata"])
):
has, file_code = await get_code_file_by_code(data.code)
if not has:
ip_limit["metadata"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
ip_limit["metadata"].add_ip(ip)
return APIResponse(detail=build_file_metadata(file_code))
@share_api.get("/select/")
async def get_code_file(code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
has, file_code = await get_code_file_by_code(code)
if not has:
ip_limit["error"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
if not await consume_file_usage(file_code):
return APIResponse(code=404, detail="文件已过期")
if file_code.text is not None:
filename = f"{file_code.prefix or 'Text'}{file_code.suffix or '.txt'}"
return Response(
content=file_code.text,
media_type="text/plain",
headers={
"Content-Disposition": (
f"attachment; filename*=UTF-8''{quote(filename, safe='')}"
)
},
)
return response_from_download(await get_stored_download(file_code, file_storage))
@share_api.post("/select/")
async def select_file(data: SelectFileModel, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
has, file_code = await get_code_file_by_code(data.code)
if not has:
ip_limit["error"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
detail = await build_select_detail(file_code, file_storage)
download_url = detail.get("download_url")
consumes_on_download = isinstance(download_url, str) and download_url.startswith(
"/share/download?"
)
if not consumes_on_download and not await consume_file_usage(file_code):
return APIResponse(code=404, detail="文件已过期")
if not consumes_on_download:
detail.update(build_file_metadata(file_code))
return APIResponse(detail=detail)
@share_api.get("/download")
async def download_file(key: str, code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
normalized_code = normalize_share_code(code)
# 同时接受当前窗口与上一窗口 token,避免时间窗边界竞态导致偶发 403。
# 逐个常量时间比较(hmac.compare_digest),避免 set 成员判断的时序侧信道。
valid_keys = [
await get_select_token(normalized_code, offset=0),
await get_select_token(normalized_code, offset=1),
]
if not any(
hmac.compare_digest(key.encode(), candidate.encode())
for candidate in valid_keys
):
ip_limit["error"].add_ip(ip)
raise HTTPException(status_code=403, detail="下载鉴权失败")
has, file_code = await get_code_file_by_code(normalized_code)
if not has:
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
if not await consume_file_usage(file_code):
return APIResponse(code=404, detail="文件已过期")
return (
APIResponse(detail=file_code.text)
if file_code.text
else response_from_download(await get_stored_download(file_code, file_storage))
)
chunk_api = APIRouter(prefix="/chunk", tags=["切片"])
async def parse_body_model(request: Request, model_class: type[BaseModel]):
content_type = request.headers.get("content-type", "").lower()
try:
if "application/x-www-form-urlencoded" in content_type or "multipart/form-data" in content_type:
payload = dict(await request.form())
else:
payload = await request.json()
return model_class.model_validate(payload)
except ValidationError as e:
raise HTTPException(status_code=422, detail=e.errors())
except Exception:
raise HTTPException(status_code=400, detail="请求体格式错误")
async def parse_init_chunk_upload(request: Request) -> InitChunkUploadModel:
return await parse_body_model(request, InitChunkUploadModel)
async def parse_complete_upload(request: Request) -> CompleteUploadModel:
return await parse_body_model(request, CompleteUploadModel)
@chunk_api.post("/upload/init/", dependencies=[Depends(share_required_login)])
async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chunk_upload)):
safe_file_name = await sanitize_filename(unquote(data.file_name or ""))
validate_file_type(safe_file_name)
# 服务端校验:根据 total_chunks * chunk_size 计算理论最大上传量
total_chunks = (data.file_size + data.chunk_size - 1) // data.chunk_size
max_possible_size = total_chunks * data.chunk_size
if max_possible_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"文件大小超过限制,最大为 {max_size_mb:.2f} MB"
)
# 断点续传:检查是否存在相同文件的未完成上传会话
existing_session = await UploadChunk.filter(
chunk_hash=data.file_hash,
chunk_index=-1,
file_size=data.file_size,
file_name=safe_file_name,
).first()
if existing_session:
if not existing_session.save_path:
await UploadChunk.filter(upload_id=existing_session.upload_id).delete()
await release_storage(f"chunk:{existing_session.upload_id}")
else:
await reserve_storage(
f"chunk:{existing_session.upload_id}",
existing_session.file_size,
ttl_seconds=max(
1, int(getattr(settings, "chunk_expire_hours", 24))
)
* 3600,
)
uploaded_chunks = await UploadChunk.filter(
upload_id=existing_session.upload_id, completed=True
).values_list("chunk_index", flat=True)
return APIResponse(
detail={
"existed": False,
"upload_id": existing_session.upload_id,
"chunk_size": existing_session.chunk_size,
"total_chunks": existing_session.total_chunks,
"uploaded_chunks": list(uploaded_chunks),
}
)
# 创建新的上传会话
upload_id = uuid.uuid4().hex
reservation_token = f"chunk:{upload_id}"
chunk_expire_seconds = max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600
await reserve_storage(
reservation_token, data.file_size, ttl_seconds=chunk_expire_seconds
)
_, _, _, safe_file_name, save_path = await get_chunk_file_path_name(
data.file_name, upload_id
)
try:
await UploadChunk.create(
upload_id=upload_id,
chunk_index=-1,
total_chunks=total_chunks,
file_size=data.file_size,
chunk_size=data.chunk_size,
chunk_hash=data.file_hash,
file_name=safe_file_name,
save_path=save_path,
)
except Exception:
await release_storage(reservation_token)
raise
return APIResponse(
detail={
"existed": False,
"upload_id": upload_id,
"chunk_size": data.chunk_size,
"total_chunks": total_chunks,
"uploaded_chunks": [],
}
)
@chunk_api.post(
"/upload/chunk/{upload_id}/{chunk_index}",
dependencies=[Depends(share_required_login)],
)
async def upload_chunk(
upload_id: str,
chunk_index: int,
chunk: UploadFile = File(...),
):
# 获取上传会话信息
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
await reserve_storage(
f"chunk:{upload_id}",
chunk_info.file_size,
ttl_seconds=max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600,
)
# 检查分片索引有效性
if chunk_index < 0 or chunk_index >= chunk_info.total_chunks:
raise HTTPException(status.HTTP_400_BAD_REQUEST, detail="无效的分片索引")
# 检查是否已上传(支持断点续传)
existing_chunk = await UploadChunk.filter(
upload_id=upload_id, chunk_index=chunk_index, completed=True
).first()
if existing_chunk:
return APIResponse(
detail={"chunk_hash": existing_chunk.chunk_hash, "skipped": True}
)
# 读取分片数据并计算哈希
chunk_data = await chunk.read()
if chunk_index == 0:
validate_header_bytes(chunk_info.file_name, None, chunk_data[:64])
chunk_size = len(chunk_data)
# 校验分片大小不超过声明的 chunk_size
if chunk_size > chunk_info.chunk_size:
raise HTTPException(
status.HTTP_400_BAD_REQUEST,
detail=f"分片大小超过声明值: 最大 {chunk_info.chunk_size}, 实际 {chunk_size}",
)
# 计算已上传分片数,校验累计大小不超限(用分片数 * chunk_size 估算)
uploaded_count = await UploadChunk.filter(
upload_id=upload_id, completed=True
).count()
# 已上传分片的最大可能大小 + 当前分片
max_uploaded_size = uploaded_count * chunk_info.chunk_size + chunk_size
if max_uploaded_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"累计上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
chunk_hash = hashlib.sha256(chunk_data).hexdigest()
save_path = chunk_info.save_path
# 保存分片到存储
storage = storages[settings.file_storage]()
try:
await storage.save_chunk(
upload_id, chunk_index, chunk_data, chunk_hash, save_path
)
except Exception as e:
raise HTTPException(
status.HTTP_500_INTERNAL_SERVER_ERROR, detail=f"分片保存失败: {str(e)}"
)
# 更新或创建分片记录(保存成功后再记录)
await UploadChunk.update_or_create(
upload_id=upload_id,
chunk_index=chunk_index,
defaults={
"chunk_hash": chunk_hash,
"completed": True,
"file_size": chunk_info.file_size,
"total_chunks": chunk_info.total_chunks,
"chunk_size": chunk_info.chunk_size,
"file_name": chunk_info.file_name,
"save_path": chunk_info.save_path,
},
)
return APIResponse(detail={"chunk_hash": chunk_hash})
@chunk_api.delete("/upload/{upload_id}", dependencies=[Depends(share_required_login)])
async def cancel_upload(upload_id: str):
"""取消上传并清理临时文件"""
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
save_path = chunk_info.save_path
# 清理存储中的临时文件
storage = storages[settings.file_storage]()
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning("取消分片上传:清理分片文件失败 upload_id=%s", upload_id, exc_info=True)
# 清理数据库记录
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
return APIResponse(detail={"message": "上传已取消"})
@chunk_api.get(
"/upload/status/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def get_upload_status(upload_id: str):
"""获取上传状态"""
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
# 获取已上传的分片列表
uploaded_chunks = await UploadChunk.filter(
upload_id=upload_id, completed=True
).values_list("chunk_index", flat=True)
return APIResponse(
detail={
"upload_id": upload_id,
"file_name": chunk_info.file_name,
"file_size": chunk_info.file_size,
"chunk_size": chunk_info.chunk_size,
"total_chunks": chunk_info.total_chunks,
"uploaded_chunks": list(uploaded_chunks),
"progress": len(uploaded_chunks) / chunk_info.total_chunks * 100,
}
)
@chunk_api.post(
"/upload/complete/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def complete_upload(
upload_id: str,
data: CompleteUploadModel = Depends(parse_complete_upload),
ip: str = Depends(ip_limit["upload"]),
):
# 获取上传基本信息
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
validate_expire_style(data.expire_style)
detail = await FileUploadService.complete_chunked_upload(
upload_id, chunk_info, data.expire_value, data.expire_style
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail=detail)
# ============ 预签名上传API ============
presign_api = APIRouter(prefix="/presign", tags=["预签名上传"])
def build_proxy_upload_urls(upload_id: str) -> dict:
proxy_upload_url = f"/presign/upload/proxy/{upload_id}"
return {
"proxy_upload_url": proxy_upload_url,
"legacy_proxy_upload_url": f"/api{proxy_upload_url}",
}
async def _get_valid_session(
upload_id: str, expected_mode: Optional[str] = None
) -> PresignUploadSession:
"""获取并验证会话"""
session = await PresignUploadSession.filter(upload_id=upload_id).first()
if not session:
raise HTTPException(404, "上传会话不存在")
if await session.is_expired():
await session.delete()
await release_storage(f"presign:{upload_id}")
raise HTTPException(404, "上传会话已过期")
if expected_mode and session.mode != expected_mode:
raise HTTPException(400, f"此会话不支持{expected_mode}模式")
return session
@presign_api.post("/upload/init", dependencies=[Depends(share_required_login)])
async def presign_upload_init(
data: PresignUploadInitRequest, ip: str = Depends(ip_limit["upload"])
):
"""初始化预签名上传,S3返回直传URL,其他存储返回代理URL"""
validate_file_type(data.file_name)
if data.file_size > settings.upload_size:
raise HTTPException(
403,
f"文件大小超过限制,最大为 {settings.upload_size / (1024 * 1024):.2f} MB",
)
validate_expire_style(data.expire_style)
upload_id = uuid.uuid4().hex
reservation_token = f"presign:{upload_id}"
await reserve_storage(
reservation_token, data.file_size, ttl_seconds=PRESIGN_SESSION_EXPIRES
)
try:
path, _, _, filename, save_path = await FileUploadService.generate_file_path(
data.file_name, upload_id
)
storage: FileStorageInterface = storages[settings.file_storage]()
presigned_url = await storage.generate_presigned_upload_url(
save_path, PRESIGN_SESSION_EXPIRES
)
mode = "direct" if presigned_url else "proxy"
proxy_urls = build_proxy_upload_urls(upload_id)
upload_url = presigned_url or proxy_urls["proxy_upload_url"]
await PresignUploadSession.create(
upload_id=upload_id,
file_name=filename,
file_size=data.file_size,
save_path=save_path,
mode=mode,
expire_value=data.expire_value,
expire_style=data.expire_style,
expires_at=await get_now() + timedelta(seconds=PRESIGN_SESSION_EXPIRES),
)
except Exception:
await release_storage(reservation_token)
raise
ip_limit["upload"].add_ip(ip)
detail = {
"upload_id": upload_id,
"upload_url": upload_url,
"mode": mode,
"expires_in": PRESIGN_SESSION_EXPIRES,
}
if mode == "proxy":
detail.update(proxy_urls)
return APIResponse(
detail=detail
)
@presign_api.put(
"/upload/proxy/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def presign_upload_proxy(
upload_id: str, file: UploadFile = File(...), ip: str = Depends(ip_limit["upload"])
):
"""代理模式上传,服务器转存到存储后端"""
session = await _get_valid_session(upload_id, expected_mode="proxy")
code = await FileUploadService.commit_proxy_upload(session, file)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@presign_api.post(
"/upload/confirm/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def presign_upload_confirm(upload_id: str, ip: str = Depends(ip_limit["upload"])):
"""直传确认,客户端完成S3直传后调用获取分享码"""
session = await _get_valid_session(upload_id, expected_mode="direct")
code = await FileUploadService.confirm_direct_upload(session)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@presign_api.get(
"/upload/status/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def presign_upload_status(upload_id: str):
"""查询上传会话状态"""
session = await PresignUploadSession.filter(upload_id=upload_id).first()
if not session:
raise HTTPException(404, "上传会话不存在")
return APIResponse(
detail={
"upload_id": session.upload_id,
"file_name": session.file_name,
"file_size": session.file_size,
"mode": session.mode,
"created_at": session.created_at.isoformat(),
"expires_at": session.expires_at.isoformat(),
"is_expired": await session.is_expired(),
}
)
@presign_api.delete("/upload/{upload_id}", dependencies=[Depends(share_required_login)])
async def presign_upload_cancel(upload_id: str):
"""取消上传会话"""
session = await PresignUploadSession.filter(upload_id=upload_id).first()
if not session:
raise HTTPException(404, "上传会话不存在")
if session.mode == "direct":
storage: FileStorageInterface = storages[settings.file_storage]()
try:
if await storage.file_exists(session.save_path):
temp_file_code = StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
await storage.delete_file(temp_file_code)
except Exception:
logger.warning("取消预签名会话:清理临时文件失败 upload_id=%s", upload_id, exc_info=True)
await session.delete()
await release_storage(f"presign:{upload_id}")
return APIResponse(detail={"message": "上传会话已取消"})