Compare commits

..
2 Commits
Author SHA1 Message Date
lan 222c048e38 fix: 基础目录设置 2024-01-18 18:46:16 +08:00
lan 31de5b7ea1 test:1 2023-08-16 13:03:56 +08:00
207 changed files with 1911 additions and 26161 deletions

No files matched your search

-50
View File
@@ -1,50 +0,0 @@
# Git
.git
.gitignore
.gitattributes
# IDE
.idea
.vscode
*.swp
*.swo
# Documentation
*.md
docs/
LICENSE
# Development
.venv
__pycache__
*.pyc
*.pyo
.pytest_cache
.coverage
htmlcov/
# macOS
.DS_Store
._*
# Frontend source (will be built from GitHub)
fcb-fronted/
themes/
# Test files
test_*.py
tests/
# Data (should be mounted as volume)
data/
# GitHub
.github/
# Docker
Dockerfile
docker-compose.yml
.dockerignore
# Claude
.claude/
+4
View File
@@ -0,0 +1,4 @@
# These are supported funding model platforms
github: [ vastsa ]
custom: [ https://share.lanol.cn ]
-38
View File
@@ -1,38 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Smartphone (please complete the following information):**
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
-10
View File
@@ -1,10 +0,0 @@
---
name: Custom issue template
about: Describe this issue template's purpose here.
title: ''
labels: ''
assignees: ''
---
-20
View File
@@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
-18
View File
@@ -1,18 +0,0 @@
version: 2
updates:
# Runtime deps. requirements.txt pins direct deps; requirements.lock.txt is
# the hashed, transitive-resolved lockfile. Dependabot raises version PRs
# against requirements.txt; regenerate the lockfile in the same PR
# (uv pip compile requirements.txt --generate-hashes --universal -o requirements.lock.txt).
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 130 KiB

-134
View File
@@ -1,134 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="1024" height="1024" viewBox="420 360 1208 1208">
<defs>
<linearGradient id="blue" x1="0" y1="0" x2="1" y2="1">
<stop offset="0" stop-color="#75A7FF"/>
<stop offset="0.52" stop-color="#4E86F7"/>
<stop offset="1" stop-color="#3769D8"/>
</linearGradient>
<linearGradient id="blueSoft" x1="0" y1="0" x2="1" y2="1">
<stop offset="0" stop-color="#EDF4FF"/>
<stop offset="1" stop-color="#C9DCFF"/>
</linearGradient>
<linearGradient id="ear" x1="0" y1="0" x2="0" y2="1">
<stop offset="0" stop-color="#F8FBFF"/>
<stop offset="1" stop-color="#DDE9FF"/>
</linearGradient>
<linearGradient id="earInner" x1="0" y1="0" x2="0" y2="1">
<stop offset="0" stop-color="#FFD8A8"/>
<stop offset="1" stop-color="#F3A76E"/>
</linearGradient>
<linearGradient id="tail" x1="0" y1="0" x2="1" y2="1">
<stop offset="0" stop-color="#6EA2FF"/>
<stop offset="1" stop-color="#315FC8"/>
</linearGradient>
<filter id="softShadow" x="-30%" y="-30%" width="160%" height="180%">
<feDropShadow dx="0" dy="24" stdDeviation="26" flood-color="#1D376C" flood-opacity="0.18"/>
</filter>
<filter id="tinyShadow" x="-50%" y="-50%" width="200%" height="200%">
<feDropShadow dx="0" dy="10" stdDeviation="10" flood-color="#243E72" flood-opacity="0.18"/>
</filter>
</defs>
<!-- 透明底:已去除外围白底 / 圆角框 / 光晕 -->
<!-- Floating audio particles -->
<g fill="#6A9BFA" opacity="0.9">
<circle cx="529" cy="777" r="20"/>
<circle cx="1561" cy="728" r="16"/>
<circle cx="1516" cy="1302" r="14"/>
</g>
<g fill="none" stroke="#8EB4FF" stroke-width="20" stroke-linecap="round" opacity="0.72">
<path d="M440 942c-36 36-36 94 0 130"/>
<path d="M1608 940c36 36 36 94 0 130"/>
</g>
<!-- Tail: a listening loop that wraps the box -->
<path d="M1351 1309c176 27 249 166 165 287-65 93-226 95-337 12"
fill="none" stroke="url(#tail)" stroke-width="114" stroke-linecap="round"
filter="url(#softShadow)"/>
<path d="M1418 1460c57 29 71 80 35 116-38 38-100 25-136-8"
fill="none" stroke="#DDE9FF" stroke-width="54" stroke-linecap="round"/>
<!-- Headphones behind the mascot -->
<path d="M702 1048c0-318 145-507 322-507s322 189 322 507"
fill="none" stroke="#18264A" stroke-width="86" stroke-linecap="round"
filter="url(#tinyShadow)"/>
<path d="M706 834c41-189 160-293 318-293 157 0 278 104 318 293"
fill="none" stroke="#4F86F7" stroke-width="34" stroke-linecap="round"/>
<rect x="625" y="972" width="160" height="300" rx="76" fill="#18264A"/>
<rect x="1263" y="972" width="160" height="300" rx="76" fill="#18264A"/>
<rect x="655" y="1010" width="98" height="225" rx="49" fill="url(#blue)"/>
<rect x="1295" y="1010" width="98" height="225" rx="49" fill="url(#blue)"/>
<!-- Open audio box -->
<g filter="url(#softShadow)">
<path d="M594 1191c0-49 40-89 89-89h682c49 0 89 40 89 89v331c0 77-62 139-139 139H733c-77 0-139-62-139-139z"
fill="url(#blue)"/>
<ellipse cx="1024" cy="1122" rx="430" ry="126" fill="#24345E"/>
<ellipse cx="1024" cy="1118" rx="365" ry="89" fill="#111B37"/>
<path d="M594 1216c189 62 671 62 860 0v311c0 74-60 134-134 134H728c-74 0-134-60-134-134z"
fill="url(#blue)"/>
<path d="M633 1287c181 50 601 55 782 4" fill="none" stroke="#9BC0FF" stroke-width="18" opacity="0.54"/>
</g>
<!-- Fox ears -->
<g filter="url(#softShadow)">
<path d="M719 854C664 713 683 520 751 397c26-47 91-42 115 5l170 331z"
fill="url(#ear)" stroke="#20335F" stroke-width="24" stroke-linejoin="round"/>
<path d="M1329 854c55-141 36-334-32-457-26-47-91-42-115 5l-170 331z"
fill="url(#ear)" stroke="#20335F" stroke-width="24" stroke-linejoin="round"/>
<path d="M758 726c-25-93-16-200 18-271 12-24 44-20 56 4l111 235z"
fill="url(#earInner)"/>
<path d="M1290 726c25-93 16-200-18-271-12-24-44-20-56 4l-111 235z"
fill="url(#earInner)"/>
</g>
<!-- Fox head -->
<g filter="url(#softShadow)">
<path d="M741 788c52-125 160-205 283-205s231 80 283 205c38 92 48 232 3 338-45 105-154 170-286 170s-241-65-286-170c-45-106-35-246 3-338z"
fill="#F9FBFF" stroke="#20335F" stroke-width="26"/>
<path d="M824 1068c52-35 119-51 200-51s148 16 200 51c-20 110-91 179-200 179s-180-69-200-179z"
fill="#E8F0FF"/>
<path d="M780 856c54-30 102-37 156-16" fill="none" stroke="#D1E0FF" stroke-width="26" stroke-linecap="round"/>
<path d="M1268 856c-54-30-102-37-156-16" fill="none" stroke="#D1E0FF" stroke-width="26" stroke-linecap="round"/>
</g>
<!-- Face -->
<g fill="none" stroke="#1E2E56" stroke-width="26" stroke-linecap="round">
<path d="M862 946c26 27 60 27 86 0"/>
<path d="M1100 946c26 27 60 27 86 0"/>
</g>
<path d="M972 1027c0-30 24-54 54-54s54 24 54 54c0 22-14 39-30 51-15 11-33 11-48 0-16-12-30-29-30-51z"
fill="#192544"/>
<circle cx="1045" cy="1005" r="10" fill="#FFFFFF" opacity="0.9"/>
<path d="M1026 1079c-2 46-30 73-72 77" fill="none" stroke="#1E2E56" stroke-width="20" stroke-linecap="round"/>
<path d="M1026 1079c2 46 30 73 72 77" fill="none" stroke="#1E2E56" stroke-width="20" stroke-linecap="round"/>
<circle cx="824" cy="1041" r="34" fill="#F4B18B" opacity="0.62"/>
<circle cx="1224" cy="1041" r="34" fill="#F4B18B" opacity="0.62"/>
<!-- Paws resting on the box -->
<g filter="url(#tinyShadow)">
<ellipse cx="790" cy="1190" rx="116" ry="78" fill="#F9FBFF" stroke="#20335F" stroke-width="22"/>
<ellipse cx="1258" cy="1190" rx="116" ry="78" fill="#F9FBFF" stroke="#20335F" stroke-width="22"/>
<path d="M756 1189c18 22 46 31 72 20M1224 1189c18 22 46 31 72 20"
fill="none" stroke="#CADBFA" stroke-width="15" stroke-linecap="round"/>
</g>
<!-- Audio waveform badge -->
<g transform="translate(1024 1442)" filter="url(#tinyShadow)">
<circle cx="0" cy="0" r="116" fill="#FFFFFF" opacity="0.96"/>
<circle cx="0" cy="0" r="98" fill="#EAF2FF"/>
<g stroke="#4D83F2" stroke-width="24" stroke-linecap="round">
<line x1="-50" y1="-18" x2="-50" y2="18"/>
<line x1="-17" y1="-49" x2="-17" y2="49"/>
<line x1="17" y1="-70" x2="17" y2="70"/>
<line x1="50" y1="-35" x2="50" y2="35"/>
</g>
</g>
<!-- Tiny AI spark -->
<g transform="translate(1437 520)" fill="#5E91F6">
<path d="M0-66c12 42 24 54 66 66-42 12-54 24-66 66-12-42-24-54-66-66 42-12 54-24 66-66z"/>
<circle cx="-91" cy="91" r="15" opacity="0.55"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 6.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 27 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

-67
View File
@@ -1,67 +0,0 @@
name: CI
on:
push:
branches:
- master
- dev
pull_request:
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
test:
name: Lint and test
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
# The hashed lockfile is the deploy artifact; install from it with
# --require-hashes so CI fails on lockfile drift, exactly like the
# Docker build does.
pip install --require-hashes -r requirements.lock.txt
pip install pytest pytest-asyncio httpx 'moto[s3,server]' mypy
- name: Ruff
run: pipx run ruff==0.16.6 check .
- name: Mypy ratchet (new type errors are rejected; fix and run
scripts/mypy_ratchet.py --regenerate to tighten the baseline)
run: python scripts/mypy_ratchet.py
- name: Verify lockfile matches requirements.txt
# Dependabot bumps requirements.txt but cannot regenerate the hashed
# lockfile; without this check a stale lockfile would silently keep
# the Docker build on old versions.
run: |
python - <<'PY'
import re, sys
pins = dict(re.findall(r'^([\w-]+)==([\w.]+)(?:\s*;[^#\n]*)?$', open('requirements.txt').read(), re.M))
lock = open('requirements.lock.txt').read()
stale = []
for name, ver in pins.items():
m = re.search(rf'(?mi)^{re.escape(name)}==([\w.]+)\b', lock)
if m is None or m.group(1) != ver:
stale.append((name, ver, m.group(1) if m else 'ABSENT'))
for name, req_ver, lock_ver in stale:
print(f'STALE LOCKFILE: {name} requirements.txt={req_ver} lock={lock_ver}')
sys.exit(1 if stale else 0)
PY
- name: Run tests
run: pytest -q
-125
View File
@@ -1,125 +0,0 @@
name: Build and push Docker image
on:
workflow_dispatch:
inputs:
release_tag:
description: Optional release tag created by Release Please
required: false
type: string
push:
branches:
- master
- dev
tags:
- 'v*'
env:
REGISTRY_IMAGE: ${{ secrets.DOCKER_USERNAME }}/filecodebox
jobs:
buildx:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ inputs.release_tag || github.ref }}
submodules: true
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY_IMAGE }}
flavor: |
latest=false
tags: |
type=ref,event=branch,enable=${{ inputs.release_tag == '' }}
type=ref,event=pr,enable=${{ inputs.release_tag == '' }}
type=semver,pattern={{version}},enable=${{ inputs.release_tag == '' }}
type=semver,pattern={{major}}.{{minor}},enable=${{ inputs.release_tag == '' }}
type=semver,value=${{ inputs.release_tag }},pattern={{version}},enable=${{ inputs.release_tag != '' }}
type=semver,value=${{ inputs.release_tag }},pattern={{major}}.{{minor}},enable=${{ inputs.release_tag != '' }}
type=raw,value=dev,enable=${{ github.ref == 'refs/heads/dev' }}
type=sha,prefix=edge-,enable=${{ github.ref == 'refs/heads/master' && inputs.release_tag == '' }}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') || inputs.release_tag != '' }}
- name: Verify release version
if: ${{ startsWith(github.ref, 'refs/tags/v') || inputs.release_tag != '' }}
shell: bash
env:
RELEASE_TAG: ${{ inputs.release_tag || github.ref_name }}
run: |
source_version="$(tr -d '[:space:]' < VERSION)"
tag_version="${RELEASE_TAG#v}"
test "$source_version" = "$tag_version" || {
echo "Version mismatch: VERSION=$source_version, tag=$tag_version" >&2
exit 1
}
- name: Resolve source revision
id: source-revision
shell: bash
run: |
revision="$(git rev-parse HEAD)"
echo "value=$revision" >> "$GITHUB_OUTPUT"
echo "short=${revision::12}" >> "$GITHUB_OUTPUT"
- name: Resolve latest frontend revisions
id: frontend-revisions
shell: bash
run: |
frontend_2024_ref="$(git ls-remote https://github.com/vastsa/FileCodeBoxFronted.git HEAD | cut -f1)"
frontend_2023_ref="$(git ls-remote https://github.com/vastsa/FileCodeBoxFronted2023.git HEAD | cut -f1)"
test -n "$frontend_2024_ref"
test -n "$frontend_2023_ref"
echo "frontend_2024=$frontend_2024_ref" >> "$GITHUB_OUTPUT"
echo "frontend_2023=$frontend_2023_ref" >> "$GITHUB_OUTPUT"
- name: Resolve application version
id: app-version
shell: bash
env:
RELEASE_TAG: ${{ inputs.release_tag }}
run: |
source_version="$(tr -d '[:space:]' < VERSION)"
if [[ "$GITHUB_REF" == refs/tags/v* || -n "$RELEASE_TAG" ]]; then
echo "value=$source_version" >> "$GITHUB_OUTPUT"
else
echo "value=${source_version}-dev+${{ steps.source-revision.outputs.short }}" >> "$GITHUB_OUTPUT"
fi
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver-opts: |
image=moby/buildkit:latest
network=host
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
APP_VERSION=${{ steps.app-version.outputs.value }}
VCS_REF=${{ steps.source-revision.outputs.value }}
FRONTEND_2024_REF=${{ steps.frontend-revisions.outputs.frontend_2024 }}
FRONTEND_2023_REF=${{ steps.frontend-revisions.outputs.frontend_2023 }}
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: false
-69
View File
@@ -1,69 +0,0 @@
# 构建 VitePress 站点并将其部署到 GitHub Pages 的示例工作流程
#
name: Deploy VitePress site to Pages
on:
# 在针对 `main` 分支的推送上运行。如果你
# 使用 `master` 分支作为默认分支,请将其更改为 `master`
push:
branches: [ master ]
# 允许你从 Actions 选项卡手动运行此工作流程
workflow_dispatch:
# 设置 GITHUB_TOKEN 的权限,以允许部署到 GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# 只允许同时进行一次部署,跳过正在运行和最新队列之间的运行队列
# 但是,不要取消正在进行的运行,因为我们希望允许这些生产部署完成
concurrency:
group: pages
cancel-in-progress: false
jobs:
# 构建工作
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0 # 如果未启用 lastUpdated,则不需要
- uses: pnpm/action-setup@v3
with:
version: 9
# - uses: oven-sh/setup-bun@v1 # 如果使用 Bun,请取消注释
- name: Setup Node
uses: actions/setup-node@v7
with:
node-version: 20
cache: 'pnpm'
cache-dependency-path: 'docs/pnpm-lock.yaml'
- name: Setup Pages
uses: actions/configure-pages@v6
- name: Install dependencies
working-directory: docs
run: pnpm install
- name: Build with VitePress
working-directory: docs
run: pnpm run docs:build
- name: Upload artifact
uses: actions/upload-pages-artifact@v5
with:
path: docs/.vitepress/dist
# 部署工作
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
needs: build
runs-on: ubuntu-latest
name: Deploy
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
-57
View File
@@ -1,57 +0,0 @@
name: Create release
on:
push:
branches:
- master
workflow_dispatch:
permissions:
actions: write
contents: write
issues: write
pull-requests: write
concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
jobs:
release-please:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Create or update release
id: release
uses: googleapis/release-please-action@v5
with:
config-file: release-please-config.json
manifest-file: .release-please-manifest.json
- name: Merge release pull request
if: ${{ steps.release.outputs.prs_created == 'true' }}
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
RELEASE_PR_JSON: ${{ steps.release.outputs.pr }}
run: |
RELEASE_PR="$(jq -r '.number' <<< "$RELEASE_PR_JSON")"
for attempt in {1..120}; do
if gh pr merge "$RELEASE_PR" --squash --delete-branch; then
break
fi
if [[ "$attempt" -eq 120 ]]; then
echo "Timed out waiting for release PR #$RELEASE_PR to merge" >&2
exit 1
fi
sleep 10
done
gh workflow run release.yml --ref master
- name: Publish release image
if: ${{ steps.release.outputs.release_created == 'true' }}
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
RELEASE_TAG: ${{ steps.release.outputs.tag_name }}
run: gh workflow run docker-image.yml --ref master -f release_tag="$RELEASE_TAG"
+8 -14
View File
@@ -16,6 +16,7 @@ database.db
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
@@ -30,7 +31,7 @@ share/python-wheels/
.installed.cfg
*.egg
MANIFEST
.vite/
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
@@ -79,9 +80,7 @@ docs/_build/
# PyBuilder
.pybuilder/
target/
*.db
*.db-shm
*.db-wal
# Jupyter Notebook
.ipynb_checkpoints
@@ -147,15 +146,10 @@ cython_debug/
# Project
.vscode
.DS_Store
for_test.py
.html
/evaluate/temp.py
/evaluation/back.json
data/.env
# Ignore node_modules
node_modules/
docs/.vitepress/cache/
AGENTS.md
dist/
# Frontend themes (built from GitHub during Docker build)
themes/
.backup/
/cloc-1.64.exe
-8
View File
@@ -1,8 +0,0 @@
# First-time setup: pre-commit install
# Behind a proxy: HTTPS_PROXY=http://<proxy-host>:<port> pre-commit run --all-files
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.16.6
hooks:
- id: ruff-check
args: [--fix]
-3
View File
@@ -1,3 +0,0 @@
{
".": "2.8.4"
}
+7 -82
View File
@@ -1,86 +1,11 @@
# 第一阶段:构建前端主题
# 前端产物与 CPU 架构无关,使用原生构建平台避免在 QEMU 中运行 Node/pnpm。
FROM --platform=$BUILDPLATFORM node:20-alpine AS frontend-builder
ARG FRONTEND_2024_REF=main
ARG FRONTEND_2023_REF=main
RUN apk add --no-cache git python3 make g++
RUN corepack enable && \
corepack prepare pnpm@9.15.9 --activate
WORKDIR /build
# 克隆并构建固定版本的 2024 主题
RUN git clone --filter=blob:none --no-checkout https://github.com/vastsa/FileCodeBoxFronted.git /build/fronted-2024 && \
cd /build/fronted-2024 && \
git fetch --depth 1 origin "${FRONTEND_2024_REF}" && \
git checkout --detach FETCH_HEAD && \
pnpm install --frozen-lockfile --prod=false && \
VITE_GIT_COMMIT="$(git rev-parse HEAD)" pnpm run build
# 克隆并构建固定版本的 2023 主题
RUN git clone --filter=blob:none --no-checkout https://github.com/vastsa/FileCodeBoxFronted2023.git /build/fronted-2023 && \
cd /build/fronted-2023 && \
git fetch --depth 1 origin "${FRONTEND_2023_REF}" && \
git checkout --detach FETCH_HEAD && \
npm install --legacy-peer-deps && \
npm run build
# 第二阶段:构建最终镜像
FROM python:3.12-slim-bookworm
ARG APP_VERSION
ARG VCS_REF=unknown
ARG FRONTEND_2024_REF=main
ARG FRONTEND_2023_REF=main
FROM python:3.9.5-slim-buster
LABEL author="Lan"
LABEL email="xzu@live.com"
LABEL org.opencontainers.image.version="${APP_VERSION}"
LABEL org.opencontainers.image.revision="${VCS_REF}"
LABEL org.opencontainers.image.filecodebox.frontend-2024-revision="${FRONTEND_2024_REF}"
LABEL org.opencontainers.image.filecodebox.frontend-2023-revision="${FRONTEND_2023_REF}"
LABEL email="vast@tom.com"
LABEL version="6"
COPY . /app
WORKDIR /app
# 复制项目文件(通过 .dockerignore 排除不必要的文件)
COPY . .
# 分支镜像使用带提交号的开发版本;正式镜像使用 VERSION 中的版本。
ENV APP_VERSION="${APP_VERSION}"
# 设置时区
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
echo 'Asia/Shanghai' > /etc/timezone
# 从构建阶段复制编译好的前端主题
COPY --from=frontend-builder /build/FileCodeBox/themes/2024 ./themes/2024
COPY --from=frontend-builder /build/fronted-2023/dist ./themes/2023
# 安装系统安全更新 + Python 依赖
# 依赖从带哈希的锁定文件安装(--require-hashes),保证构建可复现、防供应链篡改。
# 清理 apt 缓存,降低镜像噪音与扫描面
RUN apt-get update \
&& apt-get upgrade -y --no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir --require-hashes -r requirements.lock.txt \
&& pip cache purge || true
# 环境变量配置
ENV HOST="0.0.0.0" \
PORT=12345 \
WORKERS=1 \
APP_ENV="production" \
LOG_LEVEL="warning" \
ACCESS_LOG="false" \
FORWARDED_ALLOW_IPS="" \
TIMEOUT_KEEP_ALIVE=5 \
MALLOC_TRIM_THRESHOLD_=65536
RUN pip install -r requirements.txt
EXPOSE 12345
# 生产环境启动命令
# FORWARDED_ALLOW_IPS 默认为空:仅信任直连 IP,避免任意客户端伪造 X-Forwarded-*。
# 若前面有反向代理,请显式设置为代理网段,例如 "10.0.0.0/8,172.16.0.0/12"。
# 容器以 root 运行:兼容存量 root 属主的数据卷,以及 NAS/只读 bind mount。
CMD ["sh", "-c", "access_log_arg=--no-access-log; if [ \"${APP_ENV:-development}\" != \"production\" ] || [ \"${ACCESS_LOG:-false}\" = \"true\" ]; then access_log_arg=--access-log; fi; exec uvicorn main:app --host \"$HOST\" --port \"$PORT\" --workers \"$WORKERS\" --log-level \"$LOG_LEVEL\" \"$access_log_arg\" --timeout-keep-alive \"${TIMEOUT_KEEP_ALIVE:-5}\" --proxy-headers --forwarded-allow-ips \"${FORWARDED_ALLOW_IPS:-}\""]
CMD ["python","main.py"]
-21
View File
@@ -1,21 +0,0 @@
# Security Policy
## Supported Versions
Use this section to tell people about which versions of your project are
currently being supported with security updates.
| Version | Supported |
| ------- | ------------------ |
| 5.1.x | :white_check_mark: |
| 5.0.x | :x: |
| 4.0.x | :white_check_mark: |
| < 4.0 | :x: |
## Reporting a Vulnerability
Use this section to tell people how to report a vulnerability.
Tell them where to go, how often they can expect to get an update on a
reported vulnerability, what to expect if the vulnerability is accepted or
declined, etc.
-1
View File
@@ -1 +0,0 @@
2.8.4
-4
View File
@@ -1,4 +0,0 @@
# @Time : 2023/8/13 20:43
# @Author : Lan
# @File : __init__.py.py
# @Software: PyCharm
-4
View File
@@ -1,4 +0,0 @@
# @Time : 2023/8/14 14:38
# @Author : Lan
# @File : __init__.py.py
# @Software: PyCharm
-160
View File
@@ -1,160 +0,0 @@
"""System config write path (ConfigService) + the KeyValue JSON lock.
keyvalue_write_lock lives here because ConfigService.update_config is its
primary consumer; FileService imports it (one-way, no cycle).
"""
import asyncio
from fastapi import HTTPException
from core.settings import (
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
)
from apps.base.config import refresh_settings
from core.security import (
INTERNAL_CONFIG_KEYS,
OUTBOUND_ENDPOINT_CONFIG_KEYS,
generate_jwt_secret,
validate_outbound_endpoint,
validate_outbound_hostname,
)
from apps.base.models import KeyValue
from core.utils import hash_password, is_password_hashed, validate_background_url
# KeyValue 里的 settings/activities/presets 都是整块 JSON 读-改-写;
# 进程内写锁串行化这三个写路径,避免并发管理操作互相覆盖(last-writer-wins)。
# 多进程部署下锁不跨进程——文档已锁定单 worker 部署。
keyvalue_write_lock = asyncio.Lock()
class ConfigService:
INT_FIELDS = {
"admin_session_expire",
"enable_chunk",
"error_count",
"error_minute",
"login_count",
"login_minute",
"max_save_seconds",
"onedrive_proxy",
"open_upload",
"mcp_require_token",
"port",
"s3_proxy",
"server_port",
"server_workers",
"show_admin_addr",
"storage_limit",
"upload_count",
"upload_minute",
"upload_size",
"webdav_proxy",
}
FLOAT_FIELDS = {"opacity"}
def get_config(self):
config = dict(settings.items())
config["admin_token"] = ""
for key in INTERNAL_CONFIG_KEYS:
config.pop(key, None)
return config
async def update_config(self, data: dict):
current_config = dict(settings.items())
next_config = dict(current_config)
update_data = {
key: value
for key, value in data.items()
if key in settings.default_config and key not in INTERNAL_CONFIG_KEYS
}
admin_token = update_data.get("admin_token")
admin_password_changed = False
if admin_token is None or admin_token == "":
update_data.pop("admin_token", None)
elif not is_password_hashed(admin_token):
update_data["admin_token"] = hash_password(admin_token)
admin_password_changed = True
else:
admin_password_changed = True
for key, value in update_data.items():
if value == "" and key in self.INT_FIELDS | self.FLOAT_FIELDS:
continue
try:
if key in self.INT_FIELDS:
next_config[key] = int(value)
elif key in self.FLOAT_FIELDS:
next_config[key] = float(value)
else:
next_config[key] = value
except (TypeError, ValueError):
raise HTTPException(status_code=400, detail=f"{key} 配置值格式错误")
try:
session_expire = int(str(next_config.get("admin_session_expire")))
except (TypeError, ValueError):
raise HTTPException(
status_code=400,
detail="admin_session_expire 配置值格式错误",
)
if (
not ADMIN_SESSION_EXPIRE_MIN <= session_expire <= ADMIN_SESSION_EXPIRE_MAX
or session_expire % ADMIN_SESSION_EXPIRE_MIN != 0
):
raise HTTPException(
status_code=400,
detail="admin_session_expire 必须是 1 到 365 个整天",
)
next_config["admin_session_expire"] = session_expire
if int(next_config.get("storage_limit", 0)) < 0:
raise HTTPException(
status_code=400,
detail="storage_limit 不能小于 0",
)
# 只校验"发生变化"的值:升级前存入的旧格式 background(相对路径、含空格
# 或括号)在旧版本是合法的,若每次保存都重新校验,存量部署会连无关设置项
# 都保存不了(一律 400)。渲染侧仍然 html 转义,而任何修改都必须通过校验。
current_background = str(settings.background or "")
candidate_background = str(next_config.get("background") or "")
if candidate_background != current_background:
try:
validate_background_url(candidate_background)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
# 只校验"发生变化"的值:企业内网 minio/webdav 是正当场景,存量部署历史
# 合法写入的内网 endpoint 若每次保存都重新校验,会连无关设置都保存不了
# (background 曾有同款回归,上游 #528 修复过——本处沿用同一语义)。
# s3_hostname 是裸主机名(存储层按 https://{hostname} 拼接),单独分档校验。
for endpoint_key in OUTBOUND_ENDPOINT_CONFIG_KEYS:
if endpoint_key not in next_config:
continue
candidate = str(next_config[endpoint_key] or "")
current = str(getattr(settings, endpoint_key, "") or "")
if candidate == current:
continue
validator = (
validate_outbound_hostname
if endpoint_key == "s3_hostname"
else validate_outbound_endpoint
)
try:
# 写回规范化值(validator 去除首尾空白):校验通过但入库脏值
# 会让存储层在连接期才报错,应在校验点归一。
next_config[endpoint_key] = validator(candidate)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
if admin_password_changed:
next_config["jwt_secret"] = generate_jwt_secret()
async with keyvalue_write_lock:
await KeyValue.update_or_create(key="settings", defaults={"value": next_config})
await refresh_settings(force=True)
-77
View File
@@ -1,77 +0,0 @@
# @Time : 2023/8/15 17:43
# @Author : Lan
# @File : depends.py
# @Software: PyCharm
from fastapi import Header
from fastapi.requests import Request
from apps.admin.services import ConfigService, FileService, LocalFileService
from apps.base.auth import (
_extract_bearer_token,
_get_jwt_secret,
_require_admin_payload,
create_token,
get_admin_session_expire_seconds,
share_required_login,
verify_token,
)
# 认证原语(JWT 创建/校验、Bearer 提取、分享上传门控)已下沉 apps.base.auth
# ——base 分享面与本模块共同消费,留在 base 避免 base→admin 反向依赖。
# 此处 re-export 保持既有 import 路径兼容(tests/admin.views 等)。
__all__ = [
"_extract_bearer_token",
"_get_jwt_secret",
"_require_admin_payload",
"ADMIN_PUBLIC_ENDPOINTS",
"admin_required",
"create_token",
"get_admin_session",
"get_admin_session_expire_seconds",
"get_config_service",
"get_file_service",
"get_local_file_service",
"share_required_login",
"verify_token",
]
def get_admin_session(authorization: str = Header(default=None)) -> dict:
token = _extract_bearer_token(authorization)
payload = _require_admin_payload(authorization)
return {
"id": "admin",
"username": "admin",
"token": token,
"token_type": "Bearer",
"expires_at": payload.get("exp"),
}
ADMIN_PUBLIC_ENDPOINTS = {("POST", "/admin/login")}
async def admin_required(
authorization: str = Header(default=None),
request: Request = None, # type: ignore[assignment] # FastAPI 运行时注入 Request;可选语义由下方 if request 判断
):
"""
验证管理员权限
"""
if request and (request.method, request.url.path) in ADMIN_PUBLIC_ENDPOINTS:
return None
return _require_admin_payload(authorization)
async def get_file_service():
return FileService()
async def get_config_service():
return ConfigService()
async def get_local_file_service():
return LocalFileService()
-98
View File
@@ -1,98 +0,0 @@
"""Admin-side local (NAS) file browsing and deletion."""
from pathlib import Path
from typing import Any
from fastapi import HTTPException
from apps.base.local_share import (
MAX_LIST_ENTRIES,
format_local_ctime,
get_local_root,
normalize_local_relpath,
resolve_under_local,
)
class LocalFileService:
async def list_files(self, path: str = ""):
relpath = normalize_local_relpath(path, allow_empty=True)
directory = resolve_under_local(relpath)
if not directory.exists() or not directory.is_dir():
raise HTTPException(status_code=404, detail="目录不存在")
root = get_local_root()
items: list[dict[str, Any]] = []
try:
children = list(directory.iterdir())
except OSError as exc:
raise HTTPException(status_code=500, detail="无法读取目录") from exc
children.sort(key=lambda p: (not p.is_dir(), p.name.lower()))
truncated = False
for child in children:
if len(items) >= MAX_LIST_ENTRIES:
truncated = True
break
try:
resolved = child.resolve()
resolved.relative_to(root)
except (OSError, ValueError):
continue
if not resolved.is_file() and not resolved.is_dir():
continue
child_rel = child.name if not relpath else f"{relpath}/{child.name}"
is_dir = resolved.is_dir()
items.append(
{
"file": child.name,
"name": child.name,
"path": child_rel,
"type": "dir" if is_dir else "file",
"ctime": format_local_ctime(resolved),
"size": None if is_dir else resolved.stat().st_size,
}
)
parent = ""
if relpath:
parent_path = Path(relpath).parent.as_posix()
parent = "" if parent_path == "." else parent_path
return {
"path": relpath,
"parent": parent,
"truncated": truncated,
"items": items,
}
async def delete_file(self, filename: str):
file = LocalFileClass(filename)
if await file.exists():
await file.delete()
return "删除成功"
raise HTTPException(status_code=404, detail="文件不存在")
class LocalFileClass:
def __init__(self, file):
relpath = normalize_local_relpath(file)
self.file = relpath
self.name = Path(relpath).name
self.path = resolve_under_local(relpath)
if self.path.is_file():
self.ctime = format_local_ctime(self.path)
self.size = self.path.stat().st_size
else:
self.ctime = None
self.size = None
async def read(self) -> bytes:
with open(self.path, "rb") as fh:
return fh.read()
async def delete(self):
if not self.path.is_file():
raise HTTPException(status_code=404, detail="文件不存在")
self.path.unlink()
async def exists(self):
return self.path.is_file()
-71
View File
@@ -1,71 +0,0 @@
import datetime
from typing import Any, Optional, Union
from pydantic import BaseModel
class IDData(BaseModel):
id: int
class IDsData(BaseModel):
ids: list[int]
class ShareItem(BaseModel):
expire_value: int
expire_style: str = "day"
filename: str
class DeleteItem(BaseModel):
filename: str
class LoginData(BaseModel):
password: str
class UpdateFileData(BaseModel):
id: int
code: Optional[str] = None
prefix: Optional[str] = None
suffix: Optional[str] = None
expired_at: Optional[Union[datetime.datetime, str]] = None
expired_count: Optional[int] = None
class BatchUpdateFileData(BaseModel):
ids: list[int]
expired_at: Optional[Union[datetime.datetime, str]] = None
expired_count: Optional[int] = None
clear_expired_at: Optional[bool] = None
class FilePolicyActionData(BaseModel):
id: int
action: str
download_limit: Optional[int] = None
class BatchFilePolicyActionData(BaseModel):
ids: list[int]
action: str
download_limit: Optional[int] = None
class FileMetadataData(BaseModel):
id: int
note: Optional[str] = None
tags: Optional[list[str]] = None
class FileViewPresetData(BaseModel):
id: Optional[str] = None
name: str
filters: Optional[dict[str, Any]] = None
params: Optional[dict[str, Any]] = None
class FileViewPresetDeleteData(BaseModel):
id: str
File diff suppressed because it is too large. Load diff
-634
View File
@@ -1,634 +0,0 @@
# @Time : 2023/8/14 14:38
# @Author : Lan
# @File : views.py
# @Software: PyCharm
import datetime
from typing import Any
from collections import Counter
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException
from apps.admin.services import FileService, ConfigService, LocalFileService
from apps.admin.dependencies import (
admin_required,
get_admin_session,
get_file_service,
get_config_service,
get_local_file_service,
)
from apps.admin.schemas import (
IDData,
IDsData,
BatchUpdateFileData,
BatchFilePolicyActionData,
FilePolicyActionData,
FileMetadataData,
FileViewPresetData,
FileViewPresetDeleteData,
ShareItem,
DeleteItem,
LoginData,
UpdateFileData,
)
from core.response import APIResponse
from apps.base.models import FileCodes, KeyValue
from apps.base.quota import owned_storage_queryset
from tortoise.expressions import Q
from tortoise.functions import Count, Sum
from apps.admin.dependencies import (
create_token,
get_admin_session_expire_seconds,
verify_token,
)
from core.logger import logger
from core.settings import settings
from core.utils import get_now, hash_password, password_needs_rehash, verify_password
from apps.base.utils import ip_limit
admin_api = APIRouter(
prefix="/admin", tags=["管理"], dependencies=[Depends(admin_required)]
)
def _pick_query_text(*values: Optional[str]) -> Optional[str]:
for value in values:
normalized_value = str(value or "").strip()
if normalized_value:
return normalized_value
return None
@admin_api.post("/login")
async def login(data: LoginData, ip: str = Depends(ip_limit["login"])):
# 登录失败计入 IP 频率限制,超过 login_count/login_minute 后暂时锁定
if not verify_password(data.password, settings.admin_token):
ip_limit["login"].add_ip(ip)
raise HTTPException(status_code=401, detail="密码错误")
# 透明重哈希:存量 sha256/明文口令在登录成功(已证明持有口令)时升级为 scrypt
if password_needs_rehash(settings.admin_token):
try:
record = await KeyValue.filter(key="settings").first()
if record:
stored = dict(record.value or {})
stored["admin_token"] = hash_password(data.password)
record.value = stored
await record.save()
settings.admin_token = stored["admin_token"]
logger.info("管理员口令已升级为 scrypt 哈希")
except Exception:
logger.warning("口令哈希升级失败,保持旧格式", exc_info=True)
expires_in = get_admin_session_expire_seconds()
token = create_token({"is_admin": True}, expires_in=expires_in)
return APIResponse(
detail={
"id": "admin",
"username": "admin",
"token": token,
"token_type": "Bearer",
"expires_at": verify_token(token)["exp"],
"expires_in": expires_in,
}
)
@admin_api.get("/verify")
async def verify_admin(session: dict = Depends(get_admin_session)):
return APIResponse(detail=session)
@admin_api.post("/logout")
async def logout_admin():
return APIResponse(detail={"ok": True})
async def build_dashboard_recent_file(file_code: FileCodes) -> dict:
is_expired = await file_code.is_expired()
return {
"id": file_code.id,
"code": file_code.code,
"name": file_code.prefix + file_code.suffix,
"suffix": file_code.suffix,
"size": file_code.size,
"text": file_code.text is not None,
"expired_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"created_at": file_code.created_at,
"is_expired": is_expired,
}
def _expired_predicate(now: datetime.datetime) -> Q:
"""SQL version of FileCodes.is_expired:
expired_at IS NOT NULL AND ((expired_count < 0 AND expired_at < now) OR expired_count = 0)
"""
return Q(expired_at__isnull=False) & (
Q(expired_count__lt=0, expired_at__lt=now) | Q(expired_count=0)
)
async def _sum_size(queryset) -> int:
rows = await queryset.annotate(total=Sum("size")).values("total")
return rows[0]["total"] or 0
@admin_api.get("/dashboard")
async def dashboard(file_service: FileService = Depends(get_file_service)):
sys_start = await KeyValue.filter(key="sys_start").first()
now = await get_now()
today_start = now.replace(hour=0, minute=0, second=0, microsecond=0)
yesterday_start = today_start - datetime.timedelta(days=1)
yesterday_end = today_start - datetime.timedelta(microseconds=1)
# 简单计数全部走 SQL 聚合,避免整表载入内存(D3)。
# 健康摘要依赖逐行 rules engine(_build_file_status_insights),保留一次遍历。
total_files = await FileCodes.all().count()
expired_count = await FileCodes.filter(_expired_predicate(now)).count()
all_size = await _sum_size(owned_storage_queryset())
used_count = (
await FileCodes.all().annotate(total=Sum("used_count")).values("total")
)[0]["total"] or 0
text_count = await FileCodes.filter(text__isnull=False).count()
chunked_count = await FileCodes.filter(is_chunked=True).count()
yesterday_count = await FileCodes.filter(
created_at__gte=yesterday_start, created_at__lte=yesterday_end
).count()
yesterday_size = await _sum_size(
owned_storage_queryset(
FileCodes.filter(created_at__gte=yesterday_start, created_at__lte=yesterday_end)
)
)
today_count = await FileCodes.filter(created_at__gte=today_start).count()
today_size = await _sum_size(
owned_storage_queryset(FileCodes.filter(created_at__gte=today_start))
)
suffix_rows = (
await FileCodes.filter(text__isnull=True)
.annotate(count=Count("id"))
.values("suffix", "count")
)
suffix_counter = Counter(
{((row["suffix"] or "file") or "file"): row["count"] for row in suffix_rows}
)
recent_file_codes = await FileCodes.all().order_by("-created_at").limit(8)
health_summary = await file_service.build_file_health_summary(
await FileCodes.all(), now=now
)
recent_activities = await file_service.list_admin_activities(limit=8)
return APIResponse(
detail={
"total_files": total_files,
"storage_used": str(all_size),
"sys_uptime": sys_start.value if sys_start else None,
"yesterday_count": yesterday_count,
"yesterday_size": str(yesterday_size),
"today_count": today_count,
"today_size": str(today_size),
"active_count": total_files - expired_count,
"expired_count": expired_count,
"text_count": text_count,
"file_count": total_files - text_count,
"chunked_count": chunked_count,
"used_count": used_count,
"storage_backend": settings.file_storage,
"upload_size_limit": settings.upload_size,
"open_upload": settings.open_upload,
"enable_chunk": settings.enable_chunk,
"max_save_seconds": settings.max_save_seconds,
**health_summary,
"health_summary": health_summary,
"top_suffixes": [
{"suffix": suffix, "count": count}
for suffix, count in suffix_counter.most_common(8)
],
"recent_files": [
await build_dashboard_recent_file(file_code)
for file_code in recent_file_codes
],
"recent_activities": recent_activities["activities"],
}
)
@admin_api.get("/activities")
async def admin_activities(
limit: int = 20,
action: Optional[str] = None,
targetType: Optional[str] = None,
target_type: Optional[str] = None,
keyword: Optional[str] = None,
file_service: FileService = Depends(get_file_service),
):
result = await file_service.list_admin_activities(
limit=limit,
action=action,
target_type=_pick_query_text(targetType, target_type),
keyword=keyword,
)
return APIResponse(detail=result)
@admin_api.delete("/file/delete")
async def file_delete(
data: IDData,
file_service: FileService = Depends(get_file_service),
):
await file_service.delete_file(data.id)
return APIResponse()
async def batch_delete_files(
data: IDsData,
file_service: FileService,
):
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要删除的文件")
result = await file_service.delete_files(data.ids)
return APIResponse(detail=result)
@admin_api.delete("/file/batch-delete")
async def file_batch_delete(
data: IDsData,
file_service: FileService = Depends(get_file_service),
):
return await batch_delete_files(data, file_service)
@admin_api.post("/file/batch-delete")
async def file_batch_delete_post(
data: IDsData,
file_service: FileService = Depends(get_file_service),
):
return await batch_delete_files(data, file_service)
async def batch_update_files(
data: BatchUpdateFileData,
file_service: FileService,
):
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要更新的文件")
update_data: dict[str, Any] = {}
fields_set = data.model_fields_set
should_clear_expired_at = bool(data.clear_expired_at)
if should_clear_expired_at:
update_data["expired_at"] = None
update_data["expired_count"] = -1
elif "expired_at" in fields_set and data.expired_at != "":
# schema 声明 Union[datetime, str]——透传两种形态,tortoise 均可序列化
update_data["expired_at"] = data.expired_at
if (
not should_clear_expired_at
and "expired_count" in fields_set
and data.expired_count is not None
):
update_data["expired_count"] = data.expired_count
if not update_data:
raise HTTPException(status_code=400, detail="请选择要更新的字段")
result = await file_service.update_files(data.ids, update_data)
return APIResponse(detail=result)
@admin_api.patch("/file/batch-update")
async def file_batch_update(
data: BatchUpdateFileData,
file_service: FileService = Depends(get_file_service),
):
return await batch_update_files(data, file_service)
@admin_api.post("/file/batch-update")
async def file_batch_update_post(
data: BatchUpdateFileData,
file_service: FileService = Depends(get_file_service),
):
return await batch_update_files(data, file_service)
async def apply_file_policy_action(
data: FilePolicyActionData,
file_service: FileService,
):
download_limit = data.download_limit
detail = await file_service.apply_file_policy_action(
file_id=data.id,
action=data.action,
download_limit=download_limit,
)
return APIResponse(detail=detail)
@admin_api.patch("/file/policy-action")
async def file_policy_action(
data: FilePolicyActionData,
file_service: FileService = Depends(get_file_service),
):
return await apply_file_policy_action(data, file_service)
@admin_api.post("/file/policy-action")
async def file_policy_action_post(
data: FilePolicyActionData,
file_service: FileService = Depends(get_file_service),
):
return await apply_file_policy_action(data, file_service)
async def apply_batch_file_policy_action(
data: BatchFilePolicyActionData,
file_service: FileService,
):
if not data.ids:
raise HTTPException(status_code=400, detail="请选择要更新的文件")
download_limit = data.download_limit
result = await file_service.apply_files_policy_action(
file_ids=data.ids,
action=data.action,
download_limit=download_limit,
)
return APIResponse(detail=result)
@admin_api.patch("/file/batch-policy-action")
async def file_batch_policy_action(
data: BatchFilePolicyActionData,
file_service: FileService = Depends(get_file_service),
):
return await apply_batch_file_policy_action(data, file_service)
@admin_api.post("/file/batch-policy-action")
async def file_batch_policy_action_post(
data: BatchFilePolicyActionData,
file_service: FileService = Depends(get_file_service),
):
return await apply_batch_file_policy_action(data, file_service)
@admin_api.get("/file/list")
async def file_list(
page: int = 1,
size: int = 10,
keyword: str = "",
status: str = "",
type: str = "",
health: str = "",
sort_by: str = "created_at",
sort_order: str = "desc",
file_service: FileService = Depends(get_file_service),
):
page = max(page, 1)
size = min(max(size, 1), 100)
files, total, summary = await file_service.list_files(
page,
size,
keyword,
status=status,
file_type=type,
health=health,
sort_by=sort_by,
sort_order=sort_order,
)
return APIResponse(
detail={
"page": page,
"size": size,
"data": files,
"total": total,
"summary": summary,
}
)
@admin_api.get("/file/detail")
async def file_detail(
id: int,
file_service: FileService = Depends(get_file_service),
):
detail = await file_service.get_file_detail(id)
return APIResponse(detail=detail)
@admin_api.post("/file/detail")
async def file_detail_post(
data: IDData,
file_service: FileService = Depends(get_file_service),
):
detail = await file_service.get_file_detail(data.id)
return APIResponse(detail=detail)
async def update_file_metadata(
data: FileMetadataData,
file_service: FileService,
):
fields_set = data.model_fields_set
update_note = "note" in fields_set
update_tags = "tags" in fields_set
if not update_note and not update_tags:
raise HTTPException(status_code=400, detail="请选择要更新的元数据")
detail = await file_service.update_file_metadata(
file_id=data.id,
note=data.note,
tags=data.tags,
update_note=update_note,
update_tags=update_tags,
)
return APIResponse(detail=detail)
@admin_api.patch("/file/metadata")
async def file_metadata(
data: FileMetadataData,
file_service: FileService = Depends(get_file_service),
):
return await update_file_metadata(data, file_service)
@admin_api.post("/file/metadata")
async def file_metadata_post(
data: FileMetadataData,
file_service: FileService = Depends(get_file_service),
):
return await update_file_metadata(data, file_service)
@admin_api.get("/file/view-presets")
async def file_view_presets(
file_service: FileService = Depends(get_file_service),
):
result = await file_service.list_file_view_presets()
return APIResponse(detail=result)
async def save_file_view_preset(
data: FileViewPresetData,
file_service: FileService,
):
filters = data.filters if data.filters is not None else data.params
preset = await file_service.save_file_view_preset(
preset_id=data.id,
name=data.name,
filters=filters or {},
)
return APIResponse(detail=preset)
@admin_api.post("/file/view-presets")
async def file_view_presets_save(
data: FileViewPresetData,
file_service: FileService = Depends(get_file_service),
):
return await save_file_view_preset(data, file_service)
@admin_api.patch("/file/view-presets")
async def file_view_presets_patch(
data: FileViewPresetData,
file_service: FileService = Depends(get_file_service),
):
return await save_file_view_preset(data, file_service)
async def delete_file_view_preset(
data: FileViewPresetDeleteData,
file_service: FileService,
):
result = await file_service.delete_file_view_preset(data.id)
return APIResponse(detail=result)
@admin_api.delete("/file/view-presets")
async def file_view_presets_delete(
data: FileViewPresetDeleteData,
file_service: FileService = Depends(get_file_service),
):
return await delete_file_view_preset(data, file_service)
@admin_api.post("/file/view-presets/delete")
async def file_view_presets_delete_post(
data: FileViewPresetDeleteData,
file_service: FileService = Depends(get_file_service),
):
return await delete_file_view_preset(data, file_service)
@admin_api.get("/config/get")
async def get_config(
config_service: ConfigService = Depends(get_config_service),
):
return APIResponse(detail=config_service.get_config())
@admin_api.patch("/config/update")
async def update_config(
data: dict,
config_service: ConfigService = Depends(get_config_service),
file_service: FileService = Depends(get_file_service),
):
data.pop("themes_choices", None)
await config_service.update_config(data)
await file_service.record_admin_activity(
action="config.update",
target_type="config",
target_name="system",
count=1,
meta={"fields": sorted(data.keys())},
)
return APIResponse()
@admin_api.get("/file/download")
async def file_download(
id: int,
file_service: FileService = Depends(get_file_service),
):
file_content = await file_service.download_file(id)
return file_content
@admin_api.get("/file/preview")
async def file_preview(
id: int,
max_chars: int = 4000,
file_service: FileService = Depends(get_file_service),
):
preview = await file_service.preview_file(id, max_chars)
return APIResponse(detail=preview)
@admin_api.get("/local/lists")
async def get_local_lists(
path: str = "",
local_file_service: LocalFileService = Depends(get_local_file_service),
):
files = await local_file_service.list_files(path)
return APIResponse(detail=files)
@admin_api.delete("/local/delete")
async def delete_local_file(
item: DeleteItem,
local_file_service: LocalFileService = Depends(get_local_file_service),
file_service: FileService = Depends(get_file_service),
):
result = await local_file_service.delete_file(item.filename)
await file_service.record_admin_activity(
action="local_file.delete",
target_type="local_file",
target_name=item.filename,
count=1,
meta={"success": bool(result)},
)
return APIResponse(detail=result)
@admin_api.post("/local/share")
async def share_local_file(
item: ShareItem,
file_service: FileService = Depends(get_file_service),
):
share_info = await file_service.share_local_file(item)
await file_service.record_admin_activity(
action="local_file.share",
target_type="file",
target_id=share_info.get("id") if isinstance(share_info, dict) else None,
target_name=item.filename,
count=1,
meta={
"expire_value": item.expire_value,
"expire_style": item.expire_style,
},
)
return APIResponse(detail=share_info)
@admin_api.patch("/file/update")
async def update_file(
data: UpdateFileData,
file_service: FileService = Depends(get_file_service),
):
await file_service.update_file(
file_id=data.id,
code=data.code,
prefix=data.prefix,
suffix=data.suffix,
expired_at=data.expired_at,
expired_count=data.expired_count,
)
return APIResponse(detail="更新成功")
-4
View File
@@ -1,4 +0,0 @@
# @Time : 2023/8/13 20:43
# @Author : Lan
# @File : __init__.py.py
# @Software: PyCharm
-150
View File
@@ -1,150 +0,0 @@
"""Cross-cutting auth primitives and the share-upload gate.
Lives in apps.base because BOTH the share surface (base.views) and the
admin surface (admin.views) consume it; keeping it here avoids the
base → admin reverse dependency. Admin-only session gating stays in
apps.admin.dependencies, which re-exports these primitives for
compatibility.
"""
import base64
import hmac
import json
import time
from fastapi import Header, HTTPException
from core.settings import (
ADMIN_SESSION_EXPIRE_DEFAULT,
ADMIN_SESSION_EXPIRE_MAX,
ADMIN_SESSION_EXPIRE_MIN,
settings,
)
def _get_jwt_secret() -> bytes:
secret = getattr(settings, "jwt_secret", "")
if not secret:
raise RuntimeError("JWT签名密钥未初始化")
return secret.encode()
def get_admin_session_expire_seconds() -> int:
try:
expires_in = int(
getattr(settings, "admin_session_expire", ADMIN_SESSION_EXPIRE_DEFAULT)
)
except (TypeError, ValueError):
return ADMIN_SESSION_EXPIRE_DEFAULT
if (
not ADMIN_SESSION_EXPIRE_MIN <= expires_in <= ADMIN_SESSION_EXPIRE_MAX
or expires_in % ADMIN_SESSION_EXPIRE_MIN != 0
):
return ADMIN_SESSION_EXPIRE_DEFAULT
return expires_in
def create_token(data: dict, expires_in: int | None = None) -> str:
"""
创建JWT token
:param data: 数据负载
:param expires_in: 过期时间(秒)
"""
token_lifetime = (
get_admin_session_expire_seconds() if expires_in is None else expires_in
)
header = base64.urlsafe_b64encode(
json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(",", ":")).encode()
).decode().rstrip("=")
payload = base64.urlsafe_b64encode(
json.dumps(
{**data, "exp": int(time.time()) + token_lifetime},
separators=(",", ":"),
).encode()
).decode().rstrip("=")
signature_digest = hmac.new(
_get_jwt_secret(), f"{header}.{payload}".encode(), "sha256"
).digest()
signature = base64.urlsafe_b64encode(signature_digest).decode().rstrip("=")
return f"{header}.{payload}.{signature}"
def verify_token(token: str) -> dict:
"""
验证JWT token
:param token: JWT token
:return: 解码后的数据
"""
try:
header_b64, payload_b64, signature_b64 = token.split(".")
# 验证签名
expected_signature = hmac.new(
_get_jwt_secret(),
f"{header_b64}.{payload_b64}".encode(),
"sha256",
).digest()
expected_signature_b64 = (
base64.urlsafe_b64encode(expected_signature).decode().rstrip("=")
)
if not hmac.compare_digest(signature_b64, expected_signature_b64):
raise ValueError("无效的签名")
# 解码payload(兼容历史标准 base64 与 urlsafe base64)
padded = payload_b64 + "=" * (-len(payload_b64) % 4)
try:
payload_bytes = base64.urlsafe_b64decode(padded)
except Exception:
payload_bytes = base64.b64decode(padded)
payload = json.loads(payload_bytes)
# 检查是否过期
if payload.get("exp", 0) < time.time():
raise ValueError("token已过期")
return payload
except Exception as e:
raise ValueError(f"token验证失败: {str(e)}")
def _extract_bearer_token(authorization: str) -> str:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
token = authorization.split(" ", 1)[1].strip()
if not token:
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return token
def _require_admin_payload(authorization: str) -> dict:
token = _extract_bearer_token(authorization)
try:
payload = verify_token(token)
except ValueError as e:
raise HTTPException(status_code=401, detail=str(e))
if not payload.get("is_admin", False):
raise HTTPException(status_code=401, detail="未授权或授权校验失败")
return payload
async def share_required_login(authorization: str = Header(default=None)):
"""
验证分享上传权限
当 settings.open_upload 为False时,要求用户必须登录并具有管理员权限
当 settings.open_upload 为True时,允许游客上传
:param authorization: 认证头信息
:param request: 请求对象
:return: 验证结果
"""
if not settings.open_upload:
if not authorization or not authorization.startswith("Bearer "):
raise HTTPException(
status_code=403, detail="本站未开启游客上传,如需上传请先登录后台"
)
_require_admin_payload(authorization)
return True
-140
View File
@@ -1,140 +0,0 @@
from apps.base.models import KeyValue
from apps.base.utils import ip_limit
from core.database import db_startup_lock
from core.logger import logger
from core.security import (
generate_jwt_secret,
is_config_initialized,
is_valid_jwt_secret,
prepare_security_config,
)
from core.settings import DEFAULT_CONFIG, settings
from core.utils import hash_password
MIN_ADMIN_PASSWORD_LENGTH = 8
SETUP_CONFIG_FIELDS = {
"allowed_file_types",
"code_generate_type",
"enable_chunk",
"error_count",
"error_minute",
"login_count",
"login_minute",
"expire_style",
"max_save_seconds",
"name",
"open_upload",
"upload_count",
"upload_minute",
"upload_size",
}
async def ensure_settings_row() -> None:
initial_security = prepare_security_config(DEFAULT_CONFIG)
_, created = await KeyValue.get_or_create(
key="settings", defaults={"value": initial_security.config}
)
if created:
logger.warning(
"系统尚未初始化,请在浏览器中打开站点并完成管理员密码设置"
)
async def ensure_security_settings() -> None:
config_record = await KeyValue.filter(key="settings").first()
if not config_record:
return
current_config = {**DEFAULT_CONFIG, **(config_record.value or {})}
security_config = prepare_security_config(current_config)
if not security_config.changed:
return
config_record.value = security_config.config
await config_record.save()
if security_config.setup_required:
logger.warning("检测到空密码或旧版默认管理员密码,请通过初始化页面重新设置")
elif security_config.password_hashed:
logger.info("已将管理员密码迁移为哈希存储")
if security_config.jwt_secret_rotated:
logger.info("已生成独立 JWT 签名密钥")
await refresh_settings(force=True)
def _sync_ip_limits() -> None:
ip_limit["error"].minutes = settings.error_minute
ip_limit["error"].count = settings.error_count
ip_limit["metadata"].minutes = settings.error_minute
ip_limit["metadata"].count = settings.error_count
ip_limit["upload"].minutes = settings.upload_minute
ip_limit["upload"].count = settings.upload_count
ip_limit["login"].minutes = settings.login_minute
ip_limit["login"].count = settings.login_count
# Per-process config cache: the middleware refreshes settings on every request,
# which otherwise means one DB read per request. Writes call refresh_settings(force=True)
# so same-process visibility is immediate; other processes converge within the TTL
# (documented deployment is a single worker).
CONFIG_CACHE_TTL_SECONDS = 2.0
_config_cached_until = 0.0
async def refresh_settings(force: bool = False) -> None:
"""从数据库读取最新配置并应用到运行时。
默认带短 TTL 缓存(避免每请求查库);写配置的代码路径必须传 force=True。
"""
global _config_cached_until
import time as _time
if not force and _time.monotonic() < _config_cached_until:
return
config_record = await KeyValue.filter(key="settings").first()
settings.user_config = config_record.value if config_record and config_record.value else {}
unknown = settings.unknown_keys(settings.user_config)
if unknown:
logger.warning("配置中存在未登记的键(将被忽略其类型语义,仅透传): %s", unknown)
_sync_ip_limits()
_config_cached_until = _time.monotonic() + CONFIG_CACHE_TTL_SECONDS
def is_runtime_initialized() -> bool:
return is_config_initialized(dict(settings.items()))
async def initialize_system(
admin_password: str,
site_name: str | None = None,
setup_options: dict | None = None,
) -> None:
password = str(admin_password or "").strip()
if len(password) < MIN_ADMIN_PASSWORD_LENGTH:
raise ValueError(f"管理员密码至少需要 {MIN_ADMIN_PASSWORD_LENGTH} 位")
async with db_startup_lock():
config_record = await KeyValue.filter(key="settings").first()
current_config = {
**DEFAULT_CONFIG,
**(config_record.value if config_record and config_record.value else {}),
}
if is_config_initialized(current_config):
raise ValueError("系统已经初始化,请直接登录后台")
next_config = dict(current_config)
for key, value in (setup_options or {}).items():
if key in SETUP_CONFIG_FIELDS:
next_config[key] = value
normalized_site_name = str(site_name or "").strip()
if normalized_site_name:
next_config["name"] = normalized_site_name[:80]
next_config["admin_token"] = hash_password(password)
if not is_valid_jwt_secret(next_config.get("jwt_secret")):
next_config["jwt_secret"] = generate_jwt_secret()
await KeyValue.update_or_create(key="settings", defaults={"value": next_config})
await refresh_settings(force=True)
-136
View File
@@ -1,136 +0,0 @@
import time
from collections import OrderedDict
from ipaddress import ip_address, ip_network
from typing import Iterable, TypedDict
from fastapi import HTTPException, Request
from core.settings import settings
def _iter_trusted_proxies() -> Iterable[str]:
trusted_proxies = getattr(settings, "trusted_proxies", [])
if isinstance(trusted_proxies, str):
trusted_proxies = [item.strip() for item in trusted_proxies.split(",")]
return [item for item in trusted_proxies if item]
def _is_trusted_proxy(host: str) -> bool:
try:
remote_addr = ip_address(host)
except ValueError:
return False
for proxy in _iter_trusted_proxies():
try:
if remote_addr in ip_network(proxy, strict=False):
return True
except ValueError:
continue
return False
def _get_forwarded_for_ip(header_value: str, fallback_ip: str) -> str:
forwarded_chain = [
item.strip() for item in header_value.split(",") if item.strip()
]
if not forwarded_chain:
return fallback_ip
for candidate in reversed(forwarded_chain):
try:
ip_address(candidate)
except ValueError:
return fallback_ip
if not _is_trusted_proxy(candidate):
return candidate
return forwarded_chain[0]
def get_client_ip(request: Request) -> str:
client_host = request.client.host if request.client else "unknown"
if not _is_trusted_proxy(client_host):
return client_host
forwarded_for = request.headers.get("X-Forwarded-For")
if forwarded_for:
return _get_forwarded_for_ip(forwarded_for, client_host)
real_ip = request.headers.get("X-Real-IP")
if real_ip:
try:
ip_address(real_ip)
except ValueError:
return client_host
return real_ip
return client_host
class _IPRecord(TypedDict):
count: int
last_seen: float
# Upper bound for tracked source IPs per limiter. Without it a scan using
# spoofed/rotating addresses grows the dict forever on a small VPS.
DEFAULT_MAX_TRACKED_IPS = 10_000
class IPRateLimit:
def __init__(
self,
count: int,
minutes: int,
max_ips: int = DEFAULT_MAX_TRACKED_IPS,
):
# Ordered by last activity (oldest first) so capacity eviction and the
# periodic cleanup are both O(1)/O(expired) instead of a full rebuild.
self.ips: OrderedDict[str, _IPRecord] = OrderedDict()
self.count = count
self.minutes = minutes
self.max_ips = max(1, max_ips)
def _window_seconds(self) -> float:
# minutes is mutable at runtime via apps.base.config._sync_ip_limits.
return float(self.minutes) * 60
def check_ip(self, ip: str) -> bool:
record = self.ips.get(ip)
if record is None:
return True
if record["count"] >= self.count:
if record["last_seen"] + self._window_seconds() > time.monotonic():
return False
self.ips.pop(ip, None)
return True
def add_ip(self, ip: str) -> int:
now = time.monotonic()
record = self.ips.get(ip)
if record is None:
record = {"count": 0, "last_seen": now}
self.ips[ip] = record
else:
self.ips.move_to_end(ip)
record["count"] += 1
record["last_seen"] = now
while len(self.ips) > self.max_ips:
# Evict the least recently active IP; fail-open for the evicted
# address is preferable to unbounded memory growth.
self.ips.popitem(last=False)
return record["count"]
async def remove_expired_ip(self) -> None:
now = time.monotonic()
window = self._window_seconds()
while self.ips:
_, record = next(iter(self.ips.items()))
if record["last_seen"] + window >= now:
break
self.ips.popitem(last=False)
def __call__(self, request: Request) -> str:
ip = get_client_ip(request)
if not self.check_ip(ip):
raise HTTPException(status_code=423, detail="请求次数过多,请稍后再试")
return ip
-196
View File
@@ -1,196 +0,0 @@
"""文件类型校验:扩展名 / MIME 白名单 + magic bytes 防伪造。"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
from fastapi import HTTPException, UploadFile
from fnmatch import fnmatchcase
from core.settings import settings
@dataclass(frozen=True)
class FileKind:
name: str
extensions: tuple[str, ...]
mimes: tuple[str, ...]
signatures: tuple[bytes, ...]
FILE_KINDS: tuple[FileKind, ...] = (
FileKind("png", (".png",), ("image/png",), (b"\x89PNG\r\n\x1a\n",)),
FileKind("jpg", (".jpg", ".jpeg"), ("image/jpeg",), (b"\xff\xd8\xff",)),
FileKind("gif", (".gif",), ("image/gif",), (b"GIF87a", b"GIF89a")),
FileKind("webp", (".webp",), ("image/webp",), ()),
FileKind("bmp", (".bmp",), ("image/bmp", "image/x-ms-bmp"), (b"BM",)),
FileKind("pdf", (".pdf",), ("application/pdf",), (b"%PDF",)),
FileKind(
"zip",
(".zip", ".docx", ".xlsx", ".pptx", ".apk", ".jar"),
(
"application/zip",
"application/x-zip-compressed",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/java-archive",
"application/vnd.android.package-archive",
),
(b"PK\x03\x04", b"PK\x05\x06", b"PK\x07\x08"),
),
FileKind(
"rar",
(".rar",),
("application/x-rar-compressed", "application/vnd.rar"),
(b"Rar!\x1a\x07\x00", b"Rar!\x1a\x07\x01\x00"),
),
FileKind("7z", (".7z",), ("application/x-7z-compressed",), (b"7z\xbc\xaf'\x1c",)),
FileKind("gz", (".gz", ".tgz"), ("application/gzip", "application/x-gzip"), (b"\x1f\x8b",)),
FileKind(
"mp3",
(".mp3",),
("audio/mpeg",),
(b"ID3", b"\xff\xfb", b"\xff\xf3", b"\xff\xf2"),
),
FileKind("mp4", (".mp4", ".m4a", ".mov"), ("video/mp4", "audio/mp4", "video/quicktime"), ()),
FileKind(
"exe",
(".exe", ".dll", ".sys"),
("application/x-msdownload", "application/x-dosexec"),
(b"MZ",),
),
FileKind("elf", (".elf", ".so", ".o"), ("application/x-executable",), (b"\x7fELF",)),
)
KNOWN_EXTENSIONS = {ext for kind in FILE_KINDS for ext in kind.extensions}
def normalize_allowed_file_types() -> list[str]:
raw_value = settings.allowed_file_types
if isinstance(raw_value, str):
values = [item.strip() for item in raw_value.split(",")]
elif isinstance(raw_value, (list, tuple, set)):
values = [str(item).strip() for item in raw_value]
else:
values = []
normalized = [item.lower() for item in values if item]
return normalized or ["*"]
def _extension_of(file_name: str) -> str:
return Path(file_name or "").suffix.lower()
def _match_allow_rule(rule: str, file_name: str, content_type: str) -> bool:
if rule in {"*", "*/*"}:
return True
if "/" in rule:
return fnmatchcase(content_type, rule)
extension = rule if rule.startswith(".") else f".{rule}"
return file_name.endswith(extension)
def is_type_allowed(file_name: str, content_type: Optional[str] = None) -> bool:
allowed = normalize_allowed_file_types()
if any(rule in {"*", "*/*"} for rule in allowed):
return True
normalized_name = (file_name or "").strip().lower()
normalized_content_type = (content_type or "").strip().lower()
return any(
_match_allow_rule(rule, normalized_name, normalized_content_type)
for rule in allowed
)
def detect_file_kind(header: bytes) -> Optional[FileKind]:
if not header:
return None
if len(header) >= 12 and header.startswith(b"RIFF") and header[8:12] == b"WEBP":
return next(kind for kind in FILE_KINDS if kind.name == "webp")
if len(header) >= 12 and header[4:8] == b"ftyp":
return next(kind for kind in FILE_KINDS if kind.name == "mp4")
candidates: list[tuple[int, FileKind]] = []
for kind in FILE_KINDS:
for signature in kind.signatures:
if signature and header.startswith(signature):
candidates.append((len(signature), kind))
if not candidates:
return None
candidates.sort(key=lambda item: item[0], reverse=True)
return candidates[0][1]
def validate_file_type(file_name: str, content_type: Optional[str] = None) -> None:
if not is_type_allowed(file_name, content_type):
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
def _kind_matches_ext(kind: FileKind, extension: str) -> bool:
return extension in kind.extensions
def _kind_matches_mime(kind: FileKind, mime: str) -> bool:
return mime in kind.mimes
def validate_file_magic(
file_name: str,
content_type: Optional[str],
header: Optional[bytes],
) -> None:
"""白名单通过后,用 magic bytes 防止扩展名 / MIME 伪造。"""
validate_file_type(file_name, content_type)
if not header:
return
claimed_ext = _extension_of(file_name)
claimed_mime = (content_type or "").strip().lower()
detected = detect_file_kind(header)
if claimed_ext in KNOWN_EXTENSIONS:
if detected is None or not _kind_matches_ext(detected, claimed_ext):
raise HTTPException(
status_code=403,
detail="文件内容与扩展名不匹配,疑似伪造类型",
)
if claimed_mime and any(claimed_mime in kind.mimes for kind in FILE_KINDS):
if detected is None or not _kind_matches_mime(detected, claimed_mime):
raise HTTPException(
status_code=403,
detail="文件内容与 Content-Type 不匹配,疑似伪造类型",
)
allowed = normalize_allowed_file_types()
if detected and not any(rule in {"*", "*/*"} for rule in allowed):
allowed_by_detected = any(
is_type_allowed(f"file{ext}", mime)
for ext in detected.extensions
for mime in (detected.mimes or (None,))
)
if not allowed_by_detected:
raise HTTPException(status_code=403, detail="不允许上传该类型文件")
async def read_upload_header(file: UploadFile, size: int = 64) -> bytes:
await file.seek(0)
header = await file.read(size)
await file.seek(0)
return header or b""
async def validate_upload_file(file: UploadFile) -> None:
header = await read_upload_header(file)
validate_file_magic(file.filename or "", file.content_type, header)
def validate_header_bytes(
file_name: str, content_type: Optional[str], header: bytes
) -> None:
validate_file_magic(file_name, content_type, header)
-106
View File
@@ -1,106 +0,0 @@
"""Local NAS-style share helpers.
Files under ``data/local`` can be browsed by admin and turned into extract
codes without copying them into ``data/share``. The original file is never
deleted when a share expires or is removed from the file list.
``FileCodes.file_path`` is the sentinel ``local-ref``; ``uuid_file_name``
stores the posix relative path under ``data/local``. Accidental storage
deletes therefore look at ``data/local-ref/...`` and cannot touch the
original NAS file.
"""
from __future__ import annotations
import time
from pathlib import Path
from urllib.parse import quote
from fastapi import HTTPException
from core.errors import StorageError
from core.storage import StoredDownload
LOCAL_REF_MARKER = "local-ref"
MAX_RELPATH_LEN = 255
MAX_LIST_ENTRIES = 500
def is_local_ref(file_code) -> bool:
return (getattr(file_code, "file_path", None) or "") == LOCAL_REF_MARKER
def should_skip_storage_delete(file_code) -> bool:
return is_local_ref(file_code) or getattr(file_code, "text", None) is not None
def get_local_root() -> Path:
from core.settings import data_root
root = (Path(data_root) / "local").resolve()
root.mkdir(parents=True, exist_ok=True)
return root
def normalize_local_relpath(raw: str, *, allow_empty: bool = False) -> str:
text = str(raw or "").replace("\\", "/").strip()
if not text:
if allow_empty:
return ""
raise HTTPException(status_code=400, detail="非法文件名")
if (
text in {".", ".."}
or text.startswith("/")
or text.startswith("~")
or "\x00" in text
or ":" in text.split("/", 1)[0]
):
raise HTTPException(status_code=400, detail="非法文件名")
parts = [part for part in text.split("/") if part not in {"", "."}]
if not parts or any(part == ".." for part in parts):
raise HTTPException(status_code=400, detail="非法文件名")
relpath = "/".join(parts)
if len(relpath) > MAX_RELPATH_LEN:
raise HTTPException(status_code=400, detail="路径过长")
return relpath
def resolve_under_local(relpath: str) -> Path:
root = get_local_root()
if not relpath:
return root
candidate = (root / relpath).resolve()
try:
candidate.relative_to(root)
except ValueError as exc:
raise HTTPException(status_code=400, detail="非法文件路径") from exc
return candidate
def format_local_ctime(path: Path) -> str | None:
try:
return time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(path.stat().st_ctime))
except OSError:
return None
def build_local_ref_download(file_code) -> StoredDownload:
try:
relpath = normalize_local_relpath(getattr(file_code, "uuid_file_name", None) or "")
path = resolve_under_local(relpath)
except HTTPException as exc:
raise StorageError(status_code=404, detail="文件已过期删除") from exc
if not path.is_file():
raise StorageError(status_code=404, detail="文件已过期删除")
filename = f"{getattr(file_code, 'prefix', '') or ''}{getattr(file_code, 'suffix', '') or ''}"
if not filename:
filename = path.name
encoded_filename = quote(filename, safe="")
headers = {"Content-Disposition": f"attachment; filename*=UTF-8''{encoded_filename}"}
try:
headers["Content-Length"] = str(path.stat().st_size)
except OSError:
pass
return StoredDownload(filename=filename, headers=headers, path=path)
-231
View File
@@ -1,231 +0,0 @@
"""Stateless MCP Streamable HTTP endpoint for quick share uploads."""
import base64
import binascii
import html
import io
import json
from typing import Any
from urllib.parse import quote, urlsplit
from fastapi import APIRouter, HTTPException, Request, UploadFile
from starlette.datastructures import Headers
from starlette.responses import JSONResponse, Response
from apps.base.auth import _require_admin_payload
from apps.base.file_validation import validate_file_type, validate_upload_file
from apps.base.services import FileUploadService, validate_file_size
from apps.base.utils import ip_limit, validate_expire_style
from core.settings import settings
from core.utils import sanitize_filename
router = APIRouter(tags=["MCP"])
MCP_PROTOCOL_VERSION = "2025-03-26"
TEXT_SHARE_MAX_BYTES = 222 * 1024
MAX_MCP_BODY_BYTES = 64 * 1024 * 1024
def _jsonrpc_error(request_id, code: int, message: str) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "error": {"code": code, "message": message}}
def _jsonrpc_result(request_id, result: dict) -> dict:
return {"jsonrpc": "2.0", "id": request_id, "result": result}
def _validate_origin(request: Request) -> None:
"""Reject browser cross-site POSTs before parsing any upload tool."""
origin = request.headers.get("origin")
fetch_site = request.headers.get("sec-fetch-site", "").lower()
if fetch_site == "cross-site":
raise HTTPException(status_code=403, detail="跨站请求已拒绝")
if not origin:
return # Native MCP clients do not send Origin and use no ambient cookies.
try:
parsed = urlsplit(origin)
host = request.headers.get("host", "").lower()
if (
parsed.scheme not in {"http", "https"}
or not parsed.netloc
or parsed.netloc.lower() != host
or parsed.username is not None
or parsed.password is not None
or parsed.path not in {"", "/"}
or parsed.query
or parsed.fragment
):
raise ValueError("origin mismatch")
except (ValueError, TypeError):
raise HTTPException(status_code=403, detail="跨站请求已拒绝")
async def _read_limited_body(request: Request) -> bytes:
max_size = min(
MAX_MCP_BODY_BYTES,
max(1024 * 1024, int(settings.upload_size * 4 / 3) + 256 * 1024),
)
declared_size = request.headers.get("content-length")
if declared_size:
try:
if int(declared_size) > max_size:
raise HTTPException(status_code=413, detail="MCP 请求体过大")
except ValueError:
raise HTTPException(status_code=400, detail="Content-Length 无效")
body = bytearray()
async for chunk in request.stream():
body.extend(chunk)
if len(body) > max_size:
raise HTTPException(status_code=413, detail="MCP 请求体过大")
return bytes(body)
def _tool_definitions() -> list[dict]:
return [
{
"name": "share_text",
"description": "将文本快速上传到 FileCodeBox 并创建分享链接。",
"inputSchema": {
"type": "object",
"properties": {
"text": {"type": "string", "description": "要分享的文本"},
"expire_value": {"type": "integer", "minimum": 1, "default": 1},
"expire_style": {"type": "string", "enum": ["day", "hour", "minute", "forever", "count"], "default": "day"},
},
"required": ["text"],
},
},
{
"name": "share_file",
"description": "将 Base64 编码的文件上传到 FileCodeBox 并创建分享链接。",
"inputSchema": {
"type": "object",
"properties": {
"filename": {"type": "string", "description": "文件名"},
"content_base64": {"type": "string", "description": "文件内容的标准 Base64 编码"},
"content_type": {"type": "string", "default": "application/octet-stream"},
"expire_value": {"type": "integer", "minimum": 1, "default": 1},
"expire_style": {"type": "string", "enum": ["day", "hour", "minute", "forever", "count"], "default": "day"},
},
"required": ["filename", "content_base64"],
},
},
]
def _share_result(request: Request, code: str, name: str) -> dict:
base_url = str(request.base_url).rstrip("/")
share_url = f"{base_url}/#/?code={quote(code, safe='')}"
markdown_name = html.escape(name).replace("[", "\\[").replace("]", "\\]")
return {
"code": code,
"name": name,
"url": share_url,
"markdown": f"[{markdown_name}]({share_url})",
}
async def _call_tool(request: Request, tool_name: str, arguments: dict) -> dict:
authorization = request.headers.get("authorization", "")
if settings.mcp_require_token or not settings.open_upload:
_require_admin_payload(authorization)
ip = ip_limit["upload"](request)
if tool_name == "share_text":
text = arguments.get("text")
if not isinstance(text, str):
raise ValueError("text 必须是字符串")
if len(text.encode("utf-8")) > TEXT_SHARE_MAX_BYTES:
raise ValueError("内容过多,请改用文件分享")
expire_value = int(arguments.get("expire_value", 1))
expire_style = str(arguments.get("expire_style", "day"))
if expire_value < 1:
raise ValueError("expire_value 必须大于 0")
validate_expire_style(expire_style)
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return _share_result(request, code, "Text")
if tool_name == "share_file":
filename = arguments.get("filename")
encoded_content = arguments.get("content_base64")
content_type = str(arguments.get("content_type", "application/octet-stream"))
if not isinstance(filename, str) or not filename.strip():
raise ValueError("filename 不能为空")
if not isinstance(encoded_content, str):
raise ValueError("content_base64 必须是字符串")
try:
content = base64.b64decode(encoded_content, validate=True)
except (ValueError, binascii.Error):
raise ValueError("content_base64 不是有效的标准 Base64")
safe_name = await sanitize_filename(filename.rsplit("/", 1)[-1].rsplit("\\", 1)[-1])
validate_file_type(safe_name, content_type)
upload = UploadFile(filename=safe_name, file=io.BytesIO(content), size=len(content), headers=Headers({"content-type": content_type}))
size = await validate_file_size(upload, settings.upload_size)
await validate_upload_file(upload)
expire_value = int(arguments.get("expire_value", 1))
expire_style = str(arguments.get("expire_style", "day"))
if expire_value < 1:
raise ValueError("expire_value 必须大于 0")
validate_expire_style(expire_style)
share = await FileUploadService.create_file_share(upload, size=size, expire_value=expire_value, expire_style=expire_style)
ip_limit["upload"].add_ip(ip)
return _share_result(request, share["code"], share["name"])
raise LookupError(f"未知工具:{tool_name}")
@router.post("/mcp", include_in_schema=False)
async def mcp_endpoint(request: Request):
_validate_origin(request)
media_type = request.headers.get("content-type", "").split(";", 1)[0].strip().lower()
if media_type != "application/json":
raise HTTPException(status_code=415, detail="MCP 请求必须使用 application/json")
try:
message = json.loads(await _read_limited_body(request))
except (json.JSONDecodeError, UnicodeDecodeError):
return JSONResponse(_jsonrpc_error(None, -32700, "Parse error"), status_code=400)
if not isinstance(message, dict) or message.get("jsonrpc") != "2.0" or not isinstance(message.get("method"), str):
return JSONResponse(_jsonrpc_error(message.get("id") if isinstance(message, dict) else None, -32600, "Invalid Request"), status_code=400)
request_id = message.get("id")
method = message["method"]
if request_id is None and method.startswith("notifications/"):
return Response(status_code=202)
result: dict[str, Any] = {}
try:
if method == "initialize":
result = {
"protocolVersion": MCP_PROTOCOL_VERSION,
"capabilities": {"tools": {"listChanged": False}},
"serverInfo": {"name": "filecodebox", "version": "1.0.0"},
}
elif method == "ping":
result = {}
elif method == "tools/list":
result = {"tools": _tool_definitions()}
elif method == "tools/call":
params = message.get("params")
if not isinstance(params, dict) or not isinstance(params.get("name"), str):
return JSONResponse(_jsonrpc_error(request_id, -32602, "Invalid params"))
arguments = params.get("arguments", {})
if not isinstance(arguments, dict):
return JSONResponse(_jsonrpc_error(request_id, -32602, "Invalid params"))
try:
tool_value = await _call_tool(request, params["name"], arguments)
result = {"content": [{"type": "text", "text": json.dumps(tool_value, ensure_ascii=False)}], "structuredContent": tool_value}
except LookupError as exc:
return JSONResponse(_jsonrpc_error(request_id, -32602, str(exc)))
except HTTPException as exc:
if exc.status_code in {401, 403}:
raise
result = {"content": [{"type": "text", "text": str(exc.detail)}], "isError": True}
except (TypeError, ValueError) as exc:
result = {"content": [{"type": "text", "text": str(exc)}], "isError": True}
else:
return JSONResponse(_jsonrpc_error(request_id, -32601, "Method not found"))
return JSONResponse(_jsonrpc_result(request_id, result), headers={"MCP-Protocol-Version": MCP_PROTOCOL_VERSION})
except HTTPException as exc:
return JSONResponse(
_jsonrpc_error(request_id, -32001 if exc.status_code == 401 else -32003, str(exc.detail)),
status_code=exc.status_code,
)
-52
View File
@@ -1,52 +0,0 @@
from tortoise import connections
async def create_file_codes_table():
conn = connections.get("default")
await conn.execute_script(
"""
CREATE TABLE IF NOT EXISTS filecodes
(
id INTEGER not null
primary key autoincrement,
code VARCHAR(255) not null
unique,
prefix VARCHAR(255) default '' not null,
suffix VARCHAR(255) default '' not null,
uuid_file_name VARCHAR(255),
file_path VARCHAR(255),
size INT default 0 not null,
text TEXT,
expired_at TIMESTAMP,
expired_count INT default 0 not null,
used_count INT default 0 not null,
created_at TIMESTAMP default CURRENT_TIMESTAMP not null
);
CREATE INDEX IF NOT EXISTS idx_filecodes_code_1c7ee7
on filecodes (code);
"""
)
async def create_key_value_table():
conn = connections.get("default")
await conn.execute_script(
"""
CREATE TABLE IF NOT EXISTS keyvalue
(
id INTEGER not null
primary key autoincrement,
key VARCHAR(255) not null
unique,
value JSON,
created_at TIMESTAMP default CURRENT_TIMESTAMP not null
);
CREATE INDEX IF NOT EXISTS idx_keyvalue_key_eab890
on keyvalue (key);
"""
)
async def migrate():
await create_file_codes_table()
await create_key_value_table()
-28
View File
@@ -1,28 +0,0 @@
from tortoise import connections
async def create_upload_chunk_and_update_file_codes_table():
conn = connections.get("default")
await conn.execute_script(
"""
ALTER TABLE "filecodes" ADD "file_hash" VARCHAR(128);
ALTER TABLE "filecodes" ADD "is_chunked" BOOL NOT NULL DEFAULT False;
ALTER TABLE "filecodes" ADD "upload_id" VARCHAR(128);
CREATE TABLE "uploadchunk" (
id INTEGER not null primary key autoincrement,
"upload_id" VARCHAR(36) NOT NULL,
"chunk_index" INT NOT NULL,
"chunk_hash" VARCHAR(128) NOT NULL,
"total_chunks" INT NOT NULL,
"file_size" BIGINT NOT NULL,
"chunk_size" INT NOT NULL,
"created_at" TIMESTAMPTZ NOT NULL,
"file_name" VARCHAR(255) NOT NULL,
"completed" BOOL NOT NULL
);
"""
)
async def migrate():
await create_upload_chunk_and_update_file_codes_table()
-26
View File
@@ -1,26 +0,0 @@
from tortoise import connections
async def create_presign_upload_session_table():
conn = connections.get("default")
await conn.execute_script(
"""
CREATE TABLE IF NOT EXISTS presignuploadsession (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
upload_id VARCHAR(36) NOT NULL UNIQUE,
file_name VARCHAR(255) NOT NULL,
file_size BIGINT NOT NULL,
save_path VARCHAR(512) NOT NULL,
mode VARCHAR(10) NOT NULL,
expire_value INT NOT NULL DEFAULT 1,
expire_style VARCHAR(20) NOT NULL DEFAULT 'day',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_presignuploadsession_upload_id ON presignuploadsession (upload_id);
"""
)
async def migrate():
await create_presign_upload_session_table()
-14
View File
@@ -1,14 +0,0 @@
from tortoise import connections
async def add_save_path_to_uploadchunk():
conn = connections.get("default")
await conn.execute_script(
"""
ALTER TABLE uploadchunk ADD COLUMN save_path VARCHAR(512) NULL;
"""
)
async def migrate():
await add_save_path_to_uploadchunk()
-72
View File
@@ -1,72 +0,0 @@
from tortoise import connections
def _need_upgrade(columns: list[tuple]) -> bool:
for column in columns:
# PRAGMA table_info 返回 (cid, name, type, notnull, dflt_value, pk)
if column[1] == "size":
column_type = (column[2] or "").upper()
return "BIGINT" not in column_type
return False
async def migrate():
conn = connections.get("default")
result = await conn.execute_query("PRAGMA table_info(filecodes)")
columns = result[1] if result and len(result) > 1 else []
if not columns or not _need_upgrade(columns):
return
await conn.execute_script(
"""
BEGIN;
CREATE TABLE IF NOT EXISTS filecodes_new
(
id INTEGER not null
primary key autoincrement,
code VARCHAR(255) not null
unique,
prefix VARCHAR(255) default '' not null,
suffix VARCHAR(255) default '' not null,
uuid_file_name VARCHAR(255),
file_path VARCHAR(255),
size BIGINT default 0 not null,
text TEXT,
expired_at TIMESTAMP,
expired_count INT default 0 not null,
used_count INT default 0 not null,
created_at TIMESTAMP default CURRENT_TIMESTAMP not null,
file_hash VARCHAR(128),
is_chunked BOOL default False not null,
upload_id VARCHAR(128)
);
INSERT INTO filecodes_new (id, code, prefix, suffix, uuid_file_name, file_path, size, text,
expired_at, expired_count, used_count, created_at, file_hash,
is_chunked, upload_id)
SELECT id,
code,
prefix,
suffix,
uuid_file_name,
file_path,
size,
text,
expired_at,
expired_count,
used_count,
created_at,
file_hash,
is_chunked,
upload_id
FROM filecodes;
DROP TABLE filecodes;
ALTER TABLE filecodes_new
RENAME TO filecodes;
CREATE INDEX IF NOT EXISTS idx_filecodes_code_1c7ee7
on filecodes (code);
COMMIT;
"""
)
-19
View File
@@ -1,19 +0,0 @@
from tortoise import connections
async def migrate():
conn = connections.get("default")
await conn.execute_script(
"""
CREATE TABLE IF NOT EXISTS storagereservation (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
token VARCHAR(64) NOT NULL UNIQUE,
size BIGINT NOT NULL,
expires_at TIMESTAMP NOT NULL
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_storagereservation_token
ON storagereservation (token);
CREATE INDEX IF NOT EXISTS idx_storagereservation_expires_at
ON storagereservation (expires_at);
"""
)
-51
View File
@@ -1,51 +0,0 @@
import json
from tortoise import connections
# 配置键驼峰 -> snake_case 统一(2.5)。settings 存于 KeyValue 表的 JSON 值中,
# 迁移在 Python 层读取、改写键名后写回,保证存量部署升级后键名与代码一致。
KEY_RENAMES = {
"adminSessionExpire": "admin_session_expire",
"enableChunk": "enable_chunk",
"errorCount": "error_count",
"errorMinute": "error_minute",
"expireStyle": "expire_style",
"loginCount": "login_count",
"loginMinute": "login_minute",
"openUpload": "open_upload",
"robotsText": "robots_text",
"serverHost": "server_host",
"serverPort": "server_port",
"serverWorkers": "server_workers",
"showAdminAddr": "show_admin_addr",
"storageLimit": "storage_limit",
"themesChoices": "themes_choices",
"themesSelect": "themes_select",
"trustedProxies": "trusted_proxies",
"uploadCount": "upload_count",
"uploadMinute": "upload_minute",
"uploadSize": "upload_size",
}
async def migrate():
conn = connections.get("default")
rows = await conn.execute_query_dict(
"SELECT id, value FROM keyvalue WHERE key = 'settings'"
)
for row in rows:
value = row["value"]
if isinstance(value, str):
try:
value = json.loads(value)
except ValueError:
continue
if not isinstance(value, dict):
continue
renamed = {KEY_RENAMES.get(k, k): v for k, v in value.items()}
if renamed != value:
await conn.execute_query(
"UPDATE keyvalue SET value = ? WHERE id = ?",
[json.dumps(renamed, ensure_ascii=False), row["id"]],
)
-26
View File
@@ -1,26 +0,0 @@
from tortoise import connections
# 高频查询字段索引。真实部署的 schema 全部由迁移建立
# (init_db 不调用 generate_schemas),因此索引必须写在迁移里;
# apps/base/models.py 中的 index/Meta 仅用于测试的 generate_schemas 路径。
#
# - filecodes(expired_at):过期扫描(delete_expire_files)与后台过期统计。
# - filecodes(created_at):后台默认倒序列表、首页最近记录、今日/昨日统计。
# - uploadchunk(upload_id, chunk_index):分片状态检查与合并;其前缀同时
# 覆盖仅按 upload_id 的删除/清理查询(原先没有对应索引)。
# 已知缺口:clean_incomplete_uploads 的 chunk_index=-1 AND created_at<? 用不上
# 该复合索引,待清理任务改为分批查询后再评估。
async def migrate():
conn = connections.get("default")
await conn.execute_script(
"""
CREATE INDEX IF NOT EXISTS idx_filecodes_expired_at
ON filecodes (expired_at);
CREATE INDEX IF NOT EXISTS idx_filecodes_created_at
ON filecodes (created_at);
CREATE INDEX IF NOT EXISTS idx_uploadchunk_upload_chunk
ON uploadchunk (upload_id, chunk_index);
"""
)
-109
View File
@@ -1,109 +0,0 @@
# @Time : 2023/8/13 20:43
# @Author : Lan
# @File : models.py
# @Software: PyCharm
from typing import Any
from tortoise.models import Model
from tortoise.contrib.pydantic import pydantic_model_creator
from tortoise import fields, models
from core.utils import get_now
class FileCodes(models.Model):
id = fields.IntField(pk=True)
code = fields.CharField(max_length=255, unique=True, index=True)
prefix = fields.CharField(max_length=255, default="")
suffix = fields.CharField(max_length=255, default="")
uuid_file_name = fields.CharField(max_length=255, null=True)
file_path = fields.CharField(max_length=255, null=True)
size = fields.BigIntField(default=0)
text = fields.TextField(null=True)
# 过期扫描与后台过期统计的高频过滤列
expired_at = fields.DatetimeField(null=True, index=True)
# -1 = 时间式(看 expired_at),<=0 的 0 = 次数耗尽。默认 -1 而非 0:
# 0 语义是"次数已用完即过期",忘传该字段的创建路径会得到立即过期的记录。
expired_count = fields.IntField(default=-1)
used_count = fields.IntField(default=0)
# 后台默认按创建时间倒序、今日/昨日统计均落在该列
created_at = fields.DatetimeField(auto_now_add=True, index=True)
file_hash = fields.CharField(max_length=64, null=True)
is_chunked = fields.BooleanField(default=False)
upload_id = fields.CharField(max_length=36, null=True)
async def is_expired(self):
if self.expired_at is None:
return False
if self.expired_at and self.expired_count < 0:
return self.expired_at < await get_now()
return self.expired_count <= 0
async def get_file_path(self):
return f"{self.file_path}/{self.uuid_file_name}"
class UploadChunk(models.Model):
id = fields.IntField(pk=True)
# 上传会话/分片查询始终同时带 upload_id;复合索引前缀已覆盖仅按 upload_id
# 的删除与清理查询,单列索引会造成重复。
upload_id = fields.CharField(max_length=36)
chunk_index = fields.IntField()
chunk_hash = fields.CharField(max_length=64)
total_chunks = fields.IntField()
file_size = fields.BigIntField()
chunk_size = fields.IntField()
file_name = fields.CharField(max_length=255)
save_path = fields.CharField(max_length=512, null=True)
created_at = fields.DatetimeField(auto_now_add=True)
completed = fields.BooleanField(default=False)
class Meta:
indexes = [("upload_id", "chunk_index")]
class KeyValue(Model):
# 与 FileCodes 保持同一风格:类属性不写值注解(等号右侧的 Field 描述符
# 才是类级真身;实例级由 tortoise 运行时给出对应 Python 值)
id = fields.IntField(pk=True)
key = fields.CharField(max_length=255, description="键", index=True, unique=True)
# JSONField 实际承载 dict/list(如 settings 配置)
value: Any = fields.JSONField(description="值", null=True)
created_at = fields.DatetimeField(auto_now_add=True, description="创建时间")
class PresignUploadSession(models.Model):
"""预签名上传会话模型"""
id = fields.IntField(pk=True)
upload_id = fields.CharField(max_length=36, unique=True, index=True)
file_name = fields.CharField(max_length=255)
file_size = fields.BigIntField()
save_path = fields.CharField(max_length=512)
mode = fields.CharField(max_length=10) # "direct" 或 "proxy"
expire_value = fields.IntField(default=1)
expire_style = fields.CharField(max_length=20, default="day")
created_at = fields.DatetimeField(auto_now_add=True)
expires_at = fields.DatetimeField() # 会话过期时间
async def is_expired(self):
"""检查会话是否已过期"""
return self.expires_at < await get_now()
class StorageReservation(models.Model):
"""尚未写入 FileCodes 的上传容量预留。"""
id = fields.IntField(pk=True)
token = fields.CharField(max_length=64, unique=True, index=True)
size = fields.BigIntField()
expires_at = fields.DatetimeField(index=True)
file_codes_pydantic = pydantic_model_creator(FileCodes, name="FileCodes")
upload_chunk_pydantic = pydantic_model_creator(UploadChunk, name="UploadChunk")
key_value_pydantic = pydantic_model_creator(KeyValue, name="KeyValue")
presign_upload_session_pydantic = pydantic_model_creator(
PresignUploadSession, name="PresignUploadSession"
)
-223
View File
@@ -1,223 +0,0 @@
"""Public-facing pages and routes: setup wizard, theme assets, index,
robots.txt, and the public config endpoints."""
import html
import re
from functools import lru_cache
from pathlib import Path
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, RedirectResponse
from apps.base.config import initialize_system, is_runtime_initialized
from apps.base.setup_wizard import (
build_public_config,
build_public_meta,
build_setup_page,
build_setup_success_page,
parse_setup_options,
read_setup_payload,
setup_response,
)
from core.response import APIResponse
from core.settings import BASE_DIR, DEFAULT_CONFIG, settings
from core.version import APP_VERSION
router = APIRouter()
@router.get("/setup", include_in_schema=False)
@router.get("/setup/", include_in_schema=False)
async def setup_page():
if is_runtime_initialized():
return RedirectResponse(url="/", status_code=303)
return setup_response(build_setup_page())
@router.post("/setup", include_in_schema=False)
@router.post("/setup/", include_in_schema=False)
async def setup_submit(request: Request):
if is_runtime_initialized():
return RedirectResponse(url="/", status_code=303)
data = await read_setup_payload(request)
admin_password = str(data.get("admin_password") or "")
confirm_password = str(data.get("confirm_password") or "")
site_name = str(data.get("site_name") or "")
if admin_password != confirm_password:
return setup_response(build_setup_page("两次输入的管理员密码不一致", data), 400)
try:
setup_options = parse_setup_options(data)
await initialize_system(
admin_password=admin_password,
site_name=site_name,
setup_options=setup_options,
)
except ValueError as exc:
return setup_response(build_setup_page(str(exc), data), 400)
if "application/json" in request.headers.get("accept", ""):
return APIResponse(detail={"ok": True, "admin": "/#/admin"})
return setup_response(build_setup_success_page())
def resolve_theme_root():
themes_root = (BASE_DIR / "themes").resolve()
theme_root = (BASE_DIR / str(settings.themes_select)).resolve()
try:
theme_root.relative_to(themes_root)
except ValueError:
theme_root = (BASE_DIR / DEFAULT_CONFIG["themes_select"]).resolve()
if not theme_root.exists():
theme_root = (BASE_DIR / DEFAULT_CONFIG["themes_select"]).resolve()
return theme_root
def resolve_theme_file(*parts: str):
theme_root = resolve_theme_root()
file_path = theme_root.joinpath(*parts).resolve()
# 防止通过 /assets/../ 读取主题目录外的文件。
try:
file_path.relative_to(theme_root)
except ValueError:
raise HTTPException(status_code=404, detail="资源不存在")
if not file_path.is_file():
raise HTTPException(status_code=404, detail="资源不存在")
return file_path
# Vite 产物带内容指纹(如 index-BX2s1QqP.js),内容变化时文件名必然变化,
# 可以安全地长缓存;public/ 下的未哈希资源原地升级时名字不变,只给短缓存,
# 过期后靠 FileResponse 自带的 ETag/Last-Modified 走 304。
_HASHED_ASSET_RE = re.compile(
r"-[A-Za-z0-9_-]{8,}\.(?:js|mjs|css|map|woff2?|ttf|otf|eot|svg|png|jpe?g|gif|webp|avif|ico)$",
re.IGNORECASE,
)
_IMMUTABLE_ASSET_CACHE = "public, max-age=31536000, immutable"
_DEFAULT_ASSET_CACHE = "public, max-age=86400"
def _asset_cache_control(asset_path: str) -> str:
if _HASHED_ASSET_RE.search(asset_path):
return _IMMUTABLE_ASSET_CACHE
return _DEFAULT_ASSET_CACHE
@lru_cache(maxsize=8)
def _render_index_html(
template_path: str,
inode: int,
mtime_ns: int,
size: int,
title: str,
description: str,
keywords: str,
opacity: str,
background: str,
) -> str:
"""Render index.html once per (theme file version, site config) pair.
The stat fields keep the cache honest when a theme is replaced in place;
config values are part of the key so admin edits are visible immediately
without any cross-module invalidation hook.
"""
return (
Path(template_path)
.read_text(encoding="utf-8")
.replace("{{title}}", html.escape(title))
.replace("{{description}}", html.escape(description))
.replace("{{keywords}}", html.escape(keywords))
.replace("{{opacity}}", html.escape(opacity))
.replace("{{background}}", html.escape(background))
)
@router.get("/assets/{asset_path:path}", include_in_schema=False)
async def theme_asset(asset_path: str):
return FileResponse(
resolve_theme_file("assets", asset_path),
headers={"Cache-Control": _asset_cache_control(asset_path)},
)
async def not_found_handler(request, exc=None):
"""区分"浏览器导航到未知路径"与"API 处理器抛出的 404"。
Starlette 的 status-code handler 优先于 HTTPException 类 handler,
直接注册 index 会让全应用所有 HTTPException(404) 变成 200 HTML 页
(负路径测试抓到的存量 bug)。浏览器(Accept 含 text/html)仍拿到
主题首页做 SPA 兜底;API 客户端拿到 JSON 404。
"""
json_404 = JSONResponse(
status_code=404,
content={"code": 404, "message": "Not Found", "detail": "资源不存在"},
)
if request is not None and request.method in {"GET", "HEAD"} and "text/html" in (
request.headers.get("accept", "")
):
try:
return await index(request, exc)
except HTTPException:
return json_404
return json_404
@router.get("/")
async def index(request=None, exc=None):
# Site config is admin input (and during the setup window anyone can claim it);
# always escape before injecting into the theme template to prevent stored XSS
# (mirrors the setup page).
template_path = resolve_theme_file("index.html")
try:
stat_result = template_path.stat()
except OSError:
# resolve_theme_file 与 stat 之间文件被删除/替换时仍按 404 处理,
# 不能退回缓存内容。
raise HTTPException(status_code=404, detail="资源不存在")
content = _render_index_html(
str(template_path),
stat_result.st_ino,
stat_result.st_mtime_ns,
stat_result.st_size,
str(settings.name),
str(settings.description),
str(settings.keywords),
str(settings.opacity),
str(settings.background),
)
return HTMLResponse(
content=content,
media_type="text/html",
headers={"Cache-Control": "no-cache"},
)
@router.get("/robots.txt")
async def robots():
return HTMLResponse(content=settings.robots_text, media_type="text/plain")
@router.post("/")
async def get_config():
return APIResponse(detail=build_public_config())
@router.get("/api/v1/config")
async def get_public_config():
return APIResponse(
detail={
"config": build_public_config(),
"meta": build_public_meta(),
}
)
@router.get("/health")
async def health_check():
return APIResponse(
detail={
"status": "ok",
"version": APP_VERSION,
"storage": settings.file_storage,
"theme": settings.themes_select,
}
)
-132
View File
@@ -1,132 +0,0 @@
import datetime
from fastapi import HTTPException
from tortoise import connections
from tortoise.expressions import Q
from tortoise.functions import Sum
from apps.base.models import FileCodes, StorageReservation
from apps.base.local_share import LOCAL_REF_MARKER
from core.settings import settings
from core.utils import get_now
def owned_storage_queryset(queryset=None):
"""FileCodes that occupy FileCodeBox storage (exclude NAS local-ref rows).
SQL ``file_path != 'local-ref'`` drops NULLs, so keep null paths explicitly.
"""
qs = FileCodes.all() if queryset is None else queryset
return qs.filter(Q(file_path__isnull=True) | ~Q(file_path=LOCAL_REF_MARKER))
def _detect_sql_dialect() -> str:
"""识别当前默认连接的 SQL 方言。
优先使用 Tortoise capabilities.dialect;
回退到 db_config 中的 engine 路径字符串。
"""
conn = connections.get("default")
dialect = getattr(getattr(conn, "capabilities", None), "dialect", "") or ""
if dialect:
return dialect.lower()
try:
engine = str(connections.db_config.get("default", {}).get("engine", "")).lower()
except Exception:
engine = ""
if "postgres" in engine or "asyncpg" in engine or "psycopg" in engine:
return "postgres"
if "mysql" in engine:
return "mysql"
if "sqlite" in engine:
return "sqlite"
return "sqlite"
def _sql_placeholders(count: int) -> list[str]:
"""根据数据库方言生成参数占位符(多数据库兼容)。
SQLite 用 ?,PostgreSQL 用 $1/$2/...,MySQL 用 %s。
"""
dialect = _detect_sql_dialect()
if dialect in {"postgres", "postgresql"}:
return [f"${i}" for i in range(1, count + 1)]
if dialect == "mysql":
return ["%s"] * count
return ["?"] * count
def get_storage_limit() -> int:
try:
return max(0, int(getattr(settings, "storage_limit", 0)))
except (TypeError, ValueError):
return 0
async def get_storage_usage() -> dict[str, int | None]:
now = await get_now()
# SQL 聚合:此函数在每次上传配额检查时调用,禁止全表拉取(D3)
used_rows = await owned_storage_queryset().annotate(total=Sum("size")).values("total")
reserved_rows = await StorageReservation.filter(expires_at__gt=now).annotate(
total=Sum("size")
).values("total")
limit = get_storage_limit()
used_bytes = used_rows[0]["total"] or 0
reserved_bytes = reserved_rows[0]["total"] or 0
return {
"limit": limit,
"used": used_bytes,
"reserved": reserved_bytes,
"available": max(0, limit - used_bytes - reserved_bytes) if limit else None,
}
async def reserve_storage(token: str, size: int, ttl_seconds: int) -> None:
requested_size = max(0, int(size))
limit = get_storage_limit()
if not limit or requested_size == 0:
return
now = await get_now()
expires_at = now + datetime.timedelta(seconds=ttl_seconds)
conn = connections.get("default")
await StorageReservation.filter(token=token, expires_at__lte=now).delete()
existing = await StorageReservation.filter(token=token, expires_at__gt=now).first()
if existing:
if existing.size == requested_size:
return
raise HTTPException(status_code=409, detail="上传容量预留信息不一致")
try:
ph = _sql_placeholders(6)
affected, _ = await conn.execute_query(
f"""
INSERT INTO storagereservation (token, size, expires_at)
SELECT {ph[0]}, {ph[1]}, {ph[2]}
WHERE (
COALESCE((SELECT SUM(size) FROM filecodes WHERE file_path IS NULL OR file_path != '{LOCAL_REF_MARKER}'), 0)
+ COALESCE((SELECT SUM(size) FROM storagereservation WHERE expires_at > {ph[3]}), 0)
+ {ph[4]}
) <= {ph[5]}
""",
[token, requested_size, expires_at, now, requested_size, limit],
)
except Exception:
concurrent = await StorageReservation.filter(
token=token, size=requested_size, expires_at__gt=now
).exists()
if concurrent:
return
raise
if affected != 1:
concurrent = await StorageReservation.filter(
token=token, size=requested_size, expires_at__gt=now
).exists()
if concurrent:
return
raise HTTPException(status_code=507, detail="存储空间已达到管理员设置的容量上限")
async def release_storage(token: str) -> None:
await StorageReservation.filter(token=token).delete()
-35
View File
@@ -1,35 +0,0 @@
from pydantic import BaseModel
class SelectFileModel(BaseModel):
code: str
class InitChunkUploadModel(BaseModel):
file_name: str
chunk_size: int = 5 * 1024 * 1024
file_size: int
file_hash: str
class CompleteUploadModel(BaseModel):
expire_value: int
expire_style: str
# 预签名上传相关模型
class PresignUploadInitRequest(BaseModel):
"""预签名上传初始化请求"""
file_name: str
file_size: int
expire_value: int = 1
expire_style: str = "day"
class PresignUploadInitResponse(BaseModel):
"""预签名上传初始化响应"""
upload_id: str
upload_url: str
mode: str # "direct" 或 "proxy"
save_path: str
expires_in: int # URL过期时间(秒)
-415
View File
@@ -1,415 +0,0 @@
"""apps.base service layer: upload orchestration shared by share/chunk/presign.
Handlers in views.py keep HTTP concerns (form parsing, session lookup, status
codes for malformed requests, rate limiting); this module owns the business
workflow: quota reservation, storage writes, share-record creation, and
failure rollback/cleanup.
"""
import os
from fastapi import HTTPException, UploadFile
from fastapi.responses import FileResponse, Response, StreamingResponse
from core.logger import logger
from core.settings import settings
from core.storage import FileStorageInterface, StoredDownload, StoredFile, storages
from apps.base.file_validation import validate_upload_file
from apps.base.local_share import build_local_ref_download, is_local_ref
from apps.base.models import FileCodes, PresignUploadSession, UploadChunk
from apps.base.quota import release_storage, reserve_storage
from apps.base.utils import build_file_path, get_expire_info
import uuid
# 预签名上传会话有效期(秒)
PRESIGN_SESSION_EXPIRES = 900 # 15分钟
def stored_file_of(code: FileCodes) -> StoredFile:
"""Adapter: project an ORM FileCodes row onto the storage-layer contract."""
return StoredFile(
file_path=code.file_path,
uuid_file_name=code.uuid_file_name,
code=code.code,
prefix=code.prefix,
suffix=code.suffix,
text=code.text or "",
)
async def get_stored_download(file_code, file_storage: FileStorageInterface | None = None) -> StoredDownload:
if is_local_ref(file_code):
return build_local_ref_download(file_code)
storage = file_storage or storages[settings.file_storage]()
return await storage.get_file_response(stored_file_of(file_code))
async def rollback_saved_file(
storage: FileStorageInterface,
file_path: str,
uuid_file_name: str,
*,
context: str,
upload_id: str | None = None,
) -> None:
"""Best-effort delete of a stored file after a failed upload workflow.
Rollback failures are logged (with traceback) and swallowed: the original
error must propagate; the file, if left behind, is cleaned by the expiry
task once its orphaned record ages out.
"""
try:
await storage.delete_file(
StoredFile(file_path=file_path, uuid_file_name=uuid_file_name)
)
except Exception:
logger.warning(
"%s:回滚删除已保存文件失败%s",
context,
f" upload_id={upload_id}" if upload_id else "",
exc_info=True,
)
async def validate_file_size(file: UploadFile, max_size: int) -> int:
"""Return the upload's size, rejecting anything above max_size."""
size = file.size
if size is None:
# 无长度声明的上传(如 chunked 传输):必须走底层文件对象拿末尾偏移。
# 注意两层都不能用:UploadFile.seek 只接受单参数(TypeError);
# 底层 SpooledTemporaryFile.seek 是同步方法(不能 await)。
file.file.seek(0, 2)
size = file.file.tell()
file.file.seek(0)
if size > max_size:
max_size_mb = max_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"大小超过限制,最大为{max_size_mb:.2f} MB"
)
return size
def chunk_reservation_ttl() -> int:
ttl = max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600
return ttl
class FileUploadService:
"""统一的文件上传服务"""
@staticmethod
def _storage() -> FileStorageInterface:
return storages[settings.file_storage]()
@staticmethod
async def generate_file_path(
file_name: str, upload_id: str | None = None
) -> tuple[str, str, str, str, str]:
"""Delegates path generation to apps.base.utils.build_file_path."""
return await build_file_path(file_name, upload_id or uuid.uuid4().hex)
@staticmethod
async def create_file_record(
file_name: str,
file_size: int,
file_path: str,
expire_value: int,
expire_style: str,
**extra_fields,
) -> str:
"""统一创建FileCodes记录,返回code"""
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
prefix, suffix = os.path.splitext(file_name)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=file_name,
file_path=file_path,
size=file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
**extra_fields,
)
return code
@staticmethod
async def create_text_share(
text: str, expire_value: int, expire_style: str
) -> str:
"""文本分享:配额预留 → 建分享记录 → 释放配额。"""
text_size = len(text.encode("utf-8"))
token = f"text:{uuid.uuid4().hex}"
await reserve_storage(token, text_size, ttl_seconds=300)
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await FileCodes.create(
code=code,
text=text,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
size=text_size,
prefix="Text",
)
finally:
await release_storage(token)
return code
@staticmethod
async def create_file_share(
file: UploadFile, *, size: int, expire_value: int, expire_style: str
) -> dict[str, str]:
"""文件分享:路径生成 → 配额预留 → 存储写入 → 建分享记录,失败回滚已存文件。"""
path, suffix, prefix, uuid_file_name, save_path = (
await FileUploadService.generate_file_path(file.filename or "")
)
token = f"file:{uuid.uuid4().hex}"
await reserve_storage(token, size, ttl_seconds=3600)
storage = FileUploadService._storage()
try:
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await storage.save_file(file.file, save_path, file.content_type)
await FileCodes.create(
code=code,
prefix=prefix,
suffix=suffix,
uuid_file_name=uuid_file_name,
file_path=path,
size=size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
)
except Exception:
await rollback_saved_file(
storage, path, uuid_file_name, context="分享上传"
)
raise
finally:
await release_storage(token)
# UploadFile.filename 类型为 str | None;路径生成已用 "" 兜底,
# 响应处保持 str 以匹配 dict[str, str] 契约
return {"code": code, "name": file.filename or ""}
@staticmethod
async def complete_chunked_upload(
upload_id: str, chunk_info: UploadChunk, expire_value: int, expire_style: str
) -> dict[str, str]:
"""分片合并:配额 → 完整性/大小校验 → 合并 → 建分享记录 → 清理分片。
失败路径的配额释放与清理范围与原实现逐一对齐:
完整性校验失败仅抛 400(预留由 TTL 兜底);合并失败清理分片文件后抛 500。
"""
storage = FileUploadService._storage()
await reserve_storage(
f"chunk:{upload_id}", chunk_info.file_size, ttl_seconds=chunk_reservation_ttl()
)
completed_chunks = await UploadChunk.filter(
upload_id=upload_id, completed=True
).all()
if len(completed_chunks) != chunk_info.total_chunks:
raise HTTPException(400, "分片不完整")
# 用分片数 * chunk_size 校验最大可能大小
max_total_size = len(completed_chunks) * chunk_info.chunk_size
if max_total_size > settings.upload_size:
save_path = chunk_info.save_path
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning(
"分片超限中止:清理分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
403, f"实际上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
save_path = chunk_info.save_path
path = os.path.dirname(save_path) if save_path else ""
safe_file_name = os.path.basename(save_path) if save_path else ""
prefix, suffix = os.path.splitext(safe_file_name)
try:
records = {r.chunk_index: r for r in completed_chunks}
_, file_hash = await storage.merge_chunks(
upload_id,
chunk_info.total_chunks,
chunk_info.chunk_size,
save_path,
records,
)
expired_at, expired_count, used_count, code = await get_expire_info(
expire_value, expire_style
)
await FileCodes.create(
code=code,
file_hash=file_hash, # 使用合并后计算的哈希
is_chunked=True,
upload_id=upload_id,
size=chunk_info.file_size,
expired_at=expired_at,
expired_count=expired_count,
used_count=used_count,
file_path=path,
uuid_file_name=safe_file_name,
prefix=prefix,
suffix=suffix,
)
await storage.clean_chunks(upload_id, save_path)
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
return {"code": code, "name": safe_file_name}
except ValueError as e:
raise HTTPException(400, str(e))
except Exception as e:
# 合并失败时清理临时文件
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning(
"分片合并失败:清理临时分片文件失败 upload_id=%s",
upload_id,
exc_info=True,
)
raise HTTPException(500, f"文件合并失败: {str(e)}")
@staticmethod
async def commit_proxy_upload(
session: PresignUploadSession, file: UploadFile
) -> str:
"""预签名代理上传:配额 → 大小/类型/一致性校验 → 转存 → 建记录 → 会话清理。
校验失败不释放预留(与原实现一致,由 TTL 兜底)。
"""
await reserve_storage(
f"presign:{session.upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
if abs(file_size - session.file_size) > 1024:
raise HTTPException(400, "文件大小与声明不符")
storage = FileUploadService._storage()
try:
await storage.save_file(file.file, session.save_path, file.content_type)
except Exception as e:
raise HTTPException(500, f"文件保存失败: {str(e)}")
try:
code = await FileUploadService.create_file_record(
session.file_name,
file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
await rollback_saved_file(
storage,
os.path.dirname(session.save_path),
os.path.basename(session.save_path),
context="预签名代理上传:记录创建失败",
upload_id=session.upload_id,
)
raise
await session.delete()
await release_storage(f"presign:{session.upload_id}")
return code
@staticmethod
async def confirm_direct_upload(session: PresignUploadSession) -> str:
"""预签名直传确认:配额 → 文件存在性 → 建记录 → 会话清理。
预留失败说明配额已耗尽,此时清理远端临时文件与会话后原样抛出。
"""
try:
await reserve_storage(
f"presign:{session.upload_id}",
session.file_size,
ttl_seconds=PRESIGN_SESSION_EXPIRES,
)
except HTTPException:
storage = FileUploadService._storage()
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
finally:
await session.delete()
await release_storage(f"presign:{session.upload_id}")
raise
storage = FileUploadService._storage()
if not await storage.file_exists(session.save_path):
raise HTTPException(404, "文件未上传或上传失败")
try:
code = await FileUploadService.create_file_record(
session.file_name,
session.file_size,
os.path.dirname(session.save_path),
session.expire_value,
session.expire_style,
)
except Exception:
await rollback_saved_file(
storage,
os.path.dirname(session.save_path),
os.path.basename(session.save_path),
context="预签名确认:记录创建失败",
upload_id=session.upload_id,
)
raise
await session.delete()
await release_storage(f"presign:{session.upload_id}")
return code
def response_from_download(download: StoredDownload):
"""Build the starlette Response for a StoredDownload (view-layer duty)."""
if download.path is not None:
return FileResponse(
download.path,
media_type=download.media_type,
headers=download.headers,
filename=download.filename,
)
if download.content is not None:
return Response(
download.content, media_type=download.media_type, headers=download.headers
)
if download.stream_factory is None:
# 数据契约要求三者至少其一;防御性 500 语义
raise ValueError("StoredDownload 缺少可用的下载载荷")
return StreamingResponse(
download.stream_factory(),
media_type=download.media_type,
headers=download.headers,
background=download.background,
)
-694
View File
@@ -1,694 +0,0 @@
"""Setup wizard: form parsing rules and the two admin-facing HTML pages.
Pure presentation/logic — no router here; routes live in apps.base.pages.
"""
import html
from urllib.parse import parse_qs
from typing import Any
from fastapi import Request
from fastapi.responses import HTMLResponse
from core.settings import DEFAULT_CONFIG, settings
from core.version import APP_VERSION
FILE_SIZE_UNITS = {"KB": 1024, "MB": 1024**2, "GB": 1024**3}
SAVE_TIME_UNITS = {"second": 1, "minute": 60, "hour": 3600, "day": 86400}
EXPIRE_STYLE_OPTIONS = [
("day", "按天"),
("hour", "按小时"),
("minute", "按分钟"),
("forever", "永久"),
("count", "按取件次数"),
]
def normalize_public_flag(value) -> int:
if isinstance(value, str):
return int(value.strip().lower() in {"1", "true", "on", "yes"})
return int(bool(value))
def build_public_config() -> dict:
return {
"name": settings.name,
"description": settings.description,
"explain": settings.page_explain,
"upload_size": settings.upload_size,
"allowed_file_types": settings.allowed_file_types,
"expire_style": settings.expire_style,
"enable_chunk": settings.enable_chunk,
"open_upload": settings.open_upload,
"notify_title": settings.notify_title,
"notify_content": settings.notify_content,
"show_admin_address": normalize_public_flag(settings.show_admin_addr),
"max_save_seconds": settings.max_save_seconds,
}
def build_public_meta() -> dict:
return {
"version": APP_VERSION,
"api": {
"legacy_config": "/",
"public_config": "/api/v1/config",
"health": "/health",
},
"features": {
"chunk_upload": bool(settings.enable_chunk),
"guest_upload": bool(settings.open_upload),
"admin_address_visible": bool(normalize_public_flag(settings.show_admin_addr)),
"expiration_modes": settings.expire_style,
},
"limits": {
"upload_size": settings.upload_size,
"allowed_file_types": settings.allowed_file_types,
"max_save_seconds": settings.max_save_seconds,
"upload_window_minutes": settings.upload_minute,
"upload_window_count": settings.upload_count,
},
}
def get_form_value(data: dict[str, Any], key: str, default: str = "") -> str:
value = data.get(key, default)
if isinstance(value, list):
value = value[-1] if value else default
return str(value if value is not None else default)
def get_form_list(data: dict[str, Any], key: str) -> list[str]:
value = data.get(key, [])
if isinstance(value, list):
return [str(item) for item in value if str(item)]
if value:
return [str(value)]
return []
def normalize_bool_field(data: dict, key: str, default: bool) -> bool:
if key not in data:
return default
return get_form_value(data, key).lower() in {"1", "true", "on", "yes"}
def parse_int_field(
data: dict[str, Any],
key: str,
default: int,
label: str,
min_value: int = 0,
max_value: int | None = None,
) -> int:
raw_value = get_form_value(data, key, str(default)).strip()
try:
value = int(raw_value)
except ValueError:
raise ValueError(f"{label} 必须是整数")
if value < min_value:
raise ValueError(f"{label} 不能小于 {min_value}")
if max_value is not None and value > max_value:
raise ValueError(f"{label} 不能大于 {max_value}")
return value
def parse_allowed_file_types(value: str) -> list[str]:
items = [item.strip() for item in value.split(",") if item.strip()]
return items or ["*"]
def parse_setup_options(data: dict) -> dict:
upload_size_unit = get_form_value(data, "upload_size_unit", "MB").upper()
if upload_size_unit not in FILE_SIZE_UNITS:
raise ValueError("文件大小单位不正确")
upload_size_value = parse_int_field(
data, "upload_size_value", 10, "文件大小限制", min_value=1
)
save_time_unit = get_form_value(data, "save_time_unit", "day")
if save_time_unit not in SAVE_TIME_UNITS:
raise ValueError("最长保存时间单位不正确")
save_time_value = parse_int_field(
data, "save_time_value", 0, "最长保存时间", min_value=0
)
expire_styles = get_form_list(data, "expire_style")
valid_expire_styles = {style for style, _label in EXPIRE_STYLE_OPTIONS}
expire_styles = [style for style in expire_styles if style in valid_expire_styles]
if not expire_styles:
raise ValueError("至少需要选择一种过期方式")
code_generate_type = get_form_value(
data, "code_generate_type", DEFAULT_CONFIG["code_generate_type"]
)
if code_generate_type not in {"number", "secret"}:
raise ValueError("提取码类型不正确")
return {
"allowed_file_types": parse_allowed_file_types(
get_form_value(data, "allowed_file_types", "*")
),
"code_generate_type": code_generate_type,
"enable_chunk": int(normalize_bool_field(data, "enable_chunk", False)),
"error_count": parse_int_field(
data, "error_count", DEFAULT_CONFIG["error_count"], "取件错误次数限制", 1
),
"error_minute": parse_int_field(
data, "error_minute", DEFAULT_CONFIG["error_minute"], "取件错误检测窗口", 1
),
"login_count": parse_int_field(
data, "login_count", DEFAULT_CONFIG["login_count"], "登录失败次数限制", 1
),
"login_minute": parse_int_field(
data, "login_minute", DEFAULT_CONFIG["login_minute"], "登录失败检测窗口", 1
),
"expire_style": expire_styles,
"max_save_seconds": save_time_value * SAVE_TIME_UNITS[save_time_unit],
"open_upload": int(normalize_bool_field(data, "open_upload", True)),
"upload_count": parse_int_field(
data, "upload_count", DEFAULT_CONFIG["upload_count"], "上传次数限制", 1
),
"upload_minute": parse_int_field(
data, "upload_minute", DEFAULT_CONFIG["upload_minute"], "上传检测窗口", 1
),
"upload_size": upload_size_value * FILE_SIZE_UNITS[upload_size_unit],
}
def build_expire_style_inputs(selected_styles: list[str]) -> str:
inputs = []
selected = set(selected_styles)
for style, label in EXPIRE_STYLE_OPTIONS:
checked = " checked" if style in selected else ""
inputs.append(
f'<label class="check"><input type="checkbox" name="expire_style" value="{style}"{checked}> {label}</label>'
)
return "\n ".join(inputs)
def build_setup_page(error: str = "", form: dict | None = None) -> str:
form = form or {}
escaped_error = html.escape(error)
escaped_site_name = html.escape(
get_form_value(form, "site_name", DEFAULT_CONFIG["name"])
)
escaped_allowed_types = html.escape(get_form_value(form, "allowed_file_types", "*"))
upload_size_value = html.escape(get_form_value(form, "upload_size_value", "10"))
upload_size_unit = get_form_value(form, "upload_size_unit", "MB").upper()
save_time_value = html.escape(get_form_value(form, "save_time_value", "0"))
save_time_unit = get_form_value(form, "save_time_unit", "day")
upload_minute = html.escape(
get_form_value(form, "upload_minute", str(DEFAULT_CONFIG["upload_minute"]))
)
upload_count = html.escape(
get_form_value(form, "upload_count", str(DEFAULT_CONFIG["upload_count"]))
)
error_minute = html.escape(
get_form_value(form, "error_minute", str(DEFAULT_CONFIG["error_minute"]))
)
error_count = html.escape(
get_form_value(form, "error_count", str(DEFAULT_CONFIG["error_count"]))
)
login_minute = html.escape(
get_form_value(form, "login_minute", str(DEFAULT_CONFIG["login_minute"]))
)
login_count = html.escape(
get_form_value(form, "login_count", str(DEFAULT_CONFIG["login_count"]))
)
open_upload_checked = (
" checked" if normalize_bool_field(form, "open_upload", True) else ""
)
chunk_checked = (
" checked" if normalize_bool_field(form, "enable_chunk", False) else ""
)
code_generate_type = get_form_value(
form, "code_generate_type", DEFAULT_CONFIG["code_generate_type"]
)
selected_expire_styles = get_form_list(form, "expire_style") or list(
DEFAULT_CONFIG["expire_style"]
)
expire_style_inputs = build_expire_style_inputs(selected_expire_styles)
size_unit_options = "\n".join(
f'<option value="{unit}"{" selected" if unit == upload_size_unit else ""}>{unit}</option>'
for unit in FILE_SIZE_UNITS
)
save_time_unit_options = "\n".join(
f'<option value="{unit}"{" selected" if unit == save_time_unit else ""}>{label}</option>'
for unit, label in [
("second", "秒"),
("minute", "分钟"),
("hour", "小时"),
("day", "天"),
]
)
code_type_options = "\n".join(
f'<option value="{value}"{" selected" if value == code_generate_type else ""}>{label}</option>'
for value, label in [("number", "数字"), ("secret", "随机字符")]
)
error_block = (
f'<div class="alert" role="alert">{escaped_error}</div>' if escaped_error else ""
)
return f"""<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>初始化 FileCodeBox</title>
<style>
:root {{
color-scheme: light;
--bg: #f5f5f7;
--panel: rgba(255, 255, 255, .86);
--panel-strong: rgba(255, 255, 255, .96);
--text: #18181b;
--muted: #71717a;
--line: rgba(228, 228, 231, .9);
--line-strong: rgba(212, 212, 216, .95);
--primary: #18181b;
--primary-soft: #f4f4f5;
--danger-bg: #fef2f2;
--danger: #b91c1c;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0;
min-height: 100vh;
display: grid;
place-items: center;
padding: 14px;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: var(--bg);
color: var(--text);
}}
main {{
width: min(100%, 1040px);
padding: 14px;
border: 1px solid rgba(255, 255, 255, .8);
border-radius: 20px;
background: rgba(255, 255, 255, .62);
box-shadow: 0 22px 70px -34px rgba(24, 24, 27, .32);
backdrop-filter: blur(22px);
}}
.setup-header {{
display: flex;
align-items: center;
justify-content: space-between;
gap: 16px;
padding: 10px 10px 12px;
}}
.brand-mark {{
flex: 0 0 auto;
width: 40px;
height: 40px;
display: grid;
place-items: center;
border-radius: 14px;
background: #18181b;
color: #fff;
font-weight: 800;
letter-spacing: 0;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .42);
}}
.title-wrap {{
min-width: 0;
display: flex;
align-items: center;
gap: 12px;
}}
h1 {{
margin: 0;
font-size: 21px;
line-height: 1.25;
letter-spacing: 0;
font-weight: 750;
}}
p {{
margin: 2px 0 0;
color: var(--muted);
line-height: 1.35;
font-size: 13px;
}}
form {{
margin: 0;
padding: 12px;
border: 1px solid var(--line);
border-radius: 16px;
background: var(--panel);
}}
.panel-grid {{
display: grid;
grid-template-columns: 1.05fr 1fr 1fr;
gap: 10px;
}}
.panel {{
min-width: 0;
padding: 10px;
border: 1px solid var(--line);
border-radius: 14px;
background: rgba(255, 255, 255, .68);
}}
.panel-wide {{
grid-column: 1 / -1;
display: grid;
grid-template-columns: minmax(0, 1.3fr) minmax(0, .9fr);
gap: 12px;
align-items: end;
}}
.panel-title {{
margin: 0 0 2px;
color: #18181b;
font-size: 13px;
font-weight: 760;
}}
label {{
display: block;
margin: 8px 0 5px;
font-weight: 650;
font-size: 12px;
color: #3f3f46;
}}
input, select {{
width: 100%;
height: 34px;
border: 1px solid var(--line);
border-radius: 10px;
padding: 0 10px;
font: inherit;
color: var(--text);
outline: none;
background: rgba(255, 255, 255, .84);
transition: border-color .2s ease, box-shadow .2s ease, background-color .2s ease;
}}
input:focus, select:focus {{
border-color: #a1a1aa;
background: #fff;
box-shadow: 0 10px 22px -12px rgba(24, 24, 27, .2), 0 0 0 3px rgba(24, 24, 27, .06);
}}
.grid {{
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 6px 12px;
}}
.full {{ grid-column: 1 / -1; }}
.span-2 {{ grid-column: span 2; }}
.row {{
display: grid;
grid-template-columns: minmax(0, 1fr) 88px;
gap: 8px;
}}
.checks {{
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-top: 5px;
}}
.check {{
display: flex;
align-items: center;
gap: 8px;
min-height: 36px;
margin: 0;
padding: 0 9px;
border: 1px solid var(--line);
border-radius: 10px;
font-weight: 550;
font-size: 13px;
color: var(--text);
white-space: nowrap;
background: rgba(255, 255, 255, .72);
transition: border-color .2s ease, background-color .2s ease, box-shadow .2s ease;
}}
.check:hover {{
border-color: var(--line-strong);
background: #fff;
box-shadow: 0 8px 20px -8px rgba(0, 0, 0, .1);
}}
.check input {{
width: 16px;
height: 16px;
padding: 0;
box-shadow: none;
}}
.section-title {{
display: none;
}}
.help {{
margin: 4px 0 0;
font-size: 12px;
color: var(--muted);
}}
.alert {{
margin-bottom: 8px;
padding: 9px 12px;
border-radius: 12px;
background: var(--danger-bg);
color: var(--danger);
font-size: 13px;
}}
button {{
width: 100%;
height: 38px;
margin-top: 12px;
border: 0;
border-radius: 12px;
background: var(--primary);
color: white;
font: inherit;
font-weight: 700;
cursor: pointer;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .42);
transition: background-color .2s ease, box-shadow .2s ease;
}}
button:hover {{
background: #27272a;
box-shadow: 0 16px 34px -14px rgba(24, 24, 27, .56);
}}
.hint {{
margin-top: 8px;
margin-bottom: 0;
font-size: 12px;
}}
.compact-help {{
display: inline;
margin-left: 6px;
color: var(--muted);
font-weight: 500;
font-size: 12px;
}}
.topline {{
display: none;
}}
.badge {{
min-height: 26px;
display: inline-flex;
align-items: center;
padding: 0 10px;
border: 1px solid var(--line);
border-radius: 999px;
background: rgba(255, 255, 255, .76);
color: #52525b;
font-size: 12px;
font-weight: 650;
white-space: nowrap;
}}
@media (max-width: 640px) {{
body {{ padding: 10px; }}
main {{ padding: 10px; }}
.setup-header {{ align-items: flex-start; padding: 6px 4px 10px; }}
.title-wrap {{ align-items: flex-start; }}
.panel-grid, .panel-wide {{ grid-template-columns: 1fr; }}
.grid, .checks {{ grid-template-columns: 1fr; }}
.span-2 {{ grid-column: auto; }}
.row {{ grid-template-columns: 1fr; }}
.badge {{ display: none; }}
}}
</style>
</head>
<body>
<main>
<div class="setup-header">
<div class="title-wrap">
<div class="brand-mark">FCB</div>
<div>
<h1>初始化 FileCodeBox</h1>
<p>首次配置管理员密码、上传限制和取件策略,后续可在后台调整。</p>
</div>
</div>
<span class="badge">首次配置向导</span>
</div>
<form method="post" action="/setup" autocomplete="off">
{error_block}
<div class="panel-grid">
<section class="panel">
<div class="panel-title">基础设置</div>
<label for="site_name">站点名称</label>
<input id="site_name" name="site_name" maxlength="80" value="{escaped_site_name}" placeholder="文件快递柜 - FileCodeBox">
<label for="admin_password">管理员密码</label>
<input id="admin_password" name="admin_password" type="password" minlength="8" required autofocus>
<label for="confirm_password">确认管理员密码</label>
<input id="confirm_password" name="confirm_password" type="password" minlength="8" required>
</section>
<section class="panel">
<div class="panel-title">上传设置</div>
<label for="upload_size_value">单文件大小限制</label>
<div class="row">
<input id="upload_size_value" name="upload_size_value" type="number" min="1" value="{upload_size_value}" required>
<select name="upload_size_unit" aria-label="文件大小单位">
{size_unit_options}
</select>
</div>
<label for="upload_count">上传频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="upload_count" name="upload_count" type="number" min="1" value="{upload_count}" required>
<input name="upload_minute" type="number" min="1" value="{upload_minute}" aria-label="上传检测窗口分钟" required>
</div>
<div class="grid">
<input type="hidden" name="open_upload" value="0">
<label class="check"><input type="checkbox" name="open_upload" value="1"{open_upload_checked}> 允许游客上传</label>
<input type="hidden" name="enable_chunk" value="0">
<label class="check"><input type="checkbox" name="enable_chunk" value="1"{chunk_checked}> 启用切片上传</label>
</div>
</section>
<section class="panel">
<div class="panel-title">取件与保存</div>
<label for="error_count">取件错误频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="error_count" name="error_count" type="number" min="1" value="{error_count}" required>
<input name="error_minute" type="number" min="1" value="{error_minute}" aria-label="取件错误检测窗口分钟" required>
</div>
<label for="login_count">管理员登录失败频率 <span class="compact-help">次数 / 分钟</span></label>
<div class="row">
<input id="login_count" name="login_count" type="number" min="1" value="{login_count}" required>
<input name="login_minute" type="number" min="1" value="{login_minute}" aria-label="登录失败检测窗口分钟" required>
</div>
<label for="save_time_value">最长保存时间</label>
<div class="row">
<input id="save_time_value" name="save_time_value" type="number" min="0" value="{save_time_value}" required>
<select name="save_time_unit" aria-label="最长保存时间单位">
{save_time_unit_options}
</select>
</div>
<label for="code_generate_type">提取码类型</label>
<select id="code_generate_type" name="code_generate_type">
{code_type_options}
</select>
</section>
<section class="panel panel-wide">
<div>
<div class="panel-title">可用策略</div>
<label>允许的过期方式</label>
<div class="checks">
{expire_style_inputs}
</div>
</div>
<div>
<label for="allowed_file_types">允许文件类型 <span class="compact-help">逗号分隔,* 表示不限制</span></label>
<input id="allowed_file_types" name="allowed_file_types" value="{escaped_allowed_types}" placeholder="* 或 .zip, image/*">
</div>
</section>
</div>
<button type="submit">完成初始化</button>
</form>
</main>
</body>
</html>"""
def build_setup_success_page() -> str:
return """<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="refresh" content="2;url=/#/admin">
<title>初始化完成</title>
<style>
body {
margin: 0;
min-height: 100vh;
display: grid;
place-items: center;
padding: 24px;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: #f6f8fb;
color: #172033;
}
main {
width: min(100%, 420px);
padding: 28px;
border: 1px solid #d9e1ec;
border-radius: 8px;
background: #fff;
text-align: center;
box-shadow: 0 18px 50px rgba(23, 32, 51, .08);
}
h1 { margin: 0 0 10px; font-size: 24px; letter-spacing: 0; }
p { margin: 0 0 22px; color: #60708a; line-height: 1.65; }
a {
display: inline-flex;
align-items: center;
justify-content: center;
min-height: 42px;
padding: 0 18px;
border-radius: 6px;
background: #2563eb;
color: white;
text-decoration: none;
font-weight: 700;
}
</style>
</head>
<body>
<main>
<h1>初始化完成</h1>
<p>管理员密码已设置,请使用刚才的密码登录后台。</p>
<a href="/#/admin">进入后台</a>
</main>
</body>
</html>"""
def setup_response(content: str, status_code: int = 200) -> HTMLResponse:
return HTMLResponse(
content=content,
status_code=status_code,
media_type="text/html",
headers={"Cache-Control": "no-store"},
)
def is_setup_path(path: str) -> bool:
return path.rstrip("/") == "/setup"
def wants_html_response(request: Request) -> bool:
if request.method not in {"GET", "HEAD"}:
return False
accept = request.headers.get("accept", "")
return not accept or "text/html" in accept or "*/*" in accept
async def read_setup_payload(request: Request) -> dict:
content_type = request.headers.get("content-type", "")
if "application/json" in content_type:
data = await request.json()
return data if isinstance(data, dict) else {}
body = (await request.body()).decode("utf-8")
return {
key: values if len(values) > 1 else values[-1]
for key, values in parse_qs(body).items()
}
-132
View File
@@ -1,132 +0,0 @@
# @Time : 2023/8/15 22:00
# @Author : Lan
# @File : tasks.py
# @Software: PyCharm
import asyncio
import datetime
import os
from tortoise.expressions import Q
from apps.base.models import (
FileCodes,
PresignUploadSession,
StorageReservation,
UploadChunk,
)
from apps.base.utils import ip_limit, get_chunk_file_path_name
from apps.base.config import refresh_settings
from core.logger import logger
from core.settings import settings, data_root
from core.storage import FileStorageInterface, StoredFile, storages
from apps.base.services import stored_file_of
from apps.base.local_share import is_local_ref
from core.utils import get_now
async def delete_expire_files():
while True:
try:
await refresh_settings()
file_storage: FileStorageInterface = storages[settings.file_storage]()
# 遍历 share目录下的所有文件夹,删除空的文件夹,并判断父目录是否为空,如果为空也删除
if settings.file_storage == "local":
for root, dirs, files in os.walk(f"{data_root}/share/data"):
if not dirs and not files:
os.rmdir(root)
await ip_limit["error"].remove_expired_ip()
await ip_limit["metadata"].remove_expired_ip()
await ip_limit["upload"].remove_expired_ip()
await StorageReservation.filter(expires_at__lte=await get_now()).delete()
expire_data = await FileCodes.filter(
Q(expired_at__lt=await get_now()) | Q(expired_count=0)
).all()
for exp in expire_data:
try:
if not is_local_ref(exp):
await file_storage.delete_file(stored_file_of(exp))
except Exception as e:
logger.error(f"删除过期文件失败 code={exp.code}: {e}")
try:
await exp.delete()
except Exception as e:
logger.error(f"删除记录失败 code={exp.code}: {e}")
except Exception as e:
logger.error(e)
finally:
await asyncio.sleep(600)
async def clean_incomplete_uploads():
while True:
try:
await refresh_settings()
file_storage: FileStorageInterface = storages[settings.file_storage]()
expire_hours = getattr(settings, "chunk_expire_hours", 24)
now = await get_now()
expire_time = now - datetime.timedelta(hours=expire_hours)
expired_sessions = await UploadChunk.filter(
chunk_index=-1, created_at__lt=expire_time
).all()
for session in expired_sessions:
try:
save_path = session.save_path
if not save_path:
_, _, _, _, save_path = await get_chunk_file_path_name(
session.file_name, session.upload_id
)
await file_storage.clean_chunks(session.upload_id, save_path)
except Exception as e:
logger.error(
f"清理分片文件失败 upload_id={session.upload_id}: {e}"
)
try:
await UploadChunk.filter(upload_id=session.upload_id).delete()
await StorageReservation.filter(
token=f"chunk:{session.upload_id}"
).delete()
logger.info(f"已清理过期上传会话 upload_id={session.upload_id}")
except Exception as e:
logger.error(
f"删除分片记录失败 upload_id={session.upload_id}: {e}"
)
except Exception as e:
logger.error(f"清理未完成上传任务异常: {e}")
finally:
await asyncio.sleep(3600)
async def clean_expired_presign_sessions():
while True:
try:
await refresh_settings()
storage: FileStorageInterface = storages[settings.file_storage]()
now = await get_now()
expired_sessions = await PresignUploadSession.filter(
expires_at__lt=now
).all()
for session in expired_sessions:
if session.mode == "direct":
try:
if await storage.file_exists(session.save_path):
await storage.delete_file(
StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
)
except Exception as e:
logger.error(
"清理过期直传文件失败 "
f"upload_id={session.upload_id}: {e}"
)
await StorageReservation.filter(
token=f"presign:{session.upload_id}"
).delete()
await session.delete()
except Exception as e:
logger.error(f"清理过期预签名会话异常: {e}")
finally:
await asyncio.sleep(900)
-131
View File
@@ -1,131 +0,0 @@
import datetime
import os
import uuid
from urllib.parse import unquote
from fastapi import UploadFile, HTTPException
from typing import Optional, Tuple
from apps.base.dependencies import IPRateLimit
from apps.base.models import FileCodes
from core.settings import settings
from core.utils import (
get_random_num,
get_random_string,
max_save_times_desc,
sanitize_filename,
get_now,
)
def validate_expire_style(expire_style: str) -> str:
"""校验过期方式是否在管理员配置的白名单内。"""
if expire_style not in settings.expire_style:
raise HTTPException(status_code=400, detail="过期时间类型错误")
return expire_style
async def build_file_path(
file_name: str, file_uuid: str
) -> Tuple[str, str, str, str, str]:
"""Single source of storage path generation (date dir + UUID), shared by
regular, chunked, and presigned uploads.
Always use get_now() (UTC+8); do not switch to server-local time.
"""
today = await get_now()
storage_path = settings.storage_path.strip("/")
filename = await sanitize_filename(unquote(file_name or ""))
base_path = f"share/data/{today.strftime('%Y/%m/%d')}/{file_uuid}"
path = f"{storage_path}/{base_path}" if storage_path else base_path
prefix, suffix = os.path.splitext(filename)
save_path = f"{path}/{prefix}{suffix}"
return path, suffix, prefix, filename, save_path
async def get_file_path_name(file: UploadFile) -> Tuple[str, str, str, str, str]:
return await build_file_path(file.filename or "", uuid.uuid4().hex)
async def get_chunk_file_path_name(
file_name: str, upload_id: str
) -> Tuple[str, str, str, str, str]:
return await build_file_path(file_name, upload_id)
async def get_expire_info(
expire_value: int, expire_style: str
) -> Tuple[Optional[datetime.datetime], int, int, str]:
expired_count, used_count = -1, 0
now = await get_now()
code = None
max_timedelta = (
datetime.timedelta(seconds=settings.max_save_seconds)
if settings.max_save_seconds > 0
else datetime.timedelta(days=7)
)
detail = (
await max_save_times_desc(settings.max_save_seconds)
if settings.max_save_seconds > 0
else "7天"
)
detail = f"限制最长时间为 {detail[0]},可换用其他方式"
expire_styles = {
"day": lambda: now + datetime.timedelta(days=expire_value),
"hour": lambda: now + datetime.timedelta(hours=expire_value),
"minute": lambda: now + datetime.timedelta(minutes=expire_value),
"count": lambda: (now + datetime.timedelta(days=1), expire_value),
"forever": lambda: (None, None),
}
if expire_style in expire_styles:
result = expire_styles[expire_style]()
if isinstance(result, tuple):
expired_at, extra = result
if expire_style == "count":
expired_count = extra
else:
expired_at = result
if expired_at and expired_at - now > max_timedelta:
raise HTTPException(status_code=403, detail=detail)
else:
expired_at = now + datetime.timedelta(days=1)
if not code:
code = await get_random_code()
return expired_at, expired_count, used_count, code
def get_code_generate_type() -> str:
code_generate_type = getattr(settings, "code_generate_type", "secret")
if code_generate_type in {"secret", "string"}:
return "secret"
return "number"
async def get_random_code(style: str | None = None) -> str:
code_style = style or get_code_generate_type()
if code_style == "num":
code_style = "number"
if code_style == "string":
code_style = "secret"
while True:
code = (
await get_random_num()
if code_style == "number"
else await get_random_string()
)
if not await FileCodes.filter(code=code).exists():
return str(code)
ip_limit = {
"error": IPRateLimit(count=settings.error_count, minutes=settings.error_minute),
"metadata": IPRateLimit(count=settings.error_count, minutes=settings.error_minute),
"upload": IPRateLimit(count=settings.upload_count, minutes=settings.upload_minute),
"login": IPRateLimit(count=settings.login_count, minutes=settings.login_minute),
}
-694
View File
@@ -1,694 +0,0 @@
import hashlib
import hmac
import os
import uuid
from datetime import timedelta
from urllib.parse import quote, unquote
from typing import Optional, Tuple, Union
from fastapi import APIRouter, Form, Request, UploadFile, File, Depends, HTTPException
from pydantic import BaseModel, ValidationError
from starlette import status
from starlette.responses import Response
from tortoise.expressions import Case, F, Q, When
from apps.base.auth import share_required_login
from apps.base.models import FileCodes, UploadChunk, PresignUploadSession
from apps.base.quota import release_storage, reserve_storage
from apps.base.services import (
PRESIGN_SESSION_EXPIRES,
FileUploadService,
get_stored_download,
response_from_download,
stored_file_of,
validate_file_size,
)
from core.storage import StoredFile
from core.logger import logger
from apps.base.schemas import (
SelectFileModel,
InitChunkUploadModel,
CompleteUploadModel,
PresignUploadInitRequest,
)
from apps.base.file_validation import validate_file_type, validate_upload_file, validate_header_bytes
from apps.base.utils import (
ip_limit,
get_chunk_file_path_name,
validate_expire_style,
)
from apps.base.local_share import is_local_ref
from core.response import APIResponse
from core.settings import settings
from core.storage import storages, FileStorageInterface
from core.utils import (
get_file_url as get_proxy_file_url,
get_select_token,
get_now,
sanitize_filename,
)
share_api = APIRouter(prefix="/share", tags=["分享"])
# ============ 公共服务层 ============
def normalize_share_code(code: str) -> str:
return str(code or "").strip()
@share_api.post("/text/", dependencies=[Depends(share_required_login)])
async def share_text(
text: str = Form(...),
expire_value: int = Form(default=1, gt=0),
expire_style: str = Form(default="day"),
ip: str = Depends(ip_limit["upload"]),
):
validate_expire_style(expire_style)
text_size = len(text.encode("utf-8"))
max_txt_size = 222 * 1024
if text_size > max_txt_size:
raise HTTPException(status_code=403, detail="内容过多,建议采用文件形式")
code = await FileUploadService.create_text_share(text, expire_value, expire_style)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code})
@share_api.post("/file/", dependencies=[Depends(share_required_login)])
async def share_file(
expire_value: int = Form(default=1, gt=0),
expire_style: str = Form(default="day"),
file: UploadFile = File(...),
ip: str = Depends(ip_limit["upload"]),
):
file_size = await validate_file_size(file, settings.upload_size)
await validate_upload_file(file)
validate_expire_style(expire_style)
detail = await FileUploadService.create_file_share(
file, size=file_size, expire_value=expire_value, expire_style=expire_style
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail=detail)
async def get_code_file_by_code(
code: str, check: bool = True
) -> Tuple[bool, Union[FileCodes, str]]:
normalized_code = normalize_share_code(code)
if not normalized_code:
return False, "文件不存在"
file_code = await FileCodes.filter(code=normalized_code).first()
if not file_code:
return False, "文件不存在"
if await file_code.is_expired() and check:
return False, "文件已过期"
return True, file_code
async def consume_file_usage(file_code: FileCodes) -> bool:
"""原子校验分享状态并记录一次实际领取。"""
now = await get_now()
eligible = (
Q(expired_count__gt=0)
| Q(expired_count__lt=0, expired_at__gt=now)
| Q(expired_count__lt=0, expired_at=None)
)
updated = await FileCodes.filter(Q(id=file_code.id) & eligible).update(
expired_count=Case(
When(expired_count__gt=0, then=F("expired_count") - 1),
default=F("expired_count"),
),
used_count=F("used_count") + 1,
)
if not updated:
return False
await file_code.refresh_from_db()
return True
def build_file_metadata(file_code: FileCodes) -> dict:
is_text = file_code.text is not None
remaining_downloads = (
file_code.expired_count if file_code.expired_count > 0 else None
)
return {
"code": file_code.code,
"name": file_code.prefix + file_code.suffix,
"size": file_code.size,
"type": "text" if is_text else "file",
"is_text": is_text,
"created_at": file_code.created_at,
"expired_at": file_code.expired_at,
"expired_count": file_code.expired_count,
"used_count": file_code.used_count,
"remaining_downloads": remaining_downloads,
}
async def build_select_detail(
file_code: FileCodes, file_storage: FileStorageInterface
) -> dict:
metadata = build_file_metadata(file_code)
if file_code.text is not None:
download_url = None
elif file_code.expired_count >= 0 or is_local_ref(file_code):
# 次数限制与 NAS 引用都必须走下载接口,避免直链绕过次数或打到错误存储。
download_url = await get_proxy_file_url(file_code.code)
else:
download_url = await file_storage.get_file_url(stored_file_of(file_code))
content = file_code.text if file_code.text is not None else None
return {
**metadata,
"text": content if content is not None else download_url,
"content": content,
"download_url": download_url,
}
@share_api.get("/metadata/")
async def get_file_metadata(code: str, ip: str = Depends(ip_limit["metadata"])):
has, file_code = await get_code_file_by_code(code)
if not has:
ip_limit["metadata"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
ip_limit["metadata"].add_ip(ip)
return APIResponse(detail=build_file_metadata(file_code))
@share_api.post("/metadata/")
async def post_file_metadata(
data: SelectFileModel, ip: str = Depends(ip_limit["metadata"])
):
has, file_code = await get_code_file_by_code(data.code)
if not has:
ip_limit["metadata"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
ip_limit["metadata"].add_ip(ip)
return APIResponse(detail=build_file_metadata(file_code))
@share_api.get("/select/")
async def get_code_file(code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
has, file_code = await get_code_file_by_code(code)
if not has:
ip_limit["error"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
if not await consume_file_usage(file_code):
return APIResponse(code=404, detail="文件已过期")
if file_code.text is not None:
filename = f"{file_code.prefix or 'Text'}{file_code.suffix or '.txt'}"
return Response(
content=file_code.text,
media_type="text/plain",
headers={
"Content-Disposition": (
f"attachment; filename*=UTF-8''{quote(filename, safe='')}"
)
},
)
return response_from_download(await get_stored_download(file_code, file_storage))
@share_api.post("/select/")
async def select_file(data: SelectFileModel, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
has, file_code = await get_code_file_by_code(data.code)
if not has:
ip_limit["error"].add_ip(ip)
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
detail = await build_select_detail(file_code, file_storage)
download_url = detail.get("download_url")
consumes_on_download = isinstance(download_url, str) and download_url.startswith(
"/share/download?"
)
if not consumes_on_download and not await consume_file_usage(file_code):
return APIResponse(code=404, detail="文件已过期")
if not consumes_on_download:
detail.update(build_file_metadata(file_code))
return APIResponse(detail=detail)
@share_api.get("/download")
async def download_file(key: str, code: str, ip: str = Depends(ip_limit["error"])):
file_storage: FileStorageInterface = storages[settings.file_storage]()
normalized_code = normalize_share_code(code)
# 同时接受当前窗口与上一窗口 token,避免时间窗边界竞态导致偶发 403。
# 逐个常量时间比较(hmac.compare_digest),避免 set 成员判断的时序侧信道。
valid_keys = [
await get_select_token(normalized_code, offset=0),
await get_select_token(normalized_code, offset=1),
]
if not any(
hmac.compare_digest(key.encode(), candidate.encode())
for candidate in valid_keys
):
ip_limit["error"].add_ip(ip)
raise HTTPException(status_code=403, detail="下载鉴权失败")
has, file_code = await get_code_file_by_code(normalized_code)
if not has:
return APIResponse(code=404, detail=file_code)
assert isinstance(file_code, FileCodes)
if not await consume_file_usage(file_code):
return APIResponse(code=404, detail="文件已过期")
return (
APIResponse(detail=file_code.text)
if file_code.text
else response_from_download(await get_stored_download(file_code, file_storage))
)
chunk_api = APIRouter(prefix="/chunk", tags=["切片"])
async def parse_body_model(request: Request, model_class: type[BaseModel]):
content_type = request.headers.get("content-type", "").lower()
try:
if "application/x-www-form-urlencoded" in content_type or "multipart/form-data" in content_type:
payload = dict(await request.form())
else:
payload = await request.json()
return model_class.model_validate(payload)
except ValidationError as e:
raise HTTPException(status_code=422, detail=e.errors())
except Exception:
raise HTTPException(status_code=400, detail="请求体格式错误")
async def parse_init_chunk_upload(request: Request) -> InitChunkUploadModel:
return await parse_body_model(request, InitChunkUploadModel)
async def parse_complete_upload(request: Request) -> CompleteUploadModel:
return await parse_body_model(request, CompleteUploadModel)
@chunk_api.post("/upload/init/", dependencies=[Depends(share_required_login)])
async def init_chunk_upload(data: InitChunkUploadModel = Depends(parse_init_chunk_upload)):
safe_file_name = await sanitize_filename(unquote(data.file_name or ""))
validate_file_type(safe_file_name)
# 服务端校验:根据 total_chunks * chunk_size 计算理论最大上传量
total_chunks = (data.file_size + data.chunk_size - 1) // data.chunk_size
max_possible_size = total_chunks * data.chunk_size
if max_possible_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"文件大小超过限制,最大为 {max_size_mb:.2f} MB"
)
# 断点续传:检查是否存在相同文件的未完成上传会话
existing_session = await UploadChunk.filter(
chunk_hash=data.file_hash,
chunk_index=-1,
file_size=data.file_size,
file_name=safe_file_name,
).first()
if existing_session:
if not existing_session.save_path:
await UploadChunk.filter(upload_id=existing_session.upload_id).delete()
await release_storage(f"chunk:{existing_session.upload_id}")
else:
await reserve_storage(
f"chunk:{existing_session.upload_id}",
existing_session.file_size,
ttl_seconds=max(
1, int(getattr(settings, "chunk_expire_hours", 24))
)
* 3600,
)
uploaded_chunks = await UploadChunk.filter(
upload_id=existing_session.upload_id, completed=True
).values_list("chunk_index", flat=True)
return APIResponse(
detail={
"existed": False,
"upload_id": existing_session.upload_id,
"chunk_size": existing_session.chunk_size,
"total_chunks": existing_session.total_chunks,
"uploaded_chunks": list(uploaded_chunks),
}
)
# 创建新的上传会话
upload_id = uuid.uuid4().hex
reservation_token = f"chunk:{upload_id}"
chunk_expire_seconds = max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600
await reserve_storage(
reservation_token, data.file_size, ttl_seconds=chunk_expire_seconds
)
_, _, _, safe_file_name, save_path = await get_chunk_file_path_name(
data.file_name, upload_id
)
try:
await UploadChunk.create(
upload_id=upload_id,
chunk_index=-1,
total_chunks=total_chunks,
file_size=data.file_size,
chunk_size=data.chunk_size,
chunk_hash=data.file_hash,
file_name=safe_file_name,
save_path=save_path,
)
except Exception:
await release_storage(reservation_token)
raise
return APIResponse(
detail={
"existed": False,
"upload_id": upload_id,
"chunk_size": data.chunk_size,
"total_chunks": total_chunks,
"uploaded_chunks": [],
}
)
@chunk_api.post(
"/upload/chunk/{upload_id}/{chunk_index}",
dependencies=[Depends(share_required_login)],
)
async def upload_chunk(
upload_id: str,
chunk_index: int,
chunk: UploadFile = File(...),
):
# 获取上传会话信息
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
await reserve_storage(
f"chunk:{upload_id}",
chunk_info.file_size,
ttl_seconds=max(1, int(getattr(settings, "chunk_expire_hours", 24))) * 3600,
)
# 检查分片索引有效性
if chunk_index < 0 or chunk_index >= chunk_info.total_chunks:
raise HTTPException(status.HTTP_400_BAD_REQUEST, detail="无效的分片索引")
# 检查是否已上传(支持断点续传)
existing_chunk = await UploadChunk.filter(
upload_id=upload_id, chunk_index=chunk_index, completed=True
).first()
if existing_chunk:
return APIResponse(
detail={"chunk_hash": existing_chunk.chunk_hash, "skipped": True}
)
# 读取分片数据并计算哈希
chunk_data = await chunk.read()
if chunk_index == 0:
validate_header_bytes(chunk_info.file_name, None, chunk_data[:64])
chunk_size = len(chunk_data)
# 校验分片大小不超过声明的 chunk_size
if chunk_size > chunk_info.chunk_size:
raise HTTPException(
status.HTTP_400_BAD_REQUEST,
detail=f"分片大小超过声明值: 最大 {chunk_info.chunk_size}, 实际 {chunk_size}",
)
# 计算已上传分片数,校验累计大小不超限(用分片数 * chunk_size 估算)
uploaded_count = await UploadChunk.filter(
upload_id=upload_id, completed=True
).count()
# 已上传分片的最大可能大小 + 当前分片
max_uploaded_size = uploaded_count * chunk_info.chunk_size + chunk_size
if max_uploaded_size > settings.upload_size:
max_size_mb = settings.upload_size / (1024 * 1024)
raise HTTPException(
status_code=403, detail=f"累计上传大小超过限制,最大为 {max_size_mb:.2f} MB"
)
chunk_hash = hashlib.sha256(chunk_data).hexdigest()
save_path = chunk_info.save_path
# 保存分片到存储
storage = storages[settings.file_storage]()
try:
await storage.save_chunk(
upload_id, chunk_index, chunk_data, chunk_hash, save_path
)
except Exception as e:
raise HTTPException(
status.HTTP_500_INTERNAL_SERVER_ERROR, detail=f"分片保存失败: {str(e)}"
)
# 更新或创建分片记录(保存成功后再记录)
await UploadChunk.update_or_create(
upload_id=upload_id,
chunk_index=chunk_index,
defaults={
"chunk_hash": chunk_hash,
"completed": True,
"file_size": chunk_info.file_size,
"total_chunks": chunk_info.total_chunks,
"chunk_size": chunk_info.chunk_size,
"file_name": chunk_info.file_name,
"save_path": chunk_info.save_path,
},
)
return APIResponse(detail={"chunk_hash": chunk_hash})
@chunk_api.delete("/upload/{upload_id}", dependencies=[Depends(share_required_login)])
async def cancel_upload(upload_id: str):
"""取消上传并清理临时文件"""
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
save_path = chunk_info.save_path
# 清理存储中的临时文件
storage = storages[settings.file_storage]()
if save_path:
try:
await storage.clean_chunks(upload_id, save_path)
except Exception:
logger.warning("取消分片上传:清理分片文件失败 upload_id=%s", upload_id, exc_info=True)
# 清理数据库记录
await UploadChunk.filter(upload_id=upload_id).delete()
await release_storage(f"chunk:{upload_id}")
return APIResponse(detail={"message": "上传已取消"})
@chunk_api.get(
"/upload/status/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def get_upload_status(upload_id: str):
"""获取上传状态"""
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
# 获取已上传的分片列表
uploaded_chunks = await UploadChunk.filter(
upload_id=upload_id, completed=True
).values_list("chunk_index", flat=True)
return APIResponse(
detail={
"upload_id": upload_id,
"file_name": chunk_info.file_name,
"file_size": chunk_info.file_size,
"chunk_size": chunk_info.chunk_size,
"total_chunks": chunk_info.total_chunks,
"uploaded_chunks": list(uploaded_chunks),
"progress": len(uploaded_chunks) / chunk_info.total_chunks * 100,
}
)
@chunk_api.post(
"/upload/complete/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def complete_upload(
upload_id: str,
data: CompleteUploadModel = Depends(parse_complete_upload),
ip: str = Depends(ip_limit["upload"]),
):
# 获取上传基本信息
chunk_info = await UploadChunk.filter(upload_id=upload_id, chunk_index=-1).first()
if not chunk_info:
raise HTTPException(status.HTTP_404_NOT_FOUND, detail="上传会话不存在")
validate_expire_style(data.expire_style)
detail = await FileUploadService.complete_chunked_upload(
upload_id, chunk_info, data.expire_value, data.expire_style
)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail=detail)
# ============ 预签名上传API ============
presign_api = APIRouter(prefix="/presign", tags=["预签名上传"])
def build_proxy_upload_urls(upload_id: str) -> dict:
proxy_upload_url = f"/presign/upload/proxy/{upload_id}"
return {
"proxy_upload_url": proxy_upload_url,
"legacy_proxy_upload_url": f"/api{proxy_upload_url}",
}
async def _get_valid_session(
upload_id: str, expected_mode: Optional[str] = None
) -> PresignUploadSession:
"""获取并验证会话"""
session = await PresignUploadSession.filter(upload_id=upload_id).first()
if not session:
raise HTTPException(404, "上传会话不存在")
if await session.is_expired():
await session.delete()
await release_storage(f"presign:{upload_id}")
raise HTTPException(404, "上传会话已过期")
if expected_mode and session.mode != expected_mode:
raise HTTPException(400, f"此会话不支持{expected_mode}模式")
return session
@presign_api.post("/upload/init", dependencies=[Depends(share_required_login)])
async def presign_upload_init(
data: PresignUploadInitRequest, ip: str = Depends(ip_limit["upload"])
):
"""初始化预签名上传,S3返回直传URL,其他存储返回代理URL"""
validate_file_type(data.file_name)
if data.file_size > settings.upload_size:
raise HTTPException(
403,
f"文件大小超过限制,最大为 {settings.upload_size / (1024 * 1024):.2f} MB",
)
validate_expire_style(data.expire_style)
upload_id = uuid.uuid4().hex
reservation_token = f"presign:{upload_id}"
await reserve_storage(
reservation_token, data.file_size, ttl_seconds=PRESIGN_SESSION_EXPIRES
)
try:
path, _, _, filename, save_path = await FileUploadService.generate_file_path(
data.file_name, upload_id
)
storage: FileStorageInterface = storages[settings.file_storage]()
presigned_url = await storage.generate_presigned_upload_url(
save_path, PRESIGN_SESSION_EXPIRES
)
mode = "direct" if presigned_url else "proxy"
proxy_urls = build_proxy_upload_urls(upload_id)
upload_url = presigned_url or proxy_urls["proxy_upload_url"]
await PresignUploadSession.create(
upload_id=upload_id,
file_name=filename,
file_size=data.file_size,
save_path=save_path,
mode=mode,
expire_value=data.expire_value,
expire_style=data.expire_style,
expires_at=await get_now() + timedelta(seconds=PRESIGN_SESSION_EXPIRES),
)
except Exception:
await release_storage(reservation_token)
raise
ip_limit["upload"].add_ip(ip)
detail = {
"upload_id": upload_id,
"upload_url": upload_url,
"mode": mode,
"expires_in": PRESIGN_SESSION_EXPIRES,
}
if mode == "proxy":
detail.update(proxy_urls)
return APIResponse(
detail=detail
)
@presign_api.put(
"/upload/proxy/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def presign_upload_proxy(
upload_id: str, file: UploadFile = File(...), ip: str = Depends(ip_limit["upload"])
):
"""代理模式上传,服务器转存到存储后端"""
session = await _get_valid_session(upload_id, expected_mode="proxy")
code = await FileUploadService.commit_proxy_upload(session, file)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@presign_api.post(
"/upload/confirm/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def presign_upload_confirm(upload_id: str, ip: str = Depends(ip_limit["upload"])):
"""直传确认,客户端完成S3直传后调用获取分享码"""
session = await _get_valid_session(upload_id, expected_mode="direct")
code = await FileUploadService.confirm_direct_upload(session)
ip_limit["upload"].add_ip(ip)
return APIResponse(detail={"code": code, "name": session.file_name})
@presign_api.get(
"/upload/status/{upload_id}", dependencies=[Depends(share_required_login)]
)
async def presign_upload_status(upload_id: str):
"""查询上传会话状态"""
session = await PresignUploadSession.filter(upload_id=upload_id).first()
if not session:
raise HTTPException(404, "上传会话不存在")
return APIResponse(
detail={
"upload_id": session.upload_id,
"file_name": session.file_name,
"file_size": session.file_size,
"mode": session.mode,
"created_at": session.created_at.isoformat(),
"expires_at": session.expires_at.isoformat(),
"is_expired": await session.is_expired(),
}
)
@presign_api.delete("/upload/{upload_id}", dependencies=[Depends(share_required_login)])
async def presign_upload_cancel(upload_id: str):
"""取消上传会话"""
session = await PresignUploadSession.filter(upload_id=upload_id).first()
if not session:
raise HTTPException(404, "上传会话不存在")
if session.mode == "direct":
storage: FileStorageInterface = storages[settings.file_storage]()
try:
if await storage.file_exists(session.save_path):
temp_file_code = StoredFile(
file_path=os.path.dirname(session.save_path),
uuid_file_name=os.path.basename(session.save_path),
)
await storage.delete_file(temp_file_code)
except Exception:
logger.warning("取消预签名会话:清理临时文件失败 upload_id=%s", upload_id, exc_info=True)
await session.delete()
await release_storage(f"presign:{upload_id}")
return APIResponse(detail={"message": "上传会话已取消"})
-4
View File
@@ -1,4 +0,0 @@
# @Time : 2023/8/11 20:06
# @Author : Lan
# @File : __init__.py.py
# @Software: PyCharm
+45 -145
View File
@@ -1,157 +1,57 @@
import asyncio
import glob
import importlib
import os
from contextlib import asynccontextmanager
from typing import IO
import datetime
from tortoise import Tortoise
from sqlalchemy import Boolean, Column, Integer, String, DateTime, JSON, Text, select, insert, delete
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.ext.asyncio.session import AsyncSession
from core.logger import logger
from core.settings import data_root
from settings import settings
engine = create_async_engine(settings.DATABASE_URL)
Base = declarative_base()
_DB_FILE = os.path.join(data_root, "filecodebox.db")
_STARTUP_LOCK_FILE = os.path.join(data_root, "filecodebox.startup.lock")
class Options(Base):
__tablename__ = 'options'
id = Column(Integer, primary_key=True, index=True)
key = Column(String, unique=True, index=True)
value = Column(JSON)
def get_db_config() -> dict:
return {
"connections": {
"default": {
"engine": "tortoise.backends.sqlite",
"credentials": {
"file_path": _DB_FILE,
"journal_mode": "WAL",
"busy_timeout": 10000,
"foreign_keys": "ON",
# 小内存 VPS 下的 IOPS 与内存折中。Tortoise 的 SqliteClient 会把
# credentials 中 file_path 以外的键逐条执行成 PRAGMA,且 SQLite
# 后端只维持单条连接,因此这些设置在进程生命周期内全局生效。
# synchronous=NORMAL:WAL 下应用崩溃不丢已提交事务,
# 仅 OS/断电可能丢最后若干事务,省去每次提交的 fsync。
# temp_store=MEMORY:临时表与排序走内存,小库收益明显;
# 后台大排序的内存峰值需在压测中关注。
# cache_size=-8000:page cache 上限约 8MB(负数单位为 KiB)。
"synchronous": "NORMAL",
"temp_store": "MEMORY",
"cache_size": -8000,
},
}
},
"apps": {
"models": {
"models": ["apps.base.models"],
"default_connection": "default",
}
},
"use_tz": False,
"timezone": "Asia/Shanghai",
}
class Codes(Base):
__tablename__ = "codes"
id = Column(Integer, primary_key=True, index=True)
code = Column(String(10), unique=True, index=True)
key = Column(String(30), unique=True)
name = Column(String(500))
size = Column(Integer)
type = Column(String(20))
text = Column(Text)
used = Column(Boolean, default=False)
count = Column(Integer, default=-1)
use_time = Column(DateTime, default=datetime.datetime.now, onupdate=datetime.datetime.now)
exp_time = Column(DateTime, nullable=True)
def _lock_file(file_obj: IO[str]) -> None:
if os.name == "nt":
import msvcrt
# Windows 需要锁定至少 1 字节
if os.fstat(file_obj.fileno()).st_size == 0:
file_obj.write("0")
file_obj.flush()
msvcrt.locking(file_obj.fileno(), msvcrt.LK_LOCK, 1) # type: ignore[attr-defined]
else:
import fcntl
fcntl.flock(file_obj.fileno(), fcntl.LOCK_EX)
async def init_models(s):
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
print(
f'初始化数据库成功!\n'
f'如您未配置.env文件,将为您随机生成信息\n'
f'您的后台地址为:/{settings.ADMIN_ADDRESS}\n'
f'您的管理员密码为:{settings.ADMIN_PASSWORD}\n'
f'请尽快修改后台信息!\n'
f'FileCodeBox https://github.com/vastsa/FileCodeBox'
)
await settings.updates(await conn.execute(select(Options).filter()))
def _unlock_file(file_obj: IO[str]) -> None:
if os.name == "nt":
import msvcrt
msvcrt.locking(file_obj.fileno(), msvcrt.LK_UNLCK, 1) # type: ignore[attr-defined]
else:
import fcntl
fcntl.flock(file_obj.fileno(), fcntl.LOCK_UN)
async def get_config(key):
async with engine.begin() as conn:
return await conn.scalar(select(Options.value).filter(Options.key == key))
@asynccontextmanager
async def db_startup_lock():
os.makedirs(data_root, exist_ok=True)
lock_file = open(_STARTUP_LOCK_FILE, "a+", encoding="utf-8")
try:
await asyncio.to_thread(_lock_file, lock_file)
yield
finally:
await asyncio.to_thread(_unlock_file, lock_file)
lock_file.close()
async def init_db():
try:
db_config = get_db_config()
if not Tortoise._inited:
await Tortoise.init(config=db_config)
async with db_startup_lock():
# 创建migrations表
await Tortoise.get_connection("default").execute_script("""
CREATE TABLE IF NOT EXISTS migrates (
id INTEGER PRIMARY KEY AUTOINCREMENT,
migration_file VARCHAR(255) NOT NULL UNIQUE,
executed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
# 执行迁移
await execute_migrations()
except Exception as e:
logger.error(f"数据库初始化失败: {str(e)}")
raise
async def execute_migrations():
"""执行数据库迁移"""
try:
# 收集迁移文件
migration_files = []
for root, dirs, files in os.walk("apps"):
if "migrations" in dirs:
migration_path = os.path.join(root, "migrations")
migration_files.extend(glob.glob(os.path.join(migration_path, "migrations_*.py")))
# 按文件名排序
migration_files.sort()
for migration_file in migration_files:
file_name = os.path.basename(migration_file)
# 检查是否已执行
executed = await Tortoise.get_connection("default").execute_query(
"SELECT id FROM migrates WHERE migration_file = ?", [file_name]
)
if not executed[1]:
logger.info(f"执行迁移: {file_name}")
# 导入并执行migration
module_path = migration_file.replace("/", ".").replace("\\", ".").replace(".py", "")
try:
migration_module = importlib.import_module(module_path)
if hasattr(migration_module, "migrate"):
await migration_module.migrate()
# 记录执行
await Tortoise.get_connection("default").execute_query(
"INSERT INTO migrates (migration_file) VALUES (?)",
[file_name]
)
logger.info(f"迁移完成: {file_name}")
except Exception as e:
logger.error(f"迁移 {file_name} 执行失败: {str(e)}")
raise
except Exception as e:
logger.error(f"迁移过程发生错误: {str(e)}")
raise
async def get_session():
async with AsyncSession(engine, expire_on_commit=False) as s:
yield s
+52
View File
@@ -0,0 +1,52 @@
from typing import Union
from datetime import datetime, timedelta
from fastapi import Header, HTTPException, Request
from settings import settings
async def admin_required(pwd: Union[str, None] = Header(default=None), request: Request = None):
if 'share' in request.url.path:
if pwd != settings.ADMIN_PASSWORD and not settings.ENABLE_UPLOAD:
raise HTTPException(status_code=403, detail='本站上传功能已关闭,仅管理员可用')
else:
if settings.ADMIN_PASSWORD is None:
raise HTTPException(status_code=404, detail='您未设置管理员密码,无法使用此功能,请更新配置文件后,重启系统')
if not pwd or pwd != settings.ADMIN_PASSWORD:
raise HTTPException(status_code=401, detail="密码错误,请重新登录")
class IPRateLimit:
def __init__(self, count, minutes):
self.ips = {}
self.count = count
self.minutes = minutes
def check_ip(self, ip):
# 检查ip是否被禁止
if ip in self.ips:
if self.ips[ip]['count'] >= self.count:
if self.ips[ip]['time'] + timedelta(minutes=self.minutes) > datetime.now():
return False
else:
self.ips.pop(ip)
return True
def add_ip(self, ip):
ip_info = self.ips.get(ip, {'count': 0, 'time': datetime.now()})
ip_info['count'] += 1
ip_info['time'] = datetime.now()
self.ips[ip] = ip_info
return ip_info['count']
async def remove_expired_ip(self):
for ip in list(self.ips.keys()):
if self.ips[ip]['time'] + timedelta(minutes=self.minutes) < datetime.now():
self.ips.pop(ip)
def __call__(self, request: Request):
ip = request.headers.get('X-Real-IP', request.headers.get('X-Forwarded-For', request.client.host))
if not self.check_ip(ip):
raise HTTPException(status_code=400, detail=f"请求次数过多,请稍后再试")
return ip
-13
View File
@@ -1,13 +0,0 @@
"""Framework-neutral errors raised by storage backends.
main.py registers an exception handler that renders these exactly like
FastAPI's HTTPException responses ({"detail": ...} with the same status
code), so callers need no per-site wrapping while core/ stays framework-free.
"""
class StorageError(Exception):
def __init__(self, status_code: int, detail: str):
self.status_code = status_code
self.detail = detail
super().__init__(detail)
-62
View File
@@ -1,62 +0,0 @@
import logging
import os
import sys
PRODUCTION_ENVIRONMENTS = {"production", "prod"}
LOG_LEVELS = {
"debug": logging.DEBUG,
"info": logging.INFO,
"warning": logging.WARNING,
"error": logging.ERROR,
"critical": logging.CRITICAL,
}
def is_production_environment() -> bool:
environment = os.getenv("APP_ENV", "development").strip().lower()
return environment in PRODUCTION_ENVIRONMENTS
def get_log_level_name() -> str:
configured_level = os.getenv("LOG_LEVEL", "").strip().lower()
if configured_level in LOG_LEVELS:
return configured_level
return "warning" if is_production_environment() else "info"
def get_log_level() -> int:
return LOG_LEVELS[get_log_level_name()]
def is_access_log_enabled() -> bool:
configured_value = os.getenv("ACCESS_LOG")
if configured_value is not None:
return configured_value.strip().lower() in {"1", "true", "yes", "on"}
return not is_production_environment()
def setup_logger():
_logger = logging.getLogger("FileCodeBox")
level = get_log_level()
_logger.setLevel(level)
_logger.propagate = False
if not _logger.handlers:
console_handler = logging.StreamHandler(sys.stdout)
_logger.addHandler(console_handler)
else:
console_handler = _logger.handlers[0]
console_handler.setLevel(level)
formatter = logging.Formatter(
"%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
console_handler.setFormatter(formatter)
return _logger
# 创建全局logger实例
logger = setup_logger()
-11
View File
@@ -1,11 +0,0 @@
from typing import Generic, Optional, TypeVar
from pydantic import BaseModel
T = TypeVar("T")
class APIResponse(BaseModel, Generic[T]):
code: int = 200
message: str = "ok"
detail: Optional[T] = None
-172
View File
@@ -1,172 +0,0 @@
import copy
import ipaddress
import os
import secrets
import socket
from dataclasses import dataclass
from typing import Any
from urllib.parse import urlsplit
from core.utils import hash_password, is_password_hashed, verify_password
JWT_SECRET_MIN_LENGTH = 32
INTERNAL_CONFIG_KEYS = {"jwt_secret"}
LEGACY_DEFAULT_ADMIN_TOKEN = "FileCodeBox" + "2023"
@dataclass
class SecurityConfigResult:
config: dict[str, Any]
changed: bool = False
password_hashed: bool = False
jwt_secret_rotated: bool = False
setup_required: bool = False
def generate_jwt_secret() -> str:
return secrets.token_urlsafe(48)
def is_valid_jwt_secret(secret: Any) -> bool:
return isinstance(secret, str) and len(secret) >= JWT_SECRET_MIN_LENGTH
def is_config_initialized(config: dict[str, Any]) -> bool:
"""Cheap initialization probe — runs on EVERY request via the middleware.
Must never run a slow hash: a scrypt-stored token cannot be the legacy
default (which only ever existed as plaintext or sha256), so those tokens
are initialized without any verification. Only plaintext/sha256 tokens go
through the legacy-default comparison, which is microseconds.
"""
admin_token = str(config.get("admin_token") or "")
if not admin_token:
return False
if is_password_hashed(admin_token) and admin_token.startswith("scrypt$"):
return True
return not verify_password(LEGACY_DEFAULT_ADMIN_TOKEN, admin_token)
def prepare_security_config(config: dict[str, Any]) -> SecurityConfigResult:
next_config = copy.deepcopy(config)
result = SecurityConfigResult(config=next_config)
admin_token = str(next_config.get("admin_token") or "")
if not admin_token:
result.setup_required = True
elif verify_password(LEGACY_DEFAULT_ADMIN_TOKEN, admin_token):
next_config["admin_token"] = ""
result.setup_required = True
result.changed = True
elif not is_password_hashed(admin_token):
next_config["admin_token"] = hash_password(admin_token)
result.password_hashed = True
result.changed = True
jwt_secret = next_config.get("jwt_secret")
if not is_valid_jwt_secret(jwt_secret):
next_config["jwt_secret"] = generate_jwt_secret()
result.jwt_secret_rotated = True
result.changed = True
return result
# 出站端点白名单/黑名单(SSRF 防护)。
# 这些配置项会被服务器直接当作上游地址访问:被劫持的管理员会话可以把实例
# 变成内网代理跳板,或把数据发往任意地址。校验只在写入入口(update_config),
# 已存值不受影响;APP_ENV 非 production(本地开发,如 minio/webdav)时放行。
OUTBOUND_ENDPOINT_CONFIG_KEYS = ("s3_endpoint_url", "s3_hostname", "webdav_url")
_ENDPOINT_SCHEME_WHITELIST = {"http", "https"}
_ENDPOINT_HOST_DENY_SUFFIXES = (
".local",
".internal",
".lan",
".home.arpa",
".corp",
".localhost",
)
def _endpoint_ip_is_denied(ip_str: str) -> bool:
try:
ip = ipaddress.ip_address(ip_str)
except ValueError:
return False
return (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_reserved
or ip.is_multicast
or ip.is_unspecified
)
def _endpoint_host_is_denied(host: str) -> bool:
host = host.strip("[]").lower()
if host in {"localhost"} or host.endswith(_ENDPOINT_HOST_DENY_SUFFIXES):
return True
# 字面 IP 直接判;非字面形态(速记 "127.1"、十进制 2130706433、十六进制、
# *.localhost、nip.io 类 DNS 映射)交给解析后逐 IP 复判——静态字符串
# 黑名单对它们全部失效,这是探测中实测确认的绕过面。
if _endpoint_ip_is_denied(host):
return True
try:
infos = socket.getaddrinfo(host, None)
except (socket.gaierror, UnicodeError, OSError):
return False
for info in infos:
# getaddrinfo 的 sockaddr 首元素在 IPv4/IPv6 下均为 str(int 仅出现在
# AF_UNIX 等异形族)——显式 str() 让类型检查器与运行时一致。
if _endpoint_ip_is_denied(str(info[4][0])):
return True
return False
def validate_outbound_endpoint(value: Any) -> str:
"""校验完整 URL 形态的出站端点(s3_endpoint_url / webdav_url)。"""
endpoint = str(value or "").strip()
if not endpoint:
return ""
if os.environ.get("APP_ENV", "development") != "production":
return endpoint
parts = urlsplit(endpoint)
if parts.scheme.lower() not in _ENDPOINT_SCHEME_WHITELIST:
raise ValueError(f"端点协议必须是 http(s):{endpoint}")
host = parts.hostname or ""
if not host:
raise ValueError(f"端点缺少主机名:{endpoint}")
if _endpoint_host_is_denied(host):
raise ValueError(f"端点不允许指向内网/保留地址:{endpoint}")
return endpoint
def validate_outbound_hostname(value: Any) -> str:
"""校验裸主机名形态的出站端点(s3_hostname,存储层拼接 https://{hostname})。"""
hostname = str(value or "").strip()
if not hostname:
return ""
if os.environ.get("APP_ENV", "development") != "production":
return hostname
if "://" in hostname or "/" in hostname or "@" in hostname:
raise ValueError(f"s3_hostname 应为裸主机名,不含协议、路径或 userinfo:{hostname}")
# 取主机部分:[v6] 形态取括号内;host:port 取冒号前;裸 IPv6(多个冒号)整体
if "[" in hostname:
host = hostname.split("[", 1)[1].split("]", 1)[0]
rest = hostname.split("]", 1)[1] if "]" in hostname else ""
if rest and not rest.startswith(":"):
raise ValueError(f"s3_hostname 的 IPv6 括号后只允许端口:{hostname}")
elif hostname.count(":") == 1:
host = hostname.split(":")[0]
else:
host = hostname
if not host or _endpoint_host_is_denied(host):
raise ValueError(f"端点不允许指向内网/保留地址:{hostname}")
return hostname
-126
View File
@@ -1,126 +0,0 @@
from typing import Any
# @Time : 2023/8/15 09:51
# @Author : Lan
# @File : settings.py
# @Software: PyCharm
from pathlib import Path
ADMIN_SESSION_EXPIRE_DEFAULT = 30 * 24 * 60 * 60
ADMIN_SESSION_EXPIRE_MIN = 24 * 60 * 60
ADMIN_SESSION_EXPIRE_MAX = 365 * 24 * 60 * 60
BASE_DIR = Path(__file__).resolve().parent.parent
data_root = BASE_DIR / "data"
if not data_root.exists():
data_root.mkdir(parents=True, exist_ok=True)
DEFAULT_CONFIG: dict[str, Any] = {
"file_storage": "local",
"storage_path": "",
"storage_limit": 0,
"name": "文件快递柜 - FileCodeBox",
"description": "开箱即用的文件快传系统",
"notify_title": "系统通知",
"notify_content": '欢迎使用 FileCodeBox,本程序开源于 <a href="https://github.com/vastsa/FileCodeBox" target="_blank">Github</a> ,欢迎Star和Fork。',
"page_explain": "请勿上传或分享违法内容。根据《中华人民共和国网络安全法》、《中华人民共和国刑法》、《中华人民共和国治安管理处罚法》等相关规定。 传播或存储违法、违规内容,会受到相关处罚,严重者将承担刑事责任。本站坚决配合相关部门,确保网络内容的安全,和谐,打造绿色网络环境。",
"keywords": "FileCodeBox, 文件快递柜, 口令传送箱, 匿名口令分享文本, 文件",
"s3_access_key_id": "",
"s3_secret_access_key": "",
"s3_bucket_name": "",
"s3_endpoint_url": "",
"s3_region_name": "auto",
"s3_signature_version": "s3v2",
"s3_hostname": "",
"s3_addressing_style": "auto",
"s3_proxy": 0,
"max_save_seconds": 0,
"aws_session_token": "",
"onedrive_domain": "",
"onedrive_client_id": "",
"onedrive_username": "",
"onedrive_password": "",
"onedrive_root_path": "filebox_storage",
"onedrive_proxy": 0,
"webdav_root_path": "filebox_storage",
"webdav_proxy": 0,
"admin_token": "",
"jwt_secret": "",
"admin_session_expire": ADMIN_SESSION_EXPIRE_DEFAULT,
"open_upload": 1,
"mcp_require_token": 0,
"upload_size": 1024 * 1024 * 10,
"allowed_file_types": ["*"],
"expire_style": ["day", "hour", "minute", "forever", "count"],
"code_generate_type": "secret",
"upload_minute": 1,
"enable_chunk": 0,
"webdav_url": "",
"webdav_password": "",
"webdav_username": "",
"opacity": 0.9,
"background": "",
"upload_count": 10,
"themes_choices": [
{
"name": "2023",
"key": "themes/2023",
"author": "Lan",
"version": "1.0",
},
{
"name": "2024",
"key": "themes/2024",
"author": "Lan",
"version": "1.0",
},
],
"themes_select": "themes/2024",
"error_minute": 1,
"error_count": 10,
"login_count": 5,
"login_minute": 15,
"server_workers": 1,
"server_host": "0.0.0.0",
"server_port": 12345,
"show_admin_addr": 0,
"robots_text": "User-agent: *\nDisallow: /",
"trusted_proxies": [],
"chunk_expire_hours": 24,
"opendal_scheme": "s3",
}
class Settings:
def __init__(self, defaults=None):
self.default_config = defaults or {}
self.user_config = {}
def __getattr__(self, attr):
if attr in self.user_config:
return self.user_config[attr]
if attr in self.default_config:
return self.default_config[attr]
raise AttributeError(
f"'{self.__class__.__name__}' object has no attribute '{attr}'"
)
def __setattr__(self, key, value):
if key in ["default_config", "user_config"]:
super().__setattr__(key, value)
else:
self.user_config[key] = value
def unknown_keys(self, config: dict) -> list[str]:
"""Keys in `config` that DEFAULT_CONFIG does not define.
Unknown keys silently fall through __getattr__ and crash later at the
usage site; callers (refresh_settings) surface them at load time.
"""
return sorted(k for k in config if k not in self.default_config)
def items(self):
return {**self.default_config, **self.user_config}.items()
settings = Settings(DEFAULT_CONFIG)
+140
View File
@@ -0,0 +1,140 @@
import os
import asyncio
from datetime import datetime
from pathlib import Path
from typing import BinaryIO
from fastapi import UploadFile
from core.database import Codes
from settings import settings
if settings.STORAGE_ENGINE == 'aliyunsystem':
try:
import oss2
except ImportError:
os.system('pip install oss2')
import oss2
class AliyunFileStorage:
def __init__(self):
auth = oss2.Auth(settings.KeyId, settings.KeySecret)
self.bucket = oss2.Bucket(auth, settings.OSS_ENDPOINT, settings.BUCKET_NAME)
def upload_file(self, upload_filepath, remote_filepath):
self.bucket.put_object_from_file(remote_filepath, upload_filepath)
async def get_text(self, file: UploadFile, key: str):
ext = file.filename.split('.')[-1]
now = datetime.now()
path = f"FileCodeBox/upload/{now.year}/{now.month}/{now.day}"
text = f"{path}/{f'{key}.{ext}'}"
return f"https://{settings.BUCKET_NAME}.{settings.OSS_ENDPOINT}/{text}"
async def get_url(self, info: Codes):
text = info.text.strip(f"https://{settings.BUCKET_NAME}.{settings.OSS_ENDPOINT}/")
url = self.bucket.sign_url('GET', text, settings.ACCESSTIME, slash_safe=True)
return url
@staticmethod
async def get_size(file: UploadFile):
f = file.file
f.seek(0, os.SEEK_END)
size = f.tell()
f.seek(0, os.SEEK_SET)
return size
@staticmethod
def _save(filepath, file: BinaryIO):
with open(filepath, 'wb') as f:
chunk_size = 256 * 1024
chunk = file.read(chunk_size)
while chunk:
f.write(chunk)
chunk = file.read(chunk_size)
async def save_file(self, file: UploadFile, remote_filepath: str):
now = int(datetime.now().timestamp())
upload_filepath = settings.DATA_ROOT + str(now)
await asyncio.to_thread(self._save, upload_filepath, file.file)
self.upload_file(upload_filepath, remote_filepath)
remote_filepath = remote_filepath.strip(f"https://{settings.BUCKET_NAME}.{settings.OSS_ENDPOINT}/")
self.upload_file(upload_filepath, remote_filepath)
await asyncio.to_thread(os.remove, upload_filepath)
async def delete_files(self, texts):
tasks = [self.delete_file(text) for text in texts]
await asyncio.gather(*tasks)
async def delete_file(self, text: str):
text = text.strip(f"https://{settings.BUCKET_NAME}.{settings.OSS_ENDPOINT}/")
self.bucket.delete_object(text)
class FileSystemStorage:
def __init__(self):
self.DATA_ROOT = Path(settings.DATA_ROOT)
self.STATIC_URL = settings.STATIC_URL
self.NAME = "filesystem"
self.DOWN_PATH = '/select'
async def get_filepath(self, text: str):
return self.DATA_ROOT / text.lstrip(self.STATIC_URL + '/')
async def get_url(self, info: Codes):
return f'{self.DOWN_PATH}?code={info.code}'
async def get_text(self, file: UploadFile, key: str):
ext = file.filename.split('.')[-1]
now = datetime.now()
path = self.DATA_ROOT / f"upload/{now.year}/{now.month}/{now.day}/"
if not path.exists():
path.mkdir(parents=True)
text = f"{self.STATIC_URL}/{(path / f'{key}.{ext}').relative_to(self.DATA_ROOT)}"
return text
@staticmethod
async def get_size(file: UploadFile):
f = file.file
f.seek(0, os.SEEK_END)
size = f.tell()
f.seek(0, os.SEEK_SET)
return size
@staticmethod
def _save(filepath, file: BinaryIO):
with open(filepath, 'wb') as f:
chunk_size = 256 * 1024
chunk = file.read(chunk_size)
while chunk:
f.write(chunk)
chunk = file.read(chunk_size)
async def save_file(self, file: UploadFile, text: str):
filepath = await self.get_filepath(text)
await asyncio.to_thread(self._save, filepath, file.file)
async def delete_file(self, text: str):
filepath = await self.get_filepath(text)
if filepath.exists():
await asyncio.to_thread(os.remove, filepath)
await asyncio.to_thread(self.judge_delete_folder, filepath)
async def delete_files(self, texts):
tasks = [self.delete_file(text) for text in texts]
await asyncio.gather(*tasks)
def judge_delete_folder(self, filepath):
current = filepath.parent
while current != self.DATA_ROOT:
if not list(current.iterdir()):
os.rmdir(current)
current = current.parent
else:
break
STORAGE_ENGINE = {
"filesystem": FileSystemStorage,
"aliyunsystem": AliyunFileStorage
}
-25
View File
@@ -1,25 +0,0 @@
"""Storage backend package (split from the former 1400+ line storage.py).
Public surface is re-exported here — `from core.storage import X` keeps
working for every historical name.
"""
from core.storage._base import ( # noqa: F401
FileStorageInterface,
S3_MIN_MULTIPART_PART_SIZE,
StoredDownload,
StoredFile,
build_attachment_headers,
)
from core.storage.local import SystemFileStorage # noqa: F401
from core.storage.s3 import S3FileStorage # noqa: F401
from core.storage.onedrive import OneDriveFileStorage # noqa: F401
from core.storage.opendal import OpenDALFileStorage # noqa: F401
from core.storage.webdav import WebDAVFileStorage # noqa: F401
storages = {
"local": SystemFileStorage,
"s3": S3FileStorage,
"onedrive": OneDriveFileStorage,
"opendal": OpenDALFileStorage,
"webdav": WebDAVFileStorage,
}
-181
View File
@@ -1,181 +0,0 @@
# @Time : 2023/8/11 20:06
# @Author : Lan
# @File : storage.py
# @Software: PyCharm
import hashlib
from typing import BinaryIO, Optional
from urllib.parse import quote
from collections.abc import Callable
from typing import Any
from dataclasses import dataclass
from pathlib import Path
@dataclass
class StoredDownload:
"""Framework-free description of a file download.
Backends return this; the view layer builds the starlette Response:
- ``path`` set -> FileResponse (local files, Range support for free)
- ``content`` set -> small full-read Response (legacy OpenDAL fallback)
- ``stream_factory`` set -> StreamingResponse(stream_factory(), ...)
``background`` is an optional response-sent cleanup hook.
"""
filename: str
headers: dict
media_type: str = "application/octet-stream"
path: Path | None = None
content: bytes | None = None
stream_factory: Callable[[], Any] | None = None
background: Any = None # starlette BackgroundTask(core 层不 import starlette)
@dataclass
class StoredFile:
"""Plain, framework- and ORM-free description of a stored file.
Storage backends accept this instead of ORM models so core/ never imports
apps/. Callers (views/tasks/services) build it from their own records.
"""
file_path: str
uuid_file_name: str
code: str = ""
prefix: str = ""
suffix: str = ""
text: str = ""
def get_file_path(self) -> str:
return f"{self.file_path}/{self.uuid_file_name}"
# S3 multipart 除最后一片外每部分最小 5MB(服务端强制,小于即 EntityTooSmall)
S3_MIN_MULTIPART_PART_SIZE = 5 * 1024 * 1024
def build_attachment_headers(filename: str, content_length=None) -> dict:
"""所有存储后端统一的下载响应头。
Content-Disposition: attachment 是防御存储型 XSS 的关键——同源下载路径
(/share/download)因此永不内联渲染 HTML/SVG。此函数是唯一构造点,
新增后端必须复用(tests/test_attachment_guard.py 有源码级 tripwire)。
"""
encoded_filename = quote(filename, safe="")
headers = {"Content-Disposition": f"attachment; filename*=UTF-8''{encoded_filename}"}
if content_length is not None:
headers["Content-Length"] = str(content_length)
return headers
class FileStorageInterface:
@staticmethod
def _get_chunk_record(chunk_records: dict, index: int):
"""Look up the caller-provided record for chunk `index`.
Records are plain objects exposing ``chunk_hash``; fetching them from
the DB is the caller's job (keeps storage ORM-free).
"""
chunk_record = chunk_records.get(index)
if not chunk_record:
raise ValueError(f"分片{index}记录不存在")
return chunk_record
def _verify_and_hash_chunk(
self,
index: int,
chunk_record,
chunk_data: bytes,
file_sha256,
) -> None:
"""Verify a chunk against its recorded hash, then stripe it into the
whole-file digest. Raises the shared ValueError wording on mismatch."""
current_hash = hashlib.sha256(chunk_data).hexdigest()
if current_hash != chunk_record.chunk_hash:
raise ValueError(
f"分片{index}哈希不匹配: 期望 {chunk_record.chunk_hash}, 实际 {current_hash}"
)
file_sha256.update(chunk_data)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""Save a binary stream (caller owns closing the stream)."""
raise NotImplementedError
async def delete_file(self, file_code: StoredFile):
"""
删除文件
"""
raise NotImplementedError
async def get_file_url(self, file_code: StoredFile):
"""
获取文件分享的url
如果服务不支持直接访问文件,可以通过服务器中转下载。
此时,此方法可以调用 utils.py 中的 `get_file_url` 方法,获取服务器中转下载的url
"""
raise NotImplementedError
async def get_file_response(self, file_code: StoredFile):
"""
获取文件响应
如果服务不支持直接访问文件,则需要实现该方法,返回文件响应
其余情况,可以不实现该方法
"""
raise NotImplementedError
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片文件
:param upload_id: 上传会话ID
:param chunk_index: 分片索引
:param chunk_data: 分片数据
:param chunk_hash: 分片哈希值
:param save_path: 文件保存路径
"""
raise NotImplementedError
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并分片文件并返回文件路径和完整哈希值
:param upload_id: 上传会话ID
:param chunk_info: 分片信息
:param save_path: 文件保存路径
:return: (文件路径, 文件哈希值)
"""
raise NotImplementedError
async def generate_presigned_upload_url(self, save_path: str, expires_in: int = 900) -> Optional[str]:
"""
生成预签名上传URL
:param save_path: 文件保存路径
:param expires_in: URL过期时间(秒),默认15分钟
:return: 预签名URL,如果不支持直传则返回None
"""
return None # 默认不支持直传,使用代理模式
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在
:param save_path: 文件路径
:return: 文件是否存在
"""
raise NotImplementedError
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
raise NotImplementedError
-184
View File
@@ -1,184 +0,0 @@
import hashlib
from core.logger import logger
import shutil
from typing import BinaryIO, Optional
import aiofiles
import asyncio
from pathlib import Path
from core.errors import StorageError
from core.settings import data_root
from core.utils import get_file_url, sanitize_filename
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class SystemFileStorage(FileStorageInterface):
def __init__(self):
self.chunk_size = 256 * 1024
self.root_path = data_root
def _resolve_safe_path(self, relative_path: str) -> Path:
"""将相对路径解析到数据根目录内,阻止路径穿越。"""
root = self.root_path.resolve()
raw = str(relative_path or "").replace("\\", "/").lstrip("/")
if any(part == ".." for part in raw.split("/")):
raise ValueError("非法文件路径")
candidate = (root / raw).resolve()
try:
candidate.relative_to(root)
except ValueError as exc:
raise ValueError("非法文件路径") from exc
return candidate
def _save(self, file, save_path):
with open(save_path, "wb") as f:
chunk = file.read(self.chunk_size)
while chunk:
f.write(chunk)
chunk = file.read(self.chunk_size)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
path_obj = Path(str(save_path).replace("\\", "/"))
directory = str(path_obj.parent).replace("\\", "/").lstrip("/")
# 提取原始文件名并进行清理
filename = await sanitize_filename(path_obj.name)
# 构建安全的完整保存路径
safe_save_path = self._resolve_safe_path(f"{directory}/{filename}" if directory not in {"", "."} else filename)
# 确保目录存在
if not safe_save_path.parent.exists():
safe_save_path.parent.mkdir(parents=True)
await asyncio.to_thread(self._save, stream, safe_save_path)
async def delete_file(self, file_code: StoredFile):
save_path = self._resolve_safe_path(file_code.get_file_path())
if save_path.exists():
save_path.unlink()
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
file_path = self._resolve_safe_path(file_code.get_file_path())
if not file_path.exists():
raise StorageError(status_code=404, detail="文件已过期删除")
filename = f"{file_code.prefix}{file_code.suffix}"
try:
headers = build_attachment_headers(filename, file_path.stat().st_size)
except OSError:
# 文件大小不可得时省略 Content-Length
headers = build_attachment_headers(filename)
return StoredDownload(
filename=filename,
headers=headers,
path=file_path,
)
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片文件到本地文件系统
:param upload_id: 上传会话ID
:param chunk_index: 分片索引
:param chunk_data: 分片数据
:param chunk_hash: 分片哈希值
:param save_path: 文件保存路径
"""
# 先校验目标文件路径合法,再将分片落到同级 chunks 目录。
self._resolve_safe_path(save_path)
chunk_path = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
)
if not chunk_path.parent.exists():
chunk_path.parent.mkdir(parents=True, exist_ok=True)
# 使用临时文件写入,确保原子性
temp_path = chunk_path.with_suffix('.tmp')
try:
async with aiofiles.open(temp_path, "wb") as f:
await f.write(chunk_data)
# 原子重命名
temp_path.rename(chunk_path)
except Exception as e:
if temp_path.exists():
temp_path.unlink()
raise e
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并本地文件系统的分片文件并返回文件路径和完整哈希值
:param upload_id: 上传会话ID
:param chunk_info: 分片信息
:param save_path: 文件保存路径
:return: (文件路径, 文件哈希值)
"""
output_path = self._resolve_safe_path(save_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
chunk_base_dir = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id)
)
file_sha256 = hashlib.sha256()
# 使用临时文件写入,确保原子性
temp_output = output_path.with_suffix('.merging')
try:
async with aiofiles.open(temp_output, "wb") as out_file:
for i in range(total_chunks):
# 获取分片记录
chunk_record = self._get_chunk_record(chunk_records, i)
chunk_path = chunk_base_dir / f"{i}.part"
if not chunk_path.exists():
raise ValueError(f"分片{i}文件不存在")
async with aiofiles.open(chunk_path, "rb") as in_file:
chunk_data = await in_file.read()
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
# 原子重命名
temp_output.rename(output_path)
except Exception as e:
if temp_output.exists():
temp_output.unlink()
raise e
return str(output_path), file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理本地文件系统的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = self._resolve_safe_path(
str(Path(save_path).parent / "chunks" / upload_id)
)
if chunk_dir.exists():
try:
shutil.rmtree(chunk_dir)
except Exception as e:
logger.warning(f"清理本地分片目录失败: {e}")
# 清理父级 chunks 目录(如果为空)
chunks_parent = chunk_dir.parent
if chunks_parent.exists() and not any(chunks_parent.iterdir()):
try:
chunks_parent.rmdir()
except Exception as e:
logger.warning(f"清理 chunks 父目录失败: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于本地文件系统
:param save_path: 文件路径
:return: 文件是否存在
"""
try:
file_path = self._resolve_safe_path(save_path)
except ValueError:
return False
return file_path.exists()
-339
View File
@@ -1,339 +0,0 @@
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
import aiofiles
import aiohttp
import asyncio
from pathlib import Path
import datetime
import re
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url, sanitize_filename
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class OneDriveFileStorage(FileStorageInterface):
def __init__(self):
try:
import msal
from office365.graph_client import GraphClient
from office365.runtime.client_request_exception import (
ClientRequestException,
)
except ImportError:
raise ImportError("请先安装`msal`和`Office365-REST-Python-Client`")
self.msal = msal
self.domain = settings.onedrive_domain
self.client_id = settings.onedrive_client_id
self.username = settings.onedrive_username
self.password = settings.onedrive_password
self.proxy = settings.onedrive_proxy
self._ClientRequestException = ClientRequestException
try:
client = GraphClient(self.acquire_token_pwd)
self.root_path = (
client.me.drive.root.get_by_path(settings.onedrive_root_path)
.get()
.execute_query()
)
except ClientRequestException as e:
if e.code == "itemNotFound":
client.me.drive.root.create_folder(settings.onedrive_root_path)
self.root_path = (
client.me.drive.root.get_by_path(
settings.onedrive_root_path)
.get()
.execute_query()
)
else:
raise e
except Exception as e:
raise Exception("OneDrive验证失败,请检查配置是否正确\n" + str(e))
def acquire_token_pwd(self):
authority_url = f"https://login.microsoftonline.com/{self.domain}"
app = self.msal.PublicClientApplication(
authority=authority_url, client_id=self.client_id
)
result = app.acquire_token_by_username_password(
username=self.username,
password=self.password,
scopes=["https://graph.microsoft.com/.default"],
)
return result
def _get_path_str(self, path):
if isinstance(path, str):
path = path.replace("\\", "/").replace("//", "/").split("/")
elif isinstance(path, Path):
path = str(path).replace("\\", "/").replace("//", "/").split("/")
else:
raise TypeError("path must be str or Path")
path[-1] = path[-1].split(".")[0]
return "/".join(path)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""保存文件(自动创建目录;修复旧实现把 save_path 字符串当函数调用的崩溃)"""
content = await asyncio.to_thread(stream.read)
normalized = str(save_path).replace("\\", "/")
name = await sanitize_filename(Path(normalized).name)
dir_path = "/".join(normalized.split("/")[:-1])
current_folder = self.root_path
for part in dir_path.split("/"):
if not part:
continue
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
await asyncio.to_thread(
lambda: current_folder.get_by_path(name)
.upload(name, content)
.execute_query()
)
def _delete(self, save_path):
path = self._get_path_str(save_path)
try:
self.root_path.get_by_path(path).delete_object().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
pass
else:
raise e
async def delete_file(self, file_code: StoredFile):
await asyncio.to_thread(self._delete, file_code.get_file_path())
def _convert_link_to_download_link(self, link):
p1 = re.search(r"https://(.+)\.sharepoint\.com", link).group(1)
p2 = re.search(r"personal/(.+)/", link).group(1)
p3 = re.search(rf"{p2}/(.+)", link).group(1)
return f"https://{p1}.sharepoint.com/personal/{p2}/_layouts/52/download.aspx?share={p3}"
def _get_file_url(self, save_path, name):
path = self._get_path_str(save_path)
remote_file = self.root_path.get_by_path(path + "/" + name)
expiration_datetime = datetime.datetime.now(
tz=datetime.timezone.utc
) + datetime.timedelta(hours=1)
expiration_datetime = expiration_datetime.strftime(
"%Y-%m-%dT%H:%M:%SZ")
permission = remote_file.create_link(
"view", "anonymous", expiration_datetime=expiration_datetime
).execute_query()
return self._convert_link_to_download_link(permission.link.webUrl)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
try:
link = await asyncio.to_thread(
self._get_file_url, file_code.get_file_path(), filename
)
except self._ClientRequestException as e:
# 对象不存在时前置 404(与 local/S3/WebDAV 语义对齐),
# 不再落入外层兜底的 503
if str(getattr(e, "code", "")).lower() in {"itemnotfound", "404"}:
raise StorageError(
status_code=404, detail="文件已过期删除"
) from e
raise
content_length = None # 初始化为 None,表示未知大小
# 创建ClientSession并复用
session = aiohttp.ClientSession()
# 尝试发送HEAD请求获取Content-Length
try:
async with session.head(link) as resp:
if resp.status == 200 and 'Content-Length' in resp.headers:
content_length = int(resp.headers['Content-Length'])
except Exception:
# 如果HEAD请求失败,则不提供 Content-Length
pass
async def stream_generator():
try:
async with session.get(link) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"从OneDrive获取文件失败: {resp.status}"
)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except StorageError:
raise
except Exception:
raise StorageError(status_code=503, detail="服务代理下载异常,请稍后再试")
async def get_file_url(self, file_code: StoredFile):
if self.proxy:
return await get_file_url(file_code.code)
else:
return await asyncio.to_thread(
self._get_file_url,
file_code.get_file_path(),
f"{file_code.prefix}{file_code.suffix}",
)
def _save_chunk(self, chunk_path: str, chunk_data: bytes):
"""同步保存分片到 OneDrive"""
path_parts = chunk_path.replace("\\", "/").split("/")
filename = path_parts[-1]
dir_path = "/".join(path_parts[:-1])
# 确保目录存在
current_folder = self.root_path
for part in dir_path.split("/"):
if part:
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
# 上传分片
current_folder.upload(filename, chunk_data).execute_query()
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 OneDrive"""
chunk_path = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
await asyncio.to_thread(self._save_chunk, chunk_path, chunk_data)
def _read_chunk(self, chunk_path: str) -> bytes:
"""同步读取分片"""
path = self._get_path_str(chunk_path)
file_obj = self.root_path.get_by_path(path).get().execute_query()
return file_obj.get_content().execute_query().value
def _upload_merged(self, save_path: str, data: bytes):
"""同步上传合并后的文件"""
path_parts = save_path.replace("\\", "/").split("/")
filename = path_parts[-1]
dir_path = "/".join(path_parts[:-1])
# 确保目录存在
current_folder = self.root_path
for part in dir_path.split("/"):
if part:
try:
current_folder = current_folder.get_by_path(part).get().execute_query()
except self._ClientRequestException as e:
if e.code == "itemNotFound":
current_folder = current_folder.create_folder(part).execute_query()
else:
raise e
current_folder.upload(filename, data).execute_query()
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""合并 OneDrive 上的分片文件,使用临时文件避免内存问题"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
chunk_data = await asyncio.to_thread(self._read_chunk, chunk_path)
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 读取临时文件并上传
async with aiofiles.open(temp_path, 'rb') as f:
merged_content = await f.read()
await asyncio.to_thread(self._upload_merged, save_path, merged_content)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
def _delete_chunk_dir(self, chunk_dir: str):
"""同步删除分片目录"""
try:
path = self._get_path_str(chunk_dir)
self.root_path.get_by_path(path).delete_object().execute_query()
except self._ClientRequestException as e:
if e.code != "itemNotFound":
raise e
async def clean_chunks(self, upload_id: str, save_path: str):
"""清理 OneDrive 上的临时分片文件"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
try:
await asyncio.to_thread(self._delete_chunk_dir, chunk_dir)
except Exception as e:
logger.warning(f"清理 OneDrive 分片时出错: {e}")
def _file_exists(self, save_path: str) -> bool:
"""同步检查文件是否存在"""
try:
path = self._get_path_str(save_path)
self.root_path.get_by_path(path).get().execute_query()
return True
except self._ClientRequestException as e:
if e.code == "itemNotFound":
return False
raise e
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于OneDrive
:param save_path: 文件路径
:return: 文件是否存在
"""
return await asyncio.to_thread(self._file_exists, save_path)
-160
View File
@@ -1,160 +0,0 @@
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
import aiofiles
import asyncio
from pathlib import Path
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class OpenDALFileStorage(FileStorageInterface):
def __init__(self):
try:
import opendal
except ImportError:
raise ImportError('请先安装 `opendal`, 例如: "pip install opendal"')
self.service = settings.opendal_scheme
service_settings = {}
for key, value in settings.items():
if key.startswith("opendal_" + self.service):
setting_name = key.split("_", 2)[2]
service_settings[setting_name] = value
self.operator = opendal.AsyncOperator(
settings.opendal_scheme, **service_settings
)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
# 使用 asyncio.to_thread 避免阻塞事件循环
content = await asyncio.to_thread(stream.read)
await self.operator.write(save_path, content)
async def delete_file(self, file_code: StoredFile):
await self.operator.delete(file_code.get_file_path())
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
content_length = None # 初始化为 None,表示未知大小
# 尝试获取文件大小
try:
stat_result = await self.operator.stat(file_code.get_file_path())
if hasattr(stat_result, 'content_length') and stat_result.content_length:
content_length = stat_result.content_length
elif hasattr(stat_result, 'size') and stat_result.size:
content_length = stat_result.size
except Exception:
# 如果获取大小失败,则不提供 Content-Length
pass
# 尝试使用流式读取器
try:
# OpenDAL 可能提供 reader 方法返回一个异步读取器
reader = await self.operator.reader(file_code.get_file_path())
except AttributeError:
# 如果 reader 方法不存在,回退到全量读取(兼容旧版本)
content = await self.operator.read(file_code.get_file_path())
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
content=content,
)
async def stream_generator():
chunk_size = 65536
while True:
chunk = await reader.read(chunk_size)
if not chunk:
break
yield chunk
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
)
except Exception as e:
logger.info(e)
raise StorageError(status_code=404, detail="文件已过期删除")
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 OpenDAL 存储"""
chunk_path = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
await self.operator.write(chunk_path, chunk_data)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""合并 OpenDAL 存储上的分片文件,使用临时文件避免内存问题"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
chunk_data = await self.operator.read(chunk_path)
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 读取临时文件并写入存储
async with aiofiles.open(temp_path, 'rb') as f:
merged_content = await f.read()
await self.operator.write(save_path, merged_content)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""清理 OpenDAL 存储上的临时分片文件"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
try:
# OpenDAL 支持递归删除
await self.operator.remove_all(chunk_dir)
except Exception as e:
logger.warning(f"清理 OpenDAL 分片时出错: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于OpenDAL存储
:param save_path: 文件路径
:return: 文件是否存在
"""
try:
await self.operator.stat(save_path)
return True
except Exception:
return False
-345
View File
@@ -1,345 +0,0 @@
from botocore.exceptions import ClientError
import hashlib
from core.logger import logger
from typing import BinaryIO, Optional
import aiohttp
import asyncio
from pathlib import Path
import aioboto3
from botocore.config import Config
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
S3_MIN_MULTIPART_PART_SIZE,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class S3FileStorage(FileStorageInterface):
def __init__(self):
self.access_key_id = settings.s3_access_key_id
self.secret_access_key = settings.s3_secret_access_key
self.bucket_name = settings.s3_bucket_name
self.s3_hostname = settings.s3_hostname
self.region_name = settings.s3_region_name
self.signature_version = settings.s3_signature_version
self.endpoint_url = settings.s3_endpoint_url or f"https://{self.s3_hostname}"
self.aws_session_token = settings.aws_session_token
self.addressing_style = str(settings.s3_addressing_style or "auto").lower()
self.proxy = settings.s3_proxy
self.session = aioboto3.Session(
aws_access_key_id=self.access_key_id,
aws_secret_access_key=self.secret_access_key,
)
if not settings.s3_endpoint_url:
self.endpoint_url = f"https://{self.s3_hostname}"
else:
# 如果提供了 s3_endpoint_url,则优先使用它
self.endpoint_url = settings.s3_endpoint_url
def _client_config(self) -> Config:
config = {"signature_version": self.signature_version}
s3_config = {}
if self.addressing_style in {"path", "virtual", "auto"}:
s3_config["addressing_style"] = self.addressing_style
if s3_config:
config["s3"] = s3_config
return Config(**config)
def _client(self):
return self.session.client(
"s3",
endpoint_url=self.endpoint_url,
aws_session_token=self.aws_session_token,
region_name=self.region_name,
config=self._client_config(),
)
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
async with self._client() as s3:
# 使用 upload_fileobj 流式上传,避免将整个文件加载到内存
await s3.upload_fileobj(
stream,
self.bucket_name,
save_path,
ExtraArgs={"ContentType": content_type or "application/octet-stream"},
)
async def delete_file(self, file_code: StoredFile):
async with self._client() as s3:
await s3.delete_object(
Bucket=self.bucket_name, Key=file_code.get_file_path()
)
async def get_file_response(self, file_code: StoredFile):
try:
filename = file_code.prefix + file_code.suffix
content_length = None # 初始化为 None,表示未知大小
async with self._client() as s3:
# 尝试获取文件大小(HEAD请求);对象不存在时前置 404——
# 与 local 后端语义一致(M2 行为统一),不能签出 200 的坏流
try:
head_response = await s3.head_object(
Bucket=self.bucket_name,
Key=file_code.get_file_path()
)
# 从HEAD响应中获取Content-Length
if 'ContentLength' in head_response:
content_length = head_response['ContentLength']
elif 'Content-Length' in head_response['ResponseMetadata']['HTTPHeaders']:
content_length = int(head_response['ResponseMetadata']['HTTPHeaders']['Content-Length'])
except ClientError as e:
error_code = e.response.get("Error", {}).get("Code", "")
if error_code in {"404", "NoSuchKey", "NotFound"}:
raise StorageError(
status_code=404, detail="文件已过期删除"
) from e
# 其他 HEAD 错误不阻断:流式下载阶段会给出真实状态
except Exception:
# 如果HEAD请求失败,则不提供 Content-Length
pass
link = await s3.generate_presigned_url(
"get_object",
Params={
"Bucket": self.bucket_name,
"Key": file_code.get_file_path(),
},
ExpiresIn=3600,
)
# 创建ClientSession并传递给生成器复用
session = aiohttp.ClientSession()
async def stream_generator():
try:
async with session.get(link) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"从S3获取文件失败: {resp.status}"
)
# 设置块大小(例如64KB)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except StorageError:
raise
except Exception:
raise StorageError(status_code=503, detail="服务代理下载异常,请稍后再试")
async def get_file_url(self, file_code: StoredFile):
if file_code.prefix == "文本分享":
return file_code.text
if self.proxy:
return await get_file_url(file_code.code)
else:
async with self._client() as s3:
result = await s3.generate_presigned_url(
"get_object",
Params={
"Bucket": self.bucket_name,
"Key": file_code.get_file_path(),
},
ExpiresIn=3600,
)
return result
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""
保存分片到 S3(使用独立对象存储每个分片)
注意:这里不使用 S3 原生的 multipart upload,而是将每个分片作为独立对象存储
"""
chunk_key = str(Path(save_path).parent / "chunks" / upload_id / f"{chunk_index}.part")
async with self._client() as s3:
# 将分片作为独立对象上传
await s3.put_object(
Bucket=self.bucket_name,
Key=chunk_key,
Body=chunk_data,
Metadata={
'chunk-hash': chunk_hash,
'chunk-index': str(chunk_index)
}
)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并 S3 上的分片文件
使用 S3 的 multipart upload API 实现流式合并,避免内存问题
"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
async with self._client() as s3:
# 创建 multipart upload
mpu = await s3.create_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
ContentType='application/octet-stream'
)
mpu_id = mpu['UploadId']
parts = []
try:
# 按顺序读取、验证每个分片;S3 multipart 规范要求除最后一片外
# 每个部分 ≥5MB(EntityTooSmall),而分片大小由客户端决定(常见
# 2-4MB)——因此缓冲到 S3_MIN_MULTIPART_PART_SIZE 再上传 part,
# 内存上界 = 5MB + 单个分片,不破坏流式合并的初衷。
part_buffer = bytearray()
part_number = 0
async def _flush_part():
nonlocal part_number
if not part_buffer:
return
part_number += 1
part_response = await s3.upload_part(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id,
PartNumber=part_number,
Body=bytes(part_buffer),
)
parts.append({
'PartNumber': part_number,
'ETag': part_response['ETag']
})
part_buffer.clear()
for i in range(total_chunks):
chunk_key = f"{chunk_dir}/{i}.part"
chunk_record = self._get_chunk_record(chunk_records, i)
try:
response = await s3.get_object(
Bucket=self.bucket_name,
Key=chunk_key
)
chunk_data = await response['Body'].read()
except Exception as e:
raise ValueError(f"分片{i}文件不存在: {e}")
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
part_buffer.extend(chunk_data)
if len(part_buffer) >= S3_MIN_MULTIPART_PART_SIZE:
await _flush_part()
# 收尾:剩余缓冲作为最后一个 part(S3 允许最后一片小于 5MB;
# 恰好整除时缓冲为空,跳过)
await _flush_part()
# 完成 multipart upload
await s3.complete_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id,
MultipartUpload={'Parts': parts}
)
except Exception as e:
# 出错时取消 multipart upload
await s3.abort_multipart_upload(
Bucket=self.bucket_name,
Key=save_path,
UploadId=mpu_id
)
raise e
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理 S3 上的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
async with self._client() as s3:
try:
# 列出并删除所有分片对象
paginator = s3.get_paginator('list_objects_v2')
async for page in paginator.paginate(Bucket=self.bucket_name, Prefix=chunk_dir):
objects = page.get('Contents', [])
if objects:
delete_objects = [{'Key': obj['Key']} for obj in objects]
await s3.delete_objects(
Bucket=self.bucket_name,
Delete={'Objects': delete_objects}
)
except Exception as e:
logger.warning(f"清理 S3 分片数据时出错: {e}")
async def generate_presigned_upload_url(self, save_path: str, expires_in: int = 900) -> Optional[str]:
"""
生成S3预签名上传URL
:param save_path: 文件保存路径
:param expires_in: URL过期时间(秒),默认15分钟
:return: 预签名PUT URL
"""
async with self._client() as s3:
return await s3.generate_presigned_url(
"put_object",
Params={
"Bucket": self.bucket_name,
"Key": save_path,
},
ExpiresIn=expires_in,
)
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于S3
:param save_path: 文件路径
:return: 文件是否存在
"""
async with self._client() as s3:
last_error = None
for attempt in range(3):
try:
await s3.head_object(Bucket=self.bucket_name, Key=save_path)
return True
except Exception as e:
last_error = e
if attempt < 2:
await asyncio.sleep(0.2 * (attempt + 1))
try:
result = await s3.list_objects_v2(
Bucket=self.bucket_name,
Prefix=save_path,
MaxKeys=1,
)
for item in result.get("Contents", []):
if item.get("Key") == save_path:
return True
except Exception as e:
last_error = e
logger.warning(f"S3文件确认失败 key={save_path}: {last_error}")
return False
-354
View File
@@ -1,354 +0,0 @@
import base64
import hashlib
import os
import tempfile
from core.logger import logger
from typing import BinaryIO, Optional
from urllib.parse import quote, unquote
import aiofiles
import aiohttp
import asyncio
from pathlib import Path
import re
from core.errors import StorageError
from core.settings import settings
from core.utils import get_file_url, sanitize_filename
from starlette.background import BackgroundTask
from core.storage._base import (
FileStorageInterface,
StoredDownload,
StoredFile,
build_attachment_headers,
)
class WebDAVFileStorage(FileStorageInterface):
def __init__(self):
if not hasattr(self, "_initialized"):
self.base_url = settings.webdav_url.rstrip("/") + "/"
# aiohttp 4.0 移除 BasicAuth(auth=...) 参数——改用编码后的 Authorization 头
self.auth_headers = {
"Authorization": aiohttp.encode_basic_auth(
settings.webdav_username, settings.webdav_password
)
}
self._initialized = True
def _build_url(self, path: str) -> str:
encoded_path = quote(str(path.replace("\\", "/").lstrip("/")).lstrip("/"))
return f"{self.base_url}{encoded_path}"
async def _mkdir_p(self, directory_path: str):
"""递归创建目录(类似mkdir -p)"""
path_obj = Path(unquote(directory_path))
current_path = ""
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 逐级检查目录是否存在
for part in path_obj.parts:
current_path = str(Path(current_path) / part)
url = self._build_url(current_path)
# 检查目录是否存在
async with session.head(url) as resp:
if resp.status == 404:
# 创建目录
async with session.request("MKCOL", url) as mkcol_resp:
if mkcol_resp.status not in (200, 201, 409):
content = await mkcol_resp.text()
raise StorageError(
status_code=mkcol_resp.status,
detail=f"目录创建失败: {content[:200]}",
)
async def _is_dir_empty(self, dir_path: str) -> bool:
"""检查目录是否为空"""
url = self._build_url(dir_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.request("PROPFIND", url, headers={"Depth": "1"}) as resp:
if resp.status != 207: # 207 是 Multi-Status 响应
return False
content = await resp.text()
# 如果只有一个 response(当前目录),说明目录为空
return content.count("<D:response>") <= 1
async def _delete_empty_dirs(self, file_path: str, session: aiohttp.ClientSession):
"""递归删除空目录"""
path_obj = Path(file_path)
current_path = path_obj.parent
while str(current_path) != ".":
if not await self._is_dir_empty(str(current_path)):
break
url = self._build_url(str(current_path))
async with session.delete(url) as resp:
if resp.status not in (200, 204, 404):
break
current_path = current_path.parent
async def save_file(
self, stream: BinaryIO, save_path: str, content_type: Optional[str] = None
):
"""保存文件(自动创建目录,流式上传)"""
path_obj = Path(save_path)
directory_path = str(path_obj.parent)
# 提取原始文件名并进行清理
filename = await sanitize_filename(path_obj.name)
# 构建安全的保存路径
safe_save_path = str(Path(directory_path) / filename)
try:
# 先创建目录结构
await self._mkdir_p(directory_path)
# 上传文件(流式)
url = self._build_url(safe_save_path)
async def file_sender():
"""流式读取文件内容"""
chunk_size = 256 * 1024 # 256KB chunks
while True:
chunk = await asyncio.to_thread(stream.read, chunk_size)
if not chunk:
break
yield chunk
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.put(
url,
data=file_sender(),
headers={"Content-Type": content_type or "application/octet-stream"}
) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"文件上传失败: {content[:200]}",
)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def delete_file(self, file_code: StoredFile):
"""删除WebDAV文件及空目录"""
file_path = file_code.get_file_path()
url = self._build_url(file_path)
try:
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 删除文件
async with session.delete(url) as resp:
if resp.status not in (200, 204, 404):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"WebDAV删除失败: {content[:200]}",
)
# 使用同一个 session 删除空目录
await self._delete_empty_dirs(file_path, session)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def get_file_url(self, file_code: StoredFile):
return await get_file_url(file_code.code)
async def get_file_response(self, file_code: StoredFile):
"""获取文件响应(代理模式)"""
try:
filename = file_code.prefix + file_code.suffix
url = self._build_url(file_code.get_file_path())
content_length = None # 初始化为 None,表示未知大小
# 创建ClientSession并复用(包含认证头)
session = aiohttp.ClientSession(headers={
"Authorization": f"Basic {base64.b64encode(f'{settings.webdav_username}:{settings.webdav_password}'.encode()).decode()}"
})
# 尝试发送HEAD请求获取Content-Length;对象不存在时前置 404
# (与 local/S3 语义对齐),连接层错误映射 503——两者都不能
# 静默吞掉后签出 200 坏流。异常路径回收 session 防泄漏。
try:
try:
async with session.head(url) as resp:
if resp.status == 404:
raise StorageError(
status_code=404, detail="文件已过期删除"
)
if resp.status == 200 and 'Content-Length' in resp.headers:
content_length = int(resp.headers['Content-Length'])
except StorageError:
raise
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}"
) from e
except Exception:
# 其他 HEAD 异常不阻断:流式下载阶段会给出真实状态
pass
except BaseException:
await session.close()
raise
async def stream_generator():
try:
async with session.get(url) as resp:
if resp.status != 200:
raise StorageError(
status_code=resp.status,
detail=f"文件获取失败{resp.status}: {await resp.text()}",
)
chunk_size = 65536
while True:
chunk = await resp.content.read(chunk_size)
if not chunk:
break
yield chunk
finally:
await session.close()
headers = build_attachment_headers(filename, content_length)
return StoredDownload(
filename=filename,
headers=headers,
stream_factory=stream_generator,
# 兜底关闭会话:客户端中断时与 generator finally 双保险
background=BackgroundTask(session.close),
)
except aiohttp.ClientError as e:
raise StorageError(
status_code=503, detail=f"WebDAV连接异常: {str(e)}")
async def save_chunk(self, upload_id: str, chunk_index: int, chunk_data: bytes, chunk_hash: str, save_path: str):
"""保存分片到 WebDAV"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
chunk_path = f"{chunk_dir}/{chunk_index}.part"
# 先创建目录结构
await self._mkdir_p(chunk_dir)
chunk_url = self._build_url(chunk_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.put(chunk_url, data=chunk_data) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"分片上传失败: {content[:200]}"
)
async def merge_chunks(self, upload_id: str, total_chunks: int, chunk_size: int, save_path: str, chunk_records: dict) -> tuple[str, str]:
"""
合并 WebDAV 上的分片文件
使用临时文件避免内存问题
"""
file_sha256 = hashlib.sha256()
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
# 使用临时文件存储合并数据,避免内存问题
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_path = temp_file.name
try:
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
# 按顺序读取并验证每个分片,写入临时文件
async with aiofiles.open(temp_path, 'wb') as out_file:
for i in range(total_chunks):
chunk_path = f"{chunk_dir}/{i}.part"
chunk_url = self._build_url(chunk_path)
# 获取分片记录
chunk_record = self._get_chunk_record(chunk_records, i)
# 下载分片数据
async with session.get(chunk_url) as resp:
if resp.status != 200:
raise ValueError(f"分片{i}文件不存在或无法访问")
chunk_data = await resp.read()
# 验证哈希
self._verify_and_hash_chunk(i, chunk_record, chunk_data, file_sha256)
await out_file.write(chunk_data)
del chunk_data # 释放内存
# 确保目标目录存在
output_dir = str(Path(save_path).parent)
await self._mkdir_p(output_dir)
# 流式上传合并后的文件
output_url = self._build_url(save_path)
async def file_sender():
async with aiofiles.open(temp_path, 'rb') as f:
while True:
chunk = await f.read(256 * 1024)
if not chunk:
break
yield chunk
async with session.put(output_url, data=file_sender()) as resp:
if resp.status not in (200, 201, 204):
content = await resp.text()
raise StorageError(
status_code=resp.status,
detail=f"合并文件上传失败: {content[:200]}"
)
finally:
# 清理临时文件
if os.path.exists(temp_path):
os.unlink(temp_path)
return save_path, file_sha256.hexdigest()
async def clean_chunks(self, upload_id: str, save_path: str):
"""
清理 WebDAV 上的临时分片文件
:param upload_id: 上传会话ID
:param save_path: 文件保存路径
"""
chunk_dir = str(Path(save_path).parent / "chunks" / upload_id)
chunk_dir_url = self._build_url(chunk_dir)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
try:
# 检查分片目录是否存在
async with session.request("PROPFIND", chunk_dir_url, headers={"Depth": "1"}) as resp:
if resp.status == 207: # 207 表示 Multi-Status
# 获取目录下的所有分片文件
xml_data = await resp.text()
file_paths = re.findall(
r'<D:href>(.*?)</D:href>', xml_data)
for file_path in file_paths:
if file_path.endswith(".part"):
# 删除分片文件
file_url = self._build_url(file_path)
async with session.delete(file_url) as delete_resp:
if delete_resp.status not in (200, 204, 404):
logger.warning(f"删除分片文件失败: {file_path}")
# 删除分片目录
async with session.delete(chunk_dir_url) as delete_resp:
if delete_resp.status not in (200, 204, 404):
logger.warning(f"删除分片目录失败: {chunk_dir_url}")
else:
logger.info(f"分片目录不存在: {chunk_dir_url}")
except Exception as e:
logger.warning(f"清理 WebDAV 分片时出错: {e}")
async def file_exists(self, save_path: str) -> bool:
"""
检查文件是否存在于WebDAV
:param save_path: 文件路径
:return: 文件是否存在
"""
url = self._build_url(save_path)
async with aiohttp.ClientSession(headers=self.auth_headers) as session:
async with session.head(url) as resp:
return resp.status == 200
+37 -225
View File
@@ -1,231 +1,43 @@
# @Time : 2023/8/13 19:54
# @Author : Lan
# @File : utils.py
# @Software: PyCharm
import datetime
import hashlib
import hmac
import os
import re
import secrets
import string
import time
import random
import asyncio
from sqlalchemy import or_, select, delete
from sqlalchemy.ext.asyncio.session import AsyncSession
from .database import Codes, engine
from .depends import IPRateLimit
from .storage import STORAGE_ENGINE
from settings import settings
from core.settings import settings
storage = STORAGE_ENGINE[settings.STORAGE_ENGINE]()
# 错误IP限制器
error_ip_limit = IPRateLimit(settings.ERROR_COUNT, settings.ERROR_MINUTE)
# 上传文件限制器
upload_ip_limit = IPRateLimit(settings.UPLOAD_COUNT, settings.UPLOAD_MINUTE)
async def get_random_num():
"""
获取随机数
:return:
"""
return secrets.randbelow(90000) + 10000
async def delete_expire_files():
while True:
async with AsyncSession(engine, expire_on_commit=False) as s:
await error_ip_limit.remove_expired_ip()
await upload_ip_limit.remove_expired_ip()
query = select(Codes).where(or_(Codes.exp_time < datetime.datetime.now(), Codes.count == 0))
exps = (await s.execute(query)).scalars().all()
files = []
exps_ids = []
for exp in exps:
if exp.type != "text":
files.append(exp.text)
exps_ids.append(exp.id)
await storage.delete_files(files)
query = delete(Codes).where(Codes.id.in_(exps_ids))
await s.execute(query)
await s.commit()
await asyncio.sleep(settings.DELETE_EXPIRE_FILES_INTERVAL * 60)
r_s = string.ascii_uppercase + string.digits
def validate_background_url(value) -> str:
"""Validate the site background config before it reaches the theme template.
Themes inject this value into inline CSS ``url('...')`` where html escaping
cannot neutralize a single-quote breakout, so only well-formed http(s) URLs
(or an empty string) are accepted.
"""
value = str(value or "").strip()
if not value:
return ""
from urllib.parse import urlparse
parsed = urlparse(value)
if parsed.scheme not in ("http", "https") or not parsed.netloc:
raise ValueError("background 必须是 http(s) 完整 URL 或留空")
# quote()/whitespace would break out of the CSS url('') quoting context
if any(ch in value for ch in ("'", '"', "(", ")", " ", "\\", ";")):
raise ValueError("background URL 含有不允许的字符")
return value
async def get_random_string():
"""
获取随机字符串
:return:
"""
return "".join(secrets.choice(r_s) for _ in range(5))
async def get_now():
"""
获取当前时间
:return:
"""
return datetime.datetime.now(datetime.timezone(datetime.timedelta(hours=8)))
async def get_select_token(code: str, offset: int = 0):
"""
获取下载token
:param code: 取件码
:param offset: 时间窗口偏移(0=当前窗口,1=上一个窗口)。
用于兼容窗口边界竞态:用户在某窗口末尾获取的 token,
请求到达服务器时可能已进入下一窗口。
:return:
"""
token = getattr(settings, "jwt_secret", "")
if not token:
raise RuntimeError("应用签名密钥未初始化")
# 每个窗口约 1000 秒;offset 允许校验上一窗口,避免边界竞态
time_factor = int(time.time() / 1000) - max(0, int(offset))
return hashlib.sha256(
f"{code}{time_factor}000{token}".encode()
).hexdigest()
async def get_file_url(code: str):
"""
对于需要通过服务器中转下载的服务,获取文件下载地址
:param code:
:return:
"""
return f"/share/download?key={await get_select_token(code)}&code={code}"
async def max_save_times_desc(max_save_seconds: int):
"""
获取最大保存时间的描述
:param max_save_seconds:
:return:
"""
def gen_desc_zh(value: int, desc: str):
if value > 0:
return f"{value}{desc}"
else:
return ""
def gen_desc_en(value: int, desc: str):
if value > 0:
ret = f"{value} {desc}"
if value > 1:
ret += "s"
ret += " "
return ret
else:
return ""
max_timedelta = datetime.timedelta(seconds=max_save_seconds)
desc_zh, desc_en = "最长保存时间:", "Max save time: "
desc_zh += gen_desc_zh(max_timedelta.days, "天")
desc_en += gen_desc_en(max_timedelta.days, "day")
desc_zh += gen_desc_zh(max_timedelta.seconds // 3600, "小时")
desc_en += gen_desc_en(max_timedelta.seconds // 3600, "hour")
desc_zh += gen_desc_zh(max_timedelta.seconds % 3600 // 60, "分钟")
desc_en += gen_desc_en(max_timedelta.seconds % 3600 // 60, "minute")
desc_zh += gen_desc_zh(max_timedelta.seconds % 60, "秒")
desc_en += gen_desc_en(max_timedelta.seconds % 60, "second")
return desc_zh, desc_en
# scrypt work factors (OWASP-recommended memory-hard parameters; ~50ms/verify)
SCRYPT_N = 2**14
SCRYPT_R = 8
SCRYPT_P = 1
_SCRYPT_MAXMEM = 64 * 1024 * 1024 # hashlib default cap (32MB) is too low for n=2^14,r=8
PASSWORD_SCHEME = "scrypt" # current scheme for new hashes; sha256/plaintext stay verifiable
def hash_password(password: str) -> str:
"""
使用 scrypt(memory-hard)哈希密码
返回格式: scrypt$<n>$<r>$<p>$<salt>$<hash>
"""
salt = os.urandom(16).hex()
password_hash = hashlib.scrypt(
password.encode(),
salt=salt.encode(),
n=SCRYPT_N,
r=SCRYPT_R,
p=SCRYPT_P,
maxmem=_SCRYPT_MAXMEM,
).hex()
return f"scrypt${SCRYPT_N}${SCRYPT_R}${SCRYPT_P}${salt}${password_hash}"
def verify_password(password: str, hashed: str) -> bool:
"""
验证密码是否匹配
支持格式: scrypt$n$r$p$salt$hash(现行)、sha256$salt$hash(历史)、明文(最老)
"""
if not hashed:
return False
if hashed.startswith("scrypt$"):
parts = hashed.split("$")
if len(parts) != 6:
return False
try:
scheme, n, r, p, salt, stored_hash = parts
password_hash = hashlib.scrypt(
password.encode(),
salt=salt.encode(),
n=int(n),
r=int(r),
p=int(p),
maxmem=_SCRYPT_MAXMEM,
).hex()
# 默认解释器对超范围的 n/r/p 抛 TypeError/ValueError;部分构建会抛
# OverflowError,一并视为校验失败,避免坏掉的存量哈希影响登录路径。
except (ValueError, TypeError, OverflowError):
return False
return hmac.compare_digest(password_hash, stored_hash)
# sha256 格式: sha256$salt$hash(历史数据,验证逻辑保持原样)
if hashed.startswith("sha256$"):
parts = hashed.split("$")
if len(parts) != 3:
return False
_, salt, stored_hash = parts
password_hash = hashlib.sha256(f"{salt}{password}".encode()).hexdigest()
return hmac.compare_digest(password_hash, stored_hash)
# 旧格式: 明文比较 (兼容迁移前的数据)
return hmac.compare_digest(str(password), str(hashed))
def is_password_hashed(password: str) -> bool:
"""
检查密码是否已经是哈希格式(现行或历史哈希均视为已哈希)
"""
if password.startswith("scrypt$") and len(password.split("$")) == 6:
return True
return password.startswith("sha256$") and len(password.split("$")) == 3
def password_needs_rehash(hashed: str) -> bool:
"""True when the stored hash uses a legacy scheme (sha256/plaintext)."""
return bool(hashed) and not hashed.startswith(f"{PASSWORD_SCHEME}$")
async def sanitize_filename(filename: str) -> str:
"""
安全处理文件名:
1. 剥离路径只保留文件名
2. 替换非法字符
3. 处理空文件名情况
"""
filename = os.path.basename(filename)
illegal_chars = r'[\\/*?:"<>|\x00-\x1F]' # 包含控制字符
# 替换非法字符为下划线
cleaned = re.sub(illegal_chars, "_", filename)
# 处理空格(可选替换为_)
cleaned = cleaned.replace(" ", "_")
# 处理连续下划线
cleaned = re.sub(r"_+", "_", cleaned)
# 处理首尾特殊字符
cleaned = cleaned.strip("._")
# 处理空文件名情况
if not cleaned:
cleaned = "unnamed_file"
# 长度限制(按需调整)
return cleaned[:255]
async def get_code(s: AsyncSession):
code = random.randint(10000, 99999)
while (await s.execute(select(Codes.id).where(Codes.code == code))).scalar():
code = random.randint(10000, 99999)
return str(code)
-20
View File
@@ -1,20 +0,0 @@
import os
import re
from pathlib import Path
VERSION_FILE = Path(__file__).resolve().parents[1] / "VERSION"
VERSION_PATTERN = re.compile(
r"^\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$"
)
def load_app_version() -> str:
version = os.getenv("APP_VERSION") or VERSION_FILE.read_text(encoding="utf-8")
version = version.strip()
if not VERSION_PATTERN.fullmatch(version):
raise RuntimeError(f"无效的应用版本号: {version!r}")
return version
APP_VERSION = load_app_version()
-24
View File
@@ -1,24 +0,0 @@
version: "3"
services:
file-code-box:
image: lanol/filecodebox:2.8.4 # x-release-please-version
volumes:
- fcb-data:/app/data:rw
# NAS 已有文件:把共享目录挂到 /app/data/local,即可在后台「本地分享」里生成提取码,无需再上传
# - /volume1/share:/app/data/local:ro
restart: unless-stopped
ports:
- "12345:12345"
environment:
- APP_ENV=production
- LOG_LEVEL=warning
- ACCESS_LOG=false
- WORKERS=1
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
volumes:
fcb-data:
external: false
-328
View File
@@ -1,328 +0,0 @@
import { defineConfig } from 'vitepress'
// https://vitepress.dev/reference/site-config
export default defineConfig({
title: 'FileCodeBox',
description: 'FileCodeBox 自托管文件与文本分享服务文档',
lang: 'zh-CN',
lastUpdated: true,
cleanUrls: true,
sitemap: {
hostname: 'https://fcb-docs.aiuo.net',
},
head: [
['meta', { name: 'theme-color', content: '#111111' }],
['link', { rel: 'icon', href: '/logo_small.png' }],
],
locales: {
root: {
label: '简体中文',
lang: 'zh-CN',
title: 'FileCodeBox',
description: '匿名口令分享文本,文件',
themeConfig: {
logo: '/logo_small.png',
nav: [
{ text: '首页', link: '/' },
{
text: '使用指南',
items: [
{ text: '快速开始', link: '/guide/getting-started' },
{ text: '上传与分享', link: '/guide/upload' },
{ text: '管理面板', link: '/guide/management' },
{ text: '存储配置', link: '/guide/storage' },
{ text: 'MCP 接入', link: '/guide/mcp' },
],
},
{ text: 'API', link: '/api/' },
{
text: '资源',
items: [
{ text: '在线体验', link: 'https://share.lanol.cn' },
{ text: '部署案例', link: '/showcase' },
{ text: '更新日志', link: '/changelog' },
{ text: '贡献指南', link: '/contributing' },
{ text: 'GitHub', link: 'https://github.com/vastsa/FileCodeBox' },
],
},
],
sidebar: {
'/guide/': [
{
text: '开始',
items: [
{
text: '什么是 FileCodeBox',
link: '/guide/introduction',
},
{ text: '快速开始', link: '/guide/getting-started' },
],
},
{
text: '使用',
items: [
{ text: '文件上传', link: '/guide/upload' },
{ text: '文件分享', link: '/guide/share' },
{ text: '文件管理', link: '/guide/management' },
{ text: 'MCP 接入', link: '/guide/mcp' },
],
},
{
text: '运维',
items: [
{ text: '存储配置', link: '/guide/storage' },
{ text: '安全设置', link: '/guide/security' },
{ text: '系统配置', link: '/guide/configuration' },
{ text: '版本发布', link: '/guide/releasing' },
],
},
],
'/api/': [
{
text: 'API 参考',
items: [
{ text: 'API 概览', link: '/api/' },
{ text: '预签名上传', link: '/api/presign-upload' },
{ text: '分享接口', link: '/api/#分享接口' },
{ text: '管理接口', link: '/api/#管理接口' },
{ text: '错误响应', link: '/api/#错误响应' },
{ text: '状态码说明', link: '/api/#状态码说明' },
],
},
],
'/showcase': [
{
text: '优秀案例',
items: [
{ text: '案例展示', link: '/showcase' },
],
},
],
},
socialLinks: [
{ icon: 'github', link: 'https://github.com/vastsa/FileCodeBox' },
],
footer: {
message: '基于 LGPL-3.0 许可证发布',
copyright: 'Copyright © 2022-present FileCodeBox',
},
editLink: {
pattern: 'https://github.com/vastsa/FileCodeBox/edit/master/docs/:path',
text: '在 GitHub 上编辑此页',
},
docFooter: {
prev: '上一篇',
next: '下一篇',
},
lastUpdated: {
text: '最后更新',
},
returnToTopLabel: '返回顶部',
skipToContentLabel: '跳转到正文',
navMenuLabel: '导航菜单',
langMenuLabel: '切换语言',
sidebarMenuLabel: '文档目录',
darkModeSwitchLabel: '外观',
lightModeSwitchTitle: '切换到浅色模式',
darkModeSwitchTitle: '切换到深色模式',
search: {
provider: 'local',
},
outline: {
level: [2, 3],
label: '目录',
},
},
},
en: {
label: 'English',
lang: 'en-US',
title: 'FileCodeBox',
description: 'Simple and efficient file sharing tool',
themeConfig: {
logo: '/logo_small.png',
nav: [
{ text: 'Home', link: '/en/' },
{
text: 'Guides',
items: [
{ text: 'Getting Started', link: '/en/guide/getting-started' },
{ text: 'Upload & Share', link: '/en/guide/upload' },
{ text: 'Admin Panel', link: '/en/guide/management' },
{ text: 'MCP Integration', link: '/en/guide/mcp' },
{ text: 'Storage', link: '/en/guide/storage' },
],
},
{ text: 'API', link: '/en/api/' },
{
text: 'Resources',
items: [
{ text: 'Live Demo', link: 'https://share.lanol.cn' },
{ text: 'Showcase', link: '/en/showcase' },
{ text: 'Changelog', link: '/en/changelog' },
{ text: 'Contributing', link: '/en/contributing' },
{ text: 'GitHub', link: 'https://github.com/vastsa/FileCodeBox' },
],
},
],
sidebar: {
'/en/guide/': [
{
text: 'Start',
items: [
{
text: 'What is FileCodeBox',
link: '/en/guide/introduction',
},
{
text: 'Getting Started',
link: '/en/guide/getting-started',
},
],
},
{
text: 'Use',
items: [
{ text: 'File Upload', link: '/en/guide/upload' },
{ text: 'File Sharing', link: '/en/guide/share' },
{ text: 'File Management', link: '/en/guide/management' },
{ text: 'MCP Integration', link: '/en/guide/mcp' },
],
},
{
text: 'Operate',
items: [
{ text: 'Storage Configuration', link: '/en/guide/storage' },
{ text: 'Security Settings', link: '/en/guide/security' },
{
text: 'System Configuration',
link: '/en/guide/configuration',
},
{ text: 'Releasing', link: '/en/guide/releasing' },
],
},
],
'/en/api/': [
{
text: 'API Reference',
items: [
{ text: 'API Overview', link: '/en/api/' },
{ text: 'Presigned Upload', link: '/en/api/presign-upload' },
{ text: 'Share API', link: '/en/api/#share-api' },
{ text: 'Admin API', link: '/en/api/#admin-api' },
{ text: 'Error Response', link: '/en/api/#error-response' },
{ text: 'Status Codes', link: '/en/api/#status-codes' },
],
},
],
'/en/showcase': [
{
text: 'Showcase',
items: [
{ text: 'Case Studies', link: '/en/showcase' },
],
},
],
},
socialLinks: [
{ icon: 'github', link: 'https://github.com/vastsa/FileCodeBox' },
],
footer: {
message: 'Released under the LGPL-3.0 license',
copyright: 'Copyright © 2022-present FileCodeBox',
},
editLink: {
pattern: 'https://github.com/vastsa/FileCodeBox/edit/master/docs/:path',
text: 'Edit this page on GitHub',
},
docFooter: {
prev: 'Previous',
next: 'Next',
},
lastUpdated: {
text: 'Last updated',
},
returnToTopLabel: 'Return to top',
skipToContentLabel: 'Skip to content',
navMenuLabel: 'Navigation',
langMenuLabel: 'Change language',
sidebarMenuLabel: 'Documentation',
darkModeSwitchLabel: 'Appearance',
lightModeSwitchTitle: 'Switch to light theme',
darkModeSwitchTitle: 'Switch to dark theme',
search: {
provider: 'local',
},
outline: {
level: [2, 3],
label: 'On this page',
},
},
},
},
themeConfig: {
// 语言切换器
langMenuLabel: '切换语言',
// 社交链接
socialLinks: [
{ icon: 'github', link: 'https://github.com/vastsa/FileCodeBox' },
],
// 页脚
footer: {
message: '基于 LGPL-3.0 license 发布',
copyright: 'Copyright © 2022-present FileCodeBox',
},
// 搜索
search: {
provider: 'local',
options: {
locales: {
zh: {
translations: {
button: {
buttonText: '搜索文档',
buttonAriaLabel: '搜索文档',
},
modal: {
noResultsText: '无法找到相关结果',
resetButtonTitle: '清除查询条件',
footer: {
selectText: '选择',
navigateText: '切换',
},
},
},
},
en: {
translations: {
button: {
buttonText: 'Search',
buttonAriaLabel: 'Search docs',
},
modal: {
noResultsText: 'No results found',
resetButtonTitle: 'Clear query',
footer: {
selectText: 'to select',
navigateText: 'to navigate',
},
},
},
},
},
},
},
outline: {
level: [2, 3],
label: '目录',
},
externalLinkIcon: true,
},
})
-317
View File
@@ -1,317 +0,0 @@
:root {
--vp-c-brand-1: #6d5dfc;
--vp-c-brand-2: #5848e8;
--vp-c-brand-3: #4938ce;
--vp-c-brand-soft: rgba(109, 93, 252, 0.12);
--vp-c-bg: #ffffff;
--vp-c-bg-alt: #f7f7f8;
--vp-c-bg-soft: #f2f2f3;
--vp-c-divider: #e8e8eb;
--vp-c-gutter: #eeeeef;
--vp-c-text-1: #18181b;
--vp-c-text-2: #52525b;
--vp-c-text-3: #71717a;
--vp-button-brand-bg: #18181b;
--vp-button-brand-hover-bg: #2f2f33;
--vp-button-brand-active-bg: #09090b;
--vp-button-brand-text: #ffffff;
--vp-home-hero-name-color: transparent;
--vp-home-hero-name-background: linear-gradient(120deg, #18181b 15%, #6d5dfc 85%);
--vp-home-hero-image-background-image: radial-gradient(circle, rgba(109, 93, 252, 0.24), transparent 66%);
--vp-home-hero-image-filter: blur(48px);
--vp-font-family-base: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", "Noto Sans SC", sans-serif;
--vp-font-family-mono: "SFMono-Regular", Consolas, "Liberation Mono", monospace;
}
.dark {
--vp-c-brand-1: #a99fff;
--vp-c-brand-2: #8d80ff;
--vp-c-brand-3: #7565ff;
--vp-c-brand-soft: rgba(169, 159, 255, 0.14);
--vp-c-bg: #0b0b0d;
--vp-c-bg-alt: #111114;
--vp-c-bg-soft: #18181c;
--vp-c-divider: #27272c;
--vp-c-gutter: #18181c;
--vp-c-text-1: #fafafa;
--vp-c-text-2: #c4c4ca;
--vp-c-text-3: #92929b;
--vp-button-brand-bg: #f4f4f5;
--vp-button-brand-hover-bg: #ffffff;
--vp-button-brand-active-bg: #e4e4e7;
--vp-button-brand-text: #18181b;
--vp-home-hero-name-background: linear-gradient(120deg, #ffffff 12%, #a99fff 88%);
}
html {
scroll-behavior: smooth;
}
body {
letter-spacing: -0.01em;
}
.VPNavBar {
border-bottom: 1px solid color-mix(in srgb, var(--vp-c-divider) 76%, transparent);
background: color-mix(in srgb, var(--vp-c-bg) 86%, transparent) !important;
backdrop-filter: blur(18px) saturate(160%);
}
.VPNavBarTitle .logo {
width: 28px;
height: 28px;
}
.VPNavBarTitle .title {
font-weight: 750;
letter-spacing: -0.03em;
}
.VPNavBarMenuLink,
.VPNavBarMenuGroup .text {
font-size: 14px;
font-weight: 550;
}
.VPSidebar {
border-right: 1px solid var(--vp-c-divider);
background: color-mix(in srgb, var(--vp-c-bg) 96%, var(--vp-c-bg-alt));
}
.VPSidebarItem.level-0 > .item > .text {
font-size: 12px;
font-weight: 700;
letter-spacing: 0.08em;
text-transform: uppercase;
color: var(--vp-c-text-3);
}
.VPSidebarItem.level-1 .link {
margin: 2px 0;
padding: 5px 10px;
border-radius: 8px;
transition: color 0.18s ease, background-color 0.18s ease;
}
.VPSidebarItem.level-1 .link:hover,
.VPSidebarItem.level-1.is-active > .item .link {
background: var(--vp-c-brand-soft);
}
.vp-doc {
font-size: 16px;
}
.vp-doc h1 {
margin-bottom: 18px;
font-size: clamp(32px, 5vw, 44px);
line-height: 1.12;
letter-spacing: -0.045em;
}
.vp-doc h2 {
margin-top: 48px;
padding-top: 12px;
border-top: 1px solid var(--vp-c-divider);
letter-spacing: -0.025em;
}
.vp-doc h3 {
letter-spacing: -0.02em;
}
.vp-doc p,
.vp-doc li {
line-height: 1.8;
}
.vp-doc a {
text-underline-offset: 3px;
}
.vp-doc div[class*="language-"] {
margin: 22px 0;
border: 1px solid var(--vp-c-divider);
border-radius: 14px;
box-shadow: 0 14px 36px rgba(24, 24, 27, 0.06);
}
.dark .vp-doc div[class*="language-"] {
box-shadow: 0 14px 38px rgba(0, 0, 0, 0.24);
}
.vp-doc table {
display: table;
width: 100%;
font-size: 14px;
}
.vp-doc th {
background: var(--vp-c-bg-soft);
}
.custom-block {
border-radius: 12px;
}
.VPHome {
padding-bottom: 72px !important;
}
.VPHero {
padding-top: 80px !important;
}
.VPHero .name,
.VPHero .text {
letter-spacing: -0.055em;
}
.VPHero .tagline {
max-width: 620px;
line-height: 1.65;
}
.VPHero .image-bg {
opacity: 0.7;
}
.VPHero .image-src {
max-width: 250px;
max-height: 250px;
}
.VPButton {
border-radius: 10px !important;
font-weight: 650 !important;
}
.VPFeature {
border: 1px solid var(--vp-c-divider) !important;
border-radius: 16px !important;
background: color-mix(in srgb, var(--vp-c-bg) 94%, var(--vp-c-bg-soft)) !important;
transition: transform 0.2s ease, border-color 0.2s ease, box-shadow 0.2s ease;
}
.VPFeature:hover {
transform: translateY(-3px);
border-color: var(--vp-c-brand-1) !important;
box-shadow: 0 18px 42px rgba(24, 24, 27, 0.08);
}
.VPFeature .icon {
display: inline-flex;
align-items: center;
justify-content: center;
width: 38px;
height: 38px;
border-radius: 10px;
background: var(--vp-c-brand-soft);
color: var(--vp-c-brand-1);
font-family: var(--vp-font-family-mono);
font-size: 12px;
font-weight: 700;
}
.VPFeature .title {
letter-spacing: -0.02em;
}
.VPHome .vp-doc {
max-width: 1152px;
margin: 36px auto 0;
padding: 0 24px;
}
.doc-paths {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 14px;
margin: 28px 0 34px;
}
.doc-path {
display: flex;
min-height: 154px;
flex-direction: column;
padding: 20px;
border: 1px solid var(--vp-c-divider);
border-radius: 14px;
background: var(--vp-c-bg);
color: var(--vp-c-text-1) !important;
text-decoration: none !important;
transition: transform 0.2s ease, border-color 0.2s ease, box-shadow 0.2s ease;
}
.doc-path:hover {
transform: translateY(-3px);
border-color: var(--vp-c-brand-1);
box-shadow: 0 16px 36px rgba(24, 24, 27, 0.08);
}
.doc-path:focus-visible {
outline: 3px solid var(--vp-c-brand-soft);
outline-offset: 3px;
border-color: var(--vp-c-brand-1);
}
.doc-path span {
margin-bottom: 16px;
color: var(--vp-c-brand-1);
font-size: 12px;
font-weight: 700;
letter-spacing: 0.08em;
text-transform: uppercase;
}
.doc-path strong {
margin-bottom: 8px;
font-size: 17px;
letter-spacing: -0.02em;
}
.doc-path small {
margin-top: auto;
color: var(--vp-c-text-2);
line-height: 1.55;
}
@media (max-width: 767px) {
.VPHero {
padding-top: 42px !important;
}
.VPHero .image-src {
max-width: 180px;
max-height: 180px;
}
.VPHome .vp-doc {
margin-top: 20px;
padding: 0 20px;
}
.doc-paths {
grid-template-columns: 1fr;
}
.doc-path {
min-height: 132px;
}
.vp-doc table {
display: block;
}
}
@media (prefers-reduced-motion: reduce) {
html {
scroll-behavior: auto;
}
*,
*::before,
*::after {
transition-duration: 0.01ms !important;
}
}
-6
View File
@@ -1,6 +0,0 @@
import DefaultTheme from 'vitepress/theme'
import './custom.css'
export default {
...DefaultTheme,
}
-334
View File
@@ -1,334 +0,0 @@
# FileCodeBox API 文档
## API 版本: 2.5.6
## 目录
- [认证](#认证)
- [分享接口](#分享接口)
- [管理接口](#管理接口)
## 认证
部分接口需要在请求头中携带 `Authorization` 进行认证:
```
Authorization: Bearer <token>
```
### 获取 Token
当管理面板关闭了游客上传(`openUpload=0`)时,需要先登录获取 token:
```bash
curl -X POST "http://localhost:12345/admin/login" \
-H "Content-Type: application/json" \
-d '{"password": "<初始化时设置的管理员密码>"}'
```
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"token": "xxx.xxx.xxx",
"token_type": "Bearer"
}
}
```
## 分享接口
### 分享文本
**POST** `/share/text/`
分享文本内容,获取分享码。
**请求参数:**
| 参数名 | 类型 | 必填 | 默认值 | 描述 |
|-------|------|------|--------|------|
| text | string | 是 | - | 要分享的文本内容 |
| expire_value | integer | 否 | 1 | 过期时间值 |
| expire_style | string | 否 | "day" | 过期时间单位(day/hour/minute/count/forever) |
**cURL 示例:**
```bash
# 游客上传(openUpload=1 时)
curl -X POST "http://localhost:12345/share/text/" \
-F "text=这是要分享的文本内容" \
-F "expire_value=1" \
-F "expire_style=day"
# 需要认证时(openUpload=0 时)
curl -X POST "http://localhost:12345/share/text/" \
-H "Authorization: Bearer xxx.xxx.xxx" \
-F "text=这是要分享的文本内容"
```
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123"
}
}
```
### 分享文件
**POST** `/share/file/`
上传并分享文件,获取分享码。
**请求参数:**
| 参数名 | 类型 | 必填 | 默认值 | 描述 |
|-------|------|------|--------|------|
| file | file | 是 | - | 要上传的文件 |
| expire_value | integer | 否 | 1 | 过期时间值 |
| expire_style | string | 否 | "day" | 过期时间单位(day/hour/minute/count/forever) |
**cURL 示例:**
```bash
# 上传文件(默认1天有效期)
curl -X POST "http://localhost:12345/share/file/" \
-F "file=@/path/to/file.txt"
# 上传文件(7天有效期)
curl -X POST "http://localhost:12345/share/file/" \
-F "file=@/path/to/file.txt" \
-F "expire_value=7" \
-F "expire_style=day"
# 上传文件(可下载10次)
curl -X POST "http://localhost:12345/share/file/" \
-F "file=@/path/to/file.txt" \
-F "expire_value=10" \
-F "expire_style=count"
# 需要认证时
curl -X POST "http://localhost:12345/share/file/" \
-H "Authorization: Bearer xxx.xxx.xxx" \
-F "file=@/path/to/file.txt"
```
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt"
}
}
```
### 获取文件信息
**GET** `/share/select/`
通过分享码获取文件信息(直接下载文件)。
**请求参数:**
| 参数名 | 类型 | 必填 | 描述 |
|-------|------|------|------|
| code | string | 是 | 文件分享码 |
**cURL 示例:**
```bash
# 通过取件码下载文件
curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
"size": 1024,
"text": "文件内容或下载链接"
}
}
```
### 选择文件
**POST** `/share/select/`
通过分享码选择文件。
**请求参数:**
| 参数名 | 类型 | 必填 | 描述 |
|-------|------|------|------|
| code | string | 是 | 文件分享码 |
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
"size": 1024,
"text": "文件内容或下载链接"
}
}
```
### 下载文件
**GET** `/share/download`
下载分享的文件。
**请求参数:**
| 参数名 | 类型 | 必填 | 描述 |
|-------|------|------|------|
| key | string | 是 | 下载密钥 |
| code | string | 是 | 文件分享码 |
## 管理接口
### 管理员登录
**POST** `/admin/login`
管理员登录获取token。
**请求参数:**
| 参数名 | 类型 | 必填 | 描述 |
|-------|------|------|------|
| password | string | 是 | 管理员密码 |
### 仪表盘数据
**GET** `/admin/dashboard`
获取系统仪表盘数据。
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"total_files": 100,
"storage_used": "1610612736",
"sys_uptime": 1725489600.0,
"yesterday_count": 50,
"yesterday_size": "524288000",
"today_count": 30,
"today_size": "314572800",
}
}
```
### 文件列表
**GET** `/admin/file/list`
获取系统中的文件列表。
**请求参数:**
| 参数名 | 类型 | 必填 | 默认值 | 描述 |
|-------|------|------|--------|------|
| page | integer | 否 | 1 | 当前页码 |
| size | integer | 否 | 10 | 每页数量 |
| keyword | string | 否 | "" | 搜索关键词 |
| sort_by | string | 否 | "created_at" | 排序字段(created_at/expired_at/name/size/used_count/code) |
| sort_order | string | 否 | "desc" | 排序方向(asc/desc) |
**响应示例:**
```json
{
"code": 200,
"message": "success",
"detail": {
"page": 1,
"size": 10,
"total": 100,
"data": [
{
"id": 1,
"name": "example.txt",
"size": 1024,
"created_at": "2024-01-01 12:00:00"
}
]
}
}
```
### 删除文件
**DELETE** `/admin/file/delete`
删除系统中的文件。
**请求参数:**
| 参数名 | 类型 | 必填 | 描述 |
|-------|------|------|------|
| id | integer | 是 | 文件ID |
### 获取配置
**GET** `/admin/config/get`
获取系统配置信息。
### 更新配置
**PATCH** `/admin/config/update`
更新系统配置信息。
## 错误响应
当发生错误时,API会返回对应的错误信息:
```json
{
"code": 422,
"detail": [
{
"loc": ["body", "password"],
"msg": "密码不能为空",
"type": "value_error"
}
]
}
```
## 状态码说明
- 200: 请求成功
- 401: 未授权
- 403: 禁止访问
- 404: 资源不存在
- 422: 请求参数验证错误
- 500: 服务器内部错误
-451
View File
@@ -1,451 +0,0 @@
# 预签名上传 API 文档
## 概述
预签名上传功能提供统一的文件上传接口,根据后端存储类型自动选择最优上传方式:
- **S3 存储**: 返回预签名 URL,客户端直传 S3(减少服务器带宽压力)
- **其他存储**: 返回代理上传 URL,通过服务器中转上传
## 上传流程
### 流程图
```
┌─────────┐ 1. 初始化上传 ┌─────────┐
│ 客户端 │ ──────────────────────▶ │ 服务器 │
└─────────┘ └─────────┘
│ │
│◀─────── 返回 upload_url + mode ──┤
│ │
│ ┌─────────────────────────────────────────┐
│ │ if mode == "direct" (S3存储) │
│ │ 2a. PUT 文件到 upload_url (S3) │
│ │ 3a. POST /confirm 确认上传 │
│ │ │
│ │ if mode == "proxy" (其他存储) │
│ │ 2b. PUT 文件到 upload_url (服务器) │
│ │ (自动返回分享码,无需确认) │
│ └─────────────────────────────────────────┘
│
▼
获取分享码
```
---
## API 端点
### 1. 初始化上传
初始化预签名上传会话,获取上传 URL 和模式。
**请求**
```
POST /presign/upload/init
Content-Type: application/json
```
**请求体**
| 字段 | 类型 | 必填 | 默认值 | 说明 |
| ------------ | ------- | ---- | ------ | --------------------------------------- |
| file_name | string | ✅ | - | 文件名(含扩展名) |
| file_size | integer | ✅ | - | 文件大小(字节) |
| expire_value | integer | ❌ | 1 | 过期时间值 |
| expire_style | string | ❌ | "day" | 过期类型:day/hour/minute/forever/count |
**请求示例**
```json
{
"file_name": "document.pdf",
"file_size": 1048576,
"expire_value": 7,
"expire_style": "day"
}
```
**响应**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"upload_url": "https://bucket.s3.amazonaws.com/path?X-Amz-Signature=...",
"mode": "direct",
"expires_in": 900
}
}
```
**响应字段说明**
| 字段 | 类型 | 说明 |
| ---------- | ------- | ----------------------------------------------------- |
| upload_id | string | 上传会话 ID,后续操作需要 |
| upload_url | string | 上传目标 URL |
| mode | string | 上传模式:`direct`(直传 S3)或 `proxy`(服务器代理) |
| expires_in | integer | URL 有效期(秒),默认 900 秒(15 分钟) |
**错误响应**
| 状态码 | 说明 |
| ------ | ------------------------------ |
| 400 | 过期时间类型错误 |
| 403 | 文件大小超过限制 / IP 频率限制 |
---
### 2a. 直传模式 - 上传文件到 S3
当 `mode == "direct"` 时,客户端直接将文件 PUT 到返回的预签名 URL。
**请求**
```
PUT {upload_url}
Content-Type: application/octet-stream
[文件二进制内容]
```
**注意事项**
- 直接使用返回的 `upload_url`,不要修改
- Content-Type 建议使用 `application/octet-stream`
- 这是直接请求 S3,不经过服务器
**JavaScript 示例**
```javascript
const response = await fetch(uploadUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': 'application/octet-stream',
},
})
if (response.ok) {
// 上传成功,调用确认接口
}
```
---
### 2b. 代理模式 - 上传文件到服务器
当 `mode == "proxy"` 时,客户端将文件 PUT 到服务器代理端点。
**请求**
```
PUT /presign/upload/proxy/{upload_id}
Content-Type: multipart/form-data
file: [文件]
```
**路径参数**
| 参数 | 说明 |
| --------- | ------------------------- |
| upload_id | 初始化时返回的上传会话 ID |
**响应**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**注意**: 代理模式上传成功后直接返回分享码,无需调用确认接口。
**错误响应**
| 状态码 | 说明 |
| ------ | --------------------------------------- |
| 400 | 文件大小与声明不符 / 会话不支持代理上传 |
| 404 | 上传会话不存在或已过期 |
| 500 | 文件保存失败 |
---
### 3. 确认上传(仅直传模式)
直传模式下,客户端完成 S3 上传后调用此接口确认并获取分享码。
**请求**
```
POST /presign/upload/confirm/{upload_id}
Content-Type: application/json
```
**路径参数**
| 参数 | 说明 |
| --------- | ------------------------- |
| upload_id | 初始化时返回的上传会话 ID |
**请求体**
| 字段 | 类型 | 必填 | 默认值 | 说明 |
| ------------ | ------- | ---- | ------ | ---------- |
| expire_value | integer | ❌ | 1 | 过期时间值 |
| expire_style | string | ❌ | "day" | 过期类型 |
**请求示例**
```json
{
"expire_value": 7,
"expire_style": "day"
}
```
**响应**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**错误响应**
| 状态码 | 说明 |
| ------ | --------------------------------------------- |
| 400 | 会话不支持直传确认 |
| 404 | 上传会话不存在或已过期 / 文件未上传或上传失败 |
---
### 4. 查询上传状态
查询上传会话的当前状态。
**请求**
```
GET /presign/upload/status/{upload_id}
```
**响应**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"file_name": "document.pdf",
"file_size": 1048576,
"mode": "direct",
"created_at": "2024-01-01T12:00:00",
"expires_at": "2024-01-01T12:15:00",
"is_expired": false
}
}
```
**错误响应**
| 状态码 | 说明 |
| ------ | -------------- |
| 404 | 上传会话不存在 |
---
### 5. 取消上传
取消上传会话并清理相关资源。
**请求**
```
DELETE /presign/upload/{upload_id}
```
**响应**
```json
{
"code": 200,
"detail": {
"message": "上传会话已取消"
}
}
```
**错误响应**
| 状态码 | 说明 |
| ------ | -------------- |
| 404 | 上传会话不存在 |
---
## 前端集成示例
### 完整上传流程(JavaScript/TypeScript)
```typescript
interface PresignInitResponse {
upload_id: string
upload_url: string
mode: 'direct' | 'proxy'
expires_in: number
}
interface UploadResult {
code: string
name: string
}
async function uploadFile(
file: File,
expireValue: number = 1,
expireStyle: string = 'day'
): Promise<UploadResult> {
// 1. 初始化上传
const initResponse = await fetch('/presign/upload/init', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
file_name: file.name,
file_size: file.size,
expire_value: expireValue,
expire_style: expireStyle,
}),
})
const initData = await initResponse.json()
if (initData.code !== 200) {
throw new Error(initData.detail)
}
const { upload_id, upload_url, mode } = initData.detail as PresignInitResponse
// 2. 根据模式上传文件
if (mode === 'direct') {
// 直传模式:上传到S3
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: file,
headers: { 'Content-Type': 'application/octet-stream' },
})
if (!uploadResponse.ok) {
throw new Error('S3上传失败')
}
// 3. 确认上传
const confirmResponse = await fetch(
`/presign/upload/confirm/${upload_id}`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
expire_value: expireValue,
expire_style: expireStyle,
}),
}
)
const confirmData = await confirmResponse.json()
if (confirmData.code !== 200) {
throw new Error(confirmData.detail)
}
return confirmData.detail
} else {
// 代理模式:上传到服务器
const formData = new FormData()
formData.append('file', file)
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: formData,
})
const uploadData = await uploadResponse.json()
if (uploadData.code !== 200) {
throw new Error(uploadData.detail)
}
return uploadData.detail
}
}
// 使用示例
const file = document.querySelector('input[type="file"]').files[0]
const result = await uploadFile(file, 7, 'day')
console.log('分享码:', result.code)
```
### Vue 3 组件示例
```vue
<template>
<div>
<input type="file" @change="handleFileSelect" />
<button @click="upload" :disabled="!selectedFile || uploading">
{{ uploading ? '上传中...' : '上传' }}
</button>
<div v-if="shareCode">分享码: {{ shareCode }}</div>
<div v-if="error" class="error">{{ error }}</div>
</div>
</template>
<script setup lang="ts">
import { ref } from 'vue'
const selectedFile = ref<File | null>(null)
const uploading = ref(false)
const shareCode = ref('')
const error = ref('')
function handleFileSelect(e: Event) {
const input = e.target as HTMLInputElement
selectedFile.value = input.files?.[0] || null
}
async function upload() {
if (!selectedFile.value) return
uploading.value = true
error.value = ''
try {
const result = await uploadFile(selectedFile.value)
shareCode.value = result.code
} catch (e) {
error.value = e.message
} finally {
uploading.value = false
}
}
</script>
```
---
## 注意事项
1. **会话有效期**: 上传会话默认 15 分钟后过期,请在有效期内完成上传
2. **文件大小限制**: 受系统配置 `upload_size` 限制
3. **过期类型**: 支持 `day`、`hour`、`minute`、`forever`、`count`
4. **CORS**: 直传模式下,S3 需要配置正确的 CORS 策略
5. **重试机制**: 建议实现上传失败重试逻辑
-158
View File
@@ -1,158 +0,0 @@
# Changelog
## [2.8.4](https://github.com/vastsa/FileCodeBox/compare/v2.8.3...v2.8.4) (2026-10-25)
### Performance Improvements
* **db:** tune SQLite connection pragmas for small VPS ([2073f9b](https://github.com/vastsa/FileCodeBox/commit/2073f9ba4b6d17e2a9c81832c7d77468e9a15814))
* **db:** tune SQLite connection pragmas for small VPS ([ec714dc](https://github.com/vastsa/FileCodeBox/commit/ec714dcd6fec2dc2097d05a250ea3962d01d66aa))
* **rate-limit:** bound tracked IPs and use monotonic clock ([496a713](https://github.com/vastsa/FileCodeBox/commit/496a7139bdb2cdac7f516986b613715a693fff13))
* **startup:** drop duplicate Tortoise init, keep ORM exception handlers ([4f141ee](https://github.com/vastsa/FileCodeBox/commit/4f141eeb4940311ad6dcfe2bc11650fa505f825b))
* **web:** cache rendered index.html and add theme asset cache headers ([5b5d70c](https://github.com/vastsa/FileCodeBox/commit/5b5d70c3b7eb3b8efb3d77d731c860e88d1e7a92))
## [2.8.3](https://github.com/vastsa/FileCodeBox/compare/v2.8.2...v2.8.3) (2026-10-10)
### Bug Fixes
* remove HTML preview endpoint ([65f4929](https://github.com/vastsa/FileCodeBox/commit/65f4929d405408e1c9e58127e0097f8b78b1819f))
## [2.8.2](https://github.com/vastsa/FileCodeBox/compare/v2.8.1...v2.8.2) (2026-10-10)
### Bug Fixes
* type MCP protocol results ([da374b4](https://github.com/vastsa/FileCodeBox/commit/da374b459537ee079e1d03da121a935c8fb18f0e))
## [2.8.1](https://github.com/vastsa/FileCodeBox/compare/v2.8.0...v2.8.1) (2026-10-10)
### Bug Fixes
* restore lint and Docker theme build ([2f5eab1](https://github.com/vastsa/FileCodeBox/commit/2f5eab18b1ab28fcb4f416248076ce574998d718))
## [2.8.0](https://github.com/vastsa/FileCodeBox/compare/v2.7.1...v2.8.0) (2026-10-10)
### Features
* add MCP quick share and safe HTML preview ([63e72c1](https://github.com/vastsa/FileCodeBox/commit/63e72c145576d169265e4fa8cf373e33531a06a0))
## [2.7.1](https://github.com/vastsa/FileCodeBox/compare/v2.7.0...v2.7.1) (2026-09-19)
### Bug Fixes
* keep Docker as root, close s3_hostname SSRF, unbreak 404 tests ([031d300](https://github.com/vastsa/FileCodeBox/commit/031d300caea99117e82b3bddc3e5d0532cf3efa6))
## [2.7.0](https://github.com/vastsa/FileCodeBox/compare/v2.6.1...v2.7.0) (2026-09-16)
### Features
* mint extract codes from data/local without re-upload ([93a30a2](https://github.com/vastsa/FileCodeBox/commit/93a30a257cc5b0ee23a374521e1e12602c3959da))
## [2.6.1](https://github.com/vastsa/FileCodeBox/compare/v2.6.0...v2.6.1) (2026-09-14)
### Bug Fixes
* don't let a legacy background value block settings saves ([eb2db8f](https://github.com/vastsa/FileCodeBox/commit/eb2db8f507e3198e0ae45b976bdd90eab2f7d1ed))
* 存量 background 值不再阻塞后台设置保存([#514](https://github.com/vastsa/FileCodeBox/issues/514) 升级回归) ([0d82fbe](https://github.com/vastsa/FileCodeBox/commit/0d82fbe26ad06ca77af7b2b5aa05f514e8362295))
## [2.6.0](https://github.com/vastsa/FileCodeBox/compare/v2.5.6...v2.6.0) (2026-09-14)
### Features
* declare missing config keys (opendal_scheme, chunk_expire_hours) and add config schema guard ([c5d525d](https://github.com/vastsa/FileCodeBox/commit/c5d525df08bb9534d1efb3ab5ff9357b52240fe9))
* rename all config keys to snake_case with DB migration 007 ([5121db6](https://github.com/vastsa/FileCodeBox/commit/5121db6edcdac57318d058e28d7249efb9ea4d03))
* upgrade password hashing to memory-hard scrypt with transparent rehash ([443cfe7](https://github.com/vastsa/FileCodeBox/commit/443cfe7472693bb954845f4cc4ac77e3abfcbd23))
### Bug Fixes
* FileCodes.expired_count defaults to -1 (time-based) instead of instantly-expired 0 (D4) ([3c93718](https://github.com/vastsa/FileCodeBox/commit/3c93718d4b2f3a8e93978c7eb7a4a3b2553b0855))
* save_file accepts plain binary streams; repair admin local file share (D2) ([bb95017](https://github.com/vastsa/FileCodeBox/commit/bb95017aa398e80e5eca512aa9a330aaeeceb364))
* serialize KeyValue JSON read-modify-write sections behind a process lock (D5) ([0038636](https://github.com/vastsa/FileCodeBox/commit/0038636a2a5fc8e73f944c38789dce8de0354c3a))
* snake_case the public config contract (D7 follow-up) ([800bac0](https://github.com/vastsa/FileCodeBox/commit/800bac01285dd6eb827e28460996997dbb80e2c3))
* validate background config as http(s) URL to close CSS quote-breakout in 2023 theme ([4cab625](https://github.com/vastsa/FileCodeBox/commit/4cab625d5ab2fe5921f8f621ebe015623afb6fb8))
### Performance Improvements
* cache settings refresh behind a 2s TTL, force refresh on write paths (D6) ([61d72ab](https://github.com/vastsa/FileCodeBox/commit/61d72ab6ccba5df121945ef825de2967c45888b4))
* dashboard counters and storage quota use SQL aggregation (D3) ([baecebd](https://github.com/vastsa/FileCodeBox/commit/baecebd81efc9677acac29d5335e281d6bb5a252))
* make is_config_initialized cheap — no scrypt on the per-request probe (D1) ([ec424dd](https://github.com/vastsa/FileCodeBox/commit/ec424dd9093d94b69ec3a118f321cd1f85c7dd54))
## [2.5.6](https://github.com/vastsa/FileCodeBox/compare/v2.5.5...v2.5.6) (2026-09-01)
### Bug Fixes
* **logging:** reduce production log noise ([5537f56](https://github.com/vastsa/FileCodeBox/commit/5537f56b884a15eda015665c67c3f8469e4e9d89))
## [2.5.5](https://github.com/vastsa/FileCodeBox/compare/v2.5.4...v2.5.5) (2026-08-29)
### Bug Fixes
* **chunk-upload:** persist sanitized filenames ([da1dd06](https://github.com/vastsa/FileCodeBox/commit/da1dd066551cc7cc9a631a0c982b5675bd4b1967))
* harden chunk upload filename metadata ([5a7195c](https://github.com/vastsa/FileCodeBox/commit/5a7195c5c7d1028500b0e363c6b47130d260e11c))
* sanitize chunk upload file_name to prevent path traversal ([cebac32](https://github.com/vastsa/FileCodeBox/commit/cebac32145f948bfca7c44c663a0bcdcd9c56830))
## [2.5.4](https://github.com/vastsa/FileCodeBox/compare/v2.5.3...v2.5.4) (2026-07-23)
### Bug Fixes
* run Docker as root for volume compatibility ([fbcd4a5](https://github.com/vastsa/FileCodeBox/commit/fbcd4a5b2b788cdfc8e86d920f9c5a7299dc839a))
* run Docker image as root for data volume compatibility ([fc0cbeb](https://github.com/vastsa/FileCodeBox/commit/fc0cbeb4f0c7485d2e7abc0501a5fd9b1456cc22))
## [2.5.3](https://github.com/vastsa/FileCodeBox/compare/v2.5.2...v2.5.3) (2026-07-23)
### Bug Fixes
* add magic-byte checks, login rate limits, and non-root Docker ([eda0a31](https://github.com/vastsa/FileCodeBox/commit/eda0a3127ab562b6a2cf828c0df4a93933c44526))
* magic-byte file checks, admin login rate limit, non-root Docker ([0ee2f04](https://github.com/vastsa/FileCodeBox/commit/0ee2f046dd057fecceaee1059d72d67de3038f19))
## [2.5.2](https://github.com/vastsa/FileCodeBox/compare/v2.5.1...v2.5.2) (2026-07-23)
### Bug Fixes
* harden share expiry, download token, quota SQL, and CORS ([ae77026](https://github.com/vastsa/FileCodeBox/commit/ae7702603a99dc77b9b48e78746058b2a74b0bf5))
* harden share expiry, download token, quota SQL, and CORS ([a41693f](https://github.com/vastsa/FileCodeBox/commit/a41693f90ded0234e0883c399254319bef04924a))
## [2.5.1](https://github.com/vastsa/FileCodeBox/compare/v2.5.0...v2.5.1) (2026-07-21)
### Bug Fixes
* patch CVE dependencies and path traversal risks ([00f3abd](https://github.com/vastsa/FileCodeBox/commit/00f3abd3b20fe2344e4dbf8d53714773e23a1050))
## [2.5.0](https://github.com/vastsa/FileCodeBox/compare/v2.4.0...v2.5.0) (2026-07-11)
### Features
* add global storage capacity limit ([818dd47](https://github.com/vastsa/FileCodeBox/commit/818dd47b2a4136d4b0d68b2484ac2d01a7ae6a3c))
* automate version releases ([ee8f908](https://github.com/vastsa/FileCodeBox/commit/ee8f9082ab59418a4a7721a05fc358215230981f))
* configure admin session lifetime ([#484](https://github.com/vastsa/FileCodeBox/issues/484)) ([93ff291](https://github.com/vastsa/FileCodeBox/commit/93ff291366aaac9d04c5259d3ecf01b20aed3fd0))
### Bug Fixes
* build frontend on native Docker platform ([2c51384](https://github.com/vastsa/FileCodeBox/commit/2c51384c80a9ff3f4eb98eb876759b8949dfae89))
* harden automated release workflow ([46883c9](https://github.com/vastsa/FileCodeBox/commit/46883c90ff52894631fe53bbd9b9c4100a7b27f1))
* harden share retrieval and admin visibility ([#482](https://github.com/vastsa/FileCodeBox/issues/482), [#480](https://github.com/vastsa/FileCodeBox/issues/480)) ([8d7d856](https://github.com/vastsa/FileCodeBox/commit/8d7d856c62d73badd0797eb4daec8d2ff10a403a))
* resolve docs dependency audit warnings ([36300ef](https://github.com/vastsa/FileCodeBox/commit/36300ef54ee09081a6cf6a5fe4e69cee7aad0162))
## [Unreleased]
### Added
- 增加全局存储容量上限配置,并对普通上传、分片上传、预签名直传和后台文件分享统一执行并发安全的容量预留。
- 增加存储预留迁移及过期上传清理,失败、取消或过期的上传会释放预留容量。
View File
Whitespace-only changes.
-334
View File
@@ -1,334 +0,0 @@
# FileCodeBox API Documentation
## API Version: 2.5.6
## Table of Contents
- [Authentication](#authentication)
- [Share API](#share-api)
- [Admin API](#admin-api)
## Authentication
Some APIs require `Authorization` header for authentication:
```
Authorization: Bearer <token>
```
### Get Token
When guest upload is disabled in admin panel (`openUpload=0`), you need to login first:
```bash
curl -X POST "http://localhost:12345/admin/login" \
-H "Content-Type: application/json" \
-d '{"password": "<admin-password-set-during-setup>"}'
```
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"token": "xxx.xxx.xxx",
"token_type": "Bearer"
}
}
```
## Share API
### Share Text
**POST** `/share/text/`
Share text content and get a share code.
**Parameters:**
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| text | string | Yes | - | Text content to share |
| expire_value | integer | No | 1 | Expiration time value |
| expire_style | string | No | "day" | Expiration time unit(day/hour/minute/count/forever) |
**cURL Example:**
```bash
# Guest upload (when openUpload=1)
curl -X POST "http://localhost:12345/share/text/" \
-F "text=This is the text content to share" \
-F "expire_value=1" \
-F "expire_style=day"
# When authentication required (openUpload=0)
curl -X POST "http://localhost:12345/share/text/" \
-H "Authorization: Bearer xxx.xxx.xxx" \
-F "text=This is the text content to share"
```
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123"
}
}
```
### Share File
**POST** `/share/file/`
Upload and share a file, get a share code.
**Parameters:**
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| file | file | Yes | - | File to upload |
| expire_value | integer | No | 1 | Expiration time value |
| expire_style | string | No | "day" | Expiration time unit(day/hour/minute/count/forever) |
**cURL Example:**
```bash
# Upload file (default 1 day expiration)
curl -X POST "http://localhost:12345/share/file/" \
-F "file=@/path/to/file.txt"
# Upload file (7 days expiration)
curl -X POST "http://localhost:12345/share/file/" \
-F "file=@/path/to/file.txt" \
-F "expire_value=7" \
-F "expire_style=day"
# Upload file (10 downloads limit)
curl -X POST "http://localhost:12345/share/file/" \
-F "file=@/path/to/file.txt" \
-F "expire_value=10" \
-F "expire_style=count"
# When authentication required
curl -X POST "http://localhost:12345/share/file/" \
-H "Authorization: Bearer xxx.xxx.xxx" \
-F "file=@/path/to/file.txt"
```
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt"
}
}
```
### Get File Info
**GET** `/share/select/`
Get file information by share code (direct file download).
**Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| code | string | Yes | File share code |
**cURL Example:**
```bash
# Download file by extraction code
curl -L "http://localhost:12345/share/select/?code=abc123" -o downloaded_file
```
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
"size": 1024,
"text": "File content or download link"
}
}
```
### Select File
**POST** `/share/select/`
Select file by share code.
**Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| code | string | Yes | File share code |
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"code": "abc123",
"name": "example.txt",
"size": 1024,
"text": "File content or download link"
}
}
```
### Download File
**GET** `/share/download`
Download shared file.
**Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| key | string | Yes | Download key |
| code | string | Yes | File share code |
## Admin API
### Admin Login
**POST** `/admin/login`
Admin login to get token.
**Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| password | string | Yes | Admin password |
### Dashboard Data
**GET** `/admin/dashboard`
Get system dashboard data.
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"total_files": 100,
"storage_used": "1610612736",
"sys_uptime": 1725489600.0,
"yesterday_count": 50,
"yesterday_size": "524288000",
"today_count": 30,
"today_size": "314572800",
}
}
```
### File List
**GET** `/admin/file/list`
Get system file list.
**Parameters:**
| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| page | integer | No | 1 | Current page |
| size | integer | No | 10 | Page size |
| keyword | string | No | "" | Search keyword |
| sort_by | string | No | "created_at" | Sort field(created_at/expired_at/name/size/used_count/code) |
| sort_order | string | No | "desc" | Sort direction(asc/desc) |
**Response Example:**
```json
{
"code": 200,
"message": "success",
"detail": {
"page": 1,
"size": 10,
"total": 100,
"data": [
{
"id": 1,
"name": "example.txt",
"size": 1024,
"created_at": "2024-01-01 12:00:00"
}
]
}
}
```
### Delete File
**DELETE** `/admin/file/delete`
Delete file from system.
**Parameters:**
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| id | integer | Yes | File ID |
### Get Config
**GET** `/admin/config/get`
Get system configuration.
### Update Config
**PATCH** `/admin/config/update`
Update system configuration.
## Error Response
When an error occurs, the API will return corresponding error message:
```json
{
"code": 422,
"detail": [
{
"loc": ["body", "password"],
"msg": "Password cannot be empty",
"type": "value_error"
}
]
}
```
## Status Codes
- 200: Success
- 401: Unauthorized
- 403: Forbidden
- 404: Not Found
- 422: Validation Error
- 500: Internal Server Error
-389
View File
@@ -1,389 +0,0 @@
# Presigned Upload API
## Overview
The presigned upload feature provides a unified file upload interface that automatically picks the optimal upload method based on the configured storage backend:
- **S3 storage**: returns a presigned URL so the client uploads directly to S3 (reduces server bandwidth usage)
- **Other storage**: returns a proxy upload URL and the file is transferred through the server
## Upload Flow
### Flow Diagram
```
┌─────────┐ 1. Initialize upload ┌─────────┐
│ Client │ ───────────────────────▶ │ Server │
└─────────┘ └─────────┘
│ │
│◀──── upload_url + mode ──────────┤
│ │
│ ┌─────────────────────────────────────────┐
│ │ if mode == "direct" (S3 storage) │
│ │ 2a. PUT file to upload_url (S3) │
│ │ 3a. POST /confirm to finish │
│ │ │
│ │ if mode == "proxy" (other storage) │
│ │ 2b. PUT file to upload_url (server) │
│ │ (share code returned automatically, │
│ │ no confirmation needed) │
│ └─────────────────────────────────────────┘
│
▼
Get share code
```
---
## API Endpoints
### 1. Initialize Upload
Initialize a presigned upload session and get the upload URL and mode.
**Request**
```
POST /presign/upload/init
Content-Type: application/json
```
**Request Body**
| Field | Type | Required | Default | Description |
| ------------ | ------- | -------- | ------- | --------------------------------------- |
| file_name | string | ✅ | - | File name (with extension) |
| file_size | integer | ✅ | - | File size in bytes |
| expire_value | integer | ❌ | 1 | Expiration time value |
| expire_style | string | ❌ | "day" | Expiration type: day/hour/minute/forever/count |
**Request Example**
```json
{
"file_name": "document.pdf",
"file_size": 1048576,
"expire_value": 7,
"expire_style": "day"
}
```
**Response**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"upload_url": "https://bucket.s3.amazonaws.com/path?X-Amz-Signature=...",
"mode": "direct",
"expires_in": 900
}
}
```
**Response Fields**
| Field | Type | Description |
| ---------- | ------- | --------------------------------------------------------------- |
| upload_id | string | Upload session ID, required by all subsequent operations |
| upload_url | string | Target upload URL |
| mode | string | Upload mode: `direct` (direct to S3) or `proxy` (via server) |
| expires_in | integer | URL validity in seconds, default 900 seconds (15 minutes) |
In `proxy` mode the response also includes `proxy_upload_url` (and `legacy_proxy_upload_url`); `upload_url` equals the proxy URL.
**Error Responses**
| Status | Description |
| ------ | -------------------------------------------------- |
| 400 | Invalid expiration type |
| 403 | File size exceeds limit / IP rate limit exceeded |
---
### 2a. Direct Mode - Upload File to S3
When `mode == "direct"`, the client PUTs the file directly to the returned presigned URL.
**Request**
```
PUT {upload_url}
Content-Type: application/octet-stream
[binary file content]
```
**Notes**
- Use the returned `upload_url` as-is, do not modify it
- `application/octet-stream` is the recommended Content-Type
- The request goes directly to S3, it does not pass through the server
**JavaScript Example**
```javascript
const response = await fetch(uploadUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': 'application/octet-stream',
},
})
if (response.ok) {
// Upload succeeded, call the confirm endpoint
}
```
---
### 2b. Proxy Mode - Upload File to Server
When `mode == "proxy"`, the client PUTs the file to the server proxy endpoint.
**Request**
```
PUT /presign/upload/proxy/{upload_id}
Content-Type: multipart/form-data
file: [file]
```
**Path Parameters**
| Parameter | Description |
| --------- | ----------------------------------------------- |
| upload_id | Upload session ID returned by the init endpoint |
**Response**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**Note**: In proxy mode the share code is returned immediately after upload; no confirmation call is needed.
**Error Responses**
| Status | Description |
| ------ | ------------------------------------------------------------------ |
| 400 | File size mismatches declared size / session does not support proxy |
| 404 | Upload session does not exist or has expired |
| 500 | Failed to save file |
---
### 3. Confirm Upload (direct mode only)
In direct mode, call this endpoint after the S3 upload completes to obtain the share code.
**Request**
```
POST /presign/upload/confirm/{upload_id}
Content-Type: application/json
```
**Path Parameters**
| Parameter | Description |
| --------- | ----------------------------------------------- |
| upload_id | Upload session ID returned by the init endpoint |
**Response**
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "document.pdf"
}
}
```
**Error Responses**
| Status | Description |
| ------ | ------------------------------------------------------------------------------- |
| 400 | Session does not support direct confirmation |
| 404 | Upload session does not exist or has expired / file not uploaded or upload failed |
---
### 4. Query Upload Status
Query the current state of an upload session.
**Request**
```
GET /presign/upload/status/{upload_id}
```
**Response**
```json
{
"code": 200,
"detail": {
"upload_id": "a1b2c3d4e5f6...",
"file_name": "document.pdf",
"file_size": 1048576,
"mode": "direct",
"created_at": "2024-01-01T12:00:00",
"expires_at": "2024-01-01T12:15:00",
"is_expired": false
}
}
```
**Error Responses**
| Status | Description |
| ------ | ---------------------------------- |
| 404 | Upload session does not exist |
---
### 5. Cancel Upload
Cancel an upload session and clean up related resources.
**Request**
```
DELETE /presign/upload/{upload_id}
```
**Response**
```json
{
"code": 200,
"detail": {
"message": "Upload session cancelled"
}
}
```
**Error Responses**
| Status | Description |
| ------ | ---------------------------------- |
| 404 | Upload session does not exist |
---
## Frontend Integration Example
### Complete Upload Flow (JavaScript/TypeScript)
```typescript
interface PresignInitResponse {
upload_id: string
upload_url: string
mode: 'direct' | 'proxy'
expires_in: number
}
interface UploadResult {
code: string
name: string
}
async function uploadFile(
file: File,
expireValue: number = 1,
expireStyle: string = 'day'
): Promise<UploadResult> {
// 1. Initialize upload
const initResponse = await fetch('/presign/upload/init', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
file_name: file.name,
file_size: file.size,
expire_value: expireValue,
expire_style: expireStyle,
}),
})
const initData = await initResponse.json()
if (initData.code !== 200) {
throw new Error(initData.detail)
}
const { upload_id, upload_url, mode } = initData.detail as PresignInitResponse
// 2. Upload file based on mode
if (mode === 'direct') {
// Direct mode: upload to S3
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: file,
headers: { 'Content-Type': 'application/octet-stream' },
})
if (!uploadResponse.ok) {
throw new Error('S3 upload failed')
}
// 3. Confirm upload
const confirmResponse = await fetch(
`/presign/upload/confirm/${upload_id}`,
{
method: 'POST',
headers: { 'Content-Type': 'application/json' },
}
)
const confirmData = await confirmResponse.json()
if (confirmData.code !== 200) {
throw new Error(confirmData.detail)
}
return confirmData.detail
} else {
// Proxy mode: upload through the server
const formData = new FormData()
formData.append('file', file)
const uploadResponse = await fetch(upload_url, {
method: 'PUT',
body: formData,
})
const uploadData = await uploadResponse.json()
if (uploadData.code !== 200) {
throw new Error(uploadData.detail)
}
return uploadData.detail
}
}
// Usage example
const file = document.querySelector('input[type="file"]').files[0]
const result = await uploadFile(file, 7, 'day')
console.log('Share code:', result.code)
```
---
## Notes
1. **Session validity**: upload sessions expire after 15 minutes by default; complete the upload within the window
2. **File size limit**: bounded by the `upload_size` system setting
3. **Expiration types**: `day`, `hour`, `minute`, `forever`, `count`
4. **CORS**: in direct mode, S3 must have a proper CORS policy configured
5. **Retry**: implement upload retry logic on the client side for failed uploads
-8
View File
@@ -1,8 +0,0 @@
## [Unreleased]
<!-- Current release: 2.8.4 x-release-please-version -->
### Added
- Added a global storage capacity limit with concurrency-safe reservations for regular, chunked, presigned, and admin file uploads.
- Added the storage reservation migration and stale upload cleanup so failed, cancelled, or expired uploads release reserved capacity.
View File
Whitespace-only changes.
-214
View File
@@ -1,214 +0,0 @@
# Configuration Guide
FileCodeBox provides rich configuration options. Prefer the admin panel; directly editing the database can introduce invalid types or break security settings.
## Configuration Methods
FileCodeBox supports two configuration methods:
1. **Admin Panel Configuration** (Recommended): Access `/admin` to enter the admin panel and modify settings on the settings page
2. **Database Configuration**: Configuration is stored in the `data/filecodebox.db` database
::: tip Note
On first startup, the system uses default configuration from `core/settings.py`. Modified configurations are saved to the database.
:::
## Basic Settings
### Site Information
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `name` | string | `文件快递柜 - FileCodeBox` | Site name, displayed in page title and navigation bar |
| `description` | string | `开箱即用的文件快传系统` | Site description, used for SEO |
| `keywords` | string | `FileCodeBox, 文件快递柜...` | Site keywords, used for SEO |
| `server_host` | string | `0.0.0.0` | Service listening address |
| `server_port` | int | `12345` | Service listening port |
| `server_workers` | int | `1` | Worker count; keep one worker for SQLite deployments |
### Notification Settings
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `notify_title` | string | `系统通知` | Notification title |
| `notify_content` | string | Welcome message | Notification content, supports HTML |
| `page_explain` | string | Legal disclaimer | Footer explanation text |
| `robots_text` | string | `User-agent: *\nDisallow: /` | robots.txt content |
## Upload Settings
### File Upload Limits
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `open_upload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
| `upload_size` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `enable_chunk` | int | `0` | Enable chunked upload (1=enabled, 0=disabled) |
| `allowed_file_types` | list | `["*"]` | Allowed extensions; `*` allows every file type |
::: warning Note
`upload_size` is in bytes. 10MB = 10 * 1024 * 1024 = 10485760 bytes
:::
### Upload Rate Limiting
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `upload_minute` | int | `1` | Upload limit time window (minutes) |
| `upload_count` | int | `10` | Maximum uploads allowed within the time window |
Example: Default configuration allows up to 10 uploads per minute.
### File Expiration Settings
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `expire_style` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `max_save_seconds` | int | `0` | Maximum file retention time (seconds), 0 means no limit |
Expiration methods explained:
- `day` - Expire by days
- `hour` - Expire by hours
- `minute` - Expire by minutes
- `forever` - Never expire
- `count` - Expire by download count
## Theme Settings
### Theme Selection
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `themes_select` | string | `themes/2024` | Currently active theme |
| `themes_choices` | list | See below | Available themes list |
Default available themes:
```json
[
{
"name": "2023",
"key": "themes/2023",
"author": "Lan",
"version": "1.0"
},
{
"name": "2024",
"key": "themes/2024",
"author": "Lan",
"version": "1.0"
}
]
```
### Interface Style
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `opacity` | float | `0.9` | Interface opacity (0-1) |
| `background` | string | `""` | Custom background image URL, empty uses default background |
## Admin Settings
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `admin_token` | string | Set during setup | Admin login password |
| `show_admin_addr` | int | `0` | Show admin panel entry on homepage (1=show, 0=hide) |
| `admin_session_expire` | int | `2592000` | Admin session lifetime in seconds (30 days) |
::: danger Security Warning
The setup page is shown automatically while the system is uninitialized. Complete setup before exposing a production service to the public internet.
:::
## Security Settings
### Error Rate Limiting
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `error_minute` | int | `1` | Error limit time window (minutes) |
| `error_count` | int | `10` | Maximum errors allowed within the time window |
| `trusted_proxies` | list | `[]` | Trusted reverse-proxy IPs used to resolve client addresses |
This setting prevents brute-force attacks on extraction codes.
## Storage Settings
### Storage Type
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `file_storage` | string | `local` | Storage backend type |
| `storage_path` | string | `""` | Custom storage path |
| `storage_limit` | int | `0` | Total storage quota in bytes; 0 means unlimited |
Supported storage types:
- `local` - Local storage
- `s3` - S3-compatible storage (AWS S3, Aliyun OSS, MinIO, etc.)
- `onedrive` - OneDrive storage
- `webdav` - WebDAV storage
- `opendal` - OpenDAL storage
For detailed storage configuration, see [Storage Configuration](/en/guide/storage).
## Configuration Examples
### Example 1: Small Personal Use
Suitable for personal or small team use with relaxed limits:
```python
{
"name": "My File Share",
"upload_size": 52428800, # 50MB
"upload_minute": 5, # 5 minutes
"upload_count": 20, # Max 20 uploads
"expire_style": ["day", "hour", "forever"],
"show_admin_addr": 1
}
```
### Example 2: Public Service
Suitable for public services requiring stricter limits:
```python
{
"name": "Public File Box",
"upload_size": 10485760, # 10MB
"upload_minute": 1, # 1 minute
"upload_count": 5, # Max 5 uploads
"error_minute": 5, # 5 minutes
"error_count": 3, # Max 3 errors
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # Max retention 1 day
"show_admin_addr": 0
}
```
### Example 3: Enterprise Internal Use
Suitable for enterprise internal use with large file and chunked upload support:
```python
{
"name": "Enterprise File Transfer",
"upload_size": 1073741824, # 1GB
"enable_chunk": 1, # Enable chunked upload
"upload_minute": 10, # 10 minutes
"upload_count": 100, # Max 100 uploads
"expire_style": ["day", "forever"],
"file_storage": "s3", # Use S3 storage
"show_admin_addr": 1
}
```
::: warning Configuration examples
These dictionaries illustrate combinations of values; they are not editable configuration files. Set the admin password through first-run setup or the admin panel.
:::
## Next Steps
- [Storage Configuration](/en/guide/storage) - Learn how to configure different storage backends
- [Security Settings](/en/guide/security) - Learn how to enhance system security
- [File Sharing](/en/guide/share) - Learn about file sharing features
-91
View File
@@ -1,91 +0,0 @@
# Getting Started
## Introduction
FileCodeBox is a simple and efficient file sharing tool that supports temporary file transfer, sharing, and management. This guide will help you quickly deploy and use FileCodeBox.
## Features
- 🚀 Quick Deployment: Support Docker one-click deployment
- 🔒 Secure & Reliable: File access requires extraction code
- ⏱️ Time Control: Support setting file expiration time
- 📊 Download Limit: Can limit file download times
- 🖼️ File Preview: Support preview of images, videos, audio, and other formats
- 📱 Responsive Design: Perfect adaptation for mobile and desktop
## Deployment Methods
### Docker Deployment (Recommended)
```bash
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
The named volume persists configuration and uploaded files across container upgrades.
| Variable | Default | Description |
|----------|---------|-------------|
| `HOST` | `0.0.0.0` | Service listening address |
| `PORT` | `12345` | Service port |
| `WORKERS` | `1` | Worker count; keep one worker for SQLite deployments |
| `APP_ENV` | `production` (Docker) | Runtime environment; production disables HTTP access logs by default |
| `LOG_LEVEL` | `warning` (Docker) | `debug`, `info`, `warning`, or `error` |
| `ACCESS_LOG` | `false` (production) | Enable HTTP access logs; set to `true` when debugging requests |
| `TIMEOUT_KEEP_ALIVE` | `5` | HTTP Keep-Alive idle timeout in seconds; raise it behind a reverse proxy to reduce reconnects |
| `MALLOC_TRIM_THRESHOLD_` | `65536` | glibc heap trim threshold in bytes; lowers resident memory after large transfers (glibc images only) |
Production deployments output only warning, error, and other key events by default. Docker retains at most three 10 MB log files.
### Manual Deployment
1. Clone the repository
```bash
git clone https://github.com/vastsa/FileCodeBox.git
```
2. Install dependencies
```bash
cd FileCodeBox
pip install -r requirements.txt
```
3. Start the service
```bash
python main.py
```
## Usage
1. Access the System
Open browser and visit `http://localhost:12345`
2. Upload Files
- Click upload button or drag files to upload area
- Set file expiration time and download limit
- Get share link and extraction code
3. Download Files
- Visit share link
- Enter extraction code
- Download file
4. Admin Panel
- Open `http://localhost:12345` on first run and complete setup
- Visit `http://localhost:12345/#/admin`
- Enter the admin password set during setup
- Enter admin panel
- View system information, files, activity records, and configuration
## Next Steps
- [Storage Configuration](/en/guide/storage) - Learn how to configure different storage methods
- [Security Settings](/en/guide/security) - Learn how to enhance system security
- [API Documentation](/en/api/) - Learn how to integrate through API
-251
View File
@@ -1,251 +0,0 @@
<div align="center">
<img src="/screenshots/retrieve.webp" alt="FileCodeBox" width="860" />
<p><em>Share text and files anonymously with a passcode, like picking up a package</em></p>
[![GitHub stars](https://img.shields.io/github/stars/vastsa/FileCodeBox)](https://github.com/vastsa/FileCodeBox/stargazers)
[![GitHub forks](https://img.shields.io/github/forks/vastsa/FileCodeBox)](https://github.com/vastsa/FileCodeBox/network)
[![GitHub issues](https://img.shields.io/github/issues/vastsa/FileCodeBox)](https://github.com/vastsa/FileCodeBox/issues)
[![GitHub license](https://img.shields.io/github/license/vastsa/FileCodeBox)](https://github.com/vastsa/FileCodeBox/blob/master/LICENSE)
[![QQ Group](https://img.shields.io/badge/QQ%20Group-739673698-blue.svg)](https://qm.qq.com/q/PemPzhdEIM)
[![Python Version](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://www.python.org)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.68+-green.svg)](https://fastapi.tiangolo.com)
[![Vue Version](https://img.shields.io/badge/Vue.js-3.x-brightgreen.svg)](https://v3.vuejs.org)
</div>
## 📝 Introduction
FileCodeBox is a lightweight file sharing tool developed with FastAPI + Vue3. It allows users to share text and files easily, where recipients only need a passcode to retrieve the files, just like picking up a package from a delivery locker.
## 🖼️ Preview
<div align="center">
<h3>
<a href="https://github.com/vastsa/FileCodeBoxFronted" target="_blank">
<img src="https://img.shields.io/badge/Frontend-FileCodeBoxFronted-blue?style=for-the-badge&logo=github" alt="Frontend Repository">
</a>
&nbsp;&nbsp;&nbsp;
<a href="https://share.lanol.cn" target="_blank">
<img src="https://img.shields.io/badge/Demo-share.lanol.cn-green?style=for-the-badge&logo=internet-explorer" alt="Demo Site">
</a>
</h3>
</div>
## 🎯 Use Cases
<table>
<tr>
<td align="center">
<h4>📁 Temporary File Sharing</h4>
Quick file sharing without registration
</td>
<td align="center">
<h4>📝 Quick Text Sharing</h4>
Share code snippets and text content
</td>
<td align="center">
<h4>🕶️ Anonymous Transfer</h4>
Privacy-protected file transfer
</td>
</tr>
<tr>
<td align="center">
<h4>💾 Temporary Storage</h4>
File storage with expiration time
</td>
<td align="center">
<h4>🔄 Cross-platform Transfer</h4>
Quick file transfer between devices
</td>
<td align="center">
<h4>🌐 Private Share Service</h4>
Build your own file sharing service
</td>
</tr>
</table>
## ✨ Core Features
<table>
<tr>
<td align="center">
<h4>🚀 Lightweight</h4>
Based on FastAPI + SQLite3 + Vue3 + ElementUI
</td>
<td align="center">
<h4>📤 Easy Upload</h4>
Support copy-paste and drag-drop
</td>
<td align="center">
<h4>📦 Multiple Types</h4>
Support text and various file types
</td>
</tr>
<tr>
<td align="center">
<h4>🔒 Security</h4>
- IP upload limits
- Error attempt limits
- File expiration
</td>
<td align="center">
<h4>🎫 Passcode Sharing</h4>
Random codes with customizable limits
</td>
<td align="center">
<h4>🌍 Multi-language</h4>
Support for Simplified Chinese, Traditional Chinese, and English
</td>
</tr>
<tr>
<td align="center">
<h4>🎭 Anonymous</h4>
No registration required
</td>
<td align="center">
<h4>🛠 Admin Panel</h4>
File and system management
</td>
<td align="center">
<h4>🐳 Docker</h4>
One-click deployment
</td>
</tr>
<tr>
<td align="center">
<h4>💾 Storage Options</h4>
Local, S3, OneDrive support
</td>
<td align="center">
<h4>📱 Responsive</h4>
Mobile-friendly design
</td>
<td align="center">
<h4>💻 CLI Support</h4>
Command-line download
</td>
</tr>
</table>
## 🚀 Quick Start
### Docker Deployment
```bash
docker run -d --restart unless-stopped \
-p 12345:12345 \
-v fcb-data:/app/data \
-e APP_ENV=production \
-e LOG_LEVEL=warning \
--log-opt max-size=10m \
--log-opt max-file=3 \
--name filecodebox \
lanol/filecodebox:latest
```
### Manual Deployment
1. Clone the repository
```bash
git clone https://github.com/vastsa/FileCodeBox.git
```
2. Install dependencies
```bash
cd FileCodeBox
pip install -r requirements.txt
```
3. Start the service
```bash
python main.py
```
## 📖 Usage Guide
### Share Files
1. Open the website, click "Share File"
2. Select or drag files
3. Set expiration time and count
4. Get the passcode
### Retrieve Files
1. Open the website, enter passcode
2. Click retrieve
3. Download file or view text
### Admin Panel
1. Visit `/admin`
2. Enter admin password
3. Manage files and settings
## 🛠 Development Guide
### Project Structure
```
FileCodeBox/
├── apps/ # Application code
│ ├── admin/ # Admin backend
│ └── base/ # Base functions
├── core/ # Core functions
├── data/ # Data directory
└── fcb-fronted/ # Frontend code
```
### Development Environment
- Python 3.8+
- Node.js 14+
- Vue 3
- FastAPI
### Local Development
1. Backend development
```bash
python main.py
```
2. Frontend development
```bash
cd fcb-fronted
npm install
npm run dev
```
## 🤝 Contributing
1. Fork the project
2. Create your feature branch `git checkout -b feature/xxx`
3. Commit your changes `git commit -m 'Add xxx'`
4. Push to the branch `git push origin feature/xxx`
5. Open a Pull Request
## ❓ FAQ
### Q: How to modify upload size limit?
A: Change `upload_size` in admin panel
### Q: How to configure storage engine?
A: Select storage engine and configure parameters in admin panel
### Q: How to backup data?
A: Backup the `data` directory
For more questions, visit [Wiki](https://github.com/vastsa/FileCodeBox/wiki/常见问题)
## 😀 Project Statistics and Analytics
<div align="center">
<a href="https://hellogithub.com/repository/75ad7ffedd404a6485b4d621ec5b47e6" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=75ad7ffedd404a6485b4d621ec5b47e6&claim_uid=beSz6INEkCM4mDH" alt="Featured|HelloGitHub" style="width: 200px; height: 45px;" width="200" height="45" /></a>
![Repobeats](https://repobeats.axiom.co/api/embed/7a6c92f1d96ee57e6fb67f0df371528397b0c9ac.svg)
[![Star History](https://api.star-history.com/svg?repos=vastsa/FileCodeBox&type=Date)](https://star-history.com/#vastsa/FileCodeBox&Date)
</div>
## 📜 Disclaimer
This project is open-source for learning purposes only. It should not be used for any illegal purposes. The author is not responsible for any consequences. Please retain the project address and copyright information when using it.
-431
View File
@@ -1,431 +0,0 @@
# Admin Panel
FileCodeBox provides a fully-featured admin panel that allows administrators to conveniently manage files, view system status, and modify configurations. This document introduces the various features and usage of the admin panel.
## Accessing the Admin Panel
<p align="center">
<img src="/screenshots/login.webp" alt="Admin login" width="640" />
</p>
### Login Method
The admin panel is located at the `/admin` path. Access method:
1. On first run, set the admin password on the setup page
2. Visit `http://your-domain.com/admin` in your browser
3. Enter the admin password (the value of the `admin_token` configuration)
4. Click the login button
::: tip Tip
The setup page is shown automatically while the system is uninitialized. Complete setup before exposing a production service to the public internet.
:::
### Show Admin Entry
By default, the admin panel entry is not shown on the homepage. You can control whether to show it via configuration:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `show_admin_addr` | int | `0` | Show admin entry on homepage (1=show, 0=hide) |
::: warning Security Recommendation
For public services, it's recommended to keep `show_admin_addr` at `0` and access the admin panel directly via the `/admin` path to reduce the risk of malicious scanning.
:::
### Authentication Mechanism
The admin panel uses JWT (JSON Web Token) for authentication:
1. After successful login, the server returns a Token containing admin identity
2. Subsequent requests carry the Token via `Authorization: Bearer <token>` header
3. Token is used to verify admin identity, ensuring only authorized users can access admin functions
## Dashboard
<p align="center">
<img src="/screenshots/dashboard.webp" alt="Dashboard" width="860" />
</p>
After logging in, you first see the dashboard page, which displays the overall system status.
### Statistics
The dashboard displays the following key metrics:
| Metric | Description |
|--------|-------------|
| **Total Files** (`totalFiles`) | Total number of files stored in the system |
| **Storage Used** (`storageUsed`) | Total storage space occupied by all files (bytes) |
| **System Uptime** (`sysUptime`) | Time when the system was first started |
| **Yesterday's Uploads** (`yesterdayCount`) | Number of files uploaded yesterday |
| **Yesterday's Upload Size** (`yesterdaySize`) | Total size of files uploaded yesterday (bytes) |
| **Today's Uploads** (`todayCount`) | Number of files uploaded today so far |
| **Today's Upload Size** (`todaySize`) | Total size of files uploaded today (bytes) |
### Metric Notes
- **Total Files**: Includes all unexpired files and text shares
- **Storage Used**: Shows actual storage space occupied by files, excluding database and other system files
- **Yesterday/Today Statistics**: Calculated based on file creation time, useful for understanding system usage trends
::: tip Tip
Storage usage is displayed in bytes. For example, `10485760` represents approximately 10MB.
:::
## File Management
<p align="center">
<img src="/screenshots/files.webp" alt="File management" width="860" />
</p>
### File List
The file management page displays all shared files in the system, supporting pagination and search.
**List information includes:**
- File ID
- Extraction code (code)
- Filename prefix (prefix)
- File extension (suffix)
- File size
- Creation time
- Expiration time
- Remaining download count
### Search Files
Use the search function to quickly find specific files:
1. Enter keywords in the search box
2. System performs fuzzy matching based on filename prefix (prefix)
3. Search results update in real-time
**Search examples:**
- Enter `report` to find all files with "report" in the filename
- Enter `.pdf` to find all PDF files (if the filename contains this string)
### Pagination
The file list supports paginated display:
| Parameter | Default | Description |
|-----------|---------|-------------|
| `page` | `1` | Current page number |
| `size` | `10` | Items per page |
### Delete Files
Administrators can delete any file:
1. Find the file to delete in the file list
2. Click the delete button
3. Confirm the delete operation
::: danger Warning
Delete operations are irreversible! Files will be permanently deleted from the storage backend, and database records will also be removed.
:::
**Delete process:**
1. System first deletes the actual file from the storage backend (local/S3/OneDrive, etc.)
2. Then deletes the file record from the database
3. After deletion, the corresponding extraction code becomes invalid
### Download Files
Administrators can directly download any file:
1. Find the target file in the file list
2. Click the download button
3. File will be downloaded via browser
For text shares, the system returns text content directly instead of downloading a file.
### Modify File Information
Administrators can modify some information of shared files:
| Modifiable Field | Description |
|------------------|-------------|
| `code` | Extraction code (must be unique, cannot duplicate other files) |
| `prefix` | Filename prefix |
| `suffix` | File extension |
| `expired_at` | Expiration time |
| `expired_count` | Remaining download count |
**Modify extraction code:**
```
Original code: abc123
New code: myfile2024
```
::: warning Note
When modifying extraction codes, the system checks if the new code is already in use. If an identical extraction code exists, the modification will fail.
:::
## Local File Management
In addition to managing shared files, the admin panel has a Local Share page for browsing `data/local` (including subfolders) and minting extract codes.
Mount a NAS share at `/app/data/local` to generate extract codes for files already on disk, without downloading and re-uploading them.
```yaml
volumes:
- fcb-data:/app/data:rw
# Read-only NAS share for admin browsing and extract-code generation
- /volume1/share:/app/data/local:ro
```
### View Local Files
The Local Share page supports nested folders. List API:
```
GET /admin/local/lists?path=movies
```
`path` is relative to `data/local`; leave it empty for the root. Response:
| Field | Description |
|------|------|
| `path` | Current directory |
| `parent` | Parent directory |
| `truncated` | Whether the listing was truncated |
| `items[].type` | `dir` or `file` |
| `items[].path` | Path relative to `data/local`, used as `filename` when sharing |
### Share Local Files
1. Open the target subfolder and pick a file
2. Set expiration method and value
3. Create an extract code
Sharing is a **zero-copy reference**: the file is not copied into `data/share` and does not count against storage quota. Expiring or deleting the share record **does not delete the original NAS file**.
**Share parameters:**
| Parameter | Description |
|-----------|-------------|
| `filename` | Path relative to `data/local`, e.g. `movies/a.mp4` |
| `expire_style` | Expiration method (day/hour/minute/forever/count) |
| `expire_value` | Expiration value (days/hours/minutes/download count) |
## System Settings
<p align="center">
<img src="/screenshots/settings.webp" alt="System settings" width="860" />
</p>
### View Configuration
On the system settings page, you can view the current values of all configuration items. Configuration items are grouped by category:
- Basic settings (site name, description, etc.)
- Upload settings (file size limits, rate limits, etc.)
- Storage settings (storage type, path, etc.)
- Theme settings (theme selection, opacity, etc.)
- Security settings (admin password, error limits, etc.)
### Modify Configuration
Administrators can modify most configurations through the admin panel:
1. Go to the system settings page
2. Find the configuration item to modify
3. Enter the new value
4. Click the save button
**Modifiable configuration items:**
| Category | Example Settings |
|----------|------------------|
| Basic Settings | `name`, `description`, `keywords`, `notify_title`, `notify_content` |
| Upload Settings | `upload_size`, `upload_minute`, `upload_count`, `open_upload`, `enable_chunk` |
| Expiration Settings | `expire_style`, `max_save_seconds` |
| Theme Settings | `themes_select`, `opacity`, `background` |
| Security Settings | `admin_token`, `show_admin_addr`, `error_minute`, `error_count` |
| Storage Settings | `file_storage`, `storage_path` and storage backend-specific configurations |
::: warning Note
- `admin_token` (admin password) cannot be set to empty
- `themes_choices` (theme list) cannot be modified through the admin panel
- After modifying storage settings, existing files will not be automatically migrated
:::
### Configuration Effect
Configuration changes take effect immediately without restarting the service. Configurations are saved in the database and persist after restart.
**Configuration storage location:**
- Database: `data/filecodebox.db`
- Table name: `keyvalue`
- Key name: `settings`
## API Endpoints
All admin panel functions are implemented through REST APIs. Here are the main endpoints:
### Authentication Endpoint
**Login**
```
POST /admin/login
Content-Type: application/json
{
"password": "your-admin-password"
}
```
Response:
```json
{
"code": 200,
"detail": {
"token": "eyJhbGciOiJIUzI1NiIs...",
"token_type": "Bearer"
}
}
```
### Dashboard Endpoint
**Get Statistics**
```
GET /admin/dashboard
Authorization: Bearer <token>
```
### File Management Endpoints
**Get File List**
```
GET /admin/file/list?page=1&size=10&keyword=
Authorization: Bearer <token>
```
**Delete File**
```
DELETE /admin/file/delete
Authorization: Bearer <token>
Content-Type: application/json
{
"id": 123
}
```
**Download File**
```
GET /admin/file/download?id=123
Authorization: Bearer <token>
```
**Modify File Information**
```
PATCH /admin/file/update
Authorization: Bearer <token>
Content-Type: application/json
{
"id": 123,
"code": "newcode",
"expired_at": "2024-12-31T23:59:59"
}
```
### Local File Endpoints
**Get Local File List**
```
GET /admin/local/lists?path=movies
Authorization: Bearer <token>
```
**Delete Local File**
```
DELETE /admin/local/delete
Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "movies/example.txt"
}
```
**Share Local File**
```
POST /admin/local/share
Authorization: Bearer <token>
Content-Type: application/json
{
"filename": "movies/example.txt",
"expire_style": "day",
"expire_value": 7
}
```
### Configuration Endpoints
**Get Configuration**
```
GET /admin/config/get
Authorization: Bearer <token>
```
**Update Configuration**
```
PATCH /admin/config/update
Authorization: Bearer <token>
Content-Type: application/json
{
"admin_token": "new-password",
"upload_size": 52428800
}
```
## Common Issues
### Forgot Admin Password
If you forgot the admin password, you can reset it through the following methods:
1. Stop the FileCodeBox service
2. Open `data/filecodebox.db` using an SQLite tool
3. Query the record with `key='settings'` in the `keyvalue` table
4. Modify the `admin_token` value in the JSON
5. Restart the service
```sql
-- View current configuration
SELECT * FROM keyvalue WHERE key = 'settings';
-- Or delete configuration to restore default password
DELETE FROM keyvalue WHERE key = 'settings';
```
### File Deletion Failed
If an error occurs when deleting files, possible reasons:
1. **Storage backend connection failed**: Check if storage configuration is correct
2. **File no longer exists**: File may have been manually deleted
3. **Insufficient permissions**: Check write permissions for storage directory
### Configuration Changes Not Taking Effect
If configuration changes don't take effect:
1. Check if you clicked the save button
2. Refresh the page to see if configuration was saved
3. Check browser console for error messages
4. Confirm configuration value format is correct (e.g., don't enter strings for numeric types)
## Next Steps
- [Configuration Guide](/en/guide/configuration) - Learn detailed descriptions of all configuration options
- [Security Settings](/en/guide/security) - Learn how to enhance system security
- [Storage Configuration](/en/guide/storage) - Configure different storage backends
-24
View File
@@ -1,24 +0,0 @@
# MCP Integration
FileCodeBox provides a **Model Context Protocol (MCP)** endpoint over Streamable HTTP. Compatible AI clients can create text or file shares through it.
## Connection
Add a remote HTTP server in your MCP client:
- **Transport**: Streamable HTTP
- **Server URL**: `http://<FileCodeBox host>:12345/mcp`
- **Authentication**: no token required by default
For a local instance, use `http://localhost:12345/mcp`. If the administrator disables guest uploads or enables MCP token validation in the admin settings, configure `Authorization: Bearer <admin token>` in the client. Never commit the token to public configuration or source code.
## Available tools
- `share_text`: create a text share. Required: `text`; optional: `expire_value`, `expire_style`.
- `share_file`: upload a Base64-encoded file and create a share. Required: `filename`, `content_base64`; optional: `content_type`, `expire_value`, `expire_style`.
`expire_style` accepts `day`, `hour`, `minute`, `forever`, or `count`; `expire_value` must be greater than zero. Successful calls return a share code, URL, and Markdown link.
## Security and limits
MCP uploads follow FileCodeBox's guest-upload switch, admin-token validation, upload-size and file-type limits, and upload rate limits. Expose the endpoint only to trusted AI clients. Use HTTPS for public deployments and keep the admin token secure.
-26
View File
@@ -1,26 +0,0 @@
# Releasing
The project uses Semantic Versioning and Conventional Commits.
Complete these settings once in the backend GitHub repository:
- In `Settings → Actions → General → Workflow permissions`, grant read/write
access and allow GitHub Actions to create pull requests. The frontend repository
only needs the default read-only Actions permission.
- Keep squash merging enabled. The release workflow waits for required checks
before merging the release pull request.
- Add `DOCKER_USERNAME` and `DOCKER_PASSWORD` Actions secrets to the backend repository.
1. Merge commits using Conventional Commit types such as `feat:` and `fix:` into the default branch.
2. Release Please creates and merges the release pull request and updates `VERSION` and the changelog.
3. The workflow then creates `vX.Y.Z`, a GitHub Release, and dispatches the production image build.
4. Releases publish `X.Y.Z`, `X.Y`, and `latest`; branches publish `dev` or `edge-*` images.
5. Each image build resolves the latest commit from both frontend default branches automatically.
Every `fix:`, `feat:`, or breaking change merged into the default branch therefore
creates a Patch, Minor, or Major release automatically. Non-release commits such
as `docs:` and `chore:` do not publish a release on their own.
The backend runtime version comes from `VERSION`. Container branch builds inject a
commit-qualified development version through `APP_VERSION`; release tags must exactly
match `VERSION`. OCI image labels record the resolved backend and frontend commits for traceability.
-321
View File
@@ -1,321 +0,0 @@
# Security Settings
FileCodeBox provides multiple layers of security mechanisms to protect your file sharing service. This document explains how to properly configure security options to ensure secure system operation.
## Admin Password
### First-Run Setup
::: danger Important Security Warning
FileCodeBox does not ship with a default admin password. Open the site in your browser on first run and set the admin password on the setup page.
:::
Change the admin password through the admin panel:
1. Access `/admin` to enter the admin panel
2. Log in with the current password
3. Go to the "System Settings" page
4. Find the `admin_token` configuration item
5. Enter a new secure password and save
`admin_token` is stored as a password hash. Do not write a plaintext password to the database, or admin login will stop working.
### Password Security Recommendations
- Use a strong password with at least 16 characters
- Include uppercase and lowercase letters, numbers, and special characters
- Avoid common words or personal information
- Change password regularly
```python
# Recommended password format example
"admin_token": "Xk9#mP2$vL5@nQ8&wR3"
```
### Hide Admin Entry
By default, the admin panel entry is hidden. You can control whether to show the admin entry on the homepage via the `show_admin_addr` configuration:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `show_admin_addr` | int | `0` | Show admin entry (1=show, 0=hide) |
::: tip Recommendation
For public services, it's recommended to keep `show_admin_addr` at `0` and access the admin panel directly via the `/admin` path.
:::
## IP Rate Limiting
FileCodeBox has built-in IP-based rate limiting mechanisms to effectively prevent abuse and attacks.
### Upload Rate Limiting
Limit the number of uploads from a single IP within a specified time:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `upload_minute` | int | `1` | Upload limit time window (minutes) |
| `upload_count` | int | `10` | Maximum uploads allowed within the time window |
**How it works:**
- System records upload requests from each IP
- When an IP's upload count reaches `upload_count` within `upload_minute` minutes
- Subsequent upload requests from that IP will be rejected with HTTP 423 error
- Counter resets after the time window expires
**Configuration examples:**
```python
# Relaxed configuration: Max 20 uploads in 5 minutes
{
"upload_minute": 5,
"upload_count": 20
}
# Strict configuration: Max 3 uploads in 1 minute
{
"upload_minute": 1,
"upload_count": 3
}
```
### Error Rate Limiting
Limit the number of error attempts from a single IP to prevent brute-force attacks on extraction codes:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `error_minute` | int | `1` | Error limit time window (minutes) |
| `error_count` | int | `10` | Maximum errors allowed within the time window |
**How it works:**
- When a user enters an incorrect extraction code, the system records the error count for that IP
- When error count reaches `error_count`, that IP will be temporarily locked
- Lock duration is `error_minute` minutes
- During lockout, all extraction requests from that IP will be rejected
**Configuration example:**
```python
# Anti-brute-force configuration: Max 3 errors in 5 minutes
{
"error_minute": 5,
"error_count": 3
}
```
::: warning Note
The default allows up to 10 errors per IP each minute. Public services can tighten this limit while accounting for users behind shared network addresses.
:::
## Upload Restrictions
### File Size Limit
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `upload_size` | int | `10485760` | Maximum single file upload size (bytes), default 10MB |
| `open_upload` | int | `1` | Enable upload functionality (1=enabled, 0=disabled) |
**Common size conversions:**
- 10MB = 10 * 1024 * 1024 = `10485760`
- 50MB = 50 * 1024 * 1024 = `52428800`
- 100MB = 100 * 1024 * 1024 = `104857600`
- 1GB = 1024 * 1024 * 1024 = `1073741824`
### File Expiration Settings
Through file expiration mechanisms, you can automatically clean up expired files, reducing storage usage and security risks:
| Setting | Type | Default | Description |
|---------|------|---------|-------------|
| `expire_style` | list | `["day","hour","minute","forever","count"]` | Available expiration methods |
| `max_save_seconds` | int | `0` | Maximum file retention time (seconds), 0 means no limit |
**Expiration methods explained:**
- `day` - Expire by days
- `hour` - Expire by hours
- `minute` - Expire by minutes
- `forever` - Never expire (requires alphanumeric extraction code)
- `count` - Expire by download count
**Security recommendations:**
For public services, it's recommended to:
1. Remove the `forever` option to avoid permanent file storage
2. Set `max_save_seconds` to limit maximum retention time
3. Prefer using `count` method for automatic deletion after download
```python
# Recommended configuration for public services
{
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400 # Max retention 1 day
}
```
### Disable Upload Functionality
In some cases, you may need to temporarily disable upload functionality:
```python
{
"open_upload": 0 # Disable upload functionality
}
```
## Reverse Proxy Security Configuration
In production environments, Nginx or other reverse proxy servers are typically used. Here are security configuration recommendations:
### Nginx Configuration Example
```nginx
server {
listen 80;
server_name your-domain.com;
# Force HTTPS redirect
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name your-domain.com;
# SSL certificate configuration
ssl_certificate /path/to/cert.pem;
ssl_certificate_key /path/to/key.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers on;
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# Limit request body size (match upload_size configuration)
client_max_body_size 100M;
# Pass real IP
location / {
proxy_pass http://127.0.0.1:12345;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Static resource caching
location /assets {
proxy_pass http://127.0.0.1:12345;
proxy_cache_valid 200 7d;
add_header Cache-Control "public, max-age=604800";
}
}
```
### Key Security Configuration Notes
**1. Pass Real IP**
FileCodeBox's IP limiting functionality depends on obtaining the client's real IP. The system obtains IP in the following order:
1. `X-Real-IP` request header
2. `X-Forwarded-For` request header
3. Direct client connection IP
Ensure the reverse proxy correctly sets these headers:
```nginx
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
```
**2. Request Body Size Limit**
Nginx's `client_max_body_size` should match or be slightly larger than FileCodeBox's `upload_size` configuration:
```nginx
client_max_body_size 100M; # Allow max 100MB uploads
```
**3. HTTPS Encryption**
It's strongly recommended to enable HTTPS in production environments:
- Protect uploaded file content
- Protect admin login credentials
- Prevent man-in-the-middle attacks
### Caddy Configuration Example
```nginx
your-domain.com {
reverse_proxy localhost:12345
header {
X-Frame-Options "SAMEORIGIN"
X-Content-Type-Options "nosniff"
X-XSS-Protection "1; mode=block"
Strict-Transport-Security "max-age=31536000; includeSubDomains"
}
}
```
## Security Checklist
Before deploying FileCodeBox, confirm the following security configurations:
- [ ] Completed first-run setup and set the admin password `admin_token`
- [ ] Hidden admin entry `show_admin_addr: 0`
- [ ] Configured appropriate upload rate limiting
- [ ] Configured error rate limiting to prevent brute-force attacks
- [ ] Set reasonable file size limits
- [ ] Configured file expiration policy
- [ ] Enabled HTTPS encryption
- [ ] Reverse proxy correctly passes real IP
- [ ] Set security response headers
## Recommended Security Configurations
### Public Service Configuration
```python
{
"admin_token": "your-very-secure-password",
"show_admin_addr": 0,
"upload_size": 10485760, # 10MB
"upload_minute": 1,
"upload_count": 5,
"error_minute": 5,
"error_count": 3,
"expire_style": ["hour", "minute", "count"],
"max_save_seconds": 86400, # Max 1 day
"open_upload": 1
}
```
### Internal Service Configuration
```python
{
"admin_token": "internal-secure-password",
"show_admin_addr": 1,
"upload_size": 104857600, # 100MB
"upload_minute": 5,
"upload_count": 50,
"error_minute": 1,
"error_count": 5,
"expire_style": ["day", "hour", "forever"],
"max_save_seconds": 0, # No limit
"open_upload": 1
}
```
## Next Steps
- [Configuration Guide](/en/guide/configuration) - Learn about all configuration options
- [Storage Configuration](/en/guide/storage) - Configure secure storage backends
- [File Sharing](/en/guide/share) - Learn about file sharing features
-350
View File
@@ -1,350 +0,0 @@
# File Sharing
FileCodeBox provides simple and easy-to-use file and text sharing functionality. Users can securely share and retrieve files using extraction codes.
## Sharing Methods
<p align="center">
<img src="/screenshots/send.webp" alt="Send page" width="720" />
</p>
FileCodeBox supports two sharing methods:
1. **Text Sharing** - Share text content directly, suitable for code snippets, configuration files, etc.
2. **File Sharing** - Upload files for sharing, supports various file formats
## Text Sharing
### How to Use
1. Select the "Text Share" tab on the homepage
2. Enter or paste the content to share in the text box
3. Select expiration method and time
4. Click the "Share" button
5. Get the extraction code
### Text Size Limit
::: warning Note
The maximum content size for text sharing is **222KB** (227,328 bytes). If content exceeds this limit, it's recommended to use file sharing instead.
:::
Text content size is calculated using UTF-8 encoding. Chinese characters typically occupy 3 bytes.
### API Endpoint
**POST** `/share/text/`
Request parameters:
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `text` | string | Yes | Text content to share |
| `expire_value` | int | No | Expiration value, default 1 |
| `expire_style` | string | No | Expiration method, default `day` |
Response example:
```json
{
"code": 200,
"detail": {
"code": "123456"
}
}
```
## File Sharing
### How to Use
1. Select the "File Share" tab on the homepage
2. Click the upload area or drag files to the upload area
3. Select expiration method and time
4. Click the "Upload" button
5. Get the extraction code
### File Size Limit
The default maximum single file upload size is **10MB**. Administrators can modify this limit via the `upload_size` configuration.
::: tip Tip
If you need to upload large files, contact the administrator to enable chunked upload functionality or adjust the `upload_size` configuration.
:::
### Supported Upload Methods
- **Click Upload** - Click the upload area to select files
- **Drag Upload** - Drag files to the upload area
- **Paste Upload** - Paste images from clipboard (supported by some themes)
### API Endpoint
**POST** `/share/file/`
Request parameters (multipart/form-data):
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `file` | file | Yes | File to upload |
| `expire_value` | int | No | Expiration value, default 1 |
| `expire_style` | string | No | Expiration method, default `day` |
Response example:
```json
{
"code": 200,
"detail": {
"code": "654321",
"name": "example.pdf"
}
}
```
## Expiration Settings
FileCodeBox supports multiple flexible expiration methods:
| Expiration Method | Parameter Value | Description |
|-------------------|-----------------|-------------|
| By Days | `day` | File expires after specified days |
| By Hours | `hour` | File expires after specified hours |
| By Minutes | `minute` | File expires after specified minutes |
| Never Expire | `forever` | File is permanently valid |
| By Count | `count` | File expires after specified download count |
::: info Note
- Administrators can control available expiration methods via the `expire_style` configuration
- Administrators can limit maximum file retention time via the `max_save_seconds` configuration
:::
### Expiration Method Examples
```bash
# File expires after 3 days
expire_value=3, expire_style=day
# File expires after 12 hours
expire_value=12, expire_style=hour
# File expires after 30 minutes
expire_value=30, expire_style=minute
# File never expires
expire_value=1, expire_style=forever
# File expires after 5 downloads
expire_value=5, expire_style=count
```
## Retrieving Files
<p align="center">
<img src="/screenshots/retrieve.webp" alt="Retrieve page" width="720" />
</p>
### How to Use
1. Enter the extraction code in the "Retrieve File" area on the homepage
2. Click the "Retrieve" button
3. System displays file information (filename, size, etc.)
4. Click the "Download" button to download the file, or view text content directly
### Extraction Code Notes
- Extraction codes are typically **6-digit numbers**
- Files that never expire use **alphanumeric** extraction codes
- Extraction codes are case-sensitive (for alphanumeric codes)
### API Endpoints
**Query File Information**
**POST** `/share/select/`
Request parameters:
```json
{
"code": "123456"
}
```
Response example (file):
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "example.pdf",
"size": 1048576,
"text": "https://example.com/download/..."
}
}
```
Response example (text):
```json
{
"code": 200,
"detail": {
"code": "123456",
"name": "Text",
"size": 1024,
"text": "This is the shared text content..."
}
}
```
**Direct File Download**
**GET** `/share/select/?code=123456`
This endpoint returns file content directly, suitable for direct browser access.
## Chunked Upload (Large Files)
For large file uploads, FileCodeBox supports chunked upload functionality. This feature requires administrator enablement (`enable_chunk=1`).
### Chunked Upload Flow
```mermaid
sequenceDiagram
participant C as Client
participant S as Server
C->>S: 1. Initialize upload (POST /chunk/upload/init/)
S-->>C: Return upload_id and chunk info
loop Each chunk
C->>S: 2. Upload chunk (POST /chunk/upload/chunk/{upload_id}/{chunk_index})
S-->>C: Return chunk hash
end
C->>S: 3. Complete upload (POST /chunk/upload/complete/{upload_id})
S-->>C: Return extraction code
```
### 1. Initialize Upload
**POST** `/chunk/upload/init/`
Request parameters:
```json
{
"file_name": "large_file.zip",
"file_size": 104857600,
"chunk_size": 5242880,
"file_hash": "sha256_hash_of_file"
}
```
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `file_name` | string | Yes | Filename |
| `file_size` | int | Yes | Total file size (bytes) |
| `chunk_size` | int | No | Chunk size, default 5MB |
| `file_hash` | string | Yes | SHA256 hash of the file |
Response example:
```json
{
"code": 200,
"detail": {
"existed": false,
"upload_id": "abc123def456",
"chunk_size": 5242880,
"total_chunks": 20,
"uploaded_chunks": []
}
}
```
### 2. Upload Chunk
**POST** `/chunk/upload/chunk/{upload_id}/{chunk_index}`
- `upload_id` - Upload session ID returned during initialization
- `chunk_index` - Chunk index, starting from 0
Request body: Chunk file data (multipart/form-data)
Response example:
```json
{
"code": 200,
"detail": {
"chunk_hash": "sha256_hash_of_chunk"
}
}
```
### 3. Complete Upload
**POST** `/chunk/upload/complete/{upload_id}`
Request parameters:
```json
{
"expire_value": 1,
"expire_style": "day"
}
```
Response example:
```json
{
"code": 200,
"detail": {
"code": "789012",
"name": "large_file.zip"
}
}
```
### Resume Upload
Chunked upload supports resume functionality. If upload is interrupted:
1. Call the initialization endpoint again with the same `file_hash`
2. Server returns `uploaded_chunks` list containing already uploaded chunk indices
3. Client only needs to upload chunks not in the list
## Error Handling
### Common Error Codes
| Error Code | Description | Solution |
|------------|-------------|----------|
| 403 | File size exceeds limit | Reduce file size or contact administrator to adjust limit |
| 403 | Content too large | Text exceeds 222KB, use file sharing instead |
| 403 | Upload rate limit | Wait a while before retrying |
| 404 | File not found | Check if extraction code is correct |
| 404 | File expired | File has expired or download count exhausted |
### Rate Limiting
To prevent abuse, the system has rate limits on upload and retrieval operations:
- **Upload limit**: Default max 10 uploads per minute
- **Error limit**: Default max 1 error attempt per minute
::: tip Tip
If you encounter rate limiting, wait for the limit time window to pass before retrying.
:::
## Next Steps
- [Configuration Guide](/en/guide/configuration) - Learn how to configure sharing-related settings
- [Storage Configuration](/en/guide/storage) - Learn about file storage methods
- [Security Settings](/en/guide/security) - Learn about security-related configurations
- [Admin Panel](/en/guide/management) - Learn how to manage shared files
-396
View File
@@ -1,396 +0,0 @@
# Storage Configuration
FileCodeBox supports multiple storage backends. You can choose the appropriate storage method based on your needs. This document details the configuration methods for various storage backends.
## Storage Types Overview
| Storage Type | Config Value | Description |
|--------------|--------------|-------------|
| Local Storage | `local` | Default storage method, files saved on local server |
| S3-Compatible Storage | `s3` | Supports AWS S3, Aliyun OSS, MinIO, etc. |
| OneDrive | `onedrive` | Microsoft OneDrive cloud storage (work/school accounts only) |
| WebDAV | `webdav` | Storage services supporting WebDAV protocol |
| OpenDAL | `opendal` | Integrate more storage services via OpenDAL |
## Local Storage
Local storage is the default storage method. Files are saved in the server's `data/` directory.
### Configuration Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file_storage` | string | `local` | Storage type |
| `storage_path` | string | `""` | Custom storage path (optional) |
### Configuration Example
```bash
file_storage=local
storage_path=
```
### Notes
- Files are stored by default in `data/share/data/` directory
- Subdirectories are automatically created by date: `year/month/day/fileID/`
- In production, it's recommended to mount the `data/` directory to persistent storage
- Existing NAS files can be mounted at `data/local` (including subfolders). Admin Local Share references them in place to mint extract codes and does not copy them into `data/share`
## S3-Compatible Storage
Supports all S3-compatible object storage services, including AWS S3, Aliyun OSS, MinIO, Tencent Cloud COS, etc.
### Configuration Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file_storage` | string | - | Set to `s3` |
| `s3_access_key_id` | string | `""` | Access Key ID |
| `s3_secret_access_key` | string | `""` | Secret Access Key |
| `s3_bucket_name` | string | `""` | Bucket name |
| `s3_endpoint_url` | string | `""` | S3 endpoint URL |
| `s3_region_name` | string | `auto` | Region name |
| `s3_signature_version` | string | `s3v2` | Signature version (`s3v2` or `s3v4`) |
| `s3_hostname` | string | `""` | S3 hostname (alternative) |
| `s3_proxy` | int | `0` | Download through server proxy (1=yes, 0=no) |
| `aws_session_token` | string | `""` | AWS session token (optional) |
### AWS S3 Configuration Example
```bash
file_storage=s3
s3_access_key_id=AKIAIOSFODNN7EXAMPLE
s3_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
s3_bucket_name=my-filecodebox-bucket
s3_endpoint_url=https://s3.amazonaws.com
s3_region_name=us-east-1
s3_signature_version=s3v4
```
### Aliyun OSS Configuration Example
```bash
file_storage=s3
s3_access_key_id=YourAccessKeyId
s3_secret_access_key=YourSecretAccessKey
s3_bucket_name=bucket-name
s3_endpoint_url=https://bucket-name.oss-cn-hangzhou.aliyuncs.com
s3_region_name=oss-cn-hangzhou
s3_signature_version=s3v4
```
::: tip Aliyun OSS Endpoint Format
Endpoint URL format: `https://<bucket-name>.<region>.aliyuncs.com`
Common regions:
- Hangzhou: `oss-cn-hangzhou`
- Shanghai: `oss-cn-shanghai`
- Beijing: `oss-cn-beijing`
- Shenzhen: `oss-cn-shenzhen`
:::
### MinIO Configuration Example
```bash
file_storage=s3
s3_access_key_id=minioadmin
s3_secret_access_key=minioadmin
s3_bucket_name=filecodebox
s3_endpoint_url=http://localhost:9000
s3_region_name=us-east-1
s3_signature_version=s3v4
```
::: warning MinIO Notes
- `s3_endpoint_url` should be the MinIO API interface address
- `s3_region_name` should match the `Server Location` in MinIO configuration
- Ensure the bucket is created and has correct access permissions
:::
### Tencent Cloud COS Configuration Example
```bash
file_storage=s3
s3_access_key_id=YourSecretId
s3_secret_access_key=YourSecretKey
s3_bucket_name=bucket-name-1250000000
s3_endpoint_url=https://cos.ap-guangzhou.myqcloud.com
s3_region_name=ap-guangzhou
s3_signature_version=s3v4
```
### Proxy Download
When `s3_proxy=1`, file downloads are proxied through the server instead of directly from S3. This is useful when:
- S3 bucket doesn't allow public access
- Need to hide the actual storage address
- Network environment restricts direct S3 access
## OneDrive Storage
OneDrive storage supports saving files to Microsoft OneDrive cloud storage.
::: warning Important Limitation
OneDrive storage **only supports work or school accounts** and requires admin permissions to authorize the API. Personal accounts cannot use this feature.
:::
### Configuration Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file_storage` | string | - | Set to `onedrive` |
| `onedrive_domain` | string | `""` | Azure AD domain |
| `onedrive_client_id` | string | `""` | Application (client) ID |
| `onedrive_username` | string | `""` | Account email |
| `onedrive_password` | string | `""` | Account password |
| `onedrive_root_path` | string | `filebox_storage` | Storage root directory in OneDrive |
| `onedrive_proxy` | int | `0` | Download through server proxy |
### Configuration Example
```bash
file_storage=onedrive
onedrive_domain=contoso.onmicrosoft.com
onedrive_client_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
onedrive_username=user@contoso.onmicrosoft.com
onedrive_password=your_password
onedrive_root_path=filebox_storage
```
### Azure App Registration Steps
To use OneDrive storage, you need to register an application in the Azure portal:
#### 1. Get Domain
Log in to [Azure Portal](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade), hover over your account in the top right corner, and the **Domain** shown in the popup is the `onedrive_domain` value.
#### 2. Register Application
1. Click **+ New registration** in the top left
2. Enter application name (e.g., FileCodeBox)
3. **Supported account types**: Select "Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts"
4. **Redirect URI**: Select `Web`, enter `http://localhost`
5. Click **Register**
#### 3. Get Client ID
After registration, find the **Application (client) ID** in the **Essentials** section on the app overview page. This is the `onedrive_client_id` value.
#### 4. Configure Authentication
1. Select **Authentication** in the left menu
2. Find **Allow public client flows**, select **Yes**
3. Click **Save**
#### 5. Configure API Permissions
1. Select **API permissions** in the left menu
2. Click **+ Add a permission**
3. Select **Microsoft Graph** → **Delegated permissions**
4. Check the following permissions:
- `openid`
- `Files.Read`
- `Files.Read.All`
- `Files.ReadWrite`
- `Files.ReadWrite.All`
- `User.Read`
5. Click **Add permissions**
6. Click **Grant admin consent for xxx**
7. After confirmation, permission status should show **Granted**
### Install Dependencies
Using OneDrive storage requires additional Python dependencies:
```bash
pip install msal Office365-REST-Python-Client
```
### Verify Configuration
You can use the following code to test if the configuration is correct:
```python
import msal
from office365.graph_client import GraphClient
domain = 'your_domain'
client_id = 'your_client_id'
username = 'your_username'
password = 'your_password'
def acquire_token_pwd():
authority_url = f'https://login.microsoftonline.com/{domain}'
app = msal.PublicClientApplication(
authority=authority_url,
client_id=client_id
)
result = app.acquire_token_by_username_password(
username=username,
password=password,
scopes=['https://graph.microsoft.com/.default']
)
return result
# Test connection
client = GraphClient(acquire_token_pwd)
me = client.me.get().execute_query()
print(f"Login successful: {me.user_principal_name}")
```
## WebDAV Storage
WebDAV storage supports saving files to any service that supports the WebDAV protocol, such as Nextcloud, ownCloud, Nutstore, etc.
### Configuration Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file_storage` | string | - | Set to `webdav` |
| `webdav_url` | string | `""` | WebDAV server URL |
| `webdav_username` | string | `""` | WebDAV username |
| `webdav_password` | string | `""` | WebDAV password |
| `webdav_root_path` | string | `filebox_storage` | Storage root directory in WebDAV |
| `webdav_proxy` | int | `0` | Download through server proxy |
### Configuration Example
```bash
file_storage=webdav
webdav_url=https://dav.example.com/remote.php/dav/files/username/
webdav_username=your_username
webdav_password=your_password
webdav_root_path=filebox_storage
```
### Nextcloud Configuration Example
```bash
file_storage=webdav
webdav_url=https://your-nextcloud.com/remote.php/dav/files/username/
webdav_username=your_username
webdav_password=your_app_password
webdav_root_path=FileCodeBox
```
::: tip Nextcloud App Password
It's recommended to create an app password in Nextcloud instead of using your main password:
1. Log in to Nextcloud
2. Go to **Settings** → **Security**
3. Create a new app password in **Devices & sessions**
:::
### Nutstore Configuration Example
```bash
file_storage=webdav
webdav_url=https://dav.jianguoyun.com/dav/
webdav_username=your_email@example.com
webdav_password=your_app_password
webdav_root_path=FileCodeBox
```
::: tip Nutstore App Password
Nutstore requires an app password:
1. Log in to Nutstore web version
2. Go to **Account Info** → **Security Options**
3. Add an app password
:::
## OpenDAL Storage
OpenDAL is a unified data access layer that supports multiple storage services. Through OpenDAL, you can use Google Cloud Storage, Azure Blob Storage, and more.
### Configuration Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `file_storage` | string | Set to `opendal` |
| `opendal_scheme` | string | Storage service type (e.g., `gcs`, `azblob`) |
| `opendal_<scheme>_<setting>` | string | Service-specific configuration parameters |
### Install Dependencies
```bash
pip install opendal
```
### Google Cloud Storage Configuration Example
```bash
file_storage=opendal
opendal_scheme=gcs
opendal_gcs_root=/filecodebox
opendal_gcs_bucket=your-bucket-name
opendal_gcs_credential=base64_encoded_credential
```
### Azure Blob Storage Configuration Example
```bash
file_storage=opendal
opendal_scheme=azblob
opendal_azblob_root=/filecodebox
opendal_azblob_container=your-container
opendal_azblob_account_name=your_account
opendal_azblob_account_key=your_key
```
### Supported Services
OpenDAL supports numerous storage services. For the complete list, see the [OpenDAL Official Documentation](https://opendal.apache.org/docs/rust/opendal/services/index.html).
Common services include:
- `gcs` - Google Cloud Storage
- `azblob` - Azure Blob Storage
- `obs` - Huawei Cloud OBS
- `oss` - Aliyun OSS (via OpenDAL)
- `cos` - Tencent Cloud COS (via OpenDAL)
- `hdfs` - Hadoop HDFS
- `ftp` - FTP server
- `sftp` - SFTP server
::: warning OpenDAL Notes
1. Services integrated via OpenDAL download through server proxy, consuming both storage service and server bandwidth
2. Compared to native S3/OneDrive support, OpenDAL may lack some debugging information
3. OpenDAL is written in Rust with good performance
:::
## Storage Selection Recommendations
| Scenario | Recommended Storage | Reason |
|----------|---------------------|--------|
| Personal/Small deployment | Local storage | Simple and easy, no extra configuration needed |
| Enterprise intranet | MinIO + S3 | Self-hosted object storage, data control |
| Public cloud deployment | Corresponding cloud provider S3 | Fast same-region access, low cost |
| Existing OneDrive | OneDrive | Utilize existing resources |
| Existing WebDAV | WebDAV | Good compatibility |
| Special storage needs | OpenDAL | Supports more storage services |
## Common Issues
### S3 Upload Failure
1. Check if Access Key and Secret Key are correct
2. Confirm bucket name and region configuration are correct
3. Check bucket access permission settings
4. Confirm signature version (`s3v2` or `s3v4`) matches service provider requirements
### OneDrive Authentication Failure
1. Confirm using a work/school account, not a personal account
2. Check if Azure app has been granted admin consent
3. Confirm API permissions are fully configured
4. Verify username and password are correct
### WebDAV Connection Failure
1. Check if WebDAV URL format is correct
2. Confirm username and password (or app password) are correct
3. Check if server supports WebDAV protocol
4. Confirm network connection is normal
Loaded 100 of 207 files, more files were not shown because too many files have changed in this diff. Show more