gitea-admin/gh-christianlouis-docuelevate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: merge main branch and renumber migration 037→040

Browse Source 
Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).

Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
  (classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
  factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
  QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
  new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models

Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)

For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45
This commit is contained in:
copilot-swe-agent[bot]
2026-03-20 23:54:04 +00:00
parent d1ebac74a1
commit e518bce922
153 changed files with 16306 additions and 922 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.demo
+34
View File
@@ -3,10 +3,25 @@ WORKDIR=/workdir
DATABASE_URL=sqlite:///./app/database.db
REDIS_URL=redis://redis:6379/0
EXTERNAL_HOSTNAME=docuelevate.example.com
# PUBLIC_BASE_URL=https://docuelevate.example.com # Full URL with scheme; required when X-Forwarded-Proto is not forwarded by your proxy
GOTENBERG_URL=http://gotenberg:3000
ALLOW_FILE_DELETE=true # Allow deletion of file records
COMPLIANCE_ENABLED=true # Enable compliance templates dashboard (GDPR, HIPAA, SOC 2)
# **Database Connection Pool** (PostgreSQL / MySQL only; ignored for SQLite)
# DB_POOL_SIZE=10 # Persistent connections per worker (default: 10)
# DB_MAX_OVERFLOW=20 # Extra connections under burst (default: 20)
# DB_POOL_TIMEOUT=30 # Seconds to wait for a pool connection (default: 30)
# DB_POOL_RECYCLE=1800 # Recycle connections after N seconds (default: 1800)
# **Per-User Upload Rate Limiting** (health-aware, Redis-backed)
# UPLOAD_RATE_LIMIT_PER_USER=20 # Max uploads per user per window (default: 20)
# UPLOAD_RATE_LIMIT_WINDOW=60 # Sliding window in seconds (default: 60)
# **System Reset / Factory Reset**
# FACTORY_RESET_ON_STARTUP=false # Wipe all user data on every startup (demo/testing only)
# ENABLE_FACTORY_RESET=false # Show the System Reset page in admin UI
# **Logging**
# LOG_LEVEL controls the Python root-logger level.
# Accepted values: DEBUG, INFO, WARNING, ERROR, CRITICAL (default: INFO).
@@ -160,6 +175,16 @@ AUTH_ENABLED=true
# Generate a secure random string, for example:
# python -c "import secrets; print(secrets.token_hex(32))"
SESSION_SECRET=b39fd43f68d0491ca942f28a16e484b1e763fe9accf4445ca2669a5f3b179eb4
# Session lifetime in days (default: 30). Common values: 30, 60, 90.
# Determines how long a user stays logged in before needing to re-authenticate.
# SESSION_LIFETIME_DAYS=30
# Override with a custom value (takes precedence over SESSION_LIFETIME_DAYS):
# SESSION_LIFETIME_CUSTOM_DAYS=
# Time-to-live in seconds for QR code login challenges (default: 120 = 2 minutes).
# QR_LOGIN_CHALLENGE_TTL_SECONDS=120
ADMIN_USERNAME=admin
ADMIN_PASSWORD=your_secure_password
ADMIN_GROUP_NAME=admin
@@ -433,6 +458,15 @@ ONEDRIVE_TENANT_ID=common
ONEDRIVE_REFRESH_TOKEN=your-refresh-token
ONEDRIVE_FOLDER_PATH=Documents/Uploads
# SharePoint
SHAREPOINT_CLIENT_ID=your-client-id
SHAREPOINT_CLIENT_SECRET=your-client-secret
SHAREPOINT_TENANT_ID=common
SHAREPOINT_REFRESH_TOKEN=your-refresh-token
SHAREPOINT_SITE_URL=https://tenant.sharepoint.com/sites/sitename
SHAREPOINT_DOCUMENT_LIBRARY=Documents
SHAREPOINT_FOLDER_PATH=Uploads
# WebDAV
# WEBDAV_ENABLED=true # Set to false to disable WebDAV uploads without removing credentials
WEBDAV_URL=https://webdav.example.com/path
.github/workflows/ci.yml
+13 -1
View File
@@ -44,6 +44,18 @@ jobs:
- run: ruff check app/ tests/
- run: ruff format --check app/ tests/
migration-chain:
name: Alembic Migration Chain Check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Validate migration chain
run: python scripts/check_alembic_migrations.py
html-lint:
name: HTML Accessibility Lint
runs-on: ubuntu-latest
@@ -138,7 +150,7 @@ jobs:
build:
name: Build & Push Docker Image
runs-on: ubuntu-latest
needs: [run-tests, mypy, dependency-scan, html-lint]
needs: [run-tests, mypy, dependency-scan, html-lint, migration-chain]
if: github.event_name == 'push'
steps:
- name: Checkout Code
.pre-commit-config.yaml
+10
View File
@@ -48,6 +48,16 @@ repos:
.env.demo
)$
# Alembic migration chain validation
- repo: local
hooks:
- id: check-alembic-migrations
name: Check Alembic migration chain
entry: python scripts/check_alembic_migrations.py
language: python
pass_filenames: false
files: ^migrations/versions/.*\.py$
# Conventional commits validation
- repo: https://github.com/compilerla/conventional-pre-commit
rev: v3.0.0
BUILD_DATE
+1 -1
View File
@@ -1 +1 @@
2026-03-16T21:40:03Z
2026-03-20T23:38:07Z
CHANGELOG.md
+333
View File
@@ -10,6 +10,339 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
<!-- version list -->
## v0.163.0 (2026-03-20)
### Bug Fixes
- **security**: Escape HTML in folder browser to prevent XSS from folder names
([`b3a2387`](https://github.com/christianlouis/DocuElevate/commit/b3a238744d1618b7f891b7c43f243dd9099ecf49))
### Code Style
- Apply ruff auto-fix
([`65cf33c`](https://github.com/christianlouis/DocuElevate/commit/65cf33ce89cfcb7f81f31278a2cf6523c32a4754))
### Documentation
- Update setup guides and API docs for folder browser and system credentials
([`d6c21b8`](https://github.com/christianlouis/DocuElevate/commit/d6c21b8026ccc8fb71ed12394faba08214e0793d))
### Features
- **api**: Add folder browser API endpoints and UI for Dropbox and OneDrive
([`a842306`](https://github.com/christianlouis/DocuElevate/commit/a8423064ec72bf8014ca37936960c5da9ddcea1f))
- **auth**: Default to system-wide app credentials in OAuth wizards for user mode
([`23c8c76`](https://github.com/christianlouis/DocuElevate/commit/23c8c76b392a95e723f25f8cb3714d4e5f256b30))
- **ui**: Replace manual credential fields with OAuth wizard flow for watch folder sources
([`ed01952`](https://github.com/christianlouis/DocuElevate/commit/ed0195261032fb92df5f655a73f056ac380fe8b6))
### Testing
- Add tests for folder browser APIs and system credentials toggle
([`1e1e6e6`](https://github.com/christianlouis/DocuElevate/commit/1e1e6e62807a6d4b7bc84b2d8c967dceb97f1cb6))
## v0.162.0 (2026-03-20)
## v0.161.0 (2026-03-20)
### Documentation
- Update scaling, health probe, and beat scheduler documentation
([`4d019d5`](https://github.com/christianlouis/DocuElevate/commit/4d019d53d9ad161a68639d6ba2109ba3ac77df34))
### Features
- **scaling**: Enable horizontal scaling for API and worker pods
([`f75b125`](https://github.com/christianlouis/DocuElevate/commit/f75b12599291050f97aa452d980f17547ddd7bd6))
## v0.160.3 (2026-03-20)
### Bug Fixes
- **mobile**: Wire i18n reactivity, translate all screens, sync language with server
([`3b5ca04`](https://github.com/christianlouis/DocuElevate/commit/3b5ca04ebc8b6dd20f877bc4797e364e0997d840))
### Chores
- **mobile**: Upgrade ESLint to v9 with flat config and fix expo-localization version
([`0e6a4c5`](https://github.com/christianlouis/DocuElevate/commit/0e6a4c5084b3704653965f0091c36b9c78b8ad60))
## v0.160.2 (2026-03-20)
### Bug Fixes
- **dropbox**: Fix Invalid redirect_uri error by adding PUBLIC_BASE_URL config and URL-encoding
([`5e3e2b1`](https://github.com/christianlouis/DocuElevate/commit/5e3e2b19997d3af6570fbaa1e75a49cbfe6cf78d))
## v0.160.1 (2026-03-20)
### Bug Fixes
- **mobile**: Update expo-localization version from ~16.0.6 to ~16.1.0
([`78c3717`](https://github.com/christianlouis/DocuElevate/commit/78c3717661923b43f1762fa7728c75e803938fb7))
## v0.160.0 (2026-03-20)
### Bug Fixes
- **mobile**: Address code review feedback - error handling, filename collision, hash display
([`6541529`](https://github.com/christianlouis/DocuElevate/commit/65415292507aa37408428400aef7e87e006abfd0))
### Features
- **mobile**: Add pre-login legal pages, multi-image selection, file detail view, search, i18n, HEIC
support
([`67c17e7`](https://github.com/christianlouis/DocuElevate/commit/67c17e7baa8edf76be394d0aff42c2adeae351e1))
## v0.159.0 (2026-03-19)
### Code Style
- Apply ruff auto-fix
([`910fb29`](https://github.com/christianlouis/DocuElevate/commit/910fb297ba1122b751250b323a330e15e276daf6))
### Features
- **integrations**: Add Dropbox connection test and global-credential sharing
([`d1f9819`](https://github.com/christianlouis/DocuElevate/commit/d1f9819f4e12320bb901b76366fa8d4a8cd23a66))
## v0.158.4 (2026-03-19)
### Bug Fixes
- **ui**: Show proper error when signup username has invalid characters
([`689c616`](https://github.com/christianlouis/DocuElevate/commit/689c616e448bf3edfcb91b347ae9f6fac43b1983))
## v0.158.3 (2026-03-19)
### Bug Fixes
- **upload**: Reject exact duplicates at upload time and prevent duplicate mobile uploads
([`d5c18cc`](https://github.com/christianlouis/DocuElevate/commit/d5c18ccf07efa28532724a78e63cc6bd7eab9515))
### Refactoring
- **mobile**: Extract normalizeFileUri to shared utility module
([`ec88221`](https://github.com/christianlouis/DocuElevate/commit/ec882214e29d22da9d8025ea857ba754fbddae37))
## v0.158.2 (2026-03-19)
### Bug Fixes
- **mobile**: Add user feedback when server URL is unavailable
([`1366317`](https://github.com/christianlouis/DocuElevate/commit/136631762bc81399aecd5766663da7d923cba985))
- **mobile**: Apple App Store compliance fixes
([`5c15a23`](https://github.com/christianlouis/DocuElevate/commit/5c15a2395a0dcfa5fe5cf1a34a98ae743d29eabc))
- **mobile**: Fix file sharing deep-link conflicts and add MIME type inference
([`1559686`](https://github.com/christianlouis/DocuElevate/commit/1559686f903808e2ab4547f21575460aaeca839e))
- **mobile**: Fix shared file upload hanging by copying to cache
([`f549505`](https://github.com/christianlouis/DocuElevate/commit/f549505bfd625100e77a170cfcae372913680835))
### Documentation
- Add Apple App Store Compliance audit report
([`1572f32`](https://github.com/christianlouis/DocuElevate/commit/1572f322d72583c45c88e55ea51b2456e548de4c))
### Refactoring
- **mobile**: Extract shared MIME type utility and improve error handling
([`cfe83d7`](https://github.com/christianlouis/DocuElevate/commit/cfe83d7efa7f93ca51bfcac417cfafc650bbc36b))
## v0.158.1 (2026-03-19)
### Bug Fixes
- **mobile**: Add shared file to ShareContext directly in +not-found.tsx
([`71a7a57`](https://github.com/christianlouis/DocuElevate/commit/71a7a57adc1bb6de88c060dba563afa433283e38))
## v0.158.0 (2026-03-19)
### Bug Fixes
- Address code review feedback (assertion, exc_info logging)
([`faa68ad`](https://github.com/christianlouis/DocuElevate/commit/faa68adaa143774a8757fda4fd0248cc3aef551e))
- **config**: Add SETTING_METADATA for db pool and upload rate limit settings
([`1d7286c`](https://github.com/christianlouis/DocuElevate/commit/1d7286c4c68de903951d9819bcb58ae1300dab7b))
- **config**: Remove duplicate dictionary keys and class fields from merge
([`d34b8bc`](https://github.com/christianlouis/DocuElevate/commit/d34b8bceb9f96719bf5c922156d5487a4c6426b4))
- **db**: Use NullPool for SQLite and expose pool tuning settings
([`56bf665`](https://github.com/christianlouis/DocuElevate/commit/56bf66539757cf1d98fe251ad64fb75b481462ef))
- **tests**: Add docstring to rate limiter no-op override
([`c9f9001`](https://github.com/christianlouis/DocuElevate/commit/c9f900124407baa230b38f6e1d0f5ee78e0bedcc))
- **tests**: Disable upload rate limiter in test client fixture
([`6bf121f`](https://github.com/christianlouis/DocuElevate/commit/6bf121f02f2c32f7b73435f42d30b9681710089e))
### Features
- **api**: Add per-user health-aware upload rate limiting
([`571cc81`](https://github.com/christianlouis/DocuElevate/commit/571cc817893a1f4cbd7cfbee7f1c46ca1a4f14d8))
## v0.157.2 (2026-03-19)
### Bug Fixes
- **ui**: Improve devices page table layout to prevent horizontal scrolling
([`b12e891`](https://github.com/christianlouis/DocuElevate/commit/b12e8916824273eefb85f0bb498d4991e8c66d5e))
## v0.157.1 (2026-03-19)
### Bug Fixes
- **mobile**: Resolve iOS "unmatched route docuelevate://" error in Open In share flow
([`f2b7db8`](https://github.com/christianlouis/DocuElevate/commit/f2b7db88ba86e894a9bf2e4e7d6a54cd7f423194))
## v0.157.0 (2026-03-19)
### Features
- **api**: Allow disabled tokens/devices to be deleted & reactivated; add token lifetime
([`e4749b4`](https://github.com/christianlouis/DocuElevate/commit/e4749b4e7cdd78ab95627736d3fcd84fecc55e53))
## v0.156.3 (2026-03-18)
### Bug Fixes
- **auth**: Exempt /api/qr-auth/claim from CSRF to fix mobile QR login
([`a4aaebf`](https://github.com/christianlouis/DocuElevate/commit/a4aaebfe6649ef051790b6af24e0739bdd9ecfed))
### Documentation
- **changelog**: Update changelog [skip ci]
([`d8d2016`](https://github.com/christianlouis/DocuElevate/commit/d8d2016f8597f775cdf3848e7baeaae98cb998b0))
## Unreleased
## v0.156.2 (2026-03-18)
### Bug Fixes
- **qr-login**: Render QR code server-side using segno instead of CDN JS library
([`a8eb650`](https://github.com/christianlouis/DocuElevate/commit/a8eb6504ac100c14b10f57511fde8f1202deeffe))
### Chores
- Initial plan for server-side QR code rendering
([`6727253`](https://github.com/christianlouis/DocuElevate/commit/6727253958a6ae8436fc1184736ec43eef2ab820))
- Remove accidentally committed =1.6.0 file
([`ba8c88b`](https://github.com/christianlouis/DocuElevate/commit/ba8c88bc17d3245cba6079ac5ba3be5fddf36366))
## v0.156.1 (2026-03-18)
### Bug Fixes
- Add missing SETTING_METADATA entries for db pool and upload rate limit settings
([`dc0a19b`](https://github.com/christianlouis/DocuElevate/commit/dc0a19bd118d3503ad50608f55fb0bc4ce104948))
## v0.156.0 (2026-03-17)
### Bug Fixes
- **storage**: Use RuntimeError instead of bare Exception in SharePoint task
([`2b698cc`](https://github.com/christianlouis/DocuElevate/commit/2b698cc6940fb731b1ab87300ad0f7fdebc8f024))
- **test**: Add missing _should_upload_to_sharepoint mock to send_to_all tests
([`bcf2d00`](https://github.com/christianlouis/DocuElevate/commit/bcf2d00c3324a3ded3852e16759ea4d0af3af666))
### Documentation
- Add SharePoint setup guide and update all references
([`13aa14b`](https://github.com/christianlouis/DocuElevate/commit/13aa14b8e4102437f72f6c260b2795a6ee761eb9))
### Features
- **storage**: Add SharePoint integration for document storage
([`b85fc1d`](https://github.com/christianlouis/DocuElevate/commit/b85fc1d277475c06c1efa100c391b7a3c43e7c25))
## v0.155.1 (2026-03-17)
### Bug Fixes
- **api**: Add ttl_seconds to QR challenge response and fix client-side countdown
([`0f6a1ee`](https://github.com/christianlouis/DocuElevate/commit/0f6a1ee1ec8186d70afc17abe50258c968058c92))
- **mobile**: Replace gap with marginLeft for React Native compatibility
([`70b193e`](https://github.com/christianlouis/DocuElevate/commit/70b193e07d2f85e353b6b12c56bf5bcbf828ee24))
### Documentation
- Update QR code login documentation with scanner and TTL details
([`723b14e`](https://github.com/christianlouis/DocuElevate/commit/723b14e660737887c454b8e8300ac38bb390841f))
## v0.155.0 (2026-03-17)
## v0.154.0 (2026-03-17)
### Bug Fixes
- **db**: Address code review feedback - fix comment stripping, type hints, test skip, and docs
([`a007b4f`](https://github.com/christianlouis/DocuElevate/commit/a007b4fd98ed327a9874082766e0f3b8c32a1a68))
### Features
- **db**: Add migration chain CI validation, pre-commit hook, script template, docs, and tests
([`aa49fa3`](https://github.com/christianlouis/DocuElevate/commit/aa49fa3ae6ae0bfbb0f9efa6304cf109eca1c402))
## v0.153.1 (2026-03-16)
### Bug Fixes
- **mobile**: Add root index.tsx redirect to prevent stale Hello World screen
([`1350aa6`](https://github.com/christianlouis/DocuElevate/commit/1350aa6a5ede19d924a9a4c277ef8089b940ea3c))
## v0.153.0 (2026-03-16)
### Bug Fixes
- **system-reset**: Address code review feedback
([`7dffdc0`](https://github.com/christianlouis/DocuElevate/commit/7dffdc05542cbfe37a91f86b661790038195fef5))
### Features
- **system-reset**: Add system reset and factory reset feature
([`a88d790`](https://github.com/christianlouis/DocuElevate/commit/a88d790445e5aae9f8ba04e735501d6c93589dec))
### Testing
- **system-reset**: Add comprehensive tests and documentation
([`96bfba8`](https://github.com/christianlouis/DocuElevate/commit/96bfba8057e506f561d21318b0205e521cf67109))
## v0.152.0 (2026-03-16)
### Features
GIT_SHA
+1 -1
View File
@@ -1 +1 @@
793b6d3
6f5a73f
RUNTIME_INFO
+6 -6
View File
@@ -1,10 +1,10 @@
DocuElevate Build Information
==============================
Version: 0.152.0
Build Date: 2026-03-16T21:40:03Z
Git Commit: 793b6d3e2562518cb404579f656e82a6dc60da02
Git Short SHA: 793b6d3
Version: 0.163.0
Build Date: 2026-03-20T23:38:07Z
Git Commit: 6f5a73f98ae3e890a232e5202521b1a263cfe5d0
Git Short SHA: 6f5a73f
Git Branch: main
Commit Date: 2026-03-16T22:39:40+01:00
Build Timestamp: 2026-03-16T21:40:03Z
Commit Date: 2026-03-21T00:37:49+01:00
Build Timestamp: 2026-03-20T23:38:07Z
==============================
VERSION
+1 -1
View File
@@ -1 +1 @@
0.152.0
0.163.0
app/api/__init__.py
+8
View File
@@ -13,6 +13,7 @@ from app.api.automation import router as automation_router
from app.api.azure import router as azure_router
from app.api.backup import router as backup_router
from app.api.billing import router as billing_router
from app.api.classification_rules import router as classification_rules_router
from app.api.compliance import router as compliance_router
from app.api.database import router as database_router
from app.api.diagnostic import router as diagnostic_router
@@ -34,16 +35,19 @@ from app.api.pipelines import router as pipelines_router
from app.api.plans import router as plans_router
from app.api.process import router as process_router
from app.api.profile import router as profile_router
from app.api.qr_auth import router as qr_auth_router
from app.api.queue import router as queue_router
from app.api.routing_rules import router as routing_rules_router
from app.api.saved_searches import router as saved_searches_router
from app.api.scheduled_jobs import router as scheduled_jobs_router
from app.api.search import router as search_router
from app.api.sessions import router as sessions_router
from app.api.settings import router as settings_router
from app.api.shared_links import public_router as shared_links_public_router
from app.api.shared_links import router as shared_links_router
from app.api.similarity import router as similarity_router
from app.api.subscriptions import router as subscriptions_router
from app.api.system_reset import router as system_reset_router
from app.api.translation import router as translation_router
from app.api.url_upload import router as url_upload_router
@@ -97,6 +101,10 @@ router.include_router(scheduled_jobs_router)
router.include_router(audit_logs_router)
router.include_router(i18n_router)
router.include_router(mobile_router)
router.include_router(sessions_router)
router.include_router(qr_auth_router)
router.include_router(compliance_router)
router.include_router(system_reset_router)
router.include_router(translation_router)
router.include_router(classification_rules_router)
router.include_router(automation_router)
app/api/api_tokens.py
+123 -26
View File
@@ -13,7 +13,7 @@ plaintext is returned exactly once at creation time.
import hashlib
import logging
import secrets
from datetime import datetime, timezone
from datetime import datetime, timedelta, timezone
from typing import Annotated, Any
from fastapi import APIRouter, Depends, HTTPException, Request, status
@@ -42,6 +42,9 @@ TOKEN_HASH_ITERATIONS = 100_000
#: PBKDF2 salt for API token hashing (not secret, but fixed for determinism).
TOKEN_HASH_SALT = b"api-token-v1"
#: Name prefix used for tokens created by the mobile app flow.
MOBILE_TOKEN_PREFIX = "Mobile App"
# ---------------------------------------------------------------------------
# Auth helper
@@ -91,6 +94,21 @@ def hash_token(token: str) -> str:
return dk.hex()
def _token_to_dict(t: ApiToken) -> dict[str, Any]:
"""Convert an ``ApiToken`` ORM instance to a serialisable dict."""
return {
"id": t.id,
"name": t.name,
"token_prefix": t.token_prefix,
"is_active": t.is_active,
"last_used_at": t.last_used_at,
"last_used_ip": t.last_used_ip,
"created_at": t.created_at,
"revoked_at": t.revoked_at,
"expires_at": t.expires_at,
}
# ---------------------------------------------------------------------------
# Pydantic schemas
# ---------------------------------------------------------------------------
@@ -100,6 +118,12 @@ class TokenCreate(BaseModel):
"""Schema for creating a new API token."""
name: str = Field(..., min_length=1, max_length=255, description="Human-readable label for the token")
expires_in_days: int | None = Field(
default=None,
ge=1,
le=3650, # Maximum 10 years; keeps tokens from being effectively permanent while allowing long-lived CI/CD tokens.
description="Optional lifetime in days. If omitted the token never expires.",
)
class TokenResponse(BaseModel):
@@ -113,6 +137,7 @@ class TokenResponse(BaseModel):
last_used_ip: str | None
created_at: datetime | None
revoked_at: datetime | None
expires_at: datetime | None
model_config = {"from_attributes": True}
@@ -143,11 +168,16 @@ async def create_token(
token_hash_value = hash_token(plaintext)
prefix = plaintext[:12] # "de_" prefix + 9 random chars = 12 chars total
expires_at = None
if body.expires_in_days is not None:
expires_at = datetime.now(timezone.utc) + timedelta(days=body.expires_in_days)
db_token = ApiToken(
owner_id=owner_id,
name=body.name,
token_hash=token_hash_value,
token_prefix=prefix,
expires_at=expires_at,
)
try:
db.add(db_token)
@@ -168,6 +198,7 @@ async def create_token(
"last_used_ip": db_token.last_used_ip,
"created_at": db_token.created_at,
"revoked_at": db_token.revoked_at,
"expires_at": db_token.expires_at,
"token": plaintext,
}
@@ -177,48 +208,114 @@ async def list_tokens(
owner_id: CurrentOwner,
db: DbSession,
) -> list[dict[str, Any]]:
"""List all API tokens for the authenticated user."""
tokens = db.query(ApiToken).filter(ApiToken.owner_id == owner_id).order_by(ApiToken.created_at.desc()).all()
return [
{
"id": t.id,
"name": t.name,
"token_prefix": t.token_prefix,
"is_active": t.is_active,
"last_used_at": t.last_used_at,
"last_used_ip": t.last_used_ip,
"created_at": t.created_at,
"revoked_at": t.revoked_at,
}
for t in tokens
]
"""List non-mobile API tokens for the authenticated user.
Mobile tokens (whose names start with ``"Mobile App"``) are excluded
from this list; they are managed on the dedicated Devices page via
``GET /api/api-tokens/mobile``.
"""
tokens = (
db.query(ApiToken)
.filter(
ApiToken.owner_id == owner_id,
~ApiToken.name.startswith(MOBILE_TOKEN_PREFIX),
)
.order_by(ApiToken.created_at.desc())
.all()
)
return [_token_to_dict(t) for t in tokens]
@router.get("/mobile", response_model=list[TokenResponse])
async def list_mobile_tokens(
owner_id: CurrentOwner,
db: DbSession,
) -> list[dict[str, Any]]:
"""List mobile API tokens for the authenticated user.
Returns tokens whose names start with ``"Mobile App"`` — these are
created via the mobile SSO flow or QR code login.
"""
tokens = (
db.query(ApiToken)
.filter(
ApiToken.owner_id == owner_id,
ApiToken.name.startswith(MOBILE_TOKEN_PREFIX),
)
.order_by(ApiToken.created_at.desc())
.all()
)
return [_token_to_dict(t) for t in tokens]
@router.delete("/{token_id}", status_code=status.HTTP_200_OK)
async def revoke_token(
async def revoke_or_delete_token(
token_id: int,
owner_id: CurrentOwner,
db: DbSession,
) -> dict[str, str]:
"""Revoke (soft-delete) an API token.
"""Revoke or permanently delete an API token.
The token row is kept for audit purposes but marked inactive with a
``revoked_at`` timestamp.
* **Active token** – soft-revoked: the row is kept for audit purposes
but marked inactive with a ``revoked_at`` timestamp.
* **Already-revoked token** – hard-deleted: the row is permanently
removed from the database.
"""
db_token = db.query(ApiToken).filter(ApiToken.id == token_id, ApiToken.owner_id == owner_id).first()
if not db_token:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Token not found")
if not db_token.is_active:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Token is already revoked")
if db_token.is_active:
# Soft-revoke the active token.
try:
db_token.is_active = False
db_token.revoked_at = datetime.now(timezone.utc)
db.commit()
except Exception:
db.rollback()
raise
logger.info("API token revoked: id=%s owner=%s", token_id, owner_id)
return {"detail": "Token revoked"}
# Hard-delete an already-revoked token.
try:
db_token.is_active = False
db_token.revoked_at = datetime.now(timezone.utc)
db.delete(db_token)
db.commit()
except Exception:
db.rollback()
raise
logger.info("API token permanently deleted: id=%s owner=%s", token_id, owner_id)
return {"detail": "Token deleted"}
logger.info("API token revoked: id=%s owner=%s", token_id, owner_id)
return {"detail": "Token revoked"}
@router.post("/{token_id}/reactivate", status_code=status.HTTP_200_OK, response_model=TokenResponse)
async def reactivate_token(
token_id: int,
owner_id: CurrentOwner,
db: DbSession,
) -> dict[str, Any]:
"""Reactivate a previously revoked API token.
Clears the ``revoked_at`` timestamp and sets ``is_active`` back to
``True``. The token can be used for authentication again immediately.
If the token had an ``expires_at`` in the past the caller should
consider re-creating a new token instead.
"""
db_token = db.query(ApiToken).filter(ApiToken.id == token_id, ApiToken.owner_id == owner_id).first()
if not db_token:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Token not found")
if db_token.is_active:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Token is already active")
try:
db_token.is_active = True
db_token.revoked_at = None
db.commit()
db.refresh(db_token)
except Exception:
db.rollback()
raise
logger.info("API token reactivated: id=%s owner=%s", token_id, owner_id)
return _token_to_dict(db_token)
app/api/classification_rules.py
+325
View File
@@ -0,0 +1,325 @@
"""Classification Rules API endpoints.
Provides CRUD operations for managing custom document classification rules.
System-wide rules (``owner_id IS NULL``) can only be managed by admins.
"""
from __future__ import annotations
import logging
from typing import Annotated, Any
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from app.auth import require_login
from app.database import get_db
from app.models import ClassificationRuleModel
from app.utils.classification_rules import (
BUILTIN_CATEGORIES,
RULE_TYPE_CONTENT,
RULE_TYPE_FILENAME,
RULE_TYPE_METADATA,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/classification-rules", tags=["classification"])
DbSession = Annotated[Session, Depends(get_db)]
_VALID_RULE_TYPES = {RULE_TYPE_FILENAME, RULE_TYPE_CONTENT, RULE_TYPE_METADATA}
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_user_id(request: Request) -> str:
"""Extract the user identifier from the request session."""
user = getattr(request.state, "user", None)
if user and hasattr(user, "get"):
return user.get("sub") or user.get("email") or "anonymous"
return "anonymous"
def _is_admin(request: Request) -> bool:
"""Check whether the current user is an admin."""
user = getattr(request.state, "user", None)
if user and hasattr(user, "get"):
groups = user.get("groups", [])
return "admin" in groups or "Admin" in groups
return False
# ---------------------------------------------------------------------------
# Pydantic schemas
# ---------------------------------------------------------------------------
class RuleCreate(BaseModel):
"""Schema for creating a classification rule."""
name: str = Field(..., min_length=1, max_length=255)
category: str = Field(..., min_length=1, max_length=100)
rule_type: str = Field(..., description="One of: filename_pattern, content_keyword, metadata_match")
pattern: str = Field(..., min_length=1, max_length=1000)
priority: int = Field(default=0, ge=0, le=1000)
case_sensitive: bool = False
enabled: bool = True
class RuleUpdate(BaseModel):
"""Schema for updating a classification rule."""
name: str | None = Field(default=None, min_length=1, max_length=255)
category: str | None = Field(default=None, min_length=1, max_length=100)
rule_type: str | None = Field(default=None)
pattern: str | None = Field(default=None, min_length=1, max_length=1000)
priority: int | None = Field(default=None, ge=0, le=1000)
case_sensitive: bool | None = None
enabled: bool | None = None
class RuleResponse(BaseModel):
"""Schema for a classification rule response."""
id: int
owner_id: str | None
name: str
category: str
rule_type: str
pattern: str
priority: int
case_sensitive: bool
enabled: bool
model_config = {"from_attributes": True}
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.get("/categories")
@require_login
async def list_categories(request: Request) -> dict[str, str]:
"""Return all built-in classification categories.
Custom categories created via rules are not included here; they are
discovered dynamically when rules are evaluated.
"""
return BUILTIN_CATEGORIES
@router.get("/rule-types")
@require_login
async def list_rule_types(request: Request) -> list[dict[str, str]]:
"""Return the supported rule types with descriptions."""
return [
{
"type": RULE_TYPE_FILENAME,
"label": "Filename Pattern",
"description": "Regex pattern matched against the original filename.",
},
{
"type": RULE_TYPE_CONTENT,
"label": "Content Keyword",
"description": "Pipe-separated keywords matched against the OCR text.",
},
{
"type": RULE_TYPE_METADATA,
"label": "Metadata Match",
"description": "field=value pattern matched against existing AI metadata.",
},
]
@router.get("/")
@require_login
async def list_rules(request: Request, db: DbSession) -> list[dict[str, Any]]:
"""List classification rules visible to the current user.
Returns both system rules (``owner_id IS NULL``) and the user's own rules.
"""
user_id = _get_user_id(request)
rules = (
db.query(ClassificationRuleModel)
.filter((ClassificationRuleModel.owner_id.is_(None)) | (ClassificationRuleModel.owner_id == user_id))
.order_by(ClassificationRuleModel.priority.desc(), ClassificationRuleModel.id)
.all()
)
return [
{
"id": r.id,
"owner_id": r.owner_id,
"name": r.name,
"category": r.category,
"rule_type": r.rule_type,
"pattern": r.pattern,
"priority": r.priority,
"case_sensitive": r.case_sensitive,
"enabled": r.enabled,
}
for r in rules
]
@router.post("/", status_code=status.HTTP_201_CREATED)
@require_login
async def create_rule(request: Request, body: RuleCreate, db: DbSession) -> dict[str, Any]:
"""Create a new custom classification rule.
The rule is owned by the current user. Admins may create system-wide
rules by setting ``owner_id`` to ``null`` (not yet exposed).
"""
if body.rule_type not in _VALID_RULE_TYPES:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid rule_type. Must be one of: {', '.join(sorted(_VALID_RULE_TYPES))}",
)
user_id = _get_user_id(request)
# Check for duplicate name within the user's scope
existing = (
db.query(ClassificationRuleModel)
.filter(ClassificationRuleModel.owner_id == user_id, ClassificationRuleModel.name == body.name)
.first()
)
if existing:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=f"A rule named '{body.name}' already exists.",
)
rule = ClassificationRuleModel(
owner_id=user_id,
name=body.name,
category=body.category,
rule_type=body.rule_type,
pattern=body.pattern,
priority=body.priority,
case_sensitive=body.case_sensitive,
enabled=body.enabled,
)
try:
db.add(rule)
db.commit()
db.refresh(rule)
except Exception:
db.rollback()
raise
logger.info("Classification rule created: id=%s, user=%s", rule.id, user_id)
return {
"id": rule.id,
"owner_id": rule.owner_id,
"name": rule.name,
"category": rule.category,
"rule_type": rule.rule_type,
"pattern": rule.pattern,
"priority": rule.priority,
"case_sensitive": rule.case_sensitive,
"enabled": rule.enabled,
}
@router.get("/{rule_id}")
@require_login
async def get_rule(request: Request, rule_id: int, db: DbSession) -> dict[str, Any]:
"""Get a single classification rule by ID."""
user_id = _get_user_id(request)
rule = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.id == rule_id).first()
if rule is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
# Users can see system rules and their own rules
if rule.owner_id is not None and rule.owner_id != user_id and not _is_admin(request):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
return {
"id": rule.id,
"owner_id": rule.owner_id,
"name": rule.name,
"category": rule.category,
"rule_type": rule.rule_type,
"pattern": rule.pattern,
"priority": rule.priority,
"case_sensitive": rule.case_sensitive,
"enabled": rule.enabled,
}
@router.put("/{rule_id}")
@require_login
async def update_rule(request: Request, rule_id: int, body: RuleUpdate, db: DbSession) -> dict[str, Any]:
"""Update an existing classification rule.
Users can only update their own rules. Admins can update any rule.
"""
user_id = _get_user_id(request)
rule = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.id == rule_id).first()
if rule is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
if rule.owner_id != user_id and not _is_admin(request):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Cannot modify this rule")
if body.rule_type is not None and body.rule_type not in _VALID_RULE_TYPES:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid rule_type. Must be one of: {', '.join(sorted(_VALID_RULE_TYPES))}",
)
update_data = body.model_dump(exclude_unset=True)
for field_name, value in update_data.items():
setattr(rule, field_name, value)
try:
db.commit()
db.refresh(rule)
except Exception:
db.rollback()
raise
logger.info("Classification rule updated: id=%s, user=%s", rule.id, user_id)
return {
"id": rule.id,
"owner_id": rule.owner_id,
"name": rule.name,
"category": rule.category,
"rule_type": rule.rule_type,
"pattern": rule.pattern,
"priority": rule.priority,
"case_sensitive": rule.case_sensitive,
"enabled": rule.enabled,
}
@router.delete("/{rule_id}", status_code=status.HTTP_204_NO_CONTENT)
@require_login
async def delete_rule(request: Request, rule_id: int, db: DbSession) -> None:
"""Delete a classification rule.
Users can only delete their own rules. Admins can delete any rule.
"""
user_id = _get_user_id(request)
rule = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.id == rule_id).first()
if rule is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
if rule.owner_id != user_id and not _is_admin(request):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Cannot delete this rule")
try:
db.delete(rule)
db.commit()
except Exception:
db.rollback()
raise
logger.info("Classification rule deleted: id=%s, user=%s", rule_id, user_id)
app/api/diagnostic.py
+61
View File
@@ -21,6 +21,67 @@ _DEFAULT_REDIS_URL = "redis://localhost:6379/0"
router = APIRouter()
# ---------------------------------------------------------------------------
# Unauthenticated probe endpoints for Kubernetes liveness / readiness checks.
# These intentionally skip authentication so that kubelet can reach them
# without credentials. They live under /diagnostic/healthz/* so that the
# existing authenticated /diagnostic/health endpoint is unaffected.
# ---------------------------------------------------------------------------
@router.get("/diagnostic/healthz/live")
async def liveness_probe() -> JSONResponse:
"""Lightweight liveness probe for Kubernetes.
Returns **200 OK** as long as the process is running. Kubernetes uses
this to decide whether to *restart* the container — it should therefore
be as cheap as possible and **never** check external dependencies.
**Authentication:** None (designed for kubelet probes).
"""
return JSONResponse(content={"status": "ok"}, status_code=200)
@router.get("/diagnostic/healthz/ready")
async def readiness_probe() -> JSONResponse:
"""Readiness probe for Kubernetes.
Verifies that the application can serve traffic by checking the database
and Redis. Kubernetes uses this to decide whether to *route traffic* to
the pod.
Returns **200 OK** when all critical subsystems are reachable, or
**503 Service Unavailable** when the database is down.
**Authentication:** None (designed for kubelet probes).
"""
checks: dict[str, dict[str, str]] = {}
db_ok = False
# ── Database check ─────────────────────────────────────────────────
try:
with engine.connect() as conn:
conn.execute(text("SELECT 1"))
checks["database"] = {"status": "ok"}
db_ok = True
except Exception as exc:
logger.warning("Readiness probe: database check failed: %s", exc)
checks["database"] = {"status": "error", "detail": str(exc)}
# ── Redis check ────────────────────────────────────────────────────
try:
redis_url = settings.redis_url or _DEFAULT_REDIS_URL
r = redis_lib.from_url(redis_url, socket_connect_timeout=2, socket_timeout=2)
r.ping()
checks["redis"] = {"status": "ok"}
except Exception as exc:
logger.warning("Readiness probe: Redis check failed: %s", exc)
checks["redis"] = {"status": "error", "detail": str(exc)}
http_status = 503 if not db_ok else 200
overall = "ready" if db_ok else "not_ready"
return JSONResponse(content={"status": overall, "checks": checks}, status_code=http_status)
@router.get("/diagnostic/health")
@require_login
app/api/dropbox.py
+173
View File
@@ -5,6 +5,7 @@ Dropbox API endpoints
import logging
import os
from typing import Annotated, Optional
from urllib.parse import quote
import httpx
from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
@@ -23,6 +24,93 @@ logger = logging.getLogger(__name__)
router = APIRouter()
def _build_dropbox_redirect_uri(request: Request) -> str:
"""Build the Dropbox OAuth callback redirect URI.
Uses ``PUBLIC_BASE_URL`` when configured (recommended for deployments behind
a reverse proxy that doesn't forward ``X-Forwarded-Proto``). Falls back to
deriving the URI from the incoming request's scheme and host headers.
"""
if settings.public_base_url:
return settings.public_base_url.rstrip("/") + "/dropbox-callback"
return f"{request.url.scheme}://{request.url.netloc}/dropbox-callback"
@router.get("/dropbox/global-authorize-url")
@require_login
async def dropbox_global_authorize_url(request: Request):
"""Return the Dropbox OAuth authorization URL using the global app credentials.
This endpoint is used when ``DROPBOX_ALLOW_GLOBAL_CREDENTIALS_FOR_INTEGRATIONS``
is enabled so that users can authorize their personal Dropbox integration without
needing to supply their own app key/secret. Only the public ``app_key`` is
embedded in the URL; the ``app_secret`` is never sent to the browser.
"""
if not settings.dropbox_allow_global_credentials_for_integrations:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Global credentials for integrations are not enabled",
)
if not settings.dropbox_app_key or not settings.dropbox_app_secret:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Global Dropbox credentials are not configured",
)
redirect_uri = _build_dropbox_redirect_uri(request)
authorize_url = (
"https://www.dropbox.com/oauth2/authorize"
f"?client_id={settings.dropbox_app_key}"
"&response_type=code"
"&token_access_type=offline"
f"&redirect_uri={quote(redirect_uri, safe='')}"
)
return {"authorize_url": authorize_url}
@router.post("/dropbox/exchange-token-global")
@require_login
async def exchange_dropbox_token_global(
request: Request,
code: Annotated[str, Form(...)],
redirect_uri: Annotated[str, Form(...)],
):
"""Exchange an authorization code using the global Dropbox app credentials.
Used when ``DROPBOX_ALLOW_GLOBAL_CREDENTIALS_FOR_INTEGRATIONS`` is enabled so
that the ``app_secret`` is never exposed to the browser. Only the OAuth code
and redirect URI need to be supplied by the client.
"""
if not settings.dropbox_allow_global_credentials_for_integrations:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Global credentials for integrations are not enabled",
)
if not settings.dropbox_app_key or not settings.dropbox_app_secret:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Global Dropbox credentials are not configured",
)
token_url = "https://api.dropboxapi.com/oauth2/token"
payload = {
"client_id": settings.dropbox_app_key,
"client_secret": settings.dropbox_app_secret,
"code": code,
"redirect_uri": redirect_uri,
"grant_type": "authorization_code",
}
token_data = exchange_oauth_token(provider_name="Dropbox", token_url=token_url, payload=payload)
return {
"refresh_token": token_data["refresh_token"],
"access_token": token_data["access_token"],
"expires_in": token_data.get("expires_in", 14400),
# Return the public app_key so the callback can store it in the integration
"app_key": settings.dropbox_app_key,
}
@router.post("/dropbox/exchange-token")
@require_login
async def exchange_dropbox_token(
@@ -210,6 +298,91 @@ async def test_dropbox_token(request: Request):
return {"status": "error", "message": f"Connection error: {str(e)}"}
@router.post("/dropbox/list-folders")
@require_login
async def list_dropbox_folders(
request: Request,
access_token: Annotated[str, Form(...)],
path: Annotated[str, Form()] = "",
):
"""
List folders in a Dropbox account for the directory selector.
Accepts an OAuth access token (short-lived) and a path to list.
Returns a flat list of folder entries under the given path.
"""
try:
# Normalize path: Dropbox API uses "" for root, otherwise "/path"
folder_path = path.strip()
if folder_path == "/":
folder_path = ""
elif folder_path and not folder_path.startswith("/"):
folder_path = f"/{folder_path}"
headers = {
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/json",
}
payload = {
"path": folder_path,
"recursive": False,
"include_deleted": False,
"include_has_explicit_shared_members": False,
"include_mounted_folders": True,
}
response = requests.post(
"https://api.dropboxapi.com/2/files/list_folder",
headers=headers,
json=payload,
timeout=settings.http_request_timeout,
)
if response.status_code == 401:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Access token is invalid or expired. Please re-authorize.",
)
if response.status_code != 200:
logger.error(f"Dropbox list_folder failed: {response.status_code} {response.text}")
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail=f"Failed to list Dropbox folders: {response.text}",
)
data = response.json()
folders = []
for entry in data.get("entries", []):
if entry.get(".tag") == "folder":
folders.append(
{
"name": entry["name"],
"path": entry["path_display"],
"id": entry.get("id", ""),
}
)
# Sort folders alphabetically
folders.sort(key=lambda f: f["name"].lower())
return {
"folders": folders,
"path": folder_path or "/",
"has_more": data.get("has_more", False),
}
except HTTPException:
raise
except Exception as e:
logger.exception(f"Error listing Dropbox folders: {e}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to list folders: {str(e)}",
)
@router.post("/dropbox/save-settings")
@require_login
async def save_dropbox_settings(
app/api/files.py
+64 -22
View File
@@ -20,6 +20,7 @@ from sqlalchemy.orm import Session
from app.auth import require_login
from app.config import settings
from app.database import get_db
from app.middleware.upload_rate_limit import require_upload_rate_limit
from app.models import FileProcessingStep, FileRecord, ProcessingLog
from app.tasks.convert_to_pdf import convert_to_pdf
from app.tasks.process_document import process_document
@@ -346,7 +347,9 @@ def bulk_delete_files(request: Request, file_ids: List[int], db: DbSession):
try:
# Find all file records
file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
query = apply_owner_filter(query, request)
file_records = query.all()
if not file_records:
raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -385,7 +388,9 @@ def bulk_reprocess_files(request: Request, file_ids: List[int], db: DbSession):
"""
try:
# Find all file records
file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
query = apply_owner_filter(query, request)
file_records = query.all()
if not file_records:
raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -457,7 +462,9 @@ def bulk_reprocess_files_cloud_ocr(request: Request, file_ids: List[int], db: Db
Useful for re-running OCR on files with poor text quality or missing OCR text.
"""
try:
file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
query = apply_owner_filter(query, request)
file_records = query.all()
if not file_records:
raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -537,7 +544,9 @@ def bulk_download_files(request: Request, file_ids: List[int], db: DbSession):
Files not found on disk are silently skipped.
"""
try:
file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
query = apply_owner_filter(query, request)
file_records = query.all()
if not file_records:
raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -619,7 +628,9 @@ def reprocess_single_file(request: Request, file_id: int, db: DbSession):
"""
try:
# Find the file record
file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
query = db.query(FileRecord).filter(FileRecord.id == file_id)
query = apply_owner_filter(query, request)
file_record = query.first()
if not file_record:
raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -675,7 +686,9 @@ def reprocess_with_cloud_ocr(request: Request, file_id: int, db: DbSession):
"""
try:
# Find the file record
file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
query = db.query(FileRecord).filter(FileRecord.id == file_id)
query = apply_owner_filter(query, request)
file_record = query.first()
if not file_record:
raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -938,7 +951,9 @@ def retry_subtask(
"""
try:
# Find the file record
file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
query = db.query(FileRecord).filter(FileRecord.id == file_id)
query = apply_owner_filter(query, request)
file_record = query.first()
if not file_record:
raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -1080,7 +1095,9 @@ def get_file_preview(
try:
# Find the file record
file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
query = db.query(FileRecord).filter(FileRecord.id == file_id)
query = apply_owner_filter(query, request)
file_record = query.first()
if not file_record:
raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -1160,7 +1177,9 @@ def download_file(
try:
# Find the file record
file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
query = db.query(FileRecord).filter(FileRecord.id == file_id)
query = apply_owner_filter(query, request)
file_record = query.first()
if not file_record:
raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -1249,7 +1268,12 @@ async def _save_upload_file_chunks(file: UploadFile, target_path: str, max_size:
def _check_for_exact_duplicate(db: DbSession, target_path: str, safe_filename: str) -> dict | None:
"""Check for an exact duplicate of the uploaded file and return a warning if found."""
"""Check for an exact duplicate of the uploaded file.
Returns a dict with duplicate info when the file's SHA-256 hash matches an
already-processed document, or ``None`` when no duplicate is found (or
deduplication is disabled).
"""
if not settings.enable_deduplication:
return None
@@ -1268,8 +1292,8 @@ def _check_for_exact_duplicate(db: DbSession, target_path: str, safe_filename: s
"original_file_id": existing.id,
"original_filename": existing.original_filename,
"message": (
"This file appears to be an exact duplicate of an already-processed document. "
"It will still be queued but will be flagged as a duplicate."
"This file is an exact duplicate of an already-processed document. "
"It has not been queued for processing again."
),
}
except Exception as e:
@@ -1280,7 +1304,12 @@ def _check_for_exact_duplicate(db: DbSession, target_path: str, safe_filename: s
@router.post("/ui-upload")
@require_login
async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...)):
async def ui_upload(
request: Request,
db: DbSession,
file: UploadFile = File(...),
_rate_ok: None = Depends(require_upload_rate_limit),
):
"""Endpoint to accept a user-uploaded file and enqueue it for processing."""
workdir = settings.workdir
@@ -1366,6 +1395,25 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
logger.info(f"Saved uploaded file '{safe_filename}' as '{target_filename}'")
file_size = written_size
# ── Early duplicate rejection ──────────────────────────────────────────
# Check for exact duplicates (same SHA-256 hash) BEFORE enqueuing a
# processing task. When deduplication is enabled and the file already
# exists, we skip processing entirely, clean up the temp file, and
# return the existing file's information to the caller.
exact_duplicate = _check_for_exact_duplicate(db, target_path, safe_filename)
if exact_duplicate:
# Remove the just-saved temp file — it's a duplicate.
try:
os.remove(target_path)
except OSError:
pass
return {
"status": "duplicate",
"original_filename": safe_filename,
"stored_filename": target_filename,
"duplicate_of": exact_duplicate,
}
# Determine if the file is a PDF or needs conversion
mime_type, _ = mimetypes.guess_type(target_path)
file_ext = os.path.splitext(target_path)[1].lower()
@@ -1429,6 +1477,8 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
".tif",
".webp",
".svg",
".heic",
".heif",
}:
# If it's an image, convert to PDF first
task = convert_to_pdf.delay(target_path, original_filename=safe_filename, owner_id=upload_owner_id)
@@ -1442,20 +1492,12 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
logger.warning(f"Unsupported MIME type {mime_type} for {target_path}, attempting conversion")
task = convert_to_pdf.delay(target_path, original_filename=safe_filename, owner_id=upload_owner_id)
# Check for exact duplicates (same SHA-256 hash) before returning.
# This gives the caller an immediate warning without waiting for the pipeline.
# Only performed when deduplication is enabled in settings.
exact_duplicate_warning = _check_for_exact_duplicate(db, target_path, safe_filename)
response: dict = {
return {
"task_id": task.id,
"status": "queued",
"original_filename": safe_filename,
"stored_filename": target_filename,
}
if exact_duplicate_warning:
response["duplicate_warning"] = exact_duplicate_warning
return response
# ---------------------------------------------------------------------------
app/api/integrations.py
+57
View File
@@ -32,6 +32,21 @@ from app.utils.encryption import decrypt_value, encrypt_value
from app.utils.subscription import get_tier, get_user_tier_id
from app.utils.user_scope import get_current_owner_id
# Optional Dropbox SDK — imported at module level so tests can patch it cleanly.
try:
import dropbox as dbx_lib
from dropbox.exceptions import AuthError as _DropboxAuthError
from dropbox.exceptions import BadInputError as _DropboxBadInputError
except ImportError: # pragma: no cover
dbx_lib = None # type: ignore[assignment]
class _DropboxAuthError(Exception): # type: ignore[no-redef]
"""Stub — only used when the dropbox package is missing."""
class _DropboxBadInputError(Exception): # type: ignore[no-redef]
"""Stub — only used when the dropbox package is missing."""
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/integrations", tags=["integrations"])
@@ -550,6 +565,47 @@ def _test_s3_connection(config: dict[str, Any] | None, credentials: dict[str, An
return {"success": False, "message": "S3 connection failed"}
def _test_dropbox_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
"""Test a Dropbox connection by verifying OAuth credentials via the Dropbox API."""
if dbx_lib is None:
return {"success": False, "message": "dropbox package is not installed"} # pragma: no cover
creds = credentials or {}
app_key = creds.get("app_key", "")
app_secret = creds.get("app_secret", "")
refresh_token = creds.get("refresh_token", "")
if not refresh_token:
return {"success": False, "message": "Missing required credential: refresh_token"}
if not app_key or not app_secret:
return {"success": False, "message": "Missing required credentials: app_key and app_secret"}
try:
dbx = dbx_lib.Dropbox(
app_key=app_key,
app_secret=app_secret,
oauth2_refresh_token=refresh_token,
)
account = dbx.users_get_current_account()
display_name = getattr(account, "name", None)
name_str = ""
if display_name:
name_str = f" ({getattr(display_name, 'display_name', '') or ''})"
return {"success": True, "message": f"Dropbox connection successful{name_str}"}
except _DropboxAuthError as exc:
logger.warning("Dropbox auth error: %s", exc)
return {
"success": False,
"message": "Dropbox authentication failed — check app_key, app_secret, and refresh_token",
}
except _DropboxBadInputError as exc:
logger.warning("Dropbox bad input error: %s", exc)
return {"success": False, "message": "Dropbox connection failed — invalid credentials format"}
except Exception as exc: # noqa: BLE001
logger.warning("Dropbox connection error: %s", exc)
return {"success": False, "message": "Dropbox connection failed — check credentials and network connectivity"}
def _test_webdav_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
"""Test a WebDAV/Nextcloud connection by issuing an HTTP PROPFIND."""
import urllib.request
@@ -596,6 +652,7 @@ def _test_webdav_connection(config: dict[str, Any] | None, credentials: dict[str
_CONNECTION_TESTERS: dict[str, Any] = {
IntegrationType.DROPBOX: _test_dropbox_connection,
IntegrationType.IMAP: _test_imap_connection,
IntegrationType.S3: _test_s3_connection,
IntegrationType.WEBDAV: _test_webdav_connection,
app/api/mobile.py
+23 -8
View File
@@ -120,6 +120,7 @@ class WhoAmIResponse(BaseModel):
email: str | None
avatar_url: str | None
is_admin: bool
preferred_language: str | None
# ---------------------------------------------------------------------------
@@ -273,31 +274,44 @@ async def list_devices(
return [_device_to_response(d) for d in devices]
@router.delete("/devices/{device_id}", status_code=status.HTTP_204_NO_CONTENT)
@router.delete("/devices/{device_id}", status_code=status.HTTP_200_OK)
@require_login
async def deactivate_device(
request: Request,
device_id: int,
owner_id: CurrentOwner,
db: DbSession,
) -> None:
"""Deactivate a push-notification device registration.
) -> dict[str, str]:
"""Deactivate or permanently delete a push-notification device registration.
The device record is kept for audit purposes but will no longer receive
push notifications.
* **Active device** – soft-deactivated: the record is kept for audit
purposes but will no longer receive push notifications.
* **Already-inactive device** – hard-deleted: the record is permanently
removed from the database.
"""
device = db.get(MobileDevice, device_id)
if not device or device.owner_id != owner_id:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Device not found")
device.is_active = False
if device.is_active:
device.is_active = False
try:
db.commit()
except Exception:
db.rollback()
raise
logger.info("Mobile device deactivated: id=%s owner=%s", device_id, owner_id)
return {"detail": "Device deactivated"}
# Hard-delete an already-inactive device.
try:
db.delete(device)
db.commit()
except Exception:
db.rollback()
raise
logger.info("Mobile device deactivated: id=%s owner=%s", device_id, owner_id)
logger.info("Mobile device permanently deleted: id=%s owner=%s", device_id, owner_id)
return {"detail": "Device deleted"}
@router.get("/whoami", response_model=WhoAmIResponse)
@@ -344,4 +358,5 @@ async def whoami(
"email": email,
"avatar_url": avatar_url,
"is_admin": is_admin,
"preferred_language": profile.preferred_language if profile else None,
}
app/api/onedrive.py
+97
View File
@@ -56,6 +56,7 @@ async def exchange_onedrive_token(
# Return just what's needed by the frontend
return {
"refresh_token": token_data["refresh_token"],
"access_token": token_data.get("access_token", ""),
"expires_in": token_data.get("expires_in", 3600),
}
@@ -183,6 +184,102 @@ async def test_onedrive_token(request: Request):
return {"status": "error", "message": f"Connection error: {str(e)}"}
@router.post("/onedrive/list-folders")
@require_login
async def list_onedrive_folders(
request: Request,
access_token: Annotated[str, Form(...)],
path: Annotated[str, Form()] = "",
):
"""
List folders in a OneDrive account for the directory selector.
Accepts an OAuth access token (short-lived) and a path to list.
Returns a flat list of folder entries under the given path.
"""
try:
folder_path = path.strip().strip("/")
headers = {
"Authorization": f"Bearer {access_token}",
}
# Build the Graph API URL for listing children
if not folder_path or folder_path == "root":
url = "https://graph.microsoft.com/v1.0/me/drive/root/children"
else:
url = f"https://graph.microsoft.com/v1.0/me/drive/root:/{folder_path}:/children"
# Only request folders and minimal fields
params = {
"$filter": "folder ne null",
"$select": "name,id,parentReference,folder",
"$top": "200",
}
response = requests.get(
url,
headers=headers,
params=params,
timeout=settings.http_request_timeout,
)
if response.status_code == 401:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Access token is invalid or expired. Please re-authorize.",
)
if response.status_code != 200:
logger.error(f"OneDrive list children failed: {response.status_code} {response.text}")
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail=f"Failed to list OneDrive folders: {response.text}",
)
data = response.json()
folders = []
for item in data.get("value", []):
if "folder" in item:
parent_path = ""
if item.get("parentReference", {}).get("path"):
# parentReference.path looks like /drive/root:/some/path
raw_parent = item["parentReference"]["path"]
prefix = "/drive/root:"
if raw_parent.startswith(prefix):
parent_path = raw_parent[len(prefix) :]
elif raw_parent == "/drive/root":
parent_path = ""
item_path = f"{parent_path}/{item['name']}" if parent_path else f"/{item['name']}"
folders.append(
{
"name": item["name"],
"path": item_path,
"id": item.get("id", ""),
"child_count": item.get("folder", {}).get("childCount", 0),
}
)
# Sort folders alphabetically
folders.sort(key=lambda f: f["name"].lower())
return {
"folders": folders,
"path": f"/{folder_path}" if folder_path else "/",
}
except HTTPException:
raise
except Exception as e:
logger.exception(f"Error listing OneDrive folders: {e}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to list folders: {str(e)}",
)
def format_time_remaining(time_delta):
"""Format a timedelta into a human-readable string."""
if time_delta.total_seconds() <= 0:
app/api/pipelines.py
+8 -2
View File
@@ -117,8 +117,14 @@ PIPELINE_STEP_TYPES: dict[str, dict[str, Any]] = {
},
"classify": {
"label": "Document Classification",
"description": "Classify the document type using AI without full metadata extraction.",
"config_schema": {},
"description": "Classify the document type using built-in and custom rules (filename patterns, content keywords, metadata matching).",
"config_schema": {
"use_builtin_rules": {
"type": "boolean",
"default": True,
"description": "Include the pre-built classification rules (invoice, contract, receipt, etc.).",
},
},
},
}
app/api/qr_auth.py
+233
View File
@@ -0,0 +1,233 @@
"""QR code login API endpoints for mobile app authentication.
Provides a secure challenge-response flow for logging into the mobile app
by scanning a QR code displayed in the web interface:
1. **Web user** calls ``POST /qr-auth/challenge`` → receives a time-limited
challenge token (encoded in the QR code).
2. **Web UI** polls ``GET /qr-auth/challenge/{id}/status`` to detect when
the mobile app has claimed the challenge.
3. **Mobile app** scans the QR code and calls ``POST /qr-auth/claim`` with
the challenge token + device name → receives an API token.
Security properties:
* Challenges expire after a configurable TTL (default 2 minutes).
* Single-use: once claimed, a challenge cannot be reused (replay-safe).
* Cryptographically random 64-byte tokens.
* IP addresses are logged for audit.
"""
from __future__ import annotations
import base64
import io
import logging
from datetime import datetime
from typing import Annotated, Any
import segno
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from app.auth import require_login
from app.database import get_db
from app.middleware.audit_log import get_client_ip
from app.utils.session_manager import (
claim_qr_challenge,
create_qr_challenge,
get_challenge_status,
)
from app.utils.user_scope import get_current_owner_id
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/qr-auth", tags=["qr-auth"])
DbSession = Annotated[Session, Depends(get_db)]
# ---------------------------------------------------------------------------
# Auth helper
# ---------------------------------------------------------------------------
def _get_owner_id(request: Request) -> str:
"""Return the current user's owner ID, raising 401 if unauthenticated."""
owner_id = get_current_owner_id(request)
if not owner_id:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
return owner_id
CurrentOwner = Annotated[str, Depends(_get_owner_id)]
# ---------------------------------------------------------------------------
# Request / Response schemas
# ---------------------------------------------------------------------------
class CreateChallengeResponse(BaseModel):
"""Response after creating a QR login challenge."""
challenge_id: int
challenge_token: str
expires_at: datetime
ttl_seconds: int = Field(description="Seconds until the challenge expires (use for client-side countdown).")
qr_payload: str = Field(description="The string to encode in the QR code.")
qr_code_svg: str = Field(description="Base64-encoded SVG data URI of the QR code, ready for use in an <img> src.")
class ChallengeStatusResponse(BaseModel):
"""Response for polling the status of a QR challenge."""
id: int
status: str # "pending", "claimed", "expired", "cancelled"
device_name: str | None = None
claimed_at: datetime | None = None
expires_at: datetime
class ClaimChallengeRequest(BaseModel):
"""Request body for claiming a QR login challenge."""
challenge_token: str = Field(min_length=1, max_length=256)
device_name: str = Field(
default="Mobile App",
min_length=1,
max_length=120,
description="Human-readable device name.",
)
class ClaimChallengeResponse(BaseModel):
"""Response after successfully claiming a QR challenge."""
token: str
token_id: int
name: str
owner_id: str
created_at: datetime
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
# QR code rendering parameters
_QR_ERROR_LEVEL = "M" # Medium error correction (~15% recovery); sufficient for on-screen display
_QR_SCALE = 4 # Each QR module is rendered as 4×4 SVG pixels
def _generate_qr_svg(payload: str) -> str:
"""Generate a QR code for *payload* and return it as a base64 SVG data URI.
Using ``segno`` (pure-Python, no Pillow dependency) and SVG output so the
QR code scales crisply at any resolution without requiring a canvas or any
client-side JavaScript library.
"""
qr = segno.make(payload, error=_QR_ERROR_LEVEL)
buf = io.BytesIO()
qr.save(buf, kind="svg", scale=_QR_SCALE, xmldecl=False, svgclass=None, lineclass=None, omitsize=True)
svg_bytes = buf.getvalue()
return "data:image/svg+xml;base64," + base64.b64encode(svg_bytes).decode("ascii")
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.post("/challenge", status_code=status.HTTP_201_CREATED, response_model=CreateChallengeResponse)
@require_login
async def create_challenge(
request: Request,
owner_id: CurrentOwner,
db: DbSession,
) -> dict[str, Any]:
"""Create a new QR login challenge.
The returned ``qr_payload`` should be encoded into a QR code and
displayed to the user. The mobile app scans this QR code and
calls the ``/claim`` endpoint.
"""
ip = get_client_ip(request)
challenge = create_qr_challenge(db, owner_id, ip_address=ip)
# The QR payload is a JSON-like string with enough info for the mobile
# app to know the server URL and challenge token.
base_url = str(request.base_url).rstrip("/")
qr_payload = f"docuelevate://qr-login?token={challenge.challenge_token}&server={base_url}"
# Compute the TTL in seconds so the client can run a countdown timer
# without comparing absolute timestamps (which breaks when client and
# server clocks are out of sync).
ttl_seconds = max(0, int((challenge.expires_at - challenge.created_at).total_seconds()))
return {
"challenge_id": challenge.id,
"challenge_token": challenge.challenge_token,
"expires_at": challenge.expires_at,
"ttl_seconds": ttl_seconds,
"qr_payload": qr_payload,
"qr_code_svg": _generate_qr_svg(qr_payload),
}
@router.get("/challenge/{challenge_id}/status", response_model=ChallengeStatusResponse)
@require_login
async def poll_challenge_status(
request: Request,
challenge_id: int,
owner_id: CurrentOwner,
db: DbSession,
) -> dict[str, Any]:
"""Poll the status of a QR login challenge.
The web UI calls this endpoint every few seconds to check if the
mobile app has scanned the QR code and claimed the challenge.
"""
result = get_challenge_status(db, challenge_id, owner_id)
if not result:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Challenge not found")
return result
@router.post("/claim", response_model=ClaimChallengeResponse)
async def claim_challenge(
request: Request,
body: ClaimChallengeRequest,
db: DbSession,
) -> dict[str, Any]:
"""Claim a QR login challenge and receive an API token.
This endpoint is called by the mobile app after scanning a QR code.
It does **not** require authentication — the challenge token itself
serves as proof that the user authorized this login from their web
session.
"""
ip = get_client_ip(request)
result = claim_qr_challenge(db, body.challenge_token, device_name=body.device_name, ip_address=ip)
if not result:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Invalid, expired, or already claimed challenge.",
)
try:
from app.utils.audit_service import record_event
record_event(
db,
action="qr_login_claimed",
user=result["owner_id"],
resource_type="session",
ip_address=ip,
details={"device_name": body.device_name, "token_id": result["token_id"]},
severity="info",
)
except Exception:
logger.debug("Failed to write QR login audit event", exc_info=True)
return result
app/api/sessions.py
+196
View File
@@ -0,0 +1,196 @@
"""API endpoints for managing user sessions.
Provides endpoints for listing active sessions, revoking individual sessions,
and the "log off everywhere" feature that invalidates all sessions and API
tokens across all devices.
"""
from __future__ import annotations
import logging
from datetime import datetime
from typing import Annotated, Any
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import BaseModel
from sqlalchemy.orm import Session
from app.auth import require_login
from app.database import get_db
from app.middleware.audit_log import get_client_ip
from app.utils.session_manager import (
get_session_lifetime_days,
list_user_sessions,
revoke_all_sessions,
revoke_session,
)
from app.utils.user_scope import get_current_owner_id
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/sessions", tags=["sessions"])
DbSession = Annotated[Session, Depends(get_db)]
# ---------------------------------------------------------------------------
# Auth helper
# ---------------------------------------------------------------------------
def _get_owner_id(request: Request) -> str:
"""Return the current user's owner ID, raising 401 if unauthenticated."""
owner_id = get_current_owner_id(request)
if not owner_id:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
return owner_id
CurrentOwner = Annotated[str, Depends(_get_owner_id)]
# ---------------------------------------------------------------------------
# Response schemas
# ---------------------------------------------------------------------------
class SessionResponse(BaseModel):
"""Serialised user session for the management UI."""
id: int
device_info: str | None
ip_address: str | None
created_at: datetime
last_active_at: datetime
expires_at: datetime
is_current: bool = False
class SessionListResponse(BaseModel):
"""Response for listing active sessions."""
sessions: list[SessionResponse]
session_lifetime_days: int
class RevokeAllResponse(BaseModel):
"""Response after revoking all sessions."""
revoked_count: int
message: str
# ---------------------------------------------------------------------------
# Endpoints
# ---------------------------------------------------------------------------
@router.get("/", response_model=SessionListResponse)
@require_login
async def list_sessions(
request: Request,
owner_id: CurrentOwner,
db: DbSession,
) -> dict[str, Any]:
"""List all active sessions for the current user."""
sessions = list_user_sessions(db, owner_id)
# Determine which session is the current one
current_token = request.session.get("_session_token")
session_list = []
for s in sessions:
session_list.append(
{
"id": s.id,
"device_info": s.device_info,
"ip_address": s.ip_address,
"created_at": s.created_at,
"last_active_at": s.last_active_at,
"expires_at": s.expires_at,
"is_current": s.session_token == current_token if current_token else False,
}
)
return {
"sessions": session_list,
"session_lifetime_days": get_session_lifetime_days(),
}
@router.delete("/{session_id}", status_code=status.HTTP_204_NO_CONTENT)
@require_login
async def revoke_single_session(
request: Request,
session_id: int,
owner_id: CurrentOwner,
db: DbSession,
) -> None:
"""Revoke a specific session by ID."""
success = revoke_session(db, session_id, owner_id)
if not success:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Session not found")
try:
from app.utils.audit_service import record_event
record_event(
db,
action="session_revoked",
user=owner_id,
resource_type="session",
resource_id=str(session_id),
ip_address=get_client_ip(request),
severity="info",
)
except Exception:
logger.debug("Failed to write session revocation audit event", exc_info=True)
@router.post("/revoke-all", response_model=RevokeAllResponse)
@require_login
async def revoke_all(
request: Request,
owner_id: CurrentOwner,
db: DbSession,
) -> dict[str, Any]:
"""Revoke all sessions except the current one ("log off everywhere").
Also revokes all active API tokens for the user, which invalidates
mobile app sessions and any programmatic access.
"""
# Find current session to preserve it
current_token = request.session.get("_session_token")
current_session_id = None
if current_token:
from app.models import UserSession
current = db.query(UserSession).filter(UserSession.session_token == current_token).first()
if current:
current_session_id = current.id
count = revoke_all_sessions(
db,
owner_id,
except_session_id=current_session_id,
revoke_api_tokens=True,
)
try:
from app.utils.audit_service import record_event
record_event(
db,
action="revoke_all_sessions",
user=owner_id,
resource_type="session",
ip_address=get_client_ip(request),
details={"revoked_count": count},
severity="warning",
)
except Exception:
logger.debug("Failed to write revoke-all audit event", exc_info=True)
return {
"revoked_count": count,
"message": f"Successfully revoked {count} session(s) and all API tokens.",
}
app/api/system_reset.py
+124
View File
@@ -0,0 +1,124 @@
"""
System reset API endpoints for DocuElevate.
Provides admin-only REST endpoints for:
- Full system reset (wipe all user data)
- Reset with re-import (move originals → reimport folder, wipe, re-ingest)
Both operations require the ``ENABLE_FACTORY_RESET=True`` feature flag and
admin privileges.
"""
import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import BaseModel
from sqlalchemy.orm import Session
from app.config import settings
from app.database import get_db
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/admin/system-reset", tags=["system-reset"])
def _require_admin(request: Request) -> dict:
"""Ensure the caller is an admin. Raises 403 otherwise."""
user = request.session.get("user")
if not user or not user.get("is_admin"):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required")
return user
AdminUser = Annotated[dict, Depends(_require_admin)]
def _require_feature_enabled() -> None:
"""Raise 404 when the factory-reset feature flag is off."""
if not settings.enable_factory_reset:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="System reset is not enabled. Set ENABLE_FACTORY_RESET=True to activate.",
)
class ResetRequest(BaseModel):
"""Body for system reset endpoints. Requires explicit confirmation."""
confirmation: str
@router.post("/full")
async def full_reset(
body: ResetRequest,
_admin: AdminUser,
db: Session = Depends(get_db),
) -> dict:
"""Wipe all user data (database + work-files).
The caller must send ``{"confirmation": "DELETE"}`` to proceed.
"""
_require_feature_enabled()
if body.confirmation != "DELETE":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Confirmation required: send {"confirmation": "DELETE"} to proceed.',
)
from app.utils.system_reset import perform_full_reset
try:
result = perform_full_reset(db)
except Exception as exc:
logger.exception("Full system reset failed")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"System reset failed: {exc}",
) from exc
return {"status": "ok", "result": result}
@router.post("/reimport")
async def reset_and_reimport(
body: ResetRequest,
_admin: AdminUser,
db: Session = Depends(get_db),
) -> dict:
"""Move original files to a reimport folder, wipe everything, and
configure the reimport folder as a watch folder for automatic
re-ingestion.
The caller must send ``{"confirmation": "REIMPORT"}`` to proceed.
"""
_require_feature_enabled()
if body.confirmation != "REIMPORT":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail='Confirmation required: send {"confirmation": "REIMPORT"} to proceed.',
)
from app.utils.system_reset import perform_reset_and_reimport
try:
result = perform_reset_and_reimport(db)
except Exception as exc:
logger.exception("Reset-and-reimport failed")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Reset and reimport failed: {exc}",
) from exc
return {"status": "ok", "result": result}
@router.get("/status")
async def reset_status(_admin: AdminUser) -> dict:
"""Return whether the system reset feature is enabled."""
return {
"enabled": settings.enable_factory_reset,
"factory_reset_on_startup": settings.factory_reset_on_startup,
}
app/api/url_upload.py
+7 -2
View File
@@ -11,11 +11,12 @@ from typing import Optional
import aiofiles
import httpx
from fastapi import APIRouter, HTTPException, Request
from fastapi import APIRouter, Depends, HTTPException, Request
from pydantic import BaseModel, HttpUrl, field_validator
from app.auth import require_login
from app.config import settings
from app.middleware.upload_rate_limit import require_upload_rate_limit
from app.tasks.process_document import process_document
from app.utils.allowed_types import ALLOWED_MIME_TYPES
from app.utils.filename_utils import sanitize_filename
@@ -107,7 +108,11 @@ def validate_file_type(content_type: str, filename: str) -> bool:
@router.post("/process-url")
@require_login
async def process_url(request: Request, url_request: URLUploadRequest):
async def process_url(
request: Request,
url_request: URLUploadRequest,
_rate_ok: None = Depends(require_upload_rate_limit),
):
"""
Download a file from a URL and enqueue it for processing.
app/auth.py
+122 -3
View File
@@ -103,11 +103,18 @@ if AUTH_ENABLED and settings.social_auth_apple_enabled:
logger.warning("SOCIAL_AUTH_APPLE_ENABLED=true but client ID/team ID not configured")
if AUTH_ENABLED and settings.social_auth_dropbox_enabled:
if settings.social_auth_dropbox_client_id and settings.social_auth_dropbox_client_secret:
# Determine which credentials to use for Dropbox social login
_dropbox_client_id = settings.social_auth_dropbox_client_id
_dropbox_client_secret = settings.social_auth_dropbox_client_secret
if settings.social_auth_dropbox_use_global_credentials and not _dropbox_client_id:
_dropbox_client_id = settings.dropbox_app_key
_dropbox_client_secret = settings.dropbox_app_secret
if _dropbox_client_id and _dropbox_client_secret:
oauth.register(
name="dropbox",
client_id=settings.social_auth_dropbox_client_id,
client_secret=settings.social_auth_dropbox_client_secret,
client_id=_dropbox_client_id,
client_secret=_dropbox_client_secret,
authorize_url="https://www.dropbox.com/oauth2/authorize",
access_token_url="https://api.dropboxapi.com/oauth2/token",
userinfo_endpoint="https://api.dropboxapi.com/2/users/get_current_account",
@@ -129,6 +136,25 @@ def get_current_user(request: Request):
return api_user
session_user = request.session.get("user")
if session_user:
# Validate server-side session if a session token is present
session_token = request.session.get("_session_token")
if session_token:
try:
from app.database import SessionLocal
from app.utils.session_manager import validate_session
db = SessionLocal()
try:
valid = validate_session(db, session_token)
if not valid:
logger.debug("[AUTH] get_current_user: server-side session invalid — clearing")
request.session.pop("user", None)
request.session.pop("_session_token", None)
return None
finally:
db.close()
except Exception:
logger.debug("[AUTH] get_current_user: session validation error", exc_info=True)
logger.debug(
"[AUTH] get_current_user: resolved from session (user=%s)",
session_user.get("preferred_username") or session_user.get("email") or session_user.get("id"),
@@ -167,6 +193,16 @@ def _resolve_bearer_user(request: Request, db: Session) -> dict | None:
logger.debug("[AUTH] _resolve_bearer_user: no active API token matched the provided hash")
return None
# Reject tokens that have passed their optional expiry.
if db_token.expires_at is not None:
now_utc = datetime.now(timezone.utc)
expires_aware = db_token.expires_at
if expires_aware.tzinfo is None:
expires_aware = expires_aware.replace(tzinfo=timezone.utc)
if now_utc > expires_aware:
logger.debug("[AUTH] _resolve_bearer_user: API token id=%s has expired", db_token.id)
return None
logger.debug(
"[AUTH] _resolve_bearer_user: matched API token id=%s owner=%s",
db_token.id,
@@ -481,6 +517,27 @@ async def social_callback(request: Request, provider: str, db: Session = Depends
request.session["user"] = user_data
# Create server-side session for tracking and revocation
try:
from app.utils.session_manager import create_session
_session_user_id = (
user_data.get("sub")
or user_data.get("preferred_username")
or user_data.get("email")
or user_data.get("id")
)
if _session_user_id:
user_session = create_session(
db,
user_id=_session_user_id,
ip_address=get_client_ip(request),
user_agent=request.headers.get("user-agent"),
)
request.session["_session_token"] = user_session.session_token
except Exception:
logger.debug("[AUTH] Failed to create server-side session for social user", exc_info=True)
# Auto-create or update UserProfile
_ensure_user_profile(db, user_data, is_admin=False)
@@ -665,6 +722,27 @@ async def oauth_callback(request: Request, db: Session = Depends(get_db)):
request.session["user"] = user_data
# Create server-side session for tracking and revocation
try:
from app.utils.session_manager import create_session
_session_user_id = (
user_data.get("sub")
or user_data.get("preferred_username")
or user_data.get("email")
or user_data.get("id")
)
if _session_user_id:
user_session = create_session(
db,
user_id=_session_user_id,
ip_address=get_client_ip(request),
user_agent=request.headers.get("user-agent"),
)
request.session["_session_token"] = user_session.session_token
except Exception:
logger.debug("[AUTH] Failed to create server-side session for OAuth user", exc_info=True)
# Auto-create or update UserProfile so the user appears in admin user management
_ensure_user_profile(db, user_data, is_admin=is_admin)
@@ -918,6 +996,19 @@ async def auth(request: Request, db: Session = Depends(get_db)):
return RedirectResponse(url="/login?error=Invalid+username+or+password", status_code=302)
user_data = _build_session_user(local_user)
request.session["user"] = user_data
# Create server-side session for tracking and revocation
try:
from app.utils.session_manager import create_session
user_session = create_session(
db,
user_id=local_user.email,
ip_address=get_client_ip(request),
user_agent=request.headers.get("user-agent"),
)
request.session["_session_token"] = user_session.session_token
except Exception:
logger.debug("[AUTH] Failed to create server-side session", exc_info=True)
logger.info("[SECURITY] LOCAL_LOGIN_SUCCESS user=%s", local_user.email)
_record_login_event(db, request, local_user.email, success=True)
_ensure_user_profile(db, user_data, is_admin=bool(local_user.is_admin))
@@ -974,6 +1065,20 @@ async def auth(request: Request, db: Session = Depends(get_db)):
"is_admin": True,
}
request.session["user"] = admin_user_data
# Create server-side session for tracking and revocation
try:
from app.utils.session_manager import create_session
admin_user_id = settings.admin_username or "admin"
user_session = create_session(
db,
user_id=admin_user_id,
ip_address=get_client_ip(request),
user_agent=request.headers.get("user-agent"),
)
request.session["_session_token"] = user_session.session_token
except Exception:
logger.debug("[AUTH] Failed to create server-side session for admin", exc_info=True)
logger.info("[SECURITY] LOCAL_LOGIN_SUCCESS user=%s", username)
_record_login_event(db, request, username, success=True)
_ensure_user_profile(db, admin_user_data, is_admin=True)
@@ -1024,6 +1129,20 @@ async def logout(request: Request, db: Session = Depends(get_db)):
)
except Exception:
logger.debug("Failed to write logout audit event for user=%s", username, exc_info=True)
# Revoke server-side session
session_token = request.session.get("_session_token")
if session_token:
try:
from app.utils.session_manager import validate_session
user_session = validate_session(db, session_token)
if user_session:
user_session.is_revoked = True
user_session.revoked_at = datetime.now(timezone.utc)
db.commit()
except Exception:
logger.debug("[AUTH] Failed to revoke server-side session", exc_info=True)
request.session.pop("_session_token", None)
request.session.pop("user", None)
return RedirectResponse(url="/login?message=You+have+been+logged+out+successfully", status_code=302)
app/celery_worker.py
+2
View File
@@ -23,6 +23,7 @@ from app.tasks.batch_tasks import ( # noqa: F401
sync_search_index,
)
from app.tasks.check_credentials import check_credentials
from app.tasks.classify_document import classify_document_task # noqa: F401
from app.tasks.compute_embedding import backfill_missing_embeddings, compute_document_embedding # noqa: F401
from app.tasks.convert_to_pdf import convert_to_pdf # noqa: F401
from app.tasks.convert_to_pdfa import convert_to_pdfa # noqa: F401
@@ -53,6 +54,7 @@ from app.tasks.upload_to_onedrive import upload_to_onedrive # noqa: F401
from app.tasks.upload_to_paperless import upload_to_paperless # noqa: F401
from app.tasks.upload_to_s3 import upload_to_s3 # noqa: F401
from app.tasks.upload_to_sftp import upload_to_sftp # noqa: F401
from app.tasks.upload_to_sharepoint import upload_to_sharepoint # noqa: F401
from app.tasks.upload_to_user_integration import upload_to_user_integration # noqa: F401
from app.tasks.upload_to_webdav import upload_to_webdav # noqa: F401
from app.tasks.upload_with_rclone import send_to_all_rclone_destinations, upload_with_rclone # noqa: F401
app/config.py
+129
View File
@@ -13,6 +13,24 @@ class Settings(BaseSettings):
database_url: str
redis_url: str
# Database connection-pool tuning (ignored for SQLite, which uses NullPool).
db_pool_size: int = Field(
default=10,
description="Number of persistent connections kept in the pool per worker process.",
)
db_max_overflow: int = Field(
default=20,
description="Additional connections allowed beyond db_pool_size under burst load.",
)
db_pool_timeout: int = Field(
default=30,
description="Seconds to wait for a connection from the pool before raising a TimeoutError.",
)
db_pool_recycle: int = Field(
default=1800,
description="Recycle (close and reopen) connections after this many seconds to avoid stale connections.",
)
openai_api_key: str
openai_base_url: str = "https://api.openai.com/v1" # Default to OpenAI's endpoint
openai_model: str = "gpt-4o-mini" # Default model
@@ -102,6 +120,16 @@ class Settings(BaseSettings):
dropbox_app_secret: Optional[str] = None
dropbox_folder: Optional[str] = None
dropbox_refresh_token: Optional[str] = None
dropbox_allow_global_credentials_for_integrations: bool = Field(
default=False,
description=(
"When True, users may authorize their personal Dropbox integrations using the global "
"DROPBOX_APP_KEY / DROPBOX_APP_SECRET credentials configured by the admin, without "
"needing to create their own Dropbox app. The Dropbox OAuth flow is initiated "
"server-side so the app secret is never exposed to the browser. "
"Default: False (each user must supply their own app credentials)."
),
)
# Making Nextcloud optional
nextcloud_enabled: bool = Field(
@@ -165,6 +193,35 @@ class Settings(BaseSettings):
google_docai_processor_id: Optional[str] = None
google_docai_location: str = "us" # Processor location, e.g. "us" or "eu"
external_hostname: str = "localhost" # Default to localhost
public_base_url: Optional[str] = Field(
default=None,
description=(
"The full public base URL of the application, including scheme "
"(e.g., 'https://docuelevate.example.com'). "
"When set, this overrides the auto-detected URL for OAuth redirect URIs. "
"This is required when the application is behind a reverse proxy that does "
"not forward X-Forwarded-Proto headers correctly."
),
)
# ---------------------------------------------------------------------------
# Document Translation Settings
# ---------------------------------------------------------------------------
# Default target language for automatic document translation (ISO 639-1 code).
# After OCR / metadata extraction, if the detected document language differs
# from this value the system translates the extracted text into this language
# and stores it alongside the original. Other language translations are
# generated on the fly via the AI provider and are NOT persisted.
# Per-user overrides are stored in UserProfile.default_document_language.
default_document_language: str = Field(
default="en",
description=(
"ISO 639-1 language code for the default translation target "
"(e.g. 'en', 'de', 'fr'). Documents whose detected language "
"differs are automatically translated into this language after "
"processing. Default: 'en' (English)."
),
)
# ---------------------------------------------------------------------------
# Document Translation Settings
@@ -190,6 +247,26 @@ class Settings(BaseSettings):
admin_username: Optional[str] = None
admin_password: Optional[str] = None
session_secret: Optional[str] = None
session_lifetime_days: int = Field(
default=30,
description=(
"Session lifetime in days. Common values: 30, 60, 90. "
"Determines how long a user stays logged in before being required to re-authenticate. "
"Applies to both browser sessions and the session cookie max_age."
),
)
session_lifetime_custom_days: int | None = Field(
default=None,
description=(
"Override session_lifetime_days with a custom value. "
"When set, this takes precedence over session_lifetime_days. "
"Useful for admin-configured non-standard durations."
),
)
qr_login_challenge_ttl_seconds: int = Field(
default=120,
description="Time-to-live in seconds for QR login challenges (default: 2 minutes).",
)
admin_group_name: str = "admin"
# Multi-user settings
@@ -279,6 +356,16 @@ class Settings(BaseSettings):
social_auth_dropbox_enabled: bool = False
social_auth_dropbox_client_id: Optional[str] = None
social_auth_dropbox_client_secret: Optional[str] = None
social_auth_dropbox_use_global_credentials: bool = Field(
default=False,
description=(
"When True, Dropbox social login uses the global DROPBOX_APP_KEY / DROPBOX_APP_SECRET "
"credentials (the storage integration credentials) instead of requiring separate "
"SOCIAL_AUTH_DROPBOX_CLIENT_ID / SOCIAL_AUTH_DROPBOX_CLIENT_SECRET values. "
"Requires SOCIAL_AUTH_DROPBOX_ENABLED=True and the global Dropbox app credentials to be set. "
"Default: False."
),
)
# Local user signup
allow_local_signup: bool = Field(
@@ -589,6 +676,15 @@ class Settings(BaseSettings):
onedrive_refresh_token: Optional[str] = None # Required for personal accounts
onedrive_folder_path: Optional[str] = None
# SharePoint settings
sharepoint_client_id: Optional[str] = None
sharepoint_client_secret: Optional[str] = None
sharepoint_tenant_id: Optional[str] = "common"
sharepoint_refresh_token: Optional[str] = None
sharepoint_site_url: Optional[str] = None # e.g. https://tenant.sharepoint.com/sites/sitename
sharepoint_document_library: Optional[str] = "Documents" # Document library name
sharepoint_folder_path: Optional[str] = None # Subfolder inside the library
# AWS S3 settings
s3_enabled: bool = Field(
default=True,
@@ -639,6 +735,25 @@ class Settings(BaseSettings):
),
)
# System reset / factory reset settings
factory_reset_on_startup: bool = Field(
default=False,
description=(
"When enabled, DocuElevate wipes all user data (database rows and "
"work-files on disk) on every startup so the instance always comes "
"up in a clean, fresh state. Useful for demo or testing environments. "
"Default: False."
),
)
enable_factory_reset: bool = Field(
default=False,
description=(
"Show the 'System Reset' page in the admin UI. When enabled, "
"administrators can trigger a full data wipe or a wipe-and-reimport "
"directly from the web interface. Default: False."
),
)
# PDF/A archival conversion settings
enable_pdfa_conversion: bool = Field(
default=False,
@@ -1073,6 +1188,20 @@ class Settings(BaseSettings):
),
)
# Per-user upload rate limiting (health-aware, Redis-backed sliding window)
upload_rate_limit_per_user: int = Field(
default=20,
description=(
"Maximum number of file uploads allowed per user within the sliding window. "
"The effective limit may be reduced dynamically when the system is under heavy load "
"(high queue depth or CPU usage). Set to 0 to disable per-user upload rate limiting."
),
)
upload_rate_limit_window: int = Field(
default=60,
description="Sliding window size in seconds for per-user upload rate limiting (default: 60).",
)
# Rate Limiting Configuration (see SECURITY_AUDIT.md and docs/API.md)
# Protects against DoS attacks and API abuse
rate_limiting_enabled: bool = Field(
app/database.py
+31 -2
View File
@@ -10,6 +10,7 @@ from typing import Any
from sqlalchemy import create_engine, exc
from sqlalchemy.engine.url import make_url
from sqlalchemy.orm import Session, declarative_base, sessionmaker
from sqlalchemy.pool import NullPool, QueuePool
from app.config import settings
@@ -17,9 +18,37 @@ logger = logging.getLogger(__name__)
Base = declarative_base()
# Parse the DATABASE_URL
# ---------------------------------------------------------------------------
# Engine construction
# ---------------------------------------------------------------------------
DB_URL = settings.database_url
engine = create_engine(DB_URL, connect_args={"check_same_thread": False})
_parsed_url = make_url(DB_URL)
_connect_args: dict[str, Any] = {}
_engine_kwargs: dict[str, Any] = {
"pool_pre_ping": True, # detect stale / dropped connections before use
}
if _parsed_url.get_backend_name() == "sqlite":
# SQLite does not benefit from connection pooling and is prone to
# QueuePool exhaustion under concurrent access. NullPool opens a fresh
# connection for each request and closes it immediately afterwards,
# completely avoiding the "QueuePool limit reached" TimeoutError.
_connect_args["check_same_thread"] = False
_engine_kwargs["poolclass"] = NullPool
else:
# PostgreSQL / MySQL — use a bounded QueuePool with configurable limits.
_engine_kwargs["poolclass"] = QueuePool
_engine_kwargs.update(
{
"pool_size": settings.db_pool_size,
"max_overflow": settings.db_max_overflow,
"pool_timeout": settings.db_pool_timeout,
"pool_recycle": settings.db_pool_recycle,
}
)
engine = create_engine(DB_URL, connect_args=_connect_args, **_engine_kwargs)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
app/main.py
+18 -1
View File
@@ -170,6 +170,12 @@ async def lifespan(app: FastAPI):
# Startup: Initialize database
init_db() # Create tables if they don't exist
# Factory reset on startup — wipe all user data before anything else
if settings.factory_reset_on_startup:
from app.utils.system_reset import perform_startup_reset
perform_startup_reset()
# Load settings from database after DB initialization
from app.database import SessionLocal
from app.utils.config_loader import load_settings_from_db
@@ -318,8 +324,19 @@ app.add_middleware(CSRFMiddleware, config=settings)
# See SECURITY_AUDIT.md – Infrastructure Security section
app.add_middleware(AuditLogMiddleware, config=settings)
# 3) Session Middleware (for request.session to work)
app.add_middleware(SessionMiddleware, secret_key=SESSION_SECRET)
def _get_session_max_age() -> int:
"""Compute session max-age at startup time."""
try:
from app.utils.session_manager import get_session_max_age_seconds
return get_session_max_age_seconds()
except Exception:
return 30 * 86400 # 30 days default fallback
app.add_middleware(SessionMiddleware, secret_key=SESSION_SECRET, max_age=_get_session_max_age())
# 3a) CORS Middleware - handles cross-origin requests and preflight (OPTIONS) responses.
# Disabled by default: set CORS_ENABLED=True only when NOT using a reverse proxy
app/middleware/csrf.py
+7
View File
@@ -20,6 +20,9 @@ How it works:
Exempt paths (CSRF is not checked even for state-changing methods):
- ``/oauth-callback`` – OAuth 2.0 callback; protected by the ``state`` parameter.
- ``/api/qr-auth/claim`` – Called by the unauthenticated mobile app; the
cryptographically-random, single-use challenge token provides equivalent
protection.
"""
import logging
@@ -39,6 +42,10 @@ CSRF_PROTECTED_METHODS = {"POST", "PUT", "DELETE", "PATCH"}
# their own replay-protection mechanism).
CSRF_EXEMPT_PATHS = {
"/oauth-callback",
# The mobile app calls this endpoint without a browser session/CSRF token.
# The cryptographically-random, single-use challenge token already provides
# equivalent protection against cross-site request forgery.
"/api/qr-auth/claim",
}
app/middleware/upload_rate_limit.py
+290
View File
@@ -0,0 +1,290 @@
"""Per-user, health-aware upload rate limiter for DocuElevate.
This module provides a FastAPI dependency that enforces per-user upload rate
limits using a Redis-backed sliding window counter. The effective limit is
dynamically reduced when the system is under heavy load (high Celery queue
depth or elevated CPU load average), ensuring the server remains responsive
to all users even during bulk-upload scenarios.
Usage in an endpoint::
from app.middleware.upload_rate_limit import require_upload_rate_limit
@router.post("/ui-upload")
@require_login
async def ui_upload(
request: Request,
_rate_ok: None = Depends(require_upload_rate_limit),
...
):
...
See ``docs/ConfigurationGuide.md`` for the configuration options
(``UPLOAD_RATE_LIMIT_PER_USER``, ``UPLOAD_RATE_LIMIT_WINDOW``).
"""
from __future__ import annotations
import logging
import os
import time
from typing import Any
import redis
from fastapi import HTTPException, Request, status
from app.config import settings
from app.utils.user_scope import get_current_owner_id
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Redis key prefix
# ---------------------------------------------------------------------------
_KEY_PREFIX = "docuelevate:upload_rate"
# ---------------------------------------------------------------------------
# Health-check queue names (Celery defaults used by DocuElevate)
# ---------------------------------------------------------------------------
_CELERY_QUEUES = ("document_processor", "default", "celery")
# ---------------------------------------------------------------------------
# Singleton Redis client (lazy-initialised; fail-open when unavailable)
# ---------------------------------------------------------------------------
_redis_client: redis.Redis | None = None
def _get_redis() -> redis.Redis | None:
"""Return a shared Redis client, or *None* when Redis is unavailable."""
global _redis_client
if _redis_client is not None:
return _redis_client
try:
_redis_client = redis.Redis.from_url(
settings.redis_url,
decode_responses=True,
socket_connect_timeout=2,
socket_timeout=2,
)
# Quick connectivity check – raises on failure.
_redis_client.ping()
return _redis_client
except Exception: # noqa: BLE001
logger.debug("Redis unavailable for upload rate limiter – falling back to allow-all", exc_info=True)
_redis_client = None
return None
# ---------------------------------------------------------------------------
# Health metrics helpers
# ---------------------------------------------------------------------------
def _get_queue_depth(r: redis.Redis) -> int:
"""Return the total number of pending tasks across all Celery queues."""
total = 0
for queue_name in _CELERY_QUEUES:
try:
total += r.llen(queue_name)
except Exception: # noqa: BLE001, S110
logger.debug("Could not read queue length for %r", queue_name, exc_info=True)
return total
def _get_cpu_load_ratio() -> float:
"""Return the 1-minute load average divided by the number of CPU cores.
Returns ``0.0`` on platforms that do not support :func:`os.getloadavg`
(e.g. Windows) so that the limiter never penalises on those systems.
"""
try:
load_1m = os.getloadavg()[0]
cpu_count = os.cpu_count() or 1
return load_1m / cpu_count
except (OSError, AttributeError):
return 0.0
def compute_effective_limit(
base_limit: int,
queue_depth: int = 0,
cpu_load_ratio: float = 0.0,
) -> tuple[int, float, str]:
"""Compute the effective upload rate limit based on system health.
The function applies a *reduction factor* (``0.0 < factor ≤ 1.0``) to the
configured base limit. Both queue depth and CPU load contribute
independently; the lowest factor wins.
Args:
base_limit: The configured maximum uploads per window.
queue_depth: Total pending tasks in Celery queues.
cpu_load_ratio: 1-minute load average divided by CPU count.
Returns:
A 3-tuple of ``(effective_limit, factor, reason)`` where *reason*
is a human-readable tag for logging.
"""
factor = 1.0
reason = "normal"
# --- Queue-depth thresholds ---
if queue_depth > 200:
factor, reason = min(factor, 0.10), f"critical_queue({queue_depth})"
elif queue_depth > 100:
factor, reason = min(factor, 0.25), f"high_queue({queue_depth})"
elif queue_depth > 50:
factor, reason = min(factor, 0.50), f"moderate_queue({queue_depth})"
# --- CPU-load thresholds ---
if cpu_load_ratio > 3.0:
new_factor = 0.10
if new_factor < factor:
factor, reason = new_factor, f"critical_cpu({cpu_load_ratio:.1f})"
elif cpu_load_ratio > 2.0:
new_factor = 0.25
if new_factor < factor:
factor, reason = new_factor, f"high_cpu({cpu_load_ratio:.1f})"
elif cpu_load_ratio > 1.5:
new_factor = 0.50
if new_factor < factor:
factor, reason = new_factor, f"moderate_cpu({cpu_load_ratio:.1f})"
effective = max(1, int(base_limit * factor))
return effective, factor, reason
# ---------------------------------------------------------------------------
# Core sliding-window check (Redis sorted set)
# ---------------------------------------------------------------------------
def _check_and_record(
r: redis.Redis,
user_id: str,
window: int,
effective_limit: int,
) -> dict[str, Any] | None:
"""Atomically check the user's upload count and record the new upload.
Uses a Redis sorted set where each member is a unique timestamp-based ID
and the score is the Unix timestamp. Entries older than *window* seconds
are pruned on every call so the set never grows unbounded.
Returns:
``None`` if the request is allowed, or a ``dict`` with ``count``,
``limit``, and ``retry_after`` if the limit is exceeded.
"""
key = f"{_KEY_PREFIX}:{user_id}"
now = time.time()
window_start = now - window
pipe = r.pipeline(transaction=True)
# 1. Remove entries outside the window
pipe.zremrangebyscore(key, "-inf", window_start)
# 2. Count current entries
pipe.zcard(key)
# 3. Retrieve the oldest entry's score (to compute retry_after)
pipe.zrange(key, 0, 0, withscores=True)
results = pipe.execute()
current_count: int = results[1]
oldest_entries: list = results[2]
if current_count >= effective_limit:
# Compute how long until the oldest entry expires from the window.
if oldest_entries:
oldest_score = oldest_entries[0][1]
retry_after = max(1, int((oldest_score + window) - now))
else:
retry_after = max(1, window // 2)
return {
"count": current_count,
"limit": effective_limit,
"retry_after": retry_after,
}
# 4. Record this upload (unique member = timestamp with random suffix)
member = f"{now}:{os.urandom(4).hex()}"
pipe2 = r.pipeline(transaction=True)
pipe2.zadd(key, {member: now})
pipe2.expire(key, window + 60) # TTL slightly longer than window
pipe2.execute()
return None
# ---------------------------------------------------------------------------
# FastAPI dependency
# ---------------------------------------------------------------------------
async def require_upload_rate_limit(request: Request) -> None:
"""FastAPI dependency that enforces per-user upload rate limits.
The dependency is designed to **fail open**: if Redis is unavailable the
request is allowed through so that uploads are never blocked by a
monitoring outage.
Raises:
HTTPException: 429 Too Many Requests when the per-user upload limit
is exceeded. The ``Retry-After`` header indicates how many
seconds the client should wait before retrying.
"""
r = _get_redis()
if r is None:
# Redis unavailable – fail open.
return
# Identify the user (owner_id for multi-user, IP fallback).
user_id = get_current_owner_id(request)
if not user_id:
user_id = f"ip:{request.client.host}" if request.client else "ip:unknown"
base_limit: int = settings.upload_rate_limit_per_user
window: int = settings.upload_rate_limit_window
# Gather health metrics and compute effective limit.
try:
queue_depth = _get_queue_depth(r)
except Exception: # noqa: BLE001
queue_depth = 0
cpu_load_ratio = _get_cpu_load_ratio()
effective_limit, factor, health_reason = compute_effective_limit(base_limit, queue_depth, cpu_load_ratio)
# Sliding-window check.
try:
rejection = _check_and_record(r, user_id, window, effective_limit)
except Exception as exc: # noqa: BLE001
logger.warning("Upload rate-limit check failed (allowing request): %s", exc)
return
if rejection is not None:
retry_after = rejection["retry_after"]
logger.warning(
"Upload rate limit exceeded: user=%s count=%d/%d window=%ds health=%s retry_after=%ds",
user_id,
rejection["count"],
rejection["limit"],
window,
health_reason,
retry_after,
)
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=(
f"Upload rate limit exceeded ({rejection['count']}/{rejection['limit']} "
f"in {window}s). Retry after {retry_after}s."
),
headers={"Retry-After": str(retry_after)},
)
if factor < 1.0:
logger.info(
"Upload allowed with reduced limit: user=%s effective=%d/%d health=%s",
user_id,
effective_limit,
base_limit,
health_reason,
)
app/models.py
+130
View File
@@ -625,6 +625,7 @@ class IntegrationType:
EMAIL = "EMAIL"
PAPERLESS = "PAPERLESS"
RCLONE = "RCLONE"
SHAREPOINT = "SHAREPOINT"
ICLOUD = "ICLOUD"
ALL = {
@@ -642,6 +643,7 @@ class IntegrationType:
EMAIL,
PAPERLESS,
RCLONE,
SHAREPOINT,
ICLOUD,
}
@@ -806,6 +808,9 @@ class ApiToken(Base):
created_at = Column(DateTime(timezone=True), server_default=func.now())
revoked_at = Column(DateTime(timezone=True), nullable=True)
# Optional expiry: if set, the token is rejected after this timestamp.
expires_at = Column(DateTime(timezone=True), nullable=True)
class SharedLink(Base):
"""Shareable, time-limited or view-limited document link.
@@ -957,6 +962,51 @@ class ScheduledJob(Base):
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
class ClassificationRuleModel(Base):
"""Custom document classification rule.
Rules are evaluated during the ``classify`` pipeline step to assign a
category to a document. System-wide rules have ``owner_id IS NULL``;
user-specific rules belong to a single owner.
"""
__tablename__ = "classification_rules"
id = Column(Integer, primary_key=True, index=True)
# NULL = system-wide rule visible to all users.
owner_id = Column(String, nullable=True, index=True)
# Human-readable rule name (unique per owner).
name = Column(String(255), nullable=False)
# Target category (e.g. "invoice", "contract", "receipt").
category = Column(String(100), nullable=False, index=True)
# Rule type: "filename_pattern", "content_keyword", or "metadata_match".
rule_type = Column(String(50), nullable=False)
# The matching pattern:
# - filename_pattern: a regex
# - content_keyword: pipe-separated keywords
# - metadata_match: "field=value"
pattern = Column(String(1000), nullable=False)
# Higher priority rules are evaluated first (default 0).
priority = Column(Integer, nullable=False, default=0)
# Whether pattern matching is case-sensitive.
case_sensitive = Column(Boolean, nullable=False, default=False)
# Disabled rules are skipped during classification.
enabled = Column(Boolean, nullable=False, default=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
__table_args__ = (UniqueConstraint("owner_id", "name", name="uq_classification_rules_owner_name"),)
class MobileDevice(Base):
"""Registered mobile device for push notifications.
@@ -991,6 +1041,86 @@ class MobileDevice(Base):
__table_args__ = (UniqueConstraint("owner_id", "push_token", name="uq_mobile_device_owner_token"),)
class UserSession(Base):
"""Server-side session tracking for invalidation and device management.
Each row represents an active browser or app session. The ``session_token``
is stored in the user's cookie and validated on every authenticated request.
Revoking a row (``is_revoked=True``) immediately terminates that session
on the next request.
"""
__tablename__ = "user_sessions"
id = Column(Integer, primary_key=True, index=True)
# Cryptographically random token stored in the session cookie.
session_token = Column(String(128), unique=True, nullable=False, index=True)
# Stable owner identifier — matches FileRecord.owner_id.
user_id = Column(String, nullable=False, index=True)
# Client metadata for display in the session management UI.
ip_address = Column(String(45), nullable=True)
user_agent = Column(String(512), nullable=True)
device_info = Column(String(255), nullable=True)
is_revoked = Column(Boolean, nullable=False, default=False)
created_at = Column(DateTime(timezone=True), server_default=func.now())
last_active_at = Column(DateTime(timezone=True), server_default=func.now())
expires_at = Column(DateTime(timezone=True), nullable=False)
revoked_at = Column(DateTime(timezone=True), nullable=True)
class QRLoginChallenge(Base):
"""Time-limited QR code login challenge for mobile app authentication.
A logged-in web user generates a challenge that produces a QR code. The
mobile app scans the QR code and calls the claim endpoint with the
``challenge_token``. The server verifies the challenge is still valid,
unclaimed, and unexpired, then issues an API token for the mobile app.
Security properties:
* Time-bound (default 2 minutes).
* Single-use (``is_claimed`` prevents replay).
* Cryptographically random 64-byte token.
* Bound to the creating user — only that user's mobile device receives a
token.
"""
__tablename__ = "qr_login_challenges"
id = Column(Integer, primary_key=True, index=True)
# Cryptographically random token encoded in the QR code.
challenge_token = Column(String(128), unique=True, nullable=False, index=True)
# The user who created this challenge (from the web session).
user_id = Column(String, nullable=False, index=True)
# Whether the challenge has been successfully claimed by a mobile app.
is_claimed = Column(Boolean, nullable=False, default=False)
# Whether the challenge has been explicitly cancelled or expired.
is_cancelled = Column(Boolean, nullable=False, default=False)
# IP address of the web client that created the challenge.
created_by_ip = Column(String(45), nullable=True)
# IP address of the mobile client that claimed the challenge.
claimed_by_ip = Column(String(45), nullable=True)
# Device name provided by the mobile app when claiming.
device_name = Column(String(255), nullable=True)
# The API token ID that was issued to the mobile app (for audit trail).
issued_token_id = Column(Integer, nullable=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
expires_at = Column(DateTime(timezone=True), nullable=False)
claimed_at = Column(DateTime(timezone=True), nullable=True)
class ComplianceTemplate(Base):
"""Pre-built compliance configuration templates (GDPR, HIPAA, SOC2).
app/tasks/classify_document.py
+174
View File
@@ -0,0 +1,174 @@
"""Celery task for rule-based document classification.
This task is executed as a pipeline step (``step_type="classify"``). It
applies built-in and user-defined classification rules against the document's
filename, OCR text, and existing AI metadata to assign a ``document_type``
category.
The result is stored in the ``ai_metadata`` JSON blob on the
:class:`~app.models.FileRecord` (field ``classification``).
"""
from __future__ import annotations
import json
import logging
from typing import Any
from app.celery_app import celery
from app.database import SessionLocal
from app.models import ClassificationRuleModel, FileRecord
from app.tasks.retry_config import BaseTaskWithRetry
from app.utils import log_task_progress
from app.utils.classification_rules import (
ClassificationResult,
classify_document,
db_rule_to_engine_rule,
)
logger = logging.getLogger(__name__)
STEP_NAME = "classify_document"
def _load_custom_rules(owner_id: str | None) -> list[Any]:
"""Load enabled custom classification rules from the database.
Returns engine-level :class:`ClassificationRule` dataclass instances.
Rules are loaded in priority-descending order. System rules
(``owner_id IS NULL``) and the user's own rules are both included.
"""
with SessionLocal() as db:
query = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.enabled.is_(True))
if owner_id:
query = query.filter(
(ClassificationRuleModel.owner_id.is_(None)) | (ClassificationRuleModel.owner_id == owner_id)
)
else:
query = query.filter(ClassificationRuleModel.owner_id.is_(None))
rules = query.order_by(ClassificationRuleModel.priority.desc()).all()
return [db_rule_to_engine_rule(r) for r in rules]
@celery.task(base=BaseTaskWithRetry, bind=True)
def classify_document_task(
self: Any,
file_id: int,
owner_id: str | None = None,
) -> dict[str, Any]:
"""Classify a document using rule-based matching.
This task:
1. Loads the :class:`FileRecord` from the database.
2. Gathers filename, OCR text, and existing AI metadata.
3. Loads built-in + user-defined classification rules.
4. Runs the classification engine.
5. Persists the result into ``ai_metadata.classification``.
Args:
file_id: Primary key of the :class:`FileRecord` to classify.
owner_id: Owner identifier for loading user-specific rules.
Returns:
Dict with ``category``, ``confidence``, and ``matched_rules``.
"""
task_id = self.request.id
log_task_progress(
task_id,
STEP_NAME,
"in_progress",
f"Starting classification for file {file_id}",
file_id=file_id,
)
try:
with SessionLocal() as db:
file_record: FileRecord | None = db.query(FileRecord).filter(FileRecord.id == file_id).first()
if file_record is None:
log_task_progress(
task_id,
STEP_NAME,
"failure",
f"FileRecord {file_id} not found",
file_id=file_id,
)
return {"status": "error", "detail": "File not found"}
# Gather inputs
filename = file_record.original_filename or ""
text = file_record.ocr_text or ""
existing_metadata: dict[str, Any] = {}
if file_record.ai_metadata:
try:
existing_metadata = json.loads(file_record.ai_metadata)
except (json.JSONDecodeError, TypeError):
logger.warning("Failed to parse ai_metadata for file %s, starting fresh", file_id)
existing_metadata = {}
# Load custom rules
effective_owner = owner_id or file_record.owner_id
custom_rules = _load_custom_rules(effective_owner)
# Run classification engine
result: ClassificationResult = classify_document(
filename=filename,
text=text,
metadata=existing_metadata,
custom_rules=custom_rules,
)
# Persist result into ai_metadata
classification_data = {
"category": result.category,
"confidence": result.confidence,
"matched_rules": [
{
"rule_name": m.rule_name,
"rule_type": m.rule_type,
"category": m.category,
"confidence": m.confidence,
}
for m in result.matched_rules
],
}
existing_metadata["classification"] = classification_data
# If no document_type was set yet, populate it from the classification
if not existing_metadata.get("document_type"):
from app.utils.classification_rules import BUILTIN_CATEGORIES
existing_metadata["document_type"] = BUILTIN_CATEGORIES.get(
result.category, result.category.replace("_", " ").title()
)
file_record.ai_metadata = json.dumps(existing_metadata, ensure_ascii=False)
db.commit()
log_task_progress(
task_id,
STEP_NAME,
"success",
f"Classified as '{result.category}' with confidence {result.confidence}",
file_id=file_id,
detail=f"Matched {len(result.matched_rules)} rule(s)",
)
return {
"status": "success",
"category": result.category,
"confidence": result.confidence,
"matched_rules": len(result.matched_rules),
}
except Exception as e:
logger.exception("Classification failed for file %s: %s", file_id, e)
log_task_progress(
task_id,
STEP_NAME,
"failure",
f"Classification failed: {e}",
file_id=file_id,
)
raise
app/tasks/convert_to_pdf.py
+1 -1
View File
@@ -205,7 +205,7 @@ def convert_to_pdf(
".pdf", # PDF (already in PDF format but can be processed)
}
IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tiff", ".tif", ".webp", ".svg"}
IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tiff", ".tif", ".webp", ".svg", ".heic", ".heif"}
HTML_EXTENSIONS = {".html", ".htm"}
app/tasks/send_to_all.py
+19
View File
@@ -18,6 +18,7 @@ from app.tasks.upload_to_onedrive import upload_to_onedrive
from app.tasks.upload_to_paperless import upload_to_paperless
from app.tasks.upload_to_s3 import upload_to_s3
from app.tasks.upload_to_sftp import upload_to_sftp
from app.tasks.upload_to_sharepoint import upload_to_sharepoint
from app.tasks.upload_to_webdav import upload_to_webdav
from app.utils.config_validator import get_provider_status
from app.utils.logging import log_task_progress
@@ -121,6 +122,18 @@ def _should_upload_to_icloud():
return bool(getattr(settings, "icloud_enabled", True) and settings.icloud_username and settings.icloud_password)
def _should_upload_to_sharepoint():
return bool(
settings.sharepoint_client_id
and settings.sharepoint_client_secret
and settings.sharepoint_site_url
and (
settings.sharepoint_refresh_token
or (settings.sharepoint_tenant_id and settings.sharepoint_tenant_id != "common")
)
)
def get_configured_services_from_validator():
"""
Use the config validator to determine which services are configured and enabled.
@@ -140,6 +153,7 @@ def get_configured_services_from_validator():
"Email": "email",
"OneDrive": "onedrive",
"S3 Storage": "s3",
"SharePoint": "sharepoint",
"iCloud Drive": "icloud",
}
@@ -250,6 +264,11 @@ def send_to_all_destinations(self, file_path: str, use_validator=True, file_id:
"should_upload": _should_upload_to_s3,
"upload_func": upload_to_s3,
},
{
"name": "sharepoint",
"should_upload": _should_upload_to_sharepoint,
"upload_func": upload_to_sharepoint,
},
{
"name": "icloud",
"should_upload": _should_upload_to_icloud,
app/tasks/upload_to_sharepoint.py
+338
View File
@@ -0,0 +1,338 @@
#!/usr/bin/env python3
"""Upload documents to Microsoft SharePoint via the Microsoft Graph API.
This module authenticates using MSAL (same OAuth2 flow as OneDrive) and
uploads files to a configurable SharePoint Online document library using
the chunked upload session approach for reliability with large files.
Key differences from the OneDrive provider:
- Uses ``/sites/{siteId}/drives/{driveId}`` instead of ``/me/drive``
- Requires a SharePoint site URL to resolve the site and drive IDs
- Targets a named document library (default: ``Documents``)
"""
import logging
import os
import time
import urllib.parse
import msal
import requests
from app.celery_app import celery
from app.config import settings
from app.tasks.retry_config import UploadTaskWithRetry
from app.utils import log_task_progress
logger = logging.getLogger(__name__)
def get_sharepoint_token() -> str:
"""Acquire a Microsoft Graph API access token for SharePoint.
Uses MSAL ``ConfidentialClientApplication`` with the refresh-token flow
(delegated permissions) or the client-credentials flow (application
permissions) depending on configuration.
Returns:
A valid access token string.
Raises:
ValueError: When required settings are missing or token acquisition fails.
"""
if not settings.sharepoint_client_id or not settings.sharepoint_client_secret:
raise ValueError("SharePoint client ID and client secret must be configured")
tenant = settings.sharepoint_tenant_id or "common"
logger.info("Using SharePoint tenant: %s", tenant)
scopes = ["https://graph.microsoft.com/.default"]
if settings.sharepoint_refresh_token:
app = msal.ConfidentialClientApplication(
client_id=settings.sharepoint_client_id,
client_credential=settings.sharepoint_client_secret,
authority=f"https://login.microsoftonline.com/{tenant}",
)
logger.info("Attempting to acquire SharePoint token using refresh token")
token_response = app.acquire_token_by_refresh_token(
refresh_token=settings.sharepoint_refresh_token, scopes=scopes
)
if "access_token" not in token_response:
error = token_response.get("error", "")
error_desc = token_response.get("error_description", "Unknown error")
logger.error("Failed to get SharePoint access token: %s - %s", error, error_desc)
raise ValueError(f"Failed to get SharePoint access token: {error} - {error_desc}")
if "refresh_token" in token_response:
settings.sharepoint_refresh_token = token_response["refresh_token"]
logger.info("Updated SharePoint refresh token in memory")
return token_response["access_token"]
elif settings.sharepoint_tenant_id and settings.sharepoint_tenant_id != "common":
authority = f"https://login.microsoftonline.com/{settings.sharepoint_tenant_id}"
app = msal.ConfidentialClientApplication(
client_id=settings.sharepoint_client_id,
client_credential=settings.sharepoint_client_secret,
authority=authority,
)
token_response = app.acquire_token_for_client(scopes=scopes)
if "access_token" not in token_response:
error = token_response.get("error", "")
error_desc = token_response.get("error_description", "Unknown error")
raise ValueError(f"Failed to get SharePoint access token: {error} - {error_desc}")
return token_response["access_token"]
else:
raise ValueError("For SharePoint, either a refresh token or a non-'common' tenant ID is required")
def resolve_sharepoint_drive(access_token: str, site_url: str, library_name: str) -> tuple[str, str]:
"""Resolve the Graph API site ID and drive ID for a SharePoint site.
Args:
access_token: Valid Microsoft Graph API token.
site_url: Full SharePoint site URL, e.g.
``https://tenant.sharepoint.com/sites/sitename``.
library_name: Display name of the document library (e.g. ``Documents``).
Returns:
A ``(site_id, drive_id)`` tuple.
Raises:
ValueError: When the site URL cannot be parsed.
RuntimeError: When the Graph API call fails.
"""
parsed = urllib.parse.urlparse(site_url)
hostname = parsed.hostname
site_path = parsed.path.rstrip("/")
if not hostname or not site_path:
raise ValueError(
f"Invalid SharePoint site URL '{site_url}'. Expected format: https://tenant.sharepoint.com/sites/sitename"
)
headers = {"Authorization": f"Bearer {access_token}"}
# Resolve site ID
site_api_url = f"https://graph.microsoft.com/v1.0/sites/{hostname}:{site_path}"
logger.info("Resolving SharePoint site: %s", site_api_url)
resp = requests.get(site_api_url, headers=headers, timeout=settings.http_request_timeout)
if resp.status_code != 200:
raise RuntimeError(f"Failed to resolve SharePoint site: {resp.status_code} - {resp.text}")
site_id = resp.json()["id"]
logger.info("Resolved SharePoint site ID: %s", site_id)
# Resolve drive ID from the document library name
drives_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives"
resp = requests.get(drives_url, headers=headers, timeout=settings.http_request_timeout)
if resp.status_code != 200:
raise RuntimeError(f"Failed to list SharePoint drives: {resp.status_code} - {resp.text}")
drives = resp.json().get("value", [])
drive_id = None
for drive in drives:
if drive.get("name", "").lower() == library_name.lower():
drive_id = drive["id"]
break
if not drive_id:
available = [d.get("name") for d in drives]
raise RuntimeError(f"Document library '{library_name}' not found on site. Available libraries: {available}")
logger.info("Resolved SharePoint drive ID: %s (library: %s)", drive_id, library_name)
return site_id, drive_id
def create_sharepoint_upload_session(
filename: str, folder_path: str | None, drive_id: str, site_id: str, access_token: str
) -> str:
"""Create a resumable upload session on a SharePoint document library.
Args:
filename: Name of the file to upload.
folder_path: Optional subfolder path inside the library.
drive_id: Graph API drive ID of the document library.
site_id: Graph API site ID.
access_token: Valid access token.
Returns:
The upload session URL for chunked PUT requests.
Raises:
RuntimeError: When session creation fails.
"""
base_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives/{drive_id}"
if folder_path:
folder_path = folder_path.strip("/")
path_components = folder_path.split("/")
encoded_path = "/".join(urllib.parse.quote(component) for component in path_components)
encoded_filename = urllib.parse.quote(filename)
item_path = f"/root:/{encoded_path}/{encoded_filename}:/createUploadSession"
else:
encoded_filename = urllib.parse.quote(filename)
item_path = f"/root:/{encoded_filename}:/createUploadSession"
url = f"{base_url}{item_path}"
request_body = {"item": {"@microsoft.graph.conflictBehavior": "replace"}}
headers = {"Authorization": f"Bearer {access_token}", "Content-Type": "application/json"}
logger.info("Creating SharePoint upload session for %s at path %s", filename, folder_path)
response = requests.post(url, headers=headers, json=request_body, timeout=settings.http_request_timeout)
if response.status_code == 200:
upload_url = response.json().get("uploadUrl")
logger.info("SharePoint upload session created for %s", filename)
return upload_url
else:
raise RuntimeError(f"Failed to create SharePoint upload session: {response.status_code} - {response.text}")
def upload_large_file_sharepoint(file_path: str, upload_url: str) -> dict:
"""Upload a file to SharePoint using a chunked upload session.
Args:
file_path: Local path to the file.
upload_url: The upload session URL from ``create_sharepoint_upload_session``.
Returns:
The Graph API response dict containing file metadata.
Raises:
RuntimeError: When a chunk upload fails after retries.
"""
file_size = os.path.getsize(file_path)
chunk_size = 10 * 1024 * 1024 # 10 MB
response = None
with open(file_path, "rb") as f:
chunk_number = 0
while True:
chunk = f.read(chunk_size)
if not chunk:
break
chunk_start = chunk_number * chunk_size
chunk_end = chunk_start + len(chunk) - 1
content_range = f"bytes {chunk_start}-{chunk_end}/{file_size}"
headers = {"Content-Length": str(len(chunk)), "Content-Range": content_range}
max_retries = 3
retry_delay = 2
for attempt in range(max_retries):
try:
response = requests.put(
upload_url, headers=headers, data=chunk, timeout=settings.http_request_timeout
)
if response.status_code in (201, 202):
break
else:
logger.warning(
"SharePoint chunk upload failed (attempt %d): %d", attempt + 1, response.status_code
)
if attempt < max_retries - 1:
time.sleep(retry_delay * (attempt + 1))
except Exception as e:
logger.warning("SharePoint chunk upload error (attempt %d): %s", attempt + 1, str(e))
if attempt < max_retries - 1:
time.sleep(retry_delay * (attempt + 1))
if response is None or response.status_code not in (201, 202):
status = response.status_code if response else "no response"
text = response.text if response else ""
raise RuntimeError(f"Failed to upload chunk after {max_retries} attempts: {status} - {text}")
chunk_number += 1
return response.json() if response else {}
@celery.task(base=UploadTaskWithRetry, bind=True)
def upload_to_sharepoint(self, file_path: str, file_id: int = None, folder_override: str = None):
"""Upload a file to SharePoint Online.
Args:
file_path: Path to the file to upload.
file_id: Optional file ID to associate with logs.
folder_override: Optional folder path override.
Returns:
A dict with upload status and file details.
Raises:
FileNotFoundError: When the file does not exist.
ValueError: When SharePoint is not configured.
RuntimeError: When the upload fails.
"""
task_id = self.request.id
logger.info("[%s] Starting SharePoint upload: %s", task_id, file_path)
log_task_progress(
task_id,
"upload_to_sharepoint",
"in_progress",
f"Uploading to SharePoint: {os.path.basename(file_path)}",
file_id=file_id,
)
if not os.path.exists(file_path):
error_msg = f"File not found: {file_path}"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
raise FileNotFoundError(error_msg)
filename = os.path.basename(file_path)
if not settings.sharepoint_client_id:
error_msg = "SharePoint client ID is not configured"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
raise ValueError(error_msg)
if not settings.sharepoint_site_url:
error_msg = "SharePoint site URL is not configured"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
raise ValueError(error_msg)
try:
access_token = get_sharepoint_token()
library_name = settings.sharepoint_document_library or "Documents"
site_id, drive_id = resolve_sharepoint_drive(access_token, settings.sharepoint_site_url, library_name)
folder_path = folder_override if folder_override is not None else settings.sharepoint_folder_path
upload_url = create_sharepoint_upload_session(filename, folder_path, drive_id, site_id, access_token)
result = upload_large_file_sharepoint(file_path, upload_url)
web_url = result.get("webUrl", "Not available")
logger.info("[%s] Successfully uploaded %s to SharePoint", task_id, filename)
logger.info("[%s] File accessible at: %s", task_id, web_url)
log_task_progress(
task_id, "upload_to_sharepoint", "success", f"Uploaded to SharePoint: {filename}", file_id=file_id
)
return {
"status": "Completed",
"file_path": file_path,
"sharepoint_path": f"{folder_path or ''}/{filename}",
"web_url": web_url,
}
except Exception as e:
error_msg = f"Failed to upload {filename} to SharePoint: {str(e)}"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
raise RuntimeError(error_msg) from e
app/tasks/upload_to_user_integration.py
+108
View File
@@ -571,6 +571,113 @@ def _upload_rclone(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], t
return {"status": "Completed", "rclone_dest": dest}
def _upload_sharepoint(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
"""Upload *file_path* to SharePoint using per-user MSAL credentials."""
import urllib.parse
import msal
import requests as _requests
client_id = creds.get("client_id") or ""
client_secret = creds.get("client_secret") or ""
refresh_token = creds.get("refresh_token") or ""
tenant = cfg.get("tenant_id") or "common"
site_url = cfg.get("site_url") or ""
library_name = cfg.get("document_library") or "Documents"
folder_path = cfg.get("folder_path") or ""
if not (client_id and client_secret):
raise ValueError("SharePoint integration is missing client_id or client_secret in credentials")
if not site_url:
raise ValueError("SharePoint integration is missing site_url in config")
scopes = ["https://graph.microsoft.com/.default"]
msal_app = msal.ConfidentialClientApplication(
client_id=client_id,
client_credential=client_secret,
authority=f"https://login.microsoftonline.com/{tenant}",
)
if refresh_token:
token_resp = msal_app.acquire_token_by_refresh_token(refresh_token=refresh_token, scopes=scopes)
else:
token_resp = msal_app.acquire_token_for_client(scopes=scopes)
if "access_token" not in token_resp:
raise ValueError(f"SharePoint token acquisition failed: {token_resp.get('error_description', 'unknown')}")
access_token = token_resp["access_token"]
headers = {"Authorization": f"Bearer {access_token}"}
# Resolve site ID
parsed = urllib.parse.urlparse(site_url)
hostname = parsed.hostname
site_path = parsed.path.rstrip("/")
if not hostname or not site_path:
raise ValueError(f"Invalid SharePoint site URL: {site_url}")
resp = _requests.get(f"https://graph.microsoft.com/v1.0/sites/{hostname}:{site_path}", headers=headers, timeout=30)
resp.raise_for_status()
site_id = resp.json()["id"]
# Resolve drive ID
resp = _requests.get(f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives", headers=headers, timeout=30)
resp.raise_for_status()
drive_id = None
for drive in resp.json().get("value", []):
if drive.get("name", "").lower() == library_name.lower():
drive_id = drive["id"]
break
if not drive_id:
raise RuntimeError(f"Document library '{library_name}' not found on SharePoint site")
filename = os.path.basename(file_path)
# Build upload-session URL
base_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives/{drive_id}"
if folder_path:
folder_path = folder_path.strip("/")
encoded_path = "/".join(urllib.parse.quote(p) for p in folder_path.split("/"))
encoded_file = urllib.parse.quote(filename)
item_path = f"/root:/{encoded_path}/{encoded_file}:/createUploadSession"
else:
encoded_file = urllib.parse.quote(filename)
item_path = f"/root:/{encoded_file}:/createUploadSession"
session_url = f"{base_url}{item_path}"
session_headers = {"Authorization": f"Bearer {access_token}", "Content-Type": "application/json"}
resp = _requests.post(
session_url,
headers=session_headers,
json={"item": {"@microsoft.graph.conflictBehavior": "replace"}},
timeout=30,
)
resp.raise_for_status()
upload_url = resp.json()["uploadUrl"]
file_size = os.path.getsize(file_path)
chunk_size = 10 * 1024 * 1024
with open(file_path, "rb") as fh:
chunk_num = 0
while True:
chunk = fh.read(chunk_size)
if not chunk:
break
start = chunk_num * chunk_size
end = start + len(chunk) - 1
upload_headers = {
"Content-Length": str(len(chunk)),
"Content-Range": f"bytes {start}-{end}/{file_size}",
}
upload_resp = _requests.put(upload_url, headers=upload_headers, data=chunk, timeout=120)
if upload_resp.status_code not in (201, 202):
raise RuntimeError(f"SharePoint chunk upload failed: {upload_resp.status_code}")
chunk_num += 1
logger.info("[%s] SharePoint upload complete: %s/%s", task_id, folder_path, filename)
return {"status": "Completed", "sharepoint_folder": folder_path, "filename": filename}
def _upload_icloud(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
"""Upload *file_path* to iCloud Drive using per-user credentials.
@@ -615,6 +722,7 @@ _UPLOAD_HANDLERS = {
IntegrationType.PAPERLESS: _upload_paperless,
IntegrationType.EMAIL: _upload_email,
IntegrationType.RCLONE: _upload_rclone,
IntegrationType.SHAREPOINT: _upload_sharepoint,
IntegrationType.ICLOUD: _upload_icloud,
}
app/utils/allowed_types.py
+9 -1
View File
@@ -68,6 +68,8 @@ IMAGE_MIME_TYPES: set[str] = {
"image/tiff",
"image/webp",
"image/svg+xml",
"image/heic",
"image/heif",
}
# ---------------------------------------------------------------------------
@@ -124,6 +126,8 @@ ALLOWED_EXTENSIONS: set[str] = {
".tif",
".webp",
".svg",
".heic",
".heif",
# Web
".html",
".htm",
@@ -234,7 +238,7 @@ FILE_TYPE_CATEGORIES: dict[str, dict] = {
},
"images": {
"label": "Images",
"description": "Image files (.jpg, .png, .gif, .bmp, .tiff, .webp, .svg)",
"description": "Image files (.jpg, .png, .gif, .bmp, .tiff, .webp, .svg, .heic, .heif)",
"mime_types": frozenset(
{
"image/jpeg",
@@ -245,6 +249,8 @@ FILE_TYPE_CATEGORIES: dict[str, dict] = {
"image/tiff",
"image/webp",
"image/svg+xml",
"image/heic",
"image/heif",
}
),
"extensions": frozenset(
@@ -258,6 +264,8 @@ FILE_TYPE_CATEGORIES: dict[str, dict] = {
".tif",
".webp",
".svg",
".heic",
".heif",
}
),
},
app/utils/classification_rules.py
+378
View File
@@ -0,0 +1,378 @@
"""
Rule-based document classification engine.
Provides pre-built categories and a rule matcher that classifies documents
using filename patterns, content keywords, and metadata fields. Custom
rules stored in the database are evaluated alongside the built-in defaults.
Usage::
from app.utils.classification_rules import classify_document
result = classify_document(
filename="2024-03-01_Invoice_Acme.pdf",
text="Invoice total: $1,234.56",
metadata={"absender": "Acme Corp"},
custom_rules=custom_rules_from_db,
)
# result -> ClassificationResult(category="invoice", confidence=85, matched_rules=[...])
"""
from __future__ import annotations
import logging
import re
from dataclasses import dataclass, field
from typing import Any
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Pre-built categories
# ---------------------------------------------------------------------------
#: Canonical category names recognized by the system. Users may also define
#: their own categories via custom rules.
BUILTIN_CATEGORIES: dict[str, str] = {
"invoice": "Invoice",
"contract": "Contract",
"receipt": "Receipt",
"letter": "Letter",
"report": "Report",
"bank_statement": "Bank Statement",
"tax_document": "Tax Document",
"insurance": "Insurance Document",
"payslip": "Payslip",
"unknown": "Unknown",
}
# ---------------------------------------------------------------------------
# Rule type constants
# ---------------------------------------------------------------------------
RULE_TYPE_FILENAME = "filename_pattern"
RULE_TYPE_CONTENT = "content_keyword"
RULE_TYPE_METADATA = "metadata_match"
# ---------------------------------------------------------------------------
# Data classes
# ---------------------------------------------------------------------------
@dataclass
class ClassificationRule:
"""A single classification rule."""
name: str
category: str
rule_type: str # filename_pattern | content_keyword | metadata_match
pattern: str # regex for filename, keyword(s) for content, "field=value" for metadata
priority: int = 0 # higher = evaluated first
case_sensitive: bool = False
def __post_init__(self) -> None:
if self.rule_type not in (RULE_TYPE_FILENAME, RULE_TYPE_CONTENT, RULE_TYPE_METADATA):
raise ValueError(f"Invalid rule_type: {self.rule_type!r}")
@dataclass
class MatchedRule:
"""Records which rule matched and why."""
rule_name: str
rule_type: str
category: str
confidence: int
@dataclass
class ClassificationResult:
"""The outcome of running the classification engine on a document."""
category: str
confidence: int # 0 – 100
matched_rules: list[MatchedRule] = field(default_factory=list)
# ---------------------------------------------------------------------------
# Built-in rules
# ---------------------------------------------------------------------------
BUILTIN_RULES: list[ClassificationRule] = [
# ── Invoice ───────────────────────────────────────────────────────────
ClassificationRule("builtin_invoice_filename", "invoice", RULE_TYPE_FILENAME, r"(?i)invoice|rechnung|facture"),
ClassificationRule(
"builtin_invoice_content",
"invoice",
RULE_TYPE_CONTENT,
"invoice number|invoice total|amount due|rechnung|rechnungsnummer|total amount|bill to",
),
ClassificationRule("builtin_invoice_metadata", "invoice", RULE_TYPE_METADATA, "document_type=Invoice"),
ClassificationRule(
"builtin_invoice_kommunikationsart", "invoice", RULE_TYPE_METADATA, "kommunikationsart=Rechnung"
),
# ── Contract ──────────────────────────────────────────────────────────
ClassificationRule("builtin_contract_filename", "contract", RULE_TYPE_FILENAME, r"(?i)contract|vertrag|agreement"),
ClassificationRule(
"builtin_contract_content",
"contract",
RULE_TYPE_CONTENT,
"hereby agrees|terms and conditions|vertrag|agreement between|party agrees|effective date",
),
ClassificationRule("builtin_contract_metadata", "contract", RULE_TYPE_METADATA, "document_type=Contract"),
ClassificationRule(
"builtin_contract_kommunikationsart", "contract", RULE_TYPE_METADATA, "kommunikationsart=Vertrag"
),
# ── Receipt ───────────────────────────────────────────────────────────
ClassificationRule("builtin_receipt_filename", "receipt", RULE_TYPE_FILENAME, r"(?i)receipt|quittung|beleg"),
ClassificationRule(
"builtin_receipt_content",
"receipt",
RULE_TYPE_CONTENT,
"receipt|quittung|payment received|thank you for your purchase|transaction id",
),
ClassificationRule("builtin_receipt_metadata", "receipt", RULE_TYPE_METADATA, "document_type=Receipt"),
ClassificationRule(
"builtin_receipt_kommunikationsart", "receipt", RULE_TYPE_METADATA, "kommunikationsart=Quittung"
),
# ── Letter ────────────────────────────────────────────────────────────
ClassificationRule("builtin_letter_filename", "letter", RULE_TYPE_FILENAME, r"(?i)letter|brief|schreiben"),
ClassificationRule(
"builtin_letter_content",
"letter",
RULE_TYPE_CONTENT,
"dear sir|dear madam|sehr geehrte|to whom it may concern|sincerely|mit freundlichen",
),
# ── Report ────────────────────────────────────────────────────────────
ClassificationRule("builtin_report_filename", "report", RULE_TYPE_FILENAME, r"(?i)report|bericht"),
ClassificationRule(
"builtin_report_content",
"report",
RULE_TYPE_CONTENT,
"executive summary|table of contents|annual report|quarterly report|findings",
),
# ── Bank statement ────────────────────────────────────────────────────
ClassificationRule(
"builtin_bank_filename",
"bank_statement",
RULE_TYPE_FILENAME,
r"(?i)bank.?statement|kontoauszug",
),
ClassificationRule(
"builtin_bank_content",
"bank_statement",
RULE_TYPE_CONTENT,
"account statement|kontoauszug|opening balance|closing balance|account number",
),
ClassificationRule(
"builtin_bank_kommunikationsart", "bank_statement", RULE_TYPE_METADATA, "kommunikationsart=Kontoauszug"
),
# ── Tax document ──────────────────────────────────────────────────────
ClassificationRule("builtin_tax_filename", "tax_document", RULE_TYPE_FILENAME, r"(?i)tax|steuer|steuerbescheid"),
ClassificationRule(
"builtin_tax_content",
"tax_document",
RULE_TYPE_CONTENT,
"tax return|steuerbescheid|taxable income|finanzamt|tax assessment",
),
# ── Insurance ─────────────────────────────────────────────────────────
ClassificationRule(
"builtin_insurance_filename", "insurance", RULE_TYPE_FILENAME, r"(?i)insurance|versicherung|police"
),
ClassificationRule(
"builtin_insurance_content",
"insurance",
RULE_TYPE_CONTENT,
"insurance policy|versicherung|policennummer|coverage|premium|deductible",
),
# ── Payslip ───────────────────────────────────────────────────────────
ClassificationRule(
"builtin_payslip_filename", "payslip", RULE_TYPE_FILENAME, r"(?i)payslip|gehaltsabrechnung|lohnabrechnung"
),
ClassificationRule(
"builtin_payslip_content",
"payslip",
RULE_TYPE_CONTENT,
"gross salary|net salary|gehaltsabrechnung|lohnabrechnung|bruttolohn|nettolohn",
),
]
# ---------------------------------------------------------------------------
# Confidence scoring
# ---------------------------------------------------------------------------
#: Base confidence for each rule type when it matches.
_CONFIDENCE_MAP: dict[str, int] = {
RULE_TYPE_FILENAME: 60,
RULE_TYPE_CONTENT: 70,
RULE_TYPE_METADATA: 90,
}
#: Extra confidence per additional matching rule of the same category (capped).
_CONFIDENCE_BONUS_PER_EXTRA_RULE = 10
# ---------------------------------------------------------------------------
# Matching helpers
# ---------------------------------------------------------------------------
def _match_filename(rule: ClassificationRule, filename: str) -> bool:
"""Return True if *rule.pattern* (regex) matches anywhere in *filename*."""
if not filename:
return False
flags = 0 if rule.case_sensitive else re.IGNORECASE
return bool(re.search(rule.pattern, filename, flags))
def _match_content(rule: ClassificationRule, text: str) -> bool:
"""Return True if any keyword in *rule.pattern* appears in *text*.
Keywords are separated by ``|`` (pipe).
"""
if not text:
return False
keywords = [kw.strip() for kw in rule.pattern.split("|") if kw.strip()]
text_lower = text if rule.case_sensitive else text.lower()
return any((kw if rule.case_sensitive else kw.lower()) in text_lower for kw in keywords)
def _match_metadata(rule: ClassificationRule, metadata: dict[str, Any] | None) -> bool:
"""Return True if *rule.pattern* (``field=value``) matches *metadata*.
Pattern format: ``field_name=expected_value``.
"""
if not metadata:
return False
if "=" not in rule.pattern:
return False
field_name, expected_value = rule.pattern.split("=", 1)
actual = metadata.get(field_name.strip())
if actual is None:
return False
if rule.case_sensitive:
return str(actual) == expected_value.strip()
return str(actual).lower() == expected_value.strip().lower()
_MATCHERS: dict[str, tuple] = {
RULE_TYPE_FILENAME: (_match_filename, "filename"),
RULE_TYPE_CONTENT: (_match_content, "text"),
RULE_TYPE_METADATA: (_match_metadata, "metadata"),
}
def _evaluate_rule(
rule: ClassificationRule,
filename: str,
text: str,
metadata: dict[str, Any] | None,
) -> MatchedRule | None:
"""Evaluate a single rule against the document. Return a :class:`MatchedRule` on match."""
entry = _MATCHERS.get(rule.rule_type)
if entry is None:
return None
matcher, arg_key = entry
arg_map = {"filename": filename, "text": text, "metadata": metadata}
matched = matcher(rule, arg_map[arg_key])
if matched:
return MatchedRule(
rule_name=rule.name,
rule_type=rule.rule_type,
category=rule.category,
confidence=_CONFIDENCE_MAP.get(rule.rule_type, 50),
)
return None
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def classify_document(
filename: str = "",
text: str = "",
metadata: dict[str, Any] | None = None,
custom_rules: list[ClassificationRule] | None = None,
) -> ClassificationResult:
"""Classify a document by evaluating built-in and custom rules.
Rules are evaluated in priority order (highest first, then built-in before
custom for the same priority). The category with the most rule matches
wins; ties are broken by cumulative confidence.
Args:
filename: Original filename of the document.
text: Extracted / OCR text of the document.
metadata: Previously-extracted AI metadata dict (e.g. from ``ai_metadata``).
custom_rules: Optional list of user-defined :class:`ClassificationRule` objects.
Returns:
A :class:`ClassificationResult` with the best matching category,
overall confidence score, and the list of rules that fired.
"""
all_rules = list(BUILTIN_RULES)
if custom_rules:
all_rules.extend(custom_rules)
# Sort by priority descending (higher priority first)
all_rules.sort(key=lambda r: r.priority, reverse=True)
matches: list[MatchedRule] = []
for rule in all_rules:
result = _evaluate_rule(rule, filename, text, metadata)
if result is not None:
matches.append(result)
if not matches:
return ClassificationResult(category="unknown", confidence=0, matched_rules=[])
# Aggregate by category: pick the one with the most matches, then highest
# cumulative confidence as tiebreaker.
category_scores: dict[str, list[MatchedRule]] = {}
for m in matches:
category_scores.setdefault(m.category, []).append(m)
best_category = max(
category_scores,
key=lambda cat: (len(category_scores[cat]), sum(m.confidence for m in category_scores[cat])),
)
best_matches = category_scores[best_category]
base_confidence = max(m.confidence for m in best_matches)
bonus = min(
(len(best_matches) - 1) * _CONFIDENCE_BONUS_PER_EXTRA_RULE,
100 - base_confidence,
)
final_confidence = min(base_confidence + bonus, 100)
return ClassificationResult(
category=best_category,
confidence=final_confidence,
matched_rules=best_matches,
)
def db_rule_to_engine_rule(db_rule: Any) -> ClassificationRule:
"""Convert a database ``ClassificationRuleModel`` row to an engine :class:`ClassificationRule`.
Args:
db_rule: A SQLAlchemy model instance with ``name``, ``category``,
``rule_type``, ``pattern``, ``priority``, and ``case_sensitive`` attributes.
Returns:
A :class:`ClassificationRule` dataclass instance.
"""
return ClassificationRule(
name=db_rule.name,
category=db_rule.category,
rule_type=db_rule.rule_type,
pattern=db_rule.pattern,
priority=db_rule.priority,
case_sensitive=getattr(db_rule, "case_sensitive", False),
)
app/utils/config_validator/providers.py
+22
View File
@@ -296,6 +296,28 @@ def get_provider_status() -> dict[str, dict[str, object]]:
},
}
# Check SharePoint configuration
providers["SharePoint"] = {
"name": "SharePoint",
"icon": "fa-brands fa-microsoft",
"configured": bool(
getattr(settings, "sharepoint_client_id", None)
and getattr(settings, "sharepoint_client_secret", None)
and getattr(settings, "sharepoint_site_url", None)
),
"enabled": True,
"description": "Store documents in Microsoft SharePoint Online",
"details": {
"client_id": getattr(settings, "sharepoint_client_id", "Not set"),
"client_secret": mask_sensitive_value(getattr(settings, "sharepoint_client_secret", None)),
"tenant_id": getattr(settings, "sharepoint_tenant_id", "Not set"),
"refresh_token": mask_sensitive_value(getattr(settings, "sharepoint_refresh_token", None)),
"site_url": getattr(settings, "sharepoint_site_url", "Not set"),
"document_library": getattr(settings, "sharepoint_document_library", "Not set"),
"folder_path": getattr(settings, "sharepoint_folder_path", "Not set"),
},
}
# Check S3 configuration
providers["S3 Storage"] = {
"name": "S3 Storage",
app/utils/session_manager.py
+505
View File
@@ -0,0 +1,505 @@
"""Server-side session management utilities.
Provides helpers for creating, validating, and revoking user sessions.
Sessions are tracked in the ``user_sessions`` table and referenced by a
cryptographically random token stored in the browser cookie. This enables
the "log off everywhere" feature and per-session revocation.
"""
from __future__ import annotations
import logging
import secrets
from datetime import datetime, timedelta, timezone
from sqlalchemy.orm import Session
from app.config import settings
from app.models import ApiToken, QRLoginChallenge, UserSession
logger = logging.getLogger(__name__)
def _ensure_tz_aware(dt: datetime | None) -> datetime | None:
"""Return *dt* with UTC tzinfo if it is naive, or unchanged if already aware.
SQLite does not persist timezone information, so datetimes read back from
the database are offset-naive. This helper normalises them for safe
comparison with ``datetime.now(timezone.utc)``.
"""
if dt is not None and dt.tzinfo is None:
return dt.replace(tzinfo=timezone.utc)
return dt
def get_session_lifetime_days() -> int:
"""Return the effective session lifetime in days.
If ``session_lifetime_custom_days`` is set it takes precedence over
``session_lifetime_days``.
"""
custom = getattr(settings, "session_lifetime_custom_days", None)
if custom is not None and isinstance(custom, int) and custom > 0:
return custom
return max(1, getattr(settings, "session_lifetime_days", 30))
def get_session_max_age_seconds() -> int:
"""Return the session max-age in seconds for the cookie."""
return get_session_lifetime_days() * 86400
def create_session(
db: Session,
user_id: str,
ip_address: str | None = None,
user_agent: str | None = None,
) -> UserSession:
"""Create a new server-side session record.
Args:
db: Database session.
user_id: Stable owner identifier.
ip_address: Client IP address.
user_agent: Client User-Agent header.
Returns:
The newly created ``UserSession`` instance.
"""
session_token = secrets.token_urlsafe(64)
now = datetime.now(timezone.utc)
lifetime_days = get_session_lifetime_days()
expires_at = now + timedelta(days=lifetime_days)
device_info = _parse_device_info(user_agent)
user_session = UserSession(
session_token=session_token,
user_id=user_id,
ip_address=ip_address,
user_agent=(user_agent or "")[:512],
device_info=device_info,
created_at=now,
last_active_at=now,
expires_at=expires_at,
)
try:
db.add(user_session)
db.commit()
db.refresh(user_session)
except Exception:
db.rollback()
logger.exception("Failed to create session for user_id=%s", user_id)
raise
logger.info(
"[SESSION] Created session id=%s user=%s device=%r expires=%s",
user_session.id,
user_id,
device_info,
expires_at.isoformat(),
)
return user_session
def validate_session(db: Session, session_token: str) -> UserSession | None:
"""Validate a session token and return the session if valid.
A session is valid when:
* It exists in the database.
* ``is_revoked`` is ``False``.
* ``expires_at`` is in the future.
Side-effect: updates ``last_active_at`` on valid sessions.
Returns:
The ``UserSession`` if valid, else ``None``.
"""
if not session_token:
return None
now = datetime.now(timezone.utc)
user_session = db.query(UserSession).filter(UserSession.session_token == session_token).first()
if not user_session:
logger.debug("[SESSION] Token not found in database")
return None
if user_session.is_revoked:
logger.debug("[SESSION] Session id=%s is revoked", user_session.id)
return None
if user_session.expires_at:
expires = _ensure_tz_aware(user_session.expires_at)
if expires < now:
logger.debug("[SESSION] Session id=%s has expired", user_session.id)
return None
# Update last_active_at (throttled to avoid excessive writes)
last_active = _ensure_tz_aware(user_session.last_active_at)
if not last_active or (now - last_active).total_seconds() > 60:
try:
user_session.last_active_at = now
db.commit()
except Exception:
db.rollback()
logger.debug("[SESSION] Failed to update last_active_at for session id=%s", user_session.id)
return user_session
def revoke_session(db: Session, session_id: int, user_id: str) -> bool:
"""Revoke a single session by ID.
Args:
db: Database session.
session_id: The session record ID to revoke.
user_id: The owner — ensures a user can only revoke their own sessions.
Returns:
``True`` if the session was found and revoked, ``False`` otherwise.
"""
user_session = db.get(UserSession, session_id)
if not user_session or user_session.user_id != user_id:
return False
now = datetime.now(timezone.utc)
user_session.is_revoked = True
user_session.revoked_at = now
try:
db.commit()
except Exception:
db.rollback()
raise
logger.info("[SESSION] Revoked session id=%s user=%s", session_id, user_id)
return True
def revoke_all_sessions(
db: Session,
user_id: str,
*,
except_session_id: int | None = None,
revoke_api_tokens: bool = True,
) -> int:
"""Revoke all active sessions for a user ("log off everywhere").
Args:
db: Database session.
user_id: The owner whose sessions should be revoked.
except_session_id: If provided, keep this session active (the
current browser session).
revoke_api_tokens: If ``True``, also revoke all active API tokens.
Returns:
Number of sessions revoked.
"""
now = datetime.now(timezone.utc)
query = db.query(UserSession).filter(
UserSession.user_id == user_id,
UserSession.is_revoked.is_(False),
)
if except_session_id is not None:
query = query.filter(UserSession.id != except_session_id)
sessions = query.all()
count = 0
for s in sessions:
s.is_revoked = True
s.revoked_at = now
count += 1
if revoke_api_tokens:
tokens = (
db.query(ApiToken)
.filter(
ApiToken.owner_id == user_id,
ApiToken.is_active.is_(True),
)
.all()
)
for t in tokens:
t.is_active = False
t.revoked_at = now
try:
db.commit()
except Exception:
db.rollback()
raise
logger.info(
"[SESSION] Revoked all sessions for user=%s (count=%d, except_session_id=%s, tokens_revoked=%s)",
user_id,
count,
except_session_id,
revoke_api_tokens,
)
return count
def list_user_sessions(db: Session, user_id: str) -> list[UserSession]:
"""Return all non-revoked, non-expired sessions for a user.
Results are ordered by most recently active first.
"""
now = datetime.now(timezone.utc)
sessions = (
db.query(UserSession)
.filter(
UserSession.user_id == user_id,
UserSession.is_revoked.is_(False),
)
.order_by(UserSession.last_active_at.desc())
.all()
)
# Filter expired sessions in Python to handle timezone-naive datetimes (SQLite)
result = []
for s in sessions:
expires = _ensure_tz_aware(s.expires_at)
if expires and expires > now:
result.append(s)
return result
def cleanup_expired_sessions(db: Session) -> int:
"""Delete sessions that expired more than 7 days ago.
Intended to be called periodically (e.g. via Celery beat) to keep the
table from growing unbounded.
Returns:
Number of rows deleted.
"""
cutoff = datetime.now(timezone.utc) - timedelta(days=7)
count = db.query(UserSession).filter(UserSession.expires_at < cutoff).delete(synchronize_session=False)
try:
db.commit()
except Exception:
db.rollback()
raise
if count:
logger.info("[SESSION] Cleaned up %d expired sessions", count)
return count
# ---------------------------------------------------------------------------
# QR login helpers
# ---------------------------------------------------------------------------
def create_qr_challenge(db: Session, user_id: str, ip_address: str | None = None) -> QRLoginChallenge:
"""Create a new QR login challenge.
Args:
db: Database session.
user_id: The authenticated web user creating the challenge.
ip_address: IP address of the web client.
Returns:
The newly created ``QRLoginChallenge``.
"""
token = secrets.token_urlsafe(64)
ttl = getattr(settings, "qr_login_challenge_ttl_seconds", 120)
now = datetime.now(timezone.utc)
expires_at = now + timedelta(seconds=ttl)
challenge = QRLoginChallenge(
challenge_token=token,
user_id=user_id,
created_by_ip=ip_address,
created_at=now,
expires_at=expires_at,
)
try:
db.add(challenge)
db.commit()
db.refresh(challenge)
except Exception:
db.rollback()
logger.exception("Failed to create QR login challenge for user_id=%s", user_id)
raise
logger.info("[QR_AUTH] Challenge created: id=%s user=%s expires=%s", challenge.id, user_id, expires_at.isoformat())
return challenge
def validate_qr_challenge(db: Session, challenge_token: str) -> QRLoginChallenge | None:
"""Validate a QR challenge token without claiming it.
Returns the challenge if it exists, is not expired, not claimed,
and not cancelled. Returns ``None`` otherwise.
"""
if not challenge_token:
return None
now = datetime.now(timezone.utc)
challenge = db.query(QRLoginChallenge).filter(QRLoginChallenge.challenge_token == challenge_token).first()
if not challenge:
return None
if challenge.is_claimed or challenge.is_cancelled:
return None
expires = _ensure_tz_aware(challenge.expires_at)
if expires and expires < now:
return None
return challenge
def claim_qr_challenge(
db: Session,
challenge_token: str,
device_name: str = "Mobile App",
ip_address: str | None = None,
) -> dict | None:
"""Claim a QR challenge and issue an API token.
This is the critical security path. The challenge is validated,
marked as claimed atomically, and an API token is issued for the
user who created the challenge.
Args:
db: Database session.
challenge_token: The token from the QR code.
device_name: Name provided by the mobile app.
ip_address: IP address of the claiming mobile device.
Returns:
Dict with ``token`` (plaintext), ``token_id``, ``name``, ``owner_id``
and ``created_at`` on success, or ``None`` if the challenge is invalid.
"""
from app.api.api_tokens import generate_api_token, hash_token
challenge = validate_qr_challenge(db, challenge_token)
if not challenge:
logger.warning("[QR_AUTH] Invalid or expired challenge token attempted")
return None
now = datetime.now(timezone.utc)
# Mark as claimed first to prevent race conditions
challenge.is_claimed = True
challenge.claimed_at = now
challenge.claimed_by_ip = ip_address
challenge.device_name = device_name
# Generate API token for the mobile app
token_name = f"Mobile App (QR) – {device_name}"
plaintext = generate_api_token()
token_hash_value = hash_token(plaintext)
prefix = plaintext[:12]
db_token = ApiToken(
owner_id=challenge.user_id,
name=token_name,
token_hash=token_hash_value,
token_prefix=prefix,
)
try:
db.add(db_token)
db.flush()
challenge.issued_token_id = db_token.id
db.commit()
db.refresh(db_token)
except Exception:
db.rollback()
logger.exception("[QR_AUTH] Failed to issue token for challenge id=%s", challenge.id)
raise
logger.info(
"[QR_AUTH] Challenge claimed: id=%s user=%s device=%r token_id=%s",
challenge.id,
challenge.user_id,
device_name,
db_token.id,
)
return {
"token": plaintext,
"token_id": db_token.id,
"name": token_name,
"owner_id": challenge.user_id,
"created_at": db_token.created_at,
}
def get_challenge_status(db: Session, challenge_id: int, user_id: str) -> dict | None:
"""Get the current status of a QR challenge (for polling from the web UI).
Returns:
Dict with ``status`` ("pending", "claimed", "expired", "cancelled")
and metadata, or ``None`` if the challenge doesn't belong to the user.
"""
challenge = db.get(QRLoginChallenge, challenge_id)
if not challenge or challenge.user_id != user_id:
return None
now = datetime.now(timezone.utc)
expires = _ensure_tz_aware(challenge.expires_at)
if challenge.is_claimed:
status = "claimed"
elif challenge.is_cancelled:
status = "cancelled"
elif expires and expires < now:
status = "expired"
else:
status = "pending"
return {
"id": challenge.id,
"status": status,
"device_name": challenge.device_name,
"claimed_at": challenge.claimed_at,
"expires_at": challenge.expires_at,
}
def _parse_device_info(user_agent: str | None) -> str | None:
"""Extract a human-readable device description from User-Agent.
This is a lightweight parser — not a full UA library — that covers
the most common browsers and platforms.
"""
if not user_agent:
return None
ua = user_agent.lower()
# Platform detection
platform = "Unknown"
if "iphone" in ua:
platform = "iPhone"
elif "ipad" in ua:
platform = "iPad"
elif "android" in ua:
platform = "Android"
elif "macintosh" in ua or "mac os" in ua:
platform = "macOS"
elif "windows" in ua:
platform = "Windows"
elif "linux" in ua:
platform = "Linux"
elif "cros" in ua:
platform = "ChromeOS"
# Browser detection
browser = "Unknown Browser"
if "edg/" in ua or "edge/" in ua:
browser = "Edge"
elif "opr/" in ua or "opera" in ua:
browser = "Opera"
elif "chrome/" in ua and "safari/" in ua:
browser = "Chrome"
elif "safari/" in ua and "chrome/" not in ua:
browser = "Safari"
elif "firefox/" in ua:
browser = "Firefox"
elif "docuelevate" in ua:
browser = "DocuElevate App"
return f"{browser} on {platform}"
app/utils/settings_service.py
+219 -2
View File
@@ -39,6 +39,50 @@ SETTING_METADATA = {
"required": True,
"restart_required": True,
},
"db_pool_size": {
"category": "Core",
"description": (
"Number of persistent database connections kept in the pool per worker process. "
"Ignored for SQLite (which uses NullPool). Default: 10."
),
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": True,
},
"db_max_overflow": {
"category": "Core",
"description": (
"Additional database connections allowed beyond db_pool_size under burst load. "
"Ignored for SQLite. Default: 20."
),
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": True,
},
"db_pool_timeout": {
"category": "Core",
"description": (
"Seconds to wait for a database connection from the pool before raising a TimeoutError. "
"Ignored for SQLite. Default: 30."
),
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": True,
},
"db_pool_recycle": {
"category": "Core",
"description": (
"Recycle (close and reopen) database connections after this many seconds "
"to avoid stale connections. Ignored for SQLite. Default: 1800."
),
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": True,
},
"workdir": {
"category": "Core",
"description": "Working directory for file storage and processing",
@@ -55,6 +99,18 @@ SETTING_METADATA = {
"required": True, # Required for OAuth redirects and external URLs
"restart_required": True,
},
"public_base_url": {
"category": "Core",
"description": (
"Full public base URL including scheme (e.g., https://docuelevate.example.com). "
"When set, overrides auto-detected URLs for OAuth redirect URIs. "
"Required when behind a reverse proxy that does not forward X-Forwarded-Proto."
),
"type": "string",
"sensitive": False,
"required": False,
"restart_required": True,
},
"debug": {
"category": "Core",
"description": "Enable debug mode for verbose logging",
@@ -134,6 +190,30 @@ SETTING_METADATA = {
"required": True, # Required when auth_enabled=True (validated in config.py)
"restart_required": True,
},
"session_lifetime_days": {
"category": "Authentication",
"description": "Session lifetime in days (default 30). Determines how long a user stays logged in.",
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": True,
},
"session_lifetime_custom_days": {
"category": "Authentication",
"description": "Override session_lifetime_days with a custom value. Takes precedence when set.",
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": True,
},
"qr_login_challenge_ttl_seconds": {
"category": "Authentication",
"description": "Time-to-live in seconds for QR login challenges (default 120).",
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": False,
},
"admin_username": {
"category": "Authentication",
"description": "Admin username for local authentication",
@@ -302,6 +382,19 @@ SETTING_METADATA = {
"required": False,
"restart_required": True,
},
"social_auth_dropbox_use_global_credentials": {
"category": "Social Login",
"description": (
"When True, Dropbox social login uses the global DROPBOX_APP_KEY / DROPBOX_APP_SECRET "
"credentials instead of requiring separate SOCIAL_AUTH_DROPBOX_CLIENT_ID / "
"SOCIAL_AUTH_DROPBOX_CLIENT_SECRET values. "
"Requires SOCIAL_AUTH_DROPBOX_ENABLED=True and global Dropbox credentials to be set."
),
"type": "boolean",
"sensitive": False,
"required": False,
"restart_required": True,
},
"social_auth_dropbox_enabled": {
"category": "Social Login",
"description": (
@@ -695,6 +788,18 @@ SETTING_METADATA = {
"required": False,
"restart_required": False,
},
"dropbox_allow_global_credentials_for_integrations": {
"category": "Storage Providers",
"description": (
"When True, users may authorize their personal Dropbox integrations using the global "
"DROPBOX_APP_KEY / DROPBOX_APP_SECRET credentials configured by the admin, without "
"needing to create their own Dropbox app."
),
"type": "boolean",
"sensitive": False,
"required": False,
"restart_required": False,
},
# Storage Providers - Nextcloud
"nextcloud_enabled": {
"category": "Storage Providers",
@@ -875,6 +980,63 @@ SETTING_METADATA = {
"required": False,
"restart_required": False,
},
# Storage Providers - SharePoint
"sharepoint_client_id": {
"category": "Storage Providers",
"description": "SharePoint Azure AD application (client) ID",
"type": "string",
"sensitive": False,
"required": False,
"restart_required": False,
},
"sharepoint_client_secret": {
"category": "Storage Providers",
"description": "SharePoint Azure AD client secret",
"type": "string",
"sensitive": True,
"required": False,
"restart_required": False,
},
"sharepoint_tenant_id": {
"category": "Storage Providers",
"description": "SharePoint Azure AD tenant ID (use 'common' for multi-tenant apps)",
"type": "string",
"sensitive": False,
"required": False,
"restart_required": False,
},
"sharepoint_refresh_token": {
"category": "Storage Providers",
"description": "SharePoint OAuth refresh token",
"type": "string",
"sensitive": True,
"required": False,
"restart_required": False,
},
"sharepoint_site_url": {
"category": "Storage Providers",
"description": "SharePoint site URL (e.g. https://tenant.sharepoint.com/sites/sitename)",
"type": "string",
"sensitive": False,
"required": False,
"restart_required": False,
},
"sharepoint_document_library": {
"category": "Storage Providers",
"description": "SharePoint document library name (default: 'Documents')",
"type": "string",
"sensitive": False,
"required": False,
"restart_required": False,
},
"sharepoint_folder_path": {
"category": "Storage Providers",
"description": "Subfolder path inside the SharePoint document library",
"type": "string",
"sensitive": False,
"required": False,
"restart_required": False,
},
# Storage Providers - WebDAV
"webdav_enabled": {
"category": "Storage Providers",
@@ -1912,6 +2074,28 @@ SETTING_METADATA = {
"required": False,
"restart_required": False,
},
"factory_reset_on_startup": {
"category": "Feature Flags",
"description": (
"Wipe all user data on every startup so the instance always starts fresh. "
"Useful for demo/testing environments. Default: False."
),
"type": "boolean",
"sensitive": False,
"required": False,
"restart_required": True,
},
"enable_factory_reset": {
"category": "Feature Flags",
"description": (
"Show the System Reset page in the admin UI. Allows administrators to "
"trigger a full data wipe or a wipe-and-reimport from the web interface. Default: False."
),
"type": "boolean",
"sensitive": False,
"required": False,
"restart_required": False,
},
# Backup / Restore
"backup_enabled": {
"category": "Backup",
@@ -1935,14 +2119,26 @@ SETTING_METADATA = {
"category": "Backup",
"description": (
"Storage provider for remote backup copies. "
"Accepted values: s3, dropbox, google_drive, onedrive, nextcloud, webdav, ftp, sftp, email. "
"Accepted values: s3, dropbox, google_drive, onedrive, sharepoint, nextcloud, webdav, ftp, sftp, email. "
"Leave empty to keep backups local only."
),
"type": "string",
"sensitive": False,
"required": False,
"restart_required": False,
"options": ["", "s3", "dropbox", "google_drive", "onedrive", "nextcloud", "webdav", "ftp", "sftp", "email"],
"options": [
"",
"s3",
"dropbox",
"google_drive",
"onedrive",
"sharepoint",
"nextcloud",
"webdav",
"ftp",
"sftp",
"email",
],
},
"backup_remote_folder": {
"category": "Backup",
@@ -2454,6 +2650,27 @@ SETTING_METADATA = {
"required": False,
"restart_required": False,
},
# Per-user upload rate limiting
"upload_rate_limit_per_user": {
"category": "Security",
"description": (
"Maximum number of uploads a single user may submit within upload_rate_limit_window seconds. "
"The health-aware limiter may reduce this dynamically under high Redis queue depth or CPU load. "
"Default: 20."
),
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": False,
},
"upload_rate_limit_window": {
"category": "Security",
"description": ("Sliding window in seconds over which upload_rate_limit_per_user is enforced. Default: 60."),
"type": "integer",
"sensitive": False,
"required": False,
"restart_required": False,
},
# Rate Limiting
"rate_limiting_enabled": {
"category": "Security",
app/utils/system_reset.py
+296
View File
@@ -0,0 +1,296 @@
"""
System reset utilities for DocuElevate.
Provides functions to:
- Wipe all user data (database rows + work-files on disk) for a fresh start.
- Wipe with re-import: move original files to a dedicated folder, wipe
everything, then let the watch-folder mechanism re-ingest the files.
Security: All public functions in this module require admin-level access.
They MUST only be invoked from admin-guarded API/view endpoints.
"""
import logging
import shutil
from pathlib import Path
from sqlalchemy.orm import Session
from app.config import settings
logger = logging.getLogger(__name__)
# Subdirectories inside *workdir* that contain user-generated data.
# Everything else (app code, static assets, config) is left untouched.
_USER_DATA_SUBDIRS = ("original", "processed", "tmp", "pdfa", "backups")
# JSON cache files written by watch-folder / ingest tasks.
_CACHE_FILES = (
"watch_folder_processed.json",
"ftp_ingest_processed.json",
"sftp_ingest_processed.json",
"dropbox_ingest_processed.json",
"gdrive_ingest_processed.json",
"onedrive_ingest_processed.json",
"nextcloud_ingest_processed.json",
"s3_ingest_processed.json",
"webdav_ingest_processed.json",
"processed_mails.json",
"credential_failures.json",
)
# The folder name used for storing files prior to re-import.
REIMPORT_FOLDER_NAME = "reimport"
def _wipe_workdir_data(workdir: str) -> dict[str, int]:
"""Delete user data subdirectories and cache files inside *workdir*.
Leaves the workdir directory itself intact so the application can
continue to write into it. Also leaves any files that do not belong
to the known data subdirectories or caches.
Returns:
A dict with counts of deleted directories and files.
"""
workdir_path = Path(workdir)
deleted_dirs = 0
deleted_files = 0
# Remove data subdirectories
for subdir in _USER_DATA_SUBDIRS:
target = workdir_path / subdir
if target.is_dir():
shutil.rmtree(target)
logger.info("Deleted data directory: %s", target)
deleted_dirs += 1
# Remove cache / state JSON files
for cache_file in _CACHE_FILES:
target = workdir_path / cache_file
if target.is_file():
target.unlink()
logger.info("Deleted cache file: %s", target)
deleted_files += 1
# Also remove user_wf_*.json files (per-user watch folder caches)
for f in workdir_path.glob("user_wf_*.json"):
f.unlink()
logger.info("Deleted user watch-folder cache: %s", f)
deleted_files += 1
# Remove loose files in workdir root that are user uploads (uuid-named
# files like "a1b2c3d4-…pdf") but NOT application config files.
for entry in workdir_path.iterdir():
if entry.is_file() and entry.suffix.lower() in {
".pdf",
".png",
".jpg",
".jpeg",
".tiff",
".tif",
".docx",
".doc",
".xlsx",
".xls",
".pptx",
".heic",
".heif",
".webp",
".bmp",
".gif",
".txt",
".rtf",
".odt",
".ods",
".odp",
".csv",
".pages",
".numbers",
".keynote",
}:
entry.unlink()
logger.info("Deleted loose workdir file: %s", entry)
deleted_files += 1
return {"deleted_dirs": deleted_dirs, "deleted_files": deleted_files}
def _wipe_database(db: Session) -> dict[str, int]:
"""Delete all user-generated rows from the database.
Preserves schema (tables, migrations) and system-seeded rows that will
be re-created on the next startup (subscription plans, default pipeline,
scheduled jobs, compliance templates).
Returns:
A dict mapping table name → number of rows deleted.
"""
from app.models import (
AuditLog,
BackupRecord,
DocumentMetadata,
FileProcessingStep,
FileRecord,
InAppNotification,
ProcessingLog,
SavedSearch,
SettingsAuditLog,
SharedLink,
UserImapAccount,
UserIntegration,
UserNotificationPreference,
UserNotificationTarget,
)
# Order matters: delete children before parents to respect FK constraints.
tables_to_wipe: list[tuple[str, type]] = [
("file_processing_steps", FileProcessingStep),
("processing_logs", ProcessingLog),
("shared_links", SharedLink),
("in_app_notifications", InAppNotification),
("user_notification_preferences", UserNotificationPreference),
("user_notification_targets", UserNotificationTarget),
("user_imap_accounts", UserImapAccount),
("user_integrations", UserIntegration),
("saved_searches", SavedSearch),
("settings_audit_log", SettingsAuditLog),
("audit_logs", AuditLog),
("backup_records", BackupRecord),
("document_metadata", DocumentMetadata),
("files", FileRecord),
]
result: dict[str, int] = {}
for table_name, model in tables_to_wipe:
try:
count = db.query(model).delete()
result[table_name] = count
logger.info("Wiped %d rows from %s", count, table_name)
except Exception:
logger.exception("Failed to wipe table %s during system reset", table_name)
db.rollback()
raise
db.commit()
return result
def perform_full_reset(db: Session) -> dict:
"""Perform a complete system reset: wipe database rows + work-files.
Args:
db: An active SQLAlchemy session.
Returns:
Summary dict with ``database`` and ``filesystem`` sub-dicts.
"""
logger.warning(">>> SYSTEM RESET: wiping all user data <<<")
db_result = _wipe_database(db)
fs_result = _wipe_workdir_data(settings.workdir)
logger.warning(">>> SYSTEM RESET complete <<<")
return {"database": db_result, "filesystem": fs_result}
def perform_reset_and_reimport(db: Session) -> dict:
"""Move original files to a reimport folder, wipe everything, then
configure the reimport folder as a watch folder for re-ingestion.
The watch-folder scanner (``scan_all_watch_folders``) will pick up
the files on its next periodic run and process them exactly as if
they had been freshly uploaded — respecting the same backoff
strategy, size limits, and rate limits.
Args:
db: An active SQLAlchemy session.
Returns:
Summary dict with ``database``, ``filesystem``, and ``reimport`` sub-dicts.
"""
workdir_path = Path(settings.workdir)
reimport_dir = workdir_path / REIMPORT_FOLDER_NAME
original_dir = workdir_path / "original"
# 1. Collect original files
files_moved = 0
reimport_dir.mkdir(parents=True, exist_ok=True)
if original_dir.is_dir():
for entry in original_dir.iterdir():
if entry.is_file():
# Validate the resolved path stays within original_dir (path traversal guard)
try:
entry.resolve().relative_to(original_dir.resolve())
except ValueError:
logger.warning("Skipping file outside original dir: %s", entry)
continue
dest = reimport_dir / entry.name
# Avoid overwriting: append counter if name clash
if dest.exists():
stem = dest.stem
suffix = dest.suffix
counter = 1
while dest.exists():
dest = reimport_dir / f"{stem}_{counter}{suffix}"
counter += 1
shutil.copy2(str(entry), str(dest))
files_moved += 1
logger.info("Copied %d original files to reimport folder: %s", files_moved, reimport_dir)
# 2. Perform the full reset (wipe DB + other workdir data)
reset_result = perform_full_reset(db)
# 3. Ensure the reimport folder survived the wipe (it's not in _USER_DATA_SUBDIRS)
# and set up watch folder config to point at it.
_configure_reimport_watch_folder(str(reimport_dir))
reset_result["reimport"] = {
"files_moved": files_moved,
"reimport_folder": str(reimport_dir),
}
logger.warning(">>> SYSTEM RESET with re-import configured — %d files staged <<<", files_moved)
return reset_result
def _configure_reimport_watch_folder(reimport_path: str) -> None:
"""Append *reimport_path* to the application's watch-folder list.
The watch-folder scanner uses ``settings.watch_folders`` (a
comma-separated string). We mutate the runtime setting so the
next scan picks up the folder. We also set
``watch_folder_delete_after_process = True`` so files are cleaned
up after successful processing.
"""
current = getattr(settings, "watch_folders", None) or ""
folders = [f.strip() for f in current.split(",") if f.strip()]
if reimport_path not in folders:
folders.append(reimport_path)
# Mutate runtime settings (not persisted to .env — ephemeral)
object.__setattr__(settings, "watch_folders", ",".join(folders))
object.__setattr__(settings, "watch_folder_delete_after_process", True)
logger.info("Configured reimport watch folder: %s", reimport_path)
def perform_startup_reset() -> None:
"""Called during application startup when ``FACTORY_RESET_ON_STARTUP=True``.
Wipes database and filesystem data so the instance starts completely
fresh. Uses its own DB session so it runs before the normal lifespan
seeding logic.
"""
from app.database import SessionLocal
logger.warning("FACTORY_RESET_ON_STARTUP is enabled — wiping all data")
db = SessionLocal()
try:
perform_full_reset(db)
except Exception:
logger.exception("Factory reset on startup failed")
db.rollback()
finally:
db.close()
app/views/__init__.py
+6
View File
@@ -10,6 +10,7 @@ from app.views.audit_logs import router as audit_logs_router
from app.views.backup import router as backup_router
from app.views.compliance import router as compliance_router
from app.views.db_wizard import router as db_wizard_router
from app.views.devices import router as devices_router # Mobile devices dashboard
from app.views.dropbox import router as dropbox_router
from app.views.filemanager import router as filemanager_router
@@ -26,6 +27,7 @@ from app.views.onedrive import router as onedrive_router
from app.views.pipelines import router as pipelines_router # Processing pipelines
from app.views.plans import router as plans_router # Admin Plan Designer
from app.views.profile import router as profile_router # User self-service profile
from app.views.qr_login import router as qr_login_router # QR code mobile login
from app.views.queue import router as queue_router
from app.views.scheduled_jobs import router as scheduled_jobs_router # Scheduled batch jobs
from app.views.search import router as search_router
@@ -34,6 +36,7 @@ from app.views.share import router as share_router
from app.views.shared_links import router as shared_links_router
from app.views.status import router as status_router
from app.views.subscriptions import router as subscriptions_router # Pricing + subscription pages
from app.views.system_reset import router as system_reset_router # System reset / factory reset
from app.views.wizard import router as wizard_router
# Create a main router that includes all the view routers
@@ -60,6 +63,7 @@ router.include_router(plans_router) # Admin Plan Designer
router.include_router(onboarding_router) # User onboarding wizard
router.include_router(pipelines_router) # Processing pipelines
router.include_router(profile_router) # User self-service profile settings
router.include_router(qr_login_router) # QR code mobile login page
router.include_router(imap_accounts_router) # Per-user IMAP ingestion accounts
router.include_router(integrations_router) # Unified integrations dashboard
router.include_router(notifications_router) # User notification dashboard
@@ -67,3 +71,5 @@ router.include_router(scheduled_jobs_router) # Admin scheduled batch jobs
router.include_router(audit_logs_router) # Comprehensive audit log viewer
router.include_router(help_router) # Built-in help / How-To docs
router.include_router(compliance_router) # Compliance templates dashboard
router.include_router(devices_router) # Mobile devices dashboard
router.include_router(system_reset_router) # System reset / factory reset
app/views/base.py
+1
View File
@@ -94,6 +94,7 @@ def _inject_global_context(ctx: dict) -> None:
"allow_signup",
getattr(settings, "multi_user_enabled", False) and getattr(settings, "allow_local_signup", False),
)
ctx.setdefault("enable_factory_reset", getattr(settings, "enable_factory_reset", False))
req = ctx.get("request")
if req is not None:
app/views/devices.py
+25
View File
@@ -0,0 +1,25 @@
"""View route for the Devices management page.
Renders the ``devices.html`` template where users can see their registered
mobile devices, mobile API tokens (created via the mobile SSO flow or QR
code login), and revoke access per-device.
"""
import logging
from fastapi import APIRouter, Request
from app.views.base import require_login, templates
logger = logging.getLogger(__name__)
router = APIRouter()
@router.get("/devices", include_in_schema=False)
@require_login
async def devices_page(request: Request):
"""Render the Devices management page."""
return templates.TemplateResponse(
"devices.html",
{"request": request, "page_title": "Devices"},
)
app/views/dropbox.py
+27 -1
View File
@@ -14,6 +14,19 @@ from app.views.base import APIRouter, Depends, get_db, require_login, settings,
router = APIRouter()
def _get_dropbox_callback_url(request: Request) -> str:
"""Return the Dropbox OAuth callback URL.
Uses ``PUBLIC_BASE_URL`` when configured so that the redirect URI displayed
to the user (and registered in the Dropbox developer console) matches the
one used in the OAuth authorization request. Falls back to deriving the URL
from the incoming request when ``PUBLIC_BASE_URL`` is not set.
"""
if settings.public_base_url:
return settings.public_base_url.rstrip("/") + "/dropbox-callback"
return f"{request.url.scheme}://{request.url.netloc}/dropbox-callback"
@router.get("/dropbox-setup")
@require_login
async def dropbox_setup_page(
@@ -30,6 +43,8 @@ async def dropbox_setup_page(
path from the integration's existing config is pre-populated; global
admin credentials are never exposed in this mode.
"""
callback_url = _get_dropbox_callback_url(request)
if integration_id is not None:
owner_id = get_current_owner_id(request)
integration = (
@@ -46,6 +61,12 @@ async def dropbox_setup_page(
cfg = {}
# Support both "folder" (DROPBOX destination) and "folder_path" (WATCH_FOLDER source)
folder_path = cfg.get("folder", cfg.get("folder_path", ""))
# Determine if global credentials are available for users to reuse
global_creds_available = bool(
settings.dropbox_allow_global_credentials_for_integrations
and settings.dropbox_app_key
and settings.dropbox_app_secret
)
return templates.TemplateResponse(
"dropbox.html",
{
@@ -56,9 +77,12 @@ async def dropbox_setup_page(
"integration_name": integration.name,
"integration_type": integration.integration_type,
"folder_path": folder_path,
"app_key_value": "",
# Only expose the public app key (not the secret) when global creds are allowed
"app_key_value": settings.dropbox_app_key if global_creds_available else "",
"app_secret_value": "",
"refresh_token_value": "",
"global_creds_available": global_creds_available,
"callback_url": callback_url,
},
)
@@ -78,6 +102,7 @@ async def dropbox_setup_page(
"integration_id": integration_id,
"integration_name": None,
"integration_type": None,
"callback_url": callback_url,
},
)
@@ -108,5 +133,6 @@ async def dropbox_callback(request: Request, code: str = None, error: str = None
"app_key_value": "", # The callback will prioritize sessionStorage values
"app_secret_value": "", # The callback will prioritize sessionStorage values
"folder_path": "", # The callback will prioritize sessionStorage values
"callback_url": _get_dropbox_callback_url(request),
},
)
app/views/files.py
+1 -3
View File
@@ -396,9 +396,7 @@ _STEP_TYPE_TO_STAGES: dict[str, list[str]] = {
"embed_metadata": ["embed_metadata_into_pdf"],
"compute_embedding": ["compute_embedding"],
"send_to_destinations": ["finalize_document_storage", "send_to_all_destinations"],
# "classify" is defined in PIPELINE_STEP_TYPES but has no Celery log stages yet.
# When a classify task is implemented, add its stage key(s) here.
"classify": [],
"classify": ["classify_document"],
}
# These internal bookkeeping stages are always shown in the flow regardless of
app/views/google_drive.py
+11 -4
View File
@@ -45,6 +45,9 @@ async def google_drive_setup_page(
except (json.JSONDecodeError, TypeError):
cfg = {}
folder_id = cfg.get("folder_id", "")
# Provide system-wide OAuth credentials when available so users can
# authorize without registering their own Google Cloud app.
has_system_credentials = bool(settings.google_drive_client_id and settings.google_drive_client_secret)
return templates.TemplateResponse(
"google_drive.html",
{
@@ -58,10 +61,13 @@ async def google_drive_setup_page(
"use_oauth": True,
"oauth_configured": bool(integration.credentials),
"sa_configured": False,
"client_id": False,
"client_id_value": "",
"client_secret": False,
"client_secret_value": "",
"has_system_credentials": has_system_credentials,
"client_id": bool(settings.google_drive_client_id) if has_system_credentials else False,
"client_id_value": (settings.google_drive_client_id or "" if has_system_credentials else ""),
"client_secret": bool(settings.google_drive_client_secret) if has_system_credentials else False,
"client_secret_value": (
settings.google_drive_client_secret or "" if has_system_credentials else ""
),
"refresh_token": False,
"refresh_token_value": "",
"has_credentials_json": False,
@@ -90,6 +96,7 @@ async def google_drive_setup_page(
"use_oauth": use_oauth,
"oauth_configured": oauth_configured,
"sa_configured": sa_configured,
"has_system_credentials": bool(settings.google_drive_client_id and settings.google_drive_client_secret),
"client_id": bool(settings.google_drive_client_id),
"client_id_value": settings.google_drive_client_id or "",
"client_secret": bool(settings.google_drive_client_secret),
app/views/onedrive.py
+10 -5
View File
@@ -44,6 +44,9 @@ async def onedrive_setup_page(
cfg = {}
# Support both "folder_path" (WATCH_FOLDER / ONEDRIVE destination)
folder_path = cfg.get("folder_path", cfg.get("folder", ""))
# Provide system-wide app credentials when available so users can
# authorize without registering their own Azure/OneDrive app.
has_system_credentials = bool(settings.onedrive_client_id and settings.onedrive_client_secret)
return templates.TemplateResponse(
"onedrive.html",
{
@@ -54,11 +57,12 @@ async def onedrive_setup_page(
"integration_name": integration.name,
"integration_type": integration.integration_type,
"folder_path": folder_path,
"client_id": False,
"client_id_value": "",
"client_secret": False,
"client_secret_value": "",
"tenant_id": "common",
"has_system_credentials": has_system_credentials,
"client_id": bool(settings.onedrive_client_id) if has_system_credentials else False,
"client_id_value": settings.onedrive_client_id or "" if has_system_credentials else "",
"client_secret": bool(settings.onedrive_client_secret) if has_system_credentials else False,
"client_secret_value": (settings.onedrive_client_secret or "" if has_system_credentials else ""),
"tenant_id": settings.onedrive_tenant_id or "common",
"refresh_token": False,
"refresh_token_value": "",
},
@@ -75,6 +79,7 @@ async def onedrive_setup_page(
"request": request,
"user_mode": False,
"is_configured": is_configured,
"has_system_credentials": bool(settings.onedrive_client_id and settings.onedrive_client_secret),
"client_id": bool(settings.onedrive_client_id),
"client_id_value": settings.onedrive_client_id or "",
"client_secret": bool(settings.onedrive_client_secret),
app/views/qr_login.py
+26
View File
@@ -0,0 +1,26 @@
"""View route for the QR code mobile login page.
Route:
GET /qr-login — renders the QR login page (requires login)
"""
from __future__ import annotations
import logging
from fastapi import Request
from app.views.base import APIRouter, require_login, templates
logger = logging.getLogger(__name__)
router = APIRouter()
@router.get("/qr-login", include_in_schema=False)
@require_login
async def qr_login_page(request: Request):
"""Serve the QR code login page for mobile app authentication."""
return templates.TemplateResponse(
"qr_login.html",
{"request": request},
)
app/views/system_reset.py
+40
View File
@@ -0,0 +1,40 @@
"""
System reset view — admin-only UI page.
Renders a confirmation-heavy page that allows administrators to:
1. **Full Reset** — wipe all user data (DB + disk) for a fresh start.
2. **Reset & Re-import** — move originals to a reimport folder, wipe,
and let the watch-folder mechanism re-ingest them.
Both options are gated behind the ``ENABLE_FACTORY_RESET`` feature flag.
"""
import logging
from fastapi import Depends, Request
from fastapi.responses import RedirectResponse, Response
from sqlalchemy.orm import Session
from app.config import settings
from app.views.base import APIRouter, get_db, require_login, templates
from app.views.settings import require_admin_access
logger = logging.getLogger(__name__)
router = APIRouter()
@router.get("/admin/system-reset")
@require_login
@require_admin_access
async def system_reset_page(request: Request, db: Session = Depends(get_db)) -> Response:
"""Render the system reset administration page."""
if not settings.enable_factory_reset:
return RedirectResponse(url="/settings", status_code=302)
return templates.TemplateResponse(
"system_reset.html",
{
"request": request,
"factory_reset_on_startup": settings.factory_reset_on_startup,
},
)
docker-compose.yaml
+25 -4
View File
@@ -3,7 +3,7 @@ services:
build:
context: .
dockerfile: Dockerfile
container_name: document_api
# No container_name — allows `docker compose up --scale api=N`
restart: always
# We'll keep the code in /app, but set working_dir to the shared data directory
@@ -24,7 +24,7 @@ services:
depends_on:
- redis
- worker
- beat
# Mount the shared working directory for data
volumes:
@@ -34,13 +34,14 @@ services:
build:
context: .
dockerfile: Dockerfile
container_name: document_worker
# No container_name — allows `docker compose up --scale worker=N`
restart: always
# same shared working directory
working_dir: /workdir
command: ["celery", "-A", "app.celery_worker", "worker", "-B", "--loglevel=info", "-Q", "document_processor,default,celery"]
# Workers process tasks only — no -B flag (Beat runs in the dedicated beat service)
command: ["celery", "-A", "app.celery_worker", "worker", "--loglevel=info", "-Q", "document_processor,default,celery"]
env_file:
- .env
environment:
@@ -54,6 +55,26 @@ services:
volumes:
- /var/docparse/workdir:/workdir
# Dedicated Celery Beat scheduler — exactly one instance must run at all times.
# Beat publishes periodic tasks to the Redis broker; workers pick them up.
# Do NOT scale this service (replicas must stay at 1).
beat:
build:
context: .
dockerfile: Dockerfile
container_name: document_beat
restart: always
working_dir: /workdir
command: ["celery", "-A", "app.celery_worker", "beat", "--loglevel=info"]
env_file:
- .env
environment:
- PYTHONPATH=/app
depends_on:
- redis
volumes:
- /var/docparse/workdir:/workdir
gotenberg:
image: gotenberg/gotenberg:latest
container_name: gotenberg
docs/API.md
+371 -17
View File
@@ -27,9 +27,11 @@ DocuElevate implements rate limiting to protect against abuse and DoS attacks. R
### Default Limits
- **Default endpoints**: 100 requests per minute
- **File upload**: 600 requests per minute
- **File upload**: 600 requests per minute (global) + 20 per user per 60 s (per-user, health-aware)
- **Authentication**: 10 requests per minute
**Per-user upload rate limiting**: Upload endpoints (`/api/ui-upload`, `/api/process-url`) enforce a per-user sliding-window limit that adapts to system load. Under heavy queue depth or high CPU usage, the effective limit is reduced automatically. See the [Configuration Guide](ConfigurationGuide.md#per-user-upload-rate-limiting) for details.
**Note**: Document processing endpoints (OCR, metadata extraction) use built-in queue throttling to control processing rates and prevent upstream API overloads. No additional API-level rate limit is applied to processing endpoints.
### Rate Limit Headers
@@ -53,6 +55,10 @@ RATE_LIMITING_ENABLED=true
RATE_LIMIT_DEFAULT=100/minute
RATE_LIMIT_UPLOAD=600/minute
RATE_LIMIT_AUTH=10/minute
# Per-user upload rate limiting (health-aware)
UPLOAD_RATE_LIMIT_PER_USER=20 # Max uploads per user per window
UPLOAD_RATE_LIMIT_WINDOW=60 # Sliding window in seconds
```
See [Configuration Guide](ConfigurationGuide.md) for more details.
@@ -115,7 +121,8 @@ curl -X GET "http://<your-docuelevate-instance>/api/files" \
|--------|----------|-------------|
| `POST` | `/api/api-tokens/` | Create a new token |
| `GET` | `/api/api-tokens/` | List all your tokens |
| `DELETE` | `/api/api-tokens/{id}` | Revoke a token |
| `DELETE` | `/api/api-tokens/{id}` | Revoke (active) or permanently delete (revoked) a token |
| `POST` | `/api/api-tokens/{id}/reactivate` | Reactivate a revoked token |
### Session Authentication
@@ -235,17 +242,33 @@ The DocuElevate browser extension uses this endpoint to send files directly from
**POST** `/api/ui-upload`
Upload one or more files from your computer for processing.
Upload a file from your computer for processing.
**Request**:
- Multipart form data with file(s)
- Multipart form data with a single `file` field
**Response**:
**Response** (new file):
```json
{
"success": true,
"file_ids": [123, 124],
"message": "Files uploaded and queued for processing"
"task_id": "abc-123",
"status": "queued",
"original_filename": "invoice.pdf",
"stored_filename": "a1b2c3d4.pdf"
}
```
**Response** (exact duplicate, when `ENABLE_DEDUPLICATION=True`):
```json
{
"status": "duplicate",
"original_filename": "invoice.pdf",
"stored_filename": "e5f6a7b8.pdf",
"duplicate_of": {
"duplicate_type": "exact",
"original_file_id": 42,
"original_filename": "invoice.pdf",
"message": "This file is an exact duplicate of an already-processed document. It has not been queued for processing again."
}
}
```
@@ -1356,7 +1379,7 @@ Test an integration connection without saving. Useful for "Test connection" UI b
{"success": true, "message": "IMAP connection successful"}
```
Supported connection tests: `IMAP`, `S3`, `WEBDAV`, `NEXTCLOUD`. Other types return a message that testing is not yet supported.
Supported connection tests: `DROPBOX`, `IMAP`, `S3`, `WEBDAV`, `NEXTCLOUD`. Other types return a message that testing is not yet supported.
### GET /api/integrations/quota/
@@ -1381,6 +1404,55 @@ Get the current user's integration quota usage.
}
```
## Cloud Provider Folder Browser
Browse folders in connected cloud storage providers. These endpoints are used by the OAuth callback pages to let users select a target folder after authorization.
### POST /api/dropbox/list-folders
List folders in a Dropbox account. Requires a short-lived OAuth access token obtained during the authorization flow.
**Request (form-data):**
| Field | Type | Required | Description |
|----------------|--------|----------|--------------------------------------|
| `access_token` | string | Yes | Dropbox OAuth access token |
| `path` | string | No | Folder path to list (default: root) |
**Response (200):**
```json
{
"folders": [
{ "name": "Documents", "path": "/Documents", "id": "id:abc123" },
{ "name": "Photos", "path": "/Photos", "id": "id:def456" }
],
"path": "/",
"has_more": false
}
```
### POST /api/onedrive/list-folders
List folders in a OneDrive account. Requires a short-lived OAuth access token obtained during the authorization flow.
**Request (form-data):**
| Field | Type | Required | Description |
|----------------|--------|----------|--------------------------------------|
| `access_token` | string | Yes | Microsoft Graph access token |
| `path` | string | No | Folder path to list (default: root) |
**Response (200):**
```json
{
"folders": [
{ "name": "Documents", "path": "/Documents", "id": "abc123", "child_count": 5 },
{ "name": "Pictures", "path": "/Pictures", "id": "def456", "child_count": 12 }
],
"path": "/"
}
```
## Webhooks
Manage webhook configurations for notifying external systems when document events occur. All webhook endpoints require admin access.
@@ -1571,6 +1643,47 @@ Lightweight endpoint returning the total number of queued + in-progress items. D
## Diagnostic
### GET /api/diagnostic/healthz/live
Lightweight liveness probe for Kubernetes. Returns **200 OK** as long as the process is running. This endpoint does **not** check external dependencies and is intentionally cheap.
**Authentication:** None (designed for kubelet probes)
**Response (200 OK):**
```json
{
"status": "ok"
}
```
### GET /api/diagnostic/healthz/ready
Readiness probe for Kubernetes. Verifies that the application can serve traffic by checking database and Redis connectivity.
**Authentication:** None (designed for kubelet probes)
**Response (200 OK) – ready to serve traffic:**
```json
{
"status": "ready",
"checks": {
"database": {"status": "ok"},
"redis": {"status": "ok"}
}
}
```
**Response (503 Service Unavailable) – database unreachable:**
```json
{
"status": "not_ready",
"checks": {
"database": {"status": "error", "detail": "..."},
"redis": {"status": "ok"}
}
}
```
### GET /api/diagnostic/health
System health endpoint designed for monitoring tools such as Grafana, Uptime Kuma, Prometheus blackbox exporter, or any HTTP-based health checker.
@@ -2127,12 +2240,14 @@ Usage tracking records when each token was last used and from which IP address.
### POST /api/api-tokens/
Create a new API token.
Create a new API token. Optionally specify a lifetime in days via
`expires_in_days` (1–3650). If omitted the token never expires.
**Request:**
```json
{
"name": "CI Pipeline"
"name": "CI Pipeline",
"expires_in_days": 90
}
```
@@ -2147,7 +2262,8 @@ Create a new API token.
"last_used_at": null,
"last_used_ip": null,
"created_at": "2026-03-08T12:00:00Z",
"revoked_at": null
"revoked_at": null,
"expires_at": "2026-06-06T12:00:00Z"
}
```
@@ -2169,15 +2285,20 @@ List all tokens for the authenticated user. The full token value is never includ
"last_used_at": "2026-03-08T15:30:00Z",
"last_used_ip": "203.0.113.42",
"created_at": "2026-03-08T12:00:00Z",
"revoked_at": null
"revoked_at": null,
"expires_at": "2026-06-06T12:00:00Z"
}
]
```
### DELETE /api/api-tokens/{token_id}
Revoke a token. The token is soft-deleted (kept for audit purposes) and can no
longer be used for authentication.
Revoke or permanently delete a token:
* **Active token** – soft-revoked (kept for audit purposes, marked inactive).
Response: `{"detail": "Token revoked"}`
* **Already-revoked token** – permanently deleted from the database.
Response: `{"detail": "Token deleted"}`
**Response (200):**
```json
@@ -2186,6 +2307,13 @@ longer be used for authentication.
}
```
### POST /api/api-tokens/{token_id}/reactivate
Reactivate a previously revoked token. Clears `revoked_at` and sets
`is_active` back to `true`.
**Response (200):** The updated `TokenResponse` object.
### Using API Tokens
Include the token in the `Authorization` header of any API request:
@@ -2214,6 +2342,164 @@ print(response.json())
```
## Classification Rules
The classification rules API lets you manage custom document classification rules. Rules are evaluated during the `classify` pipeline step to assign a category to each document based on filename patterns, content keywords, and metadata fields.
### Built-in Categories
```bash
GET /api/classification-rules/categories
```
Returns the pre-built classification categories.
**Response (200):**
```json
{
"invoice": "Invoice",
"contract": "Contract",
"receipt": "Receipt",
"letter": "Letter",
"report": "Report",
"bank_statement": "Bank Statement",
"tax_document": "Tax Document",
"insurance": "Insurance Document",
"payslip": "Payslip",
"unknown": "Unknown"
}
```
### Rule Types
```bash
GET /api/classification-rules/rule-types
```
Returns the supported rule types with descriptions.
**Response (200):**
```json
[
{
"type": "filename_pattern",
"label": "Filename Pattern",
"description": "Regex pattern matched against the original filename."
},
{
"type": "content_keyword",
"label": "Content Keyword",
"description": "Pipe-separated keywords matched against the OCR text."
},
{
"type": "metadata_match",
"label": "Metadata Match",
"description": "field=value pattern matched against existing AI metadata."
}
]
```
### List Rules
```bash
GET /api/classification-rules/
```
List all classification rules visible to the current user (system rules + own rules).
**Response (200):**
```json
[
{
"id": 1,
"owner_id": "user@example.com",
"name": "German Invoice Filename",
"category": "invoice",
"rule_type": "filename_pattern",
"pattern": "(?i)rechnung",
"priority": 10,
"case_sensitive": false,
"enabled": true
}
]
```
### Create Rule
```bash
POST /api/classification-rules/
```
Create a new custom classification rule.
**Request:**
```json
{
"name": "German Invoice Filename",
"category": "invoice",
"rule_type": "filename_pattern",
"pattern": "(?i)rechnung",
"priority": 10,
"case_sensitive": false,
"enabled": true
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | Yes | Unique rule name (per user) |
| `category` | string | Yes | Target category (e.g. `invoice`, `contract`, or custom) |
| `rule_type` | string | Yes | One of: `filename_pattern`, `content_keyword`, `metadata_match` |
| `pattern` | string | Yes | Regex (filename), pipe-separated keywords (content), or `field=value` (metadata) |
| `priority` | integer | No | Higher priority rules are evaluated first (default: 0) |
| `case_sensitive` | boolean | No | Case-sensitive matching (default: false) |
| `enabled` | boolean | No | Whether the rule is active (default: true) |
**Response (201):**
```json
{
"id": 1,
"owner_id": "user@example.com",
"name": "German Invoice Filename",
"category": "invoice",
"rule_type": "filename_pattern",
"pattern": "(?i)rechnung",
"priority": 10,
"case_sensitive": false,
"enabled": true
}
```
### Get Rule
```bash
GET /api/classification-rules/{rule_id}
```
### Update Rule
```bash
PUT /api/classification-rules/{rule_id}
```
**Request (partial update):**
```json
{
"priority": 20,
"enabled": false
}
```
### Delete Rule
```bash
DELETE /api/classification-rules/{rule_id}
```
**Response:** `204 No Content`
## Automation (Zapier / Make.com)
Manage automation hook subscriptions for integrating DocuElevate with external platforms like Zapier and Make.com. All endpoints require API token authentication (`Authorization: Bearer <token>`).
@@ -2366,6 +2652,8 @@ The `id` field is unique per event and is used by Zapier for deduplication. If a
Automation hook deliveries follow the same retry policy as regular webhooks: up to 3 retries with exponential backoff (60 s, 300 s, 900 s) and ±20% jitter.
## Further Assistance
## Further Assistance
For additional help with the API, please contact our support team or refer to the [Development Guide](../CONTRIBUTING.md).
@@ -2420,9 +2708,14 @@ List all registered push-notification devices for the current user.
### DELETE /api/mobile/devices/{device_id}
Deactivate a push-notification device. The device will no longer receive push notifications.
Deactivate or permanently delete a push-notification device:
**Response (204 No Content)**
* **Active device** – soft-deactivated (record kept, will no longer receive push notifications).
Response: `{"detail": "Device deactivated"}`
* **Already-inactive device** – permanently deleted from the database.
Response: `{"detail": "Device deleted"}`
**Response (200)**
### GET /api/mobile/whoami
@@ -2557,3 +2850,64 @@ query GetDocument($id: Int!) {
}
```
Variables: `{ "id": 42 }`
## System Reset
Admin-only endpoints for resetting the system to a clean state. Requires `ENABLE_FACTORY_RESET=True`.
### GET /api/admin/system-reset/status
Check whether the system reset feature is enabled.
**Response (200):**
```json
{
"enabled": true,
"factory_reset_on_startup": false
}
```
### POST /api/admin/system-reset/full
Wipe all user data (database + work-files).
**Request:**
```json
{
"confirmation": "DELETE"
}
```
**Response (200):**
```json
{
"status": "ok",
"result": {
"database": { "files": 42, "processing_logs": 100 },
"filesystem": { "deleted_dirs": 5, "deleted_files": 12 }
}
}
```
### POST /api/admin/system-reset/reimport
Move original files to a reimport folder, wipe everything, and configure the reimport folder as a watch folder for re-ingestion.
**Request:**
```json
{
"confirmation": "REIMPORT"
}
```
**Response (200):**
```json
{
"status": "ok",
"result": {
"database": { "files": 42 },
"filesystem": { "deleted_dirs": 5, "deleted_files": 12 },
"reimport": { "files_moved": 42, "reimport_folder": "/workdir/reimport" }
}
}
```
docs/AppleAppStoreCompliance.md
+285
View File
@@ -0,0 +1,285 @@
# Apple App Store Compliance Audit Report
This document details the findings from a comprehensive audit of the DocuElevate mobile app against Apple's App Store Review Guidelines, Human Interface Guidelines (HIG), and privacy requirements. It covers all areas of compliance, risks for rejection, and recommendations.
> **Last Audited:** March 2026
> **App Version:** 1.0.0
> **Expo SDK:** 54.0.0
> **Bundle ID:** `org.docuelevate.mobile`
---
## Executive Summary
The DocuElevate mobile app is broadly compliant with Apple's App Store requirements. The following issues were identified and resolved as part of this audit:
| Issue | Severity | Status |
|-------|----------|--------|
| Unused `fetch` background mode declared | High | ✅ Fixed |
| Missing privacy manifest for required reason APIs | High | ✅ Fixed |
| No account deletion option (Guideline 5.1.1(v)) | Critical | ✅ Fixed |
| No Privacy Policy / Terms of Service links in-app | High | ✅ Fixed |
| Emoji used as UI icons instead of platform-native icons | Medium | ✅ Fixed |
| Missing app version display | Low | ✅ Fixed |
| Unused `Switch` import in ProfileScreen | Low | ✅ Fixed |
---
## 1. Human Interface Guidelines (HIG)
### 1.1 Navigation & Tab Bar ✅
- The app uses a standard bottom tab bar with three tabs: Upload, Files, and Profile.
- Tab icons use **Ionicons** (an icon set that closely maps to Apple's SF Symbols).
- Active/inactive tab colors follow iOS conventions (`#1e40af` active, `#9ca3af` inactive).
- Header styling uses a solid color background with white text, consistent with iOS navigation bar patterns.
### 1.2 Icons & Visual Assets ✅
- **App icon:** Custom `icon.png` provided at root level; Expo handles generating all required sizes.
- **Splash screen:** Uses branded splash with `contain` resize mode and matching background color.
- **Adaptive icon (Android):** Properly configured with foreground image and background color.
- **Action buttons:** Previously used emoji characters (📷, 🖼️, 📄) which render inconsistently across iOS versions. **Fixed:** Now using Ionicons (`camera-outline`, `images-outline`, `document-outline`).
- **Status indicators:** Previously used emoji (✅, ❌, ⏳, ⚙️). **Fixed:** Now using Ionicons with semantic colors.
### 1.3 Typography & Colors ✅
- Uses system fonts (default React Native text rendering uses San Francisco on iOS).
- Color palette (`#1e40af` primary blue, semantic reds/greens/grays) provides sufficient contrast ratios.
- Text sizes follow iOS recommended minimums (body text ≥ 13pt).
### 1.4 Touch Targets ✅
- All interactive elements have `minHeight: 44` or `minHeight: 48` (meets Apple's 44×44pt minimum).
- Back links, cancel buttons, and retry buttons all meet minimum touch target requirements.
### 1.5 Safe Areas ✅
- The app uses `react-native-safe-area-context` (`SafeAreaProvider`) to respect device notches, Dynamic Island, and home indicator.
### 1.6 Dark Mode ✅
- `userInterfaceStyle: "automatic"` is set in `app.json`, enabling automatic dark mode support.
---
## 2. Privacy & Data Usage
### 2.1 Permission Descriptions ✅
All iOS permission strings (Info.plist keys) are present and provide clear, specific descriptions of why each permission is needed:
| Permission | Key | Description |
|-----------|-----|-------------|
| Camera | `NSCameraUsageDescription` | "DocuElevate uses the camera to scan QR codes for login and to capture documents for upload." |
| Photo Library (Read) | `NSPhotoLibraryUsageDescription` | "DocuElevate accesses your photo library to select documents for upload." |
| Photo Library (Write) | `NSPhotoLibraryAddUsageDescription` | "DocuElevate saves scanned documents to your photo library." |
**Assessment:** All descriptions clearly explain the purpose, which is a requirement for App Review approval.
### 2.2 Push Notifications ✅
- Push notification permission is requested at runtime (not at launch) when the user enters the authenticated area.
- The app works gracefully without push notifications if permission is denied.
- Device tokens are registered via a dedicated backend endpoint.
### 2.3 Background Modes ✅ (Fixed)
- **Previous state:** `UIBackgroundModes` included `["fetch", "remote-notification"]`.
- **Issue:** The app does not implement background fetch (`application:performFetchWithCompletionHandler:`). Apple may reject apps that declare background modes they don't actively use (Guideline 2.5.4).
- **Fix:** Removed `fetch` from `UIBackgroundModes`. Only `remote-notification` remains, which is required for push notification delivery.
### 2.4 Privacy Manifest ✅ (Fixed)
Starting in Spring 2024, Apple requires a privacy manifest (`PrivacyInfo.xcprivacy`) for apps using specific APIs. The following required reason APIs are used by the app's dependencies:
| API Category | Reason Code | Justification |
|-------------|-------------|---------------|
| `NSPrivacyAccessedAPICategoryUserDefaults` | `CA92.1` | Used by `@react-native-async-storage/async-storage` for user preferences |
| `NSPrivacyAccessedAPICategoryFileTimestamp` | `C617.1` | Used by `expo-file-system` to read file metadata |
| `NSPrivacyAccessedAPICategoryDiskSpace` | `E174.1` | Used by Expo runtime for storage space checks |
| `NSPrivacyAccessedAPICategorySystemBootTime` | `35F9.1` | Used by React Native's timing APIs |
The privacy manifest is configured via `expo-build-properties` plugin in `app.json`, which ensures it is included in the generated Xcode project during EAS Build.
### 2.5 Tracking & Analytics ✅
- `NSPrivacyTracking: false` — the app does **not** track users.
- `NSPrivacyCollectedDataTypes: []` — no data types are collected for tracking.
- No analytics SDKs (Firebase Analytics, Amplitude, Mixpanel, etc.) are included.
- No App Tracking Transparency (ATT) prompt is needed.
### 2.6 Encryption Declaration ✅
- `ITSAppUsesNonExemptEncryption: false` — the app uses only standard HTTPS/TLS for network communication, which is exempt from export compliance requirements.
### 2.7 Data Storage Security ✅
- API tokens are stored in the device keychain via `expo-secure-store` (uses iOS Keychain Services).
- No sensitive data is stored in `AsyncStorage` or `UserDefaults`.
- Server URL is stored in secure storage, not in plain text files.
---
## 3. App Store Review Guidelines Compliance
### 3.1 Functionality (Guideline 2.x) ✅
- **2.1 App Completeness:** The app provides a complete, functional experience. All advertised features (camera capture, file upload, document list, push notifications) work as described.
- **2.3 Accurate Metadata:** App name ("DocuElevate"), description, and screenshots should accurately reflect the app's functionality.
- **2.5.4 Background Modes:** Only `remote-notification` is declared, which is actively used. ✅ Fixed.
### 3.2 Content & Intellectual Property (Guideline 3.x) ✅
- No third-party trademarked content is used.
- The app does not display user-generated content publicly (documents are private to each user).
- No copyrighted content is bundled with the app.
### 3.3 Business (Guideline 3.1.x) ✅
- The app does not include in-app purchases, subscriptions, or payment processing.
- No physical goods or services are sold through the app.
- Authentication is handled via self-hosted or enterprise SSO — no Apple Sign-In requirement applies (Apple Sign-In is required only when third-party social login options like Google/Facebook are offered as the primary login method; enterprise SSO to a self-hosted server is exempt).
### 3.4 Safety & Privacy (Guideline 5.x) ✅
- **5.1.1 Data Collection and Storage:** The app collects only what is necessary for its functionality (server URL, auth token, push token).
- **5.1.1(v) Account Deletion:** ✅ Fixed. Users can now initiate account deletion from the Profile screen, which opens the server's account deletion page in the browser.
- **5.1.2 Data Use and Sharing:** No data is shared with third parties or used for advertising.
### 3.5 Privacy Policy ✅ (Fixed)
- **Requirement:** Apple requires all apps to have an accessible privacy policy.
- **Fix:** Privacy Policy and Terms of Service links are now accessible from the Profile screen, opening the server's hosted policy pages.
- **App Store Connect:** The privacy policy URL must also be provided in App Store Connect during submission.
### 3.6 Login & Authentication ✅
- Two login methods are available: SSO (browser-based OAuth) and QR code scanning.
- Both methods provide clear error messages on failure.
- The app correctly handles authentication cancellation.
- Session restoration on app launch is implemented.
- **Demo Account:** For App Review, a demo account may need to be provided in App Store Connect's review notes. Ensure the review team can access a test server.
---
## 4. Technical Compliance
### 4.1 API Usage ✅
- No private APIs are used (all functionality comes from Expo SDK and React Native public APIs).
- No deprecated APIs are used that would trigger rejection.
### 4.2 Network Security ✅
- The app validates server URLs require `http://` or `https://` scheme.
- All API calls use Bearer token authentication over HTTPS.
- App Transport Security (ATS) is not explicitly disabled — default iOS ATS rules apply.
### 4.3 Deep Linking ✅
- Custom URL scheme `docuelevate://` is properly registered.
- Deep link handling for QR login (`docuelevate://qr-login`) and file sharing is implemented correctly.
- `WebBrowser.openAuthSessionAsync` is used for OAuth, which properly handles the authentication session lifecycle.
### 4.4 Document Handling ✅
- `CFBundleDocumentTypes` properly declares supported file types.
- `LSSupportsOpeningDocumentsInPlace: false` ensures iOS copies shared files to the app's accessible Inbox directory, avoiding security-scoped URL issues.
- The `+not-found.tsx` handler correctly intercepts iOS "Open In…" file paths.
- `UploadScreen` uses `expo-file-system` to copy external files to cache before uploading for reliable file access.
### 4.5 Crash Resistance ✅
- All network calls are wrapped in try/catch blocks.
- Error states are displayed to users with actionable recovery options (retry buttons).
- Permission denials are handled gracefully with explanatory messages.
---
## 5. Onboarding & First-Run Experience
### 5.1 Welcome Screen ✅
- Clean, informative welcome screen with app branding and feature highlights.
- Clear "Get Started" call-to-action leading to the login screen.
- No misleading claims or functionality promises.
### 5.2 Login Flow ✅
- Server URL entry with input validation.
- Two clear authentication options (SSO and QR code).
- Error handling with user-friendly alert dialogs.
- Back navigation available from all auth screens.
### 5.3 First-Run Permissions ✅
- Camera permission is requested at the point of use (when tapping Camera button), not at launch.
- Photo library permission is requested at the point of use.
- Push notification permission is requested after authentication, not before.
- All permission requests include clear usage descriptions.
---
## 6. Remaining Recommendations
### 6.1 App Store Connect Preparation
Before submission, ensure the following are configured in App Store Connect:
- [ ] **Privacy Policy URL** — must point to the server's `/privacy` endpoint
- [ ] **App Store description** — accurate description of features
- [ ] **Screenshots** — for iPhone and iPad (since `supportsTablet: true`)
- [ ] **App category** — "Business" or "Productivity"
- [ ] **Age rating** — complete the questionnaire (likely 4+)
- [ ] **Review notes** — provide demo server URL and test credentials for the Apple review team
- [ ] **Privacy Nutrition Labels** — declare data types collected (device ID for push notifications, authentication tokens)
### 6.2 Accessibility Enhancements (Recommended)
While the app includes `accessibilityRole` and `accessibilityLabel` on interactive elements, consider:
- Adding `accessibilityHint` to buttons where the action isn't immediately obvious.
- Testing with VoiceOver to ensure all screens are fully navigable.
- Ensuring all status changes are announced to screen readers.
### 6.3 iPad Support
The app declares `supportsTablet: true`. Ensure:
- UI scales appropriately on iPad screen sizes.
- Split View and Slide Over multitasking work correctly.
- Touch targets remain accessible on larger screens.
### 6.4 Localization (Future Enhancement)
- The app currently uses English-only strings.
- For broader App Store reach, consider localizing the app name, description, and in-app strings.
---
## 7. Compliance Checklist Summary
| Area | Status | Notes |
|------|--------|-------|
| Human Interface Guidelines | ✅ Pass | Ionicons used for platform-consistent iconography |
| App Icons & Visual Assets | ✅ Pass | All required assets provided |
| Device Data Usage | ✅ Pass | Camera, photos, notifications properly handled |
| Privacy Disclosures | ✅ Pass | Info.plist keys and privacy manifest configured |
| Background Modes | ✅ Pass | Only `remote-notification` declared |
| Restricted APIs | ✅ Pass | No private or deprecated APIs used |
| Content Standards | ✅ Pass | No misleading or inappropriate content |
| Functionality | ✅ Pass | Complete, functional app experience |
| Business Model | ✅ Pass | No IAP conflicts |
| Safety & Privacy | ✅ Pass | Account deletion available, privacy policy linked |
| Onboarding | ✅ Pass | Clear, permission-respectful first-run experience |
| Privacy Manifest | ✅ Pass | Required reason APIs declared |
---
## References
- [Apple App Store Review Guidelines](https://developer.apple.com/app-store/review/guidelines/)
- [Apple Human Interface Guidelines](https://developer.apple.com/design/human-interface-guidelines/)
- [Apple Privacy Manifest Requirements](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files)
- [App Store Connect Help](https://developer.apple.com/help/app-store-connect/)
docs/AuthenticationSetup.md
+56
View File
@@ -154,6 +154,62 @@ DocuElevate can work with any OpenID Connect-compliant provider, not just Authen
OAUTH_PROVIDER_NAME=Auth0
```
## Server-Side Session Management
DocuElevate supports server-side session tracking. Every login creates a `UserSession` record that can be listed and revoked individually or all at once ("log off everywhere").
### Configuration
| Variable | Description | Default |
|----------|-------------|---------|
| `SESSION_LIFETIME_DAYS` | Number of days before a session expires | `30` |
| `SESSION_LIFETIME_CUSTOM_DAYS` | Override for `SESSION_LIFETIME_DAYS` when set | — |
### Managing Sessions
Users can manage their active sessions from the **Profile → Security** section:
- **View active sessions** — see browser, device, IP address, and last activity for each session.
- **Revoke a single session** — immediately invalidate one session.
- **Log off everywhere** — revoke all sessions (optionally keeping the current one) and all API tokens at once.
Expired sessions are automatically cleaned up by a periodic background task.
### API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/sessions` | List the current user's active sessions |
| `DELETE` | `/api/sessions/{id}` | Revoke a single session |
| `POST` | `/api/sessions/revoke-all` | Revoke all sessions for the current user |
## QR Code Login
QR code login allows users to authenticate a mobile device by scanning a QR code displayed in the web UI, without manually entering credentials on the phone.
### How It Works
1. The authenticated web user opens the **QR Login** page and a challenge QR code is displayed.
2. The user opens the DocuElevate mobile app and taps **Scan QR Code to Login**, which opens the device camera.
3. The mobile app scans the QR code. The QR code contains both the challenge token and the server URL (`docuelevate://qr-login?token=...&server=...`), so there is no need to enter the server URL manually.
4. An API token is issued for the mobile device and the web UI is notified via polling.
> **Note:** The countdown timer on the web page uses server-relative time (TTL in seconds) rather than absolute timestamps, so it works correctly even when the client's clock is not in sync with the server.
### Configuration
| Variable | Description | Default |
|----------|-------------|---------|
| `QR_LOGIN_CHALLENGE_TTL_SECONDS` | How long a QR challenge is valid (seconds) | `120` |
### API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/qr-auth/challenge` | Create a new QR login challenge (returns `ttl_seconds` for client countdown) |
| `GET` | `/api/qr-auth/challenge/{id}/status` | Poll the status of a challenge |
| `POST` | `/api/qr-auth/claim` | Claim a challenge from a mobile device |
## Security Considerations
1. **Always use HTTPS** in production to protect authentication tokens and passwords
docs/ConfigurationGuide.md
+120 -2
View File
@@ -11,12 +11,19 @@ Configuration is primarily done through environment variables specified in a `.e
| **Variable** | **Description** | **Example** |
|------------------------|----------------------------------------------------------|--------------------------------|
| `DATABASE_URL` | Path/URL to the SQLite database (or other SQL backend). Use the [Database Wizard](/database-wizard) for guided setup. See [Database Configuration](DatabaseConfiguration.md). | `sqlite:///./app/database.db` |
| `DB_POOL_SIZE` | Number of persistent connections in the pool per worker (PostgreSQL/MySQL only; ignored for SQLite). | `10` |
| `DB_MAX_OVERFLOW` | Additional connections beyond `DB_POOL_SIZE` under burst load (PostgreSQL/MySQL only). | `20` |
| `DB_POOL_TIMEOUT` | Seconds to wait for a pool connection before raising `TimeoutError` (PostgreSQL/MySQL only). | `30` |
| `DB_POOL_RECYCLE` | Recycle connections after this many seconds to avoid stale connections (PostgreSQL/MySQL only). | `1800` |
| `REDIS_URL` | URL for Redis, used by Celery for broker & result store. | `redis://redis:6379/0` |
| `WORKDIR` | Working directory for the application. | `/workdir` |
| `GOTENBERG_URL` | Gotenberg PDF processing URL. | `http://gotenberg:3000` |
| `EXTERNAL_HOSTNAME` | The external hostname for the application. | `docuelevate.example.com` |
| `PUBLIC_BASE_URL` | Full public base URL including scheme (e.g., `https://docuelevate.example.com`). When set, overrides auto-detected URLs used for OAuth redirect URIs. **Required when your reverse proxy does not forward `X-Forwarded-Proto` headers.** | *(not set)* |
| `ALLOW_FILE_DELETE` | Enable file deletion in the web interface (`true`/`false`). | `true` |
| `COMPLIANCE_ENABLED` | Enable the compliance templates dashboard (GDPR, HIPAA, SOC 2). | `true` |
| `FACTORY_RESET_ON_STARTUP` | Wipe all user data on every startup (demo/testing). | `false` |
| `ENABLE_FACTORY_RESET` | Show the System Reset page in the admin UI. | `false` |
### Batch Processing Settings
@@ -79,6 +86,28 @@ Control how the web UI queues and paces file uploads to avoid overwhelming the b
**Example**: With `UPLOAD_CONCURRENCY=3` and `UPLOAD_QUEUE_DELAY_MS=500`, a directory of 5,000 files is uploaded ≈ 3 at a time with 500 ms pacing – the backend processes files at its own rate while the queue drains in the background without triggering API rate limits.
### Per-User Upload Rate Limiting
Server-side rate limiting that prevents any single user from overwhelming the system with bulk uploads. The limiter uses a Redis-backed sliding window and dynamically adjusts limits based on system health.
| **Variable** | **Description** | **Default** |
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------|-------------|
| `UPLOAD_RATE_LIMIT_PER_USER` | Maximum uploads allowed per user within the sliding window. Effective limit may be reduced under load. | `20` |
| `UPLOAD_RATE_LIMIT_WINDOW` | Sliding window size in seconds. | `60` |
**Health-aware dynamic limiting**: The effective per-user limit is automatically reduced when the system is under heavy load:
| **System condition** | **Effective limit** | **Trigger** |
|--------------------------------|---------------------|--------------------------------|
| Normal | 100 % of base | Queue < 50, CPU load normal |
| Moderate load | 50 % of base | Queue 50–100 or CPU > 1.5× |
| High load | 25 % of base | Queue 100–200 or CPU > 2× |
| Critical load | 10 % of base | Queue > 200 or CPU > 3× |
When a user exceeds the limit, the server returns **HTTP 429 Too Many Requests** with a `Retry-After` header. The browser client (see *Client-Side Upload Throttling* above) automatically pauses and retries.
> **Note**: The limiter fails open — if Redis is unavailable, all uploads are allowed through so that a monitoring outage never blocks document processing.
### File Upload Size Limits
**Security Feature**: Control file upload sizes to prevent resource exhaustion attacks. See [SECURITY_AUDIT.md](../SECURITY_AUDIT.md#5-file-upload-size-limits) for security details.
@@ -364,6 +393,9 @@ Credentials are encrypted at rest using Fernet encryption.
|-------------------------|---------------------------------------------------------------|
| `AUTH_ENABLED` | Enable or disable authentication (`true`/`false`). |
| `SESSION_SECRET` | Secret key used to encrypt sessions and cookies (at least 32 chars). |
| `SESSION_LIFETIME_DAYS` | Number of days before a server-side session expires. Default: `30`. |
| `SESSION_LIFETIME_CUSTOM_DAYS` | Override for `SESSION_LIFETIME_DAYS` when set. |
| `QR_LOGIN_CHALLENGE_TTL_SECONDS` | How long a QR login challenge is valid (seconds). Default: `120`. |
| `ADMIN_USERNAME` | Username for basic authentication (when not using OIDC). |
| `ADMIN_PASSWORD` | Password for basic authentication (when not using OIDC). |
| `ADMIN_GROUP_NAME` | Group name in OIDC claims that grants admin access. Default: `admin`. |
@@ -1284,6 +1316,20 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
For detailed setup instructions, see the [OneDrive Setup Guide](OneDriveSetup.md).
### SharePoint Online
| **Variable** | **Description** |
|---------------------------------|-------------------------------------------------------|
| `SHAREPOINT_CLIENT_ID` | Azure AD application client ID |
| `SHAREPOINT_CLIENT_SECRET` | Azure AD application client secret |
| `SHAREPOINT_TENANT_ID` | Azure AD tenant ID (use "common" for multi-tenant apps) |
| `SHAREPOINT_REFRESH_TOKEN` | OAuth 2.0 refresh token |
| `SHAREPOINT_SITE_URL` | SharePoint site URL (e.g. `https://tenant.sharepoint.com/sites/sitename`) |
| `SHAREPOINT_DOCUMENT_LIBRARY` | Document library name (default: `Documents`) |
| `SHAREPOINT_FOLDER_PATH` | Subfolder path inside the document library |
SharePoint uses the same Microsoft Graph API as OneDrive. See the [OneDrive Setup Guide](OneDriveSetup.md) for Azure AD app registration instructions — the same app registration can be reused for SharePoint with the `Sites.ReadWrite.All` permission.
### Amazon S3
| **Variable** | **Description** |
@@ -1556,14 +1602,30 @@ DocuElevate detects and flags documents that share the same content, even if the
### Exact Duplicate Detection (SHA-256)
When `ENABLE_DEDUPLICATION=True` (the default), each new document is hashed with SHA-256 before processing begins. If the hash matches an existing file record the new document is stored as a duplicate (`is_duplicate=True`, `duplicate_of_id=<original_id>`) and no further processing is performed.
When `ENABLE_DEDUPLICATION=True` (the default), each new document is hashed with SHA-256 before processing begins. If the hash matches an existing file record the upload is rejected immediately — no processing task is created, and the temporary file is removed from disk. The `/api/ui-upload` response returns `"status": "duplicate"` together with a `duplicate_of` object that identifies the original file.
If the same file somehow reaches the Celery worker (e.g. via a watch-folder ingest) it is still caught there and stored as a duplicate (`is_duplicate=True`, `duplicate_of_id=<original_id>`) with no further processing.
| Variable | Description | Default |
|---|---|---|
| `ENABLE_DEDUPLICATION` | Hash-based exact duplicate detection on ingest. | `True` |
| `SHOW_DEDUPLICATION_STEP` | Show the "Check for Duplicates" step in the processing timeline UI. | `True` |
An immediate duplicate warning is also included in the `/api/ui-upload` JSON response so the frontend can alert the user before the pipeline completes.
When the upload is an exact duplicate the `/api/ui-upload` response looks like:
```json
{
"status": "duplicate",
"original_filename": "invoice.pdf",
"stored_filename": "abc-123.pdf",
"duplicate_of": {
"duplicate_type": "exact",
"original_file_id": 42,
"original_filename": "invoice.pdf",
"message": "This file is an exact duplicate of an already-processed document. It has not been queued for processing again."
}
}
```
### Near-Duplicate Detection (Content Similarity)
@@ -1643,6 +1705,7 @@ For example:
| S3 | `docs/uploads/` | `docs/uploads/pdfa/` |
| Nextcloud | `/Files` | `/Files/pdfa` |
| OneDrive | `Documents/Uploads` | `Documents/Uploads/pdfa` |
| SharePoint | `Uploads` | `Uploads/pdfa` |
| Google Drive | *(folder ID)* | `GOOGLE_DRIVE_PDFA_FOLDER_ID` |
Set `PDFA_UPLOAD_FOLDER` to an empty string to upload PDF/A files into the
@@ -1849,6 +1912,15 @@ ONEDRIVE_TENANT_ID=common
ONEDRIVE_REFRESH_TOKEN=your_refresh_token
ONEDRIVE_FOLDER_PATH=Documents/Uploads
# SharePoint Online
SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
SHAREPOINT_CLIENT_SECRET=your_client_secret
SHAREPOINT_TENANT_ID=your-tenant-id
SHAREPOINT_REFRESH_TOKEN=your_refresh_token
SHAREPOINT_SITE_URL=https://tenant.sharepoint.com/sites/sitename
SHAREPOINT_DOCUMENT_LIBRARY=Documents
SHAREPOINT_FOLDER_PATH=Uploads
# Amazon S3
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
@@ -1876,6 +1948,52 @@ BACKUP_RETAIN_WEEKLY=13
You can choose which document storage services to use by only including the relevant environment variables. For example, if you only want to use Dropbox, include only the Dropbox variables and omit the Paperless NGX and Nextcloud variables.
## System Reset / Factory Reset
DocuElevate provides two mechanisms for resetting the system to a clean state. Both are **disabled by default** and must be explicitly enabled.
### Automatic Reset on Startup
Set `FACTORY_RESET_ON_STARTUP=true` to wipe all user data (database rows and work-files) every time the application starts. This is useful for demo, testing, or ephemeral environments where you always want a fresh instance.
```dotenv
FACTORY_RESET_ON_STARTUP=true
```
> **Warning:** This destroys all documents, processing history, audit logs, and backups on every restart. Application settings and configuration are preserved.
### Admin UI Reset Page
Set `ENABLE_FACTORY_RESET=true` to display the **System Reset** page in the admin navigation menu. From this page, administrators can:
| Action | Confirmation | Description |
|--------|-------------|-------------|
| **Full Reset** | Type `DELETE` | Wipes all database rows and work-files. The system returns to its initial state. |
| **Reset & Re-import** | Type `REIMPORT` | Copies original files to a `reimport/` folder inside the workdir, wipes everything, then configures the reimport folder as a watch folder so files are automatically re-ingested with the same processing pipeline, rate limits, and backoff strategy as regular uploads. |
```dotenv
ENABLE_FACTORY_RESET=true
```
### API Endpoints
When `ENABLE_FACTORY_RESET=true`, two admin-only API endpoints are available:
- `POST /api/admin/system-reset/full` — body: `{"confirmation": "DELETE"}`
- `POST /api/admin/system-reset/reimport` — body: `{"confirmation": "REIMPORT"}`
- `GET /api/admin/system-reset/status` — returns current feature-flag state
### What Gets Deleted
| Deleted | Preserved |
|---------|-----------|
| All document records (`files` table) | Application settings (`application_settings` table) |
| Processing logs and steps | User accounts and profiles |
| Audit logs | Subscription plans |
| Backup records | Pipelines and scheduled jobs |
| Original, processed, and temporary files | The workdir directory itself |
| Watch-folder caches and ingestion state | OAuth and integration configuration |
## Configuration File Location
The `.env` file should be placed at the root of the project directory. When using Docker Compose, you can reference it with the `env_file` directive in your `docker-compose.yml`.
docs/CredentialRotationGuide.md
+10 -1
View File
@@ -11,7 +11,7 @@ Credentials fall into two categories:
| Category | Examples |
|---|---|
| **API keys** | OpenAI API key, Azure AI key, Paperless-ngx API token, AWS access keys |
| **OAuth tokens / secrets** | Dropbox, Google Drive, OneDrive, Authentik client secrets and refresh tokens |
| **OAuth tokens / secrets** | Dropbox, Google Drive, OneDrive, SharePoint, Authentik client secrets and refresh tokens |
| **Passwords** | Admin password, Nextcloud, Email (SMTP), IMAP, FTP, SFTP, WebDAV |
| **Private keys** | SFTP private key and passphrase |
@@ -119,6 +119,15 @@ For service-account credentials (`google_drive_credentials_json`):
4. Re-authorize via the OAuth flow to get a fresh `onedrive_refresh_token`.
5. Delete the old client secret in Azure.
### SharePoint (Microsoft OAuth)
1. SharePoint uses the same Azure AD app registration as OneDrive.
2. In **Azure App Registrations**, navigate to **Certificates & secrets** for your app.
3. Add a new client secret.
4. Update `sharepoint_client_secret` in DocuElevate.
5. Re-authorize via the OAuth flow to get a fresh `sharepoint_refresh_token`.
6. Delete the old client secret in Azure.
### Authentik (OIDC)
1. In your Authentik admin panel, navigate to the DocuElevate application and regenerate the client secret.
docs/DatabaseConfiguration.md
+46 -8
View File
@@ -287,6 +287,27 @@ alembic revision --autogenerate -m "describe your change"
Review the generated file in `migrations/versions/` before applying it.
> **Tip:** For detailed guidance on naming conventions, idempotent patterns, parallel-branch workflows, and resolving merge conflicts, see the [Migration Workflow Guide](MigrationWorkflow.md).
### Validating the Migration Chain
A CI check and pre-commit hook validate that the migration chain has no broken
references, duplicate revisions, or diverged heads. Run the check locally:
```bash
python scripts/check_alembic_migrations.py
python scripts/check_alembic_migrations.py --verbose # extra detail
```
If you see **"Multiple migration heads detected"**, two branches added
migrations from the same parent. Create a merge migration:
```bash
alembic merge heads -m "merge_parallel_branches"
```
For a complete walk-through, see the [Migration Workflow Guide](MigrationWorkflow.md).
### Automating Migrations in Docker Compose
Add a short-lived `migrate` service that runs before the API and Worker:
@@ -316,18 +337,26 @@ The Helm chart includes a pre-install and pre-upgrade Job hook that runs `alembi
## Connection Pooling
SQLAlchemy manages a connection pool automatically. The defaults are suitable for most deployments. For high-concurrency or Kubernetes deployments you may want to tune:
SQLAlchemy manages a connection pool automatically. DocuElevate selects the pool
strategy based on the database backend:
- **SQLite** — uses `NullPool` (a fresh connection per request, closed immediately).
This avoids the `QueuePool limit reached` `TimeoutError` that can occur under
concurrent load because SQLite does not benefit from persistent connection pooling.
- **PostgreSQL / MySQL** — uses a bounded `QueuePool` whose size is configurable
via environment variables.
```bash
# Optional — these are set via environment variables if you extend app/database.py
# Typical production values:
DB_POOL_SIZE=10 # Number of persistent connections per worker
DB_MAX_OVERFLOW=20 # Additional connections allowed beyond pool_size
DB_POOL_TIMEOUT=30 # Seconds to wait for a connection from the pool
DB_POOL_RECYCLE=1800 # Recycle connections after 30 minutes (avoids stale connections)
# Tune these for PostgreSQL / MySQL (ignored when using SQLite):
DB_POOL_SIZE=10 # Number of persistent connections per worker (default: 10)
DB_MAX_OVERFLOW=20 # Additional connections allowed beyond pool_size (default: 20)
DB_POOL_TIMEOUT=30 # Seconds to wait for a connection from the pool (default: 30)
DB_POOL_RECYCLE=1800 # Recycle connections after 30 minutes (default: 1800)
```
> **Note:** These environment variables are not exposed in the default `app/config.py`. If you need to tune them, extend the database engine creation in `app/database.py`.
All backends also enable `pool_pre_ping`, which sends a lightweight health-check
before each connection is handed out. This detects stale or dropped connections
and transparently reconnects.
For **PgBouncer** (external connection pooling), point `DATABASE_URL` at your PgBouncer instance and use transaction-mode pooling:
@@ -491,4 +520,13 @@ Then retry `alembic upgrade head`.
Either increase `max_connections` in `postgresql.conf` or add PgBouncer in front of PostgreSQL. The default PostgreSQL `max_connections` is `100`; reduce `DB_POOL_SIZE` per worker to stay within this limit.
### "QueuePool limit reached" TimeoutError (SQLite)
If you see `TimeoutError: QueuePool limit of size 5 overflow 10 reached`, your
deployment is still running an older version of DocuElevate that used a bounded
connection pool for SQLite. Upgrade to the latest release — SQLite now uses
`NullPool`, which eliminates this error entirely. If you are already on the
latest version and are still seeing pool exhaustion, ensure you are not
overriding the engine creation manually.
For more help, see the [Troubleshooting Guide](Troubleshooting.md).
docs/DeploymentGuide.md
+11 -7
View File
@@ -19,7 +19,7 @@ This guide covers all supported deployment methods for DocuElevate.
- Access to required external services (if configured):
- AI provider API key (OpenAI, Anthropic, Gemini, or other configured provider)
- Azure Document Intelligence
- Dropbox, Google Drive, OneDrive, S3, or other storage APIs
- Dropbox, Google Drive, OneDrive, SharePoint, S3, or other storage APIs
- SMTP / IMAP server (for email processing)
- Notification services (Discord, Telegram, etc.)
@@ -349,16 +349,18 @@ workdir:
## Scaling
DocuElevate is designed for horizontal scaling. Both API and worker pods are stateless and can be scaled independently.
### Docker Compose
Add more worker containers:
Scale workers (task processing) and API pods (request handling) independently:
```yaml
worker:
deploy:
replicas: 3
```bash
docker compose up -d --scale worker=3 --scale api=2
```
> **Note:** The `beat` service (Celery Beat scheduler) must always run as exactly **one** instance. Do not scale it. It publishes periodic tasks to the Redis broker; workers pick them up.
### Kubernetes / Helm
Enable HPA:
@@ -377,13 +379,15 @@ worker:
maxReplicas: 10
```
The Helm chart deploys a separate **beat** pod (always 1 replica, `Recreate` strategy) so that scheduled tasks are never duplicated when workers scale.
---
## Monitoring
- **Docker Compose**: `docker-compose logs -f`, `docker stats`
- **Kubernetes**: `kubectl logs -l app.kubernetes.io/component=api -f`
- **Prometheus / Grafana**: Scrape the `/api/health` endpoint for readiness; add custom metrics as needed.
- **Prometheus / Grafana**: Scrape the `/api/diagnostic/healthz/ready` endpoint for readiness; add custom metrics as needed.
- **Uptime Kuma**: Set `UPTIME_KUMA_URL` to your push URL for heartbeat monitoring.
---
docs/DropboxSetup.md
+28 -3
View File
@@ -28,9 +28,10 @@ End users authorize their own Dropbox integration from the **Integrations** dash
1. Navigate to `/integrations` and click **+ Add Destination** (or **+ Add Source** for Watch Folder).
2. Create a Dropbox destination integration (or a Watch Folder with `source_type = dropbox`).
3. Click the **Authorize** button next to the integration — it links directly to the OAuth wizard pre-loaded with your integration's configuration.
4. Enter your Dropbox App Key and App Secret in the wizard (or use the global admin credentials if pre-configured).
4. If the administrator has configured system-wide Dropbox app credentials (`DROPBOX_APP_KEY` / `DROPBOX_APP_SECRET`), the wizard defaults to using them — no need to register your own Dropbox app. Uncheck the toggle to use custom credentials if needed.
5. Click **Start Authentication Flow**, authorize access in Dropbox, and the refresh token is automatically saved to your personal integration record.
6. The page redirects back to `/integrations` on success. Re-authorization is available at any time via the **Re-Authorize** button.
6. After authorization, an interactive **folder browser** lets you select the target folder directly from your Dropbox — no need to manually type folder paths.
7. The page redirects to `/integrations` on success. Re-authorization is available at any time via the **Re-Authorize** button.
> **Note:** Your credentials are stored encrypted per-integration and are never mixed with other users' data. Each user can have multiple Dropbox integrations with independent tokens.
@@ -128,7 +129,31 @@ If you encounter issues with Dropbox integration:
1. **Authentication Errors**: Make sure your App Key and App Secret are correct
2. **Token Expired**: Click "Refresh Token" button on the setup page to obtain a new token
3. **Folder Permissions**: Ensure your app has the correct permissions enabled for file operations
4. **Invalid Redirect URI**: Verify that the redirect URI in your app settings matches the one used in the authentication flow
4. **Invalid Redirect URI**: See section below for the most common cause and fix.
5. **Rate Limiting**: Dropbox API has rate limits; if exceeded, wait and try again
### Fixing "Invalid redirect_uri" Error
This error appears on the Dropbox authorization page when the redirect URI in the OAuth request does not match any URI registered in your Dropbox app console.
**Most common cause**: The application is deployed behind a reverse proxy (Traefik, Nginx, Caddy) that does **not** forward the `X-Forwarded-Proto: https` header to DocuElevate. Without this header, the server cannot determine that it is being accessed over HTTPS and may construct an `http://` redirect URI, while the registered URI in Dropbox is `https://`.
**Fix**:
Option 1 – Configure your proxy to forward `X-Forwarded-Proto`:
```nginx
proxy_set_header X-Forwarded-Proto $scheme;
```
Option 2 – Set `PUBLIC_BASE_URL` in your environment (recommended for most deployments):
```bash
PUBLIC_BASE_URL=https://docuelevate.example.com
```
When `PUBLIC_BASE_URL` is set, DocuElevate uses it directly for all OAuth redirect URIs instead of trying to infer the scheme from request headers. This is the most reliable option.
After setting `PUBLIC_BASE_URL`, ensure the Dropbox app console redirect URI matches exactly (e.g., `https://docuelevate.example.com/dropbox-callback`). The setup wizard at `/dropbox-setup` will show you the exact URI to register.
For more general configuration issues, see the [Configuration Troubleshooting Guide](ConfigurationTroubleshooting.md).
docs/GoogleDriveSetup.md
+1 -1
View File
@@ -30,7 +30,7 @@ End users can authorize their own Google Drive integration directly from the **I
1. Navigate to `/integrations` and click **+ Add Destination** (or **+ Add Source** for Watch Folder).
2. Create a Google Drive destination integration (or a Watch Folder with `source_type = google_drive`).
3. Click the **Authorize** button — it opens the OAuth wizard pre-loaded with your integration's configuration.
4. Enter your Google OAuth Client ID and Client Secret in the wizard.
4. If the administrator has configured system-wide Google Drive app credentials (`GOOGLE_DRIVE_CLIENT_ID` / `GOOGLE_DRIVE_CLIENT_SECRET`), the wizard defaults to using them — no need to register your own Google Cloud app. Uncheck the toggle to use custom credentials if needed.
5. Click **Start Authentication Flow** and authorize access in Google.
6. Credentials are saved automatically to your personal integration record; the page redirects back to `/integrations`.
7. Re-authorization is available at any time via the **Re-Authorize** button.
docs/KubernetesDeployment.md
+13 -5
View File
@@ -373,6 +373,8 @@ worker:
replicaCount: 4
```
> **Beat scheduler:** The Helm chart deploys a dedicated `beat` pod (always exactly 1 replica with `Recreate` strategy) that publishes periodic tasks to the Redis broker. Workers consume these tasks — scaling workers does **not** duplicate scheduled jobs.
### Horizontal Pod Autoscaler
```yaml
@@ -433,24 +435,30 @@ externalRedis:
### Kubernetes Probes
The Helm chart configures liveness and readiness probes on the API pods via `/api/health`. Default settings:
The Helm chart configures **unauthenticated** liveness and readiness probes on the API pods so kubelet can reach them without credentials. Default settings:
```yaml
api:
livenessProbe:
httpGet:
path: /api/health
path: /api/diagnostic/healthz/live
port: 8000
initialDelaySeconds: 30
periodSeconds: 30
periodSeconds: 20
readinessProbe:
httpGet:
path: /api/health
path: /api/diagnostic/healthz/ready
port: 8000
initialDelaySeconds: 10
initialDelaySeconds: 15
periodSeconds: 10
```
| Endpoint | Auth | Purpose |
|----------|------|---------|
| `/api/diagnostic/healthz/live` | None | Lightweight liveness check — returns 200 if the process is running |
| `/api/diagnostic/healthz/ready` | None | Readiness check — verifies database and Redis connectivity (503 when DB is down) |
| `/api/diagnostic/health` | Required | Full health status for monitoring dashboards (Grafana, Uptime Kuma) |
### Prometheus Scraping
Add annotations to expose metrics (if using a Prometheus-compatible exporter):
docs/MigrationWorkflow.md
+360
View File
@@ -0,0 +1,360 @@
# Migration Workflow
This guide explains how to create, test, and merge Alembic database migrations in DocuElevate — especially when **multiple feature branches** add migrations in parallel.
## Table of Contents
- [Quick Reference](#quick-reference)
- [Creating a New Migration](#creating-a-new-migration)
- [Migration Naming Convention](#migration-naming-convention)
- [Idempotent Migration Patterns](#idempotent-migration-patterns)
- [Parallel Branch Development](#parallel-branch-development)
- [Resolving Migration Conflicts](#resolving-migration-conflicts)
- [CI Validation](#ci-validation)
- [Pre-commit Hook](#pre-commit-hook)
- [Troubleshooting](#troubleshooting)
---
## Quick Reference
```bash
# Create a new migration after editing app/models.py
alembic revision --autogenerate -m "add_foobar_column"
# Apply all pending migrations
alembic upgrade head
# Check current database version
alembic current
# View migration history
alembic history --verbose
# Detect multiple heads (diverged branches)
alembic heads
# Create a merge migration to resolve multiple heads
alembic merge heads -m "merge_parallel_branches"
# Validate migration chain integrity (CI script)
python scripts/check_alembic_migrations.py
python scripts/check_alembic_migrations.py --verbose
```
---
## Creating a New Migration
1. **Edit `app/models.py`** — add or modify SQLAlchemy model classes.
2. **Generate the migration** from the repo root. Use `--rev-id` to set the
revision identifier directly (avoids renaming afterwards):
```bash
alembic revision --autogenerate --rev-id 037_add_my_new_table -m "add my new table"
```
This creates `migrations/versions/037_add_my_new_table_add_my_new_table.py`
with `revision = "037_add_my_new_table"`. Rename the file to match:
```bash
mv migrations/versions/037_add_my_new_table_add_my_new_table.py \
migrations/versions/037_add_my_new_table.py
```
Alternatively, generate with the default hash and then rename:
```bash
alembic revision --autogenerate -m "add_my_new_table"
# Rename: mv migrations/versions/<hash>_add_my_new_table.py migrations/versions/037_add_my_new_table.py
# Update revision inside the file to match the filename stem.
```
Alembic uses the `migrations/script.py.mako` template to generate the file. The template includes inline comments about idempotent patterns — read them.
3. **Review the generated code** — autogenerate is helpful but not perfect. Check:
- Are new tables and columns detected correctly?
- Does the `downgrade()` reverse all changes?
- Are SQLite-incompatible operations wrapped in `batch_alter_table()`?
4. **Test the migration** against a fresh database:
```bash
# Apply
alembic upgrade head
# Rollback
alembic downgrade -1
# Re-apply
alembic upgrade head
```
5. **Run the chain validation**:
```bash
python scripts/check_alembic_migrations.py
```
---
## Migration Naming Convention
All migration files follow a **sequential numeric prefix** scheme:
```
NNN_short_description.py
```
| Component | Rule |
|-----------|------|
| `NNN` | Three-digit zero-padded number, incrementing from the previous migration |
| `short_description` | Lowercase snake_case summary of the change |
The **`revision`** variable inside the file **must match the filename stem** exactly:
```python
# File: migrations/versions/037_add_classification_rules.py
revision: str = "037_add_classification_rules"
down_revision: Union[str, None] = "036_add_document_translation_fields"
```
The CI check (`scripts/check_alembic_migrations.py`) enforces this consistency.
---
## Idempotent Migration Patterns
Migrations should be **idempotent** — safe to run even if the change already exists. This is critical for SQLite compatibility and for recovering from partial failures.
### Add a Column (only if missing)
```python
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
if "my_table" in inspector.get_table_names():
existing = {c["name"] for c in inspector.get_columns("my_table")}
if "new_col" not in existing:
with op.batch_alter_table("my_table") as batch_op:
batch_op.add_column(sa.Column("new_col", sa.String(128), nullable=True))
```
### Create a Table (only if missing)
```python
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
if "new_table" not in inspector.get_table_names():
op.create_table(
"new_table",
sa.Column("id", sa.Integer(), primary_key=True),
sa.Column("name", sa.String(255), nullable=False),
)
```
### Drop a Column (only if present)
```python
def downgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
if "my_table" in inspector.get_table_names():
existing = {c["name"] for c in inspector.get_columns("my_table")}
if "new_col" in existing:
with op.batch_alter_table("my_table") as batch_op:
batch_op.drop_column("new_col")
```
### Use `batch_alter_table` for SQLite
SQLite does not support `ALTER TABLE DROP COLUMN` or `ALTER TABLE RENAME COLUMN` natively. Alembic's `batch_alter_table` context manager works around this by recreating the table:
```python
with op.batch_alter_table("users") as batch_op:
batch_op.add_column(sa.Column("phone", sa.String(20), nullable=True))
batch_op.drop_column("fax")
```
---
## Parallel Branch Development
When two feature branches both add migrations from the same parent, the migration chain **diverges** into multiple heads. This is normal and expected — Alembic supports it — but the heads must be merged before the code reaches `main`.
### Example
```
main: 001 → 002 → 003
↘ Branch A: 004_add_widgets
↘ Branch B: 004_add_gadgets ← two heads!
```
### How to Avoid Conflicts
1. **Coordinate** — if two developers are both adding migrations, assign different sequence numbers (e.g., `037_` and `038_`). Even if both depend on `036_`, different numbers prevent filename collisions.
2. **Rebase early** — before opening a PR, rebase your branch onto the latest `main`:
```bash
git fetch origin main
git rebase origin/main
```
If `main` now has a new migration `037_*`, renumber yours to `038_*` and update `down_revision` to point at `037_*`.
3. **Check for multiple heads** locally:
```bash
python scripts/check_alembic_migrations.py
# or
alembic heads
```
---
## Resolving Migration Conflicts
If your PR's CI check reports **"Multiple migration heads detected"**, follow these steps:
### Step 1 — Update Your Branch
```bash
git fetch origin main
git merge origin/main
# or
git rebase origin/main
```
### Step 2 — Check Heads
```bash
python scripts/check_alembic_migrations.py --verbose
```
The output lists the conflicting heads.
### Step 3 — Create a Merge Migration
```bash
alembic merge heads -m "merge_parallel_branches"
```
This generates a new migration with **two parents** (a merge point):
```python
down_revision = ("037_add_widgets", "037_add_gadgets")
```
### Step 4 — Rename and Validate
Rename the merge migration to the next sequence number:
```bash
mv migrations/versions/<hash>_merge_parallel_branches.py \
migrations/versions/038_merge_parallel_branches.py
```
Update the `revision` inside to match, then validate:
```bash
python scripts/check_alembic_migrations.py
```
### Step 5 — Test
```bash
alembic upgrade head
alembic downgrade -1
alembic upgrade head
```
---
## CI Validation
The CI pipeline (`.github/workflows/ci.yml`) includes a **migration-chain** job that runs:
```bash
python scripts/check_alembic_migrations.py
```
This script checks for:
| Check | Description |
|-------|-------------|
| Multiple heads | Diverged migration chains that need a merge migration |
| Broken references | A `down_revision` that points to a non-existent revision |
| Duplicate revisions | Two files declaring the same `revision` identifier |
| Filename mismatches | The `revision` variable doesn't match the filename stem |
The job runs in Stage 1 (fast-fail gates) alongside lint checks. If it fails, the build is blocked until the migration chain is fixed.
---
## Pre-commit Hook
A local pre-commit hook is configured in `.pre-commit-config.yaml` that runs the same check whenever you commit a change to `migrations/versions/`:
```yaml
- repo: local
hooks:
- id: check-alembic-migrations
name: Check Alembic migration chain
entry: python scripts/check_alembic_migrations.py
language: python
pass_filenames: false
files: ^migrations/versions/.*\.py$
```
Install the hook:
```bash
pip install pre-commit
pre-commit install
```
---
## Troubleshooting
### "Multiple migration heads detected"
See [Resolving Migration Conflicts](#resolving-migration-conflicts) above.
### "Broken chain: revision X references down_revision Y which does not exist"
You removed or renamed a migration that another migration depends on. Either restore the missing file or update the dependent migration's `down_revision`.
### "Filename mismatch: file declares revision=X but filename stem is Y"
The `revision` string inside the Python file must match the filename (without `.py`). Rename the file or update the variable.
### "relation already exists" when running `alembic upgrade head`
The database has a table that a pending migration tries to create. Stamp the current state:
```bash
alembic stamp head
```
### Autogenerate doesn't detect my changes
Ensure all models are imported in `migrations/env.py`. The `from app.models import ...` block at the top must include your new model class.
### SQLite "no such column" after downgrade
SQLite has limited `ALTER TABLE` support. Always use `op.batch_alter_table()` for column operations on existing tables.
---
## Further Reading
- [Alembic Tutorial](https://alembic.sqlalchemy.org/en/latest/tutorial.html)
- [Alembic Branch / Merge](https://alembic.sqlalchemy.org/en/latest/branches.html)
- [Database Configuration Guide](DatabaseConfiguration.md)
docs/MobileApp.md
+175 -10
View File
@@ -8,12 +8,18 @@ DocuElevate includes a native mobile application for iOS and Android built with
|---------|-----|---------|
| SSO login (OAuth2) | ✅ | ✅ |
| Local / basic auth login | ✅ | ✅ |
| QR code login (scan from web) | ✅ | ✅ |
| Auto-generated API token | ✅ | ✅ |
| Camera capture → upload | ✅ | ✅ |
| File picker upload | ✅ | ✅ |
| Multi-image selection from library | ✅ | ✅ |
| Share Sheet / Share Intent | ✅ | ✅ |
| Push notifications | ✅ | ✅ |
| Document list | ✅ | ✅ |
| Document list with search | ✅ | ✅ |
| File detail view with processing logs | ✅ | ✅ |
| Pre-login legal pages (GDPR) | ✅ | ✅ |
| Localization (EN, DE, ES, FR, IT) | ✅ | ✅ |
| Language selection | ✅ | ✅ |
| Dark mode | ✅ | ✅ |
## Getting Started (Development)
@@ -112,6 +118,18 @@ When developing with **Expo Go** the app does not have the `docuelevate://` cust
No extra configuration is needed — just run `npx expo start` and scan the QR code with the **Expo Go** app.
### QR Code Login Flow
As an alternative to SSO, users can log in by scanning a QR code displayed in the web UI:
1. The authenticated web user navigates to **Profile → Security & Sessions → Log in on mobile via QR code**.
2. A QR code is displayed containing a deep link: `docuelevate://qr-login?token=<challenge_token>&server=<server_url>`.
3. In the mobile app, the user taps **Scan QR Code to Login**, which opens the device camera.
4. The app scans the QR code, extracts both the server URL and the challenge token, and calls `POST /api/qr-auth/claim`.
5. An API token is issued and stored securely — no need to enter the server URL manually.
> **Note:** The QR code already contains the server URL, so users do not need to type it in when using QR login.
### Auto-generated Mobile Token
When the mobile app completes login it automatically creates a named API token (`"Mobile App – <device name>"`) via `POST /api/mobile/generate-token`. This token:
@@ -158,8 +176,8 @@ curl -X DELETE -H "Authorization: Bearer <token>" https://your-server/api/mobile
1. Open the **Upload** tab.
2. Tap **Photos**.
3. Select an existing photo from the device's photo library.
4. The image is uploaded and queued for processing.
3. Select one or more photos from the device's photo library (multi-selection is supported).
4. All selected images are uploaded and queued for processing.
### File Picker
@@ -185,6 +203,32 @@ The app registers itself as a share target so any file can be sent directly to D
The URL may arrive as a standard `file://` path **or** under the app's custom `docuelevate://` scheme (e.g. `docuelevate://private/var/mobile/Library/…/file.pdf`). The root layout detects the custom-scheme form and rewrites it to a `file://` URL before forwarding it to the Upload screen through `ShareContext`.
##### Handling "unmatched route" errors from "Open In…"
iOS sometimes delivers the file path under the `docuelevate://` scheme, e.g.:
```
docuelevate://private/var/mobile/Library/Mobile Documents/…/Invoice.pdf
```
expo-router strips the scheme and tries to match `/private/var/mobile/…` as an in-app route. Because no such route exists, it previously threw an **"unmatched route docuelevate://"** error and the upload never completed.
The fix is a catch-all `+not-found.tsx` route (see `mobile/app/+not-found.tsx`). When expo-router cannot match the path, it renders this screen instead. The screen detects that the path is a filesystem path rather than a real in-app route, adds the file directly to `ShareContext`, and redirects to the Upload tab. `UploadScreen` picks up the pending file and begins uploading automatically. The `Linking` listener in the root layout may also fire for the same URL; `ShareContext.addPendingFile` deduplicates by URI so the file is only uploaded once.
##### File accessibility and local caching
Shared files may reference paths outside the app's sandbox or use security-scoped URLs that React Native's `fetch` cannot read directly. To guarantee reliable uploads:
- **`LSSupportsOpeningDocumentsInPlace`** is set to `false` in `app.json`, which tells iOS to copy shared files into the app's `Documents/Inbox` directory before handing them to the app.
- **`UploadScreen`** uses `expo-file-system` (`FileSystem.copyAsync`) to copy any `file://` URI that is outside the app's cache/documents directory to a local cache path before uploading. This ensures the file is readable regardless of its origin.
- **MIME type inference**: Both `+not-found.tsx` and the `Linking` handler in `_layout.tsx` infer the MIME type from the file extension (e.g. `.pdf` → `application/pdf`) so the server receives a correct `Content-Type` instead of `application/octet-stream`.
##### iOS Action / Share Extension (future enhancement)
Apps like DeepL ("Translate in DeepL") and Microsoft Word ("Convert to Word") appear as **Action Extensions** in the iOS share sheet — a system-level feature that requires a separate Xcode target built with Swift or Objective-C. A proper Action Extension runs in its own process and must share authentication credentials with the main app via an iOS **App Group** (shared keychain / shared container).
This level of iOS-native integration is a planned future enhancement. Until it is available, the recommended workflow is the current one: tap **Share → DocuElevate** (the app appears in the "Open With" row of the share sheet via `CFBundleDocumentTypes`).
#### Android implementation
`app.json` declares `ACTION_SEND` and `ACTION_SEND_MULTIPLE` intent filters for `mimeType: "*/*"` in the `android.intentFilters` section. Incoming content URIs are received the same way as on iOS.
@@ -202,6 +246,75 @@ If a file upload fails (e.g. due to network issues or a server error), the faile
The retry re-uses the original file URI so no re-selection is needed.
## Document Search
The **Files** tab includes a search bar at the top that lets users search through their processed documents by filename. Searches are debounced (400ms) to avoid excessive API calls. Clear the search with the ✕ button to return to the full list.
## File Detail View
Tapping any document in the **Files** tab opens a detail view showing:
- **File metadata**: filename, file size, MIME type, upload date, and file hash
- **Processing status**: current status with a colour-coded icon
- **Processing log**: chronological list of processing steps with individual status indicators and timestamps
Pull-to-refresh updates the detail view. This replicates the web interface at `/files/{id}` and `/files/{id}/detail` in a mobile-friendly layout.
## Legal & Compliance
### GDPR & Apple App Store Compliance
Privacy Policy, Terms of Service, and Imprint links are accessible **before login** from both the **Welcome Screen** and the **Login Screen**. This ensures compliance with:
- **GDPR** (General Data Protection Regulation) – users must be able to review the privacy policy before providing personal data
- **Apple App Store Review Guidelines** – apps must provide accessible privacy information before account creation
Post-login, the same links are available in the **Profile** tab under the "Legal" section.
## Localization (i18n)
The mobile app supports five languages with automatic device-locale detection:
| Language | Code | Status |
|----------|------|--------|
| English | `en` | ✅ Complete |
| German (Deutsch) | `de` | ✅ Complete |
| Spanish (Español) | `es` | ✅ Complete |
| French (Français) | `fr` | ✅ Complete |
| Italian (Italiano) | `it` | ✅ Complete |
### How it works
Language priority (highest to lowest):
1. **Server preference** — `preferred_language` returned by `GET /api/mobile/whoami` on login or app resume. Allows a language set on the desktop web interface to propagate to mobile automatically.
2. **AsyncStorage** — the last language explicitly selected on the device, used as an offline fallback when the server is unreachable.
3. **Device locale** — detected via `expo-localization` on first launch.
4. **English** — final fallback when none of the above match a supported locale.
When a user selects a language on mobile the choice is:
- Applied immediately to all screens (via `LocaleContext`)
- Persisted locally to AsyncStorage
- Synced to the server via `POST /api/i18n/language` (fire-and-forget), so the next desktop login reflects the same preference.
> **Note**: If the server's preferred language is not supported by the mobile app (e.g. a locale added to the web frontend but not yet translated for mobile), the mobile app falls back to the next priority in the list above.
### Adding a new language
1. Create a new translation file in `mobile/src/i18n/` (e.g. `pt.json` for Portuguese)
2. Copy the structure from `en.json` and translate all values
3. Import the new file in `mobile/src/i18n/index.ts`
4. Add it to the `translations` object and `getSupportedLanguages()` array
## User Settings
The **Profile** tab includes a **Settings** section where users can:
- **Change language**: Select from the supported languages (English, German, Spanish, French, Italian)
- View server connection details
- Access legal documents (Privacy Policy, Terms of Service, Imprint)
- Sign out or delete their account
## Mobile API Endpoints
The backend exposes a dedicated `/api/mobile/` namespace:
@@ -212,7 +325,8 @@ The backend exposes a dedicated `/api/mobile/` namespace:
| `POST` | `/api/mobile/register-device` | Bearer | Register Expo push token |
| `GET` | `/api/mobile/devices` | Bearer | List registered devices |
| `DELETE` | `/api/mobile/devices/{id}` | Bearer | Deactivate a device |
| `GET` | `/api/mobile/whoami` | Bearer | Get current user profile |
| `GET` | `/api/mobile/whoami` | Bearer | Get current user profile (includes `preferred_language`) |
| `POST` | `/api/i18n/language` | Bearer | Sync language preference to server |
All other API endpoints (file upload, file listing, etc.) work with Bearer token authentication.
@@ -256,7 +370,7 @@ Re-registering the same token is safe (idempotent).
### GET /api/mobile/whoami
Returns the current user's profile.
Returns the current user's profile, including the server-stored language preference.
**Response (200):**
```json
@@ -265,10 +379,15 @@ Returns the current user's profile.
"display_name": "John Doe",
"email": "john@example.com",
"avatar_url": "https://www.gravatar.com/avatar/...",
"is_admin": false
"is_admin": false,
"preferred_language": "de"
}
```
`preferred_language` is `null` when no preference has been saved. The mobile
app applies this value on login / app resume, falling back to AsyncStorage and
then the device locale when it is `null` or unsupported.
## Configuration
No server-side configuration is required to enable the mobile app. The Expo push notification routing does not need FCM or APNs credentials on the server.
@@ -279,7 +398,20 @@ If you wish to use **direct FCM/APNs** without Expo's relay, replace the `send_e
```
mobile/
├── App.tsx # Root component
├── App.tsx # Root component (legacy, not used at runtime)
├── app/ # Expo Router file-based routes
│ ├── _layout.tsx # Root layout (AuthGuard + providers)
│ ├── index.tsx # Root redirect → /(auth)/
│ ├── (auth)/ # Unauthenticated route group
│ │ ├── _layout.tsx # Stack navigator (headerless)
│ │ ├── index.tsx # Welcome screen
│ │ ├── login.tsx # Login screen
│ │ └── qr-scanner.tsx # QR code scanner screen
│ └── (tabs)/ # Authenticated route group
│ ├── _layout.tsx # Tab navigator
│ ├── index.tsx # Upload screen (default tab)
│ ├── files.tsx # Files screen
│ └── profile.tsx # Profile screen
├── app.json # Expo/EAS configuration
├── eas.json # EAS Build profiles
├── package.json
@@ -291,16 +423,48 @@ mobile/
├── hooks/
│ └── usePushNotifications.ts # Push token registration
├── screens/
│ ├── LoginScreen.tsx # Server URL + SSO button
│ ├── LoginScreen.tsx # Server URL + SSO button + QR code scanner
│ ├── QRScannerScreen.tsx # Camera-based QR code scanner for login
│ ├── UploadScreen.tsx # Camera capture + photo library + file picker
│ ├── FilesScreen.tsx # Processed document list
│ └── ProfileScreen.tsx # User profile + sign out
│ ├── FilesScreen.tsx # Processed document list with search
│ ├── FileDetailScreen.tsx # File detail view with processing logs
│ ├── ProfileScreen.tsx # User profile + settings + sign out
│ └── WelcomeScreen.tsx # Pre-login welcome with legal links
├── i18n/ # Localization (i18n)
│ ├── index.ts # i18n module (locale detection, t() function)
│ ├── en.json # English translations
│ ├── de.json # German translations
│ ├── es.json # Spanish translations
│ ├── fr.json # French translations
│ └── it.json # Italian translations
├── utils/
│ ├── mimeTypes.ts # MIME type mapping for file extensions
│ └── normalizeUri.ts # URI normalization for deduplication
└── services/
└── api.ts # DocuElevate REST API client
```
## Troubleshooting
### App shows "Hello World" / default Expo page after update
If the iOS or Android app shows a generic "Hello World – This is the first page of your app" screen instead of the DocuElevate UI, it means a stale default `index.tsx` file (generated by Expo CLI scaffolding) is being picked up in the `mobile/app/` directory.
**To fix:**
1. Delete any leftover default `mobile/app/index.tsx` that is **not** the repository version (the repo version contains a `<Redirect>` to `/(auth)/`).
2. Clear the Metro bundler cache and rebuild:
```bash
cd mobile
npx expo start --clear
```
3. For production builds, run a clean EAS build:
```bash
eas build --platform ios --clear-cache
```
The repository includes a root `app/index.tsx` that immediately redirects to the authentication flow, so this issue should not recur once the correct file is present.
### "Session expired Local session" during iOS build
EAS stores an Apple ID session locally (in `~/.expo/`) to manage code-signing certificates and provisioning profiles. This session expires after a few weeks.
@@ -362,3 +526,4 @@ eas build --platform ios
- [API Documentation](./API.md)
- [Configuration Guide](./ConfigurationGuide.md)
- [Deployment Guide](./DeploymentGuide.md)
- [Apple App Store Compliance Audit](./AppleAppStoreCompliance.md)
docs/OneDriveSetup.md
+3 -2
View File
@@ -29,9 +29,10 @@ End users authorize their own OneDrive integration from the **Integrations** das
1. Navigate to `/integrations` and click **+ Add Destination** (or **+ Add Source** for Watch Folder).
2. Create a OneDrive destination integration (or a Watch Folder with `source_type = onedrive`).
3. Click the **Authorize** button next to the integration — it links directly to the OAuth wizard pre-loaded with your integration's configuration.
4. Enter your Azure AD Client ID and Client Secret in the wizard.
4. If the administrator has configured system-wide OneDrive app credentials (`ONEDRIVE_CLIENT_ID` / `ONEDRIVE_CLIENT_SECRET`), the wizard defaults to using them — no need to register your own Azure AD app. Uncheck the toggle to use custom credentials if needed.
5. Click **Start Authentication Flow**, authorize access via Microsoft, and the refresh token is automatically saved to your personal integration record.
6. The page redirects back to `/integrations` on success. Re-authorization is available at any time via the **Re-Authorize** button.
6. After authorization, an interactive **folder browser** lets you select the target folder directly from your OneDrive — no need to manually type folder paths.
7. The page redirects to `/integrations` on success. Re-authorization is available at any time via the **Re-Authorize** button.
> **Note:** Your credentials are stored encrypted per-integration and are never mixed with other users' data. Each user can have multiple OneDrive integrations with independent tokens.
docs/ProductionReadiness.md
+38 -15
View File
@@ -34,7 +34,7 @@ Use this checklist to track readiness before going live.
- [ ] **Redis** — Running and accessible only from internal network
- [ ] **Meilisearch** — Running and accessible only from internal network
- [ ] **Worker replicas** — At least 2 workers configured for redundancy
- [ ] **Monitoring** — `/api/health` polled by uptime checker
- [ ] **Monitoring** — `/api/diagnostic/health` polled by uptime checker
- [ ] **Backups** — Automated backup of database, workdir, and Meilisearch data
- [ ] **Log retention** — Logs shipped to a persistent store or aggregator
- [ ] **Secrets management** — API keys not committed to source control
@@ -285,22 +285,24 @@ For SSO/OIDC (Authentik, Keycloak, Auth0, etc.) see the [Authentication Setup Gu
### Docker Compose
Use the `deploy.replicas` setting (requires Docker Swarm mode) or simply run multiple workers:
```yaml
worker:
deploy:
replicas: 3
```
Or scale after deployment:
Scale workers independently:
```bash
docker-compose up -d --scale worker=3
docker compose up -d --scale worker=3
```
Each worker processes tasks from the Celery queue independently. Ensure the shared `workdir` volume is accessible from all worker containers.
> **Important:** The `beat` service (Celery Beat scheduler) must always run as exactly **one** instance. It is defined as a dedicated service in `docker-compose.yaml` with a fixed `container_name`. Do not scale it.
### Scaling the API
API pods are fully stateless (sessions use encrypted cookies, not server-side state) and can be scaled behind a load balancer:
```bash
docker compose up -d --scale api=3
```
### Kubernetes (Helm)
```yaml
@@ -339,11 +341,32 @@ celery -A app.celery_worker worker -Q default,celery --concurrency=2
### Health Check Endpoint
DocuElevate exposes `/api/health` for readiness probing. Configure your uptime monitor to poll this endpoint:
DocuElevate exposes three health-related endpoints:
| Endpoint | Auth | Purpose |
|----------|------|---------|
| `GET /api/diagnostic/healthz/live` | None | Lightweight liveness probe — returns 200 if the process is running |
| `GET /api/diagnostic/healthz/ready` | None | Readiness probe — checks database and Redis (503 when DB is down) |
| `GET /api/diagnostic/health` | Required | Full status for monitoring dashboards (Grafana, Uptime Kuma) |
For **Kubernetes probes**, use the unauthenticated endpoints:
```yaml
livenessProbe:
httpGet:
path: /api/diagnostic/healthz/live
port: 8000
readinessProbe:
httpGet:
path: /api/diagnostic/healthz/ready
port: 8000
```
For **uptime monitors** (Uptime Kuma, Grafana, etc.), use the authenticated endpoint:
```bash
curl http://docuelevate.example.com/api/health
# Expected: {"status": "ok", ...}
curl http://docuelevate.example.com/api/diagnostic/health
# Expected: {"status": "healthy", ...}
```
Set `UPTIME_KUMA_URL` to your Uptime Kuma push URL for heartbeat monitoring:
@@ -502,4 +525,4 @@ For a dedicated Kubernetes deployment guide, including architecture diagrams, PV
- **Image Pull Policy**: Use `IfNotPresent` in production with pinned image tags (not `latest`) for reproducible deployments.
- **Liveness & Readiness Probes**: Already configured in the Helm chart via `/api/health`. Verify they are tuned to your startup time.
- **Liveness & Readiness Probes**: Already configured in the Helm chart via unauthenticated endpoints (`/api/diagnostic/healthz/live` and `/api/diagnostic/healthz/ready`). Verify they are tuned to your startup time.
docs/README.md
+1
View File
@@ -22,6 +22,7 @@ Welcome to the DocuElevate documentation. This directory contains comprehensive
- [Google Drive Setup](GoogleDriveSetup.md) - How to set up Google Drive integration
- [Dropbox Setup](DropboxSetup.md) - How to set up Dropbox integration
- [OneDrive Setup](OneDriveSetup.md) - How to set up Microsoft OneDrive/Graph integration
- [SharePoint Setup](SharePointSetup.md) - How to set up Microsoft SharePoint Online integration
- [Amazon S3 Setup](AmazonS3Setup.md) - How to set up Amazon S3 integration
- [Authentication Setup](AuthenticationSetup.md) - How to set up user authentication
- [Notifications Setup](NotificationsSetup.md) - How to set up system notifications
docs/SettingsManagement.md
+1 -1
View File
@@ -28,7 +28,7 @@ Settings are organized into logical categories for easy navigation:
- **Authentication**: Login settings, session secrets, OAuth configuration, admin group
- **AI Services**: AI provider selection, model configuration, embeddings, and credentials (OpenAI, Azure, Anthropic, Gemini, Ollama, OpenRouter, Portkey, LiteLLM)
- **OCR Engines**: OCR provider selection and configuration (Tesseract, EasyOCR, Mistral, Google DocAI, AWS Textract)
- **Storage Providers**: Dropbox, Google Drive, OneDrive, S3, FTP, SFTP, WebDAV, Nextcloud, Paperless
- **Storage Providers**: Dropbox, Google Drive, OneDrive, SharePoint, S3, FTP, SFTP, WebDAV, Nextcloud, Paperless
- **Email**: SMTP configuration for sending emails
- **IMAP**: Email ingestion configuration (supports two mailbox accounts)
- **Monitoring**: Uptime Kuma integration
docs/SharePointSetup.md
+185
View File
@@ -0,0 +1,185 @@
# Setting up SharePoint Integration
This guide explains how to set up the Microsoft SharePoint Online integration for DocuElevate.
## Required Configuration Parameters
| **Variable** | **Description** |
|---------------------------------|-------------------------------------------------------|
| `SHAREPOINT_CLIENT_ID` | Azure AD application client ID |
| `SHAREPOINT_CLIENT_SECRET` | Azure AD application client secret |
| `SHAREPOINT_TENANT_ID` | Azure AD tenant ID (use "common" for multi-tenant apps) |
| `SHAREPOINT_REFRESH_TOKEN` | OAuth 2.0 refresh token |
| `SHAREPOINT_SITE_URL` | SharePoint site URL (e.g. `https://tenant.sharepoint.com/sites/sitename`) |
| `SHAREPOINT_DOCUMENT_LIBRARY` | Document library name (default: `Documents`) |
| `SHAREPOINT_FOLDER_PATH` | Subfolder path inside the document library |
For a complete list of configuration options, see the [Configuration Guide](ConfigurationGuide.md).
## Overview
SharePoint Online integration uses the same Microsoft Graph API as OneDrive. The key difference is that SharePoint targets a **site-specific document library** rather than a personal OneDrive. Documents are uploaded via chunked upload sessions for reliability with large files.
> **Tip:** If you already have an Azure AD app registration for OneDrive, you can reuse it for SharePoint — just add the `Sites.ReadWrite.All` permission.
## Setup Steps
### 1. Register an application in Azure Active Directory
If you don't already have an app registration (e.g. from OneDrive setup):
1. Go to the [Azure Portal](https://portal.azure.com/)
2. Navigate to **Azure Active Directory** > **App registrations**
3. Click **New registration**
4. Enter a name for your application (e.g., "DocuElevate")
5. For **Supported account types**, select:
- **Single tenant**: "Accounts in this organizational directory only"
- **Multi-tenant**: "Accounts in any organizational directory"
6. For **Redirect URI**, select "Web" and enter your callback URL (e.g., `https://your-domain.com/onedrive-callback`)
7. Click **Register**
### 2. Get Application (client) ID
1. After registration, note the **Application (client) ID** from the overview page
2. Set this value as `SHAREPOINT_CLIENT_ID`
### 3. Create a client secret
1. In your application page, go to **Certificates & secrets**
2. Under **Client secrets**, click **New client secret**
3. Add a description and select an expiration period
4. Click **Add** and immediately copy the secret value (it will only be shown once)
5. Set this value as `SHAREPOINT_CLIENT_SECRET`
### 4. Configure API permissions
1. In your application page, go to **API permissions**
2. Click **Add a permission**
3. Select **Microsoft Graph**
4. For **delegated permissions** (user-context access), add:
- `Sites.ReadWrite.All` — Read and write items in all site collections
- `offline_access` — Required for refresh tokens
5. For **application permissions** (app-only access without a user), add:
- `Sites.ReadWrite.All` — Read and write items in all site collections
6. Click **Add permissions**
7. Click **Grant admin consent** (requires admin privileges)
> **Important:** SharePoint access requires `Sites.ReadWrite.All` rather than the `Files.ReadWrite` permission used by OneDrive.
### 5. Get your Tenant ID
1. In the Azure Portal, find your **Tenant ID** (also called "Directory ID")
2. It is on the **Azure Active Directory** overview page
3. Set this value as `SHAREPOINT_TENANT_ID`
### 6. Generate a Refresh Token
#### Using the OneDrive Auth Wizard
The SharePoint integration reuses the same MSAL token flow as OneDrive:
1. Navigate to `/onedrive-setup`
2. Enter your SharePoint Client ID and Tenant ID
3. Click **Start Authentication Flow** and follow the prompts
4. Copy the generated refresh token and set it as `SHAREPOINT_REFRESH_TOKEN`
#### Manual Method
1. Open the following URL in your browser (replace placeholders):
```
https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/v2.0/authorize?client_id=YOUR_CLIENT_ID&response_type=code&redirect_uri=YOUR_REDIRECT_URI&response_mode=query&scope=https://graph.microsoft.com/.default offline_access&prompt=consent
```
2. Sign in with your Microsoft work account
3. After authentication, copy the `code` parameter from the redirect URL
4. Exchange the code for tokens:
```bash
curl -X POST https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/v2.0/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "client_id=YOUR_CLIENT_ID&scope=https://graph.microsoft.com/.default offline_access&code=YOUR_AUTH_CODE&redirect_uri=YOUR_REDIRECT_URI&grant_type=authorization_code&client_secret=YOUR_CLIENT_SECRET"
```
5. From the response JSON, copy the `refresh_token` value
6. Set this as `SHAREPOINT_REFRESH_TOKEN`
### 7. Find your SharePoint Site URL
Your SharePoint site URL follows the pattern:
```
https://YOUR-TENANT.sharepoint.com/sites/SITE-NAME
```
For example:
- `https://contoso.sharepoint.com/sites/documents`
- `https://contoso.sharepoint.com/sites/engineering-team`
Set this as `SHAREPOINT_SITE_URL`.
### 8. Choose your Document Library
Each SharePoint site has one or more document libraries. The default library is usually called `Documents` (or `Shared Documents`). You can find your library names by navigating to your SharePoint site in a browser and looking at the left sidebar.
Set the library name as `SHAREPOINT_DOCUMENT_LIBRARY` (default: `Documents`).
### 9. Set the Upload Folder (Optional)
If you want documents to be uploaded into a subfolder inside the library, set `SHAREPOINT_FOLDER_PATH`. For example, `Uploads` or `DocuElevate/Processed`.
## App-Only Access (No User Token)
For fully automated scenarios without user interaction:
1. Add **Application permissions** (not Delegated) for `Sites.ReadWrite.All`
2. Grant admin consent
3. Set `SHAREPOINT_TENANT_ID` to your organization's tenant ID
4. Leave `SHAREPOINT_REFRESH_TOKEN` empty — the app will use the client credentials flow
> **Note:** Client credentials flow requires a specific tenant ID (not "common").
## Configuration Examples
**With Refresh Token (Delegated Permissions):**
```dotenv
SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
SHAREPOINT_CLIENT_SECRET=your_client_secret
SHAREPOINT_TENANT_ID=87654321-4321-4321-4321-210987654321
SHAREPOINT_REFRESH_TOKEN=your_refresh_token
SHAREPOINT_SITE_URL=https://contoso.sharepoint.com/sites/documents
SHAREPOINT_DOCUMENT_LIBRARY=Documents
SHAREPOINT_FOLDER_PATH=Uploads
```
**App-Only Access (Application Permissions):**
```dotenv
SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
SHAREPOINT_CLIENT_SECRET=your_client_secret
SHAREPOINT_TENANT_ID=87654321-4321-4321-4321-210987654321
# No refresh token needed for app-only access
SHAREPOINT_SITE_URL=https://contoso.sharepoint.com/sites/documents
SHAREPOINT_DOCUMENT_LIBRARY=Shared Documents
SHAREPOINT_FOLDER_PATH=DocuElevate/Processed
```
## Troubleshooting
### "Failed to resolve SharePoint site"
- Verify `SHAREPOINT_SITE_URL` is correct and accessible
- Ensure your app has `Sites.ReadWrite.All` permission with admin consent
- Check that the site exists and your account has access to it
### "Document library not found"
- Verify the library name in `SHAREPOINT_DOCUMENT_LIBRARY` matches exactly (case-insensitive)
- Navigate to your SharePoint site in a browser to confirm the library name
- Common names: `Documents`, `Shared Documents`
### Token errors
- If using a refresh token, try re-authorizing via the OAuth flow
- Ensure `offline_access` scope is included in your permissions
- For app-only access, verify the tenant ID is not set to "common"
### Permission errors
- Ensure an admin has granted consent for `Sites.ReadWrite.All`
- Verify the app registration has the correct permissions
- Check that the site's sharing settings allow API access
docs/StorageArchitecture.md
+1
View File
@@ -341,6 +341,7 @@ in task messages or logs.
| `S3` | boto3 `upload_file`, per-user access key |
| `GOOGLE_DRIVE` | Google Drive API v3, OAuth or service account |
| `ONEDRIVE` | Microsoft Graph API, MSAL confidential-client |
| `SHAREPOINT` | Microsoft Graph API, site/drive resolution + chunked upload |
| `WEBDAV` | HTTP PUT request, Basic Auth |
| `NEXTCLOUD` | WebDAV (same as WEBDAV, Nextcloud-compatible path) |
| `FTP` | ftplib FTPS (TLS preferred, plaintext configurable) |
docs/UserGuide.md
+21 -3
View File
@@ -87,7 +87,7 @@ DocuElevate provides multiple convenient ways to upload documents to the system.
#### Supported File Types
- **Documents**: PDF, Word (.doc, .docx), Excel (.xls, .xlsx), PowerPoint (.ppt, .pptx)
- **Images**: JPEG, PNG, GIF, BMP, TIFF, WebP, SVG
- **Images**: JPEG, PNG, GIF, BMP, TIFF, WebP, SVG, HEIC, HEIF
- **Text**: Plain text (.txt), CSV, RTF, HTML, XML, Markdown
- **Maximum file size**: 500MB per file
@@ -170,7 +170,7 @@ The **Integrations** page (`/integrations`) provides a unified view of all your
- **S3** — bucket, region, access key, secret key
- **WebDAV / Nextcloud** — URL, folder, username, password
- **FTP / SFTP** — host, port, remote path, username, password
- **Dropbox / Google Drive / OneDrive** — folder path, with a link to the OAuth setup page
- **Dropbox / Google Drive / OneDrive / SharePoint** — folder path, with a link to the OAuth setup page
- **Email Forward** — recipient email address
- **Watch Folder** — source type (Local, S3, Dropbox, Google Drive, OneDrive, Nextcloud, WebDAV), per-type config fields, delete after processing toggle
- **Paperless NGX** — URL and API token
@@ -582,7 +582,25 @@ Processing pipelines let you define exactly what happens to your documents when
| `embed_metadata` | Write extracted metadata into the PDF document properties |
| `compute_embedding` | Compute semantic embeddings for similarity search |
| `send_to_destinations` | Upload the processed document to all configured storage destinations |
| `classify` | Classify the document type with AI |
| `classify` | Classify the document type using rules (filename patterns, content keywords, metadata) |
#### Classify step – rule-based document classification
The `classify` step assigns a category to each document by evaluating **built-in** and **custom** classification rules. Rules are matched against three signals:
- **Filename patterns** — regex matched against the original filename (e.g. `(?i)invoice` matches filenames containing "invoice").
- **Content keywords** — pipe-separated keywords matched against the OCR text (e.g. `invoice number|amount due`).
- **Metadata match** — `field=value` matched against existing AI metadata (e.g. `document_type=Invoice`).
**Pre-built categories** include: Invoice, Contract, Receipt, Letter, Report, Bank Statement, Tax Document, Insurance, and Payslip. You can also define your own custom categories.
The classification result is stored in the document's `ai_metadata` under the `classification` key with the matched category, confidence score, and list of matched rules. If no `document_type` was previously set by AI metadata extraction, the classify step will also populate it.
> **Tip:** Manage custom classification rules via **Settings → Classification Rules** or the `/api/classification-rules/` API. See the [API Documentation](./API.md#classification-rules) for details.
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `use_builtin_rules` | boolean | `true` | Include the pre-built classification rules |
#### OCR step options
frontend/static/js/common.js
+15
View File
@@ -215,6 +215,9 @@ function _makeMenuLink(href, iconClass, label, extraClasses = '') {
linksDiv.appendChild(
_makeMenuLink('/api-tokens', 'fas fa-key text-yellow-500', window.__i18n.apiTokens || 'API Tokens', 'text-gray-700')
);
linksDiv.appendChild(
_makeMenuLink('/devices', 'fas fa-mobile-alt text-blue-500', window.__i18n.devices || 'Devices', 'text-gray-700')
);
linksDiv.appendChild(
_makeMenuLink('/shared-links', 'fas fa-share-alt text-blue-400', window.__i18n.sharedLinks || 'Shared Links', 'text-gray-700')
);
@@ -311,6 +314,18 @@ function _makeMenuLink(href, iconClass, label, extraClasses = '') {
tokensLink.appendChild(document.createTextNode(window.__i18n.apiTokens || 'API Tokens'));
mobileAuthSection.appendChild(tokensLink);
// Devices link
const devicesLink = document.createElement('a');
devicesLink.href = '/devices';
devicesLink.className =
'flex items-center px-3 py-3 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50';
const devicesIcon = document.createElement('i');
devicesIcon.className = 'fas fa-mobile-alt mr-2 text-blue-500';
devicesIcon.setAttribute('aria-hidden', 'true');
devicesLink.appendChild(devicesIcon);
devicesLink.appendChild(document.createTextNode(window.__i18n.devices || 'Devices'));
mobileAuthSection.appendChild(devicesLink);
// Shared Links link
const sharedLinksLink = document.createElement('a');
sharedLinksLink.href = '/shared-links';
frontend/static/js/upload.js
+11 -3
View File
@@ -433,9 +433,17 @@ function _uploadSingleFile(file, progressBar, statusEl, onTerminal) {
if (xhr.status === 200) {
const result = JSON.parse(xhr.responseText);
progressBar.style.width = '100%';
progressBar.className = 'file-progress-bar bg-green-500 h-2 rounded-full';
statusEl.textContent = `Success: Task ID: ${result.task_id}`;
statusEl.className = 'text-xs text-green-600 mt-1';
if (result.status === 'duplicate' && result.duplicate_of) {
// Exact duplicate – no processing task was created
progressBar.className = 'file-progress-bar bg-yellow-400 h-2 rounded-full';
statusEl.textContent = `Duplicate – already processed (file #${result.duplicate_of.original_file_id})`;
statusEl.className = 'text-xs text-yellow-600 mt-1';
} else {
progressBar.className = 'file-progress-bar bg-green-500 h-2 rounded-full';
statusEl.textContent = `Success: Task ID: ${result.task_id}`;
statusEl.className = 'text-xs text-green-600 mt-1';
}
_onUploadSuccess();
onTerminal();
resolve({ rateLimited: false, retryAfterSeconds: 0 });
frontend/templates/api_tokens.html
+117 -8
View File
@@ -33,6 +33,20 @@
aria-required="true"
/>
</div>
<div class="sm:w-44">
<label for="token-lifetime" class="sr-only">{{ _("api_tokens.expires_in_days_label") }}</label>
<input
id="token-lifetime"
type="number"
x-model.number="newTokenExpiresDays"
placeholder="{{ _('api_tokens.expires_at_placeholder') }}"
min="1"
max="3650"
class="w-full px-4 py-2 border border-gray-300 dark:border-gray-600 rounded-md shadow-sm
focus:outline-none focus:ring-2 focus:ring-indigo-500 dark:bg-gray-700 dark:text-white text-sm"
aria-label="{{ _('api_tokens.expires_in_days_label') }}"
/>
</div>
<button
type="submit"
:disabled="creating || !newTokenName.trim()"
@@ -143,6 +157,7 @@
<th scope="col" class="px-6 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("api_tokens.col_created") }}</th>
<th scope="col" class="px-6 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("api_tokens.col_last_used") }}</th>
<th scope="col" class="px-6 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("api_tokens.col_last_ip") }}</th>
<th scope="col" class="px-6 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("api_tokens.col_expires") }}</th>
<th scope="col" class="px-6 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("common.status") }}</th>
<th scope="col" class="px-6 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider sr-only">{{ _("common.actions") }}</th>
</tr>
@@ -162,28 +177,62 @@
<code x-show="token.last_used_ip" class="bg-gray-100 dark:bg-gray-700 px-2 py-0.5 rounded text-xs font-mono" x-text="token.last_used_ip"></code>
<span x-show="!token.last_used_ip" class="text-gray-400">—</span>
</td>
<td class="px-6 py-4 whitespace-nowrap text-gray-500 dark:text-gray-400">
<span x-text="token.expires_at ? formatDate(token.expires_at) : '{{ _('api_tokens.expires_never') }}'"></span>
</td>
<td class="px-6 py-4 whitespace-nowrap">
<span
class="inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium"
:class="token.is_active ? 'bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-400' : 'bg-red-100 text-red-800 dark:bg-red-900/30 dark:text-red-400'"
x-text="token.is_active ? '{{ _('api_tokens.status_active') }}' : '{{ _('api_tokens.status_revoked') }}'"
:class="tokenStatusClass(token)"
x-text="tokenStatusLabel(token)"
></span>
</td>
<td class="px-6 py-4 whitespace-nowrap text-right">
<!-- Revoke button – shown for active tokens -->
<button
x-show="token.is_active"
type="button"
@click="revokeToken(token)"
:disabled="revoking === token.id"
:disabled="acting === token.id"
class="inline-flex items-center px-3 py-1.5 text-sm font-medium text-red-600 hover:text-red-800
dark:text-red-400 dark:hover:text-red-300 hover:bg-red-50 dark:hover:bg-red-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-red-500 disabled:opacity-50 transition-colors"
style="min-height:36px; min-width:44px;"
:aria-label="'{{ _('api_tokens.revoke_prefix') }} ' + token.name"
>
<i :class="revoking === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-trash-alt'" class="mr-1" aria-hidden="true"></i>
<i :class="acting === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-ban'" class="mr-1" aria-hidden="true"></i>
{{ _("api_tokens.revoke") }}
</button>
<!-- Reactivate button – shown for revoked tokens -->
<button
x-show="!token.is_active"
type="button"
@click="reactivateToken(token)"
:disabled="acting === token.id"
class="inline-flex items-center px-3 py-1.5 text-sm font-medium text-green-600 hover:text-green-800
dark:text-green-400 dark:hover:text-green-300 hover:bg-green-50 dark:hover:bg-green-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-green-500 disabled:opacity-50 transition-colors mr-1"
style="min-height:36px; min-width:44px;"
:aria-label="'{{ _('api_tokens.reactivate_prefix') }} ' + token.name"
>
<i :class="acting === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-redo'" class="mr-1" aria-hidden="true"></i>
{{ _("api_tokens.reactivate") }}
</button>
<!-- Delete button – shown for revoked tokens -->
<button
x-show="!token.is_active"
type="button"
@click="deleteToken(token)"
:disabled="acting === token.id"
class="inline-flex items-center px-3 py-1.5 text-sm font-medium text-red-600 hover:text-red-800
dark:text-red-400 dark:hover:text-red-300 hover:bg-red-50 dark:hover:bg-red-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-red-500 disabled:opacity-50 transition-colors"
style="min-height:36px; min-width:44px;"
:aria-label="'{{ _('api_tokens.delete_prefix') }} ' + token.name"
>
<i :class="acting === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-trash-alt'" class="mr-1" aria-hidden="true"></i>
{{ _("api_tokens.delete") }}
</button>
</td>
</tr>
</template>
@@ -209,9 +258,10 @@ function apiTokens() {
tokens: [],
loading: true,
creating: false,
revoking: null,
acting: null,
error: null,
newTokenName: '',
newTokenExpiresDays: null,
newlyCreatedToken: null,
copied: false,
baseUrl: window.location.origin,
@@ -238,13 +288,15 @@ function apiTokens() {
this.error = null;
this.newlyCreatedToken = null;
try {
const body = { name: this.newTokenName.trim() };
if (this.newTokenExpiresDays) body.expires_in_days = parseInt(this.newTokenExpiresDays);
const res = await fetch('/api/api-tokens/', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-CSRF-Token': csrfToken,
},
body: JSON.stringify({ name: this.newTokenName.trim() }),
body: JSON.stringify(body),
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
@@ -253,6 +305,7 @@ function apiTokens() {
const data = await res.json();
this.newlyCreatedToken = data.token;
this.newTokenName = '';
this.newTokenExpiresDays = null;
await this.loadTokens();
} catch (e) {
this.error = e.message;
@@ -263,7 +316,7 @@ function apiTokens() {
async revokeToken(token) {
if (!confirm(`Revoke token "${token.name}"? This cannot be undone.`)) return;
this.revoking = token.id;
this.acting = token.id;
this.error = null;
try {
const res = await fetch(`/api/api-tokens/${token.id}`, {
@@ -278,10 +331,66 @@ function apiTokens() {
} catch (e) {
this.error = e.message;
} finally {
this.revoking = null;
this.acting = null;
}
},
async reactivateToken(token) {
if (!confirm({{ _("api_tokens.reactivate_confirm") | tojson }})) return;
this.acting = token.id;
this.error = null;
try {
const res = await fetch(`/api/api-tokens/${token.id}/reactivate`, {
method: 'POST',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to reactivate token');
}
await this.loadTokens();
} catch (e) {
this.error = e.message;
} finally {
this.acting = null;
}
},
async deleteToken(token) {
if (!confirm({{ _("api_tokens.delete_confirm") | tojson }})) return;
this.acting = token.id;
this.error = null;
try {
const res = await fetch(`/api/api-tokens/${token.id}`, {
method: 'DELETE',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to delete token');
}
await this.loadTokens();
} catch (e) {
this.error = e.message;
} finally {
this.acting = null;
}
},
tokenStatusClass(token) {
if (!token.is_active) return 'bg-red-100 text-red-800 dark:bg-red-900/30 dark:text-red-400';
if (token.expires_at && new Date(token.expires_at) < new Date())
return 'bg-yellow-100 text-yellow-800 dark:bg-yellow-900/30 dark:text-yellow-400';
return 'bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-400';
},
tokenStatusLabel(token) {
if (!token.is_active) return '{{ _("api_tokens.status_revoked") }}';
if (token.expires_at && new Date(token.expires_at) < new Date())
return '{{ _("api_tokens.status_expired") }}';
return '{{ _("api_tokens.status_active") }}';
},
copyToken() {
if (this.newlyCreatedToken) {
navigator.clipboard.writeText(this.newlyCreatedToken);
frontend/templates/base.html
+11
View File
@@ -177,6 +177,11 @@
<a href="/admin/backup" role="menuitem" class="flex items-center px-4 py-2 text-sm text-gray-700 hover:bg-gray-100">
<i class="fas fa-database w-4 mr-2 text-green-600" aria-hidden="true"></i> {{ _("nav.backup_restore") }}
</a>
{% if enable_factory_reset %}
<a href="/admin/system-reset" role="menuitem" class="flex items-center px-4 py-2 text-sm text-red-600 hover:bg-red-50">
<i class="fas fa-skull-crossbones w-4 mr-2 text-red-500" aria-hidden="true"></i> {{ _("nav.system_reset") }}
</a>
{% endif %}
<a href="/admin/audit-logs" role="menuitem" class="flex items-center px-4 py-2 text-sm text-gray-700 hover:bg-gray-100">
<i class="fas fa-shield-halved w-4 mr-2 text-indigo-500" aria-hidden="true"></i> Audit Logs
</a>
@@ -445,6 +450,11 @@
<a href="/admin/backup" class="block px-3 py-3 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50">
<i class="fas fa-database mr-2 text-green-500" aria-hidden="true"></i> {{ _("nav.backup_restore") }}
</a>
{% if enable_factory_reset %}
<a href="/admin/system-reset" class="block px-3 py-3 rounded-md text-base font-medium text-red-600 hover:text-red-800 hover:bg-red-50">
<i class="fas fa-skull-crossbones mr-2 text-red-500" aria-hidden="true"></i> {{ _("nav.system_reset") }}
</a>
{% endif %}
<a href="/status" class="block px-3 py-3 rounded-md text-base font-medium text-gray-700 hover:text-gray-900 hover:bg-gray-50"
{% if request and request.url.path == '/status' %}aria-current="page"{% endif %}>
<i class="fas fa-circle-dot mr-2 text-gray-400" aria-hidden="true"></i> {{ _("nav.status") }}
@@ -560,6 +570,7 @@
profileSettings: {{ _("nav.profile_settings") | tojson }},
mySubscription: {{ _("nav.my_subscription") | tojson }},
apiTokens: {{ _("nav.api_tokens") | tojson }},
devices: {{ _("nav.devices") | tojson }},
sharedLinks: {{ _("nav.shared_links") | tojson }},
signOut: {{ _("nav.sign_out") | tojson }},
logIn: {{ _("nav.login") | tojson }},
frontend/templates/devices.html
+459
View File
@@ -0,0 +1,459 @@
{% extends "base.html" %}
{% block title %}{{ _("devices.page_title") }}{% endblock %}
{% block content %}
<div x-data="devicesPage()" x-init="init()" class="container mx-auto px-4 py-8 max-w-6xl">
<!-- ── Header ─────────────────────────────────────────────────────────── -->
<header class="mb-8">
<h1 class="text-2xl font-bold text-gray-900 dark:text-white flex items-center gap-2">
<i class="fas fa-mobile-alt text-blue-500" aria-hidden="true"></i>
{{ _("devices.heading") }}
</h1>
<p class="mt-2 text-gray-600 dark:text-gray-400 text-sm leading-relaxed max-w-2xl">
{{ _("devices.intro") }}
</p>
</header>
<!-- ── Mobile App Tokens ──────────────────────────────────────────────── -->
<section class="bg-white dark:bg-gray-800 shadow rounded-lg overflow-hidden mb-6" aria-labelledby="mobile-tokens-heading">
<div class="px-6 py-4 border-b border-gray-200 dark:border-gray-700">
<h2 id="mobile-tokens-heading" class="text-lg font-semibold text-gray-900 dark:text-white">
<i class="fas fa-key text-yellow-500 mr-2" aria-hidden="true"></i>{{ _("devices.mobile_tokens_heading") }}
</h2>
<p class="text-sm text-gray-500 dark:text-gray-400 mt-1">{{ _("devices.mobile_tokens_description") }}</p>
</div>
<!-- Loading -->
<template x-if="loadingTokens">
<div class="p-8 text-center text-gray-500 dark:text-gray-400">
<i class="fas fa-spinner fa-spin text-2xl mb-2" aria-hidden="true"></i>
<p class="text-sm">{{ _("devices.loading") }}</p>
</div>
</template>
<!-- Empty state -->
<template x-if="!loadingTokens && mobileTokens.length === 0">
<div class="p-8 text-center text-gray-500 dark:text-gray-400">
<i class="fas fa-mobile-alt text-4xl mb-3 text-gray-300 dark:text-gray-600" aria-hidden="true"></i>
<p class="font-medium">{{ _("devices.no_mobile_tokens") }}</p>
<p class="text-sm mt-1">{{ _("devices.no_mobile_tokens_help") }}</p>
</div>
</template>
<!-- Tokens table -->
<template x-if="!loadingTokens && mobileTokens.length > 0">
<div class="overflow-x-auto">
<table class="w-full text-sm" aria-label="{{ _('devices.mobile_tokens_heading') }}">
<thead>
<tr class="bg-gray-50 dark:bg-gray-750 text-left">
<th scope="col" class="px-4 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("devices.col_device") }}</th>
<th scope="col" class="px-4 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("devices.col_token_prefix") }}</th>
<th scope="col" class="px-4 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("devices.col_created") }}</th>
<th scope="col" class="px-4 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("devices.col_last_used") }}</th>
<th scope="col" class="px-4 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider">{{ _("devices.col_status") }}</th>
<th scope="col" class="px-4 py-3 font-medium text-gray-500 dark:text-gray-400 uppercase text-xs tracking-wider sr-only">{{ _("common.actions") }}</th>
</tr>
</thead>
<tbody class="divide-y divide-gray-200 dark:divide-gray-700">
<template x-for="token in mobileTokens" :key="token.id">
<tr class="hover:bg-gray-50 dark:hover:bg-gray-750 transition-colors">
<td class="px-4 py-3 whitespace-nowrap">
<div class="flex items-center gap-2">
<i class="fas fa-mobile-alt text-gray-400" aria-hidden="true"></i>
<span class="font-medium text-gray-900 dark:text-white" x-text="formatDeviceName(token.name)"></span>
</div>
</td>
<td class="px-4 py-3 whitespace-nowrap">
<code class="bg-gray-100 dark:bg-gray-700 px-2 py-1 rounded text-xs font-mono" x-text="token.token_prefix + '…'"></code>
</td>
<td class="px-4 py-3 whitespace-nowrap text-sm text-gray-500 dark:text-gray-400" x-text="formatDate(token.created_at)"></td>
<td class="px-4 py-3 whitespace-nowrap text-sm text-gray-500 dark:text-gray-400">
<span x-text="token.last_used_at ? formatDate(token.last_used_at) : '—'"></span>
<span x-show="token.last_used_ip" class="block text-xs text-gray-400 mt-0.5">
<i class="fas fa-globe mr-1" aria-hidden="true"></i><span x-text="token.last_used_ip"></span>
</span>
</td>
<td class="px-4 py-3 whitespace-nowrap">
<span
class="inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium"
:class="token.is_active ? 'bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-400' : 'bg-red-100 text-red-800 dark:bg-red-900/30 dark:text-red-400'"
x-text="token.is_active ? '{{ _('devices.status_active') }}' : '{{ _('devices.status_revoked') }}'"
></span>
</td>
<td class="px-4 py-3 whitespace-nowrap text-right">
<div class="inline-flex items-center gap-1">
<!-- Revoke button – shown for active tokens -->
<button
x-show="token.is_active"
type="button"
@click="revokeToken(token)"
:disabled="actingToken === token.id"
:title="'{{ _('devices.revoke_token') }}'"
class="inline-flex items-center justify-center w-11 h-11 text-sm text-red-600 hover:text-red-800
dark:text-red-400 dark:hover:text-red-300 hover:bg-red-50 dark:hover:bg-red-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-red-500 disabled:opacity-50 transition-colors"
:aria-label="'{{ _('devices.revoke_token') }} ' + token.name"
>
<i :class="actingToken === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-sign-out-alt'" aria-hidden="true"></i>
</button>
<!-- Reactivate button – shown for revoked tokens -->
<button
x-show="!token.is_active"
type="button"
@click="reactivateMobileToken(token)"
:disabled="actingToken === token.id"
:title="'{{ _('devices.reactivate_token') }}'"
class="inline-flex items-center justify-center w-11 h-11 text-sm text-green-600 hover:text-green-800
dark:text-green-400 dark:hover:text-green-300 hover:bg-green-50 dark:hover:bg-green-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-green-500 disabled:opacity-50 transition-colors"
:aria-label="'{{ _('devices.reactivate_token') }} ' + token.name"
>
<i :class="actingToken === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-redo'" aria-hidden="true"></i>
</button>
<!-- Delete button – shown for revoked tokens -->
<button
x-show="!token.is_active"
type="button"
@click="deleteMobileToken(token)"
:disabled="actingToken === token.id"
:title="'{{ _('devices.delete_token') }}'"
class="inline-flex items-center justify-center w-11 h-11 text-sm text-red-600 hover:text-red-800
dark:text-red-400 dark:hover:text-red-300 hover:bg-red-50 dark:hover:bg-red-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-red-500 disabled:opacity-50 transition-colors"
:aria-label="'{{ _('devices.delete_token') }} ' + token.name"
>
<i :class="actingToken === token.id ? 'fas fa-spinner fa-spin' : 'fas fa-trash-alt'" aria-hidden="true"></i>
</button>
</div>
</td>
</tr>
</template>
</tbody>
</table>
</div>
</template>
<!-- Error -->
<template x-if="tokenError">
<div class="m-4 bg-red-50 dark:bg-red-900/20 border border-red-200 dark:border-red-800 text-red-700 dark:text-red-400 p-3 rounded text-sm" role="alert">
<i class="fas fa-exclamation-triangle mr-1" aria-hidden="true"></i>
<span x-text="tokenError"></span>
</div>
</template>
</section>
<!-- ── Registered Devices (Push Notifications) ────────────────────────── -->
<section class="bg-white dark:bg-gray-800 shadow rounded-lg overflow-hidden mb-6" aria-labelledby="devices-heading">
<div class="px-6 py-4 border-b border-gray-200 dark:border-gray-700">
<h2 id="devices-heading" class="text-lg font-semibold text-gray-900 dark:text-white">
<i class="fas fa-bell text-purple-500 mr-2" aria-hidden="true"></i>{{ _("devices.registered_devices_heading") }}
</h2>
<p class="text-sm text-gray-500 dark:text-gray-400 mt-1">{{ _("devices.registered_devices_description") }}</p>
</div>
<!-- Loading -->
<template x-if="loadingDevices">
<div class="p-8 text-center text-gray-500 dark:text-gray-400">
<i class="fas fa-spinner fa-spin text-2xl mb-2" aria-hidden="true"></i>
<p class="text-sm">{{ _("devices.loading") }}</p>
</div>
</template>
<!-- Empty state -->
<template x-if="!loadingDevices && devices.length === 0">
<div class="p-8 text-center text-gray-500 dark:text-gray-400">
<i class="fas fa-bell-slash text-4xl mb-3 text-gray-300 dark:text-gray-600" aria-hidden="true"></i>
<p class="font-medium">{{ _("devices.no_devices") }}</p>
<p class="text-sm mt-1">{{ _("devices.no_devices_help") }}</p>
</div>
</template>
<!-- Devices list -->
<template x-if="!loadingDevices && devices.length > 0">
<div class="divide-y divide-gray-200 dark:divide-gray-700">
<template x-for="device in devices" :key="device.id">
<div class="flex items-center justify-between px-6 py-4 hover:bg-gray-50 dark:hover:bg-gray-750 transition-colors">
<div class="flex items-center gap-3 min-w-0">
<i
:class="device.platform === 'ios' ? 'fab fa-apple' :
device.platform === 'android' ? 'fab fa-android text-green-500' :
'fas fa-globe'"
class="text-lg text-gray-400 flex-shrink-0"
aria-hidden="true"
></i>
<div class="min-w-0">
<div class="text-sm font-medium text-gray-900 dark:text-white truncate">
<span x-text="device.device_name || 'Unknown Device'"></span>
<span
class="ml-2 inline-flex items-center px-2 py-0.5 rounded text-xs font-medium"
:class="device.is_active ? 'bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-400' : 'bg-gray-100 text-gray-600 dark:bg-gray-700 dark:text-gray-400'"
x-text="device.is_active ? '{{ _('devices.status_active') }}' : '{{ _('devices.status_inactive') }}'"
></span>
</div>
<div class="text-xs text-gray-500 dark:text-gray-400 space-x-3 mt-0.5">
<span>
<i class="fas fa-microchip mr-1" aria-hidden="true"></i>
<span x-text="device.platform.charAt(0).toUpperCase() + device.platform.slice(1)"></span>
</span>
<span x-show="device.last_seen_at">
<i class="fas fa-clock mr-1" aria-hidden="true"></i>{{ _("devices.col_last_seen") }}:
<span x-text="formatDate(device.last_seen_at)"></span>
</span>
<span>
<i class="fas fa-calendar mr-1" aria-hidden="true"></i>
<span x-text="formatDate(device.created_at)"></span>
</span>
</div>
</div>
</div>
<button
x-show="device.is_active"
type="button"
@click="deactivateDevice(device)"
:disabled="actingDevice === device.id"
class="flex-shrink-0 inline-flex items-center px-3 py-1.5 text-sm font-medium text-red-600 hover:text-red-800
dark:text-red-400 dark:hover:text-red-300 hover:bg-red-50 dark:hover:bg-red-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-red-500 disabled:opacity-50 transition-colors"
style="min-height:36px; min-width:44px;"
:aria-label="'{{ _('devices.deactivate_device') }} ' + (device.device_name || 'device')"
>
<i :class="actingDevice === device.id ? 'fas fa-spinner fa-spin' : 'fas fa-power-off'" class="mr-1" aria-hidden="true"></i>
{{ _("devices.deactivate_device") }}
</button>
<!-- Delete button – shown for already-inactive devices -->
<button
x-show="!device.is_active"
type="button"
@click="deleteDevice(device)"
:disabled="actingDevice === device.id"
class="flex-shrink-0 inline-flex items-center px-3 py-1.5 text-sm font-medium text-red-600 hover:text-red-800
dark:text-red-400 dark:hover:text-red-300 hover:bg-red-50 dark:hover:bg-red-900/20 rounded-md
focus:outline-none focus:ring-2 focus:ring-red-500 disabled:opacity-50 transition-colors"
style="min-height:36px; min-width:44px;"
:aria-label="'{{ _('devices.delete_device') }} ' + (device.device_name || 'device')"
>
<i :class="actingDevice === device.id ? 'fas fa-spinner fa-spin' : 'fas fa-trash-alt'" class="mr-1" aria-hidden="true"></i>
{{ _("devices.delete_device") }}
</button>
</div>
</template>
</div>
</template>
<!-- Error -->
<template x-if="deviceError">
<div class="m-4 bg-red-50 dark:bg-red-900/20 border border-red-200 dark:border-red-800 text-red-700 dark:text-red-400 p-3 rounded text-sm" role="alert">
<i class="fas fa-exclamation-triangle mr-1" aria-hidden="true"></i>
<span x-text="deviceError"></span>
</div>
</template>
</section>
<!-- ── QR Login CTA ───────────────────────────────────────────────────── -->
<div class="text-center">
<a
href="/qr-login"
class="inline-flex items-center px-5 py-2.5 bg-blue-600 hover:bg-blue-700 text-white text-sm font-medium
rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500 transition-colors"
style="min-height:44px;"
>
<i class="fas fa-qrcode mr-2" aria-hidden="true"></i>
{{ _("devices.qr_login_cta") }}
</a>
</div>
<!-- ── Status banner ──────────────────────────────────────────────────── -->
<div
x-show="banner.visible"
x-transition
class="mt-6 rounded-lg p-3 text-sm"
:class="banner.error
? 'bg-red-50 dark:bg-red-900/30 text-red-800 dark:text-red-200 border border-red-300 dark:border-red-700'
: 'bg-green-50 dark:bg-green-900/30 text-green-800 dark:text-green-200 border border-green-300 dark:border-green-700'"
role="alert"
aria-live="polite"
>
<span x-text="banner.message"></span>
</div>
</div>
<script>
function devicesPage() {
const csrfToken = '{{ csrf_token | default("") }}';
return {
mobileTokens: [],
devices: [],
loadingTokens: true,
loadingDevices: true,
actingToken: null,
actingDevice: null,
tokenError: null,
deviceError: null,
banner: { visible: false, error: false, message: '' },
async init() {
await Promise.all([this.loadMobileTokens(), this.loadDevices()]);
},
async loadMobileTokens() {
this.loadingTokens = true;
this.tokenError = null;
try {
const res = await fetch('/api/api-tokens/mobile', {
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) throw new Error('Failed to load mobile tokens');
this.mobileTokens = await res.json();
} catch (e) {
this.tokenError = e.message;
} finally {
this.loadingTokens = false;
}
},
async loadDevices() {
this.loadingDevices = true;
this.deviceError = null;
try {
const res = await fetch('/api/mobile/devices', {
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) throw new Error('Failed to load devices');
this.devices = await res.json();
} catch (e) {
this.deviceError = e.message;
} finally {
this.loadingDevices = false;
}
},
async revokeToken(token) {
if (!confirm({{ _("devices.confirm_revoke_token") | tojson }})) return;
this.actingToken = token.id;
try {
const res = await fetch(`/api/api-tokens/${token.id}`, {
method: 'DELETE',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to revoke token');
}
await this.loadMobileTokens();
this._showBanner({{ _("devices.token_revoked_success") | tojson }}, false);
} catch (e) {
this._showBanner(e.message, true);
} finally {
this.actingToken = null;
}
},
async reactivateMobileToken(token) {
if (!confirm({{ _("devices.reactivate_token_confirm") | tojson }})) return;
this.actingToken = token.id;
try {
const res = await fetch(`/api/api-tokens/${token.id}/reactivate`, {
method: 'POST',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to reactivate token');
}
await this.loadMobileTokens();
this._showBanner({{ _("devices.token_reactivated_success") | tojson }}, false);
} catch (e) {
this._showBanner(e.message, true);
} finally {
this.actingToken = null;
}
},
async deleteMobileToken(token) {
if (!confirm({{ _("devices.delete_token_confirm") | tojson }})) return;
this.actingToken = token.id;
try {
const res = await fetch(`/api/api-tokens/${token.id}`, {
method: 'DELETE',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to delete token');
}
await this.loadMobileTokens();
this._showBanner({{ _("devices.token_deleted_success") | tojson }}, false);
} catch (e) {
this._showBanner(e.message, true);
} finally {
this.actingToken = null;
}
},
async deactivateDevice(device) {
if (!confirm({{ _("devices.confirm_deactivate_device") | tojson }})) return;
this.actingDevice = device.id;
try {
const res = await fetch(`/api/mobile/devices/${device.id}`, {
method: 'DELETE',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to remove device');
}
await this.loadDevices();
this._showBanner({{ _("devices.device_removed_success") | tojson }}, false);
} catch (e) {
this._showBanner(e.message, true);
} finally {
this.actingDevice = null;
}
},
async deleteDevice(device) {
if (!confirm({{ _("devices.delete_device_confirm") | tojson }})) return;
this.actingDevice = device.id;
try {
const res = await fetch(`/api/mobile/devices/${device.id}`, {
method: 'DELETE',
headers: { 'X-CSRF-Token': csrfToken },
});
if (!res.ok) {
const data = await res.json().catch(() => ({}));
throw new Error(data.detail || 'Failed to delete device');
}
await this.loadDevices();
this._showBanner({{ _("devices.device_deleted_success") | tojson }}, false);
} catch (e) {
this._showBanner(e.message, true);
} finally {
this.actingDevice = null;
}
},
/** Extract the device name from the full token name (e.g. "Mobile App – iPhone 15 Pro" → "iPhone 15 Pro"). */
formatDeviceName(name) {
if (!name) return 'Unknown Device';
// Match either em dash (–) or hyphen (-) separators used by the mobile flows.
const match = name.match(/[–\-]\s*(.+)$/);
return match ? match[1].trim() : name;
},
formatDate(d) {
if (!d) return '—';
const dt = new Date(d);
return dt.toLocaleDateString(undefined, { year: 'numeric', month: 'short', day: 'numeric' }) +
' ' + dt.toLocaleTimeString(undefined, { hour: '2-digit', minute: '2-digit' });
},
_showBanner(msg, isError) {
this.banner = { visible: true, error: isError, message: msg };
setTimeout(() => { this.banner.visible = false; }, 5000);
},
};
}
</script>
{% endblock %}
frontend/templates/dropbox.html
+65 -3
View File
@@ -104,7 +104,7 @@
<h3 class="text-xl font-medium mb-4">Step 3: Set OAuth 2 Redirect URI</h3>
<ol class="list-decimal ml-6 space-y-3">
<li>In your app's settings page, go to the "OAuth 2" section</li>
<li>Add a redirect URI: <code class="bg-gray-100 p-1">{{ request.url.scheme }}://{{ request.url.netloc }}/dropbox-callback</code></li>
<li>Add a redirect URI: <code class="bg-gray-100 p-1">{{ callback_url }}</code></li>
<li>Click "Add" to save the redirect URI</li>
</ol>
</div>
@@ -116,6 +116,34 @@
</h2>
<div class="space-y-4">
{% if user_mode and global_creds_available %}
<!-- Global credentials mode: no app credentials required from the user -->
<div class="bg-blue-50 border border-blue-200 rounded-md p-4" role="note">
<div class="flex items-start">
<svg class="h-5 w-5 text-blue-400 mt-0.5 mr-3 flex-shrink-0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true">
<path fill-rule="evenodd" d="M18 10a8 8 0 11-16 0 8 8 0 0116 0zm-7-4a1 1 0 11-2 0 1 1 0 012 0zM9 9a1 1 0 000 2v3a1 1 0 001 1h1a1 1 0 100-2v-3a1 1 0 00-1-1H9z" clip-rule="evenodd" />
</svg>
<div>
<p class="text-sm font-medium text-blue-800">Using shared application credentials</p>
<p class="text-sm text-blue-700 mt-1">Your administrator has enabled shared Dropbox app credentials. You can authorize your account without supplying your own App Key and Secret.</p>
</div>
</div>
</div>
{% if folder_path %}
<div class="bg-gray-50 border border-gray-200 rounded-md px-3 py-2">
<p class="text-xs text-gray-500">Target folder (from integration settings)</p>
<p class="text-sm font-mono text-gray-700">{{ folder_path }}</p>
</div>
{% endif %}
<div>
<button id="start-auth-flow-global" class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-blue-600 hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-blue-500">
<i class="fab fa-dropbox mr-2" aria-hidden="true"></i>
Authorize with Dropbox
</button>
</div>
{% else %}
<div>
<label for="app-key" class="block text-sm font-medium text-gray-700">App Key <span class="text-red-500" aria-hidden="true">*</span></label>
<input type="text" id="app-key" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your Dropbox app key" value="{{ app_key_value }}">
@@ -146,6 +174,7 @@
Start Authentication Flow
</button>
</div>
{% endif %}
<!-- Token validation and status (admin mode only) -->
{% if not user_mode %}
@@ -268,6 +297,10 @@ DROPBOX_FOLDER={{ folder_path|default('/Documents/Uploads', true) }}</code></pre
<script>
document.addEventListener('DOMContentLoaded', function() {
const userMode = {{ 'true' if user_mode else 'false' }};
const globalCredsAvailable = {{ 'true' if global_creds_available else 'false' }};
// Redirect URI for OAuth: prefer server-provided value (respects PUBLIC_BASE_URL),
// fall back to window.location.origin for resilience.
const dropboxCallbackUrl = {{ callback_url | tojson }} || (window.location.origin + "/dropbox-callback");
// Store integration_id if provided (for per-user OAuth flow)
const integrationId = "{{ integration_id or '' }}";
@@ -277,6 +310,7 @@ document.addEventListener('DOMContentLoaded', function() {
// Elements
const startAuthFlowBtn = document.getElementById('start-auth-flow');
const startAuthFlowGlobalBtn = document.getElementById('start-auth-flow-global');
const testTokenBtn = document.getElementById('test-token');
const refreshTokenBtn = document.getElementById('refresh-token-btn');
const tokenStatus = document.getElementById('token-status');
@@ -328,11 +362,38 @@ document.addEventListener('DOMContentLoaded', function() {
}
});
// Global-credentials "Authorize with Dropbox" button (user mode, admin-provided creds)
if (startAuthFlowGlobalBtn) {
startAuthFlowGlobalBtn.addEventListener('click', async function() {
startAuthFlowGlobalBtn.disabled = true;
startAuthFlowGlobalBtn.innerHTML = '<span class="animate-spin inline-block mr-2">⟳</span> Redirecting…';
try {
const resp = await fetch('/api/dropbox/global-authorize-url');
if (!resp.ok) {
const err = await resp.json().catch(() => ({}));
showModal('error', 'Error', err.detail || 'Could not retrieve authorization URL.');
startAuthFlowGlobalBtn.disabled = false;
startAuthFlowGlobalBtn.innerHTML = '<i class="fab fa-dropbox mr-2" aria-hidden="true"></i>Authorize with Dropbox';
return;
}
const data = await resp.json();
// Signal to the callback that global credentials should be used for the exchange
sessionStorage.setItem('dropbox_use_global_creds', 'true');
window.location.href = data.authorize_url;
} catch (err) {
showModal('error', 'Network Error', err.message || 'Unknown error');
startAuthFlowGlobalBtn.disabled = false;
startAuthFlowGlobalBtn.innerHTML = '<i class="fab fa-dropbox mr-2" aria-hidden="true"></i>Authorize with Dropbox';
}
});
}
// Start Authentication Flow button click
startAuthFlowBtn.addEventListener('click', function() {
if (startAuthFlowBtn) {
startAuthFlowBtn.addEventListener('click', function() {
const appKey = document.getElementById('app-key').value.trim();
const appSecret = appSecretInput.value.trim();
const redirectUri = window.location.origin + "/dropbox-callback";
const redirectUri = dropboxCallbackUrl;
if (!appKey) {
showModal('error', 'Validation Error', 'Please enter your App Key');
@@ -362,6 +423,7 @@ document.addEventListener('DOMContentLoaded', function() {
// Redirect the user to the Dropbox login page
window.location.href = authUrl;
});
} // end if (startAuthFlowBtn)
// Test Token button click (admin mode only)
if (testTokenBtn) {
frontend/templates/dropbox_callback.html
+252 -12
View File
@@ -59,6 +59,40 @@
</div>
</div>
<!-- Folder selector (shown after authorization) -->
<div id="folder-selector" class="hidden mt-6 p-4 bg-white border border-gray-200 rounded-lg">
<h3 class="font-medium text-lg mb-3">
<svg class="inline-block h-5 w-5 mr-1 text-gray-500" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true">
<path fill-rule="evenodd" d="M2 6a2 2 0 012-2h4l2 2h4a2 2 0 012 2v1H8a3 3 0 00-3 3v1.5a1.5 1.5 0 01-3 0V6z" clip-rule="evenodd" />
<path d="M6 12a2 2 0 012-2h8a2 2 0 012 2v2a2 2 0 01-2 2H2h2a2 2 0 002-2v-2z" />
</svg>
Select Folder
</h3>
<p class="text-sm text-gray-600 mb-3">Browse your Dropbox to select a folder for this integration.</p>
<div class="mb-3">
<div id="folder-breadcrumb" class="flex items-center text-sm text-gray-500 mb-2">
<button type="button" class="folder-nav-btn text-indigo-600 hover:text-indigo-800 font-medium" data-path="" aria-label="Navigate to root folder">/ Root</button>
</div>
</div>
<div id="folder-list" class="border border-gray-200 rounded-md max-h-64 overflow-y-auto">
<div class="p-4 text-center text-gray-500">
<div class="animate-spin inline-block h-5 w-5 border-2 border-gray-300 border-t-indigo-600 rounded-full mb-2" role="status" aria-label="Loading folders"></div>
<p class="text-sm">Loading folders…</p>
</div>
</div>
<div class="mt-3 flex items-center gap-3">
<label for="selected-folder-path" class="text-sm font-medium text-gray-700 whitespace-nowrap">Selected folder:</label>
<input id="selected-folder-path" type="text" class="flex-1 px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 text-sm" placeholder="/" />
<button id="save-folder-btn" type="button" class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500">
Save Folder
</button>
</div>
<p id="folder-save-status" class="mt-2 text-sm hidden" aria-live="polite"></p>
</div>
<div class="mt-6 p-4 bg-gray-100 rounded-md">
<h3 class="font-medium text-lg mb-2">Configuration for Worker Nodes</h3>
<p class="text-sm text-gray-600 mb-3">
@@ -94,6 +128,7 @@ document.addEventListener('DOMContentLoaded', function() {
const code = "{{ code }}";
// Get credentials from session storage (these take precedence over server-provided values)
const useGlobalCreds = sessionStorage.getItem('dropbox_use_global_creds') === 'true';
const appKey = sessionStorage.getItem('dropbox_app_key') || "{{ app_key_value }}";
const appSecret = sessionStorage.getItem('dropbox_app_secret') || "{{ app_secret_value }}";
const folderPath = sessionStorage.getItem('dropbox_folder_path') || "{{ folder_path }}" || '/Documents/Uploads';
@@ -107,20 +142,100 @@ document.addEventListener('DOMContentLoaded', function() {
}
}
const redirectUri = window.location.origin + "/dropbox-callback";
// Use server-provided callback URL (respects PUBLIC_BASE_URL) with fallback to window.location.origin
const redirectUri = {{ callback_url | tojson }} || (window.location.origin + "/dropbox-callback");
// Automatically exchange the code for a refresh token
if (code) {
if (!appKey || !appSecret) {
showError("Missing App Key or App Secret. Please go back to the setup page and try again.");
return;
if (useGlobalCreds) {
// Global credentials: the server handles the exchange using the admin app secret
exchangeCodeGlobal(code, redirectUri);
} else {
if (!appKey || !appSecret) {
showError("Missing App Key or App Secret. Please go back to the setup page and try again.");
return;
}
exchangeCode(code, appKey, appSecret, redirectUri);
}
exchangeCode(code, appKey, appSecret, redirectUri);
} else {
showError("No authorization code was found in the URL");
}
function exchangeCodeGlobal(code, redirectUri) {
const formData = new FormData();
formData.append('code', code);
formData.append('redirect_uri', redirectUri);
document.getElementById('processing-message').innerHTML =
'<p>Exchanging authorization code using shared credentials…</p>';
fetch('/api/dropbox/exchange-token-global', {
method: 'POST',
body: formData
})
.then(response => {
if (!response.ok) {
return response.json().then(err => {
throw new Error(err.detail || 'Failed to exchange token');
});
}
return response.json();
})
.then(data => {
if (data.refresh_token) {
const resolvedAppKey = data.app_key || '';
if (integrationId) {
const creds = {
refresh_token: data.refresh_token,
// Store public app_key with the integration (no secret stored browser-side)
app_key: resolvedAppKey,
// Flag so the backend knows to use global app_secret for future operations
use_global_app_secret: true,
};
const body = { credentials: creds };
document.getElementById('processing-message').innerHTML =
'<p>Saving credentials to your integration…</p>';
return fetch(`/api/integrations/${integrationId}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
}).then(response => {
if (!response.ok) {
return response.json().then(err => {
throw new Error('Failed to save credentials: ' + (err.detail || 'Unknown error'));
});
}
return response.json();
}).then(() => {
// Clean up
sessionStorage.removeItem('dropbox_use_global_creds');
sessionStorage.removeItem('oauth_integration_id');
document.getElementById('processing-message').innerHTML =
'<p class="text-green-600 font-semibold">✓ Dropbox authorized successfully!</p>' +
'<p class="text-sm text-gray-500 mt-2">Redirecting to Integrations…</p>';
document.querySelector('.animate-spin').parentNode.classList.add('hidden');
setTimeout(() => { window.location.href = '/integrations'; }, 2000);
});
}
// Global admin flow — the exchange-token-global endpoint is intended for
// per-user integrations, so reaching here without an integrationId is unexpected.
console.warn('dropbox_callback: global creds flow reached without integration_id');
sessionStorage.removeItem('dropbox_use_global_creds');
showSuccess(data.refresh_token, resolvedAppKey, '', folderPath);
setTimeout(() => { window.location.href = '/status'; }, 10000);
} else {
throw new Error('No refresh token was received from the server');
}
})
.catch(error => {
showError(error.message);
});
}
function exchangeCode(code, appKey, appSecret, redirectUri) {
const formData = new FormData();
formData.append('client_id', appKey);
@@ -173,18 +288,21 @@ document.addEventListener('DOMContentLoaded', function() {
}
return response.json();
}).then(() => {
// Clean up
// Clean up session storage
sessionStorage.removeItem('dropbox_app_key');
sessionStorage.removeItem('dropbox_app_secret');
sessionStorage.removeItem('dropbox_folder_path');
sessionStorage.removeItem('oauth_integration_id');
sessionStorage.removeItem('dropbox_use_system_creds');
// Show brief success then redirect to integrations
document.getElementById('processing-message').innerHTML =
'<p class="text-green-600 font-semibold">✓ Dropbox authorized successfully!</p>' +
'<p class="text-sm text-gray-500 mt-2">Redirecting to Integrations...</p>';
// Hide processing spinner, show success
document.querySelector('.animate-spin').parentNode.classList.add('hidden');
setTimeout(() => { window.location.href = '/integrations'; }, 2000);
document.getElementById('processing-message').innerHTML =
'<p class="text-green-600 font-semibold">✓ Dropbox authorized successfully!</p>';
document.getElementById('success-container').classList.remove('hidden');
// Show folder browser with the access token
initFolderBrowser(data.access_token, integrationId);
});
}
@@ -279,6 +397,128 @@ DROPBOX_FOLDER=${folderPath || '/Documents/Uploads'}`;
});
}
}
// ── Folder browser ────────────────────────────────────────────────
function escapeHtml(str) {
const div = document.createElement('div');
div.appendChild(document.createTextNode(str));
return div.innerHTML;
}
function initFolderBrowser(accessToken, integrationId) {
const folderSelector = document.getElementById('folder-selector');
if (!folderSelector || !integrationId) return;
folderSelector.classList.remove('hidden');
let currentPath = '';
const folderList = document.getElementById('folder-list');
const breadcrumb = document.getElementById('folder-breadcrumb');
const selectedInput = document.getElementById('selected-folder-path');
const saveBtn = document.getElementById('save-folder-btn');
const saveStatus = document.getElementById('folder-save-status');
function loadFolders(path) {
currentPath = path;
folderList.innerHTML = '<div class="p-4 text-center text-gray-500"><div class="animate-spin inline-block h-5 w-5 border-2 border-gray-300 border-t-indigo-600 rounded-full mb-2" role="status" aria-label="Loading folders"></div><p class="text-sm">Loading folders…</p></div>';
const formData = new FormData();
formData.append('access_token', accessToken);
formData.append('path', path);
fetch('/api/dropbox/list-folders', { method: 'POST', body: formData })
.then(r => r.json())
.then(data => {
if (data.folders && data.folders.length > 0) {
folderList.innerHTML = data.folders.map(f =>
`<button type="button" class="folder-item w-full text-left px-4 py-3 hover:bg-indigo-50 border-b border-gray-100 flex items-center gap-3 text-sm focus:outline-none focus:bg-indigo-50" data-path="${escapeHtml(f.path)}" aria-label="Open folder ${escapeHtml(f.name)}">` +
`<svg class="h-5 w-5 text-yellow-400 flex-shrink-0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true"><path fill-rule="evenodd" d="M2 6a2 2 0 012-2h4l2 2h4a2 2 0 012 2v1H8a3 3 0 00-3 3v1.5a1.5 1.5 0 01-3 0V6z" clip-rule="evenodd" /><path d="M6 12a2 2 0 012-2h8a2 2 0 012 2v2a2 2 0 01-2 2H2h2a2 2 0 002-2v-2z" /></svg>` +
`<span class="font-medium text-gray-700">${escapeHtml(f.name)}</span>` +
`</button>`
).join('');
folderList.querySelectorAll('.folder-item').forEach(btn => {
btn.addEventListener('click', () => {
const p = btn.getAttribute('data-path');
selectedInput.value = p;
loadFolders(p);
});
});
} else {
folderList.innerHTML = '<div class="p-4 text-center text-gray-400 text-sm">No subfolders found</div>';
}
updateBreadcrumb(path);
})
.catch(err => {
folderList.innerHTML = `<div class="p-4 text-center text-red-500 text-sm">Failed to load folders: ${escapeHtml(err.message)}</div>`;
});
}
function updateBreadcrumb(path) {
const parts = path.split('/').filter(Boolean);
let html = '<button type="button" class="folder-nav-btn text-indigo-600 hover:text-indigo-800 font-medium" data-path="" aria-label="Navigate to root folder">/ Root</button>';
let accumulated = '';
for (const part of parts) {
accumulated += '/' + part;
html += `<span class="mx-1 text-gray-400">/</span>`;
html += `<button type="button" class="folder-nav-btn text-indigo-600 hover:text-indigo-800 font-medium" data-path="${escapeHtml(accumulated)}" aria-label="Navigate to ${escapeHtml(part)}">${escapeHtml(part)}</button>`;
}
breadcrumb.innerHTML = html;
breadcrumb.querySelectorAll('.folder-nav-btn').forEach(btn => {
btn.addEventListener('click', () => {
const p = btn.getAttribute('data-path');
selectedInput.value = p || '/';
loadFolders(p);
});
});
}
// Save selected folder to integration config
saveBtn.addEventListener('click', () => {
const folderPath = selectedInput.value.trim() || '/';
saveBtn.disabled = true;
saveBtn.textContent = 'Saving…';
// Get the current integration config, update folder_path, then PUT back
fetch(`/api/integrations/${integrationId}`)
.then(r => r.json())
.then(intg => {
const cfg = intg.config || {};
// Update the correct folder key based on integration type
if (cfg.source_type) {
cfg.folder_path = folderPath;
} else {
cfg.folder = folderPath;
}
return fetch(`/api/integrations/${integrationId}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ config: cfg }),
});
})
.then(r => {
if (!r.ok) throw new Error('Failed to save folder');
return r.json();
})
.then(() => {
saveStatus.textContent = '✓ Folder saved! Redirecting…';
saveStatus.className = 'mt-2 text-sm text-green-600';
saveStatus.classList.remove('hidden');
saveBtn.textContent = 'Saved ✓';
setTimeout(() => { window.location.href = '/integrations'; }, 1500);
})
.catch(err => {
saveStatus.textContent = 'Error: ' + err.message;
saveStatus.className = 'mt-2 text-sm text-red-600';
saveStatus.classList.remove('hidden');
saveBtn.disabled = false;
saveBtn.textContent = 'Save Folder';
});
});
// Load root folders initially
loadFolders('');
}
});
</script>
{% endblock %}
frontend/templates/files.html
+1
View File
@@ -524,6 +524,7 @@
<option value="webdav" {% if storage_provider == "webdav" %}selected{% endif %}>WebDAV</option>
<option value="ftp" {% if storage_provider == "ftp" %}selected{% endif %}>FTP</option>
<option value="sftp" {% if storage_provider == "sftp" %}selected{% endif %}>SFTP</option>
<option value="sharepoint" {% if storage_provider == "sharepoint" %}selected{% endif %}>SharePoint</option>
<option value="icloud" {% if storage_provider == "icloud" %}selected{% endif %}>iCloud Drive</option>
</select>
</div>
frontend/templates/google_drive.html
+46 -8
View File
@@ -167,14 +167,31 @@
<p class="mb-4">Now enter your Client ID and Client Secret below, and we'll help you complete the OAuth flow. You can set the folder ID after authentication is complete.</p>
<div class="space-y-4">
<div>
<label for="client-id" class="block text-sm font-medium text-gray-700">Client ID</label>
<input type="text" id="client-id" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your OAuth Client ID" value="{{ client_id_value }}">
{% if user_mode and has_system_credentials %}
<!-- System credentials toggle (user mode only) -->
<div class="bg-green-50 border border-green-200 rounded-md p-4">
<label class="flex items-center gap-3 cursor-pointer">
<input type="checkbox" id="use-system-creds" checked class="rounded border-gray-300 text-green-600 focus:ring-green-500 h-5 w-5" />
<div>
<span class="text-sm font-medium text-green-800">Use DocuElevate's app credentials</span>
<p class="text-xs text-green-600 mt-0.5">Recommended — authorize with your Google account without needing your own Google Cloud app registration.</p>
</div>
</label>
</div>
{% endif %}
<div>
<label for="client-secret" class="block text-sm font-medium text-gray-700">Client Secret</label>
<input type="password" id="client-secret" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your OAuth Client Secret" value="{{ client_secret_value }}">
<div id="custom-creds-section" {% if user_mode and has_system_credentials %}class="hidden"{% endif %}>
<div class="space-y-4">
<div>
<label for="client-id" class="block text-sm font-medium text-gray-700">Client ID</label>
<input type="text" id="client-id" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your OAuth Client ID" value="{{ client_id_value if not (user_mode and has_system_credentials) else '' }}">
</div>
<div>
<label for="client-secret" class="block text-sm font-medium text-gray-700">Client Secret</label>
<input type="password" id="client-secret" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your OAuth Client Secret" value="{{ client_secret_value if not (user_mode and has_system_credentials) else '' }}">
</div>
</div>
</div>
<div>
@@ -430,6 +447,9 @@ GOOGLE_DRIVE_FOLDER_ID={{ folder_id|default('YOUR_FOLDER_ID', true) }}
<script>
document.addEventListener('DOMContentLoaded', function() {
const userMode = {{ 'true' if user_mode else 'false' }};
const hasSystemCredentials = {{ 'true' if (user_mode and has_system_credentials) else 'false' }};
const systemClientId = {{ (client_id_value or '') | tojson }};
const systemClientSecret = {{ (client_secret_value or '') | tojson }};
// Store integration_id if provided (for per-user OAuth flow)
const integrationId = "{{ integration_id or '' }}";
@@ -437,6 +457,19 @@ document.addEventListener('DOMContentLoaded', function() {
sessionStorage.setItem('oauth_integration_id', integrationId);
}
// System credentials toggle
const useSystemCredsCheckbox = document.getElementById('use-system-creds');
const customCredsSection = document.getElementById('custom-creds-section');
if (useSystemCredsCheckbox && customCredsSection) {
useSystemCredsCheckbox.addEventListener('change', function() {
if (this.checked) {
customCredsSection.classList.add('hidden');
} else {
customCredsSection.classList.remove('hidden');
}
});
}
// Form elements
const clientIdInput = document.getElementById('client-id');
const clientSecretInput = document.getElementById('client-secret');
@@ -719,8 +752,10 @@ GOOGLE_DRIVE_FOLDER_ID=${folderId}
// Start OAuth flow button
if (startOauthFlowBtn) {
startOauthFlowBtn.addEventListener('click', function() {
const clientId = clientIdInput.value.trim();
const clientSecret = clientSecretInput.value.trim();
// Determine which credentials to use
const useSystemCreds = hasSystemCredentials && useSystemCredsCheckbox && useSystemCredsCheckbox.checked;
const clientId = useSystemCreds ? systemClientId : clientIdInput.value.trim();
const clientSecret = useSystemCreds ? systemClientSecret : clientSecretInput.value.trim();
const folderId = folderIdInput ? folderIdInput.value.trim() : '';
if (!clientId) {
@@ -736,6 +771,9 @@ GOOGLE_DRIVE_FOLDER_ID=${folderId}
// Save values to session storage for use after redirect
sessionStorage.setItem('google_drive_client_id', clientId);
sessionStorage.setItem('google_drive_client_secret', clientSecret);
if (useSystemCreds) {
sessionStorage.setItem('google_drive_use_system_creds', 'true');
}
// In admin mode store folder_id; in user mode the config is already set
if (!userMode && folderId) {
frontend/templates/help.html
+9 -1
View File
@@ -33,7 +33,7 @@
"name": "Which cloud storage providers does DocuElevate support?",
"acceptedAnswer": {
"@type": "Answer",
"text": "DocuElevate integrates with Dropbox, Google Drive, OneDrive, Amazon S3, Nextcloud, WebDAV, SFTP, FTP, and Paperless-ngx."
"text": "DocuElevate integrates with Dropbox, Google Drive, OneDrive, SharePoint, Amazon S3, Nextcloud, WebDAV, SFTP, FTP, and Paperless-ngx."
}
},
{
@@ -188,6 +188,14 @@
</div>
</div>
<div class="flex items-start bg-white shadow rounded-lg p-5">
<i class="fab fa-microsoft text-purple-600 text-xl mr-3 mt-0.5" aria-hidden="true"></i>
<div>
<h3 class="font-semibold text-gray-800 text-sm">SharePoint</h3>
<p class="text-gray-500 text-xs">Upload to SharePoint Online document libraries via Graph API.</p>
</div>
</div>
<div class="flex items-start bg-white shadow rounded-lg p-5">
<i class="fab fa-aws text-yellow-600 text-xl mr-3 mt-0.5" aria-hidden="true"></i>
<div>
frontend/templates/integrations_dashboard.html
+1 -1
View File
@@ -1392,7 +1392,7 @@ function integrationsDashboard() {
body: JSON.stringify({
integration_type: intg.integration_type,
config: intg.config,
credentials: creds,
credentials: creds.credentials,
}),
});
const data = await resp.json();
frontend/templates/onedrive.html
+53 -14
View File
@@ -130,20 +130,37 @@
</h2>
<div class="space-y-4">
<div>
<label for="client-id" class="block text-sm font-medium text-gray-700">Client ID <span class="text-red-500" aria-hidden="true">*</span></label>
<input type="text" id="client-id" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your Azure AD application client ID" value="{{ client_id_value }}">
{% if user_mode and has_system_credentials %}
<!-- System credentials toggle (user mode only) -->
<div class="bg-green-50 border border-green-200 rounded-md p-4">
<label class="flex items-center gap-3 cursor-pointer">
<input type="checkbox" id="use-system-creds" checked class="rounded border-gray-300 text-green-600 focus:ring-green-500 h-5 w-5" />
<div>
<span class="text-sm font-medium text-green-800">Use DocuElevate's app credentials</span>
<p class="text-xs text-green-600 mt-0.5">Recommended — authorize with your OneDrive account without needing your own Azure app registration.</p>
</div>
</label>
</div>
{% endif %}
<div>
<label for="client-secret" class="block text-sm font-medium text-gray-700">Client Secret <span class="text-red-500" aria-hidden="true">*</span></label>
<input type="password" id="client-secret" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your Azure AD application client secret" value="{{ client_secret_value }}">
</div>
<div id="custom-creds-section" {% if user_mode and has_system_credentials %}class="hidden"{% endif %}>
<div class="space-y-4">
<div>
<label for="client-id" class="block text-sm font-medium text-gray-700">Client ID <span class="text-red-500" aria-hidden="true">*</span></label>
<input type="text" id="client-id" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your Azure AD application client ID" value="{{ client_id_value if not (user_mode and has_system_credentials) else '' }}">
</div>
<div>
<label for="tenant-id" class="block text-sm font-medium text-gray-700">Tenant ID (Optional)</label>
<input type="text" id="tenant-id" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="common" value="{{ tenant_id }}">
<p class="text-xs text-gray-500 mt-1">Use "common" for personal accounts or your organization's Tenant ID for corporate accounts</p>
<div>
<label for="client-secret" class="block text-sm font-medium text-gray-700">Client Secret <span class="text-red-500" aria-hidden="true">*</span></label>
<input type="password" id="client-secret" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="Enter your Azure AD application client secret" value="{{ client_secret_value if not (user_mode and has_system_credentials) else '' }}">
</div>
<div>
<label for="tenant-id" class="block text-sm font-medium text-gray-700">Tenant ID (Optional)</label>
<input type="text" id="tenant-id" class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm py-2 px-3 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" placeholder="common" value="{{ tenant_id }}">
<p class="text-xs text-gray-500 mt-1">Use "common" for personal accounts or your organization's Tenant ID for corporate accounts</p>
</div>
</div>
</div>
{% if not user_mode %}
@@ -289,6 +306,10 @@ ONEDRIVE_FOLDER_PATH={{ folder_path|default('Documents/Uploads', true) }}</code>
<script>
document.addEventListener('DOMContentLoaded', function() {
const userMode = {{ 'true' if user_mode else 'false' }};
const hasSystemCredentials = {{ 'true' if (user_mode and has_system_credentials) else 'false' }};
const systemClientId = {{ (client_id_value or '') | tojson }};
const systemClientSecret = {{ (client_secret_value or '') | tojson }};
const systemTenantId = {{ (tenant_id or 'common') | tojson }};
// Store integration_id if provided (for per-user OAuth flow)
const integrationId = "{{ integration_id or '' }}";
@@ -302,6 +323,19 @@ document.addEventListener('DOMContentLoaded', function() {
const refreshTokenBtn = document.getElementById('refresh-token-btn');
const tokenStatus = document.getElementById('token-status');
const clientSecretInput = document.getElementById('client-secret');
const useSystemCredsCheckbox = document.getElementById('use-system-creds');
const customCredsSection = document.getElementById('custom-creds-section');
// Toggle custom credentials section visibility
if (useSystemCredsCheckbox && customCredsSection) {
useSystemCredsCheckbox.addEventListener('change', function() {
if (this.checked) {
customCredsSection.classList.add('hidden');
} else {
customCredsSection.classList.remove('hidden');
}
});
}
// Modal elements
const resultModal = document.getElementById('resultModal');
@@ -351,10 +385,12 @@ document.addEventListener('DOMContentLoaded', function() {
// Start Authentication Flow button click
startAuthFlowBtn.addEventListener('click', function() {
const clientId = document.getElementById('client-id').value.trim();
const clientSecret = clientSecretInput.value.trim();
// Determine which credentials to use
const useSystemCreds = hasSystemCredentials && useSystemCredsCheckbox && useSystemCredsCheckbox.checked;
const clientId = useSystemCreds ? systemClientId : document.getElementById('client-id').value.trim();
const clientSecret = useSystemCreds ? systemClientSecret : clientSecretInput.value.trim();
const redirectUri = window.location.origin + "/onedrive-callback";
const tenantId = document.getElementById('tenant-id').value.trim() || 'common';
const tenantId = useSystemCreds ? systemTenantId : (document.getElementById('tenant-id').value.trim() || 'common');
if (!clientId) {
showModal('error', 'Validation Error', 'Please enter your Client ID');
@@ -370,6 +406,9 @@ document.addEventListener('DOMContentLoaded', function() {
sessionStorage.setItem('onedrive_client_id', clientId);
sessionStorage.setItem('onedrive_client_secret', clientSecret);
sessionStorage.setItem('onedrive_tenant_id', tenantId);
if (useSystemCreds) {
sessionStorage.setItem('onedrive_use_system_creds', 'true');
}
// In admin mode also store folder path; in user mode the config is already set
if (!userMode) {
frontend/templates/onedrive_callback.html
+157 -6
View File
@@ -59,6 +59,40 @@
</div>
</div>
<!-- Folder selector (shown after authorization) -->
<div id="folder-selector" class="hidden mt-6 p-4 bg-white border border-gray-200 rounded-lg">
<h3 class="font-medium text-lg mb-3">
<svg class="inline-block h-5 w-5 mr-1 text-gray-500" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true">
<path fill-rule="evenodd" d="M2 6a2 2 0 012-2h4l2 2h4a2 2 0 012 2v1H8a3 3 0 00-3 3v1.5a1.5 1.5 0 01-3 0V6z" clip-rule="evenodd" />
<path d="M6 12a2 2 0 012-2h8a2 2 0 012 2v2a2 2 0 01-2 2H2h2a2 2 0 002-2v-2z" />
</svg>
Select Folder
</h3>
<p class="text-sm text-gray-600 mb-3">Browse your OneDrive to select a folder for this integration.</p>
<div class="mb-3">
<div id="folder-breadcrumb" class="flex items-center text-sm text-gray-500 mb-2">
<button type="button" class="folder-nav-btn text-indigo-600 hover:text-indigo-800 font-medium" data-path="" aria-label="Navigate to root folder">/ Root</button>
</div>
</div>
<div id="folder-list" class="border border-gray-200 rounded-md max-h-64 overflow-y-auto">
<div class="p-4 text-center text-gray-500">
<div class="animate-spin inline-block h-5 w-5 border-2 border-gray-300 border-t-indigo-600 rounded-full mb-2" role="status" aria-label="Loading folders"></div>
<p class="text-sm">Loading folders…</p>
</div>
</div>
<div class="mt-3 flex items-center gap-3">
<label for="selected-folder-path" class="text-sm font-medium text-gray-700 whitespace-nowrap">Selected folder:</label>
<input id="selected-folder-path" type="text" class="flex-1 px-3 py-2 border border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 text-sm" placeholder="/" />
<button id="save-folder-btn" type="button" class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500">
Save Folder
</button>
</div>
<p id="folder-save-status" class="mt-2 text-sm hidden" aria-live="polite"></p>
</div>
<div class="mt-6 p-4 bg-gray-100 rounded-md">
<h3 class="font-medium text-lg mb-2">Configuration for Worker Nodes</h3>
<p class="text-sm text-gray-600 mb-3">
@@ -177,19 +211,22 @@ document.addEventListener('DOMContentLoaded', function() {
}
return response.json();
}).then(() => {
// Clean up
// Clean up session storage
sessionStorage.removeItem('onedrive_client_id');
sessionStorage.removeItem('onedrive_client_secret');
sessionStorage.removeItem('onedrive_tenant_id');
sessionStorage.removeItem('onedrive_folder_path');
sessionStorage.removeItem('oauth_integration_id');
sessionStorage.removeItem('onedrive_use_system_creds');
// Show brief success then redirect to integrations
document.getElementById('processing-message').innerHTML =
'<p class="text-green-600 font-semibold">✓ OneDrive authorized successfully!</p>' +
'<p class="text-sm text-gray-500 mt-2">Redirecting to Integrations...</p>';
// Hide processing spinner, show success
document.querySelector('.animate-spin').parentNode.classList.add('hidden');
setTimeout(() => { window.location.href = '/integrations'; }, 2000);
document.getElementById('processing-message').innerHTML =
'<p class="text-green-600 font-semibold">✓ OneDrive authorized successfully!</p>';
document.getElementById('success-container').classList.remove('hidden');
// Show folder browser with the access token
initFolderBrowser(data.access_token, integrationId);
});
}
@@ -291,6 +328,120 @@ ONEDRIVE_FOLDER_PATH=${folderPath || 'Documents/Uploads'}`;
});
}
}
// ── Folder browser ────────────────────────────────────────────────
function escapeHtml(str) {
const div = document.createElement('div');
div.appendChild(document.createTextNode(str));
return div.innerHTML;
}
function initFolderBrowser(accessToken, integrationId) {
const folderSelector = document.getElementById('folder-selector');
if (!folderSelector || !integrationId || !accessToken) return;
folderSelector.classList.remove('hidden');
const folderList = document.getElementById('folder-list');
const breadcrumb = document.getElementById('folder-breadcrumb');
const selectedInput = document.getElementById('selected-folder-path');
const saveBtn = document.getElementById('save-folder-btn');
const saveStatus = document.getElementById('folder-save-status');
function loadFolders(path) {
folderList.innerHTML = '<div class="p-4 text-center text-gray-500"><div class="animate-spin inline-block h-5 w-5 border-2 border-gray-300 border-t-indigo-600 rounded-full mb-2" role="status" aria-label="Loading folders"></div><p class="text-sm">Loading folders…</p></div>';
const formData = new FormData();
formData.append('access_token', accessToken);
formData.append('path', path);
fetch('/api/onedrive/list-folders', { method: 'POST', body: formData })
.then(r => r.json())
.then(data => {
if (data.folders && data.folders.length > 0) {
folderList.innerHTML = data.folders.map(f =>
`<button type="button" class="folder-item w-full text-left px-4 py-3 hover:bg-indigo-50 border-b border-gray-100 flex items-center gap-3 text-sm focus:outline-none focus:bg-indigo-50" data-path="${escapeHtml(f.path)}" aria-label="Open folder ${escapeHtml(f.name)}">` +
`<svg class="h-5 w-5 text-blue-400 flex-shrink-0" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 20 20" fill="currentColor" aria-hidden="true"><path fill-rule="evenodd" d="M2 6a2 2 0 012-2h4l2 2h4a2 2 0 012 2v1H8a3 3 0 00-3 3v1.5a1.5 1.5 0 01-3 0V6z" clip-rule="evenodd" /><path d="M6 12a2 2 0 012-2h8a2 2 0 012 2v2a2 2 0 01-2 2H2h2a2 2 0 002-2v-2z" /></svg>` +
`<span class="font-medium text-gray-700">${escapeHtml(f.name)}</span>` +
`</button>`
).join('');
folderList.querySelectorAll('.folder-item').forEach(btn => {
btn.addEventListener('click', () => {
const p = btn.getAttribute('data-path');
selectedInput.value = p;
loadFolders(p.replace(/^\//, ''));
});
});
} else {
folderList.innerHTML = '<div class="p-4 text-center text-gray-400 text-sm">No subfolders found</div>';
}
updateBreadcrumb(path);
})
.catch(err => {
folderList.innerHTML = `<div class="p-4 text-center text-red-500 text-sm">Failed to load folders: ${escapeHtml(err.message)}</div>`;
});
}
function updateBreadcrumb(path) {
const parts = path.split('/').filter(Boolean);
let html = '<button type="button" class="folder-nav-btn text-indigo-600 hover:text-indigo-800 font-medium" data-path="" aria-label="Navigate to root folder">/ Root</button>';
let accumulated = '';
for (const part of parts) {
accumulated += '/' + part;
html += `<span class="mx-1 text-gray-400">/</span>`;
html += `<button type="button" class="folder-nav-btn text-indigo-600 hover:text-indigo-800 font-medium" data-path="${escapeHtml(accumulated.replace(/^\//, ''))}" aria-label="Navigate to ${escapeHtml(part)}">${escapeHtml(part)}</button>`;
}
breadcrumb.innerHTML = html;
breadcrumb.querySelectorAll('.folder-nav-btn').forEach(btn => {
btn.addEventListener('click', () => {
const p = btn.getAttribute('data-path');
selectedInput.value = p ? '/' + p : '/';
loadFolders(p);
});
});
}
// Save selected folder to integration config
saveBtn.addEventListener('click', () => {
const folderPath = selectedInput.value.trim() || '/';
saveBtn.disabled = true;
saveBtn.textContent = 'Saving…';
fetch(`/api/integrations/${integrationId}`)
.then(r => r.json())
.then(intg => {
const cfg = intg.config || {};
cfg.folder_path = folderPath;
return fetch(`/api/integrations/${integrationId}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ config: cfg }),
});
})
.then(r => {
if (!r.ok) throw new Error('Failed to save folder');
return r.json();
})
.then(() => {
saveStatus.textContent = '✓ Folder saved! Redirecting…';
saveStatus.className = 'mt-2 text-sm text-green-600';
saveStatus.classList.remove('hidden');
saveBtn.textContent = 'Saved ✓';
setTimeout(() => { window.location.href = '/integrations'; }, 1500);
})
.catch(err => {
saveStatus.textContent = 'Error: ' + err.message;
saveStatus.className = 'mt-2 text-sm text-red-600';
saveStatus.classList.remove('hidden');
saveBtn.disabled = false;
saveBtn.textContent = 'Save Folder';
});
});
// Load root folders initially
loadFolders('');
}
});
</script>
{% endblock %}
frontend/templates/profile.html
+194
View File
@@ -343,9 +343,203 @@
</section>
</template>
<!-- ── Security & Sessions card ──────────────────────────────────────── -->
<section
class="bg-white dark:bg-gray-800 shadow rounded-lg p-6 mb-6"
aria-labelledby="security-heading"
x-data="sessionManager()"
x-init="loadSessions()"
>
<h2 id="security-heading" class="text-base font-semibold text-gray-900 dark:text-white mb-1">
<i class="fas fa-shield-alt text-gray-400 mr-2" aria-hidden="true"></i>{{ _("sessions.security_heading") }}
</h2>
<p class="text-sm text-gray-500 dark:text-gray-400 mb-4">
{{ _("sessions.security_subtitle") }}
</p>
<!-- Session lifetime info -->
<div class="text-xs text-gray-400 dark:text-gray-500 mb-4" x-show="lifetimeDays > 0">
<i class="fas fa-clock mr-1" aria-hidden="true"></i>
<span x-text="'{{ _("sessions.session_lifetime") }}'.replace('{days}', lifetimeDays)"></span>
</div>
<!-- Active sessions list -->
<div class="space-y-3 mb-5">
<template x-for="session in sessions" :key="session.id">
<div
class="flex items-center justify-between border border-gray-200 dark:border-gray-700 rounded-lg p-3"
:class="session.is_current ? 'bg-blue-50 dark:bg-blue-900/20 border-blue-300 dark:border-blue-700' : ''"
>
<div class="flex items-center gap-3 min-w-0">
<i
:class="session.device_info && session.device_info.includes('iPhone') ? 'fas fa-mobile-alt' :
session.device_info && session.device_info.includes('iPad') ? 'fas fa-tablet-alt' :
session.device_info && session.device_info.includes('Android') ? 'fas fa-mobile-alt' :
session.device_info && session.device_info.includes('App') ? 'fas fa-mobile-alt' :
'fas fa-desktop'"
class="text-gray-400 text-lg flex-shrink-0"
aria-hidden="true"
></i>
<div class="min-w-0">
<div class="text-sm font-medium text-gray-900 dark:text-white truncate">
<span x-text="session.device_info || 'Unknown Device'"></span>
<span
x-show="session.is_current"
class="ml-2 inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-blue-100 text-blue-800 dark:bg-blue-900 dark:text-blue-200"
>{{ _("sessions.current_session") }}</span>
</div>
<div class="text-xs text-gray-500 dark:text-gray-400 space-x-3">
<span x-show="session.ip_address">
<i class="fas fa-globe mr-1" aria-hidden="true"></i><span x-text="session.ip_address"></span>
</span>
<span>
<i class="fas fa-clock mr-1" aria-hidden="true"></i>{{ _("sessions.last_active") }}
<span x-text="timeAgo(session.last_active_at)"></span>
</span>
</div>
</div>
</div>
<button
x-show="!session.is_current"
@click="revokeSession(session.id)"
class="flex-shrink-0 text-red-600 hover:text-red-800 dark:text-red-400 dark:hover:text-red-300 text-sm font-medium px-3 py-1 rounded hover:bg-red-50 dark:hover:bg-red-900/20 transition"
style="min-height:44px; min-width:44px;"
:aria-label="'{{ _("sessions.revoke") }}'"
>
<i class="fas fa-sign-out-alt mr-1" aria-hidden="true"></i>{{ _("sessions.revoke") }}
</button>
</div>
</template>
<p
x-show="sessions.length <= 1"
class="text-sm text-gray-500 dark:text-gray-400 italic"
>{{ _("sessions.no_other_sessions") }}</p>
</div>
<!-- Log off everywhere + QR login row -->
<div class="flex flex-col sm:flex-row gap-3">
<button
@click="revokeAllSessions()"
class="inline-flex items-center justify-center px-4 py-2 border border-red-300 dark:border-red-700 rounded-lg text-sm font-medium text-red-700 dark:text-red-300 bg-white dark:bg-gray-800 hover:bg-red-50 dark:hover:bg-red-900/20 transition"
style="min-height:44px;"
:disabled="revoking"
>
<i class="fas fa-power-off mr-2" aria-hidden="true"></i>
<span x-text="revoking ? '{{ _("profile.saving") }}' : '{{ _("sessions.log_off_everywhere") }}'"></span>
</button>
<a
href="/qr-login"
class="inline-flex items-center justify-center px-4 py-2 border border-gray-300 dark:border-gray-600 rounded-lg text-sm font-medium text-gray-700 dark:text-gray-300 bg-white dark:bg-gray-800 hover:bg-gray-50 dark:hover:bg-gray-700 transition"
style="min-height:44px;"
>
<i class="fas fa-qrcode mr-2" aria-hidden="true"></i>
{{ _("sessions.qr_login_link") }}
</a>
</div>
<!-- Status banner for session actions -->
<div
x-show="sessionBanner.visible"
x-transition
class="mt-4 rounded-lg p-3 text-sm"
:class="sessionBanner.error
? 'bg-red-50 dark:bg-red-900/30 text-red-800 dark:text-red-200 border border-red-300 dark:border-red-700'
: 'bg-green-50 dark:bg-green-900/30 text-green-800 dark:text-green-200 border border-green-300 dark:border-green-700'"
role="alert"
aria-live="polite"
>
<span x-text="sessionBanner.message"></span>
</div>
</section>
</div><!-- /container -->
<script>
/* ── Session Manager Alpine component ──────────────────────────────────── */
function sessionManager() {
return {
sessions: [],
lifetimeDays: 0,
revoking: false,
sessionBanner: { visible: false, error: false, message: '' },
_csrfToken() {
return document.cookie
.split('; ')
.find(row => row.startsWith('csrf_token='))
?.split('=')[1];
},
async loadSessions() {
try {
const res = await fetch('/api/sessions/');
if (res.ok) {
const data = await res.json();
this.sessions = data.sessions || [];
this.lifetimeDays = data.session_lifetime_days || 30;
}
} catch (_e) { /* silently ignore */ }
},
async revokeSession(sessionId) {
if (!confirm({{ _("sessions.confirm_revoke_one") | tojson }})) return;
const csrf = this._csrfToken();
try {
const res = await fetch(`/api/sessions/${sessionId}`, {
method: 'DELETE',
headers: csrf ? { 'X-CSRF-Token': csrf } : {},
});
if (res.ok || res.status === 204) {
this.sessions = this.sessions.filter(s => s.id !== sessionId);
this._showSessionBanner({{ _("sessions.revoked_success") | tojson }}, false);
}
} catch (_e) {
this._showSessionBanner('Network error — please try again.', true);
}
},
async revokeAllSessions() {
if (!confirm({{ _("sessions.confirm_revoke_all") | tojson }})) return;
this.revoking = true;
const csrf = this._csrfToken();
try {
const res = await fetch('/api/sessions/revoke-all', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
...(csrf ? { 'X-CSRF-Token': csrf } : {}),
},
});
if (res.ok) {
await this.loadSessions();
this._showSessionBanner({{ _("sessions.revoked_all_success") | tojson }}, false);
}
} catch (_e) {
this._showSessionBanner('Network error — please try again.', true);
} finally {
this.revoking = false;
}
},
timeAgo(dateStr) {
if (!dateStr) return 'unknown';
const now = new Date();
const then = new Date(dateStr);
const diff = Math.floor((now - then) / 1000);
if (diff < 60) return 'just now';
if (diff < 3600) return Math.floor(diff / 60) + 'm ago';
if (diff < 86400) return Math.floor(diff / 3600) + 'h ago';
return Math.floor(diff / 86400) + 'd ago';
},
_showSessionBanner(msg, err) {
this.sessionBanner = { visible: true, error: err, message: msg };
if (!err) setTimeout(() => { this.sessionBanner.visible = false; }, 4000);
},
};
}
/* ── Profile Settings Alpine component ─────────────────────────────────── */
function profileSettings() {
return {
// ── State ──────────────────────────────────────────────────────────────
frontend/templates/qr_login.html
+229
View File
@@ -0,0 +1,229 @@
{% extends "base.html" %}
{% block title %}{{ _("qr_login.page_title") }}{% endblock %}
{% block content %}
<div
x-data="qrLoginPage()"
x-init="generateChallenge()"
class="container mx-auto px-4 py-8 max-w-xl"
>
<!-- ── Header ─────────────────────────────────────────────────────────── -->
<header class="mb-8 text-center">
<h1 class="text-2xl font-bold text-gray-900 dark:text-white flex items-center justify-center gap-2">
<i class="fas fa-qrcode text-blue-500" aria-hidden="true"></i>
{{ _("qr_login.heading") }}
</h1>
<p class="mt-2 text-sm text-gray-500 dark:text-gray-400">
{{ _("qr_login.subtitle") }}
</p>
</header>
<!-- ── QR Code Card ───────────────────────────────────────────────────── -->
<section
class="bg-white dark:bg-gray-800 shadow rounded-lg p-8 mb-6 text-center"
aria-labelledby="qr-heading"
>
<!-- Pending state: show QR code -->
<template x-if="status === 'pending'">
<div>
<div
class="mx-auto mb-4 bg-white p-4 inline-block rounded-lg shadow-inner"
id="qr-container"
aria-label="{{ _('qr_login.description') }}"
>
<img
:src="qrCodeSvg"
width="256"
height="256"
alt="{{ _('qr_login.description') }}"
id="qr-image"
/>
</div>
<p class="text-sm text-gray-500 dark:text-gray-400 mb-2">
{{ _("qr_login.description") }}
</p>
<div class="flex items-center justify-center gap-2 text-xs text-gray-400 dark:text-gray-500">
<i class="fas fa-hourglass-half animate-pulse" aria-hidden="true"></i>
<span x-text="'{{ _("qr_login.time_remaining") }}'.replace('{seconds}', countdown)"></span>
</div>
<p class="mt-3 text-sm text-blue-600 dark:text-blue-400">
<i class="fas fa-spinner fa-spin mr-1" aria-hidden="true"></i>
{{ _("qr_login.pending_message") }}
</p>
</div>
</template>
<!-- Claimed state: success -->
<template x-if="status === 'claimed'">
<div class="py-8">
<i class="fas fa-check-circle text-green-500 text-5xl mb-4" aria-hidden="true"></i>
<p class="text-lg font-semibold text-green-700 dark:text-green-400 mb-2">
{{ _("qr_login.claimed_message") }}
</p>
<p x-show="deviceName" class="text-sm text-gray-500 dark:text-gray-400"
x-text="'{{ _("qr_login.claimed_device") }}'.replace('{device_name}', deviceName)">
</p>
</div>
</template>
<!-- Expired state -->
<template x-if="status === 'expired'">
<div class="py-8">
<i class="fas fa-clock text-yellow-500 text-5xl mb-4" aria-hidden="true"></i>
<p class="text-base text-gray-700 dark:text-gray-300 mb-4">
{{ _("qr_login.expired_message") }}
</p>
<button
@click="generateChallenge()"
class="inline-flex items-center px-4 py-2 bg-blue-600 hover:bg-blue-700 text-white font-medium rounded-lg transition"
style="min-height:44px;"
>
<i class="fas fa-redo mr-2" aria-hidden="true"></i>
{{ _("qr_login.generate_new") }}
</button>
</div>
</template>
<!-- Error state -->
<template x-if="status === 'error'">
<div class="py-8">
<i class="fas fa-exclamation-triangle text-red-500 text-5xl mb-4" aria-hidden="true"></i>
<p class="text-base text-gray-700 dark:text-gray-300 mb-4" x-text="errorMsg"></p>
<button
@click="generateChallenge()"
class="inline-flex items-center px-4 py-2 bg-blue-600 hover:bg-blue-700 text-white font-medium rounded-lg transition"
style="min-height:44px;"
>
<i class="fas fa-redo mr-2" aria-hidden="true"></i>
{{ _("qr_login.generate_new") }}
</button>
</div>
</template>
</section>
<!-- ── How it works ───────────────────────────────────────────────────── -->
<section class="bg-white dark:bg-gray-800 shadow rounded-lg p-6">
<h2 class="text-base font-semibold text-gray-900 dark:text-white mb-3">
<i class="fas fa-info-circle text-gray-400 mr-2" aria-hidden="true"></i>
{{ _("qr_login.how_it_works") }}
</h2>
<ol class="list-decimal list-inside space-y-2 text-sm text-gray-600 dark:text-gray-400">
<li>{{ _("qr_login.step_1") }}</li>
<li>{{ _("qr_login.step_2") }}</li>
<li>{{ _("qr_login.step_3") }}</li>
</ol>
</section>
</div>
<!-- QR code is rendered server-side; no external QR library needed -->
<script>
function qrLoginPage() {
return {
status: 'loading', // loading | pending | claimed | expired | error
challengeId: null,
challengeToken: '',
qrPayload: '',
qrCodeSvg: '',
expiresAt: null,
countdown: 0,
deviceName: '',
errorMsg: '',
_pollTimer: null,
_countdownTimer: null,
_ttlSeconds: 0,
_receivedAt: null,
_csrfToken() {
return document.cookie
.split('; ')
.find(row => row.startsWith('csrf_token='))
?.split('=')[1];
},
async generateChallenge() {
this.status = 'loading';
this._stopTimers();
const csrf = this._csrfToken();
try {
const res = await fetch('/api/qr-auth/challenge', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
...(csrf ? { 'X-CSRF-Token': csrf } : {}),
},
});
if (!res.ok) {
this.status = 'error';
this.errorMsg = 'Failed to generate QR code. Please try again.';
return;
}
const data = await res.json();
this.challengeId = data.challenge_id;
this.challengeToken = data.challenge_token;
this.qrPayload = data.qr_payload;
this.qrCodeSvg = data.qr_code_svg;
this.expiresAt = new Date(data.expires_at);
this._ttlSeconds = data.ttl_seconds || 120;
this._receivedAt = Date.now();
this.status = 'pending';
this.deviceName = '';
// Start polling and countdown
this._startPolling();
this._startCountdown();
} catch (_e) {
this.status = 'error';
this.errorMsg = 'Network error — please check your connection and try again.';
}
},
_startPolling() {
this._pollTimer = setInterval(async () => {
if (this.status !== 'pending') { this._stopTimers(); return; }
try {
const res = await fetch(`/api/qr-auth/challenge/${this.challengeId}/status`);
if (!res.ok) return;
const data = await res.json();
if (data.status === 'claimed') {
this.status = 'claimed';
this.deviceName = data.device_name || '';
this._stopTimers();
} else if (data.status === 'expired') {
this.status = 'expired';
this._stopTimers();
} else if (data.status === 'cancelled') {
this.status = 'expired';
this._stopTimers();
}
} catch (_e) { /* ignore transient errors */ }
}, 2000);
},
_startCountdown() {
this._updateCountdown();
this._countdownTimer = setInterval(() => {
this._updateCountdown();
if (this.countdown <= 0 && this.status === 'pending') {
this.status = 'expired';
this._stopTimers();
}
}, 1000);
},
_updateCountdown() {
if (!this._receivedAt) { this.countdown = 0; return; }
const elapsed = (Date.now() - this._receivedAt) / 1000;
const remaining = Math.max(0, Math.floor(this._ttlSeconds - elapsed));
this.countdown = remaining;
},
_stopTimers() {
if (this._pollTimer) { clearInterval(this._pollTimer); this._pollTimer = null; }
if (this._countdownTimer) { clearInterval(this._countdownTimer); this._countdownTimer = null; }
},
};
}
</script>
{% endblock %}
frontend/templates/signup.html
+14 -1
View File
@@ -32,6 +32,14 @@
error: '',
async submit() {
this.error = '';
if (this.username.length < 3 || this.username.length > 64) {
this.error = 'Username must be between 3 and 64 characters.';
return;
}
if (!/^[a-zA-Z0-9_-]+$/.test(this.username)) {
this.error = 'Username may only contain letters, numbers, hyphens, and underscores. Dots and other special characters are not allowed.';
return;
}
if (this.password !== this.password_confirm) {
this.error = 'Passwords do not match.';
return;
@@ -58,7 +66,12 @@
}
} else {
const data = await resp.json();
this.error = data.detail || 'Registration failed. Please try again.';
const detail = data.detail;
if (Array.isArray(detail)) {
this.error = detail.map(e => e.msg || String(e)).join(' ') || 'Registration failed. Please try again.';
} else {
this.error = detail || 'Registration failed. Please try again.';
}
}
} catch(e) {
this.error = 'Network error. Please try again.';
frontend/templates/system_reset.html
+261
View File
@@ -0,0 +1,261 @@
{% extends "base.html" %}
{% block title %}{{ _("system_reset.page_title") }} - DocuElevate{% endblock %}
{% block head_extra %}
<style>
.reset-card { transition: box-shadow 0.2s ease; }
.reset-card:hover { box-shadow: 0 4px 20px rgba(0,0,0,.08); }
.confirmation-input { font-family: 'Courier New', monospace; letter-spacing: 0.1em; text-transform: uppercase; }
</style>
{% endblock %}
{% block content %}
<div class="container mx-auto px-4 py-8" x-data="systemResetApp()">
<!-- Header -->
<div class="mb-6">
<div class="flex items-center gap-3 mb-2">
<h1 class="text-3xl font-bold text-gray-900 dark:text-white">
<i class="fas fa-skull-crossbones mr-2 text-red-600" aria-hidden="true"></i>
{{ _("system_reset.heading") }}
</h1>
</div>
<p class="text-sm text-gray-500 dark:text-gray-400">
{{ _("system_reset.subtitle") }}
</p>
</div>
<!-- Factory Reset on Startup banner -->
{% if factory_reset_on_startup %}
<div class="mb-6 rounded-lg border border-yellow-300 bg-yellow-50 dark:bg-yellow-900/20 dark:border-yellow-700 p-4" role="alert">
<div class="flex items-start gap-3">
<i class="fas fa-exclamation-triangle text-yellow-600 mt-0.5" aria-hidden="true"></i>
<div>
<p class="font-semibold text-yellow-800 dark:text-yellow-200">{{ _("system_reset.startup_reset_active") }}</p>
<p class="text-sm text-yellow-700 dark:text-yellow-300 mt-1">{{ _("system_reset.startup_reset_desc") }}</p>
</div>
</div>
</div>
{% endif %}
<!-- Warning banner -->
<div class="mb-8 rounded-lg border-2 border-red-300 bg-red-50 dark:bg-red-900/20 dark:border-red-700 p-6" role="alert">
<div class="flex items-start gap-3">
<i class="fas fa-radiation text-red-600 text-2xl mt-0.5" aria-hidden="true"></i>
<div>
<p class="font-bold text-red-800 dark:text-red-200 text-lg">{{ _("system_reset.danger_zone") }}</p>
<p class="text-sm text-red-700 dark:text-red-300 mt-1">{{ _("system_reset.danger_desc") }}</p>
</div>
</div>
</div>
<div class="grid grid-cols-1 lg:grid-cols-2 gap-8">
<!-- Full Reset Card -->
<div class="reset-card rounded-xl border-2 border-red-200 dark:border-red-800 bg-white dark:bg-gray-800 p-6">
<div class="flex items-center gap-3 mb-4">
<div class="h-10 w-10 rounded-full bg-red-100 dark:bg-red-900 flex items-center justify-center">
<i class="fas fa-trash-alt text-red-600" aria-hidden="true"></i>
</div>
<div>
<h2 class="text-xl font-bold text-gray-900 dark:text-white">{{ _("system_reset.full_reset_title") }}</h2>
<p class="text-sm text-gray-500 dark:text-gray-400">{{ _("system_reset.full_reset_subtitle") }}</p>
</div>
</div>
<div class="space-y-3 mb-6">
<p class="text-sm text-gray-700 dark:text-gray-300">{{ _("system_reset.full_reset_desc") }}</p>
<ul class="text-sm text-gray-600 dark:text-gray-400 space-y-1 ml-4 list-disc">
<li>{{ _("system_reset.full_reset_item_db") }}</li>
<li>{{ _("system_reset.full_reset_item_files") }}</li>
<li>{{ _("system_reset.full_reset_item_cache") }}</li>
<li>{{ _("system_reset.full_reset_item_settings_kept") }}</li>
</ul>
</div>
<div class="border-t border-gray-200 dark:border-gray-700 pt-4">
<label for="fullResetConfirm" class="block text-sm font-semibold text-red-700 dark:text-red-400 mb-2">
{{ _("system_reset.type_delete") }}
</label>
<input id="fullResetConfirm"
type="text"
x-model="fullResetInput"
class="confirmation-input w-full px-3 py-2 border-2 border-red-300 dark:border-red-700 rounded-lg
text-center text-lg bg-white dark:bg-gray-900 text-gray-900 dark:text-white
focus:outline-none focus:ring-2 focus:ring-red-500 focus:border-red-500"
placeholder="DELETE"
autocomplete="off"
aria-describedby="fullResetHelp" />
<p id="fullResetHelp" class="text-xs text-gray-500 dark:text-gray-400 mt-1">{{ _("system_reset.type_delete_help") }}</p>
<button @click="executeFullReset()"
:disabled="fullResetInput !== 'DELETE' || loading"
type="button"
class="mt-4 w-full inline-flex items-center justify-center px-4 py-3 border border-transparent
text-base font-bold rounded-lg text-white
bg-red-600 hover:bg-red-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-red-500
disabled:opacity-40 disabled:cursor-not-allowed transition min-h-[44px]"
aria-label="{{ _('system_reset.full_reset_button') }}">
<template x-if="loading && activeAction === 'full'">
<i class="fas fa-spinner fa-spin mr-2" aria-hidden="true"></i>
</template>
<i x-show="!(loading && activeAction === 'full')" class="fas fa-trash-alt mr-2" aria-hidden="true"></i>
{{ _("system_reset.full_reset_button") }}
</button>
</div>
</div>
<!-- Reset & Re-import Card -->
<div class="reset-card rounded-xl border-2 border-orange-200 dark:border-orange-800 bg-white dark:bg-gray-800 p-6">
<div class="flex items-center gap-3 mb-4">
<div class="h-10 w-10 rounded-full bg-orange-100 dark:bg-orange-900 flex items-center justify-center">
<i class="fas fa-recycle text-orange-600" aria-hidden="true"></i>
</div>
<div>
<h2 class="text-xl font-bold text-gray-900 dark:text-white">{{ _("system_reset.reimport_title") }}</h2>
<p class="text-sm text-gray-500 dark:text-gray-400">{{ _("system_reset.reimport_subtitle") }}</p>
</div>
</div>
<div class="space-y-3 mb-6">
<p class="text-sm text-gray-700 dark:text-gray-300">{{ _("system_reset.reimport_desc") }}</p>
<ol class="text-sm text-gray-600 dark:text-gray-400 space-y-1 ml-4 list-decimal">
<li>{{ _("system_reset.reimport_step_1") }}</li>
<li>{{ _("system_reset.reimport_step_2") }}</li>
<li>{{ _("system_reset.reimport_step_3") }}</li>
</ol>
<p class="text-xs text-gray-500 dark:text-gray-400 italic">{{ _("system_reset.reimport_note") }}</p>
</div>
<div class="border-t border-gray-200 dark:border-gray-700 pt-4">
<label for="reimportConfirm" class="block text-sm font-semibold text-orange-700 dark:text-orange-400 mb-2">
{{ _("system_reset.type_reimport") }}
</label>
<input id="reimportConfirm"
type="text"
x-model="reimportInput"
class="confirmation-input w-full px-3 py-2 border-2 border-orange-300 dark:border-orange-700 rounded-lg
text-center text-lg bg-white dark:bg-gray-900 text-gray-900 dark:text-white
focus:outline-none focus:ring-2 focus:ring-orange-500 focus:border-orange-500"
placeholder="REIMPORT"
autocomplete="off"
aria-describedby="reimportHelp" />
<p id="reimportHelp" class="text-xs text-gray-500 dark:text-gray-400 mt-1">{{ _("system_reset.type_reimport_help") }}</p>
<button @click="executeReimport()"
:disabled="reimportInput !== 'REIMPORT' || loading"
type="button"
class="mt-4 w-full inline-flex items-center justify-center px-4 py-3 border border-transparent
text-base font-bold rounded-lg text-white
bg-orange-600 hover:bg-orange-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-orange-500
disabled:opacity-40 disabled:cursor-not-allowed transition min-h-[44px]"
aria-label="{{ _('system_reset.reimport_button') }}">
<template x-if="loading && activeAction === 'reimport'">
<i class="fas fa-spinner fa-spin mr-2" aria-hidden="true"></i>
</template>
<i x-show="!(loading && activeAction === 'reimport')" class="fas fa-recycle mr-2" aria-hidden="true"></i>
{{ _("system_reset.reimport_button") }}
</button>
</div>
</div>
</div>
<!-- Result banner (shown after a reset completes) -->
<div x-show="resultMessage" x-cloak
class="mt-8 rounded-lg p-4"
:class="resultSuccess ? 'bg-green-50 dark:bg-green-900/20 border border-green-300 dark:border-green-700' :
'bg-red-50 dark:bg-red-900/20 border border-red-300 dark:border-red-700'"
:role="resultSuccess ? 'status' : 'alert'" aria-live="polite">
<div class="flex items-start gap-3">
<i :class="resultSuccess ? 'fas fa-check-circle text-green-600' : 'fas fa-times-circle text-red-600'" aria-hidden="true"></i>
<div>
<p class="font-semibold" :class="resultSuccess ? 'text-green-800 dark:text-green-200' : 'text-red-800 dark:text-red-200'"
x-text="resultMessage"></p>
<pre x-show="resultDetail" x-text="resultDetail"
class="mt-2 text-xs overflow-x-auto whitespace-pre-wrap"
:class="resultSuccess ? 'text-green-700 dark:text-green-300' : 'text-red-700 dark:text-red-300'"></pre>
</div>
</div>
</div>
</div>
<script>
function systemResetApp() {
const __i18n = {
successFull: {{ _("system_reset.js_success_full") | tojson }},
successReimport: {{ _("system_reset.js_success_reimport") | tojson }},
errorGeneric: {{ _("system_reset.js_error_generic") | tojson }},
};
return {
fullResetInput: '',
reimportInput: '',
loading: false,
activeAction: null,
resultMessage: null,
resultDetail: null,
resultSuccess: false,
async executeFullReset() {
if (this.fullResetInput !== 'DELETE') return;
this.loading = true;
this.activeAction = 'full';
this.resultMessage = null;
try {
const csrfToken = document.querySelector('meta[name="csrf-token"]')?.content || '';
const resp = await fetch('/api/admin/system-reset/full', {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'X-CSRF-Token': csrfToken },
body: JSON.stringify({ confirmation: 'DELETE' }),
});
const data = await resp.json();
if (resp.ok) {
this.resultSuccess = true;
this.resultMessage = __i18n.successFull;
this.resultDetail = JSON.stringify(data.result, null, 2);
} else {
this.resultSuccess = false;
this.resultMessage = data.detail || __i18n.errorGeneric;
}
} catch (err) {
this.resultSuccess = false;
this.resultMessage = __i18n.errorGeneric;
} finally {
this.loading = false;
this.fullResetInput = '';
}
},
async executeReimport() {
if (this.reimportInput !== 'REIMPORT') return;
this.loading = true;
this.activeAction = 'reimport';
this.resultMessage = null;
try {
const csrfToken = document.querySelector('meta[name="csrf-token"]')?.content || '';
const resp = await fetch('/api/admin/system-reset/reimport', {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'X-CSRF-Token': csrfToken },
body: JSON.stringify({ confirmation: 'REIMPORT' }),
});
const data = await resp.json();
if (resp.ok) {
this.resultSuccess = true;
this.resultMessage = __i18n.successReimport;
this.resultDetail = JSON.stringify(data.result, null, 2);
} else {
this.resultSuccess = false;
this.resultMessage = data.detail || __i18n.errorGeneric;
}
} catch (err) {
this.resultSuccess = false;
this.resultMessage = __i18n.errorGeneric;
} finally {
this.loading = false;
this.reimportInput = '';
}
},
};
}
</script>
{% endblock %}
frontend/translations/en.json
+118
View File
@@ -316,6 +316,7 @@
"admin_users.total_no_users": "No users",
"admin_users.total_one_user": "1 user",
"api_tokens.col_created": "Created",
"api_tokens.col_expires": "Expires",
"api_tokens.col_last_ip": "Last IP",
"api_tokens.col_last_used": "Last Used",
"api_tokens.col_name": "Name",
@@ -327,6 +328,14 @@
"api_tokens.create_heading": "Create New Token",
"api_tokens.create_token": "Create Token",
"api_tokens.creating": "Creating…",
"api_tokens.delete": "Delete",
"api_tokens.delete_confirm": "Permanently delete this revoked token? This cannot be undone.",
"api_tokens.delete_prefix": "Permanently delete token",
"api_tokens.expires_at_label": "Expires (optional)",
"api_tokens.expires_at_placeholder": "e.g. 30, 90, 365 days",
"api_tokens.expires_in_days_label": "Token lifetime (days)",
"api_tokens.expires_never": "Never",
"api_tokens.expires_on": "Expires",
"api_tokens.heading": "API Tokens",
"api_tokens.intro": "Create personal API tokens to interact with the DocuElevate API programmatically. Use tokens for webhook uploads, CI/CD pipelines, or any script that needs to upload or retrieve documents.",
"api_tokens.loading_tokens": "Loading tokens…",
@@ -334,9 +343,13 @@
"api_tokens.no_tokens_heading": "No API tokens yet",
"api_tokens.no_tokens_help": "Create your first token above to get started.",
"api_tokens.page_title": "API Tokens – DocuElevate",
"api_tokens.reactivate": "Reactivate",
"api_tokens.reactivate_confirm": "Reactivate this token? It will be usable again immediately.",
"api_tokens.reactivate_prefix": "Reactivate token",
"api_tokens.revoke": "Revoke",
"api_tokens.revoke_prefix": "Revoke token",
"api_tokens.status_active": "Active",
"api_tokens.status_expired": "Expired",
"api_tokens.status_revoked": "Revoked",
"api_tokens.table_aria": "API Tokens",
"api_tokens.token_created": "Token created successfully!",
@@ -608,6 +621,46 @@
"dashboard.title": "Dashboard",
"dashboard.total_files": "Total Files",
"dashboard.welcome": "Welcome to DocuElevate",
"devices.col_created": "Connected",
"devices.col_device": "Device",
"devices.col_last_ip": "Last IP",
"devices.col_last_seen": "Last Seen",
"devices.col_last_used": "Last Used",
"devices.col_platform": "Platform",
"devices.col_push_token": "Push Token",
"devices.col_status": "Status",
"devices.col_token_prefix": "Token Prefix",
"devices.confirm_deactivate_device": "Remove this device? It will stop receiving push notifications.",
"devices.confirm_revoke_token": "Revoke access for this device? It will need to log in again.",
"devices.deactivate_device": "Remove",
"devices.delete_device": "Delete",
"devices.delete_device_confirm": "Permanently delete this inactive device? This cannot be undone.",
"devices.delete_token": "Delete",
"devices.delete_token_confirm": "Permanently delete this revoked token? This cannot be undone.",
"devices.device_deleted_success": "Device permanently deleted.",
"devices.device_removed_success": "Device removed successfully.",
"devices.heading": "Mobile Devices",
"devices.intro": "Manage your mobile app connections and registered devices. You can revoke access for individual devices here.",
"devices.loading": "Loading devices…",
"devices.mobile_tokens_description": "These tokens were created when you logged in via the mobile app or scanned a QR code. Revoking a token will sign the device out.",
"devices.mobile_tokens_heading": "Mobile App Tokens",
"devices.no_devices": "No registered devices",
"devices.no_devices_help": "Install the DocuElevate mobile app and log in to register a device for push notifications.",
"devices.no_mobile_tokens": "No mobile app tokens",
"devices.no_mobile_tokens_help": "Log in via the mobile app or scan a QR code to create a mobile token.",
"devices.page_title": "Devices – DocuElevate",
"devices.qr_login_cta": "Connect a new device via QR code",
"devices.reactivate_token": "Reactivate",
"devices.reactivate_token_confirm": "Reactivate this token? The device will be able to use it again immediately.",
"devices.registered_devices_description": "Devices registered for push notifications from the DocuElevate mobile app.",
"devices.registered_devices_heading": "Registered Devices",
"devices.revoke_token": "Revoke",
"devices.status_active": "Active",
"devices.status_inactive": "Inactive",
"devices.status_revoked": "Revoked",
"devices.token_deleted_success": "Token permanently deleted.",
"devices.token_reactivated_success": "Token reactivated successfully.",
"devices.token_revoked_success": "Device token revoked successfully.",
"duplicates.file_id_label": "File ID",
"duplicates.file_id_placeholder": "e.g. 42",
"duplicates.find_btn": "Find",
@@ -1152,6 +1205,7 @@
"nav.dark_mode": "Dark Mode",
"nav.dashboard": "Dashboard",
"nav.developer_docs": "Developer Docs",
"nav.devices": "Devices",
"nav.duplicates": "Duplicates",
"nav.file_manager": "File Manager",
"nav.files": "Files",
@@ -1184,6 +1238,7 @@
"nav.skip_to_content": "Skip to main content",
"nav.status": "Status",
"nav.subscription": "Subscription",
"nav.system_reset": "System Reset",
"nav.toggle_dark_mode": "Toggle dark mode",
"nav.toggle_nav": "Toggle navigation menu",
"nav.upload": "Upload",
@@ -1451,6 +1506,20 @@
"profile.theme_system": "System Default",
"profile.update_password": "Update Password",
"profile.updating": "Updating…",
"qr_login.claimed_device": "Device: {device_name}",
"qr_login.claimed_message": "QR code login successful! Your mobile device is now connected.",
"qr_login.description": "Scan this QR code with the DocuElevate mobile app to log in instantly.",
"qr_login.expired_message": "This QR code has expired. Please generate a new one.",
"qr_login.generate_new": "Generate New QR Code",
"qr_login.heading": "Mobile App QR Login",
"qr_login.how_it_works": "How it works",
"qr_login.page_title": "QR Code Login – DocuElevate",
"qr_login.pending_message": "Waiting for mobile app to scan…",
"qr_login.step_1": "Open the DocuElevate app on your phone",
"qr_login.step_2": "Tap \"Scan QR Code\" on the login screen",
"qr_login.step_3": "Point your camera at this QR code",
"qr_login.subtitle": "Log in to the mobile app by scanning a QR code from this page.",
"qr_login.time_remaining": "Expires in {seconds} seconds",
"queue.active_tasks": "Active Tasks",
"queue.auto_refresh_1": "Auto-refreshes every",
"queue.auto_refresh_2": "seconds",
@@ -1512,6 +1581,25 @@
"search.saved_label": "Saved Searches",
"search.saved_loading": "Loading...",
"search.title": "Search Documents",
"sessions.active_sessions": "Active Sessions",
"sessions.confirm_revoke_all": "This will log you out of all other devices and browsers, and revoke all API tokens. Continue?",
"sessions.confirm_revoke_one": "Are you sure you want to end this session?",
"sessions.current_session": "This device",
"sessions.device_info": "Device",
"sessions.expires": "Expires",
"sessions.ip_address": "IP Address",
"sessions.last_active": "Last active",
"sessions.log_off_everywhere": "Log Off All Other Sessions",
"sessions.log_off_everywhere_desc": "End all other browser sessions and revoke all API tokens. Your current session will remain active.",
"sessions.no_other_sessions": "No other active sessions found.",
"sessions.qr_login_link": "Log in on mobile via QR code",
"sessions.revoke": "End Session",
"sessions.revoked_all_success": "All other sessions have been ended.",
"sessions.revoked_success": "Session ended successfully.",
"sessions.security_heading": "Security & Sessions",
"sessions.security_subtitle": "Manage your active sessions across devices and browsers.",
"sessions.session_lifetime": "Session lifetime: {days} days",
"sessions.started": "Started",
"settings.audit_log_btn": "Audit Log",
"settings.autocomplete_hint": "Type to search known values, or enter any custom value.",
"settings.autocomplete_no_matches": "No matches — you can still type a custom value",
@@ -1723,6 +1811,36 @@
"subscription.upgrade_info": "Upgrades take effect immediately. Downgrades are scheduled for the end of your current billing period.",
"subscription.upgrade_to_prefix": "Upgrade to",
"subscription.usage_heading": "Usage",
"system_reset.danger_desc": "The actions below will permanently destroy data. They cannot be undone. Application settings and configuration are preserved, but all documents, files, processing history, and audit logs will be deleted.",
"system_reset.danger_zone": "Danger Zone — Irreversible Actions",
"system_reset.full_reset_button": "Wipe All Data",
"system_reset.full_reset_desc": "Permanently deletes all user data from the database and removes all work files from disk. The application will be in its initial state after this operation.",
"system_reset.full_reset_item_cache": "All watch-folder caches and ingestion state",
"system_reset.full_reset_item_db": "All document records, processing logs, and audit history",
"system_reset.full_reset_item_files": "All original, processed, and temporary files on disk",
"system_reset.full_reset_item_settings_kept": "Application settings and configuration are preserved",
"system_reset.full_reset_subtitle": "Wipe everything and start fresh",
"system_reset.full_reset_title": "Full System Reset",
"system_reset.heading": "System Reset",
"system_reset.js_error_generic": "An error occurred. Please check the server logs for details.",
"system_reset.js_success_full": "System reset complete. All user data has been wiped.",
"system_reset.js_success_reimport": "Reset complete. Original files have been staged for re-import via the watch folder.",
"system_reset.page_title": "System Reset",
"system_reset.reimport_button": "Reset & Re-import",
"system_reset.reimport_desc": "Copies your original files to a special reimport folder, wipes everything, then lets the watch-folder mechanism re-process them as if they were freshly uploaded.",
"system_reset.reimport_note": "Re-imported files will go through the full processing pipeline with the same rate limits and backoff strategy as regular uploads.",
"system_reset.reimport_step_1": "Original files are copied to a dedicated reimport folder",
"system_reset.reimport_step_2": "All data (database + work files) is wiped clean",
"system_reset.reimport_step_3": "The reimport folder is configured as a watch folder for automatic re-ingestion",
"system_reset.reimport_subtitle": "Wipe and re-process all original files",
"system_reset.reimport_title": "Reset & Re-import",
"system_reset.startup_reset_active": "Factory Reset on Startup is ACTIVE",
"system_reset.startup_reset_desc": "FACTORY_RESET_ON_STARTUP is enabled. All user data is wiped every time the application starts.",
"system_reset.subtitle": "Reset DocuElevate to a clean, fresh state. All user data will be permanently deleted.",
"system_reset.type_delete": "Type DELETE to confirm",
"system_reset.type_delete_help": "You must type the word DELETE in capital letters to enable the reset button.",
"system_reset.type_reimport": "Type REIMPORT to confirm",
"system_reset.type_reimport_help": "You must type the word REIMPORT in capital letters to enable the button.",
"terms.cookie_link": "Cookie Policy",
"terms.heading": "Terms of Service",
"terms.last_updated": "Last Updated:",
helm/docuelevate/templates/beat-deployment.yaml
+83
View File
@@ -0,0 +1,83 @@
{{- /*
Celery Beat scheduler — publishes periodic tasks to the broker.
Exactly ONE replica must run; never scale this deployment.
*/ -}}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "docuelevate.fullname" . }}-beat
namespace: {{ .Release.Namespace }}
labels:
{{- include "docuelevate.labels" . | nindent 4 }}
app.kubernetes.io/component: beat
spec:
replicas: 1
strategy:
type: Recreate # Prevent two Beat instances from running simultaneously
selector:
matchLabels:
{{- include "docuelevate.selectorLabels" . | nindent 6 }}
app.kubernetes.io/component: beat
template:
metadata:
labels:
{{- include "docuelevate.selectorLabels" . | nindent 8 }}
app.kubernetes.io/component: beat
{{- with .Values.beat.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
serviceAccountName: {{ include "docuelevate.serviceAccountName" . }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.beat.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: beat
image: {{ include "docuelevate.image" . }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- celery
- -A
- app.celery_worker
- beat
- --loglevel=info
envFrom:
- configMapRef:
name: {{ include "docuelevate.fullname" . }}-config
- secretRef:
name: {{ include "docuelevate.fullname" . }}-secret
{{- with .Values.beat.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
resources:
{{- toYaml .Values.beat.resources | nindent 12 }}
volumeMounts:
- name: workdir
mountPath: /workdir
volumes:
- name: workdir
{{- if .Values.workdir.persistence.enabled }}
persistentVolumeClaim:
claimName: {{ .Values.workdir.persistence.existingClaim | default (printf "%s-workdir" (include "docuelevate.fullname" .)) }}
{{- else }}
emptyDir: {}
{{- end }}
{{- with .Values.beat.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.beat.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.beat.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
helm/docuelevate/templates/worker-deployment.yaml
-1
View File
@@ -42,7 +42,6 @@ spec:
- -A
- app.celery_worker
- worker
- -B
- --loglevel=info
- -Q
- document_processor,default,celery
helm/docuelevate/values.yaml
+33 -4
View File
@@ -121,10 +121,10 @@ api:
type: ClusterIP
port: 8000
# Liveness / readiness probes
# Liveness / readiness probes (unauthenticated endpoints for kubelet)
livenessProbe:
httpGet:
path: /api/health
path: /api/diagnostic/healthz/live
port: 8000
initialDelaySeconds: 30
periodSeconds: 20
@@ -132,7 +132,7 @@ api:
readinessProbe:
httpGet:
path: /api/health
path: /api/diagnostic/healthz/ready
port: 8000
initialDelaySeconds: 15
periodSeconds: 10
@@ -191,7 +191,36 @@ worker:
drop: ["ALL"]
# ---------------------------------------------------------------------------
# Shared workdir volume (api + worker mount the same PVC)
# Celery Beat scheduler (singleton — always exactly 1 replica)
# Beat publishes periodic tasks; workers consume them from the broker.
# ---------------------------------------------------------------------------
beat:
resources:
requests:
cpu: 100m
memory: 256Mi
limits:
cpu: 500m
memory: 512Mi
podAnnotations: {}
nodeSelector: {}
tolerations: []
affinity: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: false
capabilities:
drop: ["ALL"]
# ---------------------------------------------------------------------------
# Shared workdir volume (api + worker + beat mount the same PVC)
# ---------------------------------------------------------------------------
workdir:
persistence:
migrations/env.py
+1
View File
@@ -24,6 +24,7 @@ from app.models import ( # noqa: F401
ApplicationSettings,
AuditLog,
BackupRecord,
ClassificationRuleModel,
ComplianceTemplate,
DocumentMetadata,
FileProcessingStep,
migrations/script.py.mako
+40
View File
@@ -0,0 +1,40 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
depends_on: Union[str, None] = None
def upgrade() -> None:
"""${message}."""
# Use ``op.batch_alter_table()`` for SQLite compatibility.
# Always check whether the table/column already exists before altering
# to keep migrations idempotent (safe to re-run).
#
# Example – add a column only if it is missing:
#
# conn = op.get_bind()
# inspector = sa.inspect(conn)
# if "my_table" in inspector.get_table_names():
# existing = {c["name"] for c in inspector.get_columns("my_table")}
# if "new_col" not in existing:
# with op.batch_alter_table("my_table") as batch_op:
# batch_op.add_column(sa.Column("new_col", sa.String(128), nullable=True))
${upgrades if upgrades else "pass"}
def downgrade() -> None:
"""Reverse ${message}."""
${downgrades if downgrades else "pass"}
migrations/versions/037_add_user_sessions_and_qr_challenges.py
+72
View File
@@ -0,0 +1,72 @@
"""Add user_sessions and qr_login_challenges tables.
Adds server-side session tracking (user_sessions) for the "log off
everywhere" feature and per-session revocation, and QR login challenges
(qr_login_challenges) for secure mobile app authentication via QR code.
Revision ID: 037_add_user_sessions_and_qr_challenges
Revises: 036_add_document_translation_fields
Create Date: 2026-03-16
"""
from typing import Union
import sqlalchemy as sa
from alembic import op
revision: str = "037_add_user_sessions_and_qr_challenges"
down_revision: Union[str, None] = "036_add_document_translation_fields"
depends_on: Union[str, None] = None
def upgrade() -> None:
"""Create user_sessions and qr_login_challenges tables."""
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if "user_sessions" not in existing_tables:
op.create_table(
"user_sessions",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("session_token", sa.String(128), nullable=False, unique=True, index=True),
sa.Column("user_id", sa.String(), nullable=False, index=True),
sa.Column("ip_address", sa.String(45), nullable=True),
sa.Column("user_agent", sa.String(512), nullable=True),
sa.Column("device_info", sa.String(255), nullable=True),
sa.Column("is_revoked", sa.Boolean(), nullable=False, server_default="0"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column("last_active_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
)
if "qr_login_challenges" not in existing_tables:
op.create_table(
"qr_login_challenges",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("challenge_token", sa.String(128), nullable=False, unique=True, index=True),
sa.Column("user_id", sa.String(), nullable=False, index=True),
sa.Column("is_claimed", sa.Boolean(), nullable=False, server_default="0"),
sa.Column("is_cancelled", sa.Boolean(), nullable=False, server_default="0"),
sa.Column("created_by_ip", sa.String(45), nullable=True),
sa.Column("claimed_by_ip", sa.String(45), nullable=True),
sa.Column("device_name", sa.String(255), nullable=True),
sa.Column("issued_token_id", sa.Integer(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("claimed_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
"""Drop user_sessions and qr_login_challenges tables."""
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_tables = set(inspector.get_table_names())
if "qr_login_challenges" in existing_tables:
op.drop_table("qr_login_challenges")
if "user_sessions" in existing_tables:
op.drop_table("user_sessions")
migrations/versions/038_add_api_token_expires_at.py
+43
View File
@@ -0,0 +1,43 @@
"""Add expires_at column to api_tokens table.
Allows API tokens to be issued with an optional lifetime. If ``expires_at``
is set, the token is automatically rejected after that timestamp.
Revision ID: 038_add_api_token_expires_at
Revises: 037_add_user_sessions_and_qr_challenges
Create Date: 2026-03-18
"""
from typing import Union
import sqlalchemy as sa
from alembic import op
revision: str = "038_add_api_token_expires_at"
down_revision: Union[str, None] = "037_add_user_sessions_and_qr_challenges"
depends_on: Union[str, None] = None
def upgrade() -> None:
"""Add expires_at column to api_tokens (idempotent)."""
conn = op.get_bind()
inspector = sa.inspect(conn)
if "api_tokens" not in inspector.get_table_names():
return
existing_columns = {col["name"] for col in inspector.get_columns("api_tokens")}
if "expires_at" not in existing_columns:
op.add_column(
"api_tokens",
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
"""Remove expires_at column from api_tokens."""
conn = op.get_bind()
inspector = sa.inspect(conn)
if "api_tokens" not in inspector.get_table_names():
return
existing_columns = {col["name"] for col in inspector.get_columns("api_tokens")}
if "expires_at" in existing_columns:
op.drop_column("api_tokens", "expires_at")
migrations/versions/039_add_classification_rules.py
+46
View File
@@ -0,0 +1,46 @@
"""Add classification_rules table for custom document classification rules.
Revision ID: 039_add_classification_rules
Revises: 038_add_api_token_expires_at
Create Date: 2026-03-17
"""
from typing import Union
import sqlalchemy as sa
from alembic import op
revision: str = "039_add_classification_rules"
down_revision: Union[str, None] = "038_add_api_token_expires_at"
depends_on: Union[str, None] = None
def upgrade() -> None:
"""Create classification_rules table."""
op.create_table(
"classification_rules",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("owner_id", sa.String(), nullable=True),
sa.Column("name", sa.String(255), nullable=False),
sa.Column("category", sa.String(100), nullable=False),
sa.Column("rule_type", sa.String(50), nullable=False),
sa.Column("pattern", sa.String(1000), nullable=False),
sa.Column("priority", sa.Integer(), nullable=False, server_default="0"),
sa.Column("case_sensitive", sa.Boolean(), nullable=False, server_default="0"),
sa.Column("enabled", sa.Boolean(), nullable=False, server_default="1"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("owner_id", "name", name="uq_classification_rules_owner_name"),
)
op.create_index("ix_classification_rules_id", "classification_rules", ["id"])
op.create_index("ix_classification_rules_owner_id", "classification_rules", ["owner_id"])
op.create_index("ix_classification_rules_category", "classification_rules", ["category"])
def downgrade() -> None:
"""Drop classification_rules table."""
op.drop_index("ix_classification_rules_category", "classification_rules")
op.drop_index("ix_classification_rules_owner_id", "classification_rules")
op.drop_index("ix_classification_rules_id", "classification_rules")
op.drop_table("classification_rules")
migrations/versions/037_add_automation_hooks.py → migrations/versions/040_add_automation_hooks.py
+4 -4
View File
@@ -1,7 +1,7 @@
"""Add automation_hooks table for Zapier / Make.com webhook subscriptions.
Revision ID: 037_add_automation_hooks
Revises: 036_add_document_translation_fields
Revision ID: 040_add_automation_hooks
Revises: 039_add_classification_rules
Create Date: 2026-03-09
"""
@@ -10,8 +10,8 @@ from typing import Union
import sqlalchemy as sa
from alembic import op
revision: str = "037_add_automation_hooks"
down_revision: Union[str, None] = "036_add_document_translation_fields"
revision: str = "040_add_automation_hooks"
down_revision: Union[str, None] = "039_add_classification_rules"
depends_on: Union[str, None] = None
mobile/README.md
+14
View File
@@ -163,6 +163,16 @@ The app registers itself as a share target so any file can be sent directly to D
The root layout (`app/_layout.tsx`) listens for incoming URLs via `Linking.addEventListener` (warm start) and `Linking.getInitialURL()` (cold start). If the URL uses the `docuelevate://` scheme it is automatically rewritten to `file://` before being forwarded. Incoming files are stored in `ShareContext` and automatically uploaded by `UploadScreen`.
#### Handling "unmatched route" errors from "Open In…"
iOS sometimes delivers the file path under the `docuelevate://` scheme:
```
docuelevate://private/var/mobile/Library/Mobile Documents/…/Invoice.pdf
```
expo-router strips the scheme and tries to match `/private/var/mobile/…` as an in-app route. The catch-all `app/+not-found.tsx` intercepts this, detects the filesystem-path pattern, adds the file directly to `ShareContext`, and redirects to the Upload tab. `UploadScreen` picks up the pending file and begins uploading automatically. The `Linking` listener in the root layout may also fire for the same URL; `ShareContext` deduplicates by URI to prevent double uploads.
**Supported iOS file types:** PDF, images (JPEG / PNG / GIF / BMP / TIFF / WebP), plain text, Word (`.docx`, `.doc`), Excel (`.xlsx`, `.xls`), PowerPoint (`.pptx`, `.ppt`), and any other file (`public.data`).
To use the share sheet:
@@ -174,6 +184,10 @@ To use the share sheet:
> **Note:** `CFBundleDocumentTypes` with `LSHandlerRank: Alternate` means DocuElevate appears in the share sheet as an option but does **not** become the default app for any file type.
#### iOS Action Extension (future enhancement)
Apps like DeepL ("Translate in DeepL") appear as **Action Extensions** in the iOS share sheet, which requires a separate Xcode target and native Swift code. This is planned as a future enhancement. The current `CFBundleDocumentTypes` approach places DocuElevate in the "Open With" row of the share sheet.
### Android – how it works
`app.json` declares `intentFilters` for `ACTION_SEND` and `ACTION_SEND_MULTIPLE` with `mimeType: "*/*"`. When a user shares a file from another app and selects DocuElevate, Android delivers the content URI through the share intent, which is captured via `Linking.getInitialURL()` and processed the same way as on iOS.
mobile/app.json
+27 -5
View File
@@ -19,12 +19,12 @@
"bundleIdentifier": "org.docuelevate.mobile",
"appleTeamId": "975U2ZESBM",
"infoPlist": {
"NSCameraUsageDescription": "DocuElevate uses the camera to capture documents for upload.",
"NSCameraUsageDescription": "DocuElevate uses the camera to scan QR codes for login and to capture documents for upload.",
"NSPhotoLibraryUsageDescription": "DocuElevate accesses your photo library to select documents for upload.",
"NSPhotoLibraryAddUsageDescription": "DocuElevate saves scanned documents to your photo library.",
"UIBackgroundModes": ["fetch", "remote-notification"],
"UIBackgroundModes": ["remote-notification"],
"ITSAppUsesNonExemptEncryption": false,
"LSSupportsOpeningDocumentsInPlace": true,
"LSSupportsOpeningDocumentsInPlace": false,
"CFBundleDocumentTypes": [
{
"CFBundleTypeName": "All Documents",
@@ -86,7 +86,29 @@
"expo-build-properties",
{
"ios": {
"buildReactNativeFromSource": true
"buildReactNativeFromSource": true,
"privacyManifests": {
"NSPrivacyAccessedAPITypes": [
{
"NSPrivacyAccessedAPIType": "NSPrivacyAccessedAPICategoryUserDefaults",
"NSPrivacyAccessedAPITypeReasons": ["CA92.1"]
},
{
"NSPrivacyAccessedAPIType": "NSPrivacyAccessedAPICategoryFileTimestamp",
"NSPrivacyAccessedAPITypeReasons": ["C617.1"]
},
{
"NSPrivacyAccessedAPIType": "NSPrivacyAccessedAPICategoryDiskSpace",
"NSPrivacyAccessedAPITypeReasons": ["E174.1"]
},
{
"NSPrivacyAccessedAPIType": "NSPrivacyAccessedAPICategorySystemBootTime",
"NSPrivacyAccessedAPITypeReasons": ["35F9.1"]
}
],
"NSPrivacyCollectedDataTypes": [],
"NSPrivacyTracking": false
}
}
}
],
@@ -101,7 +123,7 @@
[
"expo-camera",
{
"cameraPermission": "DocuElevate uses the camera to capture documents for upload."
"cameraPermission": "DocuElevate uses the camera to scan QR codes for login and to capture documents for upload."
}
],
"expo-document-picker",
mobile/app/(auth)/_layout.tsx
+1
View File
@@ -12,6 +12,7 @@ export default function AuthLayout() {
<Stack screenOptions={{ headerShown: false }}>
<Stack.Screen name="index" />
<Stack.Screen name="login" />
<Stack.Screen name="qr-scanner" />
</Stack>
);
}

Some files were not shown because too many files have changed in this diff Show More