Merge pull request #821 from christianlouis/copilot/sub-pr-816

fix(integrations): harden saved-integration test endpoint & resolve merge conflicts

.dockerignore

@@ -0,0 +1,91 @@
+# =============================================================================
+# Docker build context exclusions
+# Reducing the build context speeds up builds and prevents unnecessary cache
+# invalidation when unrelated files change.
+# =============================================================================
+
+# ── Version control ──────────────────────────────────────────────────────────
+.git
+
+# ── GitHub / CI tooling ──────────────────────────────────────────────────────
+.github
+
+# ── IDE / local dev ──────────────────────────────────────────────────────────
+.vscode
+.jules
+
+# ── Pre-commit / linting config (not needed at runtime) ──────────────────────
+.pre-commit-config.yaml
+pyproject.toml
+codecov.yml
+crowdin.yml
+
+# ── Test suite ───────────────────────────────────────────────────────────────
+tests/
+requirements-dev.txt
+coverage.json
+COVERAGE_REPORT.md
+.coverage
+htmlcov/
+.pytest_cache/
+junit.xml
+coverage.xml
+
+# ── Mobile app / browser extension / legacy placeholder ─────────────────────
+# backend/ is an empty placeholder directory not part of the Python application
+mobile/
+browser-extension/
+backend/
+
+# ── Helm charts ──────────────────────────────────────────────────────────────
+helm/
+
+# ── Scripts (run before Docker build, output files are COPYd separately) ─────
+scripts/
+
+# ── Benchmark and one-off utility scripts ────────────────────────────────────
+benchmark_*.py
+fix_test*.py
+run_fast_tests.sh
+
+# ── Root-level Markdown files (docs/ is kept for docs-builder stage) ─────────
+# Note: *.md only matches files at the root level, not inside subdirectories
+*.md
+
+# ── Python bytecode / compiled artifacts ─────────────────────────────────────
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+*.so
+*.egg
+*.egg-info/
+
+# ── Virtual environments ──────────────────────────────────────────────────────
+.venv/
+venv/
+env/
+
+# ── Environment / secret files ───────────────────────────────────────────────
+.env
+.env.local
+.env.*.local
+
+# ── Runtime state files ───────────────────────────────────────────────────────
+*.log
+celerybeat-schedule
+celerybeat.pid
+
+# ── Build artifacts ───────────────────────────────────────────────────────────
+build/
+dist/
+.cache/
+.mypy_cache/
+.ruff_cache/
+site/
+docs_build/
+
+# ── Editor temp files ─────────────────────────────────────────────────────────
+*.swp
+*.swo
+*~

.env.demo

@@ -3,10 +3,43 @@ WORKDIR=/workdir
 DATABASE_URL=sqlite:///./app/database.db
 REDIS_URL=redis://redis:6379/0
 EXTERNAL_HOSTNAME=docuelevate.example.com
+# PUBLIC_BASE_URL=https://docuelevate.example.com  # Full URL with scheme; required when X-Forwarded-Proto is not forwarded by your proxy
 GOTENBERG_URL=http://gotenberg:3000
 ALLOW_FILE_DELETE=true  # Allow deletion of file records
 COMPLIANCE_ENABLED=true  # Enable compliance templates dashboard (GDPR, HIPAA, SOC 2)
 
+# **Database Connection Pool** (PostgreSQL / MySQL only; ignored for SQLite)
+# DB_POOL_SIZE=10         # Persistent connections per worker (default: 10)
+# DB_MAX_OVERFLOW=20      # Extra connections under burst (default: 20)
+# DB_POOL_TIMEOUT=30      # Seconds to wait for a pool connection (default: 30)
+# DB_POOL_RECYCLE=1800    # Recycle connections after N seconds (default: 1800)
+
+# **Per-User Upload Rate Limiting** (health-aware, Redis-backed)
+# UPLOAD_RATE_LIMIT_PER_USER=20   # Max uploads per user per window (default: 20)
+# UPLOAD_RATE_LIMIT_WINDOW=60     # Sliding window in seconds (default: 60)
+
+# **System Reset / Factory Reset**
+# FACTORY_RESET_ON_STARTUP=false  # Wipe all user data on every startup (demo/testing only)
+# ENABLE_FACTORY_RESET=false      # Show the System Reset page in admin UI
+
+# **Logging**
+# LOG_LEVEL controls the Python root-logger level.
+# Accepted values: DEBUG, INFO, WARNING, ERROR, CRITICAL (default: INFO).
+# When DEBUG=true and LOG_LEVEL is not set, the level is automatically lowered to DEBUG.
+# LOG_LEVEL=INFO
+# DEBUG=false
+
+# Log output format: "text" (human-readable, default) or "json" (structured JSON lines).
+# Use "json" when shipping logs to Grafana Loki, Splunk, ELK, Datadog, or any SIEM.
+# LOG_FORMAT=text
+
+# Forward application logs to a syslog receiver (in addition to stdout).
+# Useful for traditional (non-container) deployments and centralised SIEM ingestion.
+# LOG_SYSLOG_ENABLED=false
+# LOG_SYSLOG_HOST=localhost
+# LOG_SYSLOG_PORT=514
+# LOG_SYSLOG_PROTOCOL=udp          # udp | tcp
+
 # **UI / Appearance**
 # Default colour scheme: system (follow OS), light, or dark
 # Individual users can always override with the navbar dark-mode toggle.
@@ -142,6 +175,16 @@ AUTH_ENABLED=true
 # Generate a secure random string, for example:
 # python -c "import secrets; print(secrets.token_hex(32))"
 SESSION_SECRET=b39fd43f68d0491ca942f28a16e484b1e763fe9accf4445ca2669a5f3b179eb4
+
+# Session lifetime in days (default: 30). Common values: 30, 60, 90.
+# Determines how long a user stays logged in before needing to re-authenticate.
+# SESSION_LIFETIME_DAYS=30
+# Override with a custom value (takes precedence over SESSION_LIFETIME_DAYS):
+# SESSION_LIFETIME_CUSTOM_DAYS=
+
+# Time-to-live in seconds for QR code login challenges (default: 120 = 2 minutes).
+# QR_LOGIN_CHALLENGE_TTL_SECONDS=120
+
 ADMIN_USERNAME=admin
 ADMIN_PASSWORD=your_secure_password
 ADMIN_GROUP_NAME=admin
@@ -240,6 +283,13 @@ OPENAI_MODEL=gpt-4o-mini
 # AZURE_OPENAI_API_VERSION=2024-02-01
 # AI_MODEL=gpt-4o   # deployment name in Azure
 
+# **Document Translation**
+# After processing, documents whose detected language differs from the default
+# target language are automatically translated.  Only the original and this
+# default-language version are persisted; other translations are on-the-fly.
+# Users can override this in their profile settings.
+# DEFAULT_DOCUMENT_LANGUAGE=en
+
 # Azure Document Intelligence (OCR – separate from AI provider above)
 # **Email Settings (shared SMTP – password reset, verification, and system notifications)**
 EMAIL_HOST=smtp.example.com
@@ -408,6 +458,15 @@ ONEDRIVE_TENANT_ID=common
 ONEDRIVE_REFRESH_TOKEN=your-refresh-token
 ONEDRIVE_FOLDER_PATH=Documents/Uploads
 
+# SharePoint
+SHAREPOINT_CLIENT_ID=your-client-id
+SHAREPOINT_CLIENT_SECRET=your-client-secret
+SHAREPOINT_TENANT_ID=common
+SHAREPOINT_REFRESH_TOKEN=your-refresh-token
+SHAREPOINT_SITE_URL=https://tenant.sharepoint.com/sites/sitename
+SHAREPOINT_DOCUMENT_LIBRARY=Documents
+SHAREPOINT_FOLDER_PATH=Uploads
+
 # WebDAV
 # WEBDAV_ENABLED=true  # Set to false to disable WebDAV uploads without removing credentials
 WEBDAV_URL=https://webdav.example.com/path
@@ -577,9 +636,26 @@ EMBEDDING_MAX_TOKENS=8000
 # Profiling is only active when SENTRY_TRACES_SAMPLE_RATE > 0. Default: 0.0 (disabled).
 # SENTRY_PROFILES_SAMPLE_RATE=0.0
 #
-# Attach PII (IP addresses, user agents) to Sentry events.
-# Disable (default) to stay GDPR/CCPA compliant.
-# SENTRY_SEND_DEFAULT_PII=false
+# Attach PII (IP addresses, user agents) to Sentry events.
+# Disable (default) to stay GDPR/CCPA compliant.
+# SENTRY_SEND_DEFAULT_PII=false
+#
+# --- Browser (JavaScript) SDK ---
+# The same DSN is reused for the Sentry Browser SDK which is injected into
+# every rendered page.  The DSN is a *public* key and is intentionally
+# embedded in client-side code.
+#
+# Fraction of browser navigations captured for client-side performance tracing.
+# 0.0 (default) disables browser tracing; 1.0 captures every navigation.
+# SENTRY_JS_TRACES_SAMPLE_RATE=0.0
+#
+# Fraction of browser sessions recorded by Sentry Session Replay.
+# 0.0 (default) disables session recording; 1.0 records every session.
+# SENTRY_JS_REPLAY_SESSION_SAMPLE_RATE=0.0
+#
+# Fraction of error sessions recorded by Sentry Session Replay.
+# Defaults to 0.1 (10 %) so errors are captured with replay context.
+# SENTRY_JS_REPLAY_ON_ERROR_SAMPLE_RATE=0.1
 
 # **Mobile App – Push Notifications**
 # Push notifications are delivered via Expo's push notification service

.github/workflows/ci.yml

@@ -44,6 +44,18 @@ jobs:
       - run: ruff check app/ tests/
       - run: ruff format --check app/ tests/
 
+  migration-chain:
+    name: Alembic Migration Chain Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Validate migration chain
+        run: python scripts/check_alembic_migrations.py
+
   html-lint:
     name: HTML Accessibility Lint
     runs-on: ubuntu-latest
@@ -138,7 +150,7 @@ jobs:
   build:
     name: Build & Push Docker Image
     runs-on: ubuntu-latest
-    needs: [run-tests, mypy, dependency-scan, html-lint]
+    needs: [run-tests, mypy, dependency-scan, html-lint, migration-chain]
     if: github.event_name == 'push'
     steps:
       - name: Checkout Code

.gitignore

@@ -200,3 +200,6 @@ cython_debug/
 # Build metadata files - generated at build time
 GIT_SHA
 RUNTIME_INFO
+
+# Frontend build tooling
+frontend/node_modules/

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "vendor/embed-pdf-viewer"]
+	path = vendor/embed-pdf-viewer
+	url = https://github.com/embedpdf/embed-pdf-viewer.git

.jules/sentinel.md

@@ -1,4 +1,8 @@
-## 2024-05-24 - SSRF in WebDAV connection test
-**Vulnerability:** The `_test_webdav_connection` function had a custom SSRF check that failed to resolve DNS names, allowing attackers to bypass the check by providing a domain that resolves to an internal IP (e.g., `127.0.0.1`).
-**Learning:** DNS resolution is required for robust SSRF protection when validating URLs provided by users.
-**Prevention:** Use a centralized `is_private_ip` function (now in `app/utils/network.py`) that resolves the hostname to its IPs and checks if any are private.
+## 2026-03-20 - Safe Path Traversal Prevention in Low-Level Utilities
+**Vulnerability:** The generic file utility `hash_file` in `app/utils/file_operations.py` accepted any file path and was vulnerable to reading arbitrary files via path traversal (e.g., `../../../etc/passwd`) or absolute paths if an attacker could control the `filepath` argument.
+**Learning:** Naively checking for `".." in path` breaks legitimate relative paths used internally by the application. Blocking absolute paths entirely also breaks functionality. Input validation should occur at the API boundary, but for defense-in-depth, low-level utilities must enforce expected boundaries (e.g., the application's `workdir`).
+**Prevention:** Use `pathlib.Path.resolve()` on both the target path and the allowed base directory (`settings.workdir`). Ensure the resolved target path is strictly within the allowed boundary using `filepath_obj.relative_to(workdir_obj)`, catching the `ValueError` that is raised when the path is out of bounds. This safely blocks both relative traversal attacks and arbitrary absolute paths.
+## 2025-05-18 - [SSRF Bypass via DNS Resolution Failure]
+**Vulnerability:** The `is_private_ip` function in `app/utils/network.py` failed open (returned `False`) when a hostname could not be resolved (`socket.gaierror`).
+**Learning:** This fail-open pattern was originally added to allow external domains in tests, but in production, it created a severe SSRF risk. An attacker could bypass SSRF protections by providing a URL that fails to resolve during the security check but resolves later (DNS rebinding), or by exploiting internal routing behaviors via unresolvable addresses.
+**Prevention:** Always fail securely in network authorization functions. If a domain cannot be resolved to verify its safety, the request must be blocked (`return True` / default-deny). Tests should mock DNS resolution correctly instead of compromising production security logic.

.pre-commit-config.yaml

@@ -48,6 +48,16 @@ repos:
             .env.demo
           )$
 
+  # Alembic migration chain validation
+  - repo: local
+    hooks:
+      - id: check-alembic-migrations
+        name: Check Alembic migration chain
+        entry: python scripts/check_alembic_migrations.py
+        language: python
+        pass_filenames: false
+        files: ^migrations/versions/.*\.py$
+
   # Conventional commits validation
   - repo: https://github.com/compilerla/conventional-pre-commit
     rev: v3.0.0

BUILD_DATE

@@ -1 +1 @@
-2026-03-15T21:39:26Z
+2026-03-23T14:11:22Z

Dockerfile

@@ -1,14 +1,47 @@
-# Use multi-stage build for a smaller final image
-FROM python:3.14.1 AS builder
+# syntax=docker/dockerfile:1
 
-WORKDIR /app
+# ── Stage 1: Python dependency builder ──────────────────────────────────────
+# Use the same slim variant as the runtime to keep Python versions in sync.
+# build-essential + libffi-dev cover the few packages (e.g. cryptography) that
+# need a C compiler; they are discarded after this stage.
+FROM python:3.14.3-slim AS builder
 
-# Copy requirements first for better layer caching
-COPY requirements.txt /app/
-RUN pip install --no-cache-dir -r requirements.txt
+WORKDIR /build
 
-# ── Documentation build stage ───────────────────────────────────────────────
-FROM python:3.14.1-slim AS docs-builder
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        build-essential \
+        libffi-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Create an isolated virtual environment so only installed packages are copied
+# to the runtime image (no pip, setuptools, or other builder artefacts).
+RUN python -m venv /opt/venv
+
+ENV PATH="/opt/venv/bin:$PATH" \
+    PYTHONDONTWRITEBYTECODE=1 \
+    PIP_NO_CACHE_DIR=1
+
+COPY requirements.txt /build/
+RUN pip install --no-cache-dir -r requirements.txt \
+    # Remove bytecode and cache to keep the venv lean
+    && find /opt/venv -type f -name "*.pyc" -delete \
+    && find /opt/venv -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
+
+# ── Stage 2: Frontend asset builder (Tailwind CSS) ──────────────────────────
+FROM node:20-alpine AS frontend-builder
+
+WORKDIR /frontend
+
+# Install dependencies first (layer-cached unless package.json/lockfile changes)
+COPY frontend/package.json frontend/package-lock.json ./
+RUN npm ci
+
+# Copy source files and compile Tailwind CSS
+COPY frontend/ ./
+RUN npm run build
+
+# ── Stage 3: Documentation builder ──────────────────────────────────────────
+FROM python:3.14.3-slim AS docs-builder
 
 WORKDIR /docs
 
@@ -23,14 +56,13 @@ COPY mkdocs.yml /docs/mkdocs.yml
 # Build the static documentation site
 RUN mkdocs build --config-file /docs/mkdocs.yml --site-dir /docs/docs_build
 
-# Second stage for the actual runtime
+# ── Stage 4: Runtime image ───────────────────────────────────────────────────
 FROM python:3.14.3-slim
 
 WORKDIR /app
 
-# Copy installed packages from builder stage
-COPY --from=builder /usr/local/lib/python3.14/site-packages /usr/local/lib/python3.14/site-packages
-COPY --from=builder /usr/local/bin /usr/local/bin
+# Copy only the pre-built virtual environment from the builder
+COPY --from=builder /opt/venv /opt/venv
 
 # Install system-level OCR tools required for local OCR workflows:
 #   tesseract-ocr   – OCR engine used by pytesseract and ocrmypdf
@@ -49,6 +81,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 # Copy application code
 COPY ./app /app/app
 COPY ./frontend /app/frontend
+# Overlay compiled Tailwind CSS from the frontend build stage
+COPY --from=frontend-builder /frontend/static/styles.css /app/frontend/static/styles.css
 COPY ./migrations /app/migrations
 COPY ./alembic.ini /app/alembic.ini
 COPY ./LICENSE /app/LICENSE
@@ -62,15 +96,14 @@ COPY ./RUNTIME_INFO /app/RUNTIME_INFO
 # Copy the pre-built MkDocs documentation site (served at /help)
 COPY --from=docs-builder /docs/docs_build /app/docs_build
 
-# Create runtime_info directory
-RUN mkdir -p /app/runtime_info
-
-# Create necessary directories
-RUN mkdir -p /workdir
+# Create necessary runtime directories in a single layer
+RUN mkdir -p /app/runtime_info /workdir
 
 # Set environment variables
-ENV PYTHONPATH=/app
-ENV PYTHONUNBUFFERED=1
+ENV PATH="/opt/venv/bin:$PATH" \
+    PYTHONPATH=/app \
+    PYTHONUNBUFFERED=1 \
+    PYTHONDONTWRITEBYTECODE=1
 
 # Expose the port the app runs on
 EXPOSE 8000

Dockerfile.local

@@ -1,13 +1,31 @@
+# syntax=docker/dockerfile:1
+
 # Local development Dockerfile (avoids CI-only build metadata files)
-FROM python:3.14.1 AS builder
 
-WORKDIR /app
+# ── Stage 1: Python dependency builder ──────────────────────────────────────
+FROM python:3.14.3-slim AS builder
 
-COPY requirements.txt /app/
-RUN pip install --no-cache-dir -r requirements.txt
+WORKDIR /build
 
-# ── Documentation build stage ───────────────────────────────────────────────
-FROM python:3.14.1-slim AS docs-builder
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        build-essential \
+        libffi-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Create an isolated virtual environment
+RUN python -m venv /opt/venv
+
+ENV PATH="/opt/venv/bin:$PATH" \
+    PYTHONDONTWRITEBYTECODE=1 \
+    PIP_NO_CACHE_DIR=1
+
+COPY requirements.txt /build/
+RUN pip install --no-cache-dir -r requirements.txt \
+    && find /opt/venv -type f -name "*.pyc" -delete \
+    && find /opt/venv -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
+
+# ── Stage 2: Documentation builder ──────────────────────────────────────────
+FROM python:3.14.3-slim AS docs-builder
 
 WORKDIR /docs
 
@@ -19,23 +37,25 @@ COPY mkdocs.yml /docs/mkdocs.yml
 
 RUN mkdocs build --config-file /docs/mkdocs.yml --site-dir /docs/docs_build
 
-FROM python:3.14.1-slim
+# ── Stage 3: Runtime image ───────────────────────────────────────────────────
+FROM python:3.14.3-slim
 
 WORKDIR /app
 
-COPY --from=builder /usr/local/lib/python3.14/site-packages /usr/local/lib/python3.14/site-packages
-COPY --from=builder /usr/local/bin /usr/local/bin
+COPY --from=builder /opt/venv /opt/venv
 
 # Install system-level OCR tools required for local OCR workflows:
 #   tesseract-ocr   – OCR engine used by pytesseract and ocrmypdf
 #   ghostscript     – required by ocrmypdf for PDF/PS operations
 #   poppler-utils   – provides pdfinfo/pdftoppm used by pdf2image
 #   unpaper         – optional deskewing pre-processor used by ocrmypdf
+#   wget            – used by ocr_language_manager to download tessdata files
 RUN apt-get update && apt-get install -y --no-install-recommends \
         tesseract-ocr \
         ghostscript \
         poppler-utils \
         unpaper \
+        wget \
     && apt-get clean && rm -rf /var/lib/apt/lists/*
 
 COPY ./app /app/app
@@ -53,11 +73,13 @@ COPY --from=docs-builder /docs/docs_build /app/docs_build
 RUN echo "local" > /app/GIT_SHA \
     && echo "local" > /app/RUNTIME_INFO
 
-RUN mkdir -p /app/runtime_info
-RUN mkdir -p /workdir
+# Create necessary runtime directories in a single layer
+RUN mkdir -p /app/runtime_info /workdir
 
-ENV PYTHONPATH=/app
-ENV PYTHONUNBUFFERED=1
+ENV PATH="/opt/venv/bin:$PATH" \
+    PYTHONPATH=/app \
+    PYTHONUNBUFFERED=1 \
+    PYTHONDONTWRITEBYTECODE=1
 
 EXPOSE 8000
 

GIT_SHA

@@ -1 +1 @@
-237af31
+34457f9

RUNTIME_INFO

@@ -1,10 +1,10 @@
 DocuElevate Build Information
 ==============================
-Version: 0.145.2
-Build Date: 2026-03-15T21:39:26Z
-Git Commit: 237af31f5fe598cfc3c08f2bbba79b3d0925787e
-Git Short SHA: 237af31
+Version: 0.172.2
+Build Date: 2026-03-23T14:11:22Z
+Git Commit: 34457f977509ce145b7411e83982a96b0fd0e33e
+Git Short SHA: 34457f9
 Git Branch: main
-Commit Date: 2026-03-15T22:39:04+01:00
-Build Timestamp: 2026-03-15T21:39:26Z
+Commit Date: 2026-03-23T15:10:59+01:00
+Build Timestamp: 2026-03-23T14:11:22Z
 ==============================

SECURITY_AUDIT.md

@@ -1,6 +1,6 @@
 # Security Audit Report
 
-**Date:** 2026-02-12
+**Date:** 2026-03-23
 **Status:** Bandit Security Scan Completed - All Critical/High/Medium Issues Resolved
 
 ## Executive Summary
@@ -9,6 +9,15 @@ This document tracks security vulnerabilities found in DocuElevate and their rem
 
 ## Recent Security Fixes
 
+### Insecure API Endpoint Exposing Integration Credentials ✅ FIXED (2026-03-23)
+**Severity:** HIGH
+
+**Issue:** The endpoint `GET /api/integrations/{integration_id}/credentials` exposed integration credentials (e.g. passwords, API keys) in plaintext over the API. Although requiring login, this allowed anyone with an active user session to extract the raw credentials. The frontend used this endpoint for testing integration connections.
+
+**Remediation:**
+- Removed the `/credentials` endpoint entirely.
+- Added a new `POST /api/integrations/{integration_id}/test` endpoint that securely runs connection tests server-side without returning the decrypted credentials to the client.
+
 ### CVE-2023-36464: PyPDF2/pypdf Infinite Loop Vulnerability ✅ FIXED (2026-02-12)
 
 **Severity:** Moderate (CVSS: 5.5)

VERSION

@@ -1 +1 @@
-0.145.2
+0.172.2

app/api/__init__.py

@@ -9,9 +9,12 @@ from fastapi import APIRouter
 from app.api.admin_users import router as admin_users_router
 from app.api.api_tokens import router as api_tokens_router
 from app.api.audit_logs import router as audit_logs_router
+from app.api.automation import router as automation_router
 from app.api.azure import router as azure_router
 from app.api.backup import router as backup_router
 from app.api.billing import router as billing_router
+from app.api.classification_rules import router as classification_rules_router
+from app.api.comments import router as comments_router
 from app.api.compliance import router as compliance_router
 from app.api.database import router as database_router
 from app.api.diagnostic import router as diagnostic_router
@@ -33,16 +36,21 @@ from app.api.pipelines import router as pipelines_router
 from app.api.plans import router as plans_router
 from app.api.process import router as process_router
 from app.api.profile import router as profile_router
+from app.api.qr_auth import router as qr_auth_router
 from app.api.queue import router as queue_router
 from app.api.routing_rules import router as routing_rules_router
 from app.api.saved_searches import router as saved_searches_router
 from app.api.scheduled_jobs import router as scheduled_jobs_router
 from app.api.search import router as search_router
+from app.api.sessions import router as sessions_router
 from app.api.settings import router as settings_router
 from app.api.shared_links import public_router as shared_links_public_router
 from app.api.shared_links import router as shared_links_router
+from app.api.sharing import router as sharing_router
 from app.api.similarity import router as similarity_router
 from app.api.subscriptions import router as subscriptions_router
+from app.api.system_reset import router as system_reset_router
+from app.api.translation import router as translation_router
 from app.api.url_upload import router as url_upload_router
 
 # Import all the individual routers
@@ -95,4 +103,12 @@ router.include_router(scheduled_jobs_router)
 router.include_router(audit_logs_router)
 router.include_router(i18n_router)
 router.include_router(mobile_router)
+router.include_router(sessions_router)
+router.include_router(qr_auth_router)
 router.include_router(compliance_router)
+router.include_router(system_reset_router)
+router.include_router(translation_router)
+router.include_router(classification_rules_router)
+router.include_router(automation_router)
+router.include_router(comments_router)
+router.include_router(sharing_router)

app/api/api_tokens.py

@@ -13,7 +13,7 @@ plaintext is returned exactly once at creation time.
 import hashlib
 import logging
 import secrets
-from datetime import datetime, timezone
+from datetime import datetime, timedelta, timezone
 from typing import Annotated, Any
 
 from fastapi import APIRouter, Depends, HTTPException, Request, status
@@ -42,6 +42,9 @@ TOKEN_HASH_ITERATIONS = 100_000
 #: PBKDF2 salt for API token hashing (not secret, but fixed for determinism).
 TOKEN_HASH_SALT = b"api-token-v1"
 
+#: Name prefix used for tokens created by the mobile app flow.
+MOBILE_TOKEN_PREFIX = "Mobile App"
+
 
 # ---------------------------------------------------------------------------
 # Auth helper
@@ -91,6 +94,21 @@ def hash_token(token: str) -> str:
     return dk.hex()
 
 
+def _token_to_dict(t: ApiToken) -> dict[str, Any]:
+    """Convert an ``ApiToken`` ORM instance to a serialisable dict."""
+    return {
+        "id": t.id,
+        "name": t.name,
+        "token_prefix": t.token_prefix,
+        "is_active": t.is_active,
+        "last_used_at": t.last_used_at,
+        "last_used_ip": t.last_used_ip,
+        "created_at": t.created_at,
+        "revoked_at": t.revoked_at,
+        "expires_at": t.expires_at,
+    }
+
+
 # ---------------------------------------------------------------------------
 # Pydantic schemas
 # ---------------------------------------------------------------------------
@@ -100,6 +118,12 @@ class TokenCreate(BaseModel):
     """Schema for creating a new API token."""
 
     name: str = Field(..., min_length=1, max_length=255, description="Human-readable label for the token")
+    expires_in_days: int | None = Field(
+        default=None,
+        ge=1,
+        le=3650,  # Maximum 10 years; keeps tokens from being effectively permanent while allowing long-lived CI/CD tokens.
+        description="Optional lifetime in days. If omitted the token never expires.",
+    )
 
 
 class TokenResponse(BaseModel):
@@ -113,6 +137,7 @@ class TokenResponse(BaseModel):
     last_used_ip: str | None
     created_at: datetime | None
     revoked_at: datetime | None
+    expires_at: datetime | None
 
     model_config = {"from_attributes": True}
 
@@ -143,11 +168,16 @@ async def create_token(
     token_hash_value = hash_token(plaintext)
     prefix = plaintext[:12]  # "de_" prefix + 9 random chars = 12 chars total
 
+    expires_at = None
+    if body.expires_in_days is not None:
+        expires_at = datetime.now(timezone.utc) + timedelta(days=body.expires_in_days)
+
     db_token = ApiToken(
         owner_id=owner_id,
         name=body.name,
         token_hash=token_hash_value,
         token_prefix=prefix,
+        expires_at=expires_at,
     )
     try:
         db.add(db_token)
@@ -168,6 +198,7 @@ async def create_token(
         "last_used_ip": db_token.last_used_ip,
         "created_at": db_token.created_at,
         "revoked_at": db_token.revoked_at,
+        "expires_at": db_token.expires_at,
         "token": plaintext,
     }
 
@@ -177,48 +208,114 @@ async def list_tokens(
     owner_id: CurrentOwner,
     db: DbSession,
 ) -> list[dict[str, Any]]:
-    """List all API tokens for the authenticated user."""
-    tokens = db.query(ApiToken).filter(ApiToken.owner_id == owner_id).order_by(ApiToken.created_at.desc()).all()
-    return [
-        {
-            "id": t.id,
-            "name": t.name,
-            "token_prefix": t.token_prefix,
-            "is_active": t.is_active,
-            "last_used_at": t.last_used_at,
-            "last_used_ip": t.last_used_ip,
-            "created_at": t.created_at,
-            "revoked_at": t.revoked_at,
-        }
-        for t in tokens
-    ]
+    """List non-mobile API tokens for the authenticated user.
+
+    Mobile tokens (whose names start with ``"Mobile App"``) are excluded
+    from this list; they are managed on the dedicated Devices page via
+    ``GET /api/api-tokens/mobile``.
+    """
+    tokens = (
+        db.query(ApiToken)
+        .filter(
+            ApiToken.owner_id == owner_id,
+            ~ApiToken.name.startswith(MOBILE_TOKEN_PREFIX),
+        )
+        .order_by(ApiToken.created_at.desc())
+        .all()
+    )
+    return [_token_to_dict(t) for t in tokens]
+
+
+@router.get("/mobile", response_model=list[TokenResponse])
+async def list_mobile_tokens(
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> list[dict[str, Any]]:
+    """List mobile API tokens for the authenticated user.
+
+    Returns tokens whose names start with ``"Mobile App"`` — these are
+    created via the mobile SSO flow or QR code login.
+    """
+    tokens = (
+        db.query(ApiToken)
+        .filter(
+            ApiToken.owner_id == owner_id,
+            ApiToken.name.startswith(MOBILE_TOKEN_PREFIX),
+        )
+        .order_by(ApiToken.created_at.desc())
+        .all()
+    )
+    return [_token_to_dict(t) for t in tokens]
 
 
 @router.delete("/{token_id}", status_code=status.HTTP_200_OK)
-async def revoke_token(
+async def revoke_or_delete_token(
     token_id: int,
     owner_id: CurrentOwner,
     db: DbSession,
 ) -> dict[str, str]:
-    """Revoke (soft-delete) an API token.
+    """Revoke or permanently delete an API token.
 
-    The token row is kept for audit purposes but marked inactive with a
-    ``revoked_at`` timestamp.
+    * **Active token** – soft-revoked: the row is kept for audit purposes
+      but marked inactive with a ``revoked_at`` timestamp.
+    * **Already-revoked token** – hard-deleted: the row is permanently
+      removed from the database.
     """
     db_token = db.query(ApiToken).filter(ApiToken.id == token_id, ApiToken.owner_id == owner_id).first()
     if not db_token:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Token not found")
 
-    if not db_token.is_active:
-        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Token is already revoked")
+    if db_token.is_active:
+        # Soft-revoke the active token.
+        try:
+            db_token.is_active = False
+            db_token.revoked_at = datetime.now(timezone.utc)
+            db.commit()
+        except Exception:
+            db.rollback()
+            raise
+        logger.info("API token revoked: id=%s owner=%s", token_id, owner_id)
+        return {"detail": "Token revoked"}
 
+    # Hard-delete an already-revoked token.
     try:
-        db_token.is_active = False
-        db_token.revoked_at = datetime.now(timezone.utc)
+        db.delete(db_token)
         db.commit()
     except Exception:
         db.rollback()
         raise
+    logger.info("API token permanently deleted: id=%s owner=%s", token_id, owner_id)
+    return {"detail": "Token deleted"}
 
-    logger.info("API token revoked: id=%s owner=%s", token_id, owner_id)
-    return {"detail": "Token revoked"}
+
+@router.post("/{token_id}/reactivate", status_code=status.HTTP_200_OK, response_model=TokenResponse)
+async def reactivate_token(
+    token_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Reactivate a previously revoked API token.
+
+    Clears the ``revoked_at`` timestamp and sets ``is_active`` back to
+    ``True``.  The token can be used for authentication again immediately.
+    If the token had an ``expires_at`` in the past the caller should
+    consider re-creating a new token instead.
+    """
+    db_token = db.query(ApiToken).filter(ApiToken.id == token_id, ApiToken.owner_id == owner_id).first()
+    if not db_token:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Token not found")
+
+    if db_token.is_active:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Token is already active")
+
+    try:
+        db_token.is_active = True
+        db_token.revoked_at = None
+        db.commit()
+        db.refresh(db_token)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("API token reactivated: id=%s owner=%s", token_id, owner_id)
+    return _token_to_dict(db_token)

app/api/automation.py

@@ -0,0 +1,311 @@
+"""API endpoints for Zapier / Make.com automation integration.
+
+Provides a REST hooks subscription interface for outgoing triggers and
+incoming action endpoints that external automation platforms can call.
+
+Outgoing triggers:
+    External platforms subscribe to DocuElevate events via
+    ``POST /api/automation/hooks/subscribe``.  When a subscribed event
+    fires, DocuElevate POSTs a flat Zapier-compatible JSON payload to the
+    registered ``target_url``.
+
+Incoming actions:
+    ``POST /api/automation/actions/upload`` allows automation platforms to
+    push documents into DocuElevate for processing.
+
+Authentication:
+    All endpoints require a valid API token via ``Authorization: Bearer``
+    header.
+"""
+
+import json
+import logging
+import os
+import tempfile
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.config import settings
+from app.database import get_db
+from app.models import AutomationHook
+from app.utils.automation_hooks import SAMPLE_PAYLOADS
+from app.utils.webhook import VALID_EVENTS
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/automation", tags=["automation"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+# ---------------------------------------------------------------------------
+# Auth helper – require a valid API token (Bearer)
+# ---------------------------------------------------------------------------
+
+
+def _require_api_user(request: Request) -> dict:
+    """Ensure the caller is authenticated via session or API token.
+
+    Raises:
+        HTTPException: 401 if not authenticated, 403 if automation hooks are disabled.
+    """
+    if not settings.automation_hooks_enabled:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Automation hooks are disabled",
+        )
+
+    # Check for API-token user first (set by auth middleware)
+    user = getattr(request.state, "api_token_user", None)
+    if user:
+        return user
+
+    # Fall back to session user
+    user = request.session.get("user")
+    if user:
+        return user
+
+    raise HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Authentication required (Bearer token or session)",
+    )
+
+
+AuthUser = Annotated[dict, Depends(_require_api_user)]
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class HookSubscribe(BaseModel):
+    """Schema for subscribing to automation hook events."""
+
+    target_url: str = Field(..., min_length=1, max_length=2048, description="URL to POST event payloads to")
+    events: list[str] = Field(..., min_length=1, description="Event types to subscribe to")
+    secret: str | None = Field(default=None, max_length=512, description="Optional HMAC-SHA256 signing secret")
+    hook_type: str = Field(
+        default="generic",
+        max_length=50,
+        description="Platform identifier (zapier, make, generic)",
+    )
+    description: str | None = Field(default=None, max_length=500, description="Optional human-readable label")
+
+
+class HookResponse(BaseModel):
+    """Schema returned when listing or creating hooks."""
+
+    id: int
+    target_url: str
+    events: list[str]
+    is_active: bool
+    hook_type: str
+    description: str | None
+    has_secret: bool
+
+    model_config = {"from_attributes": True}
+
+
+class ActionUploadResponse(BaseModel):
+    """Response after an automation action uploads a document."""
+
+    status: str
+    filename: str
+    task_id: str | None = None
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _validate_events(events: list[str]) -> None:
+    """Raise 422 if any event name is not recognised."""
+    invalid = set(events) - VALID_EVENTS
+    if invalid:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"Invalid event(s): {', '.join(sorted(invalid))}. Valid: {', '.join(sorted(VALID_EVENTS))}",
+        )
+
+
+def _hook_to_response(hook: AutomationHook) -> dict[str, Any]:
+    """Convert a DB model instance to a response dict."""
+    try:
+        events = json.loads(hook.events)
+    except (json.JSONDecodeError, TypeError):
+        events = []
+    return {
+        "id": hook.id,
+        "target_url": hook.target_url,
+        "events": events,
+        "is_active": hook.is_active,
+        "hook_type": hook.hook_type,
+        "description": hook.description,
+        "has_secret": hook.secret is not None and len(hook.secret) > 0,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Outgoing triggers – REST hooks subscription endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post(
+    "/hooks/subscribe",
+    status_code=status.HTTP_201_CREATED,
+    summary="Subscribe to automation events (REST hooks)",
+)
+def subscribe_hook(body: HookSubscribe, db: DbSession, user: AuthUser) -> dict[str, Any]:
+    """Register a new automation hook subscription.
+
+    Zapier and Make.com call this endpoint to subscribe to DocuElevate
+    events.  When an event fires, a flat JSON payload is POSTed to
+    ``target_url``.
+    """
+    _validate_events(body.events)
+
+    hook = AutomationHook(
+        target_url=body.target_url,
+        secret=body.secret,
+        events=json.dumps(sorted(body.events)),
+        is_active=True,
+        hook_type=body.hook_type or "generic",
+        description=body.description,
+    )
+    try:
+        db.add(hook)
+        db.commit()
+        db.refresh(hook)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Automation hook %d created (type=%s) for events %s", hook.id, hook.hook_type, body.events)
+    return _hook_to_response(hook)
+
+
+@router.delete(
+    "/hooks/{hook_id}",
+    status_code=status.HTTP_204_NO_CONTENT,
+    summary="Unsubscribe an automation hook",
+)
+def unsubscribe_hook(hook_id: int, db: DbSession, user: AuthUser) -> None:
+    """Remove an automation hook subscription.
+
+    Zapier calls this endpoint when a Zap is turned off or deleted.
+    """
+    hook = db.query(AutomationHook).filter(AutomationHook.id == hook_id).first()
+    if not hook:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Hook not found")
+
+    try:
+        db.delete(hook)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Automation hook %d deleted", hook_id)
+
+
+@router.get("/hooks", summary="List automation hook subscriptions")
+def list_hooks(db: DbSession, user: AuthUser) -> list[dict[str, Any]]:
+    """Return all active automation hook subscriptions."""
+    hooks = db.query(AutomationHook).order_by(AutomationHook.id).all()
+    return [_hook_to_response(h) for h in hooks]
+
+
+# ---------------------------------------------------------------------------
+# Outgoing triggers – sample data for Zapier field mapping
+# ---------------------------------------------------------------------------
+
+
+@router.get("/triggers/sample/{event}", summary="Get sample trigger data")
+def get_trigger_sample(event: str, user: AuthUser) -> list[dict[str, Any]]:
+    """Return sample payload data for the given event type.
+
+    Zapier uses this during Zap setup to discover available fields and
+    provide a mapping interface.  The response is wrapped in an array
+    as Zapier expects.
+    """
+    if event not in VALID_EVENTS:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail=f"Unknown event: {event}. Valid: {', '.join(sorted(VALID_EVENTS))}",
+        )
+
+    sample = SAMPLE_PAYLOADS.get(event, {"id": "evt_sample", "event": event, "timestamp": 0})
+    return [sample]
+
+
+# ---------------------------------------------------------------------------
+# Outgoing triggers – list valid events
+# ---------------------------------------------------------------------------
+
+
+@router.get("/events", summary="List valid automation event types")
+def list_events(user: AuthUser) -> list[str]:
+    """Return the list of valid event types that automation hooks can subscribe to."""
+    return sorted(VALID_EVENTS)
+
+
+# ---------------------------------------------------------------------------
+# Incoming actions – endpoints that Zapier / Make.com can call
+# ---------------------------------------------------------------------------
+
+
+@router.post("/actions/upload", summary="Upload a document (incoming action)")
+def action_upload(
+    request: Request,
+    db: DbSession,
+    user: AuthUser,
+    file: UploadFile = File(...),
+) -> dict[str, Any]:
+    """Accept a document upload from an automation platform.
+
+    This endpoint allows Zapier or Make.com to push a document into
+    DocuElevate for processing.  The file is saved to the work directory
+    and a background processing task is queued.
+    """
+    if not file.filename:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Filename is required")
+
+    # Sanitise filename to prevent path traversal attacks
+    safe_filename = os.path.basename(file.filename)
+    if not safe_filename:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Filename is required")
+
+    owner_id = user.get("preferred_username") or user.get("email") or user.get("id", "automation")
+    workdir = settings.workdir or tempfile.gettempdir()
+    upload_dir = os.path.join(workdir, "uploads")
+    os.makedirs(upload_dir, exist_ok=True)
+
+    dest_path = os.path.join(upload_dir, safe_filename)
+    try:
+        contents = file.file.read()
+        with open(dest_path, "wb") as f:
+            f.write(contents)
+    except Exception as exc:
+        logger.error("Failed to save uploaded file: %s", exc)
+        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="Failed to save file")
+
+    # Queue background processing
+    task_id = None
+    try:
+        from app.tasks.process_document import process_document
+
+        result = process_document.delay(dest_path, owner_id)
+        task_id = result.id
+        logger.info("Automation upload queued: file=%s, task=%s, owner=%s", safe_filename, task_id, owner_id)
+    except Exception as exc:
+        logger.warning("Could not queue processing task (Celery may be unavailable): %s", exc)
+
+    return {
+        "status": "accepted",
+        "filename": safe_filename,
+        "task_id": task_id,
+    }

app/api/billing.py

@@ -168,9 +168,8 @@ async def create_checkout_session(
     checkout_session = client.checkout.sessions.create(params=session_params)
 
     logger.info(
-        "Created Stripe checkout session %s for user %s plan %s",
+        "Created Stripe checkout session %s for plan %s",
         checkout_session.id,
-        owner_id,
         body.plan_id,
     )
     return {"checkout_url": checkout_session.url, "session_id": checkout_session.id}
@@ -213,7 +212,7 @@ async def create_portal_session(
         }
     )
 
-    logger.info("Created Stripe portal session for user %s", owner_id)
+    logger.info("Created Stripe portal session for user")
     return {"portal_url": portal.url}
 
 
@@ -261,7 +260,7 @@ async def stripe_webhook(request: Request, db: Session = Depends(get_db)) -> dic
 @require_login
 async def billing_success(request: Request) -> Any:
     """Show a success page after a completed Stripe Checkout."""
-    return _templates.TemplateResponse("billing_success.html", {"request": request})
+    return _templates.TemplateResponse(request, "billing_success.html")
 
 
 # ---------------------------------------------------------------------------

app/api/classification_rules.py

@@ -0,0 +1,325 @@
+"""Classification Rules API endpoints.
+
+Provides CRUD operations for managing custom document classification rules.
+System-wide rules (``owner_id IS NULL``) can only be managed by admins.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.database import get_db
+from app.models import ClassificationRuleModel
+from app.utils.classification_rules import (
+    BUILTIN_CATEGORIES,
+    RULE_TYPE_CONTENT,
+    RULE_TYPE_FILENAME,
+    RULE_TYPE_METADATA,
+)
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/classification-rules", tags=["classification"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+_VALID_RULE_TYPES = {RULE_TYPE_FILENAME, RULE_TYPE_CONTENT, RULE_TYPE_METADATA}
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_user_id(request: Request) -> str:
+    """Extract the user identifier from the request session."""
+    user = getattr(request.state, "user", None)
+    if user and hasattr(user, "get"):
+        return user.get("sub") or user.get("email") or "anonymous"
+    return "anonymous"
+
+
+def _is_admin(request: Request) -> bool:
+    """Check whether the current user is an admin."""
+    user = getattr(request.state, "user", None)
+    if user and hasattr(user, "get"):
+        groups = user.get("groups", [])
+        return "admin" in groups or "Admin" in groups
+    return False
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class RuleCreate(BaseModel):
+    """Schema for creating a classification rule."""
+
+    name: str = Field(..., min_length=1, max_length=255)
+    category: str = Field(..., min_length=1, max_length=100)
+    rule_type: str = Field(..., description="One of: filename_pattern, content_keyword, metadata_match")
+    pattern: str = Field(..., min_length=1, max_length=1000)
+    priority: int = Field(default=0, ge=0, le=1000)
+    case_sensitive: bool = False
+    enabled: bool = True
+
+
+class RuleUpdate(BaseModel):
+    """Schema for updating a classification rule."""
+
+    name: str | None = Field(default=None, min_length=1, max_length=255)
+    category: str | None = Field(default=None, min_length=1, max_length=100)
+    rule_type: str | None = Field(default=None)
+    pattern: str | None = Field(default=None, min_length=1, max_length=1000)
+    priority: int | None = Field(default=None, ge=0, le=1000)
+    case_sensitive: bool | None = None
+    enabled: bool | None = None
+
+
+class RuleResponse(BaseModel):
+    """Schema for a classification rule response."""
+
+    id: int
+    owner_id: str | None
+    name: str
+    category: str
+    rule_type: str
+    pattern: str
+    priority: int
+    case_sensitive: bool
+    enabled: bool
+
+    model_config = {"from_attributes": True}
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/categories")
+@require_login
+async def list_categories(request: Request) -> dict[str, str]:
+    """Return all built-in classification categories.
+
+    Custom categories created via rules are not included here; they are
+    discovered dynamically when rules are evaluated.
+    """
+    return BUILTIN_CATEGORIES
+
+
+@router.get("/rule-types")
+@require_login
+async def list_rule_types(request: Request) -> list[dict[str, str]]:
+    """Return the supported rule types with descriptions."""
+    return [
+        {
+            "type": RULE_TYPE_FILENAME,
+            "label": "Filename Pattern",
+            "description": "Regex pattern matched against the original filename.",
+        },
+        {
+            "type": RULE_TYPE_CONTENT,
+            "label": "Content Keyword",
+            "description": "Pipe-separated keywords matched against the OCR text.",
+        },
+        {
+            "type": RULE_TYPE_METADATA,
+            "label": "Metadata Match",
+            "description": "field=value pattern matched against existing AI metadata.",
+        },
+    ]
+
+
+@router.get("/")
+@require_login
+async def list_rules(request: Request, db: DbSession) -> list[dict[str, Any]]:
+    """List classification rules visible to the current user.
+
+    Returns both system rules (``owner_id IS NULL``) and the user's own rules.
+    """
+    user_id = _get_user_id(request)
+    rules = (
+        db.query(ClassificationRuleModel)
+        .filter((ClassificationRuleModel.owner_id.is_(None)) | (ClassificationRuleModel.owner_id == user_id))
+        .order_by(ClassificationRuleModel.priority.desc(), ClassificationRuleModel.id)
+        .all()
+    )
+    return [
+        {
+            "id": r.id,
+            "owner_id": r.owner_id,
+            "name": r.name,
+            "category": r.category,
+            "rule_type": r.rule_type,
+            "pattern": r.pattern,
+            "priority": r.priority,
+            "case_sensitive": r.case_sensitive,
+            "enabled": r.enabled,
+        }
+        for r in rules
+    ]
+
+
+@router.post("/", status_code=status.HTTP_201_CREATED)
+@require_login
+async def create_rule(request: Request, body: RuleCreate, db: DbSession) -> dict[str, Any]:
+    """Create a new custom classification rule.
+
+    The rule is owned by the current user.  Admins may create system-wide
+    rules by setting ``owner_id`` to ``null`` (not yet exposed).
+    """
+    if body.rule_type not in _VALID_RULE_TYPES:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Invalid rule_type. Must be one of: {', '.join(sorted(_VALID_RULE_TYPES))}",
+        )
+
+    user_id = _get_user_id(request)
+
+    # Check for duplicate name within the user's scope
+    existing = (
+        db.query(ClassificationRuleModel)
+        .filter(ClassificationRuleModel.owner_id == user_id, ClassificationRuleModel.name == body.name)
+        .first()
+    )
+    if existing:
+        raise HTTPException(
+            status_code=status.HTTP_409_CONFLICT,
+            detail=f"A rule named '{body.name}' already exists.",
+        )
+
+    rule = ClassificationRuleModel(
+        owner_id=user_id,
+        name=body.name,
+        category=body.category,
+        rule_type=body.rule_type,
+        pattern=body.pattern,
+        priority=body.priority,
+        case_sensitive=body.case_sensitive,
+        enabled=body.enabled,
+    )
+    try:
+        db.add(rule)
+        db.commit()
+        db.refresh(rule)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Classification rule created: id=%s, user=%s", rule.id, user_id)
+    return {
+        "id": rule.id,
+        "owner_id": rule.owner_id,
+        "name": rule.name,
+        "category": rule.category,
+        "rule_type": rule.rule_type,
+        "pattern": rule.pattern,
+        "priority": rule.priority,
+        "case_sensitive": rule.case_sensitive,
+        "enabled": rule.enabled,
+    }
+
+
+@router.get("/{rule_id}")
+@require_login
+async def get_rule(request: Request, rule_id: int, db: DbSession) -> dict[str, Any]:
+    """Get a single classification rule by ID."""
+    user_id = _get_user_id(request)
+    rule = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.id == rule_id).first()
+    if rule is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
+
+    # Users can see system rules and their own rules
+    if rule.owner_id is not None and rule.owner_id != user_id and not _is_admin(request):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
+
+    return {
+        "id": rule.id,
+        "owner_id": rule.owner_id,
+        "name": rule.name,
+        "category": rule.category,
+        "rule_type": rule.rule_type,
+        "pattern": rule.pattern,
+        "priority": rule.priority,
+        "case_sensitive": rule.case_sensitive,
+        "enabled": rule.enabled,
+    }
+
+
+@router.put("/{rule_id}")
+@require_login
+async def update_rule(request: Request, rule_id: int, body: RuleUpdate, db: DbSession) -> dict[str, Any]:
+    """Update an existing classification rule.
+
+    Users can only update their own rules.  Admins can update any rule.
+    """
+    user_id = _get_user_id(request)
+    rule = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.id == rule_id).first()
+    if rule is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
+
+    if rule.owner_id != user_id and not _is_admin(request):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Cannot modify this rule")
+
+    if body.rule_type is not None and body.rule_type not in _VALID_RULE_TYPES:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Invalid rule_type. Must be one of: {', '.join(sorted(_VALID_RULE_TYPES))}",
+        )
+
+    update_data = body.model_dump(exclude_unset=True)
+    for field_name, value in update_data.items():
+        setattr(rule, field_name, value)
+
+    try:
+        db.commit()
+        db.refresh(rule)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Classification rule updated: id=%s, user=%s", rule.id, user_id)
+    return {
+        "id": rule.id,
+        "owner_id": rule.owner_id,
+        "name": rule.name,
+        "category": rule.category,
+        "rule_type": rule.rule_type,
+        "pattern": rule.pattern,
+        "priority": rule.priority,
+        "case_sensitive": rule.case_sensitive,
+        "enabled": rule.enabled,
+    }
+
+
+@router.delete("/{rule_id}", status_code=status.HTTP_204_NO_CONTENT)
+@require_login
+async def delete_rule(request: Request, rule_id: int, db: DbSession) -> None:
+    """Delete a classification rule.
+
+    Users can only delete their own rules.  Admins can delete any rule.
+    """
+    user_id = _get_user_id(request)
+    rule = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.id == rule_id).first()
+    if rule is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Rule not found")
+
+    if rule.owner_id != user_id and not _is_admin(request):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Cannot delete this rule")
+
+    try:
+        db.delete(rule)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Classification rule deleted: id=%s, user=%s", rule_id, user_id)

app/api/comments.py

@@ -0,0 +1,751 @@
+"""Document comments and annotations API endpoints.
+
+Provides CRUD operations for threaded comments on documents,
+text annotations on PDF pages, and a list of mentionable users
+for the @mention feature.
+"""
+
+import json
+import logging
+import re
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Body, Depends, HTTPException, Request, status
+from sqlalchemy.orm import Session
+
+from app.auth import get_current_user_id, require_login
+from app.database import get_db
+from app.models import (
+    FILE_SHARE_ROLE_VIEWER,
+    DocumentAnnotation,
+    DocumentComment,
+    FileRecord,
+    FileShare,
+    UserProfile,
+)
+from app.utils.user_scope import get_current_owner_id, has_file_role
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(tags=["comments"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# Constraints
+MAX_COMMENT_BODY_LENGTH = 10_000
+MAX_ANNOTATION_CONTENT_LENGTH = 5_000
+
+# Allowed annotation types
+ALLOWED_ANNOTATION_TYPES = frozenset({"note", "highlight", "underline", "strikethrough"})
+
+# Simple pattern for @mentions – matches @username tokens inside comment body
+_MENTION_PATTERN = re.compile(r"@([\w.\-]+)")
+
+
+def _extract_mentions(body: str) -> list[str]:
+    """Extract unique @mentioned usernames from a comment body.
+
+    Args:
+        body: The raw comment text.
+
+    Returns:
+        A deduplicated list of mentioned usernames (without the ``@`` prefix).
+    """
+    return list(dict.fromkeys(_MENTION_PATTERN.findall(body)))
+
+
+def _serialize_comment(c: DocumentComment) -> dict[str, Any]:
+    """Serialize a DocumentComment to a JSON-friendly dict.
+
+    Args:
+        c: The comment model instance.
+
+    Returns:
+        A dictionary representation of the comment.
+    """
+    mentions: list[str] = []
+    if c.mentions:
+        try:
+            mentions = json.loads(c.mentions)
+        except (json.JSONDecodeError, TypeError):
+            pass
+    return {
+        "id": c.id,
+        "file_id": c.file_id,
+        "user_id": c.user_id,
+        "parent_id": c.parent_id,
+        "body": c.body,
+        "mentions": mentions,
+        "is_resolved": c.is_resolved,
+        "created_at": c.created_at.isoformat() if c.created_at else None,
+        "updated_at": c.updated_at.isoformat() if c.updated_at else None,
+    }
+
+
+def _serialize_annotation(a: DocumentAnnotation) -> dict[str, Any]:
+    """Serialize a DocumentAnnotation to a JSON-friendly dict.
+
+    Args:
+        a: The annotation model instance.
+
+    Returns:
+        A dictionary representation of the annotation.
+    """
+    return {
+        "id": a.id,
+        "file_id": a.file_id,
+        "user_id": a.user_id,
+        "page": a.page,
+        "x": a.x,
+        "y": a.y,
+        "width": a.width,
+        "height": a.height,
+        "content": a.content,
+        "annotation_type": a.annotation_type,
+        "color": a.color,
+        "created_at": a.created_at.isoformat() if a.created_at else None,
+        "updated_at": a.updated_at.isoformat() if a.updated_at else None,
+    }
+
+
+def _build_thread_tree(comments: list[DocumentComment]) -> list[dict[str, Any]]:
+    """Organize a flat list of comments into a threaded tree structure.
+
+    Top-level comments (``parent_id is None``) appear as root nodes.
+    Replies are nested inside their parent's ``replies`` list.
+
+    Args:
+        comments: All comments for a given document, ordered by ``created_at``.
+
+    Returns:
+        A list of root-level comment dicts, each with a ``replies`` key.
+    """
+    by_id: dict[int, dict[str, Any]] = {}
+    roots: list[dict[str, Any]] = []
+
+    for c in comments:
+        node = _serialize_comment(c)
+        node["replies"] = []
+        by_id[c.id] = node
+
+    for c in comments:
+        node = by_id[c.id]
+        if c.parent_id and c.parent_id in by_id:
+            by_id[c.parent_id]["replies"].append(node)
+        else:
+            roots.append(node)
+
+    return roots
+
+
+# ---------------------------------------------------------------------------
+# Comments endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/files/{file_id}/comments")
+@require_login
+def list_comments(request: Request, file_id: int, db: DbSession):
+    """List all comments for a document, organized into threads.
+
+    Returns a threaded tree where top-level comments contain nested
+    ``replies``.  Requires at least viewer access.
+
+    Path Parameters:
+        file_id: The ID of the document.
+
+    Returns:
+        A dict with ``file_id``, ``comments`` (threaded), and ``total``.
+    """
+    user_id = get_current_owner_id(request)
+    user = request.session.get("user")
+    is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if not is_admin and not has_file_role(file_record, user_id, db, minimum_role=FILE_SHARE_ROLE_VIEWER):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    comments = (
+        db.query(DocumentComment).filter(DocumentComment.file_id == file_id).order_by(DocumentComment.created_at).all()
+    )
+
+    return {
+        "file_id": file_id,
+        "comments": _build_thread_tree(comments),
+        "total": len(comments),
+    }
+
+
+@router.post("/files/{file_id}/comments", status_code=status.HTTP_201_CREATED)
+@require_login
+def create_comment(
+    request: Request,
+    file_id: int,
+    db: DbSession,
+    body: str = Body(..., embed=True),
+    parent_id: int | None = Body(None, embed=True),
+):
+    """Create a new comment on a document.
+
+    Automatically extracts @mentions from the comment body and stores
+    them for later notification or UI highlighting.  When multi-user
+    mode is enabled, any mentioned user that does not already have
+    access to the document is automatically granted ``viewer`` access by
+    the file owner so they can read the file and continue the discussion.
+
+    Path Parameters:
+        file_id: The ID of the document to comment on.
+
+    Request body (JSON):
+        body: Comment text (required, max 10 000 characters).
+        parent_id: ID of the parent comment for threaded replies (optional).
+
+    Returns:
+        The created comment object.
+    """
+    user_id = get_current_user_id(request)
+    owner_id = get_current_owner_id(request)
+    user = request.session.get("user")
+    is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if not is_admin and not has_file_role(file_record, owner_id, db, minimum_role=FILE_SHARE_ROLE_VIEWER):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if not isinstance(body, str) or not body.strip():
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="body is required and must be non-empty",
+        )
+    body = body.strip()
+    if len(body) > MAX_COMMENT_BODY_LENGTH:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"body must be at most {MAX_COMMENT_BODY_LENGTH} characters",
+        )
+
+    if parent_id is not None:
+        parent = (
+            db.query(DocumentComment)
+            .filter(DocumentComment.id == parent_id, DocumentComment.file_id == file_id)
+            .first()
+        )
+        if not parent:
+            raise HTTPException(
+                status_code=status.HTTP_404_NOT_FOUND,
+                detail="Parent comment not found",
+            )
+
+    mentions = _extract_mentions(body)
+
+    comment = DocumentComment(
+        file_id=file_id,
+        user_id=user_id,
+        parent_id=parent_id,
+        body=body,
+        mentions=json.dumps(mentions) if mentions else None,
+    )
+
+    try:
+        db.add(comment)
+        db.flush()  # write comment so we can get its id before committing
+
+        # Auto-share the file with mentioned users that don't have access yet.
+        # Only do this in multi-user mode and only when the file has an owner
+        # (unowned files are already visible to all authenticated users).
+        if mentions and file_record.owner_id is not None:
+            from app.config import settings as _settings
+
+            if _settings.multi_user_enabled:
+                for mentioned_user in mentions:
+                    # Skip the file owner (already has full access) and the commenter
+                    # themselves (they already have access to be posting a comment).
+                    if mentioned_user in {file_record.owner_id, owner_id}:
+                        continue
+                    existing_share = (
+                        db.query(FileShare)
+                        .filter(
+                            FileShare.file_id == file_id,
+                            FileShare.shared_with_user_id == mentioned_user,
+                        )
+                        .first()
+                    )
+                    if not existing_share:
+                        auto_share = FileShare(
+                            file_id=file_id,
+                            owner_id=file_record.owner_id,
+                            shared_with_user_id=mentioned_user,
+                            role=FILE_SHARE_ROLE_VIEWER,
+                        )
+                        db.add(auto_share)
+                        logger.info(
+                            "Auto-shared file_id=%s with mentioned user=%s as viewer",
+                            file_id,
+                            mentioned_user,
+                        )
+
+        db.commit()
+        db.refresh(comment)
+    except HTTPException:
+        raise
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create comment on file_id=%s", file_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to create comment",
+        )
+
+    logger.info("Comment created: id=%s, file_id=%s, user=%s", comment.id, file_id, user_id)
+    return _serialize_comment(comment)
+
+
+@router.put("/files/{file_id}/comments/{comment_id}")
+@require_login
+def update_comment(
+    request: Request,
+    file_id: int,
+    comment_id: int,
+    db: DbSession,
+    body: str = Body(..., embed=True),
+):
+    """Update the body of an existing comment.
+
+    Only the comment author may update the comment.  Mentions are
+    re-extracted from the updated body.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        comment_id: The ID of the comment to update.
+
+    Request body (JSON):
+        body: New comment text (required).
+
+    Returns:
+        The updated comment object.
+    """
+    user_id = get_current_user_id(request)
+
+    comment = (
+        db.query(DocumentComment).filter(DocumentComment.id == comment_id, DocumentComment.file_id == file_id).first()
+    )
+    if not comment:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Comment not found")
+
+    if comment.user_id != user_id:
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="You can only edit your own comments")
+
+    if not isinstance(body, str) or not body.strip():
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="body is required and must be non-empty",
+        )
+    body = body.strip()
+    if len(body) > MAX_COMMENT_BODY_LENGTH:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"body must be at most {MAX_COMMENT_BODY_LENGTH} characters",
+        )
+
+    mentions = _extract_mentions(body)
+    comment.body = body
+    comment.mentions = json.dumps(mentions) if mentions else None
+
+    try:
+        db.commit()
+        db.refresh(comment)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to update comment id=%s", comment_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to update comment",
+        )
+
+    logger.info("Comment updated: id=%s, user=%s", comment_id, user_id)
+    return _serialize_comment(comment)
+
+
+@router.delete("/files/{file_id}/comments/{comment_id}", status_code=status.HTTP_204_NO_CONTENT)
+@require_login
+def delete_comment(request: Request, file_id: int, comment_id: int, db: DbSession):
+    """Delete a comment.
+
+    Only the comment author may delete the comment.  Replies to the
+    deleted comment are **not** removed — they become orphaned root
+    comments so that conversation context is preserved.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        comment_id: The ID of the comment to delete.
+    """
+    user_id = get_current_user_id(request)
+
+    comment = (
+        db.query(DocumentComment).filter(DocumentComment.id == comment_id, DocumentComment.file_id == file_id).first()
+    )
+    if not comment:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Comment not found")
+
+    if comment.user_id != user_id:
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="You can only delete your own comments")
+
+    try:
+        db.delete(comment)
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to delete comment id=%s", comment_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to delete comment",
+        )
+
+    logger.info("Comment deleted: id=%s, user=%s", comment_id, user_id)
+
+
+@router.patch("/files/{file_id}/comments/{comment_id}/resolve")
+@require_login
+def resolve_comment(
+    request: Request,
+    file_id: int,
+    comment_id: int,
+    db: DbSession,
+    is_resolved: bool = Body(..., embed=True),
+):
+    """Mark a top-level comment thread as resolved or unresolved.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        comment_id: The ID of the comment to resolve / unresolve.
+
+    Request body (JSON):
+        is_resolved: ``true`` to resolve, ``false`` to unresolve.
+
+    Returns:
+        The updated comment object.
+    """
+    comment = (
+        db.query(DocumentComment).filter(DocumentComment.id == comment_id, DocumentComment.file_id == file_id).first()
+    )
+    if not comment:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Comment not found")
+
+    comment.is_resolved = is_resolved
+
+    try:
+        db.commit()
+        db.refresh(comment)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to resolve comment id=%s", comment_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to update comment",
+        )
+
+    logger.info("Comment %s: id=%s", "resolved" if is_resolved else "unresolved", comment_id)
+    return _serialize_comment(comment)
+
+
+# ---------------------------------------------------------------------------
+# Annotations endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/files/{file_id}/annotations")
+@require_login
+def list_annotations(request: Request, file_id: int, db: DbSession):
+    """List all annotations for a document.
+
+    Requires at least viewer access.
+
+    Path Parameters:
+        file_id: The ID of the document.
+
+    Returns:
+        A dict with ``file_id``, ``annotations``, and ``total``.
+    """
+    user_id = get_current_owner_id(request)
+    user = request.session.get("user")
+    is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if not is_admin and not has_file_role(file_record, user_id, db, minimum_role=FILE_SHARE_ROLE_VIEWER):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    annotations = (
+        db.query(DocumentAnnotation)
+        .filter(DocumentAnnotation.file_id == file_id)
+        .order_by(DocumentAnnotation.page, DocumentAnnotation.created_at)
+        .all()
+    )
+
+    return {
+        "file_id": file_id,
+        "annotations": [_serialize_annotation(a) for a in annotations],
+        "total": len(annotations),
+    }
+
+
+@router.post("/files/{file_id}/annotations", status_code=status.HTTP_201_CREATED)
+@require_login
+def create_annotation(
+    request: Request,
+    file_id: int,
+    db: DbSession,
+    page: int = Body(..., embed=True),
+    x: float = Body(..., embed=True),
+    y: float = Body(..., embed=True),
+    content: str = Body(..., embed=True),
+    width: float = Body(0, embed=True),
+    height: float = Body(0, embed=True),
+    annotation_type: str = Body("note", embed=True),
+    color: str | None = Body(None, embed=True),
+):
+    """Create a new annotation on a PDF page.
+
+    Path Parameters:
+        file_id: The ID of the document.
+
+    Request body (JSON):
+        page: Page number (1-based, required).
+        x: Horizontal position on the page (required).
+        y: Vertical position on the page (required).
+        content: Annotation text (required, max 5 000 characters).
+        width: Width of the annotation bounding box (default 0).
+        height: Height of the annotation bounding box (default 0).
+        annotation_type: One of ``note``, ``highlight``, ``underline``,
+            ``strikethrough`` (default ``note``).
+        color: Optional CSS colour string (e.g. ``#ff0000``).
+
+    Returns:
+        The created annotation object.
+    """
+    user_id = get_current_user_id(request)
+    owner_id = get_current_owner_id(request)
+    user = request.session.get("user")
+    is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if not is_admin and not has_file_role(file_record, owner_id, db, minimum_role=FILE_SHARE_ROLE_VIEWER):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if not isinstance(content, str) or not content.strip():
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="content is required and must be non-empty",
+        )
+    content = content.strip()
+    if len(content) > MAX_ANNOTATION_CONTENT_LENGTH:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"content must be at most {MAX_ANNOTATION_CONTENT_LENGTH} characters",
+        )
+
+    if page < 1:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="page must be >= 1",
+        )
+
+    if annotation_type not in ALLOWED_ANNOTATION_TYPES:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"annotation_type must be one of: {', '.join(sorted(ALLOWED_ANNOTATION_TYPES))}",
+        )
+
+    annotation = DocumentAnnotation(
+        file_id=file_id,
+        user_id=user_id,
+        page=page,
+        x=x,
+        y=y,
+        width=width,
+        height=height,
+        content=content,
+        annotation_type=annotation_type,
+        color=color,
+    )
+
+    try:
+        db.add(annotation)
+        db.commit()
+        db.refresh(annotation)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create annotation on file_id=%s", file_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to create annotation",
+        )
+
+    logger.info("Annotation created: id=%s, file_id=%s, user=%s", annotation.id, file_id, user_id)
+    return _serialize_annotation(annotation)
+
+
+@router.put("/files/{file_id}/annotations/{annotation_id}")
+@require_login
+def update_annotation(
+    request: Request,
+    file_id: int,
+    annotation_id: int,
+    db: DbSession,
+    content: str | None = Body(None, embed=True),
+    x: float | None = Body(None, embed=True),
+    y: float | None = Body(None, embed=True),
+    width: float | None = Body(None, embed=True),
+    height: float | None = Body(None, embed=True),
+    annotation_type: str | None = Body(None, embed=True),
+    color: str | None = Body(None, embed=True),
+):
+    """Update an existing annotation.
+
+    Only the annotation author may update the annotation.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        annotation_id: The ID of the annotation to update.
+
+    Request body (JSON):
+        Any subset of ``content``, ``x``, ``y``, ``width``, ``height``,
+        ``annotation_type``, and ``color``.
+
+    Returns:
+        The updated annotation object.
+    """
+    user_id = get_current_user_id(request)
+
+    annotation = (
+        db.query(DocumentAnnotation)
+        .filter(DocumentAnnotation.id == annotation_id, DocumentAnnotation.file_id == file_id)
+        .first()
+    )
+    if not annotation:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Annotation not found")
+
+    if annotation.user_id != user_id:
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="You can only edit your own annotations")
+
+    if content is not None:
+        content = content.strip() if isinstance(content, str) else ""
+        if not content:
+            raise HTTPException(
+                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+                detail="content must be non-empty",
+            )
+        if len(content) > MAX_ANNOTATION_CONTENT_LENGTH:
+            raise HTTPException(
+                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+                detail=f"content must be at most {MAX_ANNOTATION_CONTENT_LENGTH} characters",
+            )
+        annotation.content = content
+
+    if x is not None:
+        annotation.x = x
+    if y is not None:
+        annotation.y = y
+    if width is not None:
+        annotation.width = width
+    if height is not None:
+        annotation.height = height
+    if annotation_type is not None:
+        if annotation_type not in ALLOWED_ANNOTATION_TYPES:
+            raise HTTPException(
+                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+                detail=f"annotation_type must be one of: {', '.join(sorted(ALLOWED_ANNOTATION_TYPES))}",
+            )
+        annotation.annotation_type = annotation_type
+    if color is not None:
+        annotation.color = color
+
+    try:
+        db.commit()
+        db.refresh(annotation)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to update annotation id=%s", annotation_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to update annotation",
+        )
+
+    logger.info("Annotation updated: id=%s, user=%s", annotation_id, user_id)
+    return _serialize_annotation(annotation)
+
+
+@router.delete("/files/{file_id}/annotations/{annotation_id}", status_code=status.HTTP_204_NO_CONTENT)
+@require_login
+def delete_annotation(request: Request, file_id: int, annotation_id: int, db: DbSession):
+    """Delete an annotation.
+
+    Only the annotation author may delete the annotation.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        annotation_id: The ID of the annotation to delete.
+    """
+    user_id = get_current_user_id(request)
+
+    annotation = (
+        db.query(DocumentAnnotation)
+        .filter(DocumentAnnotation.id == annotation_id, DocumentAnnotation.file_id == file_id)
+        .first()
+    )
+    if not annotation:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Annotation not found")
+
+    if annotation.user_id != user_id:
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="You can only delete your own annotations")
+
+    try:
+        db.delete(annotation)
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to delete annotation id=%s", annotation_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to delete annotation",
+        )
+
+    logger.info("Annotation deleted: id=%s, user=%s", annotation_id, user_id)
+
+
+# ---------------------------------------------------------------------------
+# Mentionable users endpoint
+# ---------------------------------------------------------------------------
+
+
+@router.get("/users/mentionable")
+@require_login
+def list_mentionable_users(request: Request, db: DbSession):
+    """List users that can be @mentioned in comments.
+
+    Returns all user profiles that are not blocked, sorted by
+    ``display_name``.
+
+    Returns:
+        A list of ``{user_id, display_name}`` objects.
+    """
+    profiles = db.query(UserProfile).filter(UserProfile.is_blocked.is_(False)).order_by(UserProfile.display_name).all()
+
+    return [
+        {
+            "user_id": p.user_id,
+            "display_name": p.display_name or p.user_id,
+        }
+        for p in profiles
+    ]

app/api/diagnostic.py

@@ -21,6 +21,67 @@ _DEFAULT_REDIS_URL = "redis://localhost:6379/0"
 
 router = APIRouter()
 
+# ---------------------------------------------------------------------------
+# Unauthenticated probe endpoints for Kubernetes liveness / readiness checks.
+# These intentionally skip authentication so that kubelet can reach them
+# without credentials.  They live under /diagnostic/healthz/* so that the
+# existing authenticated /diagnostic/health endpoint is unaffected.
+# ---------------------------------------------------------------------------
+
+
+@router.get("/diagnostic/healthz/live")
+async def liveness_probe() -> JSONResponse:
+    """Lightweight liveness probe for Kubernetes.
+
+    Returns **200 OK** as long as the process is running.  Kubernetes uses
+    this to decide whether to *restart* the container — it should therefore
+    be as cheap as possible and **never** check external dependencies.
+
+    **Authentication:** None (designed for kubelet probes).
+    """
+    return JSONResponse(content={"status": "ok"}, status_code=200)
+
+
+@router.get("/diagnostic/healthz/ready")
+async def readiness_probe() -> JSONResponse:
+    """Readiness probe for Kubernetes.
+
+    Verifies that the application can serve traffic by checking the database
+    and Redis.  Kubernetes uses this to decide whether to *route traffic* to
+    the pod.
+
+    Returns **200 OK** when all critical subsystems are reachable, or
+    **503 Service Unavailable** when the database is down.
+
+    **Authentication:** None (designed for kubelet probes).
+    """
+    checks: dict[str, dict[str, str]] = {}
+    db_ok = False
+
+    # ── Database check ─────────────────────────────────────────────────
+    try:
+        with engine.connect() as conn:
+            conn.execute(text("SELECT 1"))
+        checks["database"] = {"status": "ok"}
+        db_ok = True
+    except Exception as exc:
+        logger.warning("Readiness probe: database check failed: %s", exc)
+        checks["database"] = {"status": "error", "detail": str(exc)}
+
+    # ── Redis check ────────────────────────────────────────────────────
+    try:
+        redis_url = settings.redis_url or _DEFAULT_REDIS_URL
+        r = redis_lib.from_url(redis_url, socket_connect_timeout=2, socket_timeout=2)
+        r.ping()
+        checks["redis"] = {"status": "ok"}
+    except Exception as exc:
+        logger.warning("Readiness probe: Redis check failed: %s", exc)
+        checks["redis"] = {"status": "error", "detail": str(exc)}
+
+    http_status = 503 if not db_ok else 200
+    overall = "ready" if db_ok else "not_ready"
+    return JSONResponse(content={"status": overall, "checks": checks}, status_code=http_status)
+
 
 @router.get("/diagnostic/health")
 @require_login

app/api/dropbox.py

@@ -5,7 +5,9 @@ Dropbox API endpoints
 import logging
 import os
 from typing import Annotated, Optional
+from urllib.parse import quote
 
+import httpx
 import requests
 from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
 from sqlalchemy.orm import Session
@@ -23,6 +25,93 @@ logger = logging.getLogger(__name__)
 router = APIRouter()
 
 
+def _build_dropbox_redirect_uri(request: Request) -> str:
+    """Build the Dropbox OAuth callback redirect URI.
+
+    Uses ``PUBLIC_BASE_URL`` when configured (recommended for deployments behind
+    a reverse proxy that doesn't forward ``X-Forwarded-Proto``).  Falls back to
+    deriving the URI from the incoming request's scheme and host headers.
+    """
+    if settings.public_base_url:
+        return settings.public_base_url.rstrip("/") + "/dropbox-callback"
+    return f"{request.url.scheme}://{request.url.netloc}/dropbox-callback"
+
+
+@router.get("/dropbox/global-authorize-url")
+@require_login
+async def dropbox_global_authorize_url(request: Request):
+    """Return the Dropbox OAuth authorization URL using the global app credentials.
+
+    This endpoint is used when ``DROPBOX_ALLOW_GLOBAL_CREDENTIALS_FOR_INTEGRATIONS``
+    is enabled so that users can authorize their personal Dropbox integration without
+    needing to supply their own app key/secret.  Only the public ``app_key`` is
+    embedded in the URL; the ``app_secret`` is never sent to the browser.
+    """
+    if not settings.dropbox_allow_global_credentials_for_integrations:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Global credentials for integrations are not enabled",
+        )
+    if not settings.dropbox_app_key or not settings.dropbox_app_secret:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Global Dropbox credentials are not configured",
+        )
+    redirect_uri = _build_dropbox_redirect_uri(request)
+    authorize_url = (
+        "https://www.dropbox.com/oauth2/authorize"
+        f"?client_id={settings.dropbox_app_key}"
+        "&response_type=code"
+        "&token_access_type=offline"
+        f"&redirect_uri={quote(redirect_uri, safe='')}"
+    )
+    return {"authorize_url": authorize_url}
+
+
+@router.post("/dropbox/exchange-token-global")
+@require_login
+async def exchange_dropbox_token_global(
+    request: Request,
+    code: Annotated[str, Form(...)],
+    redirect_uri: Annotated[str, Form(...)],
+):
+    """Exchange an authorization code using the global Dropbox app credentials.
+
+    Used when ``DROPBOX_ALLOW_GLOBAL_CREDENTIALS_FOR_INTEGRATIONS`` is enabled so
+    that the ``app_secret`` is never exposed to the browser.  Only the OAuth code
+    and redirect URI need to be supplied by the client.
+    """
+    if not settings.dropbox_allow_global_credentials_for_integrations:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Global credentials for integrations are not enabled",
+        )
+    if not settings.dropbox_app_key or not settings.dropbox_app_secret:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Global Dropbox credentials are not configured",
+        )
+
+    token_url = "https://api.dropboxapi.com/oauth2/token"
+    payload = {
+        "client_id": settings.dropbox_app_key,
+        "client_secret": settings.dropbox_app_secret,
+        "code": code,
+        "redirect_uri": redirect_uri,
+        "grant_type": "authorization_code",
+    }
+
+    token_data = exchange_oauth_token(provider_name="Dropbox", token_url=token_url, payload=payload)
+
+    return {
+        "refresh_token": token_data["refresh_token"],
+        "access_token": token_data["access_token"],
+        "expires_in": token_data.get("expires_in", 14400),
+        # Return the public app_key so the callback can store it in the integration
+        "app_key": settings.dropbox_app_key,
+    }
+
+
 @router.post("/dropbox/exchange-token")
 @require_login
 async def exchange_dropbox_token(
@@ -132,57 +221,60 @@ async def test_dropbox_token(request: Request):
                 "message": "Dropbox credentials are not fully configured",
             }
 
-        # Check token validity by getting current account info
-        headers = {"Authorization": f"Bearer {settings.dropbox_refresh_token}"}
-        response = requests.post(
-            "https://api.dropboxapi.com/2/users/get_current_account",
-            headers=headers,
-            timeout=settings.http_request_timeout,
-        )
-
-        # If token is invalid, try refreshing it
-        if response.status_code == 401:
-            logger.info("Dropbox access token invalid or expired, trying to refresh")
-
-            # Get a new access token using the refresh token
-            refresh_url = "https://api.dropbox.com/oauth2/token"
-            refresh_data = {
-                "grant_type": "refresh_token",
-                "refresh_token": settings.dropbox_refresh_token,
-                "client_id": settings.dropbox_app_key,
-                "client_secret": settings.dropbox_app_secret,
-            }
-
-            refresh_response = requests.post(refresh_url, data=refresh_data, timeout=settings.http_request_timeout)
-
-            if refresh_response.status_code != 200:
-                logger.error(f"Failed to refresh Dropbox token: {refresh_response.text}")
-                return {
-                    "status": "error",
-                    "message": "Refresh token has expired or is invalid",
-                    "needs_reauth": True,
-                }
-
-            token_info = refresh_response.json()
-            access_token = token_info.get("access_token")
-
-            # Try again with the new access token
-            headers = {"Authorization": f"Bearer {access_token}"}
-            response = requests.post(
+        async with httpx.AsyncClient() as client:
+            # Check token validity by getting current account info
+            headers = {"Authorization": f"Bearer {settings.dropbox_refresh_token}"}
+            response = await client.post(
                 "https://api.dropboxapi.com/2/users/get_current_account",
                 headers=headers,
                 timeout=settings.http_request_timeout,
             )
 
-        if response.status_code != 200:
-            logger.error(f"Dropbox token test failed: {response.status_code} {response.text}")
-            return {
-                "status": "error",
-                "message": f"Token validation failed with status {response.status_code}: {response.text}",
-            }
+            # If token is invalid, try refreshing it
+            if response.status_code == 401:
+                logger.info("Dropbox access token invalid or expired, trying to refresh")
 
-        # Get account info
-        account_info = response.json()
+                # Get a new access token using the refresh token
+                refresh_url = "https://api.dropbox.com/oauth2/token"
+                refresh_data = {
+                    "grant_type": "refresh_token",
+                    "refresh_token": settings.dropbox_refresh_token,
+                    "client_id": settings.dropbox_app_key,
+                    "client_secret": settings.dropbox_app_secret,
+                }
+
+                refresh_response = await client.post(
+                    refresh_url, data=refresh_data, timeout=settings.http_request_timeout
+                )
+
+                if refresh_response.status_code != 200:
+                    logger.error(f"Failed to refresh Dropbox token: {refresh_response.text}")
+                    return {
+                        "status": "error",
+                        "message": "Refresh token has expired or is invalid",
+                        "needs_reauth": True,
+                    }
+
+                token_info = refresh_response.json()
+                access_token = token_info.get("access_token")
+
+                # Try again with the new access token
+                headers = {"Authorization": f"Bearer {access_token}"}
+                response = await client.post(
+                    "https://api.dropboxapi.com/2/users/get_current_account",
+                    headers=headers,
+                    timeout=settings.http_request_timeout,
+                )
+
+            if response.status_code != 200:
+                logger.error(f"Dropbox token test failed: {response.status_code} {response.text}")
+                return {
+                    "status": "error",
+                    "message": f"Token validation failed with status {response.status_code}: {response.text}",
+                }
+
+            # Get account info
+            account_info = response.json()
         account_email = account_info.get("email", "Unknown account")
         account_name = account_info.get("name", {}).get("display_name", "Unknown user")
 
@@ -207,6 +299,91 @@ async def test_dropbox_token(request: Request):
         return {"status": "error", "message": f"Connection error: {str(e)}"}
 
 
+@router.post("/dropbox/list-folders")
+@require_login
+async def list_dropbox_folders(
+    request: Request,
+    access_token: Annotated[str, Form(...)],
+    path: Annotated[str, Form()] = "",
+):
+    """
+    List folders in a Dropbox account for the directory selector.
+
+    Accepts an OAuth access token (short-lived) and a path to list.
+    Returns a flat list of folder entries under the given path.
+    """
+    try:
+        # Normalize path: Dropbox API uses "" for root, otherwise "/path"
+        folder_path = path.strip()
+        if folder_path == "/":
+            folder_path = ""
+        elif folder_path and not folder_path.startswith("/"):
+            folder_path = f"/{folder_path}"
+
+        headers = {
+            "Authorization": f"Bearer {access_token}",
+            "Content-Type": "application/json",
+        }
+
+        payload = {
+            "path": folder_path,
+            "recursive": False,
+            "include_deleted": False,
+            "include_has_explicit_shared_members": False,
+            "include_mounted_folders": True,
+        }
+
+        response = requests.post(
+            "https://api.dropboxapi.com/2/files/list_folder",
+            headers=headers,
+            json=payload,
+            timeout=settings.http_request_timeout,
+        )
+
+        if response.status_code == 401:
+            raise HTTPException(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                detail="Access token is invalid or expired. Please re-authorize.",
+            )
+
+        if response.status_code != 200:
+            logger.error(f"Dropbox list_folder failed: {response.status_code} {response.text}")
+            raise HTTPException(
+                status_code=status.HTTP_502_BAD_GATEWAY,
+                detail=f"Failed to list Dropbox folders: {response.text}",
+            )
+
+        data = response.json()
+        folders = []
+        for entry in data.get("entries", []):
+            if entry.get(".tag") == "folder":
+                folders.append(
+                    {
+                        "name": entry["name"],
+                        "path": entry["path_display"],
+                        "id": entry.get("id", ""),
+                    }
+                )
+
+        # Sort folders alphabetically
+        folders.sort(key=lambda f: f["name"].lower())
+
+        return {
+            "folders": folders,
+            "path": folder_path or "/",
+            "has_more": data.get("has_more", False),
+        }
+
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.exception(f"Error listing Dropbox folders: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"Failed to list folders: {str(e)}",
+        )
+
+
 @router.post("/dropbox/save-settings")
 @require_login
 async def save_dropbox_settings(

app/api/duplicates.py

@@ -73,33 +73,38 @@ def list_duplicate_groups(
     groups = []
     total_duplicate_files = 0
 
-    for filehash in dup_hashes:
-        # Find the original (non-duplicate) record with this hash
-        original = (
-            db.query(FileRecord)
-            .filter(FileRecord.filehash == filehash, FileRecord.is_duplicate.is_(False))
-            .order_by(FileRecord.id.asc())
-            .first()
+    if dup_hashes:
+        # Fetch all matching files (both original and duplicates) in a single batch query
+        all_records = (
+            db.query(FileRecord).filter(FileRecord.filehash.in_(dup_hashes)).order_by(FileRecord.id.asc()).all()
         )
 
-        # Find all duplicate records for this hash
-        duplicates = (
-            db.query(FileRecord)
-            .filter(FileRecord.filehash == filehash, FileRecord.is_duplicate.is_(True))
-            .order_by(FileRecord.id.asc())
-            .all()
-        )
+        # Group records by hash
+        originals_by_hash = {}
+        duplicates_by_hash = {h: [] for h in dup_hashes}
 
-        total_duplicate_files += len(duplicates)
+        for record in all_records:
+            h = record.filehash
+            if not record.is_duplicate:
+                # Store only the first original record per hash, matching the old .first() behaviour
+                if h not in originals_by_hash:
+                    originals_by_hash[h] = record
+            else:
+                duplicates_by_hash[h].append(record)
+                total_duplicate_files += 1
 
-        groups.append(
-            {
-                "filehash": filehash,
-                "original": _file_record_to_dict(original) if original else None,
-                "duplicates": [_file_record_to_dict(d) for d in duplicates],
-                "duplicate_count": len(duplicates),
-            }
-        )
+        for filehash in dup_hashes:
+            original = originals_by_hash.get(filehash)
+            duplicates = duplicates_by_hash.get(filehash, [])
+
+            groups.append(
+                {
+                    "filehash": filehash,
+                    "original": _file_record_to_dict(original) if original else None,
+                    "duplicates": [_file_record_to_dict(d) for d in duplicates],
+                    "duplicate_count": len(duplicates),
+                }
+            )
 
     total_pages = (total_groups + per_page - 1) // per_page if total_groups > 0 else 1
 

app/api/files.py

@@ -11,6 +11,7 @@ import zipfile
 from datetime import datetime, timezone
 from typing import Annotated, List, Optional
 
+import aiofiles
 from fastapi import APIRouter, Depends, File, HTTPException, Query, Request, UploadFile, status
 from fastapi.responses import StreamingResponse
 from sqlalchemy import asc, desc
@@ -19,6 +20,7 @@ from sqlalchemy.orm import Session
 from app.auth import require_login
 from app.config import settings
 from app.database import get_db
+from app.middleware.upload_rate_limit import require_upload_rate_limit
 from app.models import FileProcessingStep, FileRecord, ProcessingLog
 from app.tasks.convert_to_pdf import convert_to_pdf
 from app.tasks.process_document import process_document
@@ -28,7 +30,7 @@ from app.utils.file_queries import apply_status_filter
 from app.utils.file_status import get_files_processing_status
 from app.utils.filename_utils import sanitize_filename
 from app.utils.input_validation import validate_search_query, validate_sort_field, validate_sort_order
-from app.utils.user_scope import apply_owner_filter, get_current_owner_id
+from app.utils.user_scope import apply_owner_filter, get_current_owner_id, get_file_role
 
 # Set up logging
 logger = logging.getLogger(__name__)
@@ -298,6 +300,7 @@ def delete_file_record(request: Request, file_id: int, db: DbSession):
     """
     Delete a file record from the database.
     This only removes the database entry, not the actual file.
+    Only the file owner (or an admin) may delete a document.
     """
     # Check if file deletion is allowed
     if not settings.allow_file_delete:
@@ -312,6 +315,18 @@ def delete_file_record(request: Request, file_id: int, db: DbSession):
         if not file_record:
             raise HTTPException(status_code=404, detail=f"File record with ID {file_id} not found")
 
+        # Enforce owner-only deletion in multi-user mode
+        user = request.session.get("user")
+        is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+        if not is_admin:
+            owner_id = get_current_owner_id(request)
+            role = get_file_role(file_record, owner_id, db)
+            if role != "owner":
+                raise HTTPException(
+                    status_code=403,
+                    detail="Only the file owner can delete this document",
+                )
+
         # Log the deletion
         logger.info(f"Deleting file record: ID={file_id}, Filename={file_record.original_filename}")
 
@@ -338,6 +353,7 @@ def bulk_delete_files(request: Request, file_ids: List[int], db: DbSession):
     """
     Delete multiple file records from the database.
     This only removes the database entries, not the actual files.
+    Only the file owner (or an admin) may delete each document.
     """
     # Check if file deletion is allowed
     if not settings.allow_file_delete:
@@ -345,11 +361,25 @@ def bulk_delete_files(request: Request, file_ids: List[int], db: DbSession):
 
     try:
         # Find all file records
-        file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
+        query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
+        query = apply_owner_filter(query, request)
+        file_records = query.all()
 
         if not file_records:
             raise HTTPException(status_code=404, detail="No files found with the provided IDs")
 
+        # Enforce owner-only deletion in multi-user mode
+        user = request.session.get("user")
+        is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+        if not is_admin:
+            owner_id = get_current_owner_id(request)
+            non_owner_ids = [f.id for f in file_records if get_file_role(f, owner_id, db) != "owner"]
+            if non_owner_ids:
+                raise HTTPException(
+                    status_code=403,
+                    detail=f"You can only delete files you own. Not owner of file IDs: {non_owner_ids}",
+                )
+
         deleted_count = len(file_records)
         deleted_ids = [f.id for f in file_records]
 
@@ -384,7 +414,9 @@ def bulk_reprocess_files(request: Request, file_ids: List[int], db: DbSession):
     """
     try:
         # Find all file records
-        file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
+        query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
+        query = apply_owner_filter(query, request)
+        file_records = query.all()
 
         if not file_records:
             raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -456,7 +488,9 @@ def bulk_reprocess_files_cloud_ocr(request: Request, file_ids: List[int], db: Db
     Useful for re-running OCR on files with poor text quality or missing OCR text.
     """
     try:
-        file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
+        query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
+        query = apply_owner_filter(query, request)
+        file_records = query.all()
 
         if not file_records:
             raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -536,7 +570,9 @@ def bulk_download_files(request: Request, file_ids: List[int], db: DbSession):
     Files not found on disk are silently skipped.
     """
     try:
-        file_records = db.query(FileRecord).filter(FileRecord.id.in_(file_ids)).all()
+        query = db.query(FileRecord).filter(FileRecord.id.in_(file_ids))
+        query = apply_owner_filter(query, request)
+        file_records = query.all()
 
         if not file_records:
             raise HTTPException(status_code=404, detail="No files found with the provided IDs")
@@ -618,7 +654,9 @@ def reprocess_single_file(request: Request, file_id: int, db: DbSession):
     """
     try:
         # Find the file record
-        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+        query = db.query(FileRecord).filter(FileRecord.id == file_id)
+        query = apply_owner_filter(query, request)
+        file_record = query.first()
 
         if not file_record:
             raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -674,7 +712,9 @@ def reprocess_with_cloud_ocr(request: Request, file_id: int, db: DbSession):
     """
     try:
         # Find the file record
-        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+        query = db.query(FileRecord).filter(FileRecord.id == file_id)
+        query = apply_owner_filter(query, request)
+        file_record = query.first()
 
         if not file_record:
             raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -937,7 +977,9 @@ def retry_subtask(
     """
     try:
         # Find the file record
-        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+        query = db.query(FileRecord).filter(FileRecord.id == file_id)
+        query = apply_owner_filter(query, request)
+        file_record = query.first()
 
         if not file_record:
             raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -1079,7 +1121,9 @@ def get_file_preview(
 
     try:
         # Find the file record
-        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+        query = db.query(FileRecord).filter(FileRecord.id == file_id)
+        query = apply_owner_filter(query, request)
+        file_record = query.first()
 
         if not file_record:
             raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -1159,7 +1203,9 @@ def download_file(
 
     try:
         # Find the file record
-        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+        query = db.query(FileRecord).filter(FileRecord.id == file_id)
+        query = apply_owner_filter(query, request)
+        file_record = query.first()
 
         if not file_record:
             raise HTTPException(status_code=404, detail=f"File with ID {file_id} not found")
@@ -1217,9 +1263,79 @@ def download_file(
         raise HTTPException(status_code=500, detail=f"Error downloading file: {str(e)}")
 
 
+async def _save_upload_file_chunks(file: UploadFile, target_path: str, max_size: int) -> int:
+    """Save an uploaded file in chunks and enforce the maximum size limit."""
+    try:
+        written_size = 0
+        with open(target_path, "wb") as f:
+            chunk_size = 65536  # 64 KB chunks
+            while True:
+                chunk = await file.read(chunk_size)
+                if not chunk:
+                    break
+                written_size += len(chunk)
+                if written_size > max_size:
+                    # Exceeded limit mid-stream; clean up and reject
+                    f.close()
+                    os.remove(target_path)
+                    raise HTTPException(
+                        status_code=413,
+                        detail=f"File too large: exceeded {max_size} bytes during upload. "
+                        f"See SECURITY_AUDIT.md for configuration details.",
+                    )
+                f.write(chunk)
+        return written_size
+    except HTTPException:
+        raise
+    except Exception as e:
+        if os.path.exists(target_path):
+            os.remove(target_path)
+        raise HTTPException(status_code=500, detail=f"Failed to save file: {e}")
+
+
+def _check_for_exact_duplicate(db: DbSession, target_path: str, safe_filename: str) -> dict | None:
+    """Check for an exact duplicate of the uploaded file.
+
+    Returns a dict with duplicate info when the file's SHA-256 hash matches an
+    already-processed document, or ``None`` when no duplicate is found (or
+    deduplication is disabled).
+    """
+    if not settings.enable_deduplication:
+        return None
+
+    try:
+        filehash = hash_file(target_path)
+        existing = (
+            db.query(FileRecord)
+            .filter(FileRecord.filehash == filehash, FileRecord.is_duplicate.is_(False))
+            .order_by(FileRecord.id.asc())
+            .first()
+        )
+        if existing:
+            logger.info(f"Exact duplicate detected on upload: '{safe_filename}' matches file ID {existing.id}")
+            return {
+                "duplicate_type": "exact",
+                "original_file_id": existing.id,
+                "original_filename": existing.original_filename,
+                "message": (
+                    "This file is an exact duplicate of an already-processed document. "
+                    "It has not been queued for processing again."
+                ),
+            }
+    except Exception as e:
+        logger.warning(f"Duplicate check failed for uploaded file '{safe_filename}': {e}")
+
+    return None
+
+
 @router.post("/ui-upload")
 @require_login
-async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...)):
+async def ui_upload(
+    request: Request,
+    db: DbSession,
+    file: UploadFile = File(...),
+    _rate_ok: None = Depends(require_upload_rate_limit),
+):
     """Endpoint to accept a user-uploaded file and enqueue it for processing."""
     workdir = settings.workdir
 
@@ -1277,7 +1393,7 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
     # enforcing the size limit during the read so memory usage stays bounded.
     try:
         written_size = 0
-        with open(target_path, "wb") as f:
+        async with aiofiles.open(target_path, "wb") as f:
             chunk_size = 65536  # 64 KB chunks
             while True:
                 chunk = await file.read(chunk_size)
@@ -1286,14 +1402,14 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
                 written_size += len(chunk)
                 if written_size > max_size:
                     # Exceeded limit mid-stream; clean up and reject
-                    f.close()
+                    await f.close()
                     os.remove(target_path)
                     raise HTTPException(
                         status_code=413,
                         detail=f"File too large: exceeded {max_size} bytes during upload. "
                         f"See SECURITY_AUDIT.md for configuration details.",
                     )
-                f.write(chunk)
+                await f.write(chunk)
     except HTTPException:
         raise
     except Exception as e:
@@ -1305,6 +1421,25 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
     logger.info(f"Saved uploaded file '{safe_filename}' as '{target_filename}'")
     file_size = written_size
 
+    # ── Early duplicate rejection ──────────────────────────────────────────
+    # Check for exact duplicates (same SHA-256 hash) BEFORE enqueuing a
+    # processing task.  When deduplication is enabled and the file already
+    # exists, we skip processing entirely, clean up the temp file, and
+    # return the existing file's information to the caller.
+    exact_duplicate = _check_for_exact_duplicate(db, target_path, safe_filename)
+    if exact_duplicate:
+        # Remove the just-saved temp file — it's a duplicate.
+        try:
+            os.remove(target_path)
+        except OSError:
+            pass
+        return {
+            "status": "duplicate",
+            "original_filename": safe_filename,
+            "stored_filename": target_filename,
+            "duplicate_of": exact_duplicate,
+        }
+
     # Determine if the file is a PDF or needs conversion
     mime_type, _ = mimetypes.guess_type(target_path)
     file_ext = os.path.splitext(target_path)[1].lower()
@@ -1368,6 +1503,8 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
         ".tif",
         ".webp",
         ".svg",
+        ".heic",
+        ".heif",
     }:
         # If it's an image, convert to PDF first
         task = convert_to_pdf.delay(target_path, original_filename=safe_filename, owner_id=upload_owner_id)
@@ -1381,42 +1518,12 @@ async def ui_upload(request: Request, db: DbSession, file: UploadFile = File(...
         logger.warning(f"Unsupported MIME type {mime_type} for {target_path}, attempting conversion")
         task = convert_to_pdf.delay(target_path, original_filename=safe_filename, owner_id=upload_owner_id)
 
-    # Check for exact duplicates (same SHA-256 hash) before returning.
-    # This gives the caller an immediate warning without waiting for the pipeline.
-    # Only performed when deduplication is enabled in settings.
-    exact_duplicate_warning = None
-    if settings.enable_deduplication:
-        try:
-            filehash = hash_file(target_path)
-            existing = (
-                db.query(FileRecord)
-                .filter(FileRecord.filehash == filehash, FileRecord.is_duplicate.is_(False))
-                .order_by(FileRecord.id.asc())
-                .first()
-            )
-            if existing:
-                exact_duplicate_warning = {
-                    "duplicate_type": "exact",
-                    "original_file_id": existing.id,
-                    "original_filename": existing.original_filename,
-                    "message": (
-                        "This file appears to be an exact duplicate of an already-processed document. "
-                        "It will still be queued but will be flagged as a duplicate."
-                    ),
-                }
-                logger.info(f"Exact duplicate detected on upload: '{safe_filename}' matches file ID {existing.id}")
-        except Exception as e:
-            logger.warning(f"Duplicate check failed for uploaded file '{safe_filename}': {e}")
-
-    response: dict = {
+    return {
         "task_id": task.id,
         "status": "queued",
         "original_filename": safe_filename,
         "stored_filename": target_filename,
     }
-    if exact_duplicate_warning:
-        response["duplicate_warning"] = exact_duplicate_warning
-    return response
 
 
 # ---------------------------------------------------------------------------
@@ -1458,7 +1565,7 @@ def claim_file(request: Request, file_id: int, db: DbSession):
         logger.exception(f"Error claiming file {file_id}: {e}")
         raise HTTPException(status_code=500, detail="Failed to claim document")
 
-    logger.info(f"File {file_id} claimed by user '{owner_id}'")
+    logger.info("File %d claimed by user", file_id)
     return {"status": "success", "message": "Document claimed successfully", "file_id": file_id, "owner_id": owner_id}
 
 
@@ -1498,7 +1605,7 @@ def bulk_claim_files(request: Request, file_ids: list[int], db: DbSession):
         logger.exception(f"Error during bulk claim: {e}")
         raise HTTPException(status_code=500, detail="Failed to claim documents")
 
-    logger.info(f"Bulk claim by '{owner_id}': claimed={claimed}, skipped={[s['file_id'] for s in skipped]}")
+    logger.info("Bulk claim: claimed=%s, skipped=%s", claimed, [s["file_id"] for s in skipped])
     return {
         "status": "success",
         "claimed_count": len(claimed),
@@ -1551,8 +1658,7 @@ def assign_owner(request: Request, db: DbSession, owner_id: str = Query(...), fi
         logger.exception(f"Error assigning owner: {e}")
         raise HTTPException(status_code=500, detail="Failed to assign owner")
 
-    admin_name = get_current_owner_id(request) or "admin"
-    logger.info(f"Admin '{admin_name}' assigned owner_id='{owner_id}' to {updated} file(s)")
+    logger.info("Admin assigned owner to %d file(s)", updated)
     return {
         "status": "success",
         "message": f"Assigned owner to {updated} document(s)",

app/api/google_drive.py

@@ -363,7 +363,7 @@ def format_time_remaining(time_delta):
 
 @router.post("/google-drive/save-settings")
 @require_login
-async def save_dropbox_settings(
+async def save_google_drive_settings(
     request: Request,
     refresh_token: Annotated[str, Form(...)],
     client_id: Annotated[Optional[str], Form()] = None,

app/api/integrations.py

@@ -32,6 +32,21 @@ from app.utils.encryption import decrypt_value, encrypt_value
 from app.utils.subscription import get_tier, get_user_tier_id
 from app.utils.user_scope import get_current_owner_id
 
+# Optional Dropbox SDK — imported at module level so tests can patch it cleanly.
+try:
+    import dropbox as dbx_lib
+    from dropbox.exceptions import AuthError as _DropboxAuthError
+    from dropbox.exceptions import BadInputError as _DropboxBadInputError
+except ImportError:  # pragma: no cover
+    dbx_lib = None  # type: ignore[assignment]
+
+    class _DropboxAuthError(Exception):  # type: ignore[no-redef]
+        """Stub — only used when the dropbox package is missing."""
+
+    class _DropboxBadInputError(Exception):  # type: ignore[no-redef]
+        """Stub — only used when the dropbox package is missing."""
+
+
 logger = logging.getLogger(__name__)
 router = APIRouter(prefix="/integrations", tags=["integrations"])
 
@@ -455,31 +470,6 @@ def delete_integration(
     logger.info("User %s deleted integration %d", owner_id, integration_id)
 
 
-@router.get("/{integration_id}/credentials", summary="Retrieve decrypted credentials for an integration")
-def get_integration_credentials(
-    integration_id: int,
-    request: Request,
-    db: DbSession,
-    owner_id: CurrentOwner,
-) -> dict[str, Any]:
-    """Return the decrypted credentials dict for a saved integration.
-
-    This endpoint is intended for internal use by background tasks that need
-    to authenticate with a third-party service.  Treat the response as
-    sensitive — it contains plaintext secrets.
-    """
-    integration = (
-        db.query(UserIntegration)
-        .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
-        .first()
-    )
-    if not integration:
-        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Integration not found")
-
-    credentials = _decode_credentials(integration.credentials)
-    return {"credentials": credentials or {}}
-
-
 # ---------------------------------------------------------------------------
 # Connection test helpers
 # ---------------------------------------------------------------------------
@@ -550,9 +540,50 @@ def _test_s3_connection(config: dict[str, Any] | None, credentials: dict[str, An
         return {"success": False, "message": "S3 connection failed"}
 
 
+def _test_dropbox_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
+    """Test a Dropbox connection by verifying OAuth credentials via the Dropbox API."""
+    if dbx_lib is None:
+        return {"success": False, "message": "dropbox package is not installed"}  # pragma: no cover
+
+    creds = credentials or {}
+    app_key = creds.get("app_key", "")
+    app_secret = creds.get("app_secret", "")
+    refresh_token = creds.get("refresh_token", "")
+
+    if not refresh_token:
+        return {"success": False, "message": "Missing required credential: refresh_token"}
+    if not app_key or not app_secret:
+        return {"success": False, "message": "Missing required credentials: app_key and app_secret"}
+
+    try:
+        dbx = dbx_lib.Dropbox(
+            app_key=app_key,
+            app_secret=app_secret,
+            oauth2_refresh_token=refresh_token,
+        )
+        account = dbx.users_get_current_account()
+        display_name = getattr(account, "name", None)
+        name_str = ""
+        if display_name:
+            name_str = f" ({getattr(display_name, 'display_name', '') or ''})"
+        return {"success": True, "message": f"Dropbox connection successful{name_str}"}
+    except _DropboxAuthError as exc:
+        logger.warning("Dropbox auth error: %s", exc)
+        return {
+            "success": False,
+            "message": "Dropbox authentication failed — check app_key, app_secret, and refresh_token",
+        }
+    except _DropboxBadInputError as exc:
+        logger.warning("Dropbox bad input error: %s", exc)
+        return {"success": False, "message": "Dropbox connection failed — invalid credentials format"}
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("Dropbox connection error: %s", exc)
+        return {"success": False, "message": "Dropbox connection failed — check credentials and network connectivity"}
+
+
 def _test_webdav_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
     """Test a WebDAV/Nextcloud connection by issuing an HTTP PROPFIND."""
-    import urllib.request
+    import httpx
 
     cfg = config or {}
     creds = credentials or {}
@@ -579,23 +610,21 @@ def _test_webdav_connection(config: dict[str, Any] | None, credentials: dict[str
             return {"success": False, "message": "URLs pointing to internal or private networks are not allowed"}
 
     try:
-        import base64
+        auth = (username, password) if username and password else None
+        headers = {"Depth": "0"}
 
-        req = urllib.request.Request(url, method="PROPFIND")  # noqa: S310
-        if username and password:
-            token = base64.b64encode(f"{username}:{password}".encode()).decode()
-            req.add_header("Authorization", f"Basic {token}")
-        req.add_header("Depth", "0")
-        with urllib.request.urlopen(req, timeout=10) as resp:  # noqa: S310
-            if resp.status < 400:
-                return {"success": True, "message": "WebDAV connection successful"}
-            return {"success": False, "message": f"WebDAV returned HTTP {resp.status}"}
+        # Use httpx for secure connection testing, avoiding urllib vulnerabilities
+        resp = httpx.request("PROPFIND", url, auth=auth, headers=headers, timeout=10.0, follow_redirects=False)
+        if resp.status_code < 400:
+            return {"success": True, "message": "WebDAV connection successful"}
+        return {"success": False, "message": f"WebDAV returned HTTP {resp.status_code}"}
     except Exception as exc:  # noqa: BLE001
         logger.warning("WebDAV connection error for %s: %s", hostname, exc)
         return {"success": False, "message": "WebDAV connection failed — check URL and credentials"}
 
 
 _CONNECTION_TESTERS: dict[str, Any] = {
+    IntegrationType.DROPBOX: _test_dropbox_connection,
     IntegrationType.IMAP: _test_imap_connection,
     IntegrationType.S3: _test_s3_connection,
     IntegrationType.WEBDAV: _test_webdav_connection,
@@ -608,6 +637,62 @@ _CONNECTION_TESTERS: dict[str, Any] = {
 # ---------------------------------------------------------------------------
 
 
+@router.post("/{integration_id}/test", summary="Test a saved integration connection")
+def test_saved_integration_connection(
+    integration_id: int,
+    request: Request,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Test connection for an already-saved integration."""
+    integration = (
+        db.query(UserIntegration)
+        .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+        .first()
+    )
+    if not integration:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Integration not found")
+
+    tester = _CONNECTION_TESTERS.get(integration.integration_type)
+    if tester is None:
+        return {
+            "success": False,
+            "message": f"Connection testing is not yet supported for '{integration.integration_type}'. "
+            "The integration can still be saved and will be validated on first use.",
+        }
+
+    try:
+        config = json.loads(integration.config) if integration.config else {}
+    except (json.JSONDecodeError, TypeError):
+        logger.warning(
+            "Invalid JSON in integration.config for integration_id=%s, type=%s",
+            integration.id,
+            integration.integration_type,
+        )
+        return {
+            "success": False,
+            "message": (
+                "Saved integration configuration is invalid and cannot be tested. "
+                "Please edit and re-save the integration, then try again."
+            ),
+        }
+
+    credentials = _decode_credentials(integration.credentials) or {}
+
+    try:
+        return tester(config, credentials)
+    except Exception:
+        logger.exception(
+            "Unexpected error testing integration_id=%s, type=%s",
+            integration.id,
+            integration.integration_type,
+        )
+        return {
+            "success": False,
+            "message": "An unexpected error occurred while testing the connection. Please check your configuration.",
+        }
+
+
 @router.post("/test", summary="Test an integration connection without saving")
 def test_integration_connection(
     request: Request,

app/api/local_auth.py

@@ -101,9 +101,9 @@ async def signup_page(request: Request) -> Any:
     if not settings.allow_local_signup:
         return RedirectResponse(url="/login?error=Registration+is+not+enabled", status_code=302)
     return templates.TemplateResponse(
+        request,
         "signup.html",
-        {
-            "request": request,
+        context={
             "csrf_token": getattr(request.state, "csrf_token", ""),
             "app_version": settings.version,
         },
@@ -113,16 +113,16 @@ async def signup_page(request: Request) -> Any:
 @router.get("/verify-email-sent", include_in_schema=False)
 async def verify_email_sent_page(request: Request) -> Any:
     """Render the verify-email-sent confirmation page."""
-    return templates.TemplateResponse("verify_email_sent.html", {"request": request})
+    return templates.TemplateResponse(request, "verify_email_sent.html")
 
 
 @router.get("/forgot-username", include_in_schema=False)
 async def forgot_username_page(request: Request) -> Any:
     """Render the forgot-username page where users can request a username reminder email."""
     return templates.TemplateResponse(
+        request,
         "forgot_username.html",
-        {
-            "request": request,
+        context={
             "csrf_token": getattr(request.state, "csrf_token", ""),
             "app_version": settings.version,
         },
@@ -133,9 +133,9 @@ async def forgot_username_page(request: Request) -> Any:
 async def forgot_password_page(request: Request) -> Any:
     """Render the forgot-password page where users can request a reset email."""
     return templates.TemplateResponse(
+        request,
         "forgot_password.html",
-        {
-            "request": request,
+        context={
             "csrf_token": getattr(request.state, "csrf_token", ""),
             "app_version": settings.version,
         },
@@ -147,9 +147,9 @@ async def reset_password_page(request: Request) -> Any:
     """Render the password reset form page."""
     token = request.query_params.get("token", "")
     return templates.TemplateResponse(
+        request,
         "password_reset_form.html",
-        {
-            "request": request,
+        context={
             "token": token,
             "csrf_token": getattr(request.state, "csrf_token", ""),
             "app_version": settings.version,

app/api/mobile.py

@@ -120,6 +120,7 @@ class WhoAmIResponse(BaseModel):
     email: str | None
     avatar_url: str | None
     is_admin: bool
+    preferred_language: str | None
 
 
 # ---------------------------------------------------------------------------
@@ -273,31 +274,44 @@ async def list_devices(
     return [_device_to_response(d) for d in devices]
 
 
-@router.delete("/devices/{device_id}", status_code=status.HTTP_204_NO_CONTENT)
+@router.delete("/devices/{device_id}", status_code=status.HTTP_200_OK)
 @require_login
 async def deactivate_device(
     request: Request,
     device_id: int,
     owner_id: CurrentOwner,
     db: DbSession,
-) -> None:
-    """Deactivate a push-notification device registration.
+) -> dict[str, str]:
+    """Deactivate or permanently delete a push-notification device registration.
 
-    The device record is kept for audit purposes but will no longer receive
-    push notifications.
+    * **Active device** – soft-deactivated: the record is kept for audit
+      purposes but will no longer receive push notifications.
+    * **Already-inactive device** – hard-deleted: the record is permanently
+      removed from the database.
     """
     device = db.get(MobileDevice, device_id)
     if not device or device.owner_id != owner_id:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Device not found")
 
-    device.is_active = False
+    if device.is_active:
+        device.is_active = False
+        try:
+            db.commit()
+        except Exception:
+            db.rollback()
+            raise
+        logger.info("Mobile device deactivated: id=%s owner=%s", device_id, owner_id)
+        return {"detail": "Device deactivated"}
+
+    # Hard-delete an already-inactive device.
     try:
+        db.delete(device)
         db.commit()
     except Exception:
         db.rollback()
         raise
-
-    logger.info("Mobile device deactivated: id=%s owner=%s", device_id, owner_id)
+    logger.info("Mobile device permanently deleted: id=%s owner=%s", device_id, owner_id)
+    return {"detail": "Device deleted"}
 
 
 @router.get("/whoami", response_model=WhoAmIResponse)
@@ -344,4 +358,5 @@ async def whoami(
         "email": email,
         "avatar_url": avatar_url,
         "is_admin": is_admin,
+        "preferred_language": profile.preferred_language if profile else None,
     }

app/api/notifications.py

@@ -452,17 +452,16 @@ async def update_preferences(
             )
 
     try:
+        # Pre-fetch existing preferences for this user to avoid N+1 queries
+        existing_prefs = (
+            db.query(UserNotificationPreference).filter(UserNotificationPreference.owner_id == owner_id).all()
+        )
+
+        # Build a fast lookup dictionary keyed by (event_type, channel_type, target_id)
+        prefs_dict = {(pref.event_type, pref.channel_type, pref.target_id): pref for pref in existing_prefs}
+
         for item in body.preferences:
-            existing = (
-                db.query(UserNotificationPreference)
-                .filter(
-                    UserNotificationPreference.owner_id == owner_id,
-                    UserNotificationPreference.event_type == item.event_type,
-                    UserNotificationPreference.channel_type == item.channel_type,
-                    UserNotificationPreference.target_id == item.target_id,
-                )
-                .first()
-            )
+            existing = prefs_dict.get((item.event_type, item.channel_type, item.target_id))
             if existing:
                 existing.is_enabled = item.is_enabled
             else:

app/api/onedrive.py

@@ -3,10 +3,10 @@ OneDrive API endpoints
 """
 
 import logging
-import os
 from datetime import datetime, timedelta
 from typing import Annotated, Optional
 
+import httpx
 import requests
 from fastapi import APIRouter, Depends, Form, HTTPException, Request, status
 from sqlalchemy.orm import Session
@@ -14,6 +14,7 @@ from sqlalchemy.orm import Session
 from app.auth import require_login
 from app.config import settings
 from app.database import get_db
+from app.utils.env_utils import update_env_file
 from app.utils.oauth_helper import exchange_oauth_token
 from app.utils.settings_service import save_setting_to_db
 from app.utils.settings_sync import notify_settings_updated
@@ -56,6 +57,7 @@ async def exchange_onedrive_token(
     # Return just what's needed by the frontend
     return {
         "refresh_token": token_data["refresh_token"],
+        "access_token": token_data.get("access_token", ""),
         "expires_in": token_data.get("expires_in", 3600),
     }
 
@@ -92,17 +94,18 @@ async def test_onedrive_token(request: Request):
             "scope": "offline_access Files.ReadWrite",
         }
 
-        response = requests.post(token_url, data=refresh_data, timeout=settings.http_request_timeout)
+        async with httpx.AsyncClient(timeout=settings.http_request_timeout) as client:
+            response = await client.post(token_url, data=refresh_data)
 
-        if response.status_code != 200:
-            logger.error(f"Failed to refresh OneDrive token: {response.text}")
-            return {
-                "status": "error",
-                "message": "Refresh token has expired or is invalid",
-                "needs_reauth": True,
-            }
+            if response.status_code != 200:
+                logger.error(f"Failed to refresh OneDrive token: {response.text}")
+                return {
+                    "status": "error",
+                    "message": "Refresh token has expired or is invalid",
+                    "needs_reauth": True,
+                }
 
-        token_data = response.json()
+            token_data = response.json()
         access_token = token_data.get("access_token")
         expires_in = token_data.get("expires_in", 3600)  # Default to 1 hour if not specified
 
@@ -115,32 +118,7 @@ async def test_onedrive_token(request: Request):
             settings.onedrive_refresh_token = new_refresh_token
 
             # Also try to update .env file if it exists
-            try:
-                env_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), ".env")
-                if os.path.exists(env_path):
-                    with open(env_path, "r") as f:
-                        env_lines = f.readlines()
-
-                    updated_lines = []
-                    updated = False
-
-                    for line in env_lines:
-                        if line.startswith("ONEDRIVE_REFRESH_TOKEN="):
-                            updated_lines.append(f"ONEDRIVE_REFRESH_TOKEN={new_refresh_token}\n")
-                            updated = True
-                        else:
-                            updated_lines.append(line)
-
-                    if not updated:
-                        updated_lines.append(f"ONEDRIVE_REFRESH_TOKEN={new_refresh_token}\n")
-
-                    with open(env_path, "w") as f:
-                        f.writelines(updated_lines)
-
-                    logger.info("Updated refresh token in .env file")
-
-            except Exception as e:
-                logger.warning(f"Failed to update refresh token in .env file: {e}")
+            update_env_file({"ONEDRIVE_REFRESH_TOKEN": new_refresh_token})
 
             # Persist the rotated refresh token to the database
             try:
@@ -164,17 +142,18 @@ async def test_onedrive_token(request: Request):
         user_info_url = "https://graph.microsoft.com/v1.0/me"
         headers = {"Authorization": f"Bearer {access_token}"}
 
-        user_response = requests.get(user_info_url, headers=headers, timeout=settings.http_request_timeout)
+        async with httpx.AsyncClient(timeout=settings.http_request_timeout) as client:
+            user_response = await client.get(user_info_url, headers=headers)
 
-        if user_response.status_code != 200:
-            logger.error(f"OneDrive token test failed: {user_response.status_code} {user_response.text}")
-            return {
-                "status": "error",
-                "message": f"Token validation failed with status {user_response.status_code}: {user_response.text}",
-            }
+            if user_response.status_code != 200:
+                logger.error(f"OneDrive token test failed: {user_response.status_code} {user_response.text}")
+                return {
+                    "status": "error",
+                    "message": f"Token validation failed with status {user_response.status_code}: {user_response.text}",
+                }
 
-        # Get user info
-        user_info = user_response.json()
+            # Get user info
+            user_info = user_response.json()
         display_name = user_info.get("displayName", "Unknown user")
         email = user_info.get("userPrincipalName", "Unknown email")
 
@@ -206,6 +185,102 @@ async def test_onedrive_token(request: Request):
         return {"status": "error", "message": f"Connection error: {str(e)}"}
 
 
+@router.post("/onedrive/list-folders")
+@require_login
+async def list_onedrive_folders(
+    request: Request,
+    access_token: Annotated[str, Form(...)],
+    path: Annotated[str, Form()] = "",
+):
+    """
+    List folders in a OneDrive account for the directory selector.
+
+    Accepts an OAuth access token (short-lived) and a path to list.
+    Returns a flat list of folder entries under the given path.
+    """
+    try:
+        folder_path = path.strip().strip("/")
+
+        headers = {
+            "Authorization": f"Bearer {access_token}",
+        }
+
+        # Build the Graph API URL for listing children
+        if not folder_path or folder_path == "root":
+            url = "https://graph.microsoft.com/v1.0/me/drive/root/children"
+        else:
+            url = f"https://graph.microsoft.com/v1.0/me/drive/root:/{folder_path}:/children"
+
+        # Only request folders and minimal fields
+        params = {
+            "$filter": "folder ne null",
+            "$select": "name,id,parentReference,folder",
+            "$top": "200",
+        }
+
+        response = requests.get(
+            url,
+            headers=headers,
+            params=params,
+            timeout=settings.http_request_timeout,
+        )
+
+        if response.status_code == 401:
+            raise HTTPException(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                detail="Access token is invalid or expired. Please re-authorize.",
+            )
+
+        if response.status_code != 200:
+            logger.error(f"OneDrive list children failed: {response.status_code} {response.text}")
+            raise HTTPException(
+                status_code=status.HTTP_502_BAD_GATEWAY,
+                detail=f"Failed to list OneDrive folders: {response.text}",
+            )
+
+        data = response.json()
+        folders = []
+        for item in data.get("value", []):
+            if "folder" in item:
+                parent_path = ""
+                if item.get("parentReference", {}).get("path"):
+                    # parentReference.path looks like /drive/root:/some/path
+                    raw_parent = item["parentReference"]["path"]
+                    prefix = "/drive/root:"
+                    if raw_parent.startswith(prefix):
+                        parent_path = raw_parent[len(prefix) :]
+                    elif raw_parent == "/drive/root":
+                        parent_path = ""
+
+                item_path = f"{parent_path}/{item['name']}" if parent_path else f"/{item['name']}"
+
+                folders.append(
+                    {
+                        "name": item["name"],
+                        "path": item_path,
+                        "id": item.get("id", ""),
+                        "child_count": item.get("folder", {}).get("childCount", 0),
+                    }
+                )
+
+        # Sort folders alphabetically
+        folders.sort(key=lambda f: f["name"].lower())
+
+        return {
+            "folders": folders,
+            "path": f"/{folder_path}" if folder_path else "/",
+        }
+
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.exception(f"Error listing OneDrive folders: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"Failed to list folders: {str(e)}",
+        )
+
+
 def format_time_remaining(time_delta):
     """Format a timedelta into a human-readable string."""
     if time_delta.total_seconds() <= 0:
@@ -246,75 +321,26 @@ async def save_onedrive_settings(
             user.get("preferred_username") or user.get("username") or user.get("email") or user.get("id") or "wizard"
         )
 
-        # Best-effort .env file write
-        try:
-            env_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), ".env")
-            if not os.path.exists(env_path):
-                logger.warning(f".env file not found at {env_path}, skipping file write")
-            else:
-                logger.info(f"Updating OneDrive settings in {env_path}")
+        # Build settings dictionary mapped to database/memory keys
+        onedrive_settings = {
+            "onedrive_refresh_token": refresh_token,
+            "onedrive_client_id": client_id,
+            "onedrive_client_secret": client_secret,
+            "onedrive_tenant_id": tenant_id,
+            "onedrive_folder_path": folder_path,
+        }
 
-                with open(env_path, "r") as f:
-                    env_lines = f.readlines()
+        # Filter out None values
+        onedrive_settings = {k: v for k, v in onedrive_settings.items() if v is not None}
 
-                onedrive_settings = {"ONEDRIVE_REFRESH_TOKEN": refresh_token}
-                if client_id:
-                    onedrive_settings["ONEDRIVE_CLIENT_ID"] = client_id
-                if client_secret:
-                    onedrive_settings["ONEDRIVE_CLIENT_SECRET"] = client_secret
-                if tenant_id:
-                    onedrive_settings["ONEDRIVE_TENANT_ID"] = tenant_id
-                if folder_path:
-                    onedrive_settings["ONEDRIVE_FOLDER_PATH"] = folder_path
+        # Best-effort .env file write using the new utility
+        env_settings = {k.upper(): v for k, v in onedrive_settings.items()}
+        update_env_file(env_settings)
 
-                updated = set()
-                new_env_lines = []
-                for line in env_lines:
-                    stripped_line = line.rstrip()
-                    is_updated = False
-                    for key, value in onedrive_settings.items():
-                        if stripped_line.startswith(f"{key}=") or stripped_line.startswith(f"# {key}="):
-                            new_env_lines.append(f"{key}={value}")
-                            updated.add(key)
-                            is_updated = True
-                            break
-                    if not is_updated:
-                        new_env_lines.append(stripped_line)
-
-                for key, value in onedrive_settings.items():
-                    if key not in updated:
-                        new_env_lines.append(f"{key}={value}")
-
-                with open(env_path, "w") as f:
-                    f.write("\n".join(new_env_lines) + "\n")
-
-                logger.info("Successfully updated OneDrive settings in .env file")
-        except Exception as env_err:
-            logger.warning(f"Failed to write .env file (non-fatal): {env_err}")
-
-        # Update the settings in memory
-        if refresh_token:
-            settings.onedrive_refresh_token = refresh_token
-        if client_id:
-            settings.onedrive_client_id = client_id
-        if client_secret:
-            settings.onedrive_client_secret = client_secret
-        if tenant_id:
-            settings.onedrive_tenant_id = tenant_id
-        if folder_path:
-            settings.onedrive_folder_path = folder_path
-
-        # Persist to database (primary)
-        if refresh_token:
-            save_setting_to_db(db, "onedrive_refresh_token", refresh_token, changed_by=changed_by)
-        if client_id:
-            save_setting_to_db(db, "onedrive_client_id", client_id, changed_by=changed_by)
-        if client_secret:
-            save_setting_to_db(db, "onedrive_client_secret", client_secret, changed_by=changed_by)
-        if tenant_id:
-            save_setting_to_db(db, "onedrive_tenant_id", tenant_id, changed_by=changed_by)
-        if folder_path:
-            save_setting_to_db(db, "onedrive_folder_path", folder_path, changed_by=changed_by)
+        # Update in-memory settings and persist to database dynamically
+        for key, value in onedrive_settings.items():
+            setattr(settings, key, value)
+            save_setting_to_db(db, key, value, changed_by=changed_by)
 
         notify_settings_updated()
 

app/api/pipelines.py

@@ -117,8 +117,14 @@ PIPELINE_STEP_TYPES: dict[str, dict[str, Any]] = {
     },
     "classify": {
         "label": "Document Classification",
-        "description": "Classify the document type using AI without full metadata extraction.",
-        "config_schema": {},
+        "description": "Classify the document type using built-in and custom rules (filename patterns, content keywords, metadata matching).",
+        "config_schema": {
+            "use_builtin_rules": {
+                "type": "boolean",
+                "default": True,
+                "description": "Include the pre-built classification rules (invoice, contract, receipt, etc.).",
+            },
+        },
     },
 }
 

app/api/plans.py

@@ -196,11 +196,20 @@ def seed_plans(db: DbSession, _admin: AdminUser) -> dict[str, Any]:
 def reorder_plans(body: ReorderBody, db: DbSession, _admin: AdminUser) -> dict[str, Any]:
     """Update sort_order for each plan_id in *body.order* (position = index in list)."""
     updated = 0
-    for sort_order, plan_id in enumerate(body.order):
-        plan = db.query(SubscriptionPlan).filter(SubscriptionPlan.plan_id == plan_id).first()
+
+    # Fetch all requested plans in a single query to avoid N+1
+    plan_ids = body.order
+    plans = db.query(SubscriptionPlan).filter(SubscriptionPlan.plan_id.in_(plan_ids)).all()
+
+    # Build a map for fast O(1) lookup
+    plan_map = {p.plan_id: p for p in plans}
+
+    for sort_order, plan_id in enumerate(plan_ids):
+        plan = plan_map.get(plan_id)
         if plan:
             plan.sort_order = sort_order
             updated += 1
+
     try:
         db.commit()
     except Exception:

app/api/profile.py

@@ -99,6 +99,8 @@ class ProfileResponse(BaseModel):
     contact_email: str | None
     preferred_language: str | None
     preferred_theme: str | None
+    default_document_language: str | None
+    """ISO 639-1 code for the user's preferred document translation target language."""
     avatar_url: str
     """Gravatar URL or ``data:`` URI for a custom uploaded avatar."""
     is_local_user: bool
@@ -112,6 +114,10 @@ class ProfileUpdateRequest(BaseModel):
     contact_email: str | None = Field(default=None, max_length=255, description="Contact / notification e-mail")
     preferred_language: str | None = Field(default=None, description="ISO 639-1 language code, e.g. 'en', 'de'")
     preferred_theme: str | None = Field(default=None, description="Colour scheme: 'light', 'dark', or 'system'")
+    default_document_language: str | None = Field(
+        default=None,
+        description="ISO 639-1 code for the default document translation target language, e.g. 'en', 'de'",
+    )
 
 
 class ChangePasswordRequest(BaseModel):
@@ -149,6 +155,7 @@ async def get_profile(request: Request, db: DbSession) -> ProfileResponse:
         contact_email=profile.contact_email,  # type: ignore[arg-type]
         preferred_language=profile.preferred_language,  # type: ignore[arg-type]
         preferred_theme=profile.preferred_theme,  # type: ignore[arg-type]
+        default_document_language=profile.default_document_language,  # type: ignore[arg-type]
         avatar_url=avatar_url,
         is_local_user=is_local,
     )
@@ -201,6 +208,16 @@ async def update_profile(
             )
         profile.preferred_theme = theme or None  # type: ignore[assignment]
 
+    # Validate default document language
+    if body.default_document_language is not None:
+        doc_lang = body.default_document_language.lower().strip()
+        if doc_lang and doc_lang not in SUPPORTED_LANGUAGE_CODES:
+            raise HTTPException(
+                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+                detail=f"Unsupported language code: {doc_lang}",
+            )
+        profile.default_document_language = doc_lang or None  # type: ignore[assignment]
+
     if body.display_name is not None:
         profile.display_name = body.display_name.strip() or None  # type: ignore[assignment]
 
@@ -225,6 +242,7 @@ async def update_profile(
         contact_email=profile.contact_email,  # type: ignore[arg-type]
         preferred_language=profile.preferred_language,  # type: ignore[arg-type]
         preferred_theme=profile.preferred_theme,  # type: ignore[arg-type]
+        default_document_language=profile.default_document_language,  # type: ignore[arg-type]
         avatar_url=avatar_url,
         is_local_user=is_local,
     )

app/api/qr_auth.py

@@ -0,0 +1,249 @@
+"""QR code login API endpoints for mobile app authentication.
+
+Provides a secure challenge-response flow for logging into the mobile app
+by scanning a QR code displayed in the web interface:
+
+1. **Web user** calls ``POST /qr-auth/challenge`` → receives a time-limited
+   challenge token (encoded in the QR code).
+2. **Web UI** polls ``GET /qr-auth/challenge/{id}/status`` to detect when
+   the mobile app has claimed the challenge.
+3. **Mobile app** scans the QR code and calls ``POST /qr-auth/claim`` with
+   the challenge token + device name → receives an API token.
+
+Security properties:
+* Challenges expire after a configurable TTL (default 2 minutes).
+* Single-use: once claimed, a challenge cannot be reused (replay-safe).
+* Cryptographically random 64-byte tokens.
+* IP addresses are logged for audit.
+"""
+
+from __future__ import annotations
+
+import base64
+import io
+import logging
+from datetime import datetime
+from typing import Annotated, Any
+
+import segno
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.config import settings
+from app.database import get_db
+from app.middleware.audit_log import get_client_ip
+from app.utils.session_manager import (
+    claim_qr_challenge,
+    create_qr_challenge,
+    get_challenge_status,
+)
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/qr-auth", tags=["qr-auth"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+# ---------------------------------------------------------------------------
+# Auth helper
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if not owner_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+
+# ---------------------------------------------------------------------------
+# Request / Response schemas
+# ---------------------------------------------------------------------------
+
+
+class CreateChallengeResponse(BaseModel):
+    """Response after creating a QR login challenge."""
+
+    challenge_id: int
+    challenge_token: str
+    expires_at: datetime
+    ttl_seconds: int = Field(description="Seconds until the challenge expires (use for client-side countdown).")
+    qr_payload: str = Field(description="The string to encode in the QR code.")
+    qr_code_svg: str = Field(description="Base64-encoded SVG data URI of the QR code, ready for use in an <img> src.")
+
+
+class ChallengeStatusResponse(BaseModel):
+    """Response for polling the status of a QR challenge."""
+
+    id: int
+    status: str  # "pending", "claimed", "expired", "cancelled"
+    device_name: str | None = None
+    claimed_at: datetime | None = None
+    expires_at: datetime
+
+
+class ClaimChallengeRequest(BaseModel):
+    """Request body for claiming a QR login challenge."""
+
+    challenge_token: str = Field(min_length=1, max_length=256)
+    device_name: str = Field(
+        default="Mobile App",
+        min_length=1,
+        max_length=120,
+        description="Human-readable device name.",
+    )
+
+
+class ClaimChallengeResponse(BaseModel):
+    """Response after successfully claiming a QR challenge."""
+
+    token: str
+    token_id: int
+    name: str
+    owner_id: str
+    created_at: datetime
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+# QR code rendering parameters
+_QR_ERROR_LEVEL = "M"  # Medium error correction (~15% recovery); sufficient for on-screen display
+_QR_SCALE = 4  # Each QR module is rendered as 4×4 SVG pixels
+
+
+def _generate_qr_svg(payload: str) -> str:
+    """Generate a QR code for *payload* and return it as a base64 SVG data URI.
+
+    Using ``segno`` (pure-Python, no Pillow dependency) and SVG output so the
+    QR code scales crisply at any resolution without requiring a canvas or any
+    client-side JavaScript library.
+    """
+    qr = segno.make(payload, error=_QR_ERROR_LEVEL)
+    buf = io.BytesIO()
+    qr.save(buf, kind="svg", scale=_QR_SCALE, xmldecl=False, svgclass=None, lineclass=None, omitsize=True)
+    svg_bytes = buf.getvalue()
+    return "data:image/svg+xml;base64," + base64.b64encode(svg_bytes).decode("ascii")
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post("/challenge", status_code=status.HTTP_201_CREATED, response_model=CreateChallengeResponse)
+@require_login
+async def create_challenge(
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Create a new QR login challenge.
+
+    The returned ``qr_payload`` should be encoded into a QR code and
+    displayed to the user.  The mobile app scans this QR code and
+    calls the ``/claim`` endpoint.
+    """
+    if not settings.qr_login_enabled:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="QR login feature is currently disabled. Please contact your administrator to enable it.",
+        )
+    ip = get_client_ip(request)
+    challenge = create_qr_challenge(db, owner_id, ip_address=ip)
+
+    # The QR payload is a JSON-like string with enough info for the mobile
+    # app to know the server URL and challenge token.
+    base_url = str(request.base_url).rstrip("/")
+    qr_payload = f"docuelevate://qr-login?token={challenge.challenge_token}&server={base_url}"
+
+    # Compute the TTL in seconds so the client can run a countdown timer
+    # without comparing absolute timestamps (which breaks when client and
+    # server clocks are out of sync).
+    ttl_seconds = max(0, int((challenge.expires_at - challenge.created_at).total_seconds()))
+
+    return {
+        "challenge_id": challenge.id,
+        "challenge_token": challenge.challenge_token,
+        "expires_at": challenge.expires_at,
+        "ttl_seconds": ttl_seconds,
+        "qr_payload": qr_payload,
+        "qr_code_svg": _generate_qr_svg(qr_payload),
+    }
+
+
+@router.get("/challenge/{challenge_id}/status", response_model=ChallengeStatusResponse)
+@require_login
+async def poll_challenge_status(
+    request: Request,
+    challenge_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Poll the status of a QR login challenge.
+
+    The web UI calls this endpoint every few seconds to check if the
+    mobile app has scanned the QR code and claimed the challenge.
+    """
+    if not settings.qr_login_enabled:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="QR login feature is currently disabled. Please contact your administrator to enable it.",
+        )
+    result = get_challenge_status(db, challenge_id, owner_id)
+    if not result:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Challenge not found")
+    return result
+
+
+@router.post("/claim", response_model=ClaimChallengeResponse)
+async def claim_challenge(
+    request: Request,
+    body: ClaimChallengeRequest,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Claim a QR login challenge and receive an API token.
+
+    This endpoint is called by the mobile app after scanning a QR code.
+    It does **not** require authentication — the challenge token itself
+    serves as proof that the user authorized this login from their web
+    session.
+    """
+    if not settings.qr_login_enabled:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="QR login feature is currently disabled. Please contact your administrator to enable it.",
+        )
+    ip = get_client_ip(request)
+    result = claim_qr_challenge(db, body.challenge_token, device_name=body.device_name, ip_address=ip)
+
+    if not result:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="Invalid, expired, or already claimed challenge.",
+        )
+
+    try:
+        from app.utils.audit_service import record_event
+
+        record_event(
+            db,
+            action="qr_login_claimed",
+            user=result["owner_id"],
+            resource_type="session",
+            ip_address=ip,
+            details={"device_name": body.device_name, "token_id": result["token_id"]},
+            severity="info",
+        )
+    except Exception:
+        logger.debug("Failed to write QR login audit event", exc_info=True)
+
+    return result

app/api/sessions.py

@@ -0,0 +1,196 @@
+"""API endpoints for managing user sessions.
+
+Provides endpoints for listing active sessions, revoking individual sessions,
+and the "log off everywhere" feature that invalidates all sessions and API
+tokens across all devices.
+"""
+
+from __future__ import annotations
+
+import logging
+from datetime import datetime
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.database import get_db
+from app.middleware.audit_log import get_client_ip
+from app.utils.session_manager import (
+    get_session_lifetime_days,
+    list_user_sessions,
+    revoke_all_sessions,
+    revoke_session,
+)
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/sessions", tags=["sessions"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+# ---------------------------------------------------------------------------
+# Auth helper
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if not owner_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+
+# ---------------------------------------------------------------------------
+# Response schemas
+# ---------------------------------------------------------------------------
+
+
+class SessionResponse(BaseModel):
+    """Serialised user session for the management UI."""
+
+    id: int
+    device_info: str | None
+    ip_address: str | None
+    created_at: datetime
+    last_active_at: datetime
+    expires_at: datetime
+    is_current: bool = False
+
+
+class SessionListResponse(BaseModel):
+    """Response for listing active sessions."""
+
+    sessions: list[SessionResponse]
+    session_lifetime_days: int
+
+
+class RevokeAllResponse(BaseModel):
+    """Response after revoking all sessions."""
+
+    revoked_count: int
+    message: str
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/", response_model=SessionListResponse)
+@require_login
+async def list_sessions(
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """List all active sessions for the current user."""
+    sessions = list_user_sessions(db, owner_id)
+
+    # Determine which session is the current one
+    current_token = request.session.get("_session_token")
+
+    session_list = []
+    for s in sessions:
+        session_list.append(
+            {
+                "id": s.id,
+                "device_info": s.device_info,
+                "ip_address": s.ip_address,
+                "created_at": s.created_at,
+                "last_active_at": s.last_active_at,
+                "expires_at": s.expires_at,
+                "is_current": s.session_token == current_token if current_token else False,
+            }
+        )
+
+    return {
+        "sessions": session_list,
+        "session_lifetime_days": get_session_lifetime_days(),
+    }
+
+
+@router.delete("/{session_id}", status_code=status.HTTP_204_NO_CONTENT)
+@require_login
+async def revoke_single_session(
+    request: Request,
+    session_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> None:
+    """Revoke a specific session by ID."""
+    success = revoke_session(db, session_id, owner_id)
+    if not success:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Session not found")
+
+    try:
+        from app.utils.audit_service import record_event
+
+        record_event(
+            db,
+            action="session_revoked",
+            user=owner_id,
+            resource_type="session",
+            resource_id=str(session_id),
+            ip_address=get_client_ip(request),
+            severity="info",
+        )
+    except Exception:
+        logger.debug("Failed to write session revocation audit event", exc_info=True)
+
+
+@router.post("/revoke-all", response_model=RevokeAllResponse)
+@require_login
+async def revoke_all(
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Revoke all sessions except the current one ("log off everywhere").
+
+    Also revokes all active API tokens for the user, which invalidates
+    mobile app sessions and any programmatic access.
+    """
+    # Find current session to preserve it
+    current_token = request.session.get("_session_token")
+    current_session_id = None
+    if current_token:
+        from app.models import UserSession
+
+        current = db.query(UserSession).filter(UserSession.session_token == current_token).first()
+        if current:
+            current_session_id = current.id
+
+    count = revoke_all_sessions(
+        db,
+        owner_id,
+        except_session_id=current_session_id,
+        revoke_api_tokens=True,
+    )
+
+    try:
+        from app.utils.audit_service import record_event
+
+        record_event(
+            db,
+            action="revoke_all_sessions",
+            user=owner_id,
+            resource_type="session",
+            ip_address=get_client_ip(request),
+            details={"revoked_count": count},
+            severity="warning",
+        )
+    except Exception:
+        logger.debug("Failed to write revoke-all audit event", exc_info=True)
+
+    return {
+        "revoked_count": count,
+        "message": f"Successfully revoked {count} session(s) and all API tokens.",
+    }

app/api/settings.py

@@ -55,6 +55,12 @@ class SettingUpdate(BaseModel):
     value: Optional[str] = Field(None, description="Setting value (None to delete)")
 
 
+class SettingValueUpdate(BaseModel):
+    """Model for updating a setting value by key (key is provided in the URL path)."""
+
+    value: Optional[str] = Field(None, description="Setting value (None to delete)")
+
+
 class SettingResponse(BaseModel):
     """Model for setting response"""
 
@@ -323,6 +329,62 @@ async def update_setting(
         )
 
 
+@router.put("/{key}")
+async def put_setting(
+    key: str,
+    body: SettingValueUpdate,
+    request: Request,
+    db: DbSession,
+    admin: AdminUser,
+):
+    """
+    Update a specific setting by key (RESTful PUT).
+
+    Accepts a body with only ``value``; the key is taken from the URL path.
+    This is the endpoint used by the admin Connections wizard.
+    Admin only.
+    """
+    validate_setting_key(key)
+    try:
+        if body.value is not None:
+            is_valid, error_message = validate_setting_value(key, body.value)
+            if not is_valid:
+                raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=error_message)
+
+        user = request.session.get("user", {}) if hasattr(request, "session") else {}
+        changed_by = (
+            user.get("preferred_username") or user.get("username") or user.get("email") or user.get("id") or "admin"
+        )
+
+        success = save_setting_to_db(db, key, body.value, changed_by=changed_by)
+        if not success:
+            raise HTTPException(
+                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+                detail="Failed to save setting to database",
+            )
+
+        notify_settings_updated()
+
+        metadata = get_setting_metadata(key)
+        restart_required = metadata.get("restart_required", False)
+
+        return {
+            "success": True,
+            "message": f"Setting '{key}' updated successfully",
+            "restart_required": restart_required,
+            "key": key,
+            "value": body.value,
+        }
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Error updating setting {key}: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"Failed to update setting: {key}",
+        )
+
+
 @router.delete("/{key}")
 async def delete_setting(key: str, request: Request, db: DbSession, admin: AdminUser):
     """

app/api/shared_links.py

@@ -313,16 +313,18 @@ async def list_shared_links(
     active_only: bool = Query(False, description="When true, only return active (non-revoked) links"),
 ) -> list[dict[str, Any]]:
     """List all shared links created by the authenticated user."""
-    q = db.query(SharedLink).filter(SharedLink.owner_id == owner_id)
+    q = (
+        db.query(SharedLink, FileRecord.original_filename)
+        .outerjoin(FileRecord, SharedLink.file_id == FileRecord.id)
+        .filter(SharedLink.owner_id == owner_id)
+    )
     if active_only:
         q = q.filter(SharedLink.is_active.is_(True))
-    links = q.order_by(SharedLink.created_at.desc()).all()
+    links_with_filenames = q.order_by(SharedLink.created_at.desc()).all()
 
     base_url = str(request.base_url).rstrip("/")
     result = []
-    for link in links:
-        file_record = db.query(FileRecord).filter(FileRecord.id == link.file_id).first()
-        filename = file_record.original_filename if file_record else None
+    for link, filename in links_with_filenames:
         result.append(_link_to_dict(link, base_url, filename))
     return result
 

app/api/sharing.py

@@ -0,0 +1,355 @@
+"""File-sharing API endpoints.
+
+Provides CRUD operations for ``FileShare`` records, which grant named
+users ``viewer`` or ``editor`` access to a document owned by someone
+else.  Only the file owner may create, update, or revoke shares.
+"""
+
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Body, Depends, HTTPException, Request, status
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.database import get_db
+from app.models import FILE_SHARE_ROLE_VIEWER, FILE_SHARE_ROLES, FileRecord, FileShare, UserProfile
+from app.utils.user_scope import get_current_owner_id, get_file_role
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(tags=["sharing"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _serialize_share(share: FileShare) -> dict[str, Any]:
+    """Serialize a ``FileShare`` to a JSON-friendly dict."""
+    return {
+        "id": share.id,
+        "file_id": share.file_id,
+        "owner_id": share.owner_id,
+        "shared_with_user_id": share.shared_with_user_id,
+        "role": share.role,
+        "created_at": share.created_at.isoformat() if share.created_at else None,
+        "updated_at": share.updated_at.isoformat() if share.updated_at else None,
+    }
+
+
+def _require_owner(file_record: FileRecord, user_id: str | None, db: Session) -> None:
+    """Raise 403 unless the calling user is the file owner."""
+    if get_file_role(file_record, user_id, db) != "owner":
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Only the file owner can manage shares",
+        )
+
+
+# ---------------------------------------------------------------------------
+# List shares
+# ---------------------------------------------------------------------------
+
+
+@router.get("/files/{file_id}/shares")
+@require_login
+def list_shares(request: Request, file_id: int, db: DbSession):
+    """List all shares for a document.
+
+    Only the file owner (or an admin) may call this endpoint.
+
+    Path Parameters:
+        file_id: The ID of the document.
+
+    Returns:
+        A list of share objects.
+    """
+    user_id = get_current_owner_id(request)
+    user = request.session.get("user")
+    is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    role = get_file_role(file_record, user_id, db)
+    if role is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    if role != "owner" and not is_admin:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Only the file owner can view shares",
+        )
+
+    shares = db.query(FileShare).filter(FileShare.file_id == file_id).all()
+    return [_serialize_share(s) for s in shares]
+
+
+# ---------------------------------------------------------------------------
+# Create share
+# ---------------------------------------------------------------------------
+
+
+@router.post("/files/{file_id}/shares", status_code=status.HTTP_201_CREATED)
+@require_login
+def create_share(
+    request: Request,
+    file_id: int,
+    db: DbSession,
+    shared_with_user_id: str = Body(..., embed=True),
+    role: str = Body(FILE_SHARE_ROLE_VIEWER, embed=True),
+):
+    """Share a document with another user.
+
+    Only the file owner may share the document.  Sharing with a user
+    that already has access updates their role instead of creating a
+    duplicate record.
+
+    Path Parameters:
+        file_id: The ID of the document to share.
+
+    Request body (JSON):
+        shared_with_user_id: The stable user identifier of the recipient.
+        role: ``"viewer"`` (default) or ``"editor"``.
+
+    Returns:
+        The created or updated share object.
+    """
+    owner_id = get_current_owner_id(request)
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    _require_owner(file_record, owner_id, db)
+
+    if role not in FILE_SHARE_ROLES:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"role must be one of: {', '.join(FILE_SHARE_ROLES)}",
+        )
+
+    if not shared_with_user_id or not shared_with_user_id.strip():
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="shared_with_user_id must be a non-empty string",
+        )
+    shared_with_user_id = shared_with_user_id.strip()
+
+    # Cannot share with yourself
+    if shared_with_user_id == owner_id:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="You cannot share a file with yourself",
+        )
+
+    try:
+        existing = (
+            db.query(FileShare)
+            .filter(FileShare.file_id == file_id, FileShare.shared_with_user_id == shared_with_user_id)
+            .first()
+        )
+
+        if existing:
+            # Update role if different
+            if existing.role != role:
+                existing.role = role
+                db.commit()
+                db.refresh(existing)
+                logger.info(
+                    "Share updated: file_id=%s, shared_with=%s, role=%s, by owner=%s",
+                    file_id,
+                    shared_with_user_id,
+                    role,
+                    owner_id,
+                )
+            return _serialize_share(existing)
+
+        share = FileShare(
+            file_id=file_id,
+            owner_id=owner_id,
+            shared_with_user_id=shared_with_user_id,
+            role=role,
+        )
+        db.add(share)
+        db.commit()
+        db.refresh(share)
+    except HTTPException:
+        raise
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create share: file_id=%s, shared_with=%s", file_id, shared_with_user_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to create share",
+        )
+
+    logger.info(
+        "Share created: id=%s, file_id=%s, shared_with=%s, role=%s, by owner=%s",
+        share.id,
+        file_id,
+        shared_with_user_id,
+        role,
+        owner_id,
+    )
+    return _serialize_share(share)
+
+
+# ---------------------------------------------------------------------------
+# Update share role
+# ---------------------------------------------------------------------------
+
+
+@router.put("/files/{file_id}/shares/{share_id}")
+@require_login
+def update_share(
+    request: Request,
+    file_id: int,
+    share_id: int,
+    db: DbSession,
+    role: str = Body(..., embed=True),
+):
+    """Update the role of an existing share.
+
+    Only the file owner may change the role of a share.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        share_id: The ID of the share record to update.
+
+    Request body (JSON):
+        role: New role — ``"viewer"`` or ``"editor"``.
+
+    Returns:
+        The updated share object.
+    """
+    owner_id = get_current_owner_id(request)
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    _require_owner(file_record, owner_id, db)
+
+    if role not in FILE_SHARE_ROLES:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"role must be one of: {', '.join(FILE_SHARE_ROLES)}",
+        )
+
+    share = db.query(FileShare).filter(FileShare.id == share_id, FileShare.file_id == file_id).first()
+    if not share:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Share not found")
+
+    try:
+        share.role = role
+        db.commit()
+        db.refresh(share)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to update share: share_id=%s", share_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to update share",
+        )
+
+    logger.info("Share updated: id=%s, file_id=%s, new_role=%s, by owner=%s", share_id, file_id, role, owner_id)
+    return _serialize_share(share)
+
+
+# ---------------------------------------------------------------------------
+# Revoke share
+# ---------------------------------------------------------------------------
+
+
+@router.delete("/files/{file_id}/shares/{share_id}", status_code=status.HTTP_200_OK)
+@require_login
+def revoke_share(request: Request, file_id: int, share_id: int, db: DbSession):
+    """Revoke a share, removing the user's access.
+
+    Only the file owner may revoke shares.
+
+    Path Parameters:
+        file_id: The ID of the document.
+        share_id: The ID of the share record to delete.
+
+    Returns:
+        A success message.
+    """
+    owner_id = get_current_owner_id(request)
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    _require_owner(file_record, owner_id, db)
+
+    share = db.query(FileShare).filter(FileShare.id == share_id, FileShare.file_id == file_id).first()
+    if not share:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Share not found")
+
+    try:
+        db.delete(share)
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to revoke share: share_id=%s", share_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to revoke share",
+        )
+
+    logger.info("Share revoked: id=%s, file_id=%s, by owner=%s", share_id, file_id, owner_id)
+    return {"status": "success", "message": "Share revoked successfully"}
+
+
+# ---------------------------------------------------------------------------
+# List users that the file is already shared with  (for the share-picker UI)
+# ---------------------------------------------------------------------------
+
+
+@router.get("/files/{file_id}/shared-with")
+@require_login
+def list_shared_with(request: Request, file_id: int, db: DbSession):
+    """Return the list of users a document is shared with and their roles.
+
+    Accessible to any user that has at least viewer access to the file,
+    so that editors/viewers can see who else has access.
+
+    Path Parameters:
+        file_id: The ID of the document.
+
+    Returns:
+        A list of ``{share_id, user_id, display_name, role}`` objects.
+    """
+    user_id = get_current_owner_id(request)
+    user = request.session.get("user")
+    is_admin = isinstance(user, dict) and bool(user.get("is_admin"))
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    role = get_file_role(file_record, user_id, db)
+    if role is None and not is_admin:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    shares = db.query(FileShare).filter(FileShare.file_id == file_id).all()
+
+    results = []
+    for s in shares:
+        profile = db.query(UserProfile).filter(UserProfile.user_id == s.shared_with_user_id).first()
+        results.append(
+            {
+                "share_id": s.id,
+                "user_id": s.shared_with_user_id,
+                "display_name": (profile.display_name if profile and profile.display_name else s.shared_with_user_id),
+                "role": s.role,
+            }
+        )
+    return results

app/api/system_reset.py

@@ -0,0 +1,124 @@
+"""
+System reset API endpoints for DocuElevate.
+
+Provides admin-only REST endpoints for:
+- Full system reset (wipe all user data)
+- Reset with re-import (move originals → reimport folder, wipe, re-ingest)
+
+Both operations require the ``ENABLE_FACTORY_RESET=True`` feature flag and
+admin privileges.
+"""
+
+import logging
+from typing import Annotated
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel
+from sqlalchemy.orm import Session
+
+from app.config import settings
+from app.database import get_db
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/admin/system-reset", tags=["system-reset"])
+
+
+def _require_admin(request: Request) -> dict:
+    """Ensure the caller is an admin.  Raises 403 otherwise."""
+    user = request.session.get("user")
+    if not user or not user.get("is_admin"):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required")
+    return user
+
+
+AdminUser = Annotated[dict, Depends(_require_admin)]
+
+
+def _require_feature_enabled() -> None:
+    """Raise 404 when the factory-reset feature flag is off."""
+    if not settings.enable_factory_reset:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="System reset is not enabled. Set ENABLE_FACTORY_RESET=True to activate.",
+        )
+
+
+class ResetRequest(BaseModel):
+    """Body for system reset endpoints.  Requires explicit confirmation."""
+
+    confirmation: str
+
+
+@router.post("/full")
+async def full_reset(
+    body: ResetRequest,
+    _admin: AdminUser,
+    db: Session = Depends(get_db),
+) -> dict:
+    """Wipe all user data (database + work-files).
+
+    The caller must send ``{"confirmation": "DELETE"}`` to proceed.
+    """
+    _require_feature_enabled()
+
+    if body.confirmation != "DELETE":
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail='Confirmation required: send {"confirmation": "DELETE"} to proceed.',
+        )
+
+    from app.utils.system_reset import perform_full_reset
+
+    try:
+        result = perform_full_reset(db)
+    except Exception as exc:
+        logger.exception("Full system reset failed")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"System reset failed: {exc}",
+        ) from exc
+
+    return {"status": "ok", "result": result}
+
+
+@router.post("/reimport")
+async def reset_and_reimport(
+    body: ResetRequest,
+    _admin: AdminUser,
+    db: Session = Depends(get_db),
+) -> dict:
+    """Move original files to a reimport folder, wipe everything, and
+    configure the reimport folder as a watch folder for automatic
+    re-ingestion.
+
+    The caller must send ``{"confirmation": "REIMPORT"}`` to proceed.
+    """
+    _require_feature_enabled()
+
+    if body.confirmation != "REIMPORT":
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail='Confirmation required: send {"confirmation": "REIMPORT"} to proceed.',
+        )
+
+    from app.utils.system_reset import perform_reset_and_reimport
+
+    try:
+        result = perform_reset_and_reimport(db)
+    except Exception as exc:
+        logger.exception("Reset-and-reimport failed")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail=f"Reset and reimport failed: {exc}",
+        ) from exc
+
+    return {"status": "ok", "result": result}
+
+
+@router.get("/status")
+async def reset_status(_admin: AdminUser) -> dict:
+    """Return whether the system reset feature is enabled."""
+    return {
+        "enabled": settings.enable_factory_reset,
+        "factory_reset_on_startup": settings.factory_reset_on_startup,
+    }

app/api/translation.py

@@ -0,0 +1,156 @@
+"""
+API endpoints for document translation.
+
+Provides on-the-fly translation via the AI provider and access to the
+persisted default-language translation.
+"""
+
+import logging
+from typing import Annotated
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi.responses import JSONResponse
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.config import settings
+from app.database import get_db
+from app.models import FileRecord
+from app.utils.ai_provider import get_ai_provider
+from app.utils.user_scope import apply_owner_filter
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter()
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# Maximum characters sent to the AI provider for a single translation request.
+_MAX_TRANSLATION_INPUT = 50_000
+
+
+def _get_file_or_404(db: Session, file_id: int, request: Request) -> FileRecord:
+    """Fetch a FileRecord visible to the current user or raise 404."""
+    query = db.query(FileRecord).filter(FileRecord.id == file_id)
+    query = apply_owner_filter(query, request)
+    record = query.first()
+    if not record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+    return record
+
+
+@router.get("/files/{file_id}/translation/default")
+@require_login
+def get_default_translation(
+    request: Request,
+    file_id: int,
+    db: DbSession,
+) -> JSONResponse:
+    """Return the persisted default-language translation for a document.
+
+    Returns 404 if no default-language translation has been generated yet
+    (e.g. because the document is already in the default language).
+    """
+    record = _get_file_or_404(db, file_id, request)
+
+    if not record.default_language_text:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="No default-language translation available for this file",
+        )
+
+    return JSONResponse(
+        content={
+            "file_id": record.id,
+            "detected_language": record.detected_language,
+            "default_language_code": record.default_language_code,
+            "text": record.default_language_text,
+        }
+    )
+
+
+@router.get("/files/{file_id}/translate")
+@require_login
+def translate_on_the_fly(
+    request: Request,
+    file_id: int,
+    db: DbSession,
+    lang: str = Query(..., min_length=2, max_length=10, description="Target language ISO 639-1 code"),
+) -> JSONResponse:
+    """Translate a document's extracted text into an arbitrary language on the fly.
+
+    The translation is generated via the configured AI provider and is **not**
+    persisted.  For the default-language translation, use the
+    ``/files/{file_id}/translation/default`` endpoint instead.
+    """
+    record = _get_file_or_404(db, file_id, request)
+
+    source_text = record.ocr_text
+    if not source_text:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail="No extracted text available for this file — translation requires OCR text",
+        )
+
+    # If the requested language matches what is already stored, return it directly.
+    if record.default_language_code and lang == record.default_language_code and record.default_language_text:
+        return JSONResponse(
+            content={
+                "file_id": record.id,
+                "source_language": record.detected_language,
+                "target_language": lang,
+                "text": record.default_language_text,
+                "cached": True,
+            }
+        )
+
+    # If the detected language already matches, return the original text.
+    detected = record.detected_language
+    if detected and detected == lang:
+        return JSONResponse(
+            content={
+                "file_id": record.id,
+                "source_language": detected,
+                "target_language": lang,
+                "text": source_text,
+                "cached": True,
+            }
+        )
+
+    # Truncate to keep AI costs bounded.
+    text_to_translate = source_text[:_MAX_TRANSLATION_INPUT]
+
+    try:
+        provider = get_ai_provider()
+        model = settings.ai_model or settings.openai_model
+        translated = provider.chat_completion(
+            messages=[
+                {
+                    "role": "system",
+                    "content": (
+                        f"You are a professional translator. Translate the following text "
+                        f"into {lang}. Preserve the original formatting, paragraph structure, "
+                        f"and meaning. Do not add any commentary — output ONLY the translated text."
+                    ),
+                },
+                {"role": "user", "content": text_to_translate},
+            ],
+            model=model,
+            temperature=0.3,
+        )
+    except Exception as exc:
+        logger.exception(f"On-the-fly translation failed for file {file_id}: {exc}")
+        raise HTTPException(
+            status_code=status.HTTP_502_BAD_GATEWAY,
+            detail="Translation failed — the AI provider returned an error",
+        )
+
+    return JSONResponse(
+        content={
+            "file_id": record.id,
+            "source_language": detected or "unknown",
+            "target_language": lang,
+            "text": translated,
+            "cached": False,
+        }
+    )

app/api/url_upload.py

@@ -9,12 +9,14 @@ import urllib.parse
 import uuid
 from typing import Optional
 
-import requests
-from fastapi import APIRouter, HTTPException, Request
+import aiofiles
+import httpx
+from fastapi import APIRouter, Depends, HTTPException, Request
 from pydantic import BaseModel, HttpUrl, field_validator
 
 from app.auth import require_login
 from app.config import settings
+from app.middleware.upload_rate_limit import require_upload_rate_limit
 from app.tasks.process_document import process_document
 from app.utils.allowed_types import ALLOWED_MIME_TYPES
 from app.utils.filename_utils import sanitize_filename
@@ -106,7 +108,11 @@ def validate_file_type(content_type: str, filename: str) -> bool:
 
 @router.post("/process-url")
 @require_login
-async def process_url(request: Request, url_request: URLUploadRequest):
+async def process_url(
+    request: Request,
+    url_request: URLUploadRequest,
+    _rate_ok: None = Depends(require_upload_rate_limit),
+):
     """
     Download a file from a URL and enqueue it for processing.
 
@@ -153,67 +159,73 @@ async def process_url(request: Request, url_request: URLUploadRequest):
         logger.info(f"Downloading file from URL: {url}")
 
         # Use configured timeout to prevent hanging
-        response = requests.get(
-            url,
+        async with httpx.AsyncClient(
             timeout=settings.http_request_timeout,
-            stream=True,  # Stream to handle large files
-            allow_redirects=True,  # Follow redirects
+            follow_redirects=True,
             headers={
                 "User-Agent": "DocuElevate/1.0",  # Identify ourselves
             },
-        )
-        response.raise_for_status()
+        ) as client:
+            async with client.stream("GET", url) as response:
+                response.raise_for_status()
 
-        # Validate content type
-        content_type = response.headers.get("Content-Type", "")
-        if not validate_file_type(content_type, safe_filename):
-            raise HTTPException(
-                status_code=400,
-                detail=f"Unsupported file type: {content_type}. "
-                "Supported types: PDF, Office documents, images, plain text",
-            )
+                # Validate content type
+                content_type = response.headers.get("Content-Type", "")
+                if not validate_file_type(content_type, safe_filename):
+                    raise HTTPException(
+                        status_code=400,
+                        detail=f"Unsupported file type: {content_type}. "
+                        "Supported types: PDF, Office documents, images, plain text",
+                    )
 
-        # Check content length before downloading
-        content_length = response.headers.get("Content-Length")
-        if content_length:
-            file_size = int(content_length)
-            max_size = settings.max_upload_size
-            if file_size > max_size:
-                raise HTTPException(
-                    status_code=413,
-                    detail=f"File too large: {file_size} bytes (max {max_size} bytes)",
-                )
-
-        # Generate unique filename
-        unique_id = str(uuid.uuid4())
-        if "." in safe_filename:
-            file_extension = safe_filename.rsplit(".", 1)[1]
-            target_filename = f"{unique_id}.{file_extension}"
-        else:
-            target_filename = unique_id
-
-        target_path = os.path.join(settings.workdir, target_filename)
-
-        # Download file in chunks to handle large files
-        downloaded_size = 0
-        max_size = settings.max_upload_size
-
-        with open(target_path, "wb") as f:
-            for chunk in response.iter_content(chunk_size=8192):
-                if chunk:
-                    f.write(chunk)
-                    downloaded_size += len(chunk)
-
-                    # Check size during download
-                    if downloaded_size > max_size:
-                        # Remove partial file
-                        f.close()
-                        os.remove(target_path)
+                # Check content length before downloading
+                content_length = response.headers.get("Content-Length")
+                if content_length:
+                    file_size = int(content_length)
+                    max_size = settings.max_upload_size
+                    if file_size > max_size:
                         raise HTTPException(
                             status_code=413,
-                            detail=f"File too large: exceeded {max_size} bytes during download",
+                            detail=f"File too large: {file_size} bytes (max {max_size} bytes)",
                         )
 
+                # Generate unique filename
+                unique_id = str(uuid.uuid4())
+
+                # Check for extension using original_filename to avoid any CodeQL issues
+                # with safe_filename which is derived from the URL directly.
+                if "." in original_filename:
+                    _, ext = os.path.splitext(original_filename)
+                    # Strip out the leading dot and any non-alphanumeric chars
+                    clean_ext = "".join(c for c in ext if c.isalnum())
+                    if not clean_ext:
+                        clean_ext = "bin"
+                    target_filename = f"{unique_id}.{clean_ext}"
+                else:
+                    target_filename = unique_id
+
+                target_path = os.path.join(settings.workdir, target_filename)
+
+                # Download file in chunks to handle large files
+                downloaded_size = 0
+                max_size = settings.max_upload_size
+
+                async with aiofiles.open(target_path, "wb") as f:
+                    async for chunk in response.aiter_bytes(chunk_size=8192):
+                        if chunk:
+                            await f.write(chunk)
+                            downloaded_size += len(chunk)
+
+                            # Check size during download
+                            if downloaded_size > max_size:
+                                # Remove partial file
+                                await f.close()
+                                os.remove(target_path)
+                                raise HTTPException(
+                                    status_code=413,
+                                    detail=f"File too large: exceeded {max_size} bytes during download",
+                                )
+
         logger.info(f"Downloaded file from URL '{url}' as '{target_filename}' ({downloaded_size} bytes)")
 
         # Enqueue for processing
@@ -227,19 +239,19 @@ async def process_url(request: Request, url_request: URLUploadRequest):
             "size": downloaded_size,
         }
 
-    except requests.exceptions.Timeout:
+    except httpx.TimeoutException:
         logger.error(f"Timeout while downloading file from URL: {url}")
         raise HTTPException(status_code=408, detail="Request timeout: server took too long to respond")
 
-    except requests.exceptions.ConnectionError as e:
+    except httpx.ConnectError as e:
         logger.error(f"Connection error while downloading file from URL: {url} - {str(e)}")
         raise HTTPException(status_code=502, detail=f"Failed to connect to URL: {str(e)}")
 
-    except requests.exceptions.HTTPError as e:
+    except httpx.HTTPStatusError as e:
         logger.error(f"HTTP error while downloading file from URL: {url} - {str(e)}")
         raise HTTPException(status_code=e.response.status_code, detail=f"HTTP error: {str(e)}")
 
-    except requests.exceptions.RequestException as e:
+    except httpx.RequestError as e:
         logger.error(f"Error downloading file from URL: {url} - {str(e)}")
         raise HTTPException(status_code=500, detail=f"Failed to download file: {str(e)}")
 

app/auth.py

@@ -45,78 +45,264 @@ OAUTH_PROVIDER_NAME = "Single Sign-On"
 # Social login providers that are enabled and registered
 SOCIAL_PROVIDERS: dict[str, dict[str, str]] = {}
 
-if AUTH_ENABLED and settings.authentik_client_id and settings.authentik_client_secret:
-    oauth.register(
-        name="authentik",
-        client_id=settings.authentik_client_id,
-        client_secret=settings.authentik_client_secret,
-        server_metadata_url=settings.authentik_config_url,
-        client_kwargs={"scope": "openid profile email"},
-    )
-    OAUTH_CONFIGURED = True
-    OAUTH_PROVIDER_NAME = settings.oauth_provider_name or "Authentik SSO"
 
-# --- Social Login Providers ---------------------------------------------------
-if AUTH_ENABLED and settings.social_auth_google_enabled:
-    if settings.social_auth_google_client_id and settings.social_auth_google_client_secret:
-        oauth.register(
-            name="google",
-            client_id=settings.social_auth_google_client_id,
-            client_secret=settings.social_auth_google_client_secret,
-            server_metadata_url="https://accounts.google.com/.well-known/openid-configuration",
+# ---------------------------------------------------------------------------
+# Helpers for dynamic (re-)registration of OAuth providers
+# ---------------------------------------------------------------------------
+
+
+def _register_oauth_client(name: str, **kwargs: object) -> None:
+    """Register (or re-register) an authlib OAuth client, clearing any cached instance.
+
+    authlib caches the constructed client object in ``oauth._clients`` after the
+    first ``register()`` call.  Subsequent ``register()`` calls overwrite the
+    registry entry but the stale cached client is still returned by
+    ``create_client()`` / ``__getattr__``.  Popping the name from ``_clients``
+    before re-registering ensures the new credentials are picked up immediately.
+
+    Args:
+        name: Provider name (e.g. ``"google"``, ``"github"``).
+        **kwargs: Keyword arguments forwarded verbatim to ``oauth.register()``.
+    """
+    oauth._clients.pop(name, None)
+    oauth.register(name, **kwargs)
+
+
+def _dropbox_userinfo_compliance_fix(client, user_cls, token, data):
+    """Normalize Dropbox userinfo response for authlib compatibility.
+
+    Dropbox's /2/users/get_current_account returns a non-standard response
+    format. This compliance fix normalizes the response data — the HTTP
+    method (POST) is handled by authlib's compliance infrastructure.
+
+    Args:
+        client: The OAuth client instance (required by authlib compliance fix interface).
+        user_cls: The user class (required by authlib compliance fix interface).
+        token: The OAuth token dict.
+        data: The raw userinfo response dict from Dropbox.
+
+    Returns:
+        The normalized userinfo dict with ``sub`` and ``name`` fields.
+    """
+    # Dropbox returns account_id instead of sub
+    if "account_id" in data and "sub" not in data:
+        data["sub"] = data["account_id"]
+    # Normalize name field
+    name_info = data.get("name", {})
+    if isinstance(name_info, dict) and "display_name" in name_info:
+        data["name"] = name_info["display_name"]
+    return data
+
+
+def _setup_social_providers() -> None:
+    """Register all configured OAuth / social-login providers from current settings.
+
+    This function is **idempotent**: it clears ``SOCIAL_PROVIDERS``,
+    ``OAUTH_CONFIGURED``, and ``OAUTH_PROVIDER_NAME`` before rebuilding them,
+    and calls :func:`_register_oauth_client` (which also clears the authlib
+    client cache) so that credential changes in the database are reflected
+    without an application restart.
+
+    Can safely be called multiple times, e.g. after a settings reload.
+    """
+    global OAUTH_CONFIGURED, OAUTH_PROVIDER_NAME
+
+    SOCIAL_PROVIDERS.clear()
+    OAUTH_CONFIGURED = False
+    OAUTH_PROVIDER_NAME = "Single Sign-On"
+
+    if not AUTH_ENABLED:
+        return
+
+    # --- Authentik / OIDC ---
+    if settings.authentik_client_id and settings.authentik_client_secret:
+        _register_oauth_client(
+            "authentik",
+            client_id=settings.authentik_client_id,
+            client_secret=settings.authentik_client_secret,
+            server_metadata_url=settings.authentik_config_url,
             client_kwargs={"scope": "openid profile email"},
         )
-        SOCIAL_PROVIDERS["google"] = {"name": "Google", "icon": "fab fa-google", "color": "red"}
-        logger.info("Social login provider registered: Google")
-    else:
-        logger.warning("SOCIAL_AUTH_GOOGLE_ENABLED=true but client ID/secret not configured")
+        OAUTH_CONFIGURED = True
+        OAUTH_PROVIDER_NAME = settings.oauth_provider_name or "Authentik SSO"
 
-if AUTH_ENABLED and settings.social_auth_microsoft_enabled:
-    if settings.social_auth_microsoft_client_id and settings.social_auth_microsoft_client_secret:
-        tenant = settings.social_auth_microsoft_tenant or "common"
-        oauth.register(
-            name="microsoft",
-            client_id=settings.social_auth_microsoft_client_id,
-            client_secret=settings.social_auth_microsoft_client_secret,
-            server_metadata_url=f"https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration",
-            client_kwargs={"scope": "openid profile email"},
-        )
-        SOCIAL_PROVIDERS["microsoft"] = {"name": "Microsoft", "icon": "fab fa-microsoft", "color": "blue"}
-        logger.info("Social login provider registered: Microsoft (tenant=%s)", tenant)
-    else:
-        logger.warning("SOCIAL_AUTH_MICROSOFT_ENABLED=true but client ID/secret not configured")
+    # --- Social Login Providers ---
 
-if AUTH_ENABLED and settings.social_auth_apple_enabled:
-    if settings.social_auth_apple_client_id and settings.social_auth_apple_team_id:
-        oauth.register(
-            name="apple",
-            client_id=settings.social_auth_apple_client_id,
-            server_metadata_url="https://appleid.apple.com/.well-known/openid-configuration",
-            client_kwargs={
-                "scope": "openid name email",
-                "response_mode": "form_post",
-            },
-        )
-        SOCIAL_PROVIDERS["apple"] = {"name": "Apple", "icon": "fab fa-apple", "color": "gray"}
-        logger.info("Social login provider registered: Apple")
-    else:
-        logger.warning("SOCIAL_AUTH_APPLE_ENABLED=true but client ID/team ID not configured")
+    # Google
+    if settings.social_auth_google_enabled:
+        _google_client_id = settings.social_auth_google_client_id
+        _google_client_secret = settings.social_auth_google_client_secret
+        if settings.social_auth_google_use_global_credentials and not (_google_client_id and _google_client_secret):
+            _google_client_id = settings.google_drive_client_id
+            _google_client_secret = settings.google_drive_client_secret
 
-if AUTH_ENABLED and settings.social_auth_dropbox_enabled:
-    if settings.social_auth_dropbox_client_id and settings.social_auth_dropbox_client_secret:
-        oauth.register(
-            name="dropbox",
-            client_id=settings.social_auth_dropbox_client_id,
-            client_secret=settings.social_auth_dropbox_client_secret,
-            authorize_url="https://www.dropbox.com/oauth2/authorize",
-            access_token_url="https://api.dropboxapi.com/oauth2/token",
-            userinfo_endpoint="https://api.dropboxapi.com/2/users/get_current_account",
-            client_kwargs={"token_endpoint_auth_method": "client_secret_post"},
-        )
-        SOCIAL_PROVIDERS["dropbox"] = {"name": "Dropbox", "icon": "fab fa-dropbox", "color": "blue"}
-        logger.info("Social login provider registered: Dropbox")
-    else:
-        logger.warning("SOCIAL_AUTH_DROPBOX_ENABLED=true but client ID/secret not configured")
+        if _google_client_id and _google_client_secret:
+            _register_oauth_client(
+                "google",
+                client_id=_google_client_id,
+                client_secret=_google_client_secret,
+                server_metadata_url="https://accounts.google.com/.well-known/openid-configuration",
+                client_kwargs={"scope": "openid profile email"},
+            )
+            SOCIAL_PROVIDERS["google"] = {"name": "Google", "icon": "fab fa-google", "color": "red"}
+            logger.info("Social login provider registered: Google")
+        else:
+            logger.warning("SOCIAL_AUTH_GOOGLE_ENABLED=true but client ID/secret not configured")
+
+    # Microsoft
+    if settings.social_auth_microsoft_enabled:
+        _microsoft_client_id = settings.social_auth_microsoft_client_id
+        _microsoft_client_secret = settings.social_auth_microsoft_client_secret
+        if settings.social_auth_microsoft_use_global_credentials and not (
+            _microsoft_client_id and _microsoft_client_secret
+        ):
+            _microsoft_client_id = settings.onedrive_client_id
+            _microsoft_client_secret = settings.onedrive_client_secret
+
+        if _microsoft_client_id and _microsoft_client_secret:
+            tenant = settings.social_auth_microsoft_tenant or "common"
+            _register_oauth_client(
+                "microsoft",
+                client_id=_microsoft_client_id,
+                client_secret=_microsoft_client_secret,
+                server_metadata_url=f"https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration",
+                client_kwargs={"scope": "openid profile email"},
+            )
+            SOCIAL_PROVIDERS["microsoft"] = {"name": "Microsoft", "icon": "fab fa-microsoft", "color": "blue"}
+            logger.info("Social login provider registered: Microsoft (tenant=%s)", tenant)
+        else:
+            logger.warning("SOCIAL_AUTH_MICROSOFT_ENABLED=true but client ID/secret not configured")
+
+    # Apple
+    if settings.social_auth_apple_enabled:
+        if settings.social_auth_apple_client_id and settings.social_auth_apple_team_id:
+            _register_oauth_client(
+                "apple",
+                client_id=settings.social_auth_apple_client_id,
+                server_metadata_url="https://appleid.apple.com/.well-known/openid-configuration",
+                client_kwargs={
+                    "scope": "openid name email",
+                    "response_mode": "form_post",
+                },
+            )
+            SOCIAL_PROVIDERS["apple"] = {"name": "Apple", "icon": "fab fa-apple", "color": "gray"}
+            logger.info("Social login provider registered: Apple")
+        else:
+            logger.warning("SOCIAL_AUTH_APPLE_ENABLED=true but client ID/team ID not configured")
+
+    # Dropbox
+    if settings.social_auth_dropbox_enabled:
+        _dropbox_client_id = settings.social_auth_dropbox_client_id
+        _dropbox_client_secret = settings.social_auth_dropbox_client_secret
+        if settings.social_auth_dropbox_use_global_credentials and not (_dropbox_client_id and _dropbox_client_secret):
+            _dropbox_client_id = settings.dropbox_app_key
+            _dropbox_client_secret = settings.dropbox_app_secret
+
+        if _dropbox_client_id and _dropbox_client_secret:
+            _register_oauth_client(
+                "dropbox",
+                client_id=_dropbox_client_id,
+                client_secret=_dropbox_client_secret,
+                authorize_url="https://www.dropbox.com/oauth2/authorize",
+                access_token_url="https://api.dropboxapi.com/oauth2/token",
+                userinfo_endpoint="https://api.dropboxapi.com/2/users/get_current_account",
+                userinfo_compliance_fix=_dropbox_userinfo_compliance_fix,
+                client_kwargs={
+                    "token_endpoint_auth_method": "client_secret_post",
+                    "token_access_type": "offline",
+                },
+            )
+            SOCIAL_PROVIDERS["dropbox"] = {"name": "Dropbox", "icon": "fab fa-dropbox", "color": "blue"}
+            logger.info("Social login provider registered: Dropbox")
+        else:
+            logger.warning("SOCIAL_AUTH_DROPBOX_ENABLED=true but client ID/secret not configured")
+
+    # GitHub
+    if settings.social_auth_github_enabled:
+        if settings.social_auth_github_client_id and settings.social_auth_github_client_secret:
+            _register_oauth_client(
+                "github",
+                client_id=settings.social_auth_github_client_id,
+                client_secret=settings.social_auth_github_client_secret,
+                authorize_url="https://github.com/login/oauth/authorize",
+                access_token_url="https://github.com/login/oauth/access_token",
+                userinfo_endpoint="https://api.github.com/user",
+                client_kwargs={"scope": "read:user user:email"},
+            )
+            SOCIAL_PROVIDERS["github"] = {"name": "GitHub", "icon": "fab fa-github", "color": "gray"}
+            logger.info("Social login provider registered: GitHub")
+        else:
+            logger.warning("SOCIAL_AUTH_GITHUB_ENABLED=true but client ID/secret not configured")
+
+    # Keycloak
+    if settings.social_auth_keycloak_enabled:
+        _kc_server = settings.social_auth_keycloak_server_url
+        _kc_realm = settings.social_auth_keycloak_realm
+        if (
+            settings.social_auth_keycloak_client_id
+            and settings.social_auth_keycloak_client_secret
+            and _kc_server
+            and _kc_realm
+        ):
+            _kc_base = f"{_kc_server.rstrip('/')}/realms/{_kc_realm}"
+            _register_oauth_client(
+                "keycloak",
+                client_id=settings.social_auth_keycloak_client_id,
+                client_secret=settings.social_auth_keycloak_client_secret,
+                server_metadata_url=f"{_kc_base}/.well-known/openid-configuration",
+                client_kwargs={"scope": "openid profile email"},
+            )
+            SOCIAL_PROVIDERS["keycloak"] = {"name": "Keycloak", "icon": "fas fa-key", "color": "gray"}
+            logger.info("Social login provider registered: Keycloak (realm=%s)", _kc_realm)
+        else:
+            logger.warning("SOCIAL_AUTH_KEYCLOAK_ENABLED=true but required settings not configured")
+
+    # Generic OAuth2
+    if settings.social_auth_generic_oauth2_enabled:
+        if (
+            settings.social_auth_generic_oauth2_client_id
+            and settings.social_auth_generic_oauth2_client_secret
+            and settings.social_auth_generic_oauth2_authorize_url
+            and settings.social_auth_generic_oauth2_token_url
+        ):
+            _register_oauth_client(
+                "generic_oauth2",
+                client_id=settings.social_auth_generic_oauth2_client_id,
+                client_secret=settings.social_auth_generic_oauth2_client_secret,
+                authorize_url=settings.social_auth_generic_oauth2_authorize_url,
+                access_token_url=settings.social_auth_generic_oauth2_token_url,
+                userinfo_endpoint=settings.social_auth_generic_oauth2_userinfo_url,
+                client_kwargs={"scope": settings.social_auth_generic_oauth2_scope},
+            )
+            _generic_name = settings.social_auth_generic_oauth2_name or "OAuth2"
+            SOCIAL_PROVIDERS["generic_oauth2"] = {
+                "name": _generic_name,
+                "icon": "fas fa-sign-in-alt",
+                "color": "indigo",
+            }
+            logger.info("Social login provider registered: Generic OAuth2")
+        else:
+            logger.warning("SOCIAL_AUTH_GENERIC_OAUTH2_ENABLED=true but required settings not configured")
+
+
+def refresh_social_providers() -> None:
+    """Re-register all OAuth providers from the *current* settings object.
+
+    Call this after loading or reloading settings from the database so that
+    providers configured (or updated) through the admin UI take effect
+    immediately — **no application restart required**.
+
+    This function is safe to call multiple times and is idempotent.
+    """
+    logger.info("Refreshing social login provider registrations from current settings")
+    _setup_social_providers()
+
+
+# Perform the initial registration from environment / default settings at
+# import time.  The lifespan hook and settings_sync will call
+# refresh_social_providers() again after DB settings are loaded so that
+# any providers configured only in the database are also active.
+_setup_social_providers()
 
 router = APIRouter()
 
@@ -125,8 +311,36 @@ def get_current_user(request: Request):
     # Check for Bearer token auth first (API tokens)
     api_user = getattr(request.state, "api_token_user", None)
     if isinstance(api_user, dict):
+        logger.debug("[AUTH] get_current_user: resolved from API token (user_id=%s)", api_user.get("id"))
         return api_user
-    return request.session.get("user")
+    session_user = request.session.get("user")
+    if session_user:
+        # Validate server-side session if a session token is present
+        session_token = request.session.get("_session_token")
+        if session_token:
+            try:
+                from app.database import SessionLocal
+                from app.utils.session_manager import validate_session
+
+                db = SessionLocal()
+                try:
+                    valid = validate_session(db, session_token)
+                    if not valid:
+                        logger.debug("[AUTH] get_current_user: server-side session invalid — clearing")
+                        request.session.pop("user", None)
+                        request.session.pop("_session_token", None)
+                        return None
+                finally:
+                    db.close()
+            except Exception:
+                logger.debug("[AUTH] get_current_user: session validation error", exc_info=True)
+        logger.debug(
+            "[AUTH] get_current_user: resolved from session (user=%s)",
+            session_user.get("preferred_username") or session_user.get("email") or session_user.get("id"),
+        )
+    else:
+        logger.debug("[AUTH] get_current_user: no user in session or API token")
+    return session_user
 
 
 def _resolve_bearer_user(request: Request, db: Session) -> dict | None:
@@ -141,10 +355,12 @@ def _resolve_bearer_user(request: Request, db: Session) -> dict | None:
     """
     auth_header = request.headers.get("authorization", "")
     if not isinstance(auth_header, str) or not auth_header.startswith("Bearer "):
+        logger.debug("[AUTH] _resolve_bearer_user: no Bearer token in Authorization header")
         return None
 
     raw_token = auth_header[7:]
     if not raw_token or not isinstance(raw_token, str):
+        logger.debug("[AUTH] _resolve_bearer_user: empty or invalid token after 'Bearer ' prefix")
         return None
 
     from app.api.api_tokens import hash_token
@@ -153,8 +369,25 @@ def _resolve_bearer_user(request: Request, db: Session) -> dict | None:
     token_hash = hash_token(raw_token)
     db_token = db.query(ApiToken).filter(ApiToken.token_hash == token_hash, ApiToken.is_active.is_(True)).first()
     if db_token is None:
+        logger.debug("[AUTH] _resolve_bearer_user: no active API token matched the provided hash")
         return None
 
+    # Reject tokens that have passed their optional expiry.
+    if db_token.expires_at is not None:
+        now_utc = datetime.now(timezone.utc)
+        expires_aware = db_token.expires_at
+        if expires_aware.tzinfo is None:
+            expires_aware = expires_aware.replace(tzinfo=timezone.utc)
+        if now_utc > expires_aware:
+            logger.debug("[AUTH] _resolve_bearer_user: API token id=%s has expired", db_token.id)
+            return None
+
+    logger.debug(
+        "[AUTH] _resolve_bearer_user: matched API token id=%s owner=%s",
+        db_token.id,
+        db_token.owner_id,
+    )
+
     # Update usage tracking
     try:
         db_token.last_used_at = datetime.now(timezone.utc)
@@ -205,16 +438,18 @@ def require_login(func):
 
     @wraps(func)
     async def wrapper(request: Request, *args, **kwargs):
+        url_path = urlparse(str(request.url)).path
         # Check session auth first
         if request.session.get("user"):
+            logger.debug("[AUTH] require_login: session auth OK for %s", url_path)
             if inspect.iscoroutinefunction(func):
                 return await func(*args, request=request, **kwargs)
             else:
                 return func(*args, request=request, **kwargs)
 
         # Fall back to Bearer token auth for API endpoints
-        url_path = urlparse(str(request.url)).path
         if url_path.startswith("/api/"):
+            logger.debug("[AUTH] require_login: no session, trying Bearer token for %s", url_path)
             try:
                 from app.database import SessionLocal
 
@@ -228,17 +463,22 @@ def require_login(func):
 
             if api_user:
                 request.state.api_token_user = api_user
+                logger.debug(
+                    "[AUTH] require_login: Bearer token auth OK for %s (user=%s)", url_path, api_user.get("id")
+                )
                 if inspect.iscoroutinefunction(func):
                     return await func(*args, request=request, **kwargs)
                 else:
                     return func(*args, request=request, **kwargs)
 
+            logger.debug("[AUTH] require_login: no valid auth for API endpoint %s — returning 401", url_path)
             return JSONResponse(
                 status_code=status.HTTP_401_UNAUTHORIZED,
                 content={"error": "Not authenticated"},
             )
 
         # Non-API endpoint with no session — redirect to login
+        logger.debug("[AUTH] require_login: no session for %s — redirecting to /login", url_path)
         request.session["redirect_after_login"] = str(request.url)
         return RedirectResponse(url="/login", status_code=status.HTTP_302_FOUND)
 
@@ -287,13 +527,21 @@ async def login(request: Request):
             get_client_ip(request),
         )
 
+    error = request.query_params.get("error")
+    message = request.query_params.get("message")
+    show_oauth = OAUTH_CONFIGURED
+
+    # SSO Auto Login: redirect directly to SSO provider if configured
+    if show_oauth and settings.sso_auto_login is True and not error and not message:
+        return RedirectResponse(url="/oauth-login", status_code=status.HTTP_302_FOUND)
+
     return templates.TemplateResponse(
+        request,
         "login.html",
-        {
-            "request": request,
-            "error": request.query_params.get("error"),
-            "message": request.query_params.get("message"),
-            "show_oauth": OAUTH_CONFIGURED,
+        context={
+            "error": error,
+            "message": message,
+            "show_oauth": show_oauth,
             "oauth_provider_name": OAUTH_PROVIDER_NAME,
             "social_providers": SOCIAL_PROVIDERS,
             "app_version": settings.version,
@@ -307,9 +555,15 @@ async def login(request: Request):
 async def oauth_login(request: Request):
     """Handle OAuth login flow"""
     if not OAUTH_CONFIGURED:
+        logger.debug("[AUTH] oauth_login: OAuth not configured — redirecting to /login")
         return RedirectResponse(url="/login?error=OAuth+not+configured", status_code=status.HTTP_302_FOUND)
 
     redirect_uri = request.url_for("oauth_callback")
+    logger.debug(
+        "[AUTH] oauth_login: initiating Authentik OAuth redirect_uri=%s session_keys=%s",
+        redirect_uri,
+        list(request.session.keys()),
+    )
     return await oauth.authentik.authorize_redirect(request, redirect_uri)
 
 
@@ -324,13 +578,23 @@ async def social_login(request: Request, provider: str):
         A redirect to the provider's authorization page, or back to /login on error.
     """
     if provider not in SOCIAL_PROVIDERS:
+        logger.debug(
+            "[AUTH] social_login: unknown provider=%r (registered=%s)", provider, list(SOCIAL_PROVIDERS.keys())
+        )
         return RedirectResponse(url="/login?error=Unknown+social+provider", status_code=status.HTTP_302_FOUND)
 
     redirect_uri = request.url_for("social_callback", provider=provider)
     oauth_client = getattr(oauth, provider, None)
     if oauth_client is None:
+        logger.debug("[AUTH] social_login: provider=%r registered but OAuth client not configured", provider)
         return RedirectResponse(url="/login?error=Provider+not+configured", status_code=status.HTTP_302_FOUND)
 
+    logger.debug(
+        "[AUTH] social_login: initiating %s OAuth, redirect_uri=%s session_keys=%s",
+        provider,
+        redirect_uri,
+        list(request.session.keys()),
+    )
     return await oauth_client.authorize_redirect(request, redirect_uri)
 
 
@@ -363,6 +627,17 @@ def _normalize_social_userinfo(provider: str, token: dict, raw_userinfo: dict |
             "picture": userinfo.get("profile_photo_url", ""),
         }
 
+    if provider == "github":
+        # GitHub returns login, id, name, email, avatar_url
+        email = userinfo.get("email", "")
+        return {
+            "sub": str(userinfo.get("id", "")),
+            "email": email,
+            "name": userinfo.get("name", "") or userinfo.get("login", ""),
+            "preferred_username": userinfo.get("login", email),
+            "picture": userinfo.get("avatar_url", ""),
+        }
+
     # Standard OIDC providers (Google, Microsoft, Apple)
     return {
         "sub": userinfo.get("sub", ""),
@@ -389,27 +664,39 @@ async def social_callback(request: Request, provider: str, db: Session = Depends
         A redirect to the user's original destination or the upload page.
     """
     if provider not in SOCIAL_PROVIDERS:
+        logger.debug("[AUTH] social_callback: unknown provider=%r", provider)
         return RedirectResponse(url="/login?error=Unknown+social+provider", status_code=status.HTTP_302_FOUND)
 
     oauth_client = getattr(oauth, provider, None)
     if oauth_client is None:
+        logger.debug("[AUTH] social_callback: provider=%r not configured", provider)
         return RedirectResponse(url="/login?error=Provider+not+configured", status_code=status.HTTP_302_FOUND)
 
     try:
+        logger.debug("[AUTH] social_callback: exchanging auth code for provider=%s", provider)
         token = await oauth_client.authorize_access_token(request)
 
         # Try standard OIDC userinfo first, fall back to token-embedded userinfo
         raw_userinfo = token.get("userinfo")
         if not raw_userinfo:
+            logger.debug("[AUTH] social_callback: no userinfo in token, fetching from userinfo endpoint")
             try:
                 resp = await oauth_client.userinfo(token=token)
                 raw_userinfo = resp if isinstance(resp, dict) else resp.json() if hasattr(resp, "json") else {}
             except Exception:
+                logger.debug("[AUTH] social_callback: userinfo endpoint failed, using empty dict", exc_info=True)
                 raw_userinfo = {}
 
         user_data = _normalize_social_userinfo(provider, token, raw_userinfo)
+        logger.debug(
+            "[AUTH] social_callback: normalized user_data email=%s sub=%s provider=%s",
+            user_data.get("email"),
+            user_data.get("sub"),
+            provider,
+        )
 
         if not user_data.get("email"):
+            logger.debug("[AUTH] social_callback: no email in user_data — aborting")
             return RedirectResponse(
                 url="/login?error=Could+not+retrieve+email+from+provider",
                 status_code=status.HTTP_302_FOUND,
@@ -428,6 +715,27 @@ async def social_callback(request: Request, provider: str, db: Session = Depends
 
         request.session["user"] = user_data
 
+        # Create server-side session for tracking and revocation
+        try:
+            from app.utils.session_manager import create_session
+
+            _session_user_id = (
+                user_data.get("sub")
+                or user_data.get("preferred_username")
+                or user_data.get("email")
+                or user_data.get("id")
+            )
+            if _session_user_id:
+                user_session = create_session(
+                    db,
+                    user_id=_session_user_id,
+                    ip_address=get_client_ip(request),
+                    user_agent=request.headers.get("user-agent"),
+                )
+                request.session["_session_token"] = user_session.session_token
+        except Exception:
+            logger.debug("[AUTH] Failed to create server-side session for social user", exc_info=True)
+
         # Auto-create or update UserProfile
         _ensure_user_profile(db, user_data, is_admin=False)
 
@@ -442,21 +750,29 @@ async def social_callback(request: Request, provider: str, db: Session = Depends
         )
 
         # Mobile app flow: issue an inline API token and redirect back to the app.
+        logger.debug(
+            "[MOBILE] social_callback: checking for mobile redirect (session has mobile_redirect_uri=%s)",
+            "mobile_redirect_uri" in request.session,
+        )
         mobile_resp = _create_mobile_redirect(request, db)
         if mobile_resp:
+            logger.info("[MOBILE] social_callback: returning mobile redirect response for provider=%s", provider)
             return mobile_resp
 
         if user_id:
             profile = db.query(_UserProfile).filter(_UserProfile.user_id == user_id).first()
             if profile and not profile.onboarding_completed:
+                logger.debug("[AUTH] social_callback: user=%s needs onboarding, redirecting", user_id)
                 post_onboarding = request.session.pop("redirect_after_login", "/upload")
                 request.session["post_onboarding_redirect"] = post_onboarding
                 return RedirectResponse(url="/onboarding", status_code=status.HTTP_302_FOUND)
 
         redirect_url = request.session.pop("redirect_after_login", "/upload")
+        logger.debug("[AUTH] social_callback: login complete, redirecting to %s", redirect_url)
         return RedirectResponse(url=redirect_url, status_code=status.HTTP_302_FOUND)
     except Exception as e:
         logger.warning("[SECURITY] SOCIAL_LOGIN_FAILURE provider=%s error=%s", provider, type(e).__name__)
+        logger.debug("[AUTH] social_callback: full exception for provider=%s", provider, exc_info=True)
         return RedirectResponse(
             url="/login?error=Social+login+failed.+Please+try+again.", status_code=status.HTTP_302_FOUND
         )
@@ -561,15 +877,23 @@ def _ensure_user_profile(db: Session, user_data: dict, is_admin: bool = False) -
 async def oauth_callback(request: Request, db: Session = Depends(get_db)):
     """Handle OAuth callback from provider"""
     try:
+        logger.debug("[AUTH] oauth_callback: exchanging authorization code for token")
         token = await oauth.authentik.authorize_access_token(request)
         userinfo = token.get("userinfo")
         if not userinfo:
+            logger.debug("[AUTH] oauth_callback: no userinfo in token response — aborting")
             return RedirectResponse(
                 url="/login?error=Failed+to+retrieve+user+information", status_code=status.HTTP_302_FOUND
             )
 
         # Store user info in session
         user_data = dict(userinfo)
+        logger.debug(
+            "[AUTH] oauth_callback: received userinfo email=%s sub=%s groups=%s",
+            user_data.get("email"),
+            user_data.get("sub"),
+            user_data.get("groups", []),
+        )
 
         # Add Gravatar picture if no picture is provided
         if not user_data.get("picture") and user_data.get("email"):
@@ -584,12 +908,39 @@ async def oauth_callback(request: Request, db: Session = Depends(get_db)):
             groups = user_data.get("groups", [])
             admin_group = (settings.admin_group_name or "admin").strip().lower()
             is_admin = admin_group in [group.lower() for group in groups]
+            logger.debug(
+                "[AUTH] oauth_callback: admin group check — looking for %r in %s → is_admin=%s",
+                admin_group,
+                [g.lower() for g in groups],
+                is_admin,
+            )
 
         # Set is_admin flag (defaults to False for OAuth users unless they're in admin group)
         user_data["is_admin"] = is_admin
 
         request.session["user"] = user_data
 
+        # Create server-side session for tracking and revocation
+        try:
+            from app.utils.session_manager import create_session
+
+            _session_user_id = (
+                user_data.get("sub")
+                or user_data.get("preferred_username")
+                or user_data.get("email")
+                or user_data.get("id")
+            )
+            if _session_user_id:
+                user_session = create_session(
+                    db,
+                    user_id=_session_user_id,
+                    ip_address=get_client_ip(request),
+                    user_agent=request.headers.get("user-agent"),
+                )
+                request.session["_session_token"] = user_session.session_token
+        except Exception:
+            logger.debug("[AUTH] Failed to create server-side session for OAuth user", exc_info=True)
+
         # Auto-create or update UserProfile so the user appears in admin user management
         _ensure_user_profile(db, user_data, is_admin=is_admin)
 
@@ -623,15 +974,18 @@ async def oauth_callback(request: Request, db: Session = Depends(get_db)):
         if user_id:
             profile = db.query(_UserProfile).filter(_UserProfile.user_id == user_id).first()
             if profile and not profile.onboarding_completed:
+                logger.debug("[AUTH] oauth_callback: user=%s needs onboarding, redirecting", user_id)
                 post_onboarding = request.session.pop("redirect_after_login", "/upload")
                 request.session["post_onboarding_redirect"] = post_onboarding
                 return RedirectResponse(url="/onboarding", status_code=status.HTTP_302_FOUND)
 
         # Redirect to original destination or default
         redirect_url = request.session.pop("redirect_after_login", "/upload")
+        logger.debug("[AUTH] oauth_callback: login complete, redirecting to %s", redirect_url)
         return RedirectResponse(url=redirect_url, status_code=status.HTTP_302_FOUND)
     except Exception as e:
         logger.warning(f"[SECURITY] OAUTH_LOGIN_FAILURE error={type(e).__name__}")
+        logger.debug("[AUTH] oauth_callback: full exception details", exc_info=True)
         return RedirectResponse(url=f"/login?error=Authentication+failed:+{str(e)}", status_code=status.HTTP_302_FOUND)
 
 
@@ -840,6 +1194,19 @@ async def auth(request: Request, db: Session = Depends(get_db)):
                 return RedirectResponse(url="/login?error=Invalid+username+or+password", status_code=302)
             user_data = _build_session_user(local_user)
             request.session["user"] = user_data
+            # Create server-side session for tracking and revocation
+            try:
+                from app.utils.session_manager import create_session
+
+                user_session = create_session(
+                    db,
+                    user_id=local_user.email,
+                    ip_address=get_client_ip(request),
+                    user_agent=request.headers.get("user-agent"),
+                )
+                request.session["_session_token"] = user_session.session_token
+            except Exception:
+                logger.debug("[AUTH] Failed to create server-side session", exc_info=True)
             logger.info("[SECURITY] LOCAL_LOGIN_SUCCESS user=%s", local_user.email)
             _record_login_event(db, request, local_user.email, success=True)
             _ensure_user_profile(db, user_data, is_admin=bool(local_user.is_admin))
@@ -896,6 +1263,20 @@ async def auth(request: Request, db: Session = Depends(get_db)):
             "is_admin": True,
         }
         request.session["user"] = admin_user_data
+        # Create server-side session for tracking and revocation
+        try:
+            from app.utils.session_manager import create_session
+
+            admin_user_id = settings.admin_username or "admin"
+            user_session = create_session(
+                db,
+                user_id=admin_user_id,
+                ip_address=get_client_ip(request),
+                user_agent=request.headers.get("user-agent"),
+            )
+            request.session["_session_token"] = user_session.session_token
+        except Exception:
+            logger.debug("[AUTH] Failed to create server-side session for admin", exc_info=True)
         logger.info("[SECURITY] LOCAL_LOGIN_SUCCESS user=%s", username)
         _record_login_event(db, request, username, success=True)
         _ensure_user_profile(db, admin_user_data, is_admin=True)
@@ -931,6 +1312,7 @@ async def logout(request: Request, db: Session = Depends(get_db)):
     username = "unknown"
     if isinstance(user, dict):
         username = user.get("preferred_username") or user.get("email") or "unknown"
+    logger.debug("[AUTH] logout: clearing session for user=%s client_ip=%s", username, get_client_ip(request))
     logger.info(f"[SECURITY] LOGOUT user={username}")
     try:
         from app.utils.audit_service import record_event
@@ -945,6 +1327,20 @@ async def logout(request: Request, db: Session = Depends(get_db)):
         )
     except Exception:
         logger.debug("Failed to write logout audit event for user=%s", username, exc_info=True)
+    # Revoke server-side session
+    session_token = request.session.get("_session_token")
+    if session_token:
+        try:
+            from app.utils.session_manager import validate_session
+
+            user_session = validate_session(db, session_token)
+            if user_session:
+                user_session.is_revoked = True
+                user_session.revoked_at = datetime.now(timezone.utc)
+                db.commit()
+        except Exception:
+            logger.debug("[AUTH] Failed to revoke server-side session", exc_info=True)
+    request.session.pop("_session_token", None)
     request.session.pop("user", None)
     return RedirectResponse(url="/login?message=You+have+been+logged+out+successfully", status_code=302)
 

app/celery_app.py

@@ -1,10 +1,15 @@
 # app/celery_app.py
 
+import logging
+import os
+
 from celery import Celery
 from celery.signals import task_failure, worker_ready
 
 from app.config import settings
 
+logger = logging.getLogger(__name__)
+
 celery = Celery(
     "document_processor",
     broker=settings.redis_url,
@@ -21,6 +26,64 @@ celery.conf.task_routes = {
     "app.tasks.*": {"queue": "document_processor"},
 }
 
+# Mapping of document pipeline task names to the positional index of ``file_id``
+# in their ``args`` tuple.  These indices correspond to the task signatures:
+#   process_with_ocr(filename, file_id, ...)          → index 1
+#   extract_metadata_with_gpt(filename, text, file_id) → index 2
+#   embed_metadata_into_pdf(path, text, metadata, file_id) → index 3
+# Tasks that always pass ``file_id`` as a keyword argument
+# (e.g. ``process_document``, ``finalize_document_storage``) are not listed
+# here — their ``file_id`` is found via ``kwargs`` instead.
+_FILE_ID_ARG_INDEX: dict[str, int] = {
+    "app.tasks.process_with_ocr.process_with_ocr": 1,
+    "app.tasks.extract_metadata_with_gpt.extract_metadata_with_gpt": 2,
+    "app.tasks.embed_metadata_into_pdf.embed_metadata_into_pdf": 3,
+}
+
+
+def _dispatch_user_failure_notification(sender, exception, args: list | None, kwargs: dict | None) -> None:
+    """Best-effort per-user failure notification for document pipeline tasks.
+
+    Extracts ``file_id`` from the failed task's arguments, looks up the owning
+    user from the database, and dispatches a ``document.failed`` notification.
+    """
+    from app.database import SessionLocal
+    from app.models import FileRecord
+    from app.utils.user_notification import notify_user_document_failed
+
+    task_name = sender.name if sender else ""
+    if not task_name.startswith("app.tasks."):
+        return
+
+    # 1. Resolve file_id from kwargs or positional args
+    file_id = (kwargs or {}).get("file_id")
+    if file_id is None:
+        idx = _FILE_ID_ARG_INDEX.get(task_name)
+        if idx is not None and args and len(args) > idx:
+            val = args[idx]
+            if isinstance(val, int):
+                file_id = val
+
+    if file_id is None:
+        return
+
+    # 2. Look up owner from the database
+    with SessionLocal() as db:
+        record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+        if not record or not record.owner_id:
+            return
+        owner_id = record.owner_id
+        filename = record.original_filename or record.local_filename or "unknown"
+
+    # 3. Dispatch per-user notification
+    error_msg = f"{type(exception).__name__}: {exception}" if exception else "Unknown error"
+    notify_user_document_failed(
+        owner_id=owner_id,
+        filename=os.path.basename(filename),
+        error=error_msg,
+        file_id=file_id,
+    )
+
 
 @worker_ready.connect
 def init_sentry_on_worker_ready(**kwargs):
@@ -48,6 +111,10 @@ def task_failure_handler(
                 kwargs=kwargs or {},
             )
         except Exception as e:
-            import logging
+            logger.exception(f"Failed to send task failure notification: {e}")
 
-            logging.exception(f"Failed to send task failure notification: {e}")
+    # Also dispatch a per-user failure notification for document pipeline tasks
+    try:
+        _dispatch_user_failure_notification(sender, exception, args, kwargs)
+    except Exception:
+        logger.warning("Could not dispatch per-user failure notification", exc_info=True)

app/celery_worker.py

@@ -10,6 +10,7 @@ from app import tasks  # noqa: F401 - Imports app/tasks.py so Celery can registe
 # Import the shared Celery instance
 from app.celery_app import celery
 from app.config import settings
+from app.tasks.automation_tasks import deliver_automation_hook_task  # noqa: F401
 from app.tasks.backup_tasks import cleanup_old_backups, create_backup  # noqa: F401
 from app.tasks.batch_tasks import (  # noqa: F401
     backfill_missing_metadata,
@@ -22,6 +23,7 @@ from app.tasks.batch_tasks import (  # noqa: F401
     sync_search_index,
 )
 from app.tasks.check_credentials import check_credentials
+from app.tasks.classify_document import classify_document_task  # noqa: F401
 from app.tasks.compute_embedding import backfill_missing_embeddings, compute_document_embedding  # noqa: F401
 from app.tasks.convert_to_pdf import convert_to_pdf  # noqa: F401
 from app.tasks.convert_to_pdfa import convert_to_pdfa  # noqa: F401
@@ -39,6 +41,7 @@ from app.tasks.refine_text_with_gpt import refine_text_with_gpt  # noqa: F401
 from app.tasks.rotate_pdf_pages import rotate_pdf_pages  # noqa: F401
 from app.tasks.send_to_all import send_to_all_destinations  # noqa: F401
 from app.tasks.subscription_tasks import apply_pending_subscription_changes_all  # noqa: F401
+from app.tasks.translate_to_default_language import translate_to_default_language  # noqa: F401
 
 # Import new send tasks
 from app.tasks.upload_to_dropbox import upload_to_dropbox  # noqa: F401
@@ -51,6 +54,7 @@ from app.tasks.upload_to_onedrive import upload_to_onedrive  # noqa: F401
 from app.tasks.upload_to_paperless import upload_to_paperless  # noqa: F401
 from app.tasks.upload_to_s3 import upload_to_s3  # noqa: F401
 from app.tasks.upload_to_sftp import upload_to_sftp  # noqa: F401
+from app.tasks.upload_to_sharepoint import upload_to_sharepoint  # noqa: F401
 from app.tasks.upload_to_user_integration import upload_to_user_integration  # noqa: F401
 from app.tasks.upload_to_webdav import upload_to_webdav  # noqa: F401
 from app.tasks.upload_with_rclone import send_to_all_rclone_destinations, upload_with_rclone  # noqa: F401

app/config.py

@@ -13,6 +13,24 @@ class Settings(BaseSettings):
 
     database_url: str
     redis_url: str
+
+    # Database connection-pool tuning (ignored for SQLite, which uses NullPool).
+    db_pool_size: int = Field(
+        default=10,
+        description="Number of persistent connections kept in the pool per worker process.",
+    )
+    db_max_overflow: int = Field(
+        default=20,
+        description="Additional connections allowed beyond db_pool_size under burst load.",
+    )
+    db_pool_timeout: int = Field(
+        default=30,
+        description="Seconds to wait for a connection from the pool before raising a TimeoutError.",
+    )
+    db_pool_recycle: int = Field(
+        default=1800,
+        description="Recycle (close and reopen) connections after this many seconds to avoid stale connections.",
+    )
     openai_api_key: str
     openai_base_url: str = "https://api.openai.com/v1"  # Default to OpenAI's endpoint
     openai_model: str = "gpt-4o-mini"  # Default model
@@ -48,6 +66,51 @@ class Settings(BaseSettings):
     workdir: str
     debug: bool = False  # Default to False
 
+    # Logging level for the application.  Accepts standard Python level names:
+    # DEBUG, INFO, WARNING, ERROR, CRITICAL.  When *debug* is True and
+    # *log_level* has not been explicitly set, the effective level is forced to
+    # DEBUG so that all ``logger.debug()`` calls produce output.
+    log_level: str = Field(
+        default="INFO",
+        description=(
+            "Python logging level for the application root logger.  "
+            "Accepts: DEBUG, INFO, WARNING, ERROR, CRITICAL.  "
+            "When DEBUG=True and LOG_LEVEL is not explicitly set, "
+            "the effective level is automatically lowered to DEBUG."
+        ),
+    )
+
+    # Log output format.  ``text`` is the human-readable default.
+    # ``json`` emits one JSON object per line, ideal for log collectors
+    # (Promtail, Fluentd, Filebeat, Datadog agent) and SIEM ingestion.
+    log_format: str = Field(
+        default="text",
+        description=(
+            "Log output format: 'text' (human-readable, default) or "
+            "'json' (structured JSON lines for SIEM / log aggregation)."
+        ),
+    )
+
+    # Optional syslog forwarding for application logs (not just audit events).
+    # When enabled, a Python SysLogHandler is added to the root logger so that
+    # every log message is also sent to the configured syslog receiver.
+    log_syslog_enabled: bool = Field(
+        default=False,
+        description="Forward application logs to a syslog receiver in addition to stdout.",
+    )
+    log_syslog_host: str = Field(
+        default="localhost",
+        description="Hostname or IP of the syslog receiver for application logs.",
+    )
+    log_syslog_port: int = Field(
+        default=514,
+        description="Port of the syslog receiver for application logs.",
+    )
+    log_syslog_protocol: str = Field(
+        default="udp",
+        description="Protocol for syslog transport: 'udp' or 'tcp'.",
+    )
+
     # Making Dropbox optional
     dropbox_enabled: bool = Field(
         default=True,
@@ -57,6 +120,16 @@ class Settings(BaseSettings):
     dropbox_app_secret: Optional[str] = None
     dropbox_folder: Optional[str] = None
     dropbox_refresh_token: Optional[str] = None
+    dropbox_allow_global_credentials_for_integrations: bool = Field(
+        default=False,
+        description=(
+            "When True, users may authorize their personal Dropbox integrations using the global "
+            "DROPBOX_APP_KEY / DROPBOX_APP_SECRET credentials configured by the admin, without "
+            "needing to create their own Dropbox app. The Dropbox OAuth flow is initiated "
+            "server-side so the app secret is never exposed to the browser. "
+            "Default: False (each user must supply their own app credentials)."
+        ),
+    )
 
     # Making Nextcloud optional
     nextcloud_enabled: bool = Field(
@@ -120,12 +193,65 @@ class Settings(BaseSettings):
     google_docai_processor_id: Optional[str] = None
     google_docai_location: str = "us"  # Processor location, e.g. "us" or "eu"
     external_hostname: str = "localhost"  # Default to localhost
+    public_base_url: Optional[str] = Field(
+        default=None,
+        description=(
+            "The full public base URL of the application, including scheme "
+            "(e.g., 'https://docuelevate.example.com'). "
+            "When set, this overrides the auto-detected URL for OAuth redirect URIs. "
+            "This is required when the application is behind a reverse proxy that does "
+            "not forward X-Forwarded-Proto headers correctly."
+        ),
+    )
+
+    # ---------------------------------------------------------------------------
+    # Document Translation Settings
+    # ---------------------------------------------------------------------------
+    # Default target language for automatic document translation (ISO 639-1 code).
+    # After OCR / metadata extraction, if the detected document language differs
+    # from this value the system translates the extracted text into this language
+    # and stores it alongside the original.  Other language translations are
+    # generated on the fly via the AI provider and are NOT persisted.
+    # Per-user overrides are stored in UserProfile.default_document_language.
+    default_document_language: str = Field(
+        default="en",
+        description=(
+            "ISO 639-1 language code for the default translation target "
+            "(e.g. 'en', 'de', 'fr'). Documents whose detected language "
+            "differs are automatically translated into this language after "
+            "processing. Default: 'en' (English)."
+        ),
+    )
 
     # Authentication settings
     auth_enabled: bool = True  # Default to enabled
     admin_username: Optional[str] = None
     admin_password: Optional[str] = None
     session_secret: Optional[str] = None
+    session_lifetime_days: int = Field(
+        default=30,
+        description=(
+            "Session lifetime in days.  Common values: 30, 60, 90.  "
+            "Determines how long a user stays logged in before being required to re-authenticate.  "
+            "Applies to both browser sessions and the session cookie max_age."
+        ),
+    )
+    session_lifetime_custom_days: int | None = Field(
+        default=None,
+        description=(
+            "Override session_lifetime_days with a custom value.  "
+            "When set, this takes precedence over session_lifetime_days.  "
+            "Useful for admin-configured non-standard durations."
+        ),
+    )
+    qr_login_enabled: bool = Field(
+        default=True,
+        description="Enable QR code-based login for mobile device authentication (default: True).",
+    )
+    qr_login_challenge_ttl_seconds: int = Field(
+        default=120,
+        description="Time-to-live in seconds for QR login challenges (default: 2 minutes).",
+    )
     admin_group_name: str = "admin"
 
     # Multi-user settings
@@ -183,12 +309,55 @@ class Settings(BaseSettings):
     authentik_client_secret: Optional[str] = None
     authentik_config_url: Optional[str] = None
     oauth_provider_name: Optional[str] = None  # Name to display for the OAuth provider
+    sso_auto_login: bool = Field(
+        default=False,
+        description=(
+            "Automatically redirect to SSO login when authentication is required. "
+            "When enabled, users are sent directly to the SSO provider instead of "
+            "seeing the login page. Only effective when OIDC is configured."
+        ),
+    )
+
+    # Keycloak SSO
+    social_auth_keycloak_enabled: bool = False
+    social_auth_keycloak_client_id: Optional[str] = None
+    social_auth_keycloak_client_secret: Optional[str] = None
+    social_auth_keycloak_server_url: Optional[str] = None
+    social_auth_keycloak_realm: Optional[str] = None
+
+    # Generic OAuth2 SSO
+    social_auth_generic_oauth2_enabled: bool = False
+    social_auth_generic_oauth2_client_id: Optional[str] = None
+    social_auth_generic_oauth2_client_secret: Optional[str] = None
+    social_auth_generic_oauth2_authorize_url: Optional[str] = None
+    social_auth_generic_oauth2_token_url: Optional[str] = None
+    social_auth_generic_oauth2_userinfo_url: Optional[str] = None
+    social_auth_generic_oauth2_scope: str = "openid profile email"
+    social_auth_generic_oauth2_name: str = "OAuth2"
+
+    # SAML2 SSO
+    social_auth_saml2_enabled: bool = False
+    social_auth_saml2_entity_id: Optional[str] = None
+    social_auth_saml2_sso_url: Optional[str] = None
+    social_auth_saml2_certificate: Optional[str] = None
+    social_auth_saml2_name: str = "SAML2"
 
     # Social Login Providers
     # Google OAuth2
     social_auth_google_enabled: bool = False
     social_auth_google_client_id: Optional[str] = None
     social_auth_google_client_secret: Optional[str] = None
+    social_auth_google_use_global_credentials: bool = Field(
+        default=False,
+        description=(
+            "When True, Google social login uses the global GOOGLE_DRIVE_CLIENT_ID / "
+            "GOOGLE_DRIVE_CLIENT_SECRET credentials (the Google Drive OAuth integration credentials) "
+            "instead of requiring separate SOCIAL_AUTH_GOOGLE_CLIENT_ID / "
+            "SOCIAL_AUTH_GOOGLE_CLIENT_SECRET values. "
+            "Requires SOCIAL_AUTH_GOOGLE_ENABLED=True and the global Google Drive OAuth credentials to be set. "
+            "Default: False."
+        ),
+    )
 
     # Microsoft OAuth2 (Azure AD / Microsoft Entra ID)
     social_auth_microsoft_enabled: bool = False
@@ -203,6 +372,17 @@ class Settings(BaseSettings):
             "Default: common."
         ),
     )
+    social_auth_microsoft_use_global_credentials: bool = Field(
+        default=False,
+        description=(
+            "When True, Microsoft social login uses the global ONEDRIVE_CLIENT_ID / "
+            "ONEDRIVE_CLIENT_SECRET credentials (the OneDrive integration credentials) "
+            "instead of requiring separate SOCIAL_AUTH_MICROSOFT_CLIENT_ID / "
+            "SOCIAL_AUTH_MICROSOFT_CLIENT_SECRET values. "
+            "Requires SOCIAL_AUTH_MICROSOFT_ENABLED=True and the global OneDrive credentials to be set. "
+            "Default: False."
+        ),
+    )
 
     # Apple Sign-In
     social_auth_apple_enabled: bool = False
@@ -215,6 +395,21 @@ class Settings(BaseSettings):
     social_auth_dropbox_enabled: bool = False
     social_auth_dropbox_client_id: Optional[str] = None
     social_auth_dropbox_client_secret: Optional[str] = None
+    social_auth_dropbox_use_global_credentials: bool = Field(
+        default=False,
+        description=(
+            "When True, Dropbox social login uses the global DROPBOX_APP_KEY / DROPBOX_APP_SECRET "
+            "credentials (the storage integration credentials) instead of requiring separate "
+            "SOCIAL_AUTH_DROPBOX_CLIENT_ID / SOCIAL_AUTH_DROPBOX_CLIENT_SECRET values. "
+            "Requires SOCIAL_AUTH_DROPBOX_ENABLED=True and the global Dropbox app credentials to be set. "
+            "Default: False."
+        ),
+    )
+
+    # GitHub OAuth2
+    social_auth_github_enabled: bool = False
+    social_auth_github_client_id: Optional[str] = None
+    social_auth_github_client_secret: Optional[str] = None
 
     # Local user signup
     allow_local_signup: bool = Field(
@@ -525,6 +720,15 @@ class Settings(BaseSettings):
     onedrive_refresh_token: Optional[str] = None  # Required for personal accounts
     onedrive_folder_path: Optional[str] = None
 
+    # SharePoint settings
+    sharepoint_client_id: Optional[str] = None
+    sharepoint_client_secret: Optional[str] = None
+    sharepoint_tenant_id: Optional[str] = "common"
+    sharepoint_refresh_token: Optional[str] = None
+    sharepoint_site_url: Optional[str] = None  # e.g. https://tenant.sharepoint.com/sites/sitename
+    sharepoint_document_library: Optional[str] = "Documents"  # Document library name
+    sharepoint_folder_path: Optional[str] = None  # Subfolder inside the library
+
     # AWS S3 settings
     s3_enabled: bool = Field(
         default=True,
@@ -575,6 +779,25 @@ class Settings(BaseSettings):
         ),
     )
 
+    # System reset / factory reset settings
+    factory_reset_on_startup: bool = Field(
+        default=False,
+        description=(
+            "When enabled, DocuElevate wipes all user data (database rows and "
+            "work-files on disk) on every startup so the instance always comes "
+            "up in a clean, fresh state.  Useful for demo or testing environments. "
+            "Default: False."
+        ),
+    )
+    enable_factory_reset: bool = Field(
+        default=False,
+        description=(
+            "Show the 'System Reset' page in the admin UI.  When enabled, "
+            "administrators can trigger a full data wipe or a wipe-and-reimport "
+            "directly from the web interface.  Default: False."
+        ),
+    )
+
     # PDF/A archival conversion settings
     enable_pdfa_conversion: bool = Field(
         default=False,
@@ -684,6 +907,11 @@ class Settings(BaseSettings):
         ),
     )
 
+    # Telegram Bot
+    telegram_bot_token: Optional[str] = None
+    telegram_chat_id: Optional[str] = None
+    telegram_enabled: bool = False
+
     # Notification settings
     notification_urls: Union[List[str], str] = Field(
         default_factory=list,
@@ -718,6 +946,12 @@ class Settings(BaseSettings):
         description="Enable webhook delivery for document events",
     )
 
+    # Automation hooks (Zapier / Make.com)
+    automation_hooks_enabled: bool = Field(
+        default=True,
+        description="Enable Zapier / Make.com automation hook subscriptions and delivery",
+    )
+
     # ── Backup / restore settings ──────────────────────────────────────────────
     backup_enabled: bool = Field(
         default=True,
@@ -1003,6 +1237,20 @@ class Settings(BaseSettings):
         ),
     )
 
+    # Per-user upload rate limiting (health-aware, Redis-backed sliding window)
+    upload_rate_limit_per_user: int = Field(
+        default=20,
+        description=(
+            "Maximum number of file uploads allowed per user within the sliding window. "
+            "The effective limit may be reduced dynamically when the system is under heavy load "
+            "(high queue depth or CPU usage). Set to 0 to disable per-user upload rate limiting."
+        ),
+    )
+    upload_rate_limit_window: int = Field(
+        default=60,
+        description="Sliding window size in seconds for per-user upload rate limiting (default: 60).",
+    )
+
     # Rate Limiting Configuration (see SECURITY_AUDIT.md and docs/API.md)
     # Protects against DoS attacks and API abuse
     rate_limiting_enabled: bool = Field(
@@ -1129,6 +1377,40 @@ class Settings(BaseSettings):
         ),
     )
 
+    # ---------------------------------------------------------------------------
+    # Observability – Sentry Browser JavaScript SDK (client-side)
+    # ---------------------------------------------------------------------------
+    # The same SENTRY_DSN is reused for the browser SDK.  The DSN is a *public*
+    # key in Sentry's model and is intentionally embedded in client-side code.
+    # All three settings below default to 0.0 / disabled so that operators opt-in
+    # to the level of browser monitoring they want.
+    # ---------------------------------------------------------------------------
+    sentry_js_traces_sample_rate: float = Field(
+        default=0.0,
+        description=(
+            "Fraction of browser page-loads captured for client-side performance tracing "
+            "(0.0 – 1.0).  0.0 disables browser tracing; 1.0 captures every navigation. "
+            "Only active when SENTRY_DSN is set."
+        ),
+    )
+    sentry_js_replay_session_sample_rate: float = Field(
+        default=0.0,
+        description=(
+            "Fraction of sessions recorded by Sentry Session Replay (0.0 – 1.0). "
+            "0.0 disables session recording; 1.0 records every session. "
+            "Only active when SENTRY_DSN is set."
+        ),
+    )
+    sentry_js_replay_on_error_sample_rate: float = Field(
+        default=0.1,
+        description=(
+            "Fraction of sessions with an error that will be recorded by Sentry Session "
+            "Replay (0.0 – 1.0).  Defaults to 0.1 (10 %) so that errors are captured "
+            "with replay context even when session-level recording is disabled.  "
+            "Only active when SENTRY_DSN is set."
+        ),
+    )
+
     @model_validator(mode="before")
     @classmethod
     def strip_outer_quotes(cls, data: Any) -> Any:

app/database.py

@@ -10,6 +10,7 @@ from typing import Any
 from sqlalchemy import create_engine, exc
 from sqlalchemy.engine.url import make_url
 from sqlalchemy.orm import Session, declarative_base, sessionmaker
+from sqlalchemy.pool import NullPool, QueuePool
 
 from app.config import settings
 
@@ -17,9 +18,37 @@ logger = logging.getLogger(__name__)
 
 Base = declarative_base()
 
-# Parse the DATABASE_URL
+# ---------------------------------------------------------------------------
+# Engine construction
+# ---------------------------------------------------------------------------
 DB_URL = settings.database_url
-engine = create_engine(DB_URL, connect_args={"check_same_thread": False})
+_parsed_url = make_url(DB_URL)
+
+_connect_args: dict[str, Any] = {}
+_engine_kwargs: dict[str, Any] = {
+    "pool_pre_ping": True,  # detect stale / dropped connections before use
+}
+
+if _parsed_url.get_backend_name() == "sqlite":
+    # SQLite does not benefit from connection pooling and is prone to
+    # QueuePool exhaustion under concurrent access.  NullPool opens a fresh
+    # connection for each request and closes it immediately afterwards,
+    # completely avoiding the "QueuePool limit reached" TimeoutError.
+    _connect_args["check_same_thread"] = False
+    _engine_kwargs["poolclass"] = NullPool
+else:
+    # PostgreSQL / MySQL — use a bounded QueuePool with configurable limits.
+    _engine_kwargs["poolclass"] = QueuePool
+    _engine_kwargs.update(
+        {
+            "pool_size": settings.db_pool_size,
+            "max_overflow": settings.db_max_overflow,
+            "pool_timeout": settings.db_pool_timeout,
+            "pool_recycle": settings.db_pool_recycle,
+        }
+    )
+
+engine = create_engine(DB_URL, connect_args=_connect_args, **_engine_kwargs)
 SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
 
 
@@ -271,6 +300,7 @@ def _ensure_indexes(engine: Any, inspector: Any) -> None:
                 if table not in columns_by_table:
                     columns_by_table[table] = {col["name"] for col in inspector.get_columns(table)}
                 if column in columns_by_table[table]:
+                    # SECURITY: Quoted identifiers to prevent SQL injection during index creation
                     quoted_idx = preparer.quote(idx_name)
                     quoted_table = preparer.quote(table)
                     quoted_col = preparer.quote(column)

app/main.py

@@ -1,8 +1,11 @@
 #!/usr/bin/env python3
+import json as _json_mod
 import logging
 import os
 import pathlib
 from contextlib import asynccontextmanager
+from datetime import datetime as _dt
+from datetime import timezone as _tz
 
 from fastapi import FastAPI, HTTPException, Request, status
 from fastapi.middleware.cors import CORSMiddleware
@@ -36,6 +39,114 @@ from app.views import router as frontend_router
 # Explicitly include the files router
 from app.views.files import router as files_router
 
+# ---------------------------------------------------------------------------
+# Configure Python root logging level early so that *all* loggers (including
+# those already created via ``logging.getLogger(__name__)`` in other modules)
+# respect the configured level.
+#
+# Standard behaviour (matches Django, Flask, 12-factor conventions):
+#   • ``LOG_LEVEL`` env var takes precedence when explicitly set.
+#   • When ``DEBUG=True`` and ``LOG_LEVEL`` is **not** set, the effective
+#     level is automatically lowered to ``DEBUG``.
+#   • Default (neither flag set): ``INFO``.
+#
+# ``LOG_FORMAT=json`` enables structured JSON lines on stdout, suitable for
+# Promtail, Fluentd, Filebeat, Datadog, Splunk UF, or any log collector.
+#
+# ``LOG_SYSLOG_ENABLED=true`` adds a Python SysLogHandler so that every log
+# message is also forwarded to the configured syslog receiver — useful for
+# traditional (non-container) deployments and centralised SIEM ingestion.
+#
+# Noisy third-party loggers (httpx, httpcore, authlib, etc.) are pinned to
+# WARNING when the app-level is DEBUG to keep output useful.
+# ---------------------------------------------------------------------------
+_explicit_log_level = os.environ.get("LOG_LEVEL")
+if settings.debug and _explicit_log_level is None:
+    _effective_level = "DEBUG"
+else:
+    _effective_level = settings.log_level.upper()
+
+_effective_level_int = getattr(logging, _effective_level, logging.INFO)
+
+
+class _JsonFormatter(logging.Formatter):
+    """Emit one JSON object per log line for machine consumption.
+
+    Fields emitted: ``timestamp``, ``level``, ``logger``, ``message``,
+    ``module``, ``funcName``, ``lineno``, and — when present — ``exc_info``.
+    Compatible with Grafana Loki, Splunk, ELK, Datadog, and most SIEM tools.
+    """
+
+    def format(self, record: logging.LogRecord) -> str:
+        log_entry: dict = {
+            "timestamp": _dt.fromtimestamp(record.created, tz=_tz.utc).isoformat(),
+            "level": record.levelname,
+            "logger": record.name,
+            "message": record.getMessage(),
+            "module": record.module,
+            "funcName": record.funcName,
+            "lineno": record.lineno,
+        }
+        if record.exc_info and record.exc_info[1] is not None:
+            log_entry["exc_info"] = self.formatException(record.exc_info)
+        return _json_mod.dumps(log_entry, default=str)
+
+
+# Choose formatter based on LOG_FORMAT setting
+if settings.log_format.lower() == "json":
+    _handler = logging.StreamHandler()
+    _handler.setFormatter(_JsonFormatter())
+    logging.root.handlers = [_handler]
+    logging.root.setLevel(_effective_level_int)
+else:
+    logging.basicConfig(
+        level=_effective_level_int,
+        format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S",
+        force=True,
+    )
+
+# Optional: forward application logs to a syslog receiver
+if settings.log_syslog_enabled:
+    import logging.handlers as _lh
+    import socket as _socket
+
+    _proto = settings.log_syslog_protocol.lower()
+    _socktype = _socket.SOCK_STREAM if _proto == "tcp" else _socket.SOCK_DGRAM
+    _syslog_handler = _lh.SysLogHandler(
+        address=(settings.log_syslog_host, settings.log_syslog_port),
+        socktype=_socktype,
+    )
+    _syslog_handler.setLevel(_effective_level_int)
+    # Use the same formatter as stdout (text or JSON)
+    if settings.log_format.lower() == "json":
+        _syslog_handler.setFormatter(_JsonFormatter())
+    else:
+        _syslog_handler.setFormatter(logging.Formatter("%(name)s - %(levelname)s - %(message)s"))
+    logging.root.addHandler(_syslog_handler)
+
+# Keep noisy third-party loggers quiet at DEBUG level
+if _effective_level_int <= logging.DEBUG:
+    for _noisy in (
+        "httpx",
+        "httpcore",
+        "authlib",
+        "urllib3",
+        "hpack",
+        "multipart",
+        "watchfiles",
+    ):
+        logging.getLogger(_noisy).setLevel(logging.WARNING)
+
+_startup_logger = logging.getLogger(__name__)
+_startup_logger.info(
+    "Root logging level set to %s (debug=%s, format=%s, syslog=%s)",
+    _effective_level,
+    settings.debug,
+    settings.log_format,
+    settings.log_syslog_enabled,
+)
+
 # Load configuration from .env for the session key
 config = Config(".env")
 # Use settings.session_secret which has proper validation
@@ -59,6 +170,12 @@ async def lifespan(app: FastAPI):
     # Startup: Initialize database
     init_db()  # Create tables if they don't exist
 
+    # Factory reset on startup — wipe all user data before anything else
+    if settings.factory_reset_on_startup:
+        from app.utils.system_reset import perform_startup_reset
+
+        perform_startup_reset()
+
     # Load settings from database after DB initialization
     from app.database import SessionLocal
     from app.utils.config_loader import load_settings_from_db
@@ -72,6 +189,18 @@ async def lifespan(app: FastAPI):
     finally:
         db.close()
 
+    # Re-register OAuth / social-login providers now that DB settings are
+    # loaded.  auth.py runs its initial registration at import time (before
+    # the lifespan runs), so providers that are only configured in the
+    # database would not be registered yet.  Calling refresh here ensures
+    # they are active immediately on startup without any manual restart.
+    try:
+        from app.auth import refresh_social_providers
+
+        refresh_social_providers()
+    except Exception as e:
+        logging.warning(f"Could not refresh social login providers on startup: {e}")
+
     # Initialize Sentry after DB settings are loaded so that values configured
     # via the database UI (e.g. SENTRY_DSN) are respected in addition to env vars.
     init_sentry()
@@ -207,8 +336,19 @@ app.add_middleware(CSRFMiddleware, config=settings)
 #    See SECURITY_AUDIT.md – Infrastructure Security section
 app.add_middleware(AuditLogMiddleware, config=settings)
 
+
 # 3) Session Middleware (for request.session to work)
-app.add_middleware(SessionMiddleware, secret_key=SESSION_SECRET)
+def _get_session_max_age() -> int:
+    """Compute session max-age at startup time."""
+    try:
+        from app.utils.session_manager import get_session_max_age_seconds
+
+        return get_session_max_age_seconds()
+    except Exception:
+        return 30 * 86400  # 30 days default fallback
+
+
+app.add_middleware(SessionMiddleware, secret_key=SESSION_SECRET, max_age=_get_session_max_age())
 
 # 3a) CORS Middleware - handles cross-origin requests and preflight (OPTIONS) responses.
 #     Disabled by default: set CORS_ENABLED=True only when NOT using a reverse proxy
@@ -284,15 +424,13 @@ async def http_exception_handler(request: Request, exc: HTTPException):
     # For frontend routes, return appropriate HTML templates
     # Handle 404 errors with a custom template
     if exc.status_code == 404:
-        return _error_templates.TemplateResponse(
-            "404.html", {"request": request}, status_code=status.HTTP_404_NOT_FOUND
-        )
+        return _error_templates.TemplateResponse(request, "404.html", status_code=status.HTTP_404_NOT_FOUND)
 
     # For other HTTP errors, we could create specific templates or use a generic one
     # For now, return a simple error page
     return _error_templates.TemplateResponse(
+        request,
         "404.html",  # Reuse 404 template for other errors, or create a generic error template
-        {"request": request},
         status_code=exc.status_code,
     )
 
@@ -312,8 +450,9 @@ async def custom_500_handler(request: Request, exc: Exception):
 
     # Serve the 500 template for non-API routes
     return _error_templates.TemplateResponse(
+        request,
         "500.html",
-        {"request": request, "exc": exc},
+        context={"exc": exc},
         status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
     )
 

app/middleware/csrf.py

@@ -20,6 +20,9 @@ How it works:
 
 Exempt paths (CSRF is not checked even for state-changing methods):
 - ``/oauth-callback`` – OAuth 2.0 callback; protected by the ``state`` parameter.
+- ``/api/qr-auth/claim`` – Called by the unauthenticated mobile app; the
+  cryptographically-random, single-use challenge token provides equivalent
+  protection.
 """
 
 import logging
@@ -39,6 +42,10 @@ CSRF_PROTECTED_METHODS = {"POST", "PUT", "DELETE", "PATCH"}
 # their own replay-protection mechanism).
 CSRF_EXEMPT_PATHS = {
     "/oauth-callback",
+    # The mobile app calls this endpoint without a browser session/CSRF token.
+    # The cryptographically-random, single-use challenge token already provides
+    # equivalent protection against cross-site request forgery.
+    "/api/qr-auth/claim",
 }
 
 

app/middleware/upload_rate_limit.py

@@ -0,0 +1,290 @@
+"""Per-user, health-aware upload rate limiter for DocuElevate.
+
+This module provides a FastAPI dependency that enforces per-user upload rate
+limits using a Redis-backed sliding window counter.  The effective limit is
+dynamically reduced when the system is under heavy load (high Celery queue
+depth or elevated CPU load average), ensuring the server remains responsive
+to all users even during bulk-upload scenarios.
+
+Usage in an endpoint::
+
+    from app.middleware.upload_rate_limit import require_upload_rate_limit
+
+    @router.post("/ui-upload")
+    @require_login
+    async def ui_upload(
+        request: Request,
+        _rate_ok: None = Depends(require_upload_rate_limit),
+        ...
+    ):
+        ...
+
+See ``docs/ConfigurationGuide.md`` for the configuration options
+(``UPLOAD_RATE_LIMIT_PER_USER``, ``UPLOAD_RATE_LIMIT_WINDOW``).
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import time
+from typing import Any
+
+import redis
+from fastapi import HTTPException, Request, status
+
+from app.config import settings
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Redis key prefix
+# ---------------------------------------------------------------------------
+_KEY_PREFIX = "docuelevate:upload_rate"
+
+# ---------------------------------------------------------------------------
+# Health-check queue names (Celery defaults used by DocuElevate)
+# ---------------------------------------------------------------------------
+_CELERY_QUEUES = ("document_processor", "default", "celery")
+
+# ---------------------------------------------------------------------------
+# Singleton Redis client (lazy-initialised; fail-open when unavailable)
+# ---------------------------------------------------------------------------
+_redis_client: redis.Redis | None = None
+
+
+def _get_redis() -> redis.Redis | None:
+    """Return a shared Redis client, or *None* when Redis is unavailable."""
+    global _redis_client
+    if _redis_client is not None:
+        return _redis_client
+    try:
+        _redis_client = redis.Redis.from_url(
+            settings.redis_url,
+            decode_responses=True,
+            socket_connect_timeout=2,
+            socket_timeout=2,
+        )
+        # Quick connectivity check – raises on failure.
+        _redis_client.ping()
+        return _redis_client
+    except Exception:  # noqa: BLE001
+        logger.debug("Redis unavailable for upload rate limiter – falling back to allow-all", exc_info=True)
+        _redis_client = None
+        return None
+
+
+# ---------------------------------------------------------------------------
+# Health metrics helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_queue_depth(r: redis.Redis) -> int:
+    """Return the total number of pending tasks across all Celery queues."""
+    total = 0
+    for queue_name in _CELERY_QUEUES:
+        try:
+            total += r.llen(queue_name)
+        except Exception:  # noqa: BLE001, S110
+            logger.debug("Could not read queue length for %r", queue_name, exc_info=True)
+    return total
+
+
+def _get_cpu_load_ratio() -> float:
+    """Return the 1-minute load average divided by the number of CPU cores.
+
+    Returns ``0.0`` on platforms that do not support :func:`os.getloadavg`
+    (e.g. Windows) so that the limiter never penalises on those systems.
+    """
+    try:
+        load_1m = os.getloadavg()[0]
+        cpu_count = os.cpu_count() or 1
+        return load_1m / cpu_count
+    except (OSError, AttributeError):
+        return 0.0
+
+
+def compute_effective_limit(
+    base_limit: int,
+    queue_depth: int = 0,
+    cpu_load_ratio: float = 0.0,
+) -> tuple[int, float, str]:
+    """Compute the effective upload rate limit based on system health.
+
+    The function applies a *reduction factor* (``0.0 < factor ≤ 1.0``) to the
+    configured base limit.  Both queue depth and CPU load contribute
+    independently; the lowest factor wins.
+
+    Args:
+        base_limit: The configured maximum uploads per window.
+        queue_depth: Total pending tasks in Celery queues.
+        cpu_load_ratio: 1-minute load average divided by CPU count.
+
+    Returns:
+        A 3-tuple of ``(effective_limit, factor, reason)`` where *reason*
+        is a human-readable tag for logging.
+    """
+    factor = 1.0
+    reason = "normal"
+
+    # --- Queue-depth thresholds ---
+    if queue_depth > 200:
+        factor, reason = min(factor, 0.10), f"critical_queue({queue_depth})"
+    elif queue_depth > 100:
+        factor, reason = min(factor, 0.25), f"high_queue({queue_depth})"
+    elif queue_depth > 50:
+        factor, reason = min(factor, 0.50), f"moderate_queue({queue_depth})"
+
+    # --- CPU-load thresholds ---
+    if cpu_load_ratio > 3.0:
+        new_factor = 0.10
+        if new_factor < factor:
+            factor, reason = new_factor, f"critical_cpu({cpu_load_ratio:.1f})"
+    elif cpu_load_ratio > 2.0:
+        new_factor = 0.25
+        if new_factor < factor:
+            factor, reason = new_factor, f"high_cpu({cpu_load_ratio:.1f})"
+    elif cpu_load_ratio > 1.5:
+        new_factor = 0.50
+        if new_factor < factor:
+            factor, reason = new_factor, f"moderate_cpu({cpu_load_ratio:.1f})"
+
+    effective = max(1, int(base_limit * factor))
+    return effective, factor, reason
+
+
+# ---------------------------------------------------------------------------
+# Core sliding-window check (Redis sorted set)
+# ---------------------------------------------------------------------------
+
+
+def _check_and_record(
+    r: redis.Redis,
+    user_id: str,
+    window: int,
+    effective_limit: int,
+) -> dict[str, Any] | None:
+    """Atomically check the user's upload count and record the new upload.
+
+    Uses a Redis sorted set where each member is a unique timestamp-based ID
+    and the score is the Unix timestamp.  Entries older than *window* seconds
+    are pruned on every call so the set never grows unbounded.
+
+    Returns:
+        ``None`` if the request is allowed, or a ``dict`` with ``count``,
+        ``limit``, and ``retry_after`` if the limit is exceeded.
+    """
+    key = f"{_KEY_PREFIX}:{user_id}"
+    now = time.time()
+    window_start = now - window
+
+    pipe = r.pipeline(transaction=True)
+    # 1. Remove entries outside the window
+    pipe.zremrangebyscore(key, "-inf", window_start)
+    # 2. Count current entries
+    pipe.zcard(key)
+    # 3. Retrieve the oldest entry's score (to compute retry_after)
+    pipe.zrange(key, 0, 0, withscores=True)
+    results = pipe.execute()
+
+    current_count: int = results[1]
+    oldest_entries: list = results[2]
+
+    if current_count >= effective_limit:
+        # Compute how long until the oldest entry expires from the window.
+        if oldest_entries:
+            oldest_score = oldest_entries[0][1]
+            retry_after = max(1, int((oldest_score + window) - now))
+        else:
+            retry_after = max(1, window // 2)
+        return {
+            "count": current_count,
+            "limit": effective_limit,
+            "retry_after": retry_after,
+        }
+
+    # 4. Record this upload (unique member = timestamp with random suffix)
+    member = f"{now}:{os.urandom(4).hex()}"
+    pipe2 = r.pipeline(transaction=True)
+    pipe2.zadd(key, {member: now})
+    pipe2.expire(key, window + 60)  # TTL slightly longer than window
+    pipe2.execute()
+
+    return None
+
+
+# ---------------------------------------------------------------------------
+# FastAPI dependency
+# ---------------------------------------------------------------------------
+
+
+async def require_upload_rate_limit(request: Request) -> None:
+    """FastAPI dependency that enforces per-user upload rate limits.
+
+    The dependency is designed to **fail open**: if Redis is unavailable the
+    request is allowed through so that uploads are never blocked by a
+    monitoring outage.
+
+    Raises:
+        HTTPException: 429 Too Many Requests when the per-user upload limit
+            is exceeded.  The ``Retry-After`` header indicates how many
+            seconds the client should wait before retrying.
+    """
+    r = _get_redis()
+    if r is None:
+        # Redis unavailable – fail open.
+        return
+
+    # Identify the user (owner_id for multi-user, IP fallback).
+    user_id = get_current_owner_id(request)
+    if not user_id:
+        user_id = f"ip:{request.client.host}" if request.client else "ip:unknown"
+
+    base_limit: int = settings.upload_rate_limit_per_user
+    window: int = settings.upload_rate_limit_window
+
+    # Gather health metrics and compute effective limit.
+    try:
+        queue_depth = _get_queue_depth(r)
+    except Exception:  # noqa: BLE001
+        queue_depth = 0
+
+    cpu_load_ratio = _get_cpu_load_ratio()
+    effective_limit, factor, health_reason = compute_effective_limit(base_limit, queue_depth, cpu_load_ratio)
+
+    # Sliding-window check.
+    try:
+        rejection = _check_and_record(r, user_id, window, effective_limit)
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("Upload rate-limit check failed (allowing request): %s", exc)
+        return
+
+    if rejection is not None:
+        retry_after = rejection["retry_after"]
+        logger.warning(
+            "Upload rate limit exceeded: user=%s count=%d/%d window=%ds health=%s retry_after=%ds",
+            user_id,
+            rejection["count"],
+            rejection["limit"],
+            window,
+            health_reason,
+            retry_after,
+        )
+        raise HTTPException(
+            status_code=status.HTTP_429_TOO_MANY_REQUESTS,
+            detail=(
+                f"Upload rate limit exceeded ({rejection['count']}/{rejection['limit']} "
+                f"in {window}s). Retry after {retry_after}s."
+            ),
+            headers={"Retry-After": str(retry_after)},
+        )
+
+    if factor < 1.0:
+        logger.info(
+            "Upload allowed with reduced limit: user=%s effective=%d/%d health=%s",
+            user_id,
+            effective_limit,
+            base_limit,
+            health_reason,
+        )

app/models.py

@@ -84,6 +84,19 @@ class FileRecord(Base):
     # Processing pipeline assigned to this file (NULL = use system default)
     pipeline_id = Column(Integer, ForeignKey(_PIPELINES_ID_FK), nullable=True, index=True)
 
+    # Detected document language (ISO 639-1 code, e.g. "de", "en", "fr")
+    # Extracted from AI metadata during processing; cached here for fast access.
+    detected_language = Column(String(10), nullable=True)
+
+    # Default-language translation of the extracted text.
+    # Stored when the detected language differs from the user's/system default
+    # document language.  Only the original text and this translation are persisted;
+    # other languages are translated on the fly via the AI provider.
+    default_language_text = Column(Text, nullable=True)
+
+    # ISO 639-1 code of the default-language translation stored above (e.g. "en").
+    default_language_code = Column(String(10), nullable=True)
+
     # Timestamp when we inserted this record
     created_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
 
@@ -198,6 +211,28 @@ class WebhookConfig(Base):
     updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
 
 
+class AutomationHook(Base):
+    """Zapier / Make.com compatible webhook subscription for automation triggers.
+
+    External automation platforms subscribe to DocuElevate events via the REST
+    hooks protocol.  When an event fires, DocuElevate POSTs a Zapier-compatible
+    flat JSON payload to ``target_url``.  The ``hook_type`` field records which
+    platform created the subscription (informational only).
+    """
+
+    __tablename__ = "automation_hooks"
+
+    id = Column(Integer, primary_key=True, index=True)
+    target_url = Column(String, nullable=False)  # URL to POST events to
+    secret = Column(String, nullable=True)  # Optional HMAC-SHA256 signing secret
+    events = Column(Text, nullable=False)  # JSON list of subscribed event names
+    is_active = Column(Boolean, default=True, nullable=False)
+    hook_type = Column(String(50), nullable=False, default="generic")  # zapier | make | generic
+    description = Column(String, nullable=True)  # Optional human-readable label
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
 class LocalUser(Base):
     """A locally-registered user authenticated by email and bcrypt password.
 
@@ -282,6 +317,12 @@ class UserProfile(Base):
     # NULL means "auto-detect from browser Accept-Language header"
     preferred_language = Column(String(10), nullable=True)
 
+    # Default document language for translated versions (ISO 639-1 code).
+    # When a document's detected language differs from this value, the system
+    # automatically generates and stores a translation into this language.
+    # NULL means "use the global DEFAULT_DOCUMENT_LANGUAGE setting".
+    default_document_language = Column(String(10), nullable=True)
+
     # UI colour scheme preference: "light" | "dark" | "system" (NULL = "system")
     preferred_theme = Column(String(10), nullable=True)
 
@@ -584,6 +625,7 @@ class IntegrationType:
     EMAIL = "EMAIL"
     PAPERLESS = "PAPERLESS"
     RCLONE = "RCLONE"
+    SHAREPOINT = "SHAREPOINT"
     ICLOUD = "ICLOUD"
 
     ALL = {
@@ -601,6 +643,7 @@ class IntegrationType:
         EMAIL,
         PAPERLESS,
         RCLONE,
+        SHAREPOINT,
         ICLOUD,
     }
 
@@ -765,6 +808,9 @@ class ApiToken(Base):
     created_at = Column(DateTime(timezone=True), server_default=func.now())
     revoked_at = Column(DateTime(timezone=True), nullable=True)
 
+    # Optional expiry: if set, the token is rejected after this timestamp.
+    expires_at = Column(DateTime(timezone=True), nullable=True)
+
 
 class SharedLink(Base):
     """Shareable, time-limited or view-limited document link.
@@ -916,6 +962,51 @@ class ScheduledJob(Base):
     updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
 
 
+class ClassificationRuleModel(Base):
+    """Custom document classification rule.
+
+    Rules are evaluated during the ``classify`` pipeline step to assign a
+    category to a document.  System-wide rules have ``owner_id IS NULL``;
+    user-specific rules belong to a single owner.
+    """
+
+    __tablename__ = "classification_rules"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # NULL = system-wide rule visible to all users.
+    owner_id = Column(String, nullable=True, index=True)
+
+    # Human-readable rule name (unique per owner).
+    name = Column(String(255), nullable=False)
+
+    # Target category (e.g. "invoice", "contract", "receipt").
+    category = Column(String(100), nullable=False, index=True)
+
+    # Rule type: "filename_pattern", "content_keyword", or "metadata_match".
+    rule_type = Column(String(50), nullable=False)
+
+    # The matching pattern:
+    #   - filename_pattern: a regex
+    #   - content_keyword: pipe-separated keywords
+    #   - metadata_match: "field=value"
+    pattern = Column(String(1000), nullable=False)
+
+    # Higher priority rules are evaluated first (default 0).
+    priority = Column(Integer, nullable=False, default=0)
+
+    # Whether pattern matching is case-sensitive.
+    case_sensitive = Column(Boolean, nullable=False, default=False)
+
+    # Disabled rules are skipped during classification.
+    enabled = Column(Boolean, nullable=False, default=True)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    __table_args__ = (UniqueConstraint("owner_id", "name", name="uq_classification_rules_owner_name"),)
+
+
 class MobileDevice(Base):
     """Registered mobile device for push notifications.
 
@@ -950,6 +1041,86 @@ class MobileDevice(Base):
     __table_args__ = (UniqueConstraint("owner_id", "push_token", name="uq_mobile_device_owner_token"),)
 
 
+class UserSession(Base):
+    """Server-side session tracking for invalidation and device management.
+
+    Each row represents an active browser or app session.  The ``session_token``
+    is stored in the user's cookie and validated on every authenticated request.
+    Revoking a row (``is_revoked=True``) immediately terminates that session
+    on the next request.
+    """
+
+    __tablename__ = "user_sessions"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Cryptographically random token stored in the session cookie.
+    session_token = Column(String(128), unique=True, nullable=False, index=True)
+
+    # Stable owner identifier — matches FileRecord.owner_id.
+    user_id = Column(String, nullable=False, index=True)
+
+    # Client metadata for display in the session management UI.
+    ip_address = Column(String(45), nullable=True)
+    user_agent = Column(String(512), nullable=True)
+    device_info = Column(String(255), nullable=True)
+
+    is_revoked = Column(Boolean, nullable=False, default=False)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    last_active_at = Column(DateTime(timezone=True), server_default=func.now())
+    expires_at = Column(DateTime(timezone=True), nullable=False)
+    revoked_at = Column(DateTime(timezone=True), nullable=True)
+
+
+class QRLoginChallenge(Base):
+    """Time-limited QR code login challenge for mobile app authentication.
+
+    A logged-in web user generates a challenge that produces a QR code.  The
+    mobile app scans the QR code and calls the claim endpoint with the
+    ``challenge_token``.  The server verifies the challenge is still valid,
+    unclaimed, and unexpired, then issues an API token for the mobile app.
+
+    Security properties:
+    * Time-bound (default 2 minutes).
+    * Single-use (``is_claimed`` prevents replay).
+    * Cryptographically random 64-byte token.
+    * Bound to the creating user — only that user's mobile device receives a
+      token.
+    """
+
+    __tablename__ = "qr_login_challenges"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Cryptographically random token encoded in the QR code.
+    challenge_token = Column(String(128), unique=True, nullable=False, index=True)
+
+    # The user who created this challenge (from the web session).
+    user_id = Column(String, nullable=False, index=True)
+
+    # Whether the challenge has been successfully claimed by a mobile app.
+    is_claimed = Column(Boolean, nullable=False, default=False)
+
+    # Whether the challenge has been explicitly cancelled or expired.
+    is_cancelled = Column(Boolean, nullable=False, default=False)
+
+    # IP address of the web client that created the challenge.
+    created_by_ip = Column(String(45), nullable=True)
+
+    # IP address of the mobile client that claimed the challenge.
+    claimed_by_ip = Column(String(45), nullable=True)
+
+    # Device name provided by the mobile app when claiming.
+    device_name = Column(String(255), nullable=True)
+
+    # The API token ID that was issued to the mobile app (for audit trail).
+    issued_token_id = Column(Integer, nullable=True)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    expires_at = Column(DateTime(timezone=True), nullable=False)
+    claimed_at = Column(DateTime(timezone=True), nullable=True)
+
+
 class ComplianceTemplate(Base):
     """Pre-built compliance configuration templates (GDPR, HIPAA, SOC2).
 
@@ -1021,3 +1192,97 @@ class PipelineRoutingRule(Base):
 
     created_at = Column(DateTime(timezone=True), server_default=func.now())
     updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class DocumentComment(Base):
+    """Threaded comment on a document.
+
+    Supports threaded replies via ``parent_id`` and @mentions via the
+    ``mentions`` column (comma-separated user identifiers).
+    """
+
+    __tablename__ = "document_comments"
+
+    id = Column(Integer, primary_key=True, index=True)
+    file_id = Column(Integer, ForeignKey(_FILES_ID_FK), nullable=False, index=True)
+    user_id = Column(String, nullable=False, index=True)
+    parent_id = Column(Integer, ForeignKey("document_comments.id"), nullable=True, index=True)
+    body = Column(Text, nullable=False)
+    mentions = Column(Text, nullable=True)
+    is_resolved = Column(Boolean, nullable=False, default=False, server_default="0")
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class DocumentAnnotation(Base):
+    """Text annotation on a specific page and position of a PDF document.
+
+    Stores the bounding-box coordinates (``x``, ``y``, ``width``,
+    ``height``) relative to the page dimensions so that the annotation
+    can be rendered on top of the PDF viewer.
+    """
+
+    __tablename__ = "document_annotations"
+
+    id = Column(Integer, primary_key=True, index=True)
+    file_id = Column(Integer, ForeignKey(_FILES_ID_FK), nullable=False, index=True)
+    user_id = Column(String, nullable=False, index=True)
+    page = Column(Integer, nullable=False)
+    x = Column(Float, nullable=False)
+    y = Column(Float, nullable=False)
+    width = Column(Float, nullable=False, default=0)
+    height = Column(Float, nullable=False, default=0)
+    content = Column(Text, nullable=False)
+    annotation_type = Column(String(50), nullable=False, default="note", server_default="note")
+    color = Column(String(20), nullable=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+# ---------------------------------------------------------------------------
+# File sharing
+# ---------------------------------------------------------------------------
+
+# Valid roles for FileShare.role
+FILE_SHARE_ROLE_VIEWER = "viewer"
+FILE_SHARE_ROLE_EDITOR = "editor"
+FILE_SHARE_ROLES = (FILE_SHARE_ROLE_VIEWER, FILE_SHARE_ROLE_EDITOR)
+
+
+class FileShare(Base):
+    """Grants a named user access to a ``FileRecord`` owned by someone else.
+
+    The ``owner_id`` column records who created the share (must be the file
+    owner).  ``shared_with_user_id`` is the recipient's stable user
+    identifier (the same kind of string used in ``FileRecord.owner_id``).
+
+    Roles
+    -----
+    ``viewer``  — can read the file, comments, and annotations; may add
+                  comments/annotations; cannot delete or share.
+    ``editor``  — all viewer rights plus the ability to edit document
+                  metadata; cannot delete or re-share.
+
+    Only the file owner may create, update, or revoke shares.
+    """
+
+    __tablename__ = "file_shares"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # The document being shared.
+    file_id = Column(Integer, ForeignKey(_FILES_ID_FK), nullable=False, index=True)
+
+    # The user who granted the share (must match FileRecord.owner_id).
+    owner_id = Column(String, nullable=False, index=True)
+
+    # The user receiving the share.
+    shared_with_user_id = Column(String, nullable=False, index=True)
+
+    # "viewer" or "editor"
+    role = Column(String(20), nullable=False, default=FILE_SHARE_ROLE_VIEWER)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+    __table_args__ = (UniqueConstraint("file_id", "shared_with_user_id", name="uq_file_share_file_user"),)

app/tasks/automation_tasks.py

@@ -0,0 +1,44 @@
+"""Celery task for asynchronous automation hook delivery with retry and backoff.
+
+Uses :class:`~app.tasks.retry_config.BaseTaskWithRetry` so failed deliveries
+are automatically retried with exponential backoff (default: 60 s, 300 s,
+900 s) and ±20 % jitter.
+"""
+
+import logging
+from typing import Any
+
+from app.celery_app import celery
+from app.tasks.retry_config import BaseTaskWithRetry
+from app.utils.webhook import deliver_webhook
+
+logger = logging.getLogger(__name__)
+
+
+@celery.task(base=BaseTaskWithRetry, bind=True, name="automation.deliver_hook")
+def deliver_automation_hook_task(self, url: str, payload: dict[str, Any], secret: str | None = None) -> dict[str, Any]:
+    """Deliver an automation hook payload to *url* with automatic retries.
+
+    Args:
+        url: Target webhook URL (provided by Zapier / Make.com).
+        payload: The flat Zapier-compatible payload.
+        secret: Optional shared secret for HMAC-SHA256 signing.
+
+    Returns:
+        A dict with ``status`` and ``url`` on success.
+
+    Raises:
+        RuntimeError: Re-raised to trigger Celery retry on delivery failure.
+    """
+    logger.info(
+        "Delivering automation hook to %s (attempt %d/%d)",
+        url,
+        self.request.retries + 1,
+        self.max_retries + 1,
+    )
+
+    success = deliver_webhook(url, payload, secret)
+    if success:
+        return {"status": "delivered", "url": url}
+
+    raise RuntimeError(f"Automation hook delivery to {url} failed")

app/tasks/classify_document.py

@@ -0,0 +1,174 @@
+"""Celery task for rule-based document classification.
+
+This task is executed as a pipeline step (``step_type="classify"``).  It
+applies built-in and user-defined classification rules against the document's
+filename, OCR text, and existing AI metadata to assign a ``document_type``
+category.
+
+The result is stored in the ``ai_metadata`` JSON blob on the
+:class:`~app.models.FileRecord` (field ``classification``).
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from typing import Any
+
+from app.celery_app import celery
+from app.database import SessionLocal
+from app.models import ClassificationRuleModel, FileRecord
+from app.tasks.retry_config import BaseTaskWithRetry
+from app.utils import log_task_progress
+from app.utils.classification_rules import (
+    ClassificationResult,
+    classify_document,
+    db_rule_to_engine_rule,
+)
+
+logger = logging.getLogger(__name__)
+
+STEP_NAME = "classify_document"
+
+
+def _load_custom_rules(owner_id: str | None) -> list[Any]:
+    """Load enabled custom classification rules from the database.
+
+    Returns engine-level :class:`ClassificationRule` dataclass instances.
+    Rules are loaded in priority-descending order.  System rules
+    (``owner_id IS NULL``) and the user's own rules are both included.
+    """
+    with SessionLocal() as db:
+        query = db.query(ClassificationRuleModel).filter(ClassificationRuleModel.enabled.is_(True))
+        if owner_id:
+            query = query.filter(
+                (ClassificationRuleModel.owner_id.is_(None)) | (ClassificationRuleModel.owner_id == owner_id)
+            )
+        else:
+            query = query.filter(ClassificationRuleModel.owner_id.is_(None))
+        rules = query.order_by(ClassificationRuleModel.priority.desc()).all()
+        return [db_rule_to_engine_rule(r) for r in rules]
+
+
+@celery.task(base=BaseTaskWithRetry, bind=True)
+def classify_document_task(
+    self: Any,
+    file_id: int,
+    owner_id: str | None = None,
+) -> dict[str, Any]:
+    """Classify a document using rule-based matching.
+
+    This task:
+    1. Loads the :class:`FileRecord` from the database.
+    2. Gathers filename, OCR text, and existing AI metadata.
+    3. Loads built-in + user-defined classification rules.
+    4. Runs the classification engine.
+    5. Persists the result into ``ai_metadata.classification``.
+
+    Args:
+        file_id: Primary key of the :class:`FileRecord` to classify.
+        owner_id: Owner identifier for loading user-specific rules.
+
+    Returns:
+        Dict with ``category``, ``confidence``, and ``matched_rules``.
+    """
+    task_id = self.request.id
+
+    log_task_progress(
+        task_id,
+        STEP_NAME,
+        "in_progress",
+        f"Starting classification for file {file_id}",
+        file_id=file_id,
+    )
+
+    try:
+        with SessionLocal() as db:
+            file_record: FileRecord | None = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+            if file_record is None:
+                log_task_progress(
+                    task_id,
+                    STEP_NAME,
+                    "failure",
+                    f"FileRecord {file_id} not found",
+                    file_id=file_id,
+                )
+                return {"status": "error", "detail": "File not found"}
+
+            # Gather inputs
+            filename = file_record.original_filename or ""
+            text = file_record.ocr_text or ""
+            existing_metadata: dict[str, Any] = {}
+            if file_record.ai_metadata:
+                try:
+                    existing_metadata = json.loads(file_record.ai_metadata)
+                except (json.JSONDecodeError, TypeError):
+                    logger.warning("Failed to parse ai_metadata for file %s, starting fresh", file_id)
+                    existing_metadata = {}
+
+            # Load custom rules
+            effective_owner = owner_id or file_record.owner_id
+            custom_rules = _load_custom_rules(effective_owner)
+
+            # Run classification engine
+            result: ClassificationResult = classify_document(
+                filename=filename,
+                text=text,
+                metadata=existing_metadata,
+                custom_rules=custom_rules,
+            )
+
+            # Persist result into ai_metadata
+            classification_data = {
+                "category": result.category,
+                "confidence": result.confidence,
+                "matched_rules": [
+                    {
+                        "rule_name": m.rule_name,
+                        "rule_type": m.rule_type,
+                        "category": m.category,
+                        "confidence": m.confidence,
+                    }
+                    for m in result.matched_rules
+                ],
+            }
+
+            existing_metadata["classification"] = classification_data
+
+            # If no document_type was set yet, populate it from the classification
+            if not existing_metadata.get("document_type"):
+                from app.utils.classification_rules import BUILTIN_CATEGORIES
+
+                existing_metadata["document_type"] = BUILTIN_CATEGORIES.get(
+                    result.category, result.category.replace("_", " ").title()
+                )
+
+            file_record.ai_metadata = json.dumps(existing_metadata, ensure_ascii=False)
+            db.commit()
+
+        log_task_progress(
+            task_id,
+            STEP_NAME,
+            "success",
+            f"Classified as '{result.category}' with confidence {result.confidence}",
+            file_id=file_id,
+            detail=f"Matched {len(result.matched_rules)} rule(s)",
+        )
+
+        return {
+            "status": "success",
+            "category": result.category,
+            "confidence": result.confidence,
+            "matched_rules": len(result.matched_rules),
+        }
+
+    except Exception as e:
+        logger.exception("Classification failed for file %s: %s", file_id, e)
+        log_task_progress(
+            task_id,
+            STEP_NAME,
+            "failure",
+            f"Classification failed: {e}",
+            file_id=file_id,
+        )
+        raise

app/tasks/convert_to_pdf.py

@@ -205,7 +205,7 @@ def convert_to_pdf(
         ".pdf",  # PDF (already in PDF format but can be processed)
     }
 
-    IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tiff", ".tif", ".webp", ".svg"}
+    IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tiff", ".tif", ".webp", ".svg", ".heic", ".heif"}
 
     HTML_EXTENSIONS = {".html", ".htm"}
 

app/tasks/convert_to_pdfa.py

@@ -78,6 +78,7 @@ def _convert_pdf_to_pdfa(input_path: str, output_path: str, pdfa_format: str = "
         output_type,
         "--quiet",
         "--invalidate-digital-signatures",
+        "--",
         input_path,
         output_path,
     ]

app/tasks/embed_metadata_into_pdf.py

@@ -216,6 +216,30 @@ def embed_metadata_into_pdf(self, local_file_path: str, extracted_text: str, met
                     except Exception as search_exc:
                         logger.warning(f"[{task_id}] Meilisearch indexing failed (non-fatal): {search_exc}")
 
+                    # Cache the detected language on the FileRecord and trigger
+                    # default-language translation when the document is in a
+                    # different language.
+                    detected_lang = metadata.get("language") if metadata else None
+                    if detected_lang and extracted_text:
+                        try:
+                            file_record.detected_language = detected_lang
+                            db.commit()
+
+                            from app.tasks.translate_to_default_language import translate_to_default_language
+
+                            translate_to_default_language.delay(
+                                file_id,
+                                extracted_text,
+                                detected_lang,
+                                owner_id=file_record.owner_id,
+                            )
+                            logger.info(
+                                f"[{task_id}] Queued default-language translation for file {file_id} "
+                                f"(detected: {detected_lang})"
+                            )
+                        except Exception as trans_exc:
+                            logger.warning(f"[{task_id}] Could not queue translation task (non-fatal): {trans_exc}")
+
         # Persist the metadata into a JSON file with the same base name.
         # Include file path references for traceability
         logger.info(f"[{task_id}] Persisting metadata to JSON")

app/tasks/finalize_document_storage.py

@@ -21,8 +21,9 @@ from app.tasks.send_to_all import (
 # Import database and logging utils from main
 from app.utils import log_task_progress
 
-# Import notification utility
+# Import notification utilities
 from app.utils.notification import notify_file_processed
+from app.utils.user_notification import notify_user_document_processed
 
 logger = logging.getLogger(__name__)
 
@@ -139,4 +140,15 @@ def finalize_document_storage(self, original_file: str, processed_file: str, met
     except Exception as e:
         logger.warning(f"[WARNING] Failed to send file processed notification: {e}")
 
+    # 6. Send per-user notification
+    if owner_id:
+        try:
+            notify_user_document_processed(
+                owner_id=owner_id,
+                filename=os.path.basename(processed_file),
+                file_id=file_id,
+            )
+        except Exception as e:
+            logger.warning(f"[WARNING] Failed to send per-user processed notification: {e}")
+
     return {"status": "Completed", "file": processed_file}

app/tasks/send_to_all.py

@@ -18,6 +18,7 @@ from app.tasks.upload_to_onedrive import upload_to_onedrive
 from app.tasks.upload_to_paperless import upload_to_paperless
 from app.tasks.upload_to_s3 import upload_to_s3
 from app.tasks.upload_to_sftp import upload_to_sftp
+from app.tasks.upload_to_sharepoint import upload_to_sharepoint
 from app.tasks.upload_to_webdav import upload_to_webdav
 from app.utils.config_validator import get_provider_status
 from app.utils.logging import log_task_progress
@@ -121,6 +122,18 @@ def _should_upload_to_icloud():
     return bool(getattr(settings, "icloud_enabled", True) and settings.icloud_username and settings.icloud_password)
 
 
+def _should_upload_to_sharepoint():
+    return bool(
+        settings.sharepoint_client_id
+        and settings.sharepoint_client_secret
+        and settings.sharepoint_site_url
+        and (
+            settings.sharepoint_refresh_token
+            or (settings.sharepoint_tenant_id and settings.sharepoint_tenant_id != "common")
+        )
+    )
+
+
 def get_configured_services_from_validator():
     """
     Use the config validator to determine which services are configured and enabled.
@@ -140,6 +153,7 @@ def get_configured_services_from_validator():
         "Email": "email",
         "OneDrive": "onedrive",
         "S3 Storage": "s3",
+        "SharePoint": "sharepoint",
         "iCloud Drive": "icloud",
     }
 
@@ -250,6 +264,11 @@ def send_to_all_destinations(self, file_path: str, use_validator=True, file_id:
             "should_upload": _should_upload_to_s3,
             "upload_func": upload_to_s3,
         },
+        {
+            "name": "sharepoint",
+            "should_upload": _should_upload_to_sharepoint,
+            "upload_func": upload_to_sharepoint,
+        },
         {
             "name": "icloud",
             "should_upload": _should_upload_to_icloud,

app/tasks/translate_to_default_language.py

@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""Celery task to translate extracted document text into the default target language.
+
+This task is triggered after metadata extraction when the detected document
+language differs from the user's (or system) default document language.  The
+translated text is persisted in ``FileRecord.default_language_text`` so that
+users can always read a reference copy in their preferred language.
+
+Other ad-hoc translations are generated on the fly via the ``/api/files/{id}/translate``
+endpoint and are NOT persisted.
+"""
+
+import logging
+
+from app.celery_app import celery
+from app.config import settings
+from app.database import SessionLocal
+from app.models import FileRecord, UserProfile
+from app.tasks.retry_config import BaseTaskWithRetry
+from app.utils import log_task_progress
+from app.utils.ai_provider import get_ai_provider
+
+logger = logging.getLogger(__name__)
+
+
+def _resolve_default_language(owner_id: str | None) -> str:
+    """Return the default document language for the given owner.
+
+    Resolution order:
+    1. ``UserProfile.default_document_language`` (per-user override)
+    2. ``settings.default_document_language`` (global setting)
+    """
+    if owner_id:
+        with SessionLocal() as db:
+            profile = db.query(UserProfile).filter_by(user_id=owner_id).first()
+            if profile and profile.default_document_language:
+                return profile.default_document_language
+    return settings.default_document_language
+
+
+@celery.task(base=BaseTaskWithRetry, bind=True)
+def translate_to_default_language(
+    self,
+    file_id: int,
+    extracted_text: str,
+    detected_language: str,
+    owner_id: str | None = None,
+) -> dict:
+    """Translate *extracted_text* into the default document language and persist the result.
+
+    Args:
+        file_id: Primary key of the :class:`FileRecord`.
+        extracted_text: The OCR / refined text in the document's original language.
+        detected_language: ISO 639-1 code of the document's detected language.
+        owner_id: Owner identifier used to resolve per-user language preference.
+
+    Returns:
+        A dict with ``status``, ``target_language``, and the translated text length.
+    """
+    task_id = self.request.id
+    target_language = _resolve_default_language(owner_id)
+
+    # Nothing to do when the document is already in the target language.
+    if detected_language == target_language:
+        logger.info(
+            f"[{task_id}] Document {file_id} already in target language '{target_language}', skipping translation"
+        )
+        log_task_progress(
+            task_id,
+            "translate_to_default_language",
+            "skipped",
+            f"Document already in {target_language}",
+            file_id=file_id,
+        )
+        return {"status": "skipped", "reason": "already_in_target_language"}
+
+    logger.info(f"[{task_id}] Translating document {file_id} from '{detected_language}' to '{target_language}'")
+    log_task_progress(
+        task_id,
+        "translate_to_default_language",
+        "in_progress",
+        f"Translating from {detected_language} to {target_language}",
+        file_id=file_id,
+    )
+
+    try:
+        provider = get_ai_provider()
+        model = settings.ai_model or settings.openai_model
+        translated_text = provider.chat_completion(
+            messages=[
+                {
+                    "role": "system",
+                    "content": (
+                        f"You are a professional translator. Translate the following text "
+                        f"from {detected_language} to {target_language}. "
+                        f"Preserve the original formatting, paragraph structure, and meaning. "
+                        f"Do not add any commentary or explanation — output ONLY the translated text."
+                    ),
+                },
+                {"role": "user", "content": extracted_text},
+            ],
+            model=model,
+            temperature=0.3,
+        )
+
+        # Persist the translation.
+        with SessionLocal() as db:
+            record = db.query(FileRecord).filter_by(id=file_id).first()
+            if record:
+                record.default_language_text = translated_text
+                record.default_language_code = target_language
+                record.detected_language = detected_language
+                db.commit()
+                logger.info(
+                    f"[{task_id}] Stored default-language translation ({len(translated_text)} chars) for file {file_id}"
+                )
+
+        log_task_progress(
+            task_id,
+            "translate_to_default_language",
+            "success",
+            f"Translated {len(extracted_text)} → {len(translated_text)} chars ({detected_language} → {target_language})",
+            file_id=file_id,
+        )
+
+        return {
+            "status": "success",
+            "target_language": target_language,
+            "translated_length": len(translated_text),
+        }
+
+    except Exception as exc:
+        logger.exception(f"[{task_id}] Translation failed for file {file_id}: {exc}")
+        log_task_progress(
+            task_id,
+            "translate_to_default_language",
+            "failure",
+            f"Exception: {exc}",
+            file_id=file_id,
+        )
+        raise

app/tasks/upload_to_email.py

@@ -11,6 +11,7 @@ from email.mime.image import MIMEImage
 from email.mime.multipart import MIMEMultipart
 from email.mime.text import MIMEText
 
+import pypdf
 from jinja2 import Environment, FileSystemLoader, select_autoescape
 
 from app.celery_app import celery
@@ -23,6 +24,15 @@ logger = logging.getLogger(__name__)
 # Constants
 _LOGO_FILENAME = "logo.png"
 
+# Mapping from PDF metadata keys (with leading slash stripped) to application-specific names.
+# This mirrors the inverse of the mapping used in app/tasks/embed_metadata_into_pdf.py.
+_PDF_METADATA_KEY_MAP = {
+    "Title": "filename",
+    "Author": "absender",
+    "Subject": "document_type",
+    "Keywords": "tags",
+}
+
 
 def get_email_template(template_name="default.html"):
     """
@@ -63,9 +73,12 @@ def extract_metadata_from_file(file_path):
     """
     Try to extract metadata from a file using several methods:
     1. Check for a .json metadata file with the same name
-    2. Extract metadata from PDF if it's embedded
+    2. Extract embedded metadata from PDF using pypdf
 
-    Returns a dictionary of metadata or None if not found
+    JSON metadata takes precedence; embedded PDF metadata fills in any missing
+    fields using the application's standard key mapping (e.g., /Title → filename).
+
+    Returns a dictionary of metadata (may be empty if none found).
     """
     metadata = {}
 
@@ -76,12 +89,28 @@ def extract_metadata_from_file(file_path):
             with open(metadata_path, "r", encoding="utf-8") as f:
                 metadata = json.load(f)
                 logger.info(f"Loaded metadata from external JSON file: {metadata_path}")
-                return metadata
         except Exception as e:
             logger.warning(f"Failed to load metadata from JSON file: {str(e)}")
 
-    # TODO: For PDF files, try to extract embedded metadata using PyPDF2
-    # This would require additional dependencies, so for now we'll just check for external JSON
+    # Try to extract embedded metadata from PDF
+    if file_path.lower().endswith(".pdf") and os.path.exists(file_path):
+        try:
+            with open(file_path, "rb") as f:
+                pdf_reader = pypdf.PdfReader(f)
+                pdf_metadata = pdf_reader.metadata
+                if pdf_metadata:
+                    for key, value in pdf_metadata.items():
+                        # Remove the leading slash from PDF metadata keys (e.g., '/Title' -> 'Title')
+                        clean_key = key[1:] if key.startswith("/") else key
+                        # Map to application-specific key names where possible
+                        mapped_key = _PDF_METADATA_KEY_MAP.get(clean_key, clean_key)
+                        # Only set if not already present (JSON metadata takes precedence)
+                        if mapped_key not in metadata:
+                            metadata[mapped_key] = str(value)
+
+                    logger.info(f"Extracted embedded metadata from PDF: {file_path}")
+        except Exception as e:
+            logger.warning(f"Failed to extract metadata from PDF {file_path}: {str(e)}")
 
     return metadata
 

app/tasks/upload_to_sharepoint.py

@@ -0,0 +1,338 @@
+#!/usr/bin/env python3
+"""Upload documents to Microsoft SharePoint via the Microsoft Graph API.
+
+This module authenticates using MSAL (same OAuth2 flow as OneDrive) and
+uploads files to a configurable SharePoint Online document library using
+the chunked upload session approach for reliability with large files.
+
+Key differences from the OneDrive provider:
+- Uses ``/sites/{siteId}/drives/{driveId}`` instead of ``/me/drive``
+- Requires a SharePoint site URL to resolve the site and drive IDs
+- Targets a named document library (default: ``Documents``)
+"""
+
+import logging
+import os
+import time
+import urllib.parse
+
+import msal
+import requests
+
+from app.celery_app import celery
+from app.config import settings
+from app.tasks.retry_config import UploadTaskWithRetry
+from app.utils import log_task_progress
+
+logger = logging.getLogger(__name__)
+
+
+def get_sharepoint_token() -> str:
+    """Acquire a Microsoft Graph API access token for SharePoint.
+
+    Uses MSAL ``ConfidentialClientApplication`` with the refresh-token flow
+    (delegated permissions) or the client-credentials flow (application
+    permissions) depending on configuration.
+
+    Returns:
+        A valid access token string.
+
+    Raises:
+        ValueError: When required settings are missing or token acquisition fails.
+    """
+    if not settings.sharepoint_client_id or not settings.sharepoint_client_secret:
+        raise ValueError("SharePoint client ID and client secret must be configured")
+
+    tenant = settings.sharepoint_tenant_id or "common"
+    logger.info("Using SharePoint tenant: %s", tenant)
+
+    scopes = ["https://graph.microsoft.com/.default"]
+
+    if settings.sharepoint_refresh_token:
+        app = msal.ConfidentialClientApplication(
+            client_id=settings.sharepoint_client_id,
+            client_credential=settings.sharepoint_client_secret,
+            authority=f"https://login.microsoftonline.com/{tenant}",
+        )
+
+        logger.info("Attempting to acquire SharePoint token using refresh token")
+        token_response = app.acquire_token_by_refresh_token(
+            refresh_token=settings.sharepoint_refresh_token, scopes=scopes
+        )
+
+        if "access_token" not in token_response:
+            error = token_response.get("error", "")
+            error_desc = token_response.get("error_description", "Unknown error")
+            logger.error("Failed to get SharePoint access token: %s - %s", error, error_desc)
+            raise ValueError(f"Failed to get SharePoint access token: {error} - {error_desc}")
+
+        if "refresh_token" in token_response:
+            settings.sharepoint_refresh_token = token_response["refresh_token"]
+            logger.info("Updated SharePoint refresh token in memory")
+
+        return token_response["access_token"]
+
+    elif settings.sharepoint_tenant_id and settings.sharepoint_tenant_id != "common":
+        authority = f"https://login.microsoftonline.com/{settings.sharepoint_tenant_id}"
+        app = msal.ConfidentialClientApplication(
+            client_id=settings.sharepoint_client_id,
+            client_credential=settings.sharepoint_client_secret,
+            authority=authority,
+        )
+
+        token_response = app.acquire_token_for_client(scopes=scopes)
+
+        if "access_token" not in token_response:
+            error = token_response.get("error", "")
+            error_desc = token_response.get("error_description", "Unknown error")
+            raise ValueError(f"Failed to get SharePoint access token: {error} - {error_desc}")
+
+        return token_response["access_token"]
+
+    else:
+        raise ValueError("For SharePoint, either a refresh token or a non-'common' tenant ID is required")
+
+
+def resolve_sharepoint_drive(access_token: str, site_url: str, library_name: str) -> tuple[str, str]:
+    """Resolve the Graph API site ID and drive ID for a SharePoint site.
+
+    Args:
+        access_token: Valid Microsoft Graph API token.
+        site_url: Full SharePoint site URL, e.g.
+            ``https://tenant.sharepoint.com/sites/sitename``.
+        library_name: Display name of the document library (e.g. ``Documents``).
+
+    Returns:
+        A ``(site_id, drive_id)`` tuple.
+
+    Raises:
+        ValueError: When the site URL cannot be parsed.
+        RuntimeError: When the Graph API call fails.
+    """
+    parsed = urllib.parse.urlparse(site_url)
+    hostname = parsed.hostname
+    site_path = parsed.path.rstrip("/")
+
+    if not hostname or not site_path:
+        raise ValueError(
+            f"Invalid SharePoint site URL '{site_url}'. Expected format: https://tenant.sharepoint.com/sites/sitename"
+        )
+
+    headers = {"Authorization": f"Bearer {access_token}"}
+
+    # Resolve site ID
+    site_api_url = f"https://graph.microsoft.com/v1.0/sites/{hostname}:{site_path}"
+    logger.info("Resolving SharePoint site: %s", site_api_url)
+    resp = requests.get(site_api_url, headers=headers, timeout=settings.http_request_timeout)
+
+    if resp.status_code != 200:
+        raise RuntimeError(f"Failed to resolve SharePoint site: {resp.status_code} - {resp.text}")
+
+    site_id = resp.json()["id"]
+    logger.info("Resolved SharePoint site ID: %s", site_id)
+
+    # Resolve drive ID from the document library name
+    drives_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives"
+    resp = requests.get(drives_url, headers=headers, timeout=settings.http_request_timeout)
+
+    if resp.status_code != 200:
+        raise RuntimeError(f"Failed to list SharePoint drives: {resp.status_code} - {resp.text}")
+
+    drives = resp.json().get("value", [])
+    drive_id = None
+    for drive in drives:
+        if drive.get("name", "").lower() == library_name.lower():
+            drive_id = drive["id"]
+            break
+
+    if not drive_id:
+        available = [d.get("name") for d in drives]
+        raise RuntimeError(f"Document library '{library_name}' not found on site. Available libraries: {available}")
+
+    logger.info("Resolved SharePoint drive ID: %s (library: %s)", drive_id, library_name)
+    return site_id, drive_id
+
+
+def create_sharepoint_upload_session(
+    filename: str, folder_path: str | None, drive_id: str, site_id: str, access_token: str
+) -> str:
+    """Create a resumable upload session on a SharePoint document library.
+
+    Args:
+        filename: Name of the file to upload.
+        folder_path: Optional subfolder path inside the library.
+        drive_id: Graph API drive ID of the document library.
+        site_id: Graph API site ID.
+        access_token: Valid access token.
+
+    Returns:
+        The upload session URL for chunked PUT requests.
+
+    Raises:
+        RuntimeError: When session creation fails.
+    """
+    base_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives/{drive_id}"
+
+    if folder_path:
+        folder_path = folder_path.strip("/")
+        path_components = folder_path.split("/")
+        encoded_path = "/".join(urllib.parse.quote(component) for component in path_components)
+        encoded_filename = urllib.parse.quote(filename)
+        item_path = f"/root:/{encoded_path}/{encoded_filename}:/createUploadSession"
+    else:
+        encoded_filename = urllib.parse.quote(filename)
+        item_path = f"/root:/{encoded_filename}:/createUploadSession"
+
+    url = f"{base_url}{item_path}"
+    request_body = {"item": {"@microsoft.graph.conflictBehavior": "replace"}}
+    headers = {"Authorization": f"Bearer {access_token}", "Content-Type": "application/json"}
+
+    logger.info("Creating SharePoint upload session for %s at path %s", filename, folder_path)
+    response = requests.post(url, headers=headers, json=request_body, timeout=settings.http_request_timeout)
+
+    if response.status_code == 200:
+        upload_url = response.json().get("uploadUrl")
+        logger.info("SharePoint upload session created for %s", filename)
+        return upload_url
+    else:
+        raise RuntimeError(f"Failed to create SharePoint upload session: {response.status_code} - {response.text}")
+
+
+def upload_large_file_sharepoint(file_path: str, upload_url: str) -> dict:
+    """Upload a file to SharePoint using a chunked upload session.
+
+    Args:
+        file_path: Local path to the file.
+        upload_url: The upload session URL from ``create_sharepoint_upload_session``.
+
+    Returns:
+        The Graph API response dict containing file metadata.
+
+    Raises:
+        RuntimeError: When a chunk upload fails after retries.
+    """
+    file_size = os.path.getsize(file_path)
+    chunk_size = 10 * 1024 * 1024  # 10 MB
+
+    response = None
+    with open(file_path, "rb") as f:
+        chunk_number = 0
+        while True:
+            chunk = f.read(chunk_size)
+            if not chunk:
+                break
+
+            chunk_start = chunk_number * chunk_size
+            chunk_end = chunk_start + len(chunk) - 1
+            content_range = f"bytes {chunk_start}-{chunk_end}/{file_size}"
+
+            headers = {"Content-Length": str(len(chunk)), "Content-Range": content_range}
+
+            max_retries = 3
+            retry_delay = 2
+
+            for attempt in range(max_retries):
+                try:
+                    response = requests.put(
+                        upload_url, headers=headers, data=chunk, timeout=settings.http_request_timeout
+                    )
+                    if response.status_code in (201, 202):
+                        break
+                    else:
+                        logger.warning(
+                            "SharePoint chunk upload failed (attempt %d): %d", attempt + 1, response.status_code
+                        )
+                        if attempt < max_retries - 1:
+                            time.sleep(retry_delay * (attempt + 1))
+                except Exception as e:
+                    logger.warning("SharePoint chunk upload error (attempt %d): %s", attempt + 1, str(e))
+                    if attempt < max_retries - 1:
+                        time.sleep(retry_delay * (attempt + 1))
+
+            if response is None or response.status_code not in (201, 202):
+                status = response.status_code if response else "no response"
+                text = response.text if response else ""
+                raise RuntimeError(f"Failed to upload chunk after {max_retries} attempts: {status} - {text}")
+
+            chunk_number += 1
+
+    return response.json() if response else {}
+
+
+@celery.task(base=UploadTaskWithRetry, bind=True)
+def upload_to_sharepoint(self, file_path: str, file_id: int = None, folder_override: str = None):
+    """Upload a file to SharePoint Online.
+
+    Args:
+        file_path: Path to the file to upload.
+        file_id: Optional file ID to associate with logs.
+        folder_override: Optional folder path override.
+
+    Returns:
+        A dict with upload status and file details.
+
+    Raises:
+        FileNotFoundError: When the file does not exist.
+        ValueError: When SharePoint is not configured.
+        RuntimeError: When the upload fails.
+    """
+    task_id = self.request.id
+    logger.info("[%s] Starting SharePoint upload: %s", task_id, file_path)
+    log_task_progress(
+        task_id,
+        "upload_to_sharepoint",
+        "in_progress",
+        f"Uploading to SharePoint: {os.path.basename(file_path)}",
+        file_id=file_id,
+    )
+
+    if not os.path.exists(file_path):
+        error_msg = f"File not found: {file_path}"
+        logger.error("[%s] %s", task_id, error_msg)
+        log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
+        raise FileNotFoundError(error_msg)
+
+    filename = os.path.basename(file_path)
+
+    if not settings.sharepoint_client_id:
+        error_msg = "SharePoint client ID is not configured"
+        logger.error("[%s] %s", task_id, error_msg)
+        log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
+        raise ValueError(error_msg)
+
+    if not settings.sharepoint_site_url:
+        error_msg = "SharePoint site URL is not configured"
+        logger.error("[%s] %s", task_id, error_msg)
+        log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
+        raise ValueError(error_msg)
+
+    try:
+        access_token = get_sharepoint_token()
+
+        library_name = settings.sharepoint_document_library or "Documents"
+        site_id, drive_id = resolve_sharepoint_drive(access_token, settings.sharepoint_site_url, library_name)
+
+        folder_path = folder_override if folder_override is not None else settings.sharepoint_folder_path
+
+        upload_url = create_sharepoint_upload_session(filename, folder_path, drive_id, site_id, access_token)
+        result = upload_large_file_sharepoint(file_path, upload_url)
+
+        web_url = result.get("webUrl", "Not available")
+        logger.info("[%s] Successfully uploaded %s to SharePoint", task_id, filename)
+        logger.info("[%s] File accessible at: %s", task_id, web_url)
+        log_task_progress(
+            task_id, "upload_to_sharepoint", "success", f"Uploaded to SharePoint: {filename}", file_id=file_id
+        )
+
+        return {
+            "status": "Completed",
+            "file_path": file_path,
+            "sharepoint_path": f"{folder_path or ''}/{filename}",
+            "web_url": web_url,
+        }
+
+    except Exception as e:
+        error_msg = f"Failed to upload {filename} to SharePoint: {str(e)}"
+        logger.error("[%s] %s", task_id, error_msg)
+        log_task_progress(task_id, "upload_to_sharepoint", "failure", error_msg, file_id=file_id)
+        raise RuntimeError(error_msg) from e

app/tasks/upload_to_user_integration.py

@@ -555,8 +555,9 @@ def _upload_rclone(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], t
     dest = dest.replace("//", "/")
 
     try:
+        # SECURITY: Separate options from positional arguments using -- to prevent command injection
         result = subprocess.run(  # nosec B603  # noqa: S603 S607
-            ["rclone", "copyto", f"--config={conf_path}", file_path, dest],  # noqa: S603 S607
+            ["rclone", "copyto", f"--config={conf_path}", "--", file_path, dest],  # noqa: S603 S607
             capture_output=True,
             text=True,
             timeout=300,
@@ -571,6 +572,113 @@ def _upload_rclone(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], t
     return {"status": "Completed", "rclone_dest": dest}
 
 
+def _upload_sharepoint(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to SharePoint using per-user MSAL credentials."""
+    import urllib.parse
+
+    import msal
+    import requests as _requests
+
+    client_id = creds.get("client_id") or ""
+    client_secret = creds.get("client_secret") or ""
+    refresh_token = creds.get("refresh_token") or ""
+    tenant = cfg.get("tenant_id") or "common"
+    site_url = cfg.get("site_url") or ""
+    library_name = cfg.get("document_library") or "Documents"
+    folder_path = cfg.get("folder_path") or ""
+
+    if not (client_id and client_secret):
+        raise ValueError("SharePoint integration is missing client_id or client_secret in credentials")
+    if not site_url:
+        raise ValueError("SharePoint integration is missing site_url in config")
+
+    scopes = ["https://graph.microsoft.com/.default"]
+    msal_app = msal.ConfidentialClientApplication(
+        client_id=client_id,
+        client_credential=client_secret,
+        authority=f"https://login.microsoftonline.com/{tenant}",
+    )
+
+    if refresh_token:
+        token_resp = msal_app.acquire_token_by_refresh_token(refresh_token=refresh_token, scopes=scopes)
+    else:
+        token_resp = msal_app.acquire_token_for_client(scopes=scopes)
+
+    if "access_token" not in token_resp:
+        raise ValueError(f"SharePoint token acquisition failed: {token_resp.get('error_description', 'unknown')}")
+
+    access_token = token_resp["access_token"]
+    headers = {"Authorization": f"Bearer {access_token}"}
+
+    # Resolve site ID
+    parsed = urllib.parse.urlparse(site_url)
+    hostname = parsed.hostname
+    site_path = parsed.path.rstrip("/")
+    if not hostname or not site_path:
+        raise ValueError(f"Invalid SharePoint site URL: {site_url}")
+
+    resp = _requests.get(f"https://graph.microsoft.com/v1.0/sites/{hostname}:{site_path}", headers=headers, timeout=30)
+    resp.raise_for_status()
+    site_id = resp.json()["id"]
+
+    # Resolve drive ID
+    resp = _requests.get(f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives", headers=headers, timeout=30)
+    resp.raise_for_status()
+    drive_id = None
+    for drive in resp.json().get("value", []):
+        if drive.get("name", "").lower() == library_name.lower():
+            drive_id = drive["id"]
+            break
+    if not drive_id:
+        raise RuntimeError(f"Document library '{library_name}' not found on SharePoint site")
+
+    filename = os.path.basename(file_path)
+
+    # Build upload-session URL
+    base_url = f"https://graph.microsoft.com/v1.0/sites/{site_id}/drives/{drive_id}"
+    if folder_path:
+        folder_path = folder_path.strip("/")
+        encoded_path = "/".join(urllib.parse.quote(p) for p in folder_path.split("/"))
+        encoded_file = urllib.parse.quote(filename)
+        item_path = f"/root:/{encoded_path}/{encoded_file}:/createUploadSession"
+    else:
+        encoded_file = urllib.parse.quote(filename)
+        item_path = f"/root:/{encoded_file}:/createUploadSession"
+
+    session_url = f"{base_url}{item_path}"
+    session_headers = {"Authorization": f"Bearer {access_token}", "Content-Type": "application/json"}
+    resp = _requests.post(
+        session_url,
+        headers=session_headers,
+        json={"item": {"@microsoft.graph.conflictBehavior": "replace"}},
+        timeout=30,
+    )
+    resp.raise_for_status()
+    upload_url = resp.json()["uploadUrl"]
+
+    file_size = os.path.getsize(file_path)
+    chunk_size = 10 * 1024 * 1024
+    with open(file_path, "rb") as fh:
+        chunk_num = 0
+        while True:
+            chunk = fh.read(chunk_size)
+            if not chunk:
+                break
+            start = chunk_num * chunk_size
+            end = start + len(chunk) - 1
+            upload_headers = {
+                "Content-Length": str(len(chunk)),
+                "Content-Range": f"bytes {start}-{end}/{file_size}",
+            }
+            upload_resp = _requests.put(upload_url, headers=upload_headers, data=chunk, timeout=120)
+            if upload_resp.status_code not in (201, 202):
+                raise RuntimeError(f"SharePoint chunk upload failed: {upload_resp.status_code}")
+            chunk_num += 1
+
+    logger.info("[%s] SharePoint upload complete: %s/%s", task_id, folder_path, filename)
+    return {"status": "Completed", "sharepoint_folder": folder_path, "filename": filename}
+
+
 def _upload_icloud(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
     """Upload *file_path* to iCloud Drive using per-user credentials.
 
@@ -615,6 +723,7 @@ _UPLOAD_HANDLERS = {
     IntegrationType.PAPERLESS: _upload_paperless,
     IntegrationType.EMAIL: _upload_email,
     IntegrationType.RCLONE: _upload_rclone,
+    IntegrationType.SHAREPOINT: _upload_sharepoint,
     IntegrationType.ICLOUD: _upload_icloud,
 }
 

app/tasks/upload_with_rclone.py

@@ -55,12 +55,12 @@ def upload_with_rclone(self, file_path: str, destination: str):
 
     try:
         # Ensure the remote path exists (create folders if needed)
-        mkdir_cmd = ["rclone", "mkdir", "--config", rclone_config_path, destination]
+        mkdir_cmd = ["rclone", "mkdir", "--config", rclone_config_path, "--", destination]
 
         subprocess.run(mkdir_cmd, check=True, capture_output=True)  # noqa: S603
 
         # Construct the upload command
-        upload_cmd = ["rclone", "copy", "--config", rclone_config_path, file_path, destination, "--progress"]
+        upload_cmd = ["rclone", "copy", "--config", rclone_config_path, "--progress", "--", file_path, destination]
 
         log_task_progress(task_id, "rclone_upload", "in_progress", f"Executing rclone copy to {destination}")
 
@@ -71,7 +71,7 @@ def upload_with_rclone(self, file_path: str, destination: str):
         if result.returncode == 0:
             # Try to get a public link if possible
             try:
-                link_cmd = ["rclone", "link", "--config", rclone_config_path, f"{destination}/{filename}"]
+                link_cmd = ["rclone", "link", "--config", rclone_config_path, "--", f"{destination}/{filename}"]
                 link_result = subprocess.run(link_cmd, capture_output=True, text=True, check=False)  # noqa: S603
                 public_url = link_result.stdout.strip() if link_result.returncode == 0 else None
             except (subprocess.SubprocessError, OSError) as e:

app/utils/allowed_types.py

@@ -68,6 +68,8 @@ IMAGE_MIME_TYPES: set[str] = {
     "image/tiff",
     "image/webp",
     "image/svg+xml",
+    "image/heic",
+    "image/heif",
 }
 
 # ---------------------------------------------------------------------------
@@ -124,6 +126,8 @@ ALLOWED_EXTENSIONS: set[str] = {
     ".tif",
     ".webp",
     ".svg",
+    ".heic",
+    ".heif",
     # Web
     ".html",
     ".htm",
@@ -234,7 +238,7 @@ FILE_TYPE_CATEGORIES: dict[str, dict] = {
     },
     "images": {
         "label": "Images",
-        "description": "Image files (.jpg, .png, .gif, .bmp, .tiff, .webp, .svg)",
+        "description": "Image files (.jpg, .png, .gif, .bmp, .tiff, .webp, .svg, .heic, .heif)",
         "mime_types": frozenset(
             {
                 "image/jpeg",
@@ -245,6 +249,8 @@ FILE_TYPE_CATEGORIES: dict[str, dict] = {
                 "image/tiff",
                 "image/webp",
                 "image/svg+xml",
+                "image/heic",
+                "image/heif",
             }
         ),
         "extensions": frozenset(
@@ -258,6 +264,8 @@ FILE_TYPE_CATEGORIES: dict[str, dict] = {
                 ".tif",
                 ".webp",
                 ".svg",
+                ".heic",
+                ".heif",
             }
         ),
     },

app/utils/automation_hooks.py

@@ -0,0 +1,188 @@
+"""Automation hook utilities for Zapier / Make.com integration.
+
+Provides helpers to build Zapier-compatible flat payloads, query active
+automation hook subscriptions, and fan-out event delivery to all matching
+hooks via Celery tasks.
+
+The payload format is intentionally *flat* (no nested ``data`` key) so that
+Zapier and Make.com can map fields without JSONPath expressions.  An ``id``
+field is included for Zapier deduplication.
+"""
+
+import json
+import logging
+import time
+import uuid
+from typing import Any
+
+from app.config import settings
+from app.database import SessionLocal
+from app.models import AutomationHook
+from app.utils.webhook import VALID_EVENTS
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# Payload helpers
+# ---------------------------------------------------------------------------
+
+
+def build_zapier_payload(event: str, data: dict[str, Any]) -> dict[str, Any]:
+    """Build a flat, Zapier-compatible webhook payload.
+
+    Zapier works best with flat JSON objects that include an ``id`` field
+    for deduplication.  This function merges event metadata into the
+    top-level object alongside the event-specific *data*.
+
+    Args:
+        event: The event name (e.g. ``document.processed``).
+        data: Event-specific key/value pairs.
+
+    Returns:
+        A flat dictionary suitable for Zapier / Make.com consumption.
+    """
+    return {
+        "id": f"evt_{uuid.uuid4().hex[:16]}",
+        "event": event,
+        "timestamp": time.time(),
+        **data,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Sample payloads (used by the /triggers/sample endpoint)
+# ---------------------------------------------------------------------------
+
+#: Example payloads that Zapier uses for field-mapping during Zap creation.
+SAMPLE_PAYLOADS: dict[str, dict[str, Any]] = {
+    "document.uploaded": {
+        "id": "evt_sample0001",
+        "event": "document.uploaded",
+        "timestamp": 1710000000.0,
+        "document_id": 42,
+        "filename": "invoice_2024.pdf",
+        "content_type": "application/pdf",
+        "size_bytes": 204800,
+        "owner_id": "user@example.com",
+    },
+    "document.processed": {
+        "id": "evt_sample0002",
+        "event": "document.processed",
+        "timestamp": 1710000060.0,
+        "document_id": 42,
+        "filename": "invoice_2024.pdf",
+        "status": "processed",
+        "title": "Invoice #1234",
+        "owner_id": "user@example.com",
+    },
+    "document.failed": {
+        "id": "evt_sample0003",
+        "event": "document.failed",
+        "timestamp": 1710000120.0,
+        "document_id": 42,
+        "filename": "corrupt.pdf",
+        "status": "failed",
+        "error": "Unable to extract text from document",
+        "owner_id": "user@example.com",
+    },
+    "user.signup": {
+        "id": "evt_sample0004",
+        "event": "user.signup",
+        "timestamp": 1710000180.0,
+        "user_id": "newuser@example.com",
+        "display_name": "Jane Doe",
+    },
+    "user.plan_changed": {
+        "id": "evt_sample0005",
+        "event": "user.plan_changed",
+        "timestamp": 1710000240.0,
+        "user_id": "user@example.com",
+        "old_tier": "free",
+        "new_tier": "pro",
+    },
+    "user.payment_issue": {
+        "id": "evt_sample0006",
+        "event": "user.payment_issue",
+        "timestamp": 1710000300.0,
+        "user_id": "user@example.com",
+        "issue": "Credit card declined",
+    },
+}
+
+
+# ---------------------------------------------------------------------------
+# Database queries
+# ---------------------------------------------------------------------------
+
+
+def get_active_hooks_for_event(event: str) -> list[dict[str, Any]]:
+    """Return all active automation hooks subscribed to *event*.
+
+    Args:
+        event: The event name to filter on.
+
+    Returns:
+        A list of dicts with ``id``, ``target_url``, ``secret``, and
+        ``events`` keys.
+    """
+    db = SessionLocal()
+    try:
+        hooks = db.query(AutomationHook).filter(AutomationHook.is_active.is_(True)).all()
+        result: list[dict[str, Any]] = []
+        for hook in hooks:
+            try:
+                subscribed = json.loads(hook.events)
+            except (json.JSONDecodeError, TypeError):
+                subscribed = []
+            if event in subscribed:
+                result.append(
+                    {
+                        "id": hook.id,
+                        "target_url": hook.target_url,
+                        "secret": hook.secret,
+                        "events": subscribed,
+                    }
+                )
+        return result
+    finally:
+        db.close()
+
+
+# ---------------------------------------------------------------------------
+# Dispatch
+# ---------------------------------------------------------------------------
+
+
+def dispatch_automation_hooks(event: str, data: dict[str, Any]) -> None:
+    """Fan-out an event to all matching active automation hooks.
+
+    Builds a Zapier-compatible flat payload and queues a Celery task for
+    each matching hook so delivery is asynchronous with automatic retries.
+
+    Args:
+        event: Event name (must be in :data:`VALID_EVENTS`).
+        data: Event-specific payload data.
+    """
+    if not settings.automation_hooks_enabled:
+        return
+
+    if event not in VALID_EVENTS:
+        logger.warning("Ignoring unknown automation hook event: %s", event)
+        return
+
+    hooks = get_active_hooks_for_event(event)
+    if not hooks:
+        logger.debug("No active automation hooks for event %s", event)
+        return
+
+    payload = build_zapier_payload(event, data)
+
+    from app.tasks.automation_tasks import deliver_automation_hook_task
+
+    for hook in hooks:
+        try:
+            deliver_automation_hook_task.delay(hook["target_url"], payload, hook["secret"])
+            logger.debug("Queued automation hook delivery to %s for event %s", hook["target_url"], event)
+        except Exception as exc:
+            logger.error("Failed to queue automation hook to %s: %s", hook["target_url"], exc)

app/utils/classification_rules.py

@@ -0,0 +1,378 @@
+"""
+Rule-based document classification engine.
+
+Provides pre-built categories and a rule matcher that classifies documents
+using filename patterns, content keywords, and metadata fields.  Custom
+rules stored in the database are evaluated alongside the built-in defaults.
+
+Usage::
+
+    from app.utils.classification_rules import classify_document
+
+    result = classify_document(
+        filename="2024-03-01_Invoice_Acme.pdf",
+        text="Invoice total: $1,234.56",
+        metadata={"absender": "Acme Corp"},
+        custom_rules=custom_rules_from_db,
+    )
+    # result -> ClassificationResult(category="invoice", confidence=85, matched_rules=[...])
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from dataclasses import dataclass, field
+from typing import Any
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Pre-built categories
+# ---------------------------------------------------------------------------
+
+#: Canonical category names recognized by the system.  Users may also define
+#: their own categories via custom rules.
+BUILTIN_CATEGORIES: dict[str, str] = {
+    "invoice": "Invoice",
+    "contract": "Contract",
+    "receipt": "Receipt",
+    "letter": "Letter",
+    "report": "Report",
+    "bank_statement": "Bank Statement",
+    "tax_document": "Tax Document",
+    "insurance": "Insurance Document",
+    "payslip": "Payslip",
+    "unknown": "Unknown",
+}
+
+# ---------------------------------------------------------------------------
+# Rule type constants
+# ---------------------------------------------------------------------------
+
+RULE_TYPE_FILENAME = "filename_pattern"
+RULE_TYPE_CONTENT = "content_keyword"
+RULE_TYPE_METADATA = "metadata_match"
+
+
+# ---------------------------------------------------------------------------
+# Data classes
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class ClassificationRule:
+    """A single classification rule."""
+
+    name: str
+    category: str
+    rule_type: str  # filename_pattern | content_keyword | metadata_match
+    pattern: str  # regex for filename, keyword(s) for content, "field=value" for metadata
+    priority: int = 0  # higher = evaluated first
+    case_sensitive: bool = False
+
+    def __post_init__(self) -> None:
+        if self.rule_type not in (RULE_TYPE_FILENAME, RULE_TYPE_CONTENT, RULE_TYPE_METADATA):
+            raise ValueError(f"Invalid rule_type: {self.rule_type!r}")
+
+
+@dataclass
+class MatchedRule:
+    """Records which rule matched and why."""
+
+    rule_name: str
+    rule_type: str
+    category: str
+    confidence: int
+
+
+@dataclass
+class ClassificationResult:
+    """The outcome of running the classification engine on a document."""
+
+    category: str
+    confidence: int  # 0 – 100
+    matched_rules: list[MatchedRule] = field(default_factory=list)
+
+
+# ---------------------------------------------------------------------------
+# Built-in rules
+# ---------------------------------------------------------------------------
+
+BUILTIN_RULES: list[ClassificationRule] = [
+    # ── Invoice ───────────────────────────────────────────────────────────
+    ClassificationRule("builtin_invoice_filename", "invoice", RULE_TYPE_FILENAME, r"(?i)invoice|rechnung|facture"),
+    ClassificationRule(
+        "builtin_invoice_content",
+        "invoice",
+        RULE_TYPE_CONTENT,
+        "invoice number|invoice total|amount due|rechnung|rechnungsnummer|total amount|bill to",
+    ),
+    ClassificationRule("builtin_invoice_metadata", "invoice", RULE_TYPE_METADATA, "document_type=Invoice"),
+    ClassificationRule(
+        "builtin_invoice_kommunikationsart", "invoice", RULE_TYPE_METADATA, "kommunikationsart=Rechnung"
+    ),
+    # ── Contract ──────────────────────────────────────────────────────────
+    ClassificationRule("builtin_contract_filename", "contract", RULE_TYPE_FILENAME, r"(?i)contract|vertrag|agreement"),
+    ClassificationRule(
+        "builtin_contract_content",
+        "contract",
+        RULE_TYPE_CONTENT,
+        "hereby agrees|terms and conditions|vertrag|agreement between|party agrees|effective date",
+    ),
+    ClassificationRule("builtin_contract_metadata", "contract", RULE_TYPE_METADATA, "document_type=Contract"),
+    ClassificationRule(
+        "builtin_contract_kommunikationsart", "contract", RULE_TYPE_METADATA, "kommunikationsart=Vertrag"
+    ),
+    # ── Receipt ───────────────────────────────────────────────────────────
+    ClassificationRule("builtin_receipt_filename", "receipt", RULE_TYPE_FILENAME, r"(?i)receipt|quittung|beleg"),
+    ClassificationRule(
+        "builtin_receipt_content",
+        "receipt",
+        RULE_TYPE_CONTENT,
+        "receipt|quittung|payment received|thank you for your purchase|transaction id",
+    ),
+    ClassificationRule("builtin_receipt_metadata", "receipt", RULE_TYPE_METADATA, "document_type=Receipt"),
+    ClassificationRule(
+        "builtin_receipt_kommunikationsart", "receipt", RULE_TYPE_METADATA, "kommunikationsart=Quittung"
+    ),
+    # ── Letter ────────────────────────────────────────────────────────────
+    ClassificationRule("builtin_letter_filename", "letter", RULE_TYPE_FILENAME, r"(?i)letter|brief|schreiben"),
+    ClassificationRule(
+        "builtin_letter_content",
+        "letter",
+        RULE_TYPE_CONTENT,
+        "dear sir|dear madam|sehr geehrte|to whom it may concern|sincerely|mit freundlichen",
+    ),
+    # ── Report ────────────────────────────────────────────────────────────
+    ClassificationRule("builtin_report_filename", "report", RULE_TYPE_FILENAME, r"(?i)report|bericht"),
+    ClassificationRule(
+        "builtin_report_content",
+        "report",
+        RULE_TYPE_CONTENT,
+        "executive summary|table of contents|annual report|quarterly report|findings",
+    ),
+    # ── Bank statement ────────────────────────────────────────────────────
+    ClassificationRule(
+        "builtin_bank_filename",
+        "bank_statement",
+        RULE_TYPE_FILENAME,
+        r"(?i)bank.?statement|kontoauszug",
+    ),
+    ClassificationRule(
+        "builtin_bank_content",
+        "bank_statement",
+        RULE_TYPE_CONTENT,
+        "account statement|kontoauszug|opening balance|closing balance|account number",
+    ),
+    ClassificationRule(
+        "builtin_bank_kommunikationsart", "bank_statement", RULE_TYPE_METADATA, "kommunikationsart=Kontoauszug"
+    ),
+    # ── Tax document ──────────────────────────────────────────────────────
+    ClassificationRule("builtin_tax_filename", "tax_document", RULE_TYPE_FILENAME, r"(?i)tax|steuer|steuerbescheid"),
+    ClassificationRule(
+        "builtin_tax_content",
+        "tax_document",
+        RULE_TYPE_CONTENT,
+        "tax return|steuerbescheid|taxable income|finanzamt|tax assessment",
+    ),
+    # ── Insurance ─────────────────────────────────────────────────────────
+    ClassificationRule(
+        "builtin_insurance_filename", "insurance", RULE_TYPE_FILENAME, r"(?i)insurance|versicherung|police"
+    ),
+    ClassificationRule(
+        "builtin_insurance_content",
+        "insurance",
+        RULE_TYPE_CONTENT,
+        "insurance policy|versicherung|policennummer|coverage|premium|deductible",
+    ),
+    # ── Payslip ───────────────────────────────────────────────────────────
+    ClassificationRule(
+        "builtin_payslip_filename", "payslip", RULE_TYPE_FILENAME, r"(?i)payslip|gehaltsabrechnung|lohnabrechnung"
+    ),
+    ClassificationRule(
+        "builtin_payslip_content",
+        "payslip",
+        RULE_TYPE_CONTENT,
+        "gross salary|net salary|gehaltsabrechnung|lohnabrechnung|bruttolohn|nettolohn",
+    ),
+]
+
+
+# ---------------------------------------------------------------------------
+# Confidence scoring
+# ---------------------------------------------------------------------------
+
+#: Base confidence for each rule type when it matches.
+_CONFIDENCE_MAP: dict[str, int] = {
+    RULE_TYPE_FILENAME: 60,
+    RULE_TYPE_CONTENT: 70,
+    RULE_TYPE_METADATA: 90,
+}
+
+#: Extra confidence per additional matching rule of the same category (capped).
+_CONFIDENCE_BONUS_PER_EXTRA_RULE = 10
+
+
+# ---------------------------------------------------------------------------
+# Matching helpers
+# ---------------------------------------------------------------------------
+
+
+def _match_filename(rule: ClassificationRule, filename: str) -> bool:
+    """Return True if *rule.pattern* (regex) matches anywhere in *filename*."""
+    if not filename:
+        return False
+    flags = 0 if rule.case_sensitive else re.IGNORECASE
+    return bool(re.search(rule.pattern, filename, flags))
+
+
+def _match_content(rule: ClassificationRule, text: str) -> bool:
+    """Return True if any keyword in *rule.pattern* appears in *text*.
+
+    Keywords are separated by ``|`` (pipe).
+    """
+    if not text:
+        return False
+    keywords = [kw.strip() for kw in rule.pattern.split("|") if kw.strip()]
+    text_lower = text if rule.case_sensitive else text.lower()
+    return any((kw if rule.case_sensitive else kw.lower()) in text_lower for kw in keywords)
+
+
+def _match_metadata(rule: ClassificationRule, metadata: dict[str, Any] | None) -> bool:
+    """Return True if *rule.pattern* (``field=value``) matches *metadata*.
+
+    Pattern format: ``field_name=expected_value``.
+    """
+    if not metadata:
+        return False
+    if "=" not in rule.pattern:
+        return False
+    field_name, expected_value = rule.pattern.split("=", 1)
+    actual = metadata.get(field_name.strip())
+    if actual is None:
+        return False
+    if rule.case_sensitive:
+        return str(actual) == expected_value.strip()
+    return str(actual).lower() == expected_value.strip().lower()
+
+
+_MATCHERS: dict[str, tuple] = {
+    RULE_TYPE_FILENAME: (_match_filename, "filename"),
+    RULE_TYPE_CONTENT: (_match_content, "text"),
+    RULE_TYPE_METADATA: (_match_metadata, "metadata"),
+}
+
+
+def _evaluate_rule(
+    rule: ClassificationRule,
+    filename: str,
+    text: str,
+    metadata: dict[str, Any] | None,
+) -> MatchedRule | None:
+    """Evaluate a single rule against the document.  Return a :class:`MatchedRule` on match."""
+    entry = _MATCHERS.get(rule.rule_type)
+    if entry is None:
+        return None
+
+    matcher, arg_key = entry
+    arg_map = {"filename": filename, "text": text, "metadata": metadata}
+    matched = matcher(rule, arg_map[arg_key])
+
+    if matched:
+        return MatchedRule(
+            rule_name=rule.name,
+            rule_type=rule.rule_type,
+            category=rule.category,
+            confidence=_CONFIDENCE_MAP.get(rule.rule_type, 50),
+        )
+    return None
+
+
+# ---------------------------------------------------------------------------
+# Public API
+# ---------------------------------------------------------------------------
+
+
+def classify_document(
+    filename: str = "",
+    text: str = "",
+    metadata: dict[str, Any] | None = None,
+    custom_rules: list[ClassificationRule] | None = None,
+) -> ClassificationResult:
+    """Classify a document by evaluating built-in and custom rules.
+
+    Rules are evaluated in priority order (highest first, then built-in before
+    custom for the same priority).  The category with the most rule matches
+    wins; ties are broken by cumulative confidence.
+
+    Args:
+        filename: Original filename of the document.
+        text: Extracted / OCR text of the document.
+        metadata: Previously-extracted AI metadata dict (e.g. from ``ai_metadata``).
+        custom_rules: Optional list of user-defined :class:`ClassificationRule` objects.
+
+    Returns:
+        A :class:`ClassificationResult` with the best matching category,
+        overall confidence score, and the list of rules that fired.
+    """
+    all_rules = list(BUILTIN_RULES)
+    if custom_rules:
+        all_rules.extend(custom_rules)
+
+    # Sort by priority descending (higher priority first)
+    all_rules.sort(key=lambda r: r.priority, reverse=True)
+
+    matches: list[MatchedRule] = []
+    for rule in all_rules:
+        result = _evaluate_rule(rule, filename, text, metadata)
+        if result is not None:
+            matches.append(result)
+
+    if not matches:
+        return ClassificationResult(category="unknown", confidence=0, matched_rules=[])
+
+    # Aggregate by category: pick the one with the most matches, then highest
+    # cumulative confidence as tiebreaker.
+    category_scores: dict[str, list[MatchedRule]] = {}
+    for m in matches:
+        category_scores.setdefault(m.category, []).append(m)
+
+    best_category = max(
+        category_scores,
+        key=lambda cat: (len(category_scores[cat]), sum(m.confidence for m in category_scores[cat])),
+    )
+
+    best_matches = category_scores[best_category]
+    base_confidence = max(m.confidence for m in best_matches)
+    bonus = min(
+        (len(best_matches) - 1) * _CONFIDENCE_BONUS_PER_EXTRA_RULE,
+        100 - base_confidence,
+    )
+    final_confidence = min(base_confidence + bonus, 100)
+
+    return ClassificationResult(
+        category=best_category,
+        confidence=final_confidence,
+        matched_rules=best_matches,
+    )
+
+
+def db_rule_to_engine_rule(db_rule: Any) -> ClassificationRule:
+    """Convert a database ``ClassificationRuleModel`` row to an engine :class:`ClassificationRule`.
+
+    Args:
+        db_rule: A SQLAlchemy model instance with ``name``, ``category``,
+            ``rule_type``, ``pattern``, ``priority``, and ``case_sensitive`` attributes.
+
+    Returns:
+        A :class:`ClassificationRule` dataclass instance.
+    """
+    return ClassificationRule(
+        name=db_rule.name,
+        category=db_rule.category,
+        rule_type=db_rule.rule_type,
+        pattern=db_rule.pattern,
+        priority=db_rule.priority,
+        case_sensitive=getattr(db_rule, "case_sensitive", False),
+    )

app/utils/config_validator/providers.py

@@ -296,6 +296,28 @@ def get_provider_status() -> dict[str, dict[str, object]]:
         },
     }
 
+    # Check SharePoint configuration
+    providers["SharePoint"] = {
+        "name": "SharePoint",
+        "icon": "fa-brands fa-microsoft",
+        "configured": bool(
+            getattr(settings, "sharepoint_client_id", None)
+            and getattr(settings, "sharepoint_client_secret", None)
+            and getattr(settings, "sharepoint_site_url", None)
+        ),
+        "enabled": True,
+        "description": "Store documents in Microsoft SharePoint Online",
+        "details": {
+            "client_id": getattr(settings, "sharepoint_client_id", "Not set"),
+            "client_secret": mask_sensitive_value(getattr(settings, "sharepoint_client_secret", None)),
+            "tenant_id": getattr(settings, "sharepoint_tenant_id", "Not set"),
+            "refresh_token": mask_sensitive_value(getattr(settings, "sharepoint_refresh_token", None)),
+            "site_url": getattr(settings, "sharepoint_site_url", "Not set"),
+            "document_library": getattr(settings, "sharepoint_document_library", "Not set"),
+            "folder_path": getattr(settings, "sharepoint_folder_path", "Not set"),
+        },
+    }
+
     # Check S3 configuration
     providers["S3 Storage"] = {
         "name": "S3 Storage",

app/utils/db_migrate.py

@@ -12,6 +12,7 @@ The utility:
 """
 
 import logging
+import re
 from typing import Any
 
 from sqlalchemy import MetaData, create_engine, inspect, text
@@ -84,6 +85,9 @@ def preview_migration(source_url: str) -> dict[str, Any]:
         total = 0
         with src_engine.connect() as conn:
             for table_name in tables:
+                if not re.match(r"^[a-zA-Z0-9_]+$", table_name):
+                    logger.warning(f"Skipping table with invalid name format: {table_name}")
+                    continue
                 # table_name is safe — sourced from inspect().get_table_names(), not user input
                 quoted_table = conn.dialect.identifier_preparer.quote(table_name)
                 row = conn.execute(text(f"SELECT COUNT(*) FROM {quoted_table}")).fetchone()  # noqa: S608

app/utils/env_utils.py

@@ -0,0 +1,55 @@
+import logging
+import os
+from typing import Dict
+
+logger = logging.getLogger(__name__)
+
+
+def update_env_file(settings_to_update: Dict[str, str]) -> bool:
+    """
+    Updates the .env file with the given settings (best-effort).
+    Creates or modifies existing keys.
+
+    Args:
+        settings_to_update: A dictionary mapping uppercase env var names to their new string values.
+
+    Returns:
+        True if the file was successfully updated, False otherwise.
+    """
+    try:
+        env_path = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), ".env")
+        if not os.path.exists(env_path):
+            logger.warning(f".env file not found at {env_path}, skipping file write")
+            return False
+
+        logger.info(f"Updating settings in {env_path}")
+
+        with open(env_path, "r") as f:
+            env_lines = f.readlines()
+
+        updated = set()
+        new_env_lines = []
+        for line in env_lines:
+            stripped_line = line.rstrip()
+            is_updated = False
+            for key, value in settings_to_update.items():
+                if stripped_line.startswith(f"{key}=") or stripped_line.startswith(f"# {key}="):
+                    new_env_lines.append(f"{key}={value}")
+                    updated.add(key)
+                    is_updated = True
+                    break
+            if not is_updated:
+                new_env_lines.append(stripped_line)
+
+        for key, value in settings_to_update.items():
+            if key not in updated:
+                new_env_lines.append(f"{key}={value}")
+
+        with open(env_path, "w") as f:
+            f.write("\n".join(new_env_lines) + "\n")
+
+        logger.info("Successfully updated settings in .env file")
+        return True
+    except Exception as env_err:
+        logger.warning(f"Failed to write .env file (non-fatal): {env_err}")
+        return False

app/utils/file_operations.py

@@ -7,8 +7,19 @@ def hash_file(filepath: str | Path, chunk_size: int = 65536) -> str:
     Returns the SHA-256 hash of the file at 'filepath'.
     Reads the file in chunks to handle large files efficiently.
     """
+    from app.config import settings
+
+    filepath_obj = Path(filepath).resolve()
+    workdir_obj = Path(settings.workdir).resolve()
+
+    # Security check: Ensure the resolved path is strictly within the allowed workdir
+    try:
+        filepath_obj.relative_to(workdir_obj)
+    except ValueError:
+        raise FileNotFoundError(f"Access denied: path traversal attempt or file outside workdir '{filepath}'")
+
     sha256 = hashlib.sha256()
-    with open(filepath, "rb") as f:
+    with open(filepath_obj, "rb") as f:
         while True:
             data = f.read(chunk_size)
             if not data:

app/utils/i18n.py

@@ -34,89 +34,91 @@ logger = logging.getLogger(__name__)
 
 SUPPORTED_LANGUAGES: list[dict[str, str]] = [
     # --- Tier 1: Primary European languages ---
-    {"code": "en", "name": "English", "native": "English", "flag": "🇬🇧"},
-    {"code": "de", "name": "German", "native": "Deutsch", "flag": "🇩🇪"},
-    {"code": "fr", "name": "French", "native": "Français", "flag": "🇫🇷"},
-    {"code": "es", "name": "Spanish", "native": "Español", "flag": "🇪🇸"},
-    {"code": "it", "name": "Italian", "native": "Italiano", "flag": "🇮🇹"},
-    {"code": "pt", "name": "Portuguese", "native": "Português", "flag": "🇵🇹"},
+    # flag: lowercase ISO 3166-1 alpha-2 country code used with the flag-icons CSS library
+    # (e.g. "gb" → <span class="fi fi-gb">). Regional codes like "gb-wls" are also supported.
+    {"code": "en", "name": "English", "native": "English", "flag": "gb"},
+    {"code": "de", "name": "German", "native": "Deutsch", "flag": "de"},
+    {"code": "fr", "name": "French", "native": "Français", "flag": "fr"},
+    {"code": "es", "name": "Spanish", "native": "Español", "flag": "es"},
+    {"code": "it", "name": "Italian", "native": "Italiano", "flag": "it"},
+    {"code": "pt", "name": "Portuguese", "native": "Português", "flag": "pt"},
     # --- Tier 2: Western & Northern European ---
-    {"code": "nl", "name": "Dutch", "native": "Nederlands", "flag": "🇳🇱"},
-    {"code": "nb", "name": "Norwegian Bokmål", "native": "Norsk bokmål", "flag": "🇳🇴"},
-    {"code": "no", "name": "Norwegian", "native": "Norsk", "flag": "🇳🇴"},
-    {"code": "da", "name": "Danish", "native": "Dansk", "flag": "🇩🇰"},
-    {"code": "sv", "name": "Swedish", "native": "Svenska", "flag": "🇸🇪"},
-    {"code": "fi", "name": "Finnish", "native": "Suomi", "flag": "🇫🇮"},
-    {"code": "is", "name": "Icelandic", "native": "Íslenska", "flag": "🇮🇸"},
-    {"code": "ga", "name": "Irish", "native": "Gaeilge", "flag": "🇮🇪"},
-    {"code": "lb", "name": "Luxembourgish", "native": "Lëtzebuergesch", "flag": "🇱🇺"},
-    {"code": "ca", "name": "Catalan", "native": "Català", "flag": "🏴"},
-    {"code": "cy", "name": "Welsh", "native": "Cymraeg", "flag": "🏴󠁧󠁢󠁷󠁬󠁳󠁿"},  # Wales subdivision flag (U+1F3F4 + tag chars)
-    {"code": "fy", "name": "Western Frisian", "native": "Frysk", "flag": "🇳🇱"},
-    {"code": "gl", "name": "Galician", "native": "Galego", "flag": "🇪🇸"},
-    {"code": "li", "name": "Limburgish", "native": "Limburgs", "flag": "🇳🇱"},
-    {"code": "vls", "name": "Flemish", "native": "West-Vlams", "flag": "🇧🇪"},
-    {"code": "nds", "name": "Low German", "native": "Plattdüütsch", "flag": "🇩🇪"},
+    {"code": "nl", "name": "Dutch", "native": "Nederlands", "flag": "nl"},
+    {"code": "nb", "name": "Norwegian Bokmål", "native": "Norsk bokmål", "flag": "no"},
+    {"code": "no", "name": "Norwegian", "native": "Norsk", "flag": "no"},
+    {"code": "da", "name": "Danish", "native": "Dansk", "flag": "dk"},
+    {"code": "sv", "name": "Swedish", "native": "Svenska", "flag": "se"},
+    {"code": "fi", "name": "Finnish", "native": "Suomi", "flag": "fi"},
+    {"code": "is", "name": "Icelandic", "native": "Íslenska", "flag": "is"},
+    {"code": "ga", "name": "Irish", "native": "Gaeilge", "flag": "ie"},
+    {"code": "lb", "name": "Luxembourgish", "native": "Lëtzebuergesch", "flag": "lu"},
+    {"code": "ca", "name": "Catalan", "native": "Català", "flag": "es"},  # no dedicated ISO flag; use Spain
+    {"code": "cy", "name": "Welsh", "native": "Cymraeg", "flag": "gb-wls"},  # flag-icons GB region code
+    {"code": "fy", "name": "Western Frisian", "native": "Frysk", "flag": "nl"},
+    {"code": "gl", "name": "Galician", "native": "Galego", "flag": "es"},
+    {"code": "li", "name": "Limburgish", "native": "Limburgs", "flag": "nl"},
+    {"code": "vls", "name": "Flemish", "native": "West-Vlams", "flag": "be"},
+    {"code": "nds", "name": "Low German", "native": "Plattdüütsch", "flag": "de"},
     # --- Tier 3: Central & Eastern European ---
-    {"code": "pl", "name": "Polish", "native": "Polski", "flag": "🇵🇱"},
-    {"code": "cs", "name": "Czech", "native": "Čeština", "flag": "🇨🇿"},
-    {"code": "sk", "name": "Slovak", "native": "Slovenčina", "flag": "🇸🇰"},
-    {"code": "hu", "name": "Hungarian", "native": "Magyar", "flag": "🇭🇺"},
-    {"code": "sl", "name": "Slovenian", "native": "Slovenščina", "flag": "🇸🇮"},
-    {"code": "hr", "name": "Croatian", "native": "Hrvatski", "flag": "🇭🇷"},
-    {"code": "ro", "name": "Romanian", "native": "Română", "flag": "🇷🇴"},
-    {"code": "bg", "name": "Bulgarian", "native": "Български", "flag": "🇧🇬"},
-    {"code": "el", "name": "Greek", "native": "Ελληνικά", "flag": "🇬🇷"},
-    {"code": "et", "name": "Estonian", "native": "Eesti", "flag": "🇪🇪"},
-    {"code": "lv", "name": "Latvian", "native": "Latviešu", "flag": "🇱🇻"},
-    {"code": "lt", "name": "Lithuanian", "native": "Lietuvių", "flag": "🇱🇹"},
-    {"code": "sr", "name": "Serbian", "native": "Српски", "flag": "🇷🇸"},
+    {"code": "pl", "name": "Polish", "native": "Polski", "flag": "pl"},
+    {"code": "cs", "name": "Czech", "native": "Čeština", "flag": "cz"},
+    {"code": "sk", "name": "Slovak", "native": "Slovenčina", "flag": "sk"},
+    {"code": "hu", "name": "Hungarian", "native": "Magyar", "flag": "hu"},
+    {"code": "sl", "name": "Slovenian", "native": "Slovenščina", "flag": "si"},
+    {"code": "hr", "name": "Croatian", "native": "Hrvatski", "flag": "hr"},
+    {"code": "ro", "name": "Romanian", "native": "Română", "flag": "ro"},
+    {"code": "bg", "name": "Bulgarian", "native": "Български", "flag": "bg"},
+    {"code": "el", "name": "Greek", "native": "Ελληνικά", "flag": "gr"},
+    {"code": "et", "name": "Estonian", "native": "Eesti", "flag": "ee"},
+    {"code": "lv", "name": "Latvian", "native": "Latviešu", "flag": "lv"},
+    {"code": "lt", "name": "Lithuanian", "native": "Lietuvių", "flag": "lt"},
+    {"code": "sr", "name": "Serbian", "native": "Српски", "flag": "rs"},
     # --- Tier 4: Non-EU European, Middle Eastern & African ---
-    {"code": "tr", "name": "Turkish", "native": "Türkçe", "flag": "🇹🇷"},
-    {"code": "uk", "name": "Ukrainian", "native": "Українська", "flag": "🇺🇦"},
-    {"code": "he", "name": "Hebrew", "native": "עברית", "flag": "🇮🇱"},
-    {"code": "ar", "name": "Arabic", "native": "العربية", "flag": "🇸🇦"},
-    {"code": "fa", "name": "Persian", "native": "فارسی", "flag": "🇮🇷"},
-    {"code": "af", "name": "Afrikaans", "native": "Afrikaans", "flag": "🇿🇦"},
+    {"code": "tr", "name": "Turkish", "native": "Türkçe", "flag": "tr"},
+    {"code": "uk", "name": "Ukrainian", "native": "Українська", "flag": "ua"},
+    {"code": "he", "name": "Hebrew", "native": "עברית", "flag": "il"},
+    {"code": "ar", "name": "Arabic", "native": "العربية", "flag": "sa"},
+    {"code": "fa", "name": "Persian", "native": "فارسی", "flag": "ir"},
+    {"code": "af", "name": "Afrikaans", "native": "Afrikaans", "flag": "za"},
     # --- Tier 5: Asian languages ---
-    {"code": "zh", "name": "Chinese", "native": "中文", "flag": "🇨🇳"},
-    {"code": "zh-TW", "name": "Traditional Chinese", "native": "繁體中文", "flag": "🇹🇼"},
-    {"code": "ja", "name": "Japanese", "native": "日本語", "flag": "🇯🇵"},
-    {"code": "ko", "name": "Korean", "native": "한국어", "flag": "🇰🇷"},
-    {"code": "vi", "name": "Vietnamese", "native": "Tiếng Việt", "flag": "🇻🇳"},
-    {"code": "pa", "name": "Punjabi", "native": "ਪੰਜਾਬੀ", "flag": "🇮🇳"},
-    {"code": "kn", "name": "Kannada", "native": "ಕನ್ನಡ", "flag": "🇮🇳"},
-    {"code": "hi", "name": "Hindi", "native": "हिन्दी", "flag": "🇮🇳"},
-    {"code": "bn", "name": "Bengali", "native": "বাংলা", "flag": "🇧🇩"},
-    {"code": "gu", "name": "Gujarati", "native": "ગુજરાતી", "flag": "🇮🇳"},
-    {"code": "ml", "name": "Malayalam", "native": "മലയാളം", "flag": "🇮🇳"},
-    {"code": "mr", "name": "Marathi", "native": "मराठी", "flag": "🇮🇳"},
-    {"code": "ta", "name": "Tamil", "native": "தமிழ்", "flag": "🇮🇳"},
-    {"code": "te", "name": "Telugu", "native": "తెలుగు", "flag": "🇮🇳"},
-    {"code": "ur", "name": "Urdu", "native": "اردو", "flag": "🇵🇰"},
-    {"code": "si", "name": "Sinhala", "native": "සිංහල", "flag": "🇱🇰"},
-    {"code": "ne", "name": "Nepali", "native": "नेपाली", "flag": "🇳🇵"},
-    {"code": "th", "name": "Thai", "native": "ไทย", "flag": "🇹🇭"},
-    {"code": "km", "name": "Khmer", "native": "ខ្មែរ", "flag": "🇰🇭"},
-    {"code": "id", "name": "Indonesian", "native": "Bahasa Indonesia", "flag": "🇮🇩"},
-    {"code": "ms", "name": "Malay", "native": "Bahasa Melayu", "flag": "🇲🇾"},
-    {"code": "jv", "name": "Javanese", "native": "Basa Jawa", "flag": "🇮🇩"},
-    {"code": "tl", "name": "Tagalog", "native": "Filipino", "flag": "🇵🇭"},
-    {"code": "mn", "name": "Mongolian", "native": "Монгол", "flag": "🇲🇳"},
-    {"code": "kk", "name": "Kazakh", "native": "Қазақ тілі", "flag": "🇰🇿"},
-    {"code": "uz", "name": "Uzbek", "native": "Oʻzbekcha", "flag": "🇺🇿"},
-    {"code": "az", "name": "Azerbaijani", "native": "Azərbaycan dili", "flag": "🇦🇿"},
-    {"code": "hy", "name": "Armenian", "native": "Հայերեն", "flag": "🇦🇲"},
-    {"code": "ka", "name": "Georgian", "native": "ქართული", "flag": "🇬🇪"},
+    {"code": "zh", "name": "Chinese", "native": "中文", "flag": "cn"},
+    {"code": "zh-TW", "name": "Traditional Chinese", "native": "繁體中文", "flag": "tw"},
+    {"code": "ja", "name": "Japanese", "native": "日本語", "flag": "jp"},
+    {"code": "ko", "name": "Korean", "native": "한국어", "flag": "kr"},
+    {"code": "vi", "name": "Vietnamese", "native": "Tiếng Việt", "flag": "vn"},
+    {"code": "pa", "name": "Punjabi", "native": "ਪੰਜਾਬੀ", "flag": "in"},
+    {"code": "kn", "name": "Kannada", "native": "ಕನ್ನಡ", "flag": "in"},
+    {"code": "hi", "name": "Hindi", "native": "हिन्दी", "flag": "in"},
+    {"code": "bn", "name": "Bengali", "native": "বাংলা", "flag": "bd"},
+    {"code": "gu", "name": "Gujarati", "native": "ગુજરાતી", "flag": "in"},
+    {"code": "ml", "name": "Malayalam", "native": "മലയാളം", "flag": "in"},
+    {"code": "mr", "name": "Marathi", "native": "मराठी", "flag": "in"},
+    {"code": "ta", "name": "Tamil", "native": "தமிழ்", "flag": "in"},
+    {"code": "te", "name": "Telugu", "native": "తెలుగు", "flag": "in"},
+    {"code": "ur", "name": "Urdu", "native": "اردو", "flag": "pk"},
+    {"code": "si", "name": "Sinhala", "native": "සිංහල", "flag": "lk"},
+    {"code": "ne", "name": "Nepali", "native": "नेपाली", "flag": "np"},
+    {"code": "th", "name": "Thai", "native": "ไทย", "flag": "th"},
+    {"code": "km", "name": "Khmer", "native": "ខ្មែរ", "flag": "kh"},
+    {"code": "id", "name": "Indonesian", "native": "Bahasa Indonesia", "flag": "id"},
+    {"code": "ms", "name": "Malay", "native": "Bahasa Melayu", "flag": "my"},
+    {"code": "jv", "name": "Javanese", "native": "Basa Jawa", "flag": "id"},
+    {"code": "tl", "name": "Tagalog", "native": "Filipino", "flag": "ph"},
+    {"code": "mn", "name": "Mongolian", "native": "Монгол", "flag": "mn"},
+    {"code": "kk", "name": "Kazakh", "native": "Қазақ тілі", "flag": "kz"},
+    {"code": "uz", "name": "Uzbek", "native": "Oʻzbekcha", "flag": "uz"},
+    {"code": "az", "name": "Azerbaijani", "native": "Azərbaycan dili", "flag": "az"},
+    {"code": "hy", "name": "Armenian", "native": "Հայերեն", "flag": "am"},
+    {"code": "ka", "name": "Georgian", "native": "ქართული", "flag": "ge"},
     # --- Tier 6: African languages ---
-    {"code": "sw", "name": "Swahili", "native": "Kiswahili", "flag": "🇰🇪"},
-    {"code": "am", "name": "Amharic", "native": "አማርኛ", "flag": "🇪🇹"},
-    {"code": "ha", "name": "Hausa", "native": "Hausa", "flag": "🇳🇬"},
-    {"code": "yo", "name": "Yoruba", "native": "Yorùbá", "flag": "🇳🇬"},
-    {"code": "ig", "name": "Igbo", "native": "Igbo", "flag": "🇳🇬"},
-    {"code": "zu", "name": "Zulu", "native": "isiZulu", "flag": "🇿🇦"},
+    {"code": "sw", "name": "Swahili", "native": "Kiswahili", "flag": "ke"},
+    {"code": "am", "name": "Amharic", "native": "አማርኛ", "flag": "et"},
+    {"code": "ha", "name": "Hausa", "native": "Hausa", "flag": "ng"},
+    {"code": "yo", "name": "Yoruba", "native": "Yorùbá", "flag": "ng"},
+    {"code": "ig", "name": "Igbo", "native": "Igbo", "flag": "ng"},
+    {"code": "zu", "name": "Zulu", "native": "isiZulu", "flag": "za"},
     # --- Tier 7: Constructed & other languages ---
-    {"code": "eo", "name": "Esperanto", "native": "Esperanto", "flag": "🌍"},
+    {"code": "eo", "name": "Esperanto", "native": "Esperanto", "flag": "un"},  # UN flag for international language
 ]
 
 SUPPORTED_LANGUAGE_CODES: set[str] = {lang["code"] for lang in SUPPORTED_LANGUAGES}

app/utils/network.py

@@ -27,8 +27,8 @@ def is_private_ip(hostname: str) -> bool:
                     return True
             return False
         except (socket.gaierror, socket.error):
-            # Cannot resolve - allow for testing/development
-            # In production, DNS should work properly
-            # Log this for debugging
-            logger.warning(f"Could not resolve hostname: {hostname}")
-            return False  # Changed from True to False to allow external domains in tests
+            # Cannot resolve.
+            # Fail securely: block unresolved domains to prevent DNS rebinding
+            # and SSRF bypasses via unresolvable addresses.
+            logger.warning(f"Could not resolve hostname (blocking securely): {hostname}")
+            return True

app/utils/session_manager.py

@@ -0,0 +1,505 @@
+"""Server-side session management utilities.
+
+Provides helpers for creating, validating, and revoking user sessions.
+Sessions are tracked in the ``user_sessions`` table and referenced by a
+cryptographically random token stored in the browser cookie.  This enables
+the "log off everywhere" feature and per-session revocation.
+"""
+
+from __future__ import annotations
+
+import logging
+import secrets
+from datetime import datetime, timedelta, timezone
+
+from sqlalchemy.orm import Session
+
+from app.config import settings
+from app.models import ApiToken, QRLoginChallenge, UserSession
+
+logger = logging.getLogger(__name__)
+
+
+def _ensure_tz_aware(dt: datetime | None) -> datetime | None:
+    """Return *dt* with UTC tzinfo if it is naive, or unchanged if already aware.
+
+    SQLite does not persist timezone information, so datetimes read back from
+    the database are offset-naive.  This helper normalises them for safe
+    comparison with ``datetime.now(timezone.utc)``.
+    """
+    if dt is not None and dt.tzinfo is None:
+        return dt.replace(tzinfo=timezone.utc)
+    return dt
+
+
+def get_session_lifetime_days() -> int:
+    """Return the effective session lifetime in days.
+
+    If ``session_lifetime_custom_days`` is set it takes precedence over
+    ``session_lifetime_days``.
+    """
+    custom = getattr(settings, "session_lifetime_custom_days", None)
+    if custom is not None and isinstance(custom, int) and custom > 0:
+        return custom
+    return max(1, getattr(settings, "session_lifetime_days", 30))
+
+
+def get_session_max_age_seconds() -> int:
+    """Return the session max-age in seconds for the cookie."""
+    return get_session_lifetime_days() * 86400
+
+
+def create_session(
+    db: Session,
+    user_id: str,
+    ip_address: str | None = None,
+    user_agent: str | None = None,
+) -> UserSession:
+    """Create a new server-side session record.
+
+    Args:
+        db: Database session.
+        user_id: Stable owner identifier.
+        ip_address: Client IP address.
+        user_agent: Client User-Agent header.
+
+    Returns:
+        The newly created ``UserSession`` instance.
+    """
+    session_token = secrets.token_urlsafe(64)
+    now = datetime.now(timezone.utc)
+    lifetime_days = get_session_lifetime_days()
+    expires_at = now + timedelta(days=lifetime_days)
+
+    device_info = _parse_device_info(user_agent)
+
+    user_session = UserSession(
+        session_token=session_token,
+        user_id=user_id,
+        ip_address=ip_address,
+        user_agent=(user_agent or "")[:512],
+        device_info=device_info,
+        created_at=now,
+        last_active_at=now,
+        expires_at=expires_at,
+    )
+    try:
+        db.add(user_session)
+        db.commit()
+        db.refresh(user_session)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create session for user_id=%s", user_id)
+        raise
+
+    logger.info(
+        "[SESSION] Created session id=%s user=%s device=%r expires=%s",
+        user_session.id,
+        user_id,
+        device_info,
+        expires_at.isoformat(),
+    )
+    return user_session
+
+
+def validate_session(db: Session, session_token: str) -> UserSession | None:
+    """Validate a session token and return the session if valid.
+
+    A session is valid when:
+    * It exists in the database.
+    * ``is_revoked`` is ``False``.
+    * ``expires_at`` is in the future.
+
+    Side-effect: updates ``last_active_at`` on valid sessions.
+
+    Returns:
+        The ``UserSession`` if valid, else ``None``.
+    """
+    if not session_token:
+        return None
+
+    now = datetime.now(timezone.utc)
+    user_session = db.query(UserSession).filter(UserSession.session_token == session_token).first()
+
+    if not user_session:
+        logger.debug("[SESSION] Token not found in database")
+        return None
+
+    if user_session.is_revoked:
+        logger.debug("[SESSION] Session id=%s is revoked", user_session.id)
+        return None
+
+    if user_session.expires_at:
+        expires = _ensure_tz_aware(user_session.expires_at)
+        if expires < now:
+            logger.debug("[SESSION] Session id=%s has expired", user_session.id)
+            return None
+
+    # Update last_active_at (throttled to avoid excessive writes)
+    last_active = _ensure_tz_aware(user_session.last_active_at)
+    if not last_active or (now - last_active).total_seconds() > 60:
+        try:
+            user_session.last_active_at = now
+            db.commit()
+        except Exception:
+            db.rollback()
+            logger.debug("[SESSION] Failed to update last_active_at for session id=%s", user_session.id)
+
+    return user_session
+
+
+def revoke_session(db: Session, session_id: int, user_id: str) -> bool:
+    """Revoke a single session by ID.
+
+    Args:
+        db: Database session.
+        session_id: The session record ID to revoke.
+        user_id: The owner — ensures a user can only revoke their own sessions.
+
+    Returns:
+        ``True`` if the session was found and revoked, ``False`` otherwise.
+    """
+    user_session = db.get(UserSession, session_id)
+    if not user_session or user_session.user_id != user_id:
+        return False
+
+    now = datetime.now(timezone.utc)
+    user_session.is_revoked = True
+    user_session.revoked_at = now
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("[SESSION] Revoked session id=%s user=%s", session_id, user_id)
+    return True
+
+
+def revoke_all_sessions(
+    db: Session,
+    user_id: str,
+    *,
+    except_session_id: int | None = None,
+    revoke_api_tokens: bool = True,
+) -> int:
+    """Revoke all active sessions for a user ("log off everywhere").
+
+    Args:
+        db: Database session.
+        user_id: The owner whose sessions should be revoked.
+        except_session_id: If provided, keep this session active (the
+            current browser session).
+        revoke_api_tokens: If ``True``, also revoke all active API tokens.
+
+    Returns:
+        Number of sessions revoked.
+    """
+    now = datetime.now(timezone.utc)
+    query = db.query(UserSession).filter(
+        UserSession.user_id == user_id,
+        UserSession.is_revoked.is_(False),
+    )
+    if except_session_id is not None:
+        query = query.filter(UserSession.id != except_session_id)
+
+    sessions = query.all()
+    count = 0
+    for s in sessions:
+        s.is_revoked = True
+        s.revoked_at = now
+        count += 1
+
+    if revoke_api_tokens:
+        tokens = (
+            db.query(ApiToken)
+            .filter(
+                ApiToken.owner_id == user_id,
+                ApiToken.is_active.is_(True),
+            )
+            .all()
+        )
+        for t in tokens:
+            t.is_active = False
+            t.revoked_at = now
+
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info(
+        "[SESSION] Revoked all sessions for user=%s (count=%d, except_session_id=%s, tokens_revoked=%s)",
+        user_id,
+        count,
+        except_session_id,
+        revoke_api_tokens,
+    )
+    return count
+
+
+def list_user_sessions(db: Session, user_id: str) -> list[UserSession]:
+    """Return all non-revoked, non-expired sessions for a user.
+
+    Results are ordered by most recently active first.
+    """
+    now = datetime.now(timezone.utc)
+    sessions = (
+        db.query(UserSession)
+        .filter(
+            UserSession.user_id == user_id,
+            UserSession.is_revoked.is_(False),
+        )
+        .order_by(UserSession.last_active_at.desc())
+        .all()
+    )
+    # Filter expired sessions in Python to handle timezone-naive datetimes (SQLite)
+    result = []
+    for s in sessions:
+        expires = _ensure_tz_aware(s.expires_at)
+        if expires and expires > now:
+            result.append(s)
+    return result
+
+
+def cleanup_expired_sessions(db: Session) -> int:
+    """Delete sessions that expired more than 7 days ago.
+
+    Intended to be called periodically (e.g. via Celery beat) to keep the
+    table from growing unbounded.
+
+    Returns:
+        Number of rows deleted.
+    """
+    cutoff = datetime.now(timezone.utc) - timedelta(days=7)
+    count = db.query(UserSession).filter(UserSession.expires_at < cutoff).delete(synchronize_session=False)
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+    if count:
+        logger.info("[SESSION] Cleaned up %d expired sessions", count)
+    return count
+
+
+# ---------------------------------------------------------------------------
+# QR login helpers
+# ---------------------------------------------------------------------------
+
+
+def create_qr_challenge(db: Session, user_id: str, ip_address: str | None = None) -> QRLoginChallenge:
+    """Create a new QR login challenge.
+
+    Args:
+        db: Database session.
+        user_id: The authenticated web user creating the challenge.
+        ip_address: IP address of the web client.
+
+    Returns:
+        The newly created ``QRLoginChallenge``.
+    """
+    token = secrets.token_urlsafe(64)
+    ttl = getattr(settings, "qr_login_challenge_ttl_seconds", 120)
+    now = datetime.now(timezone.utc)
+    expires_at = now + timedelta(seconds=ttl)
+
+    challenge = QRLoginChallenge(
+        challenge_token=token,
+        user_id=user_id,
+        created_by_ip=ip_address,
+        created_at=now,
+        expires_at=expires_at,
+    )
+    try:
+        db.add(challenge)
+        db.commit()
+        db.refresh(challenge)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create QR login challenge for user_id=%s", user_id)
+        raise
+
+    logger.info("[QR_AUTH] Challenge created: id=%s user=%s expires=%s", challenge.id, user_id, expires_at.isoformat())
+    return challenge
+
+
+def validate_qr_challenge(db: Session, challenge_token: str) -> QRLoginChallenge | None:
+    """Validate a QR challenge token without claiming it.
+
+    Returns the challenge if it exists, is not expired, not claimed,
+    and not cancelled.  Returns ``None`` otherwise.
+    """
+    if not challenge_token:
+        return None
+
+    now = datetime.now(timezone.utc)
+    challenge = db.query(QRLoginChallenge).filter(QRLoginChallenge.challenge_token == challenge_token).first()
+
+    if not challenge:
+        return None
+    if challenge.is_claimed or challenge.is_cancelled:
+        return None
+
+    expires = _ensure_tz_aware(challenge.expires_at)
+    if expires and expires < now:
+        return None
+
+    return challenge
+
+
+def claim_qr_challenge(
+    db: Session,
+    challenge_token: str,
+    device_name: str = "Mobile App",
+    ip_address: str | None = None,
+) -> dict | None:
+    """Claim a QR challenge and issue an API token.
+
+    This is the critical security path.  The challenge is validated,
+    marked as claimed atomically, and an API token is issued for the
+    user who created the challenge.
+
+    Args:
+        db: Database session.
+        challenge_token: The token from the QR code.
+        device_name: Name provided by the mobile app.
+        ip_address: IP address of the claiming mobile device.
+
+    Returns:
+        Dict with ``token`` (plaintext), ``token_id``, ``name``, ``owner_id``
+        and ``created_at`` on success, or ``None`` if the challenge is invalid.
+    """
+    from app.api.api_tokens import generate_api_token, hash_token
+
+    challenge = validate_qr_challenge(db, challenge_token)
+    if not challenge:
+        logger.warning("[QR_AUTH] Invalid or expired challenge token attempted")
+        return None
+
+    now = datetime.now(timezone.utc)
+
+    # Mark as claimed first to prevent race conditions
+    challenge.is_claimed = True
+    challenge.claimed_at = now
+    challenge.claimed_by_ip = ip_address
+    challenge.device_name = device_name
+
+    # Generate API token for the mobile app
+    token_name = f"Mobile App (QR) – {device_name}"
+    plaintext = generate_api_token()
+    token_hash_value = hash_token(plaintext)
+    prefix = plaintext[:12]
+
+    db_token = ApiToken(
+        owner_id=challenge.user_id,
+        name=token_name,
+        token_hash=token_hash_value,
+        token_prefix=prefix,
+    )
+
+    try:
+        db.add(db_token)
+        db.flush()
+        challenge.issued_token_id = db_token.id
+        db.commit()
+        db.refresh(db_token)
+    except Exception:
+        db.rollback()
+        logger.exception("[QR_AUTH] Failed to issue token for challenge id=%s", challenge.id)
+        raise
+
+    logger.info(
+        "[QR_AUTH] Challenge claimed: id=%s user=%s device=%r token_id=%s",
+        challenge.id,
+        challenge.user_id,
+        device_name,
+        db_token.id,
+    )
+
+    return {
+        "token": plaintext,
+        "token_id": db_token.id,
+        "name": token_name,
+        "owner_id": challenge.user_id,
+        "created_at": db_token.created_at,
+    }
+
+
+def get_challenge_status(db: Session, challenge_id: int, user_id: str) -> dict | None:
+    """Get the current status of a QR challenge (for polling from the web UI).
+
+    Returns:
+        Dict with ``status`` ("pending", "claimed", "expired", "cancelled")
+        and metadata, or ``None`` if the challenge doesn't belong to the user.
+    """
+    challenge = db.get(QRLoginChallenge, challenge_id)
+    if not challenge or challenge.user_id != user_id:
+        return None
+
+    now = datetime.now(timezone.utc)
+    expires = _ensure_tz_aware(challenge.expires_at)
+
+    if challenge.is_claimed:
+        status = "claimed"
+    elif challenge.is_cancelled:
+        status = "cancelled"
+    elif expires and expires < now:
+        status = "expired"
+    else:
+        status = "pending"
+
+    return {
+        "id": challenge.id,
+        "status": status,
+        "device_name": challenge.device_name,
+        "claimed_at": challenge.claimed_at,
+        "expires_at": challenge.expires_at,
+    }
+
+
+def _parse_device_info(user_agent: str | None) -> str | None:
+    """Extract a human-readable device description from User-Agent.
+
+    This is a lightweight parser — not a full UA library — that covers
+    the most common browsers and platforms.
+    """
+    if not user_agent:
+        return None
+
+    ua = user_agent.lower()
+
+    # Platform detection
+    platform = "Unknown"
+    if "iphone" in ua:
+        platform = "iPhone"
+    elif "ipad" in ua:
+        platform = "iPad"
+    elif "android" in ua:
+        platform = "Android"
+    elif "macintosh" in ua or "mac os" in ua:
+        platform = "macOS"
+    elif "windows" in ua:
+        platform = "Windows"
+    elif "linux" in ua:
+        platform = "Linux"
+    elif "cros" in ua:
+        platform = "ChromeOS"
+
+    # Browser detection
+    browser = "Unknown Browser"
+    if "edg/" in ua or "edge/" in ua:
+        browser = "Edge"
+    elif "opr/" in ua or "opera" in ua:
+        browser = "Opera"
+    elif "chrome/" in ua and "safari/" in ua:
+        browser = "Chrome"
+    elif "safari/" in ua and "chrome/" not in ua:
+        browser = "Safari"
+    elif "firefox/" in ua:
+        browser = "Firefox"
+    elif "docuelevate" in ua:
+        browser = "DocuElevate App"
+
+    return f"{browser} on {platform}"

app/utils/settings_service.py

@@ -39,6 +39,50 @@ SETTING_METADATA = {
         "required": True,
         "restart_required": True,
     },
+    "db_pool_size": {
+        "category": "Core",
+        "description": (
+            "Number of persistent database connections kept in the pool per worker process. "
+            "Ignored for SQLite (which uses NullPool). Default: 10."
+        ),
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "db_max_overflow": {
+        "category": "Core",
+        "description": (
+            "Additional database connections allowed beyond db_pool_size under burst load. "
+            "Ignored for SQLite. Default: 20."
+        ),
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "db_pool_timeout": {
+        "category": "Core",
+        "description": (
+            "Seconds to wait for a database connection from the pool before raising a TimeoutError. "
+            "Ignored for SQLite. Default: 30."
+        ),
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "db_pool_recycle": {
+        "category": "Core",
+        "description": (
+            "Recycle (close and reopen) database connections after this many seconds "
+            "to avoid stale connections. Ignored for SQLite. Default: 1800."
+        ),
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     "workdir": {
         "category": "Core",
         "description": "Working directory for file storage and processing",
@@ -55,6 +99,18 @@ SETTING_METADATA = {
         "required": True,  # Required for OAuth redirects and external URLs
         "restart_required": True,
     },
+    "public_base_url": {
+        "category": "Core",
+        "description": (
+            "Full public base URL including scheme (e.g., https://docuelevate.example.com). "
+            "When set, overrides auto-detected URLs for OAuth redirect URIs. "
+            "Required when behind a reverse proxy that does not forward X-Forwarded-Proto."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     "debug": {
         "category": "Core",
         "description": "Enable debug mode for verbose logging",
@@ -134,6 +190,38 @@ SETTING_METADATA = {
         "required": True,  # Required when auth_enabled=True (validated in config.py)
         "restart_required": True,
     },
+    "session_lifetime_days": {
+        "category": "Authentication",
+        "description": "Session lifetime in days (default 30). Determines how long a user stays logged in.",
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "session_lifetime_custom_days": {
+        "category": "Authentication",
+        "description": "Override session_lifetime_days with a custom value. Takes precedence when set.",
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "qr_login_enabled": {
+        "category": "Authentication",
+        "description": "Enable QR code-based login for mobile device authentication.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "qr_login_challenge_ttl_seconds": {
+        "category": "Authentication",
+        "description": "Time-to-live in seconds for QR login challenges (default 120).",
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "admin_username": {
         "category": "Authentication",
         "description": "Admin username for local authentication",
@@ -182,6 +270,17 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "sso_auto_login": {
+        "category": "Authentication",
+        "description": (
+            "Automatically redirect to SSO login when authentication is required. "
+            "Skips the login page and sends users directly to the configured SSO provider."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Social Login Providers
     "social_auth_google_enabled": {
         "category": "Social Login",
@@ -212,6 +311,20 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "social_auth_google_use_global_credentials": {
+        "category": "Social Login",
+        "description": (
+            "When True, Google social login uses the global GOOGLE_DRIVE_CLIENT_ID / "
+            "GOOGLE_DRIVE_CLIENT_SECRET credentials (the Google Drive OAuth integration) "
+            "instead of requiring separate SOCIAL_AUTH_GOOGLE_CLIENT_ID / "
+            "SOCIAL_AUTH_GOOGLE_CLIENT_SECRET values. "
+            "Requires SOCIAL_AUTH_GOOGLE_ENABLED=True and global Google Drive OAuth credentials to be set."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     "social_auth_microsoft_enabled": {
         "category": "Social Login",
         "description": (
@@ -254,6 +367,20 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "social_auth_microsoft_use_global_credentials": {
+        "category": "Social Login",
+        "description": (
+            "When True, Microsoft social login uses the global ONEDRIVE_CLIENT_ID / "
+            "ONEDRIVE_CLIENT_SECRET credentials (the OneDrive integration credentials) "
+            "instead of requiring separate SOCIAL_AUTH_MICROSOFT_CLIENT_ID / "
+            "SOCIAL_AUTH_MICROSOFT_CLIENT_SECRET values. "
+            "Requires SOCIAL_AUTH_MICROSOFT_ENABLED=True and global OneDrive credentials to be set."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     "social_auth_apple_enabled": {
         "category": "Social Login",
         "description": (
@@ -302,6 +429,19 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "social_auth_dropbox_use_global_credentials": {
+        "category": "Social Login",
+        "description": (
+            "When True, Dropbox social login uses the global DROPBOX_APP_KEY / DROPBOX_APP_SECRET "
+            "credentials instead of requiring separate SOCIAL_AUTH_DROPBOX_CLIENT_ID / "
+            "SOCIAL_AUTH_DROPBOX_CLIENT_SECRET values. "
+            "Requires SOCIAL_AUTH_DROPBOX_ENABLED=True and global Dropbox credentials to be set."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     "social_auth_dropbox_enabled": {
         "category": "Social Login",
         "description": (
@@ -329,6 +469,182 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "social_auth_github_enabled": {
+        "category": "Social Login",
+        "description": (
+            "Enable GitHub Sign-In. Requires SOCIAL_AUTH_GITHUB_CLIENT_ID and "
+            "SOCIAL_AUTH_GITHUB_CLIENT_SECRET from GitHub Developer Settings."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+        "help_link": "https://github.com/settings/developers",
+        "help_link_label": "GitHub Developer Settings",
+    },
+    "social_auth_github_client_id": {
+        "category": "Social Login",
+        "description": "GitHub OAuth2 client ID from GitHub Developer Settings.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_github_client_secret": {
+        "category": "Social Login",
+        "description": "GitHub OAuth2 client secret from GitHub Developer Settings.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    # Keycloak SSO
+    "social_auth_keycloak_enabled": {
+        "category": "Social Login",
+        "description": "Enable Keycloak SSO. Requires server URL, realm, client ID, and client secret.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_keycloak_client_id": {
+        "category": "Social Login",
+        "description": "Keycloak OAuth2 client ID.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_keycloak_client_secret": {
+        "category": "Social Login",
+        "description": "Keycloak OAuth2 client secret.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_keycloak_server_url": {
+        "category": "Social Login",
+        "description": "Keycloak server base URL (e.g. https://keycloak.example.com).",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_keycloak_realm": {
+        "category": "Social Login",
+        "description": "Keycloak realm name.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    # Generic OAuth2 SSO
+    "social_auth_generic_oauth2_enabled": {
+        "category": "Social Login",
+        "description": "Enable a generic OAuth2 SSO provider.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_client_id": {
+        "category": "Social Login",
+        "description": "Generic OAuth2 client ID.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_client_secret": {
+        "category": "Social Login",
+        "description": "Generic OAuth2 client secret.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_authorize_url": {
+        "category": "Social Login",
+        "description": "Generic OAuth2 authorization URL.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_token_url": {
+        "category": "Social Login",
+        "description": "Generic OAuth2 token endpoint URL.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_userinfo_url": {
+        "category": "Social Login",
+        "description": "Generic OAuth2 userinfo endpoint URL.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_scope": {
+        "category": "Social Login",
+        "description": "Space-separated list of OAuth2 scopes to request (default: openid profile email).",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_generic_oauth2_name": {
+        "category": "Social Login",
+        "description": "Display name for the generic OAuth2 provider button on the login page.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    # SAML2 SSO
+    "social_auth_saml2_enabled": {
+        "category": "Social Login",
+        "description": "Enable SAML2 SSO authentication.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_saml2_entity_id": {
+        "category": "Social Login",
+        "description": "SAML2 Identity Provider Entity ID.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_saml2_sso_url": {
+        "category": "Social Login",
+        "description": "SAML2 Identity Provider SSO URL.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_saml2_certificate": {
+        "category": "Social Login",
+        "description": "SAML2 Identity Provider X.509 certificate (PEM format).",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_saml2_name": {
+        "category": "Social Login",
+        "description": "Display name for the SAML2 provider.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     # AI Services
     "openai_api_key": {
         "category": "AI Services",
@@ -522,6 +838,19 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    # Document Translation
+    "default_document_language": {
+        "category": "AI Services",
+        "description": (
+            "ISO 639-1 language code for the default document translation target "
+            "(e.g. 'en', 'de', 'fr'). Documents whose detected language differs "
+            "are automatically translated into this language after processing."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # OCR Engine Configuration
     "ocr_providers": {
         "category": "OCR Engines",
@@ -682,6 +1011,18 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    "dropbox_allow_global_credentials_for_integrations": {
+        "category": "Storage Providers",
+        "description": (
+            "When True, users may authorize their personal Dropbox integrations using the global "
+            "DROPBOX_APP_KEY / DROPBOX_APP_SECRET credentials configured by the admin, without "
+            "needing to create their own Dropbox app."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Storage Providers - Nextcloud
     "nextcloud_enabled": {
         "category": "Storage Providers",
@@ -862,6 +1203,63 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    # Storage Providers - SharePoint
+    "sharepoint_client_id": {
+        "category": "Storage Providers",
+        "description": "SharePoint Azure AD application (client) ID",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "sharepoint_client_secret": {
+        "category": "Storage Providers",
+        "description": "SharePoint Azure AD client secret",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": False,
+    },
+    "sharepoint_tenant_id": {
+        "category": "Storage Providers",
+        "description": "SharePoint Azure AD tenant ID (use 'common' for multi-tenant apps)",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "sharepoint_refresh_token": {
+        "category": "Storage Providers",
+        "description": "SharePoint OAuth refresh token",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": False,
+    },
+    "sharepoint_site_url": {
+        "category": "Storage Providers",
+        "description": "SharePoint site URL (e.g. https://tenant.sharepoint.com/sites/sitename)",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "sharepoint_document_library": {
+        "category": "Storage Providers",
+        "description": "SharePoint document library name (default: 'Documents')",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "sharepoint_folder_path": {
+        "category": "Storage Providers",
+        "description": "Subfolder path inside the SharePoint document library",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Storage Providers - WebDAV
     "webdav_enabled": {
         "category": "Storage Providers",
@@ -1777,6 +2175,30 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    "telegram_enabled": {
+        "category": "Notifications",
+        "description": "Enable Telegram bot notifications.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "telegram_bot_token": {
+        "category": "Notifications",
+        "description": "Telegram Bot API token from @BotFather.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": False,
+    },
+    "telegram_chat_id": {
+        "category": "Notifications",
+        "description": "Telegram chat ID to send notifications to.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Notifications Settings
     "notification_urls": {
         "category": "Notifications",
@@ -1875,6 +2297,18 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    "automation_hooks_enabled": {
+        "category": "Feature Flags",
+        "description": (
+            "Enable Zapier / Make.com automation hook subscriptions and delivery. "
+            "When enabled, external automation platforms can subscribe to DocuElevate events "
+            "via the REST hooks protocol. Default: True."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "compliance_enabled": {
         "category": "Feature Flags",
         "description": (
@@ -1887,6 +2321,28 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    "factory_reset_on_startup": {
+        "category": "Feature Flags",
+        "description": (
+            "Wipe all user data on every startup so the instance always starts fresh. "
+            "Useful for demo/testing environments. Default: False."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "enable_factory_reset": {
+        "category": "Feature Flags",
+        "description": (
+            "Show the System Reset page in the admin UI.  Allows administrators to "
+            "trigger a full data wipe or a wipe-and-reimport from the web interface. Default: False."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Backup / Restore
     "backup_enabled": {
         "category": "Backup",
@@ -1910,14 +2366,26 @@ SETTING_METADATA = {
         "category": "Backup",
         "description": (
             "Storage provider for remote backup copies. "
-            "Accepted values: s3, dropbox, google_drive, onedrive, nextcloud, webdav, ftp, sftp, email. "
+            "Accepted values: s3, dropbox, google_drive, onedrive, sharepoint, nextcloud, webdav, ftp, sftp, email. "
             "Leave empty to keep backups local only."
         ),
         "type": "string",
         "sensitive": False,
         "required": False,
         "restart_required": False,
-        "options": ["", "s3", "dropbox", "google_drive", "onedrive", "nextcloud", "webdav", "ftp", "sftp", "email"],
+        "options": [
+            "",
+            "s3",
+            "dropbox",
+            "google_drive",
+            "onedrive",
+            "sharepoint",
+            "nextcloud",
+            "webdav",
+            "ftp",
+            "sftp",
+            "email",
+        ],
     },
     "backup_remote_folder": {
         "category": "Backup",
@@ -2429,6 +2897,27 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    # Per-user upload rate limiting
+    "upload_rate_limit_per_user": {
+        "category": "Security",
+        "description": (
+            "Maximum number of uploads a single user may submit within upload_rate_limit_window seconds. "
+            "The health-aware limiter may reduce this dynamically under high Redis queue depth or CPU load. "
+            "Default: 20."
+        ),
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "upload_rate_limit_window": {
+        "category": "Security",
+        "description": ("Sliding window in seconds over which upload_rate_limit_per_user is enforced. Default: 60."),
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Rate Limiting
     "rate_limiting_enabled": {
         "category": "Security",
@@ -2628,6 +3117,63 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    # Logging
+    "log_level": {
+        "category": "Observability",
+        "description": (
+            "Python logging level for the application root logger. "
+            "Accepts: DEBUG, INFO, WARNING, ERROR, CRITICAL. "
+            "When DEBUG=True and LOG_LEVEL is not explicitly set, "
+            "the effective level is automatically lowered to DEBUG."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "log_format": {
+        "category": "Observability",
+        "description": (
+            "Log output format: 'text' (human-readable, default) or "
+            "'json' (structured JSON lines for SIEM / log aggregation)."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "log_syslog_enabled": {
+        "category": "Observability",
+        "description": "Forward application logs to a syslog receiver in addition to stdout.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "log_syslog_host": {
+        "category": "Observability",
+        "description": "Hostname or IP of the syslog receiver for application logs.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "log_syslog_port": {
+        "category": "Observability",
+        "description": "Port of the syslog receiver for application logs.",
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "log_syslog_protocol": {
+        "category": "Observability",
+        "description": "Protocol for syslog transport: 'udp' or 'tcp'.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
     # Observability – Sentry
     "sentry_dsn": {
         "category": "Observability",
@@ -2686,6 +3232,43 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "sentry_js_traces_sample_rate": {
+        "category": "Observability",
+        "description": (
+            "Fraction of browser page-loads captured for client-side Sentry performance tracing (0.0–1.0). "
+            "0.0 (default) disables browser tracing; 1.0 captures every navigation. "
+            "Only active when SENTRY_DSN is set."
+        ),
+        "type": "float",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "sentry_js_replay_session_sample_rate": {
+        "category": "Observability",
+        "description": (
+            "Fraction of sessions recorded by Sentry Session Replay (0.0–1.0). "
+            "0.0 (default) disables session recording; 1.0 records every session. "
+            "Only active when SENTRY_DSN is set."
+        ),
+        "type": "float",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "sentry_js_replay_on_error_sample_rate": {
+        "category": "Observability",
+        "description": (
+            "Fraction of error sessions recorded by Sentry Session Replay (0.0–1.0). "
+            "Defaults to 0.1 (10%) so that errors are captured with replay context "
+            "even when session-level recording is disabled. "
+            "Only active when SENTRY_DSN is set."
+        ),
+        "type": "float",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
 }
 
 

app/utils/settings_sync.py

@@ -71,6 +71,16 @@ def notify_settings_updated() -> None:
     except Exception as exc:
         logger.warning(f"Could not reload in-process settings: {exc}")
 
+    # Re-register OAuth / social-login providers so that any provider whose
+    # credentials were just saved (or updated) in the database is active
+    # immediately on the login page — no restart required.
+    try:
+        from app.auth import refresh_social_providers
+
+        refresh_social_providers()
+    except Exception as exc:
+        logger.warning(f"Could not refresh social login providers after settings update: {exc}")
+
     # Re-check OCR language availability in the background whenever settings
     # are updated.  This ensures that if a user changes tesseract_language or
     # easyocr_languages via the UI, the new language data is downloaded without

app/utils/system_reset.py

@@ -0,0 +1,296 @@
+"""
+System reset utilities for DocuElevate.
+
+Provides functions to:
+- Wipe all user data (database rows + work-files on disk) for a fresh start.
+- Wipe with re-import: move original files to a dedicated folder, wipe
+  everything, then let the watch-folder mechanism re-ingest the files.
+
+Security: All public functions in this module require admin-level access.
+They MUST only be invoked from admin-guarded API/view endpoints.
+"""
+
+import logging
+import shutil
+from pathlib import Path
+
+from sqlalchemy.orm import Session
+
+from app.config import settings
+
+logger = logging.getLogger(__name__)
+
+# Subdirectories inside *workdir* that contain user-generated data.
+# Everything else (app code, static assets, config) is left untouched.
+_USER_DATA_SUBDIRS = ("original", "processed", "tmp", "pdfa", "backups")
+
+# JSON cache files written by watch-folder / ingest tasks.
+_CACHE_FILES = (
+    "watch_folder_processed.json",
+    "ftp_ingest_processed.json",
+    "sftp_ingest_processed.json",
+    "dropbox_ingest_processed.json",
+    "gdrive_ingest_processed.json",
+    "onedrive_ingest_processed.json",
+    "nextcloud_ingest_processed.json",
+    "s3_ingest_processed.json",
+    "webdav_ingest_processed.json",
+    "processed_mails.json",
+    "credential_failures.json",
+)
+
+# The folder name used for storing files prior to re-import.
+REIMPORT_FOLDER_NAME = "reimport"
+
+
+def _wipe_workdir_data(workdir: str) -> dict[str, int]:
+    """Delete user data subdirectories and cache files inside *workdir*.
+
+    Leaves the workdir directory itself intact so the application can
+    continue to write into it.  Also leaves any files that do not belong
+    to the known data subdirectories or caches.
+
+    Returns:
+        A dict with counts of deleted directories and files.
+    """
+    workdir_path = Path(workdir)
+    deleted_dirs = 0
+    deleted_files = 0
+
+    # Remove data subdirectories
+    for subdir in _USER_DATA_SUBDIRS:
+        target = workdir_path / subdir
+        if target.is_dir():
+            shutil.rmtree(target)
+            logger.info("Deleted data directory: %s", target)
+            deleted_dirs += 1
+
+    # Remove cache / state JSON files
+    for cache_file in _CACHE_FILES:
+        target = workdir_path / cache_file
+        if target.is_file():
+            target.unlink()
+            logger.info("Deleted cache file: %s", target)
+            deleted_files += 1
+
+    # Also remove user_wf_*.json files (per-user watch folder caches)
+    for f in workdir_path.glob("user_wf_*.json"):
+        f.unlink()
+        logger.info("Deleted user watch-folder cache: %s", f)
+        deleted_files += 1
+
+    # Remove loose files in workdir root that are user uploads (uuid-named
+    # files like "a1b2c3d4-…pdf") but NOT application config files.
+    for entry in workdir_path.iterdir():
+        if entry.is_file() and entry.suffix.lower() in {
+            ".pdf",
+            ".png",
+            ".jpg",
+            ".jpeg",
+            ".tiff",
+            ".tif",
+            ".docx",
+            ".doc",
+            ".xlsx",
+            ".xls",
+            ".pptx",
+            ".heic",
+            ".heif",
+            ".webp",
+            ".bmp",
+            ".gif",
+            ".txt",
+            ".rtf",
+            ".odt",
+            ".ods",
+            ".odp",
+            ".csv",
+            ".pages",
+            ".numbers",
+            ".keynote",
+        }:
+            entry.unlink()
+            logger.info("Deleted loose workdir file: %s", entry)
+            deleted_files += 1
+
+    return {"deleted_dirs": deleted_dirs, "deleted_files": deleted_files}
+
+
+def _wipe_database(db: Session) -> dict[str, int]:
+    """Delete all user-generated rows from the database.
+
+    Preserves schema (tables, migrations) and system-seeded rows that will
+    be re-created on the next startup (subscription plans, default pipeline,
+    scheduled jobs, compliance templates).
+
+    Returns:
+        A dict mapping table name → number of rows deleted.
+    """
+    from app.models import (
+        AuditLog,
+        BackupRecord,
+        DocumentMetadata,
+        FileProcessingStep,
+        FileRecord,
+        InAppNotification,
+        ProcessingLog,
+        SavedSearch,
+        SettingsAuditLog,
+        SharedLink,
+        UserImapAccount,
+        UserIntegration,
+        UserNotificationPreference,
+        UserNotificationTarget,
+    )
+
+    # Order matters: delete children before parents to respect FK constraints.
+    tables_to_wipe: list[tuple[str, type]] = [
+        ("file_processing_steps", FileProcessingStep),
+        ("processing_logs", ProcessingLog),
+        ("shared_links", SharedLink),
+        ("in_app_notifications", InAppNotification),
+        ("user_notification_preferences", UserNotificationPreference),
+        ("user_notification_targets", UserNotificationTarget),
+        ("user_imap_accounts", UserImapAccount),
+        ("user_integrations", UserIntegration),
+        ("saved_searches", SavedSearch),
+        ("settings_audit_log", SettingsAuditLog),
+        ("audit_logs", AuditLog),
+        ("backup_records", BackupRecord),
+        ("document_metadata", DocumentMetadata),
+        ("files", FileRecord),
+    ]
+
+    result: dict[str, int] = {}
+    for table_name, model in tables_to_wipe:
+        try:
+            count = db.query(model).delete()
+            result[table_name] = count
+            logger.info("Wiped %d rows from %s", count, table_name)
+        except Exception:
+            logger.exception("Failed to wipe table %s during system reset", table_name)
+            db.rollback()
+            raise
+
+    db.commit()
+    return result
+
+
+def perform_full_reset(db: Session) -> dict:
+    """Perform a complete system reset: wipe database rows + work-files.
+
+    Args:
+        db: An active SQLAlchemy session.
+
+    Returns:
+        Summary dict with ``database`` and ``filesystem`` sub-dicts.
+    """
+    logger.warning(">>> SYSTEM RESET: wiping all user data <<<")
+
+    db_result = _wipe_database(db)
+    fs_result = _wipe_workdir_data(settings.workdir)
+
+    logger.warning(">>> SYSTEM RESET complete <<<")
+    return {"database": db_result, "filesystem": fs_result}
+
+
+def perform_reset_and_reimport(db: Session) -> dict:
+    """Move original files to a reimport folder, wipe everything, then
+    configure the reimport folder as a watch folder for re-ingestion.
+
+    The watch-folder scanner (``scan_all_watch_folders``) will pick up
+    the files on its next periodic run and process them exactly as if
+    they had been freshly uploaded — respecting the same backoff
+    strategy, size limits, and rate limits.
+
+    Args:
+        db: An active SQLAlchemy session.
+
+    Returns:
+        Summary dict with ``database``, ``filesystem``, and ``reimport`` sub-dicts.
+    """
+    workdir_path = Path(settings.workdir)
+    reimport_dir = workdir_path / REIMPORT_FOLDER_NAME
+    original_dir = workdir_path / "original"
+
+    # 1. Collect original files
+    files_moved = 0
+    reimport_dir.mkdir(parents=True, exist_ok=True)
+
+    if original_dir.is_dir():
+        for entry in original_dir.iterdir():
+            if entry.is_file():
+                # Validate the resolved path stays within original_dir (path traversal guard)
+                try:
+                    entry.resolve().relative_to(original_dir.resolve())
+                except ValueError:
+                    logger.warning("Skipping file outside original dir: %s", entry)
+                    continue
+                dest = reimport_dir / entry.name
+                # Avoid overwriting: append counter if name clash
+                if dest.exists():
+                    stem = dest.stem
+                    suffix = dest.suffix
+                    counter = 1
+                    while dest.exists():
+                        dest = reimport_dir / f"{stem}_{counter}{suffix}"
+                        counter += 1
+                shutil.copy2(str(entry), str(dest))
+                files_moved += 1
+
+    logger.info("Copied %d original files to reimport folder: %s", files_moved, reimport_dir)
+
+    # 2. Perform the full reset (wipe DB + other workdir data)
+    reset_result = perform_full_reset(db)
+
+    # 3. Ensure the reimport folder survived the wipe (it's not in _USER_DATA_SUBDIRS)
+    #    and set up watch folder config to point at it.
+    _configure_reimport_watch_folder(str(reimport_dir))
+
+    reset_result["reimport"] = {
+        "files_moved": files_moved,
+        "reimport_folder": str(reimport_dir),
+    }
+    logger.warning(">>> SYSTEM RESET with re-import configured — %d files staged <<<", files_moved)
+    return reset_result
+
+
+def _configure_reimport_watch_folder(reimport_path: str) -> None:
+    """Append *reimport_path* to the application's watch-folder list.
+
+    The watch-folder scanner uses ``settings.watch_folders`` (a
+    comma-separated string).  We mutate the runtime setting so the
+    next scan picks up the folder.  We also set
+    ``watch_folder_delete_after_process = True`` so files are cleaned
+    up after successful processing.
+    """
+    current = getattr(settings, "watch_folders", None) or ""
+    folders = [f.strip() for f in current.split(",") if f.strip()]
+
+    if reimport_path not in folders:
+        folders.append(reimport_path)
+
+    # Mutate runtime settings (not persisted to .env — ephemeral)
+    object.__setattr__(settings, "watch_folders", ",".join(folders))
+    object.__setattr__(settings, "watch_folder_delete_after_process", True)
+    logger.info("Configured reimport watch folder: %s", reimport_path)
+
+
+def perform_startup_reset() -> None:
+    """Called during application startup when ``FACTORY_RESET_ON_STARTUP=True``.
+
+    Wipes database and filesystem data so the instance starts completely
+    fresh.  Uses its own DB session so it runs before the normal lifespan
+    seeding logic.
+    """
+    from app.database import SessionLocal
+
+    logger.warning("FACTORY_RESET_ON_STARTUP is enabled — wiping all data")
+    db = SessionLocal()
+    try:
+        perform_full_reset(db)
+    except Exception:
+        logger.exception("Factory reset on startup failed")
+        db.rollback()
+    finally:
+        db.close()

app/utils/user_scope.py

@@ -11,22 +11,47 @@ import logging
 
 from fastapi import Request
 from sqlalchemy import or_
-from sqlalchemy.orm import Query
+from sqlalchemy.orm import Query, Session
 from sqlalchemy.sql import false
 
 from app.config import settings
-from app.models import FileRecord
+from app.models import FILE_SHARE_ROLE_EDITOR, FILE_SHARE_ROLE_VIEWER, FileRecord, FileShare
 
 logger = logging.getLogger(__name__)
 
+# Role hierarchy: higher index = more rights
+_ROLE_RANK: dict[str, int] = {
+    FILE_SHARE_ROLE_VIEWER: 1,
+    FILE_SHARE_ROLE_EDITOR: 2,
+    "owner": 3,
+}
+
+
+def _owner_id_from_user(user: dict) -> str | None:
+    """Extract the owner identifier from a user dict.
+
+    Priority: ``sub`` (OAuth subject) → ``preferred_username`` → ``email`` → ``id``.
+    """
+    return user.get("sub") or user.get("preferred_username") or user.get("email") or user.get("id")
+
 
 def get_current_owner_id(request: Request) -> str | None:
     """Extract the owner identifier for the current authenticated user.
 
-    The owner ID is derived from the user's session data.  It uses the
-    ``sub`` claim (OAuth subject) when available, falling back to
-    ``preferred_username`` or ``email``.  Returns ``None`` when no user
-    is authenticated.
+    The owner ID is derived from the user's session data or, when no session
+    is present, from a valid Bearer API token in the ``Authorization`` header.
+    This ensures that both browser-based (session cookie) and mobile/API
+    (Bearer token) requests are correctly identified.
+
+    Priority for user resolution:
+
+    1. Session ``user`` dict (set by OAuth or local login).
+    2. ``request.state.api_token_user`` (set by ``require_login`` or an
+       earlier call to this function during the same request).
+    3. Direct Bearer token look-up against the database.
+
+    Within the resolved user dict the owner ID is chosen as:
+    ``sub`` → ``preferred_username`` → ``email`` → ``id``.
 
     Args:
         request: The current FastAPI request with session data.
@@ -34,19 +59,50 @@ def get_current_owner_id(request: Request) -> str | None:
     Returns:
         A stable string identifier for the user, or ``None``.
     """
+    # 1. Session-based auth (most common for web UI)
     user = request.session.get("user")
-    if not user or not isinstance(user, dict):
-        return None
-    # Prefer 'sub' (OAuth subject), then 'preferred_username', then 'email', then 'id'
-    return user.get("sub") or user.get("preferred_username") or user.get("email") or user.get("id")
+    if user and isinstance(user, dict):
+        return _owner_id_from_user(user)
+
+    # 2. Already-resolved API token user (cached by require_login or a
+    #    prior dependency call during this request)
+    api_user = getattr(request.state, "api_token_user", None)
+    if isinstance(api_user, dict):
+        return _owner_id_from_user(api_user)
+
+    # 3. Direct Bearer token resolution – necessary when this function is
+    #    invoked as a FastAPI dependency (via Depends) which runs *before*
+    #    the @require_login decorator wrapper has had a chance to resolve
+    #    the token and populate request.state.api_token_user.
+    auth_header = request.headers.get("authorization", "")
+    if isinstance(auth_header, str) and auth_header.startswith("Bearer "):
+        try:
+            from app.auth import _resolve_bearer_user
+            from app.database import SessionLocal
+
+            db = SessionLocal()
+            try:
+                resolved = _resolve_bearer_user(request, db)
+            finally:
+                db.close()
+
+            if resolved:
+                # Cache so subsequent calls (and require_login) skip the DB
+                request.state.api_token_user = resolved
+                return _owner_id_from_user(resolved)
+        except Exception:
+            logger.debug("Bearer token resolution failed in get_current_owner_id", exc_info=True)
+
+    return None
 
 
 def apply_owner_filter(query: Query, request: Request) -> Query:
     """Conditionally filter a ``FileRecord`` query by the current user.
 
     When multi-user mode is enabled, only files whose ``owner_id``
-    matches the authenticated user are returned.  Admin users bypass
-    the filter and see all documents.
+    matches the authenticated user are returned, **plus** any files that
+    have been explicitly shared with the user via ``FileShare``.  Admin
+    users bypass the filter and see all documents.
 
     When ``unowned_docs_visible_to_all`` is ``True`` (default), documents
     with ``owner_id IS NULL`` (unclaimed) are also included for every
@@ -74,11 +130,89 @@ def apply_owner_filter(query: Query, request: Request) -> Query:
         # No authenticated user — return empty result set
         return query.filter(false())
 
-    # Build filter: user's own documents
+    # Build filter: user's own documents + documents shared with them
     conditions = [FileRecord.owner_id == owner_id]
 
+    # Include files explicitly shared with this user
+    from sqlalchemy import select as sa_select
+
+    conditions.append(FileRecord.id.in_(sa_select(FileShare.file_id).where(FileShare.shared_with_user_id == owner_id)))
+
     # Optionally include unclaimed (owner_id IS NULL) documents
     if settings.unowned_docs_visible_to_all:
         conditions.append(FileRecord.owner_id.is_(None))
 
     return query.filter(or_(*conditions))
+
+
+def get_file_role(file_record: FileRecord, user_id: str | None, db: Session) -> str | None:
+    """Return the effective role a user has on a ``FileRecord``.
+
+    Roles (in descending order of privilege):
+
+    ``"owner"``   — the user's ``owner_id`` matches ``file_record.owner_id``,
+                    or multi-user mode is disabled (everyone is effectively an
+                    owner in single-user mode).
+    ``"editor"``  — the user has an explicit ``FileShare`` with role=editor.
+    ``"viewer"``  — the user has an explicit ``FileShare`` with role=viewer,
+                    or the file is unclaimed (``owner_id IS NULL``) and
+                    ``unowned_docs_visible_to_all`` is True.
+    ``None``      — no access.
+
+    Args:
+        file_record: The ``FileRecord`` to check.
+        user_id: The stable identifier of the requesting user.
+        db: An active SQLAlchemy session.
+
+    Returns:
+        One of ``"owner"``, ``"editor"``, ``"viewer"``, or ``None``.
+    """
+    if not settings.multi_user_enabled:
+        # Single-user mode: full access for everyone
+        return "owner"
+
+    if user_id is None:
+        return None
+
+    # Owner always has full access
+    if file_record.owner_id == user_id:
+        return "owner"
+
+    # Unclaimed document — limited access when setting allows it
+    if file_record.owner_id is None and settings.unowned_docs_visible_to_all:
+        return FILE_SHARE_ROLE_VIEWER
+
+    # Check for an explicit share
+    share = (
+        db.query(FileShare)
+        .filter(FileShare.file_id == file_record.id, FileShare.shared_with_user_id == user_id)
+        .first()
+    )
+    if share:
+        return share.role
+
+    return None
+
+
+def has_file_role(
+    file_record: FileRecord,
+    user_id: str | None,
+    db: Session,
+    minimum_role: str = FILE_SHARE_ROLE_VIEWER,
+) -> bool:
+    """Return ``True`` if the user's effective role meets the minimum required.
+
+    Args:
+        file_record: The document to check.
+        user_id: Requesting user's stable identifier.
+        db: Active SQLAlchemy session.
+        minimum_role: The minimum role required (``"viewer"``, ``"editor"``,
+            or ``"owner"``).
+
+    Returns:
+        ``True`` when the user's role rank is >= the minimum rank.
+    """
+    role = get_file_role(file_record, user_id, db)
+    if role is None:
+        return False
+    return _ROLE_RANK.get(role, 0) >= _ROLE_RANK.get(minimum_role, 0)

app/utils/webhook.py

@@ -145,6 +145,8 @@ def dispatch_webhook_event(event: str, data: dict[str, Any]) -> None:
     It delegates to :func:`deliver_webhook_task` (Celery) for each matching
     webhook so delivery happens asynchronously with automatic retries.
 
+    Also dispatches to automation hooks (Zapier / Make.com) if enabled.
+
     Args:
         event: Event name (must be in :data:`VALID_EVENTS`).
         data: Event-specific payload data.
@@ -156,16 +158,23 @@ def dispatch_webhook_event(event: str, data: dict[str, Any]) -> None:
     webhooks = get_active_webhooks_for_event(event)
     if not webhooks:
         logger.debug("No active webhooks for event %s", event)
-        return
+    else:
+        payload = build_payload(event, data)
 
-    payload = build_payload(event, data)
+        # Import here to avoid circular dependency with celery_app
+        from app.tasks.webhook_tasks import deliver_webhook_task
 
-    # Import here to avoid circular dependency with celery_app
-    from app.tasks.webhook_tasks import deliver_webhook_task
+        for wh in webhooks:
+            try:
+                deliver_webhook_task.delay(wh["url"], payload, wh["secret"])
+                logger.debug("Queued webhook delivery to %s for event %s", wh["url"], event)
+            except Exception as exc:
+                logger.error("Failed to queue webhook to %s: %s", wh["url"], exc)
 
-    for wh in webhooks:
-        try:
-            deliver_webhook_task.delay(wh["url"], payload, wh["secret"])
-            logger.debug("Queued webhook delivery to %s for event %s", wh["url"], event)
-        except Exception as exc:
-            logger.error("Failed to queue webhook to %s: %s", wh["url"], exc)
+    # Also fan-out to Zapier / Make.com automation hooks
+    try:
+        from app.utils.automation_hooks import dispatch_automation_hooks
+
+        dispatch_automation_hooks(event, data)
+    except Exception as exc:
+        logger.error("Failed to dispatch automation hooks for event %s: %s", event, exc)

app/views/__init__.py

@@ -10,6 +10,7 @@ from app.views.audit_logs import router as audit_logs_router
 from app.views.backup import router as backup_router
 from app.views.compliance import router as compliance_router
 from app.views.db_wizard import router as db_wizard_router
+from app.views.devices import router as devices_router  # Mobile devices dashboard
 from app.views.dropbox import router as dropbox_router
 from app.views.filemanager import router as filemanager_router
 
@@ -26,6 +27,7 @@ from app.views.onedrive import router as onedrive_router
 from app.views.pipelines import router as pipelines_router  # Processing pipelines
 from app.views.plans import router as plans_router  # Admin Plan Designer
 from app.views.profile import router as profile_router  # User self-service profile
+from app.views.qr_login import router as qr_login_router  # QR code mobile login
 from app.views.queue import router as queue_router
 from app.views.scheduled_jobs import router as scheduled_jobs_router  # Scheduled batch jobs
 from app.views.search import router as search_router
@@ -34,6 +36,7 @@ from app.views.share import router as share_router
 from app.views.shared_links import router as shared_links_router
 from app.views.status import router as status_router
 from app.views.subscriptions import router as subscriptions_router  # Pricing + subscription pages
+from app.views.system_reset import router as system_reset_router  # System reset / factory reset
 from app.views.wizard import router as wizard_router
 
 # Create a main router that includes all the view routers
@@ -60,6 +63,7 @@ router.include_router(plans_router)  # Admin Plan Designer
 router.include_router(onboarding_router)  # User onboarding wizard
 router.include_router(pipelines_router)  # Processing pipelines
 router.include_router(profile_router)  # User self-service profile settings
+router.include_router(qr_login_router)  # QR code mobile login page
 router.include_router(imap_accounts_router)  # Per-user IMAP ingestion accounts
 router.include_router(integrations_router)  # Unified integrations dashboard
 router.include_router(notifications_router)  # User notification dashboard
@@ -67,3 +71,5 @@ router.include_router(scheduled_jobs_router)  # Admin scheduled batch jobs
 router.include_router(audit_logs_router)  # Comprehensive audit log viewer
 router.include_router(help_router)  # Built-in help / How-To docs
 router.include_router(compliance_router)  # Compliance templates dashboard
+router.include_router(devices_router)  # Mobile devices dashboard
+router.include_router(system_reset_router)  # System reset / factory reset

app/views/base.py

@@ -94,6 +94,22 @@ def _inject_global_context(ctx: dict) -> None:
         "allow_signup",
         getattr(settings, "multi_user_enabled", False) and getattr(settings, "allow_local_signup", False),
     )
+    ctx.setdefault("enable_factory_reset", getattr(settings, "enable_factory_reset", False))
+
+    # Sentry Browser SDK config (injected into every page so the JS SDK can initialise)
+    # Normalize empty-string DSN to None so the {% if sentry_dsn %} template guard works correctly.
+    _raw_dsn = getattr(settings, "sentry_dsn", None)
+    ctx.setdefault("sentry_dsn", _raw_dsn if _raw_dsn else None)
+    ctx.setdefault("sentry_environment", getattr(settings, "sentry_environment", "production"))
+    ctx.setdefault("sentry_js_traces_sample_rate", getattr(settings, "sentry_js_traces_sample_rate", 0.0))
+    ctx.setdefault(
+        "sentry_js_replay_session_sample_rate",
+        getattr(settings, "sentry_js_replay_session_sample_rate", 0.0),
+    )
+    ctx.setdefault(
+        "sentry_js_replay_on_error_sample_rate",
+        getattr(settings, "sentry_js_replay_on_error_sample_rate", 0.1),
+    )
 
     req = ctx.get("request")
     if req is not None:
@@ -146,12 +162,36 @@ def _inject_global_context(ctx: dict) -> None:
 
 
 def template_response_with_version(*args, **kwargs):
-    """Wrapper for TemplateResponse to include version and CSRF token in all templates"""
-    # If context dict is provided, add version to it
-    if len(args) >= 2 and isinstance(args[1], dict):
-        _inject_global_context(args[1])
-    elif "context" in kwargs and isinstance(kwargs["context"], dict):
+    """Wrapper for TemplateResponse to include version and CSRF token in all templates.
+
+    Handles both old-style and new-style Starlette TemplateResponse calls:
+      - Old-style (Starlette <1.0): TemplateResponse(name, {"request": req, ...}, ...)
+      - New-style (Starlette 1.0+): TemplateResponse(request, name, context={...}, ...)
+    """
+    if len(args) >= 1 and isinstance(args[0], str):
+        # Old-style call: first positional arg is the template name (string).
+        # Convert to new-style: (request, name, context=..., ...)
+        name = args[0]
+        if len(args) >= 2 and isinstance(args[1], dict):
+            context = args[1]
+            # Old-style may have status_code as 3rd positional arg
+            if len(args) >= 3 and "status_code" not in kwargs:
+                kwargs["status_code"] = args[2]
+        else:
+            context = kwargs.pop("context", {})
+        request_obj = context.pop("request", None)
+        if request_obj is not None:
+            context["request"] = request_obj
+        _inject_global_context(context)
+        if request_obj is not None:
+            return original_template_response(request_obj, name, context=context, **kwargs)
+        return original_template_response(name, context=context, **kwargs)
+
+    # New-style call: (request, name, context=..., ...)
+    if "context" in kwargs and isinstance(kwargs["context"], dict):
         _inject_global_context(kwargs["context"])
+    elif len(args) >= 3 and isinstance(args[2], dict):
+        _inject_global_context(args[2])
     return original_template_response(*args, **kwargs)
 
 

app/views/devices.py

@@ -0,0 +1,25 @@
+"""View route for the Devices management page.
+
+Renders the ``devices.html`` template where users can see their registered
+mobile devices, mobile API tokens (created via the mobile SSO flow or QR
+code login), and revoke access per-device.
+"""
+
+import logging
+
+from fastapi import APIRouter, Request
+
+from app.views.base import require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/devices", include_in_schema=False)
+@require_login
+async def devices_page(request: Request):
+    """Render the Devices management page."""
+    return templates.TemplateResponse(
+        "devices.html",
+        {"request": request, "page_title": "Devices"},
+    )

app/views/dropbox.py

@@ -14,6 +14,19 @@ from app.views.base import APIRouter, Depends, get_db, require_login, settings,
 router = APIRouter()
 
 
+def _get_dropbox_callback_url(request: Request) -> str:
+    """Return the Dropbox OAuth callback URL.
+
+    Uses ``PUBLIC_BASE_URL`` when configured so that the redirect URI displayed
+    to the user (and registered in the Dropbox developer console) matches the
+    one used in the OAuth authorization request.  Falls back to deriving the URL
+    from the incoming request when ``PUBLIC_BASE_URL`` is not set.
+    """
+    if settings.public_base_url:
+        return settings.public_base_url.rstrip("/") + "/dropbox-callback"
+    return f"{request.url.scheme}://{request.url.netloc}/dropbox-callback"
+
+
 @router.get("/dropbox-setup")
 @require_login
 async def dropbox_setup_page(
@@ -30,6 +43,8 @@ async def dropbox_setup_page(
     path from the integration's existing config is pre-populated; global
     admin credentials are never exposed in this mode.
     """
+    callback_url = _get_dropbox_callback_url(request)
+
     if integration_id is not None:
         owner_id = get_current_owner_id(request)
         integration = (
@@ -46,6 +61,12 @@ async def dropbox_setup_page(
                     cfg = {}
             # Support both "folder" (DROPBOX destination) and "folder_path" (WATCH_FOLDER source)
             folder_path = cfg.get("folder", cfg.get("folder_path", ""))
+            # Determine if global credentials are available for users to reuse
+            global_creds_available = bool(
+                settings.dropbox_allow_global_credentials_for_integrations
+                and settings.dropbox_app_key
+                and settings.dropbox_app_secret
+            )
             return templates.TemplateResponse(
                 "dropbox.html",
                 {
@@ -56,9 +77,12 @@ async def dropbox_setup_page(
                     "integration_name": integration.name,
                     "integration_type": integration.integration_type,
                     "folder_path": folder_path,
-                    "app_key_value": "",
+                    # Only expose the public app key (not the secret) when global creds are allowed
+                    "app_key_value": settings.dropbox_app_key if global_creds_available else "",
                     "app_secret_value": "",
                     "refresh_token_value": "",
+                    "global_creds_available": global_creds_available,
+                    "callback_url": callback_url,
                 },
             )
 
@@ -78,6 +102,7 @@ async def dropbox_setup_page(
             "integration_id": integration_id,
             "integration_name": None,
             "integration_type": None,
+            "callback_url": callback_url,
         },
     )
 
@@ -108,5 +133,6 @@ async def dropbox_callback(request: Request, code: str = None, error: str = None
             "app_key_value": "",  # The callback will prioritize sessionStorage values
             "app_secret_value": "",  # The callback will prioritize sessionStorage values
             "folder_path": "",  # The callback will prioritize sessionStorage values
+            "callback_url": _get_dropbox_callback_url(request),
         },
     )

app/views/files.py

@@ -19,6 +19,43 @@ router = APIRouter()
 _FILE_NOT_FOUND = "File not found"
 
 
+def _resolve_owner_context(request: Request, file_record, db: Session) -> dict:
+    """Return owner display info and the current user's effective role.
+
+    Returns a dict with:
+    - ``current_user_role``: one of "owner" / "editor" / "viewer" / None
+    - ``owner_display``: human-readable owner string (display_name or user_id)
+    - ``multi_user_enabled``: whether multi-user mode is active
+    """
+    from app.config import settings
+    from app.models import UserProfile
+    from app.utils.user_scope import get_current_owner_id, get_file_role
+
+    multi_user_enabled = settings.multi_user_enabled
+
+    current_owner_id = get_current_owner_id(request)
+    user_session = request.session.get("user")
+    is_admin = isinstance(user_session, dict) and bool(user_session.get("is_admin"))
+
+    if is_admin:
+        current_user_role: str | None = "owner"
+    else:
+        current_user_role = get_file_role(file_record, current_owner_id, db)
+
+    # Build a human-readable owner label
+    if file_record.owner_id:
+        profile = db.query(UserProfile).filter(UserProfile.user_id == file_record.owner_id).first()
+        owner_display: str | None = profile.display_name if profile and profile.display_name else file_record.owner_id
+    else:
+        owner_display = None  # No owner (unowned)
+
+    return {
+        "current_user_role": current_user_role,
+        "owner_display": owner_display,
+        "multi_user_enabled": multi_user_enabled,
+    }
+
+
 @router.get("/files")
 @require_login
 def files_page(
@@ -206,10 +243,93 @@ def files_page(
 
 @router.get("/files/{file_id}")
 @require_login
+def file_summary_page(request: Request, file_id: int, db: Session = Depends(get_db)):
+    """
+    Return the file summary page — a concise overview with links to detail, processing, and annotations views.
+    """
+    try:
+        import json
+        import os
+
+        from app.models import FileRecord
+
+        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+
+        if not file_record:
+            return templates.TemplateResponse(
+                "file_summary.html",
+                {"request": request, "file": None, "error": f"File with ID {file_id} not found"},
+            )
+
+        from app.config import settings
+
+        workdir = os.path.realpath(settings.workdir)
+
+        def _safe_exists(path: str | None) -> bool:
+            """Return True only when *path* exists and resides within workdir."""
+            if not path:
+                return False
+            resolved = os.path.realpath(path)
+            try:
+                common = os.path.commonpath([resolved, workdir])
+            except ValueError:
+                return False
+            return common == workdir and os.path.exists(resolved)
+
+        original_file_exists = _safe_exists(file_record.original_file_path)
+        processed_file_exists = _safe_exists(file_record.processed_file_path)
+
+        # Load AI metadata — JSON sidecar file first, then DB column
+        gpt_metadata = None
+        if file_record.processed_file_path:
+            metadata_path = os.path.splitext(os.path.realpath(file_record.processed_file_path))[0] + ".json"
+            if _safe_exists(metadata_path):
+                try:
+                    with open(metadata_path, "r", encoding="utf-8") as f:
+                        gpt_metadata = json.load(f)
+                except Exception as e:
+                    logger.warning(f"Failed to load metadata sidecar for file {file_id}: {e}")
+
+        if gpt_metadata is None and file_record.ai_metadata:
+            try:
+                gpt_metadata = json.loads(file_record.ai_metadata)
+            except Exception as e:
+                logger.warning(f"Failed to parse ai_metadata for file {file_id}: {e}")
+
+        # Quick processing status
+        try:
+            from app.utils.step_manager import get_step_summary as _get_step_summary
+
+            step_summary = _get_step_summary(db, file_id)
+        except Exception:
+            step_summary = None
+
+        pipeline_info = _resolve_pipeline(db, file_record)
+        owner_ctx = _resolve_owner_context(request, file_record, db)
+
+        return templates.TemplateResponse(
+            "file_summary.html",
+            {
+                "request": request,
+                "file": file_record,
+                "gpt_metadata": gpt_metadata,
+                "original_file_exists": original_file_exists,
+                "processed_file_exists": processed_file_exists,
+                "step_summary": step_summary,
+                "pipeline_info": pipeline_info,
+                **owner_ctx,
+            },
+        )
+    except Exception as e:
+        logger.error(f"Error retrieving file summary {file_id}: {str(e)}")
+        return templates.TemplateResponse("file_summary.html", {"request": request, "file": None, "error": str(e)})
+
+
+@router.get("/files/{file_id}/detail")
+@require_login
 def file_view_page(request: Request, file_id: int, db: Session = Depends(get_db)):
     """
-    Return the document view page — document-centric view with metadata, preview, and extracted text.
-    Process-oriented details are available via /files/{file_id}/detail.
+    Return the document detail page — document-centric view with metadata, preview, and extracted text.
     """
     try:
         import json
@@ -272,6 +392,7 @@ def file_view_page(request: Request, file_id: int, db: Session = Depends(get_db)
 
         # Resolve the pipeline assigned to this file (explicit or system default)
         pipeline_info = _resolve_pipeline(db, file_record)
+        owner_ctx = _resolve_owner_context(request, file_record, db)
 
         return templates.TemplateResponse(
             "file_view.html",
@@ -283,6 +404,7 @@ def file_view_page(request: Request, file_id: int, db: Session = Depends(get_db)
                 "processed_file_exists": processed_file_exists,
                 "step_summary": step_summary,
                 "pipeline_info": pipeline_info,
+                **owner_ctx,
             },
         )
     except Exception as e:
@@ -290,11 +412,11 @@ def file_view_page(request: Request, file_id: int, db: Session = Depends(get_db)
         return templates.TemplateResponse("file_view.html", {"request": request, "file": None, "error": str(e)})
 
 
-@router.get("/files/{file_id}/detail")
+@router.get("/files/{file_id}/process")
 @require_login
 def file_detail_page(request: Request, file_id: int, db: Session = Depends(get_db)):
     """
-    Return the file detail page showing processing history and file information
+    Return the file processing page showing processing history and pipeline information.
     """
     try:
         import json
@@ -375,6 +497,77 @@ def file_detail_page(request: Request, file_id: int, db: Session = Depends(get_d
         return templates.TemplateResponse("file_detail.html", {"request": request, "file": None, "error": str(e)})
 
 
+@router.get("/files/{file_id}/annotations")
+@require_login
+def file_annotations_page(request: Request, file_id: int, db: Session = Depends(get_db)):
+    """
+    Return the comments & annotations page for a file.
+    """
+    try:
+        import os
+
+        from app.models import FileRecord
+
+        file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+
+        if not file_record:
+            return templates.TemplateResponse(
+                "file_annotations.html",
+                {"request": request, "file": None, "error": f"File with ID {file_id} not found"},
+            )
+
+        from app.config import settings
+
+        workdir = os.path.realpath(settings.workdir)
+
+        def _safe_exists(path: str | None) -> bool:
+            """Return True only when *path* exists and resides within workdir."""
+            if not path:
+                return False
+            resolved = os.path.realpath(path)
+            try:
+                common = os.path.commonpath([resolved, workdir])
+            except ValueError:
+                return False
+            return common == workdir and os.path.exists(resolved)
+
+        original_file_exists = _safe_exists(file_record.original_file_path)
+        processed_file_exists = _safe_exists(file_record.processed_file_path)
+
+        # Determine whether the file is a PDF (for EmbedPDF viewer)
+        mime = file_record.mime_type or ""
+        is_pdf = mime == "application/pdf" or (file_record.original_filename or "").lower().endswith(".pdf")
+
+        # Determine the current user's role on this file (and owner display info)
+        owner_ctx = _resolve_owner_context(request, file_record, db)
+
+        return templates.TemplateResponse(
+            "file_annotations.html",
+            {
+                "request": request,
+                "file": file_record,
+                "original_file_exists": original_file_exists,
+                "processed_file_exists": processed_file_exists,
+                "is_pdf": is_pdf,
+                **owner_ctx,
+            },
+        )
+    except Exception as e:
+        logger.error(f"Error retrieving annotations for file {file_id}: {str(e)}")
+        return templates.TemplateResponse("file_annotations.html", {"request": request, "file": None, "error": str(e)})
+
+
+@router.get("/files/{file_id}/comments")
+@require_login
+def file_comments_redirect(request: Request, file_id: int):
+    """
+    Redirect /files/{file_id}/comments to /files/{file_id}/annotations.
+    """
+    from starlette.responses import RedirectResponse
+
+    return RedirectResponse(url=f"/files/{file_id}/annotations", status_code=302)
+
+
 # ---------------------------------------------------------------------------
 # Pipeline ↔ Celery-log stage mapping
 # ---------------------------------------------------------------------------
@@ -396,9 +589,7 @@ _STEP_TYPE_TO_STAGES: dict[str, list[str]] = {
     "embed_metadata": ["embed_metadata_into_pdf"],
     "compute_embedding": ["compute_embedding"],
     "send_to_destinations": ["finalize_document_storage", "send_to_all_destinations"],
-    # "classify" is defined in PIPELINE_STEP_TYPES but has no Celery log stages yet.
-    # When a classify task is implemented, add its stage key(s) here.
-    "classify": [],
+    "classify": ["classify_document"],
 }
 
 # These internal bookkeeping stages are always shown in the flow regardless of
@@ -832,6 +1023,34 @@ def get_processed_text(request: Request, file_id: int, db: Session = Depends(get
         )
 
 
+@router.get("/files/{file_id}/text/default-language")
+@require_login
+def get_default_language_text(request: Request, file_id: int, db: Session = Depends(get_db)):
+    """Return the persisted default-language translation for the file view."""
+    from fastapi import status
+    from fastapi.responses import JSONResponse
+
+    from app.models import FileRecord
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=_FILE_NOT_FOUND)
+
+    if not file_record.default_language_text:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail="No default-language translation available",
+        )
+
+    return JSONResponse(
+        content={
+            "text": file_record.default_language_text,
+            "language_code": file_record.default_language_code,
+            "detected_language": file_record.detected_language,
+        }
+    )
+
+
 @router.get("/duplicates")
 @require_login
 def duplicates_page(

app/views/google_drive.py

@@ -45,6 +45,9 @@ async def google_drive_setup_page(
                 except (json.JSONDecodeError, TypeError):
                     cfg = {}
             folder_id = cfg.get("folder_id", "")
+            # Provide system-wide OAuth credentials when available so users can
+            # authorize without registering their own Google Cloud app.
+            has_system_credentials = bool(settings.google_drive_client_id and settings.google_drive_client_secret)
             return templates.TemplateResponse(
                 "google_drive.html",
                 {
@@ -58,10 +61,13 @@ async def google_drive_setup_page(
                     "use_oauth": True,
                     "oauth_configured": bool(integration.credentials),
                     "sa_configured": False,
-                    "client_id": False,
-                    "client_id_value": "",
-                    "client_secret": False,
-                    "client_secret_value": "",
+                    "has_system_credentials": has_system_credentials,
+                    "client_id": bool(settings.google_drive_client_id) if has_system_credentials else False,
+                    "client_id_value": (settings.google_drive_client_id or "" if has_system_credentials else ""),
+                    "client_secret": bool(settings.google_drive_client_secret) if has_system_credentials else False,
+                    "client_secret_value": (
+                        settings.google_drive_client_secret or "" if has_system_credentials else ""
+                    ),
                     "refresh_token": False,
                     "refresh_token_value": "",
                     "has_credentials_json": False,
@@ -90,6 +96,7 @@ async def google_drive_setup_page(
             "use_oauth": use_oauth,
             "oauth_configured": oauth_configured,
             "sa_configured": sa_configured,
+            "has_system_credentials": bool(settings.google_drive_client_id and settings.google_drive_client_secret),
             "client_id": bool(settings.google_drive_client_id),
             "client_id_value": settings.google_drive_client_id or "",
             "client_secret": bool(settings.google_drive_client_secret),

app/views/onedrive.py

@@ -44,6 +44,9 @@ async def onedrive_setup_page(
                     cfg = {}
             # Support both "folder_path" (WATCH_FOLDER / ONEDRIVE destination)
             folder_path = cfg.get("folder_path", cfg.get("folder", ""))
+            # Provide system-wide app credentials when available so users can
+            # authorize without registering their own Azure/OneDrive app.
+            has_system_credentials = bool(settings.onedrive_client_id and settings.onedrive_client_secret)
             return templates.TemplateResponse(
                 "onedrive.html",
                 {
@@ -54,11 +57,12 @@ async def onedrive_setup_page(
                     "integration_name": integration.name,
                     "integration_type": integration.integration_type,
                     "folder_path": folder_path,
-                    "client_id": False,
-                    "client_id_value": "",
-                    "client_secret": False,
-                    "client_secret_value": "",
-                    "tenant_id": "common",
+                    "has_system_credentials": has_system_credentials,
+                    "client_id": bool(settings.onedrive_client_id) if has_system_credentials else False,
+                    "client_id_value": settings.onedrive_client_id or "" if has_system_credentials else "",
+                    "client_secret": bool(settings.onedrive_client_secret) if has_system_credentials else False,
+                    "client_secret_value": (settings.onedrive_client_secret or "" if has_system_credentials else ""),
+                    "tenant_id": settings.onedrive_tenant_id or "common",
                     "refresh_token": False,
                     "refresh_token_value": "",
                 },
@@ -75,6 +79,7 @@ async def onedrive_setup_page(
             "request": request,
             "user_mode": False,
             "is_configured": is_configured,
+            "has_system_credentials": bool(settings.onedrive_client_id and settings.onedrive_client_secret),
             "client_id": bool(settings.onedrive_client_id),
             "client_id_value": settings.onedrive_client_id or "",
             "client_secret": bool(settings.onedrive_client_secret),

app/views/qr_login.py

@@ -0,0 +1,26 @@
+"""View route for the QR code mobile login page.
+
+Route:
+  GET /qr-login  — renders the QR login page (requires login)
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import Request
+
+from app.views.base import APIRouter, require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/qr-login", include_in_schema=False)
+@require_login
+async def qr_login_page(request: Request):
+    """Serve the QR code login page for mobile app authentication."""
+    return templates.TemplateResponse(
+        "qr_login.html",
+        {"request": request},
+    )

app/views/settings.py

@@ -195,6 +195,346 @@ async def credentials_page(request: Request, db: Session = Depends(get_db)):
         )
 
 
+@router.get("/admin/connections")
+@require_login
+@require_admin_access
+async def connections_page(request: Request, db: Session = Depends(get_db)):
+    """
+    Connections management page - admin only.
+
+    Allows administrators to configure external authentication providers,
+    SSO settings, and service integrations through a wizard-like interface.
+    """
+    try:
+        db_settings = get_all_settings_from_db(db)
+
+        def _get_effective(key: str):
+            """Return DB value if present, else fall back to settings attr."""
+            if key in db_settings and db_settings[key] is not None:
+                return db_settings[key]
+            return getattr(settings, key, None)
+
+        def _is_truthy(val) -> bool:
+            if isinstance(val, bool):
+                return val
+            if isinstance(val, str):
+                return val.lower() in ("true", "1", "yes")
+            return bool(val)
+
+        # Build service status list
+        services = []
+
+        # --- SSO (Authentik / OIDC) ---
+        _oidc_linked = bool(_get_effective("authentik_client_id") and _get_effective("authentik_client_secret"))
+        services.append(
+            {
+                "key": "oidc",
+                "name": _get_effective("oauth_provider_name") or "Single Sign-On",
+                "icon": "fas fa-lock",
+                "type": "SSO",
+                "linked": _oidc_linked,
+                "description": "OpenID Connect SSO provider",
+                "settings_keys": [
+                    "authentik_client_id",
+                    "authentik_client_secret",
+                    "authentik_config_url",
+                    "oauth_provider_name",
+                ],
+            }
+        )
+
+        # --- Google ---
+        _google_id = _get_effective("social_auth_google_client_id")
+        _google_secret = _get_effective("social_auth_google_client_secret")
+        if _is_truthy(_get_effective("social_auth_google_use_global_credentials")) and not (
+            _google_id and _google_secret
+        ):
+            _google_id = _google_id or _get_effective("google_drive_client_id")
+            _google_secret = _google_secret or _get_effective("google_drive_client_secret")
+        _google_linked = bool(
+            _is_truthy(_get_effective("social_auth_google_enabled")) and _google_id and _google_secret
+        )
+        services.append(
+            {
+                "key": "google",
+                "name": "Google",
+                "icon": "fab fa-google",
+                "type": "Sign-in authentication",
+                "linked": _google_linked,
+                "description": "Sign-in authentication",
+                "settings_keys": [
+                    "social_auth_google_enabled",
+                    "social_auth_google_client_id",
+                    "social_auth_google_client_secret",
+                    "social_auth_google_use_global_credentials",
+                ],
+            }
+        )
+
+        # --- GitHub ---
+        _github_linked = bool(
+            _is_truthy(_get_effective("social_auth_github_enabled"))
+            and _get_effective("social_auth_github_client_id")
+            and _get_effective("social_auth_github_client_secret")
+        )
+        services.append(
+            {
+                "key": "github",
+                "name": "GitHub",
+                "icon": "fab fa-github",
+                "type": "Sign-in authentication",
+                "linked": _github_linked,
+                "description": "Sign-in authentication",
+                "settings_keys": [
+                    "social_auth_github_enabled",
+                    "social_auth_github_client_id",
+                    "social_auth_github_client_secret",
+                ],
+            }
+        )
+
+        # --- Microsoft ---
+        _ms_id = _get_effective("social_auth_microsoft_client_id")
+        _ms_secret = _get_effective("social_auth_microsoft_client_secret")
+        if _is_truthy(_get_effective("social_auth_microsoft_use_global_credentials")) and not (_ms_id and _ms_secret):
+            _ms_id = _ms_id or _get_effective("onedrive_client_id")
+            _ms_secret = _ms_secret or _get_effective("onedrive_client_secret")
+        _microsoft_linked = bool(_is_truthy(_get_effective("social_auth_microsoft_enabled")) and _ms_id and _ms_secret)
+        services.append(
+            {
+                "key": "microsoft",
+                "name": "Microsoft",
+                "icon": "fab fa-microsoft",
+                "type": "Sign-in authentication",
+                "linked": _microsoft_linked,
+                "description": "Sign-in authentication",
+                "settings_keys": [
+                    "social_auth_microsoft_enabled",
+                    "social_auth_microsoft_client_id",
+                    "social_auth_microsoft_client_secret",
+                    "social_auth_microsoft_tenant",
+                    "social_auth_microsoft_use_global_credentials",
+                ],
+            }
+        )
+
+        # --- Apple ---
+        _apple_linked = bool(
+            _is_truthy(_get_effective("social_auth_apple_enabled"))
+            and _get_effective("social_auth_apple_client_id")
+            and _get_effective("social_auth_apple_team_id")
+        )
+        services.append(
+            {
+                "key": "apple",
+                "name": "Apple",
+                "icon": "fab fa-apple",
+                "type": "Sign-in authentication",
+                "linked": _apple_linked,
+                "description": "Sign-in authentication",
+                "settings_keys": [
+                    "social_auth_apple_enabled",
+                    "social_auth_apple_client_id",
+                    "social_auth_apple_team_id",
+                    "social_auth_apple_key_id",
+                    "social_auth_apple_private_key",
+                ],
+            }
+        )
+
+        # --- Dropbox ---
+        _dbx_id = _get_effective("social_auth_dropbox_client_id")
+        _dbx_secret = _get_effective("social_auth_dropbox_client_secret")
+        if _is_truthy(_get_effective("social_auth_dropbox_use_global_credentials")) and not (_dbx_id and _dbx_secret):
+            _dbx_id = _dbx_id or _get_effective("dropbox_app_key")
+            _dbx_secret = _dbx_secret or _get_effective("dropbox_app_secret")
+        _dropbox_linked = bool(_is_truthy(_get_effective("social_auth_dropbox_enabled")) and _dbx_id and _dbx_secret)
+        services.append(
+            {
+                "key": "dropbox",
+                "name": "Dropbox",
+                "icon": "fab fa-dropbox",
+                "type": "Sign-in authentication",
+                "linked": _dropbox_linked,
+                "description": "Sign-in authentication",
+                "settings_keys": [
+                    "social_auth_dropbox_enabled",
+                    "social_auth_dropbox_client_id",
+                    "social_auth_dropbox_client_secret",
+                    "social_auth_dropbox_use_global_credentials",
+                ],
+            }
+        )
+
+        # --- Keycloak ---
+        _keycloak_linked = bool(
+            _is_truthy(_get_effective("social_auth_keycloak_enabled"))
+            and _get_effective("social_auth_keycloak_client_id")
+            and _get_effective("social_auth_keycloak_client_secret")
+            and _get_effective("social_auth_keycloak_server_url")
+            and _get_effective("social_auth_keycloak_realm")
+        )
+        services.append(
+            {
+                "key": "keycloak",
+                "name": "Keycloak",
+                "icon": "fas fa-key",
+                "type": "SSO",
+                "linked": _keycloak_linked,
+                "description": "SSO",
+                "settings_keys": [
+                    "social_auth_keycloak_enabled",
+                    "social_auth_keycloak_client_id",
+                    "social_auth_keycloak_client_secret",
+                    "social_auth_keycloak_server_url",
+                    "social_auth_keycloak_realm",
+                ],
+            }
+        )
+
+        # --- Generic OAuth2 ---
+        _generic_oauth2_linked = bool(
+            _is_truthy(_get_effective("social_auth_generic_oauth2_enabled"))
+            and _get_effective("social_auth_generic_oauth2_client_id")
+            and _get_effective("social_auth_generic_oauth2_client_secret")
+            and _get_effective("social_auth_generic_oauth2_authorize_url")
+            and _get_effective("social_auth_generic_oauth2_token_url")
+        )
+        services.append(
+            {
+                "key": "generic_oauth2",
+                "name": "Generic OAuth2",
+                "icon": "fas fa-sign-in-alt",
+                "type": "SSO",
+                "linked": _generic_oauth2_linked,
+                "description": "SSO",
+                "settings_keys": [
+                    "social_auth_generic_oauth2_enabled",
+                    "social_auth_generic_oauth2_client_id",
+                    "social_auth_generic_oauth2_client_secret",
+                    "social_auth_generic_oauth2_authorize_url",
+                    "social_auth_generic_oauth2_token_url",
+                    "social_auth_generic_oauth2_userinfo_url",
+                    "social_auth_generic_oauth2_scope",
+                    "social_auth_generic_oauth2_name",
+                ],
+            }
+        )
+
+        # --- SAML2 ---
+        _saml2_configured = bool(
+            _is_truthy(_get_effective("social_auth_saml2_enabled"))
+            and _get_effective("social_auth_saml2_sso_url")
+            and _get_effective("social_auth_saml2_entity_id")
+        )
+        services.append(
+            {
+                "key": "saml2",
+                "name": settings.social_auth_saml2_name or "SAML2",
+                "icon": "fas fa-id-badge",
+                "type": "SSO (SAML)",
+                "linked": _saml2_configured,
+                "description": "SSO (SAML)",
+                "settings_keys": [
+                    "social_auth_saml2_enabled",
+                    "social_auth_saml2_entity_id",
+                    "social_auth_saml2_sso_url",
+                    "social_auth_saml2_certificate",
+                    "social_auth_saml2_name",
+                ],
+            }
+        )
+
+        # --- SMTP Mail ---
+        _smtp_configured = bool(_get_effective("email_host") and _get_effective("email_username"))
+        services.append(
+            {
+                "key": "smtp",
+                "name": "SMTP Mail",
+                "icon": "fas fa-envelope",
+                "type": "Email Notifications",
+                "linked": _smtp_configured,
+                "description": "Email Notifications",
+                "settings_keys": [
+                    "email_host",
+                    "email_port",
+                    "email_username",
+                    "email_password",
+                    "email_use_tls",
+                    "email_sender",
+                ],
+            }
+        )
+
+        # --- Telegram Bot ---
+        _telegram_configured = bool(
+            _is_truthy(_get_effective("telegram_enabled")) and _get_effective("telegram_bot_token")
+        )
+        services.append(
+            {
+                "key": "telegram",
+                "name": "Telegram Bot",
+                "icon": "fab fa-telegram",
+                "type": "Notifications",
+                "linked": _telegram_configured,
+                "description": "Configure Telegram bot connectivity, access controls, and feedback behavior.",
+                "settings_keys": [
+                    "telegram_enabled",
+                    "telegram_bot_token",
+                    "telegram_chat_id",
+                ],
+            }
+        )
+
+        # Get setting details for the modal forms
+        service_settings = {}
+        for svc in services:
+            svc_settings = []
+            for skey in svc["settings_keys"]:
+                meta = get_setting_metadata(skey)
+                # Get current effective value
+                val = _get_effective(skey)
+                display_val = val
+                if meta.get("sensitive") and val:
+                    display_val = mask_sensitive_value(val)
+                svc_settings.append(
+                    {
+                        "key": skey,
+                        "value": val,
+                        "display_value": display_val if display_val is not None else "",
+                        "metadata": meta,
+                    }
+                )
+            service_settings[svc["key"]] = svc_settings
+
+        # Feature toggles
+        sso_auto_login = _is_truthy(_get_effective("sso_auto_login"))
+        qr_login_enabled = _is_truthy(_get_effective("qr_login_enabled"))
+        frontend_url_configured = bool(_get_effective("public_base_url"))
+
+        return templates.TemplateResponse(
+            "admin_connections.html",
+            {
+                "request": request,
+                "services": services,
+                "service_settings": service_settings,
+                "sso_auto_login": sso_auto_login,
+                "oauth_configured": _oidc_linked,
+                "qr_login_enabled": qr_login_enabled,
+                "frontend_url_configured": frontend_url_configured,
+                "app_version": settings.version,
+            },
+        )
+    except HTTPException:
+        raise
+    except Exception as e:
+        logger.error(f"Error loading connections page: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to load connections page",
+        )
+
+
 @router.get("/admin/settings/audit-log")
 @require_login
 @require_admin_access

app/views/share.py

@@ -23,6 +23,7 @@ templates = Jinja2Templates(directory=str(_templates_dir))
 async def shared_link_view(request: Request, token: str):
     """Render the public share landing page for a given token."""
     return templates.TemplateResponse(
+        request,
         "shared_link_view.html",
-        {"request": request, "token": token},
+        context={"token": token},
     )

app/views/system_reset.py

@@ -0,0 +1,40 @@
+"""
+System reset view — admin-only UI page.
+
+Renders a confirmation-heavy page that allows administrators to:
+1. **Full Reset** — wipe all user data (DB + disk) for a fresh start.
+2. **Reset & Re-import** — move originals to a reimport folder, wipe,
+   and let the watch-folder mechanism re-ingest them.
+
+Both options are gated behind the ``ENABLE_FACTORY_RESET`` feature flag.
+"""
+
+import logging
+
+from fastapi import Depends, Request
+from fastapi.responses import RedirectResponse, Response
+from sqlalchemy.orm import Session
+
+from app.config import settings
+from app.views.base import APIRouter, get_db, require_login, templates
+from app.views.settings import require_admin_access
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/admin/system-reset")
+@require_login
+@require_admin_access
+async def system_reset_page(request: Request, db: Session = Depends(get_db)) -> Response:
+    """Render the system reset administration page."""
+    if not settings.enable_factory_reset:
+        return RedirectResponse(url="/settings", status_code=302)
+
+    return templates.TemplateResponse(
+        "system_reset.html",
+        {
+            "request": request,
+            "factory_reset_on_startup": settings.factory_reset_on_startup,
+        },
+    )

benchmark_duplicates.py

@@ -0,0 +1,101 @@
+import time
+import os
+import sys
+import asyncio
+
+# Mock settings before app imports to bypass validation
+os.environ["DATABASE_URL"] = "sqlite:///:memory:"
+os.environ["REDIS_URL"] = "redis://localhost:6379"
+os.environ["OPENAI_API_KEY"] = "mock_key"
+os.environ["WORKDIR"] = "/tmp/workdir"
+os.environ["AZURE_AI_KEY"] = "mock"
+os.environ["AZURE_REGION"] = "mock"
+os.environ["AZURE_ENDPOINT"] = "http://mock"
+os.environ["GOTENBERG_URL"] = "http://mock"
+os.environ["SESSION_SECRET"] = "mock_secret_mock_secret_mock_secret_mock_secret"
+
+# Ensure app package is accessible
+sys.path.insert(0, os.path.abspath(os.path.dirname(__file__)))
+
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+from app.database import Base
+from app.models import FileRecord
+from app.api.duplicates import list_duplicate_groups
+
+# Mocking Request object
+class MockRequest:
+    def __init__(self):
+        self.session = {"user": {"username": "testuser"}}
+        self.state = type('State', (), {'user': {"username": "testuser"}})()
+
+    class MockURL:
+        def include_query_params(self, **kwargs):
+            return f"http://testserver/api/duplicates?page={kwargs.get('page')}"
+    url = MockURL()
+
+def setup_db():
+    engine = create_engine('sqlite:///:memory:')
+    Base.metadata.create_all(engine)
+    Session = sessionmaker(bind=engine)
+    db = Session()
+    return db
+
+def populate_data(db, num_groups, duplicates_per_group):
+    for i in range(num_groups):
+        filehash = f"hash_{i}"
+        # Original
+        original = FileRecord(
+            filehash=filehash,
+            local_filename=f"orig_{i}.txt",
+            file_size=100,
+            is_duplicate=False
+        )
+        db.add(original)
+        # Duplicates
+        for j in range(duplicates_per_group):
+            dup = FileRecord(
+                filehash=filehash,
+                local_filename=f"dup_{i}_{j}.txt",
+                file_size=100,
+                is_duplicate=True
+            )
+            db.add(dup)
+    db.commit()
+
+async def run_benchmark(db):
+    request = MockRequest()
+    start_time = time.time()
+
+    # Run the function we want to benchmark
+    result = list_duplicate_groups(request=request, db=db, page=1, per_page=500)
+    if asyncio.iscoroutine(result):
+        result = await result
+
+    end_time = time.time()
+    return end_time - start_time, result
+
+async def main():
+    db = setup_db()
+    # 500 groups, each with 20 duplicates = 10500 records total
+    print("Populating data...")
+    populate_data(db, 500, 20)
+    print("Data populated. Running baseline benchmark...")
+
+    # Warmup
+    result = list_duplicate_groups(request=MockRequest(), db=db, page=1, per_page=500)
+    if asyncio.iscoroutine(result):
+        await result
+
+    # Benchmark
+    total_time = 0
+    iterations = 10
+    for _ in range(iterations):
+        time_taken, _ = await run_benchmark(db)
+        total_time += time_taken
+
+    avg_time = total_time / iterations
+    print(f"Average time over {iterations} iterations: {avg_time:.4f} seconds")
+
+if __name__ == "__main__":
+    asyncio.run(main())

benchmark_notifications.py

@@ -0,0 +1,50 @@
+import json
+import time
+import pytest
+from app.database import get_db
+from app.models import UserNotificationTarget, UserNotificationPreference
+from app.main import app
+from tests.test_notifications_api import _make_client, _OWNER, _cleanup
+import statistics
+
+def run_benchmark(notif_engine, notif_session, client, items_count, iterations=5):
+    # Setup
+    target = UserNotificationTarget(
+        owner_id=_OWNER,
+        channel_type="webhook",
+        name="My Webhook",
+        config=json.dumps({"url": "https://x.com"}),
+    )
+    notif_session.add(target)
+    notif_session.commit()
+    notif_session.refresh(target)
+
+    # Generate big payload
+    preferences = []
+    for i in range(items_count):
+        preferences.append({
+            "event_type": f"event.type.{i}",
+            "channel_type": "webhook",
+            "is_enabled": True,
+            "target_id": target.id,
+        })
+
+    payload = {"preferences": preferences}
+
+    # Warm up
+    client.put("/api/user-notifications/preferences", json=payload)
+
+    times = []
+    for _ in range(iterations):
+        # Alter the values a bit so it's a real update
+        for p in payload["preferences"]:
+            p["is_enabled"] = not p["is_enabled"]
+
+        start = time.time()
+        resp = client.put("/api/user-notifications/preferences", json=payload)
+        end = time.time()
+
+        assert resp.status_code == 200
+        times.append(end - start)
+
+    return statistics.mean(times)

benchmark_onedrive.py

@@ -0,0 +1,69 @@
+import asyncio
+import time
+import httpx
+from unittest.mock import patch, MagicMock, AsyncMock
+from app.api.onedrive import test_onedrive_token
+from app.config import settings
+
+settings.onedrive_refresh_token = "dummy"
+settings.onedrive_client_id = "dummy"
+settings.onedrive_client_secret = "dummy"
+
+class DummyRequest:
+    def __init__(self):
+        self.session = {"user": "dummy"}
+
+async def run_benchmark(func_name, mock_post, mock_get):
+    mock_post_resp = MagicMock()
+    mock_post_resp.status_code = 200
+    mock_post_resp.json.return_value = {
+        "access_token": "dummy_access",
+        "expires_in": 3600
+    }
+    mock_post.return_value = mock_post_resp
+
+    mock_get_resp = MagicMock()
+    mock_get_resp.status_code = 200
+    mock_get_resp.json.return_value = {
+        "displayName": "Test User",
+        "userPrincipalName": "test@example.com"
+    }
+    mock_get.return_value = mock_get_resp
+
+    start_time = time.time()
+    for _ in range(100):
+        await test_onedrive_token(DummyRequest())
+    end_time = time.time()
+    print(f"{func_name} took {end_time - start_time:.4f} seconds")
+
+async def run_benchmark_async(func_name, mock_post, mock_get):
+    mock_post_resp = MagicMock()
+    mock_post_resp.status_code = 200
+    mock_post_resp.json = MagicMock(return_value={
+        "access_token": "dummy_access",
+        "expires_in": 3600
+    })
+    mock_post.return_value = mock_post_resp
+
+    mock_get_resp = MagicMock()
+    mock_get_resp.status_code = 200
+    mock_get_resp.json = MagicMock(return_value={
+        "displayName": "Test User",
+        "userPrincipalName": "test@example.com"
+    })
+    mock_get.return_value = mock_get_resp
+
+    start_time = time.time()
+    for _ in range(100):
+        await test_onedrive_token(DummyRequest())
+    end_time = time.time()
+    print(f"{func_name} took {end_time - start_time:.4f} seconds")
+
+
+@patch('app.api.onedrive.requests.get')
+@patch('app.api.onedrive.requests.post')
+def benchmark_sync(mock_post, mock_get):
+    asyncio.run(run_benchmark("Sync requests (baseline)", mock_post, mock_get))
+
+if __name__ == "__main__":
+    benchmark_sync()

benchmark_url_upload.py

@@ -0,0 +1,49 @@
+import asyncio
+import time
+from unittest.mock import Mock, patch
+
+from app.api.url_upload import process_url, URLUploadRequest
+from app.config import settings
+
+async def main():
+    # Mock request and URLUploadRequest
+    request = Mock()
+    url_request = URLUploadRequest(url="https://example.com/file.pdf")
+
+    # Generate a large chunk
+    large_chunk = b"A" * 8192
+    num_chunks = 10000 # 8192 * 10000 = ~80MB
+
+    mock_response = Mock()
+    mock_response.status_code = 200
+    mock_response.headers = {"Content-Type": "application/pdf"}
+    mock_response.iter_content = Mock(return_value=[large_chunk] * num_chunks)
+
+    # For async client later
+    class AsyncMockResponse:
+        def __init__(self):
+            self.status_code = 200
+            self.headers = {"Content-Type": "application/pdf"}
+        def raise_for_status(self):
+            pass
+        async def aiter_bytes(self, chunk_size):
+            for _ in range(num_chunks):
+                yield large_chunk
+
+    async_mock_response = AsyncMockResponse()
+
+    # We will mock requests.get for synchronous, httpx.AsyncClient.get for asynchronous
+
+    # Test sync
+    start_time = time.time()
+    with patch("app.api.url_upload.requests.get", return_value=mock_response), \
+         patch("app.api.url_upload.process_document"):
+        try:
+            await process_url(request=request, url_request=url_request)
+        except Exception as e:
+            print(f"Error: {e}")
+    end_time = time.time()
+    print(f"Original execution time (sync writing): {end_time - start_time:.4f} seconds")
+
+if __name__ == "__main__":
+    asyncio.run(main())

benchmark_url_upload2.py

@@ -0,0 +1,75 @@
+import asyncio
+import time
+import os
+import shutil
+import tempfile
+from unittest.mock import Mock, patch
+from fastapi import HTTPException
+
+from app.api.url_upload import process_url, URLUploadRequest
+from app.config import settings
+
+async def main():
+    # Setup test dir
+    test_dir = tempfile.mkdtemp()
+    settings.workdir = test_dir
+
+    # Mock request and URLUploadRequest
+    request = Mock()
+    url_request = URLUploadRequest(url="https://example.com/file.pdf")
+
+    # Generate a large chunk
+    chunk_size = 8192
+    num_chunks = 20000 # 20000 * 8192 = ~160MB
+    large_chunk = b"A" * chunk_size
+
+    class SyncMockResponse:
+        def __init__(self):
+            self.status_code = 200
+            self.headers = {"Content-Type": "application/pdf"}
+        def raise_for_status(self):
+            pass
+        def iter_content(self, chunk_size):
+            for _ in range(num_chunks):
+                # sleep slightly to simulate network latency, otherwise OS file cache obscures the difference
+                time.sleep(0.0001)
+                yield large_chunk
+
+    sync_mock_response = SyncMockResponse()
+
+    class AsyncMockResponse:
+        def __init__(self):
+            self.status_code = 200
+            self.headers = {"Content-Type": "application/pdf"}
+            self.is_success = True
+            self.status_code = 200
+        def raise_for_status(self):
+            pass
+        async def aiter_bytes(self, chunk_size=8192):
+            for _ in range(num_chunks):
+                await asyncio.sleep(0.0001)
+                yield large_chunk
+
+    class AsyncMockContext:
+        async def __aenter__(self):
+            return AsyncMockResponse()
+        async def __aexit__(self, exc_type, exc_val, exc_tb):
+            pass
+
+    async_mock_response = AsyncMockResponse()
+
+    # Test sync
+    start_time = time.time()
+    with patch("app.api.url_upload.requests.get", return_value=sync_mock_response), \
+         patch("app.api.url_upload.process_document"):
+        try:
+            await process_url(request=request, url_request=url_request)
+        except Exception as e:
+            print(f"Error (sync): {e}")
+    end_time = time.time()
+    print(f"Original execution time (sync writing): {end_time - start_time:.4f} seconds")
+
+    shutil.rmtree(test_dir)
+
+if __name__ == "__main__":
+    asyncio.run(main())

docker-compose.yaml

@@ -3,7 +3,7 @@ services:
     build:
       context: .
       dockerfile: Dockerfile
-    container_name: document_api
+    # No container_name — allows `docker compose up --scale api=N`
     restart: always
 
     # We'll keep the code in /app, but set working_dir to the shared data directory
@@ -24,7 +24,7 @@ services:
 
     depends_on:
       - redis
-      - worker
+      - beat
 
     # Mount the shared working directory for data
     volumes:
@@ -34,13 +34,14 @@ services:
     build:
       context: .
       dockerfile: Dockerfile
-    container_name: document_worker
+    # No container_name — allows `docker compose up --scale worker=N`
     restart: always
 
     # same shared working directory
     working_dir: /workdir
 
-    command: ["celery", "-A", "app.celery_worker", "worker", "-B", "--loglevel=info", "-Q", "document_processor,default,celery"]
+    # Workers process tasks only — no -B flag (Beat runs in the dedicated beat service)
+    command: ["celery", "-A", "app.celery_worker", "worker", "--loglevel=info", "-Q", "document_processor,default,celery"]
     env_file:
       - .env
     environment:
@@ -54,6 +55,26 @@ services:
     volumes:
       - /var/docparse/workdir:/workdir
 
+  # Dedicated Celery Beat scheduler — exactly one instance must run at all times.
+  # Beat publishes periodic tasks to the Redis broker; workers pick them up.
+  # Do NOT scale this service (replicas must stay at 1).
+  beat:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    container_name: document_beat
+    restart: always
+    working_dir: /workdir
+    command: ["celery", "-A", "app.celery_worker", "beat", "--loglevel=info"]
+    env_file:
+      - .env
+    environment:
+      - PYTHONPATH=/app
+    depends_on:
+      - redis
+    volumes:
+      - /var/docparse/workdir:/workdir
+
   gotenberg:
     image: gotenberg/gotenberg:latest
     container_name: gotenberg

docs/API.md

@@ -27,9 +27,11 @@ DocuElevate implements rate limiting to protect against abuse and DoS attacks. R
 ### Default Limits
 
 - **Default endpoints**: 100 requests per minute
-- **File upload**: 600 requests per minute
+- **File upload**: 600 requests per minute (global) + 20 per user per 60 s (per-user, health-aware)
 - **Authentication**: 10 requests per minute
 
+**Per-user upload rate limiting**: Upload endpoints (`/api/ui-upload`, `/api/process-url`) enforce a per-user sliding-window limit that adapts to system load. Under heavy queue depth or high CPU usage, the effective limit is reduced automatically. See the [Configuration Guide](ConfigurationGuide.md#per-user-upload-rate-limiting) for details.
+
 **Note**: Document processing endpoints (OCR, metadata extraction) use built-in queue throttling to control processing rates and prevent upstream API overloads. No additional API-level rate limit is applied to processing endpoints.
 
 ### Rate Limit Headers
@@ -53,6 +55,10 @@ RATE_LIMITING_ENABLED=true
 RATE_LIMIT_DEFAULT=100/minute
 RATE_LIMIT_UPLOAD=600/minute
 RATE_LIMIT_AUTH=10/minute
+
+# Per-user upload rate limiting (health-aware)
+UPLOAD_RATE_LIMIT_PER_USER=20   # Max uploads per user per window
+UPLOAD_RATE_LIMIT_WINDOW=60     # Sliding window in seconds
 ```
 
 See [Configuration Guide](ConfigurationGuide.md) for more details.
@@ -115,7 +121,8 @@ curl -X GET "http://<your-docuelevate-instance>/api/files" \
 |--------|----------|-------------|
 | `POST` | `/api/api-tokens/` | Create a new token |
 | `GET` | `/api/api-tokens/` | List all your tokens |
-| `DELETE` | `/api/api-tokens/{id}` | Revoke a token |
+| `DELETE` | `/api/api-tokens/{id}` | Revoke (active) or permanently delete (revoked) a token |
+| `POST` | `/api/api-tokens/{id}/reactivate` | Reactivate a revoked token |
 
 ### Session Authentication
 
@@ -235,17 +242,33 @@ The DocuElevate browser extension uses this endpoint to send files directly from
 
 **POST** `/api/ui-upload`
 
-Upload one or more files from your computer for processing.
+Upload a file from your computer for processing.
 
 **Request**:
-- Multipart form data with file(s)
+- Multipart form data with a single `file` field
 
-**Response**:
+**Response** (new file):
 ```json
 {
-  "success": true,
-  "file_ids": [123, 124],
-  "message": "Files uploaded and queued for processing"
+  "task_id": "abc-123",
+  "status": "queued",
+  "original_filename": "invoice.pdf",
+  "stored_filename": "a1b2c3d4.pdf"
+}
+```
+
+**Response** (exact duplicate, when `ENABLE_DEDUPLICATION=True`):
+```json
+{
+  "status": "duplicate",
+  "original_filename": "invoice.pdf",
+  "stored_filename": "e5f6a7b8.pdf",
+  "duplicate_of": {
+    "duplicate_type": "exact",
+    "original_file_id": 42,
+    "original_filename": "invoice.pdf",
+    "message": "This file is an exact duplicate of an already-processed document. It has not been queued for processing again."
+  }
 }
 ```
 
@@ -1356,7 +1379,7 @@ Test an integration connection without saving. Useful for "Test connection" UI b
 {"success": true, "message": "IMAP connection successful"}
 ```
 
-Supported connection tests: `IMAP`, `S3`, `WEBDAV`, `NEXTCLOUD`. Other types return a message that testing is not yet supported.
+Supported connection tests: `DROPBOX`, `IMAP`, `S3`, `WEBDAV`, `NEXTCLOUD`. Other types return a message that testing is not yet supported.
 
 ### GET /api/integrations/quota/
 
@@ -1381,6 +1404,55 @@ Get the current user's integration quota usage.
 }
 ```
 
+## Cloud Provider Folder Browser
+
+Browse folders in connected cloud storage providers. These endpoints are used by the OAuth callback pages to let users select a target folder after authorization.
+
+### POST /api/dropbox/list-folders
+
+List folders in a Dropbox account. Requires a short-lived OAuth access token obtained during the authorization flow.
+
+**Request (form-data):**
+
+| Field          | Type   | Required | Description                          |
+|----------------|--------|----------|--------------------------------------|
+| `access_token` | string | Yes      | Dropbox OAuth access token           |
+| `path`         | string | No       | Folder path to list (default: root)  |
+
+**Response (200):**
+```json
+{
+  "folders": [
+    { "name": "Documents", "path": "/Documents", "id": "id:abc123" },
+    { "name": "Photos", "path": "/Photos", "id": "id:def456" }
+  ],
+  "path": "/",
+  "has_more": false
+}
+```
+
+### POST /api/onedrive/list-folders
+
+List folders in a OneDrive account. Requires a short-lived OAuth access token obtained during the authorization flow.
+
+**Request (form-data):**
+
+| Field          | Type   | Required | Description                          |
+|----------------|--------|----------|--------------------------------------|
+| `access_token` | string | Yes      | Microsoft Graph access token         |
+| `path`         | string | No       | Folder path to list (default: root)  |
+
+**Response (200):**
+```json
+{
+  "folders": [
+    { "name": "Documents", "path": "/Documents", "id": "abc123", "child_count": 5 },
+    { "name": "Pictures", "path": "/Pictures", "id": "def456", "child_count": 12 }
+  ],
+  "path": "/"
+}
+```
+
 ## Webhooks
 
 Manage webhook configurations for notifying external systems when document events occur. All webhook endpoints require admin access.
@@ -1571,6 +1643,47 @@ Lightweight endpoint returning the total number of queued + in-progress items. D
 
 ## Diagnostic
 
+### GET /api/diagnostic/healthz/live
+
+Lightweight liveness probe for Kubernetes.  Returns **200 OK** as long as the process is running.  This endpoint does **not** check external dependencies and is intentionally cheap.
+
+**Authentication:** None (designed for kubelet probes)
+
+**Response (200 OK):**
+```json
+{
+  "status": "ok"
+}
+```
+
+### GET /api/diagnostic/healthz/ready
+
+Readiness probe for Kubernetes.  Verifies that the application can serve traffic by checking database and Redis connectivity.
+
+**Authentication:** None (designed for kubelet probes)
+
+**Response (200 OK) – ready to serve traffic:**
+```json
+{
+  "status": "ready",
+  "checks": {
+    "database": {"status": "ok"},
+    "redis":    {"status": "ok"}
+  }
+}
+```
+
+**Response (503 Service Unavailable) – database unreachable:**
+```json
+{
+  "status": "not_ready",
+  "checks": {
+    "database": {"status": "error", "detail": "..."},
+    "redis":    {"status": "ok"}
+  }
+}
+```
+
 ### GET /api/diagnostic/health
 
 System health endpoint designed for monitoring tools such as Grafana, Uptime Kuma, Prometheus blackbox exporter, or any HTTP-based health checker.
@@ -2127,12 +2240,14 @@ Usage tracking records when each token was last used and from which IP address.
 
 ### POST /api/api-tokens/
 
-Create a new API token.
+Create a new API token.  Optionally specify a lifetime in days via
+`expires_in_days` (1–3650).  If omitted the token never expires.
 
 **Request:**
 ```json
 {
-  "name": "CI Pipeline"
+  "name": "CI Pipeline",
+  "expires_in_days": 90
 }
 ```
 
@@ -2147,7 +2262,8 @@ Create a new API token.
   "last_used_at": null,
   "last_used_ip": null,
   "created_at": "2026-03-08T12:00:00Z",
-  "revoked_at": null
+  "revoked_at": null,
+  "expires_at": "2026-06-06T12:00:00Z"
 }
 ```
 
@@ -2169,15 +2285,20 @@ List all tokens for the authenticated user. The full token value is never includ
     "last_used_at": "2026-03-08T15:30:00Z",
     "last_used_ip": "203.0.113.42",
     "created_at": "2026-03-08T12:00:00Z",
-    "revoked_at": null
+    "revoked_at": null,
+    "expires_at": "2026-06-06T12:00:00Z"
   }
 ]
 ```
 
 ### DELETE /api/api-tokens/{token_id}
 
-Revoke a token. The token is soft-deleted (kept for audit purposes) and can no
-longer be used for authentication.
+Revoke or permanently delete a token:
+
+* **Active token** – soft-revoked (kept for audit purposes, marked inactive).
+  Response: `{"detail": "Token revoked"}`
+* **Already-revoked token** – permanently deleted from the database.
+  Response: `{"detail": "Token deleted"}`
 
 **Response (200):**
 ```json
@@ -2186,6 +2307,13 @@ longer be used for authentication.
 }
 ```
 
+### POST /api/api-tokens/{token_id}/reactivate
+
+Reactivate a previously revoked token.  Clears `revoked_at` and sets
+`is_active` back to `true`.
+
+**Response (200):** The updated `TokenResponse` object.
+
 ### Using API Tokens
 
 Include the token in the `Authorization` header of any API request:
@@ -2214,6 +2342,316 @@ print(response.json())
 ```
 
 
+## Classification Rules
+
+The classification rules API lets you manage custom document classification rules.  Rules are evaluated during the `classify` pipeline step to assign a category to each document based on filename patterns, content keywords, and metadata fields.
+
+### Built-in Categories
+
+```bash
+GET /api/classification-rules/categories
+```
+
+Returns the pre-built classification categories.
+
+**Response (200):**
+```json
+{
+  "invoice": "Invoice",
+  "contract": "Contract",
+  "receipt": "Receipt",
+  "letter": "Letter",
+  "report": "Report",
+  "bank_statement": "Bank Statement",
+  "tax_document": "Tax Document",
+  "insurance": "Insurance Document",
+  "payslip": "Payslip",
+  "unknown": "Unknown"
+}
+```
+
+### Rule Types
+
+```bash
+GET /api/classification-rules/rule-types
+```
+
+Returns the supported rule types with descriptions.
+
+**Response (200):**
+```json
+[
+  {
+    "type": "filename_pattern",
+    "label": "Filename Pattern",
+    "description": "Regex pattern matched against the original filename."
+  },
+  {
+    "type": "content_keyword",
+    "label": "Content Keyword",
+    "description": "Pipe-separated keywords matched against the OCR text."
+  },
+  {
+    "type": "metadata_match",
+    "label": "Metadata Match",
+    "description": "field=value pattern matched against existing AI metadata."
+  }
+]
+```
+
+### List Rules
+
+```bash
+GET /api/classification-rules/
+```
+
+List all classification rules visible to the current user (system rules + own rules).
+
+**Response (200):**
+```json
+[
+  {
+    "id": 1,
+    "owner_id": "user@example.com",
+    "name": "German Invoice Filename",
+    "category": "invoice",
+    "rule_type": "filename_pattern",
+    "pattern": "(?i)rechnung",
+    "priority": 10,
+    "case_sensitive": false,
+    "enabled": true
+  }
+]
+```
+
+### Create Rule
+
+```bash
+POST /api/classification-rules/
+```
+
+Create a new custom classification rule.
+
+**Request:**
+```json
+{
+  "name": "German Invoice Filename",
+  "category": "invoice",
+  "rule_type": "filename_pattern",
+  "pattern": "(?i)rechnung",
+  "priority": 10,
+  "case_sensitive": false,
+  "enabled": true
+}
+```
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | string | Yes | Unique rule name (per user) |
+| `category` | string | Yes | Target category (e.g. `invoice`, `contract`, or custom) |
+| `rule_type` | string | Yes | One of: `filename_pattern`, `content_keyword`, `metadata_match` |
+| `pattern` | string | Yes | Regex (filename), pipe-separated keywords (content), or `field=value` (metadata) |
+| `priority` | integer | No | Higher priority rules are evaluated first (default: 0) |
+| `case_sensitive` | boolean | No | Case-sensitive matching (default: false) |
+| `enabled` | boolean | No | Whether the rule is active (default: true) |
+
+**Response (201):**
+```json
+{
+  "id": 1,
+  "owner_id": "user@example.com",
+  "name": "German Invoice Filename",
+  "category": "invoice",
+  "rule_type": "filename_pattern",
+  "pattern": "(?i)rechnung",
+  "priority": 10,
+  "case_sensitive": false,
+  "enabled": true
+}
+```
+
+### Get Rule
+
+```bash
+GET /api/classification-rules/{rule_id}
+```
+
+### Update Rule
+
+```bash
+PUT /api/classification-rules/{rule_id}
+```
+
+**Request (partial update):**
+```json
+{
+  "priority": 20,
+  "enabled": false
+}
+```
+
+### Delete Rule
+
+```bash
+DELETE /api/classification-rules/{rule_id}
+```
+
+**Response:** `204 No Content`
+
+
+
+## Automation (Zapier / Make.com)
+
+Manage automation hook subscriptions for integrating DocuElevate with external platforms like Zapier and Make.com. All endpoints require API token authentication (`Authorization: Bearer <token>`).
+
+### Supported Events
+
+The automation system shares event types with the [Webhooks](#webhooks) subsystem:
+
+| Event | Description |
+|-------|-------------|
+| `document.uploaded` | A new document has been ingested |
+| `document.processed` | A document finished processing successfully |
+| `document.failed` | Document processing failed |
+| `user.signup` | A new user account was created |
+| `user.plan_changed` | A user's subscription plan changed |
+| `user.payment_issue` | A payment issue was reported for a user |
+
+### GET /api/automation/events
+
+List all valid event types that automation hooks can subscribe to.
+
+**Response (200):**
+```json
+["document.failed", "document.processed", "document.uploaded", "user.payment_issue", "user.plan_changed", "user.signup"]
+```
+
+### POST /api/automation/hooks/subscribe
+
+Subscribe to DocuElevate events. Zapier and Make.com call this endpoint to register a webhook URL that receives event notifications.
+
+**Request:**
+```bash
+curl -X POST "http://your-instance/api/automation/hooks/subscribe" \
+  -H "Authorization: Bearer de_your_token_here" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "target_url": "https://hooks.zapier.com/hooks/catch/123456/abcdef/",
+    "events": ["document.processed", "document.uploaded"],
+    "hook_type": "zapier",
+    "secret": "optional-signing-secret",
+    "description": "My Zap for processed documents"
+  }'
+```
+
+**Response (201):**
+```json
+{
+  "id": 1,
+  "target_url": "https://hooks.zapier.com/hooks/catch/123456/abcdef/",
+  "events": ["document.processed", "document.uploaded"],
+  "is_active": true,
+  "hook_type": "zapier",
+  "description": "My Zap for processed documents",
+  "has_secret": true
+}
+```
+
+### GET /api/automation/hooks
+
+List all automation hook subscriptions.
+
+**Response (200):**
+```json
+[
+  {
+    "id": 1,
+    "target_url": "https://hooks.zapier.com/hooks/catch/123456/abcdef/",
+    "events": ["document.processed", "document.uploaded"],
+    "is_active": true,
+    "hook_type": "zapier",
+    "description": "My Zap for processed documents",
+    "has_secret": true
+  }
+]
+```
+
+### DELETE /api/automation/hooks/{hook_id}
+
+Unsubscribe an automation hook. Zapier calls this when a Zap is turned off or deleted.
+
+**Response (204):** No content.
+
+### GET /api/automation/triggers/sample/{event}
+
+Get sample trigger data for Zapier field mapping. Zapier uses this during Zap setup to discover available fields.
+
+**Request:**
+```bash
+curl "http://your-instance/api/automation/triggers/sample/document.processed" \
+  -H "Authorization: Bearer de_your_token_here"
+```
+
+**Response (200):**
+```json
+[
+  {
+    "id": "evt_sample0002",
+    "event": "document.processed",
+    "timestamp": 1710000060.0,
+    "document_id": 42,
+    "filename": "invoice_2024.pdf",
+    "status": "processed",
+    "title": "Invoice #1234",
+    "owner_id": "user@example.com"
+  }
+]
+```
+
+### POST /api/automation/actions/upload
+
+Upload a document from an automation platform. This incoming action endpoint allows Zapier or Make.com to push documents into DocuElevate for processing.
+
+**Request:**
+```bash
+curl -X POST "http://your-instance/api/automation/actions/upload" \
+  -H "Authorization: Bearer de_your_token_here" \
+  -F "file=@/path/to/document.pdf"
+```
+
+**Response (200):**
+```json
+{
+  "status": "accepted",
+  "filename": "document.pdf",
+  "task_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
+}
+```
+
+### Zapier-Compatible Payload Format
+
+When events fire, automation hooks receive a **flat JSON payload** (no nested `data` key) that Zapier and Make.com can easily map:
+
+```json
+{
+  "id": "evt_a1b2c3d4e5f67890",
+  "event": "document.processed",
+  "timestamp": 1710000060.0,
+  "document_id": 42,
+  "filename": "invoice_2024.pdf",
+  "status": "processed",
+  "title": "Invoice #1234",
+  "owner_id": "user@example.com"
+}
+```
+
+The `id` field is unique per event and is used by Zapier for deduplication. If a `secret` was provided during subscription, an `X-Webhook-Signature` header with an HMAC-SHA256 signature is included.
+
+### Retry Behavior
+
+Automation hook deliveries follow the same retry policy as regular webhooks: up to 3 retries with exponential backoff (60 s, 300 s, 900 s) and ±20% jitter.
+
+
 ## Further Assistance
 
 For additional help with the API, please contact our support team or refer to the [Development Guide](../CONTRIBUTING.md).
@@ -2268,9 +2706,14 @@ List all registered push-notification devices for the current user.
 
 ### DELETE /api/mobile/devices/{device_id}
 
-Deactivate a push-notification device.  The device will no longer receive push notifications.
+Deactivate or permanently delete a push-notification device:
 
-**Response (204 No Content)**
+* **Active device** – soft-deactivated (record kept, will no longer receive push notifications).
+  Response: `{"detail": "Device deactivated"}`
+* **Already-inactive device** – permanently deleted from the database.
+  Response: `{"detail": "Device deleted"}`
+
+**Response (200)**
 
 ### GET /api/mobile/whoami
 
@@ -2405,3 +2848,401 @@ query GetDocument($id: Int!) {
 }
 ```
 Variables: `{ "id": 42 }`
+
+## System Reset
+
+Admin-only endpoints for resetting the system to a clean state.  Requires `ENABLE_FACTORY_RESET=True`.
+
+### GET /api/admin/system-reset/status
+
+Check whether the system reset feature is enabled.
+
+**Response (200):**
+```json
+{
+  "enabled": true,
+  "factory_reset_on_startup": false
+}
+```
+
+### POST /api/admin/system-reset/full
+
+Wipe all user data (database + work-files).
+
+**Request:**
+```json
+{
+  "confirmation": "DELETE"
+}
+```
+
+**Response (200):**
+```json
+{
+  "status": "ok",
+  "result": {
+    "database": { "files": 42, "processing_logs": 100 },
+    "filesystem": { "deleted_dirs": 5, "deleted_files": 12 }
+  }
+}
+```
+
+### POST /api/admin/system-reset/reimport
+
+Move original files to a reimport folder, wipe everything, and configure the reimport folder as a watch folder for re-ingestion.
+
+**Request:**
+```json
+{
+  "confirmation": "REIMPORT"
+}
+```
+
+**Response (200):**
+```json
+{
+  "status": "ok",
+  "result": {
+    "database": { "files": 42 },
+    "filesystem": { "deleted_dirs": 5, "deleted_files": 12 },
+    "reimport": { "files_moved": 42, "reimport_folder": "/workdir/reimport" }
+  }
+}
+```
+
+---
+
+## Comments & Annotations
+
+Threaded comments and PDF annotations for document collaboration.
+
+### List Comments
+
+**GET** `/api/files/{file_id}/comments`
+
+Returns all comments for a document, organized into a threaded tree.
+
+**Response (200):**
+```json
+{
+  "file_id": 1,
+  "comments": [
+    {
+      "id": 1,
+      "file_id": 1,
+      "user_id": "alice",
+      "parent_id": null,
+      "body": "Please review section 3.",
+      "mentions": ["bob"],
+      "is_resolved": false,
+      "created_at": "2026-03-21T12:00:00+00:00",
+      "updated_at": "2026-03-21T12:00:00+00:00",
+      "replies": [
+        {
+          "id": 2,
+          "file_id": 1,
+          "user_id": "bob",
+          "parent_id": 1,
+          "body": "Done!",
+          "mentions": [],
+          "is_resolved": false,
+          "created_at": "2026-03-21T12:05:00+00:00",
+          "updated_at": "2026-03-21T12:05:00+00:00",
+          "replies": []
+        }
+      ]
+    }
+  ],
+  "total": 2
+}
+```
+
+### Create Comment
+
+**POST** `/api/files/{file_id}/comments`
+
+Create a new comment on a document.  @mentions are automatically extracted from the body.
+
+**Request:**
+```json
+{
+  "body": "Hey @bob, please review this section.",
+  "parent_id": null
+}
+```
+
+**Response (201):**
+```json
+{
+  "id": 3,
+  "file_id": 1,
+  "user_id": "alice",
+  "parent_id": null,
+  "body": "Hey @bob, please review this section.",
+  "mentions": ["bob"],
+  "is_resolved": false,
+  "created_at": "2026-03-21T12:10:00+00:00",
+  "updated_at": "2026-03-21T12:10:00+00:00"
+}
+```
+
+### Update Comment
+
+**PUT** `/api/files/{file_id}/comments/{comment_id}`
+
+Update the body of an existing comment.  Only the comment author may update it.
+
+**Request:**
+```json
+{
+  "body": "Updated comment text @charlie"
+}
+```
+
+### Delete Comment
+
+**DELETE** `/api/files/{file_id}/comments/{comment_id}`
+
+Delete a comment.  Only the comment author may delete it.
+
+**Response:** `204 No Content`
+
+### Resolve / Unresolve Comment
+
+**PATCH** `/api/files/{file_id}/comments/{comment_id}/resolve`
+
+Mark a comment thread as resolved or unresolved.
+
+**Request:**
+```json
+{
+  "is_resolved": true
+}
+```
+
+### List Annotations
+
+**GET** `/api/files/{file_id}/annotations`
+
+Returns all PDF page annotations for a document, ordered by page then creation time.
+
+**Response (200):**
+```json
+{
+  "file_id": 1,
+  "annotations": [
+    {
+      "id": 1,
+      "file_id": 1,
+      "user_id": "alice",
+      "page": 1,
+      "x": 100.0,
+      "y": 200.0,
+      "width": 150.0,
+      "height": 20.0,
+      "content": "Important paragraph",
+      "annotation_type": "highlight",
+      "color": "#ffff00",
+      "created_at": "2026-03-21T12:00:00+00:00",
+      "updated_at": "2026-03-21T12:00:00+00:00"
+    }
+  ],
+  "total": 1
+}
+```
+
+### Create Annotation
+
+**POST** `/api/files/{file_id}/annotations`
+
+Create a new annotation on a PDF page.
+
+**Request:**
+```json
+{
+  "page": 1,
+  "x": 100.0,
+  "y": 200.0,
+  "width": 150.0,
+  "height": 20.0,
+  "content": "Important paragraph",
+  "annotation_type": "highlight",
+  "color": "#ffff00"
+}
+```
+
+Allowed `annotation_type` values: `note`, `highlight`, `underline`, `strikethrough`.
+
+### Update Annotation
+
+**PUT** `/api/files/{file_id}/annotations/{annotation_id}`
+
+Update an existing annotation.  Only the annotation author may update it.
+
+**Request** (all fields optional):
+```json
+{
+  "content": "Updated note",
+  "color": "#00ff00"
+}
+```
+
+### Delete Annotation
+
+**DELETE** `/api/files/{file_id}/annotations/{annotation_id}`
+
+Delete an annotation.  Only the annotation author may delete it.
+
+**Response:** `204 No Content`
+
+### List Mentionable Users
+
+**GET** `/api/users/mentionable`
+
+Returns all non-blocked user profiles for the @mention autocomplete.
+
+**Response (200):**
+```json
+[
+  { "user_id": "alice", "display_name": "Alice Anderson" },
+  { "user_id": "bob", "display_name": "Bob Baker" }
+]
+```
+
+---
+
+## File Sharing & Permissions
+
+DocuElevate supports per-user document sharing with role-based access control.
+
+### Roles
+
+| Role     | View | Comment / Annotate | Edit metadata | Delete | Share |
+|----------|------|--------------------|---------------|--------|-------|
+| `owner`  | ✓    | ✓                  | ✓             | ✓      | ✓     |
+| `editor` | ✓    | ✓                  | ✓             | ✗      | ✗     |
+| `viewer` | ✓    | ✓                  | ✗             | ✗      | ✗     |
+
+- Only the **file owner** can share a document, change roles, or delete the document.
+- When a user is **@mentioned** in a comment they are automatically granted `viewer` access to the document (multi-user mode only).
+
+---
+
+### List Shares
+
+**GET** `/api/files/{file_id}/shares`
+
+Returns all active shares for a document.  Only the file owner (or an admin) may call this endpoint.
+
+**Response (200):**
+```json
+[
+  {
+    "id": 1,
+    "file_id": 42,
+    "owner_id": "alice",
+    "shared_with_user_id": "bob",
+    "role": "viewer",
+    "created_at": "2026-03-22T10:00:00+00:00",
+    "updated_at": "2026-03-22T10:00:00+00:00"
+  }
+]
+```
+
+**Error Responses:**
+- `403`: Not the file owner
+- `404`: File not found
+
+---
+
+### Create Share
+
+**POST** `/api/files/{file_id}/shares`
+
+Share a document with another user.  Only the file owner may call this endpoint.  If the user already has a share, their role is updated.
+
+**Request:**
+```json
+{
+  "shared_with_user_id": "bob",
+  "role": "viewer"
+}
+```
+
+`role` must be `"viewer"` (default) or `"editor"`.
+
+**Response (201):**
+```json
+{
+  "id": 1,
+  "file_id": 42,
+  "owner_id": "alice",
+  "shared_with_user_id": "bob",
+  "role": "viewer",
+  "created_at": "2026-03-22T10:00:00+00:00",
+  "updated_at": "2026-03-22T10:00:00+00:00"
+}
+```
+
+**Error Responses:**
+- `403`: Not the file owner
+- `422`: Invalid role, empty user ID, or sharing with self
+
+---
+
+### Update Share Role
+
+**PUT** `/api/files/{file_id}/shares/{share_id}`
+
+Change the role of an existing share.  Only the file owner may call this endpoint.
+
+**Request:**
+```json
+{
+  "role": "editor"
+}
+```
+
+**Response (200):** Updated share object.
+
+**Error Responses:**
+- `403`: Not the file owner
+- `404`: Share not found
+- `422`: Invalid role
+
+---
+
+### Revoke Share
+
+**DELETE** `/api/files/{file_id}/shares/{share_id}`
+
+Remove a user's access to a document.  Only the file owner may revoke shares.
+
+**Response (200):**
+```json
+{ "status": "success", "message": "Share revoked successfully" }
+```
+
+**Error Responses:**
+- `403`: Not the file owner
+- `404`: Share not found
+
+---
+
+### List Shared With
+
+**GET** `/api/files/{file_id}/shared-with`
+
+Returns who a document is shared with.  Accessible to any user with at least `viewer` access (owner, editors, and viewers can all call this).
+
+**Response (200):**
+```json
+[
+  {
+    "share_id": 1,
+    "user_id": "bob",
+    "display_name": "Bob Baker",
+    "role": "viewer"
+  }
+]
+```

docs/AppleAppStoreCompliance.md

@@ -0,0 +1,285 @@
+# Apple App Store Compliance Audit Report
+
+This document details the findings from a comprehensive audit of the DocuElevate mobile app against Apple's App Store Review Guidelines, Human Interface Guidelines (HIG), and privacy requirements. It covers all areas of compliance, risks for rejection, and recommendations.
+
+> **Last Audited:** March 2026
+> **App Version:** 1.0.0
+> **Expo SDK:** 54.0.0
+> **Bundle ID:** `org.docuelevate.mobile`
+
+---
+
+## Executive Summary
+
+The DocuElevate mobile app is broadly compliant with Apple's App Store requirements. The following issues were identified and resolved as part of this audit:
+
+| Issue | Severity | Status |
+|-------|----------|--------|
+| Unused `fetch` background mode declared | High | ✅ Fixed |
+| Missing privacy manifest for required reason APIs | High | ✅ Fixed |
+| No account deletion option (Guideline 5.1.1(v)) | Critical | ✅ Fixed |
+| No Privacy Policy / Terms of Service links in-app | High | ✅ Fixed |
+| Emoji used as UI icons instead of platform-native icons | Medium | ✅ Fixed |
+| Missing app version display | Low | ✅ Fixed |
+| Unused `Switch` import in ProfileScreen | Low | ✅ Fixed |
+
+---
+
+## 1. Human Interface Guidelines (HIG)
+
+### 1.1 Navigation & Tab Bar ✅
+
+- The app uses a standard bottom tab bar with three tabs: Upload, Files, and Profile.
+- Tab icons use **Ionicons** (an icon set that closely maps to Apple's SF Symbols).
+- Active/inactive tab colors follow iOS conventions (`#1e40af` active, `#9ca3af` inactive).
+- Header styling uses a solid color background with white text, consistent with iOS navigation bar patterns.
+
+### 1.2 Icons & Visual Assets ✅
+
+- **App icon:** Custom `icon.png` provided at root level; Expo handles generating all required sizes.
+- **Splash screen:** Uses branded splash with `contain` resize mode and matching background color.
+- **Adaptive icon (Android):** Properly configured with foreground image and background color.
+- **Action buttons:** Previously used emoji characters (📷, 🖼️, 📄) which render inconsistently across iOS versions. **Fixed:** Now using Ionicons (`camera-outline`, `images-outline`, `document-outline`).
+- **Status indicators:** Previously used emoji (✅, ❌, ⏳, ⚙️). **Fixed:** Now using Ionicons with semantic colors.
+
+### 1.3 Typography & Colors ✅
+
+- Uses system fonts (default React Native text rendering uses San Francisco on iOS).
+- Color palette (`#1e40af` primary blue, semantic reds/greens/grays) provides sufficient contrast ratios.
+- Text sizes follow iOS recommended minimums (body text ≥ 13pt).
+
+### 1.4 Touch Targets ✅
+
+- All interactive elements have `minHeight: 44` or `minHeight: 48` (meets Apple's 44×44pt minimum).
+- Back links, cancel buttons, and retry buttons all meet minimum touch target requirements.
+
+### 1.5 Safe Areas ✅
+
+- The app uses `react-native-safe-area-context` (`SafeAreaProvider`) to respect device notches, Dynamic Island, and home indicator.
+
+### 1.6 Dark Mode ✅
+
+- `userInterfaceStyle: "automatic"` is set in `app.json`, enabling automatic dark mode support.
+
+---
+
+## 2. Privacy & Data Usage
+
+### 2.1 Permission Descriptions ✅
+
+All iOS permission strings (Info.plist keys) are present and provide clear, specific descriptions of why each permission is needed:
+
+| Permission | Key | Description |
+|-----------|-----|-------------|
+| Camera | `NSCameraUsageDescription` | "DocuElevate uses the camera to scan QR codes for login and to capture documents for upload." |
+| Photo Library (Read) | `NSPhotoLibraryUsageDescription` | "DocuElevate accesses your photo library to select documents for upload." |
+| Photo Library (Write) | `NSPhotoLibraryAddUsageDescription` | "DocuElevate saves scanned documents to your photo library." |
+
+**Assessment:** All descriptions clearly explain the purpose, which is a requirement for App Review approval.
+
+### 2.2 Push Notifications ✅
+
+- Push notification permission is requested at runtime (not at launch) when the user enters the authenticated area.
+- The app works gracefully without push notifications if permission is denied.
+- Device tokens are registered via a dedicated backend endpoint.
+
+### 2.3 Background Modes ✅ (Fixed)
+
+- **Previous state:** `UIBackgroundModes` included `["fetch", "remote-notification"]`.
+- **Issue:** The app does not implement background fetch (`application:performFetchWithCompletionHandler:`). Apple may reject apps that declare background modes they don't actively use (Guideline 2.5.4).
+- **Fix:** Removed `fetch` from `UIBackgroundModes`. Only `remote-notification` remains, which is required for push notification delivery.
+
+### 2.4 Privacy Manifest ✅ (Fixed)
+
+Starting in Spring 2024, Apple requires a privacy manifest (`PrivacyInfo.xcprivacy`) for apps using specific APIs. The following required reason APIs are used by the app's dependencies:
+
+| API Category | Reason Code | Justification |
+|-------------|-------------|---------------|
+| `NSPrivacyAccessedAPICategoryUserDefaults` | `CA92.1` | Used by `@react-native-async-storage/async-storage` for user preferences |
+| `NSPrivacyAccessedAPICategoryFileTimestamp` | `C617.1` | Used by `expo-file-system` to read file metadata |
+| `NSPrivacyAccessedAPICategoryDiskSpace` | `E174.1` | Used by Expo runtime for storage space checks |
+| `NSPrivacyAccessedAPICategorySystemBootTime` | `35F9.1` | Used by React Native's timing APIs |
+
+The privacy manifest is configured via `expo-build-properties` plugin in `app.json`, which ensures it is included in the generated Xcode project during EAS Build.
+
+### 2.5 Tracking & Analytics ✅
+
+- `NSPrivacyTracking: false` — the app does **not** track users.
+- `NSPrivacyCollectedDataTypes: []` — no data types are collected for tracking.
+- No analytics SDKs (Firebase Analytics, Amplitude, Mixpanel, etc.) are included.
+- No App Tracking Transparency (ATT) prompt is needed.
+
+### 2.6 Encryption Declaration ✅
+
+- `ITSAppUsesNonExemptEncryption: false` — the app uses only standard HTTPS/TLS for network communication, which is exempt from export compliance requirements.
+
+### 2.7 Data Storage Security ✅
+
+- API tokens are stored in the device keychain via `expo-secure-store` (uses iOS Keychain Services).
+- No sensitive data is stored in `AsyncStorage` or `UserDefaults`.
+- Server URL is stored in secure storage, not in plain text files.
+
+---
+
+## 3. App Store Review Guidelines Compliance
+
+### 3.1 Functionality (Guideline 2.x) ✅
+
+- **2.1 App Completeness:** The app provides a complete, functional experience. All advertised features (camera capture, file upload, document list, push notifications) work as described.
+- **2.3 Accurate Metadata:** App name ("DocuElevate"), description, and screenshots should accurately reflect the app's functionality.
+- **2.5.4 Background Modes:** Only `remote-notification` is declared, which is actively used. ✅ Fixed.
+
+### 3.2 Content & Intellectual Property (Guideline 3.x) ✅
+
+- No third-party trademarked content is used.
+- The app does not display user-generated content publicly (documents are private to each user).
+- No copyrighted content is bundled with the app.
+
+### 3.3 Business (Guideline 3.1.x) ✅
+
+- The app does not include in-app purchases, subscriptions, or payment processing.
+- No physical goods or services are sold through the app.
+- Authentication is handled via self-hosted or enterprise SSO — no Apple Sign-In requirement applies (Apple Sign-In is required only when third-party social login options like Google/Facebook are offered as the primary login method; enterprise SSO to a self-hosted server is exempt).
+
+### 3.4 Safety & Privacy (Guideline 5.x) ✅
+
+- **5.1.1 Data Collection and Storage:** The app collects only what is necessary for its functionality (server URL, auth token, push token).
+- **5.1.1(v) Account Deletion:** ✅ Fixed. Users can now initiate account deletion from the Profile screen, which opens the server's account deletion page in the browser.
+- **5.1.2 Data Use and Sharing:** No data is shared with third parties or used for advertising.
+
+### 3.5 Privacy Policy ✅ (Fixed)
+
+- **Requirement:** Apple requires all apps to have an accessible privacy policy.
+- **Fix:** Privacy Policy and Terms of Service links are now accessible from the Profile screen, opening the server's hosted policy pages.
+- **App Store Connect:** The privacy policy URL must also be provided in App Store Connect during submission.
+
+### 3.6 Login & Authentication ✅
+
+- Two login methods are available: SSO (browser-based OAuth) and QR code scanning.
+- Both methods provide clear error messages on failure.
+- The app correctly handles authentication cancellation.
+- Session restoration on app launch is implemented.
+- **Demo Account:** For App Review, a demo account may need to be provided in App Store Connect's review notes. Ensure the review team can access a test server.
+
+---
+
+## 4. Technical Compliance
+
+### 4.1 API Usage ✅
+
+- No private APIs are used (all functionality comes from Expo SDK and React Native public APIs).
+- No deprecated APIs are used that would trigger rejection.
+
+### 4.2 Network Security ✅
+
+- The app validates server URLs require `http://` or `https://` scheme.
+- All API calls use Bearer token authentication over HTTPS.
+- App Transport Security (ATS) is not explicitly disabled — default iOS ATS rules apply.
+
+### 4.3 Deep Linking ✅
+
+- Custom URL scheme `docuelevate://` is properly registered.
+- Deep link handling for QR login (`docuelevate://qr-login`) and file sharing is implemented correctly.
+- `WebBrowser.openAuthSessionAsync` is used for OAuth, which properly handles the authentication session lifecycle.
+
+### 4.4 Document Handling ✅
+
+- `CFBundleDocumentTypes` properly declares supported file types.
+- `LSSupportsOpeningDocumentsInPlace: false` ensures iOS copies shared files to the app's accessible Inbox directory, avoiding security-scoped URL issues.
+- The `+not-found.tsx` handler correctly intercepts iOS "Open In…" file paths.
+- `UploadScreen` uses `expo-file-system` to copy external files to cache before uploading for reliable file access.
+
+### 4.5 Crash Resistance ✅
+
+- All network calls are wrapped in try/catch blocks.
+- Error states are displayed to users with actionable recovery options (retry buttons).
+- Permission denials are handled gracefully with explanatory messages.
+
+---
+
+## 5. Onboarding & First-Run Experience
+
+### 5.1 Welcome Screen ✅
+
+- Clean, informative welcome screen with app branding and feature highlights.
+- Clear "Get Started" call-to-action leading to the login screen.
+- No misleading claims or functionality promises.
+
+### 5.2 Login Flow ✅
+
+- Server URL entry with input validation.
+- Two clear authentication options (SSO and QR code).
+- Error handling with user-friendly alert dialogs.
+- Back navigation available from all auth screens.
+
+### 5.3 First-Run Permissions ✅
+
+- Camera permission is requested at the point of use (when tapping Camera button), not at launch.
+- Photo library permission is requested at the point of use.
+- Push notification permission is requested after authentication, not before.
+- All permission requests include clear usage descriptions.
+
+---
+
+## 6. Remaining Recommendations
+
+### 6.1 App Store Connect Preparation
+
+Before submission, ensure the following are configured in App Store Connect:
+
+- [ ] **Privacy Policy URL** — must point to the server's `/privacy` endpoint
+- [ ] **App Store description** — accurate description of features
+- [ ] **Screenshots** — for iPhone and iPad (since `supportsTablet: true`)
+- [ ] **App category** — "Business" or "Productivity"
+- [ ] **Age rating** — complete the questionnaire (likely 4+)
+- [ ] **Review notes** — provide demo server URL and test credentials for the Apple review team
+- [ ] **Privacy Nutrition Labels** — declare data types collected (device ID for push notifications, authentication tokens)
+
+### 6.2 Accessibility Enhancements (Recommended)
+
+While the app includes `accessibilityRole` and `accessibilityLabel` on interactive elements, consider:
+
+- Adding `accessibilityHint` to buttons where the action isn't immediately obvious.
+- Testing with VoiceOver to ensure all screens are fully navigable.
+- Ensuring all status changes are announced to screen readers.
+
+### 6.3 iPad Support
+
+The app declares `supportsTablet: true`. Ensure:
+
+- UI scales appropriately on iPad screen sizes.
+- Split View and Slide Over multitasking work correctly.
+- Touch targets remain accessible on larger screens.
+
+### 6.4 Localization (Future Enhancement)
+
+- The app currently uses English-only strings.
+- For broader App Store reach, consider localizing the app name, description, and in-app strings.
+
+---
+
+## 7. Compliance Checklist Summary
+
+| Area | Status | Notes |
+|------|--------|-------|
+| Human Interface Guidelines | ✅ Pass | Ionicons used for platform-consistent iconography |
+| App Icons & Visual Assets | ✅ Pass | All required assets provided |
+| Device Data Usage | ✅ Pass | Camera, photos, notifications properly handled |
+| Privacy Disclosures | ✅ Pass | Info.plist keys and privacy manifest configured |
+| Background Modes | ✅ Pass | Only `remote-notification` declared |
+| Restricted APIs | ✅ Pass | No private or deprecated APIs used |
+| Content Standards | ✅ Pass | No misleading or inappropriate content |
+| Functionality | ✅ Pass | Complete, functional app experience |
+| Business Model | ✅ Pass | No IAP conflicts |
+| Safety & Privacy | ✅ Pass | Account deletion available, privacy policy linked |
+| Onboarding | ✅ Pass | Clear, permission-respectful first-run experience |
+| Privacy Manifest | ✅ Pass | Required reason APIs declared |
+
+---
+
+## References
+
+- [Apple App Store Review Guidelines](https://developer.apple.com/app-store/review/guidelines/)
+- [Apple Human Interface Guidelines](https://developer.apple.com/design/human-interface-guidelines/)
+- [Apple Privacy Manifest Requirements](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files)
+- [App Store Connect Help](https://developer.apple.com/help/app-store-connect/)

docs/AuthenticationSetup.md

@@ -15,6 +15,7 @@ This guide explains how to configure authentication for DocuElevate to secure yo
 | `AUTHENTIK_CLIENT_SECRET`  | Client secret for OpenID Connect authentication          |
 | `AUTHENTIK_CONFIG_URL`     | OpenID Connect discovery URL                             |
 | `OAUTH_PROVIDER_NAME`      | Display name for the OAuth provider button               |
+| `SSO_AUTO_LOGIN`           | Auto-redirect to SSO login (skips the login page)        |
 
 For a complete list of configuration options, see the [Configuration Guide](ConfigurationGuide.md).
 
@@ -24,7 +25,12 @@ DocuElevate supports multiple authentication methods that can be used independen
 
 1. **Simple Authentication** - Basic username/password authentication managed by DocuElevate
 2. **OpenID Connect** - Integration with identity providers like Authentik, Keycloak, or Auth0
-3. **Social Login** - Sign in with Google, Microsoft, Apple, or Dropbox accounts (see [Social Login Setup Guide](SocialLoginSetup.md))
+3. **Social Login** - Sign in with Google, Microsoft, Apple, Dropbox, or GitHub accounts (see [Social Login Setup Guide](SocialLoginSetup.md))
+4. **Keycloak SSO** - Self-hosted identity management via Keycloak
+5. **Generic OAuth2** - Any OAuth2-compatible identity provider
+6. **SAML2** - Enterprise SSO via SAML 2.0
+
+> **Admin Connections Page:** You can configure all authentication providers through the admin **Connections** page at `/admin/connections`.
 
 ## Session Security
 
@@ -154,6 +160,62 @@ DocuElevate can work with any OpenID Connect-compliant provider, not just Authen
    OAUTH_PROVIDER_NAME=Auth0
    ```
 
+## Server-Side Session Management
+
+DocuElevate supports server-side session tracking. Every login creates a `UserSession` record that can be listed and revoked individually or all at once ("log off everywhere").
+
+### Configuration
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `SESSION_LIFETIME_DAYS` | Number of days before a session expires | `30` |
+| `SESSION_LIFETIME_CUSTOM_DAYS` | Override for `SESSION_LIFETIME_DAYS` when set | — |
+
+### Managing Sessions
+
+Users can manage their active sessions from the **Profile → Security** section:
+
+- **View active sessions** — see browser, device, IP address, and last activity for each session.
+- **Revoke a single session** — immediately invalidate one session.
+- **Log off everywhere** — revoke all sessions (optionally keeping the current one) and all API tokens at once.
+
+Expired sessions are automatically cleaned up by a periodic background task.
+
+### API Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/sessions` | List the current user's active sessions |
+| `DELETE` | `/api/sessions/{id}` | Revoke a single session |
+| `POST` | `/api/sessions/revoke-all` | Revoke all sessions for the current user |
+
+## QR Code Login
+
+QR code login allows users to authenticate a mobile device by scanning a QR code displayed in the web UI, without manually entering credentials on the phone.
+
+### How It Works
+
+1. The authenticated web user opens the **QR Login** page and a challenge QR code is displayed.
+2. The user opens the DocuElevate mobile app and taps **Scan QR Code to Login**, which opens the device camera.
+3. The mobile app scans the QR code. The QR code contains both the challenge token and the server URL (`docuelevate://qr-login?token=...&server=...`), so there is no need to enter the server URL manually.
+4. An API token is issued for the mobile device and the web UI is notified via polling.
+
+> **Note:** The countdown timer on the web page uses server-relative time (TTL in seconds) rather than absolute timestamps, so it works correctly even when the client's clock is not in sync with the server.
+
+### Configuration
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `QR_LOGIN_CHALLENGE_TTL_SECONDS` | How long a QR challenge is valid (seconds) | `120` |
+
+### API Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `POST` | `/api/qr-auth/challenge` | Create a new QR login challenge (returns `ttl_seconds` for client countdown) |
+| `GET` | `/api/qr-auth/challenge/{id}/status` | Poll the status of a challenge |
+| `POST` | `/api/qr-auth/claim` | Claim a challenge from a mobile device |
+
 ## Security Considerations
 
 1. **Always use HTTPS** in production to protect authentication tokens and passwords

docs/ConfigurationGuide.md

@@ -11,12 +11,19 @@ Configuration is primarily done through environment variables specified in a `.e
 | **Variable**           | **Description**                                          | **Example**                    |
 |------------------------|----------------------------------------------------------|--------------------------------|
 | `DATABASE_URL`         | Path/URL to the SQLite database (or other SQL backend). Use the [Database Wizard](/database-wizard) for guided setup. See [Database Configuration](DatabaseConfiguration.md). | `sqlite:///./app/database.db`  |
+| `DB_POOL_SIZE`         | Number of persistent connections in the pool per worker (PostgreSQL/MySQL only; ignored for SQLite). | `10` |
+| `DB_MAX_OVERFLOW`      | Additional connections beyond `DB_POOL_SIZE` under burst load (PostgreSQL/MySQL only). | `20` |
+| `DB_POOL_TIMEOUT`      | Seconds to wait for a pool connection before raising `TimeoutError` (PostgreSQL/MySQL only). | `30` |
+| `DB_POOL_RECYCLE`      | Recycle connections after this many seconds to avoid stale connections (PostgreSQL/MySQL only). | `1800` |
 | `REDIS_URL`            | URL for Redis, used by Celery for broker & result store. | `redis://redis:6379/0`         |
 | `WORKDIR`              | Working directory for the application.                  | `/workdir`                     |
 | `GOTENBERG_URL`        | Gotenberg PDF processing URL.                           | `http://gotenberg:3000`        |
 | `EXTERNAL_HOSTNAME`    | The external hostname for the application.             | `docuelevate.example.com`      |
+| `PUBLIC_BASE_URL`      | Full public base URL including scheme (e.g., `https://docuelevate.example.com`). When set, overrides auto-detected URLs used for OAuth redirect URIs. **Required when your reverse proxy does not forward `X-Forwarded-Proto` headers.** | *(not set)* |
 | `ALLOW_FILE_DELETE`    | Enable file deletion in the web interface (`true`/`false`). | `true`                      |
 | `COMPLIANCE_ENABLED`   | Enable the compliance templates dashboard (GDPR, HIPAA, SOC 2). | `true`                      |
+| `FACTORY_RESET_ON_STARTUP` | Wipe all user data on every startup (demo/testing). | `false` |
+| `ENABLE_FACTORY_RESET` | Show the System Reset page in the admin UI.         | `false` |
 
 ### Batch Processing Settings
 
@@ -79,6 +86,28 @@ Control how the web UI queues and paces file uploads to avoid overwhelming the b
 
 **Example**: With `UPLOAD_CONCURRENCY=3` and `UPLOAD_QUEUE_DELAY_MS=500`, a directory of 5,000 files is uploaded ≈ 3 at a time with 500 ms pacing – the backend processes files at its own rate while the queue drains in the background without triggering API rate limits.
 
+### Per-User Upload Rate Limiting
+
+Server-side rate limiting that prevents any single user from overwhelming the system with bulk uploads. The limiter uses a Redis-backed sliding window and dynamically adjusts limits based on system health.
+
+| **Variable**                   | **Description**                                                                                                              | **Default** |
+|--------------------------------|------------------------------------------------------------------------------------------------------------------------------|-------------|
+| `UPLOAD_RATE_LIMIT_PER_USER`   | Maximum uploads allowed per user within the sliding window. Effective limit may be reduced under load.                       | `20`        |
+| `UPLOAD_RATE_LIMIT_WINDOW`     | Sliding window size in seconds.                                                                                              | `60`        |
+
+**Health-aware dynamic limiting**: The effective per-user limit is automatically reduced when the system is under heavy load:
+
+| **System condition**           | **Effective limit** | **Trigger**                    |
+|--------------------------------|---------------------|--------------------------------|
+| Normal                         | 100 % of base       | Queue < 50, CPU load normal    |
+| Moderate load                  | 50 % of base        | Queue 50–100 or CPU > 1.5×    |
+| High load                      | 25 % of base        | Queue 100–200 or CPU > 2×     |
+| Critical load                  | 10 % of base        | Queue > 200 or CPU > 3×       |
+
+When a user exceeds the limit, the server returns **HTTP 429 Too Many Requests** with a `Retry-After` header. The browser client (see *Client-Side Upload Throttling* above) automatically pauses and retries.
+
+> **Note**: The limiter fails open — if Redis is unavailable, all uploads are allowed through so that a monitoring outage never blocks document processing.
+
 ### File Upload Size Limits
 
 **Security Feature**: Control file upload sizes to prevent resource exhaustion attacks. See [SECURITY_AUDIT.md](../SECURITY_AUDIT.md#5-file-upload-size-limits) for security details.
@@ -364,6 +393,9 @@ Credentials are encrypted at rest using Fernet encryption.
 |-------------------------|---------------------------------------------------------------|
 | `AUTH_ENABLED`          | Enable or disable authentication (`true`/`false`).           |
 | `SESSION_SECRET`        | Secret key used to encrypt sessions and cookies (at least 32 chars). |
+| `SESSION_LIFETIME_DAYS` | Number of days before a server-side session expires. Default: `30`. |
+| `SESSION_LIFETIME_CUSTOM_DAYS` | Override for `SESSION_LIFETIME_DAYS` when set.        |
+| `QR_LOGIN_CHALLENGE_TTL_SECONDS` | How long a QR login challenge is valid (seconds). Default: `120`. |
 | `ADMIN_USERNAME`        | Username for basic authentication (when not using OIDC).     |
 | `ADMIN_PASSWORD`        | Password for basic authentication (when not using OIDC).     |
 | `ADMIN_GROUP_NAME`      | Group name in OIDC claims that grants admin access. Default: `admin`. |
@@ -374,7 +406,7 @@ Credentials are encrypted at rest using Fernet encryption.
 
 ### Social Login Providers
 
-Social login lets users sign in with their existing Google, Microsoft, Apple, or Dropbox accounts. Each provider is independently enabled and configured. For detailed setup instructions see the [Social Login Setup Guide](SocialLoginSetup.md).
+Social login lets users sign in with their existing Google, Microsoft, Apple, Dropbox, or GitHub accounts. Each provider is independently enabled and configured. For detailed setup instructions see the [Social Login Setup Guide](SocialLoginSetup.md).
 
 | **Variable** | **Description** | **Default** |
 |---|---|---|
@@ -393,6 +425,33 @@ Social login lets users sign in with their existing Google, Microsoft, Apple, or
 | `SOCIAL_AUTH_DROPBOX_ENABLED` | Enable Dropbox Sign-In. | `false` |
 | `SOCIAL_AUTH_DROPBOX_CLIENT_ID` | Dropbox OAuth2 App Key. | *(empty)* |
 | `SOCIAL_AUTH_DROPBOX_CLIENT_SECRET` | Dropbox OAuth2 App Secret. | *(empty)* |
+| `SOCIAL_AUTH_GITHUB_ENABLED` | Enable GitHub Sign-In. | `false` |
+| `SOCIAL_AUTH_GITHUB_CLIENT_ID` | GitHub OAuth2 client ID from GitHub Developer Settings. | *(empty)* |
+| `SOCIAL_AUTH_GITHUB_CLIENT_SECRET` | GitHub OAuth2 client secret. | *(empty)* |
+| `SSO_AUTO_LOGIN` | Automatically redirect to SSO login when authentication is required. | `false` |
+
+### SSO Providers
+
+| **Variable** | **Description** | **Default** |
+|---|---|---|
+| `SOCIAL_AUTH_KEYCLOAK_ENABLED` | Enable Keycloak SSO. | `false` |
+| `SOCIAL_AUTH_KEYCLOAK_CLIENT_ID` | Keycloak OAuth2 client ID. | *(empty)* |
+| `SOCIAL_AUTH_KEYCLOAK_CLIENT_SECRET` | Keycloak OAuth2 client secret. | *(empty)* |
+| `SOCIAL_AUTH_KEYCLOAK_SERVER_URL` | Keycloak server base URL (e.g. `https://keycloak.example.com`). | *(empty)* |
+| `SOCIAL_AUTH_KEYCLOAK_REALM` | Keycloak realm name. | *(empty)* |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_ENABLED` | Enable a generic OAuth2 SSO provider. | `false` |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_CLIENT_ID` | Generic OAuth2 client ID. | *(empty)* |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_CLIENT_SECRET` | Generic OAuth2 client secret. | *(empty)* |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_AUTHORIZE_URL` | Generic OAuth2 authorization URL. | *(empty)* |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_TOKEN_URL` | Generic OAuth2 token endpoint URL. | *(empty)* |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_USERINFO_URL` | Generic OAuth2 userinfo endpoint URL. | *(empty)* |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_SCOPE` | Space-separated list of OAuth2 scopes. | `openid profile email` |
+| `SOCIAL_AUTH_GENERIC_OAUTH2_NAME` | Display name for the provider button. | `OAuth2` |
+| `SOCIAL_AUTH_SAML2_ENABLED` | Enable SAML2 SSO authentication. | `false` |
+| `SOCIAL_AUTH_SAML2_ENTITY_ID` | SAML2 Identity Provider Entity ID. | *(empty)* |
+| `SOCIAL_AUTH_SAML2_SSO_URL` | SAML2 Identity Provider SSO URL. | *(empty)* |
+| `SOCIAL_AUTH_SAML2_CERTIFICATE` | SAML2 Identity Provider X.509 certificate (PEM format). | *(empty)* |
+| `SOCIAL_AUTH_SAML2_NAME` | Display name for the SAML2 provider. | `SAML2` |
 
 ### Multi-User Mode
 
@@ -457,6 +516,87 @@ default overage buffer applied across all plans.
 
 DocuElevate supports HTTP security headers to improve browser-side security. **These headers are disabled by default** since most deployments use a reverse proxy (Traefik, Nginx, etc.) that already adds them. Enable only if deploying directly without a reverse proxy. See [Deployment Guide - Security Headers](DeploymentGuide.md#security-headers) for detailed configuration examples.
 
+### Application Logging
+
+DocuElevate uses Python's standard `logging` module. Two environment variables control log verbosity:
+
+| **Variable** | **Description** | **Default** |
+|-------------|----------------|-------------|
+| `LOG_LEVEL` | Root logger level. Accepts standard Python level names: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`. | `INFO` |
+| `DEBUG` | Enable debug mode. When `true` **and** `LOG_LEVEL` is **not** explicitly set, the effective log level is automatically lowered to `DEBUG`. | `false` |
+
+**Precedence rules (standard behaviour):**
+
+1. If `LOG_LEVEL` is explicitly set, it always wins — regardless of `DEBUG`.
+2. If only `DEBUG=true` is set (no `LOG_LEVEL`), the effective level becomes `DEBUG`.
+3. If neither is set, the default level is `INFO`.
+
+```bash
+# Typical production (default)
+# LOG_LEVEL=INFO
+
+# Quick debug mode — sets level to DEBUG automatically
+DEBUG=true
+
+# Explicit level override (DEBUG flag is ignored for level selection)
+LOG_LEVEL=WARNING
+```
+
+> **Tip:** At `DEBUG` level, noisy third-party libraries (httpx, authlib, urllib3, etc.) are automatically pinned to `WARNING` so that application debug output remains readable.
+
+#### Structured JSON Logging
+
+Set `LOG_FORMAT=json` to emit structured JSON lines on stdout — one JSON object per log message. This is the standard format for log collectors and SIEM tools:
+
+| **Variable** | **Description** | **Default** |
+|-------------|----------------|-------------|
+| `LOG_FORMAT` | Log output format: `text` (human-readable) or `json` (structured JSON lines). | `text` |
+
+Each JSON log line contains: `timestamp` (ISO 8601), `level`, `logger`, `message`, `module`, `funcName`, `lineno`, and `exc_info` (when an exception is logged).
+
+```bash
+# Enable JSON logging for SIEM / log aggregation
+LOG_FORMAT=json
+```
+
+**Example JSON output:**
+```json
+{"timestamp": "2025-03-16T09:18:05.192000+00:00", "level": "INFO", "logger": "app.auth", "message": "[SECURITY] OAUTH_LOGIN_SUCCESS user=alice@example.com admin=False", "module": "auth", "funcName": "oauth_callback", "lineno": 654}
+```
+
+**Compatible with:**
+- **Grafana Loki** — Promtail scrapes JSON from Docker stdout
+- **Splunk** — Universal Forwarder or HEC with JSON sourcetype
+- **ELK / OpenSearch** — Filebeat with JSON codec
+- **Datadog** — Agent auto-parses JSON logs
+- **Fluentd / Vector** — JSON input plugin
+- **Docker log drivers** — `--log-driver=json-file` (default) preserves structure
+
+#### Syslog Forwarding (Application Logs)
+
+For traditional (non-container) deployments, application logs can be forwarded directly to a syslog receiver. This is **separate** from audit-log SIEM forwarding (see below) — it sends _every_ Python log message, not just audit events.
+
+| **Variable** | **Description** | **Default** |
+|-------------|----------------|-------------|
+| `LOG_SYSLOG_ENABLED` | Forward application logs to a syslog receiver in addition to stdout. | `false` |
+| `LOG_SYSLOG_HOST` | Hostname or IP of the syslog receiver. | `localhost` |
+| `LOG_SYSLOG_PORT` | Port of the syslog receiver. | `514` |
+| `LOG_SYSLOG_PROTOCOL` | Protocol: `udp` or `tcp`. | `udp` |
+
+```bash
+# Forward all application logs to syslog
+LOG_SYSLOG_ENABLED=true
+LOG_SYSLOG_HOST=syslog.internal.example.com
+LOG_SYSLOG_PORT=514
+LOG_SYSLOG_PROTOCOL=udp
+
+# Combine with JSON format for structured syslog messages
+LOG_FORMAT=json
+LOG_SYSLOG_ENABLED=true
+```
+
+> **Note:** When `LOG_FORMAT=json`, syslog messages are also sent as JSON. When `LOG_FORMAT=text`, syslog messages use the standard `name - level - message` format.
+
 ### Audit Logging
 
 DocuElevate provides comprehensive audit logging that records significant actions (logins, document CRUD, settings changes) to an append-only database table. Every entry captures the timestamp, user, action, resource, client IP, and optional JSON details.
@@ -652,7 +792,7 @@ SECURITY_HEADER_CSP_VALUE="default-src 'self'; script-src 'self'; style-src 'sel
 SECURITY_HEADER_CSP_VALUE="default-src 'self'; script-src 'self' https://cdn.example.com; style-src 'self' 'unsafe-inline';"
 ```
 
-**Note:** The default policy includes `'unsafe-inline'` for compatibility with Tailwind CSS and inline JavaScript. For stricter security, use nonces or hashes.
+**Note:** The default policy includes `'unsafe-inline'` for compatibility with inline JavaScript. Tailwind CSS v3 is compiled at build time into a static file served from `'self'`, so no external style CDN is needed.
 
 #### X-Frame-Options
 
@@ -853,6 +993,48 @@ OPENAI_API_KEY=sk-ant-...   # passed as the api_key to LiteLLM
 
 ---
 
+### Document Translation
+
+After processing, DocuElevate can automatically translate a document's extracted text into a configurable *default language* (e.g. English). This reference translation is stored alongside the original text so users always have a version in a language they understand.
+
+Other languages are translated **on the fly** via the AI provider and are not persisted.
+
+#### Settings
+
+| **Variable**                 | **Description**                                                                                           | **Default** |
+|------------------------------|-----------------------------------------------------------------------------------------------------------|-------------|
+| `DEFAULT_DOCUMENT_LANGUAGE`  | ISO 639-1 code for the default translation target (e.g. `en`, `de`, `fr`). Documents whose detected language differs are automatically translated into this language after processing. | `en`        |
+
+Each user can override this global default in their profile (`UserProfile.default_document_language`).
+
+#### How It Works
+
+1. During metadata extraction the AI detects the document language (stored as `detected_language` on the file record).
+2. If the detected language differs from the default target language, a background Celery task (`translate_to_default_language`) translates the extracted text.
+3. The translated text is persisted in `default_language_text` and the target code in `default_language_code`.
+4. The file detail view shows both the original text and the default-language version.
+5. Users can also request on-the-fly translations to any language via the **Translate** dropdown.
+
+#### API Endpoints
+
+| **Endpoint**                                  | **Method** | **Description**                                                        |
+|-----------------------------------------------|------------|------------------------------------------------------------------------|
+| `/api/files/{id}/translation/default`         | GET        | Returns the persisted default-language translation (404 if unavailable)|
+| `/api/files/{id}/translate?lang=xx`           | GET        | On-the-fly translation to any ISO 639-1 language code                  |
+| `/files/{id}/text/default-language`           | GET        | View endpoint returning the default-language text as JSON              |
+
+#### Example
+
+```bash
+# Get the stored English translation of a German document
+curl http://localhost:8000/api/files/42/translation/default
+
+# Translate on the fly to French
+curl "http://localhost:8000/api/files/42/translate?lang=fr"
+```
+
+---
+
 ### OCR Providers
 
 DocuElevate supports multiple OCR engines that can be used individually or in combination. Configure the list of active providers with `OCR_PROVIDERS` and tune each provider with the settings below.
@@ -1161,6 +1343,20 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
 
 For detailed setup instructions, see the [OneDrive Setup Guide](OneDriveSetup.md).
 
+### SharePoint Online
+
+| **Variable**                    | **Description**                                       |
+|---------------------------------|-------------------------------------------------------|
+| `SHAREPOINT_CLIENT_ID`          | Azure AD application client ID                        |
+| `SHAREPOINT_CLIENT_SECRET`      | Azure AD application client secret                    |
+| `SHAREPOINT_TENANT_ID`          | Azure AD tenant ID (use "common" for multi-tenant apps) |
+| `SHAREPOINT_REFRESH_TOKEN`      | OAuth 2.0 refresh token                               |
+| `SHAREPOINT_SITE_URL`           | SharePoint site URL (e.g. `https://tenant.sharepoint.com/sites/sitename`) |
+| `SHAREPOINT_DOCUMENT_LIBRARY`   | Document library name (default: `Documents`)          |
+| `SHAREPOINT_FOLDER_PATH`        | Subfolder path inside the document library            |
+
+SharePoint uses the same Microsoft Graph API as OneDrive. See the [OneDrive Setup Guide](OneDriveSetup.md) for Azure AD app registration instructions — the same app registration can be reused for SharePoint with the `Sites.ReadWrite.All` permission.
+
 ### Amazon S3
 
 | **Variable**                    | **Description**                                       |
@@ -1206,6 +1402,9 @@ For detailed setup instructions, see the [Amazon S3 Setup Guide](AmazonS3Setup.m
 | `NOTIFY_ON_USER_SIGNUP`    | Send admin notification when a new user signs up (`True`/`False`, default `True`) |
 | `NOTIFY_ON_PLAN_CHANGE`    | Send admin notification when a user changes their subscription plan (`True`/`False`, default `True`) |
 | `NOTIFY_ON_PAYMENT_ISSUE`  | Send admin notification when a payment issue is reported for a user (`True`/`False`, default `True`) |
+| `TELEGRAM_ENABLED`         | Enable Telegram bot notifications. | `false` |
+| `TELEGRAM_BOT_TOKEN`       | Telegram Bot API token from @BotFather. | *(empty)* |
+| `TELEGRAM_CHAT_ID`         | Telegram chat ID to send notifications to. | *(empty)* |
 
 #### User-Event Notifications
 
@@ -1285,6 +1484,26 @@ Configurations are stored in the database and managed through the API (see [API
 
 Webhook URLs, secrets, and subscribed events are configured per-webhook via the `/api/webhooks/` endpoints (admin access required). Each delivery includes an optional HMAC-SHA256 signature for verification and is retried with exponential backoff on failure.
 
+### Automation Hooks (Zapier / Make.com)
+
+Automation hooks enable integration with external automation platforms such as
+[Zapier](https://zapier.com) and [Make.com](https://make.com) (formerly Integromat).
+
+| **Variable**               | **Description**                                                                                | **Default** |
+|----------------------------|------------------------------------------------------------------------------------------------|-------------|
+| `AUTOMATION_HOOKS_ENABLED` | Enable or disable Zapier / Make.com automation hook subscriptions and delivery (`True`/`False`) | `True`      |
+
+When enabled, external platforms can:
+
+- **Subscribe** to DocuElevate events via `POST /api/automation/hooks/subscribe` (outgoing triggers)
+- **Send documents** to DocuElevate via `POST /api/automation/actions/upload` (incoming actions)
+- **Discover fields** via `GET /api/automation/triggers/sample/{event}` (Zapier field mapping)
+
+Automation hooks share the same event types as webhooks (`document.uploaded`, `document.processed`,
+`document.failed`, `user.signup`, `user.plan_changed`, `user.payment_issue`) and use a flat
+Zapier-compatible JSON payload format.  See the [API docs](API.md#automation-zapier--makecom) for
+endpoint details and payload examples.
+
 ### Backup & Restore
 
 DocuElevate automatically backs up the database on a scheduled basis.
@@ -1386,6 +1605,8 @@ No additional configuration is required — the auto-fill uses the authenticated
 
 DocuElevate integrates with [Sentry](https://sentry.io) for real-time error tracking and performance monitoring.  See [SentrySetup.md](./SentrySetup.md) for a full setup guide.
 
+### Server-side (Python SDK)
+
 | Variable | Description | Default |
 |---|---|---|
 | `SENTRY_DSN` | Sentry DSN URL.  When set, error reporting and performance tracing are enabled automatically.  Leave blank to disable. | *(unset)* |
@@ -1394,18 +1615,33 @@ DocuElevate integrates with [Sentry](https://sentry.io) for real-time error trac
 | `SENTRY_PROFILES_SAMPLE_RATE` | Fraction of profiled transactions sent to Sentry (0.0 – 1.0).  Only active when traces > 0. | `0.0` |
 | `SENTRY_SEND_DEFAULT_PII` | Attach PII (IP addresses, user agents) to Sentry events.  Disabled by default for GDPR/CCPA compliance. | `false` |
 
+### Browser SDK (JavaScript)
+
+The Sentry Browser SDK is loaded automatically on every rendered page when `SENTRY_DSN` is set.  The same DSN is used for both server and browser — the DSN is a *public* key in Sentry's security model and is intentionally embedded in client-side code.
+
+| Variable | Description | Default |
+|---|---|---|
+| `SENTRY_JS_TRACES_SAMPLE_RATE` | Fraction of browser page-loads captured for client-side performance tracing (0.0 – 1.0). | `0.0` |
+| `SENTRY_JS_REPLAY_SESSION_SAMPLE_RATE` | Fraction of sessions recorded by [Sentry Session Replay](https://docs.sentry.io/product/session-replay/) (0.0 – 1.0). | `0.0` |
+| `SENTRY_JS_REPLAY_ON_ERROR_SAMPLE_RATE` | Fraction of error sessions captured with session replay context (0.0 – 1.0). | `0.1` |
+
 ```bash
-# Minimal example
+# Minimal example (server + browser)
 SENTRY_DSN=https://<key>@o<org>.ingest.sentry.io/<project>
 SENTRY_ENVIRONMENT=production
 
-# Optional tuning
+# Optional server-side tuning
 SENTRY_TRACES_SAMPLE_RATE=0.1
 SENTRY_PROFILES_SAMPLE_RATE=0.0
 SENTRY_SEND_DEFAULT_PII=false
+
+# Optional browser-side tuning
+SENTRY_JS_TRACES_SAMPLE_RATE=0.1
+SENTRY_JS_REPLAY_SESSION_SAMPLE_RATE=0.0
+SENTRY_JS_REPLAY_ON_ERROR_SAMPLE_RATE=0.1
 ```
 
-> **Note:** Sentry is completely opt-in — if `SENTRY_DSN` is not set, the SDK is never initialised and no data leaves your infrastructure.
+> **Note:** Sentry is completely opt-in — if `SENTRY_DSN` is not set, neither SDK is initialised and no data leaves your infrastructure.
 
 ## Duplicate Document Detection
 
@@ -1413,14 +1649,30 @@ DocuElevate detects and flags documents that share the same content, even if the
 
 ### Exact Duplicate Detection (SHA-256)
 
-When `ENABLE_DEDUPLICATION=True` (the default), each new document is hashed with SHA-256 before processing begins. If the hash matches an existing file record the new document is stored as a duplicate (`is_duplicate=True`, `duplicate_of_id=<original_id>`) and no further processing is performed.
+When `ENABLE_DEDUPLICATION=True` (the default), each new document is hashed with SHA-256 before processing begins. If the hash matches an existing file record the upload is rejected immediately — no processing task is created, and the temporary file is removed from disk. The `/api/ui-upload` response returns `"status": "duplicate"` together with a `duplicate_of` object that identifies the original file.
+
+If the same file somehow reaches the Celery worker (e.g. via a watch-folder ingest) it is still caught there and stored as a duplicate (`is_duplicate=True`, `duplicate_of_id=<original_id>`) with no further processing.
 
 | Variable | Description | Default |
 |---|---|---|
 | `ENABLE_DEDUPLICATION` | Hash-based exact duplicate detection on ingest. | `True` |
 | `SHOW_DEDUPLICATION_STEP` | Show the "Check for Duplicates" step in the processing timeline UI. | `True` |
 
-An immediate duplicate warning is also included in the `/api/ui-upload` JSON response so the frontend can alert the user before the pipeline completes.
+When the upload is an exact duplicate the `/api/ui-upload` response looks like:
+
+```json
+{
+  "status": "duplicate",
+  "original_filename": "invoice.pdf",
+  "stored_filename": "abc-123.pdf",
+  "duplicate_of": {
+    "duplicate_type": "exact",
+    "original_file_id": 42,
+    "original_filename": "invoice.pdf",
+    "message": "This file is an exact duplicate of an already-processed document. It has not been queued for processing again."
+  }
+}
+```
 
 ### Near-Duplicate Detection (Content Similarity)
 
@@ -1500,6 +1752,7 @@ For example:
 | S3           | `docs/uploads/`             | `docs/uploads/pdfa/`             |
 | Nextcloud    | `/Files`                    | `/Files/pdfa`                    |
 | OneDrive     | `Documents/Uploads`         | `Documents/Uploads/pdfa`         |
+| SharePoint   | `Uploads`                   | `Uploads/pdfa`                   |
 | Google Drive | *(folder ID)*               | `GOOGLE_DRIVE_PDFA_FOLDER_ID`    |
 
 Set `PDFA_UPLOAD_FOLDER` to an empty string to upload PDF/A files into the
@@ -1706,6 +1959,15 @@ ONEDRIVE_TENANT_ID=common
 ONEDRIVE_REFRESH_TOKEN=your_refresh_token
 ONEDRIVE_FOLDER_PATH=Documents/Uploads
 
+# SharePoint Online
+SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
+SHAREPOINT_CLIENT_SECRET=your_client_secret
+SHAREPOINT_TENANT_ID=your-tenant-id
+SHAREPOINT_REFRESH_TOKEN=your_refresh_token
+SHAREPOINT_SITE_URL=https://tenant.sharepoint.com/sites/sitename
+SHAREPOINT_DOCUMENT_LIBRARY=Documents
+SHAREPOINT_FOLDER_PATH=Uploads
+
 # Amazon S3
 AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
 AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
@@ -1733,6 +1995,52 @@ BACKUP_RETAIN_WEEKLY=13
 
 You can choose which document storage services to use by only including the relevant environment variables. For example, if you only want to use Dropbox, include only the Dropbox variables and omit the Paperless NGX and Nextcloud variables.
 
+## System Reset / Factory Reset
+
+DocuElevate provides two mechanisms for resetting the system to a clean state.  Both are **disabled by default** and must be explicitly enabled.
+
+### Automatic Reset on Startup
+
+Set `FACTORY_RESET_ON_STARTUP=true` to wipe all user data (database rows and work-files) every time the application starts.  This is useful for demo, testing, or ephemeral environments where you always want a fresh instance.
+
+```dotenv
+FACTORY_RESET_ON_STARTUP=true
+```
+
+> **Warning:** This destroys all documents, processing history, audit logs, and backups on every restart.  Application settings and configuration are preserved.
+
+### Admin UI Reset Page
+
+Set `ENABLE_FACTORY_RESET=true` to display the **System Reset** page in the admin navigation menu.  From this page, administrators can:
+
+| Action | Confirmation | Description |
+|--------|-------------|-------------|
+| **Full Reset** | Type `DELETE` | Wipes all database rows and work-files.  The system returns to its initial state. |
+| **Reset & Re-import** | Type `REIMPORT` | Copies original files to a `reimport/` folder inside the workdir, wipes everything, then configures the reimport folder as a watch folder so files are automatically re-ingested with the same processing pipeline, rate limits, and backoff strategy as regular uploads. |
+
+```dotenv
+ENABLE_FACTORY_RESET=true
+```
+
+### API Endpoints
+
+When `ENABLE_FACTORY_RESET=true`, two admin-only API endpoints are available:
+
+- `POST /api/admin/system-reset/full` — body: `{"confirmation": "DELETE"}`
+- `POST /api/admin/system-reset/reimport` — body: `{"confirmation": "REIMPORT"}`
+- `GET  /api/admin/system-reset/status` — returns current feature-flag state
+
+### What Gets Deleted
+
+| Deleted | Preserved |
+|---------|-----------|
+| All document records (`files` table) | Application settings (`application_settings` table) |
+| Processing logs and steps | User accounts and profiles |
+| Audit logs | Subscription plans |
+| Backup records | Pipelines and scheduled jobs |
+| Original, processed, and temporary files | The workdir directory itself |
+| Watch-folder caches and ingestion state | OAuth and integration configuration |
+
 ## Configuration File Location
 
 The `.env` file should be placed at the root of the project directory. When using Docker Compose, you can reference it with the `env_file` directive in your `docker-compose.yml`.

docs/CredentialRotationGuide.md

@@ -11,7 +11,7 @@ Credentials fall into two categories:
 | Category | Examples |
 |---|---|
 | **API keys** | OpenAI API key, Azure AI key, Paperless-ngx API token, AWS access keys |
-| **OAuth tokens / secrets** | Dropbox, Google Drive, OneDrive, Authentik client secrets and refresh tokens |
+| **OAuth tokens / secrets** | Dropbox, Google Drive, OneDrive, SharePoint, Authentik client secrets and refresh tokens |
 | **Passwords** | Admin password, Nextcloud, Email (SMTP), IMAP, FTP, SFTP, WebDAV |
 | **Private keys** | SFTP private key and passphrase |
 
@@ -119,6 +119,15 @@ For service-account credentials (`google_drive_credentials_json`):
 4. Re-authorize via the OAuth flow to get a fresh `onedrive_refresh_token`.
 5. Delete the old client secret in Azure.
 
+### SharePoint (Microsoft OAuth)
+
+1. SharePoint uses the same Azure AD app registration as OneDrive.
+2. In **Azure App Registrations**, navigate to **Certificates & secrets** for your app.
+3. Add a new client secret.
+4. Update `sharepoint_client_secret` in DocuElevate.
+5. Re-authorize via the OAuth flow to get a fresh `sharepoint_refresh_token`.
+6. Delete the old client secret in Azure.
+
 ### Authentik (OIDC)
 
 1. In your Authentik admin panel, navigate to the DocuElevate application and regenerate the client secret.