🧪 Extract and complete tests for saved searches API

Extracted existing `TestSavedSearchesCRUD` from `tests/test_api_advanced_filters.py` into a dedicated `tests/test_api_saved_searches.py` file to better organize testing logic and reflect the application's file structure.

Significantly improved code coverage of `app/api/saved_searches.py` from 0% (missing configuration imports during tests) to 100% by testing previously untested edge cases including:
- Reaching the maximum saved search limit per user.
- Database commit errors (`HTTP_500_INTERNAL_SERVER_ERROR`) during create, update, and delete actions.
- Validation failures for `filters` field checking for non-dict types (`status.HTTP_422_UNPROCESSABLE_ENTITY`).
- Conflicting names during updates where an existing saved search matches the new name.
- Proper fallback logic across authentication methods for `_get_user_id`.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

.env.demo

@@ -5,6 +5,7 @@ REDIS_URL=redis://redis:6379/0
 EXTERNAL_HOSTNAME=docuelevate.example.com
 GOTENBERG_URL=http://gotenberg:3000
 ALLOW_FILE_DELETE=true  # Allow deletion of file records
+COMPLIANCE_ENABLED=true  # Enable compliance templates dashboard (GDPR, HIPAA, SOC 2)
 
 # **UI / Appearance**
 # Default colour scheme: system (follow OS), light, or dark
@@ -96,6 +97,22 @@ MAX_UPLOAD_SIZE=1073741824
 # Allowed request headers (use * to allow all)
 # CORS_ALLOWED_HEADERS=*
 
+# **Audit Logging & SIEM Integration** (see docs/ConfigurationGuide.md#audit-logging)
+# Enable HTTP request audit logging middleware
+AUDIT_LOGGING_ENABLED=true
+# Include client IP in audit log entries (disable for GDPR-sensitive deployments)
+AUDIT_LOG_INCLUDE_CLIENT_IP=true
+
+# Forward audit events to an external SIEM system (Syslog, Splunk, Logstash, Grafana, etc.)
+# AUDIT_SIEM_ENABLED=false
+# AUDIT_SIEM_TRANSPORT=syslog          # syslog | http
+# AUDIT_SIEM_SYSLOG_HOST=localhost
+# AUDIT_SIEM_SYSLOG_PORT=514
+# AUDIT_SIEM_SYSLOG_PROTOCOL=udp       # udp | tcp
+# AUDIT_SIEM_HTTP_URL=                  # e.g. https://splunk:8088/services/collector/event
+# AUDIT_SIEM_HTTP_TOKEN=                # Bearer / HEC token
+# AUDIT_SIEM_HTTP_CUSTOM_HEADERS=       # Comma-separated Key:Value pairs
+
 # **Rate Limiting** (see SECURITY_AUDIT.md and docs/API.md)
 # Protects against DoS attacks and API abuse by limiting request rates per IP/user
 # Enabled by default - highly recommended for production
@@ -158,6 +175,33 @@ AUTHENTIK_CLIENT_SECRET=<yourAuthentikClientSecret>
 AUTHENTIK_CONFIG_URL=<ConfigUrlOfYourApp, e.g. https://authentik.example.com/application/o/docuelevate/.well-known/openid-configuration>
 OAUTH_PROVIDER_NAME="Authentik SSO"
 
+# **Social Login Providers**
+# Enable one or more social login providers to let users sign in with existing accounts.
+# Each provider requires separate OAuth credentials. See docs/SocialLoginSetup.md for details.
+
+# Google Sign-In (https://console.cloud.google.com/apis/credentials)
+# SOCIAL_AUTH_GOOGLE_ENABLED=false
+# SOCIAL_AUTH_GOOGLE_CLIENT_ID=your-google-client-id.apps.googleusercontent.com
+# SOCIAL_AUTH_GOOGLE_CLIENT_SECRET=your-google-client-secret
+
+# Microsoft Sign-In / Azure AD (https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps)
+# SOCIAL_AUTH_MICROSOFT_ENABLED=false
+# SOCIAL_AUTH_MICROSOFT_CLIENT_ID=your-microsoft-application-id
+# SOCIAL_AUTH_MICROSOFT_CLIENT_SECRET=your-microsoft-client-secret
+# SOCIAL_AUTH_MICROSOFT_TENANT=common  # common | organizations | consumers | <tenant-id>
+
+# Apple Sign-In (https://developer.apple.com/account/resources)
+# SOCIAL_AUTH_APPLE_ENABLED=false
+# SOCIAL_AUTH_APPLE_CLIENT_ID=com.example.docuelevate
+# SOCIAL_AUTH_APPLE_TEAM_ID=ABCDE12345
+# SOCIAL_AUTH_APPLE_KEY_ID=FGHIJ67890
+# SOCIAL_AUTH_APPLE_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----"
+
+# Dropbox Sign-In (https://www.dropbox.com/developers/apps)
+# SOCIAL_AUTH_DROPBOX_ENABLED=false
+# SOCIAL_AUTH_DROPBOX_CLIENT_ID=your-dropbox-app-key
+# SOCIAL_AUTH_DROPBOX_CLIENT_SECRET=your-dropbox-app-secret
+
 # **AI/ML Services**
 # Select your AI provider: openai | azure | anthropic | gemini | ollama | openrouter | portkey | litellm
 AI_PROVIDER=openai
@@ -210,6 +254,7 @@ EMAIL_SENDER=DocuElevate System <docuelevate@example.com>
 # These settings are intentionally separate from the shared EMAIL_* settings above.
 # Configuring EMAIL_HOST for password reset / notifications does NOT automatically
 # enable the email destination – you must set DEST_EMAIL_HOST to activate it.
+# DEST_EMAIL_ENABLED=true  # Set to false to disable email delivery without removing credentials
 DEST_EMAIL_HOST=smtp.example.com
 DEST_EMAIL_PORT=587
 DEST_EMAIL_USERNAME=docuelevate@example.com
@@ -294,8 +339,15 @@ IMAP2_DELETE_AFTER_PROCESS=false
 # Use for pre-production instances that share a mailbox with production.
 IMAP_READONLY_MODE=false
 
+# Controls which attachment types are ingested from IMAP emails.
+# 'documents_only' (default) – PDFs and office files only; images are skipped.
+# 'all' – all supported file types including images.
+# Per-user IMAP accounts can override this global default.
+IMAP_ATTACHMENT_FILTER=documents_only
+
 # **Storage/Document Services**
 # Amazon S3
+# S3_ENABLED=true  # Set to false to disable S3 uploads without removing credentials
 AWS_REGION=us-east-1
 AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
 AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
@@ -305,12 +357,14 @@ S3_STORAGE_CLASS=STANDARD
 S3_ACL=private
 
 # NextCloud
+# NEXTCLOUD_ENABLED=true  # Set to false to disable NextCloud uploads without removing credentials
 NEXTCLOUD_UPLOAD_URL=https://nextcloud.example.com/remote.php/dav/files/<USERNAME>
 NEXTCLOUD_FOLDER="<NEXTCLOUD_FOLDER_PATH>"
 NEXTCLOUD_USERNAME=<NEXTCLOUD_USERNAME>
 NEXTCLOUD_PASSWORD=<NEXTCLOUD_PASSWORD>
 
 # Paperless-ngx
+# PAPERLESS_ENABLED=true  # Set to false to disable Paperless uploads without removing credentials
 PAPERLESS_HOST=https://paperless.example.com
 PAPERLESS_NGX_API_TOKEN=<PAPERLESS_API_TOKEN>
 # Optional: Name of the custom field in Paperless-ngx to store the "absender" (sender) value
@@ -327,12 +381,14 @@ PAPERLESS_NGX_API_TOKEN=<PAPERLESS_API_TOKEN>
 # PAPERLESS_CUSTOM_FIELDS_MAPPING=
 
 # Dropbox
+# DROPBOX_ENABLED=true  # Set to false to disable Dropbox uploads without removing credentials
 DROPBOX_APP_KEY=<DROPBOX_APP_KEY>
 DROPBOX_APP_SECRET=<DROPBOX_APP_SECRET>
 DROPBOX_REFRESH_TOKEN=<DROPBOX_REFRESH_TOKEN>
 DROPBOX_FOLDER="/Documents/Uploads"
 
 # Google Drive
+# GOOGLE_DRIVE_ENABLED=true  # Set to false to disable Google Drive uploads without removing credentials
 # Service Account Method:
 GOOGLE_DRIVE_CREDENTIALS_JSON={"type":"service_account","project_id":"your-project","private_key_id":"key-id","private_key":"-----BEGIN PRIVATE KEY-----\nYOUR_PRIVATE_KEY\n-----END PRIVATE KEY-----\n","client_email":"service-account@project.iam.gserviceaccount.com","client_id":"client-id","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://oauth2.googleapis.com/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_x509_cert_url":"https://www.googleapis.com/robot/v1/metadata/x509/service-account%40project.iam.gserviceaccount.com"}
 GOOGLE_DRIVE_FOLDER_ID=<YOUR_FOLDER_ID>
@@ -345,6 +401,7 @@ GOOGLE_DRIVE_CLIENT_SECRET=your-oauth-client-secret  # Required for OAuth method
 GOOGLE_DRIVE_REFRESH_TOKEN=your-oauth-refresh-token  # Required for OAuth method
 
 # OneDrive
+# ONEDRIVE_ENABLED=true  # Set to false to disable OneDrive uploads without removing credentials
 ONEDRIVE_CLIENT_ID=your-client-id
 ONEDRIVE_CLIENT_SECRET=your-client-secret
 ONEDRIVE_TENANT_ID=common
@@ -352,6 +409,7 @@ ONEDRIVE_REFRESH_TOKEN=your-refresh-token
 ONEDRIVE_FOLDER_PATH=Documents/Uploads
 
 # WebDAV
+# WEBDAV_ENABLED=true  # Set to false to disable WebDAV uploads without removing credentials
 WEBDAV_URL=https://webdav.example.com/path
 WEBDAV_USERNAME=webdav_user
 WEBDAV_PASSWORD=your_secure_webdav_password
@@ -359,6 +417,7 @@ WEBDAV_FOLDER=/Documents/Uploads
 WEBDAV_VERIFY_SSL=True
 
 # FTP
+# FTP_ENABLED=true  # Set to false to disable FTP uploads without removing credentials
 # Security Note: FTP_USE_TLS=True is strongly recommended for secure connections
 # Set FTP_ALLOW_PLAINTEXT=False in production to prevent unencrypted FTP
 FTP_HOST=ftp.example.com
@@ -370,6 +429,7 @@ FTP_USE_TLS=True
 FTP_ALLOW_PLAINTEXT=True
 
 # SFTP
+# SFTP_ENABLED=true  # Set to false to disable SFTP uploads without removing credentials
 # Security Note: Host key verification is enabled by default (False)
 # Only set to True in development/testing environments if needed
 # When false, configure SSH known_hosts for proper host key verification
@@ -382,6 +442,16 @@ SFTP_PASSWORD=your_secure_sftp_password
 SFTP_FOLDER=/Documents/Uploads
 SFTP_DISABLE_HOST_KEY_VERIFICATION=False  # Default is False (secure); set to True only for testing
 
+# iCloud Drive
+# ICLOUD_ENABLED=true  # Set to false to disable iCloud uploads without removing credentials
+# Requires an Apple ID with iCloud Drive enabled.
+# For accounts with two-factor authentication (most accounts), generate an
+# app-specific password at https://appleid.apple.com/account/manage
+ICLOUD_USERNAME=your_apple_id@example.com
+ICLOUD_PASSWORD=your-app-specific-password
+ICLOUD_FOLDER=Documents/Uploads
+# ICLOUD_COOKIE_DIRECTORY=/path/to/cookie/dir  # Optional: defaults to ~/.pyicloud
+
 # **HTTP Request Settings**
 # Timeout for HTTP requests - set higher to handle large PDF files (up to 1GB)
 HTTP_REQUEST_TIMEOUT=120  # Timeout in seconds (default: 120 for large file operations)
@@ -478,3 +548,44 @@ EMBEDDING_MODEL=text-embedding-3-small
 # Maximum tokens to send to the embedding model.  Set below the model's
 # context window (e.g. 8000 for an 8192-token model).
 EMBEDDING_MAX_TOKENS=8000
+
+# **Support / Help Center – Zammad Integration**
+# Base URL of your Zammad instance (required for chat and ticket form).
+# ZAMMAD_URL=https://zammad.example.com
+# Show a live-chat widget on the Help Center page (requires an online Zammad agent).
+# ZAMMAD_CHAT_ENABLED=false
+# Zammad chat topic ID (see Zammad → Channels → Chat → Topics).
+# ZAMMAD_CHAT_ID=1
+# Show a "Submit a Ticket" feedback form on the Help Center page.
+# ZAMMAD_FORM_ENABLED=false
+# Support e-mail address displayed on the Help Center page.
+# SUPPORT_EMAIL=support@example.com
+
+# **Observability – Sentry Error & Performance Monitoring**
+# Sentry DSN – obtain from https://sentry.io (Project → Settings → Client Keys).
+# Leave commented out (or set to empty) to disable Sentry entirely.
+# SENTRY_DSN=https://<key>@o<org>.ingest.sentry.io/<project>
+#
+# Environment label shown in the Sentry dashboard (e.g. development / staging / production).
+# SENTRY_ENVIRONMENT=production
+#
+# Fraction of requests to capture for performance tracing (0.0–1.0).
+# 0.0 disables tracing; 1.0 captures every request. Default: 0.1 (10 %).
+# SENTRY_TRACES_SAMPLE_RATE=0.1
+#
+# Fraction of profiled transactions to send to Sentry (0.0–1.0).
+# Profiling is only active when SENTRY_TRACES_SAMPLE_RATE > 0. Default: 0.0 (disabled).
+# SENTRY_PROFILES_SAMPLE_RATE=0.0
+#
+# Attach PII (IP addresses, user agents) to Sentry events.
+# Disable (default) to stay GDPR/CCPA compliant.
+# SENTRY_SEND_DEFAULT_PII=false
+
+# **Mobile App – Push Notifications**
+# Push notifications are delivered via Expo's push notification service
+# (https://expo.dev/notifications) which routes to APNs (iOS) and FCM (Android).
+# No additional credentials are required on the server side.
+# The mobile app registers its Expo push token via POST /api/mobile/register-device.
+#
+# To use native FCM/APNs directly (without Expo relay), replace the
+# send_expo_push_notification function in app/utils/push_notification.py.

.github/copilot-instructions.md

@@ -163,6 +163,13 @@ pytest --tb=short -q
 - Keep JavaScript minimal - prefer server-side rendering
 - Follow existing template structure and patterns
 
+### Internationalization (i18n) & Localization (l10n)
+- **Always** use the `_("key")` helper in Jinja2 templates and `translate("key", locale)` in Python for every user-visible string — never hardcode UI text.
+- **Only add new keys to `frontend/translations/en.json`** — that is the one and only file you must touch when introducing new UI strings.
+- Do **not** manually edit any non-English translation file (`de.json`, `fr.json`, etc.). An external automation script syncs all other language files from `en.json` automatically.
+- Key naming convention: `<section>.<descriptor>` in snake_case, e.g. `language.search_placeholder`, `nav.help`, `common.cancel`.
+- The `test_all_languages_have_same_keys` check has been intentionally removed — key completeness across locales is enforced by the external sync script, not by the test suite.
+
 ### Testing
 - Write tests in `tests/` directory, mirroring `app/` structure
 - Use pytest markers: `@pytest.mark.unit`, `@pytest.mark.integration`, etc.

.github/workflows/ci.yml

@@ -17,6 +17,7 @@ concurrency:
 
 env:
   IMAGE_NAME: christianlouis/docuelevate
+  FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
 
 jobs:
   # ══════════════════════════════════════════════════════════════════════════

.github/workflows/codeql.yml

@@ -8,6 +8,9 @@ on:
   schedule:
     - cron: '37 1 * * 1'
 
+env:
+  FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
+
 jobs:
   analyze:
     name: Analyze (${{ matrix.language }})
@@ -26,6 +29,8 @@ jobs:
           build-mode: none
         - language: javascript
           build-mode: none
+        - language: javascript-typescript
+          build-mode: none
         - language: python
           build-mode: none
     steps:

.github/workflows/release.yml

@@ -12,6 +12,9 @@ permissions:
   pull-requests: write
   packages: write
 
+env:
+  FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
+
 jobs:
   release:
     name: Semantic Release

.github/workflows/ruff-auto-fix.yml

@@ -16,6 +16,9 @@ permissions:
   contents: write
   pull-requests: write
 
+env:
+  FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
+
 jobs:
   ruff-auto-fix:
     name: Auto-fix Ruff Issues

.jules/sentinel.md

@@ -0,0 +1,4 @@
+## 2024-05-24 - SSRF in WebDAV connection test
+**Vulnerability:** The `_test_webdav_connection` function had a custom SSRF check that failed to resolve DNS names, allowing attackers to bypass the check by providing a domain that resolves to an internal IP (e.g., `127.0.0.1`).
+**Learning:** DNS resolution is required for robust SSRF protection when validating URLs provided by users.
+**Prevention:** Use a centralized `is_private_ip` function (now in `app/utils/network.py`) that resolves the hostname to its IPs and checks if any are private.

BUILD_DATE

@@ -1 +1 @@
-2026-03-08T12:15:34Z
+2026-03-15T21:39:26Z

GIT_SHA

@@ -1 +1 @@
-640662e
+237af31

README.md

@@ -24,121 +24,154 @@
 </div>
 
 <div align="center">
-  <a href="https://www.docuelevate.org"><img src="frontend/static/hero.png" alt="DocuElevate Logo" width="80%" /></a>
+  <a href="https://www.docuelevate.org"><img src="frontend/static/hero.png" alt="DocuElevate Hero" width="80%" /></a>
 </div>
 
 ## Overview
 
-DocuElevate automates the handling, extraction, and processing of documents using a variety of services, including:
+DocuElevate is an intelligent document processing system that automates the ingestion, OCR, AI-powered metadata extraction, and distribution of documents. It supports a wide range of AI providers, OCR engines, and cloud storage destinations out of the box.
 
-- **AI Provider** (pluggable – OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Portkey, and more) for metadata extraction and text refinement.
-- **Dropbox**, **Nextcloud**, and **Google Drive** for file storage and uploads.
-- **Paperless NGX** for document indexing and management.
-- **Azure Document Intelligence** for OCR on PDFs.
-- **Gotenberg** for file-to-PDF conversions.
-- **Authentik** for authentication and user management.
+**Key capabilities:**
 
-It is designed for flexibility and configurability through environment variables, making it easily customizable for different workflows. The system can fetch documents from multiple IMAP mailboxes, process them (OCR, metadata extraction, PDF conversion), and store them in the desired destinations.
+- **AI-Powered Metadata Extraction** — pluggable AI providers including OpenAI, Anthropic Claude, Google Gemini, Ollama (local), OpenRouter, Portkey, and Azure OpenAI via LiteLLM
+- **Multi-Engine OCR** — Azure Document Intelligence, Tesseract, EasyOCR, Mistral OCR, Google Cloud Document AI, and AWS Textract with configurable merge strategies
+- **12 Storage Destinations** — Dropbox, Google Drive, OneDrive, Amazon S3, Nextcloud, WebDAV, FTP, SFTP, iCloud Drive, Email (SMTP), Paperless-ngx, and Rclone
+- **Multi-Channel Ingestion** — web upload, browser extension, mobile app, CLI, REST API, IMAP email, and watched folders (local, cloud, FTP/SFTP)
+- **Processing Pipelines** — customizable multi-step workflows with conditional routing rules
+- **Full-Text Search** — powered by Meilisearch for instant document discovery
+- **Multi-User with SSO** — local accounts, OAuth2/OIDC (Authentik), and social login (Google, Microsoft, Apple, Dropbox)
 
-The project includes a **UI** for uploading and managing files, and an API documentation page is available at `/docs` (powered by **FastAPI**).
-
-## Documentation Index
-
-- [User Guide](docs/UserGuide.md) - How to use DocuElevate
-- [Browser Extension Guide](docs/BrowserExtension.md) - Install and use the browser extension
-- [API Documentation](docs/API.md) - API reference
-- [Deployment Guide](docs/DeploymentGuide.md) - How to deploy DocuElevate
-- [Configuration Guide](docs/ConfigurationGuide.md) - Available configuration options
-- [Build Metadata](docs/BuildMetadata.md) - Automated version and build information
-- [CI/CD Tools Guide](docs/CIToolsGuide.md) - CI/CD pipeline and tool documentation
-- [CI Workflow Guide](docs/CIWorkflow.md) - Detailed workflow documentation
-- [Development Guide](CONTRIBUTING.md) - How to contribute to DocuElevate
-- [Troubleshooting](docs/Troubleshooting.md) - Common issues and solutions
+The project ships with a web UI, a REST + GraphQL API, a CLI tool, a native mobile app (iOS & Android), a browser extension, and Helm charts for Kubernetes deployment.
 
 ## Screenshots
 
 <div align="center">
   <img src="docs/upload-view.png" alt="DocuElevate Upload Interface" width="80%" />
-  <p><em>Upload interface for adding new documents</em></p>
+  <p><em>Upload interface — drag-and-drop file upload with real-time progress</em></p>
 
   <img src="docs/files-view.png" alt="DocuElevate Files View" width="80%" />
-  <p><em>Files view with processed documents and metadata</em></p>
+  <p><em>Files view — processed documents with AI-extracted metadata</em></p>
+
+  <img src="docs/status-view.png" alt="DocuElevate Status View" width="80%" />
+  <p><em>Status view — system health and service monitoring</em></p>
 </div>
 
 > **Note:** Screenshots may not reflect the very latest UI. For the most current look, visit [docuelevate.org](https://www.docuelevate.org).
 
-## Workflow Process
-
-DocuElevate follows a streamlined document processing workflow:
+## Workflow
 
 <div align="center">
   <img src="docs/workflow-diagram.png" alt="DocuElevate Workflow" width="90%" />
 </div>
 
-### Document Ingestion
-Documents enter DocuElevate through four possible channels:
-1. **Web Upload**: Users manually upload files via the web interface
-2. **Browser Extension**: Send files directly from your browser with one click
-3. **Email Attachments**: Automatic polling of configured IMAP mailboxes (supports multiple accounts)
-4. **API**: Direct programmatic uploads via the REST API
+### Ingestion
+
+Documents enter DocuElevate through multiple channels:
+
+| Channel | Description |
+|---------|-------------|
+| **Web Upload** | Drag-and-drop interface with real-time progress (up to 1 GB per file) |
+| **Browser Extension** | Clip web pages or send files from Chrome, Firefox, or Edge |
+| **Mobile App** | Capture documents with the device camera or upload from the photo library |
+| **CLI** | Batch uploads and scripted workflows via the `docuelevate` command-line tool |
+| **REST API** | Programmatic uploads with full API-token authentication |
+| **Email (IMAP)** | Automatic polling of multiple mailboxes with attachment filtering |
+| **Watched Folders** | Monitor local paths, FTP, SFTP, S3, Dropbox, Google Drive, OneDrive, Nextcloud, or WebDAV for new files |
 
 ### Processing Pipeline
-Every document goes through the following steps:
-1. **PDF Conversion**: Non-PDF files are converted to PDF format using Gotenberg
-2. **OCR Processing**: Azure Document Intelligence extracts text from images/scans
-3. **Metadata Extraction**: The configured AI provider analyzes document content to identify:
-   - Document type (invoice, receipt, contract, etc.)
-   - Key entities (dates, names, amounts, account numbers)
-   - Important data points specific to the document type
-4. **Enrichment**: Metadata is attached to the document in a structured format
+
+Each document passes through a configurable set of steps:
+
+1. **PDF Conversion** — Non-PDF files are converted using Gotenberg, with optional PDF/A archival conversion
+2. **OCR** — Text extraction via one or more OCR engines (Azure, Tesseract, EasyOCR, Mistral, Google Document AI, AWS Textract) with configurable merge strategies
+3. **AI Metadata Extraction** — The configured AI provider classifies the document and extracts structured metadata (type, dates, amounts, entities)
+4. **Enrichment** — Metadata is embedded into the PDF and stored alongside the document
+5. **Embedding Generation** — Vector embeddings for similarity search and duplicate detection
+
+Steps can be customized using **Pipelines** and **Routing Rules** for conditional processing.
 
 ### Distribution
-Processed documents with their metadata can be automatically sent to:
-- **Dropbox**: For cloud storage and sharing
-- **Nextcloud**: For self-hosted file storage
-- **Google Drive**: For Google Workspace integration
-- **Paperless-NGX**: For advanced document management with search capabilities
 
-Users can choose to send documents to any combination of these destinations through configuration settings or manual selection.
+Processed documents are distributed to any combination of configured destinations:
+
+| Destination | Type |
+|------------|------|
+| **Dropbox** | Cloud storage |
+| **Google Drive** | Cloud storage |
+| **OneDrive** | Cloud storage |
+| **Amazon S3** | Object storage |
+| **Nextcloud** | Self-hosted cloud |
+| **WebDAV** | Protocol-based |
+| **FTP / SFTP** | File transfer |
+| **iCloud Drive** | Apple cloud |
+| **Email (SMTP)** | Send as attachment |
+| **Paperless-ngx** | Document management system |
+| **Rclone** | 70+ cloud providers via Rclone |
 
 ## Features
 
-- **Intuitive File Upload**:
-  - Drag-and-drop file upload on both Upload and Files pages—upload anywhere on the Files page
-  - Real-time upload progress with validation
-  - Support for PDF, Office documents, images, and more (up to 500MB per file)
-- **Browser Extension**:
-  - Send files directly from your browser to DocuElevate with one click
-  - Compatible with Chrome, Firefox, Edge, and other Chromium-based browsers
-  - Context menu integration for quick access
-  - See [Browser Extension Guide](docs/BrowserExtension.md) for installation and usage
-- **Document Upload & Storage**:
-  - Manual uploads (via API or UI) to Dropbox, Nextcloud, Google Drive, or Paperless
-- **OCR Processing (Azure)**:
-  - Extract text from scanned PDFs using Azure Document Intelligence
-- **Metadata Extraction (AI Provider)**:
-  - Use any supported AI provider (OpenAI, Anthropic, Gemini, Ollama, etc.) to classify, label, or otherwise enrich the text with structured metadata
-- **PDF Conversion (Gotenberg)**:
-  - Convert non-PDF attachments (e.g., Word docs, images) into PDFs
-- **Document Management (Paperless NGX)**:
-  - Store processed documents and metadata in a Paperless NGX instance
-- **IMAP Integration**:
-  - Fetch documents from multiple mailboxes (including Gmail) and automatically enqueue them for processing
-- **Authentication**:
-  - Secure access to the system using **Authentik** for OAuth2-based login
+### Document Processing
+- **Multi-engine OCR** with quality checks and configurable merge strategies (AI merge, longest, primary)
+- **AI metadata extraction** using any supported provider (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Portkey, Azure OpenAI)
+- **PDF conversion** via Gotenberg with optional PDF/A archival format
+- **Duplicate detection** — exact (SHA-256) and near-duplicate (content similarity with vector embeddings)
+- **Customizable pipelines** — define multi-step processing workflows with conditional routing rules
 
-## Frameworks Used
+### Document Management
+- **Full-text search** powered by Meilisearch with saved searches
+- **File detail view** with metadata, text preview, processing history, and similarity analysis
+- **Shared links** for public document access with expiration controls
+- **Bulk operations** — reprocess, delete, or reassign documents in batch
 
-- **FastAPI**: High-performance web framework for APIs.
-- **Celery**: Task queue for asynchronous processing.
-- **Redis**: Message broker and result backend.
-- **SQLAlchemy**: ORM for database interactions.
-- **Tailwind CSS**: Utility-first CSS framework.
-- **Docker**: Containerization for easy deployment.
+### Multi-Channel Ingestion
+- **Web UI** — drag-and-drop upload with real-time progress
+- **Browser extension** — clip web pages or send files from Chrome, Firefox, Edge ([guide](docs/BrowserExtension.md))
+- **Mobile app** — iOS and Android with camera capture, push notifications, and SSO ([guide](docs/MobileApp.md))
+- **CLI tool** — batch uploads, downloads, search, and API-token management ([guide](docs/CLIGuide.md))
+- **REST API & GraphQL** — full programmatic access with Swagger documentation at `/docs`
+- **IMAP email** — poll multiple mailboxes with attachment filtering and auto-processing
+- **Watched folders** — local filesystem, FTP, SFTP, and cloud storage providers
+
+### Administration
+- **Multi-user mode** with per-user document isolation and ownership
+- **Subscription & billing** — Stripe integration with configurable plans and quotas
+- **Scheduled jobs** — IMAP polling, watched folder scans, automated backups, uptime monitoring
+- **Audit logging** with SIEM integration support
+- **Compliance templates** — GDPR, HIPAA, SOC 2
+- **Admin dashboard** — user management, queue monitoring, credential management, backup/restore
+
+### Authentication & Security
+- **Local accounts** with self-service registration and password reset
+- **OAuth2/OIDC** via Authentik or any OIDC provider
+- **Social login** — Google, Microsoft, Apple, Dropbox
+- **API tokens** for CLI, mobile, and automation access
+- **Security headers** — HSTS, CSP, X-Frame-Options, X-Content-Type-Options
+- **Rate limiting** with configurable per-endpoint controls
+
+### Notifications
+- **100+ notification backends** via Apprise — Discord, Telegram, Slack, Microsoft Teams, Email, webhooks, and more
+- **Configurable events** — task failures, credential issues, file processed, user signup, payment issues
+- **In-app notification inbox** with per-user preferences
+- **Webhooks** — push events to external systems with HMAC signature verification and retry
+
+## Tech Stack
+
+| Component | Technology |
+|-----------|-----------|
+| **Backend** | FastAPI, Celery, Redis, SQLAlchemy, Alembic |
+| **Frontend** | Jinja2, Tailwind CSS |
+| **Search** | Meilisearch |
+| **Mobile** | React Native (Expo) — iOS & Android |
+| **AI** | LiteLLM (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Portkey) |
+| **OCR** | Azure Document Intelligence, Tesseract, EasyOCR, Mistral, Google Doc AI, AWS Textract |
+| **PDF** | Gotenberg, pypdf |
+| **Auth** | Authlib (OAuth2/OIDC), MSAL, social providers |
+| **Infrastructure** | Docker, Docker Compose, Helm/Kubernetes |
+| **Docs** | MkDocs Material |
 
 ## Quick Start
 
-For detailed installation and deployment instructions, please refer to the [Deployment Guide](docs/DeploymentGuide.md).
+For detailed installation and deployment instructions, see the [Deployment Guide](docs/DeploymentGuide.md).
 
 ```bash
 # Clone the repository
@@ -147,20 +180,96 @@ cd DocuElevate
 
 # Configure environment variables
 cp .env.demo .env
-# Edit .env with your settings
+# Edit .env with your settings (see Configuration Guide for all options)
 
 # Run with Docker Compose
-docker-compose up -d
+docker compose up -d
 ```
 
-The API will be available at **`http://localhost:8000`**, and the API documentation is available at **`http://localhost:8000/docs`**.
+The web UI is available at **`http://localhost:8000`** and the interactive API documentation at **`http://localhost:8000/docs`**.
+
+### Kubernetes / Helm
+
+```bash
+helm repo add docuelevate https://christianlouis.github.io/DocuElevate
+helm install docuelevate docuelevate/docuelevate -f values.yaml
+```
+
+See the [Kubernetes Deployment Guide](docs/KubernetesDeployment.md) for full details.
+
+## Documentation
+
+### Getting Started
+
+| Guide | Description |
+|-------|-------------|
+| [Setup Wizard](docs/SetupWizard.md) | Interactive first-run setup |
+| [User Guide](docs/UserGuide.md) | How to use DocuElevate |
+| [Browser Extension](docs/BrowserExtension.md) | Install and use the browser extension |
+| [Mobile App](docs/MobileApp.md) | iOS and Android mobile app |
+| [CLI Guide](docs/CLIGuide.md) | Command-line tool for automation |
+
+### How-To Guides
+
+| Guide | Description |
+|-------|-------------|
+| [How-To Overview](docs/HowToGuides.md) | Index of all how-to guides |
+| [Email Ingestion](docs/howto/EmailIngestion.md) | Set up IMAP email polling |
+| [Watched Folder](docs/howto/WatchedFolderSetup.md) | Monitor local or remote folders |
+| [Mobile Scanning](docs/howto/MobileScanning.md) | Scan documents with your phone |
+
+### Reference
+
+| Guide | Description |
+|-------|-------------|
+| [API Documentation](docs/API.md) | REST & GraphQL API reference |
+| [Configuration Guide](docs/ConfigurationGuide.md) | All environment variables |
+| [Configuration Master](docs/ConfigurationMaster.md) | Configuration overview |
+| [Settings Management](docs/SettingsManagement.md) | Runtime settings UI |
+
+### Deployment & Operations
+
+| Guide | Description |
+|-------|-------------|
+| [Deployment Guide](docs/DeploymentGuide.md) | Docker Compose deployment |
+| [Kubernetes / Helm](docs/KubernetesDeployment.md) | Kubernetes deployment with Helm charts |
+| [Production Readiness](docs/ProductionReadiness.md) | Checklist for production environments |
+| [Database Configuration](docs/DatabaseConfiguration.md) | Database setup and migration |
+| [Backup & Restore](docs/ConfigurationGuide.md#backup--restore) | Automated backup configuration |
+
+### Storage Integration Setup
+
+| Guide | Description |
+|-------|-------------|
+| [Dropbox](docs/DropboxSetup.md) | Dropbox OAuth setup |
+| [Google Drive](docs/GoogleDriveSetup.md) | Google Drive service account / OAuth |
+| [OneDrive](docs/OneDriveSetup.md) | Microsoft OneDrive setup |
+| [Amazon S3](docs/AmazonS3Setup.md) | S3 bucket configuration |
+| [Authentication](docs/AuthenticationSetup.md) | OAuth2, OIDC, and social login |
+| [Notifications](docs/NotificationsSetup.md) | Notification backend setup |
+
+### Security & Compliance
+
+| Guide | Description |
+|-------|-------------|
+| [Credential Rotation](docs/CredentialRotationGuide.md) | Rotate secrets safely |
+| [Licensing Compliance](docs/LicensingCompliance.md) | Dependency licenses |
+| [Privacy & GDPR](docs/PrivacyCompliance.md) | Privacy compliance |
+
+### Development
+
+| Guide | Description |
+|-------|-------------|
+| [Contributing](CONTRIBUTING.md) | Code style, commits, and PR process |
+| [Troubleshooting](docs/Troubleshooting.md) | Common issues and solutions |
+| [Configuration Troubleshooting](docs/ConfigurationTroubleshooting.md) | Configuration-specific issues |
+| [Build Metadata](docs/BuildMetadata.md) | Version and build information |
+| [Internationalization](docs/InternationalizationGuide.md) | Translation and localization |
 
 ## Development & Testing
 
 ### Running Tests
 
-DocuElevate includes comprehensive test coverage. To run tests:
-
 ```bash
 # Install development dependencies
 pip install -r requirements-dev.txt
@@ -175,21 +284,21 @@ pytest --cov=app --cov-report=term-missing
 pytest -m unit
 ```
 
-Tests are automatically configured with the necessary environment variables - **no manual setup required!**
+Tests are automatically configured with the necessary environment variables — **no manual setup required!**
 
-For detailed testing information, including integration tests with Docker and authentication testing, see the [Contributing Guide](CONTRIBUTING.md#running-tests).
+For detailed testing information, see the [Contributing Guide](CONTRIBUTING.md#running-tests).
 
 ### Contributing
 
 We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for:
-- Code style guidelines
+- Code style guidelines (Ruff for formatting and linting)
 - Commit message format (Conventional Commits)
 - Testing requirements
 - Pull request process
 
 ## License
 
-This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
+This project is licensed under the Apache License 2.0 — see the [LICENSE](LICENSE) file for details.
 
 ## Third-Party Software
 
@@ -214,13 +323,10 @@ The following is a summary of the licenses used by our direct dependencies:
 | Uvicorn    | BSD     |
 | SQLAlchemy | MIT     |
 | Pydantic   | MIT     |
-| openai     | MIT     |
 | litellm    | MIT     |
 | pypdf      | BSD     |
 | Requests   | Apache 2.0 |
-| puremagic  | MIT     |
-| filetype   | MIT     |
-| Dropbox    | MIT     |
+| Dropbox SDK | MIT    |
 | Azure AI Document Intelligence | MIT |
 | Authlib    | BSD     |
 | Starlette  | BSD     |
@@ -229,15 +335,15 @@ The following is a summary of the licenses used by our direct dependencies:
 | Microsoft Graph Core | MIT |
 | MSAL       | MIT     |
 | Boto3      | Apache 2.0 |
-| Paramiko   | LGPL-2.1|
+| Paramiko   | LGPL-2.1 |
 | Apprise    | MIT     |
-| Redis      | BSD     |
-| Gotenberg  | MIT     |
+| Redis (py) | BSD     |
+| Gotenberg Client | MIT |
+| Meilisearch | MIT   |
 
 For a comprehensive list of all dependencies and their licenses, run:
 
-```
+```bash
 pip install pip-licenses
 pip-licenses
-
 ```

RUNTIME_INFO

@@ -1,10 +1,10 @@
 DocuElevate Build Information
 ==============================
-Version: 0.93.1
-Build Date: 2026-03-08T12:15:34Z
-Git Commit: 640662e9b265d22885adf9cab5699399624bc0a1
-Git Short SHA: 640662e
+Version: 0.145.2
+Build Date: 2026-03-15T21:39:26Z
+Git Commit: 237af31f5fe598cfc3c08f2bbba79b3d0925787e
+Git Short SHA: 237af31
 Git Branch: main
-Commit Date: 2026-03-08T13:15:14+01:00
-Build Timestamp: 2026-03-08T12:15:34Z
+Commit Date: 2026-03-15T22:39:04+01:00
+Build Timestamp: 2026-03-15T21:39:26Z
 ==============================

SECURITY_AUDIT.md

@@ -861,3 +861,21 @@ Security headers implementation is complete and production-ready. The middleware
 ---
 
 **Next Audit Due:** 2026-05-07 (Quarterly)
+
+## Per-User IMAP Account Passwords (Added 2026-03-08)
+
+### Known Limitation: Plain-text Password Storage
+
+IMAP account passwords in the `user_imap_accounts` table are stored in plain text in the database.
+
+**Risk:** Anyone with direct database access (DBA, backup access) can read IMAP credentials for all users.
+
+**Mitigations in place:**
+- Database itself should be protected with appropriate OS-level file permissions (SQLite) or network ACLs (PostgreSQL/MySQL).
+- Passwords are never returned in API responses (the `_to_response` serialiser omits them).
+- Only the account owner can read or update their own accounts (ownership enforced at the API layer).
+- Passwords are never logged.
+
+**Future improvement:** Encrypt IMAP passwords at rest using `cryptography.fernet` (symmetric encryption with the app's `SESSION_SECRET` as key material). This is tracked as a TODO item in `app/api/imap_accounts.py` and should be implemented before this feature is used in high-security environments.
+
+**Recommended admin action:** Use app-specific passwords (Gmail, Outlook) rather than account passwords where possible, so that compromised IMAP credentials can be revoked without affecting the user's primary account.

VERSION

@@ -1 +1 @@
-0.93.1
+0.145.2

app/api/__init__.py

@@ -7,26 +7,40 @@ import logging
 from fastapi import APIRouter
 
 from app.api.admin_users import router as admin_users_router
+from app.api.api_tokens import router as api_tokens_router
+from app.api.audit_logs import router as audit_logs_router
 from app.api.azure import router as azure_router
 from app.api.backup import router as backup_router
 from app.api.billing import router as billing_router
+from app.api.compliance import router as compliance_router
 from app.api.database import router as database_router
 from app.api.diagnostic import router as diagnostic_router
 from app.api.dropbox import router as dropbox_router
 from app.api.duplicates import router as duplicates_router
 from app.api.files import router as files_router
 from app.api.google_drive import router as google_drive_router
+from app.api.i18n import router as i18n_router
+from app.api.imap_accounts import router as imap_accounts_router
+from app.api.imap_profiles import router as imap_profiles_router
+from app.api.integrations import router as integrations_router
 from app.api.logs import router as logs_router
+from app.api.mobile import router as mobile_router
+from app.api.notifications import router as notifications_router
 from app.api.onboarding import router as onboarding_router
 from app.api.onedrive import router as onedrive_router
 from app.api.openai import router as openai_router
 from app.api.pipelines import router as pipelines_router
 from app.api.plans import router as plans_router
 from app.api.process import router as process_router
+from app.api.profile import router as profile_router
 from app.api.queue import router as queue_router
+from app.api.routing_rules import router as routing_rules_router
 from app.api.saved_searches import router as saved_searches_router
+from app.api.scheduled_jobs import router as scheduled_jobs_router
 from app.api.search import router as search_router
 from app.api.settings import router as settings_router
+from app.api.shared_links import public_router as shared_links_public_router
+from app.api.shared_links import router as shared_links_router
 from app.api.similarity import router as similarity_router
 from app.api.subscriptions import router as subscriptions_router
 from app.api.url_upload import router as url_upload_router
@@ -43,6 +57,7 @@ router = APIRouter()
 
 # Include all the routers
 router.include_router(admin_users_router)
+router.include_router(api_tokens_router)
 router.include_router(user_router)
 router.include_router(backup_router)
 router.include_router(files_router)
@@ -60,6 +75,8 @@ router.include_router(search_router)
 router.include_router(queue_router)
 router.include_router(saved_searches_router)
 router.include_router(similarity_router)
+router.include_router(shared_links_router)
+router.include_router(shared_links_public_router)
 router.include_router(duplicates_router)
 router.include_router(webhooks_router)
 router.include_router(database_router)
@@ -68,3 +85,14 @@ router.include_router(plans_router)
 router.include_router(onboarding_router)
 router.include_router(billing_router)
 router.include_router(pipelines_router)
+router.include_router(profile_router)
+router.include_router(routing_rules_router)
+router.include_router(imap_accounts_router)
+router.include_router(imap_profiles_router)
+router.include_router(integrations_router)
+router.include_router(notifications_router)
+router.include_router(scheduled_jobs_router)
+router.include_router(audit_logs_router)
+router.include_router(i18n_router)
+router.include_router(mobile_router)
+router.include_router(compliance_router)

app/api/api_tokens.py

@@ -0,0 +1,224 @@
+"""API endpoints for managing personal API tokens.
+
+Provides CRUD operations so users can create, list, and revoke tokens
+that grant programmatic access to the DocuElevate API (e.g. webhook
+uploads, scripted integrations).
+
+Tokens use ``secrets.token_urlsafe`` from the Python standard library
+(no extra dependencies) and are prefixed with ``de_`` for easy
+identification.  Only a PBKDF2-HMAC-SHA256 hash is persisted; the
+plaintext is returned exactly once at creation time.
+"""
+
+import hashlib
+import logging
+import secrets
+from datetime import datetime, timezone
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import ApiToken
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/api-tokens", tags=["api-tokens"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Constants
+# ---------------------------------------------------------------------------
+
+#: Prefix prepended to every generated token for easy identification.
+TOKEN_PREFIX = "de_"
+#: Number of random bytes for the token body (32 → 43 URL-safe chars).
+TOKEN_BYTES = 32
+#: PBKDF2 iteration count for hashing API tokens.
+TOKEN_HASH_ITERATIONS = 100_000
+#: PBKDF2 salt for API token hashing (not secret, but fixed for determinism).
+TOKEN_HASH_SALT = b"api-token-v1"
+
+
+# ---------------------------------------------------------------------------
+# Auth helper
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if not owner_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def generate_api_token() -> str:
+    """Generate a new API token with the ``de_`` prefix.
+
+    Returns:
+        A URL-safe random token string, e.g. ``de_Ab3xY…``.
+    """
+    return TOKEN_PREFIX + secrets.token_urlsafe(TOKEN_BYTES)
+
+
+def hash_token(token: str) -> str:
+    """Return a PBKDF2-HMAC-SHA256 hex digest of *token*.
+
+    Args:
+        token: The plaintext API token.
+
+    Returns:
+        64-character lowercase hex string.
+    """
+    dk = hashlib.pbkdf2_hmac(
+        "sha256",
+        token.encode("utf-8"),
+        TOKEN_HASH_SALT,
+        TOKEN_HASH_ITERATIONS,
+    )
+    return dk.hex()
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class TokenCreate(BaseModel):
+    """Schema for creating a new API token."""
+
+    name: str = Field(..., min_length=1, max_length=255, description="Human-readable label for the token")
+
+
+class TokenResponse(BaseModel):
+    """Schema returned when listing tokens (plaintext is never included)."""
+
+    id: int
+    name: str
+    token_prefix: str
+    is_active: bool
+    last_used_at: datetime | None
+    last_used_ip: str | None
+    created_at: datetime | None
+    revoked_at: datetime | None
+
+    model_config = {"from_attributes": True}
+
+
+class TokenCreatedResponse(TokenResponse):
+    """Schema returned once at creation time — includes the full plaintext token."""
+
+    token: str = Field(..., description="The full API token. Store it securely — it will not be shown again.")
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post("/", status_code=status.HTTP_201_CREATED, response_model=TokenCreatedResponse)
+async def create_token(
+    body: TokenCreate,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Create a new personal API token.
+
+    The full token is returned **only once** in the response.  Subsequent
+    ``GET`` requests will only show the prefix for identification.
+    """
+    plaintext = generate_api_token()
+    token_hash_value = hash_token(plaintext)
+    prefix = plaintext[:12]  # "de_" prefix + 9 random chars = 12 chars total
+
+    db_token = ApiToken(
+        owner_id=owner_id,
+        name=body.name,
+        token_hash=token_hash_value,
+        token_prefix=prefix,
+    )
+    try:
+        db.add(db_token)
+        db.commit()
+        db.refresh(db_token)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("API token created: id=%s owner=%s name=%r", db_token.id, owner_id, body.name)
+
+    return {
+        "id": db_token.id,
+        "name": db_token.name,
+        "token_prefix": db_token.token_prefix,
+        "is_active": db_token.is_active,
+        "last_used_at": db_token.last_used_at,
+        "last_used_ip": db_token.last_used_ip,
+        "created_at": db_token.created_at,
+        "revoked_at": db_token.revoked_at,
+        "token": plaintext,
+    }
+
+
+@router.get("/", response_model=list[TokenResponse])
+async def list_tokens(
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> list[dict[str, Any]]:
+    """List all API tokens for the authenticated user."""
+    tokens = db.query(ApiToken).filter(ApiToken.owner_id == owner_id).order_by(ApiToken.created_at.desc()).all()
+    return [
+        {
+            "id": t.id,
+            "name": t.name,
+            "token_prefix": t.token_prefix,
+            "is_active": t.is_active,
+            "last_used_at": t.last_used_at,
+            "last_used_ip": t.last_used_ip,
+            "created_at": t.created_at,
+            "revoked_at": t.revoked_at,
+        }
+        for t in tokens
+    ]
+
+
+@router.delete("/{token_id}", status_code=status.HTTP_200_OK)
+async def revoke_token(
+    token_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Revoke (soft-delete) an API token.
+
+    The token row is kept for audit purposes but marked inactive with a
+    ``revoked_at`` timestamp.
+    """
+    db_token = db.query(ApiToken).filter(ApiToken.id == token_id, ApiToken.owner_id == owner_id).first()
+    if not db_token:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Token not found")
+
+    if not db_token.is_active:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Token is already revoked")
+
+    try:
+        db_token.is_active = False
+        db_token.revoked_at = datetime.now(timezone.utc)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("API token revoked: id=%s owner=%s", token_id, owner_id)
+    return {"detail": "Token revoked"}

app/api/audit_logs.py

@@ -0,0 +1,117 @@
+"""
+Audit log REST API endpoints.
+
+Provides read-only access to the comprehensive audit log for admin users.
+Events are append-only — there are no update or delete endpoints.
+"""
+
+import logging
+from datetime import datetime
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, Query, Request
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.database import get_db
+from app.utils.audit_service import count_events, query_events
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter()
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+@router.get("/audit-logs")
+@require_login
+async def list_audit_logs(
+    request: Request,
+    db: DbSession,
+    action: Annotated[str | None, Query(description="Filter by action (exact match)")] = None,
+    user: Annotated[str | None, Query(description="Filter by username")] = None,
+    resource_type: Annotated[str | None, Query(description="Filter by resource type")] = None,
+    severity: Annotated[str | None, Query(description="Filter by severity level")] = None,
+    since: Annotated[datetime | None, Query(description="Only events at or after this ISO-8601 timestamp")] = None,
+    until: Annotated[datetime | None, Query(description="Only events at or before this ISO-8601 timestamp")] = None,
+    limit: Annotated[int, Query(ge=1, le=500, description="Max rows to return")] = 50,
+    offset: Annotated[int, Query(ge=0, description="Rows to skip for pagination")] = 0,
+) -> dict[str, Any]:
+    """Return audit log entries with optional filtering and pagination.
+
+    Requires authentication. Returns events in reverse chronological order.
+    """
+    entries = query_events(
+        db,
+        action=action,
+        user=user,
+        resource_type=resource_type,
+        severity=severity,
+        since=since,
+        until=until,
+        limit=limit,
+        offset=offset,
+    )
+    total = count_events(
+        db,
+        action=action,
+        user=user,
+        resource_type=resource_type,
+        severity=severity,
+        since=since,
+        until=until,
+    )
+    return {
+        "items": [_serialize(e) for e in entries],
+        "total": total,
+        "limit": limit,
+        "offset": offset,
+    }
+
+
+@router.get("/audit-logs/actions")
+@require_login
+async def list_distinct_actions(
+    request: Request,
+    db: DbSession,
+) -> list[str]:
+    """Return the distinct action values present in the audit log."""
+    from app.models import AuditLog
+
+    rows = db.query(AuditLog.action).distinct().order_by(AuditLog.action).all()
+    return [r[0] for r in rows]
+
+
+@router.get("/audit-logs/users")
+@require_login
+async def list_distinct_users(
+    request: Request,
+    db: DbSession,
+) -> list[str]:
+    """Return the distinct user values present in the audit log."""
+    from app.models import AuditLog
+
+    rows = db.query(AuditLog.user).distinct().order_by(AuditLog.user).all()
+    return [r[0] for r in rows]
+
+
+# ------------------------------------------------------------------
+# Helpers
+# ------------------------------------------------------------------
+
+
+def _serialize(entry) -> dict[str, Any]:
+    """Convert an AuditLog row to a JSON-safe dict."""
+    import json as _json
+
+    return {
+        "id": entry.id,
+        "timestamp": entry.timestamp.isoformat() if entry.timestamp else None,
+        "user": entry.user,
+        "action": entry.action,
+        "resource_type": entry.resource_type,
+        "resource_id": entry.resource_id,
+        "ip_address": entry.ip_address,
+        "details": _json.loads(entry.details) if entry.details else None,
+        "severity": entry.severity,
+    }

app/api/billing.py

@@ -30,6 +30,7 @@ from app.auth import require_login
 from app.config import settings
 from app.database import get_db
 from app.models import SubscriptionPlan, UserProfile
+from app.utils.i18n import translate as _translate
 from app.utils.user_scope import get_current_owner_id
 
 logger = logging.getLogger(__name__)
@@ -37,6 +38,7 @@ router = APIRouter(prefix="/billing", tags=["billing"])
 
 _templates_dir = pathlib.Path(__file__).parents[2] / "frontend" / "templates"
 _templates = Jinja2Templates(directory=str(_templates_dir))
+_templates.env.globals["_"] = lambda key, **kwargs: _translate(key, "en", **kwargs)
 
 
 def _get_stripe() -> stripe.StripeClient | None:
@@ -262,6 +264,205 @@ async def billing_success(request: Request) -> Any:
     return _templates.TemplateResponse("billing_success.html", {"request": request})
 
 
+# ---------------------------------------------------------------------------
+# Admin: Stripe status + sync helpers
+# ---------------------------------------------------------------------------
+
+
+def _require_admin(request: Request) -> None:
+    """Raise 403 if the current session user is not an admin."""
+    user = request.session.get("user") or {}
+    if not user.get("is_admin"):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required.")
+
+
+@router.get("/stripe/status", summary="Check Stripe connection and plan sync status (admin only)")
+@require_login
+async def stripe_status(request: Request, db: Session = Depends(get_db)) -> dict[str, Any]:
+    """Return Stripe connection health and per-plan price-ID sync status.
+
+    Returns a JSON object with:
+    - ``configured``: whether STRIPE_SECRET_KEY is set
+    - ``connection``: ``"ok"`` or an error string (live/test mode label)
+    - ``mode``: ``"live"`` | ``"test"`` | ``null``
+    - ``plans``: list of plan objects with ``plan_id``, ``name``,
+      ``stripe_price_id_monthly``, ``stripe_price_id_yearly``, ``synced``
+
+    Raises:
+        403: Not admin.
+        503: Stripe not configured.
+    """
+    _require_admin(request)
+
+    if not settings.stripe_secret_key:
+        return {
+            "configured": False,
+            "connection": "not_configured",
+            "mode": None,
+            "plans": [],
+        }
+
+    client = _get_stripe()
+    # Probe Stripe with a lightweight account fetch
+    mode: str | None = None
+    connection_status = "ok"
+    try:
+        account = client.accounts.retrieve("me")  # type: ignore[arg-type]
+        livemode = getattr(account, "livemode", None)
+        if livemode is True:
+            mode = "live"
+        elif livemode is False:
+            mode = "test"
+        else:
+            mode = "test" if settings.stripe_secret_key.startswith("sk_test_") else "live"
+    except Exception:
+        logger.exception("Stripe connection check failed")
+        connection_status = "error"
+        mode = "test" if settings.stripe_secret_key.startswith("sk_test_") else "live"
+
+    plans = db.query(SubscriptionPlan).order_by(SubscriptionPlan.sort_order).all()
+    plan_statuses = []
+    for plan in plans:
+        has_monthly = bool(plan.stripe_price_id_monthly)
+        has_yearly = bool(plan.stripe_price_id_yearly)
+        is_paid = plan.price_monthly > 0 or plan.price_yearly > 0
+        synced = (not is_paid) or (has_monthly and (not plan.price_yearly or has_yearly))
+        plan_statuses.append(
+            {
+                "plan_id": plan.plan_id,
+                "name": plan.name,
+                "price_monthly": plan.price_monthly,
+                "price_yearly": plan.price_yearly,
+                "stripe_price_id_monthly": plan.stripe_price_id_monthly,
+                "stripe_price_id_yearly": plan.stripe_price_id_yearly,
+                "synced": synced,
+            }
+        )
+
+    return {
+        "configured": True,
+        "connection": connection_status,
+        "mode": mode,
+        "webhook_secret_configured": bool(settings.stripe_webhook_secret),
+        "plans": plan_statuses,
+        "webhook_endpoint": str(request.base_url).rstrip("/") + "/api/billing/webhook",
+    }
+
+
+@router.post("/stripe/sync-plans", summary="Auto-create Stripe products and prices for all plans (admin only)")
+@require_login
+async def stripe_sync_plans(request: Request, db: Session = Depends(get_db)) -> dict[str, Any]:
+    """Create Stripe Product + Price objects for every paid plan that is missing them.
+
+    For each paid plan (``price_monthly > 0``) that lacks a ``stripe_price_id_monthly``,
+    this endpoint:
+
+    1. Creates a Stripe *Product* named after the plan.
+    2. Creates a Stripe *Price* for the monthly amount.
+    3. Optionally creates a yearly Price if ``price_yearly > 0``.
+    4. Persists the resulting ``price_id`` values back into ``SubscriptionPlan``.
+
+    Already-synced plans (those that already have ``stripe_price_id_monthly``) are
+    skipped — existing prices in Stripe are never modified.
+
+    Raises:
+        403: Not admin.
+        503: Stripe not configured.
+    """
+    _require_admin(request)
+
+    client = _get_stripe()
+    if not client:
+        raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail="Billing is not configured.")
+
+    plans = db.query(SubscriptionPlan).order_by(SubscriptionPlan.sort_order).all()
+    results: list[dict[str, Any]] = []
+
+    for plan in plans:
+        is_paid = plan.price_monthly > 0 or plan.price_yearly > 0
+        if not is_paid:
+            results.append({"plan_id": plan.plan_id, "name": plan.name, "status": "skipped_free"})
+            continue
+
+        already_has_monthly = bool(plan.stripe_price_id_monthly)
+        already_has_yearly = bool(plan.stripe_price_id_yearly)
+
+        if already_has_monthly and (not plan.price_yearly or already_has_yearly):
+            results.append({"plan_id": plan.plan_id, "name": plan.name, "status": "already_synced"})
+            continue
+
+        try:
+            # Create (or look up) the Stripe Product for this plan
+            product = client.products.create(
+                params={
+                    "name": str(plan.name),
+                    "metadata": {"docuelevate_plan_id": plan.plan_id},
+                }
+            )
+
+            changed = False
+
+            # Monthly price
+            if not already_has_monthly and plan.price_monthly > 0:
+                monthly_price = client.prices.create(
+                    params={
+                        "product": product.id,
+                        "unit_amount": int(round(plan.price_monthly * 100)),
+                        "currency": "usd",
+                        "recurring": {"interval": "month"},
+                        "metadata": {"docuelevate_plan_id": plan.plan_id, "billing_cycle": "monthly"},
+                    }
+                )
+                plan.stripe_price_id_monthly = monthly_price.id
+                changed = True
+
+            # Yearly price
+            if not already_has_yearly and plan.price_yearly > 0:
+                yearly_price = client.prices.create(
+                    params={
+                        "product": product.id,
+                        "unit_amount": int(round(plan.price_yearly * 100)),
+                        "currency": "usd",
+                        "recurring": {"interval": "year"},
+                        "metadata": {"docuelevate_plan_id": plan.plan_id, "billing_cycle": "yearly"},
+                    }
+                )
+                plan.stripe_price_id_yearly = yearly_price.id
+                changed = True
+
+            if changed:
+                db.commit()
+                logger.info(
+                    "Stripe sync: created product/prices for plan %s (product %s)",
+                    plan.plan_id,
+                    product.id,
+                )
+
+            results.append(
+                {
+                    "plan_id": plan.plan_id,
+                    "name": plan.name,
+                    "status": "created",
+                    "stripe_price_id_monthly": plan.stripe_price_id_monthly,
+                    "stripe_price_id_yearly": plan.stripe_price_id_yearly,
+                }
+            )
+
+        except Exception as exc:
+            db.rollback()
+            logger.error("Stripe sync failed for plan %s: %s", plan.plan_id, exc)
+            results.append(
+                {
+                    "plan_id": plan.plan_id,
+                    "name": str(plan.name),
+                    "status": "error",
+                    "detail": str(exc),
+                }
+            )
+
+    return {"results": results}
+
+
 def _handle_stripe_event(db: Session, event: Any) -> None:
     """Dispatch Stripe event to the appropriate handler.
 

app/api/compliance.py

@@ -0,0 +1,183 @@
+"""API endpoints for managing compliance templates (GDPR, HIPAA, SOC2).
+
+All endpoints require admin privileges.
+
+Available routes:
+    GET   /api/compliance/templates               – list all compliance templates
+    GET   /api/compliance/templates/{name}         – get a single template with checks
+    POST  /api/compliance/templates/{name}/apply   – one-click apply a template
+    GET   /api/compliance/templates/{name}/status  – evaluate compliance status
+    GET   /api/compliance/summary                  – overall compliance dashboard data
+"""
+
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.utils.compliance_service import (
+    COMPLIANCE_TEMPLATES,
+    apply_template,
+    evaluate_template_status,
+    get_all_templates,
+    get_compliance_summary,
+    get_template_by_name,
+)
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/compliance", tags=["compliance"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+# ---------------------------------------------------------------------------
+# Authorisation helper
+# ---------------------------------------------------------------------------
+
+
+def _require_admin(request: Request) -> dict:
+    """Ensure the caller is an admin; raises HTTP 403 otherwise."""
+    user = request.session.get("user")
+    if not user or not user.get("is_admin"):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required")
+    return user
+
+
+AdminUser = Annotated[dict, Depends(_require_admin)]
+
+
+# ---------------------------------------------------------------------------
+# Pydantic response models
+# ---------------------------------------------------------------------------
+
+
+class CheckResult(BaseModel):
+    """Individual compliance check result."""
+
+    key: str
+    label: str
+    description: str
+    expected: str
+    actual: str
+    passing: bool
+
+
+class TemplateStatusResponse(BaseModel):
+    """Status evaluation for a compliance template."""
+
+    status: str
+    total: int
+    passed: int
+    failed: int
+    check_results: list[CheckResult]
+
+
+class TemplateResponse(BaseModel):
+    """Full compliance template representation."""
+
+    id: int
+    name: str
+    display_name: str
+    description: str | None
+    enabled: bool
+    status: str
+    applied_at: str | None
+    applied_by: str | None
+    settings: dict[str, str]
+    checks: list[dict[str, Any]]
+    check_count: int
+
+
+class ApplyResponse(BaseModel):
+    """Result of applying a compliance template."""
+
+    success: bool
+    template: str | None = None
+    applied_settings: dict[str, str] | None = None
+    errors: list[str] | None = None
+    error: str | None = None
+    status: TemplateStatusResponse | None = None
+
+
+class SummaryTemplateResponse(BaseModel):
+    """Per-template summary for the compliance dashboard."""
+
+    name: str
+    display_name: str
+    enabled: bool
+    status: str
+    total: int
+    passed: int
+    failed: int
+    applied_at: str | None
+    applied_by: str | None
+
+
+class ComplianceSummaryResponse(BaseModel):
+    """Overall compliance dashboard summary."""
+
+    overall_status: str
+    total_checks: int
+    total_passed: int
+    total_failed: int
+    templates: list[SummaryTemplateResponse]
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/templates", response_model=list[TemplateResponse])
+async def list_templates(db: DbSession, admin: AdminUser) -> list[dict[str, Any]]:
+    """List all compliance templates with their current status."""
+    return get_all_templates(db)
+
+
+@router.get("/templates/{name}", response_model=TemplateResponse)
+async def get_template(name: str, db: DbSession, admin: AdminUser) -> dict[str, Any]:
+    """Get a single compliance template by name."""
+    templates = get_all_templates(db)
+    for t in templates:
+        if t["name"] == name:
+            return t
+    raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=f"Template '{name}' not found")
+
+
+@router.post("/templates/{name}/apply", response_model=ApplyResponse)
+async def apply_compliance_template(name: str, db: DbSession, admin: AdminUser) -> dict[str, Any]:
+    """Apply a compliance template (one-click).
+
+    Writes all template settings to the database and evaluates the resulting
+    compliance status.
+    """
+    if name not in COMPLIANCE_TEMPLATES:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=f"Template '{name}' not found")
+
+    template = get_template_by_name(db, name)
+    if template is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=f"Template '{name}' not found")
+
+    admin_email = admin.get("email", "admin")
+    result = apply_template(db, name, applied_by=admin_email)
+    if not result.get("success") and result.get("error"):
+        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=result["error"])
+    return result
+
+
+@router.get("/templates/{name}/status", response_model=TemplateStatusResponse)
+async def get_template_status(name: str, db: DbSession, admin: AdminUser) -> dict[str, Any]:
+    """Evaluate the live compliance status of a template."""
+    template = get_template_by_name(db, name)
+    if template is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=f"Template '{name}' not found")
+    return evaluate_template_status(db, name)
+
+
+@router.get("/summary", response_model=ComplianceSummaryResponse)
+async def compliance_summary(db: DbSession, admin: AdminUser) -> dict[str, Any]:
+    """Overall compliance dashboard summary across all templates."""
+    return get_compliance_summary(db)

app/api/graphql_api.py

@@ -0,0 +1,431 @@
+"""
+GraphQL API endpoint for DocuElevate.
+
+Provides a flexible query interface alongside the existing REST API.
+Schema covers: documents, pipelines, settings, and users.
+
+Endpoint: /graphql
+GraphiQL playground: /graphql (via browser)
+"""
+
+from __future__ import annotations
+
+import logging
+from datetime import datetime
+from typing import Annotated, Any
+
+import strawberry
+from fastapi import Depends, Request
+from sqlalchemy.orm import Session
+from strawberry.fastapi import GraphQLRouter
+
+from app.auth import get_current_user
+from app.config import settings
+from app.database import get_db
+from app.models import ApplicationSettings, FileRecord, Pipeline, PipelineStep, UserProfile
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# Strawberry types
+# ---------------------------------------------------------------------------
+
+
+@strawberry.type
+class DocumentType:
+    """A processed document stored in the system."""
+
+    id: int
+    owner_id: str | None
+    original_filename: str | None
+    local_filename: str
+    file_size: int
+    mime_type: str | None
+    document_title: str | None
+    is_duplicate: bool
+    ocr_quality_score: int | None
+    pipeline_id: int | None
+    created_at: datetime | None
+
+
+@strawberry.type
+class PipelineStepType:
+    """A single step within a processing pipeline."""
+
+    id: int
+    pipeline_id: int
+    position: int
+    step_type: str
+    label: str | None
+    enabled: bool
+    created_at: datetime | None
+
+
+@strawberry.type
+class PipelineType:
+    """A processing pipeline with its ordered steps."""
+
+    id: int
+    owner_id: str | None
+    name: str
+    description: str | None
+    is_default: bool
+    is_active: bool
+    steps: list[PipelineStepType]
+    created_at: datetime | None
+    updated_at: datetime | None
+
+
+@strawberry.type
+class SettingType:
+    """An application configuration setting stored in the database."""
+
+    id: int
+    key: str
+    value: str | None
+    created_at: datetime | None
+    updated_at: datetime | None
+
+
+@strawberry.type
+class UserType:
+    """A user profile in the system."""
+
+    id: int
+    user_id: str
+    display_name: str | None
+    is_blocked: bool
+    subscription_tier: str | None
+    onboarding_completed: bool
+    created_at: datetime | None
+
+
+# ---------------------------------------------------------------------------
+# Conversion helpers
+# ---------------------------------------------------------------------------
+
+
+def _document_from_record(rec: FileRecord) -> DocumentType:
+    return DocumentType(
+        id=rec.id,
+        owner_id=rec.owner_id,
+        original_filename=rec.original_filename,
+        local_filename=rec.local_filename,
+        file_size=rec.file_size,
+        mime_type=rec.mime_type,
+        document_title=rec.document_title,
+        is_duplicate=rec.is_duplicate,
+        ocr_quality_score=rec.ocr_quality_score,
+        pipeline_id=rec.pipeline_id,
+        created_at=rec.created_at,
+    )
+
+
+def _pipeline_step_from_record(step: PipelineStep) -> PipelineStepType:
+    return PipelineStepType(
+        id=step.id,
+        pipeline_id=step.pipeline_id,
+        position=step.position,
+        step_type=step.step_type,
+        label=step.label,
+        enabled=step.enabled,
+        created_at=step.created_at,
+    )
+
+
+def _pipeline_from_record(pipeline: Pipeline, db: Session) -> PipelineType:
+    steps = db.query(PipelineStep).filter(PipelineStep.pipeline_id == pipeline.id).order_by(PipelineStep.position).all()
+    return PipelineType(
+        id=pipeline.id,
+        owner_id=pipeline.owner_id,
+        name=pipeline.name,
+        description=pipeline.description,
+        is_default=pipeline.is_default,
+        is_active=pipeline.is_active,
+        steps=[_pipeline_step_from_record(s) for s in steps],
+        created_at=pipeline.created_at,
+        updated_at=pipeline.updated_at,
+    )
+
+
+def _setting_from_record(setting: ApplicationSettings) -> SettingType:
+    return SettingType(
+        id=setting.id,
+        key=setting.key,
+        value=setting.value,
+        created_at=setting.created_at,
+        updated_at=setting.updated_at,
+    )
+
+
+def _user_from_profile(profile: UserProfile) -> UserType:
+    return UserType(
+        id=profile.id,
+        user_id=profile.user_id,
+        display_name=profile.display_name,
+        is_blocked=profile.is_blocked,
+        subscription_tier=profile.subscription_tier,
+        onboarding_completed=profile.onboarding_completed,
+        created_at=profile.created_at,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Context helpers
+# ---------------------------------------------------------------------------
+
+# Keys that contain sensitive data and must never be returned via GraphQL
+_SENSITIVE_SETTING_KEYS: frozenset[str] = frozenset(
+    {
+        "openai_api_key",
+        "azure_ai_key",
+        "session_secret",
+        "database_url",
+        "redis_url",
+        "dropbox_app_secret",
+        "dropbox_refresh_token",
+        "google_drive_credentials_json",
+        "onedrive_client_secret",
+        "onedrive_refresh_token",
+        "smtp_password",
+        "nextcloud_password",
+        "s3_secret_access_key",
+        "ftp_password",
+        "sftp_password",
+        "webdav_password",
+        "stripe_secret_key",
+        "stripe_webhook_secret",
+        "sentry_dsn",
+        "social_auth_google_client_secret",
+        "social_auth_microsoft_client_secret",
+        "social_auth_apple_private_key",
+        "social_auth_dropbox_app_secret",
+    }
+)
+
+
+def _get_current_user_id(user: dict[str, Any] | None) -> str | None:
+    """Extract the stable user identifier from the user dict."""
+    if not user:
+        return None
+    return user.get("preferred_username") or user.get("email") or user.get("id") or None
+
+
+def _get_db_and_user(info: strawberry.types.Info) -> tuple[Session, dict[str, Any] | None]:
+    """Extract the database session and current user from the Strawberry context."""
+    db: Session = info.context["db"]
+    user: dict[str, Any] | None = info.context.get("user")
+    return db, user
+
+
+def _require_auth(user: dict[str, Any] | None) -> None:
+    """Raise an error when authentication is enabled and no valid user is present."""
+    if settings.auth_enabled and not user:
+        raise strawberry.exceptions.StrawberryGraphQLError("Authentication required")
+
+
+def _require_admin(user: dict[str, Any] | None) -> None:
+    """Raise an error when the current user is not an admin.
+
+    When ``auth_enabled`` is *False* (single-user / development mode) all
+    callers are implicitly treated as administrators.
+    """
+    if not settings.auth_enabled:
+        # Single-user mode: no auth, treat caller as admin
+        return
+    _require_auth(user)
+    if not (user and user.get("is_admin")):
+        raise strawberry.exceptions.StrawberryGraphQLError("Admin access required")
+
+
+# ---------------------------------------------------------------------------
+# Query resolvers
+# ---------------------------------------------------------------------------
+
+
+@strawberry.type
+class Query:
+    """Root query type for the DocuElevate GraphQL API."""
+
+    @strawberry.field(description="List documents, optionally filtered by owner.")
+    def documents(
+        self,
+        info: strawberry.types.Info,
+        owner_id: str | None = None,
+        limit: int = 20,
+        offset: int = 0,
+    ) -> list[DocumentType]:
+        """Return a paginated list of documents.
+
+        When *auth_enabled* the caller must be authenticated.  Non-admin users
+        receive only their own documents; admins may query any *owner_id*.
+        """
+        db, user = _get_db_and_user(info)
+        _require_auth(user)
+
+        limit = max(1, min(limit, 100))
+        offset = max(0, offset)
+
+        query = db.query(FileRecord)
+
+        if settings.auth_enabled and user:
+            is_admin = user.get("is_admin", False)
+            current_user_id = _get_current_user_id(user)
+            if not is_admin:
+                # Non-admins can only see their own documents
+                query = query.filter(FileRecord.owner_id == current_user_id)
+            elif owner_id:
+                query = query.filter(FileRecord.owner_id == owner_id)
+        elif owner_id:
+            query = query.filter(FileRecord.owner_id == owner_id)
+
+        records = query.order_by(FileRecord.created_at.desc()).offset(offset).limit(limit).all()
+        return [_document_from_record(r) for r in records]
+
+    @strawberry.field(description="Fetch a single document by ID.")
+    def document(self, info: strawberry.types.Info, id: int) -> DocumentType | None:
+        """Return one document by its primary key, or *null* if not found."""
+        db, user = _get_db_and_user(info)
+        _require_auth(user)
+
+        rec = db.query(FileRecord).filter(FileRecord.id == id).first()
+        if rec is None:
+            return None
+
+        if settings.auth_enabled and user:
+            is_admin = user.get("is_admin", False)
+            current_user_id = _get_current_user_id(user)
+            if not is_admin and rec.owner_id != current_user_id:
+                return None
+
+        return _document_from_record(rec)
+
+    @strawberry.field(description="List processing pipelines.")
+    def pipelines(
+        self,
+        info: strawberry.types.Info,
+        owner_id: str | None = None,
+        limit: int = 20,
+        offset: int = 0,
+    ) -> list[PipelineType]:
+        """Return a paginated list of pipelines."""
+        db, user = _get_db_and_user(info)
+        _require_auth(user)
+
+        limit = max(1, min(limit, 100))
+        offset = max(0, offset)
+
+        query = db.query(Pipeline)
+
+        if settings.auth_enabled and user:
+            is_admin = user.get("is_admin", False)
+            current_user_id = _get_current_user_id(user)
+            if not is_admin:
+                query = query.filter((Pipeline.owner_id == current_user_id) | (Pipeline.owner_id.is_(None)))
+            elif owner_id:
+                query = query.filter(Pipeline.owner_id == owner_id)
+        elif owner_id:
+            query = query.filter(Pipeline.owner_id == owner_id)
+
+        rows = query.order_by(Pipeline.id).offset(offset).limit(limit).all()
+        return [_pipeline_from_record(p, db) for p in rows]
+
+    @strawberry.field(description="Fetch a single pipeline by ID.")
+    def pipeline(self, info: strawberry.types.Info, id: int) -> PipelineType | None:
+        """Return one pipeline by its primary key, or *null* if not found."""
+        db, user = _get_db_and_user(info)
+        _require_auth(user)
+
+        row = db.query(Pipeline).filter(Pipeline.id == id).first()
+        if row is None:
+            return None
+
+        if settings.auth_enabled and user:
+            is_admin = user.get("is_admin", False)
+            current_user_id = _get_current_user_id(user)
+            if not is_admin and row.owner_id is not None and row.owner_id != current_user_id:
+                return None
+
+        return _pipeline_from_record(row, db)
+
+    @strawberry.field(description="List non-sensitive application settings (admin only).")
+    def settings(
+        self,
+        info: strawberry.types.Info,
+        limit: int = 50,
+        offset: int = 0,
+    ) -> list[SettingType]:
+        """Return application settings stored in the database.
+
+        Sensitive keys (API secrets, passwords, etc.) are automatically
+        excluded.  Requires admin privileges when auth is enabled.
+        """
+        db, user = _get_db_and_user(info)
+        _require_admin(user)
+
+        limit = max(1, min(limit, 200))
+        offset = max(0, offset)
+
+        rows = (
+            db.query(ApplicationSettings)
+            .filter(ApplicationSettings.key.notin_(_SENSITIVE_SETTING_KEYS))
+            .order_by(ApplicationSettings.key)
+            .offset(offset)
+            .limit(limit)
+            .all()
+        )
+        return [_setting_from_record(r) for r in rows]
+
+    @strawberry.field(description="List user profiles (admin only).")
+    def users(
+        self,
+        info: strawberry.types.Info,
+        limit: int = 20,
+        offset: int = 0,
+    ) -> list[UserType]:
+        """Return a paginated list of user profiles.  Requires admin privileges."""
+        db, user = _get_db_and_user(info)
+        _require_admin(user)
+
+        limit = max(1, min(limit, 100))
+        offset = max(0, offset)
+
+        rows = db.query(UserProfile).order_by(UserProfile.user_id).offset(offset).limit(limit).all()
+        return [_user_from_profile(r) for r in rows]
+
+    @strawberry.field(description="Fetch a user profile by user_id (admin only).")
+    def user(self, info: strawberry.types.Info, user_id: str) -> UserType | None:
+        """Return one user profile by *user_id*, or *null* if not found."""
+        db, user = _get_db_and_user(info)
+        _require_admin(user)
+
+        row = db.query(UserProfile).filter(UserProfile.user_id == user_id).first()
+        return _user_from_profile(row) if row else None
+
+
+# ---------------------------------------------------------------------------
+# Schema and router
+# ---------------------------------------------------------------------------
+
+schema = strawberry.Schema(query=Query)
+
+
+async def get_graphql_context(
+    request: Request,
+    db: Annotated[Session, Depends(get_db)],
+) -> dict[str, Any]:
+    """Build the per-request context injected into every resolver."""
+    try:
+        user = get_current_user(request)
+    except Exception:
+        logger.debug("Could not resolve current user for GraphQL context", exc_info=True)
+        user = None
+    return {"request": request, "db": db, "user": user}
+
+
+graphql_router = GraphQLRouter(
+    schema,
+    context_getter=get_graphql_context,
+    graphql_ide="graphiql",
+)

app/api/i18n.py

@@ -0,0 +1,136 @@
+"""API endpoints for internationalization (i18n).
+
+Provides endpoints for:
+* Listing available languages
+* Getting/setting user language preference (persisted in session + cookie + DB)
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import APIRouter, Depends, Request, Response
+from pydantic import BaseModel
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import UserProfile
+from app.utils.i18n import (
+    DEFAULT_LANGUAGE,
+    SUPPORTED_LANGUAGE_CODES,
+    SUPPORTED_LANGUAGES,
+    detect_language,
+)
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/i18n", tags=["i18n"])
+
+
+class LanguageInfo(BaseModel):
+    """Schema for a supported language."""
+
+    code: str
+    name: str
+    native: str
+    flag: str
+
+
+class LanguageListResponse(BaseModel):
+    """Response for the list-languages endpoint."""
+
+    languages: list[LanguageInfo]
+    current: str
+    default: str
+
+
+class SetLanguageRequest(BaseModel):
+    """Request body for setting the preferred language."""
+
+    language: str
+
+
+class SetLanguageResponse(BaseModel):
+    """Response after changing the language."""
+
+    language: str
+    message: str
+
+
+@router.get("/languages", response_model=LanguageListResponse)
+async def list_languages(request: Request) -> LanguageListResponse:
+    """Return all supported UI languages and the current active language."""
+    current = detect_language(request)
+    return LanguageListResponse(
+        languages=[LanguageInfo(**lang) for lang in SUPPORTED_LANGUAGES],
+        current=current,
+        default=DEFAULT_LANGUAGE,
+    )
+
+
+@router.post("/language", response_model=SetLanguageResponse)
+async def set_language(
+    body: SetLanguageRequest,
+    request: Request,
+    response: Response,
+    db: Session = Depends(get_db),
+) -> SetLanguageResponse:
+    """Set the preferred UI language.
+
+    Persists the choice in:
+    1. The server-side session
+    2. A ``docuelevate_lang`` cookie (30-day expiry)
+    3. The ``UserProfile.preferred_language`` column (if authenticated)
+    """
+    lang = body.language.lower().strip()
+    if lang not in SUPPORTED_LANGUAGE_CODES:
+        lang = DEFAULT_LANGUAGE
+
+    # 1. Session
+    if hasattr(request, "session"):
+        request.session["preferred_language"] = lang
+
+    # 2. Cookie (30 days)
+    response.set_cookie(
+        key="docuelevate_lang",
+        value=lang,
+        max_age=30 * 24 * 60 * 60,
+        httponly=False,
+        samesite="lax",
+    )
+
+    # 3. Database (if user is authenticated)
+    _persist_language_to_profile(request, db, lang)
+
+    language_name = next(
+        (entry["native"] for entry in SUPPORTED_LANGUAGES if entry["code"] == lang),
+        lang,
+    )
+    logger.info("Language preference set to '%s'", lang)
+    return SetLanguageResponse(
+        language=lang,
+        message=f"Language changed to {language_name}",
+    )
+
+
+def _persist_language_to_profile(request: Request, db: Session, lang: str) -> None:
+    """Write language preference to the UserProfile row, if the user is logged in."""
+    user_id: str | None = None
+    if hasattr(request, "session"):
+        user = request.session.get("user")
+        if isinstance(user, dict):
+            user_id = user.get("preferred_username") or user.get("email") or user.get("id")
+        elif isinstance(user, str):
+            user_id = user
+
+    if not user_id:
+        return
+
+    try:
+        profile = db.query(UserProfile).filter(UserProfile.user_id == user_id).first()
+        if profile:
+            profile.preferred_language = lang  # type: ignore[attr-defined]
+            db.commit()
+    except Exception:
+        db.rollback()
+        logger.debug("Could not persist language preference for user_id=%s", user_id)

app/api/imap_accounts.py

@@ -0,0 +1,371 @@
+"""API endpoints for managing per-user IMAP ingestion accounts.
+
+Provides CRUD operations for a user's IMAP accounts, quota enforcement
+against their subscription plan's ``max_mailboxes`` limit, and a
+test-connection endpoint so users can verify credentials before saving.
+"""
+
+import imaplib
+import logging
+from datetime import datetime, timezone
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import UserImapAccount
+from app.utils.encryption import decrypt_value, encrypt_value
+from app.utils.subscription import get_tier, get_user_tier_id
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/imap-accounts", tags=["imap-accounts"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Auth helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if owner_id is None:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+
+# ---------------------------------------------------------------------------
+# Quota helpers
+# ---------------------------------------------------------------------------
+
+_FREE_TIER_ID = "free"
+
+
+def _get_max_mailboxes(tier: dict[str, Any]) -> int | None:
+    """Return the maximum number of IMAP accounts allowed by *tier*.
+
+    Returns:
+        ``None``  — unlimited (paid tiers with ``max_mailboxes == 0``)
+        ``0``     — no mailboxes allowed (free tier)
+        positive  — the configured limit
+    """
+    tier_id: str = tier.get("id", _FREE_TIER_ID)
+    max_mb: int = tier.get("max_mailboxes", 0)
+
+    # Free tier: 0 means "no access" (not "unlimited")
+    if tier_id == _FREE_TIER_ID:
+        return 0
+
+    # Paid tiers: 0 means unlimited
+    if max_mb == 0:
+        return None
+
+    return max_mb
+
+
+def _check_quota(db: Session, owner_id: str) -> None:
+    """Raise 403 if the user has reached their IMAP account quota."""
+    tier_id = get_user_tier_id(db, owner_id)
+    tier = get_tier(tier_id, db)
+    max_mb = _get_max_mailboxes(tier)
+
+    if max_mb == 0:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail=("Your current plan does not include email ingestion. Upgrade to a paid plan to add IMAP accounts."),
+        )
+
+    if max_mb is not None:
+        current_count = db.query(UserImapAccount).filter(UserImapAccount.owner_id == owner_id).count()
+        if current_count >= max_mb:
+            raise HTTPException(
+                status_code=status.HTTP_403_FORBIDDEN,
+                detail=(
+                    f"You have reached your plan limit of {max_mb} IMAP account(s). "
+                    "Please delete an existing account or upgrade your plan."
+                ),
+            )
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class ImapAccountCreate(BaseModel):
+    """Schema for creating a new IMAP account."""
+
+    name: str = Field(..., min_length=1, max_length=255, description="Human-readable label")
+    host: str = Field(..., min_length=1, max_length=255, description="IMAP server hostname")
+    port: int = Field(default=993, ge=1, le=65535, description="IMAP server port")
+    username: str = Field(..., min_length=1, max_length=255, description="IMAP login username")
+    password: str = Field(..., min_length=1, max_length=1024, description="IMAP login password")
+    use_ssl: bool = Field(default=True, description="Use SSL/TLS connection")
+    delete_after_process: bool = Field(default=False, description="Delete emails from mailbox after processing")
+    is_active: bool = Field(default=True, description="Whether to poll this mailbox")
+    profile_id: int | None = Field(
+        default=None,
+        description=(
+            "ID of the ImapIngestionProfile that controls which attachment types to ingest. "
+            "Null inherits the global imap_attachment_filter setting."
+        ),
+    )
+
+
+class ImapAccountUpdate(BaseModel):
+    """Schema for updating an existing IMAP account (all fields optional)."""
+
+    name: str | None = Field(default=None, min_length=1, max_length=255)
+    host: str | None = Field(default=None, min_length=1, max_length=255)
+    port: int | None = Field(default=None, ge=1, le=65535)
+    username: str | None = Field(default=None, min_length=1, max_length=255)
+    password: str | None = Field(default=None, min_length=1, max_length=1024)
+    use_ssl: bool | None = None
+    delete_after_process: bool | None = None
+    is_active: bool | None = None
+    profile_id: int | None = Field(
+        default=None,
+        description=(
+            "ID of the ImapIngestionProfile to use.  "
+            "Explicitly sending null clears the override (falls back to global setting)."
+        ),
+    )
+
+
+class ImapTestRequest(BaseModel):
+    """Schema for testing an IMAP connection without saving it."""
+
+    host: str = Field(..., min_length=1, max_length=255)
+    port: int = Field(default=993, ge=1, le=65535)
+    username: str = Field(..., min_length=1, max_length=255)
+    password: str = Field(..., min_length=1, max_length=1024)
+    use_ssl: bool = Field(default=True)
+
+
+# ---------------------------------------------------------------------------
+# Serialisation helpers
+# ---------------------------------------------------------------------------
+
+
+def _to_response(acct: UserImapAccount) -> dict[str, Any]:
+    """Serialize a ``UserImapAccount`` row to a response dict.
+
+    Passwords are never included in responses.
+    """
+    return {
+        "id": acct.id,
+        "owner_id": acct.owner_id,
+        "name": acct.name,
+        "host": acct.host,
+        "port": acct.port,
+        "username": acct.username,
+        "use_ssl": acct.use_ssl,
+        "delete_after_process": acct.delete_after_process,
+        "is_active": acct.is_active,
+        "profile_id": acct.profile_id,
+        "last_checked_at": acct.last_checked_at.isoformat() if acct.last_checked_at else None,
+        "last_error": acct.last_error,
+        "created_at": acct.created_at.isoformat() if acct.created_at else None,
+        "updated_at": acct.updated_at.isoformat() if acct.updated_at else None,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Connection test helper
+# ---------------------------------------------------------------------------
+
+
+def _test_imap_connection(host: str, port: int, username: str, password: str, use_ssl: bool) -> dict[str, Any]:
+    """Attempt to connect and log in to the IMAP server.
+
+    Returns a dict with ``{"success": bool, "message": str}``.
+    """
+    try:
+        if use_ssl:
+            mail = imaplib.IMAP4_SSL(host, port)
+        else:
+            mail = imaplib.IMAP4(host, port)
+
+        mail.login(username, password)
+        mail.logout()
+        return {"success": True, "message": "Connection successful"}
+    except OSError as exc:
+        logger.warning("IMAP network error for %s@%s: %s", username, host, exc)
+        return {"success": False, "message": f"Connection error: {exc}"}
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("IMAP error for %s@%s: %s", username, host, exc)
+        return {"success": False, "message": f"IMAP error: {exc}"}
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/", summary="List IMAP accounts for the current user")
+def list_imap_accounts(request: Request, db: DbSession, owner_id: CurrentOwner) -> list[dict[str, Any]]:
+    """Return all IMAP accounts belonging to the authenticated user."""
+    accounts = db.query(UserImapAccount).filter(UserImapAccount.owner_id == owner_id).order_by(UserImapAccount.id).all()
+    return [_to_response(a) for a in accounts]
+
+
+@router.post("/", status_code=status.HTTP_201_CREATED, summary="Create a new IMAP account")
+def create_imap_account(
+    request: Request, body: ImapAccountCreate, db: DbSession, owner_id: CurrentOwner
+) -> dict[str, Any]:
+    """Create a new IMAP ingestion account for the current user.
+
+    Quota is enforced against the user's subscription plan's ``max_mailboxes``
+    limit before the account is persisted.
+    """
+    _check_quota(db, owner_id)
+
+    acct = UserImapAccount(
+        owner_id=owner_id,
+        name=body.name,
+        host=body.host,
+        port=body.port,
+        username=body.username,
+        password=encrypt_value(body.password),
+        use_ssl=body.use_ssl,
+        delete_after_process=body.delete_after_process,
+        is_active=body.is_active,
+        profile_id=body.profile_id,
+    )
+    try:
+        db.add(acct)
+        db.commit()
+        db.refresh(acct)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s created IMAP account %d (%s)", owner_id, acct.id, body.host)
+    return _to_response(acct)
+
+
+@router.get("/{account_id}", summary="Get a single IMAP account")
+def get_imap_account(account_id: int, request: Request, db: DbSession, owner_id: CurrentOwner) -> dict[str, Any]:
+    """Return a single IMAP account by ID (must belong to the current user)."""
+    acct = (
+        db.query(UserImapAccount).filter(UserImapAccount.id == account_id, UserImapAccount.owner_id == owner_id).first()
+    )
+    if not acct:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="IMAP account not found")
+    return _to_response(acct)
+
+
+@router.put("/{account_id}", summary="Update an IMAP account")
+def update_imap_account(
+    account_id: int,
+    request: Request,
+    body: ImapAccountUpdate,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Update an existing IMAP account.  Only provided fields are changed."""
+    acct = (
+        db.query(UserImapAccount).filter(UserImapAccount.id == account_id, UserImapAccount.owner_id == owner_id).first()
+    )
+    if not acct:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="IMAP account not found")
+
+    if body.name is not None:
+        acct.name = body.name
+    if body.host is not None:
+        acct.host = body.host
+    if body.port is not None:
+        acct.port = body.port
+    if body.username is not None:
+        acct.username = body.username
+    if body.password is not None:
+        acct.password = encrypt_value(body.password)
+    if body.use_ssl is not None:
+        acct.use_ssl = body.use_ssl
+    if body.delete_after_process is not None:
+        acct.delete_after_process = body.delete_after_process
+    if body.is_active is not None:
+        acct.is_active = body.is_active
+    # profile_id: update whenever the field is explicitly present in the request payload
+    # (including sending null to clear the override).
+    if "profile_id" in body.model_fields_set:
+        acct.profile_id = body.profile_id
+
+    # Reset last_error so the next poll gives a fresh result
+    acct.last_error = None
+    acct.updated_at = datetime.now(timezone.utc)
+
+    try:
+        db.commit()
+        db.refresh(acct)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s updated IMAP account %d", owner_id, account_id)
+    return _to_response(acct)
+
+
+@router.delete("/{account_id}", status_code=status.HTTP_204_NO_CONTENT, summary="Delete an IMAP account")
+def delete_imap_account(account_id: int, request: Request, db: DbSession, owner_id: CurrentOwner) -> None:
+    """Delete an IMAP account permanently."""
+    acct = (
+        db.query(UserImapAccount).filter(UserImapAccount.id == account_id, UserImapAccount.owner_id == owner_id).first()
+    )
+    if not acct:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="IMAP account not found")
+
+    try:
+        db.delete(acct)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s deleted IMAP account %d", owner_id, account_id)
+
+
+@router.post("/{account_id}/test", summary="Test an existing IMAP account's connection")
+def test_saved_imap_account(account_id: int, request: Request, db: DbSession, owner_id: CurrentOwner) -> dict[str, Any]:
+    """Test the connection for an already-saved IMAP account."""
+    acct = (
+        db.query(UserImapAccount).filter(UserImapAccount.id == account_id, UserImapAccount.owner_id == owner_id).first()
+    )
+    if not acct:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="IMAP account not found")
+
+    return _test_imap_connection(acct.host, acct.port, acct.username, decrypt_value(acct.password), acct.use_ssl)
+
+
+@router.post("/test", summary="Test an IMAP connection without saving")
+def test_imap_connection(request: Request, body: ImapTestRequest, owner_id: CurrentOwner) -> dict[str, Any]:
+    """Test IMAP credentials without persisting anything.
+
+    Useful for the "Test connection" button in the UI before the user saves
+    a new account.
+    """
+    return _test_imap_connection(body.host, body.port, body.username, body.password, body.use_ssl)
+
+
+@router.get("/quota/", summary="Get IMAP account quota information for the current user")
+def get_imap_quota(request: Request, db: DbSession, owner_id: CurrentOwner) -> dict[str, Any]:
+    """Return the user's current IMAP account usage vs. their plan quota."""
+    tier_id = get_user_tier_id(db, owner_id)
+    tier = get_tier(tier_id, db)
+    max_mb = _get_max_mailboxes(tier)
+    current_count = db.query(UserImapAccount).filter(UserImapAccount.owner_id == owner_id).count()
+
+    return {
+        "current_count": current_count,
+        "max_mailboxes": max_mb,  # None = unlimited, 0 = not allowed
+        "can_add": max_mb is None or (max_mb > 0 and current_count < max_mb),
+        "tier_id": tier_id,
+        "tier_name": tier.get("name", tier_id),
+    }

app/api/imap_profiles.py

@@ -0,0 +1,257 @@
+"""API endpoints for managing IMAP ingestion profiles.
+
+Ingestion profiles allow fine-grained control over which attachment types are
+accepted when ingesting emails via IMAP.  Each profile carries a list of enabled
+file-type categories (e.g. ``["pdf", "office", "images"]``) drawn from the
+canonical set defined in :mod:`app.utils.allowed_types`.
+
+Built-in system profiles (``is_builtin=True``) are read-only and cannot be
+deleted or modified.  Users may create their own profiles which are private to
+their ``owner_id``.  System-level global profiles (``owner_id=None``) are visible
+to all users but can only be created by administrators.
+"""
+
+import json
+import logging
+from datetime import datetime, timezone
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import ImapIngestionProfile
+from app.utils.allowed_types import FILE_TYPE_CATEGORIES
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/imap-profiles", tags=["imap-profiles"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Auth helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if owner_id is None:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+_VALID_CATEGORIES = set(FILE_TYPE_CATEGORIES.keys())
+
+
+class ImapProfileCreate(BaseModel):
+    """Schema for creating a new ingestion profile."""
+
+    name: str = Field(..., min_length=1, max_length=255, description="Human-readable profile name")
+    description: str | None = Field(default=None, description="Optional description")
+    allowed_categories: list[str] = Field(
+        ...,
+        min_length=1,
+        description=(f"List of enabled file-type category keys.  Valid values: {sorted(_VALID_CATEGORIES)}"),
+    )
+
+
+class ImapProfileUpdate(BaseModel):
+    """Schema for updating an existing profile (all fields optional)."""
+
+    name: str | None = Field(default=None, min_length=1, max_length=255)
+    description: str | None = None
+    allowed_categories: list[str] | None = Field(default=None, min_length=1)
+
+
+# ---------------------------------------------------------------------------
+# Validation helpers
+# ---------------------------------------------------------------------------
+
+
+def _validate_categories(categories: list[str]) -> list[str]:
+    """Raise 422 if any category key is unknown; return the cleaned list."""
+    unknown = [c for c in categories if c not in _VALID_CATEGORIES]
+    if unknown:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"Unknown category key(s): {unknown}. Valid keys: {sorted(_VALID_CATEGORIES)}",
+        )
+    # Deduplicate while preserving order
+    seen: set[str] = set()
+    result: list[str] = []
+    for cat in categories:
+        if cat not in seen:
+            seen.add(cat)
+            result.append(cat)
+    return result
+
+
+# ---------------------------------------------------------------------------
+# Serialisation
+# ---------------------------------------------------------------------------
+
+
+def _to_response(profile: ImapIngestionProfile) -> dict[str, Any]:
+    """Serialize a profile row to a response dict."""
+    try:
+        categories = json.loads(profile.allowed_categories)
+    except (ValueError, TypeError):
+        categories = []
+
+    # Enrich categories with display metadata
+    categories_detail = [
+        {
+            "key": cat,
+            "label": FILE_TYPE_CATEGORIES[cat]["label"] if cat in FILE_TYPE_CATEGORIES else cat,
+            "description": FILE_TYPE_CATEGORIES[cat]["description"] if cat in FILE_TYPE_CATEGORIES else "",
+        }
+        for cat in categories
+    ]
+
+    return {
+        "id": profile.id,
+        "name": profile.name,
+        "description": profile.description,
+        "owner_id": profile.owner_id,
+        "allowed_categories": categories,
+        "categories_detail": categories_detail,
+        "is_builtin": profile.is_builtin,
+        "created_at": profile.created_at.isoformat() if profile.created_at else None,
+        "updated_at": profile.updated_at.isoformat() if profile.updated_at else None,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/categories", summary="List available file-type categories")
+def list_categories(request: Request, owner_id: CurrentOwner) -> list[dict[str, Any]]:
+    """Return the full list of file-type categories that can be used in profiles."""
+    return [
+        {
+            "key": key,
+            "label": info["label"],
+            "description": info["description"],
+        }
+        for key, info in FILE_TYPE_CATEGORIES.items()
+    ]
+
+
+@router.get("/", summary="List ingestion profiles visible to the current user")
+def list_profiles(request: Request, db: DbSession, owner_id: CurrentOwner) -> list[dict[str, Any]]:
+    """Return all profiles: system-global (owner_id=NULL) and the user's own profiles."""
+    profiles = (
+        db.query(ImapIngestionProfile)
+        .filter(
+            # SQLAlchemy requires `== None` for IS NULL comparison in ORM filters
+            (ImapIngestionProfile.owner_id == None) | (ImapIngestionProfile.owner_id == owner_id)  # noqa: E711
+        )
+        .order_by(ImapIngestionProfile.is_builtin.desc(), ImapIngestionProfile.id)
+        .all()
+    )
+    return [_to_response(p) for p in profiles]
+
+
+@router.post("/", status_code=status.HTTP_201_CREATED, summary="Create a new ingestion profile")
+def create_profile(request: Request, body: ImapProfileCreate, db: DbSession, owner_id: CurrentOwner) -> dict[str, Any]:
+    """Create a new ingestion profile owned by the current user."""
+    categories = _validate_categories(body.allowed_categories)
+
+    profile = ImapIngestionProfile(
+        name=body.name,
+        description=body.description,
+        owner_id=owner_id,
+        allowed_categories=json.dumps(categories),
+        is_builtin=False,
+    )
+    try:
+        db.add(profile)
+        db.commit()
+        db.refresh(profile)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s created IMAP ingestion profile %d ('%s')", owner_id, profile.id, body.name)
+    return _to_response(profile)
+
+
+@router.get("/{profile_id}", summary="Get a single ingestion profile")
+def get_profile(profile_id: int, request: Request, db: DbSession, owner_id: CurrentOwner) -> dict[str, Any]:
+    """Return a single profile by ID.  Only the owner or system profiles are accessible."""
+    profile = db.query(ImapIngestionProfile).filter(ImapIngestionProfile.id == profile_id).first()
+    if not profile or (profile.owner_id is not None and profile.owner_id != owner_id):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Ingestion profile not found")
+    return _to_response(profile)
+
+
+@router.put("/{profile_id}", summary="Update an ingestion profile")
+def update_profile(
+    profile_id: int,
+    request: Request,
+    body: ImapProfileUpdate,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Update an existing ingestion profile.  Built-in profiles cannot be modified."""
+    profile = db.query(ImapIngestionProfile).filter(ImapIngestionProfile.id == profile_id).first()
+    if not profile or (profile.owner_id is not None and profile.owner_id != owner_id):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Ingestion profile not found")
+    if profile.is_builtin:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Built-in profiles cannot be modified.",
+        )
+
+    if body.name is not None:
+        profile.name = body.name
+    if "description" in body.model_fields_set:
+        profile.description = body.description
+    if body.allowed_categories is not None:
+        categories = _validate_categories(body.allowed_categories)
+        profile.allowed_categories = json.dumps(categories)
+
+    profile.updated_at = datetime.now(timezone.utc)
+
+    try:
+        db.commit()
+        db.refresh(profile)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s updated IMAP ingestion profile %d", owner_id, profile_id)
+    return _to_response(profile)
+
+
+@router.delete("/{profile_id}", status_code=status.HTTP_204_NO_CONTENT, summary="Delete an ingestion profile")
+def delete_profile(profile_id: int, request: Request, db: DbSession, owner_id: CurrentOwner) -> None:
+    """Delete an ingestion profile.  Built-in profiles cannot be deleted."""
+    profile = db.query(ImapIngestionProfile).filter(ImapIngestionProfile.id == profile_id).first()
+    if not profile or (profile.owner_id is not None and profile.owner_id != owner_id):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Ingestion profile not found")
+    if profile.is_builtin:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Built-in profiles cannot be deleted.",
+        )
+
+    try:
+        db.delete(profile)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s deleted IMAP ingestion profile %d", owner_id, profile_id)

app/api/integrations.py

@@ -0,0 +1,683 @@
+"""API endpoints for managing per-user integrations (sources and destinations).
+
+Provides CRUD operations for :class:`~app.models.UserIntegration` records.
+Each record represents one ingestion source (e.g. IMAP, Watch Folder) or
+storage destination (e.g. S3, Dropbox, Google Drive) configured by a user.
+
+Sensitive credentials are encrypted at rest using Fernet symmetric encryption
+(keyed from ``SESSION_SECRET``) via :mod:`app.utils.encryption`.  Credential
+values are **never** returned in API responses.
+
+Subscription quota enforcement
+------------------------------
+On creation, the endpoint checks the user's subscription tier limits:
+
+* **Destinations** — ``max_storage_destinations`` from the plan.
+* **Sources (IMAP)** — ``max_mailboxes`` from the plan.
+
+Exceeding the quota returns HTTP 403 with an actionable error message.
+"""
+
+import json
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import IntegrationDirection, IntegrationType, UserIntegration
+from app.utils.encryption import decrypt_value, encrypt_value
+from app.utils.subscription import get_tier, get_user_tier_id
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/integrations", tags=["integrations"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Auth helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if owner_id is None:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+# ---------------------------------------------------------------------------
+# Quota helpers
+# ---------------------------------------------------------------------------
+
+_FREE_TIER_ID = "free"
+
+# Source types that consume the mailbox quota
+_MAILBOX_SOURCE_TYPES = {IntegrationType.IMAP}
+
+
+def _get_max_destinations(tier: dict[str, Any]) -> int | None:
+    """Return the maximum number of storage destinations allowed by *tier*.
+
+    Returns:
+        ``None``  — unlimited (paid tiers with ``max_storage_destinations == 0``)
+        positive  — the configured limit
+    """
+    tier_id: str = tier.get("id", _FREE_TIER_ID)
+    max_dest: int = tier.get("max_storage_destinations", 0)
+
+    # Free tier: the value itself is the limit (e.g. 1)
+    if tier_id == _FREE_TIER_ID:
+        return max_dest if max_dest > 0 else 1  # safe default
+
+    # Paid tiers: 0 means unlimited
+    if max_dest == 0:
+        return None
+
+    return max_dest
+
+
+def _get_max_sources(tier: dict[str, Any]) -> int | None:
+    """Return the maximum number of IMAP source integrations allowed by *tier*.
+
+    Returns:
+        ``None``  — unlimited (paid tiers with ``max_mailboxes == 0``)
+        ``0``     — no mailboxes allowed (free tier)
+        positive  — the configured limit
+    """
+    tier_id: str = tier.get("id", _FREE_TIER_ID)
+    max_mb: int = tier.get("max_mailboxes", 0)
+
+    # Free tier: 0 means "no access" (not "unlimited")
+    if tier_id == _FREE_TIER_ID:
+        return 0
+
+    # Paid tiers: 0 means unlimited
+    if max_mb == 0:
+        return None
+
+    return max_mb
+
+
+def _check_quota(db: Session, owner_id: str, direction: str, integration_type: str) -> None:
+    """Raise 403 if the user has reached their integration quota.
+
+    Quota rules:
+    * DESTINATION integrations are limited by ``max_storage_destinations``.
+    * SOURCE integrations of type IMAP are limited by ``max_mailboxes``.
+    * Other SOURCE types (WATCH_FOLDER, WEBHOOK) are not quota-limited yet.
+    """
+    tier_id = get_user_tier_id(db, owner_id)
+    tier = get_tier(tier_id, db)
+
+    if direction == IntegrationDirection.DESTINATION:
+        max_dest = _get_max_destinations(tier)
+        if max_dest is not None:
+            current_count = (
+                db.query(UserIntegration)
+                .filter(
+                    UserIntegration.owner_id == owner_id,
+                    UserIntegration.direction == IntegrationDirection.DESTINATION,
+                )
+                .count()
+            )
+            if current_count >= max_dest:
+                raise HTTPException(
+                    status_code=status.HTTP_403_FORBIDDEN,
+                    detail=(
+                        f"You have reached your plan limit of {max_dest} storage destination(s). "
+                        "Please remove an existing destination or upgrade your plan."
+                    ),
+                )
+
+    elif direction == IntegrationDirection.SOURCE and integration_type in _MAILBOX_SOURCE_TYPES:
+        max_src = _get_max_sources(tier)
+
+        if max_src == 0:
+            raise HTTPException(
+                status_code=status.HTTP_403_FORBIDDEN,
+                detail="Your current plan does not include email ingestion. Upgrade to a paid plan to add IMAP sources.",
+            )
+
+        if max_src is not None:
+            current_count = (
+                db.query(UserIntegration)
+                .filter(
+                    UserIntegration.owner_id == owner_id,
+                    UserIntegration.direction == IntegrationDirection.SOURCE,
+                    UserIntegration.integration_type.in_(list(_MAILBOX_SOURCE_TYPES)),
+                )
+                .count()
+            )
+            if current_count >= max_src:
+                raise HTTPException(
+                    status_code=status.HTTP_403_FORBIDDEN,
+                    detail=(
+                        f"You have reached your plan limit of {max_src} IMAP source(s). "
+                        "Please remove an existing source or upgrade your plan."
+                    ),
+                )
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+_VALID_DIRECTIONS = IntegrationDirection.ALL
+_VALID_TYPES = IntegrationType.ALL
+
+
+class IntegrationCreate(BaseModel):
+    """Schema for creating a new integration."""
+
+    direction: str = Field(..., description="'SOURCE' or 'DESTINATION'")
+    integration_type: str = Field(..., description="Integration type (e.g. 'IMAP', 'S3', 'DROPBOX')")
+    name: str = Field(..., min_length=1, max_length=255, description="Human-readable label")
+    config: dict[str, Any] | None = Field(default=None, description="Non-sensitive configuration (JSON object)")
+    credentials: dict[str, Any] | None = Field(
+        default=None, description="Sensitive credentials (JSON object, encrypted at rest)"
+    )
+    is_active: bool = Field(default=True, description="Whether the integration is active")
+
+
+class IntegrationUpdate(BaseModel):
+    """Schema for updating an existing integration (all fields optional)."""
+
+    name: str | None = Field(default=None, min_length=1, max_length=255)
+    config: dict[str, Any] | None = None
+    credentials: dict[str, Any] | None = None
+    is_active: bool | None = None
+
+
+class IntegrationTestRequest(BaseModel):
+    """Schema for testing an integration connection without saving it."""
+
+    integration_type: str = Field(..., description="Integration type (e.g. 'IMAP', 'S3', 'DROPBOX')")
+    config: dict[str, Any] | None = Field(default=None, description="Non-sensitive configuration")
+    credentials: dict[str, Any] | None = Field(default=None, description="Credentials for the connection test")
+
+
+# ---------------------------------------------------------------------------
+# Validation helpers
+# ---------------------------------------------------------------------------
+
+
+def _validate_direction(direction: str) -> None:
+    """Raise 400 if *direction* is not a known value."""
+    if direction not in _VALID_DIRECTIONS:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Invalid direction '{direction}'. Must be one of: {sorted(_VALID_DIRECTIONS)}",
+        )
+
+
+def _validate_integration_type(integration_type: str) -> None:
+    """Raise 400 if *integration_type* is not a known value."""
+    if integration_type not in _VALID_TYPES:
+        raise HTTPException(
+            status_code=status.HTTP_400_BAD_REQUEST,
+            detail=f"Invalid integration_type '{integration_type}'. Must be one of: {sorted(_VALID_TYPES)}",
+        )
+
+
+# ---------------------------------------------------------------------------
+# Serialisation helpers
+# ---------------------------------------------------------------------------
+
+
+def _to_response(integration: UserIntegration) -> dict[str, Any]:
+    """Serialise a :class:`UserIntegration` row to a response dict.
+
+    Credentials are **never** included; only a boolean flag indicating
+    whether credentials have been configured is returned.
+    """
+    config_data: dict[str, Any] | None = None
+    if integration.config:
+        try:
+            config_data = json.loads(integration.config)
+        except (json.JSONDecodeError, TypeError):
+            config_data = None
+
+    return {
+        "id": integration.id,
+        "owner_id": integration.owner_id,
+        "direction": integration.direction,
+        "integration_type": integration.integration_type,
+        "name": integration.name,
+        "config": config_data,
+        "has_credentials": bool(integration.credentials),
+        "is_active": integration.is_active,
+        "last_used_at": integration.last_used_at.isoformat() if integration.last_used_at else None,
+        "last_error": integration.last_error,
+        "created_at": integration.created_at.isoformat() if integration.created_at else None,
+        "updated_at": integration.updated_at.isoformat() if integration.updated_at else None,
+    }
+
+
+def _encode_credentials(credentials: dict[str, Any] | None) -> str | None:
+    """Serialise *credentials* dict to an encrypted JSON string for storage."""
+    if not credentials:
+        return None
+    plaintext = json.dumps(credentials)
+    return encrypt_value(plaintext)
+
+
+def _decode_credentials(stored: str | None) -> dict[str, Any] | None:
+    """Decrypt and deserialise stored credentials back to a dict.
+
+    Returns ``None`` when *stored* is empty or cannot be decoded.
+    """
+    if not stored:
+        return None
+    plaintext = decrypt_value(stored)
+    if not plaintext:
+        return None
+    try:
+        return json.loads(plaintext)
+    except (json.JSONDecodeError, TypeError):
+        logger.error("Failed to decode credentials JSON after decryption")
+        return None
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/", summary="List integrations for the current user")
+def list_integrations(
+    request: Request,
+    db: DbSession,
+    owner_id: CurrentOwner,
+    direction: str | None = None,
+    integration_type: str | None = None,
+) -> list[dict[str, Any]]:
+    """Return all integrations belonging to the authenticated user.
+
+    Optional query-string filters:
+
+    - ``direction`` — ``SOURCE`` or ``DESTINATION``
+    - ``integration_type`` — e.g. ``IMAP``, ``S3``, ``DROPBOX``
+    """
+    query = db.query(UserIntegration).filter(UserIntegration.owner_id == owner_id)
+
+    if direction is not None:
+        _validate_direction(direction)
+        query = query.filter(UserIntegration.direction == direction)
+
+    if integration_type is not None:
+        _validate_integration_type(integration_type)
+        query = query.filter(UserIntegration.integration_type == integration_type)
+
+    integrations = query.order_by(UserIntegration.id).all()
+    return [_to_response(i) for i in integrations]
+
+
+@router.post("/", status_code=status.HTTP_201_CREATED, summary="Create a new integration")
+def create_integration(
+    request: Request,
+    body: IntegrationCreate,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Create a new source or destination integration for the current user.
+
+    ``credentials`` are encrypted at rest using Fernet symmetric encryption
+    before being persisted and are **never** returned in API responses.
+
+    Quota is enforced against the user's subscription plan before the
+    integration is persisted.
+    """
+    _validate_direction(body.direction)
+    _validate_integration_type(body.integration_type)
+
+    _check_quota(db, owner_id, body.direction, body.integration_type)
+
+    integration = UserIntegration(
+        owner_id=owner_id,
+        direction=body.direction,
+        integration_type=body.integration_type,
+        name=body.name,
+        config=json.dumps(body.config) if body.config is not None else None,
+        credentials=_encode_credentials(body.credentials),
+        is_active=body.is_active,
+    )
+
+    try:
+        db.add(integration)
+        db.commit()
+        db.refresh(integration)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info(
+        "User %s created %s integration %d (%s)",
+        owner_id,
+        body.direction,
+        integration.id,
+        body.integration_type,
+    )
+    return _to_response(integration)
+
+
+@router.get("/{integration_id}", summary="Get a single integration")
+def get_integration(
+    integration_id: int,
+    request: Request,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Return a single integration by ID (must belong to the current user)."""
+    integration = (
+        db.query(UserIntegration)
+        .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+        .first()
+    )
+    if not integration:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Integration not found")
+    return _to_response(integration)
+
+
+@router.put("/{integration_id}", summary="Update an integration")
+def update_integration(
+    integration_id: int,
+    request: Request,
+    body: IntegrationUpdate,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Update an existing integration.  Only provided fields are changed.
+
+    When ``credentials`` is supplied the stored value is replaced in full
+    with the freshly encrypted version of the new credentials dict.
+    """
+    integration = (
+        db.query(UserIntegration)
+        .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+        .first()
+    )
+    if not integration:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Integration not found")
+
+    if body.name is not None:
+        integration.name = body.name
+    if body.config is not None:
+        integration.config = json.dumps(body.config)
+    if body.credentials is not None:
+        integration.credentials = _encode_credentials(body.credentials)
+    if body.is_active is not None:
+        integration.is_active = body.is_active
+
+    # Reset last_error so the next operation gives a fresh result
+    integration.last_error = None
+
+    try:
+        db.commit()
+        db.refresh(integration)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s updated integration %d", owner_id, integration_id)
+    return _to_response(integration)
+
+
+@router.delete("/{integration_id}", status_code=status.HTTP_204_NO_CONTENT, summary="Delete an integration")
+def delete_integration(
+    integration_id: int,
+    request: Request,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> None:
+    """Delete an integration permanently."""
+    integration = (
+        db.query(UserIntegration)
+        .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+        .first()
+    )
+    if not integration:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Integration not found")
+
+    try:
+        db.delete(integration)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("User %s deleted integration %d", owner_id, integration_id)
+
+
+@router.get("/{integration_id}/credentials", summary="Retrieve decrypted credentials for an integration")
+def get_integration_credentials(
+    integration_id: int,
+    request: Request,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Return the decrypted credentials dict for a saved integration.
+
+    This endpoint is intended for internal use by background tasks that need
+    to authenticate with a third-party service.  Treat the response as
+    sensitive — it contains plaintext secrets.
+    """
+    integration = (
+        db.query(UserIntegration)
+        .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+        .first()
+    )
+    if not integration:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Integration not found")
+
+    credentials = _decode_credentials(integration.credentials)
+    return {"credentials": credentials or {}}
+
+
+# ---------------------------------------------------------------------------
+# Connection test helpers
+# ---------------------------------------------------------------------------
+
+
+def _test_imap_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
+    """Test an IMAP connection using the provided config and credentials."""
+    import imaplib
+
+    cfg = config or {}
+    creds = credentials or {}
+    host = cfg.get("host", "")
+    port = int(cfg.get("port", 993))
+    username = cfg.get("username", "")
+    password = creds.get("password", "")
+    use_ssl = cfg.get("use_ssl", True)
+
+    if not host or not username or not password:
+        return {"success": False, "message": "Missing required fields: host, username, and password"}
+
+    try:
+        if use_ssl:
+            mail = imaplib.IMAP4_SSL(host, port)
+        else:
+            mail = imaplib.IMAP4(host, port)
+        mail.login(username, password)
+        mail.logout()
+        return {"success": True, "message": "IMAP connection successful"}
+    except OSError as exc:
+        logger.warning("IMAP network error for %s@%s: %s", username, host, exc)
+        return {"success": False, "message": "IMAP connection failed — check host, port, and network connectivity"}
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("IMAP error for %s@%s: %s", username, host, exc)
+        return {"success": False, "message": "IMAP authentication or connection failed"}
+
+
+def _test_s3_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
+    """Test an S3 connection by calling HeadBucket."""
+    try:
+        import boto3
+        from botocore.exceptions import BotoCoreError, ClientError
+    except ImportError:
+        return {"success": False, "message": "boto3 is not installed"}
+
+    cfg = config or {}
+    creds = credentials or {}
+    bucket = cfg.get("bucket", "")
+    region = cfg.get("region", "us-east-1")
+
+    if not bucket:
+        return {"success": False, "message": "Missing required field: bucket"}
+
+    try:
+        client = boto3.client(
+            "s3",
+            region_name=region,
+            aws_access_key_id=creds.get("access_key_id", ""),
+            aws_secret_access_key=creds.get("secret_access_key", ""),
+            endpoint_url=cfg.get("endpoint_url"),
+        )
+        client.head_bucket(Bucket=bucket)
+        return {"success": True, "message": f"S3 bucket '{bucket}' is accessible"}
+    except (BotoCoreError, ClientError) as exc:
+        logger.warning("S3 connection error for bucket '%s': %s", bucket, exc)
+        return {"success": False, "message": "S3 connection failed — check bucket name, region, and credentials"}
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("S3 unexpected error for bucket '%s': %s", bucket, exc)
+        return {"success": False, "message": "S3 connection failed"}
+
+
+def _test_webdav_connection(config: dict[str, Any] | None, credentials: dict[str, Any] | None) -> dict[str, Any]:
+    """Test a WebDAV/Nextcloud connection by issuing an HTTP PROPFIND."""
+    import urllib.request
+
+    cfg = config or {}
+    creds = credentials or {}
+    url = cfg.get("url", "")
+    username = creds.get("username", "")
+    password = creds.get("password", "")
+
+    if not url:
+        return {"success": False, "message": "Missing required field: url"}
+
+    # Only allow http/https to prevent file:// or other custom scheme attacks
+    from urllib.parse import urlparse
+
+    parsed = urlparse(url)
+    if parsed.scheme not in ("http", "https"):
+        return {"success": False, "message": "URL must use http or https scheme"}
+
+    # Block requests to private/internal IPs to prevent SSRF
+    hostname = parsed.hostname or ""
+    if hostname:
+        from app.utils.network import is_private_ip
+
+        if is_private_ip(hostname):
+            return {"success": False, "message": "URLs pointing to internal or private networks are not allowed"}
+
+    try:
+        import base64
+
+        req = urllib.request.Request(url, method="PROPFIND")  # noqa: S310
+        if username and password:
+            token = base64.b64encode(f"{username}:{password}".encode()).decode()
+            req.add_header("Authorization", f"Basic {token}")
+        req.add_header("Depth", "0")
+        with urllib.request.urlopen(req, timeout=10) as resp:  # noqa: S310
+            if resp.status < 400:
+                return {"success": True, "message": "WebDAV connection successful"}
+            return {"success": False, "message": f"WebDAV returned HTTP {resp.status}"}
+    except Exception as exc:  # noqa: BLE001
+        logger.warning("WebDAV connection error for %s: %s", hostname, exc)
+        return {"success": False, "message": "WebDAV connection failed — check URL and credentials"}
+
+
+_CONNECTION_TESTERS: dict[str, Any] = {
+    IntegrationType.IMAP: _test_imap_connection,
+    IntegrationType.S3: _test_s3_connection,
+    IntegrationType.WEBDAV: _test_webdav_connection,
+    IntegrationType.NEXTCLOUD: _test_webdav_connection,
+}
+
+
+# ---------------------------------------------------------------------------
+# Test & quota endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post("/test", summary="Test an integration connection without saving")
+def test_integration_connection(
+    request: Request,
+    body: IntegrationTestRequest,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Test integration credentials without persisting anything.
+
+    Useful for the "Test connection" button in the UI before the user saves
+    a new integration.  Returns ``{"success": bool, "message": str}``.
+    """
+    _validate_integration_type(body.integration_type)
+
+    tester = _CONNECTION_TESTERS.get(body.integration_type)
+    if tester is None:
+        return {
+            "success": False,
+            "message": f"Connection testing is not yet supported for '{body.integration_type}'. "
+            "The integration can still be saved and will be validated on first use.",
+        }
+
+    return tester(body.config, body.credentials)
+
+
+@router.get("/quota/", summary="Get integration quota information for the current user")
+def get_integration_quota(
+    request: Request,
+    db: DbSession,
+    owner_id: CurrentOwner,
+) -> dict[str, Any]:
+    """Return the user's current integration usage vs. their plan quota.
+
+    Includes separate counts for destinations and IMAP sources.
+    """
+    tier_id = get_user_tier_id(db, owner_id)
+    tier = get_tier(tier_id, db)
+
+    max_dest = _get_max_destinations(tier)
+    max_src = _get_max_sources(tier)
+
+    dest_count = (
+        db.query(UserIntegration)
+        .filter(
+            UserIntegration.owner_id == owner_id,
+            UserIntegration.direction == IntegrationDirection.DESTINATION,
+        )
+        .count()
+    )
+
+    src_count = (
+        db.query(UserIntegration)
+        .filter(
+            UserIntegration.owner_id == owner_id,
+            UserIntegration.direction == IntegrationDirection.SOURCE,
+            UserIntegration.integration_type.in_(list(_MAILBOX_SOURCE_TYPES)),
+        )
+        .count()
+    )
+
+    return {
+        "tier_id": tier_id,
+        "tier_name": tier.get("name", tier_id),
+        "destinations": {
+            "current_count": dest_count,
+            "max_allowed": max_dest,
+            "can_add": max_dest is None or dest_count < max_dest,
+        },
+        "sources": {
+            "current_count": src_count,
+            "max_allowed": max_src,
+            "can_add": max_src is None or (max_src > 0 and src_count < max_src),
+        },
+    }

app/api/local_auth.py

@@ -26,6 +26,7 @@ from starlette.responses import RedirectResponse
 from app.config import settings
 from app.database import get_db
 from app.models import LocalUser, UserProfile
+from app.utils.i18n import translate as _translate
 from app.utils.local_auth import (
     build_session_user,
     generate_token,
@@ -41,6 +42,7 @@ router = APIRouter(tags=["local-auth"])
 
 _templates_dir = pathlib.Path(__file__).parents[2] / "frontend" / "templates"
 templates = Jinja2Templates(directory=str(_templates_dir))
+templates.env.globals["_"] = lambda key, **kwargs: _translate(key, "en", **kwargs)
 
 DbSession = Annotated[Session, Depends(get_db)]
 

app/api/mobile.py

@@ -0,0 +1,347 @@
+"""Mobile app API endpoints.
+
+Provides endpoints specifically designed for the DocuElevate native mobile
+app (iOS / Android via React Native / Expo):
+
+* ``POST /mobile/generate-token`` – exchange an active session for a
+  long-lived API token that the mobile app stores securely.  The token is
+  auto-named "Mobile App – <device_name>" and is identical to regular API
+  tokens (Bearer auth works everywhere).
+
+* ``POST /mobile/register-device`` – register a push-notification device
+  token (Expo push token) so the user receives push notifications when
+  documents finish processing.
+
+* ``GET /mobile/devices`` – list registered devices for the current user.
+
+* ``DELETE /mobile/devices/{device_id}`` – deactivate a device.
+
+* ``GET /mobile/whoami`` – lightweight profile endpoint for the mobile app
+  to verify authentication state.
+"""
+
+import logging
+from datetime import datetime, timezone
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.api.api_tokens import generate_api_token, hash_token
+from app.auth import require_login
+from app.database import get_db
+from app.models import ApiToken, MobileDevice
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/mobile", tags=["mobile"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+
+# ---------------------------------------------------------------------------
+# Auth helper
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if not owner_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+
+# ---------------------------------------------------------------------------
+# Request / Response schemas
+# ---------------------------------------------------------------------------
+
+
+class GenerateTokenRequest(BaseModel):
+    """Request body for auto-generating a mobile app token."""
+
+    device_name: str = Field(
+        default="Mobile App",
+        min_length=1,
+        max_length=120,
+        description="Human-readable device name used to label the token.",
+    )
+
+
+class GenerateTokenResponse(BaseModel):
+    """Response containing the one-time-visible API token."""
+
+    token: str
+    token_id: int
+    name: str
+    created_at: datetime
+
+
+class RegisterDeviceRequest(BaseModel):
+    """Request body for registering a push-notification device token."""
+
+    push_token: str = Field(
+        min_length=1,
+        max_length=512,
+        description="Expo push token (ExponentPushToken[…]) obtained from the mobile app.",
+    )
+    device_name: str | None = Field(
+        default=None,
+        max_length=255,
+        description="Optional human-readable device name (e.g. 'John's iPhone').",
+    )
+    platform: str = Field(
+        default="ios",
+        description="Device platform: 'ios', 'android', or 'web'.",
+    )
+
+
+class DeviceResponse(BaseModel):
+    """Serialised MobileDevice record."""
+
+    id: int
+    device_name: str | None
+    platform: str
+    push_token_preview: str
+    is_active: bool
+    created_at: datetime
+    last_seen_at: datetime | None
+
+
+class WhoAmIResponse(BaseModel):
+    """Lightweight profile response for the mobile app."""
+
+    owner_id: str
+    display_name: str | None
+    email: str | None
+    avatar_url: str | None
+    is_admin: bool
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _device_to_response(device: MobileDevice) -> dict[str, Any]:
+    """Convert a MobileDevice ORM object to a serialisable dict."""
+    # Show only first 20 chars of the push token for security.
+    token_preview = device.push_token[:20] + "…" if len(device.push_token) > 20 else device.push_token
+    return {
+        "id": device.id,
+        "device_name": device.device_name,
+        "platform": device.platform,
+        "push_token_preview": token_preview,
+        "is_active": device.is_active,
+        "created_at": device.created_at,
+        "last_seen_at": device.last_seen_at,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post("/generate-token", status_code=status.HTTP_201_CREATED, response_model=GenerateTokenResponse)
+@require_login
+async def generate_mobile_token(
+    request: Request,
+    body: GenerateTokenRequest,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Generate a long-lived API token for the mobile app.
+
+    The mobile app calls this endpoint immediately after SSO login to obtain
+    a Bearer token it can store in the secure keychain.  The returned token
+    is functionally identical to manually-created API tokens and works with
+    every authenticated endpoint.
+
+    The token is shown **exactly once** in the response; subsequent requests
+    show only the prefix for identification.
+    """
+    token_name = f"Mobile App – {body.device_name}"
+    plaintext = generate_api_token()
+    token_hash_value = hash_token(plaintext)
+    prefix = plaintext[:12]
+
+    db_token = ApiToken(
+        owner_id=owner_id,
+        name=token_name,
+        token_hash=token_hash_value,
+        token_prefix=prefix,
+    )
+    try:
+        db.add(db_token)
+        db.commit()
+        db.refresh(db_token)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create mobile API token for owner_id=%s", owner_id)
+        raise
+
+    logger.info("Mobile API token created: id=%s owner=%s device=%r", db_token.id, owner_id, body.device_name)
+
+    return {
+        "token": plaintext,
+        "token_id": db_token.id,
+        "name": token_name,
+        "created_at": db_token.created_at,
+    }
+
+
+@router.post("/register-device", status_code=status.HTTP_201_CREATED, response_model=DeviceResponse)
+@require_login
+async def register_device(
+    request: Request,
+    body: RegisterDeviceRequest,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Register or refresh a push-notification device token.
+
+    If the same ``push_token`` is already registered for this user the
+    record is reactivated and ``last_seen_at`` is updated rather than
+    creating a duplicate.
+    """
+    platform = body.platform.lower()
+    if platform not in {"ios", "android", "web"}:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="platform must be one of: ios, android, web",
+        )
+
+    now = datetime.now(timezone.utc)
+
+    # Upsert: reuse existing record if the token is already known.
+    existing = (
+        db.query(MobileDevice)
+        .filter(MobileDevice.owner_id == owner_id, MobileDevice.push_token == body.push_token)
+        .first()
+    )
+    if existing:
+        existing.is_active = True
+        existing.last_seen_at = now
+        if body.device_name:
+            existing.device_name = body.device_name
+        try:
+            db.commit()
+            db.refresh(existing)
+        except Exception:
+            db.rollback()
+            raise
+        logger.info("Mobile device refreshed: id=%s owner=%s", existing.id, owner_id)
+        return _device_to_response(existing)
+
+    device = MobileDevice(
+        owner_id=owner_id,
+        device_name=body.device_name,
+        platform=platform,
+        push_token=body.push_token,
+        is_active=True,
+        last_seen_at=now,
+    )
+    try:
+        db.add(device)
+        db.commit()
+        db.refresh(device)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to register mobile device for owner_id=%s", owner_id)
+        raise
+
+    logger.info("Mobile device registered: id=%s owner=%s platform=%s", device.id, owner_id, platform)
+    return _device_to_response(device)
+
+
+@router.get("/devices", response_model=list[DeviceResponse])
+@require_login
+async def list_devices(
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> list[dict[str, Any]]:
+    """List all registered push-notification devices for the current user."""
+    devices = (
+        db.query(MobileDevice).filter(MobileDevice.owner_id == owner_id).order_by(MobileDevice.created_at.desc()).all()
+    )
+    return [_device_to_response(d) for d in devices]
+
+
+@router.delete("/devices/{device_id}", status_code=status.HTTP_204_NO_CONTENT)
+@require_login
+async def deactivate_device(
+    request: Request,
+    device_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> None:
+    """Deactivate a push-notification device registration.
+
+    The device record is kept for audit purposes but will no longer receive
+    push notifications.
+    """
+    device = db.get(MobileDevice, device_id)
+    if not device or device.owner_id != owner_id:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Device not found")
+
+    device.is_active = False
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Mobile device deactivated: id=%s owner=%s", device_id, owner_id)
+
+
+@router.get("/whoami", response_model=WhoAmIResponse)
+@require_login
+async def whoami(
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Return basic profile information for the authenticated user.
+
+    The mobile app calls this after token exchange to populate the user
+    profile screen and verify that the stored token is still valid.
+    """
+    from app.auth import get_gravatar_url
+    from app.models import LocalUser, UserProfile
+
+    profile = db.query(UserProfile).filter(UserProfile.user_id == owner_id).first()
+    local_user = db.query(LocalUser).filter(LocalUser.email == owner_id).first()
+
+    display_name: str | None = None
+    email: str | None = None
+    avatar_url: str | None = None
+    is_admin = False
+
+    if profile:
+        display_name = profile.display_name
+
+    if local_user:
+        email = local_user.email
+        is_admin = bool(local_user.is_admin)
+        if not display_name and local_user.display_name:
+            display_name = local_user.display_name
+    elif "@" in owner_id:
+        # SSO users commonly have their email as owner_id
+        email = owner_id
+
+    if email:
+        avatar_url = get_gravatar_url(email)
+
+    return {
+        "owner_id": owner_id,
+        "display_name": display_name,
+        "email": email,
+        "avatar_url": avatar_url,
+        "is_admin": is_admin,
+    }

app/api/notifications.py

@@ -0,0 +1,484 @@
+"""API endpoints for per-user notification targets, preferences, and in-app inbox.
+
+Users can define notification targets (email via SMTP, webhook via HTTP POST)
+and configure which document events trigger which targets.  In-app notifications
+are always created and surfaced via the bell icon / inbox endpoints.
+"""
+
+import json
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import InAppNotification, UserNotificationPreference, UserNotificationTarget
+from app.utils.user_notification import USER_EVENT_LABELS
+from app.utils.user_scope import get_current_owner_id
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/user-notifications", tags=["user-notifications"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Auth helper (mirrors api_tokens.py pattern)
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if not owner_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+VALID_CHANNEL_TYPES = {"email", "webhook"}
+VALID_EVENT_TYPES = set(USER_EVENT_LABELS.keys())
+
+
+class NotificationTargetCreate(BaseModel):
+    """Schema for creating a new notification target."""
+
+    channel_type: str = Field(..., pattern="^(email|webhook)$")
+    name: str = Field(..., min_length=1, max_length=255)
+    config: dict[str, Any] = Field(default_factory=dict)
+    is_active: bool = True
+
+
+class NotificationTargetUpdate(BaseModel):
+    """Schema for updating an existing notification target."""
+
+    name: str | None = Field(None, min_length=1, max_length=255)
+    config: dict[str, Any] | None = None
+    is_active: bool | None = None
+
+
+class PreferenceItem(BaseModel):
+    """A single preference toggle for one event+channel combination."""
+
+    is_enabled: bool
+    target_id: int | None = None
+
+
+class PreferenceItemFull(BaseModel):
+    """Full preference item including event and channel type (used in bulk update)."""
+
+    event_type: str
+    channel_type: str
+    is_enabled: bool
+    target_id: int | None = None
+
+
+class PreferencesUpdate(BaseModel):
+    """Bulk preferences update payload — a flat list of preference items."""
+
+    preferences: list[PreferenceItemFull]
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _mask_email_config(config: dict[str, Any]) -> dict[str, Any]:
+    """Return a copy of an email config dict with the password masked."""
+    masked = dict(config)
+    if masked.get("smtp_password"):
+        masked["smtp_password"] = "****"
+    return masked
+
+
+def _target_to_dict(target: UserNotificationTarget) -> dict[str, Any]:
+    """Serialize a UserNotificationTarget to a response dict, masking secrets."""
+    config: dict[str, Any] = {}
+    if target.config:
+        try:
+            config = json.loads(target.config)
+        except (json.JSONDecodeError, ValueError):
+            config = {}
+
+    if target.channel_type == "email":
+        config = _mask_email_config(config)
+
+    return {
+        "id": target.id,
+        "channel_type": target.channel_type,
+        "name": target.name,
+        "config": config,
+        "is_active": target.is_active,
+        "created_at": target.created_at,
+        "updated_at": target.updated_at,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Inbox endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/inbox")
+async def list_inbox(
+    owner_id: CurrentOwner,
+    db: DbSession,
+    skip: int = 0,
+    limit: int = 50,
+) -> list[dict[str, Any]]:
+    """List in-app notifications for the authenticated user, newest first."""
+    notifications = (
+        db.query(InAppNotification)
+        .filter(InAppNotification.owner_id == owner_id)
+        .order_by(InAppNotification.created_at.desc())
+        .offset(skip)
+        .limit(limit)
+        .all()
+    )
+    return [
+        {
+            "id": n.id,
+            "event_type": n.event_type,
+            "title": n.title,
+            "message": n.message,
+            "is_read": n.is_read,
+            "file_id": n.file_id,
+            "created_at": n.created_at,
+        }
+        for n in notifications
+    ]
+
+
+@router.get("/inbox/unread-count")
+async def unread_count(
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, int]:
+    """Return the number of unread in-app notifications."""
+    count = (
+        db.query(InAppNotification)
+        .filter(InAppNotification.owner_id == owner_id, InAppNotification.is_read == False)  # noqa: E712
+        .count()
+    )
+    return {"count": count}
+
+
+@router.post("/inbox/{notification_id}/read", status_code=status.HTTP_200_OK)
+async def mark_read(
+    notification_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Mark a single in-app notification as read."""
+    notif = (
+        db.query(InAppNotification)
+        .filter(InAppNotification.id == notification_id, InAppNotification.owner_id == owner_id)
+        .first()
+    )
+    if not notif:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Notification not found")
+    try:
+        notif.is_read = True
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+    return {"detail": "Marked as read"}
+
+
+@router.post("/inbox/read-all", status_code=status.HTTP_200_OK)
+async def mark_all_read(
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Mark all in-app notifications as read for the authenticated user."""
+    try:
+        db.query(InAppNotification).filter(
+            InAppNotification.owner_id == owner_id,
+            InAppNotification.is_read == False,  # noqa: E712
+        ).update({"is_read": True})
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+    return {"detail": "All notifications marked as read"}
+
+
+# ---------------------------------------------------------------------------
+# Notification target endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/targets")
+async def list_targets(
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> list[dict[str, Any]]:
+    """List all notification targets for the authenticated user."""
+    targets = (
+        db.query(UserNotificationTarget)
+        .filter(UserNotificationTarget.owner_id == owner_id)
+        .order_by(UserNotificationTarget.created_at.desc())
+        .all()
+    )
+    return [_target_to_dict(t) for t in targets]
+
+
+@router.post("/targets", status_code=status.HTTP_201_CREATED)
+async def create_target(
+    body: NotificationTargetCreate,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Create a new notification target (email or webhook)."""
+    target = UserNotificationTarget(
+        owner_id=owner_id,
+        channel_type=body.channel_type,
+        name=body.name,
+        config=json.dumps(body.config),
+        is_active=body.is_active,
+    )
+    try:
+        db.add(target)
+        db.commit()
+        db.refresh(target)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Notification target created: id=%s owner=%s type=%s", target.id, owner_id, body.channel_type)
+    return _target_to_dict(target)
+
+
+@router.put("/targets/{target_id}", status_code=status.HTTP_200_OK)
+async def update_target(
+    target_id: int,
+    body: NotificationTargetUpdate,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Update an existing notification target."""
+    target = (
+        db.query(UserNotificationTarget)
+        .filter(UserNotificationTarget.id == target_id, UserNotificationTarget.owner_id == owner_id)
+        .first()
+    )
+    if not target:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Target not found")
+
+    try:
+        if body.name is not None:
+            target.name = body.name
+        if body.config is not None:
+            # Merge new config over existing, preserving masked password field if unchanged
+            existing_config: dict[str, Any] = {}
+            if target.config:
+                try:
+                    existing_config = json.loads(target.config)
+                except (json.JSONDecodeError, ValueError):
+                    existing_config = {}
+            merged = dict(existing_config)
+            for k, v in body.config.items():
+                # Skip writing back a masked password placeholder
+                if k == "smtp_password" and v == "****":
+                    continue
+                merged[k] = v
+            target.config = json.dumps(merged)
+        if body.is_active is not None:
+            target.is_active = body.is_active
+        db.commit()
+        db.refresh(target)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Notification target updated: id=%s owner=%s", target_id, owner_id)
+    return _target_to_dict(target)
+
+
+@router.delete("/targets/{target_id}", status_code=status.HTTP_200_OK)
+async def delete_target(
+    target_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Delete a notification target and its associated preferences."""
+    target = (
+        db.query(UserNotificationTarget)
+        .filter(UserNotificationTarget.id == target_id, UserNotificationTarget.owner_id == owner_id)
+        .first()
+    )
+    if not target:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Target not found")
+
+    try:
+        # Remove any preferences that reference this target
+        db.query(UserNotificationPreference).filter(
+            UserNotificationPreference.owner_id == owner_id,
+            UserNotificationPreference.target_id == target_id,
+        ).delete()
+        db.delete(target)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Notification target deleted: id=%s owner=%s", target_id, owner_id)
+    return {"detail": "Target deleted"}
+
+
+@router.post("/targets/{target_id}/test", status_code=status.HTTP_200_OK)
+async def test_target(
+    target_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Send a test notification to the specified target."""
+    target = (
+        db.query(UserNotificationTarget)
+        .filter(UserNotificationTarget.id == target_id, UserNotificationTarget.owner_id == owner_id)
+        .first()
+    )
+    if not target:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Target not found")
+
+    config: dict[str, Any] = {}
+    if target.config:
+        try:
+            config = json.loads(target.config)
+        except (json.JSONDecodeError, ValueError):
+            config = {}
+
+    title = "DocuElevate Test Notification"
+    message = f"This is a test notification from DocuElevate for target '{target.name}'."
+
+    if target.channel_type == "email":
+        from app.utils.user_notification import _send_email_notification
+
+        ok = _send_email_notification(config, title, message)
+    elif target.channel_type == "webhook":
+        from app.utils.user_notification import _send_webhook_notification
+
+        ok = _send_webhook_notification(config, "test", title, message)
+    else:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unknown channel type")
+
+    if not ok:
+        raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail="Failed to send test notification")
+
+    return {"detail": "Test notification sent"}
+
+
+# ---------------------------------------------------------------------------
+# Preferences endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("/preferences")
+async def get_preferences(
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Return all notification preferences for the authenticated user.
+
+    Response structure:
+        {
+          "event_types": ["document.processed", "document.failed"],
+          "event_labels": {"document.processed": "Document Processed", ...},
+          "preferences": {
+            "document.processed": {
+              "in_app": {"is_enabled": true, "target_id": null},
+              "email": {"is_enabled": false, "target_id": 1},
+              ...
+            }
+          }
+        }
+    """
+    prefs = db.query(UserNotificationPreference).filter(UserNotificationPreference.owner_id == owner_id).all()
+
+    # Build nested dict: event_type -> channel_type -> {is_enabled, target_id}
+    result: dict[str, dict[str, dict[str, Any]]] = {}
+    for pref in prefs:
+        result.setdefault(pref.event_type, {})[pref.channel_type] = {
+            "is_enabled": pref.is_enabled,
+            "target_id": pref.target_id,
+        }
+
+    return {
+        "event_types": list(USER_EVENT_LABELS.keys()),
+        "event_labels": USER_EVENT_LABELS,
+        "preferences": result,
+    }
+
+
+@router.put("/preferences", status_code=status.HTTP_200_OK)
+async def update_preferences(
+    body: PreferencesUpdate,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Bulk upsert notification preferences for the authenticated user.
+
+    Validates that any referenced target_id belongs to the requesting user.
+    """
+    # Collect all target IDs referenced in the payload for ownership validation
+    referenced_target_ids: set[int] = set()
+    for item in body.preferences:
+        if item.target_id is not None:
+            referenced_target_ids.add(item.target_id)
+
+    if referenced_target_ids:
+        owned_ids = {
+            row.id
+            for row in db.query(UserNotificationTarget.id)
+            .filter(
+                UserNotificationTarget.owner_id == owner_id,
+                UserNotificationTarget.id.in_(referenced_target_ids),
+            )
+            .all()
+        }
+        invalid = referenced_target_ids - owned_ids
+        if invalid:
+            raise HTTPException(
+                status_code=status.HTTP_400_BAD_REQUEST,
+                detail=f"Invalid or inaccessible target_id(s): {sorted(invalid)}",
+            )
+
+    try:
+        for item in body.preferences:
+            existing = (
+                db.query(UserNotificationPreference)
+                .filter(
+                    UserNotificationPreference.owner_id == owner_id,
+                    UserNotificationPreference.event_type == item.event_type,
+                    UserNotificationPreference.channel_type == item.channel_type,
+                    UserNotificationPreference.target_id == item.target_id,
+                )
+                .first()
+            )
+            if existing:
+                existing.is_enabled = item.is_enabled
+            else:
+                db.add(
+                    UserNotificationPreference(
+                        owner_id=owner_id,
+                        event_type=item.event_type,
+                        channel_type=item.channel_type,
+                        target_id=item.target_id,
+                        is_enabled=item.is_enabled,
+                    )
+                )
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Notification preferences updated for owner=%s", owner_id)
+    return {"detail": "Preferences updated"}

app/api/pipelines.py

@@ -51,7 +51,48 @@ PIPELINE_STEP_TYPES: dict[str, dict[str, Any]] = {
                 "type": "boolean",
                 "default": False,
                 "description": "Always use cloud OCR even if the PDF already has embedded text.",
-            }
+            },
+            "ocr_language": {
+                "type": "select",
+                "default": "auto",
+                "description": (
+                    "Language(s) used for OCR text extraction.  Applies to Tesseract and EasyOCR "
+                    "providers; Azure and Mistral perform auto-detection by default.  "
+                    "Use Tesseract codes such as 'eng', 'deu', or 'eng+deu' for multi-language "
+                    "documents.  'auto' falls back to the global system setting."
+                ),
+                "options": [
+                    {"value": "auto", "label": "Auto (use system default)"},
+                    {"value": "ara", "label": "Arabic"},
+                    {"value": "chi_sim", "label": "Chinese (Simplified)"},
+                    {"value": "chi_tra", "label": "Chinese (Traditional)"},
+                    {"value": "ces", "label": "Czech"},
+                    {"value": "dan", "label": "Danish"},
+                    {"value": "nld", "label": "Dutch"},
+                    {"value": "eng", "label": "English"},
+                    {"value": "fin", "label": "Finnish"},
+                    {"value": "fra", "label": "French"},
+                    {"value": "deu", "label": "German"},
+                    {"value": "ell", "label": "Greek"},
+                    {"value": "heb", "label": "Hebrew"},
+                    {"value": "hin", "label": "Hindi"},
+                    {"value": "hun", "label": "Hungarian"},
+                    {"value": "ita", "label": "Italian"},
+                    {"value": "jpn", "label": "Japanese"},
+                    {"value": "kor", "label": "Korean"},
+                    {"value": "nor", "label": "Norwegian"},
+                    {"value": "pol", "label": "Polish"},
+                    {"value": "por", "label": "Portuguese"},
+                    {"value": "ron", "label": "Romanian"},
+                    {"value": "rus", "label": "Russian"},
+                    {"value": "spa", "label": "Spanish"},
+                    {"value": "swe", "label": "Swedish"},
+                    {"value": "tha", "label": "Thai"},
+                    {"value": "tur", "label": "Turkish"},
+                    {"value": "ukr", "label": "Ukrainian"},
+                    {"value": "vie", "label": "Vietnamese"},
+                ],
+            },
         },
     },
     "extract_metadata": {
@@ -291,15 +332,15 @@ def create_pipeline(request: Request, db: DbSession, body: PipelineCreate) -> di
         db.add(pipeline)
         db.commit()
         db.refresh(pipeline)
-    except Exception as exc:
+    except Exception:
         db.rollback()
-        logger.exception(f"Failed to create pipeline user={user_id}: {exc}")
+        logger.exception("Failed to create pipeline user=%s", user_id)
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail="Failed to create pipeline",
         )
 
-    logger.info(f"Pipeline created: id={pipeline.id}, owner={user_id}, name={name!r}")
+    logger.info("Pipeline created: id=%s, owner=%s, name=%r", pipeline.id, user_id, name)
     return _serialize_pipeline(pipeline)
 
 
@@ -387,15 +428,15 @@ def update_pipeline(pipeline_id: int, request: Request, db: DbSession, body: Pip
     try:
         db.commit()
         db.refresh(pipeline)
-    except Exception as exc:
+    except Exception:
         db.rollback()
-        logger.exception(f"Failed to update pipeline id={pipeline_id}: {exc}")
+        logger.exception("Failed to update pipeline id=%s", pipeline_id)
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail="Failed to update pipeline",
         )
 
-    logger.info(f"Pipeline updated: id={pipeline_id}, user={user_id}")
+    logger.info("Pipeline updated: id=%s, user=%s", pipeline_id, user_id)
     return _serialize_pipeline(pipeline, include_steps=True, db=db)
 
 
@@ -425,15 +466,15 @@ def delete_pipeline(pipeline_id: int, request: Request, db: DbSession) -> None:
         db.query(PipelineStep).filter(PipelineStep.pipeline_id == pipeline_id).delete()
         db.delete(pipeline)
         db.commit()
-    except Exception as exc:
+    except Exception:
         db.rollback()
-        logger.exception(f"Failed to delete pipeline id={pipeline_id}: {exc}")
+        logger.exception("Failed to delete pipeline id=%s", pipeline_id)
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail="Failed to delete pipeline",
         )
 
-    logger.info(f"Pipeline deleted: id={pipeline_id}, user={user_id}")
+    logger.info("Pipeline deleted: id=%s, user=%s", pipeline_id, user_id)
 
 
 # ---------------------------------------------------------------------------

app/api/plans.py

@@ -75,6 +75,8 @@ class PlanUpsert(BaseModel):
     sort_order: int = 0
     features: list[str] = []
     api_access: bool = False
+    stripe_price_id_monthly: str | None = None
+    stripe_price_id_yearly: str | None = None
 
 
 class ReorderBody(BaseModel):
@@ -121,6 +123,8 @@ def _plan_to_response(plan: SubscriptionPlan) -> dict[str, Any]:
         "sort_order": plan.sort_order,
         "features": features,
         "api_access": plan.api_access,
+        "stripe_price_id_monthly": plan.stripe_price_id_monthly,
+        "stripe_price_id_yearly": plan.stripe_price_id_yearly,
         "created_at": plan.created_at.isoformat() if plan.created_at else None,
         "updated_at": plan.updated_at.isoformat() if plan.updated_at else None,
     }
@@ -151,6 +155,8 @@ def _apply_body(plan: SubscriptionPlan, body: PlanUpsert) -> None:
     plan.sort_order = body.sort_order
     plan.features = json.dumps(body.features)
     plan.api_access = body.api_access
+    plan.stripe_price_id_monthly = body.stripe_price_id_monthly or None
+    plan.stripe_price_id_yearly = body.stripe_price_id_yearly or None
 
 
 # ---------------------------------------------------------------------------

app/api/profile.py

@@ -0,0 +1,340 @@
+"""User self-service profile API.
+
+Provides endpoints for the authenticated user to view and update their own
+profile settings without requiring admin access.
+
+Routes:
+  GET    /api/profile          — read current user's profile
+  PATCH  /api/profile          — update display name, language, theme
+  POST   /api/profile/avatar   — upload a new profile picture (JPEG/PNG/GIF/WebP, max 2 MB)
+  DELETE /api/profile/avatar   — remove custom avatar (reverts to Gravatar)
+  POST   /api/profile/change-password — change password (local-auth users only)
+"""
+
+from __future__ import annotations
+
+import base64
+import logging
+from hashlib import md5
+from typing import Annotated
+
+from fastapi import APIRouter, Depends, File, HTTPException, Request, Response, UploadFile, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.database import get_db
+from app.models import LocalUser, UserProfile
+from app.utils.i18n import SUPPORTED_LANGUAGE_CODES
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/profile", tags=["profile"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# Maximum avatar upload size: 2 MB
+_MAX_AVATAR_BYTES = 2 * 1024 * 1024
+
+# Allowed MIME types for avatar uploads
+_ALLOWED_AVATAR_TYPES = {"image/jpeg", "image/png", "image/gif", "image/webp"}
+
+# Valid theme values
+_VALID_THEMES = {"light", "dark", "system"}
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_user_id(request: Request) -> str:
+    """Return the stable user identifier from the session.
+
+    Raises HTTP 401 if no user is logged in.
+    """
+    user = request.session.get("user")
+    if not user or not isinstance(user, dict):
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    uid = user.get("sub") or user.get("preferred_username") or user.get("email") or user.get("id")
+    if not uid:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Cannot determine user identity")
+    return uid
+
+
+def _gravatar_url(email: str | None) -> str:
+    """Generate a Gravatar URL for *email*, falling back to identicon."""
+    if not email:
+        return "https://www.gravatar.com/avatar/?d=identicon"
+    # MD5 used for Gravatar URL generation only — not for security
+    h = md5(email.strip().lower().encode(), usedforsecurity=False).hexdigest()
+    return f"https://www.gravatar.com/avatar/{h}?d=identicon"
+
+
+def _get_or_create_profile(db: Session, user_id: str) -> UserProfile:
+    """Return the UserProfile for *user_id*, creating a stub if one doesn't exist."""
+    profile = db.query(UserProfile).filter(UserProfile.user_id == user_id).first()
+    if profile is None:
+        profile = UserProfile(user_id=user_id)
+        db.add(profile)
+        try:
+            db.commit()
+            db.refresh(profile)
+        except Exception:
+            db.rollback()
+            raise
+    return profile
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class ProfileResponse(BaseModel):
+    """Response body for GET /api/profile."""
+
+    user_id: str
+    display_name: str | None
+    contact_email: str | None
+    preferred_language: str | None
+    preferred_theme: str | None
+    avatar_url: str
+    """Gravatar URL or ``data:`` URI for a custom uploaded avatar."""
+    is_local_user: bool
+    """True when the account was created via local email/password sign-up."""
+
+
+class ProfileUpdateRequest(BaseModel):
+    """Request body for PATCH /api/profile."""
+
+    display_name: str | None = Field(default=None, max_length=255, description="Human-readable display name")
+    contact_email: str | None = Field(default=None, max_length=255, description="Contact / notification e-mail")
+    preferred_language: str | None = Field(default=None, description="ISO 639-1 language code, e.g. 'en', 'de'")
+    preferred_theme: str | None = Field(default=None, description="Colour scheme: 'light', 'dark', or 'system'")
+
+
+class ChangePasswordRequest(BaseModel):
+    """Request body for POST /api/profile/change-password."""
+
+    current_password: str = Field(..., min_length=1, max_length=128)
+    new_password: str = Field(..., min_length=8, max_length=128)
+    new_password_confirm: str = Field(..., min_length=8, max_length=128)
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("", response_model=ProfileResponse)
+@require_login
+async def get_profile(request: Request, db: DbSession) -> ProfileResponse:
+    """Return the current user's profile settings."""
+    user_id = _get_user_id(request)
+    profile = _get_or_create_profile(db, user_id)
+
+    session_user = request.session.get("user", {})
+    email = session_user.get("email") if isinstance(session_user, dict) else None
+
+    # Determine avatar: prefer stored data, fall back to Gravatar
+    avatar_url = profile.avatar_data if profile.avatar_data else _gravatar_url(email)  # type: ignore[attr-defined]
+
+    # Check whether this is a local (email/password) account
+    is_local = db.query(LocalUser).filter(LocalUser.username == user_id).first() is not None
+
+    return ProfileResponse(
+        user_id=user_id,
+        display_name=profile.display_name,  # type: ignore[arg-type]
+        contact_email=profile.contact_email,  # type: ignore[arg-type]
+        preferred_language=profile.preferred_language,  # type: ignore[arg-type]
+        preferred_theme=profile.preferred_theme,  # type: ignore[arg-type]
+        avatar_url=avatar_url,
+        is_local_user=is_local,
+    )
+
+
+@router.patch("", response_model=ProfileResponse)
+@require_login
+async def update_profile(
+    body: ProfileUpdateRequest, request: Request, response: Response, db: DbSession
+) -> ProfileResponse:
+    """Update the current user's editable profile settings."""
+    user_id = _get_user_id(request)
+    profile = _get_or_create_profile(db, user_id)
+
+    # Validate language code
+    if body.preferred_language is not None:
+        lang = body.preferred_language.lower().strip()
+        if lang and lang not in SUPPORTED_LANGUAGE_CODES:
+            raise HTTPException(
+                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+                detail=f"Unsupported language code: {lang}",
+            )
+        profile.preferred_language = lang or None  # type: ignore[assignment]
+
+        # Keep session and cookie in sync so detect_language() picks up
+        # the new preference immediately (without a DB round-trip).
+        if hasattr(request, "session"):
+            if lang:
+                request.session["preferred_language"] = lang
+            else:
+                request.session.pop("preferred_language", None)
+        if lang:
+            response.set_cookie(
+                key="docuelevate_lang",
+                value=lang,
+                max_age=30 * 24 * 60 * 60,
+                httponly=False,
+                samesite="lax",
+            )
+        else:
+            response.delete_cookie(key="docuelevate_lang")
+
+    # Validate theme
+    if body.preferred_theme is not None:
+        theme = body.preferred_theme.lower().strip()
+        if theme and theme not in _VALID_THEMES:
+            raise HTTPException(
+                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+                detail=f"Invalid theme: {theme}. Must be one of: {', '.join(sorted(_VALID_THEMES))}",
+            )
+        profile.preferred_theme = theme or None  # type: ignore[assignment]
+
+    if body.display_name is not None:
+        profile.display_name = body.display_name.strip() or None  # type: ignore[assignment]
+
+    if body.contact_email is not None:
+        profile.contact_email = body.contact_email.strip() or None  # type: ignore[assignment]
+
+    try:
+        db.commit()
+        db.refresh(profile)
+    except Exception:
+        db.rollback()
+        raise
+
+    session_user = request.session.get("user", {})
+    email = session_user.get("email") if isinstance(session_user, dict) else None
+    avatar_url = profile.avatar_data if profile.avatar_data else _gravatar_url(email)  # type: ignore[attr-defined]
+    is_local = db.query(LocalUser).filter(LocalUser.username == user_id).first() is not None
+
+    return ProfileResponse(
+        user_id=user_id,
+        display_name=profile.display_name,  # type: ignore[arg-type]
+        contact_email=profile.contact_email,  # type: ignore[arg-type]
+        preferred_language=profile.preferred_language,  # type: ignore[arg-type]
+        preferred_theme=profile.preferred_theme,  # type: ignore[arg-type]
+        avatar_url=avatar_url,
+        is_local_user=is_local,
+    )
+
+
+@router.post("/avatar", status_code=status.HTTP_200_OK)
+@require_login
+async def upload_avatar(
+    request: Request,
+    db: DbSession,
+    file: UploadFile = File(..., description="Profile picture (JPEG, PNG, GIF or WebP; max 2 MB)"),
+) -> dict:
+    """Upload a new profile picture.
+
+    The image is stored as a base64-encoded data URL in ``UserProfile.avatar_data``.
+    Accepts JPEG, PNG, GIF, or WebP files up to 2 MB.
+    """
+    user_id = _get_user_id(request)
+
+    content_type = (file.content_type or "").lower()
+    if content_type not in _ALLOWED_AVATAR_TYPES:
+        raise HTTPException(
+            status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
+            detail=f"Unsupported image type '{content_type}'. Allowed: JPEG, PNG, GIF, WebP.",
+        )
+
+    # Check declared size first (available when the client sends a Content-Length header)
+    if file.size is not None and file.size > _MAX_AVATAR_BYTES:
+        raise HTTPException(
+            status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
+            detail="Avatar image must be 2 MB or smaller.",
+        )
+
+    # Read up to one byte past the limit so we can detect oversized uploads
+    raw = await file.read(_MAX_AVATAR_BYTES + 1)
+    if len(raw) > _MAX_AVATAR_BYTES:
+        raise HTTPException(
+            status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
+            detail="Avatar image must be 2 MB or smaller.",
+        )
+
+    b64 = base64.b64encode(raw).decode("ascii")
+    data_url = f"data:{content_type};base64,{b64}"
+
+    profile = _get_or_create_profile(db, user_id)
+    profile.avatar_data = data_url  # type: ignore[assignment]
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    return {"avatar_url": data_url}
+
+
+@router.delete("/avatar", status_code=status.HTTP_200_OK)
+@require_login
+async def delete_avatar(request: Request, db: DbSession) -> dict:
+    """Remove the custom avatar and revert to the Gravatar fallback."""
+    user_id = _get_user_id(request)
+    profile = _get_or_create_profile(db, user_id)
+    profile.avatar_data = None  # type: ignore[assignment]
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    session_user = request.session.get("user", {})
+    email = session_user.get("email") if isinstance(session_user, dict) else None
+    return {"avatar_url": _gravatar_url(email)}
+
+
+@router.post("/change-password", status_code=status.HTTP_200_OK)
+@require_login
+async def change_password(body: ChangePasswordRequest, request: Request, db: DbSession) -> dict:
+    """Change the password for local (email/password) accounts.
+
+    Raises 403 if the account is not a local account or the current password is wrong.
+    Raises 422 if the new passwords do not match.
+    """
+    from app.utils.local_auth import hash_password, verify_password
+
+    user_id = _get_user_id(request)
+
+    local_user = db.query(LocalUser).filter(LocalUser.username == user_id).first()
+    if local_user is None:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Password change is only available for local accounts.",
+        )
+
+    if not verify_password(body.current_password, local_user.hashed_password):
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail="Current password is incorrect.",
+        )
+
+    if body.new_password != body.new_password_confirm:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="New passwords do not match.",
+        )
+
+    local_user.hashed_password = hash_password(body.new_password)
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Password changed for local user: %s", user_id)
+    return {"detail": "Password changed successfully."}

app/api/routing_rules.py

@@ -0,0 +1,468 @@
+"""Routing rules API endpoints.
+
+Provides full CRUD for pipeline routing rules that conditionally assign
+documents to pipelines based on document properties (file type, category,
+metadata fields, size, etc.).
+
+Rules are evaluated in ascending ``position`` order.  The first rule whose
+condition matches wins and routes the document to the specified target
+pipeline.  If no rule matches, the caller falls back to the owner's (or
+system) default pipeline.
+"""
+
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Body, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.auth import require_login
+from app.database import get_db
+from app.models import Pipeline, PipelineRoutingRule
+from app.utils.routing_engine import (
+    BUILTIN_FIELDS,
+    VALID_OPERATORS,
+    _evaluate_condition,
+    _resolve_field,
+)
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/routing-rules", tags=["routing-rules"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+MAX_RULES_PER_OWNER = 100
+MAX_NAME_LENGTH = 255
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _get_user_id(request: Request) -> str:
+    """Return the authenticated user identifier."""
+    user = getattr(request.state, "user", None)
+    if user:
+        if isinstance(user, dict):
+            return user.get("sub", user.get("email", "anonymous"))
+        return getattr(user, "sub", getattr(user, "email", "anonymous"))
+    return "anonymous"
+
+
+def _is_admin(request: Request) -> bool:
+    """Return ``True`` when the current user has admin privileges."""
+    user = getattr(request.state, "user", None)
+    if not user:
+        return False
+    groups = user.get("groups", []) if isinstance(user, dict) else getattr(user, "groups", [])
+    return "admin" in groups
+
+
+def _can_access_rule(rule: PipelineRoutingRule, user_id: str, admin: bool) -> bool:
+    """Check whether the user is allowed to read this rule."""
+    if admin:
+        return True
+    return rule.owner_id == user_id
+
+
+def _can_write_rule(rule: PipelineRoutingRule, user_id: str, admin: bool) -> bool:
+    """Check whether the user is allowed to modify this rule."""
+    if rule.owner_id is None:
+        return admin
+    return rule.owner_id == user_id
+
+
+def _validate_field(field: str) -> None:
+    """Raise 422 if the field name is invalid."""
+    if field in BUILTIN_FIELDS:
+        return
+    if field.startswith("metadata.") and len(field) > len("metadata."):
+        return
+    raise HTTPException(
+        status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+        detail=(
+            f"Invalid field '{field}'. "
+            f"Valid built-in fields: {sorted(BUILTIN_FIELDS)}. "
+            "For AI metadata, use 'metadata.<key>'."
+        ),
+    )
+
+
+def _validate_operator(operator: str) -> None:
+    """Raise 422 if the operator is not recognised."""
+    if operator not in VALID_OPERATORS:
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail=f"Invalid operator '{operator}'. Valid operators: {sorted(VALID_OPERATORS)}",
+        )
+
+
+def _serialize_rule(rule: PipelineRoutingRule) -> dict[str, Any]:
+    """Serialize a routing rule to a JSON-compatible dict."""
+    return {
+        "id": rule.id,
+        "owner_id": rule.owner_id,
+        "name": rule.name,
+        "position": rule.position,
+        "field": rule.field,
+        "operator": rule.operator,
+        "value": rule.value,
+        "target_pipeline_id": rule.target_pipeline_id,
+        "is_active": rule.is_active,
+        "created_at": rule.created_at.isoformat() if rule.created_at else None,
+        "updated_at": rule.updated_at.isoformat() if rule.updated_at else None,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Pydantic request models
+# ---------------------------------------------------------------------------
+
+
+class RoutingRuleCreate(BaseModel):
+    """Request body for creating a routing rule."""
+
+    name: str = Field(..., min_length=1, max_length=MAX_NAME_LENGTH)
+    field: str = Field(..., min_length=1, max_length=255)
+    operator: str = Field(..., min_length=1, max_length=50)
+    value: str = Field(..., max_length=1024)
+    target_pipeline_id: int
+    position: int | None = None
+    is_active: bool = True
+
+
+class RoutingRuleUpdate(BaseModel):
+    """Request body for updating a routing rule."""
+
+    name: str | None = Field(None, min_length=1, max_length=MAX_NAME_LENGTH)
+    field: str | None = Field(None, min_length=1, max_length=255)
+    operator: str | None = Field(None, min_length=1, max_length=50)
+    value: str | None = Field(None, max_length=1024)
+    target_pipeline_id: int | None = None
+    position: int | None = None
+    is_active: bool | None = None
+
+
+class RoutingRuleEvaluateRequest(BaseModel):
+    """Request body for dry-run rule evaluation."""
+
+    file_type: str | None = None
+    filename: str | None = None
+    size: int | None = None
+    document_type: str | None = None
+    metadata: dict[str, Any] | None = None
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("")
+@require_login
+def list_routing_rules(request: Request, db: DbSession) -> list[dict[str, Any]]:
+    """List all routing rules accessible by the current user.
+
+    Returns the user's own rules plus any system-wide rules (``owner_id=NULL``).
+    Rules are sorted by position.
+    """
+    user_id = _get_user_id(request)
+
+    rules = (
+        db.query(PipelineRoutingRule)
+        .filter((PipelineRoutingRule.owner_id == user_id) | (PipelineRoutingRule.owner_id.is_(None)))
+        .order_by(
+            PipelineRoutingRule.owner_id.is_(None).asc(),
+            PipelineRoutingRule.position.asc(),
+        )
+        .all()
+    )
+
+    return [_serialize_rule(r) for r in rules]
+
+
+@router.post("", status_code=status.HTTP_201_CREATED)
+@require_login
+def create_routing_rule(request: Request, db: DbSession, body: RoutingRuleCreate) -> dict[str, Any]:
+    """Create a new routing rule for the current user.
+
+    Returns:
+        The created routing rule.
+
+    Raises:
+        HTTPException 422: If the field or operator is invalid.
+        HTTPException 404: If the target pipeline does not exist.
+        HTTPException 409: If the maximum number of rules is reached.
+    """
+    user_id = _get_user_id(request)
+
+    _validate_field(body.field)
+    _validate_operator(body.operator)
+
+    # Verify target pipeline exists and is accessible.
+    pipeline = db.query(Pipeline).filter(Pipeline.id == body.target_pipeline_id).first()
+    if not pipeline:
+        raise HTTPException(
+            status_code=status.HTTP_404_NOT_FOUND,
+            detail=f"Target pipeline {body.target_pipeline_id} not found",
+        )
+
+    # Enforce per-owner limit.
+    count = db.query(PipelineRoutingRule).filter(PipelineRoutingRule.owner_id == user_id).count()
+    if count >= MAX_RULES_PER_OWNER:
+        raise HTTPException(
+            status_code=status.HTTP_409_CONFLICT,
+            detail=f"Maximum of {MAX_RULES_PER_OWNER} routing rules per user reached",
+        )
+
+    # Auto-assign position if not specified.
+    position = body.position
+    if position is None:
+        max_pos = (
+            db.query(PipelineRoutingRule.position)
+            .filter(PipelineRoutingRule.owner_id == user_id)
+            .order_by(PipelineRoutingRule.position.desc())
+            .first()
+        )
+        position = (max_pos[0] + 1) if max_pos else 0
+
+    rule = PipelineRoutingRule(
+        owner_id=user_id,
+        name=body.name.strip(),
+        position=position,
+        field=body.field,
+        operator=body.operator,
+        value=body.value,
+        target_pipeline_id=body.target_pipeline_id,
+        is_active=body.is_active,
+    )
+
+    try:
+        db.add(rule)
+        db.commit()
+        db.refresh(rule)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create routing rule for user=%s", user_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to create routing rule",
+        )
+
+    logger.info("Routing rule created: id=%s, user=%s", rule.id, user_id)
+    return _serialize_rule(rule)
+
+
+@router.get("/operators")
+def list_operators() -> dict[str, Any]:
+    """Return the list of supported operators and fields.
+
+    This is a public endpoint (no auth required) so that UIs can populate
+    dropdowns without hard-coding the catalogue.
+    """
+    return {
+        "operators": sorted(VALID_OPERATORS),
+        "builtin_fields": sorted(BUILTIN_FIELDS),
+        "metadata_prefix": "metadata.",
+    }
+
+
+@router.post("/evaluate")
+@require_login
+def evaluate_rules(request: Request, db: DbSession, body: RoutingRuleEvaluateRequest) -> dict[str, Any]:
+    """Dry-run rule evaluation against the provided document properties.
+
+    Returns the first matching rule and target pipeline (if any), or
+    indicates that no rule matched (default pipeline will be used).
+    """
+    user_id = _get_user_id(request)
+
+    doc_props: dict[str, Any] = {
+        "file_type": body.file_type,
+        "filename": body.filename,
+        "size": body.size,
+        "document_type": body.document_type,
+        "metadata": body.metadata or {},
+    }
+
+    rules = (
+        db.query(PipelineRoutingRule)
+        .filter(
+            PipelineRoutingRule.is_active.is_(True),
+            (PipelineRoutingRule.owner_id == user_id) | (PipelineRoutingRule.owner_id.is_(None)),
+        )
+        .order_by(
+            PipelineRoutingRule.owner_id.is_(None).asc(),
+            PipelineRoutingRule.position.asc(),
+        )
+        .all()
+    )
+
+    for rule in rules:
+        actual = _resolve_field(rule.field, doc_props)
+        if _evaluate_condition(actual, rule.operator, rule.value):
+            pipeline = db.query(Pipeline).filter(Pipeline.id == rule.target_pipeline_id).first()
+            return {
+                "matched": True,
+                "rule": _serialize_rule(rule),
+                "target_pipeline": {
+                    "id": pipeline.id,
+                    "name": pipeline.name,
+                    "is_active": pipeline.is_active,
+                }
+                if pipeline
+                else None,
+            }
+
+    return {"matched": False, "rule": None, "target_pipeline": None}
+
+
+@router.put("/reorder")
+@require_login
+def reorder_routing_rules(
+    request: Request,
+    db: DbSession,
+    rule_ids: list[int] = Body(..., embed=True),
+) -> list[dict[str, Any]]:
+    """Reorder the caller's routing rules.
+
+    Expects a JSON body ``{"rule_ids": [3, 1, 2]}`` where the list
+    contains the IDs of the caller's rules in the desired order.
+    """
+    user_id = _get_user_id(request)
+
+    rules = (
+        db.query(PipelineRoutingRule)
+        .filter(PipelineRoutingRule.owner_id == user_id, PipelineRoutingRule.id.in_(rule_ids))
+        .all()
+    )
+
+    rule_map = {r.id: r for r in rules}
+
+    if len(rule_map) != len(rule_ids) or set(rule_map.keys()) != set(rule_ids):
+        raise HTTPException(
+            status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+            detail="rule_ids must contain exactly the IDs of your routing rules",
+        )
+
+    for pos, rid in enumerate(rule_ids):
+        rule_map[rid].position = pos
+
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to reorder routing rules for user=%s", user_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to reorder routing rules",
+        )
+
+    ordered = sorted(rules, key=lambda r: r.position)
+    return [_serialize_rule(r) for r in ordered]
+
+
+@router.get("/{rule_id}")
+@require_login
+def get_routing_rule(rule_id: int, request: Request, db: DbSession) -> dict[str, Any]:
+    """Return a single routing rule by ID."""
+    user_id = _get_user_id(request)
+    admin = _is_admin(request)
+
+    rule = db.query(PipelineRoutingRule).filter(PipelineRoutingRule.id == rule_id).first()
+    if not rule or not _can_access_rule(rule, user_id, admin):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Routing rule not found")
+
+    return _serialize_rule(rule)
+
+
+@router.put("/{rule_id}")
+@require_login
+def update_routing_rule(rule_id: int, request: Request, db: DbSession, body: RoutingRuleUpdate) -> dict[str, Any]:
+    """Update a routing rule.
+
+    Only the fields present in the request body are updated.
+    """
+    user_id = _get_user_id(request)
+    admin = _is_admin(request)
+
+    rule = db.query(PipelineRoutingRule).filter(PipelineRoutingRule.id == rule_id).first()
+    if not rule or not _can_access_rule(rule, user_id, admin):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Routing rule not found")
+
+    if not _can_write_rule(rule, user_id, admin):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Cannot modify this rule")
+
+    if body.field is not None:
+        _validate_field(body.field)
+        rule.field = body.field
+
+    if body.operator is not None:
+        _validate_operator(body.operator)
+        rule.operator = body.operator
+
+    if body.value is not None:
+        rule.value = body.value
+
+    if body.name is not None:
+        rule.name = body.name.strip()
+
+    if body.target_pipeline_id is not None:
+        pipeline = db.query(Pipeline).filter(Pipeline.id == body.target_pipeline_id).first()
+        if not pipeline:
+            raise HTTPException(
+                status_code=status.HTTP_404_NOT_FOUND,
+                detail=f"Target pipeline {body.target_pipeline_id} not found",
+            )
+        rule.target_pipeline_id = body.target_pipeline_id
+
+    if body.position is not None:
+        rule.position = body.position
+
+    if body.is_active is not None:
+        rule.is_active = body.is_active
+
+    try:
+        db.commit()
+        db.refresh(rule)
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to update routing rule id=%s", rule_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to update routing rule",
+        )
+
+    logger.info("Routing rule updated: id=%s, user=%s", rule_id, user_id)
+    return _serialize_rule(rule)
+
+
+@router.delete("/{rule_id}", status_code=status.HTTP_204_NO_CONTENT)
+@require_login
+def delete_routing_rule(rule_id: int, request: Request, db: DbSession) -> None:
+    """Delete a routing rule."""
+    user_id = _get_user_id(request)
+    admin = _is_admin(request)
+
+    rule = db.query(PipelineRoutingRule).filter(PipelineRoutingRule.id == rule_id).first()
+    if not rule or not _can_access_rule(rule, user_id, admin):
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Routing rule not found")
+
+    if not _can_write_rule(rule, user_id, admin):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Cannot modify this rule")
+
+    try:
+        db.delete(rule)
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to delete routing rule id=%s", rule_id)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to delete routing rule",
+        )
+
+    logger.info("Routing rule deleted: id=%s, user=%s", rule_id, user_id)

app/api/saved_searches.py

@@ -190,15 +190,15 @@ def create_saved_search(
         db.add(saved_search)
         db.commit()
         db.refresh(saved_search)
-    except Exception as exc:
+    except Exception:
         db.rollback()
-        logger.exception(f"Failed to create saved search for user={user_id}: {exc}")
+        logger.exception("Failed to create saved search for user=%s", user_id)
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail="Failed to save search",
         )
 
-    logger.info(f"Saved search created: user={user_id}, name={name!r}")
+    logger.info("Saved search created: user=%s, name=%r", user_id, name)
     return _serialize_saved_search(saved_search)
 
 
@@ -263,15 +263,15 @@ def update_saved_search(
     try:
         db.commit()
         db.refresh(saved_search)
-    except Exception as exc:
+    except Exception:
         db.rollback()
-        logger.exception(f"Failed to update saved search id={search_id}, user={user_id}: {exc}")
+        logger.exception("Failed to update saved search id=%s, user=%s", search_id, user_id)
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail="Failed to update saved search",
         )
 
-    logger.info(f"Saved search updated: id={search_id}, user={user_id}")
+    logger.info("Saved search updated: id=%s, user=%s", search_id, user_id)
     return _serialize_saved_search(saved_search)
 
 
@@ -294,12 +294,12 @@ def delete_saved_search(search_id: int, request: Request, db: DbSession):
     try:
         db.delete(saved_search)
         db.commit()
-    except Exception as exc:
+    except Exception:
         db.rollback()
-        logger.exception(f"Failed to delete saved search id={search_id}, user={user_id}: {exc}")
+        logger.exception("Failed to delete saved search id=%s, user=%s", search_id, user_id)
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail="Failed to delete saved search",
         )
 
-    logger.info(f"Saved search deleted: id={search_id}, user={user_id}")
+    logger.info("Saved search deleted: id=%s, user=%s", search_id, user_id)

app/api/scheduled_jobs.py

@@ -0,0 +1,350 @@
+"""
+Admin API endpoints for managing scheduled batch processing jobs.
+
+All endpoints require admin privileges (checked via session ``is_admin`` flag).
+
+Available routes:
+    GET  /api/admin/scheduled-jobs         – list all scheduled jobs
+    PATCH /api/admin/scheduled-jobs/{id}   – update schedule / enable-disable
+    POST /api/admin/scheduled-jobs/{id}/run-now – trigger a job immediately
+"""
+
+import logging
+from datetime import datetime, timezone
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Request, status
+from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import ScheduledJob
+
+logger = logging.getLogger(__name__)
+router = APIRouter(prefix="/admin/scheduled-jobs", tags=["admin-scheduled-jobs"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Authorisation helper
+# ---------------------------------------------------------------------------
+
+
+def _require_admin(request: Request) -> dict:
+    """Ensure the caller is an admin; raises HTTP 403 otherwise."""
+    user = request.session.get("user")
+    if not user or not user.get("is_admin"):
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin access required")
+    return user
+
+
+AdminUser = Annotated[dict, Depends(_require_admin)]
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class ScheduledJobResponse(BaseModel):
+    """Read model for a scheduled job."""
+
+    id: int
+    name: str
+    display_name: str
+    description: str | None
+    task_name: str
+    enabled: bool
+    schedule_type: str
+    cron_minute: str
+    cron_hour: str
+    cron_day_of_week: str
+    cron_day_of_month: str
+    cron_month_of_year: str
+    interval_seconds: int | None
+    last_run_at: datetime | None
+    last_run_status: str | None
+    last_run_detail: str | None
+    created_at: datetime | None
+    updated_at: datetime | None
+
+    model_config = {"from_attributes": True}
+
+
+class ScheduledJobUpdate(BaseModel):
+    """Writable fields for a scheduled job update (all optional)."""
+
+    enabled: bool | None = Field(None, description="Whether the job is active")
+    schedule_type: str | None = Field(None, pattern="^(cron|interval)$", description="'cron' or 'interval'")
+    cron_minute: str | None = Field(None, max_length=50)
+    cron_hour: str | None = Field(None, max_length=50)
+    cron_day_of_week: str | None = Field(None, max_length=50)
+    cron_day_of_month: str | None = Field(None, max_length=50)
+    cron_month_of_year: str | None = Field(None, max_length=50)
+    interval_seconds: int | None = Field(None, ge=60, description="Interval in seconds (min 60)")
+
+
+# ---------------------------------------------------------------------------
+# Default job definitions – seeded into the DB on first startup
+# ---------------------------------------------------------------------------
+
+DEFAULT_JOBS: list[dict[str, Any]] = [
+    {
+        "name": "process-new-documents",
+        "display_name": "Process New Documents",
+        "description": (
+            "Scans for documents that have been uploaded but never processed "
+            "and queues them through the full processing pipeline. "
+            "Runs hourly by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.process_new_documents",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "0",
+        "cron_hour": "*/1",
+        "cron_day_of_week": "*",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "reprocess-failed-documents",
+        "display_name": "Reprocess Failed Documents",
+        "description": (
+            "Finds documents whose last processing attempt failed and re-queues "
+            "them for reprocessing. Only picks up files that are not currently "
+            "being processed. Runs every 6 hours by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.reprocess_failed_documents",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "30",
+        "cron_hour": "*/6",
+        "cron_day_of_week": "*",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "cleanup-temp-files",
+        "display_name": "Clean Up Temporary Files",
+        "description": (
+            "Removes stale files from the workdir/tmp directory. "
+            "Only files older than 24 hours that are not referenced by any active "
+            "processing job are deleted. Runs daily at 03:30 UTC by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.cleanup_temp_files",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "30",
+        "cron_hour": "3",
+        "cron_day_of_week": "*",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "expire-shared-links",
+        "display_name": "Expire Stale Shared Links",
+        "description": (
+            "Marks shared document links as inactive when their expiry time has passed. "
+            "Access is already blocked at request time, but this task keeps the "
+            "management UI counts accurate. Runs daily at 01:00 UTC by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.expire_shared_links",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "0",
+        "cron_hour": "1",
+        "cron_day_of_week": "*",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "prune-processing-logs",
+        "display_name": "Prune Old Processing Logs",
+        "description": (
+            "Deletes processing log entries and settings audit log entries older than "
+            "30 days to prevent unbounded database growth. "
+            "Runs weekly on Sunday at 04:00 UTC by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.prune_processing_logs",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "0",
+        "cron_hour": "4",
+        "cron_day_of_week": "0",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "prune-old-notifications",
+        "display_name": "Prune Old Notifications",
+        "description": (
+            "Deletes read in-app notifications older than 30 days. "
+            "Unread notifications are never deleted. "
+            "Runs weekly on Sunday at 04:30 UTC by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.prune_old_notifications",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "30",
+        "cron_hour": "4",
+        "cron_day_of_week": "0",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "backfill-missing-metadata",
+        "display_name": "Backfill Missing AI Metadata",
+        "description": (
+            "Re-triggers AI metadata extraction for documents that have extracted "
+            "text but no AI metadata yet (e.g., processed before an AI provider "
+            "was configured). Processes up to 50 documents per run. "
+            "Runs every 6 hours by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.backfill_missing_metadata",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "0",
+        "cron_hour": "*/6",
+        "cron_day_of_week": "*",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+    {
+        "name": "sync-search-index",
+        "display_name": "Sync Search Index",
+        "description": (
+            "Indexes documents that have OCR text or AI metadata but are missing "
+            "from the Meilisearch search index. Useful after enabling search on "
+            "an existing installation or after an index rebuild. "
+            "Processes up to 100 documents per run. "
+            "Runs hourly by default."
+        ),
+        "task_name": "app.tasks.batch_tasks.sync_search_index",
+        "enabled": True,
+        "schedule_type": "cron",
+        "cron_minute": "15",
+        "cron_hour": "*/1",
+        "cron_day_of_week": "*",
+        "cron_day_of_month": "*",
+        "cron_month_of_year": "*",
+        "interval_seconds": None,
+    },
+]
+
+
+def seed_default_scheduled_jobs(db: Session) -> None:
+    """
+    Insert the built-in scheduled jobs if they do not already exist.
+
+    Called from the FastAPI lifespan handler so the records are available
+    immediately after the first startup.
+    """
+    for job_data in DEFAULT_JOBS:
+        existing = db.query(ScheduledJob).filter(ScheduledJob.name == job_data["name"]).first()
+        if existing is None:
+            db.add(ScheduledJob(**job_data))
+    try:
+        db.commit()
+    except Exception as exc:
+        db.rollback()
+        logger.error("Failed to seed default scheduled jobs: %s", exc)
+
+
+# ---------------------------------------------------------------------------
+# Endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.get("", response_model=list[ScheduledJobResponse])
+def list_scheduled_jobs(request: Request, db: DbSession, _admin: AdminUser) -> list[ScheduledJobResponse]:
+    """
+    Return all scheduled jobs ordered by display name.
+
+    Requires admin privileges.
+    """
+    jobs = db.query(ScheduledJob).order_by(ScheduledJob.display_name).all()
+    return jobs  # type: ignore[return-value]
+
+
+@router.patch("/{job_id}", response_model=ScheduledJobResponse)
+def update_scheduled_job(
+    job_id: int,
+    payload: ScheduledJobUpdate,
+    request: Request,
+    db: DbSession,
+    _admin: AdminUser,
+) -> ScheduledJobResponse:
+    """
+    Update schedule configuration or enabled state for a job.
+
+    Only the fields included in the request body are modified.
+    Changes to the Celery Beat schedule take effect after the worker restarts.
+
+    Requires admin privileges.
+    """
+    job = db.query(ScheduledJob).filter(ScheduledJob.id == job_id).first()
+    if job is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Scheduled job not found")
+
+    update_data = payload.model_dump(exclude_none=True)
+    if not update_data:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="No fields to update")
+
+    for field, value in update_data.items():
+        setattr(job, field, value)
+
+    job.updated_at = datetime.now(timezone.utc)
+
+    try:
+        db.commit()
+        db.refresh(job)
+    except Exception as exc:
+        db.rollback()
+        logger.error("Failed to update scheduled job %s: %s", job_id, exc)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to update scheduled job",
+        ) from exc
+
+    logger.info("Admin updated scheduled job %s (id=%s): %s", job.name, job_id, update_data)
+    return job  # type: ignore[return-value]
+
+
+@router.post("/{job_id}/run-now")
+def run_scheduled_job_now(
+    job_id: int,
+    request: Request,
+    db: DbSession,
+    _admin: AdminUser,
+) -> dict[str, Any]:
+    """
+    Immediately dispatch the Celery task for the given scheduled job.
+
+    The task is sent to the default queue; its result is tracked asynchronously
+    via the ``last_run_at`` / ``last_run_status`` fields updated by the task
+    itself.
+
+    Requires admin privileges.
+    """
+    job = db.query(ScheduledJob).filter(ScheduledJob.id == job_id).first()
+    if job is None:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Scheduled job not found")
+
+    from app.celery_app import celery as celery_app
+
+    task = celery_app.send_task(job.task_name)
+    logger.info("Admin triggered scheduled job %s (id=%s) manually, task_id=%s", job.name, job_id, task.id)
+
+    return {
+        "status": "dispatched",
+        "job_id": job_id,
+        "job_name": job.name,
+        "task_id": task.id,
+    }

app/api/shared_links.py

@@ -0,0 +1,500 @@
+"""API endpoints for document sharing via expiring links.
+
+Authenticated users can create time-limited or view-limited shareable
+links for their documents.  Each link has a cryptographically random
+token that forms a public ``/share/<token>`` URL.  Optional password
+protection is supported; only a PBKDF2-HMAC-SHA256 hash is stored.
+
+Public consumers access files through the ``/share/<token>/download``
+and ``/share/<token>/info`` endpoints — no authentication required.
+"""
+
+import hashlib
+import logging
+import os
+import secrets
+from datetime import datetime, timezone
+from typing import Annotated, Any
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi.responses import FileResponse
+from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.orm import Session
+
+from app.database import get_db
+from app.models import FileRecord, SharedLink
+from app.utils.user_scope import apply_owner_filter, get_current_owner_id
+
+logger = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/shared-links", tags=["shared-links"])
+public_router = APIRouter(tags=["shared-links-public"])
+
+DbSession = Annotated[Session, Depends(get_db)]
+
+# ---------------------------------------------------------------------------
+# Constants
+# ---------------------------------------------------------------------------
+
+#: PBKDF2 iteration count — matches OWASP 2023 recommendation for PBKDF2-HMAC-SHA256.
+_PWD_HASH_ITERATIONS = 600_000
+#: Length of the random per-password salt in bytes (128-bit entropy).
+_PWD_SALT_BYTES = 16
+
+# Valid expiry durations (in hours) presented in the UI.
+EXPIRY_OPTIONS: dict[str, int] = {
+    "1h": 1,
+    "6h": 6,
+    "12h": 12,
+    "24h": 24,
+    "3d": 72,
+    "7d": 168,
+    "14d": 336,
+    "30d": 720,
+}
+
+
+# ---------------------------------------------------------------------------
+# Auth helper
+# ---------------------------------------------------------------------------
+
+
+def _get_owner_id(request: Request) -> str:
+    """Return the current user's owner ID, raising 401 if unauthenticated."""
+    owner_id = get_current_owner_id(request)
+    if not owner_id:
+        raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
+    return owner_id
+
+
+CurrentOwner = Annotated[str, Depends(_get_owner_id)]
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _generate_token() -> str:
+    """Generate a 43-character URL-safe random token."""
+    return secrets.token_urlsafe(32)
+
+
+def _hash_password(password: str) -> str:
+    """Hash *password* with PBKDF2-HMAC-SHA256 and a random per-password salt.
+
+    The returned string uses the format ``{salt_hex}:{dk_hex}`` so that
+    both the salt and the digest can be recovered from a single column.
+
+    Args:
+        password: Plaintext password string.
+
+    Returns:
+        String in the form ``<32-char salt hex>:<64-char digest hex>``,
+        totalling 97 characters (well within the 128-char column limit).
+    """
+    salt = secrets.token_bytes(_PWD_SALT_BYTES)
+    dk = hashlib.pbkdf2_hmac(
+        "sha256",
+        password.encode("utf-8"),
+        salt,
+        _PWD_HASH_ITERATIONS,
+    )
+    return f"{salt.hex()}:{dk.hex()}"
+
+
+def _verify_password(password: str, stored_hash: str) -> bool:
+    """Verify *password* against a hash produced by :func:`_hash_password`.
+
+    Uses constant-time comparison to prevent timing attacks.
+
+    Args:
+        password: Plaintext password to check.
+        stored_hash: The value previously returned by :func:`_hash_password`.
+
+    Returns:
+        ``True`` if *password* matches, ``False`` otherwise.
+    """
+    try:
+        salt_hex, dk_hex = stored_hash.split(":", 1)
+        salt = bytes.fromhex(salt_hex)
+    except (ValueError, TypeError):
+        return False
+    dk = hashlib.pbkdf2_hmac(
+        "sha256",
+        password.encode("utf-8"),
+        salt,
+        _PWD_HASH_ITERATIONS,
+    )
+    return secrets.compare_digest(dk.hex(), dk_hex)
+
+
+def _is_link_valid(link: SharedLink) -> bool:
+    """Return True when *link* is active, unexpired, and within view limit."""
+    if not link.is_active:
+        return False
+    now = datetime.now(timezone.utc)
+    if link.expires_at is not None:
+        exp = link.expires_at
+        if exp.tzinfo is None:
+            exp = exp.replace(tzinfo=timezone.utc)
+        if now > exp:
+            return False
+    if link.max_views is not None and link.view_count >= link.max_views:
+        return False
+    return True
+
+
+def _resolve_file_path(file_record: FileRecord) -> str | None:
+    """Return the best available file path for *file_record*.
+
+    Checks processed path first, then original, then local (tmp) path.
+    Returns ``None`` when no file exists on disk.
+    """
+    from app.config import settings
+
+    workdir = os.path.realpath(settings.workdir)
+    candidates = [
+        file_record.processed_file_path,
+        file_record.original_file_path,
+        file_record.local_filename,
+    ]
+    for path in candidates:
+        if not path:
+            continue
+        # Guard against path traversal in DB values.
+        real = os.path.realpath(path)
+        if not real.startswith(workdir + os.sep) and real != workdir:
+            logger.warning("Shared link file path outside workdir rejected: %s", path)
+            continue
+        if os.path.exists(real):
+            return real
+    return None
+
+
+# ---------------------------------------------------------------------------
+# Pydantic schemas
+# ---------------------------------------------------------------------------
+
+
+class SharedLinkCreate(BaseModel):
+    """Schema for creating a new shared link."""
+
+    file_id: int = Field(..., description="ID of the file to share")
+    expires_in_hours: int | None = Field(
+        None,
+        ge=1,
+        le=720,
+        description="Expiry in hours (1–720). NULL means the link never expires.",
+    )
+    max_views: int | None = Field(
+        None,
+        ge=1,
+        le=10_000,
+        description="Maximum number of downloads/views. NULL means unlimited.",
+    )
+    password: str | None = Field(
+        None,
+        min_length=1,
+        max_length=128,
+        description="Optional password protecting the link.",
+    )
+    label: str | None = Field(
+        None,
+        max_length=255,
+        description="Optional human-readable label for the link.",
+    )
+
+    @field_validator("expires_in_hours")
+    @classmethod
+    def validate_expiry(cls, v: int | None) -> int | None:
+        if v is not None and v not in range(1, 721):
+            raise ValueError("expires_in_hours must be between 1 and 720")
+        return v
+
+
+class SharedLinkResponse(BaseModel):
+    """Shared link info returned to the authenticated owner."""
+
+    id: int
+    token: str
+    file_id: int
+    label: str | None
+    expires_at: datetime | None
+    max_views: int | None
+    view_count: int
+    has_password: bool
+    is_active: bool
+    created_at: datetime | None
+    revoked_at: datetime | None
+    # Filled in by the endpoint, not stored in DB.
+    share_url: str = ""
+    original_filename: str | None = None
+
+    model_config = {"from_attributes": True}
+
+
+class SharedLinkInfoResponse(BaseModel):
+    """Public metadata about a shared link (used on the share landing page)."""
+
+    token: str
+    label: str | None
+    original_filename: str | None
+    expires_at: datetime | None
+    max_views: int | None
+    view_count: int
+    has_password: bool
+    is_valid: bool
+
+
+# ---------------------------------------------------------------------------
+# Private (authenticated) endpoints
+# ---------------------------------------------------------------------------
+
+
+@router.post("/", status_code=status.HTTP_201_CREATED, response_model=SharedLinkResponse)
+async def create_shared_link(
+    body: SharedLinkCreate,
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Create a new shareable link for a document.
+
+    The caller must own the file (or be in single-user mode).
+    Returns the full link metadata including the generated token.
+    """
+    # Verify the file exists and belongs to the caller.
+    q = db.query(FileRecord).filter(FileRecord.id == body.file_id)
+    q = apply_owner_filter(q, request)
+    file_record = q.first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    token = _generate_token()
+    expires_at = None
+    if body.expires_in_hours is not None:
+        expires_at = datetime.now(timezone.utc).replace(microsecond=0)
+        from datetime import timedelta
+
+        expires_at = expires_at + timedelta(hours=body.expires_in_hours)
+
+    password_hash = _hash_password(body.password) if body.password else None
+
+    db_link = SharedLink(
+        token=token,
+        file_id=body.file_id,
+        owner_id=owner_id,
+        label=body.label,
+        expires_at=expires_at,
+        max_views=body.max_views,
+        view_count=0,
+        password_hash=password_hash,
+    )
+    try:
+        db.add(db_link)
+        db.commit()
+        db.refresh(db_link)
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Shared link created: id=%s owner=%s file_id=%s", db_link.id, owner_id, body.file_id)
+
+    base_url = str(request.base_url).rstrip("/")
+    return _link_to_dict(db_link, base_url, file_record.original_filename)
+
+
+@router.get("/", response_model=list[SharedLinkResponse])
+async def list_shared_links(
+    request: Request,
+    owner_id: CurrentOwner,
+    db: DbSession,
+    active_only: bool = Query(False, description="When true, only return active (non-revoked) links"),
+) -> list[dict[str, Any]]:
+    """List all shared links created by the authenticated user."""
+    q = db.query(SharedLink).filter(SharedLink.owner_id == owner_id)
+    if active_only:
+        q = q.filter(SharedLink.is_active.is_(True))
+    links = q.order_by(SharedLink.created_at.desc()).all()
+
+    base_url = str(request.base_url).rstrip("/")
+    result = []
+    for link in links:
+        file_record = db.query(FileRecord).filter(FileRecord.id == link.file_id).first()
+        filename = file_record.original_filename if file_record else None
+        result.append(_link_to_dict(link, base_url, filename))
+    return result
+
+
+@router.delete("/{link_id}", status_code=status.HTTP_200_OK)
+async def revoke_shared_link(
+    link_id: int,
+    owner_id: CurrentOwner,
+    db: DbSession,
+) -> dict[str, str]:
+    """Revoke (soft-delete) a shared link.
+
+    The record is kept for audit purposes but the link immediately
+    stops working for recipients.
+    """
+    db_link = db.query(SharedLink).filter(SharedLink.id == link_id, SharedLink.owner_id == owner_id).first()
+    if not db_link:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Shared link not found")
+
+    if not db_link.is_active:
+        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Link is already revoked")
+
+    try:
+        db_link.is_active = False
+        db_link.revoked_at = datetime.now(timezone.utc)
+        db.commit()
+    except Exception:
+        db.rollback()
+        raise
+
+    logger.info("Shared link revoked: id=%s owner=%s", link_id, owner_id)
+    return {"detail": "Link revoked"}
+
+
+# ---------------------------------------------------------------------------
+# Public endpoints (no authentication required)
+# ---------------------------------------------------------------------------
+
+
+@public_router.get("/share/{token}/info", response_model=SharedLinkInfoResponse)
+def get_shared_link_info(
+    token: str,
+    db: DbSession,
+) -> dict[str, Any]:
+    """Return public metadata about a shared link.
+
+    Used by the share landing page to decide whether to show a password
+    prompt or a direct download button.  Never returns sensitive data.
+    """
+    link = db.query(SharedLink).filter(SharedLink.token == token).first()
+    if not link:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Link not found")
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == link.file_id).first()
+    filename = file_record.original_filename if file_record else None
+
+    return {
+        "token": link.token,
+        "label": link.label,
+        "original_filename": filename,
+        "expires_at": link.expires_at,
+        "max_views": link.max_views,
+        "view_count": link.view_count,
+        "has_password": link.password_hash is not None,
+        "is_valid": _is_link_valid(link),
+    }
+
+
+@public_router.get("/share/{token}/download")
+def download_via_shared_link(
+    token: str,
+    db: DbSession,
+) -> FileResponse:
+    """Download a file via a shared link that does NOT require a password.
+
+    For password-protected links use ``POST /api/share/{token}/download``
+    with ``{"password": "<value>"}`` in the JSON body instead.
+
+    Increments the view counter and validates expiry / view limit before
+    serving the file.
+    """
+    return _serve_shared_file(token, db, password=None)
+
+
+class PasswordBody(BaseModel):
+    """Request body for password-protected shared link downloads."""
+
+    password: str = Field(..., min_length=1, max_length=128, description="Password for the shared link")
+
+
+@public_router.post("/share/{token}/download")
+def download_via_shared_link_with_password(
+    token: str,
+    body: PasswordBody,
+    db: DbSession,
+) -> FileResponse:
+    """Download a password-protected file via a shared link.
+
+    Accepts the password in the JSON request body to avoid it appearing in
+    server access logs, browser history, or ``Referer`` headers.
+    """
+    return _serve_shared_file(token, db, password=body.password)
+
+
+def _serve_shared_file(token: str, db: Session, password: str | None) -> FileResponse:
+    """Core download logic shared by the GET and POST download endpoints."""
+    link = db.query(SharedLink).filter(SharedLink.token == token).first()
+    if not link:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Link not found or expired")
+
+    if not _is_link_valid(link):
+        raise HTTPException(status_code=status.HTTP_410_GONE, detail="Link has expired or reached its view limit")
+
+    # Password check
+    if link.password_hash is not None:
+        if not password:
+            raise HTTPException(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                detail="This link requires a password",
+            )
+        if not _verify_password(password, link.password_hash):
+            raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Incorrect password")
+
+    file_record = db.query(FileRecord).filter(FileRecord.id == link.file_id).first()
+    if not file_record:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not found")
+
+    file_path = _resolve_file_path(file_record)
+    if not file_path:
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="File not available on disk")
+
+    # Increment view count — fail the request if this cannot be persisted so
+    # that view-limited links are not bypassed during temporary DB outages.
+    try:
+        link.view_count = (link.view_count or 0) + 1
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.error("Failed to increment view_count for shared link id=%s — aborting download", link.id)
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="Service temporarily unavailable. Please try again.",
+        )
+
+    return FileResponse(
+        path=file_path,
+        media_type=file_record.mime_type or "application/octet-stream",
+        headers={
+            "Content-Disposition": f'attachment; filename="{file_record.original_filename or "document"}"',
+        },
+    )
+
+
+# ---------------------------------------------------------------------------
+# Internal helper
+# ---------------------------------------------------------------------------
+
+
+def _link_to_dict(link: SharedLink, base_url: str, original_filename: str | None) -> dict[str, Any]:
+    """Serialise a ``SharedLink`` ORM row to a plain dict."""
+    return {
+        "id": link.id,
+        "token": link.token,
+        "file_id": link.file_id,
+        "label": link.label,
+        "expires_at": link.expires_at,
+        "max_views": link.max_views,
+        "view_count": link.view_count,
+        "has_password": link.password_hash is not None,
+        "is_active": link.is_active,
+        "created_at": link.created_at,
+        "revoked_at": link.revoked_at,
+        "share_url": f"{base_url}/share/{link.token}",
+        "original_filename": original_filename,
+    }

app/api/url_upload.py

@@ -2,7 +2,6 @@
 API endpoint for processing files from URLs
 """
 
-import ipaddress
 import logging
 import mimetypes
 import os
@@ -19,6 +18,7 @@ from app.config import settings
 from app.tasks.process_document import process_document
 from app.utils.allowed_types import ALLOWED_MIME_TYPES
 from app.utils.filename_utils import sanitize_filename
+from app.utils.network import is_private_ip
 
 # Set up logging
 logger = logging.getLogger(__name__)
@@ -42,37 +42,6 @@ class URLUploadRequest(BaseModel):
         return v
 
 
-def is_private_ip(hostname: str) -> bool:
-    """
-    Check if a hostname resolves to a private/internal IP address.
-    Protects against SSRF attacks by blocking access to internal networks.
-    """
-    try:
-        # Try to parse as IP address directly
-        ip = ipaddress.ip_address(hostname)
-        return ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved
-    except ValueError:
-        # Not a direct IP, try to resolve hostname
-        try:
-            import socket
-
-            # Get all IP addresses for this hostname
-            addr_info = socket.getaddrinfo(hostname, None)
-            for info in addr_info:
-                ip_str = info[4][0]
-                ip = ipaddress.ip_address(ip_str)
-                # Block if ANY resolved IP is private/internal
-                if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
-                    return True
-            return False
-        except (socket.gaierror, socket.error):
-            # Cannot resolve - allow for testing/development
-            # In production, DNS should work properly
-            # Log this for debugging
-            logger.warning(f"Could not resolve hostname: {hostname}")
-            return False  # Changed from True to False to allow external domains in tests
-
-
 def validate_url_safety(url: str) -> None:
     """
     Validate that URL is safe to fetch (SSRF protection).

app/api/user.py

@@ -12,7 +12,7 @@ from sqlalchemy.orm import Session
 
 from app.auth import require_login
 from app.database import get_db
-from app.models import FileRecord
+from app.models import FileRecord, UserProfile
 
 # Set up logging
 logger = logging.getLogger(__name__)
@@ -22,7 +22,7 @@ router = APIRouter()
 DbSession = Annotated[Session, Depends(get_db)]
 
 
-async def whoami_handler(request: Request):
+async def whoami_handler(request: Request, db: Session):
     """
     Returns user info if logged in, else 401.
     """
@@ -41,20 +41,33 @@ async def whoami_handler(request: Request):
 
     # Add the gravatar URL to the user object instead of creating a new response
     user_response = user.copy()  # Create a copy to avoid modifying the session
-    user_response["picture"] = gravatar_url
+
+    # Check if the user has a custom avatar stored in their profile
+    user_id = user.get("sub") or user.get("preferred_username") or user.get("email") or user.get("id")
+    if user_id:
+        try:
+            profile = db.query(UserProfile).filter(UserProfile.user_id == user_id).first()
+            if profile and profile.avatar_data:
+                user_response["picture"] = profile.avatar_data
+            else:
+                user_response["picture"] = gravatar_url
+        except Exception:
+            user_response["picture"] = gravatar_url
+    else:
+        user_response["picture"] = gravatar_url
 
     return user_response
 
 
 # Register the same handler under two different paths
 @router.get("/whoami")
-async def whoami(request: Request):
-    return await whoami_handler(request)
+async def whoami(request: Request, db: DbSession):
+    return await whoami_handler(request, db)
 
 
 @router.get("/auth/whoami")
-async def auth_whoami(request: Request):
-    return await whoami_handler(request)
+async def auth_whoami(request: Request, db: DbSession):
+    return await whoami_handler(request, db)
 
 
 @router.get("/users/search")

app/auth.py

@@ -2,24 +2,28 @@ import hashlib
 import inspect
 import logging
 import pathlib
+from datetime import datetime, timezone
 from functools import wraps
-from urllib.parse import urlparse
+from urllib.parse import urlencode, urlparse
 
 from authlib.integrations.starlette_client import OAuth
 from fastapi import APIRouter, Depends, Request, status
 from fastapi.responses import JSONResponse
 from fastapi.templating import Jinja2Templates
+from sqlalchemy import func
 from sqlalchemy.orm import Session
 from starlette.responses import RedirectResponse
 
 from app.config import settings
 from app.database import get_db
+from app.middleware.audit_log import get_client_ip
 
 # Conditional imports: only used when multi_user_enabled=True. Imported here at
 # module level (not inside auth()) so they don't incur repeated import overhead.
 # Guards at call-sites ensure they are never *called* in single-user mode.
 from app.models import LocalUser as _LocalUser
 from app.models import UserProfile as _UserProfile
+from app.utils.i18n import translate as _translate
 from app.utils.local_auth import build_session_user as _build_session_user
 from app.utils.local_auth import verify_password as _verify_password
 
@@ -32,11 +36,15 @@ AUTH_ENABLED = settings.auth_enabled
 # Set up templates for authentication
 templates_dir = pathlib.Path(__file__).parents[1] / "frontend" / "templates"
 templates = Jinja2Templates(directory=str(templates_dir))
+templates.env.globals["_"] = lambda key, **kwargs: _translate(key, "en", **kwargs)
 
 # Configure OAuth provider if credentials are provided
 OAUTH_CONFIGURED = False
 OAUTH_PROVIDER_NAME = "Single Sign-On"
 
+# Social login providers that are enabled and registered
+SOCIAL_PROVIDERS: dict[str, dict[str, str]] = {}
+
 if AUTH_ENABLED and settings.authentik_client_id and settings.authentik_client_secret:
     oauth.register(
         name="authentik",
@@ -48,13 +56,128 @@ if AUTH_ENABLED and settings.authentik_client_id and settings.authentik_client_s
     OAUTH_CONFIGURED = True
     OAUTH_PROVIDER_NAME = settings.oauth_provider_name or "Authentik SSO"
 
+# --- Social Login Providers ---------------------------------------------------
+if AUTH_ENABLED and settings.social_auth_google_enabled:
+    if settings.social_auth_google_client_id and settings.social_auth_google_client_secret:
+        oauth.register(
+            name="google",
+            client_id=settings.social_auth_google_client_id,
+            client_secret=settings.social_auth_google_client_secret,
+            server_metadata_url="https://accounts.google.com/.well-known/openid-configuration",
+            client_kwargs={"scope": "openid profile email"},
+        )
+        SOCIAL_PROVIDERS["google"] = {"name": "Google", "icon": "fab fa-google", "color": "red"}
+        logger.info("Social login provider registered: Google")
+    else:
+        logger.warning("SOCIAL_AUTH_GOOGLE_ENABLED=true but client ID/secret not configured")
+
+if AUTH_ENABLED and settings.social_auth_microsoft_enabled:
+    if settings.social_auth_microsoft_client_id and settings.social_auth_microsoft_client_secret:
+        tenant = settings.social_auth_microsoft_tenant or "common"
+        oauth.register(
+            name="microsoft",
+            client_id=settings.social_auth_microsoft_client_id,
+            client_secret=settings.social_auth_microsoft_client_secret,
+            server_metadata_url=f"https://login.microsoftonline.com/{tenant}/v2.0/.well-known/openid-configuration",
+            client_kwargs={"scope": "openid profile email"},
+        )
+        SOCIAL_PROVIDERS["microsoft"] = {"name": "Microsoft", "icon": "fab fa-microsoft", "color": "blue"}
+        logger.info("Social login provider registered: Microsoft (tenant=%s)", tenant)
+    else:
+        logger.warning("SOCIAL_AUTH_MICROSOFT_ENABLED=true but client ID/secret not configured")
+
+if AUTH_ENABLED and settings.social_auth_apple_enabled:
+    if settings.social_auth_apple_client_id and settings.social_auth_apple_team_id:
+        oauth.register(
+            name="apple",
+            client_id=settings.social_auth_apple_client_id,
+            server_metadata_url="https://appleid.apple.com/.well-known/openid-configuration",
+            client_kwargs={
+                "scope": "openid name email",
+                "response_mode": "form_post",
+            },
+        )
+        SOCIAL_PROVIDERS["apple"] = {"name": "Apple", "icon": "fab fa-apple", "color": "gray"}
+        logger.info("Social login provider registered: Apple")
+    else:
+        logger.warning("SOCIAL_AUTH_APPLE_ENABLED=true but client ID/team ID not configured")
+
+if AUTH_ENABLED and settings.social_auth_dropbox_enabled:
+    if settings.social_auth_dropbox_client_id and settings.social_auth_dropbox_client_secret:
+        oauth.register(
+            name="dropbox",
+            client_id=settings.social_auth_dropbox_client_id,
+            client_secret=settings.social_auth_dropbox_client_secret,
+            authorize_url="https://www.dropbox.com/oauth2/authorize",
+            access_token_url="https://api.dropboxapi.com/oauth2/token",
+            userinfo_endpoint="https://api.dropboxapi.com/2/users/get_current_account",
+            client_kwargs={"token_endpoint_auth_method": "client_secret_post"},
+        )
+        SOCIAL_PROVIDERS["dropbox"] = {"name": "Dropbox", "icon": "fab fa-dropbox", "color": "blue"}
+        logger.info("Social login provider registered: Dropbox")
+    else:
+        logger.warning("SOCIAL_AUTH_DROPBOX_ENABLED=true but client ID/secret not configured")
+
 router = APIRouter()
 
 
 def get_current_user(request: Request):
+    # Check for Bearer token auth first (API tokens)
+    api_user = getattr(request.state, "api_token_user", None)
+    if isinstance(api_user, dict):
+        return api_user
     return request.session.get("user")
 
 
+def _resolve_bearer_user(request: Request, db: Session) -> dict | None:
+    """Resolve a user from a Bearer API token in the Authorization header.
+
+    If the header is present and the token is valid, updates usage tracking
+    (last_used_at, last_used_ip) and returns a synthetic user dict compatible
+    with the session user format.
+
+    Returns:
+        A user dict or ``None`` if no valid Bearer token is present.
+    """
+    auth_header = request.headers.get("authorization", "")
+    if not isinstance(auth_header, str) or not auth_header.startswith("Bearer "):
+        return None
+
+    raw_token = auth_header[7:]
+    if not raw_token or not isinstance(raw_token, str):
+        return None
+
+    from app.api.api_tokens import hash_token
+    from app.models import ApiToken
+
+    token_hash = hash_token(raw_token)
+    db_token = db.query(ApiToken).filter(ApiToken.token_hash == token_hash, ApiToken.is_active.is_(True)).first()
+    if db_token is None:
+        return None
+
+    # Update usage tracking
+    try:
+        db_token.last_used_at = datetime.now(timezone.utc)
+        # Extract client IP (respect X-Forwarded-For from reverse proxy)
+        client_ip = request.headers.get("x-forwarded-for", "").split(",")[0].strip()
+        if not client_ip and request.client:
+            client_ip = request.client.host
+        db_token.last_used_ip = client_ip or None
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.debug("Failed to update API token usage tracking for token_id=%s", db_token.id)
+
+    # Build a synthetic user dict that mimics the session user format
+    return {
+        "id": db_token.owner_id,
+        "email": db_token.owner_id,
+        "preferred_username": db_token.owner_id,
+        "is_admin": False,
+        "_api_token_id": db_token.id,
+    }
+
+
 def get_current_user_id(request: Request) -> str:
     """Return a stable string identifier for the authenticated user.
 
@@ -82,28 +205,42 @@ def require_login(func):
 
     @wraps(func)
     async def wrapper(request: Request, *args, **kwargs):
-        if not request.session.get("user"):
-            # For API endpoints return 401 instead of storing the URL in the session
-            # and redirecting to /login.  Without this guard, the /api/auth/whoami
-            # probe issued by common.js on every page load would overwrite
-            # redirect_after_login with the API URL, causing the post-login redirect
-            # to land on a JSON endpoint rather than the original page.
-            url_path = urlparse(str(request.url)).path
-            if url_path.startswith("/api/"):
-                return JSONResponse(
-                    status_code=status.HTTP_401_UNAUTHORIZED,
-                    content={"error": "Not authenticated"},
-                )
-            request.session["redirect_after_login"] = str(request.url)
-            return RedirectResponse(url="/login", status_code=status.HTTP_302_FOUND)
-        # Pass request as a keyword argument so that endpoints whose first
-        # parameter is a path variable (e.g. pipeline_id) are not accidentally
-        # bound to the request object when FastAPI supplies all arguments as
-        # keyword arguments.
-        if inspect.iscoroutinefunction(func):
-            return await func(*args, request=request, **kwargs)
-        else:
-            return func(*args, request=request, **kwargs)
+        # Check session auth first
+        if request.session.get("user"):
+            if inspect.iscoroutinefunction(func):
+                return await func(*args, request=request, **kwargs)
+            else:
+                return func(*args, request=request, **kwargs)
+
+        # Fall back to Bearer token auth for API endpoints
+        url_path = urlparse(str(request.url)).path
+        if url_path.startswith("/api/"):
+            try:
+                from app.database import SessionLocal
+
+                db = SessionLocal()
+                try:
+                    api_user = _resolve_bearer_user(request, db)
+                finally:
+                    db.close()
+            except Exception:
+                api_user = None
+
+            if api_user:
+                request.state.api_token_user = api_user
+                if inspect.iscoroutinefunction(func):
+                    return await func(*args, request=request, **kwargs)
+                else:
+                    return func(*args, request=request, **kwargs)
+
+            return JSONResponse(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                content={"error": "Not authenticated"},
+            )
+
+        # Non-API endpoint with no session — redirect to login
+        request.session["redirect_after_login"] = str(request.url)
+        return RedirectResponse(url="/login", status_code=status.HTTP_302_FOUND)
 
     return wrapper
 
@@ -118,6 +255,38 @@ def get_gravatar_url(email):
 
 async def login(request: Request):
     """Show login page with appropriate authentication options."""
+    # Persist the mobile deep-link redirect URI in the session so it survives
+    # the OAuth provider round-trip and is available when auth completes.
+    # Accepted schemes:
+    #   • "docuelevate://" — production / EAS builds (custom app scheme)
+    #   • "exp://"         — Expo Go development client
+    # Only custom (non-HTTP) schemes are accepted to prevent open-redirect abuse.
+    _MOBILE_ALLOWED_SCHEMES = ("docuelevate://", "exp://")
+    if request.query_params.get("mobile") == "1":
+        redirect_uri = request.query_params.get("redirect_uri", "")
+        logger.debug(
+            "[MOBILE] Login page opened with mobile=1: redirect_uri=%r client_ip=%s",
+            redirect_uri,
+            get_client_ip(request),
+        )
+        if any(redirect_uri.startswith(s) for s in _MOBILE_ALLOWED_SCHEMES):
+            request.session["mobile_redirect_uri"] = redirect_uri
+            logger.info(
+                "[MOBILE] Mobile redirect URI stored in session: %r",
+                redirect_uri,
+            )
+        else:
+            logger.warning(
+                "[MOBILE] Rejected redirect_uri with disallowed scheme: %r (allowed: %s)",
+                redirect_uri,
+                ", ".join(_MOBILE_ALLOWED_SCHEMES),
+            )
+    else:
+        logger.debug(
+            "[MOBILE] Login page opened without mobile=1 (standard browser flow) client_ip=%s",
+            get_client_ip(request),
+        )
+
     return templates.TemplateResponse(
         "login.html",
         {
@@ -126,6 +295,7 @@ async def login(request: Request):
             "message": request.query_params.get("message"),
             "show_oauth": OAUTH_CONFIGURED,
             "oauth_provider_name": OAUTH_PROVIDER_NAME,
+            "social_providers": SOCIAL_PROVIDERS,
             "app_version": settings.version,
             "csrf_token": getattr(request.state, "csrf_token", ""),
             # "Create account" link is only shown when multi-user mode AND local signup are both enabled
@@ -143,6 +313,155 @@ async def oauth_login(request: Request):
     return await oauth.authentik.authorize_redirect(request, redirect_uri)
 
 
+async def social_login(request: Request, provider: str):
+    """Initiate a social login flow for the given provider.
+
+    Args:
+        request: The current FastAPI request.
+        provider: One of the registered social provider keys (google, microsoft, apple, dropbox).
+
+    Returns:
+        A redirect to the provider's authorization page, or back to /login on error.
+    """
+    if provider not in SOCIAL_PROVIDERS:
+        return RedirectResponse(url="/login?error=Unknown+social+provider", status_code=status.HTTP_302_FOUND)
+
+    redirect_uri = request.url_for("social_callback", provider=provider)
+    oauth_client = getattr(oauth, provider, None)
+    if oauth_client is None:
+        return RedirectResponse(url="/login?error=Provider+not+configured", status_code=status.HTTP_302_FOUND)
+
+    return await oauth_client.authorize_redirect(request, redirect_uri)
+
+
+def _normalize_social_userinfo(provider: str, token: dict, raw_userinfo: dict | None) -> dict:
+    """Normalize the userinfo payload from different social providers into a common format.
+
+    Returns a dict with keys: sub, email, name, preferred_username, picture.
+
+    Args:
+        provider: The social provider key (google, microsoft, apple, dropbox).
+        token: The OAuth token response from the provider. Included for future
+            provider-specific claim extraction (e.g. ``id_token`` claims).
+        raw_userinfo: The raw userinfo dict (may be None for providers without standard OIDC userinfo).
+
+    Returns:
+        A normalized user-data dict compatible with the session user format.
+    """
+    userinfo: dict = raw_userinfo or {}
+
+    if provider == "dropbox":
+        # Dropbox returns a non-standard userinfo response
+        email = userinfo.get("email", "")
+        name_info = userinfo.get("name", {})
+        display_name = name_info.get("display_name", "") if isinstance(name_info, dict) else str(name_info)
+        return {
+            "sub": userinfo.get("account_id", email),
+            "email": email,
+            "name": display_name,
+            "preferred_username": email,
+            "picture": userinfo.get("profile_photo_url", ""),
+        }
+
+    # Standard OIDC providers (Google, Microsoft, Apple)
+    return {
+        "sub": userinfo.get("sub", ""),
+        "email": userinfo.get("email", ""),
+        "name": userinfo.get("name", ""),
+        "preferred_username": userinfo.get("email", ""),
+        "picture": userinfo.get("picture", ""),
+    }
+
+
+async def social_callback(request: Request, provider: str, db: Session = Depends(get_db)):
+    """Handle the OAuth callback from a social login provider.
+
+    After the user authorizes with the social provider, this endpoint exchanges
+    the authorization code for tokens, extracts user information, creates or
+    updates the user profile, and establishes a session.
+
+    Args:
+        request: The current FastAPI request.
+        provider: One of the registered social provider keys.
+        db: Database session (injected).
+
+    Returns:
+        A redirect to the user's original destination or the upload page.
+    """
+    if provider not in SOCIAL_PROVIDERS:
+        return RedirectResponse(url="/login?error=Unknown+social+provider", status_code=status.HTTP_302_FOUND)
+
+    oauth_client = getattr(oauth, provider, None)
+    if oauth_client is None:
+        return RedirectResponse(url="/login?error=Provider+not+configured", status_code=status.HTTP_302_FOUND)
+
+    try:
+        token = await oauth_client.authorize_access_token(request)
+
+        # Try standard OIDC userinfo first, fall back to token-embedded userinfo
+        raw_userinfo = token.get("userinfo")
+        if not raw_userinfo:
+            try:
+                resp = await oauth_client.userinfo(token=token)
+                raw_userinfo = resp if isinstance(resp, dict) else resp.json() if hasattr(resp, "json") else {}
+            except Exception:
+                raw_userinfo = {}
+
+        user_data = _normalize_social_userinfo(provider, token, raw_userinfo)
+
+        if not user_data.get("email"):
+            return RedirectResponse(
+                url="/login?error=Could+not+retrieve+email+from+provider",
+                status_code=status.HTTP_302_FOUND,
+            )
+
+        # Add Gravatar if no picture provided
+        if not user_data.get("picture") and user_data.get("email"):
+            user_data["picture"] = get_gravatar_url(user_data["email"])
+
+        # Tag the login source for audit/debugging
+        user_data["auth_provider"] = provider
+
+        # Social login users are never admin by default (admin must be granted
+        # via the Authentik/OIDC admin group or manually in the admin panel)
+        user_data["is_admin"] = False
+
+        request.session["user"] = user_data
+
+        # Auto-create or update UserProfile
+        _ensure_user_profile(db, user_data, is_admin=False)
+
+        provider_name = SOCIAL_PROVIDERS[provider]["name"]
+        logger.info(
+            "[SECURITY] SOCIAL_LOGIN_SUCCESS provider=%s user=%s", provider_name, user_data.get("email", "unknown")
+        )
+
+        # Redirect first-time users to onboarding
+        user_id = (
+            user_data.get("sub") or user_data.get("preferred_username") or user_data.get("email") or user_data.get("id")
+        )
+
+        # Mobile app flow: issue an inline API token and redirect back to the app.
+        mobile_resp = _create_mobile_redirect(request, db)
+        if mobile_resp:
+            return mobile_resp
+
+        if user_id:
+            profile = db.query(_UserProfile).filter(_UserProfile.user_id == user_id).first()
+            if profile and not profile.onboarding_completed:
+                post_onboarding = request.session.pop("redirect_after_login", "/upload")
+                request.session["post_onboarding_redirect"] = post_onboarding
+                return RedirectResponse(url="/onboarding", status_code=status.HTTP_302_FOUND)
+
+        redirect_url = request.session.pop("redirect_after_login", "/upload")
+        return RedirectResponse(url=redirect_url, status_code=status.HTTP_302_FOUND)
+    except Exception as e:
+        logger.warning("[SECURITY] SOCIAL_LOGIN_FAILURE provider=%s error=%s", provider, type(e).__name__)
+        return RedirectResponse(
+            url="/login?error=Social+login+failed.+Please+try+again.", status_code=status.HTTP_302_FOUND
+        )
+
+
 def _ensure_user_profile(db: Session, user_data: dict, is_admin: bool = False) -> None:
     """Create or update a UserProfile row for *user_data*.
 
@@ -276,11 +595,31 @@ async def oauth_callback(request: Request, db: Session = Depends(get_db)):
 
         # Log the successful authentication
         logger.info("[SECURITY] OAUTH_LOGIN_SUCCESS user=%s admin=%s", user_data.get("email", "unknown"), is_admin)
+        _record_login_event(
+            db,
+            request,
+            user_data.get("email") or user_data.get("preferred_username") or "unknown",
+            success=True,
+            method="oauth",
+        )
 
         # Redirect first-time users to onboarding
         user_id = (
             user_data.get("sub") or user_data.get("preferred_username") or user_data.get("email") or user_data.get("id")
         )
+
+        # Mobile app flow: issue an inline API token and redirect back to the app.
+        # This check runs before onboarding so native-app users are never sent
+        # to the web-based onboarding wizard.
+        logger.debug(
+            "[MOBILE] oauth_callback: checking for mobile redirect (session has mobile_redirect_uri=%s)",
+            "mobile_redirect_uri" in request.session,
+        )
+        mobile_resp = _create_mobile_redirect(request, db)
+        if mobile_resp:
+            logger.info("[MOBILE] oauth_callback: returning mobile redirect response")
+            return mobile_resp
+
         if user_id:
             profile = db.query(_UserProfile).filter(_UserProfile.user_id == user_id).first()
             if profile and not profile.onboarding_completed:
@@ -296,6 +635,140 @@ async def oauth_callback(request: Request, db: Session = Depends(get_db)):
         return RedirectResponse(url=f"/login?error=Authentication+failed:+{str(e)}", status_code=status.HTTP_302_FOUND)
 
 
+def _record_login_event(
+    db: Session,
+    request: Request,
+    username: str,
+    *,
+    success: bool,
+    method: str = "local",
+    detail: str | None = None,
+) -> None:
+    """Write a login or login-failure audit event to the database.
+
+    Failures are silently swallowed so that an audit-service error never
+    prevents a legitimate login or surfaces an unrelated 500 error to the user.
+
+    Args:
+        db: Active database session.
+        request: The current HTTP request (used to extract the client IP).
+        username: The username that attempted authentication.
+        success: ``True`` for a successful login, ``False`` for a failure.
+        method: Authentication method, e.g. ``"local"`` or ``"oauth"``.
+        detail: Optional extra context for failures (e.g. ``"wrong_password"``).
+    """
+    try:
+        from app.utils.audit_service import record_event
+
+        action = "login" if success else "login.failure"
+        details: dict = {"method": method}
+        if detail:
+            details["reason"] = detail
+        record_event(
+            db,
+            action=action,
+            user=username,
+            resource_type="session",
+            ip_address=get_client_ip(request),
+            details=details,
+            severity="info" if success else "warning",
+        )
+    except Exception:
+        logger.debug("Failed to write login audit event for user=%s", username, exc_info=True)
+
+
+def _create_mobile_redirect(request: Request, db: Session) -> RedirectResponse | None:
+    """Generate a mobile API token and return a redirect to the mobile app.
+
+    If ``mobile_redirect_uri`` is stored in the session (set when the login
+    page was opened with ``?mobile=1&redirect_uri=docuelevate://...``), this
+    function creates a long-lived API token, appends it as a ``?token=``
+    query parameter to the redirect URI, and returns the redirect so that
+    ``WebBrowser.openAuthSessionAsync`` in the Expo app intercepts the
+    deep link and stores the token.
+
+    Returns ``None`` when the request is not part of a mobile SSO flow.
+
+    Args:
+        request: The current FastAPI request.  The ``user`` dict must already
+            be stored in ``request.session`` before calling this function.
+        db: Active database session used to persist the new API token.
+
+    Returns:
+        A ``RedirectResponse`` to the deep-link URI with ``?token=<plaintext>``,
+        or ``None`` if no mobile redirect URI is pending.
+    """
+    mobile_redirect_uri = request.session.pop("mobile_redirect_uri", None)
+    if not mobile_redirect_uri:
+        logger.debug("[MOBILE] _create_mobile_redirect: no mobile_redirect_uri in session — skipping mobile flow")
+        return None
+
+    logger.info(
+        "[MOBILE] _create_mobile_redirect: mobile flow detected, redirect_uri=%r",
+        mobile_redirect_uri,
+    )
+
+    user = request.session.get("user") or {}
+    owner_id = user.get("sub") or user.get("preferred_username") or user.get("email") or user.get("id")
+    logger.debug(
+        "[MOBILE] Resolving owner_id from session user: sub=%r preferred_username=%r email=%r id=%r → owner_id=%r",
+        user.get("sub"),
+        user.get("preferred_username"),
+        user.get("email"),
+        user.get("id"),
+        owner_id,
+    )
+    if not owner_id:
+        logger.warning("Mobile SSO redirect requested but no owner_id could be resolved from session")
+        return None
+
+    # Lazy imports to avoid circular dependency via app.api.__init__
+    from app.api.api_tokens import generate_api_token, hash_token
+    from app.models import ApiToken as _ApiToken
+
+    plaintext = generate_api_token()
+    token_hash_value = hash_token(plaintext)
+    prefix = plaintext[:12]
+
+    db_token = _ApiToken(
+        owner_id=owner_id,
+        name="Mobile App",
+        token_hash=token_hash_value,
+        token_prefix=prefix,
+    )
+    try:
+        db.add(db_token)
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create mobile API token for owner_id=%s", owner_id)
+        return None
+
+    # Safely append the token as a query parameter, preserving any existing params.
+    separator = "&" if "?" in mobile_redirect_uri else "?"
+    redirect_url = f"{mobile_redirect_uri}{separator}{urlencode({'token': plaintext})}"
+
+    # Log the full redirect URL at DEBUG so it's visible when debug logging is enabled.
+    # At INFO level, log a sanitised version (scheme + host only, token prefix only)
+    # so the plaintext token is never written to persistent info logs.
+    parsed = urlparse(redirect_url)
+    existing_params = f"&{parsed.query.replace(f'token={plaintext}', '')}" if parsed.query else ""
+    sanitised_url = (
+        f"{parsed.scheme}://{parsed.netloc}{parsed.path}?token={prefix}…[redacted]{existing_params.rstrip('&')}"
+    )
+    logger.info(
+        "[MOBILE] MOBILE_SSO_TOKEN_ISSUED owner=%s token_id=%s redirect_target=%s",
+        owner_id,
+        db_token.id,
+        sanitised_url,
+    )
+    logger.debug(
+        "[MOBILE] Full redirect URL being sent to client: %s",
+        redirect_url,
+    )
+    return RedirectResponse(url=redirect_url, status_code=status.HTTP_302_FOUND)
+
+
 async def auth(request: Request, db: Session = Depends(get_db)):
     """Handle local username/password authentication.
 
@@ -312,25 +785,75 @@ async def auth(request: Request, db: Session = Depends(get_db)):
     username = form_data.get("username")
     password = form_data.get("password")
 
+    logger.debug(
+        "[AUTH] Login attempt: username=%r password_provided=%s multi_user_enabled=%s",
+        username,
+        bool(password),
+        settings.multi_user_enabled,
+    )
+
     # --- LocalUser check (multi-user mode only) ---
     if settings.multi_user_enabled:
+        if not username:
+            logger.warning(
+                "[AUTH] LOGIN_FAILURE reason=empty_username multi_user_enabled=%s form_keys=%s content_type=%s",
+                settings.multi_user_enabled,
+                list(form_data.keys()),
+                request.headers.get("content-type", "<missing>"),
+            )
+            return RedirectResponse(url="/login?error=Invalid+username+or+password", status_code=302)
+
+        username_lower = username.lower()
         local_user = (
-            db.query(_LocalUser).filter((_LocalUser.username == username) | (_LocalUser.email == username)).first()
+            db.query(_LocalUser)
+            .filter(
+                (func.lower(_LocalUser.username) == username_lower) | (func.lower(_LocalUser.email) == username_lower)
+            )
+            .first()
+        )
+        logger.debug(
+            "[AUTH] LocalUser lookup: username=%r found=%s",
+            username,
+            local_user is not None,
         )
         if local_user is not None:
             if not local_user.is_active:
-                logger.warning("[SECURITY] LOCAL_LOGIN_UNVERIFIED user=%s", username)
+                logger.warning(
+                    "[SECURITY] LOCAL_LOGIN_UNVERIFIED user=%s is_active=%s",
+                    username,
+                    local_user.is_active,
+                )
+                _record_login_event(db, request, username, success=False, detail="account_not_verified")
                 return RedirectResponse(
                     url="/login?error=Please+verify+your+email+address+before+logging+in",
                     status_code=302,
                 )
-            if not _verify_password(password or "", local_user.hashed_password):
-                logger.warning("[SECURITY] LOCAL_LOGIN_FAILURE user=%s", username)
+            pw_ok = _verify_password(password or "", local_user.hashed_password)
+            logger.debug(
+                "[AUTH] Password verification: user=%s ok=%s",
+                username,
+                pw_ok,
+            )
+            if not pw_ok:
+                logger.warning("[SECURITY] LOCAL_LOGIN_FAILURE reason=wrong_password user=%s", username)
+                _record_login_event(db, request, username, success=False, detail="wrong_password")
                 return RedirectResponse(url="/login?error=Invalid+username+or+password", status_code=302)
             user_data = _build_session_user(local_user)
             request.session["user"] = user_data
             logger.info("[SECURITY] LOCAL_LOGIN_SUCCESS user=%s", local_user.email)
+            _record_login_event(db, request, local_user.email, success=True)
             _ensure_user_profile(db, user_data, is_admin=bool(local_user.is_admin))
+
+            # Mobile app flow: issue an inline API token and redirect back to the app.
+            logger.debug(
+                "[MOBILE] local auth: checking for mobile redirect (session has mobile_redirect_uri=%s)",
+                "mobile_redirect_uri" in request.session,
+            )
+            mobile_resp = _create_mobile_redirect(request, db)
+            if mobile_resp:
+                logger.info("[MOBILE] local auth: returning mobile redirect response")
+                return mobile_resp
+
             profile = db.query(_UserProfile).filter(_UserProfile.user_id == local_user.email).first()
             if profile and not profile.onboarding_completed:
                 post_onboarding = request.session.pop("redirect_after_login", "/upload")
@@ -339,16 +862,29 @@ async def auth(request: Request, db: Session = Depends(get_db)):
             redirect_url = request.session.pop("redirect_after_login", "/upload")
             return RedirectResponse(url=redirect_url, status_code=302)
 
+        # Local user not found; fall through to admin credential check below.
+        logger.debug(
+            "[AUTH] No LocalUser matched username=%r; falling through to admin credential check",
+            username,
+        )
+
     # --- Admin credentials (always available as a fallback / single-user mode) ---
     # Guard: only attempt the match when credentials are actually configured.
     # Without this guard, Python's `None == None` would be True when neither
     # ADMIN_USERNAME nor ADMIN_PASSWORD is set, allowing any request that omits
     # those form fields to be authenticated as an admin — creating a phantom
     # "None@local.docuelevate" admin profile with full privileges.
+    admin_configured = bool(settings.admin_username and settings.admin_password)
+    logger.debug(
+        "[AUTH] Admin credential check: admin_configured=%s username_match=%s multi_user_enabled=%s",
+        admin_configured,
+        (username or "").lower() == settings.admin_username.lower() if admin_configured else False,
+        settings.multi_user_enabled,
+    )
     if (
         settings.admin_username
         and settings.admin_password
-        and username == settings.admin_username
+        and (username or "").lower() == settings.admin_username.lower()
         and password == settings.admin_password
     ):
         admin_user_data = {
@@ -361,21 +897,54 @@ async def auth(request: Request, db: Session = Depends(get_db)):
         }
         request.session["user"] = admin_user_data
         logger.info("[SECURITY] LOCAL_LOGIN_SUCCESS user=%s", username)
+        _record_login_event(db, request, username, success=True)
         _ensure_user_profile(db, admin_user_data, is_admin=True)
+
+        # Mobile app flow: issue an inline API token and redirect back to the app.
+        logger.debug(
+            "[MOBILE] admin auth: checking for mobile redirect (session has mobile_redirect_uri=%s)",
+            "mobile_redirect_uri" in request.session,
+        )
+        mobile_resp = _create_mobile_redirect(request, db)
+        if mobile_resp:
+            logger.info("[MOBILE] admin auth: returning mobile redirect response")
+            return mobile_resp
+
         redirect_url = request.session.pop("redirect_after_login", "/upload")
         return RedirectResponse(url=redirect_url, status_code=302)
     else:
-        logger.warning("[SECURITY] LOCAL_LOGIN_FAILURE user=%s", username)
+        logger.warning(
+            "[SECURITY] LOCAL_LOGIN_FAILURE reason=no_match user=%r "
+            "multi_user_enabled=%s admin_configured=%s form_empty=%s",
+            username,
+            settings.multi_user_enabled,
+            admin_configured,
+            not username and not password,
+        )
+        _record_login_event(db, request, username or "anonymous", success=False, detail="invalid_credentials")
         return RedirectResponse(url="/login?error=Invalid+username+or+password", status_code=302)
 
 
-async def logout(request: Request):
+async def logout(request: Request, db: Session = Depends(get_db)):
     """Handle user logout"""
     user = request.session.get("user")
     username = "unknown"
     if isinstance(user, dict):
         username = user.get("preferred_username") or user.get("email") or "unknown"
     logger.info(f"[SECURITY] LOGOUT user={username}")
+    try:
+        from app.utils.audit_service import record_event
+
+        record_event(
+            db,
+            action="logout",
+            user=username,
+            resource_type="session",
+            ip_address=get_client_ip(request),
+            severity="info",
+        )
+    except Exception:
+        logger.debug("Failed to write logout audit event for user=%s", username, exc_info=True)
     request.session.pop("user", None)
     return RedirectResponse(url="/login?message=You+have+been+logged+out+successfully", status_code=302)
 
@@ -384,6 +953,8 @@ if AUTH_ENABLED:
     router.add_api_route("/login", login, methods=["GET"])
     router.add_api_route("/oauth-login", oauth_login, methods=["GET"])
     router.add_api_route("/oauth-callback", oauth_callback, methods=["GET"])
+    router.add_api_route("/social-login/{provider}", social_login, methods=["GET"])
+    router.add_api_route("/social-callback/{provider}", social_callback, methods=["GET"])
     router.add_api_route("/auth", auth, methods=["POST"])
     router.add_api_route("/logout", logout, methods=["GET"])
 
@@ -392,7 +963,7 @@ if AUTH_ENABLED:
 @require_login
 async def whoami(request: Request):
     """API endpoint to get current user information"""
-    user = request.session.get("user")
+    user = get_current_user(request)
     return user or {"error": "Not authenticated"}
 
 

app/celery_app.py

@@ -1,7 +1,7 @@
 # app/celery_app.py
 
 from celery import Celery
-from celery.signals import task_failure
+from celery.signals import task_failure, worker_ready
 
 from app.config import settings
 
@@ -22,6 +22,14 @@ celery.conf.task_routes = {
 }
 
 
+@worker_ready.connect
+def init_sentry_on_worker_ready(**kwargs):
+    """Initialise Sentry SDK in the Celery worker process."""
+    from app.utils.sentry import init_sentry
+
+    init_sentry(integrations_extra=["celery"])
+
+
 @task_failure.connect
 def task_failure_handler(
     sender=None, task_id=None, exception=None, args=None, kwargs=None, traceback=None, einfo=None, **kw

app/celery_worker.py

@@ -1,5 +1,7 @@
 #!/usr/bin/env python3
 
+import logging
+
 from celery.schedules import crontab
 
 # Ensure tasks are loaded
@@ -9,6 +11,16 @@ from app import tasks  # noqa: F401 - Imports app/tasks.py so Celery can registe
 from app.celery_app import celery
 from app.config import settings
 from app.tasks.backup_tasks import cleanup_old_backups, create_backup  # noqa: F401
+from app.tasks.batch_tasks import (  # noqa: F401
+    backfill_missing_metadata,
+    cleanup_temp_files,
+    expire_shared_links,
+    process_new_documents,
+    prune_old_notifications,
+    prune_processing_logs,
+    reprocess_failed_documents,
+    sync_search_index,
+)
 from app.tasks.check_credentials import check_credentials
 from app.tasks.compute_embedding import backfill_missing_embeddings, compute_document_embedding  # noqa: F401
 from app.tasks.convert_to_pdf import convert_to_pdf  # noqa: F401
@@ -33,11 +45,13 @@ from app.tasks.upload_to_dropbox import upload_to_dropbox  # noqa: F401
 from app.tasks.upload_to_email import upload_to_email  # noqa: F401
 from app.tasks.upload_to_ftp import upload_to_ftp  # noqa: F401
 from app.tasks.upload_to_google_drive import upload_to_google_drive  # noqa: F401
+from app.tasks.upload_to_icloud import upload_to_icloud  # noqa: F401
 from app.tasks.upload_to_nextcloud import upload_to_nextcloud  # noqa: F401
 from app.tasks.upload_to_onedrive import upload_to_onedrive  # noqa: F401
 from app.tasks.upload_to_paperless import upload_to_paperless  # noqa: F401
 from app.tasks.upload_to_s3 import upload_to_s3  # noqa: F401
 from app.tasks.upload_to_sftp import upload_to_sftp  # noqa: F401
+from app.tasks.upload_to_user_integration import upload_to_user_integration  # noqa: F401
 from app.tasks.upload_to_webdav import upload_to_webdav  # noqa: F401
 from app.tasks.upload_with_rclone import send_to_all_rclone_destinations, upload_with_rclone  # noqa: F401
 from app.tasks.uptime_kuma_tasks import ping_uptime_kuma  # noqa: F401
@@ -47,6 +61,8 @@ from app.tasks.webhook_tasks import deliver_webhook_task  # noqa: F401
 # Register the settings reload signal handler so workers pick up config changes
 from app.utils.settings_sync import register_settings_reload_signal
 
+logger = logging.getLogger(__name__)
+
 register_settings_reload_signal()
 
 celery.conf.task_routes = {
@@ -63,15 +79,13 @@ def test_task():
 check_credentials.apply_async(countdown=10)  # Run 10 seconds after worker starts
 
 celery.conf.beat_schedule = {
-    "poll-inboxes-every-minute": (
-        {
-            "task": "app.tasks.imap_tasks.pull_all_inboxes",
-            "schedule": crontab(minute="*/1"),  # every 1 minute
-            "options": {"expires": 55},  # Ensure tasks don't pile up
-        }
-        if (settings.imap1_host or settings.imap2_host)
-        else None
-    ),
+    # IMAP polling — always enabled because per-user IMAP integrations may
+    # exist in the database even when no system-level IMAP hosts are configured.
+    "poll-inboxes-every-minute": {
+        "task": "app.tasks.imap_tasks.pull_all_inboxes",
+        "schedule": crontab(minute="*/1"),  # every 1 minute
+        "options": {"expires": 55},  # Ensure tasks don't pile up
+    },
     # Add Uptime Kuma ping task if configured
     "ping-uptime-kuma": (
         {
@@ -100,27 +114,15 @@ celery.conf.beat_schedule = {
         "schedule": crontab(minute="*/1"),  # Every minute
         "options": {"expires": 55},  # Must complete within 55 seconds
     },
-    # Watch folder scanning — polls local paths, FTP, SFTP, and cloud ingest folders.
+    # Watch folder scanning — always enabled because per-user WATCH_FOLDER
+    # integrations may exist in the database even when no system-level watch
+    # folder settings are configured.
     # Schedule is controlled by WATCH_FOLDER_POLL_INTERVAL (default: 1 minute).
-    "scan-watch-folders": (
-        {
-            "task": "app.tasks.watch_folder_tasks.scan_all_watch_folders",
-            "schedule": crontab(minute=f"*/{max(1, settings.watch_folder_poll_interval)}"),
-            "options": {"expires": 55},
-        }
-        if (
-            settings.watch_folders
-            or settings.ftp_ingest_enabled
-            or settings.sftp_ingest_enabled
-            or settings.dropbox_ingest_enabled
-            or settings.google_drive_ingest_enabled
-            or settings.onedrive_ingest_enabled
-            or settings.nextcloud_ingest_enabled
-            or settings.s3_ingest_enabled
-            or settings.webdav_ingest_enabled
-        )
-        else None
-    ),
+    "scan-watch-folders": {
+        "task": "app.tasks.watch_folder_tasks.scan_all_watch_folders",
+        "schedule": crontab(minute=f"*/{max(1, settings.watch_folder_poll_interval)}"),
+        "options": {"expires": 55},
+    },
     # Backfill embeddings for files that were processed before the
     # embedding pipeline was enabled, or where the embedding task failed.
     "backfill-missing-embeddings": {
@@ -172,3 +174,66 @@ celery.conf.beat_schedule = {
 
 # Remove None entries from beat_schedule
 celery.conf.beat_schedule = {k: v for k, v in celery.conf.beat_schedule.items() if v is not None}
+
+# ---------------------------------------------------------------------------
+# Load admin-managed scheduled jobs from the database
+# ---------------------------------------------------------------------------
+# These jobs are defined in the ``scheduled_jobs`` table (seeded by
+# ``app.api.scheduled_jobs.seed_default_scheduled_jobs``) and can be
+# enabled/disabled and rescheduled via the admin UI at /admin/scheduled-jobs.
+# The schedule is read once at worker startup; changes take effect after
+# the worker is restarted.
+
+
+def _load_db_scheduled_jobs() -> None:
+    """
+    Extend ``celery.conf.beat_schedule`` with entries from the ``scheduled_jobs``
+    database table.
+
+    Only rows with ``enabled=True`` are added.  Rows whose ``name`` key
+    already exists in the static schedule (defined above) are skipped so
+    that hardcoded entries cannot be overridden accidentally.
+
+    Failures are logged as warnings and do not prevent the worker from
+    starting.
+    """
+    try:
+        from app.database import SessionLocal
+        from app.models import ScheduledJob
+
+        with SessionLocal() as db:
+            jobs = db.query(ScheduledJob).filter(ScheduledJob.enabled.is_(True)).all()
+
+        added = 0
+        for job in jobs:
+            if job.name in celery.conf.beat_schedule:
+                # Static entry takes precedence; skip silently.
+                continue
+
+            if job.schedule_type == "interval" and job.interval_seconds:
+                from celery.schedules import schedule as interval_schedule
+
+                sched = interval_schedule(run_every=job.interval_seconds)
+            else:
+                # Default to cron.
+                sched = crontab(
+                    minute=job.cron_minute,
+                    hour=job.cron_hour,
+                    day_of_week=job.cron_day_of_week,
+                    day_of_month=job.cron_day_of_month,
+                    month_of_year=job.cron_month_of_year,
+                )
+
+            celery.conf.beat_schedule[job.name] = {
+                "task": job.task_name,
+                "schedule": sched,
+                "options": {"expires": 3600},
+            }
+            added += 1
+
+        logger.info("Loaded %d scheduled job(s) from database into Celery Beat.", added)
+    except Exception as exc:
+        logger.warning("Could not load scheduled jobs from database: %s", exc)
+
+
+_load_db_scheduled_jobs()

app/cli.py

@@ -0,0 +1,680 @@
+"""DocuElevate command-line interface.
+
+Provides a pipe-friendly CLI for scripting and automation against the
+DocuElevate REST API.  Authentication is via personal API tokens (the
+same tokens managed at ``/api-tokens`` in the web UI).
+
+Usage::
+
+    docuelevate --url http://my-instance --token de_xxx list
+    DOCUELEVATE_URL=http://my-instance DOCUELEVATE_API_TOKEN=de_xxx docuelevate list
+
+Commands
+--------
+upload      Upload one or more local files for processing.
+download    Download a processed (or original) file by ID.
+search      Full-text search across all documents.
+list        List documents with optional filtering.
+token       Sub-commands: create / list / revoke API tokens.
+"""
+
+import json
+import sys
+from pathlib import Path
+from typing import Any
+from urllib.parse import unquote
+
+import click
+import requests
+
+# ---------------------------------------------------------------------------
+# Environment-variable defaults
+# ---------------------------------------------------------------------------
+
+ENV_URL = "DOCUELEVATE_URL"
+ENV_TOKEN = "DOCUELEVATE_API_TOKEN"
+
+_DEFAULT_URL = "http://localhost:8000"
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _build_headers(token: str) -> dict[str, str]:
+    """Return Authorization headers for the given API token."""
+    return {"Authorization": f"Bearer {token}"}
+
+
+def _api(
+    method: str,
+    base_url: str,
+    path: str,
+    token: str,
+    timeout: int = 60,
+    **kwargs: Any,
+) -> requests.Response:
+    """Make an authenticated API request and return the response.
+
+    Args:
+        method: HTTP method (GET, POST, DELETE, …).
+        base_url: The base URL of the DocuElevate instance.
+        path: API path starting with ``/``.
+        token: Plaintext API token.
+        timeout: Request timeout in seconds (default: 60).
+        **kwargs: Extra keyword arguments forwarded to :func:`requests.request`.
+
+    Returns:
+        The :class:`requests.Response` object.
+
+    Raises:
+        click.ClickException: On network errors.
+    """
+    url = base_url.rstrip("/") + path
+    headers = _build_headers(token)
+    try:
+        resp = requests.request(method, url, headers=headers, timeout=timeout, **kwargs)
+    except requests.ConnectionError as exc:
+        raise click.ClickException(f"Could not connect to {base_url}: {exc}") from exc
+    except requests.Timeout as exc:
+        raise click.ClickException(f"Request timed out: {exc}") from exc
+    return resp
+
+
+def _require_ok(resp: requests.Response) -> dict[str, Any] | list[Any]:
+    """Assert a successful HTTP response and return parsed JSON.
+
+    Args:
+        resp: The response to check.
+
+    Returns:
+        Parsed JSON payload.
+
+    Raises:
+        click.ClickException: If the response status indicates an error.
+    """
+    if resp.status_code >= 400:
+        try:
+            detail = resp.json().get("detail", resp.text)
+        except Exception:
+            detail = resp.text
+        raise click.ClickException(f"API error {resp.status_code}: {detail}")
+    try:
+        return resp.json()
+    except Exception:
+        return {}
+
+
+def _output(data: Any, fmt: str) -> None:
+    """Write *data* to stdout in the requested format.
+
+    Args:
+        data: The value to serialise (dict, list, or primitive).
+        fmt: Either ``"json"`` (machine-readable) or ``"table"`` (human-readable).
+    """
+    if fmt == "json":
+        click.echo(json.dumps(data, indent=2, default=str))
+    else:
+        _print_table(data)
+
+
+def _print_table(data: Any) -> None:
+    """Pretty-print a list of dicts as a fixed-width table.
+
+    Falls back to JSON if the data is not a homogeneous list of dicts.
+
+    Args:
+        data: Data to render.
+    """
+    if isinstance(data, dict):
+        # Single-object output — print as key: value pairs
+        for key, value in data.items():
+            click.echo(f"  {key}: {value}")
+        return
+
+    if not isinstance(data, list) or not data:
+        click.echo(json.dumps(data, indent=2, default=str))
+        return
+
+    if not isinstance(data[0], dict):
+        for item in data:
+            click.echo(str(item))
+        return
+
+    # Determine column widths
+    keys = list(data[0].keys())
+    widths: dict[str, int] = {k: len(k) for k in keys}
+    for row in data:
+        for k in keys:
+            widths[k] = max(widths[k], len(str(row.get(k, ""))))
+
+    header = "  ".join(k.upper().ljust(widths[k]) for k in keys)
+    separator = "  ".join("-" * widths[k] for k in keys)
+    click.echo(header)
+    click.echo(separator)
+    for row in data:
+        click.echo("  ".join(str(row.get(k, "")).ljust(widths[k]) for k in keys))
+
+
+# ---------------------------------------------------------------------------
+# Root command group
+# ---------------------------------------------------------------------------
+
+
+@click.group(context_settings={"help_option_names": ["-h", "--help"]})
+@click.option(
+    "--url",
+    envvar=ENV_URL,
+    default=_DEFAULT_URL,
+    show_default=True,
+    show_envvar=True,
+    help="Base URL of the DocuElevate instance.",
+    metavar="URL",
+)
+@click.option(
+    "--token",
+    envvar=ENV_TOKEN,
+    default=None,
+    show_envvar=True,
+    help="API token (de_…).  Required for all commands except help.",
+    metavar="TOKEN",
+)
+@click.option(
+    "--format",
+    "fmt",
+    type=click.Choice(["table", "json"], case_sensitive=False),
+    default="table",
+    show_default=True,
+    help="Output format.  Use 'json' for machine-readable / pipe-friendly output.",
+)
+@click.option(
+    "--timeout",
+    default=60,
+    show_default=True,
+    envvar="DOCUELEVATE_TIMEOUT",
+    show_envvar=True,
+    type=int,
+    help="HTTP request timeout in seconds.",
+)
+@click.version_option(package_name="docuelevate", prog_name="docuelevate")
+@click.pass_context
+def cli(ctx: click.Context, url: str, token: str | None, fmt: str, timeout: int) -> None:
+    """DocuElevate CLI — interact with DocuElevate from the command line.
+
+    Configure the target instance and credentials via options or environment
+    variables:
+
+    \b
+      DOCUELEVATE_URL            Base URL of the instance (default: http://localhost:8000)
+      DOCUELEVATE_API_TOKEN      Personal API token (de_…)
+      DOCUELEVATE_TIMEOUT        HTTP request timeout in seconds (default: 60)
+
+    Examples:
+
+    \b
+      # Upload a file
+      docuelevate --token de_xxx upload report.pdf
+
+    \b
+      # List files as JSON for further processing
+      docuelevate --token de_xxx --format json list | jq '.[].original_filename'
+
+    \b
+      # Search for invoices
+      docuelevate --token de_xxx search "invoice amazon"
+    """
+    ctx.ensure_object(dict)
+    ctx.obj["url"] = url
+    ctx.obj["token"] = token
+    ctx.obj["fmt"] = fmt
+    ctx.obj["timeout"] = timeout
+
+
+def _get_token(ctx: click.Context) -> str:
+    """Return the token from context, raising ClickException if absent.
+
+    Args:
+        ctx: The current Click context.
+
+    Returns:
+        The API token string.
+
+    Raises:
+        click.ClickException: If no token has been provided.
+    """
+    token = ctx.obj.get("token")
+    if not token:
+        raise click.ClickException(f"No API token provided.  Use --token or set the {ENV_TOKEN} environment variable.")
+    return token
+
+
+# ---------------------------------------------------------------------------
+# list command
+# ---------------------------------------------------------------------------
+
+
+@cli.command("list")
+@click.option("--page", default=1, show_default=True, help="Page number.")
+@click.option("--per-page", default=25, show_default=True, help="Items per page (max 200).")
+@click.option("--search", default=None, help="Filter by filename substring.")
+@click.option("--mime-type", default=None, help="Filter by MIME type (e.g. application/pdf).")
+@click.option("--status", "file_status", default=None, help="Filter by status: pending, processing, completed, failed.")
+@click.option("--sort-by", default="created_at", show_default=True, help="Sort field.")
+@click.option("--sort-order", type=click.Choice(["asc", "desc"]), default="desc", show_default=True)
+@click.pass_context
+def list_files(
+    ctx: click.Context,
+    page: int,
+    per_page: int,
+    search: str | None,
+    mime_type: str | None,
+    file_status: str | None,
+    sort_by: str,
+    sort_order: str,
+) -> None:
+    """List documents stored in DocuElevate.
+
+    Examples:
+
+    \b
+      docuelevate list
+      docuelevate list --status completed --per-page 10
+      docuelevate --format json list | jq '.[].original_filename'
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    fmt: str = ctx.obj["fmt"]
+    timeout: int = ctx.obj["timeout"]
+
+    params: dict[str, Any] = {
+        "page": page,
+        "per_page": per_page,
+        "sort_by": sort_by,
+        "sort_order": sort_order,
+    }
+    if search:
+        params["search"] = search
+    if mime_type:
+        params["mime_type"] = mime_type
+    if file_status:
+        params["status"] = file_status
+
+    resp = _api("GET", url, "/api/files", token, timeout=timeout, params=params)
+    payload = _require_ok(resp)
+
+    # Extract the list from the paginated response
+    files: list[dict[str, Any]] = payload.get("files", payload) if isinstance(payload, dict) else payload  # type: ignore[assignment]
+    pagination: dict[str, Any] = payload.get("pagination", {}) if isinstance(payload, dict) else {}
+
+    if fmt == "json":
+        _output(files, fmt)
+    else:
+        # Trim fields for readable table
+        rows = [
+            {
+                "id": f.get("id"),
+                "filename": f.get("original_filename"),
+                "size": f.get("file_size"),
+                "status": f.get("status"),
+                "created_at": str(f.get("created_at", ""))[:19],
+            }
+            for f in files
+        ]
+        _output(rows, fmt)
+        if pagination:
+            click.echo(f"\nPage {pagination.get('page')}/{pagination.get('pages')}  ({pagination.get('total')} total)")
+
+
+# ---------------------------------------------------------------------------
+# upload command
+# ---------------------------------------------------------------------------
+
+
+@cli.command("upload")
+@click.argument("files", nargs=-1, required=True, type=click.Path(exists=True, readable=True))
+@click.option(
+    "--batch-size",
+    default=5,
+    show_default=True,
+    help="Maximum number of concurrent uploads (sequential when 1).",
+)
+@click.pass_context
+def upload_files(ctx: click.Context, files: tuple[str, ...], batch_size: int) -> None:
+    """Upload one or more local files for processing.
+
+    Supports glob patterns and multiple arguments for batch uploads.
+
+    Examples:
+
+    \b
+      docuelevate upload report.pdf
+      docuelevate upload *.pdf invoice_*.png
+      docuelevate upload --batch-size 3 /scans/*.pdf
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    fmt: str = ctx.obj["fmt"]
+    timeout: int = ctx.obj["timeout"]
+
+    results: list[dict[str, Any]] = []
+    failed = 0
+
+    for i, file_path in enumerate(files, 1):
+        path = Path(file_path)
+        click.echo(f"[{i}/{len(files)}] Uploading {path.name}…", err=True)
+        try:
+            with path.open("rb") as fh:
+                resp = _api(
+                    "POST",
+                    url,
+                    "/api/ui-upload",
+                    token,
+                    timeout=timeout,
+                    files={"file": (path.name, fh)},
+                )
+            if resp.status_code >= 400:
+                try:
+                    detail = resp.json().get("detail", resp.text)
+                except Exception:
+                    detail = resp.text
+                click.echo(f"  ERROR {resp.status_code}: {detail}", err=True)
+                results.append({"file": path.name, "status": "error", "detail": detail})
+                failed += 1
+            else:
+                data = resp.json()
+                results.append({"file": path.name, "status": "queued", **data})
+                click.echo(f"  OK  task_id={data.get('task_id', '?')}", err=True)
+        except click.ClickException:
+            raise
+        except Exception as exc:
+            click.echo(f"  ERROR: {exc}", err=True)
+            results.append({"file": path.name, "status": "error", "detail": str(exc)})
+            failed += 1
+
+    _output(results, fmt)
+
+    if failed:
+        click.echo(f"\n{failed}/{len(files)} upload(s) failed.", err=True)
+        sys.exit(1)
+
+
+# ---------------------------------------------------------------------------
+# download command
+# ---------------------------------------------------------------------------
+
+
+@cli.command("download")
+@click.argument("file_id", type=int)
+@click.option(
+    "--output",
+    "-o",
+    default=None,
+    help="Destination file path.  Defaults to the server-provided filename in the current directory.",
+    type=click.Path(),
+)
+@click.option(
+    "--version",
+    type=click.Choice(["processed", "original"]),
+    default="processed",
+    show_default=True,
+    help="Which version to download.",
+)
+@click.pass_context
+def download_file(ctx: click.Context, file_id: int, output: str | None, version: str) -> None:
+    """Download a file by its numeric ID.
+
+    Examples:
+
+    \b
+      docuelevate download 42
+      docuelevate download 42 --version original -o /tmp/orig.pdf
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    timeout: int = ctx.obj["timeout"]
+
+    resp = _api(
+        "GET",
+        url,
+        f"/api/files/{file_id}/download",
+        token,
+        timeout=timeout,
+        params={"version": version},
+        stream=True,
+    )
+    _require_ok(resp)
+
+    # Determine output filename
+    if output:
+        dest = Path(output)
+    else:
+        content_disp = resp.headers.get("content-disposition", "")
+        filename = f"file_{file_id}"
+        for raw_part in content_disp.split(";"):
+            clean = raw_part.strip()
+            if clean.startswith("filename="):
+                filename = clean[len("filename=") :].strip('"').strip("'")
+                break
+            if clean.startswith("filename*="):
+                raw = clean[len("filename*=") :]
+                if raw.upper().startswith("UTF-8''"):
+                    filename = unquote(raw[7:])
+                    break
+        dest = Path(filename)
+
+    with dest.open("wb") as fh:
+        for chunk in resp.iter_content(chunk_size=65536):
+            fh.write(chunk)
+
+    click.echo(f"Downloaded {dest} ({dest.stat().st_size} bytes)")
+
+
+# ---------------------------------------------------------------------------
+# search command
+# ---------------------------------------------------------------------------
+
+
+@cli.command("search")
+@click.argument("query")
+@click.option("--mime-type", default=None, help="Filter by MIME type.")
+@click.option("--document-type", default=None, help="Filter by document type (e.g. Invoice).")
+@click.option("--tags", default=None, help="Filter by tag.")
+@click.option("--language", default=None, help="Filter by language code (e.g. en, de).")
+@click.option("--page", default=1, show_default=True)
+@click.option("--per-page", default=20, show_default=True, help="Results per page (max 100).")
+@click.pass_context
+def search(
+    ctx: click.Context,
+    query: str,
+    mime_type: str | None,
+    document_type: str | None,
+    tags: str | None,
+    language: str | None,
+    page: int,
+    per_page: int,
+) -> None:
+    """Full-text search across all documents.
+
+    Examples:
+
+    \b
+      docuelevate search "invoice amazon"
+      docuelevate search "contract" --document-type Contract --language en
+      docuelevate --format json search "receipt" | jq '.[].file_id'
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    fmt: str = ctx.obj["fmt"]
+    timeout: int = ctx.obj["timeout"]
+
+    params: dict[str, Any] = {"q": query, "page": page, "per_page": per_page}
+    if mime_type:
+        params["mime_type"] = mime_type
+    if document_type:
+        params["document_type"] = document_type
+    if tags:
+        params["tags"] = tags
+    if language:
+        params["language"] = language
+
+    resp = _api("GET", url, "/api/search", token, timeout=timeout, params=params)
+    payload = _require_ok(resp)
+
+    results: list[dict[str, Any]] = (
+        payload.get("results", payload) if isinstance(payload, dict) else payload  # type: ignore[assignment]
+    )
+    total: int = payload.get("total", len(results)) if isinstance(payload, dict) else len(results)
+    pages: int = payload.get("pages", 1) if isinstance(payload, dict) else 1
+
+    if fmt == "json":
+        _output(results, fmt)
+    else:
+        rows = [
+            {
+                "file_id": r.get("file_id"),
+                "filename": r.get("original_filename"),
+                "type": r.get("document_type"),
+                "tags": ",".join(r.get("tags") or []),
+            }
+            for r in results
+        ]
+        _output(rows, fmt)
+        click.echo(f"\nPage {page}/{pages}  ({total} total results)")
+
+
+# ---------------------------------------------------------------------------
+# token sub-group
+# ---------------------------------------------------------------------------
+
+
+@cli.group("token")
+@click.pass_context
+def token_group(ctx: click.Context) -> None:
+    """Manage personal API tokens.
+
+    Tokens can be created, listed, and revoked.  Token rotation is achieved
+    by creating a new token before revoking the old one.
+
+    Examples:
+
+    \b
+      docuelevate token create "CI Pipeline"
+      docuelevate token list
+      docuelevate token revoke 3
+    """
+
+
+@token_group.command("create")
+@click.argument("name")
+@click.pass_context
+def token_create(ctx: click.Context, name: str) -> None:
+    """Create a new personal API token.
+
+    The full token value is printed exactly once.  Store it securely.
+
+    Examples:
+
+    \b
+      docuelevate token create "My script"
+      docuelevate --format json token create "CI" | jq -r '.token'
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    fmt: str = ctx.obj["fmt"]
+    timeout: int = ctx.obj["timeout"]
+
+    resp = _api("POST", url, "/api/api-tokens/", token, timeout=timeout, json={"name": name})
+    payload = _require_ok(resp)
+
+    if fmt == "json":
+        _output(payload, fmt)
+    else:
+        if not isinstance(payload, dict):
+            raise click.ClickException("Unexpected API response format.")
+        click.echo("Token created successfully:")
+        click.echo(f"  ID:     {payload.get('id')}")
+        click.echo(f"  Name:   {payload.get('name')}")
+        click.echo(f"  Prefix: {payload.get('token_prefix')}")
+        click.echo(f"  Token:  {payload.get('token')}")
+        click.echo()
+        click.echo("Store this token securely — it will not be shown again.", err=True)
+
+
+@token_group.command("list")
+@click.pass_context
+def token_list(ctx: click.Context) -> None:
+    """List all your API tokens (active and revoked).
+
+    Examples:
+
+    \b
+      docuelevate token list
+      docuelevate --format json token list | jq '.[] | select(.is_active)'
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    fmt: str = ctx.obj["fmt"]
+    timeout: int = ctx.obj["timeout"]
+
+    resp = _api("GET", url, "/api/api-tokens/", token, timeout=timeout)
+    payload = _require_ok(resp)
+
+    if fmt == "json":
+        _output(payload, fmt)
+    else:
+        if not isinstance(payload, list):
+            raise click.ClickException("Unexpected API response format.")
+        rows = [
+            {
+                "id": t.get("id"),
+                "name": t.get("name"),
+                "prefix": t.get("token_prefix"),
+                "active": t.get("is_active"),
+                "last_used": str(t.get("last_used_at") or "never")[:19],
+                "created": str(t.get("created_at") or "")[:19],
+            }
+            for t in payload
+        ]
+        _output(rows, fmt)
+
+
+@token_group.command("revoke")
+@click.argument("token_id", type=int)
+@click.option("--yes", "-y", is_flag=True, help="Skip confirmation prompt.")
+@click.pass_context
+def token_revoke(ctx: click.Context, token_id: int, yes: bool) -> None:
+    """Revoke an API token by its numeric ID.
+
+    The token is soft-deleted (kept for audit) but immediately invalidated.
+
+    Examples:
+
+    \b
+      docuelevate token revoke 3
+      docuelevate token revoke 3 --yes
+    """
+    token = _get_token(ctx)
+    url: str = ctx.obj["url"]
+    timeout: int = ctx.obj["timeout"]
+
+    if not yes:
+        click.confirm(f"Revoke token {token_id}?", abort=True)
+
+    resp = _api("DELETE", url, f"/api/api-tokens/{token_id}", token, timeout=timeout)
+    _require_ok(resp)
+    click.echo(f"Token {token_id} revoked.")
+
+
+# ---------------------------------------------------------------------------
+# Entry point
+# ---------------------------------------------------------------------------
+
+
+def main() -> None:
+    """Entry point for the ``docuelevate`` console script."""
+    cli(auto_envvar_prefix="DOCUELEVATE")  # type: ignore[call-arg]
+
+
+if __name__ == "__main__":
+    main()

app/config.py

@@ -49,18 +49,30 @@ class Settings(BaseSettings):
     debug: bool = False  # Default to False
 
     # Making Dropbox optional
+    dropbox_enabled: bool = Field(
+        default=True,
+        description="Enable Dropbox as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     dropbox_app_key: Optional[str] = None
     dropbox_app_secret: Optional[str] = None
     dropbox_folder: Optional[str] = None
     dropbox_refresh_token: Optional[str] = None
 
     # Making Nextcloud optional
+    nextcloud_enabled: bool = Field(
+        default=True,
+        description="Enable Nextcloud as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     nextcloud_upload_url: Optional[str] = None
     nextcloud_username: Optional[str] = None
     nextcloud_password: Optional[str] = None
     nextcloud_folder: Optional[str] = None
 
     # Making Paperless optional
+    paperless_enabled: bool = Field(
+        default=True,
+        description="Enable Paperless-ngx as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     paperless_ngx_api_token: Optional[str] = None
     paperless_host: Optional[str] = None
     paperless_custom_field_absender: Optional[str] = None  # Name of the "absender" custom field in Paperless
@@ -166,12 +178,44 @@ class Settings(BaseSettings):
         ),
     )
 
-    # Authentik
+    # Authentik / Generic OIDC
     authentik_client_id: Optional[str] = None
     authentik_client_secret: Optional[str] = None
     authentik_config_url: Optional[str] = None
     oauth_provider_name: Optional[str] = None  # Name to display for the OAuth provider
 
+    # Social Login Providers
+    # Google OAuth2
+    social_auth_google_enabled: bool = False
+    social_auth_google_client_id: Optional[str] = None
+    social_auth_google_client_secret: Optional[str] = None
+
+    # Microsoft OAuth2 (Azure AD / Microsoft Entra ID)
+    social_auth_microsoft_enabled: bool = False
+    social_auth_microsoft_client_id: Optional[str] = None
+    social_auth_microsoft_client_secret: Optional[str] = None
+    social_auth_microsoft_tenant: str = Field(
+        default="common",
+        description=(
+            "Azure AD tenant ID or one of 'common', 'organizations', 'consumers'. "
+            "Use 'common' to allow any Microsoft account and any Azure AD org. "
+            "Use a specific tenant ID (GUID) to restrict to a single organization. "
+            "Default: common."
+        ),
+    )
+
+    # Apple Sign-In
+    social_auth_apple_enabled: bool = False
+    social_auth_apple_client_id: Optional[str] = None
+    social_auth_apple_team_id: Optional[str] = None
+    social_auth_apple_key_id: Optional[str] = None
+    social_auth_apple_private_key: Optional[str] = None
+
+    # Dropbox OAuth2
+    social_auth_dropbox_enabled: bool = False
+    social_auth_dropbox_client_id: Optional[str] = None
+    social_auth_dropbox_client_secret: Optional[str] = None
+
     # Local user signup
     allow_local_signup: bool = Field(
         default=False,
@@ -394,6 +438,10 @@ class Settings(BaseSettings):
     imap2_delete_after_process: bool = False
 
     # Google Drive settings
+    google_drive_enabled: bool = Field(
+        default=True,
+        description="Enable Google Drive as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     google_drive_credentials_json: Optional[str] = ""
     google_drive_folder_id: Optional[str] = ""
     google_drive_delegate_to: Optional[str] = ""  # Optional delegated user email
@@ -405,6 +453,10 @@ class Settings(BaseSettings):
     google_drive_refresh_token: Optional[str] = ""
 
     # WebDAV settings
+    webdav_enabled: bool = Field(
+        default=True,
+        description="Enable WebDAV as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     webdav_url: Optional[str] = None
     webdav_username: Optional[str] = None
     webdav_password: Optional[str] = None
@@ -412,6 +464,10 @@ class Settings(BaseSettings):
     webdav_verify_ssl: bool = True
 
     # FTP settings
+    ftp_enabled: bool = Field(
+        default=True,
+        description="Enable FTP as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     ftp_host: Optional[str] = None
     ftp_port: Optional[int] = 21
     ftp_username: Optional[str] = None
@@ -421,6 +477,10 @@ class Settings(BaseSettings):
     ftp_allow_plaintext: bool = True  # Default to allowing plaintext fallback
 
     # SFTP settings
+    sftp_enabled: bool = Field(
+        default=True,
+        description="Enable SFTP as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     sftp_host: Optional[str] = None
     sftp_port: Optional[int] = 22
     sftp_username: Optional[str] = None
@@ -442,6 +502,10 @@ class Settings(BaseSettings):
     email_default_recipient: Optional[str] = None
 
     # Email destination settings (dedicated SMTP for document delivery – decoupled from shared email above)
+    dest_email_enabled: bool = Field(
+        default=True,
+        description="Enable Email as an upload destination. Set to False to disable document delivery via email even when credentials are configured.",
+    )
     dest_email_host: Optional[str] = None
     dest_email_port: Optional[int] = 587
     dest_email_username: Optional[str] = None
@@ -451,6 +515,10 @@ class Settings(BaseSettings):
     dest_email_default_recipient: Optional[str] = None  # Fallback recipient for document delivery
 
     # OneDrive settings
+    onedrive_enabled: bool = Field(
+        default=True,
+        description="Enable OneDrive as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     onedrive_client_id: Optional[str] = None
     onedrive_client_secret: Optional[str] = None
     onedrive_tenant_id: Optional[str] = "common"  # Default to "common" for personal accounts
@@ -458,6 +526,10 @@ class Settings(BaseSettings):
     onedrive_folder_path: Optional[str] = None
 
     # AWS S3 settings
+    s3_enabled: bool = Field(
+        default=True,
+        description="Enable Amazon S3 as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
     aws_access_key_id: Optional[str] = None
     aws_secret_access_key: Optional[str] = None
     aws_region: Optional[str] = "us-east-1"  # Default region
@@ -466,6 +538,16 @@ class Settings(BaseSettings):
     s3_storage_class: Optional[str] = "STANDARD"  # Default storage class
     s3_acl: Optional[str] = "private"  # Default ACL
 
+    # iCloud Drive settings
+    icloud_enabled: bool = Field(
+        default=True,
+        description="Enable iCloud Drive as an upload destination. Set to False to disable uploads even when credentials are configured.",
+    )
+    icloud_username: Optional[str] = None  # Apple ID email address
+    icloud_password: Optional[str] = None  # App-specific password (required for 2FA accounts)
+    icloud_folder: Optional[str] = None  # Target folder path in iCloud Drive (e.g. "Documents/Uploads")
+    icloud_cookie_directory: Optional[str] = None  # Directory for session cookies (default: ~/.pyicloud)
+
     # Uptime Kuma settings
     uptime_kuma_url: Optional[str] = None
     uptime_kuma_ping_interval: int = 5  # Default ping interval in minutes
@@ -484,6 +566,14 @@ class Settings(BaseSettings):
 
     # Feature flags
     allow_file_delete: bool = True  # Default to allowing file deletion from database
+    compliance_enabled: bool = Field(
+        default=True,
+        description=(
+            "Enable the compliance templates dashboard (GDPR, HIPAA, SOC 2). "
+            "When enabled, admins can view compliance status and apply "
+            "pre-built regulatory configurations. Default: True."
+        ),
+    )
 
     # PDF/A archival conversion settings
     enable_pdfa_conversion: bool = Field(
@@ -557,6 +647,17 @@ class Settings(BaseSettings):
         ),
     )
 
+    imap_attachment_filter: str = Field(
+        default="documents_only",
+        description=(
+            "Controls which attachment types are ingested from IMAP emails. "
+            "Accepted values: "
+            "'documents_only' – ingest only PDFs and office files (Word, Excel, PowerPoint, ODT, etc.); "
+            "'all' – ingest all supported file types including images. "
+            "This is the global default; individual user IMAP accounts can override it."
+        ),
+    )
+
     # Batch processing settings
     processall_throttle_threshold: int = Field(
         default=20,
@@ -849,6 +950,49 @@ class Settings(BaseSettings):
         ),
     )
 
+    # SIEM / External Audit Log Forwarding
+    # Forward audit events to external SIEM systems for centralised monitoring.
+    audit_siem_enabled: bool = Field(
+        default=False,
+        description="Enable forwarding of audit events to an external SIEM system.",
+    )
+    audit_siem_transport: str = Field(
+        default="syslog",
+        description=(
+            "Transport used to forward audit events. "
+            "Options: 'syslog' (RFC 5424 over UDP/TCP), 'http' (JSON POST to a webhook URL, "
+            "compatible with Splunk HEC, Logstash HTTP input, Grafana Loki, etc.)."
+        ),
+    )
+    audit_siem_syslog_host: str = Field(
+        default="localhost",
+        description="Hostname or IP of the syslog receiver.",
+    )
+    audit_siem_syslog_port: int = Field(
+        default=514,
+        description="Port of the syslog receiver.",
+    )
+    audit_siem_syslog_protocol: str = Field(
+        default="udp",
+        description="Protocol for syslog transport: 'udp' or 'tcp'.",
+    )
+    audit_siem_http_url: str = Field(
+        default="",
+        description=(
+            "HTTP endpoint URL for SIEM webhook delivery. "
+            "Supports Splunk HEC (https://splunk:8088/services/collector/event), "
+            "Logstash HTTP input, Grafana Loki push API, or any JSON-accepting endpoint."
+        ),
+    )
+    audit_siem_http_token: str = Field(
+        default="",
+        description="Bearer / HEC token included in the Authorization header of SIEM HTTP requests.",
+    )
+    audit_siem_http_custom_headers: str = Field(
+        default="",
+        description="Comma-separated 'Key:Value' pairs of extra headers for SIEM HTTP requests.",
+    )
+
     # UI / Appearance
     ui_default_color_scheme: str = Field(
         default="system",
@@ -914,6 +1058,77 @@ class Settings(BaseSettings):
         description="Allowed request headers for CORS. Use ['*'] to allow all headers.",
     )
 
+    # ---------------------------------------------------------------------------
+    # Support / Help Center – Zammad integration
+    # ---------------------------------------------------------------------------
+    zammad_url: Optional[str] = Field(
+        default=None,
+        description=(
+            "Base URL of your Zammad instance (e.g. https://zammad.example.com). "
+            "Required for the chat widget and feedback form on the Help Center page."
+        ),
+    )
+    zammad_chat_enabled: bool = Field(
+        default=False,
+        description="Show the Zammad live-chat widget on the Help Center page.",
+    )
+    zammad_chat_id: int = Field(
+        default=1,
+        description="Zammad chat topic ID to use for the live-chat widget.",
+    )
+    zammad_form_enabled: bool = Field(
+        default=False,
+        description="Show the Zammad feedback / ticket form on the Help Center page.",
+    )
+    support_email: Optional[str] = Field(
+        default=None,
+        description="Support e-mail address displayed on the Help Center page.",
+    )
+
+    # ---------------------------------------------------------------------------
+    # Observability – Sentry error & performance monitoring
+    # ---------------------------------------------------------------------------
+    sentry_dsn: Optional[str] = Field(
+        default=None,
+        description=(
+            "Sentry Data Source Name (DSN).  When set, error reporting and "
+            "performance tracing are enabled automatically.  Leave blank (or unset) "
+            "to disable Sentry entirely."
+        ),
+    )
+    sentry_environment: str = Field(
+        default="production",
+        description=(
+            "Environment tag sent to Sentry (e.g. 'development', 'staging', 'production'). "
+            "Helps you filter events in the Sentry dashboard."
+        ),
+    )
+    sentry_traces_sample_rate: float = Field(
+        default=0.1,
+        description=(
+            "Fraction of transactions to capture for performance monitoring (0.0–1.0). "
+            "Set to 0.0 to disable tracing, 1.0 to capture every transaction. "
+            "Values above 0 may increase Sentry quota usage."
+        ),
+    )
+    sentry_profiles_sample_rate: float = Field(
+        default=0.0,
+        description=(
+            "Fraction of profiled transactions to send to Sentry (0.0–1.0). "
+            "Profiling is only active when traces_sample_rate > 0. "
+            "Defaults to 0.0 (disabled) to minimise overhead."
+        ),
+    )
+    sentry_send_default_pii: bool = Field(
+        default=False,
+        description=(
+            "Whether to attach personally identifiable information (PII) – such as "
+            "IP addresses and user agents – to Sentry events.  Disabled by default "
+            "for privacy compliance (GDPR / CCPA).  Enable only if your Sentry "
+            "project is configured to handle PII."
+        ),
+    )
+
     @model_validator(mode="before")
     @classmethod
     def strip_outer_quotes(cls, data: Any) -> Any:

app/database.py

@@ -210,8 +210,10 @@ def _run_schema_migrations(engine: Any) -> None:
             if unique_filehash_indexes:
                 logger.info("Migrating files: dropping unique index on 'filehash'")
                 with engine.begin() as conn:
+                    preparer = conn.dialect.identifier_preparer
                     for index in unique_filehash_indexes:
-                        conn.execute(text(f"DROP INDEX IF EXISTS {index['name']}"))
+                        quoted_idx = preparer.quote(index["name"])
+                        conn.execute(text(f"DROP INDEX IF EXISTS {quoted_idx}"))
                 logger.info("Migration complete: unique index on 'filehash' removed")
         except Exception as exc:
             logger.warning(f"Skipping filehash unique index drop: {exc}")
@@ -263,12 +265,16 @@ def _ensure_indexes(engine: Any, inspector: Any) -> None:
     table_names = inspector.get_table_names()
     columns_by_table: dict[str, set[str]] = {}
     with engine.begin() as conn:
+        preparer = conn.dialect.identifier_preparer
         for idx_name, table, column in _PERF_INDEXES:
             if table in table_names:
                 if table not in columns_by_table:
                     columns_by_table[table] = {col["name"] for col in inspector.get_columns(table)}
                 if column in columns_by_table[table]:
-                    conn.execute(text(f"CREATE INDEX IF NOT EXISTS {idx_name} ON {table} ({column})"))
+                    quoted_idx = preparer.quote(idx_name)
+                    quoted_table = preparer.quote(table)
+                    quoted_col = preparer.quote(column)
+                    conn.execute(text(f"CREATE INDEX IF NOT EXISTS {quoted_idx} ON {quoted_table} ({quoted_col})"))
 
     logger.info("Performance indexes ensured")
 

app/main.py

@@ -16,6 +16,7 @@ from starlette.middleware.trustedhost import TrustedHostMiddleware
 from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware
 
 from app.api import router as api_router
+from app.api.graphql_api import graphql_router
 from app.api.local_auth import router as local_auth_router
 from app.auth import router as auth_router
 from app.config import settings
@@ -27,6 +28,7 @@ from app.middleware.request_size_limit import RequestSizeLimitMiddleware
 from app.middleware.security_headers import SecurityHeadersMiddleware
 from app.utils.config_validator import check_all_configs
 from app.utils.notification import init_apprise, notify_shutdown, notify_startup
+from app.utils.sentry import init_sentry
 
 # Import the routers - now using views directly instead of frontend
 from app.views import router as frontend_router
@@ -70,6 +72,10 @@ async def lifespan(app: FastAPI):
     finally:
         db.close()
 
+    # Initialize Sentry after DB settings are loaded so that values configured
+    # via the database UI (e.g. SENTRY_DSN) are respected in addition to env vars.
+    init_sentry()
+
     # Ensure OCR language data is available (background download, non-blocking)
     from app.utils.ocr_language_manager import ensure_ocr_languages_async
 
@@ -127,6 +133,34 @@ async def lifespan(app: FastAPI):
     except Exception:
         logging.debug("Default pipeline seeding skipped — DB may not be ready yet")  # noqa: S110
 
+    # Seed the default scheduled batch processing jobs so they appear in the
+    # admin UI (/admin/scheduled-jobs) on first startup.
+    try:
+        from app.api.scheduled_jobs import seed_default_scheduled_jobs as _seed_jobs
+        from app.database import SessionLocal as _SessionLocal  # noqa: F811
+
+        _db_jobs = _SessionLocal()
+        try:
+            _seed_jobs(_db_jobs)
+        finally:
+            _db_jobs.close()
+    except Exception:
+        logging.debug("Scheduled jobs seeding skipped — DB may not be ready yet")  # noqa: S110
+
+    # Seed the built-in compliance templates (GDPR, HIPAA, SOC2) so they
+    # are available in the admin compliance dashboard on first startup.
+    try:
+        from app.database import SessionLocal as _SessionLocal  # noqa: F811
+        from app.utils.compliance_service import seed_compliance_templates as _seed_compliance
+
+        _db_compliance = _SessionLocal()
+        try:
+            _seed_compliance(_db_compliance)
+        finally:
+            _db_compliance.close()
+    except Exception:
+        logging.debug("Compliance template seeding skipped — DB may not be ready yet")  # noqa: S110
+
     # Application is now running
     yield
 
@@ -137,7 +171,12 @@ async def lifespan(app: FastAPI):
     notify_shutdown()
 
 
-app = FastAPI(title="DocuElevate", lifespan=lifespan)
+app = FastAPI(
+    title="DocuElevate",
+    lifespan=lifespan,
+    docs_url="/admin/api-docs",
+    redoc_url="/admin/api-redoc",
+)
 
 # Initialize rate limiter and attach to app state
 limiter = create_limiter(redis_url=settings.redis_url, enabled=settings.rate_limiting_enabled)
@@ -202,17 +241,36 @@ if os.path.exists(static_dir):
 else:
     print(f"WARNING: Static directory not found at {static_dir}. Static files will not be served.")
 
-# Mount the built MkDocs documentation site at /help/
+# Mount the built MkDocs developer documentation at /developer-docs/
+# These docs target administrators and developers, not end-users.
+# The user-facing Help Center is served by the /help view instead.
 # The docs are pre-built into docs_build/ during the Docker image build.
 # When running locally, run `mkdocs build` from the repo root first.
 docs_build_dir = pathlib.Path(__file__).parents[1] / "docs_build"
 if os.path.exists(docs_build_dir):
-    app.mount("/help", StaticFiles(directory=str(docs_build_dir), html=True), name="help_docs")
+    app.mount("/developer-docs", StaticFiles(directory=str(docs_build_dir), html=True), name="developer_docs")
 else:
-    print(f"INFO: Help docs not found at {docs_build_dir}. Run 'mkdocs build' to generate them.")
+    print(f"INFO: Developer docs not found at {docs_build_dir}. Run 'mkdocs build' to generate them.")
 
 
 # Custom exception handlers that return JSON for API routes and HTML for frontend routes
+# These use their own separate templates instance so that patches in tests on individual
+# view modules do not affect the error handler rendering.
+_error_templates_dir = pathlib.Path(__file__).parents[1] / "frontend" / "templates"
+_error_templates = Jinja2Templates(directory=str(_error_templates_dir))
+# Register the i18n translate helper as a global so error templates can use {{ _("key") }}.
+# Error pages use the default language (English); request-specific locale is not needed here.
+from app.utils.i18n import SUPPORTED_LANGUAGES as _SUPPORTED_LANGUAGES  # noqa: E402
+from app.utils.i18n import get_suggested_languages as _get_suggested_languages  # noqa: E402
+from app.utils.i18n import translate as _translate_fn  # noqa: E402
+
+_error_templates.env.globals["_"] = lambda key, **kwargs: _translate_fn(key, "en", **kwargs)
+_error_templates.env.globals["min"] = min
+_error_templates.env.globals["max"] = max
+_error_templates.env.globals["supported_languages"] = _SUPPORTED_LANGUAGES
+_error_templates.env.globals["suggested_languages"] = _get_suggested_languages("en", "")
+
+
 @app.exception_handler(HTTPException)
 async def http_exception_handler(request: Request, exc: HTTPException):
     """
@@ -224,15 +282,15 @@ async def http_exception_handler(request: Request, exc: HTTPException):
         return JSONResponse(status_code=exc.status_code, content={"detail": exc.detail})
 
     # For frontend routes, return appropriate HTML templates
-    templates = Jinja2Templates(directory=str(static_dir.parent / "templates"))
-
     # Handle 404 errors with a custom template
     if exc.status_code == 404:
-        return templates.TemplateResponse("404.html", {"request": request}, status_code=status.HTTP_404_NOT_FOUND)
+        return _error_templates.TemplateResponse(
+            "404.html", {"request": request}, status_code=status.HTTP_404_NOT_FOUND
+        )
 
     # For other HTTP errors, we could create specific templates or use a generic one
     # For now, return a simple error page
-    return templates.TemplateResponse(
+    return _error_templates.TemplateResponse(
         "404.html",  # Reuse 404 template for other errors, or create a generic error template
         {"request": request},
         status_code=exc.status_code,
@@ -253,8 +311,7 @@ async def custom_500_handler(request: Request, exc: Exception):
         )
 
     # Serve the 500 template for non-API routes
-    templates = Jinja2Templates(directory=str(static_dir.parent / "templates"))
-    return templates.TemplateResponse(
+    return _error_templates.TemplateResponse(
         "500.html",
         {"request": request, "exc": exc},
         status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
@@ -272,3 +329,4 @@ app.include_router(files_router)  # Explicitly include the files router
 app.include_router(auth_router)
 app.include_router(local_auth_router)
 app.include_router(api_router, prefix="/api")
+app.include_router(graphql_router, prefix="/graphql")

app/middleware/csrf.py

@@ -109,6 +109,13 @@ class CSRFMiddleware(BaseHTTPMiddleware):
 
         # Validate for state-changing methods on non-exempt paths.
         if request.method in CSRF_PROTECTED_METHODS and request.url.path not in CSRF_EXEMPT_PATHS:
+            # Bearer-authenticated requests (API tokens) are exempt from CSRF
+            # because the token itself acts as proof of intent — it cannot be
+            # injected by a cross-site request from a browser.
+            auth_header = request.headers.get("authorization", "")
+            if auth_header.startswith("Bearer "):
+                return await call_next(request)
+
             submitted_token = await self._get_submitted_token(request)
             if not submitted_token or not secrets.compare_digest(csrf_token, submitted_token):
                 logger.warning(f"[SECURITY] CSRF_VALIDATION_FAILED method={request.method} path={request.url.path}")
@@ -148,10 +155,26 @@ class CSRFMiddleware(BaseHTTPMiddleware):
 
         # 2. For URL-encoded form bodies only (plain HTML form submissions).
         content_type = request.headers.get("content-type", "")
+        logger.debug("CSRF: content_type=%r method=%s path=%s", content_type, request.method, request.url.path)
         if "application/x-www-form-urlencoded" in content_type:
             try:
+                # Cache the raw body bytes before parsing the form.  Starlette's
+                # BaseHTTPMiddleware uses _CachedRequest.wrapped_receive to relay
+                # the body to downstream handlers.  When form() is called it
+                # internally uses stream() which sets _stream_consumed=True but
+                # does NOT populate _body.  wrapped_receive then sees a consumed
+                # stream and forwards an empty body, so the auth endpoint gets
+                # form_keys=[].  Calling body() first stores the bytes in _body;
+                # wrapped_receive detects this and replays the real body to any
+                # downstream handler (e.g. the /auth endpoint).
+                await request.body()
                 form = await request.form()
                 token = form.get("csrf_token")
+                logger.debug(
+                    "CSRF: form_keys=%s csrf_token_present=%s",
+                    list(form.keys()),
+                    bool(token),
+                )
                 if token:
                     return str(token)
             except Exception as exc:

app/models.py

@@ -7,6 +7,7 @@ from app.database import Base
 # Foreign key constants
 _FILES_ID_FK = "files.id"
 _PIPELINES_ID_FK = "pipelines.id"
+_ROUTING_RULES_TABLE = "pipeline_routing_rules"
 
 
 class DocumentMetadata(Base):
@@ -146,6 +147,27 @@ class SettingsAuditLog(Base):
     action = Column(String, nullable=False)  # "update" or "delete"
 
 
+class AuditLog(Base):
+    """Comprehensive audit log for compliance tracking.
+
+    Records all significant actions: login/logout, document CRUD, settings
+    changes, and administrative operations.  Rows are append-only; the API
+    and service layer never update or delete entries.
+    """
+
+    __tablename__ = "audit_logs"
+
+    id = Column(Integer, primary_key=True, index=True)
+    timestamp = Column(DateTime(timezone=True), server_default=func.now(), nullable=False, index=True)
+    user = Column(String, nullable=False, index=True)  # Username or "anonymous" / "system"
+    action = Column(String, nullable=False, index=True)  # e.g. "login", "document.create", "settings.update"
+    resource_type = Column(String, nullable=True, index=True)  # e.g. "document", "user", "settings"
+    resource_id = Column(String, nullable=True)  # ID of the affected resource
+    ip_address = Column(String, nullable=True)  # Client IP address
+    details = Column(Text, nullable=True)  # JSON-encoded extra context
+    severity = Column(String(16), nullable=False, server_default="info")  # info / warning / error / critical
+
+
 class SavedSearch(Base):
     """User-defined saved search filters for quick access to frequently used filter combinations."""
 
@@ -256,6 +278,17 @@ class UserProfile(Base):
     preferred_destination = Column(String(50), nullable=True)
     stripe_customer_id = Column(String(64), nullable=True)
 
+    # UI language preference for i18n (ISO 639-1 code, e.g. "en", "de", "fr")
+    # NULL means "auto-detect from browser Accept-Language header"
+    preferred_language = Column(String(10), nullable=True)
+
+    # UI colour scheme preference: "light" | "dark" | "system" (NULL = "system")
+    preferred_theme = Column(String(10), nullable=True)
+
+    # Custom profile avatar stored as a base64 data-URL (e.g. "data:image/png;base64,...")
+    # NULL means use the Gravatar fallback derived from the user's e-mail address.
+    avatar_data = Column(Text, nullable=True)
+
     created_at = Column(DateTime(timezone=True), server_default=func.now())
     updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
 
@@ -376,6 +409,103 @@ class PipelineStep(Base):
     updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
 
 
+class ImapIngestionProfile(Base):
+    """Named ingestion profile controlling which attachment types are accepted from IMAP emails.
+
+    Profiles group file-type categories (e.g. "pdf", "office", "images") so users
+    can precisely control what gets ingested from each mailbox.
+
+    System-provided built-in profiles (``is_builtin=True``) are seeded by the
+    migration and cannot be deleted or renamed.  Users may create their own profiles
+    (``owner_id`` set to their identifier) or rely on the global system profiles
+    (``owner_id=None``).
+
+    ``allowed_categories`` stores a JSON list of category strings, e.g.::
+
+        '["pdf", "office", "opendocument", "text", "web"]'
+
+    Valid category names are defined in ``app.utils.allowed_types.FILE_TYPE_CATEGORIES``.
+    """
+
+    __tablename__ = "imap_ingestion_profiles"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Human-readable profile name (e.g. "Documents Only", "Documents + Images")
+    name = Column(String(255), nullable=False)
+
+    # Optional description shown in the UI
+    description = Column(Text, nullable=True)
+
+    # Owner of this profile. NULL = global/system profile available to all users.
+    owner_id = Column(String, nullable=True, index=True)
+
+    # JSON-encoded list of enabled category keys.  Example: '["pdf","office","text"]'
+    # See FILE_TYPE_CATEGORIES in app/utils/allowed_types.py for valid values.
+    allowed_categories = Column(Text, nullable=False, default='["pdf","office","opendocument","text","web"]')
+
+    # Built-in system profiles that cannot be deleted or modified via the API.
+    is_builtin = Column(Boolean, nullable=False, default=False)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class UserImapAccount(Base):
+    """Per-user IMAP ingestion account.
+
+    Each row represents one IMAP mailbox that a user wants DocuElevate to
+    poll for document attachments.  The periodic ``pull_all_inboxes`` Celery
+    task iterates over all active accounts and processes any new emails.
+
+    Quota enforcement: the user's subscription plan's ``max_mailboxes`` field
+    controls how many accounts a user may configure (0 = unlimited for paid
+    plans; free tier is not permitted any accounts).
+    """
+
+    __tablename__ = "user_imap_accounts"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Stable owner identifier — matches FileRecord.owner_id
+    owner_id = Column(String, nullable=False, index=True)
+
+    # Human-readable label chosen by the user (e.g. "Work Gmail", "Scanner mailbox")
+    name = Column(String(255), nullable=False)
+
+    # IMAP connection settings
+    host = Column(String(255), nullable=False)
+    port = Column(Integer, nullable=False, default=993)
+    username = Column(String(255), nullable=False)
+    # Password stored encrypted using Fernet symmetric encryption via
+    # app.utils.encryption.encrypt_value / decrypt_value (keyed from SESSION_SECRET).
+    # New records are always encrypted; legacy plaintext records are transparently
+    # handled by decrypt_value which returns the value unchanged when no "enc:" prefix
+    # is present.
+    password = Column(String(1024), nullable=False)
+    use_ssl = Column(Boolean, nullable=False, default=True)
+
+    # Processing options
+    # When True, emails are deleted from the mailbox after their attachments are processed
+    delete_after_process = Column(Boolean, nullable=False, default=False)
+
+    # Optional reference to an ImapIngestionProfile.
+    # NULL means "use the global imap_attachment_filter setting" (system default).
+    profile_id = Column(Integer, ForeignKey("imap_ingestion_profiles.id"), nullable=True)
+
+    # When False the account is not polled by the periodic task (but not deleted)
+    is_active = Column(Boolean, nullable=False, default=True)
+
+    # Last time this mailbox was successfully polled
+    last_checked_at = Column(DateTime(timezone=True), nullable=True)
+
+    # Last error message if the most recent poll failed (NULL = last poll succeeded)
+    last_error = Column(Text, nullable=True)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
 class BackupRecord(Base):
     """Tracks database backup files and their retention metadata.
 
@@ -418,3 +548,476 @@ class BackupRecord(Base):
     remote_path = Column(String(1024), nullable=True)
 
     created_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+
+
+# ---------------------------------------------------------------------------
+# Integration direction / type constants
+# ---------------------------------------------------------------------------
+
+
+class IntegrationDirection:
+    """Direction of data flow for a UserIntegration."""
+
+    SOURCE = "SOURCE"
+    DESTINATION = "DESTINATION"
+
+    ALL = {SOURCE, DESTINATION}
+
+
+class IntegrationType:
+    """Supported integration types for UserIntegration."""
+
+    # Source integrations (ingestion)
+    IMAP = "IMAP"
+    WATCH_FOLDER = "WATCH_FOLDER"
+    WEBHOOK = "WEBHOOK"
+
+    # Destination integrations (storage / output)
+    S3 = "S3"
+    DROPBOX = "DROPBOX"
+    GOOGLE_DRIVE = "GOOGLE_DRIVE"
+    ONEDRIVE = "ONEDRIVE"
+    WEBDAV = "WEBDAV"
+    NEXTCLOUD = "NEXTCLOUD"
+    FTP = "FTP"
+    SFTP = "SFTP"
+    EMAIL = "EMAIL"
+    PAPERLESS = "PAPERLESS"
+    RCLONE = "RCLONE"
+    ICLOUD = "ICLOUD"
+
+    ALL = {
+        IMAP,
+        WATCH_FOLDER,
+        WEBHOOK,
+        S3,
+        DROPBOX,
+        GOOGLE_DRIVE,
+        ONEDRIVE,
+        WEBDAV,
+        NEXTCLOUD,
+        FTP,
+        SFTP,
+        EMAIL,
+        PAPERLESS,
+        RCLONE,
+        ICLOUD,
+    }
+
+
+class UserIntegration(Base):
+    """Generic per-user integration record (source or destination).
+
+    Replaces ad-hoc per-integration-type tables with a single, extensible
+    model that supports any combination of ingestion sources and storage
+    destinations without schema changes when new integrations are added.
+
+    ``config`` holds non-sensitive connection settings as a JSON string
+    (e.g. host, port, bucket name, folder path).
+
+    ``credentials`` holds sensitive secrets (passwords, tokens, API keys)
+    as a Fernet-encrypted JSON string.  Always use
+    ``app.utils.encryption.encrypt_value`` / ``decrypt_value`` when
+    writing / reading this field.
+
+    Example config + credentials shapes by integration type:
+
+    IMAP:
+      config      = {"host": "imap.example.com", "port": 993,
+                     "username": "user@example.com", "use_ssl": true,
+                     "delete_after_process": false,
+                     "gmail_apply_labels": true}
+      credentials = {"password": "secret"}
+
+    WATCH_FOLDER (local):
+      config      = {"source_type": "local",
+                     "folder_path": "/data/inbox",
+                     "delete_after_process": false}
+
+    WATCH_FOLDER (s3):
+      config      = {"source_type": "s3", "bucket": "my-bucket",
+                     "region": "us-east-1", "prefix": "inbox/",
+                     "endpoint_url": null, "delete_after_process": false}
+      credentials = {"access_key_id": "AKI…", "secret_access_key": "…"}
+
+    WATCH_FOLDER (dropbox):
+      config      = {"source_type": "dropbox",
+                     "folder_path": "/Inbox/Scanner",
+                     "delete_after_process": false}
+      credentials = {"refresh_token": "…", "app_key": "…",
+                     "app_secret": "…"}
+
+    WATCH_FOLDER (google_drive):
+      config      = {"source_type": "google_drive",
+                     "folder_id": "1abc…",
+                     "delete_after_process": false}
+      credentials = {"credentials_json": "{…service-account…}"}
+
+    WATCH_FOLDER (onedrive):
+      config      = {"source_type": "onedrive",
+                     "folder_path": "/Documents/Inbox",
+                     "delete_after_process": false}
+      credentials = {"refresh_token": "…", "client_id": "…",
+                     "client_secret": "…"}
+
+    WATCH_FOLDER (nextcloud):
+      config      = {"source_type": "nextcloud",
+                     "url": "https://cloud.example.com",
+                     "folder_path": "/Documents/Inbox",
+                     "delete_after_process": false}
+      credentials = {"username": "user", "password": "secret"}
+
+    WATCH_FOLDER (webdav):
+      config      = {"source_type": "webdav",
+                     "url": "https://webdav.example.com/dav/",
+                     "folder_path": "/remote.php/webdav/Inbox",
+                     "delete_after_process": false}
+      credentials = {"username": "user", "password": "secret"}
+
+    S3:
+      config      = {"bucket": "my-bucket", "region": "us-east-1",
+                     "endpoint_url": null, "folder_prefix": ""}
+      credentials = {"access_key_id": "AKI…", "secret_access_key": "…"}
+
+    DROPBOX:
+      config      = {"folder": "/DocuElevate"}
+      credentials = {"refresh_token": "…", "app_key": "…",
+                     "app_secret": "…"}
+
+    GOOGLE_DRIVE:
+      config      = {"folder_id": "1abc…"}
+      credentials = {"credentials_json": "{…service-account or OAuth…}"}
+
+    WEBDAV / NEXTCLOUD:
+      config      = {"url": "https://cloud.example.com/dav/",
+                     "folder": "/Documents"}
+      credentials = {"username": "user", "password": "secret"}
+    """
+
+    __tablename__ = "user_integrations"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Stable owner identifier — matches FileRecord.owner_id / UserImapAccount.owner_id
+    owner_id = Column(String, nullable=False, index=True)
+
+    # "SOURCE" or "DESTINATION"  (see IntegrationDirection)
+    direction = Column(String(20), nullable=False, index=True)
+
+    # One of the IntegrationType constants (e.g. "IMAP", "S3", "DROPBOX")
+    integration_type = Column(String(50), nullable=False, index=True)
+
+    # Human-readable label chosen by the user (e.g. "Work Gmail", "S3 Archive")
+    name = Column(String(255), nullable=False)
+
+    # Non-sensitive connection configuration (JSON string)
+    config = Column(Text, nullable=True)
+
+    # Sensitive credentials — always stored encrypted via encrypt_value()
+    credentials = Column(Text, nullable=True)
+
+    # When False the integration is not polled / used by background tasks
+    is_active = Column(Boolean, nullable=False, default=True)
+
+    # Timestamp of the last successful use of this integration
+    last_used_at = Column(DateTime(timezone=True), nullable=True)
+
+    # Last error message if the most recent operation failed (NULL = last op succeeded)
+    last_error = Column(Text, nullable=True)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class ApiToken(Base):
+    """Personal API token for programmatic access.
+
+    Users can create multiple tokens, each with a human-readable name.
+    Only the SHA-256 hash of the token is stored; the plaintext is shown
+    exactly once at creation time.  A short prefix (first 8 chars) is
+    persisted for easy identification in the UI.
+
+    Usage tracking records the timestamp and IP address of the most
+    recent request that used the token.
+    """
+
+    __tablename__ = "api_tokens"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Stable owner identifier — matches FileRecord.owner_id / UserIntegration.owner_id
+    owner_id = Column(String, nullable=False, index=True)
+
+    # Human-readable label chosen by the user (e.g. "CI Pipeline", "Webhook Upload")
+    name = Column(String(255), nullable=False)
+
+    # SHA-256 hex digest of the full token value
+    token_hash = Column(String(64), nullable=False, unique=True, index=True)
+
+    # First 12 characters of the token for display (e.g. "de_Ab3xY7kL…")
+    token_prefix = Column(String(16), nullable=False)
+
+    # Usage tracking
+    last_used_at = Column(DateTime(timezone=True), nullable=True)
+    last_used_ip = Column(String(45), nullable=True)  # IPv6 max length
+
+    is_active = Column(Boolean, nullable=False, default=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    revoked_at = Column(DateTime(timezone=True), nullable=True)
+
+
+class SharedLink(Base):
+    """Shareable, time-limited or view-limited document link.
+
+    A ``SharedLink`` grants unauthenticated access to one ``FileRecord``
+    via a cryptographically random URL token.  The link may optionally
+    expire after a given datetime, be limited to a fixed number of views,
+    and require a password.  Only a PBKDF2-HMAC-SHA256 hash of the
+    password is stored.
+
+    Owners can view and revoke their active links from the management UI.
+    """
+
+    __tablename__ = "shared_links"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Unique URL-safe token — forms the public /share/<token> URL.
+    token = Column(String(64), nullable=False, unique=True, index=True)
+
+    # File this link grants access to.
+    file_id = Column(Integer, ForeignKey(_FILES_ID_FK), nullable=False, index=True)
+
+    # Owner who created the link (matches FileRecord.owner_id).
+    owner_id = Column(String, nullable=False, index=True)
+
+    # Optional human-readable description chosen by the creator.
+    label = Column(String(255), nullable=True)
+
+    # Time-based expiration (NULL = never expires).
+    expires_at = Column(DateTime(timezone=True), nullable=True)
+
+    # View-count limit (NULL = unlimited).
+    max_views = Column(Integer, nullable=True)
+
+    # Cumulative view count (incremented on every successful access).
+    view_count = Column(Integer, nullable=False, default=0)
+
+    # Optional password protection — stores PBKDF2-HMAC-SHA256 hex digest.
+    password_hash = Column(String(128), nullable=True)
+
+    # Whether the link is still valid (set to False to revoke immediately).
+    is_active = Column(Boolean, nullable=False, default=True)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    revoked_at = Column(DateTime(timezone=True), nullable=True)
+
+
+class UserNotificationTarget(Base):
+    """Per-user notification target (email or webhook channel)."""
+
+    __tablename__ = "user_notification_targets"
+
+    id = Column(Integer, primary_key=True, index=True)
+    owner_id = Column(String, nullable=False, index=True)
+    channel_type = Column(String(20), nullable=False)  # "email" or "webhook"
+    name = Column(String(255), nullable=False)  # Human-readable label
+    config = Column(Text, nullable=True)  # JSON: smtp config or webhook url
+    is_active = Column(Boolean, nullable=False, default=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class UserNotificationPreference(Base):
+    """Mapping: which user events trigger which notification channel."""
+
+    __tablename__ = "user_notification_preferences"
+
+    id = Column(Integer, primary_key=True, index=True)
+    owner_id = Column(String, nullable=False, index=True)
+    event_type = Column(String(50), nullable=False)  # "document.processed", "document.failed"
+    channel_type = Column(String(20), nullable=False)  # "in_app", "email", "webhook"
+    target_id = Column(Integer, nullable=True)  # NULL = in_app, else UserNotificationTarget.id
+    is_enabled = Column(Boolean, nullable=False, default=True)
+
+    __table_args__ = (UniqueConstraint("owner_id", "event_type", "channel_type", "target_id"),)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class InAppNotification(Base):
+    """In-app notification record for the bell icon / inbox."""
+
+    __tablename__ = "in_app_notifications"
+
+    id = Column(Integer, primary_key=True, index=True)
+    owner_id = Column(String, nullable=False, index=True)
+    event_type = Column(String(50), nullable=False)  # "document.processed", "document.failed"
+    title = Column(String(255), nullable=False)
+    message = Column(Text, nullable=True)
+    is_read = Column(Boolean, nullable=False, default=False, index=True)
+    file_id = Column(Integer, nullable=True)  # Optional link to FileRecord
+    created_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
+
+
+class ScheduledJob(Base):
+    """
+    Configuration record for an admin-managed scheduled batch processing job.
+
+    Each row represents one recurring job entry.  The Celery Beat schedule is
+    built from these rows at worker startup; changes take effect after the
+    worker process is restarted.
+
+    Schedule types
+    --------------
+    - ``"cron"``     – standard cron expression fields (minute/hour/…)
+    - ``"interval"`` – fixed interval in seconds (e.g. 3600 for hourly)
+    """
+
+    __tablename__ = "scheduled_jobs"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Unique machine-readable key used as the Celery Beat schedule entry name.
+    name = Column(String(100), unique=True, nullable=False, index=True)
+
+    # Human-readable display name shown in the admin UI.
+    display_name = Column(String(255), nullable=False)
+
+    # Short description of what the job does.
+    description = Column(Text, nullable=True)
+
+    # Fully-qualified Celery task name, e.g. "app.tasks.batch_tasks.process_new_documents".
+    task_name = Column(String(255), nullable=False)
+
+    # Whether the job is active.  Inactive jobs are excluded from the beat schedule.
+    enabled = Column(Boolean, nullable=False, default=True)
+
+    # Schedule type: "cron" or "interval".
+    schedule_type = Column(String(20), nullable=False, default="cron")
+
+    # --- Cron fields (used when schedule_type == "cron") ---
+    cron_minute = Column(String(50), nullable=False, default="0")
+    cron_hour = Column(String(50), nullable=False, default="*")
+    cron_day_of_week = Column(String(50), nullable=False, default="*")
+    cron_day_of_month = Column(String(50), nullable=False, default="*")
+    cron_month_of_year = Column(String(50), nullable=False, default="*")
+
+    # --- Interval field (used when schedule_type == "interval") ---
+    # Interval in seconds; e.g. 3600 = hourly, 86400 = daily.
+    interval_seconds = Column(Integer, nullable=True)
+
+    # Timestamps populated by the worker after each execution.
+    last_run_at = Column(DateTime(timezone=True), nullable=True)
+    last_run_status = Column(String(20), nullable=True)  # "success", "failed", "running"
+    last_run_detail = Column(Text, nullable=True)  # Brief result summary or error
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class MobileDevice(Base):
+    """Registered mobile device for push notifications.
+
+    Stores the push token (Expo push token, FCM token, or APNs token) for a
+    specific user device so that document-processing events can be forwarded
+    as push notifications to the native mobile app.
+    """
+
+    __tablename__ = "mobile_devices"
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # User that owns this device registration.
+    owner_id = Column(String, nullable=False, index=True)
+
+    # Human-readable name the user gave this device (e.g. "John's iPhone").
+    device_name = Column(String(255), nullable=True)
+
+    # Platform: "ios", "android", or "web".
+    platform = Column(String(20), nullable=False, default="ios")
+
+    # Expo push token (ExponentPushToken[…]) or raw FCM/APNs token.
+    push_token = Column(String(512), nullable=False)
+
+    # Whether push notifications are enabled for this device.
+    is_active = Column(Boolean, nullable=False, default=True)
+
+    # Timestamps.
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    last_seen_at = Column(DateTime(timezone=True), nullable=True)
+
+    __table_args__ = (UniqueConstraint("owner_id", "push_token", name="uq_mobile_device_owner_token"),)
+
+
+class ComplianceTemplate(Base):
+    """Pre-built compliance configuration templates (GDPR, HIPAA, SOC2).
+
+    Each row represents an applied compliance template.  The ``settings_json``
+    column stores the concrete setting key/value pairs that were written when
+    the template was applied.  ``status`` tracks the current compliance posture.
+    """
+
+    __tablename__ = "compliance_templates"
+
+    id = Column(Integer, primary_key=True, index=True)
+    name = Column(String(50), unique=True, nullable=False, index=True)  # GDPR, HIPAA, SOC2
+    display_name = Column(String(100), nullable=False)
+    description = Column(Text, nullable=True)
+    settings_json = Column(Text, nullable=False, default="{}")  # JSON of applied settings
+    enabled = Column(Boolean, nullable=False, default=False)
+    status = Column(String(20), nullable=False, default="not_applied")  # not_applied, compliant, partial, non_compliant
+    applied_at = Column(DateTime(timezone=True), nullable=True)
+    applied_by = Column(String(255), nullable=True)
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
+
+
+class PipelineRoutingRule(Base):
+    """Conditional routing rule that assigns documents to pipelines.
+
+    Rules are evaluated in ascending ``position`` order for a given owner.
+    The first rule whose condition matches the document properties wins and
+    the document is routed to ``target_pipeline_id``.  If no rule matches,
+    the caller falls back to the owner's (or system) default pipeline.
+
+    Supported fields:
+        file_type, document_type, category, filename, size, and any key
+        inside the AI-extracted metadata JSON (prefixed ``metadata.``).
+
+    Supported operators:
+        equals, not_equals, contains, not_contains, regex, gt, lt, gte, lte.
+    """
+
+    __tablename__ = _ROUTING_RULES_TABLE
+
+    id = Column(Integer, primary_key=True, index=True)
+
+    # Owner of this rule.  NULL = system-wide rule (admin only).
+    owner_id = Column(String, nullable=True, index=True)
+
+    # Human-readable label for the rule.
+    name = Column(String(255), nullable=False)
+
+    # Evaluation order (lower = earlier).  First matching rule wins.
+    position = Column(Integer, nullable=False, default=0)
+
+    # The document property to evaluate.
+    # Built-in: file_type, document_type, category, filename, size.
+    # For AI metadata fields, use the "metadata.<key>" prefix.
+    field = Column(String(255), nullable=False)
+
+    # Comparison operator.
+    operator = Column(String(50), nullable=False)
+
+    # Value to compare against (always stored as text; cast as needed).
+    value = Column(String(1024), nullable=False)
+
+    # Target pipeline when the condition matches.
+    target_pipeline_id = Column(Integer, ForeignKey(_PIPELINES_ID_FK), nullable=False, index=True)
+
+    # Soft-disable without deleting.
+    is_active = Column(Boolean, nullable=False, default=True)
+
+    created_at = Column(DateTime(timezone=True), server_default=func.now())
+    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

app/tasks/batch_tasks.py

@@ -0,0 +1,669 @@
+"""
+Scheduled batch processing tasks for DocuElevate.
+
+This module provides Celery tasks that can be scheduled via Celery Beat
+and managed through the admin UI (``/admin/scheduled-jobs``):
+
+Core batch jobs
+---------------
+- ``process_new_documents``      – Queue documents that have never been processed.
+- ``reprocess_failed_documents`` – Re-queue documents whose processing failed.
+- ``cleanup_temp_files``         – Remove stale files from the ``workdir/tmp`` directory.
+
+Maintenance / housekeeping jobs
+--------------------------------
+- ``expire_shared_links``        – Auto-revoke SharedLinks whose ``expires_at`` has passed.
+- ``prune_processing_logs``      – Delete old rows from ``processing_logs`` and
+                                   ``settings_audit_log`` to prevent unbounded table growth.
+- ``prune_old_notifications``    – Delete old read ``in_app_notifications`` rows.
+- ``backfill_missing_metadata``  – Re-trigger AI metadata extraction for completed files
+                                   that have OCR text but no ``ai_metadata``.
+- ``sync_search_index``          – Index documents in Meilisearch that have OCR text /
+                                   metadata but are not yet in the search index.
+
+Each task records its execution result back to the ``ScheduledJob`` table so
+the admin UI can display last-run times and statuses.
+"""
+
+import json
+import logging
+import os
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+from app.celery_app import celery
+from app.config import settings
+from app.database import SessionLocal
+from app.models import (
+    FileProcessingStep,
+    FileRecord,
+    InAppNotification,
+    ProcessingLog,
+    ScheduledJob,
+    SettingsAuditLog,
+    SharedLink,
+)
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Internal helpers
+# ---------------------------------------------------------------------------
+
+_PROCESSING_STEPS = {
+    "create_file_record",
+    "check_text",
+    "extract_text",
+    "process_with_ocr",
+    "process_with_azure_document_intelligence",
+    "extract_metadata_with_gpt",
+    "embed_metadata_into_pdf",
+    "finalize_document_storage",
+    "send_to_all_destinations",
+}
+
+
+def _update_job_status(job_name: str, status: str, detail: str) -> None:
+    """Persist run status back to the ScheduledJob row for display in the UI."""
+    try:
+        with SessionLocal() as db:
+            job = db.query(ScheduledJob).filter(ScheduledJob.name == job_name).first()
+            if job:
+                job.last_run_at = datetime.now(timezone.utc)
+                job.last_run_status = status
+                job.last_run_detail = detail
+                db.commit()
+    except Exception as exc:  # pragma: no cover – best-effort status update
+        logger.warning("Could not update ScheduledJob status for %s: %s", job_name, exc)
+
+
+# ---------------------------------------------------------------------------
+# Task: process new (unprocessed) documents
+# ---------------------------------------------------------------------------
+
+
+@celery.task(name="app.tasks.batch_tasks.process_new_documents")
+def process_new_documents() -> dict:
+    """
+    Queue all documents that have never been processed.
+
+    A document is considered *new* when it has no ``FileProcessingStep`` rows
+    that match the core pipeline steps.  The task loads each qualifying
+    ``FileRecord``, verifies that the original file still exists on disk, and
+    dispatches ``process_document`` for each one.
+
+    Returns a summary dict with ``queued`` and ``skipped`` counts.
+    """
+    from app.tasks.process_document import process_document  # avoid circular import
+
+    job_name = "process-new-documents"
+    logger.info("[batch] Starting process_new_documents task")
+
+    try:
+        with SessionLocal() as db:
+            # Files that already have at least one processing step recorded.
+            processed_file_ids = (
+                db.query(FileProcessingStep.file_id)
+                .filter(FileProcessingStep.step_name.in_(_PROCESSING_STEPS))
+                .distinct()
+                .subquery()
+            )
+
+            # Candidate files: non-duplicate records with no processing steps yet.
+            candidates = (
+                db.query(FileRecord)
+                .filter(FileRecord.is_duplicate.is_(False))
+                .filter(~FileRecord.id.in_(db.query(processed_file_ids.c.file_id)))
+                .all()
+            )
+
+        queued = 0
+        skipped = 0
+        for record in candidates:
+            if not record.local_filename or not os.path.exists(record.local_filename):
+                logger.warning(
+                    "[batch] Skipping file_id=%s — local file not found: %s",
+                    record.id,
+                    record.local_filename,
+                )
+                skipped += 1
+                continue
+            process_document.delay(
+                record.local_filename,
+                original_filename=record.original_filename,
+                file_id=record.id,
+                owner_id=record.owner_id,
+            )
+            queued += 1
+
+        detail = f"Queued {queued} document(s) for processing; skipped {skipped} (file not on disk)."
+        logger.info("[batch] process_new_documents: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"queued": queued, "skipped": skipped}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] process_new_documents failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"queued": 0, "skipped": 0, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: reprocess failed documents
+# ---------------------------------------------------------------------------
+
+
+@celery.task(name="app.tasks.batch_tasks.reprocess_failed_documents")
+def reprocess_failed_documents() -> dict:
+    """
+    Re-queue documents whose most-recent processing attempt failed.
+
+    Only files that have at least one ``FileProcessingStep`` with
+    ``status == "failure"`` **and** no currently ``in_progress`` steps are
+    selected so that actively-running jobs are not interrupted.
+
+    Returns a summary dict with ``queued`` and ``skipped`` counts.
+    """
+    from app.tasks.process_document import process_document  # avoid circular import
+
+    job_name = "reprocess-failed-documents"
+    logger.info("[batch] Starting reprocess_failed_documents task")
+
+    try:
+        with SessionLocal() as db:
+            # Files with at least one failed step.
+            failed_file_ids = (
+                db.query(FileProcessingStep.file_id)
+                .filter(FileProcessingStep.step_name.in_(_PROCESSING_STEPS))
+                .filter(FileProcessingStep.status == "failure")
+                .distinct()
+                .subquery()
+            )
+
+            # Exclude files that are currently being processed.
+            in_progress_file_ids = (
+                db.query(FileProcessingStep.file_id)
+                .filter(FileProcessingStep.status == "in_progress")
+                .distinct()
+                .subquery()
+            )
+
+            candidates = (
+                db.query(FileRecord)
+                .filter(FileRecord.is_duplicate.is_(False))
+                .filter(FileRecord.id.in_(db.query(failed_file_ids.c.file_id)))
+                .filter(~FileRecord.id.in_(db.query(in_progress_file_ids.c.file_id)))
+                .all()
+            )
+
+        queued = 0
+        skipped = 0
+        for record in candidates:
+            if not record.local_filename or not os.path.exists(record.local_filename):
+                logger.warning(
+                    "[batch] Skipping file_id=%s — local file not found: %s",
+                    record.id,
+                    record.local_filename,
+                )
+                skipped += 1
+                continue
+            process_document.delay(
+                record.local_filename,
+                original_filename=record.original_filename,
+                file_id=record.id,
+                owner_id=record.owner_id,
+            )
+            queued += 1
+
+        detail = f"Re-queued {queued} failed document(s); skipped {skipped} (file not on disk)."
+        logger.info("[batch] reprocess_failed_documents: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"queued": queued, "skipped": skipped}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] reprocess_failed_documents failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"queued": 0, "skipped": 0, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: clean up temporary files
+# ---------------------------------------------------------------------------
+
+#: Files in ``workdir/tmp`` that are older than this threshold are deleted.
+_TEMP_FILE_MAX_AGE_HOURS: int = 24
+
+
+@celery.task(name="app.tasks.batch_tasks.cleanup_temp_files")
+def cleanup_temp_files(max_age_hours: int = _TEMP_FILE_MAX_AGE_HOURS) -> dict:
+    """
+    Delete stale files from the ``workdir/tmp`` directory.
+
+    A file is considered stale when **both** of the following are true:
+
+    1. Its modification time is older than *max_age_hours* (default 24 h).
+    2. No ``FileRecord.local_filename`` points to it  **or**  the file is not
+       referenced by any active in-progress processing step.
+
+    This prevents accidental deletion of files that are being actively
+    processed by the pipeline.
+
+    Args:
+        max_age_hours: Minimum age (in hours) before a temp file is eligible
+                       for deletion.  Defaults to 24.
+
+    Returns:
+        A summary dict with ``deleted`` and ``skipped`` counts.
+    """
+    job_name = "cleanup-temp-files"
+    logger.info("[batch] Starting cleanup_temp_files (max_age_hours=%s)", max_age_hours)
+
+    tmp_dir = Path(settings.workdir) / "tmp"
+    if not tmp_dir.exists():
+        detail = "workdir/tmp does not exist; nothing to clean."
+        logger.info("[batch] cleanup_temp_files: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"deleted": 0, "skipped": 0}
+
+    cutoff = datetime.now(timezone.utc) - timedelta(hours=max_age_hours)
+
+    deleted = 0
+    skipped = 0
+    errors = 0
+
+    try:
+        with SessionLocal() as db:
+            # Collect filenames actively referenced by in-progress processing steps.
+            in_progress_filenames: set[str] = set()
+            in_progress_records = (
+                db.query(FileRecord.local_filename)
+                .join(FileProcessingStep, FileProcessingStep.file_id == FileRecord.id)
+                .filter(FileProcessingStep.status == "in_progress")
+                .distinct()
+                .all()
+            )
+            for row in in_progress_records:
+                if row.local_filename:
+                    in_progress_filenames.add(os.path.basename(row.local_filename))
+
+            # Also collect all filenames referenced by FileRecord.local_filename
+            # that point into workdir/tmp (files still in the tmp pipeline).
+            active_tmp_filenames: set[str] = set()
+            tmp_dir_str = str(tmp_dir.resolve())
+            active_records = (
+                db.query(FileRecord.local_filename).filter(FileRecord.local_filename.like(f"{tmp_dir_str}%")).all()
+            )
+            for row in active_records:
+                if row.local_filename:
+                    active_tmp_filenames.add(os.path.basename(row.local_filename))
+
+        protected_basenames = in_progress_filenames | active_tmp_filenames
+
+        for entry in tmp_dir.iterdir():
+            if not entry.is_file():
+                continue
+
+            # Check modification time.
+            try:
+                mtime = datetime.fromtimestamp(entry.stat().st_mtime, tz=timezone.utc)
+            except OSError:  # pragma: no cover – only reachable if file vanishes between iterdir() and stat()
+                skipped += 1
+                continue
+
+            if mtime >= cutoff:
+                skipped += 1
+                continue
+
+            if entry.name in protected_basenames:
+                logger.debug("[batch] cleanup_temp_files: keeping protected file %s", entry.name)
+                skipped += 1
+                continue
+
+            try:
+                entry.unlink()
+                logger.debug("[batch] cleanup_temp_files: deleted %s", entry)
+                deleted += 1
+            except OSError as exc:
+                logger.warning("[batch] cleanup_temp_files: could not delete %s: %s", entry, exc)
+                errors += 1
+
+        detail = f"Deleted {deleted} stale temp file(s); skipped {skipped} (too new or protected); {errors} error(s)."
+        status = "failed" if errors and not deleted else "success"
+        logger.info("[batch] cleanup_temp_files: %s", detail)
+        _update_job_status(job_name, status, detail)
+        return {"deleted": deleted, "skipped": skipped, "errors": errors}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] cleanup_temp_files failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"deleted": 0, "skipped": 0, "errors": 1, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: expire stale shared links
+# ---------------------------------------------------------------------------
+
+
+@celery.task(name="app.tasks.batch_tasks.expire_shared_links")
+def expire_shared_links() -> dict:
+    """
+    Auto-revoke SharedLinks whose ``expires_at`` timestamp has passed.
+
+    The ``_is_link_valid`` helper in the shared-links API already blocks
+    access at request time, but the database rows remain flagged as
+    ``is_active=True``.  This task sweeps those rows and sets
+    ``is_active=False`` + ``revoked_at`` so the management UI reflects
+    the true state and counts are accurate.
+
+    Returns a summary dict with ``revoked`` count.
+    """
+    job_name = "expire-shared-links"
+    logger.info("[batch] Starting expire_shared_links task")
+
+    try:
+        now = datetime.now(timezone.utc)
+        with SessionLocal() as db:
+            stale = (
+                db.query(SharedLink)
+                .filter(
+                    SharedLink.is_active.is_(True),
+                    SharedLink.expires_at.isnot(None),
+                    SharedLink.expires_at < now,
+                )
+                .all()
+            )
+            for link in stale:
+                link.is_active = False
+                link.revoked_at = now
+            db.commit()
+            revoked = len(stale)
+
+        detail = f"Revoked {revoked} expired shared link(s)."
+        logger.info("[batch] expire_shared_links: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"revoked": revoked}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] expire_shared_links failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"revoked": 0, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: prune old processing logs
+# ---------------------------------------------------------------------------
+
+#: Default retention period for processing logs and audit log rows.
+_LOG_RETENTION_DAYS: int = 30
+
+
+@celery.task(name="app.tasks.batch_tasks.prune_processing_logs")
+def prune_processing_logs(retention_days: int = _LOG_RETENTION_DAYS) -> dict:
+    """
+    Delete ``processing_logs`` and ``settings_audit_log`` rows older than
+    *retention_days* (default 30) to prevent unbounded table growth.
+
+    Rows for the most recent *retention_days* days are kept so that recent
+    activity is still visible in the logs/audit UI.
+
+    Args:
+        retention_days: Number of days of history to keep (default 30).
+
+    Returns:
+        A summary dict with ``processing_logs_deleted`` and
+        ``audit_log_deleted`` counts.
+    """
+    job_name = "prune-processing-logs"
+    logger.info("[batch] Starting prune_processing_logs (retention_days=%s)", retention_days)
+
+    cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
+
+    try:
+        with SessionLocal() as db:
+            pl_deleted = db.query(ProcessingLog).filter(ProcessingLog.timestamp < cutoff).delete()
+            al_deleted = db.query(SettingsAuditLog).filter(SettingsAuditLog.changed_at < cutoff).delete()
+            db.commit()
+
+        detail = (
+            f"Deleted {pl_deleted} processing log row(s) and "
+            f"{al_deleted} settings audit log row(s) older than {retention_days} days."
+        )
+        logger.info("[batch] prune_processing_logs: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"processing_logs_deleted": pl_deleted, "audit_log_deleted": al_deleted}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] prune_processing_logs failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"processing_logs_deleted": 0, "audit_log_deleted": 0, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: prune old in-app notifications
+# ---------------------------------------------------------------------------
+
+#: Default retention period for read notifications.
+_NOTIFICATION_RETENTION_DAYS: int = 30
+
+
+@celery.task(name="app.tasks.batch_tasks.prune_old_notifications")
+def prune_old_notifications(retention_days: int = _NOTIFICATION_RETENTION_DAYS) -> dict:
+    """
+    Delete ``in_app_notifications`` rows that are already read and older than
+    *retention_days* days (default 30) to prevent unbounded table growth.
+
+    Unread notifications are always kept regardless of age so users do not
+    miss important alerts.
+
+    Args:
+        retention_days: Number of days of read-notification history to keep
+                        (default 30).
+
+    Returns:
+        A summary dict with ``deleted`` count.
+    """
+    job_name = "prune-old-notifications"
+    logger.info("[batch] Starting prune_old_notifications (retention_days=%s)", retention_days)
+
+    cutoff = datetime.now(timezone.utc) - timedelta(days=retention_days)
+
+    try:
+        with SessionLocal() as db:
+            deleted = (
+                db.query(InAppNotification)
+                .filter(
+                    InAppNotification.is_read.is_(True),
+                    InAppNotification.created_at < cutoff,
+                )
+                .delete()
+            )
+            db.commit()
+
+        detail = f"Deleted {deleted} old read notification(s) older than {retention_days} days."
+        logger.info("[batch] prune_old_notifications: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"deleted": deleted}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] prune_old_notifications failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"deleted": 0, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: backfill missing AI metadata
+# ---------------------------------------------------------------------------
+
+#: Maximum number of files to process per backfill run.
+_METADATA_BACKFILL_BATCH_SIZE: int = 50
+
+
+@celery.task(name="app.tasks.batch_tasks.backfill_missing_metadata")
+def backfill_missing_metadata(batch_size: int = _METADATA_BACKFILL_BATCH_SIZE) -> dict:
+    """
+    Re-trigger AI metadata extraction for documents that have OCR text but
+    no ``ai_metadata``.
+
+    This handles the common case where a document was processed before the AI
+    metadata extraction step was configured (e.g., before an OpenAI API key
+    was added), or where the extraction previously failed.
+
+    Only files that are **not** currently in-progress and have non-empty
+    ``ocr_text`` are selected.  A configurable *batch_size* caps the number
+    of tasks queued per run to avoid overwhelming the AI provider.
+
+    Args:
+        batch_size: Maximum number of files to queue per run (default 50).
+
+    Returns:
+        A summary dict with ``queued`` count.
+    """
+    from app.tasks.extract_metadata_with_gpt import extract_metadata_with_gpt  # avoid circular import
+
+    job_name = "backfill-missing-metadata"
+    logger.info("[batch] Starting backfill_missing_metadata (batch_size=%s)", batch_size)
+
+    try:
+        with SessionLocal() as db:
+            # Files currently being processed — skip them.
+            in_progress_file_ids = (
+                db.query(FileProcessingStep.file_id)
+                .filter(FileProcessingStep.status == "in_progress")
+                .distinct()
+                .subquery()
+            )
+
+            candidates = (
+                db.query(FileRecord)
+                .filter(FileRecord.is_duplicate.is_(False))
+                .filter(FileRecord.ocr_text.isnot(None))
+                .filter(FileRecord.ocr_text != "")
+                .filter((FileRecord.ai_metadata.is_(None)) | (FileRecord.ai_metadata == ""))
+                .filter(~FileRecord.id.in_(db.query(in_progress_file_ids.c.file_id)))
+                .limit(batch_size)
+                .all()
+            )
+
+        queued = 0
+        for record in candidates:
+            filename = record.local_filename or record.original_filename or f"file_{record.id}"
+            extract_metadata_with_gpt.delay(
+                filename,
+                record.ocr_text,
+                file_id=record.id,
+            )
+            queued += 1
+
+        detail = f"Queued {queued} document(s) for AI metadata backfill."
+        logger.info("[batch] backfill_missing_metadata: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"queued": queued}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] backfill_missing_metadata failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"queued": 0, "error": str(exc)}
+
+
+# ---------------------------------------------------------------------------
+# Task: sync Meilisearch search index
+# ---------------------------------------------------------------------------
+
+#: Maximum documents to index per sync run.
+_SEARCH_SYNC_BATCH_SIZE: int = 100
+
+
+@celery.task(name="app.tasks.batch_tasks.sync_search_index")
+def sync_search_index(batch_size: int = _SEARCH_SYNC_BATCH_SIZE) -> dict:
+    """
+    Index documents in Meilisearch that have OCR text or AI metadata but are
+    not yet present in the search index.
+
+    This is useful after:
+    - Enabling Meilisearch for the first time on an existing installation.
+    - Recovering from a Meilisearch index wipe or migration.
+    - Documents processed before search indexing was added to the pipeline.
+
+    The task queries the Meilisearch index for existing document IDs, then
+    finds ``FileRecord`` rows that have processable content (``ocr_text`` or
+    ``ai_metadata``) but are absent from the index, and re-indexes them.
+
+    A configurable *batch_size* caps the number of documents indexed per run.
+
+    Args:
+        batch_size: Maximum number of documents to index per run (default 100).
+
+    Returns:
+        A summary dict with ``indexed`` and ``skipped`` counts.
+    """
+    from app.utils.meilisearch_client import get_meilisearch_client, index_document
+
+    job_name = "sync-search-index"
+    logger.info("[batch] Starting sync_search_index (batch_size=%s)", batch_size)
+
+    client = get_meilisearch_client()
+    if client is None:
+        detail = "Meilisearch is not configured; skipping search index sync."
+        logger.info("[batch] sync_search_index: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"indexed": 0, "skipped": 0, "reason": "meilisearch_not_configured"}
+
+    try:
+        # Fetch the set of file_ids already in the Meilisearch index.
+        index = client.get_index(settings.meilisearch_index_name)
+        # Fetch up to 10 000 IDs — sufficient to determine gaps for most installs.
+        existing_result = index.get_documents({"fields": ["file_id"], "limit": 10000})
+        existing_ids: set[int] = {doc["file_id"] for doc in existing_result.results if "file_id" in doc}
+    except Exception as exc:
+        detail = f"Error fetching existing Meilisearch IDs: {exc}"
+        logger.error("[batch] sync_search_index: %s", detail)
+        _update_job_status(job_name, "failed", detail)
+        return {"indexed": 0, "skipped": 0, "error": str(exc)}
+
+    try:
+        with SessionLocal() as db:
+            # Files with indexable content that are not already in the index.
+            candidates = (
+                db.query(FileRecord)
+                .filter(FileRecord.is_duplicate.is_(False))
+                .filter(
+                    (FileRecord.ocr_text.isnot(None) & (FileRecord.ocr_text != ""))
+                    | (FileRecord.ai_metadata.isnot(None) & (FileRecord.ai_metadata != ""))
+                )
+                .filter(~FileRecord.id.in_(existing_ids) if existing_ids else True)  # type: ignore[arg-type]
+                .limit(batch_size)
+                .all()
+            )
+
+        indexed = 0
+        skipped = 0
+        for record in candidates:
+            metadata: dict = {}
+            if record.ai_metadata:
+                try:
+                    metadata = json.loads(record.ai_metadata)
+                except (json.JSONDecodeError, ValueError):
+                    pass
+
+            success = index_document(record, record.ocr_text or "", metadata)
+            if success:
+                indexed += 1
+            else:
+                skipped += 1
+
+        detail = f"Indexed {indexed} document(s) into Meilisearch; {skipped} skipped (indexing error)."
+        logger.info("[batch] sync_search_index: %s", detail)
+        _update_job_status(job_name, "success", detail)
+        return {"indexed": indexed, "skipped": skipped}
+
+    except Exception as exc:
+        detail = f"Error: {exc}"
+        logger.error("[batch] sync_search_index failed: %s", exc, exc_info=True)
+        _update_job_status(job_name, "failed", detail)
+        return {"indexed": 0, "skipped": 0, "error": str(exc)}

app/tasks/extract_metadata_with_gpt.py

@@ -14,6 +14,7 @@ from app.tasks.embed_metadata_into_pdf import embed_metadata_into_pdf
 from app.tasks.retry_config import BaseTaskWithRetry
 from app.utils import log_task_progress
 from app.utils.ai_provider import get_ai_provider
+from app.utils.filename_utils import VALID_FILENAME_RE
 
 logger = logging.getLogger(__name__)
 
@@ -75,7 +76,7 @@ def extract_metadata_with_gpt(self, filename: str, cleaned_text: str, file_id: i
         "Your task is to analyze the given text and return a well-structured JSON object.\n\n"
         "Extract and return the following fields:\n"
         "1. **filename**: Machine-readable filename "
-        "(YYYY-MM-DD_DescriptiveTitle, use only letters, numbers, periods, and underscores).\n"
+        "(YYYY-MM-DD_DescriptiveTitle, use only letters, numbers, spaces, dashes, periods, and underscores).\n"
         '2. **empfaenger**: The recipient, or "Unknown" if not found.\n'
         '3. **absender**: The sender, or "Unknown" if not found.\n'
         "4. **correspondent**: The entity or company that issued the document "
@@ -148,12 +149,12 @@ def extract_metadata_with_gpt(self, filename: str, cleaned_text: str, file_id: i
         suggested_filename = metadata.get("filename", "")
         if suggested_filename:
             # Check if filename contains only safe characters AND explicitly check for ".."
-            # Defense in depth: While the regex [\w\-\. ]+ already excludes / and \,
+            # Defense in depth: While the regex VALID_FILENAME_PATTERN already excludes / and \,
             # we explicitly reject ".." to guard against:
             # 1. Potential locale-specific \w behavior
             # 2. Files literally named ".." which are valid but problematic
             # 3. Future code changes that might relax the regex
-            if not re.match(r"^[\w\-\. ]+$", suggested_filename) or ".." in suggested_filename:
+            if not VALID_FILENAME_RE.match(suggested_filename) or ".." in suggested_filename:
                 logger.warning(f"[{task_id}] Invalid filename format from GPT: '{suggested_filename}', using fallback")
                 # Reset to empty to trigger fallback to original filename
                 metadata["filename"] = ""

app/tasks/finalize_document_storage.py

@@ -10,8 +10,13 @@ from app.database import SessionLocal
 from app.models import FileRecord
 from app.tasks.retry_config import BaseTaskWithRetry
 
-# Import the aggregator task and validator
-from app.tasks.send_to_all import get_configured_services_from_validator, send_to_all_destinations
+# Import the aggregator tasks and validator
+from app.tasks.send_to_all import (
+    get_configured_services_from_validator,
+    get_user_destination_count,
+    send_to_all_destinations,
+    send_to_user_destinations,
+)
 
 # Import database and logging utils from main
 from app.utils import log_task_progress
@@ -26,8 +31,16 @@ logger = logging.getLogger(__name__)
 def finalize_document_storage(self, original_file: str, processed_file: str, metadata: dict, file_id: int = None):
     """
     Final storage step after embedding metadata.
-    We will now call 'send_to_all_destinations' to push the final PDF to Dropbox/Nextcloud/Paperless.
-    After uploading, send a notification about the processed file.
+    Routes the processed document to the appropriate destination(s):
+
+    1. If the document has an identified owner and that owner has active
+       DESTINATION UserIntegrations, the file is uploaded to each of those
+       integrations (user-specific routing).
+    2. Otherwise the file is forwarded to the globally-configured destinations
+       via :func:`send_to_all_destinations` (system-wide fallback).
+
+    After queuing uploads, optional PDF/A archival conversion and embedding
+    computation are triggered, and a completion notification is sent.
     """
     task_id = self.request.id
     logger.info(f"[{task_id}] Finalizing document storage for {processed_file}")
@@ -41,7 +54,8 @@ def finalize_document_storage(self, original_file: str, processed_file: str, met
         file_id=file_id,
     )
 
-    # Get file_id from database if not provided (fallback logic)
+    # 2. Resolve file_id and owner_id from the database
+    owner_id = None
     if file_id is None:
         with SessionLocal() as db:
             # Only as a last resort, try to find by exact match on local_filename
@@ -49,32 +63,52 @@ def finalize_document_storage(self, original_file: str, processed_file: str, met
             file_record = db.query(FileRecord).filter(FileRecord.local_filename == tmp_path).first()
             if file_record:
                 file_id = file_record.id
+                owner_id = file_record.owner_id
+    else:
+        with SessionLocal() as db:
+            file_record = db.query(FileRecord).filter(FileRecord.id == file_id).first()
+            if file_record:
+                owner_id = file_record.owner_id
 
-    # 2. Determine Configured Destinations
-    # This is needed for the notification message later
+    # 3. Determine configured destinations for notification
     configured_destinations = []
     try:
         configured_services = get_configured_services_from_validator()
-        # Get list of service names that are configured
         for service_name, is_configured in configured_services.items():
             if is_configured:
-                # Format service names for display
                 display_name = service_name.replace("_", " ").title()
                 configured_destinations.append(display_name)
     except Exception as e:
         logger.warning(f"[WARNING] Could not determine configured destinations: {e}")
         configured_destinations = ["configured destinations"]
 
-    # 3. Queue Uploads
-    logger.info(f"[{task_id}] Queueing uploads to all destinations")
+    # 4. Queue Uploads — prefer user-specific destinations when available
     log_task_progress(
         task_id, "finalize_document_storage", "success", "Queuing uploads to destinations", file_id=file_id
     )
 
-    # Note: send_to_all_destinations is asynchronous and queues upload tasks
-    send_to_all_destinations.delay(processed_file, True, file_id)
+    user_dest_count = 0
+    if owner_id:
+        try:
+            user_dest_count = get_user_destination_count(owner_id)
+        except Exception as e:
+            logger.warning("[%s] Could not query user destination count for owner=%s: %s", task_id, owner_id, e)
 
-    # 3a. Trigger PDF/A archival conversion if enabled (from feature branch)
+    if owner_id and user_dest_count > 0:
+        # User has configured their own destinations → use those exclusively
+        logger.info(
+            "[%s] Routing to %d user-specific destination(s) for owner=%s",
+            task_id,
+            user_dest_count,
+            owner_id,
+        )
+        send_to_user_destinations.delay(processed_file, owner_id, file_id)
+    else:
+        # No user-specific destinations → fall back to global configuration
+        logger.info("[%s] No user-specific destinations found; using global destinations", task_id)
+        send_to_all_destinations.delay(processed_file, True, file_id)
+
+    # 4a. Trigger PDF/A archival conversion if enabled
     if settings.enable_pdfa_conversion:
         try:
             from app.tasks.convert_to_pdfa import convert_to_pdfa
@@ -84,7 +118,7 @@ def finalize_document_storage(self, original_file: str, processed_file: str, met
         except Exception as e:
             logger.warning(f"[{task_id}] Could not queue PDF/A conversion: {e}")
 
-    # 3b. Queue embedding computation (from main branch)
+    # 4b. Queue embedding computation
     if file_id is not None:
         try:
             from app.tasks.compute_embedding import compute_document_embedding
@@ -94,9 +128,8 @@ def finalize_document_storage(self, original_file: str, processed_file: str, met
         except Exception as e:
             logger.warning(f"[{task_id}] Could not queue embedding task: {e}")
 
-    # 4. Send Notification
+    # 5. Send Notification
     try:
-        # Get file information
         file_size = os.path.getsize(processed_file) if os.path.exists(processed_file) else 0
         filename = os.path.basename(processed_file)
 

app/tasks/imap_tasks.py

@@ -13,13 +13,79 @@ from celery import shared_task
 from app.config import settings
 from app.tasks.convert_to_pdf import convert_to_pdf  # new conversion task
 from app.tasks.process_document import process_document  # Updated import
-from app.utils.allowed_types import ALLOWED_EXTENSIONS, ALLOWED_MIME_TYPES
+from app.utils.allowed_types import (
+    ALL_CATEGORIES,
+    DEFAULT_CATEGORIES,
+    get_allowed_types_for_categories,
+)
+
+# Database session for per-user IMAP accounts (imported lazily to avoid circular imports)
+_db_session_factory = None
+
+# Maximum length to store as last_error to prevent DB bloat
+_MAX_ERROR_LENGTH = 500
+
+
+def _get_db_session():
+    """Return a new SQLAlchemy session (lazy import to avoid startup issues)."""
+    global _db_session_factory  # noqa: PLW0603
+    if _db_session_factory is None:
+        from app.database import SessionLocal
+
+        _db_session_factory = SessionLocal
+    return _db_session_factory()
+
 
 logger = logging.getLogger(__name__)
 
 # Initialize Redis connection using Celery's Redis settings
 redis_client = redis.StrictRedis.from_url(settings.redis_url, decode_responses=True)
 
+
+def _decrypt_imap_password(password: str | None) -> str | None:
+    """Decrypt an IMAP account password stored in the database.
+
+    Passwords are stored encrypted (Fernet, ``enc:`` prefix) for new records;
+    legacy plaintext records are returned unchanged so existing accounts
+    continue to work until they are next updated via the API.
+    """
+    from app.utils.encryption import decrypt_value
+
+    return decrypt_value(password)
+
+
+def _resolve_categories_for_profile(profile_id: int | None) -> list[str]:
+    """Return the list of allowed categories for a profile ID.
+
+    Loads the profile from the database.  If ``profile_id`` is ``None`` or the
+    profile is not found, falls back to the global ``settings.imap_attachment_filter``
+    string (``'documents_only'`` → default categories; ``'all'`` → all categories).
+    """
+    if profile_id is not None:
+        try:
+            from app.models import ImapIngestionProfile
+
+            db = _get_db_session()
+            try:
+                profile = db.query(ImapIngestionProfile).filter(ImapIngestionProfile.id == profile_id).first()
+                if profile:
+                    return json.loads(profile.allowed_categories)
+            finally:
+                db.close()
+        except Exception as exc:  # noqa: BLE001
+            logger.warning(
+                "Could not load IMAP ingestion profile %d (%s: %s) — using global default",
+                profile_id,
+                type(exc).__name__,
+                exc,
+            )
+
+    # Fall back to global setting
+    if settings.imap_attachment_filter == "all":
+        return ALL_CATEGORIES
+    return DEFAULT_CATEGORIES
+
+
 LOCK_KEY = "imap_lock"  # Unique key for locking
 LOCK_EXPIRE = 300  # Lock expires in 5 minutes
 
@@ -82,6 +148,10 @@ def pull_all_inboxes():
     Periodic Celery task that checks all configured IMAP mailboxes
     and fetches attachments from new emails.
     Ensures only one instance runs at a time using Redis-based locking.
+
+    Processes:
+    1. System-level mailboxes configured via environment variables (IMAP1, IMAP2).
+    2. Per-user IMAP accounts stored in the ``user_imap_accounts`` database table.
     """
     if not acquire_lock():
         logger.info("Skipping execution: Another instance is running.")
@@ -112,12 +182,158 @@ def pull_all_inboxes():
             delete_after_process=settings.imap2_delete_after_process,
         )
 
+        # Per-user IMAP accounts from the database
+        _pull_user_imap_accounts()
+
+        # Per-user IMAP integrations from the UserIntegration model
+        _pull_user_integration_imap()
+
         logger.info("Finished pull_all_inboxes")
 
     finally:
         release_lock()
 
 
+def _pull_user_imap_accounts() -> None:
+    """Iterate over all active per-user IMAP accounts and pull their inboxes."""
+    try:
+        from app.models import UserImapAccount
+
+        db = _get_db_session()
+        try:
+            accounts = db.query(UserImapAccount).filter(UserImapAccount.is_active.is_(True)).all()
+            logger.info("Processing %d per-user IMAP account(s)", len(accounts))
+            for acct in accounts:
+                # Use a descriptive identifier for logging and processed-email cache keys
+                account_identifier = f"user_{acct.owner_id}_{acct.id}"
+                try:
+                    pull_inbox(
+                        mailbox_key=account_identifier,
+                        host=acct.host,
+                        port=acct.port,
+                        username=acct.username,
+                        password=_decrypt_imap_password(acct.password),
+                        use_ssl=acct.use_ssl,
+                        delete_after_process=acct.delete_after_process,
+                        owner_id=acct.owner_id,
+                        allowed_categories=_resolve_categories_for_profile(acct.profile_id),
+                    )
+                    # Record successful poll
+                    acct.last_checked_at = datetime.now(timezone.utc)
+                    acct.last_error = None
+                    db.commit()
+                except Exception as exc:  # noqa: BLE001
+                    error_msg = str(exc)[:_MAX_ERROR_LENGTH]
+                    logger.error(
+                        "Error pulling user IMAP account %d (%s@%s): %s",
+                        acct.id,
+                        acct.username,
+                        acct.host,
+                        error_msg,
+                    )
+                    try:
+                        acct.last_checked_at = datetime.now(timezone.utc)
+                        acct.last_error = error_msg
+                        db.commit()
+                    except Exception:  # noqa: BLE001
+                        db.rollback()
+        finally:
+            db.close()
+    except Exception as exc:  # noqa: BLE001
+        logger.error("Failed to process per-user IMAP accounts: %s", exc)
+
+
+def _pull_user_integration_imap() -> None:
+    """Iterate over all active IMAP UserIntegrations and pull their inboxes.
+
+    This polls the ``user_integrations`` table for records with
+    ``integration_type='IMAP'``, ``direction='SOURCE'``, and ``is_active=True``.
+    Each integration's config/credentials are decoded and passed to
+    :func:`pull_inbox` with the owning user's ``owner_id`` so that ingested
+    documents are correctly attributed.
+
+    Individual connection failures are caught and recorded on the integration
+    without crashing the polling loop.
+    """
+    try:
+        import json as _json
+
+        from app.models import IntegrationDirection, IntegrationType, UserIntegration
+        from app.utils.encryption import decrypt_value
+
+        db = _get_db_session()
+        try:
+            integrations = (
+                db.query(UserIntegration)
+                .filter(
+                    UserIntegration.integration_type == IntegrationType.IMAP,
+                    UserIntegration.direction == IntegrationDirection.SOURCE,
+                    UserIntegration.is_active.is_(True),
+                )
+                .all()
+            )
+            logger.info("Processing %d IMAP UserIntegration(s)", len(integrations))
+            for integ in integrations:
+                account_identifier = f"integration_{integ.owner_id}_{integ.id}"
+                try:
+                    cfg = _json.loads(integ.config) if integ.config else {}
+                    raw_creds = decrypt_value(integ.credentials) if integ.credentials else None
+                    creds = _json.loads(raw_creds) if raw_creds else {}
+
+                    host = cfg.get("host")
+                    port = int(cfg.get("port", 993))
+                    username = cfg.get("username")
+                    password = creds.get("password")
+                    use_ssl = cfg.get("use_ssl", True)
+                    delete_after = cfg.get("delete_after_process", False)
+                    gmail_labels = cfg.get("gmail_apply_labels", True)
+                    # Integrations can store a profile_id in config; fall back to global default
+                    profile_id = cfg.get("profile_id")
+                    allowed_categories = _resolve_categories_for_profile(profile_id)
+
+                    if not (host and username and password):
+                        logger.warning(
+                            "IMAP integration %d (owner %s) has incomplete config — skipping.",
+                            integ.id,
+                            integ.owner_id,
+                        )
+                        continue
+
+                    pull_inbox(
+                        mailbox_key=account_identifier,
+                        host=host,
+                        port=port,
+                        username=username,
+                        password=password,
+                        use_ssl=use_ssl,
+                        delete_after_process=delete_after,
+                        owner_id=integ.owner_id,
+                        gmail_apply_labels=gmail_labels,
+                        allowed_categories=allowed_categories,
+                    )
+                    integ.last_used_at = datetime.now(timezone.utc)
+                    integ.last_error = None
+                    db.commit()
+                except Exception as exc:  # noqa: BLE001
+                    error_msg = str(exc)[:_MAX_ERROR_LENGTH]
+                    logger.error(
+                        "Error pulling IMAP integration %d (owner %s): %s",
+                        integ.id,
+                        integ.owner_id,
+                        error_msg,
+                    )
+                    try:
+                        integ.last_used_at = datetime.now(timezone.utc)
+                        integ.last_error = error_msg
+                        db.commit()
+                    except Exception:  # noqa: BLE001
+                        db.rollback()
+        finally:
+            db.close()
+    except Exception as exc:  # noqa: BLE001
+        logger.error("Failed to process IMAP UserIntegrations: %s", exc)
+
+
 def check_and_pull_mailbox(
     mailbox_key: str,
     host: str | None,
@@ -144,7 +360,18 @@ def check_and_pull_mailbox(
     )
 
 
-def pull_inbox(mailbox_key, host, port, username, password, use_ssl, delete_after_process):
+def pull_inbox(
+    mailbox_key,
+    host,
+    port,
+    username,
+    password,
+    use_ssl,
+    delete_after_process,
+    owner_id=None,
+    gmail_apply_labels=True,
+    allowed_categories=None,
+):
     """
     Connects to the IMAP inbox, fetches new unread emails from the last 3 days,
     and processes attachments while preserving the original unread status.
@@ -154,8 +381,28 @@ def pull_inbox(mailbox_key, host, port, username, password, use_ssl, delete_afte
       - Runs an X-GM-RAW query: "in:anywhere in:unread newer_than:3d has:attachment".
 
     For non-Gmail mailboxes, it falls back to selecting the INBOX with a SINCE/UNSEEN filter.
+
+    Args:
+        owner_id: Optional user identifier. When provided, ingested documents are
+                  attributed to this user via ``process_document`` / ``convert_to_pdf``.
+        gmail_apply_labels: Whether to apply Gmail-specific labels and stars to
+                  processed emails. Only relevant for Gmail hosts. Defaults to True.
+        allowed_categories: List of file-type category keys to ingest (e.g.
+                  ``["pdf", "office", "images"]``).  ``None`` falls back to the
+                  global ``settings.imap_attachment_filter`` mapping.
     """
-    logger.info("Connecting to %s at %s:%s (SSL=%s)", mailbox_key, host, port, use_ssl)
+    if allowed_categories is None:
+        allowed_categories = _resolve_categories_for_profile(None)
+
+    effective_mime_types, effective_extensions = get_allowed_types_for_categories(allowed_categories)
+    logger.info(
+        "Connecting to %s at %s:%s (SSL=%s) — categories: %s",
+        mailbox_key,
+        host,
+        port,
+        use_ssl,
+        allowed_categories,
+    )
     processed_emails = load_processed_emails()
 
     try:
@@ -209,19 +456,23 @@ def pull_inbox(mailbox_key, host, port, username, password, use_ssl, delete_afte
                 continue
 
             # For Gmail, check if the email already has the "Ingested" label.
-            if is_gmail_host:
+            if is_gmail_host and gmail_apply_labels:
                 if email_already_has_label(mail, num, "Ingested"):
                     logger.info("Skipping email %s in %s, already labeled 'Ingested'.", msg_id, mailbox_key)
                     continue
 
-            # Process attachments (and convert non-PDF files).
-            # We call the function without assigning its return value since it is not used.
-            fetch_attachments_and_enqueue(email_message)
+            # Process attachments using the resolved mime types / extensions.
+            fetch_attachments_and_enqueue(
+                email_message,
+                owner_id=owner_id,
+                effective_mime_types=effective_mime_types,
+                effective_extensions=effective_extensions,
+            )
 
             if settings.imap_readonly_mode:
                 logger.info("Readonly mode: skipping mailbox modifications for %s in %s", msg_id, mailbox_key)
             else:
-                if is_gmail_host:
+                if is_gmail_host and gmail_apply_labels:
                     mark_as_processed_with_star(mail, num)
                     mark_as_processed_with_label(mail, num, label="Ingested")
 
@@ -245,33 +496,39 @@ def pull_inbox(mailbox_key, host, port, username, password, use_ssl, delete_afte
         logger.exception("Error pulling mailbox %s: %s", mailbox_key, e)
 
 
-def fetch_attachments_and_enqueue(email_message):
+def fetch_attachments_and_enqueue(
+    email_message,
+    owner_id: str | None = None,
+    effective_mime_types: frozenset[str] | None = None,
+    effective_extensions: frozenset[str] | None = None,
+):
     """
     Extracts attachments from the email and processes only allowed file types.
 
-    Files are accepted if either:
-    1. They have a MIME type from the ALLOWED_MIME_TYPES set, OR
-    2. They have a '.pdf' file extension (regardless of MIME type)
+    The caller is responsible for computing ``effective_mime_types`` and
+    ``effective_extensions`` from the relevant :class:`ImapIngestionProfile` (or
+    the global default) via :func:`app.utils.allowed_types.get_allowed_types_for_categories`
+    before calling this function.  ``pull_inbox`` does this automatically.
 
-    Allowed file types include:
-      - PDF: application/pdf or *.pdf extension
-      - Microsoft Office files:
-          - Word: application/msword,
-            application/vnd.openxmlformats-officedocument.wordprocessingml.document
-          - Excel: application/vnd.ms-excel,
-            application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
-          - PowerPoint: application/vnd.ms-powerpoint,
-            application/vnd.openxmlformats-officedocument.presentationml.presentation
-      - Other meaningful attachments:
-          - Plain text: text/plain
-          - CSV: text/csv
-          - Rich Text Format: application/rtf, text/rtf
+    If either set is ``None`` the function falls back to the default category list
+    so the function still works correctly when called directly in tests or from
+    other contexts.
 
     If the attachment is a PDF (by extension or MIME type), it is enqueued for upload;
     any other allowed file is enqueued for conversion to PDF.
 
+    Args:
+        email_message: The parsed email message to extract attachments from.
+        owner_id: Optional user identifier forwarded to ``process_document`` /
+                  ``convert_to_pdf`` for multi-tenant attribution.
+        effective_mime_types: Pre-computed frozenset of allowed MIME type strings.
+        effective_extensions: Pre-computed frozenset of allowed file extension strings.
+
     Returns True if at least one allowed attachment was processed.
     """
+    if effective_mime_types is None or effective_extensions is None:
+        effective_mime_types, effective_extensions = get_allowed_types_for_categories(DEFAULT_CATEGORIES)
+
     has_attachment = False
     for part in email_message.walk():
         if part.get_content_maintype() == "multipart":
@@ -286,9 +543,15 @@ def fetch_attachments_and_enqueue(email_message):
 
         mime_type = part.get_content_type()
         file_ext = os.path.splitext(filename)[1].lower()
+
         # Accept file if it has an allowed MIME type, an allowed extension, OR is a PDF by extension
-        if mime_type not in ALLOWED_MIME_TYPES and file_ext not in ALLOWED_EXTENSIONS and not is_pdf_by_extension:
-            logger.info("Skipping attachment %s with MIME type %s", filename, mime_type)
+        if mime_type not in effective_mime_types and file_ext not in effective_extensions and not is_pdf_by_extension:
+            logger.info(
+                "Skipping attachment %s (MIME: %s, ext: %s) — not in effective allowed set",
+                filename,
+                mime_type,
+                file_ext,
+            )
             continue
 
         file_path = os.path.join(settings.workdir, filename)
@@ -297,11 +560,11 @@ def fetch_attachments_and_enqueue(email_message):
 
         # If it's a PDF by MIME type or extension, process it directly
         if mime_type == "application/pdf" or is_pdf_by_extension:
-            process_document.delay(file_path)
+            process_document.delay(file_path, owner_id=owner_id)
             logger.info("Enqueued PDF for upload: %s (MIME: %s)", filename, mime_type)
-        elif mime_type in ALLOWED_MIME_TYPES:
+        elif mime_type in effective_mime_types:
             # Other allowed files are sent for conversion
-            convert_to_pdf.delay(file_path)
+            convert_to_pdf.delay(file_path, owner_id=owner_id)
             logger.info("Enqueued file for conversion to PDF: %s", filename)
 
         has_attachment = True

app/tasks/process_document.py

@@ -1,10 +1,14 @@
 #!/usr/bin/env python3
 
+from __future__ import annotations
+
+import json
 import logging
 import mimetypes
 import os
 import shutil
 import uuid
+from typing import TYPE_CHECKING
 
 import pypdf  # Upgraded from PyPDF2 to fix CVE-2023-36464
 from pypdf.errors import PdfReadError
@@ -12,7 +16,7 @@ from pypdf.errors import PdfReadError
 from app.celery_app import celery
 from app.config import settings
 from app.database import SessionLocal
-from app.models import FileRecord
+from app.models import FileRecord, Pipeline, PipelineStep
 from app.tasks.extract_metadata_with_gpt import extract_metadata_with_gpt
 from app.tasks.process_with_ocr import process_with_ocr
 from app.tasks.retry_config import BaseTaskWithRetry
@@ -20,9 +24,75 @@ from app.utils import get_unique_filepath_with_counter, hash_file, log_task_prog
 from app.utils.step_manager import initialize_file_steps
 from app.utils.text_quality import check_text_quality, detect_pdf_text_source
 
+if TYPE_CHECKING:
+    from sqlalchemy.orm import Session
+
 logger = logging.getLogger(__name__)
 
 
+def _get_pipeline_ocr_language(db: "Session", file_record: FileRecord, owner_id: str | None) -> str | None:
+    """Look up the OCR language override from the file's pipeline OCR step config.
+
+    Resolution order:
+    1. Explicit pipeline assigned to the file (``file_record.pipeline_id``).
+    2. User's own default pipeline (``owner_id``, ``is_default=True``).
+    3. System default pipeline (``owner_id=NULL``, ``is_default=True``).
+
+    Returns the ``ocr_language`` value from the pipeline's OCR step config, or
+    ``None`` when no override is configured.
+    """
+    pipeline = None
+
+    if file_record.pipeline_id:
+        pipeline = db.query(Pipeline).filter(Pipeline.id == file_record.pipeline_id).first()
+
+    if pipeline is None and owner_id:
+        pipeline = (
+            db.query(Pipeline)
+            .filter(
+                Pipeline.owner_id == owner_id,
+                Pipeline.is_default.is_(True),
+                Pipeline.is_active.is_(True),
+            )
+            .first()
+        )
+
+    if pipeline is None:
+        pipeline = (
+            db.query(Pipeline)
+            .filter(
+                Pipeline.owner_id.is_(None),
+                Pipeline.is_default.is_(True),
+                Pipeline.is_active.is_(True),
+            )
+            .first()
+        )
+
+    if pipeline is None:
+        return None
+
+    ocr_step = (
+        db.query(PipelineStep)
+        .filter(
+            PipelineStep.pipeline_id == pipeline.id,
+            PipelineStep.step_type == "ocr",
+            PipelineStep.enabled.is_(True),
+        )
+        .first()
+    )
+
+    if ocr_step is None or not ocr_step.config:
+        return None
+
+    try:
+        step_config = json.loads(ocr_step.config)
+        lang = step_config.get("ocr_language")
+        # "auto" is treated as no override
+        return lang if lang and lang != "auto" else None
+    except Exception:
+        return None
+
+
 @celery.task(base=BaseTaskWithRetry, bind=True)
 def process_document(
     self,
@@ -109,6 +179,7 @@ def process_document(
         )
 
     # Acquire DB session in the task
+    ocr_language: str | None = None  # Pipeline OCR language override resolved inside DB session
     with SessionLocal() as db:
         # When file_id is provided, we are reprocessing an existing file.
         # Skip the duplicate check and reuse the existing record.
@@ -305,6 +376,14 @@ def process_document(
         new_record.local_filename = new_local_path
         db.commit()
 
+        # Look up pipeline OCR language override before the session closes.
+        # This reads the OCR step config from the file's assigned pipeline (or
+        # the user/system default pipeline) so the language is available when
+        # dispatching process_with_ocr below.
+        ocr_language = _get_pipeline_ocr_language(db, new_record, owner_id)
+        if ocr_language:
+            logger.info(f"[{task_id}] Pipeline OCR language override: {ocr_language!r}")
+
         # Store file_id before session closes to avoid DetachedInstanceError
         file_id = new_record.id
 
@@ -334,7 +413,7 @@ def process_document(
             "Queued for forced OCR processing",
             file_id=file_id,
         )
-        process_with_ocr.delay(new_filename, file_id)
+        process_with_ocr.delay(new_filename, file_id, language=ocr_language)
         return {"file": new_local_path, "status": "Queued for forced OCR", "file_id": file_id}
 
     # If the file is not a PDF, skip embedded text check and convert to PDF first
@@ -491,7 +570,7 @@ def process_document(
                     "Queued for OCR (text quality too low)",
                     file_id=file_id,
                 )
-                process_with_ocr.delay(new_filename, file_id, extracted_text)
+                process_with_ocr.delay(new_filename, file_id, extracted_text, language=ocr_language)
                 return {
                     "file": new_local_path,
                     "status": "Queued for OCR (poor embedded text quality)",
@@ -564,5 +643,5 @@ def process_document(
         "Queued for OCR processing",
         file_id=file_id,
     )
-    process_with_ocr.delay(new_filename, file_id)
+    process_with_ocr.delay(new_filename, file_id, language=ocr_language)
     return {"file": new_local_path, "status": "Queued for OCR", "file_id": file_id}

app/tasks/process_with_ocr.py

@@ -17,7 +17,6 @@ task with a multi-engine OCR pipeline that:
 
 import logging
 import os
-from typing import Optional
 
 from app.celery_app import celery
 from app.config import settings
@@ -33,7 +32,13 @@ logger = logging.getLogger(__name__)
 
 
 @celery.task(base=OcrTaskWithRetry, bind=True)
-def process_with_ocr(self, filename: str, file_id: Optional[int] = None, original_text: Optional[str] = None):
+def process_with_ocr(
+    self,
+    filename: str,
+    file_id: int | None = None,
+    original_text: str | None = None,
+    language: str | None = None,
+):
     """Run the configured OCR providers on *filename* and continue the pipeline.
 
     When multiple OCR providers are configured the results are merged using the
@@ -47,6 +52,10 @@ def process_with_ocr(self, filename: str, file_id: Optional[int] = None, origina
         filename: Base name of the file inside ``<workdir>/tmp/``.
         file_id: Optional database record ID passed through to downstream tasks.
         original_text: Optional original embedded text for head-to-head comparison.
+        language: Optional Tesseract-style language code(s) (e.g. ``"eng+deu"``)
+            to override the global OCR language settings for this specific run.
+            Pass ``None`` or ``"auto"`` to use the global settings.  This
+            enables per-pipeline language configuration.
     """
     task_id = self.request.id
     log_task_progress(
@@ -62,7 +71,7 @@ def process_with_ocr(self, filename: str, file_id: Optional[int] = None, origina
         if not os.path.exists(tmp_file_path):
             raise FileNotFoundError(f"Local file not found: {tmp_file_path}")
 
-        providers = get_ocr_providers()
+        providers = get_ocr_providers(language=language)
         provider_names = [p.name for p in providers]
         logger.info(f"[{task_id}] Running {len(providers)} OCR provider(s): {provider_names}")
 
@@ -122,7 +131,12 @@ def process_with_ocr(self, filename: str, file_id: Optional[int] = None, origina
         # PDF with ocrmypdf to embed an invisible text layer so the output is
         # selectable/searchable in PDF viewers.
         if searchable_pdf_path is None:
-            lang = getattr(settings, "tesseract_language", None) or "eng"
+            # Use the per-call language override; fall back to global setting
+            embed_lang = (
+                language
+                if language and language != "auto"
+                else (getattr(settings, "tesseract_language", None) or "eng")
+            )
             log_task_progress(
                 task_id,
                 "embed_text_layer",
@@ -130,7 +144,7 @@ def process_with_ocr(self, filename: str, file_id: Optional[int] = None, origina
                 "Embedding searchable text layer into PDF",
                 file_id=file_id,
             )
-            embedded = embed_text_layer(tmp_file_path, tmp_file_path, language=lang)
+            embedded = embed_text_layer(tmp_file_path, tmp_file_path, language=embed_lang)
             if embedded:
                 searchable_pdf_path = tmp_file_path
                 log_task_progress(

app/tasks/send_to_all.py

@@ -6,12 +6,13 @@ import os
 from app.celery_app import celery
 from app.config import settings
 from app.database import SessionLocal
-from app.models import FileRecord
+from app.models import FileRecord, IntegrationDirection, UserIntegration
 from app.tasks.retry_config import BaseTaskWithRetry
 from app.tasks.upload_to_dropbox import upload_to_dropbox
 from app.tasks.upload_to_email import upload_to_email
 from app.tasks.upload_to_ftp import upload_to_ftp
 from app.tasks.upload_to_google_drive import upload_to_google_drive
+from app.tasks.upload_to_icloud import upload_to_icloud
 from app.tasks.upload_to_nextcloud import upload_to_nextcloud
 from app.tasks.upload_to_onedrive import upload_to_onedrive
 from app.tasks.upload_to_paperless import upload_to_paperless
@@ -25,18 +26,32 @@ logger = logging.getLogger(__name__)
 
 
 def _should_upload_to_dropbox():
-    return bool(settings.dropbox_app_key and settings.dropbox_app_secret and settings.dropbox_refresh_token)
+    return bool(
+        getattr(settings, "dropbox_enabled", True)
+        and settings.dropbox_app_key
+        and settings.dropbox_app_secret
+        and settings.dropbox_refresh_token
+    )
 
 
 def _should_upload_to_nextcloud():
-    return bool(settings.nextcloud_upload_url and settings.nextcloud_username and settings.nextcloud_password)
+    return bool(
+        getattr(settings, "nextcloud_enabled", True)
+        and settings.nextcloud_upload_url
+        and settings.nextcloud_username
+        and settings.nextcloud_password
+    )
 
 
 def _should_upload_to_paperless():
-    return bool(settings.paperless_ngx_api_token and settings.paperless_host)
+    return bool(
+        getattr(settings, "paperless_enabled", True) and settings.paperless_ngx_api_token and settings.paperless_host
+    )
 
 
 def _should_upload_to_google_drive():
+    if not getattr(settings, "google_drive_enabled", True):
+        return False
     # Check for OAuth configuration
     if getattr(settings, "google_drive_use_oauth", False):
         return bool(
@@ -51,20 +66,33 @@ def _should_upload_to_google_drive():
 
 
 def _should_upload_to_webdav():
-    return bool(settings.webdav_url and settings.webdav_username and settings.webdav_password)
+    return bool(
+        getattr(settings, "webdav_enabled", True)
+        and settings.webdav_url
+        and settings.webdav_username
+        and settings.webdav_password
+    )
 
 
 def _should_upload_to_ftp():
-    return bool(settings.ftp_host and settings.ftp_username and settings.ftp_password)
+    return bool(
+        getattr(settings, "ftp_enabled", True) and settings.ftp_host and settings.ftp_username and settings.ftp_password
+    )
 
 
 def _should_upload_to_sftp():
-    return bool(settings.sftp_host and settings.sftp_username and (settings.sftp_password or settings.sftp_private_key))
+    return bool(
+        getattr(settings, "sftp_enabled", True)
+        and settings.sftp_host
+        and settings.sftp_username
+        and (settings.sftp_password or settings.sftp_private_key)
+    )
 
 
 def _should_upload_to_email():
     return bool(
-        settings.dest_email_host
+        getattr(settings, "dest_email_enabled", True)
+        and settings.dest_email_host
         and settings.dest_email_username
         and settings.dest_email_password
         and settings.dest_email_default_recipient
@@ -72,18 +100,32 @@ def _should_upload_to_email():
 
 
 def _should_upload_to_onedrive():
-    return bool(settings.onedrive_client_id and settings.onedrive_client_secret and settings.onedrive_refresh_token)
+    return bool(
+        getattr(settings, "onedrive_enabled", True)
+        and settings.onedrive_client_id
+        and settings.onedrive_client_secret
+        and settings.onedrive_refresh_token
+    )
 
 
 def _should_upload_to_s3():
-    return bool(settings.s3_bucket_name and settings.aws_access_key_id and settings.aws_secret_access_key)
+    return bool(
+        getattr(settings, "s3_enabled", True)
+        and settings.s3_bucket_name
+        and settings.aws_access_key_id
+        and settings.aws_secret_access_key
+    )
+
+
+def _should_upload_to_icloud():
+    return bool(getattr(settings, "icloud_enabled", True) and settings.icloud_username and settings.icloud_password)
 
 
 def get_configured_services_from_validator():
     """
-    Use the config validator to determine which services are configured properly.
+    Use the config validator to determine which services are configured and enabled.
     Returns a dictionary with service names as keys and boolean values indicating
-    whether they're properly configured.
+    whether they're properly configured AND explicitly enabled.
     """
     providers = get_provider_status()
 
@@ -98,12 +140,14 @@ def get_configured_services_from_validator():
         "Email": "email",
         "OneDrive": "onedrive",
         "S3 Storage": "s3",
+        "iCloud Drive": "icloud",
     }
 
     result = {}
     for provider_name, internal_name in service_map.items():
         if provider_name in providers:
-            result[internal_name] = providers[provider_name].get("configured", False)
+            provider = providers[provider_name]
+            result[internal_name] = provider.get("configured", False) and provider.get("enabled", True)
 
     return result
 
@@ -206,6 +250,11 @@ def send_to_all_destinations(self, file_path: str, use_validator=True, file_id:
             "should_upload": _should_upload_to_s3,
             "upload_func": upload_to_s3,
         },
+        {
+            "name": "icloud",
+            "should_upload": _should_upload_to_icloud,
+            "upload_func": upload_to_icloud,
+        },
     ]
 
     # Optionally get configuration status from validator
@@ -261,3 +310,123 @@ def send_to_all_destinations(self, file_path: str, use_validator=True, file_id:
     log_task_progress(task_id, "send_to_all_destinations", "success", f"Queued {queued_count} uploads", file_id=file_id)
 
     return {"status": "Queued", "file_path": file_path, "tasks": results}
+
+
+@celery.task(base=BaseTaskWithRetry, bind=True)
+def send_to_user_destinations(self, file_path: str, owner_id: str, file_id: int | None = None):
+    """Dispatch uploads to all active DESTINATION UserIntegrations for *owner_id*.
+
+    This is the user-specific counterpart of :func:`send_to_all_destinations`.
+    It queries the ``user_integrations`` table for records where:
+
+    * ``owner_id`` matches the document owner,
+    * ``direction == "DESTINATION"``, and
+    * ``is_active == True``.
+
+    One :func:`upload_to_user_integration` Celery task is queued for each
+    matching integration so that uploads proceed asynchronously and
+    independently.
+
+    Args:
+        file_path: Absolute path to the processed document file.
+        owner_id: The stable user identifier from ``FileRecord.owner_id``.
+        file_id: Optional ``FileRecord.id`` used for progress logging.
+
+    Returns:
+        A dict summarising how many integrations were queued.
+    """
+    from app.tasks.upload_to_user_integration import upload_to_user_integration
+
+    task_id = self.request.id
+    filename = os.path.basename(file_path)
+
+    if not os.path.exists(file_path):
+        error_msg = f"File not found: {file_path}"
+        logger.error("[%s] %s", task_id, error_msg)
+        log_task_progress(task_id, "send_to_user_destinations", "failure", error_msg, file_id=file_id)
+        raise FileNotFoundError(error_msg)
+
+    logger.info("[%s] Sending %s to user destinations for owner=%s", task_id, filename, owner_id)
+    log_task_progress(
+        task_id,
+        "send_to_user_destinations",
+        "in_progress",
+        f"Distributing {filename} to user integrations",
+        file_id=file_id,
+    )
+
+    with SessionLocal() as db:
+        integrations = (
+            db.query(UserIntegration)
+            .filter(
+                UserIntegration.owner_id == owner_id,
+                UserIntegration.direction == IntegrationDirection.DESTINATION,
+                UserIntegration.is_active.is_(True),
+            )
+            .all()
+        )
+        # Snapshot the IDs so we don't keep the session open
+        integration_ids = [(i.id, i.name, i.integration_type) for i in integrations]
+
+    queued = 0
+    task_results: dict[str, str] = {}
+
+    for int_id, int_name, int_type in integration_ids:
+        logger.info("[%s] Queueing upload for integration %d (%s '%s')", task_id, int_id, int_type, int_name)
+        log_task_progress(
+            task_id,
+            f"queue_user_integration_{int_id}",
+            "in_progress",
+            f"Queueing upload to {int_type} '{int_name}'",
+            file_id=file_id,
+        )
+        try:
+            celery_task = upload_to_user_integration.delay(file_path, int_id, file_id)
+            task_results[f"integration_{int_id}_task_id"] = celery_task.id
+            queued += 1
+            log_task_progress(
+                task_id,
+                f"queue_user_integration_{int_id}",
+                "success",
+                f"Queued upload to {int_type} '{int_name}'",
+                file_id=file_id,
+            )
+        except Exception as exc:  # noqa: BLE001
+            error_msg = str(exc)
+            logger.error("[%s] Failed to queue upload for integration %d: %s", task_id, int_id, error_msg)
+            task_results[f"integration_{int_id}_error"] = error_msg
+            log_task_progress(
+                task_id,
+                f"queue_user_integration_{int_id}",
+                "failure",
+                f"Failed to queue {int_type} '{int_name}': {error_msg}",
+                file_id=file_id,
+            )
+
+    logger.info("[%s] Queued %d user-integration upload(s) for owner=%s", task_id, queued, owner_id)
+    log_task_progress(
+        task_id,
+        "send_to_user_destinations",
+        "success",
+        f"Queued {queued} user-integration upload(s)",
+        file_id=file_id,
+    )
+    return {"status": "Queued", "file_path": file_path, "queued": queued, "tasks": task_results}
+
+
+def get_user_destination_count(owner_id: str) -> int:
+    """Return the number of active DESTINATION integrations for *owner_id*.
+
+    A count of zero means no user-specific destinations are configured and
+    the caller should fall back to the global :func:`send_to_all_destinations`.
+    """
+    with SessionLocal() as db:
+        return (
+            db.query(UserIntegration)
+            .filter(
+                UserIntegration.owner_id == owner_id,
+                UserIntegration.direction == IntegrationDirection.DESTINATION,
+                UserIntegration.is_active.is_(True),
+            )
+            .count()
+        )

app/tasks/upload_to_icloud.py

@@ -0,0 +1,177 @@
+#!/usr/bin/env python3
+
+"""Upload files to Apple iCloud Drive via the pyicloud library.
+
+This module uses the ``pyicloud`` library to authenticate with Apple's iCloud
+service and upload files to iCloud Drive.  Because Apple does not offer a public
+REST API for iCloud Drive, this integration relies on the *unofficial*
+reverse-engineered protocol implemented by ``pyicloud``.
+
+Requirements
+~~~~~~~~~~~~
+* An Apple ID with iCloud Drive enabled.
+* An **app-specific password** generated at https://appleid.apple.com (required
+  when two-factor authentication is active – which is the default for all modern
+  Apple IDs).
+* The ``pyicloud`` Python package (``pip install pyicloud``).
+
+Configuration
+~~~~~~~~~~~~~
+Set the following environment variables (or ``app/config.py`` fields):
+
+* ``ICLOUD_USERNAME`` – Apple ID email address.
+* ``ICLOUD_PASSWORD`` – App-specific password.
+* ``ICLOUD_FOLDER``   – Target folder path inside iCloud Drive, using ``/`` as
+  the separator (e.g. ``Documents/Uploads``).  The folder is created
+  automatically if it does not exist.
+* ``ICLOUD_COOKIE_DIRECTORY`` – (Optional) Directory for persisting session
+  cookies so that re-authentication is avoided between task runs.  Defaults to
+  ``~/.pyicloud``.
+"""
+
+import logging
+import os
+
+from app.celery_app import celery
+from app.config import settings
+from app.tasks.retry_config import UploadTaskWithRetry
+from app.utils import log_task_progress
+
+logger = logging.getLogger(__name__)
+
+
+def _get_icloud_api(
+    username: str,
+    password: str,
+    cookie_directory: str | None = None,
+):
+    """Return an authenticated ``PyiCloudService`` instance.
+
+    Args:
+        username: Apple ID email address.
+        password: App-specific password.
+        cookie_directory: Optional directory for session cookies.
+
+    Returns:
+        An authenticated ``PyiCloudService`` instance.
+
+    Raises:
+        ImportError: If ``pyicloud`` is not installed.
+        ValueError: If authentication fails or 2FA is required interactively.
+    """
+    from pyicloud import PyiCloudService  # noqa: S404 – unofficial third-party iCloud client
+
+    kwargs: dict = {}
+    if cookie_directory:
+        kwargs["cookie_directory"] = cookie_directory
+
+    api = PyiCloudService(username, password, **kwargs)
+
+    # If 2SA/2FA is required the user must use an app-specific password instead.
+    if api.requires_2sa or api.requires_2fa:
+        raise ValueError(
+            "iCloud account requires two-factor authentication.  "
+            "Please generate an app-specific password at https://appleid.apple.com "
+            "and use it as ICLOUD_PASSWORD."
+        )
+
+    return api
+
+
+def _navigate_to_folder(drive_root, folder_path: str):
+    """Navigate into (or create) the folder hierarchy described by *folder_path*.
+
+    Args:
+        drive_root: The iCloud Drive root node (``api.drive``).
+        folder_path: ``/``-separated path such as ``Documents/Uploads``.
+
+    Returns:
+        The drive node representing the target folder.
+    """
+    node = drive_root
+    if not folder_path:
+        return node
+
+    parts = [p for p in folder_path.strip("/").split("/") if p]
+    for part in parts:
+        children = {child.name: child for child in node.dir()}
+        if part in children:
+            node = children[part]
+        else:
+            # Create the missing folder
+            node = node.mkdir(part)
+    return node
+
+
+@celery.task(base=UploadTaskWithRetry, bind=True)
+def upload_to_icloud(self, file_path: str, file_id: int = None, folder_override: str = None):
+    """Upload a file to Apple iCloud Drive.
+
+    Args:
+        file_path: Local path to the file to upload.
+        file_id: Optional ``FileRecord.id`` for progress logging.
+        folder_override: If provided, overrides the default ``ICLOUD_FOLDER``
+            setting for this upload.
+    """
+    task_id = self.request.id
+    logger.info(f"[{task_id}] Starting iCloud Drive upload: {file_path}")
+    log_task_progress(
+        task_id,
+        "upload_to_icloud",
+        "in_progress",
+        f"Uploading to iCloud Drive: {os.path.basename(file_path)}",
+        file_id=file_id,
+    )
+
+    # ------------------------------------------------------------------
+    # Validate inputs
+    # ------------------------------------------------------------------
+    if not os.path.exists(file_path):
+        error_msg = f"File not found: {file_path}"
+        logger.error(f"[{task_id}] {error_msg}")
+        log_task_progress(task_id, "upload_to_icloud", "failure", error_msg, file_id=file_id)
+        raise FileNotFoundError(error_msg)
+
+    if not settings.icloud_username or not settings.icloud_password:
+        error_msg = "iCloud credentials are not configured (ICLOUD_USERNAME / ICLOUD_PASSWORD)"
+        logger.error(f"[{task_id}] {error_msg}")
+        log_task_progress(task_id, "upload_to_icloud", "failure", error_msg, file_id=file_id)
+        raise ValueError(error_msg)
+
+    filename = os.path.basename(file_path)
+    target_folder = folder_override if folder_override is not None else (settings.icloud_folder or "")
+
+    # ------------------------------------------------------------------
+    # Authenticate & upload
+    # ------------------------------------------------------------------
+    try:
+        api = _get_icloud_api(
+            settings.icloud_username,
+            settings.icloud_password,
+            settings.icloud_cookie_directory,
+        )
+
+        folder_node = _navigate_to_folder(api.drive, target_folder)
+
+        with open(file_path, "rb") as fh:
+            folder_node.upload(fh)
+
+        logger.info(f"[{task_id}] Successfully uploaded {filename} to iCloud Drive folder '{target_folder}'")
+        log_task_progress(
+            task_id,
+            "upload_to_icloud",
+            "success",
+            f"Uploaded to iCloud Drive: {filename}",
+            file_id=file_id,
+        )
+        return {
+            "status": "Completed",
+            "file": file_path,
+            "icloud_folder": target_folder or "/",
+        }
+
+    except Exception as e:
+        error_msg = f"Error uploading {filename} to iCloud Drive: {e}"
+        logger.error(f"[{task_id}] {error_msg}")
+        log_task_progress(task_id, "upload_to_icloud", "failure", error_msg, file_id=file_id)
+        raise RuntimeError(error_msg) from e

app/tasks/upload_to_user_integration.py

@@ -0,0 +1,763 @@
+#!/usr/bin/env python3
+"""
+Upload dispatcher for user-specific destination integrations.
+
+This module provides a Celery task that uploads a processed document to a
+specific ``UserIntegration`` record using that integration's own stored
+config and decrypted credentials — instead of the global application settings.
+
+It is the per-destination counterpart of :func:`send_to_all_destinations`
+and is dispatched by :func:`send_to_user_destinations` once per active
+DESTINATION integration that belongs to the document's owner.
+
+Credential shapes per integration type (mirrors the UserIntegration docstring):
+
+DROPBOX     credentials = {"refresh_token", "app_key", "app_secret"}
+            config      = {"folder": "/DocuElevate"}
+
+S3          credentials = {"access_key_id", "secret_access_key"}
+            config      = {"bucket", "region", "endpoint_url", "folder_prefix"}
+
+GOOGLE_DRIVE
+  OAuth     credentials = {"client_id", "client_secret", "refresh_token"}
+            config      = {"folder_id"}
+  SA        credentials = {"credentials_json"}
+            config      = {"folder_id"}
+
+ONEDRIVE    credentials = {"client_id", "client_secret", "refresh_token"}
+            config      = {"folder_path", "tenant_id"}
+
+WEBDAV /
+NEXTCLOUD   credentials = {"username", "password"}
+            config      = {"url", "folder"}
+
+FTP         credentials = {"password"}
+            config      = {"host", "username", "port", "folder", "use_tls"}
+
+SFTP        credentials = {"password"} or {"private_key"}
+            config      = {"host", "username", "port", "folder"}
+
+EMAIL       credentials = {"password"}
+            config      = {"host", "username", "port", "recipient",
+                           "use_tls", "sender_name"}
+
+PAPERLESS   credentials = {"api_token"}
+            config      = {"host"}
+
+RCLONE      credentials = {"rclone_conf"}   (full rclone config file text)
+            config      = {"remote": "myremote:", "folder": "dest/path"}
+"""
+
+import ftplib  # nosec B402
+import json
+import logging
+import os
+import subprocess  # nosec B404
+from datetime import datetime, timezone
+from typing import Any
+from urllib.parse import urljoin
+
+from app.celery_app import celery
+from app.database import SessionLocal
+from app.models import IntegrationType, UserIntegration
+from app.tasks.retry_config import UploadTaskWithRetry
+from app.utils.encryption import decrypt_value
+from app.utils.logging import log_task_progress
+
+logger = logging.getLogger(__name__)
+
+# Maximum characters to store in UserIntegration.last_error
+_MAX_ERROR_LENGTH = 500
+
+
+# ---------------------------------------------------------------------------
+# Per-type upload helpers
+# ---------------------------------------------------------------------------
+
+
+def _upload_dropbox(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to Dropbox using per-user OAuth credentials."""
+    import dropbox
+
+    app_key = creds.get("app_key") or ""
+    app_secret = creds.get("app_secret") or ""
+    refresh_token = creds.get("refresh_token") or ""
+
+    if not (app_key and app_secret and refresh_token):
+        raise ValueError("Dropbox integration is missing app_key, app_secret or refresh_token in credentials")
+
+    dbx = dropbox.Dropbox(app_key=app_key, app_secret=app_secret, oauth2_refresh_token=refresh_token)
+
+    remote_folder = cfg.get("folder", "/DocuElevate").rstrip("/")
+    filename = os.path.basename(file_path)
+    remote_path = f"{remote_folder}/{filename}"
+    if not remote_path.startswith("/"):
+        remote_path = "/" + remote_path
+
+    file_size = os.path.getsize(file_path)
+    with open(file_path, "rb") as fh:
+        if file_size > 10 * 1024 * 1024:
+            chunk_size = 4 * 1024 * 1024
+            session_start = dbx.files_upload_session_start(fh.read(chunk_size))
+            cursor = dropbox.files.UploadSessionCursor(session_start.session_id, fh.tell())
+            while fh.tell() < file_size:
+                if (file_size - fh.tell()) <= chunk_size:
+                    dbx.files_upload_session_finish(
+                        fh.read(chunk_size),
+                        cursor,
+                        dropbox.files.CommitInfo(path=remote_path, mode=dropbox.files.WriteMode.overwrite),
+                    )
+                else:
+                    dbx.files_upload_session_append_v2(fh.read(chunk_size), cursor)
+                    cursor.offset = fh.tell()
+        else:
+            dbx.files_upload(fh.read(), remote_path, mode=dropbox.files.WriteMode.overwrite)
+
+    logger.info("[%s] Dropbox upload complete: %s", task_id, remote_path)
+    return {"status": "Completed", "dropbox_path": remote_path}
+
+
+def _upload_s3(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to Amazon S3 (or S3-compatible) using per-user credentials."""
+    import boto3
+    from botocore.exceptions import ClientError
+
+    bucket = cfg.get("bucket") or ""
+    region = cfg.get("region") or "us-east-1"
+    endpoint_url = cfg.get("endpoint_url") or None
+    folder_prefix = cfg.get("folder_prefix") or ""
+    storage_class = cfg.get("storage_class") or "STANDARD"
+
+    access_key = creds.get("access_key_id") or ""
+    secret_key = creds.get("secret_access_key") or ""
+
+    if not bucket:
+        raise ValueError("S3 integration is missing bucket in config")
+    if not (access_key and secret_key):
+        raise ValueError("S3 integration is missing access_key_id or secret_access_key in credentials")
+
+    client_kwargs: dict[str, Any] = {
+        "region_name": region,
+        "aws_access_key_id": access_key,
+        "aws_secret_access_key": secret_key,
+    }
+    if endpoint_url:
+        client_kwargs["endpoint_url"] = endpoint_url
+
+    s3 = boto3.client("s3", **client_kwargs)
+
+    filename = os.path.basename(file_path)
+    s3_key = f"{folder_prefix.rstrip('/')}/{filename}" if folder_prefix else filename
+
+    try:
+        s3.upload_file(file_path, bucket, s3_key, ExtraArgs={"StorageClass": storage_class})
+    except ClientError as exc:
+        raise RuntimeError(f"S3 upload failed: {exc}") from exc
+
+    logger.info("[%s] S3 upload complete: s3://%s/%s", task_id, bucket, s3_key)
+    return {"status": "Completed", "s3_bucket": bucket, "s3_key": s3_key}
+
+
+def _upload_google_drive(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to Google Drive using per-user OAuth or service-account credentials."""
+    from googleapiclient.discovery import build
+    from googleapiclient.http import MediaFileUpload
+
+    folder_id = cfg.get("folder_id") or ""
+    filename = os.path.basename(file_path)
+
+    # Prefer OAuth credentials (client_id + client_secret + refresh_token)
+    client_id = creds.get("client_id") or ""
+    client_secret = creds.get("client_secret") or ""
+    refresh_token = creds.get("refresh_token") or ""
+    credentials_json = creds.get("credentials_json") or ""
+
+    if client_id and client_secret and refresh_token:
+        from google.auth.transport.requests import Request
+        from google.oauth2.credentials import Credentials as OAuthCredentials
+
+        google_creds = OAuthCredentials(
+            None,
+            refresh_token=refresh_token,
+            token_uri="https://oauth2.googleapis.com/token",
+            client_id=client_id,
+            client_secret=client_secret,
+            scopes=["https://www.googleapis.com/auth/drive.file"],
+        )
+        google_creds.refresh(Request())
+        service = build("drive", "v3", credentials=google_creds)
+    elif credentials_json:
+        from google.oauth2.service_account import Credentials as SACredentials
+
+        creds_dict = json.loads(credentials_json)
+        sa_creds = SACredentials.from_service_account_info(creds_dict, scopes=["https://www.googleapis.com/auth/drive"])
+        service = build("drive", "v3", credentials=sa_creds)
+    else:
+        raise ValueError("Google Drive integration requires either OAuth credentials or credentials_json")
+
+    file_metadata: dict[str, Any] = {"name": filename}
+    if folder_id:
+        file_metadata["parents"] = [folder_id]
+
+    media = MediaFileUpload(file_path, mimetype="application/pdf", resumable=True)
+    file_obj = service.files().create(body=file_metadata, media_body=media, fields="id,name,webViewLink").execute()
+
+    gdrive_id = file_obj.get("id")
+    web_link = file_obj.get("webViewLink")
+    logger.info("[%s] Google Drive upload complete: %s (%s)", task_id, gdrive_id, web_link)
+    return {"status": "Completed", "google_drive_file_id": gdrive_id, "web_link": web_link}
+
+
+def _upload_onedrive(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to OneDrive using per-user MSAL credentials."""
+    import urllib.parse
+
+    import msal
+    import requests as _requests
+
+    client_id = creds.get("client_id") or ""
+    client_secret = creds.get("client_secret") or ""
+    refresh_token = creds.get("refresh_token") or ""
+    tenant = cfg.get("tenant_id") or "common"
+    folder_path = cfg.get("folder_path") or ""
+
+    if not (client_id and client_secret):
+        raise ValueError("OneDrive integration is missing client_id or client_secret in credentials")
+
+    scopes = ["https://graph.microsoft.com/.default"]
+    msal_app = msal.ConfidentialClientApplication(
+        client_id=client_id,
+        client_credential=client_secret,
+        authority=f"https://login.microsoftonline.com/{tenant}",
+    )
+
+    if refresh_token:
+        token_resp = msal_app.acquire_token_by_refresh_token(refresh_token=refresh_token, scopes=scopes)
+    else:
+        token_resp = msal_app.acquire_token_for_client(scopes=scopes)
+
+    if "access_token" not in token_resp:
+        raise ValueError(f"OneDrive token acquisition failed: {token_resp.get('error_description', 'unknown')}")
+
+    access_token = token_resp["access_token"]
+    filename = os.path.basename(file_path)
+
+    # Build upload-session URL
+    if folder_path:
+        folder_path = folder_path.strip("/")
+        encoded_path = "/".join(urllib.parse.quote(p) for p in folder_path.split("/"))
+        encoded_file = urllib.parse.quote(filename)
+        item_path = f"/root:/{encoded_path}/{encoded_file}:/createUploadSession"
+    else:
+        encoded_file = urllib.parse.quote(filename)
+        item_path = f"/root:/{encoded_file}:/createUploadSession"
+
+    session_url = f"https://graph.microsoft.com/v1.0/me/drive{item_path}"
+    headers = {"Authorization": f"Bearer {access_token}", "Content-Type": "application/json"}
+    resp = _requests.post(
+        session_url, headers=headers, json={"item": {"@microsoft.graph.conflictBehavior": "replace"}}, timeout=30
+    )
+    resp.raise_for_status()
+    upload_url = resp.json()["uploadUrl"]
+
+    file_size = os.path.getsize(file_path)
+    chunk_size = 10 * 1024 * 1024
+    with open(file_path, "rb") as fh:
+        chunk_num = 0
+        while True:
+            chunk = fh.read(chunk_size)
+            if not chunk:
+                break
+            start = chunk_num * chunk_size
+            end = start + len(chunk) - 1
+            upload_headers = {
+                "Content-Length": str(len(chunk)),
+                "Content-Range": f"bytes {start}-{end}/{file_size}",
+            }
+            upload_resp = _requests.put(upload_url, headers=upload_headers, data=chunk, timeout=120)
+            if upload_resp.status_code not in (201, 202):
+                raise RuntimeError(f"OneDrive chunk upload failed: {upload_resp.status_code}")
+            chunk_num += 1
+
+    logger.info("[%s] OneDrive upload complete: %s/%s", task_id, folder_path, filename)
+    return {"status": "Completed", "onedrive_folder": folder_path, "filename": filename}
+
+
+def _upload_webdav(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to a WebDAV server using per-user credentials."""
+    import requests as _requests
+
+    url = cfg.get("url") or ""
+    folder = cfg.get("folder") or ""
+    username = creds.get("username") or ""
+    password = creds.get("password") or ""
+    verify_ssl = cfg.get("verify_ssl", True)
+
+    if not url:
+        raise ValueError("WebDAV integration is missing url in config")
+
+    filename = os.path.basename(file_path)
+    folder = folder.lstrip("/")
+    target = urljoin(url.rstrip("/") + "/", folder)
+    if not target.endswith("/"):
+        target += "/"
+    dest = urljoin(target, filename)
+
+    with open(file_path, "rb") as fh:
+        resp = _requests.put(dest, auth=(username, password), data=fh, verify=verify_ssl, timeout=120)
+
+    if resp.status_code not in (200, 201, 204):
+        raise RuntimeError(f"WebDAV upload failed: {resp.status_code} {resp.text[:200]}")
+
+    logger.info("[%s] WebDAV upload complete: %s", task_id, dest)
+    return {"status": "Completed", "webdav_url": dest}
+
+
+def _upload_nextcloud(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to Nextcloud (WebDAV) using per-user credentials."""
+    # Nextcloud uses WebDAV under the hood; reuse the WebDAV helper.
+    return _upload_webdav(file_path, cfg, creds, task_id)
+
+
+def _upload_ftp(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to an FTP/FTPS server using per-user credentials."""
+    host = cfg.get("host") or ""
+    port = int(cfg.get("port") or 21)
+    username = cfg.get("username") or ""
+    folder = cfg.get("folder") or ""
+    use_tls = cfg.get("use_tls", True)
+    password = creds.get("password") or ""
+    filename = os.path.basename(file_path)
+
+    if not host:
+        raise ValueError("FTP integration is missing host in config")
+
+    ftp: ftplib.FTP
+    if use_tls:
+        ftp = ftplib.FTP_TLS()  # nosec B321  # noqa: S321
+        ftp.connect(host=host, port=port)
+        ftp.login(user=username, passwd=password)
+        ftp.prot_p()
+    else:
+        ftp = ftplib.FTP()  # nosec B321  # noqa: S321
+        ftp.connect(host=host, port=port)
+        ftp.login(user=username, passwd=password)
+
+    if folder:
+        folder_stripped = folder.lstrip("/")
+        try:
+            ftp.cwd(folder_stripped)
+        except ftplib.error_perm:
+            parts = folder_stripped.split("/")
+            current = ""
+            for part in parts:
+                if not part:
+                    continue
+                current += f"/{part}"
+                try:
+                    ftp.cwd(current)
+                except ftplib.error_perm:
+                    ftp.mkd(current)
+                    ftp.cwd(current)
+
+    with open(file_path, "rb") as fh:
+        ftp.storbinary(f"STOR {filename}", fh)
+    ftp.quit()
+
+    logger.info("[%s] FTP upload complete: %s/%s", task_id, host, filename)
+    return {"status": "Completed", "ftp_host": host, "filename": filename}
+
+
+def _upload_sftp(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to an SFTP server using per-user credentials."""
+    import paramiko
+
+    host = cfg.get("host") or ""
+    port = int(cfg.get("port") or 22)
+    username = cfg.get("username") or ""
+    folder = cfg.get("folder") or ""
+    password = creds.get("password") or ""
+    private_key_text = creds.get("private_key") or ""
+    filename = os.path.basename(file_path)
+
+    if not host:
+        raise ValueError("SFTP integration is missing host in config")
+
+    ssh = paramiko.SSHClient()
+    ssh.load_system_host_keys()
+    ssh.set_missing_host_key_policy(paramiko.RejectPolicy())
+
+    connect_kwargs: dict[str, Any] = {"hostname": host, "port": port, "username": username}
+    if private_key_text:
+        import io
+
+        pkey = paramiko.RSAKey.from_private_key(io.StringIO(private_key_text))
+        connect_kwargs["pkey"] = pkey
+    elif password:
+        connect_kwargs["password"] = password
+    else:
+        raise ValueError("SFTP integration requires password or private_key in credentials")
+
+    ssh.connect(**connect_kwargs)
+    sftp = ssh.open_sftp()
+
+    remote_path = f"{folder.rstrip('/')}/{filename}" if folder else filename
+    if folder and folder.startswith("/") and not remote_path.startswith("/"):
+        remote_path = "/" + remote_path
+
+    sftp.put(file_path, remote_path)
+    sftp.close()
+    ssh.close()
+
+    logger.info("[%s] SFTP upload complete: %s:%s", task_id, host, remote_path)
+    return {"status": "Completed", "sftp_host": host, "sftp_path": remote_path}
+
+
+def _upload_paperless(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to a Paperless-ngx instance using per-user API token."""
+    import time
+
+    import requests as _requests
+
+    host = (cfg.get("host") or "").rstrip("/")
+    api_token = creds.get("api_token") or ""
+    filename = os.path.basename(file_path)
+
+    if not host:
+        raise ValueError("Paperless integration is missing host in config")
+    if not api_token:
+        raise ValueError("Paperless integration is missing api_token in credentials")
+
+    headers = {"Authorization": f"Token {api_token}"}
+    post_url = f"{host}/api/documents/post_document/"
+
+    with open(file_path, "rb") as fh:
+        resp = _requests.post(
+            post_url,
+            headers=headers,
+            files={"document": (filename, fh, "application/pdf")},
+            data={"title": filename},
+            timeout=120,
+        )
+    resp.raise_for_status()
+    raw_task_id = resp.text.strip().strip('"').strip("'")
+
+    # Poll for completion (up to 30 s)
+    task_url = f"{host}/api/tasks/"
+    doc_id = None
+    for _ in range(10):
+        time.sleep(3)
+        try:
+            poll_resp = _requests.get(task_url, headers=headers, params={"task_id": raw_task_id}, timeout=30)
+            poll_resp.raise_for_status()
+            tasks_data = poll_resp.json()
+            if isinstance(tasks_data, dict) and "results" in tasks_data:
+                tasks_data = tasks_data["results"]
+            if tasks_data:
+                info = tasks_data[0]
+                status = info.get("status")
+                if status == "SUCCESS":
+                    doc_id = info.get("related_document")
+                    break
+                elif status == "FAILURE":
+                    raise RuntimeError(f"Paperless processing failed: {info.get('result')}")
+        except RuntimeError:
+            raise
+        except Exception as poll_exc:
+            logger.warning("[%s] Paperless poll error: %s", task_id, poll_exc)
+
+    logger.info("[%s] Paperless upload complete: doc_id=%s", task_id, doc_id)
+    return {"status": "Completed", "paperless_host": host, "paperless_document_id": doc_id}
+
+
+def _upload_email(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Send *file_path* as an email attachment using per-user SMTP credentials."""
+    import smtplib
+    from email.mime.application import MIMEApplication
+    from email.mime.multipart import MIMEMultipart
+    from email.mime.text import MIMEText
+
+    host = cfg.get("host") or ""
+    port = int(cfg.get("port") or 587)
+    username = cfg.get("username") or ""
+    recipient = cfg.get("recipient") or ""
+    use_tls = cfg.get("use_tls", True)
+    sender_name = cfg.get("sender_name") or "DocuElevate"
+    password = creds.get("password") or ""
+    filename = os.path.basename(file_path)
+
+    if not (host and recipient):
+        raise ValueError("Email integration is missing host or recipient in config")
+
+    msg = MIMEMultipart()
+    msg["From"] = f"{sender_name} <{username}>" if username else sender_name
+    msg["To"] = recipient
+    msg["Subject"] = f"Document: {filename}"
+    msg.attach(MIMEText(f"Please find the attached document: {filename}", "plain"))
+
+    with open(file_path, "rb") as fh:
+        part = MIMEApplication(fh.read(), Name=filename)
+    part["Content-Disposition"] = f'attachment; filename="{filename}"'
+    msg.attach(part)
+
+    if use_tls:
+        import ssl
+
+        tls_context = ssl.create_default_context()
+        with smtplib.SMTP(host, port, timeout=30) as smtp:
+            smtp.starttls(context=tls_context)
+            if username and password:
+                smtp.login(username, password)
+            smtp.sendmail(msg["From"], [recipient], msg.as_string())
+    else:
+        # Plaintext SMTP — only use when explicitly configured and TLS is unavailable.
+        # Credentials and content will be transmitted without encryption.
+        with smtplib.SMTP(host, port, timeout=30) as smtp:  # nosec B608
+            if username and password:
+                smtp.login(username, password)
+            smtp.sendmail(msg["From"], [recipient], msg.as_string())
+
+    logger.info("[%s] Email upload complete: sent to %s", task_id, recipient)
+    return {"status": "Completed", "recipient": recipient}
+
+
+def _upload_rclone(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Copy *file_path* to an rclone remote using per-user rclone config."""
+    import re
+    import tempfile
+
+    remote = cfg.get("remote") or ""
+    folder = cfg.get("folder") or ""
+    rclone_conf_text = creds.get("rclone_conf") or ""
+    filename = os.path.basename(file_path)
+
+    if not remote:
+        raise ValueError("Rclone integration is missing remote in config")
+    if not rclone_conf_text:
+        raise ValueError("Rclone integration is missing rclone_conf in credentials")
+
+    # Validate remote and folder to prevent shell metacharacter injection.
+    # rclone remote names are alphanumeric + hyphens/underscores followed by ':'.
+    # folder paths must not contain shell-dangerous characters.
+    _SAFE_REMOTE_RE = re.compile(r"^[A-Za-z0-9_\-]+:(/[A-Za-z0-9_.@\-/ ]*)?$")
+    _SAFE_FOLDER_RE = re.compile(r"^[A-Za-z0-9_.@\-/ ]*$")
+    if not _SAFE_REMOTE_RE.match(remote):
+        raise ValueError(f"Rclone remote contains unsafe characters: {remote!r}")
+    if folder and not _SAFE_FOLDER_RE.match(folder):
+        raise ValueError(f"Rclone folder contains unsafe characters: {folder!r}")
+
+    # Write the user's rclone config to a temp file so we don't touch the system config
+    with tempfile.NamedTemporaryFile(mode="w", suffix=".conf", delete=False) as tmp_conf:
+        tmp_conf.write(rclone_conf_text)
+        conf_path = tmp_conf.name
+
+    dest = f"{remote.rstrip('/')}/{folder.strip('/')}/{filename}" if folder else f"{remote.rstrip('/')}/{filename}"
+    dest = dest.replace("//", "/")
+
+    try:
+        result = subprocess.run(  # nosec B603  # noqa: S603 S607
+            ["rclone", "copyto", f"--config={conf_path}", file_path, dest],  # noqa: S603 S607
+            capture_output=True,
+            text=True,
+            timeout=300,
+            check=False,
+        )
+        if result.returncode != 0:
+            raise RuntimeError(f"rclone exited {result.returncode}: {result.stderr[:300]}")
+    finally:
+        os.unlink(conf_path)
+
+    logger.info("[%s] Rclone upload complete: %s", task_id, dest)
+    return {"status": "Completed", "rclone_dest": dest}
+
+
+def _upload_icloud(file_path: str, cfg: dict[str, Any], creds: dict[str, Any], task_id: str) -> dict[str, Any]:
+    """Upload *file_path* to iCloud Drive using per-user credentials.
+
+    Expected *cfg* keys:
+        * ``folder`` – target folder path inside iCloud Drive (e.g. ``Documents/Uploads``).
+        * ``cookie_directory`` – (optional) path for session cookie persistence.
+
+    Expected *creds* keys:
+        * ``username`` – Apple ID email address.
+        * ``password`` – app-specific password.
+    """
+    from app.tasks.upload_to_icloud import _get_icloud_api, _navigate_to_folder
+
+    username = creds.get("username") or ""
+    password = creds.get("password") or ""
+    folder = cfg.get("folder") or ""
+    cookie_directory = cfg.get("cookie_directory") or None
+
+    if not username or not password:
+        raise ValueError("iCloud integration is missing username or password in credentials")
+
+    api = _get_icloud_api(username, password, cookie_directory)
+    folder_node = _navigate_to_folder(api.drive, folder)
+
+    with open(file_path, "rb") as fh:
+        folder_node.upload(fh)
+
+    logger.info("[%s] iCloud Drive upload complete: folder=%s", task_id, folder or "/")
+    return {"status": "Completed", "icloud_folder": folder or "/"}
+
+
+# Map IntegrationType → upload helper
+_UPLOAD_HANDLERS = {
+    IntegrationType.DROPBOX: _upload_dropbox,
+    IntegrationType.S3: _upload_s3,
+    IntegrationType.GOOGLE_DRIVE: _upload_google_drive,
+    IntegrationType.ONEDRIVE: _upload_onedrive,
+    IntegrationType.WEBDAV: _upload_webdav,
+    IntegrationType.NEXTCLOUD: _upload_nextcloud,
+    IntegrationType.FTP: _upload_ftp,
+    IntegrationType.SFTP: _upload_sftp,
+    IntegrationType.PAPERLESS: _upload_paperless,
+    IntegrationType.EMAIL: _upload_email,
+    IntegrationType.RCLONE: _upload_rclone,
+    IntegrationType.ICLOUD: _upload_icloud,
+}
+
+
+# ---------------------------------------------------------------------------
+# Celery task
+# ---------------------------------------------------------------------------
+
+
+@celery.task(base=UploadTaskWithRetry, bind=True)
+def upload_to_user_integration(self, file_path: str, integration_id: int, file_id: int | None = None) -> dict[str, Any]:
+    """Upload *file_path* to the destination described by the given UserIntegration record.
+
+    This task is dispatched once per active DESTINATION UserIntegration that
+    belongs to a document's owner.  Credentials are decrypted at runtime so
+    they never travel across the Celery message bus in plaintext.
+
+    Args:
+        file_path: Absolute path to the processed document file.
+        integration_id: Primary key of the ``UserIntegration`` record.
+        file_id: Optional ``FileRecord.id`` used for progress logging.
+
+    Returns:
+        A dict with at least ``{"status": "Completed", ...}`` on success.
+
+    Raises:
+        FileNotFoundError: When *file_path* does not exist.
+        ValueError: When the integration record is not found or has missing config.
+        RuntimeError: When the underlying upload operation fails.
+    """
+    task_id = self.request.id
+    filename = os.path.basename(file_path)
+
+    log_task_progress(
+        task_id,
+        f"upload_to_user_integration_{integration_id}",
+        "in_progress",
+        f"Uploading {filename} to integration {integration_id}",
+        file_id=file_id,
+    )
+
+    if not os.path.exists(file_path):
+        error_msg = f"File not found: {file_path}"
+        logger.error("[%s] %s", task_id, error_msg)
+        log_task_progress(
+            task_id, f"upload_to_user_integration_{integration_id}", "failure", error_msg, file_id=file_id
+        )
+        raise FileNotFoundError(error_msg)
+
+    with SessionLocal() as db:
+        integration: UserIntegration | None = (
+            db.query(UserIntegration).filter(UserIntegration.id == integration_id).first()
+        )
+        if integration is None:
+            error_msg = f"UserIntegration {integration_id} not found"
+            logger.error("[%s] %s", task_id, error_msg)
+            log_task_progress(
+                task_id, f"upload_to_user_integration_{integration_id}", "failure", error_msg, file_id=file_id
+            )
+            raise ValueError(error_msg)
+
+        itype = integration.integration_type
+        int_name = integration.name
+        owner_id = integration.owner_id
+
+        # Parse config (non-sensitive) and decrypt credentials (sensitive)
+        try:
+            cfg: dict[str, Any] = json.loads(integration.config) if integration.config else {}
+        except json.JSONDecodeError as exc:
+            raise ValueError(f"Integration {integration_id} has invalid JSON in config: {exc}") from exc
+
+        try:
+            raw_creds = decrypt_value(integration.credentials) if integration.credentials else None
+            creds: dict[str, Any] = json.loads(raw_creds) if raw_creds else {}
+        except json.JSONDecodeError as exc:
+            raise ValueError(f"Integration {integration_id} has invalid JSON in credentials: {exc}") from exc
+
+    handler = _UPLOAD_HANDLERS.get(itype)
+    if handler is None:
+        error_msg = f"No upload handler registered for integration type '{itype}' (integration {integration_id})"
+        logger.warning("[%s] %s", task_id, error_msg)
+        log_task_progress(
+            task_id, f"upload_to_user_integration_{integration_id}", "skipped", error_msg, file_id=file_id
+        )
+        return {"status": "Skipped", "reason": error_msg}
+
+    logger.info(
+        "[%s] Uploading %s via %s integration '%s' (id=%d, owner=%s)",
+        task_id,
+        filename,
+        itype,
+        int_name,
+        integration_id,
+        owner_id,
+    )
+
+    try:
+        result = handler(file_path, cfg, creds, task_id)
+
+        # Update last_used_at on success
+        with SessionLocal() as db:
+            integ = db.query(UserIntegration).filter(UserIntegration.id == integration_id).first()
+            if integ:
+                integ.last_used_at = datetime.now(timezone.utc)
+                integ.last_error = None
+                db.commit()
+
+        log_task_progress(
+            task_id,
+            f"upload_to_user_integration_{integration_id}",
+            "success",
+            f"Uploaded to {itype} '{int_name}': {filename}",
+            file_id=file_id,
+        )
+        return result
+
+    except Exception as exc:
+        error_msg = str(exc)[:_MAX_ERROR_LENGTH]
+        logger.error(
+            "[%s] Upload to integration %d (%s '%s') failed: %s",
+            task_id,
+            integration_id,
+            itype,
+            int_name,
+            error_msg,
+        )
+
+        # Persist error for operator visibility
+        try:
+            with SessionLocal() as db:
+                integ = db.query(UserIntegration).filter(UserIntegration.id == integration_id).first()
+                if integ:
+                    integ.last_used_at = datetime.now(timezone.utc)
+                    integ.last_error = error_msg
+                    db.commit()
+        except Exception as db_exc:  # noqa: BLE001
+            logger.warning("[%s] Could not persist last_error for integration %d: %s", task_id, integration_id, db_exc)
+
+        log_task_progress(
+            task_id,
+            f"upload_to_user_integration_{integration_id}",
+            "failure",
+            f"Upload to {itype} '{int_name}' failed: {error_msg}",
+            file_id=file_id,
+        )
+        raise

app/utils/allowed_types.py

@@ -131,3 +131,157 @@ ALLOWED_EXTENSIONS: set[str] = {
     ".md",
     ".markdown",
 }
+
+# ---------------------------------------------------------------------------
+# Fine-grained file-type categories used by IMAP ingestion profiles.
+# Each category groups related MIME types and extensions so that users can
+# enable/disable a logical collection of formats (e.g. "images") rather than
+# having to manage individual MIME strings.
+# ---------------------------------------------------------------------------
+
+FILE_TYPE_CATEGORIES: dict[str, dict] = {
+    "pdf": {
+        "label": "PDF",
+        "description": "PDF documents (.pdf)",
+        "mime_types": frozenset({"application/pdf"}),
+        "extensions": frozenset({".pdf"}),
+    },
+    "office": {
+        "label": "Microsoft Office",
+        "description": "Word, Excel and PowerPoint files (.doc, .docx, .xls, .xlsx, .ppt, .pptx, …)",
+        "mime_types": frozenset(
+            {
+                "application/msword",
+                "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+                "application/vnd.openxmlformats-officedocument.wordprocessingml.template",
+                "application/vnd.ms-word.document.macroEnabled.12",
+                "application/vnd.ms-word.template.macroEnabled.12",
+                "application/vnd.ms-excel",
+                "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+                "application/vnd.openxmlformats-officedocument.spreadsheetml.template",
+                "application/vnd.ms-excel.sheet.macroEnabled.12",
+                "application/vnd.ms-excel.sheet.binary.macroEnabled.12",
+                "application/vnd.ms-powerpoint",
+                "application/vnd.openxmlformats-officedocument.presentationml.presentation",
+                "application/vnd.openxmlformats-officedocument.presentationml.template",
+                "application/vnd.openxmlformats-officedocument.presentationml.slideshow",
+                "application/vnd.ms-powerpoint.presentation.macroEnabled.12",
+            }
+        ),
+        "extensions": frozenset(
+            {
+                ".doc",
+                ".docx",
+                ".docm",
+                ".dot",
+                ".dotx",
+                ".dotm",
+                ".xls",
+                ".xlsx",
+                ".xlsm",
+                ".xlsb",
+                ".xlt",
+                ".xltx",
+                ".xlw",
+                ".ppt",
+                ".pptx",
+                ".pptm",
+                ".pps",
+                ".ppsx",
+                ".pot",
+                ".potx",
+            }
+        ),
+    },
+    "opendocument": {
+        "label": "OpenDocument (LibreOffice)",
+        "description": "LibreOffice / OpenOffice files (.odt, .ods, .odp, …)",
+        "mime_types": frozenset(
+            {
+                "application/vnd.oasis.opendocument.text",
+                "application/vnd.oasis.opendocument.spreadsheet",
+                "application/vnd.oasis.opendocument.presentation",
+                "application/vnd.oasis.opendocument.graphics",
+                "application/vnd.oasis.opendocument.formula",
+            }
+        ),
+        "extensions": frozenset({".odt", ".ods", ".odp", ".odg", ".odf"}),
+    },
+    "text": {
+        "label": "Text & Data",
+        "description": "Plain text, CSV and RTF files (.txt, .csv, .rtf)",
+        "mime_types": frozenset(
+            {
+                "text/plain",
+                "text/csv",
+                "application/rtf",
+                "text/rtf",
+            }
+        ),
+        "extensions": frozenset({".txt", ".csv", ".rtf"}),
+    },
+    "web": {
+        "label": "Web & Markup",
+        "description": "HTML and Markdown files (.html, .htm, .md, .markdown)",
+        "mime_types": frozenset(
+            {
+                "text/html",
+                "text/markdown",
+                "text/x-markdown",
+            }
+        ),
+        "extensions": frozenset({".html", ".htm", ".md", ".markdown"}),
+    },
+    "images": {
+        "label": "Images",
+        "description": "Image files (.jpg, .png, .gif, .bmp, .tiff, .webp, .svg)",
+        "mime_types": frozenset(
+            {
+                "image/jpeg",
+                "image/jpg",
+                "image/png",
+                "image/gif",
+                "image/bmp",
+                "image/tiff",
+                "image/webp",
+                "image/svg+xml",
+            }
+        ),
+        "extensions": frozenset(
+            {
+                ".jpg",
+                ".jpeg",
+                ".png",
+                ".gif",
+                ".bmp",
+                ".tiff",
+                ".tif",
+                ".webp",
+                ".svg",
+            }
+        ),
+    },
+}
+
+# Default categories for the "documents only" built-in profile (no images)
+DEFAULT_CATEGORIES: list[str] = ["pdf", "office", "opendocument", "text", "web"]
+# All categories including images
+ALL_CATEGORIES: list[str] = ["pdf", "office", "opendocument", "text", "web", "images"]
+
+
+def get_allowed_types_for_categories(
+    categories: list[str],
+) -> tuple[frozenset[str], frozenset[str]]:
+    """Return ``(mime_types, extensions)`` for the given category list.
+
+    Unknown category names are silently ignored so that future categories
+    don't break existing profiles.
+    """
+    mime_types: set[str] = set()
+    extensions: set[str] = set()
+    for cat in categories:
+        info = FILE_TYPE_CATEGORIES.get(cat)
+        if info:
+            mime_types |= info["mime_types"]
+            extensions |= info["extensions"]
+    return frozenset(mime_types), frozenset(extensions)

app/utils/audit_service.py

@@ -0,0 +1,331 @@
+"""
+Comprehensive audit-event service for DocuElevate.
+
+Provides helpers to **record** audit events (append-only database writes)
+and to optionally **forward** them to external SIEM systems.
+
+Supported SIEM transports:
+* **Syslog** – RFC 5424 structured-data messages over UDP or TCP.
+* **HTTP** – JSON POST payloads compatible with Splunk HEC, Logstash
+  HTTP input, Grafana Loki push API, and any generic webhook endpoint.
+"""
+
+import json
+import logging
+import re
+import socket
+import threading
+from datetime import datetime, timezone
+from typing import Any
+
+import httpx
+from fastapi import Request
+from sqlalchemy.orm import Session
+
+from app.config import settings
+from app.middleware.audit_log import get_client_ip, get_username
+from app.models import AuditLog
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Public helpers
+# ---------------------------------------------------------------------------
+
+
+def record_event(
+    db: Session,
+    *,
+    action: str,
+    user: str = "system",
+    resource_type: str | None = None,
+    resource_id: str | None = None,
+    ip_address: str | None = None,
+    details: dict[str, Any] | None = None,
+    severity: str = "info",
+) -> AuditLog:
+    """Persist an audit event and optionally forward it to SIEM.
+
+    Args:
+        db: Active SQLAlchemy session.
+        action: Short action identifier (e.g. ``"login"``, ``"document.create"``).
+        user: Username performing the action.
+        resource_type: Category of the affected resource (``"document"``, ``"user"`` …).
+        resource_id: Identifier of the affected resource.
+        ip_address: Client IP address (``None`` when not applicable).
+        details: Arbitrary key/value context serialised as JSON.
+        severity: One of ``info``, ``warning``, ``error``, ``critical``.
+
+    Returns:
+        The newly created :class:`AuditLog` row.
+    """
+    details_json = json.dumps(details, default=str) if details else None
+
+    entry = AuditLog(
+        user=user,
+        action=action,
+        resource_type=resource_type,
+        resource_id=str(resource_id) if resource_id is not None else None,
+        ip_address=ip_address,
+        details=details_json,
+        severity=severity,
+    )
+    db.add(entry)
+    db.commit()
+    db.refresh(entry)
+
+    # Fire-and-forget SIEM forwarding in a background thread so we never
+    # block the request path.
+    if settings.audit_siem_enabled:
+        payload = _build_siem_payload(entry)
+        thread = threading.Thread(target=_forward_to_siem, args=(payload,), daemon=True)
+        thread.start()
+
+    return entry
+
+
+def record_event_from_request(
+    db: Session,
+    request: Request,
+    *,
+    action: str,
+    resource_type: str | None = None,
+    resource_id: str | None = None,
+    details: dict[str, Any] | None = None,
+    severity: str = "info",
+) -> AuditLog:
+    """Convenience wrapper that extracts user and IP from a :class:`Request`.
+
+    Args:
+        db: Active SQLAlchemy session.
+        request: The current HTTP request.
+        action: Short action identifier.
+        resource_type: Category of the affected resource.
+        resource_id: Identifier of the affected resource.
+        details: Arbitrary key/value context serialised as JSON.
+        severity: One of ``info``, ``warning``, ``error``, ``critical``.
+
+    Returns:
+        The newly created :class:`AuditLog` row.
+    """
+    return record_event(
+        db,
+        action=action,
+        user=get_username(request),
+        resource_type=resource_type,
+        resource_id=resource_id,
+        ip_address=get_client_ip(request),
+        details=details,
+        severity=severity,
+    )
+
+
+def query_events(
+    db: Session,
+    *,
+    action: str | None = None,
+    user: str | None = None,
+    resource_type: str | None = None,
+    severity: str | None = None,
+    since: datetime | None = None,
+    until: datetime | None = None,
+    limit: int = 200,
+    offset: int = 0,
+) -> list[AuditLog]:
+    """Query audit log entries with optional filtering.
+
+    Args:
+        db: Active SQLAlchemy session.
+        action: Filter by action string (exact match).
+        user: Filter by username (exact match).
+        resource_type: Filter by resource type (exact match).
+        severity: Filter by severity level (exact match).
+        since: Only events at or after this timestamp.
+        until: Only events at or before this timestamp.
+        limit: Maximum number of rows to return.
+        offset: Number of rows to skip (for pagination).
+
+    Returns:
+        List of :class:`AuditLog` rows ordered by *timestamp descending*.
+    """
+    q = db.query(AuditLog)
+    if action:
+        q = q.filter(AuditLog.action == action)
+    if user:
+        q = q.filter(AuditLog.user == user)
+    if resource_type:
+        q = q.filter(AuditLog.resource_type == resource_type)
+    if severity:
+        q = q.filter(AuditLog.severity == severity)
+    if since:
+        q = q.filter(AuditLog.timestamp >= since)
+    if until:
+        q = q.filter(AuditLog.timestamp <= until)
+    return q.order_by(AuditLog.timestamp.desc()).offset(offset).limit(limit).all()
+
+
+def count_events(
+    db: Session,
+    *,
+    action: str | None = None,
+    user: str | None = None,
+    resource_type: str | None = None,
+    severity: str | None = None,
+    since: datetime | None = None,
+    until: datetime | None = None,
+) -> int:
+    """Return the total count of events matching the given filters.
+
+    Args:
+        db: Active SQLAlchemy session.
+        action: Filter by action string.
+        user: Filter by username.
+        resource_type: Filter by resource type.
+        severity: Filter by severity level.
+        since: Only events at or after this timestamp.
+        until: Only events at or before this timestamp.
+
+    Returns:
+        Integer count.
+    """
+    q = db.query(AuditLog)
+    if action:
+        q = q.filter(AuditLog.action == action)
+    if user:
+        q = q.filter(AuditLog.user == user)
+    if resource_type:
+        q = q.filter(AuditLog.resource_type == resource_type)
+    if severity:
+        q = q.filter(AuditLog.severity == severity)
+    if since:
+        q = q.filter(AuditLog.timestamp >= since)
+    if until:
+        q = q.filter(AuditLog.timestamp <= until)
+    return q.count()
+
+
+# ---------------------------------------------------------------------------
+# SIEM forwarding internals
+# ---------------------------------------------------------------------------
+
+_SYSLOG_FACILITY_LOCAL0 = 16
+_SYSLOG_SEVERITY_MAP = {
+    "info": 6,
+    "warning": 4,
+    "error": 3,
+    "critical": 2,
+}
+
+
+def _build_siem_payload(entry: AuditLog) -> dict[str, Any]:
+    """Convert an :class:`AuditLog` row into a plain dict for SIEM delivery."""
+    ts = entry.timestamp if entry.timestamp else datetime.now(timezone.utc)
+    return {
+        "id": entry.id,
+        "timestamp": ts.isoformat(),
+        "user": entry.user,
+        "action": entry.action,
+        "resource_type": entry.resource_type,
+        "resource_id": entry.resource_id,
+        "ip_address": entry.ip_address,
+        "details": entry.details,
+        "severity": entry.severity,
+        "source": "docuelevate",
+    }
+
+
+def _forward_to_siem(payload: dict[str, Any]) -> None:
+    """Route a SIEM payload to the configured transport."""
+    transport = settings.audit_siem_transport.lower()
+    try:
+        if transport == "syslog":
+            _send_syslog(payload)
+        elif transport == "http":
+            _send_http(payload)
+        else:
+            logger.warning("Unknown SIEM transport %r; skipping forwarding", transport)
+    except Exception:
+        logger.exception("Failed to forward audit event to SIEM (%s)", transport)
+
+
+def _send_syslog(payload: dict[str, Any]) -> None:
+    """Send a RFC 5424 syslog message to the configured receiver."""
+    severity_num = _SYSLOG_SEVERITY_MAP.get(payload.get("severity", "info"), 6)
+    priority = _SYSLOG_FACILITY_LOCAL0 * 8 + severity_num
+    ts = payload.get("timestamp", datetime.now(timezone.utc).isoformat())
+    hostname = socket.gethostname()
+    app_name = "docuelevate"
+    msg_id = payload.get("action", "-")
+
+    # Structured data (SD) element with key event fields.
+    sd = (
+        f'[docuelevate@0 user="{payload.get("user", "-")}" '
+        f'action="{payload.get("action", "-")}" '
+        f'resource_type="{payload.get("resource_type", "-")}" '
+        f'resource_id="{payload.get("resource_id", "-")}" '
+        f'ip="{payload.get("ip_address", "-")}"]'
+    )
+    message = json.dumps(payload, default=str)
+    syslog_msg = f"<{priority}>1 {ts} {hostname} {app_name} - {msg_id} {sd} {message}"
+
+    proto = settings.audit_siem_syslog_protocol.lower()
+    host = settings.audit_siem_syslog_host
+    port = settings.audit_siem_syslog_port
+
+    if proto == "tcp":
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
+            sock.settimeout(5)
+            sock.connect((host, port))
+            sock.sendall(syslog_msg.encode("utf-8"))
+    else:
+        with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
+            sock.settimeout(5)
+            sock.sendto(syslog_msg.encode("utf-8"), (host, port))
+
+    logger.debug("Syslog audit event sent to %s:%s (%s)", host, port, proto)
+
+
+def _send_http(payload: dict[str, Any]) -> None:
+    """POST a JSON audit event to the configured HTTP endpoint."""
+    url = settings.audit_siem_http_url
+    if not url:
+        logger.warning("SIEM HTTP URL not configured; skipping HTTP forwarding")
+        return
+
+    headers: dict[str, str] = {"Content-Type": "application/json"}
+    token = settings.audit_siem_http_token
+    if token:
+        headers["Authorization"] = f"Bearer {token}"
+
+    # Parse custom headers (comma-separated "Key:Value" pairs).
+    # Reject headers that could override security-critical ones already set,
+    # and validate that header names contain only RFC 7230 token characters.
+    _PROTECTED_HEADERS = {"authorization", "content-type", "host"}
+    _VALID_HEADER_NAME = re.compile(r"^[A-Za-z0-9!#$%&'*+\-.^_`|~]+$")
+    raw_custom = settings.audit_siem_http_custom_headers
+    if raw_custom:
+        for raw_pair in raw_custom.split(","):
+            pair = raw_pair.strip()
+            if ":" in pair:
+                k, _, v = pair.partition(":")
+                name = k.strip()
+                if not name or not _VALID_HEADER_NAME.match(name):
+                    logger.warning("Skipping invalid SIEM custom header name: %r", name)
+                    continue
+                if name.lower() in _PROTECTED_HEADERS:
+                    logger.warning("Skipping protected SIEM custom header: %r", name)
+                    continue
+                headers[name] = v.strip()
+
+    # Wrap in Splunk HEC-style envelope when URL contains ``/services/collector``.
+    body: dict[str, Any]
+    if "/services/collector" in url:
+        body = {"event": payload, "sourcetype": "docuelevate:audit", "source": "docuelevate"}
+    else:
+        body = payload
+
+    with httpx.Client(timeout=10) as client:
+        resp = client.post(url, json=body, headers=headers)
+        resp.raise_for_status()
+
+    logger.debug("HTTP audit event forwarded to %s (status %s)", url, resp.status_code)

app/utils/compliance_service.py

@@ -0,0 +1,433 @@
+"""Compliance service for managing GDPR, HIPAA, and SOC2 compliance templates.
+
+Provides pre-built compliance configurations that can be applied with one click
+to ensure the DocuElevate instance meets regulatory requirements.
+"""
+
+import json
+import logging
+from datetime import datetime, timezone
+from typing import Any
+
+from sqlalchemy.orm import Session
+
+from app.models import ComplianceTemplate
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Pre-built compliance template definitions
+# ---------------------------------------------------------------------------
+
+COMPLIANCE_TEMPLATES: dict[str, dict[str, Any]] = {
+    "gdpr": {
+        "display_name": "GDPR (General Data Protection Regulation)",
+        "description": (
+            "European Union regulation for data protection and privacy. "
+            "Enforces data minimisation, encryption at rest, audit logging, "
+            "and limits PII exposure in telemetry."
+        ),
+        "settings": {
+            "auth_enabled": "True",
+            "sentry_send_default_pii": "False",
+            "security_headers_enabled": "True",
+            "security_header_hsts_enabled": "True",
+            "security_header_csp_enabled": "True",
+            "security_header_x_frame_options_enabled": "True",
+            "enable_deduplication": "True",
+        },
+        "checks": [
+            {
+                "key": "auth_enabled",
+                "expected": "True",
+                "label": "Authentication enabled",
+                "description": "User authentication must be enabled to control access to personal data.",
+            },
+            {
+                "key": "sentry_send_default_pii",
+                "expected": "False",
+                "label": "PII excluded from telemetry",
+                "description": "Personally identifiable information must not be sent to external monitoring services.",
+            },
+            {
+                "key": "security_headers_enabled",
+                "expected": "True",
+                "label": "Security headers enabled",
+                "description": "HTTP security headers protect against common web vulnerabilities.",
+            },
+            {
+                "key": "security_header_hsts_enabled",
+                "expected": "True",
+                "label": "HSTS enabled",
+                "description": "HTTP Strict Transport Security ensures encrypted connections.",
+            },
+            {
+                "key": "security_header_csp_enabled",
+                "expected": "True",
+                "label": "Content Security Policy enabled",
+                "description": "CSP headers prevent cross-site scripting and data injection attacks.",
+            },
+            {
+                "key": "security_header_x_frame_options_enabled",
+                "expected": "True",
+                "label": "Clickjacking protection enabled",
+                "description": "X-Frame-Options header prevents clickjacking attacks.",
+            },
+            {
+                "key": "enable_deduplication",
+                "expected": "True",
+                "label": "Deduplication enabled",
+                "description": "Data minimisation: avoid storing duplicate documents.",
+            },
+        ],
+    },
+    "hipaa": {
+        "display_name": "HIPAA (Health Insurance Portability and Accountability Act)",
+        "description": (
+            "United States regulation for protecting health information. "
+            "Requires strong access controls, audit trails, encryption, "
+            "and strict session management."
+        ),
+        "settings": {
+            "auth_enabled": "True",
+            "multi_user_enabled": "True",
+            "sentry_send_default_pii": "False",
+            "security_headers_enabled": "True",
+            "security_header_hsts_enabled": "True",
+            "security_header_csp_enabled": "True",
+            "security_header_x_frame_options_enabled": "True",
+            "enable_deduplication": "True",
+        },
+        "checks": [
+            {
+                "key": "auth_enabled",
+                "expected": "True",
+                "label": "Authentication enabled",
+                "description": "Access controls are required to protect electronic Protected Health Information (ePHI).",
+            },
+            {
+                "key": "multi_user_enabled",
+                "expected": "True",
+                "label": "Multi-user mode enabled",
+                "description": "Individual user accounts required for access accountability.",
+            },
+            {
+                "key": "sentry_send_default_pii",
+                "expected": "False",
+                "label": "PII excluded from telemetry",
+                "description": "Protected Health Information must not be sent to external services.",
+            },
+            {
+                "key": "security_headers_enabled",
+                "expected": "True",
+                "label": "Security headers enabled",
+                "description": "Security headers protect ePHI during transmission.",
+            },
+            {
+                "key": "security_header_hsts_enabled",
+                "expected": "True",
+                "label": "HSTS enabled",
+                "description": "Encrypted transport required for all ePHI transmissions.",
+            },
+            {
+                "key": "security_header_csp_enabled",
+                "expected": "True",
+                "label": "Content Security Policy enabled",
+                "description": "CSP prevents injection attacks that could expose ePHI.",
+            },
+            {
+                "key": "security_header_x_frame_options_enabled",
+                "expected": "True",
+                "label": "Clickjacking protection enabled",
+                "description": "Prevents embedding the application in unauthorized frames.",
+            },
+            {
+                "key": "enable_deduplication",
+                "expected": "True",
+                "label": "Deduplication enabled",
+                "description": "Minimise data footprint for ePHI.",
+            },
+        ],
+    },
+    "soc2": {
+        "display_name": "SOC 2 (Service Organization Control 2)",
+        "description": (
+            "Trust Service Criteria framework for service organisations. "
+            "Focuses on security, availability, processing integrity, "
+            "confidentiality, and privacy."
+        ),
+        "settings": {
+            "auth_enabled": "True",
+            "multi_user_enabled": "True",
+            "sentry_send_default_pii": "False",
+            "security_headers_enabled": "True",
+            "security_header_hsts_enabled": "True",
+            "security_header_csp_enabled": "True",
+            "security_header_x_frame_options_enabled": "True",
+            "enable_deduplication": "True",
+        },
+        "checks": [
+            {
+                "key": "auth_enabled",
+                "expected": "True",
+                "label": "Authentication enabled",
+                "description": "Logical access controls required (CC6.1).",
+            },
+            {
+                "key": "multi_user_enabled",
+                "expected": "True",
+                "label": "Multi-user mode enabled",
+                "description": "Individual user accounts for access management (CC6.2).",
+            },
+            {
+                "key": "sentry_send_default_pii",
+                "expected": "False",
+                "label": "PII excluded from telemetry",
+                "description": "Confidential information must not leak to external services (CC6.7).",
+            },
+            {
+                "key": "security_headers_enabled",
+                "expected": "True",
+                "label": "Security headers enabled",
+                "description": "Protection against common web threats (CC6.6).",
+            },
+            {
+                "key": "security_header_hsts_enabled",
+                "expected": "True",
+                "label": "HSTS enabled",
+                "description": "Encrypted transport in transit (CC6.7).",
+            },
+            {
+                "key": "security_header_csp_enabled",
+                "expected": "True",
+                "label": "Content Security Policy enabled",
+                "description": "Application-level security controls (CC6.6).",
+            },
+            {
+                "key": "security_header_x_frame_options_enabled",
+                "expected": "True",
+                "label": "Clickjacking protection enabled",
+                "description": "UI redress attack prevention (CC6.6).",
+            },
+            {
+                "key": "enable_deduplication",
+                "expected": "True",
+                "label": "Deduplication enabled",
+                "description": "Data integrity through deduplication (PI1.1).",
+            },
+        ],
+    },
+}
+
+
+def seed_compliance_templates(db: Session) -> None:
+    """Create or update the built-in compliance template rows.
+
+    Called once at application startup to ensure the ``compliance_templates``
+    table always contains the latest definitions.
+    """
+    for name, defn in COMPLIANCE_TEMPLATES.items():
+        existing = db.query(ComplianceTemplate).filter(ComplianceTemplate.name == name).first()
+        if existing is None:
+            template = ComplianceTemplate(
+                name=name,
+                display_name=defn["display_name"],
+                description=defn["description"],
+                settings_json=json.dumps(defn["settings"]),
+                enabled=False,
+                status="not_applied",
+            )
+            db.add(template)
+            logger.info(f"Seeded compliance template: {name}")
+        else:
+            # Update display_name and description if changed, but preserve user state
+            existing.display_name = defn["display_name"]
+            existing.description = defn["description"]
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to seed compliance templates")
+
+
+def get_all_templates(db: Session) -> list[dict[str, Any]]:
+    """Return all compliance templates with their current status."""
+    templates = db.query(ComplianceTemplate).order_by(ComplianceTemplate.name).all()
+    result = []
+    for t in templates:
+        defn = COMPLIANCE_TEMPLATES.get(t.name, {})
+        checks = defn.get("checks", [])
+        result.append(
+            {
+                "id": t.id,
+                "name": t.name,
+                "display_name": t.display_name,
+                "description": t.description,
+                "enabled": t.enabled,
+                "status": t.status,
+                "applied_at": t.applied_at.isoformat() if t.applied_at else None,
+                "applied_by": t.applied_by,
+                "settings": json.loads(t.settings_json) if t.settings_json else {},
+                "checks": checks,
+                "check_count": len(checks),
+            }
+        )
+    return result
+
+
+def get_template_by_name(db: Session, name: str) -> ComplianceTemplate | None:
+    """Retrieve a single compliance template by name."""
+    return db.query(ComplianceTemplate).filter(ComplianceTemplate.name == name).first()
+
+
+def evaluate_template_status(db: Session, name: str) -> dict[str, Any]:
+    """Evaluate the compliance status of a template against live settings.
+
+    Returns a dict with ``status``, ``total``, ``passed``, ``failed``, and
+    a list of individual ``check_results``.
+    """
+    from app.config import settings as app_settings
+    from app.utils.settings_service import get_all_settings_from_db
+
+    defn = COMPLIANCE_TEMPLATES.get(name)
+    if defn is None:
+        return {"status": "unknown", "total": 0, "passed": 0, "failed": 0, "check_results": []}
+
+    db_settings = get_all_settings_from_db(db)
+    checks = defn.get("checks", [])
+    results: list[dict[str, Any]] = []
+    passed = 0
+
+    for check in checks:
+        key = check["key"]
+        expected = check["expected"]
+
+        # Resolve effective value: DB > config object
+        if key in db_settings and db_settings[key] is not None:
+            actual = str(db_settings[key])
+        else:
+            actual = str(getattr(app_settings, key, ""))
+
+        is_passing = actual.lower() == expected.lower()
+        if is_passing:
+            passed += 1
+
+        results.append(
+            {
+                "key": key,
+                "label": check["label"],
+                "description": check["description"],
+                "expected": expected,
+                "actual": actual,
+                "passing": is_passing,
+            }
+        )
+
+    total = len(checks)
+    if passed == total:
+        status = "compliant"
+    elif passed > 0:
+        status = "partial"
+    else:
+        status = "non_compliant"
+
+    return {
+        "status": status,
+        "total": total,
+        "passed": passed,
+        "failed": total - passed,
+        "check_results": results,
+    }
+
+
+def apply_template(db: Session, name: str, applied_by: str = "admin") -> dict[str, Any]:
+    """Apply a compliance template by writing its settings to the database.
+
+    Returns a summary of what was applied.
+    """
+    from app.utils.settings_service import save_setting_to_db
+
+    defn = COMPLIANCE_TEMPLATES.get(name)
+    if defn is None:
+        return {"success": False, "error": f"Unknown template: {name}"}
+
+    template = get_template_by_name(db, name)
+    if template is None:
+        return {"success": False, "error": f"Template not found in database: {name}"}
+
+    applied_settings: dict[str, str] = {}
+    errors: list[str] = []
+
+    for key, value in defn["settings"].items():
+        try:
+            save_setting_to_db(db, key, value, changed_by=f"compliance:{name}")
+            applied_settings[key] = value
+        except Exception as e:
+            errors.append(f"{key}: {e}")
+            logger.error(f"Failed to apply compliance setting {key}={value}: {e}")
+
+    # Update the template record
+    now = datetime.now(timezone.utc)
+    template.enabled = True
+    template.settings_json = json.dumps(applied_settings)
+    template.applied_at = now
+    template.applied_by = applied_by
+
+    # Evaluate and store status
+    eval_result = evaluate_template_status(db, name)
+    template.status = eval_result["status"]
+
+    try:
+        db.commit()
+    except Exception:
+        db.rollback()
+        logger.exception(f"Failed to update compliance template record: {name}")
+        return {"success": False, "error": "Database commit failed"}
+
+    logger.info(f"Applied compliance template '{name}' by {applied_by}: {len(applied_settings)} settings written")
+
+    return {
+        "success": len(errors) == 0,
+        "template": name,
+        "applied_settings": applied_settings,
+        "errors": errors,
+        "status": eval_result,
+    }
+
+
+def get_compliance_summary(db: Session) -> dict[str, Any]:
+    """Return a high-level compliance dashboard summary across all templates."""
+    templates = db.query(ComplianceTemplate).order_by(ComplianceTemplate.name).all()
+    summary: list[dict[str, Any]] = []
+    total_checks = 0
+    total_passed = 0
+
+    for t in templates:
+        eval_result = evaluate_template_status(db, t.name)
+        total_checks += eval_result["total"]
+        total_passed += eval_result["passed"]
+        summary.append(
+            {
+                "name": t.name,
+                "display_name": t.display_name,
+                "enabled": t.enabled,
+                "status": eval_result["status"],
+                "total": eval_result["total"],
+                "passed": eval_result["passed"],
+                "failed": eval_result["failed"],
+                "applied_at": t.applied_at.isoformat() if t.applied_at else None,
+                "applied_by": t.applied_by,
+            }
+        )
+
+    overall = "compliant" if total_checks > 0 and total_passed == total_checks else "non_compliant"
+    if 0 < total_passed < total_checks:
+        overall = "partial"
+
+    return {
+        "overall_status": overall,
+        "total_checks": total_checks,
+        "total_passed": total_passed,
+        "total_failed": total_checks - total_passed,
+        "templates": summary,
+    }

app/utils/config_validator/providers.py

@@ -144,7 +144,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "dropbox_app_secret", None)
             and getattr(settings, "dropbox_refresh_token", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "dropbox_enabled", True),
         "description": "Upload files to Dropbox cloud storage",
         "details": {
             "folder": getattr(settings, "dropbox_folder", "Not set"),
@@ -161,7 +161,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
         "configured": bool(
             getattr(settings, "dest_email_host", None) and getattr(settings, "dest_email_default_recipient", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "dest_email_enabled", True),
         "description": "Send documents via email",
         "details": {
             "host": getattr(settings, "dest_email_host", "Not set"),
@@ -183,7 +183,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "ftp_username", None)
             and getattr(settings, "ftp_password", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "ftp_enabled", True),
         "description": "Upload files to FTP server",
         "details": {
             "host": getattr(settings, "ftp_host", "Not set"),
@@ -214,7 +214,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
         "name": "Google Drive",
         "icon": "fa-brands fa-google-drive",
         "configured": is_configured and bool(getattr(settings, "google_drive_folder_id", None)),
-        "enabled": True,
+        "enabled": getattr(settings, "google_drive_enabled", True),
         "description": "Store documents in Google Drive",
         "details": {
             "auth_type": "OAuth" if use_oauth else "Service Account",
@@ -250,7 +250,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "nextcloud_username", None)
             and getattr(settings, "nextcloud_password", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "nextcloud_enabled", True),
         "description": "Store documents in NextCloud",
         "details": {
             "url": getattr(settings, "nextcloud_upload_url", "Not set"),
@@ -270,7 +270,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "onedrive_client_secret", None)
             and getattr(settings, "onedrive_refresh_token", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "onedrive_enabled", True),
         "description": "Store documents in Microsoft OneDrive",
         "details": {
             "client_id": getattr(settings, "onedrive_client_id", "Not set"),
@@ -288,7 +288,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
         "configured": bool(
             getattr(settings, "paperless_host", None) and getattr(settings, "paperless_ngx_api_token", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "paperless_enabled", True),
         "description": "Document management system for digital archives",
         "details": {
             "host": getattr(settings, "paperless_host", "Not set"),
@@ -305,7 +305,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "aws_access_key_id", None)
             and getattr(settings, "aws_secret_access_key", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "s3_enabled", True),
         "description": "Store documents in S3-compatible object storage",
         "details": {
             "bucket": getattr(settings, "s3_bucket_name", "Not set"),
@@ -327,7 +327,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "sftp_username", None)
             and (getattr(settings, "sftp_password", None) or getattr(settings, "sftp_private_key", None))
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "sftp_enabled", True),
         "description": "Upload files to SFTP server",
         "details": {
             "host": getattr(settings, "sftp_host", "Not set"),
@@ -362,7 +362,7 @@ def get_provider_status() -> dict[str, dict[str, object]]:
             and getattr(settings, "webdav_username", None)
             and getattr(settings, "webdav_password", None)
         ),
-        "enabled": True,
+        "enabled": getattr(settings, "webdav_enabled", True),
         "description": "Store documents on WebDAV servers",
         "details": {
             "url": getattr(settings, "webdav_url", "Not set"),
@@ -373,4 +373,19 @@ def get_provider_status() -> dict[str, dict[str, object]]:
         },
     }
 
+    # Check iCloud Drive configuration
+    providers["iCloud Drive"] = {
+        "name": "iCloud Drive",
+        "icon": "fa-brands fa-apple",
+        "configured": bool(getattr(settings, "icloud_username", None) and getattr(settings, "icloud_password", None)),
+        "enabled": getattr(settings, "icloud_enabled", True),
+        "description": "Store documents in Apple iCloud Drive",
+        "details": {
+            "username": getattr(settings, "icloud_username", "Not set"),
+            "password": mask_sensitive_value(getattr(settings, "icloud_password", None)),
+            "folder": getattr(settings, "icloud_folder", "Not set"),
+            "cookie_directory": getattr(settings, "icloud_cookie_directory", "Not set"),
+        },
+    }
+
     return providers

app/utils/config_validator/validators.py

@@ -59,13 +59,43 @@ def validate_auth_config() -> list[str]:
             and getattr(settings, "authentik_config_url", None)
         )
 
-        if not using_simple_auth and not using_oidc:
-            issues.append("Neither simple authentication nor OIDC are properly configured")
+        # Check if any social login provider is enabled
+        using_social_login = any(
+            getattr(settings, f"social_auth_{p}_enabled", False) for p in ("google", "microsoft", "apple", "dropbox")
+        )
+
+        if not using_simple_auth and not using_oidc and not using_social_login:
+            issues.append("Neither simple authentication, OIDC, nor social login are properly configured")
 
         # If using OIDC, check for provider name
         if using_oidc and not getattr(settings, "oauth_provider_name", None):
             issues.append("OAUTH_PROVIDER_NAME is not configured but OIDC is enabled")
 
+        # Validate individual social login provider configs
+        if getattr(settings, "social_auth_google_enabled", False):
+            if not getattr(settings, "social_auth_google_client_id", None):
+                issues.append("SOCIAL_AUTH_GOOGLE_CLIENT_ID is required when Google login is enabled")
+            if not getattr(settings, "social_auth_google_client_secret", None):
+                issues.append("SOCIAL_AUTH_GOOGLE_CLIENT_SECRET is required when Google login is enabled")
+
+        if getattr(settings, "social_auth_microsoft_enabled", False):
+            if not getattr(settings, "social_auth_microsoft_client_id", None):
+                issues.append("SOCIAL_AUTH_MICROSOFT_CLIENT_ID is required when Microsoft login is enabled")
+            if not getattr(settings, "social_auth_microsoft_client_secret", None):
+                issues.append("SOCIAL_AUTH_MICROSOFT_CLIENT_SECRET is required when Microsoft login is enabled")
+
+        if getattr(settings, "social_auth_apple_enabled", False):
+            if not getattr(settings, "social_auth_apple_client_id", None):
+                issues.append("SOCIAL_AUTH_APPLE_CLIENT_ID is required when Apple login is enabled")
+            if not getattr(settings, "social_auth_apple_team_id", None):
+                issues.append("SOCIAL_AUTH_APPLE_TEAM_ID is required when Apple login is enabled")
+
+        if getattr(settings, "social_auth_dropbox_enabled", False):
+            if not getattr(settings, "social_auth_dropbox_client_id", None):
+                issues.append("SOCIAL_AUTH_DROPBOX_CLIENT_ID is required when Dropbox login is enabled")
+            if not getattr(settings, "social_auth_dropbox_client_secret", None):
+                issues.append("SOCIAL_AUTH_DROPBOX_CLIENT_SECRET is required when Dropbox login is enabled")
+
     return issues
 
 

app/utils/db_migrate.py

@@ -32,8 +32,10 @@ _TABLE_ORDER = [
     "processing_logs",
     "application_settings",
     "settings_audit_log",
+    "audit_logs",
     "saved_searches",
     "webhook_configs",
+    "shared_links",
 ]
 
 
@@ -83,7 +85,8 @@ def preview_migration(source_url: str) -> dict[str, Any]:
         with src_engine.connect() as conn:
             for table_name in tables:
                 # table_name is safe — sourced from inspect().get_table_names(), not user input
-                row = conn.execute(text(f'SELECT COUNT(*) FROM "{table_name}"')).fetchone()  # noqa: S608
+                quoted_table = conn.dialect.identifier_preparer.quote(table_name)
+                row = conn.execute(text(f"SELECT COUNT(*) FROM {quoted_table}")).fetchone()  # noqa: S608
                 count = row[0] if row else 0
                 result.append({"name": table_name, "row_count": count})
                 total += count

app/utils/filename_utils.py

@@ -8,6 +8,11 @@ from pathlib import Path
 
 logger = logging.getLogger(__name__)
 
+# Pattern for valid filenames (alphanumeric, dash, underscore, period, and space)
+# Used for validating GPT-provided filenames and other inputs
+VALID_FILENAME_PATTERN = r"^[\w\-\. ]+$"
+VALID_FILENAME_RE = re.compile(VALID_FILENAME_PATTERN)
+
 
 def get_unique_filename(original_path: str, check_exists_func: Callable[[str], bool] | None = None) -> str:
     """

app/utils/i18n.py

@@ -0,0 +1,765 @@
+"""Internationalization (i18n) and localization (l10n) utilities.
+
+Provides a JSON-based translation system for the DocuElevate UI with:
+
+* **77 supported languages** covering European, Asian, Middle-Eastern, African, and other languages
+* Browser ``Accept-Language`` detection with cookie & user-profile persistence
+* AI-powered fallback translation via the configured LLM provider
+* Locale-aware date, number, and file-size formatting helpers
+* Jinja2 integration via a ``_()`` global function
+
+Language resolution order:
+    1. User profile ``preferred_language`` (persisted in DB)
+    2. ``docuelevate_lang`` cookie
+    3. ``Accept-Language`` HTTP header
+    4. Default (``en``)
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+from datetime import date, datetime
+from functools import lru_cache
+from pathlib import Path
+from typing import Any
+
+from starlette.requests import Request
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Supported languages (ordered by priority)
+# ---------------------------------------------------------------------------
+
+SUPPORTED_LANGUAGES: list[dict[str, str]] = [
+    # --- Tier 1: Primary European languages ---
+    {"code": "en", "name": "English", "native": "English", "flag": "🇬🇧"},
+    {"code": "de", "name": "German", "native": "Deutsch", "flag": "🇩🇪"},
+    {"code": "fr", "name": "French", "native": "Français", "flag": "🇫🇷"},
+    {"code": "es", "name": "Spanish", "native": "Español", "flag": "🇪🇸"},
+    {"code": "it", "name": "Italian", "native": "Italiano", "flag": "🇮🇹"},
+    {"code": "pt", "name": "Portuguese", "native": "Português", "flag": "🇵🇹"},
+    # --- Tier 2: Western & Northern European ---
+    {"code": "nl", "name": "Dutch", "native": "Nederlands", "flag": "🇳🇱"},
+    {"code": "nb", "name": "Norwegian Bokmål", "native": "Norsk bokmål", "flag": "🇳🇴"},
+    {"code": "no", "name": "Norwegian", "native": "Norsk", "flag": "🇳🇴"},
+    {"code": "da", "name": "Danish", "native": "Dansk", "flag": "🇩🇰"},
+    {"code": "sv", "name": "Swedish", "native": "Svenska", "flag": "🇸🇪"},
+    {"code": "fi", "name": "Finnish", "native": "Suomi", "flag": "🇫🇮"},
+    {"code": "is", "name": "Icelandic", "native": "Íslenska", "flag": "🇮🇸"},
+    {"code": "ga", "name": "Irish", "native": "Gaeilge", "flag": "🇮🇪"},
+    {"code": "lb", "name": "Luxembourgish", "native": "Lëtzebuergesch", "flag": "🇱🇺"},
+    {"code": "ca", "name": "Catalan", "native": "Català", "flag": "🏴"},
+    {"code": "cy", "name": "Welsh", "native": "Cymraeg", "flag": "🏴󠁧󠁢󠁷󠁬󠁳󠁿"},  # Wales subdivision flag (U+1F3F4 + tag chars)
+    {"code": "fy", "name": "Western Frisian", "native": "Frysk", "flag": "🇳🇱"},
+    {"code": "gl", "name": "Galician", "native": "Galego", "flag": "🇪🇸"},
+    {"code": "li", "name": "Limburgish", "native": "Limburgs", "flag": "🇳🇱"},
+    {"code": "vls", "name": "Flemish", "native": "West-Vlams", "flag": "🇧🇪"},
+    {"code": "nds", "name": "Low German", "native": "Plattdüütsch", "flag": "🇩🇪"},
+    # --- Tier 3: Central & Eastern European ---
+    {"code": "pl", "name": "Polish", "native": "Polski", "flag": "🇵🇱"},
+    {"code": "cs", "name": "Czech", "native": "Čeština", "flag": "🇨🇿"},
+    {"code": "sk", "name": "Slovak", "native": "Slovenčina", "flag": "🇸🇰"},
+    {"code": "hu", "name": "Hungarian", "native": "Magyar", "flag": "🇭🇺"},
+    {"code": "sl", "name": "Slovenian", "native": "Slovenščina", "flag": "🇸🇮"},
+    {"code": "hr", "name": "Croatian", "native": "Hrvatski", "flag": "🇭🇷"},
+    {"code": "ro", "name": "Romanian", "native": "Română", "flag": "🇷🇴"},
+    {"code": "bg", "name": "Bulgarian", "native": "Български", "flag": "🇧🇬"},
+    {"code": "el", "name": "Greek", "native": "Ελληνικά", "flag": "🇬🇷"},
+    {"code": "et", "name": "Estonian", "native": "Eesti", "flag": "🇪🇪"},
+    {"code": "lv", "name": "Latvian", "native": "Latviešu", "flag": "🇱🇻"},
+    {"code": "lt", "name": "Lithuanian", "native": "Lietuvių", "flag": "🇱🇹"},
+    {"code": "sr", "name": "Serbian", "native": "Српски", "flag": "🇷🇸"},
+    # --- Tier 4: Non-EU European, Middle Eastern & African ---
+    {"code": "tr", "name": "Turkish", "native": "Türkçe", "flag": "🇹🇷"},
+    {"code": "uk", "name": "Ukrainian", "native": "Українська", "flag": "🇺🇦"},
+    {"code": "he", "name": "Hebrew", "native": "עברית", "flag": "🇮🇱"},
+    {"code": "ar", "name": "Arabic", "native": "العربية", "flag": "🇸🇦"},
+    {"code": "fa", "name": "Persian", "native": "فارسی", "flag": "🇮🇷"},
+    {"code": "af", "name": "Afrikaans", "native": "Afrikaans", "flag": "🇿🇦"},
+    # --- Tier 5: Asian languages ---
+    {"code": "zh", "name": "Chinese", "native": "中文", "flag": "🇨🇳"},
+    {"code": "zh-TW", "name": "Traditional Chinese", "native": "繁體中文", "flag": "🇹🇼"},
+    {"code": "ja", "name": "Japanese", "native": "日本語", "flag": "🇯🇵"},
+    {"code": "ko", "name": "Korean", "native": "한국어", "flag": "🇰🇷"},
+    {"code": "vi", "name": "Vietnamese", "native": "Tiếng Việt", "flag": "🇻🇳"},
+    {"code": "pa", "name": "Punjabi", "native": "ਪੰਜਾਬੀ", "flag": "🇮🇳"},
+    {"code": "kn", "name": "Kannada", "native": "ಕನ್ನಡ", "flag": "🇮🇳"},
+    {"code": "hi", "name": "Hindi", "native": "हिन्दी", "flag": "🇮🇳"},
+    {"code": "bn", "name": "Bengali", "native": "বাংলা", "flag": "🇧🇩"},
+    {"code": "gu", "name": "Gujarati", "native": "ગુજરાતી", "flag": "🇮🇳"},
+    {"code": "ml", "name": "Malayalam", "native": "മലയാളം", "flag": "🇮🇳"},
+    {"code": "mr", "name": "Marathi", "native": "मराठी", "flag": "🇮🇳"},
+    {"code": "ta", "name": "Tamil", "native": "தமிழ்", "flag": "🇮🇳"},
+    {"code": "te", "name": "Telugu", "native": "తెలుగు", "flag": "🇮🇳"},
+    {"code": "ur", "name": "Urdu", "native": "اردو", "flag": "🇵🇰"},
+    {"code": "si", "name": "Sinhala", "native": "සිංහල", "flag": "🇱🇰"},
+    {"code": "ne", "name": "Nepali", "native": "नेपाली", "flag": "🇳🇵"},
+    {"code": "th", "name": "Thai", "native": "ไทย", "flag": "🇹🇭"},
+    {"code": "km", "name": "Khmer", "native": "ខ្មែរ", "flag": "🇰🇭"},
+    {"code": "id", "name": "Indonesian", "native": "Bahasa Indonesia", "flag": "🇮🇩"},
+    {"code": "ms", "name": "Malay", "native": "Bahasa Melayu", "flag": "🇲🇾"},
+    {"code": "jv", "name": "Javanese", "native": "Basa Jawa", "flag": "🇮🇩"},
+    {"code": "tl", "name": "Tagalog", "native": "Filipino", "flag": "🇵🇭"},
+    {"code": "mn", "name": "Mongolian", "native": "Монгол", "flag": "🇲🇳"},
+    {"code": "kk", "name": "Kazakh", "native": "Қазақ тілі", "flag": "🇰🇿"},
+    {"code": "uz", "name": "Uzbek", "native": "Oʻzbekcha", "flag": "🇺🇿"},
+    {"code": "az", "name": "Azerbaijani", "native": "Azərbaycan dili", "flag": "🇦🇿"},
+    {"code": "hy", "name": "Armenian", "native": "Հայերեն", "flag": "🇦🇲"},
+    {"code": "ka", "name": "Georgian", "native": "ქართული", "flag": "🇬🇪"},
+    # --- Tier 6: African languages ---
+    {"code": "sw", "name": "Swahili", "native": "Kiswahili", "flag": "🇰🇪"},
+    {"code": "am", "name": "Amharic", "native": "አማርኛ", "flag": "🇪🇹"},
+    {"code": "ha", "name": "Hausa", "native": "Hausa", "flag": "🇳🇬"},
+    {"code": "yo", "name": "Yoruba", "native": "Yorùbá", "flag": "🇳🇬"},
+    {"code": "ig", "name": "Igbo", "native": "Igbo", "flag": "🇳🇬"},
+    {"code": "zu", "name": "Zulu", "native": "isiZulu", "flag": "🇿🇦"},
+    # --- Tier 7: Constructed & other languages ---
+    {"code": "eo", "name": "Esperanto", "native": "Esperanto", "flag": "🌍"},
+]
+
+SUPPORTED_LANGUAGE_CODES: set[str] = {lang["code"] for lang in SUPPORTED_LANGUAGES}
+DEFAULT_LANGUAGE = "en"
+
+# Lookup map for fast code → language-dict resolution
+_LANG_CODE_MAP: dict[str, dict[str, str]] = {lang["code"]: lang for lang in SUPPORTED_LANGUAGES}
+
+# Global-usage order used to fill remaining slots in the smart suggestions list
+_POPULAR_LANGUAGE_CODES: list[str] = ["en", "zh", "es", "ar", "fr", "de", "ja", "pt", "hi", "ko"]
+
+# ---------------------------------------------------------------------------
+# Translation file loading
+# ---------------------------------------------------------------------------
+
+_TRANSLATIONS_DIR = Path(__file__).resolve().parent.parent.parent / "frontend" / "translations"
+_translation_cache: dict[str, dict[str, str]] = {}
+
+
+def _load_translations(locale: str) -> dict[str, str]:
+    """Load the translation JSON file for *locale*, with caching."""
+    if locale in _translation_cache:
+        return _translation_cache[locale]
+
+    filepath = _TRANSLATIONS_DIR / f"{locale}.json"
+    if not filepath.is_file():
+        logger.warning("Translation file not found for locale '%s'", locale)
+        _translation_cache[locale] = {}
+        return {}
+
+    try:
+        data: dict[str, str] = json.loads(filepath.read_text(encoding="utf-8"))
+        _translation_cache[locale] = data
+        return data
+    except (json.JSONDecodeError, OSError):
+        logger.exception("Failed to load translations for '%s'", locale)
+        _translation_cache[locale] = {}
+        return {}
+
+
+def reload_translations() -> None:
+    """Clear the translation cache so files are re-read on next access."""
+    _translation_cache.clear()
+
+
+# ---------------------------------------------------------------------------
+# Core translation function
+# ---------------------------------------------------------------------------
+
+
+def translate(key: str, locale: str | None = None, **kwargs: Any) -> str:
+    """Return the translated string for *key* in *locale*.
+
+    Falls back through:
+        1. Requested *locale*
+        2. English (``en``)
+        3. The raw key itself (to keep the UI functional)
+
+    Positional placeholders ``{0}``, ``{1}`` or named placeholders
+    ``{name}`` in the translated string are interpolated via *kwargs*.
+    """
+    locale = locale if locale and locale in SUPPORTED_LANGUAGE_CODES else DEFAULT_LANGUAGE
+
+    translations = _load_translations(locale)
+    value = translations.get(key)
+
+    # Fallback to English
+    if value is None and locale != DEFAULT_LANGUAGE:
+        en_translations = _load_translations(DEFAULT_LANGUAGE)
+        value = en_translations.get(key)
+
+    # Fallback to key itself
+    if value is None:
+        value = key
+
+    if kwargs:
+        try:
+            value = value.format(**kwargs)
+        except (KeyError, IndexError):
+            pass  # Return unformatted string rather than crash
+
+    return value
+
+
+# ---------------------------------------------------------------------------
+# AI fallback translation (best-effort, non-blocking)
+# ---------------------------------------------------------------------------
+
+_ai_translation_cache: dict[tuple[str, str], str] = {}
+
+
+def translate_with_ai_fallback(text: str, target_locale: str) -> str:
+    """Translate *text* using the configured AI provider as a fallback.
+
+    Returns the original *text* unchanged when:
+    * The target locale is English (source language)
+    * The AI provider is unavailable or returns an error
+    * The translation has already been cached
+
+    Results are cached in-memory for the lifetime of the process.
+    """
+    if target_locale == DEFAULT_LANGUAGE or target_locale not in SUPPORTED_LANGUAGE_CODES:
+        return text
+
+    cache_key = (text, target_locale)
+    if cache_key in _ai_translation_cache:
+        return _ai_translation_cache[cache_key]
+
+    target_name = next(
+        (lang["name"] for lang in SUPPORTED_LANGUAGES if lang["code"] == target_locale),
+        target_locale,
+    )
+
+    try:
+        from litellm import completion  # type: ignore[import-untyped]
+
+        from app.config import settings
+
+        model = getattr(settings, "ai_model", None) or getattr(settings, "openai_model", "gpt-4o-mini")
+        response = completion(
+            model=model,
+            messages=[
+                {
+                    "role": "system",
+                    "content": (
+                        f"You are a professional translator. Translate the following UI text "
+                        f"from English to {target_name}. Return ONLY the translated text, "
+                        f"nothing else. Keep any HTML tags, placeholders like {{name}}, "
+                        f"and special characters intact."
+                    ),
+                },
+                {"role": "user", "content": text},
+            ],
+            max_tokens=256,
+            temperature=0.1,
+        )
+        translated = response.choices[0].message.content.strip()
+        _ai_translation_cache[cache_key] = translated
+        return translated
+    except Exception:
+        logger.debug("AI fallback translation failed for '%s' → %s", text[:50], target_locale)
+        return text
+
+
+# ---------------------------------------------------------------------------
+# Language detection
+# ---------------------------------------------------------------------------
+
+
+def detect_language(request: Request) -> str:
+    """Determine the preferred UI language from the request context.
+
+    Resolution order:
+        1. ``preferred_language`` stored in the user session
+        2. ``docuelevate_lang`` cookie
+        3. ``Accept-Language`` HTTP header (best match)
+        4. Default → ``en``
+    """
+    # 1. User session preference
+    if hasattr(request, "session"):
+        session_lang = request.session.get("preferred_language")
+        if isinstance(session_lang, str) and session_lang in SUPPORTED_LANGUAGE_CODES:
+            return session_lang
+
+    # 2. Cookie
+    if hasattr(request, "cookies"):
+        cookie_lang = request.cookies.get("docuelevate_lang")
+        if isinstance(cookie_lang, str) and cookie_lang in SUPPORTED_LANGUAGE_CODES:
+            return cookie_lang
+
+    # 3. Accept-Language header
+    accept = ""
+    if hasattr(request, "headers"):
+        accept = request.headers.get("accept-language", "")
+    lang = _parse_accept_language(accept)
+    if lang:
+        return lang
+
+    return DEFAULT_LANGUAGE
+
+
+def _parse_accept_language_entries(header: str) -> list[tuple[float, str]]:
+    """Parse an ``Accept-Language`` header into quality-sorted ``(q, tag)`` pairs."""
+    if not header:
+        return []
+
+    entries: list[tuple[float, str]] = []
+    for raw_part in header.split(","):
+        part = raw_part.strip()
+        if not part:
+            continue
+        if ";q=" in part:
+            lang_tag, _, q_str = part.partition(";q=")
+            try:
+                quality = float(q_str.strip())
+            except ValueError:
+                quality = 0.0
+        else:
+            lang_tag = part
+            quality = 1.0
+        entries.append((quality, lang_tag.strip().lower()))
+
+    entries.sort(key=lambda e: e[0], reverse=True)
+    return entries
+
+
+def _parse_accept_language(header: str) -> str | None:
+    """Extract the best matching language from an ``Accept-Language`` header.
+
+    Parses quality values and returns the highest-priority match among
+    :data:`SUPPORTED_LANGUAGE_CODES`, or ``None`` if nothing matches.
+    """
+    for _quality, tag in _parse_accept_language_entries(header):
+        code = tag.split("-")[0]
+        if code in SUPPORTED_LANGUAGE_CODES:
+            return code
+
+    return None
+
+
+# Maximum number of languages shown in the compact nav-bar dropdown
+_SUGGESTED_LANGUAGES_MAX = 6
+
+
+def get_suggested_languages(current_locale: str, accept_language_header: str = "") -> list[dict[str, str]]:
+    """Return up to :data:`_SUGGESTED_LANGUAGES_MAX` suggested languages for the compact picker.
+
+    Selection priority:
+        1. The currently active language (always included first).
+        2. Languages listed in the browser's ``Accept-Language`` header.
+        3. Popular global languages (by estimated speaker count) as fillers.
+
+    The resulting list is de-duplicated and capped at
+    :data:`_SUGGESTED_LANGUAGES_MAX` entries.
+    """
+    candidates: list[str] = []
+
+    # 1. Active locale first
+    if current_locale in SUPPORTED_LANGUAGE_CODES:
+        candidates.append(current_locale)
+
+    # 2. Browser preferences
+    for _quality, tag in _parse_accept_language_entries(accept_language_header):
+        if len(candidates) >= _SUGGESTED_LANGUAGES_MAX:
+            break
+        code = tag.split("-")[0]
+        if code in SUPPORTED_LANGUAGE_CODES and code not in candidates:
+            candidates.append(code)
+
+    # 3. Popular language fillers
+    for code in _POPULAR_LANGUAGE_CODES:
+        if len(candidates) >= _SUGGESTED_LANGUAGES_MAX:
+            break
+        if code not in candidates and code in SUPPORTED_LANGUAGE_CODES:
+            candidates.append(code)
+
+    return [_LANG_CODE_MAP[c] for c in candidates[:_SUGGESTED_LANGUAGES_MAX] if c in _LANG_CODE_MAP]
+
+
+# ---------------------------------------------------------------------------
+# Localization helpers (l10n)
+# ---------------------------------------------------------------------------
+
+# Locale-specific formatting rules for date/number display
+_LOCALE_FORMATS: dict[str, dict[str, Any]] = {
+    "en": {
+        "date": "%B %d, %Y",
+        "date_short": "%m/%d/%Y",
+        "datetime": "%B %d, %Y %I:%M %p",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "de": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "fr": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u202f",
+        "decimal_sep": ",",
+    },
+    "es": {
+        "date": "%d de %B de %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d de %B de %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "it": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "pt": {
+        "date": "%d de %B de %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d de %B de %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "nl": {
+        "date": "%d %B %Y",
+        "date_short": "%d-%m-%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "nb": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "da": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "sv": {
+        "date": "%d %B %Y",
+        "date_short": "%Y-%m-%d",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "fi": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "is": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "ga": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "lb": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "ca": {
+        "date": "%d de %B de %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d de %B de %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "pl": {
+        "date": "%d %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "cs": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "sk": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "hu": {
+        "date": "%Y. %B %d.",
+        "date_short": "%Y.%m.%d.",
+        "datetime": "%Y. %B %d. %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "sl": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "hr": {
+        "date": "%d. %B %Y.",
+        "date_short": "%d.%m.%Y.",
+        "datetime": "%d. %B %Y. %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "ro": {
+        "date": "%d %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "bg": {
+        "date": "%d %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "el": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "et": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "lv": {
+        "date": "%Y. gada %d. %B",
+        "date_short": "%d.%m.%Y.",
+        "datetime": "%Y. gada %d. %B %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "lt": {
+        "date": "%Y m. %B %d d.",
+        "date_short": "%Y-%m-%d",
+        "datetime": "%Y m. %B %d d. %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "tr": {
+        "date": "%d %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "uk": {
+        "date": "%d %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "zh": {
+        "date": "%Y年%m月%d日",
+        "date_short": "%Y/%m/%d",
+        "datetime": "%Y年%m月%d日 %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "ru": {
+        "date": "%d %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    # --- New languages ---
+    "no": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "cy": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "fy": {
+        "date": "%d %B %Y",
+        "date_short": "%d-%m-%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "gl": {
+        "date": "%d de %B de %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d de %B de %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "li": {
+        "date": "%d %B %Y",
+        "date_short": "%d-%m-%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "vls": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "nds": {
+        "date": "%d. %B %Y",
+        "date_short": "%d.%m.%Y",
+        "datetime": "%d. %B %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "sr": {
+        "date": "%d. %B %Y.",
+        "date_short": "%d.%m.%Y.",
+        "datetime": "%d. %B %Y. %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "he": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "ar": {
+        "date": "%d %B %Y",
+        "date_short": "%Y/%m/%d",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "fa": {
+        "date": "%d %B %Y",
+        "date_short": "%Y/%m/%d",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "af": {
+        "date": "%d %B %Y",
+        "date_short": "%Y/%m/%d",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+    "ja": {
+        "date": "%Y年%m月%d日",
+        "date_short": "%Y/%m/%d",
+        "datetime": "%Y年%m月%d日 %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "ko": {
+        "date": "%Y년 %m월 %d일",
+        "date_short": "%Y.%m.%d",
+        "datetime": "%Y년 %m월 %d일 %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "vi": {
+        "date": "ngày %d tháng %m năm %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "ngày %d tháng %m năm %Y %H:%M",
+        "thousands_sep": ".",
+        "decimal_sep": ",",
+    },
+    "pa": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "kn": {
+        "date": "%d %B %Y",
+        "date_short": "%d/%m/%Y",
+        "datetime": "%d %B %Y %H:%M",
+        "thousands_sep": ",",
+        "decimal_sep": ".",
+    },
+    "eo": {
+        "date": "%d-a de %B %Y",
+        "date_short": "%Y-%m-%d",
+        "datetime": "%d-a de %B %Y %H:%M",
+        "thousands_sep": "\u00a0",
+        "decimal_sep": ",",
+    },
+}
+
+
+def format_date(value: date | datetime | None, locale: str = DEFAULT_LANGUAGE, short: bool = False) -> str:
+    """Format a date/datetime value according to the locale conventions."""
+    if value is None:
+        return ""
+    fmt_key = "date_short" if short else "date"
+    fmt = _LOCALE_FORMATS.get(locale, _LOCALE_FORMATS[DEFAULT_LANGUAGE])[fmt_key]
+    return value.strftime(fmt)
+
+
+def format_datetime(value: datetime | None, locale: str = DEFAULT_LANGUAGE) -> str:
+    """Format a datetime value according to the locale conventions."""
+    if value is None:
+        return ""
+    fmt = _LOCALE_FORMATS.get(locale, _LOCALE_FORMATS[DEFAULT_LANGUAGE])["datetime"]
+    return value.strftime(fmt)
+
+
+def format_number(value: int | float, locale: str = DEFAULT_LANGUAGE) -> str:
+    """Format a number with locale-appropriate thousand separators."""
+    lf = _LOCALE_FORMATS.get(locale, _LOCALE_FORMATS[DEFAULT_LANGUAGE])
+    if isinstance(value, float):
+        int_part, _, dec_part = f"{value:,.2f}".partition(".")
+        formatted_int = int_part.replace(",", lf["thousands_sep"])
+        return f"{formatted_int}{lf['decimal_sep']}{dec_part}"
+    return f"{value:,}".replace(",", lf["thousands_sep"])
+
+
+@lru_cache(maxsize=32)
+def get_language_info(code: str) -> dict[str, str] | None:
+    """Return the metadata dict for a supported language code, or ``None``."""
+    for lang in SUPPORTED_LANGUAGES:
+        if lang["code"] == code:
+            return lang
+    return None

app/utils/local_auth.py

@@ -32,8 +32,12 @@ def hash_password(plain: str) -> str:
 def verify_password(plain: str, hashed: str) -> bool:
     """Return True when *plain* matches the stored bcrypt *hashed* string."""
     try:
-        return bcrypt.checkpw(plain.encode("utf-8"), hashed.encode("utf-8"))
-    except Exception:
+        result = bcrypt.checkpw(plain.encode("utf-8"), hashed.encode("utf-8"))
+        if not result:
+            logger.debug("verify_password: mismatch password_provided=%s", bool(plain))
+        return result
+    except Exception as exc:
+        logger.warning("verify_password: exception type=%s msg=%s", type(exc).__name__, exc)
         return False
 
 

app/utils/network.py

@@ -0,0 +1,34 @@
+import ipaddress
+import logging
+import socket
+
+logger = logging.getLogger(__name__)
+
+
+def is_private_ip(hostname: str) -> bool:
+    """
+    Check if a hostname resolves to a private/internal IP address.
+    Protects against SSRF attacks by blocking access to internal networks.
+    """
+    try:
+        # Try to parse as IP address directly
+        ip = ipaddress.ip_address(hostname)
+        return ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved
+    except ValueError:
+        # Not a direct IP, try to resolve hostname
+        try:
+            # Get all IP addresses for this hostname
+            addr_info = socket.getaddrinfo(hostname, None)
+            for info in addr_info:
+                ip_str = info[4][0]
+                ip = ipaddress.ip_address(ip_str)
+                # Block if ANY resolved IP is private/internal
+                if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
+                    return True
+            return False
+        except (socket.gaierror, socket.error):
+            # Cannot resolve - allow for testing/development
+            # In production, DNS should work properly
+            # Log this for debugging
+            logger.warning(f"Could not resolve hostname: {hostname}")
+            return False  # Changed from True to False to allow external domains in tests

app/utils/ocr_provider.py

@@ -192,6 +192,95 @@ class OCRResult:
         )
 
 
+# ---------------------------------------------------------------------------
+# Multi-language support
+# ---------------------------------------------------------------------------
+
+#: Canonical list of supported OCR languages for pipeline configuration.
+#: Keys are display names; values are Tesseract language code(s).
+#: Tesseract codes are used as the canonical format because they are the most
+#: widely applicable across self-hosted providers (Tesseract + ocrmypdf).
+#: "auto" falls back to the global ``tesseract_language`` / ``easyocr_languages``
+#: settings (i.e. no per-call override).
+OCR_LANGUAGES: Dict[str, str] = {
+    "Auto (use system default)": "auto",
+    "Arabic": "ara",
+    "Chinese (Simplified)": "chi_sim",
+    "Chinese (Traditional)": "chi_tra",
+    "Czech": "ces",
+    "Danish": "dan",
+    "Dutch": "nld",
+    "English": "eng",
+    "Finnish": "fin",
+    "French": "fra",
+    "German": "deu",
+    "Greek": "ell",
+    "Hebrew": "heb",
+    "Hindi": "hin",
+    "Hungarian": "hun",
+    "Italian": "ita",
+    "Japanese": "jpn",
+    "Korean": "kor",
+    "Norwegian": "nor",
+    "Polish": "pol",
+    "Portuguese": "por",
+    "Romanian": "ron",
+    "Russian": "rus",
+    "Spanish": "spa",
+    "Swedish": "swe",
+    "Thai": "tha",
+    "Turkish": "tur",
+    "Ukrainian": "ukr",
+    "Vietnamese": "vie",
+}
+
+#: Mapping from Tesseract language codes to EasyOCR language codes.
+#: Used when ``TesseractOCRProvider``-style codes are specified but EasyOCR is
+#: the active provider.  Codes not present in this map are passed through as-is
+#: (EasyOCR accepts its own ISO 639-1 codes such as ``"en"`` or ``"de"``).
+TESSERACT_TO_EASYOCR: Dict[str, str] = {
+    "ara": "ar",
+    "ces": "cs",
+    "chi_sim": "ch_sim",
+    "chi_tra": "ch_tra",
+    "dan": "da",
+    "deu": "de",
+    "ell": "el",
+    "eng": "en",
+    "fin": "fi",
+    "fra": "fr",
+    "heb": "he",
+    "hin": "hi",
+    "hun": "hu",
+    "ita": "it",
+    "jpn": "ja",
+    "kor": "ko",
+    "nld": "nl",
+    "nor": "no",
+    "pol": "pl",
+    "por": "pt",
+    "ron": "ro",
+    "rus": "ru",
+    "spa": "es",
+    "swe": "sv",
+    "tha": "th",
+    "tur": "tr",
+    "ukr": "uk",
+    "vie": "vi",
+}
+
+
+def _tesseract_codes_to_easyocr(tesseract_lang: str) -> List[str]:
+    """Convert a Tesseract language string (e.g. ``"eng+deu"``) to a list of
+    EasyOCR language codes (e.g. ``["en", "de"]``).
+
+    Unknown codes are passed through unchanged, so native EasyOCR codes such
+    as ``"en"`` also work transparently.
+    """
+    codes = [part.strip() for part in tesseract_lang.split("+") if part.strip()]
+    return [TESSERACT_TO_EASYOCR.get(code, code) for code in codes]
+
+
 class OCRProvider(ABC):
     """Abstract base class for OCR providers.
 
@@ -290,10 +379,24 @@ class TesseractOCRProvider(OCRProvider):
     - ``tesseract_cmd`` – path to the ``tesseract`` binary (optional).
     - ``tesseract_language`` – Tesseract language code(s), e.g. ``"eng"`` or
       ``"eng+deu"`` (default: ``"eng"``).
+
+    The optional *language* constructor argument overrides the global
+    ``tesseract_language`` setting for this specific provider instance, enabling
+    per-pipeline language configuration.
     """
 
     name = "tesseract"
 
+    def __init__(self, language: Optional[str] = None) -> None:
+        """Initialise the Tesseract provider.
+
+        Args:
+            language: Optional Tesseract language code(s) to use instead of the
+                global ``tesseract_language`` setting (e.g. ``"eng+deu"``).
+                Pass ``None`` or ``"auto"`` to use the global setting.
+        """
+        self._language_override: Optional[str] = language if language and language != "auto" else None
+
     def process(self, file_path: str) -> OCRResult:
         try:
             import pytesseract
@@ -308,7 +411,7 @@ class TesseractOCRProvider(OCRProvider):
         if tesseract_cmd:
             pytesseract.pytesseract.tesseract_cmd = tesseract_cmd
 
-        lang = getattr(settings, "tesseract_language", None) or "eng"
+        lang = self._language_override or getattr(settings, "tesseract_language", None) or "eng"
 
         # Ensure language data files are present; attempt download if missing.
         from app.utils.ocr_language_manager import ensure_tesseract_languages  # noqa: PLC0415
@@ -349,10 +452,26 @@ class EasyOCRProvider(OCRProvider):
     - ``easyocr_languages`` – comma-separated list of language codes
       (default: ``"en"``).
     - ``easyocr_gpu`` – whether to use GPU acceleration (default: ``False``).
+
+    The optional *language* constructor argument accepts a Tesseract-style
+    language string (e.g. ``"eng+deu"``) which is automatically translated to
+    EasyOCR codes (e.g. ``["en", "de"]``), overriding the global
+    ``easyocr_languages`` setting for this provider instance.
     """
 
     name = "easyocr"
 
+    def __init__(self, language: Optional[str] = None) -> None:
+        """Initialise the EasyOCR provider.
+
+        Args:
+            language: Optional Tesseract-style language code(s) (e.g. ``"eng+deu"``)
+                or a comma-separated EasyOCR language list (e.g. ``"en,de"``).
+                Pass ``None`` or ``"auto"`` to use the global ``easyocr_languages``
+                setting.
+        """
+        self._language_override: Optional[str] = language if language and language != "auto" else None
+
     def process(self, file_path: str) -> OCRResult:
         try:
             import easyocr
@@ -363,8 +482,12 @@ class EasyOCRProvider(OCRProvider):
                 "Install them with: pip install easyocr pdf2image"
             ) from exc
 
-        lang_str = getattr(settings, "easyocr_languages", None) or "en"
-        langs = [lang.strip() for lang in lang_str.split(",") if lang.strip()]
+        if self._language_override:
+            # Convert Tesseract-style codes to EasyOCR codes
+            langs = _tesseract_codes_to_easyocr(self._language_override)
+        else:
+            lang_str = getattr(settings, "easyocr_languages", None) or "en"
+            langs = [lang.strip() for lang in lang_str.split(",") if lang.strip()]
         gpu = getattr(settings, "easyocr_gpu", False)
 
         logger.info(f"[EasyOCR] Processing {os.path.basename(file_path)} (langs={langs}, gpu={gpu})")
@@ -679,23 +802,36 @@ KNOWN_OCR_PROVIDERS: List[str] = sorted(_PROVIDER_MAP.keys())
 MAX_OCR_TEXT_FOR_AI_MERGE = 4000
 
 
-def get_ocr_providers() -> List[OCRProvider]:
+def get_ocr_providers(language: Optional[str] = None) -> List[OCRProvider]:
     """Return a list of configured OCR provider instances.
 
     Reads ``settings.ocr_providers`` (comma-separated provider names) and
     returns one instantiated provider per entry.  Falls back to ``["azure"]``
     when the setting is absent.
+
+    Args:
+        language: Optional Tesseract-style language code(s) (e.g. ``"eng+deu"``)
+            to override the global language settings for providers that support
+            per-call language configuration (Tesseract and EasyOCR).  Pass
+            ``None`` or ``"auto"`` to use the global settings.
     """
     raw = getattr(settings, "ocr_providers", None) or "azure"
     provider_names = [name.strip().lower() for name in raw.split(",") if name.strip()]
 
+    # Normalise "auto" to None so providers fall back to global settings
+    effective_language = language if language and language != "auto" else None
+
     providers: List[OCRProvider] = []
     for name in provider_names:
         cls = _PROVIDER_MAP.get(name)
         if cls is None:
             logger.warning(f"Unknown OCR provider '{name}' in OCR_PROVIDERS – skipping.")
             continue
-        providers.append(cls())
+        # Pass language override to providers that support per-call language config
+        if effective_language is not None and name in ("tesseract", "easyocr"):
+            providers.append(cls(language=effective_language))
+        else:
+            providers.append(cls())
         logger.debug(f"Registered OCR provider: {name}")
 
     if not providers:

app/utils/push_notification.py

@@ -0,0 +1,130 @@
+"""Push notification sender for the DocuElevate mobile app.
+
+Uses the **Expo Push Notification** service to deliver notifications to both
+iOS (via APNs) and Android (via FCM) without requiring server-side APNs keys
+or FCM credentials.  The mobile app obtains an ``ExponentPushToken[…]`` at
+startup and registers it with the backend via the mobile API.
+
+Reference: https://docs.expo.dev/push-notifications/sending-notifications/
+"""
+
+import logging
+from typing import Any
+
+import httpx
+
+from app.database import SessionLocal
+from app.models import MobileDevice
+
+logger = logging.getLogger(__name__)
+
+EXPO_PUSH_URL = "https://exp.host/--/api/v2/push/send"
+
+# Maximum tokens per batch request (Expo limit).
+_EXPO_BATCH_LIMIT = 100
+
+
+def send_expo_push_notification(
+    tokens: list[str],
+    title: str,
+    body: str,
+    data: dict[str, Any] | None = None,
+    sound: str = "default",
+    badge: int | None = None,
+) -> list[dict[str, Any]]:
+    """Send a push notification to one or more Expo push tokens.
+
+    Args:
+        tokens: List of Expo push tokens (``ExponentPushToken[…]``).
+        title: Notification title shown in the system tray.
+        body: Notification body text.
+        data: Optional JSON-serialisable dict attached to the notification
+              (available in the app via ``notification.request.content.data``).
+        sound: Notification sound.  Use ``"default"`` or ``None`` for silent.
+        badge: iOS badge count.  Pass ``0`` to clear.
+
+    Returns:
+        List of Expo push receipt dicts (one per token).
+    """
+    if not tokens:
+        return []
+
+    results: list[dict[str, Any]] = []
+
+    # Send in batches to stay within Expo's per-request limit.
+    for i in range(0, len(tokens), _EXPO_BATCH_LIMIT):
+        batch = tokens[i : i + _EXPO_BATCH_LIMIT]
+        messages = []
+        for token in batch:
+            msg: dict[str, Any] = {
+                "to": token,
+                "title": title,
+                "body": body,
+                "sound": sound,
+            }
+            if data:
+                msg["data"] = data
+            if badge is not None:
+                msg["badge"] = badge
+            messages.append(msg)
+
+        try:
+            resp = httpx.post(
+                EXPO_PUSH_URL,
+                json=messages,
+                headers={
+                    "Accept": "application/json",
+                    "Accept-Encoding": "gzip, deflate",
+                    "Content-Type": "application/json",
+                },
+                timeout=15,
+            )
+            resp.raise_for_status()
+            payload = resp.json()
+            batch_results = payload.get("data", [])
+            results.extend(batch_results)
+            logger.debug("Expo push batch sent: %d tokens, %d results", len(batch), len(batch_results))
+        except httpx.HTTPStatusError as exc:
+            logger.error("Expo push HTTP error: %s – %s", exc.response.status_code, exc.response.text)
+        except Exception:
+            logger.exception("Expo push notification failed for batch starting at index %d", i)
+
+    return results
+
+
+def send_push_to_owner(
+    owner_id: str,
+    title: str,
+    body: str,
+    data: dict[str, Any] | None = None,
+) -> None:
+    """Look up all active push tokens for *owner_id* and send them a notification.
+
+    This function is safe to call from Celery task workers.  Database errors
+    and push failures are logged but never raised so that the caller task is
+    not retried due to a notification failure.
+    """
+    db = SessionLocal()
+    try:
+        devices = (
+            db.query(MobileDevice)
+            .filter(
+                MobileDevice.owner_id == owner_id,
+                MobileDevice.is_active.is_(True),
+                MobileDevice.push_token.isnot(None),
+            )
+            .all()
+        )
+        tokens = [d.push_token for d in devices if d.push_token]
+    except Exception:
+        logger.exception("Failed to query mobile devices for owner_id=%s", owner_id)
+        return
+    finally:
+        db.close()
+
+    if not tokens:
+        logger.debug("No active push tokens for owner_id=%s", owner_id)
+        return
+
+    logger.info("Sending push notification to %d device(s) for owner_id=%s", len(tokens), owner_id)
+    send_expo_push_notification(tokens=tokens, title=title, body=body, data=data)

app/utils/routing_engine.py

@@ -0,0 +1,223 @@
+"""Routing engine for conditional document-to-pipeline assignment.
+
+Evaluates a set of :class:`PipelineRoutingRule` rows against document
+properties and returns the first matching target pipeline (if any).
+
+Supported document fields
+-------------------------
+* ``file_type`` – MIME type of the file (e.g. ``application/pdf``)
+* ``filename``  – original filename
+* ``size``      – file size in bytes (numeric comparison)
+* ``document_type`` – AI-classified document type (e.g. ``Invoice``)
+* ``category``  – alias for ``document_type``
+* ``metadata.<key>`` – arbitrary key inside the AI-extracted JSON metadata
+
+Supported comparison operators
+------------------------------
+* ``equals`` / ``not_equals``
+* ``contains`` / ``not_contains`` (substring match, case-insensitive)
+* ``regex`` (Python ``re`` full-match, case-insensitive)
+* ``gt`` / ``lt`` / ``gte`` / ``lte`` (numeric comparison)
+"""
+
+import json
+import logging
+import re
+from typing import Any
+
+from sqlalchemy.orm import Session
+
+from app.models import Pipeline, PipelineRoutingRule
+
+logger = logging.getLogger(__name__)
+
+# Operators recognised by the engine.
+VALID_OPERATORS: frozenset[str] = frozenset(
+    {
+        "equals",
+        "not_equals",
+        "contains",
+        "not_contains",
+        "regex",
+        "gt",
+        "lt",
+        "gte",
+        "lte",
+    }
+)
+
+# Fields that are resolved directly from the FileRecord.
+BUILTIN_FIELDS: frozenset[str] = frozenset(
+    {
+        "file_type",
+        "filename",
+        "size",
+        "document_type",
+        "category",
+    }
+)
+
+
+def _resolve_field(field: str, doc_props: dict[str, Any]) -> Any:
+    """Resolve a *field* name to its actual value from *doc_props*.
+
+    ``doc_props`` is expected to contain top-level keys that mirror the
+    built-in field names **plus** a ``metadata`` dict with the parsed
+    AI metadata JSON.
+    """
+    if field == "category":
+        # ``category`` is an alias for ``document_type``.
+        field = "document_type"
+
+    if field.startswith("metadata."):
+        meta_key = field[len("metadata.") :]
+        metadata = doc_props.get("metadata") or {}
+        return metadata.get(meta_key)
+
+    return doc_props.get(field)
+
+
+def _to_float(value: Any) -> float | None:
+    """Try to convert *value* to a float for numeric comparison."""
+    if value is None:
+        return None
+    try:
+        return float(value)
+    except (TypeError, ValueError):
+        return None
+
+
+def _evaluate_condition(actual: Any, operator: str, expected: str) -> bool:
+    """Return ``True`` when *actual* satisfies *operator* against *expected*.
+
+    All string comparisons are case-insensitive.  Numeric operators (``gt``,
+    ``lt``, ``gte``, ``lte``) attempt to cast both sides to ``float``.
+    """
+    if actual is None:
+        # If the document property is missing, the rule cannot match
+        # (except for ``not_equals`` / ``not_contains`` which should match).
+        if operator == "not_equals":
+            return True
+        if operator == "not_contains":
+            return True
+        return False
+
+    actual_str = str(actual).lower()
+    expected_lower = expected.lower()
+
+    if operator == "equals":
+        return actual_str == expected_lower
+    if operator == "not_equals":
+        return actual_str != expected_lower
+    if operator == "contains":
+        return expected_lower in actual_str
+    if operator == "not_contains":
+        return expected_lower not in actual_str
+    if operator == "regex":
+        try:
+            return bool(re.fullmatch(expected, str(actual), flags=re.IGNORECASE))
+        except re.error:
+            logger.warning("Invalid regex in routing rule: %s", expected)
+            return False
+
+    # Numeric operators
+    actual_num = _to_float(actual)
+    expected_num = _to_float(expected)
+    if actual_num is None or expected_num is None:
+        return False
+
+    if operator == "gt":
+        return actual_num > expected_num
+    if operator == "lt":
+        return actual_num < expected_num
+    if operator == "gte":
+        return actual_num >= expected_num
+    if operator == "lte":
+        return actual_num <= expected_num
+
+    return False
+
+
+def build_document_properties(file_record: Any) -> dict[str, Any]:
+    """Build the property dict that the engine evaluates against.
+
+    Args:
+        file_record: A :class:`FileRecord` instance (or any object with the
+            same attributes).
+
+    Returns:
+        A dict with ``file_type``, ``filename``, ``size``, ``document_type``,
+        and ``metadata`` keys.
+    """
+    metadata: dict[str, Any] = {}
+    raw_meta = getattr(file_record, "ai_metadata", None)
+    if raw_meta:
+        try:
+            metadata = json.loads(raw_meta) if isinstance(raw_meta, str) else raw_meta
+        except (json.JSONDecodeError, TypeError):
+            metadata = {}
+
+    return {
+        "file_type": getattr(file_record, "mime_type", None),
+        "filename": getattr(file_record, "original_filename", None),
+        "size": getattr(file_record, "file_size", None),
+        "document_type": metadata.get("document_type"),
+        "metadata": metadata,
+    }
+
+
+def evaluate_routing_rules(
+    db: Session,
+    owner_id: str | None,
+    doc_props: dict[str, Any],
+) -> Pipeline | None:
+    """Evaluate routing rules and return the first matching pipeline.
+
+    Rules are fetched for the given *owner_id* **plus** any system-wide rules
+    (``owner_id IS NULL``).  Owner rules are evaluated first (by position),
+    then system rules.
+
+    Args:
+        db: Active database session.
+        owner_id: The document owner's identifier (may be ``None``).
+        doc_props: Document property dict as produced by
+            :func:`build_document_properties`.
+
+    Returns:
+        The first matching :class:`Pipeline`, or ``None`` when no rule
+        matches (caller should fall back to the default pipeline).
+    """
+    # Fetch active rules for the owner + system rules, ordered by position.
+    rules = (
+        db.query(PipelineRoutingRule)
+        .filter(
+            PipelineRoutingRule.is_active.is_(True),
+            (PipelineRoutingRule.owner_id == owner_id) | (PipelineRoutingRule.owner_id.is_(None)),
+        )
+        .order_by(
+            # Owner-specific rules take priority over system rules.
+            PipelineRoutingRule.owner_id.is_(None).asc(),
+            PipelineRoutingRule.position.asc(),
+        )
+        .all()
+    )
+
+    for rule in rules:
+        actual = _resolve_field(rule.field, doc_props)
+        if _evaluate_condition(actual, rule.operator, rule.value):
+            pipeline = db.query(Pipeline).filter(Pipeline.id == rule.target_pipeline_id).first()
+            if pipeline and pipeline.is_active:
+                logger.info(
+                    "Routing rule matched: rule_id=%s, name=%s, target_pipeline=%s",
+                    rule.id,
+                    rule.name,
+                    rule.target_pipeline_id,
+                )
+                return pipeline
+            logger.warning(
+                "Routing rule %s matched but target pipeline %s is inactive or missing",
+                rule.id,
+                rule.target_pipeline_id,
+            )
+
+    return None

app/utils/sentry.py

@@ -0,0 +1,113 @@
+"""
+Sentry integration utilities for DocuElevate.
+
+Call ``init_sentry()`` early in your application entry point (before any
+request handling) to enable error tracking and performance monitoring.  The
+function is a no-op when ``SENTRY_DSN`` is not configured, so it is safe to
+call unconditionally in all environments.
+
+Example (FastAPI)::
+
+    from app.utils.sentry import init_sentry
+    init_sentry()
+
+Example (Celery worker)::
+
+    from app.utils.sentry import init_sentry
+    init_sentry(integrations_extra=["celery"])
+"""
+
+from __future__ import annotations
+
+import logging
+
+from app.config import settings
+
+logger = logging.getLogger(__name__)
+
+
+def init_sentry(*, integrations_extra: list[str] | None = None) -> bool:
+    """
+    Initialise the Sentry SDK if ``SENTRY_DSN`` is configured.
+
+    Args:
+        integrations_extra: Optional list of additional integration names to
+            activate.  Currently recognised values: ``"celery"``.  The
+            ``FastApiIntegration``, ``SqlalchemyIntegration``, and
+            ``LoggingIntegration`` are always included when the SDK is
+            initialised.
+
+    Returns:
+        ``True`` when Sentry was successfully initialised, ``False`` otherwise
+        (e.g. DSN not configured or SDK not installed).
+    """
+    dsn = settings.sentry_dsn
+    if not dsn:
+        logger.debug("Sentry DSN not configured – error monitoring disabled")
+        return False
+
+    try:
+        import sentry_sdk
+        from sentry_sdk.integrations.fastapi import FastApiIntegration
+        from sentry_sdk.integrations.logging import LoggingIntegration
+        from sentry_sdk.integrations.sqlalchemy import SqlalchemyIntegration
+        from sentry_sdk.integrations.starlette import StarletteIntegration
+    except ImportError:
+        logger.warning(
+            "sentry-sdk is not installed.  Install it with: pip install 'sentry-sdk[fastapi,celery,sqlalchemy]'"
+        )
+        return False
+
+    integrations = [
+        StarletteIntegration(transaction_style="url"),
+        FastApiIntegration(transaction_style="url"),
+        SqlalchemyIntegration(),
+        LoggingIntegration(
+            level=logging.INFO,  # Breadcrumbs from INFO+
+            event_level=logging.ERROR,  # Send Sentry events for ERROR+
+        ),
+    ]
+
+    if integrations_extra and "celery" in integrations_extra:
+        try:
+            from sentry_sdk.integrations.celery import CeleryIntegration
+
+            integrations.append(CeleryIntegration(monitor_beat_tasks=True))
+        except ImportError:
+            logger.warning("CeleryIntegration not available – skipping")
+
+    # Clamp sample rates to [0.0, 1.0]
+    traces_rate = max(0.0, min(1.0, settings.sentry_traces_sample_rate))
+    profiles_rate = max(0.0, min(1.0, settings.sentry_profiles_sample_rate))
+
+    version = _get_app_version()
+
+    sentry_sdk.init(
+        dsn=dsn,
+        environment=settings.sentry_environment,
+        release=version,
+        integrations=integrations,
+        traces_sample_rate=traces_rate,
+        profiles_sample_rate=profiles_rate,
+        send_default_pii=settings.sentry_send_default_pii,
+        # Attach a request body snapshot to every event (helps debugging)
+        max_request_body_size="medium",
+        # Keep the SDK from attaching local variable values to stack frames
+        # by default; enable explicitly if needed for deeper debugging.
+        attach_stacktrace=True,
+    )
+
+    logger.info(
+        "Sentry initialised (environment=%s, traces_sample_rate=%s)",
+        settings.sentry_environment,
+        traces_rate,
+    )
+    return True
+
+
+def _get_app_version() -> str | None:
+    """Return the application version string for Sentry release tracking."""
+    try:
+        return settings.version or None
+    except AttributeError:  # pragma: no cover
+        return None

app/utils/settings_service.py

@@ -182,6 +182,153 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    # Social Login Providers
+    "social_auth_google_enabled": {
+        "category": "Social Login",
+        "description": (
+            "Enable Google Sign-In. Requires SOCIAL_AUTH_GOOGLE_CLIENT_ID and "
+            "SOCIAL_AUTH_GOOGLE_CLIENT_SECRET from the Google Cloud Console."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+        "help_link": "https://console.cloud.google.com/apis/credentials",
+        "help_link_label": "Google Cloud Console",
+    },
+    "social_auth_google_client_id": {
+        "category": "Social Login",
+        "description": "Google OAuth2 client ID from the Google Cloud Console.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_google_client_secret": {
+        "category": "Social Login",
+        "description": "Google OAuth2 client secret from the Google Cloud Console.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_microsoft_enabled": {
+        "category": "Social Login",
+        "description": (
+            "Enable Microsoft Sign-In (Azure AD / Microsoft Entra ID). Requires "
+            "SOCIAL_AUTH_MICROSOFT_CLIENT_ID and SOCIAL_AUTH_MICROSOFT_CLIENT_SECRET "
+            "from Azure App Registrations."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+        "help_link": "https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade",
+        "help_link_label": "Azure Portal",
+    },
+    "social_auth_microsoft_client_id": {
+        "category": "Social Login",
+        "description": "Microsoft OAuth2 application (client) ID from Azure App Registrations.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_microsoft_client_secret": {
+        "category": "Social Login",
+        "description": "Microsoft OAuth2 client secret from Azure App Registrations.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_microsoft_tenant": {
+        "category": "Social Login",
+        "description": (
+            "Azure AD tenant ID or one of 'common', 'organizations', 'consumers'. "
+            "Use 'common' to allow any Microsoft account. Use a specific GUID to "
+            "restrict to a single organization."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_apple_enabled": {
+        "category": "Social Login",
+        "description": (
+            "Enable Sign in with Apple. Requires an Apple Developer account with "
+            "a Services ID configured for Sign in with Apple."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+        "help_link": "https://developer.apple.com/account/resources/identifiers/list/serviceId",
+        "help_link_label": "Apple Developer Portal",
+    },
+    "social_auth_apple_client_id": {
+        "category": "Social Login",
+        "description": "Apple Services ID (e.g. com.example.docuelevate).",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_apple_team_id": {
+        "category": "Social Login",
+        "description": "Apple Developer Team ID (10-character alphanumeric string).",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_apple_key_id": {
+        "category": "Social Login",
+        "description": "Apple Sign-In private key ID from the Apple Developer Portal.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_apple_private_key": {
+        "category": "Social Login",
+        "description": (
+            "Apple Sign-In private key (PEM format). Generate this in the Apple Developer Portal. "
+            "Paste the entire key content including BEGIN/END headers."
+        ),
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_dropbox_enabled": {
+        "category": "Social Login",
+        "description": (
+            "Enable Dropbox Sign-In. Uses the same Dropbox App you may already have "
+            "configured for storage, or a separate one."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_dropbox_client_id": {
+        "category": "Social Login",
+        "description": "Dropbox OAuth2 App Key from the Dropbox App Console.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "social_auth_dropbox_client_secret": {
+        "category": "Social Login",
+        "description": "Dropbox OAuth2 App Secret from the Dropbox App Console.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
     # AI Services
     "openai_api_key": {
         "category": "AI Services",
@@ -495,6 +642,14 @@ SETTING_METADATA = {
         "options": ["us", "eu"],
     },
     # Storage Providers - Dropbox
+    "dropbox_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable Dropbox as an upload destination. When disabled, no documents will be sent to Dropbox even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "dropbox_app_key": {
         "category": "Storage Providers",
         "description": "Dropbox app key for OAuth authentication",
@@ -528,6 +683,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - Nextcloud
+    "nextcloud_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable Nextcloud as an upload destination. When disabled, no documents will be sent to Nextcloud even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "nextcloud_upload_url": {
         "category": "Storage Providers",
         "description": "Nextcloud WebDAV upload URL",
@@ -561,6 +724,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - Paperless-ngx
+    "paperless_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable Paperless-ngx as an upload destination. When disabled, no documents will be sent to Paperless-ngx even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "paperless_ngx_api_token": {
         "category": "Storage Providers",
         "description": "Paperless-ngx API authentication token",
@@ -578,6 +749,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - Google Drive
+    "google_drive_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable Google Drive as an upload destination. When disabled, no documents will be sent to Google Drive even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "google_drive_credentials_json": {
         "category": "Storage Providers",
         "description": "Google Drive service account credentials JSON",
@@ -635,6 +814,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - OneDrive
+    "onedrive_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable OneDrive as an upload destination. When disabled, no documents will be sent to OneDrive even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "onedrive_client_id": {
         "category": "Storage Providers",
         "description": "OneDrive OAuth client ID",
@@ -676,6 +863,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - WebDAV
+    "webdav_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable WebDAV as an upload destination. When disabled, no documents will be sent to WebDAV even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "webdav_url": {
         "category": "Storage Providers",
         "description": "WebDAV server URL",
@@ -717,6 +912,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - FTP
+    "ftp_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable FTP as an upload destination. When disabled, no documents will be sent to FTP even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "ftp_host": {
         "category": "Storage Providers",
         "description": "FTP server hostname or IP address",
@@ -774,6 +977,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Storage Providers - SFTP
+    "sftp_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable SFTP as an upload destination. When disabled, no documents will be sent to SFTP even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "sftp_host": {
         "category": "Storage Providers",
         "description": "SFTP server hostname or IP address",
@@ -838,7 +1049,56 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    # Storage Providers - iCloud Drive
+    "icloud_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable iCloud Drive as an upload destination. When disabled, no documents will be sent to iCloud Drive even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "icloud_username": {
+        "category": "Storage Providers",
+        "description": "Apple ID email address for iCloud Drive authentication",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "icloud_password": {
+        "category": "Storage Providers",
+        "description": "App-specific password for iCloud Drive (generate at https://appleid.apple.com)",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": False,
+    },
+    "icloud_folder": {
+        "category": "Storage Providers",
+        "description": "Target folder path in iCloud Drive (e.g. Documents/Uploads)",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "icloud_cookie_directory": {
+        "category": "Storage Providers",
+        "description": "Directory for persisting iCloud session cookies (default: ~/.pyicloud)",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Storage Providers - AWS S3
+    "s3_enabled": {
+        "category": "Storage Providers",
+        "description": "Enable Amazon S3 as an upload destination. When disabled, no documents will be sent to S3 even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "aws_access_key_id": {
         "category": "Storage Providers",
         "description": "AWS access key ID for S3",
@@ -972,6 +1232,14 @@ SETTING_METADATA = {
         "restart_required": False,
     },
     # Email Destination Settings (dedicated SMTP for document delivery)
+    "dest_email_enabled": {
+        "category": "Email Destination",
+        "description": "Enable Email as an upload destination. When disabled, no documents will be delivered via email even if credentials are configured.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     "dest_email_host": {
         "category": "Email Destination",
         "description": "SMTP server hostname for document delivery (separate from shared email settings)",
@@ -1392,6 +1660,18 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    "imap_attachment_filter": {
+        "category": "IMAP",
+        "description": (
+            "Controls which attachment types are ingested from IMAP emails. "
+            "Accepted values: 'documents_only' (PDFs and office files only, default) or 'all' (including images). "
+            "Per-user IMAP accounts can override this global default."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Monitoring - Uptime Kuma
     "uptime_kuma_url": {
         "category": "Monitoring",
@@ -1595,6 +1875,18 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    "compliance_enabled": {
+        "category": "Feature Flags",
+        "description": (
+            "Enable the compliance templates dashboard (GDPR, HIPAA, SOC 2). "
+            "When enabled, admins can view compliance status and apply "
+            "pre-built regulatory configurations. Default: True."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Backup / Restore
     "backup_enabled": {
         "category": "Backup",
@@ -2065,6 +2357,78 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": True,
     },
+    "audit_siem_enabled": {
+        "category": "Security",
+        "description": "Enable forwarding of audit events to an external SIEM system (Syslog, Splunk, Logstash, etc.).",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "audit_siem_transport": {
+        "category": "Security",
+        "description": (
+            "Transport used to forward audit events. 'syslog' sends RFC 5424 messages over UDP/TCP. "
+            "'http' sends JSON POST payloads to a webhook URL (Splunk HEC, Logstash, Grafana Loki, etc.)."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+        "options": ["syslog", "http"],
+    },
+    "audit_siem_syslog_host": {
+        "category": "Security",
+        "description": "Hostname or IP of the syslog receiver.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "audit_siem_syslog_port": {
+        "category": "Security",
+        "description": "Port of the syslog receiver. Default: 514.",
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "audit_siem_syslog_protocol": {
+        "category": "Security",
+        "description": "Protocol for syslog transport: 'udp' or 'tcp'. Default: udp.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+        "options": ["udp", "tcp"],
+    },
+    "audit_siem_http_url": {
+        "category": "Security",
+        "description": (
+            "HTTP endpoint URL for SIEM webhook delivery. Supports Splunk HEC, "
+            "Logstash HTTP input, Grafana Loki push API, or any JSON-accepting endpoint."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "audit_siem_http_token": {
+        "category": "Security",
+        "description": "Bearer / HEC token included in the Authorization header of SIEM HTTP requests.",
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": False,
+    },
+    "audit_siem_http_custom_headers": {
+        "category": "Security",
+        "description": "Comma-separated 'Key:Value' pairs of extra headers for SIEM HTTP requests.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
     # Rate Limiting
     "rate_limiting_enabled": {
         "category": "Security",
@@ -2223,6 +2587,105 @@ SETTING_METADATA = {
         "required": False,
         "restart_required": False,
     },
+    # Support / Help Center – Zammad integration
+    "zammad_url": {
+        "category": "Support",
+        "description": "Base URL of your Zammad instance (e.g. https://zammad.example.com). Required for the chat widget and feedback form.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "zammad_chat_enabled": {
+        "category": "Support",
+        "description": "Show a Zammad live-chat widget on the Help Center page.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "zammad_chat_id": {
+        "category": "Support",
+        "description": "Zammad chat topic ID (see Channels → Chat → Topics in Zammad admin). Default: 1.",
+        "type": "integer",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "zammad_form_enabled": {
+        "category": "Support",
+        "description": "Show a 'Submit a Ticket' feedback form on the Help Center page.",
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    "support_email": {
+        "category": "Support",
+        "description": "Support e-mail address displayed on the Help Center page.",
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": False,
+    },
+    # Observability – Sentry
+    "sentry_dsn": {
+        "category": "Observability",
+        "description": (
+            "Sentry DSN (Data Source Name) URL. When set, runtime errors and "
+            "performance traces are automatically sent to Sentry. "
+            "Leave blank to disable Sentry entirely."
+        ),
+        "type": "string",
+        "sensitive": True,
+        "required": False,
+        "restart_required": True,
+    },
+    "sentry_environment": {
+        "category": "Observability",
+        "description": (
+            "Environment label attached to every Sentry event "
+            "(e.g. 'development', 'staging', 'production'). "
+            "Helps you filter events in the Sentry dashboard. Default: 'production'."
+        ),
+        "type": "string",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "sentry_traces_sample_rate": {
+        "category": "Observability",
+        "description": (
+            "Fraction of transactions captured for Sentry performance monitoring (0.0–1.0). "
+            "0.0 disables tracing; 1.0 captures every request. Default: 0.1 (10%)."
+        ),
+        "type": "float",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "sentry_profiles_sample_rate": {
+        "category": "Observability",
+        "description": (
+            "Fraction of profiled transactions sent to Sentry (0.0–1.0). "
+            "Only active when sentry_traces_sample_rate > 0. Default: 0.0 (disabled)."
+        ),
+        "type": "float",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
+    "sentry_send_default_pii": {
+        "category": "Observability",
+        "description": (
+            "Attach personally identifiable information (PII) such as IP addresses "
+            "to Sentry events. Disabled by default for GDPR/CCPA compliance."
+        ),
+        "type": "boolean",
+        "sensitive": False,
+        "required": False,
+        "restart_required": True,
+    },
 }
 
 

app/utils/user_notification.py

@@ -0,0 +1,246 @@
+"""Per-user notification dispatch service.
+
+Handles user-centric events (document.processed, document.failed) by:
+1. Always creating an InAppNotification record
+2. Sending via configured email/webhook targets (UserNotificationTarget)
+   if the user has enabled that channel/event combination.
+"""
+
+import json
+import logging
+import smtplib
+from email.mime.multipart import MIMEMultipart
+from email.mime.text import MIMEText
+from typing import Any
+
+import httpx
+
+from app.database import SessionLocal
+from app.models import InAppNotification, UserNotificationPreference, UserNotificationTarget
+
+logger = logging.getLogger(__name__)
+
+# Supported user-centric event types
+EVENT_DOCUMENT_PROCESSED = "document.processed"
+EVENT_DOCUMENT_FAILED = "document.failed"
+
+USER_EVENT_LABELS: dict[str, str] = {
+    EVENT_DOCUMENT_PROCESSED: "Document Processed",
+    EVENT_DOCUMENT_FAILED: "Document Processing Failed",
+}
+
+
+def create_in_app_notification(
+    owner_id: str,
+    event_type: str,
+    title: str,
+    message: str,
+    file_id: int | None = None,
+) -> InAppNotification | None:
+    """Persist an InAppNotification record for the given user.
+
+    Returns:
+        The created InAppNotification, or None on error.
+    """
+    db = SessionLocal()
+    try:
+        notif = InAppNotification(
+            owner_id=owner_id,
+            event_type=event_type,
+            title=title,
+            message=message,
+            file_id=file_id,
+        )
+        db.add(notif)
+        db.commit()
+        db.refresh(notif)
+        return notif
+    except Exception:
+        db.rollback()
+        logger.exception("Failed to create in-app notification for owner_id=%s", owner_id)
+        return None
+    finally:
+        db.close()
+
+
+def _send_email_notification(target_config: dict[str, Any], title: str, message: str) -> bool:
+    """Send an email notification via the configured SMTP target.
+
+    Args:
+        target_config: dict with keys: smtp_host, smtp_port, smtp_username,
+                       smtp_password, smtp_use_tls, recipient_email
+        title: Email subject
+        message: Email body text
+
+    Returns:
+        True if the email was sent successfully, False otherwise.
+    """
+    try:
+        smtp_host = target_config.get("smtp_host", "")
+        smtp_port = int(target_config.get("smtp_port", 587))
+        smtp_username = target_config.get("smtp_username", "")
+        smtp_password = target_config.get("smtp_password", "")
+        smtp_use_tls = bool(target_config.get("smtp_use_tls", True))
+        recipient_email = target_config.get("recipient_email", "")
+        sender_email = target_config.get("sender_email") or smtp_username or "noreply@docuelevate.local"
+
+        if not smtp_host or not recipient_email:
+            logger.warning("Email notification target missing smtp_host or recipient_email")
+            return False
+
+        msg = MIMEMultipart("alternative")
+        msg["Subject"] = title
+        msg["From"] = sender_email
+        msg["To"] = recipient_email
+        msg.attach(MIMEText(message, "plain"))
+
+        with smtplib.SMTP(smtp_host, smtp_port, timeout=30) as server:
+            if smtp_use_tls:
+                server.starttls()
+            if smtp_username and smtp_password:
+                server.login(smtp_username, smtp_password)
+            server.send_message(msg)
+
+        logger.info("Email notification sent to %s", recipient_email)
+        return True
+    except Exception:
+        logger.exception("Failed to send email notification")
+        return False
+
+
+def _send_webhook_notification(target_config: dict[str, Any], event_type: str, title: str, message: str) -> bool:
+    """Send a webhook POST notification to the configured URL.
+
+    Args:
+        target_config: dict with keys: url, secret (optional HMAC header value)
+        event_type: The event type string
+        title: Notification title
+        message: Notification body
+
+    Returns:
+        True if the webhook was delivered successfully, False otherwise.
+    """
+    try:
+        url = target_config.get("url", "")
+        secret = target_config.get("secret", "")
+
+        if not url:
+            logger.warning("Webhook notification target missing url")
+            return False
+
+        payload = {
+            "event": event_type,
+            "title": title,
+            "message": message,
+        }
+        headers = {"Content-Type": "application/json"}
+        if secret:
+            headers["X-DocuElevate-Secret"] = secret
+
+        response = httpx.post(url, json=payload, headers=headers, timeout=10)
+        response.raise_for_status()
+        logger.info("Webhook notification sent to %s (status %s)", url, response.status_code)
+        return True
+    except Exception:
+        logger.exception("Failed to send webhook notification to %s", target_config.get("url", ""))
+        return False
+
+
+def dispatch_user_notification(
+    owner_id: str,
+    event_type: str,
+    title: str,
+    message: str,
+    file_id: int | None = None,
+) -> None:
+    """Dispatch a user notification for the given event.
+
+    Always creates an in-app notification. Also sends via email/webhook
+    targets if the user has configured and enabled them for this event.
+
+    Args:
+        owner_id: The user's stable identifier.
+        event_type: e.g. "document.processed" or "document.failed"
+        title: Short notification title.
+        message: Longer notification body.
+        file_id: Optional FileRecord.id to link.
+    """
+    # 1. Always create an in-app notification
+    create_in_app_notification(
+        owner_id=owner_id,
+        event_type=event_type,
+        title=title,
+        message=message,
+        file_id=file_id,
+    )
+
+    # 2. Check for configured email/webhook preferences
+    db = SessionLocal()
+    try:
+        prefs = (
+            db.query(UserNotificationPreference)
+            .filter(
+                UserNotificationPreference.owner_id == owner_id,
+                UserNotificationPreference.event_type == event_type,
+                UserNotificationPreference.is_enabled == True,  # noqa: E712
+                UserNotificationPreference.channel_type.in_(["email", "webhook"]),
+            )
+            .all()
+        )
+
+        for pref in prefs:
+            if not pref.target_id:
+                continue
+            target = db.get(UserNotificationTarget, pref.target_id)
+            if not target or not target.is_active:
+                continue
+            config: dict[str, Any] = {}
+            if target.config:
+                try:
+                    config = json.loads(target.config)
+                except (json.JSONDecodeError, ValueError):
+                    config = {}
+
+            if pref.channel_type == "email":
+                _send_email_notification(config, title, message)
+            elif pref.channel_type == "webhook":
+                _send_webhook_notification(config, event_type, title, message)
+    except Exception:
+        logger.exception("Error dispatching user notification for owner_id=%s event=%s", owner_id, event_type)
+    finally:
+        db.close()
+
+    # 3. Send push notifications to registered mobile devices
+    try:
+        from app.utils.push_notification import send_push_to_owner
+
+        send_push_to_owner(
+            owner_id=owner_id,
+            title=title,
+            body=message,
+            data={"event_type": event_type, "file_id": file_id},
+        )
+    except Exception:
+        logger.exception("Error sending push notification for owner_id=%s event=%s", owner_id, event_type)
+
+
+def notify_user_document_processed(owner_id: str, filename: str, file_id: int | None = None) -> None:
+    """Notify a user that their document was successfully processed."""
+    dispatch_user_notification(
+        owner_id=owner_id,
+        event_type=EVENT_DOCUMENT_PROCESSED,
+        title=f"Document processed: {filename}",
+        message=f"Your document '{filename}' has been successfully processed and uploaded.",
+        file_id=file_id,
+    )
+
+
+def notify_user_document_failed(owner_id: str, filename: str, error: str, file_id: int | None = None) -> None:
+    """Notify a user that their document processing failed."""
+    dispatch_user_notification(
+        owner_id=owner_id,
+        event_type=EVENT_DOCUMENT_FAILED,
+        title=f"Document processing failed: {filename}",
+        message=f"Processing of '{filename}' failed: {error}",
+        file_id=file_id,
+    )

app/views/__init__.py

@@ -5,7 +5,10 @@ Aggregated view routers for the application.
 from fastapi import APIRouter
 
 from app.views.admin_users import router as admin_users_router
+from app.views.api_tokens import router as api_tokens_router
+from app.views.audit_logs import router as audit_logs_router
 from app.views.backup import router as backup_router
+from app.views.compliance import router as compliance_router
 from app.views.db_wizard import router as db_wizard_router
 from app.views.dropbox import router as dropbox_router
 from app.views.filemanager import router as filemanager_router
@@ -14,14 +17,21 @@ from app.views.filemanager import router as filemanager_router
 from app.views.general import router as general_router
 from app.views.google_drive import router as google_drive_router
 from app.views.help import router as help_router  # Built-in help / How-To docs
+from app.views.imap_accounts import router as imap_accounts_router
+from app.views.integrations import router as integrations_router  # Unified integrations dashboard
 from app.views.license_routes import router as license_router  # Add the license router
+from app.views.notifications import router as notifications_router
 from app.views.onboarding import router as onboarding_router
 from app.views.onedrive import router as onedrive_router
 from app.views.pipelines import router as pipelines_router  # Processing pipelines
 from app.views.plans import router as plans_router  # Admin Plan Designer
+from app.views.profile import router as profile_router  # User self-service profile
 from app.views.queue import router as queue_router
+from app.views.scheduled_jobs import router as scheduled_jobs_router  # Scheduled batch jobs
 from app.views.search import router as search_router
 from app.views.settings import router as settings_router
+from app.views.share import router as share_router
+from app.views.shared_links import router as shared_links_router
 from app.views.status import router as status_router
 from app.views.subscriptions import router as subscriptions_router  # Pricing + subscription pages
 from app.views.wizard import router as wizard_router
@@ -31,6 +41,9 @@ router = APIRouter()
 router.include_router(wizard_router)  # Wizard first (for /setup)
 router.include_router(db_wizard_router)  # Database wizard
 router.include_router(admin_users_router)  # Admin user management
+router.include_router(api_tokens_router)  # API token management
+router.include_router(shared_links_router)  # Shared links management
+router.include_router(share_router)  # Public share landing pages (no auth)
 router.include_router(backup_router)  # Backup dashboard
 router.include_router(general_router)
 router.include_router(status_router)
@@ -46,4 +59,11 @@ router.include_router(subscriptions_router)  # Pricing + subscription pages
 router.include_router(plans_router)  # Admin Plan Designer
 router.include_router(onboarding_router)  # User onboarding wizard
 router.include_router(pipelines_router)  # Processing pipelines
+router.include_router(profile_router)  # User self-service profile settings
+router.include_router(imap_accounts_router)  # Per-user IMAP ingestion accounts
+router.include_router(integrations_router)  # Unified integrations dashboard
+router.include_router(notifications_router)  # User notification dashboard
+router.include_router(scheduled_jobs_router)  # Admin scheduled batch jobs
+router.include_router(audit_logs_router)  # Comprehensive audit log viewer
 router.include_router(help_router)  # Built-in help / How-To docs
+router.include_router(compliance_router)  # Compliance templates dashboard

app/views/api_tokens.py

@@ -0,0 +1,24 @@
+"""View route for the API Tokens management page.
+
+Renders the ``api_tokens.html`` template where users can create, view,
+and revoke their personal API tokens for programmatic access.
+"""
+
+import logging
+
+from fastapi import APIRouter, Request
+
+from app.views.base import require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/api-tokens")
+@require_login
+async def api_tokens_page(request: Request):
+    """Render the API Tokens management page."""
+    return templates.TemplateResponse(
+        "api_tokens.html",
+        {"request": request, "page_title": "API Tokens"},
+    )

app/views/audit_logs.py

@@ -0,0 +1,46 @@
+"""
+Audit log viewer UI — admin-only page with filtering and SIEM status.
+"""
+
+import logging
+
+from fastapi import Depends, HTTPException, Request, status
+from sqlalchemy.orm import Session
+
+from app.views.base import APIRouter, get_db, require_login, settings, templates
+from app.views.settings import require_admin_access
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/admin/audit-logs")
+@require_login
+@require_admin_access
+async def audit_logs_page(request: Request, db: Session = Depends(get_db)):
+    """Comprehensive audit log viewer with filtering controls.
+
+    Displays a chronological log of all significant actions: logins,
+    document operations, settings changes, and admin actions.  The
+    actual data is fetched client-side via the ``/api/audit-logs`` JSON
+    endpoint so that filters, pagination, and live refresh work without
+    full-page reloads.
+    """
+    try:
+        siem_enabled = settings.audit_siem_enabled
+        siem_transport = settings.audit_siem_transport if siem_enabled else None
+        return templates.TemplateResponse(
+            "audit_logs.html",
+            {
+                "request": request,
+                "app_version": settings.version,
+                "siem_enabled": siem_enabled,
+                "siem_transport": siem_transport,
+            },
+        )
+    except Exception as e:
+        logger.error("Error loading audit logs page: %s", e)
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to load audit logs page",
+        )

app/views/base.py

@@ -11,7 +11,17 @@ from sqlalchemy.orm import Session  # noqa: F401
 
 from app.auth import require_login  # noqa: F401
 from app.config import settings
-from app.database import get_db  # noqa: F401
+from app.database import SessionLocal, get_db  # noqa: F401
+from app.models import UserProfile
+from app.utils.i18n import (
+    SUPPORTED_LANGUAGES,
+    detect_language,
+    format_date,
+    format_datetime,
+    format_number,
+    get_suggested_languages,
+    translate,
+)
 
 # Set up Jinja2 templates
 templates_dir = Path(__file__).parent.parent.parent / "frontend" / "templates"
@@ -21,10 +31,58 @@ templates = Jinja2Templates(directory=str(templates_dir))
 templates.env.globals["min"] = min
 templates.env.globals["max"] = max
 
+# ---------------------------------------------------------------------------
+# i18n Jinja2 integration
+# ---------------------------------------------------------------------------
+# The _() function is available in every template to translate UI strings.
+# Usage: {{ _("nav.dashboard") }} or {{ _("upload.max_size", size="10 MB") }}
+# The locale is automatically resolved from the request context.
+# A default English implementation is registered as a global so error handlers
+# that don't go through _inject_global_context still have the function available.
+# ---------------------------------------------------------------------------
+
+templates.env.globals["supported_languages"] = SUPPORTED_LANGUAGES
+templates.env.globals["_"] = lambda key, **kwargs: translate(key, "en", **kwargs)
+
 # Customize Jinja2Templates to include app_version in all templates
 original_template_response = templates.TemplateResponse
 
 
+def _hydrate_language_from_db(request: Request, session_user: object) -> None:
+    """Load the user's preferred language from the DB into the session.
+
+    Called once per session when ``preferred_language`` is not yet in the
+    session.  A lightweight DB query fetches the stored preference so that
+    :func:`detect_language` picks it up from the session on all subsequent
+    requests without further DB access.
+    """
+    from app.utils.i18n import SUPPORTED_LANGUAGE_CODES
+
+    user_id: str | None = None
+    if isinstance(session_user, dict):
+        user_id = (
+            session_user.get("sub")
+            or session_user.get("preferred_username")
+            or session_user.get("email")
+            or session_user.get("id")
+        )
+    elif isinstance(session_user, str):
+        user_id = session_user
+
+    if not user_id:
+        return
+
+    db = SessionLocal()
+    try:
+        profile = db.query(UserProfile).filter(UserProfile.user_id == user_id).first()
+        if profile and profile.preferred_language and profile.preferred_language in SUPPORTED_LANGUAGE_CODES:
+            request.session["preferred_language"] = profile.preferred_language
+    except Exception:  # noqa: BLE001 — intentionally broad; DB may be temporarily unavailable
+        logger.debug("Could not hydrate language preference for user_id=%s", user_id)
+    finally:
+        db.close()
+
+
 def _inject_global_context(ctx: dict) -> None:
     """Inject shared global variables into every template context dict."""
     ctx.setdefault("version", settings.version)
@@ -48,8 +106,43 @@ def _inject_global_context(ctx: dict) -> None:
             session_user = req.session.get("user")
         # When auth is disabled every visitor is effectively "logged in"
         ctx.setdefault("is_logged_in", not getattr(settings, "auth_enabled", True) or session_user is not None)
+
+        # --- Hydrate session language from DB (once per session) ---
+        # If the session doesn't have a preferred_language yet but the user
+        # is logged in, load the stored preference from the database so that
+        # detect_language() picks it up from the session on this and all
+        # subsequent requests.
+        if hasattr(req, "session") and "preferred_language" not in req.session and session_user is not None:
+            _hydrate_language_from_db(req, session_user)
+
+        # --- i18n: detect language and register template helpers ---
+        current_locale = detect_language(req)
+        ctx.setdefault("current_locale", current_locale)
+
+        # Smart language suggestions for the compact nav-bar dropdown (5-7 languages)
+        accept_header = req.headers.get("accept-language", "") if hasattr(req, "headers") else ""
+        ctx.setdefault("suggested_languages", get_suggested_languages(current_locale, accept_header))
+
+        def _translate(key: str, **kwargs: object) -> str:
+            return translate(key, current_locale, **kwargs)
+
+        def _format_date(value: object, short: bool = False) -> str:
+            return format_date(value, current_locale, short=short)  # type: ignore[arg-type]
+
+        def _format_datetime(value: object) -> str:
+            return format_datetime(value, current_locale)  # type: ignore[arg-type]
+
+        def _format_number(value: object) -> str:
+            return format_number(value, current_locale)  # type: ignore[arg-type]
+
+        ctx.setdefault("_", _translate)
+        ctx.setdefault("format_date_l10n", _format_date)
+        ctx.setdefault("format_datetime_l10n", _format_datetime)
+        ctx.setdefault("format_number_l10n", _format_number)
     else:
         ctx.setdefault("is_logged_in", not getattr(settings, "auth_enabled", True))
+        ctx.setdefault("current_locale", "en")
+        ctx.setdefault("_", lambda key, **kw: translate(key, "en", **kw))
 
 
 def template_response_with_version(*args, **kwargs):

app/views/compliance.py

@@ -0,0 +1,48 @@
+"""Admin view: compliance templates dashboard page."""
+
+import logging
+
+from fastapi import HTTPException, Request, status
+from fastapi.responses import RedirectResponse
+
+from app.views.base import APIRouter, require_login, settings, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+def _require_admin(request: Request):
+    """Return the session user if they are an admin, else None."""
+    user = request.session.get("user")
+    if not user or not user.get("is_admin"):
+        logger.warning("Non-admin user attempted to access /admin/compliance")
+        return None
+    return user
+
+
+@router.get("/admin/compliance")
+@require_login
+async def compliance_page(request: Request):
+    """Admin compliance templates dashboard page.
+
+    Displays GDPR, HIPAA, and SOC2 compliance templates with their current
+    status and one-click apply functionality.
+    """
+    user = _require_admin(request)
+    if user is None:
+        return RedirectResponse(url="/", status_code=status.HTTP_302_FOUND)
+
+    try:
+        return templates.TemplateResponse(
+            "compliance.html",
+            {
+                "request": request,
+                "app_version": settings.version,
+            },
+        )
+    except Exception as e:
+        logger.error(f"Error loading compliance page: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to load compliance page",
+        )

app/views/dropbox.py

@@ -2,32 +2,82 @@
 Dropbox integration views for setup and OAuth callback.
 """
 
-from fastapi import Request
+import json
 
-from app.views.base import APIRouter, require_login, settings, templates
+from fastapi import Query, Request
+from sqlalchemy.orm import Session
+
+from app.models import UserIntegration
+from app.utils.user_scope import get_current_owner_id
+from app.views.base import APIRouter, Depends, get_db, require_login, settings, templates
 
 router = APIRouter()
 
 
 @router.get("/dropbox-setup")
 @require_login
-async def dropbox_setup_page(request: Request):
+async def dropbox_setup_page(
+    request: Request,
+    integration_id: int | None = Query(None),
+    db: Session = Depends(get_db),
+):
     """
     Setup page for the Dropbox integration.
-    Shows configuration status and setup instructions.
+
+    When ``integration_id`` is provided the page operates in **user mode**:
+    the OAuth wizard saves credentials to the named per-user integration
+    record rather than to the global application settings.  Only the folder
+    path from the integration's existing config is pre-populated; global
+    admin credentials are never exposed in this mode.
     """
-    # Check Dropbox configuration
+    if integration_id is not None:
+        owner_id = get_current_owner_id(request)
+        integration = (
+            db.query(UserIntegration)
+            .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+            .first()
+        )
+        if integration:
+            cfg: dict = {}
+            if integration.config:
+                try:
+                    cfg = json.loads(integration.config)
+                except (json.JSONDecodeError, TypeError):
+                    cfg = {}
+            # Support both "folder" (DROPBOX destination) and "folder_path" (WATCH_FOLDER source)
+            folder_path = cfg.get("folder", cfg.get("folder_path", ""))
+            return templates.TemplateResponse(
+                "dropbox.html",
+                {
+                    "request": request,
+                    "user_mode": True,
+                    "is_configured": bool(integration.credentials),
+                    "integration_id": integration_id,
+                    "integration_name": integration.name,
+                    "integration_type": integration.integration_type,
+                    "folder_path": folder_path,
+                    "app_key_value": "",
+                    "app_secret_value": "",
+                    "refresh_token_value": "",
+                },
+            )
+
+    # ── Admin / global mode ──────────────────────────────────────────────────
     is_configured = bool(settings.dropbox_app_key and settings.dropbox_app_secret and settings.dropbox_refresh_token)
 
     return templates.TemplateResponse(
         "dropbox.html",
         {
             "request": request,
+            "user_mode": False,
             "is_configured": is_configured,
             "app_key_value": settings.dropbox_app_key or "",
             "app_secret_value": settings.dropbox_app_secret if settings.dropbox_app_secret else "",
             "refresh_token_value": settings.dropbox_refresh_token if settings.dropbox_refresh_token else "",
-            "folder_path": settings.dropbox_folder or "/Documents/Uploads",  # Default folder path
+            "folder_path": settings.dropbox_folder or "/Documents/Uploads",
+            "integration_id": integration_id,
+            "integration_name": None,
+            "integration_type": None,
         },
     )
 

app/views/google_drive.py

@@ -2,47 +2,90 @@
 Google Drive integration views for setup and OAuth callback.
 """
 
+import json
 import urllib.parse
 
-from fastapi import Request
+from fastapi import Query, Request
 from fastapi.responses import RedirectResponse
+from sqlalchemy.orm import Session
 
-from app.views.base import APIRouter, require_login, settings, templates
+from app.models import UserIntegration
+from app.utils.user_scope import get_current_owner_id
+from app.views.base import APIRouter, Depends, get_db, require_login, settings, templates
 
 router = APIRouter()
 
 
 @router.get("/google-drive-setup")
 @require_login
-async def google_drive_setup_page(request: Request):
+async def google_drive_setup_page(
+    request: Request,
+    integration_id: int | None = Query(None),
+    db: Session = Depends(get_db),
+):
     """
     Setup page for the Google Drive integration.
-    Shows configuration status and setup instructions.
+
+    When ``integration_id`` is provided the page operates in **user mode**:
+    the OAuth wizard saves credentials to the named per-user integration
+    record rather than to the global application settings.
     """
-    # Check if using OAuth
+    if integration_id is not None:
+        owner_id = get_current_owner_id(request)
+        integration = (
+            db.query(UserIntegration)
+            .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+            .first()
+        )
+        if integration:
+            cfg: dict = {}
+            if integration.config:
+                try:
+                    cfg = json.loads(integration.config)
+                except (json.JSONDecodeError, TypeError):
+                    cfg = {}
+            folder_id = cfg.get("folder_id", "")
+            return templates.TemplateResponse(
+                "google_drive.html",
+                {
+                    "request": request,
+                    "user_mode": True,
+                    "is_configured": bool(integration.credentials),
+                    "integration_id": integration_id,
+                    "integration_name": integration.name,
+                    "integration_type": integration.integration_type,
+                    "folder_id": folder_id,
+                    "use_oauth": True,
+                    "oauth_configured": bool(integration.credentials),
+                    "sa_configured": False,
+                    "client_id": False,
+                    "client_id_value": "",
+                    "client_secret": False,
+                    "client_secret_value": "",
+                    "refresh_token": False,
+                    "refresh_token_value": "",
+                    "has_credentials_json": False,
+                },
+            )
+
+    # ── Admin / global mode ──────────────────────────────────────────────────
     use_oauth = getattr(settings, "google_drive_use_oauth", False)
 
-    # Check Google Drive OAuth configuration
     oauth_configured = bool(
         settings.google_drive_client_id and settings.google_drive_client_secret and settings.google_drive_refresh_token
     )
-
-    # Check Google Drive service account configuration
     sa_configured = bool(settings.google_drive_credentials_json)
-
-    # Overall configuration status
     is_configured = (use_oauth and oauth_configured) or (not use_oauth and sa_configured)
-
     if settings.google_drive_folder_id:
         is_configured = is_configured and True
     else:
         is_configured = False
 
-    # Get configuration values to display status (hide sensitive values)
     return templates.TemplateResponse(
         "google_drive.html",
         {
             "request": request,
+            "user_mode": False,
             "is_configured": is_configured,
             "use_oauth": use_oauth,
             "oauth_configured": oauth_configured,
@@ -55,6 +98,9 @@ async def google_drive_setup_page(request: Request):
             "refresh_token_value": settings.google_drive_refresh_token or "",
             "folder_id": settings.google_drive_folder_id or "",
             "has_credentials_json": bool(settings.google_drive_credentials_json),
+            "integration_id": integration_id,
+            "integration_name": None,
+            "integration_type": None,
         },
     )
 

app/views/help.py

@@ -1,27 +1,59 @@
 """
-Help documentation view routes.
+Help Center view routes.
 
-Serves the built MkDocs documentation site at /help.
-The static site is built during the Docker image build and placed at docs_build/.
+Renders a user-facing, SEO-optimised Help Center page at ``/help``.
+The page is designed for SaaS end-users and includes guidance on
+features, integrations, workflows, and optional Zammad support widgets
+(live-chat and feedback form).
+
+The developer-oriented MkDocs documentation is served separately at
+``/docs`` and is intentionally **not** cross-linked from this page.
 """
 
 import logging
 import pathlib
 
 from fastapi import Request
-from fastapi.responses import RedirectResponse
 
-from app.views.base import APIRouter
+from app.config import settings
+from app.views.base import APIRouter, templates
 
 logger = logging.getLogger(__name__)
 
 router = APIRouter()
 
-# Path to the built MkDocs documentation
+# Path to the built MkDocs documentation (kept for reference / backwards compat)
 _DOCS_BUILD_DIR = pathlib.Path(__file__).parents[2] / "docs_build"
 
 
 @router.get("/help", include_in_schema=False)
-async def help_redirect(request: Request) -> RedirectResponse:
-    """Redirect /help to /help/ so the MkDocs index is served correctly."""
-    return RedirectResponse(url="/help/", status_code=301)
+async def help_center(request: Request):
+    """Render the end-user Help Center page."""
+    # Extract user context for Zammad widget pre-fill ----------------------
+    session_user = None
+    if hasattr(request, "session"):
+        session_user = request.session.get("user")
+
+    user_name = ""
+    user_email = ""
+    user_id = ""
+    if session_user and isinstance(session_user, dict):
+        user_name = session_user.get("name") or session_user.get("display_name") or ""
+        user_email = session_user.get("email") or ""
+        user_id = session_user.get("preferred_username") or session_user.get("email") or session_user.get("id") or ""
+
+    return templates.TemplateResponse(
+        "help.html",
+        {
+            "request": request,
+            "external_hostname": settings.external_hostname,
+            "zammad_url": settings.zammad_url,
+            "zammad_chat_enabled": settings.zammad_chat_enabled,
+            "zammad_chat_id": settings.zammad_chat_id,
+            "zammad_form_enabled": settings.zammad_form_enabled,
+            "support_email": settings.support_email,
+            "user_name": user_name,
+            "user_email": user_email,
+            "user_id": user_id,
+        },
+    )

app/views/imap_accounts.py

@@ -0,0 +1,105 @@
+"""User-facing view for the per-user IMAP ingestion dashboard."""
+
+import json
+import logging
+
+from fastapi import Request
+from sqlalchemy.orm import Session
+
+from app.models import ImapIngestionProfile, UserImapAccount
+from app.utils.allowed_types import DEFAULT_CATEGORIES, FILE_TYPE_CATEGORIES
+from app.utils.subscription import get_tier, get_user_tier_id
+from app.utils.user_scope import get_current_owner_id
+from app.views.base import APIRouter, Depends, get_db, require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+def _get_max_mailboxes(tier: dict) -> int | None:
+    """Mirror of the quota helper from the API module — avoids a circular import."""
+    tier_id: str = tier.get("id", "free")
+    max_mb: int = tier.get("max_mailboxes", 0)
+    if tier_id == "free":
+        return 0
+    if max_mb == 0:
+        return None
+    return max_mb
+
+
+def _serialize_profile(profile: ImapIngestionProfile) -> dict:
+    """Serialize a profile for JSON embedding in the template."""
+    try:
+        categories = json.loads(profile.allowed_categories)
+    except (ValueError, TypeError):
+        categories = []
+    return {
+        "id": profile.id,
+        "name": profile.name,
+        "description": profile.description,
+        "owner_id": profile.owner_id,
+        "allowed_categories": categories,
+        "is_builtin": profile.is_builtin,
+    }
+
+
+@router.get("/imap-accounts")
+@require_login
+async def imap_accounts_page(request: Request, db: Session = Depends(get_db)):
+    """IMAP ingestion account management page for the current user."""
+    owner_id = get_current_owner_id(request)
+
+    accounts: list[UserImapAccount] = []
+    current_count = 0
+    max_mailboxes: int | None = 0
+    can_add = False
+    tier_name = "Free"
+    tier_id = "free"
+
+    if owner_id:
+        accounts = (
+            db.query(UserImapAccount).filter(UserImapAccount.owner_id == owner_id).order_by(UserImapAccount.id).all()
+        )
+        current_count = len(accounts)
+        tier_id = get_user_tier_id(db, owner_id)
+        tier = get_tier(tier_id, db)
+        tier_name = tier.get("name", tier_id)
+        max_mailboxes = _get_max_mailboxes(tier)
+        can_add = max_mailboxes is None or (max_mailboxes > 0 and current_count < max_mailboxes)
+
+    # Load ingestion profiles: system-global + user's own
+    profiles = (
+        db.query(ImapIngestionProfile)
+        .filter(
+            # SQLAlchemy requires `== None` for IS NULL comparison in ORM filters
+            (ImapIngestionProfile.owner_id == None) | (ImapIngestionProfile.owner_id == owner_id)  # noqa: E711
+        )
+        .order_by(ImapIngestionProfile.is_builtin.desc(), ImapIngestionProfile.id)
+        .all()
+    )
+
+    # Category definitions for the UI checkbox builder
+    categories = [
+        {
+            "key": key,
+            "label": info["label"],
+            "description": info["description"],
+        }
+        for key, info in FILE_TYPE_CATEGORIES.items()
+    ]
+
+    return templates.TemplateResponse(
+        "imap_accounts.html",
+        {
+            "request": request,
+            "accounts": accounts,
+            "profiles": [_serialize_profile(p) for p in profiles],
+            "categories": categories,
+            "default_categories": DEFAULT_CATEGORIES,
+            "current_count": current_count,
+            "max_mailboxes": max_mailboxes,
+            "can_add": can_add,
+            "tier_id": tier_id,
+            "tier_name": tier_name,
+        },
+    )

app/views/integrations.py

@@ -0,0 +1,116 @@
+"""User-facing view for the unified Sources & Destinations dashboard."""
+
+import logging
+
+from fastapi import HTTPException, Request, status
+from sqlalchemy.orm import Session
+
+from app.models import IntegrationDirection, IntegrationType, UserIntegration
+from app.utils.subscription import get_tier, get_user_tier_id
+from app.utils.user_scope import get_current_owner_id
+from app.views.base import APIRouter, Depends, get_db, require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+# Source types that consume the mailbox quota (mirrored from app.api.integrations)
+_MAILBOX_SOURCE_TYPES = {IntegrationType.IMAP}
+_FREE_TIER_ID = "free"
+
+
+def _get_max_destinations(tier: dict) -> int | None:
+    """Return the maximum number of storage destinations allowed by *tier*.
+
+    Returns ``None`` for unlimited, or a positive int for the cap.
+    """
+    tier_id: str = tier.get("id", _FREE_TIER_ID)
+    max_dest: int = tier.get("max_storage_destinations", 0)
+    if tier_id == _FREE_TIER_ID:
+        return max_dest if max_dest > 0 else 1
+    if max_dest == 0:
+        return None
+    return max_dest
+
+
+def _get_max_sources(tier: dict) -> int | None:
+    """Return the maximum number of IMAP sources allowed by *tier*.
+
+    Returns ``None`` for unlimited, ``0`` for no access, or a positive int.
+    """
+    tier_id: str = tier.get("id", _FREE_TIER_ID)
+    max_mb: int = tier.get("max_mailboxes", 0)
+    if tier_id == _FREE_TIER_ID:
+        return 0
+    if max_mb == 0:
+        return None
+    return max_mb
+
+
+@router.get("/integrations")
+@require_login
+async def integrations_dashboard(request: Request, db: Session = Depends(get_db)):
+    """Render the unified Sources & Destinations dashboard."""
+    try:
+        owner_id = get_current_owner_id(request)
+
+        dest_count = 0
+        src_count = 0
+        max_destinations: int | None = 1
+        max_sources: int | None = 0
+        can_add_destination = False
+        can_add_source = False
+        tier_name = "Free"
+        tier_id = "free"
+
+        if owner_id:
+            from sqlalchemy import func
+
+            dest_count = (
+                db.query(func.count())
+                .select_from(UserIntegration)
+                .filter(
+                    UserIntegration.owner_id == owner_id,
+                    UserIntegration.direction == IntegrationDirection.DESTINATION,
+                )
+                .scalar()
+            ) or 0
+            src_count = (
+                db.query(func.count())
+                .select_from(UserIntegration)
+                .filter(
+                    UserIntegration.owner_id == owner_id,
+                    UserIntegration.direction == IntegrationDirection.SOURCE,
+                    UserIntegration.integration_type.in_(list(_MAILBOX_SOURCE_TYPES)),
+                )
+                .scalar()
+            ) or 0
+            tier_id = get_user_tier_id(db, owner_id)
+            tier = get_tier(tier_id, db)
+            tier_name = tier.get("name", tier_id)
+            max_destinations = _get_max_destinations(tier)
+            max_sources = _get_max_sources(tier)
+            can_add_destination = max_destinations is None or dest_count < max_destinations
+            can_add_source = max_sources is None or (max_sources > 0 and src_count < max_sources)
+
+        return templates.TemplateResponse(
+            "integrations_dashboard.html",
+            {
+                "request": request,
+                "dest_count": dest_count,
+                "src_count": src_count,
+                "max_destinations": max_destinations,
+                "max_sources": max_sources,
+                "can_add_destination": can_add_destination,
+                "can_add_source": can_add_source,
+                "tier_id": tier_id,
+                "tier_name": tier_name,
+            },
+        )
+    except HTTPException:
+        raise
+    except Exception as exc:
+        logger.error(f"Error loading integrations dashboard: {exc}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to load integrations dashboard",
+        )

app/views/notifications.py

@@ -0,0 +1,20 @@
+"""View route for the notifications dashboard."""
+
+import logging
+
+from fastapi import Request
+
+from app.views.base import APIRouter, require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/notifications")
+@require_login
+async def notifications_dashboard(request: Request):
+    """Render the notifications dashboard."""
+    return templates.TemplateResponse(
+        "notifications_dashboard.html",
+        {"request": request, "page_title": "Notifications"},
+    )

app/views/onboarding.py

@@ -27,6 +27,7 @@ _DESTINATION_META: list[dict] = [
     {"id": "webdav", "name": "WebDAV", "icon": "fas fa-server"},
     {"id": "sftp", "name": "SFTP", "icon": "fas fa-terminal"},
     {"id": "ftp", "name": "FTP", "icon": "fas fa-server"},
+    {"id": "icloud", "name": "iCloud Drive", "icon": "fab fa-apple"},
 ]
 
 
@@ -51,6 +52,7 @@ def _get_configured_destinations(cfg: Settings) -> list[dict]:
         "webdav": bool(cfg.webdav_url and cfg.webdav_username),
         "sftp": bool(cfg.sftp_host and cfg.sftp_username),
         "ftp": bool(cfg.ftp_host and cfg.ftp_username),
+        "icloud": bool(cfg.icloud_username and cfg.icloud_password),
     }
     return [meta for meta in _DESTINATION_META if checks.get(meta["id"], False)]
 

app/views/onedrive.py

@@ -2,39 +2,90 @@
 OneDrive integration views for setup and OAuth callback.
 """
 
-from fastapi import Request
+import json
 
-from app.views.base import APIRouter, require_login, settings, templates
+from fastapi import Query, Request
+from sqlalchemy.orm import Session
+
+from app.models import UserIntegration
+from app.utils.user_scope import get_current_owner_id
+from app.views.base import APIRouter, Depends, get_db, require_login, settings, templates
 
 router = APIRouter()
 
 
 @router.get("/onedrive-setup")
 @require_login
-async def onedrive_setup_page(request: Request):
+async def onedrive_setup_page(
+    request: Request,
+    integration_id: int | None = Query(None),
+    db: Session = Depends(get_db),
+):
     """
     Setup page for the OneDrive integration.
-    Shows configuration status and setup instructions.
+
+    When ``integration_id`` is provided the page operates in **user mode**:
+    the OAuth wizard saves credentials to the named per-user integration
+    record rather than to the global application settings.
     """
-    # Check OneDrive configuration
+    if integration_id is not None:
+        owner_id = get_current_owner_id(request)
+        integration = (
+            db.query(UserIntegration)
+            .filter(UserIntegration.id == integration_id, UserIntegration.owner_id == owner_id)
+            .first()
+        )
+        if integration:
+            cfg: dict = {}
+            if integration.config:
+                try:
+                    cfg = json.loads(integration.config)
+                except (json.JSONDecodeError, TypeError):
+                    cfg = {}
+            # Support both "folder_path" (WATCH_FOLDER / ONEDRIVE destination)
+            folder_path = cfg.get("folder_path", cfg.get("folder", ""))
+            return templates.TemplateResponse(
+                "onedrive.html",
+                {
+                    "request": request,
+                    "user_mode": True,
+                    "is_configured": bool(integration.credentials),
+                    "integration_id": integration_id,
+                    "integration_name": integration.name,
+                    "integration_type": integration.integration_type,
+                    "folder_path": folder_path,
+                    "client_id": False,
+                    "client_id_value": "",
+                    "client_secret": False,
+                    "client_secret_value": "",
+                    "tenant_id": "common",
+                    "refresh_token": False,
+                    "refresh_token_value": "",
+                },
+            )
+
+    # ── Admin / global mode ──────────────────────────────────────────────────
     is_configured = bool(
         settings.onedrive_client_id and settings.onedrive_client_secret and settings.onedrive_refresh_token
     )
 
-    # Get configuration values to display status (hide sensitive values)
     return templates.TemplateResponse(
         "onedrive.html",
         {
             "request": request,
+            "user_mode": False,
             "is_configured": is_configured,
             "client_id": bool(settings.onedrive_client_id),
-            "client_id_value": settings.onedrive_client_id or "",  # Pass the actual value for the form
+            "client_id_value": settings.onedrive_client_id or "",
             "client_secret": bool(settings.onedrive_client_secret),
             "client_secret_value": settings.onedrive_client_secret if settings.onedrive_client_secret else "",
             "tenant_id": settings.onedrive_tenant_id,
             "refresh_token": bool(settings.onedrive_refresh_token),
             "refresh_token_value": settings.onedrive_refresh_token if settings.onedrive_refresh_token else "",
-            "folder_path": settings.onedrive_folder_path or "Documents/Uploads",  # Default folder path
+            "folder_path": settings.onedrive_folder_path or "Documents/Uploads",
+            "integration_id": integration_id,
+            "integration_name": None,
+            "integration_type": None,
         },
     )
 

app/views/plans.py

@@ -1,14 +1,13 @@
-"""View route for the admin Plan Designer page."""
+"""View routes for admin plan management pages."""
 
 from fastapi import Request
 from fastapi.responses import HTMLResponse
 from fastapi.routing import APIRouter
-from fastapi.templating import Jinja2Templates
 
 from app.auth import require_login
+from app.views.base import templates
 
 router = APIRouter()
-templates = Jinja2Templates(directory="frontend/templates")
 
 
 @router.get("/admin/plans", response_class=HTMLResponse)
@@ -16,3 +15,10 @@ templates = Jinja2Templates(directory="frontend/templates")
 async def plan_designer(request: Request) -> HTMLResponse:
     """Admin Plan Designer page."""
     return templates.TemplateResponse("admin_plans.html", {"request": request})
+
+
+@router.get("/admin/stripe-wizard", response_class=HTMLResponse)
+@require_login
+async def stripe_wizard(request: Request) -> HTMLResponse:
+    """Admin Stripe Setup Wizard page."""
+    return templates.TemplateResponse("admin_stripe_wizard.html", {"request": request})

app/views/profile.py

@@ -0,0 +1,40 @@
+"""View route for the user self-service profile settings page.
+
+Route:
+  GET /profile  — renders the profile settings HTML page (requires login)
+"""
+
+from __future__ import annotations
+
+import logging
+
+from fastapi import Depends, Request
+from sqlalchemy.orm import Session
+
+from app.models import UserProfile
+from app.utils.i18n import SUPPORTED_LANGUAGES
+from app.views.base import APIRouter, get_db, require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/profile", include_in_schema=False)
+@require_login
+async def profile_page(request: Request, db: Session = Depends(get_db)):
+    """Serve the user profile settings page."""
+    user = request.session.get("user") or {}
+    user_id = user.get("sub") or user.get("preferred_username") or user.get("email") or user.get("id")
+
+    profile = None
+    if user_id:
+        profile = db.query(UserProfile).filter(UserProfile.user_id == user_id).first()
+
+    return templates.TemplateResponse(
+        "profile.html",
+        {
+            "request": request,
+            "profile": profile,
+            "supported_languages": SUPPORTED_LANGUAGES,
+        },
+    )

app/views/scheduled_jobs.py

@@ -0,0 +1,44 @@
+"""Admin view: scheduled batch processing jobs management page."""
+
+import logging
+
+from fastapi import HTTPException, Request, status
+from fastapi.responses import RedirectResponse
+
+from app.views.base import APIRouter, require_login, settings, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+def _require_admin(request: Request):
+    """Return the session user if they are an admin, else None."""
+    user = request.session.get("user")
+    if not user or not user.get("is_admin"):
+        logger.warning("Non-admin user attempted to access /admin/scheduled-jobs")
+        return None
+    return user
+
+
+@router.get("/admin/scheduled-jobs")
+@require_login
+async def scheduled_jobs_page(request: Request):
+    """Admin scheduled jobs management page."""
+    user = _require_admin(request)
+    if user is None:
+        return RedirectResponse(url="/", status_code=status.HTTP_302_FOUND)
+
+    try:
+        return templates.TemplateResponse(
+            "admin_scheduled_jobs.html",
+            {
+                "request": request,
+                "app_version": settings.version,
+            },
+        )
+    except Exception as e:
+        logger.error(f"Error loading scheduled jobs page: {e}")
+        raise HTTPException(
+            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+            detail="Failed to load scheduled jobs page",
+        )

app/views/share.py

@@ -0,0 +1,28 @@
+"""Public view for accessing a shared document link.
+
+This route does NOT require authentication — it is the landing page
+that link recipients visit.  The page fetches link metadata via the
+public ``/api/share/{token}/info`` JSON endpoint and then renders the
+appropriate download UI (password gate or direct download button).
+"""
+
+import logging
+import pathlib
+
+from fastapi import APIRouter, Request
+from fastapi.templating import Jinja2Templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+_templates_dir = pathlib.Path(__file__).parents[2] / "frontend" / "templates"
+templates = Jinja2Templates(directory=str(_templates_dir))
+
+
+@router.get("/share/{token}")
+async def shared_link_view(request: Request, token: str):
+    """Render the public share landing page for a given token."""
+    return templates.TemplateResponse(
+        "shared_link_view.html",
+        {"request": request, "token": token},
+    )

app/views/shared_links.py

@@ -0,0 +1,24 @@
+"""View route for the Shared Links management page.
+
+Renders the ``shared_links.html`` template where authenticated users can
+create, view, and revoke their document share links.
+"""
+
+import logging
+
+from fastapi import APIRouter, Request
+
+from app.views.base import require_login, templates
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.get("/shared-links")
+@require_login
+async def shared_links_page(request: Request):
+    """Render the Shared Links management page."""
+    return templates.TemplateResponse(
+        "shared_links.html",
+        {"request": request, "page_title": "Shared Links"},
+    )

app/views/status.py

@@ -6,7 +6,8 @@ import logging
 import os
 from datetime import datetime
 
-from fastapi import Request
+from fastapi import Request, status
+from fastapi.responses import RedirectResponse
 
 from app.utils.config_validator import get_provider_status
 from app.views.base import APIRouter, require_login, settings, templates
@@ -15,12 +16,25 @@ logger = logging.getLogger(__name__)
 router = APIRouter()
 
 
+def _require_admin(request: Request):
+    """Return the session user if they are an admin, else return None."""
+    user = request.session.get("user")
+    if not user or not user.get("is_admin"):
+        user_email = user.get("email", "anonymous") if user else "anonymous"
+        logger.warning(f"Non-admin user {user_email} attempted to access /status")
+        return None
+    return user
+
+
 @router.get("/status")
 @require_login
 async def status_dashboard(request: Request):
     """
-    Status dashboard showing all configured integration targets
+    Status dashboard showing all configured integration targets (admin only).
     """
+    user = _require_admin(request)
+    if user is None:
+        return RedirectResponse(url="/", status_code=status.HTTP_302_FOUND)
     # Get provider status
     providers = get_provider_status()
 

codecov.yml

@@ -0,0 +1,71 @@
+# Codecov Configuration for DocuElevate
+# Documentation: https://docs.codecov.com/docs/codecov-yaml
+#
+# SETUP INSTRUCTIONS:
+# 1. Go to https://app.codecov.io and sign in with GitHub
+# 2. Add the DocuElevate repository
+# 3. Copy the repository upload token
+# 4. Add it as a GitHub Actions secret named CODECOV_TOKEN
+#    (Settings → Secrets and variables → Actions → New repository secret)
+# 5. Push to trigger CI — Codecov will begin reporting coverage
+
+# ── Coverage thresholds ────────────────────────────────────────────────────
+coverage:
+  # Minimum acceptable overall project coverage
+  # CI will fail if the total drops below this percentage
+  status:
+    project:
+      default:
+        # Fail if overall project coverage drops below 60%
+        target: 60%
+        # Allow up to 2% drop compared to the base branch before failing
+        threshold: 2%
+        # Branches to enforce the threshold against
+        branches:
+          - main
+          - develop
+    patch:
+      default:
+        # Fail if the lines changed in a PR are covered below 70%
+        target: 70%
+        # Allow up to 5% slack on patch coverage (newly added/changed lines)
+        threshold: 5%
+
+  # Lines that are never counted toward coverage (mirrors pyproject.toml)
+  ignore:
+    - "migrations/**"
+    - "tests/**"
+    - "frontend/**"
+    - "docs/**"
+    - "scripts/**"
+    - "**/__pycache__/**"
+    - "**/conftest.py"
+
+# ── Pull-request comments ─────────────────────────────────────────────────
+comment:
+  # Post a coverage summary comment on every PR
+  layout: "condensed_header, condensed_files, condensed_footer"
+  behavior: default          # update the existing comment instead of posting a new one
+  require_changes: false     # always post, even if coverage hasn't changed
+  require_base: false        # post even when there is no base report to compare against
+  require_head: true         # only post when a head report is available
+  hide_project_coverage: false
+
+# ── Upload settings ───────────────────────────────────────────────────────
+# Flags let you split coverage by test type and track each independently
+# Each flag maps to the `flags` parameter in the codecov/codecov-action step
+flag_management:
+  individual_flags:
+    - name: unittests
+      paths:
+        - app/
+      carryforward: true
+    - name: integration
+      paths:
+        - app/
+      carryforward: true
+
+# ── Miscellaneous ─────────────────────────────────────────────────────────
+github_checks:
+  # Annotate individual lines in PR diffs with coverage status
+  annotations: true

crowdin.yml

@@ -0,0 +1,11 @@
+"project_id_env": "CROWDIN_PROJECT_ID"
+"api_token_env": "CROWDIN_PERSONAL_TOKEN"
+"base_path": "."
+"base_url": "https://api.crowdin.com"
+
+"preserve_hierarchy": true
+
+files:
+  - source: /frontend/translations/en.json
+    translation: /frontend/translations/%two_letters_code%.json
+    type: json

docs/API.md

@@ -2,6 +2,9 @@
 
 DocuElevate provides a powerful REST API for programmatic access to all its features. This document serves as a reference for the available endpoints and their usage.
 
+> **Looking for a quick way to script against DocuElevate?**
+> The built-in [CLI tool](./CLIGuide.md) wraps the API and is ready to use from a terminal or shell script — no HTTP client code required.
+
 ## API Overview
 
 - Base URL: `http://<your-docuelevate-instance>/api`
@@ -86,7 +89,38 @@ def make_api_request(url, max_retries=3):
 
 ## Authentication
 
-When authentication is enabled, you must include an authentication token in your requests:
+When authentication is enabled, you must include an authentication token in your requests.
+
+### API Tokens (Recommended)
+
+DocuElevate supports personal API tokens for programmatic access. Tokens are the recommended
+way to authenticate scripts, CI/CD pipelines, and webhook integrations.
+
+**Creating a token:**
+
+1. Log in to DocuElevate and navigate to **API Tokens** (available in your user menu or at `/api-tokens`).
+2. Enter a descriptive name (e.g. "CI Pipeline", "Scanner Integration") and click **Create Token**.
+3. Copy the token immediately — it is shown only once.
+
+**Using a token:**
+
+```bash
+curl -X GET "http://<your-docuelevate-instance>/api/files" \
+  -H "Authorization: Bearer <your-api-token>"
+```
+
+**Managing tokens programmatically:**
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `POST` | `/api/api-tokens/` | Create a new token |
+| `GET` | `/api/api-tokens/` | List all your tokens |
+| `DELETE` | `/api/api-tokens/{id}` | Revoke a token |
+
+### Session Authentication
+
+Browser-based users authenticate via OAuth or local login. Session cookies are set
+automatically and used for subsequent requests:
 
 ```bash
 curl -X GET "http://<your-docuelevate-instance>/api/files" \
@@ -1218,6 +1252,135 @@ Send a processed file to Google Drive.
 }
 ```
 
+## Integrations
+
+Manage per-user integrations (sources and destinations). All endpoints require authentication and are scoped to the current user's integrations. Subscription-tier quota enforcement is applied on creation.
+
+### Quota Enforcement
+
+When creating an integration, the API checks the user's subscription tier:
+
+| Tier | Storage Destinations | IMAP Sources |
+|------|---------------------|--------------|
+| **Free** | 1 | 0 |
+| **Starter** | 2 | 1 |
+| **Professional** | 5 | 3 |
+| **Power** | 10 | Unlimited |
+
+Exceeding a quota returns HTTP 403 with a descriptive error message.
+
+### GET /api/integrations/
+
+List all integrations for the current user. Supports optional query-string filters.
+
+**Query Parameters:**
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `direction` | string | Filter by `SOURCE` or `DESTINATION` |
+| `integration_type` | string | Filter by type (e.g. `IMAP`, `S3`, `DROPBOX`) |
+
+**Response (200):**
+
+```json
+[
+  {
+    "id": 1,
+    "owner_id": "user@example.com",
+    "direction": "DESTINATION",
+    "integration_type": "S3",
+    "name": "Archive Bucket",
+    "config": {"bucket": "my-bucket", "region": "us-east-1"},
+    "has_credentials": true,
+    "is_active": true,
+    "last_used_at": null,
+    "last_error": null,
+    "created_at": "2025-01-01T00:00:00",
+    "updated_at": "2025-01-01T00:00:00"
+  }
+]
+```
+
+### POST /api/integrations/
+
+Create a new integration. Quota is enforced before creation.
+
+**Request:**
+
+```json
+{
+  "direction": "DESTINATION",
+  "integration_type": "S3",
+  "name": "Archive Bucket",
+  "config": {"bucket": "my-bucket", "region": "us-east-1"},
+  "credentials": {"access_key_id": "AKIA...", "secret_access_key": "..."},
+  "is_active": true
+}
+```
+
+**Response (201):** The created integration (same shape as list response).
+
+**Response (403):** Quota exceeded.
+
+```json
+{
+  "detail": "You have reached your plan limit of 1 storage destination(s). Please remove an existing destination or upgrade your plan."
+}
+```
+
+### PUT /api/integrations/{id}
+
+Update an existing integration. Only provided fields are changed.
+
+### DELETE /api/integrations/{id}
+
+Delete an integration permanently. Returns 204 on success.
+
+### POST /api/integrations/test
+
+Test an integration connection without saving. Useful for "Test connection" UI buttons.
+
+**Request:**
+
+```json
+{
+  "integration_type": "IMAP",
+  "config": {"host": "imap.gmail.com", "port": 993, "username": "user@example.com", "use_ssl": true},
+  "credentials": {"password": "app-password"}
+}
+```
+
+**Response (200):**
+
+```json
+{"success": true, "message": "IMAP connection successful"}
+```
+
+Supported connection tests: `IMAP`, `S3`, `WEBDAV`, `NEXTCLOUD`. Other types return a message that testing is not yet supported.
+
+### GET /api/integrations/quota/
+
+Get the current user's integration quota usage.
+
+**Response (200):**
+
+```json
+{
+  "tier_id": "starter",
+  "tier_name": "Starter",
+  "destinations": {
+    "current_count": 1,
+    "max_allowed": 2,
+    "can_add": true
+  },
+  "sources": {
+    "current_count": 0,
+    "max_allowed": 1,
+    "can_add": true
+  }
+}
+```
+
 ## Webhooks
 
 Manage webhook configurations for notifying external systems when document events occur. All webhook endpoints require admin access.
@@ -1625,12 +1788,27 @@ Returns the catalogue of built-in step types.
     "label": "OCR Processing",
     "description": "Extract text using Azure Document Intelligence or local Tesseract.",
     "config_schema": {
-      "force_cloud_ocr": { "type": "boolean", "default": false }
+      "force_cloud_ocr": { "type": "boolean", "default": false },
+      "ocr_language": {
+        "type": "select",
+        "default": "auto",
+        "description": "Language(s) for OCR. Overrides the global setting for Tesseract/EasyOCR. Azure/Mistral auto-detect.",
+        "options": [
+          { "value": "auto", "label": "Auto (use system default)" },
+          { "value": "eng",  "label": "English" },
+          { "value": "deu",  "label": "German" },
+          { "value": "fra",  "label": "French" },
+          { "value": "spa",  "label": "Spanish" },
+          "..."
+        ]
+      }
     }
   }
 }
 ```
 
+The `ocr_language` field accepts Tesseract language codes (e.g. `"eng"`, `"deu"`, `"eng+deu"` for multi-language) or `"auto"` to fall back to the global system setting.  The full list of 28 supported language codes is returned by the step-types endpoint.
+
 ### List pipelines
 
 ```bash
@@ -1723,12 +1901,23 @@ Content-Type: application/json
 
 {
   "step_type": "ocr",
-  "label": "Cloud OCR",
-  "config": { "force_cloud_ocr": true },
+  "label": "German OCR",
+  "config": { "force_cloud_ocr": false, "ocr_language": "deu" },
   "enabled": true
 }
 ```
 
+Multi-language (Tesseract `+`-separated codes):
+
+```bash
+{
+  "step_type": "ocr",
+  "config": { "ocr_language": "eng+deu" }
+}
+```
+
+Use `"ocr_language": "auto"` (or omit the field) to fall back to the global system language setting.
+
 ### Update step
 
 ```bash
@@ -1771,6 +1960,448 @@ Pass no `pipeline_id` query parameter (or omit it) to clear the assignment.
 ```
 
 
+## Routing Rules
+
+Routing rules let you conditionally assign documents to different pipelines
+based on file properties such as type, size, filename, or AI-extracted
+metadata.  Rules are evaluated in **position order** (lowest first); the first
+rule that matches wins.  If no rule matches, the system falls back to the
+owner's (or global) default pipeline.
+
+### Supported operators and fields
+
+```bash
+GET /api/routing-rules/operators
+```
+
+Returns the catalogue of valid operators and built-in fields so UIs can
+populate dropdowns without hard-coding values.
+
+**Response (200):**
+```json
+{
+  "operators": ["contains", "equals", "gt", "gte", "lt", "lte", "not_contains", "not_equals", "regex"],
+  "builtin_fields": ["category", "document_type", "file_type", "filename", "size"],
+  "metadata_prefix": "metadata."
+}
+```
+
+> **Tip:** For AI metadata fields use the `metadata.` prefix, e.g.
+> `metadata.sender`, `metadata.amount`.
+
+### List routing rules
+
+```bash
+GET /api/routing-rules
+```
+
+Returns the current user's rules **plus** any system-wide rules
+(`owner_id = null`), ordered by position.
+
+**Response (200):**
+```json
+[
+  {
+    "id": 1,
+    "owner_id": "alice",
+    "name": "Route invoices",
+    "position": 0,
+    "field": "document_type",
+    "operator": "equals",
+    "value": "Invoice",
+    "target_pipeline_id": 3,
+    "is_active": true,
+    "created_at": "2026-03-09T12:00:00+00:00",
+    "updated_at": "2026-03-09T12:00:00+00:00"
+  }
+]
+```
+
+### Create routing rule
+
+```bash
+POST /api/routing-rules
+Content-Type: application/json
+
+{
+  "name": "Route invoices",
+  "field": "document_type",
+  "operator": "equals",
+  "value": "Invoice",
+  "target_pipeline_id": 3
+}
+```
+
+Optional fields: `position` (auto-assigned if omitted), `is_active` (default `true`).
+
+**Response (201 Created):** The created rule object.
+
+### Get routing rule
+
+```bash
+GET /api/routing-rules/{rule_id}
+```
+
+**Response (200):** A single rule object.
+
+### Update routing rule
+
+```bash
+PUT /api/routing-rules/{rule_id}
+Content-Type: application/json
+
+{ "name": "Renamed rule", "operator": "contains", "is_active": false }
+```
+
+Only the supplied fields are updated.
+
+**Response (200):** The updated rule object.
+
+### Delete routing rule
+
+```bash
+DELETE /api/routing-rules/{rule_id}
+```
+
+Returns **204 No Content**.
+
+### Reorder routing rules
+
+```bash
+PUT /api/routing-rules/reorder
+Content-Type: application/json
+
+{ "rule_ids": [3, 1, 2] }
+```
+
+Provide the complete ordered list of your rule IDs.  Positions are reassigned
+0, 1, 2, … in the given order.
+
+### Evaluate rules (dry run)
+
+```bash
+POST /api/routing-rules/evaluate
+Content-Type: application/json
+
+{
+  "file_type": "application/pdf",
+  "filename": "invoice_2024.pdf",
+  "size": 204800,
+  "document_type": "Invoice",
+  "metadata": { "sender": "Acme Corp" }
+}
+```
+
+Tests which rule (if any) would match the given properties **without**
+actually routing a document.
+
+**Response (200) – match found:**
+```json
+{
+  "matched": true,
+  "rule": { "id": 1, "name": "Route invoices", "..." : "..." },
+  "target_pipeline": { "id": 3, "name": "Invoice Pipeline", "is_active": true }
+}
+```
+
+**Response (200) – no match:**
+```json
+{
+  "matched": false,
+  "rule": null,
+  "target_pipeline": null
+}
+```
+
+
+## API Tokens
+
+Personal API tokens allow programmatic access to the DocuElevate API without
+session cookies. Tokens are ideal for CI/CD pipelines, webhook integrations,
+and automation scripts.
+
+Each token is prefixed with `de_` for easy identification. Only a SHA-256 hash
+is stored server-side; the plaintext is returned exactly once at creation time.
+
+Usage tracking records when each token was last used and from which IP address.
+
+### POST /api/api-tokens/
+
+Create a new API token.
+
+**Request:**
+```json
+{
+  "name": "CI Pipeline"
+}
+```
+
+**Response (201 Created):**
+```json
+{
+  "id": 1,
+  "name": "CI Pipeline",
+  "token_prefix": "de_Ab3xY7kL",
+  "token": "de_Ab3xY7kLmN9pQrStUvWxYz0123456789abcdef",
+  "is_active": true,
+  "last_used_at": null,
+  "last_used_ip": null,
+  "created_at": "2026-03-08T12:00:00Z",
+  "revoked_at": null
+}
+```
+
+> **Important:** The `token` field is only included in the creation response.
+> Copy it immediately — it will not be shown again.
+
+### GET /api/api-tokens/
+
+List all tokens for the authenticated user. The full token value is never included.
+
+**Response (200):**
+```json
+[
+  {
+    "id": 1,
+    "name": "CI Pipeline",
+    "token_prefix": "de_Ab3xY7kL",
+    "is_active": true,
+    "last_used_at": "2026-03-08T15:30:00Z",
+    "last_used_ip": "203.0.113.42",
+    "created_at": "2026-03-08T12:00:00Z",
+    "revoked_at": null
+  }
+]
+```
+
+### DELETE /api/api-tokens/{token_id}
+
+Revoke a token. The token is soft-deleted (kept for audit purposes) and can no
+longer be used for authentication.
+
+**Response (200):**
+```json
+{
+  "detail": "Token revoked"
+}
+```
+
+### Using API Tokens
+
+Include the token in the `Authorization` header of any API request:
+
+```bash
+# Upload a document
+curl -X POST "http://your-instance/api/files/ui-upload" \
+  -H "Authorization: Bearer de_your_token_here" \
+  -F "file=@/path/to/document.pdf"
+
+# List files
+curl -X GET "http://your-instance/api/files" \
+  -H "Authorization: Bearer de_your_token_here"
+```
+
+**Python example:**
+```python
+import requests
+
+response = requests.post(
+    "http://your-instance/api/files/ui-upload",
+    headers={"Authorization": "Bearer de_your_token_here"},
+    files={"file": open("document.pdf", "rb")},
+)
+print(response.json())
+```
+
+
 ## Further Assistance
 
 For additional help with the API, please contact our support team or refer to the [Development Guide](../CONTRIBUTING.md).
+
+## Mobile App API
+
+The mobile API provides endpoints used by the native iOS and Android app.  All endpoints require authentication (Bearer token or active session cookie).
+
+For full mobile app documentation see [MobileApp.md](./MobileApp.md).
+
+### POST /api/mobile/generate-token
+
+Exchange an active web session for a long-lived API token scoped to the mobile app.
+
+**Request:**
+```json
+{ "device_name": "John's iPhone" }
+```
+
+**Response (201 Created):**
+```json
+{
+  "token": "de_AbCdEfGhIjKl...",
+  "token_id": 42,
+  "name": "Mobile App – John's iPhone",
+  "created_at": "2026-03-10T09:30:00Z"
+}
+```
+
+> The `token` is shown **once only**.
+
+### POST /api/mobile/register-device
+
+Register an Expo push token to receive push notifications.
+
+**Request:**
+```json
+{
+  "push_token": "ExponentPushToken[xxxxxx]",
+  "device_name": "John's iPhone",
+  "platform": "ios"
+}
+```
+
+**Response (201 Created):** Device record with `id`, `platform`, `is_active`, `created_at`.
+
+### GET /api/mobile/devices
+
+List all registered push-notification devices for the current user.
+
+**Response (200 OK):** Array of device records.
+
+### DELETE /api/mobile/devices/{device_id}
+
+Deactivate a push-notification device.  The device will no longer receive push notifications.
+
+**Response (204 No Content)**
+
+### GET /api/mobile/whoami
+
+Return basic profile information for the authenticated user.
+
+**Response (200 OK):**
+```json
+{
+  "owner_id": "john@example.com",
+  "display_name": "John Doe",
+  "email": "john@example.com",
+  "avatar_url": "https://www.gravatar.com/avatar/...",
+  "is_admin": false
+}
+```
+
+---
+
+## GraphQL API
+
+DocuElevate exposes a GraphQL API at `/graphql` alongside the REST API.  It
+supports flexible queries with field selection, making it ideal for dashboards
+and integrations that only need a subset of the available data.
+
+### Endpoint
+
+| Method | URL | Description |
+|--------|-----|-------------|
+| `POST` | `/graphql` | Execute a GraphQL query or mutation |
+| `GET`  | `/graphql` | Open the GraphiQL interactive playground |
+
+### Authentication
+
+The GraphQL endpoint honours the same authentication rules as the REST API:
+
+- **`AUTH_ENABLED=False`** (default, single-user mode): all queries are
+  allowed without credentials.
+- **`AUTH_ENABLED=True`** (multi-user mode): a valid session cookie **or**
+  an `Authorization: Bearer <token>` API token is required.  Admin-only
+  queries (settings, users) additionally require the `is_admin` flag.
+
+### Available Queries
+
+| Field | Returns | Notes |
+|-------|---------|-------|
+| `documents(ownerId, limit, offset)` | `[DocumentType]` | Paginated list of documents |
+| `document(id)` | `DocumentType` | Single document by primary key |
+| `pipelines(ownerId, limit, offset)` | `[PipelineType]` | Paginated list of pipelines with steps |
+| `pipeline(id)` | `PipelineType` | Single pipeline by primary key |
+| `settings(limit, offset)` | `[SettingType]` | Non-sensitive app settings (**admin only**) |
+| `users(limit, offset)` | `[UserType]` | User profiles (**admin only**) |
+| `user(userId)` | `UserType` | Single user profile (**admin only**) |
+
+> **Note:** Sensitive configuration keys (API secrets, passwords, tokens) are
+> automatically excluded from the `settings` query regardless of the caller's
+> privilege level.
+
+### GraphiQL Playground
+
+Navigate to `http://<your-instance>/graphql` in a browser to open the
+interactive GraphiQL IDE, which provides schema documentation, auto-complete,
+and the ability to run queries directly.
+
+### Example Queries
+
+**List recent documents:**
+```graphql
+{
+  documents(limit: 5) {
+    id
+    originalFilename
+    mimeType
+    fileSize
+    documentTitle
+    createdAt
+  }
+}
+```
+
+**Fetch a pipeline with its steps:**
+```graphql
+{
+  pipeline(id: 1) {
+    id
+    name
+    description
+    isDefault
+    isActive
+    steps {
+      position
+      stepType
+      label
+      enabled
+    }
+  }
+}
+```
+
+**List application settings (admin only):**
+```graphql
+{
+  settings {
+    key
+    value
+    updatedAt
+  }
+}
+```
+
+**List user profiles (admin only):**
+```graphql
+{
+  users(limit: 10) {
+    userId
+    displayName
+    subscriptionTier
+    isBlocked
+  }
+}
+```
+
+**Using variables:**
+```graphql
+query GetDocument($id: Int!) {
+  document(id: $id) {
+    id
+    originalFilename
+    documentTitle
+    isDuplicate
+    ocrQualityScore
+  }
+}
+```
+Variables: `{ "id": 42 }`

docs/AuthenticationSetup.md

@@ -20,10 +20,11 @@ For a complete list of configuration options, see the [Configuration Guide](Conf
 
 ## Authentication Methods
 
-DocuElevate supports two primary authentication methods:
+DocuElevate supports multiple authentication methods that can be used independently or together:
 
 1. **Simple Authentication** - Basic username/password authentication managed by DocuElevate
 2. **OpenID Connect** - Integration with identity providers like Authentik, Keycloak, or Auth0
+3. **Social Login** - Sign in with Google, Microsoft, Apple, or Dropbox accounts (see [Social Login Setup Guide](SocialLoginSetup.md))
 
 ## Session Security
 
@@ -189,3 +190,11 @@ If you encounter issues with authentication:
    - For most providers, you can visit the `/.well-known/openid-configuration` endpoint to verify their settings
 
 For more general configuration issues, see the [Configuration Troubleshooting Guide](ConfigurationTroubleshooting.md).
+
+## Social Login
+
+DocuElevate supports social login with Google, Microsoft, Apple, and Dropbox. Social login allows users to authenticate using their existing accounts with these providers, without needing a separate DocuElevate password.
+
+Social login can be used alongside any other authentication method (simple auth, OIDC, local signup). Each social provider is independently configured.
+
+For detailed setup instructions, prerequisites, and provider-specific configuration, see the **[Social Login Setup Guide](SocialLoginSetup.md)**.

docs/BillingSetup.md

@@ -8,7 +8,8 @@ This guide covers how to configure Stripe billing and local user sign-up in Docu
 - [Stripe Billing Integration](#stripe-billing-integration)
   - [Prerequisites](#prerequisites)
   - [Configuration](#configuration)
-  - [Setting Up Plans](#setting-up-plans)
+  - [Stripe Setup Wizard (recommended)](#stripe-setup-wizard-recommended)
+  - [Setting Up Plans Manually](#setting-up-plans-manually)
   - [Webhook Configuration](#webhook-configuration)
   - [Billing Flows](#billing-flows)
 - [Compliance Notes](#compliance-notes)
@@ -75,7 +76,6 @@ DocuElevate integrates with [Stripe](https://stripe.com) to handle subscription
 ### Prerequisites
 
 - A Stripe account (sign up at [stripe.com](https://stripe.com))
-- Products and prices created in the Stripe Dashboard for each paid plan
 - A publicly reachable webhook endpoint (or use [Stripe CLI](https://stripe.com/docs/stripe-cli) for local testing)
 
 ### Configuration
@@ -90,16 +90,75 @@ STRIPE_CANCEL_URL=https://app.example.com/pricing               # Optional overr
 
 > **Security:** Never commit your Stripe secret key. Store it in your environment or secrets manager.
 
-### Setting Up Plans
+### Stripe Setup Wizard (recommended)
 
-After starting DocuElevate, go to **Admin → Plans** to configure each plan:
+DocuElevate includes a built-in **Stripe Setup Wizard** at `/admin/stripe-wizard` that guides you through the complete setup in three steps:
 
-1. Open the **Plan Designer** for a paid tier (e.g. Starter, Professional).
-2. Enter the **Stripe Price ID (monthly)** from your Stripe Dashboard (e.g. `price_1OtAbc...`).
-3. Optionally enter the **Stripe Price ID (yearly)** for annual billing.
-4. Save the plan.
+1. **Verify API Keys** — checks that your Stripe secret key is configured and the connection to Stripe is working.
+2. **Sync Plans to Stripe** — automatically creates Stripe **Products** and **Prices** for every paid plan defined in DocuElevate, then stores the resulting `price_id` values back in the database. Free plans are skipped; plans that already have a price ID are left unchanged.
+3. **Configure Webhook** — shows the exact webhook endpoint URL to register in the Stripe Dashboard and which events to subscribe to.
 
-Stripe Price IDs look like `price_1OtAbcDefGhIjKlMnOpQrSt`. Find them in **Products** in your Stripe Dashboard.
+You can also reach the wizard from the **Admin → Plans** page via the **Stripe Setup** button.
+
+#### Auto-sync API
+
+The sync step is also available as an API endpoint for automation:
+
+```bash
+curl -X POST https://your-app.example.com/api/billing/stripe/sync-plans \
+  -H "Cookie: <admin-session-cookie>"
+```
+
+Response:
+
+```json
+{
+  "results": [
+    {
+      "plan_id": "starter",
+      "name": "Starter",
+      "status": "created",
+      "stripe_price_id_monthly": "price_1OtAbc...",
+      "stripe_price_id_yearly": "price_1OtDef..."
+    },
+    {
+      "plan_id": "free",
+      "name": "Free",
+      "status": "skipped_free"
+    }
+  ]
+}
+```
+
+Possible `status` values:
+
+| Status | Meaning |
+|--------|---------|
+| `created` | Stripe product and price(s) were created and saved |
+| `already_synced` | Plan already had a price ID — no changes made |
+| `skipped_free` | Free plan (price is $0) — no Stripe price needed |
+| `error` | Stripe API call failed — `detail` contains the error message |
+
+#### Stripe connection status API
+
+```bash
+curl https://your-app.example.com/api/billing/stripe/status
+```
+
+Returns connection health, API mode (test/live), webhook secret status, and per-plan sync status.
+
+### Setting Up Plans Manually
+
+If you prefer to enter price IDs yourself rather than using the wizard:
+
+1. Create **Products** and **Prices** in the [Stripe Dashboard](https://dashboard.stripe.com/products).
+2. Go to **Admin → Plans** in DocuElevate and click the **Edit** (pencil) button for a paid plan.
+3. Scroll to the **Stripe Integration** section in the plan editor.
+4. Enter the **Stripe Price ID (monthly)** (e.g. `price_1OtAbc...`).
+5. Optionally enter the **Stripe Price ID (yearly)** for annual billing.
+6. Save the plan.
+
+> **Stable link:** DocuElevate stores the Stripe `customer_id` in the `UserProfile.stripe_customer_id` column and matches it on every webhook event. This is the stable link between Stripe billing profiles and DocuElevate accounts. It is set automatically when a user completes their first checkout.
 
 ### Webhook Configuration
 
@@ -110,6 +169,7 @@ Stripe webhooks allow DocuElevate to sync subscription status in real time.
 1. Go to **Developers → Webhooks** in the Stripe Dashboard.
 2. Click **Add endpoint**.
 3. Set the endpoint URL to: `https://your-app-domain.com/api/billing/webhook`
+   (The Stripe Setup Wizard shows the exact URL for your deployment.)
 4. Select the following events:
    - `checkout.session.completed`
    - `customer.subscription.updated`
@@ -126,8 +186,10 @@ stripe login
 # Forward webhooks to your local server
 stripe listen --forward-to http://localhost:8000/api/billing/webhook
 
-# Trigger a test event
+# Trigger test events
 stripe trigger checkout.session.completed
+stripe trigger customer.subscription.updated
+stripe trigger customer.subscription.deleted
 ```
 
 ### Billing Flows
@@ -139,7 +201,7 @@ stripe trigger checkout.session.completed
 3. DocuElevate calls `POST /api/billing/create-checkout-session`.
 4. User is redirected to Stripe Checkout.
 5. After payment, Stripe fires `checkout.session.completed`.
-6. DocuElevate webhook handler activates the subscription tier.
+6. DocuElevate webhook handler activates the subscription tier and stores the Stripe `customer_id`.
 7. User is redirected to `/api/billing/success`.
 
 #### Manage or cancel subscription

docs/CIToolsGuide.md

@@ -128,18 +128,112 @@ pytest -m integration
 #### Codecov - Coverage Tracking
 
 **What it does:**
-- Visualizes test coverage trends
-- Comments on PRs with coverage changes
-- Tracks coverage over time
-- Provides coverage badges
+- Visualizes test coverage trends over time
+- Comments on PRs with a per-file coverage diff
+- Enforces project-level and patch-level coverage thresholds
+- Provides embeddable coverage badges
+- Annotates PR diff lines with coverage status
 
 **Why we chose it:**
-- Free for open-source
-- Excellent PR integration
-- Clear coverage visualization
-- Industry standard
+- Free for open-source projects
+- Excellent GitHub PR integration
+- Clear coverage visualization and trend graphs
+- Industry standard with broad toolchain support
 
-**Configuration:** Integrated in `.github/workflows/tests.yaml`
+**Configuration files:**
+- `codecov.yml` — repository-level settings (thresholds, flags, PR comments)
+- `.github/workflows/ci.yml` (`run-tests` job) — uploads `coverage.xml` via `codecov/codecov-action@v5`
+- `pyproject.toml` (`[tool.coverage.*]`) — what pytest-cov measures and omits
+
+##### Initial Setup (Repository Admins)
+
+1. **Sign in to Codecov** at <https://app.codecov.io> using your GitHub account.
+2. **Add the repository**: click *Add new repository* and select `DocuElevate`.
+3. **Copy the upload token** shown on the repository settings page.
+4. **Store the token as a GitHub Actions secret**:
+   - Navigate to *Settings → Secrets and variables → Actions → New repository secret*
+   - Name: `CODECOV_TOKEN`
+   - Value: paste the token copied in step 3
+5. **Push a commit** to trigger CI. The `run-tests` job will upload `coverage.xml` and Codecov will begin reporting.
+
+##### Adjusting Coverage Thresholds
+
+Edit `codecov.yml` at the repository root:
+
+```yaml
+coverage:
+  status:
+    project:
+      default:
+        target: 60%      # Minimum overall project coverage
+        threshold: 2%    # Allowed drop compared to base branch
+    patch:
+      default:
+        target: 70%      # Minimum coverage of lines changed in a PR
+        threshold: 5%    # Allowed slack on patch coverage
+```
+
+Commit and push the change — Codecov picks it up automatically.
+
+##### Using Coverage Flags
+
+Flags let you track unit and integration coverage separately.
+The `run-tests` job in `ci.yml` uploads a single unified report; to split it add a
+`flags` parameter to the upload step for each job:
+
+```yaml
+- name: Upload Unified Coverage to Codecov
+  uses: codecov/codecov-action@v5
+  with:
+    token: ${{ secrets.CODECOV_TOKEN }}
+    files: ./coverage.xml
+    flags: unittests       # matches flag_management in codecov.yml
+    fail_ci_if_error: true
+```
+
+The flag names (`unittests`, `integration`) are defined in `codecov.yml` under
+`flag_management.individual_flags`.
+
+##### Embedding the Coverage Badge
+
+Add the following Markdown to `README.md`, replacing `<owner>` and `<repo>`:
+
+```markdown
+<!-- Public repository (no token needed) -->
+[![codecov](https://codecov.io/gh/<owner>/<repo>/graph/badge.svg)](https://codecov.io/gh/<owner>/<repo>)
+
+<!-- Private repository (include token for badge access) -->
+[![codecov](https://codecov.io/gh/<owner>/<repo>/graph/badge.svg?token=<TOKEN>)](https://codecov.io/gh/<owner>/<repo>)
+```
+
+The exact badge snippet (with the correct token pre-filled) is shown on the Codecov repository overview page under *Settings → Badge*.
+
+##### Accessing Coverage Reports
+
+| Where | What you see |
+|-------|-------------|
+| Codecov dashboard (<https://app.codecov.io>) | Full report, trend graphs, file explorer |
+| GitHub PR comment | Per-file diff, overall Δ, patch coverage |
+| GitHub Checks tab | Pass/fail status for project and patch thresholds |
+| CI artifacts | `coverage.xml` (machine-readable), HTML report (human-readable) |
+
+To view the HTML report locally:
+
+```bash
+pytest --cov=app --cov-report=html
+open htmlcov/index.html
+```
+
+##### Troubleshooting
+
+| Symptom | Likely cause | Fix |
+|---------|-------------|-----|
+| "Token is invalid" upload error | `CODECOV_TOKEN` secret missing or wrong | Re-copy the token from Codecov settings and update the secret |
+| No PR comment posted | Repository is private and token not set | Set `CODECOV_TOKEN` — required for private repos |
+| Coverage always 0% | `coverage.xml` not generated | Confirm pytest runs with `--cov=app --cov-report=xml:coverage.xml` |
+| CI fails with "coverage decreased" | Patch coverage dropped below threshold | Improve test coverage for the changed lines, or adjust `threshold` in `codecov.yml` |
+| Codecov not picking up `codecov.yml` | YAML syntax error | Validate the file locally with `python3 -c "import yaml; yaml.safe_load(open('codecov.yml'))"`, or use the web validator at the Codecov dashboard (*Settings → YAML*) |
+| Flags not appearing in dashboard | Flag name mismatch | Ensure the `flags:` value in the upload step matches an entry in `codecov.yml` |
 
 #### Pre-commit - Local Quality Gates
 

docs/CLIGuide.md

@@ -0,0 +1,348 @@
+# DocuElevate CLI Guide
+
+The `docuelevate` command-line tool lets you interact with your DocuElevate instance
+from a terminal, shell script, or CI/CD pipeline.  It is ideal for:
+
+- Batch uploads from a script or cron job
+- Downloading processed documents programmatically
+- Searching documents in automation workflows
+- Rotating API tokens safely without touching the web UI
+
+---
+
+## Installation
+
+The CLI is included in the standard DocuElevate package.  After installing the
+Python package (e.g. inside the Docker image or a virtualenv), the `docuelevate`
+command is available:
+
+```bash
+pip install docuelevate          # or: pip install -e . inside the repo
+docuelevate --help
+```
+
+---
+
+## Authentication
+
+All commands require an API token.  Create one at `/api-tokens` in the web UI,
+or with the `docuelevate token create` command itself.
+
+Provide the token in either of two ways:
+
+| Method | Example |
+|--------|---------|
+| `--token` flag | `docuelevate --token de_xxxxx list` |
+| Environment variable | `export DOCUELEVATE_API_TOKEN=de_xxxxx` |
+
+The environment variable is recommended for scripts so that secrets never appear
+in shell history or process listings.
+
+---
+
+## Configuration
+
+| Option / Variable | Default | Description |
+|-------------------|---------|-------------|
+| `--url` / `DOCUELEVATE_URL` | `http://localhost:8000` | Base URL of the DocuElevate instance |
+| `--token` / `DOCUELEVATE_API_TOKEN` | _(none)_ | API token for authentication |
+| `--format` | `table` | Output format: `table` (human-readable) or `json` (pipe-friendly) |
+| `--timeout` / `DOCUELEVATE_TIMEOUT` | `60` | HTTP request timeout in seconds |
+
+Setting both `DOCUELEVATE_URL` and `DOCUELEVATE_API_TOKEN` in your environment
+removes the need for flags on every invocation:
+
+```bash
+export DOCUELEVATE_URL=https://docs.example.com
+export DOCUELEVATE_API_TOKEN=de_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+docuelevate list
+```
+
+---
+
+## Commands
+
+### `list` — List documents
+
+```
+docuelevate [OPTIONS] list [OPTIONS]
+```
+
+Returns a paginated list of documents stored in DocuElevate.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--page` | `1` | Page number |
+| `--per-page` | `25` | Items per page (max 200) |
+| `--search` | — | Filter by filename substring |
+| `--mime-type` | — | Filter by MIME type (e.g. `application/pdf`) |
+| `--status` | — | Filter by status: `pending`, `processing`, `completed`, `failed` |
+| `--sort-by` | `created_at` | Sort field |
+| `--sort-order` | `desc` | Sort direction: `asc` or `desc` |
+
+**Examples:**
+
+```bash
+# Human-readable table
+docuelevate list
+
+# Only completed PDFs
+docuelevate list --status completed --mime-type application/pdf
+
+# Pipe filenames to another command
+docuelevate --format json list | jq -r '.[].filename'
+
+# Search by filename
+docuelevate list --search invoice
+```
+
+---
+
+### `upload` — Upload files
+
+```
+docuelevate [OPTIONS] upload [OPTIONS] FILES...
+```
+
+Uploads one or more local files to DocuElevate for processing.  Multiple file
+paths (or shell globs) can be provided for batch uploads.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--batch-size` | `5` | Maximum uploads before reporting progress |
+
+**Examples:**
+
+```bash
+# Upload a single file
+docuelevate upload report.pdf
+
+# Batch upload — all PDFs in a folder
+docuelevate upload /scans/*.pdf
+
+# Upload multiple files explicitly
+docuelevate upload invoice.pdf contract.pdf receipt.png
+
+# JSON output to capture task IDs
+docuelevate --format json upload *.pdf | jq '.[].task_id'
+```
+
+---
+
+### `download` — Download a file
+
+```
+docuelevate [OPTIONS] download [OPTIONS] FILE_ID
+```
+
+Downloads a processed (or original) file by its numeric ID.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `-o` / `--output` | _(server filename)_ | Destination file path |
+| `--version` | `processed` | `processed` or `original` |
+
+**Examples:**
+
+```bash
+# Download processed version of file #42
+docuelevate download 42
+
+# Save to a specific path
+docuelevate download 42 -o /tmp/invoice.pdf
+
+# Download the original (unprocessed) upload
+docuelevate download 42 --version original -o original.pdf
+```
+
+---
+
+### `search` — Full-text search
+
+```
+docuelevate [OPTIONS] search [OPTIONS] QUERY
+```
+
+Searches across document text, filenames, tags, and metadata using Meilisearch.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--mime-type` | — | Filter by MIME type |
+| `--document-type` | — | Filter by document type (e.g. `Invoice`) |
+| `--tags` | — | Filter by tag |
+| `--language` | — | Filter by language code (e.g. `en`, `de`) |
+| `--page` | `1` | Page number |
+| `--per-page` | `20` | Results per page (max 100) |
+
+**Examples:**
+
+```bash
+# Simple search
+docuelevate search "amazon invoice"
+
+# With filters
+docuelevate search "contract" --document-type Contract --language en
+
+# Pipe file IDs to the download command
+docuelevate --format json search "Q1 report" | jq -r '.[].file_id'
+```
+
+---
+
+### `token` — Manage API tokens
+
+The `token` sub-group provides commands to create, list, and revoke personal API
+tokens — enabling **token rotation** without logging into the web UI.
+
+#### `token create`
+
+```
+docuelevate token create NAME
+```
+
+Creates a new token.  The full token value is printed exactly once — store it
+securely.
+
+```bash
+# Create a new token
+docuelevate --token de_existing token create "CI Pipeline"
+
+# Capture the new token value in a script
+NEW_TOKEN=$(docuelevate --format json --token de_existing token create "Rotation" \
+            | jq -r '.token')
+```
+
+#### `token list`
+
+```
+docuelevate token list
+```
+
+Lists all your tokens (active and revoked).
+
+```bash
+docuelevate token list
+
+# JSON for scripting
+docuelevate --format json token list | jq '.[] | select(.is_active) | .id'
+```
+
+#### `token revoke`
+
+```
+docuelevate token revoke [--yes] TOKEN_ID
+```
+
+Revokes a token by its numeric ID.  The token is immediately invalidated.
+
+| Option | Description |
+|--------|-------------|
+| `--yes` / `-y` | Skip confirmation prompt |
+
+```bash
+# Interactive confirmation
+docuelevate token revoke 3
+
+# Non-interactive (for scripts)
+docuelevate token revoke 3 --yes
+```
+
+---
+
+## Token Rotation
+
+Rotate an API token safely without any downtime:
+
+```bash
+# 1. Create the replacement token
+NEW_TOKEN=$(docuelevate --format json --token "$OLD_TOKEN" \
+            token create "Rotated $(date +%Y-%m-%d)" | jq -r '.token')
+
+# 2. Update consumers to use NEW_TOKEN, then revoke the old one
+OLD_ID=$(docuelevate --format json --token "$OLD_TOKEN" token list \
+         | jq '.[] | select(.is_active and (.token_prefix == "de_old_prefix")) | .id')
+docuelevate --token "$NEW_TOKEN" token revoke --yes "$OLD_ID"
+```
+
+---
+
+## Output Formats
+
+### Table (default)
+
+Human-readable, suitable for terminal use:
+
+```
+ID  FILENAME           SIZE   STATUS     CREATED_AT
+--  -----------------  -----  ---------  -------------------
+42  invoice_2026.pdf   98304  completed  2026-03-01T10:30:00
+43  contract.pdf       51200  pending    2026-03-01T11:00:00
+```
+
+### JSON (`--format json`)
+
+Machine-readable, pipe-friendly, suitable for `jq`, shell scripts, and CI:
+
+```json
+[
+  {
+    "id": 42,
+    "filename": "invoice_2026.pdf",
+    "size": 98304,
+    "status": "completed",
+    "created_at": "2026-03-01T10:30:00"
+  }
+]
+```
+
+---
+
+## Pipe-Friendly Examples
+
+```bash
+# Download all completed PDFs in a folder
+docuelevate --format json list --status completed --mime-type application/pdf \
+  | jq -r '.[].id' \
+  | xargs -I {} docuelevate download {} -o /backup/{}.pdf
+
+# Count documents by status
+docuelevate --format json list --per-page 200 \
+  | jq 'group_by(.status) | map({status: .[0].status, count: length})'
+
+# Search and get filenames
+docuelevate --format json search "2026 invoice" \
+  | jq -r '.[].filename'
+
+# Batch upload all new files and capture task IDs
+find /inbox -name "*.pdf" | xargs docuelevate upload \
+  && echo "All uploaded"
+```
+
+---
+
+## Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Success |
+| `1` | One or more uploads failed (partial failure) |
+| `2` | Invalid options or arguments |
+| other | Fatal error (network, API, authentication) |
+
+---
+
+## Environment Variables Reference
+
+| Variable | Description |
+|----------|-------------|
+| `DOCUELEVATE_URL` | Base URL of the DocuElevate instance |
+| `DOCUELEVATE_API_TOKEN` | Personal API token (`de_…`) |
+| `DOCUELEVATE_TIMEOUT` | HTTP request timeout in seconds (default: 60) |
+
+---
+
+## See Also
+
+- [API Documentation](./API.md) — full REST API reference
+- [User Guide](./UserGuide.md) — web UI guide including API token management
+- [Configuration Guide](./ConfigurationGuide.md) — server-side configuration

docs/ComplianceGuide.md

@@ -0,0 +1,190 @@
+# Compliance Templates Guide
+
+DocuElevate includes pre-built compliance templates for **GDPR**, **HIPAA**, and **SOC 2** that help you configure your instance to meet regulatory requirements. This guide covers how to use the compliance dashboard, apply templates, and monitor your compliance status.
+
+## Overview
+
+The compliance templates feature provides:
+
+- **Pre-built configurations** for GDPR, HIPAA, and SOC 2
+- **One-click apply** to configure all required settings at once
+- **Compliance status dashboard** to monitor your regulatory posture
+- **Individual check results** showing which settings are compliant and which need attention
+
+## Accessing the Dashboard
+
+The compliance dashboard is available to **admin users only**.
+
+1. Log in as an administrator
+2. Click **Admin** in the navigation bar
+3. Select **Compliance** from the dropdown menu
+
+Or navigate directly to: `/admin/compliance`
+
+## Available Templates
+
+### GDPR (General Data Protection Regulation)
+
+The European Union regulation for data protection and privacy. The GDPR template enforces:
+
+| Setting | Value | Purpose |
+|---------|-------|---------|
+| `AUTH_ENABLED` | `True` | Controls access to personal data |
+| `SENTRY_SEND_DEFAULT_PII` | `False` | Prevents PII leaking to external services |
+| `SECURITY_HEADERS_ENABLED` | `True` | Protects against common web vulnerabilities |
+| `SECURITY_HEADER_HSTS_ENABLED` | `True` | Ensures encrypted connections |
+| `SECURITY_HEADER_CSP_ENABLED` | `True` | Prevents XSS and injection attacks |
+| `SECURITY_HEADER_X_FRAME_OPTIONS_ENABLED` | `True` | Prevents clickjacking |
+| `ENABLE_DEDUPLICATION` | `True` | Data minimisation — avoids duplicate storage |
+
+### HIPAA (Health Insurance Portability and Accountability Act)
+
+United States regulation for protecting health information. The HIPAA template includes all GDPR settings plus:
+
+| Setting | Value | Purpose |
+|---------|-------|---------|
+| `MULTI_USER_ENABLED` | `True` | Individual accounts for access accountability |
+
+### SOC 2 (Service Organization Control 2)
+
+Trust Service Criteria framework for service organisations. The SOC 2 template includes the same settings as HIPAA, mapped to SOC 2 Trust Service Criteria (CC6.x, PI1.x).
+
+## Applying a Template
+
+1. Navigate to the **Compliance** dashboard (`/admin/compliance`)
+2. Find the template you want to apply (GDPR, HIPAA, or SOC 2)
+3. Click **Apply Template**
+4. Confirm the action in the dialog
+5. The template settings are written to the database immediately
+
+> **Note:** Applying a template writes configuration values to the database. Some settings (e.g., security headers) may require a restart to take effect. Check the Settings page for restart indicators.
+
+## Understanding Compliance Status
+
+Each template shows one of four statuses:
+
+| Status | Badge | Meaning |
+|--------|-------|---------|
+| **Compliant** | Green | All checks are passing |
+| **Partial** | Yellow | Some checks are passing, others are not |
+| **Non-Compliant** | Red | No checks are passing |
+| **Not Applied** | Grey | Template has never been applied |
+
+### Individual Checks
+
+Click **Show Details** on any template card to see individual check results:
+
+- ✅ **Passing** — The setting matches the expected compliance value
+- ❌ **Failing** — The setting does not match; the current and expected values are shown
+
+## API Endpoints
+
+The compliance feature exposes the following API endpoints under `/api/compliance/`:
+
+### List Templates
+
+```bash
+GET /api/compliance/templates
+```
+
+Returns all compliance templates with their current status.
+
+### Get Single Template
+
+```bash
+GET /api/compliance/templates/{name}
+```
+
+Returns a single template by name (`gdpr`, `hipaa`, or `soc2`).
+
+### Apply Template
+
+```bash
+POST /api/compliance/templates/{name}/apply
+```
+
+Applies a compliance template, writing all its settings to the database.
+
+### Get Template Status
+
+```bash
+GET /api/compliance/templates/{name}/status
+```
+
+Evaluates the live compliance status of a template against current settings.
+
+**Response example:**
+
+```json
+{
+  "status": "partial",
+  "total": 7,
+  "passed": 5,
+  "failed": 2,
+  "check_results": [
+    {
+      "key": "auth_enabled",
+      "label": "Authentication enabled",
+      "description": "User authentication must be enabled to control access to personal data.",
+      "expected": "True",
+      "actual": "True",
+      "passing": true
+    }
+  ]
+}
+```
+
+### Compliance Summary
+
+```bash
+GET /api/compliance/summary
+```
+
+Returns an overall compliance summary across all templates.
+
+**Response example:**
+
+```json
+{
+  "overall_status": "partial",
+  "total_checks": 22,
+  "total_passed": 18,
+  "total_failed": 4,
+  "templates": [
+    {
+      "name": "gdpr",
+      "display_name": "GDPR (General Data Protection Regulation)",
+      "enabled": true,
+      "status": "compliant",
+      "total": 7,
+      "passed": 7,
+      "failed": 0,
+      "applied_at": "2026-03-09T12:00:00+00:00",
+      "applied_by": "admin@example.com"
+    }
+  ]
+}
+```
+
+> **Note:** All API endpoints require admin authentication.
+
+## Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `COMPLIANCE_ENABLED` | `True` | Enable the compliance templates dashboard. Set to `False` to hide the feature. |
+
+## Best Practices
+
+1. **Apply templates before going live** — Set up compliance before processing real documents
+2. **Monitor status regularly** — Check the compliance dashboard after configuration changes
+3. **Use the refresh button** — After changing settings elsewhere, refresh the compliance page to see updated status
+4. **Combine templates** — You can apply multiple templates; settings overlap is handled automatically
+5. **Review after updates** — After upgrading DocuElevate, review your compliance status as new checks may be added
+
+## Related Documentation
+
+- [Configuration Guide](./ConfigurationGuide.md) — Full list of configuration options
+- [Privacy & Compliance Guide](./PrivacyCompliance.md) — Privacy notice and GDPR compliance details
+- [Deployment Guide](./DeploymentGuide.md) — Production deployment with security best practices
+- [Security Audit](../SECURITY_AUDIT.md) — Security findings and mitigations

docs/ConfigurationGuide.md

@@ -16,6 +16,7 @@ Configuration is primarily done through environment variables specified in a `.e
 | `GOTENBERG_URL`        | Gotenberg PDF processing URL.                           | `http://gotenberg:3000`        |
 | `EXTERNAL_HOSTNAME`    | The external hostname for the application.             | `docuelevate.example.com`      |
 | `ALLOW_FILE_DELETE`    | Enable file deletion in the web interface (`true`/`false`). | `true`                      |
+| `COMPLIANCE_ENABLED`   | Enable the compliance templates dashboard (GDPR, HIPAA, SOC 2). | `true`                      |
 
 ### Batch Processing Settings
 
@@ -267,12 +268,33 @@ DocuElevate can poll a WebDAV folder for new files. It reuses the existing WebDA
 | `WEBDAV_INGEST_FOLDER`                | WebDAV folder path to poll. Uses the existing WebDAV URL and credentials.                   | *(empty)* |
 | `WEBDAV_INGEST_DELETE_AFTER_PROCESS`  | Delete files from WebDAV after they are downloaded and enqueued.                            | `false`     |
 
+#### Per-User Watch Folder Integrations
+
+In addition to system-level watch folders, each user can configure personal watch folder sources through the **Integrations** dashboard (`/integrations`). Documents ingested from per-user watch folder integrations are automatically attributed to the owning user's `owner_id`.
+
+Per-user watch folder integrations are stored in the `user_integrations` table with `integration_type='WATCH_FOLDER'` and `direction='SOURCE'`. The `config` JSON field stores:
+- `source_type` — the type of source to scan (`local`, `s3`, `dropbox`, `google_drive`, `onedrive`, `nextcloud`, `webdav`; default: `local`)
+- `folder_path` — path to the directory/folder to scan (used by local, Dropbox, OneDrive, Nextcloud, WebDAV)
+- `delete_after_process` — whether to remove source files after ingestion (default: `false`)
+
+Additional type-specific config fields:
+- **S3**: `bucket`, `region`, `prefix`, `endpoint_url`
+- **Google Drive**: `folder_id`
+- **Nextcloud / WebDAV**: `url`, `folder_path`
+
+> **Security**: Path traversal protection is enforced on local watch folder paths. Relative paths, `..` components, and symlink escapes are rejected. Cloud source types use per-user encrypted credentials instead.
+
+- Individual scan failures are handled gracefully and recorded on the integration's `last_error` field without interrupting the scanning of other integrations.
+- The scan runs alongside the system-level watch folder polling cycle.
+
 ### IMAP Email Ingestion
 
-DocuElevate can automatically pull document attachments from IMAP mailboxes — no need to forward emails manually. Configure one or two mailboxes and DocuElevate polls them on the schedule you set.
+DocuElevate can automatically pull document attachments from IMAP mailboxes — no need to forward emails manually. Configure one or two system-wide mailboxes using environment variables, and/or let each user configure their own IMAP sources via the **Integrations** dashboard.
 
 > **For HP Scanners (Scan to Email)**: If your scanner is set up to email scanned documents to a dedicated mailbox, configure that mailbox in DocuElevate using the settings below. DocuElevate will automatically retrieve the scanned PDFs from the inbox and process them. You do **not** need to configure DocuElevate as an email server — it acts as an email *client* that reads from your existing mailbox.
 
+#### System-Level IMAP Configuration
+
 | **Variable**                  | **Description**                                              | **Example**       |
 |-------------------------------|--------------------------------------------------------------|-------------------|
 | `IMAP1_HOST`                  | Hostname for first IMAP server.                             | `mail.example.com`|
@@ -282,6 +304,59 @@ DocuElevate can automatically pull document attachments from IMAP mailboxes —
 | `IMAP1_SSL`                   | Use SSL (`true`/`false`).                                   | `true`            |
 | `IMAP1_POLL_INTERVAL_MINUTES` | Frequency in minutes to poll for new mail.                  | `5`               |
 | `IMAP_READONLY_MODE`          | When `true`, fetches and processes attachments but does **not** modify the mailbox (no starring, labeling, deleting, or flag changes). Use for pre-production instances sharing a mailbox with production. Default: `false`. | `false` |
+| `IMAP_ATTACHMENT_FILTER`      | System-wide fallback for which attachment types are ingested when no ingestion profile is assigned to a mailbox. `documents_only` (default) ingests PDFs and office files only — images are skipped. `all` ingests every supported file type including images. Individual IMAP accounts can override this using ingestion profiles. | `documents_only` |
+
+#### IMAP Ingestion Profiles
+
+For fine-grained control, DocuElevate supports **Ingestion Profiles** — named configurations that let you choose exactly which file-type categories to accept from each mailbox.
+
+Each profile contains a list of enabled **categories**:
+
+| Category | Description |
+|----------|-------------|
+| `pdf` | PDF documents (`.pdf`) |
+| `office` | Microsoft Office files (Word, Excel, PowerPoint — `.docx`, `.xlsx`, `.pptx`, …) |
+| `opendocument` | LibreOffice/OpenOffice files (`.odt`, `.ods`, `.odp`, …) |
+| `text` | Plain text, CSV and RTF files (`.txt`, `.csv`, `.rtf`) |
+| `web` | HTML and Markdown files (`.html`, `.htm`, `.md`, `.markdown`) |
+| `images` | Image files (`.jpg`, `.png`, `.gif`, `.bmp`, `.tiff`, `.webp`, `.svg`) |
+
+Two built-in system profiles are seeded automatically:
+
+| Profile | Categories |
+|---------|------------|
+| **Documents Only** | pdf, office, opendocument, text, web (no images) |
+| **All Files** | All categories, including images |
+
+Users can create their own custom profiles via the **Email Ingestion** dashboard (`/imap-accounts`) by clicking the **Manage profiles** link or the **+** button next to the profile dropdown. Custom profiles are private to the creating user and can be freely edited or deleted.
+
+**API endpoints for ingestion profiles:**
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/api/imap-profiles/` | List all visible profiles (system + user's own) |
+| `POST` | `/api/imap-profiles/` | Create a new profile |
+| `GET` | `/api/imap-profiles/categories` | List available file-type categories |
+| `GET` | `/api/imap-profiles/{id}` | Get a single profile |
+| `PUT` | `/api/imap-profiles/{id}` | Update a profile (not built-in) |
+| `DELETE` | `/api/imap-profiles/{id}` | Delete a profile (not built-in) |
+
+#### Per-User IMAP Integrations
+
+In addition to system-level mailboxes, each user can configure personal IMAP sources through the **Integrations** dashboard (`/integrations`). Documents ingested from per-user IMAP integrations are automatically attributed to the owning user's `owner_id`.
+
+Per-user IMAP integrations are stored in the `user_integrations` table with `integration_type='IMAP'` and `direction='SOURCE'`. The `config` JSON field stores:
+- `host` — IMAP server hostname (required)
+- `port` — IMAP server port (default: `993`)
+- `username` — IMAP login username (required)
+- `use_ssl` — whether to use SSL/TLS (default: `true`)
+- `delete_after_process` — whether to delete emails from the mailbox after processing (default: `false`)
+- `gmail_apply_labels` — whether to apply Gmail-specific labels and stars to processed emails (default: `true`). When enabled, processed emails are starred and tagged with an "Ingested" label. Only applies to Gmail hosts.
+
+Credentials are encrypted at rest using Fernet encryption.
+
+- Individual connection failures are handled gracefully and recorded on the integration's `last_error` field without interrupting the polling of other integrations.
+- The polling loop runs every minute and processes all active IMAP sources (system-level and per-user) in sequence.
 
 ### Authentication
 
@@ -297,6 +372,28 @@ DocuElevate can automatically pull document attachments from IMAP mailboxes —
 | `AUTHENTIK_CONFIG_URL`  | Configuration URL for Authentik OpenID Connect.             |
 | `OAUTH_PROVIDER_NAME`   | Display name for the OAuth provider button.                  |
 
+### Social Login Providers
+
+Social login lets users sign in with their existing Google, Microsoft, Apple, or Dropbox accounts. Each provider is independently enabled and configured. For detailed setup instructions see the [Social Login Setup Guide](SocialLoginSetup.md).
+
+| **Variable** | **Description** | **Default** |
+|---|---|---|
+| `SOCIAL_AUTH_GOOGLE_ENABLED` | Enable Google Sign-In. | `false` |
+| `SOCIAL_AUTH_GOOGLE_CLIENT_ID` | Google OAuth2 client ID from the Google Cloud Console. | *(empty)* |
+| `SOCIAL_AUTH_GOOGLE_CLIENT_SECRET` | Google OAuth2 client secret. | *(empty)* |
+| `SOCIAL_AUTH_MICROSOFT_ENABLED` | Enable Microsoft Sign-In (Azure AD / Microsoft Entra ID). | `false` |
+| `SOCIAL_AUTH_MICROSOFT_CLIENT_ID` | Microsoft application (client) ID from Azure App Registrations. | *(empty)* |
+| `SOCIAL_AUTH_MICROSOFT_CLIENT_SECRET` | Microsoft client secret. | *(empty)* |
+| `SOCIAL_AUTH_MICROSOFT_TENANT` | Azure AD tenant: `common`, `organizations`, `consumers`, or a tenant GUID. | `common` |
+| `SOCIAL_AUTH_APPLE_ENABLED` | Enable Sign in with Apple. | `false` |
+| `SOCIAL_AUTH_APPLE_CLIENT_ID` | Apple Services ID (e.g. `com.example.docuelevate`). | *(empty)* |
+| `SOCIAL_AUTH_APPLE_TEAM_ID` | Apple Developer Team ID. | *(empty)* |
+| `SOCIAL_AUTH_APPLE_KEY_ID` | Apple Sign-In private key ID. | *(empty)* |
+| `SOCIAL_AUTH_APPLE_PRIVATE_KEY` | Apple Sign-In private key (PEM format). | *(empty)* |
+| `SOCIAL_AUTH_DROPBOX_ENABLED` | Enable Dropbox Sign-In. | `false` |
+| `SOCIAL_AUTH_DROPBOX_CLIENT_ID` | Dropbox OAuth2 App Key. | *(empty)* |
+| `SOCIAL_AUTH_DROPBOX_CLIENT_SECRET` | Dropbox OAuth2 App Secret. | *(empty)* |
+
 ### Multi-User Mode
 
 When multi-user mode is enabled, each authenticated user gets their own isolated document space.
@@ -316,8 +413,11 @@ Requires `AUTH_ENABLED=true`.
 
 #### Unclaimed Documents
 
-Documents ingested without a user session (e.g. via IMAP polling, API calls without authentication,
-or legacy imports) have `owner_id = NULL`. These are called **unclaimed** documents.
+Documents ingested via **system-level** sources (environment variable IMAP mailboxes, system watch folders)
+without a user session have `owner_id = NULL` unless `DEFAULT_OWNER_ID` is set. These are called **unclaimed** documents.
+
+Documents ingested via **per-user integrations** (IMAP or Watch Folder integrations configured through
+the Integrations dashboard) are automatically attributed to the owning user's `owner_id` and are never unclaimed.
 
 - When `UNOWNED_DOCS_VISIBLE_TO_ALL=true` (default), every authenticated user sees unclaimed
   documents alongside their own files. This allows users to discover and claim them.
@@ -357,6 +457,61 @@ default overage buffer applied across all plans.
 
 DocuElevate supports HTTP security headers to improve browser-side security. **These headers are disabled by default** since most deployments use a reverse proxy (Traefik, Nginx, etc.) that already adds them. Enable only if deploying directly without a reverse proxy. See [Deployment Guide - Security Headers](DeploymentGuide.md#security-headers) for detailed configuration examples.
 
+### Audit Logging
+
+DocuElevate provides comprehensive audit logging that records significant actions (logins, document CRUD, settings changes) to an append-only database table. Every entry captures the timestamp, user, action, resource, client IP, and optional JSON details.
+
+| **Variable**                   | **Description**                                                                                   | **Default** |
+|--------------------------------|---------------------------------------------------------------------------------------------------|-------------|
+| `AUDIT_LOGGING_ENABLED`       | Enable the HTTP request audit-logging middleware.                                                  | `true`      |
+| `AUDIT_LOG_INCLUDE_CLIENT_IP` | Include the client IP address in audit log entries. Disable for GDPR-sensitive deployments.       | `true`      |
+
+#### SIEM Integration
+
+Audit events can be forwarded in real time to external SIEM systems for centralised monitoring, alerting, and long-term retention. Two transports are supported:
+
+* **Syslog** – RFC 5424 structured-data messages over UDP or TCP. Works with rsyslog, syslog-ng, Graylog, Datadog, etc.
+* **HTTP** – JSON POST payloads compatible with Splunk HEC, Logstash HTTP input, Grafana Loki push API, and any generic webhook.
+
+| **Variable**                        | **Description**                                                                                   | **Default**   |
+|-------------------------------------|---------------------------------------------------------------------------------------------------|---------------|
+| `AUDIT_SIEM_ENABLED`               | Enable forwarding of audit events to an external SIEM system.                                     | `false`       |
+| `AUDIT_SIEM_TRANSPORT`             | Transport: `syslog` or `http`.                                                                    | `syslog`      |
+| `AUDIT_SIEM_SYSLOG_HOST`           | Hostname or IP of the syslog receiver.                                                            | `localhost`   |
+| `AUDIT_SIEM_SYSLOG_PORT`           | Port of the syslog receiver.                                                                      | `514`         |
+| `AUDIT_SIEM_SYSLOG_PROTOCOL`       | Protocol for syslog: `udp` or `tcp`.                                                              | `udp`         |
+| `AUDIT_SIEM_HTTP_URL`              | HTTP endpoint URL for SIEM delivery (e.g. Splunk HEC, Logstash, Loki).                           | *(empty)*     |
+| `AUDIT_SIEM_HTTP_TOKEN`            | Bearer / HEC token for the SIEM HTTP endpoint.                                                    | *(empty)*     |
+| `AUDIT_SIEM_HTTP_CUSTOM_HEADERS`   | Comma-separated `Key:Value` extra headers for SIEM HTTP requests.                                 | *(empty)*     |
+
+**Example – Syslog to rsyslog:**
+
+```bash
+AUDIT_SIEM_ENABLED=true
+AUDIT_SIEM_TRANSPORT=syslog
+AUDIT_SIEM_SYSLOG_HOST=syslog.internal.example.com
+AUDIT_SIEM_SYSLOG_PORT=514
+AUDIT_SIEM_SYSLOG_PROTOCOL=udp
+```
+
+**Example – Splunk HEC:**
+
+```bash
+AUDIT_SIEM_ENABLED=true
+AUDIT_SIEM_TRANSPORT=http
+AUDIT_SIEM_HTTP_URL=https://splunk.example.com:8088/services/collector/event
+AUDIT_SIEM_HTTP_TOKEN=your-hec-token
+```
+
+**Example – Logstash HTTP input:**
+
+```bash
+AUDIT_SIEM_ENABLED=true
+AUDIT_SIEM_TRANSPORT=http
+AUDIT_SIEM_HTTP_URL=https://logstash.example.com:8080
+AUDIT_SIEM_HTTP_TOKEN=
+```
+
 ### Rate Limiting
 
 DocuElevate implements rate limiting to protect against DoS attacks and API abuse. **Rate limiting is enabled by default** and uses Redis for distributed rate limiting across multiple workers.
@@ -846,6 +1001,7 @@ TESSERACT_LANGUAGE=eng+deu
 
 | **Variable**                        | **Description**                                                                                     |
 |-------------------------------------|-----------------------------------------------------------------------------------------------------|
+| `PAPERLESS_ENABLED`                 | Set to `false` to disable Paperless-ngx uploads without removing credentials. Default: `true`       |
 | `PAPERLESS_NGX_API_TOKEN`           | API token for Paperless NGX.                                                                        |
 | `PAPERLESS_HOST`                    | Root URL for Paperless NGX (e.g. `https://paperless.example.com`).                                 |
 | `PAPERLESS_CUSTOM_FIELD_ABSENDER`   | (Optional, Legacy) Name of the custom field in Paperless-ngx to store the sender ("absender") information. If set, the extracted sender will be automatically set as a custom field after document upload. Example: `Absender` or `Sender` |
@@ -888,6 +1044,7 @@ PAPERLESS_CUSTOM_FIELDS_MAPPING='{"absender": "Sender", "empfaenger": "Recipient
 
 | **Variable**            | **Description**                                  |
 |-------------------------|--------------------------------------------------|
+| `DROPBOX_ENABLED`       | Set to `false` to disable Dropbox uploads without removing credentials. Default: `true` |
 | `DROPBOX_APP_KEY`       | Dropbox API app key.                             |
 | `DROPBOX_APP_SECRET`    | Dropbox API app secret.                          |
 | `DROPBOX_REFRESH_TOKEN` | OAuth2 refresh token for Dropbox.                |
@@ -899,6 +1056,7 @@ For detailed setup instructions, see the [Dropbox Setup Guide](DropboxSetup.md).
 
 | **Variable**            | **Description**                                               |
 |-------------------------|---------------------------------------------------------------|
+| `NEXTCLOUD_ENABLED`     | Set to `false` to disable Nextcloud uploads without removing credentials. Default: `true` |
 | `NEXTCLOUD_UPLOAD_URL`  | Nextcloud WebDAV URL (e.g. `https://nc.example.com/remote.php/dav/files/<USERNAME>`). |
 | `NEXTCLOUD_USERNAME`    | Nextcloud login username.                                    |
 | `NEXTCLOUD_PASSWORD`    | Nextcloud login password.                                    |
@@ -908,6 +1066,7 @@ For detailed setup instructions, see the [Dropbox Setup Guide](DropboxSetup.md).
 
 | **Variable**                    | **Description**                                       |
 |---------------------------------|-------------------------------------------------------|
+| `GOOGLE_DRIVE_ENABLED`          | Set to `false` to disable Google Drive uploads without removing credentials. Default: `true` |
 | `GOOGLE_DRIVE_USE_OAUTH`        | Set to `true` to use OAuth flow (recommended)         |
 | `GOOGLE_DRIVE_CLIENT_ID`        | OAuth Client ID (required if using OAuth flow)        |
 | `GOOGLE_DRIVE_CLIENT_SECRET`    | OAuth Client Secret (required if using OAuth flow)    |
@@ -924,6 +1083,7 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
 
 | **Variable**            | **Description**                                               |
 |-------------------------|---------------------------------------------------------------|
+| `WEBDAV_ENABLED`        | Set to `false` to disable WebDAV uploads without removing credentials. Default: `true` |
 | `WEBDAV_URL`            | WebDAV server URL (e.g. `https://webdav.example.com/path`).   |
 | `WEBDAV_USERNAME`       | WebDAV authentication username.                               |
 | `WEBDAV_PASSWORD`       | WebDAV authentication password.                               |
@@ -934,6 +1094,7 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
 
 | **Variable**            | **Description**                                               |
 |-------------------------|---------------------------------------------------------------|
+| `FTP_ENABLED`           | Set to `false` to disable FTP uploads without removing credentials. Default: `true` |
 | `FTP_HOST`              | FTP server hostname or IP address.                            |
 | `FTP_PORT`              | FTP port (default: `21`).                                     |
 | `FTP_USERNAME`          | FTP authentication username.                                  |
@@ -946,6 +1107,7 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
 
 | **Variable**                  | **Description**                                         |
 |------------------------------|-------------------------------------------------------|
+| `SFTP_ENABLED`               | Set to `false` to disable SFTP uploads without removing credentials. Default: `true` |
 | `SFTP_HOST`                  | SFTP server hostname or IP address.                    |
 | `SFTP_PORT`                  | SFTP port (default: `22`).                             |
 | `SFTP_USERNAME`              | SFTP authentication username.                          |
@@ -977,6 +1139,7 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
 
 | **Variable**                     | **Description**                                                     |
 |----------------------------------|---------------------------------------------------------------------|
+| `DEST_EMAIL_ENABLED`            | Set to `false` to disable email delivery without removing credentials. Default: `true` |
 | `DEST_EMAIL_HOST`               | SMTP server hostname for document delivery.                          |
 | `DEST_EMAIL_PORT`               | SMTP port for document delivery (default: `587`).                    |
 | `DEST_EMAIL_USERNAME`           | SMTP authentication username for document delivery.                  |
@@ -989,6 +1152,7 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
 
 | **Variable**                    | **Description**                                       |
 |---------------------------------|-------------------------------------------------------|
+| `ONEDRIVE_ENABLED`              | Set to `false` to disable OneDrive uploads without removing credentials. Default: `true` |
 | `ONEDRIVE_CLIENT_ID`            | Azure AD application client ID                        |
 | `ONEDRIVE_CLIENT_SECRET`        | Azure AD application client secret                    |
 | `ONEDRIVE_TENANT_ID`            | Azure AD tenant ID: use "common" for personal accounts or your tenant ID for corporate accounts |
@@ -1001,6 +1165,7 @@ For detailed setup instructions, see the [OneDrive Setup Guide](OneDriveSetup.md
 
 | **Variable**                    | **Description**                                       |
 |---------------------------------|-------------------------------------------------------|
+| `S3_ENABLED`                    | Set to `false` to disable S3 uploads without removing credentials. Default: `true` |
 | `AWS_ACCESS_KEY_ID`             | AWS IAM access key ID                                 |
 | `AWS_SECRET_ACCESS_KEY`         | AWS IAM secret access key                             |
 | `AWS_REGION`                    | AWS region where your S3 bucket is located (default: `us-east-1`) |
@@ -1011,6 +1176,23 @@ For detailed setup instructions, see the [OneDrive Setup Guide](OneDriveSetup.md
 
 For detailed setup instructions, see the [Amazon S3 Setup Guide](AmazonS3Setup.md).
 
+### iCloud Drive (Apple)
+
+| **Variable**                    | **Description**                                       |
+|---------------------------------|-------------------------------------------------------|
+| `ICLOUD_ENABLED`                | Set to `false` to disable iCloud uploads without removing credentials. Default: `true` |
+| `ICLOUD_USERNAME`               | Apple ID email address                                |
+| `ICLOUD_PASSWORD`               | App-specific password (generate at [appleid.apple.com](https://appleid.apple.com/account/manage)) |
+| `ICLOUD_FOLDER`                 | Target folder path in iCloud Drive (e.g. `Documents/Uploads`) |
+| `ICLOUD_COOKIE_DIRECTORY`       | Optional directory for session cookie persistence (default: `~/.pyicloud`) |
+
+> **Note:** Apple does not provide a public REST API for iCloud Drive. This
+> integration uses the [pyicloud](https://github.com/picklepete/pyicloud)
+> library which relies on an unofficial, reverse-engineered protocol. Because
+> most Apple IDs have two-factor authentication enabled, you **must** generate
+> an [app-specific password](https://support.apple.com/en-us/102654) and use
+> it as `ICLOUD_PASSWORD`.
+
 ### Notification System
 
 | **Variable**                | **Description**                                           |
@@ -1043,6 +1225,55 @@ payment processors.
 
 For detailed setup instructions, see the [Notifications Setup Guide](NotificationsSetup.md).
 
+#### Per-User Notification System
+
+In addition to the system-level Apprise notifications, DocuElevate includes a **per-user notification system** that gives each user full control over how they are notified about their own document events.
+
+**Notification Dashboard** — available at `/notifications` for every logged-in user. It has three tabs:
+
+| Tab | Description |
+|-----|-------------|
+| **Inbox** | In-app bell-icon notification feed. Persisted in the database; shows unread count badge in the navigation bar. Users can mark individual items or all items as read. |
+| **Targets** | User-defined notification channels: **Email (SMTP)** and **Webhook (HTTP POST)**. Each target can be tested independently from the UI. |
+| **Preferences** | Event/channel matrix. Users choose which channels are triggered for each event type. In-app notifications are always enabled. |
+
+**User-centric event types:**
+
+| Event | Description |
+|-------|-------------|
+| `document.processed` | A document uploaded by the user was successfully processed and uploaded to destinations |
+| `document.failed` | A document uploaded by the user failed during processing |
+
+**Email target configuration fields:**
+
+| Field | Description |
+|-------|-------------|
+| `smtp_host` | SMTP server hostname |
+| `smtp_port` | SMTP port (default `587`) |
+| `smtp_username` | SMTP login username |
+| `smtp_password` | SMTP login password (stored in database, masked in UI) |
+| `smtp_use_tls` | Enable STARTTLS (`true`/`false`, default `true`) |
+| `sender_email` | From address (defaults to `smtp_username` if omitted) |
+| `recipient_email` | Destination address for this target |
+
+**Webhook target configuration fields:**
+
+| Field | Description |
+|-------|-------------|
+| `url` | HTTP(S) URL to POST the notification payload to |
+| `secret` | Optional secret string sent as `X-DocuElevate-Secret` header |
+
+**Webhook payload format:**
+```json
+{
+  "event": "document.processed",
+  "title": "Document processed: invoice.pdf",
+  "message": "Your document 'invoice.pdf' has been successfully processed and uploaded."
+}
+```
+
+> **Note:** There are no additional environment variables for the per-user notification system — all settings are stored in the database and managed through the user-facing `/notifications` dashboard.
+
 ### Webhooks
 
 Webhooks notify external systems via HTTP POST when document events occur.
@@ -1118,6 +1349,64 @@ Administrators can set the **site-wide default** colour scheme that is applied w
 UI_DEFAULT_COLOR_SCHEME=dark
 ```
 
+### Support / Help Center – Zammad Integration
+
+The Help Center page (`/help`) can optionally integrate with a [Zammad](https://zammad.com/) instance to offer live chat and a ticket-creation form directly within DocuElevate.
+
+| **Variable**            | **Description**                                                                                       | **Default** |
+|-------------------------|-------------------------------------------------------------------------------------------------------|-------------|
+| `ZAMMAD_URL`            | Base URL of your Zammad instance (e.g. `https://zammad.example.com`). Required for chat and form.     | *(unset)*   |
+| `ZAMMAD_CHAT_ENABLED`   | Show a Zammad live-chat widget on the Help Center page.                                               | `false`     |
+| `ZAMMAD_CHAT_ID`        | Zammad chat topic ID (see *Channels → Chat → Topics* in Zammad admin).                               | `1`         |
+| `ZAMMAD_FORM_ENABLED`   | Show a "Submit a Ticket" feedback form on the Help Center page.                                       | `false`     |
+| `SUPPORT_EMAIL`         | Support e-mail address displayed on the Help Center page.                                             | *(unset)*   |
+
+**Example:**
+
+```dotenv
+ZAMMAD_URL=https://zammad.example.com
+ZAMMAD_CHAT_ENABLED=true
+ZAMMAD_CHAT_ID=1
+ZAMMAD_FORM_ENABLED=true
+SUPPORT_EMAIL=support@example.com
+```
+
+> **Note:** The live-chat widget requires at least one Zammad agent to be online. If no agent is available, the widget will not appear. Enable Zammad's debug mode (`debug: true`) for troubleshooting.
+
+#### Automatic User Context (Auto-Fill)
+
+When a user is logged in, DocuElevate automatically passes their identity to the Zammad widgets:
+
+- **Ticket form:** The user's **name** and **email** are pre-filled in the form fields. A *DocuElevate User Context* block containing the user's name, email, and username is appended to the ticket body so the support agent can immediately identify the requester.
+- **Live chat:** The user's **name** and **email** are passed to the Zammad chat widget constructor. Depending on your Zammad version, the agent may see this information in the chat session details.
+
+No additional configuration is required — the auto-fill uses the authenticated session data (OAuth, local login, or admin credentials). Anonymous visitors see the standard Zammad widgets without pre-filled data.
+
+## Observability – Sentry
+
+DocuElevate integrates with [Sentry](https://sentry.io) for real-time error tracking and performance monitoring.  See [SentrySetup.md](./SentrySetup.md) for a full setup guide.
+
+| Variable | Description | Default |
+|---|---|---|
+| `SENTRY_DSN` | Sentry DSN URL.  When set, error reporting and performance tracing are enabled automatically.  Leave blank to disable. | *(unset)* |
+| `SENTRY_ENVIRONMENT` | Environment label attached to every Sentry event (`development`, `staging`, `production`, …). | `production` |
+| `SENTRY_TRACES_SAMPLE_RATE` | Fraction of requests captured for performance tracing (0.0 – 1.0).  `0.0` disables tracing entirely. | `0.1` |
+| `SENTRY_PROFILES_SAMPLE_RATE` | Fraction of profiled transactions sent to Sentry (0.0 – 1.0).  Only active when traces > 0. | `0.0` |
+| `SENTRY_SEND_DEFAULT_PII` | Attach PII (IP addresses, user agents) to Sentry events.  Disabled by default for GDPR/CCPA compliance. | `false` |
+
+```bash
+# Minimal example
+SENTRY_DSN=https://<key>@o<org>.ingest.sentry.io/<project>
+SENTRY_ENVIRONMENT=production
+
+# Optional tuning
+SENTRY_TRACES_SAMPLE_RATE=0.1
+SENTRY_PROFILES_SAMPLE_RATE=0.0
+SENTRY_SEND_DEFAULT_PII=false
+```
+
+> **Note:** Sentry is completely opt-in — if `SENTRY_DSN` is not set, the SDK is never initialised and no data leaves your infrastructure.
+
 ## Duplicate Document Detection
 
 DocuElevate detects and flags documents that share the same content, even if they arrive as separate uploads.

docs/DropboxSetup.md

@@ -15,23 +15,37 @@ For a complete list of configuration options, see the [Configuration Guide](Conf
 
 ## Setup Methods
 
-You can set up Dropbox integration in two ways:
+DocuElevate supports two distinct Dropbox OAuth flows:
 
-1. **Using the Built-in Setup Wizard (Recommended)**: An interactive setup experience available at `/dropbox-setup` in the web interface
-2. **Manual Setup**: Following the step-by-step instructions in this document
+1. **Per-User Integration Wizard (Recommended for end users)**: Triggered from the Integrations dashboard (`/integrations`) by clicking **Authorize** on a Dropbox destination or Dropbox-backed Watch Folder. Credentials are saved securely to your personal integration record — global settings are never exposed.
+2. **System-Level Setup Wizard**: Available at `/dropbox-setup` for administrators configuring the global system-wide Dropbox connection. Generates environment variables for all worker nodes.
+3. **Manual Setup**: Following the step-by-step instructions in this document.
 
-## Using the Setup Wizard
+## Per-User OAuth Flow (Integrations Dashboard)
 
-The easiest way to set up Dropbox integration is to use the built-in setup wizard:
+End users authorize their own Dropbox integration from the **Integrations** dashboard:
 
-1. Navigate to the `/dropbox-setup` page in your DocuElevate instance
-2. Follow the on-screen instructions to create a Dropbox app
-3. Enter your App Key and App Secret in the wizard
-4. Optionally specify a custom folder path for uploads
-5. Click "Start Authentication Flow" to begin the authorization process
-6. Complete the Dropbox authentication process
-7. The system will automatically exchange the authorization code for a refresh token
-8. Copy the generated environment variables for your worker nodes
+1. Navigate to `/integrations` and click **+ Add Destination** (or **+ Add Source** for Watch Folder).
+2. Create a Dropbox destination integration (or a Watch Folder with `source_type = dropbox`).
+3. Click the **Authorize** button next to the integration — it links directly to the OAuth wizard pre-loaded with your integration's configuration.
+4. Enter your Dropbox App Key and App Secret in the wizard (or use the global admin credentials if pre-configured).
+5. Click **Start Authentication Flow**, authorize access in Dropbox, and the refresh token is automatically saved to your personal integration record.
+6. The page redirects back to `/integrations` on success. Re-authorization is available at any time via the **Re-Authorize** button.
+
+> **Note:** Your credentials are stored encrypted per-integration and are never mixed with other users' data. Each user can have multiple Dropbox integrations with independent tokens.
+
+## Using the System-Level Setup Wizard (Admin)
+
+The easiest way to configure the global Dropbox integration is to use the built-in setup wizard:
+
+1. Navigate to the `/dropbox-setup` page in your DocuElevate instance.
+2. Follow the on-screen instructions to create a Dropbox app.
+3. Enter your App Key and App Secret in the wizard.
+4. Optionally specify a custom folder path for uploads.
+5. Click **Start Authentication Flow** to begin the authorization process.
+6. Complete the Dropbox authentication process.
+7. The system will automatically exchange the authorization code for a refresh token.
+8. Copy the generated environment variables for your worker nodes.
 
 The wizard handles all the token exchange steps and provides you with the exact configuration needed for your environment.