Merge branch 'main' into copilot/add-document-sharing-feature

2026-03-08 23:12:27 +01:00
parent 06091546b0 84e8ea2fb2
commit 201d7c8fdc
31 changed files with 5345 additions and 28 deletions
@@ -2,6 +2,9 @@

 DocuElevate provides a powerful REST API for programmatic access to all its features. This document serves as a reference for the available endpoints and their usage.

+> **Looking for a quick way to script against DocuElevate?**
+> The built-in [CLI tool](./CLIGuide.md) wraps the API and is ready to use from a terminal or shell script — no HTTP client code required.
+
 ## API Overview

 - Base URL: `http://<your-docuelevate-instance>/api`
@@ -1785,12 +1788,27 @@ Returns the catalogue of built-in step types.
    "label": "OCR Processing",
    "description": "Extract text using Azure Document Intelligence or local Tesseract.",
    "config_schema": {
-      "force_cloud_ocr": { "type": "boolean", "default": false }
+      "force_cloud_ocr": { "type": "boolean", "default": false },
+      "ocr_language": {
+        "type": "select",
+        "default": "auto",
+        "description": "Language(s) for OCR. Overrides the global setting for Tesseract/EasyOCR. Azure/Mistral auto-detect.",
+        "options": [
+          { "value": "auto", "label": "Auto (use system default)" },
+          { "value": "eng",  "label": "English" },
+          { "value": "deu",  "label": "German" },
+          { "value": "fra",  "label": "French" },
+          { "value": "spa",  "label": "Spanish" },
+          "..."
+        ]
+      }
    }
  }
 }
 ```

+The `ocr_language` field accepts Tesseract language codes (e.g. `"eng"`, `"deu"`, `"eng+deu"` for multi-language) or `"auto"` to fall back to the global system setting.  The full list of 28 supported language codes is returned by the step-types endpoint.
+
 ### List pipelines

 ```bash
@@ -1883,12 +1901,23 @@ Content-Type: application/json

 {
  "step_type": "ocr",
-  "label": "Cloud OCR",
-  "config": { "force_cloud_ocr": true },
+  "label": "German OCR",
+  "config": { "force_cloud_ocr": false, "ocr_language": "deu" },
  "enabled": true
 }
 ```

+Multi-language (Tesseract `+`-separated codes):
+
+```bash
+{
+  "step_type": "ocr",
+  "config": { "ocr_language": "eng+deu" }
+}
+```
+
+Use `"ocr_language": "auto"` (or omit the field) to fall back to the global system language setting.
+
 ### Update step

 ```bash
@@ -0,0 +1,348 @@
+# DocuElevate CLI Guide
+
+The `docuelevate` command-line tool lets you interact with your DocuElevate instance
+from a terminal, shell script, or CI/CD pipeline.  It is ideal for:
+
+- Batch uploads from a script or cron job
+- Downloading processed documents programmatically
+- Searching documents in automation workflows
+- Rotating API tokens safely without touching the web UI
+
+---
+
+## Installation
+
+The CLI is included in the standard DocuElevate package.  After installing the
+Python package (e.g. inside the Docker image or a virtualenv), the `docuelevate`
+command is available:
+
+```bash
+pip install docuelevate          # or: pip install -e . inside the repo
+docuelevate --help
+```
+
+---
+
+## Authentication
+
+All commands require an API token.  Create one at `/api-tokens` in the web UI,
+or with the `docuelevate token create` command itself.
+
+Provide the token in either of two ways:
+
+| Method | Example |
+|--------|---------|
+| `--token` flag | `docuelevate --token de_xxxxx list` |
+| Environment variable | `export DOCUELEVATE_API_TOKEN=de_xxxxx` |
+
+The environment variable is recommended for scripts so that secrets never appear
+in shell history or process listings.
+
+---
+
+## Configuration
+
+| Option / Variable | Default | Description |
+|-------------------|---------|-------------|
+| `--url` / `DOCUELEVATE_URL` | `http://localhost:8000` | Base URL of the DocuElevate instance |
+| `--token` / `DOCUELEVATE_API_TOKEN` | _(none)_ | API token for authentication |
+| `--format` | `table` | Output format: `table` (human-readable) or `json` (pipe-friendly) |
+| `--timeout` / `DOCUELEVATE_TIMEOUT` | `60` | HTTP request timeout in seconds |
+
+Setting both `DOCUELEVATE_URL` and `DOCUELEVATE_API_TOKEN` in your environment
+removes the need for flags on every invocation:
+
+```bash
+export DOCUELEVATE_URL=https://docs.example.com
+export DOCUELEVATE_API_TOKEN=de_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+docuelevate list
+```
+
+---
+
+## Commands
+
+### `list` — List documents
+
+```
+docuelevate [OPTIONS] list [OPTIONS]
+```
+
+Returns a paginated list of documents stored in DocuElevate.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--page` | `1` | Page number |
+| `--per-page` | `25` | Items per page (max 200) |
+| `--search` | — | Filter by filename substring |
+| `--mime-type` | — | Filter by MIME type (e.g. `application/pdf`) |
+| `--status` | — | Filter by status: `pending`, `processing`, `completed`, `failed` |
+| `--sort-by` | `created_at` | Sort field |
+| `--sort-order` | `desc` | Sort direction: `asc` or `desc` |
+
+**Examples:**
+
+```bash
+# Human-readable table
+docuelevate list
+
+# Only completed PDFs
+docuelevate list --status completed --mime-type application/pdf
+
+# Pipe filenames to another command
+docuelevate --format json list | jq -r '.[].filename'
+
+# Search by filename
+docuelevate list --search invoice
+```
+
+---
+
+### `upload` — Upload files
+
+```
+docuelevate [OPTIONS] upload [OPTIONS] FILES...
+```
+
+Uploads one or more local files to DocuElevate for processing.  Multiple file
+paths (or shell globs) can be provided for batch uploads.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--batch-size` | `5` | Maximum uploads before reporting progress |
+
+**Examples:**
+
+```bash
+# Upload a single file
+docuelevate upload report.pdf
+
+# Batch upload — all PDFs in a folder
+docuelevate upload /scans/*.pdf
+
+# Upload multiple files explicitly
+docuelevate upload invoice.pdf contract.pdf receipt.png
+
+# JSON output to capture task IDs
+docuelevate --format json upload *.pdf | jq '.[].task_id'
+```
+
+---
+
+### `download` — Download a file
+
+```
+docuelevate [OPTIONS] download [OPTIONS] FILE_ID
+```
+
+Downloads a processed (or original) file by its numeric ID.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `-o` / `--output` | _(server filename)_ | Destination file path |
+| `--version` | `processed` | `processed` or `original` |
+
+**Examples:**
+
+```bash
+# Download processed version of file #42
+docuelevate download 42
+
+# Save to a specific path
+docuelevate download 42 -o /tmp/invoice.pdf
+
+# Download the original (unprocessed) upload
+docuelevate download 42 --version original -o original.pdf
+```
+
+---
+
+### `search` — Full-text search
+
+```
+docuelevate [OPTIONS] search [OPTIONS] QUERY
+```
+
+Searches across document text, filenames, tags, and metadata using Meilisearch.
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--mime-type` | — | Filter by MIME type |
+| `--document-type` | — | Filter by document type (e.g. `Invoice`) |
+| `--tags` | — | Filter by tag |
+| `--language` | — | Filter by language code (e.g. `en`, `de`) |
+| `--page` | `1` | Page number |
+| `--per-page` | `20` | Results per page (max 100) |
+
+**Examples:**
+
+```bash
+# Simple search
+docuelevate search "amazon invoice"
+
+# With filters
+docuelevate search "contract" --document-type Contract --language en
+
+# Pipe file IDs to the download command
+docuelevate --format json search "Q1 report" | jq -r '.[].file_id'
+```
+
+---
+
+### `token` — Manage API tokens
+
+The `token` sub-group provides commands to create, list, and revoke personal API
+tokens — enabling **token rotation** without logging into the web UI.
+
+#### `token create`
+
+```
+docuelevate token create NAME
+```
+
+Creates a new token.  The full token value is printed exactly once — store it
+securely.
+
+```bash
+# Create a new token
+docuelevate --token de_existing token create "CI Pipeline"
+
+# Capture the new token value in a script
+NEW_TOKEN=$(docuelevate --format json --token de_existing token create "Rotation" \
+            | jq -r '.token')
+```
+
+#### `token list`
+
+```
+docuelevate token list
+```
+
+Lists all your tokens (active and revoked).
+
+```bash
+docuelevate token list
+
+# JSON for scripting
+docuelevate --format json token list | jq '.[] | select(.is_active) | .id'
+```
+
+#### `token revoke`
+
+```
+docuelevate token revoke [--yes] TOKEN_ID
+```
+
+Revokes a token by its numeric ID.  The token is immediately invalidated.
+
+| Option | Description |
+|--------|-------------|
+| `--yes` / `-y` | Skip confirmation prompt |
+
+```bash
+# Interactive confirmation
+docuelevate token revoke 3
+
+# Non-interactive (for scripts)
+docuelevate token revoke 3 --yes
+```
+
+---
+
+## Token Rotation
+
+Rotate an API token safely without any downtime:
+
+```bash
+# 1. Create the replacement token
+NEW_TOKEN=$(docuelevate --format json --token "$OLD_TOKEN" \
+            token create "Rotated $(date +%Y-%m-%d)" | jq -r '.token')
+
+# 2. Update consumers to use NEW_TOKEN, then revoke the old one
+OLD_ID=$(docuelevate --format json --token "$OLD_TOKEN" token list \
+         | jq '.[] | select(.is_active and (.token_prefix == "de_old_prefix")) | .id')
+docuelevate --token "$NEW_TOKEN" token revoke --yes "$OLD_ID"
+```
+
+---
+
+## Output Formats
+
+### Table (default)
+
+Human-readable, suitable for terminal use:
+
+```
+ID  FILENAME           SIZE   STATUS     CREATED_AT
+--  -----------------  -----  ---------  -------------------
+42  invoice_2026.pdf   98304  completed  2026-03-01T10:30:00
+43  contract.pdf       51200  pending    2026-03-01T11:00:00
+```
+
+### JSON (`--format json`)
+
+Machine-readable, pipe-friendly, suitable for `jq`, shell scripts, and CI:
+
+```json
+[
+  {
+    "id": 42,
+    "filename": "invoice_2026.pdf",
+    "size": 98304,
+    "status": "completed",
+    "created_at": "2026-03-01T10:30:00"
+  }
+]
+```
+
+---
+
+## Pipe-Friendly Examples
+
+```bash
+# Download all completed PDFs in a folder
+docuelevate --format json list --status completed --mime-type application/pdf \
+  | jq -r '.[].id' \
+  | xargs -I {} docuelevate download {} -o /backup/{}.pdf
+
+# Count documents by status
+docuelevate --format json list --per-page 200 \
+  | jq 'group_by(.status) | map({status: .[0].status, count: length})'
+
+# Search and get filenames
+docuelevate --format json search "2026 invoice" \
+  | jq -r '.[].filename'
+
+# Batch upload all new files and capture task IDs
+find /inbox -name "*.pdf" | xargs docuelevate upload \
+  && echo "All uploaded"
+```
+
+---
+
+## Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| `0` | Success |
+| `1` | One or more uploads failed (partial failure) |
+| `2` | Invalid options or arguments |
+| other | Fatal error (network, API, authentication) |
+
+---
+
+## Environment Variables Reference
+
+| Variable | Description |
+|----------|-------------|
+| `DOCUELEVATE_URL` | Base URL of the DocuElevate instance |
+| `DOCUELEVATE_API_TOKEN` | Personal API token (`de_…`) |
+| `DOCUELEVATE_TIMEOUT` | HTTP request timeout in seconds (default: 60) |
+
+---
+
+## See Also
+
+- [API Documentation](./API.md) — full REST API reference
+- [User Guide](./UserGuide.md) — web UI guide including API token management
+- [Configuration Guide](./ConfigurationGuide.md) — server-side configuration
@@ -1070,6 +1070,55 @@ payment processors.

 For detailed setup instructions, see the [Notifications Setup Guide](NotificationsSetup.md).

+#### Per-User Notification System
+
+In addition to the system-level Apprise notifications, DocuElevate includes a **per-user notification system** that gives each user full control over how they are notified about their own document events.
+
+**Notification Dashboard** — available at `/notifications` for every logged-in user. It has three tabs:
+
+| Tab | Description |
+|-----|-------------|
+| **Inbox** | In-app bell-icon notification feed. Persisted in the database; shows unread count badge in the navigation bar. Users can mark individual items or all items as read. |
+| **Targets** | User-defined notification channels: **Email (SMTP)** and **Webhook (HTTP POST)**. Each target can be tested independently from the UI. |
+| **Preferences** | Event/channel matrix. Users choose which channels are triggered for each event type. In-app notifications are always enabled. |
+
+**User-centric event types:**
+
+| Event | Description |
+|-------|-------------|
+| `document.processed` | A document uploaded by the user was successfully processed and uploaded to destinations |
+| `document.failed` | A document uploaded by the user failed during processing |
+
+**Email target configuration fields:**
+
+| Field | Description |
+|-------|-------------|
+| `smtp_host` | SMTP server hostname |
+| `smtp_port` | SMTP port (default `587`) |
+| `smtp_username` | SMTP login username |
+| `smtp_password` | SMTP login password (stored in database, masked in UI) |
+| `smtp_use_tls` | Enable STARTTLS (`true`/`false`, default `true`) |
+| `sender_email` | From address (defaults to `smtp_username` if omitted) |
+| `recipient_email` | Destination address for this target |
+
+**Webhook target configuration fields:**
+
+| Field | Description |
+|-------|-------------|
+| `url` | HTTP(S) URL to POST the notification payload to |
+| `secret` | Optional secret string sent as `X-DocuElevate-Secret` header |
+
+**Webhook payload format:**
+```json
+{
+  "event": "document.processed",
+  "title": "Document processed: invoice.pdf",
+  "message": "Your document 'invoice.pdf' has been successfully processed and uploaded."
+}
+```
+
+> **Note:** There are no additional environment variables for the per-user notification system — all settings are stored in the database and managed through the user-facing `/notifications` dashboard.
+
 ### Webhooks

 Webhooks notify external systems via HTTP POST when document events occur.
@@ -568,13 +568,50 @@ Processing pipelines let you define exactly what happens to your documents when
 |-----------|-------------|
 | `convert_to_pdf` | Convert non-PDF files to PDF using Gotenberg |
 | `check_duplicates` | Detect duplicate files by content hash |
-| `ocr` | Extract text with Azure Document Intelligence or local Tesseract |
+| `ocr` | Extract text with OCR (supports multi-language configuration, see below) |
 | `extract_metadata` | Extract structured metadata (type, sender, tags) with AI |
 | `embed_metadata` | Write extracted metadata into the PDF document properties |
 | `compute_embedding` | Compute semantic embeddings for similarity search |
 | `send_to_destinations` | Upload the processed document to all configured storage destinations |
 | `classify` | Classify the document type with AI |

+#### OCR step options
+
+The `ocr` step supports two optional configuration fields:
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `force_cloud_ocr` | boolean | `false` | Always run cloud OCR even if the PDF already has embedded text |
+| `ocr_language` | string | `"auto"` | Language(s) to use for OCR text extraction (see below) |
+
+**`ocr_language` — per-pipeline language override**
+
+This option enables manual language control per pipeline, overriding the global Tesseract/EasyOCR language settings for all documents processed by that pipeline. The following values are supported (28 languages total):
+
+| Value | Language | Value | Language |
+|-------|----------|-------|----------|
+| `auto` | Auto (use system default) | `jpn` | Japanese |
+| `ara` | Arabic | `kor` | Korean |
+| `chi_sim` | Chinese (Simplified) | `nor` | Norwegian |
+| `chi_tra` | Chinese (Traditional) | `pol` | Polish |
+| `ces` | Czech | `por` | Portuguese |
+| `dan` | Danish | `ron` | Romanian |
+| `nld` | Dutch | `rus` | Russian |
+| `eng` | English | `spa` | Spanish |
+| `fin` | Finnish | `swe` | Swedish |
+| `fra` | French | `tha` | Thai |
+| `deu` | German | `tur` | Turkish |
+| `ell` | Greek | `ukr` | Ukrainian |
+| `heb` | Hebrew | `vie` | Vietnamese |
+| `hin` | Hindi | | |
+| `hun` | Hungarian | | |
+| `ita` | Italian | | |
+
+> **Notes:**
+> - The language override applies to **Tesseract** and **EasyOCR** providers. **Azure Document Intelligence** and **Mistral OCR** perform automatic language detection regardless of this setting.
+> - For multi-language documents with Tesseract, combine codes with `+`, e.g. `eng+deu`.
+> - Setting `ocr_language` to `auto` or leaving it unset uses the global `TESSERACT_LANGUAGE` / `EASYOCR_LANGUAGES` environment variables.
+
 ### Assigning a pipeline to a file

 You can assign (or change) the pipeline for an individual document via the file detail page or the API: