From fcf2452b625109dbd7f7464929d5482b94db4a32 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sun, 15 Mar 2026 17:51:09 +0000 Subject: [PATCH] docs: rewrite README.md and update UserGuide and Troubleshooting docs - Rewrite README.md to reflect current state of the project: - Updated Overview with all 12 storage, 7 AI, 6 OCR providers - Comprehensive Features section (mobile, CLI, browser ext, pipelines, etc.) - Updated Workflow with all ingestion channels and distribution targets - Expanded Documentation index with all doc links organized by category - Updated Tech Stack table (Meilisearch, MkDocs, Expo, etc.) - Added Kubernetes/Helm quick start - Added status-view screenshot - Updated dependency licenses table - Updated docs/UserGuide.md with cross-references to Mobile App, CLI, Browser Extension, and API docs - Expanded docs/Troubleshooting.md from 175 to 300+ lines with new sections for Search, Pipelines, Mobile App, CLI, Performance, and updated all existing sections with current information Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com> --- README.md | 300 +++++++++++++++++++++++++++------------- docs/Troubleshooting.md | 236 +++++++++++++++++++++++++------ docs/UserGuide.md | 9 ++ 3 files changed, 409 insertions(+), 136 deletions(-) diff --git a/README.md b/README.md index a149a4e8..e4235cd4 100644 --- a/README.md +++ b/README.md @@ -24,121 +24,154 @@
- DocuElevate Logo + DocuElevate Hero
## Overview -DocuElevate automates the handling, extraction, and processing of documents using a variety of services, including: +DocuElevate is an intelligent document processing system that automates the ingestion, OCR, AI-powered metadata extraction, and distribution of documents. It supports a wide range of AI providers, OCR engines, and cloud storage destinations out of the box. -- **AI Provider** (pluggable – OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Portkey, and more) for metadata extraction and text refinement. -- **Dropbox**, **Nextcloud**, and **Google Drive** for file storage and uploads. -- **Paperless NGX** for document indexing and management. -- **Azure Document Intelligence** for OCR on PDFs. -- **Gotenberg** for file-to-PDF conversions. -- **Authentik** for authentication and user management. +**Key capabilities:** -It is designed for flexibility and configurability through environment variables, making it easily customizable for different workflows. The system can fetch documents from multiple IMAP mailboxes, process them (OCR, metadata extraction, PDF conversion), and store them in the desired destinations. +- **AI-Powered Metadata Extraction** — pluggable AI providers including OpenAI, Anthropic Claude, Google Gemini, Ollama (local), OpenRouter, Portkey, and Azure OpenAI via LiteLLM +- **Multi-Engine OCR** — Azure Document Intelligence, Tesseract, EasyOCR, Mistral OCR, Google Cloud Document AI, and AWS Textract with configurable merge strategies +- **12 Storage Destinations** — Dropbox, Google Drive, OneDrive, Amazon S3, Nextcloud, WebDAV, FTP, SFTP, iCloud Drive, Email (SMTP), Paperless-ngx, and Rclone +- **Multi-Channel Ingestion** — web upload, browser extension, mobile app, CLI, REST API, IMAP email, and watched folders (local, cloud, FTP/SFTP) +- **Processing Pipelines** — customizable multi-step workflows with conditional routing rules +- **Full-Text Search** — powered by Meilisearch for instant document discovery +- **Multi-User with SSO** — local accounts, OAuth2/OIDC (Authentik), and social login (Google, Microsoft, Apple, Dropbox) -The project includes a **UI** for uploading and managing files, and an API documentation page is available at `/docs` (powered by **FastAPI**). - -## Documentation Index - -- [User Guide](docs/UserGuide.md) - How to use DocuElevate -- [Browser Extension Guide](docs/BrowserExtension.md) - Install and use the browser extension -- [API Documentation](docs/API.md) - API reference -- [Deployment Guide](docs/DeploymentGuide.md) - How to deploy DocuElevate -- [Configuration Guide](docs/ConfigurationGuide.md) - Available configuration options -- [Build Metadata](docs/BuildMetadata.md) - Automated version and build information -- [CI/CD Tools Guide](docs/CIToolsGuide.md) - CI/CD pipeline and tool documentation -- [CI Workflow Guide](docs/CIWorkflow.md) - Detailed workflow documentation -- [Development Guide](CONTRIBUTING.md) - How to contribute to DocuElevate -- [Troubleshooting](docs/Troubleshooting.md) - Common issues and solutions +The project ships with a web UI, a REST + GraphQL API, a CLI tool, a native mobile app (iOS & Android), a browser extension, and Helm charts for Kubernetes deployment. ## Screenshots
DocuElevate Upload Interface -

Upload interface for adding new documents

+

Upload interface — drag-and-drop file upload with real-time progress

DocuElevate Files View -

Files view with processed documents and metadata

+

Files view — processed documents with AI-extracted metadata

+ + DocuElevate Status View +

Status view — system health and service monitoring

> **Note:** Screenshots may not reflect the very latest UI. For the most current look, visit [docuelevate.org](https://www.docuelevate.org). -## Workflow Process - -DocuElevate follows a streamlined document processing workflow: +## Workflow
DocuElevate Workflow
-### Document Ingestion -Documents enter DocuElevate through four possible channels: -1. **Web Upload**: Users manually upload files via the web interface -2. **Browser Extension**: Send files directly from your browser with one click -3. **Email Attachments**: Automatic polling of configured IMAP mailboxes (supports multiple accounts) -4. **API**: Direct programmatic uploads via the REST API +### Ingestion + +Documents enter DocuElevate through multiple channels: + +| Channel | Description | +|---------|-------------| +| **Web Upload** | Drag-and-drop interface with real-time progress (up to 1 GB per file) | +| **Browser Extension** | Clip web pages or send files from Chrome, Firefox, or Edge | +| **Mobile App** | Capture documents with the device camera or upload from the photo library | +| **CLI** | Batch uploads and scripted workflows via the `docuelevate` command-line tool | +| **REST API** | Programmatic uploads with full API-token authentication | +| **Email (IMAP)** | Automatic polling of multiple mailboxes with attachment filtering | +| **Watched Folders** | Monitor local paths, FTP, SFTP, S3, Dropbox, Google Drive, OneDrive, Nextcloud, or WebDAV for new files | ### Processing Pipeline -Every document goes through the following steps: -1. **PDF Conversion**: Non-PDF files are converted to PDF format using Gotenberg -2. **OCR Processing**: Azure Document Intelligence extracts text from images/scans -3. **Metadata Extraction**: The configured AI provider analyzes document content to identify: - - Document type (invoice, receipt, contract, etc.) - - Key entities (dates, names, amounts, account numbers) - - Important data points specific to the document type -4. **Enrichment**: Metadata is attached to the document in a structured format + +Each document passes through a configurable set of steps: + +1. **PDF Conversion** — Non-PDF files are converted using Gotenberg, with optional PDF/A archival conversion +2. **OCR** — Text extraction via one or more OCR engines (Azure, Tesseract, EasyOCR, Mistral, Google Document AI, AWS Textract) with configurable merge strategies +3. **AI Metadata Extraction** — The configured AI provider classifies the document and extracts structured metadata (type, dates, amounts, entities) +4. **Enrichment** — Metadata is embedded into the PDF and stored alongside the document +5. **Embedding Generation** — Vector embeddings for similarity search and duplicate detection + +Steps can be customized using **Pipelines** and **Routing Rules** for conditional processing. ### Distribution -Processed documents with their metadata can be automatically sent to: -- **Dropbox**: For cloud storage and sharing -- **Nextcloud**: For self-hosted file storage -- **Google Drive**: For Google Workspace integration -- **Paperless-NGX**: For advanced document management with search capabilities -Users can choose to send documents to any combination of these destinations through configuration settings or manual selection. +Processed documents are distributed to any combination of configured destinations: + +| Destination | Type | +|------------|------| +| **Dropbox** | Cloud storage | +| **Google Drive** | Cloud storage | +| **OneDrive** | Cloud storage | +| **Amazon S3** | Object storage | +| **Nextcloud** | Self-hosted cloud | +| **WebDAV** | Protocol-based | +| **FTP / SFTP** | File transfer | +| **iCloud Drive** | Apple cloud | +| **Email (SMTP)** | Send as attachment | +| **Paperless-ngx** | Document management system | +| **Rclone** | 70+ cloud providers via Rclone | ## Features -- **Intuitive File Upload**: - - Drag-and-drop file upload on both Upload and Files pages—upload anywhere on the Files page - - Real-time upload progress with validation - - Support for PDF, Office documents, images, and more (up to 500MB per file) -- **Browser Extension**: - - Send files directly from your browser to DocuElevate with one click - - Compatible with Chrome, Firefox, Edge, and other Chromium-based browsers - - Context menu integration for quick access - - See [Browser Extension Guide](docs/BrowserExtension.md) for installation and usage -- **Document Upload & Storage**: - - Manual uploads (via API or UI) to Dropbox, Nextcloud, Google Drive, or Paperless -- **OCR Processing (Azure)**: - - Extract text from scanned PDFs using Azure Document Intelligence -- **Metadata Extraction (AI Provider)**: - - Use any supported AI provider (OpenAI, Anthropic, Gemini, Ollama, etc.) to classify, label, or otherwise enrich the text with structured metadata -- **PDF Conversion (Gotenberg)**: - - Convert non-PDF attachments (e.g., Word docs, images) into PDFs -- **Document Management (Paperless NGX)**: - - Store processed documents and metadata in a Paperless NGX instance -- **IMAP Integration**: - - Fetch documents from multiple mailboxes (including Gmail) and automatically enqueue them for processing -- **Authentication**: - - Secure access to the system using **Authentik** for OAuth2-based login +### Document Processing +- **Multi-engine OCR** with quality checks and configurable merge strategies (AI merge, longest, primary) +- **AI metadata extraction** using any supported provider (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Portkey, Azure OpenAI) +- **PDF conversion** via Gotenberg with optional PDF/A archival format +- **Duplicate detection** — exact (SHA-256) and near-duplicate (content similarity with vector embeddings) +- **Customizable pipelines** — define multi-step processing workflows with conditional routing rules -## Frameworks Used +### Document Management +- **Full-text search** powered by Meilisearch with saved searches +- **File detail view** with metadata, text preview, processing history, and similarity analysis +- **Shared links** for public document access with expiration controls +- **Bulk operations** — reprocess, delete, or reassign documents in batch -- **FastAPI**: High-performance web framework for APIs. -- **Celery**: Task queue for asynchronous processing. -- **Redis**: Message broker and result backend. -- **SQLAlchemy**: ORM for database interactions. -- **Tailwind CSS**: Utility-first CSS framework. -- **Docker**: Containerization for easy deployment. +### Multi-Channel Ingestion +- **Web UI** — drag-and-drop upload with real-time progress +- **Browser extension** — clip web pages or send files from Chrome, Firefox, Edge ([guide](docs/BrowserExtension.md)) +- **Mobile app** — iOS and Android with camera capture, push notifications, and SSO ([guide](docs/MobileApp.md)) +- **CLI tool** — batch uploads, downloads, search, and API-token management ([guide](docs/CLIGuide.md)) +- **REST API & GraphQL** — full programmatic access with Swagger documentation at `/docs` +- **IMAP email** — poll multiple mailboxes with attachment filtering and auto-processing +- **Watched folders** — local filesystem, FTP, SFTP, and cloud storage providers + +### Administration +- **Multi-user mode** with per-user document isolation and ownership +- **Subscription & billing** — Stripe integration with configurable plans and quotas +- **Scheduled jobs** — IMAP polling, watched folder scans, automated backups, uptime monitoring +- **Audit logging** with SIEM integration support +- **Compliance templates** — GDPR, HIPAA, SOC 2 +- **Admin dashboard** — user management, queue monitoring, credential management, backup/restore + +### Authentication & Security +- **Local accounts** with self-service registration and password reset +- **OAuth2/OIDC** via Authentik or any OIDC provider +- **Social login** — Google, Microsoft, Apple, Dropbox +- **API tokens** for CLI, mobile, and automation access +- **Security headers** — HSTS, CSP, X-Frame-Options, X-Content-Type-Options +- **Rate limiting** with configurable per-endpoint controls + +### Notifications +- **100+ notification backends** via Apprise — Discord, Telegram, Slack, Microsoft Teams, Email, webhooks, and more +- **Configurable events** — task failures, credential issues, file processed, user signup, payment issues +- **In-app notification inbox** with per-user preferences +- **Webhooks** — push events to external systems with HMAC signature verification and retry + +## Tech Stack + +| Component | Technology | +|-----------|-----------| +| **Backend** | FastAPI, Celery, Redis, SQLAlchemy, Alembic | +| **Frontend** | Jinja2, Tailwind CSS | +| **Search** | Meilisearch | +| **Mobile** | React Native (Expo) — iOS & Android | +| **AI** | LiteLLM (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Portkey) | +| **OCR** | Azure Document Intelligence, Tesseract, EasyOCR, Mistral, Google Doc AI, AWS Textract | +| **PDF** | Gotenberg, pypdf | +| **Auth** | Authlib (OAuth2/OIDC), MSAL, social providers | +| **Infrastructure** | Docker, Docker Compose, Helm/Kubernetes | +| **Docs** | MkDocs Material | ## Quick Start -For detailed installation and deployment instructions, please refer to the [Deployment Guide](docs/DeploymentGuide.md). +For detailed installation and deployment instructions, see the [Deployment Guide](docs/DeploymentGuide.md). ```bash # Clone the repository @@ -147,20 +180,96 @@ cd DocuElevate # Configure environment variables cp .env.demo .env -# Edit .env with your settings +# Edit .env with your settings (see Configuration Guide for all options) # Run with Docker Compose -docker-compose up -d +docker compose up -d ``` -The API will be available at **`http://localhost:8000`**, and the API documentation is available at **`http://localhost:8000/docs`**. +The web UI is available at **`http://localhost:8000`** and the interactive API documentation at **`http://localhost:8000/docs`**. + +### Kubernetes / Helm + +```bash +helm repo add docuelevate https://christianlouis.github.io/DocuElevate +helm install docuelevate docuelevate/docuelevate -f values.yaml +``` + +See the [Kubernetes Deployment Guide](docs/KubernetesDeployment.md) for full details. + +## Documentation + +### Getting Started + +| Guide | Description | +|-------|-------------| +| [Setup Wizard](docs/SetupWizard.md) | Interactive first-run setup | +| [User Guide](docs/UserGuide.md) | How to use DocuElevate | +| [Browser Extension](docs/BrowserExtension.md) | Install and use the browser extension | +| [Mobile App](docs/MobileApp.md) | iOS and Android mobile app | +| [CLI Guide](docs/CLIGuide.md) | Command-line tool for automation | + +### How-To Guides + +| Guide | Description | +|-------|-------------| +| [How-To Overview](docs/HowToGuides.md) | Index of all how-to guides | +| [Email Ingestion](docs/howto/EmailIngestion.md) | Set up IMAP email polling | +| [Watched Folder](docs/howto/WatchedFolderSetup.md) | Monitor local or remote folders | +| [Mobile Scanning](docs/howto/MobileScanning.md) | Scan documents with your phone | + +### Reference + +| Guide | Description | +|-------|-------------| +| [API Documentation](docs/API.md) | REST & GraphQL API reference | +| [Configuration Guide](docs/ConfigurationGuide.md) | All environment variables | +| [Configuration Master](docs/ConfigurationMaster.md) | Configuration overview | +| [Settings Management](docs/SettingsManagement.md) | Runtime settings UI | + +### Deployment & Operations + +| Guide | Description | +|-------|-------------| +| [Deployment Guide](docs/DeploymentGuide.md) | Docker Compose deployment | +| [Kubernetes / Helm](docs/KubernetesDeployment.md) | Kubernetes deployment with Helm charts | +| [Production Readiness](docs/ProductionReadiness.md) | Checklist for production environments | +| [Database Configuration](docs/DatabaseConfiguration.md) | Database setup and migration | +| [Backup & Restore](docs/ConfigurationGuide.md#backup--restore) | Automated backup configuration | + +### Storage Integration Setup + +| Guide | Description | +|-------|-------------| +| [Dropbox](docs/DropboxSetup.md) | Dropbox OAuth setup | +| [Google Drive](docs/GoogleDriveSetup.md) | Google Drive service account / OAuth | +| [OneDrive](docs/OneDriveSetup.md) | Microsoft OneDrive setup | +| [Amazon S3](docs/AmazonS3Setup.md) | S3 bucket configuration | +| [Authentication](docs/AuthenticationSetup.md) | OAuth2, OIDC, and social login | +| [Notifications](docs/NotificationsSetup.md) | Notification backend setup | + +### Security & Compliance + +| Guide | Description | +|-------|-------------| +| [Credential Rotation](docs/CredentialRotationGuide.md) | Rotate secrets safely | +| [Licensing Compliance](docs/LicensingCompliance.md) | Dependency licenses | +| [Privacy & GDPR](docs/PrivacyCompliance.md) | Privacy compliance | + +### Development + +| Guide | Description | +|-------|-------------| +| [Contributing](CONTRIBUTING.md) | Code style, commits, and PR process | +| [Troubleshooting](docs/Troubleshooting.md) | Common issues and solutions | +| [Configuration Troubleshooting](docs/ConfigurationTroubleshooting.md) | Configuration-specific issues | +| [Build Metadata](docs/BuildMetadata.md) | Version and build information | +| [Internationalization](docs/InternationalizationGuide.md) | Translation and localization | ## Development & Testing ### Running Tests -DocuElevate includes comprehensive test coverage. To run tests: - ```bash # Install development dependencies pip install -r requirements-dev.txt @@ -175,21 +284,21 @@ pytest --cov=app --cov-report=term-missing pytest -m unit ``` -Tests are automatically configured with the necessary environment variables - **no manual setup required!** +Tests are automatically configured with the necessary environment variables — **no manual setup required!** -For detailed testing information, including integration tests with Docker and authentication testing, see the [Contributing Guide](CONTRIBUTING.md#running-tests). +For detailed testing information, see the [Contributing Guide](CONTRIBUTING.md#running-tests). ### Contributing We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for: -- Code style guidelines +- Code style guidelines (Ruff for formatting and linting) - Commit message format (Conventional Commits) - Testing requirements - Pull request process ## License -This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details. +This project is licensed under the Apache License 2.0 — see the [LICENSE](LICENSE) file for details. ## Third-Party Software @@ -214,13 +323,10 @@ The following is a summary of the licenses used by our direct dependencies: | Uvicorn | BSD | | SQLAlchemy | MIT | | Pydantic | MIT | -| openai | MIT | | litellm | MIT | | pypdf | BSD | | Requests | Apache 2.0 | -| puremagic | MIT | -| filetype | MIT | -| Dropbox | MIT | +| Dropbox SDK | MIT | | Azure AI Document Intelligence | MIT | | Authlib | BSD | | Starlette | BSD | @@ -229,15 +335,15 @@ The following is a summary of the licenses used by our direct dependencies: | Microsoft Graph Core | MIT | | MSAL | MIT | | Boto3 | Apache 2.0 | -| Paramiko | LGPL-2.1| +| Paramiko | LGPL-2.1 | | Apprise | MIT | -| Redis | BSD | -| Gotenberg | MIT | +| Redis (py) | BSD | +| Gotenberg Client | MIT | +| Meilisearch | MIT | For a comprehensive list of all dependencies and their licenses, run: -``` +```bash pip install pip-licenses pip-licenses - ``` diff --git a/docs/Troubleshooting.md b/docs/Troubleshooting.md index 5cdded08..6a322785 100644 --- a/docs/Troubleshooting.md +++ b/docs/Troubleshooting.md @@ -2,6 +2,8 @@ This document provides solutions to common problems encountered when using DocuElevate. +> **Tip:** For configuration-specific issues, see also the [Configuration Troubleshooting](ConfigurationTroubleshooting.md) guide. + ## Common Issues ### Application Won't Start @@ -14,7 +16,7 @@ This document provides solutions to common problems encountered when using DocuE #### Possible Solutions 1. **Check environment variables** ```bash - docker-compose config + docker compose config ``` Ensure all required variables are set properly in your `.env` file. @@ -30,6 +32,18 @@ This document provides solutions to common problems encountered when using DocuE ``` Ensure the port isn't already in use by another application. +4. **Check Redis connectivity** + ```bash + docker compose logs redis + ``` + Ensure Redis is running — both the API server and Celery worker depend on it. + +5. **Check database migrations** + ```bash + docker compose exec api alembic upgrade head + ``` + Ensure the database schema is up-to-date. + ### Document Upload Fails #### Symptoms @@ -39,41 +53,51 @@ This document provides solutions to common problems encountered when using DocuE #### Possible Solutions 1. **Check file size limits** - - Default maximum file size is 100MB - - Adjust `client_max_body_size` in your reverse proxy configuration + - Default maximum file size is 1 GB (`MAX_UPLOAD_SIZE`) + - Individual file limit: `MAX_SINGLE_FILE_SIZE` (default: same as `MAX_UPLOAD_SIZE`) + - If using a reverse proxy, adjust `client_max_body_size` (Nginx) or equivalent 2. **Verify storage space** ```bash df -h ``` - Ensure there's sufficient disk space available. + Ensure there's sufficient disk space on the workdir volume. 3. **Check worker process** ```bash - docker-compose logs worker + docker compose logs worker ``` Verify the Celery worker is running and processing tasks. +4. **Check upload quota** + If multi-user mode and subscriptions are enabled, verify the user hasn't exceeded their daily upload limit (`DEFAULT_DAILY_UPLOAD_LIMIT`). + ### OCR or Text Extraction Issues #### Symptoms - Documents upload but text isn't extracted - Poor quality text extraction -- API errors related to Azure services +- API errors related to OCR services #### Possible Solutions -1. **Verify API credentials** - Check the Azure Document Intelligence API key and endpoint in your `.env` file. +1. **Verify the configured OCR provider** + Check which provider is set via the `OCR_PROVIDER` environment variable (defaults to Azure Document Intelligence). -2. **Check document quality** +2. **Verify API credentials** + Check the credentials for your configured OCR provider in your `.env` file: + - **Azure**: `AZURE_DI_KEY` and `AZURE_DI_ENDPOINT` + - **Tesseract**: No credentials required (local), but ensure `TESSERACT_LANGUAGES` is set + - **EasyOCR**: No credentials required (local) + - **Mistral**: `MISTRAL_OCR_API_KEY` + - **Google Document AI**: `GOOGLE_DOCAI_PROJECT_ID`, `GOOGLE_DOCAI_LOCATION`, `GOOGLE_DOCAI_PROCESSOR_ID` + - **AWS Textract**: `AWS_TEXTRACT_ACCESS_KEY_ID`, `AWS_TEXTRACT_SECRET_ACCESS_KEY`, `AWS_TEXTRACT_REGION` + +3. **Check document quality** - Ensure documents are clearly scanned - Try preprocessing images to improve quality before upload -3. **Test API connectivity** - ```bash - curl -X GET -H "Ocp-Apim-Subscription-Key: YOUR_KEY" "YOUR_ENDPOINT" - ``` - Ensure the API is accessible from your server. +4. **Try multi-provider OCR** + Configure `OCR_PROVIDERS` (comma-separated list) with a merge strategy (`OCR_MERGE_STRATEGY`: `ai_merge`, `longest`, or `primary`) for better results. ### Email Integration Problems @@ -84,92 +108,226 @@ This document provides solutions to common problems encountered when using DocuE #### Possible Solutions 1. **Verify IMAP settings** - Check host, port, username, and password in your configuration. + Check host, port, username, and password for `IMAP1_*` / `IMAP2_*` in your configuration. 2. **Test IMAP connectivity** ```bash - telnet mail.example.com 993 + docker compose exec api python -c "import imaplib; m = imaplib.IMAP4_SSL('mail.example.com', 993); print('OK')" ``` - Ensure the IMAP server is accessible. + Ensure the IMAP server is accessible from the container. -3. **Enable less secure apps** - For Gmail and some providers, you may need to enable access for less secure apps or use app-specific passwords. +3. **Check for app-specific passwords** + For Gmail and some providers, you must use app-specific passwords instead of your account password. 4. **Check firewall settings** - Ensure your server can make outbound connections to the mail server. + Ensure your server can make outbound connections to the mail server on port 993 (IMAP SSL). + +5. **Check attachment filter** + If only certain attachments are expected, verify `IMAP_ATTACHMENT_FILTER` is set correctly (`documents_only` or `all`). ### Storage Integration Issues #### Symptoms -- Files aren't appearing in Dropbox/Nextcloud/Paperless +- Files aren't appearing in configured storage destinations - Authentication errors in logs - API rate limiting errors #### Possible Solutions 1. **Verify API credentials** - Double-check all API keys, tokens, and secrets. + Double-check all API keys, tokens, and secrets for the relevant service. 2. **Check access permissions** - Ensure the application has write permissions to the specified folders. + Ensure the application has write permissions to the specified folders/buckets. 3. **Refresh tokens** - For OAuth-based services like Dropbox, try generating new refresh tokens. + For OAuth-based services like Dropbox, Google Drive, and OneDrive, try re-authorizing through the integration setup pages. 4. **Examine detailed logs** ```bash - docker-compose logs worker | grep -i dropbox + docker compose logs worker | grep -i "upload_to" ``` - Look for specific error messages related to the service. + Look for specific error messages related to the storage service. + +5. **Check integration status** + Visit the **Integrations** page in the web UI to verify the connection status of each configured storage backend. + +## Search Issues + +### Symptoms +- Search returns no results or incomplete results +- Search page shows an error + +### Possible Solutions +1. **Check Meilisearch is running** + ```bash + docker compose logs meilisearch + ``` + Ensure the Meilisearch container is healthy and accepting connections. + +2. **Verify Meilisearch URL** + Check `MEILISEARCH_URL` in your `.env` file (default: `http://meilisearch:7700`). + +3. **Rebuild the search index** + If documents are missing from search results, reprocessing them will re-index their content. + +## Pipeline & Routing Issues + +### Symptoms +- Documents are not processed according to pipeline steps +- Routing rules don't match expected documents + +### Possible Solutions +1. **Verify pipeline assignment** + On the file detail page, check which pipeline (if any) is assigned. The system pipeline applies to all documents by default. + +2. **Test routing rules** + Use the **Evaluate** button on the Routing Rules page to test whether a rule matches a specific document. + +3. **Check step ordering** + Pipeline steps execute in order — ensure OCR comes before metadata extraction if the AI step depends on extracted text. ## Database Issues -#### Symptoms +### Symptoms - Application errors related to database connections - Missing or corrupt data - Slow performance -#### Possible Solutions +### Possible Solutions 1. **Check database connection string** Verify the `DATABASE_URL` variable in your `.env` file. 2. **Inspect database integrity** + For SQLite: ```bash sqlite3 database.db "PRAGMA integrity_check;" ``` - (For SQLite databases) + For PostgreSQL (recommended for production): + ```bash + docker compose exec api python -c "from app.database import engine; print(engine.url)" + ``` 3. **Perform database migrations** ```bash - docker-compose exec api alembic upgrade head + docker compose exec api alembic upgrade head ``` Ensure the database schema is up-to-date. +4. **Consider PostgreSQL for production** + SQLite is suitable for small deployments, but PostgreSQL is recommended for multi-user production environments. See the [Database Configuration Guide](DatabaseConfiguration.md). + ## Authentication Problems -#### Symptoms +### Symptoms - Unable to log in - Redirect loops during authentication - OAuth errors -#### Possible Solutions -1. **Verify Authentik configuration** - Check client ID, client secret, and configuration URL. +### Possible Solutions +1. **Verify OAuth/OIDC configuration** + Check client ID, client secret, and configuration URL for your identity provider. 2. **Check callback URLs** - Ensure the redirect URIs are correctly configured in your OAuth provider. + Ensure the redirect URIs are correctly configured in your OAuth provider. The callback URL is typically `https://your-domain/auth/callback`. 3. **Clear browser cookies and cache** Authentication issues can sometimes be resolved by clearing browser data. +4. **Check social login credentials** + If using social login (Google, Microsoft, Apple, Dropbox), verify the corresponding `SOCIAL_AUTH_*` environment variables. + +5. **Verify `EXTERNAL_HOSTNAME`** + The `EXTERNAL_HOSTNAME` setting must match the domain users access DocuElevate from — OAuth redirect URLs depend on it. + +## Mobile App Issues + +### Symptoms +- Can't connect to DocuElevate from the mobile app +- Push notifications not received +- Login fails + +### Possible Solutions +1. **Verify the server URL** + Ensure the mobile app is configured with the correct DocuElevate server URL (including `https://`). + +2. **Check API token** + Generate a fresh API token from the web UI (Profile → API Tokens) and enter it in the mobile app settings. + +3. **Check network connectivity** + The mobile device must be able to reach your DocuElevate server. If using a private network, ensure VPN is connected. + +4. **Push notifications** + Push notifications require a valid Expo push token. Check the app settings and ensure notifications are enabled at the OS level. + +See the [Mobile App Guide](MobileApp.md) for detailed setup instructions. + +## CLI Issues + +### Symptoms +- CLI commands fail with connection errors +- Authentication rejected + +### Possible Solutions +1. **Verify URL and token** + ```bash + docuelevate --url https://your-instance --token de_xxx list + ``` + Ensure the URL is correct (include the scheme) and the API token is valid. + +2. **Check environment variables** + The CLI reads `DOCUELEVATE_URL` and `DOCUELEVATE_API_TOKEN` from the environment. Verify they are exported. + +3. **Test API directly** + ```bash + curl -H "Authorization: Bearer de_xxx" https://your-instance/api/files + ``` + If this fails, the issue is with the server, not the CLI. + +See the [CLI Guide](CLIGuide.md) for detailed usage. + +## Performance Issues + +### Symptoms +- Slow document processing +- High memory usage +- Queue backing up + +### Possible Solutions +1. **Check worker concurrency** + The Celery worker processes tasks in parallel. If the queue is backing up, consider scaling workers or adjusting concurrency. + +2. **Enable batch throttling** + Set `PROCESSALL_THROTTLE_THRESHOLD` and `PROCESSALL_THROTTLE_DELAY` to prevent overwhelming external APIs. + +3. **Monitor the queue** + Visit the **Admin → Queue** page to see pending, active, and failed tasks. + +4. **Use PostgreSQL** + SQLite can become a bottleneck under load. Migrate to PostgreSQL for better concurrent performance. See the [Database Configuration Guide](DatabaseConfiguration.md). + +5. **Check Redis memory** + ```bash + docker compose exec redis redis-cli info memory + ``` + Ensure Redis has sufficient memory for the task queue and cache. + ## Getting Additional Help If you continue to experience issues after trying these solutions: -1. **Check the logs** for detailed error messages +1. **Check the logs** for detailed error messages: ```bash - docker-compose logs --tail=100 + docker compose logs --tail=200 ``` -2. **Open an issue** on the [GitHub repository](https://github.com/christianlouis/document-processor/issues) +2. **Check the status page** at `/status` in the web UI for an overview of all service connections. -3. **Contact the developer** via the information provided on the About page +3. **Open an issue** on the [GitHub repository](https://github.com/christianlouis/DocuElevate/issues) with: + - A description of the problem + - Relevant log output + - Your DocuElevate version (shown on the About page or in the `VERSION` file) + +4. **Consult additional documentation**: + - [Configuration Guide](ConfigurationGuide.md) — All environment variables + - [Configuration Troubleshooting](ConfigurationTroubleshooting.md) — Configuration-specific issues + - [Deployment Guide](DeploymentGuide.md) — Infrastructure and deployment diff --git a/docs/UserGuide.md b/docs/UserGuide.md index 957d005d..8d6b0a82 100644 --- a/docs/UserGuide.md +++ b/docs/UserGuide.md @@ -63,6 +63,15 @@ DocuElevate features a simple navigation system with the following main sections - **Search**: Dedicated full-text search across all document content - **About**: Information about DocuElevate +### Other Ways to Use DocuElevate + +Beyond the web interface, DocuElevate is available through several additional clients: + +- **Mobile App (iOS & Android)** — Capture documents with your phone camera or upload from your photo library. See the [Mobile App Guide](MobileApp.md) for setup and usage. +- **Browser Extension** — Clip web pages or send files to DocuElevate directly from Chrome, Firefox, or Edge. See the [Browser Extension Guide](BrowserExtension.md) for installation. +- **CLI Tool** — Upload, download, search, and manage documents from the command line or scripts. See the [CLI Guide](CLIGuide.md) for details. +- **REST & GraphQL API** — Full programmatic access for automation and integrations. See the [API Documentation](API.md). + ## Uploading Documents DocuElevate provides multiple convenient ways to upload documents to the system.