copilot-swe-agent[bot]
b4e066bf2d
- Fixed XSS vulnerability in file detail template - Use tojson filter to safely encode file.id into JavaScript - CodeQL scan confirms no security issues Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Intelligent Document Processing & Management
DocuElevate
Overview
DocuElevate automates the handling, extraction, and processing of documents using a variety of services, including:
- OpenAI for metadata extraction and text refinement.
- Dropbox, Nextcloud, and Google Drive for file storage and uploads.
- Paperless NGX for document indexing and management.
- Azure Document Intelligence for OCR on PDFs.
- Gotenberg for file-to-PDF conversions.
- Authentik for authentication and user management.
It is designed for flexibility and configurability through environment variables, making it easily customizable for different workflows. The system can fetch documents from multiple IMAP mailboxes, process them (OCR, metadata extraction, PDF conversion), and store them in the desired destinations.
The project includes a UI for uploading and managing files, and an API documentation page is available at /docs (powered by FastAPI).
Documentation Index
- User Guide - How to use DocuElevate
- API Documentation - API reference
- Deployment Guide - How to deploy DocuElevate
- Configuration Guide - Available configuration options
- Build Metadata - Automated version and build information
- Development Guide - How to contribute to DocuElevate
- Troubleshooting - Common issues and solutions
Screenshots
Upload interface for adding new documents
Files view with processed documents and metadata
Workflow Process
DocuElevate follows a streamlined document processing workflow:
Document Ingestion
Documents enter DocuElevate through three possible channels:
- Web Upload: Users manually upload files via the web interface
- Email Attachments: Automatic polling of configured IMAP mailboxes (supports multiple accounts)
- API: Direct programmatic uploads via the REST API
Processing Pipeline
Every document goes through the following steps:
- PDF Conversion: Non-PDF files are converted to PDF format using Gotenberg
- OCR Processing: Azure Document Intelligence extracts text from images/scans
- Metadata Extraction: OpenAI analyzes document content to identify:
- Document type (invoice, receipt, contract, etc.)
- Key entities (dates, names, amounts, account numbers)
- Important data points specific to the document type
- Enrichment: Metadata is attached to the document in a structured format
Distribution
Processed documents with their metadata can be automatically sent to:
- Dropbox: For cloud storage and sharing
- Nextcloud: For self-hosted file storage
- Google Drive: For Google Workspace integration
- Paperless-NGX: For advanced document management with search capabilities
Users can choose to send documents to any combination of these destinations through configuration settings or manual selection.
Features
- Document Upload & Storage:
- Manual uploads (via API or UI) to Dropbox, Nextcloud, Google Drive, or Paperless.
- OCR Processing (Azure):
- Extract text from scanned PDFs using Azure Document Intelligence.
- Metadata Extraction (OpenAI):
- Use GPT to classify, label, or otherwise enrich the text with structured metadata.
- PDF Conversion (Gotenberg):
- Convert non-PDF attachments (e.g., Word docs, images) into PDFs.
- Document Management (Paperless NGX):
- Store processed documents and metadata in a Paperless NGX instance.
- IMAP Integration:
- Fetch documents from multiple mailboxes (including Gmail) and automatically enqueue them for processing.
- Authentication:
- Secure access to the system using Authentik for OAuth2-based login.
Frameworks Used
- FastAPI: High-performance web framework for APIs.
- Celery: Task queue for asynchronous processing.
- Redis: Message broker and result backend.
- SQLAlchemy: ORM for database interactions.
- Tailwind CSS: Utility-first CSS framework.
- Docker: Containerization for easy deployment.
Quick Start
For detailed installation and deployment instructions, please refer to the Deployment Guide.
# Clone the repository
git clone <repository_url>
cd document-processor
# Configure environment variables
cp .env.example .env
# Edit .env with your settings
# Run with Docker Compose
docker-compose up -d
The API will be available at http://localhost:8000, and the API documentation is available at http://localhost:8000/docs.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Third-Party Software
This project uses various third-party libraries and components. See NOTICE for attributions and the attribution page in the application for more details.
LGPL Compliance
This project uses Paramiko which is licensed under LGPL-2.1. In accordance with the LGPL license:
- The source code for Paramiko can be obtained from https://github.com/paramiko/paramiko
- A copy of the LGPL license is available in the application at
/licenses/lgpl.txt - Users have the right to modify and redistribute Paramiko under the terms of the LGPL
Dependency Licenses
The following is a summary of the licenses used by our direct dependencies:
| Dependency | License |
|---|---|
| FastAPI | MIT |
| Celery | BSD |
| Uvicorn | BSD |
| SQLAlchemy | MIT |
| Pydantic | MIT |
| OpenAI | MIT |
| PyPDF2 | BSD |
| Requests | Apache 2.0 |
| Dropbox | MIT |
| Azure AI Document Intelligence | MIT |
| Authlib | BSD |
| Starlette | BSD |
| Alembic | MIT |
| Google API Client | Apache 2.0 |
| Microsoft Graph Core | MIT |
| MSAL | MIT |
| Boto3 | Apache 2.0 |
| Paramiko | LGPL-2.1 |
| Apprise | MIT |
| Redis | BSD |
| Gotenberg | MIT |
For a comprehensive list of all dependencies and their licenses, run:
pip install pip-licenses
pip-licenses