gitea-admin/gh-christianlouis-docuelevate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add comprehensive documentation including user guide, API reference, and deployment instructions

Browse Source
This commit is contained in:
Christian Krakau-Louis
2025-03-31 10:58:01 +02:00
parent 92a387a759
commit 5aff586fb0
7 changed files with 635 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTRIBUTING.md
+80
View File
@@ -0,0 +1,80 @@
# Contributing to DocuNova
Thank you for your interest in contributing to DocuNova! This document provides guidelines and instructions for contributing to the project.
## Code of Conduct
By participating in this project, you agree to abide by the [Code of Conduct](CODE_OF_CONDUCT.md).
## How to Contribute
### Reporting Bugs
If you find a bug in the codebase, please submit an issue on GitHub with:
1. A clear title and description
2. Steps to reproduce the issue
3. Expected behavior
4. Actual behavior
5. Environment information (OS, Docker version, etc.)
### Feature Requests
We welcome feature requests! Please submit an issue with:
1. A clear title and description
2. The problem the feature would solve
3. Any ideas you have for implementing the feature
### Pull Requests
1. Fork the repository
2. Create a new branch for your changes
3. Make your changes
4. Run the tests to ensure everything works
5. Submit a pull request with a clear description of the changes
## Development Environment
### Setting Up Your Environment
```bash
# Clone the repository
git clone https://github.com/christianlouis/document-processor.git
cd document-processor
# Create a virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt
```
### Running Tests
```bash
pytest
```
### Code Style
We use:
- Black for Python code formatting
- Flake8 for linting
- isort for import sorting
```bash
# Format code
black .
# Check linting
flake8
# Sort imports
isort .
```
## Project Structure
docs/API.md
+127
View File
@@ -0,0 +1,127 @@
# API Documentation
DocuNova provides a powerful REST API for programmatic access to all its features. This document serves as a reference for the available endpoints and their usage.
## API Overview
- Base URL: `http://<your-docunova-instance>/api`
- Authentication: OAuth2 (when enabled)
- Response Format: JSON
## Interactive API Documentation
The most up-to-date and interactive API documentation is available at:
`http://<your-docunova-instance>/docs`
This Swagger UI provides a complete reference with the ability to try out API calls directly from your browser.
## Authentication
When authentication is enabled, you must include an authentication token in your requests:
```bash
curl -X GET "http://<your-docunova-instance>/api/files" \
-H "Authorization: Bearer <your-token>"
```
## Common Endpoints
### Document Upload
**POST** `/api/upload`
Upload one or more files for processing.
**Request**:
- Multipart form data with file(s)
**Response**:
```json
{
"success": true,
"file_ids": [123, 124],
"message": "Files uploaded and queued for processing"
}
```
### Get Files
**GET** `/api/files`
Retrieve a list of processed files.
**Parameters**:
- `limit` (optional): Maximum number of files to return
- `offset` (optional): Pagination offset
- `search` (optional): Search term
**Response**:
```json
[
{
"id": 123,
"original_filename": "invoice.pdf",
"file_size": 1024000,
"mime_type": "application/pdf",
"created_at": "2023-04-15T12:30:45Z"
},
...
]
```
### File Metadata
**GET** `/api/files/{file_id}/metadata`
Retrieve metadata for a specific file.
**Response**:
```json
{
"document_type": "invoice",
"date": "2023-04-10",
"vendor": "Acme Corp",
"amount": "$1,234.56",
"extracted_text": "..."
}
```
### Process Control
**POST** `/api/files/{file_id}/reprocess`
Reprocess a specific file.
**Response**:
```json
{
"success": true,
"message": "File queued for reprocessing"
}
```
## Error Handling
Errors follow standard HTTP status codes with descriptive messages:
```json
{
"detail": "File not found",
"status_code": 404
}
```
## Rate Limiting
The API implements rate limiting to ensure system stability. If you exceed the limits, you'll receive a `429 Too Many Requests` response.
## Client Libraries
For easy integration with your applications, consider using one of our client libraries:
- Python: [github.com/christianlouis/docunova-python-client](https://github.com/christianlouis/docunova-python-client)
## Further Assistance
For additional help with the API, please contact our support team or refer to the [Development Guide](../CONTRIBUTING.md).
docs/ConfigurationGuide.md
+8 -11
View File
@@ -28,17 +28,6 @@ DocuNova can monitor multiple IMAP mailboxes for document attachments. Each mail
| `IMAP1_PASSWORD` | IMAP password (first mailbox). | `*******` | | `IMAP1_PASSWORD` | IMAP password (first mailbox). | `*******` |
| `IMAP1_SSL` | Use SSL (`true`/`false`). | `true` | | `IMAP1_SSL` | Use SSL (`true`/`false`). | `true` |
| `IMAP1_POLL_INTERVAL_MINUTES` | Frequency in minutes to poll for new mail. | `5` | | `IMAP1_POLL_INTERVAL_MINUTES` | Frequency in minutes to poll for new mail. | `5` |
| `IMAP1_DELETE_AFTER_PROCESS` | Delete emails after processing (`true`/`false`). | `false` |
To configure additional mailboxes, use `IMAP2_`, `IMAP3_`, etc.
### OpenAI & Azure Document Intelligence
| **Variable** | **Description** | **How to Obtain** |
|---------------------------------|------------------------------------------|--------------------------------------------------------------------------|
| `OPENAI_API_KEY` | OpenAI API key for GPT metadata extraction. | [OpenAI API keys](https://platform.openai.com/account/api-keys) |
| `AZURE_DOCUMENT_INTELLIGENCE_KEY` | Azure Document Intelligence API key for OCR. | [Azure Portal](https://portal.azure.com/) |
| `AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT` | Endpoint URL for Azure Doc Intelligence API. | [Azure Portal](https://portal.azure.com/) |
### Authentication ### Authentication
@@ -49,6 +38,14 @@ To configure additional mailboxes, use `IMAP2_`, `IMAP3_`, etc.
| `AUTHENTIK_CLIENT_SECRET` | Client secret for Authentik OAuth2. | | `AUTHENTIK_CLIENT_SECRET` | Client secret for Authentik OAuth2. |
| `AUTHENTIK_CONFIG_URL` | Configuration URL for Authentik OpenID Connect. | | `AUTHENTIK_CONFIG_URL` | Configuration URL for Authentik OpenID Connect. |
### OpenAI & Azure Document Intelligence
| **Variable** | **Description** | **How to Obtain** |
|---------------------------------|------------------------------------------|--------------------------------------------------------------------------|
| `OPENAI_API_KEY` | OpenAI API key for GPT metadata extraction. | [OpenAI API keys](https://platform.openai.com/account/api-keys) |
| `AZURE_DOCUMENT_INTELLIGENCE_KEY` | Azure Document Intelligence API key for OCR. | [Azure Portal](https://portal.azure.com/) |
| `AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT` | Endpoint URL for Azure Doc Intelligence API. | [Azure Portal](https://portal.azure.com/) |
### Paperless NGX ### Paperless NGX
| **Variable** | **Description** | | **Variable** | **Description** |
docs/DeploymentGuide.md
+142
View File
@@ -0,0 +1,142 @@
# Deployment Guide
This guide provides instructions for deploying DocuNova in various environments.
## Prerequisites
- Docker and Docker Compose
- Access to required external services (if configured):
- OpenAI API
- Azure Document Intelligence
- Dropbox API
- Nextcloud instance
- Paperless NGX instance
- SMTP server (for email notifications)
- IMAP server(s) (for email attachment processing)
## Docker Deployment
Docker is the recommended deployment method for DocuNova.
### Step 1: Clone the Repository
```bash
git clone https://github.com/christianlouis/document-processor.git
cd document-processor
```
### Step 2: Configure Environment Variables
Create a `.env` file based on the example:
```bash
cp .env.example .env
```
Edit the `.env` file with your configuration settings. See the [Configuration Guide](ConfigurationGuide.md) for details.
### Step 3: Run with Docker Compose
```bash
docker-compose up -d
```
This will start:
- The DocuNova API server
- A worker for background tasks
- Redis for message broker and result storage
- Gotenberg for PDF processing
### Step 4: Verify the Installation
Access the web interface at `http://localhost:8000` and the API documentation at `http://localhost:8000/docs`.
## Production Considerations
### Reverse Proxy Setup
For production use, we recommend setting up a reverse proxy (like Nginx or Traefik) to handle HTTPS and domain routing:
```nginx
server {
listen 80;
server_name docunova.example.com;
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
### Persistent Storage
The Docker setup uses volumes for persistent storage. For production, consider:
```yaml
volumes:
- /path/to/persistent/storage:/workdir
```
### Security
1. **Always use HTTPS** in production
2. Enable authentication by setting `AUTH_ENABLED=true`
3. Use strong passwords for all services
4. Limit access to the Docker host
5. Regularly update the application and dependencies
## Scaling
For high-volume deployments:
1. Increase worker processes by adding more worker containers:
```yaml
worker:
image: christianlouis/document-processor:latest
deploy:
replicas: 3
```
2. Consider using dedicated Redis and database servers
3. Monitor system performance and adjust resources as needed
## Monitoring
Monitor your DocuNova deployment using:
- Docker's built-in logging: `docker-compose logs -f`
- Container metrics: `docker stats`
- External monitoring tools like Prometheus and Grafana
## Backup Procedures
Regularly back up the following:
1. The `/workdir` directory containing all processed documents
2. The database file (if using SQLite) or database contents (if using another DBMS)
3. The `.env` configuration file
## Updates
To update DocuNova to a newer version:
```bash
# Pull the latest changes
git pull
# Pull the latest Docker images
docker-compose pull
# Restart the services
docker-compose down
docker-compose up -d
```
## Troubleshooting
See the [Troubleshooting](Troubleshooting.md) guide for common deployment issues and solutions.
docs/README.md
+34
View File
@@ -0,0 +1,34 @@
# DocuNova Documentation
Welcome to the DocuNova documentation. This directory contains comprehensive guides to help you install, configure, and use DocuNova effectively.
## Available Documentation
- [User Guide](UserGuide.md) - How to use DocuNova's features and interface
- [API Documentation](API.md) - Complete API reference for developers
- [Deployment Guide](DeploymentGuide.md) - How to deploy DocuNova in various environments
- [Configuration Guide](ConfigurationGuide.md) - All available configuration options
- [Troubleshooting](Troubleshooting.md) - Solutions to common issues
## Additional Resources
- [Contributing Guide](../CONTRIBUTING.md) - How to contribute to DocuNova development
- [License Information](../LICENSE) - Apache License 2.0 details
## Support
If you need additional assistance beyond what's covered in these guides:
1. Check the [GitHub repository](https://github.com/christianlouis/document-processor) for updates and issues
2. Contact the developer through the information provided on the About page
## Screenshots
![Upload Interface](upload-view.png)
*DocuNova's upload interface*
![Files View](files-view.png)
*Document management interface*
![Workflow Diagram](workflow-diagram.png)
*DocuNova processing workflow*
docs/Troubleshooting.md
+175
View File
@@ -0,0 +1,175 @@
# Troubleshooting Guide
This document provides solutions to common problems encountered when using DocuNova.
## Common Issues
### Application Won't Start
#### Symptoms
- Docker containers exit immediately
- Web interface not accessible
- Error logs show startup failures
#### Possible Solutions
1. **Check environment variables**
```bash
docker-compose config
```
Ensure all required variables are set properly in your `.env` file.
2. **Verify permissions**
```bash
ls -la /path/to/workdir
```
Make sure the application has write permissions to the working directory.
3. **Check port availability**
```bash
netstat -tuln | grep 8000
```
Ensure the port isn't already in use by another application.
### Document Upload Fails
#### Symptoms
- Error messages during upload
- Files appear to upload but aren't processed
- Browser console errors
#### Possible Solutions
1. **Check file size limits**
- Default maximum file size is 100MB
- Adjust `client_max_body_size` in your reverse proxy configuration
2. **Verify storage space**
```bash
df -h
```
Ensure there's sufficient disk space available.
3. **Check worker process**
```bash
docker-compose logs worker
```
Verify the Celery worker is running and processing tasks.
### OCR or Text Extraction Issues
#### Symptoms
- Documents upload but text isn't extracted
- Poor quality text extraction
- API errors related to Azure services
#### Possible Solutions
1. **Verify API credentials**
Check the Azure Document Intelligence API key and endpoint in your `.env` file.
2. **Check document quality**
- Ensure documents are clearly scanned
- Try preprocessing images to improve quality before upload
3. **Test API connectivity**
```bash
curl -X GET -H "Ocp-Apim-Subscription-Key: YOUR_KEY" "YOUR_ENDPOINT"
```
Ensure the API is accessible from your server.
### Email Integration Problems
#### Symptoms
- Email attachments aren't being processed
- IMAP connection errors in logs
- Authentication failures
#### Possible Solutions
1. **Verify IMAP settings**
Check host, port, username, and password in your configuration.
2. **Test IMAP connectivity**
```bash
telnet mail.example.com 993
```
Ensure the IMAP server is accessible.
3. **Enable less secure apps**
For Gmail and some providers, you may need to enable access for less secure apps or use app-specific passwords.
4. **Check firewall settings**
Ensure your server can make outbound connections to the mail server.
### Storage Integration Issues
#### Symptoms
- Files aren't appearing in Dropbox/Nextcloud/Paperless
- Authentication errors in logs
- API rate limiting errors
#### Possible Solutions
1. **Verify API credentials**
Double-check all API keys, tokens, and secrets.
2. **Check access permissions**
Ensure the application has write permissions to the specified folders.
3. **Refresh tokens**
For OAuth-based services like Dropbox, try generating new refresh tokens.
4. **Examine detailed logs**
```bash
docker-compose logs worker | grep -i dropbox
```
Look for specific error messages related to the service.
## Database Issues
#### Symptoms
- Application errors related to database connections
- Missing or corrupt data
- Slow performance
#### Possible Solutions
1. **Check database connection string**
Verify the `DATABASE_URL` variable in your `.env` file.
2. **Inspect database integrity**
```bash
sqlite3 database.db "PRAGMA integrity_check;"
```
(For SQLite databases)
3. **Perform database migrations**
```bash
docker-compose exec api alembic upgrade head
```
Ensure the database schema is up-to-date.
## Authentication Problems
#### Symptoms
- Unable to log in
- Redirect loops during authentication
- OAuth errors
#### Possible Solutions
1. **Verify Authentik configuration**
Check client ID, client secret, and configuration URL.
2. **Check callback URLs**
Ensure the redirect URIs are correctly configured in your OAuth provider.
3. **Clear browser cookies and cache**
Authentication issues can sometimes be resolved by clearing browser data.
## Getting Additional Help
If you continue to experience issues after trying these solutions:
1. **Check the logs** for detailed error messages
```bash
docker-compose logs --tail=100
```
2. **Open an issue** on the [GitHub repository](https://github.com/christianlouis/document-processor/issues)
3. **Contact the developer** via the information provided on the About page
docs/UserGuide.md
+69
View File
@@ -0,0 +1,69 @@
# User Guide
This guide helps you get started with using DocuNova for document management and processing.
## Getting Started
DocuNova offers an intuitive web interface for uploading, managing, and processing your documents.
### Accessing the System
1. Navigate to your DocuNova instance (typically at `http://your-server-address:8000`)
2. If authentication is enabled, you'll be prompted to log in using your credentials
### Main Interface
DocuNova features a simple navigation system with the following main sections:
- **Home**: Dashboard and overview
- **Upload**: For adding new documents to the system
- **Files**: For viewing and managing processed documents
- **About**: Information about DocuNova
## Uploading Documents
### Web Upload
1. Navigate to the **Upload** page
2. Drag and drop files onto the upload area, or click to browse your files
3. Select files to upload (supported formats include PDF, Word documents, images, etc.)
4. Click "Upload" to begin the process
5. Your documents will be processed automatically according to the system configuration
### Email Attachments
If configured, DocuNova can automatically fetch documents from email attachments:
1. Send an email with attachments to the configured email account
2. DocuNova will poll the mailbox at the configured interval
3. Attachments will be automatically downloaded and processed
4. No further action is required
## Managing Documents
The **Files** page provides access to all processed documents:
1. Navigate to the **Files** page
2. Use the search box to find specific documents
3. Click on any file to view its details
4. Sort the list by any column by clicking on the column header
## Document Processing Features
Depending on the system configuration, DocuNova can perform:
- **OCR** - Extract text from images and scanned PDFs
- **Metadata Extraction** - Automatically identify document types, dates, and other key information
- **PDF Conversion** - Convert various file formats to PDF
- **Document Distribution** - Store documents in Dropbox, Nextcloud, or Paperless NGX
## API Access
For programmatic access, DocuNova provides a comprehensive REST API:
1. Navigate to `/docs` on your DocuNova instance
2. The interactive Swagger documentation allows you to test API endpoints directly
3. Obtain an API token if authentication is enabled
## Troubleshooting
If you encounter issues while using DocuNova, please refer to the [Troubleshooting Guide](Troubleshooting.md).