feat(storage): add Evernote destination (#862)

* feat(storage): add Evernote destination

* style: apply ruff auto-fix

- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

---------

Co-authored-by: Christian Krakau-Louis <christian@Christians-Mac-mini-7.local>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Christian Krakau-Louis
2026-05-22 12:13:46 +02:00
committed by GitHub
parent 8e744c076d
commit 4b46c4baf8
20 changed files with 507 additions and 2 deletions
+4 -1
View File
@@ -35,7 +35,7 @@ DocuElevate is an intelligent document processing system that automates the inge
- **AI-Powered Metadata Extraction** — pluggable AI providers including OpenAI, Anthropic Claude, Google Gemini, Ollama (local), OpenRouter, Portkey, and Azure OpenAI via LiteLLM - **AI-Powered Metadata Extraction** — pluggable AI providers including OpenAI, Anthropic Claude, Google Gemini, Ollama (local), OpenRouter, Portkey, and Azure OpenAI via LiteLLM
- **Multi-Engine OCR** — Azure Document Intelligence, Tesseract, EasyOCR, Mistral OCR, Google Cloud Document AI, and AWS Textract with configurable merge strategies - **Multi-Engine OCR** — Azure Document Intelligence, Tesseract, EasyOCR, Mistral OCR, Google Cloud Document AI, and AWS Textract with configurable merge strategies
- **12 Storage Destinations** — Dropbox, Google Drive, OneDrive, Amazon S3, Nextcloud, WebDAV, FTP, SFTP, iCloud Drive, Email (SMTP), Paperless-ngx, and Rclone - **13 Storage Destinations** — Dropbox, Google Drive, OneDrive, Amazon S3, Nextcloud, WebDAV, FTP, SFTP, iCloud Drive, Email (SMTP), Paperless-ngx, Evernote, and Rclone
- **Multi-Channel Ingestion** — web upload, browser extension, mobile app, CLI, REST API, IMAP email, and watched folders (local, cloud, FTP/SFTP) - **Multi-Channel Ingestion** — web upload, browser extension, mobile app, CLI, REST API, IMAP email, and watched folders (local, cloud, FTP/SFTP)
- **Processing Pipelines** — customizable multi-step workflows with conditional routing rules - **Processing Pipelines** — customizable multi-step workflows with conditional routing rules
- **Full-Text Search** — powered by Meilisearch for instant document discovery - **Full-Text Search** — powered by Meilisearch for instant document discovery
@@ -106,6 +106,7 @@ Processed documents are distributed to any combination of configured destination
| **iCloud Drive** | Apple cloud | | **iCloud Drive** | Apple cloud |
| **Email (SMTP)** | Send as attachment | | **Email (SMTP)** | Send as attachment |
| **Paperless-ngx** | Document management system | | **Paperless-ngx** | Document management system |
| **Evernote** | Notes with PDF attachments |
| **Rclone** | 70+ cloud providers via Rclone | | **Rclone** | 70+ cloud providers via Rclone |
## Features ## Features
@@ -245,6 +246,7 @@ See the [Kubernetes Deployment Guide](docs/KubernetesDeployment.md) for full det
| [Google Drive](docs/GoogleDriveSetup.md) | Google Drive service account / OAuth | | [Google Drive](docs/GoogleDriveSetup.md) | Google Drive service account / OAuth |
| [OneDrive](docs/OneDriveSetup.md) | Microsoft OneDrive setup | | [OneDrive](docs/OneDriveSetup.md) | Microsoft OneDrive setup |
| [Amazon S3](docs/AmazonS3Setup.md) | S3 bucket configuration | | [Amazon S3](docs/AmazonS3Setup.md) | S3 bucket configuration |
| [Evernote](docs/EvernoteSetup.md) | Evernote note creation |
| [Authentication](docs/AuthenticationSetup.md) | OAuth2, OIDC, and social login | | [Authentication](docs/AuthenticationSetup.md) | OAuth2, OIDC, and social login |
| [Notifications](docs/NotificationsSetup.md) | Notification backend setup | | [Notifications](docs/NotificationsSetup.md) | Notification backend setup |
@@ -327,6 +329,7 @@ The following is a summary of the licenses used by our direct dependencies:
| pypdf | BSD | | pypdf | BSD |
| Requests | Apache 2.0 | | Requests | Apache 2.0 |
| Dropbox SDK | MIT | | Dropbox SDK | MIT |
| Evernote SDK | BSD |
| Azure AI Document Intelligence | MIT | | Azure AI Document Intelligence | MIT |
| Authlib | BSD | | Authlib | BSD |
| Starlette | BSD | | Starlette | BSD |
+11
View File
@@ -154,6 +154,17 @@ class Settings(BaseSettings):
# "language": "Language", "correspondent": "Correspondent"} # "language": "Language", "correspondent": "Correspondent"}
paperless_custom_fields_mapping: Optional[str] = None paperless_custom_fields_mapping: Optional[str] = None
# Evernote destination settings
evernote_enabled: bool = Field(
default=True,
description="Enable Evernote as an upload destination. Set to False to disable uploads even when credentials are configured.",
)
evernote_auth_token: Optional[str] = None
evernote_sandbox: bool = False
evernote_notebook_guid: Optional[str] = None
evernote_default_tags: Optional[str] = None
evernote_include_metadata: bool = True
azure_ai_key: str azure_ai_key: str
azure_region: str azure_region: str
azure_endpoint: str azure_endpoint: str
+12
View File
@@ -10,6 +10,7 @@ from app.models import FileRecord, IntegrationDirection, UserIntegration
from app.tasks.retry_config import BaseTaskWithRetry from app.tasks.retry_config import BaseTaskWithRetry
from app.tasks.upload_to_dropbox import upload_to_dropbox from app.tasks.upload_to_dropbox import upload_to_dropbox
from app.tasks.upload_to_email import upload_to_email from app.tasks.upload_to_email import upload_to_email
from app.tasks.upload_to_evernote import upload_to_evernote
from app.tasks.upload_to_ftp import upload_to_ftp from app.tasks.upload_to_ftp import upload_to_ftp
from app.tasks.upload_to_google_drive import upload_to_google_drive from app.tasks.upload_to_google_drive import upload_to_google_drive
from app.tasks.upload_to_icloud import upload_to_icloud from app.tasks.upload_to_icloud import upload_to_icloud
@@ -100,6 +101,11 @@ def _should_upload_to_email():
) )
def _should_upload_to_evernote():
token = getattr(settings, "evernote_auth_token", None)
return bool(getattr(settings, "evernote_enabled", True) and isinstance(token, str) and token.strip())
def _should_upload_to_onedrive(): def _should_upload_to_onedrive():
return bool( return bool(
getattr(settings, "onedrive_enabled", True) getattr(settings, "onedrive_enabled", True)
@@ -151,6 +157,7 @@ def get_configured_services_from_validator():
"FTP Storage": "ftp", "FTP Storage": "ftp",
"SFTP Storage": "sftp", "SFTP Storage": "sftp",
"Email": "email", "Email": "email",
"Evernote": "evernote",
"OneDrive": "onedrive", "OneDrive": "onedrive",
"S3 Storage": "s3", "S3 Storage": "s3",
"SharePoint": "sharepoint", "SharePoint": "sharepoint",
@@ -254,6 +261,11 @@ def send_to_all_destinations(self, file_path: str, use_validator=True, file_id:
"should_upload": _should_upload_to_email, "should_upload": _should_upload_to_email,
"upload_func": upload_to_email, "upload_func": upload_to_email,
}, },
{
"name": "evernote",
"should_upload": _should_upload_to_evernote,
"upload_func": upload_to_evernote,
},
{ {
"name": "onedrive", "name": "onedrive",
"should_upload": _should_upload_to_onedrive, "should_upload": _should_upload_to_onedrive,
+242
View File
@@ -0,0 +1,242 @@
#!/usr/bin/env python3
import hashlib
import json
import logging
import mimetypes
import os
from html import escape
from typing import Any
from app.celery_app import celery
from app.config import settings
from app.tasks.retry_config import UploadTaskWithRetry
from app.utils import log_task_progress
logger = logging.getLogger(__name__)
_UNKNOWN_PLACEHOLDERS = {"", "Unknown", "unknown", "N/A", "n/a", "None", "none", "null"}
_MAX_EVERNOTE_TITLE_LENGTH = 255
def _get_evernote_sdk():
"""Import the Evernote SDK lazily so the missing dependency error is actionable."""
try:
from evernote.edam.notestore import NoteStore
from evernote.edam.type import ttypes as Types
from evernote.edam.userstore import UserStore
from thrift.protocol import TBinaryProtocol
from thrift.transport import THttpClient
except ImportError as exc:
raise RuntimeError("Evernote upload requires the evernote3 package. Install requirements.txt again.") from exc
return NoteStore, Types, UserStore, TBinaryProtocol, THttpClient
def _build_thrift_client(client_cls, url: str, binary_protocol, http_transport):
transport = http_transport.THttpClient(url)
protocol = binary_protocol.TBinaryProtocol(transport)
return client_cls(protocol)
def _get_note_store(auth_token: str):
NoteStore, Types, UserStore, TBinaryProtocol, THttpClient = _get_evernote_sdk()
base_url = (
"https://sandbox.evernote.com" if getattr(settings, "evernote_sandbox", False) else "https://www.evernote.com"
)
user_store = _build_thrift_client(UserStore.Client, f"{base_url}/edam/user", TBinaryProtocol, THttpClient)
user = user_store.getUser(auth_token)
shard_id = getattr(user, "shardId", None)
if not shard_id:
raise RuntimeError("Evernote user response did not include a shard ID")
note_store_url = f"{base_url}/shard/{shard_id}/notestore"
note_store = _build_thrift_client(NoteStore.Client, note_store_url, TBinaryProtocol, THttpClient)
return note_store, Types
def _load_metadata(file_path: str) -> dict[str, Any]:
"""Load extracted DocuElevate metadata from the companion JSON file, when present."""
json_path = os.path.splitext(file_path)[0] + ".json"
if not os.path.exists(json_path):
return {}
try:
with open(json_path, "r", encoding="utf-8") as metadata_file:
data = json.load(metadata_file)
except Exception as exc: # noqa: BLE001
logger.warning("Failed to load Evernote metadata from %s: %s", json_path, exc)
return {}
return data if isinstance(data, dict) else {}
def _normalize_metadata_value(value: Any) -> str:
if value is None:
return ""
if isinstance(value, (list, tuple, set)):
normalized = ", ".join(str(item) for item in value if item is not None)
elif isinstance(value, dict):
normalized = json.dumps(value, ensure_ascii=False, sort_keys=True)
else:
normalized = str(value)
normalized = normalized.strip()
return "" if normalized in _UNKNOWN_PLACEHOLDERS else normalized
def _metadata_rows(metadata: dict[str, Any]) -> list[tuple[str, str]]:
rows = []
for key in sorted(metadata):
value = _normalize_metadata_value(metadata[key])
if value:
rows.append((key, value))
return rows
def _extract_tags(metadata: dict[str, Any]) -> list[str]:
tags: list[str] = []
def add_tag(value: Any) -> None:
normalized = _normalize_metadata_value(value)
if normalized and normalized not in tags:
tags.append(normalized)
default_tags = getattr(settings, "evernote_default_tags", None)
if default_tags:
for tag in str(default_tags).split(","):
add_tag(tag)
metadata_tags = metadata.get("tags")
if isinstance(metadata_tags, str):
for tag in metadata_tags.split(","):
add_tag(tag)
elif isinstance(metadata_tags, (list, tuple, set)):
for tag in metadata_tags:
add_tag(tag)
return tags
def _note_title(file_path: str, metadata: dict[str, Any]) -> str:
title = (
_normalize_metadata_value(metadata.get("title"))
or _normalize_metadata_value(metadata.get("filename"))
or os.path.basename(file_path)
)
return title[:_MAX_EVERNOTE_TITLE_LENGTH]
def _build_enml(metadata: dict[str, Any], resource_hash: str, resource_mime: str, include_metadata: bool) -> str:
body_parts = ['<?xml version="1.0" encoding="UTF-8"?>']
body_parts.append('<!DOCTYPE en-note SYSTEM "http://xml.evernote.com/pub/enml2.dtd">')
body_parts.append("<en-note>")
if include_metadata:
rows = _metadata_rows(metadata)
if rows:
body_parts.append("<div><b>DocuElevate metadata</b></div>")
for key, value in rows:
body_parts.append(f"<div><b>{escape(key)}:</b> {escape(value)}</div>")
body_parts.append("<br/>")
body_parts.append(f'<en-media type="{escape(resource_mime)}" hash="{resource_hash}"/>')
body_parts.append("</en-note>")
return "".join(body_parts)
def _create_evernote_note(file_path: str, metadata: dict[str, Any], task_id: str):
auth_token = getattr(settings, "evernote_auth_token", None)
if not auth_token:
raise ValueError("Evernote auth token is not configured (EVERNOTE_AUTH_TOKEN)")
note_store, Types = _get_note_store(auth_token)
filename = os.path.basename(file_path)
with open(file_path, "rb") as pdf_file:
resource_body = pdf_file.read()
body_hash = hashlib.md5(resource_body).digest() # noqa: S324 - Evernote API requires MD5 resource hashes.
body_hash_hex = hashlib.md5(resource_body).hexdigest() # noqa: S324 - Evernote ENML references MD5 hashes.
resource_mime = mimetypes.guess_type(filename)[0] or "application/pdf"
data = Types.Data()
data.size = len(resource_body)
data.bodyHash = body_hash
data.body = resource_body
resource = Types.Resource()
resource.mime = resource_mime
resource.data = data
resource.attributes = Types.ResourceAttributes(fileName=filename)
note = Types.Note()
note.title = _note_title(file_path, metadata)
note.content = _build_enml(
metadata,
body_hash_hex,
resource_mime,
include_metadata=getattr(settings, "evernote_include_metadata", True),
)
note.resources = [resource]
notebook_guid = getattr(settings, "evernote_notebook_guid", None)
if notebook_guid:
note.notebookGuid = notebook_guid
tag_names = _extract_tags(metadata)
if tag_names:
note.tagNames = tag_names
created_note = note_store.createNote(auth_token, note)
logger.info("[%s] Created Evernote note %s for %s", task_id, getattr(created_note, "guid", None), file_path)
return created_note
@celery.task(base=UploadTaskWithRetry, bind=True)
def upload_to_evernote(self, file_path: str, file_id: int = None):
"""
Upload a document to Evernote by creating a note with metadata and a PDF attachment.
Args:
file_path: Path to the PDF file to upload
file_id: Optional file ID to associate with logs
"""
task_id = self.request.id
filename = os.path.basename(file_path)
logger.info("[%s] Starting Evernote upload: %s", task_id, file_path)
log_task_progress(
task_id, "upload_to_evernote", "in_progress", f"Uploading to Evernote: {filename}", file_id=file_id
)
if not os.path.exists(file_path):
error_msg = f"File not found: {file_path}"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_evernote", "failure", error_msg, file_id=file_id)
raise FileNotFoundError(error_msg)
if not getattr(settings, "evernote_auth_token", None):
error_msg = "Evernote auth token is not configured (EVERNOTE_AUTH_TOKEN)"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_evernote", "failure", error_msg, file_id=file_id)
raise ValueError(error_msg)
try:
metadata = _load_metadata(file_path)
created_note = _create_evernote_note(file_path, metadata, task_id)
except Exception as exc:
error_msg = f"Failed to upload to Evernote: {exc}"
logger.error("[%s] %s", task_id, error_msg)
log_task_progress(task_id, "upload_to_evernote", "failure", error_msg, file_id=file_id)
raise
note_guid = getattr(created_note, "guid", None)
log_task_progress(task_id, "upload_to_evernote", "success", f"Uploaded to Evernote: {note_guid}", file_id=file_id)
return {
"status": "Completed",
"file_path": file_path,
"evernote_note_guid": note_guid,
"evernote_title": getattr(created_note, "title", None),
"evernote_notebook_guid": getattr(created_note, "notebookGuid", None),
}
+15
View File
@@ -174,6 +174,21 @@ def get_provider_status() -> dict[str, dict[str, object]]:
}, },
} }
providers["Evernote"] = {
"name": "Evernote",
"icon": "fa-brands fa-evernote",
"configured": bool(getattr(settings, "evernote_auth_token", None)),
"enabled": getattr(settings, "evernote_enabled", True),
"description": "Create Evernote notes with document metadata and PDF attachments",
"details": {
"auth_token": mask_sensitive_value(getattr(settings, "evernote_auth_token", None)),
"sandbox": getattr(settings, "evernote_sandbox", False),
"notebook_guid": getattr(settings, "evernote_notebook_guid", "Not set"),
"default_tags": getattr(settings, "evernote_default_tags", "Not set"),
"include_metadata": getattr(settings, "evernote_include_metadata", True),
},
}
# Add FTP configuration to providers # Add FTP configuration to providers
providers["FTP Storage"] = { providers["FTP Storage"] = {
"name": "FTP Storage", "name": "FTP Storage",
+6
View File
@@ -145,6 +145,12 @@ def validate_storage_configs() -> dict[str, list[str]]:
email_issues.append("DEST_EMAIL_DEFAULT_RECIPIENT is not configured") email_issues.append("DEST_EMAIL_DEFAULT_RECIPIENT is not configured")
issues["email"] = email_issues issues["email"] = email_issues
# Validate Evernote
evernote_issues = []
if not getattr(settings, "evernote_auth_token", None):
evernote_issues.append("EVERNOTE_AUTH_TOKEN is not configured")
issues["evernote"] = evernote_issues
# Validate S3 # Validate S3
s3_issues = [] s3_issues = []
if not getattr(settings, "s3_bucket_name", None): if not getattr(settings, "s3_bucket_name", None):
+2
View File
@@ -717,6 +717,7 @@ def _compute_processing_flow(logs, pipeline_steps=None):
"upload_to_ftp": "FTP Storage", "upload_to_ftp": "FTP Storage",
"upload_to_sftp": "SFTP Storage", "upload_to_sftp": "SFTP Storage",
"upload_to_email": "Email", "upload_to_email": "Email",
"upload_to_evernote": "Evernote",
"queue_dropbox": "Dropbox", "queue_dropbox": "Dropbox",
"queue_nextcloud": "Nextcloud", "queue_nextcloud": "Nextcloud",
"queue_paperless": "Paperless-ngx", "queue_paperless": "Paperless-ngx",
@@ -727,6 +728,7 @@ def _compute_processing_flow(logs, pipeline_steps=None):
"queue_ftp": "FTP Storage", "queue_ftp": "FTP Storage",
"queue_sftp": "SFTP Storage", "queue_sftp": "SFTP Storage",
"queue_email": "Email", "queue_email": "Email",
"queue_evernote": "Evernote",
} }
# Create a map of step names to their log entries # Create a map of step names to their log entries
+13
View File
@@ -1330,6 +1330,19 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
| `DEST_EMAIL_SENDER` | From address for delivered documents (e.g., `"DocuElevate Delivery <docuelevate@example.com>"`). | | `DEST_EMAIL_SENDER` | From address for delivered documents (e.g., `"DocuElevate Delivery <docuelevate@example.com>"`). |
| `DEST_EMAIL_DEFAULT_RECIPIENT` | Fallback recipient email when none is specified for a delivery task. | | `DEST_EMAIL_DEFAULT_RECIPIENT` | Fallback recipient email when none is specified for a delivery task. |
### Evernote
| **Variable** | **Description** |
|-------------------------------|---------------------------------------------------------------------|
| `EVERNOTE_ENABLED` | Set to `false` to disable Evernote uploads without removing credentials. Default: `true` |
| `EVERNOTE_AUTH_TOKEN` | Evernote developer token or OAuth access token used to create notes. |
| `EVERNOTE_SANDBOX` | Use Evernote sandbox API endpoints. Default: `false` |
| `EVERNOTE_NOTEBOOK_GUID` | Optional target notebook GUID. If omitted, Evernote uses the default notebook. |
| `EVERNOTE_DEFAULT_TAGS` | Optional comma-separated tags applied to every created note. |
| `EVERNOTE_INCLUDE_METADATA` | Include extracted metadata in the Evernote note body. Default: `true` |
For detailed setup instructions, see the [Evernote Setup Guide](EvernoteSetup.md).
### OneDrive / Microsoft Graph ### OneDrive / Microsoft Graph
| **Variable** | **Description** | | **Variable** | **Description** |
+1
View File
@@ -13,6 +13,7 @@ DocuElevate is designed to be highly configurable through environment variables,
- [Dropbox Setup](DropboxSetup.md) - How to set up Dropbox integration - [Dropbox Setup](DropboxSetup.md) - How to set up Dropbox integration
- [OneDrive Setup](OneDriveSetup.md) - How to set up Microsoft OneDrive/Graph integration - [OneDrive Setup](OneDriveSetup.md) - How to set up Microsoft OneDrive/Graph integration
- [Amazon S3 Setup](AmazonS3Setup.md) - How to set up Amazon S3 integration - [Amazon S3 Setup](AmazonS3Setup.md) - How to set up Amazon S3 integration
- [Evernote Setup](EvernoteSetup.md) - How to set up Evernote note creation
- [Authentication Setup](AuthenticationSetup.md) - How to set up user authentication - [Authentication Setup](AuthenticationSetup.md) - How to set up user authentication
- [Notifications Setup](NotificationsSetup.md) - How to set up system notifications - [Notifications Setup](NotificationsSetup.md) - How to set up system notifications
+51
View File
@@ -0,0 +1,51 @@
# Setting up Evernote Integration
This guide explains how to configure DocuElevate to create Evernote notes for processed documents.
## Overview
The Evernote destination creates one note per processed document. The note contains:
- A visible metadata section populated from DocuElevate's extracted metadata JSON
- The processed PDF attached as an Evernote resource
- Optional tags from `EVERNOTE_DEFAULT_TAGS` plus extracted document tags
## Required Configuration
| Variable | Description |
|----------|-------------|
| `EVERNOTE_ENABLED` | Set to `false` to disable Evernote uploads without removing credentials. Default: `true` |
| `EVERNOTE_AUTH_TOKEN` | Evernote developer token or OAuth access token with note creation permissions |
## Optional Configuration
| Variable | Description |
|----------|-------------|
| `EVERNOTE_SANDBOX` | Use Evernote sandbox API endpoints. Default: `false` |
| `EVERNOTE_NOTEBOOK_GUID` | Target notebook GUID. If omitted, Evernote uses the account default notebook |
| `EVERNOTE_DEFAULT_TAGS` | Comma-separated tags to apply to every created note, for example `docuelevate,archive` |
| `EVERNOTE_INCLUDE_METADATA` | Include extracted metadata in the note body. Default: `true` |
## Example
```dotenv
EVERNOTE_ENABLED=true
EVERNOTE_AUTH_TOKEN=your-evernote-token
EVERNOTE_NOTEBOOK_GUID=optional-notebook-guid
EVERNOTE_DEFAULT_TAGS=docuelevate,processed
EVERNOTE_INCLUDE_METADATA=true
```
## Metadata and Attachments
DocuElevate reads the companion metadata file next to the processed PDF, for example `invoice.pdf` and `invoice.json`. Non-empty metadata fields are rendered into the Evernote note body. Values such as `Unknown`, empty strings, and null values are skipped.
If the metadata contains a `tags` field, those tags are applied to the note together with any tags configured in `EVERNOTE_DEFAULT_TAGS`.
The processed PDF is attached directly to the note using Evernote's resource model, so it appears as a normal Evernote attachment.
## Notes
- Evernote tokens can expire or be revoked. If uploads start failing with authentication errors, generate or refresh the token and update `EVERNOTE_AUTH_TOKEN`.
- If `EVERNOTE_NOTEBOOK_GUID` points to a missing or inaccessible notebook, Evernote will reject the note creation request.
- Evernote enforces account upload quotas and per-note size limits. Large PDFs may fail if they exceed those limits.
+1
View File
@@ -24,6 +24,7 @@ Welcome to the DocuElevate documentation. This directory contains comprehensive
- [OneDrive Setup](OneDriveSetup.md) - How to set up Microsoft OneDrive/Graph integration - [OneDrive Setup](OneDriveSetup.md) - How to set up Microsoft OneDrive/Graph integration
- [SharePoint Setup](SharePointSetup.md) - How to set up Microsoft SharePoint Online integration - [SharePoint Setup](SharePointSetup.md) - How to set up Microsoft SharePoint Online integration
- [Amazon S3 Setup](AmazonS3Setup.md) - How to set up Amazon S3 integration - [Amazon S3 Setup](AmazonS3Setup.md) - How to set up Amazon S3 integration
- [Evernote Setup](EvernoteSetup.md) - How to set up Evernote note creation
- [Authentication Setup](AuthenticationSetup.md) - How to set up user authentication - [Authentication Setup](AuthenticationSetup.md) - How to set up user authentication
- [Notifications Setup](NotificationsSetup.md) - How to set up system notifications - [Notifications Setup](NotificationsSetup.md) - How to set up system notifications
+1 -1
View File
@@ -28,7 +28,7 @@ Settings are organized into logical categories for easy navigation:
- **Authentication**: Login settings, session secrets, OAuth configuration, admin group - **Authentication**: Login settings, session secrets, OAuth configuration, admin group
- **AI Services**: AI provider selection, model configuration, embeddings, and credentials (OpenAI, Azure, Anthropic, Gemini, Ollama, OpenRouter, Portkey, LiteLLM) - **AI Services**: AI provider selection, model configuration, embeddings, and credentials (OpenAI, Azure, Anthropic, Gemini, Ollama, OpenRouter, Portkey, LiteLLM)
- **OCR Engines**: OCR provider selection and configuration (Tesseract, EasyOCR, Mistral, Google DocAI, AWS Textract) - **OCR Engines**: OCR provider selection and configuration (Tesseract, EasyOCR, Mistral, Google DocAI, AWS Textract)
- **Storage Providers**: Dropbox, Google Drive, OneDrive, SharePoint, S3, FTP, SFTP, WebDAV, Nextcloud, Paperless - **Storage Providers**: Dropbox, Google Drive, OneDrive, SharePoint, S3, FTP, SFTP, WebDAV, Nextcloud, Paperless, Evernote
- **Email**: SMTP configuration for sending emails - **Email**: SMTP configuration for sending emails
- **IMAP**: Email ingestion configuration (supports two mailbox accounts) - **IMAP**: Email ingestion configuration (supports two mailbox accounts)
- **Monitoring**: Uptime Kuma integration - **Monitoring**: Uptime Kuma integration
+1
View File
@@ -125,6 +125,7 @@ Several cloud storage integrations include their own guided configuration pages
| Google Drive | `/google-drive-setup` | [GoogleDriveSetup.md](GoogleDriveSetup.md) | | Google Drive | `/google-drive-setup` | [GoogleDriveSetup.md](GoogleDriveSetup.md) |
| OneDrive / SharePoint | `/onedrive-setup` | [OneDriveSetup.md](OneDriveSetup.md) | | OneDrive / SharePoint | `/onedrive-setup` | [OneDriveSetup.md](OneDriveSetup.md) |
| Amazon S3 | Configured via settings | [AmazonS3Setup.md](AmazonS3Setup.md) | | Amazon S3 | Configured via settings | [AmazonS3Setup.md](AmazonS3Setup.md) |
| Evernote | Configured via settings | [EvernoteSetup.md](EvernoteSetup.md) |
These pages are accessed **after** the main Setup Wizard is complete and are independent wizard flows specific to each integration. These pages are accessed **after** the main Setup Wizard is complete and are independent wizard flows specific to each integration.
+1
View File
@@ -55,6 +55,7 @@ nav:
- Google Drive: GoogleDriveSetup - Google Drive: GoogleDriveSetup
- OneDrive: OneDriveSetup - OneDrive: OneDriveSetup
- Amazon S3: AmazonS3Setup - Amazon S3: AmazonS3Setup
- Evernote: EvernoteSetup
- Authentication: AuthenticationSetup - Authentication: AuthenticationSetup
- Notifications: NotificationsSetup - Notifications: NotificationsSetup
- Security: - Security:
+3
View File
@@ -37,6 +37,9 @@ paramiko>=3.4.0 # SSH/SFTP implementation for Python (LGPL license)
# iCloud Drive # iCloud Drive
pyicloud>=2.4.0 # Unofficial Apple iCloud API client (MIT license) pyicloud>=2.4.0 # Unofficial Apple iCloud API client (MIT license)
# Evernote
evernote3>=1.25.14 # Evernote Cloud API SDK for Python 3 (BSD license)
# Safe XML parsing (protection against XML bomb / XXE attacks) # Safe XML parsing (protection against XML bomb / XXE attacks)
defusedxml>=0.7.1 defusedxml>=0.7.1
+8
View File
@@ -35,6 +35,7 @@ class TestValidateStorageConfigs:
"google_drive", "google_drive",
"onedrive", "onedrive",
"email", "email",
"evernote",
"paperless", "paperless",
"uptime_kuma", "uptime_kuma",
] ]
@@ -83,6 +84,13 @@ class TestValidateStorageConfigs:
assert "DEST_EMAIL_HOST is not configured" in result["email"] assert "DEST_EMAIL_HOST is not configured" in result["email"]
assert "DEST_EMAIL_DEFAULT_RECIPIENT is not configured" in result["email"] assert "DEST_EMAIL_DEFAULT_RECIPIENT is not configured" in result["email"]
def test_evernote_missing_token(self):
"""Test validation when Evernote destination auth token is missing."""
with patch("app.utils.config_validator.validators.settings") as mock_settings:
mock_settings.evernote_auth_token = None
result = validate_storage_configs()
assert "EVERNOTE_AUTH_TOKEN is not configured" in result["evernote"]
@pytest.mark.unit @pytest.mark.unit
class TestValidateEmailConfig: class TestValidateEmailConfig:
@@ -663,6 +663,7 @@ def _all_should_upload_false():
"ftp", "ftp",
"sftp", "sftp",
"email", "email",
"evernote",
"onedrive", "onedrive",
"s3", "s3",
"sharepoint", "sharepoint",
+19
View File
@@ -7,6 +7,7 @@ import pytest
from app.tasks.send_to_all import ( from app.tasks.send_to_all import (
_should_upload_to_dropbox, _should_upload_to_dropbox,
_should_upload_to_email, _should_upload_to_email,
_should_upload_to_evernote,
_should_upload_to_ftp, _should_upload_to_ftp,
_should_upload_to_google_drive, _should_upload_to_google_drive,
_should_upload_to_icloud, _should_upload_to_icloud,
@@ -128,6 +129,14 @@ class TestShouldUploadFunctions:
assert _should_upload_to_email() is True assert _should_upload_to_email() is True
@patch("app.tasks.send_to_all.settings")
def test_should_upload_to_evernote_configured(self, mock_settings):
"""Test Evernote upload check."""
mock_settings.evernote_enabled = True
mock_settings.evernote_auth_token = "token"
assert _should_upload_to_evernote() is True
@patch("app.tasks.send_to_all.settings") @patch("app.tasks.send_to_all.settings")
def test_should_upload_to_onedrive_configured(self, mock_settings): def test_should_upload_to_onedrive_configured(self, mock_settings):
"""Test OneDrive upload check.""" """Test OneDrive upload check."""
@@ -248,6 +257,14 @@ class TestShouldUploadEnabledFlag:
assert _should_upload_to_email() is False assert _should_upload_to_email() is False
@patch("app.tasks.send_to_all.settings")
def test_evernote_disabled_with_credentials(self, mock_settings):
"""Test Evernote upload is blocked when disabled even with valid credentials."""
mock_settings.evernote_enabled = False
mock_settings.evernote_auth_token = "token"
assert _should_upload_to_evernote() is False
@patch("app.tasks.send_to_all.settings") @patch("app.tasks.send_to_all.settings")
def test_onedrive_disabled_with_credentials(self, mock_settings): def test_onedrive_disabled_with_credentials(self, mock_settings):
"""Test OneDrive upload is blocked when disabled even with valid credentials.""" """Test OneDrive upload is blocked when disabled even with valid credentials."""
@@ -289,6 +306,7 @@ class TestGetConfiguredServicesFromValidator:
"Dropbox": {"configured": True, "enabled": True}, "Dropbox": {"configured": True, "enabled": True},
"NextCloud": {"configured": False, "enabled": True}, "NextCloud": {"configured": False, "enabled": True},
"S3 Storage": {"configured": True, "enabled": True}, "S3 Storage": {"configured": True, "enabled": True},
"Evernote": {"configured": True, "enabled": True},
} }
result = get_configured_services_from_validator() result = get_configured_services_from_validator()
@@ -296,6 +314,7 @@ class TestGetConfiguredServicesFromValidator:
assert result["dropbox"] is True assert result["dropbox"] is True
assert result["nextcloud"] is False assert result["nextcloud"] is False
assert result["s3"] is True assert result["s3"] is True
assert result["evernote"] is True
@patch("app.tasks.send_to_all.get_provider_status") @patch("app.tasks.send_to_all.get_provider_status")
def test_handles_missing_providers(self, mock_get_status): def test_handles_missing_providers(self, mock_get_status):
+2
View File
@@ -8,6 +8,7 @@ from unittest.mock import Mock, patch
import pytest import pytest
from app.tasks.upload_to_email import upload_to_email from app.tasks.upload_to_email import upload_to_email
from app.tasks.upload_to_evernote import upload_to_evernote
from app.tasks.upload_to_ftp import upload_to_ftp from app.tasks.upload_to_ftp import upload_to_ftp
from app.tasks.upload_to_google_drive import upload_to_google_drive from app.tasks.upload_to_google_drive import upload_to_google_drive
from app.tasks.upload_to_onedrive import upload_to_onedrive from app.tasks.upload_to_onedrive import upload_to_onedrive
@@ -447,6 +448,7 @@ def test_all_upload_tasks_have_consistent_signature(sample_text_file):
(upload_to_webdav, "app.tasks.upload_to_webdav"), (upload_to_webdav, "app.tasks.upload_to_webdav"),
(upload_to_google_drive, "app.tasks.upload_to_google_drive"), (upload_to_google_drive, "app.tasks.upload_to_google_drive"),
(upload_to_email, "app.tasks.upload_to_email"), (upload_to_email, "app.tasks.upload_to_email"),
(upload_to_evernote, "app.tasks.upload_to_evernote"),
] ]
import inspect import inspect
+113
View File
@@ -0,0 +1,113 @@
import hashlib
import json
from types import SimpleNamespace
from unittest.mock import patch
import pytest
from app.tasks.upload_to_evernote import _build_enml, upload_to_evernote
class _FakeTypes:
class Data:
pass
class Resource:
pass
class ResourceAttributes:
def __init__(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
class Note:
pass
class _FakeNoteStore:
def __init__(self):
self.calls = []
def createNote(self, auth_token, note):
self.calls.append((auth_token, note))
note.guid = "note-guid-123"
return note
_fake_note_store = _FakeNoteStore()
@pytest.fixture(autouse=True)
def reset_fake_store():
global _fake_note_store
_fake_note_store = _FakeNoteStore()
def test_build_enml_escapes_metadata():
enml = _build_enml({"sender": "A&B <Corp>"}, "abc123", "application/pdf", include_metadata=True)
assert "A&amp;B &lt;Corp&gt;" in enml
assert '<en-media type="application/pdf" hash="abc123"/>' in enml
@pytest.mark.unit
def test_upload_to_evernote_creates_note_with_metadata_and_pdf(tmp_path):
pdf_path = tmp_path / "invoice.pdf"
pdf_bytes = b"%PDF-1.4 test content"
pdf_path.write_bytes(pdf_bytes)
pdf_path.with_suffix(".json").write_text(
json.dumps(
{
"title": "Invoice May",
"absender": "Example GmbH",
"tags": ["invoice", "finance"],
"empty": "Unknown",
}
),
encoding="utf-8",
)
with (
patch("app.tasks.upload_to_evernote._get_note_store", return_value=(_fake_note_store, _FakeTypes)),
patch("app.tasks.upload_to_evernote.log_task_progress"),
patch("app.tasks.upload_to_evernote.settings") as mock_settings,
):
mock_settings.evernote_auth_token = "auth-token"
mock_settings.evernote_sandbox = True
mock_settings.evernote_notebook_guid = "notebook-guid"
mock_settings.evernote_default_tags = "docuelevate,archive"
mock_settings.evernote_include_metadata = True
result = upload_to_evernote.apply(args=[str(pdf_path)], kwargs={"file_id": 7}).get()
created_note = _fake_note_store.calls[0][1]
resource = created_note.resources[0]
assert result["status"] == "Completed"
assert result["evernote_note_guid"] == "note-guid-123"
assert created_note.title == "Invoice May"
assert created_note.notebookGuid == "notebook-guid"
assert created_note.tagNames == ["docuelevate", "archive", "invoice", "finance"]
assert "Example GmbH" in created_note.content
assert "empty" not in created_note.content
assert f'hash="{hashlib.md5(pdf_bytes).hexdigest()}"' in created_note.content # noqa: S324
assert resource.mime == "application/pdf"
assert resource.attributes.fileName == "invoice.pdf"
assert resource.data.body == pdf_bytes
assert resource.data.bodyHash == hashlib.md5(pdf_bytes).digest() # noqa: S324
@pytest.mark.unit
def test_upload_to_evernote_requires_token(tmp_path):
pdf_path = tmp_path / "document.pdf"
pdf_path.write_bytes(b"%PDF-1.4")
with (
patch("app.tasks.upload_to_evernote.log_task_progress"),
patch("app.tasks.upload_to_evernote.settings", SimpleNamespace(evernote_auth_token=None)),
):
result = upload_to_evernote.apply(args=[str(pdf_path)])
assert result.failed()
assert isinstance(result.result, ValueError)
assert "EVERNOTE_AUTH_TOKEN" in str(result.result)