gitea-admin/gh-christianlouis-docuelevate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: merge main into feature branch - resolve all merge conflicts cleanly

Browse Source 
Merges origin/main (v0.156.0) into the classification feature branch,
properly resolving all 23 merge conflicts:

- Auto-generated files (BUILD_DATE, VERSION, etc.): accept main's version
- Non-classification files (SharePoint, QR auth, session mgmt, mobile): accept main's version
- Classification files (api/__init__.py, models.py, migrations/env.py, conftest.py):
  keep classification additions alongside main's content

Previously the branch was incorrectly removing files from main (SharePoint
integration, QR scanner, session management). This merge properly preserves
all main branch content while maintaining the classification feature additions.

Migration chain validated: 038_add_classification_rules chains from
037_add_user_sessions_and_qr_challenges.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-03-17 16:14:45 +00:00
parent 342e2d1614
commit a08b103271
37 changed files with 1814 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/AuthenticationSetup.md
+7 -4
View File
@@ -190,20 +190,23 @@ QR code login allows users to authenticate a mobile device by scanning a QR code
### How It Works
1. The authenticated web user opens the **QR Login** page and a challenge QR code is displayed.
2. The mobile app scans the QR code and calls the claim endpoint.
3. An API token is issued for the mobile device and the web UI is notified via polling.
2. The user opens the DocuElevate mobile app and taps **Scan QR Code to Login**, which opens the device camera.
3. The mobile app scans the QR code. The QR code contains both the challenge token and the server URL (`docuelevate://qr-login?token=...&server=...`), so there is no need to enter the server URL manually.
4. An API token is issued for the mobile device and the web UI is notified via polling.
> **Note:** The countdown timer on the web page uses server-relative time (TTL in seconds) rather than absolute timestamps, so it works correctly even when the client's clock is not in sync with the server.
### Configuration
| Variable | Description | Default |
|----------|-------------|---------|
| `QR_LOGIN_CHALLENGE_TTL_SECONDS` | How long a QR challenge is valid | `120` |
| `QR_LOGIN_CHALLENGE_TTL_SECONDS` | How long a QR challenge is valid (seconds) | `120` |
### API Endpoints
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/qr-auth/challenge` | Create a new QR login challenge |
| `POST` | `/api/qr-auth/challenge` | Create a new QR login challenge (returns `ttl_seconds` for client countdown) |
| `GET` | `/api/qr-auth/challenge/{id}/status` | Poll the status of a challenge |
| `POST` | `/api/qr-auth/claim` | Claim a challenge from a mobile device |
docs/ConfigurationGuide.md
+24
View File
@@ -1289,6 +1289,20 @@ For detailed setup instructions, see the [Google Drive Setup Guide](GoogleDriveS
For detailed setup instructions, see the [OneDrive Setup Guide](OneDriveSetup.md).
### SharePoint Online
| **Variable** | **Description** |
|---------------------------------|-------------------------------------------------------|
| `SHAREPOINT_CLIENT_ID` | Azure AD application client ID |
| `SHAREPOINT_CLIENT_SECRET` | Azure AD application client secret |
| `SHAREPOINT_TENANT_ID` | Azure AD tenant ID (use "common" for multi-tenant apps) |
| `SHAREPOINT_REFRESH_TOKEN` | OAuth 2.0 refresh token |
| `SHAREPOINT_SITE_URL` | SharePoint site URL (e.g. `https://tenant.sharepoint.com/sites/sitename`) |
| `SHAREPOINT_DOCUMENT_LIBRARY` | Document library name (default: `Documents`) |
| `SHAREPOINT_FOLDER_PATH` | Subfolder path inside the document library |
SharePoint uses the same Microsoft Graph API as OneDrive. See the [OneDrive Setup Guide](OneDriveSetup.md) for Azure AD app registration instructions — the same app registration can be reused for SharePoint with the `Sites.ReadWrite.All` permission.
### Amazon S3
| **Variable** | **Description** |
@@ -1628,6 +1642,7 @@ For example:
| S3 | `docs/uploads/` | `docs/uploads/pdfa/` |
| Nextcloud | `/Files` | `/Files/pdfa` |
| OneDrive | `Documents/Uploads` | `Documents/Uploads/pdfa` |
| SharePoint | `Uploads` | `Uploads/pdfa` |
| Google Drive | *(folder ID)* | `GOOGLE_DRIVE_PDFA_FOLDER_ID` |
Set `PDFA_UPLOAD_FOLDER` to an empty string to upload PDF/A files into the
@@ -1834,6 +1849,15 @@ ONEDRIVE_TENANT_ID=common
ONEDRIVE_REFRESH_TOKEN=your_refresh_token
ONEDRIVE_FOLDER_PATH=Documents/Uploads
# SharePoint Online
SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
SHAREPOINT_CLIENT_SECRET=your_client_secret
SHAREPOINT_TENANT_ID=your-tenant-id
SHAREPOINT_REFRESH_TOKEN=your_refresh_token
SHAREPOINT_SITE_URL=https://tenant.sharepoint.com/sites/sitename
SHAREPOINT_DOCUMENT_LIBRARY=Documents
SHAREPOINT_FOLDER_PATH=Uploads
# Amazon S3
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
docs/CredentialRotationGuide.md
+10 -1
View File
@@ -11,7 +11,7 @@ Credentials fall into two categories:
| Category | Examples |
|---|---|
| **API keys** | OpenAI API key, Azure AI key, Paperless-ngx API token, AWS access keys |
| **OAuth tokens / secrets** | Dropbox, Google Drive, OneDrive, Authentik client secrets and refresh tokens |
| **OAuth tokens / secrets** | Dropbox, Google Drive, OneDrive, SharePoint, Authentik client secrets and refresh tokens |
| **Passwords** | Admin password, Nextcloud, Email (SMTP), IMAP, FTP, SFTP, WebDAV |
| **Private keys** | SFTP private key and passphrase |
@@ -119,6 +119,15 @@ For service-account credentials (`google_drive_credentials_json`):
4. Re-authorize via the OAuth flow to get a fresh `onedrive_refresh_token`.
5. Delete the old client secret in Azure.
### SharePoint (Microsoft OAuth)
1. SharePoint uses the same Azure AD app registration as OneDrive.
2. In **Azure App Registrations**, navigate to **Certificates & secrets** for your app.
3. Add a new client secret.
4. Update `sharepoint_client_secret` in DocuElevate.
5. Re-authorize via the OAuth flow to get a fresh `sharepoint_refresh_token`.
6. Delete the old client secret in Azure.
### Authentik (OIDC)
1. In your Authentik admin panel, navigate to the DocuElevate application and regenerate the client secret.
docs/DeploymentGuide.md
+1 -1
View File
@@ -19,7 +19,7 @@ This guide covers all supported deployment methods for DocuElevate.
- Access to required external services (if configured):
- AI provider API key (OpenAI, Anthropic, Gemini, or other configured provider)
- Azure Document Intelligence
- Dropbox, Google Drive, OneDrive, S3, or other storage APIs
- Dropbox, Google Drive, OneDrive, SharePoint, S3, or other storage APIs
- SMTP / IMAP server (for email processing)
- Notification services (Discord, Telegram, etc.)
docs/MobileApp.md
+17 -2
View File
@@ -8,6 +8,7 @@ DocuElevate includes a native mobile application for iOS and Android built with
|---------|-----|---------|
| SSO login (OAuth2) | ✅ | ✅ |
| Local / basic auth login | ✅ | ✅ |
| QR code login (scan from web) | ✅ | ✅ |
| Auto-generated API token | ✅ | ✅ |
| Camera capture → upload | ✅ | ✅ |
| File picker upload | ✅ | ✅ |
@@ -112,6 +113,18 @@ When developing with **Expo Go** the app does not have the `docuelevate://` cust
No extra configuration is needed — just run `npx expo start` and scan the QR code with the **Expo Go** app.
### QR Code Login Flow
As an alternative to SSO, users can log in by scanning a QR code displayed in the web UI:
1. The authenticated web user navigates to **Profile → Security & Sessions → Log in on mobile via QR code**.
2. A QR code is displayed containing a deep link: `docuelevate://qr-login?token=<challenge_token>&server=<server_url>`.
3. In the mobile app, the user taps **Scan QR Code to Login**, which opens the device camera.
4. The app scans the QR code, extracts both the server URL and the challenge token, and calls `POST /api/qr-auth/claim`.
5. An API token is issued and stored securely — no need to enter the server URL manually.
> **Note:** The QR code already contains the server URL, so users do not need to type it in when using QR login.
### Auto-generated Mobile Token
When the mobile app completes login it automatically creates a named API token (`"Mobile App – <device name>"`) via `POST /api/mobile/generate-token`. This token:
@@ -286,7 +299,8 @@ mobile/
│ ├── (auth)/ # Unauthenticated route group
│ │ ├── _layout.tsx # Stack navigator (headerless)
│ │ ├── index.tsx # Welcome screen
│ │ └── login.tsx # Login screen
│ │ ├── login.tsx # Login screen
│ │ └── qr-scanner.tsx # QR code scanner screen
│ └── (tabs)/ # Authenticated route group
│ ├── _layout.tsx # Tab navigator
│ ├── index.tsx # Upload screen (default tab)
@@ -303,7 +317,8 @@ mobile/
├── hooks/
│ └── usePushNotifications.ts # Push token registration
├── screens/
│ ├── LoginScreen.tsx # Server URL + SSO button
│ ├── LoginScreen.tsx # Server URL + SSO button + QR code scanner
│ ├── QRScannerScreen.tsx # Camera-based QR code scanner for login
│ ├── UploadScreen.tsx # Camera capture + photo library + file picker
│ ├── FilesScreen.tsx # Processed document list
│ └── ProfileScreen.tsx # User profile + sign out
docs/README.md
+1
View File
@@ -22,6 +22,7 @@ Welcome to the DocuElevate documentation. This directory contains comprehensive
- [Google Drive Setup](GoogleDriveSetup.md) - How to set up Google Drive integration
- [Dropbox Setup](DropboxSetup.md) - How to set up Dropbox integration
- [OneDrive Setup](OneDriveSetup.md) - How to set up Microsoft OneDrive/Graph integration
- [SharePoint Setup](SharePointSetup.md) - How to set up Microsoft SharePoint Online integration
- [Amazon S3 Setup](AmazonS3Setup.md) - How to set up Amazon S3 integration
- [Authentication Setup](AuthenticationSetup.md) - How to set up user authentication
- [Notifications Setup](NotificationsSetup.md) - How to set up system notifications
docs/SettingsManagement.md
+1 -1
View File
@@ -28,7 +28,7 @@ Settings are organized into logical categories for easy navigation:
- **Authentication**: Login settings, session secrets, OAuth configuration, admin group
- **AI Services**: AI provider selection, model configuration, embeddings, and credentials (OpenAI, Azure, Anthropic, Gemini, Ollama, OpenRouter, Portkey, LiteLLM)
- **OCR Engines**: OCR provider selection and configuration (Tesseract, EasyOCR, Mistral, Google DocAI, AWS Textract)
- **Storage Providers**: Dropbox, Google Drive, OneDrive, S3, FTP, SFTP, WebDAV, Nextcloud, Paperless
- **Storage Providers**: Dropbox, Google Drive, OneDrive, SharePoint, S3, FTP, SFTP, WebDAV, Nextcloud, Paperless
- **Email**: SMTP configuration for sending emails
- **IMAP**: Email ingestion configuration (supports two mailbox accounts)
- **Monitoring**: Uptime Kuma integration
docs/SharePointSetup.md
+185
View File
@@ -0,0 +1,185 @@
# Setting up SharePoint Integration
This guide explains how to set up the Microsoft SharePoint Online integration for DocuElevate.
## Required Configuration Parameters
| **Variable** | **Description** |
|---------------------------------|-------------------------------------------------------|
| `SHAREPOINT_CLIENT_ID` | Azure AD application client ID |
| `SHAREPOINT_CLIENT_SECRET` | Azure AD application client secret |
| `SHAREPOINT_TENANT_ID` | Azure AD tenant ID (use "common" for multi-tenant apps) |
| `SHAREPOINT_REFRESH_TOKEN` | OAuth 2.0 refresh token |
| `SHAREPOINT_SITE_URL` | SharePoint site URL (e.g. `https://tenant.sharepoint.com/sites/sitename`) |
| `SHAREPOINT_DOCUMENT_LIBRARY` | Document library name (default: `Documents`) |
| `SHAREPOINT_FOLDER_PATH` | Subfolder path inside the document library |
For a complete list of configuration options, see the [Configuration Guide](ConfigurationGuide.md).
## Overview
SharePoint Online integration uses the same Microsoft Graph API as OneDrive. The key difference is that SharePoint targets a **site-specific document library** rather than a personal OneDrive. Documents are uploaded via chunked upload sessions for reliability with large files.
> **Tip:** If you already have an Azure AD app registration for OneDrive, you can reuse it for SharePoint — just add the `Sites.ReadWrite.All` permission.
## Setup Steps
### 1. Register an application in Azure Active Directory
If you don't already have an app registration (e.g. from OneDrive setup):
1. Go to the [Azure Portal](https://portal.azure.com/)
2. Navigate to **Azure Active Directory** > **App registrations**
3. Click **New registration**
4. Enter a name for your application (e.g., "DocuElevate")
5. For **Supported account types**, select:
- **Single tenant**: "Accounts in this organizational directory only"
- **Multi-tenant**: "Accounts in any organizational directory"
6. For **Redirect URI**, select "Web" and enter your callback URL (e.g., `https://your-domain.com/onedrive-callback`)
7. Click **Register**
### 2. Get Application (client) ID
1. After registration, note the **Application (client) ID** from the overview page
2. Set this value as `SHAREPOINT_CLIENT_ID`
### 3. Create a client secret
1. In your application page, go to **Certificates & secrets**
2. Under **Client secrets**, click **New client secret**
3. Add a description and select an expiration period
4. Click **Add** and immediately copy the secret value (it will only be shown once)
5. Set this value as `SHAREPOINT_CLIENT_SECRET`
### 4. Configure API permissions
1. In your application page, go to **API permissions**
2. Click **Add a permission**
3. Select **Microsoft Graph**
4. For **delegated permissions** (user-context access), add:
- `Sites.ReadWrite.All` — Read and write items in all site collections
- `offline_access` — Required for refresh tokens
5. For **application permissions** (app-only access without a user), add:
- `Sites.ReadWrite.All` — Read and write items in all site collections
6. Click **Add permissions**
7. Click **Grant admin consent** (requires admin privileges)
> **Important:** SharePoint access requires `Sites.ReadWrite.All` rather than the `Files.ReadWrite` permission used by OneDrive.
### 5. Get your Tenant ID
1. In the Azure Portal, find your **Tenant ID** (also called "Directory ID")
2. It is on the **Azure Active Directory** overview page
3. Set this value as `SHAREPOINT_TENANT_ID`
### 6. Generate a Refresh Token
#### Using the OneDrive Auth Wizard
The SharePoint integration reuses the same MSAL token flow as OneDrive:
1. Navigate to `/onedrive-setup`
2. Enter your SharePoint Client ID and Tenant ID
3. Click **Start Authentication Flow** and follow the prompts
4. Copy the generated refresh token and set it as `SHAREPOINT_REFRESH_TOKEN`
#### Manual Method
1. Open the following URL in your browser (replace placeholders):
```
https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/v2.0/authorize?client_id=YOUR_CLIENT_ID&response_type=code&redirect_uri=YOUR_REDIRECT_URI&response_mode=query&scope=https://graph.microsoft.com/.default offline_access&prompt=consent
```
2. Sign in with your Microsoft work account
3. After authentication, copy the `code` parameter from the redirect URL
4. Exchange the code for tokens:
```bash
curl -X POST https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/v2.0/token \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "client_id=YOUR_CLIENT_ID&scope=https://graph.microsoft.com/.default offline_access&code=YOUR_AUTH_CODE&redirect_uri=YOUR_REDIRECT_URI&grant_type=authorization_code&client_secret=YOUR_CLIENT_SECRET"
```
5. From the response JSON, copy the `refresh_token` value
6. Set this as `SHAREPOINT_REFRESH_TOKEN`
### 7. Find your SharePoint Site URL
Your SharePoint site URL follows the pattern:
```
https://YOUR-TENANT.sharepoint.com/sites/SITE-NAME
```
For example:
- `https://contoso.sharepoint.com/sites/documents`
- `https://contoso.sharepoint.com/sites/engineering-team`
Set this as `SHAREPOINT_SITE_URL`.
### 8. Choose your Document Library
Each SharePoint site has one or more document libraries. The default library is usually called `Documents` (or `Shared Documents`). You can find your library names by navigating to your SharePoint site in a browser and looking at the left sidebar.
Set the library name as `SHAREPOINT_DOCUMENT_LIBRARY` (default: `Documents`).
### 9. Set the Upload Folder (Optional)
If you want documents to be uploaded into a subfolder inside the library, set `SHAREPOINT_FOLDER_PATH`. For example, `Uploads` or `DocuElevate/Processed`.
## App-Only Access (No User Token)
For fully automated scenarios without user interaction:
1. Add **Application permissions** (not Delegated) for `Sites.ReadWrite.All`
2. Grant admin consent
3. Set `SHAREPOINT_TENANT_ID` to your organization's tenant ID
4. Leave `SHAREPOINT_REFRESH_TOKEN` empty — the app will use the client credentials flow
> **Note:** Client credentials flow requires a specific tenant ID (not "common").
## Configuration Examples
**With Refresh Token (Delegated Permissions):**
```dotenv
SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
SHAREPOINT_CLIENT_SECRET=your_client_secret
SHAREPOINT_TENANT_ID=87654321-4321-4321-4321-210987654321
SHAREPOINT_REFRESH_TOKEN=your_refresh_token
SHAREPOINT_SITE_URL=https://contoso.sharepoint.com/sites/documents
SHAREPOINT_DOCUMENT_LIBRARY=Documents
SHAREPOINT_FOLDER_PATH=Uploads
```
**App-Only Access (Application Permissions):**
```dotenv
SHAREPOINT_CLIENT_ID=12345678-1234-1234-1234-123456789012
SHAREPOINT_CLIENT_SECRET=your_client_secret
SHAREPOINT_TENANT_ID=87654321-4321-4321-4321-210987654321
# No refresh token needed for app-only access
SHAREPOINT_SITE_URL=https://contoso.sharepoint.com/sites/documents
SHAREPOINT_DOCUMENT_LIBRARY=Shared Documents
SHAREPOINT_FOLDER_PATH=DocuElevate/Processed
```
## Troubleshooting
### "Failed to resolve SharePoint site"
- Verify `SHAREPOINT_SITE_URL` is correct and accessible
- Ensure your app has `Sites.ReadWrite.All` permission with admin consent
- Check that the site exists and your account has access to it
### "Document library not found"
- Verify the library name in `SHAREPOINT_DOCUMENT_LIBRARY` matches exactly (case-insensitive)
- Navigate to your SharePoint site in a browser to confirm the library name
- Common names: `Documents`, `Shared Documents`
### Token errors
- If using a refresh token, try re-authorizing via the OAuth flow
- Ensure `offline_access` scope is included in your permissions
- For app-only access, verify the tenant ID is not set to "common"
### Permission errors
- Ensure an admin has granted consent for `Sites.ReadWrite.All`
- Verify the app registration has the correct permissions
- Check that the site's sharing settings allow API access
docs/StorageArchitecture.md
+1
View File
@@ -341,6 +341,7 @@ in task messages or logs.
| `S3` | boto3 `upload_file`, per-user access key |
| `GOOGLE_DRIVE` | Google Drive API v3, OAuth or service account |
| `ONEDRIVE` | Microsoft Graph API, MSAL confidential-client |
| `SHAREPOINT` | Microsoft Graph API, site/drive resolution + chunked upload |
| `WEBDAV` | HTTP PUT request, Basic Auth |
| `NEXTCLOUD` | WebDAV (same as WEBDAV, Nextcloud-compatible path) |
| `FTP` | ftplib FTPS (TLS preferred, plaintext configurable) |
docs/UserGuide.md
+1 -1
View File
@@ -170,7 +170,7 @@ The **Integrations** page (`/integrations`) provides a unified view of all your
- **S3** — bucket, region, access key, secret key
- **WebDAV / Nextcloud** — URL, folder, username, password
- **FTP / SFTP** — host, port, remote path, username, password
- **Dropbox / Google Drive / OneDrive** — folder path, with a link to the OAuth setup page
- **Dropbox / Google Drive / OneDrive / SharePoint** — folder path, with a link to the OAuth setup page
- **Email Forward** — recipient email address
- **Watch Folder** — source type (Local, S3, Dropbox, Google Drive, OneDrive, Nextcloud, WebDAV), per-type config fields, delete after processing toggle
- **Paperless NGX** — URL and API token