Merge origin/main into copilot/add-sharepoint-integration.
All four conflicts were resolved by keeping both the SharePoint
additions (from this branch) and the iCloud additions (from main):
- app/models.py: added both SHAREPOINT and ICLOUD to IntegrationType
- app/tasks/send_to_all.py: added both to service_map and services list
- app/tasks/upload_to_user_integration.py: kept both upload handlers
- frontend/templates/files.html: added both filter options
No database migration conflicts — SharePoint does not require schema changes.
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add Microsoft SharePoint Online as a storage provider using the
Microsoft Graph API. Includes OAuth2 authentication via MSAL,
site/drive resolution, chunked upload sessions, and metadata sync.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add app/tasks/upload_to_user_integration.py: new Celery task that
uploads a processed document to a specific UserIntegration using its
own per-user config and Fernet-decrypted credentials. Supports all
DESTINATION types: Dropbox, S3, Google Drive, OneDrive, WebDAV,
Nextcloud, FTP, SFTP, Paperless-ngx, Email (SMTP), and Rclone.
- Extend app/tasks/send_to_all.py: add send_to_user_destinations task
(queries active DESTINATION UserIntegrations for an owner and
dispatches one upload_to_user_integration task per integration) and
get_user_destination_count helper used by finalize_document_storage.
- Refactor app/tasks/finalize_document_storage.py: after processing,
look up the document owner; if the owner has active DESTINATION
integrations route exclusively to those (user-specific routing),
otherwise fall back to the global send_to_all_destinations.
- Update tests/test_finalize_storage.py: add autouse fixture to prevent
Redis hangs, update all existing tests with new mock parameters, add
TestFinalizeDocumentStorageUserRouting class with four new tests that
validate user-specific vs global routing decisions.
- Add tests/test_user_integration_upload.py: 14 new unit tests covering
upload_to_user_integration (handler dispatch, error persistence,
last_used_at update, credential decryption, skip for unknown types)
and send_to_user_destinations / get_user_destination_count.
- Update docs/StorageArchitecture.md: document the user-specific
destination routing feature, supported types, multiple-destination
behaviour, and global fallback semantics.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Create app/templates/email/default.html (fixes 'default.html not found' error)
- Add DEST_EMAIL_* settings to app/config.py (decoupled from shared EMAIL_* settings)
- Update upload_to_email task to use dest_email_* settings exclusively
- Update _should_upload_to_email() to check dest_email_* settings
- Update config validator, providers, and settings_service for dest_email_*
- Update .env.demo and docs/ConfigurationGuide.md
- Update all tests to use dest_email_* settings where appropriate"
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace LIKE queries with exact matches in fallback lookups
- Add comments clarifying fallback queries should not be needed
- Fix duplicate comment in embed_metadata_into_pdf
- Add missing file_id parameter to log_task_progress call
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added database logging to all major processing tasks
- Created API endpoints for retrieving processing logs
- Updated frontend to display processing logs per file
- Logging includes: process_document, convert_to_pdf, extract_metadata_with_gpt, embed_metadata_into_pdf, finalize_document_storage, send_to_all_destinations
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Implemented `google_drive_callback.html` for processing Google Drive authorization, including UI for success and error states.
- Added JavaScript functionality for exchanging authorization codes, saving settings, and handling folder selection.
- Created `google_drive_callback_error.html` to display error messages during the authorization process.
copilot-swe-agent[bot]