- Sanitise uploaded filenames with os.path.basename() to prevent path traversal
- Change TestWebhookDispatchIntegration marker from unit to integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add REST hooks subscription endpoints, incoming action endpoints, and
Zapier-compatible flat payload format for automation platform integration.
- AutomationHook model for webhook subscriptions
- POST /api/automation/hooks/subscribe and DELETE /hooks/{id}
- GET /api/automation/triggers/sample/{event} for Zapier field mapping
- POST /api/automation/actions/upload for incoming document uploads
- Celery task with retry for async hook delivery
- Integration with existing webhook dispatch flow
- 30 passing tests covering all new functionality
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Test coverage:
- Add config=None branch test for dropbox, onedrive, google_drive views (100% coverage)
- Add WATCH_FOLDER source-type test (folder_path vs folder key)
- Add integration-not-found fallback-to-admin-mode test
WCAG 2.1 AA fixes:
- Add aria-labelledby="modalTitle" to role="dialog" modals in setup templates
- Add aria-hidden="true" to decorative SVGs in callback templates
- Add role="status" aria-label="Loading" to spinner divs
- Add aria-live="polite" to processing-message and success/folder-selection regions
- Add role="alert" aria-live="assertive" to error containers
- Update "Return to Setup" link to preserve integration_id in user mode
Docs: update DropboxSetup.md, GoogleDriveSetup.md, OneDriveSetup.md with per-user OAuth flow section
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add test_gmail_labels_disabled_when_gmail_apply_labels_false to verify
that Gmail star/label operations are skipped when gmail_apply_labels=False.
Add four tests to TestPullUserIntegrationWatchFolders for cloud source
type dispatching: S3, Dropbox, unknown provider, and explicit local type.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add `from app.tasks.upload_to_user_integration import upload_to_user_integration`
to app/celery_worker.py — fixes TestCeleryWorkerConfig test that enforces
every tasks/ module is registered with Celery.
- Add tests/test_upload_handlers.py with 43 unit tests covering all 11
per-type upload handler functions (_upload_dropbox, _upload_s3,
_upload_google_drive, _upload_onedrive, _upload_webdav, _upload_nextcloud,
_upload_ftp, _upload_sftp, _upload_paperless, _upload_email, _upload_rclone)
plus 2 additional finalize_document_storage branch tests. All external
libraries (dropbox, boto3, msal, paramiko, smtplib, subprocess, requests)
are mocked so tests are hermetic and fast. Coverage on changed files:
upload_to_user_integration.py 94.71%, finalize_document_storage.py 95.51%
(both well above the 70% Codecov diff threshold).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Move init_sentry() from module level into the FastAPI lifespan context
manager, immediately after load_settings_from_db() completes. This
ensures that SENTRY_DSN and other Sentry settings configured via the
database admin UI are picked up on every restart.
Also update tests and docs accordingly.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Use per-password random salt with PBKDF2-HMAC-SHA256 (stored as salt:hash)
- Increase PBKDF2 iterations to 600,000 (OWASP 2023 recommendation)
- Password for downloads now accepted via POST body (never URL query param)
- Fail download request if view count cannot be incremented (prevents bypass)
- Update tests to match new hashing format and POST password download
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SharedLink model with token, expiry, view limit, password hash
- Add migration 025_add_shared_links
- Add API endpoints: create, list, revoke (auth) + public info/download
- Add management UI at /shared-links with revoke controls
- Add public share landing page at /share/{token}
- Add Share button on file_view.html
- Add Shared Links to user dropdown in common.js
- Write 35 unit tests covering all scenarios
- Update UserGuide.md with sharing documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix _get_pipeline_ocr_language: remove redundant `or None` in step_config.get()
- Add Session type hint to _get_pipeline_ocr_language db parameter via TYPE_CHECKING
- Update process_with_ocr to use modern str | None syntax instead of Optional[str]
- Fix test_get_pipeline_ocr_language_explicit_pipeline_takes_priority: properly add
sys_step to db_session so the system pipeline step is persisted in the test DB
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add app/tasks/upload_to_user_integration.py: new Celery task that
uploads a processed document to a specific UserIntegration using its
own per-user config and Fernet-decrypted credentials. Supports all
DESTINATION types: Dropbox, S3, Google Drive, OneDrive, WebDAV,
Nextcloud, FTP, SFTP, Paperless-ngx, Email (SMTP), and Rclone.
- Extend app/tasks/send_to_all.py: add send_to_user_destinations task
(queries active DESTINATION UserIntegrations for an owner and
dispatches one upload_to_user_integration task per integration) and
get_user_destination_count helper used by finalize_document_storage.
- Refactor app/tasks/finalize_document_storage.py: after processing,
look up the document owner; if the owner has active DESTINATION
integrations route exclusively to those (user-specific routing),
otherwise fall back to the global send_to_all_destinations.
- Update tests/test_finalize_storage.py: add autouse fixture to prevent
Redis hangs, update all existing tests with new mock parameters, add
TestFinalizeDocumentStorageUserRouting class with four new tests that
validate user-specific vs global routing decisions.
- Add tests/test_user_integration_upload.py: 14 new unit tests covering
upload_to_user_integration (handler dispatch, error persistence,
last_used_at update, credential decryption, skip for unknown types)
and send_to_user_destinations / get_user_destination_count.
- Update docs/StorageArchitecture.md: document the user-specific
destination routing feature, supported types, multiple-destination
behaviour, and global fallback semantics.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Convert f-string log interpolation to %s-style formatting in
app/api/pipelines.py and app/api/saved_searches.py to prevent
clear-text logging of request-derived data (CodeQL: clear-text
logging of sensitive information)
- Replace plain hashlib.sha256() with PBKDF2-HMAC-SHA256 via
hash_token() in app/auth.py for Bearer token verification,
consistent with how tokens are stored in api_tokens.py (CodeQL:
use of weak cryptographic hashing on sensitive data)
- Remove redundant {exc} from logger.exception() calls (the
traceback is already captured by logger.exception())
- Update test to verify PBKDF2 hash instead of plain SHA-256
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Setup pages now accept an integration_id query param to save OAuth
credentials to the user's personal UserIntegration record instead
of global settings. The integrations dashboard shows an "Authorize"
button for OAuth types (Dropbox, Google Drive, OneDrive) that need
credentials.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add owner_id parameter to pull_inbox() and fetch_attachments_and_enqueue()
to attribute ingested documents to the correct user
- Add _pull_user_integration_imap() to poll IMAP sources from UserIntegration model
- Add _pull_user_integration_watch_folders() to scan watch folders from UserIntegration model
- Add _is_safe_watch_path() for path traversal security on user-configured paths
- Add _scan_user_watch_folder() that passes owner_id to _enqueue_file()
- Update _enqueue_file() to forward owner_id to process_document/convert_to_pdf
- Update celery beat schedule to always enable IMAP and watch folder polling
(user integrations can exist without system-level config)
- Ensure individual connection failures don't crash the polling loop
- Update existing tests for new function signatures
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Enable Zammad settings via mock and verify user data appears in the
rendered HTML. Add tests for chat widget, email-only fallback, and
display_name fallback. Also improve JS variable naming in help.html.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Pass authenticated user's name, email, and username to Zammad form
and chat widgets. For the ticket form: pre-fill name/email fields
and append a User Context metadata block to the ticket body via
$.ajaxPrefilter. For the chat widget: pass name/email to the
ZammadChat constructor.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The oauthLink() function in integrations_dashboard.html returned
/dropbox, /google-drive, /onedrive which are not valid routes
(404). The actual view routes are /dropbox-setup,
/google-drive-setup, /onedrive-setup.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]