Commit Graph

1829 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] d6c21b8026 docs: update setup guides and API docs for folder browser and system credentials
Updated DropboxSetup.md, OneDriveSetup.md, and GoogleDriveSetup.md to
document the new system credentials toggle and folder browser features.
Added API documentation for POST /api/dropbox/list-folders and
POST /api/onedrive/list-folders endpoints.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:43:08 +00:00
copilot-swe-agent[bot] 1e1e6e6280 test: add tests for folder browser APIs and system credentials toggle
Added tests for POST /api/dropbox/list-folders (7 tests) and
POST /api/onedrive/list-folders (7 tests) covering success, subfolder
navigation, empty directories, auth errors, API errors, path
normalization, and alphabetical sorting.

Added view tests for system credentials toggle visibility in Dropbox
setup wizard.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:40:51 +00:00
copilot-swe-agent[bot] a8423064ec feat(api): add folder browser API endpoints and UI for Dropbox and OneDrive
Added POST /api/dropbox/list-folders and POST /api/onedrive/list-folders
endpoints that accept an OAuth access_token and return folder listings.

After successful OAuth authorization in the callback pages, users now
see an interactive folder browser to select the target folder for their
integration. The selected folder is saved to the integration config.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:21:52 +00:00
copilot-swe-agent[bot] 23c8c76b39 feat(auth): default to system-wide app credentials in OAuth wizards for user mode
When system-wide Dropbox, Google Drive, or OneDrive app credentials are
configured by the admin, user-mode OAuth wizards now default to using
them. A toggle lets users switch to custom credentials if needed. This
removes the need for end users to register their own cloud provider apps.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:17:40 +00:00
copilot-swe-agent[bot] ed01952610 feat(ui): replace manual credential fields with OAuth wizard flow for watch folder sources
Watch folders using Dropbox, Google Drive, or OneDrive now show only the
folder path field and an info box directing users to use the Authorize
button after saving — matching the destination integration pattern.
Manual credential fields (refresh token, app key, app secret, etc.)
have been removed for these OAuth-backed source types.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-20 13:12:39 +00:00
copilot-swe-agent[bot] 954e67640c Initial plan 2026-03-09 23:13:07 +00:00
github-actions[bot] e709f2646c chore(release): update build metadata files [skip ci] 2026-03-09 23:01:00 +00:00
semantic-release d5ed512612 0.114.0
Automatically generated by python-semantic-release
2026-03-09 23:00:57 +00:00
Christian Krakau-Louis 5fd3f0661b Merge pull request #577 from christianlouis/copilot/create-user-auth-workflow
fix(templates): remove orphan `</div>` in google_drive.html breaking HTML accessibility lint
2026-03-10 00:00:39 +01:00
copilot-swe-agent[bot] 256d9cb674 fix(templates): remove orphan </div> in google_drive.html causing HTML accessibility lint failure
The extra </div> at line 260 (before the {% endif %} closing the admin
auth-method block) was introduced when the per-user OAuth wizard PR
restructured the admin section. Removing it balances the div tree so
that the outer container div is properly closed by line 427.

djlint frontend/templates/ --lint now reports 0 errors across 58 files.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 21:46:38 +00:00
github-actions[bot] 5d5784f069 chore(release): update build metadata files [skip ci] 2026-03-09 21:30:44 +00:00
semantic-release 91549c8f5e 0.113.0
Automatically generated by python-semantic-release
2026-03-09 21:30:41 +00:00
Christian Krakau-Louis 4555cdb7df Merge pull request #576 from christianlouis/copilot/add-scheduled-batch-processing
feat: scheduled batch processing — 8 admin-managed jobs, DB-driven beat schedule, 100% coverage
2026-03-09 22:30:20 +01:00
copilot-swe-agent[bot] 0529ae53ff fix: 100% coverage, WCAG accessibility fixes, and documentation updates for per-user OAuth wizards
Test coverage:
- Add config=None branch test for dropbox, onedrive, google_drive views (100% coverage)
- Add WATCH_FOLDER source-type test (folder_path vs folder key)
- Add integration-not-found fallback-to-admin-mode test

WCAG 2.1 AA fixes:
- Add aria-labelledby="modalTitle" to role="dialog" modals in setup templates
- Add aria-hidden="true" to decorative SVGs in callback templates
- Add role="status" aria-label="Loading" to spinner divs
- Add aria-live="polite" to processing-message and success/folder-selection regions
- Add role="alert" aria-live="assertive" to error containers
- Update "Return to Setup" link to preserve integration_id in user mode

Docs: update DropboxSetup.md, GoogleDriveSetup.md, OneDriveSetup.md with per-user OAuth flow section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 20:19:19 +00:00
copilot-swe-agent[bot] ec77c51cb5 fix: code standards audit — 100% coverage, Pydantic v2 model_config, asyncio.run(), targeted edge-case tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 19:00:13 +00:00
copilot-swe-agent[bot] 2d01e71afd test(views): add user-mode coverage tests and fix dashboard helper duplication
- Add tests for user-mode with invalid JSON config (covers exception handling path)
- Add tests for user-mode with valid config (verifies folder path pre-population)
- Extract _watchFolderOAuthSource() helper to reduce isOAuthType/oauthLink duplication

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:32:04 +00:00
copilot-swe-agent[bot] 5bc1fac79c feat(tasks): add 5 additional scheduled jobs (expire_shared_links, prune_processing_logs, prune_old_notifications, backfill_missing_metadata, sync_search_index)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:25:26 +00:00
copilot-swe-agent[bot] c71f33a214 feat(integrations): add per-user OAuth wizard with user-mode for Dropbox, OneDrive, Google Drive
- Add user_mode to dropbox/onedrive/google_drive setup views that loads integration config
- Show user-friendly auth wizard when integration_id is provided (user mode)
- In user mode: show integration name, current folder, back-to-integrations link
- In callback templates: only save credentials (not config) for user integrations
- In integrations dashboard: show Authorize/Re-Authorize button for all OAuth types
- Add WATCH_FOLDER OAuth support: detect source_type in config for auth button
- isOAuthType() and oauthLink() now accept full integration object

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:24:07 +00:00
copilot-swe-agent[bot] c2d56d7475 feat(tasks): add scheduled batch processing infrastructure (model, migration, tasks, API, view, template)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:01:34 +00:00
copilot-swe-agent[bot] 80b1b01beb Initial plan 2026-03-09 17:58:17 +00:00
copilot-swe-agent[bot] 07d90758b8 Initial plan 2026-03-09 17:52:43 +00:00
github-actions[bot] 04d51bab22 chore(release): update build metadata files [skip ci] 2026-03-09 12:01:14 +00:00
semantic-release 8fa3e9bc4a 0.112.0
Automatically generated by python-semantic-release
2026-03-09 12:01:11 +00:00
Christian Krakau-Louis 2132e7e8d1 Merge pull request #575 from christianlouis/copilot/fix-imap-integration-ui-issues
feat(integrations): IMAP Gmail labels/delete controls, Watch Folder cloud sources, and comprehensive test coverage
2026-03-09 13:00:50 +01:00
copilot-swe-agent[bot] 9a342af609 fix(imap): normalize CRLF line endings to LF in imap_tasks.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 09:42:20 +00:00
copilot-swe-agent[bot] 1741ab1011 test(tasks): add comprehensive tests for per-user cloud scan functions and gmail_apply_labels
Add 68 new unit tests covering:
- _scan_user_s3_folder: 10 tests (bucket config, credentials, client creation,
  pagination, download, cache, file types, delete, error handling)
- _scan_user_dropbox_folder: 9 tests (credentials, folder path, auth, listing,
  download, cache, non-file entries, delete, download failure)
- _scan_user_google_drive_folder: 8 tests (credentials JSON, folder ID, auth,
  download, cache, delete, download/listing failures)
- _scan_user_onedrive_folder: 10 tests (credentials, folder path, token exchange,
  download, folders, cache, download URL, delete, download/listing failures)
- _scan_user_nextcloud_folder: 9 tests (settings, PROPFIND, XML parse, download,
  self-entry, cache, delete, download failure, absolute href)
- _scan_user_webdav_folder: 9 tests (URL config, PROPFIND, XML parse, download,
  directories, cache, delete, download failure, absolute href)
- Dispatch tests for google_drive, onedrive, nextcloud, webdav source types
- gmail_apply_labels forwarding and default-to-true in IMAP integration

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-09 09:29:43 +00:00
copilot-swe-agent[bot] 4acad71c8f fix: address code review feedback - fix type hint and rename test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:38:50 +00:00
copilot-swe-agent[bot] 8c62a6b714 test(integrations): add tests for gmail_apply_labels and cloud watch folder dispatch
docs(integrations): update UserGuide and ConfigurationGuide for new IMAP and Watch Folder features

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:35:30 +00:00
copilot-swe-agent[bot] 79fc11b452 test: add tests for gmail_apply_labels=False and cloud source type dispatching
Add test_gmail_labels_disabled_when_gmail_apply_labels_false to verify
that Gmail star/label operations are skipped when gmail_apply_labels=False.

Add four tests to TestPullUserIntegrationWatchFolders for cloud source
type dispatching: S3, Dropbox, unknown provider, and explicit local type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-09 02:26:10 +00:00
copilot-swe-agent[bot] e7d1289f1b feat(integrations): add IMAP Gmail labels/delete and Watch Folder cloud source support
- Add delete_after_process and gmail_apply_labels checkboxes to IMAP integration UI
- Add source_type selector (Local/S3/Dropbox/Google Drive/OneDrive/Nextcloud/WebDAV) to Watch Folder UI
- Add per-source-type config and credential fields for watch folders
- Add delete_after_process checkbox to Watch Folder settings
- Fix field name mismatch (path → folder_path) in Watch Folder UI
- Update pull_inbox() to accept and respect gmail_apply_labels parameter
- Update _pull_user_integration_imap() to read gmail_apply_labels from config
- Add per-user cloud scanning functions for S3, Dropbox, Google Drive, OneDrive, Nextcloud, WebDAV
- Update _pull_user_integration_watch_folders() to dispatch based on source_type
- Update IMAP and WATCH_FOLDER config shape documentation in models.py

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:15:03 +00:00
copilot-swe-agent[bot] 61e402f785 Initial plan 2026-03-09 01:52:58 +00:00
github-actions[bot] 5f548e29fa chore(release): update build metadata files [skip ci] 2026-03-09 01:47:12 +00:00
semantic-release aea9e227db 0.111.1
Automatically generated by python-semantic-release
2026-03-09 01:47:09 +00:00
Christian Krakau-Louis b71b18d184 Merge pull request #574 from christianlouis/copilot/fix-cors-header-test-error
fix(migrations): resolve duplicate Alembic heads causing test INTERNALERROR
2026-03-09 02:46:51 +01:00
copilot-swe-agent[bot] c374ba3ae9 fix(migrations): resolve duplicate Alembic heads causing test failures
Two migration files (025_add_shared_links and 025_add_user_notifications)
both had down_revision pointing to 024_add_api_tokens, creating multiple
migration heads. This caused alembic stamp/upgrade head to fail with
'Multiple heads are present; please specify a single target revision'
whenever init_db() ran (e.g. in TestClient fixtures).

Fix: chain 025_add_user_notifications from 025_add_shared_links so the
migration history is linear:
024_add_api_tokens → 025_add_shared_links → 025_add_user_notifications

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 01:11:26 +00:00
copilot-swe-agent[bot] 32e73727ad Initial plan 2026-03-09 00:47:56 +00:00
github-actions[bot] e4157e3373 chore(release): update build metadata files [skip ci] 2026-03-09 00:06:00 +00:00
semantic-release 708962b6a7 0.111.0
Automatically generated by python-semantic-release
2026-03-09 00:05:57 +00:00
Christian Krakau-Louis 0b9bba7158 Merge pull request #562 from christianlouis/copilot/refactor-dynamic-routing-user-destinations
fix(tasks): register upload_to_user_integration in Celery and add handler coverage
2026-03-09 01:05:36 +01:00
github-actions[bot] 39150cb9ab chore(release): update build metadata files [skip ci] 2026-03-08 23:46:58 +00:00
semantic-release ea2e927054 0.110.1
Automatically generated by python-semantic-release
2026-03-08 23:46:54 +00:00
Christian Krakau-Louis e916001e08 Merge pull request #573 from christianlouis/copilot/fix-sentry-settings-startup
fix(sentry): initialize Sentry after DB settings load in lifespan
2026-03-09 00:46:36 +01:00
github-actions[bot] b22f9a6af6 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-08 22:40:15 +00:00
copilot-swe-agent[bot] 5fb11b978c fix(tasks): register upload_to_user_integration in celery_worker and add handler tests
- Add `from app.tasks.upload_to_user_integration import upload_to_user_integration`
  to app/celery_worker.py — fixes TestCeleryWorkerConfig test that enforces
  every tasks/ module is registered with Celery.

- Add tests/test_upload_handlers.py with 43 unit tests covering all 11
  per-type upload handler functions (_upload_dropbox, _upload_s3,
  _upload_google_drive, _upload_onedrive, _upload_webdav, _upload_nextcloud,
  _upload_ftp, _upload_sftp, _upload_paperless, _upload_email, _upload_rclone)
  plus 2 additional finalize_document_storage branch tests. All external
  libraries (dropbox, boto3, msal, paramiko, smtplib, subprocess, requests)
  are mocked so tests are hermetic and fast. Coverage on changed files:
  upload_to_user_integration.py 94.71%, finalize_document_storage.py 95.51%
  (both well above the 70% Codecov diff threshold).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 22:21:15 +00:00
copilot-swe-agent[bot] 5864a4ed06 fix(sentry): initialize Sentry after DB settings are loaded in lifespan
Move init_sentry() from module level into the FastAPI lifespan context
manager, immediately after load_settings_from_db() completes. This
ensures that SENTRY_DSN and other Sentry settings configured via the
database admin UI are picked up on every restart.

Also update tests and docs accordingly.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 22:17:18 +00:00
github-actions[bot] a171186f93 chore(release): update build metadata files [skip ci] 2026-03-08 22:13:18 +00:00
semantic-release 50271ff21f 0.110.0
Automatically generated by python-semantic-release
2026-03-08 22:13:16 +00:00
Christian Krakau-Louis 382e082198 Merge pull request #569 from christianlouis/copilot/add-document-sharing-feature
feat: document sharing with expiring links
2026-03-08 23:12:58 +01:00
github-actions[bot] e5d4f8b58f style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-08 22:12:44 +00:00
Christian Krakau-Louis 201d7c8fdc Merge branch 'main' into copilot/add-document-sharing-feature 2026-03-08 23:12:27 +01:00