Commit Graph

1833 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] a08b103271 fix: merge main into feature branch - resolve all merge conflicts cleanly
Merges origin/main (v0.156.0) into the classification feature branch,
properly resolving all 23 merge conflicts:

- Auto-generated files (BUILD_DATE, VERSION, etc.): accept main's version
- Non-classification files (SharePoint, QR auth, session mgmt, mobile): accept main's version
- Classification files (api/__init__.py, models.py, migrations/env.py, conftest.py):
  keep classification additions alongside main's content

Previously the branch was incorrectly removing files from main (SharePoint
integration, QR scanner, session management). This merge properly preserves
all main branch content while maintaining the classification feature additions.

Migration chain validated: 038_add_classification_rules chains from
037_add_user_sessions_and_qr_challenges.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 16:14:45 +00:00
copilot-swe-agent[bot] 342e2d1614 fix: merge main into feature branch - resolve all 18 conflicts cleanly
Merges origin/main (v0.155.0) into the classification feature branch,
resolving all 18 conflicted files by accepting main's version and
re-applying only classification-specific additions:

- Renumber migration from 037 to 038 (chains from 037_user_sessions)
- Re-add ClassificationRuleModel to models.py, env.py, conftest.py
- Re-add classification_rules_router to api/__init__.py
- All session management, QR auth, and devices code preserved from main

Migration chain validated. 62 classification tests pass.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 13:22:28 +00:00
copilot-swe-agent[bot] 7e1ed14ee6 fix: address code review feedback - remove extra blank lines from merge
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 11:12:21 +00:00
copilot-swe-agent[bot] c7d67031d4 fix: resolve merge conflicts with main branch
Merge origin/main into feature branch, resolving 18 conflicts:
- app/api/__init__.py: add classification_rules_router alongside system_reset_router
- app/config.py: keep system reset settings from main
- app/models.py: keep ClassificationRuleModel alongside main's models
- app/utils/settings_service.py: keep factory reset settings from main
- app/views/__init__.py: keep system_reset_router from main
- tests/conftest.py: add ClassificationRuleModel import
- migrations/env.py: add ClassificationRuleModel import
- .env.demo, docs/*, frontend/*, mobile/*: keep additions from main
- BUILD_DATE, GIT_SHA, RUNTIME_INFO, VERSION, CHANGELOG.md: accept main's version

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-17 11:09:39 +00:00
copilot-swe-agent[bot] bdc26846bd refactor: address code review feedback - simplify dispatch, add warning log
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:36:24 +00:00
copilot-swe-agent[bot] 651b48658c fix: resolve merge conflicts with main branch
Merge origin/main into feature branch, resolving 3 conflicts:
- app/api/__init__.py: add classification_rules_router alongside new
  routers from main (audit_logs, i18n, mobile, compliance, translation)
- app/models.py: keep ClassificationRuleModel alongside new models from
  main (MobileDevice, ComplianceTemplate, PipelineRoutingRule)
- tests/conftest.py: import both ClassificationRuleModel and new models
  from main (AuditLog, ComplianceTemplate)

Also renumber migration from 027 to 037 to chain from the latest
migration on main (036_add_document_translation_fields).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 22:33:59 +00:00
copilot-swe-agent[bot] 23498e0a98 fix: address code review feedback on spelling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:40:09 +00:00
copilot-swe-agent[bot] bd898605de docs: add classification rules API and user guide documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:39:09 +00:00
copilot-swe-agent[bot] df051e8b81 feat(classify): add rule-based document classification engine, task, and API
Implements the classify pipeline step with:
- Classification rules engine (app/utils/classification_rules.py) with
  pre-built categories (invoice, contract, receipt, letter, report,
  bank_statement, tax_document, insurance, payslip) and support for
  filename patterns, content keywords, and metadata matching rules
- Celery task (app/tasks/classify_document.py) that runs as a pipeline step
- CRUD API (app/api/classification_rules.py) for managing custom rules
- ClassificationRuleModel in app/models.py with migration 027
- Updated pipeline step config_schema and stage mapping
- Comprehensive tests for engine, API, and task

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 23:38:05 +00:00
copilot-swe-agent[bot] e6dfa079cc Initial plan 2026-03-09 23:05:35 +00:00
github-actions[bot] e709f2646c chore(release): update build metadata files [skip ci] 2026-03-09 23:01:00 +00:00
semantic-release d5ed512612 0.114.0
Automatically generated by python-semantic-release
2026-03-09 23:00:57 +00:00
Christian Krakau-Louis 5fd3f0661b Merge pull request #577 from christianlouis/copilot/create-user-auth-workflow
fix(templates): remove orphan `</div>` in google_drive.html breaking HTML accessibility lint
2026-03-10 00:00:39 +01:00
copilot-swe-agent[bot] 256d9cb674 fix(templates): remove orphan </div> in google_drive.html causing HTML accessibility lint failure
The extra </div> at line 260 (before the {% endif %} closing the admin
auth-method block) was introduced when the per-user OAuth wizard PR
restructured the admin section. Removing it balances the div tree so
that the outer container div is properly closed by line 427.

djlint frontend/templates/ --lint now reports 0 errors across 58 files.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 21:46:38 +00:00
github-actions[bot] 5d5784f069 chore(release): update build metadata files [skip ci] 2026-03-09 21:30:44 +00:00
semantic-release 91549c8f5e 0.113.0
Automatically generated by python-semantic-release
2026-03-09 21:30:41 +00:00
Christian Krakau-Louis 4555cdb7df Merge pull request #576 from christianlouis/copilot/add-scheduled-batch-processing
feat: scheduled batch processing — 8 admin-managed jobs, DB-driven beat schedule, 100% coverage
2026-03-09 22:30:20 +01:00
copilot-swe-agent[bot] 0529ae53ff fix: 100% coverage, WCAG accessibility fixes, and documentation updates for per-user OAuth wizards
Test coverage:
- Add config=None branch test for dropbox, onedrive, google_drive views (100% coverage)
- Add WATCH_FOLDER source-type test (folder_path vs folder key)
- Add integration-not-found fallback-to-admin-mode test

WCAG 2.1 AA fixes:
- Add aria-labelledby="modalTitle" to role="dialog" modals in setup templates
- Add aria-hidden="true" to decorative SVGs in callback templates
- Add role="status" aria-label="Loading" to spinner divs
- Add aria-live="polite" to processing-message and success/folder-selection regions
- Add role="alert" aria-live="assertive" to error containers
- Update "Return to Setup" link to preserve integration_id in user mode

Docs: update DropboxSetup.md, GoogleDriveSetup.md, OneDriveSetup.md with per-user OAuth flow section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 20:19:19 +00:00
copilot-swe-agent[bot] ec77c51cb5 fix: code standards audit — 100% coverage, Pydantic v2 model_config, asyncio.run(), targeted edge-case tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 19:00:13 +00:00
copilot-swe-agent[bot] 2d01e71afd test(views): add user-mode coverage tests and fix dashboard helper duplication
- Add tests for user-mode with invalid JSON config (covers exception handling path)
- Add tests for user-mode with valid config (verifies folder path pre-population)
- Extract _watchFolderOAuthSource() helper to reduce isOAuthType/oauthLink duplication

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:32:04 +00:00
copilot-swe-agent[bot] 5bc1fac79c feat(tasks): add 5 additional scheduled jobs (expire_shared_links, prune_processing_logs, prune_old_notifications, backfill_missing_metadata, sync_search_index)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:25:26 +00:00
copilot-swe-agent[bot] c71f33a214 feat(integrations): add per-user OAuth wizard with user-mode for Dropbox, OneDrive, Google Drive
- Add user_mode to dropbox/onedrive/google_drive setup views that loads integration config
- Show user-friendly auth wizard when integration_id is provided (user mode)
- In user mode: show integration name, current folder, back-to-integrations link
- In callback templates: only save credentials (not config) for user integrations
- In integrations dashboard: show Authorize/Re-Authorize button for all OAuth types
- Add WATCH_FOLDER OAuth support: detect source_type in config for auth button
- isOAuthType() and oauthLink() now accept full integration object

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:24:07 +00:00
copilot-swe-agent[bot] c2d56d7475 feat(tasks): add scheduled batch processing infrastructure (model, migration, tasks, API, view, template)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 18:01:34 +00:00
copilot-swe-agent[bot] 80b1b01beb Initial plan 2026-03-09 17:58:17 +00:00
copilot-swe-agent[bot] 07d90758b8 Initial plan 2026-03-09 17:52:43 +00:00
github-actions[bot] 04d51bab22 chore(release): update build metadata files [skip ci] 2026-03-09 12:01:14 +00:00
semantic-release 8fa3e9bc4a 0.112.0
Automatically generated by python-semantic-release
2026-03-09 12:01:11 +00:00
Christian Krakau-Louis 2132e7e8d1 Merge pull request #575 from christianlouis/copilot/fix-imap-integration-ui-issues
feat(integrations): IMAP Gmail labels/delete controls, Watch Folder cloud sources, and comprehensive test coverage
2026-03-09 13:00:50 +01:00
copilot-swe-agent[bot] 9a342af609 fix(imap): normalize CRLF line endings to LF in imap_tasks.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 09:42:20 +00:00
copilot-swe-agent[bot] 1741ab1011 test(tasks): add comprehensive tests for per-user cloud scan functions and gmail_apply_labels
Add 68 new unit tests covering:
- _scan_user_s3_folder: 10 tests (bucket config, credentials, client creation,
  pagination, download, cache, file types, delete, error handling)
- _scan_user_dropbox_folder: 9 tests (credentials, folder path, auth, listing,
  download, cache, non-file entries, delete, download failure)
- _scan_user_google_drive_folder: 8 tests (credentials JSON, folder ID, auth,
  download, cache, delete, download/listing failures)
- _scan_user_onedrive_folder: 10 tests (credentials, folder path, token exchange,
  download, folders, cache, download URL, delete, download/listing failures)
- _scan_user_nextcloud_folder: 9 tests (settings, PROPFIND, XML parse, download,
  self-entry, cache, delete, download failure, absolute href)
- _scan_user_webdav_folder: 9 tests (URL config, PROPFIND, XML parse, download,
  directories, cache, delete, download failure, absolute href)
- Dispatch tests for google_drive, onedrive, nextcloud, webdav source types
- gmail_apply_labels forwarding and default-to-true in IMAP integration

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-09 09:29:43 +00:00
copilot-swe-agent[bot] 4acad71c8f fix: address code review feedback - fix type hint and rename test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:38:50 +00:00
copilot-swe-agent[bot] 8c62a6b714 test(integrations): add tests for gmail_apply_labels and cloud watch folder dispatch
docs(integrations): update UserGuide and ConfigurationGuide for new IMAP and Watch Folder features

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:35:30 +00:00
copilot-swe-agent[bot] 79fc11b452 test: add tests for gmail_apply_labels=False and cloud source type dispatching
Add test_gmail_labels_disabled_when_gmail_apply_labels_false to verify
that Gmail star/label operations are skipped when gmail_apply_labels=False.

Add four tests to TestPullUserIntegrationWatchFolders for cloud source
type dispatching: S3, Dropbox, unknown provider, and explicit local type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-09 02:26:10 +00:00
copilot-swe-agent[bot] e7d1289f1b feat(integrations): add IMAP Gmail labels/delete and Watch Folder cloud source support
- Add delete_after_process and gmail_apply_labels checkboxes to IMAP integration UI
- Add source_type selector (Local/S3/Dropbox/Google Drive/OneDrive/Nextcloud/WebDAV) to Watch Folder UI
- Add per-source-type config and credential fields for watch folders
- Add delete_after_process checkbox to Watch Folder settings
- Fix field name mismatch (path → folder_path) in Watch Folder UI
- Update pull_inbox() to accept and respect gmail_apply_labels parameter
- Update _pull_user_integration_imap() to read gmail_apply_labels from config
- Add per-user cloud scanning functions for S3, Dropbox, Google Drive, OneDrive, Nextcloud, WebDAV
- Update _pull_user_integration_watch_folders() to dispatch based on source_type
- Update IMAP and WATCH_FOLDER config shape documentation in models.py

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 02:15:03 +00:00
copilot-swe-agent[bot] 61e402f785 Initial plan 2026-03-09 01:52:58 +00:00
github-actions[bot] 5f548e29fa chore(release): update build metadata files [skip ci] 2026-03-09 01:47:12 +00:00
semantic-release aea9e227db 0.111.1
Automatically generated by python-semantic-release
2026-03-09 01:47:09 +00:00
Christian Krakau-Louis b71b18d184 Merge pull request #574 from christianlouis/copilot/fix-cors-header-test-error
fix(migrations): resolve duplicate Alembic heads causing test INTERNALERROR
2026-03-09 02:46:51 +01:00
copilot-swe-agent[bot] c374ba3ae9 fix(migrations): resolve duplicate Alembic heads causing test failures
Two migration files (025_add_shared_links and 025_add_user_notifications)
both had down_revision pointing to 024_add_api_tokens, creating multiple
migration heads. This caused alembic stamp/upgrade head to fail with
'Multiple heads are present; please specify a single target revision'
whenever init_db() ran (e.g. in TestClient fixtures).

Fix: chain 025_add_user_notifications from 025_add_shared_links so the
migration history is linear:
024_add_api_tokens → 025_add_shared_links → 025_add_user_notifications

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-09 01:11:26 +00:00
copilot-swe-agent[bot] 32e73727ad Initial plan 2026-03-09 00:47:56 +00:00
github-actions[bot] e4157e3373 chore(release): update build metadata files [skip ci] 2026-03-09 00:06:00 +00:00
semantic-release 708962b6a7 0.111.0
Automatically generated by python-semantic-release
2026-03-09 00:05:57 +00:00
Christian Krakau-Louis 0b9bba7158 Merge pull request #562 from christianlouis/copilot/refactor-dynamic-routing-user-destinations
fix(tasks): register upload_to_user_integration in Celery and add handler coverage
2026-03-09 01:05:36 +01:00
github-actions[bot] 39150cb9ab chore(release): update build metadata files [skip ci] 2026-03-08 23:46:58 +00:00
semantic-release ea2e927054 0.110.1
Automatically generated by python-semantic-release
2026-03-08 23:46:54 +00:00
Christian Krakau-Louis e916001e08 Merge pull request #573 from christianlouis/copilot/fix-sentry-settings-startup
fix(sentry): initialize Sentry after DB settings load in lifespan
2026-03-09 00:46:36 +01:00
github-actions[bot] b22f9a6af6 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-08 22:40:15 +00:00
copilot-swe-agent[bot] 5fb11b978c fix(tasks): register upload_to_user_integration in celery_worker and add handler tests
- Add `from app.tasks.upload_to_user_integration import upload_to_user_integration`
  to app/celery_worker.py — fixes TestCeleryWorkerConfig test that enforces
  every tasks/ module is registered with Celery.

- Add tests/test_upload_handlers.py with 43 unit tests covering all 11
  per-type upload handler functions (_upload_dropbox, _upload_s3,
  _upload_google_drive, _upload_onedrive, _upload_webdav, _upload_nextcloud,
  _upload_ftp, _upload_sftp, _upload_paperless, _upload_email, _upload_rclone)
  plus 2 additional finalize_document_storage branch tests. All external
  libraries (dropbox, boto3, msal, paramiko, smtplib, subprocess, requests)
  are mocked so tests are hermetic and fast. Coverage on changed files:
  upload_to_user_integration.py 94.71%, finalize_document_storage.py 95.51%
  (both well above the 70% Codecov diff threshold).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 22:21:15 +00:00
copilot-swe-agent[bot] 5864a4ed06 fix(sentry): initialize Sentry after DB settings are loaded in lifespan
Move init_sentry() from module level into the FastAPI lifespan context
manager, immediately after load_settings_from_db() completes. This
ensures that SENTRY_DSN and other Sentry settings configured via the
database admin UI are picked up on every restart.

Also update tests and docs accordingly.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 22:17:18 +00:00
github-actions[bot] a171186f93 chore(release): update build metadata files [skip ci] 2026-03-08 22:13:18 +00:00