Commit Graph

480 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 58af9e5a29 feat(observability): add Sentry error and performance monitoring integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 21:09:17 +00:00
Christian Krakau-Louis 8faf164e9e Merge pull request #560 from christianlouis/copilot/add-webhook-snippet-and-api-tokens
fix(security): resolve CodeQL clear-text logging and weak hashing alerts
2026-03-08 21:42:28 +01:00
github-actions[bot] fc9caf4f9b style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-08 20:39:10 +00:00
copilot-swe-agent[bot] a7eff2c033 fix(security): address CodeQL clear-text logging and weak hashing alerts
- Convert f-string log interpolation to %s-style formatting in
  app/api/pipelines.py and app/api/saved_searches.py to prevent
  clear-text logging of request-derived data (CodeQL: clear-text
  logging of sensitive information)
- Replace plain hashlib.sha256() with PBKDF2-HMAC-SHA256 via
  hash_token() in app/auth.py for Bearer token verification,
  consistent with how tokens are stored in api_tokens.py (CodeQL:
  use of weak cryptographic hashing on sensitive data)
- Remove redundant {exc} from logger.exception() calls (the
  traceback is already captured by logger.exception())
- Update test to verify PBKDF2 hash instead of plain SHA-256

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 19:33:55 +00:00
Christian Krakau-Louis af696daf55 Merge pull request #558 from christianlouis/copilot/fix-user-specific-tokens
feat(auth): make OAuth credentials user-specific via UserIntegration records
2026-03-08 20:23:18 +01:00
Christian Krakau-Louis 1ce9e97ad0 Merge pull request #559 from christianlouis/copilot/refactor-multi-tenant-polling-engine
feat(tasks): Multi-tenant polling engine for IMAP and watch folder ingestion
2026-03-08 20:23:04 +01:00
Christian Krakau-Louis 52ebbad335 Potential fix for code scanning alert no. 344: Use of a broken or weak cryptographic hashing algorithm on sensitive data
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-03-08 20:22:39 +01:00
copilot-swe-agent[bot] cfc52fd355 feat(auth): make OAuth credentials user-specific via UserIntegration records
Setup pages now accept an integration_id query param to save OAuth
credentials to the user's personal UserIntegration record instead
of global settings. The integrations dashboard shows an "Authorize"
button for OAuth types (Dropbox, Google Drive, OneDrive) that need
credentials.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:48:11 +00:00
copilot-swe-agent[bot] 4f9f458151 test(tasks): add comprehensive tests for multi-tenant polling functions
- Add tests for owner_id passthrough in fetch_attachments_and_enqueue
- Add tests for _pull_user_integration_imap (success, failure, DB error, incomplete config)
- Add tests for _is_safe_watch_path path traversal security
- Add tests for _enqueue_file owner_id forwarding
- Add tests for _scan_user_watch_folder (scan, skip processed, nonexistent dir)
- Add tests for _pull_user_integration_watch_folders (success, unsafe path, failure, DB error)
- Add tests verifying scan_all_watch_folders calls user integration polling
- Add tests verifying pull_all_inboxes calls both legacy and integration polling

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:43:18 +00:00
copilot-swe-agent[bot] c3bb93c197 feat(api): add personal API tokens and enhance webhook integration UI
- Add ApiToken model with SHA-256 hashed storage and usage tracking
- Create API token CRUD endpoints (POST/GET/DELETE /api/api-tokens/)
- Add Bearer token authentication to require_login decorator
- Exempt Bearer-authenticated requests from CSRF validation
- Add API tokens management page with create/revoke/copy UI
- Enhance webhook integration type with detailed explanation and code snippets
- Add navigation links (desktop + mobile) to API tokens page
- Include 19 tests covering CRUD, auth resolution, and utility functions
- Create migration 024_add_api_tokens

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:42:55 +00:00
copilot-swe-agent[bot] 019807d0f5 feat(tasks): refactor IMAP and watch folder polling to support multi-tenant user attribution
- Add owner_id parameter to pull_inbox() and fetch_attachments_and_enqueue()
  to attribute ingested documents to the correct user
- Add _pull_user_integration_imap() to poll IMAP sources from UserIntegration model
- Add _pull_user_integration_watch_folders() to scan watch folders from UserIntegration model
- Add _is_safe_watch_path() for path traversal security on user-configured paths
- Add _scan_user_watch_folder() that passes owner_id to _enqueue_file()
- Update _enqueue_file() to forward owner_id to process_document/convert_to_pdf
- Update celery beat schedule to always enable IMAP and watch folder polling
  (user integrations can exist without system-level config)
- Ensure individual connection failures don't crash the polling loop
- Update existing tests for new function signatures

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:38:22 +00:00
copilot-swe-agent[bot] 8098856186 test(support): improve Zammad user context tests with meaningful assertions
Enable Zammad settings via mock and verify user data appears in the
rendered HTML. Add tests for chat widget, email-only fallback, and
display_name fallback. Also improve JS variable naming in help.html.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:33:39 +00:00
copilot-swe-agent[bot] 209e82f7e5 feat(support): auto-fill Zammad chat and ticket widgets with user context
Pass authenticated user's name, email, and username to Zammad form
and chat widgets. For the ticket form: pre-fill name/email fields
and append a User Context metadata block to the ticket body via
$.ajaxPrefilter. For the chat widget: pass name/email to the
ZammadChat constructor.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:27:14 +00:00
copilot-swe-agent[bot] ba387b8e8b fix(ui): correct OAuth setup page links in integrations dashboard
The oauthLink() function in integrations_dashboard.html returned
/dropbox, /google-drive, /onedrive which are not valid routes
(404). The actual view routes are /dropbox-setup,
/google-drive-setup, /onedrive-setup.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 18:21:52 +00:00
Christian Krakau-Louis f714a9ed31 Merge pull request #556 from christianlouis/copilot/feature-universal-dashboard
feat(ui): Universal Sources & Destinations integrations dashboard
2026-03-08 19:08:11 +01:00
Christian Krakau-Louis 58733d6450 Merge pull request #555 from christianlouis/copilot/restructure-help-page
feat(help): replace MkDocs redirect with user-facing Help Center + Zammad integration
2026-03-08 18:58:02 +01:00
copilot-swe-agent[bot] 351d6f3f4f fix: update docs endpoint test and add Zammad settings metadata
- Update test_docs_endpoint to use new /admin/api-docs path
- Add SETTING_METADATA entries for zammad_url, zammad_chat_enabled,
  zammad_chat_id, zammad_form_enabled, support_email

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:37:29 +00:00
copilot-swe-agent[bot] 2c652dcc3e feat(ui): add unified Sources & Destinations integrations dashboard
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:34:22 +00:00
copilot-swe-agent[bot] af3eed4040 feat(help): restructure /help as user-facing Help Center with Zammad integration
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
  ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 17:25:34 +00:00
Christian Krakau-Louis a7a807bb11 Merge pull request #551 from christianlouis/copilot/verify-stripe-billing-implementation
fix(billing): cast SQLAlchemy Column[str] to str to resolve mypy typeddict-item error
2026-03-08 18:11:58 +01:00
copilot-swe-agent[bot] c4b4643f6e fix(integrations): sanitize error messages and add SSRF protection for WebDAV test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:57:27 +00:00
copilot-swe-agent[bot] 35abdb0e69 feat(integrations): add quota enforcement, connection test, and quota endpoint
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:45:41 +00:00
copilot-swe-agent[bot] 002cdf312b fix(billing): address code review - remove duplicate mode logic, accessibility improvements, robust test assertions
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:18:53 +00:00
copilot-swe-agent[bot] 223539bebb feat(billing): expose Stripe price IDs in plan API, add Stripe Setup Wizard and sync endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:14:42 +00:00
Christian Krakau-Louis 89378b2270 Merge pull request #553 from christianlouis/copilot/restrict-status-page-access
feat(status): restrict status page to admin-only, move to admin menu
2026-03-08 16:14:26 +01:00
copilot-swe-agent[bot] feea051c4f feat(status): restrict status page to admin-only and move to admin menu
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:10:46 +00:00
copilot-swe-agent[bot] 743627ecc2 feat(models): add UserIntegration model, encrypt IMAP passwords, add integrations API
- Add IntegrationDirection/IntegrationType constants and UserIntegration SQLAlchemy model
  with owner_id, direction, integration_type, name, config (JSON), credentials (encrypted),
  is_active, last_used_at, last_error, created_at, updated_at fields
- Fix UserImapAccount password encryption: encrypt_value() on create/update,
  decrypt_value() in IMAP tasks and test-connection endpoint (backward compatible)
- Add Alembic migration 023_add_user_integrations
- Add app/api/integrations.py: CRUD + credentials endpoint with owner-scoped access
- Register integrations router in app/api/__init__.py
- Add 39 tests in tests/test_api_integrations.py covering model, CRUD, encryption

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 15:02:38 +00:00
Christian Krakau-Louis 6175489ba0 Merge pull request #543 from christianlouis/copilot/add-per-user-imap-ingestion
feat(imap): per-user IMAP ingestion accounts with quota enforcement
2026-03-08 15:45:25 +01:00
copilot-swe-agent[bot] 087f3c10e9 fix(auth): cache request body in CSRF middleware to prevent login failures
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:48:17 +00:00
copilot-swe-agent[bot] 43bcfe5653 feat(imap): add per-user IMAP ingestion accounts with quota enforcement
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 13:01:14 +00:00
Christian Krakau-Louis 6497d35d84 Merge pull request #534 from christianlouis/copilot/increase-test-coverage-watch-folder-tasks
test: increase watch_folder_tasks.py coverage from 37% to 98%
2026-03-08 13:26:23 +01:00
Christian Krakau-Louis c26ecfce08 Merge pull request #535 from christianlouis/copilot/increase-test-coverage-backup-view
Increase test coverage for app/views/backup.py to 100%
2026-03-08 13:23:29 +01:00
Christian Krakau-Louis 06eb6e636e Merge pull request #533 from christianlouis/copilot/increase-test-coverage-backup-tasks
test: increase backup_tasks.py coverage from 66% to 99.79%
2026-03-08 13:20:50 +01:00
copilot-swe-agent[bot] 3ac5a97467 test: increase watch_folder_tasks coverage from 37% to 98%
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:19:29 +00:00
copilot-swe-agent[bot] 19b4ea4f6b test(backup): add comprehensive unit tests for app/views/backup.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:15:39 +00:00
Christian Krakau-Louis 640662e9b2 Merge pull request #540 from christianlouis/copilot/fix-local-user-login-issue
fix(auth): local user login fails after password reset
2026-03-08 13:15:14 +01:00
Christian Krakau-Louis 9e9a4f9b6a Merge pull request #536 from christianlouis/copilot/increase-test-coverage-subscription-tasks
test: 100% coverage for app/tasks/subscription_tasks.py
2026-03-08 13:12:31 +01:00
copilot-swe-agent[bot] 9a95d5cfd5 test: increase backup_tasks.py coverage to 99.79%
Add comprehensive tests covering all previously untested code paths in
app/tasks/backup_tasks.py, raising coverage from 66.39% to 99.79%.

New test classes:
- TestDumpPostgresqlBranches / TestDumpMysqlBranches: URL branches
- TestRestoreSqliteBranches: error paths and rollback scenarios
- TestRestorePostgresqlBranches / TestRestoreMysqlBranches: URL branches
- TestApplyRetentionOSError: OSError on file removal + remote record keeping
- TestPruneRemoteBackups: full pruning lifecycle
- TestDeleteRemoteCopy: S3, Dropbox, known/unknown destinations
- TestUploadRemote: all destination branches (S3, Dropbox, email, Nextcloud, WebDAV)
- TestEmailBackup: recipient error, TLS, no-auth paths
- TestCreateBackupAdditional: dump failure, remote upload, PG/MySQL failures
- TestCleanupOldBackupsTask: validates all tiers are processed

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:11:07 +00:00
copilot-swe-agent[bot] 86f9f5f9b1 fix(auth): activate account on password reset and fix is_active check order
- reset_password sets is_active=True so users with unverified accounts
  can log in after using the forgot-password flow
- admin set_password also sets is_active=True for the same reason
- auth() now checks is_active before verifying the password, ensuring
  inactive users always see the email-verification prompt regardless of
  password correctness (avoids leaking password validity)"

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:07:25 +00:00
Christian Krakau-Louis a1dcc1d2b5 Merge pull request #538 from christianlouis/copilot/increase-test-coverage-subscriptions-again
test: increase coverage for app/views/subscriptions.py to 100%
2026-03-08 13:05:26 +01:00
copilot-swe-agent[bot] a580527de4 test: increase coverage for app/tasks/subscription_tasks.py to 100%
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 12:05:17 +00:00
Christian Krakau-Louis b7df2ca74b Merge pull request #532 from christianlouis/copilot/increase-test-coverage-subscriptions
test(subscriptions): increase app/api/subscriptions.py coverage from 49% to 100%
2026-03-08 13:03:23 +01:00
Christian Krakau-Louis 13e8d5098c Merge pull request #537 from christianlouis/copilot/increase-test-coverage-url-upload
test: increase url_upload.py coverage from 91% to 100%
2026-03-08 12:59:53 +01:00
copilot-swe-agent[bot] e96ddc7865 test: increase coverage for app/views/subscriptions.py to 100%
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 11:57:37 +00:00
copilot-swe-agent[bot] cd19ac661e test(subscriptions): add comprehensive API tests reaching 100% coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 11:54:56 +00:00
copilot-swe-agent[bot] afe42270bb test: increase url_upload.py test coverage to 100%
Add 10 new tests in TestURLUploadCoverageGaps to cover previously
uncovered lines and branches in app/api/url_upload.py:

- Line 41: validate_url_scheme raises ValueError for non-http scheme
- Lines 65->61, 67: is_private_ip DNS path with public IP resolution
- Line 87: validate_url_safety with ftp:// scheme (direct call)
- Line 107: validate_url_safety blocks metadata.google.internal
- Line 130->135: validate_file_type with no file extension
- Line 177: sanitize_filename returning empty string defaults to 'download'
- Line 234->233: iter_content empty bytes chunks (if chunk: False branch)
- Line 285: OSError cleanup path removes existing partial file
- Line 291->293: unexpected exception before target_path assigned (stays None)

Coverage: 91.16% -> 100%

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 11:53:45 +00:00
Christian Krakau-Louis 660b18ee97 Merge pull request #529 from christianlouis/copilot/increase-test-coverage-pipelines
test: increase coverage for app/views/pipelines.py to 100%
2026-03-08 12:52:08 +01:00
Christian Krakau-Louis 1cb39ca027 Merge pull request #528 from christianlouis/copilot/increase-test-coverage-google-drive
test: increase app/api/google_drive.py coverage to 100%
2026-03-08 12:51:53 +01:00
Christian Krakau-Louis 8292704703 Merge pull request #531 from christianlouis/copilot/increase-test-coverage-local-auth
test(local_auth): increase coverage for app/utils/local_auth.py to 100%
2026-03-08 12:51:12 +01:00
copilot-swe-agent[bot] 14c9432640 test: increase app/api/google_drive.py coverage to 100%
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-08 11:44:30 +00:00