Commit Graph

565 Commits

Author SHA1 Message Date
google-labs-jules[bot] 10df0b21bf 🧪 Extract and complete tests for saved searches API
Extracted existing `TestSavedSearchesCRUD` from `tests/test_api_advanced_filters.py` into a dedicated `tests/test_api_saved_searches.py` file to better organize testing logic and reflect the application's file structure.

Significantly improved code coverage of `app/api/saved_searches.py` from 0% (missing configuration imports during tests) to 100% by testing previously untested edge cases including:
- Reaching the maximum saved search limit per user.
- Database commit errors (`HTTP_500_INTERNAL_SERVER_ERROR`) during create, update, and delete actions.
- Validation failures for `filters` field checking for non-dict types (`status.HTTP_422_UNPROCESSABLE_ENTITY`).
- Conflicting names during updates where an existing saved search matches the new name.
- Proper fallback logic across authentication methods for `_get_user_id`.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-23 16:31:34 +00:00
copilot-swe-agent[bot] f58b8943fb fix(auth): case-insensitive login + mobile auth debug logging
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 21:15:19 +00:00
copilot-swe-agent[bot] fe48e2aedc test: fix webdav localhost assertion to match actual error message
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 20:49:52 +00:00
google-labs-jules[bot] 8079db7893 🛡️ Sentinel: [HIGH] Fix SSRF in WebDAV connection test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 04:10:11 +00:00
Christian Krakau-Louis 996deb76ff Merge pull request #638 from christianlouis/security-fix-sql-injection-index-mgmt-16594348298290829845
🔒 [security fix] Fix SQL injection in index management queries
2026-03-14 12:45:01 +01:00
Christian Krakau-Louis 51cf0995f7 Merge pull request #637 from christianlouis/test-improvement-api-tokens-revoke-error-paths-8200211521533028876
🧪 [testing improvement] Add error path tests for API token revocation
2026-03-14 12:44:44 +01:00
Christian Krakau-Louis 777910b653 Merge pull request #640 from christianlouis/refactor-filename-regex-constant-13933144971632372772
Refactor filename regex to shared constant
2026-03-14 12:44:11 +01:00
copilot-swe-agent[bot] 1416d901d0 fix(i18n): persist language preference server-side, fix dropdown rendering and avatar 404
- Fix language dropdown in base.html by moving tojson data out of HTML
  attribute into a script tag (prevents quote conflicts breaking Alpine.js)
- Fix avatar placeholder 404 by correcting filename reference from
  avatar-placeholder.svg to default-avatar.svg
- Add session hydration from DB in _inject_global_context() so
  detect_language() uses the stored preference on every request
- Sync session and cookie in PATCH /api/profile when language changes
- Reload page after language change in profile to reflect new locale
- Add tests for session/cookie sync and DB hydration

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 10:30:50 +00:00
Christian Krakau-Louis 91855b0763 Merge pull request #643 from christianlouis/copilot/sub-pr-637
test: exercise real auth path in test_revoke_token_unauthenticated
2026-03-14 11:03:55 +01:00
copilot-swe-agent[bot] 9e9b1fb158 test: strengthen test_revoke_token_database_error to verify rollback is called
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:58:46 +00:00
copilot-swe-agent[bot] 751b16d804 test: clarify caller cleanup responsibility in _make_unauthenticated_client docstring
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:55:33 +00:00
copilot-swe-agent[bot] 33eded02b1 test(database): add unit test for quoted identifier in filehash index drop
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:54:28 +00:00
copilot-swe-agent[bot] 96ef7d4769 test: exercise real auth path in test_revoke_token_unauthenticated
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:54:20 +00:00
google-labs-jules[bot] 917b282614 Refactor: extract filename regex to shared constant
Move the valid filename regex pattern to a shared constant in app/utils/filename_utils.py and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase. Also normalized line endings in app/tasks/extract_metadata_with_gpt.py.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:42:40 +00:00
google-labs-jules[bot] fdf5053ace test: add error path tests for API token revocation
- Enhanced existing tests for 400 (already revoked) and 404 (not found)
- Added test_revoke_token_unauthenticated (401)
- Added test_revoke_token_database_error (500 + rollback check)
- Added test_revoke_token_invalid_id_format (422)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:41:38 +00:00
copilot-swe-agent[bot] 08fabe154a fix(test): remove tests that check specific translation content
Remove three tests that asserted exact translated strings for German,
French, and Chinese locales. These tests broke whenever translation
files were updated externally.

Replace content-checking tests with behavioral assertions:
- Translated values are non-empty strings (not the raw key)
- Fallback and None-locale return the English translation
- Placeholder interpolation injects the kwarg value

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 08:47:56 +00:00
Christian Krakau-Louis 64d3f27afe Merge pull request #633 from christianlouis/copilot/enhance-language-selector-menu
feat(ui): smart compact language selector with flags and search
2026-03-14 09:34:42 +01:00
Christian Krakau-Louis 74eaf6fcef Merge pull request #631 from christianlouis/copilot/remove-translation-test
test(i18n): stop asserting specific translation content in kwargs interpolation test
2026-03-14 09:33:33 +01:00
copilot-swe-agent[bot] 3557590679 feat(ui): smart compact language selector with search and flags
- Add get_suggested_languages() to i18n.py returning ≤6 ranked
  suggestions (current locale → Accept-Language header → popular
  language fallbacks); refactor _parse_accept_language to share
  a common _parse_accept_language_entries() helper
- Inject suggested_languages into every template context (base.py)
- Redesign nav-bar language dropdown (base.html): button shows
  current-language flag emoji; dropdown lists 5-7 suggestions with
  flags and native names; Alpine.js search input filters all 77
  languages live; footer shows count and Search shortcut
- Add language.search_placeholder and language.no_results keys to
  all 77 translation JSON files (en values; external script
  propagates translations to other locales)
- Remove test_all_languages_have_same_keys (external sync script
  owns key completeness); add TestGetSuggestedLanguages (7 unit
  tests); update test_language_selector_in_nav for new HTML
- Update InternationalizationGuide.md: single-step en.json-only
  workflow for adding new translation keys
- Update .github/copilot-instructions.md: add i18n/l10n section
  documenting the en.json-only rule for future agents

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 21:31:16 +00:00
copilot-swe-agent[bot] 59d851272d feat(i18n): update language support to 77 languages
- Remove Russian (ru) per requirement
- Keep Esperanto (eo) per user feedback
- Add 29 new languages: am, az, bn, gu, ha, hi, hy, id, ig, jv, ka,
  kk, km, ml, mn, mr, ms, ne, si, sw, ta, te, th, tl, ur, uz, yo,
  zh-TW, zu
- Create 29 new translation JSON files seeded from en.json
- Remove frontend/translations/ru.json
- Update SUPPORTED_LANGUAGES list and docstring (49 → 77)
- Update tests: count 49→77, code set updated
- Update docs/InternationalizationGuide.md: count and table

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 18:36:07 +00:00
copilot-swe-agent[bot] 9b32cbc91b test(i18n): remove translation content assertion from test_translate_with_kwargs_german
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 16:47:48 +00:00
copilot-swe-agent[bot] 87d85624be feat(i18n): expand supported languages from 31 to 49 with flags and locale formats
Add 18 previously missing language entries to SUPPORTED_LANGUAGES in
app/utils/i18n.py so every translation JSON file in frontend/translations/
is properly indexed and served by the language selector:

New languages: af, ar, cy, eo, fa, fy, gl, he, ja, kn, ko, li, nds, no, pa, sr, vi, vls

Each new language entry includes:
- Correct ISO 639-1/639-3 code matching its JSON filename
- Native name and display name
- Appropriate country/language flag emoji
- Locale-specific date and number formatting rules in _LOCALE_FORMATS

Also: rename nb "Norwegian" → "Norwegian Bokmål" to distinguish it from no "Norwegian".

Update tests/test_i18n.py:
- Count assertions: 31 → 49
- Expected code set expanded to all 49 codes
- Fix test_unsupported_language_fallback (ja/ko now supported, use xx/yy)

Update docs/InternationalizationGuide.md:
- Language count: 10 → 49
- Full language table with flags, native names, tiers
- Complete file structure listing all 49 JSON files

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 14:03:31 +00:00
copilot-swe-agent[bot] f70fa65969 Merge branch 'main' into copilot/add-conditional-routing
Resolve conflicts in app/api/__init__.py and app/models.py.
Renumber migration 027_add_routing_rules → 035_add_routing_rules.
Fix migration chain: down_revision → 034_add_user_profile_settings.
Add PipelineRoutingRule to migrations/env.py.
2026-03-12 22:07:54 +00:00
Christian Krakau-Louis 4345d5128f Merge pull request #607 from christianlouis/copilot/add-user-settings-management
fix(tests): update whoami_handler call sites to pass required db argument
2026-03-12 22:44:13 +01:00
copilot-swe-agent[bot] 270e1a56b5 fix(tests): pass mock db to whoami_handler in test_api_auth_enabled.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 17:03:30 +00:00
copilot-swe-agent[bot] 9583d6d96f fix(profile): address code review feedback - early size check, CSRF helper, test constants
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 13:10:46 +00:00
copilot-swe-agent[bot] b0d6f1ab60 feat(profile): add user self-service profile settings page and API
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 13:00:19 +00:00
copilot-swe-agent[bot] b483ed16fc fix(i18n): register _ in all Jinja2 template instances, sync translation files, fix pipelines.html
- Register _() translation function in app/auth.py, app/api/local_auth.py, and
  app/api/billing.py template env globals to fix jinja2.exceptions.UndefinedError
- Update test_settings_template_has_db_wizard_link to assert i18n key instead of
  literal "DB Wizard" (template now uses {{ _("settings.db_wizard_btn") }})
- Add 418 missing en.json keys to all 30 non-English translation files as English
  fallbacks to fix test_all_languages_have_same_keys
- Fix orphan </template> tag in pipelines.html by adding missing
  <template x-if="pipelineModal.saving"> opening tag (fixes djlint H025)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 12:46:11 +00:00
copilot-swe-agent[bot] be6b49c872 fix(tests): fix two failing tests - missing DB table and MagicMock IP address
- tests/test_database.py: add user_imap_accounts table to the regression
  test's initial DB setup (migration 022 creates it before rev 026, so it
  must exist for migration 032's ALTER TABLE to succeed)
- tests/test_local_auth.py: configure mock_request.headers.get to return
  None and client=None so get_client_ip() returns "unknown" instead of an
  un-serialisable MagicMock that broke the audit_logs INSERT in
  test_local_login_success and test_local_login_by_email

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 10:49:49 +00:00
copilot-swe-agent[bot] 8060a79b9c style: fix ruff formatting in tests/test_imap_profiles.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 10:25:32 +00:00
Christian Krakau-Louis 7f644a2388 Merge pull request #604 from christianlouis/copilot/configure-attachment-ingestion
feat(imap): fine-grained attachment ingestion profiles with per-category selection
2026-03-12 08:53:45 +01:00
Christian Krakau-Louis 87fca21c7f Merge pull request #605 from christianlouis/copilot/enable-disable-storage-locations
feat(storage): explicit enable/disable flag per global storage destination
2026-03-12 08:51:14 +01:00
copilot-swe-agent[bot] 2e087731b9 fix(imap): address code review feedback on ingestion profiles
- Improve warning log in _resolve_categories_for_profile() to include
  exception type name for better troubleshooting
- Add SQLAlchemy IS NULL comment to imap_profiles.py filter
- Pass default_categories from server to template to avoid hardcoded
  category list in JS (now uses {{ default_categories | tojson }})
- Simplify view profiles query (remove redundant unauthenticated path)
- Update docs: ConfigurationGuide.md and EmailIngestion.md with
  full profiles documentation including category table and API reference

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 01:19:32 +00:00
copilot-swe-agent[bot] c9f554465d feat(imap): add ImapIngestionProfile model, API, migration and UI
Replaces the simple binary attachment_filter string with a full
ingestion profiles system:

- Add FILE_TYPE_CATEGORIES dict to allowed_types.py (6 categories:
  pdf, office, opendocument, text, web, images) + DEFAULT_CATEGORIES
  / ALL_CATEGORIES + get_allowed_types_for_categories() helper
- Add ImapIngestionProfile model (id, name, description, owner_id,
  allowed_categories JSON, is_builtin)
- Update UserImapAccount: replace attachment_filter string with
  profile_id FK to imap_ingestion_profiles
- Migration 033: creates profiles table, seeds 2 built-in profiles
  (Documents Only, All Files), migrates attachment_filter → profile_id
- New /api/imap-profiles/ CRUD endpoints (list, create, get, update,
  delete) with category validation
- Register imap_profiles router in app/api/__init__.py
- Update imap_tasks.py: replace attachment_filter string param with
  profile-based allowed_categories; add _resolve_categories_for_profile()
- Update imap_accounts.py API to use profile_id instead of attachment_filter
- Update imap_accounts view to pass profiles + categories to template
- Full UI overhaul: profiles panel + profile create/edit modal with
  category checkboxes; profile selector in account modal
- 17 new tests (141 total), all passing

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 01:14:19 +00:00
copilot-swe-agent[bot] b72ab3b318 feat(storage): add explicit enable/disable flag for each global storage destination
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 01:04:28 +00:00
copilot-swe-agent[bot] 554bb21d32 feat(imap): add attachment type filter for IMAP ingestion
Add a configurable switch to control which attachment types are ingested
via IMAP. Images are excluded by default; office files and PDFs are ingested.

- Add global `IMAP_ATTACHMENT_FILTER` config setting (default: `documents_only`)
- Add `attachment_filter` column to `UserImapAccount` model for per-user override
- Migration 032 adds the column to `user_imap_accounts` table
- Update `fetch_attachments_and_enqueue()` to respect filter (documents_only/all)
- Update `pull_inbox()`, `_pull_user_imap_accounts()`, and
  `_pull_user_integration_imap()` to pass the resolved filter
- Update IMAP accounts API (schemas, create/update handlers, response serializer)
- Update IMAP accounts UI to show attachment filter dropdown in modal and
  display filter badges on account cards
- Add 6 new tests covering attachment filter behaviour
- Update ConfigurationGuide.md, EmailIngestion.md, and .env.demo

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 00:59:54 +00:00
Christian Krakau-Louis 14b892b6d4 Merge pull request #601 from christianlouis/copilot/improve-code-test-coverage
test: improve coverage for app/utils/user_notification.py from ~50% to 100%
2026-03-12 01:55:49 +01:00
copilot-swe-agent[bot] 78204c2490 test(notifications): improve coverage for user_notification.py to 100%
Add 22 new unit tests in tests/test_user_notification_service.py covering
all previously-uncovered branches in app/utils/user_notification.py:

- create_in_app_notification: exception/rollback path
- _send_email_notification: SMTP success (TLS+creds, no-TLS, no-creds),
  SMTP exception, sender_email fallbacks
- _send_webhook_notification: success with/without secret, HTTP errors
- dispatch_user_notification: email/webhook pref dispatch, no target_id
  skip, inactive target skip, invalid JSON config, null config, outer
  exception handling, push notification sent/exception, unknown channel_type

Combined coverage: 49.52% → 100%

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 00:44:59 +00:00
Christian Krakau-Louis 0aa1df3920 Merge pull request #588 from christianlouis/copilot/add-social-logins-configuration
feat(auth): add social login support for Google, Microsoft, Apple, and Dropbox
2026-03-12 01:40:04 +01:00
copilot-swe-agent[bot] 7438551080 test: improve coverage for app/utils/audit_service.py from 59% to 100%
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 00:28:42 +00:00
Christian Krakau-Louis 80dd117ebf Merge pull request #587 from christianlouis/copilot/add-apple-drive-icloud-support
feat(storage): add Apple iCloud Drive storage provider
2026-03-12 01:14:37 +01:00
copilot-swe-agent[bot] 054af636e3 fix(migrations): merge main and rechain compliance migration as 031 after 030_add_mobile_devices
Merge main branch into compliance templates feature branch.

Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.

Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
2026-03-12 00:00:29 +00:00
copilot-swe-agent[bot] 50af0ea679 fix(tests): add iCloud mocks to all test files and address code review feedback
- Add _should_upload_to_icloud mock to test_coverage_uploads_notification.py
  (_all_should_upload_false helper + 3 inline patch blocks)
- Add cfg.icloud_username/password = None to all onboarding test mocks
- Add iCloud creds to fully-configured onboarding test
- Fix noqa comment accuracy (unofficial third-party, not first-party)
- Replace generic Exception with RuntimeError in upload error handler

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 23:59:19 +00:00
Christian Krakau-Louis 39db5fc564 Merge pull request #600 from christianlouis/copilot/add-graphql-api
feat: add GraphQL API alongside REST at /graphql
2026-03-12 00:43:10 +01:00
copilot-swe-agent[bot] 528f0a624d fix: remove accidental pip artifact file and update docs for iCloud Drive
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 23:38:35 +00:00
copilot-swe-agent[bot] 30f06e0b32 feat(storage): add Apple iCloud Drive storage provider
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 23:38:35 +00:00
copilot-swe-agent[bot] 9bc23aa40b test(tasks): add _should_upload_to_icloud mock to send_to_all tests
Add icloud upload check mock alongside existing _should_upload_to_*
function mocks in all TestSendToAllDestinations test methods.

Changes:
- Import _should_upload_to_icloud from app.tasks.send_to_all
- Add @patch decorator for _should_upload_to_icloud in 9 test methods
- Add mock_icloud parameter to each test method signature
- Set mock_icloud.return_value = False where other mocks are set to False

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-11 23:38:34 +00:00
Christian Krakau-Louis 1c4a261f01 Merge pull request #599 from christianlouis/copilot/fix-main-test-failures
fix: resolve 47 failing CI tests across i18n, templates, and migrations
2026-03-12 00:34:18 +01:00
copilot-swe-agent[bot] df4c91a586 fix: resolve all 47 failing tests in main
- Fix detect_language() to safely handle unhashable session values and
  requests missing cookies/headers attributes (TypeError + AttributeError)
- Add default English `_()` translation function to Jinja2 template
  environment globals so error pages always have it available
- Fix app/main.py exception handlers to use a dedicated error templates
  instance with `_` registered, keeping it separate from view templates
  to avoid test patches breaking error rendering
- Fix app/views/plans.py to import shared templates from app.views.base
  instead of creating its own Jinja2Templates instance
- Make migration 029_add_user_language_preference idempotent: skip
  ALTER TABLE if user_profiles table does not exist
- Update test_i18n.py expectations to reflect 31 supported languages
- Create 21 missing translation files (nb, da, sv, fi, is, ga, lb, ca,
  cs, sk, hu, sl, hr, ro, bg, el, et, lv, lt, tr, uk) with English
  placeholder translations
- Update de.json with 117 missing translation keys including proper
  German translations
- Update es, fr, it, nl, pl, pt, ru, zh translation files with missing
  keys using English fallbacks

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 22:28:01 +00:00
copilot-swe-agent[bot] a41ded535f feat(api): add GraphQL endpoint at /graphql with Strawberry
- Add strawberry-graphql[fastapi] dependency
- Implement GraphQL schema covering documents, pipelines, settings, users
- Enable GraphiQL playground at /graphql
- Mount GraphQL router in main.py
- Add 24 tests for all query types and auth enforcement
- Update docs/API.md with GraphQL documentation section

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-11 22:17:25 +00:00