Commit Graph

565 Commits

Author SHA1 Message Date
Christian Krakau-Louis b3d1824d66 Merge pull request #700 from christianlouis/extract-pdf-metadata-upload-to-email-12238364102013658937
Extract embedded PDF metadata using pypdf
2026-03-16 10:52:51 +01:00
Christian Krakau-Louis 00be8d7b8e Merge pull request #703 from christianlouis/perf/fix-n-plus-1-shared-links-7028382104319809262
 fix N+1 query in list_shared_links
2026-03-16 10:52:34 +01:00
Christian Krakau-Louis 6e2e54aac6 Merge pull request #705 from christianlouis/perf/fix-n-plus-1-duplicates-17979853535459179799
 Fix N+1 Query in Duplicate Grouping List
2026-03-16 10:51:59 +01:00
Christian Krakau-Louis 056292dbe6 Merge pull request #710 from christianlouis/jules/perf-opt-reorder-plans-5035457001465312501
 Optimize reorder_plans by fixing N+1 database query
2026-03-16 10:51:02 +01:00
Christian Krakau-Louis c00a35bbac Merge pull request #711 from christianlouis/performance-optimize-dropbox-token-2748375428782384195
 Optimize Dropbox token check by using async httpx
2026-03-16 10:49:34 +01:00
Christian Krakau-Louis 5b3c7da644 Merge pull request #719 from christianlouis/jules-fix-database-index-quoting-4036216779366574643
🔒 fix: explicitly quote identifiers in CREATE INDEX for saved_searches
2026-03-16 10:48:47 +01:00
Christian Krakau-Louis 1d51266208 Merge pull request #693 from christianlouis/fix-rclone-command-injection-16912117938106238295
🔒 fix(tasks): prevent command injection in rclone commands
2026-03-16 10:46:40 +01:00
Christian Krakau-Louis 3088459c70 Merge pull request #713 from christianlouis/perf/fix-notifications-api-n-plus-1-17980566457111073264
 Fix N+1 Query in Notification Preferences Update
2026-03-16 10:46:12 +01:00
github-actions[bot] b4118f6162 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-16 09:37:40 +00:00
google-labs-jules[bot] 71d2d4100e 🔒 Prevent SQL injection by explicitly quoting identifier in CREATE INDEX
While `_ensure_indexes` was already secured, the `CREATE INDEX` for
`ix_saved_searches_user_id` was hardcoded. This commit explicitly
quotes it to unify our security posture against SQL injection
and keep static analyzers happy.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:37:10 +00:00
google-labs-jules[bot] fe20e02f78 perf(api): fix n+1 query issue in user notification preferences update
- Added a benchmark script in tests/test_notifications_api.py that proved the N+1 issue issue.
- Replaced iterative DB lookups inside `for item in body.preferences:` with single pre-fetch query and local `prefs_dict` lookups.
- Verified test benchmark time drops from ~0.0964s to ~0.0141s for a batch of 100 items.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:20:41 +00:00
google-labs-jules[bot] 84c6e1c5dd perf: optimize dropbox token refresh by replacing blocking requests with httpx
Replaced the synchronous `requests.post` calls in `app/api/dropbox.py` with asynchronous `httpx.AsyncClient().post` calls. This ensures that the FastAPI event loop is not blocked during network I/O, allowing better concurrent performance.

Also updated the `test_api_dropbox.py` tests to use `httpx.AsyncClient.post` in mocks and properly construct `httpx.RequestError` in exception handling tests.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:19:55 +00:00
google-labs-jules[bot] d8372c6fb8 perf(api): optimize reorder_plans to prevent N+1 queries
Replaced the loop over `body.order` which generated an N+1 issue
with a single bulk query fetching all relevant `SubscriptionPlan`
records via the `.in_()` clause.

Added an in-memory dictionary map of `plan_id` to `SubscriptionPlan`
objects to allow `O(1)` lookups while updating the order.

Benchmark speedup: 14.71x faster on 500 records.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:19:50 +00:00
google-labs-jules[bot] e4e3ac4077 perf(duplicates): fix N+1 query in group listing
Replaced the loop over duplicate hashes that resulted in O(N) database queries
per page with a single efficient `in_` batch query to retrieve both originals
and duplicates. The records are then grouped in memory using dictionaries.

This resolves the N+1 performance bottleneck and reduces response time from
an average of 1.65 seconds to ~0.45 seconds locally for 500 groups.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:16:31 +00:00
google-labs-jules[bot] 040f4dcdd4 fix N+1 query in list_shared_links
Replaced the N+1 query in `list_shared_links` which fetched `FileRecord` for each link. It now uses a single query with an `outerjoin` to fetch `original_filename` alongside the `SharedLink` object.
Measured a significant improvement from ~0.4547s to ~0.0579s per 1000 links.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:14:36 +00:00
github-actions[bot] 2cfbea29a9 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-16 09:12:15 +00:00
google-labs-jules[bot] 433d1eb639 🔒 Fix potential SQL injection in db_migrate preview
Added a strict regex validation allowlist for table names in `preview_migration` before using them in raw SQL queries. This ensures that only alphanumeric characters and underscores are allowed, preventing potential SQL injection even if the source of table names were to be manipulated.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:11:57 +00:00
google-labs-jules[bot] df64aece2c feat: Extract embedded PDF metadata using pypdf in upload_to_email
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 09:05:02 +00:00
google-labs-jules[bot] 5e911ed268 🔒 fix(tasks): prevent command injection in rclone commands
Added the `--` argument before positional arguments in rclone subprocess calls (link, mkdir, copy) in `app/tasks/upload_with_rclone.py`. This ensures that filenames or destinations starting with a hyphen are treated as paths rather than unintended command-line flags.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-16 08:57:57 +00:00
copilot-swe-agent[bot] f58b8943fb fix(auth): case-insensitive login + mobile auth debug logging
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 21:15:19 +00:00
copilot-swe-agent[bot] 14285567d5 fix(auth): Expo Go support via Linking.createURL; safe token URL construction; clean up return type annotation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 17:42:36 +00:00
copilot-swe-agent[bot] 5c373e7224 fix(auth): store mobile_redirect_uri in session and redirect to deep-link after SSO; fix SafeAreaView deprecation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 17:35:49 +00:00
github-actions[bot] b7a195d3bc style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-15 04:10:28 +00:00
google-labs-jules[bot] 8079db7893 🛡️ Sentinel: [HIGH] Fix SSRF in WebDAV connection test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-15 04:10:11 +00:00
Christian Krakau-Louis 996deb76ff Merge pull request #638 from christianlouis/security-fix-sql-injection-index-mgmt-16594348298290829845
🔒 [security fix] Fix SQL injection in index management queries
2026-03-14 12:45:01 +01:00
Christian Krakau-Louis d5fb0da335 Merge pull request #639 from christianlouis/refactor-audit-logs-annotated-b008-6104828608171783200
🧹 Use Annotated pattern in audit_logs.py to resolve Ruff B008
2026-03-14 12:44:26 +01:00
Christian Krakau-Louis 777910b653 Merge pull request #640 from christianlouis/refactor-filename-regex-constant-13933144971632372772
Refactor filename regex to shared constant
2026-03-14 12:44:11 +01:00
Christian Krakau-Louis ee95173de0 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-14 12:43:59 +01:00
copilot-swe-agent[bot] f41ecc9b2c fix(i18n): add noqa comment to intentionally broad exception in hydration helper
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 10:34:31 +00:00
copilot-swe-agent[bot] 1416d901d0 fix(i18n): persist language preference server-side, fix dropdown rendering and avatar 404
- Fix language dropdown in base.html by moving tojson data out of HTML
  attribute into a script tag (prevents quote conflicts breaking Alpine.js)
- Fix avatar placeholder 404 by correcting filename reference from
  avatar-placeholder.svg to default-avatar.svg
- Add session hydration from DB in _inject_global_context() so
  detect_language() uses the stored preference on every request
- Sync session and cookie in PATCH /api/profile when language changes
- Reload page after language change in profile to reflect new locale
- Add tests for session/cookie sync and DB hydration

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 10:30:50 +00:00
github-actions[bot] de0b674025 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-14 09:45:05 +00:00
Christian Krakau-Louis b148675ba8 Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-14 10:44:14 +01:00
google-labs-jules[bot] 917b282614 Refactor: extract filename regex to shared constant
Move the valid filename regex pattern to a shared constant in app/utils/filename_utils.py and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase. Also normalized line endings in app/tasks/extract_metadata_with_gpt.py.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:42:40 +00:00
google-labs-jules[bot] f96fcbe6c9 style: use Annotated pattern in audit_logs.py to resolve Ruff B008
Refactor `app/api/audit_logs.py` to use the `Annotated` type hint pattern for FastAPI dependencies (`Depends`) and query parameters (`Query`).

- Resolves B008: Function-call in default argument.
- Improves code maintainability and readability by following modern FastAPI best practices.
- Maintains consistency with other modules in the codebase (e.g., `files.py`, `integrations.py`).
- No changes to API runtime behavior.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:42:23 +00:00
github-actions[bot] 0b064b9d20 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-14 09:42:23 +00:00
google-labs-jules[bot] 120002b394 fix(database): quote identifiers in index management queries to prevent SQL injection
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:42:04 +00:00
copilot-swe-agent[bot] 0592c59174 fix(ui): add missing suggested_languages global to error templates
The error templates (404.html, 500.html) extend base.html which uses
{{ suggested_languages | tojson }} in the language selector dropdown.
The _error_templates instance in app/main.py was missing this global,
causing Jinja2 Undefined objects to be passed to the tojson filter,
resulting in "TypeError: Object of type Undefined is not JSON serializable"
errors in 35 tests.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 09:07:17 +00:00
Christian Krakau-Louis 64d3f27afe Merge pull request #633 from christianlouis/copilot/enhance-language-selector-menu
feat(ui): smart compact language selector with flags and search
2026-03-14 09:34:42 +01:00
google-labs-jules[bot] 82b1f89332 fix(security): prevent potential SQL injection in database migration
Replaced manual double-quoting of table names with SQLAlchemy's dialect-specific
identifier preparer in `app/utils/db_migrate.py`. This ensures proper quoting
for any database dialect and acts as a defense-in-depth measure against
SQL injection or syntax errors if a table name contains unexpected characters.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-14 03:39:53 +00:00
copilot-swe-agent[bot] 3557590679 feat(ui): smart compact language selector with search and flags
- Add get_suggested_languages() to i18n.py returning ≤6 ranked
  suggestions (current locale → Accept-Language header → popular
  language fallbacks); refactor _parse_accept_language to share
  a common _parse_accept_language_entries() helper
- Inject suggested_languages into every template context (base.py)
- Redesign nav-bar language dropdown (base.html): button shows
  current-language flag emoji; dropdown lists 5-7 suggestions with
  flags and native names; Alpine.js search input filters all 77
  languages live; footer shows count and Search shortcut
- Add language.search_placeholder and language.no_results keys to
  all 77 translation JSON files (en values; external script
  propagates translations to other locales)
- Remove test_all_languages_have_same_keys (external sync script
  owns key completeness); add TestGetSuggestedLanguages (7 unit
  tests); update test_language_selector_in_nav for new HTML
- Update InternationalizationGuide.md: single-step en.json-only
  workflow for adding new translation keys
- Update .github/copilot-instructions.md: add i18n/l10n section
  documenting the en.json-only rule for future agents

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 21:31:16 +00:00
copilot-swe-agent[bot] 59d851272d feat(i18n): update language support to 77 languages
- Remove Russian (ru) per requirement
- Keep Esperanto (eo) per user feedback
- Add 29 new languages: am, az, bn, gu, ha, hi, hy, id, ig, jv, ka,
  kk, km, ml, mn, mr, ms, ne, si, sw, ta, te, th, tl, ur, uz, yo,
  zh-TW, zu
- Create 29 new translation JSON files seeded from en.json
- Remove frontend/translations/ru.json
- Update SUPPORTED_LANGUAGES list and docstring (49 → 77)
- Update tests: count 49→77, code set updated
- Update docs/InternationalizationGuide.md: count and table

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 18:36:07 +00:00
copilot-swe-agent[bot] ce4e797bd1 docs(i18n): add comment on Welsh subdivision flag Unicode representation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 14:07:07 +00:00
copilot-swe-agent[bot] 87d85624be feat(i18n): expand supported languages from 31 to 49 with flags and locale formats
Add 18 previously missing language entries to SUPPORTED_LANGUAGES in
app/utils/i18n.py so every translation JSON file in frontend/translations/
is properly indexed and served by the language selector:

New languages: af, ar, cy, eo, fa, fy, gl, he, ja, kn, ko, li, nds, no, pa, sr, vi, vls

Each new language entry includes:
- Correct ISO 639-1/639-3 code matching its JSON filename
- Native name and display name
- Appropriate country/language flag emoji
- Locale-specific date and number formatting rules in _LOCALE_FORMATS

Also: rename nb "Norwegian" → "Norwegian Bokmål" to distinguish it from no "Norwegian".

Update tests/test_i18n.py:
- Count assertions: 31 → 49
- Expected code set expanded to all 49 codes
- Fix test_unsupported_language_fallback (ja/ko now supported, use xx/yy)

Update docs/InternationalizationGuide.md:
- Language count: 10 → 49
- Full language table with flags, native names, tiers
- Complete file structure listing all 49 JSON files

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-13 14:03:31 +00:00
copilot-swe-agent[bot] f70fa65969 Merge branch 'main' into copilot/add-conditional-routing
Resolve conflicts in app/api/__init__.py and app/models.py.
Renumber migration 027_add_routing_rules → 035_add_routing_rules.
Fix migration chain: down_revision → 034_add_user_profile_settings.
Add PipelineRoutingRule to migrations/env.py.
2026-03-12 22:07:54 +00:00
Christian Krakau-Louis 4345d5128f Merge pull request #607 from christianlouis/copilot/add-user-settings-management
fix(tests): update whoami_handler call sites to pass required db argument
2026-03-12 22:44:13 +01:00
copilot-swe-agent[bot] 9583d6d96f fix(profile): address code review feedback - early size check, CSRF helper, test constants
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 13:10:46 +00:00
copilot-swe-agent[bot] b0d6f1ab60 feat(profile): add user self-service profile settings page and API
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 13:00:19 +00:00
copilot-swe-agent[bot] b483ed16fc fix(i18n): register _ in all Jinja2 template instances, sync translation files, fix pipelines.html
- Register _() translation function in app/auth.py, app/api/local_auth.py, and
  app/api/billing.py template env globals to fix jinja2.exceptions.UndefinedError
- Update test_settings_template_has_db_wizard_link to assert i18n key instead of
  literal "DB Wizard" (template now uses {{ _("settings.db_wizard_btn") }})
- Add 418 missing en.json keys to all 30 non-English translation files as English
  fallbacks to fix test_all_languages_have_same_keys
- Fix orphan </template> tag in pipelines.html by adding missing
  <template x-if="pipelineModal.saving"> opening tag (fixes djlint H025)

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-12 12:46:11 +00:00
Christian Krakau-Louis 7f644a2388 Merge pull request #604 from christianlouis/copilot/configure-attachment-ingestion
feat(imap): fine-grained attachment ingestion profiles with per-category selection
2026-03-12 08:53:45 +01:00
Christian Krakau-Louis 87fca21c7f Merge pull request #605 from christianlouis/copilot/enable-disable-storage-locations
feat(storage): explicit enable/disable flag per global storage destination
2026-03-12 08:51:14 +01:00