- Added a benchmark script in tests/test_notifications_api.py that proved the N+1 issue issue.
- Replaced iterative DB lookups inside `for item in body.preferences:` with single pre-fetch query and local `prefs_dict` lookups.
- Verified test benchmark time drops from ~0.0964s to ~0.0141s for a batch of 100 items.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added a strict regex validation allowlist for table names in `preview_migration` before using them in raw SQL queries. This ensures that only alphanumeric characters and underscores are allowed, preventing potential SQL injection even if the source of table names were to be manipulated.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added the `--` argument before positional arguments in rclone subprocess calls (link, mkdir, copy) in `app/tasks/upload_with_rclone.py`. This ensures that filenames or destinations starting with a hyphen are treated as paths rather than unintended command-line flags.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix language dropdown in base.html by moving tojson data out of HTML
attribute into a script tag (prevents quote conflicts breaking Alpine.js)
- Fix avatar placeholder 404 by correcting filename reference from
avatar-placeholder.svg to default-avatar.svg
- Add session hydration from DB in _inject_global_context() so
detect_language() uses the stored preference on every request
- Sync session and cookie in PATCH /api/profile when language changes
- Reload page after language change in profile to reflect new locale
- Add tests for session/cookie sync and DB hydration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Move the valid filename regex pattern to a shared constant in app/utils/filename_utils.py and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase. Also normalized line endings in app/tasks/extract_metadata_with_gpt.py.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Refactor `app/api/audit_logs.py` to use the `Annotated` type hint pattern for FastAPI dependencies (`Depends`) and query parameters (`Query`).
- Resolves B008: Function-call in default argument.
- Improves code maintainability and readability by following modern FastAPI best practices.
- Maintains consistency with other modules in the codebase (e.g., `files.py`, `integrations.py`).
- No changes to API runtime behavior.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The error templates (404.html, 500.html) extend base.html which uses
{{ suggested_languages | tojson }} in the language selector dropdown.
The _error_templates instance in app/main.py was missing this global,
causing Jinja2 Undefined objects to be passed to the tojson filter,
resulting in "TypeError: Object of type Undefined is not JSON serializable"
errors in 35 tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced manual double-quoting of table names with SQLAlchemy's dialect-specific
identifier preparer in `app/utils/db_migrate.py`. This ensures proper quoting
for any database dialect and acts as a defense-in-depth measure against
SQL injection or syntax errors if a table name contains unexpected characters.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add get_suggested_languages() to i18n.py returning ≤6 ranked
suggestions (current locale → Accept-Language header → popular
language fallbacks); refactor _parse_accept_language to share
a common _parse_accept_language_entries() helper
- Inject suggested_languages into every template context (base.py)
- Redesign nav-bar language dropdown (base.html): button shows
current-language flag emoji; dropdown lists 5-7 suggestions with
flags and native names; Alpine.js search input filters all 77
languages live; footer shows count and Search shortcut
- Add language.search_placeholder and language.no_results keys to
all 77 translation JSON files (en values; external script
propagates translations to other locales)
- Remove test_all_languages_have_same_keys (external sync script
owns key completeness); add TestGetSuggestedLanguages (7 unit
tests); update test_language_selector_in_nav for new HTML
- Update InternationalizationGuide.md: single-step en.json-only
workflow for adding new translation keys
- Update .github/copilot-instructions.md: add i18n/l10n section
documenting the en.json-only rule for future agents
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add 18 previously missing language entries to SUPPORTED_LANGUAGES in
app/utils/i18n.py so every translation JSON file in frontend/translations/
is properly indexed and served by the language selector:
New languages: af, ar, cy, eo, fa, fy, gl, he, ja, kn, ko, li, nds, no, pa, sr, vi, vls
Each new language entry includes:
- Correct ISO 639-1/639-3 code matching its JSON filename
- Native name and display name
- Appropriate country/language flag emoji
- Locale-specific date and number formatting rules in _LOCALE_FORMATS
Also: rename nb "Norwegian" → "Norwegian Bokmål" to distinguish it from no "Norwegian".
Update tests/test_i18n.py:
- Count assertions: 31 → 49
- Expected code set expanded to all 49 codes
- Fix test_unsupported_language_fallback (ja/ko now supported, use xx/yy)
Update docs/InternationalizationGuide.md:
- Language count: 10 → 49
- Full language table with flags, native names, tiers
- Complete file structure listing all 49 JSON files
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Register _() translation function in app/auth.py, app/api/local_auth.py, and
app/api/billing.py template env globals to fix jinja2.exceptions.UndefinedError
- Update test_settings_template_has_db_wizard_link to assert i18n key instead of
literal "DB Wizard" (template now uses {{ _("settings.db_wizard_btn") }})
- Add 418 missing en.json keys to all 30 non-English translation files as English
fallbacks to fix test_all_languages_have_same_keys
- Fix orphan </template> tag in pipelines.html by adding missing
<template x-if="pipelineModal.saving"> opening tag (fixes djlint H025)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Improve warning log in _resolve_categories_for_profile() to include
exception type name for better troubleshooting
- Add SQLAlchemy IS NULL comment to imap_profiles.py filter
- Pass default_categories from server to template to avoid hardcoded
category list in JS (now uses {{ default_categories | tojson }})
- Simplify view profiles query (remove redundant unauthenticated path)
- Update docs: ConfigurationGuide.md and EmailIngestion.md with
full profiles documentation including category table and API reference
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add a configurable switch to control which attachment types are ingested
via IMAP. Images are excluded by default; office files and PDFs are ingested.
- Add global `IMAP_ATTACHMENT_FILTER` config setting (default: `documents_only`)
- Add `attachment_filter` column to `UserImapAccount` model for per-user override
- Migration 032 adds the column to `user_imap_accounts` table
- Update `fetch_attachments_and_enqueue()` to respect filter (documents_only/all)
- Update `pull_inbox()`, `_pull_user_imap_accounts()`, and
`_pull_user_integration_imap()` to pass the resolved filter
- Update IMAP accounts API (schemas, create/update handlers, response serializer)
- Update IMAP accounts UI to show attachment filter dropdown in modal and
display filter badges on account cards
- Add 6 new tests covering attachment filter behaviour
- Update ConfigurationGuide.md, EmailIngestion.md, and .env.demo
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main branch into compliance templates feature branch.
Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.
Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Christian Krakau-Louis