Move the valid filename regex pattern to a shared constant in `app/utils/filename_utils.py` and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase.
Also normalized line endings to LF in affected files to ensure CI compatibility.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add 18 previously missing language entries to SUPPORTED_LANGUAGES in
app/utils/i18n.py so every translation JSON file in frontend/translations/
is properly indexed and served by the language selector:
New languages: af, ar, cy, eo, fa, fy, gl, he, ja, kn, ko, li, nds, no, pa, sr, vi, vls
Each new language entry includes:
- Correct ISO 639-1/639-3 code matching its JSON filename
- Native name and display name
- Appropriate country/language flag emoji
- Locale-specific date and number formatting rules in _LOCALE_FORMATS
Also: rename nb "Norwegian" → "Norwegian Bokmål" to distinguish it from no "Norwegian".
Update tests/test_i18n.py:
- Count assertions: 31 → 49
- Expected code set expanded to all 49 codes
- Fix test_unsupported_language_fallback (ja/ko now supported, use xx/yy)
Update docs/InternationalizationGuide.md:
- Language count: 10 → 49
- Full language table with flags, native names, tiers
- Complete file structure listing all 49 JSON files
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Register _() translation function in app/auth.py, app/api/local_auth.py, and
app/api/billing.py template env globals to fix jinja2.exceptions.UndefinedError
- Update test_settings_template_has_db_wizard_link to assert i18n key instead of
literal "DB Wizard" (template now uses {{ _("settings.db_wizard_btn") }})
- Add 418 missing en.json keys to all 30 non-English translation files as English
fallbacks to fix test_all_languages_have_same_keys
- Fix orphan </template> tag in pipelines.html by adding missing
<template x-if="pipelineModal.saving"> opening tag (fixes djlint H025)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Improve warning log in _resolve_categories_for_profile() to include
exception type name for better troubleshooting
- Add SQLAlchemy IS NULL comment to imap_profiles.py filter
- Pass default_categories from server to template to avoid hardcoded
category list in JS (now uses {{ default_categories | tojson }})
- Simplify view profiles query (remove redundant unauthenticated path)
- Update docs: ConfigurationGuide.md and EmailIngestion.md with
full profiles documentation including category table and API reference
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add a configurable switch to control which attachment types are ingested
via IMAP. Images are excluded by default; office files and PDFs are ingested.
- Add global `IMAP_ATTACHMENT_FILTER` config setting (default: `documents_only`)
- Add `attachment_filter` column to `UserImapAccount` model for per-user override
- Migration 032 adds the column to `user_imap_accounts` table
- Update `fetch_attachments_and_enqueue()` to respect filter (documents_only/all)
- Update `pull_inbox()`, `_pull_user_imap_accounts()`, and
`_pull_user_integration_imap()` to pass the resolved filter
- Update IMAP accounts API (schemas, create/update handlers, response serializer)
- Update IMAP accounts UI to show attachment filter dropdown in modal and
display filter badges on account cards
- Add 6 new tests covering attachment filter behaviour
- Update ConfigurationGuide.md, EmailIngestion.md, and .env.demo
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main branch into compliance templates feature branch.
Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.
Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix detect_language() to safely handle unhashable session values and
requests missing cookies/headers attributes (TypeError + AttributeError)
- Add default English `_()` translation function to Jinja2 template
environment globals so error pages always have it available
- Fix app/main.py exception handlers to use a dedicated error templates
instance with `_` registered, keeping it separate from view templates
to avoid test patches breaking error rendering
- Fix app/views/plans.py to import shared templates from app.views.base
instead of creating its own Jinja2Templates instance
- Make migration 029_add_user_language_preference idempotent: skip
ALTER TABLE if user_profiles table does not exist
- Update test_i18n.py expectations to reflect 31 supported languages
- Create 21 missing translation files (nb, da, sv, fi, is, ga, lb, ca,
cs, sk, hu, sl, hr, ro, bg, el, et, lv, lt, tr, uk) with English
placeholder translations
- Update de.json with 117 missing translation keys including proper
German translations
- Update es, fr, it, nl, pl, pt, ru, zh translation files with missing
keys using English fallbacks
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Resolve conflict in app/api/__init__.py (keep both audit_logs_router and i18n_router)
- Incorporate AuditLog model, audit_service, audit_logs API/views from main
- Relink migration from 026→027 to 028→029 (chain after 028_add_audit_logs)
- Update migrations/env.py with full model import list from main
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Create i18n utility module (app/utils/i18n.py) with translation loading,
browser language detection, AI fallback, and l10n helpers
- Add JSON translation files for EN, DE, FR, ES, IT, PT, NL, PL, ZH, RU
- Add preferred_language column to UserProfile model with migration
- Register _() translation function as Jinja2 global
- Update base.html with translated navigation, footer, cookie notice
- Add language selector dropdown in nav bar (desktop + mobile)
- Create API endpoints for language preference (POST/GET /api/i18n/)
- Support language detection: user profile > cookie > Accept-Language > default
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add user_mode to dropbox/onedrive/google_drive setup views that loads integration config
- Show user-friendly auth wizard when integration_id is provided (user mode)
- In user mode: show integration name, current folder, back-to-integrations link
- In callback templates: only save credentials (not config) for user integrations
- In integrations dashboard: show Authorize/Re-Authorize button for all OAuth types
- Add WATCH_FOLDER OAuth support: detect source_type in config for auth button
- isOAuthType() and oauthLink() now accept full integration object
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add `from app.tasks.upload_to_user_integration import upload_to_user_integration`
to app/celery_worker.py — fixes TestCeleryWorkerConfig test that enforces
every tasks/ module is registered with Celery.
- Add tests/test_upload_handlers.py with 43 unit tests covering all 11
per-type upload handler functions (_upload_dropbox, _upload_s3,
_upload_google_drive, _upload_onedrive, _upload_webdav, _upload_nextcloud,
_upload_ftp, _upload_sftp, _upload_paperless, _upload_email, _upload_rclone)
plus 2 additional finalize_document_storage branch tests. All external
libraries (dropbox, boto3, msal, paramiko, smtplib, subprocess, requests)
are mocked so tests are hermetic and fast. Coverage on changed files:
upload_to_user_integration.py 94.71%, finalize_document_storage.py 95.51%
(both well above the 70% Codecov diff threshold).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Move init_sentry() from module level into the FastAPI lifespan context
manager, immediately after load_settings_from_db() completes. This
ensures that SENTRY_DSN and other Sentry settings configured via the
database admin UI are picked up on every restart.
Also update tests and docs accordingly.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
google-labs-jules[bot]