- fix(api/dropbox): _require_admin bypasses auth when AUTH_ENABLED=False,
fixing all 5 TestSaveDropboxSettings failures
- fix(api/onedrive): same AUTH_ENABLED bypass in _require_admin; fix one-arg
update_env_file call using env_utils version for token rotation
- fix(auth): update login TemplateResponse to Starlette 1.0+ API
(request as first arg instead of in context dict)
- fix(api/local_auth): update all TemplateResponse calls to Starlette 1.0+ API
- fix(views/share): update TemplateResponse call to Starlette 1.0+ API
- fix(api/billing): update TemplateResponse call to Starlette 1.0+ API
- fix(tests/test_imap_tasks): mock is_private_ip for tests using
imap.example.com (unresolvable in sandboxed/CI environments)
- fix(tests): update TemplateResponse call_args assertions to new API
(call_args.kwargs['context'] instead of call_args[0][1])
- fix(tests): update fake_original signatures in dark_mode tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/52d7b7b7-3a71-4a96-b2b1-b675b8a6d3b4
This commit safely handles the dynamic table names in database migration queries
by leveraging `sqlalchemy.select` and `sqlalchemy.table` in `app/utils/db_migrate.py`.
It addresses the `# noqa: S608` exception that was in place for string interpolation
SQL queries which are a known security anti-pattern.
Additionally, this commit includes the latest updates to `app/views/base.py`
from the `main` branch to handle backward compatibility across Starlette
versions (<1.0 vs 1.0+) when invoking `Jinja2Templates.TemplateResponse`,
resolving previous merge conflicts in the PR.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The transition to a newer FastAPI/Starlette version changed the signature of `Jinja2Templates.TemplateResponse` from `(name, context)` to `(request, name, context)`.
The `app/views/base.py:template_response_with_version` wrapper naively forwarded positional arguments `*args` to `original_template_response`. This caused the template name (`"files.html"`) to be passed as the `request` parameter, and the context dictionary to be passed as the `name` parameter. This resulted in Jinja2 attempting to cache the template using a dictionary as the cache key, which triggered a `TypeError: unhashable type: 'dict'`.
This commit updates the wrapper to automatically translate the legacy positional arguments `(name: str, context: dict)` into the explicit keyword arguments `request=context.get("request"), name=name, context=context` required by modern Starlette, preventing template rendering crashes across the application and restoring passing CI test suites.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract all OAuth registration into _setup_social_providers() with a
_register_oauth_client() helper that clears the authlib _clients cache
so credentials can change without a restart
- Add refresh_social_providers() public function called after every
settings reload (lifespan startup + settings_sync live reload)
- Fix connections page linked status to use _get_effective() (DB-aware)
instead of the stale startup-time SOCIAL_PROVIDERS dict
- Fix oauth_configured template variable similarly
- Add tests: DB-driven linked status, stale-provider clearing,
register_oauth_client cache-clear, refresh function coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ef15910f-fd25-469a-814b-9e1fb40659c9
- Add `qr_login_enabled` boolean config field (default True) to app/config.py
- Add `qr_login_enabled` entry to settings metadata in app/utils/settings_service.py
- Fix app/views/settings.py to use `qr_login_enabled` directly instead of
deriving enablement status from qr_login_challenge_ttl_seconds (integer TTL)
- Fix admin_connections.html: remove hardcoded `disabled` attribute from the
Mobile Phone Upload toggle and wire up onchange handler so toggling actually
persists the setting via toggleSetting('qr_login_enabled', this.checked)
- Gate all three QR auth API endpoints on settings.qr_login_enabled so the
feature is actually disabled when the toggle is turned off
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/8421cdb2-d92d-4b83-9cda-c44983d35173
- Add PUT /{key} endpoint to settings API with SettingValueUpdate body model (only
requires value, key comes from URL path) — fixes 405 Method Not Allowed errors
from the admin Connections wizard which used PUT to save settings
- Fix grey toggles on /admin/connections: they appeared grey because all saves were
silently failing with 405; now saves succeed and toggles reflect actual state
- Add social_auth_google_use_global_credentials config field and auth.py logic to
reuse google_drive_client_id/google_drive_client_secret for Google Sign-In
- Add social_auth_microsoft_use_global_credentials config field and auth.py logic to
reuse onedrive_client_id/onedrive_client_secret for Microsoft Sign-In
- Also apply consistent both-field check for Dropbox global credentials fallback
- Add settings metadata entries for the two new boolean settings
- Add Google and Microsoft settings_keys to admin_connections service definitions
- Add JS visibility toggle logic for Google/Microsoft credential fields in admin UI
- Add 6 new unit/integration tests for PUT endpoint and SettingValueUpdate model
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ac66041a-2cbd-4d90-8f8e-3588c629d4d8
- Add 3 new config fields for browser SDK sample rates:
sentry_js_traces_sample_rate (default 0.0),
sentry_js_replay_session_sample_rate (default 0.0),
sentry_js_replay_on_error_sample_rate (default 0.1)
- Expose Sentry config to Jinja2 templates via _inject_global_context;
empty-string DSN normalized to None so {% if sentry_dsn %} guard works
- Load Sentry Browser SDK bundle.tracing.replay.min.js from the official
Sentry CDN in base.html when SENTRY_DSN is configured, with Sentry.init()
for error capture, browser tracing and session replay
- Register new JS settings fields in SETTING_METADATA so they appear on the
admin Settings → Observability page
- Update .env.demo with commented-out examples for SENTRY_JS_* variables
- Update docs/ConfigurationGuide.md and docs/SentrySetup.md with full
browser SDK documentation, env-specific examples and troubleshooting
- Add TestSentryJsTemplateContext (5 tests) and TestSentryJsConfig (4 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/4a945567-67df-4264-aaed-75eb2e236e9c
- /files/<id> → new summary page with navigation cards
- /files/<id>/detail → document detail with metadata, preview, text
- /files/<id>/process → processing pipeline status and history
- /files/<id>/annotations → comments & annotations with EmbedPDF viewer
- /files/<id>/comments → redirects to /annotations
- Added embed-pdf-viewer as git submodule for PDF annotation viewer
- Updated all navigation links across templates
- Updated all tests to use new URL structure
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/12276514-bd3d-4e3e-84d9-5977d1f82b19
Added tests for POST /api/dropbox/list-folders (7 tests) and
POST /api/onedrive/list-folders (7 tests) covering success, subfolder
navigation, empty directories, auth errors, API errors, path
normalization, and alphabetical sorting.
Added view tests for system credentials toggle visibility in Dropbox
setup wizard.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When system-wide Dropbox, Google Drive, or OneDrive app credentials are
configured by the admin, user-mode OAuth wizards now default to using
them. A toggle lets users switch to custom credentials if needed. This
removes the need for end users to register their own cloud provider apps.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add PUBLIC_BASE_URL optional config to override auto-detected OAuth redirect URIs
when behind a reverse proxy that doesn't forward X-Forwarded-Proto headers
- Add _build_dropbox_redirect_uri() helper in app/api/dropbox.py
- URL-encode redirect_uri in server-side Dropbox authorization URL
- Add _get_dropbox_callback_url() helper in app/views/dropbox.py
- Pass callback_url to both setup and callback templates
- Update templates to use server-provided callback_url instead of window.location.origin
- Update settings_service.py to register new setting
- Update .env.demo, ConfigurationGuide.md, and DropboxSetup.md documentation
- Add tests for new helper functions and global-authorize-url endpoint
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merges origin/main (v0.155.0) into the classification feature branch,
resolving all 18 conflicted files by accepting main's version and
re-applying only classification-specific additions:
- Renumber migration from 037 to 038 (chains from 037_user_sessions)
- Re-add ClassificationRuleModel to models.py, env.py, conftest.py
- Re-add classification_rules_router to api/__init__.py
- All session management, QR auth, and devices code preserved from main
Migration chain validated. 62 classification tests pass.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge origin/main into feature branch, resolving 3 conflicts:
- app/api/__init__.py: add classification_rules_router alongside new
routers from main (audit_logs, i18n, mobile, compliance, translation)
- app/models.py: keep ClassificationRuleModel alongside new models from
main (MobileDevice, ComplianceTemplate, PipelineRoutingRule)
- tests/conftest.py: import both ClassificationRuleModel and new models
from main (AuditLog, ComplianceTemplate)
Also renumber migration from 027 to 037 to chain from the latest
migration on main (036_add_document_translation_fields).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix language dropdown in base.html by moving tojson data out of HTML
attribute into a script tag (prevents quote conflicts breaking Alpine.js)
- Fix avatar placeholder 404 by correcting filename reference from
avatar-placeholder.svg to default-avatar.svg
- Add session hydration from DB in _inject_global_context() so
detect_language() uses the stored preference on every request
- Sync session and cookie in PATCH /api/profile when language changes
- Reload page after language change in profile to reflect new locale
- Add tests for session/cookie sync and DB hydration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add get_suggested_languages() to i18n.py returning ≤6 ranked
suggestions (current locale → Accept-Language header → popular
language fallbacks); refactor _parse_accept_language to share
a common _parse_accept_language_entries() helper
- Inject suggested_languages into every template context (base.py)
- Redesign nav-bar language dropdown (base.html): button shows
current-language flag emoji; dropdown lists 5-7 suggestions with
flags and native names; Alpine.js search input filters all 77
languages live; footer shows count and Search shortcut
- Add language.search_placeholder and language.no_results keys to
all 77 translation JSON files (en values; external script
propagates translations to other locales)
- Remove test_all_languages_have_same_keys (external sync script
owns key completeness); add TestGetSuggestedLanguages (7 unit
tests); update test_language_selector_in_nav for new HTML
- Update InternationalizationGuide.md: single-step en.json-only
workflow for adding new translation keys
- Update .github/copilot-instructions.md: add i18n/l10n section
documenting the en.json-only rule for future agents
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Improve warning log in _resolve_categories_for_profile() to include
exception type name for better troubleshooting
- Add SQLAlchemy IS NULL comment to imap_profiles.py filter
- Pass default_categories from server to template to avoid hardcoded
category list in JS (now uses {{ default_categories | tojson }})
- Simplify view profiles query (remove redundant unauthenticated path)
- Update docs: ConfigurationGuide.md and EmailIngestion.md with
full profiles documentation including category table and API reference
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main branch into compliance templates feature branch.
Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.
Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix detect_language() to safely handle unhashable session values and
requests missing cookies/headers attributes (TypeError + AttributeError)
- Add default English `_()` translation function to Jinja2 template
environment globals so error pages always have it available
- Fix app/main.py exception handlers to use a dedicated error templates
instance with `_` registered, keeping it separate from view templates
to avoid test patches breaking error rendering
- Fix app/views/plans.py to import shared templates from app.views.base
instead of creating its own Jinja2Templates instance
- Make migration 029_add_user_language_preference idempotent: skip
ALTER TABLE if user_profiles table does not exist
- Update test_i18n.py expectations to reflect 31 supported languages
- Create 21 missing translation files (nb, da, sv, fi, is, ga, lb, ca,
cs, sk, hu, sl, hr, ro, bg, el, et, lv, lt, tr, uk) with English
placeholder translations
- Update de.json with 117 missing translation keys including proper
German translations
- Update es, fr, it, nl, pl, pt, ru, zh translation files with missing
keys using English fallbacks
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Resolve conflict in app/api/__init__.py (keep both audit_logs_router and i18n_router)
- Incorporate AuditLog model, audit_service, audit_logs API/views from main
- Relink migration from 026→027 to 028→029 (chain after 028_add_audit_logs)
- Update migrations/env.py with full model import list from main
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Implements the classify pipeline step with:
- Classification rules engine (app/utils/classification_rules.py) with
pre-built categories (invoice, contract, receipt, letter, report,
bank_statement, tax_document, insurance, payslip) and support for
filename patterns, content keywords, and metadata matching rules
- Celery task (app/tasks/classify_document.py) that runs as a pipeline step
- CRUD API (app/api/classification_rules.py) for managing custom rules
- ClassificationRuleModel in app/models.py with migration 027
- Updated pipeline step config_schema and stage mapping
- Comprehensive tests for engine, API, and task
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Create i18n utility module (app/utils/i18n.py) with translation loading,
browser language detection, AI fallback, and l10n helpers
- Add JSON translation files for EN, DE, FR, ES, IT, PT, NL, PL, ZH, RU
- Add preferred_language column to UserProfile model with migration
- Register _() translation function as Jinja2 global
- Update base.html with translated navigation, footer, cookie notice
- Add language selector dropdown in nav bar (desktop + mobile)
- Create API endpoints for language preference (POST/GET /api/i18n/)
- Support language detection: user profile > cookie > Accept-Language > default
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Christian Krakau-Louis