- Resolve merge conflicts in app/api/onedrive.py and tests/test_api_google_drive_final.py
- Fix legacy Dict[str, str] type hints in update_env_file functions to use dict[str, str]
- Add admin-only access (_require_admin dependency) to save-settings endpoints
in google_drive.py, onedrive.py, and dropbox.py
- Fix in_memory_only response field to reflect actual env_write_success status
- Update tests to override _require_admin dependency for save-settings endpoint tests
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Renamed `save_dropbox_settings` inside `app/api/google_drive.py` to `save_google_drive_settings` to fix a copy-paste naming error.
- Extracted duplicate `.env` file updating logic from `app/api/google_drive.py`, `app/api/onedrive.py`, and `app/api/dropbox.py` into a new reusable helper function `update_env_file` inside `app/utils/settings_service.py`.
- Refactored the three API endpoints to use the new helper function, significantly reducing complexity and code duplication.
- Updated relevant test files (`tests/test_api_google_drive_final.py`) to reflect the new function name.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add `qr_login_enabled` boolean config field (default True) to app/config.py
- Add `qr_login_enabled` entry to settings metadata in app/utils/settings_service.py
- Fix app/views/settings.py to use `qr_login_enabled` directly instead of
deriving enablement status from qr_login_challenge_ttl_seconds (integer TTL)
- Fix admin_connections.html: remove hardcoded `disabled` attribute from the
Mobile Phone Upload toggle and wire up onchange handler so toggling actually
persists the setting via toggleSetting('qr_login_enabled', this.checked)
- Gate all three QR auth API endpoints on settings.qr_login_enabled so the
feature is actually disabled when the toggle is turned off
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/8421cdb2-d92d-4b83-9cda-c44983d35173
- Add PUT /{key} endpoint to settings API with SettingValueUpdate body model (only
requires value, key comes from URL path) — fixes 405 Method Not Allowed errors
from the admin Connections wizard which used PUT to save settings
- Fix grey toggles on /admin/connections: they appeared grey because all saves were
silently failing with 405; now saves succeed and toggles reflect actual state
- Add social_auth_google_use_global_credentials config field and auth.py logic to
reuse google_drive_client_id/google_drive_client_secret for Google Sign-In
- Add social_auth_microsoft_use_global_credentials config field and auth.py logic to
reuse onedrive_client_id/onedrive_client_secret for Microsoft Sign-In
- Also apply consistent both-field check for Dropbox global credentials fallback
- Add settings metadata entries for the two new boolean settings
- Add Google and Microsoft settings_keys to admin_connections service definitions
- Add JS visibility toggle logic for Google/Microsoft credential fields in admin UI
- Add 6 new unit/integration tests for PUT endpoint and SettingValueUpdate model
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ac66041a-2cbd-4d90-8f8e-3588c629d4d8
- Add 3 new config fields for browser SDK sample rates:
sentry_js_traces_sample_rate (default 0.0),
sentry_js_replay_session_sample_rate (default 0.0),
sentry_js_replay_on_error_sample_rate (default 0.1)
- Expose Sentry config to Jinja2 templates via _inject_global_context;
empty-string DSN normalized to None so {% if sentry_dsn %} guard works
- Load Sentry Browser SDK bundle.tracing.replay.min.js from the official
Sentry CDN in base.html when SENTRY_DSN is configured, with Sentry.init()
for error capture, browser tracing and session replay
- Register new JS settings fields in SETTING_METADATA so they appear on the
admin Settings → Observability page
- Update .env.demo with commented-out examples for SENTRY_JS_* variables
- Update docs/ConfigurationGuide.md and docs/SentrySetup.md with full
browser SDK documentation, env-specific examples and troubleshooting
- Add TestSentryJsTemplateContext (5 tests) and TestSentryJsConfig (4 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/4a945567-67df-4264-aaed-75eb2e236e9c
Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).
Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
(classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models
Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)
For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45
Add a configurable switch to control which attachment types are ingested
via IMAP. Images are excluded by default; office files and PDFs are ingested.
- Add global `IMAP_ATTACHMENT_FILTER` config setting (default: `documents_only`)
- Add `attachment_filter` column to `UserImapAccount` model for per-user override
- Migration 032 adds the column to `user_imap_accounts` table
- Update `fetch_attachments_and_enqueue()` to respect filter (documents_only/all)
- Update `pull_inbox()`, `_pull_user_imap_accounts()`, and
`_pull_user_integration_imap()` to pass the resolved filter
- Update IMAP accounts API (schemas, create/update handlers, response serializer)
- Update IMAP accounts UI to show attachment filter dropdown in modal and
display filter badges on account cards
- Add 6 new tests covering attachment filter behaviour
- Update ConfigurationGuide.md, EmailIngestion.md, and .env.demo
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main branch into compliance templates feature branch.
Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.
Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
Add iCloud Drive as a new storage destination using the pyicloud library.
Includes upload task, configuration, user integration handler, provider
status, onboarding support, and comprehensive tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add REST hooks subscription endpoints, incoming action endpoints, and
Zapier-compatible flat payload format for automation platform integration.
- AutomationHook model for webhook subscriptions
- POST /api/automation/hooks/subscribe and DELETE /hooks/{id}
- GET /api/automation/triggers/sample/{event} for Zapier field mapping
- POST /api/automation/actions/upload for incoming document uploads
- Celery task with retry for async hook delivery
- Integration with existing webhook dispatch flow
- 30 passing tests covering all new functionality
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Create app/templates/email/default.html (fixes 'default.html not found' error)
- Add DEST_EMAIL_* settings to app/config.py (decoupled from shared EMAIL_* settings)
- Update upload_to_email task to use dest_email_* settings exclusively
- Update _should_upload_to_email() to check dest_email_* settings
- Update config validator, providers, and settings_service for dest_email_*
- Update .env.demo and docs/ConfigurationGuide.md
- Update all tests to use dest_email_* settings where appropriate"
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Merge main (pipelines feature) into branch, resolving conflicts in
app/api/__init__.py and app/views/__init__.py by keeping all routers
(onboarding + billing from our branch, pipelines from main)
- Fix migration 018 down_revision to depend on both 017_add_onboarding_fields
and 017_add_pipelines (Alembic multi-head merge pattern)
- Fix test_auth_module.py: add multi_user_enabled=False to three admin-auth
tests that call auth() directly without FastAPI DI
- Add missing SETTING_METADATA entries for allow_local_signup and all five
Stripe config keys (fixes test_all_config_settings_have_metadata)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add "DB Wizard" link button to settings page header
- Add help_link to database_url SETTING_METADATA pointing to /database-wizard
- Add help_link rendering in settings template for any setting with a help_link
- Fix SQLite whitespace path handling in build_connection_string
- Add dark mode CSS overrides for wizard template
- Add aria-describedby for all form inputs with help text
- Add prefers-reduced-motion media query for smooth scrolling
- Expand test coverage: 106 tests (up from 49)
- db_wizard.py: 100% coverage
- db_wizard view: 100% coverage
- database.py API: 97.37% coverage
- db_migrate.py: 96.60% coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When enabled, IMAP processing will fetch and process attachments but
will NOT modify the mailbox state (no starring, labeling, deleting,
or flag changes). This allows preprod instances to safely share a
Gmail inbox with production without interfering with production
email processing.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Raise quality acceptance threshold from 65→85 (configurable via TEXT_QUALITY_THRESHOLD)
- Reject text with significant issues (excessive_typos, garbage_characters,
incoherent_text, fragmented_sentences) even when score is above threshold
(configurable via TEXT_QUALITY_SIGNIFICANT_ISSUES)
- Add compare_text_quality() for AI-powered head-to-head comparison of
original embedded text vs fresh OCR output
- Update process_document to pass original text to OCR task for comparison
- Update process_with_ocr to run comparison and keep the higher-quality text
- Add new settings to settings_service.py metadata
- Update docs/ConfigurationGuide.md with new settings
- Add comprehensive tests for new threshold and comparison logic
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Christian Krakau-Louis