- Add _require_admin + AdminUser dependency to google_drive.py, dropbox.py, onedrive.py
and switch save-settings endpoints from @require_login to Depends(_require_admin) so
tests can use dependency_overrides to bypass auth
- Wrap lifespan shutdown section (logging.info + notify_shutdown) in try/except to
silence OSError and other exceptions during shutdown (test_lifespan_shutdown_*)
- Add @patch("app.tasks.imap_tasks.is_private_ip", return_value=False) to 5 IMAP
tests that use imap.example.com (unresolvable in CI, causing is_private_ip to return
True and pull_inbox to return early before any IMAP operations)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/06fb29ae-8e36-4f13-89b8-68c06378e8a6
- fix(api/dropbox): _require_admin bypasses auth when AUTH_ENABLED=False,
fixing all 5 TestSaveDropboxSettings failures
- fix(api/onedrive): same AUTH_ENABLED bypass in _require_admin; fix one-arg
update_env_file call using env_utils version for token rotation
- fix(auth): update login TemplateResponse to Starlette 1.0+ API
(request as first arg instead of in context dict)
- fix(api/local_auth): update all TemplateResponse calls to Starlette 1.0+ API
- fix(views/share): update TemplateResponse call to Starlette 1.0+ API
- fix(api/billing): update TemplateResponse call to Starlette 1.0+ API
- fix(tests/test_imap_tasks): mock is_private_ip for tests using
imap.example.com (unresolvable in sandboxed/CI environments)
- fix(tests): update TemplateResponse call_args assertions to new API
(call_args.kwargs['context'] instead of call_args[0][1])
- fix(tests): update fake_original signatures in dark_mode tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/52d7b7b7-3a71-4a96-b2b1-b675b8a6d3b4
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract all OAuth registration into _setup_social_providers() with a
_register_oauth_client() helper that clears the authlib _clients cache
so credentials can change without a restart
- Add refresh_social_providers() public function called after every
settings reload (lifespan startup + settings_sync live reload)
- Fix connections page linked status to use _get_effective() (DB-aware)
instead of the stale startup-time SOCIAL_PROVIDERS dict
- Fix oauth_configured template variable similarly
- Add tests: DB-driven linked status, stale-provider clearing,
register_oauth_client cache-clear, refresh function coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ef15910f-fd25-469a-814b-9e1fb40659c9
- Add UserSession and QRLoginChallenge models for session tracking
and mobile QR authentication
- Add session_manager utility with create/validate/revoke/cleanup
functions and QR challenge helpers
- Add /api/sessions endpoints for listing, revoking, and
'log off everywhere' functionality
- Add /api/qr-auth endpoints for challenge creation, polling, and
claiming with API token issuance
- Add session config fields (lifetime, custom override, QR TTL)
- Update get_current_user to validate server-side sessions
- Create server-side sessions on all login paths (local, OAuth,
social, admin)
- Revoke server-side session on logout
- Configure SessionMiddleware max_age from session lifetime settings
- Graceful degradation: old sessions without _session_token continue
to work
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add LOG_FORMAT setting (text/json) for structured JSON log output
- Add LOG_SYSLOG_* settings for direct syslog forwarding of app logs
- JSON format compatible with Grafana Loki, Splunk, ELK, Datadog
- Syslog forwarding uses Python's SysLogHandler (UDP/TCP)
- Update .env.demo and ConfigurationGuide.md with all new settings
- Add tests for JSON formatter and syslog config fields
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add `log_level` setting to config.py (default: INFO, env: LOG_LEVEL)
- Configure Python root logger in main.py with standard precedence:
LOG_LEVEL explicit > DEBUG=true implies DEBUG > default INFO
- Add timestamp to log format for production readability
- Suppress noisy third-party loggers at DEBUG level
- Add comprehensive debug logging to all auth functions
- Add LOG_LEVEL/DEBUG to .env.demo and ConfigurationGuide.md
- Add tests for logging config and auth debug output
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The error templates (404.html, 500.html) extend base.html which uses
{{ suggested_languages | tojson }} in the language selector dropdown.
The _error_templates instance in app/main.py was missing this global,
causing Jinja2 Undefined objects to be passed to the tojson filter,
resulting in "TypeError: Object of type Undefined is not JSON serializable"
errors in 35 tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge main branch into compliance templates feature branch.
Main had advanced with migrations 027-030 (ensure_shared_links,
audit_logs, user_language_preference, mobile_devices) since this
branch forked. Our compliance migration was 027 with down_revision
026, which conflicted with main's 027_ensure_shared_links_table.
Changes:
- Merge main (including i18n, audit logs, mobile, GraphQL features)
- Resolve conflicts in app/api/__init__.py, app/models.py, tests/conftest.py
- Rename 027_add_compliance_templates → 031_add_compliance_templates
- Rechain: down_revision 026_add_scheduled_jobs → 030_add_mobile_devices
- Add ComplianceTemplate to migrations/env.py imports
- Alembic now has single head: 031_add_compliance_templates
- Fix detect_language() to safely handle unhashable session values and
requests missing cookies/headers attributes (TypeError + AttributeError)
- Add default English `_()` translation function to Jinja2 template
environment globals so error pages always have it available
- Fix app/main.py exception handlers to use a dedicated error templates
instance with `_` registered, keeping it separate from view templates
to avoid test patches breaking error rendering
- Fix app/views/plans.py to import shared templates from app.views.base
instead of creating its own Jinja2Templates instance
- Make migration 029_add_user_language_preference idempotent: skip
ALTER TABLE if user_profiles table does not exist
- Update test_i18n.py expectations to reflect 31 supported languages
- Create 21 missing translation files (nb, da, sv, fi, is, ga, lb, ca,
cs, sk, hu, sl, hr, ro, bg, el, et, lv, lt, tr, uk) with English
placeholder translations
- Update de.json with 117 missing translation keys including proper
German translations
- Update es, fr, it, nl, pl, pt, ru, zh translation files with missing
keys using English fallbacks
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Move init_sentry() from module level into the FastAPI lifespan context
manager, immediately after load_settings_from_db() completes. This
ensures that SENTRY_DSN and other Sentry settings configured via the
database admin UI are picked up on every restart.
Also update tests and docs accordingly.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace MkDocs redirect with a branded, SEO-optimised Help Center template
- Add sections: Quick Start, Sources, Destinations, Workflows, FAQ, Support
- Integrate optional Zammad live-chat widget and ticket form
- Add config settings: ZAMMAD_URL, ZAMMAD_CHAT_ENABLED, ZAMMAD_CHAT_ID,
ZAMMAD_FORM_ENABLED, SUPPORT_EMAIL
- Move MkDocs developer docs from /help to /developer-docs
- Move interactive API docs (Swagger/ReDoc) to /admin/api-docs and /admin/api-redoc
- Add API Docs and Developer Docs links to Admin menu (desktop + mobile)
- Update navigation Help link from /help/ to /help
- Update .env.demo with Zammad configuration examples
- Document new settings in docs/ConfigurationGuide.md
- Rewrite tests to cover new Help Center behaviour
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add MkDocs Material docs build stage to Dockerfile and Dockerfile.local
- Mount pre-built docs as static files at /help/ in FastAPI (app/main.py)
- Add app/views/help.py with /help → /help/ permanent redirect route
- Register help router in app/views/__init__.py
- Add Help nav link to base.html (public + app nav, desktop + mobile)
- Create how-to guides: HP printer, ScanSnap, watched folder, email ingestion, mobile scanning
- Update mkdocs.yml with How-To Guides section and Material theme palette
- Add optional docs service (squidfunk/mkdocs-material) to docker-compose.yaml with docs profile
- Add mkdocs-material to requirements-dev.txt
- Add /docs_build to .gitignore
- Add tests for help view (8 tests, 100% coverage on help.py)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The pipeline management UI showed an empty list after first boot because no
default system pipeline was created. This adds seed_default_pipeline() which:
- Creates a system-owned (owner_id=NULL), is_default=True pipeline named
"Standard Processing Pipeline" at application startup
- Steps mirror the current hardcoded Celery processing workflow:
convert_to_pdf → check_duplicates → ocr → extract_metadata →
embed_metadata → compute_embedding → send_to_destinations
- Is idempotent: no-op if any system pipeline already exists
- Handles missing pipelines table gracefully (during first migration run)
Also wires the seeder into app/main.py lifespan startup using the same
pattern as seed_default_plans.
9 new tests added covering creation, step order, idempotency, and API visibility.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add SubscriptionPlan model and subscription_plans table (migration 015)
- Add billing cycle/period/allow_overage fields to UserProfile (migration 016)
- Add subscription_overage_percent config field (replaces overage_factor)
- Rewrite check_upload_allowed: use overage_percent, yearly carry-over, no daily cap
- Add seed_default_plans(), _plan_to_dict(), get_year_file_count(), _months_elapsed()
- Update get_tier/get_all_tiers to be DB-first with TIER_DEFAULTS fallback
- Add TIER_DEFAULTS alias (TIERS kept for backward compat)
- New /api/plans/ CRUD endpoints (admin-only except list/get)
- New /admin/plans Plan Designer page with Alpine.js UI
- Add Plan Designer link to admin navigation in base.html
- Remove 'Files per day' row from pricing comparison table
- Add billing cycle + period start to admin users edit modal
- Seed default plans on startup in lifespan handler
- Rewrite docs/SubscriptionTiers.md with full plan/overage/API docs
- Fix all tests in test_subscription.py (remove daily cap tests, add overage/carry-over tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add app/utils/ocr_language_manager.py: detects tessdata dir, downloads
missing .traineddata files via wget/curl from tessdata_fast GitHub repo,
pre-downloads EasyOCR models, exposes async background-thread helper
- TesseractOCRProvider.process() calls ensure_tesseract_languages() before
running pytesseract; raises clear error if languages remain unavailable
- EasyOCRProvider.process() logs informational message when models download
- app/main.py: calls ensure_ocr_languages_async() at startup
- app/utils/settings_sync.py: triggers language re-check after every
settings reload so UI changes take effect without container restart
- app/api/settings.py: adds POST /api/settings/install-ocr-languages
endpoint for on-demand language installation from the admin UI
- Dockerfile: adds wget for runtime tessdata downloads
- docs/ConfigurationGuide.md: documents automatic language download
- tests/test_ocr_language_manager.py: 29 unit tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add CORSMiddleware (disabled by default, enabled via CORS_ENABLED=true)
- Add cors_enabled, cors_allowed_origins, cors_allow_credentials,
cors_allowed_methods, cors_allowed_headers settings to config.py
- Add parse_comma_separated_list validator for CORS list env vars
- Insert CORS middleware between SessionMiddleware and ProxyHeaders
so preflight runs before CSRF/auth but after proxy-header processing
- Document CORS env vars in .env.demo with rationale for proxy-first approach
- Mark CORS TODO as completed in SECURITY_AUDIT.md
- Add tests/test_cors.py with 12 unit and integration tests
Closes#175
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add RequestSizeLimitMiddleware that checks Content-Length header
before request body is read: non-multipart requests capped at
MAX_REQUEST_BODY_SIZE (default 1 MB), multipart uploads capped at
MAX_UPLOAD_SIZE (default 1 GB). Returns HTTP 413 on violation.
- Register middleware in app/main.py
- Add max_request_body_size setting to app/config.py
- Fix ui_upload in files.py to check Content-Length early and read
in 64 KB chunks (bounded memory usage), removing the post-write
os.path.getsize check
- Document MAX_REQUEST_BODY_SIZE in .env.demo and ConfigurationGuide.md
- Mark SECURITY_AUDIT.md item #4 as resolved
- Add 9 tests in test_request_size_limit.py
- Update test_upload_file_too_large to use patch.object instead of
the now-unused os.path.getsize mock
Closes#173
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Break long line in lifespan function to multiple lines
- Remove test_startup.py (not needed, existing tests validate startup)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Convert sync and async startup handlers to single lifespan function
- Remove deprecated @app.on_event("startup") and @app.on_event("shutdown")
- Add asynccontextmanager import for lifespan pattern
- Consolidate all startup/shutdown logic into one place
- Fixes Python 3.14 compatibility issue causing startup failure
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Modified exception handlers in app/main.py to check if request path starts with /api/
- HTTPException handler returns JSON for API routes, HTML for frontend routes
- General exception handler (500) also checks and returns appropriate format
- Enhanced frontend deleteFile() to handle non-JSON responses gracefully
- Added content-type checking before parsing JSON
- Added comprehensive tests for API error handling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]