Fixed a `SyntaxError: keyword argument repeated` in `app/api/url_upload.py` where `event_hooks` was being passed twice as a keyword argument to `httpx.AsyncClient`. Combined both SSRF redirect validation hooks into a single list `event_hooks={"response": [validate_redirect, verify_redirect]}`. This restores critical SSRF protections via redirect validation and fixes the application crash.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: The `/process-url` endpoint used `httpx.AsyncClient` with `follow_redirects=True`. While the initial user-provided URL was validated against SSRF protections (blocking private/internal IPs), the client implicitly followed subsequent HTTP redirects without validating their target locations. This allowed an attacker to bypass the initial check by supplying a valid URL that redirected to an internal IP or cloud metadata endpoint.
🎯 Impact: An attacker could potentially access internal network services or cloud metadata endpoints.
🔧 Fix: Implemented an `event_hooks` listener (`validate_redirect`) on the `httpx.AsyncClient` that intercepts responses, extracts the `Location` header, resolves the absolute target URL, and applies the same `validate_url_safety` check before allowing the redirect to be followed.
✅ Verification: Ran `pytest tests/test_url_upload.py`, formatting checks via `ruff format` and linting via `ruff check`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds validation using `is_private_ip()` for user-provided hosts in `_test_imap_connection` and `_test_s3_connection` to prevent Server-Side Request Forgery vulnerabilities.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add _require_admin + AdminUser dependency to google_drive.py, dropbox.py, onedrive.py
and switch save-settings endpoints from @require_login to Depends(_require_admin) so
tests can use dependency_overrides to bypass auth
- Wrap lifespan shutdown section (logging.info + notify_shutdown) in try/except to
silence OSError and other exceptions during shutdown (test_lifespan_shutdown_*)
- Add @patch("app.tasks.imap_tasks.is_private_ip", return_value=False) to 5 IMAP
tests that use imap.example.com (unresolvable in CI, causing is_private_ip to return
True and pull_inbox to return early before any IMAP operations)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/06fb29ae-8e36-4f13-89b8-68c06378e8a6
The test_successful_conversion test asserts that '--' precedes the input/output
file paths in the ocrmypdf command as a security measure against argument
injection (file paths starting with '-' being interpreted as options).
The implementation was missing this separator, causing the test to fail and
triggering a downstream pytest INTERNALERROR (OSError: Bad file descriptor)
when pytest's terminal writer tried to report the failure.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/bd7ef195-4b30-456e-8118-5c200fb4bf45
Move the valid filename regex pattern to a shared constant in `app/utils/filename_utils.py` and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase.
Changes:
- Defined `VALID_FILENAME_PATTERN` and `VALID_FILENAME_RE` in `app/utils/filename_utils.py`.
- Updated `app/tasks/extract_metadata_with_gpt.py` to use `VALID_FILENAME_RE`.
- Updated `tests/test_path_traversal_security.py` to use `VALID_FILENAME_PATTERN`.
This refactoring addresses the duplication mentioned in the TODO in `tests/test_path_traversal_security.py`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- fix(api/dropbox): _require_admin bypasses auth when AUTH_ENABLED=False,
fixing all 5 TestSaveDropboxSettings failures
- fix(api/onedrive): same AUTH_ENABLED bypass in _require_admin; fix one-arg
update_env_file call using env_utils version for token rotation
- fix(auth): update login TemplateResponse to Starlette 1.0+ API
(request as first arg instead of in context dict)
- fix(api/local_auth): update all TemplateResponse calls to Starlette 1.0+ API
- fix(views/share): update TemplateResponse call to Starlette 1.0+ API
- fix(api/billing): update TemplateResponse call to Starlette 1.0+ API
- fix(tests/test_imap_tasks): mock is_private_ip for tests using
imap.example.com (unresolvable in sandboxed/CI environments)
- fix(tests): update TemplateResponse call_args assertions to new API
(call_args.kwargs['context'] instead of call_args[0][1])
- fix(tests): update fake_original signatures in dark_mode tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/52d7b7b7-3a71-4a96-b2b1-b675b8a6d3b4
- Resolve merge conflicts in app/api/onedrive.py and tests/test_api_google_drive_final.py
- Fix legacy Dict[str, str] type hints in update_env_file functions to use dict[str, str]
- Add admin-only access (_require_admin dependency) to save-settings endpoints
in google_drive.py, onedrive.py, and dropbox.py
- Fix in_memory_only response field to reflect actual env_write_success status
- Update tests to override _require_admin dependency for save-settings endpoint tests
- Resolve merge conflicts in .jules/sentinel.md and app/utils/network.py
- Refactor join_url() to use urllib.parse.urlsplit/urlunsplit and posixpath
instead of sentinel-string hack, preventing corruption for any input URL
- Fix test to use pytest tmp_path fixture instead of hard-coded /tmp/workdir
This commit safely handles the dynamic table names in database migration queries
by leveraging `sqlalchemy.select` and `sqlalchemy.table` in `app/utils/db_migrate.py`.
It addresses the `# noqa: S608` exception that was in place for string interpolation
SQL queries which are a known security anti-pattern.
Additionally, this commit includes the latest updates to `app/views/base.py`
from the `main` branch to handle backward compatibility across Starlette
versions (<1.0 vs 1.0+) when invoking `Jinja2Templates.TemplateResponse`,
resolving previous merge conflicts in the PR.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: CRITICAL
💡 Vulnerability: The generic file hashing utility `app/utils/file_operations.py:hash_file` was vulnerable to path traversal. An attacker controlling the `filepath` argument could read arbitrary files on the system by passing relative paths like `../../../etc/passwd` or providing absolute paths directly.
🎯 Impact: This could lead to Arbitrary File Read and potential information disclosure.
🔧 Fix: Used `pathlib.Path.resolve()` to resolve both the target file path and the allowed base directory (`settings.workdir`). Added a strict check to ensure the resolved target path is strictly within the allowed boundary using `filepath_obj.relative_to(workdir_obj)`, catching the `ValueError` raised when the path is out of bounds. This safely blocks both relative traversal attacks and arbitrary absolute paths, without breaking legitimate relative application paths.
✅ Verification: Ran the test suite `pytest tests/test_path_traversal_security.py -v` successfully, which explicitly checks for `FileNotFoundError` upon traversal attempts.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Modified `is_private_ip` in `app/utils/network.py` to fail securely by returning True (blocking the request) when a hostname cannot be resolved. The previous implementation failed open, creating a risk for Server-Side Request Forgery (SSRF) and DNS rebinding attacks.
Updated corresponding tests to expect the secure behavior and correctly appended the security finding to the Sentinel journal.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Renamed `save_dropbox_settings` inside `app/api/google_drive.py` to `save_google_drive_settings` to fix a copy-paste naming error.
- Extracted duplicate `.env` file updating logic from `app/api/google_drive.py`, `app/api/onedrive.py`, and `app/api/dropbox.py` into a new reusable helper function `update_env_file` inside `app/utils/settings_service.py`.
- Refactored the three API endpoints to use the new helper function, significantly reducing complexity and code duplication.
- Updated relevant test files (`tests/test_api_google_drive_final.py`) to reflect the new function name.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The transition to a newer FastAPI/Starlette version changed the signature of `Jinja2Templates.TemplateResponse` from `(name, context)` to `(request, name, context)`.
The `app/views/base.py:template_response_with_version` wrapper naively forwarded positional arguments `*args` to `original_template_response`. This caused the template name (`"files.html"`) to be passed as the `request` parameter, and the context dictionary to be passed as the `name` parameter. This resulted in Jinja2 attempting to cache the template using a dictionary as the cache key, which triggered a `TypeError: unhashable type: 'dict'`.
This commit updates the wrapper to automatically translate the legacy positional arguments `(name: str, context: dict)` into the explicit keyword arguments `request=context.get("request"), name=name, context=context` required by modern Starlette, preventing template rendering crashes across the application and restoring passing CI test suites.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The `preview_migration` function in `app/utils/db_migrate.py` used string
interpolation to dynamically execute a COUNT query on the source database
(`f"SELECT COUNT(*) FROM {quoted_table}"`).
While the table name was quoted via the dialect's identifier preparer and
validated with a regex, string interpolation for raw SQL should be avoided
as it represents an anti-pattern and a theoretical risk for SQL injection
if validation controls are ever bypassed or modified.
This commit replaces the raw string interpolation with safe, parameterized
SQLAlchemy Core query construction `select(func.count()).select_from(table(table_name))`,
which automatically handles table quoting and execution safely. It also removes
the unused `text` import to keep the code clean.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Modified `is_private_ip` in `app/utils/network.py` to fail securely by returning True (blocking the request) when a hostname cannot be resolved. The previous implementation failed open, creating a risk for Server-Side Request Forgery (SSRF) and DNS rebinding attacks.
Updated corresponding tests to expect the secure behavior.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract all OAuth registration into _setup_social_providers() with a
_register_oauth_client() helper that clears the authlib _clients cache
so credentials can change without a restart
- Add refresh_social_providers() public function called after every
settings reload (lifespan startup + settings_sync live reload)
- Fix connections page linked status to use _get_effective() (DB-aware)
instead of the stale startup-time SOCIAL_PROVIDERS dict
- Fix oauth_configured template variable similarly
- Add tests: DB-driven linked status, stale-provider clearing,
register_oauth_client cache-clear, refresh function coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ef15910f-fd25-469a-814b-9e1fb40659c9
- Add `qr_login_enabled` boolean config field (default True) to app/config.py
- Add `qr_login_enabled` entry to settings metadata in app/utils/settings_service.py
- Fix app/views/settings.py to use `qr_login_enabled` directly instead of
deriving enablement status from qr_login_challenge_ttl_seconds (integer TTL)
- Fix admin_connections.html: remove hardcoded `disabled` attribute from the
Mobile Phone Upload toggle and wire up onchange handler so toggling actually
persists the setting via toggleSetting('qr_login_enabled', this.checked)
- Gate all three QR auth API endpoints on settings.qr_login_enabled so the
feature is actually disabled when the toggle is turned off
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/8421cdb2-d92d-4b83-9cda-c44983d35173
- Add PUT /{key} endpoint to settings API with SettingValueUpdate body model (only
requires value, key comes from URL path) — fixes 405 Method Not Allowed errors
from the admin Connections wizard which used PUT to save settings
- Fix grey toggles on /admin/connections: they appeared grey because all saves were
silently failing with 405; now saves succeed and toggles reflect actual state
- Add social_auth_google_use_global_credentials config field and auth.py logic to
reuse google_drive_client_id/google_drive_client_secret for Google Sign-In
- Add social_auth_microsoft_use_global_credentials config field and auth.py logic to
reuse onedrive_client_id/onedrive_client_secret for Microsoft Sign-In
- Also apply consistent both-field check for Dropbox global credentials fallback
- Add settings metadata entries for the two new boolean settings
- Add Google and Microsoft settings_keys to admin_connections service definitions
- Add JS visibility toggle logic for Google/Microsoft credential fields in admin UI
- Add 6 new unit/integration tests for PUT endpoint and SettingValueUpdate model
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ac66041a-2cbd-4d90-8f8e-3588c629d4d8
google-labs-jules[bot]