Combined duplicated `event_hooks` keyword arguments into a single dictionary parameter with both `validate_redirect` and `verify_redirect` in `app/api/url_upload.py`. This fixes a `SyntaxError: keyword argument repeated: event_hooks` and ensures that all redirect validations run.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Combined duplicated `event_hooks` keyword arguments into a single dictionary parameter with both `validate_redirect` and `verify_redirect` in `app/api/url_upload.py`. This fixes a `SyntaxError: keyword argument repeated: event_hooks` and ensures that all redirect validations run.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: The `/process-url` endpoint used `httpx.AsyncClient` with `follow_redirects=True`. While the initial user-provided URL was validated against SSRF protections (blocking private/internal IPs), the client implicitly followed subsequent HTTP redirects without validating their target locations. This allowed an attacker to bypass the initial check by supplying a valid URL that redirected to an internal IP or cloud metadata endpoint.
🎯 Impact: An attacker could potentially access internal network services or cloud metadata endpoints.
🔧 Fix: Implemented an `event_hooks` listener (`validate_redirect`) on the `httpx.AsyncClient` that intercepts responses, extracts the `Location` header, resolves the absolute target URL, and applies the same `validate_url_safety` check before allowing the redirect to be followed.
✅ Verification: Ran `pytest tests/test_url_upload.py`, formatting checks via `ruff format` and linting via `ruff check`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Resolve merge conflicts with main (PR #834 also fixed S3 SSRF)
- Add isinstance(endpoint_url, str) type check before urlparse to prevent TypeError on non-string values
- Reject endpoint_url with empty/missing hostname after parsing (malformed URLs like 'https://')
- Keep scheme validation (http/https only) and private IP blocking via is_private_ip()
- Add logger.warning for SSRF block events
- Add regression tests: non-string endpoint_url and empty hostname cases
- Update sentinel.md with consolidated SSRF entry
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds missing unit tests for `_test_imap_connection` and `_test_s3_connection` to cover the new `is_private_ip()` SSRF blocking logic and satisfy Codecov checks.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds validation using `is_private_ip()` for user-provided hosts in `_test_imap_connection` and `_test_s3_connection` to prevent Server-Side Request Forgery vulnerabilities.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Restored mobile/ from d22175310a711e7ebdd8062ae29a54f0136dc3f6^
(parent commit d94e9ca4bc).
Commit d22175310a (google-labs-jules[bot], 2026-03-23T14:45:22Z) introduced
an SSRF security fix for IMAP connections but unintentionally deleted or
truncated a large number of files across the repository, including 24 files
under mobile/.
This commit targets only the mobile/ directory and restores the following
files to their pre-d2217531 state:
- mobile/README.md
- mobile/app.json
- mobile/app/(tabs)/_layout.tsx
- mobile/app/(tabs)/file-detail.tsx (re-added)
- mobile/app/+not-found.tsx (re-added)
- mobile/app/_layout.tsx
- mobile/eslint.config.js (re-added)
- mobile/package-lock.json
- mobile/package.json
- mobile/src/context/ShareContext.tsx
- mobile/src/i18n/de.json (re-added)
- mobile/src/i18n/en.json (re-added)
- mobile/src/i18n/es.json (re-added)
- mobile/src/i18n/fr.json (re-added)
- mobile/src/i18n/index.ts (re-added)
- mobile/src/i18n/it.json (re-added)
- mobile/src/screens/FileDetailScreen.tsx (re-added)
- mobile/src/screens/FilesScreen.tsx
- mobile/src/screens/LoginScreen.tsx
- mobile/src/screens/ProfileScreen.tsx
- mobile/src/screens/UploadScreen.tsx
- mobile/src/screens/WelcomeScreen.tsx
- mobile/src/services/api.ts
- mobile/src/utils/mimeTypes.ts (re-added)
- mobile/src/utils/normalizeUri.ts (re-added)
Security fixes introduced by d2217531 that are unrelated to mobile/
(IMAP SSRF fix in app/utils/network.py and app/tasks/imap_tasks.py)
are preserved — this restore targets only files under mobile/.
- Add _require_admin + AdminUser dependency to google_drive.py, dropbox.py, onedrive.py
and switch save-settings endpoints from @require_login to Depends(_require_admin) so
tests can use dependency_overrides to bypass auth
- Wrap lifespan shutdown section (logging.info + notify_shutdown) in try/except to
silence OSError and other exceptions during shutdown (test_lifespan_shutdown_*)
- Add @patch("app.tasks.imap_tasks.is_private_ip", return_value=False) to 5 IMAP
tests that use imap.example.com (unresolvable in CI, causing is_private_ip to return
True and pull_inbox to return early before any IMAP operations)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/06fb29ae-8e36-4f13-89b8-68c06378e8a6
The test_successful_conversion test asserts that '--' precedes the input/output
file paths in the ocrmypdf command as a security measure against argument
injection (file paths starting with '-' being interpreted as options).
The implementation was missing this separator, causing the test to fail and
triggering a downstream pytest INTERNALERROR (OSError: Bad file descriptor)
when pytest's terminal writer tried to report the failure.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/bd7ef195-4b30-456e-8118-5c200fb4bf45
Migration files 038-042 were accidentally deleted by commit d2217531
("Sentinel: Fix SSRF in IMAP connections"), which broke container
startup because existing databases had alembic_version stamped to
042_add_file_shares — a revision Alembic could no longer find.
Restored from the parent of that commit:
- 038_add_api_token_expires_at.py
- 039_add_classification_rules.py
- 040_add_automation_hooks.py
- 041_add_document_comments_and_annotations.py
- 042_add_file_shares.py
Alembic now resolves a clean single-head chain (001→042).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/f6165a49-2ec0-4158-9f1f-d508bb0489fe
Move the valid filename regex pattern to a shared constant in `app/utils/filename_utils.py` and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase.
Changes:
- Defined `VALID_FILENAME_PATTERN` and `VALID_FILENAME_RE` in `app/utils/filename_utils.py`.
- Updated `app/tasks/extract_metadata_with_gpt.py` to use `VALID_FILENAME_RE`.
- Updated `tests/test_path_traversal_security.py` to use `VALID_FILENAME_PATTERN`.
This refactoring addresses the duplication mentioned in the TODO in `tests/test_path_traversal_security.py`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- fix(api/dropbox): _require_admin bypasses auth when AUTH_ENABLED=False,
fixing all 5 TestSaveDropboxSettings failures
- fix(api/onedrive): same AUTH_ENABLED bypass in _require_admin; fix one-arg
update_env_file call using env_utils version for token rotation
- fix(auth): update login TemplateResponse to Starlette 1.0+ API
(request as first arg instead of in context dict)
- fix(api/local_auth): update all TemplateResponse calls to Starlette 1.0+ API
- fix(views/share): update TemplateResponse call to Starlette 1.0+ API
- fix(api/billing): update TemplateResponse call to Starlette 1.0+ API
- fix(tests/test_imap_tasks): mock is_private_ip for tests using
imap.example.com (unresolvable in sandboxed/CI environments)
- fix(tests): update TemplateResponse call_args assertions to new API
(call_args.kwargs['context'] instead of call_args[0][1])
- fix(tests): update fake_original signatures in dark_mode tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/52d7b7b7-3a71-4a96-b2b1-b675b8a6d3b4
- Resolve merge conflicts in app/api/onedrive.py and tests/test_api_google_drive_final.py
- Fix legacy Dict[str, str] type hints in update_env_file functions to use dict[str, str]
- Add admin-only access (_require_admin dependency) to save-settings endpoints
in google_drive.py, onedrive.py, and dropbox.py
- Fix in_memory_only response field to reflect actual env_write_success status
- Update tests to override _require_admin dependency for save-settings endpoint tests
- Resolve merge conflicts in .jules/sentinel.md and app/utils/network.py
- Refactor join_url() to use urllib.parse.urlsplit/urlunsplit and posixpath
instead of sentinel-string hack, preventing corruption for any input URL
- Fix test to use pytest tmp_path fixture instead of hard-coded /tmp/workdir
- Resolve add/add conflict in tests/test_api_saved_searches.py by keeping the improved HEAD version
- Resolve content conflict in tests/test_api_advanced_filters.py by keeping HEAD (no CRUD tests)
- Remove no-op test_get_user_id_branches (was just 'pass')
- Remove unused 'from fastapi import Request' import (fixes Ruff F401)
- Fix duplicate 'session = {}' assignment in MockRequest (fixes Ruff F811)
This commit safely handles the dynamic table names in database migration queries
by leveraging `sqlalchemy.select` and `sqlalchemy.table` in `app/utils/db_migrate.py`.
It addresses the `# noqa: S608` exception that was in place for string interpolation
SQL queries which are a known security anti-pattern.
Additionally, this commit includes the latest updates to `app/views/base.py`
from the `main` branch to handle backward compatibility across Starlette
versions (<1.0 vs 1.0+) when invoking `Jinja2Templates.TemplateResponse`,
resolving previous merge conflicts in the PR.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: CRITICAL
💡 Vulnerability: The generic file hashing utility `app/utils/file_operations.py:hash_file` was vulnerable to path traversal. An attacker controlling the `filepath` argument could read arbitrary files on the system by passing relative paths like `../../../etc/passwd` or providing absolute paths directly.
🎯 Impact: This could lead to Arbitrary File Read and potential information disclosure.
🔧 Fix: Used `pathlib.Path.resolve()` to resolve both the target file path and the allowed base directory (`settings.workdir`). Added a strict check to ensure the resolved target path is strictly within the allowed boundary using `filepath_obj.relative_to(workdir_obj)`, catching the `ValueError` raised when the path is out of bounds. This safely blocks both relative traversal attacks and arbitrary absolute paths, without breaking legitimate relative application paths.
✅ Verification: Ran the test suite `pytest tests/test_path_traversal_security.py -v` successfully, which explicitly checks for `FileNotFoundError` upon traversal attempts.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Extracted existing `TestSavedSearchesCRUD` from `tests/test_api_advanced_filters.py` into a dedicated `tests/test_api_saved_searches.py` file to better organize testing logic and reflect the application's file structure.
Significantly improved code coverage of `app/api/saved_searches.py` from 0% (missing configuration imports during tests) to 100% by testing previously untested edge cases including:
- Reaching the maximum saved search limit per user.
- Database commit errors (`HTTP_500_INTERNAL_SERVER_ERROR`) during create, update, and delete actions.
- Validation failures for `filters` field checking for non-dict types (`status.HTTP_422_UNPROCESSABLE_ENTITY`).
- Conflicting names during updates where an existing saved search matches the new name.
- Proper fallback logic across authentication methods for `_get_user_id`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Modified `is_private_ip` in `app/utils/network.py` to fail securely by returning True (blocking the request) when a hostname cannot be resolved. The previous implementation failed open, creating a risk for Server-Side Request Forgery (SSRF) and DNS rebinding attacks.
Updated corresponding tests to expect the secure behavior and correctly appended the security finding to the Sentinel journal.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Renamed `save_dropbox_settings` inside `app/api/google_drive.py` to `save_google_drive_settings` to fix a copy-paste naming error.
- Extracted duplicate `.env` file updating logic from `app/api/google_drive.py`, `app/api/onedrive.py`, and `app/api/dropbox.py` into a new reusable helper function `update_env_file` inside `app/utils/settings_service.py`.
- Refactored the three API endpoints to use the new helper function, significantly reducing complexity and code duplication.
- Updated relevant test files (`tests/test_api_google_drive_final.py`) to reflect the new function name.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The `preview_migration` function in `app/utils/db_migrate.py` used string
interpolation to dynamically execute a COUNT query on the source database
(`f"SELECT COUNT(*) FROM {quoted_table}"`).
While the table name was quoted via the dialect's identifier preparer and
validated with a regex, string interpolation for raw SQL should be avoided
as it represents an anti-pattern and a theoretical risk for SQL injection
if validation controls are ever bypassed or modified.
This commit replaces the raw string interpolation with safe, parameterized
SQLAlchemy Core query construction `select(func.count()).select_from(table(table_name))`,
which automatically handles table quoting and execution safely. It also removes
the unused `text` import to keep the code clean.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Renamed `save_dropbox_settings` inside `app/api/google_drive.py` to `save_google_drive_settings` to fix a copy-paste naming error.
- Extracted duplicate `.env` file updating logic from `app/api/google_drive.py`, `app/api/onedrive.py`, and `app/api/dropbox.py` into a new reusable helper function `update_env_file` inside `app/utils/settings_service.py`.
- Refactored the three API endpoints to use the new helper function, significantly reducing complexity and code duplication.
- Updated relevant test files (`tests/test_api_google_drive_final.py`) to reflect the new function name.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Move the valid filename regex pattern to a shared constant in `app/utils/filename_utils.py` and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase.
Normalized line endings in `app/tasks/extract_metadata_with_gpt.py` from CRLF to LF to ensure consistency and prevent CI issues.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The transition to a newer FastAPI/Starlette version changed the signature of `Jinja2Templates.TemplateResponse` from `(name, context)` to `(request, name, context)`.
The `app/views/base.py:template_response_with_version` wrapper naively forwarded positional arguments `*args` to `original_template_response`. This caused the template name (`"files.html"`) to be passed as the `request` parameter, and the context dictionary to be passed as the `name` parameter. This resulted in Jinja2 attempting to cache the template using a dictionary as the cache key, which triggered a `TypeError: unhashable type: 'dict'`.
This commit updates the wrapper to automatically translate the legacy positional arguments `(name: str, context: dict)` into the explicit keyword arguments `request=context.get("request"), name=name, context=context` required by modern Starlette, preventing template rendering crashes across the application and restoring passing CI test suites.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Move the valid filename regex pattern to a shared constant in `app/utils/filename_utils.py` and update both the task logic and security tests to use it. This eliminates duplication and ensures consistency across the codebase.
Also normalized line endings to LF in affected files to ensure CI compatibility.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added `mock_task.delay.assert_called_once_with(str(test_file))` to all integration tests involving background task enqueuing in `app/api/process.py` endpoints to ensure background tasks are called with the correct file path arguments.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds missing unit tests for the send_to_dropbox_endpoint in app/api/process.py, covering both success (queued) and error (file not found) states to ensure better robustness and API reliability.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The `preview_migration` function in `app/utils/db_migrate.py` used string
interpolation to dynamically execute a COUNT query on the source database
(`f"SELECT COUNT(*) FROM {quoted_table}"`).
While the table name was quoted via the dialect's identifier preparer and
validated with a regex, string interpolation for raw SQL should be avoided
as it represents an anti-pattern and a theoretical risk for SQL injection
if validation controls are ever bypassed or modified.
This commit replaces the raw string interpolation with safe, parameterized
SQLAlchemy Core query construction `select(func.count()).select_from(table(table_name))`,
which automatically handles table quoting and execution safely. It also removes
the unused `text` import to keep the code clean.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Modified `is_private_ip` in `app/utils/network.py` to fail securely by returning True (blocking the request) when a hostname cannot be resolved. The previous implementation failed open, creating a risk for Server-Side Request Forgery (SSRF) and DNS rebinding attacks.
Updated corresponding tests to expect the secure behavior.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The Tailwind CSS CLI is a devDependency in frontend/package.json. Using
`npm ci --omit=dev` skipped installing it, causing the Docker build to
fail with 'sh: tailwindcss: not found' at the `npm run build` step.
Since this is a multi-stage build where the frontend-builder stage is
discarded after compilation, all dependencies (including devDependencies)
are needed during the build but do not bloat the final image.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/f16fabb4-4d8c-48e3-8d26-c2d38cc7129f
- Extract all OAuth registration into _setup_social_providers() with a
_register_oauth_client() helper that clears the authlib _clients cache
so credentials can change without a restart
- Add refresh_social_providers() public function called after every
settings reload (lifespan startup + settings_sync live reload)
- Fix connections page linked status to use _get_effective() (DB-aware)
instead of the stale startup-time SOCIAL_PROVIDERS dict
- Fix oauth_configured template variable similarly
- Add tests: DB-driven linked status, stale-provider clearing,
register_oauth_client cache-clear, refresh function coverage
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ef15910f-fd25-469a-814b-9e1fb40659c9
The toggles used Tailwind CSS v3 JIT pseudo-element utilities
(after:content-[''], peer-checked:after:translate-x-full, etc.)
that are not available in Tailwind v2.2.19 CDN.
Added .doc-toggle / .doc-toggle-track CSS classes to styles.css
using native CSS ::after pseudo-elements and adjacent-sibling
selectors — works across all Tailwind versions and browsers.
Updated all three toggle instances in admin_connections.html
(SSO auto-login, QR login, and JS-created service settings toggles).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/64581c3a-6c34-4bb6-bb6b-6331945fed04
- Add `qr_login_enabled` boolean config field (default True) to app/config.py
- Add `qr_login_enabled` entry to settings metadata in app/utils/settings_service.py
- Fix app/views/settings.py to use `qr_login_enabled` directly instead of
deriving enablement status from qr_login_challenge_ttl_seconds (integer TTL)
- Fix admin_connections.html: remove hardcoded `disabled` attribute from the
Mobile Phone Upload toggle and wire up onchange handler so toggling actually
persists the setting via toggleSetting('qr_login_enabled', this.checked)
- Gate all three QR auth API endpoints on settings.qr_login_enabled so the
feature is actually disabled when the toggle is turned off
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/8421cdb2-d92d-4b83-9cda-c44983d35173
- Add PUT /{key} endpoint to settings API with SettingValueUpdate body model (only
requires value, key comes from URL path) — fixes 405 Method Not Allowed errors
from the admin Connections wizard which used PUT to save settings
- Fix grey toggles on /admin/connections: they appeared grey because all saves were
silently failing with 405; now saves succeed and toggles reflect actual state
- Add social_auth_google_use_global_credentials config field and auth.py logic to
reuse google_drive_client_id/google_drive_client_secret for Google Sign-In
- Add social_auth_microsoft_use_global_credentials config field and auth.py logic to
reuse onedrive_client_id/onedrive_client_secret for Microsoft Sign-In
- Also apply consistent both-field check for Dropbox global credentials fallback
- Add settings metadata entries for the two new boolean settings
- Add Google and Microsoft settings_keys to admin_connections service definitions
- Add JS visibility toggle logic for Google/Microsoft credential fields in admin UI
- Add 6 new unit/integration tests for PUT endpoint and SettingValueUpdate model
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/ac66041a-2cbd-4d90-8f8e-3588c629d4d8
- Add 3 new config fields for browser SDK sample rates:
sentry_js_traces_sample_rate (default 0.0),
sentry_js_replay_session_sample_rate (default 0.0),
sentry_js_replay_on_error_sample_rate (default 0.1)
- Expose Sentry config to Jinja2 templates via _inject_global_context;
empty-string DSN normalized to None so {% if sentry_dsn %} guard works
- Load Sentry Browser SDK bundle.tracing.replay.min.js from the official
Sentry CDN in base.html when SENTRY_DSN is configured, with Sentry.init()
for error capture, browser tracing and session replay
- Register new JS settings fields in SETTING_METADATA so they appear on the
admin Settings → Observability page
- Update .env.demo with commented-out examples for SENTRY_JS_* variables
- Update docs/ConfigurationGuide.md and docs/SentrySetup.md with full
browser SDK documentation, env-specific examples and troubleshooting
- Add TestSentryJsTemplateContext (5 tests) and TestSentryJsConfig (4 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/4a945567-67df-4264-aaed-75eb2e236e9c
- /files/<id> → new summary page with navigation cards
- /files/<id>/detail → document detail with metadata, preview, text
- /files/<id>/process → processing pipeline status and history
- /files/<id>/annotations → comments & annotations with EmbedPDF viewer
- /files/<id>/comments → redirects to /annotations
- Added embed-pdf-viewer as git submodule for PDF annotation viewer
- Updated all navigation links across templates
- Updated all tests to use new URL structure
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/12276514-bd3d-4e3e-84d9-5977d1f82b19
The `_test_webdav_connection` function previously used `urllib.request.urlopen`
to verify connection credentials. This triggers a Bandit B310 warning because
`urllib` supports multiple schemes (like file://, ftp://) and implicitly follows
redirects.
Although scheme checking and a basic `is_private_ip` validation were implemented,
using `urllib.request` remains risky because a public URL could return an
HTTP redirect to a private IP (e.g., 127.0.0.1) which `urllib` would blindly follow,
causing an SSRF (Server-Side Request Forgery) bypass.
This commit replaces `urllib.request` with `httpx.request` using explicitly
`follow_redirects=False`. This eliminates the B310 vulnerability, ensures
requests only hit the specified URL without following potentially malicious
redirects, and standardizes the application on `httpx` for safer HTTP connections.
In addition to fixing the vulnerability, test coverage is added for the
new WebDAV connections logic.
CI issues (missing imports / unformatted code) are resolved.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The `_test_webdav_connection` function previously used `urllib.request.urlopen`
to verify connection credentials. This triggers a Bandit B310 warning because
`urllib` supports multiple schemes (like file://, ftp://) and implicitly follows
redirects.
Although scheme checking and a basic `is_private_ip` validation were implemented,
using `urllib.request` remains risky because a public URL could return an
HTTP redirect to a private IP (e.g., 127.0.0.1) which `urllib` would blindly follow,
causing an SSRF (Server-Side Request Forgery) bypass.
This commit replaces `urllib.request` with `httpx.request` using explicitly
`follow_redirects=False`. This eliminates the B310 vulnerability, ensures
requests only hit the specified URL without following potentially malicious
redirects, and standardizes the application on `httpx` for safer HTTP connections.
In addition to fixing the vulnerability, test coverage is added for the
new WebDAV connections logic.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The `_test_webdav_connection` function previously used `urllib.request.urlopen`
to verify connection credentials. This triggers a Bandit B310 warning because
`urllib` supports multiple schemes (like file://, ftp://) and implicitly follows
redirects.
Although scheme checking and a basic `is_private_ip` validation were implemented,
using `urllib.request` remains risky because a public URL could return an
HTTP redirect to a private IP (e.g., 127.0.0.1) which `urllib` would blindly follow,
causing an SSRF (Server-Side Request Forgery) bypass.
This commit replaces `urllib.request` with `httpx.request` using explicitly
`follow_redirects=False`. This eliminates the B310 vulnerability, ensures
requests only hit the specified URL without following potentially malicious
redirects, and standardizes the application on `httpx` for safer HTTP connections.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add DocumentComment and DocumentAnnotation models to app/models.py
- Create migration 041_add_document_comments_and_annotations
- Add API endpoints for CRUD operations on comments and annotations
- Add threaded comment support with parent_id relationships
- Add @mention extraction from comment body text
- Add resolve/unresolve comment thread endpoint
- Add mentionable users endpoint (GET /api/users/mentionable)
- Add 43 unit tests covering all endpoints and edge cases
- Add 29 i18n translation keys to en.json
- Update API documentation in docs/API.md
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/3894af37-0f19-457b-8811-f1feb18b17ef
Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).
Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
(classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models
Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)
For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45
Added escapeHtml() utility function to both Dropbox and OneDrive
callback pages. Folder names, paths, and error messages inserted into
innerHTML via template literals are now escaped to prevent potential
cross-site scripting from malicious folder names.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Updated DropboxSetup.md, OneDriveSetup.md, and GoogleDriveSetup.md to
document the new system credentials toggle and folder browser features.
Added API documentation for POST /api/dropbox/list-folders and
POST /api/onedrive/list-folders endpoints.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added tests for POST /api/dropbox/list-folders (7 tests) and
POST /api/onedrive/list-folders (7 tests) covering success, subfolder
navigation, empty directories, auth errors, API errors, path
normalization, and alphabetical sorting.
Added view tests for system credentials toggle visibility in Dropbox
setup wizard.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added POST /api/dropbox/list-folders and POST /api/onedrive/list-folders
endpoints that accept an OAuth access_token and return folder listings.
After successful OAuth authorization in the callback pages, users now
see an interactive folder browser to select the target folder for their
integration. The selected folder is saved to the integration config.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When system-wide Dropbox, Google Drive, or OneDrive app credentials are
configured by the admin, user-mode OAuth wizards now default to using
them. A toggle lets users switch to custom credentials if needed. This
removes the need for end users to register their own cloud provider apps.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Watch folders using Dropbox, Google Drive, or OneDrive now show only the
folder path field and an info box directing users to use the Authorize
button after saving — matching the destination integration pattern.
Manual credential fields (refresh token, app key, app secret, etc.)
have been removed for these OAuth-backed source types.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add LocaleProvider + useLocale() hook with AsyncStorage persistence to mobile i18n
- Replace all hardcoded English strings in every screen with t() calls
- Add missing profile.settings/language keys to all 5 translation files (en/de/es/fr/it)
- Wrap app root in LocaleProvider; apply server preferred_language on login in AuthGuard
- Tab labels and header titles now re-render on language switch
- ProfileScreen: use useLocale() context, sync language to server via POST /api/i18n/language
- Backend: add preferred_language field to GET /api/mobile/whoami response
- Mobile API: add preferred_language to WhoAmIResponse type + setServerLanguage() method
- Tests: add test_whoami_returns_preferred_language and test_whoami_no_profile_preferred_language_is_null
- Docs: update MobileApp.md with language sync priority and whoami response format
Language priority: server preference > AsyncStorage > device locale > English fallback
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add PUBLIC_BASE_URL optional config to override auto-detected OAuth redirect URIs
when behind a reverse proxy that doesn't forward X-Forwarded-Proto headers
- Add _build_dropbox_redirect_uri() helper in app/api/dropbox.py
- URL-encode redirect_uri in server-side Dropbox authorization URL
- Add _get_dropbox_callback_url() helper in app/views/dropbox.py
- Pass callback_url to both setup and callback templates
- Update templates to use server-provided callback_url instead of window.location.origin
- Update settings_service.py to register new setting
- Update .env.demo, ConfigurationGuide.md, and DropboxSetup.md documentation
- Add tests for new helper functions and global-authorize-url endpoint
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Privacy Policy, Terms of Service, and Imprint links to WelcomeScreen
and LoginScreen for GDPR/Apple compliance (pre-login access)
- Enable multiple image selection in photo library picker
- Add HEIC/HEIF image support to backend (allowed_types, convert_to_pdf, upload handler)
- Create FileDetailScreen with processing status and logs
- Add search bar to FilesScreen with debounced search
- Set up i18n with expo-localization (EN, DE, ES, FR, IT)
- Add language selector to ProfileScreen settings
- Add Imprint link to ProfileScreen legal section
- Update docs and tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When a username like 'christianlouis.de' (containing a dot) was submitted
on the signup page, FastAPI returned a 422 with detail as an array of
Pydantic validation error objects. The JS code assigned that array directly
to `this.error`, causing Alpine.js x-text to render '[object Object]'.
Two fixes applied in signup.html:
1. Client-side validation: check username length and pattern in submit()
before the API call, with clear human-readable error messages.
2. Server error handling: detect when data.detail is an Array and extract
each entry's .msg field, joining them into a readable string.
Also adds a regression test to confirm the 422 response format for an
invalid username (with dot) includes a list detail with msg fields.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Address code review feedback:
- Extract normalizeFileUri to mobile/src/utils/normalizeUri.ts
- Import shared function in ShareContext and UploadScreen
- Move os import to top of test file
- Update test docstring to reflect new behavior
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Move duplicate check before task enqueue in ui_upload endpoint
- Clean up temp file and return status "duplicate" for exact duplicates
- Add URI-level dedup guard in mobile UploadScreen to prevent repeated uploads
- Improve ShareContext URI normalization (collapse slashes, decode percent-encoding)
- Guard +not-found.tsx effect against re-firing for the same pathname
- Update mobile UploadResponse type and handlers for duplicate status
- Update web frontend upload.js to show duplicate status
- Update API and Configuration docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract EXT_TO_MIME map and mimeTypeFromFilename to shared module
at mobile/src/utils/mimeTypes.ts (used by _layout.tsx and +not-found.tsx)
- Add error logging to ensureLocalUri catch block for debugging
- Add error handling to Linking.openURL calls in ProfileScreen
- Fix incorrect LSSupportsOpeningDocumentsInPlace docs in audit report
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Files shared via iOS Share Sheet / "Open In…" may reference paths
outside the app's sandbox or use security-scoped URLs that React
Native's fetch cannot read. This caused uploads to hang indefinitely
with a spinning indicator.
Fixes:
- Set LSSupportsOpeningDocumentsInPlace to false so iOS copies shared
files to the app's accessible Inbox directory
- Use expo-file-system to copy external file:// URIs to the app's
cache directory before uploading (ensureLocalUri helper)
- Apply ensureLocalUri to both initial uploads and retries
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Skip known deep-link paths (qr-login, callback) in makeUrlHandler
to prevent docuelevate://qr-login URLs from being treated as shared
files and creating phantom upload errors
- Infer MIME type from file extension for files shared via iOS Share
Sheet / "Open In…" so the server receives correct Content-Type
instead of application/octet-stream
- Default login screen server URL to https://app.docuelevate.org
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Show alert dialogs when Privacy Policy, Terms of Service, or
account deletion links cannot be opened due to missing server URL.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
When iOS delivers a file via "Open In…", expo-router strips the
docuelevate:// scheme and routes to +not-found.tsx. Previously, this
screen only redirected to the Upload tab and relied on the Linking
handler in _layout.tsx to add the file to ShareContext. This was
unreliable because expo-router may consume the URL event before the
Linking handler fires.
Now +not-found.tsx directly reconstructs the file:// URI from the
pathname and adds it to ShareContext before redirecting. ShareContext
deduplicates by URI to prevent double uploads if both mechanisms fire.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Widen page container from max-w-4xl (896px) to max-w-6xl (1152px)
- Convert action buttons (Revoke/Reactivate/Delete) to icon-only (44×44px)
with aria-label and title tooltip for accessibility
- Reduce table cell padding from px-6 py-4 to px-4 py-3
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge origin/main into branch, resolving conflict in app/database.py.
Combined improvements from both branches:
- Keep pool_pre_ping=True and structured variable approach from feature branch
- Add explicit QueuePool import and poolclass assignment from main
The QR code on /qr-login was not rendering because it depended on loading
qrcode@1.5.4 from the jsdelivr CDN, which may be blocked in some network
environments.
- Add segno>=1.6.0 (pure-Python QR library, no Pillow needed) to requirements.txt
- Generate QR code as a base64 SVG data URI server-side in the challenge endpoint
- Add qr_code_svg field to CreateChallengeResponse Pydantic model
- Replace canvas+CDN script in qr_login.html with an <img :src="qrCodeSvg">
- Remove the $nextTick/QRCode.toCanvas() client-side rendering block
- Extract QR rendering parameters (_QR_ERROR_LEVEL, _QR_SCALE) as module constants
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add db_pool_size, db_max_overflow, db_pool_timeout, db_pool_recycle fields to app/config.py
- Add upload_rate_limit_per_user, upload_rate_limit_window fields to app/config.py
- Update app/database.py to use NullPool for SQLite and QueuePool with config-driven
pool settings for PostgreSQL/MySQL
- Add all 6 settings to SETTING_METADATA in app/utils/settings_service.py
Fixes test_all_config_settings_have_metadata failure
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add missing SETTING_METADATA entries for db_pool_size, db_max_overflow,
db_pool_timeout, db_pool_recycle, upload_rate_limit_per_user, and
upload_rate_limit_window so the test_all_config_settings_have_metadata
test passes.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merges origin/main (v0.155.0) into the classification feature branch,
resolving all 18 conflicted files by accepting main's version and
re-applying only classification-specific additions:
- Renumber migration from 037 to 038 (chains from 037_user_sessions)
- Re-add ClassificationRuleModel to models.py, env.py, conftest.py
- Re-add classification_rules_router to api/__init__.py
- All session management, QR auth, and devices code preserved from main
Migration chain validated. 62 classification tests pass.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Override require_upload_rate_limit with a no-op in the test client
fixture so that upload-heavy test suites (test_file_upload.py) are not
rejected with 429 Too Many Requests when Redis is available in CI.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Introduces a Redis-backed sliding-window rate limiter for upload
endpoints (/api/ui-upload, /api/process-url) that:
- Enforces per-user limits (default: 20 uploads / 60 s)
- Dynamically reduces limits under system stress (queue depth, CPU load)
- Returns 429 with Retry-After header when exceeded
- Fails open when Redis is unavailable
- Works with the existing client-side adaptive back-off
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
SQLite engines now use NullPool instead of QueuePool, eliminating the
"QueuePool limit of size 5 overflow 10 reached" TimeoutError under
concurrent load. PostgreSQL/MySQL engines use a configurable QueuePool
with sensible defaults (pool_size=10, max_overflow=20) exposed via
DB_POOL_SIZE, DB_MAX_OVERFLOW, DB_POOL_TIMEOUT, DB_POOL_RECYCLE env
vars. pool_pre_ping is enabled on all backends.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The QR login page countdown timer compared the server's UTC expiration
timestamp against the client's local clock, causing the QR code to appear
immediately expired when the client clock was ahead of the server.
Changes:
- Add ttl_seconds field to CreateChallengeResponse (seconds until expiry)
- Frontend countdown now uses relative elapsed time since response was
received, eliminating clock-skew issues
- Mobile app: replace alert-only QR button with actual camera-based
QR code scanner using expo-camera
- Add QRScannerScreen with barcode scanning, permission handling, and
scan area overlay
- Update camera permission description to mention QR code scanning
- Add tests for ttl_seconds computation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Update DeploymentGuide.md with scaling instructions and beat service info
- Update KubernetesDeployment.md with unauthenticated probe paths and beat note
- Update ProductionReadiness.md with new health endpoints table and beat guidance
- Update API.md with new healthz/live and healthz/ready endpoint docs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add unauthenticated /api/diagnostic/healthz/live and /healthz/ready
probe endpoints for Kubernetes liveness/readiness checks
- Separate Celery Beat into dedicated beat service in docker-compose.yaml
- Remove container_name from api and worker services to allow scaling
- Create Helm beat-deployment.yaml for standalone Beat scheduler pod
- Remove -B flag from worker-deployment.yaml so workers can scale safely
- Add beat section and fix probe paths in Helm values.yaml
- Add tests for the new probe endpoints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Without a root app/index.tsx in the repo, a stale default Expo Router
scaffold file (showing "Hello World") could be picked up from a previous
build or CLI scaffolding and displayed instead of the real app.
The new index.tsx immediately redirects to /(auth)/, and the existing
AuthGuard in _layout.tsx forwards authenticated users to /(tabs)/.
Also registers the index screen in the root Stack and updates
docs/MobileApp.md with an expanded project structure and a new
troubleshooting entry.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add path traversal guard in reimport file copy loop
- Improve error log message context for table wipe failures
- Use conditional role=alert/status on result banner for accessibility
- Make test assertions more specific (exact status codes)
- Rename ambiguous view test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Merge origin/main into feature branch, resolving 3 conflicts:
- app/api/__init__.py: add classification_rules_router alongside new
routers from main (audit_logs, i18n, mobile, compliance, translation)
- app/models.py: keep ClassificationRuleModel alongside new models from
main (MobileDevice, ComplianceTemplate, PipelineRoutingRule)
- tests/conftest.py: import both ClassificationRuleModel and new models
from main (AuditLog, ComplianceTemplate)
Also renumber migration from 027 to 037 to chain from the latest
migration on main (036_add_document_translation_fields).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- 21 tests covering unit, integration, API, and view layers
- Update ConfigurationGuide.md with System Reset section
- Update API.md with system reset endpoint docs
- All tests pass, ruff clean
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add UserSession and QRLoginChallenge models for session tracking
and mobile QR authentication
- Add session_manager utility with create/validate/revoke/cleanup
functions and QR challenge helpers
- Add /api/sessions endpoints for listing, revoking, and
'log off everywhere' functionality
- Add /api/qr-auth endpoints for challenge creation, polling, and
claiming with API token issuance
- Add session config fields (lifetime, custom override, QR TTL)
- Update get_current_user to validate server-side sessions
- Create server-side sessions on all login paths (local, OAuth,
social, admin)
- Revoke server-side session on logout
- Configure SessionMiddleware max_age from session lifetime settings
- Graceful degradation: old sessions without _session_token continue
to work
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merge origin/main into copilot/add-sharepoint-integration.
All four conflicts were resolved by keeping both the SharePoint
additions (from this branch) and the iCloud additions (from main):
- app/models.py: added both SHAREPOINT and ICLOUD to IntegrationType
- app/tasks/send_to_all.py: added both to service_map and services list
- app/tasks/upload_to_user_integration.py: kept both upload handlers
- frontend/templates/files.html: added both filter options
No database migration conflicts — SharePoint does not require schema changes.
Extract APP_SCHEME_PREFIX constant for the custom URL scheme string,
and derive the photo library fallback filename extension from the
asset's MIME type instead of always using .jpg.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
On iOS the Share Sheet / "Open In" action may deliver the file path
under the app's custom docuelevate:// scheme instead of a file:// URL,
causing an "Unmatched Route" error. The URL handler now detects this
and rewrites the URL to file:// before processing.
Also adds a Photo Library button to the Upload screen so users can
select existing photos from their device library, not just capture
new ones with the camera.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Change `appVersionSource` from `"local"` to `"remote"` in `eas.json` so EAS
tracks build numbers on its servers. This ensures every CI build gets a
unique, ever-increasing version without needing to commit bumps back to
the repo — fixing the App Store Connect "bundle version already used"
rejection.
Also bump `ios.buildNumber` to "7" and `android.versionCode` to 7 in
`app.json` (above the previously uploaded version "6") so the remote
version initializes correctly.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The per-user notification functions (notify_user_document_processed /
notify_user_document_failed) were defined but never called from the
document processing pipeline.
- Call notify_user_document_processed in finalize_document_storage
when owner_id is available (creates in-app + email/webhook notifications)
- Add _dispatch_user_failure_notification helper to celery_app.py that
extracts file_id from failed task args and dispatches
notify_user_document_failed for document pipeline tasks
- Add comprehensive tests for both success and failure notification paths
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add paths filter so builds only trigger when mobile/ files change
- Add submit_ios job to automatically submit iOS builds to App Store Connect
- Update docs/MobileApp.md and mobile/README.md with CI/CD documentation
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- app.json: add CFBundleDocumentTypes to iOS infoPlist so the app
appears in the iOS Share Sheet; add ACTION_SEND/SEND_MULTIPLE
intentFilters for Android share intent support
- src/context/ShareContext.tsx (new): React context that queues files
received from the share sheet and delivers them to UploadScreen
- app/_layout.tsx: wrap in ShareProvider; add Linking handler
(makeUrlHandler factory + getInitialURL cold-start + addEventListener
warm-start) to capture file:// and content:// URLs
- src/services/api.ts: fix FileRecord interface (original_filename,
nested ProcessingStatus, mime_type); fix UploadResponse interface;
fix listFiles() (per_page param, unwrap data.files); add
getFileStatus(fileId) for single-file status polling
- src/screens/FilesScreen.tsx: use file.original_filename and
file.processing_status.status; fix statusEmoji to use actual backend
status values (completed/pending/duplicate)
- src/screens/UploadScreen.tsx: consume ShareContext for auto-upload of
shared files; add 5-second polling loop (search by filename → file_id
→ getFileStatus) to show real-time server processing status;
uploadFile wrapped in useCallback; proper effect dependency arrays
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Windows does not render regional indicator emoji as graphical flags,
showing raw text (DE, GB) instead. Replace all emoji flag values in
SUPPORTED_LANGUAGES with lowercase ISO 3166-1 alpha-2 country codes,
add the flag-icons@7.3.2 CSS library via CDN, and update templates to
render <span class="fi fi-{code}"> instead of emoji text.
Special cases:
- Welsh (cy): uses flag-icons region code "gb-wls"
- Catalan (ca): falls back to "es" (no dedicated ISO flag)
- Esperanto (eo): uses "un" (UN flag for international language)
<option> elements in profile.html no longer display flags since CSS
icon classes cannot be applied inside native <option> tags.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The tests were patching `app.api.url_upload.requests.get` but the module
uses `httpx.AsyncClient`. Updated 4 tests across 2 files to use the
correct `httpx.AsyncClient.stream` mock pattern with `AsyncMock`,
matching the existing working tests in test_url_upload.py.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Import translate_to_default_language in celery_worker.py
- Add default_document_language to SETTING_METADATA in settings_service.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- audit_logs.py: Remove _db_dep pattern that fails with latest FastAPI on
Python 3.11. Use clean DbSession = Annotated[Session, Depends(get_db)]
without default values.
- billing.py: Remove owner_id from log messages to fix CodeQL clear-text
logging of sensitive information alerts.
- files.py: Remove owner_id from log messages to fix CodeQL clear-text
logging of sensitive information alerts.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
get_current_owner_id() only checked the session for authenticated users.
When the mobile app sends Authorization: Bearer <token>, there is no
session cookie, so the Depends(_get_owner_id) dependency raised HTTP 401
before the @require_login wrapper could resolve the Bearer token.
The function now checks three sources in order:
1. Session user dict (existing behavior)
2. request.state.api_token_user (cached by require_login or prior call)
3. Direct Bearer token resolution via _resolve_bearer_user (new)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added a new test file `tests/test_api_saved_searches.py` containing a comprehensive test suite for the CRUD operations provided in `app/api/saved_searches.py`. The suite validates happy paths, error conditions (like missing filters, name limits, duplicates), and user isolation using an in-memory SQLite database. Fixed Ruff formatting error that caused the CI pipeline to fail in the previous commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added a new test file `tests/test_api_saved_searches.py` containing a comprehensive test suite for the CRUD operations provided in `app/api/saved_searches.py`. The suite validates happy paths, error conditions (like missing filters, name limits, duplicates), and user isolation using an in-memory SQLite database.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by deriving the extension from original_filename and filtering out all non-alphanumerics.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by deriving the extension from original_filename and filtering out all non-alphanumerics.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Remove _db_dep singleton and its use as default value in function signatures
- Use DbSession = Annotated[Session, Depends(get_db)] directly (matches files.py pattern)
- Declare db: DbSession without a default (FastAPI DI provides the session)
- Delete 10 experimental test_*.py files left at repo root from B008 debugging
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by deriving the extension from original_filename and filtering out all non-alphanumerics.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaces the synchronous `requests.post` and `requests.get` calls in
`app/api/onedrive.py:test_onedrive_token` with an asynchronous
`httpx.AsyncClient` implementation. This unblocks the FastAPI event loop
when this endpoint is hit.
Fixed unused requests import in `app/api/onedrive.py` and sorted imports in the testing files updated previously to adhere to the repository formatting (`ruff check --fix`). Tests in `test_api_onedrive_extended.py` were also migrated to use AsyncMock properly.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add LOG_FORMAT setting (text/json) for structured JSON log output
- Add LOG_SYSLOG_* settings for direct syslog forwarding of app logs
- JSON format compatible with Grafana Loki, Splunk, ELK, Datadog
- Syslog forwarding uses Python's SysLogHandler (UDP/TCP)
- Update .env.demo and ConfigurationGuide.md with all new settings
- Add tests for JSON formatter and syslog config fields
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths, and filtering out non-alphanumerics from the file extension.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaces the synchronous `requests.post` and `requests.get` calls in
`app/api/onedrive.py:test_onedrive_token` with an asynchronous
`httpx.AsyncClient` implementation. This unblocks the FastAPI event loop
when this endpoint is hit.
Fixed unused requests import in `app/api/onedrive.py` and sorted imports in the testing files updated previously to adhere to the repository formatting (`ruff check --fix`).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
While `_ensure_indexes` was already secured, the `CREATE INDEX` for
`ix_saved_searches_user_id` was hardcoded. This commit explicitly
quotes it to unify our security posture against SQL injection
and keep static analyzers happy.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit improves test coverage for the `register_settings_reload_signal` function in `app/utils/settings_sync.py`.
🎯 **What:** The testing gap addressed was that the `_reload_if_stale` inner Celery `task_prerun` signal handler was entirely untested, specifically around exception handling (e.g. Redis timeouts or OCR manager errors) and the code branch where Redis returns no version key.
📊 **Coverage:** The following scenarios are now tested:
- Redis returning `None` for the version.
- Redis throwing an exception (handled gracefully).
- `ensure_ocr_languages_async` throwing an exception (caught and logged without failing the task).
✨ **Result:** Test coverage for `register_settings_reload_signal` is now 100%. Total coverage for `app/utils/settings_sync.py` has been substantially improved.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add `log_level` setting to config.py (default: INFO, env: LOG_LEVEL)
- Configure Python root logger in main.py with standard precedence:
LOG_LEVEL explicit > DEBUG=true implies DEBUG > default INFO
- Add timestamp to log format for production readability
- Suppress noisy third-party loggers at DEBUG level
- Add comprehensive debug logging to all auth functions
- Add LOG_LEVEL/DEBUG to .env.demo and ConfigurationGuide.md
- Add tests for logging config and auth debug output
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaces the synchronous `requests.post` and `requests.get` calls in
`app/api/onedrive.py:test_onedrive_token` with an asynchronous
`httpx.AsyncClient` implementation. This unblocks the FastAPI event loop
when this endpoint is hit.
Tests were updated to mock `httpx.AsyncClient` and a sync wrapper using `asyncio.run` was added to integration tests to maintain test coverage without massive test refactoring.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added a strict regex validation allowlist for table names in `preview_migration` before using them in raw SQL queries. This ensures that only alphanumeric characters and underscores are allowed, preventing potential SQL injection even if the source of table names were to be manipulated. Formatted code with ruff format.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaces the synchronous `requests.post` and `requests.get` calls in
`app/api/onedrive.py:test_onedrive_token` with an asynchronous
`httpx.AsyncClient` implementation. This unblocks the FastAPI event loop
when this endpoint is hit.
Tests were updated to mock `httpx.AsyncClient` and a sync wrapper using `asyncio.run` was added to integration tests to maintain test coverage without massive test refactoring.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The previous commit introduced a benchmark test with unsorted imports inside the test method, which caused the Ruff Lint & Format CI check to fail with `I001 [*] Import block is un-sorted or un-formatted`. This commit runs `ruff format` and `ruff check --fix` on `tests/test_notifications_api.py` to fix the issue.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added a benchmark script in tests/test_notifications_api.py that proved the N+1 issue issue.
- Replaced iterative DB lookups inside `for item in body.preferences:` with single pre-fetch query and local `prefs_dict` lookups.
- Verified test benchmark time drops from ~0.0964s to ~0.0141s for a batch of 100 items.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced the synchronous `requests.post` calls in `app/api/dropbox.py` with asynchronous `httpx.AsyncClient().post` calls. This ensures that the FastAPI event loop is not blocked during network I/O, allowing better concurrent performance.
Also updated the `test_api_dropbox.py` tests to use `httpx.AsyncClient.post` in mocks and properly construct `httpx.RequestError` in exception handling tests.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced the loop over `body.order` which generated an N+1 issue
with a single bulk query fetching all relevant `SubscriptionPlan`
records via the `.in_()` clause.
Added an in-memory dictionary map of `plan_id` to `SubscriptionPlan`
objects to allow `O(1)` lookups while updating the order.
Benchmark speedup: 14.71x faster on 500 records.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added tests to `tests/test_api_advanced_filters.py` to cover missing edge cases and error handling for the `PUT /api/saved-searches/{id}` endpoint. New test coverage includes duplicate name conflicts (409), validation errors for names exceeding max length (422), empty names (422), empty filters (422), and payloads containing only invalid filter keys (422).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced the loop over duplicate hashes that resulted in O(N) database queries
per page with a single efficient `in_` batch query to retrieve both originals
and duplicates. The records are then grouped in memory using dictionaries.
This resolves the N+1 performance bottleneck and reduces response time from
an average of 1.65 seconds to ~0.45 seconds locally for 500 groups.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds unit tests for the notify_settings_updated function in
app/utils/settings_sync.py to verify that exceptions during Redis publish,
settings reload, and OCR language check are properly caught and logged as
warnings without raising up the call stack.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced the N+1 query in `list_shared_links` which fetched `FileRecord` for each link. It now uses a single query with an `outerjoin` to fetch `original_filename` alongside the `SharedLink` object.
Measured a significant improvement from ~0.4547s to ~0.0579s per 1000 links.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added a strict regex validation allowlist for table names in `preview_migration` before using them in raw SQL queries. This ensures that only alphanumeric characters and underscores are allowed, preventing potential SQL injection even if the source of table names were to be manipulated.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Enhance the coverage and robustness of the `generate_api_token` helper
in `app/api/api_tokens.py` by introducing three unit tests.
The new tests verify:
- The exact character length of the generated string based on `TOKEN_BYTES`.
- The character set strictly adheres to URL-safe characters and the expected `TOKEN_PREFIX`.
- `secrets.token_urlsafe` is explicitly called with `TOKEN_BYTES`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds `--` separator to `rclone copy`, `mkdir`, and `link` commands in `upload_with_rclone.py`. This explicitly tells rclone to stop processing options and treat subsequent arguments strictly as positional arguments, preventing malicious user-controlled paths (starting with `-`) from being executed as arbitrary command flags.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds a specific unit test `test_hash_token_known_value` to `tests/test_api_tokens.py` to assert that the `hash_token` pure function accurately computes the expected PBKDF2 digest for a known input string. This provides a hard check against any accidental regressions to the cryptographic hashing logic, iteration counts, or salt values used.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Extracted file chunk saving and duplicate detection logic into separate helper functions (`_save_upload_file_chunks` and `_check_for_exact_duplicate`) to improve readability and maintainability of the `ui_upload` endpoint in `app/api/files.py`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The function handling the `/google-drive/save-settings` endpoint was incorrectly named `save_dropbox_settings`, likely due to a copy-paste error. This commits renames it to `save_google_drive_settings` and updates all the tests referencing it.
Tested using standard procedures, although test execution resulted in missing dependency errors due to lack of network access in the environment.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Adds test coverage for the 500 Internal Server Error path when deleting
a saved search fails due to a database error. The 404 path was already
covered, so this brings full coverage to the deletion error handling in
app/api/saved_searches.py.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Added the `--` argument before positional arguments in rclone subprocess calls (link, mkdir, copy) in `app/tasks/upload_with_rclone.py`. This ensures that filenames or destinations starting with a hyphen are treated as paths rather than unintended command-line flags.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Refactor `app/api/audit_logs.py` to use the `Annotated` type hint pattern while maintaining default values for dependencies using module-level singletons.
- Resolves B008: Function-call in default argument.
- Maintains compatibility with decorators (e.g., `@require_login`) that call the function without explicitly providing the `db` argument.
- Uses standard FastAPI patterns for query parameters with constant defaults.
- No changes to API runtime behavior.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add Microsoft SharePoint Online as a storage provider using the
Microsoft Graph API. Includes OAuth2 authentication via MSAL,
site/drive resolution, chunked upload sessions, and metadata sync.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Sanitise uploaded filenames with os.path.basename() to prevent path traversal
- Change TestWebhookDispatchIntegration marker from unit to integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add REST hooks subscription endpoints, incoming action endpoints, and
Zapier-compatible flat payload format for automation platform integration.
- AutomationHook model for webhook subscriptions
- POST /api/automation/hooks/subscribe and DELETE /hooks/{id}
- GET /api/automation/triggers/sample/{event} for Zapier field mapping
- POST /api/automation/actions/upload for incoming document uploads
- Celery task with retry for async hook delivery
- Integration with existing webhook dispatch flow
- 30 passing tests covering all new functionality
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Implements the classify pipeline step with:
- Classification rules engine (app/utils/classification_rules.py) with
pre-built categories (invoice, contract, receipt, letter, report,
bank_statement, tax_document, insurance, payslip) and support for
filename patterns, content keywords, and metadata matching rules
- Celery task (app/tasks/classify_document.py) that runs as a pipeline step
- CRUD API (app/api/classification_rules.py) for managing custom rules
- ClassificationRuleModel in app/models.py with migration 027
- Updated pipeline step config_schema and stage mapping
- Comprehensive tests for engine, API, and task
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
## 2026-03-20 - Safe Path Traversal Prevention in Low-Level Utilities
**Vulnerability:** The generic file utility `hash_file` in `app/utils/file_operations.py` accepted any file path and was vulnerable to reading arbitrary files via path traversal (e.g., `../../../etc/passwd`) or absolute paths if an attacker could control the `filepath` argument.
**Learning:**Naively checking for `".." in path` breaks legitimate relative paths used internally by the application. Blocking absolute paths entirely also breaks functionality. Input validation should occur at the API boundary, but for defense-in-depth, low-level utilities must enforce expected boundaries (e.g., the application's `workdir`).
**Prevention:**Use `pathlib.Path.resolve()` on both the target path and the allowed base directory (`settings.workdir`). Ensure the resolved target path is strictly within the allowed boundary using `filepath_obj.relative_to(workdir_obj)`, catching the `ValueError` that is raised when the path is out of bounds. This safely blocks both relative traversal attacks and arbitrary absolute paths.
## 2025-05-18 - [SSRF Bypass via DNS Resolution Failure]
**Vulnerability:** The `is_private_ip` function in `app/utils/network.py` failed open (returned `False`) when a hostname could not be resolved (`socket.gaierror`).
**Learning:** This fail-open pattern was originally added to allow external domains in tests, but in production, it created a severe SSRF risk. An attacker could bypass SSRF protections by providing a URL that fails to resolve during the security check but resolves later (DNS rebinding), or by exploiting internal routing behaviors via unresolvable addresses.
**Prevention:** Always fail securely in network authorization functions. If a domain cannot be resolved to verify its safety, the request must be blocked (`return True` / default-deny). Tests should mock DNS resolution correctly instead of compromising production security logic.
## 2026-06-01 - [Fix XSS in status_dashboard.html]
**Vulnerability:** A Cross-Site Scripting (XSS) vulnerability existed in `frontend/templates/status_dashboard.html` where untrusted configuration settings (`value`), external service messages (`data.message`), and token expirations (`data.token_info.expires_in_human`) were injected directly into the DOM via `.innerHTML` without sanitization.
**Learning:**Even internal or admin-focused dashboards can be vulnerable if they display external or user-configurable data without escaping. Constructing HTML strings dynamically from unvalidated sources is a common vector for DOM-based XSS.
**Prevention:**Always use a sanitization function like `escapeHtml` to escape dangerous characters (`<`, `>`, `&`, `"`, `'`) before assigning dynamic content to `.innerHTML`, or prefer `.textContent` when only plaintext is intended.
-**Database-backed settings management with encryption**
-**Setup wizard for first-time configuration**
-**Admin UI for runtime configuration**
-**Automated semantic versioning and releases**
- OAuth2 authentication with admin group support
- Basic web UI and REST API
DocuElevate ships continuously via automated semantic versioning. Use **GitHub Releases** for the latest build artifacts and **GitHub Milestones** (below) for roadmap tracking.
### Last Shipped Milestone: v0.5.0 (Released February 8, 2026)
- Database-backed settings management with encryption
- Setup wizard for first-time configuration
- Admin UI for runtime configuration
-Release automation via semantic-release
### Important Note on Versioning
As of February 2026, DocuElevate uses **automated semantic versioning**:
@@ -130,87 +128,78 @@ As of February 2026, DocuElevate uses **automated semantic versioning**:
@@ -35,7 +35,7 @@ DocuElevate is an intelligent document processing system that automates the inge
- **AI-Powered Metadata Extraction** — pluggable AI providers including OpenAI, Anthropic Claude, Google Gemini, Ollama (local), OpenRouter, Portkey, and Azure OpenAI via LiteLLM
- **Multi-Engine OCR** — Azure Document Intelligence, Tesseract, EasyOCR, Mistral OCR, Google Cloud Document AI, and AWS Textract with configurable merge strategies
DocuElevate aims to be the premier open-source intelligent document processing platform, providing seamless integration with cloud storage providers, advanced AI-powered metadata extraction, and enterprise-grade security and scalability.
## How to Read This Roadmap
DocuElevate ships frequently (automated semantic versioning), so this roadmap is organized around **milestone outcomes** and **themes**, not exact build numbers.
- **P0** = required for the milestone to feel “done”
- **P1** = strongly desired; may slip if needed
- **P2** = nice-to-have / opportunistic
For the detailed milestone breakdown and target dates, see [MILESTONES.md](MILESTONES.md).
## Release Naming
Each major milestone release carries a codename to anchor key project moments. These names appear in the status dashboard, build metadata, and changelog. For details, see [docs/ReleaseNaming.md](docs/ReleaseNaming.md).
- **P1:** SCIM provisioning, SAML/Okta/Azure AD hardening, quotas/billing at org level
- **P2:** multi-region deployment patterns and disaster recovery playbooks
### Features - v0.5.0 "Foundation"
- **Workflow Automation** → _evolving into v0.7.0 "Conductor"_
- [ ] Custom processing pipelines
- [ ] Conditional routing based on document type
- [ ] Scheduled batch processing
- [ ] Webhook support for external integrations
- [ ] Rule-based document classification
## Release Plan (Extended)
- **Advanced AI Features**
- [ ] Custom AI models for specialized document types
- [ ] Multi-language OCR support
- [ ] Document similarity detection
- [ ] Automatic duplicate detection
- [ ] Intelligent document splitting
This plan extends the existing milestones with a clearer thematic arc and a forward-looking “beyond v2.0” horizon. Each milestone links to an epic issue that owns scope and sub-issues.
**Status:** Bandit Security Scan Completed - All Critical/High/Medium Issues Resolved
## Executive Summary
@@ -9,15 +9,6 @@ This document tracks security vulnerabilities found in DocuElevate and their rem
## Recent Security Fixes
### Insecure API Endpoint Exposing Integration Credentials ✅ FIXED (2026-03-23)
**Severity:** HIGH
**Issue:** The endpoint `GET /api/integrations/{integration_id}/credentials` exposed integration credentials (e.g. passwords, API keys) in plaintext over the API. Although requiring login, this allowed anyone with an active user session to extract the raw credentials. The frontend used this endpoint for testing integration connections.
**Remediation:**
- Removed the `/credentials` endpoint entirely.
- Added a new `POST /api/integrations/{integration_id}/test` endpoint that securely runs connection tests server-side without returning the decrypted credentials to the client.
This guide explains how to configure DocuElevate to create Evernote notes for processed documents.
## Overview
The Evernote destination creates one note per processed document. The note contains:
- A visible metadata section populated from DocuElevate's extracted metadata JSON
- The processed PDF attached as an Evernote resource
- Optional tags from `EVERNOTE_DEFAULT_TAGS` plus extracted document tags
## Required Configuration
| Variable | Description |
|----------|-------------|
| `EVERNOTE_ENABLED` | Set to `false` to disable Evernote uploads without removing credentials. Default: `true` |
| `EVERNOTE_AUTH_TOKEN` | Evernote developer token or OAuth access token with note creation permissions |
## Optional Configuration
| Variable | Description |
|----------|-------------|
| `EVERNOTE_SANDBOX` | Use Evernote sandbox API endpoints. Default: `false` |
| `EVERNOTE_NOTEBOOK_GUID` | Target notebook GUID. If omitted, Evernote uses the account default notebook |
| `EVERNOTE_DEFAULT_TAGS` | Comma-separated tags to apply to every created note, for example `docuelevate,archive` |
| `EVERNOTE_INCLUDE_METADATA` | Include extracted metadata in the note body. Default: `true` |
## Example
```dotenv
EVERNOTE_ENABLED=true
EVERNOTE_AUTH_TOKEN=your-evernote-token
EVERNOTE_NOTEBOOK_GUID=optional-notebook-guid
EVERNOTE_DEFAULT_TAGS=docuelevate,processed
EVERNOTE_INCLUDE_METADATA=true
```
## Metadata and Attachments
DocuElevate reads the companion metadata file next to the processed PDF, for example `invoice.pdf` and `invoice.json`. Non-empty metadata fields are rendered into the Evernote note body. Values such as `Unknown`, empty strings, and null values are skipped.
If the metadata contains a `tags` field, those tags are applied to the note together with any tags configured in `EVERNOTE_DEFAULT_TAGS`.
The processed PDF is attached directly to the note using Evernote's resource model, so it appears as a normal Evernote attachment.
## Notes
- Evernote tokens can expire or be revoked. If uploads start failing with authentication errors, generate or refresh the token and update `EVERNOTE_AUTH_TOKEN`.
- If `EVERNOTE_NOTEBOOK_GUID` points to a missing or inaccessible notebook, Evernote will reject the note creation request.
- Evernote enforces account upload quotas and per-note size limits. Large PDFs may fail if they exceed those limits.
# Should complete without raising despite the notify_shutdown error
asyncwithlifespan(app):
pass
@pytest.mark.unit
classTestExceptionHandlers:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
github-actions[bot]