Compare commits

...

35 Commits

Author SHA1 Message Date
github-actions[bot] 8e744c076d chore(release): update build metadata files [skip ci] 2026-05-17 14:20:12 +00:00
semantic-release cc494c6937 0.172.12
Automatically generated by python-semantic-release
2026-05-17 14:20:08 +00:00
Christian Krakau-Louis e2fa96318f 🛡️ Sentinel: [HIGH] Fix SSRF in webhook delivery (#846)
* fix: validate webhook targets before delivery

* test: cover webhook SSRF validation
2026-05-17 16:19:41 +02:00
github-actions[bot] 416c3c4758 chore(release): update build metadata files [skip ci] 2026-05-17 14:07:02 +00:00
semantic-release 1c7ef28c5d 0.172.11
Automatically generated by python-semantic-release
2026-05-17 14:06:59 +00:00
Christian Krakau-Louis 1a0218799b fix: escape search result template values (#853) 2026-05-17 16:06:39 +02:00
github-actions[bot] 048f28a671 docs(changelog): update changelog [skip ci] 2026-05-17 13:45:16 +00:00
dependabot[bot] b818f07c80 Update setuptools build requirement (#847)
Co-authored-by: Christian Krakau-Louis <christianlouis@gmail.com>
2026-05-17 15:44:52 +02:00
github-actions[bot] 06507ed8bf chore(release): update build metadata files [skip ci] 2026-05-17 12:40:24 +00:00
semantic-release 044a9a86d6 0.172.10
Automatically generated by python-semantic-release
2026-05-17 12:40:21 +00:00
Christian Krakau-Louis 62d4ca6367 Merge pull request #860 from christianlouis/fix-httpx-asyncclient-syntax-error-15225710494108486471
Fix SyntaxError caused by duplicate `event_hooks` in `httpx.AsyncClient` instantiation
2026-05-17 14:39:59 +02:00
Christian Krakau-Louis 871f788f0b fix(url-upload): handle unsafe redirects as client errors 2026-05-17 13:12:55 +02:00
google-labs-jules[bot] 18f5596b01 Fix SyntaxError caused by duplicate event_hooks in httpx.AsyncClient instantiation
Combined duplicated `event_hooks` keyword arguments into a single dictionary parameter with both `validate_redirect` and `verify_redirect` in `app/api/url_upload.py`. This fixes a `SyntaxError: keyword argument repeated: event_hooks` and ensures that all redirect validations run.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-05-17 03:01:11 +00:00
google-labs-jules[bot] afb8b367ee Fix SyntaxError caused by duplicate event_hooks in httpx.AsyncClient instantiation
Combined duplicated `event_hooks` keyword arguments into a single dictionary parameter with both `validate_redirect` and `verify_redirect` in `app/api/url_upload.py`. This fixes a `SyntaxError: keyword argument repeated: event_hooks` and ensures that all redirect validations run.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-05-17 02:58:26 +00:00
github-actions[bot] 58b14ae769 docs(changelog): update changelog [skip ci] 2026-04-07 09:35:53 +00:00
Christian Krakau-Louis 23c5bac666 Merge pull request #838 from christianlouis/dependabot/pip/pip-licenses-5.5.5
Bump pip-licenses from 5.5.1 to 5.5.5
2026-04-07 11:35:15 +02:00
Christian Krakau-Louis d925dc5cd3 Merge pull request #839 from christianlouis/dependabot/pip/stripe-gte-7.0.0-and-lt-16.0.0
Update stripe requirement from <15.0.0,>=7.0.0 to >=7.0.0,<16.0.0
2026-04-07 11:35:04 +02:00
github-actions[bot] b8ddd2f8d2 chore(release): update build metadata files [skip ci] 2026-04-07 09:34:57 +00:00
semantic-release 301ca9d186 0.172.9
Automatically generated by python-semantic-release
2026-04-07 09:34:54 +00:00
Christian Krakau-Louis 3bd8a52ea2 Merge pull request #836 from christianlouis/sentinel/fix-s3-ssrf-vulnerability-8738702433604673313
🛡️ Sentinel: [HIGH] Fix SSRF in S3 connection test endpoint_url
2026-04-07 11:34:28 +02:00
Christian Krakau-Louis 789e8c6236 Merge pull request #840 from christianlouis/sentinel/ssrf-redirect-bypass-15997970627137004397
🛡️ Sentinel: [HIGH] Fix SSRF bypass via HTTP redirects in url_upload
2026-04-07 11:34:11 +02:00
Christian Krakau-Louis a3ea215a1c Merge branch 'main' into sentinel/ssrf-redirect-bypass-15997970627137004397 2026-04-07 11:33:59 +02:00
github-actions[bot] c6e0b80bec docs(changelog): update changelog [skip ci] 2026-04-07 09:33:42 +00:00
Christian Krakau-Louis e86e1b9f13 Merge pull request #845 from christianlouis/sentinel-ssrf-httpx-redirect-fix-3175117423198883580
🛡️ Sentinel: [HIGH] Fix SSRF bypass via httpx redirects in /process-url
2026-04-07 11:33:11 +02:00
google-labs-jules[bot] 46a9a30af0 🛡️ Sentinel: [HIGH] Fix SSRF bypass via httpx redirects
🚨 Severity: HIGH
💡 Vulnerability: The `/process-url` endpoint used `httpx.AsyncClient` with `follow_redirects=True`. While the initial user-provided URL was validated against SSRF protections (blocking private/internal IPs), the client implicitly followed subsequent HTTP redirects without validating their target locations. This allowed an attacker to bypass the initial check by supplying a valid URL that redirected to an internal IP or cloud metadata endpoint.
🎯 Impact: An attacker could potentially access internal network services or cloud metadata endpoints.
🔧 Fix: Implemented an `event_hooks` listener (`validate_redirect`) on the `httpx.AsyncClient` that intercepts responses, extracts the `Location` header, resolves the absolute target URL, and applies the same `validate_url_safety` check before allowing the redirect to be followed.
 Verification: Ran `pytest tests/test_url_upload.py`, formatting checks via `ruff format` and linting via `ruff check`.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-04-06 02:55:58 +00:00
google-labs-jules[bot] bdfa3ba1e0 style: sort imports in test_url_upload.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-30 03:31:23 +00:00
github-actions[bot] 8295279ec9 style: apply ruff auto-fix
- Auto-formatted code with ruff format
- Applied ruff linting fixes with --fix

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-30 03:24:20 +00:00
google-labs-jules[bot] 152ee15b06 test: add coverage for url_upload redirect SSRF bypass prevention hook
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-30 03:23:56 +00:00
google-labs-jules[bot] a75e8b9297 🛡️ Sentinel: [HIGH] Fix SSRF bypass via HTTP redirects in url_upload
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-30 03:02:05 +00:00
dependabot[bot] ee664f83fb Update stripe requirement from <15.0.0,>=7.0.0 to >=7.0.0,<16.0.0
Updates the requirements on [stripe](https://github.com/stripe/stripe-python) to permit the latest version.
- [Release notes](https://github.com/stripe/stripe-python/releases)
- [Changelog](https://github.com/stripe/stripe-python/blob/master/CHANGELOG.md)
- [Commits](https://github.com/stripe/stripe-python/compare/v7.0.0...v15.0.0)

---
updated-dependencies:
- dependency-name: stripe
  dependency-version: 15.0.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-29 23:04:45 +00:00
dependabot[bot] 91ef089aa7 Bump pip-licenses from 5.5.1 to 5.5.5
Bumps [pip-licenses](https://github.com/raimon49/pip-licenses) from 5.5.1 to 5.5.5.
- [Release notes](https://github.com/raimon49/pip-licenses/releases)
- [Changelog](https://github.com/raimon49/pip-licenses/blob/master/CHANGELOG.md)
- [Commits](https://github.com/raimon49/pip-licenses/compare/v-5.5.1...v-5.5.5)

---
updated-dependencies:
- dependency-name: pip-licenses
  dependency-version: 5.5.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-29 23:04:35 +00:00
google-labs-jules[bot] 925864ddca Close as obsolete
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-27 14:51:28 +00:00
copilot-swe-agent[bot] 57db4c7c82 fix(api): resolve merge conflicts, add type safety for endpoint_url in S3 connection test
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/8a9f717e-a6cb-45f0-8f2a-0e5d1d404657

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-27 14:50:51 +00:00
copilot-swe-agent[bot] 35752c9092 fix(api): resolve merge conflicts, add type safety for endpoint_url in S3 connection test
- Resolve merge conflicts with main (PR #834 also fixed S3 SSRF)
- Add isinstance(endpoint_url, str) type check before urlparse to prevent TypeError on non-string values
- Reject endpoint_url with empty/missing hostname after parsing (malformed URLs like 'https://')
- Keep scheme validation (http/https only) and private IP blocking via is_private_ip()
- Add logger.warning for SSRF block events
- Add regression tests: non-string endpoint_url and empty hostname cases
- Update sentinel.md with consolidated SSRF entry

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-27 14:48:52 +00:00
google-labs-jules[bot] c547ad1acc 🛡️ Sentinel: [HIGH] Fix SSRF vulnerability in S3 connection test endpoint_url
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-27 04:18:28 +00:00
17 changed files with 428 additions and 29 deletions
+2
View File
@@ -200,3 +200,5 @@ cython_debug/
# Build metadata files - generated at build time
GIT_SHA
RUNTIME_INFO
node_modules
frontend/node_modules
+9
View File
@@ -19,3 +19,12 @@
**Vulnerability:** The `_test_imap_connection` and `_test_s3_connection` functions in `app/api/integrations.py` did not validate user-provided `host` and `endpoint_url` variables against `is_private_ip()`. This allowed an attacker to test the presence of internal IMAP servers or direct S3 SDK API calls to internal infrastructure via SSRF.
**Learning:** Any time a new generic connection or integration test is added, SSRF validation may be forgotten if the core network utility (`is_private_ip`) is not systematically applied to all outbound network operations, regardless of the protocol (e.g., IMAP, S3).
**Prevention:** Establish a pattern where any user-configurable host or endpoint URL is immediately passed through the centralized `is_private_ip` validation function before any network call or third-party client initialization.
## 2024-05-27 - SSRF Bypass via HTTP Redirects
**Vulnerability:** In `app/api/url_upload.py`, the `validate_url_safety` function was correctly verifying the initially requested URL to prevent fetching internal IPs or cloud metadata endpoints. However, the subsequent `httpx.AsyncClient` was configured with `follow_redirects=True` without validating the destination of those redirects. An attacker could bypass SSRF protections by providing a URL to an attacker-controlled server that responds with a 301/302 redirect pointing to an internal target (e.g., `http://127.0.0.1` or `http://169.254.169.254`).
**Learning:** Checking the URL before sending the request is insufficient if the HTTP client automatically follows redirects. The target of every single redirect must be subject to the same strict validation as the initial request.
**Prevention:** Avoid `follow_redirects=True` for user-provided URLs when possible. If redirects must be followed, attach an event hook (e.g., `event_hooks={"response": [hook_function]}`) to the `httpx` client to intercept the response, calculate the redirect destination from the `Location` header, and run the URL safety validation logic before the redirect is actually followed.
## 2026-03-27 - SSRF Bypass via HTTP Redirects in httpx
**Vulnerability:** The `/process-url` endpoint used `httpx.AsyncClient(follow_redirects=True)` after validating the initial user-provided URL against SSRF protections. However, it did not validate the target URLs of any subsequent HTTP redirects, allowing an attacker to provide a safe URL that redirects to an internal/private IP, bypassing the security check.
**Learning:** Initial URL validation is insufficient when the HTTP client is configured to follow redirects automatically. The client must be explicitly configured to validate every redirect target.
**Prevention:** When using `httpx.AsyncClient(follow_redirects=True)` for user-provided URLs, always implement a redirect validator hook function (e.g., using `event_hooks={'response': [validate_redirect]}`) that resolves the `Location` header and passes it through the same SSRF validation logic before the redirect is followed.
+1 -1
View File
@@ -1 +1 @@
2026-03-25T07:54:28Z
2026-05-17T14:20:08Z
+114
View File
@@ -10,6 +10,120 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
<!-- version list -->
## v0.172.12 (2026-05-17)
### Bug Fixes
- Validate webhook targets before delivery
([#846](https://github.com/christianlouis/DocuElevate/pull/846),
[`e2fa963`](https://github.com/christianlouis/DocuElevate/commit/e2fa96318f5bd45607baa0fe08a0bf14e1ca83d4))
### Testing
- Cover webhook SSRF validation ([#846](https://github.com/christianlouis/DocuElevate/pull/846),
[`e2fa963`](https://github.com/christianlouis/DocuElevate/commit/e2fa96318f5bd45607baa0fe08a0bf14e1ca83d4))
## v0.172.11 (2026-05-17)
### Bug Fixes
- Escape search result template values
([#853](https://github.com/christianlouis/DocuElevate/pull/853),
[`1a02187`](https://github.com/christianlouis/DocuElevate/commit/1a0218799b9a1eb4154e2f4fbb2572cb3922106a))
### Documentation
- **changelog**: Update changelog [skip ci]
([`048f28a`](https://github.com/christianlouis/DocuElevate/commit/048f28a6717fa7f5cf4b235f9142e625e80e5d59))
## Unreleased
## v0.172.10 (2026-05-17)
### Bug Fixes
- **url-upload**: Handle unsafe redirects as client errors
([`871f788`](https://github.com/christianlouis/DocuElevate/commit/871f788f0bd782ba8ad3a7d70e5cd4ccd24f749b))
### Documentation
- **changelog**: Update changelog [skip ci]
([`58b14ae`](https://github.com/christianlouis/DocuElevate/commit/58b14ae769b85e25290126256de936743609af06))
## Unreleased
## v0.172.9 (2026-04-07)
### Bug Fixes
- **api**: Resolve merge conflicts, add type safety for endpoint_url in S3 connection test
([`57db4c7`](https://github.com/christianlouis/DocuElevate/commit/57db4c7c82f4a8df2e7e5e5505e1d5c01768fc16))
### Chores
- **ci**: Ignore CVE-2026-4539 in pip-audit until pygments releases a fix
([`6927e76`](https://github.com/christianlouis/DocuElevate/commit/6927e7643f9cbe1664f4a1a093511df5b079ed0a))
### Code Style
- Apply ruff auto-fix
([`8295279`](https://github.com/christianlouis/DocuElevate/commit/8295279ec93570da4eb0445ede8084d1eb2aba99))
- Sort imports in test_url_upload.py
([`bdfa3ba`](https://github.com/christianlouis/DocuElevate/commit/bdfa3ba1e0a5702414e3b449fbde6a6d3149557a))
### Documentation
- **changelog**: Update changelog [skip ci]
([`c6e0b80`](https://github.com/christianlouis/DocuElevate/commit/c6e0b80becab81a75aea4ee78f5aaf8b6ac54854))
- **changelog**: Update changelog [skip ci]
([`9b9882c`](https://github.com/christianlouis/DocuElevate/commit/9b9882c4d62691d0ddd20444e3b77bfe6eecc8c3))
- **changelog**: Update changelog [skip ci]
([`69053bf`](https://github.com/christianlouis/DocuElevate/commit/69053bfb08d3e2f12a86878044667ac500888837))
- **changelog**: Update changelog [skip ci]
([`76f202f`](https://github.com/christianlouis/DocuElevate/commit/76f202f7f1b94e39a4e79cd984770310599405bf))
### Testing
- Add coverage for url_upload redirect SSRF bypass prevention hook
([`152ee15`](https://github.com/christianlouis/DocuElevate/commit/152ee15b06ebf7beb6216423b4c8d93ec2243165))
- Add tests for SSRF validation in integrations
([`470f08d`](https://github.com/christianlouis/DocuElevate/commit/470f08d89322f2904b78a8b0f820973611486c26))
## Unreleased
### Chores
- **ci**: Ignore CVE-2026-4539 in pip-audit until pygments releases a fix
([`6927e76`](https://github.com/christianlouis/DocuElevate/commit/6927e7643f9cbe1664f4a1a093511df5b079ed0a))
### Documentation
- **changelog**: Update changelog [skip ci]
([`9b9882c`](https://github.com/christianlouis/DocuElevate/commit/9b9882c4d62691d0ddd20444e3b77bfe6eecc8c3))
- **changelog**: Update changelog [skip ci]
([`69053bf`](https://github.com/christianlouis/DocuElevate/commit/69053bfb08d3e2f12a86878044667ac500888837))
- **changelog**: Update changelog [skip ci]
([`76f202f`](https://github.com/christianlouis/DocuElevate/commit/76f202f7f1b94e39a4e79cd984770310599405bf))
### Testing
- Add tests for SSRF validation in integrations
([`470f08d`](https://github.com/christianlouis/DocuElevate/commit/470f08d89322f2904b78a8b0f820973611486c26))
## Unreleased
### Chores
+1 -1
View File
@@ -1 +1 @@
12a35f9
e2fa963
+6 -6
View File
@@ -1,10 +1,10 @@
DocuElevate Build Information
==============================
Version: 0.172.8
Build Date: 2026-03-25T07:54:28Z
Git Commit: 12a35f9b301a5a4430265f32f6552bd131e0d5e2
Git Short SHA: 12a35f9
Version: 0.172.12
Build Date: 2026-05-17T14:20:08Z
Git Commit: e2fa96318f5bd45607baa0fe08a0bf14e1ca83d4
Git Short SHA: e2fa963
Git Branch: main
Commit Date: 2026-03-25T08:54:06+01:00
Build Timestamp: 2026-03-25T07:54:28Z
Commit Date: 2026-05-17T16:19:41+02:00
Build Timestamp: 2026-05-17T14:20:08Z
==============================
+1 -1
View File
@@ -1 +1 @@
0.172.8
0.172.12
+29
View File
@@ -28,6 +28,10 @@ logger = logging.getLogger(__name__)
router = APIRouter()
class UnsafeRedirectError(httpx.RequestError):
"""Raised when a redirect target fails URL safety checks."""
class URLUploadRequest(BaseModel):
"""Request model for URL-based file upload"""
@@ -106,6 +110,26 @@ def validate_file_type(content_type: str, filename: str) -> bool:
return False
async def verify_redirect(response: httpx.Response) -> None:
"""
Event hook to intercept redirects and validate the new destination URL.
Prevents SSRF bypasses via redirects to internal networks or metadata endpoints.
"""
if response.status_code in (301, 302, 303, 307, 308):
location = response.headers.get("Location")
if location:
# Resolve relative redirects
new_url = str(response.url.join(location))
# Validate the new URL
try:
validate_url_safety(new_url)
except HTTPException as e:
raise UnsafeRedirectError(
f"Redirect to unsafe URL blocked: {e.detail}",
request=response.request,
) from e
@router.post("/process-url")
@require_login
async def process_url(
@@ -162,6 +186,7 @@ async def process_url(
async with httpx.AsyncClient(
timeout=settings.http_request_timeout,
follow_redirects=True,
event_hooks={"response": [verify_redirect]},
headers={
"User-Agent": "DocuElevate/1.0", # Identify ourselves
},
@@ -251,6 +276,10 @@ async def process_url(
logger.error(f"HTTP error while downloading file from URL: {url} - {str(e)}")
raise HTTPException(status_code=e.response.status_code, detail=f"HTTP error: {str(e)}")
except UnsafeRedirectError as e:
logger.warning(f"Unsafe redirect blocked while downloading file from URL: {url} - {str(e)}")
raise HTTPException(status_code=400, detail=str(e))
except httpx.RequestError as e:
logger.error(f"Error downloading file from URL: {url} - {str(e)}")
raise HTTPException(status_code=500, detail=f"Failed to download file: {str(e)}")
+21
View File
@@ -12,11 +12,13 @@ import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from typing import Any
from urllib.parse import urlparse
import httpx
from app.database import SessionLocal
from app.models import InAppNotification, UserNotificationPreference, UserNotificationTarget
from app.utils.network import is_private_ip
logger = logging.getLogger(__name__)
@@ -28,6 +30,11 @@ USER_EVENT_LABELS: dict[str, str] = {
EVENT_DOCUMENT_PROCESSED: "Document Processed",
EVENT_DOCUMENT_FAILED: "Document Processing Failed",
}
METADATA_ENDPOINTS = {
"169.254.169.254",
"169.254.169.253",
"metadata.google.internal",
}
def create_in_app_notification(
@@ -128,6 +135,20 @@ def _send_webhook_notification(target_config: dict[str, Any], event_type: str, t
logger.warning("Webhook notification target missing url")
return False
parsed_url = urlparse(url)
if parsed_url.scheme not in {"http", "https"}:
logger.warning("Webhook notification to %s blocked: invalid scheme %s", url, parsed_url.scheme)
return False
hostname = parsed_url.hostname
if not hostname:
logger.warning("Webhook notification to %s blocked: missing hostname", url)
return False
if hostname in METADATA_ENDPOINTS or is_private_ip(hostname):
logger.warning("Webhook notification to %s blocked: private or metadata endpoint", url)
return False
payload = {
"event": event_type,
"title": title,
+21
View File
@@ -18,11 +18,13 @@ import json
import logging
import time
from typing import Any
from urllib.parse import urlparse
import requests
from app.database import SessionLocal
from app.models import WebhookConfig
from app.utils.network import is_private_ip
logger = logging.getLogger(__name__)
@@ -40,6 +42,11 @@ VALID_EVENTS: frozenset[str] = frozenset(
#: Timeout (seconds) for outgoing webhook HTTP requests.
WEBHOOK_TIMEOUT = 10
METADATA_ENDPOINTS = {
"169.254.169.254",
"169.254.169.253",
"metadata.google.internal",
}
def compute_signature(payload_bytes: bytes, secret: str) -> str:
@@ -67,6 +74,20 @@ def deliver_webhook(url: str, payload: dict[str, Any], secret: str | None = None
Returns:
``True`` when the remote server responds with a 2xx status.
"""
parsed_url = urlparse(url)
if parsed_url.scheme not in {"http", "https"}:
logger.warning("Webhook to %s blocked: invalid scheme %s", url, parsed_url.scheme)
return False
hostname = parsed_url.hostname
if not hostname:
logger.warning("Webhook to %s blocked: missing hostname", url)
return False
if hostname in METADATA_ENDPOINTS or is_private_ip(hostname):
logger.warning("Webhook to %s blocked: private or metadata endpoint", url)
return False
body = json.dumps(payload, default=str, sort_keys=True)
body_bytes = body.encode("utf-8")
+39 -11
View File
@@ -1538,6 +1538,28 @@
});
}
function escapeHtml(str) {
if (!str) return '';
return String(str)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#039;');
}
function sanitizeHighlight(html) {
if (!html) return '';
let safe = String(html)
.replace(/<mark>/gi, '\x00MARK_OPEN\x00')
.replace(/<\/mark>/gi, '\x00MARK_CLOSE\x00');
safe = escapeHtml(safe);
safe = safe
.replace(/\x00MARK_OPEN\x00/g, '<mark>')
.replace(/\x00MARK_CLOSE\x00/g, '</mark>');
return safe;
}
function renderSearchResults(data, q) {
const panel = document.getElementById('search-results-panel');
const list = document.getElementById('search-results-list');
@@ -1555,25 +1577,31 @@
list.innerHTML = results.map(hit => {
const fmt = hit._formatted || {};
const title = fmt.document_title || hit.document_title || hit.original_filename || __i18n.untitled;
const filename = fmt.original_filename || hit.original_filename || '';
const snippet = fmt.ocr_text || '';
const tags = Array.isArray(hit.tags) ? hit.tags.join(', ') : (hit.tags || '');
const docType = hit.document_type || '';
const titleRaw = fmt.document_title || hit.document_title || hit.original_filename || __i18n.untitled;
const filenameRaw = fmt.original_filename || hit.original_filename || '';
const snippetRaw = fmt.ocr_text || '';
const tagsRaw = Array.isArray(hit.tags) ? hit.tags.join(', ') : (hit.tags || '');
const docTypeRaw = hit.document_type || '';
const safeTitle = fmt.document_title ? sanitizeHighlight(titleRaw) : escapeHtml(titleRaw);
const safeFilename = escapeHtml(filenameRaw);
const safeSnippet = sanitizeHighlight(snippetRaw);
const safeTags = escapeHtml(tagsRaw);
const safeDocType = escapeHtml(docTypeRaw);
return `<div style="padding: 0.75rem 1rem; border-bottom: 1px solid #f3f4f6; display: flex; gap: 0.75rem; align-items: flex-start;">
<div style="flex-shrink: 0; color: #3b82f6; font-size: 1.25rem; padding-top: 0.1rem;">
<i class="fas fa-file-pdf"></i>
</div>
<div style="flex: 1; min-width: 0;">
<div style="font-weight: 600; font-size: 0.9rem; color: #111827;">${title}</div>
${filename ? `<div style="font-size: 0.8rem; color: #6b7280; margin-top: 0.15rem;">${filename}</div>` : ''}
${docType ? `<span style="display: inline-block; margin-top: 0.25rem; padding: 0.1rem 0.5rem; background: #eff6ff; color: #1d4ed8; border-radius: 9999px; font-size: 0.75rem;">${docType}</span>` : ''}
${tags ? `<span style="display: inline-block; margin-top: 0.25rem; margin-left: 0.25rem; padding: 0.1rem 0.5rem; background: #f0fdf4; color: #15803d; border-radius: 9999px; font-size: 0.75rem;">${tags}</span>` : ''}
${snippet ? `<div style="margin-top: 0.4rem; font-size: 0.8rem; color: #374151; white-space: pre-wrap; word-break: break-word;">…${snippet}…</div>` : ''}
<div style="font-weight: 600; font-size: 0.9rem; color: #111827;">${safeTitle}</div>
${safeFilename ? `<div style="font-size: 0.8rem; color: #6b7280; margin-top: 0.15rem;">${safeFilename}</div>` : ''}
${safeDocType ? `<span style="display: inline-block; margin-top: 0.25rem; padding: 0.1rem 0.5rem; background: #eff6ff; color: #1d4ed8; border-radius: 9999px; font-size: 0.75rem;">${safeDocType}</span>` : ''}
${safeTags ? `<span style="display: inline-block; margin-top: 0.25rem; margin-left: 0.25rem; padding: 0.1rem 0.5rem; background: #f0fdf4; color: #15803d; border-radius: 9999px; font-size: 0.75rem;">${safeTags}</span>` : ''}
${safeSnippet ? `<div style="margin-top: 0.4rem; font-size: 0.8rem; color: #374151; white-space: pre-wrap; word-break: break-word;">…${safeSnippet}…</div>` : ''}
</div>
<div style="flex-shrink: 0;">
<a href="/files/${hit.file_id}" style="padding: 0.25rem 0.6rem; background: #f3f4f6; color: #374151; border-radius: 0.25rem; font-size: 0.8rem; text-decoration: none; white-space: nowrap;" title="${__i18n.viewFile}">
<a href="/files/${escapeHtml(hit.file_id)}" style="padding: 0.25rem 0.6rem; background: #f3f4f6; color: #374151; border-radius: 0.25rem; font-size: 0.8rem; text-decoration: none; white-space: nowrap;" title="${__i18n.viewFile}">
<i class="fas fa-external-link-alt"></i>
</a>
</div>
+1 -1
View File
@@ -1,5 +1,5 @@
[build-system]
requires = ["setuptools>=45", "wheel"]
requires = ["setuptools>=82.0.1", "wheel"]
build-backend = "setuptools.build_meta"
[project]
+1 -1
View File
@@ -34,7 +34,7 @@ pip-audit>=2.7.0 # Dependency vulnerability scanning against OSV/PyPA advisory
pre-commit>=3.6.0
# License compliance
pip-licenses==5.5.1 # For license compliance checking
pip-licenses==5.5.5 # For license compliance checking
# Release automation
python-semantic-release>=9.0.0
+1 -1
View File
@@ -51,7 +51,7 @@ pytesseract>=0.3.10 # Python wrapper for Tesseract OCR
pdf2image>=1.17.0 # Convert PDF pages to images (used by Tesseract and EasyOCR providers)
ocrmypdf>=16.0.0,<18.0.0 # Post-processing: embeds searchable text layers into PDFs via Tesseract
meilisearch>=0.31.0 # Full-text search engine client
stripe>=7.0.0,<15.0.0 # Stripe billing SDK (MIT license)
stripe>=7.0.0,<16.0.0 # Stripe billing SDK (MIT license)
# Error and performance monitoring
sentry-sdk[fastapi,celery,sqlalchemy]>=2.20.0,<3.0.0
+58
View File
@@ -465,6 +465,19 @@ class TestURLUploadEndpoint:
data = response.json()
assert "Failed to download file" in data["detail"]
@patch("app.api.url_upload.httpx.AsyncClient.stream")
def test_process_url_unsafe_redirect_returns_400(self, mock_stream, client):
"""Test unsafe redirects are reported as a client error instead of HTTP 500."""
from app.api.url_upload import UnsafeRedirectError
mock_stream.side_effect = UnsafeRedirectError("Redirect to unsafe URL blocked: Unsafe URL")
response = client.post("/api/process-url", json={"url": "https://example.com/file.pdf"})
assert response.status_code == 400
data = response.json()
assert "Redirect to unsafe URL blocked" in data["detail"]
@patch("app.api.url_upload.httpx.AsyncClient.stream")
def test_process_url_oserror_during_save(self, mock_stream, client, tmp_path, monkeypatch):
"""Test handling of OSError when saving file"""
@@ -879,3 +892,48 @@ class TestURLUploadCoverageGaps:
# Generic exception (not HTTPException/OSError/RequestException) is caught and returns 500
assert response.status_code == 500
assert "Unexpected error" in response.json()["detail"]
@pytest.mark.asyncio
async def test_verify_redirect_allows_safe_url(self):
"""Test verify_redirect allows safe redirects (lines 115, 118, 120-121)"""
import httpx
from app.api.url_upload import verify_redirect
req = httpx.Request("GET", "http://example.com")
resp = httpx.Response(301, headers={"Location": "https://google.com"}, request=req)
# Should not raise any exception
await verify_redirect(resp)
@pytest.mark.asyncio
@patch("app.api.url_upload.validate_url_safety")
async def test_verify_redirect_blocks_unsafe_url(self, mock_validate):
"""Test verify_redirect blocks unsafe redirects (lines 122-125)"""
import httpx
from fastapi import HTTPException
from app.api.url_upload import verify_redirect
mock_validate.side_effect = HTTPException(status_code=400, detail="Unsafe URL")
req = httpx.Request("GET", "http://example.com")
resp = httpx.Response(301, headers={"Location": "http://127.0.0.1"}, request=req)
with pytest.raises(httpx.RequestError) as exc_info:
await verify_redirect(resp)
assert "Redirect to unsafe URL blocked" in str(exc_info.value)
@pytest.mark.asyncio
async def test_verify_redirect_ignores_non_redirects(self):
"""Test verify_redirect ignores 200 OK responses"""
import httpx
from app.api.url_upload import verify_redirect
req = httpx.Request("GET", "http://example.com")
resp = httpx.Response(200, request=req)
# Should not raise any exception and should ignore missing Location header
await verify_redirect(resp)
+81 -6
View File
@@ -235,7 +235,10 @@ class TestSendWebhookNotification:
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
with patch("app.utils.user_notification.httpx.post", return_value=mock_response) as mock_post:
with (
patch("app.utils.user_notification.is_private_ip", return_value=False),
patch("app.utils.user_notification.httpx.post", return_value=mock_response) as mock_post,
):
result = _send_webhook_notification(
{"url": "https://hook.example.com/test", "secret": "mysecret"},
"document.processed",
@@ -256,7 +259,10 @@ class TestSendWebhookNotification:
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
with patch("app.utils.user_notification.httpx.post", return_value=mock_response) as mock_post:
with (
patch("app.utils.user_notification.is_private_ip", return_value=False),
patch("app.utils.user_notification.httpx.post", return_value=mock_response) as mock_post,
):
result = _send_webhook_notification(
{"url": "https://hook.example.com/test"},
"document.failed",
@@ -272,9 +278,12 @@ class TestSendWebhookNotification:
"""_send_webhook_notification returns False when httpx raises."""
from app.utils.user_notification import _send_webhook_notification
with patch(
"app.utils.user_notification.httpx.post",
side_effect=Exception("connection error"),
with (
patch("app.utils.user_notification.is_private_ip", return_value=False),
patch(
"app.utils.user_notification.httpx.post",
side_effect=Exception("connection error"),
),
):
result = _send_webhook_notification(
{"url": "https://hook.example.com/test"},
@@ -300,7 +309,10 @@ class TestSendWebhookNotification:
)
)
with patch("app.utils.user_notification.httpx.post", return_value=mock_response):
with (
patch("app.utils.user_notification.is_private_ip", return_value=False),
patch("app.utils.user_notification.httpx.post", return_value=mock_response),
):
result = _send_webhook_notification(
{"url": "https://hook.example.com/test"},
"document.processed",
@@ -310,6 +322,69 @@ class TestSendWebhookNotification:
assert result is False
def test_blocks_private_webhook_target(self):
"""Webhook delivery is skipped for private network targets."""
from app.utils.user_notification import _send_webhook_notification
with (
patch("app.utils.user_notification.is_private_ip", return_value=True),
patch("app.utils.user_notification.httpx.post") as mock_post,
):
result = _send_webhook_notification(
{"url": "https://10.0.0.5/test"},
"document.processed",
"T",
"M",
)
assert result is False
mock_post.assert_not_called()
def test_blocks_metadata_webhook_target(self):
"""Webhook delivery is skipped for cloud metadata endpoints."""
from app.utils.user_notification import _send_webhook_notification
with patch("app.utils.user_notification.httpx.post") as mock_post:
result = _send_webhook_notification(
{"url": "http://169.254.169.254/latest/meta-data"},
"document.processed",
"T",
"M",
)
assert result is False
mock_post.assert_not_called()
def test_blocks_invalid_webhook_scheme(self):
"""Webhook delivery is skipped for unsupported URL schemes."""
from app.utils.user_notification import _send_webhook_notification
with patch("app.utils.user_notification.httpx.post") as mock_post:
result = _send_webhook_notification(
{"url": "file:///etc/passwd"},
"document.processed",
"T",
"M",
)
assert result is False
mock_post.assert_not_called()
def test_blocks_webhook_without_hostname(self):
"""Webhook delivery is skipped when the URL has no hostname."""
from app.utils.user_notification import _send_webhook_notification
with patch("app.utils.user_notification.httpx.post") as mock_post:
result = _send_webhook_notification(
{"url": "https:///missing-host"},
"document.processed",
"T",
"M",
)
assert result is False
mock_post.assert_not_called()
# ---------------------------------------------------------------------------
# dispatch_user_notification preference loop
+42
View File
@@ -81,6 +81,7 @@ class TestDeliverWebhook:
def test_success_returns_true(self, mocker):
"""A 200 response returns True."""
mocker.patch("app.utils.webhook.is_private_ip", return_value=False)
mock_post = mocker.patch("app.utils.webhook.requests.post")
mock_post.return_value = MagicMock(ok=True, status_code=200)
@@ -90,6 +91,7 @@ class TestDeliverWebhook:
def test_non_2xx_returns_false(self, mocker):
"""A non-2xx response returns False."""
mocker.patch("app.utils.webhook.is_private_ip", return_value=False)
mock_post = mocker.patch("app.utils.webhook.requests.post")
mock_post.return_value = MagicMock(ok=False, status_code=500)
@@ -100,6 +102,7 @@ class TestDeliverWebhook:
"""A network error returns False."""
import requests
mocker.patch("app.utils.webhook.is_private_ip", return_value=False)
mocker.patch("app.utils.webhook.requests.post", side_effect=requests.ConnectionError("fail"))
result = deliver_webhook("https://example.com/hook", {"event": "test"})
@@ -107,6 +110,7 @@ class TestDeliverWebhook:
def test_signature_header_included_when_secret(self, mocker):
"""X-Webhook-Signature header is present when a secret is supplied."""
mocker.patch("app.utils.webhook.is_private_ip", return_value=False)
mock_post = mocker.patch("app.utils.webhook.requests.post")
mock_post.return_value = MagicMock(ok=True, status_code=200)
@@ -118,6 +122,7 @@ class TestDeliverWebhook:
def test_no_signature_header_without_secret(self, mocker):
"""X-Webhook-Signature header is absent when no secret is supplied."""
mocker.patch("app.utils.webhook.is_private_ip", return_value=False)
mock_post = mocker.patch("app.utils.webhook.requests.post")
mock_post.return_value = MagicMock(ok=True, status_code=200)
@@ -126,6 +131,43 @@ class TestDeliverWebhook:
headers = call_kwargs.kwargs.get("headers") or call_kwargs[1].get("headers")
assert "X-Webhook-Signature" not in headers
def test_private_target_is_blocked(self, mocker):
"""Private network webhook targets are not called."""
mocker.patch("app.utils.webhook.is_private_ip", return_value=True)
mock_post = mocker.patch("app.utils.webhook.requests.post")
result = deliver_webhook("https://10.0.0.5/hook", {"event": "test"})
assert result is False
mock_post.assert_not_called()
def test_metadata_target_is_blocked(self, mocker):
"""Cloud metadata webhook targets are not called."""
mock_post = mocker.patch("app.utils.webhook.requests.post")
result = deliver_webhook("http://169.254.169.254/latest/meta-data", {"event": "test"})
assert result is False
mock_post.assert_not_called()
def test_invalid_scheme_is_blocked(self, mocker):
"""Unsupported webhook URL schemes are not called."""
mock_post = mocker.patch("app.utils.webhook.requests.post")
result = deliver_webhook("file:///etc/passwd", {"event": "test"})
assert result is False
mock_post.assert_not_called()
def test_missing_hostname_is_blocked(self, mocker):
"""Webhook URLs without a hostname are not called."""
mock_post = mocker.patch("app.utils.webhook.requests.post")
result = deliver_webhook("https:///missing-host", {"event": "test"})
assert result is False
mock_post.assert_not_called()
# ---------------------------------------------------------------------------
# Unit tests get_active_webhooks_for_event (DB)