Adds validation to webhook URLs before attempting to deliver them to prevent
SSRF attacks targeting private IP ranges, local host, and cloud metadata endpoints.
Validates URL schema, hostname, and applies `is_private_ip()`. Also resolved ruff linting
errors. Tests have been expanded to ensure validation covers all cases.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Modified `is_private_ip` in `app/utils/network.py` to fail securely by returning True (blocking the request) when a hostname cannot be resolved. The previous implementation failed open, creating a risk for Server-Side Request Forgery (SSRF) and DNS rebinding attacks.
Updated corresponding tests to expect the secure behavior and correctly appended the security finding to the Sentinel journal.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add 10 new tests in TestURLUploadCoverageGaps to cover previously
uncovered lines and branches in app/api/url_upload.py:
- Line 41: validate_url_scheme raises ValueError for non-http scheme
- Lines 65->61, 67: is_private_ip DNS path with public IP resolution
- Line 87: validate_url_safety with ftp:// scheme (direct call)
- Line 107: validate_url_safety blocks metadata.google.internal
- Line 130->135: validate_file_type with no file extension
- Line 177: sanitize_filename returning empty string defaults to 'download'
- Line 234->233: iter_content empty bytes chunks (if chunk: False branch)
- Line 285: OSError cleanup path removes existing partial file
- Line 291->293: unexpected exception before target_path assigned (stays None)
Coverage: 91.16% -> 100%
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace "hash_file" test step with actual MAIN_PROCESSING_STEPS names
- Fix test_get_step_summary to use upload_to_* instead of queue_* for upload counts
- All 12 step_manager tests now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
google-labs-jules[bot]