Combined duplicated `event_hooks` keyword arguments into a single dictionary parameter with both `validate_redirect` and `verify_redirect` in `app/api/url_upload.py`. This fixes a `SyntaxError: keyword argument repeated: event_hooks` and ensures that all redirect validations run.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: The `/process-url` endpoint used `httpx.AsyncClient` with `follow_redirects=True`. While the initial user-provided URL was validated against SSRF protections (blocking private/internal IPs), the client implicitly followed subsequent HTTP redirects without validating their target locations. This allowed an attacker to bypass the initial check by supplying a valid URL that redirected to an internal IP or cloud metadata endpoint.
🎯 Impact: An attacker could potentially access internal network services or cloud metadata endpoints.
🔧 Fix: Implemented an `event_hooks` listener (`validate_redirect`) on the `httpx.AsyncClient` that intercepts responses, extracts the `Location` header, resolves the absolute target URL, and applies the same `validate_url_safety` check before allowing the redirect to be followed.
✅ Verification: Ran `pytest tests/test_url_upload.py`, formatting checks via `ruff format` and linting via `ruff check`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
🚨 Severity: HIGH
💡 Vulnerability: User-provided IMAP `host` in `_test_imap_connection` and `pull_inbox` was not validated against private IPs, creating an SSRF risk.
🎯 Impact: Attackers could abuse the endpoints to port-scan or interact with internal/private network services.
🔧 Fix: Integrated `is_private_ip` from `app.utils.network` to block connections resolving to private, loopback, link-local, or reserved IPs.
✅ Verification: Ran `test_imap_tasks.py` and `test_api_imap_accounts.py` successfully. Checked `ruff` output and diffs. Removed all scratch files from the commit.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Introduces a Redis-backed sliding-window rate limiter for upload
endpoints (/api/ui-upload, /api/process-url) that:
- Enforces per-user limits (default: 20 uploads / 60 s)
- Dynamically reduces limits under system stress (queue depth, CPU load)
- Returns 429 with Retry-After header when exceeded
- Fails open when Redis is unavailable
- Works with the existing client-side adaptive back-off
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by deriving the extension from original_filename and filtering out all non-alphanumerics.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths, and filtering out non-alphanumerics from the file extension.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Added dependencies `aiofiles` and `types-aiofiles` to resolve MyPy typing CI failures, and mitigated CodeQL security alerts regarding user-provided path extensions by leveraging `os.path.basename` around the generated target file paths.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replaced synchronous `requests.get` and `open().write` in the `process_url` endpoint with `httpx.AsyncClient` and `aiofiles.open`. This prevents the FastAPI event loop from blocking during large file downloads.
Updated test suite in `tests/test_url_upload.py` to use `AsyncMock` to mock `httpx.AsyncClient.stream` contexts and async generators properly, covering all original conditions and HTTP error handling paths.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The @require_login decorator expects a Starlette Request as the first
parameter to access request.session, but process_url only had a
URLUploadRequest Pydantic model parameter. This caused an
AttributeError: 'URLUploadRequest' object has no attribute 'session'
when POST /api/process-url was called.
Fix: Add `request: Request` as the first parameter and rename the
Pydantic model parameter from `request` to `url_request`.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Christian Krakau-Louis