gitea-admin/gh-christianlouis-docuelevate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(api): add per-user health-aware upload rate limiting

Browse Source 
Introduces a Redis-backed sliding-window rate limiter for upload
endpoints (/api/ui-upload, /api/process-url) that:
- Enforces per-user limits (default: 20 uploads / 60 s)
- Dynamically reduces limits under system stress (queue depth, CPU load)
- Returns 429 with Retry-After header when exceeded
- Fails open when Redis is unavailable
- Works with the existing client-side adaptive back-off

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-03-17 12:12:50 +00:00
parent 56bf665397
commit 571cc81789
8 changed files with 615 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ConfigurationGuide.md
+22
View File
@@ -85,6 +85,28 @@ Control how the web UI queues and paces file uploads to avoid overwhelming the b
**Example**: With `UPLOAD_CONCURRENCY=3` and `UPLOAD_QUEUE_DELAY_MS=500`, a directory of 5,000 files is uploaded ≈ 3 at a time with 500 ms pacing – the backend processes files at its own rate while the queue drains in the background without triggering API rate limits.
### Per-User Upload Rate Limiting
Server-side rate limiting that prevents any single user from overwhelming the system with bulk uploads. The limiter uses a Redis-backed sliding window and dynamically adjusts limits based on system health.
| **Variable** | **Description** | **Default** |
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------|-------------|
| `UPLOAD_RATE_LIMIT_PER_USER` | Maximum uploads allowed per user within the sliding window. Effective limit may be reduced under load. | `20` |
| `UPLOAD_RATE_LIMIT_WINDOW` | Sliding window size in seconds. | `60` |
**Health-aware dynamic limiting**: The effective per-user limit is automatically reduced when the system is under heavy load:
| **System condition** | **Effective limit** | **Trigger** |
|--------------------------------|---------------------|--------------------------------|
| Normal | 100 % of base | Queue < 50, CPU load normal |
| Moderate load | 50 % of base | Queue 50–100 or CPU > 1.5× |
| High load | 25 % of base | Queue 100–200 or CPU > 2× |
| Critical load | 10 % of base | Queue > 200 or CPU > 3× |
When a user exceeds the limit, the server returns **HTTP 429 Too Many Requests** with a `Retry-After` header. The browser client (see *Client-Side Upload Throttling* above) automatically pauses and retries.
> **Note**: The limiter fails open — if Redis is unavailable, all uploads are allowed through so that a monitoring outage never blocks document processing.
### File Upload Size Limits
**Security Feature**: Control file upload sizes to prevent resource exhaustion attacks. See [SECURITY_AUDIT.md](../SECURITY_AUDIT.md#5-file-upload-size-limits) for security details.