Add documentation for batch processing throttling feature

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

docs/API.md

@@ -101,6 +101,29 @@ Reprocess a specific file.
 }
 ```
 
+### Batch Processing
+
+**POST** `/api/processall`
+
+Process all PDF files in the configured workdir directory.
+
+**Throttling**: For large batches (>20 files by default), tasks are automatically staggered to prevent overwhelming downstream APIs. The throttling behavior can be configured via environment variables:
+
+- `PROCESSALL_THROTTLE_THRESHOLD`: Number of files above which throttling is applied (default: 20)
+- `PROCESSALL_THROTTLE_DELAY`: Delay in seconds between each task submission when throttling (default: 3)
+
+**Example**: When processing 25 files with default settings, the first file is queued immediately, the second after 3 seconds, the third after 6 seconds, etc., spreading the load over 72 seconds total.
+
+**Response**:
+```json
+{
+  "message": "Enqueued 25 PDFs for processing (throttled over 72 seconds)",
+  "pdf_files": ["file1.pdf", "file2.pdf", ...],
+  "task_ids": ["a1b2c3...", "d4e5f6...", ...],
+  "throttled": true
+}
+```
+
 **POST** `/send_to_google_drive/`
 
 Send a processed file to Google Drive.

docs/ConfigurationGuide.md

@@ -17,6 +17,20 @@ Configuration is primarily done through environment variables specified in a `.e
 | `EXTERNAL_HOSTNAME`    | The external hostname for the application.             | `docuelevate.example.com`      |
 | `ALLOW_FILE_DELETE`    | Enable file deletion in the web interface (`true`/`false`). | `true`                      |
 
+### Batch Processing Settings
+
+Control how the `/processall` endpoint handles large batches of files to prevent overwhelming downstream APIs.
+
+| **Variable**                      | **Description**                                                                                    | **Default** |
+|-----------------------------------|----------------------------------------------------------------------------------------------------|-------------|
+| `PROCESSALL_THROTTLE_THRESHOLD`   | Number of files above which throttling is applied. Files <= threshold are processed immediately.  | `20`        |
+| `PROCESSALL_THROTTLE_DELAY`       | Delay in seconds between each task submission when throttling is active.                          | `3`         |
+
+**Example Usage**: When processing 25 files with default settings:
+- Files are staggered: file 0 at 0s, file 1 at 3s, file 2 at 6s, etc.
+- Total queue time: (25-1) × 3 = 72 seconds
+- Prevents API rate limit issues and ensures smooth processing
+
 ### IMAP Configuration
 
 DocuElevate can monitor multiple IMAP mailboxes for document attachments. Each mailbox uses a numbered prefix (e.g., `IMAP1_`, `IMAP2_`).