feat(tasks): add retry logic with exponential backoff and jitter
- Rewrite app/tasks/retry_config.py with compute_countdown() function implementing per-retry delays with ±20% jitter (default: 60s, 300s, 900s) - Add BaseTaskWithRetry.retry() override to inject proper countdown - Add OcrTaskWithRetry (120s, 600s, 1800s) for OCR/AI tasks - Add UploadTaskWithRetry for cloud-storage upload tasks - Add config settings: TASK_RETRY_MAX_RETRIES, TASK_RETRY_DELAYS, TASK_RETRY_JITTER - Update process_with_ocr and process_with_azure tasks to use OcrTaskWithRetry - Update all 11 upload tasks to use UploadTaskWithRetry - Add 38 unit tests in tests/test_retry_config.py - Update docs/ConfigurationGuide.md and .env.demo Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
@@ -9,7 +9,7 @@ from dropbox.exceptions import ApiError, AuthError
|
||||
|
||||
from app.celery_app import celery
|
||||
from app.config import settings
|
||||
from app.tasks.retry_config import BaseTaskWithRetry
|
||||
from app.tasks.retry_config import UploadTaskWithRetry
|
||||
from app.utils import log_task_progress
|
||||
from app.utils.filename_utils import extract_remote_path, get_unique_filename
|
||||
|
||||
@@ -102,7 +102,7 @@ def get_dropbox_client():
|
||||
raise
|
||||
|
||||
|
||||
@celery.task(base=BaseTaskWithRetry, bind=True)
|
||||
@celery.task(base=UploadTaskWithRetry, bind=True)
|
||||
def upload_to_dropbox(self, file_path: str, file_id: int = None):
|
||||
"""
|
||||
Upload a file to Dropbox.
|
||||
|
||||
Reference in New Issue
Block a user