fix(status): prevent false Completed status when mandatory pipeline steps have not run
Add a terminal-step guard (send_to_all_destinations) to all status calculation paths so that files are only marked Completed once the entire processing pipeline has been recorded. - get_file_overall_status: require TERMINAL_STEP to be present - get_files_processing_status: same guard for bulk status - get_step_summary: count missing terminal step as queued so total_main_steps > main_completed when pipeline is incomplete - apply_status_filter: SQL sub-query requires terminal step for completed filter - process_document: call initialize_file_steps after creating a new file record so all mandatory steps are pre-created as pending Define TERMINAL_STEP constant in step_manager.py and reference it in file_status.py and file_queries.py to avoid magic strings. Tests updated: add send_to_all_destinations to completed-file fixtures; add test verifying initialize_file_steps is called for new files. Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
@@ -7,7 +7,7 @@ from typing import Dict, List
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models import FileProcessingStep, FileRecord, ProcessingLog
|
||||
from app.utils.step_manager import get_file_overall_status
|
||||
from app.utils.step_manager import TERMINAL_STEP, get_file_overall_status
|
||||
|
||||
|
||||
def get_file_processing_status(db: Session, file_id: int) -> Dict:
|
||||
@@ -150,7 +150,14 @@ def get_files_processing_status(db: Session, file_ids: List[int]) -> Dict[int, D
|
||||
elif in_progress_steps > 0:
|
||||
status = "processing"
|
||||
elif completed_steps + skipped_steps == total_steps:
|
||||
status = "completed"
|
||||
# Only mark as completed if the terminal processing step has been
|
||||
# recorded. Without this guard, files where later pipeline steps
|
||||
# have not yet started would be falsely marked as "completed".
|
||||
existing_step_names = {s.step_name for s in file_steps}
|
||||
if TERMINAL_STEP in existing_step_names:
|
||||
status = "completed"
|
||||
else:
|
||||
status = "pending"
|
||||
else:
|
||||
status = "pending"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user