fix(status): prevent false Completed status when mandatory pipeline steps have not run

Add a terminal-step guard (send_to_all_destinations) to all status
calculation paths so that files are only marked Completed once the
entire processing pipeline has been recorded.

- get_file_overall_status: require TERMINAL_STEP to be present
- get_files_processing_status: same guard for bulk status
- get_step_summary: count missing terminal step as queued so
  total_main_steps > main_completed when pipeline is incomplete
- apply_status_filter: SQL sub-query requires terminal step for
  completed filter
- process_document: call initialize_file_steps after creating a new
  file record so all mandatory steps are pre-created as pending

Define TERMINAL_STEP constant in step_manager.py and reference it in
file_status.py and file_queries.py to avoid magic strings.

Tests updated: add send_to_all_destinations to completed-file
fixtures; add test verifying initialize_file_steps is called for
new files.

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-02-27 00:51:14 +00:00
parent 515f5b97e9
commit e6dd39c27d
7 changed files with 165 additions and 12 deletions
+9 -2
View File
@@ -7,7 +7,7 @@ from typing import Dict, List
from sqlalchemy.orm import Session
from app.models import FileProcessingStep, FileRecord, ProcessingLog
from app.utils.step_manager import get_file_overall_status
from app.utils.step_manager import TERMINAL_STEP, get_file_overall_status
def get_file_processing_status(db: Session, file_id: int) -> Dict:
@@ -150,7 +150,14 @@ def get_files_processing_status(db: Session, file_ids: List[int]) -> Dict[int, D
elif in_progress_steps > 0:
status = "processing"
elif completed_steps + skipped_steps == total_steps:
status = "completed"
# Only mark as completed if the terminal processing step has been
# recorded. Without this guard, files where later pipeline steps
# have not yet started would be falsely marked as "completed".
existing_step_names = {s.step_name for s in file_steps}
if TERMINAL_STEP in existing_step_names:
status = "completed"
else:
status = "pending"
else:
status = "pending"