feat(ocr): fine-tune OCR quality criteria with stricter threshold and head-to-head comparison

- Raise quality acceptance threshold from 65→85 (configurable via TEXT_QUALITY_THRESHOLD)
- Reject text with significant issues (excessive_typos, garbage_characters,
  incoherent_text, fragmented_sentences) even when score is above threshold
  (configurable via TEXT_QUALITY_SIGNIFICANT_ISSUES)
- Add compare_text_quality() for AI-powered head-to-head comparison of
  original embedded text vs fresh OCR output
- Update process_document to pass original text to OCR task for comparison
- Update process_with_ocr to run comparison and keep the higher-quality text
- Add new settings to settings_service.py metadata
- Update docs/ConfigurationGuide.md with new settings
- Add comprehensive tests for new threshold and comparison logic

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

app/config.py

@@ -288,6 +288,22 @@ class Settings(BaseSettings):
             "bypass the check. Default: True (enabled)."
         ),
     )
+    text_quality_threshold: int = Field(
+        default=85,
+        description=(
+            "Minimum quality score (0–100) required to accept embedded PDF text without re-OCR. "
+            "Text scoring below this threshold is discarded and the file is re-processed with OCR. "
+            "Default: 85. The stricter this value, the more files will be re-OCR'd."
+        ),
+    )
+    text_quality_significant_issues: Union[List[str], str] = Field(
+        default_factory=lambda: ["excessive_typos", "garbage_characters", "incoherent_text", "fragmented_sentences"],
+        description=(
+            "Comma-separated list of quality issue labels that force OCR re-run even when the quality "
+            "score is above TEXT_QUALITY_THRESHOLD. Any of these issues present in the AI assessment "
+            "will trigger re-OCR. Default: excessive_typos,garbage_characters,incoherent_text,fragmented_sentences"
+        ),
+    )
 
     # Processing step timeout - prevents files from getting stuck in "in_progress" state
     step_timeout: int = Field(
@@ -443,6 +459,18 @@ class Settings(BaseSettings):
             return []
         return v
 
+    @field_validator("text_quality_significant_issues", mode="before")
+    @classmethod
+    def parse_text_quality_significant_issues(cls, v: str | list[str]) -> list[str]:
+        """Parse significant issue labels from comma-separated string or list."""
+        if isinstance(v, str):
+            if "," in v:
+                return [item.strip() for item in v.split(",") if item.strip()]
+            elif v.strip():
+                return [v.strip()]
+            return []
+        return v
+
     @field_validator("cors_allowed_origins", "cors_allowed_methods", "cors_allowed_headers", mode="before")
     @classmethod
     def parse_comma_separated_list(cls, v: str | list[str]) -> list[str]: