- Update _compute_processing_flow to recognize process_with_ocr as the OCR
stage and remap legacy process_with_azure_document_intelligence log entries
for backward compatibility
- Normalize legacy OCR step name in _compute_step_summary log fallback
- Add process_with_ocr to REAL_MAIN_STEPS/REAL_STEPS in step_manager,
file_status, and file_queries (keeping legacy name for old DB entries)
- Update retry logic in api/files.py to retry failed OCR via process_with_ocr
(handles both step names as aliases)
- Fix process_document.py to log process_with_ocr as skipped (not azure step)
for the local text extraction path
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Root cause: ensure_ocr_languages_from_settings() only downloaded tessdata
when the 'tesseract' provider was active, but embed_text_layer() uses
ocrmypdf (which needs tessdata) as a fallback for ALL OCR providers.
- embed_text_layer(): call ensure_tesseract_languages(language) after
confirming ocrmypdf is on PATH, so language data is present before
ocrmypdf is invoked (prevents exit code 3 for fra/deu/etc.)
- ensure_ocr_languages_from_settings(): extend the condition from
'tesseract' in active_providers to also trigger when ocrmypdf is
on PATH, enabling proactive pre-download at startup for any config
- Tests: mock shutil.which and ensure_tesseract_languages in affected
test cases; rename azure-only test and add new test for ocrmypdf case
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add app/utils/ocr_language_manager.py: detects tessdata dir, downloads
missing .traineddata files via wget/curl from tessdata_fast GitHub repo,
pre-downloads EasyOCR models, exposes async background-thread helper
- TesseractOCRProvider.process() calls ensure_tesseract_languages() before
running pytesseract; raises clear error if languages remain unavailable
- EasyOCRProvider.process() logs informational message when models download
- app/main.py: calls ensure_ocr_languages_async() at startup
- app/utils/settings_sync.py: triggers language re-check after every
settings reload so UI changes take effect without container restart
- app/api/settings.py: adds POST /api/settings/install-ocr-languages
endpoint for on-demand language installation from the admin UI
- Dockerfile: adds wget for runtime tessdata downloads
- docs/ConfigurationGuide.md: documents automatic language download
- tests/test_ocr_language_manager.py: 29 unit tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- PDFs are now uploaded to Mistral Files API (POST /v1/files) and
processed via a signed document_url, resolving the 422 error caused
by passing data:application/pdf;base64,... to an image endpoint
- Images (JPEG/PNG/GIF/WEBP/BMP/TIFF) use base64 image_url directly
- Unsupported MIME types raise a clear ValueError
- Magic-byte fallback detects PDFs with no file extension
- Switches from openai chat completions to requests HTTP calls
- Adds helper method _upload_pdf_and_get_document()
- Adds TestMistralOCRProvider with 9 unit tests covering all paths
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
semantic-release