- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Root cause: ensure_ocr_languages_from_settings() only downloaded tessdata
when the 'tesseract' provider was active, but embed_text_layer() uses
ocrmypdf (which needs tessdata) as a fallback for ALL OCR providers.
- embed_text_layer(): call ensure_tesseract_languages(language) after
confirming ocrmypdf is on PATH, so language data is present before
ocrmypdf is invoked (prevents exit code 3 for fra/deu/etc.)
- ensure_ocr_languages_from_settings(): extend the condition from
'tesseract' in active_providers to also trigger when ocrmypdf is
on PATH, enabling proactive pre-download at startup for any config
- Tests: mock shutil.which and ensure_tesseract_languages in affected
test cases; rename azure-only test and add new test for ocrmypdf case
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add app/utils/ocr_language_manager.py: detects tessdata dir, downloads
missing .traineddata files via wget/curl from tessdata_fast GitHub repo,
pre-downloads EasyOCR models, exposes async background-thread helper
- TesseractOCRProvider.process() calls ensure_tesseract_languages() before
running pytesseract; raises clear error if languages remain unavailable
- EasyOCRProvider.process() logs informational message when models download
- app/main.py: calls ensure_ocr_languages_async() at startup
- app/utils/settings_sync.py: triggers language re-check after every
settings reload so UI changes take effect without container restart
- app/api/settings.py: adds POST /api/settings/install-ocr-languages
endpoint for on-demand language installation from the admin UI
- Dockerfile: adds wget for runtime tessdata downloads
- docs/ConfigurationGuide.md: documents automatic language download
- tests/test_ocr_language_manager.py: 29 unit tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- PDFs are now uploaded to Mistral Files API (POST /v1/files) and
processed via a signed document_url, resolving the 422 error caused
by passing data:application/pdf;base64,... to an image endpoint
- Images (JPEG/PNG/GIF/WEBP/BMP/TIFF) use base64 image_url directly
- Unsupported MIME types raise a clear ValueError
- Magic-byte fallback detects PDFs with no file extension
- Switches from openai chat completions to requests HTTP calls
- Adds helper method _upload_pdf_and_get_document()
- Adds TestMistralOCRProvider with 9 unit tests covering all paths
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]