feat(ocr): add multi-language OCR support with per-pipeline language override

- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-03-08 21:47:12 +00:00
parent b03ea41638
commit a2a4c6fc9a
9 changed files with 808 additions and 18 deletions
+29 -3
View File
@@ -1785,12 +1785,27 @@ Returns the catalogue of built-in step types.
"label": "OCR Processing",
"description": "Extract text using Azure Document Intelligence or local Tesseract.",
"config_schema": {
"force_cloud_ocr": { "type": "boolean", "default": false }
"force_cloud_ocr": { "type": "boolean", "default": false },
"ocr_language": {
"type": "select",
"default": "auto",
"description": "Language(s) for OCR. Overrides the global setting for Tesseract/EasyOCR. Azure/Mistral auto-detect.",
"options": [
{ "value": "auto", "label": "Auto (use system default)" },
{ "value": "eng", "label": "English" },
{ "value": "deu", "label": "German" },
{ "value": "fra", "label": "French" },
{ "value": "spa", "label": "Spanish" },
"..."
]
}
}
}
}
```
The `ocr_language` field accepts Tesseract language codes (e.g. `"eng"`, `"deu"`, `"eng+deu"` for multi-language) or `"auto"` to fall back to the global system setting. The full list of 28 supported language codes is returned by the step-types endpoint.
### List pipelines
```bash
@@ -1883,12 +1898,23 @@ Content-Type: application/json
{
"step_type": "ocr",
"label": "Cloud OCR",
"config": { "force_cloud_ocr": true },
"label": "German OCR",
"config": { "force_cloud_ocr": false, "ocr_language": "deu" },
"enabled": true
}
```
Multi-language (Tesseract `+`-separated codes):
```bash
{
"step_type": "ocr",
"config": { "ocr_language": "eng+deu" }
}
```
Use `"ocr_language": "auto"` (or omit the field) to fall back to the global system language setting.
### Update step
```bash
+38 -1
View File
@@ -519,13 +519,50 @@ Processing pipelines let you define exactly what happens to your documents when
|-----------|-------------|
| `convert_to_pdf` | Convert non-PDF files to PDF using Gotenberg |
| `check_duplicates` | Detect duplicate files by content hash |
| `ocr` | Extract text with Azure Document Intelligence or local Tesseract |
| `ocr` | Extract text with OCR (supports multi-language configuration, see below) |
| `extract_metadata` | Extract structured metadata (type, sender, tags) with AI |
| `embed_metadata` | Write extracted metadata into the PDF document properties |
| `compute_embedding` | Compute semantic embeddings for similarity search |
| `send_to_destinations` | Upload the processed document to all configured storage destinations |
| `classify` | Classify the document type with AI |
#### OCR step options
The `ocr` step supports two optional configuration fields:
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `force_cloud_ocr` | boolean | `false` | Always run cloud OCR even if the PDF already has embedded text |
| `ocr_language` | string | `"auto"` | Language(s) to use for OCR text extraction (see below) |
**`ocr_language` — per-pipeline language override**
This option enables manual language control per pipeline, overriding the global Tesseract/EasyOCR language settings for all documents processed by that pipeline. The following values are supported (28 languages total):
| Value | Language | Value | Language |
|-------|----------|-------|----------|
| `auto` | Auto (use system default) | `jpn` | Japanese |
| `ara` | Arabic | `kor` | Korean |
| `chi_sim` | Chinese (Simplified) | `nor` | Norwegian |
| `chi_tra` | Chinese (Traditional) | `pol` | Polish |
| `ces` | Czech | `por` | Portuguese |
| `dan` | Danish | `ron` | Romanian |
| `nld` | Dutch | `rus` | Russian |
| `eng` | English | `spa` | Spanish |
| `fin` | Finnish | `swe` | Swedish |
| `fra` | French | `tha` | Thai |
| `deu` | German | `tur` | Turkish |
| `ell` | Greek | `ukr` | Ukrainian |
| `heb` | Hebrew | `vie` | Vietnamese |
| `hin` | Hindi | | |
| `hun` | Hungarian | | |
| `ita` | Italian | | |
> **Notes:**
> - The language override applies to **Tesseract** and **EasyOCR** providers. **Azure Document Intelligence** and **Mistral OCR** perform automatic language detection regardless of this setting.
> - For multi-language documents with Tesseract, combine codes with `+`, e.g. `eng+deu`.
> - Setting `ocr_language` to `auto` or leaving it unset uses the global `TESSERACT_LANGUAGE` / `EASYOCR_LANGUAGES` environment variables.
### Assigning a pipeline to a file
You can assign (or change) the pipeline for an individual document via the file detail page or the API: