- Add OCR_LANGUAGES constant (28 languages, EN/DE/FR/ES/IT/PT/RU/ZH/JA/KO/AR/etc.)
- Add TESSERACT_TO_EASYOCR mapping for automatic code translation
- Add optional language constructor arg to TesseractOCRProvider/EasyOCRProvider
- Update get_ocr_providers() to accept and pass per-call language override
- Add language parameter to process_with_ocr Celery task
- Add _get_pipeline_ocr_language() helper to resolve OCR language from pipeline step config
- Update process_document to look up and pass pipeline OCR language to process_with_ocr
- Add ocr_language select config field (28 options) to pipeline OCR step schema
- Add language dropdown to pipeline UI (pipelines.html)
- Update docs/UserGuide.md and docs/API.md with language override documentation
- Add 27 new tests covering language constants, provider overrides, and pipeline lookup
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Convert f-string log interpolation to %s-style formatting in
app/api/pipelines.py and app/api/saved_searches.py to prevent
clear-text logging of request-derived data (CodeQL: clear-text
logging of sensitive information)
- Replace plain hashlib.sha256() with PBKDF2-HMAC-SHA256 via
hash_token() in app/auth.py for Bearer token verification,
consistent with how tokens are stored in api_tokens.py (CodeQL:
use of weak cryptographic hashing on sensitive data)
- Remove redundant {exc} from logger.exception() calls (the
traceback is already captured by logger.exception())
- Update test to verify PBKDF2 hash instead of plain SHA-256
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The pipeline management UI showed an empty list after first boot because no
default system pipeline was created. This adds seed_default_pipeline() which:
- Creates a system-owned (owner_id=NULL), is_default=True pipeline named
"Standard Processing Pipeline" at application startup
- Steps mirror the current hardcoded Celery processing workflow:
convert_to_pdf → check_duplicates → ocr → extract_metadata →
embed_metadata → compute_embedding → send_to_destinations
- Is idempotent: no-op if any system pipeline already exists
- Handles missing pipelines table gracefully (during first migration run)
Also wires the seeder into app/main.py lifespan startup using the same
pattern as seed_default_plans.
9 new tests added covering creation, step order, idempotency, and API visibility.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract _get_user_id into shared auth.get_current_user_id() used by both
pipelines API and the assign-pipeline endpoint in files API
- Fix aria-live attribute: use two separate static containers (polite/assertive)
instead of dynamic Alpine.js binding for correct screen reader announcements
- Fix migration comment to accurately describe batch-mode FK creation
- Remove redundant tags parameter from reorder endpoint decorator
- Rename _make_file test helper to _make_test_file_record for clarity
- Update docs/UserGuide.md and docs/API.md with full Pipelines reference
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Pipeline and PipelineStep models with user-specific ownership
- Add pipeline_id FK column to FileRecord
- Migration 017_add_pipelines (batch mode for SQLite FK compat)
- Pipeline CRUD API at /api/pipelines with step management endpoints
- Reorder steps PUT endpoint placed before parameterised {step_id} routes
- POST /api/files/{id}/assign-pipeline for per-file pipeline assignment
- Admin-only POST /api/pipelines/admin/system for system-level pipelines
- Management UI at /pipelines (Jinja2 + Alpine.js + Tailwind)
- Pipelines link added to desktop and mobile navigation
- 41 new tests in tests/test_api_pipelines.py (all passing)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]