The test was patching os.path.join, which is called inside an inner
try/except block in save_onedrive_settings. This meant the exception
was silently caught and logged, never reaching the outer exception
handler that returns HTTP 500.
Fix by patching notify_settings_updated instead, which is called in
the outer try block, so exceptions correctly propagate to the outer
handler.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
The debounceSearch() function in files.html called clearFullTextSearch()
when the query was shorter than 2 characters. Since clearFullTextSearch()
sets input.value = '', every single keystroke was immediately erased —
users could paste text but not type.
Fix: debounceSearch now only hides the results panel for short queries
without touching the input value.
Also adds a dedicated /search page with Google-style results showing
content previews (document title, filename, type badges, tag badges,
sender, and OCR text snippets with highlighted matches).
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Replace module-level getattr patch with a MagicMock that has a property
raising on google_drive_use_oauth access, as suggested by code review.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Change @patch("app.api.google_drive.get_google_drive_service") to
@patch("app.tasks.upload_to_google_drive.get_google_drive_service")
because the function is imported locally inside the endpoint function
body, not at module level
- Replace patch.object(type(settings), "google_drive_use_oauth", ...)
with patch("app.api.google_drive.getattr", ...) because Pydantic v2
models don't expose fields as regular class attributes
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Rewrite tests/test_celery_worker.py to mock check_credentials.apply_async
at import time, enabling all 35 statements to be covered without Redis
- Add tests for conditional beat schedule entries (IMAP, Uptime Kuma)
- Delete app/utils/config_validator.py — dead code shadowed by the
config_validator/ package directory (Python gives packages precedence)
- Remove both files from coverage omit in pyproject.toml
- celery_worker.py now at 100% coverage (was 0%)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add dismissable cookie notice banner to base template (essential
cookies only, ePrivacy Directive compliant, localStorage persistence)
- Expand Privacy Notice to cover all target markets: EU/GDPR,
UK GDPR, Switzerland nFADP, Ukraine, US CCPA/CPRA, Canada PIPEDA/
Law 25, Brazil LGPD/Latin America, and Asia-Pacific & Japan (APPI,
Australia Privacy Act, South Korea PIPA, Singapore PDPA, India DPDP)
- Add International Data Transfers section (SCCs, IDTAs, adequacy
decisions) and Data Minimization & Purpose Limitation section
- Update Cookie Policy with precise cookie table, ePrivacy exemption
rationale, and localStorage notice dismissal documentation
- Create docs/PrivacyCompliance.md: full multi-market compliance guide
covering cookie strategy, data transfer mechanisms, data subject
rights handling matrix with response timelines, and market-specific
notes for all supported regions
- Add docs/PrivacyCompliance.md to mkdocs.yml Compliance nav section
- Add 10 new targeted tests to test_views_general.py validating all
key compliance content areas
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Raise quality acceptance threshold from 65→85 (configurable via TEXT_QUALITY_THRESHOLD)
- Reject text with significant issues (excessive_typos, garbage_characters,
incoherent_text, fragmented_sentences) even when score is above threshold
(configurable via TEXT_QUALITY_SIGNIFICANT_ISSUES)
- Add compare_text_quality() for AI-powered head-to-head comparison of
original embedded text vs fresh OCR output
- Update process_document to pass original text to OCR task for comparison
- Update process_with_ocr to run comparison and keep the higher-quality text
- Add new settings to settings_service.py metadata
- Update docs/ConfigurationGuide.md with new settings
- Add comprehensive tests for new threshold and comparison logic
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Root cause: ensure_ocr_languages_from_settings() only downloaded tessdata
when the 'tesseract' provider was active, but embed_text_layer() uses
ocrmypdf (which needs tessdata) as a fallback for ALL OCR providers.
- embed_text_layer(): call ensure_tesseract_languages(language) after
confirming ocrmypdf is on PATH, so language data is present before
ocrmypdf is invoked (prevents exit code 3 for fra/deu/etc.)
- ensure_ocr_languages_from_settings(): extend the condition from
'tesseract' in active_providers to also trigger when ocrmypdf is
on PATH, enabling proactive pre-download at startup for any config
- Tests: mock shutil.which and ensure_tesseract_languages in affected
test cases; rename azure-only test and add new test for ocrmypdf case
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add app/utils/ocr_language_manager.py: detects tessdata dir, downloads
missing .traineddata files via wget/curl from tessdata_fast GitHub repo,
pre-downloads EasyOCR models, exposes async background-thread helper
- TesseractOCRProvider.process() calls ensure_tesseract_languages() before
running pytesseract; raises clear error if languages remain unavailable
- EasyOCRProvider.process() logs informational message when models download
- app/main.py: calls ensure_ocr_languages_async() at startup
- app/utils/settings_sync.py: triggers language re-check after every
settings reload so UI changes take effect without container restart
- app/api/settings.py: adds POST /api/settings/install-ocr-languages
endpoint for on-demand language installation from the admin UI
- Dockerfile: adds wget for runtime tessdata downloads
- docs/ConfigurationGuide.md: documents automatic language download
- tests/test_ocr_language_manager.py: 29 unit tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- PDFs are now uploaded to Mistral Files API (POST /v1/files) and
processed via a signed document_url, resolving the 422 error caused
by passing data:application/pdf;base64,... to an image endpoint
- Images (JPEG/PNG/GIF/WEBP/BMP/TIFF) use base64 image_url directly
- Unsupported MIME types raise a clear ValueError
- Magic-byte fallback detects PDFs with no file extension
- Switches from openai chat completions to requests HTTP calls
- Adds helper method _upload_pdf_and_get_document()
- Adds TestMistralOCRProvider with 9 unit tests covering all paths
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]