copilot-swe-agent[bot] 89d5df71c8 feat(ocr): fine-tune OCR quality criteria with stricter threshold and head-to-head comparison ...

- Raise quality acceptance threshold from 65→85 (configurable via TEXT_QUALITY_THRESHOLD)
- Reject text with significant issues (excessive_typos, garbage_characters,
  incoherent_text, fragmented_sentences) even when score is above threshold
  (configurable via TEXT_QUALITY_SIGNIFICANT_ISSUES)
- Add compare_text_quality() for AI-powered head-to-head comparison of
  original embedded text vs fresh OCR output
- Update process_document to pass original text to OCR task for comparison
- Update process_with_ocr to run comparison and keep the higher-quality text
- Add new settings to settings_service.py metadata
- Update docs/ConfigurationGuide.md with new settings
- Add comprehensive tests for new threshold and comparison logic

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

2026-02-25 13:03:21 +00:00

..

__init__.py

Add unit tests for utils.hash_file function

2025-03-26 16:27:36 +00:00

conftest_oauth.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

conftest.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

fixtures_integration.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

mock_oauth_server.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

README_INTEGRATION_TESTS.md

chore: apply ruff formatting and fix whitespace issues

2026-02-13 09:12:11 +00:00

README_OAUTH_TESTING.md

chore: apply ruff formatting and fix whitespace issues

2026-02-13 09:12:11 +00:00

test_ai_provider.py

feat(ai): handle temperature incompatibility for gpt-5 and o-series models, add model picker UI

2026-02-23 22:12:08 +00:00

test_api_auth_enabled.py

style: format test_api_auth_enabled.py with Black and isort

2026-02-10 15:53:11 +00:00

test_api_azure_comprehensive.py

style: apply ruff auto-fix

2026-02-14 00:05:57 +00:00

test_api_azure_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_common.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_api_diagnostic_extended.py

style: apply ruff auto-fix

2026-02-23 23:24:27 +00:00

test_api_dropbox_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_dropbox.py

style: apply ruff auto-fix

2026-02-23 09:33:48 +00:00

test_api_error_handling.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_api_files_comprehensive.py

test: fix test_retry_azure_ocr_success to patch process_with_ocr instead of legacy azure task

2026-02-24 22:55:46 +00:00

test_api_google_drive_comprehensive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_google_drive_coverage.py

style: apply ruff auto-fix

2026-02-14 00:17:16 +00:00

test_api_google_drive_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_logs.py

feat(security): add comprehensive input validation and sanitization (#172)

2026-02-22 16:02:00 +00:00

test_api_onedrive_comprehensive.py

style: apply ruff auto-fix

2026-02-23 09:33:48 +00:00

test_api_onedrive_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_openai_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_openai.py

test(api): add comprehensive tests for AI extraction endpoint reaching 100% coverage

2026-02-23 21:14:08 +00:00

test_api_process.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_api_settings_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_settings.py

fix(tests): resolve CI test failures from asyncio event loop and settings reload side-effects

2026-02-22 20:56:58 +00:00

test_api_user.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_api.py

style: Apply Black formatting to modified files

2026-02-08 08:16:54 +00:00

test_audit_logging.py

style: apply ruff auto-fix

2026-02-22 14:24:27 +00:00

test_auth_integration.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_auth_module.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_auth.py

fix: resolve 48 lint errors in test files

2026-02-15 10:24:48 +00:00

test_bulk_operations.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_celery_app.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_celery_worker.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_check_credentials_extended.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_check_credentials.py

style: apply ruff auto-fix

2026-02-23 20:23:06 +00:00

test_config_loader.py

fix(config): decrypt sensitive settings when loading from database

2026-02-23 12:54:34 +00:00

test_config_validator_coverage.py

style: apply ruff auto-fix

2026-02-14 00:12:30 +00:00

test_config_validator_reexport.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_config_validators.py

fix: correct patch paths in config validator tests

2026-02-16 15:37:22 +00:00

test_config.py

fix(ci): lint errors, remove webhook deploy, add ArgoCD GitOps k8s manifest update

2026-02-21 09:40:52 +00:00

test_convert_pdf.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_convert_to_pdf.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_cors.py

fix(tests): resolve CI test failures from asyncio event loop and settings reload side-effects

2026-02-22 20:56:58 +00:00

test_coverage_boost.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_coverage_config_settings.py

feat(api): remove redundant /env and /api/diagnostic/settings endpoints

2026-02-23 23:23:28 +00:00

test_coverage_final.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_coverage_remaining_gaps.py

feat(tests): increase code coverage from 89.28% to 92.15%

2026-02-23 23:59:50 +00:00

test_coverage_uploads_notification.py

style: run ruff format on 6 files to fix formatting issues

2026-02-13 22:39:34 +00:00

test_csrf.py

feat(auth): implement CSRF token protection for state-changing operations

2026-02-22 16:27:37 +00:00

test_database.py

test: add comprehensive tests for license_routes, database migrations, SFTP, S3, and notifications

2026-02-14 00:30:52 +00:00

test_diagnostic.py

feat(api): remove redundant /env and /api/diagnostic/settings endpoints

2026-02-23 23:23:28 +00:00

test_e2e_full_stack.py

fix: resolve all Ruff linting errors

2026-02-13 16:12:39 +00:00

test_embed_metadata.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_embed_pdf_metadata.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_encryption.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_endpoint_registration.py

feat(api): remove redundant /env and /api/diagnostic/settings endpoints

2026-02-23 23:23:28 +00:00

test_external_integrations.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_extract_metadata_gpt.py

feat: add Portkey provider support and null-content guard to AI abstraction layer

2026-02-23 19:42:03 +00:00

test_extract_metadata.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_file_detail_endpoints.py

style: run ruff format on 6 files to fix formatting issues

2026-02-13 22:39:34 +00:00

test_file_detail_enhancements.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_file_listing.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_file_queries.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_file_splitting.py

fix: resolve 48 lint errors in test files

2026-02-15 10:24:48 +00:00

test_file_status_fix.py

test: achieve 90%+ code coverage across codebase

2026-02-24 18:17:22 +00:00

test_file_upload.py

style: apply ruff auto-fix

2026-02-22 15:25:48 +00:00

test_filename_utils.py

docs: clarify behavior of extract_remote_path with multiple 'processed' dirs

2026-02-15 10:28:44 +00:00

test_files_view_extended.py

fix: Remove unused imports from test files

2026-02-13 23:40:31 +00:00

test_files_view.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_finalize_storage.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_imap_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_imap_tasks.py

test: add comprehensive tests for IMAP, OpenAI, send_to_all, config_loader, and diagnostic APIs

2026-02-14 00:34:34 +00:00

test_input_validation.py

style: apply ruff auto-fix

2026-02-22 16:12:58 +00:00

test_logging_utils.py

style: apply ruff auto-fix

2026-02-14 00:05:57 +00:00

test_main.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_migrate_logs_to_steps.py

test: add comprehensive tests for critical untested files and fix deprecated warnings

2026-02-12 17:32:17 +00:00

test_monitor_stalled_steps.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_notification_utils.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_notification.py

fix: correct notification test to handle keyword args and actual file size formatting

2026-02-16 15:56:19 +00:00

test_notifications.py

test: remove unused import from test_notifications.py

2026-02-09 20:56:35 +00:00

test_oauth_helper.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_oauth_integration_flows.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_ocr_language_manager.py

fix(ocr): ensure Tesseract language data for embed_text_layer regardless of active OCR provider

2026-02-24 18:02:19 +00:00

test_ocr_processing.py

Merge pull request #396 from christianlouis/copilot/achieve-code-coverage-goals

2026-02-24 19:25:41 +01:00

test_ocr_provider_coverage.py

test: achieve 90%+ code coverage across codebase

2026-02-24 18:17:22 +00:00

test_original_filename_preservation.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_path_traversal_security.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_process_document.py

fix(tests): update process_document test patches to use process_with_ocr

2026-02-23 23:00:45 +00:00

test_processall_throttling.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_rate_limit_decorators.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_rate_limiting.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_rclone_tasks.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_reexports.py

docs: update references to deleted app/utils.py file

2026-02-09 15:14:40 +00:00

test_request_size_limit.py

style: apply ruff auto-fix

2026-02-22 15:25:48 +00:00

test_retry_subtask_logging.py

style: run ruff format on 6 files to fix formatting issues

2026-02-13 22:39:34 +00:00

test_security_headers.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_send_to_all.py

fix: Apply ruff format to test_send_to_all.py

2026-02-16 20:40:44 +00:00

test_settings_audit_log.py

style: apply ruff auto-fix

2026-02-23 12:28:59 +00:00

test_settings_display.py

test: achieve 90%+ code coverage across codebase

2026-02-24 18:17:22 +00:00

test_settings_service_coverage.py

test: skip tests for non-existent wrapper functions in settings_service

2026-02-16 15:47:20 +00:00

test_settings.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_setup_wizard.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_step_manager.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_step_timeout.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_storage_reorganization.py

fix(tests): update process_with_azure patch target in test_storage_reorganization

2026-02-23 23:09:55 +00:00

test_text_quality.py

feat(ocr): fine-tune OCR quality criteria with stricter threshold and head-to-head comparison

2026-02-25 13:03:21 +00:00

test_upload_email.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_upload_ftp_additional.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_upload_google_drive.py

test: Add comprehensive tests for upload_to_google_drive and status views

2026-02-13 23:33:25 +00:00

test_upload_tasks_additional.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_tasks_coverage.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_tasks.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_dropbox.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_nextcloud.py

feat(tests): increase test coverage for wizard.py and upload_to_nextcloud.py

2026-02-13 23:45:11 +00:00

test_upload_to_onedrive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_paperless.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_s3.py

fix(tests): mock log_task_progress in S3 upload tests

2026-02-17 04:35:00 +00:00

test_upload_to_sftp.py

fix: make log_task_progress resilient to DB errors and fix test assertions

2026-02-17 11:40:34 +00:00

test_upload_webdav_comprehensive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_webdav_integration.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_upload_with_rclone.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_uptime_kuma.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_url_upload.py

test: Increase coverage for url_upload.py and files.py to 90%+

2026-02-13 23:36:32 +00:00

test_utils.py

test: convert unittest-style tests to pytest in test_utils.py and test_notifications.py

2026-02-09 20:55:53 +00:00

test_views_coverage.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_views_dropbox.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_views_filemanager.py

style(tests): rename _SESSION_SECRET to TEST_SESSION_SECRET; tighten non-admin redirect assertions to == 302

2026-02-24 00:04:42 +00:00

test_views_files_comprehensive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_views_general.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_views_google_drive.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_views_license.py

fix(test): fix test_get_lgpl_license_not_found JSONDecodeError

2026-02-17 12:01:40 +00:00

test_views_onedrive.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_views_settings.py

style(tests): rename _SESSION_SECRET to TEST_SESSION_SECRET; tighten non-admin redirect assertions to == 302

2026-02-24 00:04:42 +00:00

test_views_status.py

feat(api): remove redundant /env and /api/diagnostic/settings endpoints

2026-02-23 23:23:28 +00:00

test_views_wizard.py

fix: isolate token_hex mock to wizard module and fix mypy annotation

2026-02-23 10:01:07 +00:00

test_wizard_db_persist.py

fix: isolate token_hex mock to wizard module and fix mypy annotation

2026-02-23 10:01:07 +00:00