copilot-swe-agent[bot] f6fcaaeccc feat(database): add database configuration wizard and migration tool ...

Add a guided database configuration wizard and a data migration tool that
allows users to:
- Build database connection strings through a step-by-step UI
- Test database connections before applying
- Preview and execute data migrations from SQLite to PostgreSQL/MySQL
- Copy to clipboard for easy .env file updates

New files:
- app/utils/db_wizard.py — connection string builder, parser, and tester
- app/utils/db_migrate.py — table-by-table data migration utility
- app/api/database.py — REST API endpoints for wizard operations
- app/views/db_wizard.py — view route for the wizard page
- frontend/templates/db_wizard.html — multi-tab wizard UI
- tests/test_db_wizard.py — unit tests for db_wizard utilities
- tests/test_db_migrate.py — unit tests for db_migrate utilities
- tests/test_db_wizard_api.py — integration tests for API and views

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

2026-03-05 22:10:14 +00:00

..

__init__.py

Add unit tests for utils.hash_file function

2025-03-26 16:27:36 +00:00

conftest_oauth.py

fix(tests): fix OAuth integration tests failing due to Docker registry timeout

2026-02-26 14:21:58 +00:00

conftest.py

feat(webhooks): add webhook support for external integrations

2026-03-01 20:42:57 +00:00

fixtures_integration.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

mock_oauth_server.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

README_INTEGRATION_TESTS.md

fix(ci): increase quick test job timeout to 15 minutes for CI runner headroom

2026-03-01 16:46:57 +00:00

README_OAUTH_TESTING.md

chore: apply ruff formatting and fix whitespace issues

2026-02-13 09:12:11 +00:00

test_ai_provider.py

feat(ai): handle temperature incompatibility for gpt-5 and o-series models, add model picker UI

2026-02-23 22:12:08 +00:00

test_allowed_types.py

feat(upload): adaptive 429 backoff, full Gotenberg file types, directory traversal

2026-02-26 23:24:26 +00:00

test_api_advanced_filters.py

feat(search): add content-finding filters, saved searches, and text quality to Search view

2026-03-01 13:46:15 +00:00

test_api_auth_enabled.py

style: format test_api_auth_enabled.py with Black and isort

2026-02-10 15:53:11 +00:00

test_api_azure_comprehensive.py

style: apply ruff auto-fix

2026-02-14 00:05:57 +00:00

test_api_azure_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_common.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_api_diagnostic_extended.py

style: apply ruff auto-fix

2026-02-23 23:24:27 +00:00

test_api_dropbox_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_dropbox.py

style: apply ruff auto-fix

2026-02-23 09:33:48 +00:00

test_api_error_handling.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_api_files_comprehensive.py

feat(api): implement proper pagination for file list API

2026-03-01 17:31:56 +00:00

test_api_google_drive_comprehensive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_google_drive_coverage2.py

fix(tests): use targeted settings mock instead of broad getattr patch

2026-02-26 09:16:49 +00:00

test_api_google_drive_coverage.py

style: apply ruff auto-fix

2026-02-14 00:17:16 +00:00

test_api_google_drive_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_logs.py

feat(security): add comprehensive input validation and sanitization (#172)

2026-02-22 16:02:00 +00:00

test_api_onedrive_comprehensive.py

style: apply ruff auto-fix

2026-02-23 09:33:48 +00:00

test_api_onedrive_coverage.py

fix(tests): fix TestGetFullConfigException using PropertyMock on module-level settings

2026-02-26 12:04:38 +00:00

test_api_onedrive_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_openai_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_openai.py

test(api): add comprehensive tests for AI extraction endpoint reaching 100% coverage

2026-02-23 21:14:08 +00:00

test_api_process.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_api_search.py

feat(search): add content-finding filters, saved searches, and text quality to Search view

2026-03-01 13:46:15 +00:00

test_api_settings_coverage.py

test: improve coverage for process_with_ocr, api/settings, and settings_service to 90%+

2026-02-25 15:24:42 +00:00

test_api_settings_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_api_settings.py

fix(tests): resolve CI test failures from asyncio event loop and settings reload side-effects

2026-02-22 20:56:58 +00:00

test_api_user.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_api.py

style: Apply Black formatting to modified files

2026-02-08 08:16:54 +00:00

test_audit_logging.py

style: apply ruff auto-fix

2026-02-22 14:24:27 +00:00

test_auth_integration.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_auth_module.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_auth.py

fix: resolve 48 lint errors in test files

2026-02-15 10:24:48 +00:00

test_bulk_operations.py

feat: persist ocr_quality_score and use it for numeric filtering

2026-03-01 13:46:43 +00:00

test_cache.py

style: fix spelling (British to American English)

2026-03-01 14:41:29 +00:00

test_celery_app.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_celery_worker.py

fix(tasks): register missing Celery tasks in celery_worker.py

2026-03-02 18:23:44 +00:00

test_check_credentials_extended.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_check_credentials.py

style: apply ruff auto-fix

2026-02-23 20:23:06 +00:00

test_config_loader.py

fix(config): decrypt sensitive settings when loading from database

2026-02-23 12:54:34 +00:00

test_config_validator_coverage.py

style: apply ruff auto-fix

2026-02-14 00:12:30 +00:00

test_config_validator_reexport.py

test(coverage): add celery_worker coverage and remove dead config_validator.py

2026-02-25 21:08:24 +00:00

test_config_validators.py

fix: correct patch paths in config validator tests

2026-02-16 15:37:22 +00:00

test_config.py

fix(ci): lint errors, remove webhook deploy, add ArgoCD GitOps k8s manifest update

2026-02-21 09:40:52 +00:00

test_convert_pdf.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_convert_to_pdf.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_convert_to_pdfa.py

fix(pdfa): address code review - validate pdfa_format, add S3 comment, add format test

2026-03-02 14:01:26 +00:00

test_cors.py

fix(tests): resolve CI test failures from asyncio event loop and settings reload side-effects

2026-02-22 20:56:58 +00:00

test_coverage_boost.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_coverage_config_settings.py

feat(api): remove redundant /env and /api/diagnostic/settings endpoints

2026-02-23 23:23:28 +00:00

test_coverage_final.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_coverage_polish.py

test: update 3 tests broken by terminal-step completed guard

2026-02-27 01:04:42 +00:00

test_coverage_remaining_gaps.py

feat(tests): increase code coverage from 89.28% to 92.15%

2026-02-23 23:59:50 +00:00

test_coverage_uploads_notification.py

style: run ruff format on 6 files to fix formatting issues

2026-02-13 22:39:34 +00:00

test_csrf.py

feat(auth): implement CSRF token protection for state-changing operations

2026-02-22 16:27:37 +00:00

test_dark_mode.py

feat(ui): add dark mode support with system preference detection and localStorage persistence

2026-02-25 13:59:49 +00:00

test_database_indexes.py

docs: add Performance & Caching section to ConfigurationGuide

2026-03-01 14:40:19 +00:00

test_database.py

fix(database): skip create_all for Alembic-tracked databases to prevent OperationalError on webhook_configs

2026-03-02 10:25:34 +00:00

test_db_migrate.py

feat(database): add database configuration wizard and migration tool

2026-03-05 22:10:14 +00:00

test_db_wizard_api.py

feat(database): add database configuration wizard and migration tool

2026-03-05 22:10:14 +00:00

test_db_wizard.py

feat(database): add database configuration wizard and migration tool

2026-03-05 22:10:14 +00:00

test_diagnostic.py

feat(api): add GET /api/diagnostic/health endpoint for monitoring

2026-03-01 18:36:38 +00:00

test_document_preview.py

fix: address code review feedback on tests and placeholder messages

2026-03-01 07:51:07 +00:00

test_duplicates.py

fix(similarity): truncate text to fit embedding model context window, fix step tracking

2026-03-02 13:27:34 +00:00

test_e2e_full_stack.py

fix: resolve all Ruff linting errors

2026-02-13 16:12:39 +00:00

test_embed_metadata.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_embed_pdf_metadata.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_encryption.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_endpoint_registration.py

feat(api): remove redundant /env and /api/diagnostic/settings endpoints

2026-02-23 23:23:28 +00:00

test_external_integrations.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_extract_metadata_gpt.py

feat: add Portkey provider support and null-content guard to AI abstraction layer

2026-02-23 19:42:03 +00:00

test_extract_metadata.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_file_detail_endpoints.py

style: run ruff format on 6 files to fix formatting issues

2026-02-13 22:39:34 +00:00

test_file_detail_enhancements.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_file_listing.py

feat(api): implement proper pagination for file list API

2026-03-01 17:31:56 +00:00

test_file_queries.py

fix(status): prevent false Completed status when mandatory pipeline steps have not run

2026-02-27 00:51:14 +00:00

test_file_splitting.py

fix: resolve 48 lint errors in test files

2026-02-15 10:24:48 +00:00

test_file_status_fix.py

fix(tasks): resolve files stuck in Pending status despite completed processing

2026-02-27 16:38:10 +00:00

test_file_upload.py

style: apply ruff auto-fix

2026-02-22 15:25:48 +00:00

test_filename_utils.py

docs: clarify behavior of extract_remote_path with multiple 'processed' dirs

2026-02-15 10:28:44 +00:00

test_files_view_extended.py

fix: Remove unused imports from test files

2026-02-13 23:40:31 +00:00

test_files_view.py

fix(test): update drop overlay text assertion to match new directory-upload message

2026-02-26 23:50:46 +00:00

test_finalize_storage.py

feat(pdfa): add PDF/A archival conversion using ocrmypdf

2026-03-02 13:35:13 +00:00

test_imap_extended.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_imap_tasks.py

feat(imap): add IMAP_READONLY_MODE feature flag to safeguard shared mailboxes

2026-03-01 14:28:51 +00:00

test_input_validation.py

style: apply ruff auto-fix

2026-02-22 16:12:58 +00:00

test_logging_utils.py

style: apply ruff auto-fix

2026-02-14 00:05:57 +00:00

test_main.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_migrate_logs_to_steps.py

test: add comprehensive tests for critical untested files and fix deprecated warnings

2026-02-12 17:32:17 +00:00

test_monitor_stalled_steps.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_notification_utils.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_notification.py

fix: correct notification test to handle keyword args and actual file size formatting

2026-02-16 15:56:19 +00:00

test_notifications.py

test: remove unused import from test_notifications.py

2026-02-09 20:56:35 +00:00

test_oauth_helper.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_oauth_integration_flows.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_ocr_language_manager.py

fix(ocr): ensure Tesseract language data for embed_text_layer regardless of active OCR provider

2026-02-24 18:02:19 +00:00

test_ocr_processing.py

Merge pull request #396 from christianlouis/copilot/achieve-code-coverage-goals

2026-02-24 19:25:41 +01:00

test_ocr_provider_coverage.py

test: achieve 90%+ code coverage across codebase

2026-02-24 18:17:22 +00:00

test_original_filename_preservation.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_path_traversal_security.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_process_document.py

fix(status): prevent false Completed status when mandatory pipeline steps have not run

2026-02-27 00:51:14 +00:00

test_process_with_ocr_comparison.py

test: improve coverage for process_with_ocr, api/settings, and settings_service to 90%+

2026-02-25 15:24:42 +00:00

test_processall_throttling.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_queue_monitoring.py

refactor(queue): address code review feedback — extract constants and sync refresh interval

2026-02-27 00:47:58 +00:00

test_rate_limit_decorators.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_rate_limiting.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_rclone_tasks.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_reexports.py

docs: update references to deleted app/utils.py file

2026-02-09 15:14:40 +00:00

test_release_names.py

refactor(test): simplify test patches per code review feedback

2026-03-01 18:38:07 +00:00

test_request_size_limit.py

style: apply ruff auto-fix

2026-02-22 15:25:48 +00:00

test_retry_config.py

feat(tasks): add retry logic with exponential backoff and jitter

2026-03-01 17:37:41 +00:00

test_retry_subtask_logging.py

test: improve upload retry tests with legacy path assertions and priority verification

2026-02-28 12:57:50 +00:00

test_security_headers.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_send_to_all.py

fix(ci): add pytest-timeout, fix hanging test, split CI into quick + integration stages

2026-03-01 16:43:56 +00:00

test_settings_audit_log.py

style: apply ruff auto-fix

2026-02-23 12:28:59 +00:00

test_settings_display.py

test: achieve 90%+ code coverage across codebase

2026-02-24 18:17:22 +00:00

test_settings_service_coverage.py

test: skip tests for non-existent wrapper functions in settings_service

2026-02-16 15:47:20 +00:00

test_settings_service_extended.py

test: improve coverage for process_with_ocr, api/settings, and settings_service to 90%+

2026-02-25 15:24:42 +00:00

test_settings.py

feat(settings): add all missing config settings to settings page with enhanced UX

2026-03-02 16:37:28 +00:00

test_setup_wizard.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_similarity.py

feat(similarity): add similarity pairs dashboard, step tracking, and fix tests for pre-computed embeddings

2026-03-02 13:12:59 +00:00

test_step_manager.py

fix: address code review feedback - restrict terminal step check to success only, add test assertions

2026-02-27 16:39:32 +00:00

test_step_timeout.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_storage_reorganization.py

fix(tests): update process_with_azure patch target in test_storage_reorganization

2026-02-23 23:09:55 +00:00

test_text_quality.py

feat(ocr): fine-tune OCR quality criteria with stricter threshold and head-to-head comparison

2026-02-25 13:03:21 +00:00

test_upload_email.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_upload_ftp_additional.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_upload_google_drive.py

test: Add comprehensive tests for upload_to_google_drive and status views

2026-02-13 23:33:25 +00:00

test_upload_tasks_additional.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_tasks_coverage.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_tasks.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_dropbox.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_nextcloud.py

feat(tests): increase test coverage for wizard.py and upload_to_nextcloud.py

2026-02-13 23:45:11 +00:00

test_upload_to_onedrive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_paperless.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_to_s3.py

fix(tests): mock log_task_progress in S3 upload tests

2026-02-17 04:35:00 +00:00

test_upload_to_sftp.py

fix: make log_task_progress resilient to DB errors and fix test assertions

2026-02-17 11:40:34 +00:00

test_upload_webdav_comprehensive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_upload_webdav_integration.py

fix: apply ruff formatting to 6 test files

2026-02-13 16:51:07 +00:00

test_upload_with_rclone.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_uptime_kuma.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_url_upload.py

test: Increase coverage for url_upload.py and files.py to 90%+

2026-02-13 23:36:32 +00:00

test_utils.py

test: convert unittest-style tests to pytest in test_utils.py and test_notifications.py

2026-02-09 20:55:53 +00:00

test_views_coverage.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_views_dropbox.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_views_filemanager.py

style(tests): rename _SESSION_SECRET to TEST_SESSION_SECRET; tighten non-admin redirect assertions to == 302

2026-02-24 00:04:42 +00:00

test_views_files_comprehensive.py

refactor: consolidate linting tools into Ruff

2026-02-13 09:10:52 +00:00

test_views_general.py

feat(privacy): implement GDPR and global privacy compliance

2026-02-25 15:15:24 +00:00

test_views_google_drive.py

style: apply ruff auto-fix

2026-02-14 00:16:21 +00:00

test_views_license.py

fix(test): fix test_get_lgpl_license_not_found JSONDecodeError

2026-02-17 12:01:40 +00:00

test_views_onedrive.py

style: fix code formatting with black, isort, and flake8

2026-02-12 03:21:41 +00:00

test_views_search.py

feat(search): add content-finding filters, saved searches, and text quality to Search view

2026-03-01 13:46:15 +00:00

test_views_settings.py

style(tests): rename _SESSION_SECRET to TEST_SESSION_SECRET; tighten non-admin redirect assertions to == 302

2026-02-24 00:04:42 +00:00

test_views_status.py

test(status): increase code coverage for app/views/status.py to 100%

2026-03-01 17:41:50 +00:00

test_views_wizard.py

test: increase code coverage for app/views/wizard.py to 100%

2026-03-01 17:40:54 +00:00

test_webhooks.py

feat(webhooks): add webhook support for external integrations

2026-03-01 20:42:57 +00:00

test_wizard_db_persist.py

fix: isolate token_hex mock to wizard module and fix mypy annotation

2026-02-23 10:01:07 +00:00