- Update test_bulk_operations.py: Use FileProcessingStep instead of ProcessingLog for status filter tests
- Update test_file_listing.py: Use FileProcessingStep for processing status determination tests
- Update test_file_detail_endpoints.py: Replace hash_file with check_text in step summary test
- Update test_path_traversal_security.py: Import get_unique_filepath_with_counter from correct module
- Update test_process_document.py: Match current duplicate handling behavior (creates new record)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace ProcessingLog-based tests with FileProcessingStep-based tests
- Update to use step_manager functions (get_file_overall_status, get_step_summary)
- All 6 tests (3 status + 3 metrics) now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace ProcessingLog with FileProcessingStep in sample_files fixture
- Update test comments to reflect using steps instead of logs
- All 10 file_queries tests now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace "hash_file" test step with actual MAIN_PROCESSING_STEPS names
- Fix test_get_step_summary to use upload_to_* instead of queue_* for upload counts
- All 12 step_manager tests now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added endpoints for previewing original and processed PDF files.
- Implemented on-demand text extraction from original and processed PDFs.
- Updated file detail page to show original and processed file paths with existence status.
- Introduced GPT metadata display with a collapsible JSON view.
- Enhanced front-end with PDF.js for in-browser PDF rendering and improved user experience.
- Added integration tests for new features including metadata display and file previews.
- Replace unique_filepath with get_unique_filepath_with_counter in tests
- Update test expectations for -0001 suffix format
- All 40 tests passing successfully
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix _compute_status_from_logs to track latest status per unique step
- Fix _compute_step_summary to count only latest status per step
- Add comprehensive tests for both fixes
- Resolves issue where completed files showed as "Processing"
- Resolves issue where metrics showed incorrect counts
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Address code review feedback by extracting the hardcoded test URL
into a module-level constant to improve maintainability and ensure
consistency across all test methods.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add comprehensive tests to verify critical API endpoints are registered,
including the /api/process-url endpoint. These tests will prevent future
regressions where endpoints might not be properly registered in the app.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add missing pytest markers (e2e, requires_docker) to pyproject.toml
- Update CI workflow to skip E2E tests with -m "not e2e"
- Update integration test documentation with CI configuration notes
- E2E tests can still be run locally with: pytest -m e2e
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Use lazy % formatting in logging (PYL-W1203) in test_external_integrations.py
- Add @staticmethod to 3 methods not using self (PYL-R0201)
- Remove unused mock_media and mock_smtp variables (PYL-W0612)
- Remove redundant local reimports of upload_to_webdav (PYL-W0404)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add @staticmethod to 9 test methods in test_external_integrations.py
that don't use self (PYL-R0201)
- Extract hard-coded password literals to constants in 6 test files
to resolve S2068 warnings (fixtures_integration, test_imap_tasks,
test_upload_tasks, test_upload_webdav_comprehensive,
test_upload_webdav_integration, test_views_coverage)
- Migrate Form() dependency injection to Annotated type hints in
dropbox.py, google_drive.py, onedrive.py (Sonar fastapi convention)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- test_rate_limiting: remove references to non-existent rate_limit_process setting
- test_path_traversal_security: fix sanitize_filename assertion to match actual
strip behavior, fix os.path.basename test for Linux (backslash not a separator),
remove erroneous task_mock arg from embed_metadata_into_pdf direct call
- test_e2e_full_stack: add psycopg2 availability check to skip Postgres test
when driver is not installed
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add fpdf2 dependency for dynamic test PDF generation
- Create tests/test_external_integrations.py with end-to-end pipeline tests:
- OpenAI: key validation, metadata extraction via chat completion
- Azure Document Intelligence: admin connectivity, full OCR on generated PDF
- S3: bucket access, upload/download/delete pipeline
- Dropbox: token refresh, upload/download/delete pipeline
- OneDrive: token refresh, upload/download/delete pipeline
- Authentik: OIDC discovery endpoint, credential consistency
- Full pipeline: Azure OCR → OpenAI metadata extraction
- Update tests/conftest.py to capture original env vars before test overrides
- Add has_real_env() helper and original_env fixture for credential detection
- All external tests use @pytest.mark.requires_external and skipif guards
- Test files are dynamically generated with unique content per run
- Uploaded test files are cleaned up in finally blocks
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Update generate_build_metadata.sh to sync VERSION from latest git tag
- Change BUILD_DATE format from date-only to ISO 8601 with time (YYYY-MM-DDTHH:MM:SSZ)
- Fix VERSION file from 0.5.0 to 0.9.1 (matching latest git tag v0.9.1)
- Change version fallback from hardcoded '0.5.0-dev' to 'unknown' in config.py
- Update release.yml to commit all build metadata files (not just VERSION)
- Update BuildMetadata.md documentation to reflect automated versioning
- Add tests for build_date with time format and version unknown fallback
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Extract duplicated PDF text extraction into _extract_text_from_pdf helper
- Clarify empty metadata dict comment for embed_metadata_into_pdf retry
- Make test assertion for file_id passing more explicit
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add file_id parameter to process_document to skip duplicate hash check on reprocess
- Pass file_id from reprocess_single_file and bulk_reprocess_files endpoints
- Extend retry-subtask endpoint to support pipeline steps (process_document,
process_with_azure_document_intelligence, extract_metadata_with_gpt,
embed_metadata_into_pdf) in addition to upload tasks
- Add retry button for failed main pipeline steps in file detail UI
- Add comprehensive tests for reprocessing and pipeline step retry
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Created app/utils/file_queries.py with apply_status_filter function
- Updated app/api/files.py to use shared function
- Updated app/views/files.py to use shared function
- Removed unused 'or_' import from app/api/files.py
- Added comprehensive tests in tests/test_file_queries.py
- All tests pass (10 new tests, 14 existing tests verified)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add detailed test setup section to CONTRIBUTING.md explaining environment configuration
- Add testing section to README.md with quick start guide
- Create test_api_auth_enabled.py with 11 new integration tests for auth configuration
- Document that tests automatically configure required environment variables (no manual setup)
- Explain AUTH_ENABLED and SESSION_SECRET configuration for tests
- Include examples of testing with authentication enabled
- Reference integration test documentation for Docker-based tests
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix test count in SECURITY_AUDIT.md (11 tests, not 24)
- Add deprecation note for ALLOW-FROM in X-Frame-Options
- Update documentation to recommend CSP frame-ancestors instead
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add prominent documentation that splitting uses page boundaries
- Update docstring with IMPORTANT note about page-level splitting
- Add test to validate split PDFs are valid and readable
- Update ConfigurationGuide.md to emphasize page-based approach
- Update SECURITY_AUDIT.md with implementation details
- Ensures users understand no risk of corrupted PDFs
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Use BytesIO for size checking instead of temporary disk writes (major performance improvement)
- Add constant and comment for PDF overhead multiplier in tests
- Address code review feedback
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add MAX_UPLOAD_SIZE config (default 1GB) to prevent resource exhaustion
- Add MAX_SINGLE_FILE_SIZE config for optional PDF file splitting
- Implement automatic PDF splitting when files exceed single file limit
- Update upload endpoint to use configured limits instead of hardcoded 500MB
- Add comprehensive tests for upload limits and file splitting
- Document configuration in ConfigurationGuide.md and SECURITY_AUDIT.md
- Reference SECURITY_AUDIT.md in error messages for user guidance
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]