- Move get_all_settings_from_db import to module level in app/views/settings.py
- Move get_provider_status and get_settings_for_display imports to module level in app/views/status.py
- Fix CI workflow: replace deprecated 'file' parameter with 'files' in codecov-action
- Fix CI workflow: update test results upload to use codecov-action@v5 with report_type instead of deprecated test-results-action@v1
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix PLW2901: Use different variable name for stripped lines in loop
- Fix E721: Use 'is' instead of '==' for type comparisons
- Add noqa comments for intentional security warnings (S321, S507, S110, S603)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add enable_deduplication and show_deduplication_step config options
- Rename hash_file step to check_for_duplicates
- Make deduplication step conditional based on configuration
- Add is_duplicate and duplicate_of_id fields to FileRecord model
- Create database migration for new deduplication fields
- Update process_document task to log deduplication results
- Update step visualization to show/hide step based on config
- Update status calculations to include deduplication step conditionally
- Default: deduplication enabled, step displayed
- Can be configured to hide from UI while still processing
- Added endpoints for previewing original and processed PDF files.
- Implemented on-demand text extraction from original and processed PDFs.
- Updated file detail page to show original and processed file paths with existence status.
- Introduced GPT metadata display with a collapsible JSON view.
- Enhanced front-end with PDF.js for in-browser PDF rendering and improved user experience.
- Added integration tests for new features including metadata display and file previews.
- Fix _compute_status_from_logs to track latest status per unique step
- Fix _compute_step_summary to count only latest status per step
- Add comprehensive tests for both fixes
- Resolves issue where completed files showed as "Processing"
- Resolves issue where metrics showed incorrect counts
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Created app/utils/file_queries.py with apply_status_filter function
- Updated app/api/files.py to use shared function
- Updated app/views/files.py to use shared function
- Removed unused 'or_' import from app/api/files.py
- Added comprehensive tests in tests/test_file_queries.py
- All tests pass (10 new tests, 14 existing tests verified)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Redis and RabbitMQ services to CI workflow
- Fix Jinja2 template error by passing file=None in error cases
- Fix test expecting dict response format for list_files endpoint
- Fix NOT NULL constraint by providing valid local_filename
- Fix retry-subtask to validate subtask name before checking processed file
- Add mock for process_document in reprocess test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Remove HTML 'required' attributes - all fields optional
- Add source detection (DB/ENV/DEFAULT) for each setting
- Display color-coded badges showing setting source
- Update template with precedence order explanation
- Pre-fill form with current values from DB/ENV/defaults
- Update documentation with source badge explanations
- Test and verify form prefilling works correctly
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Convert require_admin_access to proper decorator pattern
- Fix redirect loop that was sending all users to /
- Add is_admin flag handling for OAuth users (checks groups)
- Update SETTING_METADATA with all 102 settings from config.py
- Improve API admin check with type hints
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added /api/files/{file_id}/reprocess endpoint for single file reprocessing
- Added /api/files/{file_id}/preview endpoint for viewing original/processed files
- Enhanced file detail view with process flow computation
- Updated frontend template with retry button, process flow visualization, and PDF previews
- Added JavaScript for async retry functionality
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fixed status filter in /files view endpoint
- Added bulk delete and reprocess API endpoints
- Added bulk selection UI with checkboxes
- Added bulk actions bar with reprocess and delete buttons
- Updated JavaScript to handle bulk operations
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fixed status filtering to occur before pagination for correct counts
- Resolved N+1 query problem by batch-fetching processing statuses
- Extracted status computation logic to shared utility function
- Changed sort indicator from ⬍ to ↕ for better browser compatibility
- Updated both API and view layers to use shared status utilities
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Updated /api/files endpoint with pagination, filtering, and sorting support
- Added /api/files/{file_id} endpoint for detailed file information
- Updated /files view to support server-side operations
- Added /files/{file_id}/detail route for file detail page
- Created new files.html with filters, status column, and pagination
- Created file_detail.html for viewing processing history
- Status computed from ProcessingLog entries (pending, processing, completed, failed)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added a new license route to serve the LGPL license text.
- Introduced a new attribution page to acknowledge third-party software used in the project.
- Updated the base HTML template to include a link to the attribution page.
- Included the license router in the main application router.
- Added the license text file for LGPL to the static licenses directory.
- Updated the NOTICE file to include detailed attributions for third-party libraries.
- Added a new requirements-dev.txt for license compliance checking.
- Updated the requirements.txt to clarify the LGPL license for Paramiko.
- Implemented `google_drive_callback.html` for processing Google Drive authorization, including UI for success and error states.
- Added JavaScript functionality for exchanging authorization codes, saving settings, and handling folder selection.
- Created `google_drive_callback_error.html` to display error messages during the authorization process.
- Added a script to save the build date during Docker image build.
- Introduced a build_date property in the Settings class to retrieve the build date from the environment or file.
- Enhanced the homepage to display system statistics including processed files and active integrations.
- Updated the about page with a comprehensive list of features.
- Added new privacy, imprint, cookies, and terms pages with relevant content.
- Improved the license page with related information links.
- Refactored the base template for better mobile responsiveness and footer links.
copilot-swe-agent[bot]