- Add date range (date_from/date_to), storage provider, and tags filters to GET /api/files
- Add SavedSearch model and migration (005_add_saved_searches)
- Add CRUD API endpoints for saved searches at /api/saved-searches
- Update files.html template with new filter controls and saved searches UI
- Update files view to pass new filter parameters to template
- Add comprehensive tests for all new functionality (26 tests)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Revert search.html and files.html links back to /files/{id}
- Add GET /files/{file_id} route (file_view_page) with workdir path-
containment guards (os.path.commonpath) to prevent traversal
- Create file_view.html: document-centric page showing AI metadata,
inline PDF preview, extracted OCR text, download actions, file info,
status pill, and link to /files/{id}/detail for process pipeline view
- Existing /files/{id}/detail route is unchanged
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Update _compute_processing_flow to recognize process_with_ocr as the OCR
stage and remap legacy process_with_azure_document_intelligence log entries
for backward compatibility
- Normalize legacy OCR step name in _compute_step_summary log fallback
- Add process_with_ocr to REAL_MAIN_STEPS/REAL_STEPS in step_manager,
file_status, and file_queries (keeping legacy name for old DB entries)
- Update retry logic in api/files.py to retry failed OCR via process_with_ocr
(handles both step names as aliases)
- Fix process_document.py to log process_with_ocr as skipped (not azure step)
for the local text extraction path
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add enable_deduplication and show_deduplication_step config options
- Rename hash_file step to check_for_duplicates
- Make deduplication step conditional based on configuration
- Add is_duplicate and duplicate_of_id fields to FileRecord model
- Create database migration for new deduplication fields
- Update process_document task to log deduplication results
- Update step visualization to show/hide step based on config
- Update status calculations to include deduplication step conditionally
- Default: deduplication enabled, step displayed
- Can be configured to hide from UI while still processing
- Added endpoints for previewing original and processed PDF files.
- Implemented on-demand text extraction from original and processed PDFs.
- Updated file detail page to show original and processed file paths with existence status.
- Introduced GPT metadata display with a collapsible JSON view.
- Enhanced front-end with PDF.js for in-browser PDF rendering and improved user experience.
- Added integration tests for new features including metadata display and file previews.
- Fix _compute_status_from_logs to track latest status per unique step
- Fix _compute_step_summary to count only latest status per step
- Add comprehensive tests for both fixes
- Resolves issue where completed files showed as "Processing"
- Resolves issue where metrics showed incorrect counts
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Created app/utils/file_queries.py with apply_status_filter function
- Updated app/api/files.py to use shared function
- Updated app/views/files.py to use shared function
- Removed unused 'or_' import from app/api/files.py
- Added comprehensive tests in tests/test_file_queries.py
- All tests pass (10 new tests, 14 existing tests verified)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Run Black formatter and isort on all app/ files
- Remove unused imports (F401) across multiple files
- Add # noqa: F401 for intentional re-exports in celery_worker.py,
tasks/__init__.py, utils.py, frontend.py, views/base.py
- Fix f-strings without placeholders (F541) in azure.py, notification.py,
check_credentials.py, upload_to_onedrive.py, settings.py
- Fix bare except (E722) in upload_to_sftp.py
- Fix block comment format (E265) in models.py
- Move imports to top of file to fix E402 in celery_app.py, celery_worker.py
- Fix line-too-long (E501) by wrapping strings in multiple files
- Remove unused variable (F841) in upload_to_nextcloud.py
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add Redis and RabbitMQ services to CI workflow
- Fix Jinja2 template error by passing file=None in error cases
- Fix test expecting dict response format for list_files endpoint
- Fix NOT NULL constraint by providing valid local_filename
- Fix retry-subtask to validate subtask name before checking processed file
- Add mock for process_document in reprocess test
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Added /api/files/{file_id}/reprocess endpoint for single file reprocessing
- Added /api/files/{file_id}/preview endpoint for viewing original/processed files
- Enhanced file detail view with process flow computation
- Updated frontend template with retry button, process flow visualization, and PDF previews
- Added JavaScript for async retry functionality
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fixed status filter in /files view endpoint
- Added bulk delete and reprocess API endpoints
- Added bulk selection UI with checkboxes
- Added bulk actions bar with reprocess and delete buttons
- Updated JavaScript to handle bulk operations
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fixed status filtering to occur before pagination for correct counts
- Resolved N+1 query problem by batch-fetching processing statuses
- Extracted status computation logic to shared utility function
- Changed sort indicator from ⬍ to ↕ for better browser compatibility
- Updated both API and view layers to use shared status utilities
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Updated /api/files endpoint with pagination, filtering, and sorting support
- Added /api/files/{file_id} endpoint for detailed file information
- Updated /files view to support server-side operations
- Added /files/{file_id}/detail route for file detail page
- Created new files.html with filters, status column, and pagination
- Created file_detail.html for viewing processing history
- Status computed from ProcessingLog entries (pending, processing, completed, failed)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]