- Create detailed implementation summary document
- Document all deliverables and technical specifications
- Include code statistics and browser compatibility matrix
- List all requirements met and acceptance criteria satisfied
- Provide success metrics and next steps
- Mark feature as production-ready
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Fix response.json() called before checking response.ok in popup.js
- Consolidate duplicate onInstalled listeners in background.js
- Remove unnecessary return true from content.js message handler
- Add better error handling for non-JSON responses
- Improve user experience by not auto-opening popup on install
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add explicit return value to message listener in content.js
- Fix documentation references in QUICKSTART.md
- Clarify that GET_PAGE_INFO listener is reserved for future use
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Refactors .github/workflows/tests.yaml so that flake8, black, mypy,
pylint, and bandit each run as their own job in parallel with the test
job. This ensures a failure in one tool never blocks the others, and
contributors see full feedback from every tool on every CI run.
- Upgrades actions/checkout to v4 and actions/setup-python to v5
- All linter jobs are enforced (no continue-on-error)
- Test artifacts (junit.xml, coverage.xml) always uploaded
- Bandit JSON report always uploaded as artifact
- Adds docs/CIWorkflow.md with maintainer documentation
- Updates CONTRIBUTING.md with CI workflow table and link
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Create comprehensive visual guide showing all UI states
- Document color scheme, typography, and accessibility features
- Include ASCII art mockups of popup interface
- Add browser support matrix and performance metrics
- Document user flow and security indicators
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add complete browser extension with popup UI and background workers
- Support Chrome, Firefox, Edge, and other Chromium-based browsers
- Include context menu integration for quick file sending
- Add comprehensive documentation for users and administrators
- Update main README and API docs to include browser extension
- Implement secure configuration storage in browser extension storage
- Add SSRF-protected URL upload endpoint integration
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Update test_bulk_operations.py: Use FileProcessingStep instead of ProcessingLog for status filter tests
- Update test_file_listing.py: Use FileProcessingStep for processing status determination tests
- Update test_file_detail_endpoints.py: Replace hash_file with check_text in step summary test
- Update test_path_traversal_security.py: Import get_unique_filepath_with_counter from correct module
- Update test_process_document.py: Match current duplicate handling behavior (creates new record)
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace ProcessingLog-based tests with FileProcessingStep-based tests
- Update to use step_manager functions (get_file_overall_status, get_step_summary)
- All 6 tests (3 status + 3 metrics) now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace ProcessingLog with FileProcessingStep in sample_files fixture
- Update test comments to reflect using steps instead of logs
- All 10 file_queries tests now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace "hash_file" test step with actual MAIN_PROCESSING_STEPS names
- Fix test_get_step_summary to use upload_to_* instead of queue_* for upload counts
- All 12 step_manager tests now pass
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Implement step timeout detection to prevent files from getting stuck in 'pending' state
- Add monitor_stalled_steps periodic task running every minute (Celery Beat)
- Automatically mark in-progress steps as failed if they exceed timeout (default: 10 minutes)
- Add step_timeout configuration setting (default: 600 seconds)
- Recover stalled steps with error message indicating when timeout was triggered
- Fix duplicate check to exclude self-comparison (file not duplicate of itself)
When processing crashes or hangs:
1. Worker detects stalled steps (in_progress for >10 minutes)
2. Marks them as failed with timeout error message
3. Updates UI to show failure status
4. Allows file to be retried or handled by user
This prevents files from being indefinitely stuck in processing state and provides
visibility into what went wrong.
- Remove duplicate record creation to avoid UNIQUE constraint on filehash
- When duplicate detected, return original file_id instead of creating new record
- Avoids sqlite3.IntegrityError: UNIQUE constraint failed
- Simpler approach: duplicates not tracked as separate records, just rejected
- Revert filehash column back to NOT NULL (required for original files)
- Fixes error: (sqlite3.IntegrityError) UNIQUE constraint failed: files.filehash
- Implement lightweight migration system for is_duplicate and duplicate_of_id columns
- Migrations run automatically on application startup
- Idempotent migrations safe to run multiple times
- Fixes SQLite OperationalError for missing columns
- Resolves issue where database schema didn't match model definitions
- Add enable_deduplication and show_deduplication_step config options
- Rename hash_file step to check_for_duplicates
- Make deduplication step conditional based on configuration
- Add is_duplicate and duplicate_of_id fields to FileRecord model
- Create database migration for new deduplication fields
- Update process_document task to log deduplication results
- Update step visualization to show/hide step based on config
- Update status calculations to include deduplication step conditionally
- Default: deduplication enabled, step displayed
- Can be configured to hide from UI while still processing
- Added endpoints for previewing original and processed PDF files.
- Implemented on-demand text extraction from original and processed PDFs.
- Updated file detail page to show original and processed file paths with existence status.
- Introduced GPT metadata display with a collapsible JSON view.
- Enhanced front-end with PDF.js for in-browser PDF rendering and improved user experience.
- Added integration tests for new features including metadata display and file previews.
- Replace unique_filepath with get_unique_filepath_with_counter in tests
- Update test expectations for -0001 suffix format
- All 40 tests passing successfully
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add original_file_path and processed_file_path columns to FileRecord model
- Create database migration for new fields
- Implement get_unique_filepath_with_counter() with -0001 suffix format
- Update process_document to save immutable copy to /workdir/original
- Add force_cloud_ocr parameter to process_document for forced OCR
- Update embed_metadata to use new collision handling
- Update metadata JSON to include file path references
- Add /files/{file_id}/reprocess-with-cloud-ocr API endpoint
- Update processed_file_path in database during embedding
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Christian Krakau-Louis