Replace the fragile global DOM query approach in `updateOverallStatus`
with per-batch closure-based counters inside `processFiles`.
- Add `total`, `done`, `updateStatus()`, and `markDone()` as closure
variables/functions within each `processFiles` invocation
- Change `_uploadSingleFile` to accept an `onTerminal` callback instead
of `statusMessage`, called when a file reaches a terminal state
- Pass `markDone` as the `onTerminal` callback from `scheduleNext`
- Remove the now-unused global `updateOverallStatus` function
The previous implementation queried `document.querySelectorAll('.file-status')`
globally and relied on text `startsWith` checks to count completed files.
This was fragile and could produce a stale done=0 count in practice.
The new approach uses deterministic closure counters, so the displayed
"Uploading files (X/N)" count correctly increments as files complete.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Add imports for convert_to_pdfa, finalize_document_storage,
process_with_ocr, upload_with_rclone, and webhook_tasks which
were missing from celery_worker.py, causing "unregistered task"
errors at runtime.
Add dynamic test that discovers all task modules in app/tasks/
and verifies each is imported in celery_worker.py.
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Replace with CSS spacing classes for accessibility
- Use |tojson filter for search index to prevent XSS
- Add IntersectionObserver cleanup via Alpine $cleanup
- Add sr-only setting key text for mobile screen readers
- Respect prefers-reduced-motion for smooth scrolling
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add real-time search/filter bar for finding settings by name, key, or description
- Add sidebar navigation with category icons and setting counts
- Make category sections collapsible with smooth animations
- Add mobile-friendly category dropdown selector
- Show setting key as code badge for quick reference
- Compact header with inline precedence/legend info
- Add intersection observer for active category tracking in sidebar
- Add no-results state with clear search action
- Maintain all existing functionality (save, revert, bulk save, alerts)
- Full dark mode compatibility via existing CSS overrides
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add EMBEDDING_MAX_TOKENS config (default 8000) for safe text truncation
- Use conservative 3 chars/token estimate (was 4) to prevent ContextWindowExceededError
- Add compute_embedding to REAL_MAIN_STEPS in both get_file_overall_status and get_step_summary
- Fix test_near_duplicates_returned to use pre-computed embeddings
- Update .env.demo and docs with EMBEDDING_MAX_TOKENS setting
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
- Add GET /api/similarity/pairs endpoint for corpus-wide pair discovery
- Add /similarity view route and similarity_dashboard.html template
- Add Similarity link to desktop and mobile nav menus
- Register compute_embedding as a tracked FileProcessingStep
- Update compute_embedding task with update_step_status calls
- Add compute_embedding to flow visualization in _compute_processing_flow
- Add backfill_missing_embeddings periodic beat task (every 5 min)
- Return clear message when embedding not yet computed in similar docs API
- Fix all tests to use pre-computed embeddings (no lazy API calls)
- Add tests for similarity pairs, backfill task, and embedding-not-computed
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
copilot-swe-agent[bot]