Commit Graph

4 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 55543be3b0 fix(similarity): address code review - column-only queries, configurable batch size, WCAG touch targets
- Use column-only query in embeddings overview to reduce memory for 100K+ files
- Add embedding_backfill_batch_size config setting (default 50)
- Fix WCAG touch target on backfill button (min-height/min-width 44px)
- Add inline comment explaining 3 chars/token truncation estimate
- Import settings in compute_embedding task for configurable batch size

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:39:10 +00:00
copilot-swe-agent[bot] c724b8d83a feat(similarity): add similarity pairs dashboard, step tracking, and fix tests for pre-computed embeddings
- Add GET /api/similarity/pairs endpoint for corpus-wide pair discovery
- Add /similarity view route and similarity_dashboard.html template
- Add Similarity link to desktop and mobile nav menus
- Register compute_embedding as a tracked FileProcessingStep
- Update compute_embedding task with update_step_status calls
- Add compute_embedding to flow visualization in _compute_processing_flow
- Add backfill_missing_embeddings periodic beat task (every 5 min)
- Return clear message when embedding not yet computed in similar docs API
- Fix all tests to use pre-computed embeddings (no lazy API calls)
- Add tests for similarity pairs, backfill task, and embedding-not-computed

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:12:59 +00:00
copilot-swe-agent[bot] 8d7c8e7c4e feat(similarity): add embedding pipeline, debug endpoints, backfill task, and scalable similarity search
- Add embedding_model config setting (replaces hardcoded text-embedding-3-small)
- Add compute_document_embedding Celery task for ingestion-time embedding
- Chain embedding task into finalize_document_storage pipeline
- Add backfill_missing_embeddings periodic task (every 5 min) for legacy files
- Add debug API endpoints: embedding-status, compute-embedding, diagnostic/embeddings, diagnostic/compute-all-embeddings
- Refactor find_similar_documents to only use pre-computed embeddings (no lazy API calls)
- Use yield_per(500) and column-only queries for 100K+ scale
- Add embedding status indicator and recompute button in file detail UI

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-02 13:01:57 +00:00
copilot-swe-agent[bot] 9748103782 feat(similarity): add document similarity detection with embeddings and cosine similarity
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-03-01 20:38:52 +00:00