4 Commits

Author	SHA1	Message	Date
copilot-swe-agent[bot]	55543be3b0	fix(similarity): address code review - column-only queries, configurable batch size, WCAG touch targets ... - Use column-only query in embeddings overview to reduce memory for 100K+ files - Add embedding_backfill_batch_size config setting (default 50) - Fix WCAG touch target on backfill button (min-height/min-width 44px) - Add inline comment explaining 3 chars/token truncation estimate - Import settings in compute_embedding task for configurable batch size Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>	2026-03-02 13:39:10 +00:00
copilot-swe-agent[bot]	8e955f3c81	fix(similarity): truncate text to fit embedding model context window, fix step tracking ... - Add EMBEDDING_MAX_TOKENS config (default 8000) for safe text truncation - Use conservative 3 chars/token estimate (was 4) to prevent ContextWindowExceededError - Add compute_embedding to REAL_MAIN_STEPS in both get_file_overall_status and get_step_summary - Fix test_near_duplicates_returned to use pre-computed embeddings - Update .env.demo and docs with EMBEDDING_MAX_TOKENS setting Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>	2026-03-02 13:27:34 +00:00
copilot-swe-agent[bot]	8d7c8e7c4e	feat(similarity): add embedding pipeline, debug endpoints, backfill task, and scalable similarity search ... - Add embedding_model config setting (replaces hardcoded text-embedding-3-small) - Add compute_document_embedding Celery task for ingestion-time embedding - Chain embedding task into finalize_document_storage pipeline - Add backfill_missing_embeddings periodic task (every 5 min) for legacy files - Add debug API endpoints: embedding-status, compute-embedding, diagnostic/embeddings, diagnostic/compute-all-embeddings - Refactor find_similar_documents to only use pre-computed embeddings (no lazy API calls) - Use yield_per(500) and column-only queries for 100K+ scale - Add embedding status indicator and recompute button in file detail UI Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>	2026-03-02 13:01:57 +00:00
copilot-swe-agent[bot]	9748103782	feat(similarity): add document similarity detection with embeddings and cosine similarity ... Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>	2026-03-01 20:38:52 +00:00