feat(similarity): add similarity pairs dashboard, step tracking, and fix tests for pre-computed embeddings

gitea-admin/gh-christianlouis-docuelevate

- Add GET /api/similarity/pairs endpoint for corpus-wide pair discovery
- Add /similarity view route and similarity_dashboard.html template
- Add Similarity link to desktop and mobile nav menus
- Register compute_embedding as a tracked FileProcessingStep
- Update compute_embedding task with update_step_status calls
- Add compute_embedding to flow visualization in _compute_processing_flow
- Add backfill_missing_embeddings periodic beat task (every 5 min)
- Return clear message when embedding not yet computed in similar docs API
- Fix all tests to use pre-computed embeddings (no lazy API calls)
- Add tests for similarity pairs, backfill task, and embedding-not-computed

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

This commit is contained in:

copilot-swe-agent[bot]

2026-03-02 13:12:59 +00:00

parent 8d7c8e7c4e

commit c724b8d83a

7 changed files with 811 additions and 43 deletions

									
										app/utils/step_manager.py
									
		+1
		
												View File
												
				@@ -23,6 +23,7 @@ BASE_MAIN_PROCESSING_STEPS = [

				    "embed_metadata_into_pdf",

				    "finalize_document_storage",

				    "send_to_all_destinations",

				    "compute_embedding",

				]

				OPTIONAL_PROCESSING_STEPS = {