feat(duplicates): add duplicate document detection and management

- Add near_duplicate_threshold config setting (default 0.85)
- New GET /api/duplicates endpoint listing all exact-duplicate groups
- New GET /api/files/{id}/duplicates endpoint returning exact + near-duplicates
- POST /api/ui-upload now returns immediate exact-duplicate warning (respects ENABLE_DEDUPLICATION)
- New /duplicates management UI with Exact Duplicates tab and Near-Duplicate Finder tab
- Add Duplicates link in admin nav menu (desktop + mobile)
- Document new config options in ConfigurationGuide.md and .env.demo
- 20 new tests covering all acceptance criteria

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
2026-03-02 10:21:57 +00:00
parent 3361042186
commit ebea83a750
10 changed files with 1293 additions and 3 deletions
+2
View File
@@ -9,6 +9,7 @@ from fastapi import APIRouter
from app.api.azure import router as azure_router
from app.api.diagnostic import router as diagnostic_router
from app.api.dropbox import router as dropbox_router
from app.api.duplicates import router as duplicates_router
from app.api.files import router as files_router
from app.api.google_drive import router as google_drive_router
from app.api.logs import router as logs_router
@@ -49,4 +50,5 @@ router.include_router(search_router)
router.include_router(queue_router)
router.include_router(saved_searches_router)
router.include_router(similarity_router)
router.include_router(duplicates_router)
router.include_router(webhooks_router)