Duplicate Documents
Documents with identical content (same SHA-256 hash) are shown as exact duplicates. Use the Near-Duplicate Finder tab to detect documents with the same scanned content but different hashes — for example, a document scanned twice.
Found {{ total_groups }} duplicate group(s) with {{ total_duplicate_files }} duplicate file(s) total.
{% for group in groups %}No exact duplicates found
Every file in the system has a unique SHA-256 hash. Use the Near-Duplicate Finder tab to check for re-scanned documents.
Find Near-Duplicates for a Document
Select a document to check whether any other files contain the same (or very similar) content — even if they were scanned at different times and have different SHA-256 hashes. Similarity is computed from OCR text embeddings; documents without extracted text cannot be compared.
How near-duplicate detection works
- After OCR processes a document, its text is converted to a numeric embedding vector using an AI language model.
- Near-duplicate detection compares these vectors using cosine similarity: a score of 1.0 means identical content, 0.0 means completely different.
- A threshold of {{ near_duplicate_threshold }} means documents must share at least {{ (near_duplicate_threshold * 100)|round|int }}% semantic similarity to be flagged.
- Documents scanned twice (even with slight differences) will typically score > 0.90.