{% extends "base.html" %} {% block title %}Duplicate Documents - DocuElevate{% endblock %} {% block head_extra %} {% endblock %} {% block content %}

Duplicate Documents

Documents with identical content (same SHA-256 hash) are shown as exact duplicates. Use the Near-Duplicate Finder tab to detect documents with the same scanned content but different hashes — for example, a document scanned twice.

{% if groups %}

Found {{ total_groups }} duplicate group(s) with {{ total_duplicate_files }} duplicate file(s) total.

{% for group in groups %}
{{ group.filehash[:16] }}… {{ group.duplicate_count }} duplicate{{ 's' if group.duplicate_count != 1 else '' }}
{% if group.original %}
Original
ID: {{ group.original.id }} {% if group.original.file_size %} · {{ (group.original.file_size / 1024)|round(1) }} KB {% endif %} {% if group.original.created_at %} · {{ group.original.created_at[:19].replace('T',' ') }} {% endif %}
{% endif %} {% for dup in group.duplicates %}
Duplicate
ID: {{ dup.id }} {% if dup.file_size %} · {{ (dup.file_size / 1024)|round(1) }} KB {% endif %} {% if dup.created_at %} · {{ dup.created_at[:19].replace('T',' ') }} {% endif %}
{% endfor %}
{% endfor %} {% if pagination.pages > 1 %} {% endif %} {% else %}

No exact duplicates found

Every file in the system has a unique SHA-256 hash. Use the Near-Duplicate Finder tab to check for re-scanned documents.

{% endif %}

How near-duplicate detection works

  • After OCR processes a document, its text is converted to a numeric embedding vector using an AI language model.
  • Near-duplicate detection compares these vectors using cosine similarity: a score of 1.0 means identical content, 0.0 means completely different.
  • A threshold of {{ near_duplicate_threshold }} means documents must share at least {{ (near_duplicate_threshold * 100)|round|int }}% semantic similarity to be flagged.
  • Documents scanned twice (even with slight differences) will typically score > 0.90.
{% endblock %} {% block scripts %} {% endblock %}