feat(duplication): implement duplicate file handling and detection across processing steps

2026-02-12 02:37:51 +01:00
parent 70757e5644
commit 9a2c2d20b1
12 changed files with 325 additions and 28 deletions
@@ -207,6 +207,8 @@ The following is a summary of the licenses used by our direct dependencies:
 | OpenAI     | MIT     |
 | PyPDF2     | BSD     |
 | Requests   | Apache 2.0 |
+| puremagic  | MIT     |
+| filetype   | MIT     |
 | Dropbox    | MIT     |
 | Azure AI Document Intelligence | MIT |
 | Authlib    | BSD     |