gitea-admin/gh-christianlouis-dmarq
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add m365 safe backfill windows

Browse Source
This commit is contained in:
Christian Krakau-Louis
2026-05-23 15:42:52 +02:00
parent 78df802a98
commit 8556ea30bf
8 changed files with 122 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/milestones.md
+1 -1
View File
@@ -206,7 +206,7 @@ Planned:
- Microsoft 365 mail source using OAuth (Graph) with least-privilege scopes. Delivered for delegated `User.Read`, `Mail.Read`, and `offline_access` with encrypted token storage, manual import, scheduled polling, UI setup, and operator docs.
- Shared mailbox and folder selection support for DMARC report collection. Delivered with shared mailbox targeting, Microsoft Graph folder listing, folder-id based imports, UI selection, and mailbox/folder context in import history.
- Import-history parity with existing sources (auditable attachment outcomes, duplicates, parse failures). Delivered for Microsoft 365 imports.
- Backfill support with safe throttling and progressive search windows.
- Backfill support with safe throttling and progressive search windows. Delivered with days-based Graph `receivedDateTime` filters, duplicate-safe reruns, and retry/backoff for throttled or temporarily unavailable Graph requests.
- Secret handling mirrors existing guidance (no raw secrets in logs; 1Password-friendly).
Exit criteria:
docs/user_guide/microsoft365.md
+3 -1
View File
@@ -39,10 +39,12 @@ DMARQ uses delegated Microsoft Graph access. That means the authorised Microsoft
## Import Behavior
DMARQ reads recent messages in the configured folder, filters for messages that look like DMARC reports, downloads Graph `fileAttachment` items, and sends `.xml`, `.zip`, `.gz`, and `.gzip` attachments through the same parser and persistence path used by upload, IMAP, and Gmail imports.
DMARQ reads messages in the configured search window, filters for messages that look like DMARC reports, downloads Graph `fileAttachment` items, and sends `.xml`, `.zip`, `.gz`, and `.gzip` attachments through the same parser and persistence path used by upload, IMAP, and Gmail imports.
Imported Graph message IDs are stored on the mail source so scheduled polling does not reprocess the same message. Import history records processed messages, imported reports, duplicates, parse failures, attachment-level details, and the mailbox/folder target used for the attempt.
Manual imports and backfills use the **days** value from the Mail Sources UI or trigger-poll endpoint. DMARQ passes that window to Microsoft Graph as a `receivedDateTime` filter, so large mailboxes can be searched without scanning unrelated older mail. If Graph returns a throttling or temporary service response, DMARQ retries with `Retry-After` or exponential backoff before surfacing the sanitized error in import history.
## Troubleshooting
- **Not authorised**: reconnect the source from Mail Sources.