feat: add source pass fail rollups

This commit is contained in:
Christian Krakau-Louis
2026-05-22 20:35:56 +02:00
parent 2165fb1054
commit 14f9f6defd
12 changed files with 503 additions and 65 deletions
+16 -15
View File
@@ -31,37 +31,38 @@ Recently improved:
- Import-history rows include sanitized per-attachment outcomes and imported report IDs.
- Mail source backfills can be launched from the UI with configurable search windows.
- The current Alpine-based UI is allowed by CSP and renders dynamic tables in real browsers.
- Sending-source summaries now retain SPF, DKIM, DMARC, and disposition pass/fail totals per IP instead of showing only the latest result.
Implementation note:
- The legacy `ReportStore` remains as a projection layer for existing report/dashboard code, but durable report data now lives in the database.
## Active Milestone: Reporting Quality and Import Confidence
Objective: make mailbox imports auditable and make report totals trustworthy.
Priority tasks:
- Report duplicate skips separately from parse failures.
- Improve source rollups so a source IP tracks pass/fail counts over time.
Quality bar:
- Importing the same mailbox twice must not change aggregate totals.
- Parse failures must be visible and actionable.
- The user should be able to tell whether a mail source is healthy without reading logs.
## Next Milestone: Meaningful Reports
## Active Milestone: Meaningful Reports
Objective: turn parsed DMARC data into administrator-friendly reports.
Priority tasks:
- Add time-series charts for volume and compliance.
- Add per-domain daily rollups.
- Add sender/source breakdowns with SPF, DKIM, and disposition counts.
- Add "what changed" summaries for newly observed senders and sudden compliance drops.
- Add exportable reports for a domain and date range.
- Add actionable recommendations for common SPF, DKIM, and DMARC failure patterns.
Quality bar:
- A domain owner can understand who sends mail as their domain, which sources fail, and what to fix next.
## Completed Milestone: Reporting Quality and Import Confidence
Objective: make mailbox imports auditable and make report totals trustworthy.
Delivered:
- Duplicate skips are reported separately from parse failures.
- Import history exposes per-attachment results and sanitized errors.
- Individual sources can be manually imported and backfilled from the UI.
- Source rollups track pass/fail counts over time by sender IP.
Quality bar:
- Importing the same mailbox twice does not change aggregate totals, parse failures are visible, and source totals are not overwritten by the latest result.
## Production Hardening
Objective: make self-hosted deployments safer.
+6 -8
View File
@@ -49,13 +49,13 @@ Delivered:
Implementation note:
- The existing `ReportStore` remains as a compatibility projection for dashboard/report code, but persisted database rows are now the durable source for uploads and mailbox imports.
## Milestone 4: Reporting Quality and Import Confidence - In Progress
## Milestone 4: Reporting Quality and Import Confidence - Complete
Status: In progress
Status: Complete
Goal: make reports trustworthy for day-to-day administration and make import failures obvious.
Recently delivered:
Delivered:
- Gmail import now handles real inbox metadata patterns and Google-style ZIP filenames.
- Gmail/IMAP imports skip duplicate report IDs to avoid inflated totals.
- Tests now cover a real Google-style ZIP attachment path rather than only mocked parser behavior.
@@ -66,16 +66,14 @@ Recently delivered:
- Import history now includes per-attachment details for imported reports, duplicates, parse errors, unsupported attachments, and imported report IDs.
- Mail sources can be backfilled from the UI with 7-day, 30-day, 90-day, or custom search windows.
- The current Alpine-based UI can run under the configured CSP, so dynamic tables render in real browsers.
Next tasks:
- Improve source aggregation so each sender IP keeps pass/fail totals instead of only the latest result.
- Source aggregation now keeps per-sender-IP SPF, DKIM, DMARC, and disposition totals instead of overwriting each IP with only the latest result.
Exit criteria:
- A user can connect a mailbox, run a backfill, see exactly what was imported or skipped, and trust that totals are not double-counted.
## Milestone 5: Dashboard and Meaningful Reports - Next
## Milestone 5: Dashboard and Meaningful Reports - In Progress
Status: Planned
Status: In progress
Goal: convert raw DMARC data into useful operational reporting.
+1 -1
View File
@@ -149,7 +149,7 @@ This file tracks the specific implementation tasks for each milestone of the DMA
### Meaningful Reports
- [ ] Add per-domain daily rollups
- [ ] Add sender/source pass/fail totals
- [x] Add sender/source pass/fail totals
- [ ] Add newly observed source detection
- [ ] Add exportable domain reports
- [ ] Add actionable recommendations for common DMARC failure patterns