Christian Krakau-Louis
93 lines
3.9 KiB
Markdown
93 lines
3.9 KiB
Markdown
# DMARQ Development Roadmap
|
|
|
|
Last updated: 2026-05-22
|
|
|
|
This roadmap tracks implementation status and near-term engineering priorities. The project has moved beyond the initial MVP and now needs production-grade import visibility, better reporting, and hardened operations.
|
|
|
|
## Current Status
|
|
|
|
Complete:
|
|
- Core DMARC aggregate parsing for XML, ZIP, and GZIP reports.
|
|
- Secure XML parsing with `defusedxml`.
|
|
- Upload validation and archive safety checks.
|
|
- IMAP mailbox ingestion.
|
|
- Gmail OAuth ingestion.
|
|
- Persistent database models and migrations for domains, reports, records, settings, users, and mail sources.
|
|
- Domain, report, settings, and mail source APIs.
|
|
- Dashboard and domain detail views.
|
|
- Logto auth integration and local development auth-disabled mode.
|
|
- DNS resolver foundation and DNS-related endpoint tests.
|
|
|
|
Recently improved:
|
|
- Gmail ingestion now matches the real Google DMARC report messages found in the connected inbox.
|
|
- Gmail import now uses the same parser path as uploads and IMAP.
|
|
- Gmail and IMAP imports now skip duplicate domain/report IDs.
|
|
- Tests cover Google-style DMARC ZIP attachment imports.
|
|
- Mail source imports now create sanitized import-history records for manual and scheduled polls.
|
|
|
|
Important gap:
|
|
- Parsed DMARC report data is still served primarily from the in-memory `ReportStore`. The database schema exists, but report upload/import paths and dashboard read paths must be completed before the persistence milestone can be called done.
|
|
|
|
## Active Milestone: Finish Report Persistence
|
|
|
|
Objective: complete the database-backed report storage promised by Milestone 3.
|
|
|
|
Priority tasks:
|
|
- Add a report persistence service that converts parsed DMARC report dicts into `Domain`, `DMARCReport`, and `ReportRecord` rows.
|
|
- Load or query persisted reports for dashboard, domain, and report endpoints.
|
|
- Add duplicate report detection against the database.
|
|
- Keep tests covering upload, Gmail import, IMAP import, and restart-style reload behavior.
|
|
|
|
Quality bar:
|
|
- Uploading or importing a report survives application restart and remains visible in report/domain endpoints.
|
|
|
|
## Next Milestone: Reporting Quality and Import Confidence
|
|
|
|
Objective: make mailbox imports auditable and make report totals trustworthy.
|
|
|
|
Priority tasks:
|
|
- Expand import attempts with message ID, source, attachment filename, outcome, and sanitized error details.
|
|
- Show import history on mail source detail pages.
|
|
- Report duplicate skips separately from parse failures.
|
|
- Add backfill controls for Gmail and IMAP sources.
|
|
- Improve source rollups so a source IP tracks pass/fail counts over time.
|
|
|
|
Quality bar:
|
|
- Importing the same mailbox twice must not change aggregate totals.
|
|
- Parse failures must be visible and actionable.
|
|
- The user should be able to tell whether a mail source is healthy without reading logs.
|
|
|
|
## Following Milestone: Meaningful Reports
|
|
|
|
Objective: turn parsed DMARC data into administrator-friendly reports.
|
|
|
|
Priority tasks:
|
|
- Add time-series charts for volume and compliance.
|
|
- Add per-domain daily rollups.
|
|
- Add sender/source breakdowns with SPF, DKIM, and disposition counts.
|
|
- Add "what changed" summaries for newly observed senders and sudden compliance drops.
|
|
- Add exportable reports for a domain and date range.
|
|
|
|
Quality bar:
|
|
- A domain owner can understand who sends mail as their domain, which sources fail, and what to fix next.
|
|
|
|
## Production Hardening
|
|
|
|
Objective: make self-hosted deployments safer.
|
|
|
|
Priority tasks:
|
|
- Document a 1Password secret-injection deployment flow.
|
|
- Keep raw secrets out of diagnostics, logs, and UI responses.
|
|
- Add startup validation for production settings.
|
|
- Add backup and restore documentation.
|
|
- Add a release checklist covering migrations, tests, and smoke checks.
|
|
|
|
## Later Milestones
|
|
|
|
- Notifications and alert rules with Apprise.
|
|
- DNS health and Cloudflare read-only inspection.
|
|
- Guided setup and operator health screens.
|
|
- Forensic/RUF report support.
|
|
|
|
See [milestones.md](../milestones.md) for the full milestone breakdown and exit criteria.
|