Christian Krakau-Louis
163 lines
5.5 KiB
Markdown
163 lines
5.5 KiB
Markdown
# DMARQ Implementation Todo List
|
|
|
|
This file tracks the specific implementation tasks for each milestone of the DMARQ project.
|
|
|
|
## Milestone 1: Minimal Viable Product (MVP)
|
|
|
|
### Infrastructure Setup
|
|
- [x] Set up FastAPI project structure
|
|
- [x] Configure Tailwind CSS
|
|
- [x] Create Docker and Docker Compose files
|
|
- [ ] Set up CI/CD pipeline (optional for MVP)
|
|
|
|
### Core DMARC Parser
|
|
- [x] Integrate parsedmarc library
|
|
- [x] Create parsing service for DMARC XML reports
|
|
- [x] Add support for ZIP/GZ compression extraction
|
|
- [x] Implement validation for uploaded reports
|
|
|
|
### Data Models
|
|
- [x] Create Domain model
|
|
- [x] Create AggregateReport model
|
|
- [x] Create ReportRecord model for individual sending sources
|
|
- [x] Design in-memory storage for MVP phase
|
|
|
|
### API Endpoints
|
|
- [x] Create domain registration endpoint
|
|
- [x] Create report upload endpoint
|
|
- [x] Create domain summary endpoints
|
|
- [x] Create detailed report view endpoints
|
|
|
|
### Frontend
|
|
- [x] Create base layout template
|
|
- [x] Implement dashboard overview page
|
|
- [x] Create domain list component
|
|
- [x] Build report upload interface
|
|
- [x] Implement domain detail view
|
|
- [x] Create report detail view
|
|
- [x] Add basic visualization components for report statistics
|
|
|
|
### Testing
|
|
- [x] Create unit tests for parser
|
|
- [x] Create API tests
|
|
- [x] Collect sample DMARC reports for testing
|
|
- [x] Manual UI testing
|
|
|
|
### Documentation
|
|
- [x] Create user guide for MVP
|
|
- [x] Document deployment instructions
|
|
- [x] Add sample screenshots
|
|
|
|
## Milestone 2: IMAP Integration
|
|
|
|
### IMAP Client
|
|
- [x] Create IMAP connection service
|
|
- [x] Implement mailbox search functionality for DMARC reports
|
|
- [x] Add attachment extraction capabilities
|
|
- [x] Create email filtering logic (by sender, subject)
|
|
- [x] Add processed email tracking
|
|
|
|
### Scheduler
|
|
- [x] Implement background task system
|
|
- [x] Create scheduler for periodic mailbox checking
|
|
- [x] Add timestamp tracking for fetched reports
|
|
- [x] Create logging for background processes
|
|
|
|
### Configuration
|
|
- [x] Create configuration model for IMAP settings
|
|
- [x] Build configuration UI
|
|
- [x] Implement secure credential storage
|
|
- [x] Add connection testing functionality
|
|
|
|
### Frontend Updates
|
|
- [x] Add IMAP configuration page
|
|
- [x] Create last sync indicator
|
|
- [x] Implement manual sync trigger button
|
|
- [x] Add status indicators for background processes
|
|
|
|
## Milestone 2A: Gmail and Mail Import Reliability
|
|
|
|
### Gmail API Import
|
|
- [x] Create Gmail OAuth mail source flow
|
|
- [x] Search Gmail for likely DMARC aggregate report messages
|
|
- [x] Match Google DMARC subjects like `Report domain: example.com Submitter: google.com`
|
|
- [x] Parse Google-style ZIP attachments such as `google.com!example.com!begin!end.zip`
|
|
- [x] Use the shared DMARC parser for Gmail imports
|
|
- [x] Track ingested Gmail message IDs
|
|
|
|
### Duplicate and Error Handling
|
|
- [x] Skip duplicate domain/report IDs during Gmail imports
|
|
- [x] Skip duplicate domain/report IDs during IMAP imports
|
|
- [x] Persist sanitized import errors for API/UI review
|
|
- [x] Count duplicate skips separately from parse failures
|
|
- [x] Show recent import history in the Mail Sources UI
|
|
- [x] Add manual import trigger per mail source
|
|
- [x] Add per-import result details for duplicates, parse failures, unsupported attachments, and imported report IDs
|
|
- [x] Add retry/backfill controls per mail source
|
|
|
|
## Milestone 3: Database Integration
|
|
|
|
### Database Setup
|
|
- [x] Set up SQLAlchemy ORM
|
|
- [x] Create SQLite database (for initial version)
|
|
- [x] Design database schema with migrations
|
|
- [x] Implement data access layer
|
|
|
|
### Model Migration
|
|
- [x] Convert report ingestion and dashboard reads from in-memory storage to database-backed storage
|
|
- [x] Create Domain table
|
|
- [x] Create AggregateReport table
|
|
- [x] Create ReportRecord table for sender details
|
|
- [x] Implement relationships between models
|
|
- [x] Persist parsed upload reports to `dmarc_reports` and `report_records`
|
|
- [x] Persist parsed Gmail/IMAP reports to `dmarc_reports` and `report_records`
|
|
- [x] Load/query persisted reports after app restart
|
|
|
|
### Domain Management
|
|
- [x] Create UI for adding/editing domains
|
|
- [x] Implement domain validation
|
|
- [x] Add domain deletion with data cleanup
|
|
- [x] Create domain filtering/search for larger sets
|
|
|
|
### Query Optimization
|
|
- [x] Add pagination for large report sets
|
|
- [x] Implement efficient queries for dashboard stats
|
|
- [x] Create data summarization for performance
|
|
- [x] Add database indexes for common queries
|
|
|
|
## Milestone 4: Dashboard Enhancements
|
|
|
|
### Data Visualization
|
|
- [x] Integrate Chart.js library
|
|
- [x] Create time-series charts for DMARC compliance
|
|
- [x] Add volume charts for email traffic
|
|
- [ ] Implement sender breakdown visualizations
|
|
- [ ] Create policy distribution charts
|
|
|
|
### Dashboard Widgets
|
|
- [ ] Create compliance rate summary widget
|
|
- [ ] Add enforcement rate widget
|
|
- [x] Implement email volume trends widget
|
|
- [ ] Create top sender sources widget
|
|
- [ ] Add alert status summary (for later integration)
|
|
|
|
### Historical Data
|
|
- [ ] Implement date range filtering
|
|
- [ ] Create historical trend analysis
|
|
- [ ] Add data aggregation for different time periods
|
|
- [ ] Implement data comparison features
|
|
|
|
### Meaningful Reports
|
|
- [ ] Add per-domain daily rollups
|
|
- [x] Add sender/source pass/fail totals
|
|
- [ ] Add newly observed source detection
|
|
- [ ] Add exportable domain reports
|
|
- [ ] Add actionable recommendations for common DMARC failure patterns
|
|
|
|
## Future Milestones
|
|
- [ ] Production secret handling guide using 1Password injection
|
|
- [ ] Apprise notifications and alert rules
|
|
- [ ] DNS health guidance and Cloudflare read-only inspection
|
|
- [ ] Guided setup and operator health pages
|
|
- [ ] Forensic/RUF report support
|