Christian Krakau-Louis
173 lines
6.8 KiB
Markdown
173 lines
6.8 KiB
Markdown
# DMARQ Architecture
|
|
|
|
## System Overview
|
|
|
|
DMARQ is designed as a self-contained application that processes DMARC reports, stores relevant data, and presents insights through a web interface. The architecture follows a layered approach with clear separation of concerns between components.
|
|
|
|
## Core Components
|
|
|
|
### Web Application (FastAPI)
|
|
- Serves the web interface
|
|
- Handles API requests
|
|
- Manages user authentication
|
|
- Coordinates background tasks
|
|
- Renders templates with Jinja2
|
|
|
|
### DMARC Processing Engine
|
|
- Parses DMARC XML reports
|
|
- Extracts meaningful data from reports
|
|
- Validates report structure
|
|
- Identifies sending sources
|
|
- Calculates compliance metrics
|
|
|
|
### Data Storage
|
|
- **MVP Phase**: In-memory storage
|
|
- **Later Phases**: SQLite or PostgreSQL database
|
|
- Stores domain configurations
|
|
- Maintains report history
|
|
- Tracks sender statistics
|
|
|
|
### Report Acquisition
|
|
- **MVP Phase**: Manual file upload
|
|
- **Later Phases**: IMAP client for automatic retrieval
|
|
- Handles compression formats (ZIP, GZ)
|
|
- Deduplicates reports
|
|
|
|
### Background Processing
|
|
- Scheduled report fetching
|
|
- Periodic DNS checks
|
|
- Alert evaluation
|
|
- Data aggregation for dashboards
|
|
|
|
## Data Flow
|
|
|
|
1. **Report Ingestion**
|
|
- Reports arrive via upload or IMAP
|
|
- System extracts and validates XML content
|
|
- Parser processes report data
|
|
- Data is stored in appropriate format
|
|
|
|
2. **Data Processing**
|
|
- Raw report data is transformed into metrics
|
|
- System calculates compliance rates
|
|
- Identifies new or problematic senders
|
|
- Updates historical records
|
|
|
|
3. **Presentation Layer**
|
|
- Dashboard displays key metrics
|
|
- Domain details show specific report data
|
|
- Charts visualize trends
|
|
- Alerts highlight issues requiring attention
|
|
|
|
## Architectural Evolution
|
|
|
|
The DMARQ architecture is designed to evolve across milestones:
|
|
|
|
### Milestone 1: Minimal Architecture
|
|
- Single FastAPI service
|
|
- In-memory data storage
|
|
- Manual file upload
|
|
- Basic template rendering
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ Web Browser │
|
|
└───────────┬─────────────┘
|
|
│
|
|
┌───────────┼─────────────┐
|
|
│ FastAPI Application │
|
|
├───────────┼─────────────┤
|
|
│ DMARC Parser │ Templates │
|
|
├─────────────┬───────────┤
|
|
│ In-Memory Store │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
### Milestone 2-3: Enhanced Architecture
|
|
- FastAPI with background tasks
|
|
- Database persistence layer
|
|
- IMAP integration
|
|
- Expanded web interface
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ Web Browser │
|
|
└───────────┬─────────────┘
|
|
│
|
|
┌───────────┼─────────────┐
|
|
│ FastAPI Application │
|
|
├─────────┬───────┬───────┤
|
|
│Templates│Parser │IMAP │
|
|
├─────────┴───────┴───────┤
|
|
│ Background Tasks │
|
|
├─────────────────────────┤
|
|
│ Database Layer │
|
|
└───────────┬─────────────┘
|
|
│
|
|
┌───────────┼─────────────┐
|
|
│ SQLite/PostgreSQL DB │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
### Milestone 5+: Full Architecture
|
|
- Authentication layer
|
|
- DNS integration
|
|
- Alert system
|
|
- Visualization enhancements
|
|
|
|
```
|
|
┌───────────────────────────────────────────────┐
|
|
│ Web Browser │
|
|
└───────────────────┬───────────────────────────┘
|
|
│
|
|
┌───────────────────┼───────────────────────────┐
|
|
│ FastAPI Application │
|
|
├────────┬──────────┬──────────┬────────┬───────┤
|
|
│Auth │Templates │Parser │IMAP │Apprise│
|
|
├────────┴──────────┴──────────┴────────┴───────┤
|
|
│ Background Tasks │
|
|
├───────────────────────────────────────────────┤
|
|
│ Database Layer │
|
|
└───────────────────┬───────────────────────────┘
|
|
│
|
|
┌───────────────────┼───────────────────────────┐
|
|
│ SQLite/PostgreSQL DB │
|
|
└───────────────────────────────────────────────┘
|
|
│ │ │
|
|
┌───┴───┐ ┌─────┴────┐ ┌─────┴────┐
|
|
│DNS API│ │Notif. API│ │Email │
|
|
└───────┘ └──────────┘ └──────────┘
|
|
```
|
|
|
|
## Technology Stack Details
|
|
|
|
### Backend Framework
|
|
- **FastAPI**: Modern, high-performance web framework
|
|
- **Pydantic**: Data validation and settings management
|
|
- **SQLAlchemy**: ORM for database interactions
|
|
- **APScheduler**: Task scheduling for background jobs
|
|
|
|
### Frontend
|
|
- **Jinja2**: Template engine for rendering HTML
|
|
- **Tailwind CSS**: Utility-first CSS framework
|
|
- **ShadCN/UI**: Reusable UI component system
|
|
- **Chart.js**: Lightweight charting library
|
|
|
|
### External Libraries
|
|
- **parsedmarc**: DMARC report parsing
|
|
- **Apprise**: Unified notification system
|
|
- **Cloudflare API** (optional): DNS integration
|
|
- **FastAPI Users**: Authentication management
|
|
|
|
### Deployment
|
|
- **Docker**: Containerization
|
|
- **Docker Compose**: Multi-container orchestration
|
|
- **SQLite/PostgreSQL**: Database options
|
|
|
|
## Security Considerations
|
|
|
|
- Sensitive credentials are stored securely
|
|
- Authentication protects access to report data
|
|
- HTTPS recommended for production deployment
|
|
- No external APIs required for core functionality
|
|
- Self-hosted approach keeps data private |