Christian Krakau-Louis
56 lines
3.1 KiB
Markdown
56 lines
3.1 KiB
Markdown
# DMARC Aggregate Format Compatibility
|
|
|
|
DMARQ imports DMARC aggregate reports from direct uploads, IMAP attachments, Gmail API attachments, and the Cloudflare Email Worker webhook. Compatibility is locked by the fixture pack in `backend/app/tests/fixtures/dmarc_aggregate`.
|
|
|
|
## Supported Aggregate Inputs
|
|
|
|
- Plain XML files with a `.xml` extension.
|
|
- ZIP archives containing an XML report.
|
|
- GZIP archives with `.gz` or `.gzip` extensions.
|
|
- RFC 7489-compatible aggregate reports without XML namespaces.
|
|
- Namespaced aggregate reports using `urn:ietf:params:xml:ns:dmarc-2.0`.
|
|
- RFC 9990-style reports with optional metadata such as `version`, `generator`, `extra_contact_info`, repeated `error` values, `np`, `fo`, `testing`, `discovery_method`, `envelope_to`, policy override reasons, `human_result`, SPF `scope`, and namespaced extension elements.
|
|
|
|
## Preserved Metadata
|
|
|
|
Newer optional fields are parsed without changing the legacy response shape that existing screens use. When a database is configured, DMARQ also persists the optional report metadata, policy metadata, record identifiers, policy override reasons, and extension payloads so exports and future views can use them.
|
|
|
|
CSV exports include the most useful aggregate metadata for operators:
|
|
|
|
- `subdomain_policy`
|
|
- `non_subdomain_policy`
|
|
- `adkim`
|
|
- `aspf`
|
|
- `failure_options`
|
|
- `testing`
|
|
- `discovery_method`
|
|
- `schema_version`
|
|
- `report_variant`
|
|
- `generator`
|
|
|
|
## Known Edge Cases
|
|
|
|
- Unknown namespaced extension fields are preserved as best-effort key/value data after namespace prefixes are stripped by the XML parser.
|
|
- Malformed optional timestamps and counts use safe defaults so one bad optional value does not reject the whole report.
|
|
- Reports with no `<record>` elements import with a zero-count summary.
|
|
- Unsupported attachments are skipped by IMAP and Gmail import paths and recorded in import details when stats are available.
|
|
- For duplicate detection, DMARQ uses the domain and `report_id` pair. Fixture report IDs must remain unique within a single test import run.
|
|
|
|
## Fixture Pack
|
|
|
|
The current fixture pack covers:
|
|
|
|
- `rfc7489-google.xml`: legacy no-namespace aggregate report.
|
|
- `rfc9990-namespaced-legacy-fields.xml`: namespaced aggregate report that keeps legacy fields working.
|
|
- `rfc9990-treewalk-extension.xml`: RFC 9990-style policy metadata, treewalk discovery, `envelope_to`, override reasons, auth-result details, and report/record extensions.
|
|
- `rfc9990-multi-auth-overrides.xml`: multiple records, multiple DKIM auth results, policy override reasons, PSD-style discovery, and nested vendor extensions.
|
|
|
|
The compatibility tests exercise every fixture through parser extraction, upload import, IMAP attachment import, and Gmail attachment import.
|
|
|
|
## Adding Fixtures
|
|
|
|
1. Add the XML file under `backend/app/tests/fixtures/dmarc_aggregate`.
|
|
2. Add its expected metadata to `DMARC_COMPATIBILITY_FIXTURES` in `backend/app/tests/test_data.py`.
|
|
3. Include a unique `report_id`, stable domain, expected variant, expected total count, and any policy fields that should be asserted.
|
|
4. Run `pytest backend/app/tests/test_dmarc_compatibility_fixtures.py backend/app/tests/test_dmarc_parser.py backend/app/tests/test_reports_api.py`.
|