test: add DMARC compatibility fixture pack

docs/index.md

@@ -22,3 +22,5 @@ DMARQ is a full-stack DMARC monitoring platform designed to help organizations t
 To get started with DMARQ, please see the [Getting Started](user_guide/getting_started.md) guide.
 
 For installation instructions, check the [Docker Setup](deployment/docker.md) or [Manual Installation](deployment/manual.md) guides. Operators should use the [Operator Runbook](deployment/operations.md) for deployment modes, verification, upgrades, and rollback, and the [Troubleshooting Playbooks](deployment/troubleshooting.md) for ingestion, authentication, DNS, database, and notification failures. For production secrets, use [Secret Handling with 1Password](deployment/secrets.md). For database operations, use [Database Backup and Restore](deployment/backups.md). For upgrades, use the [Release Checklist](deployment/release-checklist.md).
+
+For aggregate-report parser support, known edge cases, and fixture guidance, see [DMARC Aggregate Format Compatibility](reference/dmarc-compatibility.md).

docs/milestones.md

@@ -163,7 +163,7 @@ Exit criteria:
 
 ## Milestone 10: Forensic Report Support
 
-Status: In Progress
+Status: Complete
 
 Goal: support DMARC RUF/forensic reports for individual failure investigation.
 
@@ -182,7 +182,7 @@ Exit criteria:
 
 ## Milestone 11: DMARC Format Compatibility (DMARCbis) and Standards Alignment
 
-Status: In Progress
+Status: Complete
 
 Goal: keep DMARQ compatible with evolving DMARC report formats and nomenclature without breaking existing imports.
 
@@ -190,11 +190,8 @@ Delivered:
 - Add parser compatibility for RFC 9990-style aggregate report namespaces, version detection, policy metadata, identifiers, override reasons, auth-result details, and namespaced extensions.
 - Keep legacy RFC 7489-style reports backward compatible through fixture coverage.
 - Persist and CSV-export newly introduced aggregate metadata with nullable, backward-safe database fields.
-
-Planned:
-- Update domain/source reporting to include new metadata where it improves operator actionability.
-- Add fixture-driven tests for representative real-world DMARCbis-style reports.
-- Update documentation to clarify supported formats and terminology.
+- Add a fixture-driven compatibility pack covering parser, upload, IMAP, and Gmail import paths with supported-format documentation.
+- Verify domain report views and CSV exports render fixture-backed DMARCbis-style imports correctly.
 
 Exit criteria:
 - A DMARCbis-style aggregate report can be imported via upload/IMAP/Gmail and renders correctly in dashboards and exports.

docs/reference/dmarc-compatibility.md

@@ -0,0 +1,55 @@
+# DMARC Aggregate Format Compatibility
+
+DMARQ imports DMARC aggregate reports from direct uploads, IMAP attachments, Gmail API attachments, and the Cloudflare Email Worker webhook. Compatibility is locked by the fixture pack in `backend/app/tests/fixtures/dmarc_aggregate`.
+
+## Supported Aggregate Inputs
+
+- Plain XML files with a `.xml` extension.
+- ZIP archives containing an XML report.
+- GZIP archives with `.gz` or `.gzip` extensions.
+- RFC 7489-compatible aggregate reports without XML namespaces.
+- Namespaced aggregate reports using `urn:ietf:params:xml:ns:dmarc-2.0`.
+- RFC 9990-style reports with optional metadata such as `version`, `generator`, `extra_contact_info`, repeated `error` values, `np`, `fo`, `testing`, `discovery_method`, `envelope_to`, policy override reasons, `human_result`, SPF `scope`, and namespaced extension elements.
+
+## Preserved Metadata
+
+Newer optional fields are parsed without changing the legacy response shape that existing screens use. When a database is configured, DMARQ also persists the optional report metadata, policy metadata, record identifiers, policy override reasons, and extension payloads so exports and future views can use them.
+
+CSV exports include the most useful aggregate metadata for operators:
+
+- `subdomain_policy`
+- `non_subdomain_policy`
+- `adkim`
+- `aspf`
+- `failure_options`
+- `testing`
+- `discovery_method`
+- `schema_version`
+- `report_variant`
+- `generator`
+
+## Known Edge Cases
+
+- Unknown namespaced extension fields are preserved as best-effort key/value data after namespace prefixes are stripped by the XML parser.
+- Malformed optional timestamps and counts use safe defaults so one bad optional value does not reject the whole report.
+- Reports with no `<record>` elements import with a zero-count summary.
+- Unsupported attachments are skipped by IMAP and Gmail import paths and recorded in import details when stats are available.
+- For duplicate detection, DMARQ uses the domain and `report_id` pair. Fixture report IDs must remain unique within a single test import run.
+
+## Fixture Pack
+
+The current fixture pack covers:
+
+- `rfc7489-google.xml`: legacy no-namespace aggregate report.
+- `rfc9990-namespaced-legacy-fields.xml`: namespaced aggregate report that keeps legacy fields working.
+- `rfc9990-treewalk-extension.xml`: RFC 9990-style policy metadata, treewalk discovery, `envelope_to`, override reasons, auth-result details, and report/record extensions.
+- `rfc9990-multi-auth-overrides.xml`: multiple records, multiple DKIM auth results, policy override reasons, PSD-style discovery, and nested vendor extensions.
+
+The compatibility tests exercise every fixture through parser extraction, upload import, IMAP attachment import, and Gmail attachment import.
+
+## Adding Fixtures
+
+1. Add the XML file under `backend/app/tests/fixtures/dmarc_aggregate`.
+2. Add its expected metadata to `DMARC_COMPATIBILITY_FIXTURES` in `backend/app/tests/test_data.py`.
+3. Include a unique `report_id`, stable domain, expected variant, expected total count, and any policy fields that should be asserted.
+4. Run `pytest backend/app/tests/test_dmarc_compatibility_fixtures.py backend/app/tests/test_dmarc_parser.py backend/app/tests/test_reports_api.py`.