diff --git a/docs/deployment/docker.md b/docs/deployment/docker.md index 42915f8..6656c73 100644 --- a/docs/deployment/docker.md +++ b/docs/deployment/docker.md @@ -219,6 +219,8 @@ docker-compose --profile production up -d ## Updating DMARQ +Before updating a production deployment, follow the [Release Checklist](release-checklist.md) and create a database backup. + To update to a newer version: ```bash diff --git a/docs/deployment/manual.md b/docs/deployment/manual.md index 2e5ed2e..138a2e2 100644 --- a/docs/deployment/manual.md +++ b/docs/deployment/manual.md @@ -240,6 +240,8 @@ sudo systemctl enable dmarq-celery ## Updating DMARQ +Before updating a production deployment, follow the [Release Checklist](release-checklist.md) and create a database backup. + To update to a newer version: ```bash diff --git a/docs/deployment/release-checklist.md b/docs/deployment/release-checklist.md new file mode 100644 index 0000000..901d549 --- /dev/null +++ b/docs/deployment/release-checklist.md @@ -0,0 +1,94 @@ +# Release Checklist + +Use this checklist for every DMARQ release or production upgrade. It focuses on the work that most often causes avoidable downtime: database migrations, test coverage, release automation, and smoke checks. + +## Before Merging + +Confirm the release candidate is ready: + +- The pull request describes user impact, operator impact, and any migration or configuration changes. +- Local tests pass with `PYTHONPATH=backend python -m pytest`. +- Formatting and lint checks pass, or the PR explains any advisory-only lint warnings. +- New or changed environment variables are documented in deployment docs. +- New secrets are stored through the deployment secret manager, not committed to files or copied into PR text. +- Database model changes include Alembic migration coverage or a clear reason no migration is needed. +- The database backup guide has been followed for production upgrades. + +## Migration Review + +Before merging a change that touches database models, migrations, import paths, or report persistence: + +1. Inspect the migration files and confirm they match the model change. +2. Test a fresh database with `alembic upgrade head`. +3. Test an existing database upgrade from the current released version when possible. +4. Confirm downgrade or rollback expectations are documented if the change is risky. +5. Create a fresh production backup before deployment. + +## Merge And Release + +DMARQ releases from the `main` branch through GitHub Actions. + +1. Use a conventional merge commit subject: + - `fix:` for patch releases. + - `feat:` for minor releases. + - `feat!:` or `BREAKING CHANGE` for major releases. + - `docs:` for documentation-only changes that do not need a new app version. +2. Watch the Release workflow. +3. Watch the CI workflow through lint, tests, security scan, CodeQL, Docker build, and preprod manifest update. +4. If the preprod manifest update fails because the k8s state repo moved during the run, rerun the failed job after confirming the image build succeeded. +5. Pull tags locally after release automation completes. + +## Smoke Checks + +Run these checks after the deployment updates: + +```bash +curl -fsS https://your-dmarq-host.example.com/health +curl -fsS https://your-dmarq-host.example.com/api/v1/health +``` + +Then verify in the browser: + +- The login or setup page loads as expected. +- The dashboard loads without server errors. +- Domain totals and recent reports are visible. +- Mail Sources opens and recent import history is visible. +- A manual mailbox fetch can be triggered in preprod or another safe environment. +- CSV export works for at least one domain with reports. + +Check logs after smoke testing: + +```bash +docker compose logs --tail=100 backend +``` + +For systemd deployments: + +```bash +sudo journalctl -u dmarq -n 100 +``` + +There should be no repeated database, migration, authentication, or mailbox polling failures. + +## Rollback Readiness + +Before promoting a release to production, make sure these are known: + +- The previous DMARQ version or image tag. +- The backup file created before deployment. +- The database restore command for the active database type. +- The operator who can update the deployment manifest or systemd service. +- The smoke check results from preprod. + +If a rollback requires restoring the database, stop the app before restoring and verify the restored app with the same smoke checks. + +## After Release + +Record: + +- Released version or commit SHA. +- Whether a database migration ran. +- Backup file location or backup job identifier. +- CI and release workflow result. +- Smoke check result. +- Any follow-up issues found during deployment. diff --git a/docs/development/ci-cd.md b/docs/development/ci-cd.md index fb94227..cf7a120 100644 --- a/docs/development/ci-cd.md +++ b/docs/development/ci-cd.md @@ -8,6 +8,8 @@ The pipeline is defined in two workflow files: | `.github/workflows/ci.yml` | Lint → Test → Security → Docker → GitOps | Push to `main`/`develop`, pull requests, weekly schedule | | `.github/workflows/release.yml` | Semantic versioning & changelog | Push to `main` | +Use the [Release Checklist](../deployment/release-checklist.md) before merging production changes and after the deployment updates. + --- ## Pipeline Stages diff --git a/docs/development/roadmap.md b/docs/development/roadmap.md index b89b44a..bb15c5a 100644 --- a/docs/development/roadmap.md +++ b/docs/development/roadmap.md @@ -69,13 +69,14 @@ Quality bar: Objective: make self-hosted deployments safer. Priority tasks: -- Add a release checklist covering migrations, tests, and smoke checks. +- Use the production hardening docs in at least one real production upgrade and capture any gaps. Delivered: - Documented a 1Password secret-injection deployment flow for local, Docker Compose, and systemd deployments. - Redacted secret-like values from mail-source diagnostics, stored import history, OAuth error logs, and validated admin auth contexts. - Added production startup validation for stable secrets, configured auth, auth-disabled mode, and Logto TLS verification. - Added database backup and restore guidance for SQLite and PostgreSQL deployments. +- Added a release checklist covering migrations, tests, smoke checks, release automation, and rollback readiness. ## Later Milestones diff --git a/docs/index.md b/docs/index.md index 8b7b221..d441ec1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -21,4 +21,4 @@ DMARQ is a full-stack DMARC monitoring platform designed to help organizations t To get started with DMARQ, please see the [Getting Started](user_guide/getting_started.md) guide. -For installation instructions, check the [Docker Setup](deployment/docker.md) or [Manual Installation](deployment/manual.md) guides. For production secrets, use [Secret Handling with 1Password](deployment/secrets.md). For database operations, use [Database Backup and Restore](deployment/backups.md). +For installation instructions, check the [Docker Setup](deployment/docker.md) or [Manual Installation](deployment/manual.md) guides. For production secrets, use [Secret Handling with 1Password](deployment/secrets.md). For database operations, use [Database Backup and Restore](deployment/backups.md). For upgrades, use the [Release Checklist](deployment/release-checklist.md). diff --git a/docs/milestones.md b/docs/milestones.md index 6ae777b..a4747af 100644 --- a/docs/milestones.md +++ b/docs/milestones.md @@ -100,9 +100,9 @@ Delivered: - Admin authentication contexts no longer carry raw API keys after validation. - Production startup checks now fail early for missing stable secrets, missing auth configuration, auth-disabled mode, or disabled Logto TLS verification. - Database backup and restore guidance now covers SQLite and PostgreSQL deployments, restore verification, and upgrade safety checks. +- A release checklist now covers pre-merge checks, migrations, release automation, smoke checks, and rollback readiness. Planned: -- Add release checklist covering migrations, tests, and smoke checks. Exit criteria: - A self-hosted deployment can be configured without copying secrets into source-controlled files or chat logs. diff --git a/docs/todo.md b/docs/todo.md index a5daee5..d662156 100644 --- a/docs/todo.md +++ b/docs/todo.md @@ -159,6 +159,7 @@ This file tracks the specific implementation tasks for each milestone of the DMA - [x] Redact mailbox/OAuth secrets from diagnostics, logs, import history, and auth contexts - [x] Add startup checks for production-critical configuration - [x] Add backup/restore guidance for database deployments +- [x] Add release checklist covering migrations, tests, and smoke checks - [ ] Apprise notifications and alert rules - [ ] DNS health guidance and Cloudflare read-only inspection - [ ] Guided setup and operator health pages