gh-christianlouis-dmarq/docs/development/issue_generation.md

# Issue Generation from Roadmap - Summary

## What Was Done

This PR implements a complete solution for generating GitHub issues from the DMARQ roadmap documentation.

### Created Files

1. **`scripts/generate_issues.py`** (410 lines)
   - Python script that parses ROADMAP.md
   - Extracts tasks from three main sections:
     - Security Remediation Sprint
     - Milestones 4-11
     - Continuous Improvements
   - Generates structured issue data in multiple formats

2. **`generated_issues/issues.json`** (32 KB)
   - JSON file with 54 issues
   - Includes title, body, labels, milestone, and assignees
   - Ready for programmatic import via GitHub API

3. **`generated_issues/issues_preview.md`** (25 KB)
   - Human-readable markdown preview
   - Organized by milestone
   - Allows manual review before creating issues

4. **`generated_issues/create_issues.sh`** (39 KB)
   - Executable bash script
   - Uses GitHub CLI to create all 54 issues
   - Includes error handling and rate limiting

5. **`generated_issues/README.md`** (6.5 KB)
   - Comprehensive documentation
   - Usage instructions for all three import methods
   - Label and milestone reference guide

## Issue Breakdown

### Total: 54 Issues

#### Security Remediation Sprint (7 issues - HIGHEST PRIORITY)
1. **[Security Sprint] Authentication & Authorization** (CRITICAL)
2. **[Security Sprint] Secret Management** (CRITICAL)
3. **[Security Sprint] XML Parsing Security** (HIGH)
4. **[Security Sprint] Input Validation** (HIGH)
5. **[Security Sprint] Security Headers** (MEDIUM)
6. **[Security Sprint] CORS Configuration** (MEDIUM)
7. **[Security Sprint] Error Handling** (MEDIUM)

#### Milestone 4: Enhanced Dashboard & Visualization (6 issues)
- Historical trend charts
- Compliance rate visualizations
- Volume and sender analytics
- Time-series data displays
- Domain comparison views
- Security enhancements (XSS prevention, CSP)

#### Milestone 5: User Authentication & Multi-User Support (6 issues)
- FastAPI Users integration
- User registration and management
- JWT-based authentication
- Role-based access control (RBAC)
- Password reset functionality
- Security features (MFA, session management, etc.)

#### Milestone 6: Alerting & Notifications (5 issues)
- Apprise integration
- Customizable alert rules
- Multi-channel notifications
- Alert history and management
- Security features

#### Milestone 7: Advanced Rule Engine (5 issues)
- Custom alert conditions
- Threshold-based triggers
- New sender detection
- Anomaly detection
- Security features

#### Milestone 8: DNS Health & Cloudflare Integration (5 issues)
- DNS record health checks
- SPF/DKIM/DMARC validation
- Cloudflare API integration
- Configuration recommendations
- Security features

#### Milestone 9: Forensic Reports (RUF) Support (5 issues)
- Forensic report parsing
- Failure sample analysis
- PII redaction options
- Detailed authentication failure views
- Security features

#### Milestone 10: Advanced Analytics & Reporting (5 issues)
- Historical trend analysis
- Comparative reporting
- Export capabilities
- Scheduled reports
- Security features

#### Milestone 11: Enterprise Features (6 issues)
- Multi-tenant architecture
- API rate limiting
- Advanced RBAC
- SSO integration
- Compliance reporting
- Security features

#### Continuous Improvements (4 issues)
- **Code Quality**: Test coverage, dependency updates, code review, performance, tech debt
- **Security**: Monthly audits, vulnerability scanning, penetration testing, training
- **Documentation**: Keep current, API docs, security guides, playbooks, troubleshooting
- **Community**: Issue triage, PR review, engagement, feature evaluation, bug fixes

## Labels Used

### Priority
- `priority: critical` (7 issues)
- `priority: high` (19 issues)
- `priority: medium` (21 issues)
- `priority: low` (7 issues)

### Type
- `security` (18 issues)
- `security: critical` (2 issues)
- `security: high` (2 issues)
- `enhancement` (43 issues)
- `maintenance` (4 issues)
- `continuous-improvement` (4 issues)
- `documentation` (1 issue)
- `code-quality` (1 issue)

### Milestones
- `milestone-4` through `milestone-11`

## How to Create the Issues

### Option 1: Using GitHub CLI (Recommended)

```bash
# Install GitHub CLI if not already installed
# See: https://cli.github.com/

# Authenticate
gh auth login

# Navigate to the generated issues directory
cd generated_issues

# Run the script
./create_issues.sh
```

This will create all 54 issues automatically. The script includes:
- ✅ Verification that GitHub CLI is installed
- ✅ Authentication check
- ✅ Rate limiting (1 second between issues)
- ✅ Error handling
- ✅ Progress reporting

### Option 2: Manual Creation

Review `generated_issues/issues_preview.md` and create issues manually through the GitHub web interface.

### Option 3: Custom Import

Use `generated_issues/issues.json` with your own tooling or GitHub API:

```python
import json
import requests

with open('issues.json') as f:
    issues = json.load(f)

for issue in issues:
    response = requests.post(
        'https://api.github.com/repos/christianlouis/dmarq/issues',
        headers={'Authorization': f'token {YOUR_TOKEN}'},
        json={
            'title': issue['title'],
            'body': issue['body'],
            'labels': issue['labels']
        }
    )
```

## Milestones Setup

Before creating issues, you may want to create milestones:

```bash
gh milestone create "Security Remediation Sprint" --repo christianlouis/dmarq
gh milestone create "Milestone 4: Enhanced Dashboard & Visualization" --repo christianlouis/dmarq
gh milestone create "Milestone 5: User Authentication & Multi-User Support" --repo christianlouis/dmarq
gh milestone create "Milestone 6: Alerting & Notifications" --repo christianlouis/dmarq
gh milestone create "Milestone 7: Advanced Rule Engine" --repo christianlouis/dmarq
gh milestone create "Milestone 8: DNS Health & Cloudflare Integration" --repo christianlouis/dmarq
gh milestone create "Milestone 9: Forensic Reports (RUF) Support" --repo christianlouis/dmarq
gh milestone create "Milestone 10: Advanced Analytics & Reporting" --repo christianlouis/dmarq
gh milestone create "Milestone 11: Enterprise Features" --repo christianlouis/dmarq
gh milestone create "Continuous Improvements" --repo christianlouis/dmarq
```

## Recommended Workflow

1. **Review**: Read through `generated_issues/issues_preview.md`
2. **Create Milestones**: Set up milestones in GitHub (optional)
3. **Create Labels**: Ensure all required labels exist (optional - will be auto-created)
4. **Start with Security**: Consider creating Security Sprint issues first
5. **Create All Issues**: Run `./create_issues.sh`
6. **Organize**: Assign to milestones and team members
7. **Prioritize**: Start with Security Sprint!

## Regenerating Issues

If the roadmap is updated, regenerate issues:

```bash
python3 scripts/generate_issues.py
```

This will overwrite files in `generated_issues/`.

## Script Features

### Parsing Capabilities
- ✅ Extracts security sprint tasks with priorities
- ✅ Parses milestone features (skips completed milestones)
- ✅ Identifies security enhancements per milestone
- ✅ Captures continuous improvement tasks
- ✅ Preserves file references and documentation links

### Issue Generation
- ✅ Creates descriptive titles with prefixes ([Security Sprint], [M4], etc.)
- ✅ Generates detailed issue bodies with:
  - Description and context
  - Task checklists
  - Files to update (for security issues)
  - Related documentation links
  - Auto-generated footer
- ✅ Applies appropriate labels based on:
  - Priority level (critical/high/medium/low)
  - Issue type (security/enhancement/maintenance)
  - Milestone number
  - Category (documentation/code-quality)
- ✅ Assigns to appropriate milestones

### Output Formats
- ✅ JSON for programmatic import
- ✅ Markdown for human review
- ✅ Shell script for GitHub CLI

## Next Steps

1. **Review the generated issues** in `generated_issues/issues_preview.md`
2. **Create labels and milestones** if desired (optional)
3. **Run the import script** to create issues in GitHub
4. **Prioritize the Security Sprint** - start working on critical security issues
5. **Organize the remaining issues** by assigning to team members
6. **Update the roadmap** as work progresses

## Important Notes

### Security First! 🔒
The Security Remediation Sprint contains **7 critical and high-priority security issues** that should be addressed before implementing new features. These address:
- Authentication and authorization
- Secret management
- XML parsing security
- Input validation
- Security headers
- CORS configuration
- Error handling

### Issue Quantities
Some milestones have more issues than others based on the roadmap structure. This is intentional and reflects the complexity and scope of each milestone.

### Dependencies
Some milestones depend on others:
- Milestone 5 (Authentication) is foundational for later features
- Security Sprint should complete before Milestone 4
- See ROADMAP.md for detailed dependency information

## Support

- **Script Location**: `scripts/generate_issues.py`
- **Documentation**: `generated_issues/README.md`
- **Source**: `ROADMAP.md`
- **Questions**: Open an issue in the DMARQ repository

---

**Generated**: 2026-02-06  
**Total Issues**: 54  
**Ready to Import**: ✅