feat: Add security hardening, agentic coding infrastructure, and test framework

- Add SECRET_KEY and ENCRYPTION_KEY validation on startup
- Implement security headers middleware (X-Frame-Options, CSP, HSTS)
- Add CSRF protection middleware
- Create comprehensive GitHub issue templates and PR template
- Add Makefile with common development tasks
- Configure pre-commit hooks (black, ruff, mypy, bandit, detect-secrets)
- Create docs/CODING_PATTERNS.md with best practices
- Create docs/ERRORS.md documenting all error codes
- Add Architecture Decision Records (ADR) for Celery and Fernet encryption
- Create CHANGELOG.md for version tracking
- Set up pytest test infrastructure with fixtures and factories
- Add sample unit tests for security and config validation
- Create CI/CD workflows (test, lint, security)
- Add comprehensive TODO.md with milestones and progress tracking

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

.pre-commit-config.yaml

@@ -0,0 +1,103 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+
+default_language_version:
+  python: python3.11
+
+repos:
+  # General pre-commit hooks
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+      - id: trailing-whitespace
+        args: [--markdown-linebreak-ext=md]
+      - id: end-of-file-fixer
+      - id: check-yaml
+        args: [--unsafe]  # Allow custom YAML tags
+      - id: check-json
+      - id: check-added-large-files
+        args: [--maxkb=1000]
+      - id: check-merge-conflict
+      - id: check-case-conflict
+      - id: check-docstring-first
+      - id: debug-statements
+      - id: mixed-line-ending
+      - id: name-tests-test
+        args: [--pytest-test-first]
+      - id: requirements-txt-fixer
+
+  # Python code formatting with Black
+  - repo: https://github.com/psf/black
+    rev: 24.1.1
+    hooks:
+      - id: black
+        language_version: python3.11
+        args: [--line-length=100]
+        files: ^(backend/|pop3_forwarder\.py)
+
+  # Python linting with Ruff (replaces flake8, isort, etc.)
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.1.15
+    hooks:
+      - id: ruff
+        args: [--fix, --exit-non-zero-on-fix]
+        files: ^(backend/|pop3_forwarder\.py)
+
+  # Type checking with mypy
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.8.0
+    hooks:
+      - id: mypy
+        additional_dependencies: [pydantic, sqlalchemy, types-redis]
+        args: [--ignore-missing-imports, --explicit-package-bases]
+        files: ^backend/app/
+
+  # Security checks
+  - repo: https://github.com/PyCQA/bandit
+    rev: 1.7.6
+    hooks:
+      - id: bandit
+        args: [-ll, -r, backend/app/, --skip, B101]  # Skip assert warnings
+        files: ^backend/
+
+  # Check for secrets
+  - repo: https://github.com/Yelp/detect-secrets
+    rev: v1.4.0
+    hooks:
+      - id: detect-secrets
+        args: [--baseline, .secrets.baseline]
+        exclude: package.lock.json
+
+  # Dockerfile linting
+  - repo: https://github.com/hadolint/hadolint
+    rev: v2.12.0
+    hooks:
+      - id: hadolint-docker
+        args: [--ignore, DL3008, --ignore, DL3009]  # Ignore apt-get warnings
+
+  # YAML linting
+  - repo: https://github.com/adrienverge/yamllint
+    rev: v1.33.0
+    hooks:
+      - id: yamllint
+        args: [-c=.yamllint.yml]
+
+  # Markdown linting
+  - repo: https://github.com/markdownlint/markdownlint
+    rev: v0.12.0
+    hooks:
+      - id: markdownlint
+        args: [--rules, '~MD013,~MD033,~MD041']  # Ignore line length, HTML, first line
+
+  # SQL formatting (optional, if you have raw SQL files)
+  # - repo: https://github.com/sqlfluff/sqlfluff
+  #   rev: 2.3.5
+  #   hooks:
+  #     - id: sqlfluff-lint
+  #     - id: sqlfluff-fix
+
+# Configuration for specific tools
+
+# Ruff configuration (in pyproject.toml or ruff.toml)
+# Black configuration (in pyproject.toml)
+# Mypy configuration (in mypy.ini or pyproject.toml)