docs: update CI workflow and integration test documentation for two-stage pipeline

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

docs/CIWorkflow.md

@@ -11,21 +11,45 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus
 | Job | Tool | Purpose | Enforced |
 |--------|--------|---------------------------------------------|----------|
 | `lint` | Ruff | Fast Python linter (replaces Flake8, Black, isort, Bandit) | ✅ |
+| `html-lint` | djLint | HTML template accessibility linter | ✅ |
 | `dependency-scan` | pip-audit | Dependency vulnerability scanning against OSV/PyPA advisories | ✅ |
-| `test` | pytest | Unit/integration tests with coverage | ✅ |
+| `test-quick` | pytest | Unit + basic integration tests with coverage (~2 min) | ✅ |
+| `test-integration` | pytest | Docker container and external service tests (~5 min) | ✅ |
 | `mypy` | mypy | Static type checking | ✅ |
 
-`lint` and `dependency-scan` start **in parallel** at the beginning of the pipeline — neither depends on the other. `test` and `mypy` run only after both have passed.
+### Pipeline Flow
+
+```
+Stage 1 (parallel):  lint, html-lint, dependency-scan
+                          │
+Stage 2 (parallel):  test-quick + mypy
+                          │
+Stage 3:             test-integration  (only after test-quick passes)
+                          │
+Stage 4:             build  (only after all above pass)
+```
+
+The pipeline follows a **fail-early** strategy: fast linters and quick tests run first to catch regressions early. Heavier integration tests only run after the quick tests pass, saving CI time when basic issues are present.
 
 > **Note:** DocuElevate uses Ruff, a modern all-in-one Python linter that consolidates the functionality of Flake8, Black, isort, and Bandit. This streamlined approach reduces CI complexity while maintaining code quality and security standards.
 
-### Tests
+### Quick Tests (`test-quick`)
 
-- Runs pytest with coverage reporting (XML + terminal).
+- **Timeout:** 10 minutes (job), 120 seconds (per test via `pytest-timeout`)
-- Excludes E2E tests that require Docker-in-Docker (`-m "not e2e"`).
+- Runs the majority of tests (~2,790 unit + basic integration tests)
-- Uses Redis and RabbitMQ service containers.
+- Excludes tests marked `e2e`, `requires_docker`, `requires_external`, or `slow`
-- Uploads coverage and JUnit XML results to Codecov.
+- Uses a Redis service container for tests that need it
-- Uploads `junit.xml` and `coverage.xml` as workflow artifacts (always, even on failure).
+- Collects coverage and uploads to Codecov
+- Uploads `junit.xml` and `coverage.xml` as workflow artifacts
+
+### Integration Tests (`test-integration`)
+
+- **Timeout:** 20 minutes (job), 300 seconds (per test via `pytest-timeout`)
+- Runs tests marked `requires_docker`, `requires_external`, or `slow` (excluding `e2e`)
+- Uses Redis and RabbitMQ service containers
+- Docker daemon available for testcontainers (WebDAV, OAuth mock server, etc.)
+- Only runs after quick tests pass (fail-early gate)
+- Uploads `junit-integration.xml` as a workflow artifact
 
 ### Ruff Lint & Format
 
@@ -48,8 +72,9 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus
 The following artifacts are uploaded after every run:
 
 | Artifact | Contents | Condition |
-|------------------|--------------------------------------|----------------------|
+|-------------------------------|--------------------------------------|----------------------|
-| `test-results` | `junit.xml`, `coverage.xml` | Always (unless cancelled) |
+| `test-results-quick` | `junit.xml`, `coverage.xml` | Always (unless cancelled) |
+| `test-results-integration` | `junit-integration.xml` | Always (unless cancelled) |
 
 ## Running Linters Locally
 
@@ -68,8 +93,14 @@ ruff format --check app/ tests/
 # Run type checking
 mypy app/
 
-# Run tests
+# Run quick tests (same as CI quick stage)
-pytest tests/ -v --cov=app --cov-report=term -m "not e2e"
+pytest tests/ -v --timeout=120 --cov=app --cov-report=term -m "not e2e and not requires_docker and not requires_external and not slow"
+
+# Run integration tests (requires Docker)
+pytest tests/ -v --timeout=300 -m "(requires_docker or requires_external or slow) and not e2e"
+
+# Run all tests except e2e
+pytest tests/ -v --timeout=120 -m "not e2e"
 ```
 
 Or use pre-commit hooks to run checks automatically on each commit (recommended):

tests/README_INTEGRATION_TESTS.md

@@ -361,12 +361,28 @@ docker volume prune -f
 
 ### GitHub Actions - Current Configuration
 
-The DocuElevate CI workflow (`.github/workflows/tests.yaml`) **excludes E2E tests** by default because they require Docker-in-Docker (testcontainers), which requires additional configuration in GitHub Actions.
+The DocuElevate CI pipeline (`.github/workflows/ci.yml`) uses a **two-stage test strategy** for fast feedback:
+
+1. **Quick Tests** (`test-quick`) — Runs ~2,790 unit and basic integration tests in ~2 minutes. Excludes tests marked `e2e`, `requires_docker`, `requires_external`, or `slow`. This stage gates the integration tests.
+
+2. **Integration Tests** (`test-integration`) — Runs ~28 Docker-based and external service tests. Only starts after quick tests pass. Uses testcontainers for WebDAV, OAuth, etc.
+
+Both stages use `pytest-timeout` to prevent individual tests from hanging:
+- Quick tests: 120 seconds per test, 10-minute job timeout
+- Integration tests: 300 seconds per test, 20-minute job timeout
 
 ```yaml
-- name: Run Tests
+# Quick tests (Stage 2b)
-  # Exclude E2E tests - they require Docker-in-Docker (testcontainers)
+- name: Run Quick Tests
-  run: pytest tests/ -v --cov=app -m "not e2e"
+  run: >
+    pytest tests/ -v --timeout=120
+    -m "not e2e and not requires_docker and not requires_external and not slow"
+
+# Integration tests (Stage 2c) — only after quick tests pass
+- name: Run Integration Tests
+  run: >
+    pytest tests/ -v --timeout=300
+    -m "(requires_docker or requires_external or slow) and not e2e"
 ```
 
 **To run E2E tests locally:**