From c100250b23390e9f85ab8231ff5cfa6eac9274f6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Sun, 1 Mar 2026 16:45:44 +0000 Subject: [PATCH] docs: update CI workflow and integration test documentation for two-stage pipeline Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com> --- docs/CIWorkflow.md | 55 ++++++++++++++++++++++++------- tests/README_INTEGRATION_TESTS.md | 24 +++++++++++--- 2 files changed, 63 insertions(+), 16 deletions(-) diff --git a/docs/CIWorkflow.md b/docs/CIWorkflow.md index a2f17f3a..7692a3ff 100644 --- a/docs/CIWorkflow.md +++ b/docs/CIWorkflow.md @@ -11,21 +11,45 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus | Job | Tool | Purpose | Enforced | |--------|--------|---------------------------------------------|----------| | `lint` | Ruff | Fast Python linter (replaces Flake8, Black, isort, Bandit) | ✅ | +| `html-lint` | djLint | HTML template accessibility linter | ✅ | | `dependency-scan` | pip-audit | Dependency vulnerability scanning against OSV/PyPA advisories | ✅ | -| `test` | pytest | Unit/integration tests with coverage | ✅ | +| `test-quick` | pytest | Unit + basic integration tests with coverage (~2 min) | ✅ | +| `test-integration` | pytest | Docker container and external service tests (~5 min) | ✅ | | `mypy` | mypy | Static type checking | ✅ | -`lint` and `dependency-scan` start **in parallel** at the beginning of the pipeline — neither depends on the other. `test` and `mypy` run only after both have passed. +### Pipeline Flow + +``` +Stage 1 (parallel): lint, html-lint, dependency-scan + │ +Stage 2 (parallel): test-quick + mypy + │ +Stage 3: test-integration (only after test-quick passes) + │ +Stage 4: build (only after all above pass) +``` + +The pipeline follows a **fail-early** strategy: fast linters and quick tests run first to catch regressions early. Heavier integration tests only run after the quick tests pass, saving CI time when basic issues are present. > **Note:** DocuElevate uses Ruff, a modern all-in-one Python linter that consolidates the functionality of Flake8, Black, isort, and Bandit. This streamlined approach reduces CI complexity while maintaining code quality and security standards. -### Tests +### Quick Tests (`test-quick`) -- Runs pytest with coverage reporting (XML + terminal). -- Excludes E2E tests that require Docker-in-Docker (`-m "not e2e"`). -- Uses Redis and RabbitMQ service containers. -- Uploads coverage and JUnit XML results to Codecov. -- Uploads `junit.xml` and `coverage.xml` as workflow artifacts (always, even on failure). +- **Timeout:** 10 minutes (job), 120 seconds (per test via `pytest-timeout`) +- Runs the majority of tests (~2,790 unit + basic integration tests) +- Excludes tests marked `e2e`, `requires_docker`, `requires_external`, or `slow` +- Uses a Redis service container for tests that need it +- Collects coverage and uploads to Codecov +- Uploads `junit.xml` and `coverage.xml` as workflow artifacts + +### Integration Tests (`test-integration`) + +- **Timeout:** 20 minutes (job), 300 seconds (per test via `pytest-timeout`) +- Runs tests marked `requires_docker`, `requires_external`, or `slow` (excluding `e2e`) +- Uses Redis and RabbitMQ service containers +- Docker daemon available for testcontainers (WebDAV, OAuth mock server, etc.) +- Only runs after quick tests pass (fail-early gate) +- Uploads `junit-integration.xml` as a workflow artifact ### Ruff Lint & Format @@ -48,8 +72,9 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus The following artifacts are uploaded after every run: | Artifact | Contents | Condition | -|------------------|--------------------------------------|----------------------| -| `test-results` | `junit.xml`, `coverage.xml` | Always (unless cancelled) | +|-------------------------------|--------------------------------------|----------------------| +| `test-results-quick` | `junit.xml`, `coverage.xml` | Always (unless cancelled) | +| `test-results-integration` | `junit-integration.xml` | Always (unless cancelled) | ## Running Linters Locally @@ -68,8 +93,14 @@ ruff format --check app/ tests/ # Run type checking mypy app/ -# Run tests -pytest tests/ -v --cov=app --cov-report=term -m "not e2e" +# Run quick tests (same as CI quick stage) +pytest tests/ -v --timeout=120 --cov=app --cov-report=term -m "not e2e and not requires_docker and not requires_external and not slow" + +# Run integration tests (requires Docker) +pytest tests/ -v --timeout=300 -m "(requires_docker or requires_external or slow) and not e2e" + +# Run all tests except e2e +pytest tests/ -v --timeout=120 -m "not e2e" ``` Or use pre-commit hooks to run checks automatically on each commit (recommended): diff --git a/tests/README_INTEGRATION_TESTS.md b/tests/README_INTEGRATION_TESTS.md index fe99c9e8..7a6bec15 100644 --- a/tests/README_INTEGRATION_TESTS.md +++ b/tests/README_INTEGRATION_TESTS.md @@ -361,12 +361,28 @@ docker volume prune -f ### GitHub Actions - Current Configuration -The DocuElevate CI workflow (`.github/workflows/tests.yaml`) **excludes E2E tests** by default because they require Docker-in-Docker (testcontainers), which requires additional configuration in GitHub Actions. +The DocuElevate CI pipeline (`.github/workflows/ci.yml`) uses a **two-stage test strategy** for fast feedback: + +1. **Quick Tests** (`test-quick`) — Runs ~2,790 unit and basic integration tests in ~2 minutes. Excludes tests marked `e2e`, `requires_docker`, `requires_external`, or `slow`. This stage gates the integration tests. + +2. **Integration Tests** (`test-integration`) — Runs ~28 Docker-based and external service tests. Only starts after quick tests pass. Uses testcontainers for WebDAV, OAuth, etc. + +Both stages use `pytest-timeout` to prevent individual tests from hanging: +- Quick tests: 120 seconds per test, 10-minute job timeout +- Integration tests: 300 seconds per test, 20-minute job timeout ```yaml -- name: Run Tests - # Exclude E2E tests - they require Docker-in-Docker (testcontainers) - run: pytest tests/ -v --cov=app -m "not e2e" +# Quick tests (Stage 2b) +- name: Run Quick Tests + run: > + pytest tests/ -v --timeout=120 + -m "not e2e and not requires_docker and not requires_external and not slow" + +# Integration tests (Stage 2c) — only after quick tests pass +- name: Run Integration Tests + run: > + pytest tests/ -v --timeout=300 + -m "(requires_docker or requires_external or slow) and not e2e" ``` **To run E2E tests locally:**