From 9670e8486251de8f83d4faefb4ca370ced7cd8f9 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 6 Mar 2026 15:32:43 +0000 Subject: [PATCH] ci: merge quick and integration tests into a single test step Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com> --- .github/workflows/ci.yml | 18 ++++-------- docs/CIWorkflow.md | 46 ++++++++++++------------------- tests/README_INTEGRATION_TESTS.md | 23 ++++------------ 3 files changed, 28 insertions(+), 59 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 16ac0949..677150c2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -116,20 +116,12 @@ jobs: python -m pip install --upgrade pip pip install -r requirements-dev.txt - - name: Run Quick Tests + - name: Run Tests run: > - pytest tests/ -v --timeout=120 - --cov=app --cov-report=xml:coverage.xml - --junitxml=junit-quick.xml -o junit_family=legacy - -m "not e2e and not requires_docker and not requires_external and not slow" - - - name: Run Integration Tests - if: success() || failure() - run: > - pytest tests/ -v --timeout=300 - --cov=app --cov-append --cov-report=xml:coverage.xml - --junitxml=junit-integration.xml -o junit_family=legacy - -m "(requires_docker or requires_external or slow) and not e2e" + pytest tests/ -v --timeout=300 + --cov=app --cov-report=xml:coverage.xml + --junitxml=junit.xml -o junit_family=legacy + -m "not e2e" - name: Upload Unified Coverage to Codecov if: always() diff --git a/docs/CIWorkflow.md b/docs/CIWorkflow.md index 6f62ce52..616e4b07 100644 --- a/docs/CIWorkflow.md +++ b/docs/CIWorkflow.md @@ -13,8 +13,7 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus | `lint` | Ruff | Fast Python linter (replaces Flake8, Black, isort, Bandit) | ✅ | | `html-lint` | djLint | HTML template accessibility linter | ✅ | | `dependency-scan` | pip-audit | Dependency vulnerability scanning against OSV/PyPA advisories | ✅ | -| `test-quick` | pytest | Unit + basic integration tests with coverage (~2 min) | ✅ | -| `test-integration` | pytest | Docker container and external service tests (~5 min) | ✅ | +| `run-tests` | pytest | All unit + integration tests with coverage (excludes e2e) | ✅ | | `mypy` | mypy | Static type checking | ✅ | ### Pipeline Flow @@ -22,34 +21,24 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus ``` Stage 1 (parallel): lint, html-lint, dependency-scan │ -Stage 2 (parallel): test-quick + mypy +Stage 2 (parallel): run-tests + mypy │ -Stage 3: test-integration (only after test-quick passes) - │ -Stage 4: build (only after all above pass) +Stage 3: build (only after all above pass) ``` -The pipeline follows a **fail-early** strategy: fast linters and quick tests run first to catch regressions early. Heavier integration tests only run after the quick tests pass, saving CI time when basic issues are present. +The pipeline follows a **fail-early** strategy: fast linters run first to catch regressions early, then the full test suite and type checks run in parallel. > **Note:** DocuElevate uses Ruff, a modern all-in-one Python linter that consolidates the functionality of Flake8, Black, isort, and Bandit. This streamlined approach reduces CI complexity while maintaining code quality and security standards. -### Quick Tests (`test-quick`) +### All Tests (`run-tests`) -- **Timeout:** 15 minutes (job), 120 seconds (per test via `pytest-timeout`) -- Runs the majority of tests (~2,790 unit + basic integration tests) -- Excludes tests marked `e2e`, `requires_docker`, `requires_external`, or `slow` -- Uses a Redis service container for tests that need it -- Collects coverage and uploads to Codecov -- Uploads `junit.xml` and `coverage.xml` as workflow artifacts - -### Integration Tests (`test-integration`) - -- **Timeout:** 20 minutes (job), 300 seconds (per test via `pytest-timeout`) -- Runs tests marked `requires_docker`, `requires_external`, or `slow` (excluding `e2e`) +- **Timeout:** 300 seconds (per test via `pytest-timeout`) +- Runs all tests (unit, integration, and Docker-based) in a single step +- Excludes tests marked `e2e` - Uses Redis and RabbitMQ service containers - Docker daemon available for testcontainers (WebDAV, OAuth mock server, etc.) -- Only runs after quick tests pass (fail-early gate) -- Uploads `junit-integration.xml` as a workflow artifact +- Collects coverage and uploads to Codecov +- Uploads `junit.xml` and `coverage.xml` as workflow artifacts ### Ruff Lint & Format @@ -73,8 +62,7 @@ The following artifacts are uploaded after every run: | Artifact | Contents | Condition | |-------------------------------|--------------------------------------|----------------------| -| `test-results-quick` | `junit.xml`, `coverage.xml` | Always (unless cancelled) | -| `test-results-integration` | `junit-integration.xml` | Always (unless cancelled) | +| `test-results` | `junit.xml`, `coverage.xml` | Always (unless cancelled) | ## Running Linters Locally @@ -93,14 +81,14 @@ ruff format --check app/ tests/ # Run type checking mypy app/ -# Run quick tests (same as CI quick stage) -pytest tests/ -v --timeout=120 --cov=app --cov-report=term -m "not e2e and not requires_docker and not requires_external and not slow" +# Run all tests except e2e (same as CI) +pytest tests/ -v --timeout=300 --cov=app --cov-report=term -m "not e2e" -# Run integration tests (requires Docker) +# Run only fast tests locally (skip Docker/external/slow) +pytest tests/ -v --timeout=120 -m "not e2e and not requires_docker and not requires_external and not slow" + +# Run only Docker/external/slow tests locally (requires Docker) pytest tests/ -v --timeout=300 -m "(requires_docker or requires_external or slow) and not e2e" - -# Run all tests except e2e -pytest tests/ -v --timeout=120 -m "not e2e" ``` Or use pre-commit hooks to run checks automatically on each commit (recommended): diff --git a/tests/README_INTEGRATION_TESTS.md b/tests/README_INTEGRATION_TESTS.md index 3ecd3ca9..d3cd16e5 100644 --- a/tests/README_INTEGRATION_TESTS.md +++ b/tests/README_INTEGRATION_TESTS.md @@ -361,28 +361,17 @@ docker volume prune -f ### GitHub Actions - Current Configuration -The DocuElevate CI pipeline (`.github/workflows/ci.yml`) uses a **two-stage test strategy** for fast feedback: +The DocuElevate CI pipeline (`.github/workflows/ci.yml`) runs all non-e2e tests in a single step for simplicity: -1. **Quick Tests** (`test-quick`) — Runs ~2,790 unit and basic integration tests in ~2 minutes. Excludes tests marked `e2e`, `requires_docker`, `requires_external`, or `slow`. This stage gates the integration tests. - -2. **Integration Tests** (`test-integration`) — Runs ~28 Docker-based and external service tests. Only starts after quick tests pass. Uses testcontainers for WebDAV, OAuth, etc. - -Both stages use `pytest-timeout` to prevent individual tests from hanging: -- Quick tests: 120 seconds per test, 15-minute job timeout -- Integration tests: 300 seconds per test, 20-minute job timeout +- All unit, integration, and Docker-based tests run together +- Excludes tests marked `e2e` +- Uses `pytest-timeout` (300 seconds per test) to prevent individual tests from hanging ```yaml -# Quick tests (Stage 2b) -- name: Run Quick Tests - run: > - pytest tests/ -v --timeout=120 - -m "not e2e and not requires_docker and not requires_external and not slow" - -# Integration tests (Stage 2c) — only after quick tests pass -- name: Run Integration Tests +- name: Run Tests run: > pytest tests/ -v --timeout=300 - -m "(requires_docker or requires_external or slow) and not e2e" + -m "not e2e" ``` **To run E2E tests locally:**