gh-christianlouis-docuelevate/docs/CIWorkflow.md

CI Workflow Guide

This document describes the CI/CD pipeline for DocuElevate, focusing on the Run Tests & Linting workflow.

Overview

The CI workflow (.github/workflows/tests.yaml) runs automatically on every push and pull request. It is designed so that each linter and the test suite run as independent jobs, ensuring that a failure in one tool never blocks the results from another.

Workflow Jobs

Job	Tool	Purpose	Enforced
test	pytest	Unit/integration tests with coverage	✅
flake8	flake8	PEP 8 style linting	✅
black	black	Code formatting check	✅
mypy	mypy	Static type checking	✅
pylint	pylint	Code quality analysis	✅
bandit	bandit	Security vulnerability scanning	✅

All six jobs start in parallel as soon as the workflow is triggered. No job depends on or waits for any other job.

Tests

• Runs pytest with coverage reporting (XML + terminal).
• Excludes E2E tests that require Docker-in-Docker (-m "not e2e").
• Uses Redis and RabbitMQ service containers.
• Uploads coverage and JUnit XML results to Codecov.
• Uploads junit.xml and coverage.xml as workflow artifacts (always, even on failure).

Flake8

• Checks app/ against PEP 8 with max-line-length=120.
• Ignores E203 (whitespace before :) and W503 (line break before binary operator), matching the Black formatter.

Black

• Verifies that all files in app/ are formatted with black --line-length=120.
• Runs in --check mode (no files are modified).

Mypy

• Type checks app/ with --ignore-missing-imports.
• Requires full project dependencies (installs requirements-dev.txt).

Pylint

• Analyzes app/ with max-line-length=120.
• Disables C0111 (missing docstrings), C0103 (naming conventions), and R0903 (too few public methods).
• Requires full project dependencies (installs requirements-dev.txt).

Bandit

• Produces a full JSON report (bandit-report.json) uploaded as a workflow artifact.
• Fails the job if any high- or medium-severity issues are found (-ll flag).
• The JSON report is always uploaded, even if the severity check fails.

Artifacts

The following artifacts are uploaded after every run:

Artifact	Contents	Condition
test-results	junit.xml, coverage.xml	Always (unless cancelled)
bandit-report	bandit-report.json	Always (unless cancelled)

Running Linters Locally

You can run the same checks locally before pushing:

# Install dev dependencies
pip install -r requirements-dev.txt

# Run each linter
flake8 app/ --max-line-length=120 --extend-ignore=E203,W503
black --check app/ --line-length=120
mypy app/ --ignore-missing-imports
pylint app/ --max-line-length=120 --disable=C0111,C0103,R0903
bandit -r app/ -ll

# Run tests
pytest tests/ -v --cov=app --cov-report=term -m "not e2e"

Or use pre-commit hooks to run checks automatically on each commit:

pip install pre-commit
pre-commit install
pre-commit run --all-files

Design Decisions

Why Separate Jobs Instead of Steps?

Previously, all linters ran as sequential steps in a single job. This meant:

• A failure in flake8 would prevent black, mypy, pylint, and bandit from running.
• Contributors only saw feedback from the first tool that failed, not all of them.

By splitting into independent jobs:

• All tools always run regardless of other failures.
• Contributors see all feedback in a single CI run.
• Jobs run in parallel, reducing total wall-clock time.

Why Are All Linters Enforced?

All linters are set to fail the CI (no continue-on-error). This ensures:

• The codebase stays consistently formatted (Black).
• Style issues are caught early (Flake8).
• Type errors surface before merge (Mypy).
• Code quality standards are maintained (Pylint).
• Security issues are flagged immediately (Bandit).

Copilot Code Compliance

All code — whether written by hand or suggested by GitHub Copilot — goes through the same CI pipeline. Copilot-generated code is linted, type-checked, and security-scanned identically to human-written code. Contributors using Copilot should ensure suggestions pass all checks before committing.