gh-christianlouis-docuelevate/docs/CIToolsGuide.md

CI/CD Tools Guide

This document provides a comprehensive overview of the CI/CD tools used in DocuElevate, including the rationale for tool selection and the de-duplication strategy that was applied.

Overview

DocuElevate uses a streamlined, non-redundant set of CI/CD tools to ensure code quality, security, and reliability without bloat. The tool selection prioritizes:

1. No duplication: Each tool serves a unique purpose
2. Performance: Fast feedback in CI runs
3. Developer experience: Clear, actionable feedback
4. Modern tooling: Active development and support
5. Cost-effectiveness: Preference for free/open-source tools

Current Tool Stack

Core CI Tools

Tool	Purpose	Frequency	Status
Ruff	Linting, formatting, security (Python)	Every push/PR	✅ Active
Mypy	Static type checking (Python)	Every push/PR	✅ Active
pytest	Unit and integration testing	Every push/PR	✅ Active
CodeQL	Advanced security scanning	Push to main, PR, weekly	✅ Active
Codecov	Coverage tracking and reporting	Every push/PR	✅ Active
Pre-commit	Local checks before commit	Pre-commit hook	✅ Active
Dependabot	Dependency security updates	Daily	✅ Active

Tool Details

Ruff - All-in-One Python Linter

What it does:

• PEP 8 style checking (replaces Flake8)
• Code formatting (replaces Black)
• Import sorting (replaces isort)
• Security vulnerability detection (replaces Bandit)
• Code quality checks (replaces parts of Pylint)

Why we chose it:

• 10-100x faster than traditional tools
• Single configuration file (pyproject.toml)
• Written in Rust, actively maintained
• Auto-fix capability for most issues
• Comprehensive rule set (1000+ rules)

Configuration: pyproject.toml → [tool.ruff]

Commands:

# Lint code
ruff check app/ tests/

# Auto-fix issues
ruff check app/ tests/ --fix

# Format code
ruff format app/ tests/

Mypy - Type Checking

What it does:

• Static type analysis for Python code
• Catches type-related bugs before runtime
• Enforces type hint usage

Why we chose it:

• Industry standard for Python type checking
• Unique value - no other tool provides this
• Excellent IDE integration
• Configurable strictness levels

Configuration: pyproject.toml → [tool.mypy]

Commands:

mypy app/

pytest - Testing Framework

What it does:

• Runs unit and integration tests
• Generates coverage reports
• Provides test result reporting

Why we chose it:

• Modern Python testing standard
• Rich plugin ecosystem
• Excellent fixture support
• Built-in parameterization

Configuration: pyproject.toml → [tool.pytest.ini_options]

Commands:

# Run all tests
pytest tests/ -v

# Run with coverage
pytest tests/ -v --cov=app --cov-report=term-missing

# Run specific test markers
pytest -m unit
pytest -m integration

CodeQL - Advanced Security Scanning

What it does:

• Deep semantic code analysis
• Detects security vulnerabilities
• Finds complex code quality issues
• Scans Python, JavaScript, GitHub Actions

Why we chose it:

• Free for open-source projects
• Native GitHub integration
• Enterprise-grade scanning
• GitHub Advanced Security features
• Results appear in Security tab

Configuration: .github/workflows/codeql.yml

Frequency: Push to main, PRs, weekly scheduled scan

Codecov - Coverage Tracking

What it does:

• Visualizes test coverage trends over time
• Comments on PRs with a per-file coverage diff
• Enforces project-level and patch-level coverage thresholds
• Provides embeddable coverage badges
• Annotates PR diff lines with coverage status

Why we chose it:

• Free for open-source projects
• Excellent GitHub PR integration
• Clear coverage visualization and trend graphs
• Industry standard with broad toolchain support

Configuration files:

• codecov.yml — repository-level settings (thresholds, flags, PR comments)
• .github/workflows/ci.yml (run-tests job) — uploads coverage.xml via codecov/codecov-action@v5
• pyproject.toml ([tool.coverage.*]) — what pytest-cov measures and omits

Initial Setup (Repository Admins)

1. Sign in to Codecov at https://app.codecov.io using your GitHub account.
2. Add the repository: click Add new repository and select DocuElevate.
3. Copy the upload token shown on the repository settings page.
4. Store the token as a GitHub Actions secret:
   • Navigate to Settings → Secrets and variables → Actions → New repository secret
   • Name: CODECOV_TOKEN
   • Value: paste the token copied in step 3
5. Push a commit to trigger CI. The run-tests job will upload coverage.xml and Codecov will begin reporting.

Adjusting Coverage Thresholds

Edit codecov.yml at the repository root:

coverage:
  status:
    project:
      default:
        target: 60%      # Minimum overall project coverage
        threshold: 2%    # Allowed drop compared to base branch
    patch:
      default:
        target: 70%      # Minimum coverage of lines changed in a PR
        threshold: 5%    # Allowed slack on patch coverage

Commit and push the change — Codecov picks it up automatically.

Using Coverage Flags

Flags let you track unit and integration coverage separately. The run-tests job in ci.yml uploads a single unified report; to split it add a flags parameter to the upload step for each job:

- name: Upload Unified Coverage to Codecov
  uses: codecov/codecov-action@v5
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    files: ./coverage.xml
    flags: unittests       # matches flag_management in codecov.yml
    fail_ci_if_error: true

The flag names (unittests, integration) are defined in codecov.yml under flag_management.individual_flags.

Embedding the Coverage Badge

Add the following Markdown to README.md, replacing <owner> and <repo>:

<!-- Public repository (no token needed) -->
[![codecov](https://codecov.io/gh/<owner>/<repo>/graph/badge.svg)](https://codecov.io/gh/<owner>/<repo>)

<!-- Private repository (include token for badge access) -->
[![codecov](https://codecov.io/gh/<owner>/<repo>/graph/badge.svg?token=<TOKEN>)](https://codecov.io/gh/<owner>/<repo>)

The exact badge snippet (with the correct token pre-filled) is shown on the Codecov repository overview page under Settings → Badge.

Accessing Coverage Reports

Where	What you see
Codecov dashboard (https://app.codecov.io)	Full report, trend graphs, file explorer
GitHub PR comment	Per-file diff, overall Δ, patch coverage
GitHub Checks tab	Pass/fail status for project and patch thresholds
CI artifacts	coverage.xml (machine-readable), HTML report (human-readable)

To view the HTML report locally:

pytest --cov=app --cov-report=html
open htmlcov/index.html

Troubleshooting

Symptom	Likely cause	Fix
"Token is invalid" upload error	CODECOV_TOKEN secret missing or wrong	Re-copy the token from Codecov settings and update the secret
No PR comment posted	Repository is private and token not set	Set CODECOV_TOKEN — required for private repos
Coverage always 0%	coverage.xml not generated	Confirm pytest runs with --cov=app --cov-report=xml:coverage.xml
CI fails with "coverage decreased"	Patch coverage dropped below threshold	Improve test coverage for the changed lines, or adjust threshold in codecov.yml
Codecov not picking up codecov.yml	YAML syntax error	Validate the file locally with python3 -c "import yaml; yaml.safe_load(open('codecov.yml'))", or use the web validator at the Codecov dashboard (Settings → YAML)
Flags not appearing in dashboard	Flag name mismatch	Ensure the flags: value in the upload step matches an entry in codecov.yml

Pre-commit - Local Quality Gates

What it does:

• Runs checks before git commit
• Prevents committing bad code
• Enforces conventional commits
• Detects secrets and security issues

Why we chose it:

• Catches issues before CI
• Fast local feedback
• Configurable hook selection
• Wide ecosystem support

Configuration: .pre-commit-config.yaml

Hooks included:

• Ruff linting and formatting
• Mypy type checking
• Secret detection (detect-secrets)
• Conventional commit validation
• YAML/JSON validation
• Trailing whitespace removal
• Large file detection

Commands:

# Install hooks
pre-commit install

# Run on all files
pre-commit run --all-files

# Run on staged files
pre-commit run

De-duplication Strategy

Removed Tools

DeepSource ❌ Removed

Reason for removal: Redundant with Ruff + CodeQL

DeepSource provided:

• Static analysis → Now covered by Ruff
• Security scanning → Now covered by Ruff + CodeQL
• Code quality metrics → Now covered by Ruff

Why it was redundant:

• Overlapped 90% with Ruff's capabilities
• CodeQL provides superior security scanning
• No unique value proposition
• Added CI complexity without benefit

Action taken: Removed .deepsource.toml configuration file

Tools Not Configured (No Action Needed)

SonarQube ❌ Not configured

Status: No configuration found in repository

Analysis:

• Enterprise-focused tool
• Best for large organizations needing quality gates and dashboards
• Would duplicate Ruff + CodeQL capabilities
• Not needed for this project's scale

Snyk ❌ Not configured

Status: No configuration found in repository

Analysis:

• Would duplicate CodeQL for security scanning
• Would duplicate Dependabot + pip-audit for dependency vulnerabilities
• Current tools provide adequate coverage
• Not needed at this time

Tool Overlap Analysis (Before De-duplication)

Capability	Old Setup	New Setup	Status
PEP 8 Style	Flake8, Pylint	Ruff	✅ Consolidated
Code Formatting	Black	Ruff	✅ Consolidated
Import Sorting	isort	Ruff	✅ Consolidated
Security Linting	Bandit	Ruff	✅ Consolidated
Code Quality	Pylint, DeepSource	Ruff	✅ Consolidated
Security Scanning	CodeQL, DeepSource	CodeQL	✅ De-duplicated
Dependency Vulnerabilities	None	pip-audit	✅ Added
Type Checking	Mypy	Mypy	✅ Kept (unique)
Testing	pytest	pytest	✅ Kept (unique)
Coverage	Codecov	Codecov	✅ Kept (unique)

Workflow Structure

Tests & Linting Workflow (.github/workflows/tests.yaml)

Runs on every push and pull request.

Jobs (run in parallel):

1. test

   • Runs pytest with coverage
   • Uploads results to Codecov
   • Provides artifacts (junit.xml, coverage.xml)

2. lint

   • Runs Ruff check (linting)
   • Runs Ruff format (formatting validation)

3. mypy

   • Runs type checking

Result: All three jobs complete independently, providing comprehensive feedback even if one fails.

CodeQL Workflow (.github/workflows/codeql.yml)

Runs on:

• Push to main branch
• Pull requests to main
• Weekly schedule (Mondays at 1:37 AM UTC)

Languages scanned:

• Python
• JavaScript/TypeScript
• GitHub Actions

Release Workflow (.github/workflows/release.yml)

Runs on push to main branch.

Actions:

• Generates version based on conventional commits
• Updates CHANGELOG.md
• Creates Git tags
• Triggers Docker builds

Docker Build Workflows

• docker-ci.yml - Builds and pushes Docker images on main branch
• docker-build.yaml - Builds on tags and branches

Best Practices

For Contributors

1. Install pre-commit hooks (recommended):

   pip install pre-commit
   pre-commit install
   

2. Run checks locally before pushing:

   # Quick check
   ruff check app/ tests/
   mypy app/
   pytest tests/ -m unit
   
   # Full check (what CI runs)
   ruff check app/ tests/
   ruff format --check app/ tests/
   mypy app/
   pytest tests/ -v --cov=app -m "not e2e"
   

3. Use Ruff auto-fix to resolve most issues automatically:

   ruff check app/ tests/ --fix
   ruff format app/ tests/
   

4. Follow conventional commits (enforced by pre-commit):

   • feat: for new features
   • fix: for bug fixes
   • docs: for documentation
   • refactor:, test:, chore:, etc.

For Maintainers

1. Review CodeQL security alerts in the Security tab regularly
2. Monitor Codecov reports to ensure coverage doesn't drop
3. Update dependencies via Dependabot PRs promptly
4. Review CI failures for patterns indicating needed tool configuration changes

Performance Metrics

CI Run Time (Typical)

Job	Duration	Status
test	~2-3 min	✅ Fast
lint (Ruff)	~10-15 sec	✅ Very Fast
mypy	~30-45 sec	✅ Fast
Total (parallel)	~2-3 min	✅ Fast

Before De-duplication

• Total jobs: 6+ (Flake8, Black, isort, Pylint, Bandit, tests, Mypy)
• Total run time: ~5-7 minutes
• Tool overlap: High

After De-duplication

• Total jobs: 3 (Ruff, Mypy, tests)
• Total run time: ~2-3 minutes
• Tool overlap: None

Improvement: 40-50% faster CI runs with zero functionality loss.

Future Considerations

Potential Additions (Only if Needed)

1. Performance monitoring (if performance becomes an issue)

   • Tool: Lighthouse CI for frontend
   • Tool: py-spy for Python profiling

2. End-to-end testing (if integration testing is insufficient)

   • Tool: Playwright or Selenium
   • Currently handled by pytest with testcontainers locally

3. Dependency license scanning (if needed for compliance)

   • Tool: licensee or similar

Tools to Avoid (Redundant)

• ❌ SonarQube (overlaps with Ruff + CodeQL)
• ❌ Snyk (overlaps with CodeQL + Dependabot)
• ❌ Additional Python linters (Ruff is comprehensive)
• ❌ Additional formatters (Ruff format is sufficient)

Related Documentation

• CI Workflow Guide - Detailed workflow documentation
• Contributing Guide - Contribution guidelines including testing
• AGENTIC_CODING.md - Development guide for AI agents
• pyproject.toml - Tool configurations

Summary

DocuElevate's CI/CD pipeline is designed to be:

• Lean: No redundant tools
• Fast: Parallel execution, fast tools (Ruff)
• Comprehensive: Security, quality, testing all covered
• Developer-friendly: Clear feedback, auto-fix capabilities
• Maintainable: Single configuration file, modern tools

The de-duplication effort removed DeepSource and consolidated 6 separate linting tools into Ruff, resulting in faster CI runs without sacrificing code quality or security coverage.