Merge pull request #346 from christianlouis/copilot/add-dependency-vulnerability-scanning
feat(ci): add pip-audit dependency vulnerability scanning to CI/CD
This commit is contained in:
@@ -55,13 +55,38 @@ jobs:
|
|||||||
run: ruff format --check app/ tests/
|
run: ruff format --check app/ tests/
|
||||||
|
|
||||||
# ══════════════════════════════════════════════════════════════════════════
|
# ══════════════════════════════════════════════════════════════════════════
|
||||||
# Stage 2: Tests & Type Checking (run in parallel after lint passes)
|
# Stage 2a: Dependency Vulnerability Scan (runs in parallel with lint)
|
||||||
|
# ══════════════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
dependency-scan:
|
||||||
|
name: Dependency Vulnerability Scan
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Checkout Code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.11"
|
||||||
|
|
||||||
|
- name: Install pip-audit
|
||||||
|
run: pip install pip-audit>=2.7.0
|
||||||
|
|
||||||
|
- name: Run pip-audit on production dependencies
|
||||||
|
run: pip-audit -r requirements.txt --desc on
|
||||||
|
|
||||||
|
- name: Run pip-audit on dev dependencies
|
||||||
|
run: pip-audit -r requirements-dev.txt --desc on
|
||||||
|
|
||||||
|
# ══════════════════════════════════════════════════════════════════════════
|
||||||
|
# Stage 2b: Tests & Type Checking (run in parallel after lint passes)
|
||||||
# ══════════════════════════════════════════════════════════════════════════
|
# ══════════════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
test:
|
test:
|
||||||
name: Tests
|
name: Tests
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
needs: [lint] # Wait for lint to pass before running tests
|
needs: [lint, dependency-scan] # Wait for lint and dependency scan before running tests
|
||||||
services:
|
services:
|
||||||
redis:
|
redis:
|
||||||
image: redis:7
|
image: redis:7
|
||||||
@@ -128,7 +153,7 @@ jobs:
|
|||||||
mypy:
|
mypy:
|
||||||
name: Mypy
|
name: Mypy
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
needs: [lint] # Wait for lint to pass before running type checks
|
needs: [lint, dependency-scan] # Wait for lint and dependency scan before running type checks
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout Code
|
- name: Checkout Code
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
@@ -153,7 +178,7 @@ jobs:
|
|||||||
build:
|
build:
|
||||||
name: Build & Push Docker Image
|
name: Build & Push Docker Image
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
needs: [test, lint, mypy]
|
needs: [test, lint, mypy, dependency-scan]
|
||||||
if: github.event_name == 'push'
|
if: github.event_name == 'push'
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
|
|||||||
+28
-2
@@ -291,11 +291,17 @@ Tests are organized using pytest markers:
|
|||||||
|
|
||||||
Tests run automatically in GitHub Actions for all pull requests. The CI workflow is organized in stages:
|
Tests run automatically in GitHub Actions for all pull requests. The CI workflow is organized in stages:
|
||||||
|
|
||||||
**Stage 1: Ruff Lint & Format** (runs first)
|
**Stage 1: Ruff Lint & Format** (runs first, in parallel with dependency scan)
|
||||||
- Checks code style, formatting, and basic security issues
|
- Checks code style, formatting, and basic security issues
|
||||||
- Must pass before tests run
|
- Must pass before tests run
|
||||||
|
|
||||||
**Stage 2: Tests & Type Checking** (runs after lint passes)
|
**Stage 1b: Dependency Vulnerability Scan** (runs in parallel with lint)
|
||||||
|
- Runs `pip-audit` against `requirements.txt` and `requirements-dev.txt`
|
||||||
|
- Fails the build if any known vulnerabilities are detected
|
||||||
|
- Checks the OSV and PyPA advisory databases
|
||||||
|
- Runs independently at the same time as Stage 1 so it does not add to total pipeline time
|
||||||
|
|
||||||
|
**Stage 2: Tests & Type Checking** (runs after lint and dependency scan both pass)
|
||||||
|
|
||||||
| Job | Tool | What it checks |
|
| Job | Tool | What it checks |
|
||||||
|--------|--------|--------------------------------------|
|
|--------|--------|--------------------------------------|
|
||||||
@@ -339,6 +345,26 @@ ruff format app/ tests/
|
|||||||
|
|
||||||
**Note:** The pre-commit hooks and CI pipeline will automatically check (and optionally fix) these for you.
|
**Note:** The pre-commit hooks and CI pipeline will automatically check (and optionally fix) these for you.
|
||||||
|
|
||||||
|
### Dependency Vulnerability Scanning
|
||||||
|
|
||||||
|
DocuElevate uses **pip-audit** to scan dependencies for known security vulnerabilities. The CI pipeline runs this automatically and **blocks builds** if any vulnerabilities are found.
|
||||||
|
|
||||||
|
To run locally before pushing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Scan production dependencies
|
||||||
|
pip-audit -r requirements.txt --desc on
|
||||||
|
|
||||||
|
# Scan all dependencies (including dev)
|
||||||
|
pip-audit -r requirements-dev.txt --desc on
|
||||||
|
```
|
||||||
|
|
||||||
|
If pip-audit is not installed, add it with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install pip-audit
|
||||||
|
```
|
||||||
|
|
||||||
## Project Structure
|
## Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|||||||
+3
-3
@@ -246,7 +246,7 @@ ftp = ftplib.FTP() # nosec B321 - Plaintext FTP intentional when configured
|
|||||||
### Dependency Management
|
### Dependency Management
|
||||||
- ✅ Version pinning for security-critical packages (authlib, starlette)
|
- ✅ Version pinning for security-critical packages (authlib, starlette)
|
||||||
- ✅ Advisory database checks integrated into development workflow
|
- ✅ Advisory database checks integrated into development workflow
|
||||||
- ⏳ TODO: Add automated dependency vulnerability scanning in CI/CD ([#171](https://github.com/christianlouis/DocuElevate/issues/171))
|
- ✅ Automated dependency vulnerability scanning in CI/CD via pip-audit ([#171](https://github.com/christianlouis/DocuElevate/issues/171))
|
||||||
|
|
||||||
### Authentication & Secrets
|
### Authentication & Secrets
|
||||||
- ✅ Strong validation for SESSION_SECRET (minimum 32 characters)
|
- ✅ Strong validation for SESSION_SECRET (minimum 32 characters)
|
||||||
@@ -265,8 +265,8 @@ ftp = ftplib.FTP() # nosec B321 - Plaintext FTP intentional when configured
|
|||||||
- ✅ **COMPLETED:** Bandit (Python security linter) audit completed
|
- ✅ **COMPLETED:** Bandit (Python security linter) audit completed
|
||||||
- ✅ **COMPLETED:** Bandit integrated into CI pipeline (fails on high/medium severity issues)
|
- ✅ **COMPLETED:** Bandit integrated into CI pipeline (fails on high/medium severity issues)
|
||||||
- ✅ **COMPLETED:** CodeQL security scanning enabled in GitHub Actions
|
- ✅ **COMPLETED:** CodeQL security scanning enabled in GitHub Actions
|
||||||
- ⏳ **TODO:** Add dependency vulnerability scanning (Safety, pip-audit) to CI ([#171](https://github.com/christianlouis/DocuElevate/issues/171))
|
- ✅ **COMPLETED:** pip-audit dependency vulnerability scanning added to CI ([#171](https://github.com/christianlouis/DocuElevate/issues/171))
|
||||||
- ⏳ **TODO:** Make dependency scans blocking (fail on critical issues) ([#171](https://github.com/christianlouis/DocuElevate/issues/171))
|
- ✅ **COMPLETED:** Dependency scans are blocking (fail build when vulnerabilities detected) ([#171](https://github.com/christianlouis/DocuElevate/issues/171))
|
||||||
|
|
||||||
### Code Security
|
### Code Security
|
||||||
- ✅ Authentication required on all sensitive endpoints (@require_login decorator)
|
- ✅ Authentication required on all sensitive endpoints (@require_login decorator)
|
||||||
|
|||||||
@@ -217,7 +217,7 @@ DeepSource provided:
|
|||||||
|
|
||||||
**Analysis:**
|
**Analysis:**
|
||||||
- Would duplicate CodeQL for security scanning
|
- Would duplicate CodeQL for security scanning
|
||||||
- Would duplicate Dependabot for dependency vulnerabilities
|
- Would duplicate Dependabot + pip-audit for dependency vulnerabilities
|
||||||
- Current tools provide adequate coverage
|
- Current tools provide adequate coverage
|
||||||
- Not needed at this time
|
- Not needed at this time
|
||||||
|
|
||||||
@@ -231,6 +231,7 @@ DeepSource provided:
|
|||||||
| Security Linting | Bandit | Ruff | ✅ Consolidated |
|
| Security Linting | Bandit | Ruff | ✅ Consolidated |
|
||||||
| Code Quality | Pylint, DeepSource | Ruff | ✅ Consolidated |
|
| Code Quality | Pylint, DeepSource | Ruff | ✅ Consolidated |
|
||||||
| Security Scanning | CodeQL, DeepSource | CodeQL | ✅ De-duplicated |
|
| Security Scanning | CodeQL, DeepSource | CodeQL | ✅ De-duplicated |
|
||||||
|
| Dependency Vulnerabilities | None | pip-audit | ✅ Added |
|
||||||
| Type Checking | Mypy | Mypy | ✅ Kept (unique) |
|
| Type Checking | Mypy | Mypy | ✅ Kept (unique) |
|
||||||
| Testing | pytest | pytest | ✅ Kept (unique) |
|
| Testing | pytest | pytest | ✅ Kept (unique) |
|
||||||
| Coverage | Codecov | Codecov | ✅ Kept (unique) |
|
| Coverage | Codecov | Codecov | ✅ Kept (unique) |
|
||||||
|
|||||||
+3
-2
@@ -10,11 +10,12 @@ The CI workflow (`.github/workflows/tests.yaml`) runs automatically on every pus
|
|||||||
|
|
||||||
| Job | Tool | Purpose | Enforced |
|
| Job | Tool | Purpose | Enforced |
|
||||||
|--------|--------|---------------------------------------------|----------|
|
|--------|--------|---------------------------------------------|----------|
|
||||||
| `test` | pytest | Unit/integration tests with coverage | ✅ |
|
|
||||||
| `lint` | Ruff | Fast Python linter (replaces Flake8, Black, isort, Bandit) | ✅ |
|
| `lint` | Ruff | Fast Python linter (replaces Flake8, Black, isort, Bandit) | ✅ |
|
||||||
|
| `dependency-scan` | pip-audit | Dependency vulnerability scanning against OSV/PyPA advisories | ✅ |
|
||||||
|
| `test` | pytest | Unit/integration tests with coverage | ✅ |
|
||||||
| `mypy` | mypy | Static type checking | ✅ |
|
| `mypy` | mypy | Static type checking | ✅ |
|
||||||
|
|
||||||
All three jobs start **in parallel** as soon as the workflow is triggered. No job depends on or waits for any other job.
|
`lint` and `dependency-scan` start **in parallel** at the beginning of the pipeline — neither depends on the other. `test` and `mypy` run only after both have passed.
|
||||||
|
|
||||||
> **Note:** DocuElevate uses Ruff, a modern all-in-one Python linter that consolidates the functionality of Flake8, Black, isort, and Bandit. This streamlined approach reduces CI complexity while maintaining code quality and security standards.
|
> **Note:** DocuElevate uses Ruff, a modern all-in-one Python linter that consolidates the functionality of Flake8, Black, isort, and Bandit. This streamlined approach reduces CI complexity while maintaining code quality and security standards.
|
||||||
|
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ types-requests>=2.31.0
|
|||||||
types-paramiko>=3.0.0
|
types-paramiko>=3.0.0
|
||||||
|
|
||||||
# Security scanning (Ruff includes most security checks from bandit)
|
# Security scanning (Ruff includes most security checks from bandit)
|
||||||
|
pip-audit>=2.7.0 # Dependency vulnerability scanning against OSV/PyPA advisory databases
|
||||||
|
|
||||||
# Pre-commit hooks
|
# Pre-commit hooks
|
||||||
pre-commit>=3.6.0
|
pre-commit>=3.6.0
|
||||||
|
|||||||
Reference in New Issue
Block a user