diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 3d9b53c5..fdfd90a7 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -66,36 +66,50 @@ pytest --cov=app --cov-report=html ## Lint / Format Commands ```bash -# Format code with Black (line length 120) -black app/ tests/ - -# Sort imports with isort (Black-compatible profile) -isort app/ tests/ - -# Lint with flake8 (max line length 120, ignores E203/W503) -flake8 app/ --max-line-length=120 +# Format and lint with Ruff (replaces Black, isort, Flake8, Bandit — all-in-one) +ruff format app/ tests/ +ruff check app/ tests/ --fix # Type checking with mypy mypy app/ -# Security lint with bandit (excludes tests) -bandit -r app/ - # Check for dependency vulnerabilities safety check -# Run all pre-commit hooks at once +# Run all pre-commit hooks at once (recommended — runs ruff, mypy, secret detection, etc.) pre-commit run --all-files ``` +## Agent Workflow (Follow for Every Task) + +Follow these steps **in order** for every task — do not skip any: + +1. **Understand** — read the issue/request in full before writing any code +2. **Explore** — search the codebase for existing patterns and relevant implementations +3. **Plan** — outline your changes as a checklist before starting +4. **Implement** — make the smallest correct change that solves the problem +5. **Test** — write or update tests; new code requires 100% test coverage +6. **Document** — update all relevant docs in `docs/`; this is mandatory, not optional +7. **Quality Gate** — run the single gate command below and fix every failure before committing: + +```bash +ruff format app/ tests/ && \ +ruff check app/ tests/ --fix && \ +safety check && \ +pytest --tb=short -q +``` + +8. **Review** — re-read your own diff; confirm it is clean, secure, minimal, and well-documented + +> All commands in the quality gate must exit with code 0. Never submit with failures. + ## Core Principles ### Code Quality - **Security first**: treat security as a non-negotiable requirement, not an afterthought — review [SECURITY_AUDIT.md](../SECURITY_AUDIT.md) for every change - Write **clean, modern, well-documented code** — prioritize readability, maintainability, and idiomatic Python -- Always use **Black** for formatting (line length: 120) -- Use **isort** with Black profile for import sorting -- Use **flake8** for linting (ignore E203, W503) +- Use **Ruff** for all formatting, linting, import sorting, and security scanning — `ruff format` + `ruff check --fix` (replaces Black, isort, Flake8, Bandit) +- Line length: 120 characters (configured in `pyproject.toml`) - Use **type hints** for all function parameters and return values - Write **docstrings** for all public functions, classes, and modules - Maintain **100% test coverage** for new code @@ -103,7 +117,8 @@ pre-commit run --all-files ### Python Conventions - Use descriptive variable names (e.g., `user_document_path`, not `udp`) - Follow PEP 8 naming: `snake_case` for functions/variables, `PascalCase` for classes -- Use type hints from `typing` module (Dict, List, Optional, etc.) +- Use modern Python 3.10+ type hints: `list[str]`, `dict[str, Any]`, `str | None` — avoid `List`, `Dict`, `Optional` from `typing` +- Only import from `typing` for `Any`, `Callable`, `TypeVar`, `Protocol`, and other constructs unavailable natively - Prefer `pathlib.Path` over string paths for file operations - Use f-strings for string formatting, not `.format()` or `%` - Handle exceptions explicitly - avoid bare `except:` clauses @@ -114,7 +129,8 @@ pre-commit run --all-files - Validate and sanitize all user inputs - Use parameterized queries with SQLAlchemy (never raw SQL with user input) - Review [SECURITY_AUDIT.md](../SECURITY_AUDIT.md) before making security-related changes -- Run `bandit` to check for security issues in Python code +- Security linting is built into Ruff via `S` rules — runs automatically with `ruff check`; fix all `S`-prefixed findings +- Run `safety check` to scan dependencies for known CVEs before submitting any PR ### FastAPI Patterns - Organize endpoints by feature in `app/api/` directory @@ -318,7 +334,7 @@ These files are managed entirely by the semantic-release automation. - Configuration in `app/config.py` ### Common Patterns -- Use `from typing import Optional, Dict, List, Any` for type hints +- Use modern Python 3.10+ type hints: `list[str]`, `dict[str, Any]`, `str | None`; only import from `typing` for `Any`, `Callable`, `TypeVar`, `Protocol` - Import FastAPI dependencies: `from fastapi import Depends, HTTPException, status` - Get DB session: `db: Session = Depends(get_db)` - Current user: `current_user: User = Depends(get_current_user)` diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md index 8dffcbcd..9c5d041e 100644 --- a/.github/instructions/documentation.instructions.md +++ b/.github/instructions/documentation.instructions.md @@ -230,12 +230,15 @@ For API details, refer to the [API Documentation](./API.md). ``` ## Updating Documentation +Documentation updates are **mandatory** — every PR that changes code must include matching documentation updates in the same PR. There are no exceptions. + When making code changes: -1. Update relevant documentation in the same PR -2. Check for outdated information +1. **Update relevant documentation** in the same PR — never defer docs to a follow-up +2. Check for outdated information in existing docs 3. Add new sections for new features 4. Update examples if behavior changes 5. Review related documentation for consistency +6. Update `docs/ConfigurationGuide.md` and `.env.demo` for any new or changed configuration options ## Screenshots and Diagrams - Use clear, high-quality images diff --git a/.github/instructions/python-backend.instructions.md b/.github/instructions/python-backend.instructions.md index 7532cb73..c7498b95 100644 --- a/.github/instructions/python-backend.instructions.md +++ b/.github/instructions/python-backend.instructions.md @@ -7,16 +7,18 @@ applyTo: "app/**/*.py" These instructions apply to all Python code in the `app/` directory. ## Code Style -- Use **Ruff** for linting and formatting with 120 character line length +- Use **Ruff** for all formatting, linting, import sorting, and security scanning — `ruff format app/ tests/ && ruff check app/ tests/ --fix` +- Line length: 120 characters (configured in `pyproject.toml` `[tool.ruff]`) - All functions must have type hints for parameters and return values -- Use `from typing import Optional, Dict, List, Any, Union` as needed +- Use modern Python 3.10+ type hints: `list[str]`, `dict[str, Any]`, `str | None` +- Only import from `typing` for `Any`, `Callable`, `TypeVar`, `Protocol` (not `Dict`, `List`, `Optional`, `Union`) -## Import Order (Ruff enforces isort-compatible ordering) +## Import Order (enforced by Ruff `I` rules) ```python # Standard library imports import os from pathlib import Path -from typing import Optional, Dict, List +from typing import Any # Only for Any, Callable, TypeVar, Protocol # Third-party imports from fastapi import APIRouter, Depends, HTTPException @@ -33,7 +35,7 @@ from app.models import Document, User def process_document( file_path: Path, user_id: int, - metadata: Optional[Dict[str, Any]] = None + metadata: dict[str, Any] | None = None ) -> DocumentMetadata: """ Process a document and extract metadata. @@ -127,7 +129,7 @@ import logging logger = logging.getLogger(__name__) @shared_task(bind=True, max_retries=3) -def process_ocr(self, document_id: int) -> Dict[str, Any]: +def process_ocr(self, document_id: int) -> dict[str, Any]: """Process OCR for a document.""" try: # Processing logic @@ -153,10 +155,11 @@ class Settings(BaseSettings): env_file = ".env" ``` -## Security -- Never commit secrets -- Validate all user inputs -- Use parameterized queries -- Sanitize file paths -- Check file permissions -- Review SECURITY_AUDIT.md for guidelines +## Security (First and Foremost) +- **Security first**: treat every change as a potential attack surface — review `SECURITY_AUDIT.md` before making any security-related change +- Never commit secrets, tokens, or credentials +- Validate and sanitize all user inputs +- Use parameterized queries — never raw SQL with user data +- Sanitize file paths; check file permissions before access +- Security linting is built into Ruff via `S` rules — fix all `S`-prefixed findings before committing +- Run `safety check` before submitting any PR to catch dependency CVEs diff --git a/.github/instructions/testing.instructions.md b/.github/instructions/testing.instructions.md index ee9f78db..bc0284b4 100644 --- a/.github/instructions/testing.instructions.md +++ b/.github/instructions/testing.instructions.md @@ -174,7 +174,8 @@ def test_create_document(db_session): ``` ## Test Coverage Goals -- Aim for **80% code coverage** for all new code +- Achieve **100% test coverage** for all new code — use `# pragma: no cover` only for genuinely unreachable or platform-specific branches, with an inline comment explaining why +- Enforce the threshold: `pytest --cov=app --cov-fail-under=100` - Focus on critical paths and error handling - Test both success and failure scenarios - Don't test third-party library code