gh-christianlouis-docuelevate/.github/copilot-instructions.md

Copilot Instructions for DocuElevate

Project Overview

DocuElevate is an intelligent document processing system that automates handling, extraction, and processing of documents. It integrates with multiple cloud storage providers (Dropbox, Google Drive, OneDrive, S3, Nextcloud) and uses AI services (OpenAI, Azure Document Intelligence) for metadata extraction and OCR.

Tech Stack

• Backend: FastAPI, SQLAlchemy, Celery, Redis
• Frontend: Jinja2 templates, Tailwind CSS
• AI/ML: OpenAI API, Azure Document Intelligence
• Auth: Authentik (OAuth2), Basic Auth
• Infrastructure: Docker, Docker Compose, Alembic (migrations)
• Testing: Pytest, pytest-asyncio, httpx

Core Principles

Code Quality

• Always use Black for formatting (line length: 120)
• Use isort with Black profile for import sorting
• Use flake8 for linting (ignore E203, W503)
• Use type hints for all function parameters and return values
• Write docstrings for all public functions, classes, and modules
• Maintain 80% test coverage for new code

Python Conventions

• Use descriptive variable names (e.g., user_document_path, not udp)
• Follow PEP 8 naming: snake_case for functions/variables, PascalCase for classes
• Use type hints from typing module (Dict, List, Optional, etc.)
• Prefer pathlib.Path over string paths for file operations
• Use f-strings for string formatting, not .format() or %
• Handle exceptions explicitly - avoid bare except: clauses

Security Best Practices

• Never commit secrets or credentials to the repository
• Use environment variables for sensitive configuration (see .env.demo)
• Validate and sanitize all user inputs
• Use parameterized queries with SQLAlchemy (never raw SQL with user input)
• Review SECURITY_AUDIT.md before making security-related changes
• Run bandit to check for security issues in Python code

FastAPI Patterns

• Organize endpoints by feature in app/api/ directory
• Use dependency injection for database sessions and authentication
• Return Pydantic models from endpoints for automatic validation
• Use proper HTTP status codes (200, 201, 400, 401, 403, 404, 500)
• Document endpoints with docstrings for OpenAPI documentation
• Use async def for I/O-bound operations

Database (SQLAlchemy)

• All models are defined in app/models.py
• Use Alembic for schema migrations (create migration for any model change)
• Use declarative base for models
• Define relationships with relationship() and proper back_populates
• Use database sessions from app.database.get_db() dependency
• Always close sessions in finally blocks or use context managers

Celery Tasks

• Define tasks in app/tasks/ directory, organized by feature
• Use descriptive task names: module.action (e.g., document.process_ocr)
• Set appropriate retry policies and error handling
• Log progress and errors using Python's logging module
• Use bind=True for tasks that need access to task instance
• Keep tasks idempotent when possible

Frontend

• Templates are in frontend/templates/ using Jinja2
• Static files (CSS, JS, images) in frontend/static/
• Use Tailwind CSS utility classes (already configured)
• Keep JavaScript minimal - prefer server-side rendering
• Follow existing template structure and patterns

Testing

• Write tests in tests/ directory, mirroring app/ structure
• Use pytest markers: @pytest.mark.unit, @pytest.mark.integration, etc.
• Mock external services (OpenAI, Azure, cloud storage) in tests
• Use pytest.fixture for test setup and teardown
• Run tests with: pytest -v
• Check coverage with: pytest --cov=app --cov-report=term-missing

Configuration

• All configuration is in app/config.py using Pydantic Settings
• Use environment variables for configuration (12-factor app)
• Provide sensible defaults when possible
• Document all configuration options in docs/ConfigurationGuide.md

Documentation

• Keep documentation in docs/ directory in Markdown format
• Update relevant docs when adding features or changing behavior
• User-facing documentation should be clear and include examples
• Reference existing docs: docs/UserGuide.md, docs/API.md, docs/DeploymentGuide.md
• See AGENTIC_CODING.md for detailed development guide

Error Handling

• Use custom exceptions defined in application (follow existing patterns)
• Log errors with context using Python's logging module
• Return user-friendly error messages in API responses
• Include error details in development, sanitize in production

Dependencies

• Add new dependencies to requirements.txt (production) or requirements-dev.txt (development)
• Document any new dependencies and their licenses in README.md
• Check for security vulnerabilities with safety check
• Pin major versions, allow minor updates (e.g., fastapi>=0.100.0,<1.0.0)

Git Workflow

• Write clear, descriptive commit messages
• ALWAYS follow Conventional Commits format (see below)
• Keep commits focused and atomic
• Run tests and linters before committing
• Pre-commit hooks are configured (.pre-commit-config.yaml)

Conventional Commits (REQUIRED)

All commit messages MUST follow the Conventional Commits specification.

Format

<type>(<scope>): <subject>

<body>

<footer>

Commit Types and Version Impact

• feat: New feature → minor version bump (0.5.0 → 0.6.0)
• fix: Bug fix → patch version bump (0.5.0 → 0.5.1)
• perf: Performance improvement → patch version bump
• docs: Documentation only → no version bump
• style: Formatting changes → no version bump
• refactor: Code refactoring → no version bump
• test: Test changes → no version bump
• build: Build system changes → no version bump
• ci: CI/CD changes → no version bump
• chore: Other changes → no version bump

Breaking Changes

For breaking changes (major version bump), add ! after type or include BREAKING CHANGE: in footer:

feat(api)!: redesign authentication endpoints

BREAKING CHANGE: OAuth2 tokens now required instead of API keys.

Result: 0.5.0 → 1.0.0

Scope Examples

• api - REST API changes
• ui - Frontend changes
• auth - Authentication
• storage - Storage providers
• ocr - OCR processing
• tasks - Celery tasks
• config - Configuration
• docs - Documentation

Commit Examples

feat(storage): add Amazon S3 storage provider
fix(ocr): handle PDFs without text layer
docs: update deployment guide with Docker setup
refactor(tasks): consolidate duplicate code
test: add integration tests for upload API
chore: update dependencies for security fixes

Semantic Release Process

Automated Versioning

DocuElevate uses python-semantic-release for automated version management:

1. On merge to main: semantic-release analyzes commit messages
2. Automatic actions:
   • Determines next version from commit types
   • Updates VERSION file
   • Generates/updates CHANGELOG.md
   • Creates Git tag with v prefix (e.g., v0.6.0)
   • Creates GitHub Release with auto-generated notes
   • Triggers Docker image builds with version tag

Agent Rules for Versioning

• ✅ DO: Write conventional commit messages
• ✅ DO: Use appropriate commit types for your changes
• ✅ DO: Mark breaking changes explicitly
• ❌ DON'T: Manually edit VERSION file
• ❌ DON'T: Manually edit CHANGELOG.md
• ❌ DON'T: Create version tags or GitHub Releases manually

These files are managed entirely by the semantic-release automation.

File Organization

• Place API endpoints in app/api/ organized by feature
• Background tasks go in app/tasks/
• Utility functions in app/utils/
• UI routes in app/views/
• Database models in app/models.py
• Configuration in app/config.py

Common Patterns

• Use from typing import Optional, Dict, List, Any for type hints
• Import FastAPI dependencies: from fastapi import Depends, HTTPException, status
• Get DB session: db: Session = Depends(get_db)
• Current user: current_user: User = Depends(get_current_user)
• Logger: import logging; logger = logging.getLogger(__name__)

Resources

• AGENTIC_CODING.md - Comprehensive development guide
• CONTRIBUTING.md - Contribution guidelines
• SECURITY_AUDIT.md - Security considerations
• README.md - Project overview and quickstart