Add comprehensive Copilot instructions for repository

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
2026-02-07 13:57:52 +00:00
parent 0dd6ab3c76
commit b731256fef
5 changed files with 960 additions and 0 deletions
@@ -0,0 +1,267 @@
+---
+applyTo: "docs/**/*.md"
+---
+
+# Documentation Instructions
+
+These instructions apply to all documentation files in the `docs/` directory.
+
+## Documentation Structure
+- User-facing documentation in `docs/` directory
+- All documentation in Markdown format
+- Follow existing documentation style and structure
+
+## Existing Documentation
+- `docs/UserGuide.md` - How to use DocuElevate
+- `docs/API.md` - API reference and examples
+- `docs/DeploymentGuide.md` - Deployment instructions
+- `docs/ConfigurationGuide.md` - Configuration options
+- `docs/Troubleshooting.md` - Common issues and solutions
+- `AGENTIC_CODING.md` - Development guide for AI agents
+- `CONTRIBUTING.md` - Contribution guidelines
+- `README.md` - Project overview and quickstart
+
+## Markdown Style
+
+### Headers
+```markdown
+# H1 - Document Title (only one per file)
+
+## H2 - Major Sections
+
+### H3 - Subsections
+
+#### H4 - Minor subsections (use sparingly)
+```
+
+### Code Blocks
+Always specify the language for syntax highlighting:
+````markdown
+```python
+def example_function():
+    """Example Python code."""
+    return "Hello, World!"
+```
+
+```bash
+# Shell commands
+docker-compose up -d
+```
+
+```json
+{
+  "key": "value"
+}
+```
+````
+
+### Lists
+```markdown
+- Unordered list item 1
+- Unordered list item 2
+  - Nested item
+  - Another nested item
+
+1. Ordered list item 1
+2. Ordered list item 2
+3. Ordered list item 3
+```
+
+### Links
+```markdown
+[Link text](https://example.com)
+[Internal link](./UserGuide.md)
+[Link to section](#installation)
+```
+
+### Images
+```markdown
+![Alt text](path/to/image.png)
+
+<div align="center">
+  <img src="path/to/image.png" alt="Descriptive alt text" width="80%" />
+  <p><em>Image caption</em></p>
+</div>
+```
+
+### Tables
+```markdown
+| Column 1 | Column 2 | Column 3 |
+|----------|----------|----------|
+| Value 1  | Value 2  | Value 3  |
+| Value 4  | Value 5  | Value 6  |
+```
+
+### Admonitions and Notes
+```markdown
+> **Note:** This is an important note.
+
+> **Warning:** This is a warning message.
+
+> **Tip:** This is a helpful tip.
+```
+
+## Content Guidelines
+
+### Writing Style
+- Use clear, concise language
+- Write in second person (you/your) for user-facing docs
+- Use present tense
+- Avoid jargon; explain technical terms when necessary
+- Use active voice
+- Keep sentences short and focused
+
+### Documentation Types
+
+#### User Documentation
+- Focus on **how to use** features, not implementation details
+- Include step-by-step instructions
+- Provide examples for common use cases
+- Add screenshots or diagrams when helpful
+- Explain what each feature does and when to use it
+
+Example:
+```markdown
+## Uploading Documents
+
+To upload a document to DocuElevate:
+
+1. Navigate to the Upload page
+2. Click "Choose File" and select your document
+3. Select the destination (Dropbox, Google Drive, etc.)
+4. Click "Upload"
+
+The document will be automatically processed and stored in your selected destination.
+```
+
+#### API Documentation
+- Document all endpoints with examples
+- Show request and response formats
+- Include authentication requirements
+- Provide example curl commands
+- Document error responses
+
+Example:
+```markdown
+### POST /api/documents/upload
+
+Upload a new document for processing.
+
+**Authentication:** Required
+
+**Request:**
+```bash
+curl -X POST "http://localhost:8000/api/documents/upload" \
+  -H "Authorization: Bearer YOUR_TOKEN" \
+  -F "file=@document.pdf"
+```
+
+**Response (201 Created):**
+```json
+{
+  "id": 123,
+  "filename": "document.pdf",
+  "status": "processing"
+}
+```
+```
+
+#### Configuration Documentation
+- List all configuration options
+- Provide default values
+- Explain what each option does
+- Include example configurations
+- Note which options are required vs. optional
+
+Example:
+```markdown
+### OPENAI_API_KEY
+
+**Type:** String  
+**Required:** Yes  
+**Default:** None
+
+Your OpenAI API key for metadata extraction.
+
+```bash
+OPENAI_API_KEY=sk-...
+```
+```
+
+#### Troubleshooting Documentation
+- Start with the symptom/error
+- Provide clear diagnosis steps
+- Offer solutions
+- Include common causes
+
+Example:
+```markdown
+### Error: "Connection refused" when starting services
+
+**Cause:** Docker services are not running or ports are already in use.
+
+**Solution:**
+1. Check if Docker is running: `docker ps`
+2. Check port availability: `lsof -i :8000`
+3. Restart Docker services: `docker-compose restart`
+```
+
+## Code Examples
+- Always test code examples before including them
+- Use realistic examples that users can adapt
+- Include comments explaining non-obvious parts
+- Show complete examples, not just fragments
+
+## Version Information
+- Update documentation when changing features
+- Note version numbers when features are added
+- Mark deprecated features clearly
+
+## Cross-References
+- Link to related documentation
+- Reference other sections when appropriate
+- Keep the documentation interconnected
+
+Example:
+```markdown
+For deployment instructions, see the [Deployment Guide](./DeploymentGuide.md).
+
+For API details, refer to the [API Documentation](./API.md).
+```
+
+## Updating Documentation
+When making code changes:
+1. Update relevant documentation in the same PR
+2. Check for outdated information
+3. Add new sections for new features
+4. Update examples if behavior changes
+5. Review related documentation for consistency
+
+## Screenshots and Diagrams
+- Use clear, high-quality images
+- Annotate screenshots when helpful
+- Keep diagrams simple and focused
+- Update screenshots when UI changes
+- Use consistent styling in diagrams
+
+## Accessibility
+- Use descriptive alt text for images
+- Ensure proper heading hierarchy
+- Make links descriptive (avoid "click here")
+- Use semantic formatting (bold, italic, code) appropriately
+
+## README.md Specific
+- Keep README concise and focused on getting started
+- Include badges for build status, version, license
+- Show the most important features first
+- Link to detailed documentation
+- Include quick start instructions
+- Add screenshots of the main interface
+
+## Configuration Guide Updates
+When adding new configuration options:
+- Add to `docs/ConfigurationGuide.md`
+- Include type, default value, and description
+- Provide example usage
+- Note any dependencies on other config options
+- Update `.env.demo` with the new option
@@ -0,0 +1,152 @@
+---
+applyTo: "frontend/**/*"
+---
+
+# Frontend Instructions
+
+These instructions apply to all files in the `frontend/` directory (templates, CSS, JavaScript, images).
+
+## Templates (Jinja2)
+
+### Location and Structure
+- All templates in `frontend/templates/`
+- Use template inheritance with `base.html`
+- Keep templates organized by feature
+
+### Template Patterns
+```jinja2
+{% extends "base.html" %}
+
+{% block title %}Document Upload - DocuElevate{% endblock %}
+
+{% block content %}
+<div class="container mx-auto px-4 py-8">
+    <h1 class="text-2xl font-bold mb-4">{{ page_title }}</h1>
+    
+    {% if error_message %}
+    <div class="bg-red-100 border border-red-400 text-red-700 px-4 py-3 rounded mb-4">
+        {{ error_message }}
+    </div>
+    {% endif %}
+    
+    <form method="post" enctype="multipart/form-data">
+        <!-- Form content -->
+    </form>
+</div>
+{% endblock %}
+```
+
+### Tailwind CSS Usage
+- Use Tailwind utility classes (already configured)
+- Follow responsive design: `md:`, `lg:` breakpoints
+- Use existing color scheme from the project
+- Common patterns:
+  - Containers: `container mx-auto px-4`
+  - Buttons: `bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded`
+  - Cards: `bg-white shadow-md rounded-lg p-6`
+  - Forms: `w-full px-3 py-2 border rounded`
+
+### Static Files
+- CSS files in `frontend/static/css/`
+- JavaScript in `frontend/static/js/`
+- Images in `frontend/static/images/`
+- Reference with `{{ url_for('static', path='css/style.css') }}`
+
+### JavaScript
+- Keep JavaScript minimal - prefer server-side rendering
+- Use vanilla JavaScript or minimal dependencies
+- Place scripts at the end of the body
+- Use `defer` or `async` for external scripts
+```html
+<script src="{{ url_for('static', path='js/upload.js') }}" defer></script>
+```
+
+### Forms
+- Use CSRF protection when needed
+- Include proper validation
+- Show clear error messages
+- Use proper `method` (GET/POST) and `enctype` for file uploads
+```html
+<form method="post" enctype="multipart/form-data">
+    <div class="mb-4">
+        <label class="block text-gray-700 text-sm font-bold mb-2" for="file">
+            Document File
+        </label>
+        <input 
+            type="file" 
+            id="file" 
+            name="file"
+            class="w-full px-3 py-2 border rounded"
+            required
+        />
+    </div>
+    <button type="submit" class="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded">
+        Upload
+    </button>
+</form>
+```
+
+### Accessibility
+- Use semantic HTML elements (`nav`, `main`, `article`, `section`)
+- Include `alt` text for images
+- Use proper heading hierarchy (h1 → h2 → h3)
+- Add ARIA labels when needed
+- Ensure keyboard navigation works
+
+### Error Handling
+- Display user-friendly error messages
+- Use flash messages for feedback
+- Show loading states for async operations
+```jinja2
+{% with messages = get_flashed_messages(with_categories=true) %}
+    {% if messages %}
+        {% for category, message in messages %}
+        <div class="bg-{{ category }}-100 border border-{{ category }}-400 text-{{ category }}-700 px-4 py-3 rounded mb-4">
+            {{ message }}
+        </div>
+        {% endfor %}
+    {% endif %}
+{% endwith %}
+```
+
+### URL Generation
+- Always use `url_for()` for URLs, never hardcode
+- Examples:
+  - Routes: `{{ url_for('upload_document') }}`
+  - Static: `{{ url_for('static', path='css/style.css') }}`
+  - API: `{{ url_for('api_document', document_id=doc.id) }}`
+
+### Template Variables
+- Check if variables exist before using them
+- Use filters for formatting
+```jinja2
+{% if document %}
+    <p>Uploaded: {{ document.created_at|datetime }}</p>
+    <p>Size: {{ document.file_size|filesizeformat }}</p>
+{% else %}
+    <p>No document found</p>
+{% endif %}
+```
+
+### Common Components
+- Follow existing patterns for headers, footers, navigation
+- Reuse template blocks and includes
+- Keep components modular
+```jinja2
+{% include 'components/navigation.html' %}
+{% include 'components/document_card.html' with document=doc %}
+```
+
+## UI/UX Guidelines
+- Maintain consistent spacing using Tailwind's scale (4, 8, 16, etc.)
+- Use the existing color palette from the design
+- Ensure mobile responsiveness
+- Show loading indicators for long operations
+- Provide feedback for user actions (success/error messages)
+- Keep the interface clean and minimal
+
+## Performance
+- Optimize images (compress, use appropriate formats)
+- Minimize JavaScript bundle size
+- Use lazy loading for images when appropriate
+- Cache static assets
@@ -0,0 +1,164 @@
+---
+applyTo: "app/**/*.py"
+---
+
+# Python Backend Instructions
+
+These instructions apply to all Python code in the `app/` directory.
+
+## Code Style
+- Use **Black** formatter with 120 character line length
+- Use **isort** with Black profile for import organization
+- Follow **flake8** rules (ignore E203, W503 as per `.pre-commit-config.yaml`)
+- All functions must have type hints for parameters and return values
+- Use `from typing import Optional, Dict, List, Any, Union` as needed
+
+## Import Order (isort with Black profile)
+```python
+# Standard library imports
+import os
+from pathlib import Path
+from typing import Optional, Dict, List
+
+# Third-party imports
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy.orm import Session
+
+# Local application imports
+from app.config import settings
+from app.database import get_db
+from app.models import Document, User
+```
+
+## Function Definitions
+```python
+def process_document(
+    file_path: Path,
+    user_id: int,
+    metadata: Optional[Dict[str, Any]] = None
+) -> DocumentMetadata:
+    """
+    Process a document and extract metadata.
+    
+    Args:
+        file_path: Path to the document file
+        user_id: ID of the user uploading the document
+        metadata: Optional additional metadata
+        
+    Returns:
+        DocumentMetadata object with extracted information
+        
+    Raises:
+        FileNotFoundError: If file doesn't exist
+        ProcessingError: If processing fails
+    """
+    pass
+```
+
+## FastAPI Endpoints
+- Use dependency injection for DB sessions and auth
+- Return Pydantic models for automatic validation
+- Use proper status codes from `fastapi.status`
+- Add detailed docstrings for OpenAPI docs
+```python
+from fastapi import APIRouter, Depends, status
+from sqlalchemy.orm import Session
+
+router = APIRouter(prefix="/api/documents", tags=["documents"])
+
+@router.post("/", status_code=status.HTTP_201_CREATED)
+async def create_document(
+    file: UploadFile,
+    db: Session = Depends(get_db),
+    current_user: User = Depends(get_current_user)
+) -> DocumentResponse:
+    """Create and process a new document."""
+    pass
+```
+
+## Error Handling
+- Use custom exceptions from the application
+- Log errors with context using `logging.getLogger(__name__)`
+- Return user-friendly error messages
+- Never expose internal details in production errors
+```python
+import logging
+
+logger = logging.getLogger(__name__)
+
+try:
+    result = process_file(file_path)
+except FileNotFoundError:
+    logger.error(f"File not found: {file_path}")
+    raise HTTPException(status_code=404, detail="File not found")
+except Exception as e:
+    logger.exception(f"Error processing file: {str(e)}")
+    raise HTTPException(status_code=500, detail="Processing failed")
+```
+
+## Database Operations
+- Use SQLAlchemy ORM, never raw SQL with user input
+- Use `get_db()` dependency for sessions
+- Always commit in try/except blocks
+```python
+from sqlalchemy.orm import Session
+from app.database import get_db
+
+def create_document(db: Session, document_data: dict) -> Document:
+    """Create a new document in the database."""
+    db_document = Document(**document_data)
+    try:
+        db.add(db_document)
+        db.commit()
+        db.refresh(db_document)
+        return db_document
+    except Exception as e:
+        db.rollback()
+        raise
+```
+
+## Celery Tasks
+- Define in `app/tasks/` directory
+- Use descriptive names: `module.action`
+- Set retry policies
+- Log progress and errors
+```python
+from celery import shared_task
+import logging
+
+logger = logging.getLogger(__name__)
+
+@shared_task(bind=True, max_retries=3)
+def process_ocr(self, document_id: int) -> Dict[str, Any]:
+    """Process OCR for a document."""
+    try:
+        # Processing logic
+        logger.info(f"Processing OCR for document {document_id}")
+        return {"status": "success"}
+    except Exception as exc:
+        logger.exception(f"OCR processing failed for {document_id}")
+        raise self.retry(exc=exc, countdown=60)
+```
+
+## Configuration
+- All settings in `app/config.py` using Pydantic Settings
+- Use environment variables, never hardcode values
+- Provide defaults when sensible
+```python
+from pydantic_settings import BaseSettings
+
+class Settings(BaseSettings):
+    openai_api_key: str
+    max_file_size: int = 10485760  # 10MB default
+    
+    class Config:
+        env_file = ".env"
+```
+
+## Security
+- Never commit secrets
+- Validate all user inputs
+- Use parameterized queries
+- Sanitize file paths
+- Check file permissions
+- Review SECURITY_AUDIT.md for guidelines
@@ -0,0 +1,245 @@
+---
+applyTo: "tests/**/*.py"
+---
+
+# Testing Instructions
+
+These instructions apply to all test files in the `tests/` directory.
+
+## Test Organization
+- Mirror the structure of `app/` directory in `tests/`
+- Name test files with `test_` prefix (e.g., `test_api.py`)
+- Group related tests in classes with `Test` prefix
+- Use descriptive test function names: `test_<what>_<condition>_<expected>`
+
+## Pytest Configuration
+- Configuration in `pytest.ini`
+- Run tests: `pytest -v`
+- With coverage: `pytest --cov=app --cov-report=term-missing`
+- Run specific markers: `pytest -m unit` or `pytest -m integration`
+
+## Test Markers
+Use pytest markers to categorize tests:
+```python
+import pytest
+
+@pytest.mark.unit
+def test_document_validation():
+    """Test document validation logic."""
+    pass
+
+@pytest.mark.integration
+def test_document_upload_api():
+    """Test document upload endpoint."""
+    pass
+
+@pytest.mark.slow
+def test_large_file_processing():
+    """Test processing of large files."""
+    pass
+
+@pytest.mark.requires_external
+def test_openai_integration():
+    """Test OpenAI API integration."""
+    pass
+```
+
+Available markers:
+- `unit` - Unit tests for individual functions/methods
+- `integration` - Integration tests for API endpoints and workflows
+- `slow` - Tests that take significant time to run
+- `security` - Security-related tests
+- `requires_external` - Tests requiring external services (OpenAI, Azure, etc.)
+- `requires_db` - Tests requiring database
+- `requires_redis` - Tests requiring Redis
+
+## Fixtures
+Use pytest fixtures for test setup and teardown:
+```python
+import pytest
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+from app.database import Base
+
+@pytest.fixture
+def db_session():
+    """Provide a database session for tests."""
+    engine = create_engine("sqlite:///:memory:")
+    Base.metadata.create_all(engine)
+    Session = sessionmaker(bind=engine)
+    session = Session()
+    
+    yield session
+    
+    session.close()
+    Base.metadata.drop_all(engine)
+
+@pytest.fixture
+def sample_document():
+    """Provide a sample document for tests."""
+    return {
+        "filename": "test.pdf",
+        "content_type": "application/pdf",
+        "size": 1024
+    }
+```
+
+## API Testing with FastAPI
+Use `TestClient` from FastAPI:
+```python
+from fastapi.testclient import TestClient
+from app.main import app
+
+client = TestClient(app)
+
+def test_upload_document():
+    """Test document upload endpoint."""
+    with open("tests/fixtures/sample.pdf", "rb") as f:
+        response = client.post(
+            "/api/documents/upload",
+            files={"file": ("test.pdf", f, "application/pdf")}
+        )
+    
+    assert response.status_code == 201
+    assert "id" in response.json()
+```
+
+## Async Testing
+For async code, use `pytest-asyncio`:
+```python
+import pytest
+import httpx
+
+@pytest.mark.asyncio
+async def test_async_document_processing():
+    """Test async document processing."""
+    async with httpx.AsyncClient(app=app, base_url="http://test") as client:
+        response = await client.get("/api/documents/1")
+        assert response.status_code == 200
+```
+
+## Mocking External Services
+Always mock external services in tests:
+```python
+from unittest.mock import Mock, patch
+
+@pytest.mark.unit
+def test_openai_metadata_extraction(mocker):
+    """Test metadata extraction with mocked OpenAI."""
+    mock_response = {
+        "document_type": "invoice",
+        "amount": 100.00,
+        "date": "2024-01-01"
+    }
+    
+    mocker.patch(
+        "app.utils.openai_client.extract_metadata",
+        return_value=mock_response
+    )
+    
+    result = extract_document_metadata("test.pdf")
+    assert result["document_type"] == "invoice"
+
+@pytest.mark.unit
+def test_azure_ocr_processing(mocker):
+    """Test OCR with mocked Azure service."""
+    mock_text = "Sample extracted text"
+    
+    mocker.patch(
+        "app.utils.azure_client.extract_text",
+        return_value=mock_text
+    )
+    
+    result = perform_ocr("test.pdf")
+    assert result == mock_text
+```
+
+## Database Testing
+```python
+@pytest.mark.requires_db
+def test_create_document(db_session):
+    """Test document creation in database."""
+    from app.models import Document
+    
+    doc = Document(
+        filename="test.pdf",
+        user_id=1,
+        file_path="/tmp/test.pdf"
+    )
+    db_session.add(doc)
+    db_session.commit()
+    
+    assert doc.id is not None
+    assert doc.filename == "test.pdf"
+```
+
+## Test Coverage Goals
+- Aim for **80% code coverage** for all new code
+- Focus on critical paths and error handling
+- Test both success and failure scenarios
+- Don't test third-party library code
+
+## Test Structure
+Follow the Arrange-Act-Assert pattern:
+```python
+def test_document_validation():
+    """Test that invalid documents are rejected."""
+    # Arrange
+    invalid_document = {
+        "filename": "",  # Empty filename
+        "size": -1  # Invalid size
+    }
+    
+    # Act
+    result = validate_document(invalid_document)
+    
+    # Assert
+    assert result.is_valid is False
+    assert "filename" in result.errors
+    assert "size" in result.errors
+```
+
+## Parameterized Tests
+Use `pytest.mark.parametrize` for multiple test cases:
+```python
+@pytest.mark.parametrize("filename,expected", [
+    ("document.pdf", True),
+    ("image.jpg", True),
+    ("script.exe", False),
+    ("", False),
+])
+def test_allowed_file_types(filename, expected):
+    """Test file type validation."""
+    result = is_allowed_file(filename)
+    assert result == expected
+```
+
+## Test Data
+- Place test fixtures in `tests/fixtures/` directory
+- Use small sample files for testing
+- Don't commit large test files
+- Clean up test files in teardown
+
+## Error Testing
+Always test error conditions:
+```python
+def test_missing_file_raises_error():
+    """Test that missing files raise appropriate error."""
+    with pytest.raises(FileNotFoundError):
+        process_document("/nonexistent/file.pdf")
+
+def test_invalid_api_request():
+    """Test API error handling."""
+    response = client.post("/api/documents/", json={})
+    assert response.status_code == 422  # Validation error
+```
+
+## Best Practices
+- Test one thing per test function
+- Use descriptive test names
+- Keep tests independent (no dependencies between tests)
+- Use fixtures for common setup
+- Mock external dependencies
+- Test edge cases and error conditions
+- Keep tests fast (use mocks for slow operations)
+- Clean up resources after tests