From 0dd6ab3c76566f0223c0b1cfde27ae68e06e1180 Mon Sep 17 00:00:00 2001
From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com>
Date: Fri, 6 Feb 2026 22:24:32 +0000
Subject: [PATCH 1/2] Initial plan
From b731256fefae7cb0f25a10d0f530bc2ca83efc53 Mon Sep 17 00:00:00 2001
From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com>
Date: Sat, 7 Feb 2026 13:57:52 +0000
Subject: [PATCH 2/2] Add comprehensive Copilot instructions for repository
Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
---
.github/copilot-instructions.md | 132 +++++++++
.../documentation.instructions.md | 267 ++++++++++++++++++
.github/instructions/frontend.instructions.md | 152 ++++++++++
.../python-backend.instructions.md | 164 +++++++++++
.github/instructions/testing.instructions.md | 245 ++++++++++++++++
5 files changed, 960 insertions(+)
create mode 100644 .github/copilot-instructions.md
create mode 100644 .github/instructions/documentation.instructions.md
create mode 100644 .github/instructions/frontend.instructions.md
create mode 100644 .github/instructions/python-backend.instructions.md
create mode 100644 .github/instructions/testing.instructions.md
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
new file mode 100644
index 00000000..2d96b8ca
--- /dev/null
+++ b/.github/copilot-instructions.md
@@ -0,0 +1,132 @@
+# Copilot Instructions for DocuElevate
+
+## Project Overview
+
+DocuElevate is an intelligent document processing system that automates handling, extraction, and processing of documents. It integrates with multiple cloud storage providers (Dropbox, Google Drive, OneDrive, S3, Nextcloud) and uses AI services (OpenAI, Azure Document Intelligence) for metadata extraction and OCR.
+
+## Tech Stack
+
+- **Backend**: FastAPI, SQLAlchemy, Celery, Redis
+- **Frontend**: Jinja2 templates, Tailwind CSS
+- **AI/ML**: OpenAI API, Azure Document Intelligence
+- **Auth**: Authentik (OAuth2), Basic Auth
+- **Infrastructure**: Docker, Docker Compose, Alembic (migrations)
+- **Testing**: Pytest, pytest-asyncio, httpx
+
+## Core Principles
+
+### Code Quality
+- Always use **Black** for formatting (line length: 120)
+- Use **isort** with Black profile for import sorting
+- Use **flake8** for linting (ignore E203, W503)
+- Use **type hints** for all function parameters and return values
+- Write **docstrings** for all public functions, classes, and modules
+- Maintain **80% test coverage** for new code
+
+### Python Conventions
+- Use descriptive variable names (e.g., `user_document_path`, not `udp`)
+- Follow PEP 8 naming: `snake_case` for functions/variables, `PascalCase` for classes
+- Use type hints from `typing` module (Dict, List, Optional, etc.)
+- Prefer `pathlib.Path` over string paths for file operations
+- Use f-strings for string formatting, not `.format()` or `%`
+- Handle exceptions explicitly - avoid bare `except:` clauses
+
+### Security Best Practices
+- **Never commit secrets or credentials** to the repository
+- Use environment variables for sensitive configuration (see `.env.demo`)
+- Validate and sanitize all user inputs
+- Use parameterized queries with SQLAlchemy (never raw SQL with user input)
+- Review [SECURITY_AUDIT.md](../SECURITY_AUDIT.md) before making security-related changes
+- Run `bandit` to check for security issues in Python code
+
+### FastAPI Patterns
+- Organize endpoints by feature in `app/api/` directory
+- Use dependency injection for database sessions and authentication
+- Return Pydantic models from endpoints for automatic validation
+- Use proper HTTP status codes (200, 201, 400, 401, 403, 404, 500)
+- Document endpoints with docstrings for OpenAPI documentation
+- Use `async def` for I/O-bound operations
+
+### Database (SQLAlchemy)
+- All models are defined in `app/models.py`
+- Use Alembic for schema migrations (create migration for any model change)
+- Use declarative base for models
+- Define relationships with `relationship()` and proper `back_populates`
+- Use database sessions from `app.database.get_db()` dependency
+- Always close sessions in `finally` blocks or use context managers
+
+### Celery Tasks
+- Define tasks in `app/tasks/` directory, organized by feature
+- Use descriptive task names: `module.action` (e.g., `document.process_ocr`)
+- Set appropriate retry policies and error handling
+- Log progress and errors using Python's `logging` module
+- Use `bind=True` for tasks that need access to task instance
+- Keep tasks idempotent when possible
+
+### Frontend
+- Templates are in `frontend/templates/` using Jinja2
+- Static files (CSS, JS, images) in `frontend/static/`
+- Use Tailwind CSS utility classes (already configured)
+- Keep JavaScript minimal - prefer server-side rendering
+- Follow existing template structure and patterns
+
+### Testing
+- Write tests in `tests/` directory, mirroring `app/` structure
+- Use pytest markers: `@pytest.mark.unit`, `@pytest.mark.integration`, etc.
+- Mock external services (OpenAI, Azure, cloud storage) in tests
+- Use `pytest.fixture` for test setup and teardown
+- Run tests with: `pytest -v`
+- Check coverage with: `pytest --cov=app --cov-report=term-missing`
+
+### Configuration
+- All configuration is in `app/config.py` using Pydantic Settings
+- Use environment variables for configuration (12-factor app)
+- Provide sensible defaults when possible
+- Document all configuration options in `docs/ConfigurationGuide.md`
+
+### Documentation
+- Keep documentation in `docs/` directory in Markdown format
+- Update relevant docs when adding features or changing behavior
+- User-facing documentation should be clear and include examples
+- Reference existing docs: `docs/UserGuide.md`, `docs/API.md`, `docs/DeploymentGuide.md`
+- See [AGENTIC_CODING.md](../AGENTIC_CODING.md) for detailed development guide
+
+### Error Handling
+- Use custom exceptions defined in application (follow existing patterns)
+- Log errors with context using Python's `logging` module
+- Return user-friendly error messages in API responses
+- Include error details in development, sanitize in production
+
+### Dependencies
+- Add new dependencies to `requirements.txt` (production) or `requirements-dev.txt` (development)
+- Document any new dependencies and their licenses in README.md
+- Check for security vulnerabilities with `safety check`
+- Pin major versions, allow minor updates (e.g., `fastapi>=0.100.0,<1.0.0`)
+
+### Git Workflow
+- Write clear, descriptive commit messages
+- Keep commits focused and atomic
+- Run tests and linters before committing
+- Pre-commit hooks are configured (`.pre-commit-config.yaml`)
+- Follow conventional commits format when appropriate
+
+### File Organization
+- Place API endpoints in `app/api/` organized by feature
+- Background tasks go in `app/tasks/`
+- Utility functions in `app/utils/`
+- UI routes in `app/views/`
+- Database models in `app/models.py`
+- Configuration in `app/config.py`
+
+### Common Patterns
+- Use `from typing import Optional, Dict, List, Any` for type hints
+- Import FastAPI dependencies: `from fastapi import Depends, HTTPException, status`
+- Get DB session: `db: Session = Depends(get_db)`
+- Current user: `current_user: User = Depends(get_current_user)`
+- Logger: `import logging; logger = logging.getLogger(__name__)`
+
+## Resources
+- [AGENTIC_CODING.md](../AGENTIC_CODING.md) - Comprehensive development guide
+- [CONTRIBUTING.md](../CONTRIBUTING.md) - Contribution guidelines
+- [SECURITY_AUDIT.md](../SECURITY_AUDIT.md) - Security considerations
+- [README.md](../README.md) - Project overview and quickstart
diff --git a/.github/instructions/documentation.instructions.md b/.github/instructions/documentation.instructions.md
new file mode 100644
index 00000000..d815a50d
--- /dev/null
+++ b/.github/instructions/documentation.instructions.md
@@ -0,0 +1,267 @@
+---
+applyTo: "docs/**/*.md"
+---
+
+# Documentation Instructions
+
+These instructions apply to all documentation files in the `docs/` directory.
+
+## Documentation Structure
+- User-facing documentation in `docs/` directory
+- All documentation in Markdown format
+- Follow existing documentation style and structure
+
+## Existing Documentation
+- `docs/UserGuide.md` - How to use DocuElevate
+- `docs/API.md` - API reference and examples
+- `docs/DeploymentGuide.md` - Deployment instructions
+- `docs/ConfigurationGuide.md` - Configuration options
+- `docs/Troubleshooting.md` - Common issues and solutions
+- `AGENTIC_CODING.md` - Development guide for AI agents
+- `CONTRIBUTING.md` - Contribution guidelines
+- `README.md` - Project overview and quickstart
+
+## Markdown Style
+
+### Headers
+```markdown
+# H1 - Document Title (only one per file)
+
+## H2 - Major Sections
+
+### H3 - Subsections
+
+#### H4 - Minor subsections (use sparingly)
+```
+
+### Code Blocks
+Always specify the language for syntax highlighting:
+````markdown
+```python
+def example_function():
+ """Example Python code."""
+ return "Hello, World!"
+```
+
+```bash
+# Shell commands
+docker-compose up -d
+```
+
+```json
+{
+ "key": "value"
+}
+```
+````
+
+### Lists
+```markdown
+- Unordered list item 1
+- Unordered list item 2
+ - Nested item
+ - Another nested item
+
+1. Ordered list item 1
+2. Ordered list item 2
+3. Ordered list item 3
+```
+
+### Links
+```markdown
+[Link text](https://example.com)
+[Internal link](./UserGuide.md)
+[Link to section](#installation)
+```
+
+### Images
+```markdown
+
+
+
+
+
Image caption
+
+
{{ page_title }}
+
+ {% if error_message %}
+
+ {{ error_message }}
+
+ {% endif %}
+
+
+
+ {{ message }}
+
+ {% endfor %}
+ {% endif %}
+{% endwith %}
+```
+
+### URL Generation
+- Always use `url_for()` for URLs, never hardcode
+- Examples:
+ - Routes: `{{ url_for('upload_document') }}`
+ - Static: `{{ url_for('static', path='css/style.css') }}`
+ - API: `{{ url_for('api_document', document_id=doc.id) }}`
+
+### Template Variables
+- Check if variables exist before using them
+- Use filters for formatting
+```jinja2
+{% if document %}
+ Uploaded: {{ document.created_at|datetime }}
+ Size: {{ document.file_size|filesizeformat }}
+{% else %}
+ No document found
+{% endif %}
+```
+
+### Common Components
+- Follow existing patterns for headers, footers, navigation
+- Reuse template blocks and includes
+- Keep components modular
+```jinja2
+{% include 'components/navigation.html' %}
+{% include 'components/document_card.html' with document=doc %}
+```
+
+## UI/UX Guidelines
+- Maintain consistent spacing using Tailwind's scale (4, 8, 16, etc.)
+- Use the existing color palette from the design
+- Ensure mobile responsiveness
+- Show loading indicators for long operations
+- Provide feedback for user actions (success/error messages)
+- Keep the interface clean and minimal
+
+## Performance
+- Optimize images (compress, use appropriate formats)
+- Minimize JavaScript bundle size
+- Use lazy loading for images when appropriate
+- Cache static assets
diff --git a/.github/instructions/python-backend.instructions.md b/.github/instructions/python-backend.instructions.md
new file mode 100644
index 00000000..3626507b
--- /dev/null
+++ b/.github/instructions/python-backend.instructions.md
@@ -0,0 +1,164 @@
+---
+applyTo: "app/**/*.py"
+---
+
+# Python Backend Instructions
+
+These instructions apply to all Python code in the `app/` directory.
+
+## Code Style
+- Use **Black** formatter with 120 character line length
+- Use **isort** with Black profile for import organization
+- Follow **flake8** rules (ignore E203, W503 as per `.pre-commit-config.yaml`)
+- All functions must have type hints for parameters and return values
+- Use `from typing import Optional, Dict, List, Any, Union` as needed
+
+## Import Order (isort with Black profile)
+```python
+# Standard library imports
+import os
+from pathlib import Path
+from typing import Optional, Dict, List
+
+# Third-party imports
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy.orm import Session
+
+# Local application imports
+from app.config import settings
+from app.database import get_db
+from app.models import Document, User
+```
+
+## Function Definitions
+```python
+def process_document(
+ file_path: Path,
+ user_id: int,
+ metadata: Optional[Dict[str, Any]] = None
+) -> DocumentMetadata:
+ """
+ Process a document and extract metadata.
+
+ Args:
+ file_path: Path to the document file
+ user_id: ID of the user uploading the document
+ metadata: Optional additional metadata
+
+ Returns:
+ DocumentMetadata object with extracted information
+
+ Raises:
+ FileNotFoundError: If file doesn't exist
+ ProcessingError: If processing fails
+ """
+ pass
+```
+
+## FastAPI Endpoints
+- Use dependency injection for DB sessions and auth
+- Return Pydantic models for automatic validation
+- Use proper status codes from `fastapi.status`
+- Add detailed docstrings for OpenAPI docs
+```python
+from fastapi import APIRouter, Depends, status
+from sqlalchemy.orm import Session
+
+router = APIRouter(prefix="/api/documents", tags=["documents"])
+
+@router.post("/", status_code=status.HTTP_201_CREATED)
+async def create_document(
+ file: UploadFile,
+ db: Session = Depends(get_db),
+ current_user: User = Depends(get_current_user)
+) -> DocumentResponse:
+ """Create and process a new document."""
+ pass
+```
+
+## Error Handling
+- Use custom exceptions from the application
+- Log errors with context using `logging.getLogger(__name__)`
+- Return user-friendly error messages
+- Never expose internal details in production errors
+```python
+import logging
+
+logger = logging.getLogger(__name__)
+
+try:
+ result = process_file(file_path)
+except FileNotFoundError:
+ logger.error(f"File not found: {file_path}")
+ raise HTTPException(status_code=404, detail="File not found")
+except Exception as e:
+ logger.exception(f"Error processing file: {str(e)}")
+ raise HTTPException(status_code=500, detail="Processing failed")
+```
+
+## Database Operations
+- Use SQLAlchemy ORM, never raw SQL with user input
+- Use `get_db()` dependency for sessions
+- Always commit in try/except blocks
+```python
+from sqlalchemy.orm import Session
+from app.database import get_db
+
+def create_document(db: Session, document_data: dict) -> Document:
+ """Create a new document in the database."""
+ db_document = Document(**document_data)
+ try:
+ db.add(db_document)
+ db.commit()
+ db.refresh(db_document)
+ return db_document
+ except Exception as e:
+ db.rollback()
+ raise
+```
+
+## Celery Tasks
+- Define in `app/tasks/` directory
+- Use descriptive names: `module.action`
+- Set retry policies
+- Log progress and errors
+```python
+from celery import shared_task
+import logging
+
+logger = logging.getLogger(__name__)
+
+@shared_task(bind=True, max_retries=3)
+def process_ocr(self, document_id: int) -> Dict[str, Any]:
+ """Process OCR for a document."""
+ try:
+ # Processing logic
+ logger.info(f"Processing OCR for document {document_id}")
+ return {"status": "success"}
+ except Exception as exc:
+ logger.exception(f"OCR processing failed for {document_id}")
+ raise self.retry(exc=exc, countdown=60)
+```
+
+## Configuration
+- All settings in `app/config.py` using Pydantic Settings
+- Use environment variables, never hardcode values
+- Provide defaults when sensible
+```python
+from pydantic_settings import BaseSettings
+
+class Settings(BaseSettings):
+ openai_api_key: str
+ max_file_size: int = 10485760 # 10MB default
+
+ class Config:
+ env_file = ".env"
+```
+
+## Security
+- Never commit secrets
+- Validate all user inputs
+- Use parameterized queries
+- Sanitize file paths
+- Check file permissions
+- Review SECURITY_AUDIT.md for guidelines
diff --git a/.github/instructions/testing.instructions.md b/.github/instructions/testing.instructions.md
new file mode 100644
index 00000000..37b3961b
--- /dev/null
+++ b/.github/instructions/testing.instructions.md
@@ -0,0 +1,245 @@
+---
+applyTo: "tests/**/*.py"
+---
+
+# Testing Instructions
+
+These instructions apply to all test files in the `tests/` directory.
+
+## Test Organization
+- Mirror the structure of `app/` directory in `tests/`
+- Name test files with `test_` prefix (e.g., `test_api.py`)
+- Group related tests in classes with `Test` prefix
+- Use descriptive test function names: `test___`
+
+## Pytest Configuration
+- Configuration in `pytest.ini`
+- Run tests: `pytest -v`
+- With coverage: `pytest --cov=app --cov-report=term-missing`
+- Run specific markers: `pytest -m unit` or `pytest -m integration`
+
+## Test Markers
+Use pytest markers to categorize tests:
+```python
+import pytest
+
+@pytest.mark.unit
+def test_document_validation():
+ """Test document validation logic."""
+ pass
+
+@pytest.mark.integration
+def test_document_upload_api():
+ """Test document upload endpoint."""
+ pass
+
+@pytest.mark.slow
+def test_large_file_processing():
+ """Test processing of large files."""
+ pass
+
+@pytest.mark.requires_external
+def test_openai_integration():
+ """Test OpenAI API integration."""
+ pass
+```
+
+Available markers:
+- `unit` - Unit tests for individual functions/methods
+- `integration` - Integration tests for API endpoints and workflows
+- `slow` - Tests that take significant time to run
+- `security` - Security-related tests
+- `requires_external` - Tests requiring external services (OpenAI, Azure, etc.)
+- `requires_db` - Tests requiring database
+- `requires_redis` - Tests requiring Redis
+
+## Fixtures
+Use pytest fixtures for test setup and teardown:
+```python
+import pytest
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+from app.database import Base
+
+@pytest.fixture
+def db_session():
+ """Provide a database session for tests."""
+ engine = create_engine("sqlite:///:memory:")
+ Base.metadata.create_all(engine)
+ Session = sessionmaker(bind=engine)
+ session = Session()
+
+ yield session
+
+ session.close()
+ Base.metadata.drop_all(engine)
+
+@pytest.fixture
+def sample_document():
+ """Provide a sample document for tests."""
+ return {
+ "filename": "test.pdf",
+ "content_type": "application/pdf",
+ "size": 1024
+ }
+```
+
+## API Testing with FastAPI
+Use `TestClient` from FastAPI:
+```python
+from fastapi.testclient import TestClient
+from app.main import app
+
+client = TestClient(app)
+
+def test_upload_document():
+ """Test document upload endpoint."""
+ with open("tests/fixtures/sample.pdf", "rb") as f:
+ response = client.post(
+ "/api/documents/upload",
+ files={"file": ("test.pdf", f, "application/pdf")}
+ )
+
+ assert response.status_code == 201
+ assert "id" in response.json()
+```
+
+## Async Testing
+For async code, use `pytest-asyncio`:
+```python
+import pytest
+import httpx
+
+@pytest.mark.asyncio
+async def test_async_document_processing():
+ """Test async document processing."""
+ async with httpx.AsyncClient(app=app, base_url="http://test") as client:
+ response = await client.get("/api/documents/1")
+ assert response.status_code == 200
+```
+
+## Mocking External Services
+Always mock external services in tests:
+```python
+from unittest.mock import Mock, patch
+
+@pytest.mark.unit
+def test_openai_metadata_extraction(mocker):
+ """Test metadata extraction with mocked OpenAI."""
+ mock_response = {
+ "document_type": "invoice",
+ "amount": 100.00,
+ "date": "2024-01-01"
+ }
+
+ mocker.patch(
+ "app.utils.openai_client.extract_metadata",
+ return_value=mock_response
+ )
+
+ result = extract_document_metadata("test.pdf")
+ assert result["document_type"] == "invoice"
+
+@pytest.mark.unit
+def test_azure_ocr_processing(mocker):
+ """Test OCR with mocked Azure service."""
+ mock_text = "Sample extracted text"
+
+ mocker.patch(
+ "app.utils.azure_client.extract_text",
+ return_value=mock_text
+ )
+
+ result = perform_ocr("test.pdf")
+ assert result == mock_text
+```
+
+## Database Testing
+```python
+@pytest.mark.requires_db
+def test_create_document(db_session):
+ """Test document creation in database."""
+ from app.models import Document
+
+ doc = Document(
+ filename="test.pdf",
+ user_id=1,
+ file_path="/tmp/test.pdf"
+ )
+ db_session.add(doc)
+ db_session.commit()
+
+ assert doc.id is not None
+ assert doc.filename == "test.pdf"
+```
+
+## Test Coverage Goals
+- Aim for **80% code coverage** for all new code
+- Focus on critical paths and error handling
+- Test both success and failure scenarios
+- Don't test third-party library code
+
+## Test Structure
+Follow the Arrange-Act-Assert pattern:
+```python
+def test_document_validation():
+ """Test that invalid documents are rejected."""
+ # Arrange
+ invalid_document = {
+ "filename": "", # Empty filename
+ "size": -1 # Invalid size
+ }
+
+ # Act
+ result = validate_document(invalid_document)
+
+ # Assert
+ assert result.is_valid is False
+ assert "filename" in result.errors
+ assert "size" in result.errors
+```
+
+## Parameterized Tests
+Use `pytest.mark.parametrize` for multiple test cases:
+```python
+@pytest.mark.parametrize("filename,expected", [
+ ("document.pdf", True),
+ ("image.jpg", True),
+ ("script.exe", False),
+ ("", False),
+])
+def test_allowed_file_types(filename, expected):
+ """Test file type validation."""
+ result = is_allowed_file(filename)
+ assert result == expected
+```
+
+## Test Data
+- Place test fixtures in `tests/fixtures/` directory
+- Use small sample files for testing
+- Don't commit large test files
+- Clean up test files in teardown
+
+## Error Testing
+Always test error conditions:
+```python
+def test_missing_file_raises_error():
+ """Test that missing files raise appropriate error."""
+ with pytest.raises(FileNotFoundError):
+ process_document("/nonexistent/file.pdf")
+
+def test_invalid_api_request():
+ """Test API error handling."""
+ response = client.post("/api/documents/", json={})
+ assert response.status_code == 422 # Validation error
+```
+
+## Best Practices
+- Test one thing per test function
+- Use descriptive test names
+- Keep tests independent (no dependencies between tests)
+- Use fixtures for common setup
+- Mock external dependencies
+- Test edge cases and error conditions
+- Keep tests fast (use mocks for slow operations)
+- Clean up resources after tests