diff --git a/.env.demo b/.env.demo index 6c8dc303..7334834e 100644 --- a/.env.demo +++ b/.env.demo @@ -97,6 +97,22 @@ MAX_UPLOAD_SIZE=1073741824 # Allowed request headers (use * to allow all) # CORS_ALLOWED_HEADERS=* +# **Audit Logging & SIEM Integration** (see docs/ConfigurationGuide.md#audit-logging) +# Enable HTTP request audit logging middleware +AUDIT_LOGGING_ENABLED=true +# Include client IP in audit log entries (disable for GDPR-sensitive deployments) +AUDIT_LOG_INCLUDE_CLIENT_IP=true + +# Forward audit events to an external SIEM system (Syslog, Splunk, Logstash, Grafana, etc.) +# AUDIT_SIEM_ENABLED=false +# AUDIT_SIEM_TRANSPORT=syslog # syslog | http +# AUDIT_SIEM_SYSLOG_HOST=localhost +# AUDIT_SIEM_SYSLOG_PORT=514 +# AUDIT_SIEM_SYSLOG_PROTOCOL=udp # udp | tcp +# AUDIT_SIEM_HTTP_URL= # e.g. https://splunk:8088/services/collector/event +# AUDIT_SIEM_HTTP_TOKEN= # Bearer / HEC token +# AUDIT_SIEM_HTTP_CUSTOM_HEADERS= # Comma-separated Key:Value pairs + # **Rate Limiting** (see SECURITY_AUDIT.md and docs/API.md) # Protects against DoS attacks and API abuse by limiting request rates per IP/user # Enabled by default - highly recommended for production @@ -511,3 +527,12 @@ EMBEDDING_MAX_TOKENS=8000 # Attach PII (IP addresses, user agents) to Sentry events. # Disable (default) to stay GDPR/CCPA compliant. # SENTRY_SEND_DEFAULT_PII=false + +# **Mobile App – Push Notifications** +# Push notifications are delivered via Expo's push notification service +# (https://expo.dev/notifications) which routes to APNs (iOS) and FCM (Android). +# No additional credentials are required on the server side. +# The mobile app registers its Expo push token via POST /api/mobile/register-device. +# +# To use native FCM/APNs directly (without Expo relay), replace the +# send_expo_push_notification function in app/utils/push_notification.py. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 677150c2..0c206c46 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -17,6 +17,7 @@ concurrency: env: IMAGE_NAME: christianlouis/docuelevate + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true jobs: # ══════════════════════════════════════════════════════════════════════════ diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index d1f3781c..a7879042 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -8,6 +8,9 @@ on: schedule: - cron: '37 1 * * 1' +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true + jobs: analyze: name: Analyze (${{ matrix.language }}) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 56466fcb..275e0565 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -12,6 +12,9 @@ permissions: pull-requests: write packages: write +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true + jobs: release: name: Semantic Release diff --git a/.github/workflows/ruff-auto-fix.yml b/.github/workflows/ruff-auto-fix.yml index a834b523..6e1a6748 100644 --- a/.github/workflows/ruff-auto-fix.yml +++ b/.github/workflows/ruff-auto-fix.yml @@ -16,6 +16,9 @@ permissions: contents: write pull-requests: write +env: + FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true + jobs: ruff-auto-fix: name: Auto-fix Ruff Issues diff --git a/BUILD_DATE b/BUILD_DATE index ce1b9463..a9655f30 100644 --- a/BUILD_DATE +++ b/BUILD_DATE @@ -1 +1 @@ -2026-03-09T23:00:57Z +2026-03-11T23:43:28Z diff --git a/CHANGELOG.md b/CHANGELOG.md index 1568c12d..d14ac1ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,116 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 +## v0.118.0 (2026-03-11) + +### Features + +- **api**: Add GraphQL endpoint at /graphql with Strawberry + ([`a41ded5`](https://github.com/christianlouis/DocuElevate/commit/a41ded535f32d4892199208e1cab54cc189fde13)) + + +## v0.117.1 (2026-03-11) + +### Bug Fixes + +- Resolve all 47 failing tests in main + ([`df4c91a`](https://github.com/christianlouis/DocuElevate/commit/df4c91a58661e2ecf9fda15500e4ba1259674168)) + + +## v0.117.0 (2026-03-11) + +### Continuous Integration + +- Opt into Node.js 24 for all GitHub Actions workflows + ([`c306d80`](https://github.com/christianlouis/DocuElevate/commit/c306d80755193e4b12222c73ed4da442a2d5e23c)) + +### Documentation + +- **changelog**: Update changelog [skip ci] + ([`ab19ae5`](https://github.com/christianlouis/DocuElevate/commit/ab19ae57060ff2be8e9172714c5e3b308c45334d)) + +- **changelog**: Update changelog [skip ci] + ([`c5b67fe`](https://github.com/christianlouis/DocuElevate/commit/c5b67fe36464b0756553cc54614d0f77273772ec)) + +- **changelog**: Update changelog [skip ci] + ([`eb22a58`](https://github.com/christianlouis/DocuElevate/commit/eb22a580651b4de477b3b5cc945581776a6775fa)) + + +## Unreleased + +### Continuous Integration + +- Opt into Node.js 24 for all GitHub Actions workflows + ([`c306d80`](https://github.com/christianlouis/DocuElevate/commit/c306d80755193e4b12222c73ed4da442a2d5e23c)) + +### Documentation + +- **changelog**: Update changelog [skip ci] + ([`c5b67fe`](https://github.com/christianlouis/DocuElevate/commit/c5b67fe36464b0756553cc54614d0f77273772ec)) + +- **changelog**: Update changelog [skip ci] + ([`eb22a58`](https://github.com/christianlouis/DocuElevate/commit/eb22a580651b4de477b3b5cc945581776a6775fa)) + + +## Unreleased + +### Continuous Integration + +- Opt into Node.js 24 for all GitHub Actions workflows + ([`c306d80`](https://github.com/christianlouis/DocuElevate/commit/c306d80755193e4b12222c73ed4da442a2d5e23c)) + +### Documentation + +- **changelog**: Update changelog [skip ci] + ([`eb22a58`](https://github.com/christianlouis/DocuElevate/commit/eb22a580651b4de477b3b5cc945581776a6775fa)) + + +## Unreleased + +### Continuous Integration + +- Opt into Node.js 24 for all GitHub Actions workflows + ([`c306d80`](https://github.com/christianlouis/DocuElevate/commit/c306d80755193e4b12222c73ed4da442a2d5e23c)) + + +## v0.116.0 (2026-03-11) + +### Documentation + +- **changelog**: Update changelog [skip ci] + ([`d6fb787`](https://github.com/christianlouis/DocuElevate/commit/d6fb78715ae550f25a925bdc629eb2945f455009)) + +### Testing + +- **views**: Add comprehensive coverage tests for 11 view modules + ([`c8987d9`](https://github.com/christianlouis/DocuElevate/commit/c8987d966bd949634d222a7809ec52c3ed258f81)) + +- **views**: Boost coverage for 11 view modules toward 100% + ([`6fb1df1`](https://github.com/christianlouis/DocuElevate/commit/6fb1df1aade8a8c98c27fe50dbe44045cba1724d)) + + +## Unreleased + +### Testing + +- **views**: Add comprehensive coverage tests for 11 view modules + ([`c8987d9`](https://github.com/christianlouis/DocuElevate/commit/c8987d966bd949634d222a7809ec52c3ed258f81)) + +- **views**: Boost coverage for 11 view modules toward 100% + ([`6fb1df1`](https://github.com/christianlouis/DocuElevate/commit/6fb1df1aade8a8c98c27fe50dbe44045cba1724d)) + + +## v0.115.0 (2026-03-10) + + +## v0.114.1 (2026-03-10) + +### Bug Fixes + +- **db**: Add migration to create shared_links table for databases that skipped 025 + ([`289dcc3`](https://github.com/christianlouis/DocuElevate/commit/289dcc375c111c8d71bd04ef31f184a0e6a3f6f2)) + + ## v0.114.0 (2026-03-09) ### Bug Fixes diff --git a/GIT_SHA b/GIT_SHA index 082dbf0f..0996706b 100644 --- a/GIT_SHA +++ b/GIT_SHA @@ -1 +1 @@ -5fd3f06 +39db5fc diff --git a/RUNTIME_INFO b/RUNTIME_INFO index 0d52e60a..99eacc4d 100644 --- a/RUNTIME_INFO +++ b/RUNTIME_INFO @@ -1,10 +1,10 @@ DocuElevate Build Information ============================== -Version: 0.114.0 -Build Date: 2026-03-09T23:00:57Z -Git Commit: 5fd3f0661b8d86ba6b3f92481675d820aec0d53c -Git Short SHA: 5fd3f06 +Version: 0.118.0 +Build Date: 2026-03-11T23:43:28Z +Git Commit: 39db5fc564af339724d2297748172290208966e0 +Git Short SHA: 39db5fc Git Branch: main -Commit Date: 2026-03-10T00:00:39+01:00 -Build Timestamp: 2026-03-09T23:00:57Z +Commit Date: 2026-03-12T00:43:10+01:00 +Build Timestamp: 2026-03-11T23:43:28Z ============================== diff --git a/VERSION b/VERSION index 18455b77..12f9c914 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.114.0 +0.118.0 diff --git a/app/api/__init__.py b/app/api/__init__.py index b221aa87..ca331229 100644 --- a/app/api/__init__.py +++ b/app/api/__init__.py @@ -8,6 +8,7 @@ from fastapi import APIRouter from app.api.admin_users import router as admin_users_router from app.api.api_tokens import router as api_tokens_router +from app.api.audit_logs import router as audit_logs_router from app.api.azure import router as azure_router from app.api.backup import router as backup_router from app.api.billing import router as billing_router @@ -18,9 +19,11 @@ from app.api.dropbox import router as dropbox_router from app.api.duplicates import router as duplicates_router from app.api.files import router as files_router from app.api.google_drive import router as google_drive_router +from app.api.i18n import router as i18n_router from app.api.imap_accounts import router as imap_accounts_router from app.api.integrations import router as integrations_router from app.api.logs import router as logs_router +from app.api.mobile import router as mobile_router from app.api.notifications import router as notifications_router from app.api.onboarding import router as onboarding_router from app.api.onedrive import router as onedrive_router @@ -83,4 +86,7 @@ router.include_router(imap_accounts_router) router.include_router(integrations_router) router.include_router(notifications_router) router.include_router(scheduled_jobs_router) +router.include_router(audit_logs_router) +router.include_router(i18n_router) +router.include_router(mobile_router) router.include_router(compliance_router) diff --git a/app/api/audit_logs.py b/app/api/audit_logs.py new file mode 100644 index 00000000..a4ed9d10 --- /dev/null +++ b/app/api/audit_logs.py @@ -0,0 +1,115 @@ +""" +Audit log REST API endpoints. + +Provides read-only access to the comprehensive audit log for admin users. +Events are append-only — there are no update or delete endpoints. +""" + +import logging +from datetime import datetime +from typing import Any + +from fastapi import APIRouter, Depends, Query, Request +from sqlalchemy.orm import Session + +from app.auth import require_login +from app.database import get_db +from app.utils.audit_service import count_events, query_events + +logger = logging.getLogger(__name__) + +router = APIRouter() + + +@router.get("/audit-logs") +@require_login +async def list_audit_logs( + request: Request, + db: Session = Depends(get_db), + action: str | None = Query(None, description="Filter by action (exact match)"), + user: str | None = Query(None, description="Filter by username"), + resource_type: str | None = Query(None, description="Filter by resource type"), + severity: str | None = Query(None, description="Filter by severity level"), + since: datetime | None = Query(None, description="Only events at or after this ISO-8601 timestamp"), + until: datetime | None = Query(None, description="Only events at or before this ISO-8601 timestamp"), + limit: int = Query(50, ge=1, le=500, description="Max rows to return"), + offset: int = Query(0, ge=0, description="Rows to skip for pagination"), +) -> dict[str, Any]: + """Return audit log entries with optional filtering and pagination. + + Requires authentication. Returns events in reverse chronological order. + """ + entries = query_events( + db, + action=action, + user=user, + resource_type=resource_type, + severity=severity, + since=since, + until=until, + limit=limit, + offset=offset, + ) + total = count_events( + db, + action=action, + user=user, + resource_type=resource_type, + severity=severity, + since=since, + until=until, + ) + return { + "items": [_serialize(e) for e in entries], + "total": total, + "limit": limit, + "offset": offset, + } + + +@router.get("/audit-logs/actions") +@require_login +async def list_distinct_actions( + request: Request, + db: Session = Depends(get_db), +) -> list[str]: + """Return the distinct action values present in the audit log.""" + from app.models import AuditLog + + rows = db.query(AuditLog.action).distinct().order_by(AuditLog.action).all() + return [r[0] for r in rows] + + +@router.get("/audit-logs/users") +@require_login +async def list_distinct_users( + request: Request, + db: Session = Depends(get_db), +) -> list[str]: + """Return the distinct user values present in the audit log.""" + from app.models import AuditLog + + rows = db.query(AuditLog.user).distinct().order_by(AuditLog.user).all() + return [r[0] for r in rows] + + +# ------------------------------------------------------------------ +# Helpers +# ------------------------------------------------------------------ + + +def _serialize(entry) -> dict[str, Any]: + """Convert an AuditLog row to a JSON-safe dict.""" + import json as _json + + return { + "id": entry.id, + "timestamp": entry.timestamp.isoformat() if entry.timestamp else None, + "user": entry.user, + "action": entry.action, + "resource_type": entry.resource_type, + "resource_id": entry.resource_id, + "ip_address": entry.ip_address, + "details": _json.loads(entry.details) if entry.details else None, + "severity": entry.severity, + } diff --git a/app/api/graphql_api.py b/app/api/graphql_api.py new file mode 100644 index 00000000..5d4530d5 --- /dev/null +++ b/app/api/graphql_api.py @@ -0,0 +1,431 @@ +""" +GraphQL API endpoint for DocuElevate. + +Provides a flexible query interface alongside the existing REST API. +Schema covers: documents, pipelines, settings, and users. + +Endpoint: /graphql +GraphiQL playground: /graphql (via browser) +""" + +from __future__ import annotations + +import logging +from datetime import datetime +from typing import Annotated, Any + +import strawberry +from fastapi import Depends, Request +from sqlalchemy.orm import Session +from strawberry.fastapi import GraphQLRouter + +from app.auth import get_current_user +from app.config import settings +from app.database import get_db +from app.models import ApplicationSettings, FileRecord, Pipeline, PipelineStep, UserProfile + +logger = logging.getLogger(__name__) + + +# --------------------------------------------------------------------------- +# Strawberry types +# --------------------------------------------------------------------------- + + +@strawberry.type +class DocumentType: + """A processed document stored in the system.""" + + id: int + owner_id: str | None + original_filename: str | None + local_filename: str + file_size: int + mime_type: str | None + document_title: str | None + is_duplicate: bool + ocr_quality_score: int | None + pipeline_id: int | None + created_at: datetime | None + + +@strawberry.type +class PipelineStepType: + """A single step within a processing pipeline.""" + + id: int + pipeline_id: int + position: int + step_type: str + label: str | None + enabled: bool + created_at: datetime | None + + +@strawberry.type +class PipelineType: + """A processing pipeline with its ordered steps.""" + + id: int + owner_id: str | None + name: str + description: str | None + is_default: bool + is_active: bool + steps: list[PipelineStepType] + created_at: datetime | None + updated_at: datetime | None + + +@strawberry.type +class SettingType: + """An application configuration setting stored in the database.""" + + id: int + key: str + value: str | None + created_at: datetime | None + updated_at: datetime | None + + +@strawberry.type +class UserType: + """A user profile in the system.""" + + id: int + user_id: str + display_name: str | None + is_blocked: bool + subscription_tier: str | None + onboarding_completed: bool + created_at: datetime | None + + +# --------------------------------------------------------------------------- +# Conversion helpers +# --------------------------------------------------------------------------- + + +def _document_from_record(rec: FileRecord) -> DocumentType: + return DocumentType( + id=rec.id, + owner_id=rec.owner_id, + original_filename=rec.original_filename, + local_filename=rec.local_filename, + file_size=rec.file_size, + mime_type=rec.mime_type, + document_title=rec.document_title, + is_duplicate=rec.is_duplicate, + ocr_quality_score=rec.ocr_quality_score, + pipeline_id=rec.pipeline_id, + created_at=rec.created_at, + ) + + +def _pipeline_step_from_record(step: PipelineStep) -> PipelineStepType: + return PipelineStepType( + id=step.id, + pipeline_id=step.pipeline_id, + position=step.position, + step_type=step.step_type, + label=step.label, + enabled=step.enabled, + created_at=step.created_at, + ) + + +def _pipeline_from_record(pipeline: Pipeline, db: Session) -> PipelineType: + steps = db.query(PipelineStep).filter(PipelineStep.pipeline_id == pipeline.id).order_by(PipelineStep.position).all() + return PipelineType( + id=pipeline.id, + owner_id=pipeline.owner_id, + name=pipeline.name, + description=pipeline.description, + is_default=pipeline.is_default, + is_active=pipeline.is_active, + steps=[_pipeline_step_from_record(s) for s in steps], + created_at=pipeline.created_at, + updated_at=pipeline.updated_at, + ) + + +def _setting_from_record(setting: ApplicationSettings) -> SettingType: + return SettingType( + id=setting.id, + key=setting.key, + value=setting.value, + created_at=setting.created_at, + updated_at=setting.updated_at, + ) + + +def _user_from_profile(profile: UserProfile) -> UserType: + return UserType( + id=profile.id, + user_id=profile.user_id, + display_name=profile.display_name, + is_blocked=profile.is_blocked, + subscription_tier=profile.subscription_tier, + onboarding_completed=profile.onboarding_completed, + created_at=profile.created_at, + ) + + +# --------------------------------------------------------------------------- +# Context helpers +# --------------------------------------------------------------------------- + +# Keys that contain sensitive data and must never be returned via GraphQL +_SENSITIVE_SETTING_KEYS: frozenset[str] = frozenset( + { + "openai_api_key", + "azure_ai_key", + "session_secret", + "database_url", + "redis_url", + "dropbox_app_secret", + "dropbox_refresh_token", + "google_drive_credentials_json", + "onedrive_client_secret", + "onedrive_refresh_token", + "smtp_password", + "nextcloud_password", + "s3_secret_access_key", + "ftp_password", + "sftp_password", + "webdav_password", + "stripe_secret_key", + "stripe_webhook_secret", + "sentry_dsn", + "social_auth_google_client_secret", + "social_auth_microsoft_client_secret", + "social_auth_apple_private_key", + "social_auth_dropbox_app_secret", + } +) + + +def _get_current_user_id(user: dict[str, Any] | None) -> str | None: + """Extract the stable user identifier from the user dict.""" + if not user: + return None + return user.get("preferred_username") or user.get("email") or user.get("id") or None + + +def _get_db_and_user(info: strawberry.types.Info) -> tuple[Session, dict[str, Any] | None]: + """Extract the database session and current user from the Strawberry context.""" + db: Session = info.context["db"] + user: dict[str, Any] | None = info.context.get("user") + return db, user + + +def _require_auth(user: dict[str, Any] | None) -> None: + """Raise an error when authentication is enabled and no valid user is present.""" + if settings.auth_enabled and not user: + raise strawberry.exceptions.StrawberryGraphQLError("Authentication required") + + +def _require_admin(user: dict[str, Any] | None) -> None: + """Raise an error when the current user is not an admin. + + When ``auth_enabled`` is *False* (single-user / development mode) all + callers are implicitly treated as administrators. + """ + if not settings.auth_enabled: + # Single-user mode: no auth, treat caller as admin + return + _require_auth(user) + if not (user and user.get("is_admin")): + raise strawberry.exceptions.StrawberryGraphQLError("Admin access required") + + +# --------------------------------------------------------------------------- +# Query resolvers +# --------------------------------------------------------------------------- + + +@strawberry.type +class Query: + """Root query type for the DocuElevate GraphQL API.""" + + @strawberry.field(description="List documents, optionally filtered by owner.") + def documents( + self, + info: strawberry.types.Info, + owner_id: str | None = None, + limit: int = 20, + offset: int = 0, + ) -> list[DocumentType]: + """Return a paginated list of documents. + + When *auth_enabled* the caller must be authenticated. Non-admin users + receive only their own documents; admins may query any *owner_id*. + """ + db, user = _get_db_and_user(info) + _require_auth(user) + + limit = max(1, min(limit, 100)) + offset = max(0, offset) + + query = db.query(FileRecord) + + if settings.auth_enabled and user: + is_admin = user.get("is_admin", False) + current_user_id = _get_current_user_id(user) + if not is_admin: + # Non-admins can only see their own documents + query = query.filter(FileRecord.owner_id == current_user_id) + elif owner_id: + query = query.filter(FileRecord.owner_id == owner_id) + elif owner_id: + query = query.filter(FileRecord.owner_id == owner_id) + + records = query.order_by(FileRecord.created_at.desc()).offset(offset).limit(limit).all() + return [_document_from_record(r) for r in records] + + @strawberry.field(description="Fetch a single document by ID.") + def document(self, info: strawberry.types.Info, id: int) -> DocumentType | None: + """Return one document by its primary key, or *null* if not found.""" + db, user = _get_db_and_user(info) + _require_auth(user) + + rec = db.query(FileRecord).filter(FileRecord.id == id).first() + if rec is None: + return None + + if settings.auth_enabled and user: + is_admin = user.get("is_admin", False) + current_user_id = _get_current_user_id(user) + if not is_admin and rec.owner_id != current_user_id: + return None + + return _document_from_record(rec) + + @strawberry.field(description="List processing pipelines.") + def pipelines( + self, + info: strawberry.types.Info, + owner_id: str | None = None, + limit: int = 20, + offset: int = 0, + ) -> list[PipelineType]: + """Return a paginated list of pipelines.""" + db, user = _get_db_and_user(info) + _require_auth(user) + + limit = max(1, min(limit, 100)) + offset = max(0, offset) + + query = db.query(Pipeline) + + if settings.auth_enabled and user: + is_admin = user.get("is_admin", False) + current_user_id = _get_current_user_id(user) + if not is_admin: + query = query.filter((Pipeline.owner_id == current_user_id) | (Pipeline.owner_id.is_(None))) + elif owner_id: + query = query.filter(Pipeline.owner_id == owner_id) + elif owner_id: + query = query.filter(Pipeline.owner_id == owner_id) + + rows = query.order_by(Pipeline.id).offset(offset).limit(limit).all() + return [_pipeline_from_record(p, db) for p in rows] + + @strawberry.field(description="Fetch a single pipeline by ID.") + def pipeline(self, info: strawberry.types.Info, id: int) -> PipelineType | None: + """Return one pipeline by its primary key, or *null* if not found.""" + db, user = _get_db_and_user(info) + _require_auth(user) + + row = db.query(Pipeline).filter(Pipeline.id == id).first() + if row is None: + return None + + if settings.auth_enabled and user: + is_admin = user.get("is_admin", False) + current_user_id = _get_current_user_id(user) + if not is_admin and row.owner_id is not None and row.owner_id != current_user_id: + return None + + return _pipeline_from_record(row, db) + + @strawberry.field(description="List non-sensitive application settings (admin only).") + def settings( + self, + info: strawberry.types.Info, + limit: int = 50, + offset: int = 0, + ) -> list[SettingType]: + """Return application settings stored in the database. + + Sensitive keys (API secrets, passwords, etc.) are automatically + excluded. Requires admin privileges when auth is enabled. + """ + db, user = _get_db_and_user(info) + _require_admin(user) + + limit = max(1, min(limit, 200)) + offset = max(0, offset) + + rows = ( + db.query(ApplicationSettings) + .filter(ApplicationSettings.key.notin_(_SENSITIVE_SETTING_KEYS)) + .order_by(ApplicationSettings.key) + .offset(offset) + .limit(limit) + .all() + ) + return [_setting_from_record(r) for r in rows] + + @strawberry.field(description="List user profiles (admin only).") + def users( + self, + info: strawberry.types.Info, + limit: int = 20, + offset: int = 0, + ) -> list[UserType]: + """Return a paginated list of user profiles. Requires admin privileges.""" + db, user = _get_db_and_user(info) + _require_admin(user) + + limit = max(1, min(limit, 100)) + offset = max(0, offset) + + rows = db.query(UserProfile).order_by(UserProfile.user_id).offset(offset).limit(limit).all() + return [_user_from_profile(r) for r in rows] + + @strawberry.field(description="Fetch a user profile by user_id (admin only).") + def user(self, info: strawberry.types.Info, user_id: str) -> UserType | None: + """Return one user profile by *user_id*, or *null* if not found.""" + db, user = _get_db_and_user(info) + _require_admin(user) + + row = db.query(UserProfile).filter(UserProfile.user_id == user_id).first() + return _user_from_profile(row) if row else None + + +# --------------------------------------------------------------------------- +# Schema and router +# --------------------------------------------------------------------------- + +schema = strawberry.Schema(query=Query) + + +async def get_graphql_context( + request: Request, + db: Annotated[Session, Depends(get_db)], +) -> dict[str, Any]: + """Build the per-request context injected into every resolver.""" + try: + user = get_current_user(request) + except Exception: + logger.debug("Could not resolve current user for GraphQL context", exc_info=True) + user = None + return {"request": request, "db": db, "user": user} + + +graphql_router = GraphQLRouter( + schema, + context_getter=get_graphql_context, + graphql_ide="graphiql", +) diff --git a/app/api/i18n.py b/app/api/i18n.py new file mode 100644 index 00000000..d8f5bbb3 --- /dev/null +++ b/app/api/i18n.py @@ -0,0 +1,136 @@ +"""API endpoints for internationalization (i18n). + +Provides endpoints for: +* Listing available languages +* Getting/setting user language preference (persisted in session + cookie + DB) +""" + +from __future__ import annotations + +import logging + +from fastapi import APIRouter, Depends, Request, Response +from pydantic import BaseModel +from sqlalchemy.orm import Session + +from app.database import get_db +from app.models import UserProfile +from app.utils.i18n import ( + DEFAULT_LANGUAGE, + SUPPORTED_LANGUAGE_CODES, + SUPPORTED_LANGUAGES, + detect_language, +) + +logger = logging.getLogger(__name__) + +router = APIRouter(prefix="/i18n", tags=["i18n"]) + + +class LanguageInfo(BaseModel): + """Schema for a supported language.""" + + code: str + name: str + native: str + flag: str + + +class LanguageListResponse(BaseModel): + """Response for the list-languages endpoint.""" + + languages: list[LanguageInfo] + current: str + default: str + + +class SetLanguageRequest(BaseModel): + """Request body for setting the preferred language.""" + + language: str + + +class SetLanguageResponse(BaseModel): + """Response after changing the language.""" + + language: str + message: str + + +@router.get("/languages", response_model=LanguageListResponse) +async def list_languages(request: Request) -> LanguageListResponse: + """Return all supported UI languages and the current active language.""" + current = detect_language(request) + return LanguageListResponse( + languages=[LanguageInfo(**lang) for lang in SUPPORTED_LANGUAGES], + current=current, + default=DEFAULT_LANGUAGE, + ) + + +@router.post("/language", response_model=SetLanguageResponse) +async def set_language( + body: SetLanguageRequest, + request: Request, + response: Response, + db: Session = Depends(get_db), +) -> SetLanguageResponse: + """Set the preferred UI language. + + Persists the choice in: + 1. The server-side session + 2. A ``docuelevate_lang`` cookie (30-day expiry) + 3. The ``UserProfile.preferred_language`` column (if authenticated) + """ + lang = body.language.lower().strip() + if lang not in SUPPORTED_LANGUAGE_CODES: + lang = DEFAULT_LANGUAGE + + # 1. Session + if hasattr(request, "session"): + request.session["preferred_language"] = lang + + # 2. Cookie (30 days) + response.set_cookie( + key="docuelevate_lang", + value=lang, + max_age=30 * 24 * 60 * 60, + httponly=False, + samesite="lax", + ) + + # 3. Database (if user is authenticated) + _persist_language_to_profile(request, db, lang) + + language_name = next( + (entry["native"] for entry in SUPPORTED_LANGUAGES if entry["code"] == lang), + lang, + ) + logger.info("Language preference set to '%s'", lang) + return SetLanguageResponse( + language=lang, + message=f"Language changed to {language_name}", + ) + + +def _persist_language_to_profile(request: Request, db: Session, lang: str) -> None: + """Write language preference to the UserProfile row, if the user is logged in.""" + user_id: str | None = None + if hasattr(request, "session"): + user = request.session.get("user") + if isinstance(user, dict): + user_id = user.get("preferred_username") or user.get("email") or user.get("id") + elif isinstance(user, str): + user_id = user + + if not user_id: + return + + try: + profile = db.query(UserProfile).filter(UserProfile.user_id == user_id).first() + if profile: + profile.preferred_language = lang # type: ignore[attr-defined] + db.commit() + except Exception: + db.rollback() + logger.debug("Could not persist language preference for user_id=%s", user_id) diff --git a/app/api/mobile.py b/app/api/mobile.py new file mode 100644 index 00000000..465872ab --- /dev/null +++ b/app/api/mobile.py @@ -0,0 +1,347 @@ +"""Mobile app API endpoints. + +Provides endpoints specifically designed for the DocuElevate native mobile +app (iOS / Android via React Native / Expo): + +* ``POST /mobile/generate-token`` – exchange an active session for a + long-lived API token that the mobile app stores securely. The token is + auto-named "Mobile App – " and is identical to regular API + tokens (Bearer auth works everywhere). + +* ``POST /mobile/register-device`` – register a push-notification device + token (Expo push token) so the user receives push notifications when + documents finish processing. + +* ``GET /mobile/devices`` – list registered devices for the current user. + +* ``DELETE /mobile/devices/{device_id}`` – deactivate a device. + +* ``GET /mobile/whoami`` – lightweight profile endpoint for the mobile app + to verify authentication state. +""" + +import logging +from datetime import datetime, timezone +from typing import Annotated, Any + +from fastapi import APIRouter, Depends, HTTPException, Request, status +from pydantic import BaseModel, Field +from sqlalchemy.orm import Session + +from app.api.api_tokens import generate_api_token, hash_token +from app.auth import require_login +from app.database import get_db +from app.models import ApiToken, MobileDevice +from app.utils.user_scope import get_current_owner_id + +logger = logging.getLogger(__name__) +router = APIRouter(prefix="/mobile", tags=["mobile"]) + +DbSession = Annotated[Session, Depends(get_db)] + + +# --------------------------------------------------------------------------- +# Auth helper +# --------------------------------------------------------------------------- + + +def _get_owner_id(request: Request) -> str: + """Return the current user's owner ID, raising 401 if unauthenticated.""" + owner_id = get_current_owner_id(request) + if not owner_id: + raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated") + return owner_id + + +CurrentOwner = Annotated[str, Depends(_get_owner_id)] + + +# --------------------------------------------------------------------------- +# Request / Response schemas +# --------------------------------------------------------------------------- + + +class GenerateTokenRequest(BaseModel): + """Request body for auto-generating a mobile app token.""" + + device_name: str = Field( + default="Mobile App", + min_length=1, + max_length=120, + description="Human-readable device name used to label the token.", + ) + + +class GenerateTokenResponse(BaseModel): + """Response containing the one-time-visible API token.""" + + token: str + token_id: int + name: str + created_at: datetime + + +class RegisterDeviceRequest(BaseModel): + """Request body for registering a push-notification device token.""" + + push_token: str = Field( + min_length=1, + max_length=512, + description="Expo push token (ExponentPushToken[…]) obtained from the mobile app.", + ) + device_name: str | None = Field( + default=None, + max_length=255, + description="Optional human-readable device name (e.g. 'John's iPhone').", + ) + platform: str = Field( + default="ios", + description="Device platform: 'ios', 'android', or 'web'.", + ) + + +class DeviceResponse(BaseModel): + """Serialised MobileDevice record.""" + + id: int + device_name: str | None + platform: str + push_token_preview: str + is_active: bool + created_at: datetime + last_seen_at: datetime | None + + +class WhoAmIResponse(BaseModel): + """Lightweight profile response for the mobile app.""" + + owner_id: str + display_name: str | None + email: str | None + avatar_url: str | None + is_admin: bool + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _device_to_response(device: MobileDevice) -> dict[str, Any]: + """Convert a MobileDevice ORM object to a serialisable dict.""" + # Show only first 20 chars of the push token for security. + token_preview = device.push_token[:20] + "…" if len(device.push_token) > 20 else device.push_token + return { + "id": device.id, + "device_name": device.device_name, + "platform": device.platform, + "push_token_preview": token_preview, + "is_active": device.is_active, + "created_at": device.created_at, + "last_seen_at": device.last_seen_at, + } + + +# --------------------------------------------------------------------------- +# Endpoints +# --------------------------------------------------------------------------- + + +@router.post("/generate-token", status_code=status.HTTP_201_CREATED, response_model=GenerateTokenResponse) +@require_login +async def generate_mobile_token( + request: Request, + body: GenerateTokenRequest, + owner_id: CurrentOwner, + db: DbSession, +) -> dict[str, Any]: + """Generate a long-lived API token for the mobile app. + + The mobile app calls this endpoint immediately after SSO login to obtain + a Bearer token it can store in the secure keychain. The returned token + is functionally identical to manually-created API tokens and works with + every authenticated endpoint. + + The token is shown **exactly once** in the response; subsequent requests + show only the prefix for identification. + """ + token_name = f"Mobile App – {body.device_name}" + plaintext = generate_api_token() + token_hash_value = hash_token(plaintext) + prefix = plaintext[:12] + + db_token = ApiToken( + owner_id=owner_id, + name=token_name, + token_hash=token_hash_value, + token_prefix=prefix, + ) + try: + db.add(db_token) + db.commit() + db.refresh(db_token) + except Exception: + db.rollback() + logger.exception("Failed to create mobile API token for owner_id=%s", owner_id) + raise + + logger.info("Mobile API token created: id=%s owner=%s device=%r", db_token.id, owner_id, body.device_name) + + return { + "token": plaintext, + "token_id": db_token.id, + "name": token_name, + "created_at": db_token.created_at, + } + + +@router.post("/register-device", status_code=status.HTTP_201_CREATED, response_model=DeviceResponse) +@require_login +async def register_device( + request: Request, + body: RegisterDeviceRequest, + owner_id: CurrentOwner, + db: DbSession, +) -> dict[str, Any]: + """Register or refresh a push-notification device token. + + If the same ``push_token`` is already registered for this user the + record is reactivated and ``last_seen_at`` is updated rather than + creating a duplicate. + """ + platform = body.platform.lower() + if platform not in {"ios", "android", "web"}: + raise HTTPException( + status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, + detail="platform must be one of: ios, android, web", + ) + + now = datetime.now(timezone.utc) + + # Upsert: reuse existing record if the token is already known. + existing = ( + db.query(MobileDevice) + .filter(MobileDevice.owner_id == owner_id, MobileDevice.push_token == body.push_token) + .first() + ) + if existing: + existing.is_active = True + existing.last_seen_at = now + if body.device_name: + existing.device_name = body.device_name + try: + db.commit() + db.refresh(existing) + except Exception: + db.rollback() + raise + logger.info("Mobile device refreshed: id=%s owner=%s", existing.id, owner_id) + return _device_to_response(existing) + + device = MobileDevice( + owner_id=owner_id, + device_name=body.device_name, + platform=platform, + push_token=body.push_token, + is_active=True, + last_seen_at=now, + ) + try: + db.add(device) + db.commit() + db.refresh(device) + except Exception: + db.rollback() + logger.exception("Failed to register mobile device for owner_id=%s", owner_id) + raise + + logger.info("Mobile device registered: id=%s owner=%s platform=%s", device.id, owner_id, platform) + return _device_to_response(device) + + +@router.get("/devices", response_model=list[DeviceResponse]) +@require_login +async def list_devices( + request: Request, + owner_id: CurrentOwner, + db: DbSession, +) -> list[dict[str, Any]]: + """List all registered push-notification devices for the current user.""" + devices = ( + db.query(MobileDevice).filter(MobileDevice.owner_id == owner_id).order_by(MobileDevice.created_at.desc()).all() + ) + return [_device_to_response(d) for d in devices] + + +@router.delete("/devices/{device_id}", status_code=status.HTTP_204_NO_CONTENT) +@require_login +async def deactivate_device( + request: Request, + device_id: int, + owner_id: CurrentOwner, + db: DbSession, +) -> None: + """Deactivate a push-notification device registration. + + The device record is kept for audit purposes but will no longer receive + push notifications. + """ + device = db.get(MobileDevice, device_id) + if not device or device.owner_id != owner_id: + raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Device not found") + + device.is_active = False + try: + db.commit() + except Exception: + db.rollback() + raise + + logger.info("Mobile device deactivated: id=%s owner=%s", device_id, owner_id) + + +@router.get("/whoami", response_model=WhoAmIResponse) +@require_login +async def whoami( + request: Request, + owner_id: CurrentOwner, + db: DbSession, +) -> dict[str, Any]: + """Return basic profile information for the authenticated user. + + The mobile app calls this after token exchange to populate the user + profile screen and verify that the stored token is still valid. + """ + from app.auth import get_gravatar_url + from app.models import LocalUser, UserProfile + + profile = db.query(UserProfile).filter(UserProfile.user_id == owner_id).first() + local_user = db.query(LocalUser).filter(LocalUser.email == owner_id).first() + + display_name: str | None = None + email: str | None = None + avatar_url: str | None = None + is_admin = False + + if profile: + display_name = profile.display_name + + if local_user: + email = local_user.email + is_admin = bool(local_user.is_admin) + if not display_name and local_user.display_name: + display_name = local_user.display_name + elif "@" in owner_id: + # SSO users commonly have their email as owner_id + email = owner_id + + if email: + avatar_url = get_gravatar_url(email) + + return { + "owner_id": owner_id, + "display_name": display_name, + "email": email, + "avatar_url": avatar_url, + "is_admin": is_admin, + } diff --git a/app/config.py b/app/config.py index 8b669186..a5907dad 100644 --- a/app/config.py +++ b/app/config.py @@ -857,6 +857,49 @@ class Settings(BaseSettings): ), ) + # SIEM / External Audit Log Forwarding + # Forward audit events to external SIEM systems for centralised monitoring. + audit_siem_enabled: bool = Field( + default=False, + description="Enable forwarding of audit events to an external SIEM system.", + ) + audit_siem_transport: str = Field( + default="syslog", + description=( + "Transport used to forward audit events. " + "Options: 'syslog' (RFC 5424 over UDP/TCP), 'http' (JSON POST to a webhook URL, " + "compatible with Splunk HEC, Logstash HTTP input, Grafana Loki, etc.)." + ), + ) + audit_siem_syslog_host: str = Field( + default="localhost", + description="Hostname or IP of the syslog receiver.", + ) + audit_siem_syslog_port: int = Field( + default=514, + description="Port of the syslog receiver.", + ) + audit_siem_syslog_protocol: str = Field( + default="udp", + description="Protocol for syslog transport: 'udp' or 'tcp'.", + ) + audit_siem_http_url: str = Field( + default="", + description=( + "HTTP endpoint URL for SIEM webhook delivery. " + "Supports Splunk HEC (https://splunk:8088/services/collector/event), " + "Logstash HTTP input, Grafana Loki push API, or any JSON-accepting endpoint." + ), + ) + audit_siem_http_token: str = Field( + default="", + description="Bearer / HEC token included in the Authorization header of SIEM HTTP requests.", + ) + audit_siem_http_custom_headers: str = Field( + default="", + description="Comma-separated 'Key:Value' pairs of extra headers for SIEM HTTP requests.", + ) + # UI / Appearance ui_default_color_scheme: str = Field( default="system", diff --git a/app/main.py b/app/main.py index 9e4e0e66..6089699b 100644 --- a/app/main.py +++ b/app/main.py @@ -16,6 +16,7 @@ from starlette.middleware.trustedhost import TrustedHostMiddleware from uvicorn.middleware.proxy_headers import ProxyHeadersMiddleware from app.api import router as api_router +from app.api.graphql_api import graphql_router from app.api.local_auth import router as local_auth_router from app.auth import router as auth_router from app.config import settings @@ -253,6 +254,21 @@ else: # Custom exception handlers that return JSON for API routes and HTML for frontend routes +# These use their own separate templates instance so that patches in tests on individual +# view modules do not affect the error handler rendering. +_error_templates_dir = pathlib.Path(__file__).parents[1] / "frontend" / "templates" +_error_templates = Jinja2Templates(directory=str(_error_templates_dir)) +# Register the i18n translate helper as a global so error templates can use {{ _("key") }}. +# Error pages use the default language (English); request-specific locale is not needed here. +from app.utils.i18n import SUPPORTED_LANGUAGES as _SUPPORTED_LANGUAGES # noqa: E402 +from app.utils.i18n import translate as _translate_fn # noqa: E402 + +_error_templates.env.globals["_"] = lambda key, **kwargs: _translate_fn(key, "en", **kwargs) +_error_templates.env.globals["min"] = min +_error_templates.env.globals["max"] = max +_error_templates.env.globals["supported_languages"] = _SUPPORTED_LANGUAGES + + @app.exception_handler(HTTPException) async def http_exception_handler(request: Request, exc: HTTPException): """ @@ -264,15 +280,15 @@ async def http_exception_handler(request: Request, exc: HTTPException): return JSONResponse(status_code=exc.status_code, content={"detail": exc.detail}) # For frontend routes, return appropriate HTML templates - templates = Jinja2Templates(directory=str(static_dir.parent / "templates")) - # Handle 404 errors with a custom template if exc.status_code == 404: - return templates.TemplateResponse("404.html", {"request": request}, status_code=status.HTTP_404_NOT_FOUND) + return _error_templates.TemplateResponse( + "404.html", {"request": request}, status_code=status.HTTP_404_NOT_FOUND + ) # For other HTTP errors, we could create specific templates or use a generic one # For now, return a simple error page - return templates.TemplateResponse( + return _error_templates.TemplateResponse( "404.html", # Reuse 404 template for other errors, or create a generic error template {"request": request}, status_code=exc.status_code, @@ -293,8 +309,7 @@ async def custom_500_handler(request: Request, exc: Exception): ) # Serve the 500 template for non-API routes - templates = Jinja2Templates(directory=str(static_dir.parent / "templates")) - return templates.TemplateResponse( + return _error_templates.TemplateResponse( "500.html", {"request": request, "exc": exc}, status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, @@ -312,3 +327,4 @@ app.include_router(files_router) # Explicitly include the files router app.include_router(auth_router) app.include_router(local_auth_router) app.include_router(api_router, prefix="/api") +app.include_router(graphql_router, prefix="/graphql") diff --git a/app/models.py b/app/models.py index b820e5d0..e7021e91 100644 --- a/app/models.py +++ b/app/models.py @@ -146,6 +146,27 @@ class SettingsAuditLog(Base): action = Column(String, nullable=False) # "update" or "delete" +class AuditLog(Base): + """Comprehensive audit log for compliance tracking. + + Records all significant actions: login/logout, document CRUD, settings + changes, and administrative operations. Rows are append-only; the API + and service layer never update or delete entries. + """ + + __tablename__ = "audit_logs" + + id = Column(Integer, primary_key=True, index=True) + timestamp = Column(DateTime(timezone=True), server_default=func.now(), nullable=False, index=True) + user = Column(String, nullable=False, index=True) # Username or "anonymous" / "system" + action = Column(String, nullable=False, index=True) # e.g. "login", "document.create", "settings.update" + resource_type = Column(String, nullable=True, index=True) # e.g. "document", "user", "settings" + resource_id = Column(String, nullable=True) # ID of the affected resource + ip_address = Column(String, nullable=True) # Client IP address + details = Column(Text, nullable=True) # JSON-encoded extra context + severity = Column(String(16), nullable=False, server_default="info") # info / warning / error / critical + + class SavedSearch(Base): """User-defined saved search filters for quick access to frequently used filter combinations.""" @@ -256,6 +277,10 @@ class UserProfile(Base): preferred_destination = Column(String(50), nullable=True) stripe_customer_id = Column(String(64), nullable=True) + # UI language preference for i18n (ISO 639-1 code, e.g. "en", "de", "fr") + # NULL means "auto-detect from browser Accept-Language header" + preferred_language = Column(String(10), nullable=True) + created_at = Column(DateTime(timezone=True), server_default=func.now()) updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now()) @@ -835,6 +860,40 @@ class ScheduledJob(Base): updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now()) +class MobileDevice(Base): + """Registered mobile device for push notifications. + + Stores the push token (Expo push token, FCM token, or APNs token) for a + specific user device so that document-processing events can be forwarded + as push notifications to the native mobile app. + """ + + __tablename__ = "mobile_devices" + + id = Column(Integer, primary_key=True, index=True) + + # User that owns this device registration. + owner_id = Column(String, nullable=False, index=True) + + # Human-readable name the user gave this device (e.g. "John's iPhone"). + device_name = Column(String(255), nullable=True) + + # Platform: "ios", "android", or "web". + platform = Column(String(20), nullable=False, default="ios") + + # Expo push token (ExponentPushToken[…]) or raw FCM/APNs token. + push_token = Column(String(512), nullable=False) + + # Whether push notifications are enabled for this device. + is_active = Column(Boolean, nullable=False, default=True) + + # Timestamps. + created_at = Column(DateTime(timezone=True), server_default=func.now()) + last_seen_at = Column(DateTime(timezone=True), nullable=True) + + __table_args__ = (UniqueConstraint("owner_id", "push_token", name="uq_mobile_device_owner_token"),) + + class ComplianceTemplate(Base): """Pre-built compliance configuration templates (GDPR, HIPAA, SOC2). diff --git a/app/utils/audit_service.py b/app/utils/audit_service.py new file mode 100644 index 00000000..73a6af7c --- /dev/null +++ b/app/utils/audit_service.py @@ -0,0 +1,331 @@ +""" +Comprehensive audit-event service for DocuElevate. + +Provides helpers to **record** audit events (append-only database writes) +and to optionally **forward** them to external SIEM systems. + +Supported SIEM transports: +* **Syslog** – RFC 5424 structured-data messages over UDP or TCP. +* **HTTP** – JSON POST payloads compatible with Splunk HEC, Logstash + HTTP input, Grafana Loki push API, and any generic webhook endpoint. +""" + +import json +import logging +import re +import socket +import threading +from datetime import datetime, timezone +from typing import Any + +import httpx +from fastapi import Request +from sqlalchemy.orm import Session + +from app.config import settings +from app.middleware.audit_log import get_client_ip, get_username +from app.models import AuditLog + +logger = logging.getLogger(__name__) + +# --------------------------------------------------------------------------- +# Public helpers +# --------------------------------------------------------------------------- + + +def record_event( + db: Session, + *, + action: str, + user: str = "system", + resource_type: str | None = None, + resource_id: str | None = None, + ip_address: str | None = None, + details: dict[str, Any] | None = None, + severity: str = "info", +) -> AuditLog: + """Persist an audit event and optionally forward it to SIEM. + + Args: + db: Active SQLAlchemy session. + action: Short action identifier (e.g. ``"login"``, ``"document.create"``). + user: Username performing the action. + resource_type: Category of the affected resource (``"document"``, ``"user"`` …). + resource_id: Identifier of the affected resource. + ip_address: Client IP address (``None`` when not applicable). + details: Arbitrary key/value context serialised as JSON. + severity: One of ``info``, ``warning``, ``error``, ``critical``. + + Returns: + The newly created :class:`AuditLog` row. + """ + details_json = json.dumps(details, default=str) if details else None + + entry = AuditLog( + user=user, + action=action, + resource_type=resource_type, + resource_id=str(resource_id) if resource_id is not None else None, + ip_address=ip_address, + details=details_json, + severity=severity, + ) + db.add(entry) + db.commit() + db.refresh(entry) + + # Fire-and-forget SIEM forwarding in a background thread so we never + # block the request path. + if settings.audit_siem_enabled: + payload = _build_siem_payload(entry) + thread = threading.Thread(target=_forward_to_siem, args=(payload,), daemon=True) + thread.start() + + return entry + + +def record_event_from_request( + db: Session, + request: Request, + *, + action: str, + resource_type: str | None = None, + resource_id: str | None = None, + details: dict[str, Any] | None = None, + severity: str = "info", +) -> AuditLog: + """Convenience wrapper that extracts user and IP from a :class:`Request`. + + Args: + db: Active SQLAlchemy session. + request: The current HTTP request. + action: Short action identifier. + resource_type: Category of the affected resource. + resource_id: Identifier of the affected resource. + details: Arbitrary key/value context serialised as JSON. + severity: One of ``info``, ``warning``, ``error``, ``critical``. + + Returns: + The newly created :class:`AuditLog` row. + """ + return record_event( + db, + action=action, + user=get_username(request), + resource_type=resource_type, + resource_id=resource_id, + ip_address=get_client_ip(request), + details=details, + severity=severity, + ) + + +def query_events( + db: Session, + *, + action: str | None = None, + user: str | None = None, + resource_type: str | None = None, + severity: str | None = None, + since: datetime | None = None, + until: datetime | None = None, + limit: int = 200, + offset: int = 0, +) -> list[AuditLog]: + """Query audit log entries with optional filtering. + + Args: + db: Active SQLAlchemy session. + action: Filter by action string (exact match). + user: Filter by username (exact match). + resource_type: Filter by resource type (exact match). + severity: Filter by severity level (exact match). + since: Only events at or after this timestamp. + until: Only events at or before this timestamp. + limit: Maximum number of rows to return. + offset: Number of rows to skip (for pagination). + + Returns: + List of :class:`AuditLog` rows ordered by *timestamp descending*. + """ + q = db.query(AuditLog) + if action: + q = q.filter(AuditLog.action == action) + if user: + q = q.filter(AuditLog.user == user) + if resource_type: + q = q.filter(AuditLog.resource_type == resource_type) + if severity: + q = q.filter(AuditLog.severity == severity) + if since: + q = q.filter(AuditLog.timestamp >= since) + if until: + q = q.filter(AuditLog.timestamp <= until) + return q.order_by(AuditLog.timestamp.desc()).offset(offset).limit(limit).all() + + +def count_events( + db: Session, + *, + action: str | None = None, + user: str | None = None, + resource_type: str | None = None, + severity: str | None = None, + since: datetime | None = None, + until: datetime | None = None, +) -> int: + """Return the total count of events matching the given filters. + + Args: + db: Active SQLAlchemy session. + action: Filter by action string. + user: Filter by username. + resource_type: Filter by resource type. + severity: Filter by severity level. + since: Only events at or after this timestamp. + until: Only events at or before this timestamp. + + Returns: + Integer count. + """ + q = db.query(AuditLog) + if action: + q = q.filter(AuditLog.action == action) + if user: + q = q.filter(AuditLog.user == user) + if resource_type: + q = q.filter(AuditLog.resource_type == resource_type) + if severity: + q = q.filter(AuditLog.severity == severity) + if since: + q = q.filter(AuditLog.timestamp >= since) + if until: + q = q.filter(AuditLog.timestamp <= until) + return q.count() + + +# --------------------------------------------------------------------------- +# SIEM forwarding internals +# --------------------------------------------------------------------------- + +_SYSLOG_FACILITY_LOCAL0 = 16 +_SYSLOG_SEVERITY_MAP = { + "info": 6, + "warning": 4, + "error": 3, + "critical": 2, +} + + +def _build_siem_payload(entry: AuditLog) -> dict[str, Any]: + """Convert an :class:`AuditLog` row into a plain dict for SIEM delivery.""" + ts = entry.timestamp if entry.timestamp else datetime.now(timezone.utc) + return { + "id": entry.id, + "timestamp": ts.isoformat(), + "user": entry.user, + "action": entry.action, + "resource_type": entry.resource_type, + "resource_id": entry.resource_id, + "ip_address": entry.ip_address, + "details": entry.details, + "severity": entry.severity, + "source": "docuelevate", + } + + +def _forward_to_siem(payload: dict[str, Any]) -> None: + """Route a SIEM payload to the configured transport.""" + transport = settings.audit_siem_transport.lower() + try: + if transport == "syslog": + _send_syslog(payload) + elif transport == "http": + _send_http(payload) + else: + logger.warning("Unknown SIEM transport %r; skipping forwarding", transport) + except Exception: + logger.exception("Failed to forward audit event to SIEM (%s)", transport) + + +def _send_syslog(payload: dict[str, Any]) -> None: + """Send a RFC 5424 syslog message to the configured receiver.""" + severity_num = _SYSLOG_SEVERITY_MAP.get(payload.get("severity", "info"), 6) + priority = _SYSLOG_FACILITY_LOCAL0 * 8 + severity_num + ts = payload.get("timestamp", datetime.now(timezone.utc).isoformat()) + hostname = socket.gethostname() + app_name = "docuelevate" + msg_id = payload.get("action", "-") + + # Structured data (SD) element with key event fields. + sd = ( + f'[docuelevate@0 user="{payload.get("user", "-")}" ' + f'action="{payload.get("action", "-")}" ' + f'resource_type="{payload.get("resource_type", "-")}" ' + f'resource_id="{payload.get("resource_id", "-")}" ' + f'ip="{payload.get("ip_address", "-")}"]' + ) + message = json.dumps(payload, default=str) + syslog_msg = f"<{priority}>1 {ts} {hostname} {app_name} - {msg_id} {sd} {message}" + + proto = settings.audit_siem_syslog_protocol.lower() + host = settings.audit_siem_syslog_host + port = settings.audit_siem_syslog_port + + if proto == "tcp": + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: + sock.settimeout(5) + sock.connect((host, port)) + sock.sendall(syslog_msg.encode("utf-8")) + else: + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: + sock.settimeout(5) + sock.sendto(syslog_msg.encode("utf-8"), (host, port)) + + logger.debug("Syslog audit event sent to %s:%s (%s)", host, port, proto) + + +def _send_http(payload: dict[str, Any]) -> None: + """POST a JSON audit event to the configured HTTP endpoint.""" + url = settings.audit_siem_http_url + if not url: + logger.warning("SIEM HTTP URL not configured; skipping HTTP forwarding") + return + + headers: dict[str, str] = {"Content-Type": "application/json"} + token = settings.audit_siem_http_token + if token: + headers["Authorization"] = f"Bearer {token}" + + # Parse custom headers (comma-separated "Key:Value" pairs). + # Reject headers that could override security-critical ones already set, + # and validate that header names contain only RFC 7230 token characters. + _PROTECTED_HEADERS = {"authorization", "content-type", "host"} + _VALID_HEADER_NAME = re.compile(r"^[A-Za-z0-9!#$%&'*+\-.^_`|~]+$") + raw_custom = settings.audit_siem_http_custom_headers + if raw_custom: + for raw_pair in raw_custom.split(","): + pair = raw_pair.strip() + if ":" in pair: + k, _, v = pair.partition(":") + name = k.strip() + if not name or not _VALID_HEADER_NAME.match(name): + logger.warning("Skipping invalid SIEM custom header name: %r", name) + continue + if name.lower() in _PROTECTED_HEADERS: + logger.warning("Skipping protected SIEM custom header: %r", name) + continue + headers[name] = v.strip() + + # Wrap in Splunk HEC-style envelope when URL contains ``/services/collector``. + body: dict[str, Any] + if "/services/collector" in url: + body = {"event": payload, "sourcetype": "docuelevate:audit", "source": "docuelevate"} + else: + body = payload + + with httpx.Client(timeout=10) as client: + resp = client.post(url, json=body, headers=headers) + resp.raise_for_status() + + logger.debug("HTTP audit event forwarded to %s (status %s)", url, resp.status_code) diff --git a/app/utils/db_migrate.py b/app/utils/db_migrate.py index f95d97f0..3424009d 100644 --- a/app/utils/db_migrate.py +++ b/app/utils/db_migrate.py @@ -32,8 +32,10 @@ _TABLE_ORDER = [ "processing_logs", "application_settings", "settings_audit_log", + "audit_logs", "saved_searches", "webhook_configs", + "shared_links", ] diff --git a/app/utils/i18n.py b/app/utils/i18n.py new file mode 100644 index 00000000..ab94a908 --- /dev/null +++ b/app/utils/i18n.py @@ -0,0 +1,542 @@ +"""Internationalization (i18n) and localization (l10n) utilities. + +Provides a JSON-based translation system for the DocuElevate UI with: + +* **31 supported languages** covering all major European languages plus ZH +* Browser ``Accept-Language`` detection with cookie & user-profile persistence +* AI-powered fallback translation via the configured LLM provider +* Locale-aware date, number, and file-size formatting helpers +* Jinja2 integration via a ``_()`` global function + +Language resolution order: + 1. User profile ``preferred_language`` (persisted in DB) + 2. ``docuelevate_lang`` cookie + 3. ``Accept-Language`` HTTP header + 4. Default (``en``) +""" + +from __future__ import annotations + +import json +import logging +from datetime import date, datetime +from functools import lru_cache +from pathlib import Path +from typing import Any + +from starlette.requests import Request + +logger = logging.getLogger(__name__) + +# --------------------------------------------------------------------------- +# Supported languages (ordered by priority) +# --------------------------------------------------------------------------- + +SUPPORTED_LANGUAGES: list[dict[str, str]] = [ + # --- Tier 1: Primary European languages --- + {"code": "en", "name": "English", "native": "English", "flag": "🇬🇧"}, + {"code": "de", "name": "German", "native": "Deutsch", "flag": "🇩🇪"}, + {"code": "fr", "name": "French", "native": "Français", "flag": "🇫🇷"}, + {"code": "es", "name": "Spanish", "native": "Español", "flag": "🇪🇸"}, + {"code": "it", "name": "Italian", "native": "Italiano", "flag": "🇮🇹"}, + {"code": "pt", "name": "Portuguese", "native": "Português", "flag": "🇵🇹"}, + # --- Tier 2: Western & Northern European --- + {"code": "nl", "name": "Dutch", "native": "Nederlands", "flag": "🇳🇱"}, + {"code": "nb", "name": "Norwegian", "native": "Norsk", "flag": "🇳🇴"}, + {"code": "da", "name": "Danish", "native": "Dansk", "flag": "🇩🇰"}, + {"code": "sv", "name": "Swedish", "native": "Svenska", "flag": "🇸🇪"}, + {"code": "fi", "name": "Finnish", "native": "Suomi", "flag": "🇫🇮"}, + {"code": "is", "name": "Icelandic", "native": "Íslenska", "flag": "🇮🇸"}, + {"code": "ga", "name": "Irish", "native": "Gaeilge", "flag": "🇮🇪"}, + {"code": "lb", "name": "Luxembourgish", "native": "Lëtzebuergesch", "flag": "🇱🇺"}, + {"code": "ca", "name": "Catalan", "native": "Català", "flag": "🏴"}, + # --- Tier 3: Central & Eastern European --- + {"code": "pl", "name": "Polish", "native": "Polski", "flag": "🇵🇱"}, + {"code": "cs", "name": "Czech", "native": "Čeština", "flag": "🇨🇿"}, + {"code": "sk", "name": "Slovak", "native": "Slovenčina", "flag": "🇸🇰"}, + {"code": "hu", "name": "Hungarian", "native": "Magyar", "flag": "🇭🇺"}, + {"code": "sl", "name": "Slovenian", "native": "Slovenščina", "flag": "🇸🇮"}, + {"code": "hr", "name": "Croatian", "native": "Hrvatski", "flag": "🇭🇷"}, + {"code": "ro", "name": "Romanian", "native": "Română", "flag": "🇷🇴"}, + {"code": "bg", "name": "Bulgarian", "native": "Български", "flag": "🇧🇬"}, + {"code": "el", "name": "Greek", "native": "Ελληνικά", "flag": "🇬🇷"}, + {"code": "et", "name": "Estonian", "native": "Eesti", "flag": "🇪🇪"}, + {"code": "lv", "name": "Latvian", "native": "Latviešu", "flag": "🇱🇻"}, + {"code": "lt", "name": "Lithuanian", "native": "Lietuvių", "flag": "🇱🇹"}, + # --- Tier 4: Non-EU European & Other --- + {"code": "tr", "name": "Turkish", "native": "Türkçe", "flag": "🇹🇷"}, + {"code": "uk", "name": "Ukrainian", "native": "Українська", "flag": "🇺🇦"}, + {"code": "ru", "name": "Russian", "native": "Русский", "flag": "🇷🇺"}, + {"code": "zh", "name": "Chinese", "native": "中文", "flag": "🇨🇳"}, +] + +SUPPORTED_LANGUAGE_CODES: set[str] = {lang["code"] for lang in SUPPORTED_LANGUAGES} +DEFAULT_LANGUAGE = "en" + +# --------------------------------------------------------------------------- +# Translation file loading +# --------------------------------------------------------------------------- + +_TRANSLATIONS_DIR = Path(__file__).resolve().parent.parent.parent / "frontend" / "translations" +_translation_cache: dict[str, dict[str, str]] = {} + + +def _load_translations(locale: str) -> dict[str, str]: + """Load the translation JSON file for *locale*, with caching.""" + if locale in _translation_cache: + return _translation_cache[locale] + + filepath = _TRANSLATIONS_DIR / f"{locale}.json" + if not filepath.is_file(): + logger.warning("Translation file not found for locale '%s'", locale) + _translation_cache[locale] = {} + return {} + + try: + data: dict[str, str] = json.loads(filepath.read_text(encoding="utf-8")) + _translation_cache[locale] = data + return data + except (json.JSONDecodeError, OSError): + logger.exception("Failed to load translations for '%s'", locale) + _translation_cache[locale] = {} + return {} + + +def reload_translations() -> None: + """Clear the translation cache so files are re-read on next access.""" + _translation_cache.clear() + + +# --------------------------------------------------------------------------- +# Core translation function +# --------------------------------------------------------------------------- + + +def translate(key: str, locale: str | None = None, **kwargs: Any) -> str: + """Return the translated string for *key* in *locale*. + + Falls back through: + 1. Requested *locale* + 2. English (``en``) + 3. The raw key itself (to keep the UI functional) + + Positional placeholders ``{0}``, ``{1}`` or named placeholders + ``{name}`` in the translated string are interpolated via *kwargs*. + """ + locale = locale if locale and locale in SUPPORTED_LANGUAGE_CODES else DEFAULT_LANGUAGE + + translations = _load_translations(locale) + value = translations.get(key) + + # Fallback to English + if value is None and locale != DEFAULT_LANGUAGE: + en_translations = _load_translations(DEFAULT_LANGUAGE) + value = en_translations.get(key) + + # Fallback to key itself + if value is None: + value = key + + if kwargs: + try: + value = value.format(**kwargs) + except (KeyError, IndexError): + pass # Return unformatted string rather than crash + + return value + + +# --------------------------------------------------------------------------- +# AI fallback translation (best-effort, non-blocking) +# --------------------------------------------------------------------------- + +_ai_translation_cache: dict[tuple[str, str], str] = {} + + +def translate_with_ai_fallback(text: str, target_locale: str) -> str: + """Translate *text* using the configured AI provider as a fallback. + + Returns the original *text* unchanged when: + * The target locale is English (source language) + * The AI provider is unavailable or returns an error + * The translation has already been cached + + Results are cached in-memory for the lifetime of the process. + """ + if target_locale == DEFAULT_LANGUAGE or target_locale not in SUPPORTED_LANGUAGE_CODES: + return text + + cache_key = (text, target_locale) + if cache_key in _ai_translation_cache: + return _ai_translation_cache[cache_key] + + target_name = next( + (lang["name"] for lang in SUPPORTED_LANGUAGES if lang["code"] == target_locale), + target_locale, + ) + + try: + from litellm import completion # type: ignore[import-untyped] + + from app.config import settings + + model = getattr(settings, "ai_model", None) or getattr(settings, "openai_model", "gpt-4o-mini") + response = completion( + model=model, + messages=[ + { + "role": "system", + "content": ( + f"You are a professional translator. Translate the following UI text " + f"from English to {target_name}. Return ONLY the translated text, " + f"nothing else. Keep any HTML tags, placeholders like {{name}}, " + f"and special characters intact." + ), + }, + {"role": "user", "content": text}, + ], + max_tokens=256, + temperature=0.1, + ) + translated = response.choices[0].message.content.strip() + _ai_translation_cache[cache_key] = translated + return translated + except Exception: + logger.debug("AI fallback translation failed for '%s' → %s", text[:50], target_locale) + return text + + +# --------------------------------------------------------------------------- +# Language detection +# --------------------------------------------------------------------------- + + +def detect_language(request: Request) -> str: + """Determine the preferred UI language from the request context. + + Resolution order: + 1. ``preferred_language`` stored in the user session + 2. ``docuelevate_lang`` cookie + 3. ``Accept-Language`` HTTP header (best match) + 4. Default → ``en`` + """ + # 1. User session preference + if hasattr(request, "session"): + session_lang = request.session.get("preferred_language") + if isinstance(session_lang, str) and session_lang in SUPPORTED_LANGUAGE_CODES: + return session_lang + + # 2. Cookie + if hasattr(request, "cookies"): + cookie_lang = request.cookies.get("docuelevate_lang") + if isinstance(cookie_lang, str) and cookie_lang in SUPPORTED_LANGUAGE_CODES: + return cookie_lang + + # 3. Accept-Language header + accept = "" + if hasattr(request, "headers"): + accept = request.headers.get("accept-language", "") + lang = _parse_accept_language(accept) + if lang: + return lang + + return DEFAULT_LANGUAGE + + +def _parse_accept_language(header: str) -> str | None: + """Extract the best matching language from an ``Accept-Language`` header. + + Parses quality values and returns the highest-priority match among + :data:`SUPPORTED_LANGUAGE_CODES`, or ``None`` if nothing matches. + """ + if not header: + return None + + entries: list[tuple[float, str]] = [] + for raw_part in header.split(","): + part = raw_part.strip() + if not part: + continue + if ";q=" in part: + lang_tag, _, q_str = part.partition(";q=") + try: + quality = float(q_str.strip()) + except ValueError: + quality = 0.0 + else: + lang_tag = part + quality = 1.0 + entries.append((quality, lang_tag.strip().lower())) + + # Sort by quality descending + entries.sort(key=lambda e: e[0], reverse=True) + + for _quality, tag in entries: + # Try exact match first (e.g., "de", "zh") + code = tag.split("-")[0] + if code in SUPPORTED_LANGUAGE_CODES: + return code + + return None + + +# --------------------------------------------------------------------------- +# Localization helpers (l10n) +# --------------------------------------------------------------------------- + +# Locale-specific formatting rules for date/number display +_LOCALE_FORMATS: dict[str, dict[str, Any]] = { + "en": { + "date": "%B %d, %Y", + "date_short": "%m/%d/%Y", + "datetime": "%B %d, %Y %I:%M %p", + "thousands_sep": ",", + "decimal_sep": ".", + }, + "de": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "fr": { + "date": "%d %B %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": "\u202f", + "decimal_sep": ",", + }, + "es": { + "date": "%d de %B de %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d de %B de %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "it": { + "date": "%d %B %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "pt": { + "date": "%d de %B de %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d de %B de %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "nl": { + "date": "%d %B %Y", + "date_short": "%d-%m-%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "nb": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "da": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "sv": { + "date": "%d %B %Y", + "date_short": "%Y-%m-%d", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "fi": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "is": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "ga": { + "date": "%d %B %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": ",", + "decimal_sep": ".", + }, + "lb": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "ca": { + "date": "%d de %B de %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d de %B de %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "pl": { + "date": "%d %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "cs": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "sk": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "hu": { + "date": "%Y. %B %d.", + "date_short": "%Y.%m.%d.", + "datetime": "%Y. %B %d. %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "sl": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "hr": { + "date": "%d. %B %Y.", + "date_short": "%d.%m.%Y.", + "datetime": "%d. %B %Y. %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "ro": { + "date": "%d %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "bg": { + "date": "%d %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "el": { + "date": "%d %B %Y", + "date_short": "%d/%m/%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "et": { + "date": "%d. %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d. %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "lv": { + "date": "%Y. gada %d. %B", + "date_short": "%d.%m.%Y.", + "datetime": "%Y. gada %d. %B %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "lt": { + "date": "%Y m. %B %d d.", + "date_short": "%Y-%m-%d", + "datetime": "%Y m. %B %d d. %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "tr": { + "date": "%d %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": ".", + "decimal_sep": ",", + }, + "uk": { + "date": "%d %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, + "zh": { + "date": "%Y年%m月%d日", + "date_short": "%Y/%m/%d", + "datetime": "%Y年%m月%d日 %H:%M", + "thousands_sep": ",", + "decimal_sep": ".", + }, + "ru": { + "date": "%d %B %Y", + "date_short": "%d.%m.%Y", + "datetime": "%d %B %Y %H:%M", + "thousands_sep": "\u00a0", + "decimal_sep": ",", + }, +} + + +def format_date(value: date | datetime | None, locale: str = DEFAULT_LANGUAGE, short: bool = False) -> str: + """Format a date/datetime value according to the locale conventions.""" + if value is None: + return "" + fmt_key = "date_short" if short else "date" + fmt = _LOCALE_FORMATS.get(locale, _LOCALE_FORMATS[DEFAULT_LANGUAGE])[fmt_key] + return value.strftime(fmt) + + +def format_datetime(value: datetime | None, locale: str = DEFAULT_LANGUAGE) -> str: + """Format a datetime value according to the locale conventions.""" + if value is None: + return "" + fmt = _LOCALE_FORMATS.get(locale, _LOCALE_FORMATS[DEFAULT_LANGUAGE])["datetime"] + return value.strftime(fmt) + + +def format_number(value: int | float, locale: str = DEFAULT_LANGUAGE) -> str: + """Format a number with locale-appropriate thousand separators.""" + lf = _LOCALE_FORMATS.get(locale, _LOCALE_FORMATS[DEFAULT_LANGUAGE]) + if isinstance(value, float): + int_part, _, dec_part = f"{value:,.2f}".partition(".") + formatted_int = int_part.replace(",", lf["thousands_sep"]) + return f"{formatted_int}{lf['decimal_sep']}{dec_part}" + return f"{value:,}".replace(",", lf["thousands_sep"]) + + +@lru_cache(maxsize=32) +def get_language_info(code: str) -> dict[str, str] | None: + """Return the metadata dict for a supported language code, or ``None``.""" + for lang in SUPPORTED_LANGUAGES: + if lang["code"] == code: + return lang + return None diff --git a/app/utils/push_notification.py b/app/utils/push_notification.py new file mode 100644 index 00000000..f5f0e521 --- /dev/null +++ b/app/utils/push_notification.py @@ -0,0 +1,130 @@ +"""Push notification sender for the DocuElevate mobile app. + +Uses the **Expo Push Notification** service to deliver notifications to both +iOS (via APNs) and Android (via FCM) without requiring server-side APNs keys +or FCM credentials. The mobile app obtains an ``ExponentPushToken[…]`` at +startup and registers it with the backend via the mobile API. + +Reference: https://docs.expo.dev/push-notifications/sending-notifications/ +""" + +import logging +from typing import Any + +import httpx + +from app.database import SessionLocal +from app.models import MobileDevice + +logger = logging.getLogger(__name__) + +EXPO_PUSH_URL = "https://exp.host/--/api/v2/push/send" + +# Maximum tokens per batch request (Expo limit). +_EXPO_BATCH_LIMIT = 100 + + +def send_expo_push_notification( + tokens: list[str], + title: str, + body: str, + data: dict[str, Any] | None = None, + sound: str = "default", + badge: int | None = None, +) -> list[dict[str, Any]]: + """Send a push notification to one or more Expo push tokens. + + Args: + tokens: List of Expo push tokens (``ExponentPushToken[…]``). + title: Notification title shown in the system tray. + body: Notification body text. + data: Optional JSON-serialisable dict attached to the notification + (available in the app via ``notification.request.content.data``). + sound: Notification sound. Use ``"default"`` or ``None`` for silent. + badge: iOS badge count. Pass ``0`` to clear. + + Returns: + List of Expo push receipt dicts (one per token). + """ + if not tokens: + return [] + + results: list[dict[str, Any]] = [] + + # Send in batches to stay within Expo's per-request limit. + for i in range(0, len(tokens), _EXPO_BATCH_LIMIT): + batch = tokens[i : i + _EXPO_BATCH_LIMIT] + messages = [] + for token in batch: + msg: dict[str, Any] = { + "to": token, + "title": title, + "body": body, + "sound": sound, + } + if data: + msg["data"] = data + if badge is not None: + msg["badge"] = badge + messages.append(msg) + + try: + resp = httpx.post( + EXPO_PUSH_URL, + json=messages, + headers={ + "Accept": "application/json", + "Accept-Encoding": "gzip, deflate", + "Content-Type": "application/json", + }, + timeout=15, + ) + resp.raise_for_status() + payload = resp.json() + batch_results = payload.get("data", []) + results.extend(batch_results) + logger.debug("Expo push batch sent: %d tokens, %d results", len(batch), len(batch_results)) + except httpx.HTTPStatusError as exc: + logger.error("Expo push HTTP error: %s – %s", exc.response.status_code, exc.response.text) + except Exception: + logger.exception("Expo push notification failed for batch starting at index %d", i) + + return results + + +def send_push_to_owner( + owner_id: str, + title: str, + body: str, + data: dict[str, Any] | None = None, +) -> None: + """Look up all active push tokens for *owner_id* and send them a notification. + + This function is safe to call from Celery task workers. Database errors + and push failures are logged but never raised so that the caller task is + not retried due to a notification failure. + """ + db = SessionLocal() + try: + devices = ( + db.query(MobileDevice) + .filter( + MobileDevice.owner_id == owner_id, + MobileDevice.is_active.is_(True), + MobileDevice.push_token.isnot(None), + ) + .all() + ) + tokens = [d.push_token for d in devices if d.push_token] + except Exception: + logger.exception("Failed to query mobile devices for owner_id=%s", owner_id) + return + finally: + db.close() + + if not tokens: + logger.debug("No active push tokens for owner_id=%s", owner_id) + return + + logger.info("Sending push notification to %d device(s) for owner_id=%s", len(tokens), owner_id) + send_expo_push_notification(tokens=tokens, title=title, body=body, data=data) diff --git a/app/utils/settings_service.py b/app/utils/settings_service.py index 48d06012..d2f805e2 100644 --- a/app/utils/settings_service.py +++ b/app/utils/settings_service.py @@ -2077,6 +2077,78 @@ SETTING_METADATA = { "required": False, "restart_required": True, }, + "audit_siem_enabled": { + "category": "Security", + "description": "Enable forwarding of audit events to an external SIEM system (Syslog, Splunk, Logstash, etc.).", + "type": "boolean", + "sensitive": False, + "required": False, + "restart_required": False, + }, + "audit_siem_transport": { + "category": "Security", + "description": ( + "Transport used to forward audit events. 'syslog' sends RFC 5424 messages over UDP/TCP. " + "'http' sends JSON POST payloads to a webhook URL (Splunk HEC, Logstash, Grafana Loki, etc.)." + ), + "type": "string", + "sensitive": False, + "required": False, + "restart_required": False, + "options": ["syslog", "http"], + }, + "audit_siem_syslog_host": { + "category": "Security", + "description": "Hostname or IP of the syslog receiver.", + "type": "string", + "sensitive": False, + "required": False, + "restart_required": False, + }, + "audit_siem_syslog_port": { + "category": "Security", + "description": "Port of the syslog receiver. Default: 514.", + "type": "integer", + "sensitive": False, + "required": False, + "restart_required": False, + }, + "audit_siem_syslog_protocol": { + "category": "Security", + "description": "Protocol for syslog transport: 'udp' or 'tcp'. Default: udp.", + "type": "string", + "sensitive": False, + "required": False, + "restart_required": False, + "options": ["udp", "tcp"], + }, + "audit_siem_http_url": { + "category": "Security", + "description": ( + "HTTP endpoint URL for SIEM webhook delivery. Supports Splunk HEC, " + "Logstash HTTP input, Grafana Loki push API, or any JSON-accepting endpoint." + ), + "type": "string", + "sensitive": False, + "required": False, + "restart_required": False, + }, + "audit_siem_http_token": { + "category": "Security", + "description": "Bearer / HEC token included in the Authorization header of SIEM HTTP requests.", + "type": "string", + "sensitive": True, + "required": False, + "restart_required": False, + }, + "audit_siem_http_custom_headers": { + "category": "Security", + "description": "Comma-separated 'Key:Value' pairs of extra headers for SIEM HTTP requests.", + "type": "string", + "sensitive": False, + "required": False, + "restart_required": False, + }, # Rate Limiting "rate_limiting_enabled": { "category": "Security", diff --git a/app/utils/user_notification.py b/app/utils/user_notification.py index f6a277cd..791b2219 100644 --- a/app/utils/user_notification.py +++ b/app/utils/user_notification.py @@ -210,6 +210,19 @@ def dispatch_user_notification( finally: db.close() + # 3. Send push notifications to registered mobile devices + try: + from app.utils.push_notification import send_push_to_owner + + send_push_to_owner( + owner_id=owner_id, + title=title, + body=message, + data={"event_type": event_type, "file_id": file_id}, + ) + except Exception: + logger.exception("Error sending push notification for owner_id=%s event=%s", owner_id, event_type) + def notify_user_document_processed(owner_id: str, filename: str, file_id: int | None = None) -> None: """Notify a user that their document was successfully processed.""" diff --git a/app/views/__init__.py b/app/views/__init__.py index 0dd0b3ee..b2814ff5 100644 --- a/app/views/__init__.py +++ b/app/views/__init__.py @@ -6,6 +6,7 @@ from fastapi import APIRouter from app.views.admin_users import router as admin_users_router from app.views.api_tokens import router as api_tokens_router +from app.views.audit_logs import router as audit_logs_router from app.views.backup import router as backup_router from app.views.compliance import router as compliance_router from app.views.db_wizard import router as db_wizard_router @@ -61,5 +62,6 @@ router.include_router(imap_accounts_router) # Per-user IMAP ingestion accounts router.include_router(integrations_router) # Unified integrations dashboard router.include_router(notifications_router) # User notification dashboard router.include_router(scheduled_jobs_router) # Admin scheduled batch jobs +router.include_router(audit_logs_router) # Comprehensive audit log viewer router.include_router(help_router) # Built-in help / How-To docs router.include_router(compliance_router) # Compliance templates dashboard diff --git a/app/views/audit_logs.py b/app/views/audit_logs.py new file mode 100644 index 00000000..a787fece --- /dev/null +++ b/app/views/audit_logs.py @@ -0,0 +1,46 @@ +""" +Audit log viewer UI — admin-only page with filtering and SIEM status. +""" + +import logging + +from fastapi import Depends, HTTPException, Request, status +from sqlalchemy.orm import Session + +from app.views.base import APIRouter, get_db, require_login, settings, templates +from app.views.settings import require_admin_access + +logger = logging.getLogger(__name__) +router = APIRouter() + + +@router.get("/admin/audit-logs") +@require_login +@require_admin_access +async def audit_logs_page(request: Request, db: Session = Depends(get_db)): + """Comprehensive audit log viewer with filtering controls. + + Displays a chronological log of all significant actions: logins, + document operations, settings changes, and admin actions. The + actual data is fetched client-side via the ``/api/audit-logs`` JSON + endpoint so that filters, pagination, and live refresh work without + full-page reloads. + """ + try: + siem_enabled = settings.audit_siem_enabled + siem_transport = settings.audit_siem_transport if siem_enabled else None + return templates.TemplateResponse( + "audit_logs.html", + { + "request": request, + "app_version": settings.version, + "siem_enabled": siem_enabled, + "siem_transport": siem_transport, + }, + ) + except Exception as e: + logger.error("Error loading audit logs page: %s", e) + raise HTTPException( + status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, + detail="Failed to load audit logs page", + ) diff --git a/app/views/base.py b/app/views/base.py index 010d4f21..21c8b5dd 100644 --- a/app/views/base.py +++ b/app/views/base.py @@ -12,6 +12,14 @@ from sqlalchemy.orm import Session # noqa: F401 from app.auth import require_login # noqa: F401 from app.config import settings from app.database import get_db # noqa: F401 +from app.utils.i18n import ( + SUPPORTED_LANGUAGES, + detect_language, + format_date, + format_datetime, + format_number, + translate, +) # Set up Jinja2 templates templates_dir = Path(__file__).parent.parent.parent / "frontend" / "templates" @@ -21,6 +29,19 @@ templates = Jinja2Templates(directory=str(templates_dir)) templates.env.globals["min"] = min templates.env.globals["max"] = max +# --------------------------------------------------------------------------- +# i18n Jinja2 integration +# --------------------------------------------------------------------------- +# The _() function is available in every template to translate UI strings. +# Usage: {{ _("nav.dashboard") }} or {{ _("upload.max_size", size="10 MB") }} +# The locale is automatically resolved from the request context. +# A default English implementation is registered as a global so error handlers +# that don't go through _inject_global_context still have the function available. +# --------------------------------------------------------------------------- + +templates.env.globals["supported_languages"] = SUPPORTED_LANGUAGES +templates.env.globals["_"] = lambda key, **kwargs: translate(key, "en", **kwargs) + # Customize Jinja2Templates to include app_version in all templates original_template_response = templates.TemplateResponse @@ -48,8 +69,31 @@ def _inject_global_context(ctx: dict) -> None: session_user = req.session.get("user") # When auth is disabled every visitor is effectively "logged in" ctx.setdefault("is_logged_in", not getattr(settings, "auth_enabled", True) or session_user is not None) + + # --- i18n: detect language and register template helpers --- + current_locale = detect_language(req) + ctx.setdefault("current_locale", current_locale) + + def _translate(key: str, **kwargs: object) -> str: + return translate(key, current_locale, **kwargs) + + def _format_date(value: object, short: bool = False) -> str: + return format_date(value, current_locale, short=short) # type: ignore[arg-type] + + def _format_datetime(value: object) -> str: + return format_datetime(value, current_locale) # type: ignore[arg-type] + + def _format_number(value: object) -> str: + return format_number(value, current_locale) # type: ignore[arg-type] + + ctx.setdefault("_", _translate) + ctx.setdefault("format_date_l10n", _format_date) + ctx.setdefault("format_datetime_l10n", _format_datetime) + ctx.setdefault("format_number_l10n", _format_number) else: ctx.setdefault("is_logged_in", not getattr(settings, "auth_enabled", True)) + ctx.setdefault("current_locale", "en") + ctx.setdefault("_", lambda key, **kw: translate(key, "en", **kw)) def template_response_with_version(*args, **kwargs): diff --git a/app/views/plans.py b/app/views/plans.py index 6f01c40a..5eb059d6 100644 --- a/app/views/plans.py +++ b/app/views/plans.py @@ -3,12 +3,11 @@ from fastapi import Request from fastapi.responses import HTMLResponse from fastapi.routing import APIRouter -from fastapi.templating import Jinja2Templates from app.auth import require_login +from app.views.base import templates router = APIRouter() -templates = Jinja2Templates(directory="frontend/templates") @router.get("/admin/plans", response_class=HTMLResponse) diff --git a/docs/API.md b/docs/API.md index d5a0bf40..9515df4c 100644 --- a/docs/API.md +++ b/docs/API.md @@ -2063,3 +2063,191 @@ print(response.json()) ## Further Assistance For additional help with the API, please contact our support team or refer to the [Development Guide](../CONTRIBUTING.md). + +## Mobile App API + +The mobile API provides endpoints used by the native iOS and Android app. All endpoints require authentication (Bearer token or active session cookie). + +For full mobile app documentation see [MobileApp.md](./MobileApp.md). + +### POST /api/mobile/generate-token + +Exchange an active web session for a long-lived API token scoped to the mobile app. + +**Request:** +```json +{ "device_name": "John's iPhone" } +``` + +**Response (201 Created):** +```json +{ + "token": "de_AbCdEfGhIjKl...", + "token_id": 42, + "name": "Mobile App – John's iPhone", + "created_at": "2026-03-10T09:30:00Z" +} +``` + +> The `token` is shown **once only**. + +### POST /api/mobile/register-device + +Register an Expo push token to receive push notifications. + +**Request:** +```json +{ + "push_token": "ExponentPushToken[xxxxxx]", + "device_name": "John's iPhone", + "platform": "ios" +} +``` + +**Response (201 Created):** Device record with `id`, `platform`, `is_active`, `created_at`. + +### GET /api/mobile/devices + +List all registered push-notification devices for the current user. + +**Response (200 OK):** Array of device records. + +### DELETE /api/mobile/devices/{device_id} + +Deactivate a push-notification device. The device will no longer receive push notifications. + +**Response (204 No Content)** + +### GET /api/mobile/whoami + +Return basic profile information for the authenticated user. + +**Response (200 OK):** +```json +{ + "owner_id": "john@example.com", + "display_name": "John Doe", + "email": "john@example.com", + "avatar_url": "https://www.gravatar.com/avatar/...", + "is_admin": false +} +``` + +--- + +## GraphQL API + +DocuElevate exposes a GraphQL API at `/graphql` alongside the REST API. It +supports flexible queries with field selection, making it ideal for dashboards +and integrations that only need a subset of the available data. + +### Endpoint + +| Method | URL | Description | +|--------|-----|-------------| +| `POST` | `/graphql` | Execute a GraphQL query or mutation | +| `GET` | `/graphql` | Open the GraphiQL interactive playground | + +### Authentication + +The GraphQL endpoint honours the same authentication rules as the REST API: + +- **`AUTH_ENABLED=False`** (default, single-user mode): all queries are + allowed without credentials. +- **`AUTH_ENABLED=True`** (multi-user mode): a valid session cookie **or** + an `Authorization: Bearer ` API token is required. Admin-only + queries (settings, users) additionally require the `is_admin` flag. + +### Available Queries + +| Field | Returns | Notes | +|-------|---------|-------| +| `documents(ownerId, limit, offset)` | `[DocumentType]` | Paginated list of documents | +| `document(id)` | `DocumentType` | Single document by primary key | +| `pipelines(ownerId, limit, offset)` | `[PipelineType]` | Paginated list of pipelines with steps | +| `pipeline(id)` | `PipelineType` | Single pipeline by primary key | +| `settings(limit, offset)` | `[SettingType]` | Non-sensitive app settings (**admin only**) | +| `users(limit, offset)` | `[UserType]` | User profiles (**admin only**) | +| `user(userId)` | `UserType` | Single user profile (**admin only**) | + +> **Note:** Sensitive configuration keys (API secrets, passwords, tokens) are +> automatically excluded from the `settings` query regardless of the caller's +> privilege level. + +### GraphiQL Playground + +Navigate to `http:///graphql` in a browser to open the +interactive GraphiQL IDE, which provides schema documentation, auto-complete, +and the ability to run queries directly. + +### Example Queries + +**List recent documents:** +```graphql +{ + documents(limit: 5) { + id + originalFilename + mimeType + fileSize + documentTitle + createdAt + } +} +``` + +**Fetch a pipeline with its steps:** +```graphql +{ + pipeline(id: 1) { + id + name + description + isDefault + isActive + steps { + position + stepType + label + enabled + } + } +} +``` + +**List application settings (admin only):** +```graphql +{ + settings { + key + value + updatedAt + } +} +``` + +**List user profiles (admin only):** +```graphql +{ + users(limit: 10) { + userId + displayName + subscriptionTier + isBlocked + } +} +``` + +**Using variables:** +```graphql +query GetDocument($id: Int!) { + document(id: $id) { + id + originalFilename + documentTitle + isDuplicate + ocrQualityScore + } +} +``` +Variables: `{ "id": 42 }` diff --git a/docs/ConfigurationGuide.md b/docs/ConfigurationGuide.md index fb2bdc8a..07f6ef10 100644 --- a/docs/ConfigurationGuide.md +++ b/docs/ConfigurationGuide.md @@ -399,6 +399,61 @@ default overage buffer applied across all plans. DocuElevate supports HTTP security headers to improve browser-side security. **These headers are disabled by default** since most deployments use a reverse proxy (Traefik, Nginx, etc.) that already adds them. Enable only if deploying directly without a reverse proxy. See [Deployment Guide - Security Headers](DeploymentGuide.md#security-headers) for detailed configuration examples. +### Audit Logging + +DocuElevate provides comprehensive audit logging that records significant actions (logins, document CRUD, settings changes) to an append-only database table. Every entry captures the timestamp, user, action, resource, client IP, and optional JSON details. + +| **Variable** | **Description** | **Default** | +|--------------------------------|---------------------------------------------------------------------------------------------------|-------------| +| `AUDIT_LOGGING_ENABLED` | Enable the HTTP request audit-logging middleware. | `true` | +| `AUDIT_LOG_INCLUDE_CLIENT_IP` | Include the client IP address in audit log entries. Disable for GDPR-sensitive deployments. | `true` | + +#### SIEM Integration + +Audit events can be forwarded in real time to external SIEM systems for centralised monitoring, alerting, and long-term retention. Two transports are supported: + +* **Syslog** – RFC 5424 structured-data messages over UDP or TCP. Works with rsyslog, syslog-ng, Graylog, Datadog, etc. +* **HTTP** – JSON POST payloads compatible with Splunk HEC, Logstash HTTP input, Grafana Loki push API, and any generic webhook. + +| **Variable** | **Description** | **Default** | +|-------------------------------------|---------------------------------------------------------------------------------------------------|---------------| +| `AUDIT_SIEM_ENABLED` | Enable forwarding of audit events to an external SIEM system. | `false` | +| `AUDIT_SIEM_TRANSPORT` | Transport: `syslog` or `http`. | `syslog` | +| `AUDIT_SIEM_SYSLOG_HOST` | Hostname or IP of the syslog receiver. | `localhost` | +| `AUDIT_SIEM_SYSLOG_PORT` | Port of the syslog receiver. | `514` | +| `AUDIT_SIEM_SYSLOG_PROTOCOL` | Protocol for syslog: `udp` or `tcp`. | `udp` | +| `AUDIT_SIEM_HTTP_URL` | HTTP endpoint URL for SIEM delivery (e.g. Splunk HEC, Logstash, Loki). | *(empty)* | +| `AUDIT_SIEM_HTTP_TOKEN` | Bearer / HEC token for the SIEM HTTP endpoint. | *(empty)* | +| `AUDIT_SIEM_HTTP_CUSTOM_HEADERS` | Comma-separated `Key:Value` extra headers for SIEM HTTP requests. | *(empty)* | + +**Example – Syslog to rsyslog:** + +```bash +AUDIT_SIEM_ENABLED=true +AUDIT_SIEM_TRANSPORT=syslog +AUDIT_SIEM_SYSLOG_HOST=syslog.internal.example.com +AUDIT_SIEM_SYSLOG_PORT=514 +AUDIT_SIEM_SYSLOG_PROTOCOL=udp +``` + +**Example – Splunk HEC:** + +```bash +AUDIT_SIEM_ENABLED=true +AUDIT_SIEM_TRANSPORT=http +AUDIT_SIEM_HTTP_URL=https://splunk.example.com:8088/services/collector/event +AUDIT_SIEM_HTTP_TOKEN=your-hec-token +``` + +**Example – Logstash HTTP input:** + +```bash +AUDIT_SIEM_ENABLED=true +AUDIT_SIEM_TRANSPORT=http +AUDIT_SIEM_HTTP_URL=https://logstash.example.com:8080 +AUDIT_SIEM_HTTP_TOKEN= +``` + ### Rate Limiting DocuElevate implements rate limiting to protect against DoS attacks and API abuse. **Rate limiting is enabled by default** and uses Redis for distributed rate limiting across multiple workers. diff --git a/docs/InternationalizationGuide.md b/docs/InternationalizationGuide.md new file mode 100644 index 00000000..2b192602 --- /dev/null +++ b/docs/InternationalizationGuide.md @@ -0,0 +1,231 @@ +# Internationalization (i18n) & Localization (l10n) Guide + +DocuElevate supports **10 languages** for its web UI, with automatic browser +language detection, user-preference persistence, and an AI-powered fallback +translator for strings that haven't been manually translated yet. + +## Supported Languages + +| Code | Language | Native Name | Priority | +|------|------------|-------------|----------| +| `en` | English | English | Tier 1 | +| `de` | German | Deutsch | Tier 1 | +| `fr` | French | Français | Tier 1 | +| `es` | Spanish | Español | Tier 1 | +| `it` | Italian | Italiano | Tier 1 | +| `pt` | Portuguese | Português | Tier 1 | +| `nl` | Dutch | Nederlands | Tier 2 | +| `pl` | Polish | Polski | Tier 2 | +| `zh` | Chinese | 中文 | Tier 2 | +| `ru` | Russian | Русский | Tier 2 | + +> **Tier 1** languages (European priority) have complete, manually-reviewed +> translations. **Tier 2** languages have complete translations but may +> receive less frequent updates. + +## How Language Is Detected + +DocuElevate resolves the display language in the following priority order: + +1. **User profile preference** — stored in the database (`UserProfile.preferred_language`) + and loaded into the session on login +2. **Cookie** — `docuelevate_lang` cookie (30-day expiry, set when user selects a language) +3. **Browser `Accept-Language` header** — the highest-priority match among supported languages +4. **Default** — English (`en`) + +## Selecting Your Language + +### Via the Navigation Bar + +Click the 🌐 **globe icon** in the top navigation bar. A dropdown menu shows all +available languages with their native names and flag emoji. The current language +is highlighted with a blue checkmark. + +### Via the API + +```bash +# Set language to German +curl -X POST http://localhost:8000/api/i18n/language \ + -H "Content-Type: application/json" \ + -d '{"language": "de"}' + +# List all available languages +curl http://localhost:8000/api/i18n/languages +``` + +### Via Cookie (Programmatic) + +Set the `docuelevate_lang` cookie to any supported language code: + +```javascript +document.cookie = "docuelevate_lang=fr; max-age=2592000; path=/"; +location.reload(); +``` + +## For Developers + +### Translation File Structure + +Translations are stored as flat JSON files in `frontend/translations/`: + +``` +frontend/translations/ +├── en.json # English (base / reference) +├── de.json # German +├── fr.json # French +├── es.json # Spanish +├── it.json # Italian +├── pt.json # Portuguese +├── nl.json # Dutch +├── pl.json # Polish +├── zh.json # Chinese +└── ru.json # Russian +``` + +Each file is a flat key-value dictionary with dot-notation namespacing: + +```json +{ + "nav.dashboard": "Dashboard", + "nav.upload": "Upload", + "upload.max_size": "Maximum file size: {size}", + "footer.copyright": "DocuElevate {year}" +} +``` + +### Using Translations in Templates + +The `_()` function is available globally in all Jinja2 templates: + +```jinja2 +{# Simple translation #} +

{{ _("dashboard.title") }}

+ +{# Translation with placeholders #} +

{{ _("upload.max_size", size="50 MB") }}

+ +{# Translation in attributes #} + +``` + +### Using Translations in Python + +```python +from app.utils.i18n import translate + +# Basic translation +text = translate("nav.dashboard", "de") # → "Übersicht" + +# With placeholders +text = translate("footer.copyright", "fr", year="2025") # → "DocuElevate 2025" +``` + +### Localization Helpers + +Format dates, times, and numbers according to locale conventions: + +```jinja2 +{# In templates — locale is automatically detected #} +{{ format_date_l10n(document.created_at) }} +{{ format_number_l10n(file_count) }} +``` + +```python +# In Python +from app.utils.i18n import format_date, format_number + +format_date(date(2025, 3, 15), "de") # → "15. March 2025" +format_date(date(2025, 3, 15), "de", short=True) # → "15.03.2025" +format_number(1234567, "de") # → "1.234.567" +format_number(1234.56, "en") # → "1,234.56" +``` + +### Adding a New Translation Key + +1. Add the key and English text to `frontend/translations/en.json` +2. Add translations for all other languages in their respective files +3. Use `{{ _("your.new.key") }}` in templates + +### AI Fallback Translation + +When a translation key exists in English but not in the target language, +DocuElevate can use the configured AI provider (OpenAI, Anthropic, etc.) +to translate the string on-the-fly: + +```python +from app.utils.i18n import translate_with_ai_fallback + +# Falls back to AI if no manual translation exists +translated = translate_with_ai_fallback("Welcome to our platform", "de") +``` + +The AI fallback: +- Uses the `AI_MODEL` or `OPENAI_MODEL` setting +- Caches results in memory for the process lifetime +- Returns the original English text if the AI call fails +- Is designed for graceful degradation — the UI never breaks + +### Adding a New Language + +1. Create a new JSON file in `frontend/translations/` (e.g., `ja.json`) +2. Copy the structure from `en.json` and translate all values +3. Add the language to `SUPPORTED_LANGUAGES` in `app/utils/i18n.py`: + ```python + {"code": "ja", "name": "Japanese", "native": "日本語", "flag": "🇯🇵"}, + ``` +4. Add locale formatting rules to `_LOCALE_FORMATS` in the same file +5. Create a database migration if needed (the `preferred_language` column + already accepts any string up to 10 characters) + +### Database Migration + +Migration `027_add_user_language_preference` adds a `preferred_language` +column to the `user_profiles` table. This column stores the user's chosen +UI language as an ISO 639-1 code (e.g., `"de"`, `"fr"`). A `NULL` value +means "auto-detect from browser settings." + +### API Reference + +#### `GET /api/i18n/languages` + +Returns all supported languages and the current active language. + +**Response:** +```json +{ + "languages": [ + {"code": "en", "name": "English", "native": "English", "flag": "🇬🇧"}, + {"code": "de", "name": "German", "native": "Deutsch", "flag": "🇩🇪"} + ], + "current": "en", + "default": "en" +} +``` + +#### `POST /api/i18n/language` + +Set the preferred UI language. Persists in session, cookie, and database. + +**Request:** +```json +{"language": "de"} +``` + +**Response:** +```json +{ + "language": "de", + "message": "Language changed to Deutsch" +} +``` + +## Configuration + +No additional configuration is required. The i18n system works out of the box +with the default English language and automatically detects browser preferences. + +| Setting | Default | Description | +|---------|---------|-------------| +| Browser `Accept-Language` | Auto-detected | Used when no explicit preference is set | +| `docuelevate_lang` cookie | Not set | Set when user selects a language via the UI | +| `UserProfile.preferred_language` | `NULL` | Stored in DB for authenticated users | diff --git a/docs/MobileApp.md b/docs/MobileApp.md new file mode 100644 index 00000000..92e92383 --- /dev/null +++ b/docs/MobileApp.md @@ -0,0 +1,252 @@ +# Mobile App + +DocuElevate includes a native mobile application for iOS and Android built with **React Native** and **Expo**. The app allows users to capture documents with the device camera, pick files from the device storage, and receive push notifications when documents finish processing. + +## Features + +| Feature | iOS | Android | +|---------|-----|---------| +| SSO login (OAuth2) | ✅ | ✅ | +| Local / basic auth login | ✅ | ✅ | +| Auto-generated API token | ✅ | ✅ | +| Camera capture → upload | ✅ | ✅ | +| File picker upload | ✅ | ✅ | +| Share Sheet / Share Intent | ✅ | ✅ | +| Push notifications | ✅ | ✅ | +| Document list | ✅ | ✅ | +| Dark mode | ✅ | ✅ | + +## Getting Started (Development) + +### Prerequisites + +- Node.js 18 or later +- [Expo CLI](https://docs.expo.dev/get-started/installation/): `npm install -g @expo/cli` +- [Expo Go](https://expo.dev/client) app on your iOS or Android device (for development) +- A running DocuElevate server reachable from your device + +### Run in development mode + +```bash +cd mobile +npm install +npx expo start +``` + +Scan the QR code with **Expo Go** on your device. On iOS you can also use the Camera app. + +## Building for Production + +DocuElevate uses **Expo Application Services (EAS)** to produce App Store / Play Store binaries. + +```bash +# Install EAS CLI globally +npm install -g eas-cli + +# Authenticate with Expo +eas login + +# Build for iOS (requires Apple Developer account) +eas build --platform ios + +# Build for Android +eas build --platform android +``` + +See the [EAS Build documentation](https://docs.expo.dev/build/introduction/) for full setup instructions. + +## Authentication + +### SSO Login Flow + +The mobile app uses the server's existing OAuth2/SSO setup: + +1. User enters the DocuElevate server URL on the login screen. +2. The app opens `/login?mobile=1&redirect_uri=docuelevate://callback` in the **system browser** (Safari / Chrome). +3. The user authenticates via SSO or local credentials. +4. The server redirects back to `docuelevate://callback`. +5. The app calls `POST /api/mobile/generate-token` to exchange the session for a **long-lived API token**. +6. The token is stored securely in the device's keychain (`expo-secure-store`). + +### Auto-generated Mobile Token + +When the mobile app completes login it automatically creates a named API token (`"Mobile App – "`) via `POST /api/mobile/generate-token`. This token: + +- Works identically to tokens created manually in the web UI. +- Is shown in the **API Tokens** page (`/api-tokens`) and can be revoked there. +- Is stored in the device's secure keychain, never in plain storage. + +## Push Notifications + +Push notifications are delivered via the **Expo Push Notification** service, which routes through Apple Push Notification service (APNs) for iOS and Firebase Cloud Messaging (FCM) for Android. + +**No server-side APNs/FCM credentials are required** – Expo's servers handle the provider integration. + +### How it works + +1. After login, the app requests notification permission from the operating system. +2. If granted, the app obtains an **Expo Push Token** (`ExponentPushToken[…]`). +3. The token is registered with the backend via `POST /api/mobile/register-device`. +4. When a document finishes processing, the server sends a push notification to all registered devices for that user. + +### Managing registered devices + +Users can see and remove their registered devices from the **Profile** tab in the app, or via the API: + +```bash +# List registered devices +curl -H "Authorization: Bearer " https://your-server/api/mobile/devices + +# Remove a device +curl -X DELETE -H "Authorization: Bearer " https://your-server/api/mobile/devices/ +``` + +## Uploading Documents + +### Camera Capture + +1. Open the **Upload** tab. +2. Tap **Camera**. +3. Point the camera at the document and take a photo. +4. The image is immediately uploaded and queued for processing. + +### File Picker + +1. Open the **Upload** tab. +2. Tap **File Picker**. +3. Browse to and select one or more files (PDF, DOCX, images, etc.). +4. Files are uploaded and queued for processing. + +### Share Sheet (iOS) / Share Intent (Android) + +The app registers itself as a share target so any file can be sent directly to DocuElevate from another app: + +1. Open a file in Files, Mail, Safari, or any other app. +2. Tap the **Share** button (iOS) or **Share** (Android). +3. Find and tap **DocuElevate** in the share sheet. +4. The file is immediately uploaded. + +> **Note:** The app must be installed on the device for it to appear in the share sheet. + +## Mobile API Endpoints + +The backend exposes a dedicated `/api/mobile/` namespace: + +| Method | Endpoint | Auth | Description | +|--------|----------|------|-------------| +| `POST` | `/api/mobile/generate-token` | Session | Exchange SSO session for API token | +| `POST` | `/api/mobile/register-device` | Bearer | Register Expo push token | +| `GET` | `/api/mobile/devices` | Bearer | List registered devices | +| `DELETE` | `/api/mobile/devices/{id}` | Bearer | Deactivate a device | +| `GET` | `/api/mobile/whoami` | Bearer | Get current user profile | + +All other API endpoints (file upload, file listing, etc.) work with Bearer token authentication. + +### POST /api/mobile/generate-token + +Exchanges an active web session (cookie) for a permanent API token suitable for use in the mobile app. + +**Request:** +```json +{ "device_name": "John's iPhone" } +``` + +**Response (201):** +```json +{ + "token": "de_AbCdEfGhIjKl...", + "token_id": 42, + "name": "Mobile App – John's iPhone", + "created_at": "2026-03-10T09:30:00Z" +} +``` + +> ⚠️ The `token` value is returned **once only**. Store it in the device's secure keychain immediately. + +### POST /api/mobile/register-device + +Registers an Expo push token for the authenticated user. + +**Request:** +```json +{ + "push_token": "ExponentPushToken[xxxxxxxxxxxxxxxxxxxxxx]", + "device_name": "John's iPhone", + "platform": "ios" +} +``` + +Supported platforms: `ios`, `android`, `web`. + +Re-registering the same token is safe (idempotent). + +### GET /api/mobile/whoami + +Returns the current user's profile. + +**Response (200):** +```json +{ + "owner_id": "john@example.com", + "display_name": "John Doe", + "email": "john@example.com", + "avatar_url": "https://www.gravatar.com/avatar/...", + "is_admin": false +} +``` + +## Configuration + +No server-side configuration is required to enable the mobile app. The Expo push notification routing does not need FCM or APNs credentials on the server. + +If you wish to use **direct FCM/APNs** without Expo's relay, replace the `send_expo_push_notification` function in `app/utils/push_notification.py` with your own implementation. + +## Project Structure (mobile/) + +``` +mobile/ +├── App.tsx # Root component +├── app.json # Expo/EAS configuration +├── eas.json # EAS Build profiles +├── package.json +├── tsconfig.json +└── src/ + ├── context/ + │ └── AuthContext.tsx # Auth state + SSO login flow + ├── hooks/ + │ └── usePushNotifications.ts # Push token registration + ├── screens/ + │ ├── LoginScreen.tsx # Server URL + SSO button + │ ├── UploadScreen.tsx # Camera capture + file picker + │ ├── FilesScreen.tsx # Processed document list + │ └── ProfileScreen.tsx # User profile + sign out + └── services/ + └── api.ts # DocuElevate REST API client +``` + +## Troubleshooting + +### "Authentication was cancelled or failed" + +- Ensure the server URL is correct (including `https://`). +- Verify the server is reachable from your device's network. +- Confirm that `AUTH_ENABLED=True` on the server. + +### Push notifications not arriving + +1. Check that the app has notification permission (Settings → DocuElevate → Notifications). +2. Verify the device is registered: `GET /api/mobile/devices`. +3. Ensure the server can reach `https://exp.host` (outbound HTTPS on port 443). +4. On Android, add `google-services.json` to the `mobile/` directory if you are building your own binary. + +### "Connection refused" or timeout + +- Verify that the DocuElevate server is running and accessible. +- Ensure the server's `EXTERNAL_HOSTNAME` or reverse proxy is configured correctly. +- Check that the server accepts CORS requests from `docuelevate://`. + +## Related Documentation + +- [API Documentation](./API.md) +- [Configuration Guide](./ConfigurationGuide.md) +- [Deployment Guide](./DeploymentGuide.md) diff --git a/frontend/templates/audit_logs.html b/frontend/templates/audit_logs.html new file mode 100644 index 00000000..92073816 --- /dev/null +++ b/frontend/templates/audit_logs.html @@ -0,0 +1,222 @@ +{% extends "base.html" %} + +{% block title %}Audit Logs - DocuElevate{% endblock %} + +{% block content %} +
+ + +
+
+

+ Audit Logs +

+

+ Comprehensive, append-only record of all significant actions. +

+
+
+ {% if siem_enabled %} + + SIEM: {{ siem_transport|upper }} + + {% else %} + + SIEM: Off + + {% endif %} + +
+
+ + +
+
+
+ + +
+
+ + +
+
+ + +
+
+ + +
+
+
+ + +
+ + Loading… +
+ + +
+
+ + + + + + + + + + + + + + + + + + +
TimestampSeverityUserActionResourceIPDetails
+ +

No audit events recorded yet.

+

Significant actions (logins, document operations, settings changes) will appear here.

+
+
+
+ + + + +
+ + +{% endblock %} diff --git a/frontend/templates/base.html b/frontend/templates/base.html index 6d078574..eef10e65 100644 --- a/frontend/templates/base.html +++ b/frontend/templates/base.html @@ -1,5 +1,5 @@ - + {% block title %}DocuElevate{% endblock %} @@ -37,10 +37,10 @@ data-multi-user="{{ 'true' if multi_user_enabled else 'false' }}" data-allow-signup="{{ 'true' if allow_signup else 'false' }}"> - + -