fix: merge main branch and renumber migration 037→040

Resolve all merge conflicts between our automation feature branch and
current main (v0.163.0, 920 commits ahead).

Conflicts resolved:
- app/api/__init__.py: add automation_router alongside main's new routers
  (classification_rules, qr_auth, sessions, system_reset)
- app/config.py: add main's new settings (dropbox_use_global_credentials,
  factory_reset_on_startup, enable_factory_reset)
- app/models.py: add main's new models (ClassificationRuleModel, UserSession,
  QRLoginChallenge, SharePoint integration type)
- app/utils/settings_service.py: merge automation_hooks_enabled with main's
  new metadata entries
- docs/API.md: merge automation API docs with main's classification rules docs
- docs/ConfigurationGuide.md: add factory reset settings
- tests/conftest.py: import both AutomationHook and new main models

Migration renumbered:
- 037_add_automation_hooks → 040_add_automation_hooks
- down_revision: 039_add_classification_rules (was 036_add_document_translation_fields)
- Chain: 036 → 037 → 038 → 039 → 040 (automation hooks)

For all non-automation files with conflicts, main's version was taken since
our branch did not modify those files (conflicts were from a stale prior merge).

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>
Agent-Logs-Url: https://github.com/christianlouis/DocuElevate/sessions/cb62f012-3b69-4415-835e-3857ce3e9f45

docs/ProductionReadiness.md

@@ -34,7 +34,7 @@ Use this checklist to track readiness before going live.
 - [ ] **Redis** — Running and accessible only from internal network
 - [ ] **Meilisearch** — Running and accessible only from internal network
 - [ ] **Worker replicas** — At least 2 workers configured for redundancy
-- [ ] **Monitoring** — `/api/health` polled by uptime checker
+- [ ] **Monitoring** — `/api/diagnostic/health` polled by uptime checker
 - [ ] **Backups** — Automated backup of database, workdir, and Meilisearch data
 - [ ] **Log retention** — Logs shipped to a persistent store or aggregator
 - [ ] **Secrets management** — API keys not committed to source control
@@ -285,22 +285,24 @@ For SSO/OIDC (Authentik, Keycloak, Auth0, etc.) see the [Authentication Setup Gu
 
 ### Docker Compose
 
-Use the `deploy.replicas` setting (requires Docker Swarm mode) or simply run multiple workers:
-
-```yaml
-worker:
-  deploy:
-    replicas: 3
-```
-
-Or scale after deployment:
+Scale workers independently:
 
 ```bash
-docker-compose up -d --scale worker=3
+docker compose up -d --scale worker=3
 ```
 
 Each worker processes tasks from the Celery queue independently.  Ensure the shared `workdir` volume is accessible from all worker containers.
 
+> **Important:** The `beat` service (Celery Beat scheduler) must always run as exactly **one** instance.  It is defined as a dedicated service in `docker-compose.yaml` with a fixed `container_name`.  Do not scale it.
+
+### Scaling the API
+
+API pods are fully stateless (sessions use encrypted cookies, not server-side state) and can be scaled behind a load balancer:
+
+```bash
+docker compose up -d --scale api=3
+```
+
 ### Kubernetes (Helm)
 
 ```yaml
@@ -339,11 +341,32 @@ celery -A app.celery_worker worker -Q default,celery --concurrency=2
 
 ### Health Check Endpoint
 
-DocuElevate exposes `/api/health` for readiness probing.  Configure your uptime monitor to poll this endpoint:
+DocuElevate exposes three health-related endpoints:
+
+| Endpoint | Auth | Purpose |
+|----------|------|---------|
+| `GET /api/diagnostic/healthz/live` | None | Lightweight liveness probe — returns 200 if the process is running |
+| `GET /api/diagnostic/healthz/ready` | None | Readiness probe — checks database and Redis (503 when DB is down) |
+| `GET /api/diagnostic/health` | Required | Full status for monitoring dashboards (Grafana, Uptime Kuma) |
+
+For **Kubernetes probes**, use the unauthenticated endpoints:
+
+```yaml
+livenessProbe:
+  httpGet:
+    path: /api/diagnostic/healthz/live
+    port: 8000
+readinessProbe:
+  httpGet:
+    path: /api/diagnostic/healthz/ready
+    port: 8000
+```
+
+For **uptime monitors** (Uptime Kuma, Grafana, etc.), use the authenticated endpoint:
 
 ```bash
-curl http://docuelevate.example.com/api/health
-# Expected: {"status": "ok", ...}
+curl http://docuelevate.example.com/api/diagnostic/health
+# Expected: {"status": "healthy", ...}
 ```
 
 Set `UPTIME_KUMA_URL` to your Uptime Kuma push URL for heartbeat monitoring:
@@ -502,4 +525,4 @@ For a dedicated Kubernetes deployment guide, including architecture diagrams, PV
 
 - **Image Pull Policy**: Use `IfNotPresent` in production with pinned image tags (not `latest`) for reproducible deployments.
 
-- **Liveness & Readiness Probes**: Already configured in the Helm chart via `/api/health`.  Verify they are tuned to your startup time.
+- **Liveness & Readiness Probes**: Already configured in the Helm chart via unauthenticated endpoints (`/api/diagnostic/healthz/live` and `/api/diagnostic/healthz/ready`).  Verify they are tuned to your startup time.