docs: update scaling, health probe, and beat scheduler documentation

- Update DeploymentGuide.md with scaling instructions and beat service info
- Update KubernetesDeployment.md with unauthenticated probe paths and beat note
- Update ProductionReadiness.md with new health endpoints table and beat guidance
- Update API.md with new healthz/live and healthz/ready endpoint docs

Co-authored-by: christianlouis <361235+christianlouis@users.noreply.github.com>

docs/API.md

@@ -1571,6 +1571,47 @@ Lightweight endpoint returning the total number of queued + in-progress items. D
 
 ## Diagnostic
 
+### GET /api/diagnostic/healthz/live
+
+Lightweight liveness probe for Kubernetes.  Returns **200 OK** as long as the process is running.  This endpoint does **not** check external dependencies and is intentionally cheap.
+
+**Authentication:** None (designed for kubelet probes)
+
+**Response (200 OK):**
+```json
+{
+  "status": "ok"
+}
+```
+
+### GET /api/diagnostic/healthz/ready
+
+Readiness probe for Kubernetes.  Verifies that the application can serve traffic by checking database and Redis connectivity.
+
+**Authentication:** None (designed for kubelet probes)
+
+**Response (200 OK) – ready to serve traffic:**
+```json
+{
+  "status": "ready",
+  "checks": {
+    "database": {"status": "ok"},
+    "redis":    {"status": "ok"}
+  }
+}
+```
+
+**Response (503 Service Unavailable) – database unreachable:**
+```json
+{
+  "status": "not_ready",
+  "checks": {
+    "database": {"status": "error", "detail": "..."},
+    "redis":    {"status": "ok"}
+  }
+}
+```
+
 ### GET /api/diagnostic/health
 
 System health endpoint designed for monitoring tools such as Grafana, Uptime Kuma, Prometheus blackbox exporter, or any HTTP-based health checker.

docs/DeploymentGuide.md

@@ -349,16 +349,18 @@ workdir:
 
 ## Scaling
 
+DocuElevate is designed for horizontal scaling.  Both API and worker pods are stateless and can be scaled independently.
+
 ### Docker Compose
 
-Add more worker containers:
+Scale workers (task processing) and API pods (request handling) independently:
 
-```yaml
+```bash
-worker:
+docker compose up -d --scale worker=3 --scale api=2
-  deploy:
-    replicas: 3
 ```
 
+> **Note:** The `beat` service (Celery Beat scheduler) must always run as exactly **one** instance.  Do not scale it.  It publishes periodic tasks to the Redis broker; workers pick them up.
+
 ### Kubernetes / Helm
 
 Enable HPA:
@@ -377,13 +379,15 @@ worker:
     maxReplicas: 10
 ```
 
+The Helm chart deploys a separate **beat** pod (always 1 replica, `Recreate` strategy) so that scheduled tasks are never duplicated when workers scale.
+
 ---
 
 ## Monitoring
 
 - **Docker Compose**: `docker-compose logs -f`, `docker stats`
 - **Kubernetes**: `kubectl logs -l app.kubernetes.io/component=api -f`
-- **Prometheus / Grafana**: Scrape the `/api/health` endpoint for readiness; add custom metrics as needed.
+- **Prometheus / Grafana**: Scrape the `/api/diagnostic/healthz/ready` endpoint for readiness; add custom metrics as needed.
 - **Uptime Kuma**: Set `UPTIME_KUMA_URL` to your push URL for heartbeat monitoring.
 
 ---

docs/KubernetesDeployment.md

@@ -373,6 +373,8 @@ worker:
   replicaCount: 4
 ```
 
+> **Beat scheduler:** The Helm chart deploys a dedicated `beat` pod (always exactly 1 replica with `Recreate` strategy) that publishes periodic tasks to the Redis broker.  Workers consume these tasks — scaling workers does **not** duplicate scheduled jobs.
+
 ### Horizontal Pod Autoscaler
 
 ```yaml
@@ -433,24 +435,30 @@ externalRedis:
 
 ### Kubernetes Probes
 
-The Helm chart configures liveness and readiness probes on the API pods via `/api/health`.  Default settings:
+The Helm chart configures **unauthenticated** liveness and readiness probes on the API pods so kubelet can reach them without credentials.  Default settings:
 
 ```yaml
 api:
   livenessProbe:
     httpGet:
-      path: /api/health
+      path: /api/diagnostic/healthz/live
       port: 8000
     initialDelaySeconds: 30
-    periodSeconds: 30
+    periodSeconds: 20
   readinessProbe:
     httpGet:
-      path: /api/health
+      path: /api/diagnostic/healthz/ready
       port: 8000
-    initialDelaySeconds: 10
+    initialDelaySeconds: 15
     periodSeconds: 10
 ```
 
+| Endpoint | Auth | Purpose |
+|----------|------|---------|
+| `/api/diagnostic/healthz/live` | None | Lightweight liveness check — returns 200 if the process is running |
+| `/api/diagnostic/healthz/ready` | None | Readiness check — verifies database and Redis connectivity (503 when DB is down) |
+| `/api/diagnostic/health` | Required | Full health status for monitoring dashboards (Grafana, Uptime Kuma) |
+
 ### Prometheus Scraping
 
 Add annotations to expose metrics (if using a Prometheus-compatible exporter):

docs/ProductionReadiness.md

@@ -34,7 +34,7 @@ Use this checklist to track readiness before going live.
 - [ ] **Redis** — Running and accessible only from internal network
 - [ ] **Meilisearch** — Running and accessible only from internal network
 - [ ] **Worker replicas** — At least 2 workers configured for redundancy
-- [ ] **Monitoring** — `/api/health` polled by uptime checker
+- [ ] **Monitoring** — `/api/diagnostic/health` polled by uptime checker
 - [ ] **Backups** — Automated backup of database, workdir, and Meilisearch data
 - [ ] **Log retention** — Logs shipped to a persistent store or aggregator
 - [ ] **Secrets management** — API keys not committed to source control
@@ -285,22 +285,24 @@ For SSO/OIDC (Authentik, Keycloak, Auth0, etc.) see the [Authentication Setup Gu
 
 ### Docker Compose
 
-Use the `deploy.replicas` setting (requires Docker Swarm mode) or simply run multiple workers:
+Scale workers independently:
-
-```yaml
-worker:
-  deploy:
-    replicas: 3
-```
-
-Or scale after deployment:
 
 ```bash
-docker-compose up -d --scale worker=3
+docker compose up -d --scale worker=3
 ```
 
 Each worker processes tasks from the Celery queue independently.  Ensure the shared `workdir` volume is accessible from all worker containers.
 
+> **Important:** The `beat` service (Celery Beat scheduler) must always run as exactly **one** instance.  It is defined as a dedicated service in `docker-compose.yaml` with a fixed `container_name`.  Do not scale it.
+
+### Scaling the API
+
+API pods are fully stateless (sessions use encrypted cookies, not server-side state) and can be scaled behind a load balancer:
+
+```bash
+docker compose up -d --scale api=3
+```
+
 ### Kubernetes (Helm)
 
 ```yaml
@@ -339,11 +341,32 @@ celery -A app.celery_worker worker -Q default,celery --concurrency=2
 
 ### Health Check Endpoint
 
-DocuElevate exposes `/api/health` for readiness probing.  Configure your uptime monitor to poll this endpoint:
+DocuElevate exposes three health-related endpoints:
+
+| Endpoint | Auth | Purpose |
+|----------|------|---------|
+| `GET /api/diagnostic/healthz/live` | None | Lightweight liveness probe — returns 200 if the process is running |
+| `GET /api/diagnostic/healthz/ready` | None | Readiness probe — checks database and Redis (503 when DB is down) |
+| `GET /api/diagnostic/health` | Required | Full status for monitoring dashboards (Grafana, Uptime Kuma) |
+
+For **Kubernetes probes**, use the unauthenticated endpoints:
+
+```yaml
+livenessProbe:
+  httpGet:
+    path: /api/diagnostic/healthz/live
+    port: 8000
+readinessProbe:
+  httpGet:
+    path: /api/diagnostic/healthz/ready
+    port: 8000
+```
+
+For **uptime monitors** (Uptime Kuma, Grafana, etc.), use the authenticated endpoint:
 
 ```bash
-curl http://docuelevate.example.com/api/health
+curl http://docuelevate.example.com/api/diagnostic/health
-# Expected: {"status": "ok", ...}
+# Expected: {"status": "healthy", ...}
 ```
 
 Set `UPTIME_KUMA_URL` to your Uptime Kuma push URL for heartbeat monitoring:
@@ -502,4 +525,4 @@ For a dedicated Kubernetes deployment guide, including architecture diagrams, PV
 
 - **Image Pull Policy**: Use `IfNotPresent` in production with pinned image tags (not `latest`) for reproducible deployments.
 
-- **Liveness & Readiness Probes**: Already configured in the Helm chart via `/api/health`.  Verify they are tuned to your startup time.
+- **Liveness & Readiness Probes**: Already configured in the Helm chart via unauthenticated endpoints (`/api/diagnostic/healthz/live` and `/api/diagnostic/healthz/ready`).  Verify they are tuned to your startup time.