Merge pull request #753 from christianlouis/copilot/scale-worker-and-api-pods

feat(scaling): enable horizontal scaling for API and worker pods
This commit is contained in:
Christian Krakau-Louis
2026-03-20 13:55:31 +01:00
committed by GitHub
10 changed files with 384 additions and 35 deletions
+10 -6
View File
@@ -349,16 +349,18 @@ workdir:
## Scaling
DocuElevate is designed for horizontal scaling. Both API and worker pods are stateless and can be scaled independently.
### Docker Compose
Add more worker containers:
Scale workers (task processing) and API pods (request handling) independently:
```yaml
worker:
deploy:
replicas: 3
```bash
docker compose up -d --scale worker=3 --scale api=2
```
> **Note:** The `beat` service (Celery Beat scheduler) must always run as exactly **one** instance. Do not scale it. It publishes periodic tasks to the Redis broker; workers pick them up.
### Kubernetes / Helm
Enable HPA:
@@ -377,13 +379,15 @@ worker:
maxReplicas: 10
```
The Helm chart deploys a separate **beat** pod (always 1 replica, `Recreate` strategy) so that scheduled tasks are never duplicated when workers scale.
---
## Monitoring
- **Docker Compose**: `docker-compose logs -f`, `docker stats`
- **Kubernetes**: `kubectl logs -l app.kubernetes.io/component=api -f`
- **Prometheus / Grafana**: Scrape the `/api/health` endpoint for readiness; add custom metrics as needed.
- **Prometheus / Grafana**: Scrape the `/api/diagnostic/healthz/ready` endpoint for readiness; add custom metrics as needed.
- **Uptime Kuma**: Set `UPTIME_KUMA_URL` to your push URL for heartbeat monitoring.
---