Merge pull request #105 from christianlouis/copilot/debug-celery-logging-issues

Fix Celery datetime crash that silently blocked all mail processing + respect sub-5-min poll intervals

CHANGELOG.md

@@ -28,6 +28,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `duration_seconds`. The error handler no longer accesses an expired SQLAlchemy ORM attribute
   (`run.started_at`) after a session rollback, which previously caused the handler to crash and
   left runs stuck in the `running` state indefinitely.
+- **Celery datetime crash**: Fixed `TypeError: can't subtract offset-naive and offset-aware
+  datetimes` in `process_all_enabled_accounts` by wrapping `account.last_check_at` with the
+  existing `_as_utc()` helper before comparing against `datetime.now(timezone.utc)`. This
+  crash silently prevented every mail account from being processed on every scheduled run.
+- **Per-account polling interval**: Changed the Celery beat schedule for
+  `process_all_enabled_accounts` from every 5 minutes (`*/5`) to every minute (`*`). The
+  per-account `check_interval_minutes` field already gates whether an account actually gets
+  processed, so accounts configured with a 1-minute interval are now polled as expected instead
+  of being limited to 5-minute effective intervals.
 - **Processing log early-exit path**: When a mail account has no delivery method configured
   (SMTP credentials missing and Gmail API not set up), the processing run now correctly sets
   `completed_at`, `duration_seconds`, and `account.last_check_at`, preventing the account from

backend/app/workers/celery_app.py

@@ -36,7 +36,9 @@ celery_app.conf.update(
 celery_app.conf.beat_schedule = {
     "process-all-mail-accounts": {
         "task": "app.workers.tasks.process_all_enabled_accounts",
-        "schedule": crontab(minute="*/5"),  # Every 5 minutes
+        "schedule": crontab(
+            minute="*"
+        ),  # Every minute (per-account interval gates actual work)
     },
     "cleanup-old-logs": {
         "task": "app.workers.tasks.cleanup_old_logs",

backend/app/workers/tasks.py

@@ -510,8 +510,8 @@ async def process_all_enabled_accounts():
             for account in accounts:
                 # Check if it's time to check this account
                 if account.last_check_at:
-                    time_since_last_check = (
-                        datetime.now(timezone.utc) - account.last_check_at
+                    time_since_last_check = datetime.now(timezone.utc) - _as_utc(
+                        account.last_check_at
                     )
                     if time_since_last_check.total_seconds() < (
                         account.check_interval_minutes * 60

docs/TODO.md

@@ -14,6 +14,8 @@ Comprehensive task breakdown for repository improvements and production readines
 - [x] Fixed GitOps `update-k8s-manifest` job: corrected image tag computation and `yq` patterns to use `registry.cklnet.com` (private registry) matching the actual k8s manifest image references, so SHA-pinned tags are properly applied on each deploy.
 - [x] Added GitOps auto-deployment step in `ci.yml` to update preprod k8s manifest in `k8s-cluster-state` repo.
 - [x] Fixed GitOps `update-k8s-manifest` job: added PAT availability check to skip gracefully when `GH_PAT` secret is not configured, fixing 403 "Write access to repository not granted" pipeline failure.
+- [x] Fixed Celery `TypeError: can't subtract offset-naive and offset-aware datetimes` in `process_all_enabled_accounts` — all mail accounts were silently skipped on every scheduled run.
+- [x] Changed Celery beat schedule to run `process_all_enabled_accounts` every minute so accounts configured with `check_interval_minutes = 1` are polled as expected.
 
 ## 🔴 Critical - Security (In Progress)