gh-christianlouis-inboxconverge/docs/adr/010-hybrid-configuration.md

ADR 010: Hybrid Configuration Model (Database + Environment Variables)

Status: Accepted
Date: 2026-02-20
Deciders: Development Team

Context

The application requires configuration for many operational parameters:

1. Bootstrap secrets that must be available before the database is ready: DATABASE_URL, SECRET_KEY, ENCRYPTION_KEY
2. SMTP relay settings that operators want to change without redeployment: host, port, username, password
3. Processing tuning parameters that may need runtime adjustment: check interval, max emails per run, throttle rate
4. Feature flags for Gmail API, notifications, subscriptions
5. Tier limits (max accounts per subscription tier)

A pure environment-variable approach requires container redeployment for every config change. A pure database approach creates a chicken-and-egg problem for bootstrap settings.

Decision

We will use a hybrid configuration model with the following priority chain:

1. Database (app_settings table)  ← highest priority
2. Environment variable / .env file
3. Built-in default               ← lowest priority

Bootstrap settings (DATABASE_URL, SECRET_KEY, ENCRYPTION_KEY) are always resolved from environment variables only, because the database connection depends on them.

All other settings are resolved by ConfigService, which checks the database first, then falls back to environment/defaults.

Alternatives Considered

1. Environment Variables Only

• Pros: Simple, 12-factor compliant, works everywhere
• Cons: Config changes require redeployment, hard to manage across many instances, no UI for non-technical operators

2. Database-Only Configuration

• Pros: Runtime changes, admin UI possible, audit trail
• Cons: Chicken-and-egg for DATABASE_URL itself; database must exist before the app can read its own connection string

3. External Config Service (Consul, etcd)

• Pros: Distributed config, live reload, service discovery
• Cons: Additional infrastructure to deploy and operate, overkill for initial deployment

4. TOML / YAML Config Files

• Pros: Version-controlled, human-readable
• Cons: Requires file system mounts in Docker, config changes need file edits + possible restart

5. Pydantic Settings Only (env + .env files)

• Pros: Type-safe, validated at startup, no database dependency
• Cons: No runtime mutation, no admin UI, requires env changes for any tuning

Rationale

The hybrid model was chosen because:

1. Bootstrap Problem Solved: Database connection string and secrets are always env-only — no circular dependency
2. Runtime Mutability: SMTP settings, tier limits, and feature flags can be changed via admin API without redeployment
3. Operator Friendly: Non-technical operators can use the admin UI or API to adjust settings
4. Developer Friendly: Developers can use .env files for local overrides without touching the database
5. Gradual Migration: Existing env-based deployments continue to work; database settings are additive

Implementation

Data Model

# backend/app/models/database_models.py
class AppSetting(Base):
    __tablename__ = "app_settings"

    key: Mapped[str] = mapped_column(String(255), primary_key=True)
    value: Mapped[str] = mapped_column(Text, nullable=False)
    description: Mapped[str | None] = mapped_column(Text)
    updated_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True),
        default=lambda: datetime.now(timezone.utc),
        onupdate=lambda: datetime.now(timezone.utc),
    )

ConfigService

# backend/app/services/config_service.py
class ConfigService:
    def __init__(self, db: AsyncSession):
        self.db = db

    async def get(self, key: str, default: str | None = None) -> str | None:
        """Resolve setting: DB → env → default."""
        # 1. Check database
        result = await self.db.get(AppSetting, key)
        if result is not None:
            return result.value
        # 2. Check environment
        env_value = os.getenv(key)
        if env_value is not None:
            return env_value
        # 3. Return default
        return default

    async def set(self, key: str, value: str, description: str | None = None) -> None:
        """Persist setting to database (creates or updates)."""
        setting = AppSetting(key=key, value=value, description=description)
        await self.db.merge(setting)
        await self.db.commit()

Default Seeding at Startup

On first boot, sensible defaults are written to the app_settings table if they don't already exist:

# Called during lifespan startup
async def seed_default_settings(db: AsyncSession):
    defaults = {
        "CHECK_INTERVAL_MINUTES": ("5", "How often to check mail accounts"),
        "MAX_EMAILS_PER_RUN": ("50", "Maximum emails to process per run"),
        "SMTP_PORT": ("587", "SMTP relay port"),
        "TIER_FREE_MAX_ACCOUNTS": ("1", "Free tier account limit"),
        # ... etc.
    }
    for key, (value, description) in defaults.items():
        existing = await db.get(AppSetting, key)
        if existing is None:
            await db.add(AppSetting(key=key, value=value, description=description))
    await db.commit()

Admin API Endpoints

GET    /api/v1/settings              # List all database settings (admin only)
PUT    /api/v1/settings/{key}        # Create or update a setting
DELETE /api/v1/settings/{key}        # Delete a setting (falls back to env/default)
POST   /api/v1/settings/seed-defaults # Re-seed all defaults

Bootstrap Settings (Env-Only)

These are read via pydantic-settings (BaseSettings) and never looked up in the database:

Setting	Required	Purpose
DATABASE_URL	✅	PostgreSQL connection string
SECRET_KEY	✅	JWT signing key (≥32 chars)
ENCRYPTION_KEY	✅	Fernet master encryption key
GOOGLE_CLIENT_ID	Optional	Google OAuth2
GOOGLE_CLIENT_SECRET	Optional	Google OAuth2

Consequences

Positive

• Bootstrap settings always resolved from env (no circular dependency)
• Runtime-mutable settings require no redeployment
• Default values seeded at startup so application works out-of-the-box
• Clear priority chain: database overrides env, env overrides code default
• Admin API provides complete CRUD on runtime settings

Negative

• Two sources of truth for settings requires careful documentation
• Database settings take precedence over env — operators must know to check the database if env changes seem to have no effect
• ConfigService requires a database session (async), adding overhead for high-frequency config reads
• Settings cache is not implemented; every lookup hits the database (future improvement: TTL-based cache)

Future Improvements

1. In-Memory Cache: Cache settings with a short TTL (e.g., 30 seconds) to reduce database load
2. Change Notifications: Pub/sub via Redis to notify workers of setting changes
3. Typed Settings Schema: JSON Schema validation for setting values
4. Audit Trail: Record who changed each setting and when

Related Decisions

• See ADR-004 for PostgreSQL (stores app_settings table)
• See ADR-006 for key management (bootstrap secrets are env-only)
• See ADR-003 for FastAPI dependency injection of ConfigService

References

• 12-Factor App: Config
• Pydantic Settings
• SQLAlchemy ORM