Scaling

AnvilBase scales well because per-project overhead is tiny (the built-in REST engine uses ~2–5 MB pools instead of per-project containers). The constraints to manage are shared rate-limit state, Postgres connections, and upstream back-pressure.

Multi-replica control plane requires Valkey

The control plane’s rate limiter has two backends:

Backend	When	Consistency
Valkey (primary)	`VALKEY_URL` reaches a live instance	global — shared across all replicas
In-memory LRU (fallback)	Valkey unreachable at boot	per-process — each replica counts separately

If you run more than one control-plane replica, Valkey MUST be reachable. Otherwise a caller can burst N × replicas before any single replica rejects them — rate limits become a suggestion.

Detecting the degraded state

Startup log (WARN): rate limiting in degraded mode (in-memory, per-process) — grep rate_limit_for_scope=in_memory_fallback.
/health/services: warnings[] includes rate_limit_degraded_fallback_to_in_memory; the Valkey entry’s mode is in_memory_fallback.
Alert on that exact string whenever replicas > 1 (Monitoring).

Fail fast instead of degrading silently

ANVILBASE_RATE_LIMIT_REQUIRE_VALKEY=true

The control plane then panics at boot if Valkey construction fails. Pair with a Kubernetes readinessProbe so a degraded replica never takes traffic.

Per-project connection pool sizing

Each project gets its own Postgres pool inside the control-plane process.

Default cap: 20 concurrent connections per project (ANVILBASE_PROJECT_POOL_MAX, built-in DEFAULT_PLATFORM_POOL_MAX).
The 21st concurrent request gets 429 with Retry-After: 1 and {"error":{"kind":"db_pool_exhausted","code":429}}. That’s back-pressure, not an error — clients with exponential backoff recover.
Raise it (ANVILBASE_PROJECT_POOL_MAX=<N>) only if the workload justifies it, and keep the ceiling:
```
replicas × active_projects × ANVILBASE_PROJECT_POOL_MAX  <  Postgres max_connections
```
When that math gets tight, put Supavisor in front of Postgres so many logical connections share fewer physical ones.

Auth service back-pressure

The control plane’s client to the auth service enforces a 2s connect / 5s total timeout; on timeout it returns 503 with {"error":"upstream_timeout","service":"auth"}. If /health/services shows auth: unhealthy and you’re seeing 503s, scale the auth service, not the control plane.

What scales how

Component	Scaling
Control plane	horizontal replicas (Valkey required); watch the connection ceiling
Realtime	horizontal (BEAM handles huge WebSocket fan-out per node; cluster for HA) — see the note below
Postgres	vertical + read replicas; Supavisor pooling; Patroni for failover
Valkey	replicated/clustered for HA (it’s now load-bearing for rate limits)
Storage	distributed MinIO or a managed S3-compatible store

Realtime replicas

Running multiple realtime replicas is safe for delivery: each node sources its own change feed (its own per-project LISTEN) and broadcasts change events locally to the channels on that node, so a change that happens once in Postgres reaches each subscriber exactly once — no duplicate events regardless of replica count. The cost is connections: each node opens its own per-project LISTEN + RLS-gate query-pool connections, so size Postgres max_connections for roughly:

replicas × active_projects × (1 + ANVILBASE_REALTIME_QUERY_POOL_SIZE)

Known gap: per-topic broadcast/presence authorization is not yet available — any valid project JWT may join any broadcast/presence topic within its own project (cross-project isolation is enforced; intra-project topic authz is not). Don’t put secrets in broadcast/presence topic names or payloads you wouldn’t share with every member of that project.

Connection pooling (Supavisor)

Supavisor provides transaction- and session-mode pooling, exposing many client connections over a small physical pool. Use it for:

Direct DB connections from trusted services (so they don’t each hold raw Postgres connections),
Keeping active_projects × pool_max under max_connections as you grow.

Checklist before adding a replica

VALKEY_URL reachable from every replica.
Postgres max_connections absorbs replicas × active_projects × pool_max.
Rate-limiter-degraded alert active (Monitoring).
mTLS enabled end-to-end (ANVILBASE_MTLS_ENABLED=true) — internal plaintext is dangerous the moment traffic leaves one host.
ANVILBASE_RATE_LIMIT_REQUIRE_VALKEY=true + readiness probe (recommended).

Next: Upgrades.