Internal mTLS
ANVILBASE_MTLS_ENABLED=true turns on mutual TLS for service-to-service
traffic: internal listeners require a client certificate signed by the
AnvilBase internal CA, and every internal client presents its own service
certificate. Off by default; when the flag is unset or false, every service
behaves byte-for-byte as before (plaintext on the internal network).
Enabling
docker/mtls/generate-certs.sh # internal CA + one cert per service# .env:ANVILBASE_MTLS_ENABLED=truedocker compose up -dCertificates mount read-only at /etc/anvilbase/tls/. Each service reads the
same three env vars (compose sets per-service defaults):
| Variable | Meaning | Example default |
|---|---|---|
ANVILBASE_MTLS_ENABLED | one switch for BOTH legs (listener + outbound) | false |
ANVILBASE_MTLS_CERT | the service’s own cert (server and client identity) | /etc/anvilbase/tls/<service>.pem |
ANVILBASE_MTLS_KEY | the service’s private key | /etc/anvilbase/tls/<service>-key.pem |
ANVILBASE_MTLS_CA | the internal CA bundle (sole trust anchor) | /etc/anvilbase/tls/ca.pem |
Every generated certificate carries extendedKeyUsage = serverAuth, clientAuth
and SANs for the compose service name (control-plane, auth, realtime,
deno, …), localhost, and 127.0.0.1 — so one cert per service covers both
its listener and its outbound identity.
The internal edge map
Every service-to-service HTTP/WS edge in the platform, and what
ANVILBASE_MTLS_ENABLED=true does to it:
| # | Edge | Carries | With mTLS on |
|---|---|---|---|
| 1 | control-plane → auth (GoTrue/native auth proxy, users admin, invalidation) | per-project JWT secrets | mTLS — https://auth:3002, control-plane client cert; auth’s Bun listener requires + verifies it |
| 2 | auth → control-plane (POST /internal/email/send) | internal secret, email payloads | mTLS — https://control-plane:3001, auth client cert via Bun fetch tls: init |
| 3 | realtime → control-plane (GET /internal/projects/:id/jwt-secret) | per-project JWT secrets | mTLS — https, realtime client cert via :httpc ssl options (verify_peer against the internal CA) |
| 4 | control-plane → realtime (WS bridge + longpoll proxy, health probe) | user JWTs in ?token= | mTLS — wss/https://realtime:4000; Bandit serves verify_peer + fail_if_no_peer_cert, plaintext listener disabled; the WS bridge presents the control-plane cert via a rustls connector |
| 5 | control-plane → deno (function invoke, deploy sync, health probe) | project DB URLs, service-role/anon keys (headers) | TLS, server-auth only — https://deno:8082 with the internal CA pinned. Gap: Deno.serve cannot VERIFY client certificates (Deno 2.1.x), so this leg is encrypted + server-authenticated but not mutually authenticated |
| 6 | deno worker → control-plane (lazy rehydration fetch) | internal secret, function source | mTLS — https, deno client cert via Deno.createHttpClient({ caCerts, cert, key }); workers get read access to exactly the three cert files, only when the flag is on |
| 7 | Traefik → control-plane / auth | everything at the edge | mTLS — docker/traefik/dynamic/mtls.yml serversTransports.internal-mtls presents the traefik cert |
| 8 | control-plane ↔ webhooks service | delivery rows | no HTTP edge — Postgres LISTEN/NOTIFY + shared DB (see #11) |
| 9 | webhooks → customer endpoints | signed webhook payloads | external — never sends internal client certs (by design); HMAC signatures + SSRF guard apply |
| 10 | deno supervisor → worker | invocation traffic | loopback (127.0.0.1 inside one container) — plaintext by design |
| 11 | * → Postgres / Valkey / MinIO / imgproxy / SMTP | data backplane | out of scope for this switch (plaintext on the internal network). Postgres TLS, MinIO TLS and Valkey TLS are separate hardening items; until then keep them unpublished on a private network |
Additional consequence of #5/#6: the X-AnvilBase-Url context header handed to
edge functions advertises https://control-plane:3001/... when mTLS is on —
function code calling back into the REST engine must use
Deno.createHttpClient with the mounted cert material (the worker has read
access to it), since the control-plane listener rejects uncertified callers.
Plaintext refusal
With the flag on, internal listeners fail closed:
- control-plane: rustls
RequireAndVerifyClientCert— plain HTTP and cert-less TLS are both refused at the handshake. - auth:
Bun.servewithrequestCert: true+rejectUnauthorized: true. - realtime: Bandit/
:sslwithverify: :verify_peer+fail_if_no_peer_cert: true; the plaintext:httplistener is disabled. - deno: TLS listener (plain HTTP refused); client certs are accepted but not verified (the documented gap above).
A missing/corrupt cert file fails loudly at boot (control-plane, auth, realtime, deno listener) rather than silently downgrading to plaintext.
External access
With mTLS on, browsers/CLIs cannot talk to internal listeners directly —
route external traffic through Traefik, which presents its own client
certificate (docker/traefik/dynamic/mtls.yml). The compose healthchecks are
mTLS-aware (the control-plane probes itself with its own cert).
Smoke coverage
scripts/mtls-smoke.sh (CI: .github/workflows/mtls-smoke.yml — label a PR
ci:mtls, push to main touching the mTLS surface, or run it from the Actions
tab) boots postgres + minio + valkey + control-plane + auth + deno + realtime
with mTLS on and asserts:
- control-plane/auth/realtime answer mTLS health probes and reject plain-HTTP and cert-less HTTPS;
- deno answers TLS health probes and rejects plain HTTP;
- the full proxied chain works: project create →
db push→ REST query → GoTrue signup (control-plane→auth) → function deploy + invoke (control-plane→deno) → runtime-cache wipe + reinvoke (deno→control-plane mTLS rehydration).
Rotation
Re-run docker/mtls/generate-certs.sh (delete the expiring service’s
<service>.pem/<service>-key.pem first — the script skips existing files),
then roll the affected services. The CA is valid for 10 years; service certs
for 1 year.
Next: Network Security.