Scaling & Deployment Scenarios

Where to run Hoppla, which values profile to start from, the one hard database constraint, and how to grow into multiple probe regions.

Choosing a profile

Each profile is a values file that overlays the chart defaults. Keep exactly one copy per cluster under version control and always pass it with -f (never --reuse-values).

Profile	Scale	Data layer	Shape
values.self-hosted-small.yaml	≤ ~1k monitors	in-cluster	single node, 4 vCPU / 8 GB, 1 replica of everything
values.self-hosted-medium.yaml	~1k–10k	in-cluster, sized up	3+ nodes, HPA on the check pipeline, PodDisruptionBudgets
values.self-hosted-large.yaml	> 10k	external (yours)	autoscaled serving + workers, no DB/Redis pods
values.self-hosted-probe.yaml	add-on	none (uses central)	worker-only secondary region

The TimescaleDB constraint

Read this before choosing a database

Hoppla stores check results in a TimescaleDB hypertable. AWS RDS and GCP Cloud SQL do not support the TimescaleDB extension — pointing Hoppla at them fails at migration time.

Your options, in order of operational simplicity:

1. In-cluster PostgreSQL (chart default, small/medium profiles) on fast block storage. This is the supported managed-cloud path up to ~10k monitors. Treat the PVC and your pg_dumps as the crown jewels.
2. Timescale Cloud — managed TimescaleDB; works with the large profile.
3. Self-managed Postgres VM with the TimescaleDB extension.
4. Azure Database for PostgreSQL Flexible Server — the one major managed offering that allowlists timescaledb.

Redis has no such constraint — ElastiCache, Memorystore, or any Redis 7 works with the large profile.

Provider notes

Generic k8s / k3s / on-prem (reference platform)

The reference platform. ingress-nginx + cert-manager (HTTP-01), default storage class, done. Single-node k3s with the small profile is a perfectly good production setup at small scale.

Amazon EKS

• Storage: install the EBS CSI driver; set postgresql.persistence.storageClass: gp3 (and redis.persistence if you keep it in-cluster).
• Ingress: ingress-nginx behind an NLB is the path of least resistance — the chart's annotations target nginx (ingress.className: nginx). The AWS Load Balancer Controller (ALB) works too, but you own translating the annotations, and the custom-domain provisioner only creates Ingress/Certificate objects — pair ALB with customDomains.provisioner: manual unless you know your controller handles them.
• TLS: cert-manager with HTTP-01 through the NLB, or DNS-01 via Route 53 (needed for wildcard portal domains).
• DNS — EKS differs from GKE/k3s: an NLB has no static IP, only a DNS hostname (...elb.<region>.amazonaws.com), exposed under .status.loadBalancer.ingress[0].hostname (not .ip). So:
  • Point records with a CNAME to the NLB hostname, not an A record to an IP.
  • An apex domains.app can't be a CNAME — use a Route 53 Alias to the NLB, or use a subdomain. external-dns automates either.
• Database: RDS is not an option. Use gp3-backed in-cluster Postgres or Timescale Cloud.
• No IAM roles are required by Hoppla itself — it only needs outbound internet (checks), SMTP, and its own data stores.

Google GKE

• Storage: premium-rwo (pd-ssd) for Postgres; standard-rwo is fine for Redis.
• Ingress: ingress-nginx (ingress.className: nginx) behind a TCP LB is the validated path; GKE's built-in GCE ingress class has different annotation semantics (same caveat as ALB above).
• TLS: cert-manager HTTP-01, or DNS-01 via Cloud DNS for wildcards.
• Database: Cloud SQL is not an option.
• Autopilot: works, but it rewrites resource requests — review the check-pipeline HPA targets after the first day of traffic.

Azure AKS

• managed-csi-premium storage for Postgres; ingress-nginx + cert-manager — same shape as EKS/GKE.
• Azure is the one cloud where a managed database works: Flexible Server with the timescaledb extension enabled, plus the large profile.

DNS records you need

Point everything at your ingress controller's external address (the prerequisites step prints it). On IP-based load balancers (GKE, k3s/MetalLB, bare metal) that's an A record to the ingress IP — list more than one (one per node) for redundancy. On hostname load balancers (AWS NLB/ELB) there is no IP: use a CNAME to the LB hostname (apex domains → a Route 53 Alias / external-dns), and read the table's "A → ingress IP" rows as "CNAME → ingress hostname".

Record	Type	Value	When
domains.app (e.g. monitoring.example.com)	A	ingress IP	always
*.<statusRoot> (e.g. *.status.example.com)	A	ingress IP	if you publish status pages on a status subdomain
*.<portalRoot> (e.g. *.portal.example.com)	A	ingress IP	if you run white-label reseller portals
cname.<statusRoot>	A	ingress IP	the target customers point their own status domain at (covered by the *.status wildcard above)
a customer's own domain (status.client.com)	CNAME	cname.<statusRoot>	added by the customer, then verified in the dashboard

*.status and *.portal are two-level wildcards — plain HTTP-01 cannot issue certs for them, so use a DNS-01 issuer (see Configuration → TLS) or per-host certs. The app domain alone is enough for a basic single-tenant install.

Multi-probe locations

False-positive pages are the enemy: Hoppla never alerts on a single vantage point. A failed uptime check is re-queued to uptime-verify and must also fail from the secondary probe before an incident opens. On a single cluster both labels are honest-but-identical (hoppla:doctor warns). A real second region looks like this:

  CENTRAL cluster (eu)                      PROBE cluster (us)
  ┌───────────────────────────┐             ┌──────────────────────┐
  │ api · scheduler · reverb   │             │ worker: uptime-verify│
  │ dashboard · alerts · ...   │   private   │ (optionally: uptime, │
  │ workers: uptime, dns, ssl  │◄──network──►│  dns, ssl pools)     │
  │ PostgreSQL ─ Redis         │   peering   │ no API, no scheduler,│
  └───────────────────────────┘             │ no DB, no Redis      │
                                             └──────────────────────┘

1. Deploy the probe profile in the second region — helm install hoppla-probe <chart> -n hoppla -f values.self-hosted-probe.yaml — with config.DB_HOST / config.REDIS_HOST pointing at the central endpoints and PROBE_LOCATION set to an honest label (us-east, …).
2. Connectivity: the probe cluster needs central Redis (6379) and Postgres (5432). Use VPC peering, WireGuard, or Tailscale — never the public internet. DB_SSLMODE: require is preset in the profile.
3. Move the verify pool: on the CENTRAL cluster, disable its local uptime-verify pool (workers.uptime-verify.enabled: false) so the second opinion always comes from the remote region. Set the central PROBE_SECONDARY env to the probe's label.
4. Keep singletons central: exactly one scheduler (the probe profile disables it) and one alerts pool across all clusters — a second scheduler double-runs every check; a second alerts pool double-pages people.
5. Verify: central hoppla:doctor (probe labels no longer identical), then a deliberate failure — block one monitor's target from the central region only; no alert should fire, and its results show both location labels.

Latency tolerance: workers batch their writes, so transatlantic round-trips to Postgres/Redis are fine for verification workloads. If you also move PRIMARY check pools to the probe (workers.uptime.enabled: true), keep queue latency under ~100ms or throughput drops noticeably.

Scaling knobs

Reach for these in order:

1. workers.uptime.replicaCount / its HPA — the check pipeline's throughput. This is almost always the first lever.
2. UPTIME_BATCH_SIZE (per-worker env) — more checks per batch job, more memory per process.
3. redis.maxmemory — queues live here. Invariant: resources.limits.memory >= 2 × maxmemory + 64Mi (BGSAVE fork headroom; the chart refuses to render violations).
4. PostgreSQL resources + storage IOPS — check-result writes are the main DB load; prefer faster disks over more CPU.
5. api.replicaCount / HPA — only when dashboard/API latency, not check throughput, is the bottleneck.
6. Never scale scheduler (single dispatcher by design), and never run the alerts pool in more than one cluster.