Operations

Day-2 essentials and troubleshooting for a self-hosted instance: diagnostics, the license lifecycle, backups, network isolation, common issues, and how to reach support.

Diagnostics — hoppla:doctor

hoppla:doctor is your first stop for almost any question. It verifies DB/TimescaleDB, Redis, queues, the scheduler, SMTP, RLS, and the license, and exits non-zero on failure (safe to script).

kubectl -n hoppla exec deploy/hoppla-api -c api -- php artisan hoppla:doctor

Add --send-test to verify mail delivery end-to-end:

kubectl -n hoppla exec deploy/hoppla-api -c api -- \
  php artisan hoppla:doctor [email protected]

TIP

The doctor also warns when both probe labels (PROBE_LOCATION / PROBE_SECONDARY) are identical — expected on a single-cluster install, a signal to fix once you add a second probe region.

License lifecycle

Your instance is unlocked by a signed license file in the hoppla-license Secret, mounted on every PHP pod. It never validates against Hoppla's servers.

• State shows in the dashboard banner and via php artisan license:verify (and license:inspect for the dates, including updates_until).

• Renewal is just a new license file. Replace the Secret and the instance picks it up within ~60 seconds — no restart, no reinstall, no data loss:

  kubectl -n hoppla create secret generic hoppla-license \
    --from-file=hoppla.license=new-license.txt \
    --dry-run=client -o yaml | kubectl apply -f -

• After expiry there is a grace window, then the instance turns read-only: checks and alerting pause, but dashboards, history, and status pages stay up. Nothing is deleted — renewing resumes monitoring where it left off. (Note the separate updates_until window covered in Upgrading: that only affects whether newer builds run, not your current version.)

Backups

Everything durable lives in PostgreSQL plus the secrets you created (hoppla-secrets, hoppla-license). Back up both.

Enable the built-in pg_dump CronJob in your values:

backups:
  enabled: true
  schedule: "0 2 * * *"
  retentionDays: 30
  persistence:
    size: 20Gi

Or dump on demand:

kubectl -n hoppla exec statefulset/hoppla-postgresql -- \
  pg_dump -U hoppla hoppla > backup-$(date +%F).sql

The data lives on the PVC behind the Postgres StatefulSet — treat the PVC and your dumps as the crown jewels, ship them off-cluster, and test restores periodically. Redis holds only queues and cache, so it needs no backup.

Network isolation

networkPolicy:
  enabled: true
  backend: standard   # any CNI; use 'cilium' only when you run Cilium

With this on, the chart applies per-component ingress/egress rules — PostgreSQL and Redis are reachable only by the app pods. Outbound, the instance needs 80/443 (checks), 587 (SMTP), 43 (WHOIS), and 53 (DNS checks).

Common issues

Symptom	Likely cause / fix
App reachable only via port-forward	ingress.enabled is still false — add an ingress override (Installation).
Certificate not issuing	kubectl -n hoppla get certificate / describe; wildcards need DNS-01, not HTTP-01 (Configuration → TLS).
--send-test passes but real alerts don't arrive	You rolled only the API. Alerts come from the worker pods — rollout restart deploy (all of them).
Redirect loops / broken login behind Cloudflare	SSL/TLS mode is Flexible; switch to Full (Configuration → CDN).
Custom status domain shows an SSL error	The CNAME target is proxied (orange); keep it grey-cloud so the origin gets a cert.
hoppla:doctor warns probe labels identical	Single-cluster install — expected until you add a second probe region.
Live dashboard not updating	Dashboard REVERB_APP_KEY must equal the one in hoppla-secrets; check the /app path isn't cached by your CDN.
Redis OOM / chart refuses to render	The memory invariant: redis.resources.limits.memory >= 2 × maxmemory + 64Mi.

When in doubt, start here:

php artisan hoppla:doctor          # pinpoints the failing layer
kubectl -n hoppla get pods         # everything Running?
kubectl -n hoppla logs deploy/hoppla-api

Support

Email support with your customer ID (in your welcome pack). Please attach the output of php artisan hoppla:doctor — it answers most environment questions upfront.