Upgrading & Maintaining

Everything operational after install: how you learn about new releases, routine upgrades, expedited security updates, rollback, and maintaining the stack underneath (PostgreSQL/TimescaleDB, Redis, your cluster).

Routine upgrade (TL;DR)

# 0. Check your license window covers the release (see "License & update windows")
kubectl -n hoppla exec deploy/hoppla-api -- php artisan license:verify

# 1. ALWAYS back up first
kubectl -n hoppla exec statefulset/hoppla-postgresql -- \
  pg_dump -U hoppla hoppla > backup-$(date +%F).sql

# 2. Upgrade to a pinned version with YOUR values file
helm upgrade hoppla oci://ghcr.io/jkbennemann/charts/hoppla-selfhosted \
  --version <NEW_VERSION> -n hoppla -f my-values.yaml

# 3. Verify
kubectl -n hoppla exec deploy/hoppla-api -- php artisan hoppla:doctor

How you learn about new releases

Your instance is fully offline — it never phones home, so it will not tell you a new version exists. Stay informed through:

• Watch the hoppla-dist repository (GitHub → Watch → Custom → Releases). Every release lands there as a tag plus a new row in VERSIONS.md with the chart reference and image tags.
• VERSIONS.md in hoppla-dist is the canonical version table — newest on top, with the OCI chart coordinates to pass to helm upgrade.
• Security advisories go to the operator contact on your contract and are published in hoppla-dist release notes, flagged [SECURITY].

Recommended cadence: a monthly maintenance window for routine upgrades, plus expedited windows for [SECURITY] releases.

Release types

Versions follow MAJOR.MINOR.PATCH:

Type	Contains	Migrations	Action needed
Patch	Bug fixes, security fixes, dependency bumps	Additive only, usually none	Upgrade promptly; no values changes ever required.
Minor	New features, new optional values	Additive only	Read the notes; new values are optional with safe defaults.
Major	Breaking changes, required values changes, destructive migrations	Possibly destructive	Read the upgrade steps in the notes before touching anything.

Rules that keep upgrades boring:

• Pin versions. Always pass --version; never rely on latest.
• Keep one values file (my-values.yaml) under version control and always pass it with -f. Do not use --reuse-values — it freezes chart-default changes and produces drift that is miserable to debug.
• Upgrade one major at a time (1.x → 2.x → 3.x; never 1.x → 3.x).
• Migrations run automatically — an init container runs php artisan migrate --force before the new API starts; workers receive SIGTERM and drain in-flight checks gracefully (60s).

Security updates

[SECURITY] releases are patch releases, safe to apply immediately:

1. Read the advisory — it states the affected components and whether the fix is in the application images, the chart, or both.
2. Run the routine upgrade (TL;DR above). There is never a values change in a security patch.
3. If the advisory covers a base-image CVE (PHP, nginx, Alpine), the fix is simply the rebuilt image — same procedure.
4. If it covers PostgreSQL/TimescaleDB or Redis and you run the in-cluster defaults, the release bumps postgresql.image.tag / redis.image.tag and the StatefulSet rolls on upgrade. If you bring your own database, patch it on your side per the advisory.

After any security upgrade: hoppla:doctor, then spot-check login and one monitor detail page.

After every upgrade

kubectl -n hoppla exec deploy/hoppla-api -- php artisan hoppla:doctor   # all OK?
kubectl -n hoppla get pods                                              # everything Running?

Then in the dashboard: the license banner is absent (or expected), checks are ticking on a monitor's history, and there is no growth in Admin → System failed-job counts.

Rollback

helm rollback hoppla -n hoppla            # previous revision

If the failed upgrade ran new migrations, restore the database dump from step 1 first, then roll back. (Minor/patch migrations are additive, so a plain helm rollback is normally sufficient.) Roll back one revision and re-verify; never chain rollbacks blind.

License & update windows

Your license carries an updates_until date: builds published after that date run read-only until you renew (the dashboard banner says exactly that, and license:verify exits non-zero). Therefore:

• Before upgrading, compare the release date in VERSIONS.md against your updates_until (php artisan license:inspect shows it). If the release is newer, renew first — then upgrade.
• Renewing = replacing the hoppla-license Secret — picked up within 60 seconds, no restart, no reinstall, no data loss.
• An expired update window never breaks the version you are already running.

Maintaining the stack underneath

PostgreSQL / TimescaleDB (in-cluster default)

• Minor PG / TimescaleDB bumps ship as chart updates (the timescale/timescaledb tag) and roll automatically. After a TimescaleDB bump, the release notes say if ALTER EXTENSION timescaledb UPDATE; is needed — when it is, the notes include the exact step.
• Major PG upgrades (e.g. 16 → 17) are never silent: they only appear in MAJOR releases with a documented dump → upgrade → restore path.
• The data lives on the PVC behind the StatefulSet — treat the PVC + your pg_dumps as the things worth protecting. Test restores periodically.

Redis (in-cluster default)

Redis holds queues and cache, not durable data. Chart bumps roll the StatefulSet; in-flight jobs drain first (workers stop consuming on SIGTERM). Worst case after a hard restart: a few checks run twice. No backup needed.

Kubernetes / nodes

App pods are stateless and reschedule freely. For node maintenance:

• Multi-node: kubectl drain <node> as usual — PodDisruptionBudgets keep the API and workers available.
• Single-node: expect a short full outage during the node reboot; monitoring resumes by itself, and the missed window shows as a gap, not as false downs (no alerts fire for the instance being off).

TLS certificates

cert-manager renews everything automatically. After restoring from backup or moving clusters, re-check kubectl -n hoppla get certificate — all READY.

When something goes wrong

1. php artisan hoppla:doctor — it pinpoints the failing layer.
2. kubectl -n hoppla get pods + kubectl -n hoppla logs deploy/hoppla-api.
3. Roll back (above) — the previous images stay pullable regardless of your update window.
4. Support: email with your customer ID and the full hoppla:doctor output.