madbox
Blog

Getting k3s onto Rocky Linux: three firewalls and a stubborn certificate

16 July 2026

This site used to run on Docker Compose behind Traefik and nginx. I wanted to try something closer to how a real cluster is run, so I tore it down and put k3s directly on the box instead — one lightweight Kubernetes node hosting this site along with a couple of others. It went about as smoothly as homelab infrastructure ever does, which is to say: not smoothly, but instructively.

SELinux wants a say first

Rocky Linux ships with SELinux enforcing by default, and the k3s installer doesn't pull in what it needs on its own. Installing container-selinux and selinux-policy-base before running the install script saved a round of confusing failures later.

A Traefik crash that fixed itself

Right after install, the bundled Traefik job went into CrashLoopBackOff complaining that its CRDs were missing. Classic chicken-and-egg: a separate job installs the CRDs, and Traefik's own install job was racing it. No fix needed — it retried and passed once the CRD job caught up.

502s that made no sense

Every request came back 502, even plain requests to the homepage. The confusing part: curling a backend pod's IP directly from the node worked fine, which pointed away from the real problem. The actual issue was that firewalld was blocking pod-to-pod traffic — host-to-pod worked because it doesn't cross that boundary, but Traefik proxying to a backend pod does. Trusting the pod and service CIDRs in firewalld's trusted zone fixed it immediately.

The tell: if host-originated requests succeed but the same request routed through your ingress controller fails, look at pod-to-pod traffic specifically — it's a different path than anything you've tested from the node itself.

Reloading firewalld broke the exit path too

Fixing the trusted zone required a firewall-cmd --reload, which turned out to knock over k3s's own iptables rules for outbound/NodePort traffic. A systemctl restart k3s made it reconcile its rules again — worth remembering any time firewalld gets reloaded on a k3s box.

The last mile wasn't mine to fix

Even with everything above sorted, external requests still timed out. Nothing left on the box explained it — which meant it wasn't the box. The hosting provider's own cloud firewall was blocking inbound 80/443, most likely reset during a prior reprovision. That one just needed opening from the provider's control panel, not SSH.

cert-manager holds a grudge

Once traffic actually reached the cluster, Let's Encrypt certificates still refused to issue. Deleting the failed CertificateRequest did nothing — it turns out cert-manager tracks issuance backoff on the Certificate resource itself, not the request. Deleting the Certificate was the fix: since it's owned by the Ingress, it was recreated instantly with a clean history, and it issued within a minute.

Four bugs, three of them wearing a firewall costume

None of this was exotic — SELinux, two layers of firewalld, a cloud provider's own edge, and cert-manager's retry logic. But each one hid the next one behind it, so the fix only looked obvious in hindsight. All three domains are now serving real TLS certificates off a single k3s node, which is exactly the point of running a homelab: nothing here was necessary, and now I understand all of it a little better.