🕳️ HomeLab Chronicles: Episode 9 - Tunnel Vision

Hey all 👋

State of the lab: two nodes, Flux in charge, Airflow alive, backups nightly, RBAC banished to therapy. The UI works from anywhere in the house. But "in the house" is a service radius measured in Wi-Fi bars, and I wanted my Airflow at airflow.sanskararora.in, from anywhere, like a person with a real platform and not two laptops fighting a router for their lives.

The classic move is port forwarding: open a hole in the router, point it at the NodePort, pray. Every part of that sentence should upset you. My router has already been established, across two episodes, as this saga's least reliable character — and now I'd make it my front door too? Also: home IP published to the world, DDoS surface, dynamic-IP DNS jank. No.

Enter Cloudflare Tunnel. The trick that makes it homelab-perfect: a small daemon (cloudflared) runs inside the cluster and dials out to Cloudflare's edge. Traffic to my domain rides that already-open outbound connection back in. Zero inbound ports. Zero public IP. The router is demoted from front door to hallway, which is frankly all it ever deserved.

🚇 Two Tunnels Diverged in a Yellow Dashboard

First fork in the road, and I took both prongs like a professional: Cloudflare tunnels come in two flavors.

Locally-managed: cloudflared tunnel create on your machine, credentials JSON, routing rules in a config file → which for me means in git. GitOps-pure. Chef's kiss.

Remotely-managed: click "create tunnel" in the Zero Trust dashboard, get a token, routing rules live in the UI.

I wrote the manifests for flavor one, then created the tunnel in the dashboard, which is flavor two. The dashboard then helpfully offers you install commands — sudo cloudflared service install eyJ... — which are for running it on a host, i.e., not what a cluster deployment wants at all. The only thing you need from that whole screen is the token. The long eyJhIjoi... string. That's the tunnel's entire identity; treat it like a password, because it is one.

The in-cluster version is refreshingly small — a Deployment, two replicas with pod anti-affinity so the tunnel survives a node dying, cloudflared tunnel run reading TUNNEL_TOKEN from a Secret:

env:
  - name: TUNNEL_TOKEN
    valueFrom:
      secretKeyRef: {name: tunnel-token, key: token}

The token secret is created by hand with stringData: (paste as-is, the API server base64s it — hand-encoding a token that's already opaque gibberish is how you double-encode and then debug "auth errors" for an hour). The manifest holding it never touches git. .gitignore first, file second — that order, specifically, because git add -A has ended better tokens than mine.

Then in the dashboard: Public Hostname → airflow.sanskararora.in → HTTP → the Envoy gateway service's in-cluster FQDN on 8080. DNS record appears automatically. And because my HTTPRoute matches any Host header (Episode 4's hard-won lesson about hostnames), the request flows: Cloudflare edge → tunnel → Envoy → route → Airflow. Login page. On my phone. On mobile data. In a parking lot. I looked deranged and I did not care.

$ kubectl -n cloudflared logs -l app=cloudflared --tail=5
INF Registered tunnel connection ...
INF Registered tunnel connection ...

Four registered connections per replica. The dashboard's sad little "No connectors installed" flips to two healthy connectors. Lovely.

🚪 The Part You Don't Get to Skip

Now the uncomfortable paragraph. What I just built is: my Airflow admin login, on the public internet, guarded by one password, terminating inside my house. Airflow is a genuinely juicy target — it runs arbitrary code on schedule, that's its job — and it has the CVE history of software that important.

And if you're thinking "nobody knows my subdomain exists" — the TLS certificate for it landed in public Certificate Transparency logs the moment it was issued. Scanners watch those logs like RSS feeds. Your "secret" subdomain gets its first uninvited visitors in minutes, not months. Obscurity isn't a moat; it's a countdown. But in order for me to access it freely i am skipping it for now.

🧾 The Fine Print I'm Choosing to Live With

Remotely-managed means the hostname→service routing lives in Cloudflare's dashboard, not in my repo. Which dents the Episode 6 doctrine: a full cluster rebuild now restores everything except the tunnel's routing table. That's one manual step reintroduced into a system I spent three episodes making manual-step-free. It's written in the README, on the rebuild checklist, where future-me will find it and sigh.

The pure path exists — locally-managed tunnel, config in git, SOPS-encrypted token committed alongside it, flux bootstrap --decryption-provider=sops — and it's genuinely the endgame. But the dashboard tunnel was already made, it works, and I have learned (Episodes 5 through 8, inclusive) to stop refactoring things that are currently succeeding.

🧠 What This Taught Me

• Outbound-only beats port forwarding in every dimension that matters. No inbound holes, no public IP, no router in the trust path.
• Dashboard tunnel = token + rules in UI. CLI tunnel = credentials file + rules in git. Pick one before writing manifests, unlike some people.
• Two replicas + anti-affinity makes the tunnel survive exactly the node-death this cluster has proven it can produce.
• Certificate Transparency means your subdomain was never secret. Assume discovery is immediate, because it approximately is.
• Access in front of everything. Authentication at the edge, before your network — not a nice-to-have, the whole point of fronting with Cloudflare.
• stringData for token secrets. Let the API server do the base64. You will mangle it; it won't.

📋 Quick Reference (For Skimmers)

🚀 What's Next

The Pi is still in the drawer. Cilium is still a rumor. But there's now a third-voter-shaped hole in this cluster's HA story, and dqlite quorum math says three is the first number that actually means anything. So: next episode, the drawer opens. Probably. The drawer has heard promises before.

💬 Final Thoughts

Nine episodes ago this was one laptop that couldn't survive a reboot. It's now a two-node, git-defined, nightly-backed-up, tunnel-fronted, edge-authenticated platform that survives power cuts out of spite. The router remains the weakest member of the team and has been architected around accordingly.

Popcorn 🍿, coffee ☕, UPS 🔌, laminated card 🪪, chart-source bookmark 🔖 — and now a domain. The lab is public. Send help, or DAGs.