Three years of a hot tub lying to Home Assistant — and the 200 lines that finally shut it up

Three years ago I retrofitted a Balboa BWA Wi-Fi Module (part 50350) to my hot tub — a £270 aftermarket board that turns any BP-controlled Balboa spa into a TCP-speaking device on the local network. Home Assistant's stock balboa integration talks to it directly via pybalboa. The setup takes about 90 seconds and works out of the box.

Right up until it doesn't.

The pain

For three years I lived with:

• climate.spa showing Unavailable in Lovelace for hours at a stretch, seemingly at random
• HA logs full of TimeoutError and ConnectionResetError
• Rare but genuinely alarming HA restarts triggered by memory pressure while the integration was reading from a hung socket
• Automations firing into the void because the spa entity had been stale for 40 minutes and nobody knew
• The classic "is it my Wi-Fi?" rabbit hole — nope, RSSI is −44 dBm, the tub is 3 metres from the AP

I put up with it because there was no supported alternative that didn't involve opening the spa panel, voiding warranty, and soldering to the RS485 bus (which is a great fix — see HyperActiveJ's ESP32 bridge project — but not one I wanted to attempt in February with the tub full of water).

Then I got annoyed enough to actually measure it.

The measurement

I wrote a small soak harness in Python — a script that opens a TCP connection to the module, tracks whether messages are arriving, logs every state transition, and produces a CSV I can analyse. Critically, it uses the same connection.py module the HA integration would eventually use, so what I measure on my Mac is exactly what runs in Home Assistant later.

I ran it against my spa for 14.75 hours on excellent Wi-Fi. The results were worse than I expected:

The module is silently unresponsive two-thirds of the time. Overnight it can go dark for over an hour and a half. This is not a Wi-Fi problem, not a channel problem, not a router problem — the module firmware just stops sending data.

The root cause: zombie sockets

Here's what "stale socket" means in practice.

A healthy Balboa TCP session looks like this:

Client               Module
  |                    |
  |---- SYN --------->|
  |<--- SYN-ACK ------|
  |---- ACK --------->|   (connection established)
  |                    |
  |<-- status frame --|   (module streams 1 msg/sec)
  |<-- status frame --|
  |<-- status frame --|
  |         ...        |

Here's what the 50350 does after ~30 seconds:

  |<-- status frame --|
  |<-- status frame --|
  |<-- status frame --|
  |                    |   ← module goes silent, socket stays open
  |                    |
  |                    |
  |                    |   ← 40 minutes later, still nothing

The module never sends a FIN. Never sends an RST. Never closes the connection. From pybalboa's perspective, the TCP connection is fine — socket.recv() just blocks forever waiting for data that will never arrive.

Multiply by a few hours of unattended operation and the integration:

1. Freezes its entity states on their last known values (the temperature shown when I unplugged the tub? The reading from 40 minutes ago)
2. Piles up half-processed read buffers in memory
3. Occasionally trips HA's memory watchdog and restarts the whole supervisor

The fix

Once the failure mode is understood, the fix is well-known distributed-systems hygiene: you cannot trust the TCP layer to tell you the peer is alive. You have to prove liveness at the application layer.

The pattern is a supervised connection with three layers of defence:

1. Heartbeat / staleness detection

The Balboa protocol streams one status frame per second. Any silence longer than N seconds means the module has hung, regardless of what TCP thinks.

async def _heartbeat_loop(self):
    while self._running:
        await asyncio.sleep(self.config.heartbeat_interval)
        silence = time.monotonic() - self._last_frame_at
        if silence > self.config.stale_after:
            _LOGGER.warning(
                "Link stale (%.1fs > %.1fs) — tearing down",
                silence, self.config.stale_after,
            )
            await self._teardown_and_reconnect()

2. Zombie-socket rejection at connect time

A fresh connection where TCP handshake completes but no spa data arrives within the connect timeout isn't a success — it's a zombie. Reject it.

async def _connect_once(self):
    reader, writer = await asyncio.wait_for(
        asyncio.open_connection(self.host, self.port),
        timeout=self.config.connect_timeout,
    )
    try:
        # Wait for first real spa frame before declaring success
        first_frame = await asyncio.wait_for(
            self._read_frame(reader),
            timeout=self.config.connect_timeout,
        )
    except asyncio.TimeoutError:
        writer.close()
        await writer.wait_closed()
        raise NoSpaDataError("TCP ok but no spa data — zombie socket")
    return reader, writer, first_frame

3. Exponential backoff with cap

Retrying every second when the module is in a 104-minute dead zone is pointless — you just fill your logs. Back off, but not to infinity:

def _next_delay(self, attempt: int) -> float:
    delay = self.config.backoff_initial * (self.config.backoff_factor ** attempt)
    return min(delay, self.config.backoff_max)

Defaults: 5s → 10s → 20s → 40s → 80s → 120s (capped). All tunable in the HA UI, hot-applied without a restart.

The result

Wrapping pybalboa with a SpaConnectionManager that owns the connection lifecycle, and having the HA integration talk to that instead of directly to the socket, turned three years of chaos into a well-behaved integration:

• No more crashes / memory leaks
• Entity states flip to Unavailable within 30 seconds of the module going silent (was: never)
• Automations gated on binary_sensor.spa_reachable fire cleanly instead of into the void
• A rolling sensor.spa_uptime_rolling shows exactly how flaky the module is (spoiler: still 34%)

Note what this does not do: it doesn't fix the module's firmware. The 50350 still goes silent for 30 s to 100+ min at a time. Nothing running in Home Assistant can change that — the bug is on the other side of the wire. What this delivers is a shock absorber: the road is still bumpy, but the car stays on it and the passengers don't get thrown out.

Native HA design as a forcing function

The interesting side effect of writing a "properly HA-native" integration is that the platform's own type system nudges you toward good ergonomics. Instead of shoving everything into sensor entities, I got to use:

The connection-health surface alone is worth writing: exposing connection_state, uptime_rolling, connect_latency, next_attempt_at, connections_lost, and a fault event entity means you can see the module's misbehaviour instead of guessing at it. In three years of the stock integration I never knew whether "unavailable spa" was a Wi-Fi blip, a HA bug, or the module — because none of that state was visible. Now it's a live graph on my dashboard.

Try it, break it, improve it

Repo: github.com/paw2paw/balboa_robust — MIT licensed.

Install via HACS as a custom repository. Not a replacement for the stock balboa integration — if your module is healthy, keep using stock, it's maintained by the HA core team and battle-tested by thousands of users. This one only earns its keep if you're seeing the specific 33%-uptime, stale-socket, "temperature-still-shows-when-unplugged" pattern.

Two things I'd love from readers:

1. Confirmation — if you own a retrofitted 50350 and are seeing Unavailable several times a day, does this pattern generalise? File an issue with your soak numbers.
2. Upstream — the medium-term goal is to distill the heartbeat / zombie-socket / backoff patches back into a pybalboa PR so the stock integration benefits too. If you know that codebase, DM me.

The connection.py module has zero HA imports on purpose — the resilience layer is a plain Python state machine that anyone can build on. An MQTT bridge that exposes the spa to Node-RED / OpenHAB / anything-that-speaks-MQTT is a weekend's work on top. If that's interesting to you, that's next.

Appendix: the soak harness

The single most useful thing I built was not the integration. It was the ~150-line script that ran overnight and produced this:

=== SOAK SUMMARY ===
Duration:           14h 45m
Effective uptime:   33.85%
Total disconnects:  42
Stale-socket:       41
Clean disconnects:  1
Longest outage:     104m at 05:17
Signal (avg RSSI):  -44 dBm

Before that data existed, this bug was "sometimes the spa is unavailable." After that data existed, it was "the module is unresponsive two-thirds of the time and the failure mode is 98% one specific pattern." That's the difference between complaining and shipping a fix.

If you're debugging any flaky IoT device on your network, write the soak harness first. It'll pay for itself in the first hour.