For eleven days our barometer read 981.8 hPa. That number was plausible atmospheric pressure for
our altitude, it never looked odd, and nothing downstream ever had a reason to ask about it.
The sensor was fine the whole time. The value was a fossil.
The last=<hpa> line we were reading from adb shell dumpsys sensorservice is a frozen snapshot
from early boot. It stayed pinned at the same reading while the phone's uptime climbed to 11.8
days β proven at active sensor count zero, with the values frozen at 17 minutes of uptime. The
number was not wrong. It had simply stopped being measured, and it kept reporting as if it were.
A stale value that never looks stale is worse than no value, because nothing downstream has a
reason to ask.
The same sensor, a second dead read path
Having moved the read off the fossil onto a live listener, the pressure went dark again eleven
weeks later. This time the failure was a wedged adb server: the shell call hung until it timed
out, rc=124, while adb devices still listed the phone as present.
That is the ambiguous case. The device table said healthy. The read did not return. Recovery was a
scoped adb kill-server, no commit and no new tool.
Both failures are the same shape. The sensor was fine, the read path was dead, and the dead path
advertised a present device.
The corrective is not a better health check
The instinct when a reading looks wrong is to distrust the sensor, or to add a health check on the
read path. Both failures here were in the transport, and in both the transport reported itself
healthy. The fossil returned a plausible number. The wedged server returned a listed device.
Neither failure was visible from the value alone.
The fix was to make the consumer the freshness authority, and to accept that a producer's
honest answer is sometimes "no number".
The producer now reads a native sensor listener, which actually samples:
SENSOR|SHTC1 ambient temperature sensor|13|23.4052|26.16|3|243885381030238
SENSOR|SHTC1 relative humidity sensor|12|55.175|45.66|3|243885381022196
SENSOR|LPS25H Barometer Sensor|6|995.8|146.306|30.3104|243885601563443
Moving to the listener also recovered live temperature and humidity, which had been permanently
n/a before. On-change sensors with no listener report exactly 0.0, which is another way to
look healthy while saying nothing.
Four behaviours, each measurable
1. A failed read writes no state. With the read path forced dark, the producer exits rc=2
and its state file is left untouched. A failure can never become tomorrow's baseline. The exit
code is part of the signal: rc=2 is an honest n/a, and the landing layer reads it as a pass
rather than as a faked all-clear.
2. An unreachable input is announced once per outage. A second consecutive dark run emits the
offline line a single time, then the next live read clears the marker and returns rc=0. One
announcement per outage, not one per cron tick.
3. The consumer does not trust the number β it trusts its freshness. This is the part that
generalises. With the producer's state aged past its TTL, the consumer does not echo the last
good value. It drops the field and decrements its own input count:
STABLE | pressure=STALE | out_c=18.1 ... | inputs=2/3
Restore freshness, and the field comes back with the count:
STABLE | in_hpa=995.84 trend=STABLE d_hpa=+0.11 | out_c=18.1 ... | inputs=3/3
A stale producer cannot ship a number, because the consumer removes the number and shows you the
count went down.
4. The drift reference is only written when stable. The producer's state format is
<hpa> <trend> <baseline_hpa>; the baseline is updated only on a STABLE reading, so the
reference a future reading is compared against cannot itself be set by a noisy one.
What it looks like when it works
The live chain, captured while writing this:
mesh-baro β [baro-stable] 996.38 hPa β STABLE (Ξaltβ-1.9 m from baseline 996.17 hPa) rc=0
mesh-climate β in_hpa=996.38 trend=STABLE d_hpa=+0.21 | inputs=3/3 rc=0
Each stage is a one-line check that fails loudly if the contract regresses. That is the part
worth keeping: not the specific tooling, but that a freshness contract is only as good as the
assertions that would go red if it silently stopped holding.
Where this transfers
Anywhere a cached reading is displayed as though it were live. If your dashboard shows a sensor
value, the question is not "is this a good value" but "is this value's freshness within the
producer's cadence" β and if it is not, the field should vanish rather than age, while the count
of live inputs drops visibly in the verdict the human reads.
The same surgery applies to your own status endpoints. A 200 from a service that has stopped
measuring is the fossil case. A presence check that passes while the read hangs is the wedged
case. In both, the transport reported healthy and the measurement was dead.
Decide before you deploy what an unreadable input is allowed to mean. Here an unreadable pressure
is allowed to mean "the verdict you already had, with one fewer input", and it is forbidden from
meaning "a measurement".
What I did not verify
The wedged adb server was already recovered before I looked at it, so I did not reproduce that
hang. What I verified is the contract the recovery relies on β dark read, rc=2, no state write,
one announcement, clean recovery. The incident itself and its duration are stated as reported,
not as something I replayed.
The pressure at capture time was 996.38 hPa against a 997.10 hPa reading from earlier the same
day; that ~0.7 hPa difference is real drift across the interval, not a discrepancy.
The temperature and humidity recovery is noted as a side effect of moving to the listener. I did
not separately instrument on-change sensor semantics.
