Boot Confirmation: Who Decides an Update Worked?

A successful install is not a successful update. For boot confirmation, use a bootloader boot counter to catch the failures where the system never reaches userspace, and one userspace health check that confirms the slot only after the device has shown it works and can still be updated. Run a hardware watchdog under both, so a hang produces a reset that the counter records. A boot counter alone is enough only when someone can physically reach a failed device.

An update framework reports that the installation succeeded. That means only that the image reached the inactive slot and its checksum matched — nothing about whether the device will boot it, or whether the software works. Boot confirmation answers those two questions: the booted system reports back to the bootloader, which then stops treating the new slot as unproven. The decision is who may make that report, and on what evidence. It goes one level deeper than my post on choosing an A/B update layout, which covered where the slots go.

The context

The SWUpdate documentation defines a successful update as four steps: the update agent runs, the device reboots, the bootloader starts the new software, and that software runs, checks itself, and declares the transaction complete. Three of the four happen after the update agent has exited. Boot confirmation has to catch two classes of failure there, and no single component sees both.

The first class never reaches userspace. A device tree that does not match the board, a panic in early init, an unresolvable root filesystem UUID, a hang while probing a bus. No userspace runs, so none of it can report anything. Only the bootloader can notice, and only if it counts.

The second class boots cleanly and is still a dead product. The system reaches a login prompt, but the radio never associates, a migration fails, or the main service restarts in a loop. The bootloader cannot see this; as RAUC's documentation notes, such problems appear only at runtime, sometimes only after days and many reboots.

The boot confirmation options

Option A — a bootloader boot counter

U-Boot implements this as Boot Count Limit (CONFIG_BOOTCOUNT_LIMIT). After a power-on reset bootcount is initialised to 1, and while upgrade_available is non-zero each reboot increments it by one. When bootcount exceeds bootlimit, U-Boot executes altbootcmd instead of bootcmd. Two details matter on a headless device: if bootlimit is undefined the feature is disabled entirely, and if altbootcmd is undefined U-Boot drops into interactive mode and stays there. Linux must then reset bootcount to 0 to allow further boot cycles.

raghu@techveda.org:~$ fw_printenv bootcount bootlimit upgrade_available
bootcount=1
bootlimit=3
upgrade_available=1
raghu@techveda.org:~$ sudo fw_setenv upgrade_available 0
raghu@techveda.org:~$ sudo fw_setenv bootcount 0

RAUC's U-Boot backend layers its own variables on this, BOOT_ORDER and BOOT_<bootname>_LEFT, driven by a boot script. Barebox instead provides a maintained framework, bootchooser, keeping remaining_attempts and priority per target in an atomic, redundant backend — which is why RAUC's documentation recommends it.

A counter is small, runs before your code, and does not depend on the application. But it counts boots, not correctness: a device that boots with a dead application passes. The state must also be written atomically — on U-Boot, a redundant environment (CONFIG_SYS_REDUNDAND_ENVIRONMENT, spelled that way upstream) outside the bootloader partition — or a power cut mid-write leaves the device unable to decide anything.

Option B — a userspace health check

Here the running system decides. RAUC provides rauc status mark-good and mark-bad, which reach the bootloader through its bootloader interface. The documentation suggests running mark-good from a systemd unit ordered after the services that actually matter, not at the end of boot.

SWUpdate expresses the same idea through a persistent variable, ustate. It is set to 1 (INSTALLED) after an update, meaning the new software is under test; the bootloader initiates a fallback and sets it to 3 (FAILED) if that software is not running; userspace resets it to 0 once confirmed. The documentation is explicit that resetting it is project specific, and can be the last action after the application has checked itself.

This is the only mechanism that can judge whether the product works, and you choose the evidence. But it reports nothing if it never runs, so alone it leaves the first failure class uncovered. Too permissive, it confirms broken builds; too strict, it rolls back a healthy device. The SWUpdate best-practice guide warns of a deeper risk: tying the check to your application makes the application a dependency of updatability.

Option C — both, over a hardware watchdog

A counter only increments if the device reboots, and a system that hangs does not reboot. A hardware watchdog converts a hang into a reset. RAUC states this directly: the system should use a hardware watchdog during boot, and the bootloader should treat watchdog resets as failed boots.

On Linux the watchdog starts when /dev/watchdog is opened, then must be pinged by a write or by WDIOC_KEEPALIVE. If a process closes the device without first writing the magic character V, a Magic Close driver keeps it running; CONFIG_WATCHDOG_NOWAYOUT removes the ability to stop it at all. Under systemd, RuntimeWatchdogSec= programs it and systemd contacts it at least once every half of that interval — but it defaults to 0, meaning off. For coverage before Linux starts, U-Boot has CONFIG_WATCHDOG_AUTOSTART (on by default, except ARCH_SUNXI, ARCH_STM32MP and ARCH_SNAPDRAGON), and barebox uses boot.watchdog_timeout.

The decision

For a connected product you cannot physically reach, use all three layers with different jobs. The watchdog detects that progress stopped and forces a reset. The boot counter detects repeated failed attempts and switches slots. One health check decides the new software genuinely works, and confirms.

Three rules make that practical. Define the health check as the smallest set of conditions meaning the device performs its function and can be updated again, including the update agent's reachability. Keep the confirmation path independent of the application. And set bootlimit to at least 3, so one transient failure does not cause a rollback.

The boot counter alone is the right choice in two situations. If a technician can reach and reflash a failed device, a dead application and a dead boot are the same service call. And if the application team cannot commit to a stable health signal, a check whose meaning changes every release produces rollbacks nobody can explain.

A userspace check alone is the right choice when the vendor boot chain is locked and you cannot add counting logic. Put the fallback decision in an initramfs or an early service that runs before the application.

Consequences

The health check becomes release-critical code. It ships inside the image it validates, so version it with that image, and test the rollback path each release with a bundle known to fail.

You acquire a timing budget. The watchdog timeout must exceed the worst-case boot, including a filesystem check and any first-boot migration; otherwise a slow but healthy boot is indistinguishable from a broken one. U-Boot's CONFIG_WATCHDOG_TIMEOUT_MSECS defaults to 60000 with per-architecture overrides — measure it on your hardware.

Rollback is not free at the data layer either. Returning to older software leaves newer data on the writable partition, so a schema written by the new version must still load on the old one — a constraint belonging to the application, not the update framework. You also need to explain a rollback afterwards, so record the watchdog boot status (WDIOC_GETBOOTSTATUS, WDIOF_CARDRESET) and ustate or BOOT_<bootname>_LEFT.

Finally, one boot confirmation failure mode is easy to miss. If no device in your fleet has ever rolled back, the likely explanation is not that every release was perfect; it is that the health check tests nothing. Keep a rescue system regardless, because a double-copy setup still has failure modes affecting both copies.

Key takeaways

A successful installation is not a successful update; boot confirmation decides whether the new slot is kept.
A boot counter catches failures that never reach userspace; a health check catches a device that boots but does not work. Neither covers the other.
A hardware watchdog turns a hang into a reset the counter can observe; under systemd it is off by default. Keep the health check small, and always include the ability to receive another update.

Frequently asked questions

What is boot confirmation?
It is the step where a newly booted system reports back to the bootloader that it is working, so the bootloader stops treating the new slot as unproven. Until then, returning to the previous slot stays possible.

Is a bootloader boot counter enough on its own?
Only when someone can physically reach a failed device, or when the application cannot provide a stable health signal. A counter measures boots, not correctness, so a device that boots with a dead application passes it.

What should the userspace health check actually test?
The smallest set of conditions meaning the device performs its function and can still receive another update. Include the update agent's reachability, so a bad release stays replaceable.

Why is a hardware watchdog needed if the bootloader already counts boots?
A counter only increments when the device reboots. A system that hangs does not reboot, so the failure is never counted. The watchdog forces the reset that makes it visible.