By Micky Irons, founder of Mickai.
The regulator now wants the decision, not the disclaimer
In 2026 the National Association of Insurance Commissioners moved its model-evaluation work from principle to practice. The NAIC AI evaluation pilot asks a blunt question of any carrier using machine learning in underwriting and claims. Show us the decision. Not the policy that governs the model, not the fairness attestation signed once a year, but the specific path that turned one applicant into one price, on one date, under one version of the model.
That shift is larger than it looks. For a decade insurers answered AI scrutiny with documentation: governance frameworks, bias-testing summaries, vendor questionnaires. The pilot treats those as table stakes and reaches past them to the underwriting decision itself. The implicit standard is replay. If an examiner picks a declined application from eighteen months ago, can the carrier reconstruct exactly what the model saw, which features moved the outcome, which version of the weights was live, and who approved the override? Most stacks cannot, and the gap is not a paperwork problem. It is an architecture problem.
The pilot moves the burden of proof from the framework to the individual decision. Themis no longer weighs the policy. She weighs the case.
Why documentation fails the replay test
A governance document describes intent. A replay describes what actually happened. The two diverge the moment a model is retrained, a feature pipeline is patched, or an underwriter exercises discretion. Carriers routinely retrain quarterly and ship feature changes weekly. By the time an examiner asks about a decision from last spring, the model that made it may no longer exist in any runnable form, the training data may have been rotated, and the feature store may have been migrated.
So the honest answer to many examination questions today is reconstruction, not retrieval. Teams rebuild an approximation of the decision and present it as the decision. That is defensible until it is challenged in litigation or a market-conduct exam, at which point the difference between what the model did and what the team believes it did becomes the whole case. The pilot is quietly forcing carriers to confront a fact they have managed to avoid. An unauditable decision is an uninsurable liability.
The four things a real replay must carry
- The exact model artefact and version that scored the application, not a retrained successor.
- The full feature vector as it existed at decision time, including derived and third-party inputs.
- Every human action layered on the model output: overrides, manual referrals, and the identity behind each.
- A tamper-evident timestamp proving the record was sealed when the decision was made, not assembled later for the examiner.
Memory is only evidence if it was sealed at the moment it formed. A record assembled after the question is asked answers a different question.
Sealing the decision at the moment it is made
This is the tension the pilot exposes, and it is exactly the tension a Sovereign Intelligence Operating System is built to resolve. Mickai runs its fifty specialised brains (twenty-five domain and twenty-five operational) on the carrier's own hardware, fully offline-capable, which means the model, the feature pipeline, and the decision logic live inside one auditable boundary rather than scattered across vendor APIs the carrier cannot inspect.
Inside that boundary, every consequential action is written to the Open Audit Record. The OAR seals each underwriting decision and signs it with FIPS 204 ML-DSA-65, the published NIST post-quantum signature standard. Mickai did not invent that standard. It adopts it, which matters to a regulator who wants the cryptography to be recognised rather than bespoke. The signature binds the model version, the feature vector, the output, and any human override into a single record at the instant the decision is made. Replay stops being a reconstruction exercise and becomes a retrieval. The examiner asks for a decision. The carrier returns the sealed, signed record of that decision, byte for byte.
Each decision is forged into the record as it happens. The chain is built in the moment, not annealed afterward to fit the audit.
Permanence the carrier does not have to be trusted on
A signed record still raises one question an examiner is right to press. How do we know it was not re-sealed yesterday with a backdated timestamp? Mickai answers that with Pantheon, its own sovereign, Bitcoin-anchored Layer 1. Pantheon takes a hash commitment of the record and anchors it to Bitcoin, fixing the record in time against the most expensive clock in existence.
The distinction matters and is worth stating plainly. Pantheon does not move Bitcoin and is not a Bitcoin Layer 2. Anchoring is not spending. Only a hash, a fingerprint of the sealed decision, is committed, so the underwriting record itself never leaves the carrier's boundary while its existence and timing become independently verifiable. A market-conduct examiner no longer has to trust the carrier's word that a record predates a dispute. The proof sits on a public chain the carrier does not control.
This is also why the regulatory posture is evidence rather than marketing. The architecture behind this approach sits within Mickai's portfolio of 101 filed UK patent applications, around 2,234 claims, owned by Mickai LTD, with Micky Irons named as inventor. The point is not the count. The point is that sealed, signed, time-anchored decisioning is a designed system, held privately by its founder, not a slide in a compliance deck.
A hash of the decision is anchored to Bitcoin. The record stays inside the carrier. Its place in time becomes something no party can quietly revise.
What carriers should do before the pilot becomes the rule
Pilots have a way of hardening into examination standards. The carriers that struggle in two years will be the ones still answering replay requests with reconstructions. The ones that pass will have moved the auditability requirement down into the substrate, where the decision, the model version, the human override, and the cryptographic seal are captured together and proven in time rather than reassembled on request.
The NAIC pilot is not really a test of fairness frameworks. It is a test of whether the underwriting decision can replay. For most stacks that is a hard retrofit. For a sovereign operating system that seals and signs every consequential action as it happens and anchors its permanence to Bitcoin, it is simply how the decision was recorded the first time.
Written by Micky Irons. Originally published at https://mickai.co.uk/articles/naic-ai-evaluation-pilot-the-underwriting-decision-must-replay. More from Micky Irons and Mickai at mickai.co.uk.
