Decision Model Guard: Testing the Models Behind Security Decisions

A support system reads an email and decides whether to approve a refund. The email includes a sentence claiming that another employee already approved the case. That sentence is evidence supplied by the sender. If the decision model treats it as an internal approval, the sender has acquired influence over the system's decision process.

This is the problem Patronus addresses with Decision Model Guard: guardrails around classifiers, Jev-like decision models and LLM judges. Its framework combines content scanners, repeated decision checks, an application policy and a robustness audit. This article explains those mechanisms and reads one published audit critically. It was prepared with AI assistance. The reported model runs are published project results, not new measurements for this article. 1

The judge is part of the attack surface

A model that returns a label rather than a paragraph still interprets input. An email classifier, refund judge or routing model can read text about how it should classify that same text. The output may be a perfectly valid option while the reasoning implicit in the choice has crossed a trust boundary.

For an illustrative refund workflow, distinguish three inputs. The application supplies the policy. A verified internal service supplies the order status. The customer's message supplies the complaint. A claim in that message about a previous approval does not become a verified service record just because it sounds procedural. This is our example's authority model, not a measured customer incident.

Patronus' framework represents trusted application context and untrusted content separately. Its Scene builder is particularly relevant: the repository warns that manually concatenated scene strings can give a final trusted-looking action block special meaning. Application code should construct the boundary through the documented builder rather than let input authors choose block labels. Provenance must come from the application. 1

The model and the guard also have separate roles. Wolf Defender Jev scores answers to a question about an agent scene. Decision Model Guard wraps the decision process and can use other supported classifiers or judges. Changing the wrapper does not retrain the underlying model. 1, 2

Four questions before acting on an answer

The framework separates problems that a single injection score cannot resolve. Its documentation describes content scanners for suspicious claims and markers, a steering gate around repeated model calls, coverage checks for long inputs, and invariance checks for alternative representations. A policy turns their reports into an action. 1, 3

None of these observations independently grants permission to issue a refund, transmit a document or execute a tool. In our proposed deployment, the guard's action is an input to application control flow; the business service still checks the actual authorization.

Erase-and-check tests dependence on supplied text

The steering gate evaluates the original scene and variants with individual units of untrusted content removed. It then combines their answers using the declared decision type. For security questions with risky options it can retain the higher-risk result; for ordered decisions it chooses the more cautious answer; for unordered routing it can flag disagreement and route to a fallback. The exact override thresholds and policy remain configuration choices. 4

In the refund example, removing the alleged approval leaves the underlying complaint. If the answer changes from authorization to verification, that is useful evidence that the supplied claim influenced the verdict. It does not prove that the claim was false. Its authenticity requires an independent source. Nor does every changed verdict indicate an attack: removing the actual reason for a ticket can legitimately change its destination.

This explains the project's distinction between full erasure and detector-selected erasure. Selecting suspected claims avoids repeatedly removing ordinary task evidence, but a missed claim may then receive no erasure check. Full erasure examines more units and can cause more interventions on legitimate inputs. Neither choice eliminates the need to evaluate normal traffic. 1, 4

Erase-and-check also appears in research by Kumar and colleagues on adversarial prompts. Their guarantees depend on a specified token-edit threat model and safety filter. A framework using bounded sentence windows and detector-selected variants does not inherit that paper's certificate merely by using the same broad idea. 5

A coverage check must report its own limits

A long scene can contain material outside the text the model actually reads. The project's adapters have different truncation strategies; its Coverage guard checks long untrusted content in overlapping windows while retaining trusted context. Window results are combined according to the decision type. This addresses input visibility, a different problem from whether a visible instruction is classified correctly. 3

Windowing itself has a budget. The documentation sets a default maximum number of extra windows and explicitly reports partial checks or an unknown context size. Critically, the documented default incomplete_coverage_action is report. An application requiring complete inspection should deliberately select escalate and enforce that action. Adding a coverage guard without reading its report does not establish that every input was fully checked. 3, 4

Our engineering recommendation is to record the inspected extent alongside the outcome. A reassuring label with incomplete coverage should remain distinguishable from a complete decision. Also test an input long enough to exceed the configured budget. A short happy-path example cannot reveal that failure mode.

Stability checks ask a different question

Invariance re-asks the decision with different scene formats or answer wording. OptionOrder reorders the choices and maps the answers back to their original meanings. Comparing labels without this mapping would mistake a changed index for a changed decision. The guards report disagreement, and the policy chooses its treatment. 3, 4

For a legitimate stability test, the variants must preserve meaning. “Verify the return first” and “Approve the refund” are different business choices, not harmless paraphrases. Review the transformations before interpreting a changed answer as a model weakness. Conversely, consistency alone does not establish correctness: a model can repeat the same wrong answer in every format.

This distinction makes a robustness audit more informative than one aggregate attack-success figure. We want to know which perturbations change decisions, which attacks reach an unsafe target and how often the guard interrupts legitimate work. Those are separate outcomes.

Read the published Qwen audit with its denominators

The repository includes a recorded audit of Qwen/Qwen2.5-0.5B-Instruct on twelve bundled sample cases. Its configuration uses content scanners, the claim detector, a steering gate and Coverage. It omits Invariance and OptionOrder. The report records six incorrect baseline answers, so this is plainly not a demonstration of universal task competence. 6, 7

The audit implementation runs attack checks on cases the bare model originally answered correctly. Robustness and neutral-control rows measure changed answers, with different denominators. Rows therefore do not represent ten interchangeable estimates of one population-level failure rate. Here are five directly inspectable rows from the published JSON. 8

Published project run, not independently repeated here. Denominators count evaluated variants for the respective check, not independent users or deployments. Source: the repository's audit JSON and implementation. 6, 8

The first two rows demonstrate an improvement on those tested attacks. The wording row shows a remaining problem: five changed guarded answers out of twenty-one variants. The guard is not configured to address every tested failure class, and an intervention can itself affect another robustness dimension.

The neutral-control row supplies an operational cost. The guarded answer changes in 3 of 24 variants, while the guard flags 6 of 24 decisions for an action other than simple allowance. A flag and a changed answer are different measurements. Neither should be silently called a conventional classifier false-positive rate. 6

Zero observed successes also remains a result for this finite collection. The small sample, reused base cases and selected attack patterns do not establish a deployment guarantee. We checked the published arithmetic and configuration; we did not rerun inference or measure a complete refund or agent application.

A proposed deployment flow. Guard evidence and application authorization are separate decisions; partial checks and exceptions need an explicit held route.

Turn an audit into a deployment decision

Start with a decision the application actually makes. Record its question, answer options, correct labels and which options are risky or ordered. Add normal cases that quote approvals legitimately, actual claims supplied by an untrusted sender, long inputs and meaning-preserving format variants. The public evaluation format supports labeled cases and these decision types. 9

Freeze the model, tokenizer, numerical format, guard settings and case set before comparing results. Report ordinary task accuracy, unsafe-target successes, neutral interventions, incomplete coverage and model-call cost separately. These are our proposed deployment checks, informed by the project's evaluation contract. An attack-success CI threshold alone cannot enforce all of them: the documented gate excludes robustness and neutral-control outcomes from that metric. 3

Finally, exercise control flow. In the proposed application, an exception, escalation, abstention or incomplete required check must not reach an automated action. Keep the exact inspected scene with its decision and retain external service permissions independently. This is an integration requirement to test, not a result established by the published audit.

Explore the Decision Model Guard repository, its guard and policy documentation and Wolf Defender Jev. For deployment discussions, contact Patronus Protect. A concrete first step is to choose one consequential decision and test its normal, manipulated and unavailable-model routes before allowing it to trigger an action.

References

Repository sources checked 21 October 2026 at commit 9457ebd1f175a635a6e184bc3bfe8664b5c9548e; source version 0.2.0. Project documentation and recorded results are manufacturer sources.

1. Patronus Protect. Decision Model Guard, README, architecture, scene construction and limitations. Public repository.
2. Patronus Protect. Wolf Defender Jev, model card. Model interface and limitations, checked 21 October 2026.
3. Patronus Protect. Robustness: long content, invariance, calibration and the audit. Technical documentation.
4. Patronus Protect. Guards and policy. Parameters, defaults and action precedence.
5. Kumar, A., Agarwal, C., Srinivas, S., Li, A. J., Feizi, S., and Lakkaraju, H. Certifying LLM Safety against Adversarial Prompting. arXiv:2309.02705v3, 12 February 2024. Cited for its defined erasure threat model; its guarantees are not claimed for this framework.
6. Patronus Protect. Recorded Qwen audit. Raw counts and rates.
7. Patronus Protect. Qwen audit configuration. Exact enabled scanners and guards.
8. Patronus Protect. Audit implementation. Eligibility, variants and metric definitions.
9. Patronus Protect. Testing your own decision model. Labeled cases and evaluation semantics.