A Green Lint Report Is Not a Working Policy

On August 10 we filed issue #3661 in Microsoft's Agent Governance Toolkit repository: the policy linter accepted rules whose conditions can never match, and the evaluator silently applied its default action when they didn't. On October 20 a maintainer answered with a full audit of the problem. Their audit found more failure modes than our report listed, laid out where the fix will and will not land, and confirmed one uncomfortable fact: the published packages still in circulation accept never-match rules and default silently. This is what the thread says, and why it matters beyond one repository.

What we reported

The toolkit's agt lint-policy is described as linting policy files "for common mistakes." We found the case it misses: governance is enabled, the policy loads, the rule is syntactically valid, and it still matches nothing that can ever occur.

Three variants, reproduced against the published PyPI packages (agent-governance-toolkit 4.1.0, agent-os-kernel 3.7.0, clean venv):

• a deny rule on toolname (a typo for tool_name) never fires;
• a deny rule with an empty in: [] value set never fires;
• a deny rule shadowed by a higher-priority blanket allow never fires.

All three pass the linter with "No issues found." The evaluator returns allowed=True with the reason "No rules matched; default action applied" - no exception, no warning. Under default_action: deny this fails closed and costs a confusing refusal. Under default_action: allow it fails open, and allow is the default in the README quickstart policy, which is named production-policy.

The reason we filed it: this defect is invisible to review. A missing policy file shows up in doctor output and dashboards. A rule that reads correctly and covers nothing shows up nowhere - the file is present, the linter is green, the audit log shows ordinary allowed actions, application tests pass. The first honest signal is a denial that never happened and nobody was waiting for.

What happened in the thread

A community contributor reproduced the issue on current main within a day and mapped the complication: the linter our report targeted no longer exists on main, replaced in the ACS migration, while the failing legacy evaluator still shipped on PyPI. In September, PR #4130 made the one-way v4-to-v5 migrator refuse the provably unsatisfiable rules: empty in lists, empty or conditions, contradictory and siblings. In October the issue was marked stale; we rechecked against the published packages, confirmed the original reproduction still fails open, and noted that #4130 fixes the migrator path, not the 4.1.0 evaluator and linter that pip users still install. The issue stayed open.

The maintainer's audit

On October 20, maintainer MohammadHaroonAbuomar posted the project's decision. Credit where it is due: instead of triaging our three cases, they ran the current source against the whole class and listed every variant they found, including ones absent from the thread:

• A misspelled field with any operator is accepted and compiles to a Rego accessor that can never match.
• The standard v4 context key tool_name compiles to input.snapshot.tool_name, but the ACS snapshot carries the name under tool_call.name. A correctly spelled v4 deny migrates to a rule that never fires, with a green migration report.
• A deny shadowed by a higher-priority allow is accepted.
• An allow rule with not_in: [] is accepted and always fires.
• A top-level default_action: deny is dropped, and the generated bundle defaults to allow.

Separately, they noted that the README quickstart on main now uses a different (agentmesh) string-condition engine, again with default_action: allow, where an empty in list or a misspelled path returns false with no log.

Read that second case twice. Even a user who did everything right - correct field name, working deny rule, successful migration - ends up with a policy that never fires, because the field was renamed between policy versions and nothing checked. The failure survived a correct migration of a correct rule.

The fix, and its limits

The maintainers will not patch the 4.x evaluator; its source no longer exists in the repository. The plan, as they stated it:

1. Migrator (agt-policies, to ship as 5.2.0 on PyPI): map v4 context keys to snapshot paths (tool_name to tool_call.name); refuse, with errors naming the construct, unknown fields, unknown top-level keys, an allow rule with not_in: [], and a rule provably covered by a higher-priority opposite-action rule; honour default_action: deny by emitting a default-deny bundle and refuse any other value. Tests must execute the generated bundle with OPA against a real pre_tool_call snapshot and prove that a mapped tool_name deny actually fires.
2. Lint and docs (agent-compliance): the README quickstart moves to default_action: deny; agt lint-policy identifies a legacy v4 document as such and points at agt migrate v4-to-v5; a LIMITATIONS entry will state that the published 4.x packages accept never-match rules and default silently.
3. agentmesh engines, as a separate issue: raise at load time on in [], on not_in: [] in an allow rule, and on unrecognized syntax in deny rules.
4. Later: the ACS validator runs opa check --strict and verifies that a manifest's query names a rule the bundle defines.

Two honest caveats. First, this is a plan and two invited PRs, not a shipped release: as of today none of it has landed in a package you can install. Second, by the maintainers' own decision, the 4.x packages already published keep their behavior; the LIMITATIONS entry will document that rather than change it. If you installed agent-governance-toolkit 4.1.0 from PyPI, no patch is coming to that line.

A green report is not enforcement

The deeper lesson is not about one toolkit, and the maintainers' own test requirement states it best. The check that would have caught every one of these cases is the one they are now writing: execute the generated policy against realistic input and prove the deny fires. Everything before that was a statement about the policy: the file parses, the schema validates, the linter is green, the migration report is green. None of those statements is the policy working.

We have argued the same line from other angles: an instruction is not an authorization, and a public key is not permission. A lint report belongs on the same list. A policy file that reads correctly is a statement of intent; enforcement is what happens when a real request arrives. The gap between them is where every one of these bugs lived - a typo, an empty list, a renamed field, a dropped default. Each one individually boring, each one silently converting "deny" into "allow by default."

The finding direction matters more than the finding. The maintainers found more cases in one audit pass than two months of thread had listed, not because the cases were hidden, but because nobody had run the source against the class of defect. That is what a real review buys you: not a greener report, but someone executing the thing.

What to check if you run v4 policies today

• On the published 4.x packages, your linter cannot see this class of defect. Check every deny rule's field names against the evaluator's context keys by hand, and never ship an empty in or not_in list.
• After any v4-to-v5 migration, execute the generated bundle with OPA against a real pre_tool_call snapshot and prove your denies fire. A green migration report is not evidence.
• Whatever the version: prefer default_action: deny, and treat "No issues found" as the start of verification, not the end of it.

Sources: issue #3661 and the maintainer's October 20, 2026 response (github.com/microsoft/agent-governance-toolkit/issues/3661), and PR #4130 (github.com/microsoft/agent-governance-toolkit/issues/4130). This article describes only what those public sources state. The fix plan quoted above is announced, not yet released; the published 4.x packages remain affected per the maintainers' own stated LIMITATIONS plan.