Give Your Coding Agent a Deterministic Vulnerability Oracle

AI agents can write code, run tests, inspect dependencies, and open pull requests. But when they encounter a vulnerable package, their security reasoning often collapses into a web search, an opaque API score, or whatever the model remembers from training.

The problem is not simply missing vulnerability data. It is the contract between the agent and its harness—the automation layer that supplies tools, executes commands, and interprets results. If “no match” silently becomes “safe,” stale intelligence looks current, or a network failure resembles an empty result, the agent can produce a confident answer without trustworthy evidence.

I designed VulnGraph around a different premise: turn continuously changing vulnerability intelligence into a deterministic local primitive. vulngraph-data compiles upstream security sources into verified, content-addressed snapshots. vulngraph-cli installs those snapshots and checks CVEs, package versions, or entire lockfiles offline.

The result is a security tool shaped for both humans and machines: an accepted snapshot and target produce the same verdict; every verdict carries typed evidence; stale data fails explicitly; and unknown is never misrepresented as clean.

This article explains the design decisions behind that contract and shows how to embed VulnGraph into agent and harness workflows.

The dangerous gap between “not found” and “safe”

Suppose an agent is reviewing a dependency update. It needs to answer a seemingly simple question: Is this version safe to ship? A useful answer depends on distinctions that are easy for a harness to erase.

These are not cosmetic labels. They lead to different agent actions: proceed, investigate, refresh the data, or stop the workflow. Once a harness flattens them into a Boolean, the model cannot recover the missing meaning through better prompting.

Here is the distinction in practice. One command checks an exploited CVE, a real package version, and a package the snapshot has never observed:

$ vulngraph check CVE-2024-4577 npm:lodash@4.17.15 npm:no-such-package@1.0.0

  CVE-2024-4577
  verdict: ACTIVELY EXPLOITED  (confidence 0.99)
  action:  patch now
  reasons: CRITICAL_SEVERITY, HIGH_EXPLOIT_PROBABILITY,
           KNOWN_EXPLOITED, PUBLIC_EXPLOIT, ...
  cvss:    9.8
  epss:    100.0%
  kev:     listed

  npm:lodash@4.17.15
  verdict: PROOF OF CONCEPT  (confidence 0.96)
  action:  prioritize
  package: npm:lodash — 6 CVE(s) affect this version

  npm:no-such-package@1.0.0
  verdict: UNKNOWN  (confidence 0.00)
  action:  investigate
  reasons: NOT_OBSERVED

  snapshot: sha256:41ee5fbbf58f0a46b99234af89c5388e5f27e0dcd3dbd52298bcd2c2d87ca90e

The output does not ask the agent to infer policy from prose. It supplies a disposition, an action, stable reason codes, confidence, and the exact snapshot that produced the result. The human-readable rendering is useful in a terminal; --json exposes the same distinctions through a versioned machine envelope.

Split changing intelligence from deterministic execution

Vulnerability intelligence changes every day. Agent execution still needs to be reproducible. I separated those concerns instead of allowing every check to fetch and reconcile live data.

CVE List V5 · EPSS · CISA KEV · exploit sources · ATT&CK · OSV · …
                              │
                              ▼
                    vulngraph-data
              fetch → normalize → build → verify
                              │
                              ▼
            immutable data-YYYYMMDD release
       manifest + checksums + content snapshot_id
                              │
                  vulngraph update
                              ▼
             verified local snapshot
                              │
          vulngraph check --offline --json
                              ▼
                  agent or harness

The vulngraph-data repository owns the nondeterministic edge: retrieving bulk publications from upstream sources and reconciling them into one graph. Its output is deterministic. Identical inputs produce byte-identical semantic files, and their hashes produce a snapshot_id that names the graph by content. If a daily build has no semantic change, there is no new release.

The release artifact is the distribution boundary. A consumer does not need the raw source collection or the machinery that built it. It receives a fixed database archive and a manifest recording the snapshot identity, file hashes, source freshness, graph format, and engine revision.

The vulngraph-cli owns the deterministic side. vulngraph update downloads a release into a staging area, verifies the archive checksum and every file hash, recomputes the snapshot identity, checks format compatibility, sanity-opens the graph, and only then activates it. Failed updates cannot replace the last verified snapshot.

After activation, checks do not touch the network. The snapshot becomes an explicit input to the decision:

verdict = policy(snapshot_id, target)

There is no wall clock, random value, model call, or remote response inside that function. Once a snapshot passes the freshness and integrity preflight, the same snapshot and target give a developer laptop, a CI job, and an agent sandbox the same verdict.

A harness workflow that does not guess

A reliable integration has three phases. Only the update phase needs network access; discovery and checks are stable interfaces that the harness can validate.

1. Discover the contract

Do not make the agent reverse-engineer help text. Let the harness inspect the machine-readable capability document and the bundled JSON Schemas:

vulngraph --json capabilities
vulngraph schema command
vulngraph schema observation
vulngraph schema status

capabilities declares the commands, output schema versions, exit codes, and two behavioral guarantees: checks are offline and deterministic. A harness can validate support during startup instead of discovering an incompatibility halfway through a run.

2. Refresh and preflight outside the restricted step

Install or refresh the snapshot in a setup job that is allowed to use the network:

vulngraph update
vulngraph --json status

Then mount or cache the resulting VulnGraph home directory inside the agent environment. This makes network access a controlled data-provisioning concern instead of an ambient capability available during every security decision.

status reports whether a snapshot is installed, its identity, its integrity, and its freshness. A dataset older than fourteen days is not merely accompanied by a warning: check refuses to answer and exits with code 4.

3. Check the artifact the agent is actually changing

The check command accepts individual CVEs, package coordinates, and dependency files. Pointing it at a lockfile expands every dependency and applies the same version-range policy:

vulngraph --offline --json check package-lock.json > vulngraph.json

The CLI recognizes lockfiles from npm, Yarn, pnpm, Cargo, Bundler, Poetry, pip, Go, Composer, Maven, Gradle, and Pipenv. The abridged JSON response below shows the stable envelope:

{
  "schema": "vulngraph.command.v1",
  "command": "check",
  "ok": true,
  "partial": false,
  "snapshot_id": "sha256:41ee5fbb…",
  "data": [
    {
      "schema": "vulngraph.observation.v1",
      "target": { "type": "package", "value": "npm:lodash@4.17.15" },
      "verdict": {
        "disposition": "proof-of-concept",
        "confidence": 0.96,
        "action": "prioritize",
        "reason_codes": [
          "ATTACK_MAPPED",
          "HIGH_SEVERITY",
          "MULTISOURCE_CORROBORATION",
          "PUBLIC_EXPLOIT",
          "SEVERITY_SCORED",
          "VERSION_IN_AFFECTED_RANGE",
          "WEAKNESS_CLASSIFIED"
        ]
      }
    }
  ],
  "warnings": [],
  "errors": [],
  "metrics": { "elapsed_us": 781250 }
}

The important separation is that VulnGraph observes; it does not gate. A successful check exits with 0 even when it finds an actively exploited vulnerability. Exit codes describe whether the tool completed reliably; the harness applies organizational policy to the returned dispositions.

This split keeps probabilistic reasoning in the right place. The harness deterministically decides which states require a stop, review, or escalation. The agent then uses the evidence to explain the finding, inspect reachability, propose the smallest safe upgrade, and communicate the trade-off to a human.

A minimal Node.js harness

The following wrapper turns the versioned VulnGraph envelope into one of five local workflow decisions. It does not ask the model whether an operational failure is acceptable, and it never treats an unrecognized disposition as approval.

import { spawnSync } from "node:child_process";

const binary = process.env.VULNGRAPH_BIN ?? "vulngraph";
const target = process.argv[2] ?? "package-lock.json";
const run = spawnSync(
  binary,
  ["--offline", "--json", "check", target],
  { encoding: "utf8" }
);

let envelope;
try {
  envelope = JSON.parse(run.stdout);
} catch {
  throw new Error(`vulngraph returned invalid JSON: ${run.stderr}`);
}

// Exit codes describe tool reliability, not vulnerability severity.
if (run.status === 4) {
  throw new Error("VulnGraph snapshot is stale; refresh before continuing");
}
if (run.status !== 0 || !envelope.ok) {
  throw new Error(`VulnGraph failed: ${JSON.stringify(envelope.errors)}`);
}
if (envelope.schema !== "vulngraph.command.v1") {
  throw new Error(`Unsupported schema: ${envelope.schema}`);
}

const policy = new Map([
  ["actively-exploited", "block"],
  ["weaponized", "block"],
  ["proof-of-concept", "review"],
  ["scored", "review"],
  ["recorded", "monitor"],
  ["not-affected", "proceed"],
  ["unknown", "investigate"]
]);

const decisions = envelope.data.map(({ target, verdict }) => ({
  target: target.value,
  disposition: verdict.disposition,
  decision: policy.get(verdict.disposition) ?? "investigate",
  action: verdict.action,
  confidence: verdict.confidence,
  reasons: verdict.reason_codes
}));

const evidence = {
  snapshotId: envelope.snapshot_id,
  decisions
};

console.log(JSON.stringify(evidence, null, 2));

if (decisions.some(({ decision }) => decision === "block")) {
  process.exitCode = 10; // This is our harness policy, not a VulnGraph exit code.
}

The evidence object is what I would place in the agent's context. The prompt can remain narrow:

Review the attached vulnerability evidence for this dependency change.

- Never describe `unknown` as safe or clean.
- Preserve the snapshot ID in your report.
- Explain the reason codes in plain language.
- Check whether each affected dependency is reachable from the changed code.
- Propose the smallest compatible upgrade, but do not edit files until approved.

The deterministic layer establishes what was observed and which policy branch applies. The agent contributes the work it is good at: repository-specific investigation, explanation, remediation planning, and communication.

Design lessons for agent-facing security tools

Building the graph was only half the problem. Making it safe for automation required treating the interface as part of the security model.

1. Make ignorance a first-class result. unknown must survive every layer from data lookup to terminal output to agent context.
2. Return evidence, not just a score. A verdict should carry stable reasons and source observations that a human can audit.
3. Separate update time from decision time. Network retrieval belongs in a controlled provisioning phase; checks should be local and reproducible.
4. Identify the data behind every answer. A content-derived snapshot ID turns “what did the scanner know?” into an answerable question.
5. Let tools describe their contract. Capabilities, schemas, freshness, and exit semantics should be machine-readable.
6. Keep gating policy outside the observation tool. Different repositories have different risk tolerances. The harness should own that policy explicitly.

An agent does not become trustworthy because it receives more tokens or a longer system prompt. It becomes more trustworthy when its environment preserves distinctions, constrains side effects, and supplies evidence through contracts that cannot silently change meaning.

That is the role I designed VulnGraph to play: not an autonomous security authority, but a fast, local, auditable source of vulnerability evidence that gives agents and their harnesses something solid to reason from.

• Build and release pipeline: copyleftdev/vulngraph-data
• Offline CLI: copyleftdev/vulngraph-cli

Disclosure: I designed VulnGraph. This article and its cover were created with AI assistance. I reviewed the technical claims against the linked repositories and live CLI output.

If this open-source work is useful to you, you can support its continued development: