LLM Supply Chain Security Checklist: Lock Down Agents [2026]

Originally published at kunalganglani.com — read it there for inline code, hero image, and live links.

If you want a llm supply chain security checklist you can actually wire into CI, this post gives you one. The prerequisite that trips teams up is simple. You have to treat your agent’s “stuff” (model files, MCP servers, tool plugins, prompt packs) as versioned artifacts with provenance, not as “config” that changes whenever someone merges a PR.

I built this site’s multi-agent publishing pipeline with deterministic gates and idempotent publishing keys. One thing I learned the hard way is that gates beat good intentions. In one incident, rewriting slugs on live URLs burned 907K impressions of link equity. Same pattern here. If you allow “unreviewed, unpinned updates” anywhere in the chain, you are volunteering for a security incident. The attacker just gets better branding than your average feature release.

Here’s the threat model that matters, and the CI/CD controls I’d put in front of any agent that can touch real systems.

What is LLM supply chain security?

LLM supply chain security is protecting and verifying every external artifact an LLM-powered app or agent depends on. Not just code libraries. Also model weights, tool/plugin packages, Model Context Protocol (MCP) servers, and prompt bundles.

The job is to stop tampering, typosquatting, malicious updates, and unauthorized build or publish steps by enforcing provenance, signatures, pinning, and runtime least privilege.

The “dependencies” of an AI agent (beyond code libraries)

Classic appsec assumes your dependencies are things like npm packages, Docker images, and managed services. Agentic systems add a messier layer.

These are the four dependency classes I see most teams hand-wave until something breaks.

1) Model providers and model artifacts (weights, adapters, tokenizers)

Even if you call a hosted API, the “dependency” is still real. Your agent is trusting:

• the model identity (which model, which version)
• the serving stack (tool calling, safety filters, caching)
• the provider’s update policy

If you run a local LLM or ship model weights, it gets sharper:

• weight file format (.safetensors vs pickle-based formats)
• weight provenance (who produced it, from what source)
• adapter provenance (LoRA/QLoRA layers)

Hugging Face is explicit that safetensors is “a … format for storing tensors safely (as opposed to pickle).” That’s not some academic nit. Unsafe deserialization is one of the oldest “how did this become RCE” footguns in ML.

2) Tool plugins (SDKs, connectors, “agent tools”)

This is your new npm install moment, except worse, because the plugin often arrives bundled with:

• a credential (token, API key, OAuth client)
• a network path (outbound egress)
• a privileged operation (write access, deletion, money movement)

A compromised tool plugin is basically a compromised employee laptop. It can exfiltrate and mutate.

3) MCP servers

Model Context Protocol is quickly becoming the USB-C for agent tools. The MCP roadmap makes the direction obvious: more ecosystem, more servers, more enterprise hardening.

The official David Soria Parra + Den Delimarsky roadmap post (Aug 22, 2026) calls out “enterprise-ready security” and details security-focused authorization work in the 2026-07-28 spec release.

Translation. MCP servers are going to be everywhere. Which means they are going to be the dependency surface you forgot to inventory.

4) Prompt packs (prompt bundles, system prompt templates, tool instructions)

This one makes me a little angry, because it’s so avoidable.

Teams treat prompt packs like:

• “just text”
• “safe to hotfix in prod”
• “not worthy of review”

But prompts are executable policy. They define tool access patterns, escalation logic, and what data is “allowed” to be pulled into context. A malicious prompt update is a malicious code deployment. It just slips through with fewer guardrails.

If you’re building AI agents, prompts are dependencies.

Top compromise paths: models, MCP servers, tool plugins, prompt packs

When you threat model agent toolchains, the compromise paths look boringly familiar.

Good. We already know how attackers win in supply chains.

Model weights

• Typosquatting / impersonation: “official-looking” model repo with a near-identical name.
• Malicious update: same repo, new commit, new weights. Your pipeline auto-pulls “latest”.
• Unsafe deserialization: loading pickle-based formats in a process that has network and secrets.
• Backdoored weights: model behaves normally until a trigger phrase appears.

Concrete control: block unsafe formats. If your pipeline sees pickle-based model artifacts, fail the build. Prefer .safetensors where possible.

MCP servers

• Server substitution: DNS/registry compromise points your agent to a different MCP endpoint.
• Capability confusion: server claims it offers harmless tools but actually exposes destructive ones.
• Auth bypass or weak auth: bearer tokens copied from logs, dev tokens used in prod.
• Egress abuse: server becomes a data exfiltration tunnel.

Concrete control: treat MCP servers like any third-party service. Authenticate them, scope them, and constrain network egress.

Tool plugins

• Dependency confusion: internal package name resolves to public registry package.
• Malicious transitive dependency: plugin looks clean; its dependencies are not.
• Credential harvesting: plugin captures tokens it’s given and ships them out.

Concrete control: pin versions. Don’t allow “floating” tool versions in production agents.

Prompt packs

• Prompt backdoor: hidden instruction (“if user says X, exfiltrate Y”).
• Indirect prompt injection amplifier: prompt includes “always trust retrieved content” patterns.
• Secret leakage: prompts accidentally include API keys, test creds, internal URLs.

Concrete control: prompt diffs and approvals. If prompts can change without review, you’re doing production changes without change management.

The CI-ready LLM supply chain security checklist (12 gates)

This is the part you can copy into a ticket.

If you only do 3 things, do items 1, 2, and 6.

1. Inventory agent dependencies: model, tool plugins, MCP servers, prompt packs. No inventory means no security.
2. Pin everything: container images by digest, packages by exact version, prompt packs by commit SHA.
3. Require signatures for artifacts: containers, prompt bundles, plugin tarballs.
4. Verify signatures in CI: fail builds on unsigned/untrusted artifacts.
5. Generate and verify provenance for build outputs (who built it, how).
6. SBOM required for plugins/tools and server images. Fail builds if missing.
7. Block unsafe model formats (pickle-based). Allowlist .safetensors and known-safe loaders.
8. Prompt pack review workflow: mandatory diff + approval + secrets scan.
9. Tool least privilege: scoped tokens, per-tool allowlists, bounded arguments.
10. Network egress controls: default-deny outbound from tool runners and MCP servers.
11. Runtime logging for tool calls: tool name, args hash, caller, response size, latency.
12. Revocation and rollback plan: ability to yank a compromised dependency in < 30 minutes.

You’ll notice I didn’t say “write a policy doc.” Policies don’t stop builds. Gates do.

Here’s what those gates map to.

Provenance for prompts/tools/models: SLSA, in-toto, Sigstore

Most teams hear “provenance” and think it’s paperwork. It’s not. It’s cryptographic receipts.

SLSA: the maturity ladder for builds

SLSA (Supply-chain Levels for Software Artifacts) is explicitly a “checklist of standards and controls to prevent tampering, improve integrity, and secure packages and infrastructure,” with four levels of increasing assurance, per the OpenSSF SLSA working group.

The way I apply SLSA to agent toolchains is pretty simple:

• treat model weights, prompt packs, and MCP server images as “artifacts”
• treat your prompt build pipeline as a “build”
• generate provenance that ties an artifact back to its source, builder identity, and build steps

in-toto: verify the steps and the actors

The cleanest sentence on in-toto’s site is basically your justification. in-toto is designed to ensure integrity “by making it transparent to the user what steps were performed, by whom and in what order.”

That maps perfectly to prompt packs:

• Step 1: prompt authored in repo
• Step 2: reviewed by CODEOWNERS
• Step 3: scanned for secrets
• Step 4: bundled + signed

If any step is skipped, or performed by the wrong identity, verification fails.

Sigstore cosign: signing and verification as a CI enforcement point

Cosign is the practical tool I reach for because it’s widely adopted and plugs into CI cleanly. The Sigstore docs frame Cosign as the tool for signing and verifying, and it supports not just signatures but also in-toto attestations (you can see this in the Cosign docs navigation at Sigstore Cosign).

Your goal is not “we signed it once.” Your goal is “CI refuses to deploy it unless it verifies.”

Prompt packs: how I’d inventory, diff, approve, and sign them

If you only take one weird idea from this post, take this: prompt packs should ship like code.

Here’s a workflow that has worked for me when building deterministic pipelines.

Treat prompts as build inputs, not runtime strings

• Prompts live in a repo, not in a database field that anyone can edit.
• Prompts have versions (tags) and changelogs.
• Prompts are bundled into a prompt pack artifact during CI.

Concrete number: set a hard rule like “0 runtime edits to production prompts.” Every change goes through PR.

Diff prompts like you diff code

Prompt diffs are tricky because whitespace and reformatting can hide meaning changes.

Controls I like:

• a “normalized diff” view (strip trailing whitespace, collapse runs of spaces)
• highlight changes to tool instructions (those are high-risk lines)
• require at least 1 security reviewer for any changes that mention credentials, network, or tools

Scan prompt packs for secrets

Prompt bundles love to accumulate:

• internal URLs
• test tokens
• “temporary” credentials

If you already run gitleaks, reuse it here. I wrote a full setup for this in gitleaks + pre-commit + CI. Don’t invent a new scanning stack for prompts.

Sign prompt packs and verify at runtime

Signing prompts sounds overkill until the first time:

• someone hotfixes a production prompt in a panic
• an attacker lands a malicious change in a prompt repo
• your agent starts doing “creative” things with tools

The runtime check can be as simple as: “does this prompt pack’s signature verify against our trusted identity?” If not, the agent refuses to start.

Least privilege for agent tools: allowlists, scoped tokens, sandboxing, egress

If your agent can call tools, it’s a distributed system with an untrusted planner. Act accordingly.

Start with an allowlist by capability, not by name

“Allowed tools: github, slack, jira” is useless. Each of those hides dozens of operations.

Instead:

• allow specific operations (read issue, comment, open PR)
• deny destructive operations by default (delete repo, rotate secrets, transfer ownership)
• put a human-in-the-loop gate on money movement or deletions

If you want patterns, I’ve written these out in 10 HITL tool approval patterns for AI agents.

Scope tokens per tool and per environment

Concrete numbers that matter:

• dev tokens expire in 24 hours
• prod tokens expire in 7 days (or less)
• each token has a single tool scope, not “agent-do-everything”

This is boring IAM hygiene. That’s why it works.

Sandbox tool execution

If your agent runs code, or shells out, put it in a box.

I’m opinionated here. If you’re not willing to sandbox the agent, you’re not ready to give it write access.

A pragmatic path is a dedicated Linux VM runner. I’ve covered this setup in AI Agent Sandbox Linux VM [2026].

Default-deny network egress

If your MCP server or tool runner can connect to the internet freely, you’ve basically built an exfiltration appliance.

Give the agent:

• access to the APIs it needs
• access to your logging stack
• nothing else

If you need a mental model, treat it like a PCI environment. The agent is not “a developer.” It’s an untrusted workload.

Runtime monitoring and logging to catch supply-chain abuse

Supply chain attacks are nasty because they look like normal behavior. Your agent still “works.” It just leaks.

In production, I want agent logging to look closer to payment systems than web apps.

Log every tool call like it’s an audit event

At minimum:

• tool name
• tool version (or image digest)
• MCP server identity (host + cert or signed metadata)
• argument hash (store the hash, not raw secrets)
• response size in bytes
• duration in ms

If your agent averages 20 tool calls per task, you can alert on sudden shifts. “This workflow usually calls github.readFile 3 times. Today it called it 60 times.” That’s a signal.

For a concrete schema, see AI Agent Observability Logging Schema [2026].

Detect “new dependency” events

This is the runtime equivalent of “new outbound connection.”

Alert on:

• first-seen MCP server
• first-seen tool plugin version
• prompt pack version change

If you can’t answer “what changed?” during an incident in under 5 minutes, you’re going to have a bad week.

Incident response when an agent dependency is compromised

Supply chain security isn’t real until you can roll back.

Here’s the playbook I’d want written down.

1) Immediate containment (minutes)

• revoke tokens used by affected tools
• block MCP server egress at the firewall
• freeze dependency updates (no more auto-bumps)

Target: containment in < 30 minutes.

2) Blast radius analysis (hours)

• identify which agent versions ran with the compromised artifact
• list data accessed and tools invoked during that window
• check logs for unusual tool-call patterns (volume, destinations)

3) Recovery (same day)

• roll back to last known-good prompt pack / tool image digest
• rotate credentials that were accessible
• patch CI gates so the same class of change can’t ship again

I’ve shipped enough systems to know this is where teams lie to themselves. The incident “ends” when production is stable. The real work is making sure the pipeline can’t repeat the mistake.

If you need a broader governance baseline, start with AI Security Leader Playbook [2026].

Here’s the official MCP explainer if your team is still catching up. Watch it once so you can align on terminology:

[YOUTUBE:eur8dUO9mvE|What is MCP? Integrate AI Agents with Databases & APIs]

My stance: stop shipping agents with “floating dependencies”

This industry keeps reinventing the same failure mode. We spent 10 years learning to pin Docker images by digest, require SBOMs, and lock dependency graphs. Now we’re wiring agents to MCP servers and prompt packs like it’s 2012 and the internet is friendly.

If you’re building production AI systems, treat every agent artifact as a supply chain risk. Pin it. Sign it. Attest it. Sandbox it. Monitor it.

My prediction: within 12 months, “prompt pack signing” and “MCP server allowlists” will be as normal as container scanning. The teams that adopt these gates early won’t just be safer. They’ll ship faster, because they won’t be negotiating security from scratch every time someone wants to add a new tool.

Originally published on kunalganglani.com