I wish every vendor demo began with this simple prompt: "Show me a real workflow where your AI drafts or classifies a quality record, then show me the audit trail and the human signature on it." Marketing decks can talk about "AI-assisted quality" until the cows come home — but compliance is about records, reviewability, and traceability. Working software proves the promises.
I've sat through demos where the vendor spends 20 minutes on flashy NLP summaries and then glosses over what happens when an engineer edits the draft, who approved it, and how that is recorded. In our Class II environment (ISO 13485 and FDA 21 CFR Part 820 are the yardsticks), that gap is the one that turns a nice feature into regulatory risk.
Why this question matters
- AI can speed classification and draft CAPA narratives, triage complaints, or suggest root causes. That's useful.
- But regulated QMS work isn't just about a suggestion — it's about a traceable decision. If an AI suggests a CAPA, the system must show:
- who saw the suggestion
- what they changed
- when they accepted it
- an auditable signature or equivalent control
- Regulators (and notified bodies) want reviewability and traceability. "AI proposes, human approves and signs" is the right model — and it must be demonstrable in the product.
A demo checklist I now always run through
When a vendor claims "AI drafts" or "auto-classifies," I ask for a live, end-to-end demo that covers these points. If they refuse or rush through any, I get suspicious.
Ask them to perform — live in the demo — the following:
- Create a real quality record (e.g., a complaint or deviation). Have the AI draft the first-pass content or classification.
- Show the edit history: before/after text, who edited, timestamps, and commentary explaining each change.
- Show the approval step: where a person reviews the AI output, makes edits, and signs off (electronic signature or other verifiable control).
- Export the record with metadata (timestamps, user IDs, version history) in a human-readable format.
- Show how the record links to SOPs, risk assessments, existing change requests, and any related CAPAs.
- Show API/webhook access to the audit trail (so your automation or e-discovery scripts can fetch records programmatically).
- Demonstrate the role protections: can an engineer bypass approval? Can the AI draft be auto-closed without human sign-off?
If they can't do that in the demo, get it in writing and ask for a PoC in your environment.
What vendors usually show — and why it's insufficient
Typical demo flow:
- They show the AI generating a draft in a sandbox.
- They highlight accuracy metrics or model confidence.
- They show a "Save" button.
What's missing:
- Who actually accepted that save in the workflow (was it a human or an unattended automation)?
- The record of edits and why each edit happened (useful in root-cause investigations).
- Exportability of the audit trail in a format auditors can consume.
- API access to the same audit trail so you can include it in your DHF, eDMS exports, or legal holds.
Those gaps don't just annoy auditors — they create evidence disputes. If a notified body asks "who verified this AI-suggested CAPA?" you need a clear, retrievable answer.
Non-negotiables (practical, not just marketing)
In our shop, "AI-assisted" features must meet these criteria before we consider them for production:
- Human-in-loop with explicit approval step before any record is considered final.
- Immutable audit trail with time-ordered events: draft created, edits, approvals, sign-offs, exports.
- Electronic signature support that meets your applicable rules (e.g., 21 CFR Part 11 controls if you sell in the US).
- Linkability: automated suggestions must be linkable to procedures, risk files, and other records in the QMS.
- Exportable audit data via API: you should be able to pull a complete, machine-readable record for archiving or analysis.
I call this "controlled assistance": the AI helps the user, but the system enforces traceability and reviewability.
Practical API and migration checks
Engineers on the team care about two practical things:
- Can we fetch the full audit trail via REST? If the vendor only shows UI audit views, ask for an API spec and a sample JSON export.
- If we migrate later, can we export records (including signatures, timestamps, and edit history) in a standard, human-readable archive? Vendors that lock audit trails in proprietary blobs make migrations and audits miserable.
Ask for a short-lived sandbox account and a scripted check: create a draft, approve it, then call the audit-trail API and verify all events are present. That test takes 30–60 minutes and reveals a lot.
Closing thought — what I want vendors to do
If you sell QMS software with AI features, start your demo with a simple, auditable story: AI drafts a complaint summary → engineer edits → QA approves → record is exported with full audit trail. Show the API. Show the signature. Let the buyer validate the trace.
If you buy QMS software, insist on that demo. Marketing can't show you the audit trail in a way that satisfies auditors — working software can.
What's the best real demo you've seen where an AI draft was handled end-to-end, and what did it convince you of (or not)?
