Sub-100ms Patient Intake & Clinical Triage: Slashing 98% of Routing LLM Calls in Healthcare Systems

Part 4 of the "Jev-First Routing for LangGraph" series. Read Part 1: Your Agent Harness Is Paying LLM Prices for Classifier Decisions | Read Part 2: Manufacturing Safety Routing | Read Part 3: Supply Chain Hazard Triage.

Executive Summary for Healthcare CIOs, CMIOs & Engineering Leaders

• The Problem: Modern health systems, telehealth providers, and EHR platforms (Epic, Cerner) process millions of patient intake forms, nurse triage notes, and portal symptom messages annually. Multi-agent clinical intake systems typically route these encounters using a frontier LLM (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro). This creates unpredictable latency (800ms to 30s+), vulnerability to rate-limiting outages (429), and massive token expenditures on simple 4-way specialty classification.
• The Solution: Confidence-gated routing with langgraph-jev-router. TypeSafe’s Jev model evaluates the clinical presentation in ~119ms. When confident, it dispatches the encounter autonomously; when uncertain, your trusted clinical LLM router executes with 100% fail-safe reliability.
• The Empirical Proof: Benchmarked on 200 real-world clinical patient encounters from the MTSamples Database (CC0 Public Domain, fully de-identified under HIPAA Safe Harbor):
  • -98.0% LLM routing calls eliminated (49 out of 50 encounters routed autonomously)
  • 94.0% to 96.0% accuracy preserved
  • 119 ms average decision latency (vs. 307ms simulated baseline and 25s+ under API congestion)
  • 94.9% net routing cost cut
• Tunable Clinical Risk Dial: Multi-threshold calibration shows that even at an ultra-conservative 0.95 threshold, Jev eliminates 90.0% of LLM calls while matching the full LLM's accuracy (96.0%).

1. The Operational Reality of Clinical Intake & Triage

In hospital enterprise networks, ambulatory clinics, and digital health platforms, patient intake is the primary front door.

Every day, thousands of encounters arrive across digital portals:

• "62-year-old male with history of coronary artery disease presents with acute retrosternal chest pressure and dyspnea."
• "Patient with abnormal EEG during light sleep showing sharp wave activity in frontotemporal head regions."
• "34-year-old runner with acute left knee pain, swelling, and audible pop following twisting injury during marathon."
• "50-year-old female with persistent postprandial epigastric burning, nausea, and weight loss refractory to antacids."

Every encounter requires immediate, accurate routing to the appropriate specialist department:

• cardiology: Cardiovascular, thoracic, and pulmonary disorders.
• neurology: Brain, spine, and central/peripheral nervous system disorders.
• orthopedics: Musculoskeletal trauma, joint arthroplasty, and spine conditions.
• gastroenterology: Digestive tract, liver, and biliary disorders.

                            [ Patient Intake Encounter ]
                                          │
                                          ▼
                         [ Clinical Department Router ]
                                          │
       ┌──────────────────┬───────────────┴───────────────┬──────────────────┐
       ▼                  ▼                               ▼                  ▼
 [Cardiology Agent] [Neurology Agent]             [Orthopedics Agent] [Gastroenterology Agent]

The Architectural Flaw: Using Generative Reasoning for Categorical Dispatch

Many healthcare engineering teams implement this routing conditional edge using a general-purpose frontier LLM. In production, this pattern introduces three critical liabilities:

1. The Latency Trap: A 1,000ms–3,000ms LLM response time causes visible lag in interactive patient portals. During peak hours or third-party API spikes, retries push response times past 20 to 30 seconds. Patients waiting for urgent care cannot be left hanging on a frozen loading spinner.
2. The Fragility & Uptime Trap: If an external LLM vendor suffers a network outage or returns HTTP 429 RESOURCE_EXHAUSTED, the intake queue freezes. In clinical settings, routing failures cause operational chaos across nursing stations.
3. The Financial Trap: Paying generative LLM token fees ($0.015 per call) across 1,000,000 annual patient encounters means spending $15,000/year purely on categorical if/else decisions.

2. The Solution: Confidence-Gated Fast-Path Clinical Routing

langgraph-jev-router implements confidence-gated System 1 routing:

                           [ Clinical Encounter Note ]
                                       │
                                       ▼
                       ┌───────────────────────────────┐
                       │    TypeSafe Jev Fast-Path     │
                       │    (~119ms System 1 Reflex)   │
                       └───────────────┬───────────────┘
                                       │
                   ┌───────────────────┴───────────────────┐
                   │                                       │
        Confidence ≥ 0.70                       Confidence < 0.70
         (Clear pathology)                      (Complex / Multi-system)
                   │                                       │
                   ▼                                       ▼
        ┌─────────────────────┐                 ┌─────────────────────┐
        │ Direct Fast Dispatch│                 │ Existing LLM Router │
        │ ⚡ ~119ms / No LLM  │                 │ 🤖 Trusted Fallback │
        └─────────────────────┘                 └─────────────────────┘

• Unambiguous clinical presentations (e.g., "Patient with worsening chest tightness and exertional dyspnea") are recognized by Jev in ~119ms without invoking your LLM.
• Complex, multi-system presentations (e.g., "Diabetic patient with concurrent thoracic numbness, nausea, and syncope") fall below the confidence threshold and automatically route to your existing LLM router for full clinical reasoning.
• Fail-Safe Operation: If TypeSafe is ever unreachable or times out (enforced by a 5-second circuit breaker), the router logs a warning and executes your LLM router. The graph never fails.

3. Empirical Benchmark: Real-World Clinical Transcriptions

We benchmarked langgraph-jev-router on the MTSamples Clinical Database (CC0 1.0 Universal Public Domain, hosted on GitHub).

The Dataset

• 200 real clinical patient encounter summaries across 4 balanced specialist departments: cardiology, neurology, orthopedics, and gastroenterology.
• Fully de-identified under HIPAA Safe Harbor (45 CFR § 164.514) — zero Protected Health Information (PHI).

Benchmark Comparison at Default Threshold (0.70)

4. The Executive Knob: Calibration Sweep (0.60 to 0.95)

In healthcare systems, Chief Medical Information Officers (CMIOs) require quantifiable assurance that automation does not degrade clinical accuracy.

langgraph-jev-router provides a tunable threshold parameter. We executed a multi-threshold sweep (python healthcare/benchmark.py --sweep) from 0.60 to 0.95:

Why Does Jev Perform So Exceptionally in Healthcare?

Medical specialties have distinct, well-defined clinical vocabularies ("echocardiogram", "myocardial infarction", "epilepsy", "arthroplasty", "endoscopy"). Because these terms represent strong categorical signals, a typed decision model can classify 90%–98% of cases with extreme confidence and high accuracy (96.0%), only escalating genuinely ambiguous presentations to the reasoning LLM.

5. Code Implementation: Adding @jev_first to Your Clinical Graph

Adding langgraph-jev-router to an existing LangGraph healthcare application requires wrapping your router with @jev_first:

from typing import Literal
from langchain_core.messages import AIMessage, HumanMessage
from langgraph.graph import StateGraph, START, END, MessagesState
from langgraph_jev_router import jev_first, attach_routes

# 1. Define specialist departments with clinical descriptions
DEPARTMENTS = {
    "cardiology": (
        "Cardiovascular, thoracic, and pulmonary conditions (chest pain, coronary artery disease, "
        "myocardial infarction, arrhythmias, heart failure, hypertension, valve disease, and dyspnea)."
    ),
    "neurology": (
        "Brain, spinal cord, and nervous system disorders (seizures, epilepsy, migraines, "
        "neuropathies, stroke/TIA, tremors, Parkinson's disease, and cognitive impairment)."
    ),
    "orthopedics": (
        "Musculoskeletal injuries, joints, bones, and spine disorders (bone fractures, joint arthroplasty, "
        "arthritis, ligament/tendon tears, disc herniation, back/neck pain, and orthopedic trauma)."
    ),
    "gastroenterology": (
        "Digestive system, gastrointestinal tract, liver, and biliary conditions (abdominal pain, GERD, "
        "endoscopy/colonoscopy, Crohn's/colitis, bowel obstruction, GI bleed, and pancreatitis)."
    ),
}

# 2. Add one decorator to your existing clinical router
@jev_first(
    routes=DEPARTMENTS,
    question="Which medical specialist department should handle this patient clinical presentation?",
    threshold=0.85,  # High precision mode (94-96% accuracy)
)
def triage_router(state: MessagesState) -> Literal["cardiology", "neurology", "orthopedics", "gastroenterology"]:
    # ── YOUR EXISTING CLINICAL LLM ROUTER (Acts as 100% zero-risk fallback) ──
    print("🤖 Complex encounter: Falling back to Gemini LLM router...")
    # Your standard OpenAI / Gemini / Claude invocation runs here
    return "cardiology"

# 3. Build your LangGraph state graph
builder = StateGraph(MessagesState)

builder.add_node("cardiology", lambda s: {"messages": [AIMessage("Routed to Cardiology Specialist")]})
builder.add_node("neurology", lambda s: {"messages": [AIMessage("Routed to Neurology Specialist")]})
builder.add_node("orthopedics", lambda s: {"messages": [AIMessage("Routed to Orthopedics Specialist")]})
builder.add_node("gastroenterology", lambda s: {"messages": [AIMessage("Routed to Gastroenterology Specialist")]})

builder.add_conditional_edges(
    START,
    triage_router,
    ["cardiology", "neurology", "orthopedics", "gastroenterology"],
)
for node in ["cardiology", "neurology", "orthopedics", "gastroenterology"]:
    builder.add_edge(node, END)

# 4. Attach routes and compile
graph = attach_routes(builder).compile()

6. HIPAA, Privacy & Enterprise Data Security

For healthcare technology leaders, data perimeter security is mandatory:

1. What data leaves the hospital perimeter? Only the clinical symptom description and the 4 department descriptions are sent to TypeSafe's API endpoint. No patient identifiers (names, MRNs, DOBs) are ever transmitted.
2. Is patient message history stored? No. Message history is not retained or logged by default.
3. Model Version Pinning: The decision model (jev-latest or pinned jev-1-preview) is immutable, ensuring that third-party updates cannot silently shift clinical routing thresholds.
4. Instant Rollback: Setting JEV_ROUTER_DISABLED=1 instantly disengages Jev globally, returning 100% of routing to your existing LLM with zero code changes.

7. Annual ROI Projection for a Regional Health System

Consider a regional health system or digital health platform processing 1,000,000 patient portal messages and triage encounters annually:

• Traditional LLM Routing Cost (GPT-4o / Gemini Pro): 1,000,000 $\times$ $0.015 = $15,000 / year in pure routing tokens.
• Jev-First Routing Cost (at 98% avoidance):
  • 980,000 encounters handled by Jev: 980,000 $\times$ $0.000006 = $5.88
  • 20,000 encounters fallback to LLM: 20,000 $\times$ $0.015 = $300.00
  • Net Annual Routing Cost: $305.88 (97.9% direct financial savings).
• Clinical SLA & Latency Impact:
  • Triage latency drops from ~1,200ms to 119ms (a 10x speedup).
  • Across 1,000,000 encounters, over 300 hours of cumulative patient waiting latency is eliminated.
  • 100% immunity against external LLM API rate limits (429) during regional health surges.

8. What's Next in This Series

• Part 1 (Foundation): Your Agent Harness Is Paying LLM Prices for Classifier Decisions
• Part 2 (Manufacturing): Slashing Latency & Cutting 74% of LLM Calls in Plant Safety Dispatch
• Part 3 (Supply Chain): Sub-100ms Food Safety & Cargo Triage in Global Logistics
• Part 4 (This Article): Sub-100ms Patient Intake & Clinical Triage in Healthcare Systems
• Part 5 (Upcoming): Telecommunications Customer Support & Intent Routing

🛠️ Reproduce the Benchmark

All benchmark scripts, dataset files, and calibration tools are open-source and reproducible:

git clone https://github.com/JTankariya/jev-router-benchmark.git
cd jev-router-benchmark

pip install -r requirements.txt
export TYPESAFE_API_KEY="your_api_key"

# Run healthcare benchmark
python healthcare/benchmark.py --sample 50

# Run multi-threshold calibration sweep
python healthcare/benchmark.py --sweep

• Library on GitHub: https://github.com/JTankariya/langgraph-jev-router
• Benchmark Suite: https://github.com/JTankariya/jev-router-benchmark/tree/main/healthcare
• PyPI Package: pip install langgraph-jev-router

Notice: langgraph-jev-router is an independent open-source project released under the MIT license. Not affiliated with LangChain, Inc. or TypeSafe AI.