Grounded Recovery Playbooks for Healthcare Appointment Assistant Retrieval Under Load

Short answer: make failure recovery produce a verified disposition, not a better-sounding guess. For a healthcare appointment assistant, keep source identity and revision time attached to every passage, retry only idempotent reads inside a fixed deadline, and switch to a clarifying question or handoff when evidence is empty, stale, or contradictory.

I treat retrieval as an evidence pipeline with a stop condition. The language model is the last consumer of that pipeline, not its repair mechanism. A patient asking for the earliest cardiology slot next Tuesday needs a grounded statement about availability and clinic policy; a fluent answer assembled from an old index is an operational incident wearing a friendly tone.

No guesswork.

Start with dispositions, not retries

Before choosing a replica, define the states the assistant may return. grounded_answer means each operational claim has an eligible passage and citation. clarification means the request is underspecified, such as a missing location or date. handoff means the system cannot establish evidence within its deadline. These states are useful to product, clinical operations, and on-call staff because they describe truthfulness, not just transport health.

The distinction matters during a partial outage. An empty result can mean no matching appointment policy, a stale materialized view, or a query filter that excluded the right clinic. A timeout is different again. I record the query class, filter set, attempt number, source revision, and final disposition under one correlation token. Patient identifiers stay in the protected system; ordinary logs get no names or dates of birth.

I once assumed a second read would fix every miss. It did not. A replica can return a perfectly valid passage whose revision predates the scheduling source, so the response is transport-successful and still unsafe. Your mileage may vary on exact freshness windows, but the policy has to be explicit: for example, a 300 ms retrieval budget inside a 1.5 s end-to-end SLO leaves time for authorization, citation checks, and a clear handoff.

How should a healthcare appointment assistant recover retrieval failures?

Use the failure signal to choose one bounded action. On a transient read error, retry once with the remaining context deadline; on an empty or low-score result, narrow or correct filters before trying another source; on a stale revision, prefer a source with a known freshness contract; on a conflict, surface the constraint and ask for confirmation. Never broaden the search silently, because that turns a missing clinic into a plausible wrong clinic.

The recovery order I use is deliberately boring:

1. Validate the request and authorization context.
2. Query lexical and semantic indexes with specialty, location, language, and effective-date filters.
3. Check that returned passages carry non-empty source IDs, revision times, and scores above the policy threshold.
4. Retry an idempotent read once, bounded by the original deadline, then consult a known-good replica or recent snapshot.
5. If no eligible evidence remains, return handoff or clarification and preserve the attempted source set for audit.

This is a control-flow problem, not a prompt trick. The retriever may rank passages, but a policy layer decides whether they are admissible. A reranker can improve ordering; it must not discard provenance. A snapshot can explain a clinic rule, but it cannot prove that a live slot is still bookable.

Here is the smallest Go shape that keeps that boundary visible. The interfaces are generic so the policy can sit above a database extension, a hosted index, or a self-hosted service.

package retrieval

import (
    "context"
    "errors"
)

var ErrNoEvidence = errors.New("no eligible evidence")

type Passage struct {
    Text     string
    SourceID string
    Revision int64
    Score    float64
}

type Searcher interface {
    Search(context.Context, string) ([]Passage, error)
}

type Disposition struct {
    Passages    []Passage
    NeedsHandoff bool
}

func Recover(ctx context.Context, primary, replica Searcher, query string, minScore float64) (Disposition, error) {
    passages, err := primary.Search(ctx, query)
    if err == nil && eligible(passages, minScore) {
        return Disposition{Passages: passages}, nil
    }
    if replica != nil {
        fallback, fallbackErr := replica.Search(ctx, query)
        if fallbackErr == nil && eligible(fallback, minScore) {
            return Disposition{Passages: fallback}, nil
        }
    }
    return Disposition{NeedsHandoff: true}, ErrNoEvidence
}

func eligible(passages []Passage, minScore float64) bool {
    for _, p := range passages {
        if p.Text != "" && p.SourceID != "" && p.Score >= minScore {
            return true
        }
    }
    return false
}

The caller owns the deadline, so a slow primary cannot make the fallback unbounded. In production I would also carry an attempt label and a freshness verdict beside each passage, then have the answer composer require a citation marker for every slot, policy, or appointment identifier. Unsupported claims become a question or handoff. That is the recovery behavior users can understand.

Evidence checks that prevent plausible wrong answers

Grounding is more than putting a URL in the response. Verify that the cited span actually contains the claim, that the source is accessible to the intended user, and that retrieved passages do not conflict. A 200 response from a document store only proves transport success. It says nothing about whether a same-day booking rule was superseded yesterday.

Conflicts deserve a first-class branch. Suppose an availability feed says a slot is open while a clinic policy passage says same-day bookings require a call. Do not average those sources or let the model pick whichever wording sounds authoritative. Present the policy constraint, identify the conflicting evidence, and ask the patient to confirm the channel or accept a handoff.

Chunking and indexing changes need the same discipline. Keep document IDs and revision times through ingestion, embedding, reranking, and prompt assembly. For a recorded test query, assert the disposition, citation coverage, freshness verdict, and source IDs; checking only generated prose lets a regression hide behind a grammatical answer.

The practical comparison is about ownership and failure boundaries:

There is no universally correct winner. The catch is that a retrieval choice is unsuitable when its freshness contract cannot be explained to clinical operations. Stick with a simpler store when the corpus is small and curated; take on a richer stack only when its additional failure boundaries have owners.

Capacity planning for the bad minute

Recovery consumes the most capacity when a dependency is already slow. Size replicas and queue workers for the retry load, then cap concurrency so one degraded index cannot exhaust all request handlers. A queue that grows without an admission limit merely moves the outage from latency to memory.

Set separate objectives for disposition latency, retrieval availability, and grounded-answer coverage. A team might choose a 99.9% monthly objective for returning a truthful disposition within the user-facing deadline while tracking citation completeness as a second objective. Those numbers are policy choices, not universal facts; publish them so product and clinical operations can challenge the trade-off.

Alert on handoff rate, stale-source matches, and conflict rate, not only request latency. During a rollout, slice those signals by specialty, region, and query class. A healthy p95 can hide a broken filter that affects one clinic.

Verification and rollback runbook

Build a replay set containing empty results, stale revisions, duplicate passages, timeout errors, HTTP 429 responses, and conflicting clinic rules. For each case, assert that the retry count stays within the deadline and that the final disposition is correct. Keep a trace of the index build identifier, prompt version, and source revisions; otherwise a rollback leaves the team guessing which evidence produced an answer.

Shadow a new retriever against production-shaped queries with identifiers removed. Compare citation coverage, stale-source rate, handoff rate, and latency before changing chunking, embeddings, or score thresholds. Roll out behind a feature flag and keep the previous retriever addressable for one release window. If unsupported claims rise, reverse the flag and investigate the evidence path, not the prose style.

When grounding cannot be established, the assistant should be clear about what is missing and offer the next verified action. Boring is a feature here.

References

• https://arxiv.org/abs/2005.11401
• https://www.rfc-editor.org/rfc/rfc9110