Chat history is a second read path into your RAG data — gate the replay like the search

My copilot persists the source cards it cites — which documents backed each answer, scores, names, the works. That's table stakes for a trustworthy RAG product: an answer without its evidence is just vibes.

Here's the question that changed how I shipped it: six months from now, a user opens that old conversation and the cards render again. Who authorized them the second time?

The comfortable answer is "nobody has to — it's the user's own history, they already saw it." I shipped the uncomfortable answer instead, and I want to defend it:

Persistence is not permission. What a turn was allowed to show at write time proves nothing about what it may show at read time.

The short version, if you're skimming

• The moment you persist retrieval results — citations, source cards, snippets — your history endpoint becomes a second read path into the same data your search guards so carefully.
• Entitlements drift between write and read. Re-check authorization at read time, in the service, not just at the gateway.
• Degrade gracefully: when the finer-grained entitlement is off, withhold the document-derived cards but keep the conversation text. Authorization outcomes aren't all-or-nothing.
• Fail closed, and make denial look like absence: my history reads answer the same 404 for "no entitlement" as for "session doesn't exist".

Where the second read path sneaks in

Context in one sentence: I spent two weeks giving a streaming-first document-search copilot durable session history — persist every turn, restore the whole conversation after a refresh (the cursor-paging and UI-hydration half of that story is a post of its own ).

Part of that work was persisting the sources each turn emitted, pinned onto the audit event that produced them, so the transcript endpoint could replay them verbatim. And that's exactly where the trap is:

            WRITE TIME (the live turn)
user ──► chat ──► search ──► entitlement checks ──► per-doc view gate ──► answer + cards
                                                          │
                                              persisted per turn (JSONB)

            READ TIME (weeks later)
user ──► GET /sessions/{id}/turns ──► SELECT rows ──► replayed cards
                        ▲
                        └── who checks entitlements HERE?

The live search path is guarded like a fortress — entitlement toggles, a fail-closed per-document view gate, identity from the JWT only. The new history endpoint reads document-derived data out of plain database rows, and the database doesn't know about any of that. Ship it naively and you've built an unguarded side door into the exact data you spent months gating.

Nothing about this is exotic. Every RAG product that persists retrievals has this door. The only question is whether anyone put a lock on it.

Entitlements drift; rows don't

In this product, tenant admins control AI entitlements with toggles: the copilot itself, and separately the document-search capability that produces the source cards. Between the day a turn was written and the day it's replayed, weeks pass. Toggles flip. Contracts change. The rows don't care.

So I made the replay re-derive its answer from today's entitlements, not from the fact that the rows exist:

Two different denial shapes, on purpose. That's the design decision the rest of this post unpacks.

Gate one: no entitlement, no transcript

The coarse gate sits at the top of every history read. It loads the tenant's entitlements and refuses before touching a single session row (identifiers lightly renamed for the post):

async def _require_copilot_read(request: Request, auth: AuthContext) -> TenantEntitlement:
    """History reads honor the same in-service entitlement the chat path enforces
    (the service never trusts the gateway for this): a tenant with the copilot
    switched off stops being served stored transcripts."""
    entitlement = await load_tenant_entitlement(request.app.state.pool, auth.tenant_id)
    if not (entitlement.ai_enabled and entitlement.copilot_enabled):
        raise HTTPException(status_code=404, detail="Not found.")
    return entitlement

Three details doing quiet work here:

1. "Never trusts the gateway." There is a gateway in front of this service doing its own gating. The service re-checks anyway, because the day someone reroutes traffic or adds a new caller, an assumption held in another codebase is not a control.
2. 404, not 403. The turns endpoint already answers a uniform not-found for malformed, unknown, and foreign session ids, so it never confirms what exists. Entitlement denial joins the same posture — a denied caller learns nothing, not even "there's something here you can't have".
3. Denial costs zero reads. The test pins this: when the toggle is off, the session and turn queries are never even awaited.

def test_history_reads_fail_closed_when_the_copilot_is_disabled(monkeypatch):
    monkeypatch.setattr(cc, "load_tenant_entitlement",
                        AsyncMock(return_value=_entitlement(copilot_enabled=False)))
    assert client.get("/sessions", headers=_auth()).status_code == 404
    assert client.get(f"/sessions/{SESSION}/turns", headers=_auth()).status_code == 404
    cc.list_sessions.assert_not_awaited()
    cc.list_session_turns.assert_not_awaited()

Gate two: keep the words, withhold the cards

The finer case is more interesting: the copilot is on, but the document-search toggle is off. A blanket 404 would be wrong — the user's history isn't all about documents, and their conversations are still theirs. But the source cards are document-derived data: names, versions, relevance scores. They exist because a document search ran under an entitlement that is no longer granted.

So the degrade is surgical — one expression in the response builder:

# Source cards are document-derived: withhold them while the tenant's document AI is off.
sources=row["sources"] if entitlement.documents_enabled else None,

The transcript text still replays. The cards don't. The test states the contract better than I can:

def test_turns_withhold_sources_while_documents_ai_is_off(monkeypatch):
    ...
    turn = r.json()["turns"][0]
    assert turn["sources"] is None
    assert turn["final_response"] == "The answer."

This is the part I'd push hardest in a design review: graceful degradation is an authorization outcome, not an error state. Most authz discussions collapse to allow/deny, and then someone argues "deny breaks the history feature, so… allow?" — and the side door ships open. Having a middle answer (keep the conversation, withhold the derived artifacts) is what made the strict position shippable at all.

What the user sees in that withheld state — an empty gap, a placeholder, an explanation —.

The unglamorous hygiene that makes replay trustworthy

Two small things I fixed in the same arc, because a second read path deserves the same paranoia as the first:

Validate with the writer's strictness. The bound check on the persisted sources payload originally serialized with json.dumps(detail, default=str) — but the database insert serialized strictly. A payload could pass the check and then fail the entire turn record at insert time. Now the guard serializes exactly as strictly as the insert, and an unserializable payload is dropped with a warning instead of taking the transcript down with it. The turn record always lands; the transcript is never hostage to its citations.

Normalize replayed JSONB like every other JSONB read. Depending on the driver path, a JSONB column can come back as a dict or as a raw string. The replay now normalizes (json.loads when it's a string) the same way the module's other JSONB reads do — because "it worked with my driver config" is not a data contract.

Neither of these is security in the ACL sense. Both are what makes the gated replay dependable enough that you're not tempted to bypass it later.

Where I drew the line — and where you might not

Full disclosure of the trade-offs I actually made:

• The transcript text replays even when the cards are withheld. The prose answer was generated from those documents and may paraphrase them. My reasoning: the words were already delivered to this user once, and the toggle governs the document-search capability — the cards, scores, and identifiers — not speech that already happened. I think that's defensible. I don't think it's obvious.
• The gate is entitlement-level, not document-level. Re-running the per-document view gate on every history page would mean a bulk authorization call per page read — a real cost against replay traffic, and a threat-model call rather than a free upgrade. I shipped the entitlement layer first because it's one lookup that catches the whole-capability drift.
• I haven't measured the latency cost of the entitlement lookup on history reads. It's small, but "small" is a claim I didn't benchmark.

Which leaves the one call I keep turning over. When the document entitlement is revoked, I kept the words and withheld the evidence. A stricter shop would redact the whole turn; a looser one would replay everything and call the persisted rows an immutable record. If persistence is not permission — where would you have drawn the line: cards only, or the entire turn?

Thanks for sticking with an authorization post all the way to the end 🙌 If your copilot persists what it retrieves and you're now side-eyeing your own history endpoint, I'd genuinely enjoy comparing designs — find me on LinkedIn.