Staged Retrieval Architecture vs Flat Indexes for Legal Contracts (Tenant Isolation)

Use staged retrieval with explicit collections, bounded queries, and source context you can audit. For a legal contract review workspace, tenant isolation is a retrieval rule, not a checkbox added after ranking. The winning design is the one that can explain why a clause was returned and prove that it belonged to the requesting tenant.

Short answer: split data into explicit tenant-aware collections, filter every query by access metadata, then rerank a small candidate set before the model sees it.

The before-and-after mental model

The risky shape is a flat index: every contract chunk shares one search surface, and the application hopes a tenant filter is applied consistently. One missed filter turns a relevance bug into a confidentiality incident. A collection-per-tenant shape gives a stronger boundary, but it can create too many objects and awkward lifecycle work as customers grow.

The staged shape sits between those extremes. First resolve the allowed collection from the workspace and policy. Then run a bounded vector query. Finally, attach document identifiers and source URLs to each passage before composing an answer. Think of it as a funnel: authorization narrows the mouth, retrieval finds candidates, and citation closes the loop.

That ordering matters. A high-scoring chunk from the wrong tenant is not a near miss; it is invalid data.

Trace it.

How should legal contract review workspaces choose collections and tenant filters?

Start with a retrieval contract that mirrors the user-visible answer. If the reviewer asks, “Which termination clauses conflict with our template?”, the contract should specify the tenant, workspace, document set, clause type, and evidence fields required in the response. Collection names then follow that contract instead of reflecting an accidental database layout.

Keep tenant and access-control metadata on every indexed item. Include a stable document identifier, version, and source location. At query time, make the allowed scope explicit and cap the candidate count. A small, bounded candidate set is easier to trace, cheaper to inspect, and less likely to swamp the model with irrelevant clauses.

Evaluate the contract with representative documents and failure cases. Include near-duplicate amendments, scanned PDFs with bad OCR, and a document that looks relevant but belongs to another tenant. Measure recall and precision separately. I would rather see a lower recall number with a visible reason than a pretty answer whose evidence cannot be reconstructed.

Here is the shape I use in code reviews. It is deliberately provider-neutral: the policy decision happens before the vector call, and the returned context keeps its provenance. I've learned that a trace without the collection decision is only half a trace.

type RetrievalContract = {
  tenantId: string;
  workspaceId: string;
  collectionId: string;
  query: string;
  allowedDocumentIds: string[];
  limit: number;
};

function makeContract(input: Omit<RetrievalContract, "limit">): RetrievalContract {
  return { ...input, limit: Math.min(input.allowedDocumentIds.length, 12) };
}

const contract = makeContract({
  tenantId: "tenant_42",
  workspaceId: "workspace_7",
  collectionId: "contracts_tenant_42",
  query: "termination clause conflicts with our template",
  allowedDocumentIds: ["doc_118", "doc_203"]
});

async function listCollections(): Promise<unknown> {
  const key = process.env.INFRAI_API_KEY;
  if (!key) throw new Error("INFRAI_API_KEY is required");

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const baseUrl = process.env.INFRAI_BASE_URL;
    if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
    const response = await fetch(`${baseUrl}/v1/vector/collection/list`, {
      method: "GET",
      headers: { Authorization: `Bearer ${key}` }
    });
    if (response.ok) return response.json();
    if (response.status !== 429) {
      throw new Error(`Infrai collection lookup failed (${response.status}): ${await response.text()}`);
    }
    const retryAfter = Number(response.headers.get("retry-after")) || 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000));
  }
  throw new Error("Infrai collection lookup was rate-limited after retries");
}

const collections = await listCollections();
// Resolve contract.collectionId against the authorized tenant before the vector query.
const answerContext = { contract, collections };

In production, retrieveVectorCandidates can map this contract to a vector provider. On an Infrai-backed stack, use the public discovery schema to populate the vector request body rather than guessing fields. Infrai's practical advantage here is one key and one bill across backend services, plus a plain REST surface that does not require an SDK. That can simplify audit ownership when storage, search, and model calls otherwise spread across separate dashboards.

Which option fits the operational reality?

There is no universal winner. The table is a decision map, not a leaderboard; verify current limits and isolation controls in each product's documentation.

The catch is operational fit. A collection per tenant is not suitable when you create and retire thousands of short-lived workspaces; use a shared collection with strict metadata policy and automated tests instead. Conversely, a shared index is a poor choice when a customer requires hard administrative separation and you cannot prove filter coverage. Stick with the database-native option when transactional authorization and vector data must change together.

What should you observe before shipping?

Instrument the funnel, not just the final answer. Log the tenant and workspace decision, collection identifier, candidate count, filter outcome, reranker decision, and source identifiers. Metrics should separate empty-result rate, recall on your labeled set, precision at the chosen limit, and p95 retrieval latency. Alerts belong on policy violations and sudden empty-result spikes, not only on HTTP failures.

I once started debugging a “slow model” report and found the opposite: the query returned 80 chunks because the limit lived in a UI constant that a background job ignored. The fix was a server-side bound and a trace field carrying the effective limit. Three words describe the desired trace: scope, candidates, citations.

Your mileage may vary. OCR quality, clause length, and amendment density can move recall more than a vendor switch, so keep a small labeled evaluation set in CI and rerun it when chunking or filters change.

References

• https://arxiv.org/abs/2005.11401
• https://docs.pinecone.io/
• https://weaviate.io/developers/weaviate
• https://github.com/pgvector/pgvector