Online Course Retrieval Architecture: Tutor Index Economics (and Why I Chose One)

Short answer: use a staged retrieval design with explicit collections, bounded queries, and source context that can be traced back to the lesson and tenant. For an online course tutor, that decision matters more than picking a fashionable embedding model: index cost grows with every reprocessed transcript, while an unbounded query quietly grows latency and token spend.

I treat a release checklist as an experiment note, not a vendor parade. The simple approach is one giant index and a single similarity call. It is quick to demo, then difficult to explain when a learner gets an answer from the wrong course. The design that survives release has separate ingestion, querying, and citation stages, each with observable inputs and outputs. I would only copy it after measuring recall, precision, re-index cost, and citation coverage on representative lessons plus deliberately bad matches.

What should an online course tutor retrieval architecture measure before release?

Start with the answer the learner should see, then write the retrieval contract behind it. A contract can say: return up to eight chunks from the learner's enrolled course, include lesson ID and access metadata, and provide enough source text to cite the exact section. That is testable. “Find relevant context” is not.

Build a small evaluation set from real material: a coding lesson, a quiz explanation, a transcript with repeated terminology, and a lesson that should produce no result. Add adversarial pairs such as “closures in JavaScript” beside “closures in Rust.” Label the expected chunks. Recall tells me whether the right passage was retrieved; precision tells me how much irrelevant text I am paying to pass to the model.

The failure case I care about is boring but expensive: a shared chunk ID survives a course import, but its tenant metadata does not. The tutor then has a plausible answer with an invalid citation. Preserve tenant, course, enrollment scope, lesson, and source offsets on every indexed item. Filter before ranking where the store supports it, and enforce the same scope again before rendering citations. I also store an embedding version such as lesson-emb-v3; that tiny field makes a rollback or side-by-side recall test possible after a re-index.

Ship it only when the contract is observable.

No shortcut.

For a gaming course tutor, one transcript can contain the same item name in a design lesson, a patch-notes lesson, and a moderation lesson. A broad nearest-neighbor search may return the right words from the wrong audience. I would tag each chunk with game, course, lesson type, and enrollment scope, then make those fields part of the evaluation fixture. A query for “damage falloff” should retrieve the combat-design explanation for the enrolled course, while a question about moderation should either retrieve its own lesson or say that the catalog has no grounded answer. That distinction is where index cost and trust meet: fewer, better candidates reduce reranking work and leave the model less room to invent a bridge between unrelated lessons.

A staged pipeline keeps index cost and answer quality visible

Ingestion normalizes documents, chunks them by lesson boundaries, attaches metadata, and records an embedding version. Querying first applies the learner's scope and a result limit, then runs vector retrieval; a lightweight lexical check can catch exact code symbols that embeddings miss. Answer assembly receives only the bounded context and emits citations tied to stored offsets.

Each stage gets a counter and a trace ID. I want to see documents accepted, chunks written, chunks skipped, candidates returned, candidates discarded by access filters, and citations attached. If a re-index doubles chunk count, the cost signal appears before a model bill does. Keep old embedding versions long enough to compare recall, then delete them deliberately.

Here is the smallest shape of a bounded vector query. The route is a POST, and the body should match the schema exposed by your chosen service; the important contract is the explicit limit and tenant filter, not a magic collection name.

type RetrievalRequest = {
  query: string;
  collection: string;
  limit: number;
  filter: { tenant_id: string; course_id: string };
};

export async function retrieve(input: RetrievalRequest, attempt = 0): Promise<unknown> {
  const key = process.env.INFRAI_API_KEY;
  const baseUrl = process.env.INFRAI_BASE_URL;
  if (!key) throw new Error("INFRAI_API_KEY is required");
  if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");

  const response = await fetch(`${baseUrl}/v1/vector/query`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ ...input, limit: Math.min(input.limit, 8) }),
  });

  if (response.status === 429) {
    if (attempt >= 4) throw new Error("retrieval rate limit persisted after retries");
    const retryAfter = Number(response.headers.get("retry-after"));
    const delaySeconds = Number.isFinite(retryAfter) ? retryAfter : 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delaySeconds * 1000));
    return retrieve(input, attempt + 1);
  }
  if (!response.ok) {
    throw new Error(`retrieval failed (${response.status}): ${await response.text()}`);
  }
  return response.json();
}

The retry shown here is intentionally small; production code should cap attempts and use exponential backoff. A read-only query has no duplicate-write risk. Ingestion does, so give each chunk a deterministic ID such as course/lesson/source-offset/embedding-version and make upserts idempotent.

How do hosted vector options compare for a course tutor?

The right comparison axis is operational control at your expected corpus size, not a leaderboard score copied from another domain. These are credible options with different ownership costs:

Infrai's useful angle here is the plain HTTP surface: no SDK installation or client version to babysit. The same key can cover adjacent backend capabilities, while the retrieval code remains an ordinary request. That convenience is real, but it does not remove the need to design chunking, filters, and evaluation yourself.

Stick with pgvector when your course catalog already lives in Postgres and joins are the hard requirement. Choose Elasticsearch when exact phrase and faceted filters dominate. A dedicated vector service makes sense when the team wants managed index operations. Your mileage may vary; I am not sure any generic benchmark predicts the recall of your lesson transcripts and quiz vocabulary.

Release gates I would automate

First, a contract test rejects an item missing tenant or access metadata. Second, a bounded-query test fails if the requested limit exceeds the product budget. Third, a retrieval set test checks recall and precision against the labeled lessons, including “no answer” cases. Fourth, a citation test verifies that every generated claim points to a chunk offset in the learner's scope.

Keep ingestion and query traces separate. A green query test cannot prove that last night's import indexed every lesson, and a green import count cannot prove that filtering works. Store the evaluation corpus and its labels beside the test code, with a small sample run on every change and a larger run before changing the embedding model.

The final gate is a human read of ten answers: five ordinary questions, three ambiguous ones, and two questions outside the catalog. It is fast. It catches confident citations that metrics miss.

References

• https://arxiv.org/abs/2005.11401
• https://www.pinecone.io/learn/vector-database/
• https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
• https://github.com/pgvector/pgvector
• https://weaviate.io/developers/weaviate