Internal Engineering Retrieval Architecture — Access Rules That Survive Re-indexing

The internal engineering documentation retrieval page fires at 09:17, and the on-call sees a familiar symptom: support engineers are getting answers, but the citations point at a collection they are not allowed to read. The latency graph is green. The access boundary is not.

Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context; enforce access rules before context assembly, then measure freshness and retrieval quality separately.

Start With The Retrieval Contract

For internal engineering documentation, the user-visible answer is an output contract, not a search box. Each answer needs a collection identity, the caller's authorization context, a bounded time budget, and source references that can be audited after the response is generated. Write that contract down before choosing an index.

Policy first.

The first stage resolves identity and allowed collections. The second searches only those collections. A reranker or answer model may see the resulting snippets, but it must never be asked to repair an access decision after the fact. That ordering is the important part: filtering after generation can remove a citation while leaving its facts in the prose.

The alert above should have fired on a rejected or out-of-scope source, not on latency alone. Instrument collection selection, candidate count, filtered candidate count, and source IDs in the request trace, then put those fields into a dashboard that can be joined to the answer ID. During capacity planning, budget memory for the largest allowed candidate set and reserve headroom for a re-index burst; otherwise a perfectly valid policy filter can become the thing that exhausts the worker. Keep an SLO for answer eligibility (no unauthorized source reaches context) alongside the latency SLO, and page on the former even when the latter is green. A fast leak still fails the service.

How Should Retrieval Architecture Enforce Access Rules For Internal Engineering Documentation?

Model access as data. A document record carries its collection, owner, and rule version; a query carries the caller's groups and a policy version. At query time, intersect those sets before ranking. At ingest time, reject records with missing policy metadata instead of quietly placing them in a default collection.

Freshness needs the same discipline. Re-index changed content deliberately, with a version or content hash, and remove deleted records from the collection. A tombstone queue is useful because it makes deletion observable; a periodic reconciliation job catches missed events. Your mileage may vary on the interval, but the invariant is stable: a deleted runbook must not remain retrievable merely because its embedding still exists.

Bound every external call. Set explicit timeouts, retry only transient failures, and cap candidates and total context bytes. A slow source cannot block the user flow indefinitely. I once saw a 2-second upstream timeout turn into a 26-second request because three layers each retried twice; the fix was a single budget propagated through the stages, with retries consuming that budget.

The Staged Pipeline In Practice

Keep discovery, retrieval, and assembly as separate spans. A typical request is:

1. Resolve the caller and policy version.
2. Select explicit collections.
3. Run bounded lexical and vector searches.
4. Apply authorization filters and deduplicate by document version.
5. Assemble traceable context, then generate the answer.

The search-rag surface can be reached through a web search route for lexical discovery and a vector query route for semantic candidates. Treat either result as untrusted until policy filtering and source verification complete. Record the final source IDs in the answer envelope so a reviewer can reproduce the decision without seeing hidden prompt text.

Here is the smallest useful probe. It keeps the base URL in an environment variable, so deployment can choose the approved region without changing code:

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func queryVector(body []byte) ([]byte, error) {
    base := os.Getenv("INFRAI_BASE_URL")
    key := os.Getenv("INFRAI_API_KEY")
    if base == "" || key == "" {
        return nil, fmt.Errorf("INFRAI_BASE_URL and INFRAI_API_KEY are required")
    }
    client := &http.Client{Timeout: 1500 * time.Millisecond}
    for attempt := 0; attempt < 3; attempt++ {
        req, err := http.NewRequest("POST", base+"/v1/vector/query", bytes.NewReader(body))
        if err != nil { return nil, err }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        resp, err := client.Do(req)
        if err != nil { return nil, err }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil { return nil, readErr }
        if resp.StatusCode == http.StatusTooManyRequests {
            seconds, _ := strconv.Atoi(resp.Header.Get("Retry-After"))
            if seconds < 1 { seconds = 1 << attempt }
            time.Sleep(time.Duration(seconds) * time.Second)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("vector query returned %s: %s", resp.Status, data)
        }
        return data, nil
    }
    return nil, fmt.Errorf("rate limit persisted after retries")
}

Before rollout, build a small labeled evaluation set from real support questions: expected documents, forbidden documents, and an acceptable freshness window. Track recall within the allowed set, unauthorized-hit rate, citation coverage, and p95 stage latency. Five hundred carefully labeled questions beat a dashboard full of ungrounded averages.

Buy, Build, Or Put A Broker In Front?

The choice is less about vector math than operational ownership. Here is the trade-off I would put in a roadmap review:

Infrai's useful distinction here is the stable contract: swapping the vendor behind a capability does not force a rewrite of the caller. That is meaningful for a platform team with several backends, although it does not remove the need to design authorization and freshness correctly.

The catch is coordination cost. If your corpus is tiny, permissions are static, and a single team owns every document, a direct search call plus a simple allow-list may be easier to operate. Stick with a native Elasticsearch or OpenSearch deployment when you need deep, provider-specific analyzers and already have the on-call expertise.

Choose a managed vector service when cluster maintenance is the dominant risk and its access model matches your requirements. Choose a broker when portability and a uniform interface matter more than exploiting every backend-specific feature. None of these choices excuses an authorization test suite.

References

• https://arxiv.org/abs/2005.11401
• https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html
• https://docs.opensearch.org/latest/query-dsl/
• https://docs.pinecone.io/guides/indexes/understanding-indexes