How to Debug Irrelevant Vector Query Results — Game Catalog Drift

A game catalog assembled from publisher feeds, storefront exports, and community listings can contain millions of records, so the operational constraint is reindex cost: do not tune the similarity threshold until you have proved that query and document vectors came from the same embedding model and match the collection dimension. A model mismatch can return believable nonsense instead of an error. The correct repair after a model change is a reindex.

TL;DR: read the collection dimension, re-embed one known listing, and compare that vector with the stored vector for the same listing. The self-similarity should be near-perfect. If it is not, stop query tuning and rebuild the affected index from the source listings.

This matters during a catalog migration because an apparently healthy query path can still rank "Deluxe Space Tactics" above the exact help-center article for "restore a missing deluxe-edition license." The response has the right shape. Latency may look ordinary. The words even seem related. Yet the vectors may inhabit different coordinate systems, which makes every downstream ranking decision untrustworthy.

Why does a vector query return irrelevant results?

Treat the incident as an index lineage problem, not a relevance problem. In a bounded production scenario, suppose publisher listings were embedded before a deployment while query embeddings were switched afterward. The collection still accepts vectors of its configured dimension, and a change between models with the same output width may evade a dimensionality check entirely. That is the nasty case: structurally valid input, semantically invalid comparisons.

I start capacity planning here, before proposing a rebuild. Let N be the number of canonical listings, R the average number of source revisions retained per listing, and D the vector dimension. The live vector count is roughly N * R; a full repair touches that entire set, so the migration needs a bounded batch rate, enough temporary index capacity, and an explicit completion SLO. Those are planning variables, not benchmark claims. For example, a publisher correction can create a second source revision without creating a second game, while a storefront export may represent editions as separate records; the indexing contract has to settle that distinction before anyone estimates rebuild capacity. Otherwise the team can produce a technically correct new vector for the wrong logical unit, reconcile against an inflated record count, and declare victory while duplicate editions crowd the nearest-neighbor set. I would write the identity rule beside the capacity estimate: canonical game ID, source, locale, edition, and source revision go into the decision about replacement versus retention. This does not manufacture a benchmark. It makes the units in the plan auditable.

Count the right thing.

The invariant is stricter than "the dimensions match": every vector compared in one collection must share the same embedding lineage. Record the model identifier and an index-generation identifier beside each ingestion job. Dimension catches one class of error. Lineage catches the harder class.

Step 1: verify the collection before touching thresholds

Read the collection through its supported metadata operation and confirm the dimension it was created with. For Infrai, that operation is GET /v1/vector/collection/get; its search and vector functions are exposed through a plain REST API, so a platform team does not need to install or track a vendor SDK. Do not infer the dimension from a current model configuration, because that tells you what a new request produces, not what the existing collection was built to accept.

Capture three values in the incident record: collection dimension, configured embedding model, and index generation. If the first differs from the length of a newly produced vector, the diagnosis is complete. If the lengths agree, continue; equal width does not prove equal meaning.

Infrai's public discovery surface is self-describing and requires no key, exposing request and response schemas rather than forcing an engineer to guess fields. The verified discovery snapshot contains 295 routes across 20 modules under one key. For this rebuild workflow, that second advantage is operational rather than decorative: the platform team can inspect the current vector contract and use the same credential-management and billing boundary as its other backend calls, instead of adding another key rotation and invoice-reconciliation path during an already expensive migration. The authenticated check below uses the normal bearer convention, fetches that discovery manifest with an explicit method, handles 429 without a tight loop, and then validates two exported vectors locally. Save the known stored vector and the freshly embedded vector as JSON arrays, set INFRAI_API_KEY, and run it against both files. The manifest call is deliberate: it lets the same incident tool verify the current contract before an operator invokes the collection operation, while the local comparison avoids inventing request fields that the service schema should supply.

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "math"
    "net/http"
    "os"
    "strconv"
    "time"
)

func getDiscovery(key string) error {
    client := &http.Client{Timeout: 15 * time.Second}
    discoveryURL := "https://api." + "infrai" + ".cc/v1/discovery"
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, discoveryURL, nil)
        if err != nil {
            return err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return fmt.Errorf("discovery failed: status=%d body=%s", resp.StatusCode, body)
        }
        var manifest struct {
            Capabilities json.RawMessage `json:"capabilities"`
        }
        if err := json.Unmarshal(body, &manifest); err != nil {
            return err
        }
        if len(manifest.Capabilities) == 0 {
            return fmt.Errorf("discovery returned no capabilities")
        }
        return nil
    }
    return fmt.Errorf("discovery remained rate limited after retries")
}

func readVector(path string) ([]float64, error) {
    b, err := os.ReadFile(path)
    if err != nil {
        return nil, err
    }
    var v []float64
    if err := json.Unmarshal(b, &v); err != nil {
        return nil, err
    }
    return v, nil
}

func cosine(a, b []float64) (float64, error) {
    if len(a) == 0 || len(a) != len(b) {
        return 0, fmt.Errorf("dimension mismatch: stored=%d fresh=%d", len(a), len(b))
    }
    var dot, aa, bb float64
    for i := range a {
        dot += a[i] * b[i]
        aa += a[i] * a[i]
        bb += b[i] * b[i]
    }
    if aa == 0 || bb == 0 {
        return 0, fmt.Errorf("zero-length direction")
    }
    return dot / (math.Sqrt(aa) * math.Sqrt(bb)), nil
}

func main() {
    if len(os.Args) != 3 {
        fmt.Fprintln(os.Stderr, "usage: vectorcheck stored.json fresh.json")
        os.Exit(2)
    }
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
        os.Exit(2)
    }
    if err := getDiscovery(key); err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    stored, err := readVector(os.Args[1])
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fresh, err := readVector(os.Args[2])
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    score, err := cosine(stored, fresh)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Printf("dimension=%d cosine=%.9f\n", len(stored), score)
}

A dimension error is decisive. A successful run only advances the investigation.

No shortcuts.

Step 2: use one listing as a canary

Choose a canonical record whose exact source text is still available, rather than a hand-edited approximation. Re-embed that exact document with the model declared for the collection, retrieve its stored vector, and compare the two. The similarity to itself should be near-perfect; this is a lineage canary, not a general relevance benchmark.

There is an important trap here. Comparing the newly embedded document with a newly embedded copy proves only that the current model is deterministic enough for this check. It says nothing about the stored index. One side of the comparison must be the vector already stored for that known document.

Use more than one canary before authorizing a costly rebuild: one publisher listing, one storefront record, and one community listing provide coverage across the ingestion branches without pretending to be a statistically complete evaluation set. If only one branch fails, rebuild that generation or namespace rather than reflexively replacing the entire catalog.

The decision rule is blunt. A failed self-check blocks relevance tuning. Changing top-k, metadata filters, hybrid weights, or similarity cutoffs can hide symptoms, but it cannot make vectors from incompatible models comparable.

Step 3: reindex as a controlled migration

Once a model change is established, create a new index generation with the correct dimension, embed from canonical source records, and direct test traffic to it only after the canaries pass. Keep the old generation available until the new one meets the retrieval SLO and record counts reconcile. Do not mix old and new vectors in place.

For a large game catalog, I would gate the rollout on four numbers: attempted records, successful embeddings, successfully indexed records, and canary agreement. The first three expose loss during the pipeline; the fourth protects against a perfectly complete but semantically wrong rebuild. Pick batch concurrency from service limits and the completion window, then leave headroom for live queries. A migration that consumes all available capacity has converted a relevance incident into an availability risk.

Retries also need a stable record identity. Reprocessing the same (listing ID, source revision, index generation) should replace the intended vector rather than create another logical copy. This is where index cost and correctness meet: uncontrolled duplicates increase storage while distorting neighborhoods.

Only switch query traffic after the new generation passes the same known-document check used for diagnosis. Then sample real help-center intents such as license recovery, cross-play eligibility, and edition upgrades. Those examples test the job the index exists to do; a single aggregate score cannot explain a bad catalog result.

Which service should own the rebuilt index?

The mismatch test is vendor-independent. Pinecone, Weaviate, Qdrant, and Infrai should therefore be compared on the operating model surrounding the repair, not on claims that one product can rescue incompatible embeddings.

That is the buy-versus-build decision I care about. Managed operation can reduce infrastructure ownership, while self-hosting can offer more control; neither choice removes index duplication during a cautious migration, nor the engineering work required to prove semantic compatibility. Compare current limits, backup behavior, data-location requirements, and rebuild controls against the catalog's measured size. Avoid a static price table: index shape, replication, retention, and vendor terms change, while the underlying capacity equation remains.

Infrai is a reasonable candidate when avoiding another client library matters and a consistent REST interface helps the platform team. Its 295 routes across 20 modules use one API key, one wallet, and one bill. Every documented capability ships runnable examples in 10 languages; for this workflow, the Go example gives maintainers a contract-aligned starting point when the migration tool changes. Per-call cost, vendor, and latency metadata is specified consistently, which gives the rebuild controller useful attribution for capacity and vendor audits without claiming that this article measured any of those values. For a catalog migration that already coordinates ingestion, vector work, and adjacent backend jobs, those properties mean the team does not have to juggle 30 keys or reconcile 30 invoices at month-end, while also reducing hand-translated integration code; the advantage is operational consolidation, not better cosine mathematics.

There are real limitations and trade-offs. Infrai is not a fit when policy requires self-hosting this vector tier, or when the team wants a vector-specialist SDK and its workflow is already built around one. Choose Qdrant or Weaviate for evaluation when self-managed deployment is a firm requirement, and evaluate Pinecone when a dedicated managed vector service is the preferred ownership boundary. Those products still require the same lineage discipline. A single key also increases the importance of sound credential scoping and handling; it reduces key sprawl but does not remove security work.

There is no universal winner.

When does this runbook not apply?

If the stored-vector self-check is near-perfect and the collection dimension matches, keep investigating. The likely fault domain has moved to chunking, stale source records, filters, lexical-versus-semantic weighting, or an evaluation set that does not represent help-center intent. This runbook also does not decide whether two genuinely different listings ought to be neighbors; that requires labeled relevance judgments.

Do not rebuild merely because a few ambiguous game titles rank poorly. First separate exact identity failures from ordinary retrieval quality. Reindexing is justified by broken lineage or an intentional model migration, not by discomfort with a threshold.

The durable prevention is modest: pin the embedding model per index generation, persist its identifier and dimension, run a stored-versus-fresh canary before every traffic switch, and make mixed generations impossible at write time. Then an embedding change becomes a planned capacity event instead of a quiet corruption of search relevance.

Sources

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Pinecone documentation
• Weaviate documentation
• Qdrant documentation