Vector Index Over Live Web Retrieval: Ranking Signals for Listing Discovery at Scale

Own the index for the durable half of your catalogue, and use live web retrieval only as a thin freshness layer on top. For a marketplace that aggregates game-item listings from a dozen upstream sources, that ordering is what keeps ranking signals under your control while listing discovery stays current. The deciding constraint isn't recall quality — every option on the shortlist can find a plasma rifle skin. It's what the index costs to hold and to keep re-embedding once the catalogue stops being small, and that number grows with churn rather than with traffic, which is the part nobody plans for.

I keep meeting teams who build this the other way round. They start with a hosted live-search call because it demos beautifully in week one, and by month four they have no ranking signals of their own, no way to collapse the same item appearing in six feeds under four spellings, and no answer when someone asks why result #3 moved overnight.

The signal that forces the split

The retrieval architecture question usually arrives disguised as a capacity question, so do the arithmetic before the design review rather than after it. 1.2M live listings, one 1536-dimension float32 vector each, is 1,200,000 × 1536 × 4 bytes ≈ 7.4 GB of raw vectors — call it 15 GB resident once the HNSW graph and the filterable payload fields are in memory. That is still a machine you can reason about and put on a capacity plan.

Then add churn, which is where a listing corpus stops behaving like a document corpus. If a third of your catalogue is re-listed, re-priced or delisted every week, you are re-embedding something like 400k rows a week forever, and the delete path costs you the same attention as the write path.

Deletes are the part everyone forgets.

The failure mode I write the runbook against is silent staleness: a sold listing that still ranks, still gets clicked, and still burns a support ticket, because the embedding survived the row it described. Removing deleted records from the collection on the same code path that removes them from the product database is not optional hygiene here, it's the difference between a search box people trust and one they route around. Set a freshness objective you can actually verify — ours would be five minutes for price and availability changes, an hour for new descriptions — and treat any budget you cannot measure as a budget you have not set.

How should ranking signals change when listing discovery spans multiple sources?

Deduplicate first, rank second. When six upstream feeds describe one physical item, the retrieval unit is the cluster, not the row, and every ranking signal you compute on un-clustered rows is measuring your ingest pipeline instead of the market.

After that, keep vector similarity in its lane. It is a recall device: it gets 50 plausible candidates in front of your ranker cheaply and tolerantly of spelling. It is a bad ranking function, because cosine distance has no opinion about seller reputation, price recency, delivery time, or the fact that one source lies about stock. Those are business signals, they belong in a second-stage scorer you own, and they are the reason to hold the index rather than rent the answer.

Live web retrieval earns its place at exactly one point in that chain — when the user asks about something your crawler hasn't reached yet, and a bounded call to a search endpoint like /v1/web/search gets a citable URL into the answer within the freshness budget. Use it as a fallback with a hard timeout and a hard result cap, not as the primary path. Every result it returns is ranked by signals you don't control and can't tune, and that's fine for the 3% of queries that need it and wrong for the other 97%.

Buy versus build: what each option actually buys you

Infrai sits in that last row, with 295 routes across 20 modules behind one key, so the day you add a scraper or a sparse keyword pass next to vector query it's another endpoint on the same contract rather than another vendor, another SDK and another invoice. Its discovery surface is public and needs no key, which means you can read a capability's request schema and its runnable examples before you commit any code to the shape — a small thing that saves an afternoon per integration on a two-person platform team.

The catch with anything in that bottom row, Infrai included, is that index internals live on the provider's side of the contract. If your ranking plan depends on a particular HNSW configuration, a custom analyzer, or data residency in a region you name yourself, stick with pgvector or a Qdrant cluster you run — that trade-off is the whole point of the row above it. And if your users mostly type exact identifiers, item IDs and half-remembered set names, Typesense or Elasticsearch will beat a vector-first design on the queries that pay your bills.

The implementation: one query path, two stores

The safe shape is a single retrieval function with a bounded first stage, an explicit collection, and metadata filters applied in the store rather than in your service. Bounded means a top_k you wrote down and a timeout you enforce.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type vectorQuery struct {
    Collection string `json:"collection"`
    Text       string `json:"text"`
    TopK       int    `json:"top_k"`
}

// post sends one JSON request, retries on 429 with exponential backoff,
// and surfaces the response body on any non-2xx status.
func post(path string, payload any) ([]byte, error) {
    key, base := os.Getenv("INFRAI_API_KEY"), os.Getenv("INFRAI_API_BASE")
    if key == "" || base == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY and INFRAI_API_BASE must be set")
    }
    body, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }

    client := &http.Client{Timeout: 3 * time.Second}
    backoff := 500 * time.Millisecond

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", base+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        raw, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        if resp.StatusCode == http.StatusTooManyRequests {
            wait := backoff
            if s, convErr := strconv.Atoi(resp.Header.Get("Retry-After")); convErr == nil {
                wait = time.Duration(s) * time.Second
            }
            time.Sleep(wait)
            backoff *= 2
            continue
        }
        if resp.StatusCode/100 != 2 {
            return nil, fmt.Errorf("POST %s: status %d: %s", path, resp.StatusCode, raw)
        }
        return raw, nil
    }
    return nil, fmt.Errorf("POST %s: still throttled after 4 attempts", path)
}

func main() {
    raw, err := post("/v1/vector/query", vectorQuery{
        Collection: "listings-live",
        Text:       "tradable plasma rifle skin, blue, buy now",
        TopK:       50,
    })
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Println(string(raw))
}

Three habits in there earn their keep on a listing workload. The key and the base URL both come from the environment, so the same binary runs in staging against a staging collection. The method is explicit on every request, which sounds pedantic until a proxy in front of you starts making its own decisions about verbs. And the 429 path backs off with the server's own Retry-After instead of hammering — during a bulk re-index, that is the difference between a slow catch-up and a self-inflicted incident. When you extend this to the writes, derive an idempotency key from the upstream listing id plus its revision, so a retried feed webhook re-applies the same vector instead of duplicating it.

Verifying the split, and backing it out

Build the evaluation set before the rollout, not after the complaints. 200 labelled queries drawn from real search logs, each with the handful of listings a human agrees should come back, is enough to catch the regressions that matter; measure recall@20 for the vector stage and click-through on position 1–3 for the ranker, and hold both against the current production path rather than against an abstract target. I'd hold the rollout at a 5% traffic slice until recall@20 stops moving between runs, which is usually two or three days of real query mix. Your numbers will differ from mine and that's fine — the point is that the shortlist stage and the ranking stage get measured separately, because a recall problem and a ranking problem look identical from the search box.

Keep the rollback boring. One flag that routes retrieval back to the previous path, the old index kept warm and written to for a full release cycle, and an alert on the freshness objective rather than on request count — staleness is silent, and a dashboard of green request counters will happily report a search box full of sold items. Delete the old path only once the evaluation set has been green through a full churn cycle.

If you're carrying fewer than about 200k listings and your queries are mostly exact-ish, none of this applies to you yet. Put the vectors in pgvector, keep the delete path honest, and spend the saved quarter on ingest quality instead.

References

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — https://arxiv.org/abs/2005.11401
• pgvector — https://github.com/pgvector/pgvector
• Qdrant documentation — https://qdrant.tech/documentation/
• Elasticsearch kNN search — https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
• OpenSearch vector search — https://opensearch.org/docs/latest/search-plugins/vector-search/
• Typesense vector search — https://typesense.org/docs/latest/api/vector-search.html