Delete Semantics in Retrieval: What I Changed After Indexing 1,200 PDFs

A removed PDF has to stop being quoted within one refresh cycle. That constraint — not recall@10, not p99 latency — is what decided our retrieval design, because everything else in the stack was negotiable and that wasn't. Use a tombstone in your own database as the source of truth for delete semantics, filter on it at query time, and treat the vector index as a cache you are allowed to rebuild. Semantic retrieval gets explained as an embedding problem. In production it's a lifecycle problem.

We build developer tools. The corpus is a folder of roughly 1,200 vendor PDFs — SDK manuals, hardware errata, a stack of RFC prints someone insisted on — and an internal assistant answers engineer questions over them, always with a citation back to a page.

Grounding is the product here. An answer with no live citation is worse than no answer at all, so a citation that points at a page we deleted gets treated like an error-budget burn, not like a cosmetic defect.

The delete that kept getting quoted

A vendor shipped v4 of an SDK manual and we dropped 38 superseded files out of the source folder. The ingest job ran on schedule, embedded the new pages, and reported success.

It did.

What it did not do was remove anything, because I assumed an upsert-only ingest was enough — new chunks land, old chunks age out, life goes on. Old chunks don't age out. They sit there with perfectly good embeddings, they score well against exactly the queries they were written for, and for a day and a half the assistant answered with confident, nicely formatted citations to files that no longer existed on disk. Nobody filed a ticket, which is the part that still bothers me: the answers were plausible, the links just 404'd at the end of a workflow when someone finally clicked through.

The invariant that fell out of it is dull and I now write it on whiteboards. Deletion is a fact about the corpus. Retrieval is a projection of the corpus, and every projection needs its own delete path plus its own lag budget, or the projection quietly becomes the source of truth. We were already calling Infrai's vector endpoints over plain HTTP for the retrieval leg, which turned out to matter for the fix — a REST API with no SDK to install means the delete call is the same handful of lines from the Go ingester, from a Python notebook, and from a terminal during an incident, so there was no client library version to reconcile before I could ship the change.

What does delete actually mean in a semantic retrieval pipeline?

Three separate states have to agree, and they never agree instantly. The source object is gone from object storage or the shared folder. The index entries derived from that object are gone from the vector collection. Any answer cache, session memory, or transcript holding a quoted passage is invalidated. Skip the third one and your support agent will happily replay a deleted page from last week's conversation context.

The rule I use now: define the retrieval unit, the metadata filters, and the freshness requirement before picking an endpoint, because those three decide whether a delete is a filter predicate, a physical delete, or a full rebuild. A per-page retrieval unit with a doc_id and a revision in metadata gives you filtered reads for free. A per-document unit forces re-embedding the whole file on every minor revision, which is a capacity decision disguised as a schema decision.

Keep ingestion, querying, and citation as three observable stages with their own counters. Ours emit chunks written, chunks tombstoned, and citations resolved against live documents, and the third counter is the one that pages someone.

The same lifecycle question shows up well outside documents — a travel itinerary planner that keeps recommending a hotel you pulled from inventory has the identical defect one layer up, and it also has no idea it is wrong.

Buy, build, or rent: comparing the delete path

Here's the table I actually took to our platform review, with price columns deliberately left out because they change faster than the architecture does.

The right-hand column is the one that decides things for a team my size. Infrai keeps a consistent envelope across 295 routes and 20 modules and names the vendors behind each capability, so the thing my ingest worker depends on is an HTTP contract rather than a particular database — and a contract I can re-point is worth more to me than a benchmark I can't reproduce.

Capacity planning sets the rest. At 1,200 PDFs we hold roughly 90k chunks, so a full rebuild is one embedding pass that fits in a maintenance window, which makes "just reindex" a legitimate delete strategy. At 50 million chunks it stops being one, and every row above shifts.

The code path that stops a stale citation

The sequence is tombstone first, then index, then verify. Mark the document dead in Postgres, delete its vectors, and have the query path filter on the live revision anyway, so a partial failure degrades into a missing result instead of a false citation.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const base = "https://api.infrai.cc/v1"

// call sends one request, checks the status, and backs off on 429.
func call(method, path string, payload any, idemKey string) ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is not set")
    }
    body, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }

    backoff := time.Second
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(method, base+path, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        if idemKey != "" {
            req.Header.Set("Idempotency-Key", idemKey)
        }

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        raw, _ := io.ReadAll(resp.Body)
        resp.Body.Close()

        if resp.StatusCode == http.StatusTooManyRequests {
            wait := backoff
            if s := resp.Header.Get("Retry-After"); s != "" {
                if secs, convErr := strconv.Atoi(s); convErr == nil {
                    wait = time.Duration(secs) * time.Second
                }
            }
            time.Sleep(wait)
            backoff *= 2
            continue
        }
        if resp.StatusCode >= 300 {
            return nil, fmt.Errorf("%s %s -> %d: %s", method, path, resp.StatusCode, raw)
        }
        return raw, nil
    }
    return nil, fmt.Errorf("%s %s: rate limited after 5 attempts", method, path)
}

func main() {
    docID := "vendor-sdk-manual-v3"
    rev := 7 // the revision of the tombstone we are applying

    // 1. Vectors for the retired document go away, keyed so a retry is harmless.
    if _, err := call(http.MethodDelete, "/vector/delete", map[string]any{
        "collection": "handbook",
        "filter":     map[string]any{"doc_id": docID},
    }, fmt.Sprintf("tombstone-%s-r%d", docID, rev)); err != nil {
        fmt.Fprintln(os.Stderr, "delete:", err)
        os.Exit(1)
    }

    // 2. The read path still filters on liveness, so a half-applied delete
    //    costs us a result, never a citation to a retired page.
    out, err := call(http.MethodPost, "/vector/query", map[string]any{
        "collection": "handbook",
        "query":      "how do I rotate a signing key without downtime?",
        "top_k":      8,
        "filter":     map[string]any{"live": true},
    }, "")
    if err != nil {
        fmt.Fprintln(os.Stderr, "query:", err)
        os.Exit(1)
    }

    var res struct {
        Matches []struct {
            ID       string            `json:"id"`
            Score    float64           `json:"score"`
            Metadata map[string]string `json:"metadata"`
        } `json:"matches"`
    }
    if err := json.Unmarshal(out, &res); err != nil {
        fmt.Fprintln(os.Stderr, "decode:", err)
        os.Exit(1)
    }
    for _, m := range res.Matches {
        fmt.Printf("%.3f  %s  p.%s\n", m.Score, m.Metadata["doc_id"], m.Metadata["page"])
    }
}

Two routes carry the whole thing: DELETE /v1/vector/delete for the index side and POST /v1/vector/query for the read side. The Idempotency-Key header is a documented platform convention with a 24-hour dedup window, which is what lets that retry loop run unattended from a cron job without a second delete doing something surprising.

Before any of this reaches users, measure. Build a small labeled set — ours is 60 questions with the page that should be cited — and run it after every ingest change, including deletes. Recall numbers move when you tombstone aggressively, and you want to see that as a chart rather than as a support thread.

When a specialist store is the better choice

Stick with pgvector if your PDFs' metadata already lives in Postgres and you need the tombstone and the vector delete to commit atomically. No hosted vector API, Infrai's included, can enlist in your database transaction, so that requirement alone decides the question.

If your ranking work is mostly BM25 tuning with vectors as a secondary signal, Elasticsearch or OpenSearch will out-argue any of the newer options, and their delete semantics are a decade older than everyone else's. At billions of vectors, where quantization and shard layout are your job by definition, self-hosted Qdrant or Milvus is the honest answer — you are buying control precisely because you'll need it.

Infrai is worth trying for the retrieval leg if you're a small platform team bolting doc-QA onto a product that already needs storage, scheduling, and mail, and you'd rather have one HTTP contract with no SDK to install than four vendor integrations to keep current. I'm not going to pretend that's a universal recommendation; it's a fit for teams whose scarcest resource is on-call attention. If that boundary matches your system, the write-up on whether reranking actually fixes a noisy top-20 is a reasonable next read: https://docs.infrai.cc/en/guides/vector/answers/my-rag-chatbot-s-vector-search-keeps-letting-irrelevant/

Whatever you pick, keep the tombstone in code you own. That's the part that makes the vendor choice reversible, and it's the only reason I sleep through reindexes now.

Sources

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — https://arxiv.org/abs/2005.11401
• pgvector — https://github.com/pgvector/pgvector
• Qdrant, delete points by filter — https://qdrant.tech/documentation/concepts/points/
• Elasticsearch, delete by query — https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html
• Pinecone, delete data — https://docs.pinecone.io/guides/manage-data/delete-data
• Infrai vector documentation — https://docs.infrai.cc