Your Vector Database Thinks "Not Covered" Means "Covered"

TL;DR: An embedding's similarity score measures how much two texts share in topic and wording, and a negation is a one-word edit. I wrote 40 invented policy facts and tested six open embedding models. On every model, "Which plan doesn't cover X?" scored closer to "Which plan covers X?" than a plain rewording of the same question did, for 77.5–100% of the items. At a common semantic-cache cut-off (similarity 0.90, RedisVL's default), one model would serve the cached answer to 97.5% of the opposite questions and to 7.5% of the genuine rewordings. In retrieval, models found the "covered" chunk almost every time and the "not covered" chunk between 27.5% and 92.5% of the time. Adding BM25 through rank fusion made five of the six worse. Dropping English stop words, which removes "not", took BM25 to a coin flip. Re-ranking the top ten with a small cross-encoder took every model to 97.5% or better on the negated questions. Run the same test on your own model before you trust a similarity threshold.

A benefits assistant had a semantic cache in front of it, so repeated questions skipped the LLM. On Monday someone asked "Which plan covers dental implants?" and got the right answer: the Gold plan. On Tuesday someone else asked "Which plan doesn't cover dental implants?"

The cache embedded the new question, found Monday's at similarity 0.97, and returned "The Gold plan." That's the opposite of the right answer, delivered in a fraction of a second with no LLM call to blame. Someone asking "What plan pays for dental implants?", which means exactly the same as Monday's question, would have scored 0.85 and missed the cache.

The assistant is invented. The two numbers aren't: they're what all-MiniLM-L6-v2, the most downloaded sentence-similarity model on Hugging Face (about 200 million downloads a month in October 2026), gives for those three questions. The rest of this article is what happened when I ran the same comparison across six models, three tests and 40 facts.

What the score is actually measuring

Embedding models are trained to put texts that go together (a question and its answer, a title and its abstract, two captions of one picture) close together. Texts that go together share a topic and, usually, a lot of words. So the model learns a geometry where shared topic and shared wording count for most of the distance, and nothing in training forces a single "not" to move a sentence to the other side of the space.

From the vector's point of view, "Which plan doesn't cover dental implants?" is the original question with one word added. "What plan pays for dental implants?" swaps two words and adds a third. The negation is the smaller edit, so it lands closer.

This isn't new. NevIR (Weller, Lawrie and Van Durme, EACL 2024) built document pairs that differ only by negation and found most retrieval models, including state-of-the-art ones, "performing the same or worse than a random ranking", with cross-encoders doing best. Cao (ECAI 2025) tested general-purpose text embeddings and reported "a significant lack of negation awareness in these models". A June 2026 preprint on agent memory (Yadav, arXiv 2606.26511) reported that cosine similarity separated contradicted facts from duplicates with an AUROC of 0.59, close to chance, and gave the same reason: contradictions came out more similar to the original than duplicates did "because a value-flip is a smaller edit than a genuine rephrasing of the same fact." That result is small (98 labelled pairs, one embedding model), so I wanted to see how far it generalised. An August 2026 audit of similarity thresholds in agent frameworks (Frias, arXiv 2608.10216) is the closest to what follows: a production drift guard "caught 0 of 56 meaning-breaking mutations", including "withhold the study drug" → "administer the study drug", which "came in at cosine 0.9608".

Six models, 40 facts, three tests

I wrote 40 short policy facts across twelve domains: dental and health plans, home, travel and car insurance, refunds, software tiers and licences, staff benefits, airline fares, warranties, bank accounts, gym memberships and drug formularies. Each fact has three forms:

• the fact: "The Gold plan covers dental implants."
• a rewording with the same meaning: "Dental implants are paid for under the Gold plan."
• a contradiction: "The Gold plan does not cover dental implants."

The contradictions use several kinds of negation on purpose: "does not", "isn't", "excludes", "no", "never", "unpaid", "non-refundable". Each fact also has three questions: one asking which entity covers it, a rewording of that question, and the opposite question ("Which plan doesn't cover dental implants?"). Most opposite questions carry a negation word, but a few flip the meaning without one: "Which tier limits data exports?", "Who gets unpaid parental leave?", and "Which account charges foreign transaction fees?", where it's the original that says "no".

The six models are all open and run locally: all-MiniLM-L6-v2, bge-base-en-v1.5, e5-base-v2, gte-base, mxbai-embed-large-v1 and Qwen3-Embedding-0.6B, each with the query and passage prefixes its model card asks for. I didn't test hosted embedding APIs, which is the main reason to run the check below on whatever you use.

Test 1: questions, the semantic-cache case

For each of the 40 items: how similar is the original question to its rewording, and to its opposite?

The last two columns use 0.90, which is what RedisVL's SemanticCache defaults to (a cosine distance of 0.1). That's RedisVL's threshold applied to these six models, not RedisVL's own default embedding model, which I didn't test. On every model, a cache at that setting answers the opposite question more often than it answers the same question asked differently.

You can't tune your way out of it. Set the threshold low enough to catch 95% of genuine rewordings on each model, and it catches all 40 opposites, on all six. As a separation score, similarity managed an AUROC between 0.01 and 0.25, where 0.5 is chance. A number that low means the score is reliably backwards: higher similarity was evidence that the question meant the opposite.

Test 2: statements, the deduplication case

The same comparison between the fact, its rewording and its contradiction. This is what a deduplication step or a memory store faces when it decides whether a new fact is "the same" as one it already has.

This one is mixed, and it's worth saying so. Three models separate statements well, and mxbai separates them almost perfectly. Three don't: MiniLM, e5 and Qwen3 sit near chance, so "X is not covered" is as likely as a rewording to look like a duplicate of "X is covered". The models that cope with statements still failed the question test. Part of that is my data: the question rewordings change more words than the statement rewordings do, and on every model the rewordings score lower for questions. That's also how people actually ask questions, so I don't think it makes the question result less real. The safe conclusion isn't "embeddings can't do negation". It's that it depends on the model and the text shape, and you can't know which case you're in without measuring.

Test 3: retrieval, and what fixes it

The cache and deduplication cases compare two texts directly. Retrieval is subtler, because the query and the chunk aren't phrased alike. So I built an index where every item contributes two chunks about different entities: "The Gold plan covers dental implants" and "The Silver plan does not cover dental implants". Then I asked both questions. "Which plan covers dental implants?" should rank Gold's chunk above Silver's, and "Which plan doesn't cover dental implants?" should do the reverse. All 80 chunks sit in the same index, so the other 39 items act as distractors.

Every dense model answered the positive questions well (90–100%). The negated questions are where they split:

Share of the 40 opposite questions ("doesn't", "excludes", "no", "unpaid", "limits" and so on) where the right chunk ranked above its opposite. A coin flip scores 50%.

The MiniLM row is worth a second look. Counting an item as right only when both its questions came out right, the way NevIR scores it, MiniLM got 17.5%. Guessing at random would get 25%.

The fix everyone recommends made it worse

The standard advice for negation is hybrid retrieval: the "not" is a keyword, so add BM25 and fuse the rankings. I believed that going in, and it's what the outline for this article said.

In this test, BM25 on its own got 77.5% of the negated questions right, which beats two of the dense models and ties a third. Fusing it with them using reciprocal rank fusion (Cormack, Clarke and Büttcher, 2009, k = 60) lowered the score for five of the six. Only MiniLM improved, from 27.5% to 32.5%. The reason is in the wording. The keyword arm only sees the negation when the question and the chunk use the same word. In 25 of the 40 items they did ("excludes" and "excludes", "does not" and "does not"), and BM25 got all 25 right. In the other 15 they didn't ("doesn't" against "not covered", "won't refund" against "non-refundable", "no API access" against "unavailable"), and BM25 got 6 right, which is below a coin flip. There it rewards whichever chunk shares more terms with the question, and that's often the wrong one. Rank fusion gives that vote the same weight as the dense model's.

There's a second trap underneath, and it's a configuration setting. Elasticsearch's english analyzer and its stop token filter both default to the _english_ stop list, which is Lucene's ENGLISH_STOP_WORDS_SET. That list contains "no" and "not". So the keyword that's supposed to rescue you is deleted before it reaches the index. My BM25 is a simple Python one (rank_bm25, lowercase word tokens, no stemming), not Elasticsearch itself, but I gave it the same stop list. With that list applied, it scored exactly 50% on the negated questions, a coin flip, and every hybrid score dropped further. (The standard analyzer keeps stop words by default. Check which one your text fields actually use.)

What worked was the slow, boring stage. A cross-encoder reads the question and the chunk together, so "doesn't" can change its answer in a way that two separately computed vectors can't. Re-ranking each dense model's top ten with ms-marco-MiniLM-L-6-v2, a 22-million-parameter model, took every row to 97.5% on the negated questions. bge-reranker-base took them all to 100%. The dense model you started with stopped mattering here, because the right chunk and its opposite were both in every top ten. The ordering inside it was the problem. That's an easy case: an 80-chunk index where the two rivals share most of their words. In a large index full of near-identical clauses, check that both make it into the candidates you re-rank. This matches NevIR's ranking, and it's why re-ranking gets a whole lesson of its own in SophiArch's Retrieval & RAG Systems course, two lessons after the one on hybrid retrieval: the two stages fix different failures.

The re-rankers aren't perfect, and both made a couple of mistakes in the other direction. bge-reranker-base put "The standard warranty excludes accidental drops" above "The extended warranty covers accidental drops" for "Which warranty covers accidental drops?". ms-marco-MiniLM missed "Which account offers no overdraft?" and "Who refunds return shipping costs?". e5 also missed the overdraft question, and the opposite of the shipping one ("Who doesn't refund return shipping costs?"). That's 2 misses in 80 questions for each re-ranker, against 3 for the best dense model (e5) and 33 for MiniLM. One caution from Frias's audit: swapping in an NLI model as the gate "did no better" there. A re-ranker trained for search relevance and an NLI classifier aren't the same tool, and short policy questions are an easy case. So re-run the triples with the re-ranker in the loop before you rely on it.

Try it on your model before you set a threshold

This takes a few minutes and needs no labelled data. Write 20 question triples from your own domain: a question, a rewording, and its opposite. Then run:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("your-embedding-model")  # or call your hosted embedding API

triples = [
    ("Which plan covers dental implants?",
     "What plan pays for dental implants?",
     "Which plan doesn't cover dental implants?"),
    # ... 19 more from your own domain
]

backwards = 0
for question, reworded, opposite in triples:
    q, r, o = model.encode([question, reworded, opposite], normalize_embeddings=True)
    sim_r, sim_o = float(q @ r), float(q @ o)
    backwards += sim_o > sim_r
    print(f"{sim_r:.2f}  {sim_o:.2f}  {question}")

print(f"opposite closer than rewording: {backwards}/{len(triples)}")

If the opposite wins most rows, and on every model I tried it did, then no similarity threshold on that model can tell a repeat question from its opposite. Use the same triples for statements if you deduplicate or merge facts.

Where each fix goes

• Semantic caches. Don't serve a hit on similarity alone. The cheapest guard is a polarity check: if one question contains a negation word or a word like "exclude" or "without" and the other doesn't, treat it as a miss. A stronger one is to score the (new question, cached question) pair with a cross-encoder before serving. Either way, test the cache with opposite questions, not just rewordings, since rewordings are the only thing most cache evaluations try.
• Retrieval. Re-rank the top k with a cross-encoder. In this test, putting it after any of the six dense models beat every combination without one. If you also run BM25, check the analyzer's stop list for "no" and "not", and measure whether fusion helps on your queries rather than assuming it does.
• Deduplication and memory. Don't merge or drop a fact because it's close to one you already have. A contradiction is often closer than a duplicate. Compare the claims themselves, with a cross-encoder or an LLM check, before deciding they're the same.

What this test doesn't show

The facts are invented, short and in English, and there are 40 of them. Real policy text is longer, and a longer chunk dilutes the negation further, so I'd expect statement-level separation to get worse, not better. The retrieval index has 80 chunks, so both rivals always reached the top ten. In a large index, recall into the re-ranker becomes a separate thing to measure. The BM25 numbers come from a simple Python implementation without stemming, not from an Elasticsearch analyzer, so treat them as a direction, not a figure for your cluster. I only tested open models; hosted embedding APIs may do better or worse, which is the point of running the triples yourself. And the rewordings change more words than the negations do. That's deliberate, because it's the comparison a cache faces in production (people reword freely, and a negation is one word), but it means the results describe how real questions differ, not a controlled single-word edit.

None of it means vector search is broken. It means a similarity score answers "are these about the same thing?" and gets used as if it answered "do these mean the same thing?". For dental implants, those are opposite answers.

References

• Weller, O., Lawrie, D., & Van Durme, B. (2024). NevIR: Negation in Neural Information Retrieval. arXiv:2305.07614 (EACL 2024). Negation-pair benchmark; most retrievers perform at or below random, cross-encoders best.
• Cao, H. (2025). Semantic Adapter for Universal Text Embeddings: Diagnosing and Mitigating Negation Blindness to Enhance Universality. arXiv:2504.00584 (ECAI 2025). Negation awareness in general-purpose text embeddings.
• Yadav, N. (2026). Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge. arXiv:2606.26511 (v1). Cosine similarity separating contradicted facts from duplicates at AUROC 0.59; the "smaller edit" explanation.
• Frias, S. E. (2026). Similarity Gates Approve Reversals: A Validity Audit of Embedding-Cosine Thresholds in Agent Systems. arXiv:2608.10216 (v1). Cosine-threshold gates in agent frameworks (deduplication, semantic caches, drift guards) approving meaning reversals; the NLI drop-in result.
• Cormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods. SIGIR 2009. The rank-fusion method used for the hybrid rows (k = 60).
• Redis. RedisVL cache API: SemanticCache. Default distance_threshold=0.1 in cosine distance units.
• Elastic. Language analyzers, Stop token filter and Standard analyzer. The english analyzer's _english_ stop list and the standard analyzer's _none_ default.
• Apache Lucene. EnglishAnalyzer.java. ENGLISH_STOP_WORDS_SET, which includes "no" and "not".
• Model cards for the six embedding models and two re-rankers tested: all-MiniLM-L6-v2, bge-base-en-v1.5, e5-base-v2, gte-base, mxbai-embed-large-v1, Qwen3-Embedding-0.6B, ms-marco-MiniLM-L-6-v2, bge-reranker-base.
• SophiArch. Retrieval & RAG Systems. Course covering chunking, hybrid retrieval with rank fusion, query rewriting, cross-encoder re-ranking, retrieval metrics and RAG evaluation.