Embeddings Cannot Say No: An Intent Detector's Real Numbers

My message detector worked on my test set. On messages it had never seen, it missed one real case out of two.

TL;DR: I built a small detector that spots messages asking for an action. It relies on embeddings, a technique that turns a sentence into numbers. It works well, except for one thing: it does not understand negation. "The outage is fixed, thanks" fires exactly like "outage". I explain why this is a flaw of nature, not of tuning, with numbers to back it. And why the right answer is to accept the flaw rather than fix it.

This article is for developers who want to plug in AI without paying a large model on every message. No machine learning background needed: I define everything along the way.

The setup

In a personal project, an AI watches a chat between users. When a message asks for an action, a badge shows up. Create a ticket, send a document.

Calling an LLM on every message is expensive. In money and in response time. I already made that case here: put the LLM last.

So the system has two stages. Stage 1 is a small model, free to run, that sorts messages. Stage 2 is the LLM, called only when the user clicks the badge.

Stage 1 uses embeddings. An embedding turns a sentence into a list of numbers. Two sentences with close meanings give close lists of numbers. That is all you need to understand for the rest.

Deciding with a similarity score

How does an embedding make a decision? By measuring how much two sentences look alike. That score is called cosine similarity. Close to 1: the sentences are very similar. Lower: they have nothing in common.

My detector compares each message to the available tools. "Create a ticket", "send a document". If the message looks enough like a tool, it is deemed actionable.

The whole question sits inside "enough". That is where things get hard.

A fixed bar does not work

First instinct: set a bar. Above 0.85 similarity with a tool, the message is actionable.

It does not work. My model gives scores between 0.80 and 0.92, for everything. "Hello" and "create a ticket" get almost the same score.

This model sees every sentence as somewhat similar. Its scores are packed into a tiny range. And a fixed bar separates nothing inside a tiny range. Raise the bar, and you lose real cases before you lose the noise.

Compare against neutral phrases, not a bar

The solution that worked: compare, instead of measuring in the absolute.

I wrote a list of neutral phrases, with no action in them. "Hello", "thanks", "ok great". I call them anchors.

The rule becomes simple. A message is actionable if it looks more like a tool than like the best anchor. If it mostly looks like "thanks", it is chit-chat.

# The decision rule
tool_score    = similarity(message, closest_tool)
neutral_score = similarity(message, closest_anchor)

actionable = tool_score > neutral_score

One detail that matters: I split long messages on punctuation. A request drowned in politeness stands out better piece by piece. The flip side: no punctuation, no splitting. Voice dictation rarely adds any.

One false alarm out of four

I evaluated on 300 generated messages. Generated, not real: the project had no users yet. Keep that in mind, reality will do worse.

Two measures matter, and they are simple. First: out of 100 messages that deserve an action, how many the detector catches. Mine catches 92. That is called recall, and it is a good score.

Second: out of 100 harmless messages, how many it leaves alone. Mine leaves 76. In other words, 24 harmless messages out of 100 trigger the badge for nothing. That is called specificity, and it is the number that hurts.

A badge that is wrong one time out of four stops being believed. Too many false alarms kill the alarm. The first number shines in the demo. The second one is paid in production.

Embeddings cannot say no

The worst failure mode has a name: negation. "There is an outage" asks for an action. "The outage is fixed, thanks" asks for none.

To the detector, these two sentences are almost identical. Same vocabulary, so almost the same numbers. The second one triggers the badge exactly like the first.

This is not a botched setting. An embedding summarizes a sentence by its topic. And both sentences have the same topic: an outage. The "it's fixed" part weighs almost nothing in the numbers.

You can check this yourself in two minutes. Take your positive sentences, append "it's fixed, thanks" to each one. Then look at the scores: they barely move.

The mock exam caught up with me

At first I believed I could fix negation through tuning. Twelve more anchors, like "it's fixed" and "problem solved". And a higher decision bar.

On my test set, beautiful. So I gave the detector a mock exam. In jargon: a holdout. A set of fresh messages, never used during tuning. New exam questions, not the past papers.

The verdict. The "it's fixed" false alarms did disappear. But the detector dropped from catching 68 real cases out of 100 to just 48. My fix missed more than one real case out of two.

That is called overfitting. My tuning had memorized my examples instead of learning the problem. Like a student who recites past papers and sinks on a fresh question.

Worse: on the mock exam, no bar gives both a good catch rate and few false alarms. The knob I was turning has nothing left to give. The ceiling is structural.

All the precision belongs to the LLM

The numbers forced the conclusion on me. This detector will never be precise. It can, however, be exhaustive.

So I picked its role: a gate, not a judge. Bar at the lowest, to catch wide. On the mock exam: 88 real cases out of 100 caught, and 31 harmless messages out of 100 let through for nothing.

That noise is accepted, and written into the design. Because stage 2 judges behind it: the LLM, called on click, can read "it's fixed". It filters the negation and picks the right tool.

# The shipped config: a gate, not a judge
model     : multilingual-e5-small   # small, on CPU, zero training
threshold : lowest                  # catch wide, on purpose
anchors   : smalltalk + resolution  # "thanks", "it's fixed"...
result    : 88 real cases out of 100 · 31 false alarms out of 100

Last decision, the most counter-intuitive one: do not train the model. I have no real conversations. And training on generated messages means learning the generator's quirks. The mock exam had just shown me that trap. I will train once I have a few hundred real, anonymized messages.

The checklist before you ship an embedding detector

Before a similarity score goes to production, run the list.

[ ] Test on messages never seen before, not on the ones used for tuning
[ ] Count the false alarms, not just the good catches
[ ] Try negation phrases: "it's fixed", "no need anymore, thanks"
[ ] Compare against neutral phrases rather than setting an absolute bar
[ ] Give the embedding stage a gate's role: catch wide, let something else judge
[ ] Know how many false alarms the next stage can absorb
[ ] Never train on generated data