Building Amit.AI, Part 1: The Frontend

tldr:

• Most questions never reach an LLM. A tiny client-side matcher answers them instantly, and when it is unsure, it stays quiet and defers.
• Wrong-but-confident is worse than slow-but-right, so every threshold is tuned to defer.
• SSE over POST needs fetch and a hand-rolled parser, because EventSource cannot POST.
• A sleeping free-tier backend is handled with a warm-up ping during the boot animation, which is also why the boot animation exists.

Go to the chat page and you are talking to me, in terminal form. This is Part 1 of how I built it: the browser side. Part 2 covers the server.

I'll skip the secrets and the exact knobs. This is the fun-parts tour, not a blueprint.

1. The best LLM call is the one you don't make

My first version of Amit.AI was a static FAQ matcher. No AI, just "if the question looks like this, show that answer". It was fast and free, and also confidently wrong in creative ways.

Then I added a real LLM backend, and the obvious move was to send everything to it. I didn't, for three reasons:

• Free-tier LLM quotas are small, and visitors keep asking "what do you do?" and "are you available?"
• The backend sleeps when nobody is around. A FAQ answer doesn't need it awake.
• Instant answers feel better than streamed ones.

So the chat is two tiers. Tier 0 lives in the browser and answers what it is sure about. Everything else gets handed to the backend, which checks its own cache before it spends a model call, and tries a fallback model if the first one fails. Here is the whole trip of one question:

This post covers the first box.
Part 2 covers everything below it.

Basically a tiny, free intent router

Looking back, the Tier 0 matcher is a poor man's intent router: a cheap first stage that decides what to do with a message before an expensive LLM gets involved.

There are real, proper versions of this idea:

• Jev from TypeSafe AI is pitched as a model for decisions inside software rather than chat. You ask it a structured question, and it returns a typed decision with a probability, so your code can act when confidence is high and escalate when it's low.
• Laya is an open-source (Apache-2.0) "System 1" decision engine that doesn't generate text at all. It answers choice, score and yes/no questions about a piece of text, and ships presets for routing and prompt guardrails.

Mine is kind of in the same shape with none of the ML. No model, no training, no GPU, just a token-overlap score, a few thresholds and a JSON file. It runs in the visitor's browser for free. The "confidence" is whether a message clears the gates below, and low confidence means "send it to the big model".

It's much dumber than either of those, and it's also ₹0 and instant. For a portfolio chatbot with a few dozen FAQ topics, that trade is fine.

2. A matcher that knows when to shut up

The matcher returns one of three things: guard, faq or defer. There is deliberately no fourth "I'm not sure" answer. A defer is silent. The visitor just sees the typing indicator a little longer.

The first version used one similarity threshold, and it failed in a way I found hilarious. Ask "How good are you at Java?" and it happily returned my React answer, because the other words overlapped.

The fix was to stop trusting a single score. An FAQ answer only goes out when it clears every gate:

• An exact phrasing match always wins.
• Otherwise, most of the visitor's content words must be covered by one FAQ phrasing, so "Java" can't hide behind "good" and "you".
• At least two meaningful words must overlap.
• The winner must beat the runner-up by a clear margin. Two close candidates mean the matcher is guessing, so it defers.

Words like "that", "more" and "those" force a defer on any turn after the first, because the matcher has no memory. "Tell me more" means something different every time.

My rule while tuning: wrong-but-confident is worse than slow-but-right. Every change I made pushed towards deferring.

The stopword rabbit hole

A stopword is a filler word the matcher ignores when scoring, like "what", "are", "the" or "you". Only the remaining content words count towards the overlap and coverage gates above.

Here's the problem. My FAQ is written in the second person, because the terminal is me: "What are your skills?" But visitors don't know that. They look at a site about a person called Amit and ask "what are his skills?" or "what is Amit's stack?".

Walk through the first one:

• The FAQ phrasing "what are your skills" reduces to one content word: skills. ("what", "are" and "your" are all stopwords.)
• The visitor's "what are his skills" reduces to his and skills, because his wasn't a stopword.
• Only one of the visitor's two content words is covered, so coverage is 0.5, below the 0.7 gate. The matcher defers, and a question it should have answered instantly goes to the LLM.

The fix was to add his, him, her, their, amit and mondal to the stopword list. They carry no meaning on a site about exactly one person, so they should be invisible to scoring. Now both phrasings reduce to skills, and it's an instant answer.

There's a flip side. Pronouns like "it", "that" and "those" normally mean "I'm pointing at something said earlier", which is why they force a defer. But "he" and "his" are different here. On this site they can only mean me, so I deliberately left them out of that list. A question like "what are his skills?" is standalone, even though it contains a pronoun.

Small things like this are the difference between a matcher that works in a demo and one that works for strangers.

Here is that funnel as a short video: how many questions actually reach the LLM.

3. Streaming without EventSource

The backend streams its answer as Server-Sent Events, so tokens appear as they are generated. Browsers have a built-in EventSource for this, but it can only make GET requests, and I need to POST the conversation.

So I used fetch and read the body stream myself. The core is about a dozen lines:

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true }).replace(/\r\n/g, '\n');

  let sep: number;
  while ((sep = buffer.indexOf('\n\n')) !== -1) {
    const frame = parseFrame(buffer.slice(0, sep));
    buffer = buffer.slice(sep + 2);
    // token → append to the bubble, done → resolve, error → throw
  }
}

SSE frames are separated by a blank line, and a network chunk can end in the middle of one. So you buffer, split on \n\n, and keep the leftover for the next read. That one detail is where most hand-rolled parsers break.

There are three event types: token, done and error. The UI pushes an empty AI bubble on the first token and appends to it. The typing indicator stays up until that first token arrives, so the wait never looks like a freeze.

One safety habit: the message text is rendered with {{ }}, never v-html. Model output is untrusted input, and I'm not letting a chatbot inject HTML into my own page.

4. The sleeping backend

My backend runs on a free tier that goes to sleep when idle. The first request after a nap can take a long time.

I handled it in layers:

1. Tier 0 covers the common stuff. Most visitors never notice the backend is asleep.
2. A warm-up ping. The moment the chat page loads, the browser quietly pings the backend's health endpoint. By the time you have typed a question, it's often awake.
3. A generous first-token timeout, then an overall timeout. After that, a friendly offline message instead of a spinner of doom.

The boot animation (those fake > initializing... lines) exists partly because it looks cool, and partly because it buys the ping a few seconds. I'm not going to pretend I planned that first.

5. Guardrails, on both sides

A guardrail here is a cheap pattern check that runs before any answer is produced. If a message looks like something the chat shouldn't engage with, it gets a canned reply, and neither the FAQ nor the LLM ever sees it. There are two kinds:

• Jailbreak-style messages, like "ignore all previous instructions" or "pretend you are...". The reply politely says this terminal is scoped to my professional profile.
• Requests for private details, like a phone number, home address or date of birth. The reply says that isn't shared here and points to the Contact page.

The same idea exists twice, for different reasons:

So the frontend copy makes the common case fast and nice, and the backend copy is the one that actually protects anything. If they ever disagree, the server wins.

The hard part wasn't writing patterns, it was not blocking innocent questions. My first versions were too blunt:

• A bare number blocked "what's the number of projects you've built?"
• A bare address blocked "how did you address performance issues?"
• A bare dob matched inside "A*dob*e".

The fix is two rules: always use word boundaries (\b), and match the phrase, not the word. "phone number" and "home address" are the requests I care about, "number" and "address" alone are not. A guardrail that annoys real recruiters is worse than no guardrail, so when in doubt I let the message through to the server and the prompt, which are better at judging context.

6. Small touches that earn their keep

• A frontend answer cache. If you ask the same question twice in a tab, the second answer comes from sessionStorage and never leaves the browser. Storage can throw or be blocked, so an in-memory Map is the source of truth and storage is a bonus.
• History trimming. Only the last few turns go along with each deferred message. If you ask the same question twice, only the latest pair is kept, so the model isn't fed its own repeats.
• A fake thinking pause on locally answered messages. Replying in 5ms feels wrong, and a short, slightly random delay feels more human. It is a lie, and I'm at peace with it.
• A character counter ring on the input that fills as you approach the message limit, so the backend never has to say no.

What's next

That's the browser half. The interesting questions are on the other side: how do you ground an LLM on a bio without a vector database, what happens when the free model hits its limit, and how do you stop people making me say silly things?

That's Part 2.