Free AI for my Chrome extension's users, with no API key and no surprise bill

The problem: I wanted the extension to see what I'm working on

Groupify is my Chrome tab organizer. When I start a new project I usually have Pinterest, the Next.js docs, Cloudflare and Vercel open at once, and grouping by website just splits them apart. What I wanted was one click to put those tabs in one group, with a sensible name such as “Website deploy”.

That needs AI. The question was which AI.

What I tried

• Chrome's built-in AI (Gemini Nano): it runs on the user's computer, free and private, so it's the ideal choice. But it needs a capable machine and a lot of free disk space. My own computer doesn't have it, and most users won't either.
• Asking users for their own API key: the simplest technically, but my users aren't developers. Nobody signs up for a key to rename tabs.
• My own API on OpenAI or Azure: possible, but neither has a real free tier for this. A budget alert only emails me; it doesn't stop anything, so a bit of abuse still turns into a bill.

I wanted a free allocation that stops when it's used up, like the Azure speech free tier I use in another extension.

Why Cloudflare Workers AI

Workers AI gives every account a free daily allocation. On the Workers Free plan, requests past it fail instead of being charged, so the most it can ever cost me is nothing. I only needed a tiny Worker in the middle that does three things: recognise my extension, limit how much each person can use, and turn the tabs into a prompt.

How I applied it

1. The Worker does two things

One endpoint names a set of tabs; the other sorts a pile of tabs into groups by task. The Worker stores nothing and logs nothing, so it never remembers what anyone sent.

2. Send only what's needed

The extension sends each tab's title and site name, never the full address, because addresses often carry tokens or private parameters. At most 40 tabs go in one request, and long titles are cut short.

3. Only my extension gets an answer

The Worker answers only my extension's origins (the store build and my development build) through CORS. That means the extension needs no new permission, so users see no new warning when it updates. An origin can be faked, so there's also a per-IP limit per minute on top of the free allocation's hard cap. The worst case is a day without AI.

4. A model that's good at Chinese and cheap

Most of my users read Traditional Chinese, so I picked a cheap open model that handles Chinese well and told it to skip its thinking output to save allocation. Naming a group uses very little, and at my current number of users the free allocation goes a long way.

5. Off by default, and clear about what's sent

Cloud AI is a switch in Settings, off by default, with a line saying tab titles and site names go to my server and nothing is kept. The extension always tries on-device AI first, then the cloud, then names by date, so whichever step fails, users never see an error.

Things that tripped me up

• Half a Chinese character: now and then the model put a broken character in the middle of a name. The Worker asks again once when that happens, and both sides strip any broken characters.
• JSON isn't guaranteed: not every model supports JSON mode, so I take the first JSON object in the answer and check every field, for example that tab numbers aren't repeated or out of range.
• The rate limit is approximate: Cloudflare counts per location, and in my test it only started refusing around the ninth request. Good enough against abuse, but not an exact counter.
• A development build has a different ID: an unpacked extension's ID comes from its folder path, so the allow list needs that ID too.

One more small trick: telling your changes from the extension's

Once Groupify could sort tabs automatically, I hit a problem: automatic sorting must not pull apart groups I made by hand. If I put my own blog's tabs in a group called “Me”, sorting by purpose shouldn't move them into “Reading”.

But Chrome doesn't record who changed a tab or a group, only how things are now. So the extension has to tell for itself. Before it changes any group, it writes down “the next few seconds of changes are mine”. A background worker listens to Chrome's tab and group events, and treats anything outside that window as the user's decision, including dragging tabs and renaming groups in Chrome's own tab bar.

// The extension's own changes are wrapped, so they never count as the user's
async function asExtension<T>(change: () => Promise<T>): Promise<T> {
  await storage.session.set({ ownChangesUntil: Date.now() + 15_000 })
  try {
    return await change()
  } finally {
    // Chrome's events can arrive a little late, so keep the window open for a second
    await storage.session.set({ ownChangesUntil: Date.now() + 1_000 })
  }
}

const isOwnChange = async () => (await storage.session.get('ownChangesUntil')) > Date.now()

// Background worker: every other change is the user's decision
tabs.onUpdated.addListener(async (tabId, change, tab) => {
  if (change.groupId === undefined || change.groupId === NO_GROUP) return
  if (await isOwnChange()) return
  const group = await tabGroups.get(change.groupId)
  if (group.title) rememberSite(siteOf(tab.url), group.title) // e.g. "amsyn.org" -> "Me"
})

tabGroups.onUpdated.addListener(async (group) => {
  const before = lastKnownTitle(group.id) // from a snapshot the worker keeps
  if (before === group.title || (await isOwnChange())) return
  renameRulesAndLock(before, group.title) // rules and locks follow the new name
})

// Reclassify: what the user decided goes first, automatic sorting gets the rest
function reclassify(tabs: Tab[], rules: SiteRule[], mode: Mode) {
  const movable = tabs.filter((tab) => !tab.pinned && !isLocked(tab.group))
  const { claimed, rest } = splitByRules(movable, rules) // sites with a rule
  return [...groupsFromRules(claimed), ...plan(mode, rest)]
}

A few details:

• Names, not IDs: a rule is “site → group name”. Chrome gives groups new IDs when it restarts, but names stay.
• Side panel actions are recorded directly: dragging a tab, naming a new group or renaming one is already the user acting, so there's nothing to guess.
• The latest decision wins: drag a site's tab into another group and the rule updates; Settings lists every rule so you can forget one.
• Your groups lock themselves: a group you changed by hand gets a lock, so Reclassify leaves it alone.

What improved

If usage grows or someone abuses it, the worst that happens is a day without AI, and the extension goes back to naming by date. For a side project with a handful of users, that's the setup I'm most comfortable with.

Let an AI set it up for you

If you want to add free AI that can't run up a bill to your own extension or small tool, paste this into your AI coding assistant:

Add an optional shared AI feature to my browser extension, using this design:

1. Server: a Cloudflare Worker using the Workers AI binding (free daily allocation, so on the
   Workers Free plan it fails instead of billing). Pick a cheap model that handles my users'
   languages. Expose only the endpoints the feature needs, keep no logs or state, and validate
   every request body (cap the number of items and the length of each string).
2. Access: answer CORS only for my extension's origins (store ID plus my unpacked dev ID, read
   from a variable), so the extension needs no new permission. Add a per-IP rate limit with the
   Workers rate-limiting binding.
3. Data: send only what the prompt needs (for tabs: title and hostname, never full URLs).
4. Answers: ask for JSON, but parse leniently (find the first JSON object, ignore any thinking
   block) and validate every field. Retry once if the answer contains U+FFFD.
5. Extension: an opt-in setting, off by default, that says exactly what is sent and where. Try
   any on-device AI first, then the Worker, then a non-AI fallback, so no error ever reaches the
   user. Treat every HTTP error or timeout as "no AI".
6. Tests for request parsing, answer parsing and the fallback order, plus a README for the
   Worker with the deploy steps.

Don't ask me for any secret. Stop after writing the code so I can review and deploy it.

Cover: Mail sorting room, U.S. Post by libraryofcongress, CC0 1.0.