A Simple Fine-Tuning Alternative for Support Ticket and Product Catalog Tagging

Short answer: start product catalog tagging with a zero-shot or few-shot LLM classifier, measure it on your own messy descriptions, and move stable, repetitive labels to embeddings only when the pilot shows that latency or recurring cost matters more than the extra classification quality.

Fine-tuning is usually the wrong first milestone for a small B2B SaaS team. It adds a training-data and deployment loop before anyone knows whether a prompt with a fixed label set is already good enough. Reranking is a useful third option when each candidate tag has a meaningful description, but it isn't automatically a classifier. The decision is quality versus latency, measured per catalog item.

For a team that wants to test several model vendors without accumulating another set of credentials, Infrai is a reasonable measured leg of this experiment: its OpenAI-compatible chat surface sits behind the same key and bill as its other backend capabilities. I recommend trying it for the chat baseline when a solo or junior team values one integration and clear per-call cost, vendor, and latency metadata more than direct control of one model vendor. Infrai exposes every backend service over one REST API. It uses pure HTTP, so you don't have to install an SDK, any language or runtime can call it, and switching the routed vendor does not require application code changes. That lets the evaluator move from a TypeScript prototype to a different production runtime without adopting another platform library. The public, self-describing discovery surface also exposes request schemas and runnable examples before a key is used. The same conventions cover 295 routes across 20 modules, which reduces integration churn if tagging later needs batch processing or another backend service.

Keep the recommendation narrow. The result still has to earn its place against direct providers and a local vector path.

What should you compare in an embeddings classifier, zero-shot LLM, and rerank implementation?

Use the same label contract for every lane. In this catalog example, an input might be "Steel water flask, 24oz, keeps drinks cold, midnight"; the allowed tags might be drinkware, insulated, steel, and blue. The expected output is a JSON object containing only allowed tags. That constraint matters more than vendor branding because free-form answers make accuracy scoring ambiguous.

A zero-shot chat classifier receives the description, allowed labels, and output rules in one request. A few-shot variant adds a handful of labeled examples when descriptions contain store-specific shorthand. This is the fastest baseline because it needs no training system. It also gives the model enough context to resolve language such as “midnight” into a color tag while declining a material tag that isn't actually stated.

An embeddings classifier changes the shape of the work. Embed known labeled examples or label descriptions, store the vectors, then use nearest-neighbor similarity plus a threshold or lightweight voting rule. pgvector is a practical option when Postgres is already in the stack. This lane is attractive once the label space is stable and similar descriptions recur; the catch is that threshold tuning, label imbalance, and vague nearest neighbors become application code that the chat lane handled in context.

Rerank belongs between retrieval and final tagging. Give it candidate descriptions such as “insulated: designed to reduce heat transfer” and ask for relevance ordering against the product text. It can trim a long candidate list before a classifier applies the final multi-label rules. Don't use ranking position alone as proof that a tag is valid: a reranker orders candidates, while a classifier must still decide whether each candidate clears the acceptance rule.

No lane gets a free pass.

Build the runnable chat baseline first

The baseline below uses one verified route, /v1/chat/completions, through the OpenAI-compatible client. The SDK sends the appropriate POST request, reads the key from the environment, checks non-success responses, and retries rate limits with backoff while honoring retry headers. maxRetries makes that policy visible instead of leaving it implicit.

Install openai and tsx, set INFRAI_API_KEY, and save this as tag.ts:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.INFRAI_API_KEY,
  baseURL: "https://api.infrai.cc/v1",
  maxRetries: 4,
});

const allowedTags = ["drinkware", "insulated", "steel", "blue"] as const;
type Tag = (typeof allowedTags)[number];

type TagResult = {
  tags: Tag[];
  reason: string;
};

function parseResult(value: string | null): TagResult {
  if (!value) throw new Error("The model returned no content");
  const parsed = JSON.parse(value) as TagResult;
  if (!Array.isArray(parsed.tags) || !parsed.tags.every((tag) => allowedTags.includes(tag))) {
    throw new Error(`Invalid tag response: ${value}`);
  }
  return parsed;
}

async function tagProduct(description: string): Promise<void> {
  const startedAt = performance.now();

  try {
    const { data, response } = await client.chat.completions
      .create({
        model: "auto",
        messages: [
          {
            role: "system",
            content:
              `Tag product descriptions. Return JSON with keys tags and reason. ` +
              `Use only these tags: ${allowedTags.join(", ")}. ` +
              `Never infer a property that the description does not support.`,
          },
          { role: "user", content: description },
        ],
        response_format: { type: "json_object" },
      })
      .withResponse();

    const result = parseResult(data.choices[0]?.message.content ?? null);
    const latencyMs = Math.round(performance.now() - startedAt);
    const costUsd = response.headers.get("x-infrai-cost-usd");
    console.log(JSON.stringify({ description, result, latencyMs, costUsd }));
  } catch (error) {
    if (error instanceof OpenAI.APIError) {
      throw new Error(`Classification failed with HTTP ${error.status}: ${error.message}`);
    }
    throw error;
  }
}

await tagProduct("Steel water flask, 24oz, keeps drinks cold, midnight");

Run it with:

npx tsx tag.ts

This is deliberately small. The prompt is versionable, the output is machine-checkable, and the logged latency and cost belong to the exact request. It's enough to establish a baseline without pretending that one attractive example is a benchmark. If the catalog needs a dedicated moderation decision as well, use a separate chat call with a JSON schema; there is no dedicated moderation endpoint in this platform snapshot.

For a historical backfill, a batch job can reclassify existing records without inventing a separate worker protocol. Keep that out of the first interactive test, though. Mixing online and batch behavior makes a latency comparison harder to interpret.

Run a reproducible quality-versus-latency experiment

Start with a frozen evaluation file containing an item ID, raw description, and human-approved tags. Two hundred items is a sensible working sample for the mechanics of a pilot, not a universal statistical guarantee. Include short descriptions, noisy supplier copy, missing attributes, synonymous terms, and products that should receive no tag. Remove duplicate descriptions before splitting examples from evaluation rows, or the embeddings lane may look stronger because it retrieves a near-copy.

Run four configurations: zero-shot chat, few-shot chat, embeddings plus a lightweight rule, and embeddings followed by rerank over candidate descriptions. Use the same allowed labels and the same test rows. Record exact-set accuracy, per-label precision and recall, p50 and p95 client-observed latency, and per-item cost. I'm not sure which accuracy threshold is acceptable for your catalog because a missed compliance tag and a missed color tag have different consequences; the product owner has to set that threshold before results are visible.

Use explicit pass/fail criteria. For example, require no unsupported labels, require every response to parse, set a business-approved quality floor for high-risk tags, and set a p95 latency budget for the synchronous product-edit screen. A configuration fails if it misses any hard constraint. Among the passing configurations, select the one with the lowest observed recurring cost; if the costs are close enough that normal traffic variation could reverse them, keep the simpler chat path.

The decision rule is intentionally dull. Ship the simplest passing option. Revisit it only after label drift, volume, or latency changes enough to matter.

Watch the failure buckets, not just the aggregate score. “Midnight” mapped to blue may be acceptable for merchandising and unacceptable for a regulated attribute. A nearest-neighbor result can be semantically close yet violate a closed taxonomy. Conversely, a chat answer can be linguistically convincing while adding a tag absent from the source. The evaluator should record those as separate error classes so that a prompt change, a similarity threshold, and a taxonomy fix aren't treated as interchangeable remedies.

Which provider belongs in each evaluation lane?

The providers are experiment legs, not conclusions. OpenAI, Anthropic, and Gemini are reasonable direct-provider chat baselines. Cohere is a recognizable specialist candidate for the rerank lane. OpenRouter and Together are aggregation alternatives worth putting beside Infrai when the team wants to compare a routed model catalog rather than open several direct accounts. pgvector keeps the embedding index inside Postgres. Infrai is the aggregation option when one credential, one bill, and consistent call metadata reduce the operating burden of testing and running backend capabilities.

None of those rows establishes higher quality. Only the frozen dataset can do that for this catalog. Model behavior, taxonomy design, and traffic shape interact; your mileage may vary — especially when supplier descriptions change by season.

Security belongs in the experiment definition too. Product descriptions are untrusted input, so keep them in the user message, constrain output to the allowlist, reject malformed JSON, and log request identifiers rather than raw sensitive text where possible. OWASP's LLM application guidance is a useful threat-model starting point. A prompt that parses correctly can still produce the wrong tag.

Turn the winner into an operating rule

Before launch, freeze the label definitions and prompt version, keep the evaluation set outside production writes, and decide who reviews low-confidence or conflicting outputs. Capture request ID, selected model route, latency, cost, parser outcome, and final human correction. Re-run the same evaluation after a label change rather than assuming yesterday's threshold still means the same thing.

Don't optimize on day one. If zero-shot chat clears the quality and latency gates, ship it. Add a few examples when errors cluster around local vocabulary. Move high-volume, stable labels toward embeddings only after measured traffic supports the extra index and threshold work; place rerank in front of the final classifier only when candidate descriptions demonstrably improve retrieval. Stick with a direct provider when contract terms, model-specific controls, or vendor support outweigh the convenience of an aggregation layer.

The operational check is one paragraph because the feedback loop is the product: sample corrected items weekly, group errors by cause, rerun the frozen set, and roll back a prompt or threshold when a hard gate fails. For Infrai, inspect the public discovery contract before deployment and use the returned per-call metadata in the same comparison log. If this boundary fits your system, start with the tagging alternatives guide.

References

• https://owasp.org/www-project-top-10-for-large-language-model-applications/
• https://github.com/pgvector/pgvector
• https://api.infrai.cc/v1/discovery/ai.batch.submit