We are going to build a domain-specific support ticket classifier that applies transfer learning at inference time. Instead of fine-tuning model weights, we will adapt a general LLM to a custom support taxonomy using retrieval-augmented few-shot prompting. This is the fastest path to production for teams that need accurate routing without managing training infrastructure.
What you'll need
- Python 3.10 or higher
- An Oxlo.ai API key from https://portal.oxlo.ai
- The OpenAI SDK:
pip install openai - NumPy for similarity search:
pip install numpy
Step 1: Set up the Oxlo.ai client
Create a client pointing to Oxlo.ai. Because Oxlo.ai is fully OpenAI SDK compatible, this is a drop-in replacement.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
Step 2: Define the target taxonomy
Transfer learning needs a clear target domain. We will define a realistic support taxonomy for a B2B SaaS product. The model must map free-form tickets to one of these categories and severities.
TAXONOMY = {
"categories": ["Billing", "Authentication", "Integrations", "Performance", "Security"],
"severities": ["Low", "Medium", "High", "Critical"]
}
HISTORICAL_EXAMPLES = [
{
"text": "Our monthly invoice shows double the expected seat count. We need this corrected before the next billing cycle.",
"category": "Billing",
"severity": "Medium",
"reasoning": "Incorrect seat count impacts invoicing but does not block platform access."
},
{
"text": "SSO login fails for all users in the EU region with a 500 error from the identity provider callback.",
"category": "Authentication",
"severity": "Critical",
"reasoning": "Complete regional outage prevents any user access."
},
{
"text": "The Salesforce sync is missing custom field mappings for Opportunity objects.",
"category": "Integrations",
"severity": "Low",
"reasoning": "Missing field mappings degrade data quality but do not break the sync."
},
{
"text": "API latency spiked to 8 seconds for the reports endpoint starting at 14:00 UTC.",
"category": "Performance",
"severity": "High",
"reasoning": "Severe latency degrades core functionality for active users."
},
{
"text": "We noticed an exposed API key in a public log file linked from a support thread.",
"category": "Security",
"severity": "Critical",
"reasoning": "Exposed credentials require immediate rotation and audit."
}
]
Step 3: Embed the historical example bank
We encode the historical tickets with Oxlo.ai's embedding model so we can retrieve the most relevant examples for any new ticket. This is the knowledge substrate we transfer into the context window.
import numpy as np
def embed_texts(texts):
response = client.embeddings.create(
model="bge-large",
input=texts
)
return np.array([item.embedding for item in response.data])
example_texts = [ex["text"] for ex in HISTORICAL_EXAMPLES]
example_embeddings = embed_texts(example_texts)
Step 4: Retrieve relevant few-shot examples
When a new ticket arrives, we embed it and compute cosine similarity against the example bank. We pull the top three matches to condition the model on the most relevant domain patterns.
def retrieve_examples(ticket_text, k=3):
ticket_embedding = embed_texts([ticket_text])[0]
similarities = np.dot(example_embeddings, ticket_embedding)
top_indices = np.argsort(similarities)[-k:][::-1]
return [HISTORICAL_EXAMPLES[i] for i in top_indices]
Step 5: Write the system prompt
The system prompt anchors the general model to our domain. It defines the taxonomy, output schema, and reasoning style. We keep it strict so the model behaves like a specialist classifier.
SYSTEM_PROMPT = """
You are a support ticket classifier for a B2B SaaS platform.
Your job is to analyze the ticket and return a JSON object with exactly these keys:
- category: one of Billing, Authentication, Integrations, Performance, Security
- severity: one of Low, Medium, High, Critical
- reasoning: one sentence explaining why the label fits
Rules:
1. Use only the categories and severities listed above.
2. Base your decision on the semantic content of the ticket, not on urgency keywords alone.
3. Return only valid JSON. Do not wrap it in markdown fences.
"""
Step 6: Build the classifier function
We assemble the final message list. The system prompt sets the domain, the retrieved examples demonstrate the pattern, and the new ticket is the user query. Because Oxlo.ai uses flat per-request pricing, adding these few-shot examples does not require token math. You can see how this affects cost at https://oxlo.ai/pricing.
import json
def classify_ticket(ticket_text):
examples = retrieve_examples(ticket_text)
messages = [
{"role": "system", "content": SYSTEM_PROMPT}
]
for ex in examples:
messages.append({
"role": "user",
"content": f"Ticket: {ex['text']}"
})
messages.append({
"role": "assistant",
"content": json.dumps({
"category": ex["category"],
"severity": ex["severity"],
"reasoning": ex["reasoning"]
})
})
messages.append({
"role": "user",
"content": f"Ticket: {ticket_text}"
})
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=messages
)
return json.loads(response.choices[0].message.content)
Run it
Here is how we test the classifier on a held-out ticket that was not in the example bank. The model must generalize from the retrieved patterns.
if __name__ == "__main__":
new_ticket = (
"Our engineering team cannot push deployment logs to the Splunk integration "
"since this morning. The connector test returns a timeout."
)
result = classify_ticket(new_ticket)
print(json.dumps(result, indent=2))
Example output:
{
"category": "Integrations",
"severity": "High",
"reasoning": "A broken Splunk connector blocks the engineering team's observability pipeline, which is a core integration function."
}
Wrap-up and next steps
This pattern gives you a production-ready classifier in under a hundred lines. Because we are running on Oxlo.ai, you can scale up the example bank or switch to a heavier reasoning model like deepseek-r1-671b without rethinking your cost structure.
Two concrete next steps: wire this function into a webhook that listens to your support inbox, or expand the taxonomy and add vision by processing screenshot attachments with kimi-k2.6 or gemma-3-27b-it on Oxlo.ai.
