How I Built an AI Phone Receptionist: Webhooks, Stateless Turns and Structured LLM Output

Small businesses miss a lot of calls. A plumber is under a sink, a dentist's front desk is checking in patients, a salon owner is mid-colour — and the caller hangs up and dials the next business on Google.

I built an AI phone receptionist for small businesses in Canada (Aethely) to cover exactly those calls. This post walks through how a call actually flows, the design decisions that mattered, and a few things I'd tell anyone building something similar.

The shape of a call

The whole thing is a loop of webhooks between the phone network and a small serverless backend:

1. A customer calls the business. If nobody answers (or it's after hours), the carrier forwards the call to a phone number we control.
2. The telephony provider hits our "incoming call" webhook. We reply with instructions: say a greeting, then listen for speech.
3. The caller speaks. The provider transcribes it and posts the text to our "turn" webhook.
4. We ask an LLM what to say next, given the business's details and the conversation so far.
5. We reply with the next line and listen again. Repeat until the call ends.
6. When the call ends, we text the owner a summary: who called, what they need, how urgent it is, and what was booked.

Every step is a stateless HTTP request, which makes it cheap to run on serverless functions.

Step 1: Greeting and listening

The provider (Twilio, in my case) expects XML instructions. The greeting is deliberately honest — it says it's an AI:

<Response>
  <Gather input="speech" action="/api/voice/turn?s=STATE"
          speechTimeout="auto" language="en-CA" actionOnEmptyResult="true">
    <Say>Hi, thanks for calling Northside Heating. I'm the virtual assistant,
    an AI. How can I help?</Say>
  </Gather>
</Response>

actionOnEmptyResult="true" matters: if the caller says nothing, we still get a webhook, so we can handle silence ourselves instead of the call just ending.

Step 2: Keeping state without a database

Each turn is a separate request, so where does the conversation history live? You could use a database or a cache keyed by call ID. I went simpler: compress the history and carry it in the callback URL.

import { deflateRawSync, inflateRawSync } from 'node:zlib';

export const packState = (obj) =>
  deflateRawSync(Buffer.from(JSON.stringify(obj))).toString('base64url');

export const unpackState = (s) =>
  JSON.parse(inflateRawSync(Buffer.from(s, 'base64url')).toString());

A typical call is under ten turns, so the compressed history stays small. No database, no cleanup jobs, and every function invocation has everything it needs.

(If you do this, cap the number of turns and validate everything you unpack — it came from a URL.)

Step 3: Asking the model for structured output

The key decision: don't ask the model for free text. Ask for a JSON object with both what to say and what it has learned:

const TURN_SCHEMA = {
  type: 'object',
  properties: {
    say: { type: 'string' },          // next line to speak
    end_call: { type: 'boolean' },     // conversation is finished
    urgent: { type: 'boolean' },       // matches the owner's urgent rules
    transfer: { type: 'boolean' },     // put through to the owner now
    details: {
      type: 'object',
      properties: {
        name: { type: 'string' },
        callback: { type: 'string' },
        need: { type: 'string' },
        address: { type: 'string' },
        booking: { type: 'string' },
      },
    },
  },
  required: ['say', 'end_call', 'urgent', 'transfer', 'details'],
};

With a schema, the backend never has to parse prose. transfer: true becomes a <Dial> to the owner's mobile, end_call: true hangs up and sends the summary, and details fills the text message — all deterministic code paths.

Step 4: The system prompt is mostly rules

The prompt contains the business's hours, services, prices it's allowed to share, booking windows and what counts as urgent. But the most important part is the list of things it must not do:

• Only answer from the information provided. If you don't know, say so and take a message.
• Never give medical, legal or financial advice.
• Never invent prices, availability or promises.
• Keep replies short — this is a phone call, not a chat window.

Owners trusted the system once they saw those limits. Features didn't build trust; boundaries did.

Step 5: Handling the awkward parts

Silence. If the caller says nothing, ask once ("Sorry, are you still there?"). On the second silence, end politely and still send the owner whatever was captured.

Short calls and hang-ups. A status callback fires when the call ends. If it lasted only a few seconds, or was never answered, we send the caller a friendly text from the business's number so the lead isn't lost.

Security. Every webhook verifies the provider's request signature (an HMAC over the URL and parameters) before doing anything. Otherwise anyone could post fake "calls" to your endpoints.

Latency. Phone conversations feel broken if replies take too long. Keep the model's output short, keep the prompt cached, and do the summary SMS after the call — not during it.

Step 6: The summary is the product

The moment that made owners say "oh, I get it" wasn't the call itself. It was getting a text like this seconds after putting down a wrench or finishing with a client:

URGENT — New call for Northside Heating
Caller: Jordan · (780) 555-0142
Needs: Furnace not working, house is cold
Booked/requested: Today 2–4 PM

No voicemail to dig through. Everything else in the system exists to produce that message reliably.

What I'd tell anyone building this

• Disclose that it's an AI. It's the right thing to do, and callers mostly care that someone helpful picked up.
• Use structured output so the model can't break your control flow.
• Design the failure cases first: silence, hang-ups, "I don't know", and emergencies.
• Make call forwarding painless. Most owners want to keep their number. A clear guide to carrier forwarding codes (we wrote one for Canadian carriers) removes the biggest setup hurdle.

If you're curious what this looks like for a real business, there's a live demo and pricing on the Aethely AI receptionist page. I'm happy to answer questions about the architecture in the comments.