Build an AI Support Agent That Handles Email and WhatsApp With Shared Context

A customer emails your support team about a damaged order. They describe the problem, provide the order number, explain what happened. Your AI agent replies, acknowledging the damage and asking for a photo. Then it reaches out on WhatsApp: "Hi Maria, could you send us a photo of the damaged headphones?"

Maria sends the photo on WhatsApp. The AI sees it, confirms a replacement, and sends a shipping confirmation back via email with the tracking number.

Two channels. One conversation. Zero repeated context.

This is the opposite of what most support systems do. In a typical setup, email and messaging are separate silos. The WhatsApp agent does not know about the email. The email agent does not know about the photo. The customer ends up re-explaining the problem on every channel.

This example builds a single Flask app that unifies Email and WhatsApp support with shared memory, so the AI agent carries context across both channels automatically.

What This Example Builds

Products Used

• Telnyx Email API -- receive inbound emails via webhook, send replies programmatically
• Telnyx WhatsApp Business API -- receive inbound WhatsApp messages (text + images) via webhook, send replies with template and freeform support
• Telnyx AI Inference API -- OpenAI-compatible chat completions with tool-calling (send_email, send_whatsapp, update_order, resolve_case)

All APIs authenticate with the same TELNYX_API_KEY. One platform, one bill.

Cross-Channel Identity Resolution

The cross-channel experience depends on one function:

def resolve_customer(email=None, phone=None):
    """Resolve a customer by email or phone. Returns customer dict or None."""
    if email:
        customer = lookup_customer_by_email(email)
        if customer:
            return customer
    if phone:
        customer = lookup_customer_by_phone(phone)
        if customer:
            return customer
    return None

When an email arrives from maria.santos@example.com, the email webhook calls resolve_customer(email="maria.santos@example.com"). When a WhatsApp message comes in from +31626781895, the WhatsApp webhook calls resolve_customer(phone="+31626781895"). Both return the same customer record.

The customers table uses unique partial indexes:

CREATE UNIQUE INDEX idx_cust_email ON customers (email) WHERE email IS NOT NULL;
CREATE UNIQUE INDEX idx_cust_phone ON customers (phone) WHERE phone IS NOT NULL;

That shared identity is the foundation. Without it, the WhatsApp handler has no way to know that the person sending a damage photo is the same person who emailed about a damaged order.

The Cross-Channel Context Store

Every message from every channel is stored in one conversations table:

CREATE TABLE conversations (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    customer_id TEXT    NOT NULL,
    channel     TEXT    NOT NULL,    -- 'email' or 'whatsapp'
    role        TEXT    NOT NULL,    -- 'user' or 'assistant'
    content     TEXT    NOT NULL,
    timestamp   TEXT    NOT NULL
);

The store_message() function writes every interaction:

def store_message(customer_id, channel, role, content):
    conn = sqlite3.connect(DB_PATH)
    cur = conn.cursor()
    cur.execute(
        "INSERT INTO conversations (customer_id, channel, role, content, timestamp) "
        "VALUES (?, ?, ?, ?, ?)",
        (customer_id, channel, role, content, datetime.now(timezone.utc).isoformat()),
    )
    conn.commit()
    conn.close()

And get_conversation_history() retrieves the full timeline across all channels:

def get_conversation_history(customer_id):
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row
    cur = conn.cursor()
    cur.execute(
        "SELECT channel, role, content, timestamp FROM conversations "
        "WHERE customer_id = ? ORDER BY timestamp",
        (customer_id,),
    )
    rows = cur.fetchall()
    conn.close()
    return [dict(r) for r in rows]

The history for a given customer might look like:

[email]    user:      Subject: Damaged Order -- ORD-2024-7891 ...
[email]    assistant: Subject: Re: Damaged Order -- ORD-2024-7891 ...
[whatsapp] assistant: Hi Maria! Could you send us a photo of the damaged headphones?
[whatsapp] user:      Here's the photo. The left ear cup is completely cracked.
[whatsapp] assistant: Thank you for the photo. We've approved a replacement.
[email]    assistant: Subject: Replacement Shipped -- ORD-2024-7891 ...

This timeline is what the AI reads before generating any response. It is the memory that makes context-carrying work.

The AI Brain -- Tool-Calling

The AI agent uses Telnyx Inference API with four tools:

TOOLS = [
    {"type": "function", "function": {
        "name": "send_email",
        "description": "Send a formal email to the customer. Use for acknowledgments, shipping confirmations, and case summaries.",
        "parameters": {
            "type": "object",
            "properties": {
                "to_email": {"type": "string"},
                "subject": {"type": "string"},
                "body": {"type": "string"},
            },
            "required": ["to_email", "subject", "body"],
        },
    }},
    {"type": "function", "function": {
        "name": "send_whatsapp",
        "description": "Send a WhatsApp message to the customer.",
        "parameters": {
            "type": "object",
            "properties": {
                "to_number": {"type": "string"},
                "text": {"type": "string"},
                "template": {"type": "string"},
                "template_params": {"type": "array", "items": {"type": "string"}},
            },
            "required": ["to_number"],
        },
    }},
    {"type": "function", "function": {
        "name": "update_order",
        "description": "Update an order's status or details.",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "status": {"type": "string"},
                "damage_photo_url": {"type": "string"},
                "replacement_tracking": {"type": "string"},
            },
            "required": ["order_id"],
        },
    }},
    {"type": "function", "function": {
        "name": "resolve_case",
        "description": "Mark the support case as resolved.",
        "parameters": {
            "type": "object",
            "properties": {
                "summary": {"type": "string"},
            },
            "required": ["summary"],
        },
    }},
]

The system prompt guides channel selection:

Use EMAIL for: formal acknowledgments, shipping confirmations with tracking numbers, case summaries.
Use WHATSAPP for: quick updates, requesting photos of damage, short confirmations.

The agentic loop continues calling tools until the model stops requesting them:

while True:
    data = call_inference(messages)
    choice = data["choices"][0]
    msg = choice["message"]

    if not msg.get("tool_calls"):
        break  # No more tools to call -- done

    messages.append(msg)
    for tc in msg["tool_calls"]:
        fn = tc["function"]
        tool_name = fn["name"]
        tool_input = json.loads(fn.get("arguments", "{}"))
        result = execute_tool(tool_name, tool_input, customer)
        messages.append({"role": "tool", "tool_call_id": tc["id"], "content": result})

The AI decides the workflow. When Maria emails about damaged headphones, the AI chooses to: (1) acknowledge via email, (2) update the order status, and (3) reach out on WhatsApp requesting a photo. No hardcoded sequence.

WhatsApp Business API: Templates vs Freeform

WhatsApp enforces a messaging window: businesses can only send freeform messages within 24 hours of the customer's last message. Outside that window, you must use an approved template.

The send_whatsapp() function handles both:

def send_whatsapp(to_number, text, template=None, template_params=None, media_url=None):
    payload = {"from": WHATSAPP_NUMBER, "to": to_number}

    if template:
        payload["whatsapp_message"] = {
            "type": "template",
            "template": {
                "name": template,
                "language": {"policy": "deterministic", "code": "en_US"},
                "components": [{"type": "body", "parameters": [
                    {"type": "text", "text": str(p)} for p in template_params
                ]}],
            },
        }
    elif media_url:
        payload["whatsapp_message"] = {
            "type": "image",
            "image": {"link": media_url, "caption": text},
        }
    else:
        payload["whatsapp_message"] = {"type": "text", "text": {"body": text}}

The execute_tool() function includes fallback logic: if a template message fails, it retries with plain text:

try:
    send_whatsapp(to_number, text, template=template, template_params=template_params)
except requests.HTTPError:
    if template and text:
        send_whatsapp(to_number, text)  # Fall back to plain text

This graceful degradation is important in production -- template approval status and window state can be ambiguous.

The Dashboard

The Flask app publishes Server-Sent Events to a browser dashboard with four panels:

• Customer -- profile card with name, email, masked phone number
• Email Thread -- subject and body cards, inbound (blue border) and outbound (green border)
• WhatsApp -- phone-frame UI with chat bubbles, delivery ticks, image thumbnails
• Order & Timeline -- status badge transitions (delivered → damage_reported → replacement_shipping → resolved), damage photo, chronological event log

Every significant event -- identity resolution, tool calls, channel messages, order updates -- is published via sse_publish() and rendered in real time.

Try the Demo

The demo runs "Maria's Damaged Headphones" without any Telnyx credentials:

git clone https://github.com/team-telnyx/telnyx-code-examples.git
cd telnyx-code-examples/email-whatsapp-support-python
pip install -r requirements.txt
python demo/demo_server.py

Open http://localhost:5050/ and click "Run Demo Journey."

The scripted scenario:

1. Maria emails about damaged headphones (ORD-2024-7891)
2. AI acknowledges via email with reference SR-2024-0456
3. AI reaches out on WhatsApp requesting a damage photo
4. Maria sends the photo on WhatsApp with a description
5. AI confirms replacement on WhatsApp
6. AI sends shipping confirmation email with tracking number
7. Case resolved

Run It With Real Credentials

To run against real Telnyx APIs:

cp .env.example .env

Edit .env:

TELNYX_API_KEY=KEY_your_api_key_here
TELNYX_EMAIL_FROM=support@yourdomain.com
WHATSAPP_NUMBER=+15551234567
MESSAGING_PROFILE_ID=your_messaging_profile_id
AI_MODEL=meta-llama/Llama-3.3-70B-Instruct

Configure Telnyx resources:

1. Email domain -- verify a sending domain for the Email API, configure DKIM/SPF
2. WhatsApp number -- register a phone number for WhatsApp Business via the Telnyx Portal
3. Messaging Profile -- create one and assign the WhatsApp number
4. Webhook URLs -- email webhook to https://your-domain.com/webhooks/email, WhatsApp webhook to https://your-domain.com/webhooks/whatsapp

Before Using in Production

This is a working prototype. Areas to address:

1. Webhook signature verification. Verify the Telnyx signature on every inbound webhook to prevent spoofed events.
2. PostgreSQL. Replace SQLite for concurrent writes under production load.
3. WhatsApp template approval. Submit templates to Meta for approval before using in production.
4. DKIM for email deliverability. Configure DKIM, SPF, and DMARC records for your sending domain.
5. Task queue. Replace background threads with Celery for reliability at scale.
6. Media handling. Download and store WhatsApp media via the Telnyx media retrieval endpoint instead of using placeholder URLs.

Why This Pattern Is Useful

The damaged-order scenario is specific, but the pattern generalizes. Any business that communicates with customers across email and messaging faces the same problem: siloed channels, repeated context, fragmented conversation history.

The fix is identity resolution plus shared memory. If the system knows that maria.santos@example.com and +31626781895 are the same person, and every message from every channel is stored against that person's ID, then any channel can pick up where the last one left off.

Email for formal correspondence. WhatsApp for quick updates and photo collection. One AI brain that decides which channel fits each step. One conversation history that spans both.

Resources

• Full source code
• Telnyx Email API
• Telnyx WhatsApp Business API
• Telnyx AI Inference API
• Telnyx Portal