Your AI Feature's First User Should Be a Script, Not a Human

Last week I watched a demo of an AI travel assistant running on an Android phone. The model answered three questions in a row, the room nodded, and then the presenter rotated the device to show a map. The activity restarted, the conversation vanished, and the demo ended on the home screen. The model had worked perfectly while the feature had failed completely, and nobody in the room could tell you which was true.

The current agent trend is full of advice about remembering decisions instead of just data. Reasoning ledgers, memory layers, and state machines are excellent ideas for a server that never sleeps. On a phone, the ledger itself gets backgrounded, killed, or stripped of permissions before it writes its first entry. I think the first user of every mobile AI feature should be a script, not a human, because a script will not forgive the lifecycle.

My position is simple: if your AI feature cannot pass a scripted lifecycle probe, you do not have a demo problem, you have a product problem. Humans forgive lost conversations, invisible retry buttons, and silent failures. Scripts forgive nothing, which is exactly why you should let them go first.

Why Demos Lie to You

A demo is a story with a single happy path. The network is up, the permission dialog was accepted months ago, and the activity stays in the foreground for exactly ninety seconds. Your users live in the other world where airplane mode, battery saver, and a swipe-away gesture arrive without warning.

Every mobile AI failure I have debugged happened at a lifecycle boundary. The request was in flight when the app went to the background, the model returned a response that the killed activity could not deliver, or a permission revoke left the SDK in a state that only a restart could fix. None of those failures show up in a chat demo, and all of them show up in a probe.

The Probe Workflow

To run this kind of probing you need two cheap things: model access for thousands of small calls and a server that collects the results. I used MonkeyCode for both, because its current offer happens to match the workflow. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator's published terms, as of this writing, include free model access with a 10 million token allocation and a free server option, so treat the numbers as a snapshot and verify them before you build anything.

The point is not the vendor. The point is that the workflow should cost almost nothing, run every night, and produce a table you can argue about. The code below is generic, so you can point it at any model endpoint and any collector you already own.

Step 1: Write a Probe That Cannot Lie

The probe sends one fixed prompt and checks that the response contains the expected token. No chat, no creativity, no ambiguity.

# probe.py — run this on the device or emulator under each lifecycle condition
import time
import requests

PROBE_PROMPT = 'Reply with the word READY and nothing else.'
COLLECTOR = 'https://your-free-server.example/probe'  # your MonkeyCode server URL

def run_probe(label: str, model_url: str) -> dict:
    start = time.perf_counter()
    try:
        response = requests.post(model_url, json={'prompt': PROBE_PROMPT}, timeout=15)
        latency_ms = round((time.perf_counter() - start) * 1000, 1)
        text = response.json().get('text', '')
        result = {
            'label': label,
            'status': 'pass' if response.ok and 'READY' in text else 'fail',
            'latency_ms': latency_ms,
            'tokens': response.json().get('tokens', 0),
            'error': None,
        }
    except Exception as exc:
        result = {
            'label': label,
            'status': 'fail',
            'latency_ms': None,
            'tokens': 0,
            'error': str(exc),
        }
    requests.post(COLLECTOR, json=result)
    return result

The fixed prompt matters more than it looks. A free-form conversation can pass a hundred times while a single-word contract catches regressions on the first run.

Step 2: Collect Results on the Free Server

The collector is a tiny endpoint that stores each probe result in a table. You can read the table in the morning and argue about the numbers instead of arguing about feelings.

# server.py — deploy this to your MonkeyCode free server
from fastapi import FastAPI, Request
import sqlite3

app = FastAPI()

@app.post('/probe')
async def record_probe(request: Request):
    data = await request.json()
    con = sqlite3.connect('probes.db')
    con.execute(
        "INSERT INTO probes (label, status, latency_ms, tokens, error, ts) "
        "VALUES (?, ?, ?, ?, ?, datetime('now'))",
        (data['label'], data['status'], data.get('latency_ms'),
         data.get('tokens'), data.get('error')),
    )
    con.commit()
    return {'ok': True}

SQLite is enough for a single device matrix, and you can move to Postgres when the table outgrows you.

Step 3: Run the Lifecycle Matrix

This is the part that separates a probe from a benchmark. A benchmark measures speed on a clean device, while a probe measures survival across the transitions your users actually trigger.

Run the matrix on at least one real device per OS, because the emulator will not reproduce radio behavior. Schedule the probe nightly with a cron job or a WorkManager task, and let the collector accumulate a history.

What the Results Actually Mean

A pass means the feature survived one specific transition on one specific device with one specific OS version. It is not a license to stop testing, but it is a tripwire that catches regressions before your users do. A fail means you have a reproducible bug with a device name, a timestamp, and a stack trace, which is more than most bug reports ever give you.

The token math works in your favor here. A probe call costs a fraction of a chat session, so a 10 million token allocation covers thousands of nightly runs across a small device matrix. That is the correct use of free model access: spend it on the boring, repetitive, honest calls that tell you when the feature broke.

Who Should Not Use This Workflow

Teams without access to a real device should not pretend an emulator is enough, because radios, thermal throttling, and permission behavior differ. Teams shipping server-side AI can skip the lifecycle matrix entirely, since the phone is not the failure domain. Teams that cannot act on failures should also skip it, because a probe that nobody reads is just another dashboard with a green light.

The Bottom Line

Your demo will always work, and your users will not always forgive. Put a script in front of your mobile AI feature before a human ever sees it, and spend your free tokens on the thousands of small probes that tell you the truth. If you want to run this tonight, MonkeyCode's free server and model access are enough to start; verify the current terms, then let the script go first. The chat window can wait.