This is a submission for the MLH x DEV Writing Challenge.
At LA Hacks 2026, our team of three built FirstHour in 24 hours and won 2nd place in two sponsor challenges: StepFun's "Get It Done with Step 5 Preview" and Espressif's "Best Use of Espressif." This is the story of the part I owned: what happens between "the box heard a fall" and "someone is at the door."
What I Built
When an older adult falls alone in the bathroom, the hardest part is often what comes next: lying on the floor, unable to reach a phone, while nobody knows. FirstHour is built for that first hour. It's a camera-free box for the bathroom, built by Anmol Hurkat, Sakshi Kuldeep Dhariwal and me.
Here's what it does, end to end:
- Detect. An Espressif ESP32-S3-BOX-3 listens for the thud and feels the vibration. A Raspberry Pi 5 running QNX confirms the fall and starts an alarm on the device.
- Check in. A voice asks the person if they're okay, and listens. "I'm fine, but my hip hurts" is not a cancel, and silence always escalates.
- Triage. The answers become a short triage record (conscious, pain, head strike, blood thinners), refined by StepFun's step-5-preview, with a rule-based fallback so nothing waits on the model.
- Tell the family. A family member gets a text with that record and can ask questions, say they're coming, or let help in through a guarded door unlock.
Sakshi's post covers the whole project and how the demo runs. This one goes deep on step 4.
I was the comms lead. My job was the family's side of the incident: getting the right message to the right person, getting their answer back, and making sure none of it could be used to open the door when it shouldn't. I also ran our live messaging and calling tests, spent a lot of the night testing the voice check-in, and worked on the pitch, demo and video.
It sounds simple: send a text when someone falls. It isn't, for three reasons:
- The family's reply has to come back to the box, reliably, even if a process restarts.
- More than one person might get the alert. If everyone drives over, or everyone assumes someone else will, both are bad outcomes.
- The text can open a door, so the messaging layer is part of the security model.
Demo
Here's the flow from the caregiver's phone. I ran it live on real iMessage, playing the caregiver:
1. The alert. About 0.3 s after the incident escalates, Photon accepts a message like this:
FALL ALERT: Margaret (Bathroom). Down 00:32. Conscious. Right hip pain, cannot move. No head strike. NOTE: on a blood thinner. How soon can you be here?
It deliberately has no door code in it. The blood-thinner line comes from her profile, never from the AI.
2. The claim. The first caregiver to reply "on my way, 10 min" claims the incident:
- The box reads it aloud to the person on the floor: "Sarah is about 10 minutes away."
- Every other caregiver gets: "Sarah is on her way. Please stand by."
- Only Sarah gets the one-time door code.
Hedges like "I might come" never count as a claim.
3. Questions. A caregiver can text "is she still talking?" and get an answer built only from the incident's facts.
4. Nobody answers? After 20 s with no claim (or right away if there was a head strike), the backend phones the first caregiver. It's an ElevenLabs voice agent on a Photon phone number. When the call ends, the ETA they gave is posted back into the text thread: "You said 5 minutes on the call. Reply UNLOCK with the code at the door."
5. The door. UNLOCK <code> from the claiming caregiver opens the latch (simulated for the demo), but only after the Pi verifies the signed command and the latch confirms. A stranger's number, even with the right code, is refused.
Live numbers from the event:
- A real reply reached the backend 3.5 s after the phone sent it.
- The door-code text went out 0.3 s after the claim.
Code: github.com/saksh12/firsthour · Devpost: devpost.com/software/firsthour
How it works
Platforms we used: Photon 2.0 beta (iMessage + phone calls), StepFun step-5-preview, ElevenLabs (voice and the call agent), Espressif ESP32-S3-BOX-3, QNX on a Raspberry Pi 5, FastAPI, and Sentry.
The messaging layer has two pieces.
A TypeScript gateway (comms-gateway/) on Photon's 2.0 beta SDK:
- It sends alerts and polls for replies.
- Every reply it forwards to the backend is HMAC-signed and carries Photon's message id, so a replayed message is ignored. The backend trusts a sender's phone number only if it came from the gateway.
- It saves its inbound cursor and each caregiver's conversation to disk. The cursor only moves forward once the backend has accepted the message, so restarting the gateway mid-incident loses no replies.
A caregiver coordinator in the Python backend:
- It reads each reply and decides whether it's a claim, an ETA, an arrival, a question, or a message for the person.
- It tracks who is coming.
- It reports delivery honestly. "Sent" means Photon accepted the message, never "delivered", because we can't actually know that.
I wrote the gateway. For the coordinator, I proposed caregiver Q&A in the thread and several of the demo moments, drove Devin as a coding agent to build the first version (claims, Q&A, honest delivery, voice escalation, a restart-safe gateway), and got it merged against a teammate's parallel changes and a 370+ test suite. Anmol wired the Photon number to the ElevenLabs voice agent, then reworked the flow into what you see above: ask for an ETA first, and text the door code only to whoever is coming.
Where the AI is, and isn't
We used StepFun's step-5-preview for three jobs:
- refining the triage record
- rewriting a caregiver's text into one calm sentence to read aloud
- answering caregiver questions
Each call has a hard deadline and a fallback, because we measured triage at 8–22 s on the day. Nothing waits on the model.
The rules we held to:
- The model never sees the door code or anyone's phone number. The Q&A only gets a small "incident facts" object.
-
Claims and unlocks are parsed by rules, not the model.
UNLOCK <code>is a regex plus an unlock guard: enrolled number, one-time code, 3 tries, and a 10 s signed command. - Nothing said on a phone call can unlock anything. The voice agent will tell you to text the code.
In a healthcare setting, holding the AI back like this was what made us comfortable demoing it at all.
What went wrong (my favorite part)
1. A shared line can't text first. Reading Photon's beta docs before writing any code, I found that a shared iMessage line can't start a conversation. That's awkward for a fall alert. The workaround:
- each caregiver texts the line once before the demo
- the gateway remembers that conversation
- if an alert is blocked anyway, it's held and sent on that caregiver's first message
2. Real replies silently disappeared. My gateway originally only accepted replies sent to the agent number in our config. On the morning of the demo, tracing showed every real reply landing on a different number. Photon's shared plan assigns each caregiver their own line. The fix was small: accept replies on any of our project's lines.
// Before: only replies to PHOTON_AGENT_HANDLE counted,
// so every real reply was dropped.
// After: any agent line in our project is ours.
if (msg.receivedBy.kind !== "agent") return false;
The lesson: a stub mode is great for building, but you only learn how a platform really behaves by running it live.
3. Testing a voice check-in means being the person on the floor. Much of my night was spent on a teammate's laptop playing the person who fell: answering the check-in with every kind of reply I could think of. Clear answers, hesitant ones, "I'm fine, but my hip hurts", and silence. That's how you feel why the rules matter: "I'm fine, but…" must never cancel an incident, and silence must always escalate.
4. A bug no test caught. All our tests passed, but I suspected the AI calls were competing for one thread pool. I wrote a 15-line reproduction with fake latencies, and it confirmed two problems:
- A burst of answers starved the caregiver-reply rewrite, so the message was read aloud unrewritten.
- The last triage call, the only one whose result would be used, timed out waiting in the queue.
We split the pools and limited triage to one call at a time per incident.
5. A real UNLOCK was correctly refused. In a live test, a real UNLOCK text came through Photon after the 600 s unlock window had closed, and the system refused it. Annoying in the moment, and exactly what it should do.
Hackathon Experience
This was my first hackathon with real hardware. Overnight, the whole system ran on my phone's hotspot, because the venue Wi-Fi wouldn't let our laptop and the Pi see each other.
What I'll remember:
- Reading sponsor docs before writing code saved us hours. The shared-line limit would have broken the demo if we'd found it the morning of judging.
- The sponsor tables: Photon's booth confirmed that phone calls run on an SMS number, which became our escalation call.
- Pitching is its own build. I worked on our pitch, the demo run-through and the video. Turning a system with a dozen moving parts into two minutes a judge can follow took as much iteration as the code.
- Being honest with judges paid off. We said plainly that the latch is simulated, that detection was tuned on bottle drops rather than real falls, and that it's a prototype, not a medical device. The questions we got were about design decisions, not gotchas.
Huge thanks to Anmol, who got a QNX Pi talking to an ESP32 over raw USB overnight and got our phone line talking to a voice agent, and to Sakshi, who made all of it make sense to judges. And thanks to StepFun and Espressif for the 2nd-place recognition.
FirstHour is a hackathon prototype, not a medical device. "Margaret" and all phone numbers are fictional.
