Someone asks for the same thing every few months. A remote expert should be able to see what the technician sees and point at the actual bolt, instead of saying "the third one along, no, from the other side" until the wrong component comes off.
The pointing part is a week of work. An anchored marker in a live camera feed is a solved problem, and if you have ever shipped anything with world tracking in it you already know how to do it. The project is everything that is not the arrow, and almost none of it is AR.
So here are the parts that actually decide whether one of these ships.
Three products share one name
Vendors mean one of three things by remote assistance, and the cost gap between them is large enough that naming which one you are building is the cheapest decision on the project.
The annotated video call. The technician holds up a phone, the expert draws on the live feed, and the drawing stays stuck to the object when the camera moves instead of sliding around the screen. That last clause is the entire AR contribution. Everything else is a video call with a whiteboard on top.
The hands-free session. Same idea on a head-worn device, so the technician keeps both hands on the work.
The persistent layer. Content stays anchored to the equipment after the call ends, so the next person at that valve sees the note the last expert left. This one is a different product. You are no longer running calls, you are maintaining spatial content against physical assets that get replaced, repainted and moved.
Most buyers ask for the second, need the first, and work out a year in that they wanted the third.
The anchored arrow is the only AR in the AR call
Worth being precise about what "stays stuck to the object" costs, because it is the one genuinely AR requirement and it is where the demo and the deployment diverge.
A screen-space drawing is trivial. An anchored one needs a spatial reference that survives the technician turning their head away and coming back, in a room that is often the worst case for tracking: repeating metal surfaces, low light, reflective panels, and nothing with texture on it. Re-localization is the interesting failure. The marker does not vanish with an error, it quietly ends up on the wrong fitting, which is exactly the outcome the feature existed to prevent.
Holding several people inside one consistent spatial view is solved and we have built it. MR Camera is a multiplayer mixed-reality environment where several users place and interact with the same 3D models in shared space. That part is not the risk. The risk is that your anchor quality is decided by the site, not by your code, so it needs to be tested on the site.
The test that decides whether you build this at all
A senior engineer on a plain video call already fixes a great deal. Before anyone spends on AR, check whether at least one of these three is true of the actual work.
1. Pointing. The technician is looking at twenty identical fittings and the expert has to indicate one. When language cannot reliably identify the object in the frame, an anchored marker is the whole fix.
2. Hands. The technician physically cannot hold a phone. They are on a ladder, in gloves, inside a panel, or already holding something that must not move.
3. Artifact. The same fault gets diagnosed again next month by somebody else. A call ends and takes its contents with it. A session that leaves an annotated frame, a transcript and a resolution becomes something the next technician can use.
One of the three is usually enough to justify a modest build. If none is true, a scheduled call and a better camera get you most of the value for none of the budget, and anyone worth hiring will tell you that before writing a proposal.
Design the ladder down, not the happy path
Remote assistance is needed exactly where the signal is worst. Plant rooms, basements, rooftops, ship holds, rural substations. The pilot runs in a room with five bars and the work happens somewhere that barely holds a phone call, and live video is the most bandwidth-hungry thing you can ask of a weak link.
So the question is not whether the thing supports poor connectivity. It is what it does at a fraction of the bandwidth your demo assumed. Decide the rungs up front and treat each one as a supported mode with its own interface, not as an error state:
- Full video with live annotation.
- Reduced frame rate and resolution, annotation still live.
- Still captures with annotations, live audio only.
- Audio only, photos queued and sent when a bar of signal appears.
Two things follow from writing that list down. The first is that annotation has to work against a still, not only against a moving feed, which is a real difference in how you store and replay a marker. The second is that the technician needs to know which rung they are on without reading a diagnostic panel, because "the expert has gone quiet" and "the expert is on rung three" feel identical when you are up a ladder.
Anything that only works at full video is unusable exactly where a remote expert is worth the most.
Two constraints that are not software
The device has to fit the job. Head-worn hardware has to coexist with whatever the work already requires: hard hats, safety glasses, respirators, hearing protection, sterile fields. A headset that cannot be worn with mandatory PPE will not be worn, and no amount of software quality recovers from that. Then the ordinary questions. Does the battery last a shift or a demo, can it be cleaned the way the site requires, does it survive concrete.
Somebody has to answer. The most common failure is organizational. A platform with nobody on the other end is an expensive way to miss a call. Decide before launch who answers, inside what window, and what happens when they do not: night shifts, holidays, leave, and the fallback when nobody picks up while a technician stands next to stopped equipment.
The call is the demo. The session record is the product.
A call that ends and leaves nothing behind has saved one trip. A session that writes an annotated frame, a transcript, the parts used and the resolution onto the asset record has done something durable. After a year you hold a library of real faults on your real equipment, described by the people who fixed them.
It is also the integration surface, which makes it most of the build cost. Sessions have to land in the ticketing system, asset register or CRM the team already uses, and the write-back has to survive a technician who closes the app the moment the call ends, on the same bad connection that forced you down the ladder in the first place. Queue it locally, upload with an idempotency key, and never let the upload be a side effect of a screen staying open.
One more thing belongs in the scoping document rather than in a later argument: you are recording video of a workplace, staff, and sometimes a customer's premises. Retention, access and consent are a decision, and not deciding is also a decision.
What I would settle before anyone quotes
- Which of the three tests is true for this work: pointing, hands, or artifact? If the answer is none, say so out loud.
- What does the session do at the worst bandwidth on the worst site, and when do we test it there rather than here?
- Can the device be worn with the PPE these people are required to wear?
- Who answers, within what window, and what is the fallback when nobody does?
- Where does the session record land, and is it attached to the asset or the ticket?
- Can a contractor join without installing anything?
And the cheapest experiment of all: run it on phones against real faults for a few weeks before buying any headsets. You will find out which of the three tests is actually true for your work, and that answer changes the build more than any hardware decision will.
The headset is the least interesting part. What decides the outcome is whether the work has a real pointing or hands problem, whether the software still functions on the bad connection at the far site, whether a human reliably answers, and whether each session leaves something behind that makes the next one shorter.
I wrote the longer buyer-side version of this, including the cost drivers and where remote assistance does not earn its keep, over on the Null Studio blog.
