Status: building now. Started 21 Oct 2026 for the Nebius x NVIDIA Global AI Hackathon. Submission due 30 Oct 2026. Live build: atlas-live-xi.vercel.app
A few weeks ago I built a gesture interface for maintenance technicians, JAMES. One part of it, called ATLAS, drew the targeted machine as an exploded schematic you could pull apart with an open palm. For this hackathon I am rebuilding that part on its own, in the browser, and taking it much further.
ATLAS Live turns real machines, real objects and real places into holograms you can pull apart with your hands, pick with a pinch and question out loud. Every fact it speaks comes from a web search it cites.
This post is written mid-build, so it is a progress report, not a victory lap.
What it does today
| Mode | You do | It does |
|---|---|---|
| Library | Pick a machine: Perseverance, Ingenuity, the James Webb telescope, a chronograph, a concept car | Loads the real model. Open palm explodes it, pinch picks a part, ask anything |
| Scan | Hold an object up to the webcam, or drop in a photo | Nemotron vision names it and boxes its parts. Depth Anything V2 lifts the photo into a 3D point cloud, split into those parts |
| Places | Name a landmark | Google Photorealistic 3D Tiles fly you in. Cited captions for what you are looking at |
The models come from NASA (public domain) and the Khronos sample set (CC BY 4.0), with textures and logos removed.
The stack, as an exploded view
The cover of this post is the app taken apart, layer by layer, because that is honestly how I think about it:
- Hands: MediaPipe Hand Landmarker, running locally. Camera frames never leave the browser except the single frame you choose to scan.
-
Eyes: NVIDIA Nemotron 3.5 Super VL identifies a scanned object and locates its parts. A Nemotron agent answers questions and drives the hologram with tool calls (
web_search,focus_part,set_explode). - Depth: Depth Anything V2 Small via transformers.js, also in the browser.
- Facts: Tavily search. Captions and answers carry their sources.
- Ground: Google Photorealistic 3D Tiles through Cesium, on three.js.
All model calls run server-side in Vercel functions. Keys never reach the browser.
Three things that bit me in the first two days
1. Model ids die
The provider layer is any OpenAI-compatible endpoint, first key wins: Nebius Token Factory, then the NVIDIA API catalog, then fallbacks. Right now the live site runs Nemotron through NVIDIA's API catalog, and it will switch to Token Factory the moment that key is set, without a code change.
What I didn't expect: model ids get retired under you. Gemini 2.5 shut down on 16 Oct 2026, days before I started. So the app no longer trusts a hard-coded id. On start it reads the provider's live /models list, and if the configured id is gone it takes the first live id that matches a preference list:
const NEMOTRON_PREFER = {
vision: [/nemotron.*vl/i, /vl/i],
agent: [/nemotron.*super/i, /nemotron.*ultra/i, /nemotron/i],
fast: [/nemotron.*lightning/i, /nemotron.*nano/i, /nemotron/i],
};
A model that returns 429 is skipped for 15 minutes and the next one takes over.
2. One image per request
Build mode researches a landmark, plans it, has small builder agents make the parts, and then a critic compares the result with a reference photo. The natural request sends both images. NVIDIA's Nemotron VL endpoint rejects a second image in one request.
The fix was boring and it works: stitch the render and the reference into one side-by-side image and send that.
3. Some famous places look bad in 3D Tiles
Google's 3D coverage is excellent for the Colosseum and Machu Picchu and poor for others. The Taj Mahal looks melted. So I am adding photo reconstructions: a Colab notebook pulls openly licensed photos of the landmark from Wikimedia Commons, matches them (SuperPoint and LightGlue through hloc, then COLMAP), and trains a Gaussian splat with gsplat. The app shows it with the Spark viewer.
One bug worth sharing from that work. In the gsplat version I am using, the opacity reset never fires:
if step % reset_every == 0 & step > 0:
In Python & binds tighter than ==, so this reads as step % reset_every == (0 & step) > 0, a chained comparison that ends in 0 > 0. It is always false. My training loop does the check itself.
Still to do before 30 Oct
- Run the reconstruction notebook for the Taj Mahal and Charminar and wire the results in
- Test hand control on a real webcam, in poor light
- Re-test Build mode on the Eiffel Tower and the Taj Mahal
- A 3-minute video and the write-up of where Nemotron and Token Factory are used
If you have tried hand tracking in the browser and found a pinch detector that survives bad lighting, I would love to hear how you did it.
Repo (MIT): github.com/victorisaacchinta-eng/atlas-live
I wrote this with AI assistance, working from the repo and my build notes. It describes the build as of 22 Oct 2026.
