A Raspberry Pi 5 runs DNS for the network, plus dozens of other small services that don't need much CPU or memory but can't go down. A Jetson Orin Nano handles anything that needs a GPU, and its GPU and CPU share one memory pool, small enough to hold one loaded model or one active render job, never both.
One flat network
There's one LAN. Clients and services aren't split into VLANs or separated behind an internal firewall; the upstream router is the gateway, and everything else sits behind it as a peer. If one device gets compromised, there's nothing stopping it from reaching every other device on the LAN. I've thought about segmenting it and decided against it: at this size, a mesh of routing rules is more to maintain than the threat justifies. It stays flat.
The Pi carries DNS and everything that has to be up all the time. It's picked for uptime and low draw, not speed, and I don't let it run anything that could pin its CPU long enough to take DNS down with it.
One model at a time
Load two things onto the Jetson that it can't both hold, and it thrashes. So nothing talks to it directly: every job (a chat request, a background research task, a scheduled scrape) lands in a durable queue first, and a single gateway between that queue and the Jetson admits exactly one job at a time, by priority. That gateway is most of the engineering here. Skip it and two jobs landing at once both try to load into memory built for one, and now the box is fighting itself instead of doing either job.
Priority doesn't preempt. If a background scrape is already loaded and running when a chat request comes in, the chat request doesn't kill it, it queues behind it and jumps ahead of everything else waiting. The alternative, killing a running job because something more urgent showed up, throws away whatever work was already done and risks leaving the model half-loaded. Waiting for the current job to finish costs a few seconds at most; a job that has to restart from zero costs more than that most of the time.
</defs>
<g fill="none" stroke="currentColor" stroke-width="1.4">
<rect width="280" height="56" rx="2"></rect>
<rect width="280" height="76" rx="2"></rect>
<rect width="280" height="56" rx="2"></rect>
<rect width="280" height="60" rx="2"></rect>
<rect width="280" height="70" rx="2"></rect>
<rect width="280" height="76" rx="2"></rect>
<rect width="280" height="56" rx="2"></rect>
<line x1="180" y1="86" x2="180" y2="126"></line>
<line x1="180" y1="206" x2="180" y2="246"></line>
<line x1="180" y1="306" x2="180" y2="346"></line>
<line x1="180" y1="410" x2="180" y2="446"></line>
<line x1="540" y1="206" x2="540" y2="446"></line>
<path d="M 400,180 C 300,230 260,300 300,352"></path>
</g>
<g fill="currentColor">
</g>
A second machine, with an RTX 4070 Ti Super, does rendering, and it doesn't go through this gateway for that work. The gateway exists to arbitrate the Jetson's single model slot, and the RTX box was never competing for that slot to begin with, so routing it through anyway would just add latency it doesn't need. It does share one thing with the Jetson: a single LLM call both need. For that one call, the two machines are fighting over the same resource, so it queues like everything else.
The price of running it this way
A Pi 5, a Jetson Orin Nano, and a desktop GPU: together roughly the price of a decent laptop.
