A Wider Computer, Not a Bigger One: Modeling AI Inference Across Millions of Homes

I modeled an AI inference fleet distributed across ordinary homes. What survived was narrower—and more plausible—than what I started with.

Picture a detached house on a cold evening. On one garage wall, an operator-owned compute appliance draws about five kilowatts—electrically comparable to an EV charger, though it runs much longer. It serves small AI models. In winter its waste heat can warm the house; in summer that heat must be carried away. The family bought nothing. They are paid to host it.

Now repeat that arrangement across a neighborhood, then a state, then millions of homes—not to assemble one enormous computer, but to create millions of independent inference workers. Each keeps a small model resident in GPU memory. New requests route around homes that are offline. The network grows by adding locations that were already built and connected to the grid.

That network does not exist. I call the idea HEARTH. Almost none of its physical pieces are exotic; the experiment is whether they can be arranged and scheduled as one system. Not a bigger computer. A wider one.

To make the appliance less abstract, I developed three concept form factors: a wall unit, a floor-standing thermal tower, and a duct-integrated mechanical-room unit. They are appearance and installation studies, not engineered products; their job is to expose the questions that a real prototype must answer.

The model kept changing its answer

Then I priced the accelerators, and the idea stopped working.

That was the first useful result. My original comparison had focused on the dramatic expense of constructing a data center while treating the compute hardware as identical on both sides. But identical hardware does not disappear from the economics. If one location keeps an expensive GPU busier than another, utilization can overwhelm everything the cheaper building saves.

I rebuilt the model five times. Each pass introduced a constraint the previous one had missed:

What survived: small, independent inference

The fifth pass did not rescue my original proposal. It replaced it. A residential fleet is not a cheaper place to run every AI workload. It cannot pool memory across the internet, train a frontier model, or make residential latency disappear. What survived was narrower: small-model inference in which one node completes one request and no durable state is tied to a particular house.

Model size changes the economics because inference is not just a race between GPUs. During generation, a server repeatedly reads the model's weights while advancing many requests together. That is batching. A larger batch spreads each weight read across more paying tokens—but every active request also consumes working memory, usually called the KV cache.

An 8B model—roughly eight billion parameters—leaves enough memory on a 32 GB consumer GPU for a useful production batch. In my model, both the home GPU and the data-center hardware then become limited mainly by compute, allowing the consumer part's much lower purchase price to matter. At 32B, the consumer card runs short of memory first. Its batch stops growing while high-memory data-center hardware keeps filling, and the verdict reverses.

The current result is conditional. Before homeowner compensation and fleet-level operating costs, and under RTX 5090 MSRP, an 85% accelerator duty factor, identical serving-efficiency assumptions for both venues, and a quantized 8B workload, my roofline model estimates the residential hardware-and-energy stack at about 0.49 times the cost per token of the best data-center case in the sweep, based on GB200 NVL72 rack pricing. That is a reproducible scenario output, not observed production performance or a fully loaded business cost. An apples-to-apples benchmark—or the costs excluded here—could erase the advantage.

This is the distinction behind wide, not deep. HEARTH would not divide one giant model among houses or combine residential GPUs into a virtual supercomputer. Each node would answer complete, independent requests with a model it already holds locally. Adding homes increases the number and variety of requests the fleet can serve; it does not make any one node larger.

The asset is the grid endpoint

Why put that workload in homes at all? Not because residential electricity is cheap—it usually is not—or because waste heat makes energy free. The asset is the connection behind the meter.

Berkeley Lab estimated that US data centers used about 176 TWh of electricity in 2023. Its June 2026 update projects 521–843 TWh in 2030, with a 649 TWh reference case. Serving that growth is not simply a question of buying more GPUs. A large new campus may need substations, transformers, transmission upgrades, and a negotiated path to connect a load the local grid was never designed to carry. The 410 GW of prospective large loads reported in ERCOT—about 87% associated with data centers—is not a forecast of what will be built. It is evidence of how much demand is converging on the same constrained process.

The United States has about 82.5 million one-unit detached housing units. That is physical stock, not an estimate of eligible hosts: occupancy, broadband, electrical capacity, utility approval, and household consent would reduce it substantially. Each candidate already has a meter, an electrical service, and a physical building around it. A compute node installed behind that meter may avoid the transmission-scale interconnection required by a new campus. That is the inversion: instead of bringing an enormous new grid connection to the compute, bring a modest amount of compute to many connections that already exist.

Existing does not mean unused. A continuous 5 kW load is much less forgiving than an EV that charges for a few hours. Some homes would need panel or service upgrades; utilities may require review; and the local distribution transformer remains a hard physical constraint. My one-node-per-transformer rule is therefore a screening hypothesis, not a validated safety rule or national capacity estimate. Transformer loading, telemetry, and thermal behavior belong in the first utility-supervised field test.

To the household, this is a hosting contract, not an investment. The operator would own and maintain the appliance, meter and reimburse its electricity, and pay the family a share of revenue. None of that has been field-tested; fire and insurance rules, noise, summer heat rejection, ISP terms, maintenance, and upgrades remain field questions.

One request goes to one home

At the software layer, the system is deliberately less exotic. A control plane would assign replicas of small models to qualifying nodes and distribute checkpoints outside the request path. Each node would store its assigned checkpoints locally and keep its serving model resident in GPU memory. The request router would know which homes have the requested model resident, which are healthy, and which have batch capacity available.

registry ── sync ──► home node

client ──► router ──► available home ──► response
               ▲              │
               └── telemetry ─┘

During generation, one home holds the request's KV cache and returns the tokens; no activations or model layers cross the residential network. If the node disappears, the in-flight generation is lost. New work can route elsewhere, and a retryable request can start again against another replica, although it may not produce identical tokens. The system does not inherit data-center reliability merely because it has many machines.

At the network level, remote attestation, model security, and prompt and output confidentiality on hardware outside an operator-controlled facility remain unresolved. Distribution makes failures routable; it does not make operations disappear.

Five conditions decide whether this is real

1. The hardware must be legally and commercially usable. NVIDIA's current driver agreement says the software may not be used to provide commercial hosting services and that GeForce and Titan software is not licensed for data-center deployment. Whether an operator could obtain written authorization or a different license for this residential topology must be resolved with NVIDIA and qualified counsel before a hardware pilot. Warranty, support, and a credible procurement channel must also exist.
2. Edge serving must sustain the modeled batches. The same checkpoint, quantization, context length, serving stack, and request mix must be benchmarked on the candidate consumer GPU and data-center hardware. A spreadsheet cannot substitute for that result.
3. Demand must keep the silicon busy. Cheap idle GPUs are still expensive. Customers must commit enough suitable 8B inference work—classification, extraction, routing, drafting, or other bounded tasks—to maintain high utilization at a viable token price.
4. The location must pass the energy screen. Residential rates vary widely, and heat reuse is a seasonal credit rather than the thesis. Every deployment needs to work after metered electricity, host payment, and summer heat rejection.
5. The distribution grid must approve the load. A real utility must confirm that a real home, service, and transformer can carry it continuously. National averages are not permission to connect.

Those are not caveats around the result. They are the result. HEARTH exists only at their intersection.

The first pilot should contain no houses

That changes the order of the pilot. My original plan began with 250 homes and grew through increasingly expensive gates. It tested installation first and market demand later. The model eventually exposed the flaw: there is no reason to put hardware in a house before proving that anyone will buy this particular kind of inference.

The first HEARTH pilot should therefore contain no houses. Rent the candidate consumer and data-center hardware. Resolve the license question. Run the same models through the same serving stack, measure delivered tokens and power at production batch, and ask customers to pay for the workloads the residential fleet is supposed to serve. If the measured economics or demand fail, stop before an electrician is dispatched.

Only then move to 250 homes. That field trial buys answers the lab cannot: Can crews install safely and consistently? What do transformers do under continuous load? How loud and hot are the nodes in July? Can the operator keep them online without drowning in truck rolls—and will families keep hosting them?

Four million homes is not the next milestone

Four million homes is an image of the possible system, not an adoption forecast. The serious next milestone is smaller: one workload that benchmarks, one customer willing to buy it, one utility willing to supervise it, and then the first 250 homes.

I published the full feasibility study, the source model and inputs, and every assumption I used. If the idea is wrong, I want the failure to be reproducible too.

HEARTH is not a forecast. It is a claim specific enough to test—and to kill if it fails. The buildings, meters, and many of the grid and network endpoints already surround us. What is missing is the appliance, the dedicated installation, the operating system around it, and an agreement worth signing.

If those pieces work, the neighborhood still looks like a neighborhood on a cold evening. Behind some garage walls, independent machines serve requests from a network spanning the country—each complete on its own, all scheduled together.

No monument to compute. A million warm windows.