A 32 GB GPU Still Needs Host RAM Headroom

My local model was installed. Ollama answered its model-list request. The RTX 5090 reported 32,607 MiB of VRAM. I still did not start the test.

The short answer: VRAM fit does not prove that the whole host is ready for a local LLM load. On 2026-08-29, my preflight found only 7.02 GiB of free host RAM against my 16.0 GiB cold-load floor. It stopped before inference and saved a clear reason.

Canonical URL: https://bmdpat.com/blog/rtx-5090-host-ram-headroom-local-llm-2026

Why did the run stop when the GPU was ready?

I ran config/ollama/health.py --json before a small Ollama test. The check reached the local API and found all eight installed model names. It also found gemma4:26b, the model required by my local writing path.

The nvidia-smi probe reported an NVIDIA GeForce RTX 5090, driver 610.88, and 32,607 MiB of total VRAM. Those checks passed.

The host-memory check did not. Windows had 7.02 GiB free. My cold-load policy needs 16.0 GiB free before the system can ask Ollama to generate. The health command returned ok: false and named the failed guard. It made no generation request.

That last detail matters. This was not an Ollama error. It was not a model error. It was a local admission rule that stopped a risky load. The generate HTTP call was never sent, so this run has no token rate, eval count, or duration to publish.

A later runtime budget such as AgentGuard can cap spend after a request starts. This floor is earlier. It refuses the load while host RAM is 7.02 GiB free against a 16.0 GiB policy.

What is the difference between VRAM and host RAM?

VRAM holds GPU work such as model weights, runtime state, and the active context. Host RAM supports the operating system, the inference process, file reads, model setup, and every other process on the machine.

A model can fit in VRAM while the host has little working room. The GPU total does not show how much memory Windows, Ollama, browsers, build tools, and other agents need at the same time.

This is why I keep host memory separate from the VRAM budget for shared local AI workloads. The two checks protect different boundaries.

The 16.0 GiB floor is my policy for this machine. It is not an Ollama rule. Another host can use a different floor after it measures its own cold loads and normal background work.

What should a local LLM preflight check?

I use four gates.

First, query the model catalog. A missing model is a setup result, not a speed result.

Second, read free host RAM before model load. Compare it with a fixed floor for that host and workload. If the check fails, stop before the request.

Third, load one model with a small request. Save the model name, runtime version, host, and the pass or fail result.

Fourth, run the measured workload. Capture wall time, prompt tokens, output tokens, output rate, VRAM, host RAM, and stop reason.

I keep these gates separate from the broader local AI runtime preflight. Build tools and runtime support come first. Memory admission comes just before model load.

Why not let Windows page memory to disk?

Paging can keep a process alive, but that is not the same as keeping an agent host healthy. A large memory load can slow unrelated work and make wall-time results hard to compare.

My 5090 machine also runs the vault, browser tasks, and scheduled agents. A local model test must share the host. It does not own the machine.

A fixed free-memory floor makes that choice explicit. The run starts only when the machine has enough working room. If the floor is too high, measured skips will show that. If it is too low, load-time and system-health receipts will show the cost.

I do not lower the floor to make a daily test turn green. I free memory, choose a smaller model, or record the skipped run.

What should the failure receipt contain?

Save the time, host, model, runtime version, GPU, driver, total VRAM, free host RAM, required floor, and the exact failed gate. Also save whether the generation request started.

The distinction between blocked and failed is important. On 2026-08-29 my health command returned generate_ok: false and named the host RAM guard as the reason. The request never started, so there is no failed generation to explain.

That receipt prevents a later report from showing a blank token rate with no reason. It also prevents someone from blaming the model for a request that never reached it.

A useful preflight names the failed gate and whether generate started. Mine named the host RAM guard and recorded generate_ok: false, so the table does not show a blank token rate.

Accompanying prompt

What the prompt does: It turns a local LLM load plan into a host-memory gate with a clear pass rule and failure receipt.

Copy/paste this prompt:

Role:
You are reviewing memory admission for a local LLM host.

Context:
Paste the host name, model, runtime, total VRAM, free host RAM, normal background work, and planned workload.

Task:
1. Separate the VRAM check from the host RAM check.
2. Set a host-specific free-memory floor from measured cold loads and background use.
3. Define the small request that proves model load before the measured workload.
4. Define the receipt for a blocked, failed, or completed run.

Output:
- A gate table with catalog, host RAM, model load, and workload rows.
- A pass or fail rule for each row.
- The fields to save when a gate stops the run.

Constraints:
- Do not treat total VRAM as free host RAM.
- Do not lower the memory floor only to make a run pass.
- Do not invent missing measurements.
- Mark every request that never started as not attempted.

Copy the block above.

Get the next measured local AI build note from The 5090 Reports.

Get the local AI lab notes (benchmark rows, VRAM fit, quant choices, what runs on consumer GPUs), M-F only when there is something worth sending: https://bmdpat.com/newsletter?utm_source=blog_md&utm_medium=aeo&utm_campaign=rtx-5090-host-ram-headroom-local-llm-2026

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.