Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side

One RTX 5090, 128 GB of RAM, and four open models from 125B to 748B parameters. Over the past weeks I got each of them running on the same workstation, on three different runtimes. I measured all four in one session, one after the other, with the same benchmarks. This article puts the numbers side by side, so you can see what each setup is good for.

Update, October 22, 2026: I measured everything again after updating Project Maya from 1.1.17 to 1.1.21 and retuning Ornith. The first version of this article, measured on October 21, had Ornith at 33.1 tokens per second. It drafted with its built-in MTP head then, not without a draft model as I wrote. Without drafting and with more experts in VRAM, it now does 46.4. Maya 1.1.21 made DeepSeek read long prompts about 3.7 times faster. The tables below replace the old ones.

Every number below was measured on our own machine on the night of October 21 to 22, 2026, unless I say otherwise.

The four setups

Parameter counts come from the model cards and the runtimes' READMEs. All four are mixture-of-experts models: only part of the model works on each token, which is what makes them possible on one GPU. The runtimes keep the most-used experts in VRAM and the rest in RAM; Maya with DeepSeek also reads part of the model from the SSD while it generates.

How to set up three of them, step by step:

• Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant
• GLM-5.3-Flash (321B) with Project Maya and EXL3
• DeepSeek-V4.1-Flash (748B) with Project Maya

For a machine with less RAM, the oldest article in the series covers Qwen3.8-Flash-Next with EXL3 and TabbyAPI on 40+ GB of RAM.

The machine

• GPU: RTX 5090, 32 GB of VRAM, also driving the desktop. Every runtime is set to leave 3 GiB free for that.
• CPU: Ryzen 9 9950X3D (16 cores, 32 threads).
• RAM: 128 GB DDR5 as 4x32 GB at 5,600 MT/s.
• Disk: one NVMe drive.
• OS: Arch Linux (Omarchy), NVIDIA driver 615.78, CUDA 13.4.
• Versions: Strata 296b59d, Project Maya 1.1.21 (451f9cb), TabbyAPI d6e6227 with ExLlamaV3 1.6.2.

How we measured

One model at a time, started with its own script, with no other model runtime or GPU miner running. Each runtime started in its own systemd slice without a memory limit, so a shell's limits or systemd-oomd could not get in the way (the setup articles explain why).

• Decode speed: a client-side streaming bench against the OpenAI endpoint. Eight runs, alternating a ~400-word Dutch prose prompt and a coding prompt (binary search with tests), greedy decoding, 512 output tokens, median reported. Time to first token is the median over the same runs.
• Prefill and long context: a needle-in-a-haystack test with prompts of about 31K and about 125K tokens. The needle is a code hidden in Dutch filler text, with a different random seed per run so the prefix cache cannot help. The exact token count depends on each model's tokenizer: the long prompt came to 124.6K tokens for Qwen, 125.6K for DeepSeek, 127.9K for GLM and 131.5K for Ornith. Prefill speed is prompt tokens divided by the time of the whole request, which includes a short answer.
• Drafting: Strata drafts with Qwen's MTP head and Maya with GLM's MTP block. Maya measures DeepSeek's DSpark drafter at the first start and keeps the faster option; on 1.1.21 it measured 23.9 tokens per second without and 23.8 with it, so it kept no drafting. Ornith on TabbyAPI runs without drafting, with 180 of its 256 experts per layer in RAM. In an A/B at 256K context the evening before, its MTP head gave 37.8 tokens per second, the DFlash2 drafter 40.6, and no drafting 45.2. Most of Ornith's experts run on the CPU, so every drafted token the model has to check costs extra reads from RAM.
• Quality on our own work: nine tasks from our day job, each with an automatic checker: two Astro helpers in JavaScript, two WordPress PHP snippets run in a PHP sandbox, SEO meta tags as JSON, LocalBusiness JSON-LD, a Search Console tool call, a Search Console quick-win analysis and a Dutch blog intro. 54 points per round, two rounds, reasoning effort medium, each server's default sampling. The second round ran the models in reverse order. During this run, Node could not start inside the sandbox (a version-manager shim on a read-only file system), so both JavaScript tasks failed for every model. I fixed the sandbox and scored the saved answers again with the same checkers; that changed only those two tasks.

Speed

All four found the needle at both lengths. For a sense of what the prefill means: the long prompt took 29 seconds on Strata, 49 on Maya with DeepSeek, 52 on Maya with GLM and 87 on TabbyAPI with Ornith. On October 21, with Maya 1.1.17, DeepSeek needed 182 seconds for the same length.

Qwen on Strata is in a class of its own: 1.7 times the decode speed of the next one, and the fastest prefill by far. One likely reason is the number of active parameters: about 6B per token for Qwen against about 18B for GLM, according to their model cards, so far fewer weights move through the CPU and over PCIe for every token. Strata's runtime, built for this one model family, plays a part too; we did not measure how much each contributes.

The bench counts tokens from the usage numbers each API returns, not from stream chunks. That mattered for Ornith in the first version: with MTP drafting, TabbyAPI streamed it in chunks of about two tokens.

Quality on our own tasks

On October 21 the totals were 108, 108, 105 and 89, in the same order.

What went wrong:

• GLM wrote a WordPress breadcrumb snippet with a PHP syntax error in the first round, and missed the Search Console quick-win analysis in the second (0 of 3 checks), the same task it missed once on October 21.
• Ornith missed the breadcrumb snippet (it used a PHP constant that does not exist), the JSON-LD task and the Search Console analysis in the first round, and a WordPress shortcode snippet with a PHP syntax error and one check of the Dutch blog intro in the second. Several misses came from stray tokens in the middle of otherwise correct output, such as a Chinese word or the word "Logical" inside a JSON object. That happened in one to four of the nine tasks per round, on both days, with and without MTP drafting.

The time per round says as much as the score. Qwen finishes the nine tasks in under two minutes, GLM in about two and a half. DeepSeek needs 11 minutes, because it writes about 15,900 tokens per round at about 24 tokens per second. Ornith writes even more, about 22,600 tokens, mostly reasoning, and takes about 8 minutes. GLM writes the least, about 3,900 tokens, which is why it is quick despite its slower decode.

These tasks reward correct, careful output, not deep reasoning. They do not tell you which model is smartest. They do tell you which ones we can hand our everyday work to without checking every answer.

Memory and VRAM

The VRAM peak is the highest value nvidia-smi reported for the whole card during the needle tests, so it includes the desktop and two browsers (about 2.2 GiB before any model started). Each runtime keeps its 3 GiB reserve on top of its own use; with the desktop counted, GLM and Qwen left a little less than that free at their peak. Strata and Maya with GLM pin almost all RAM for their expert tiers, so nothing else heavy fits next to them. Ornith's 65.5 GB quant leaves the most room; with more of its experts in VRAM, it now uses about 1 GiB more VRAM than in the first version of this article. DeepSeek's engine works differently: it reads its model files in place, so free counts most of what it uses as page cache and reports about 100 GiB as available. Do not read that as free memory. The engine depends on that cache, and anything you start next to it pushes experts out, which then have to come from the SSD again. In practice it uses the whole machine, like the other two. RAM and swap were read once, right after the needle tests. Ornith ran right after DeepSeek and had about 2.3 GiB in swap at that point; on the previous evening's tuning runs it had none, and its decode speed here (46.4) matched those runs with the same settings (46.4 to 47.2).

Which one we use when

• Qwen3.8-Flash-Next on Strata is our default for coding and agent work. Fastest at everything, full marks in both rounds of our tasks, 512K context.
• GLM-5.3-Flash on Maya is the second opinion: a much bigger model with 512K context, at about 40% of the speed, and the one that writes the least.
• DeepSeek-V4.1-Flash on Maya is the biggest model here, and the slowest to write, with a 3-second wait before the first token. Since Maya 1.1.21 it reads long prompts about as fast as GLM. Good for hard problems where we can wait, not for interactive work.
• Ornith-1.5-122B on TabbyAPI is now the second fastest at writing, with the quickest first token, but the slowest at reading long prompts. It leaves the most RAM free. On our tasks it scored the lowest, partly because of stray tokens in its output, so we read what it writes before we run it.

Running all of this on one desktop is possible because the runtimes move experts between VRAM, RAM and the SSD while you chat. The cost is that only one model fits at a time, and switching takes a stop and a start script.

Caveats

• One machine, one session. Other hardware, other drivers or another day can give other numbers, and all four runtimes move fast: Maya 1.1.17 made DeepSeek's long-prompt reading about eight times faster than 1.1.16, and 1.1.21 made it another 3.7 times faster.
• Two rounds of nine tasks are not many. A single miss can be noise.
• GLM scored 98 here and 105 on October 21, Ornith 83 and 89. With two rounds of nine tasks, that difference is within what a few misses can do.
• Each model ran with its runtime's default sampling and reasoning effort medium. Tuning per model could change the quality ranking.

If you are thinking about running models like these for your business, Voltti has a page about local AI for businesses: what stays on your own hardware, and when a hybrid setup makes more sense.