Run DeepSeek-V4.1-Flash (748B) locally with Project Maya on one RTX 5090

DeepSeek-V4.1-Flash-1017 has 748B parameters. Even Project Maya's own 2-3 bit quant is 340 GB with its Engram tables, and my workstation has 128 GB of RAM. It still runs here: on one RTX 5090, with a 262K context, at about 21 tokens per second on Maya 1.1.17, and about 25 since 1.1.21. This guide shows how, with Project Maya, and what we measured, including where it is slow.

Every number below was measured on our own machine between October 19 and 22, 2026. Where a number comes from someone else, I name the source.

This article belongs to a series about large models on one consumer machine. A side-by-side benchmark of all four models compares them; the setup guides cover GLM-5.3-Flash with Project Maya, Qwen3.8-Flash-Next with Strata at 79 tokens per second and Qwen3.8-Flash-Next with EXL3 and TabbyAPI.

The model

DeepSeek-V4.1-Flash-1017 is an open-weight model from DeepSeek under the MIT license. Maya's README lists it at 748B parameters. The model card lists a context window of up to 1M tokens and recommends temperature 1.0 and top_p 1.0. Next to its experts it has two Engram tables, which Maya's setup describes as an n-gram memory that the engine reads a few rows from per token.

The model card does not ship a Jinja chat template, only a Python reference encoder. Maya's engine brings its own prompt handling, so you do not have to deal with that yourself.

Why Project Maya

Maya started as a runtime for GLM-5.3-Flash, and recent versions add a second engine, maya-ds4, for this model. The idea is the same: the most-used experts live on the GPU, the next ones in RAM, and the rest are read in place from the model files on the NVMe drive. The full model never has to fit in memory.

For this model Maya offers several file sets. From its README:

Maya-S is the smallest by far, so on 128 GB of RAM that is the one to take. We installed it with Maya 1.1.16, which downloaded 178.2 GB of model files, 203 GB of Engram tables from unsloth's GGUF and 8 GB for the DSpark drafter. Since 1.1.17, a fresh setup downloads Maya-S as one set of six files from Maya's own repository, 339.5 GB including the Engram tables, according to the changelog; a 1.1.16 install keeps working as it is. Plan for about 350-400 GB of free NVMe space.

The machine

• GPU: RTX 5090, 32 GB of VRAM, also driving the desktop.
• CPU: Ryzen 9 9950X3D (16 cores, 32 threads, AVX-512).
• RAM: 128 GB DDR5 as 4x32 GB at 5,600 MT/s.
• Disk: one NVMe drive.
• OS: Arch Linux (Omarchy), NVIDIA driver 615.78, CUDA 13.4.
• Maya 1.1.16 for the first measurements, 1.1.17 (commit 8d9a9b5) for the rest, and 1.1.21 (commit 451f9cb) for the October 22 update.

Install

If you already run Maya for GLM, the same checkout can hold both models. Each gets its own engine, config and port.

git clone https://github.com/mw00/project-maya.git
cd project-maya
./maya.sh --check

Then the setup. It downloads 340 GB or more, so run it in the background and outside any memory-limited scope (see the next section):

./maya.sh --setup --plain --yes --gpu 0 \
  --model DeepSeek-V4.1-Flash-1017 --ds4-quant maya-s \
  --context 262144 --models-dir ~/Models/maya --download-model \
  --no-vision --vram-reserve-mib 3072 --port 8082 --no-start

What it did on our machine (Maya 1.1.16):

1. Checked the PC, installed its Python packages and fetched llama.cpp's source at a pinned commit.
2. Compiled the maya-ds4 engine for the RTX 5090 (sm_120).
3. Downloaded the four Maya-S files (178.2 GB), the Engram file (203 GB) and the DSpark drafter (8 GB) with resumable curl commands it prints first, and checked the files against their SHA-256. The download ran at roughly 40-80 MB/s here, about an hour and a half for everything.
4. Indexed every tensor into a pack next to the model files. The weights stay where they are.
5. Wrote maya-ds4.json and run-maya-ds4.sh.

The flags that matter:

• --model DeepSeek-V4.1-Flash-1017 --ds4-quant maya-s picks the model and the smallest quant.
• --port 8082 keeps it apart from a GLM install on 8080.
• --vram-reserve-mib 3072 leaves 3 GiB of VRAM free. Our desktop runs on the same card, and with full VRAM the screen flickered.
• --no-vision skips DeepSeek's vision tower (about 1 GB).

Start it outside your shell's memory limits

Unlike the GLM engine, maya-ds4 does not pin most of the RAM: it reads the model files in place, so the RAM it uses shows up mostly as page cache (after our benchmark free reported 20 GiB used and about 100 GiB available). Do not read the available number as free memory: the engine depends on that cache, and anything you start next to it pushes experts out, which then have to come from the SSD again. In practice it uses the whole machine. The load is heavy too. Two Linux defaults get in the way:

• A shell inside a systemd scope with a memory limit. Our agent's terminal scopes have memory.max of 4 GiB; a big process started from there ends up in swap.
• systemd-oomd, which Omarchy enables for the user's app.slice. It kills a cgroup after 20 seconds above 50% memory pressure.

So we start it in its own slice, without a memory limit:

systemd-run --user --scope --collect --slice=llm.slice \
  --unit="maya-ds4-$(date +%s)" -p MemoryMax=infinity \
  ./run-maya-ds4.sh --plain

The first start after setup also runs Maya's speed calibration (below) and took about four minutes here; a normal start took 20 seconds. Then:

curl -s http://127.0.0.1:8082/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4.1-flash", "messages": [{"role": "user", "content": "Say hello in five words."}],
       "max_tokens": 2000, "reasoning_effort": "medium"}'

The dashboard at http://127.0.0.1:8082/#monitor shows the speeds and where the experts live. Keep the server on 127.0.0.1 unless you add --api-key.

The drafter: measure it on every version

DeepSeek's model comes with a DSpark drafter that proposes a few tokens at a time for the big model to check. Whether that helps depends on your PC, so Maya measures it on the first start with the GPU to itself and keeps the faster option. Upstream, it does not run while more than 1 GiB of VRAM is in use. We raised that threshold locally to 3 GiB because the desktop runs on the same card (see the GLM article), and during setup 4.2 GiB was in use, so the calibration ran at the first start instead.

The result flipped between two versions:

Running --setup again after an update drops the stored result, so the next start calibrates again and takes a few minutes longer. Read the new choice from the log instead of assuming the old one.

Benchmarks

How we measured: a client-side streaming bench against the OpenAI endpoint. Eight runs per setting, alternating a ~400-word Dutch prose prompt and a coding prompt (binary search with tests), greedy decoding, 512 output tokens, median reported. Prefill comes from a needle-in-a-haystack test at the given prompt length, with a different random seed per run so the prefix cache cannot help. No other model runtime or GPU miner ran at the same time.

Update, October 22, 2026: Maya 1.1.21 made prompt reading about three to four times faster again: 126K tokens now take 49 seconds instead of 182. Its calibration found DSpark no faster than plain decoding on this PC (23.8 against 23.9 tok/s) and kept no drafting. Decode is back at about 25 tokens per second; time to first token stayed at about 3 seconds. The quality bench scored 54 of 54 in both rounds again, at 680 seconds and about 15,900 tokens per round; the VRAM peak was 28.6 GiB. The rest of this article describes 1.1.16 and 1.1.17.

Three things stand out:

• Version 1.1.17 made long prompts usable. Its release notes say it reads prompts in 8K chunks and reads each expert once per layer for long prompts. Here, reading 247K tokens went from 46 minutes to 6.
• Decoding got slower with 1.1.17. Our bench measured 24.0 tokens per second on 1.1.16 without drafting and 20.7-21.8 on 1.1.17 with DSpark. Maya's own calibration also measured less without drafting on 1.1.17 (18.2 against 23.0), so this looks like the version, not the drafter. It is not a clean A/B: the calibration writes three answers of 384 tokens on its own prompts, our bench eight of 512. If it matters for your PC, measure both with your own prompts.
• Time to first token went from 1.7 to about 3 seconds on 1.1.17 with DSpark, about four times that of the other models on this machine. For short chat turns you notice that.

The VRAM peak for the whole card, desktop included, was 28.7 GiB, which left 3.2 GiB free. Right after the needle tests the engine process had about 600 MB in swap; the decode numbers above were measured before that and looked normal.

Quality check

We ran our own nine-task bench twice: two Astro helpers in JavaScript, two WordPress PHP snippets run in a sandbox, SEO meta tags as JSON, LocalBusiness JSON-LD, a Search Console tool call, a Search Console quick-win analysis and a Dutch blog intro, each with an automatic checker. Reasoning effort medium, Maya's default sampling.

DeepSeek scored 54 out of 54 in both rounds, 108 out of 108 in total, like Qwen3.8-Flash-Next on Strata. It also took the longest: 784 seconds per round on average, against 103 seconds for Qwen on Strata, because it writes more (about 15,700 tokens per round, against 9,000) at about a quarter of the speed.

I measured the other three models on this machine on the same day with the same benchmarks: Four big local LLMs on one RTX 5090 and 128 GB of RAM, benchmarked side by side.

Pitfalls we ran into

Nothing else fits next to it. The engine holds about 27 GiB of VRAM (29 GiB for the whole card with our desktop), and everything else you run competes with it for the page cache that holds the experts. Do not run a second model, a game or a GPU miner next to it.

Disk space and SSD reads. About 350-400 GB on disk, and the part that does not fit in RAM is read from the SSD during generation. Use a fast NVMe drive, not a SATA SSD or a hard disk.

Calibrate with the GPU to itself. Maya skips the speed calibration when too much VRAM is in use (1 GiB upstream) and tries again at the next start. On a machine where the desktop uses the same card, close what you can, or it keeps being skipped.

Reasoning effort. The setup writes "reasoning_effort": "high" as the server default. At 21 to 25 tokens per second, long reasoning takes a while: send a lower reasoning_effort from your client for interactive work. Our quality bench used medium.

Which setup we use when

On this machine, DeepSeek-V4.1-Flash is the biggest model and the slowest. It scored full marks in both rounds of our bench, but at about a quarter of the speed of Qwen3.8-Flash-Next on Strata, with a 3-second wait before the first token. We use it for a second opinion on hard problems, not for interactive agent work.

If you are thinking about running models like this for your business, Voltti has a page about local AI for businesses: what stays on your own hardware, and when a hybrid setup makes more sense.