How to Run Mistral Large 4 Locally: 52B Active, Still Needs ~400GB of RAM

Mistral Large 4 is a 1.05 trillion parameter MoE with 52B active, and Mistral calls it open-weight. As of today, October 22, you can't download a single byte of it. The Hugging Face repo is a countdown page.

So "can I run Mistral Large 4 locally?" has two answers. Today: no. On October 31, if the date holds: yes, if you have somewhere around 400GB of fast memory and some patience. Below I sort the confirmed facts from the guesses, run the hardware math, and give the llama.cpp offload command to use once GGUFs land. That should tell you whether to start clearing a shelf or just pay for tokens.

What Mistral actually shipped on October 6

The announcement describes an "open-weight hybrid instruct-and-reasoning MoE with multimodal input", trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs. The model docs page adds the details the blog post skips:

Spec	Value	Source
Total parameters	1.05T	Mistral docs
Active parameters	52B (49B excluding embedding/output layers)	Mistral docs, HF placeholder
Vision encoder	1.6B	Mistral docs
Context (API)	1M tokens	Mistral docs
Version	v26.10, Public Preview	Mistral docs
Price per 1M tokens	$1.36 input, $0.14 cached input, $4.18 output	Mistral docs
Weights	"by the end of the month", HF page says Oct 31	HF placeholder
License	not named	everywhere

That last row bothers me more than the missing weights. The docs list the license as "Open" with no name. On Mistral's models overview, Mistral Large 3 and Mistral Small 4 sit right next to it, both tagged Apache 2.0. If Large 4 were going to be Apache 2.0 too, the easy move was to just say so. Until the license file exists, "open-weight" is a promise, and you shouldn't plan a product around it.

Context is also murkier than the 1M figure suggests. That number is for Mistral's hosted API. The HF placeholder lists no context length at all, so what the downloadable checkpoint natively supports is unknown until the config.json lands.

Mistral Large 4 benchmarks, as picked by Mistral

All self-reported, all from the announcement:

DeepSWE v1.1: 61.7%
Terminal-Bench 4: 28.3%
SWE-Atlas-QnA: 59.4%
Cybench: 93%
Surge AI blind human coding eval: 3.74 out of 5, second of five. Claude Opus 5 got 4.22.

The cyber numbers are the interesting part. Mistral claims 82% on a reproduce-and-patch test where, per the announcement, Claude Opus 5.5 and GPT-6 Astra score near zero because they refuse the task. Whether that's a feature depends on which side of the firewall you sit.

The Hacker News thread was less impressed. One commenter said it "performed pretty poorly (on par with Kimi K3)" on their private adversarial programming benchmark. Another pointed out it costs "about 10x of Deepseek" on OpenRouter and asked why they'd use it. The defense was mostly about EU data residency and GDPR, which is a real reason for some teams and no reason at all for a hobbyist with a GPU.

How much memory a 1T MoE needs at Q4 and Q2

The usual mistake with MoE models is reading "52B active" as "52B-sized hardware". Every expert has to be resident somewhere, because the router picks different ones per token. You need memory for all 1.05T parameters. The 52B only decides how many bytes you read per token, which is what sets your speed.

No GGUFs exist yet, so the closest real-world anchor is Kimi K2, another ~1T MoE. Unsloth's Kimi K2 guide publishes actual file sizes. Scaling those by 1.05 gives a rough picture. These are my estimates, not published Mistral numbers:

Quant	Kimi K2 size (published)	Mistral Large 4 estimate
Q8 (full)	1.09TB	~1.14TB
UD-Q4_K_XL	588GB	~617GB
UD-Q3_K_XL	452GB	~475GB
UD-Q2_K_XL	381GB	~400GB
UD-TQ1_0 (1-bit)	245GB	~257GB

Add KV cache on top. How much depends on the attention design, which Mistral hasn't published, and at long context it won't be small.

Speed is the other catch. Kimi K2 has 32B active parameters. Mistral Large 4 has 52B. On the same box at the same quant, expect it to run roughly 1.6x slower per token, because decode on these setups is limited by memory bandwidth and you're reading about 60% more weights for each token.

Hardware that can actually load Mistral Large 4

Multi-GPU servers

Unsloth says full Q8 Kimi K2 needs at least 8x H200. An 8x H200 node has 1,128GB of HBM, which fits Q4 Mistral Large 4 comfortably with room for KV cache. 8x H100 (640GB) technically holds ~617GB of Q4 weights and leaves you almost nothing for context. Realistically that's Q3 territory.

Mac Studio

The lineup moved around this year. In early March, Cult of Mac reported that Apple had quietly dropped the 512GB M3 Ultra option, capping it at 256GB. The M5 Ultra Mac Studio brought 512GB back in September. A 512GB box fits the Q2 or Q3 estimate above, but not Q4. A 256GB Mac can't fully hold even the ~257GB 1-bit estimate, so expect to spill to SSD.

CPU offload with llama.cpp

This is how most people will run it. Keep attention and shared layers on the GPU and push the expert tensors to system RAM. llama.cpp has dedicated flags for this: --cpu-moe keeps all MoE weights on CPU, --n-cpu-moe N keeps the experts of the first N layers there, and -ot / --override-tensor gives you regex control.

Once GGUFs appear, the command will look like Unsloth's Kimi K2 recipe. The repo name below is a guess, so swap in whatever actually gets published:

./llama.cpp/llama-cli \
    -hf <publisher>/Mistral-Large-4-GGUF:UD-Q2_K_XL \
    --n-gpu-layers 99 \
    --ctx-size 16384 \
    -ot ".ffn_.*_exps.=CPU"

If you have spare VRAM, use --n-cpu-moe and lower N until you run out. Every expert layer you pull back onto the GPU helps.

For expectations, Unsloth reports 1 to 2 tokens/s for the 1-bit Kimi K2 on one 24GB GPU plus 256GB of RAM, 5+ tokens/s once combined RAM+VRAM covers the whole quant (about 247GB for that one), and under 1 token/s once it spills to disk. Mistral Large 4 reads more bytes per token, so knock those numbers down. A consumer desktop that tops out at 128 or 192GB of RAM is not in the game. Look at used EPYC or Xeon boards with lots of memory channels.

Try Mistral Large 4 through the API first

If you want to know whether it's worth 400GB of RAM, spend a few dollars first. Mistral's docs use the slug mistral-large-4-0, but I couldn't find Mistral confirming the exact API model ID, so check the list before you hardcode anything:

curl -s https://api.mistral.ai/v1/models \
  -H "Authorization: Bearer $MISTRAL_API_KEY" | jq -r '.data[].id' | grep large

curl -s https://api.mistral.ai/v1/chat/completions \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-large-4-0",
    "messages": [{"role": "user", "content": "Find the race condition in this Go snippet: ..."}]
  }'

On OpenRouter it's mistralai/mistral-large-4-0. Run your own evals on your own code, not the pelican test everyone already ran.

Some napkin math: at $4.18 per million output tokens, the $12,299 M5 Ultra config Guru3D lists (256GB memory, 4TB SSD) buys you about 2.9 billion output tokens. That Mac can't even hold the Q2 quant. Unless you need offline inference, EU-only data handling on your own metal, or you just like owning the thing (valid), the API wins on cost for a long time.

My take

Stop looking at the 52B. The 1.05T total decides your hardware, and at any quant worth using that means 400GB or more; the active count only tells you how slow it will be once loaded. For a local daily driver, a 30B to 120B MoE that fits in VRAM and runs at 30+ tokens/s will be more useful than a 1T model at 2 tokens/s. I'll download the Q2 on the 31st anyway, because of course I will. But until a license file shows up next to the weights, I'm treating "open-weight" as marketing.

So: will you run a 1T MoE at home when the weights drop, or is 52B active still out of reach for your rig? Post your setup (GPU, RAM, memory channels) and what quant you think you can fit. And if you think I'm wrong about the API being cheaper, show me your power bill.