LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

TL;DR

Liquid AI released LFM2.5-24B-A2B on Hugging Face this month: 23.97B total parameters, 2.45B active per token, 131,072-token context, GGUF on day one. For on-device use, three numbers decide whether it fits your hardware, and all three can be computed from config.json and the GGUF file list instead of taken from a launch post:

• RAM floor: about 14.5 GB for the Q4_K_M weights (4.85 bits per weight). A 24B MoE costs 24B of memory even though it computes like a 2B model.
• KV cache: 20 KiB per token in FP16, because only 10 of 40 layers use attention. A full 128K context is 2.5 GiB, not the 10 GiB a plain transformer of this width would need.
• Decode ceiling: about 67 tokens per second per 100 GB/s of memory bandwidth at Q4_K_M, since each token streams roughly 1.5 GB of weights.

This is a budget calculation, not a benchmark. We did not run the model for this post. Everything below is derived from published files, and the one speed claim from the vendor is marked as theirs.

Why does a 24B model with 2.45B active parameters still need 14.5 GB of RAM?

Mixture-of-experts models split compute from storage. The router picks 4 of 64 experts per layer for each token, so compute looks like a 2B model. But the router can pick any expert on the next token, so every expert has to be resident, or paged in from fast storage, to keep decoding smooth.

The published files make this concrete. These are the GGUF sizes from the LiquidAI/LFM2.5-24B-A2B-GGUF repository, with bits per weight computed against the 23,971,587,712 parameters in the GGUF metadata:

The last column multiplies the 2.45B active parameters by the bits per weight. It is an approximation, because K-quants mix precisions per tensor, but it is close enough for sizing.

Two things stand out. First, QAD-Q4_0 is the smallest 4-bit file. QAD in the file name points to quantization-aware distillation, which, if it works as the name suggests, would make it a better 4-bit default than a post-hoc K-quant, but the repository README is short and we did not find an accuracy table for it. Treat that as a hypothesis to test on your own task, not a claim. Second, going from Q4_K_M to Q8_0 costs 11 GB more RAM for a 1.76x increase in bits per weight. On a 16 GB machine that difference decides whether the model loads at all.

Where do the 2.45B active parameters actually go?

You can reconstruct the active count from the config, which is a good habit before trusting any "A2B" label. From config.json: hidden size 2048, MoE intermediate size 1536, 64 experts, top-4 routing, 40 layers of which 2 are dense, so 38 MoE layers.

Assuming a SwiGLU expert has three projection matrices (gate, up, down):

h, moe_inter, experts, top_k, moe_layers = 2048, 1536, 64, 4, 38
per_expert = 3 * h * moe_inter                      # 9,437,184 params
all_experts = per_expert * experts * moe_layers     # 22.95 B
active_experts = per_expert * top_k * moe_layers    # 1.43 B
total = 23_971_587_712
always_on = total - all_experts                     # 1.02 B
print(always_on + active_experts)                   # 2.4548 B

The result, 2.4548B, matches the model card's 2.45B. That also tells us how the active budget splits: about 1.43B (58%) are the four selected experts, and about 1.02B (42%) are always-on weights: the attention and convolution blocks, the two dense feed-forward layers, and the tied 128,000-token embedding and output matrix.

Why this matters on-device: the always-on 1.02B is read for every single token no matter what the router does. If you ever offload experts to slower storage, the always-on part is what you pin in fast memory.

How big is the KV cache when 30 of 40 layers are convolutions?

This is the quiet edge-AI feature of the architecture. The config's layer_types list shows a repeating pattern of three conv layers followed by one full_attention layer, 30 convolution and 10 attention layers in total. Only attention layers keep a per-token KV cache. The convolution layers carry a small fixed-size state (conv_L_cache is 3), which does not grow with context.

With 8 KV heads and a head dimension of 64 (hidden size 2048 divided by 32 attention heads; the config has no explicit head_dim, so this is our assumption), the FP16 cost per token is:

2 (K and V) x 10 attention layers x 8 KV heads x 64 dims x 2 bytes = 20,480 bytes

The right-hand column is hypothetical, a same-width model with attention in every layer, shown only to isolate what the hybrid design saves. At 128K the saving is 7.5 GiB, which is larger than the entire RAM headroom of many laptops. With -ctk q8_0 -ctv q8_0 in llama.cpp the cache shrinks further (about 1.33 GiB at 128K using 1.0625 bytes per element), though a quantized V cache generally requires flash attention to be enabled, so check your build.

The convolution state is negligible: roughly 30 layers x 2048 channels x 3 positions x 2 bytes is about 0.35 MiB.

Will it fit on my machine?

Weights plus FP16 KV cache for Q4_K_M, before the operating system, the runtime's compute buffers, and anything else you have open:

The honest reading: on a 16 GiB machine this model is borderline even at 32K context, because 1.85 GiB of headroom has to cover the OS, the compute graph, and your application. In practice that means 24 GiB is the comfortable minimum for Q4_K_M at long context, and 16 GiB is a "close everything else" configuration, or a case for mmap with some paging. Phones with 8 to 12 GiB cannot hold these weights at all, which is why the smaller LFM2.5 siblings exist for that tier.

What speed should I expect? A bandwidth ceiling, not a promise

Autoregressive decoding is bandwidth-bound: each new token must read every active weight from memory once. The ceiling is simply memory bandwidth divided by active bytes per token. For Q4_K_M that is 1.48 GB:

active_bytes = 2.45e9 * 4.85 / 8          # ~1.48e9 bytes per token
for bw_gbs in (50, 100, 200, 400):
    print(bw_gbs, round(bw_gbs * 1e9 / active_bytes, 1))
# 50 -> 33.7, 100 -> 67.4, 200 -> 134.7, 400 -> 269.4 tokens/s

The bandwidth tiers are illustrative round numbers, not specific devices. Real throughput lands below the ceiling because of compute, KV reads, expert routing overhead, and cache effects, so treat these as upper bounds you can sanity-check measurements against.

The vendor reports 184 tokens per second on an Apple M5 Max. We could not confirm which quantization was used for that figure from the model card. If it was a 4-bit-class format, 184 tokens per second times roughly 1.4 to 1.5 GB per token implies something like 255 to 270 GB/s of effective weight traffic. That is a consistency check on the claim, not a measurement of ours, and it suggests the number is plausible only on a high-bandwidth unified-memory machine, not on a typical thin laptop.

Two caveats on the ceiling. It applies to decode only; during prompt processing many tokens are in flight, the router touches most experts, and the workload becomes compute-bound. And the model is a pure reasoning model: the chat template appends <think> to every assistant turn, so a visible answer is preceded by reasoning tokens. Cheap tokens are good, but you will generate many of them.

How do I run it with a tight memory budget?

A starting point with llama.cpp, using flags that exist in the current server documentation:

llama-server \
  -m LFM2.5-24B-A2B-Q4_K_M.gguf \
  -c 32768 \
  -ctk q8_0 -ctv q8_0 -fa on \
  --jinja

If the weights do not fit in GPU memory, --n-cpu-moe N keeps the expert weights of the first N layers in system RAM, and --cpu-moe keeps all of them there. That maps well to the parameter split above: the 1.02B always-on weights stay on the accelerator, and the 1.43B-per-token expert traffic comes from RAM. Because only 4 of 64 experts are read per layer per token, the traffic per token is far smaller than the 14.5 GB resident size suggests. Measure it, because routing locality varies by workload.

Generation settings from the model card: temperature 0.1, top_k 50, repetition penalty 1.05. Tool calls are emitted as Pythonic lists between <|tool_call_start|> and <|tool_call_end|> tokens unless the system prompt asks for JSON.

What are the limits?

• We did not benchmark it. The numbers here are arithmetic on published files. Quality on your task needs your own evaluation.
• The model card says it is not recommended for knowledge-heavy tasks alone and suggests pairing it with retrieval or web search. That fits the on-device agent role it targets.
• The license is a custom lfm1.0 license, listed as "other" on Hugging Face, not Apache 2.0 or MIT. Read the LICENSE file before shipping it in a product.
• Vendor benchmark claims are the vendor's. The LiveClaw score of 48.09 against 38.19 for Qwen3.6-35B-A3B comes from the model card, and we have not reproduced it.
• Head dimension 64 and the three-matrix expert structure are inferred from config values, not stated explicitly. The reconstruction landing on 2.4548B against the stated 2.45B supports the second one.

FAQ

How much RAM does LFM2.5-24B-A2B need?
About 14.5 GB for the Q4_K_M GGUF weights plus roughly 0.6 GiB of FP16 KV cache at 32K context, so plan for 16 GiB at the very least and 24 GiB for comfort.

Why is a 2B-active model not 2B in memory?
Because the router can select any of the 64 experts for the next token. All 23.97B parameters must be resident, or quickly pageable, even though only 2.45B are read per token.

How big is the KV cache at 128K context?
2.5 GiB in FP16, from 20,480 bytes per token across 10 attention layers with 8 KV heads of dimension 64. About 1.33 GiB with a q8_0 cache.

What decode speed is realistic on a laptop?
The ceiling is about 67 tokens per second per 100 GB/s of memory bandwidth at Q4_K_M. Measured speed will be lower. Check your machine's memory bandwidth first.

Which GGUF should I pick?
Start with Q4_K_M, or try QAD-Q4_0 if you need 1 GB less and can test quality on your own prompts. Move up to Q5_K_M only if you have the RAM and see quality problems.

Further reading

• Offline AI on a Phone: Thread Pinning, Lazy Loading, and a Safety Gate That Knows When to Stay Quiet
• 4-bit GGUF Quality for MoE Models: Why Only 3B of 180B Params Fire, and How to Prove Parity

Sources: LFM2.5-24B-A2B model card and config, GGUF repository, llama.cpp server documentation.