Got a DGX Spark and want vLLM on it without the NGC container? Here is what actually happens on real hardware: NVIDIA GB10, cc 12.1 (sm_121), 48 SMs, 121.7 GiB unified memory, driver 580.159.03, CUDA 13.0, aarch64, Ubuntu 24.04.4. Model under test: unsloth/Qwen3.6-27B-NVFP4, FP8 KV, 65,536 context, TP=1, vLLM 0.27.1. Dated 2026-08-26.
The install that works
uv venv --python 3.12 ~/venvs/vllm
source ~/venvs/vllm/bin/activate
uv pip install vllm --torch-backend=auto
That resolves vLLM 0.27.1 + FlashInfer 0.6.16.post3 + torch 2.13.0+cu130 from plain PyPI in about 32 seconds. No nightly index, no container. The Spark ships with Python 3.14, which most ML wheels do not cover, so the pinned 3.12 venv is the whole trick.
The two landmines
- FlashInfer's JIT calls
ninjafrom PATH. Run the venv python by absolute path without activating and the first NVFP4 GEMM build dies withFileNotFoundError: 'ninja'. Activate the venv. - There is no prebuilt sm_121 artifact for the CUTLASS FP4 path, so the first boot JIT-builds it and looks hung. It is not. Cold start to
/health: 425 s. Warm: 255 s (weights 98-115 s, torch.compile 8.5 s cached, autotune 21 s, graph capture 13-20 s).
What the engine resolves to on sm_121
decode_backend=flashinfer-native, prefill=torch.bfloat16, decode=torch.bfloat16,
kv_cache_dtype=torch.float8_e4m3fn, arch=sm121
Default cudagraph mode is FULL_AND_PIECEWISE. XQA / TRT-LLM decode is gated off for family 12x on the 0.27.1 release (requesting it logs TRTLLM attention is not supported on this platform and falls back); the gate opens on current nightlies. After the 21.34 GiB model load, KV cache headroom is 83.65-86.23 GiB, roughly 2.5M tokens at this context length.
Booting is not correctness
A server that boots can still generate garbage, so the cell was verified with needle recall at 2.5K/10K/16K/49K token depths, temperature 0, across four configurations. All 16 probes returned the planted codeword exactly. Practical note for Qwen3.6 thinking models: pass chat_template_kwargs: {"enable_thinking": false} when you need deterministic short answers, or the reasoning stream eats your max_tokens.
Sources
Full datapoint with the release-vs-nightly gate code cites is in the FlashInfer SM121 audit thread: flashinfer-ai/flashinfer#3170. Comparable measured sm_120 cells: blackwell-serving-matrix. Identify your own cell with blackwell-doctor; it reports GB10 unified memory correctly as of v0.1.1.
