From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides — auto-updated hourly
Jahn6d ago • 3 min read
Qwen3-8B on workstation Blackwell: vLLM vs SGLang vs llama.cpp, plus an FP8 pass
Benchmarks of the same model on the same GPU across three serving stacks, then an FP8 pass on the...
#machinelearning#cuda#performance#llm
1 1
Jahn6d ago • 3 min read
The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, a...
#cuda#llm#gpu#performance
0 0