From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides — auto-updated hourly
xbill4d ago • 11 min read
Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050
Gemma 4 26B A4B, a 14.4 GB mixture-of-experts model, serves from an 8 GB RTX 3050 with most of its e...
#gemma#llamacpp#cuda#benchmarking
12 9
xbillOct 16, 2026 • 19 min read
Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs
Yes. Speculative decoding with Google's 59 MB QAT multi-token prediction (MTP) drafters speeds up Ge...
#gemma#llamacpp#cuda#benchmarking
8 3
xbill4d ago • 9 min read
How Much Context Fits Beside a 12B on an 8 GB GPU? Gemma 4 12B's Context Cache on an RTX 3050
A 4-bit context cache lets Gemma 4 12B hold 81,920 tokens beside its drafter on an 8 GB RTX 3050, 3....
#gemma#llamacpp#cuda#benchmarking
2 0
Victor2d ago • 6 min read
My AI App Waits 2.3 Seconds Before Saying a Single Word. Here's Exactly Where That Time Goes.
323 requests, 8 test conditions, one production deployment. The slow first word is not the model's.....
#performance#llm#buildinpublic#benchmarking
1 1
Alex Chen1d ago • 5 min read
SGLang vs vLLM: Your Agent Is Slow Because You Picked the Wrong Cache
If you serve LLM agents in production, you have probably seen this week's numbers: in a multi-turn.....
#llm#opensource#benchmarking#machinelearning
0 0
xbill3d ago • 10 min read
A Dense 31B on an 8 GB GPU: Gemma 4 31B on an RTX 3050
Gemma 4 31B, a dense 17.7 GB model, serves from an 8 GB RTX 3050 with 40 of its 60 layers in system ...
#gemma#llamacpp#cuda#benchmarking
10 0
xbill5d ago • 15 min read
Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050
Yes. An exact Q4_0 rebuild of Google's Gemma 4 12B QAT GGUF runs with every layer on an 8 GB RTX 305...
#gemma#llamacpp#cuda#benchmarking
7 0
RESK3d ago • 5 min read
AI Agent Evaluation End to End: How LFORLA Scores a LEGO Build From Brief to Render
A mechanism explainer of the LFORLA LEGO Set Build benchmark: how agent capability is measured end t...
#ai#benchmarking#agents#evaluation
0 0