From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides — auto-updated hourly
xbill4d ago • 11 min read
Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050
Gemma 4 26B A4B, a 14.4 GB mixture-of-experts model, serves from an 8 GB RTX 3050 with most of its e...
#gemma#llamacpp#cuda#benchmarking
12 9
xbillOct 16, 2026 • 19 min read
Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs
Yes. Speculative decoding with Google's 59 MB QAT multi-token prediction (MTP) drafters speeds up Ge...
#gemma#llamacpp#cuda#benchmarking
8 3
xbill4d ago • 9 min read
How Much Context Fits Beside a 12B on an 8 GB GPU? Gemma 4 12B's Context Cache on an RTX 3050
A 4-bit context cache lets Gemma 4 12B hold 81,920 tokens beside its drafter on an 8 GB RTX 3050, 3....
#gemma#llamacpp#cuda#benchmarking
2 0
Helgard6d ago • 7 min read
Running NVIDIA's new 120B Nemotron-3.5-Super on an 8 GB GPU — skipping the zeros, and finding the real bottleneck
NVIDIA released Nemotron-3.5-Super on October 15: a 120B mixture-of-experts model, 75 GB even in...
#ai#llm#cuda#python
0 0
xbill5d ago • 15 min read
Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050
Yes. An exact Q4_0 rebuild of Google's Gemma 4 12B QAT GGUF runs with every layer on an 8 GB RTX 305...
#gemma#llamacpp#cuda#benchmarking
7 0
Arunav Pathak1d ago • 4 min read
Project EREN: My Journey to Building a CUDA-Powered LLM Inference Engine
Hey DEV Community! 👋 This is my first post here, and I'm excited to share a project I've recently....
#ai#llm#cuda#opensource
1 0
xbill3d ago • 10 min read
A Dense 31B on an 8 GB GPU: Gemma 4 31B on an RTX 3050
Gemma 4 31B, a dense 17.7 GB model, serves from an 8 GB RTX 3050 with 40 of its 60 layers in system ...
#gemma#llamacpp#cuda#benchmarking
10 0
jamilxt2d ago • 8 min read
TorchTPU vs CUDA: Can Native PyTorch Make Google TPUs Easier to Use?
A PyTorch model can be portable in the way a recipe is portable: the ingredients and steps look...
#pytorch#tpu#cuda#ai
1 0
xbill5d ago • 13 min read
Comparing Qwen and Gemma on the Low End: Qwen Is Faster Without Drafting, Gemma Has Better Quality
Step-by-step: serving Qwen3.5-2B with llama.cpp on an 8 GB RTX 3050 and a 4 GB GTX 1650 Ti beside Ge...
#gemma#qwen#llamacpp#cuda
14 0
xbill2d ago • 9 min read
Qwen and Gemma With Thinking On: Thinking Pays Qwen on Tool Calls, Gemma Still Leads
Step-by-step: turning thinking on for Qwen3.5-2B and Gemma 4 E2B under llama.cpp on a 4 GB GTX 1650 ...
#gemma#qwen#llamacpp#cuda
1 1
Hacks.gr4d ago • 2 min read
Amazon: New TorchNeuron tool challenges Nvidia's dominance
Amazon’s TorchNeuron lets applications built with PyTorch run on its Trainium chips with a simple se...
#torchneuron#pytorch#trainium#cuda
1 0