From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides — auto-updated hourly
xbillApr 28, 2026 • 16 min read
Self-hosted Gemma 4 on TPU with vLLM, MCP, ADK, and Gemini CLI
This article provides a step by step deployment guide for Gemma 4 to v6e Trillium TPUs in an 8 core....
#vllm#googleadk#tpu#gemini
26 0
ANKUSH CHOUDHARY JOHALApr 29, 2026 • 14 min read
Performance Test: Ollama 0.5.0 vs. vLLM 0.4.0 Local LLM Inference Latency on NVIDIA RTX 5090 and AMD Radeon RX 8900 in 2026
In Q1 2026, we ran 12,000 inference requests across NVIDIA’s RTX 5090 and AMD’s Radeon RX 8900 to...
#performance#test#ollama#vllm
0 0
ANKUSH CHOUDHARY JOHALApr 29, 2026 • 14 min read
Why We Stopped Using vLLM 0.6 for Local LLMs in Favor of Ollama 0.5 for Code Tasks
After 14 months of running vLLM 0.6 in production for local code generation tasks, we’ve migrated...
#stopped#using#vllm#local
0 0
ANKUSH CHOUDHARY JOHALApr 29, 2026 • 16 min read
Comparison: vLLM 0.6 vs. Text Generation Inference 1.4 for Serving Code LLMs
Serving code LLMs at production scale is 3.2x more expensive than general-purpose LLMs when using...
#comparison#vllm#text#generation
0 0
Jangwook KimApr 28, 2026 • 10 min read
vLLM 0.8: Native Llama 4 MoE Routing Explained
How vLLM 0.8 achieves 40% throughput gains on MoE models via Expert Parallelism Load Balancing. Cove...
#vllm#moe#llama4#expertparallelism
0 0
ANKUSH CHOUDHARY JOHALApr 29, 2026 • 17 min read
LLM Inference Optimization: How to Use vLLM 0.6 and TensorRT 9.0 for 2x Throughput
\n If your LLM serving stack is stuck at 120 tokens/sec per A100, you’re leaving 50% of your...
#inference#optimization#vllm#tensorrt
0 0
ANKUSH CHOUDHARY JOHALApr 28, 2026 • 13 min read
vLLM 0.4.0 vs. Modal 0.55: LLM Serving Cost for 1M Requests
Serving 1 million LLM requests costs $1,240 with vLLM 0.4.0 on self-managed A100s, but $2,890 with.....
#vllm#modal#serving#cost
0 0
ANKUSH CHOUDHARY JOHALApr 28, 2026 • 18 min read
Deep Dive: How vLLM 0.6 Handles Batching for 2026 LLM Inference
In 2026, a single 10-trillion parameter LLM will require 240GB of VRAM just to load weights, yet...
#deep#dive#vllm#handles
0 0
ANKUSH CHOUDHARY JOHALApr 28, 2026 • 14 min read
vLLM 0.5 vs. Modal 0.60: LLM Inference Cost for 1000 RPM Workloads
At 1000 requests per minute (RPM), LLM inference costs can swing by 62% between self-hosted vLLM 0.5...
#vllm#modal#inference#cost
0 0
ANKUSH CHOUDHARY JOHALApr 28, 2026 • 14 min read
War Story: How a vLLM 0.6 OOM Error Crashed Our LLM Inference Service for 30 Minutes in 2026
At 14:17 UTC on March 12, 2026, our production LLM inference fleet running vLLM 0.6.0 hit a silent.....
#story#vllm#errors#crashed
0 0