From Dev.to Community
Tech Articles
Curated developer articles, tutorials, and guides — auto-updated hourly
prodbymarcuOct 16, 2026 • 5 min read
I put the 381 tok/s Qwen stack on one RTX 3090, real chat does 118 and the headline mode crashes
A post went around earlier this month, 82 to 381 tok/s on a single RTX 3090 running a 27B model. I.....
#llm#inference#gpu#docker
1 1
Sachin Kr. Rajput1d ago • 12 min read
One Request, End to End: Tokenize, Prefill, Decode, Stream
You type "Why is the sky blue?" and press Enter. There's a short pause. Then words start appearing,...
#llm#machinelearning#inference#performance
2 1
jamilxt3d ago • 7 min read
DeepInfer vs vLLM: DeepSeek's New Engine Is Not a Drop-In Replacement
A new inference engine can sound like a simple upgrade: install it, point it at a model, and get...
#ai#inference#devtools#programming
1 0
ai maya4d ago • 10 min read
Ling-3.1-flash and the 7:1 hybrid: how much KV cache does a 560B model need at 1M tokens?
A 560B open model keeps full attention in only 9 of 72 layers. Here is the cache and serving math fo...
#ai#machinelearning#llm#inference
0 0
ai maya3d ago • 12 min read
Can two Mac Studios really run an 816B model? A bandwidth roofline check on Muse Spark Open
Meta's 816B open model runs on two Mac Studios at 15.7 to 36.7 tokens/s. Here is the memory and band...
#ai#machinelearning#llm#inference
0 0
amrit2d ago • 5 min read
How DeepInfer Slashed Latency by 2.4 and Cut Costs by 45% – The Future of AI on Edge
DeepInfer × DeepSeek – What the “breaking‑news” rollout means for high‑performance...
#ai#inference#deeplearning
0 0
EME GUG3d ago • 7 min read
Why LLM Inference Engines Are Fast: Continuous Batching, PagedAttention and Prefix Caching Explained with vLLM
Tuần này DeepSeek open-source inference engine của họ và lên top Hacker News với gần 1400 điểm. Đọc....
#llm#vllm#inference#performance
0 0
Anshul Rajpal6d ago • 5 min read
Magnitude: The Apache-2.0 Inference Engine That Claims 2x Faster Than llama.cpp
What Is Magnitude Magnitude is an Apache-2.0 inference engine that compiles and tunes...
#ai#tools#inference#opensource
0 0
Mariano Gobea Alcoba4d ago • 5 min read
Technical Analysis of DeepSeek V4.1 Flash 1017!
Architecting Throughput: An Analysis of DeepSeek V4.1 Flash 1017 (Max) The recent release...
#deepseek#llm#aibenchmarks#inference
0 0
Reno Lu3d ago • 3 min read
Strata runs a 125B model on a 12 GB gaming GPU by leaning on system RAM, and the README spells out the cost
Strata serves Qwen3.8-Flash-Next locally with OpenAI and Anthropic endpoints. Your RAM decides which...
#llm#localai#inference#ai
0 0
TechPulse 3d ago • 2 min read
DeepSeek Open-Sources DeepInfer, Its Production Inference Engine
DeepSeek open-sourced DeepInfer on October 20, 2026, the high-performance inference framework that.....
#deepseek#ai#inference#opensource
0 0
Hacks.gr4d ago • 1 min read
Tenstorrent raises $1,5 billion and doubles its value
AI chip maker Tenstorrent has secured $1.5 billion in a new funding round led by Trace Capital and I...
#tenstorrent#ai#hardware#inference
1 0