Can a Raspberry Pi 5 run a real language model on its own, with no cloud API anywhere in the loop?
Google says yes, and the new LiteRT CLI turns that claim into a ten-minute test you can run tonight. LiteRT is the successor to TensorFlow Lite, retuned for Arm boards, and Google engineers published a step-by-step walkthrough on the Raspberry Pi blog that takes you from a blank SD card to a Gemma chat session over SSH. No monitor, no keyboard juggling, just an Ethernet cable between your laptop and the Pi.
What the setup actually looks like
Flash Raspberry Pi OS (64-bit) with Imager, tick the SSH box, set a hostname, and boot. Once uname -m prints aarch64 you are in business. The install itself is three commands: pull down the uv environment manager, spin up a Python 3.13 venv, then uv pip install litert-cli-nightly. Models come from Hugging Face, so you also need a read-scope access token exported as HUGGING_FACE_HUB_TOKEN before the first download.
The numbers that matter
Running litert lm run against gemma-4-E2B-it pulls a 2.59 GB model file, which clocked roughly 20.4 MB/s on the demo machine, so budget SD card space accordingly. Classic vision models are far lighter: litert download litert-community/efficientnet_b1 gives you a .tflite file that classifies an image in one command, and the sample run scored a tiger shark at 7.4043 against 4.7619 for the runner-up. Here is the gotcha worth knowing before you chase it: GPU acceleration exists behind V3D_WEBGPU_OVERRIDE=1 on the Pi 5's V3DV Vulkan driver, but it is experimental and currently slower than the CPU path. Skip the --gpu flag unless you are specifically testing Dawn WebGPU. Hailo AI HAT+ offload is on the roadmap, which is where the real speedup will land.
Try it on your capstone
A Pi 5, a camera module, and EfficientNet is a complete object-classification pipeline for a thesis demo, and it keeps working when campus WiFi dies. Start with the vision models rather than the LLM, since a 2.59 GB download is a painful first step on a shared connection. Full instructions and the exact commands are on the Raspberry Pi blog: Get started with LiteRT on Raspberry Pi. Grab a Pi 5 and an active cooler from Circuit.Rocks, because sustained inference will heat that chip fast.
Originally published on blog.circuit.rocks.
