A mid-range gaming GPU like an RTX 4060 ships with 8 GB of VRAM, which is enough to hold a 7-billion-parameter language model and answer faster than you can read. That card spends most of its life rendering desktop wallpaper while you pay a monthly fee to run prompts on a rented server. Off Grid AI Desktop is a free, open-source app that runs language models directly on your Windows PC.
Free, open source, runs offline. No account.
The Off Grid AI Desktop chat, running a local model fully on-device.
What You Need
On Windows, your GPU's VRAM is the number that matters most. An NVIDIA card with CUDA is the fastest path. No discrete GPU is fine too: the app falls back to your CPU and system RAM.
| Tier | GPU | VRAM | System RAM | Free disk |
|---|---|---|---|---|
| CPU only | none / integrated | n/a | 16 GB | 10 GB |
| Minimum | GTX 1660 / RTX 3050 | 6 GB | 16 GB | 20 GB |
| Recommended | RTX 4060 / 4070 | 8 to 12 GB | 16 GB | 30 GB |
| Comfortable | RTX 4080 / 4090 | 16 to 24 GB | 32 GB+ | 50 GB+ |
VRAM sets the ceiling. A model that fits entirely in VRAM runs fast. A model larger than your VRAM spills into system RAM and slows down, but it still runs.
What Off Grid AI Desktop Can Do
You get a chat window that works like ChatGPT, except the model runs on your own GPU and nothing leaves the PC.
- Chat with a local model, with temperature and context-window controls in plain sight.
- Browse and download models from Hugging Face inside the app, no command line needed.
- Render the model's HTML, SVG, Mermaid, and React output live in a sandboxed preview with no network access.
- Run any compatible GGUF file. The format is open, so you are never tied to one provider's lineup.
The win is not the list. The win is asking a model to refactor a function, draft a reply, or explain an error message with your internet unplugged, and getting an answer.
Which Models to Use
Match the model to your VRAM. Quantized GGUF files are compressed, so a model that needs 28 GB at full precision drops to roughly 8 GB at Q4.
| Your GPU | Model to start with | What to expect |
|---|---|---|
| No GPU (CPU + 16 GB RAM) | Gemma 3 1B or Qwen 2.5 3B | Works, slower. Good for short tasks and summaries. |
| 6 to 8 GB VRAM | Qwen 2.5 7B (Q4) | Fits in VRAM, fast, strong general reasoning. |
| 12 to 16 GB VRAM | Qwen 2.5 14B (Q4) | Closer to cloud quality, longer answers. |
| 24 GB VRAM | 14B at Q8 or a 32B at Q4 | Best local quality, room for long context. |
The rule of thumb: keep the whole model inside VRAM. The moment it overflows into system RAM, speed drops sharply, so size down one step before that happens.
How Hardware Acceleration Works
Off Grid AI Desktop bundles llama.cpp and offloads the model's layers to your GPU. On an NVIDIA card it uses CUDA, NVIDIA's compute framework. On AMD, Intel Arc, or other GPUs it uses Vulkan, the cross-vendor graphics API. With no usable GPU, it runs the same model on your CPU.
Each layer of the model you can fit in VRAM runs on the GPU's thousands of parallel cores instead of your CPU's handful. That is the whole speed story. The weights are quantized to 4 or 8 bits so they fit in consumer VRAM and the math stays fast. Download the file once, and it runs against your own card from then on.
Keeping It Fast
A few settings move the needle on Windows.
- Fit the model in VRAM. Check your card's VRAM, then pick a quantized model a little smaller than that. This is the single biggest speed factor.
- Prefer Q4_K_M. It is the balance point: small enough to fit, accurate enough to trust. Move to Q8 only if VRAM allows.
- Keep your GPU driver current. A fresh NVIDIA or AMD driver matters for CUDA and Vulkan performance.
- Mind the context window. Long context uses more VRAM and slows the first token. Keep it short when you only need a short answer.
Privacy: Stronger Than ChatGPT
A cloud chatbot sends every prompt to a remote data center where it is logged and may feed the next training run. With Off Grid AI Desktop, your prompt goes from your keyboard to a process on your own PC and straight back.
No account. No telemetry. No API key. The app is AGPL-3.0, so you can read the full source on GitHub and confirm nothing phones home. Disconnect the network and it keeps running.
Getting Started
- Open the repo at github.com/off-grid-ai/desktop and download the latest Windows build, or clone and build it yourself.
- Install and launch the app.
- Open the Models browser and download one model that fits your VRAM tier above.
- Open a chat, select that model, and send your first prompt.
- Disconnect your network and send another, to confirm it runs on your hardware alone.
git clone https://github.com/off-grid-ai/desktop
cd desktop
npm install
npm run dev
What's Coming
- More bundled models and one-click presets for common tasks.
- Cross-device sync so your chats and models follow you between machines, still local-first.
- Unified search across your chats and documents.
The built-in model browser. Download text, vision, image, and voice models from Hugging Face, sized to your machine.
FAQ
Q: Is it actually free?
Yes. Free and open source under AGPL-3.0. Local chat and the Models browser have no paywall.
Q: Do I need an NVIDIA GPU?
No. NVIDIA with CUDA is fastest. AMD and Intel GPUs run through Vulkan, and a CPU-only PC works too, just slower.
Q: Does it really work offline?
Yes. After a model downloads, the app needs no network. Downloading is the only online step.
Q: How much VRAM do I need?
6 to 8 GB runs a 7B model well. 12 to 16 GB handles 14B models. With no GPU you use system RAM and a smaller model.
Q: Which models can I run?
Any compatible GGUF model, such as Gemma or Qwen. You browse and download them in the app.
Q: Is my data private?
Your prompts never leave the PC. No account, no telemetry, and the source is public so you can check for yourself.
Run a real model on your own Windows PC today, with nothing leaving your drive.
