Your M-series Mac shares one pool of memory between the CPU and the GPU, so a model can use almost all of your RAM as if it were a graphics card. That hardware sits idle most of the day while you pay 20 dollars a month to send your prompts to someone else's server. Off Grid AI Desktop is a free, open-source app that runs language models directly on your Mac.
Free, open source, runs offline. No account.
What You Need
Apple Silicon does the heavy lifting here. Intel Macs can run small models on the CPU, but the experience is slow, so the tiers below assume an M-series chip.
| Tier | Chip | RAM | macOS | Free disk |
|---|---|---|---|---|
| Minimum | M1 / M2 | 8 GB | 13 Ventura | 10 GB |
| Recommended | M2 Pro / M3 | 16 GB | 14 Sonoma+ | 30 GB |
| Comfortable | M3 Max / M4 | 32 GB+ | 14+ | 50 GB+ |
More RAM means a bigger model or a longer context window. On 8 GB you run a 1B to 3B model and keep other apps light. On 32 GB you can hold a 14B model in memory and still have room to work.
What Off Grid AI Desktop Can Do
You get a chat window that behaves like ChatGPT, except the model lives on your drive and nothing leaves the machine.
- Chat with a local model, with temperature and context-window controls you can actually see and adjust.
- Browse and download models from Hugging Face inside the app, no terminal required.
- Render the model's HTML, SVG, Mermaid, and React output live in a sandboxed preview with no network access.
- Pick any compatible GGUF file and run it. The format is open, so you are not locked to one vendor's catalog.
The point is not the feature list. The point is that you can ask a model to draft an email, summarize a contract, or write a regex at 2am on a plane with no signal, and it answers.
Which Models to Use
Match the model to your RAM. Quantized GGUF files are compressed, so a model that would need 28 GB at full precision fits in 8 to 9 GB at Q4.
| Your Mac | Model to start with | What to expect |
|---|---|---|
| 8 GB (M1/M2) | Gemma 3 1B or Qwen 2.5 3B | Quick chat, summaries, short code. Snappy. |
| 16 GB (M2 Pro/M3) | Gemma 3 4B or Qwen 2.5 7B | Solid reasoning, longer answers, comfortable pace. |
| 32 GB+ (M3 Max/M4) | Qwen 2.5 14B or larger | Closer to cloud-model quality, longer context. |
Start one tier below what you think you need. A fast small model you actually use beats a large one that makes you wait.
How Hardware Acceleration Works
Off Grid AI Desktop bundles llama.cpp and runs the model through Metal, Apple's GPU framework. Because Apple Silicon uses unified memory, the GPU reads model weights straight from the same RAM the CPU uses. There is no copying weights across a PCIe bus the way a discrete graphics card does on a PC.
That single design choice is why a laptop with no separate GPU can run a 7B model at a usable speed. The weights are quantized to 4 or 8 bits, which shrinks them enough to fit and keeps the math fast. You download the file once, and it runs against your own silicon from then on.
Keeping It Fast
A few practical settings make a real difference.
- Pick the right quantization. Q4_K_M is the sweet spot for most Macs: small enough to fit, good enough to trust. Reach for Q8 only if you have RAM to spare and want maximum quality.
- Watch your context window. A long context eats memory and slows the first token. If you only need a short answer, keep the window modest.
- Close memory-hungry apps. A browser with 60 tabs competes with the model for the same unified memory. Quit what you do not need.
- Lower the temperature for facts. For code and structured answers, drop the temperature. Raise it for brainstorming.
Privacy: Stronger Than ChatGPT
When you type into a cloud chatbot, your prompt travels to a data center, gets logged, and may train the next model. With Off Grid AI Desktop, the prompt goes from your keyboard to a process on your own Mac and back.
No account. No telemetry. No API key. The app is AGPL-3.0, so you can read every line of the source on GitHub and confirm there is no phone-home. Pull the network cable and it keeps working.
Getting Started
- Open the repo at github.com/off-grid-ai/desktop and grab the latest macOS build, or clone and build it yourself.
- Install the app and launch it.
- Open the Models browser and download one model that fits your RAM tier from the table above.
- Open a chat, pick that model, and send your first prompt.
- Turn off your Wi-Fi and send another, to prove it runs on your machine alone.
git clone https://github.com/off-grid-ai/desktop
cd desktop
npm install
npm run dev
What's Coming
- More bundled models and one-click presets for common tasks.
- Cross-device sync so your chats and models follow you between machines, still local-first.
- Unified search across your chats and documents.
FAQ
Q: Is it actually free?
Yes. Free and open source under AGPL-3.0. There is no paywall on local chat or the Models browser.
Q: Does it really work offline?
Yes. Once a model is downloaded, the app needs no network. The download step is the only thing that touches the internet.
Q: Which models can I run?
Any compatible GGUF model. Gemma and Qwen are good starting points. You browse and download them inside the app.
Q: How much RAM do I need?
8 GB runs a small model. 16 GB is comfortable for 7B-class models. 32 GB and up lets you run 14B models with a long context window.
Q: Will this slow down my Mac?
Only while a model is generating, and only as much as you let it. Pick a smaller model or shorter context if you want headroom for other work.
Q: Is my data private?
Your prompts never leave the machine. No account, no telemetry, and the source is public so you can verify it.
Run a real model on your own Mac today, with nothing leaving your drive.
