The A17 Pro in your iPhone has a 16-core Neural Engine capable of 35 trillion operations per second. Apple uses it for photo processing and autocorrect. You can use it to run full AI models locally — no iCloud, no subscription, no data leaving your phone.
Off Grid is a free, open-source app that runs AI models locally on your iPhone. No internet connection after the initial model download. No Apple ID required for AI features. No data sent anywhere. This guide covers how to set it up, which models work best on iOS, and what performance to expect.
What You Need
Minimum hardware: iPhone with 6GB RAM (iPhone 13 Pro and newer). A17 or A16 chip. You can start with models as small as 80MB.
Recommended hardware: iPhone 15 Pro or newer (8GB RAM). This opens up 3B+ parameter models that produce genuinely useful output.
What you're giving up vs cloud AI: Cloud AI services like ChatGPT and Claude run models with hundreds of billions of parameters on data center GPUs. Running AI locally on your iPhone means smaller models (1B to 7B parameters). The output is less sophisticated for complex reasoning, but for everyday tasks like quick questions, summarization, drafting, and document analysis, it's surprisingly capable.
What Off Grid Can Do
Off Grid isn't just a text chatbot. It runs six AI capabilities locally in a single app, all on your device:
Text Generation
|
Image Generation
|
Vision AI
|
Attachments
|
Onboarding
|
Text generation. Run Qwen 3, Llama 3.2, Gemma 3, Phi-4, or any GGUF model. Streaming responses with markdown rendering. Metal GPU acceleration on all modern iPhones gives you 15 to 25 tokens per second.
Image generation. Local Stable Diffusion using Apple's Core ML pipeline. Neural Engine acceleration for fast generation. Models include SD 1.5, SD 2.1, and SDXL optimized for iOS.
Vision AI. Point your camera at something or attach an image and ask questions about it. SmolVLM and Qwen3-VL run at about 7 seconds on iPhone 15 Pro.
Voice transcription. Local Whisper speech to text. Hold to record, real-time partial transcription. No audio ever leaves your phone.
Tool calling. Models that support function calling can use built-in tools: web search, calculator, date/time, device info. The model chains them together in an automatic loop with runaway prevention.
Document analysis. Attach PDFs, code files, CSVs, and more to your conversations. Native PDFKit extraction on iOS.
Which Models to Use
Off Grid's model browser filters by your device's RAM so you never download something your iPhone can't run. Here's what works on real hardware:
6GB RAM iPhones (iPhone 13 Pro, 14 Pro): Stick with 1B to 2B models. Qwen 3 0.6B or SmolLM3 are good starting points. Expect 8 to 12 tokens per second. Usable for short answers and simple tasks.
8GB RAM iPhones (iPhone 15 Pro, 16 Pro): The sweet spot. Qwen 3 1.5B or Phi-4 Mini give you noticeably better output quality. 15 to 25 tokens per second with Metal GPU acceleration.
Quantization matters. A Q4_K_M quantized model uses roughly half the memory of a full precision version with minimal quality loss. Always go for Q4 or Q5 quantization on mobile.
You can also import your own .gguf files from device storage if you already have models downloaded.
How Local AI Actually Works on Your iPhone
When you run AI locally, the model weights are loaded into your iPhone's RAM and inference runs on your processor. There's no server involved. After you download a model from HuggingFace, every computation happens on your device.
Off Grid uses llama.cpp compiled for Apple Silicon with Metal GPU acceleration. The same engine powers most local AI on Mac — it runs natively on your iPhone's GPU, which is significantly faster than CPU-only inference.
For image generation, Off Grid uses Apple's own Core ML Stable Diffusion pipeline, running on the Neural Engine for maximum efficiency. The same hardware Apple uses for computational photography is generating your AI images.
The result is that your conversations, your documents, your images — none of it ever touches a network. You can verify this by turning on airplane mode. Everything works.
Memory Safety on iOS
iOS is aggressive about killing apps that use too much memory. A model that technically fits in RAM can still get killed if iOS decides it needs memory for something else.
Off Grid handles this with pre-load memory checks. Before loading any model, it calculates the actual RAM needed (model file size x 1.5 for text models) and compares it to available memory. On devices with 4GB RAM or less, GPU acceleration is automatically disabled to prevent Metal buffer allocation crashes.
You'll see a clear warning if a model won't fit, instead of the app silently closing.
The KV Cache Trick That Triples Your Speed
This is the single biggest performance win most people miss. The KV cache stores the context of your conversation. By default it uses f16 (16-bit floating point). Off Grid lets you switch to q4_0 (4-bit quantization) in settings.
Going from f16 to q4_0 roughly triples inference speed with minimal quality impact on most models. The app nudges you to optimize after your first generation.
Why Run AI Locally Instead of Using the Cloud
Running AI locally on your iPhone removes three dependencies at once.
No subscription. Cloud AI costs $20/month and up. Local AI is free after the one-time model download.
No internet. Local models work on a plane, in a basement, anywhere. No connectivity means no interruption.
No data exposure. When you type a prompt into a cloud AI, that text is stored on a server you don't control. Running locally means your medical questions, legal documents, personal journal entries, and proprietary work notes never leave your device. Off Grid is open source — you don't have to trust anyone's privacy claims. Read the code yourself.
How This Compares to Apple Intelligence
Apple Intelligence runs some tasks on-device but routes others through Apple's Private Cloud Compute servers. It's limited to Apple's own models and locked to the Apple ecosystem. You can't choose your model, you can't verify what runs locally versus in the cloud, and you can't use it on Android.
Off Grid runs everything locally. You pick the model. You verify the code. It works across iOS, Android, and macOS. No walled garden.
Getting Started
- Install Off Grid from the App Store
- Open the model browser and pick a recommended model for your device's RAM
- Download the model over WiFi (sizes range from 80MB to 4GB+)
- Turn on airplane mode to verify it works locally
- Start chatting
The first generation will be slower as the model loads into memory. Subsequent messages are faster. Go into settings and switch KV cache to q4_0 for the best speed.
Off Grid also runs natively on Apple Silicon Macs via Mac Catalyst. Same app, same models, same privacy.
What's Next
Apple's next generation chips are expected to push local AI inference even faster. Model optimization techniques keep improving quality at smaller sizes. The gap between local and cloud AI gets smaller every quarter.
Off Grid is under active development with new features shipping weekly. Voice mode, tool calling, configurable KV cache, and vision support all shipped in the last month. Check the GitHub for the latest releases.
A year from now, running AI locally on your phone won't be a power user trick. It'll be the default.
![Defluffer - reduce token usage 📉 by 45% using this one simple trick! [Earthday challenge]](https://media2.dev.to/dynamic/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fiekbgepcutl4jse0sfs0.png)
