Zed + Ollama Local Model Setup 2026: Agent, Inline Assist, and Running Zeta Edit Prediction Offline

By AICoderScope Team · 10 min read
zedollamalocal-llmsetup-guidezetaprivacyai

TL;DR: Zed is the fastest native editor with built-in local-model support — you point it at Ollama and get chat, inline assist, and (uniquely) local edit prediction with no cloud in the loop. Setup takes 15 minutes. The catch: agentic multi-file editing with a local model is still unreliable in 2026 because most local models fumble Zed’s tool calls. Use it for autocomplete and single-file work, not autonomous agents.

  • What you’ll be able to do after this guide: run the Assistant/Agent panel, inline assist (ctrl-enter), and offline edit prediction against a local model — no API key, nothing leaving your machine.
  • What you’ll need: Zed (stable), Ollama running locally, a GPU with ≥12 GB VRAM or Apple Silicon with ≥32 GB unified memory, and about 15 minutes.
  • Where this setup hits its ceiling: agent mode that edits across many files — local models get stuck on tool calls, so cloud models still win for autonomous work.

Honest take: If you already live in Zed and want free, private autocomplete plus chat, this is the best local setup available — nothing else pairs a Rust-fast editor with a locally-runnable edit-prediction model. But if your goal is a hands-off coding agent that edits ten files on its own, keep a cloud model on the Agent panel and use local only for the inline and prediction work.

Why Zed for local models

Most editors bolt local-model support on through a third-party extension that still routes a telemetry ping somewhere. Zed builds the LLM provider layer into the editor itself: Ollama, LM Studio, llama.cpp, and any OpenAI-compatible endpoint are first-class providers, configured in settings.json. Inference runs on your machine, and Zed’s request goes to localhost — you can confirm it with a network monitor.

The unique lever here, new for 2026, is edit prediction. Zed’s own edit-prediction model, Zeta2, is open-weight and published on Hugging Face. That means the “predict my next edit” feature — the thing you’d normally need a Zed Pro subscription or GitHub Copilot for — can run entirely offline through Ollama. No other mainstream editor lets you self-host its inline-completion model. If privacy is the whole point of going local, this closes the last cloud gap that autocomplete usually leaves open.

For a full feature breakdown of the editor itself, see our Zed Editor AI review. This guide is only about the local-model path.

Hardware floor

What you can run is bounded by VRAM (or unified memory on a Mac). Rough fits for the coding models Zed handles well:

VRAM / MemoryRecommended modelRealistic use case
12 GB VRAM (RTX 3060 12GB)qwen3-coder:14b (Q4)Inline assist + chat; agent marginal
16 GB VRAM (RTX 4060 Ti 16GB)devstral (Q4)Best value for tool-calling attempts
24 GB VRAM (RTX 3090 / RTX 4090)devstral (Q5) or gpt-oss:20bBest local tier; agent sometimes works
32 GB Apple unified memory (Mac Studio M3 Ultra)qwen3-coder:14bmacOS sweet spot for chat + inline
64 GB+ Apple unified memorydevstral / gpt-oss:20bBest macOS local tier

For a deeper VRAM-by-model breakdown, runaihome.com maintains a best local AI models by VRAM guide that maps cleanly onto these tiers.

Step 1: Get Ollama running

Install Ollama from ollama.com/download, then pull a coding model. For agentic work in Zed, devstral is the model to start with — Mistral built it specifically for coding agents and it handles tool calls better than most local models:

ollama pull devstral
ollama serve

On macOS, opening the Ollama app starts the server automatically. Confirm it’s up:

curl http://localhost:11434/api/tags

You should get JSON listing your pulled models. If this fails, fix it before touching Zed — the most common cause is the server not running, which we cover in our Ollama connection refused / fetch failed fix.

Step 2: Point Zed at Ollama

Zed auto-discovers models that Ollama has pulled, so the fast path needs no config at all: click the star / assistant icon, open Configure, choose Ollama under LLM Providers, confirm the host is http://localhost:11434, and click Connect. Your pulled models appear in the model dropdown.

If you want explicit control — which you do, because of the context-window trap in Step 3 — declare the model in settings.json instead. Open the command palette and run zed: open settings:

{
  "language_models": {
    "ollama": {
      "api_url": "http://localhost:11434",
      "available_models": [
        {
          "name": "devstral",
          "display_name": "Devstral (local)",
          "max_tokens": 32768,
          "supports_tools": true,
          "supports_images": false
        }
      ]
    }
  }
}

The supports_tools: true line is what makes the Agent panel even attempt tool calls with this model. Without it, Zed treats the model as chat-only and the agent can’t edit files.

Step 3: Fix the context window (the trap that breaks everyone)

This is the single mistake that makes people give up on local models in Zed. Ollama defaults to a 4,096-token context window (num_ctx). Agentic coding blows past that instantly — your system prompt, the file, and the tool schema alone can exceed it — and Ollama silently truncates the rest. The model then behaves as if it forgot half the conversation, because it did.

Override it globally in settings.json:

{
  "language_models": {
    "ollama": {
      "api_url": "http://localhost:11434",
      "context_window": 32768
    }
  }
}

Match context_window to what your VRAM can actually hold — a larger context uses more memory. On 12 GB, 8192 is realistic; on 24 GB you can push 32768 with a 14B model. If Zed starts responding slowly or the GPU spills into system RAM, drop the number.

Step 4: Set the model per feature

Zed lets you assign different models to different features. Set the Agent panel’s default in settings.json:

{
  "agent": {
    "version": "2",
    "default_model": {
      "provider": "ollama",
      "model": "devstral"
    }
  }
}

The same { "provider": "ollama", "model": "..." } shape works for inline_assistant_model, commit_message_model, and thread_summary_model under agent. A sensible local split: put your best model on the Agent, and a smaller/faster one on commit_message_model and thread_summary_model so summaries don’t stall your workflow.

Inline assist is where local models actually shine. Select code, hit ctrl-enter, describe the change, and Zed rewrites the selection in place. Because this is a single, bounded request rather than a multi-step tool loop, even a 14B model handles it well.

Step 5 (optional): Run Zeta edit prediction offline

This is the part no other editor offers. Zed’s edit-prediction model, Zeta2, is open-weight — pull it into Ollama and run inline “predict my next edit” completions locally, with none of the free-tier prediction cap (2,000/month on hosted Zeta) and no cloud round-trip.

ollama pull zeta2

Then in settings.json:

{
  "edit_predictions": {
    "provider": "ollama",
    "ollama": {
      "api_url": "http://localhost:11434",
      "model": "zeta2",
      "prompt_format": "infer",
      "max_output_tokens": 512
    }
  }
}

"prompt_format": "infer" tells Zed to auto-apply the correct Zeta prompt template based on the model name, so you don’t hand-write it. Now edit prediction — the low-latency, next-edit suggestions as you type — runs entirely on your machine. This pairs naturally with a local chat model to give you a 100% offline Zed.

Where this setup breaks (the honest section)

Every local-model guide that skips this section is selling you something. Here is where Zed + Ollama actually falls down in 2026:

  • Agentic tool-calling is unreliable. This is the big one. The Zed community’s own testing shows most local models get stuck wanting to make a tool call and never completing it, or revert to chatting instead of editing. qwen2.5-coder famously doesn’t do tool use in Zed out of the box; devstral and gpt-oss:20b are better but still miss. On a 32 GB machine, the consensus is blunt: you can’t write much useful code with small or quantized models in full agent mode. Keep a cloud model on the Agent panel for anything autonomous.
  • Context is smaller than you think. Even after the Step 3 fix, a local 14B at 8K–32K context can’t hold the codebase context a cloud model gets. Long agent threads degrade fast.
  • It’s slower. Local inference is meaningfully slower than a hosted API. For inline assist that’s fine; for a 10-step agent loop it compounds into real waiting.
  • Quantization adds instability. Aggressive quants (Q3 and below) introduce corrupted output and template bugs. Stay at Q4 or higher for anything you’ll actually use.

The takeaway isn’t “don’t go local” — it’s “go local for the right features.” Inline assist and edit prediction: excellent locally. Autonomous multi-file agent runs: keep them on the cloud, or accept that they’ll need babysitting.

The setup that actually works day to day

The pragmatic 2026 configuration for most developers is a hybrid: local Zeta2 for edit prediction, local devstral or qwen3-coder for inline assist and quick chat, and a cloud model (Claude, GPT) wired into the Agent panel for the heavy autonomous work. That keeps your keystroke-level activity — the stuff that reveals the most about what you’re building — fully private, while still giving you a capable agent when you need one. If your requirement is truly zero-cloud, everything above works offline; just set your expectations for agent mode accordingly.

FAQ

Does Zed support local models on Windows and Linux? Yes. The Ollama, LM Studio, and llama.cpp providers work on all three platforms Zed ships on. The settings.json config is identical; only the Ollama install differs.

Can I use LM Studio instead of Ollama? Yes. Start the LM Studio server (lms server start), and Zed’s LM Studio provider auto-discovers loaded models. If you’re deciding between them, we compare the two workflows in the Cline + LM Studio setup guide — the trade-offs are the same in Zed.

Why does the agent keep chatting instead of editing my files? Two usual causes: the model isn’t declared with "supports_tools": true, or the model itself can’t reliably call tools. Add the flag first; if it still won’t edit, switch to devstral or gpt-oss:20b, which are the most reliable local tool-callers in Zed today.

Do I need a Zed Pro subscription for any of this? No. Local models via Ollama and offline Zeta2 edit prediction both run without a subscription. Pro is only needed for Zed’s hosted models and the hosted Zeta prediction quota above the free 2,000/month.

Will this run on a CPU-only machine? Technically, but not usefully. A 14B model on CPU is far too slow for inline assist or edit prediction. You need a GPU or Apple Silicon with enough unified memory.

Sources

Last verified Jul 16, 2026. Zed version, Ollama version, Zeta model availability, and local tool-calling reliability change frequently — verify current versions and re-test agent behavior before relying on this setup.

Was this article helpful?