Claude Code + Ollama in 2026: One Command to Run Anthropic's Terminal Agent on a Local Model

By AICoderScope Team · 11 min read
claude-codeollamalocal-llmsetup-guideanthropicprivacyai

TL;DR: Since Ollama v0.14.0 shipped a native Anthropic-compatible API in January 2026, Claude Code runs against local models with no proxy, no LiteLLM, no config file — ollama launch claude does the whole thing in one command. It works, it’s genuinely free, and your code never leaves the machine. The catch: a local 14B–30B model is a competent assistant, not the Fable-class agent the CLI was built around.

  • What you’ll be able to do after this guide: run Claude Code entirely against Ollama — chat, file edits, tool calling, subagents, and thinking — with zero API spend and zero cloud exposure.
  • What you’ll need: Ollama v0.15 or newer (current release is v0.32.1, July 16, 2026), Claude Code, and a GPU with 12 GB+ VRAM or Apple Silicon with 32 GB+ unified memory.
  • What you’ll avoid: the two failure modes that make people give up — the silent 4K context truncation and the background-model error that manual setups hit.

Honest take: If you’re evaluating this to save money on Claude, don’t — a local model is a downgrade from Sonnet-class output, and you’ll feel it on multi-file refactors. If you’re evaluating it for privacy, offline work, or unlimited experimentation on hardware you already own, this is the best local agent setup of 2026, and ollama launch claude is the fastest path to it.

Why this suddenly works without a proxy

For most of 2025, wiring Claude Code to a local model meant running LiteLLM or a homegrown translation shim, because Claude Code speaks Anthropic’s Messages API and Ollama spoke its own dialect plus OpenAI-compat. Every guide from that era has a proxy diagram in the middle.

That layer is gone. Ollama v0.14.0 (January 10, 2026) added native support for the Anthropic /v1/messages API — the exact protocol Claude Code uses to talk to Anthropic’s servers. Per the official compatibility docs, the endpoint supports streaming, system prompts, multi-turn conversations, vision, tool calling, tool results, and extended thinking. That list covers everything Claude Code actually needs to operate as an agent: per Ollama’s Claude Code integration guide, chat, command-line execution, tool calling, file edits, subagents, web search, web fetch, vision, and thinking all function against a local model.

Later that same month, Ollama v0.15 added the ollama launch command, which collapses the remaining setup — environment variables, model download, starting the tool — into a single step. Newer releases kept polishing this specific path: v0.30.11 (June 25, 2026) made launch auto-install Claude Code if it’s missing, and v0.31.2 (July 6, 2026) made it disable Claude Code’s telemetry by default.

Path 1: ollama launch claude (use this one)

If you have Ollama v0.15+ installed, the entire setup is:

ollama launch claude

Run bare, ollama launch opens an interactive picker listing the integrations it can configure — Claude Code, OpenCode, Codex, VS Code, and Droid. Pointing it at claude skips the menu. What it does under the hood, per the CLI docs: sets ANTHROPIC_AUTH_TOKEN, ANTHROPIC_BASE_URL, and ANTHROPIC_API_KEY for the session, pulls the model if you don’t have it, installs Claude Code if you don’t have that, and drops you into a normal Claude Code session pointed at localhost:11434 instead of Anthropic.

To pick the model explicitly:

ollama launch claude --model qwen3-coder

You should see the standard Claude Code banner, with the model line showing your Ollama model instead of a Claude model. Confirm nothing is leaking to the cloud by checking what’s loaded:

ollama ps

A row with your model name and a PROCESSOR column reading 100% GPU is the goal state. If it says CPU or a split, your VRAM is too small for the model plus its context — see the trap section below.

Two more flags worth knowing: --config writes the configuration without starting the tool (useful for provisioning), and Ollama’s docs show a headless pattern for scripts and CI — ollama launch claude --model glm-4.7-flash --yes -- -p "explain this codebase" — that passes flags through to Claude Code’s non-interactive mode.

Path 2: manual environment variables

The launch command owns the session it starts. If you want the wiring permanent — dotfiles, containers, a remote Ollama box on your LAN — set the variables yourself. This is the officially documented configuration:

export ANTHROPIC_AUTH_TOKEN=ollama     # required by the client, ignored by Ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434

claude --model qwen3-coder

Swap localhost for another machine’s address if your GPU lives elsewhere. That’s the whole trick: Claude Code doesn’t have a “local mode,” it just has a base URL, and Ollama now answers on the same protocol.

One subtlety from Anthropic’s model-config docs that most guides skip: ANTHROPIC_BASE_URL changes where requests go, not which model answers them. Claude Code resolves several model roles independently — the main model, the haiku alias used for background functionality, and the subagent model. Which brings us to the second trap.

The two traps that break this setup

Trap 1: the context window silently strangles the agent

Ollama’s default context window does not fit an agentic coding tool. The FAQ still documents 4,096 tokens as the baseline, and recent releases scale the default with VRAM — under 24 GiB you’re still at 4K. Claude Code’s system prompt plus tool schemas plus one source file blows through 4K before the model sees your actual request, and Ollama truncates from the top silently. The symptoms are distinctive: the agent “forgets” instructions mid-session, re-reads the same file in a loop, or edits confidently against code it can no longer see.

Ollama’s own integration guide recommends a 64K+ context window for larger repositories. Set it server-wide:

OLLAMA_CONTEXT_LENGTH=65536 ollama serve

or set it in the Ollama app’s settings. Mind the VRAM math: the KV cache grows with the window and comes out of the same budget as the model weights. A 14B model at Q4 with a 32K window is realistic on a 16 GB card; 64K wants 24 GB. We covered the failure mode — and the per-tool fixes — in depth in the num_ctx deep dive, and everything there applies to Claude Code unchanged.

Trap 2: the background model still points at Anthropic

This is the one that bites manual setups, usually in the first session: the main model responds fine, then a background operation errors out complaining about authentication. Claude Code uses a small, fast model for background functionality — that’s the haiku alias — and it resolves independently of --model. With ANTHROPIC_API_KEY empty, those calls have nowhere to go.

The fix, straight from the model configuration reference: point the haiku alias at a local model too.

export ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3-coder
export CLAUDE_CODE_SUBAGENT_MODEL=qwen3-coder

(ANTHROPIC_SMALL_FAST_MODEL, which older guides reference, is deprecated in favor of ANTHROPIC_DEFAULT_HAIKU_MODEL.) The ollama launch path handles this class of wiring for you, which is 80% of the reason to prefer it.

Which model to run

The official docs use qwen3.5 and qwen3-coder in their examples, and the compatibility layer means anything with solid tool calling works. What separates the models here isn’t chat quality — it’s whether they can sustain Claude Code’s tool-call loop for dozens of turns without emitting a malformed call. Guidance by hardware tier:

HardwareModel to start withWhat you get
12 GB VRAM (RTX 3060 class)glm-4.7-flash or qwen3-coder small quant, 16K–32K contextChat, single-file edits; agent runs work but degrade on long threads
16–24 GB VRAM (RTX 3090 class)qwen3-coder at Q4, 32K–64K contextThe realistic sweet spot: multi-file edits, subagents, usable thinking
32 GB+ Apple unified memoryqwen3-coder, 32K contextSame tier as above at lower wattage; prompt processing is slower on long contexts
Anything, with an Ollama accountqwen3.5:cloudOllama-hosted variant — no download, no VRAM limit, but you’ve reintroduced a cloud

Third-party testing agrees with the low end: Morph’s setup guide singles out glm-4.7-flash as the lowest-hardware-requirement option with the most consistent tool calling. If you’re below 12 GB of VRAM, be honest with yourself: quantized-to-death models drop tool-call syntax constantly, and Claude Code amplifies every model weakness because it chains calls. A used RTX 3090 remains the price-performance move for this workload in 2026 — our sister site breaks down the tiers in the local-AI GPU buying guide. No GPU at all? A rented one on RunPod running Ollama gives you the same wiring with the privacy dial set to “trusted cloud” instead of “none.”

What you give up

Running Anthropic’s agent without Anthropic’s models has real costs, and the compatibility docs are upfront about the protocol-level ones:

  • No prompt caching. Anthropic’s API caches the system prompt and conversation prefix; Ollama’s compatibility layer doesn’t support it. Long sessions reprocess context on every turn, which is why local Claude Code feels slower as threads grow. (Cache-friendly pricing is also a big part of why the cloud version is cheaper than people assume.)
  • tool_choice and metadata fields are unsupported, token counts are approximations, and there’s no token-counting endpoint, batching, citations, or PDF ingestion. URL-based images don’t work — base64 only.
  • Model quality is the ceiling. A 14B–30B local model will handle “rename this API across the codebase” and explain-this-file work fine. It will not match Sonnet 5 or Fable 5 on a gnarly cross-cutting refactor, and no amount of configuration changes that. Treat local Claude Code the way we treated local Zed and local Aider: superb for the private, repetitive, high-volume work; wrong tool for the hardest 10%.

What you don’t give up is the harness. CLAUDE.md conventions, slash commands, subagents, hooks — the workflow layer we mapped in the power-user setup guide — all of it runs identically, because none of it lives on Anthropic’s side of the wire.

FAQ

Do I need a Claude subscription or API key for this?

No. ANTHROPIC_AUTH_TOKEN=ollama satisfies the client, and Ollama ignores the value. The whole point is that no Anthropic account is involved — which also means no Anthropic bill and no rate limits beyond your GPU.

Which Ollama version do I need?

v0.14.0 for the Anthropic-compatible API (manual path), v0.15 for ollama launch. As of this writing the current release is v0.32.1 (July 16, 2026), and the June–July releases specifically improved the Claude Code path — auto-install and telemetry-off-by-default among them. Update before you file bugs.

Can I switch between local and real Claude in the same day?

Yes — the wiring is session-scoped. ollama launch claude only configures the session it starts; run plain claude in another terminal and you’re back on Anthropic’s API with your normal account. If you exported the variables manually, unset them (or scope them per-project with direnv).

Is this actually private?

The inference is. Prompts, code, and completions stay between Claude Code and your Ollama server. Claude Code’s own telemetry is a separate channel — which is why it matters that ollama launch now disables it by default (v0.31.2). If you’re wiring things manually and telemetry-zero matters to you, that’s your checklist item, not the model routing.

Why does the agent keep re-reading files and forgetting instructions?

Context truncation, near-certainly. You’re on the default window and Ollama is silently dropping the top of the conversation. Set OLLAMA_CONTEXT_LENGTH=65536 (or as high as your VRAM allows) and the behavior disappears — the full diagnosis is here.

  • RTX 3090 (24 GB) — the used-market sweet spot: runs qwen3-coder at Q4 with the 64K context this workload actually wants

Sources

Last verified July 20, 2026. Ollama’s release cadence is weekly and Claude Code’s model-resolution rules change between versions; re-check the official docs above before wiring anything into CI.

Was this article helpful?