TL;DR: You don’t need Ollama or LM Studio to run Cline, Continue.dev, or Aider on a local model — llama-server from llama.cpp serves an OpenAI-compatible API directly. Two flags make or break it: --jinja (without it, agent tool calls fail) and an explicit --ctx-size (the 4,096-token default kills agent loops). Setup takes about ten minutes.

What you’ll be able to do after this guide:

• Serve a tool-calling coding model on localhost:8080 with one llama-server command that pulls the GGUF straight from Hugging Face — no Ollama, no GUI
• Wire that endpoint into Cline (OpenAI Compatible provider), Continue.dev (native llama.cpp provider), and Aider (openai/ prefix) with configs that survive agent mode
• Diagnose the three failures every first-time llama-server user hits: missing --jinja, the silent 4K context default, and Aider’s “Unknown context window” warning

Honest take: If Ollama already works for you, keep it — llama.cpp is the engine underneath Ollama anyway. Switch when you need what the wrapper hides: exact context sizing, KV-cache quantization, parallel slots, and same-day builds. For agent workloads on a 24 GB GPU, that control is worth the extra command-line friction.

---

Why run llama.cpp directly when Ollama exists

Every local-backend guide on this site so far assumes Ollama or LM Studio. Both are wrappers: Ollama embeds llama.cpp as its inference engine, and LM Studio ships llama.cpp (and MLX) under a GUI. Running llama-server directly removes the wrapper, and with it three recurring failure classes we’ve covered in separate troubleshooters:

1. The hidden context default. Ollama’s 4K num_ctx default has broken more Cline sessions than any other single setting — enough that we wrote a dedicated context-length fix guide. llama-server has the same 4,096-token default, but you set the real value once, in the launch command, where you can see it.
2. Capability detection you don’t control. Ollama decides whether a model “supports tools” by scanning its chat template, and rejects the request with a 400 if it disagrees — the error we dissected in the Ollama “does not support tools” fix. llama-server hands you the template engine directly via --jinja and lets you override the template file if a GGUF ships a broken one.
3. Release lag. llama.cpp cuts numbered builds continuously — as of this writing the current tag is b10506, built August 20, 2026, with the build before it adding a server model-caching option the same day. New architectures land here first; wrappers inherit them weeks later.

There’s also a genuinely new reason in 2026: router mode. Launch llama-server without -m and it loads models dynamically from --models-dir, routing each request by the model field in the JSON — one port serving your 30B coder for agent work and a small model for quick questions. That used to be the single biggest convenience argument for Ollama.

The trade: no model library UI, no automatic VRAM fitting, and you manage GGUF files yourself. If that sounds like a downside rather than a feature, stay on Cline + LM Studio — no judgment.

Step 1 — Install llama.cpp

Package managers carry current builds on macOS, Windows, and Linux:

# macOS
brew install llama.cpp

# Windows
winget install llama.cpp

# Linux (or any platform) — prebuilt binaries per backend:
# github.com/ggml-org/llama.cpp/releases  (CUDA, Vulkan, CPU, and more)

The release page ships separate archives per acceleration backend. For NVIDIA cards grab the CUDA build; for AMD on Windows, Vulkan is the low-friction path. Verify the install:

llama-server --version

Step 2 — Pick a model that can actually drive an agent

Cline and Continue’s agent mode live and die on tool calling (function calling). A model that chats fine but flubs tool-call JSON will loop or stall — the same failure mode we documented for Ollama in the Cline tool-use loop fix. Three GGUF picks that we verified are current and tool-call-capable, by VRAM tier:

VRAM	Model (-hf value)	Download size	Why
8 GB	bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M	~4.7 GB	The reference model in llama.cpp’s own function-calling docs; fine for chat and single-file edits, marginal for agent loops
16 GB (or 6 GB VRAM + 16 GB RAM with CPU offload)	ggml-org/gpt-oss-20b-GGUF	12.1 GB (MXFP4)	Official ggml-org conversion; strong tool-calling; runs surprisingly well partially offloaded
24 GB (RTX 3090/4090)	unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M	18.6 GB	The default local agent-coding pick; MoE with 3B active parameters, so it generates fast for its size

The -hf flag downloads from Hugging Face on first launch and caches locally — no separate download step. For a deeper dive on what fits in which GPU, runaihome.com’s best local AI models by VRAM tier covers the hardware side, and aifoss.dev tracks the open-source model landscape.

Step 3 — Launch llama-server with the flags that matter

Here is the command for the 24 GB tier, annotated:

llama-server \
  -hf unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M \
  --jinja \
  -ngl 99 \
  -c 65536 \
  --host 127.0.0.1 --port 8080

• --jinja is not optional for agent tools. llama.cpp’s function-calling docs are explicit: OpenAI-style tool calling is enabled by the --jinja flag, which activates the model’s chat template. Launch without it and Cline’s tool calls come back as plain text the extension can’t parse.
• -c 65536 sets context explicitly. The default is 4,096 tokens — Cline’s system prompt alone overflows that. Don’t rely on -c 0 (“read from model”) either: there’s an open llama.cpp issue (#18376) where -c 0 falls back to 4,096 instead of the model’s trained maximum. State the number you want.
• -ngl 99 offloads all layers to GPU. If you’re on the 16 GB-RAM offload path for gpt-oss-20b, lower it until the load fits.
• --host 127.0.0.1 keeps the server loopback-only. Add --api-key yoursecret if you bind 0.0.0.0 for LAN access.

Confirm the OpenAI-compatible surface is live:

curl http://localhost:8080/v1/models

Expected output (trimmed):

{"object":"list","data":[{"id":"unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M","object":"model", ...}]}

Copy that id value — two of the three tools below want it verbatim. The server also exposes /v1/chat/completions, /v1/completions, and /v1/embeddings, plus a built-in web UI at http://localhost:8080 (enabled by default) that’s handy for a quick smoke test before blaming your editor config.

One caveat from the official docs worth repeating: aggressive KV-cache quantization like -ctk q4_0 “can substantially degrade the model’s tool calling performance.” q8_0 for K and V is the safe ceiling if you’re squeezing a bigger context into VRAM; going lower trades exactly the capability agents depend on.

Step 4 — Per-tool configuration

Cline: OpenAI Compatible provider

Cline’s official docs cover Ollama, LM Studio, and Atomic Chat as local runtimes — llama.cpp isn’t listed, which is why searching “cline llama.cpp” gets you forum fragments. The supported route is the OpenAI Compatible provider:

1. Cline settings → API Provider → OpenAI Compatible
2. Base URL: http://localhost:8080/v1 — the /v1 suffix is required here
3. API Key: any non-empty string (local works) unless you launched with --api-key
4. Model: the exact id from your /v1/models output
5. Under Model Configuration, set the Context Window size to match your -c value — Cline uses this to decide when to condense the conversation, and the default assumption won’t match your server

Two more settings earn their keep on local models: enable Use Compact Prompt (Settings → Features), which Cline’s local-models guide recommends to cut system-prompt overhead, and keep tasks short — smaller context means faster prefill on consumer GPUs.

Continue.dev: native llama.cpp provider

Continue is the only one of the three with a first-class llama.cpp provider. In config.yaml:

models:
  - name: qwen3-coder-local
    provider: llama.cpp
    model: qwen3-coder-30b
    apiBase: http://localhost:8080
    capabilities:
      - tool_use

Note the apiBase has no /v1 suffix for this provider — Continue appends paths itself. Getting this backwards produces the same connection-refused symptoms we cataloged in the LM Studio connection error guide, so it’s the first thing to check.

The capabilities block matters for one specific reason: Continue gates Agent mode on the tool_use capability, which it autodetects from provider and model name. A custom llama-server model name may not match anything Continue recognizes, leaving Agent mode grayed out with “requires tools.” Capabilities in YAML are additive — per Continue’s docs (v1.3.40, current as of today) you can’t remove autodetected capabilities, only add missing ones. Adding tool_use unlocks Agent mode; whether the model honors it is Step 2’s problem. Chat and autocomplete config work the same as with Ollama.

Aider: the openai/ prefix

Aider treats any OpenAI-compatible endpoint as generic OpenAI:

export OPENAI_API_BASE=http://localhost:8080/v1
export OPENAI_API_KEY=local

aider --model openai/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M

The double-prefix looks odd (openai/ + the full HF id) but the rule is mechanical: whatever your server reports in /v1/models, prefixed with openai/. On first run Aider prints:

Model openai/...: Unknown context window size and costs, using sane defaults.

That’s Aider telling you its model database has no entry for your local model — the exact warning string is in its docs. It still works, but Aider’s repo-map sizing improves if you register the real context window via a .aider.model.metadata.json in your project (see Aider’s advanced model settings docs). Since Aider edits via search/replace blocks rather than tool calls, it’s the most forgiving of the three on smaller models — the Aider + Ollama guide has the model-quality discussion, and it transfers unchanged.

The three failures you’ll hit, in order

Tool calls arrive as chat text. Cline shows the model narrating <tool_call> JSON instead of executing it, or errors out parsing a response. Cause: launched without --jinja. Fix: relaunch with it, and check the server log line reading Chat format: — if it says Generic instead of a model-specific format, llama.cpp didn’t recognize the template and fell back to a mode its docs warn “may consume more tokens and be less efficient.” The unsloth Qwen3-Coder GGUFs ship corrected templates; if you grabbed a random quant, --chat-template-file overrides it.

The agent forgets what it was doing. Multi-file task starts strong, then the model re-reads files it already saw or abandons the plan. That’s context overflow at the 4,096 default — llama-server truncates or errors depending on version. Fix: explicit -c, and set Cline’s Context Window to the same number. Rough VRAM math for the KV cache: Qwen3-Coder-30B at 64K context adds several GB on top of the 18.6 GB weights, which is why the 24 GB tier is the practical floor for agent work on this model.

Aider “sane defaults” surprises. The unknown-model warning above isn’t fatal, but the default context assumption caps how much repo-map Aider sends. Register the metadata or pass --map-tokens deliberately.

llama.cpp vs Ollama vs LM Studio as an agent backend

	llama-server (b10506)	Ollama	LM Studio
Setup friction	CLI flags, manual GGUF choice	One pull command	GUI download
Context default	4,096 — set -c yourself, visibly	4,096 via hidden num_ctx	Slider in GUI
Tool calling	--jinja, template overridable	Template-scan gate, can 400	Toggle per model
Multi-model serving	Router mode (--models-dir)	Built-in	One model per server by default
KV-cache quantization	-ctk/-ctv, full control	Limited exposure	Limited exposure
New model architectures	Same-day builds	After integration	After integration
Best for	Control, agents, tinkerers	Zero-fuss default	Windows + GUI preference

When not to bother

Local models still lose to hosted frontier models on long agentic chains — that verdict from our Cursor + Ollama guide hasn’t changed. And if your machine tops out at 8 GB VRAM, a rented GPU running the exact same llama-server command is the honest middle path: a RunPod instance with a 24 GB card runs Qwen3-Coder-30B with the config above unmodified — point your apiBase at the pod URL and add --api-key.

FAQ

Do I need /v1 in the base URL or not? Cline and Aider: yes (http://localhost:8080/v1). Continue’s native llama.cpp provider: no (http://localhost:8080). This single inconsistency causes most “it connects in one tool but not the other” reports.

Can one llama-server instance serve Cline and Continue at the same time? Yes. The server handles concurrent requests through slots (--parallel, default auto). On a single GPU they interleave rather than truly parallelize, so expect both to slow down under simultaneous load.

Does Cline’s model field have to match exactly? Use the id string from /v1/models. A single-model server generally accepts any name, but router mode routes by that field — exact matching is the habit that never breaks.

Is llama.cpp faster than Ollama on the same model? Same engine, so raw inference is comparable at identical settings. Real-world gaps come from configuration: explicit -ngl, batch sizes, flash attention (-fa), and KV quantization that Ollama doesn’t expose. You tune it, you win; you don’t, it’s a wash.

Where do the downloaded models live? -hf downloads cache under your user cache directory (~/.cache/llama.cpp on Linux/macOS). Point -m at any GGUF path if you prefer managing files yourself.

Sources

• llama-server README — llama.cpp official repo
• Function calling documentation — llama.cpp official repo
• Install guide — llama.cpp official repo
• llama.cpp releases (b10506, Aug 20 2026)
• Context size flag behavior — llama.cpp issue #18376
• Qwen3-Coder-30B-A3B-Instruct GGUF — unsloth on Hugging Face
• Running gpt-oss with llama.cpp — official ggml-org guide
• Cline OpenAI Compatible provider docs
• Cline running models locally — official docs
• Continue.dev llama.cpp provider docs
• Continue.dev model capabilities (tool_use) docs
• Aider OpenAI-compatible API docs

Last updated August 20, 2026. Pricing and features change frequently; verify current state before purchasing.