TL;DR: KoboldCpp is a single-file llama.cpp wrapper — no install, no daemon, AGPL-3.0 — that speaks four API dialects from one port: its own, OpenAI, Ollama, and Anthropic. That makes it the most impersonation-capable backend in this series, and its 12,288-token default context is the least-broken default we’ve tested. Still raise it to 32K for agents.

What you’ll be able to do after this guide:

• Launch one downloaded binary and get an OpenAI-compatible server on http://localhost:5001/v1 with GPU offload figured out for you — --gpulayers defaults to autofit, the knob llama-server makes you hand-tune
• Run KoboldCpp on port 11434 so tools that only ship an Ollama provider talk to it without knowing the difference, and decide when the native OpenAI route is the better wire
• Wire it into Cline, Continue.dev, and Aider with the series-standard three-line configs, get tool calling working with --jinja --jinjatools, and close the security hole KoboldCpp ships open: by default it listens on every network interface your machine has

Honest take: KoboldCpp is the Swiss-army pick — the only backend here that can stand in for Ollama, OpenAI, and Anthropic servers at once, on hardware as odd as Vulkan-only GPUs. If you just want a coding backend and nothing else, Ollama is still less to think about. Pick KoboldCpp when the impersonation tricks or the exotic-hardware support actually solve a problem you have.

---

The zero-install seat in the local-server lineup

Seven engines into this series — Ollama, LM Studio, llama-server, vLLM, Docker Model Runner, and Jan — every entrant has demanded either an installer, a container runtime, or a Python environment. KoboldCpp demands a download. It is one executable (LostRuins/koboldcpp, 11.5k GitHub stars, AGPL-3.0) that bundles llama.cpp, a web UI, and a four-dialect API server; you run it, point it at a GGUF file, and you have a model server. Current release: v1.119, shipped August 16, 2026.

The project grew up in the AI-roleplay community — the “Kobold” in the name is KoboldAI, and the bundled Lite UI is built for storytelling — which is probably why it never shows up in coding-tool documentation. That’s an oversight. Under the hobbyist paint is the same llama.cpp engine as half this series, wrapped with three genuinely useful decisions: GPU offload is automatic (--gpulayers defaults to -1, autofit — the exact knob llama-server makes you hand-tune), the default context is 12,288 tokens rather than the 4,096 that Ollama, llama-server, and Docker Model Runner still ship, and the API layer answers in whatever dialect your client speaks: KoboldAI’s own at /api, OpenAI at /v1, Ollama at /api/chat and /api/generate, and Anthropic at /v1/messages (added in v1.115.2, June 2026, tool calling included).

Hardware reach is the other quiet strength. CUDA via --usecuda, Vulkan via --usevulkan — which covers Intel Arc and older AMD cards that Ollama’s ROCm builds ignore — Metal on Apple silicon, a community ROCm fork, and pure CPU if that’s what you have. If your GPU is the reason no other backend works, KoboldCpp is often the one that does. For which card is worth owning in the first place, runaihome.com’s local-models-by-VRAM guide is the companion read; a used RTX 3090 still runs everything in this article.

Step 0 — Launch and prove the endpoint

Download the binary for your platform from the releases page — koboldcpp.exe on Windows, koboldcpp-linux-x64 on Linux, a Mac build for Apple silicon. Double-clicking opens a launcher GUI; for a coding backend you want the flags explicit, so use the terminal:

./koboldcpp-linux-x64 --model Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf \
  --contextsize 32768 --jinja --jinjatools --host 127.0.0.1
# → Starting Kobold API on port 5001 at http://localhost:5001/api/
# → Starting OpenAI Compatible API on port 5001 at http://localhost:5001/v1/

Every flag in that line earns its place below. First, confirm the endpoint the way your coding tools will see it:

curl http://localhost:5001/v1/models
# → {"object":"list","data":[{"id":"koboldcpp/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M", ...}]}

Note the id. KoboldCpp names the served model koboldcpp/ plus your GGUF filename minus the extension — verified in the current source, which sanitizes and prefixes the basename. That full string, prefix included, is the model ID Cline and Continue need. Same rule as every backend in this series: copy the ID from /v1/models, never type it from memory.

Why --host 127.0.0.1 is in the launch line is worth a paragraph of its own. KoboldCpp’s own --host help text says it plainly: if the flag is not set, all routable interfaces are accepted. Out of the box, your model server is reachable by every device on your network — a default inherited from the multiplayer and Horde-worker use cases, and the inverse of Jan and LM Studio, which bind localhost until you say otherwise. A coding backend has no reason to be on the LAN. Bind it to loopback, and if you ever do expose it, set --password (or the KCPP_PASSWORD env var) — and read the fine print: the password gates the text endpoints, while image endpoints stay open.

Step 1 — The context default: best in the series, still not enough

The numbers across this series tell the story in one row each:

Backend	Default context	Agent-ready?
Ollama	4,096	No — the num_ctx trap
llama-server	4,096	No
Docker Model Runner	4,096	No
Jan	8,192	No
KoboldCpp	12,288	Closer — still no
vLLM	model’s full window	Yes, if it fits

Three times Ollama’s default buys real slack — a small Cline task can finish inside 12K where a 4K backend has already looped. It does not buy safety. Cline’s system prompt plus tool schemas plus two open files still clears 12K early in a session, and KoboldCpp’s failure mode when that happens is quieter than most: Context Shifting, on by default, trims old tokens without reprocessing so generation keeps flowing. For a long roleplay chat, that’s the feature it was built to be. For an agent, “trim old tokens” means shedding the instructions and file contents the task depends on — the same slow-motion forgetting we dissected for Ollama, minus any error to catch. The server does print a once-per-session warning when a request’s context exceeds the allocation; watch the terminal, not the editor.

The fix is the flag already in our launch line: --contextsize 32768, the working floor this series recommends for agent use (the flag accepts up to 524,288 if your model and VRAM agree). KV-cache memory scales with it, so 32K on a 24 GB card with a 30B-class MoE is comfortable; 128K is not. Set the same number in each tool’s config below so client-side truncation math matches the server. And leave Context Shifting on once the window is honest — at 32K it’s a safety net, not a shredder. If you’d rather overflow loudly while debugging, --noshift turns it off.

Step 2 — Tool calling: two modes, one right answer for agents

KoboldCpp handles OpenAI-style tool calls in either of two ways, and the launch flags choose. Bare, with no flags, it uses its own universal fallback: tool definitions are compressed to JSON and injected into the prompt as an ### Available Tools: block, and the model’s reply is parsed for a call. The charm of this mode is that it works on models with no tool template at all. The cost is that it’s a prompt-engineering trick, and under Cline’s multi-turn pressure, trick-mode tool calls degrade into the narrating-JSON loop faster than native ones.

--jinja --jinjatools is the right answer for the models worth running as agents. --jinja applies the GGUF’s embedded chat template on the chat-completions endpoint; --jinjatools (it requires --jinja) extends that to tool calls, so Qwen3-Coder or gpt-oss sees tools in exactly the format it was trained on. On the response side, v1.116.1 (June 28, 2026) switched the default parser to llama.cpp’s jinja tool-call parser, and v1.119 added tool-calling templates for the newest arrivals — Muse Glimmer among them, if you’ve followed that backend’s quirks.

Model picks are the series standards, because the engine is the same llama.cpp underneath: gpt-oss-20b at 16 GB, Qwen3-Coder-Next as the efficiency pick, Qwen3-Coder-30B-A3B or Qwen3.8-27B at 24 GB, Q4_K_M to start. v1.119 ships full Qwen 3.8 support, so the newest of those runs day-one.

Step 3 — Wire in the three tools

Cline: OpenAI Compatible provider

No native KoboldCpp entry in Cline’s provider list; use OpenAI Compatible, per Cline’s provider docs:

• Base URL: http://127.0.0.1:5001/v1
• API Key: anything (e.g. kcpp) unless you set --password, in which case it’s that password
• Model ID: the exact koboldcpp/... string from /v1/models
• Model Configuration: context window = the --contextsize you launched with; enable tool/computer use

The classic first-run failure is pasting the model ID without its koboldcpp/ prefix. The 404 that produces looks like a connection error; it isn’t. curl the models endpoint and recheck the string — the triage rule from the LM Studio piece holds: if curl answers and the editor doesn’t, the bug is in the editor’s config.

Continue.dev: openai provider

models:
  - name: Qwen3-Coder (KoboldCpp)
    provider: openai
    model: koboldcpp/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M
    apiBase: http://127.0.0.1:5001/v1
    apiKey: kcpp
    roles: [chat, edit, apply]

The three config mistakes that plague every OpenAI-compatible backend apply unchanged: the provider is openai (not ollama — even though KoboldCpp could answer Ollama calls, this route is cleaner), apiBase stops at /v1, and the key field never contains the word Bearer.

Aider: OpenAI-compatible env vars

export OPENAI_API_BASE=http://127.0.0.1:5001/v1
export OPENAI_API_KEY=kcpp
aider --model openai/koboldcpp/Qwen3-Coder-30B-A3B-Instruct-Q4_K_M

The doubled-up path after openai/ looks wrong and is correct: the prefix routes Aider’s LiteLLM layer to the generic OpenAI-compatible handler, and everything after it must match the served ID literally — prefix and all, per Aider’s OpenAI-compatible docs. Silence the unknown-model warnings with a .aider.model.metadata.json declaring your context size and zero costs. Aider’s search/replace-block editing makes it the most tolerant of the three if you test a smaller model; when the blocks themselves stop matching, that’s the model failing, not the server.

The port 11434 trick — and the Anthropic bonus

Here is the capability no other backend in this series has. Since v1.117.1 (July 10, 2026), KoboldCpp answers Ollama’s API — /api/chat, /api/generate, /api/tags, /api/version, embeddings included — on the same port as everything else. The server even coaches you at startup: launch on any port other than 11434 and it prints “Note: For third party Ollama API Emulation, you should set the port to 11434.” So:

./koboldcpp-linux-x64 --model your-model.gguf --contextsize 32768 \
  --jinja --jinjatools --host 127.0.0.1 --port 11434
# → Ollama Emulation is now available at port 11434.

Now any tool whose local-model support begins and ends with “we support Ollama” — and there are many — talks to KoboldCpp believing it’s Ollama, on hardware Ollama might not even run on. The emulation honors num_ctx in request options, so Ollama-provider clients that send context hints still work. When a tool offers both an Ollama provider and an OpenAI-compatible one, prefer the OpenAI route on 5001 — it’s the dialect KoboldCpp’s tool-calling paths are built around — and keep 11434 for the tools that give you no choice.

The Anthropic endpoint is the same idea aimed the other way: /v1/messages accepts Anthropic-format requests, tool definitions included, and converts them internally. That’s the dialect Claude Code speaks, which makes KoboldCpp a candidate for the same env-var redirection we walked through in the Claude Code + LM Studio guide — point ANTHROPIC_BASE_URL at http://127.0.0.1:5001 and the request format matches. We haven’t put a full Claude Code session through it yet, so treat that pairing as plumbing-verified rather than battle-tested; the LM Studio route remains the one we’ve proven end to end.

KoboldCpp vs Ollama vs llama-server: the one-binary decision

	KoboldCpp	Ollama	llama-server
Best for	API impersonation, odd GPUs, zero install	Least-fuss daily driver	Minimal, scriptable, first-party
Install	1 file, no install	Installer + background service	Build or download, CLI
Default context	12,288	4,096	4,096
GPU offload	Autofit by default	Automatic	Manual -ngl
API dialects	KoboldAI, OpenAI, Ollama, Anthropic	Ollama, OpenAI	OpenAI
Tool calling	Universal fallback or --jinja --jinjatools	Native, per-model	--jinja
Default bind	All interfaces	Localhost	Localhost
License	AGPL-3.0	MIT (server)	MIT

The default-bind row is the one to respect. Everything else about KoboldCpp’s defaults is friendlier than the competition — better context, automatic offload, more dialects — and then it undoes the goodwill by listening on your whole network until told otherwise. One flag fixes it; just don’t skip the flag.

FAQ

Does KoboldCpp download models like Ollama does? No. You bring your own GGUF file — grab quantizations from Hugging Face. That’s one honest advantage Ollama and Jan keep: a built-in model library. KoboldCpp assumes you know what file you want.

Can I run two models at once for chat and autocomplete? Not from one instance — one process serves one text model. Run two instances on different ports (say 5001 and 5002) and give Continue.dev a different apiBase per role. Ollama and Jan handle multi-model swapping for you; it’s the price of KoboldCpp’s simplicity.

Is the AGPL-3.0 license a problem for work use? Running KoboldCpp as a local server and calling its API does not touch your code — AGPL obligations attach to modifying and distributing KoboldCpp itself. If your employer has a blanket no-AGPL-software policy, that’s a policy conversation, not a licensing trap; llama-server (MIT) is the closest substitute.

Why does my Cline request hang for 30+ seconds, then work? Prompt processing on first contact — the whole system prompt and file context has to be ingested before token one. Context Shifting and fast-forwarding make subsequent turns cheap. If every turn is slow, check the terminal: the layers may not have fit on the GPU, and autofit fell back to CPU for the remainder.

Should I use the Ollama emulation for Cline or Continue.dev? No — both have OpenAI-compatible providers, so use those against /v1 on port 5001. Reserve the port-11434 trick for tools that hardcode Ollama as their only local option.

Sources

• KoboldCpp — GitHub repository and README
• KoboldCpp releases — v1.119 (Aug 16, 2026), v1.117.1, v1.116.1, v1.115.2 release notes
• KoboldCpp wiki — flags, endpoints, context shifting
• koboldcpp.py source — verified defaults: contextsize 12288, port 5001, gpulayers -1 autofit, host all-interfaces, model-ID prefix, Ollama/Anthropic routes
• Cline docs — OpenAI Compatible provider
• Continue.dev docs — OpenAI-compatible model providers
• Aider docs — OpenAI-compatible APIs

Last verified August 25, 2026. KoboldCpp releases move fast; check the releases page for current flags and defaults before filing bugs.