TL;DR: llama.cpp v0.7.0 (tagged October 15, 2026) changes llama-server’s default port from 8080 to 9931 — which silently breaks every Cline, Cursor, Continue.dev, and Aider config pointing at the old port — and adds --moe-cache-mib, a GPU cache for offloaded MoE experts that the project measured at up to 2.2x faster decode. Update if you run an MoE model bigger than your VRAM; everyone else mostly needs the one-line port fix.

What you’ll be able to do after this guide:

• Update llama.cpp to v0.7.0 and repair the broken port in your Cline, Cursor, Continue.dev, or Aider config in under five minutes — or pin the old port and change nothing else
• Size --moe-cache-mib correctly for your GPU using the ~10%-of-expert-weights rule from the actual pull request, and know the realistic speedup for your hardware (1.5–2.2x, decode only)
• Decide whether LiliCorr speculative decoding is worth setting up for coding work today (short answer: not yet, and the flag most write-ups name doesn’t exist)

Honest take: If your coding model fits entirely in VRAM — like the Qwen3-Coder-30B Q4_K_M most 24 GB owners run — v0.7.0 gives you nothing but a moved port and some new model support. The MoE cache is a genuinely big deal for the other group: people running 60–100 GB MoE models with experts offloaded to system RAM. For them, this is the largest free decode speedup llama.cpp has shipped this year, and it’s worth updating today.

---

What actually changed in v0.7.0

llama.cpp has shipped continuous bNNNN nightly builds for years — our August llama-server setup guide cited b10506. v0.7.0 is one of the project’s periodic tagged releases that bundles everything since the last tag (the nightly at tag time was b11669; nightlies have continued past b11880 since). The October 15 release notes list four changes that matter for coding-agent use:

Change	What it does	Who it affects
Default port 8080 → 9931	llama-server now listens on 9931 unless you pass --port	Everyone with an existing editor config
--moe-cache-mib N	LRU cache on the GPU for MoE experts stored in host RAM; up to ~2.2x decode	MoE models bigger than VRAM (--cpu-moe / -ncmoe setups)
LiliCorr speculative decoding	Higher-acceptance candidate selection on top of DFlash drafting	Nemotron-family users with a converted draft model
New architectures	K2 Horizon, Prism Bonsai 2 27B (ternary), EmbeddingGemma-2, Nemotron Bolt 74B A9B, maion-coder, several decision models	Anyone waiting on these models

There’s also a --spec-draft-n-ubatch flag that sizes the draft model’s batch separately to save VRAM in speculative setups, and the bundled web UI gained a Models Manager (aifoss.dev tracks the broader open-source release cadence if you want the full changelog treatment). Docker users: the official images still expose 8080 inside the container, so published port mappings keep working there.

Update commands are the same as installation:

# macOS
brew upgrade llama.cpp

# Windows
winget upgrade llama.cpp

# Linux / manual: grab the new archive per backend
# github.com/ggml-org/llama.cpp/releases/tag/v0.7.0

llama-server --version   # confirm it reports v0.7.0 or a b11xxx build

Why did Cline or Cursor stop connecting after the update?

Because your editor is still calling port 8080 and llama-server is now on 9931. The symptom is identical in every tool: the server starts fine, the model loads, and the editor reports a connection error — Cline shows a fetch failure on http://localhost:8080/v1, Cursor’s chat panel times out, Aider prints a connection refused. Nothing is broken except the port number, which makes this the kind of failure that eats an evening.

Confirm what port the server actually bound — v0.7.0 prints it at startup, or just probe both:

curl -s http://localhost:9931/v1/models

Expected output on a v0.7.0 default launch (trimmed):

{"object":"list","data":[{"id":"unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M","object":"model"}]}

If 9931 answers and 8080 doesn’t, you have two clean fixes. Pick one, not both.

Fix 1 — pin the old port (recommended). Add --port 8080 to your launch command and every existing config keeps working. Our previous guides already included an explicit --host 127.0.0.1 --port 8080; if you copied those commands, the update changes nothing for you. An explicit port also protects you from any future default change.

Fix 2 — adopt 9931 and update each tool once:

• Cline: Settings → API Provider → OpenAI Compatible → Base URL http://localhost:9931/v1
• Continue.dev: in config.yaml, change the llama.cpp provider’s apiBase to http://localhost:9931/v1
• Aider: update the base URL env var — export OPENAI_API_BASE=http://localhost:9931/v1
• Cursor: Settings → Models → Override OpenAI Base URL → http://localhost:9931/v1 (this routes Chat and Cmd+K only; Tab autocomplete stays on Cursor’s cloud regardless, as covered in the Cursor local-model setup guide)

Either fix takes two minutes. The trap is only for people who never set --port explicitly and have configs scattered across three editors.

How much faster is --moe-cache-mib, really?

Up to 2.2x on decode, with 1.5–1.8x being the more typical measured result — and zero gain on prompt processing. The flag creates a least-recently-used cache on the GPU for expert weights that live in host RAM. When a token’s routed experts are already cached, the matmul runs on the GPU without re-uploading them; only misses cross the PCIe bus. The cache path only engages for batches of 32 tokens or fewer, which is why it accelerates token generation but not prompt ingestion.

The numbers from the pull request itself (Qwen3.8-Flash-Next Q4_0 — a 93.7 GiB model with 65.4 GiB of expert weights — on an EPYC 7742 host, PCIe 4.0 x16):

GPU	Config	Decode t/s	Speedup	Cache hit rate
RTX 4090	--fit (no cache, v0.6 behavior)	25.0	1.0x	—
RTX 4090	--fit --moe-cache-mib 6544	39.4	1.57x	72%
RTX 4090	-cmoe --moe-cache-mib 11000	40.7	1.62x	77%
RTX 5090	--fit (no cache)	30.8	1.0x	—
RTX 5090	--fit --moe-cache-mib 6544	54.5	1.77x	77%
RTX 5090	-cmoe --moe-cache-mib 19000	67.8	2.20x	89%

Three practical rules fall out of the PR discussion:

1. Size the cache at roughly 10% of total expert weights. The author found that ratio worked well in his experiments — for the 65 GiB-expert model above, that’s the ~6,544 MiB figure in the table. More cache raises the hit rate further if you have spare VRAM.
2. It stacks with --fit, but the cache counts against VRAM, so fewer whole layers stay resident. The table shows both configurations beating the uncached baseline anyway.
3. PCIe bandwidth and backend matter. The benchmark machine had PCIe 4.0 x16. Testers in the thread reported smaller gains on AMD HIP, slowdowns in some Vulkan and ROCm configurations, and one Vulkan load crash. On NVIDIA CUDA with a full-width slot, the numbers above are representative; on anything else, benchmark before and after with your own workload.

For agentic coding specifically, there’s a caveat the headline number hides: agent sessions are prompt-heavy. Cline and Aider re-send large system prompts and file context on every turn, and prompt processing gets no speedup from this cache. Your perceived gain in an agent loop will be smaller than 2.2x — the cache pays off most in the long generation phases, like when the model writes a 200-line diff.

Does your setup actually benefit? The VRAM math

The cache only does something when expert weights live in host RAM. Run the check against your own model:

• Model fits entirely in VRAM → no benefit. The standard 24 GB coding pick, Qwen3-Coder-30B-A3B Q4_K_M at 18.6 GB, loads fully onto an RTX 3090 or 4090. No experts are offloaded, so there’s nothing to cache. Update for the port hygiene and move on.
• MoE model bigger than VRAM → this is for you. gpt-oss-120b (63 GB total), Qwen3.8-Flash-Next (93.7 GiB), and similar large MoE models on a 24–32 GB card run with --cpu-moe or -ncmoe, exactly the setup the cache targets. A 40–68 t/s decode on a model that physically cannot fit in your VRAM is workstation-class behavior from a consumer card.
• Dense model with CPU offload → no benefit. The cache is expert-granular. A dense 70B spilling layers to RAM gains nothing here.

The cost angle is why this release matters more in late 2026 than it would have in 2024. The hardware ladder for running large MoE models used to be “buy more VRAM,” and that ladder has gotten brutal: a used RTX 4090 runs $2,575–$3,010 and an RTX 5090 $3,822+ (street prices we track; MSRPs are fiction this year). Meanwhile the RAM side took its own hit — DDR5 64 GB kits are $680–$1,070 because HBM demand ate the fab capacity. --moe-cache-mib is the rare change that extracts a 1.5–2.2x improvement from hardware you already own, no purchase required. If you’re deciding between a RAM upgrade and a GPU upgrade for big-MoE coding work, the math shifted this month: 64 GB of DDR5 plus this cache now delivers usable decode speeds that previously argued for a second GPU. The full hardware-side analysis lives on our sister site — see runaihome.com’s VRAM-tier model guide — and how much RAM local coding setups actually need covers the system-memory side for Cursor users.

What about LiliCorr speculative decoding — should you set it up?

Not yet, for coding work — and ignore any guide telling you to pass --spec-type draft-lilicorr, because that value doesn’t exist. We checked the merged PR: LiliCorr loads through the existing DFlash path. The real invocation is:

llama-server -m target.gguf -md lilicorr-draft.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7

LiliCorr (from an NVIDIA paper) adds a candidate-selection step on top of DFlash drafting: a small transformer computes embedding vectors for each position’s top-k candidate tokens and picks the chain with the highest cosine similarity between consecutive candidates, raising the draft acceptance ratio at some extra compute cost. The catch for practical use: it requires a specially converted draft checkpoint (the PR’s converter targets the Nemotron family — a Nemotron 3.5 Bolt LiliCorr GGUF is referenced), and the PR publishes no end-to-end speedup or acceptance-length numbers — only parity checks against the vLLM implementation. Until converted drafts exist for the models people actually code with and someone publishes tokens-per-second comparisons, this is a watch-item, not a setup task. We’ll revisit when a Qwen-family draft lands.

The updated launch command for coding agents

Everything from the August guide still applies — --jinja remains mandatory for tool calling, and the 4,096-token context default still kills agent loops. Here is the v0.7.0-era command for a big-MoE setup on a 24 GB card with 64 GB+ system RAM:

llama-server \
  -hf ggml-org/gpt-oss-120b-GGUF \
  --jinja \
  -c 65536 \
  --cpu-moe \
  --moe-cache-mib 6000 \
  --host 127.0.0.1 --port 8080

For the fits-in-VRAM tier (Qwen3-Coder-30B Q4_K_M), keep your existing command and just confirm the port — --moe-cache-mib is harmless but useless there. Either way, pin --port explicitly from now on.

FAQ

Is v0.7.0 safe to update to mid-project? The port change is the only thing that breaks existing coding-agent setups, and --port 8080 neutralizes it. The server API surface (/v1/chat/completions, /v1/models) is unchanged. Vulkan and ROCm users experimenting with --moe-cache-mib should benchmark first — some configurations in the PR thread regressed.

Does --moe-cache-mib help Ollama or LM Studio users? Not yet. Both wrap llama.cpp and inherit features on their own release schedules, typically weeks later. Running llama-server directly is how you get this today — which is the core argument of our standalone llama-server guide.

What’s a reasonable --moe-cache-mib value for a 24 GB card? Start at ~10% of the model’s expert weights (for gpt-oss-120b, roughly 5,000–6,000 MiB), then raise it while VRAM headroom and the reported hit rate allow. The server logs cache hit rates; past ~85–90% you’re near the ceiling.

Does the cache speed up prompt processing? No. It only engages for batches of 32 tokens or fewer, so prompt ingestion — the slow part of agent turns with big contexts — is unchanged. Expect the gain during generation, not while the model reads your codebase.

Why 9931? The project moved off 8080 because it collides with half the web tooling on a developer machine (proxies, dashboards, dev servers). 9931 is unclaimed by anything common. Annoying for one week of config fixes, defensible long-term.

Sources

• llama.cpp v0.7.0 release notes — Official GitHub release
• PR #29887: GPU cache for MoE experts kept in host memory — benchmark tables and sizing guidance
• PR #30506: LiliCorr speculative decoding — flags and conversion requirements
• PR #30159: llama-server default port change 8080 → 9931
• llama.cpp function-calling docs — the —jinja requirement
• byteiota: llama.cpp 0.7.0 MoE cache coverage — independent write-up

Last updated October 26, 2026. llama.cpp ships nightly builds; flags and defaults verified against the v0.7.0 tag and merged PRs on this date.

Know which coding tool is worth paying for

Hands-on comparisons of AI coding assistants and what each one costs to run — including the local-model path. Sent only when something changes. Unsubscribe anytime.

Email address Subscribe