TL;DR: mlx_lm.server is the OpenAI-compatible HTTP server inside Apple’s own MLX LM package — one pip install, one command, port 8080. On an Apple Silicon Mac it runs quantized models through Metal with unified memory, ships a real LRU key-value prompt cache that agent loops actually benefit from, and implements the OpenAI tools parameter with per-model-family parsers. The catch: it defaults to 512 output tokens, it’s Mac-only in practice, and the maintainers say plainly it’s not hardened for production.

What you’ll be able to do after this guide:

• Install mlx-lm 0.31.3, serve mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit, and sanity-check the endpoint with curl
• Wire the same server into Cline (OpenAI Compatible provider), Continue.dev (provider: openai + apiBase), and Aider (openai/ prefix + OPENAI_API_BASE)
• Dodge the three traps that break agent sessions: the 512-token default, the HF-cache model ID mismatch, and the localhost-only bind

Honest take: if you code on a Mac and your local backend is only ever going to serve you, MLX LM Server is the leanest path in this series — no Electron app, no GGUF conversion step, models pulled straight from the mlx-community Hugging Face org, and KV caching that makes turn ten of a Cline session feel like turn two. If you need Linux, NVIDIA throughput, or anything multi-user, this isn’t your tool: vLLM and SGLang exist for that. And if you want MLX’s speed with a GUI wrapped around it, LM Studio already ships an MLX engine — this guide is for people who’d rather run the stack those apps wrap.

---

The twelfth backend is the one Apple wrote

Eleven servers into this series — Ollama, LM Studio, llama-server, vLLM, SGLang, KoboldCpp, LocalAI, Jan, Docker Model Runner, Lemonade, and the GGUF ecosystem they mostly share — every entry has treated Apple Silicon as one hardware target among several. MLX LM inverts that. It’s a Python package from Apple’s machine learning research group, built on the MLX array framework, and its README states the scope in one line: “generating text and fine-tuning large language models on Apple silicon with MLX.”

That scope produces a different shape of tool. There’s no model-format conversion ritual — the mlx-community organization on Hugging Face hosts thousands of pre-quantized conversions, and mlx_lm.server pulls them by repo ID the way Ollama pulls from its registry. There’s no daemon, no menu-bar app, no bundled UI. You get a Python process speaking the OpenAI API on localhost:8080, and that’s the whole product.

Version check for this writing: mlx-lm 0.31.3, released April 22, 2026, current on PyPI as of September 1, 2026. The release cadence through early 2026 was roughly monthly (v0.30.6 February 4, v0.30.7 February 12, v0.31.0 March 7, v0.31.2 April 7), and the changelog is unusually agent-relevant: the Mistral tool-call parser landed in v0.30.7, Gemma 4 arrived with a dedicated tool-call parser in v0.31.2, and v0.31.3 fixed parallel tool-call handling. Someone on that team is clearly testing against agent workloads.

One wrinkle the PyPI metadata reveals: mlx-lm now publishes [cuda] and [cpu] extras, riding MLX’s experimental CUDA backend — the core mlx>=0.31.2 dependency is pinned to macOS (platform_system == "Darwin") only. Treat Linux/CUDA as a curiosity for now; every command below assumes a Mac, because that’s the platform the project is actually built for.

Step 0 — What Mac this needs

MLX runs on any Apple Silicon Mac, but the README flags one hard requirement worth checking before you blame the tool: large models need macOS 15.0 or higher. If you’re parked on Sonoma and a 30B model fails to load, that’s the first suspect, not the model.

Memory is the real gate, and on a Mac it’s simpler than the GPU-tier math in earlier entries — unified memory means the model shares RAM with everything else, no VRAM/system split:

• 16 GB: 7B-class models at 4-bit. Usable for Continue autocomplete and chat; too cramped for a serious agent loop plus an IDE plus a browser.
• 24–32 GB: the sweet spot for this series’ standard pick. mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit weighs roughly 17 GB on disk, and being a mixture-of-experts with ~3B active parameters, it generates fast on Apple Silicon. A Mac Mini M4 Pro with 48 GB is the cheapest new hardware that runs it with headroom.
• 64 GB and up: 70B-class dense models at 4-bit, or the 30B pick with generous context. This is MacBook Pro M4 Max territory.

The deeper which-Mac-for-which-model analysis is runaihome.com’s beat — their local AI models by VRAM guide applies directly since unified memory plays the VRAM role. No Mac at all? Then MLX is simply not your backend: rent an NVIDIA GPU (RunPod by the hour) and run vLLM instead — MLX has no answer for that deployment.

Step 1 — Install, serve, sanity-check

The whole install is one line into whatever Python environment you keep for tools:

pip install mlx-lm
# or: conda install -c conda-forge mlx-lm

Serve the coding model. First launch downloads the weights from Hugging Face into the standard HF cache (~/.cache/huggingface/), so give the ~17 GB pull a few minutes:

mlx_lm.server --model mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit --port 8080
# 2026-09-01 ... INFO Starting httpd at 127.0.0.1 on port 8080...

Note what you did not configure: context length. Unlike Ollama’s notorious 4K default, mlx_lm.server doesn’t impose a truncation window — the model’s own context and your RAM are the limits, with --prefill-step-size (default 2048) chunking long prompt ingestion so memory doesn’t spike. One recurring failure mode from the Ollama entries simply doesn’t exist here.

Sanity-check the endpoint the same way as every backend in this series:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit",
       "messages": [{"role": "user", "content": "Write a Python function that reverses a linked list."}],
       "max_tokens": 800}'

A JSON body with a choices[0].message.content full of Python means you’re live. curl http://127.0.0.1:8080/v1/models lists what the server can see — it scans your HF cache for MLX models, which matters for troubleshooting later. /v1/completions also works for tools that still use the legacy text-completion shape.

The flags that matter for agent work

Flag	Default	Why an agent user cares
--max-tokens	512	The trap. See below — raise it or every long reply truncates.
--prompt-cache-size	10	LRU KV cache slots. Agent loops re-send a growing transcript; cache hits skip re-prefilling it.
--prompt-concurrency / --decode-concurrency	8 / 32	Real request concurrency — one server can take Cline and Continue hits at once.
--draft-model + --num-draft-tokens	none / 3	Speculative decoding: point at a small mlx-community model for free extra tokens/sec.
--host	127.0.0.1	Localhost-only by default. 0.0.0.0 exposes it to your LAN — read the security note first.
--temp	0.0	Deterministic by default, which is the right default for diff-editing agents.

The security note is the project’s own, verbatim from the server docs: “The MLX LM server is not recommended for production as it only implements basic security checks.” There’s no API-key auth at all. On 127.0.0.1 for one developer, fine; don’t put it on a network interface you don’t fully trust.

The 512-token trap

Here’s the problem I’d want flagged before wiring in any agent: the server’s default max_tokens is 512, both as a server flag and as the per-request fallback. Chat replies fit in 512 tokens. A Cline file-edit response — reasoning plus a complete rewritten file in a diff block — does not. The symptom is an agent that starts a code block and stops mid-line, which Cline then reports as a malformed or failed edit, and which looks exactly like the model being too weak. It isn’t; it’s a truncation ceiling.

The fix has two layers. Cline, Continue, and Aider all send an explicit max_tokens when configured to, and a per-request value overrides the server default. But belt-and-suspenders says start the server generous so an unconfigured client can’t faceplant:

mlx_lm.server \
  --model mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit \
  --port 8080 \
  --max-tokens 8192 \
  --prompt-cache-size 16

That’s the launch command the rest of this guide assumes. If you’ve read the Cline + Ollama tool-loop debugging guide, this is the same class of failure — a quiet server-side default sabotaging a client that assumes cloud-API behavior — with a different knob.

Wire it into Cline

Cline’s OpenAI Compatible provider takes four fields, and the only subtlety is the model ID:

1. Cline settings → API Provider: OpenAI Compatible
2. Base URL: http://127.0.0.1:8080/v1 — the /v1, not the full /v1/chat/completions path
3. API Key: anything non-empty (mlx works; the server never checks it)
4. Model ID: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit — the exact Hugging Face repo ID you served

That fourth field is where sessions die. The server resolves the request’s model field against the HF cache (or a local path relative to where you launched the server), so qwen3-coder, Qwen3-Coder-30B, and other Ollama-style nicknames all miss. When in doubt, copy the ID out of /v1/models verbatim.

Tool calling — the thing Cline’s agent loop lives on — is genuinely implemented server-side: the request tools array is honored and responses carry tool_calls, with dedicated parsers per model family (Mistral since v0.30.7, Gemma 4 since v0.31.2, parallel-call fixes in v0.31.3). In practice Cline also embeds its tool protocol in the prompt itself, so the load-bearing requirement is a model trained for tool use — the Qwen3-Coder pick is trained for exactly this. If you swap in some niche fine-tune and tool calls start arriving as plain text, suspect the model before the server.

Wire it into Continue.dev

Continue’s openai provider covers any OpenAI-compatible server via apiBase. In ~/.continue/config.yaml:

models:
  - name: Qwen3 Coder (MLX local)
    provider: openai
    model: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit
    apiBase: http://127.0.0.1:8080/v1
    apiKey: mlx
    roles:
      - chat
      - edit
      - apply

Same rule as Cline: model is the full repo ID. One Continue-specific note — if you want a separate small model for autocomplete (a 3B at 4-bit responds fast enough for tab-completion), you don’t need a second server: mlx_lm.server loads models per request, so a second YAML entry pointing at the same apiBase with model: mlx-community/Llama-3.2-3B-Instruct-4bit (the package’s own default model) just works, and the LRU cache keeps both warm. No other backend in this series makes multi-model this cheap to configure.

Worth saying plainly since the Continue acquisition news: the extension is frozen at v2.0.0 under Cursor’s ownership. Local YAML config like the above still works — it’s the cloud features that died — but weigh that freeze before building a new workflow on it.

Wire it into Aider

Aider goes through its OpenAI-compatible endpoint support: set the base, prefix the model with openai/:

export OPENAI_API_BASE=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=mlx

aider --model openai/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit

The double-prefix model string looks wrong and is right: openai/ routes Aider’s LiteLLM layer to the OpenAI dialect, and everything after it passes through as the server-side model ID. Aider’s whole-file and diff edit formats both produce long completions, so this is the tool where the 512-token trap bites hardest — fix it at the server with --max-tokens, because Aider’s own retries can’t out-stubborn a truncating backend. If you see the search/replace failures covered in the Aider block-match fix guide specifically on this backend, check truncation before blaming the model.

Where MLX LM Server actually sits in the series

	mlx_lm.server	Ollama	LM Studio	llama-server
Best for	Mac-only devs who want the raw stack	Cross-platform muscle memory	Mac devs who want a GUI	Fine-grained GGUF control
Install weight	pip install mlx-lm	Native app + daemon	~full desktop app	Build or download binary
Model format	MLX (mlx-community hub)	GGUF (own registry)	GGUF + MLX engines	GGUF
Context default	Model’s own limit	4K unless raised	Per-model setting	Flag-controlled
The catch	Mac-only, 512-token default, no auth	Context + connection quirks	Closed-source app	You manage everything

Performance claim discipline, as always in this series: I’m not publishing tokens-per-second numbers I haven’t measured on hardware you don’t have. The architecture arguments are checkable, though — MLX skips the GGUF conversion layer and compiles straight to Metal, unified memory means no CPU↔GPU weight shuffling, and the KV prompt cache (default 10 entries, LRU-evicted, tunable by count or bytes) is purpose-built for the send-the-whole-transcript-again pattern every coding agent produces. On turn twelve of a Cline session with 40K tokens of accumulated context, re-prefilling from zero versus resuming from cache is the difference you feel.

Problems you’ll hit, and the fixes

Agent replies truncate mid-code-block. The 512-token default. Relaunch with --max-tokens 8192 and set the client’s max output too. This is the single most likely first-session failure.

404 or model-not-found despite the server running. The request’s model string doesn’t match an HF repo ID in your cache. Hit /v1/models, copy the exact ID into Cline/Continue/Aider. Same disease as Aider’s model-not-found error, different registry.

Connection refused from a container or another machine. The default bind is 127.0.0.1. Relaunch with --host 0.0.0.0 — and remember there is no auth layer, so only on networks you trust.

Port 8080 is taken. Half the dev tools on earth default to 8080. --port 8090 and update every client’s base URL to match.

First request after a model swap takes forever. Per-request model loading means the first hit on a new model ID pays the full load from disk. That’s the feature working as designed; subsequent requests are warm.

Verdict

MLX LM Server is the best Mac-native backend in this series for a developer who wants the minimum viable stack: one pip package, Apple’s own framework, models straight from mlx-community, real prompt caching, and honest OpenAI compatibility including tools. It beats Ollama on Mac for context defaults (none of the 4K silliness) and multi-model flexibility, and it beats LM Studio for people who don’t want a desktop app supervising their inference.

It loses everywhere else. No Linux or Windows story that you’d bet a workflow on, no auth, a production disclaimer in its own docs, and a 512-token default that will absolutely eat your first agent session if you skip this guide’s launch flags. Cross-platform teams should stay with Ollama or Docker Model Runner; throughput chasers with NVIDIA hardware belong on vLLM. For the FOSS-stack view of these tools, aifoss.dev covers the open-source side of local AI tooling.

But on a personal Apple Silicon machine serving one developer’s Cline, Continue, and Aider sessions? This is the one I’d run.

FAQ

Does mlx_lm.server require an API key? No — and it can’t check one; there’s no auth implementation at all. Cline requires a non-empty key field in its config, so type any placeholder. Keep the server on 127.0.0.1 unless you fully trust the network.

Can it serve two different models at the same time? Yes, per request: send a different model (HF repo ID or local path) in each request and the server loads it on demand, keeping recent models’ KV states in the LRU prompt cache (default 10 slots, --prompt-cache-size). A big agent model and a small autocomplete model coexist behind one port.

Does tool calling work for Cline and Continue agent loops? The server honors the OpenAI tools parameter and returns tool_calls, with per-family parsers (Mistral, Gemma 4, and others) and parallel-call handling fixed in v0.31.3. Pair it with a tool-trained model — Qwen3-Coder-30B-A3B-Instruct is the safe pick — and both agents run their loops normally.

Why not just use LM Studio’s MLX engine? Same inference framework, different packaging. LM Studio adds a GUI, a model browser, and its own server on port 1234; mlx_lm.server is the bare stack in one Python process. If you already live in LM Studio, stay there — the Cline and Aider LM Studio guides cover it. This backend is for people who want fewer layers, not more.

Is there a Linux or NVIDIA version? PyPI publishes experimental mlx-lm[cuda] and [cpu] extras, but the core dependency pins macOS and the project’s stated scope is Apple Silicon. For Linux/NVIDIA, use vLLM, SGLang, or llama-server — or rent the GPU by the hour on RunPod.

Sources

• MLX LM README — ml-explore/mlx-lm (GitHub)
• MLX LM HTTP server docs (SERVER.md) — endpoints, request fields, security note
• mlx-lm releases (v0.31.3 Apr 22, v0.31.2 Apr 7, v0.31.0 Mar 7 2026 — tool-call parser changelog)
• mlx-lm server source — CLI flags, tool-call formatting, LRU prompt cache
• mlx-lm on PyPI — version, dependencies, platform pins
• mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit — Hugging Face
• MLX array framework — ml-explore/mlx (GitHub)
• Cline OpenAI Compatible provider docs
• Continue.dev OpenAI provider (apiBase) docs
• Aider OpenAI-compatible API docs

Last updated September 1, 2026. Pricing and features change frequently; verify current state before purchasing.

Recommended Gear

• Mac Mini M4 Pro — cheapest new Apple Silicon box that runs the 30B coding pick with headroom (48 GB unified memory configuration)
• MacBook Pro M4 Max — the 64 GB+ tier for 70B-class models or long-context agent sessions

Know which coding tool is worth paying for

Hands-on comparisons of AI coding assistants and what each one costs to run — including the local-model path. Sent only when something changes. Unsubscribe anytime.

Email address Subscribe