RamaLama as a Local Backend for Cline, Continue.dev, and Aider in 2026: Containerized llama.cpp, the Missing --jinja Flag, and Rootless Podman Isolation

By AICoderScope Team · 13 min read
ramalamaclinecontinue-devaiderlocal-llmsetup-guidepodman

TL;DR: RamaLama wraps llama.cpp (or vLLM, or MLX) in a rootless container, auto-detects your GPU, and pulls a matching image — CUDA, ROCm, Vulkan, even Ascend — so nothing touches your host Python or driver stack. One command serves an OpenAI-compatible API on port 8080. The catch: out of the box it never passes --jinja to llama-server, so Cline’s tool calls arrive as useless plain text until you add --runtime-args="--jinja".

What you’ll be able to do after this guide:

  • Install RamaLama 0.24.0, serve unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M in a container matched to your GPU, and sanity-check the endpoint with curl
  • Wire that server into Cline (OpenAI Compatible provider), Continue.dev (provider: openai + apiBase), and Aider (openai/ prefix + OPENAI_API_BASE)
  • Dodge the three launch traps: the missing --jinja flag, the unset context size, and the port that publishes on every network interface with no auth

Honest take: RamaLama is the backend for people who trust containers more than they trust pip install — the model process runs rootless with every Linux capability dropped and the weights mounted read-only, which is a materially better story than any other entry in this series. Pay for it in two ways: you learn three non-obvious flags before your first agent session works, and you accept a project whose own README says to expect breaking changes. On a Linux box with Podman already installed, I’d pick it over bare llama-server. Everywhere else, the simpler tools keep their lead.

The thirteenth backend runs in a box

Twelve servers into this series — Ollama, LM Studio, llama-server, vLLM, SGLang, KoboldCpp, LocalAI, Jan, Docker Model Runner, Lemonade, and MLX LM Server — every one of them has asked the same thing of your machine: install an inference stack onto the host and keep its dependencies happy. RamaLama refuses the premise. It lives in the containers GitHub organization alongside Podman and Buildah, it’s MIT-licensed, it was started by Red Hat engineer Eric Curtin, and its pitch is one sentence long: run AI models the way you run any other container.

The mechanics follow from that. ramalama serve inspects your hardware, picks a container image built for it, and launches llama-server inside that image — your host needs a container engine (Podman by default, Docker works) and nothing else. No CUDA toolkit on the host, no ROCm version matching, no Python environment. The image map as of version 0.24.0 covers NVIDIA (quay.io/ramalama/cuda), AMD (quay.io/ramalama/rocm), Vulkan/CPU (quay.io/ramalama/ramalama), Intel GPUs (intel-gpu, openvino), Apple Silicon on Asahi Linux (asahi), Huawei Ascend NPUs (cann), and Moore Threads GPUs (musa) — the widest accelerator net in this series, wider than Lemonade’s AMD-centric matrix.

Version check for this writing: RamaLama 0.24.0, released August 21, 2026, current on PyPI as of September 2, 2026. The cadence has been steady — 0.21.0 on May 14, 0.22.0 on June 5, 0.23.0 on June 24 — and the recent changelog is agent-relevant: 0.23.0 added a multi-model router mode for serve, and 0.24.0 added speculative-decoding flags with automatic draft-model pulls. The README is equally direct about maturity: “everything is under development, so expect breaking changes.”

Step 1 — Install RamaLama and a container engine

On Fedora and friends, it’s in the distro repos; anywhere else, the install script or pip works:

# Linux / macOS universal script
curl -fsSL https://ramalama.ai/install.sh | bash

# or, if you prefer explicit package managers
pip install ramalama        # any platform with Python 3.9+
sudo dnf install ramalama   # Fedora

macOS also gets a self-contained .pkg installer that bundles Python. You need Podman or Docker installed and working first — podman run hello-world is the ten-second check. If you genuinely can’t run containers, ramalama --nocontainer serve ... executes the runtime directly on the host, but that throws away the isolation that justifies the tool; at that point bare llama-server is fewer moving parts.

Hardware sizing is the same as everywhere in this series, because underneath it’s the same llama.cpp and the same GGUF files:

VRAM / unified memoryModelWeights on diskWhy
8–12 GBhf://unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M won’t fit; use a 7B-class coder~4–5 GBChat and autocomplete, not agent loops
24 GB (RTX 3090/4090)hf://unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M18.6 GBThe series-standard agent pick: MoE, ~3B active params, tool-trained
48 GB+ (Mac Mini M4 Pro 48 GB and up)Same model with long context, or 70B-class dense18.6+ GBRoom for the KV cache agent sessions actually need

The runaihome.com VRAM guide has the fuller hardware tiers. No GPU at all? A RunPod instance with a 24 GB card runs every command below unmodified.

Step 2 — The serve command that actually works for agents

Here’s the launch this whole article exists to document:

ramalama serve --name coder \
  --ctx-size 32768 \
  --host 127.0.0.1 \
  --runtime-args="--jinja" \
  hf://unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M

First run pulls the container image and the model (the hf:// transport replicates llama.cpp’s -hf logic, quant tag and all — ollama://, oci://, and modelscope:// transports also work). The server detaches by default and prints a container ID; ramalama stop coder shuts it down. Each of those three flags un-breaks something:

--runtime-args="--jinja" — without it, Cline is broken. This is the finding I want you to leave with. llama.cpp’s own docs are explicit that OpenAI-style function calling requires launching llama-server with --jinja, which activates the model’s chat template — we covered why in the llama-server guide. Reading RamaLama 0.24.0’s source (llama_cpp_commands.py, where the serve command line is assembled), the flags it passes are --host, --port, --model, --alias, --no-warmup, --temp, threads, and friends — --jinja is never among them, and no config option adds it. The escape hatch is --runtime-args, which shell-splits its string and appends it to the llama-server invocation verbatim. Skip this and Cline shows the model narrating <tool_call> JSON as chat text instead of executing tools — the same symptom family as the Ollama tool-use loop bug, different cause.

--ctx-size 32768 — the default is not what the docs imply. RamaLama’s configuration reference lists ctx_size defaulting to 0, glossed as “use model default.” What the source actually does with 0 is omit the flag entirely, which hands the decision to llama-server’s own default — 4,096 tokens as of this writing. A Cline system prompt plus one file read blows past 4K before the agent does anything, and you get the familiar amnesia spiral: the model re-reads files it just saw, forgets the plan, loops. It’s the same trap we documented for Ollama’s num_ctx wearing a different config key. 32K is the practical floor for agent work; on a 24 GB card with this model you have the headroom.

--host 127.0.0.1 — because the default publishes to your whole network. RamaLama’s default host is ::, and with that value the container port is published as -p 8080:8080 — every interface, and llama-server has no authentication of any kind. On a home network behind NAT that’s a judgment call; on café Wi-Fi it’s an open inference server with your name on it. Setting --host 127.0.0.1 makes the publish spec 127.0.0.1:8080:8080, loopback only. (Port 8080 taken? RamaLama hunts 8080–8180 for a free one — check the startup output, or pin it with -p 9090.)

Sanity-check the endpoint the same way as every backend in this series:

$ curl http://127.0.0.1:8080/v1/models
{"object":"list","data":[{"id":"unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M","object":"model", ...}]}

Copy that id string exactly — it’s what goes in every client config below (RamaLama sets it via llama-server’s --alias). Bonus: a browser pointed at http://127.0.0.1:8080 gets llama.cpp’s built-in web UI, on by default; --webui off disables it.

Step 3 — Wire in the three tools

Nothing here differs by much from the rest of the series, which is the point — it’s the same OpenAI dialect on the other side of a container boundary.

Cline (VS Code): API Provider → OpenAI Compatible. Base URL http://127.0.0.1:8080/v1, API key set to anything non-empty (ramalama works — the field is required by Cline, not by the server), Model ID = the exact string from /v1/models. Set Cline’s context window setting to match your --ctx-size. If tool calls still come back as text after all that, you launched without --jinja — relaunch; there is no client-side fix.

Continue.dev (~/.continue/config.yaml):

models:
  - name: ramalama-qwen3-coder
    provider: openai
    model: unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M
    apiBase: http://127.0.0.1:8080/v1
    apiKey: ramalama
    roles: [chat, edit, apply]

The standing caveat from the Cursor acquisition still applies: the extension is frozen at v2.0.0, so treat it as stable-but-static. It speaks this API fine.

Aider (terminal):

export OPENAI_API_BASE=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=ramalama
aider --model openai/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M

The doubled-looking openai/unsloth/... is correct — the prefix routes litellm to the OpenAI-compatible provider, the rest is the alias. Aider will print its “Unknown context window size and costs, using sane defaults” warning; that’s cosmetic, and the Aider model-not-found guide covers silencing it with .aider.model.settings.yml if it bothers you.

Router mode: two models, one port

Since 0.23.0, handing serve more than one model (or none) flips it into router mode: the models are mounted into a single container and llama.cpp’s multi-model directory feature routes each request by the model field, loading up to --models-max (default 4) concurrently. That enables the pairing this series keeps recommending — a 30B agent model for Cline plus a small fill-in-the-middle model for Continue autocomplete — behind one base URL, no second server. MLX LM got there via per-request lazy loading; RamaLama’s version is explicit and container-only. It’s also new: router mode is two releases old, in a project that warns about breaking changes, so pin your version if you build a workflow on it.

The security story, and its honest limits

What makes RamaLama genuinely different in this series is what the model process can’t do. Verified against the README and the engine source: the container runs rootless, with --cap-drop=all and --security-opt=no-new-privileges, model weights mounted read-only, and --rm wiping container-written state on exit. For ramalama run sessions, the container gets --network=none — the process literally cannot phone home. A serve container necessarily keeps a network namespace with the published API port.

Given this year’s run of agent-stack attacks — Agentjacking, GhostApproval, the Amazon Q MCP CVE — a fair question is how much this buys you. Be precise about the boundary: RamaLama sandboxes the inference server, so a malicious or compromised GGUF, a llama.cpp parser exploit, or a poisoned quant is contained with no capabilities, no host filesystem write, and (for run) no network. It does nothing about the agent side — Cline still executes on your host with whatever permissions you gave it, and a prompt-injected model can still talk it into mischief through the API response. Right defense, right layer: RamaLama hardens the layer that runs untrusted weights, which no other backend in this series hardens by default.

Where it breaks

Alpha software, moving fast. Four releases since mid-May, a README that promises breaking changes, and features like router mode that are weeks old. The counterweight: it’s the same battle-tested llama-server inside; RamaLama’s blast radius is mostly launch plumbing.

First-run downloads are chunky. You pull a GPU-specific container image and an 18.6 GB model before the first token. Subsequent launches are warm.

macOS is supported, with an asterisk. Apple Silicon GPU inference works via llama.cpp or the MLX runtime (--runtime mlx, which requires --nocontainer — MLX needs Metal, and Metal doesn’t reach into a Linux VM). Which means on a Mac you’re either giving up the container isolation or running through podman-machine’s VM path. If you’re Mac-only, the MLX LM Server guide is the shorter road.

No auth, ever. Like most of this series. The --host 127.0.0.1 flag above is not optional advice.

Throughput ceilings are llama.cpp’s. One user, one GPU: fine. Team serving or batch throughput: that’s vLLM (which RamaLama can also drive via --runtime vllm) or SGLang territory.

Performance claim discipline, as always: no tokens-per-second numbers I haven’t measured on hardware you don’t have. The architecture argument is checkable — inside the container it is llama-server, so single-user latency should track the bare llama-server setup on the same hardware, plus a container start once per launch, not per request.

Verdict

RamaLama is the right backend for a specific developer: Linux, Podman or Docker already installed, security-conscious enough to care that the thing executing downloaded weights has zero capabilities and a read-only filesystem, and comfortable reading a man page when a default bites. For that person it beats bare llama-server — same engine, no host dependency rot, real isolation — and it beats Ollama on transparency, because every llama-server flag is inspectable and overridable via --runtime-args.

It is not the beginner pick. Three flags stand between the quick-start command and a working Cline session, the project is self-declared alpha, and the Mac story sends you elsewhere. If you want zero-config, Ollama and LM Studio still win; if you want Docker-native without a new CLI, Docker Model Runner is closer to home. For the FOSS-stack view of these tools, aifoss.dev covers the open-source side of local AI tooling.

FAQ

Does RamaLama require an API key? No, and it can’t check one — llama-server inside has no auth layer. Cline and Continue require a non-empty key field, so type any placeholder. Keep the bind on 127.0.0.1 unless you fully trust the network.

Why do Cline tool calls fail when the same model works in Ollama? Ollama applies chat templates (and tool-call parsing) automatically; RamaLama’s llama-server needs --jinja for that, and RamaLama doesn’t pass it. Launch with --runtime-args="--jinja". Verify by checking the server log for a Chat format: line naming your model family rather than Generic.

Podman or Docker? Podman is the default (engine = "podman" in ramalama.conf) and the rootless security posture is its native mode. Docker works — RamaLama adds host.docker.internal mapping automatically. If both are installed, it prefers Podman.

Can I use it with Cursor? Cursor’s BYOK routing goes through Cursor’s servers, so a loopback URL won’t work directly — the Cursor + Ollama guide covers the tunnel workaround, which applies unchanged here since it’s the same OpenAI dialect.

What’s rlcr://? A fifth model transport, pointing at RamaLama’s own OCI registry (registry.ramalama.com). For coding models you’ll live on hf:// and ollama://; the OCI path matters if your team already ships artifacts through quay.io or Docker Hub and wants models versioned the same way.

Sources

Last updated September 2, 2026. Pricing and features change frequently; verify current state before purchasing.

  • RTX 3090 — the used-market 24 GB workhorse that runs the Q4_K_M agent pick with KV-cache headroom
  • Mac Mini M4 Pro — 48 GB unified memory tier for the same model via podman-machine or the MLX runtime

Was this article helpful?

Know which coding tool is worth paying for

Hands-on comparisons of AI coding assistants and what each one costs to run — including the local-model path. Sent only when something changes. Unsubscribe anytime.