TL;DR: When Ollama can’t use your GPU, it doesn’t error — it quietly runs the model on your CPU and lets your coding agent crawl at one or two tokens per second. ollama ps tells you in one column whether that’s happening. The fix is almost never “reinstall Ollama”: it’s a leftover environment variable, a driver below the floor, a Docker runtime gap, or a context request from your coding tool that pushed half the model out of VRAM.

What you’ll be able to do after this guide:

• Read the PROCESSOR column in ollama ps and know within 30 seconds whether you’re on GPU, CPU, or the worst-of-both partial split
• Trace silent CPU fallback to its actual cause: env-var sabotage, the driver floor, the Linux suspend/resume bug, Docker cgroup resets, or agent-sized context windows
• Keep a coding model at 100% GPU even when Cline or Aider requests a 64k context

Honest take: The single most common version of this problem in coding setups isn’t a broken GPU at all — it’s a model that fits in VRAM at the CLI’s default context but spills to CPU the moment your agent asks for 64k. If ollama run feels fast and Cline feels broken, check the context math before you touch a driver.

This is the fourth entry in our Ollama troubleshooting series, after connection errors, the silent context-length trap, and out-of-memory crashes. Everything below was verified on August 9, 2026 against Ollama v0.32.6 (released August 4, 2026), Cline 4.1.6, Continue.dev 1.3.40, and Aider 0.86.2.

Why you don’t get an error message

Out-of-memory failures announce themselves. GPU fallback doesn’t. Ollama takes inventory of your GPUs at server startup, and if discovery fails — wrong driver, missing container runtime, a GPU that dropped off the bus after suspend — it falls back to the CPU code path and keeps serving requests as if nothing happened. Your coding tool gets valid responses. They just take twenty times longer.

Inside the tools, that latency wears three different disguises:

• Cline appears to hang mid-task. Agent turns that took 15 seconds now take 5+ minutes, and long tool-use loops hit Cline’s request timeout and surface as retry banners — which look identical to the connection errors they aren’t.
• Continue.dev autocomplete stops feeling like autocomplete. FIM suggestions that should land in under a second arrive after you’ve already typed the line yourself. Chat still “works,” so most people blame the model.
• Aider streams its edit blocks one word at a time. Because Aider prints tokens as they arrive, it’s actually the most honest of the three — a crawling stream is your clearest visual signal that inference moved to the CPU.

None of these show an Ollama error, which is why people burn evenings adjusting tool settings when the problem is one layer down.

The 30-second diagnosis

One command, one column:

$ ollama ps
NAME               ID              SIZE     PROCESSOR      CONTEXT    UNTIL
qwen3-coder:30b    abc123def456    21 GB    100% CPU       32768      4 minutes from now

Ollama’s own FAQ defines the three states of the PROCESSOR column: 100% GPU means the model is fully in VRAM, 100% CPU means it loaded entirely into system memory, and a split like 48%/52% CPU/GPU means it’s straddling both. For interactive coding, treat anything other than 100% GPU as broken — a partial split throttles generation to the speed of the slowest layer, and official docs are blunt about it: for best performance, avoid offloading the model to CPU.

Cross-check with your GPU while a request is running. If nvidia-smi shows near-zero GPU utilization and no ollama process in its memory table while Aider is generating, inference is on the CPU no matter what you configured.

Then read the server log, because discovery failures are recorded there even though clients never see them:

# macOS
$ cat ~/.ollama/logs/server.log

# Linux (systemd)
$ journalctl -u ollama --no-pager --follow

# Windows: explorer %LOCALAPPDATA%\Ollama  →  server.log
# Docker
$ docker logs <container-name>

On NVIDIA, failed GPU initialization shows up as numeric error codes in the log — the official troubleshooting docs call out “3” (not initialized), “46” (device unavailable), “100” (no device), and “999” (unknown). On AMD, a driver mismatch looks like this, straight from the docs:

msg="failure during GPU discovery" ... error="failed to finish discovery before timeout"
msg="bootstrap discovery took" duration=30s ...

Set OLLAMA_DEBUG=1 and restart the server to get verbose discovery logging when the cause isn’t obvious.

Fix #1: the coding-agent context spill

Start here if ollama ps shows a percentage split rather than 100% CPU. A split means discovery worked fine — the model just doesn’t fit anymore, and the usual reason in coding setups is the context window your tool requested.

Ollama’s context docs recommend at least 64,000 tokens for agents and coding tools, and modern Cline and Continue.dev setups ask for exactly that. But KV cache scales with context, and on a card with less than 24 GiB of VRAM Ollama’s own default is a 4k context for a reason. The same 19–21GB model that sits at 100% GPU under ollama run can spill 30% of its layers to system RAM once a 64k KV cache muscles into VRAM alongside it. Result: the model “works” in both places and is only unusable in the one that matters.

Three levers, in the order to pull them:

Lever	Command / setting	What it buys you
Quantize the KV cache	OLLAMA_FLASH_ATTENTION=1 + OLLAMA_KV_CACHE_TYPE=q8_0	Roughly half the cache memory at f16, per official docs — often the difference between a split and 100% GPU
Right-size the context	OLLAMA_CONTEXT_LENGTH=32768 (or per-tool num_ctx)	Cache scales linearly-ish with context; 32k is a workable floor for agent work on 16–24GB cards
Evict squatters	ollama stop <model>	Models idle up to 5 minutes by default and OLLAMA_MAX_LOADED_MODELS defaults to 3 per GPU — a forgotten chat model may be occupying the VRAM your coding model needs

Watch one trap from the parallelism side: OLLAMA_NUM_PARALLEL multiplies the context allocation (the FAQ’s example: 2k context × 4 parallel = an 8k allocation). If you raised it to serve Continue.dev autocomplete and Cline simultaneously, you multiplied your KV cache too.

The full memory math — including when to accept a smaller quant instead — is in the OOM guide, and picking a model that honestly fits your card is covered in runaihome.com’s best local AI models by VRAM.

Fix #2: environment variables that force CPU on purpose

Ollama ships several switches that deliberately disable GPU use, and every one of them is something a past-you might have set during an experiment and forgotten:

• CUDA_VISIBLE_DEVICES=-1 — the documented way to force CPU on NVIDIA is setting an invalid GPU ID. If this is lingering in your shell profile or systemd unit, Ollama is doing exactly what it was told.
• OLLAMA_LLM_LIBRARY=cpu (or cpu_avx, cpu_avx2) — the library override bypasses GPU autodetection entirely. Handy for debugging a crash once; catastrophic to leave in place.
• ROCR_VISIBLE_DEVICES=-1 — same invalid-ID trick, AMD edition.
• GGML_VK_VISIBLE_DEVICES=-1 / OLLAMA_VULKAN=0 — disables Vulkan GPUs, which is the supported path for many Intel GPUs and older AMD cards.

Where to look: systemctl cat ollama on Linux (overrides live in the [Service] block via systemctl edit ollama), user environment variables on Windows (Ollama inherits them), and launchctl getenv on macOS. This class of problem survives reinstalls, which is exactly why reinstalling never fixes it.

The multi-GPU variant is subtler: CUDA_VISIBLE_DEVICES set to a numeric ID can silently point at the wrong card after a reboot, because ordering isn’t stable. The docs recommend UUIDs from nvidia-smi -L instead. On laptops with hybrid graphics, the same mechanism can pin Ollama to the iGPU — on mixed iGPU/dGPU Vulkan systems, set GGML_VK_VISIBLE_DEVICES to the discrete GPU’s index.

Fix #3: your GPU or driver is below the floor

Ollama v0.32.x supports NVIDIA GPUs with compute capability 5.0 and up, with driver 550 or newer — and cards in the 5.0–6.2 range (GTX 900/1000 series, among others) specifically need driver 570+. That second clause bites people who “already updated”: a GTX 1070 on driver 555 meets the first requirement and still fails discovery. On AMD, Linux support now requires the ROCm v7 driver stack — an older ROCm 6.x install produces the discovery-timeout log shown above and a silent CPU fallback, and the fix is upgrading via AMD’s amdgpu-install utility, then rebooting.

Check your card against NVIDIA’s compute capability list. If it’s below 5.0, stock Ollama will never use it, and you have three honest options: build Ollama locally with older-GPU support (documented, fiddly, unsupported), rent a GPU by the hour on RunPod and point your tools at the endpoint, or buy used — a RTX 3090 remains the default recommendation for 24GB-class local coding in 2026. CPU-only inference on a 20GB coding model is not a fourth option anyone should live with.

Fix #4: it worked yesterday — the suspend/resume bug

The signature developer version of this problem: Ollama on your Linux laptop or desktop ran on GPU all day, you suspended overnight, and this morning every Cline task crawls. The official docs acknowledge this directly — after a suspend/resume cycle, Ollama can fail to rediscover the NVIDIA GPU and falls back to CPU. The workaround is reloading the UVM driver, no reboot needed:

$ sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm

Then restart Ollama and confirm with ollama ps that new loads land at 100% GPU. Community reports of this class of problem — GPU inference that randomly degrades to CPU until a restart, with plenty of free VRAM showing in nvidia-smi — go back to at least January 2026 (ollama/ollama#13765), and Windows users report the same silent 100% CPU state with a healthy card sitting idle (#15516). If reloading UVM doesn’t stick, sudo nvidia-modprobe -u and a driver update are the documented next steps.

Fix #5: Docker and WSL2 — the GPU never made it inside

Running Ollama in a container adds a layer that must explicitly pass the GPU through. Three separate failure modes here:

The runtime is missing. GPU access in Docker requires the NVIDIA Container Toolkit and the --gpus=all flag. The docs’ isolation test cuts straight to it:

$ docker run --gpus all ubuntu nvidia-smi

If that command can’t print your GPU table, Ollama in Docker never had a chance — fix the toolkit install before touching Ollama. Inside WSL2, run nvidia-smi in the WSL shell first for the same reason.

GPU works, then degrades to CPU hours later. This one is nasty because startup logs look perfect. The official troubleshooting docs describe it exactly: Ollama initially works on the GPU in Docker, then switches to CPU with discovery errors in the log. The documented fix is disabling systemd cgroup management in Docker — add "exec-opts": ["native.cgroupdriver=cgroupfs"] to /etc/docker/daemon.json on the host and restart the daemon. The restart-cures-it-temporarily pattern in #13765 is what this looks like from the user side.

It’s a Mac. Docker Desktop on macOS has no GPU passthrough — containerized Ollama there is CPU-only by design, per the FAQ. Run the native app instead; it uses Apple’s Metal API directly.

AMD containers add one more: the container needs access to /dev/kfd and /dev/dri, which can require --group-add with the numeric group IDs from the host and, on SELinux systems, sudo setsebool container_use_devices=1.

Fix #6: AMD cards that ROCm doesn’t officially know

If your Radeon card isn’t on Ollama’s ROCm support list, discovery may skip it even with correct drivers. The documented escape hatch is forcing a near-match LLVM target — for example, an RX 5400 (gfx1034, unsupported) can run with HSA_OVERRIDE_GFX_VERSION="10.3.0", the closest supported target. Per-GPU overrides append the device number (HSA_OVERRIDE_GFX_VERSION_0=10.3.0). On Linux, also confirm the ollama user is in the video and/or render groups — permission failures on /dev/kfd are detected and logged, but again: logged, not surfaced to Cline.

For cards with no workable ROCm target, Vulkan is now the fallback path — enabled by default when the backend is installed, though on Linux the scheduler needs either root or sudo setcap cap_perfmon+ep /usr/local/bin/ollama to read real VRAM data for scheduling decisions.

Confirm the fix from inside your coding tool

After any change: restart Ollama, trigger a real request from the tool (not ollama run — it doesn’t reproduce the agent’s context request), and re-check:

$ ollama ps
NAME               ID              SIZE     PROCESSOR    CONTEXT    UNTIL
qwen3-coder:30b    abc123def456    21 GB    100% GPU     32768      4 minutes from now

100% GPU under the load your actual tool generates is the finish line. In Cline, a normal agent turn should stream visibly again; in Continue.dev, completions should return faster than you type; in Aider, the edit-block stream should flow rather than drip. If the split returns only when your agent connects, you’re back in Fix #1 territory — that’s a context spill, not a discovery failure, and the context-length guide covers the per-tool num_ctx settings in detail.

Driver-level rabbit holes — clean NVIDIA reinstalls, ROCm version pinning, iGPU BIOS toggles — are out of scope for a coding-tools site, and our sister site has the deep version: Ollama not using GPU on Windows, WSL2, and Linux at runaihome.com.

FAQ

Why is Ollama using the CPU when I have a GPU? One of five reasons, in rough order of frequency: the model plus its KV cache doesn’t fit in VRAM (partial or full spill), a leftover environment variable is forcing CPU, your driver is below Ollama’s floor (NVIDIA 550+, or 570+ for compute capability 5.0–6.2 cards; ROCm v7 on AMD), the GPU vanished after suspend/resume, or a container never got GPU passthrough. ollama ps plus the server log identifies which within a minute.

Is 48%/52% CPU/GPU in ollama ps okay for coding? No. A split keeps things technically functional but generation runs at close to CPU speed, which is unusable for agent loops. Quantize the KV cache, trim context, evict other models, or use a smaller quant until you’re back at 100% GPU.

Why does ollama run use the GPU but Cline runs on CPU? Because they make different requests. The CLI uses a small default context; Cline asks for a large one, and the extra KV cache pushes layers out of VRAM. Compare the CONTEXT column in ollama ps during each session — the number, not the model, is what changed.

Does reinstalling Ollama fix GPU fallback? Almost never. The common causes — environment variables, drivers, container runtimes, context math — all survive a reinstall. The one case where “turn it off and on again” legitimately helps is the post-suspend UVM bug, and even there rmmod nvidia_uvm && modprobe nvidia_uvm is faster.

Can I force Ollama to use the GPU? There’s no “force GPU” switch — discovery either succeeds or it doesn’t. What you can do is remove everything that blocks it: correct drivers, no CPU-forcing env vars, working container runtime, and a model+context allocation that fits VRAM. For unsupported AMD cards, HSA_OVERRIDE_GFX_VERSION is the closest thing to a force flag that exists.

Sources

• Ollama hardware support (GPU compatibility, driver floors, overrides) — official docs
• Ollama troubleshooting (discovery errors, suspend/resume, Docker cgroup fix) — official docs
• Ollama FAQ (ollama ps states, Docker GPU, concurrency) — official docs
• Ollama context length (agent 64k recommendation, VRAM-tiered defaults) — official docs
• Ollama v0.32.6 release — GitHub
• ollama/ollama#13765 — “Randomly uses CPU even when GPU available until I restart”
• ollama/ollama#15516 — “ollama not using gpu windows”
• NVIDIA CUDA GPU compute capability list — NVIDIA Developer
• NVIDIA Container Toolkit — GitHub
• Ollama not using GPU: Windows, WSL2, and Linux driver-level fixes — runaihome.com
• Best local AI models by VRAM — runaihome.com

Last updated August 9, 2026. Verified against Ollama v0.32.6. GPU discovery behavior changes between Ollama releases; check the official docs for the current state.