Ollama has to fit a model in VRAM, or in VRAM plus system RAM. When it cannot, it either splits the model across GPU and CPU, which is slow, or refuses to load it and logs an error such as model requires more system memory than is available. The fix is to load less: a smaller model, a lower quantization, or a shorter context.
1. Confirm the CPU/GPU split
ollama ps
The PROCESSOR column reads 100% GPU (good), 100% CPU, or a split like 48%/52% CPU/GPU when the model spilled out of VRAM.
2. Pick a smaller model or lower quantization
See what is installed and how big each build is:
ollama list
The SIZE column shows each model on disk; switch to a smaller parameter count, or a tighter quant such as a q4 build instead of q8 or fp16. See how much RAM or VRAM a model needs and how to choose a quantization.
3. Shrink the context window
A larger context costs memory for the KV cache, so lower it. The default is 4096 tokens.
On macOS or Linux:
OLLAMA_CONTEXT_LENGTH=4096 ollama serve
On Windows (PowerShell):
$env:OLLAMA_CONTEXT_LENGTH=4096; ollama serve
Each sets the default context for the server. Per model, run /set parameter num_ctx 4096 inside ollama run.
4. Free VRAM other apps are holding
nvidia-smi
Lists every process on the GPU; close browsers, games, or a second model so Ollama gets a clean 100% GPU load.
Verify it worked
ollama ps
A 100% GPU in the PROCESSOR column means the model fits.
If it fails:
- Still
100% CPUon a good GPU: the GPU was never detected, so fix that first in Ollama not using your GPU. model requires more system memory: the model is still too big; read the Windows log at%LOCALAPPDATA%\Ollama\server.log(or~/.ollama/logs/server.log) for the exact figures, then pick a smaller quantization.- Memory frees slowly after a run: Ollama unloads idle models after 5 minutes; shorten it with
OLLAMA_KEEP_ALIVE. Full behaviour in the Ollama FAQ.