# Which Local LLM Should You Run?

> Choose a local LLM by what fits your RAM and VRAM first, then by task, with starter picks for 8GB, 16GB, and 24GB machines.

- Canonical: https://guides-ai.com/guides/which-local-model-should-i-run/
- Plate 11.07 · Topic: Choosing tools (https://guides-ai.com/topics/choosing/)
- Published: 21 Oct 2026 · 3 min read
- Source site: guides-ai — https://guides-ai.com/

Answer first: run the largest model your memory holds, then the best one for your task. Chasing a bigger name than your box fits only makes it swap to disk and crawl. Decide hardware, then task, then size, in that order — the same discipline as [picking an AI model](/guides/pick-an-ai-model/).

## 1. Start with your hardware, not the model

A model's parameter count times its quantization has to fit in RAM for CPU inference or VRAM for GPU inference, with headroom for context. Do that math in [RAM and VRAM requirements](/guides/local-llm-ram-vram-requirements/) before you download anything, because it rules out most choices immediately.

## 2. Then choose by task

Within what fits, match the family to the job: general chat and drafting (`llama3.2`, `qwen3`, `gemma3`), code (`qwen2.5-coder`), vision questions about images (`llava`), and embeddings for search or RAG (`nomic-embed-text`). All of these exist in the [Ollama](/glossary/#ollama) library today.

## 3. Set the quantization tradeoff

[Quantization](/glossary/#quantization) shrinks a model so a larger one fits your box. A 4-bit quant is the usual sweet spot, trading a little quality for roughly half the memory of an 8-bit build. Weigh it in [choose a quantization](/guides/choose-llm-quantization/).

## 4. Starter picks by memory tier

Concrete starting points, assuming a 4-bit quant:

```text
8 GB:    a 3B-class model (llama3.2) for chat and drafting
16 GB:   a 7-8B-class model (qwen2.5-coder for code, qwen3 for chat)
24 GB+:  a 14B+ model, or run nomic-embed-text alongside a chat model
```

Move up a size only when a real task fails, not on spec-sheet envy.

## 5. Pull one and test

Pick your tier's model and run it to confirm it actually fits.

```bash
ollama run llama3.2 "Say hello and name yourself"
```

If it replies fast and coherently, keep it; if it stalls or swaps, drop a size. Not sure which runner? Compare [Ollama vs LM Studio](/guides/ollama-vs-lm-studio/), and browse exact tags in the [Ollama model library](https://ollama.com/library).

**If it fails:**

- Out of memory or very slow: the model plus context exceeds your RAM or VRAM — pick a smaller size or a lower quant.
- "model not found": that tag does not exist — run `ollama pull <name>` or check the name in the library.
- Answers are too weak: you undersized for the task — move one tier up if memory allows, or switch to a task-specific family.