Use a Local Model in Cursor or Claude Code
Point Cursor's chat at a local OpenAI-compatible model, and see why Claude Code cannot use a non-Claude or local model as its backend.
On this page5 sections
Short answer: you can point Cursor’s chat at a local model, with real limits. Claude Code is different. Anthropic states it does not support routing Claude Code to non-Claude models, so a local model is not a supported backend there. Here is what each can and cannot do.
1. Serve a local OpenAI-compatible endpoint
First expose a local OpenAI-compatible API. Ollama’s daemon serves one at http://localhost:11434/v1/ whenever it runs; LM Studio serves one once you start its server.
ollama pull qwen2.5-coder && ollama serve
That pulls a coding model and runs the daemon that answers OpenAI-style requests. See Ollama on Windows or LM Studio, and size it with RAM and VRAM requirements.
2. Point Cursor at a custom key and base URL
In Cursor Settings > Models, paste any string as the OpenAI key, then set the base URL override in that key’s advanced options and add your model name by hand.
Base URL: http://localhost:11434/v1
Model: qwen2.5-coder
Cursor appends /chat/completions to whatever base URL you enter.
3. Know what Cursor still sends to its servers
Here is the gotcha: custom keys only work with chat models, and every request is still routed through Cursor’s servers for final prompt building, so it is not fully offline. Tab completion, inline edit, and Cloud Agents stay on Cursor’s own models, and Zero Data Retention does not apply when you use your own key.
4. Understand what Claude Code supports
Claude Code is honest here: ANTHROPIC_BASE_URL points it at an LLM gateway, but Anthropic says it does not support routing Claude Code to non-Claude models through any gateway.
export ANTHROPIC_BASE_URL=https://your-gateway.example.com
That gateway must still serve Claude models (Anthropic, Bedrock, or Vertex) — you cannot swap in a local Llama or Qwen as a supported backend.
5. Pick the right tool for private work
So for truly local inference, use Cursor’s chat with the caveats above, or call the endpoint from your own code. Do not expect Claude Code to run offline on local weights. Details are in the Claude Code gateway docs.
If it fails:
- 404 from the endpoint: your base URL is missing
/v1, or you pointed at the root. - Cursor ignores your model: custom keys only work with chat; Tab and agent stay on Cursor’s models.
- Claude Code still reaches Anthropic:
ANTHROPIC_BASE_URLonly redirects to a Claude-format gateway, never to a local non-Claude model.