# Call any local model with the OpenAI-compatible API

> Ollama, LM Studio, llama.cpp and vLLM all expose an OpenAI-compatible /v1 API. List models, run a chat completion, and stream from one SDK.

- Canonical: https://guides-ai.com/guides/call-a-local-model-openai-compatible/
- Plate 09.14 · Topic: Local LLMs (https://guides-ai.com/topics/local-llm/)
- Published: 21 Oct 2026 · 3 min read
- Source site: guides-ai — https://guides-ai.com/

Ollama, LM Studio, the llama.cpp server and vLLM all speak the same
[OpenAI-compatible](/glossary/#openai-compatible-api) wire format under a `/v1` path. One client
library works against any of them — only the base URL and the model name change.

## 1. Find your server's base URL and port

```text
Ollama       http://localhost:11434/v1
LM Studio    http://localhost:1234/v1
llama.cpp    http://localhost:9931/v1
vLLM         http://localhost:8000/v1
```

These are each server's current default; override with its `--host`/`--port` flag or app setting (older llama.cpp builds used port 8080).

## 2. List the models with GET /v1/models

```bash
curl http://localhost:8000/v1/models
```

Every server implements this endpoint, and the `id` fields it returns are the exact model names to pass below — this example hits vLLM, so swap the port for another server.

## 3. Run a chat completion with the OpenAI SDK

```bash
pip install openai
```

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:1234/v1", api_key="not-needed")
resp = client.chat.completions.create(
    model="your-model-id",
    messages=[{"role": "user", "content": "Say this is a test"}],
)
print(resp.choices[0].message.content)
```

Point `base_url` at your server and pass any non-empty `api_key`, which local servers require as a field but ignore as a value.

## 4. Stream the response token by token

```python
stream = client.chat.completions.create(
    model="your-model-id",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)
```

With `stream=True` the text arrives in chunks at `choices[0].delta.content`, which is `None` on the final chunk — hence the `or ""`.

## 5. Check which params each server supports

Honesty check: not every server implements every field. All four handle `model`, `messages`,
`temperature`, `max_tokens` and `stream`, but coverage of `tools`, `response_format`,
`logprobs` and `/v1/embeddings` varies between projects and versions. Check the server's own
docs before you depend on a parameter, and prefer the `model` strings reported by
`/v1/models` over names copied from a blog.

## Verify it worked

A good sign: `curl .../v1/models` returns JSON with a `data` array, and a chat call prints text.

**If it fails:**

- `Connection refused` — the server isn't running or the port is wrong; recheck the list in step 1 (LM Studio starts its server from the Developer tab; vLLM and llama.cpp print the bound address on launch).
- 404 `model '...' does not exist` — the `model` string doesn't match any `id` from `/v1/models`; copy it verbatim.
- `api_key client option must be set` — the SDK needs a non-empty `api_key`, so pass any placeholder string.

Server setup lives in the dedicated guides: [Ollama](/guides/call-ollama-api/),
[LM Studio](/guides/run-llm-lm-studio/), [vLLM](/guides/vllm-serve-openai-compatible/) and
[llama.cpp](/guides/llama-cpp-quickstart/).

Next: [point Cursor or Claude Code at a local model](/guides/local-model-in-cursor-or-claude-code/).

Source: [Ollama OpenAI compatibility](https://docs.ollama.com/api/openai-compatibility) ·
[LM Studio OpenAI API](https://lmstudio.ai/docs/api/endpoints/openai) ·
[vLLM OpenAI-compatible server](https://docs.vllm.ai/en/latest/serving/online_serving/).