# Run LLMs Fast on Apple Silicon with MLX

> MLX is Apple's framework for M-series Macs. Install mlx-lm, generate from a Hugging Face model, and serve an OpenAI-style API with mlx_lm.server.

- Canonical: https://guides-ai.com/guides/local-llm-apple-silicon-mlx/
- Plate 09.12 · Topic: Local LLMs (https://guides-ai.com/topics/local-llm/)
- Published: 19 Oct 2026 · 3 min read
- Source site: guides-ai — https://guides-ai.com/

MLX is Apple's array framework, and `mlx-lm` is the package that runs language models on it. It is macOS and Apple Silicon only: there is no Windows or Linux build, and it will not use an external NVIDIA GPU. On M-series chips it is tuned for the shared CPU/GPU memory and can beat the llama.cpp backend other runners use for some models; measure your own case, since it varies by model.

## 1. Confirm you are on Apple Silicon

```bash
uname -m
```

Prints `arm64` on an M-series Mac; if you see `x86_64`, MLX will not run and you want [Ollama](/glossary/#ollama) or [LM Studio](/glossary/#lm-studio) instead. macOS 15 or newer is needed for the large-model speedups.

## 2. Install mlx-lm

```bash
pip install mlx-lm
```

Installs the CLI entry points (`mlx_lm.generate`, `mlx_lm.chat`, `mlx_lm.server`) and the Python API. Conda users can run `conda install -c conda-forge mlx-lm` instead.

## 3. Generate from a model

```bash
mlx_lm.generate --model mlx-community/Mistral-7B-Instruct-v0.3-4bit --prompt "Explain shared memory in one line"
```

Downloads the model from Hugging Face on first run, then prints a completion. Run `mlx_lm.generate --help` for every flag.

## 4. Chat and serve an OpenAI-style API

```bash
mlx_lm.chat
```

Opens an interactive REPL that keeps context for the session. For an HTTP endpoint instead:

```bash
mlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit
```

Starts a local server on port 8080 whose API is modelled on OpenAI's chat API, with a `/v1/chat/completions` endpoint; the docs call it unsuitable for production. Any OpenAI client works by pointing at `http://localhost:8080/v1`.

## 5. Get quantized models from Hugging Face

The `mlx-community` organisation on the Hub publishes ready-made MLX models, usually 4-bit, which the commands above pull by repo ID or you fetch with the [hf CLI](/guides/hugging-face-cli-download-models/). To convert and quantize a model yourself:

```bash
mlx_lm.convert --model mistralai/Mistral-7B-Instruct-v0.3 -q
```

Writes a 4-bit MLX copy to `./mlx_model`. A 4-bit build needs roughly a quarter of the 16-bit memory; see [how much RAM a model needs](/guides/local-llm-ram-vram-requirements/).

**If it fails:**

- `command not found: mlx_lm.generate`: the pip scripts are not on PATH, or you are in the wrong environment; reinstall in the active venv.
- Generation is very slow or swaps to disk: the model is larger than free memory; pick a 4-bit build, or raise the limit with `sudo sysctl iogpu.wired_limit_mb=N`.
- A model fails to load asking for remote code: add `--trust-remote-code`, but only for repos you trust. Full CLI in the [mlx-lm README](https://github.com/ml-explore/mlx-lm).