# Transformers pipeline(): Run a Hugging Face Model Locally

> Install PyTorch and transformers, chat with a small instruct model via pipeline(), set device_map, dtype and max_new_tokens, then classify an image.

- Canonical: https://guides-ai.com/guides/transformers-pipeline-quickstart/
- Plate 16.06 · Topic: Hugging Face (https://guides-ai.com/topics/huggingface/)
- Published: 12 Oct 2026 · 3 min read
- Source site: guides-ai — https://guides-ai.com/

`pipeline()` turns a Hub model id into a result on your own machine: it downloads the weights, loads the tokenizer or preprocessor, picks a device and runs the task. The example model is [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct): 1.5B parameters, a 3.1 GB download, no licence gate.

## 1. Install

Linux with an NVIDIA GPU (check with `nvidia-smi`):

```bash
pip install -U "transformers[torch]"
```

CPU only (smaller download, no CUDA libraries):

```bash
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -U transformers accelerate
```

Commands from the [transformers install page](https://huggingface.co/docs/transformers/main/en/installation); the `[torch]` extra includes `accelerate`, which `device_map="auto"` needs. On Windows the PyPI `torch` wheel is CPU-only, so install the CUDA build from the [pytorch.org selector](https://pytorch.org/get-started/locally/) first. Sizing: [RAM and VRAM per model](/guides/local-llm-ram-vram-requirements/).

## 2. Chat with an instruct model

```python
import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="Qwen/Qwen2.5-1.5B-Instruct",
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a concise assistant."},
    {"role": "user", "content": "Explain what a tokenizer does in two sentences."},
]

out = pipe(messages, max_new_tokens=128)
print(out[0]["generated_text"][-1]["content"])
```

The first run downloads the weights. A list of `role`/`content` dicts gets the model's chat template applied; `generated_text` returns the whole conversation, so `[-1]` is the new reply.

## 3. Length, device, precision

`max_new_tokens` caps the reply length. `device_map="auto"` puts layers on the GPU first and spills to CPU if VRAM runs out; use `device="cpu"` or `device=0` instead for a single fixed device, never both. `dtype` sets the load precision; the default `"auto"` keeps the checkpoint's own. Older tutorials write `torch_dtype=`, which current transformers flags as deprecated.

## 4. Same API, other modalities

```python
from transformers import pipeline

classifier = pipeline("image-classification", model="google/vit-base-patch16-224")
preds = classifier("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg")
for p in preds[:3]:
    print(f"{p['score']:.2f}  {p['label']}")
```

Prints the top three ImageNet labels with scores (`0.43  lynx, catamount` first). Only the task string and model change; `automatic-speech-recognition` works the same way but needs `ffmpeg` installed to decode audio files, see [transcribe audio locally with Whisper](/guides/whisper-transcribe-audio-locally/).

## 5. Where the weights went

Everything lands in the Hugging Face cache: `~/.cache/huggingface/hub/models--Qwen--Qwen2.5-1.5B-Instruct` (on Windows `C:\Users\<you>\.cache\huggingface\hub\…`). Later runs reuse it. [Move or clean the cache](/guides/hugging-face-cache-location/) if your system drive is small.

## If it fails

- **`torch.OutOfMemoryError: CUDA out of memory. Tried to allocate …`** Weights plus activations exceed VRAM. Use a smaller model or keep `device_map="auto"` so layers spill to CPU; a [quantized GGUF](/guides/choose-llm-quantization/) in llama.cpp needs far less memory.
- **`OSError: You are trying to access a gated repo.` … `401 Client Error`** Open the model page, accept the licence, then run `hf auth login` with a read token.
- **`ValueError: The repository … contains custom code which must be executed to correctly load the model.`** Read the repo's Python files first; only then pass `trust_remote_code=True` to `pipeline()`, because it runs that code on your machine.

---

Next: [load, stream and filter a dataset](/guides/hugging-face-datasets-load-stream-filter/) · [call a model via Inference Providers](/guides/hugging-face-inference-providers-first-request/) · [deploy it as a Gradio Space](/guides/deploy-gradio-app-hugging-face-spaces/) · [download models with the hf CLI](/guides/hugging-face-cli-download-models/) · [llama.cpp quickstart](/guides/llama-cpp-quickstart/) · [serve a model with vLLM](/guides/vllm-serve-openai-compatible/).