pipeline() turns a Hub model id into a result on your own machine: it downloads the weights, loads the tokenizer or preprocessor, picks a device and runs the task. The example model is Qwen/Qwen2.5-1.5B-Instruct: 1.5B parameters, a 3.1 GB download, no licence gate.
1. Install
Linux with an NVIDIA GPU (check with nvidia-smi):
pip install -U "transformers[torch]"
CPU only (smaller download, no CUDA libraries):
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -U transformers accelerate
Commands from the transformers install page; the [torch] extra includes accelerate, which device_map="auto" needs. On Windows the PyPI torch wheel is CPU-only, so install the CUDA build from the pytorch.org selector first. Sizing: RAM and VRAM per model.
2. Chat with an instruct model
import torch
from transformers import pipeline
pipe = pipeline(
"text-generation",
model="Qwen/Qwen2.5-1.5B-Instruct",
dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a concise assistant."},
{"role": "user", "content": "Explain what a tokenizer does in two sentences."},
]
out = pipe(messages, max_new_tokens=128)
print(out[0]["generated_text"][-1]["content"])
The first run downloads the weights. A list of role/content dicts gets the model’s chat template applied; generated_text returns the whole conversation, so [-1] is the new reply.
3. Length, device, precision
max_new_tokens caps the reply length. device_map="auto" puts layers on the GPU first and spills to CPU if VRAM runs out; use device="cpu" or device=0 instead for a single fixed device, never both. dtype sets the load precision; the default "auto" keeps the checkpoint’s own. Older tutorials write torch_dtype=, which current transformers flags as deprecated.
4. Same API, other modalities
from transformers import pipeline
classifier = pipeline("image-classification", model="google/vit-base-patch16-224")
preds = classifier("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg")
for p in preds[:3]:
print(f"{p['score']:.2f} {p['label']}")
Prints the top three ImageNet labels with scores (0.43 lynx, catamount first). Only the task string and model change; automatic-speech-recognition works the same way but needs ffmpeg installed to decode audio files, see transcribe audio locally with Whisper.
5. Where the weights went
Everything lands in the Hugging Face cache: ~/.cache/huggingface/hub/models--Qwen--Qwen2.5-1.5B-Instruct (on Windows C:\Users\<you>\.cache\huggingface\hub\…). Later runs reuse it. Move or clean the cache if your system drive is small.
If it fails
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate …Weights plus activations exceed VRAM. Use a smaller model or keepdevice_map="auto"so layers spill to CPU; a quantized GGUF in llama.cpp needs far less memory.OSError: You are trying to access a gated repo.…401 Client ErrorOpen the model page, accept the licence, then runhf auth loginwith a read token.ValueError: The repository … contains custom code which must be executed to correctly load the model.Read the repo’s Python files first; only then passtrust_remote_code=Truetopipeline(), because it runs that code on your machine.
Next: load, stream and filter a dataset · call a model via Inference Providers · deploy it as a Gradio Space · download models with the hf CLI · llama.cpp quickstart · serve a model with vLLM.