# Describe and Query Images with a Local VLM

> Load Qwen3-VL-8B-Instruct with transformers, pass an image plus a question, and get captions, chart reads and answers on your own GPU.

- Canonical: https://guides-ai.com/guides/describe-images-with-a-vision-model/
- Plate 12.12 · Topic: AI media (https://guides-ai.com/topics/media/)
- Published: 19 Oct 2026 · 3 min read
- Source site: guides-ai — https://guides-ai.com/

A vision-language model (VLM) takes an image and text together and answers in text. Unlike OCR it *understands* the picture: it captions a scene, answers questions, reads a chart, or explains a diagram. This guide uses **Qwen3-VL-8B-Instruct** (Apache-2.0), a small [multimodal](/glossary/#multimodal) model with a clean transformers API — see its [model card](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct). If you only want the characters out of a scan, [run OCR locally](/guides/run-ocr-locally/) instead.

## 1. What a vision-language model does

Give it an image and a prompt; it returns free-form text. That covers captioning, visual question answering, chart and table reading, and document or UI understanding — tasks plain OCR cannot do, because they need reasoning about what is shown, not just transcription of glyphs.

## 2. Install transformers

Qwen3-VL is recent, so the model card says to install transformers from source alongside torch:

```bash
pip install -U "git+https://github.com/huggingface/transformers" accelerate torchvision
```

The Qwen3-VL classes ship only in the latest transformers; the [transformers pipeline quickstart](/guides/transformers-pipeline-quickstart/) covers a plain release install.

## 3. Load the model and processor

```python
import torch
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen3-VL-8B-Instruct", dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")
```

Downloads the weights on first run and places them on your GPU automatically.

## 4. Ask a question about an image

```python
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "photo.jpg"},
        {"type": "text", "text": "What is in this image? Read any text you see."},
    ],
}]

inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

ids = model.generate(**inputs, max_new_tokens=128)
trimmed = [out[len(inp):] for inp, out in zip(inputs.input_ids, ids)]
print(processor.batch_decode(trimmed, skip_special_tokens=True)[0])
```

`apply_chat_template` loads `photo.jpg` (a local path or a URL), builds the prompt, and the trim drops the echoed input so you print only the answer.

## 5. Fit it on your GPU

At 9B parameters in bf16 the model wants a mid-range to large GPU; see [how much VRAM you need](/guides/local-llm-ram-vram-requirements/). No headroom? Qwen ships smaller Qwen3-VL sizes, and a [quantized](/glossary/#quantization) build — the trade-offs are in [choosing a quantization](/guides/choose-llm-quantization/) — cuts memory further.

## Verify it worked

The printed text should describe your actual image. A generic, image-free answer usually means the path was wrong and only the text prompt reached the model.

**If it fails:**

- `KeyError: 'qwen3_vl'` or an unknown-model-type error — transformers is too old; reinstall it from source.
- `torch.cuda.OutOfMemoryError` — load a smaller Qwen3-VL variant or a quantized build.
- Blank or irrelevant output — confirm `photo.jpg` exists in the working directory and is a readable image file.