---
base_model:
- Qwen/Qwen2.5-VL-7B-Instruct
---

This is the [Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) model, converted to OpenVINO, with int4 weights for the language model, int8 weights for the other models.

## Download Model

To download the model, run `pip install huggingface-hub[cli]` and then:
```
huggingface-cli download helenai/Qwen2.5-VL-7B-Instruct-ov-int4 --local-dir Qwen2.5-VL-7B-Instruct-ov-int4
```

## Run inference with OpenVINO GenAI

Use OpenVINO GenAI to run inference on this model. This model works with OpenVINO GenAI 2025.2 and later.

- Install OpenVINO GenAI and pillow:

```
pip install --upgrade openvino-genai pillow
```

- Download a test image: `curl -O "https://storage.openvinotoolkit.org/test_data/images/dog.jpg"`
- Run inference:

```python
import numpy as np
import openvino as ov
import openvino_genai
from PIL import Image

# Choose GPU instead of CPU in the line below to run the model on Intel integrated or discrete GPU
pipe = openvino_genai.VLMPipeline("Qwen2.5-VL-7B-Instruct-ov-int4", "CPU")

image = Image.open("dog.jpg")
# optional: resizing to a smaller size (depending on image and prompt) is often useful to speed up inference. 
image = image.resize((128, 128))

image_data = np.array(image.getdata()).reshape(1, image.size[1], image.size[0], 3).astype(np.uint8)
image_data = ov.Tensor(image_data)  

prompt = "Can you describe the image?"
result = pipe.generate(prompt, image=image_data, max_new_tokens=100)
print(result.texts[0])
```

See [OpenVINO GenAI repository](https://github.com/openvinotoolkit/openvino.genai?tab=readme-ov-file#performing-visual-language-text-generation)

## Model export properties

Model export command:

```
optimum-cli export openvino -m Qwen/Qwen2.5-VL-7B-Instruct --weight-format int4 Qwen2.5-VL-7B-Instruct-ov-int4
```

### Framework versions

```
openvino         : 2025.2.0-19140-c01cd93e24d-releases/2025/2
nncf             : 2.17.0.dev0+c6296072
optimum_intel    : 1.26.0.dev0+0e2ccef
optimum          : 1.27.0
pytorch          : 2.7.0+cpu
transformers     : 4.51.3
```

### LLM export properties

```
all_layers               : False
awq                      : False
backup_mode              : int8_asym
compression_format       : dequantize
gptq                     : False
group_size               : 128
ignored_scope            : []
lora_correction          : False
mode                     : int4_asym
ratio                    : 1.0
scale_estimation         : False
sensitivity_metric       : weight_quantization_error
```