|
|
---
|
|
|
base_model: Qwen/Qwen2.5-7B-Instruct
|
|
|
language:
|
|
|
- zho
|
|
|
- eng
|
|
|
- fra
|
|
|
- spa
|
|
|
- por
|
|
|
- deu
|
|
|
- ita
|
|
|
- rus
|
|
|
- jpn
|
|
|
- kor
|
|
|
- vie
|
|
|
- tha
|
|
|
- ara
|
|
|
library_name: transformers
|
|
|
license: apache-2.0
|
|
|
license_link: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct/blob/main/LICENSE
|
|
|
pipeline_tag: text-generation
|
|
|
tags:
|
|
|
- chat
|
|
|
- openvino
|
|
|
- nncf
|
|
|
- 8-bit
|
|
|
base_model_relation: quantized
|
|
|
---
|
|
|
|
|
|
This model is a quantized version of [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) and is converted to the OpenVINO format. This model was obtained via the [nncf-quantization](https://huggingface.co/spaces/echarlaix/nncf-quantization) space with [optimum-intel](https://github.com/huggingface/optimum-intel).
|
|
|
First make sure you have `optimum-intel` installed:
|
|
|
```bash
|
|
|
pip install optimum[openvino]
|
|
|
```
|
|
|
To load your model you can do as follows:
|
|
|
```python
|
|
|
from optimum.intel import OVModelForCausalLM
|
|
|
model_id = "AIFunOver/Qwen2.5-7B-Instruct-openvino-8bit"
|
|
|
model = OVModelForCausalLM.from_pretrained(model_id)
|
|
|
```
|
|
|
|
