File size: 1,382 Bytes
e68f409 4167fa1 5958ef8 a985751 8758474 a985751 5958ef8 a985751 5958ef8 80b6726 5958ef8 a985751 5958ef8 a985751 5958ef8 a985751 5958ef8 a985751 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
---
license: mit
base_model:
- Qwen/Qwen2.5-7B-Instruct-1M
tags:
- rkllm
- rknn
- airockchip
language:
- en
---
# Model Card for Model ID
## Model Details
### Model Description
- **Developed by:** JiangTao
- **Model type:** rknn
- **License:** MIT
## Uses
### Direct Use
```bash
uv venv --python=3.12
source .venv/bin/activate
uv pip install flask Werkzeug
git clone https://github.com/airockchip/rknn-llm
cd rknn-llm/examples/rkllm_server_demo
python3 flask_server.py --rkllm_model_path /path/to/model/Qwen2.5-7B-Instruct-1M_W8A8_RK3588.rkllm --target_platform rk3588
```
### Export Pipeline
```bash
# init rkllm export environment
cd ~/autodl-tmp
git clone https://github.com/airockchip/rknn-llm.git
conda init
conda create -n rkllm python=3.10
conda activate rkllm
cd rknn-llm
pip install rkllm-toolkit/rkllm_toolkit-1.1.4-cp310-cp310-linux_x86_64.whl
# processing
cd examples/DeepSeek-R1-Distill-Qwen-1.5B_Demo/export
# /root/autodl-tmp/models/Qwen2.5-7B-Instruct-1M
python generate_data_quant.py -m /root/autodl-tmp/models/Qwen2.5-7B-Instruct-1M
# modify `modelpath` in `export_rkllm.py`
python export_rkllm.py
```
### Out-of-Scope Use
+ Not supported for rk3576, rk3562
## Environmental Impact
- **Cloud Provider:** AutoDL
## Technical Specifications [optional]
#### Software
https://github.com/airockchip/rknn-llm
## Model Card Contact
[email protected] |