readme.md

介绍

SHA256 5b2d8a0ad40adbd4c02c299f2a46393a65d8e82a85ce2ec06bca6df72301ab8b Qwen2.5-7B-Instruct-1M_W8A8_RK3588.rkllm

模型

来源：https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

处理步骤

# 初始化环境
cd ~/autodl-tmp 
git clone https://github.com/airockchip/rknn-llm.git
conda init
conda create -n rkllm python=3.10
conda activate rkllm
cd rknn-llm
pip install rkllm-toolkit/rkllm_toolkit-1.1.4-cp310-cp310-linux_x86_64.whl

# 处理
cd examples/DeepSeek-R1-Distill-Qwen-1.5B_Demo/export
# /root/autodl-tmp/models/Qwen2.5-7B-Instruct-1M
python generate_data_quant.py -m ~/models/Qwen2.5-7B-Instruct-1M
# modify `modelpath` in `export_rkllm.py`
python export_rkllm.py

条件限制

• RK3588
• NPU 内核数量： 3

---

Introduction

SHA256 5b2d8a0ad40adbd4c02c299f2a46393a65d8e82a85ce2ec06bca6df72301ab8b Qwen2.5-7B-Instruct-1M_W8A8_RK3588.rkllm

Model

Source: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Processing Steps

# Initialize RKLLM export environment
cd ~/autodl-tmp 
git clone https://github.com/airockchip/rknn-llm.git
conda init
conda create -n rkllm python=3.10
conda activate rkllm
cd rknn-llm
pip install rkllm-toolkit/rkllm_toolkit-1.1.4-cp310-cp310-linux_x86_64.whl

# Processing
cd examples/DeepSeek-R1-Distill-Qwen-1.5B_Demo/export
# /root/autodl-tmp/models/Qwen2.5-7B-Instruct-1M
python generate_data_quant.py -m ~/models/Qwen2.5-7B-Instruct-1M
# Modify `modelpath` in `export_rkllm.py`
python export_rkllm.py

Requirements & Limitations

• RK3588
• NPU core count: 3