|
|
--- |
|
|
license: mit |
|
|
base_model: |
|
|
- Qwen/Qwen2.5-7B-Instruct-1M |
|
|
tags: |
|
|
- rkllm |
|
|
- rknn |
|
|
- airockchip |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
# Model Card for Model ID |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
- **Developed by:** JiangTao |
|
|
- **Model type:** rknn |
|
|
- **License:** MIT |
|
|
|
|
|
## Uses |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
```bash |
|
|
uv venv --python=3.12 |
|
|
source .venv/bin/activate |
|
|
uv pip install flask Werkzeug |
|
|
git clone https://github.com/airockchip/rknn-llm |
|
|
cd rknn-llm/examples/rkllm_server_demo |
|
|
python3 flask_server.py --rkllm_model_path /path/to/model/Qwen2.5-7B-Instruct-1M_W8A8_RK3588.rkllm --target_platform rk3588 |
|
|
``` |
|
|
|
|
|
### Export Pipeline |
|
|
|
|
|
```bash |
|
|
# init rkllm export environment |
|
|
cd ~/autodl-tmp |
|
|
git clone https://github.com/airockchip/rknn-llm.git |
|
|
conda init |
|
|
conda create -n rkllm python=3.10 |
|
|
conda activate rkllm |
|
|
cd rknn-llm |
|
|
pip install rkllm-toolkit/rkllm_toolkit-1.1.4-cp310-cp310-linux_x86_64.whl |
|
|
|
|
|
# processing |
|
|
cd examples/DeepSeek-R1-Distill-Qwen-1.5B_Demo/export |
|
|
# /root/autodl-tmp/models/Qwen2.5-7B-Instruct-1M |
|
|
python generate_data_quant.py -m /root/autodl-tmp/models/Qwen2.5-7B-Instruct-1M |
|
|
# modify `modelpath` in `export_rkllm.py` |
|
|
python export_rkllm.py |
|
|
``` |
|
|
|
|
|
### Out-of-Scope Use |
|
|
|
|
|
+ Not supported for rk3576, rk3562 |
|
|
|
|
|
## Environmental Impact |
|
|
|
|
|
- **Cloud Provider:** AutoDL |
|
|
|
|
|
## Technical Specifications [optional] |
|
|
|
|
|
#### Software |
|
|
|
|
|
https://github.com/airockchip/rknn-llm |
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
[email protected] |
