Hugging Face's logo

onnxruntime
/
DeepSeek-R1-Distill-ONNX

Text Generation
Transformers
ONNX
ONNX
ONNXRuntime
Model card Files
xet
Community
3
kvaishnavi commited on
Commit
492a4b6
·
verified ·
1 Parent(s): b8ffafb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -6
README.md CHANGED
@@ -27,12 +27,12 @@ The ONNX models are tested on:
27
  ONNX enables you to run your models on-device across CPU, GPU, NPU. With ONNX you can run your models on any machine across all silica Qualcomm, AMD, Intel, Nvidia. See table below for some key benchmarks for Windows GPU and CPU devices.
28
  | **Model** | **Precisionl** | **Device Type** | **Execution Provider** | **Device** | **Token Generation Throughput** | **Speed up vs base model**|
29
  | :------------: | :------------: | :------------: | :------------: | :------------: | :------------: | :------------:|
30
- |deepseek-ai_DeepSeek-R1-Distill-Qwen-1.5B| ONNX | fp16 | GPU CUDA | RTX 4090 | 197.195 | 4X |
31
- |deepseek-ai_DeepSeek-R1-Distill-Qwen-1.5B| ONNX | Int4 | GPU CUDA | RTX 4090 | 313.32 | 6.3X |
32
- |deepseek-ai_DeepSeek-R1-Distill-Qwen-7B| ONNX | fp16 | GPU CUDA | RTX 4090 | 57.316 | 1.3X |
33
- |deepseek-ai_DeepSeek-R1-Distill-Qwen-7B| ONNX | Int4 | GPU CUDA | RTX 4090 | 161.00 | 3.7X |
34
- |deepseek-ai_DeepSeek-R1-Distill-Qwen-7B| ONNX | Int4/bfloat16 | CPU | CPU Intel i9 | 3.184 | 20X |
35
- |deepseek-ai_DeepSeek-R1-Distill-Qwen-1.5B| ONNX | Int4 | CPU | CPU Intel i9 | 11.749 | 1.4x |
36
 
37
  CPU build specs:
38
 
 
27
  ONNX enables you to run your models on-device across CPU, GPU, NPU. With ONNX you can run your models on any machine across all silica Qualcomm, AMD, Intel, Nvidia. See table below for some key benchmarks for Windows GPU and CPU devices.
28
  | **Model** | **Precisionl** | **Device Type** | **Execution Provider** | **Device** | **Token Generation Throughput** | **Speed up vs base model**|
29
  | :------------: | :------------: | :------------: | :------------: | :------------: | :------------: | :------------:|
30
+ | deepseek-ai_DeepSeek-R1-Distill-Qwen-1.5B | ONNX | fp16 | CUDA | RTX 4090 | 197.195 | 4X |
31
+ | deepseek-ai_DeepSeek-R1-Distill-Qwen-1.5B | ONNX | int4 | CUDA | RTX 4090 | 313.32 | 6.3X |
32
+ | deepseek-ai_DeepSeek-R1-Distill-Qwen-1.5B | ONNX | int4 | CPU | Intel i9 | 11.749 | 1.4x |
33
+ | deepseek-ai_DeepSeek-R1-Distill-Qwen-7B | ONNX | fp16 | CUDA | RTX 4090 | 57.316 | 1.3X |
34
+ | deepseek-ai_DeepSeek-R1-Distill-Qwen-7B | ONNX | int4 | CUDA | RTX 4090 | 161.00 | 3.7X |
35
+ | deepseek-ai_DeepSeek-R1-Distill-Qwen-7B | ONNX | int4 | CPU | Intel i9 | 3.184 | 20X |
36
 
37
  CPU build specs:
38