|
|
--- |
|
|
license: mit |
|
|
base_model: |
|
|
- agentica-org/DeepCoder-14B-Preview |
|
|
--- |
|
|
|
|
|
# OpenVINO quant of [agentica-org/DeepCoder-14B-Preview](https://huggingface.co/agentica-org/DeepCoder-14B-Preview-int4-awq-ov) |
|
|
|
|
|
- Requires 12GB of VRAM (eg. Intel Arc A770 / B580). |
|
|
- Won't fit on 8GB A750 |
|
|
|
|
|
# Performance on an A770 with [OpenArc](https://github.com/SearchSavior/OpenArc) |
|
|
|
|
|
``` |
|
|
=== Streaming Performance === |
|
|
Total generation time: 65.078 seconds |
|
|
Prompt evaluation: 1376 tokens in 0.841 seconds (1636.58 T/s) |
|
|
Response generation: 982 tokens in (15.09 T/s) |
|
|
``` |
