File size: 544 Bytes
8290dc1 d47bf0f 8290dc1 8e4a660 8290dc1 8e4a660 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
---
license: mit
base_model:
- agentica-org/DeepCoder-14B-Preview
---
# OpenVINO quant of [agentica-org/DeepCoder-14B-Preview](https://huggingface.co/agentica-org/DeepCoder-14B-Preview-int4-awq-ov)
- Requires 12GB of VRAM (eg. Intel Arc A770 / B580).
- Won't fit on 8GB A750
# Performance on an A770 with [OpenArc](https://github.com/SearchSavior/OpenArc)
```
=== Streaming Performance ===
Total generation time: 65.078 seconds
Prompt evaluation: 1376 tokens in 0.841 seconds (1636.58 T/s)
Response generation: 982 tokens in (15.09 T/s)
``` |