|
|
--- |
|
|
library_name: transformers |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
- fr |
|
|
- es |
|
|
- it |
|
|
- pt |
|
|
- zh |
|
|
- ar |
|
|
- ru |
|
|
base_model: |
|
|
- HuggingFaceTB/SmolLM3-3B-Base |
|
|
tags: |
|
|
- openvino |
|
|
- int4 |
|
|
- quantization |
|
|
- edge-deployment |
|
|
- optimization |
|
|
- smollm3 |
|
|
inference: false |
|
|
--- |
|
|
|
|
|
# SmolLM3 INT4 OpenVINO |
|
|
|
|
|
## 🚀 Optimized for Edge Deployment |
|
|
|
|
|
This is an INT4 quantized version of [SmolLM3-3B](https://huggingface.co/HuggingFaceTB/SmolLM3-3B) using OpenVINO, designed for efficient inference on edge devices and CPUs. |
|
|
|
|
|
## Model Overview |
|
|
|
|
|
- **Base Model:** SmolLM3-3B (3B parameters) |
|
|
- **Quantization:** INT4 via OpenVINO |
|
|
- **Size Reduction:** ~75% smaller than original |
|
|
- **Target Hardware:** CPUs, Intel GPUs, NPUs |
|
|
- **Use Cases:** Local inference, edge deployment, resource-constrained environments |
|
|
|
|
|
## 🔧 Technical Details |
|
|
|
|
|
### Quantization Process |
|
|
```python |
|
|
# Quantized using OpenVINO NNCF |
|
|
# INT4 symmetric quantization |
