---
tags:
- deepsparse
---
## Usage

```python
from deepsparse import TextGeneration

prompt = "How to get in a good university?"
formatted_prompt =  f"<s> [|User|]\n{prompt}</s>[|Assistant|]\n"

model = TextGeneration(model="deployment")

print(model(formatted_prompt, max_new_tokens=500).generations[0].text)
"""
Hi there! I'm an AI assistant and I can help you with any questions you have. Here's how you can get into a good university:
1. Choose a university that you are interested in.
2. Read the university's website and learn more about their programs and courses.
3. Reach out to the university's alumni or staff members and ask for advice or guidance.
4. Attend a campus tour or visit day to get a better understanding of the university's culture and atmosphere.
5. Finally, don't wait to apply for the university of your choice.
I hope this helps you get into a good university! Let me know if you have any other questions or need any more information.
"""
```
## One-shot and Export

```bash
git clone https://github.com/neuralmagic/sparseml
pip install -e "sparseml[transformers]"
python sparseml/src/sparseml/transformers/sparsification/obcq/obcq.py TinyLlama/TinyLlama-1.1B-Chat-v0.4 open_platypus --recipe recipe.yaml --save True
python sparseml/src/sparseml/transformers/sparsification/obcq/export.py --task text-generation --model_path obcq_deployment 
cp deployment/model.onnx deployment/model-orig.onnx
python onnx_kv_inject.py --input-file deployment/model-orig.onnx --output-file deployment/model.onnx
```