Update README.md
Browse files
README.md
CHANGED
@@ -42,7 +42,6 @@ Once your server is started, you can query the model using the OpenAI API:
|
|
42 |
```python
|
43 |
from openai import OpenAI
|
44 |
|
45 |
-
# Modify OpenAI's API key and API base to use vLLM's API server.
|
46 |
openai_api_key = "EMPTY"
|
47 |
openai_api_base = "http://localhost:8000/v1"
|
48 |
client = OpenAI(
|
@@ -215,7 +214,7 @@ The model was evaluated on the test split of trl-lib/tldr using the Neural Magic
|
|
215 |
One can reproduce these results by using the following command:
|
216 |
|
217 |
```bash
|
218 |
-
lm_eval --model vllm --model_args "pretrained=RedHatAI/Sparse-Llama-3.1-8B-tldr-2of4,dtype=auto,add_bos_token" --batch-size auto --tasks tldr
|
219 |
```
|
220 |
|
221 |
<table>
|
|
|
42 |
```python
|
43 |
from openai import OpenAI
|
44 |
|
|
|
45 |
openai_api_key = "EMPTY"
|
46 |
openai_api_base = "http://localhost:8000/v1"
|
47 |
client = OpenAI(
|
|
|
214 |
One can reproduce these results by using the following command:
|
215 |
|
216 |
```bash
|
217 |
+
lm_eval --model vllm --model_args "pretrained=RedHatAI/Sparse-Llama-3.1-8B-tldr-2of4,dtype=auto,add_bos_token=True" --batch-size auto --tasks tldr
|
218 |
```
|
219 |
|
220 |
<table>
|