alexmarques commited on
Commit
eb855d8
·
verified ·
1 Parent(s): f8d85de

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -2
README.md CHANGED
@@ -42,7 +42,6 @@ Once your server is started, you can query the model using the OpenAI API:
42
  ```python
43
  from openai import OpenAI
44
 
45
- # Modify OpenAI's API key and API base to use vLLM's API server.
46
  openai_api_key = "EMPTY"
47
  openai_api_base = "http://localhost:8000/v1"
48
  client = OpenAI(
@@ -215,7 +214,7 @@ The model was evaluated on the test split of trl-lib/tldr using the Neural Magic
215
  One can reproduce these results by using the following command:
216
 
217
  ```bash
218
- lm_eval --model vllm --model_args "pretrained=RedHatAI/Sparse-Llama-3.1-8B-tldr-2of4,dtype=auto,add_bos_token" --batch-size auto --tasks tldr
219
  ```
220
 
221
  <table>
 
42
  ```python
43
  from openai import OpenAI
44
 
 
45
  openai_api_key = "EMPTY"
46
  openai_api_base = "http://localhost:8000/v1"
47
  client = OpenAI(
 
214
  One can reproduce these results by using the following command:
215
 
216
  ```bash
217
+ lm_eval --model vllm --model_args "pretrained=RedHatAI/Sparse-Llama-3.1-8B-tldr-2of4,dtype=auto,add_bos_token=True" --batch-size auto --tasks tldr
218
  ```
219
 
220
  <table>