jerryzh168 commited on
Commit
87bad56
·
verified ·
1 Parent(s): dd89b44

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -3
README.md CHANGED
@@ -23,7 +23,11 @@ pipeline_tag: text-generation
23
  Install vllm nightly to get some recent changes:
24
  ```
25
  pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
26
- pip install torchao
 
 
 
 
27
  ```
28
 
29
  ## Code Example
@@ -72,11 +76,15 @@ vllm serve pytorch/Phi-4-mini-instruct-FP8 --tokenizer microsoft/Phi-4-mini-inst
72
  Install the required packages:
73
  ```Shell
74
  pip install git+https://github.com/huggingface/transformers@main
75
- pip install torchao
76
- pip install torch
77
  pip install accelerate
78
  ```
79
 
 
 
 
 
 
80
  Example:
81
  ```Py
82
  import torch
 
23
  Install vllm nightly to get some recent changes:
24
  ```
25
  pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
26
+ pip install --pre torchao torch --index-url https://download.pytorch.org/whl/nightly/cu126
27
+ ```
28
+ To use fbgemm kernels:
29
+ ```Shell
30
+ pip install --pre fbgemm-gpu-genai --index-url https://download.pytorch.org/whl/nightly/cu126
31
  ```
32
 
33
  ## Code Example
 
76
  Install the required packages:
77
  ```Shell
78
  pip install git+https://github.com/huggingface/transformers@main
79
+ pip install --pre torchao torch --index-url https://download.pytorch.org/whl/nightly/cu126
 
80
  pip install accelerate
81
  ```
82
 
83
+ To use fbgemm kernels:
84
+ ```Shell
85
+ pip install --pre fbgemm-gpu-genai --index-url https://download.pytorch.org/whl/nightly/cu126
86
+ ```
87
+
88
  Example:
89
  ```Py
90
  import torch