tokyotech-llm
/

Llama-3.1-Swallow-70B-Instruct-v0.3

Text Generation

text-generation-inference

Model card Files Files and versions

Taishi-N324 commited on Mar 9

Commit

e182ae8

·

verified ·

1 Parent(s): 982c4f1

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -172,7 +172,7 @@ model_name = "tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.3"
 tokenizer = AutoTokenizer.from_pretrained(model_name)
 llm = LLM(
     model=model_name,
-    tensor_parallel_size=1,
 )
 sampling_params = SamplingParams(

 tokenizer = AutoTokenizer.from_pretrained(model_name)
 llm = LLM(
     model=model_name,
+    tensor_parallel_size=4,
 )
 sampling_params = SamplingParams(