pytorch
/

Qwen3-32B-FP8

Text Generation

text-generation-inference

Model card Files Files and versions

jerryzh168 commited on about 19 hours ago

Commit

051bf7e

·

verified ·

1 Parent(s): eed4706

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -181,7 +181,7 @@ tokenizer = AutoTokenizer.from_pretrained(model_name)
 model = AutoModelForCausalLM.from_pretrained(
     model_name,
     torch_dtype="auto",
-    device_map="auto"
 )
 torch.cuda.reset_peak_memory_stats()

 model = AutoModelForCausalLM.from_pretrained(
     model_name,
     torch_dtype="auto",
+    device_map="cuda:0"
 )
 torch.cuda.reset_peak_memory_stats()