Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
VoxCPM
Log In
Sign Up
ModelCloud
/
TinyLlama-1.1B-Chat-v1.0-dynamic-GPTQ-2024-8-3
like
0
Follow
ModelCloud.AI
66
Safetensors
llama
4-bit precision
gptq
Model card
Files
Files and versions
xet
Community
zx-modelcloud
commited on
Aug 2, 2024
Commit
0935d64
·
verified
·
1 Parent(s):
9a0f02a
Create README.md
Browse files
Files changed (1)
hide
show
README.md
+5
-0
README.md
ADDED
Viewed
@@ -0,0 +1,5 @@
1
+
dynamic = {
2
+
# `.*\.` matches the layers_node prefix
3
+
r".*\.(?:1[0-5])\..*": {"bits": 8,}, # match layer 10-15
4
+
r".*\.(?:1[6-9]|20|21)\..*": {"bits": 8, "group_size": 64,}, # match layer 16-21
5
+
}