experimental 4-bit dynamic quant using unsloth, these model will be used for inference and RoPE fine-tuning.
huggingkot
huggingkot
AI & ML interests
None yet
Recent Activity
updated
a collection
6 days ago
bnb-4bit-experimental
updated
a model
6 days ago
huggingkot/BlackSheep-24B-bnb-4bit
published
a model
6 days ago
huggingkot/BlackSheep-24B-bnb-4bit
Organizations
None yet
for model less than 8b
-
huggingkot/Dolphin3.0-Qwen2.5-3b-q4f16_1-MLC
Text Generation • Updated • 1 -
huggingkot/Dolphin3.0-Qwen2.5-1.5B-q4f16_1-MLC
Text Generation • Updated • 4 -
huggingkot/Dolphin3.0-Qwen2.5-0.5B-q4f16_1-MLC
Text Generation • Updated • 3 -
huggingkot/Dolphin3.0-Llama3.1-8B-q4f16_1-MLC
Text Generation • Updated • 6
https://llm.mlc.ai/docs
-
huggingkot/DeepScaleR-1.5B-Preview-q4f16_1-MLC
Text Generation • Updated • 1 -
huggingkot/Llama-3-8B-Instruct-abliterated-v2-q4f16_1-MLC
Text Generation • Updated • 3 -
huggingkot/aya-expanse-8b-q4f16_1-MLC
Text Generation • Updated -
huggingkot/LLAMA-3_8B_Unaligned_BETA-q4f16_1-MLC
Text Generation • Updated • 4
bookmark
-
huggingkot/Qwen2.5-Coder-14B-Instruct-q4f16_1-MLC
Text Generation • Updated • 1 -
huggingkot/Qwen2.5-Coder-7B-Instruct-q4f16_1-MLC
Text Generation • Updated • 12 -
huggingkot/Qwen2.5-Coder-3B-Instruct-q4f16_1-MLC
Text Generation • Updated • 7 -
huggingkot/Qwen2.5-Coder-1.5B-Instruct-q4f16_1-MLC
Text Generation • Updated • 6
experimental 4-bit dynamic quant using unsloth, these model will be used for inference and RoPE fine-tuning.
for model less than 8b
-
huggingkot/Dolphin3.0-Qwen2.5-3b-q4f16_1-MLC
Text Generation • Updated • 1 -
huggingkot/Dolphin3.0-Qwen2.5-1.5B-q4f16_1-MLC
Text Generation • Updated • 4 -
huggingkot/Dolphin3.0-Qwen2.5-0.5B-q4f16_1-MLC
Text Generation • Updated • 3 -
huggingkot/Dolphin3.0-Llama3.1-8B-q4f16_1-MLC
Text Generation • Updated • 6
-
huggingkot/Qwen2.5-Coder-14B-Instruct-q4f16_1-MLC
Text Generation • Updated • 1 -
huggingkot/Qwen2.5-Coder-7B-Instruct-q4f16_1-MLC
Text Generation • Updated • 12 -
huggingkot/Qwen2.5-Coder-3B-Instruct-q4f16_1-MLC
Text Generation • Updated • 7 -
huggingkot/Qwen2.5-Coder-1.5B-Instruct-q4f16_1-MLC
Text Generation • Updated • 6
https://llm.mlc.ai/docs
-
huggingkot/DeepScaleR-1.5B-Preview-q4f16_1-MLC
Text Generation • Updated • 1 -
huggingkot/Llama-3-8B-Instruct-abliterated-v2-q4f16_1-MLC
Text Generation • Updated • 3 -
huggingkot/aya-expanse-8b-q4f16_1-MLC
Text Generation • Updated -
huggingkot/LLAMA-3_8B_Unaligned_BETA-q4f16_1-MLC
Text Generation • Updated • 4
bookmark