|
|
--- |
|
|
library_name: transformers |
|
|
tags: |
|
|
- Assistant |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- teknium/OpenHermes-2.5 |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- meta-llama/Llama-3.2-3B |
|
|
--- |
|
|
|
|
|
# Llama-3.2-3B LoRA Fine-tune on OpenHermes |
|
|
|
|
|
## 📖 Overview |
|
|
|
|
|
This model is a **LoRA fine-tuned version of `meta-llama/Llama-3.2-3B`** on the [OpenHermes dataset](https://huggingface.co/datasets/teknium/OpenHermes-2.5). |
|
|
The goal of this run was to adapt Llama-3.2-3B for improved instruction-following using a high-quality, multi-domain SFT dataset. |
|
|
|
|
|
Fine-tuning was performed with **parameter-efficient fine-tuning (PEFT)** using **LoRA adapters**. Only \~0.75% of model parameters were trained, keeping compute and memory usage efficient while still yielding strong gains. |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚙️ Training Configuration |
|
|
|
|
|
**Base Model:** `meta-llama/Llama-3.2-3B` |
|
|
**Method:** QLoRA (LoRA rank 16, α=32, dropout=0.05) |
|
|
**Trainable Parameters:** 24.3M / 3.24B (\~0.75%) |
|
|
|
|
|
**Training Arguments:** |
|
|
|
|
|
```python |
|
|
training_args = TrainingArguments( |
|
|
output_dir="./llama_finetune_lora", |
|
|
per_device_train_batch_size=2, |
|
|
gradient_accumulation_steps=8, |
|
|
learning_rate=2e-4, |
|
|
num_train_epochs=1, |
|
|
lr_scheduler_type="cosine", |
|
|
warmup_ratio=0.03, |
|
|
weight_decay=0.01, |
|
|
logging_steps=200, |
|
|
|
|
|
evaluation_strategy="steps", |
|
|
eval_steps=200, |
|
|
save_strategy="steps", |
|
|
save_steps=1000, |
|
|
save_total_limit=2, |
|
|
load_best_model_at_end=True, |
|
|
metric_for_best_model="eval_loss", |
|
|
greater_is_better=False, |
|
|
|
|
|
bf16=True, # A100 support |
|
|
fp16=False, |
|
|
gradient_checkpointing=True, |
|
|
torch_compile=False, |
|
|
report_to="none", |
|
|
seed=42 |
|
|
) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📊 Training Metrics |
|
|
|
|
|
Run stopped at **2000 steps** (\~4.5h on A100). Loss steadily improved and validation stabilized around 0.20. |
|
|
|
|
|
| Step | Training Loss | Validation Loss | |
|
|
| ---- | ------------- | --------------- | |
|
|
| 200 | 1.2781 | 0.2202 | |
|
|
| 400 | 0.2167 | 0.2134 | |
|
|
| 600 | 0.2139 | 0.2098 | |
|
|
| 800 | 0.2120 | 0.2072 | |
|
|
| 1000 | 0.2085 | 0.2057 | |
|
|
| 1200 | 0.1996 | 0.2043 | |
|
|
| 1400 | 0.2056 | 0.2034 | |
|
|
| 1600 | 0.2016 | 0.2023 | |
|
|
| 1800 | 0.2000 | 0.2012 | |
|
|
| 2000 | 0.2027 | 0.2005 | |
|
|
|
|
|
📉 **Validation loss converged near \~0.20**, indicating effective adaptation. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
from peft import PeftModel |
|
|
|
|
|
base = "meta-llama/Llama-3.2-3B" |
|
|
adapter = "kunjcr2/llama3-3b-lora-openhermes" # replace with your Hub repo |
|
|
|
|
|
# Load base + adapter |
|
|
tok = AutoTokenizer.from_pretrained(adapter) |
|
|
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto") |
|
|
model = PeftModel.from_pretrained(model, adapter) |
|
|
|
|
|
# Generate |
|
|
prompt = "Explain the concept of binary search trees." |
|
|
inputs = tok(prompt, return_tensors="pt").to(model.device) |
|
|
outputs = model.generate(**inputs, max_new_tokens=256) |
|
|
print(tok.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📌 Notes |
|
|
|
|
|
* Training was run with **bf16 + gradient checkpointing** on A100 (40GB). |
|
|
* Only adapters are uploaded (small size). Use together with the base model. |
|
|
* Repo includes: `adapter_model.safetensors`, `adapter_config.json`, tokenizer files, and this README. |
|
|
* Training stopped early at **2000 steps (\~17% of planned)** due to good convergence. |
|
|
|
|
|
--- |
|
|
|
|
|
✨ If you like this model, feel free to try it out and extend training. Future runs could include more steps, preference tuning (DPO/ORPO), or domain-specific mixtures. |
