File size: 2,404 Bytes
4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 4b65bb6 f64d846 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
license: mit
tags:
- generated-from-train
- instruction-tuned
- phi2
- lora
- low-resource
- fine-tuning
datasets:
- yahma/alpaca-cleaned
base_model: microsoft/phi-2
widget:
- text: "### Instruction:\nExplain the concept of gravity.\n\n### Response:"
---
# 🧠 phi2-lora-instruct
This is a **LoRA fine-tuned version of Microsoft’s Phi-2** model trained on 500 examples from the [`yahma/alpaca-cleaned`](https://huggingface.co/datasets/yahma/alpaca-cleaned) instruction dataset.
### ✅ Fine-Tuned by:
**[howtomakepplragequit](https://huggingface.co/howtomakepplragequit)** — working on scalable, efficient LLM training for real-world instruction-following.
---
## 🏗️ Model Architecture
- **Base model**: [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) (2.7B parameters)
- **Adapter**: LoRA (Low-Rank Adaptation), trained with [PEFT](https://github.com/huggingface/peft)
- **Quantization**: 4-bit NF4 via `bitsandbytes` for efficient memory use
---
## 📦 Dataset
- [`yahma/alpaca-cleaned`](https://huggingface.co/datasets/yahma/alpaca-cleaned)
- Instruction-based Q&A for natural language understanding and generation
- Covers topics like science, grammar, everyday tasks, and reasoning
---
## 🛠️ Training Details
- **Training platform**: Google Colab (Free T4 GPU)
- **Epochs**: 2
- **Batch size**: 2 (with gradient accumulation)
- **Optimizer**: AdamW (via Transformers `Trainer`)
- **Training time**: ~20–30 mins
---
## 📈 Intended Use
- Ideal for **instruction-following tasks**, such as:
- Explanation
- Summarization
- List generation
- Creative writing
- Can be adapted to **custom domains** (health, code, manufacturing) by adding your own prompts + responses.
---
## 🚀 Example Prompt
Instruction:
Give three tips to improve time management.
---
## 🧪 Try it Out
To use this model in your own project:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("howtomakepplragequit/phi2-lora-instruct")
tokenizer = AutoTokenizer.from_pretrained("howtomakepplragequit/phi2-lora-instruct")
input_text = "### Instruction:\nExplain how machine learning works.\n\n### Response:"
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True)) |