|
|
--- |
|
|
base_model: meta-llama/Llama-3.3-70B-Instruct |
|
|
library_name: peft |
|
|
license: mit |
|
|
datasets: |
|
|
- tatsu-lab/alpaca |
|
|
language: |
|
|
- en |
|
|
- hi |
|
|
- ja |
|
|
- ta |
|
|
- te |
|
|
- mr |
|
|
tags: |
|
|
- llm |
|
|
- text-to-text |
|
|
- text-generation-inference |
|
|
- converstional |
|
|
- llama70b |
|
|
- lora |
|
|
- adapters |
|
|
--- |
|
|
# 🧠 Model Card: `pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat` |
|
|
|
|
|
A LoRA fine-tuned version of the **meta-llama/Llama-3.1-70B-Instruct** model on the **Alpaca dataset**, optimized using **PEFT** and accelerated on **Intel Gaudi3 HPU** hardware. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📝 Model Summary |
|
|
|
|
|
This model is a fine-tuned variant of LLaMA 3.1 70B Instruct, trained on the Alpaca dataset using Parameter-Efficient Fine-Tuning (PEFT) via LoRA. The goal of this fine-tuning was to improve instruction-following performance on lightweight resources, leveraging Intel’s Gaudi3 HPU for efficient training. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📄 Model Details |
|
|
|
|
|
* **Base Model:** `meta-llama/Llama-3.1-70B-Instruct` |
|
|
* **Fine-tuned Model:** `pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat` |
|
|
* **Fine-tuned By:** *Pranjal Singh Thakur* |
|
|
* **Dataset:** Stanford Alpaca dataset |
|
|
* **PEFT Library:** PEFT v0.12.0 |
|
|
* **Fine-tuning Technique:** LoRA |
|
|
* **Epochs:** 2 |
|
|
* **Training Hardware:** 1 Node with 8× Intel Gaudi3 HPUs |
|
|
* **Language(s):** English |
|
|
* **License:** Same as base model (LLaMA 3) |
|
|
* **Credit:** Intel for providing Gaudi3 HPU infrastructure |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Usage |
|
|
|
|
|
### Direct Use |
|
|
|
|
|
Use the model as an instruction-following chatbot or in downstream applications requiring LLM completion with lightweight deployment using LoRA adapters. |
|
|
|
|
|
```python |
|
|
from peft import PeftModel |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-70B-Instruct") |
|
|
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-70B-Instruct") |
|
|
|
|
|
model = PeftModel.from_pretrained(base_model, "pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat") |
|
|
|
|
|
inputs = tokenizer("### Instruction: Explain quantum computing in simple terms.", return_tensors="pt").to(model.device) |
|
|
outputs = model.generate(**inputs, max_new_tokens=256) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## 📊 Evaluation Results |
|
|
|
|
|
| Metric | Value | |
|
|
| ---------------------- | --------- | |
|
|
| Eval Accuracy | 73.27% | |
|
|
| Eval Loss | 1.02 | |
|
|
| Perplexity | 2.79 | |
|
|
| Evaluation Runtime | 20.97s | |
|
|
| Samples Evaluated | 101 | |
|
|
| Samples/Sec | 4.82 | |
|
|
| Max Memory Used (GB) | 126.2 | |
|
|
| Total Available Memory | 126.54 GB | |
|
|
| Memory Allocated (GB) | 41.06 | |
|
|
|
|
|
--- |
|
|
|
|
|
## 🛠 Training Configuration |
|
|
|
|
|
* **Epochs:** 2 |
|
|
* **Precision:** Likely mixed precision (bf16/fp16 on Gaudi3) |
|
|
* **Hardware:** Intel Gaudi3 HPU (8 cards, 1 node) |
|
|
* **Frameworks:** PEFT, Hugging Face Transformers |
|
|
* **Batching & Tokenization:** Not explicitly provided |
|
|
|
|
|
--- |
|
|
|
|
|
## 📦 Model Sources |
|
|
|
|
|
* **Repository:** [Hugging Face Model Card](https://huggingface.co/pranjalsingh/alpaca-Llama-3.1-70B-Instruct-chat) |
|
|
* **Dataset:** [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) |
|
|
* **Base Model:** [`meta-llama/Llama-3.1-70B-Instruct`](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct) |
|
|
|
|
|
--- |
|
|
|
|
|
## ⚠️ Limitations & Risks |
|
|
|
|
|
* Not suitable for multilingual tasks (trained only on English data). |
|
|
* May reflect biases present in the Alpaca dataset. |
|
|
* Not recommended for sensitive or safety-critical applications. |
|
|
* Fine-tuning was conducted for instruction tasks — may not generalize to other domains. |
|
|
|
|
|
--- |
|
|
|
|
|
## ♻️ Environmental Impact |
|
|
|
|
|
| Parameter | Value | |
|
|
| ----------------- | ----------------------------------------------------------- | |
|
|
| Compute Platform | Intel Gaudi3 | |
|
|
| Cards Used | 8× HPU | |
|
|
| Training Duration | \~2 Epochs | |
|
|
| Region | \[More info needed] | |
|
|
| Emission Estimate | \[Use [MLCO2](https://mlco2.github.io/impact) to calculate] | |
|
|
|
|
|
--- |
|
|
|
|
|
## 👨💻 Author & Acknowledgment |
|
|
|
|
|
* **Author:** Pranjal Singh Thakur |
|
|
* **Credit:** Intel (for compute resources using Gaudi3 HPU) |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔖 Citation |
|
|
|
|
|
Coming soon. |
