README.md

---
library_name: transformers
tags:
- agriculture
- question-answering
- LoRA
- tinyllama
- fine-tuned
- causal-lm
license: apache-2.0
---

# 🌾 AgriQA-TinyLlama-LoRA (Adapter)

A **LoRA fine-tuned TinyLlama model** for answering agriculture-related questions in a conversational format. This adapter is fine-tuned on the [AgriQA dataset](https://huggingface.co/datasets/shchoi83/agriQA) using **parameter-efficient fine-tuning (PEFT)** and is suitable for low-resource inference scenarios.

## 🧠 Model Details

- **Base Model:** [TinyLlama/TinyLlama-1.1B-Chat](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat)
- **LoRA Adapter Size:** ~2MB
- **Dataset:** [shchoi83/agriQA](https://huggingface.co/datasets/shchoi83/agriQA)
- **Task:** Question Answering (Instruction Tuning)
- **Language:** English
- **Adapter Only:** This repository only contains the **LoRA adapter**. You must load it on top of the base model.
- **Trained by:** [@theone049](https://huggingface.co/theone049)

## 🚀 Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "theone049/agriqa-tinyllama-lora-adapter")

# Inference
prompt = """### Instruction:
Answer the agricultural question.

### Input:
What is the control measure for aphid infestation in mustard crops?

### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## 📊 Training

- **Epochs:** 3  
- **Batch Size:** 8  
- **Learning Rate:** 2e-5  
- **Precision:** bf16  
- **Training Framework:** 🤗 `transformers` + `peft`  
- **Compute:** Google Colab T4 GPU

## 📁 Files

- `adapter_config.json`: Configuration of LoRA adapter.
- `adapter_model.safetensors`: The trained adapter weights.
- `README.md`: This file.

## 🛑 Limitations

- **Domain-Specific**: Works best on agri-related questions. Not suited for general conversation.
- **Small Dataset**: Initial training was done on a subset (~5k samples). Accuracy may improve with full dataset.
- **Not Merged**: Requires base model for usage.

## 📚 Citation

```bibtex
@misc{nithyanandam2024agriqa,
  title={AgriQA TinyLlama LoRA Adapter},
  author={Nithyanandam Venu},
  year={2024},
  howpublished={\url{https://huggingface.co/theone049/agriqa-tinyllama-lora-adapter}}
}
```

## ✉️ Contact

For questions or collaborations: [[email protected]](mailto:[email protected])

---

*This is part of an experimental project to improve AI Q&A for agriculture. Not for medical or pesticide advice.*