Question Answering
Transformers
Safetensors
German
phi3
text-generation
Connect-Transport
Logics Software
German support chatbot
Deutscher KI Chatbot
Kundenservice Chatbot
Deutscher Chatbot
KI-Chatbots für Unternehmen
Chatbot for SMEs
Question-answering
QLoRA fine-tuning
LLM training
custom_code
text-generation-inference
Update README.md
Browse files
README.md
CHANGED
|
@@ -129,11 +129,11 @@ eval_steps: 500 # adjust this if needed (e.g., if you use "steps", it dete
|
|
| 129 |
We follow the instructions provided in the [LLaMA-Factory Quickstart Guide](https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#quickstart):
|
| 130 |
|
| 131 |
```
|
| 132 |
-
llamafactory-cli train logicsct_train_Phi4_qlora_sft_otfq.yaml
|
| 133 |
-
llamafactory-cli chat logicsct_inference_Phi4_qlora_sft_otfq.yaml
|
| 134 |
-
llamafactory-cli export logicsct_export_Phi4_qlora_sft.yaml
|
| 135 |
-
llamafactory-cli export logicsct_export_Phi4_qlora_sft_Q4.yaml
|
| 136 |
-
llamafactory-cli chat logicsct_inference_Phi4_qlora_sft_otfq_Q4.yaml
|
| 137 |
```
|
| 138 |
|
| 139 |
### Comparison of Open Source Training/Models with OpenAI Proprietary Fine-Tuning
|
|
|
|
| 129 |
We follow the instructions provided in the [LLaMA-Factory Quickstart Guide](https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#quickstart):
|
| 130 |
|
| 131 |
```
|
| 132 |
+
llamafactory-cli train logicsct_train_Phi4_qlora_sft_otfq.yaml # VRAM used: 11093MiB for 4 bit QLoRA training
|
| 133 |
+
llamafactory-cli chat logicsct_inference_Phi4_qlora_sft_otfq.yaml # VRAM used: 30927MiB for inference of base model + QLoRA adapter
|
| 134 |
+
llamafactory-cli export logicsct_export_Phi4_qlora_sft.yaml # VRAM used: 665MiB + about 29 GB of system RAM for exporting a merged verison of the model with its adapter
|
| 135 |
+
llamafactory-cli export logicsct_export_Phi4_qlora_sft_Q4.yaml # VRAM used: 38277MiB for a 4bit quant export of the merged model
|
| 136 |
+
llamafactory-cli chat logicsct_inference_Phi4_qlora_sft_otfq_Q4.yaml # VRAM used: 9255MiB-11405MiB VRAM for inference of the 4bit quant merged model (increasing with increasing context length)
|
| 137 |
```
|
| 138 |
|
| 139 |
### Comparison of Open Source Training/Models with OpenAI Proprietary Fine-Tuning
|