metadata
license: apache-2.0
datasets:
- hackathon-somos-nlp-2023/ask2democracy-cfqa-salud-pension
language:
- es
library_name: transformers
pipeline_tag: text2text-generation
tags:
- democracy
- public debate
- question answering
- RAG
- Retrieval Augmented Generation
What's Baize-lora-7B-ask2democracy-cfqa?
This model is an open-source chat model fine-tuned with LoRA inspired by Baize project. It was trained with the Baize datasets and the ask2democracy-cfqa-salud-pension dataset, wich contains almost 4k instructions to answers questions based on a context relevant to citizen concerns and public debate in spanish.
- Developed by:
- 🇨🇴 Jorge Henao
- 🇨🇴 David Torres
Training Parameters
- Base Model: LLaMA-7B
- Training Epoch: 1
- Batch Size: 16
- Maximum Input Length: 512
- Learning Rate: 2e-4
- LoRA Rank: 8
- Updated Modules: All Linears
Training Dataset
- Ask2Democracy-cfqa-salud-pension (3806)
- Standford Alpaca (51,942)
- Quora Dialogs (54,456):
- StackOverflow Dialogs (57,046)
- Alpacaca chat Dialogs
- Medical chat Dialogs
More details can be found in the Ask2Democracy GitHub