Update README.md
Browse files
README.md
CHANGED
|
@@ -59,6 +59,10 @@ This repository provides a Japanese GPT-NeoX model of 3.6 billion parameters. Th
|
|
| 59 |
|
| 60 |
[Tianyu Zhao](https://huggingface.co/tianyuz) and [Kei Sawada](https://huggingface.co/keisawada)
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
# Limitations
|
| 63 |
* We found this verison of PPO model tends to generate repeated text more often than its SFT counterpart, and thus we set `repetition_penalty=1.1` for better generation performance. (*The same generation hyper-parameters are applied to the SFT model in aforementioned evaluation experiments.*) You can also explore other hyperparameter combinations that yield higher generation randomness/diversity for better generation quality, e.g. `temperature=0.9, repetition_penalty=1.0`.
|
| 64 |
|
|
|
|
| 59 |
|
| 60 |
[Tianyu Zhao](https://huggingface.co/tianyuz) and [Kei Sawada](https://huggingface.co/keisawada)
|
| 61 |
|
| 62 |
+
* **Release date**
|
| 63 |
+
|
| 64 |
+
March 31, 2023
|
| 65 |
+
|
| 66 |
# Limitations
|
| 67 |
* We found this verison of PPO model tends to generate repeated text more often than its SFT counterpart, and thus we set `repetition_penalty=1.1` for better generation performance. (*The same generation hyper-parameters are applied to the SFT model in aforementioned evaluation experiments.*) You can also explore other hyperparameter combinations that yield higher generation randomness/diversity for better generation quality, e.g. `temperature=0.9, repetition_penalty=1.0`.
|
| 68 |
|