Hugging Face's logo

jahidhasan
/
os-reasoning-model

Safetensors
English
gpt2
operating-systems
reasoning
education
computer-science
Model card Files
xet
Community
jahidhasan commited on
Commit
8d03952
·
verified ·
1 Parent(s): ad3f789

Upload OS Reasoning model

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. README.md +107 -0
  2. added_tokens.json +8 -0
  3. checkpoint-1500/added_tokens.json +8 -0
  4. checkpoint-1500/config.json +45 -0
  5. checkpoint-1500/generation_config.json +6 -0
  6. checkpoint-1500/merges.txt +0 -0
  7. checkpoint-1500/model.safetensors +3 -0
  8. checkpoint-1500/optimizer.pt +3 -0
  9. checkpoint-1500/rng_state.pth +3 -0
  10. checkpoint-1500/scheduler.pt +3 -0
  11. checkpoint-1500/special_tokens_map.json +50 -0
  12. checkpoint-1500/tokenizer.json +0 -0
  13. checkpoint-1500/tokenizer_config.json +77 -0
  14. checkpoint-1500/trainer_state.json +244 -0
  15. checkpoint-1500/training_args.bin +3 -0
  16. checkpoint-1500/vocab.json +0 -0
  17. checkpoint-2000/added_tokens.json +8 -0
  18. checkpoint-2000/config.json +45 -0
  19. checkpoint-2000/generation_config.json +6 -0
  20. checkpoint-2000/merges.txt +0 -0
  21. checkpoint-2000/model.safetensors +3 -0
  22. checkpoint-2000/optimizer.pt +3 -0
  23. checkpoint-2000/rng_state.pth +3 -0
  24. checkpoint-2000/scheduler.pt +3 -0
  25. checkpoint-2000/special_tokens_map.json +50 -0
  26. checkpoint-2000/tokenizer.json +0 -0
  27. checkpoint-2000/tokenizer_config.json +77 -0
  28. checkpoint-2000/trainer_state.json +314 -0
  29. checkpoint-2000/training_args.bin +3 -0
  30. checkpoint-2000/vocab.json +0 -0
  31. checkpoint-2100/added_tokens.json +8 -0
  32. checkpoint-2100/config.json +45 -0
  33. checkpoint-2100/generation_config.json +6 -0
  34. checkpoint-2100/merges.txt +0 -0
  35. checkpoint-2100/model.safetensors +3 -0
  36. checkpoint-2100/optimizer.pt +3 -0
  37. checkpoint-2100/rng_state.pth +3 -0
  38. checkpoint-2100/scheduler.pt +3 -0
  39. checkpoint-2100/special_tokens_map.json +50 -0
  40. checkpoint-2100/tokenizer.json +0 -0
  41. checkpoint-2100/tokenizer_config.json +77 -0
  42. checkpoint-2100/trainer_state.json +328 -0
  43. checkpoint-2100/training_args.bin +3 -0
  44. checkpoint-2100/vocab.json +0 -0
  45. config.json +45 -0
  46. generation_config.json +6 -0
  47. merges.txt +0 -0
  48. model.safetensors +3 -0
  49. special_tokens_map.json +50 -0
  50. tokenizer.json +0 -0
README.md ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ tags:
4
+ - operating-systems
5
+ - reasoning
6
+ - education
7
+ - computer-science
8
+ datasets:
9
+ - custom
10
+ metrics:
11
+ - accuracy
12
+ widget:
13
+ - text: "Question: What is a process in operating systems? Reasoning:"
14
+ example_title: "Process Explanation"
15
+ - text: "Question: How does virtual memory work? Reasoning:"
16
+ example_title: "Virtual Memory"
17
+ ---
18
+
19
+ # Operating System Reasoning Model
20
+
21
+ ## Model Description
22
+
23
+ This model is specifically fine-tuned for reasoning about Operating Systems concepts. It can:
24
+
25
+ - Explain OS concepts with step-by-step reasoning
26
+ - Solve OS-related problems
27
+ - Compare different OS mechanisms
28
+ - Provide educational explanations for students
29
+
30
+ ## Training Data
31
+
32
+ The model was trained on content from multiple authoritative Operating Systems textbooks and resources:
33
+
34
+ - **OSTEP (Operating Systems: Three Easy Pieces)** - 0 chapters
35
+ - **xv6 Documentation** - System implementation details
36
+ - **Academic OS Resources** - Additional educational content
37
+
38
+ Total training examples: 3354
39
+
40
+ ## Usage
41
+
42
+ ```python
43
+ from transformers import AutoTokenizer, AutoModelForCausalLM
44
+
45
+ tokenizer = AutoTokenizer.from_pretrained("jahidhasan/os-reasoning-model")
46
+ model = AutoModelForCausalLM.from_pretrained("jahidhasan/os-reasoning-model")
47
+
48
+ # Generate reasoning
49
+ question = "What is a deadlock in operating systems?"
50
+ prompt = f"Question: {question}\nReasoning:"
51
+
52
+ inputs = tokenizer(prompt, return_tensors="pt")
53
+ outputs = model.generate(**inputs, max_length=200, temperature=0.7)
54
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
55
+ print(response)
56
+ ```
57
+
58
+ ## Model Architecture
59
+
60
+ - **Base Model**: distilbert/distilgpt2
61
+ - **Parameters**: 81,917,184
62
+ - **Fine-tuning**: Specialized for OS domain reasoning
63
+
64
+ ## Performance
65
+
66
+ The model demonstrates strong performance on:
67
+ - Concept explanation tasks
68
+ - Problem-solving scenarios
69
+ - Comparative analysis
70
+ - Educational Q&A
71
+
72
+ ## Limitations
73
+
74
+ - Focused specifically on Operating Systems domain
75
+ - May not perform well on general reasoning tasks
76
+ - Requires clear, structured questions for best results
77
+
78
+ ## Citation
79
+
80
+ ```bibtex
81
+ @misc{os-reasoning-model,
82
+ author = {Jahid Hasan},
83
+ title = {Operating System Reasoning Model},
84
+ year = {2025},
85
+ publisher = {Hugging Face},
86
+ howpublished = {\url{https://huggingface.co/jahidhasan/os-reasoning-model}}
87
+ }
88
+ ```
89
+
90
+ ## Training Details
91
+
92
+ - **Training Epochs**: 5
93
+ - **Learning Rate**: 3e-5
94
+ - **Batch Size**: 16
95
+ - **Training Time**: Unknown
96
+
97
+ ## Educational Use
98
+
99
+ This model is particularly useful for:
100
+ - Computer Science students learning OS concepts
101
+ - Educators creating OS curriculum
102
+ - Self-study and review sessions
103
+ - Assignment and project assistance
104
+
105
+ ---
106
+
107
+ *Trained with ❤️ for OS education*
added_tokens.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "[/ANSWER]": 50262,
3
+ "[/OS_CONCEPT]": 50258,
4
+ "[/REASONING]": 50260,
5
+ "[ANSWER]": 50261,
6
+ "[OS_CONCEPT]": 50257,
7
+ "[REASONING]": 50259
8
+ }
checkpoint-1500/added_tokens.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "[/ANSWER]": 50262,
3
+ "[/OS_CONCEPT]": 50258,
4
+ "[/REASONING]": 50260,
5
+ "[ANSWER]": 50261,
6
+ "[OS_CONCEPT]": 50257,
7
+ "[REASONING]": 50259
8
+ }
checkpoint-1500/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_num_labels": 1,
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 50256,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 50256,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "label2id": {
16
+ "LABEL_0": 0
17
+ },
18
+ "layer_norm_epsilon": 1e-05,
19
+ "model_type": "gpt2",
20
+ "n_ctx": 1024,
21
+ "n_embd": 768,
22
+ "n_head": 12,
23
+ "n_inner": null,
24
+ "n_layer": 6,
25
+ "n_positions": 1024,
26
+ "reorder_and_upcast_attn": false,
27
+ "resid_pdrop": 0.1,
28
+ "scale_attn_by_inverse_layer_idx": false,
29
+ "scale_attn_weights": true,
30
+ "summary_activation": null,
31
+ "summary_first_dropout": 0.1,
32
+ "summary_proj_to_labels": true,
33
+ "summary_type": "cls_index",
34
+ "summary_use_proj": true,
35
+ "task_specific_params": {
36
+ "text-generation": {
37
+ "do_sample": true,
38
+ "max_length": 50
39
+ }
40
+ },
41
+ "torch_dtype": "float32",
42
+ "transformers_version": "4.53.0",
43
+ "use_cache": true,
44
+ "vocab_size": 50263
45
+ }
checkpoint-1500/generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 50256,
4
+ "eos_token_id": 50256,
5
+ "transformers_version": "4.53.0"
6
+ }
checkpoint-1500/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-1500/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f9737f4c51743981920356dc5d6ba50018754a9ee7db8fd946f658a5d23206d
3
+ size 327676360
checkpoint-1500/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b1f54cc2ca8d601f9985395c8998663489c1480eb7f2ed80f27d1d9181ec306
3
+ size 655401338
checkpoint-1500/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ca045602373bd80718d033f5ec56475d8a0d328f986673ad9e00da790d9601d
3
+ size 14244
checkpoint-1500/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48cda1885c98b0cead5fdea48b78a1926a5cdc27a238e5f4d6bb6fc551ae46d9
3
+ size 1064
checkpoint-1500/special_tokens_map.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "[OS_CONCEPT]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "[/OS_CONCEPT]",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ },
17
+ {
18
+ "content": "[REASONING]",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ {
25
+ "content": "[/REASONING]",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ {
32
+ "content": "[ANSWER]",
33
+ "lstrip": false,
34
+ "normalized": false,
35
+ "rstrip": false,
36
+ "single_word": false
37
+ },
38
+ {
39
+ "content": "[/ANSWER]",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false
44
+ }
45
+ ],
46
+ "bos_token": "<|endoftext|>",
47
+ "eos_token": "<|endoftext|>",
48
+ "pad_token": "<|endoftext|>",
49
+ "unk_token": "<|endoftext|>"
50
+ }
checkpoint-1500/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-1500/tokenizer_config.json ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "50256": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "50257": {
13
+ "content": "[OS_CONCEPT]",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "50258": {
21
+ "content": "[/OS_CONCEPT]",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "50259": {
29
+ "content": "[REASONING]",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50260": {
37
+ "content": "[/REASONING]",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "50261": {
45
+ "content": "[ANSWER]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "50262": {
53
+ "content": "[/ANSWER]",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ }
60
+ },
61
+ "additional_special_tokens": [
62
+ "[OS_CONCEPT]",
63
+ "[/OS_CONCEPT]",
64
+ "[REASONING]",
65
+ "[/REASONING]",
66
+ "[ANSWER]",
67
+ "[/ANSWER]"
68
+ ],
69
+ "bos_token": "<|endoftext|>",
70
+ "clean_up_tokenization_spaces": false,
71
+ "eos_token": "<|endoftext|>",
72
+ "extra_special_tokens": {},
73
+ "model_max_length": 1024,
74
+ "pad_token": "<|endoftext|>",
75
+ "tokenizer_class": "GPT2Tokenizer",
76
+ "unk_token": "<|endoftext|>"
77
+ }
checkpoint-1500/trainer_state.json ADDED
@@ -0,0 +1,244 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 3.5724508050089447,
6
+ "eval_steps": 500,
7
+ "global_step": 1500,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.11926058437686345,
14
+ "grad_norm": 8.05271053314209,
15
+ "learning_rate": 7.35e-06,
16
+ "loss": 5.4953,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.2385211687537269,
21
+ "grad_norm": 2.7420473098754883,
22
+ "learning_rate": 1.485e-05,
23
+ "loss": 1.2318,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.35778175313059035,
28
+ "grad_norm": 1.7430616617202759,
29
+ "learning_rate": 2.235e-05,
30
+ "loss": 0.7734,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 0.4770423375074538,
35
+ "grad_norm": 1.5376101732254028,
36
+ "learning_rate": 2.985e-05,
37
+ "loss": 0.6646,
38
+ "step": 200
39
+ },
40
+ {
41
+ "epoch": 0.5963029218843172,
42
+ "grad_norm": 1.6270289421081543,
43
+ "learning_rate": 2.9226315789473687e-05,
44
+ "loss": 0.6334,
45
+ "step": 250
46
+ },
47
+ {
48
+ "epoch": 0.7155635062611807,
49
+ "grad_norm": 1.8452305793762207,
50
+ "learning_rate": 2.8436842105263156e-05,
51
+ "loss": 0.6103,
52
+ "step": 300
53
+ },
54
+ {
55
+ "epoch": 0.8348240906380441,
56
+ "grad_norm": 1.8059115409851074,
57
+ "learning_rate": 2.7647368421052632e-05,
58
+ "loss": 0.6186,
59
+ "step": 350
60
+ },
61
+ {
62
+ "epoch": 0.9540846750149076,
63
+ "grad_norm": 1.6886439323425293,
64
+ "learning_rate": 2.6857894736842105e-05,
65
+ "loss": 0.6045,
66
+ "step": 400
67
+ },
68
+ {
69
+ "epoch": 1.071556350626118,
70
+ "grad_norm": 1.4987449645996094,
71
+ "learning_rate": 2.6068421052631578e-05,
72
+ "loss": 0.5941,
73
+ "step": 450
74
+ },
75
+ {
76
+ "epoch": 1.1908169350029816,
77
+ "grad_norm": 1.5386848449707031,
78
+ "learning_rate": 2.527894736842105e-05,
79
+ "loss": 0.5678,
80
+ "step": 500
81
+ },
82
+ {
83
+ "epoch": 1.310077519379845,
84
+ "grad_norm": 1.5747556686401367,
85
+ "learning_rate": 2.4489473684210527e-05,
86
+ "loss": 0.5671,
87
+ "step": 550
88
+ },
89
+ {
90
+ "epoch": 1.4293381037567083,
91
+ "grad_norm": 1.5854023694992065,
92
+ "learning_rate": 2.37e-05,
93
+ "loss": 0.5376,
94
+ "step": 600
95
+ },
96
+ {
97
+ "epoch": 1.5485986881335718,
98
+ "grad_norm": 1.4520505666732788,
99
+ "learning_rate": 2.2910526315789473e-05,
100
+ "loss": 0.5632,
101
+ "step": 650
102
+ },
103
+ {
104
+ "epoch": 1.6678592725104353,
105
+ "grad_norm": 1.4415792226791382,
106
+ "learning_rate": 2.2121052631578946e-05,
107
+ "loss": 0.5629,
108
+ "step": 700
109
+ },
110
+ {
111
+ "epoch": 1.7871198568872988,
112
+ "grad_norm": 1.4224036931991577,
113
+ "learning_rate": 2.1331578947368422e-05,
114
+ "loss": 0.5647,
115
+ "step": 750
116
+ },
117
+ {
118
+ "epoch": 1.906380441264162,
119
+ "grad_norm": 1.4895819425582886,
120
+ "learning_rate": 2.0542105263157895e-05,
121
+ "loss": 0.54,
122
+ "step": 800
123
+ },
124
+ {
125
+ "epoch": 2.0238521168753727,
126
+ "grad_norm": 1.5508358478546143,
127
+ "learning_rate": 1.9752631578947368e-05,
128
+ "loss": 0.5695,
129
+ "step": 850
130
+ },
131
+ {
132
+ "epoch": 2.143112701252236,
133
+ "grad_norm": 1.5374252796173096,
134
+ "learning_rate": 1.896315789473684e-05,
135
+ "loss": 0.5228,
136
+ "step": 900
137
+ },
138
+ {
139
+ "epoch": 2.2623732856290997,
140
+ "grad_norm": 1.639708161354065,
141
+ "learning_rate": 1.8173684210526317e-05,
142
+ "loss": 0.5512,
143
+ "step": 950
144
+ },
145
+ {
146
+ "epoch": 2.381633870005963,
147
+ "grad_norm": 1.6390520334243774,
148
+ "learning_rate": 1.738421052631579e-05,
149
+ "loss": 0.5252,
150
+ "step": 1000
151
+ },
152
+ {
153
+ "epoch": 2.500894454382826,
154
+ "grad_norm": 1.4625619649887085,
155
+ "learning_rate": 1.6594736842105263e-05,
156
+ "loss": 0.5339,
157
+ "step": 1050
158
+ },
159
+ {
160
+ "epoch": 2.62015503875969,
161
+ "grad_norm": 1.3197258710861206,
162
+ "learning_rate": 1.5805263157894735e-05,
163
+ "loss": 0.5479,
164
+ "step": 1100
165
+ },
166
+ {
167
+ "epoch": 2.739415623136553,
168
+ "grad_norm": 1.4443845748901367,
169
+ "learning_rate": 1.5015789473684212e-05,
170
+ "loss": 0.5134,
171
+ "step": 1150
172
+ },
173
+ {
174
+ "epoch": 2.8586762075134167,
175
+ "grad_norm": 1.7365626096725464,
176
+ "learning_rate": 1.4226315789473685e-05,
177
+ "loss": 0.5253,
178
+ "step": 1200
179
+ },
180
+ {
181
+ "epoch": 2.97793679189028,
182
+ "grad_norm": 1.7469673156738281,
183
+ "learning_rate": 1.343684210526316e-05,
184
+ "loss": 0.5452,
185
+ "step": 1250
186
+ },
187
+ {
188
+ "epoch": 3.095408467501491,
189
+ "grad_norm": 1.7984752655029297,
190
+ "learning_rate": 1.2647368421052632e-05,
191
+ "loss": 0.5178,
192
+ "step": 1300
193
+ },
194
+ {
195
+ "epoch": 3.2146690518783543,
196
+ "grad_norm": 1.5190192461013794,
197
+ "learning_rate": 1.1857894736842105e-05,
198
+ "loss": 0.5261,
199
+ "step": 1350
200
+ },
201
+ {
202
+ "epoch": 3.3339296362552178,
203
+ "grad_norm": 1.5508211851119995,
204
+ "learning_rate": 1.106842105263158e-05,
205
+ "loss": 0.5435,
206
+ "step": 1400
207
+ },
208
+ {
209
+ "epoch": 3.4531902206320813,
210
+ "grad_norm": 1.8733484745025635,
211
+ "learning_rate": 1.0278947368421052e-05,
212
+ "loss": 0.5001,
213
+ "step": 1450
214
+ },
215
+ {
216
+ "epoch": 3.5724508050089447,
217
+ "grad_norm": 1.6355196237564087,
218
+ "learning_rate": 9.489473684210527e-06,
219
+ "loss": 0.5005,
220
+ "step": 1500
221
+ }
222
+ ],
223
+ "logging_steps": 50,
224
+ "max_steps": 2100,
225
+ "num_input_tokens_seen": 0,
226
+ "num_train_epochs": 5,
227
+ "save_steps": 500,
228
+ "stateful_callbacks": {
229
+ "TrainerControl": {
230
+ "args": {
231
+ "should_epoch_stop": false,
232
+ "should_evaluate": false,
233
+ "should_log": false,
234
+ "should_save": true,
235
+ "should_training_stop": false
236
+ },
237
+ "attributes": {}
238
+ }
239
+ },
240
+ "total_flos": 782714416398336.0,
241
+ "train_batch_size": 1,
242
+ "trial_name": null,
243
+ "trial_params": null
244
+ }
checkpoint-1500/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e70b6a91213c1b38da9f91a27fc85eeb5f1a0452422f750cf236f5949b7e75b0
3
+ size 5304
checkpoint-1500/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2000/added_tokens.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "[/ANSWER]": 50262,
3
+ "[/OS_CONCEPT]": 50258,
4
+ "[/REASONING]": 50260,
5
+ "[ANSWER]": 50261,
6
+ "[OS_CONCEPT]": 50257,
7
+ "[REASONING]": 50259
8
+ }
checkpoint-2000/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_num_labels": 1,
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 50256,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 50256,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "label2id": {
16
+ "LABEL_0": 0
17
+ },
18
+ "layer_norm_epsilon": 1e-05,
19
+ "model_type": "gpt2",
20
+ "n_ctx": 1024,
21
+ "n_embd": 768,
22
+ "n_head": 12,
23
+ "n_inner": null,
24
+ "n_layer": 6,
25
+ "n_positions": 1024,
26
+ "reorder_and_upcast_attn": false,
27
+ "resid_pdrop": 0.1,
28
+ "scale_attn_by_inverse_layer_idx": false,
29
+ "scale_attn_weights": true,
30
+ "summary_activation": null,
31
+ "summary_first_dropout": 0.1,
32
+ "summary_proj_to_labels": true,
33
+ "summary_type": "cls_index",
34
+ "summary_use_proj": true,
35
+ "task_specific_params": {
36
+ "text-generation": {
37
+ "do_sample": true,
38
+ "max_length": 50
39
+ }
40
+ },
41
+ "torch_dtype": "float32",
42
+ "transformers_version": "4.53.0",
43
+ "use_cache": true,
44
+ "vocab_size": 50263
45
+ }
checkpoint-2000/generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 50256,
4
+ "eos_token_id": 50256,
5
+ "transformers_version": "4.53.0"
6
+ }
checkpoint-2000/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2000/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b553b008fcb3d780475d258c5a19d7accb45dd8e0d6460927432e8cd4f4ba96
3
+ size 327676360
checkpoint-2000/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:549bac7a1cea1d7337adac7451526fb1ca87b0df721e2eebf573b9e58e6c7e6c
3
+ size 655401338
checkpoint-2000/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e5f761a538d3d5b7c9c6f99207b1ef11f5a408dad976547562d1ba4eabc058e
3
+ size 14244
checkpoint-2000/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:770f1cbe19c3cd0e680868e46e72ffe46758d439e6cdc3fc2626f238cb58965b
3
+ size 1064
checkpoint-2000/special_tokens_map.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "[OS_CONCEPT]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "[/OS_CONCEPT]",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ },
17
+ {
18
+ "content": "[REASONING]",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ {
25
+ "content": "[/REASONING]",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ {
32
+ "content": "[ANSWER]",
33
+ "lstrip": false,
34
+ "normalized": false,
35
+ "rstrip": false,
36
+ "single_word": false
37
+ },
38
+ {
39
+ "content": "[/ANSWER]",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false
44
+ }
45
+ ],
46
+ "bos_token": "<|endoftext|>",
47
+ "eos_token": "<|endoftext|>",
48
+ "pad_token": "<|endoftext|>",
49
+ "unk_token": "<|endoftext|>"
50
+ }
checkpoint-2000/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2000/tokenizer_config.json ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "50256": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "50257": {
13
+ "content": "[OS_CONCEPT]",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "50258": {
21
+ "content": "[/OS_CONCEPT]",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "50259": {
29
+ "content": "[REASONING]",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50260": {
37
+ "content": "[/REASONING]",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "50261": {
45
+ "content": "[ANSWER]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "50262": {
53
+ "content": "[/ANSWER]",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ }
60
+ },
61
+ "additional_special_tokens": [
62
+ "[OS_CONCEPT]",
63
+ "[/OS_CONCEPT]",
64
+ "[REASONING]",
65
+ "[/REASONING]",
66
+ "[ANSWER]",
67
+ "[/ANSWER]"
68
+ ],
69
+ "bos_token": "<|endoftext|>",
70
+ "clean_up_tokenization_spaces": false,
71
+ "eos_token": "<|endoftext|>",
72
+ "extra_special_tokens": {},
73
+ "model_max_length": 1024,
74
+ "pad_token": "<|endoftext|>",
75
+ "tokenizer_class": "GPT2Tokenizer",
76
+ "unk_token": "<|endoftext|>"
77
+ }
checkpoint-2000/trainer_state.json ADDED
@@ -0,0 +1,314 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 4.763267740011926,
6
+ "eval_steps": 500,
7
+ "global_step": 2000,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.11926058437686345,
14
+ "grad_norm": 8.05271053314209,
15
+ "learning_rate": 7.35e-06,
16
+ "loss": 5.4953,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.2385211687537269,
21
+ "grad_norm": 2.7420473098754883,
22
+ "learning_rate": 1.485e-05,
23
+ "loss": 1.2318,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.35778175313059035,
28
+ "grad_norm": 1.7430616617202759,
29
+ "learning_rate": 2.235e-05,
30
+ "loss": 0.7734,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 0.4770423375074538,
35
+ "grad_norm": 1.5376101732254028,
36
+ "learning_rate": 2.985e-05,
37
+ "loss": 0.6646,
38
+ "step": 200
39
+ },
40
+ {
41
+ "epoch": 0.5963029218843172,
42
+ "grad_norm": 1.6270289421081543,
43
+ "learning_rate": 2.9226315789473687e-05,
44
+ "loss": 0.6334,
45
+ "step": 250
46
+ },
47
+ {
48
+ "epoch": 0.7155635062611807,
49
+ "grad_norm": 1.8452305793762207,
50
+ "learning_rate": 2.8436842105263156e-05,
51
+ "loss": 0.6103,
52
+ "step": 300
53
+ },
54
+ {
55
+ "epoch": 0.8348240906380441,
56
+ "grad_norm": 1.8059115409851074,
57
+ "learning_rate": 2.7647368421052632e-05,
58
+ "loss": 0.6186,
59
+ "step": 350
60
+ },
61
+ {
62
+ "epoch": 0.9540846750149076,
63
+ "grad_norm": 1.6886439323425293,
64
+ "learning_rate": 2.6857894736842105e-05,
65
+ "loss": 0.6045,
66
+ "step": 400
67
+ },
68
+ {
69
+ "epoch": 1.071556350626118,
70
+ "grad_norm": 1.4987449645996094,
71
+ "learning_rate": 2.6068421052631578e-05,
72
+ "loss": 0.5941,
73
+ "step": 450
74
+ },
75
+ {
76
+ "epoch": 1.1908169350029816,
77
+ "grad_norm": 1.5386848449707031,
78
+ "learning_rate": 2.527894736842105e-05,
79
+ "loss": 0.5678,
80
+ "step": 500
81
+ },
82
+ {
83
+ "epoch": 1.310077519379845,
84
+ "grad_norm": 1.5747556686401367,
85
+ "learning_rate": 2.4489473684210527e-05,
86
+ "loss": 0.5671,
87
+ "step": 550
88
+ },
89
+ {
90
+ "epoch": 1.4293381037567083,
91
+ "grad_norm": 1.5854023694992065,
92
+ "learning_rate": 2.37e-05,
93
+ "loss": 0.5376,
94
+ "step": 600
95
+ },
96
+ {
97
+ "epoch": 1.5485986881335718,
98
+ "grad_norm": 1.4520505666732788,
99
+ "learning_rate": 2.2910526315789473e-05,
100
+ "loss": 0.5632,
101
+ "step": 650
102
+ },
103
+ {
104
+ "epoch": 1.6678592725104353,
105
+ "grad_norm": 1.4415792226791382,
106
+ "learning_rate": 2.2121052631578946e-05,
107
+ "loss": 0.5629,
108
+ "step": 700
109
+ },
110
+ {
111
+ "epoch": 1.7871198568872988,
112
+ "grad_norm": 1.4224036931991577,
113
+ "learning_rate": 2.1331578947368422e-05,
114
+ "loss": 0.5647,
115
+ "step": 750
116
+ },
117
+ {
118
+ "epoch": 1.906380441264162,
119
+ "grad_norm": 1.4895819425582886,
120
+ "learning_rate": 2.0542105263157895e-05,
121
+ "loss": 0.54,
122
+ "step": 800
123
+ },
124
+ {
125
+ "epoch": 2.0238521168753727,
126
+ "grad_norm": 1.5508358478546143,
127
+ "learning_rate": 1.9752631578947368e-05,
128
+ "loss": 0.5695,
129
+ "step": 850
130
+ },
131
+ {
132
+ "epoch": 2.143112701252236,
133
+ "grad_norm": 1.5374252796173096,
134
+ "learning_rate": 1.896315789473684e-05,
135
+ "loss": 0.5228,
136
+ "step": 900
137
+ },
138
+ {
139
+ "epoch": 2.2623732856290997,
140
+ "grad_norm": 1.639708161354065,
141
+ "learning_rate": 1.8173684210526317e-05,
142
+ "loss": 0.5512,
143
+ "step": 950
144
+ },
145
+ {
146
+ "epoch": 2.381633870005963,
147
+ "grad_norm": 1.6390520334243774,
148
+ "learning_rate": 1.738421052631579e-05,
149
+ "loss": 0.5252,
150
+ "step": 1000
151
+ },
152
+ {
153
+ "epoch": 2.500894454382826,
154
+ "grad_norm": 1.4625619649887085,
155
+ "learning_rate": 1.6594736842105263e-05,
156
+ "loss": 0.5339,
157
+ "step": 1050
158
+ },
159
+ {
160
+ "epoch": 2.62015503875969,
161
+ "grad_norm": 1.3197258710861206,
162
+ "learning_rate": 1.5805263157894735e-05,
163
+ "loss": 0.5479,
164
+ "step": 1100
165
+ },
166
+ {
167
+ "epoch": 2.739415623136553,
168
+ "grad_norm": 1.4443845748901367,
169
+ "learning_rate": 1.5015789473684212e-05,
170
+ "loss": 0.5134,
171
+ "step": 1150
172
+ },
173
+ {
174
+ "epoch": 2.8586762075134167,
175
+ "grad_norm": 1.7365626096725464,
176
+ "learning_rate": 1.4226315789473685e-05,
177
+ "loss": 0.5253,
178
+ "step": 1200
179
+ },
180
+ {
181
+ "epoch": 2.97793679189028,
182
+ "grad_norm": 1.7469673156738281,
183
+ "learning_rate": 1.343684210526316e-05,
184
+ "loss": 0.5452,
185
+ "step": 1250
186
+ },
187
+ {
188
+ "epoch": 3.095408467501491,
189
+ "grad_norm": 1.7984752655029297,
190
+ "learning_rate": 1.2647368421052632e-05,
191
+ "loss": 0.5178,
192
+ "step": 1300
193
+ },
194
+ {
195
+ "epoch": 3.2146690518783543,
196
+ "grad_norm": 1.5190192461013794,
197
+ "learning_rate": 1.1857894736842105e-05,
198
+ "loss": 0.5261,
199
+ "step": 1350
200
+ },
201
+ {
202
+ "epoch": 3.3339296362552178,
203
+ "grad_norm": 1.5508211851119995,
204
+ "learning_rate": 1.106842105263158e-05,
205
+ "loss": 0.5435,
206
+ "step": 1400
207
+ },
208
+ {
209
+ "epoch": 3.4531902206320813,
210
+ "grad_norm": 1.8733484745025635,
211
+ "learning_rate": 1.0278947368421052e-05,
212
+ "loss": 0.5001,
213
+ "step": 1450
214
+ },
215
+ {
216
+ "epoch": 3.5724508050089447,
217
+ "grad_norm": 1.6355196237564087,
218
+ "learning_rate": 9.489473684210527e-06,
219
+ "loss": 0.5005,
220
+ "step": 1500
221
+ },
222
+ {
223
+ "epoch": 3.691711389385808,
224
+ "grad_norm": 1.238028645515442,
225
+ "learning_rate": 8.7e-06,
226
+ "loss": 0.5164,
227
+ "step": 1550
228
+ },
229
+ {
230
+ "epoch": 3.8109719737626713,
231
+ "grad_norm": 1.8155537843704224,
232
+ "learning_rate": 7.910526315789474e-06,
233
+ "loss": 0.5049,
234
+ "step": 1600
235
+ },
236
+ {
237
+ "epoch": 3.9302325581395348,
238
+ "grad_norm": 1.6747583150863647,
239
+ "learning_rate": 7.121052631578948e-06,
240
+ "loss": 0.5099,
241
+ "step": 1650
242
+ },
243
+ {
244
+ "epoch": 4.047704233750745,
245
+ "grad_norm": 1.4803907871246338,
246
+ "learning_rate": 6.331578947368422e-06,
247
+ "loss": 0.5148,
248
+ "step": 1700
249
+ },
250
+ {
251
+ "epoch": 4.166964818127608,
252
+ "grad_norm": 1.571410059928894,
253
+ "learning_rate": 5.542105263157895e-06,
254
+ "loss": 0.5128,
255
+ "step": 1750
256
+ },
257
+ {
258
+ "epoch": 4.286225402504472,
259
+ "grad_norm": 1.606655478477478,
260
+ "learning_rate": 4.752631578947368e-06,
261
+ "loss": 0.5116,
262
+ "step": 1800
263
+ },
264
+ {
265
+ "epoch": 4.405485986881335,
266
+ "grad_norm": 1.6239967346191406,
267
+ "learning_rate": 3.963157894736842e-06,
268
+ "loss": 0.5068,
269
+ "step": 1850
270
+ },
271
+ {
272
+ "epoch": 4.524746571258199,
273
+ "grad_norm": 1.3790518045425415,
274
+ "learning_rate": 3.173684210526316e-06,
275
+ "loss": 0.4962,
276
+ "step": 1900
277
+ },
278
+ {
279
+ "epoch": 4.644007155635062,
280
+ "grad_norm": 1.2910724878311157,
281
+ "learning_rate": 2.38421052631579e-06,
282
+ "loss": 0.5172,
283
+ "step": 1950
284
+ },
285
+ {
286
+ "epoch": 4.763267740011926,
287
+ "grad_norm": 2.057995557785034,
288
+ "learning_rate": 1.5947368421052633e-06,
289
+ "loss": 0.5007,
290
+ "step": 2000
291
+ }
292
+ ],
293
+ "logging_steps": 50,
294
+ "max_steps": 2100,
295
+ "num_input_tokens_seen": 0,
296
+ "num_train_epochs": 5,
297
+ "save_steps": 500,
298
+ "stateful_callbacks": {
299
+ "TrainerControl": {
300
+ "args": {
301
+ "should_epoch_stop": false,
302
+ "should_evaluate": false,
303
+ "should_log": false,
304
+ "should_save": true,
305
+ "should_training_stop": false
306
+ },
307
+ "attributes": {}
308
+ }
309
+ },
310
+ "total_flos": 1043619221864448.0,
311
+ "train_batch_size": 1,
312
+ "trial_name": null,
313
+ "trial_params": null
314
+ }
checkpoint-2000/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e70b6a91213c1b38da9f91a27fc85eeb5f1a0452422f750cf236f5949b7e75b0
3
+ size 5304
checkpoint-2000/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2100/added_tokens.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "[/ANSWER]": 50262,
3
+ "[/OS_CONCEPT]": 50258,
4
+ "[/REASONING]": 50260,
5
+ "[ANSWER]": 50261,
6
+ "[OS_CONCEPT]": 50257,
7
+ "[REASONING]": 50259
8
+ }
checkpoint-2100/config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_num_labels": 1,
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 50256,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 50256,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "label2id": {
16
+ "LABEL_0": 0
17
+ },
18
+ "layer_norm_epsilon": 1e-05,
19
+ "model_type": "gpt2",
20
+ "n_ctx": 1024,
21
+ "n_embd": 768,
22
+ "n_head": 12,
23
+ "n_inner": null,
24
+ "n_layer": 6,
25
+ "n_positions": 1024,
26
+ "reorder_and_upcast_attn": false,
27
+ "resid_pdrop": 0.1,
28
+ "scale_attn_by_inverse_layer_idx": false,
29
+ "scale_attn_weights": true,
30
+ "summary_activation": null,
31
+ "summary_first_dropout": 0.1,
32
+ "summary_proj_to_labels": true,
33
+ "summary_type": "cls_index",
34
+ "summary_use_proj": true,
35
+ "task_specific_params": {
36
+ "text-generation": {
37
+ "do_sample": true,
38
+ "max_length": 50
39
+ }
40
+ },
41
+ "torch_dtype": "float32",
42
+ "transformers_version": "4.53.0",
43
+ "use_cache": true,
44
+ "vocab_size": 50263
45
+ }
checkpoint-2100/generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 50256,
4
+ "eos_token_id": 50256,
5
+ "transformers_version": "4.53.0"
6
+ }
checkpoint-2100/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2100/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c173e96a46be22d9d0191ca38391ee5fc4d1326e59ceb072721c16bc5bbb0b98
3
+ size 327676360
checkpoint-2100/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9400d7aae5f3790d12002b9485fcc6def655b4ea4836b8fc1e60a1a1c75ecf6
3
+ size 655401338
checkpoint-2100/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6326a526ff8699f4ac51edcb6ba1aaaab54b1b963e88f0fff3fd251f6b7b78dc
3
+ size 14244
checkpoint-2100/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6483713128e4012a73332236f5e558536b9285dd2f47e8d45f1662059bbe467
3
+ size 1064
checkpoint-2100/special_tokens_map.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "[OS_CONCEPT]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "[/OS_CONCEPT]",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ },
17
+ {
18
+ "content": "[REASONING]",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ {
25
+ "content": "[/REASONING]",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ {
32
+ "content": "[ANSWER]",
33
+ "lstrip": false,
34
+ "normalized": false,
35
+ "rstrip": false,
36
+ "single_word": false
37
+ },
38
+ {
39
+ "content": "[/ANSWER]",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false
44
+ }
45
+ ],
46
+ "bos_token": "<|endoftext|>",
47
+ "eos_token": "<|endoftext|>",
48
+ "pad_token": "<|endoftext|>",
49
+ "unk_token": "<|endoftext|>"
50
+ }
checkpoint-2100/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2100/tokenizer_config.json ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "50256": {
5
+ "content": "<|endoftext|>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "50257": {
13
+ "content": "[OS_CONCEPT]",
14
+ "lstrip": false,
15
+ "normalized": false,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "50258": {
21
+ "content": "[/OS_CONCEPT]",
22
+ "lstrip": false,
23
+ "normalized": false,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "50259": {
29
+ "content": "[REASONING]",
30
+ "lstrip": false,
31
+ "normalized": false,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50260": {
37
+ "content": "[/REASONING]",
38
+ "lstrip": false,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ },
44
+ "50261": {
45
+ "content": "[ANSWER]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false,
50
+ "special": true
51
+ },
52
+ "50262": {
53
+ "content": "[/ANSWER]",
54
+ "lstrip": false,
55
+ "normalized": false,
56
+ "rstrip": false,
57
+ "single_word": false,
58
+ "special": true
59
+ }
60
+ },
61
+ "additional_special_tokens": [
62
+ "[OS_CONCEPT]",
63
+ "[/OS_CONCEPT]",
64
+ "[REASONING]",
65
+ "[/REASONING]",
66
+ "[ANSWER]",
67
+ "[/ANSWER]"
68
+ ],
69
+ "bos_token": "<|endoftext|>",
70
+ "clean_up_tokenization_spaces": false,
71
+ "eos_token": "<|endoftext|>",
72
+ "extra_special_tokens": {},
73
+ "model_max_length": 1024,
74
+ "pad_token": "<|endoftext|>",
75
+ "tokenizer_class": "GPT2Tokenizer",
76
+ "unk_token": "<|endoftext|>"
77
+ }
checkpoint-2100/trainer_state.json ADDED
@@ -0,0 +1,328 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 5.0,
6
+ "eval_steps": 500,
7
+ "global_step": 2100,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.11926058437686345,
14
+ "grad_norm": 8.05271053314209,
15
+ "learning_rate": 7.35e-06,
16
+ "loss": 5.4953,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.2385211687537269,
21
+ "grad_norm": 2.7420473098754883,
22
+ "learning_rate": 1.485e-05,
23
+ "loss": 1.2318,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.35778175313059035,
28
+ "grad_norm": 1.7430616617202759,
29
+ "learning_rate": 2.235e-05,
30
+ "loss": 0.7734,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 0.4770423375074538,
35
+ "grad_norm": 1.5376101732254028,
36
+ "learning_rate": 2.985e-05,
37
+ "loss": 0.6646,
38
+ "step": 200
39
+ },
40
+ {
41
+ "epoch": 0.5963029218843172,
42
+ "grad_norm": 1.6270289421081543,
43
+ "learning_rate": 2.9226315789473687e-05,
44
+ "loss": 0.6334,
45
+ "step": 250
46
+ },
47
+ {
48
+ "epoch": 0.7155635062611807,
49
+ "grad_norm": 1.8452305793762207,
50
+ "learning_rate": 2.8436842105263156e-05,
51
+ "loss": 0.6103,
52
+ "step": 300
53
+ },
54
+ {
55
+ "epoch": 0.8348240906380441,
56
+ "grad_norm": 1.8059115409851074,
57
+ "learning_rate": 2.7647368421052632e-05,
58
+ "loss": 0.6186,
59
+ "step": 350
60
+ },
61
+ {
62
+ "epoch": 0.9540846750149076,
63
+ "grad_norm": 1.6886439323425293,
64
+ "learning_rate": 2.6857894736842105e-05,
65
+ "loss": 0.6045,
66
+ "step": 400
67
+ },
68
+ {
69
+ "epoch": 1.071556350626118,
70
+ "grad_norm": 1.4987449645996094,
71
+ "learning_rate": 2.6068421052631578e-05,
72
+ "loss": 0.5941,
73
+ "step": 450
74
+ },
75
+ {
76
+ "epoch": 1.1908169350029816,
77
+ "grad_norm": 1.5386848449707031,
78
+ "learning_rate": 2.527894736842105e-05,
79
+ "loss": 0.5678,
80
+ "step": 500
81
+ },
82
+ {
83
+ "epoch": 1.310077519379845,
84
+ "grad_norm": 1.5747556686401367,
85
+ "learning_rate": 2.4489473684210527e-05,
86
+ "loss": 0.5671,
87
+ "step": 550
88
+ },
89
+ {
90
+ "epoch": 1.4293381037567083,
91
+ "grad_norm": 1.5854023694992065,
92
+ "learning_rate": 2.37e-05,
93
+ "loss": 0.5376,
94
+ "step": 600
95
+ },
96
+ {
97
+ "epoch": 1.5485986881335718,
98
+ "grad_norm": 1.4520505666732788,
99
+ "learning_rate": 2.2910526315789473e-05,
100
+ "loss": 0.5632,
101
+ "step": 650
102
+ },
103
+ {
104
+ "epoch": 1.6678592725104353,
105
+ "grad_norm": 1.4415792226791382,
106
+ "learning_rate": 2.2121052631578946e-05,
107
+ "loss": 0.5629,
108
+ "step": 700
109
+ },
110
+ {
111
+ "epoch": 1.7871198568872988,
112
+ "grad_norm": 1.4224036931991577,
113
+ "learning_rate": 2.1331578947368422e-05,
114
+ "loss": 0.5647,
115
+ "step": 750
116
+ },
117
+ {
118
+ "epoch": 1.906380441264162,
119
+ "grad_norm": 1.4895819425582886,
120
+ "learning_rate": 2.0542105263157895e-05,
121
+ "loss": 0.54,
122
+ "step": 800
123
+ },
124
+ {
125
+ "epoch": 2.0238521168753727,
126
+ "grad_norm": 1.5508358478546143,
127
+ "learning_rate": 1.9752631578947368e-05,
128
+ "loss": 0.5695,
129
+ "step": 850
130
+ },
131
+ {
132
+ "epoch": 2.143112701252236,
133
+ "grad_norm": 1.5374252796173096,
134
+ "learning_rate": 1.896315789473684e-05,
135
+ "loss": 0.5228,
136
+ "step": 900
137
+ },
138
+ {
139
+ "epoch": 2.2623732856290997,
140
+ "grad_norm": 1.639708161354065,
141
+ "learning_rate": 1.8173684210526317e-05,
142
+ "loss": 0.5512,
143
+ "step": 950
144
+ },
145
+ {
146
+ "epoch": 2.381633870005963,
147
+ "grad_norm": 1.6390520334243774,
148
+ "learning_rate": 1.738421052631579e-05,
149
+ "loss": 0.5252,
150
+ "step": 1000
151
+ },
152
+ {
153
+ "epoch": 2.500894454382826,
154
+ "grad_norm": 1.4625619649887085,
155
+ "learning_rate": 1.6594736842105263e-05,
156
+ "loss": 0.5339,
157
+ "step": 1050
158
+ },
159
+ {
160
+ "epoch": 2.62015503875969,
161
+ "grad_norm": 1.3197258710861206,
162
+ "learning_rate": 1.5805263157894735e-05,
163
+ "loss": 0.5479,
164
+ "step": 1100
165
+ },
166
+ {
167
+ "epoch": 2.739415623136553,
168
+ "grad_norm": 1.4443845748901367,
169
+ "learning_rate": 1.5015789473684212e-05,
170
+ "loss": 0.5134,
171
+ "step": 1150
172
+ },
173
+ {
174
+ "epoch": 2.8586762075134167,
175
+ "grad_norm": 1.7365626096725464,
176
+ "learning_rate": 1.4226315789473685e-05,
177
+ "loss": 0.5253,
178
+ "step": 1200
179
+ },
180
+ {
181
+ "epoch": 2.97793679189028,
182
+ "grad_norm": 1.7469673156738281,
183
+ "learning_rate": 1.343684210526316e-05,
184
+ "loss": 0.5452,
185
+ "step": 1250
186
+ },
187
+ {
188
+ "epoch": 3.095408467501491,
189
+ "grad_norm": 1.7984752655029297,
190
+ "learning_rate": 1.2647368421052632e-05,
191
+ "loss": 0.5178,
192
+ "step": 1300
193
+ },
194
+ {
195
+ "epoch": 3.2146690518783543,
196
+ "grad_norm": 1.5190192461013794,
197
+ "learning_rate": 1.1857894736842105e-05,
198
+ "loss": 0.5261,
199
+ "step": 1350
200
+ },
201
+ {
202
+ "epoch": 3.3339296362552178,
203
+ "grad_norm": 1.5508211851119995,
204
+ "learning_rate": 1.106842105263158e-05,
205
+ "loss": 0.5435,
206
+ "step": 1400
207
+ },
208
+ {
209
+ "epoch": 3.4531902206320813,
210
+ "grad_norm": 1.8733484745025635,
211
+ "learning_rate": 1.0278947368421052e-05,
212
+ "loss": 0.5001,
213
+ "step": 1450
214
+ },
215
+ {
216
+ "epoch": 3.5724508050089447,
217
+ "grad_norm": 1.6355196237564087,
218
+ "learning_rate": 9.489473684210527e-06,
219
+ "loss": 0.5005,
220
+ "step": 1500
221
+ },
222
+ {
223
+ "epoch": 3.691711389385808,
224
+ "grad_norm": 1.238028645515442,
225
+ "learning_rate": 8.7e-06,
226
+ "loss": 0.5164,
227
+ "step": 1550
228
+ },
229
+ {
230
+ "epoch": 3.8109719737626713,
231
+ "grad_norm": 1.8155537843704224,
232
+ "learning_rate": 7.910526315789474e-06,
233
+ "loss": 0.5049,
234
+ "step": 1600
235
+ },
236
+ {
237
+ "epoch": 3.9302325581395348,
238
+ "grad_norm": 1.6747583150863647,
239
+ "learning_rate": 7.121052631578948e-06,
240
+ "loss": 0.5099,
241
+ "step": 1650
242
+ },
243
+ {
244
+ "epoch": 4.047704233750745,
245
+ "grad_norm": 1.4803907871246338,
246
+ "learning_rate": 6.331578947368422e-06,
247
+ "loss": 0.5148,
248
+ "step": 1700
249
+ },
250
+ {
251
+ "epoch": 4.166964818127608,
252
+ "grad_norm": 1.571410059928894,
253
+ "learning_rate": 5.542105263157895e-06,
254
+ "loss": 0.5128,
255
+ "step": 1750
256
+ },
257
+ {
258
+ "epoch": 4.286225402504472,
259
+ "grad_norm": 1.606655478477478,
260
+ "learning_rate": 4.752631578947368e-06,
261
+ "loss": 0.5116,
262
+ "step": 1800
263
+ },
264
+ {
265
+ "epoch": 4.405485986881335,
266
+ "grad_norm": 1.6239967346191406,
267
+ "learning_rate": 3.963157894736842e-06,
268
+ "loss": 0.5068,
269
+ "step": 1850
270
+ },
271
+ {
272
+ "epoch": 4.524746571258199,
273
+ "grad_norm": 1.3790518045425415,
274
+ "learning_rate": 3.173684210526316e-06,
275
+ "loss": 0.4962,
276
+ "step": 1900
277
+ },
278
+ {
279
+ "epoch": 4.644007155635062,
280
+ "grad_norm": 1.2910724878311157,
281
+ "learning_rate": 2.38421052631579e-06,
282
+ "loss": 0.5172,
283
+ "step": 1950
284
+ },
285
+ {
286
+ "epoch": 4.763267740011926,
287
+ "grad_norm": 2.057995557785034,
288
+ "learning_rate": 1.5947368421052633e-06,
289
+ "loss": 0.5007,
290
+ "step": 2000
291
+ },
292
+ {
293
+ "epoch": 4.882528324388789,
294
+ "grad_norm": 1.6165767908096313,
295
+ "learning_rate": 8.052631578947369e-07,
296
+ "loss": 0.4959,
297
+ "step": 2050
298
+ },
299
+ {
300
+ "epoch": 5.0,
301
+ "grad_norm": 3.790095329284668,
302
+ "learning_rate": 1.5789473684210525e-08,
303
+ "loss": 0.4978,
304
+ "step": 2100
305
+ }
306
+ ],
307
+ "logging_steps": 50,
308
+ "max_steps": 2100,
309
+ "num_input_tokens_seen": 0,
310
+ "num_train_epochs": 5,
311
+ "save_steps": 500,
312
+ "stateful_callbacks": {
313
+ "TrainerControl": {
314
+ "args": {
315
+ "should_epoch_stop": false,
316
+ "should_evaluate": false,
317
+ "should_log": false,
318
+ "should_save": true,
319
+ "should_training_stop": true
320
+ },
321
+ "attributes": {}
322
+ }
323
+ },
324
+ "total_flos": 1095486626856960.0,
325
+ "train_batch_size": 1,
326
+ "trial_name": null,
327
+ "trial_params": null
328
+ }
checkpoint-2100/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e70b6a91213c1b38da9f91a27fc85eeb5f1a0452422f750cf236f5949b7e75b0
3
+ size 5304
checkpoint-2100/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_num_labels": 1,
3
+ "activation_function": "gelu_new",
4
+ "architectures": [
5
+ "GPT2LMHeadModel"
6
+ ],
7
+ "attn_pdrop": 0.1,
8
+ "bos_token_id": 50256,
9
+ "embd_pdrop": 0.1,
10
+ "eos_token_id": 50256,
11
+ "id2label": {
12
+ "0": "LABEL_0"
13
+ },
14
+ "initializer_range": 0.02,
15
+ "label2id": {
16
+ "LABEL_0": 0
17
+ },
18
+ "layer_norm_epsilon": 1e-05,
19
+ "model_type": "gpt2",
20
+ "n_ctx": 1024,
21
+ "n_embd": 768,
22
+ "n_head": 12,
23
+ "n_inner": null,
24
+ "n_layer": 6,
25
+ "n_positions": 1024,
26
+ "reorder_and_upcast_attn": false,
27
+ "resid_pdrop": 0.1,
28
+ "scale_attn_by_inverse_layer_idx": false,
29
+ "scale_attn_weights": true,
30
+ "summary_activation": null,
31
+ "summary_first_dropout": 0.1,
32
+ "summary_proj_to_labels": true,
33
+ "summary_type": "cls_index",
34
+ "summary_use_proj": true,
35
+ "task_specific_params": {
36
+ "text-generation": {
37
+ "do_sample": true,
38
+ "max_length": 50
39
+ }
40
+ },
41
+ "torch_dtype": "float32",
42
+ "transformers_version": "4.53.0",
43
+ "use_cache": true,
44
+ "vocab_size": 50263
45
+ }
generation_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 50256,
4
+ "eos_token_id": 50256,
5
+ "transformers_version": "4.53.0"
6
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c173e96a46be22d9d0191ca38391ee5fc4d1326e59ceb072721c16bc5bbb0b98
3
+ size 327676360
special_tokens_map.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "[OS_CONCEPT]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "[/OS_CONCEPT]",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ },
17
+ {
18
+ "content": "[REASONING]",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ {
25
+ "content": "[/REASONING]",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ },
31
+ {
32
+ "content": "[ANSWER]",
33
+ "lstrip": false,
34
+ "normalized": false,
35
+ "rstrip": false,
36
+ "single_word": false
37
+ },
38
+ {
39
+ "content": "[/ANSWER]",
40
+ "lstrip": false,
41
+ "normalized": false,
42
+ "rstrip": false,
43
+ "single_word": false
44
+ }
45
+ ],
46
+ "bos_token": "<|endoftext|>",
47
+ "eos_token": "<|endoftext|>",
48
+ "pad_token": "<|endoftext|>",
49
+ "unk_token": "<|endoftext|>"
50
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff