Model save

README.md

@@ -16,10 +16,10 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: -392272726550942272.0000
-- Ndcg: 0.9564
-- Ndcg@25: 0.5959
-- Precision@25: 0.2234
+- Loss: -438158728757996992.0000
+- Ndcg: 0.9571
+- Ndcg@25: 0.8843
+- Precision@25: 0.9634
 
 ## Model description
 
@@ -46,15 +46,18 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 4
 - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
-- num_epochs: 2
+- num_epochs: 5
 - mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss             | Epoch  | Step | Validation Loss          | Ndcg   | Ndcg@25 | Precision@25 |
 |:-------------------------:|:------:|:----:|:------------------------:|:------:|:-------:|:------------:|
-| -6819856770917832704.0000 | 1.0    | 44   | -289695770057323328.0000 | 0.9560 | 0.6207  | 0.2464       |
-| -164647509444630944.0000  | 1.9711 | 86   | -392272726550942272.0000 | 0.9564 | 0.5959  | 0.2234       |
+| -1114019463802178816.0000 | 1.0    | 44   | -109312091402722160.0000 | 0.9555 | 0.2112  | 0.0          |
+| -97186389406882208.0000   | 2.0    | 88   | -332415755916448320.0000 | 0.9562 | 0.7834  | 0.5920       |
+| -1330305825467111936.0000 | 3.0    | 132  | -391117015460168128.0000 | 0.9569 | 0.8217  | 0.9086       |
+| -355019110469337088.0000  | 4.0    | 176  | -415820224368131648.0000 | 0.9570 | 0.8829  | 0.9360       |
+| -4554729996694637568.0000 | 4.9017 | 215  | -438158728757996992.0000 | 0.9571 | 0.8843  | 0.9634       |
 
 
 ### Framework versions

config.json

@@ -1,7 +1,8 @@
 {
+  "_attn_implementation_autoset": true,
   "_name_or_path": "mistralai/Mistral-7B-Instruct-v0.3",
   "architectures": [
-    "LTRModel"
+    "MistralForCausalLM"
   ],
   "attention_dropout": 0.0,
   "bos_token_id": 1,
@@ -15925,7 +15926,7 @@
   "rope_theta": 1000000.0,
   "sliding_window": null,
   "tie_word_embeddings": false,
-  "torch_dtype": "float32",
+  "torch_dtype": "bfloat16",
   "transformers_version": "4.49.0",
   "use_cache": true,
   "vocab_size": 32768

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:447ef3af00bc298553b96416a20b9b05d450c52588ee3a4e274faecc870a6815
+oid sha256:824e8dcfabe8eae13306b90a911c7f499c28893007d76063323f973640afd0c7
 size 4323010659

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:32ddf0a3de16457bc32a43966b2bfde212cfda1c4ba6171aa8231284c3276345
+oid sha256:f1fc47b549caf15a57335bc06d3f9bf5336c1c0e7e1b99706a201d3e85dcb710
 size 5432