Add new CrossEncoder model

README.md

@@ -0,0 +1,733 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- cross-encoder
+- reranker
+- generated_from_trainer
+- dataset_size:78704
+- loss:ListNetLoss
+base_model: jhu-clsp/ettin-encoder-400m
+datasets:
+- microsoft/ms_marco
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+metrics:
+- map
+- mrr@10
+- ndcg@10
+model-index:
+- name: CrossEncoder based on jhu-clsp/ettin-encoder-400m
+  results:
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoMSMARCO R100
+      type: NanoMSMARCO_R100
+    metrics:
+    - type: map
+      value: 0.5555
+      name: Map
+    - type: mrr@10
+      value: 0.544
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.6012
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNFCorpus R100
+      type: NanoNFCorpus_R100
+    metrics:
+    - type: map
+      value: 0.3715
+      name: Map
+    - type: mrr@10
+      value: 0.584
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.4065
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNQ R100
+      type: NanoNQ_R100
+    metrics:
+    - type: map
+      value: 0.6598
+      name: Map
+    - type: mrr@10
+      value: 0.683
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.7061
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-nano-beir
+      name: Cross Encoder Nano BEIR
+    dataset:
+      name: NanoBEIR R100 mean
+      type: NanoBEIR_R100_mean
+    metrics:
+    - type: map
+      value: 0.5289
+      name: Map
+    - type: mrr@10
+      value: 0.6037
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5713
+      name: Ndcg@10
+---
+
+# CrossEncoder based on jhu-clsp/ettin-encoder-400m
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [jhu-clsp/ettin-encoder-400m](https://huggingface.co/jhu-clsp/ettin-encoder-400m) on the [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [jhu-clsp/ettin-encoder-400m](https://huggingface.co/jhu-clsp/ettin-encoder-400m) <!-- at revision 7662476d60abb071a5bd319c9f3074f3072c062d -->
+- **Maximum Sequence Length:** 7999 tokens
+- **Number of Output Labels:** 1 label
+- **Training Dataset:**
+    - [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("rahulseetharaman/reranker-msmarco-v1.1-ettin-encoder-400m-listnet")
+# Get scores for pairs of texts
+pairs = [
+    ['what is the salary of an animal shelter manager', 'Average Animal Shelter Salaries. The average salary for animal shelter jobs is $50,000. Average animal shelter salaries can vary greatly due to company, location, industry, experience and benefits. This salary was calculated using the average salary for all jobs with the term animal shelter anywhere in the job listing.'],
+    ['what is the salary of an animal shelter manager', 'The salary that an animal shelter manager earns can vary based on a variety of factors such as their specific responsibilities, their years of experience, their educational background, and the region in which the position is located. Most animal shelter manager positions do not carry particularly high salaries, but those who follow animal rescue career paths tend to be willing to sacrifice some earning potential for the prospect of being able to help animals in need.'],
+    ['what is the salary of an animal shelter manager', 'Animal shelter managers’ salaries vary depending on their state and location. For example, the Midwest sits at the bottom of the list. Based on the May, 2010 Pay Scale survey, Illinois managers made $35,382, while the average Californian animal shelter manager earned $55,224. Size. The size of animal shelter also correlates to the manager’s salary. The more animals, the more employees, the more responsibility for the manager. In May, 2010, animal shelter managers working in a shelter employing 1 to 9 people, earned $33,835 on average.'],
+    ['what is the salary of an animal shelter manager', 'Salary by Region. Salaries for animal shelter managers also vary by region. According to the Salary Expert website, the highest compensation for such positions, about $60,000 a year, is in San Francisco. Animal shelter managers in the New York and Atlanta metro areas can expect to earn about $50,000 a year.'],
+    ['what is the salary of an animal shelter manager', 'Experience and years of service factor into an animal shelter manager’s salary. In May 2010, managers with 1 to 4 years of experience earned the least amount of money, $36,061. Those with 20 years or more experience, however, make $60,521, according to Pay Scale. Size. The size of animal shelter also correlates to the manager’s salary. The more animals, the more employees, the more responsibility for the manager. In May, 2010, animal shelter managers working in a shelter employing 1 to 9 people, earned $33,835 on average.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (5,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'what is the salary of an animal shelter manager',
+    [
+        'Average Animal Shelter Salaries. The average salary for animal shelter jobs is $50,000. Average animal shelter salaries can vary greatly due to company, location, industry, experience and benefits. This salary was calculated using the average salary for all jobs with the term animal shelter anywhere in the job listing.',
+        'The salary that an animal shelter manager earns can vary based on a variety of factors such as their specific responsibilities, their years of experience, their educational background, and the region in which the position is located. Most animal shelter manager positions do not carry particularly high salaries, but those who follow animal rescue career paths tend to be willing to sacrifice some earning potential for the prospect of being able to help animals in need.',
+        'Animal shelter managers’ salaries vary depending on their state and location. For example, the Midwest sits at the bottom of the list. Based on the May, 2010 Pay Scale survey, Illinois managers made $35,382, while the average Californian animal shelter manager earned $55,224. Size. The size of animal shelter also correlates to the manager’s salary. The more animals, the more employees, the more responsibility for the manager. In May, 2010, animal shelter managers working in a shelter employing 1 to 9 people, earned $33,835 on average.',
+        'Salary by Region. Salaries for animal shelter managers also vary by region. According to the Salary Expert website, the highest compensation for such positions, about $60,000 a year, is in San Francisco. Animal shelter managers in the New York and Atlanta metro areas can expect to earn about $50,000 a year.',
+        'Experience and years of service factor into an animal shelter manager’s salary. In May 2010, managers with 1 to 4 years of experience earned the least amount of money, $36,061. Those with 20 years or more experience, however, make $60,521, according to Pay Scale. Size. The size of animal shelter also correlates to the manager’s salary. The more animals, the more employees, the more responsibility for the manager. In May, 2010, animal shelter managers working in a shelter employing 1 to 9 people, earned $33,835 on average.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Reranking
+
+* Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
+* Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
+  ```json
+  {
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | NanoMSMARCO_R100     | NanoNFCorpus_R100    | NanoNQ_R100          |
+|:------------|:---------------------|:---------------------|:---------------------|
+| map         | 0.5555 (+0.0659)     | 0.3715 (+0.1105)     | 0.6598 (+0.2401)     |
+| mrr@10      | 0.5440 (+0.0665)     | 0.5840 (+0.0842)     | 0.6830 (+0.2563)     |
+| **ndcg@10** | **0.6012 (+0.0607)** | **0.4065 (+0.0815)** | **0.7061 (+0.2055)** |
+
+#### Cross Encoder Nano BEIR
+
+* Dataset: `NanoBEIR_R100_mean`
+* Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
+  ```json
+  {
+      "dataset_names": [
+          "msmarco",
+          "nfcorpus",
+          "nq"
+      ],
+      "rerank_k": 100,
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | Value                |
+|:------------|:---------------------|
+| map         | 0.5289 (+0.1389)     |
+| mrr@10      | 0.6037 (+0.1356)     |
+| **ndcg@10** | **0.5713 (+0.1159)** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 78,704 training samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                           | docs                                                                                   | labels                                                                                 |
+  |:--------|:------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                          | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 10 characters</li><li>mean: 33.66 characters</li><li>max: 103 characters</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                       | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>a gram is times as large as a milligram</code>        | <code>['1 milligram is 1000 times smaller than 1 gram so 1000 milligram = 1 gram. 1 milligram is 1000 times smaller than 1 gram so 1000 milligram = 1 gram. 1 microgram is 1000 times smaller than a 1 milligram and 1 million times smaller than 1 gram. 1 microgram is 1000 times smaller than a 1 milligram and 1 million times smaller than 1 gram. It takes 1 million mcg = 1 gram. Kilograms are 1000 times larger than a gram (1 kg = 1000 g). Kilograms are 1000 times larger than a gram (1 kg = 1000 g). Kilograms is used to denote weights of clients, upon med doses are based.', 'So to find out how many milligrams in grams, simply multiply it by 1000 or instead, use the converter below. 1 Gram = 1000 Milligrams. Gram is a metric system unit of mass. It is one thousandth (1/1000) of the metric system base unit, kilogram. It is a very commonly used unit of mass in daily life. The abbreviation is g. Milligram is a small unit of mass in metric system and used commonly in medicine and pharmacy etc.', 'A kil...</code> | <code>[1, 1, 0, 0, 0, ...]</code> |
+  | <code>income limit for dependent on taxes</code>            | <code>['Qualifying Relative. For qualifying relatives, there is a strict limit on how much income they could make during the year. A qualifying relative cannot make more than $3,700 of gross income during the year. If he makes more than $3,700, then he cannot be a dependent. ', "Claiming an earner as a dependent can also affect that person's own tax filing. She can only claim her earned income plus $300 as a standard deduction on her return if this amounts to less than the standard deduction for that year, $5,950 as of 2012. Your dependent cannot claim anyone else as a dependent on her own return.", 'If you had $12,000 in income, for example, and $800 in eligible child care expenses, then, your credit would be 35 percent of $800, or $280. For taxpayers with incomes of more than $43,000, the percentage was 20 percent, with no upper income limit. So whether you had income of $44,000 or $440,000, your credit would be 20 percent. If you had $2,000 in care expenses, that translates to $400.', 'Hi...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>how much does it cost to have your own website</code> | <code>["Some hosts also offer discounts if you pay a year (or more) in advance. Prices vary from web host to web host but are usually (at the time I wrote this article) around $10 per month if your website is new and doesn't have much traffic or data. There are many ways to advertise your site, such as in newspapers, magazines, TV, as well as over the web. Since the cost for ads in the traditional media varies from country to country, you will have to do your own research.", 'Domain Registration. To have your own domain name (.com, .net, .org, .nu) you must register the domain. This typically costs anywhere between $10 to $12 (USD) and is billed on a yearly basis. GoDaddy and E-Starr are two companies that register your domain for less than $12 a year.', 'At minimum, you need to invest in your own domain name and hosting. Depending on the type of domain name you choose, the costs could run from just $10 a year, to hundreds or even millions ! The options for website hosting run the gamut in p...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>ListNetLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#listnetloss) with these parameters:
+  ```json
+  {
+      "activation_fn": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": 16
+  }
+  ```
+
+### Evaluation Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 1,000 evaluation samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                          | docs                                                                                   | labels                                                                                 |
+  |:--------|:-----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                         | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 11 characters</li><li>mean: 34.3 characters</li><li>max: 105 characters</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                        | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:-------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>what is the salary of an animal shelter manager</code> | <code>['Average Animal Shelter Salaries. The average salary for animal shelter jobs is $50,000. Average animal shelter salaries can vary greatly due to company, location, industry, experience and benefits. This salary was calculated using the average salary for all jobs with the term animal shelter anywhere in the job listing.', 'The salary that an animal shelter manager earns can vary based on a variety of factors such as their specific responsibilities, their years of experience, their educational background, and the region in which the position is located. Most animal shelter manager positions do not carry particularly high salaries, but those who follow animal rescue career paths tend to be willing to sacrifice some earning potential for the prospect of being able to help animals in need.', 'Animal shelter managers’ salaries vary depending on their state and location. For example, the Midwest sits at the bottom of the list. Based on the May, 2010 Pay Scale survey, Illinois managers made ...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>how long do quail eggs take to hatch</code>            | <code>['I am often asked how long it takes quail eggs to hatch. The incubation period for bobwhite quail eggs is 23-24 days. Here is a link to information on incubating bobwhite quail eggs http://www.farmingfriends.com/incubating-bobwhite-quail-eggs/. The incubation period for Coturnix (Japanese) quail eggs is 17 days. Here is a link for information about incubating japanese quail eggs. http://www.farmingfriends.com/incubating-coturnix-japanese-quail/. Usually the hen will keep sitting if she thinks the other eggs are fertile and I have read that the hen can feel movement in the eggs so will keep sitting. Quail eggs take about 17 days to hatch and hen eggs take 21 days.', 'Turkey eggs usually take 21 to 28 days to hatch depending on what they are incubated in like an incubator or by a hen. It also depends on how fertile it is and how it is cared … for. It usually takes closer to 28 days. ', 'When you are trying to hatch Tennessee red quail eggs, it will take  approximately 23 days. You shoul...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>sign of june</code>                                    | <code>['June 1 – June 20 Gemini June 21 – June 30 Cancer. Those bearing the Gemini zodiac are incredibly flexible people who can adapt to almost any situation. They also possess a tenacity that not only enables them to rise above major setbacks but to take full advantage of negative situations as well.', 'Air is your paired element and as a Gemini, you have the most fluid connection with air out of any of the zodiac signs. You often witness curiosity pushing through you like a constant breeze and once you find something that truly interests you, your will seems to push with purpose.', 'There are 12 different stones listed as birthstones for the calendar month of June, or as Sun/Star, Planetary, or Talismanic stones for the Zodiac sign of Gemini or Cancer.', 'If you were born on the 26th of June you are a Cancer in the Western zodiac.', 'Birthday Meanings Of People Born On 6th June (Zodiac Sign Gemini). IF YOUR BIRTHDAY IS June 6, then you are a Gemini, who love to joke and play around. The G...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>ListNetLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#listnetloss) with these parameters:
+  ```json
+  {
+      "activation_fn": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": 16
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 5
+- `seed`: 12
+- `bf16`: True
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 5
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.0
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch      | Step     | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10  | NanoBEIR_R100_mean_ndcg@10 |
+|:----------:|:--------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
+| -1         | -1       | -             | -               | 0.0475 (-0.4929)         | 0.2463 (-0.0787)          | 0.0589 (-0.4418)     | 0.1176 (-0.3378)           |
+| 0.0002     | 1        | 2.1526        | -               | -                        | -                         | -                    | -                          |
+| 0.0203     | 100      | 2.0847        | 2.0890          | 0.1559 (-0.3845)         | 0.2694 (-0.0556)          | 0.1030 (-0.3976)     | 0.1761 (-0.2792)           |
+| 0.0407     | 200      | 2.0834        | 2.0844          | 0.3588 (-0.1817)         | 0.2750 (-0.0500)          | 0.3993 (-0.1013)     | 0.3444 (-0.1110)           |
+| 0.0610     | 300      | 2.0775        | 2.0807          | 0.4209 (-0.1196)         | 0.3176 (-0.0075)          | 0.3781 (-0.1226)     | 0.3722 (-0.0832)           |
+| 0.0813     | 400      | 2.0934        | 2.0779          | 0.5085 (-0.0319)         | 0.3194 (-0.0056)          | 0.5146 (+0.0140)     | 0.4475 (-0.0079)           |
+| 0.1016     | 500      | 2.0693        | 2.0760          | 0.5408 (+0.0003)         | 0.3275 (+0.0024)          | 0.5793 (+0.0786)     | 0.4825 (+0.0271)           |
+| 0.1220     | 600      | 2.0753        | 2.0753          | 0.5580 (+0.0176)         | 0.3464 (+0.0213)          | 0.6250 (+0.1244)     | 0.5098 (+0.0544)           |
+| 0.1423     | 700      | 2.0712        | 2.0752          | 0.5756 (+0.0352)         | 0.3418 (+0.0168)          | 0.6753 (+0.1747)     | 0.5309 (+0.0756)           |
+| 0.1626     | 800      | 2.0688        | 2.0743          | 0.5689 (+0.0284)         | 0.3171 (-0.0079)          | 0.6427 (+0.1420)     | 0.5096 (+0.0542)           |
+| 0.1830     | 900      | 2.0723        | 2.0742          | 0.5762 (+0.0358)         | 0.3681 (+0.0431)          | 0.6712 (+0.1705)     | 0.5385 (+0.0831)           |
+| 0.2033     | 1000     | 2.0743        | 2.0744          | 0.5799 (+0.0394)         | 0.3723 (+0.0473)          | 0.6483 (+0.1477)     | 0.5335 (+0.0781)           |
+| 0.2236     | 1100     | 2.0728        | 2.0739          | 0.5995 (+0.0590)         | 0.3717 (+0.0467)          | 0.6674 (+0.1668)     | 0.5462 (+0.0908)           |
+| 0.2440     | 1200     | 2.0735        | 2.0747          | 0.5869 (+0.0465)         | 0.3735 (+0.0484)          | 0.6365 (+0.1359)     | 0.5323 (+0.0770)           |
+| 0.2643     | 1300     | 2.0677        | 2.0733          | 0.6003 (+0.0598)         | 0.3783 (+0.0532)          | 0.5812 (+0.0806)     | 0.5199 (+0.0646)           |
+| 0.2846     | 1400     | 2.0647        | 2.0739          | 0.5669 (+0.0264)         | 0.3423 (+0.0172)          | 0.6041 (+0.1034)     | 0.5044 (+0.0490)           |
+| 0.3049     | 1500     | 2.0764        | 2.0740          | 0.5728 (+0.0324)         | 0.3761 (+0.0510)          | 0.6380 (+0.1373)     | 0.5289 (+0.0736)           |
+| 0.3253     | 1600     | 2.0652        | 2.0741          | 0.6202 (+0.0797)         | 0.3806 (+0.0556)          | 0.6523 (+0.1516)     | 0.5510 (+0.0956)           |
+| 0.3456     | 1700     | 2.0733        | 2.0731          | 0.5948 (+0.0544)         | 0.3869 (+0.0619)          | 0.6421 (+0.1415)     | 0.5413 (+0.0859)           |
+| 0.3659     | 1800     | 2.0679        | 2.0734          | 0.5797 (+0.0393)         | 0.3790 (+0.0540)          | 0.6340 (+0.1333)     | 0.5309 (+0.0755)           |
+| 0.3863     | 1900     | 2.075         | 2.0733          | 0.6027 (+0.0623)         | 0.3931 (+0.0680)          | 0.6285 (+0.1278)     | 0.5414 (+0.0861)           |
+| 0.4066     | 2000     | 2.0724        | 2.0732          | 0.5920 (+0.0516)         | 0.4056 (+0.0806)          | 0.6277 (+0.1271)     | 0.5418 (+0.0864)           |
+| 0.4269     | 2100     | 2.0866        | 2.0721          | 0.6059 (+0.0655)         | 0.4060 (+0.0810)          | 0.6602 (+0.1596)     | 0.5574 (+0.1020)           |
+| 0.4472     | 2200     | 2.0738        | 2.0725          | 0.6148 (+0.0744)         | 0.4167 (+0.0917)          | 0.6356 (+0.1350)     | 0.5557 (+0.1003)           |
+| 0.4676     | 2300     | 2.0699        | 2.0722          | 0.6112 (+0.0707)         | 0.3992 (+0.0742)          | 0.6919 (+0.1913)     | 0.5674 (+0.1121)           |
+| 0.4879     | 2400     | 2.0673        | 2.0725          | 0.5909 (+0.0505)         | 0.3820 (+0.0570)          | 0.6462 (+0.1455)     | 0.5397 (+0.0843)           |
+| 0.5082     | 2500     | 2.0687        | 2.0725          | 0.5926 (+0.0521)         | 0.3838 (+0.0587)          | 0.6493 (+0.1486)     | 0.5419 (+0.0865)           |
+| 0.5286     | 2600     | 2.0649        | 2.0725          | 0.6206 (+0.0801)         | 0.3825 (+0.0574)          | 0.6708 (+0.1702)     | 0.5580 (+0.1026)           |
+| 0.5489     | 2700     | 2.0686        | 2.0729          | 0.5457 (+0.0053)         | 0.3728 (+0.0478)          | 0.6497 (+0.1491)     | 0.5227 (+0.0674)           |
+| 0.5692     | 2800     | 2.0774        | 2.0716          | 0.5827 (+0.0422)         | 0.3931 (+0.0680)          | 0.6744 (+0.1737)     | 0.5500 (+0.0947)           |
+| 0.5896     | 2900     | 2.0638        | 2.0724          | 0.6289 (+0.0884)         | 0.3915 (+0.0665)          | 0.6878 (+0.1872)     | 0.5694 (+0.1140)           |
+| 0.6099     | 3000     | 2.0509        | 2.0718          | 0.5888 (+0.0483)         | 0.3906 (+0.0656)          | 0.6806 (+0.1800)     | 0.5533 (+0.0980)           |
+| 0.6302     | 3100     | 2.0703        | 2.0715          | 0.6150 (+0.0746)         | 0.3832 (+0.0582)          | 0.6645 (+0.1638)     | 0.5542 (+0.0989)           |
+| 0.6505     | 3200     | 2.0785        | 2.0718          | 0.5982 (+0.0577)         | 0.3766 (+0.0515)          | 0.6867 (+0.1860)     | 0.5538 (+0.0984)           |
+| 0.6709     | 3300     | 2.0691        | 2.0716          | 0.6034 (+0.0629)         | 0.3814 (+0.0563)          | 0.6262 (+0.1255)     | 0.5370 (+0.0816)           |
+| 0.6912     | 3400     | 2.0648        | 2.0721          | 0.6354 (+0.0950)         | 0.3811 (+0.0560)          | 0.6841 (+0.1834)     | 0.5669 (+0.1115)           |
+| 0.7115     | 3500     | 2.0645        | 2.0719          | 0.6226 (+0.0821)         | 0.3802 (+0.0551)          | 0.6867 (+0.1860)     | 0.5631 (+0.1078)           |
+| 0.7319     | 3600     | 2.0627        | 2.0719          | 0.5720 (+0.0316)         | 0.3800 (+0.0549)          | 0.6702 (+0.1696)     | 0.5407 (+0.0854)           |
+| 0.7522     | 3700     | 2.0737        | 2.0719          | 0.5945 (+0.0540)         | 0.3793 (+0.0542)          | 0.6807 (+0.1801)     | 0.5515 (+0.0961)           |
+| 0.7725     | 3800     | 2.0659        | 2.0721          | 0.5833 (+0.0429)         | 0.3749 (+0.0498)          | 0.6565 (+0.1559)     | 0.5382 (+0.0829)           |
+| **0.7928** | **3900** | **2.063**     | **2.0724**      | **0.6012 (+0.0607)**     | **0.4065 (+0.0815)**      | **0.7061 (+0.2055)** | **0.5713 (+0.1159)**       |
+| 0.8132     | 4000     | 2.0685        | 2.0720          | 0.6246 (+0.0842)         | 0.3876 (+0.0626)          | 0.6563 (+0.1556)     | 0.5562 (+0.1008)           |
+| 0.8335     | 4100     | 2.0705        | 2.0719          | 0.6045 (+0.0641)         | 0.3731 (+0.0480)          | 0.6908 (+0.1902)     | 0.5561 (+0.1008)           |
+| 0.8538     | 4200     | 2.066         | 2.0714          | 0.6121 (+0.0717)         | 0.3783 (+0.0533)          | 0.6603 (+0.1597)     | 0.5503 (+0.0949)           |
+| 0.8742     | 4300     | 2.06          | 2.0716          | 0.6186 (+0.0782)         | 0.3877 (+0.0627)          | 0.6495 (+0.1488)     | 0.5520 (+0.0966)           |
+| 0.8945     | 4400     | 2.0777        | 2.0710          | 0.6079 (+0.0674)         | 0.3937 (+0.0686)          | 0.6654 (+0.1647)     | 0.5556 (+0.1003)           |
+| 0.9148     | 4500     | 2.0558        | 2.0725          | 0.6238 (+0.0833)         | 0.3757 (+0.0507)          | 0.6381 (+0.1374)     | 0.5458 (+0.0905)           |
+| 0.9351     | 4600     | 2.0675        | 2.0718          | 0.6313 (+0.0909)         | 0.3729 (+0.0478)          | 0.6558 (+0.1551)     | 0.5533 (+0.0979)           |
+| 0.9555     | 4700     | 2.0755        | 2.0726          | 0.6048 (+0.0644)         | 0.3500 (+0.0250)          | 0.6126 (+0.1120)     | 0.5225 (+0.0671)           |
+| 0.9758     | 4800     | 2.0658        | 2.0716          | 0.6217 (+0.0812)         | 0.3875 (+0.0625)          | 0.6695 (+0.1689)     | 0.5596 (+0.1042)           |
+| 0.9961     | 4900     | 2.0735        | 2.0716          | 0.6073 (+0.0669)         | 0.3537 (+0.0287)          | 0.6470 (+0.1463)     | 0.5360 (+0.0806)           |
+| 1.0165     | 5000     | 2.061         | 2.0712          | 0.6010 (+0.0605)         | 0.3785 (+0.0534)          | 0.6492 (+0.1486)     | 0.5429 (+0.0875)           |
+| 1.0368     | 5100     | 2.0642        | 2.0719          | 0.6026 (+0.0622)         | 0.3840 (+0.0589)          | 0.6498 (+0.1491)     | 0.5454 (+0.0901)           |
+| 1.0571     | 5200     | 2.0681        | 2.0711          | 0.6143 (+0.0739)         | 0.3802 (+0.0551)          | 0.7052 (+0.2046)     | 0.5666 (+0.1112)           |
+| 1.0775     | 5300     | 2.0596        | 2.0709          | 0.6166 (+0.0762)         | 0.3897 (+0.0647)          | 0.6642 (+0.1636)     | 0.5569 (+0.1015)           |
+| 1.0978     | 5400     | 2.0576        | 2.0710          | 0.5779 (+0.0375)         | 0.3831 (+0.0581)          | 0.6276 (+0.1269)     | 0.5295 (+0.0742)           |
+| 1.1181     | 5500     | 2.0675        | 2.0720          | 0.6164 (+0.0759)         | 0.3843 (+0.0593)          | 0.6712 (+0.1705)     | 0.5573 (+0.1019)           |
+| 1.1384     | 5600     | 2.0647        | 2.0720          | 0.5934 (+0.0529)         | 0.3860 (+0.0610)          | 0.6664 (+0.1657)     | 0.5486 (+0.0932)           |
+| 1.1588     | 5700     | 2.0587        | 2.0715          | 0.6235 (+0.0831)         | 0.3813 (+0.0562)          | 0.6858 (+0.1852)     | 0.5635 (+0.1082)           |
+| 1.1791     | 5800     | 2.0608        | 2.0718          | 0.6298 (+0.0894)         | 0.3680 (+0.0430)          | 0.6572 (+0.1565)     | 0.5517 (+0.0963)           |
+| 1.1994     | 5900     | 2.0642        | 2.0720          | 0.5838 (+0.0434)         | 0.3961 (+0.0710)          | 0.6678 (+0.1671)     | 0.5492 (+0.0939)           |
+| 1.2198     | 6000     | 2.0588        | 2.0719          | 0.5920 (+0.0516)         | 0.3869 (+0.0619)          | 0.6653 (+0.1646)     | 0.5481 (+0.0927)           |
+| 1.2401     | 6100     | 2.0631        | 2.0722          | 0.5666 (+0.0262)         | 0.3672 (+0.0421)          | 0.6692 (+0.1685)     | 0.5343 (+0.0789)           |
+| 1.2604     | 6200     | 2.0591        | 2.0724          | 0.5742 (+0.0338)         | 0.3637 (+0.0387)          | 0.6546 (+0.1540)     | 0.5309 (+0.0755)           |
+| 1.2807     | 6300     | 2.0567        | 2.0725          | 0.5693 (+0.0288)         | 0.3756 (+0.0506)          | 0.6682 (+0.1676)     | 0.5377 (+0.0823)           |
+| 1.3011     | 6400     | 2.0597        | 2.0716          | 0.5587 (+0.0183)         | 0.3599 (+0.0349)          | 0.6731 (+0.1724)     | 0.5306 (+0.0752)           |
+| 1.3214     | 6500     | 2.0647        | 2.0727          | 0.5838 (+0.0434)         | 0.3633 (+0.0383)          | 0.6556 (+0.1550)     | 0.5343 (+0.0789)           |
+| 1.3417     | 6600     | 2.0624        | 2.0712          | 0.5602 (+0.0198)         | 0.3697 (+0.0446)          | 0.6359 (+0.1353)     | 0.5219 (+0.0666)           |
+| 1.3621     | 6700     | 2.0604        | 2.0714          | 0.5784 (+0.0379)         | 0.3629 (+0.0379)          | 0.6383 (+0.1376)     | 0.5265 (+0.0711)           |
+| 1.3824     | 6800     | 2.0702        | 2.0711          | 0.5400 (-0.0004)         | 0.3814 (+0.0563)          | 0.5985 (+0.0979)     | 0.5066 (+0.0512)           |
+| 1.4027     | 6900     | 2.0569        | 2.0713          | 0.5239 (-0.0165)         | 0.3827 (+0.0576)          | 0.6110 (+0.1103)     | 0.5059 (+0.0505)           |
+| 1.4231     | 7000     | 2.0689        | 2.0726          | 0.5382 (-0.0022)         | 0.3912 (+0.0661)          | 0.6323 (+0.1316)     | 0.5205 (+0.0652)           |
+| 1.4434     | 7100     | 2.056         | 2.0717          | 0.5331 (-0.0074)         | 0.3691 (+0.0441)          | 0.6244 (+0.1238)     | 0.5089 (+0.0535)           |
+| 1.4637     | 7200     | 2.0573        | 2.0717          | 0.5668 (+0.0263)         | 0.3769 (+0.0519)          | 0.6235 (+0.1228)     | 0.5224 (+0.0670)           |
+| 1.4840     | 7300     | 2.0544        | 2.0720          | 0.5543 (+0.0139)         | 0.3875 (+0.0624)          | 0.6240 (+0.1233)     | 0.5219 (+0.0666)           |
+| 1.5044     | 7400     | 2.0684        | 2.0718          | 0.5684 (+0.0280)         | 0.3753 (+0.0503)          | 0.6075 (+0.1068)     | 0.5171 (+0.0617)           |
+| 1.5247     | 7500     | 2.0588        | 2.0718          | 0.5663 (+0.0259)         | 0.3904 (+0.0654)          | 0.6564 (+0.1557)     | 0.5377 (+0.0823)           |
+| 1.5450     | 7600     | 2.0556        | 2.0713          | 0.5764 (+0.0360)         | 0.3732 (+0.0482)          | 0.6323 (+0.1317)     | 0.5273 (+0.0719)           |
+| 1.5654     | 7700     | 2.0572        | 2.0716          | 0.5589 (+0.0185)         | 0.3876 (+0.0626)          | 0.6327 (+0.1320)     | 0.5264 (+0.0710)           |
+| 1.5857     | 7800     | 2.0662        | 2.0712          | 0.5577 (+0.0173)         | 0.3912 (+0.0662)          | 0.6744 (+0.1737)     | 0.5411 (+0.0857)           |
+| 1.6060     | 7900     | 2.063         | 2.0720          | 0.5444 (+0.0040)         | 0.3859 (+0.0609)          | 0.6401 (+0.1395)     | 0.5235 (+0.0681)           |
+| 1.6263     | 8000     | 2.0641        | 2.0728          | 0.5565 (+0.0160)         | 0.4032 (+0.0781)          | 0.6318 (+0.1311)     | 0.5305 (+0.0751)           |
+| 1.6467     | 8100     | 2.0529        | 2.0720          | 0.5508 (+0.0104)         | 0.3929 (+0.0679)          | 0.6016 (+0.1010)     | 0.5151 (+0.0598)           |
+| 1.6670     | 8200     | 2.0734        | 2.0732          | 0.5626 (+0.0221)         | 0.3920 (+0.0670)          | 0.6223 (+0.1216)     | 0.5256 (+0.0702)           |
+| 1.6873     | 8300     | 2.0645        | 2.0719          | 0.5660 (+0.0255)         | 0.3953 (+0.0702)          | 0.6511 (+0.1504)     | 0.5374 (+0.0821)           |
+| 1.7077     | 8400     | 2.0637        | 2.0719          | 0.5640 (+0.0236)         | 0.3890 (+0.0640)          | 0.6608 (+0.1602)     | 0.5379 (+0.0826)           |
+| 1.7280     | 8500     | 2.0588        | 2.0715          | 0.5875 (+0.0471)         | 0.3911 (+0.0661)          | 0.6827 (+0.1820)     | 0.5538 (+0.0984)           |
+| 1.7483     | 8600     | 2.0584        | 2.0720          | 0.5755 (+0.0351)         | 0.4047 (+0.0797)          | 0.6930 (+0.1924)     | 0.5578 (+0.1024)           |
+| 1.7687     | 8700     | 2.0622        | 2.0726          | 0.5884 (+0.0480)         | 0.3793 (+0.0542)          | 0.6777 (+0.1771)     | 0.5485 (+0.0931)           |
+| 1.7890     | 8800     | 2.059         | 2.0725          | 0.5876 (+0.0472)         | 0.3962 (+0.0712)          | 0.7014 (+0.2007)     | 0.5617 (+0.1064)           |
+| 1.8093     | 8900     | 2.0561        | 2.0720          | 0.5855 (+0.0451)         | 0.3997 (+0.0747)          | 0.6678 (+0.1671)     | 0.5510 (+0.0956)           |
+| 1.8296     | 9000     | 2.0627        | 2.0718          | 0.5876 (+0.0472)         | 0.3902 (+0.0652)          | 0.6706 (+0.1700)     | 0.5495 (+0.0941)           |
+| 1.8500     | 9100     | 2.0516        | 2.0716          | 0.5754 (+0.0349)         | 0.3946 (+0.0696)          | 0.6524 (+0.1518)     | 0.5408 (+0.0854)           |
+| 1.8703     | 9200     | 2.0553        | 2.0714          | 0.5678 (+0.0274)         | 0.3869 (+0.0619)          | 0.6543 (+0.1536)     | 0.5363 (+0.0810)           |
+| 1.8906     | 9300     | 2.0606        | 2.0714          | 0.5426 (+0.0022)         | 0.4028 (+0.0778)          | 0.6784 (+0.1778)     | 0.5413 (+0.0859)           |
+| 1.9110     | 9400     | 2.0502        | 2.0714          | 0.5644 (+0.0240)         | 0.3986 (+0.0736)          | 0.6751 (+0.1745)     | 0.5461 (+0.0907)           |
+| 1.9313     | 9500     | 2.0617        | 2.0719          | 0.5160 (-0.0244)         | 0.3915 (+0.0664)          | 0.6381 (+0.1374)     | 0.5152 (+0.0598)           |
+| 1.9516     | 9600     | 2.0525        | 2.0712          | 0.5532 (+0.0128)         | 0.3886 (+0.0636)          | 0.6453 (+0.1446)     | 0.5290 (+0.0737)           |
+| 1.9719     | 9700     | 2.059         | 2.0713          | 0.5565 (+0.0161)         | 0.3922 (+0.0671)          | 0.6576 (+0.1569)     | 0.5354 (+0.0800)           |
+| 1.9923     | 9800     | 2.0607        | 2.0723          | 0.5623 (+0.0219)         | 0.3930 (+0.0680)          | 0.6539 (+0.1533)     | 0.5364 (+0.0810)           |
+| 2.0126     | 9900     | 2.0462        | 2.0758          | 0.5482 (+0.0078)         | 0.4075 (+0.0825)          | 0.6371 (+0.1364)     | 0.5309 (+0.0756)           |
+| 2.0329     | 10000    | 2.0481        | 2.0783          | 0.5441 (+0.0037)         | 0.3783 (+0.0532)          | 0.6386 (+0.1379)     | 0.5203 (+0.0649)           |
+| 2.0533     | 10100    | 2.0392        | 2.0772          | 0.5299 (-0.0106)         | 0.3750 (+0.0499)          | 0.6398 (+0.1391)     | 0.5149 (+0.0595)           |
+| 2.0736     | 10200    | 2.0362        | 2.0779          | 0.5068 (-0.0337)         | 0.3699 (+0.0448)          | 0.6162 (+0.1156)     | 0.4976 (+0.0422)           |
+| 2.0939     | 10300    | 2.0312        | 2.0782          | 0.5188 (-0.0216)         | 0.3783 (+0.0533)          | 0.6282 (+0.1275)     | 0.5085 (+0.0531)           |
+| 2.1143     | 10400    | 2.0307        | 2.0773          | 0.5011 (-0.0393)         | 0.3768 (+0.0517)          | 0.6125 (+0.1118)     | 0.4968 (+0.0414)           |
+| 2.1346     | 10500    | 2.0425        | 2.0779          | 0.5203 (-0.0201)         | 0.3796 (+0.0545)          | 0.6380 (+0.1373)     | 0.5126 (+0.0572)           |
+| 2.1549     | 10600    | 2.0441        | 2.0793          | 0.5256 (-0.0148)         | 0.3671 (+0.0421)          | 0.6091 (+0.1085)     | 0.5006 (+0.0453)           |
+| 2.1752     | 10700    | 2.046         | 2.0786          | 0.5220 (-0.0184)         | 0.3836 (+0.0585)          | 0.6237 (+0.1231)     | 0.5097 (+0.0544)           |
+| 2.1956     | 10800    | 2.0384        | 2.0763          | 0.5089 (-0.0315)         | 0.3791 (+0.0541)          | 0.6307 (+0.1300)     | 0.5062 (+0.0509)           |
+| 2.2159     | 10900    | 2.0373        | 2.0787          | 0.5019 (-0.0386)         | 0.3699 (+0.0448)          | 0.6322 (+0.1316)     | 0.5013 (+0.0459)           |
+| 2.2362     | 11000    | 2.0377        | 2.0772          | 0.5220 (-0.0184)         | 0.3697 (+0.0447)          | 0.5991 (+0.0984)     | 0.4969 (+0.0416)           |
+| 2.2566     | 11100    | 2.0422        | 2.0766          | 0.5232 (-0.0173)         | 0.3803 (+0.0552)          | 0.6249 (+0.1243)     | 0.5094 (+0.0541)           |
+| 2.2769     | 11200    | 2.0412        | 2.0763          | 0.5138 (-0.0266)         | 0.3739 (+0.0489)          | 0.5752 (+0.0746)     | 0.4877 (+0.0323)           |
+| 2.2972     | 11300    | 2.0411        | 2.0777          | 0.5240 (-0.0164)         | 0.3728 (+0.0478)          | 0.6340 (+0.1334)     | 0.5103 (+0.0549)           |
+| 2.3175     | 11400    | 2.0392        | 2.0770          | 0.5093 (-0.0312)         | 0.3638 (+0.0387)          | 0.5971 (+0.0965)     | 0.4901 (+0.0347)           |
+| 2.3379     | 11500    | 2.0346        | 2.0798          | 0.5061 (-0.0343)         | 0.3748 (+0.0497)          | 0.6418 (+0.1412)     | 0.5076 (+0.0522)           |
+| 2.3582     | 11600    | 2.0501        | 2.0772          | 0.5231 (-0.0173)         | 0.3734 (+0.0483)          | 0.6018 (+0.1012)     | 0.4994 (+0.0441)           |
+| 2.3785     | 11700    | 2.0488        | 2.0787          | 0.4924 (-0.0480)         | 0.3830 (+0.0580)          | 0.6310 (+0.1304)     | 0.5021 (+0.0468)           |
+| 2.3989     | 11800    | 2.0407        | 2.0752          | 0.5073 (-0.0332)         | 0.3853 (+0.0602)          | 0.6053 (+0.1046)     | 0.4993 (+0.0439)           |
+| 2.4192     | 11900    | 2.039         | 2.0793          | 0.5047 (-0.0357)         | 0.3815 (+0.0565)          | 0.6049 (+0.1042)     | 0.4970 (+0.0417)           |
+| 2.4395     | 12000    | 2.0405        | 2.0762          | 0.4999 (-0.0405)         | 0.3797 (+0.0547)          | 0.6229 (+0.1222)     | 0.5009 (+0.0455)           |
+| 2.4598     | 12100    | 2.0422        | 2.0773          | 0.5257 (-0.0147)         | 0.3741 (+0.0490)          | 0.6304 (+0.1297)     | 0.5100 (+0.0547)           |
+| 2.4802     | 12200    | 2.0395        | 2.0787          | 0.5189 (-0.0215)         | 0.3614 (+0.0364)          | 0.6396 (+0.1389)     | 0.5066 (+0.0513)           |
+| 2.5005     | 12300    | 2.0357        | 2.0800          | 0.4991 (-0.0413)         | 0.3693 (+0.0442)          | 0.6140 (+0.1133)     | 0.4941 (+0.0387)           |
+| 2.5208     | 12400    | 2.0449        | 2.0766          | 0.4701 (-0.0704)         | 0.3703 (+0.0453)          | 0.6154 (+0.1147)     | 0.4852 (+0.0299)           |
+| 2.5412     | 12500    | 2.0362        | 2.0765          | 0.4795 (-0.0609)         | 0.3794 (+0.0543)          | 0.6068 (+0.1062)     | 0.4886 (+0.0332)           |
+| 2.5615     | 12600    | 2.0357        | 2.0774          | 0.4893 (-0.0511)         | 0.3635 (+0.0385)          | 0.6128 (+0.1121)     | 0.4885 (+0.0332)           |
+| 2.5818     | 12700    | 2.0478        | 2.0793          | 0.4924 (-0.0481)         | 0.3738 (+0.0488)          | 0.6028 (+0.1022)     | 0.4897 (+0.0343)           |
+| 2.6022     | 12800    | 2.0378        | 2.0777          | 0.5023 (-0.0381)         | 0.3667 (+0.0417)          | 0.6150 (+0.1144)     | 0.4947 (+0.0393)           |
+| 2.6225     | 12900    | 2.0377        | 2.0787          | 0.5118 (-0.0286)         | 0.3709 (+0.0459)          | 0.6317 (+0.1310)     | 0.5048 (+0.0494)           |
+| 2.6428     | 13000    | 2.0392        | 2.0791          | 0.5099 (-0.0305)         | 0.3829 (+0.0579)          | 0.6300 (+0.1293)     | 0.5076 (+0.0522)           |
+| 2.6631     | 13100    | 2.0367        | 2.0786          | 0.5189 (-0.0215)         | 0.3890 (+0.0639)          | 0.6537 (+0.1530)     | 0.5205 (+0.0651)           |
+| 2.6835     | 13200    | 2.0431        | 2.0771          | 0.4851 (-0.0553)         | 0.3682 (+0.0431)          | 0.6240 (+0.1234)     | 0.4924 (+0.0371)           |
+| 2.7038     | 13300    | 2.0392        | 2.0793          | 0.5057 (-0.0347)         | 0.3841 (+0.0590)          | 0.6380 (+0.1373)     | 0.5092 (+0.0539)           |
+| 2.7241     | 13400    | 2.0443        | 2.0767          | 0.4739 (-0.0665)         | 0.3669 (+0.0418)          | 0.6151 (+0.1145)     | 0.4853 (+0.0299)           |
+| 2.7445     | 13500    | 2.0501        | 2.0773          | 0.4633 (-0.0771)         | 0.3543 (+0.0293)          | 0.6069 (+0.1063)     | 0.4748 (+0.0195)           |
+| 2.7648     | 13600    | 2.0321        | 2.0780          | 0.4930 (-0.0474)         | 0.3644 (+0.0394)          | 0.6082 (+0.1075)     | 0.4885 (+0.0332)           |
+| 2.7851     | 13700    | 2.0273        | 2.0786          | 0.4572 (-0.0832)         | 0.3653 (+0.0402)          | 0.6065 (+0.1058)     | 0.4763 (+0.0210)           |
+| 2.8054     | 13800    | 2.0326        | 2.0776          | 0.4665 (-0.0739)         | 0.3709 (+0.0459)          | 0.5942 (+0.0936)     | 0.4772 (+0.0218)           |
+| 2.8258     | 13900    | 2.0489        | 2.0769          | 0.4748 (-0.0657)         | 0.3716 (+0.0466)          | 0.5893 (+0.0886)     | 0.4785 (+0.0232)           |
+| 2.8461     | 14000    | 2.0442        | 2.0775          | 0.5012 (-0.0392)         | 0.3711 (+0.0460)          | 0.5988 (+0.0981)     | 0.4903 (+0.0350)           |
+| 2.8664     | 14100    | 2.0365        | 2.0779          | 0.4766 (-0.0639)         | 0.3759 (+0.0508)          | 0.5890 (+0.0884)     | 0.4805 (+0.0251)           |
+| 2.8868     | 14200    | 2.0334        | 2.0771          | 0.4703 (-0.0701)         | 0.3729 (+0.0478)          | 0.5817 (+0.0810)     | 0.4749 (+0.0196)           |
+| 2.9071     | 14300    | 2.037         | 2.0777          | 0.4868 (-0.0536)         | 0.3748 (+0.0498)          | 0.6002 (+0.0996)     | 0.4873 (+0.0319)           |
+| 2.9274     | 14400    | 2.0465        | 2.0775          | 0.4884 (-0.0520)         | 0.3658 (+0.0408)          | 0.6351 (+0.1345)     | 0.4965 (+0.0411)           |
+| 2.9478     | 14500    | 2.0314        | 2.0779          | 0.4955 (-0.0449)         | 0.3627 (+0.0377)          | 0.6120 (+0.1114)     | 0.4901 (+0.0347)           |
+| 2.9681     | 14600    | 2.0436        | 2.0776          | 0.5073 (-0.0331)         | 0.3639 (+0.0389)          | 0.5965 (+0.0958)     | 0.4892 (+0.0339)           |
+| 2.9884     | 14700    | 2.0382        | 2.0771          | 0.4987 (-0.0417)         | 0.3728 (+0.0477)          | 0.6041 (+0.1034)     | 0.4919 (+0.0365)           |
+| 3.0087     | 14800    | 2.0339        | 2.0793          | 0.4615 (-0.0789)         | 0.3673 (+0.0423)          | 0.5634 (+0.0628)     | 0.4641 (+0.0087)           |
+| 3.0291     | 14900    | 2.0197        | 2.0827          | 0.4515 (-0.0889)         | 0.3683 (+0.0433)          | 0.5774 (+0.0767)     | 0.4658 (+0.0104)           |
+| 3.0494     | 15000    | 2.0155        | 2.0809          | 0.4226 (-0.1178)         | 0.3684 (+0.0434)          | 0.5542 (+0.0536)     | 0.4484 (-0.0069)           |
+| 3.0697     | 15100    | 2.0157        | 2.0833          | 0.4044 (-0.1360)         | 0.3721 (+0.0471)          | 0.5168 (+0.0162)     | 0.4311 (-0.0242)           |
+| 3.0901     | 15200    | 2.0132        | 2.0821          | 0.4341 (-0.1064)         | 0.3650 (+0.0400)          | 0.5546 (+0.0539)     | 0.4512 (-0.0041)           |
+| 3.1104     | 15300    | 2.0201        | 2.0829          | 0.4324 (-0.1081)         | 0.3527 (+0.0277)          | 0.5693 (+0.0687)     | 0.4515 (-0.0039)           |
+| 3.1307     | 15400    | 2.0219        | 2.0816          | 0.4606 (-0.0798)         | 0.3623 (+0.0373)          | 0.5450 (+0.0443)     | 0.4560 (+0.0006)           |
+| 3.1510     | 15500    | 2.0189        | 2.0821          | 0.4729 (-0.0675)         | 0.3579 (+0.0328)          | 0.5762 (+0.0756)     | 0.4690 (+0.0136)           |
+| 3.1714     | 15600    | 2.0126        | 2.0831          | 0.4415 (-0.0989)         | 0.3503 (+0.0253)          | 0.5613 (+0.0606)     | 0.4510 (-0.0043)           |
+| 3.1917     | 15700    | 2.0201        | 2.0843          | 0.4552 (-0.0852)         | 0.3488 (+0.0238)          | 0.5897 (+0.0891)     | 0.4646 (+0.0092)           |
+| 3.2120     | 15800    | 2.0189        | 2.0850          | 0.4860 (-0.0544)         | 0.3520 (+0.0270)          | 0.5931 (+0.0924)     | 0.4770 (+0.0217)           |
+| 3.2324     | 15900    | 2.0196        | 2.0827          | 0.4668 (-0.0736)         | 0.3485 (+0.0235)          | 0.5721 (+0.0715)     | 0.4625 (+0.0071)           |
+| 3.2527     | 16000    | 2.0195        | 2.0831          | 0.4649 (-0.0756)         | 0.3522 (+0.0272)          | 0.5928 (+0.0921)     | 0.4699 (+0.0146)           |
+| 3.2730     | 16100    | 2.0197        | 2.0830          | 0.4790 (-0.0614)         | 0.3549 (+0.0298)          | 0.5798 (+0.0792)     | 0.4712 (+0.0159)           |
+| 3.2934     | 16200    | 2.0253        | 2.0824          | 0.4680 (-0.0725)         | 0.3633 (+0.0383)          | 0.5854 (+0.0848)     | 0.4722 (+0.0169)           |
+| 3.3137     | 16300    | 2.0244        | 2.0844          | 0.4810 (-0.0594)         | 0.3678 (+0.0427)          | 0.5862 (+0.0855)     | 0.4783 (+0.0230)           |
+| 3.3340     | 16400    | 2.0164        | 2.0844          | 0.4550 (-0.0854)         | 0.3614 (+0.0364)          | 0.5888 (+0.0881)     | 0.4684 (+0.0130)           |
+| 3.3543     | 16500    | 2.0065        | 2.0835          | 0.5012 (-0.0392)         | 0.3629 (+0.0379)          | 0.6218 (+0.1212)     | 0.4953 (+0.0399)           |
+| 3.3747     | 16600    | 2.02          | 2.0830          | 0.4915 (-0.0489)         | 0.3518 (+0.0267)          | 0.5898 (+0.0891)     | 0.4777 (+0.0223)           |
+| 3.3950     | 16700    | 2.0213        | 2.0842          | 0.4778 (-0.0627)         | 0.3605 (+0.0354)          | 0.5656 (+0.0649)     | 0.4679 (+0.0126)           |
+| 3.4153     | 16800    | 2.02          | 2.0842          | 0.4533 (-0.0871)         | 0.3663 (+0.0412)          | 0.5447 (+0.0440)     | 0.4548 (-0.0006)           |
+| 3.4357     | 16900    | 2.0178        | 2.0859          | 0.4510 (-0.0894)         | 0.3620 (+0.0369)          | 0.5665 (+0.0658)     | 0.4598 (+0.0044)           |
+| 3.4560     | 17000    | 2.0154        | 2.0849          | 0.4662 (-0.0742)         | 0.3673 (+0.0422)          | 0.5558 (+0.0552)     | 0.4631 (+0.0077)           |
+| 3.4763     | 17100    | 2.0145        | 2.0842          | 0.4698 (-0.0706)         | 0.3511 (+0.0261)          | 0.5797 (+0.0790)     | 0.4669 (+0.0115)           |
+| 3.4966     | 17200    | 2.019         | 2.0845          | 0.4305 (-0.1099)         | 0.3659 (+0.0409)          | 0.5614 (+0.0607)     | 0.4526 (-0.0028)           |
+| 3.5170     | 17300    | 2.0151        | 2.0843          | 0.4671 (-0.0733)         | 0.3644 (+0.0394)          | 0.5810 (+0.0804)     | 0.4708 (+0.0155)           |
+| 3.5373     | 17400    | 2.0131        | 2.0844          | 0.4711 (-0.0693)         | 0.3579 (+0.0329)          | 0.5760 (+0.0754)     | 0.4684 (+0.0130)           |
+| 3.5576     | 17500    | 2.0145        | 2.0822          | 0.4687 (-0.0717)         | 0.3588 (+0.0338)          | 0.5853 (+0.0847)     | 0.4709 (+0.0156)           |
+| 3.5780     | 17600    | 2.0184        | 2.0840          | 0.4723 (-0.0682)         | 0.3530 (+0.0279)          | 0.5898 (+0.0891)     | 0.4717 (+0.0163)           |
+| 3.5983     | 17700    | 2.0219        | 2.0839          | 0.4657 (-0.0748)         | 0.3538 (+0.0288)          | 0.5897 (+0.0890)     | 0.4697 (+0.0144)           |
+| 3.6186     | 17800    | 2.0186        | 2.0842          | 0.4466 (-0.0938)         | 0.3542 (+0.0292)          | 0.5779 (+0.0773)     | 0.4596 (+0.0042)           |
+| 3.6390     | 17900    | 2.0097        | 2.0843          | 0.4828 (-0.0577)         | 0.3607 (+0.0357)          | 0.5720 (+0.0713)     | 0.4718 (+0.0164)           |
+| 3.6593     | 18000    | 2.0208        | 2.0844          | 0.4724 (-0.0680)         | 0.3644 (+0.0394)          | 0.5634 (+0.0627)     | 0.4667 (+0.0114)           |
+| 3.6796     | 18100    | 2.0254        | 2.0839          | 0.5004 (-0.0400)         | 0.3675 (+0.0425)          | 0.5854 (+0.0848)     | 0.4844 (+0.0291)           |
+| 3.6999     | 18200    | 2.0205        | 2.0852          | 0.4712 (-0.0692)         | 0.3729 (+0.0479)          | 0.5775 (+0.0769)     | 0.4739 (+0.0185)           |
+| 3.7203     | 18300    | 2.0213        | 2.0856          | 0.4849 (-0.0555)         | 0.3645 (+0.0395)          | 0.5771 (+0.0765)     | 0.4755 (+0.0201)           |
+| 3.7406     | 18400    | 2.0321        | 2.0850          | 0.4516 (-0.0889)         | 0.3621 (+0.0371)          | 0.5578 (+0.0571)     | 0.4571 (+0.0018)           |
+| 3.7609     | 18500    | 2.0256        | 2.0832          | 0.4442 (-0.0962)         | 0.3710 (+0.0460)          | 0.5593 (+0.0587)     | 0.4582 (+0.0028)           |
+| 3.7813     | 18600    | 2.0178        | 2.0846          | 0.4536 (-0.0869)         | 0.3689 (+0.0439)          | 0.5854 (+0.0847)     | 0.4693 (+0.0139)           |
+| 3.8016     | 18700    | 2.0156        | 2.0842          | 0.4469 (-0.0935)         | 0.3620 (+0.0370)          | 0.5622 (+0.0615)     | 0.4570 (+0.0017)           |
+| 3.8219     | 18800    | 2.0202        | 2.0836          | 0.4712 (-0.0692)         | 0.3690 (+0.0439)          | 0.5732 (+0.0726)     | 0.4711 (+0.0158)           |
+| 3.8422     | 18900    | 2.0157        | 2.0838          | 0.4609 (-0.0796)         | 0.3508 (+0.0258)          | 0.5849 (+0.0842)     | 0.4655 (+0.0101)           |
+| 3.8626     | 19000    | 2.0139        | 2.0856          | 0.4356 (-0.1048)         | 0.3519 (+0.0269)          | 0.5631 (+0.0624)     | 0.4502 (-0.0052)           |
+| 3.8829     | 19100    | 2.0227        | 2.0832          | 0.4603 (-0.0802)         | 0.3592 (+0.0342)          | 0.5955 (+0.0948)     | 0.4717 (+0.0163)           |
+| 3.9032     | 19200    | 2.0184        | 2.0840          | 0.4684 (-0.0720)         | 0.3562 (+0.0312)          | 0.5860 (+0.0853)     | 0.4702 (+0.0148)           |
+| 3.9236     | 19300    | 2.0271        | 2.0832          | 0.4545 (-0.0860)         | 0.3635 (+0.0385)          | 0.5709 (+0.0703)     | 0.4630 (+0.0076)           |
+| 3.9439     | 19400    | 2.0283        | 2.0834          | 0.4524 (-0.0880)         | 0.3569 (+0.0318)          | 0.5711 (+0.0704)     | 0.4601 (+0.0047)           |
+| 3.9642     | 19500    | 2.0148        | 2.0836          | 0.4536 (-0.0869)         | 0.3556 (+0.0306)          | 0.5735 (+0.0728)     | 0.4609 (+0.0055)           |
+| 3.9845     | 19600    | 2.021         | 2.0841          | 0.4660 (-0.0744)         | 0.3563 (+0.0313)          | 0.5622 (+0.0615)     | 0.4615 (+0.0061)           |
+| 4.0049     | 19700    | 2.0185        | 2.0835          | 0.4452 (-0.0952)         | 0.3613 (+0.0363)          | 0.5338 (+0.0332)     | 0.4468 (-0.0086)           |
+| 4.0252     | 19800    | 2.005         | 2.0859          | 0.4500 (-0.0904)         | 0.3582 (+0.0332)          | 0.5490 (+0.0484)     | 0.4524 (-0.0029)           |
+| 4.0455     | 19900    | 2.0078        | 2.0864          | 0.4131 (-0.1273)         | 0.3477 (+0.0227)          | 0.5343 (+0.0337)     | 0.4317 (-0.0237)           |
+| 4.0659     | 20000    | 2.0096        | 2.0861          | 0.4120 (-0.1285)         | 0.3549 (+0.0299)          | 0.5329 (+0.0322)     | 0.4332 (-0.0221)           |
+| 4.0862     | 20100    | 2.009         | 2.0875          | 0.4151 (-0.1253)         | 0.3475 (+0.0224)          | 0.5105 (+0.0098)     | 0.4243 (-0.0310)           |
+| 4.1065     | 20200    | 2.0131        | 2.0869          | 0.4550 (-0.0854)         | 0.3499 (+0.0248)          | 0.5395 (+0.0389)     | 0.4481 (-0.0072)           |
+| 4.1269     | 20300    | 2.0067        | 2.0873          | 0.4371 (-0.1033)         | 0.3471 (+0.0221)          | 0.5333 (+0.0326)     | 0.4392 (-0.0162)           |
+| 4.1472     | 20400    | 2.0032        | 2.0883          | 0.4493 (-0.0912)         | 0.3542 (+0.0291)          | 0.5173 (+0.0167)     | 0.4403 (-0.0151)           |
+| 4.1675     | 20500    | 2.0077        | 2.0889          | 0.4513 (-0.0892)         | 0.3424 (+0.0173)          | 0.5270 (+0.0263)     | 0.4402 (-0.0152)           |
+| 4.1878     | 20600    | 2.0103        | 2.0865          | 0.4426 (-0.0978)         | 0.3388 (+0.0138)          | 0.5171 (+0.0164)     | 0.4328 (-0.0225)           |
+| 4.2082     | 20700    | 2.0067        | 2.0872          | 0.4429 (-0.0976)         | 0.3445 (+0.0194)          | 0.4934 (-0.0073)     | 0.4269 (-0.0285)           |
+| 4.2285     | 20800    | 2.0058        | 2.0877          | 0.4382 (-0.1022)         | 0.3469 (+0.0219)          | 0.5075 (+0.0069)     | 0.4309 (-0.0245)           |
+| 4.2488     | 20900    | 2.0117        | 2.0884          | 0.4319 (-0.1085)         | 0.3432 (+0.0182)          | 0.5178 (+0.0172)     | 0.4310 (-0.0244)           |
+| 4.2692     | 21000    | 2.0143        | 2.0872          | 0.4122 (-0.1282)         | 0.3396 (+0.0146)          | 0.5132 (+0.0126)     | 0.4217 (-0.0337)           |
+| 4.2895     | 21100    | 2.0108        | 2.0875          | 0.4333 (-0.1071)         | 0.3484 (+0.0233)          | 0.5300 (+0.0293)     | 0.4372 (-0.0182)           |
+| 4.3098     | 21200    | 2.0104        | 2.0879          | 0.4051 (-0.1353)         | 0.3561 (+0.0311)          | 0.5248 (+0.0242)     | 0.4287 (-0.0267)           |
+| 4.3301     | 21300    | 2.0188        | 2.0871          | 0.4072 (-0.1332)         | 0.3533 (+0.0282)          | 0.5204 (+0.0198)     | 0.4270 (-0.0284)           |
+| 4.3505     | 21400    | 2.0051        | 2.0878          | 0.4469 (-0.0935)         | 0.3536 (+0.0285)          | 0.5331 (+0.0324)     | 0.4445 (-0.0109)           |
+| 4.3708     | 21500    | 2.0109        | 2.0873          | 0.4213 (-0.1191)         | 0.3501 (+0.0250)          | 0.5313 (+0.0306)     | 0.4342 (-0.0212)           |
+| 4.3911     | 21600    | 2.006         | 2.0872          | 0.4388 (-0.1017)         | 0.3515 (+0.0265)          | 0.5109 (+0.0103)     | 0.4337 (-0.0216)           |
+| 4.4115     | 21700    | 2.0123        | 2.0878          | 0.4168 (-0.1236)         | 0.3567 (+0.0316)          | 0.5057 (+0.0051)     | 0.4264 (-0.0290)           |
+| 4.4318     | 21800    | 2.0092        | 2.0891          | 0.4030 (-0.1375)         | 0.3562 (+0.0312)          | 0.5083 (+0.0076)     | 0.4225 (-0.0329)           |
+| 4.4521     | 21900    | 2.0045        | 2.0887          | 0.4159 (-0.1245)         | 0.3503 (+0.0253)          | 0.5123 (+0.0117)     | 0.4262 (-0.0292)           |
+| 4.4725     | 22000    | 2.0178        | 2.0861          | 0.4375 (-0.1029)         | 0.3559 (+0.0308)          | 0.5314 (+0.0307)     | 0.4416 (-0.0138)           |
+| 4.4928     | 22100    | 2.0145        | 2.0881          | 0.4408 (-0.0996)         | 0.3617 (+0.0366)          | 0.5040 (+0.0033)     | 0.4355 (-0.0199)           |
+| 4.5131     | 22200    | 2.0025        | 2.0882          | 0.4291 (-0.1113)         | 0.3592 (+0.0341)          | 0.5233 (+0.0226)     | 0.4372 (-0.0182)           |
+| 4.5334     | 22300    | 2.0113        | 2.0892          | 0.4253 (-0.1151)         | 0.3569 (+0.0318)          | 0.5032 (+0.0025)     | 0.4285 (-0.0269)           |
+| 4.5538     | 22400    | 2.0167        | 2.0883          | 0.4236 (-0.1169)         | 0.3543 (+0.0292)          | 0.5218 (+0.0211)     | 0.4332 (-0.0222)           |
+| 4.5741     | 22500    | 2.0103        | 2.0881          | 0.4151 (-0.1254)         | 0.3611 (+0.0360)          | 0.5097 (+0.0090)     | 0.4286 (-0.0268)           |
+| 4.5944     | 22600    | 2.0072        | 2.0865          | 0.4143 (-0.1261)         | 0.3554 (+0.0304)          | 0.5245 (+0.0239)     | 0.4314 (-0.0240)           |
+| 4.6148     | 22700    | 2.0001        | 2.0871          | 0.4091 (-0.1313)         | 0.3575 (+0.0325)          | 0.5231 (+0.0225)     | 0.4299 (-0.0254)           |
+| 4.6351     | 22800    | 2.0078        | 2.0881          | 0.4099 (-0.1306)         | 0.3501 (+0.0250)          | 0.5151 (+0.0145)     | 0.4250 (-0.0303)           |
+| 4.6554     | 22900    | 2.0003        | 2.0878          | 0.4104 (-0.1300)         | 0.3460 (+0.0210)          | 0.5290 (+0.0284)     | 0.4285 (-0.0269)           |
+| 4.6757     | 23000    | 2.0152        | 2.0877          | 0.4269 (-0.1135)         | 0.3495 (+0.0245)          | 0.5259 (+0.0253)     | 0.4341 (-0.0212)           |
+| 4.6961     | 23100    | 2.011         | 2.0881          | 0.4442 (-0.0962)         | 0.3516 (+0.0266)          | 0.5347 (+0.0341)     | 0.4435 (-0.0118)           |
+| 4.7164     | 23200    | 2.0012        | 2.0884          | 0.4324 (-0.1081)         | 0.3498 (+0.0247)          | 0.5252 (+0.0245)     | 0.4358 (-0.0196)           |
+| 4.7367     | 23300    | 2.0176        | 2.0876          | 0.4238 (-0.1166)         | 0.3510 (+0.0260)          | 0.5339 (+0.0333)     | 0.4363 (-0.0191)           |
+| 4.7571     | 23400    | 2.0063        | 2.0884          | 0.4280 (-0.1125)         | 0.3546 (+0.0295)          | 0.5347 (+0.0341)     | 0.4391 (-0.0163)           |
+| 4.7774     | 23500    | 2.0049        | 2.0884          | 0.4358 (-0.1047)         | 0.3497 (+0.0247)          | 0.5300 (+0.0293)     | 0.4385 (-0.0169)           |
+| 4.7977     | 23600    | 2.0078        | 2.0873          | 0.4194 (-0.1211)         | 0.3508 (+0.0257)          | 0.5396 (+0.0390)     | 0.4366 (-0.0188)           |
+| 4.8181     | 23700    | 2.0143        | 2.0875          | 0.4268 (-0.1137)         | 0.3568 (+0.0318)          | 0.5266 (+0.0260)     | 0.4367 (-0.0186)           |
+| 4.8384     | 23800    | 2.0162        | 2.0872          | 0.4271 (-0.1133)         | 0.3525 (+0.0274)          | 0.5331 (+0.0325)     | 0.4376 (-0.0178)           |
+| 4.8587     | 23900    | 2.0089        | 2.0879          | 0.4297 (-0.1107)         | 0.3512 (+0.0262)          | 0.5241 (+0.0235)     | 0.4350 (-0.0203)           |
+| 4.8790     | 24000    | 1.9988        | 2.0879          | 0.4273 (-0.1131)         | 0.3481 (+0.0231)          | 0.5238 (+0.0231)     | 0.4331 (-0.0223)           |
+| 4.8994     | 24100    | 2.0074        | 2.0881          | 0.4362 (-0.1042)         | 0.3519 (+0.0269)          | 0.5283 (+0.0276)     | 0.4388 (-0.0166)           |
+| 4.9197     | 24200    | 2.0179        | 2.0878          | 0.4362 (-0.1042)         | 0.3473 (+0.0223)          | 0.5285 (+0.0279)     | 0.4373 (-0.0180)           |
+| 4.9400     | 24300    | 2.0162        | 2.0874          | 0.4228 (-0.1176)         | 0.3495 (+0.0245)          | 0.5348 (+0.0341)     | 0.4357 (-0.0197)           |
+| 4.9604     | 24400    | 1.9986        | 2.0874          | 0.4268 (-0.1136)         | 0.3493 (+0.0242)          | 0.5364 (+0.0358)     | 0.4375 (-0.0179)           |
+| 4.9807     | 24500    | 2.0158        | 2.0874          | 0.4331 (-0.1074)         | 0.3515 (+0.0265)          | 0.5379 (+0.0373)     | 0.4408 (-0.0145)           |
+| -1         | -1       | -             | -               | 0.6012 (+0.0607)         | 0.4065 (+0.0815)          | 0.7061 (+0.2055)     | 0.5713 (+0.1159)           |
+
+* The bold row denotes the saved checkpoint.
+</details>
+
+### Framework Versions
+- Python: 3.10.18
+- Sentence Transformers: 5.0.0
+- Transformers: 4.56.0.dev0
+- PyTorch: 2.7.1+cu126
+- Accelerate: 1.9.0
+- Datasets: 4.0.0
+- Tokenizers: 0.21.4
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### ListNetLoss
+```bibtex
+@inproceedings{cao2007learning,
+    title={Learning to Rank: From Pairwise Approach to Listwise Approach},
+    author={Cao, Zhe and Qin, Tao and Liu, Tie-Yan and Tsai, Ming-Feng and Li, Hang},
+    booktitle={Proceedings of the 24th international conference on Machine learning},
+    pages={129--136},
+    year={2007}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,57 @@
+{
+  "architectures": [
+    "ModernBertForSequenceClassification"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "causal_mask": false,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 2624,
+  "is_causal": false,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 160000.0,
+  "max_position_embeddings": 7999,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 16,
+  "num_hidden_layers": 28,
+  "pad_token_id": 50283,
+  "position_embedding_type": "sans_pos",
+  "repad_logits_with_grad": false,
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid",
+    "version": "5.0.0"
+  },
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.56.0.dev0",
+  "vocab_size": 50368
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dbc6e9d6cc46a49c805d90efaf0a8af0659933ef535f9e4373baae76339236bf
+size 1583347532

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 7999,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}