Add new CrossEncoder model

README.md

@@ -0,0 +1,403 @@
+---
+tags:
+- sentence-transformers
+- cross-encoder
+- generated_from_trainer
+- dataset_size:1129
+- loss:CrossEntropyLoss
+base_model: answerdotai/ModernBERT-base
+datasets:
+- dnth/cxs-eval-dataset
+pipeline_tag: text-classification
+library_name: sentence-transformers
+metrics:
+- f1_macro
+- f1_micro
+- f1_weighted
+model-index:
+- name: CrossEncoder based on answerdotai/ModernBERT-base
+  results:
+  - task:
+      type: cross-encoder-classification
+      name: Cross Encoder Classification
+    dataset:
+      name: validation set
+      type: validation-set
+    metrics:
+    - type: f1_macro
+      value: 0.7480758658603628
+      name: F1 Macro
+    - type: f1_micro
+      value: 0.8156028368794326
+      name: F1 Micro
+    - type: f1_weighted
+      value: 0.8193029448804775
+      name: F1 Weighted
+---
+
+# CrossEncoder based on answerdotai/ModernBERT-base
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the [cxs-eval-dataset](https://huggingface.co/datasets/dnth/cxs-eval-dataset) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text pair classification.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) <!-- at revision 8949b909ec900327062f0ebf497f51aef5e6f0c8 -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Number of Output Labels:** 2 labels
+- **Training Dataset:**
+    - [cxs-eval-dataset](https://huggingface.co/datasets/dnth/cxs-eval-dataset)
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("dnth/cxs-eval-modernbert-base-classifier")
+# Get scores for pairs of texts
+pairs = [
+    ["Inflight Quality Assurance Inspector - As an Inflight Quality Assurance Inspector at Santan, you are tasked with ensuring the accuracy and efficiency of meal and product handling for inflight services. Your responsibilities include verifying cart quantities, monitoring picking and staging processes, and maintaining inventory through CMS and WMS systems. You'll oversee packers and warehouse agents, manage stock adjustments and wastage reporting, and ensure compliance with FIFO and quality assurance standards. Key tasks include verifying outbound and inbound cart quantities, ensuring accurate meal picking, confirming seal and flight details in CMS, performing inventory counts, monitoring product packing, and facilitating effective communication between all parties involved. Additionally, you'll manage stock movements, update supervisors on stock issues, and ensure timely mass production by Packers or Warehouse Agents. - Air Transport", 'Manager (Cargo) - The Manager (Cargo) leads the development of operational strategies to improve the effectiveness of air cargo and warehouse operations. He/She is responsible for driving continuous improvement and business development initiatives to optimise processes and meet customer needs. He develops Standard Operating Procedures (SOPs) and systems to mitigate safety and/or security risks and monitors the adherence to safety and/or security standards. He also advises authorities and other teams to manage incidents involving hazardous materials and/or dangerous goods. He develops the teams technical capabilities and maintains positive morale within the teams.\n\nThe Manager (Cargo) has an extensive knowledge of cargo handling procedures, international airfreight regulations and policy requirements. He is also well-versed in organisational safety regulations and practices. He possesses strong interpersonal and stakeholder management skills in order to build and maintain stakeholder relationships. In addition, he is adaptable and manages stressful situations within a tight time frame. He also has strong communication and people management skills with outstanding computer literacy and problem-solving skills.'],
+    ["Manager, Pricing Systems - Manage the systems and tools that power our pricing and revenue management strategies. As Manager of Pricing Systems, you will oversee the implementation, maintenance, and enhancement of our pricing engines and related software. You'll work with IT and commercial teams to ensure system integrity, support new pricing initiatives, and troubleshoot issues. Technical expertise in airline pricing systems is a must. - Air Transport", 'Terminal Manager - The Terminal Manager is responsible for planning the delivery of airport services and implementing projects to improve the airport user experience. He/She collaborates with internal and external stakeholders to develop and establish compliance standards for airport operations. Besides leading research to identify new service offerings for the airport, he also enforces safety and/or security standards for the organisation. He sources for contractors to perform airport operations and establishes key indicators to track their performances. He also develops on-the-job training programmes for the department.\n\nThe Terminal Manager possesses a sound knowledge of airport and airside operations, Standard Operating Procedures (SOPs) and airport security and emergency response plans. He travels across the terminals to review operational processes and resolve incidents at the airport. To ensure operational plans and projects are implemented efficiently and effectively, he displays excellent project management skills with strong communication and stakeholder management skills to interact and engage all customers and stakeholders of the organisation.'],
+    ['Executive, Inflight Operation Team Lead - As an Executive, Inflight Operation Team Lead, you will excel in leading the Hi-Lift Driver team by demonstrating expertise in operational leadership while upholding high standards of safety and quality. You will contribute significantly to departmental and company objectives, ensuring strict adherence to industry regulations and operational procedures. Your role champions driving excellence in operational and engineering tasks, supporting initiatives to boost efficiency, and collaborating with cross-functional teams to achieve organizational goals, all while ensuring compliance with regulatory and organizational standards. - Air Transport', 'Executive (Ramp / Technical Ramp) - The Executive (Ramp/Technical Ramp) performs regular audits to ensure that ramp/technical ramp operations comply with Standard Operating Procedures (SOPs). He/She recommends enhancements to address areas of non-compliance, improves operational efficiencies and develops action plans to prevent safety and/or security issues from recurring. He also develops workplace learning plans and on-the-job training programmes to enhance the capabilities of teams.\n \nThe Executive (Ramp/Technical Ramp) has a sound knowledge of airport operations and ramp/technical ramp procedures. He also has strong analytical abilities to identify root causes of ground handling related delays and their potential implications. He is results-oriented and possesses excellent communication and interpersonal skills. Furthermore, he has strong computer literacy skills and competence in learning orientation to determine learning and developmental needs of his team members.'],
+    ['Executive, Customs & Logistics - As an Executive, Customs & Logistics, you will showcase your expertise as an Inflight Customs Officer by ensuring high safety and quality standards in all assignments. Your role involves contributing to the department and company objectives while adhering to industry regulations and operational procedures. You will champion excellence in operational or engineering tasks, support initiatives that enhance operational efficiency, and collaborate with cross-functional teams to achieve organizational goals, maintaining compliance with all regulatory and organizational standards. - Air Transport', 'Executive (Load Control) - The Executive (Load Control) determines load sheet requirements with airlines and develops manpower plans to support load control operations. He/She performs regular audits to ensure that the calculation of load limits, distribution and flight performance data complies with Standard Operating Procedures (SOPs). He also recommends enhancements to address areas of non-compliance and improves operational efficiencies. He develops SOPs to ensure alignment with safety and regulatory requirements, and is responsible for manpower deployment, developing on-the-job training programmes and workplace learning plans.\n\nBesides holding an Airlines Load and Trim Certification to perform his duties, the Executive (Load Control) also possesses outstanding data management and analytical skills. Moreover, he is comfortable with number crunching and is resilient to unforeseen interruptions and stressful situations. He also has excellent verbal and written communication skills and strong people management skills in order to develop effective training programmes and learning plans for the teams.'],
+    ['Quality Assurance Engineer - As a Quality Assurance Engineer in the aviation sector, you will conduct audits and inspections to ensure adherence to safety standards, review maintenance and operational processes, and assist in quality issue investigations. Your role includes supporting safety management systems, promoting continuous quality improvement, ensuring timely resolution of audit findings, and coordinating with regulatory bodies. You will also maintain up-to-date documentation of all quality assurance activities. - Aerospace', "Technician (Assembly) - The Technician (Assembly) performs assembly tasks for aircraft components in accordance with technical manuals and standard operating procedures (SOPs). He/She operates workshop equipment, tools and machines for the assembly of aircraft components. He also keeps abreast of latest developments of related systems by updating himself through relevant manuals and other publications. He may be authorised by the organisation to perform quality control functions, including inspection of incoming materials and assembled components and parts, and registration of non-conformances. He may also be authorised to perform level 1 non-destructive testing (NDT) functions under supervision, evaluate for acceptance or rejection, and record results as specified in the work instructions. He complies with airworthiness and legislative requirements, and the organisation's safety, health and quality systems. He supports in implementation of continuous improvement initiatives and lean practices. He works in a hangar or workshop and may be required to work in shifts. He should be systematic and detail-oriented, and able to work independently and in a team to accomplish assigned tasks."],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (5, 2)
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Classification
+
+* Dataset: `validation-set`
+* Evaluated with [<code>CrossEncoderClassificationEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderClassificationEvaluator)
+
+| Metric       | Value      |
+|:-------------|:-----------|
+| **f1_macro** | **0.7481** |
+| f1_micro     | 0.8156     |
+| f1_weighted  | 0.8193     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### cxs-eval-dataset
+
+* Dataset: [cxs-eval-dataset](https://huggingface.co/datasets/dnth/cxs-eval-dataset) at [a3a8a16](https://huggingface.co/datasets/dnth/cxs-eval-dataset/tree/a3a8a167ee1ca5fd48f6c99cff7b61fc756fbb3f)
+* Size: 1,129 training samples
+* Columns: <code>query</code>, <code>result</code>, and <code>label</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                              | result                                                                                              | label                                           |
+  |:--------|:---------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------|:------------------------------------------------|
+  | type    | string                                                                                             | string                                                                                              | int                                             |
+  | details | <ul><li>min: 193 characters</li><li>mean: 661.12 characters</li><li>max: 2255 characters</li></ul> | <ul><li>min: 556 characters</li><li>mean: 1161.45 characters</li><li>max: 1836 characters</li></ul> | <ul><li>0: ~76.70%</li><li>1: ~23.30%</li></ul> |
+* Samples:
+  | query                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | result                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | label          |
+  |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
+  | <code>Executive, Inflight Operation Team Lead - As an Executive, Inflight Operation Team Lead, you will excel in leading the Hi-Lift Driver team by demonstrating expertise in operational leadership while upholding high standards of safety and quality. You will contribute significantly to departmental and company objectives, ensuring strict adherence to industry regulations and operational procedures. Your role champions driving excellence in operational and engineering tasks, supporting initiatives to boost efficiency, and collaborating with cross-functional teams to achieve organizational goals, all while ensuring compliance with regulatory and organizational standards. - Air Transport</code>                                                                                                                                                      | <code>Executive (Baggage Services) - The Executive (Baggage Services) is the main point of contact for airline representatives, authorities and other departments to determine baggage handling standards and manpower requirements for baggage handling operations. He/She develops effective plans to mitigate compliance failures. He collates statistics and reports on baggage handling failures and is in charge of implementing new procedures to reduce these occurrences. He enforces a safety and security culture in the workplace and addresses internal or external audit issues. He is responsible for developing on-the-job training programmes and workplace learning plans.<br><br>A sound knowledge of airport operations, baggage handling procedures and associated administrative procedures to process requests and documentation is expected of the Executive (Baggage Services). He has good stakeholder management skills to handle internal and external stakeholders effectively. He is adept at customer service to hand...</code> | <code>0</code> |
+  | <code>VP of Passenger Experience - Customer Experience Strategy Development: Define and implement long-term customer service strategies aligned with brand positioning, regulatory requirements, and evolving passenger expectations.<br><br>Leadership and Department Oversight: Lead cross-functional teams across reservations, in-flight services, airport staff, and support centers to ensure unified and consistent service delivery across all locations.<br><br>Performance and Quality Management: Monitor customer service KPIs (NPS, CSAT, FCR), oversee service audits, and implement improvements based on passenger feedback and competitive benchmarking. - Air Transport</code>                                                                                                                                                                                    | <code>Vice President - The Vice President formulates strategies for passenger services operations to ensure performance levels are aligned with evolving customer needs, technological advancements, airline brand identity and business objectives. He/She is responsible for spearheading continuous improvement and business development initiatives to improve the profitability of the business. He leads the development of Standard Operating Procedures (SOPs) and systems to mitigate safety and/or security risks and plays a pivotal role in influencing organisational development by making key decisions on human capital needs.<br><br>The Vice President for Passenger Services possesses exceptional leadership and stakeholder management skills to cultivate local and international relationships with senior stakeholders in the industry and further the business objectives of the organisation. Furthermore, he has an in-depth knowledge of the airports organisational structures, ground handling operations and safety a...</code> | <code>1</code> |
+  | <code>Guest Services Assistant - As a Guest Services Assistant, you will uphold the highest standards of safety, security, and service in our ground operations. Your responsibilities will include verifying documentation through passport checks and ID matches, managing reservations, and issuing boarding passes. You will ensure smooth luggage tagging and inspections, oversee guest boarding and deplaning processes, and provide specialized assistance for guests with special needs, such as wheelchair services. Handling important documents like GD and crew sign-ins, as well as collecting immigration cards, is essential. You must also conduct the mandatory security questionnaire for passengers checking in luggage. Compliance with airline standards, safety protocols, and applicable laws in all operational areas is paramount. - Air Transport</code> | <code>Executive (Ground Services) - The Executive (Ground Services) oversees activities relating to the audit of ground service standards and develops partnerships with a wide range of audiences and stakeholders. He/She reviews service standard agreements and develops action plans to improve service efficiency for the airline. He conducts rate reviews for ground handlers regarding pricing, service quality and scope of work. In addition, he also plans cost estimates for contracts with ground handling organisations. He proposes process changes to improve passenger safety and/or security standards. He contributes to organisational development by developing on-the-job training programmes and workplace learning plans.<br><br>With foresight and highly developed analytical capabilities, the Executive (Ground Services) is able to develop action plans and provide recommendations to address any identified service delivery gaps. He develops strong rapport with the stakeholders by recognising their interests ...</code> | <code>0</code> |
+* Loss: [<code>CrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#crossentropyloss)
+
+### Evaluation Dataset
+
+#### cxs-eval-dataset
+
+* Dataset: [cxs-eval-dataset](https://huggingface.co/datasets/dnth/cxs-eval-dataset) at [a3a8a16](https://huggingface.co/datasets/dnth/cxs-eval-dataset/tree/a3a8a167ee1ca5fd48f6c99cff7b61fc756fbb3f)
+* Size: 141 evaluation samples
+* Columns: <code>query</code>, <code>result</code>, and <code>label</code>
+* Approximate statistics based on the first 141 samples:
+  |         | query                                                                                              | result                                                                                              | label                                           |
+  |:--------|:---------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------|:------------------------------------------------|
+  | type    | string                                                                                             | string                                                                                              | int                                             |
+  | details | <ul><li>min: 193 characters</li><li>mean: 655.76 characters</li><li>max: 2255 characters</li></ul> | <ul><li>min: 556 characters</li><li>mean: 1091.27 characters</li><li>max: 1810 characters</li></ul> | <ul><li>0: ~77.30%</li><li>1: ~22.70%</li></ul> |
+* Samples:
+  | query                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | result                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | label          |
+  |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
+  | <code>Inflight Quality Assurance Inspector - As an Inflight Quality Assurance Inspector at Santan, you are tasked with ensuring the accuracy and efficiency of meal and product handling for inflight services. Your responsibilities include verifying cart quantities, monitoring picking and staging processes, and maintaining inventory through CMS and WMS systems. You'll oversee packers and warehouse agents, manage stock adjustments and wastage reporting, and ensure compliance with FIFO and quality assurance standards. Key tasks include verifying outbound and inbound cart quantities, ensuring accurate meal picking, confirming seal and flight details in CMS, performing inventory counts, monitoring product packing, and facilitating effective communication between all parties involved. Additionally, you'll manage stock movements, update supervisors on stock issues, and ensure timely mass production by Packers or Warehouse Agents. - Air Transport</code> | <code>Manager (Cargo) - The Manager (Cargo) leads the development of operational strategies to improve the effectiveness of air cargo and warehouse operations. He/She is responsible for driving continuous improvement and business development initiatives to optimise processes and meet customer needs. He develops Standard Operating Procedures (SOPs) and systems to mitigate safety and/or security risks and monitors the adherence to safety and/or security standards. He also advises authorities and other teams to manage incidents involving hazardous materials and/or dangerous goods. He develops the teams technical capabilities and maintains positive morale within the teams.<br><br>The Manager (Cargo) has an extensive knowledge of cargo handling procedures, international airfreight regulations and policy requirements. He is also well-versed in organisational safety regulations and practices. He possesses strong interpersonal and stakeholder management skills in order to build and maintain stakeholder re...</code> | <code>0</code> |
+  | <code>Manager, Pricing Systems - Manage the systems and tools that power our pricing and revenue management strategies. As Manager of Pricing Systems, you will oversee the implementation, maintenance, and enhancement of our pricing engines and related software. You'll work with IT and commercial teams to ensure system integrity, support new pricing initiatives, and troubleshoot issues. Technical expertise in airline pricing systems is a must. - Air Transport</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | <code>Terminal Manager - The Terminal Manager is responsible for planning the delivery of airport services and implementing projects to improve the airport user experience. He/She collaborates with internal and external stakeholders to develop and establish compliance standards for airport operations. Besides leading research to identify new service offerings for the airport, he also enforces safety and/or security standards for the organisation. He sources for contractors to perform airport operations and establishes key indicators to track their performances. He also develops on-the-job training programmes for the department.<br><br>The Terminal Manager possesses a sound knowledge of airport and airside operations, Standard Operating Procedures (SOPs) and airport security and emergency response plans. He travels across the terminals to review operational processes and resolve incidents at the airport. To ensure operational plans and projects are implemented efficiently and effectively, he displa...</code> | <code>0</code> |
+  | <code>Executive, Inflight Operation Team Lead - As an Executive, Inflight Operation Team Lead, you will excel in leading the Hi-Lift Driver team by demonstrating expertise in operational leadership while upholding high standards of safety and quality. You will contribute significantly to departmental and company objectives, ensuring strict adherence to industry regulations and operational procedures. Your role champions driving excellence in operational and engineering tasks, supporting initiatives to boost efficiency, and collaborating with cross-functional teams to achieve organizational goals, all while ensuring compliance with regulatory and organizational standards. - Air Transport</code>                                                                                                                                                                                                                                                                 | <code>Executive (Ramp / Technical Ramp) - The Executive (Ramp/Technical Ramp) performs regular audits to ensure that ramp/technical ramp operations comply with Standard Operating Procedures (SOPs). He/She recommends enhancements to address areas of non-compliance, improves operational efficiencies and develops action plans to prevent safety and/or security issues from recurring. He also develops workplace learning plans and on-the-job training programmes to enhance the capabilities of teams.<br> <br>The Executive (Ramp/Technical Ramp) has a sound knowledge of airport operations and ramp/technical ramp procedures. He also has strong analytical abilities to identify root causes of ground handling related delays and their potential implications. He is results-oriented and possesses excellent communication and interpersonal skills. Furthermore, he has strong computer literacy skills and competence in learning orientation to determine learning and developmental needs of his team members.</code>                   | <code>0</code> |
+* Loss: [<code>CrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#crossentropyloss)
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: epoch
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 32
+- `learning_rate`: 4e-05
+- `weight_decay`: 0.01
+- `num_train_epochs`: 10
+- `warmup_ratio`: 0.1
+- `bf16`: True
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: epoch
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 32
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 4e-05
+- `weight_decay`: 0.01
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 10
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch   | Step    | Training Loss | Validation Loss | validation-set_f1_macro |
+|:-------:|:-------:|:-------------:|:---------------:|:-----------------------:|
+| 0.3521  | 25      | 0.5365        | -               | -                       |
+| 0.7042  | 50      | 0.5596        | -               | -                       |
+| 1.0     | 71      | -             | 0.5498          | 0.5324                  |
+| 1.0563  | 75      | 0.5252        | -               | -                       |
+| 1.4085  | 100     | 0.4624        | -               | -                       |
+| 1.7606  | 125     | 0.4534        | -               | -                       |
+| 2.0     | 142     | -             | 0.5547          | 0.4649                  |
+| 2.1127  | 150     | 0.4883        | -               | -                       |
+| 2.4648  | 175     | 0.3718        | -               | -                       |
+| 2.8169  | 200     | 0.4136        | -               | -                       |
+| 3.0     | 213     | -             | 0.4693          | 0.5576                  |
+| 3.1690  | 225     | 0.3431        | -               | -                       |
+| 3.5211  | 250     | 0.2573        | -               | -                       |
+| 3.8732  | 275     | 0.3497        | -               | -                       |
+| 4.0     | 284     | -             | 0.3733          | 0.7594                  |
+| 4.2254  | 300     | 0.2694        | -               | -                       |
+| 4.5775  | 325     | 0.2604        | -               | -                       |
+| 4.9296  | 350     | 0.1834        | -               | -                       |
+| 5.0     | 355     | -             | 0.5547          | 0.7321                  |
+| 5.2817  | 375     | 0.1862        | -               | -                       |
+| 5.6338  | 400     | 0.1219        | -               | -                       |
+| 5.9859  | 425     | 0.1605        | -               | -                       |
+| 6.0     | 426     | -             | 0.7399          | 0.7177                  |
+| 6.3380  | 450     | 0.1093        | -               | -                       |
+| 6.6901  | 475     | 0.1003        | -               | -                       |
+| 7.0     | 497     | -             | 1.2497          | 0.7320                  |
+| 7.0423  | 500     | 0.0885        | -               | -                       |
+| 7.3944  | 525     | 0.0698        | -               | -                       |
+| 7.7465  | 550     | 0.0627        | -               | -                       |
+| **8.0** | **568** | **-**         | **1.2357**      | **0.7749**              |
+| 8.0986  | 575     | 0.0088        | -               | -                       |
+| 8.4507  | 600     | 0.0001        | -               | -                       |
+| 8.8028  | 625     | 0.0296        | -               | -                       |
+| 9.0     | 639     | -             | 1.6408          | 0.7530                  |
+| 9.1549  | 650     | 0.0098        | -               | -                       |
+| 9.5070  | 675     | 0.0124        | -               | -                       |
+| 9.8592  | 700     | 0.0012        | -               | -                       |
+| 10.0    | 710     | -             | 1.6398          | 0.7481                  |
+
+* The bold row denotes the saved checkpoint.
+
+### Framework Versions
+- Python: 3.12.11
+- Sentence Transformers: 4.1.0
+- Transformers: 4.53.0
+- PyTorch: 2.7.1+cu126
+- Accelerate: 1.8.1
+- Datasets: 3.6.0
+- Tokenizers: 0.21.2
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,49 @@
+{
+  "architectures": [
+    "ModernBertForSequenceClassification"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "repad_logits_with_grad": false,
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.linear.Identity",
+    "version": "4.1.0"
+  },
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.53.0",
+  "vocab_size": 50368
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d0018266efbf34818d0deb715a25dfcd75f1313d2194a47a39e64bfa98458b2
+size 598439784

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}