frankwong2001/1_attempt_mxbai-embed-large-v1

SentenceTransformer based on mixedbread-ai/mxbai-embed-large-v1

This is a sentence-transformers model finetuned from mixedbread-ai/mxbai-embed-large-v1 on the ssf-train-valid-full-synthetic-batch10 dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: mixedbread-ai/mxbai-embed-large-v1
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-full-synthetic-batch10

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("frankwong2001/1_attempt_mxbai-embed-large-v1")
# Run inference
queries = [
    "The Operations Risk and Control Manager is responsible for managing risk and control activities for the organisation and ensuring compliance with any applicable guidelines, laws and regulations. He/She will monitor high risk operational and emerging risk incidents with the aim of strengthening the organisation\u0027s control environment and improving control processes. He conducts investigations to identify risk incidents and determine corrective actions, and develops incident response and crisis management protocols to deal with potential emergencies. The Operations Risk and Control Manager possesses analytical capabilities and a keen eye for pinpointing sources of risks or potential crises. He is a quick thinker who is able to make decisions under tight timelines so as to address and resolve risk incidents as they arise and adapt to the changing regulatory environment.",
]
documents = [
    'The Operations Risk and Control Manager is tasked with overseeing risk and control measures within the organization, ensuring adherence to relevant guidelines, laws, and regulations. He/She will assess high-risk operational incidents and emerging threats to enhance the control framework and refine control processes. He conducts thorough investigations to pinpoint risk occurrences and formulate corrective measures, while also developing incident response and crisis management strategies for potential emergencies. The Operations Risk and Control Manager has strong analytical skills and is adept at identifying sources of risk or potential crises. He is a decisive thinker who can make timely decisions to address and resolve risk incidents as they emerge, adapting to the evolving regulatory landscape.',
    'The Operations Compliance Manager is responsible for overseeing compliance and audit processes for the organization while ensuring alignment with various industry standards and practices. He/She will evaluate low-risk operational activities and existing compliance issues to enhance the compliance framework and streamline audit processes. He conducts reviews to assess compliance violations and suggests improvements, while also creating compliance training and awareness programs for all employees. The Operations Compliance Manager possesses strong organizational skills and is effective in identifying areas of improvement or compliance gaps. He is a strategic planner who can implement changes to enhance compliance measures over time, adapting to the shifting market trends.',
    'The Arts Educators are responsible for designing, implementing, and evaluating learning experiences while utilizing effective assessment techniques to ensure that learners meet established standards. Their teaching is enriched by their own artistic practice in their selected art form. With a solid grasp of effective teaching methodologies and learning strategies, they skillfully adjust these approaches to cater to specific contexts, student needs, and educational goals. They guide learners in realizing their full potential in their craft and deepening their understanding and appreciation of artistic endeavors. Arts Educators foster creativity and equip students with the necessary tools to explore their ideas and imagination. They deliver arts education programs across various settings, including schools, universities, community centers, welfare organizations, and co-curricular activities, serving a diverse range of students. They are committed to enhancing arts education through the development and refinement of pedagogies, programs, and curricula. Additionally, they actively engage with arts and arts education organizations while mentoring emerging artists. They engage in self-reflection and adopt a critical approach to their teaching and artistic practice, often developing a distinctive teaching style that reflects their individuality.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.9665, 0.4572, 0.1590]])

Training Details

Training Dataset

ssf-train-valid-full-synthetic-batch10

• Dataset: ssf-train-valid-full-synthetic-batch10 at b687585
• Size: 4,524 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 54 tokens mean: 168.61 tokens max: 404 tokens	min: 57 tokens mean: 163.11 tokens max: 369 tokens	min: 18 tokens mean: 135.91 tokens max: 374 tokens

• Samples:

  anchor	positive	negative
  The Multi-Utility Operations Team Leader leads the day-to-day power plant operations by assigning tasks to junior team members, performs high voltage switching operational works and drives the rectification of all major plant faults, defects and outages. He/She supervises the first line maintenance works. He develops staff capabilities through on-the-job training and coaching. He monitors Permits-to-Work procedures, and ensures works are done according to Safe System of Work (SSoW) practices. In times of emergency, he facilitates the implementation of emergency response plans and relevant safety procedures. He also supervises the Emergency Response Team on site incident management. He works at the power plant station and may be required to perform shift work. He possesses good leadership and interpersonal skills in leading the operations teams. He is also systematic and able to respond to situations quickly in times of faults or outages.	The Multi-Utility Operations Team Leader is responsible for managing the daily operations of the power plant by delegating tasks to junior team members, executing high voltage switching operations, and addressing all significant plant faults, defects, and outages. He/She oversees first line maintenance activities and enhances staff capabilities through on-the-job training and coaching. He monitors Permits-to-Work procedures to ensure compliance with Safe System of Work (SSoW) practices. In emergencies, he facilitates the execution of emergency response plans and relevant safety protocols, while also supervising the Emergency Response Team during on-site incidents. He works at the power plant station and may be required to perform shift work. He demonstrates strong leadership and interpersonal skills in guiding the operations teams and is systematic, responding swiftly to faults or outages.	The Multi-Utility Operations Team Supervisor manages the daily logistics for the distribution center by assigning tasks to assistant staff, oversees low voltage electrical installation projects, and addresses all minor warehouse issues and delays. He/She coordinates routine inventory checks and enhances staff efficiency through training sessions and workshops. He monitors compliance with shipping regulations and ensures operations adhere to standard operating procedures (SOP). In critical situations, he facilitates the execution of logistical plans and relevant operational protocols, while also supervising the Inventory Management Team during stock assessments. He works at the distribution center and may be required to perform regular office hours. He demonstrates excellent organizational and communication skills in managing the logistics teams and is methodical, adapting quickly to challenges or delays.
  The Technician (Component Repair & OverhaulMechanical) performs maintenance, repair and overhaul (MRO) tasks for aircraft components in accordance with technical manuals and standard operating procedures (SOPs). He/She examines parts for maintenance, repair or replacement. He/She troubleshoots component defects and takes corrective actions to restore components to the desired performance requirements. He also performs special processes and repair of composite structures, and documents all completed tasks. He may be authorised by the organisation to perform quality control functions, including inspection of incoming materials and outgoing serviced items, and registration of non-conformances. He may also be authorised to perform level 1 non-destructive testing (NDT) functions under supervision, perform evaluations for acceptance or rejection of aircraft components, and record results as specified in the work instructions. He complies with airworthiness and legislative requirements, and t...	The Technician (Component Repair & Overhaul Mechanical) is responsible for performing maintenance, repair, and overhaul (MRO) activities on aircraft components according to technical manuals and standard operating procedures (SOPs). He/She inspects parts for maintenance, repair, or replacement needs, troubleshoots component defects, and implements corrective actions to ensure components meet performance standards. Additionally, he/she carries out special processes and repairs of composite structures while documenting all completed tasks. The technician may also be authorized to conduct quality control functions, such as inspecting incoming materials and outgoing serviced items, as well as registering non-conformances. Furthermore, he/she may perform level 1 non-destructive testing (NDT) functions under supervision, evaluate aircraft components for acceptance or rejection, and record results as outlined in work instructions. He/She adheres to airworthiness and legislative requirements, ...	The Chef prepares gourmet meals and creates unique recipes for a fine dining restaurant. He/She manages kitchen staff, ensures food safety standards are met, and collaborates with suppliers to source fresh ingredients. Additionally, he/she designs menus that highlight seasonal produce and oversees the presentation of dishes to enhance customer experience. The chef conducts food tastings and works to innovate culinary techniques, while maintaining a clean and organized kitchen environment. He/She may also participate in promotional events to showcase the restaurant's offerings and engage with guests.
  The Relationship Management Director - Small and Medium Enterprises is responsible for defining strategies for team members to achieve mass sales acquisition. He/She provides oversight to due diligence, compliance and Anti-Money Laundering (AML) processes carried out by team members. He sets policies and guidelines for ongoing support processes pertaining to credit responsibilities. He guides his team to achieve their performance targets and ensures they have the training necessary to deliver on their responsibilities. The Relationship Management Director - Small and Medium Enterprises is a strong leader who provides mentoring and coaching to his team members to allow them to succeed in their roles. He is a strong communicator with internal and external stakeholders. He is always looking for opportunities to provide enhanced services to clients. He uses analytics and problem solving capabilities to foster an environment that will yield results. He is accountable for the defined standar...	The Relationship Management Director - Small and Medium Enterprises is tasked with developing strategies that enable team members to achieve significant sales growth. He/She supervises the due diligence, compliance, and Anti-Money Laundering (AML) procedures executed by the team. He establishes policies and guidelines for ongoing support processes related to credit responsibilities. He mentors his team to meet their performance goals and ensures they receive the necessary training to fulfill their duties. The Relationship Management Director - Small and Medium Enterprises is an effective leader who provides guidance and support to help his team thrive in their positions. He excels in communication with both internal and external stakeholders. He consistently seeks opportunities to enhance client services. He leverages analytics and problem-solving skills to create a results-oriented environment. He is responsible for upholding the standards he sets for his team.	The Relationship Management Director - Large Enterprises is responsible for creating strategies for team members to achieve substantial market share. He/She oversees the financial audits, regulatory compliance, and Anti-Bribery measures conducted by team members. He formulates policies and frameworks for ongoing management processes relating to financial responsibilities. He directs his team to exceed their sales targets and ensures they have the resources needed to perform their duties. The Relationship Management Director - Large Enterprises is a proactive leader who offers training and support to his team members to enable them to excel in their functions. He is an effective communicator with clients and vendors. He frequently identifies opportunities to improve operational efficiencies. He utilizes data analysis and strategic planning to cultivate an environment that fosters success. He is responsible for the established benchmarks he sets for his team.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid-full-synthetic-batch10

• Dataset: ssf-train-valid-full-synthetic-batch10 at b687585
• Size: 1,131 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 64 tokens mean: 169.57 tokens max: 348 tokens	min: 62 tokens mean: 163.13 tokens max: 331 tokens	min: 21 tokens mean: 135.5 tokens max: 323 tokens

• Samples:

  anchor	positive	negative
  The Assistant Equipment Engineer applies engineering principles and techniques to support equipment engineering processes in a manufacturing environment to meet organisational objectives. He/She also assists in analysing equipment maintenance issues. In addition, the Assistant Equipment Engineer participates in equipment improvement projects, and partakes in the development of maintenance plans in accordance with organisational objectives. The Assistant Equipment Engineer is required to have strong communication skills, good teamwork and an analytical mind to perform his role well to achieve the desired organisational outcomes.	The Assistant Equipment Engineer utilizes engineering principles and techniques to enhance equipment engineering processes within a manufacturing setting, aligning with organizational goals. He/She also aids in evaluating equipment maintenance challenges. Furthermore, the Assistant Equipment Engineer engages in equipment enhancement initiatives and contributes to the formulation of maintenance strategies in line with organizational objectives. Strong communication skills, effective teamwork, and analytical thinking are essential for the Assistant Equipment Engineer to succeed in achieving the desired organizational results.	The Assistant Mechanical Engineer employs design principles and techniques to assist mechanical engineering tasks in a construction environment to fulfill project requirements. He/She also helps in reviewing machinery performance issues. Additionally, the Assistant Mechanical Engineer takes part in machinery optimization projects and contributes to the creation of operational strategies that meet project goals. Strong leadership abilities, effective collaboration, and critical thinking are necessary for the Assistant Mechanical Engineer to excel in reaching the intended project outcomes.
  The Brokerage Supervisor/ Freight Supervisor is responsible for liaising with customers, logistics operators and customs officials and supervising the custom clearance/freight forwarding operations to ensure goods are cleared through customs or quarantine in accordance with import and export laws and regulations. Analytical and systematic, he/she is required to supervise a freight operations team to execute operations in a timely manner to meet business and customers' requirements. He/She is also expected to work with internal and external stakeholders to accomplish his work.	The Brokerage Supervisor/Freight Supervisor is tasked with coordinating with customers, logistics providers, and customs authorities while overseeing the customs clearance and freight forwarding processes to ensure that goods comply with import and export regulations. With a strong analytical and systematic approach, he/she leads a freight operations team to execute tasks promptly, meeting both business and customer needs. Additionally, he/she collaborates with internal and external stakeholders to achieve work objectives.	The Freight Operations Manager is responsible for interacting with suppliers, transportation companies, and regulatory agencies while managing the delivery and logistics services to guarantee that products adhere to supply chain protocols. With a focus on detail-oriented and organized practices, he/she directs a logistics team to carry out operations efficiently, fulfilling both company and supplier expectations. Furthermore, he/she engages with internal and external partners to fulfill his/her duties.
  The Production Planner is responsible for managing and executing production plans and schedules to ensure that products are delivered to customers on time and within schedule. He/She plans for the entire production supply chain from feedstock to production, storage and distribution, and analyses production data to optimise production and inventory control. The Production Planner coordinates with the maintenance planning team to align production targets with the planning of maintenance and turnaround schedules. He supports the reporting of plant production status and raw materials inventories, and highlights issues that may affect production output. He monitors feedstock movement to ensure minimal interruption to the production schedule. In addition, he identifies opportunities for continuous improvement in the organisations supply chain operations. The Production Planner works closely with the production, maintenance planning, sales and logistics teams, and interfaces with suppliers an...	The Production Planner is tasked with overseeing and implementing production schedules to guarantee timely delivery of products to customers. He/She is responsible for planning the complete production supply chain, from the initial feedstock to production, storage, and distribution, while analyzing production data to enhance production efficiency and inventory management. The Production Planner collaborates with the maintenance planning team to synchronize production objectives with maintenance and turnaround schedules. He supports the reporting of plant production status and raw material inventories, addressing any issues that could impact production output. He ensures smooth feedstock movement to minimize disruptions to the production timeline and identifies opportunities for ongoing improvements in the organization's supply chain operations. The Production Planner works in close partnership with the production, maintenance planning, sales, and logistics teams, while also engaging wi...	The Software Developer creates applications and software solutions tailored to meet client needs, focusing on coding, debugging, and testing software programs. He/She collaborates with cross-functional teams to design user-friendly interfaces and enhance user experience. The Software Developer is responsible for maintaining and updating existing software, ensuring optimal performance and security standards are met. He conducts code reviews and provides technical support to other team members while staying updated on the latest industry trends and technologies.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• gradient_checkpointing: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	9	0.0499	0.0028
2.0	18	0.0089	0.0013
3.0	27	0.0038	0.0009
4.0	36	0.0031	0.0007
5.0	45	0.0034	0.0007

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}