# Utilizing Semantic Textual Similarity for Clinical Survey Data Feature
Selection: Supplementary Material

###### Contents

  1. Feature Importance By Selection Algorithm
    1. Top 20 Scores, MI
    2. Top 20 Scores, STS
    3. Top 20 Scores, MI & STS
    4. mRMR, MI
    5. mRMR, STS
    6. mRMR, MI & STS
    7. $\mu+1\sigma$, STS
    8. $\mu+1\sigma$, STS
    9. $\mu+1\sigma$, MI & STS
  2. Performance of All Feature Selection & Classifier Combinations
    1. Linear support vector machine (SVM)
    2. XGBoost
    3. MLP
    4. Gaussian Naïve Bayes (Gaussian NB)
    5. $k$-nearest neighbors ($k$-NN)

## Feature Importance By Selection Algorithm

This section contains the results for each of our selection algorithms that
use mutual information (MI), semantic textual similarity (STS), and a linear
combination of both. In all cases, we generate STS with PubMedBERT fine-tuned
on the combined vocabulary. In addition, we provide SHapley Additive
exPlanations (SHAP) values on a multilayer perceptron (MLP) model trained with
the filtered features.

### Top 20 Scores, MI

Table 1: Feature importance using MLP using MI selecting 20 features.  Feature Name | $n$ |  Relevancy | Mean Absolute SHAP
---|---|---|---
Current Medications: Confirmed with participant (choice=Metformin)_Checked | 1 |  0.0722903 | 1.37293772e-09
Color Score (C), Raw Score | 2 |  0.0696211 | 1.14889901e-07
Please specify your race: (choice=Black / African Heritage)_Checked | 3 |  0.0668161 | 0
BRS Score | 4 |  0.0660192 | 0
Color-Word Score (CW), Residual | 5 |  0.0639546 | 0
Please specify your race: (choice=Prefer not to answer)_Checked | 6 |  0.0638041 | 0
Current Medications: Confirmed with participant (choice=Metformin)_Unchecked | 7 |  0.0634184 | 0
Other:.1_married; polyam | 8 |  0.062555 | 0
Current Medications: Confirmed with participant (choice=SNRI antidepressants (venlafaxine, duloxetine, desvenlafaxine, milnacipran))_Unchecked | 9 |  0.0619615 | 0
Current Medications: Confirmed with participant (choice=None of the above)_Unchecked | 10 |  0.0611848 | 0
Magnification Score | 11 |  0.0591411 | 0
Color Score (C), Age/Ed. Predicted | 12 |  0.0588759 | 7.82877036e-09
Please specify your race: (choice=Hawaiian Native / Other Pacific Islander)_Checked | 13 |  0.0584943 | 0
Current Medications: Confirmed with participant (choice=Opioids (morphine, oxycodone, fentanyl, tramadol, methadone, buprenorphine))_Checked | 14 |  0.0583059 | 0
Total Years of Education | 15 |  0.0581167 | 3.84597352e-09
Have you ever been diagnosed with chronic pain?_No | 16 |  0.0579427 | 0
Helplessness Score | 17 |  0.0578844 | 2.42765435e-06
Current Medications: Confirmed with participant (choice=None of the above)_Checked | 18 |  0.0577642 | 0
Word Score (W), Raw Score | 19 |  0.0576109 | 2.98490162e-07
Current Medications: Confirmed with participant (choice=SSRI antidepressants (citalopram, escitalopram, fluoxetine, fluvoxamine, paroxetine, sertraline))_Unchecked | 20 |  0.0574527 | 0

### Top 20 Scores, STS

Table 2: Feature importance using MLP using STS scoring with PubMedBERT fine-tuned on the combined vocab selecting 20 features.  Feature Name | $n$ |  Relevancy | Mean Absolute SHAP
---|---|---|---
On a scale of zero to ten, with zero being no pain and ten being the worst pain, please mark your average pain level during the past week, while at rest. | 1 |  0.645585 | 1.46336854e-06
On a scale of zero to ten, with zero being no pain and ten being the worst pain , please mark your average pain level during the past week, while you were active or moving. | 2 |  0.645342 | 4.46294155e-07
Have you been experiencing pain in the past week?_No | 3 |  0.639949 | 0.00279322
Have you been experiencing pain in the past week?_Yes, and RELATED to my need for surgery | 4 |  0.638486 | 0.00297259
Have you been experiencing pain in the past week?_Yes, but NOT RELATED to my need for surgery | 5 |  0.634165 | 0.00894911
On a scale of zero to ten, with zero being no pain and ten being the worst pain, what is your average pain level at rest TODAY? | 6 |  0.619892 | 4.25537234e-07
In the past three months, did you experience daily or near daily pain?_No | 7 |  0.544496 | 0.01075854
Have you ever been diagnosed with chronic pain?_No | 8 |  0.534078 | 0.01045607
Have you ever been diagnosed with chronic pain?_Yes | 9 |  0.531654 | 0.00238557
In the past three months, did you experience daily or near daily pain?_Yes | 10 |  0.526172 | 0.00067493
Pain intensity - Baseline Average: | 11 |  0.525021 | 1.61967588e-06
Pain intensity - Baseline 1: | 12 |  0.487709 | 2.1284362e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=None)_Unchecked | 13 |  0.478695 | 0.00369703
Pain intensity - Baseline 2: | 14 |  0.475303 | 1.34237724e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Other)_Unchecked | 15 |  0.475231 | 0.00833284
Do you take any of the following medications for pain treatment at least once per week? (choice=None)_Checked | 16 |  0.461651 | 0.00497855
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Unchecked | 17 |  0.449812 | 0.0008251
Have you used opioid medications for pain management in the past?_I don’t know | 18 |  0.44269 | 0.00038904
Have you used opioid medications for pain management in the past?_yes | 19 |  0.43422 | 0.00495188
Do you take any of the following medications for pain treatment at least once per week? (choice=Other)_Checked | 20 |  0.430055 | 0.00134032

### Top 20 Scores, MI & STS

Table 3: Feature importance using MLP using linear scoring with PubMedBERT fine-tuned on the combined vocab selecting 20 features.  Feature Name | $n$ |  Relevancy | Mean Absolute SHAP
---|---|---|---
Have you ever been diagnosed with chronic pain?_No | 1 |  0.734467 | 0.01120194
Have you ever been diagnosed with chronic pain?_Yes | 2 |  0.712339 | 0.00546406
Have you been experiencing pain in the past week?_No | 3 |  0.599768 | 0.00223598
Have you been experiencing pain in the past week?_Yes, and RELATED to my need for surgery | 4 |  0.598852 | 5.00820274e-07
Have you been experiencing pain in the past week?_Yes, but NOT RELATED to my need for surgery | 5 |  0.529267 | 1.79035372e-06
In the past three months, did you experience daily or near daily pain?_Yes | 6 |  0.525755 | 0.00294529
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Unchecked | 7 |  0.512279 | 0.00100577
In the past three months, did you experience daily or near daily pain?_No | 8 |  0.504445 | 0.00930347
Helplessness Score | 9 |  0.493851 | 5.01834836e-07
Pain intensity - Baseline 2: | 10 |  0.479731 | 9.24779374e-07
Pain intensity - Baseline Average: | 11 |  0.478815 | 5.96666072e-07
Do you take any of the following medications for pain treatment at least once per week? (choice=Other)_Unchecked | 12 |  0.472314 | 0.00909019
Have you used opioid medications for pain management in the past?_yes | 13 |  0.466008 | 0.00462665
Pain intensity - CPM 1: | 14 |  0.457419 | 4.55048919e-07
Do you take any of the following medications for pain treatment at least once per week? (choice=None)_Unchecked | 15 |  0.455803 | 0.00129691
Pain intensity - CPM 2: | 16 |  0.444577 | 1.87985214e-06
Pain intensity - CPM Average: | 17 |  0.439207 | 4.87195489e-07
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Unchecked | 18 |  0.438722 | 0.00499209
Have you used opioid medications for pain management in the past?_I don’t know | 19 |  0.43665 | 0.0006287
Have you used opioid medications for pain management in the past?_no | 20 |  0.433569 | 3.60313277e-05

###  mRMR, MI

Table 4: Feature importance using MLP using mRMR with MI scores with 20 features Feature Name | $n$ |  mRMR Score | Relevancy | Redundancy |  Mean Absolute SHAP
---|---|---|---|---|---
Current Medications: Confirmed with participant (choice=Metformin)_Checked | 1 |  0.0723 | 0.0723 | 0 |  1.80786083e-07
Please specify your race: (choice=Black / African Heritage)_Checked | 2 |  0.0668 | 0.0668 | 0 |  1.31254826e-07
Please specify your race: (choice=Hawaiian Native / Other Pacific Islander)_Checked | 3 |  0.0585 | 0.0585 | 0 |  2.02387005e-07
Please specify your race: (choice=American Indian / Alaskan Native)_Checked | 4 |  0.0523 | 0.0523 | 0 |  1.2439402e-07
Baseline Emotional Distress-Anxiety - Short Form 4a T Score | 5 |  0.0461 | 0.0461 | 0 |  2.17081369e-07
Baseline Cognitive Function - Abilities - Short Form 4a T Score | 6 |  0.0119 | 0.0352 | 0.0232 |  1.46263805e-07
Please specify your ethnicity:_Non-Hispanic | 7 |  -0.0011 | 0.0227 | 0.0238 |  0.00047083
Did you find submerging your hand in the cold water bath to be:_Not Painful | 8 |  -0.0049 | 0.0497 | 0.0545 |  3.61855009e-07
Pain intensity - Baseline 2: | 9 |  -0.0145 | 0.0474 | 0.0619 |  3.04670314e-07
Pain intensity - CPM Average: | 10 |  -0.012 | 0.04 | 0.052 |  3.40698218e-07
Baseline Emotional Distress-Depression - Short Form 4a T Score | 11 |  -0.016 | 0.024 | 0.04 |  4.35499288e-07
How often have you been using cannabis (marijuana, pot, weed, grass) in the past year?_Never | 12 |  -0.0286 | 0.0408 | 0.0694 |  0.00359785
Pain intensity - CPM 1: | 13 |  -0.0479 | 0.0446 | 0.0924 |  2.01787643e-07
Moderate activities such as moving a table, pushing a vacuum cleaner, bowling, or playing golf?_Yes limited a lot | 14 |  -0.0585 | 0.0203 | 0.0789 |  0.00129293
Climbing several flights of stairs?_Yes limited a lot | 15 |  -0.0591 | 0.011 | 0.0701 |  0.00302154
Why are you having this upcoming procedure? Please mark all that apply. Please mark all that apply. (choice=My doctor said I needed the procedure)_Unchecked | 16 |  -0.0536 | 0.0293 | 0.0829 |  0.00573359
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Unchecked | 17 |  -0.0511 | 0.0502 | 0.1013 |  0.00229605
Did you find submerging your hand in the cold water bath to be:_Mildly Painful | 18 |  -0.0524 | 0.025 | 0.0773 |  6.09845743e-07
Have you ever been diagnosed with anxiety?_No | 19 |  -0.0599 | 0.0289 | 0.0888 |  1.61796364e-06
Have you ever been diagnosed with anxiety?_Yes | 20 |  -0.0611 | 0.0263 | 0.0874 |  3.27162909e-07

###  mRMR, STS

Table 5: Feature importance using MLP using mRMR with STS scores from PubMedBERT fine-tuned on the combined vocab with 20 features. Feature Name | $n$ |  mRMR | Relevancy | Redundancy |  Mean Absolute SHAP
---|---|---|---|---|---
On a scale of zero to ten, with zero being no pain and ten being the worst pain, please mark your average pain level during the past week, while at rest. | 1 |  0.6456 | 0.6456 | 0 |  1.43756747e-06
On a scale of zero to ten, with zero being no pain and ten being the worst pain , please mark your average pain level during the past week, while you were active or moving. | 2 |  0.6453 | 0.6453 | 0 |  1.76442483e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Aspirin)_Unchecked | 3 |  0.2493 | 0.397 | 0.1477 |  0.00140821
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Checked | 4 |  0.1669 | 0.3962 | 0.2294 |  0.00218598
Have you ever been diagnosed with chronic pain?_No | 5 |  0.1206 | 0.5341 | 0.4135 |  0.01126774
Have you ever been diagnosed with chronic pain?_Yes | 6 |  0.2025 | 0.5317 | 0.3291 |  0.00169776
In the past three months, did you experience daily or near daily pain?_Yes | 7 |  0.1023 | 0.5262 | 0.4239 |  0.00016405
Have you been experiencing pain in the past week?_Yes, but NOT RELATED to my need for surgery | 8 |  0.1518 | 0.6342 | 0.4823 |  0.00909981
On a scale of zero to ten, with zero being no pain and ten being the worst pain, what is your average pain level at rest TODAY? | 9 |  0.114 | 0.6199 | 0.5059 |  2.0828322e-07
Have you been experiencing pain in the past week?_No | 10 |  0.0777 | 0.6399 | 0.5622 |  0.00350571
Rumination Score | 11 |  0.0543 | 0.2666 | 0.2123 |  2.4462886e-06
Have you been experiencing pain in the past week?_Yes, and RELATED to my need for surgery | 12 |  0.0627 | 0.6385 | 0.5758 |  0.0041694
Helplessness Score | 13 |  0.0643 | 0.3075 | 0.2433 |  1.75271276e-06
Pain intensity - Baseline Average: | 14 |  0.0722 | 0.525 | 0.4528 |  2.71861889e-06
In the past three months, did you experience daily or near daily pain?_No | 15 |  0.0952 | 0.5445 | 0.4493 |  0.00994588
Pain intensity - Baseline 1: | 16 |  0.0679 | 0.4877 | 0.4199 |  1.89465065e-07
Pain intensity - Baseline 2: | 17 |  0.0749 | 0.4753 | 0.4004 |  1.72906497e-06
Why are you having this upcoming procedure? Please mark all that apply. Please mark all that apply. (choice=Decrease pain)_Unchecked | 18 |  0.0809 | 0.3686 | 0.2877 |  2.52886547e-06
Why are you having this upcoming procedure? Please mark all that apply. Please mark all that apply. (choice=Decrease pain)_Checked | 19 |  0.0727 | 0.3482 | 0.2756 |  0
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Unchecked | 20 |  0.0795 | 0.4162 | 0.3368 |  0.00644815

###  mRMR, MI & STS

Table 6: Feature importance using MLP using linear scoring with PubMedBERT fine-tuned on the combined vocab selecting 20 features.  Feature Name | $n$ |  mRMR | Relevancy | Redundancy |  Mean Absolute SHAP
---|---|---|---|---|---
Have you ever been diagnosed with chronic pain?_No | 1 |  0.7345 | 0.7345 | 0 |  0.01284267
Have you ever been diagnosed with chronic pain?_Yes | 2 |  0.7123 | 0.7123 | 0 |  0.00255372
Baseline Emotional Distress-Anxiety - Short Form 4a | 3 |  0.1982 | 0.3735 | 0.1753 |  2.61222337e-05
In the past three months, did you experience daily or near daily pain?_Yes | 4 |  0.158 | 0.5258 | 0.3677 |  0.00154897
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Unchecked | 5 |  0.1109 | 0.5123 | 0.4013 |  0.00176415
Baseline Emotional Distress-Anxiety - Short Form 4a T Score | 6 |  0.1008 | 0.3454 | 0.2446 |  1.33594353e-05
Have you used opioid medications for pain management in the past?_I don’t know | 7 |  0.0502 | 0.4366 | 0.3865 |  0.00153403
Have you been experiencing pain in the past week?_No | 8 |  0.0596 | 0.5998 | 0.5402 |  0.00508954
Please specify your ethnicity:_Hispanic | 9 |  0.0652 | 0.2196 | 0.1545 |  1.19111503e-05
On a scale of zero to ten, with zero being no pain and ten being the worst pain, what is your average pain level at rest TODAY? | 10 |  0.0667 | 0.3974 | 0.3307 |  1.6230635e-05
Have you been experiencing pain in the past week?_Yes, and RELATED to my need for surgery | 11 |  0.0858 | 0.5989 | 0.513 |  2.27270858e-05
Pain intensity - Baseline 2: | 12 |  0.0716 | 0.4797 | 0.4081 |  1.82589818e-05
In the past three months, did you experience daily or near daily pain?_No | 13 |  0.0678 | 0.5044 | 0.4366 |  0.01732398
Other:.1_married; polyam | 14 |  0.0524 | 0.3322 | 0.2798 |  1.78507086e-05
Pain intensity - Baseline 1: | 15 |  0.0609 | 0.3951 | 0.3342 |  3.09535235e-05
Why are you having this upcoming procedure? Please mark all that apply. Please mark all that apply. (choice=Decrease pain)_Unchecked | 16 |  0.0512 | 0.3042 | 0.253 |  1.89781409e-05
Pain intensity - Baseline Average: | 17 |  0.0574 | 0.4788 | 0.4214 |  2.68615232e-05
Have you been experiencing pain in the past week?_Yes, but NOT RELATED to my need for surgery | 18 |  0.0602 | 0.5293 | 0.4691 |  2.64203743e-05
Pain intensity - CPM 2: | 19 |  0.0499 | 0.4446 | 0.3947 |  1.49991297e-05
Pain intensity - CPM 1: | 20 |  0.0372 | 0.4574 | 0.4203 |  2.22239174e-05

###  $\mu+1\sigma$, STS

Table 7: Feature importance using MLP using MI scoring selecting features with scores $\sigma+1\mu$. Feature Name | $n$ |  Score | Mean Absolute SHAP
---|---|---|---
Current Medications: Confirmed with participant (choice=Metformin)_Checked | 1 |  0.0722903 | 0
Color Score (C), Raw Score | 2 |  0.0696211 | 9.05460563e-06
Please specify your race: (choice=Black / African Heritage)_Checked | 3 |  0.0668161 | 7.03006661e-10
BRS Score | 4 |  0.0660192 | 9.07897635e-09
Color-Word Score (CW), Residual | 5 |  0.0639546 | 1.20770991e-08
Please specify your race: (choice=Prefer not to answer)_Checked | 6 |  0.0638041 | 4.17793688e-09
Current Medications: Confirmed with participant (choice=Metformin)_Unchecked | 7 |  0.0634184 | 1.18285792e-09
Other:.1_married; polyam | 8 |  0.062555 | 0
Current Medications: Confirmed with participant (choice=SNRI antidepressants (venlafaxine, duloxetine, desvenlafaxine, milnacipran))_Unchecked | 9 |  0.0619615 | 0
Current Medications: Confirmed with participant (choice=None of the above)_Unchecked | 10 |  0.0611848 | 0
Magnification Score | 11 |  0.0591411 | 3.05139839e-09
Color Score (C), Age/Ed. Predicted | 12 |  0.0588759 | 2.06625958e-08
Please specify your race: (choice=Hawaiian Native / Other Pacific Islander)_Checked | 13 |  0.0584943 | 0
Current Medications: Confirmed with participant (choice=Opioids (morphine, oxycodone, fentanyl, tramadol, methadone, buprenorphine))_Checked | 14 |  0.0583059 | 0
Total Years of Education | 15 |  0.0581167 | 5.38566368e-09
Have you ever been diagnosed with chronic pain?_No | 16 |  0.0579427 | 3.60115734e-10
Helplessness Score | 17 |  0.0578844 | 8.02095497e-06
Current Medications: Confirmed with participant (choice=None of the above)_Checked | 18 |  0.0577642 | 0
Word Score (W), Raw Score | 19 |  0.0576109 | 1.09118178e-05
Current Medications: Confirmed with participant (choice=SSRI antidepressants (citalopram, escitalopram, fluoxetine, fluvoxamine, paroxetine, sertraline))_Unchecked | 20 |  0.0574527 | 4.59364778e-10
Word Score, Residual | 21 |  0.0570744 | 8.04490438e-07
SBT Score: | 22 |  0.0569778 | 1.40497951e-07
Current Medications: Confirmed with participant (choice=Benzodiazepines (diazepam, clonazepam, alprazolam))_Unchecked | 23 |  0.0568526 | 0
Color-Word Score (CW), T-Score | 24 |  0.0561718 | 1.36966936e-05
Medical History: Confirmed with participant (choice=Cancer or Hx Cancer)_Unchecked | 25 |  0.056036 | 2.47380147e-09
Medical History: Confirmed with participant (choice=Cancer or Hx Cancer)_Checked | 26 |  0.0560167 | 8.56659354e-10
Medical History: Confirmed with participant (choice=Heart Disease (CAD, CHF, PAD))_Checked | 27 |  0.0554194 | 5.05811537e-10
Total Correct | 28 |  0.0552498 | 3.05141385e-05
Medical History: Confirmed with participant (choice=Diabetes)_Checked | 29 |  0.0548924 | 3.0714916e-10
Color-Word Score (CW), Age/Ed. Predicted | 30 |  0.0540521 | 4.95443076e-07
Current Medications: Confirmed with participant (choice=SNRI antidepressants (venlafaxine, duloxetine, desvenlafaxine, milnacipran))_Checked | 31 |  0.0536455 | 2.90962588e-10
Age (E-Consent): | 32 |  0.0534179 | 9.35483487e-09
Final T-Score | 33 |  0.053231 | 2.39802603e-06
Medical History: Confirmed with participant (choice=Hypertension)_Unchecked | 34 |  0.0532083 | 0
Word Score, T-Score | 35 |  0.0532053 | 2.75284503e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Unchecked | 36 |  0.0530448 | 0.00020982

###  $\mu+1\sigma$, STS

Table 8: Feature importance using MLP using STS scoring with PubMedBERT fine-tuned on the combined vocab selecting features with scores $\sigma+1\mu$. Feature Name | $n$ |  Score | Mean Absolute SHAP
---|---|---|---
On a scale of zero to ten, with zero being no pain and ten being the worst pain, please mark your average pain level during the past week, while at rest. | 1 |  0.645585 | 2.52998313e-06
On a scale of zero to ten, with zero being no pain and ten being the worst pain , please mark your average pain level during the past week, while you were active or moving. | 2 |  0.645342 | 4.85071774e-06
Have you been experiencing pain in the past week?_No | 3 |  0.639949 | 0.00097941
Have you been experiencing pain in the past week?_Yes, and RELATED to my need for surgery | 4 |  0.638486 | 0.00827323
Have you been experiencing pain in the past week?_Yes, but NOT RELATED to my need for surgery | 5 |  0.634165 | 0.00756546
On a scale of zero to ten, with zero being no pain and ten being the worst pain, what is your average pain level at rest TODAY? | 6 |  0.619892 | 5.57362602e-06
In the past three months, did you experience daily or near daily pain?_No | 7 |  0.544496 | 0.00685028
Have you ever been diagnosed with chronic pain?_No | 8 |  0.534078 | 0.0076836
Have you ever been diagnosed with chronic pain?_Yes | 9 |  0.531654 | 0.00385837
In the past three months, did you experience daily or near daily pain?_Yes | 10 |  0.526172 | 0.00479166
Pain intensity - Baseline Average: | 11 |  0.525021 | 2.64689293e-06
Pain intensity - Baseline 1: | 12 |  0.487709 | 6.1944453e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=None)_Unchecked | 13 |  0.478695 | 0.00105593
Pain intensity - Baseline 2: | 14 |  0.475303 | 1.54633563e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Other)_Unchecked | 15 |  0.475231 | 0.00597048
Do you take any of the following medications for pain treatment at least once per week? (choice=None)_Checked | 16 |  0.461651 | 0.00330743
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Unchecked | 17 |  0.449812 | 0.00079825
Have you used opioid medications for pain management in the past?_I don’t know | 18 |  0.44269 | 0.00037989
Have you used opioid medications for pain management in the past?_yes | 19 |  0.43422 | 0.00675297
Do you take any of the following medications for pain treatment at least once per week? (choice=Other)_Checked | 20 |  0.430055 | 0.00186429
Have you used opioid medications for pain management in the past?_no | 21 |  0.427981 | 0.00257769
Do you take any of the following medications for pain treatment at least once per week? (choice=Gabapentin or Pregabalin)_Unchecked | 22 |  0.417516 | 0.0130704
Do you take any of the following medications for pain treatment at least once per week? (choice=Ibuprofen (Motrin or Advil), celecoxib (Celebrex, Naproxen or Aleve), or other NSAIDs)_Unchecked | 23 |  0.416656 | 0.00558276
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Unchecked | 24 |  0.416248 | 0.00518861
Do you take any of the following medications for pain treatment at least once per week? (choice=Acetaminophen (Tylenol))_Unchecked | 25 |  0.415403 | 0.00048819
Pain intensity - CPM Average: | 26 |  0.413333 | 3.80922354e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Checked | 27 |  0.40647 | 0.00236485
Do you take any of the following medications for pain treatment at least once per week? (choice=Aspirin)_Unchecked | 28 |  0.397003 | 0.00082881
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Checked | 29 |  0.396227 | 0.0033335
Pain intensity - CPM 1: | 30 |  0.394309 | 8.07634382e-06
Pain intensity - CPM 2: | 31 |  0.393898 | 2.06799025e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Acetaminophen (Tylenol))_Checked | 32 |  0.387619 | 0.00553789

###  $\mu+1\sigma$, MI & STS

Table 9: Feature importance using MLP using linear scoring with PubMedBERT fine-tuned on the combined vocab selecting features with scores $\sigma+1\mu$. Feature Name | $n$ |  Score | Mean Absolute SHAP
---|---|---|---
Have you ever been diagnosed with chronic pain?_No | 1 |  0.734467 | 0.00860101
Have you ever been diagnosed with chronic pain?_Yes | 2 |  0.712339 | 0.00457072
Have you been experiencing pain in the past week?_No | 3 |  0.599768 | 7.51307387e-06
Have you been experiencing pain in the past week?_Yes, and RELATED to my need for surgery | 4 |  0.598852 | 0.01351249
Have you been experiencing pain in the past week?_Yes, but NOT RELATED to my need for surgery | 5 |  0.529267 | 6.29499593e-06
In the past three months, did you experience daily or near daily pain?_Yes | 6 |  0.525755 | 0.00436844
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Unchecked | 7 |  0.512279 | 3.79335761e-05
In the past three months, did you experience daily or near daily pain?_No | 8 |  0.504445 | 0.00791056
Helplessness Score | 9 |  0.493851 | 3.21395627e-06
Pain intensity - Baseline 2: | 10 |  0.479731 | 1.51890814e-06
Pain intensity - Baseline Average: | 11 |  0.478815 | 4.47488069e-07
Do you take any of the following medications for pain treatment at least once per week? (choice=Other)_Unchecked | 12 |  0.472314 | 0.00774911
Have you used opioid medications for pain management in the past?_yes | 13 |  0.466008 | 0.00828783
Pain intensity - CPM 1: | 14 |  0.457419 | 1.58137085e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=None)_Unchecked | 15 |  0.455803 | 0.00064399
Pain intensity - CPM 2: | 16 |  0.444577 | 3.07856456e-07
Pain intensity - CPM Average: | 17 |  0.439207 | 4.12713718e-06
Do you take any of the following medications for pain treatment at least once per week? (choice=Tramadol (Ultram))_Unchecked | 18 |  0.438722 | 0.00609676
Have you used opioid medications for pain management in the past?_I don’t know | 19 |  0.43665 | 0.00050837
Have you used opioid medications for pain management in the past?_no | 20 |  0.433569 | 0.00174238
Do you take any of the following medications for pain treatment at least once per week? (choice=Duloxetine (Cymbalta) or amitriptyline (Elavil))_Checked | 21 |  0.41803 | 0.00318739
Do you take any of the following medications for pain treatment at least once per week? (choice=Ibuprofen (Motrin or Advil), celecoxib (Celebrex, Naproxen or Aleve), or other NSAIDs)_Unchecked | 22 |  0.415362 | 0.00432255
Do you take any of the following medications for pain treatment at least once per week? (choice=Gabapentin or Pregabalin)_Unchecked | 23 |  0.415204 | 0.01460327
On a scale of zero to ten, with zero being no pain and ten being the worst pain, please mark your average pain level during the past week, while at rest. | 24 |  0.404095 | 0
Do you take any of the following medications for pain treatment at least once per week? (choice=Aspirin)_Unchecked | 25 |  0.403687 | 0.00111563
Current Medications: Confirmed with participant (choice=Opioids (morphine, oxycodone, fentanyl, tramadol, methadone, buprenorphine))_Unchecked | 26 |  0.398177 | 2.50144896e-06
On a scale of zero to ten, with zero being no pain and ten being the worst pain, what is your average pain level at rest TODAY? | 27 |  0.397393 | 1.43354317e-06

## Performance of All Feature Selection & Classifier Combinations

This section contains the area under the receiver-operator curve (AUROC) and
area under the precision-recall curve (AUPRC) performance of all algorithms
tested, separated by classifier. Maximum values for each classifier are
bolded, as well as minimum absolute values for the test-train difference
column.

### Linear support vector machine (SVM)

Table 10: All results for linear SVM. Selection Hyperparameters | AUROC | AUPRC
---|---|---
Feature Selector | Scorer | Scoring Model | FT Dataset | Test | Train | $\Delta$ | Test | Train | $\Delta$
Identity | - | - | - | 0.534 | 0.858 | -0.324 | 0.033 | 0.463 | -0.430
RFE | - | - | - | 0.521 | 0.826 | -0.305 | 0.032 | 0.438 | -0.406
ReliefF | - | - | - | 0.581 | 0.785 | -0.203 | 0.037 | 0.174 | -0.138
SFM-LinearSVM | - | - | - | 0.493 | 0.817 | -0.324 | 0.031 | 0.400 | -0.369
SFM-XGBoost | - | - | - | 0.590 | 0.744 | -0.155 | 0.041 | 0.178 | -0.137
Std. Dev. | MI | - | - | 0.689 | 0.712 | -0.024 | 0.049 | 0.116 | -0.067
Std. Dev. | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.561 | 0.782 | -0.221 | 0.039 | 0.358 | -0.319
Std. Dev. | MI & STS | Bio_ClinicalBERT | clinical | 0.705 | 0.751 | -0.046 | 0.052 | 0.135 | -0.083
Std. Dev. | MI & STS | Bio_ClinicalBERT | combined | 0.725 | 0.728 | -0.003 | 0.055 | 0.101 | -0.046
Std. Dev. | MI & STS | Bio_ClinicalBERT | gen | 0.691 | 0.737 | -0.045 | 0.050 | 0.149 | -0.099
Std. Dev. | MI & STS | FastText | gen | 0.736 | 0.737 | -0.002 | 0.101 | 0.108 | -0.007
Std. Dev. | MI & STS | PubMedBERT | clinical | 0.686 | 0.763 | -0.077 | 0.048 | 0.204 | -0.155
Std. Dev. | MI & STS | PubMedBERT | combined | 0.628 | 0.770 | -0.142 | 0.045 | 0.321 | -0.277
Std. Dev. | MI & STS | PubMedBERT | gen | 0.628 | 0.770 | -0.142 | 0.041 | 0.284 | -0.243
Std. Dev. | MI & STS | Skipgram | gen | 0.661 | 0.759 | -0.098 | 0.044 | 0.166 | -0.122
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | clinical | 0.544 | 0.771 | -0.227 | 0.036 | 0.338 | -0.302
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | combined | 0.477 | 0.753 | -0.276 | 0.029 | 0.226 | -0.197
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | gen | 0.645 | 0.784 | -0.140 | 0.044 | 0.164 | -0.120
Std. Dev. | MI & STS | bert-base-uncased | clinical | 0.618 | 0.692 | -0.073 | 0.047 | 0.105 | -0.058
Std. Dev. | MI & STS | bert-base-uncased | combined | 0.526 | 0.739 | -0.213 | 0.067 | 0.151 | -0.084
Std. Dev. | MI & STS | bert-base-uncased | gen | 0.634 | 0.733 | -0.099 | 0.056 | 0.121 | -0.065
Std. Dev. | MI & STS | biomed_roberta_base | clinical | 0.680 | 0.671 | 0.010 | 0.066 | 0.093 | -0.027
Std. Dev. | MI & STS | biomed_roberta_base | combined | 0.700 | 0.752 | -0.052 | 0.051 | 0.186 | -0.134
Std. Dev. | MI & STS | biomed_roberta_base | gen | 0.741 | 0.729 | 0.012 | 0.057 | 0.110 | -0.053
Std. Dev. | STS | Bio_ClinicalBERT | ClinicalSTS | 0.733 | 0.728 | 0.005 | 0.055 | 0.158 | -0.103
Std. Dev. | STS | Bio_ClinicalBERT | clinical | 0.642 | 0.707 | -0.065 | 0.042 | 0.161 | -0.119
Std. Dev. | STS | Bio_ClinicalBERT | combined | 0.708 | 0.758 | -0.050 | 0.051 | 0.207 | -0.156
Std. Dev. | STS | Bio_ClinicalBERT | gen | 0.713 | 0.753 | -0.040 | 0.052 | 0.194 | -0.142
Std. Dev. | STS | FastText | gen | 0.559 | 0.806 | -0.247 | 0.035 | 0.249 | -0.213
Std. Dev. | STS | PubMedBERT | clinical | 0.499 | 0.776 | -0.277 | 0.031 | 0.207 | -0.176
Std. Dev. | STS | PubMedBERT | combined | 0.598 | 0.767 | -0.169 | 0.040 | 0.305 | -0.265
Std. Dev. | STS | PubMedBERT | gen | 0.616 | 0.771 | -0.155 | 0.041 | 0.292 | -0.251
Std. Dev. | STS | Skipgram | gen | 0.457 | 0.843 | -0.385 | 0.029 | 0.395 | -0.366
Std. Dev. | STS | all-MiniLM-L12-v2 | clinical | 0.753 | 0.721 | 0.032 | 0.060 | 0.117 | -0.056
Std. Dev. | STS | all-MiniLM-L12-v2 | combined | 0.508 | 0.785 | -0.276 | 0.032 | 0.297 | -0.265
Std. Dev. | STS | all-MiniLM-L12-v2 | gen | 0.709 | 0.759 | -0.050 | 0.051 | 0.203 | -0.152
Std. Dev. | STS | bert-base-uncased | clinical | 0.595 | 0.750 | -0.154 | 0.037 | 0.218 | -0.180
Std. Dev. | STS | bert-base-uncased | combined | 0.614 | 0.769 | -0.155 | 0.039 | 0.268 | -0.229
Std. Dev. | STS | bert-base-uncased | gen | 0.629 | 0.772 | -0.142 | 0.041 | 0.279 | -0.237
Std. Dev. | STS | biomed_roberta_base | clinical | 0.559 | 0.805 | -0.246 | 0.035 | 0.237 | -0.202
Std. Dev. | STS | biomed_roberta_base | combined | 0.579 | 0.786 | -0.207 | 0.037 | 0.282 | -0.244
Std. Dev. | STS | biomed_roberta_base | gen | 0.606 | 0.775 | -0.169 | 0.039 | 0.243 | -0.203
Top N | MI | - | - | 0.609 | 0.642 | -0.033 | 0.039 | 0.082 | -0.043
Top N | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.752 | 0.671 | 0.081 | 0.145 | 0.113 | 0.032
Top N | MI & STS | Bio_ClinicalBERT | clinical | 0.672 | 0.745 | -0.073 | 0.047 | 0.117 | -0.071
Top N | MI & STS | Bio_ClinicalBERT | combined | 0.664 | 0.753 | -0.089 | 0.046 | 0.152 | -0.105
Top N | MI & STS | Bio_ClinicalBERT | gen | 0.683 | 0.747 | -0.064 | 0.050 | 0.133 | -0.083
Top N | MI & STS | FastText | gen | 0.565 | 0.681 | -0.116 | 0.035 | 0.099 | -0.064
Top N | MI & STS | PubMedBERT | clinical | 0.713 | 0.733 | -0.020 | 0.052 | 0.159 | -0.107
Top N | MI & STS | PubMedBERT | combined | 0.769 | 0.721 | 0.047 | 0.070 | 0.140 | -0.070
Top N | MI & STS | PubMedBERT | gen | 0.563 | 0.776 | -0.212 | 0.037 | 0.238 | -0.201
Top N | MI & STS | Skipgram | gen | 0.782 | 0.594 | 0.188 | 0.068 | 0.077 | -0.009
Top N | MI & STS | all-MiniLM-L12-v2 | clinical | 0.696 | 0.721 | -0.025 | 0.049 | 0.131 | -0.082
Top N | MI & STS | all-MiniLM-L12-v2 | combined | 0.601 | 0.733 | -0.132 | 0.038 | 0.180 | -0.143
Top N | MI & STS | all-MiniLM-L12-v2 | gen | 0.661 | 0.760 | -0.098 | 0.045 | 0.159 | -0.114
Top N | MI & STS | bert-base-uncased | clinical | 0.594 | 0.682 | -0.088 | 0.053 | 0.091 | -0.038
Top N | MI & STS | bert-base-uncased | combined | 0.594 | 0.698 | -0.104 | 0.053 | 0.131 | -0.077
Top N | MI & STS | bert-base-uncased | gen | 0.574 | 0.734 | -0.159 | 0.037 | 0.139 | -0.102
Top N | MI & STS | biomed_roberta_base | clinical | 0.639 | 0.648 | -0.009 | 0.051 | 0.077 | -0.026
Top N | MI & STS | biomed_roberta_base | combined | 0.730 | 0.724 | 0.006 | 0.066 | 0.165 | -0.099
Top N | MI & STS | biomed_roberta_base | gen | 0.741 | 0.729 | 0.012 | 0.057 | 0.110 | -0.053
Top N | STS | Bio_ClinicalBERT | ClinicalSTS | 0.713 | 0.737 | -0.023 | 0.050 | 0.150 | -0.100
Top N | STS | Bio_ClinicalBERT | clinical | 0.680 | 0.728 | -0.048 | 0.046 | 0.162 | -0.115
Top N | STS | Bio_ClinicalBERT | combined | 0.675 | 0.713 | -0.038 | 0.046 | 0.151 | -0.106
Top N | STS | Bio_ClinicalBERT | gen | 0.649 | 0.727 | -0.078 | 0.043 | 0.145 | -0.102
Top N | STS | FastText | gen | 0.642 | 0.723 | -0.081 | 0.042 | 0.146 | -0.104
Top N | STS | PubMedBERT | clinical | 0.769 | 0.724 | 0.045 | 0.065 | 0.133 | -0.069
Top N | STS | PubMedBERT | combined | 0.758 | 0.753 | 0.004 | 0.067 | 0.171 | -0.103
Top N | STS | PubMedBERT | gen | 0.646 | 0.749 | -0.103 | 0.044 | 0.161 | -0.117
Top N | STS | Skipgram | gen | 0.725 | 0.721 | 0.004 | 0.054 | 0.130 | -0.076
Top N | STS | all-MiniLM-L12-v2 | clinical | 0.711 | 0.746 | -0.035 | 0.055 | 0.154 | -0.099
Top N | STS | all-MiniLM-L12-v2 | combined | 0.617 | 0.735 | -0.118 | 0.040 | 0.135 | -0.096
Top N | STS | all-MiniLM-L12-v2 | gen | 0.755 | 0.726 | 0.029 | 0.059 | 0.112 | -0.053
Top N | STS | bert-base-uncased | clinical | 0.646 | 0.726 | -0.080 | 0.043 | 0.183 | -0.141
Top N | STS | bert-base-uncased | combined | 0.517 | 0.754 | -0.238 | 0.030 | 0.189 | -0.159
Top N | STS | bert-base-uncased | gen | 0.653 | 0.740 | -0.087 | 0.043 | 0.182 | -0.139
Top N | STS | biomed_roberta_base | clinical | 0.533 | 0.737 | -0.204 | 0.038 | 0.240 | -0.202
Top N | STS | biomed_roberta_base | combined | 0.723 | 0.714 | 0.009 | 0.059 | 0.116 | -0.057
Top N | STS | biomed_roberta_base | gen | 0.748 | 0.720 | 0.028 | 0.068 | 0.168 | -0.100
mRMR | MI | - | - | 0.391 | 0.671 | -0.280 | 0.025 | 0.092 | -0.067
mRMR | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.687 | 0.750 | -0.062 | 0.048 | 0.126 | -0.079
mRMR | MI & STS | Bio_ClinicalBERT | clinical | 0.602 | 0.749 | -0.147 | 0.037 | 0.161 | -0.124
mRMR | MI & STS | Bio_ClinicalBERT | combined | 0.576 | 0.746 | -0.170 | 0.036 | 0.151 | -0.115
mRMR | MI & STS | Bio_ClinicalBERT | gen | 0.610 | 0.732 | -0.122 | 0.039 | 0.146 | -0.107
mRMR | MI & STS | FastText | gen | 0.234 | 0.736 | -0.502 | 0.021 | 0.153 | -0.132
mRMR | MI & STS | PubMedBERT | clinical | 0.614 | 0.757 | -0.143 | 0.041 | 0.168 | -0.127
mRMR | MI & STS | PubMedBERT | combined | 0.663 | 0.765 | -0.103 | 0.053 | 0.256 | -0.203
mRMR | MI & STS | PubMedBERT | gen | 0.679 | 0.799 | -0.120 | 0.063 | 0.301 | -0.239
mRMR | MI & STS | Skipgram | gen | 0.410 | 0.599 | -0.188 | 0.028 | 0.099 | -0.071
mRMR | MI & STS | all-MiniLM-L12-v2 | clinical | 0.603 | 0.673 | -0.070 | 0.039 | 0.103 | -0.064
mRMR | MI & STS | all-MiniLM-L12-v2 | combined | 0.620 | 0.775 | -0.155 | 0.040 | 0.150 | -0.110
mRMR | MI & STS | all-MiniLM-L12-v2 | gen | 0.682 | 0.779 | -0.097 | 0.048 | 0.156 | -0.109
mRMR | MI & STS | bert-base-uncased | clinical | 0.584 | 0.662 | -0.078 | 0.038 | 0.095 | -0.057
mRMR | MI & STS | bert-base-uncased | combined | 0.683 | 0.704 | -0.021 | 0.050 | 0.100 | -0.050
mRMR | MI & STS | bert-base-uncased | gen | 0.530 | 0.685 | -0.155 | 0.033 | 0.117 | -0.084
mRMR | MI & STS | biomed_roberta_base | clinical | 0.519 | 0.638 | -0.119 | 0.041 | 0.069 | -0.027
mRMR | MI & STS | biomed_roberta_base | combined | 0.686 | 0.665 | 0.020 | 0.195 | 0.103 | 0.092
mRMR | MI & STS | biomed_roberta_base | gen | 0.690 | 0.750 | -0.060 | 0.048 | 0.126 | -0.078
mRMR | STS | Bio_ClinicalBERT | ClinicalSTS | 0.708 | 0.702 | 0.006 | 0.052 | 0.118 | -0.066
mRMR | STS | Bio_ClinicalBERT | clinical | 0.640 | 0.757 | -0.117 | 0.042 | 0.156 | -0.114
mRMR | STS | Bio_ClinicalBERT | combined | 0.457 | 0.757 | -0.300 | 0.029 | 0.235 | -0.206
mRMR | STS | Bio_ClinicalBERT | gen | 0.506 | 0.739 | -0.233 | 0.032 | 0.199 | -0.167
mRMR | STS | FastText | gen | 0.565 | 0.779 | -0.214 | 0.035 | 0.221 | -0.186
mRMR | STS | PubMedBERT | clinical | 0.544 | 0.769 | -0.225 | 0.033 | 0.198 | -0.165
mRMR | STS | PubMedBERT | combined | 0.620 | 0.724 | -0.104 | 0.041 | 0.191 | -0.150
mRMR | STS | PubMedBERT | gen | 0.554 | 0.741 | -0.187 | 0.041 | 0.194 | -0.154
mRMR | STS | Skipgram | gen | 0.581 | 0.803 | -0.222 | 0.037 | 0.360 | -0.324
mRMR | STS | all-MiniLM-L12-v2 | clinical | 0.594 | 0.737 | -0.143 | 0.042 | 0.146 | -0.104
mRMR | STS | all-MiniLM-L12-v2 | combined | 0.526 | 0.692 | -0.166 | 0.032 | 0.103 | -0.071
mRMR | STS | all-MiniLM-L12-v2 | gen | 0.569 | 0.743 | -0.174 | 0.039 | 0.136 | -0.097
mRMR | STS | bert-base-uncased | clinical | 0.628 | 0.747 | -0.119 | 0.041 | 0.113 | -0.072
mRMR | STS | bert-base-uncased | combined | 0.493 | 0.732 | -0.239 | 0.031 | 0.135 | -0.104
mRMR | STS | bert-base-uncased | gen | 0.579 | 0.721 | -0.143 | 0.038 | 0.085 | -0.048
mRMR | STS | biomed_roberta_base | clinical | 0.523 | 0.762 | -0.239 | 0.036 | 0.140 | -0.104
mRMR | STS | biomed_roberta_base | combined | 0.722 | 0.693 | 0.029 | 0.054 | 0.105 | -0.051
mRMR | STS | biomed_roberta_base | gen | 0.587 | 0.703 | -0.116 | 0.036 | 0.109 | -0.072
Table 10: All results for linear SVM.

### XGBoost

Table 11: All results for XGBoost. Selection Hyperparameters | AUROC | AUPRC
---|---|---
Feature Selector | Scorer | Scoring Model | FT Dataset | Test | Train | $\Delta$ | Test | Train | $\Delta$
Identity | - | - | - | 0.488 | 0.784 | -0.296 | 0.024 | 0.338 | -0.314
RFE | - | - | - | 0.417 | 0.749 | -0.332 | 0.024 | 0.129 | -0.105
ReliefF | - | - | - | 0.483 | 0.727 | -0.244 | 0.024 | 0.479 | -0.455
SFM-LinearSVM | - | - | - | 0.421 | 0.728 | -0.306 | 0.024 | 0.118 | -0.094
SFM-XGBoost | - | - | - | 0.742 | 0.860 | -0.118 | 0.064 | 0.238 | -0.175
Std. Dev. | MI | - | - | 0.450 | 0.761 | -0.311 | 0.024 | 0.305 | -0.281
Std. Dev. | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.455 | 0.637 | -0.182 | 0.024 | 0.110 | -0.086
Std. Dev. | MI & STS | Bio_ClinicalBERT | clinical | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
Std. Dev. | MI & STS | Bio_ClinicalBERT | combined | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
Std. Dev. | MI & STS | Bio_ClinicalBERT | gen | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
Std. Dev. | MI & STS | FastText | gen | 0.450 | 0.761 | -0.311 | 0.024 | 0.305 | -0.281
Std. Dev. | MI & STS | PubMedBERT | clinical | 0.617 | 0.750 | -0.133 | 0.042 | 0.219 | -0.177
Std. Dev. | MI & STS | PubMedBERT | combined | 0.467 | 0.768 | -0.302 | 0.024 | 0.429 | -0.405
Std. Dev. | MI & STS | PubMedBERT | gen | 0.479 | 0.763 | -0.284 | 0.024 | 0.332 | -0.308
Std. Dev. | MI & STS | Skipgram | gen | 0.530 | 0.774 | -0.244 | 0.026 | 0.117 | -0.091
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | clinical | 0.483 | 0.628 | -0.144 | 0.024 | 0.149 | -0.125
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | combined | 0.625 | 0.688 | -0.062 | 0.046 | 0.174 | -0.127
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | gen | 0.475 | 0.609 | -0.134 | 0.024 | 0.161 | -0.137
Std. Dev. | MI & STS | bert-base-uncased | clinical | 0.492 | 0.562 | -0.070 | 0.024 | 0.084 | -0.060
Std. Dev. | MI & STS | bert-base-uncased | combined | 0.467 | 0.738 | -0.271 | 0.024 | 0.272 | -0.248
Std. Dev. | MI & STS | bert-base-uncased | gen | 0.463 | 0.670 | -0.207 | 0.024 | 0.114 | -0.090
Std. Dev. | MI & STS | biomed_roberta_base | clinical | 0.479 | 0.636 | -0.157 | 0.024 | 0.305 | -0.281
Std. Dev. | MI & STS | biomed_roberta_base | combined | 0.442 | 0.736 | -0.294 | 0.024 | 0.251 | -0.227
Std. Dev. | MI & STS | biomed_roberta_base | gen | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
Std. Dev. | STS | Bio_ClinicalBERT | ClinicalSTS | 0.434 | 0.701 | -0.267 | 0.024 | 0.119 | -0.095
Std. Dev. | STS | Bio_ClinicalBERT | clinical | 0.455 | 0.629 | -0.175 | 0.024 | 0.094 | -0.070
Std. Dev. | STS | Bio_ClinicalBERT | combined | 0.450 | 0.718 | -0.268 | 0.024 | 0.156 | -0.132
Std. Dev. | STS | Bio_ClinicalBERT | gen | 0.434 | 0.701 | -0.267 | 0.024 | 0.119 | -0.095
Std. Dev. | STS | FastText | gen | 0.450 | 0.838 | -0.388 | 0.024 | 0.303 | -0.279
Std. Dev. | STS | PubMedBERT | clinical | 0.426 | 0.639 | -0.213 | 0.024 | 0.078 | -0.054
Std. Dev. | STS | PubMedBERT | combined | 0.426 | 0.639 | -0.213 | 0.024 | 0.078 | -0.054
Std. Dev. | STS | PubMedBERT | gen | 0.426 | 0.639 | -0.213 | 0.024 | 0.078 | -0.054
Std. Dev. | STS | Skipgram | gen | 0.471 | 0.744 | -0.273 | 0.024 | 0.346 | -0.322
Std. Dev. | STS | all-MiniLM-L12-v2 | clinical | 0.479 | 0.596 | -0.116 | 0.024 | 0.086 | -0.062
Std. Dev. | STS | all-MiniLM-L12-v2 | combined | 0.455 | 0.637 | -0.182 | 0.024 | 0.110 | -0.086
Std. Dev. | STS | all-MiniLM-L12-v2 | gen | 0.434 | 0.701 | -0.267 | 0.024 | 0.119 | -0.095
Std. Dev. | STS | bert-base-uncased | clinical | 0.455 | 0.637 | -0.182 | 0.024 | 0.110 | -0.086
Std. Dev. | STS | bert-base-uncased | combined | 0.455 | 0.637 | -0.182 | 0.024 | 0.110 | -0.086
Std. Dev. | STS | bert-base-uncased | gen | 0.426 | 0.639 | -0.213 | 0.024 | 0.078 | -0.054
Std. Dev. | STS | biomed_roberta_base | clinical | 0.471 | 0.628 | -0.157 | 0.024 | 0.149 | -0.125
Std. Dev. | STS | biomed_roberta_base | combined | 0.617 | 0.748 | -0.131 | 0.042 | 0.207 | -0.166
Std. Dev. | STS | biomed_roberta_base | gen | 0.430 | 0.639 | -0.209 | 0.024 | 0.078 | -0.054
Top N | MI | - | - | 0.471 | 0.633 | -0.162 | 0.024 | 0.214 | -0.190
Top N | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.471 | 0.587 | -0.116 | 0.024 | 0.072 | -0.048
Top N | MI & STS | Bio_ClinicalBERT | clinical | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
Top N | MI & STS | Bio_ClinicalBERT | combined | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
Top N | MI & STS | Bio_ClinicalBERT | gen | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
Top N | MI & STS | FastText | gen | 0.450 | 0.743 | -0.293 | 0.024 | 0.184 | -0.160
Top N | MI & STS | PubMedBERT | clinical | 0.467 | 0.738 | -0.271 | 0.024 | 0.272 | -0.248
Top N | MI & STS | PubMedBERT | combined | 0.463 | 0.715 | -0.252 | 0.024 | 0.231 | -0.207
Top N | MI & STS | PubMedBERT | gen | 0.475 | 0.681 | -0.206 | 0.024 | 0.145 | -0.121
Top N | MI & STS | Skipgram | gen | 0.467 | 0.669 | -0.202 | 0.024 | 0.174 | -0.150
Top N | MI & STS | all-MiniLM-L12-v2 | clinical | 0.467 | 0.579 | -0.112 | 0.024 | 0.085 | -0.061
Top N | MI & STS | all-MiniLM-L12-v2 | combined | 0.467 | 0.579 | -0.112 | 0.024 | 0.085 | -0.061
Top N | MI & STS | all-MiniLM-L12-v2 | gen | 0.479 | 0.633 | -0.154 | 0.024 | 0.214 | -0.190
Top N | MI & STS | bert-base-uncased | clinical | 0.500 | 0.500 | 0.000 | 0.024 | 0.045 | -0.020
Top N | MI & STS | bert-base-uncased | combined | 0.471 | 0.703 | -0.232 | 0.024 | 0.162 | -0.138
Top N | MI & STS | bert-base-uncased | gen | 0.463 | 0.674 | -0.211 | 0.024 | 0.123 | -0.099
Top N | MI & STS | biomed_roberta_base | clinical | 0.479 | 0.636 | -0.157 | 0.024 | 0.305 | -0.281
Top N | MI & STS | biomed_roberta_base | combined | 0.475 | 0.651 | -0.175 | 0.024 | 0.179 | -0.155
Top N | MI & STS | biomed_roberta_base | gen | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
Top N | STS | Bio_ClinicalBERT | ClinicalSTS | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
Top N | STS | Bio_ClinicalBERT | clinical | 0.467 | 0.579 | -0.112 | 0.024 | 0.085 | -0.061
Top N | STS | Bio_ClinicalBERT | combined | 0.479 | 0.585 | -0.105 | 0.024 | 0.109 | -0.085
Top N | STS | Bio_ClinicalBERT | gen | 0.467 | 0.579 | -0.112 | 0.024 | 0.085 | -0.061
Top N | STS | FastText | gen | 0.496 | 0.639 | -0.143 | 0.024 | 0.116 | -0.092
Top N | STS | PubMedBERT | clinical | 0.479 | 0.596 | -0.116 | 0.024 | 0.086 | -0.062
Top N | STS | PubMedBERT | combined | 0.479 | 0.596 | -0.116 | 0.024 | 0.086 | -0.062
Top N | STS | PubMedBERT | gen | 0.471 | 0.676 | -0.205 | 0.024 | 0.129 | -0.104
Top N | STS | Skipgram | gen | 0.496 | 0.639 | -0.143 | 0.024 | 0.116 | -0.092
Top N | STS | all-MiniLM-L12-v2 | clinical | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
Top N | STS | all-MiniLM-L12-v2 | combined | 0.479 | 0.596 | -0.116 | 0.024 | 0.086 | -0.062
Top N | STS | all-MiniLM-L12-v2 | gen | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
Top N | STS | bert-base-uncased | clinical | 0.467 | 0.579 | -0.112 | 0.024 | 0.085 | -0.061
Top N | STS | bert-base-uncased | combined | 0.430 | 0.639 | -0.209 | 0.024 | 0.078 | -0.054
Top N | STS | bert-base-uncased | gen | 0.426 | 0.694 | -0.268 | 0.024 | 0.108 | -0.084
Top N | STS | biomed_roberta_base | clinical | 0.450 | 0.619 | -0.169 | 0.024 | 0.107 | -0.083
Top N | STS | biomed_roberta_base | combined | 0.479 | 0.596 | -0.116 | 0.024 | 0.086 | -0.062
Top N | STS | biomed_roberta_base | gen | 0.479 | 0.596 | -0.116 | 0.024 | 0.086 | -0.062
mRMR | MI | - | - | 0.492 | 0.654 | -0.162 | 0.024 | 0.216 | -0.192
mRMR | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
mRMR | MI & STS | Bio_ClinicalBERT | clinical | 0.388 | 0.709 | -0.320 | 0.024 | 0.092 | -0.068
mRMR | MI & STS | Bio_ClinicalBERT | combined | 0.388 | 0.709 | -0.320 | 0.024 | 0.092 | -0.068
mRMR | MI & STS | Bio_ClinicalBERT | gen | 0.421 | 0.719 | -0.298 | 0.024 | 0.099 | -0.075
mRMR | MI & STS | FastText | gen | 0.467 | 0.735 | -0.268 | 0.024 | 0.242 | -0.218
mRMR | MI & STS | PubMedBERT | clinical | 0.488 | 0.628 | -0.140 | 0.024 | 0.149 | -0.125
mRMR | MI & STS | PubMedBERT | combined | 0.471 | 0.663 | -0.192 | 0.024 | 0.140 | -0.116
mRMR | MI & STS | PubMedBERT | gen | 0.475 | 0.667 | -0.192 | 0.024 | 0.161 | -0.136
mRMR | MI & STS | Skipgram | gen | 0.483 | 0.631 | -0.148 | 0.024 | 0.181 | -0.157
mRMR | MI & STS | all-MiniLM-L12-v2 | clinical | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
mRMR | MI & STS | all-MiniLM-L12-v2 | combined | 0.526 | 0.717 | -0.191 | 0.026 | 0.088 | -0.062
mRMR | MI & STS | all-MiniLM-L12-v2 | gen | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
mRMR | MI & STS | bert-base-uncased | clinical | 0.500 | 0.500 | 0.000 | 0.024 | 0.045 | -0.020
mRMR | MI & STS | bert-base-uncased | combined | 0.459 | 0.562 | -0.104 | 0.024 | 0.060 | -0.036
mRMR | MI & STS | bert-base-uncased | gen | 0.471 | 0.575 | -0.104 | 0.024 | 0.075 | -0.051
mRMR | MI & STS | biomed_roberta_base | clinical | 0.488 | 0.534 | -0.046 | 0.024 | 0.055 | -0.030
mRMR | MI & STS | biomed_roberta_base | combined | 0.479 | 0.589 | -0.109 | 0.024 | 0.158 | -0.134
mRMR | MI & STS | biomed_roberta_base | gen | 0.459 | 0.667 | -0.209 | 0.024 | 0.110 | -0.086
mRMR | STS | Bio_ClinicalBERT | ClinicalSTS | 0.613 | 0.707 | -0.094 | 0.040 | 0.181 | -0.141
mRMR | STS | Bio_ClinicalBERT | clinical | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
mRMR | STS | Bio_ClinicalBERT | combined | 0.634 | 0.690 | -0.056 | 0.053 | 0.186 | -0.133
mRMR | STS | Bio_ClinicalBERT | gen | 0.479 | 0.586 | -0.106 | 0.024 | 0.117 | -0.093
mRMR | STS | FastText | gen | 0.426 | 0.807 | -0.382 | 0.024 | 0.181 | -0.157
mRMR | STS | PubMedBERT | clinical | 0.459 | 0.728 | -0.269 | 0.024 | 0.194 | -0.170
mRMR | STS | PubMedBERT | combined | 0.500 | 0.589 | -0.089 | 0.024 | 0.158 | -0.134
mRMR | STS | PubMedBERT | gen | 0.455 | 0.678 | -0.223 | 0.024 | 0.135 | -0.110
mRMR | STS | Skipgram | gen | 0.459 | 0.764 | -0.306 | 0.024 | 0.348 | -0.323
mRMR | STS | all-MiniLM-L12-v2 | clinical | 0.522 | 0.738 | -0.216 | 0.025 | 0.094 | -0.068
mRMR | STS | all-MiniLM-L12-v2 | combined | 0.522 | 0.738 | -0.216 | 0.025 | 0.094 | -0.068
mRMR | STS | all-MiniLM-L12-v2 | gen | 0.430 | 0.735 | -0.306 | 0.024 | 0.113 | -0.089
mRMR | STS | bert-base-uncased | clinical | 0.500 | 0.590 | -0.090 | 0.024 | 0.182 | -0.158
mRMR | STS | bert-base-uncased | combined | 0.483 | 0.694 | -0.210 | 0.024 | 0.220 | -0.196
mRMR | STS | bert-base-uncased | gen | 0.747 | 0.726 | 0.021 | 0.066 | 0.104 | -0.038
mRMR | STS | biomed_roberta_base | clinical | 0.479 | 0.692 | -0.212 | 0.024 | 0.202 | -0.178
mRMR | STS | biomed_roberta_base | combined | 0.434 | 0.786 | -0.352 | 0.024 | 0.219 | -0.195
mRMR | STS | biomed_roberta_base | gen | 0.563 | 0.637 | -0.074 | 0.029 | 0.072 | -0.043
Table 11: All results for XGBoost.

###  MLP

Table 12: All results for MLP. Selection Hyperparameters | AUROC | AUPRC
---|---|---
Feature Selector | Scorer | Scoring Model | FT Dataset | Test | Train | $\Delta$ | Test | Train | $\Delta$
Identity | - | - | - | 0.576 | 0.881 | -0.305 | 0.036 | 0.541 | -0.505
RFE | - | - | - | 0.534 | 0.834 | -0.299 | 0.034 | 0.452 | -0.418
ReliefF | - | - | - | 0.545 | 0.861 | -0.315 | 0.034 | 0.348 | -0.314
SFM-LinearSVM | - | - | - | 0.658 | 0.791 | -0.132 | 0.045 | 0.286 | -0.242
SFM-XGBoost | - | - | - | 0.430 | 0.679 | -0.249 | 0.028 | 0.151 | -0.123
Std. Dev. | MI | - | - | 0.259 | 0.494 | -0.235 | 0.021 | 0.044 | -0.022
Std. Dev. | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.638 | 0.786 | -0.148 | 0.042 | 0.336 | -0.294
Std. Dev. | MI & STS | Bio_ClinicalBERT | clinical | 0.689 | 0.765 | -0.077 | 0.049 | 0.143 | -0.094
Std. Dev. | MI & STS | Bio_ClinicalBERT | combined | 0.705 | 0.776 | -0.071 | 0.052 | 0.174 | -0.122
Std. Dev. | MI & STS | Bio_ClinicalBERT | gen | 0.650 | 0.770 | -0.120 | 0.044 | 0.218 | -0.174
Std. Dev. | MI & STS | FastText | gen | 0.738 | 0.758 | -0.019 | 0.205 | 0.123 | 0.082
Std. Dev. | MI & STS | PubMedBERT | clinical | 0.631 | 0.786 | -0.155 | 0.041 | 0.278 | -0.236
Std. Dev. | MI & STS | PubMedBERT | combined | 0.689 | 0.774 | -0.085 | 0.049 | 0.316 | -0.267
Std. Dev. | MI & STS | PubMedBERT | gen | 0.645 | 0.788 | -0.144 | 0.044 | 0.319 | -0.275
Std. Dev. | MI & STS | Skipgram | gen | 0.689 | 0.831 | -0.142 | 0.055 | 0.443 | -0.388
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | clinical | 0.583 | 0.781 | -0.199 | 0.039 | 0.328 | -0.289
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | combined | 0.556 | 0.754 | -0.198 | 0.035 | 0.212 | -0.177
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | gen | 0.664 | 0.824 | -0.160 | 0.052 | 0.231 | -0.179
Std. Dev. | MI & STS | bert-base-uncased | clinical | 0.561 | 0.743 | -0.183 | 0.035 | 0.143 | -0.108
Std. Dev. | MI & STS | bert-base-uncased | combined | 0.617 | 0.753 | -0.136 | 0.072 | 0.154 | -0.082
Std. Dev. | MI & STS | bert-base-uncased | gen | 0.653 | 0.748 | -0.095 | 0.055 | 0.143 | -0.087
Std. Dev. | MI & STS | biomed_roberta_base | clinical | 0.719 | 0.711 | 0.008 | 0.088 | 0.149 | -0.061
Std. Dev. | MI & STS | biomed_roberta_base | combined | 0.680 | 0.769 | -0.089 | 0.047 | 0.187 | -0.139
Std. Dev. | MI & STS | biomed_roberta_base | gen | 0.705 | 0.746 | -0.041 | 0.052 | 0.127 | -0.075
Std. Dev. | STS | Bio_ClinicalBERT | ClinicalSTS | 0.687 | 0.755 | -0.067 | 0.048 | 0.197 | -0.149
Std. Dev. | STS | Bio_ClinicalBERT | clinical | 0.668 | 0.727 | -0.059 | 0.046 | 0.218 | -0.172
Std. Dev. | STS | Bio_ClinicalBERT | combined | 0.628 | 0.774 | -0.146 | 0.041 | 0.284 | -0.243
Std. Dev. | STS | Bio_ClinicalBERT | gen | 0.645 | 0.783 | -0.138 | 0.042 | 0.230 | -0.187
Std. Dev. | STS | FastText | gen | 0.581 | 0.823 | -0.242 | 0.038 | 0.314 | -0.277
Std. Dev. | STS | PubMedBERT | clinical | 0.540 | 0.789 | -0.249 | 0.033 | 0.277 | -0.244
Std. Dev. | STS | PubMedBERT | combined | 0.628 | 0.784 | -0.156 | 0.043 | 0.342 | -0.299
Std. Dev. | STS | PubMedBERT | gen | 0.647 | 0.790 | -0.143 | 0.045 | 0.326 | -0.281
Std. Dev. | STS | Skipgram | gen | 0.548 | 0.859 | -0.311 | 0.035 | 0.469 | -0.435
Std. Dev. | STS | all-MiniLM-L12-v2 | clinical | 0.705 | 0.761 | -0.056 | 0.051 | 0.189 | -0.138
Std. Dev. | STS | all-MiniLM-L12-v2 | combined | 0.545 | 0.789 | -0.243 | 0.036 | 0.297 | -0.261
Std. Dev. | STS | all-MiniLM-L12-v2 | gen | 0.678 | 0.798 | -0.120 | 0.048 | 0.274 | -0.226
Std. Dev. | STS | bert-base-uncased | clinical | 0.590 | 0.774 | -0.184 | 0.037 | 0.272 | -0.235
Std. Dev. | STS | bert-base-uncased | combined | 0.614 | 0.778 | -0.163 | 0.039 | 0.299 | -0.260
Std. Dev. | STS | bert-base-uncased | gen | 0.588 | 0.793 | -0.205 | 0.037 | 0.316 | -0.279
Std. Dev. | STS | biomed_roberta_base | clinical | 0.534 | 0.815 | -0.281 | 0.034 | 0.290 | -0.256
Std. Dev. | STS | biomed_roberta_base | combined | 0.347 | 0.905 | -0.558 | 0.024 | 0.566 | -0.542
Std. Dev. | STS | biomed_roberta_base | gen | 0.584 | 0.792 | -0.208 | 0.037 | 0.252 | -0.215
Top N | MI | - | - | 0.331 | 0.521 | -0.190 | 0.023 | 0.048 | -0.025
Top N | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.782 | 0.710 | 0.072 | 0.149 | 0.131 | 0.017
Top N | MI & STS | Bio_ClinicalBERT | clinical | 0.661 | 0.758 | -0.097 | 0.045 | 0.173 | -0.128
Top N | MI & STS | Bio_ClinicalBERT | combined | 0.730 | 0.743 | -0.013 | 0.059 | 0.166 | -0.107
Top N | MI & STS | Bio_ClinicalBERT | gen | 0.675 | 0.772 | -0.097 | 0.049 | 0.188 | -0.140
Top N | MI & STS | FastText | gen | 0.171 | 0.504 | -0.333 | 0.019 | 0.046 | -0.027
Top N | MI & STS | PubMedBERT | clinical | 0.702 | 0.757 | -0.054 | 0.052 | 0.191 | -0.139
Top N | MI & STS | PubMedBERT | combined | 0.821 | 0.734 | 0.087 | 0.100 | 0.138 | -0.039
Top N | MI & STS | PubMedBERT | gen | 0.727 | 0.744 | -0.016 | 0.058 | 0.209 | -0.150
Top N | MI & STS | Skipgram | gen | 0.135 | 0.552 | -0.417 | 0.018 | 0.068 | -0.050
Top N | MI & STS | all-MiniLM-L12-v2 | clinical | 0.698 | 0.738 | -0.039 | 0.049 | 0.146 | -0.097
Top N | MI & STS | all-MiniLM-L12-v2 | combined | 0.671 | 0.741 | -0.071 | 0.046 | 0.173 | -0.127
Top N | MI & STS | all-MiniLM-L12-v2 | gen | 0.769 | 0.722 | 0.047 | 0.065 | 0.137 | -0.071
Top N | MI & STS | bert-base-uncased | clinical | 0.654 | 0.715 | -0.061 | 0.047 | 0.089 | -0.041
Top N | MI & STS | bert-base-uncased | combined | 0.610 | 0.729 | -0.119 | 0.065 | 0.127 | -0.062
Top N | MI & STS | bert-base-uncased | gen | 0.599 | 0.750 | -0.150 | 0.040 | 0.135 | -0.095
Top N | MI & STS | biomed_roberta_base | clinical | 0.708 | 0.733 | -0.025 | 0.100 | 0.125 | -0.025
Top N | MI & STS | biomed_roberta_base | combined | 0.760 | 0.729 | 0.032 | 0.062 | 0.150 | -0.088
Top N | MI & STS | biomed_roberta_base | gen | 0.722 | 0.747 | -0.025 | 0.055 | 0.125 | -0.070
Top N | STS | Bio_ClinicalBERT | ClinicalSTS | 0.707 | 0.731 | -0.025 | 0.051 | 0.102 | -0.051
Top N | STS | Bio_ClinicalBERT | clinical | 0.702 | 0.739 | -0.037 | 0.049 | 0.154 | -0.105
Top N | STS | Bio_ClinicalBERT | combined | 0.669 | 0.730 | -0.061 | 0.045 | 0.162 | -0.117
Top N | STS | Bio_ClinicalBERT | gen | 0.664 | 0.738 | -0.074 | 0.045 | 0.156 | -0.111
Top N | STS | FastText | gen | 0.638 | 0.748 | -0.110 | 0.043 | 0.159 | -0.116
Top N | STS | PubMedBERT | clinical | 0.736 | 0.738 | -0.003 | 0.055 | 0.168 | -0.112
Top N | STS | PubMedBERT | combined | 0.736 | 0.737 | -0.001 | 0.056 | 0.160 | -0.104
Top N | STS | PubMedBERT | gen | 0.639 | 0.762 | -0.122 | 0.042 | 0.187 | -0.145
Top N | STS | Skipgram | gen | 0.730 | 0.762 | -0.032 | 0.057 | 0.168 | -0.110
Top N | STS | all-MiniLM-L12-v2 | clinical | 0.719 | 0.743 | -0.024 | 0.056 | 0.124 | -0.068
Top N | STS | all-MiniLM-L12-v2 | combined | 0.595 | 0.749 | -0.154 | 0.038 | 0.167 | -0.129
Top N | STS | all-MiniLM-L12-v2 | gen | 0.736 | 0.738 | -0.002 | 0.056 | 0.105 | -0.049
Top N | STS | bert-base-uncased | clinical | 0.640 | 0.723 | -0.082 | 0.046 | 0.160 | -0.114
Top N | STS | bert-base-uncased | combined | 0.599 | 0.774 | -0.174 | 0.037 | 0.216 | -0.178
Top N | STS | bert-base-uncased | gen | 0.658 | 0.796 | -0.138 | 0.044 | 0.249 | -0.206
Top N | STS | biomed_roberta_base | clinical | 0.566 | 0.755 | -0.189 | 0.038 | 0.267 | -0.229
Top N | STS | biomed_roberta_base | combined | 0.530 | 0.820 | -0.290 | 0.032 | 0.262 | -0.230
Top N | STS | biomed_roberta_base | gen | 0.653 | 0.744 | -0.091 | 0.044 | 0.121 | -0.077
mRMR | MI | - | - | 0.430 | 0.591 | -0.161 | 0.031 | 0.075 | -0.044
mRMR | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.716 | 0.749 | -0.032 | 0.053 | 0.107 | -0.054
mRMR | MI & STS | Bio_ClinicalBERT | clinical | 0.613 | 0.735 | -0.122 | 0.039 | 0.134 | -0.095
mRMR | MI & STS | Bio_ClinicalBERT | combined | 0.603 | 0.874 | -0.271 | 0.038 | 0.316 | -0.278
mRMR | MI & STS | Bio_ClinicalBERT | gen | 0.671 | 0.740 | -0.069 | 0.046 | 0.157 | -0.111
mRMR | MI & STS | FastText | gen | 0.289 | 0.728 | -0.439 | 0.023 | 0.144 | -0.121
mRMR | MI & STS | PubMedBERT | clinical | 0.572 | 0.790 | -0.219 | 0.037 | 0.255 | -0.218
mRMR | MI & STS | PubMedBERT | combined | 0.632 | 0.825 | -0.193 | 0.052 | 0.387 | -0.336
mRMR | MI & STS | PubMedBERT | gen | 0.640 | 0.775 | -0.134 | 0.054 | 0.298 | -0.243
mRMR | MI & STS | Skipgram | gen | 0.612 | 0.669 | -0.057 | 0.042 | 0.131 | -0.089
mRMR | MI & STS | all-MiniLM-L12-v2 | clinical | 0.609 | 0.728 | -0.119 | 0.040 | 0.121 | -0.081
mRMR | MI & STS | all-MiniLM-L12-v2 | combined | 0.606 | 0.850 | -0.244 | 0.039 | 0.226 | -0.187
mRMR | MI & STS | all-MiniLM-L12-v2 | gen | 0.763 | 0.727 | 0.036 | 0.071 | 0.115 | -0.044
mRMR | MI & STS | bert-base-uncased | clinical | 0.639 | 0.705 | -0.066 | 0.042 | 0.101 | -0.059
mRMR | MI & STS | bert-base-uncased | combined | 0.713 | 0.720 | -0.006 | 0.062 | 0.104 | -0.042
mRMR | MI & STS | bert-base-uncased | gen | 0.607 | 0.704 | -0.097 | 0.038 | 0.130 | -0.092
mRMR | MI & STS | biomed_roberta_base | clinical | 0.599 | 0.694 | -0.095 | 0.052 | 0.087 | -0.035
mRMR | MI & STS | biomed_roberta_base | combined | 0.617 | 0.767 | -0.150 | 0.058 | 0.185 | -0.127
mRMR | MI & STS | biomed_roberta_base | gen | 0.594 | 0.843 | -0.250 | 0.038 | 0.263 | -0.225
mRMR | STS | Bio_ClinicalBERT | ClinicalSTS | 0.713 | 0.738 | -0.024 | 0.074 | 0.189 | -0.115
mRMR | STS | Bio_ClinicalBERT | clinical | 0.682 | 0.751 | -0.070 | 0.049 | 0.175 | -0.126
mRMR | STS | Bio_ClinicalBERT | combined | 0.529 | 0.760 | -0.231 | 0.033 | 0.236 | -0.203
mRMR | STS | Bio_ClinicalBERT | gen | 0.598 | 0.761 | -0.163 | 0.041 | 0.242 | -0.201
mRMR | STS | FastText | gen | 0.410 | 0.792 | -0.382 | 0.027 | 0.274 | -0.246
mRMR | STS | PubMedBERT | clinical | 0.587 | 0.835 | -0.248 | 0.038 | 0.277 | -0.239
mRMR | STS | PubMedBERT | combined | 0.661 | 0.741 | -0.080 | 0.048 | 0.206 | -0.158
mRMR | STS | PubMedBERT | gen | 0.653 | 0.750 | -0.097 | 0.058 | 0.239 | -0.181
mRMR | STS | Skipgram | gen | 0.399 | 0.814 | -0.414 | 0.026 | 0.422 | -0.396
mRMR | STS | all-MiniLM-L12-v2 | clinical | 0.552 | 0.806 | -0.254 | 0.038 | 0.198 | -0.160
mRMR | STS | all-MiniLM-L12-v2 | combined | 0.563 | 0.745 | -0.182 | 0.035 | 0.145 | -0.110
mRMR | STS | all-MiniLM-L12-v2 | gen | 0.585 | 0.813 | -0.228 | 0.040 | 0.266 | -0.226
mRMR | STS | bert-base-uncased | clinical | 0.587 | 0.792 | -0.205 | 0.038 | 0.208 | -0.170
mRMR | STS | bert-base-uncased | combined | 0.595 | 0.766 | -0.171 | 0.038 | 0.176 | -0.138
mRMR | STS | bert-base-uncased | gen | 0.603 | 0.746 | -0.142 | 0.040 | 0.110 | -0.070
mRMR | STS | biomed_roberta_base | clinical | 0.532 | 0.818 | -0.286 | 0.035 | 0.223 | -0.188
mRMR | STS | biomed_roberta_base | combined | 0.686 | 0.738 | -0.053 | 0.049 | 0.172 | -0.123
mRMR | STS | biomed_roberta_base | gen | 0.603 | 0.738 | -0.135 | 0.039 | 0.120 | -0.081
Table 12: All results for MLP.

###  Gaussian Naïve Bayes (Gaussian NB)

Table 13: All results for Gaussian NB. Selection Hyperparameters | AUROC | AUPRC
---|---|---
Feature Selector | Scorer | Scoring Model | FT Dataset | Test | Train | $\Delta$ | Test | Train | $\Delta$
Identity | - | - | - | 0.579 | 0.602 | -0.023 | 0.029 | 0.055 | -0.027
RFE | - | - | - | 0.603 | 0.799 | -0.196 | 0.038 | 0.132 | -0.094
ReliefF | - | - | - | 0.656 | 0.753 | -0.097 | 0.049 | 0.125 | -0.076
SFM-LinearSVM | - | - | - | 0.573 | 0.809 | -0.236 | 0.036 | 0.257 | -0.220
SFM-XGBoost | - | - | - | 0.559 | 0.761 | -0.202 | 0.036 | 0.236 | -0.200
Std. Dev. | MI | - | - | 0.533 | 0.565 | -0.032 | 0.026 | 0.051 | -0.025
Std. Dev. | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.696 | 0.789 | -0.093 | 0.051 | 0.193 | -0.141
Std. Dev. | MI & STS | Bio_ClinicalBERT | clinical | 0.620 | 0.740 | -0.120 | 0.040 | 0.140 | -0.100
Std. Dev. | MI & STS | Bio_ClinicalBERT | combined | 0.658 | 0.746 | -0.087 | 0.044 | 0.144 | -0.100
Std. Dev. | MI & STS | Bio_ClinicalBERT | gen | 0.639 | 0.754 | -0.115 | 0.042 | 0.181 | -0.139
Std. Dev. | MI & STS | FastText | gen | 0.574 | 0.613 | -0.038 | 0.028 | 0.057 | -0.029
Std. Dev. | MI & STS | PubMedBERT | clinical | 0.713 | 0.797 | -0.084 | 0.053 | 0.222 | -0.169
Std. Dev. | MI & STS | PubMedBERT | combined | 0.713 | 0.781 | -0.068 | 0.055 | 0.201 | -0.146
Std. Dev. | MI & STS | PubMedBERT | gen | 0.694 | 0.790 | -0.096 | 0.051 | 0.148 | -0.097
Std. Dev. | MI & STS | Skipgram | gen | 0.541 | 0.572 | -0.031 | 0.026 | 0.052 | -0.025
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | clinical | 0.693 | 0.786 | -0.093 | 0.051 | 0.204 | -0.153
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | combined | 0.669 | 0.753 | -0.084 | 0.052 | 0.159 | -0.108
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | gen | 0.713 | 0.746 | -0.032 | 0.081 | 0.109 | -0.028
Std. Dev. | MI & STS | bert-base-uncased | clinical | 0.709 | 0.728 | -0.019 | 0.076 | 0.114 | -0.039
Std. Dev. | MI & STS | bert-base-uncased | combined | 0.653 | 0.750 | -0.097 | 0.064 | 0.145 | -0.081
Std. Dev. | MI & STS | bert-base-uncased | gen | 0.700 | 0.742 | -0.042 | 0.082 | 0.115 | -0.033
Std. Dev. | MI & STS | biomed_roberta_base | clinical | 0.669 | 0.703 | -0.033 | 0.059 | 0.127 | -0.067
Std. Dev. | MI & STS | biomed_roberta_base | combined | 0.678 | 0.746 | -0.068 | 0.051 | 0.147 | -0.096
Std. Dev. | MI & STS | biomed_roberta_base | gen | 0.689 | 0.742 | -0.054 | 0.047 | 0.114 | -0.067
Std. Dev. | STS | Bio_ClinicalBERT | ClinicalSTS | 0.711 | 0.742 | -0.031 | 0.054 | 0.150 | -0.096
Std. Dev. | STS | Bio_ClinicalBERT | clinical | 0.672 | 0.737 | -0.065 | 0.044 | 0.146 | -0.102
Std. Dev. | STS | Bio_ClinicalBERT | combined | 0.697 | 0.752 | -0.055 | 0.053 | 0.168 | -0.115
Std. Dev. | STS | Bio_ClinicalBERT | gen | 0.680 | 0.745 | -0.065 | 0.051 | 0.153 | -0.103
Std. Dev. | STS | FastText | gen | 0.731 | 0.708 | 0.024 | 0.044 | 0.075 | -0.031
Std. Dev. | STS | PubMedBERT | clinical | 0.653 | 0.770 | -0.117 | 0.045 | 0.198 | -0.154
Std. Dev. | STS | PubMedBERT | combined | 0.661 | 0.778 | -0.117 | 0.044 | 0.202 | -0.157
Std. Dev. | STS | PubMedBERT | gen | 0.696 | 0.783 | -0.088 | 0.051 | 0.147 | -0.096
Std. Dev. | STS | Skipgram | gen | 0.756 | 0.713 | 0.043 | 0.048 | 0.075 | -0.027
Std. Dev. | STS | all-MiniLM-L12-v2 | clinical | 0.704 | 0.751 | -0.047 | 0.058 | 0.136 | -0.078
Std. Dev. | STS | all-MiniLM-L12-v2 | combined | 0.687 | 0.760 | -0.073 | 0.050 | 0.173 | -0.123
Std. Dev. | STS | all-MiniLM-L12-v2 | gen | 0.696 | 0.762 | -0.066 | 0.054 | 0.145 | -0.091
Std. Dev. | STS | bert-base-uncased | clinical | 0.664 | 0.754 | -0.090 | 0.046 | 0.199 | -0.154
Std. Dev. | STS | bert-base-uncased | combined | 0.722 | 0.774 | -0.052 | 0.055 | 0.139 | -0.084
Std. Dev. | STS | bert-base-uncased | gen | 0.679 | 0.780 | -0.101 | 0.048 | 0.141 | -0.093
Std. Dev. | STS | biomed_roberta_base | clinical | 0.726 | 0.775 | -0.049 | 0.048 | 0.113 | -0.065
Std. Dev. | STS | biomed_roberta_base | combined | 0.689 | 0.783 | -0.094 | 0.051 | 0.227 | -0.176
Std. Dev. | STS | biomed_roberta_base | gen | 0.656 | 0.778 | -0.123 | 0.044 | 0.245 | -0.200
Top N | MI | - | - | 0.521 | 0.535 | -0.014 | 0.025 | 0.048 | -0.023
Top N | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.708 | 0.709 | -0.001 | 0.195 | 0.108 | 0.087
Top N | MI & STS | Bio_ClinicalBERT | clinical | 0.628 | 0.742 | -0.114 | 0.041 | 0.171 | -0.130
Top N | MI & STS | Bio_ClinicalBERT | combined | 0.691 | 0.718 | -0.027 | 0.049 | 0.146 | -0.096
Top N | MI & STS | Bio_ClinicalBERT | gen | 0.628 | 0.737 | -0.109 | 0.041 | 0.154 | -0.113
Top N | MI & STS | FastText | gen | 0.493 | 0.739 | -0.246 | 0.035 | 0.132 | -0.097
Top N | MI & STS | PubMedBERT | clinical | 0.628 | 0.759 | -0.131 | 0.041 | 0.163 | -0.122
Top N | MI & STS | PubMedBERT | combined | 0.738 | 0.767 | -0.028 | 0.075 | 0.145 | -0.070
Top N | MI & STS | PubMedBERT | gen | 0.671 | 0.776 | -0.105 | 0.052 | 0.195 | -0.143
Top N | MI & STS | Skipgram | gen | 0.515 | 0.750 | -0.235 | 0.038 | 0.134 | -0.096
Top N | MI & STS | all-MiniLM-L12-v2 | clinical | 0.629 | 0.743 | -0.113 | 0.041 | 0.144 | -0.103
Top N | MI & STS | all-MiniLM-L12-v2 | combined | 0.702 | 0.736 | -0.034 | 0.056 | 0.128 | -0.072
Top N | MI & STS | all-MiniLM-L12-v2 | gen | 0.727 | 0.747 | -0.019 | 0.090 | 0.105 | -0.015
Top N | MI & STS | bert-base-uncased | clinical | 0.718 | 0.697 | 0.021 | 0.098 | 0.090 | 0.008
Top N | MI & STS | bert-base-uncased | combined | 0.616 | 0.734 | -0.119 | 0.050 | 0.123 | -0.073
Top N | MI & STS | bert-base-uncased | gen | 0.690 | 0.737 | -0.047 | 0.058 | 0.119 | -0.061
Top N | MI & STS | biomed_roberta_base | clinical | 0.675 | 0.698 | -0.023 | 0.056 | 0.136 | -0.080
Top N | MI & STS | biomed_roberta_base | combined | 0.675 | 0.752 | -0.077 | 0.046 | 0.140 | -0.093
Top N | MI & STS | biomed_roberta_base | gen | 0.689 | 0.742 | -0.054 | 0.047 | 0.114 | -0.067
Top N | STS | Bio_ClinicalBERT | ClinicalSTS | 0.647 | 0.741 | -0.094 | 0.042 | 0.105 | -0.063
Top N | STS | Bio_ClinicalBERT | clinical | 0.628 | 0.739 | -0.111 | 0.040 | 0.142 | -0.102
Top N | STS | Bio_ClinicalBERT | combined | 0.617 | 0.728 | -0.111 | 0.039 | 0.131 | -0.091
Top N | STS | Bio_ClinicalBERT | gen | 0.618 | 0.733 | -0.115 | 0.040 | 0.132 | -0.092
Top N | STS | FastText | gen | 0.617 | 0.744 | -0.127 | 0.040 | 0.146 | -0.105
Top N | STS | PubMedBERT | clinical | 0.664 | 0.732 | -0.068 | 0.046 | 0.129 | -0.083
Top N | STS | PubMedBERT | combined | 0.678 | 0.757 | -0.080 | 0.048 | 0.138 | -0.091
Top N | STS | PubMedBERT | gen | 0.707 | 0.776 | -0.070 | 0.054 | 0.189 | -0.135
Top N | STS | Skipgram | gen | 0.639 | 0.749 | -0.110 | 0.043 | 0.142 | -0.099
Top N | STS | all-MiniLM-L12-v2 | clinical | 0.658 | 0.746 | -0.087 | 0.044 | 0.110 | -0.067
Top N | STS | all-MiniLM-L12-v2 | combined | 0.680 | 0.746 | -0.065 | 0.048 | 0.129 | -0.081
Top N | STS | all-MiniLM-L12-v2 | gen | 0.653 | 0.738 | -0.085 | 0.043 | 0.111 | -0.068
Top N | STS | bert-base-uncased | clinical | 0.618 | 0.744 | -0.125 | 0.040 | 0.147 | -0.106
Top N | STS | bert-base-uncased | combined | 0.687 | 0.743 | -0.055 | 0.046 | 0.170 | -0.124
Top N | STS | bert-base-uncased | gen | 0.675 | 0.737 | -0.062 | 0.048 | 0.145 | -0.096
Top N | STS | biomed_roberta_base | clinical | 0.613 | 0.727 | -0.114 | 0.043 | 0.171 | -0.128
Top N | STS | biomed_roberta_base | combined | 0.731 | 0.737 | -0.006 | 0.065 | 0.132 | -0.067
Top N | STS | biomed_roberta_base | gen | 0.676 | 0.737 | -0.060 | 0.048 | 0.139 | -0.091
mRMR | MI | - | - | 0.375 | 0.712 | -0.337 | 0.025 | 0.122 | -0.097
mRMR | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.643 | 0.741 | -0.097 | 0.042 | 0.116 | -0.074
mRMR | MI & STS | Bio_ClinicalBERT | clinical | 0.602 | 0.721 | -0.119 | 0.037 | 0.126 | -0.089
mRMR | MI & STS | Bio_ClinicalBERT | combined | 0.573 | 0.732 | -0.159 | 0.036 | 0.134 | -0.099
mRMR | MI & STS | Bio_ClinicalBERT | gen | 0.616 | 0.741 | -0.125 | 0.039 | 0.113 | -0.074
mRMR | MI & STS | FastText | gen | 0.482 | 0.700 | -0.218 | 0.031 | 0.197 | -0.166
mRMR | MI & STS | PubMedBERT | clinical | 0.680 | 0.727 | -0.047 | 0.056 | 0.143 | -0.087
mRMR | MI & STS | PubMedBERT | combined | 0.674 | 0.756 | -0.083 | 0.059 | 0.146 | -0.087
mRMR | MI & STS | PubMedBERT | gen | 0.652 | 0.796 | -0.144 | 0.043 | 0.271 | -0.228
mRMR | MI & STS | Skipgram | gen | 0.738 | 0.714 | 0.025 | 0.058 | 0.111 | -0.053
mRMR | MI & STS | all-MiniLM-L12-v2 | clinical | 0.656 | 0.735 | -0.080 | 0.048 | 0.102 | -0.054
mRMR | MI & STS | all-MiniLM-L12-v2 | combined | 0.675 | 0.751 | -0.076 | 0.047 | 0.112 | -0.066
mRMR | MI & STS | all-MiniLM-L12-v2 | gen | 0.715 | 0.741 | -0.026 | 0.055 | 0.134 | -0.079
mRMR | MI & STS | bert-base-uncased | clinical | 0.700 | 0.701 | -0.001 | 0.054 | 0.125 | -0.071
mRMR | MI & STS | bert-base-uncased | combined | 0.738 | 0.713 | 0.025 | 0.067 | 0.110 | -0.042
mRMR | MI & STS | bert-base-uncased | gen | 0.632 | 0.702 | -0.069 | 0.043 | 0.122 | -0.079
mRMR | MI & STS | biomed_roberta_base | clinical | 0.638 | 0.712 | -0.074 | 0.045 | 0.085 | -0.039
mRMR | MI & STS | biomed_roberta_base | combined | 0.749 | 0.698 | 0.052 | 0.092 | 0.095 | -0.004
mRMR | MI & STS | biomed_roberta_base | gen | 0.671 | 0.751 | -0.080 | 0.046 | 0.108 | -0.062
mRMR | STS | Bio_ClinicalBERT | ClinicalSTS | 0.716 | 0.734 | -0.018 | 0.088 | 0.116 | -0.028
mRMR | STS | Bio_ClinicalBERT | clinical | 0.652 | 0.755 | -0.104 | 0.043 | 0.125 | -0.082
mRMR | STS | Bio_ClinicalBERT | combined | 0.628 | 0.735 | -0.107 | 0.042 | 0.177 | -0.134
mRMR | STS | Bio_ClinicalBERT | gen | 0.599 | 0.752 | -0.153 | 0.038 | 0.164 | -0.126
mRMR | STS | FastText | gen | 0.518 | 0.755 | -0.237 | 0.035 | 0.227 | -0.193
mRMR | STS | PubMedBERT | clinical | 0.627 | 0.757 | -0.131 | 0.041 | 0.174 | -0.133
mRMR | STS | PubMedBERT | combined | 0.617 | 0.735 | -0.118 | 0.040 | 0.159 | -0.119
mRMR | STS | PubMedBERT | gen | 0.656 | 0.743 | -0.087 | 0.047 | 0.224 | -0.177
mRMR | STS | Skipgram | gen | 0.521 | 0.801 | -0.280 | 0.034 | 0.327 | -0.293
mRMR | STS | all-MiniLM-L12-v2 | clinical | 0.663 | 0.747 | -0.084 | 0.048 | 0.113 | -0.065
mRMR | STS | all-MiniLM-L12-v2 | combined | 0.702 | 0.743 | -0.041 | 0.068 | 0.115 | -0.047
mRMR | STS | all-MiniLM-L12-v2 | gen | 0.698 | 0.749 | -0.050 | 0.072 | 0.113 | -0.041
mRMR | STS | bert-base-uncased | clinical | 0.658 | 0.756 | -0.097 | 0.045 | 0.121 | -0.077
mRMR | STS | bert-base-uncased | combined | 0.647 | 0.776 | -0.129 | 0.043 | 0.157 | -0.114
mRMR | STS | bert-base-uncased | gen | 0.658 | 0.739 | -0.081 | 0.045 | 0.110 | -0.065
mRMR | STS | biomed_roberta_base | clinical | 0.620 | 0.770 | -0.150 | 0.043 | 0.134 | -0.091
mRMR | STS | biomed_roberta_base | combined | 0.719 | 0.730 | -0.011 | 0.115 | 0.107 | 0.008
mRMR | STS | biomed_roberta_base | gen | 0.683 | 0.722 | -0.039 | 0.066 | 0.110 | -0.044
Table 13: All results for Gaussian NB.

###  $k$-nearest neighbors ($k$-NN)

Table 14: All results for $k$-NN. Selection Hyperparameters | AUROC | AUPRC
---|---|---
Feature Selector | Scorer | Scoring Model | FT Dataset | Test | Train | $\Delta$ | Test | Train | $\Delta$
Identity | - | - | - | 0.393 | 0.900 | -0.508 | 0.024 | 0.207 | -0.182
RFE | - | - | - | 0.552 | 0.948 | -0.396 | 0.029 | 0.408 | -0.379
ReliefF | - | - | - | 0.405 | 0.933 | -0.528 | 0.024 | 0.270 | -0.245
SFM-LinearSVM | - | - | - | 0.450 | 0.967 | -0.517 | 0.024 | 0.435 | -0.411
SFM-XGBoost | - | - | - | 0.588 | 0.908 | -0.320 | 0.099 | 0.210 | -0.110
Std. Dev. | MI | - | - | 0.744 | 0.908 | -0.164 | 0.072 | 0.214 | -0.143
Std. Dev. | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.405 | 0.922 | -0.517 | 0.024 | 0.246 | -0.222
Std. Dev. | MI & STS | Bio_ClinicalBERT | clinical | 0.351 | 0.908 | -0.557 | 0.024 | 0.231 | -0.207
Std. Dev. | MI & STS | Bio_ClinicalBERT | combined | 0.515 | 0.881 | -0.366 | 0.026 | 0.165 | -0.139
Std. Dev. | MI & STS | Bio_ClinicalBERT | gen | 0.567 | 0.919 | -0.352 | 0.030 | 0.230 | -0.200
Std. Dev. | MI & STS | FastText | gen | 0.446 | 0.954 | -0.508 | 0.024 | 0.339 | -0.315
Std. Dev. | MI & STS | PubMedBERT | clinical | 0.552 | 0.909 | -0.356 | 0.028 | 0.259 | -0.231
Std. Dev. | MI & STS | PubMedBERT | combined | 0.430 | 0.953 | -0.524 | 0.024 | 0.333 | -0.309
Std. Dev. | MI & STS | PubMedBERT | gen | 0.529 | 0.912 | -0.383 | 0.026 | 0.349 | -0.322
Std. Dev. | MI & STS | Skipgram | gen | 0.585 | 0.927 | -0.342 | 0.099 | 0.316 | -0.216
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | clinical | 0.459 | 0.923 | -0.464 | 0.024 | 0.404 | -0.380
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | combined | 0.446 | 0.956 | -0.510 | 0.024 | 0.364 | -0.339
Std. Dev. | MI & STS | all-MiniLM-L12-v2 | gen | 0.689 | 0.896 | -0.207 | 0.051 | 0.184 | -0.133
Std. Dev. | MI & STS | bert-base-uncased | clinical | 0.562 | 0.879 | -0.317 | 0.029 | 0.208 | -0.179
Std. Dev. | MI & STS | bert-base-uncased | combined | 0.426 | 0.933 | -0.507 | 0.024 | 0.322 | -0.298
Std. Dev. | MI & STS | bert-base-uncased | gen | 0.696 | 0.904 | -0.209 | 0.050 | 0.251 | -0.201
Std. Dev. | MI & STS | biomed_roberta_base | clinical | 0.588 | 0.958 | -0.369 | 0.033 | 0.394 | -0.361
Std. Dev. | MI & STS | biomed_roberta_base | combined | 0.580 | 0.958 | -0.379 | 0.032 | 0.373 | -0.341
Std. Dev. | MI & STS | biomed_roberta_base | gen | 0.696 | 0.799 | -0.103 | 0.047 | 0.189 | -0.142
Std. Dev. | STS | Bio_ClinicalBERT | ClinicalSTS | 0.446 | 0.820 | -0.373 | 0.024 | 0.252 | -0.228
Std. Dev. | STS | Bio_ClinicalBERT | clinical | 0.758 | 0.777 | -0.019 | 0.078 | 0.225 | -0.147
Std. Dev. | STS | Bio_ClinicalBERT | combined | 0.364 | 0.897 | -0.533 | 0.024 | 0.211 | -0.186
Std. Dev. | STS | Bio_ClinicalBERT | gen | 0.438 | 0.928 | -0.490 | 0.024 | 0.312 | -0.288
Std. Dev. | STS | FastText | gen | 0.561 | 0.908 | -0.347 | 0.029 | 0.226 | -0.197
Std. Dev. | STS | PubMedBERT | clinical | 0.588 | 0.948 | -0.360 | 0.034 | 0.428 | -0.394
Std. Dev. | STS | PubMedBERT | combined | 0.397 | 0.928 | -0.531 | 0.024 | 0.296 | -0.271
Std. Dev. | STS | PubMedBERT | gen | 0.548 | 0.905 | -0.357 | 0.028 | 0.238 | -0.210
Std. Dev. | STS | Skipgram | gen | 0.417 | 0.937 | -0.520 | 0.024 | 0.298 | -0.274
Std. Dev. | STS | all-MiniLM-L12-v2 | clinical | 0.672 | 0.864 | -0.192 | 0.044 | 0.177 | -0.133
Std. Dev. | STS | all-MiniLM-L12-v2 | combined | 0.393 | 0.910 | -0.518 | 0.024 | 0.298 | -0.274
Std. Dev. | STS | all-MiniLM-L12-v2 | gen | 0.584 | 0.921 | -0.337 | 0.033 | 0.243 | -0.210
Std. Dev. | STS | bert-base-uncased | clinical | 0.413 | 0.890 | -0.477 | 0.024 | 0.234 | -0.210
Std. Dev. | STS | bert-base-uncased | combined | 0.434 | 0.928 | -0.494 | 0.024 | 0.359 | -0.335
Std. Dev. | STS | bert-base-uncased | gen | 0.442 | 0.934 | -0.492 | 0.024 | 0.412 | -0.388
Std. Dev. | STS | biomed_roberta_base | clinical | 0.555 | 0.936 | -0.381 | 0.028 | 0.312 | -0.284
Std. Dev. | STS | biomed_roberta_base | combined | 0.368 | 0.909 | -0.541 | 0.024 | 0.225 | -0.201
Std. Dev. | STS | biomed_roberta_base | gen | 0.360 | 0.884 | -0.525 | 0.024 | 0.178 | -0.153
Top N | MI | - | - | 0.569 | 0.930 | -0.361 | 0.030 | 0.253 | -0.223
Top N | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.784 | 0.724 | 0.060 | 0.103 | 0.148 | -0.045
Top N | MI & STS | Bio_ClinicalBERT | clinical | 0.376 | 0.912 | -0.536 | 0.024 | 0.236 | -0.212
Top N | MI & STS | Bio_ClinicalBERT | combined | 0.372 | 0.903 | -0.531 | 0.024 | 0.213 | -0.189
Top N | MI & STS | Bio_ClinicalBERT | gen | 0.409 | 0.939 | -0.529 | 0.024 | 0.300 | -0.276
Top N | MI & STS | FastText | gen | 0.629 | 0.954 | -0.325 | 0.049 | 0.400 | -0.351
Top N | MI & STS | PubMedBERT | clinical | 0.421 | 0.941 | -0.520 | 0.024 | 0.396 | -0.371
Top N | MI & STS | PubMedBERT | combined | 0.769 | 0.942 | -0.173 | 0.086 | 0.406 | -0.319
Top N | MI & STS | PubMedBERT | gen | 0.351 | 0.865 | -0.514 | 0.024 | 0.207 | -0.183
Top N | MI & STS | Skipgram | gen | 0.592 | 0.950 | -0.357 | 0.034 | 0.383 | -0.349
Top N | MI & STS | all-MiniLM-L12-v2 | clinical | 0.716 | 0.807 | -0.091 | 0.054 | 0.154 | -0.100
Top N | MI & STS | all-MiniLM-L12-v2 | combined | 0.558 | 0.788 | -0.231 | 0.029 | 0.151 | -0.122
Top N | MI & STS | all-MiniLM-L12-v2 | gen | 0.413 | 0.930 | -0.516 | 0.024 | 0.310 | -0.286
Top N | MI & STS | bert-base-uncased | clinical | 0.723 | 0.819 | -0.096 | 0.056 | 0.178 | -0.122
Top N | MI & STS | bert-base-uncased | combined | 0.607 | 0.808 | -0.200 | 0.038 | 0.311 | -0.273
Top N | MI & STS | bert-base-uncased | gen | 0.599 | 0.821 | -0.222 | 0.036 | 0.245 | -0.210
Top N | MI & STS | biomed_roberta_base | clinical | 0.596 | 0.955 | -0.358 | 0.035 | 0.355 | -0.320
Top N | MI & STS | biomed_roberta_base | combined | 0.426 | 0.934 | -0.508 | 0.024 | 0.328 | -0.304
Top N | MI & STS | biomed_roberta_base | gen | 0.696 | 0.799 | -0.103 | 0.047 | 0.189 | -0.142
Top N | STS | Bio_ClinicalBERT | ClinicalSTS | 0.550 | 0.753 | -0.204 | 0.028 | 0.189 | -0.161
Top N | STS | Bio_ClinicalBERT | clinical | 0.691 | 0.785 | -0.094 | 0.046 | 0.152 | -0.106
Top N | STS | Bio_ClinicalBERT | combined | 0.601 | 0.715 | -0.114 | 0.037 | 0.179 | -0.142
Top N | STS | Bio_ClinicalBERT | gen | 0.540 | 0.717 | -0.177 | 0.027 | 0.119 | -0.092
Top N | STS | FastText | gen | 0.417 | 0.879 | -0.461 | 0.024 | 0.289 | -0.265
Top N | STS | PubMedBERT | clinical | 0.562 | 0.805 | -0.243 | 0.040 | 0.139 | -0.099
Top N | STS | PubMedBERT | combined | 0.558 | 0.810 | -0.252 | 0.040 | 0.141 | -0.101
Top N | STS | PubMedBERT | gen | 0.376 | 0.871 | -0.495 | 0.024 | 0.260 | -0.236
Top N | STS | Skipgram | gen | 0.405 | 0.894 | -0.489 | 0.024 | 0.238 | -0.214
Top N | STS | all-MiniLM-L12-v2 | clinical | 0.674 | 0.778 | -0.105 | 0.043 | 0.160 | -0.117
Top N | STS | all-MiniLM-L12-v2 | combined | 0.360 | 0.888 | -0.528 | 0.024 | 0.211 | -0.187
Top N | STS | all-MiniLM-L12-v2 | gen | 0.566 | 0.752 | -0.186 | 0.029 | 0.201 | -0.171
Top N | STS | bert-base-uncased | clinical | 0.585 | 0.808 | -0.223 | 0.033 | 0.259 | -0.227
Top N | STS | bert-base-uncased | combined | 0.405 | 0.780 | -0.375 | 0.024 | 0.223 | -0.199
Top N | STS | bert-base-uncased | gen | 0.503 | 0.840 | -0.337 | 0.025 | 0.202 | -0.177
Top N | STS | biomed_roberta_base | clinical | 0.372 | 0.890 | -0.518 | 0.024 | 0.264 | -0.240
Top N | STS | biomed_roberta_base | combined | 0.355 | 0.897 | -0.542 | 0.024 | 0.220 | -0.196
Top N | STS | biomed_roberta_base | gen | 0.624 | 0.667 | -0.043 | 0.046 | 0.143 | -0.097
mRMR | MI | - | - | 0.397 | 0.924 | -0.527 | 0.024 | 0.279 | -0.255
mRMR | MI & STS | Bio_ClinicalBERT | ClinicalSTS | 0.541 | 0.802 | -0.260 | 0.027 | 0.198 | -0.171
mRMR | MI & STS | Bio_ClinicalBERT | clinical | 0.434 | 0.837 | -0.403 | 0.024 | 0.243 | -0.219
mRMR | MI & STS | Bio_ClinicalBERT | combined | 0.364 | 0.849 | -0.485 | 0.024 | 0.184 | -0.160
mRMR | MI & STS | Bio_ClinicalBERT | gen | 0.426 | 0.862 | -0.437 | 0.024 | 0.245 | -0.221
mRMR | MI & STS | FastText | gen | 0.401 | 0.932 | -0.532 | 0.024 | 0.287 | -0.263
mRMR | MI & STS | PubMedBERT | clinical | 0.573 | 0.826 | -0.253 | 0.083 | 0.149 | -0.067
mRMR | MI & STS | PubMedBERT | combined | 0.687 | 0.820 | -0.132 | 0.050 | 0.228 | -0.178
mRMR | MI & STS | PubMedBERT | gen | 0.529 | 0.882 | -0.353 | 0.027 | 0.306 | -0.280
mRMR | MI & STS | Skipgram | gen | 0.551 | 0.919 | -0.368 | 0.028 | 0.238 | -0.210
mRMR | MI & STS | all-MiniLM-L12-v2 | clinical | 0.518 | 0.786 | -0.268 | 0.025 | 0.119 | -0.094
mRMR | MI & STS | all-MiniLM-L12-v2 | combined | 0.570 | 0.853 | -0.283 | 0.030 | 0.262 | -0.232
mRMR | MI & STS | all-MiniLM-L12-v2 | gen | 0.645 | 0.786 | -0.141 | 0.039 | 0.162 | -0.123
mRMR | MI & STS | bert-base-uncased | clinical | 0.624 | 0.768 | -0.144 | 0.046 | 0.191 | -0.145
mRMR | MI & STS | bert-base-uncased | combined | 0.463 | 0.766 | -0.303 | 0.024 | 0.206 | -0.181
mRMR | MI & STS | bert-base-uncased | gen | 0.388 | 0.790 | -0.402 | 0.024 | 0.157 | -0.132
mRMR | MI & STS | biomed_roberta_base | clinical | 0.475 | 0.748 | -0.272 | 0.024 | 0.151 | -0.127
mRMR | MI & STS | biomed_roberta_base | combined | 0.515 | 0.895 | -0.380 | 0.025 | 0.213 | -0.188
mRMR | MI & STS | biomed_roberta_base | gen | 0.559 | 0.874 | -0.315 | 0.029 | 0.270 | -0.241
mRMR | STS | Bio_ClinicalBERT | ClinicalSTS | 0.526 | 0.898 | -0.372 | 0.026 | 0.237 | -0.211
mRMR | STS | Bio_ClinicalBERT | clinical | 0.397 | 0.850 | -0.453 | 0.024 | 0.180 | -0.156
mRMR | STS | Bio_ClinicalBERT | combined | 0.397 | 0.915 | -0.518 | 0.024 | 0.253 | -0.229
mRMR | STS | Bio_ClinicalBERT | gen | 0.393 | 0.873 | -0.481 | 0.024 | 0.219 | -0.195
mRMR | STS | FastText | gen | 0.455 | 0.967 | -0.512 | 0.024 | 0.436 | -0.411
mRMR | STS | PubMedBERT | clinical | 0.569 | 0.901 | -0.332 | 0.030 | 0.328 | -0.298
mRMR | STS | PubMedBERT | combined | 0.372 | 0.907 | -0.535 | 0.024 | 0.215 | -0.191
mRMR | STS | PubMedBERT | gen | 0.368 | 0.902 | -0.534 | 0.024 | 0.212 | -0.188
mRMR | STS | Skipgram | gen | 0.388 | 0.928 | -0.539 | 0.024 | 0.302 | -0.278
mRMR | STS | all-MiniLM-L12-v2 | clinical | 0.584 | 0.884 | -0.300 | 0.083 | 0.212 | -0.129
mRMR | STS | all-MiniLM-L12-v2 | combined | 0.506 | 0.873 | -0.368 | 0.025 | 0.218 | -0.193
mRMR | STS | all-MiniLM-L12-v2 | gen | 0.426 | 0.882 | -0.457 | 0.024 | 0.226 | -0.202
mRMR | STS | bert-base-uncased | clinical | 0.413 | 0.940 | -0.527 | 0.024 | 0.299 | -0.275
mRMR | STS | bert-base-uncased | combined | 0.401 | 0.926 | -0.525 | 0.024 | 0.274 | -0.250
mRMR | STS | bert-base-uncased | gen | 0.536 | 0.891 | -0.355 | 0.027 | 0.188 | -0.161
mRMR | STS | biomed_roberta_base | clinical | 0.442 | 0.946 | -0.504 | 0.024 | 0.324 | -0.299
mRMR | STS | biomed_roberta_base | combined | 0.409 | 0.932 | -0.523 | 0.024 | 0.261 | -0.237
mRMR | STS | biomed_roberta_base | gen | 0.446 | 0.778 | -0.332 | 0.024 | 0.193 | -0.169
Table 14: All results for KNearest.