# Evaluating a Learned Admission-Prediction Model as a Replacement for
Standardized Tests in College Admissions

Hansol Lee  Stanford UniversityStanfordCAUSA<EMAIL_ADDRESS>,  René F.
Kizilcec  Cornell UniversityIthaca, NYUSA<EMAIL_ADDRESS>and  Thorsten
Joachims  Cornell UniversityIthaca, NYUSA<EMAIL_ADDRESS>
###### Abstract.

A growing number of college applications has presented an annual challenge for
college admissions in the United States. Admission offices have historically
relied on standardized test scores to organize large applicant pools into
viable subsets for review. However, this approach may be subject to bias in
test scores and selection bias in test-taking with recent trends toward test-
optional admission. We explore a machine learning-based approach to replace
the role of standardized tests in subset generation while taking into account
a wide range of factors extracted from student applications to support a more
holistic review. We evaluate the approach on data from an undergraduate
admission office at a selective US institution (13,248 applications). We find
that a prediction model trained on past admission data outperforms an SAT-
based heuristic and matches the demographic composition of the last admitted
class. We discuss the risks and opportunities for how such a learned model
could be leveraged to support human decision-making in college admissions.

higher education, college admissions, machine learning, predictive modeling,
standardized testing

††ccs: Computer systems organization Embedded systems††ccs: Computer systems
organization Redundancy††ccs: Computer systems organization Robotics††ccs:
Networks Network reliability

##  1\. Introduction

Colleges and universities across the United States receive an increasing
number of student applications for admission to their incoming class each year
(Nietzel, 2021). The Common Application is the primary tool that applicants
use to apply to colleges in the United States.111https://www.commonapp.org/ It
received over 6.6 million first-year applications during the 2021-22 admission
cycle, which constitutes a 9.1% increase over the previous year, and a 21.3%
increase over the 2019-20 cycle (Freeman et al., 2022). The growing number of
applications received by selective colleges and universities has presented an
annual challenge for college admission, especially for institutions that
follow a holistic review process. Holistic review aims to assess each
applicant as a whole by considering a wide range of factors presented in a
student’s application, which makes it a thorough but time-consuming process
(Espinosa et al., 2015; Bastedo et al., 2018; Stevens, 2009).

The large volume of applications received by colleges challenges human
reviewers to perform a thoughtful and equitable review of individual
applications given the limited admission timeline, which typically spans only
a few months. In order to manage the reviewing load, admission officers thus
lean on quantitative measures to prioritize the order of human review and to
most effectively allocate the limited reviewing resources. This has led many
selective colleges in the United States to use heuristics that are based on
standardized test scores such as the ACT or SAT to “triage” their large
applicant pool—that is, to organize their large applicant pool to better
allocate the limited resources available for application review (of Education,
2015). However, there are several limitations to using standardized test
scores to triage the applicant pool – a practice we will refer to as the “SAT-
based” approach. First, there are many unresolved concerns about gender,
racial, and socioeconomic biases in standardized test scores which undermine
the fairness of the SAT-based approach for organizing the applicant pool
(Freedle, 2003; Zwick, 2002; Dixon-RomÁN et al., 2013; Rosser, 1989; Sawyer,
2017; Reardon et al., 2018). Second, this SAT-based approach is dependent on
requiring all applicants to submit their test scores, which may impose a
significant financial burden on many applicants. Moreover, many institutions
began test-optional admission in response to testing site closures during the
coronavirus disease (COVID-19) pandemic, which makes the traditional SAT-based
heuristics incomplete, impractical, and potentially subject to selection
biases.

All these changes and arguments drive admission offices to explore
alternatives to the SAT-based approach in order to overcome the issue of bias
and costliness of standardized tests, their increasing unavailability in test-
optional admission, and their non-holistic nature as a basis for organizing
the admission process. We thus explore a machine learning-based approach that
aims to mimic holistic review by taking into account a wide range of factors
extracted from student applications, not just the standardized test scores. In
particular, we focus on answering the following research question: How well
can an admission prediction model trained on past admission data replace and
improve on the traditional SAT-based heuristic to organize the applicant pool
for review? We examine this question in the context of first-year
undergraduate admissions at a selective US institution and find that the
prediction model is better aligned with existing admission practices at the
case institution compared with an SAT-based heuristic.

##  2\. Related Work

Our work builds on a framework of human-machine collaboration, which advocates
for the design and use of machine learning systems with the intention of
augmenting, not replacing, human contributions (Miller, 2018; Wilson and
Daugherty, 2018). Autor et al. argued that machines may replace humans in
performing routine tasks while complementing humans in performing nonroutine
cognitive tasks (Autor et al., 2003). Jarrahi suggested that machines may
extend humans’ cognition by equipping human decision-makers with comprehensive
data analytics, whereas humans may offer a more holistic and intuitive
approach to decision-making (Jarrahi, 2018). In our work, we aim to leverage
the complementary strengths of machines and humans in the admission process; a
machine-learned admission prediction model can be used to organize a large
applicant pool in a way consistent with past institutional decision-making,
allowing admission officers to use the freed-up resources to engage in the
process of holistic review in a more meaningful way.

Approaches based on machine learning are increasingly studied to support
various aspects of college admissions. For example, Basu et al. used machine
learning algorithms to predict whether a student who is offered admission
would accept the offer, helping institutions to better estimate the sizes of
their entering class (Basu et al., 2019). From the applicants’ perspective,
Gupta et al. and Kiaghadi et al. used machine learning techniques to evaluate
applicants’ chances of admission to help students make informed decisions
about where to apply to college (Gupta et al., 2016; Kiaghadi and Hoseinpour,
2022). In the context of college essays, Alvero et al. explored the use of
computational text analysis to assist human readers in their evaluation of
college application essays (Alvero et al., 2020, 2022).

In particular, our work focuses on the use of machine learning to predict
admission to support the holistic review process of admission officers. As
early as the 1990s, Bruggink et al. and Moore et al. utilized domain knowledge
to build statistical models to predict undergraduate admissions (Bruggink and
Gambhir, 1996; Moore, 1998). More recently, Lux et al. used multi-layer
perceptron and support vector machine algorithms to predict admission
decisions at a small private liberal arts college (Lux et al., 2016). Rees and
Ryder evaluated the usefulness of the random forest algorithm in assisting in
the process of an internal medicine residency program in northern New England
(Rees and Ryder, 2022). Neda and Gago-Masague compared classification
performances of various machine learning algorithms trained on applications
submitted to the Computer Science department at the University of California,
Irvine (Neda and Gago-Masague, 2022).

However, most of the prior work evaluated the learned admission-prediction
model by reporting the overall model performance without explicit discussion
of the implications of their use in admission practice. For instance, one
recent study examined the feasibility of machine learning support for
undergraduate admissions by reporting an overall accuracy of 0.783 and an
AUROC of 0.871 (Neda and Gago-Masague, 2022). However, such numbers alone may
not be sufficient to inform how the learned prediction model should be used in
practice. They often lack a meaningful baseline model performance to compare
with. Additionally, they may not provide enough insights about the limitations
of the prediction model. Our work utilizes a similar modeling approach to
estimate admission probability for each applicant but provides additional
analyses of the model predictions with direct comparison to the SAT-based
approach as a baseline. We also provide specific recommendations for using the
predicted admission probabilities in practice in order to avoid potential
misuse of the prediction model that is superficial in nature.

Among the prior work that involved the use of machine learning for predicting
admission, Waters and Miikkulainen examined a contentious use of an admission
prediction model: to save time in the admission process (Waters and
Miikkulainen, 2014; Burke, 2020). In their 2014 study, Waters and Miikkulainen
used logistic regression to predict graduate admissions in the computer
science department at the University of Texas at Austin. Their work focused on
improving the efficiency of review by cutting the time spent on application
reading and the number of applications to review. We emphasize that our work
focuses on improving the process of holistic review by providing a tool that
better supports the organization of the applicant pool than the SAT-based
method previously used in admission, not by providing a shortlisting tool to
make the admission process cost-effective.

##  3\. Methods

In this section, we describe the first-year undergraduate admission process at
the case university and explain how we develop and evaluate the admission
prediction model using past admission data at the case university. We then
explore how to construct new applicant pools based on the admission prediction
model that can help admission officers with structuring the admission process.

###  3.1. Context

The case university is one of many selective institutions in the United States
where standardized test scores such as the SAT have been used in the admission
process to prioritize the review order of applications. Like many other
colleges, the case institution began test-optional undergraduate admissions
during the COVID-19 pandemic which provided an emergent need for an
alternative to the SAT-based heuristic.

Figure 1. The 2019-2020 admission timeline for early and regular decisions at
the case institution.

Admission at the case institution begins with applicants submitting their
applications for either the Early Decision (ED) or the Regular Decision (RD)
admission cycle (see Figure 1). In the 2019-20 cycle, 1,508 ED applications
were received by November 1, and admission decisions were released in mid-
December. During the RD admission cycle, 11,740 applications were submitted by
January 2, and decisions were sent out by early April. The admit rates for ED
and RD were 28.1% and 9.4%, respectively.222As an ED applicant, applicants
apply to only one institution and must enroll at that school if admitted.
Because of this binding nature, ED applicant pools usually have a higher rate
of admission relative to RD applicant pools. Application review was performed
by professional admission staff in the case institution’s undergraduate
admission office. Each admission officer was responsible for reviewing the
applications from certain geographical regions, and a larger team of external
reviewers supported their initial review of the applications. The review
process spanned about one month for ED and about three months for RD.

The institution requires all first-year applicants to apply through the Common
Application portal. Each completed application via Common Application includes
a variety of information such as a personal essay, descriptions of
extracurricular activities, honors and awards, AP and/or IB scores, SAT or ACT
scores, SAT subject scores, TOEFL/IELTS scores, and a college-specific essay.
In addition, recommendation letters from two teachers, secondary school
reports that include a recommendation letter from the guidance counselor,
school profile and the official transcript of the applicant, and the current
mid-year grade report are submitted alongside the Common Application.

Like many other selective institutions in the US, the case institution has
relied on standardized test scores such as the SAT and the ACT in order to
organize the applicant pool into three subsets, the “Top”, “Middle”, and
“Bottom” pool. Once these pools are defined using the SAT-based model, all
student applications were sent to admission staff for review. Every
application was reviewed by human reviewers regardless of their initial pool
assignment and admission staff reviewed applications in an iterative process
to fill the annual admit target (i.e., the number of spaces available for
matriculation). Grouping similar applicants together is necessary for
equitable review, since many qualified applicants apply to selective colleges
like the case institution, but the human resources and the size of the first-
year class are limited.

Indeed, this SAT-based segmentation into pools shows a correlation with the
admission outcome: 83.1% of the final admitted class for the 2019-2020
admission cycle was identified from the Top pool alone, which only consisted
of 56.6% of the entire applicant pool. However, it is clear that the SAT-based
model did not replace holistic human review: only 28.5% and 8.8% of applicants
in the Top pool were identified as female and underrepresented minority (URM)
applicants, respectively, whereas 51.4% were female and 30.0% were URM
applicants in the final admitted class. In this work, we explore an admission
prediction model that could replace the SAT-based heuristic that had been used
at the case institution for triaging the applicant pool, aligning the
formation of applicant pools with the holistic human review that follows.

###  3.2. Dataset description

The dataset we used to build the prediction model comprises the student
application data submitted to the case institution during the 2019-20
admission cycle as well as their final admission outcomes. First-year
applicants apply through the Common Application which contains a fixed set of
data fields including SAT and ACT scores, SAT subjects, Advanced Placement
(AP) International Baccalaureate (IB), English proficiency test scores
(TOEFL/IELTS), high school GPA, class rank, high school type (e.g., boarding
school), intended major, legacy status, career interests, languages spoken,
personal essays, application information (early or regular, application fee
waivers, etc.), extracurricular activities and time commitment, courses taken
in the current year, high school disciplinary records, honors and awards, and
several demographic information such as gender, ethnic background,
citizenship, age, first-generation status, and parental levels of education.
We consider all information presented in the Common Application except for
personally identifiable information such as names, addresses, contact
information, and names of high schools.

In the 2019-20 admission cycle, all applicants were still required to report
their SAT or ACT scores, and international applicants from non-English
speaking countries were also required to submit their TOEFL or IELTS scores.
In building the admission prediction model, however, we choose to remove
SAT/ACT and TOEFL/IELTS scores from our feature set in order to simulate the
test-optional admission policy. Importantly, we note that we did not have
access to other application materials apart from what is presented in the
Common Application; as a result, several crucial pieces to application review
such as college-specific essays, teacher and guidance counselor recommendation
letters, high school reports, and transcripts are omitted in the feature set.

We filter out a small number of duplicate applications. We also remove
student-athletes and Reserve Officers’ Training Corps (ROTC) applications who
were recruited to the university prior to application submission. We impute
missing values with a unique placeholder value and add an indicator variable
for missingness. Categorical features are one-hot encoded, while some
categorical values with fewer than 1% frequency are merged together as “RARE”.
The final processed data has 13,248 rows and 1,435 columns where each row
corresponds to a single completed application submitted to the case
institution for either the ED or RD admission cycle.

###  3.3. Modeling from past admissions

We frame college admission prediction as a probabilistic binary classification
problem and focus on the following two admission outcomes: admitted (including
admitted and conditionally admitted applicants) and denied (including denied,
wait-listed, and withdrawn applicants). We do not consider admitted
applicants’ matriculation decisions. We leave out a randomly sampled 20% of
the dataset for testing ($n=2650$), and train the model using the remaining
80% of the dataset. We note that 11.5% of applicants in the training data and
11.7% in the testing data were granted final admission from the case
institution in the 2019-20 admission cycle. We fit a Gradient Boosting
Decision Trees model using sklearn’s XGBoost library using its default
parameter settings (Chen and Guestrin, 2016).

Figure 2. Illustration of the SAT-based and the prediction-based “Top pool” in
the testing data ($n=2650$). The initial applicant pool is shown on the left.
The two figures on the right represent the applicant pools organized by the
SAT-based model (middle) and the prediction model (right). In the SAT-based
model, 1,509 applicants are grouped into the Top pool. In the prediction
model, the top 1,509 applicants as sorted by their predicted probability of
admission are grouped into the prediction model’s notion of the “Top pool”.

We assess the potential of the prediction model to replace and improve the
institution’s SAT-based triaging process by using the following evaluation
strategy. In the testing data, 1,509 applicants (57.0% of the testing data)
were placed in the Top pool by the SAT-based model, and 82.5% of the final
admitted class was identified from the Top pool. Similarly, we identify the
top 1,509 applications based on the predicted probability of admission from
the admission prediction model in order to simulate the model’s version of the
Top pool. We then compare the proportion of the eventually admitted class
identified in the Top pools between the SAT-based model and the prediction
model. We also compare the proportion of URM, female, and legacy applicants
represented in the SAT-based and prediction-based Top pools to the proportion
in the final admitted class (see Figure 2 for a visual illustration).

We note that the admission prediction model includes sensitive socio-
demographic attributes of applicants (e.g., gender, race, and ethnicity) and
excludes standardized test scores (i.e., SAT/ACT and TOEFL/IELTS scores). In
Section 4.2, we perform an ablation study of the prediction model to
investigate the effects of standardized test scores and applicants’ sensitive
attributes on the model behavior. Specifically, we explore how much predictive
value can be added to the model by including the standardized test scores, and
how much is lost by excluding the following sensitive socio-demographic
attributes: race/ethnicity, sex, gender identity, URM status, first-generation
status, parents’ education level, religious preference, family income, age,
citizenship, and applicants’ geographical regions. We compare the proportions
of the admitted class identified by the following three models: the baseline
model without standardized test scores but with sensitive attributes, a model
that includes the standardized test scores, and a model that excludes
sensitive attributes.

###  3.4. Construction of new applicant pools

Figure 3. Construction of the new pools based on the admission prediction
model. All applicants (left) are sorted by the predicted probability of
admission (middle). The applicants with the top 10% admission probability are
placed in Pool 10, the next 10% in Pool 9, and so on (right), to produce 10
different applicant pools with varying predicted acceptance rates.

While the SAT-based model used by the case institution defined only three
pools (Top, Middle, and Bottom), the prediction model offers a straightforward
way to create more fine-grained pools and convey their probabilistic
semantics. One approach to creating fine-grained pools is to sort applicants
by their predicted probabilities of admission and aggregate them into 10
different pools as follows: applicants with the top 10% admission probability
are placed in Pool 10, the next top 10% is placed in Pool 9, and so on (see
Figure 3 for a visual illustration). Pool 10 is then predicted to have the
highest number of admitted applicants, Pool 9 is predicted to have the second-
highest number of admitted applicants, and so on.

Note that this will not surface predicted probabilities of admission for
individual applicants to the admissions officers, but only the assignment to
pools. One can now convey to the admission officers that the density of
qualified candidates in each pool is different, but that all pools contain
qualified and not qualified candidates. To set expectations, one can also
communicate the percentage of applicants predicted to be admitted (i.e.,
predicted admit density) in each pool, as it may not vary linearly across the
pools (e.g., pool 10 may contain 50% predicted admits, while pool 9 contains
only 20%).

This use of machine-predicted probabilities to guide decision-making in
admission is not meant to make judgments about individuals; rather, it is
creating pools of applicants with different admission rates. In particular, we
conjecture that much of the uncertainty captured by the prediction model is
epistemic in nature, not aleatoric (Hüllermeier and Waegeman, 2021). This
means that the uncertainty comes from the model’s limitation to truly
understand an applicant’s qualification, not from some external randomness. In
the extreme, for a pool with a 90% acceptance rate, 9 out of 10 applicants in
that pool may individually have a 100% probability of getting admitted, while
one student has a 0% probability. The probability therefore only makes sense
if we think about pools of applicants instead of individual applicants. We
thus stress that it is important to present results to human admission staff
in terms of pools, not in terms of predicted probabilities of admission for
individuals, to avoid any misconception of what the model is capable of.

In Section 4.3, we assess the calibration of the predicted admission rates for
each pool by checking whether or not the predicted admission rate matches the
actual admission rate in each pool. In addition to analyzing the prediction
uncertainty in the aggregate pools of applicants, we further explore the
uncertainty of the prediction model by examining the distribution of the
individual predicted probabilities for denied and admitted applicants.

##  4\. Results

We evaluate the potential of the admission prediction model as a replacement
for and an improvement over the SAT-based triaging process previously used at
the case institution. We find that the prediction model outperforms the SAT-
based model by placing more admits in the Top pool and fewer admits in the
Bottom pool. The Top pool of the prediction model also more closely matches
the final admitted class in terms of the female, URM, and legacy composition
than the SAT-based Top pool. We analyze both the aggregate and individual
predicted probabilities of the admission prediction model and find that the
new applicant pools constructed from the prediction model are well-calibrated
by comparing the predicted and the actual admit rates in each pool.

###  4.1. Comparison to the SAT-based model

Figure 4. Illustration of the proportion of admitted class identified by the
SAT-based model (middle) and by the prediction model (right).

As shown in Figure 4, we find that the prediction model outperforms the SAT-
based model in having more admits in the Top pool and fewer admits in the
Bottom pool. Specifically, the Top pool of the prediction model identifies
91.9% of the final admitted class, compared to the 82.5% in the SAT-based Top
pool ($N=309$, $\chi^{2}(1)=12.206$, $p<0.001$). Conversely, the Bottom pool
of the prediction model consists of 4.5% of the admitted class, while the SAT-
based Bottom pool consists of 9.7% of the admitted class ($N=309$,
$\chi^{2}(1)=6.2642$, $p=0.006$). These results show that the prediction model
is significantly better aligned with the admission criteria of the case
institution than the SAT-based model.

Figure 5. Proportion of URM, female, and legacy applicants in each of the
following pools (from left to right): 1) the entire applicant pool ($n=2650$),
2) the Top pool of the SAT-based model ($n=1509$), 3) the Top pool of the
prediction model ($n=1509$), and 4) the final admitted class ($n=309$).

In addition, Figure 5 shows that the Top pool of the prediction model more
closely matches the final admitted class in terms of the female, URM, and
legacy distributions than the SAT-based Top pool. The URM, female, and legacy
student proportion in the overall applicant pool is 14.6%, 29.2%, and 3.0%,
respectively. We see that 8.7%, 28.6%, and 3.6% of the SAT-based Top pool are
URM, female, and legacy applicants, while 22.9%, 40.0%, and 5.0% of the
prediction-based Top pool are URM, female, and legacy applicants. The
prediction model thus produces a Top pool that more closely matches the
demographic makeup of the final admitted class, where 26.9% of admits are URM
applicants, 45.3% are female applicants, and 9.1% are legacy applicants.

###  4.2. Ablation Study

Figure 6. The proportion of total admitted class captured for each number of
applications reviewed in the testing data ($n=2650$) for the baseline model
(blue), the model that includes standardized test scores (orange), and the
model that excludes sensitive attributes (green).

We find that the model that includes standardized test scores has comparable
performance to the baseline model that excludes them ($N=309$,
$\chi^{2}(1)=0.090$, $p=0.3819$). The model that excludes sensitive attributes
performs worse than the baseline model: it identifies 87.1% of the total
admitted class in the testing set compared to the 91.9% in the baseline
prediction model ($N=309$, $\chi^{2}(1)=3.8684$, $p=0.0246$). In fact, we find
that the model without sensitive attributes (and without standardized test
scores) shows similar performance to the SAT-based model ($N=309$,
$\chi^{2}(1)=2.4592$, $p=0.0584$).

Figure 6 compares the predictive performance for these three models in terms
of the proportion of applicants in the admitted class given a number of
reviewed applicants, where applicants are reviewed in the order of their
predicted scores (from highest to lowest). The dashed vertical line indicates
the number of applicants in the SAT-based Top pool, with its corresponding
predictive performance (dashed vertical line). We use this visualization to
further understand the added predictive value of standardized test scores and
sensitive socio-demographic attributes in the admission prediction model at
all stages of a structured reviewing process. For any number of reviewed
applicants, the baseline admission prediction model (excluding standardized
test scores) is able to identify a similar proportion of admitted applicants
as the model that includes standardized test scores. The model that excludes
sensitive attributes, however, performs consistently lower in identifying
applicants in the admitted class at any stage of the review process. In fact,
it performs similarly to the SAT-based model (indicated by a black point) for
the number of reviews corresponding to the SAT-based Top pool size.

Table 1. Model comparison of the proportion of the admitted class, URM, female, and legacy applicants in each model’s Top pool ($n=1509$): SAT-based model; baseline prediction model; baseline with standardized test scores added, and baseline with sensitive socio-demographic attributes removed. | Admitted | URM | Female | Legacy
---|---|---|---|---
SAT-based model | 82.5% | 8.7% | 28.6% | 3.6%
Baseline model | 91.9% | 22.9% | 40.0% | 5.0%
Add standardized test scores | 92.6% | 20.3% | 40.8% | 5.0%
Remove sensitive attributes | 87.1% | 14.6% | 31.5% | 5.0%

We further examine how the three models compare in terms of the proportion of
URM, female, and legacy students that they predict to be in the Top pool.
Table 1 shows that there are no significant differences in the composition of
the Top pool as a result of adding standardized test scores to the baseline
model, though it reduces the proportion of URM applicants by 2.6 percentage
points (pp) ($\chi^{2}=2.98,p=0.085$). However, removing sensitive socio-
demographic attributes from the baseline model not only reduces the recall
rate as noted above, but it also significantly reduced the proportion of URM
applicants by 8.3pp ($\chi^{2}=34.0,p<0.001$) and female applicants by 8.5pp
($\chi^{2}=23.6,p<0.001$) in the Top pool. The share of legacy applicants in
the Top pool remains unchanged from adding or removing these features.

###  4.3. Evaluating uncertainty of the prediction model

Figure 7. Predicted admit rate (blue) and actual admit rate (red) for each of
the 10 pools constructed from the prediction model. Error bars indicate 95%
confidence intervals computed using the Clopper-Pearson (exact) method.

Figure 8. Predicted admit rate (blue) and actual admit rate (red) of female
(left) and URM (right) applicants for each of the 10 pools constructed from
the prediction model. Error bars indicate 95% confidence intervals computed
using the Clopper-Pearson (exact) method.

We find that new applicant pools constructed with the predicted probabilities
are reasonably well-calibrated at the level of the defined pools. We compute
the predicted admission rate by taking the average of the predicted
probabilities in each pool, and the actual admission rate by taking the
proportion of actual admits in each pool. As presented in Figure 7, we see
that the predicted admission rate closely follows the actual admission rate in
each pool ($r(10)=0.998$, $p<0.001$). Figure 8 also shows a similar trend for
female applicants ($r(10)=0.996$, $p<0.001$) as well as URM applicants
($r(10)=0.985$, $p<0.001$) across pools.

Figure 9. Density plot of estimated admission probability scores from 0.0 to
1.0 for denied (blue) and admitted (orange) applicants in the testing set.

While the model is reasonably calibrated across the ten pools, an applicant’s
placement in a given pool does not indicate whether they should be admitted or
not. For example, a student in Pool 10 is not necessarily a stronger applicant
than a student in Pool 9. We highlight this issue by examining the
distribution of predicted probabilities for individual applicants. Consider
the following distribution of predicted probabilities for denied and admitted
applicants shown in Figure 9. We see that most of the predicted probabilities
are concentrated on the lower end for the denied applicants, suggesting that
the prediction model is able to accurately discern most of the denied
applicants. On the other hand, the predicted probabilities for the admitted
applicants are more widely spread across the entire range of predicted
probabilities, implying that the prediction model has limited knowledge about
who gets admitted. This suggests that the final admission decisions may depend
largely on human evaluation of other parts of the applications such as teacher
recommendations, personal essays, and transcripts that are missing from the
prediction model; in other words, the prediction model exhibits significant
epistemic uncertainty (Hüllermeier and Waegeman, 2021). The aggregate pools
are meant to provide a coarse yet meaningful organizational structure of the
application pool for effective reading for admission officers, and as a safety
net to minimize the chance that any qualified applicant gets overlooked.

##  5\. Discussion

In this work, we set out to answer the following question: How well can a
prediction model trained on past admission data replace and improve on the
traditional SAT-based heuristic to organize the applicant pool for review? We
identify three aspects that contribute to answering this question. First, we
found that the prediction model outperforms the SAT-based model by placing
more admits in the Top pool and fewer admits in the Bottom pool. Second, the
Top pool of the prediction model more closely matches the final admitted class
in terms of the female, URM, and legacy composition than the SAT-based Top
pool. Finally, we showed that the new applicant pools constructed from the
prediction model are reasonably calibrated by comparing the predicted and the
actual acceptance rates in each pool, allowing for direct interpretation of
the prediction model’s applicant pools in a holistic admission process.

###  5.1. Implications

Many institutions began test-optional admission in response to testing site
closures during the COVID-19 pandemic, which made the traditional SAT-based
heuristic impractical. In our development and evaluation of the admission
prediction model, we found that the prediction model may serve as a practical
alternative to the SAT-based model for organizing the applicant pool; it can
be trained using other already available student information in the Common
Application excluding the standardized test scores (SAT and ACT) and English
proficiency scores (TOEFL and IELTS). Because the prediction model represents
a larger set of information provided in the student application, it can also
be seen as more holistic compared to the SAT-based model which is only based
on standardized test scores and a few demographic markers.

Comparative analysis of the prediction-based and the SAT-based Top pools
suggests that the prediction model may not only replace but also improve the
triaging process in important ways. We found that in the past application
cycle, the prediction model would have identified 9.4% more admitted
applicants in the Top pool and 5.2% fewer admitted applicants in the Bottom
pool compared to the SAT-based model. In other words, the prediction model
outperforms the SAT-based model in terms of both the true positive rate (i.e.
more admits in the Top pool) and the false negative rate (i.e. fewer admits in
the Bottom pool). The prediction model further improves on the traditional
SAT-based model as the prediction-based Top pool contains a pool of applicants
with a female, URM and legacy composition that is closer to the actual
admitted class compared to the SAT-based Top pool. These results suggest that
the prediction model is able to organize the applicant pool in a way that
better reflects the institution’s admission goals, which is in line with the
fact the prediction model is indeed trained on the institution’s past
admission decisions.

The results of the ablation study provide valuable insights into the relative
importance of different features in developing the admission prediction model.
We found that the inclusion of standardized test scores does not significantly
affect the performance of the prediction model, yet excluding sensitive socio-
demographic attributes has a negative impact in terms of both the proportion
of the admitted class and the demographic composition of the Top pool. These
results support the use of sensitive attributes in developing admission
prediction models to improve both accuracy (i.e. proportion of admitted class
identified in the Top pool) and equity (i.e. demographic composition of the
Top pool compared to that of the admitted class). Our findings contribute to
the ongoing discussions about the ethical and practical implications of the
use of sensitive attributes in predictive modeling in education (Kleinberg et
al., 2018; Yu et al., 2020, 2021; Paquette et al., 2020).

The upside of using the prediction model instead of the SAT-based model is
that the admission practices are no longer tied to the biases in the
standardized test scores that do not align with the institution’s values
(Freedle, 2003; Zwick, 2002; Dixon-RomÁN et al., 2013; Rosser, 1989; Sawyer,
2017; Reardon et al., 2018). However, because the admission prediction model
is directly informed by past admission decisions, it is important to ensure
that the past admission data is appropriate for developing and using the
resulting prediction model. So, any use of the model should be accompanied by
a governance process that ensures that the dynamics of the admission process
continue to reflect the potentially changing values of the institution. A
prediction model is a tool, where thoughtful use could untether admission
practices from biases in external scores to iteratively improve admission
practices, but thoughtless use can also cement existing inequities (Burke,
2020).

One new affordance of the prediction model is that the new applicant pools
come with predicted acceptance rates that are well-calibrated to the true
acceptance rates. In contrast, the SAT-based model did not yield pools that
are calibrated to actual acceptance rates. This suggests that the prediction
model may further inform the admission process by enabling a direct
interpretation of the applicant pools in terms of their predicted acceptance
rates. For example, the institution may set itself the goal to double the
acceptance rates in the lower pools, encouraging admission staff to find
creative new ways of identifying qualified applicants that do not fit the
typical profile. In this way, the prediction model may serve as a way to
target a subset of applicants to examine more closely and highlight new
applicants who might have been overlooked.

###  5.2. Limitations and Future Work

There are several limitations to building the admission prediction model.
First, admission decisions are not independent as they are about creating a
class, but they were modeled as if they are independent when building the
prediction model. Next, the application data used to train the admission
prediction model did not include the full information that the human reviewers
use to make their decisions. This work only included the data available in the
Common Application, but there are many other aspects of student applications
that heavily influence admission decisions (e.g. student essays, and letters
of recommendation).

Another potential direction for improving the prediction model is to account
for covariate shifts. For example, the admission prediction model was
developed on admission data where SAT/ACT scores were mandatory, which may
have a different distribution from future years where tests are optional or no
longer factor in the decision at all. Even though the SAT and ACT scores are
excluded entirely from the feature set for building the admission prediction
model, there may potentially be a large covariate shift between the year in
which the model is trained and tested and the year in which the trained model
is to be deployed. The model evaluation results presented in this work are
based on the testing set from the same year that the model is trained on, and
this may not be an accurate estimation of the model performance in another
year because of the potential shift in data distributions. Future work may
attempt to account for the potential covariate shift across datasets from
different years by training a classifier for identifying whether a given data
point belongs to the training or the testing distribution, and using it to
assign more weights to training instances that are closer to a testing
distribution (Bickel et al., 2009).

Given the interaction of the prediction model with the human decision-making
process, the full impact of the prediction model on the admission process and
final decisions requires evaluation in a controlled randomized trial. In
addition to assessing the impact of including the prediction model in the
admission practice, future work could explore ways to provide explanations to
admission staff as to how the pools are computed and why a given applicant is
placed in a particular pool. Comparing the admission decisions between the
prediction model and admission staff may help human decision-makers reflect on
their decisions and see where they may have blind spots. It may also help
improve the prediction model by identifying where the model may have blind
spots, and admission staff may be able to offer insight into where additional
data collection may be helpful to make the predictions better.

## References

  * (1)
  * Alvero et al. (2020) AJ Alvero, Noah Arthurs, Anthony Lising Antonio, Benjamin W Domingue, Ben Gebre-Medhin, Sonia Giebel, and Mitchell L Stevens. 2020\.  AI and holistic review: informing human reading in college admissions. In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_. 200–206.
  * Alvero et al. (2022) AJ Alvero, Jasmine Pal, and Katelyn M Moussavian. 2022\.  Linguistic, cultural, and narrative capital: computational and human readings of transfer admissions essays.  _J Comput Soc Sci_ 5, 2 (Sept. 2022), 1709–1734.
  * Autor et al. (2003) David H. Autor, Frank Levy, and Richard J. Murnane. 2003\.  The Skill Content of Recent Technological Change: An Empirical Exploration.  _The Quarterly Journal of Economics_ 118, 4 (2003), 1279–1333.
  * Bastedo et al. (2018) Michael Bastedo, Nicholas Bowman, Kristen Glasener, and Jandi Kelly. 2018.  What are We Talking About When We Talk About Holistic Review? Selective College Admissions and its Effects on Low-SES Students.  _The Journal of Higher Education_ 89 (04 2018), 1–24.
  * Basu et al. (2019) Kanadpriya Basu, Treena Basu, Ron Buckmire, and Nishu Lal. 2019\.  Predictive Models of Student College Commitment Decisions Using Machine Learning.  _Data_ 4, 2 (2019).  https://doi.org/10.3390/data4020065
  * Bickel et al. (2009) Steffen Bickel, Michael Brückner, and Tobias Scheffer. 2009\.  Discriminative Learning Under Covariate Shift.  _J. Mach. Learn. Res._ 10 (Dec. 2009), 2137–2155.
  * Bruggink and Gambhir (1996) Thomas H. Bruggink and Vivek Gambhir. 1996.  Statistical Models for College Admission and Enrollment: A Case Study for a Selective Liberal Arts College.  _Research in Higher Education_ 37, 2 (1996), 221–240.
  * Burke (2020) Lilah Burke. 2020\.  The Death and Life of an Admissions Algorithm.  _Inside Higher Ed_ (December 2020).  https://www.insidehighered.com/admissions/article/2020/12/14/u-texas-will-stop-using-controversial-algorithm-evaluate-phd
  * Chen and Guestrin (2016) Tianqi Chen and Carlos Guestrin. 2016.  XGBoost: A Scalable Tree Boosting System. In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ (San Francisco, California, USA) _(KDD ’16)_. Association for Computing Machinery, New York, NY, USA, 785–794.  https://doi.org/10.1145/2939672.2939785
  * Dixon-RomÁN et al. (2013) Ezekiel J. Dixon-RomÁN, Howard T. Everson, and John J. Mcardle. 2013.  Race, Poverty and SAT Scores: Modeling the Influences of Family Income on Black and White High School Students’ SAT Performance.  _Teachers College Record_ 115, 4 (2013), 1–33.  https://doi.org/10.1177/016146811311500406 arXiv:https://doi.org/10.1177/016146811311500406
  * Espinosa et al. (2015) Lorelle L Espinosa, Gary Orfield, and Matthew N Gaertner. 2015\.  Race, class, and college access: Achieving diversity in a shifting legal landscape.  _Washington, DC: American Council on Education_ (2015).
  * Freedle (2003) Roy O. Freedle. 2003\.  Correcting the SAT’s ethnic and social-class bias: A method for reestimating SAT scores.  _Harvard Educational Review_ 73, 1 (2003), 1–43.
  * Freeman et al. (2022) Mark Freeman, Brian Heseung Kim, Preston Magouirk, and Trent Kajikawa. 2022.  Deadline update: first-year application trends through March 15.  _Common App_ (March 2022).  https://s3.us-west-2.amazonaws.com/ca.research.publish/Research+briefs+2020/20211123_Deadline_Update_FY.pdf
  * Gupta et al. (2016) Narender Gupta, Aman Sawhney, and Dan Roth. 2016.  Will I Get in? Modeling the Graduate Admission Process for American Universities. In _2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW)_. 631–638.  https://doi.org/10.1109/ICDMW.2016.0095
  * Hüllermeier and Waegeman (2021) Eyke Hüllermeier and Willem Waegeman. 2021.  Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods.  _Machine Learning_ 110, 3 (March 2021), 457–506.  https://doi.org/10.1007/s10994-021-05946-3
  * Jarrahi (2018) Mohammad Hossein Jarrahi. 2018\.  Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making.  _Business Horizons_ 61, 4 (2018), 577–586.  https://doi.org/10.1016/j.bushor.2018.03.007
  * Kiaghadi and Hoseinpour (2022) Mohammadreza Kiaghadi and Pooya Hoseinpour. 2022.  University admission process: a prescriptive analytics approach.  _Artificial Intelligence Review_ (2022), 1–24.
  * Kleinberg et al. (2018) Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018.  Algorithmic Fairness.  _AEA Papers and Proceedings_ 108 (May 2018), 22–27.  https://doi.org/10.1257/pandp.20181018
  * Lux et al. (2016) Thomas Lux, Randall Pittman, Maya Shende, and Anil Shende. 2016.  Applications of Supervised Learning Techniques on Undergraduate Admissions Data. In _Proceedings of the ACM International Conference on Computing Frontiers_ (Como, Italy) _(CF ’16)_. Association for Computing Machinery, New York, NY, USA, 412–417.  https://doi.org/10.1145/2903150.2911717
  * Miller (2018) Steven M. Miller. 2018\.  AI: Augmentation, more so than automation.  _Asian Management Insights._ 5, 1 (2018), 1–20.
  * Moore (1998) James S Moore. 1998\.  An expert system approach to graduate school admission decisions and academic performance prediction.  _Omega_ 26, 5 (1998), 659–670.  https://doi.org/10.1016/S0305-0483(98)00008-5
  * Neda and Gago-Masague (2022) Barbara Martinez Neda and Sergio Gago-Masague. 2022.  Feasibility of Machine Learning Support for Holistic Review of Undergraduate Applications. In _2022 International Conference on Applied Artificial Intelligence (ICAPAI)_. Halden, Norway, 1–6.  https://doi.org/10.1109/ICAPAI55158.2022.9801571
  * Nietzel (2021) Michael Nietzel. 2021\.  “New Data: Applications Surge At Larger, Selective Colleges”.  _Forbes_ (Mar 2021).  https://www.forbes.com/sites/michaeltnietzel/2021/03/10/new-data-applications-surge-at-larger-selective-colleges/?sh=2ca76ce11e45
  * of Education (2015) United States Department of Education. 2015\.  Title VI: Admissions: Princeton University (02086002).  _United States Department of Education Office for Civil Rights, Region II_ (September 2015).  https://www2.ed.gov/about/offices/list/ocr/docs/investigations/02086002.html
  * Paquette et al. (2020) Luc Paquette, Jaclyn Ocumpaugh, Ziyue Li, Alexandra Andres, and Ryan Baker. 2020.  Who’s Learning? Using Demographics in EDM Research.  _Journal of Educational Data Mining_ 12, 3 (Oct. 2020), 1–30.  https://doi.org/10.5281/zenodo.4143612 Number: 3.
  * Reardon et al. (2018) Sean F. Reardon, Demetra Kalogrides, Erin M. Fahle, Anne Podolsky, and Rosalía C. Zárate. 2018\.  The Relationship Between Test Item Format and Gender Achievement Gaps on Math and ELA Tests in Fourth and Eighth Grades.  _Educational Researcher_ 47, 5 (2018), 284–294.  https://doi.org/10.3102/0013189X18762105 arXiv:https://doi.org/10.3102/0013189X18762105
  * Rees and Ryder (2022) Christiaan A. Rees and Hilary F. Ryder. 2022.  Machine Learning for The Prediction of Ranked Applicants and Matriculants to an Internal Medicine Residency Program.  _Teaching and Learning in Medicine_ 0, 0 (2022), 1–10.  https://doi.org/10.1080/10401334.2022.2059664 arXiv:https://doi.org/10.1080/10401334.2022.2059664  PMID: 35591808.
  * Rosser (1989) Phyllis Rosser. 1989\.  _The SAT Gender Gap: Identifying the Causes._ Center for Women Policy Studies., Washington, DC.
  * Sawyer (2017) Art Sawyer. 2017\.  How the New SAT has Disadvantaged Female Testers.  _Compass Education Group_ (October 2017).  https://www.compassprep.com/new-sat-has-disadvantaged-female-testers/
  * Stevens (2009) Mitchell L Stevens. 2009\.  _Creating a class_.  Harvard University Press.
  * Waters and Miikkulainen (2014) Austin Waters and Risto Miikkulainen. 2014.  Grade: Machine learning support for graduate admissions.  _Ai Magazine_ 35, 1 (2014), 64–64.
  * Wilson and Daugherty (2018) H James Wilson and Paul R Daugherty. 2018.  Collaborative intelligence: Humans and AI are joining forces.  _Harvard Business Review_ 96, 4 (2018), 114–123.
  * Yu et al. (2021) Renzhe Yu, Hansol Lee, and René F. Kizilcec. 2021.  Should College Dropout Prediction Models Include Protected Attributes?. In _Proceedings of the Eighth ACM Conference on Learning @ Scale_. ACM, Virtual Event Germany, 91–100.  https://doi.org/10.1145/3430895.3460139
  * Yu et al. (2020) Renzhe Yu, Qiujie Li, Christian Fischer, Shayan Doroudi, and Di Xu. 2020.  Towards Accurate and Fair Prediction of College Success: Evaluating Different Sources of Student Data. In _Proceedings of The 13th International Conference on Educational Data Mining (EDM 2020)_. International Educational Data Mining Society, 292–301.
  * Zwick (2002) Rebecca Zwick. 2002\.  Is the SAT a ‘Wealth Test’?  _Phi Delta Kappan_ 84, 4 (2002), 307–311.  https://doi.org/10.1177/003172170208400411 arXiv:https://doi.org/10.1177/003172170208400411


# Heavy Higgs bosons at a 100 TeV hadron collider

Kun Wang  College of Science, University of Shanghai for Science and
Technology, Shanghai 200093, China  Pengfu Tian  School of Physics and
Technology, Wuhan University, Wuhan 430072, China  Jingya Zhu  School of
Physics and Electronics, Henan University, Kaifeng 475004, China

###### Abstract

In this work, we consider two heavy Higgs bosons ($H$ and $A$) of about
$2.4\\!\sim\\!4.8\ \mathrm{TeV}$ in the semi-constrained NMSSM (scNMSSM) in
our previous work. We study them in the $pp\to b\bar{b}H/A\to
b\bar{b}t\bar{t}$ channel at a $100\ \mathrm{TeV}$ hardon collider. We
calculate their cross section of production and branching ratios of decay, and
compare the signal with the simulation result in a reference. We finally find
that it is hard to detect the heavy Higgs bosons in this channel at the $100\
\mathrm{TeV}$ collider with an integrated luminosity of $3\ \mathrm{ab}^{-1}$,
because their branching ratio to SM particles including $t\bar{t}$ are
seriously suppressed by the decay to SUSY particles. We will add discussions
about detecting in other channels later.

††preprint:  arXiv:

##  I Introduction

It is well known for long that the Standard Model (SM) is an incomplete
description of nature, but up to now, it is still very confusing what the new
physics beyond the SM (BSM) are. After a SM-like Higgs boson was discovered at
the LHC in 2012 Aad  _et al._ (2012); Chatrchyan  _et al._ (2012); Aad  _et
al._ (2016); Sirunyan  _et al._ (2019); Aad  _et al._ (2020); Chatrchyan  _et
al._ (2020); Sopczak (2020), it came to a natural question whether additional
Higgs bosons exist. In theory, additional Higgs bosons can exist in numerous
natural BSM models, see recent relevant studies in the Minimal Supersymmetric
Standard Model (MSSM) Medina and Schmidt (2017); Bechtle  _et al._ (2017);
Barman  _et al._ (2016); Carena  _et al._ (2016a); Djouadi and Quevillon
(2013); Bechtle  _et al._ (2013), the Next-to MSSM (NMSSM) Ma  _et al._
(2021); Shang  _et al._ (2020); Cao  _et al._ (2017); Wang  _et al._ (2017);
Carena  _et al._ (2016b); Bomark  _et al._ (2015); Cao  _et al._ (2014a); King
_et al._ (2014); Baglio  _et al._ (2014); Cao  _et al._ (2012a, b, 2013), the
Minimal dilaton model Cao  _et al._ (2016); Agarwal  _et al._ (2016); Cao  _et
al._ (2014b); Liu  _et al._ (2019), the Little Higgs Models Han  _et al._
(2013); Cheung  _et al._ (2018), etc. In experiment, many searches for heavy
neutral Higgs have been carried out in $\tau^{+}\tau^{-}$ Sirunyan  _et al._
(2018); Aaboud  _et al._ (2018a), $t\bar{t}$, $b\bar{b}$, $\mu^{+}\mu^{-}$,
$ZZ$, $WW$, $hh$, $hV$ channels, which are reviewed in Zyla  _et al._ (2020);
Aaboud  _et al._ (2018b). And recently evidence for a new scalar around 151
GeV was accumulated with significances of $5.1\sigma$ local and $4.8\sigma$
global Crivellin  _et al._ (2021), or $4.1\sigma$ local and $3.5\sigma$ global
Fowlie (2021).

Moreover, there are also several future high energy colliders in proposals
that may give great opportunities to detect heavy Higgs, for example, the
Future hadron-hadron Circular Collider (FCC-hh) at CERN Bicer  _et al._ (2014)
and the Super-$pp$-Collider (SppC) Gao (2021); Dong  _et al._ (2018) in China
both of which involve construction of a 50-100 TeV $pp$ collider Hinchliffe
_et al._ (2015). Besides, a muon collider with the energy of muti-TeV is also
in proposal Bartosik  _et al._ (2020); Alsharoa  _et al._ (2003). And numerous
works have been carried out on detecting heavy Higgs at future colliders. In
Ref. Hajer  _et al._ (2015), $pp\to b\bar{b}H/A\to b\bar{b}\tau\tau$ and
$pp\to b\bar{b}H/A\to b\bar{b}t\bar{t}$ channels at a 100-TeV $pp$-collider
were studied, the exclusion limits for the heavy Higgs searches were extend
the up to $M_{H}\sim 10\ \mathrm{TeV}$, except for the low $\tan\beta$
regions. In Ref. Baer  _et al._ (2021), $pp\to
H/A\to\tilde{\chi}^{\pm}_{1}\tilde{\chi}^{\mp}_{2}$ at a $100$-TeV hardon
collider, with the signal of $4\ell+\makebox[0.86108pt][l]{$E$}/$, was
studied. In Ref. Han  _et al._ (2021), at a multi-TeV muon collider the
discovery potential of heavy Higgs bosons in the Two Higgs Doublet Models
(2HDMs) was studied and the discrimination power among different types of
2HDMs were explored.

In this work, we study the heavy Higgs bosons from our previous work in the
semi-constrained NMSSM Wang and Zhu (2020a). We calculate their cross section
of production and branching ratios of decay, and investigate their discovery
potential in the $pp\to b\bar{b}H/A\to b\bar{b}t\bar{t}$ channel at a $100\
\mathrm{TeV}$ collider. We will supplement discussions about detecting in
other channels later.

The remind of this paper is listed as follows. In Sec. II, we give a brief
introduction to the semi-constrained NMSSM. Then in Sec. III we show our
calculations and discussions. Finally in Sec. IV, we draw our conclusions.

##  II The semi-constrained NMSSM

There are two Higgs doublets in MSSM, one generating masses to the up-type
fermions and the other to down-type ones, with the ratio of vacuum expected
values (VEVs) equal to $\tan\beta$. Thus the MSSM predicts five Higgs bosons
(two CP-even, one CP-odd, and a pair of charged ones), four neutralinos, and
two pairs of charginos. The NMSSM extends MSSM by adding a singlet superfield
$\hat{S}$ with the superpotential given by $W^{\rm
MSSM}_{\mu\\!\to\\!\lambda\hat{S}}\\!+\\!\kappa\hat{S}^{3}/3$, with the
massive $\mu$-term generated dynamically when the singlet scalar develops a
VEV Ellwanger  _et al._ (2010); Maniatis (2010). Thus the NMSSM predicts in
addition two Higgs bosons (one CP-even and the other CP-odd), and one
neutralino. We denote the heaviest CP-even Higgs boson as $H$ with the singlet
component of $|S_{33}|^{2}$, the heavy CP-odd Higgs boson as $A$ with singlet
component of $|P_{22}|^{2}$, and the lightest neutralino as
$\tilde{\chi}^{0}_{1}$ with the singlino component of $|N_{15}|^{2}$.

##  III Results and Discussions

In this work we use the surviving samples in our former work Wang and Zhu
(2020a), where we use the package NMSSMTools-5.5.2 Ellwanger  _et al._ (2005);
Ellwanger and Hugonie (2006, 2007); Das  _et al._ (2012) to scan the parameter
space and calculate corresponding quantities, and consider the following
constraints: (i) Theoretical constraints of vacuum stability and no Landau
pole below the GUT scale Ellwanger  _et al._ (2005); Ellwanger and Hugonie
(2006). (ii) Flavour constraints of rare B-meson decays, and the mass
differences of $D$ meson Tanabashi  _et al._ (2018); Aaij  _et al._ (2013);
Lees  _et al._ (2012a, b). (iii) A $123\\!\sim\\!127$-GeV Higgs boson with
signal predictions consistent globally with LHC Higgs data Aad  _et al._
(2020); Sirunyan  _et al._ (2019); Aad  _et al._ (2016). (iv) Constraints from
searches for additional Higgs and exotic decay of the SM-like Higgs, imposed
with HiggsBounds-5.5.0, including the excluding limit $19\%$ of the invisible
Higgs decay Bechtle  _et al._ (2015, 2014, 2012, 2011, 2010). (v) Constraints
from muon g-2 at $2\sigma$ level Abi  _et al._ (2021). (vi) Dark matter relic
density in upper bounds from WMAP/Planck Hinshaw  _et al._ (2013); Ade  _et
al._ (2014). (vii) Constraints of direct searches for dark matter by XENON1T
Aprile  _et al._ (2018, 2019) and PICO-60 Amole  _et al._ (2019). (viii)
Constraints from direct SUSY searches at the LHC, imposed with the package
SModelS-v1.2.2 Kraml  _et al._ (2014); Ambrogi  _et al._ (2018, 2020); Dutta
_et al._ (2018); Buckley (2015); Sjostrand  _et al._ (2006); Sjöstrand  _et
al._ (2015), and the LEP.

As shown in Ref. Wang and Zhu (2020a), the surviving samples have following
properties,

  * •

The first-two-generation squarks are heavier than $2\ \mathrm{TeV}$, and even
the lightest squark $\tilde{t}_{1}$ is heavier than about $700\ \mathrm{GeV}$.

  * •

Sleptons in the third generation can be as light as about $100\ \mathrm{GeV}$
Wang and Zhu (2020a).

  * •

The glugino mass $m_{\tilde{g}}>2\ \mathrm{TeV}$, thus due to universal
gaugino mass at the GUT scale, bino and wino mass scale are higher than $350$
and $700\ \mathrm{GeV}$ respectively Wang and Zhu (2020a).

  * •

The higgsino-like charginos and neutralinos can be about $100\sim 200\
\mathrm{GeV}$ Wang and Zhu (2020b); Wang  _et al._ (2021).

  * •

The lightest neutralino is lighter than about $60\ \mathrm{GeV}$, and usually
singlino-dominated, some mixing with higgsinos Wang and Zhu (2020c); Wang  _et
al._ (2021).

  * •

In the Higgs sector, $h_{2}$ is the SM-like Higgs of $123\sim 127\
\mathrm{GeV}$, $h_{1}$ and the light CP-odd Higgs $a_{1}$ is singlet-
dominated, and the heavy Higgs $H$ and $A$ are doublet-dominated with mass
$2.4\sim 4.8\ \mathrm{TeV}$ Wang and Zhu (2020a); Ma  _et al._ (2021).

In this work, for the heavy Higgs $H$ and $A$, we calculate the cross sections
of production and branching ratios of decay, and compare the signal in $pp\to
b\bar{b}H\to b\bar{b}t\bar{t}$ with the simulation result in Ref. Hajer  _et
al._ (2015).

Figure 1:  Surviving samples in the branching ratios versus $m_{H}$ planes,
where colors indicate $\tan\beta$, and the branching ratios are of the CP-even
heavy Higgs $H$ decay to $b\bar{b}$, $\tau^{+}\tau^{-}$, $t\bar{t}$, all
possible lighter Higgs bosons, all possible SUSY particles, and only
$\tilde{\chi}^{\pm}_{1}\tilde{\chi}^{\mp}_{2}$ respectively. Samples with
larger $\tan\beta$ are projected on the smaller ones.  Figure 2:  Same as in
Fig.1, but with colors indicate $|\kappa/\lambda|$, and samples with larger
$|\kappa/\lambda|$ are projected on the smaller ones.  Figure 3:  Same as in
Fig.1, but with the CP-even Higgs $H$ replaced by the CP-odd Higgs $A$.
Samples with larger $\tan\beta$ are projected on the smaller ones.  Figure 4:
Same as in Fig.3, but with colors indicate $|\kappa/\lambda|$, and samples
with larger $|\kappa/\lambda|$ are projected on the smaller ones.  Figure 5:
Surviving samples in $|P_{22}|^{2}$ versus $|S_{33}|^{2}$, $|\kappa/\lambda|$
versus $\lambda$, and $|\kappa/\lambda|$ versus $m_{\chi_{1}^{0}}$ planes
respectively, with colors indicate $|N_{15}|^{2}$. $|P_{22}|^{2}$,
$|S_{33}|^{2}$ and $|N_{15}|^{2}$ indicate the singlet or singlino components
in the heavy Higgs $H$, $A$, and the lightest neutralino
$\tilde{\chi}^{0}_{1}$ respectively.  Figure 6:  Surviving samples in cross
section of $pp\to b\bar{b}H\to b\bar{b}t\bar{t}$ versus the mass of heavy
Higgs planes, where colors of samples indicate $\tan\beta$. The red and green
curves indicate the model-independent exclusion and discovery reaches with an
integrated luminosity of $3\ \mathrm{ab}^{-1}$ at $100\ \mathrm{TeV}$, taken
from Fig.9 in Ref. Hajer  _et al._ (2015).

In Fig. 1 and 2, we show the decay information of the CP-even heavy Higgs $H$.
From these two figures we can see that:

  * •

The dominated branching ratios are always from the decay to SUSY particles,
which are about $0.5\sim 0.8$ in total.

  * •

Since there are two pairs of charginos, five neutralinos, several squarks and
sleptons that can be lighter than the heavy Higgs, we checked that the
branching ratios are so diffuse that the largest branching ratios
$H\to\tilde{\chi}^{\pm}_{1}\tilde{\chi}^{\mp}_{2}$ are at about $0.1\sim 0.2$
level.

  * •

The branching ratios of $H$ to light Higgs bosons are also small, about 0.2 at
most.

  * •

The branching ratios to SM particles are suppressed by the decay to SUSY
particles, that the branching ratios to $b\bar{b}$, $\tau^{+}\tau^{-}$,
$t\bar{t}$ are at most $0.4$, $0.1$, and $0.01$ at most.

  * •

The branching ratios to $b\bar{b}$ and $\tau^{+}\tau^{-}$ are roughly
proportional to $\tan\beta$, while the other ratios are suppressed by large
$\tan\beta$.

  * •

The branching ratios to $t\bar{t}$ and SUSY particles including
$\tilde{\chi}^{\pm}_{1}\tilde{\chi}^{\mp}_{2}$ are also suppressed by
$|\kappa/\lambda|$, while the branching ratio to light Higgs bosons can be
increased.

In Fig. 3 and 4, we show the same information for the CP-odd heavy Higgs boson
$A$, but we can see roughly the same behaviors with these of the CP-even $H$.
So here we omit the discussions.

In Fig. 5 we project the surviving samples in $|P_{22}|^{2}$ versus
$|S_{33}|^{2}$, $|\kappa/\lambda|$ versus $\lambda$, and $|\kappa/\lambda|$
versus $m_{\chi_{1}^{0}}$ planes respectively, with colors indicate
$|N_{15}|^{2}$. $|P_{22}|^{2}$, $|S_{33}|^{2}$ and $|N_{15}|^{2}$ indicate the
singlet or singlino components in the heavy Higgs $H$, $A$, and the lightest
neutralino $\tilde{\chi}^{0}_{1}$ respectively.

We calculate the cross sections of the process $pp\to b\bar{b}H$ in the SM
with $m_{H}$ varying from $1$ to $5\ \mathrm{TeV}$ at $\sqrt{s}=100\
\mathrm{TeV}$ with MG5_aMC_v2.6.7 Alwall  _et al._ (2014); Hirschi and
Mattelaer (2015). We get the cross section corresponding to $m_{H}$ or $m_{A}$
in our samples, timed with squire of the reduced coupling
$C_{Hbb}/C_{Hbb}^{{\rm SM}}$ and the branching ratio $Br(H/A\to t\bar{t})$.
Then we get the production rates of our samples in the $pp\to b\bar{b}H\to
b\bar{b}t\bar{t}$ channel, and show them in Fig. 6, where colors indicate
$\tan\beta$. While the red and green curves indicate the model-independent
exclusion and discovery reaches with an integrated luminosity of $3\
\mathrm{ab}^{-1}$ at $100\ \mathrm{TeV}$, taken from Fig.9 in Ref. Hajer  _et
al._ (2015). In calculating the cross section in the SM, we also did in two
schemes, getting the 4FS and 5FS cross sections Wiesemann  _et al._ (2015),
and matched them with the formula in Ref. Harlander  _et al._ (2011)

$\sigma=\frac{\sigma^{\rm 4FS}+\omega\sigma^{\rm 5FS}}{1+\omega},$ (1)

with $\omega=\ln(m_{H}/m_{b})-2$.

From Fig.6, we can see that the production rate of $pp\to b\bar{b}H\to
b\bar{b}t\bar{t}$ is decreasing with the heavy-Higgs mass. And there are some
differences among samples with the same heavy-Higgs mass and different
$\tan\beta$. In the figure, regions above the red and green curves mean where
the samples can be excluded and discovered with an integrated luminosity of
$3\ \mathrm{ab}^{-1}$ at $100\ \mathrm{TeV}$. While we can see that the
surviving samples are all below the exclusion and discovery curves, thus they
can not be discovered, or excluded if no signal is detected with an integrated
luminosity of $3\ \mathrm{ab}^{-1}$ at $100\ \mathrm{TeV}$. But the model can
be excluded if signal of $pp\to b\bar{b}H\to b\bar{b}t\bar{t}$ was detected.

##  IV Conclusion

In this work, with the surviving samples in our previous work Wang and Zhu
(2020a), we study heavy Higgs bosons in the semi-constrained NMSSM. These two
Higgs bosons can be as heavy as $2.4\sim 4.8\ \mathrm{TeV}$, so we investigate
its discovery potential at a $100\ \mathrm{TeV}$ collider. We study them in
the $pp\to b\bar{b}H\to b\bar{b}t\bar{t}$, calculating their production rates,
and comparing them with the simulation result in Ref. Hajer  _et al._ (2015).
Finally, we found that it is hard to detect the heavy Higgs in this channel at
the $100\ \mathrm{TeV}$ collider with an integrated luminosity of $3\
\mathrm{ab}^{-1}$, because the branching ratio of $H\to t\bar{t}$ are
suppressed by the decay to SUSY particles. We will supplement discussions
about detecting in other channels later.

###### Acknowledgements.

This work was supported by the National Natural Science Foundation of China
(NNSFC) under grant Nos. 11605123, and the Startup Research Foundations of
Henan University.

## References

  * Aad  _et al._ (2012) G. Aad _et al._ (ATLAS), Phys. Lett. B 716, 1 (2012), arXiv:1207.7214 [hep-ex] .
  * Chatrchyan  _et al._ (2012) S. Chatrchyan _et al._ (CMS), Phys. Lett. B 716, 30 (2012), arXiv:1207.7235 [hep-ex] .
  * Aad  _et al._ (2016) G. Aad _et al._ (ATLAS, CMS), JHEP 08, 045 (2016), arXiv:1606.02266 [hep-ex] .
  * Sirunyan  _et al._ (2019) A. M. Sirunyan _et al._ (CMS), Eur. Phys. J. C 79, 421 (2019), arXiv:1809.10733 [hep-ex] .
  * Aad  _et al._ (2020) G. Aad _et al._ (ATLAS), Phys. Rev. D 101, 012002 (2020), arXiv:1909.02845 [hep-ex] .
  * Chatrchyan  _et al._ (2020) S. Chatrchyan _et al._ (CMS), (2020), CMS-PAS-HIG-19-005 .
  * Sopczak (2020) A. Sopczak (ATLAS, CMS), PoS FFK2019, 006 (2020), arXiv:2001.05927 [hep-ex] .
  * Medina and Schmidt (2017) A. D. Medina and M. A. Schmidt, JHEP 08, 095 (2017), arXiv:1706.04994 [hep-ph] .
  * Bechtle  _et al._ (2017) P. Bechtle, H. E. Haber, S. Heinemeyer, O. Stål, T. Stefaniak, G. Weiglein, and L. Zeune, Eur. Phys. J. C 77, 67 (2017), arXiv:1608.00638 [hep-ph] .
  * Barman  _et al._ (2016) R. K. Barman, B. Bhattacherjee, A. Chakraborty, and A. Choudhury, Phys. Rev. D 94, 075013 (2016), arXiv:1607.00676 [hep-ph] .
  * Carena  _et al._ (2016a) M. Carena, J. Ellis, J. S. Lee, A. Pilaftsis, and C. E. M. Wagner, JHEP 02, 123 (2016a), arXiv:1512.00437 [hep-ph] .
  * Djouadi and Quevillon (2013) A. Djouadi and J. Quevillon, JHEP 10, 028 (2013), arXiv:1304.1787 [hep-ph] .
  * Bechtle  _et al._ (2013) P. Bechtle, S. Heinemeyer, O. Stal, T. Stefaniak, G. Weiglein, and L. Zeune, Eur. Phys. J. C 73, 2354 (2013), arXiv:1211.1955 [hep-ph] .
  * Ma  _et al._ (2021) S. Ma, K. Wang, and J. Zhu, Chin. Phys. C 45, 023113 (2021), arXiv:2006.03527 [hep-ph] .
  * Shang  _et al._ (2020) L. Shang, P. Sun, Z. Heng, Y. He, and B. Yang, Eur. Phys. J. C 80, 574 (2020).
  * Cao  _et al._ (2017) J. Cao, X. Guo, Y. He, P. Wu, and Y. Zhang, Phys. Rev. D 95, 116001 (2017), arXiv:1612.08522 [hep-ph] .
  * Wang  _et al._ (2017) F. Wang, W. Wang, L. Wu, J. M. Yang, and M. Zhang, Int. J. Mod. Phys. A 32, 1745005 (2017), arXiv:1512.08434 [hep-ph] .
  * Carena  _et al._ (2016b) M. Carena, H. E. Haber, I. Low, N. R. Shah, and C. E. M. Wagner, Phys. Rev. D 93, 035013 (2016b), arXiv:1510.09137 [hep-ph] .
  * Bomark  _et al._ (2015) N.-E. Bomark, S. Moretti, S. Munir, and L. Roszkowski, JHEP 02, 044 (2015), arXiv:1409.8393 [hep-ph] .
  * Cao  _et al._ (2014a) J. Cao, D. Li, L. Shang, P. Wu, and Y. Zhang, JHEP 12, 026 (2014a), arXiv:1409.8431 [hep-ph] .
  * King  _et al._ (2014) S. F. King, M. Mühlleitner, R. Nevzorov, and K. Walz, Phys. Rev. D 90, 095014 (2014), arXiv:1408.1120 [hep-ph] .
  * Baglio  _et al._ (2014) J. Baglio, R. Gröber, M. Mühlleitner, D. T. Nhung, H. Rzehak, M. Spira, J. Streicher, and K. Walz, Comput. Phys. Commun. 185, 3372 (2014), arXiv:1312.4788 [hep-ph] .
  * Cao  _et al._ (2012a) J.-J. Cao, Z.-X. Heng, J. M. Yang, Y.-M. Zhang, and J.-Y. Zhu, JHEP 03, 086 (2012a), arXiv:1202.5821 [hep-ph] .
  * Cao  _et al._ (2012b) J. Cao, Z. Heng, J. M. Yang, and J. Zhu, JHEP 10, 079 (2012b), arXiv:1207.3698 [hep-ph] .
  * Cao  _et al._ (2013) J. Cao, F. Ding, C. Han, J. M. Yang, and J. Zhu, JHEP 11, 018 (2013), arXiv:1309.4939 [hep-ph] .
  * Cao  _et al._ (2016) J. Cao, L. Shang, W. Su, Y. Zhang, and J. Zhu, Eur. Phys. J. C 76, 239 (2016), arXiv:1601.02570 [hep-ph] .
  * Agarwal  _et al._ (2016) B. Agarwal, J. Isaacson, and K. A. Mohan, Phys. Rev. D 94, 035027 (2016), arXiv:1604.05328 [hep-ph] .
  * Cao  _et al._ (2014b) J. Cao, Y. He, P. Wu, M. Zhang, and J. Zhu, JHEP 01, 150 (2014b), arXiv:1311.6661 [hep-ph] .
  * Liu  _et al._ (2019) L. Liu, H. Qiao, K. Wang, and J. Zhu, Chin. Phys. C 43, 023104 (2019), arXiv:1812.00107 [hep-ph] .
  * Han  _et al._ (2013) X.-F. Han, L. Wang, J. M. Yang, and J. Zhu, Phys. Rev. D 87, 055004 (2013), arXiv:1301.0090 [hep-ph] .
  * Cheung  _et al._ (2018) K. Cheung, S.-P. He, Y.-n. Mao, P.-Y. Tseng, and C. Zhang, Phys. Rev. D 98, 075023 (2018), arXiv:1809.03809 [hep-ph] .
  * Sirunyan  _et al._ (2018) A. M. Sirunyan _et al._ (CMS), JHEP 09, 007 (2018), arXiv:1803.06553 [hep-ex] .
  * Aaboud  _et al._ (2018a) M. Aaboud _et al._ (ATLAS), JHEP 01, 055 (2018a), arXiv:1709.07242 [hep-ex] .
  * Zyla  _et al._ (2020) P. A. Zyla _et al._ (Particle Data Group), PTEP 2020, 083C01 (2020).
  * Aaboud  _et al._ (2018b) M. Aaboud _et al._ (ATLAS), Phys. Rev. D 98, 052008 (2018b), arXiv:1808.02380 [hep-ex] .
  * Crivellin  _et al._ (2021) A. Crivellin, Y. Fang, O. Fischer, A. Kumar, M. Kumar, E. Malwa, B. Mellado, N. Rapheeha, X. Ruan, and Q. Sha, (2021), arXiv:2109.02650 [hep-ph] .
  * Fowlie (2021) A. Fowlie, (2021), arXiv:2109.13426 [hep-ph] .
  * Bicer  _et al._ (2014) M. Bicer _et al._ (TLEP Design Study Working Group), JHEP 01, 164 (2014), arXiv:1308.6176 [hep-ex] .
  * Gao (2021) J. Gao, Int. J. Mod. Phys. A 36, 2142005 (2021).
  * Dong  _et al._ (2018) M. Dong _et al._ (CEPC Study Group), (2018), arXiv:1811.10545 [hep-ex] .
  * Hinchliffe  _et al._ (2015) I. Hinchliffe, A. Kotwal, M. L. Mangano, C. Quigg, and L.-T. Wang, Int. J. Mod. Phys. A 30, 1544002 (2015), arXiv:1504.06108 [hep-ph] .
  * Bartosik  _et al._ (2020) N. Bartosik _et al._ , JINST 15, P05001 (2020), arXiv:2001.04431 [hep-ex] .
  * Alsharoa  _et al._ (2003) M. M. Alsharoa _et al._ (Neutrino Factory, Muon Collider), Phys. Rev. ST Accel. Beams 6, 081001 (2003), arXiv:hep-ex/0207031 .
  * Hajer  _et al._ (2015) J. Hajer, Y.-Y. Li, T. Liu, and J. F. H. Shiu, JHEP 11, 124 (2015), arXiv:1504.07617 [hep-ph] .
  * Baer  _et al._ (2021) H. Baer, C. Kao, V. Barger, R. Jain, D. Sengupta, and X. Tata, (2021), arXiv:2112.02232 [hep-ph] .
  * Han  _et al._ (2021) T. Han, S. Li, S. Su, W. Su, and Y. Wu, Phys. Rev. D 104, 055029 (2021), arXiv:2102.08386 [hep-ph] .
  * Wang and Zhu (2020a) K. Wang and J. Zhu, JHEP 06, 078 (2020a), arXiv:2002.05554 [hep-ph] .
  * Ellwanger  _et al._ (2010) U. Ellwanger, C. Hugonie, and A. M. Teixeira, Phys. Rept. 496, 1 (2010), arXiv:0910.1785 [hep-ph] .
  * Maniatis (2010) M. Maniatis, Int. J. Mod. Phys. A 25, 3505 (2010), arXiv:0906.0777 [hep-ph] .
  * Ellwanger  _et al._ (2005) U. Ellwanger, J. F. Gunion, and C. Hugonie, JHEP 02, 066 (2005), arXiv:hep-ph/0406215 .
  * Ellwanger and Hugonie (2006) U. Ellwanger and C. Hugonie, Comput. Phys. Commun. 175, 290 (2006), arXiv:hep-ph/0508022 .
  * Ellwanger and Hugonie (2007) U. Ellwanger and C. Hugonie, Comput. Phys. Commun. 177, 399 (2007), arXiv:hep-ph/0612134 .
  * Das  _et al._ (2012) D. Das, U. Ellwanger, and A. M. Teixeira, Comput. Phys. Commun. 183, 774 (2012), arXiv:1106.5633 [hep-ph] .
  * Tanabashi  _et al._ (2018) M. Tanabashi _et al._ (Particle Data Group), Phys. Rev. D 98, 030001 (2018).
  * Aaij  _et al._ (2013) R. Aaij _et al._ (LHCb), Phys. Rev. Lett. 110, 021801 (2013), arXiv:1211.2674 [hep-ex] .
  * Lees  _et al._ (2012a) J. P. Lees _et al._ (BaBar), Phys. Rev. Lett. 109, 101802 (2012a), arXiv:1205.5442 [hep-ex] .
  * Lees  _et al._ (2012b) J. P. Lees _et al._ (BaBar), Phys. Rev. Lett. 109, 191801 (2012b), arXiv:1207.2690 [hep-ex] .
  * Bechtle  _et al._ (2015) P. Bechtle, S. Heinemeyer, O. Stal, T. Stefaniak, and G. Weiglein, Eur. Phys. J. C 75, 421 (2015), arXiv:1507.06706 [hep-ph] .
  * Bechtle  _et al._ (2014) P. Bechtle, O. Brein, S. Heinemeyer, O. Stål, T. Stefaniak, G. Weiglein, and K. E. Williams, Eur. Phys. J. C 74, 2693 (2014), arXiv:1311.0055 [hep-ph] .
  * Bechtle  _et al._ (2012) P. Bechtle, O. Brein, S. Heinemeyer, O. Stal, T. Stefaniak, G. Weiglein, and K. Williams, PoS CHARGED2012, 024 (2012), arXiv:1301.2345 [hep-ph] .
  * Bechtle  _et al._ (2011) P. Bechtle, O. Brein, S. Heinemeyer, G. Weiglein, and K. E. Williams, Comput. Phys. Commun. 182, 2605 (2011), arXiv:1102.1898 [hep-ph] .
  * Bechtle  _et al._ (2010) P. Bechtle, O. Brein, S. Heinemeyer, G. Weiglein, and K. E. Williams, Comput. Phys. Commun. 181, 138 (2010), arXiv:0811.4169 [hep-ph] .
  * Abi  _et al._ (2021) B. Abi _et al._ (Muon g-2), Phys. Rev. Lett. 126, 141801 (2021), arXiv:2104.03281 [hep-ex] .
  * Hinshaw  _et al._ (2013) G. Hinshaw _et al._ (WMAP), Astrophys. J. Suppl. 208, 19 (2013), arXiv:1212.5226 [astro-ph.CO] .
  * Ade  _et al._ (2014) P. A. R. Ade _et al._ (Planck), Astron. Astrophys. 571, A16 (2014), arXiv:1303.5076 [astro-ph.CO] .
  * Aprile  _et al._ (2018) E. Aprile _et al._ (XENON), Phys. Rev. Lett. 121, 111302 (2018), arXiv:1805.12562 [astro-ph.CO] .
  * Aprile  _et al._ (2019) E. Aprile _et al._ (XENON), Phys. Rev. Lett. 122, 141301 (2019), arXiv:1902.03234 [astro-ph.CO] .
  * Amole  _et al._ (2019) C. Amole _et al._ (PICO), Phys. Rev. D 100, 022001 (2019), arXiv:1902.04031 [astro-ph.CO] .
  * Kraml  _et al._ (2014) S. Kraml, S. Kulkarni, U. Laa, A. Lessa, W. Magerl, D. Proschofsky-Spindler, and W. Waltenberger, Eur. Phys. J. C 74, 2868 (2014), arXiv:1312.4175 [hep-ph] .
  * Ambrogi  _et al._ (2018) F. Ambrogi, S. Kraml, S. Kulkarni, U. Laa, A. Lessa, V. Magerl, J. Sonneveld, M. Traub, and W. Waltenberger, Comput. Phys. Commun. 227, 72 (2018), arXiv:1701.06586 [hep-ph] .
  * Ambrogi  _et al._ (2020) F. Ambrogi _et al._ , Comput. Phys. Commun. 251, 106848 (2020), arXiv:1811.10624 [hep-ph] .
  * Dutta  _et al._ (2018) J. Dutta, S. Kraml, A. Lessa, and W. Waltenberger, LHEP 1, 5 (2018), arXiv:1803.02204 [hep-ph] .
  * Buckley (2015) A. Buckley, Eur. Phys. J. C 75, 467 (2015), arXiv:1305.4194 [hep-ph] .
  * Sjostrand  _et al._ (2006) T. Sjostrand, S. Mrenna, and P. Z. Skands, JHEP 05, 026 (2006), arXiv:hep-ph/0603175 .
  * Sjöstrand  _et al._ (2015) T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen, and P. Z. Skands, Comput. Phys. Commun. 191, 159 (2015), arXiv:1410.3012 [hep-ph] .
  * Wang and Zhu (2020b) K. Wang and J. Zhu, Chin. Phys. C 44, 061001 (2020b), arXiv:1911.08319 [hep-ph] .
  * Wang  _et al._ (2021) K. Wang, J. Zhu, and Q. Jie, Chin. Phys. C 45, 041003 (2021), arXiv:2011.12848 [hep-ph] .
  * Wang and Zhu (2020c) K. Wang and J. Zhu, Phys. Rev. D 101, 095028 (2020c), arXiv:2003.01662 [hep-ph] .
  * Alwall  _et al._ (2014) J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli, and M. Zaro, JHEP 07, 079 (2014), arXiv:1405.0301 [hep-ph] .
  * Hirschi and Mattelaer (2015) V. Hirschi and O. Mattelaer, JHEP 10, 146 (2015), arXiv:1507.00020 [hep-ph] .
  * Wiesemann  _et al._ (2015) M. Wiesemann, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, and P. Torrielli, JHEP 02, 132 (2015), arXiv:1409.5301 [hep-ph] .
  * Harlander  _et al._ (2011) R. Harlander, M. Kramer, and M. Schumacher, (2011), arXiv:1112.3478 [hep-ph] .