# Improved determination of strange distribution function from the global
analysis using BHPS model

Maral Salajegheha ID<EMAIL_ADDRESS>S. Mohammad Moosavi Nejada,b
ID<EMAIL_ADDRESS>S. Atashbar Tehranib ID<EMAIL_ADDRESS>aPhysics
Department, Yazd University, P.O.Box 89195-741, Yazd, Iran
bSchool of Particles and Accelerators, Institute for Research in Fundamental
Sciences (IPM), P.O.Box 19395-5531, Tehran, Iran

(August 27, 2024)

###### Abstract

We study the impact of intrinsic strange (IS) component of nucleon sea on the
global analysis of parton distribution functions (PDFs) considering a wide
range of experimental data. To this aim, we consider two scenarios on the
basis of BHPS model results for the IS distribution. In the first scenario, we
apply the results presented through the BHPS model and in the second scenario
we use its evolved distributions. For each scenarios, we present the limit of
the IS probability ${\cal P}_{5}^{s\bar{s}}$ for the standard tolerance
criteria $\Delta\chi^{2}=1$ and $18.112$ at $1\sigma$ and $4\sigma$ levels.
Our results show that the experimental data can tolerate an IS component with
a greater probability ${\cal P}_{5}^{s\bar{s}}$ if one employs the second
scenario. We obtain ${\cal P}_{5}^{s\bar{s}}\approx 0.01$ and ${\cal
P}_{5}^{s\bar{s}}\approx 0.025$ for $\Delta\chi^{2}=1$ and $18.112$,
respectively, at the $4\sigma$ level. We also calculate the ratio of strange-
to-light sea-quark densities $r_{s}$ in the proton both including and
excluding the IS component. Our results show that one can obtain a higher
value for the ratio $r_{s}$ if the IS component is included.

###### pacs:

14.65.Bt, 12.38.Bx, 12.38.Lg, 13.60.Hb

##  I Introduction

It is well-known that the global analysis of experimental data is the best and
most reliable way to extract the nucleon parton distribution functions (PDFs)
Harland-Lang:2014zoa ; Jimenez-Delgado:2014twa ; Abramowicz:2015mha ;
Dulat:2015mca ; Alekhin:2017kpj ; MoosaviNejad:2016ebo  as well as the
polarized PDFs Jimenez-Delgado:2014xza ; Sato:2016tuz ; Shahri:2016uzl ;
Khanpour:2017cha , the nuclear modifications of PDFs deFlorian:2011fp ;
Kovarik:2015cma ; Eskola:2016oht ; Klasen:2017kwb , and the fragmentation
functions of partons Nejad:2015fdh ; MoosaviNejad:2020svj ; Delpasand:2020mqv
; Nejad:2016epx ; Delpasand:2019xpk ; MoosaviNejad:2020vqc ;
MoosaviNejad:2017rvi ; Soleymaninia:2017xhc ; MoosaviNejad:2017bda ;
Nejad:2015far ; MoosaviNejad:2016scq . Actually, these quantities correspond
to the nonperturbative aspects of nature and experimental data will remain, at
least for the next few years, the main source of information on them.
Although, in some theoretical works on basis of lattice QCD calculations and
continuum QCD studies these quantities have been computed theoretically with
large uncertainties, see for example Refs. Lin:2020rut ; Ding:2019qlr .
In the case of PDFs, much progress has been achieved in past few decades.
However, there are still some different issues which must be studied to make
more precise predictions for hadronic collisions, specifically, at the CERN
Large Hadron Collider (LHC) and the Relativistic Heavy Ion Collider (RHIC).
One of the important issues is the accurate determination of the sea quark
distributions which we intend to study in this work. This would be possible
using the xFitter program Alekhin:2014irh  and its facilities. It is now
possible to study the PDFs in a faster way in some global analyses including a
wide range of experimental data at an arbitrary perturbative approximation up
to next-to-next-to-leading order (NNLO) (see for example Refs Bertone:2016ywq
; Giuli:2017oii ).

The sea quark distributions extracted from the global analysis of PDFs are
usually referred to as extrinsic sea quarks in the scientific literature. In
fact, these kinds of sea quarks, specifically the heavy quark (c,b) sea, arise
from the perturbative QCD mechanism and the splitting of gluons into the
quark-antiquark pairs in the Dokshitzer-Gribove-Lipatov-Alterelli-Parisi
(DGLAP) evolution equations Altarelli:1977zs ; Gribov:1972ri . Although, it is
now well-established that other kinds of sea quarks can exist inside the
nucleon which are known as the intrinsic ones. Historically, the existence of
these intrinsic sea quarks in the proton was first suggested by Brodsky,
Hoyer, Peterson, and Sakai (BHPS) in 1980 Brodsky:1980pb ; Brodsky:1981se . In
comparison with the extrinsic sea quarks, the intrinsic ones arise from the
nonperturbative mechanism in origin, and their contributions to the nucleon
wave function become important and considerable at relatively large values of
the Bjorken scaling variable $x$.
The first phenomenological attempt to study the impact of an intrinsic quark
(specifically, the charm) component of the nucleon sea on the global analysis
of PDFs was made by Harris et al. Harris:1995jx , so their analysis suggests a
probability of $0.86\pm 0.6$% for the intrinsic charm (IC). Their result was
smaller than the BHPS theoretical estimation Brodsky:1980pb ; Brodsky:1981se
which was obtained (about) $1$% using the diffractive production of
$\Lambda_{c}$. The first comprehensive global analysis of PDFs including the
IC component was made by the CTEQ Collaboration Pumplin:2007wg ;
Nadolsky:2008zw  using a wide range of the hard scattering data. They obtained
a limit up to three times larger for the IC probability in comparison with the
BHPS estimation. The studies of CTEQ Collaboration were followed in Ref.
Dulat:2013hea  where the authors reported a larger value for the IC
probability. In the following, Jimenez-Delgado et al.Jimenez-Delgado:2014zga
presented a new global analysis of PDFs considering the IC component. Their
conclusion was that the IC probability can not be greater than $0.5\%$,
although it was criticized by Brodsky and Gardner Brodsky:2015uwa . Very
recently, the authors of Ref. Hou:2017khm  have studied the possibility of a
(sizable) nonperturbative contribution to the charm PDF in the context of the
CTEQ-TEA global QCD analysis. They found that the momentum fraction of charm
quark must be less than (about) 2% for the BHPS IC model. Consequently, there
is still no agreement concerning the allowed amounts of IC probability in the
nucleon sea.

In spite of the global analysis of PDFs for studying the impact of nucleon
intrinsic charm component, there are no similar global analysis for the
intrinsic light quarks. Although, despite some studies on the distributions of
the intrinsic light quarks in the nucleon and their related probabilities
Chang:2011vx ; Chang:2011du ; Chang:2014lea ; Salajegheh:2015xoa ; An:2017flb
, a simultaneous study of the intrinsic and extrinsic light quarks has not
been performed yet in any global analysis of a wide range of the experimental
data. Among the sea quark distributions of the nucleon, the strange one has an
important role in better understanding the nucleon structure. It would be also
useful for describing some DIS processes at the LHC. Moreover, the study of
$s-\bar{s}$ asymmetry has been always very interesting Salajegheh:2015xoa ;
Vega:2015hti  because both experimental and theoretical analysis indicate its
existence in the nucleon sea. Therefore, apart from the IC component, it seems
that a simultaneous study of the intrinsic strange (IS) component and the
extrinsic one is motivated in a global analysis of PDFs and would be of
particular interest for researchers.
In this work, we will study this simultaneous analysis by virtue of the
xFitter program Alekhin:2014irh  and by considering a wide range of the
experimental data. For this purpose, considering two scenarios we use the
result of the BHPS model for the IS distribution in the proton. In the first
scenario, we apply the results presented through the BHPS model, and in the
second one we use its evolved distribution. For each scenarios, we present the
limit of the IS probability ${\cal P}_{5}^{s\bar{s}}$ for the standard
tolerance criteria $\Delta\chi^{2}=1$ and $18.112$, and also at $1\sigma$ and
$4\sigma$ level.

The contents of the present paper are as follows. In Sec. II, we shall first
describe the BHPS model briefly, and then introduce its result for the IS
distribution in the proton. We will also discuss the evolution of the IS
distribution using the non-singlet DGLAP equation and present a simple
functional form for the evolved BHPS IS distribution. In Sec. III, we present
a detailed explanation of the QCD analysis such as the treatment of heavy-
flavor contributions, PDF parameterization and the procedure for determining
the limit of the IS probability. In Sec. IV we explain our fitting procedure
by describing the essential concepts of the Hessian or error matrix approach
and Sec. V contains the global analyses of PDFs. We also determine the limit
of the IS probability in this section. In Sec. VI, we compare the extrinsic
strange distribution in the proton obtained from the global QCD analysis with
the results of the total strange distribution obtained via considering an IS
component. We also calculate the ratio of strange-to-light sea-quark densities
in the proton. Finally, our results and conclusions are summarized in Sec.
VII.

##  II Intrinsic strange distribution

Unlike the extrinsic sea quark distributions that should be determined
phenomenologically in a global analysis of experimental data, the intrinsic
ones can be calculated theoretically in a light-cone picture of the nucleon at
fixed light-front time. Actually, these components can arise from the
nonperturbative fluctuations of the nucleon state to the five-quark states
$|uudq\bar{q}\rangle$ where $q=u,d,s$ (and also heavy quarks $c$ and $b$), or
the virtual meson-baryon states in the mesonic cloud model (MCM) framework
Thomas:1983fh ; Signal:1987gz . One can find a review of these models in Refs.
Salajegheh:2015xoa ; Pumplin:2005yf ; Hobbs:2013bia . It is worth noting in
this context that, in Ref. Pumplin:2005yf  authors have presented a model in
which the light-cone probability distributions are derived directly from
Feynman diagrams. Although, there are some clear evidences for the existence
of the intrinsic quarks in proton Brodsky:2015fna , but their experimental
presence has not been yet confirmed definitively. Then, the only remaining
task is the determination of their presence probabilities in the nucleon.

According to the BHPS model, if one neglects the effect of the transverse
momentum in the five-quark transition amplitudes, the probability
distributions of intrinsic quarks $q,\bar{q}$ in the five-quark Fock state
$|uudq\bar{q}\rangle$ are given by

$\displaystyle P(x_{1},\dots,x_{5})={\cal
N}\delta\left(1-\sum_{i=1}^{5}x_{i}\right)\left[M^{2}-\sum_{i=1}^{5}\dfrac{m_{i}^{2}}{x_{i}}\right]^{-2},$
(1)

where $m_{i}$ and $x_{i}$ are the mass and the momentum fraction carried by
the quark $i$, respectively. In Eq. (1), $M$ is the mass of parent hadron,
proton in our case, and ${\cal N}$ is the normalization constant which is
determined by the following relation

${\cal P}_{5}^{q\bar{q}}=\int_{0}^{1}dx_{1}\dots dx_{5}P(x_{1},\dots,x_{5}),$
(2)

where ${\cal P}_{5}^{q\bar{q}}$ is the $|uudq\bar{q}\rangle$-state probability
within the proton. For the case of heavy quarks, it is expected to be roughly
proportional to the quark mass as $1/m_{q}^{2}$. The momentum distribution of
the intrinsic quark $\bar{q}$ is obtained by integrating Eq. (1) over
$x_{1},\cdots,x_{4}$.
For heavy quarks, BHPS assumed that the masses of nucleon and light quarks are
negligible in comparison with the heavy ones and obtained the $x$-distribution
of the IC component in proton, analytically. Although, for the case of light
quarks one can not consider such simplifying assumption and then the intrinsic
light distributions must be calculated numerically, see Refs. Chang:2011vx ;
Salajegheh:2015xoa . In Ref. Salajegheh:2015xoa , the IS distribution at the
initial scale $Q_{0}=0.5$ GeV is determined from the BHPS model and it is
fitted to a simple functional form as

$\displaystyle xs(x)_{int}=226.531\,x^{2.449}(1-x)^{8.433}.$ (3)

In this way, the momentum fraction carried by the IS component in five-quark
state $|uuds\bar{s}\rangle$ with $s=\bar{s}$, i.e.

$\displaystyle\langle x\rangle_{\textrm{IS}}=\int_{0}^{1}2xs(x)_{int}\,dx,$
(4)

is equal to $\langle x\rangle_{\textrm{IS}}=0.41$ which is smaller than the
one carried by the IC in five-quark state $|uudc\bar{c}\rangle$ ($\langle
x\rangle_{\textrm{IC}}=0.57$ Brodsky:1980pb ) at the same energy scale
$Q_{0}=0.5$ GeV. Note that, by neglecting the probability of the
$|uudq\bar{q}\rangle$-state in the proton, the quark number condition
$\int_{0}^{1}xf(x)dx=1$, ($f=q,\bar{q}$) should be always satisfied.

One of the main issues concerning the intrinsic quark distributions is their
evolution to higher energy scales $Q^{2}$. In Ref. Lyonnet:2015dca , it is
shown that the $Q^{2}$-evolution of the intrinsic quark distributions can be
well-controlled by non-singlet evolution equations. An interesting advantage
of this technique is to evolve the intrinsic quark distributions independent
of the gluon and other PDFs. On the other hand, in Refs. Chang:2011vx ;
Chang:2011du ; Chang:2014lea  it is shown that if the intrinsic light quark
distributions are evolved from the initial scales $Q_{0}=0.3$ or $0.5$ GeV, a
more convenient fit can be achieved. Therefore, using the non-singlet
evolution equations to evolve Eq. (3) to the higher scale $Q=1$ GeV, one gets

$\displaystyle xs(x)_{int}=21.5296\,x^{1.3874}(1-x)^{7.4453}.$ (5)

Note that, the momentum fraction carried by this evolved IS distribution is
$\langle x\rangle_{\textrm{IS}}=0.27$ which is smaller than the one obtained
at the scale $Q_{0}=0.5$ GeV, i.e. $\langle x\rangle_{\textrm{IS}}=0.41$.
Fig. 1 shows the behavior of IS distribution given by Eqs. (3) and (5). As is
seen, the peak position of evolved IS distribution (dashed line) is shifted
towards lower values of $x$ and its amount decreases in the peak region, by as
much as $32\%$. Then, it seems that if one uses these different IS
distributions in the analysis, various results can be obtained for any
physical quantity sensitive to the strange content of the proton.

Figure 1: Comparison between the BHPS result (3) for the IS distribution
(solid line) and its evolved distribution (dashed line) at the scale $Q=1$ GeV
(5).

Now, having the IS distribution from the BHPS model, it is important to
estimate its probability (${\cal P}_{5}^{s\bar{s}}$) in the proton. It should
be noted that, this quantity has been already calculated in Refs. Chang:2011vx
; Chang:2011du ; Chang:2014lea  and An:2017flb  so that many different results
have been reported for its value. In Ref. Chang:2011vx , Chang and Peng
assumed that the probability of intrinsic light sea quarks (with the mass
$m_{q}$) is proportional to $1/m_{q}^{2}$, similar to the heavy quark state.
Then, they concluded ${\cal P}_{5}^{s\bar{s}}/[\frac{1}{2}({\cal
P}_{5}^{u\bar{u}}+{\cal
P}_{5}^{d\bar{d}})]=m_{\bar{u}}^{2}/m_{\bar{s}}^{2}\approx 0.36$ with
$m_{\bar{u}}=0.3$ GeV$/c^{2}$ and $m_{\bar{s}}=0.5$ GeV$/c^{2}$. By comparing
the results based on the BHPS model and the data for $\bar{d}-\bar{u}$ and
$\bar{u}+\bar{d}-s-\bar{s}$, they have also extracted the probabilities for
the $|uudu\bar{u}\rangle$\- and $|uudd\bar{d}\rangle$-states as ${\cal
P}_{5}^{u\bar{u}}=0.176$ and ${\cal P}_{5}^{d\bar{d}}=0.294$, respectively.
This implies the value ${\cal P}_{5}^{s\bar{s}}=0.08478$ for the IS
probability in the proton. Their studies are followed in Ref. Chang:2011du
where the ${\cal P}_{5}^{s\bar{s}}$ is determined using the $x(s+\bar{s})$
data from the HERMES Collaboration measurement of charged kaon production in a
SIDIS reaction Airapetian:2008qf . They obtained ${\cal
P}_{5}^{s\bar{s}}=0.024$ through the analysis with the initial scale
$Q_{0}=0.5$ GeV, and ${\cal P}_{5}^{s\bar{s}}=0.029$ for the case of
$Q_{0}=0.3$ GeV. In a newer version of their studies Chang:2014lea  they
extracted the ${\cal P}_{5}^{s\bar{s}}$ in different scenarios which led to
the different values ranging from $0.0$ to $0.111$. Recently, An and Saghai
An:2017flb  have analyzed the ratios for light quark-antiquark pairs (i.e.
$u\bar{u}/d\bar{d}$ and $s\bar{s}/d\bar{d}$) and found ${\cal
P}_{5}^{s\bar{s}}=0.057$. Their analysis was based on the ratios measured from
pion and kaon electroproduction cross sections reported by the CLAS
Collaboration Park:2014zra . In spite of the analyses mentioned above for
determination of the probability of IS distribution in the proton, a
simultaneous study of the intrinsic strange component and the extrinsic sea
quarks in a global analysis of a wide range of the experimental data has not
been done, yet. One advantage of this study is to obtain the new limits on the
intrinsic strange probability in the nucleon, consistent with the large number
of the experimental data.

##  III Technical detail of QCD analysis

As was already mentioned, in our study we use the open source QCD fit
framework for PDF determination xFitter Alekhin:2014irh . Meanwhile, the
QCDNUM evolution code Botje:2010ay  is used for the evolution of parton
densities. To consider the contributions of heavy flavors, we apply the
Thorne-Roberts scheme Thorne:1997ga . We also set the charm and bottom quark
masses as $m_{c}=1.43$ GeV and $m_{b}=4.5$ GeV. The analysis is done at the
NLO QCD approximation with the strong coupling constant evaluated at NLO in
the $\overline{MS}$ scheme with $n_{f}=5$ active flavors adjusted such that
$\alpha_{s}^{(5)}(M_{Z})=0.118$ for $M_{Z}=91.1876$ GeV.

In our analysis, in addition to the gluon and valence distributions, i.e.
$xg(x)$, $xu_{v}(x)$ and $xd_{v}(x)$, we shall also parametrize the anti-quark
distributions $x\bar{u}$, $x\bar{d}$ and $x\bar{s}$, separately. This is due
to the fact that we use a wide range of the experimental data, including those
obtained from the $W$ and $Z$ bosons production in hadronic collisions. We
shall also assume that the sea quark and anti-quark distributions are the
same. The PDFs are parametrized at the initial scale $Q_{0}^{2}=1$ GeV2 and
then evolved to the energy scale of experimental data. According to the
factorization theorem of the QCD-improved parton model Collins:1989gx , by the
convolution of these evolved PDFs with the hard-scattering coefficients one
can obtain the theoretical cross sections.

The optimal functional form of the PFDs parameterization is found by the
saturation technique of $\chi^{2}$ Adloff:2000qk ; Aaron:2009aa . Following
this approach, the parameterizations used to describe the quark and gluon PDFs
at the initial scale $Q_{0}^{2}=1$ GeV2, read

$\displaystyle xg(x)$ $\displaystyle=$ $\displaystyle
A_{g}x^{{B_{g}}}(1-x)^{C_{g}}-A^{\prime}_{g}x^{{B^{\prime}_{g}}}(1-x)^{C^{\prime}_{g}},$
$\displaystyle xu_{\rm v}(x)$ $\displaystyle=$ $\displaystyle A_{u_{\rm
v}}x^{B_{u_{\rm v}}}(1-x)^{C_{u_{\rm v}}}(1+E_{u_{\rm v}}x^{2}),$
$\displaystyle xd_{\rm v}(x)$ $\displaystyle=$ $\displaystyle A_{d_{\rm
v}}x^{B_{d_{\rm v}}}(1-x)^{C_{d_{\rm v}}},$ $\displaystyle x\bar{u}(x)$
$\displaystyle=$ $\displaystyle
A_{\bar{u}}x^{B_{\bar{u}}}(1-x)^{C_{\bar{u}}},$ $\displaystyle x\bar{d}(x)$
$\displaystyle=$ $\displaystyle
A_{\bar{d}}x^{B_{\bar{d}}}(1-x)^{C_{\bar{d}}},$ $\displaystyle
x\bar{s}_{ext}(x)$ $\displaystyle=$ $\displaystyle
A_{\bar{s}}x^{B_{\bar{s}}}(1-x)^{C_{\bar{s}}}.$ (6)

where ext refers to the extrinsic strange sea distribution.

In Eq. (III), as usual, the parameter $A_{g}$ is fixed by the constraint
induced by the momentum sum rule. The parameters $A_{u_{\rm v}}$ and
$A_{d_{\rm v}}$ are determined by the virtue of the quark number sum rules for
the up and down valence quarks. For the gluon parameterization, we also make
an additional constraint $C^{\prime}_{g}=25$ as suggested in Ref.
Abramowicz:2015mha . Moreover, to ensure the same behavior of $x\bar{u}$ and
$x\bar{d}$ when $x\rightarrow 0$, we take some additional constraints as:
$A_{\bar{u}}=A_{\bar{d}}$ and $B_{\bar{u}}=B_{\bar{d}}$. For the strange
distribution, we also assume $A_{\bar{s}}=f_{s}A_{\bar{d}}/(1-f_{s})$ in which
$f_{s}$ appears as a free parameter. After imposing these constraints, the
global fit includes $16$ free parameters.
It should be also noted that, to avoid the nonperturbative effects, all DIS
data included in our global analysis should satisfy two conditions as
$Q^{2}\geq 3.5$ GeV2 and $W^{2}\geq 3.5$ GeV2, and all jet production data in
hadronic collisions should satisfy $p_{T}\geq 20$ GeV, as well. Then, all
remaining PDF parameters are determined by xFitter in minimizing the
$\chi^{2}$-function using the MINUIT program James:1975dr . To estimate the
PDF uncertainties the experimental data uncertainties are propagated to the
extracted QCD fit parameters using the asymmetric Hessian method
Pumplin:2001ct  that is applicable in the xFitter framework. The
$\chi^{2}$-function along with the asymmetric Hessian method are introduced in
Section IV.

With the above explanations, the total strange sea distribution in the proton
is written as

$\displaystyle xs_{tot}(x)=xs_{fit}(x)+{\cal P}_{5}^{s\bar{s}}xs_{BHPS}(x),$
(7)

where, the factor ${\cal P}_{5}^{s\bar{s}}$ describes the probability of
finding an IS component in the proton. The term $xs_{fit}(x)$ is determined in
global analysis of the experimental data and for the term $xs_{BHPS}$, one can
apply the functional form obtained from the BHPS model (3) or its evolved
distribution (5).

Now, we summarize our procedure for the analysis. At first, we perform a
global analysis of PDFs in the framework described in this section without
considering any intrinsic component for the strange distribution in the
proton. In the following, we repeat our analysis considering the intrinsic
strange distribution, as a certain function multiplied by the factor ${\cal
P}_{5}^{s\bar{s}}$. Then, to construct the total strange distribution (7) we
add its contribution to the fitted distribution at the initial scale
$Q_{0}^{2}=1$ GeV2. The analysis is repeated several times by varying the
value of the ${\cal P}_{5}^{s\bar{s}}$. Note that, during each analysis the
value of ${\cal P}_{5}^{s\bar{s}}$ is fixed. In this way, for each data set
(and also for all data) we can compute the deviation of $\chi^{2}$-values from
their central ones, corresponding to the analysis performed without
considering an IS component, see section IV. Then, we can investigate the
consistency of each data set with the IS component and obtain the new limits
on the IS probability in the nucleon consistent with the large number of
experimental data. For the ${\cal P}_{5}^{s\bar{s}}$, we choose a value in the
range $0.001-0.03$ in our analysis. It should be also noted that for the
analysis including an IS distribution, the momentum sum rule of PDFs at the
scale $Q_{0}^{2}$ must be modified as

$\displaystyle\int_{0}^{1}dx\,x\\{g(x)+u_{v}(x)+d_{v}(x)+2[\bar{u}(x)$
$\displaystyle+\bar{d}(x)+\bar{s}_{ext}(x)+{\cal
P}_{5}^{s\bar{s}}\bar{s}_{int}(x)]\\}=1.$ (8)

##  IV  $\chi^{2}$-minimization approach

Here we, briefly, explain our fitting procedure by describing the essential
concepts of the Hessian or error matrix approach which is a usual minimization
method. More details can be found in Refs. Pumplin:2001ct ; Pumplin:2002vw ;
Martin:2002aw ; Martin:2009iq ; Salajegheh:2018hfs ; Soleymaninia:2013cxa .
According to the fit procedure which will be presented in Sec. III, we have a
set of appropriate parameters for PDFs, i.e., $\\{p_{i}\\}(i=1,2,\cdots,n)$,
where $n$ refers to the total number of the fitted parameters. The basic
assumption of the Hessian approach is a quadratic expansion of the
$\chi^{2}_{global}$ around the minimum $\chi^{2}$ point as

$\displaystyle\Delta\chi^{2}_{global}(p)$ $\displaystyle=$
$\displaystyle\chi^{2}_{global}-\chi_{min}^{2}$ (9)
$\displaystyle=\sum_{i,j}H_{ij}(p_{i}-p_{i}^{0})(p_{j}-p_{j}^{0}),$

where, the element of the Hessian matrix $H_{ij}$ is defined as

$\displaystyle
H_{ij}=\left.\frac{1}{2}\frac{\partial^{2}\chi^{2}_{global}}{\partial
p_{i}\partial p_{j}}\right|_{min}.$ (10)

Since, the Hessian matrix and its inverse $C\equiv H^{-1}$, which is the error
matrix, are symmetric they have a set of n orthogonal eigenvectors $v_{ik}$
with eigenvalues $\lambda_{k}$:

$\displaystyle\sum_{j=1}^{n}C_{ij}v_{jk}=\lambda_{k}v_{ik},$
$\displaystyle\sum_{i=1}^{n}v_{ij}v_{ik}=\delta_{jk}.$ (11)

The parameter variation around the global minimum can be expanded in a basis
of rescaled eigenvectors $e_{ik}=\sqrt{\lambda_{k}}v_{ik}$ as

$\displaystyle p_{i}-p_{i}^{0}=\sum_{k=1}^{n}e_{ik}z_{k}.$ (12)

Replacing Eq. (12) in Eq. (9) and considering the orthogonality of $v_{k}$ we
achieve

$\displaystyle\Delta\chi^{2}_{global}=\sum_{k=1}^{n}z_{k}^{2},$ (13)

where $\sum_{k=1}^{n}z_{k}^{2}\leq T^{2}$ is the interior of a sphere of
radius T. The neighborhood parameters are given by

$\displaystyle p_{i}(s_{k}^{\pm})=p_{i}^{0}\pm te_{ik},$ (14)

where $s_{k}$ is the $k$’th set of PDFs and $t$ is adapted to make the desired
$T=(\Delta\chi^{2}_{global})^{\frac{1}{2}}$ and $t=T$ in the quadratic
approximation. The PDF uncertainties are estimated using the Hessian matrix as
follows:

$\displaystyle[\delta f(x)]^{2}$ $\displaystyle=$
$\displaystyle\Delta\chi^{2}\sum_{i,j=1}^{n}\frac{\partial
f(x,p_{i})}{\partial p_{i}}H_{ij}^{-1}\frac{\partial f(x,p_{j})}{\partial
p_{j}},$ (15)

where $H_{ij}$ is the Hessian matrix and $n$ is the number of parameters in
the global fit. The $\Delta\chi^{2}$ value determines the confidence region
and is given by

$\displaystyle\Delta\chi^{2}(p)=\sum_{i,j}H_{ij}\delta p_{i}\delta p_{j}.$
(16)

The $\Delta\chi^{2}$ value is calculated considering the confidence level $P$
which is defined as

$\displaystyle
P=\int_{0}^{\Delta\chi^{2}}\frac{1}{2\Gamma\left(\frac{N}{2}\right)}\left(\frac{s}{2}\right)^{\frac{N}{2}-1}exp\left(-\frac{s}{2}\right)ds.$
(17)

The numerical value of Eq. (17) which corresponds to 1$\sigma$ error is
$P=0.6826$. This numerical value is related to a given number of parameters
($N$) by assuming the normal distribution in the multiparameter space. In an
analysis with 16 parameters, we achieve $\Delta\chi^{2}=18.112$. The Hessian
method can also be used to estimate the polarized PDFs and fragmentation
functions Pumplin:2001ct ; Pumplin:2002vw ; Martin:2002aw ; Martin:2009iq ;
Salajegheh:2019srg ; Salajegheh:2019nea ; Salajegheh:2019ach ;
MoosaviNejad:2018ukp .

##  V fit results

Defining our framework, we are now in a situation to perform our global
analysis of PDFs considering an IS component in the proton. One of the main
tasks in this regard is an appropriate selection of the experimental data. As
is well-known, in any global analysis of PDFs, the main constraints on parton
densities come from the DIS data which include a wide range of the $x$\- and
$Q^{2}$\- values. Nevertheless, the DIS data can not determine the gluon
distribution accurately, and can not also specify the sea quark distributions,
specifically as the separate flavors. In fact, for the case of sea quarks the
DIS data can just be used to determine the sum of their distributions in some
kinematical regions. Therefore, to impose further constraints on the gluon and
sea quark distributions, various phenomenological groups employ different
types of experimental data such as those from the one-jet inclusive
productions and also the various measurements of the $W$ and $Z$ boson
productions. Nowadays, with the progresses achieved for the theoretical
calculations and having advanced computational tools, many other processes can
be also included in the analysis to get more information over the parton
distributions. For example, for the gluon distribution the inclusive isolated
prompt photon production can be used in hadronic and nuclear collisions.

In the present work, for the fixed-target DIS data we use the proton structure
functions $F_{2}^{p}$ determined by BCDMS in scattering of muons over hydrogen
targets Benvenuti:1989rh . For DIS data sets, we include the latest high-
precision neutral current (NC) and charge current (CC) HERA I+II combined data
Abramowicz:2015mha  and also the low-$x$ and $Q^{2}$ measurements from H1
obtained through the inclusive NC $e^{\pm}p$ scattering Collaboration:2010ry .
These DIS data can constrain the gluon and sea quark distributions in all
range of $x$, satisfactorily. In addition, we use the H1 and ZEUS measurements
of the bottom structure function $F_{2}^{b}$ Aaron:2009af  and reduced cross
section $\sigma_{r}^{b}$ Abramowicz:2014zub , and also the HERA combined
measurements of the charm production cross section Abramowicz:1900rp  that can
be useful to constrain the bottom and charm contents of proton at low values
of $x$. The inclusion of DIS data is completed by considering the one-jet
inclusive cross sections from the H1 Aaron:2009vs ; Aktas:2007aa ;
Aaron:2010ac  and ZEUS Chekanov:2002be ; Chekanov:2006xr , and also the dijet
cross sections from the ZEUS measurements Abramowicz:2010cka . For the one-jet
inclusive cross sections in hadronic collisions, we use the ATLAS data at a
center-of-mass energies of $7$ TeV and $2.76$ TeV Aad:2011fc ; Aad:2013lpa
and also the CMS measurements at $7$ TeV Chatrchyan:2012bja . It is now well-
established Rojo:2014kta  that the jet production cross sections reported by
the LHC experiments provide a practical tool to achieve the precious
information on the gluon and quark PDFs at large values of $x$. In the case of
Drell-Yan differential cross sections in $pp$ collisions, both high
Aad:2013iua  and low Aad:2014qja  mass ATLAS measurements at $\sqrt{s}=7$ TeV
are included in our analysis. Finally for the electroweak production of bosons
at the Tevatron, we use the CDF data Aaltonen:2010zza ; Aaltonen:2009ta  while
for the same measurements at the LHC we use the inclusive $W^{\pm}$ and $Z$
rapidity distributions at $7$ TeV from both ATLAS 2010 Aad:2011dm  and 2011
Aaboud:2016btc  data sets. The advantage of these data is to provide more
sensitivity to the flavor composition of the sea quark. The same benefit is
applicable for the valence-quark distributions at lower-$x$ so that, for
instance, the determination of the strange content of the proton would be
possible.

After introducing the experimental data used in our analysis, we present our
results obtained through the global fit. At the first step, we execute a
global analysis without including an IS component in the proton. All
experimental data along with the $\chi^{2}$-values are listed in Table 1. We
have also presented the total $\chi^{2}$ divided by the number of freedom
degrees. i.e. $\chi^{2}/d.o.f=1.178$. Note that, the only data which have a
rather large $\chi^{2}$-value are related to the $Z$ rapidity distributions
for the central channel (CC) from the ATLAS 2011 data set. The optimal values
of the PDF parameters (III) at the initial scale $Q_{0}^{2}=1$ GeV2 are listed
in Table 2, so the uncertainties are estimated by the MINUIT program
James:1975dr .

Table 1:  Experimental data included in our analysis (see the text for a detailed explanation). The values of $\chi^{2}$ and the number of data points $N_{\textrm{pts}}$ are presented for each data set. Data set | $\chi^{2}/N_{pts}$
---|---
HERA I+II NC Abramowicz:2015mha  | 1291.46/1064
HERA I+II CC Abramowicz:2015mha  | 114.8/81
H1 $F_{2}^{b}$ Aaron:2009af  | 3.73/12
ZEUS $\sigma_{r}^{b}$ Abramowicz:2014zub  | 13.07/17
H1 incl. jets Aaron:2009vs ; Aktas:2007aa ; Aaron:2010ac  | 45.62/76
ZEUS incl. jets Chekanov:2002be ; Chekanov:2006xr ; Abramowicz:2010cka  | 67.79/82
H1 low $ep$ Collaboration:2010ry  | 134.98/124
HERA $\sigma_{r}^{c}$ Abramowicz:1900rp  | 40.85/47
BCDMS $F_{2}^{p}$Benvenuti:1989rh  | 320.78/328
CDF II $W$ asym. Aaltonen:2009ta  | 21.9/13
CDF II $Z$ rap. Aaltonen:2010zza  | 29.27/28
CMS jets ($7$ TeV) Chatrchyan:2012bja  | 145.78/133
ATLAS $W^{+}$ 2010Aad:2011dm  | 15.47/11
ATLAS $W^{-}$ 2010Aad:2011dm  | 9.58/11
ATLAS $Z$ 2010 Aad:2011dm  | 2.4/8
ATLAS Drell-Yan Aad:2013iua ; Aad:2014qja  | 19.16/21
ATLAS jets ($7$ TeV)Aad:2011fc  | 60.35/90
ATLAS jets ($2.76$ TeV)Aad:2011fc  | 47.14/56
ATLAS $W^{-}$ 2011Aaboud:2016btc  | 12.12/11
ATLAS $W^{+}$ 2011Aaboud:2016btc  | 14.44/11
ATLAS CC $Z$ 2011 Aaboud:2016btc  | 46.16/24
ATLAS high-mass FC $Z$ 2011 Aaboud:2016btc  | 4.14/6
ATLAS peak FC $Z$ 2011 Aaboud:2016btc  | 12.19/9
TOTAL $\chi^{2}/d.o.f$ | $2650.56/2250=1.178$
Table 2: The optimal values of the input PDF parameters (III) at the initial scale $Q_{0}^{2}$= 1 GeV2 determined by the global analysis of the experimental data reported in Table 1. Parameter | Best value
---|---
$B_{g}$ | $-0.3698\pm 0.02833$
$C_{g}$ | $4.5158\pm 0.2681$
$A^{\prime}_{g}$ | $1.3973\pm 0.1138$
$B^{\prime}_{g}$ | $-0.3562\pm 0.0241$
$B_{u_{\rm v}}$ | $0.7923\pm 0.0082$
$C_{u_{\rm v}}$ | $2.2082\pm 0.0327$
$E_{u_{\rm v}}$ | $-1.0898\pm 0.0223$
$B_{d_{\rm v}}$ | $0.8932\pm 0.0215$
$C_{d_{\rm v}}$ | $4.2397\pm 0.0788$
$C_{\bar{u}}$ | $3.2517\pm 0.2990$
$A_{\bar{d}}$ | $0.0909\pm 0.0052$
$B_{\bar{d}}$ | $-0.1739\pm 0.0089$
$C_{\bar{d}}$ | $10.9148\pm 1.5679$
$B_{\bar{s}}$ | $0.0644\pm 0.0610$
$C_{\bar{s}}$ | $4.9168\pm 0.7289$
$f_{s}$ | $0.7166\pm 0.0532$

Now, having a basic fit to a wide range of experimental data we can
investigate, for the first time, the impact of the IS component of the nucleon
on the global analysis as well as the extracted PDFs. Therefore, a similar
analysis should be done by including the contribution of IS component so that
each analysis is related to a different value of the ${\cal
P}_{5}^{s\bar{s}}$. For example, we can start with a value of ${\cal
P}_{5}^{s\bar{s}}=0.001$ and go to the ${\cal P}_{5}^{s\bar{s}}=0.03$ by
adding a value of $0.001$ in each stage. In the next step, for each data set
and also for all data, we must calculate the deviation of $\chi^{2}$-values
from their central values corresponding to the basic analysis listed in Table
1. Therefore, the sensitivity of each data set (and also the overall analysis)
to the IS component of the proton can be investigated. Moreover, we can obtain
the new limits on the IS probability in the nucleon which is consistent with a
large number of the experimental data. To consider the IS component we apply
two scenarios: the first is based on the IS distribution resulting from the
BHPS model (3), and the second one is related to its evolved distribution (5).

Fig. 2 shows the results of our global analysis determined through the first
scenario where we have used Eq. (3) for the IS distribution of proton. In this
figure, for each data set we have shown the $(\chi^{2}-\chi_{0}^{2})$-values
as a function of IS probability ${\cal P}_{5}^{s\bar{s}}$. We have also shown
the $(\chi^{2}-\chi_{0}^{2})$-values for all data (solid line), where the
$\chi_{0}^{2}$ is obtained through our basic analysis in which the IS
component is ignored. Note that, the $\chi^{2}$ stands for the analysis
including the IS component. Generally, most of data sets have either a little
sensitivity or no sensitivity on the IS component so that their
$\chi^{2}$-value does not have a considerable deviation from the corresponding
value obtained through the basic analysis. The largest deviation for the
$\chi^{2}$ arises from the NC HERA I+II combined data Abramowicz:2015mha
(dashed line), while the smaller ones come from the CMS inclusive jets
Chatrchyan:2012bja  and BCDMS proton structure functions $F_{2}^{p}$
Benvenuti:1989rh . An interesting and considerable point is that, when an IS
component is considered in the proton the $Z$ rapidity distributions at $7$
TeV from the ATLAS 2010 Aad:2011dm  and 2011 Aaboud:2016btc  data sets (for
the forward channel (FC) in the $Z$-peak regions) are just experimental data
which show a reduction in the $\chi^{2}$.

Figure 2: The $\chi^{2}-\chi_{0}^{2}$ as a function of the IS probability
${\cal P}_{5}^{s\bar{s}}$ in the first scenario. The largest deviation for the
$\chi^{2}$ arises from the NC HERA I+II combined data Abramowicz:2015mha
(dashed line), while the smaller contributions arise from the CMS inclusive
jets Chatrchyan:2012bja  and BCDMS proton structure functions $F_{2}^{p}$
Benvenuti:1989rh .

In our QCD analysis of PDFs, we used the Hessian approach introduced in
section IV to estimate the PDFs uncertainties. Following this method, a
confidence region is identified by providing the tolerance criterion
$\Delta\chi^{2}$. No unique criterion is defined to select a correct value of
$\Delta\chi^{2}$ in various global analysis which have been done yet. In some
analysis, a value $\Delta\chi^{2}=1$ is chosen as a standard tolerance
criterion Alekhin:2017kpj ; Jimenez-Delgado:2014twa ; Jimenez-Delgado:2014zga
. To be more precise, in such analysis the $\Delta\chi^{2}$-value is
calculated such that the confidence level $P$ becomes the one-$\sigma$-error
range ($P=0.6826$) for only one parameter. In fact, the main argument for this
kind of choice is that for the Gaussian distributions the parameter errors in
$\chi^{2}$-fits should be determined by $\Delta\chi^{2}=1$ irrespective of the
number of parameters in the fit Jimenez-Delgado:2015tma . This approach is
usually called as Hessian methodology, but with no tolerance. In contrary, in
some analysis the value of $\Delta\chi^{2}>1$ is assumed for the tolerance
criterion deFlorian:2011fp ; Pumplin:2007wg ; Dulat:2013hea  in order for
inflating the uncertainty and counting the tensions among different data sets
used in the fit. However, from another point of view, the actual value of
$\Delta\chi^{2}$ depends on the number of parameters which are specified in
the fit Brodsky:2015uwa . Indeed, a value different of $\Delta\chi^{2}=1$
should be assigned for the freedom degrees $N$. For example, if there are $16$
free parameters in the analysis, like ours, the $\Delta\chi^{2}$-value is
assumed as $\Delta\chi^{2}=18.112$ at the $1\sigma$ level. This approach is
applied in some analysis such as Kovarik:2015cma ; Soleymaninia:2013cxa ;
Nejad:2015fdh .

In our analysis, we specify the limit of IS probability ${\cal
P}_{5}^{s\bar{s}}$ both for $\Delta\chi^{2}=1$ and $18.112$, and also at the
$1\sigma$ and $4\sigma$ levels. Focusing on Fig. 2, we find that with
$\Delta\chi^{2}=1$ at the $1\sigma$ level, the experimental data do not
tolerate any IS component in the proton, since even for the ${\cal
P}_{5}^{s\bar{s}}=0.001$ one has $\Delta\chi^{2}>1$. Nevertheless, at the
$4\sigma$ level an IS component with the probability ${\cal
P}_{5}^{s\bar{s}}\approx 0.007$ can be considered. Similarly, if we consider
$\Delta\chi^{2}=18.112$ corresponding to $16$ free parameters at the $1\sigma$
level, we find ${\cal P}_{5}^{s\bar{s}}\approx 0.008$ while the corresponding
value at the $4\sigma$ level is ${\cal P}_{5}^{s\bar{s}}\approx 0.016$.
Remember that, the mentioned results are related to the first scenario where
we used the IS distribution (3). However, if we apply the second scenario
using Eq. (5) with a smaller momentum fraction, the results are changed to a
considerable extent, see Fig. 3. As is seen, the sensitivity of experimental
data to the IS component is decreased in comparison with the previous one, as
was expected. In this case, considering $\Delta\chi^{2}=1$ we get ${\cal
P}_{5}^{s\bar{s}}\approx 0.001$ and ${\cal P}_{5}^{s\bar{s}}\approx 0.01$ at
the $1\sigma$ and $4\sigma$ levels, respectively. The corresponding value for
the $\Delta\chi^{2}=18.112$ is ${\cal P}_{5}^{s\bar{s}}\approx 0.011$ at the
$1\sigma$, and ${\cal P}_{5}^{s\bar{s}}\approx 0.025$ at the $4\sigma$ level.
Generally, we can assert that if one uses the evolved BHPS result for the IS
distribution, the experimental data can tolerate an IS component with a
greater probability. Indeed, this conclusion confirms the Chang and Peng
suggestion Chang:2011vx ; Chang:2011du ; Chang:2014lea  so that if the
intrinsic light quark distributions are evolved from a lower initial scale
(such as $Q_{0}=0.5$ GeV or $0.3$ GeV), a more suitable fit can be achieved.
As a last point, note that the results of our global analysis are accessible
in the LHAPDF6 format Buckley:2014ana  (For more detail see the Appendix).
These PDF sets can be applied in future studies on the impact of IS
distributions on the physical observables.

Figure 3: As in Fig. 2, but for the second scenario where we used the evolved
BHPS IS distribution (5).

##  VI strange content of proton

As a conclusion of previous section, if one uses an evolved BHPS IS
distribution in the proton a greater probability ${\cal P}_{5}^{s\bar{s}}$ can
be obtained, consistent with a wide range of the experimental data. In fact,
considering the standard tolerance criterion $\Delta\chi^{2}=1$ we obtained
the limit of IS probability as ${\cal P}_{5}^{s\bar{s}}\approx 0.01$ at the
$4\sigma$ level. Moreover, we found that at the same $\sigma$ level the limit
of IS probability for a greater value $\Delta\chi^{2}=18.112$ (corresponding
to $16$ free parameters), can even reach the ${\cal P}_{5}^{s\bar{s}}\approx
0.025$. In this section, considering these two values for ${\cal
P}_{5}^{s\bar{s}}$, we study the strange content of the proton in detail. To
further investigation, the comparisons are made at different values of
$Q^{2}$.
Fig. 4 (top panel) shows a comparison between the extrinsic strange
distribution extracted through the BHPS model in the absence of an IS
component (solid line), and the total strange distributions in the presence of
an IS component with the probabilities ${\cal P}_{5}^{s\bar{s}}=0.01$ (dashed
line) and ${\cal P}_{5}^{s\bar{s}}=0.025$ (dot-dashed line) corresponding to
two scenarios. For a more quantitative interpretation of Fig. 4, in the lower
panel we showed the ratios of both results to the extrinsic strange
distribution. We have also showed the uncertainty of the extrinsic strange
distribution in Fig. 4. Note that, the total strange distributions including
an IS component with the probability ${\cal P}_{5}^{s\bar{s}}=0.025$ is not
within the error band for large values of $x$, where the contribution of IS
component is dominant. As can be seen, inclusion of an IS component with the
probability of $1\%$ does not change the strange content of the proton,
significantly. Although, an IS component with the probability of $2.5\%$ can
have a remarkable role at the medium- and large-$x$ regions. The enhancement
shown in Fig. 4 is obviously due to the inclusion of an IS component which
increases the momentum fraction carried by the strange sea and, in conclusion,
the contribution of strange to the sea quark distribution increases, see Fig.
1. The reduction of total strange distributions for the region $x\gtrsim 0.5$
can be related to the effects of momentum sum rule (III) for which any
reduction (enhancement) in the momentum fraction carried by a specific parton
leads to an enhancement (reduction) in the momentum fraction carried by
another parton.

Figure 4: Up panel: comparison between the extrinsic strange distribution in
the proton extracted through BHPS model (solid line) and the total strange
distributions including an IS component with the probabilities ${\cal
P}_{5}^{s\bar{s}}=0.01$ (dashed line) and ${\cal P}_{5}^{s\bar{s}}=0.025$
(dot-dashed line) at the initial scale $Q_{0}^{2}=1$ GeV2. Down panel: the
total strange distributions normalized to the extrinsic one.

In Fig. 5, to study the evolution effects on the total strange distribution of
the proton, we have made a similar comparison as in Fig. 4, but for higher
energy scales $Q^{2}=100$ GeV2 (upper panel) and $Q^{2}=m_{Z}^{2}=8317$ GeV2
(lower panel). As is seen, in comparison with the initial energy scale (Fig.
4), a similar behavior is regained for the total strange distribution of
proton, but with a smaller intensity. It should be also noted that the
inclusion of an IS component into the analysis leads to a small change in the
free parameter $f_{s}$ which appears in
$A_{\bar{s}}=f_{s}A_{\bar{d}}/(1-f_{s})$. For example, considering the second
scenario the value of $f_{s}$ is changed from $0.7166$ (corresponding to the
analysis without considering an IS component) to $0.7323$ (for the analysis
including an IS component with the probability of $2.5\%$). The inclusion
effect of an IS component can also lead to a small change in the contribution
of total strange and the distribution of down-like sea quark. Basically, when
the IS probability ${\cal P}_{5}^{s\bar{s}}$ is increased in the analysis, a
larger value is obtained for the $f_{s}$.

Figure 5:  As in Fig. 4, but for higher energy scales $Q^{2}=100$ GeV2 (top
panel) and $m_{Z}^{2}$ (lower panel).

In the following, we compute the ratio of strange-to-light sea-quark densities
in the proton for two situations: with and without inclusion of an IS
component. Analytically, the global PDF fits including the dimuon production
in neutrino scattering Mason:2007zz ; Samoylov:2013xoa ; KayisTopaksu:2011mx ;
Goncharov:2001qe  suggest the strangeness suppression in the proton
Dulat:2015mca ; Harland-Lang:2014zoa ; Alekhin:2014sya . However, in Ref.
Aad:2012sb  it is shown that the QCD analysis of the ATLAS 2010 $W^{\pm}$ and
$Z$ measurements Aad:2011dm  offer the presence of strange quark at $x=0.023$
and $Q^{2}=1.9$ GeV2. In other words, their results assert the existence of
all light quarks in a similar strength in the kinematic range mentioned. This
claim has been confirmed by the analysis of ATLAS measurements of associated
$W^{\pm}$ and charm production Aad:2014xca  and also the recent ATLAS 2011
$W^{\pm}$ and $Z$ measurements Aaboud:2016btc .

To evaluate the fraction of the strange-quark density in the proton, we apply
the following definition of the quantity $r_{s}$ Aaboud:2016btc

$\displaystyle r_{s}=\frac{s+\bar{s}}{2\bar{d}},$ (18)

which is the ratio of the strange to the down sea-quark distributions for a
specific value of Bjorken variable $x$. The analysis of Refs. Aad:2014xca  and
Aaboud:2016btc  which are based on the ATLAS 2010 and 2011 $W^{\pm}/Z$
measurements have determined the values $r_{s}=1.00$ and $r_{s}=1.19$,
respectively, at $x=0.023$ and $Q^{2}=1.9$ GeV2. Since in our analysis, both
the ATLAS 2010 and 2011 data sets have been included simultaneously, then it
would be interesting if we calculate the $r_{s}$ for the analysis without
considering an IS component. For the extrinsic analysis, we obtain the value
$r_{s}=1.18_{-0.06}^{+0.066}$ at $x=0.023$ and $Q^{2}=1.9$ GeV2. This value is
a little smaller than the one obtained in the analysis of ATLAS Aaboud:2016btc
Collaboration. Note that, to determine this number they have just used their
measurements along with the HERA I+II combined data Abramowicz:2015mha , while
our analysis includes a wide range of the collider data. If we include an IS
component in the analysis, the values of $r_{s}=1.20_{-0.061}^{+0.066}$ and
$r_{s}=1.24_{-0.062}^{+0.067}$ are obtained considering the ${\cal
P}_{5}^{s\bar{s}}=0.01$ and ${\cal P}_{5}^{s\bar{s}}=0.025$, respectively, in
the same kinematical range. This means, the inclusion of an IS component in
the proton sea leads to a larger strange-quark density. In fact, when the
probability of IS distribution is increased a larger value is obtained for the
ratio $r_{s}$ in the kinematical range $x=0.023$ and $Q^{2}=1.9$ GeV2.

##  VII Summary and Conclusions

For many years, various attempts have been made to study the impact of an
intrinsic charm component of the nucleon sea on the global analysis of parton
distribution functions Harris:1995jx ; Pumplin:2007wg ; Nadolsky:2008zw ;
Dulat:2013hea ; Jimenez-Delgado:2014zga . In this regard, some studies have
been done to extract the intrinsic light quark distributions and the
corresponding probabilities in the nucleon Chang:2011vx ; Chang:2011du ;
Chang:2014lea ; Salajegheh:2015xoa ; An:2017flb . Although, a simultaneous
study of the intrinsic and extrinsic flavors have not been performed yet in
any global analysis of PDFs. In the present work, we studied for the first
time this simultaneous analysis in the case of intrinsic strange component by
virtue of the xFitter program Alekhin:2014irh  and considering a wide range of
the experimental data such as the HERA I+II combined data Abramowicz:2015mha
and the ATLAS $W^{\pm}$, $Z$ 2010 Aad:2011dm  and 2011 Aaboud:2016btc
measurements.
For our aim, we applied the analytical result of BHPS model Brodsky:1980pb ;
Brodsky:1981se  for the IS distribution, as the intrinsic contribution to the
total strange density of the proton. Considering the BHPS model we employed
two scenarios: in the first scenario we used the result of this model
directly, and in the second one we used its evolved IS distribution. The
result of BHPS model leads to a momentum fraction $\langle
x\rangle_{\textrm{IS}}=0.41$ while its evolved leads to a smaller one as
$\langle x\rangle_{\textrm{IS}}=0.27$. As a primary conclusion we found that
for both scenarios most of data sets have either a little sensitivity or no
sensitivity on the IS component. This means, their $\chi^{2}$-value dose not
have a considerable deviation from the one obtained through our basic analysis
in the absence of IS component. The most variations for the $\chi^{2}$ arise
from the NC HERA I+II combined data, the CMS inclusive jets Chatrchyan:2012bja
and the BCDMS proton structure functions $F_{2}^{p}$ Benvenuti:1989rh . Among
all data sets, the Z rapidity distributions at 7 TeV from the ATLAS 2010 data
set and its 2011 data set for the forward channel in the Z-peak regions show a
reduction in the $\chi^{2}$ when an IS component is considered in the proton
sea. Moreover, we found that if one follows the second scenario the
sensitivity of experimental data on the IS component is less.
For each scenario, we also presented the limit of the IS probability ${\cal
P}_{5}^{s\bar{s}}$ for both standard tolerance criteria $\Delta\chi^{2}=1$ and
$18.112$ corresponding to $16$ free parameters of the global fit, and also at
the $1\sigma$ and $4\sigma$ level. Following the first scenario we found that
with $\Delta\chi^{2}=1$ at the $1\sigma$ level, the experimental data do not
tolerate any IS component in the proton. However, at the $4\sigma$ level an IS
component with the probability ${\cal P}_{5}^{s\bar{s}}\approx 0.007$ can be
considered. In analogy, for $\Delta\chi^{2}=18.112$ we found the ${\cal
P}_{5}^{s\bar{s}}\approx 0.008$ and ${\cal P}_{5}^{s\bar{s}}\approx 0.016$ at
the $1\sigma$ and $4\sigma$ level, respectively. We have also concluded that
following the second scenario and using the evolved BHPS IS distribution, the
experimental data can tolerate an IS component with a greater probability
${\cal P}_{5}^{s\bar{s}}$. In fact, with $\Delta\chi^{2}=1$ one gets ${\cal
P}_{5}^{s\bar{s}}\approx 0.001$ and ${\cal P}_{5}^{s\bar{s}}\approx 0.01$ at
the $1\sigma$ and $4\sigma$ level, respectively. The corresponding values for
$\Delta\chi^{2}=18.112$ are ${\cal P}_{5}^{s\bar{s}}\approx 0.011$ at the
$1\sigma$ and ${\cal P}_{5}^{s\bar{s}}\approx 0.025$ at the $4\sigma$ level.
As a next step, we studied the strange content of the proton in detail. In
this respect, using the second scenario we compared the proton extrinsic
strange distribution (obtained through the global QCD analysis by ignoring the
IS component) with the results of the total strange distribution obtained by
including an IS component with the probabilities $1\%$ and $2.5\%$. We found
that the main changes in the total strange distribution of the proton sea (due
to the inclusion of an IS component) can be divided to two different regions:
(1) an enhancement in the region $0\lesssim x\lesssim 0.5$ and (2) a reduction
for $x\gtrsim 0.5$. We have shown that, as the probability of the IS
distribution increases the rate of variations would be more.
In this work, the effect of evolution on the total strange distribution of the
proton is also studied. It is shown that a similar behavior is obtained at
higher energies. The only remarkable point is that, as $Q^{2}$ increases the
rate of enhancement and reduction decreases, partially.
As a last conclusion, we calculated the ratio of strange-to-light sea-quark
densities $r_{s}$ in the proton both by including and excluding an IS
component. For the extrinsic analysis, we obtained
$r_{s}=1.18_{-0.06}^{+0.066}$ at $x=0.023$ and $Q^{2}=1.9$ GeV2 which is a
little smaller than the result obtained in the recent analysis of ATLAS 2011
$W^{\pm}$ and $Z$ measurements Aaboud:2016btc . We have shown that if one
includes an IS component in the analysis, the values
$r_{s}=1.20_{-0.061}^{+0.066}$ and $r_{s}=1.24_{-0.062}^{+0.067}$ are obtained
for the analysis with the ${\cal P}_{5}^{s\bar{s}}=0.01$ and ${\cal
P}_{5}^{s\bar{s}}=0.025$, respectively. Indeed, as the probability of IS
distribution increases, a larger value is obtained for the $r_{s}$ in the
kinematical range $x=0.023$ and $Q^{2}=1.9$ GeV2, as is expected.

## Appendix: The extracted PDF sets

For further studies on the impact of IS distributions on the physical
observables, it is important to have the PDF sets including the contribution
of IS component. Then, we have provided the results of our global PDF analysis
in the LHAPDF6 format, including the basic analysis (where the contribution of
an IS component is ignored) and the analysis of scenario II considering an IS
component with the probabilities $1\%$ and $2.5\%$. The grid files can be
obtained via email from the authors.

## References

  * (1) S. Dulat et al., Phys. Rev. D 93, no. 3, 033006 (2016).
  * (2) L. A. Harland-Lang, A. D. Martin, P. Motylinski and R. S. Thorne, Eur. Phys. J. C 75, no. 5, 204 (2015).
  * (3) P. Jimenez-Delgado and E. Reya, Phys. Rev. D 89, no. 7, 074049 (2014).
  * (4) H. Abramowicz et al. [H1 and ZEUS Collaborations], Eur. Phys. J. C 75, no. 12, 580 (2015).
  * (5) S. Alekhin, J. BlÃŒmlein, S. Moch and R. Placakyte, Phys. Rev. D 96, no. 1, 014011 (2017).
  * (6) S. M. Moosavi Nejad, H. Khanpour, S. Atashbar Tehrani and M. Mahdavi, Phys. Rev. C 94, no. 4, 045201 (2016).
  * (7) P. Jimenez-Delgado et al. [Jefferson Lab Angular Momentum (JAM) Collaboration], “Constraints on spin-dependent parton distributions at large x from global QCD analysis,” Phys. Lett. B 738, 263 (2014).
  * (8) N. Sato et al. [Jefferson Lab Angular Momentum Collaboration], Phys. Rev. D 93, no. 7, 074005 (2016).
  * (9) F. Taghavi-Shahri, H. Khanpour, S. Atashbar Tehrani and Z. Alizadeh Yazdi, Phys. Rev. D 93, no. 11, 114024 (2016).
  * (10) H. Khanpour, S. T. Monfared and S. Atashbar Tehrani, Phys. Rev. D 95, no. 7, 074006 (2017).
  * (11) D. de Florian, R. Sassot, P. Zurita and M. Stratmann, Phys. Rev. D 85, 074028 (2012).
  * (12) K. Kovarik et al., Phys. Rev. D 93, no. 8, 085037 (2016).
  * (13) K. J. Eskola, P. Paakkinen, H. Paukkunen and C. A. Salgado, Eur. Phys. J. C 77, no. 3, 163 (2017).
  * (14) M. Klasen, K. Kovarik and J. Potthoff, Phys. Rev. D 95, 094013 (2017).
  * (15) S. M. Moosavi Nejad, M. Soleymaninia and A. Maktoubian, Eur. Phys. J. A 52, no. 10, 316 (2016).
  * (16) S. M. Moosavi Nejad, M. Roknabady and M. Delpasand, Nucl. Phys. B 956 (2020) 115036.
  * (17) M. Delpasand and S. M. Moosavi Nejad, Eur. Phys. J. A 56 (2020) no.2, 56.
  * (18) S. M. Moosavi Nejad and M. Balali, Eur. Phys. J. C 76 (2016) no.3, 173.
  * (19) M. Delpasand and S. M. Moosavi Nejad, Phys. Rev. D 99 (2019) no.11, 114028.
  * (20) S. M. Moosavi Nejad, M. Torkian and M. Delpasand, Phys. Rev. D 102 (2020) no.3, 034025.
  * (21) S. M. Moosavi Nejad, Phys. Rev. D 96 (2017) no.11, 114021.
  * (22) M. Soleymaninia, H. Khanpour and S. M. Moosavi Nejad, Phys. Rev. D 97 (2018) no.7, 074014.
  * (23) S. M. Moosavi Nejad and M. Delpasand, Eur. Phys. J. A 53 (2017) no.9, 174.
  * (24) S. M. Moosavi Nejad and M. Delpasand, Int. J. Mod. Phys. A 30 (2015) no.32, 1550179.
  * (25) S. M. Moosavi Nejad, Eur. Phys. J. A 52 (2016) no.5, 127.
  * (26) M. Constantinou et al., arXiv:2006.08636 [hep-ph].
  * (27) M. Ding, K. Raya, D. Binosi, L. Chang, C. D. Roberts and S. M. Schmidt, Chin. Phys. C 44 (2020) no.3, 031002.
  * (28) S. Alekhin et al., Eur. Phys. J. C 75, no. 7, 304 (2015).
  * (29) V. Bertone et al. [xFitter Developers’ Team], JHEP 1608, 050 (2016).
  * (30) F. Giuli et al. [xFitter Developers’ Team], Eur. Phys. J. C 77, no. 6, 400 (2017).
  * (31) G. Altarelli and G. Parisi, Nucl. Phys. B 126, 298 (1977).
  * (32) V. N. Gribov and L. N. Lipatov, Sov. J. Nucl. Phys. 15, 438 (1972) [Yad. Fiz. 15, 781 (1972)].
  * (33) S. J. Brodsky, P. Hoyer, C. Peterson and N. Sakai, Phys. Lett. 93B, 451 (1980).
  * (34) S. J. Brodsky, C. Peterson and N. Sakai, Phys. Rev. D 23, 2745 (1981).
  * (35) B. W. Harris, J. Smith and R. Vogt, Nucl. Phys. B 461, 181 (1996).
  * (36) J. Pumplin, H. L. Lai and W. K. Tung, Phys. Rev. D 75, 054029 (2007).
  * (37) P. M. Nadolsky, H. L. Lai, Q. H. Cao, J. Huston, J. Pumplin, D. Stump, W. K. Tung and C.-P. Yuan, Phys. Rev. D 78, 013004 (2008).
  * (38) S. Dulat, T. J. Hou, J. Gao, J. Huston, J. Pumplin, C. Schmidt, D. Stump and C.-P. Yuan, Phys. Rev. D 89, no. 7, 073004 (2014).
  * (39) P. Jimenez-Delgado, T. J. Hobbs, J. T. Londergan and W. Melnitchouk, Phys. Rev. Lett. 114, no. 8, 082002 (2015).
  * (40) S. J. Brodsky and S. Gardner, Phys. Rev. Lett. 116, no. 1, 019101 (2016).
  * (41) T. J. Hou et al., arXiv:1707.00657 [hep-ph].
  * (42) W. C. Chang and J. C. Peng, Phys. Rev. Lett. 106, 252002 (2011).
  * (43) W. C. Chang and J. C. Peng, Phys. Lett. B 704, 197 (2011).
  * (44) W. C. Chang and J. C. Peng, Phys. Rev. D 92, no. 5, 054020 (2015).
  * (45) M. Salajegheh, Phys. Rev. D 92, no. 7, 074033 (2015).
  * (46) C. S. An and B. Saghai, Phys. Rev. D 95, no. 7, 074015 (2017).
  * (47) A. Vega, I. Schmidt, T. Gutsche and V. E. Lyubovitskij, Phys. Rev. D 93, no. 5, 056001 (2016).
  * (48) A. W. Thomas, Phys. Lett. 126B, 97 (1983).
  * (49) A. I. Signal and A. W. Thomas, Phys. Lett. B 191, 205 (1987).
  * (50) J. Pumplin, Phys. Rev. D 73, 114015 (2006).
  * (51) T. J. Hobbs, J. T. Londergan and W. Melnitchouk, Phys. Rev. D 89, no. 7, 074008 (2014).
  * (52) S. J. Brodsky, A. Kusina, F. Lyonnet, I. Schienbein, H. Spiesberger and R. Vogt, Adv. High Energy Phys. 2015, 231547 (2015).
  * (53) F. Lyonnet, A. Kusina, T. JeÅŸo, K. KovarÃk, F. Olness, I. Schienbein and J. Y. Yu, JHEP 1507, 141 (2015).
  * (54) A. Airapetian et al. [HERMES Collaboration], Phys. Lett. B 666, 446 (2008).
  * (55) M. Mestayer et al. [CLAS Collaboration], Phys. Rev. Lett. 113, no. 15, 152004 (2014).
  * (56) M. Botje, Comput. Phys. Commun. 182, 490 (2011).
  * (57) R. S. Thorne and R. G. Roberts, Phys. Rev. D 57, 6871 (1998).
  * (58) J. C. Collins, D. E. Soper and G. F. Sterman, “Factorization of Hard Processes in QCD,” Adv. Ser. Direct. High Energy Phys. 5, 1 (1989).
  * (59) C. Adloff et al. [H1 Collaboration], Eur. Phys. J. C 21, 33 (2001).
  * (60) F. D. Aaron et al. [H1 and ZEUS Collaborations], JHEP 1001, 109 (2010).
  * (61) F. James and M. Roos, Comput. Phys. Commun. 10, 343 (1975).
  * (62) J. Pumplin, D. Stump, R. Brock, D. Casey, J. Huston, J. Kalk, H. L. Lai and W. K. Tung, Phys. Rev. D 65, 014013 (2001).
  * (63) J. Pumplin, D. R. Stump, J. Huston, H. L. Lai, P. M. Nadolsky and W. K. Tung, JHEP 0207, 012 (2002)
  * (64) A. D. Martin, R. G. Roberts, W. J. Stirling and R. S. Thorne, Eur. Phys. J. C 28, 455 (2003); Eur. Phys. J. C 35, 325 (2004)
  * (65) A. D. Martin, W. J. Stirling, R. S. Thorne and G. Watt, Eur. Phys. J. C 63, 189 (2009)
  * (66) M. Salajegheh, S. M. Moosavi Nejad, M. Nejad, H. Khanpour and S. Atashbar Tehrani, Phys. Rev. C 97 (2018) no.5, 055201.
  * (67) M. Soleymaninia, A. N. Khorramian, S. M. Moosavi Nejad and F. Arbabifar, Phys. Rev. D 88, no. 5, 054019 (2013) Addendum: [Phys. Rev. D 89, no. 3, 039901 (2014)].
  * (68) M. Salajegheh, S. M. Moosavi Nejad and M. Delpasand, Phys. Rev. D 100 (2019) no.11, 114031.
  * (69) M. Salajegheh, S. M. Moosavi Nejad, M. Soleymaninia, H. Khanpour and S. Atashbar Tehrani, Eur. Phys. J. C 79 (2019) no.12, 999.
  * (70) M. Salajegheh, S. M. Moosavi Nejad, H. Khanpour, B. A. Kniehl and M. Soleymaninia, Phys. Rev. D 99 (2019) no.11, 114001.
  * (71) S. M. Moosavi Nejad, Eur. Phys. J. Plus 133 (2018) no.1, 25.
  * (72) A. C. Benvenuti et al. [BCDMS Collaboration], Phys. Lett. B 223, 485 (1989).
  * (73) F. D. Aaron et al. [H1 Collaboration], Eur. Phys. J. C 71, 1579 (2011).
  * (74) F. D. Aaron et al. [H1 Collaboration], Eur. Phys. J. C 65, 89 (2010).
  * (75) H. Abramowicz et al. [ZEUS Collaboration], JHEP 1409, 127 (2014).
  * (76) H. Abramowicz et al. [H1 and ZEUS Collaborations], Eur. Phys. J. C 73, no. 2, 2311 (2013).
  * (77) F. D. Aaron et al. [H1 Collaboration], Eur. Phys. J. C 65, 363 (2010).
  * (78) A. Aktas et al. [H1 Collaboration], Phys. Lett. B 653, 134 (2007).
  * (79) F. D. Aaron et al. [H1 Collaboration], Eur. Phys. J. C 67, 1 (2010).
  * (80) S. Chekanov et al. [ZEUS Collaboration], Phys. Lett. B 547, 164 (2002).
  * (81) S. Chekanov et al. [ZEUS Collaboration], Nucl. Phys. B 765, 1 (2007).
  * (82) H. Abramowicz et al. [ZEUS Collaboration], Eur. Phys. J. C 70, 965 (2010).
  * (83) G. Aad et al. [ATLAS Collaboration], Phys. Rev. D 86, 014022 (2012).
  * (84) G. Aad et al. [ATLAS Collaboration], Eur. Phys. J. C 73, no. 8, 2509 (2013).
  * (85) S. Chatrchyan et al. [CMS Collaboration], Phys. Rev. D 87, no. 11, 112002 (2013) Erratum: [Phys. Rev. D 87, no. 11, 119902 (2013)].
  * (86) J. Rojo, Int. J. Mod. Phys. A 30, 1546005 (2015).
  * (87) G. Aad et al. [ATLAS Collaboration], Phys. Lett. B 725, 223 (2013).
  * (88) G. Aad et al. [ATLAS Collaboration], JHEP 1406, 112 (2014).
  * (89) T. Aaltonen et al. [CDF Collaboration], Phys. Rev. Lett. 102, 181801 (2009).
  * (90) T. A. Aaltonen et al. [CDF Collaboration], Phys. Lett. B 692, 232 (2010).
  * (91) G. Aad et al. [ATLAS Collaboration], Phys. Rev. D 85, 072004 (2012).
  * (92) M. Aaboud et al. [ATLAS Collaboration], Eur. Phys. J. C 77, no. 6, 367 (2017).
  * (93) P. Jimenez-Delgado, T. J. Hobbs, J. T. Londergan and W. Melnitchouk, Phys. Rev. Lett. 116, no. 1, 019102 (2016).
  * (94) A. Buckley, J. Ferrando, S. Lloyd, K. NordstrÃ¶m, B. Page, M. RÃŒfenacht, M. SchÃ¶nherr and G. Watt, Eur. Phys. J. C 75, 132 (2015).
  * (95) D. Mason et al. [NuTeV Collaboration], Phys. Rev. Lett. 99, 192001 (2007).
  * (96) A. Kayis-Topaksu et al., New J. Phys. 13, 093002 (2011).
  * (97) O. Samoylov et al. [NOMAD Collaboration], Nucl. Phys. B 876, 339 (2013).
  * (98) M. Goncharov et al. [NuTeV Collaboration], Phys. Rev. D 64, 112006 (2001).
  * (99) S. Alekhin, J. Blumlein, L. Caminadac, K. Lipka, K. Lohwasser, S. Moch, R. Petti and R. Placakyte, Phys. Rev. D 91, no. 9, 094002 (2015).
  * (100) G. Aad et al. [ATLAS Collaboration], Phys. Rev. Lett. 109, 012001 (2012).
  * (101) G. Aad et al. [ATLAS Collaboration], JHEP 1405, 068 (2014).


11institutetext: University of Toronto, Toronto ON, Canada
11email<EMAIL_ADDRESS>
22institutetext: University of Toronto Mississauga, Mississauga ON, Canada
22email<EMAIL_ADDRESS>
33institutetext: Vector Institute, Toronto ON, Canada
33email<EMAIL_ADDRESS>
# Using Decomposed Prompting to Answer Questions on a Course Discussion Board

Brandon Jaipersaud 33 0009-0007-0478-7356 Paul Zhang 22 0009-0008-7674-6657
Jimmy Ba 1133 0009-0000-9062-4180 Andrew Petersen 22 0000-0003-1337-7985 Lisa
Zhang 22 0000-0002-7302-6530 Michael R. Zhang 1133 0009-0000-8281-0856

###### Abstract

We propose and evaluate a question-answering system that uses decomposed
prompting to classify and answer student questions on a course discussion
board. Our system uses a large language model (LLM) to classify questions into
one of four types: conceptual, homework, logistics, and not answerable. This
enables us to employ a different strategy for answering questions that fall
under different types. Using a variant of GPT-3, we achieve 81% classification
accuracy. We discuss our system’s performance on answering conceptual
questions from a machine learning course and various failure modes.

###### Keywords:

Course Discussion Board GPT-3 Large-Language Models Mixture of Experts
Prompting

##  1 Introduction

Course discussion boards are an important avenue for students to ask course-
related questions. However, instructors and teaching assistants spend much
time and effort responding to discussion board questions, especially for
courses with large enrollment [8, 3]. With recent advances in natural language
processing, there is an opportunity to leverage general-purpose large language
models (LLMs) to assist in answering these questions. While GPT-3 has been
used in education to adapt assignments [7, 6] and explanations [1], we are not
aware of work using LLMs for course Q&A. Furthermore, non-LLM automated
methods for answering student questions have been developed in prior work [8,
2, 3]. However, the cost of answering a question incorrectly is high:
providing an incorrect answer can be detrimental for students and increase
course staff workload.

To mitigate this risk, we propose a LLM-based question-answering system that
uses decomposed prompting [4] to answer student questions. We first prompt a
LLM to classify a student question into one of four types: conceptual,
homework, logistics, and not answerable. Then, depending on the question type,
we either ignore the question, or we further prompt a LLM to answer the
question.

This Mixture of Experts approach can mitigate risk in a few ways. First, the
risk and failure modes of an incorrect answer differ depending on the question
type (e.g. a question about a deadline vs. explaining subtle concepts). Since
each expert model can be deployed independently, educators have more control
over the system. Second, we demonstrate that while LLMs can effectively
differentiate between types of questions, different answering approaches and
contextual cues may be effective for each question type. This modular approach
suggests a framework for managing risk in real deployments and allows future
work to focus on answering specific types of questions effectively.

Section 2 describes our approach. In Section 3.1, we show that our
classification system achieves an accuracy of 81%, and analyze how the number
of few-shot examples, the task description, and the question type labels
affect classification accuracy. In Section 4.2, we use various metrics to show
that our conceptual prompt works best for answering conceptual questions from
an Intro to ML course. We use human evaluation to validate our automatic
metrics and discuss the failure modes of our system on conceptual questions.

Student Question Classification prompt Conceptual prompt Contextual prompt
Ignore AnswerAnswerconceptual homework/ logistics not answerable

Figure 1: Mixture of Experts for answering questions using decomposed
prompting.

##  2 Mixture of Experts with Decomposed Prompting

Decomposed prompting [4] is an approach that uses LLM prompting to decompose a
complex problem into simpler subtasks where each subtask can be solved by
further prompting a LLM. Our mixture of experts design (Figure 1) decomposes
the task of answering student questions into two subtasks: Question
Classification and Question Answering. We first classify questions using the
prompt in Figure 2 (left). Based on the classification, we further prompt a
LLM to answer the question using Figure 2 (right) or ignore the question.

The Question Classification task classifies student questions by type. As in
prior work [8], our question types are based on the course-specific context
required to answer the question. The four question types we use are:

  1. 1.

Conceptual Questions that can be answered without course-specific context,
e.g., How do we choose the learning rate?

  2. 2.

Homework Questions that require the corresponding homework instructions to be
answerable, e.g., What does z refer to in Lab 1?.

  3. 3.

Logistics Questions that require the course syllabus to be answerable, e.g.,
Which room is the midterm in?

  4. 4.

Not Answerable Questions that require human intervention, e.g., The instructor
isn’t here. Have office hours been cancelled?

The Question Answering task proposes answers to a question. Our intention is
for each question type to have a different prompting strategy: for example,
conceptual questions do not require specialized context in the prompt.
However, homework questions require that we identify and provide relevant
sections of the assignment handout. Likewise, logistics questions require
context from the course syllabus. Finally, not answerable questions are left
to the instructors.

##  3 Results and Analysis

###  3.1 Question Classification

We evaluate our classification system on 72 historical student questions
randomly sampled from the Fall 2022 instance of an upper-level Intro to ML
course at a research-intensive institution in North America. We choose upper-
year courses since, in our experience, there are more conceptual questions in
these courses. To form the ground-truth type labels for the 72 questions,
three course staff manually classified each question into the four types
described in Section 2. 27 of the 72 annotated questions had a disagreement
between two of the annotators, and we take the majority label as the ground
truth. For 3 questions, all annotators disagreed on the ground truth label; we
discard these questions from our analysis. Of the remaining 69 questions with
ground truth labels, there are 13 conceptual, 34 homework, 8 logistics, and 14
not answerable questions.

The LLM we use for this task is text-davinci-003, an InstructGPT [5] variant
of OpenAI’s GPT-3 model fine-tuned on human instructions. Our prompt, which
includes 31 in-context examples, is shown in Figure 2 (left); we justify our
choice of prompt below. In Table 1, we give a breakdown of the F-score by
type; the total classification accuracy of our system is 81%.

Table 1: Precision, recall, and F-score for question classification type. The overall accuracy is 81%. Type | Count | #Correct | Precision | Recall | F-Score
---|---|---|---|---|---
conceptual | 13 | 11 | 0.79 | 0.85 | 0.81
homework | 34 | 27 | 0.96 | 0.79 | 0.87
logistics | 8 | 5 | 0.63 | 0.63 | 0.63
not answerable | 14 | 13 | 0.68 | 0.93 | 0.79
|  |  |  |  |

Task Description Our prompt begins with a task description outlining when a
question belongs to each type (Figure 2 (left)). To test the importance of
this description, we evaluate three alternatives. With our descriptive prompt,
the classification accuracy is 81%. Using no task description reduces the
accuracy to 74%. Using only the first sentence of the description gives the
lowest performance at 72%. Adding a sentence stating Questions that point out
corrections or typos should be classified as “homework” gives an accuracy of
77%.

Number of Few-shot Examples We use few-shot prompting and provide example
questions and classifications (Figure 2 (left)). To test the importance of
these in-context examples, we vary the number of examples in our prompt. Table
2 shows that using 31 examples produces the highest classification accuracy.

Table 2: Classification accuracy by number of few-shot examples. Examples | 0 | 2 | 4 | 8 | 16 | 24 | 31 | 42
---|---|---|---|---|---|---|---|---
Accuracy | 42% | 61% | 58% | 67% | 65% | 77% | 81% | 70%

Question Type Labels The classification performance is sensitive to the choice
of type labels used in the prompt. Renaming the types to a, b, c, d reduces
accuracy to 70%. Renaming the types to directly answerable, needs course
material, needs administrative material, not answerable reduces accuracy to
74%. Renaming the types to conceptual, needs course material, needs
administrative material, not answerable reduces accuracy to 75%.

###  3.2 Question-Answering System

In this section, we present an initial effort to answer conceptual questions.
In addition to the 69 questions from earlier, we include 63 additional
questions from the Winter 2023 offering of the same Intro to ML course. Of
these 63 questions, there are 20 conceptual, 31 homework, 8 logistics and 4
not answerable questions.

As before, we use text-davinci-003, now with the conceptual prompt shown in
Figure 2 (right). We use a temperature of 0.7 for answer generation since we
have observed that this temperature produces more descriptive answers.

Difficulty of each Question Type To justify focusing our attention on
conceptual questions, we start by generating model answers to all 132
questions regardless of question type, using the conceptual prompt in Figure 2
(right). We then evaluate the model’s answers using the following metrics:

  * •

The Cosine similarity between the embeddings of the model answer and the
instructor answer, generated with Cohere’s Embedding API.

  * •

The ROUGE score between model and instructor answer to measure textual
similarity between the answers.

  * •

The Perplexity of the instructor answer, which measures how likely the model
is to generate the instructor answer.

The result of applying these metrics across all 132 questions is shown in
Table 3. The LLM performs best on conceptual questions across all three
metrics. This agrees with our intuition that conceptual questions are easier
for a general-purpose LLM to answer since they do not require course-specific
details.

Table 3: Similarity between instructor and model answer by question type. Question Type | Count | Cosine Similarity | ROUGE1/2/L | Perplexity
---|---|---|---|---
conceptual | 34 | 0.62 | 0.30/0.07/0.18 | 7.61
homework | 59 | 0.48 | 0.23/0.06/0.16 | 12.73
logistics | 16 | 0.43 | 0.17/0.04/0.14 | 13.01
not answerable | 23 | 0.52 | 0.19/0.03/0.13 | 34.32
|  |  |  |

Human Evaluation on Conceptual Questions The Intro to ML instructors labeled
the 28 model answers to conceptual questions with good answer or bad answer,
depending on whether the answer correctly and appropriately addresses the
student’s question. Table 4 shows the result of applying the automated metrics
to each feedback category. The good answers are semantically and textually
closer to their corresponding instructor answers. This validates our use of
the metrics in Table 3.

Furthermore, we see that 8/28 (29%) of annotated answers were labeled with
good answer. Of the 20 bad answers, we observe the following distribution of
failure modes: 3 answers address questions that are misclassified as
conceptual (rather than homework). 5 answers are factually incorrect. For
instance: “The training set can be used to tune hyperparameters”. 3 answers
are correct but inappropriate for the student’s level of knowledge. For
example: “Gradient descent can be used to optimize hyperparameters, although
this is less common” when this form of optimization was not discussed in the
course. The remaining bad answers suffer from issues such as misunderstanding
the question, incoherence, making incorrect assumptions, or adding irrelevant
information to the answer.

Table 4: Human and automatic evaluation of conceptual answers. Feedback | Count | Cosine Similarity | ROUGE1/2/L | Perplexity
---|---|---|---|---
good answer | 8 | 0.83 | 0.65/0.51/0.57 | 3.05
bad answer | 20 | 0.61 | 0.31/0.07/0.18 | 6.35
|  |  |  |

⬇

Task Description: Classify each question posted on an undergraduate course
discussion board into one of the following 4 types: conceptual", "homework",
"logistics" or "not answerable".

A question that requires instructor intervention should be classified as "not
answerable". Questions that point out contradictions in assignment
instructions and deadlines should be classified as "not answerable".
Conceptual questions should be classified as "conceptual". Homework and lab
questions that provide enough information to be answered should be classified
as "conceptual". Questions that need course content related context to be
answered such as an assignment handout should be classified as "homework".
Questions that need logistical context to be answered such as a course
syllabus should be classified as "logistics".

Question: Is the A1 Q2 code for debugging a neural network correct? It says we
should debug using a large dataset. However, using a small dataset seems to
make more sense here.

Classification: homework

... (30 more question/classification pairs)

Question: <student question>

Classification:

⬇

Task: Answer the following question that was posted by a student on the class
discussion board for an introductory machine learning course. Your answer
should be truthful, concise and helpful to the student.

Question: <student question>

Answer:

Figure 2: Prompt used for classification (left) and conceptual Q&A (right).

##  4 Conclusion

We demonstrated that decomposed prompting is a useful strategy for classifying
and answering student questions due to their context-dependent or unanswerable
nature. Furthermore, our results have shown that LLMs can classify student
questions with an accuracy of 81%, but yield poor performance when answering
conceptual questions. Many of the poor answers arose due to misalignment with
the preferences of a course instructor. Future work could aim to fine-tune the
model on course discussion board questions. An interesting direction for
future work is to combine LLMs with semantic processing or text extraction
techniques [3, 8] which should enable prompted LLMs to handle homework and
logistics questions.

## References

  * Drori et al. [2022] Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Ke, E., Liu, K., Chen, L., Tran, S., Cheng, N., et al.: A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proceedings of the National Academy of Sciences 119(32), e2123433119 (2022)
  * Feng et al. [2006] Feng, D., Shaw, E., Kim, J., Hovy, E.: An intelligent discussion-bot for answering student queries in threaded discussions. In: Proceedings of the 11th international conference on Intelligent user interfaces, pp. 171–177 (2006)
  * Goel and Polepeddi [2018] Goel, A.K., Polepeddi, L.: Jill watson: A virtual teaching assistant for online education. In: Learning engineering for online education, pp. 120–143, Routledge (2018)
  * Khot et al. [2022] Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., Sabharwal, A.: Decomposed prompting: A modular approach for solving complex tasks (2022), doi:10.48550/ARXIV.2210.02406
  * Ouyang et al. [2022] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.: Training language models to follow instructions with human feedback. arXiv:2203.02155 (2022)
  * Sarsa et al. [2022] Sarsa, S., Denny, P., Hellas, A., Leinonen, J.: Automatic generation of programming exercises and code explanations using large language models. In: Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 1, pp. 27–43 (2022)
  * Wang et al. [2022] Wang, Z., Valdez, J., Basu Mallick, D., Baraniuk, R.G.: Towards human-like educational question generation with large language models. In: Artificial Intelligence in Education: 23rd International Conference, AIED 2022, Durham, UK, July 27–31, 2022, Proceedings, Part I, pp. 153–166, Springer (2022)
  * Zylich et al. [2020] Zylich, B., Viola, A., Toggerson, B., Al-Hariri, L., Lan, A.: Exploring automated question answering methods for teaching assistance. In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Proceedings, Part I 21, pp. 610–622, Springer (2020)