# Atomic mass, Bjorken variable and scale dependence of quark transport
coefficient in Drell-Yan process for proton incident on nucleus

Wei-Jie Xu<EMAIL_ADDRESS>College of Physics and Hebei Advanced Thin Film
Laboratory, Hebei Normal University, Shijiazhuang 050024, P.R.China  Tian-Xing
Bai<EMAIL_ADDRESS>College of Physics and Hebei Advanced Thin Film
Laboratory, Hebei Normal University, Shijiazhuang 050024, P.R.China  Chun-Gui
Duan<EMAIL_ADDRESS>College of Physics and Hebei Advanced Thin Film
Laboratory, Hebei Normal University, Shijiazhuang 050024, P.R.China

###### Abstract

By means of the nuclear parton distributions determined without the fixed-
target Drell-Yan experimental data and the analytic expression of quenching
weight based on BDMPS formalism, a next-to-leading order analyses are
performed on the Drell-Yan differential cross section ratios from Fermilab
E906 and E866 Collaborations. It is found that the calculated results with
only the nuclear effects of parton distribution are not in agreement with the
E866 and E906 experimental data. The incoming parton energy loss effect can
not be ignored in the nuclear Drell-Yan reactions. The predicted results
indicate that with the quark transport coefficient as a constant, the
suppression due to the target nuclear geometry effect is approximately
$19.24\%$ for the quark transport coefficient. It is shown that we should
consider the target nuclear geometry effect in studying the Drell-Yan reaction
on nuclear targets. On the basis of Bjorken variable and scale dependence of
the quark transport coefficient, the atomic mass dependence is incorporated.
The quark transport coefficient is determined as a function of the atomic
mass, Bjorken variable $x_{2}$ and scale $Q^{2}$ by the global fit of the
experimental data. The determined constant factor $\hat{q}_{0}$ of the quark
transport coefficient is $0.061\pm 0.004$ GeV2/fm. It is found that the atomic
mass dependence has a remarkable impact on the constant factor $\hat{q}_{0}$
in the quark transport coefficient in cold nuclear matter.

Keywords: quark transport coefficient, Drell-Yan, Energy loss

###### pacs:

12.38.-t; 13.85.Qk; 24.85.+p; 25.40.-h

##  I Introduction

The insight into quark-gluon plasma(QGP) properties has been one of the most
active frontiers in nuclear physics and particle physics up to now. Jet
quenching can provide a unique window into the nature of the QGP produced in
heavy-ion collisions[1,2]. In order to realize jet quenching, it is necessary
to make certain of the energy loss mechanism theoretically because the initial
scattering partons with high transverse momentum propagate through the QGP
while losing its energy. However, the energy loss mechanism is still not
clearly known[3-9].

As for the Drell-Yan reaction[10] of hadron on nucleus and the semi-inclusive
deep inelastic scattering of lepton on nucleus, the bound nucleons in target
nucleus play the role of very nearby detectors for the traversing partons. The
hard scattering is localized in space. The properties of target nuclei are
well known. In addition, the semi-inclusive deep inelastic scattering on
nuclear targets is an ideal tool to study the energy loss of the outgoing
parton in the cold nuclear medium[11-15]. The hadron-induced Drell-Yan
reaction on nuclear targets is an excellent process to investigate the
incoming parton energy loss in cold nuclear matter[16-23]. Therefore, the
Drell-Yan reaction of hadron on nucleus and the semi-inclusive deep inelastic
scattering of lepton on nucleus can provide the essential information on the
energy loss mechanism of fast partons in cold nuclear matter which help to
explore the similar process appearing in relativistic heavy ion collisions.

According to the parton model interpretation, the Drell-Yan process in hadron-
nucleus collisions is closely related to the parton distribution functions in
nuclei. It is well known that nuclear parton distribution functions are
mutually different from those in free nucleons[24]. The nuclear modifications
to the nucleon parton distribution functions are usually referred to as the
nuclear effects on the parton distribution functions. In consideration of the
strong necessity of precise nuclear parton distributions, the global analysis
of nuclear parton distribution functions has been employed for over twenty
years[25]. Several groups have presented their global analysis of the nuclear
parton distribution functions. They are Eskola et al.(EKS98[26], EKPS[27],
EPS09[28] and EPPS16[29]), Hirai et al.(HKM[30], HKN04[31], and HKN07[32]), de
Florian et al.(nDS[33] and DSZS[34]), I. Schienbein et al.[35], K. Kovarik et
al.(nCTEQ15[36]), Marina Walt et al.(TUJU19[37]) and Rabah Abdul Khalek et
al.(nNNPDF1.0[38] and nNNPDF2.0[39]). It is noticeable that HKM, TUJU19,
nNNPDF1.0 and nNNPDF2.0 proposed the nuclear parton distributions that did not
employ the existing experimental data on the fixed-target Drell-Yan process.

As for hadron-nucleus Drell-Yan process, the energy loss of incoming partons
is another nuclear effect apart from the nuclear effects on the parton
distributions as in deep inelastic scattering. In order to investigate the
energy loss of a high energy parton in the cold nuclear matter, Baier et
al.[40,41] (BDMPS hereafter) treated the multiple scattering of the high
energy quark in the nucleus by the Glauber approximation. The medium-induced
transverse momentum broadening and induced gluon radiation spectrum of a high
energy quark traversing a nucleus were studied. BDMPS showed the radiative
energy loss of the parton per unit length, grows as the length of the nuclear
matter, as does the transverse momentum broadening. In the BDMPS formalism,
the quark transport coefficient not only determines the energy loss, but it is
also related to transverse momentum broadening of energetic partons
propagating in the medium. The quark transport coefficient is defined as

$\hat{q}=\frac{4\pi^{2}\alpha_{s}(Q^{2}_{\rm G})C_{F}}{N^{2}_{c}-1}\rho x_{\rm
G}G(x_{\rm G},Q^{2}_{\rm G}),$ (1)

where $C_{F}$ is the quark colour factor, the number of colors $N_{c}=3$,
$\rho$ is the nuclear matter density. The strong coupling constant
$\alpha_{s}$ and the gluon distribution function $G(x_{\rm G},Q^{2}_{\rm G})$
depend on the virtuality $Q^{2}_{\rm G}$. The relative calculation indicates
that the Bjorken variable $x_{\rm G}\ll 1$. The virtuality $Q^{2}_{\rm G}$ is
equal to transverse momentum broadening of energetic partons. The experimental
data from CERN experiment NA10[42] and Fermilab experiment E772[43] show that
the virtuality $Q^{2}_{\rm G}$ is less than 1 GeV2. However, it is important
to keep in mind that the Bjorken variable $x_{\rm G}$ estimated in theory is
an unmeasurable kinematic variable in experiment. The fact limits the
predictive power of BDMPS theory.

In order to improve the predictive power of theory, we explored the relation
between quark transport coefficient and the measurable kinematic variable in
deep inelastic scattering[15] or nuclear Drell-Yan reaction[22]. By means of
the semi-inclusive deep inelastic scattering of lepton on nucleus, we used the
analytic parameterization of quenching weight based on BDMPS formalism[44]
with the target nuclear geometry effect. The hadron multiplicity ratios were
calculated with comparison to the HERMES charged pions production data[45] on
the quarks hadronization occurring outside the nucleus. The relation is
discovered between quark transport coefficient and the measurable kinematic
variables in deep inelastic scattering. The quark transport coefficient is
determined as a function of the Bjorken variable and the photon
virtuality[15]. By means of the proton-nucleus Drell-Yan process, we used the
HKM nuclear parton distributions[30] determined only with lepton-nuclear deep
inelastic scattering experimental data and the analytic parameterization of
quenching weight based on BDMPS formalism. The nuclear Drell-Yan differential
cross section ratios were computed as a function of Feynman variable from
Fermilab E906[46] and E866[47] experimental data. The relation is discovered
between quark transport coefficient and the measurable kinematic variables in
Drell-Yan process. The quark transport coefficient is determined as a function
of the invariant mass of the lepton pair and the momentum fraction of the
partons in target nucleus[22]. In the two published papers[15,22], our
theoretical calculations were performed at leading order QCD approximation.
The atomic mass dependence is not involved of quark transport coefficient.

In fact, a high energy parton encounters the various nucleons in the target
nucleus when it traverses cold nuclear matter. In the definition of quark
transport coefficient, the gluon distribution function $G(x_{\rm G},Q^{2}_{\rm
G})$ should be that of the bound nucleon in the target nucleus. In other word,
the gluon distribution function $G(x_{\rm G},Q^{2}_{\rm G})$ depends on the
atomic mass number. In the present article, the next-to-leading order analysis
are performed of the differential cross section ratios in the nuclear Drell-
Yan process from Fermilab E906[46] and E866[47] experimental data. The nuclear
effects on the parton distribution functions, energy loss effect based on
BDMPS formalism and the target nuclear geometry effect are discussed
respectively. The atomic mass, Bjorken variable and scale dependence of quark
transport coefficient is explored for the first time. It is hoped to gain new
knowledge about the quark transport coefficient in cold nuclear medium.

The remainder of the paper is organized as follows. In Section II, the briefly
formalism for the differential cross section in the nuclear Drell-Yan process
is presented. In Section III, the results and discussion obtained are
presented. Finally, a summary is presented.

##  II Brief formalism for differential cross section in nuclear Drell-Yan
reaction

As for proton-nucleus Drell-Yan process[10], the perturbative QCD leading
order contribution is quark-antiquark annihilation into a lepton pair of mass
$M$. In the collinear factorization approach, the leading order(LO) Drell-Yan
cross section in Feynman variable $x_{\rm F}$ and $M$ is given by

$\frac{d^{2}\sigma^{\rm LO}}{dMdx_{\rm F}}=\frac{8\pi\alpha_{\rm
em}^{2}}{9Ms}\frac{1}{x_{1}+x_{2}}H^{\rm LO}(x_{1},x_{2},Q^{2}),$ (2)

with

$H^{\rm LO}(x_{1},x_{2},Q^{2})=\sum_{f}e^{2}_{f}[q^{\rm
p}_{f}(x_{1},Q^{2})\bar{q}^{\rm A}_{f}(x_{2},Q^{2})+\bar{q}^{\rm
p}_{f}(x_{1},Q^{2})q^{\rm A}_{f}(x_{2},Q^{2})],$ (3)

where $\alpha_{\rm em}$ is the fine structure coupling constant, $\sqrt{s}$ is
the center of mass energy of the hadronic collision, $x_{1}$(respectively
$x_{2}$) is the momentum fraction carried by the projectile (respectively
target) parton, the factorization scale $Q^{2}=M^{2}$, the sum is carried out
over the light flavor $f={\rm u,d,s}$, $q^{\rm p(A)}_{f}(x,Q^{2})$ and
${\bar{q}}^{\rm p(A)}_{f}(x,Q^{2})$ are the quark and antiquark distributions
in the proton (nucleon in the nucleus A).

In the perturbative QCD next-to-leading order(NLO) approximation, additional
emission of a parton (quark or gluon) into the final state has to be taken
into account. The next-to-leading order contribution includes quark-antiquark
annihilation processes ($q+\bar{q}\rightarrow\gamma^{*}+g$) and gluon Compton
scattering ($q+g\rightarrow\gamma^{*}+q$). The NLO correction to the Drell-Yan
cross section is given by

$\frac{d^{2}\sigma^{\rm NLO}}{dMdx_{\rm F}}=\frac{8\pi\alpha_{\rm
em}^{2}}{9Ms}\frac{\alpha_{s}(M^{2})}{2\pi}\int^{1}_{0}dz\frac{1}{x_{1}+x_{2}}H^{\rm
NLO}(x_{1},x_{2},Q^{2},z),$ (4)

with

$\displaystyle H^{\rm NLO}(x_{1},x_{2},Q^{2},z)$ $\displaystyle=$
$\displaystyle\sum_{f}e^{2}_{f}\\{q^{\rm p}_{f}(x_{1},Q^{2})\bar{q}^{\rm
A}_{f}(x_{2},Q^{2})f_{q}(z)$ (5) $\displaystyle+$ $\displaystyle g^{\rm
p}(x_{1},Q^{2})[q^{\rm A}_{f}(x_{2},Q^{2})+\bar{q}^{\rm
A}_{f}(x_{2},Q^{2})]f_{g}(z)+(x_{1}\leftrightarrow x_{2})\\},$

where $g^{\rm p(A)}(x,Q^{2})$ are the gluon distributions in the proton
(nucleon in the nucleus A). By combining the dimensionless variable
$\tau=zx_{1}x_{2}$ and the Feynman variable $x_{\rm F}=x_{1}-x_{2}$, one can
easily find

$x_{1}=\frac{1}{2}(\sqrt{x_{\rm F}^{2}+4(\tau/z)}+x_{\rm F}),\quad
x_{2}=\frac{1}{2}(\sqrt{x_{\rm F}^{2}+4(\tau/z)}-x_{\rm F}).$ (6)

The coefficient functions $f_{q,g}(z)$[48-50] are, in the DIS factorization
scheme,

$\displaystyle f_{q}(z)$ $\displaystyle=$ $\displaystyle
C_{F}[\delta(1-z)(1+\frac{4\pi^{2}}{3})-6-4z+(\frac{3}{1-z})_{+}+2(1+z^{2})(\frac{\ln(1-z)}{1-z})_{+}],$
$\displaystyle f_{g}(z)$ $\displaystyle=$
$\displaystyle\frac{1}{2}[(z^{2}+(1-z)^{2})\ln(1-z)+\frac{3}{2}-5z+\frac{9}{2}z^{2}].$
(7)

The ’plus’ distributions are defined by

$\int^{1}_{0}dxf(x)[g(x)]_{+}=\int^{1}_{0}dx(f(x)-f(1))g(x).$ (8)

Therefore, up to the next to leading order, the differential cross section in
a Drell-Yan reaction can be written as

$\frac{d^{2}\sigma}{dMdx_{\rm F}}=\frac{d^{2}\sigma^{\rm LO}}{dMdx_{\rm
F}}+\frac{d^{2}\sigma^{\rm NLO}}{dMdx_{\rm F}}.$ (9)

In the hadron-nucleus Drell-Yan reaction, the projectile suffers multiple
collisions and repeated energy losses in the nuclear matter. In other words,
each quark or gluon in the beam hardon can lose a finite fraction of its
energy in the nuclear target due to QCD bremsstrahlung. After considering the
parton energy loss in nuclei, the incident parton momentum fraction can be
shifted from $x_{1}+\Delta x_{1}$ to $x_{1}$ at the point of fusion. $\Delta
x_{1}=\Delta E/E_{\rm p}$ with $\Delta E$ ($E_{\rm p}$) the quark energy loss
in the nuclear medium (the projectile hadron energy). After adding the energy
loss of the incoming parton in the target nucleus,

$\displaystyle H^{\rm LO}(x_{1},x_{2},Q^{2})$ $\displaystyle=$
$\displaystyle\int_{0}^{(1-x_{1})E_{\rm p}}d(\Delta E)P_{q}(\Delta
E,\omega_{c},L)H^{\rm LO}(x_{1}+\Delta x_{1},x_{2},Q^{2}),$ $\displaystyle
H^{\rm NLO}(x_{1},x_{2},Q^{2},z)$ $\displaystyle=$
$\displaystyle\int_{0}^{(1-x_{1})E_{\rm p}}d(\Delta
E)\sum_{f}e^{2}_{f}\\{P_{q}(\Delta E,\omega_{c},L)q^{\rm p}_{f}(x_{1}+\Delta
x_{1},Q^{2})\bar{q}^{\rm A}_{f}(x_{2},Q^{2})f_{q}(z)$ (10) $\displaystyle+$
$\displaystyle P_{g}(\Delta E,\omega_{c},L)g^{\rm p}(x_{1}+\Delta
x_{1},Q^{2})[q^{\rm A}_{f}(x_{2},Q^{2})+\bar{q}^{\rm
A}_{f}(x_{2},Q^{2})]f_{g}(z)$ $\displaystyle+$
$\displaystyle(x_{1}\leftrightarrow x_{2})\\},$

where the quenching weight $P_{q,g}(\Delta E,\omega_{c},L)$ are the
probability that the radiated gluons carry altogether a given energy $\Delta
E$ for an incident quark and gluon, respectively. $\omega_{c}$ is the
characteristic gluon frequency, which is equal to $(1/2)\hat{q}L^{2}$ with the
path length $L$ traversed by the incoming parton.

If we do not consider the target nuclear geometry effect, the average path
length of the incident parton in the nucleus is given by $L=(3/4)R_{A}$ for
the case of a hard-sphere nucleus. The nuclear radius $R_{A}=1.12{A^{1/3}}$ fm
with atomic mass number $A$[51]. With adding the target nuclear geometry
effect[14], the colored incoming parton interacting at $y$ the coordinate
along the direction of the incident quark will traverse the path length
$L=\sqrt{R_{A}^{2}-b^{2}}+y$ with $\vec{b}$ its impact parameter. After taking
both nuclear geometry and energy loss effects into account,

$\displaystyle H^{\rm LO}(x_{1},x_{2},Q^{2})$ $\displaystyle=$
$\displaystyle\int d^{2}bdy\rho_{A}(\vec{b},y)\int_{0}^{(1-x_{1})E_{\rm
p}}d(\Delta E)P_{q}(\Delta E,\omega_{c},L)H^{\rm LO}(x_{1}+\Delta
x_{1},x_{2},Q^{2}),$ $\displaystyle H^{\rm NLO}(x_{1},x_{2},Q^{2},z)$
$\displaystyle=$ $\displaystyle\int
d^{2}bdy\rho_{A}(\vec{b},y)\int_{0}^{(1-x_{1})E_{\rm p}}d(\Delta
E)\sum_{f}e^{2}_{f}\\{P_{q}(\Delta E,\omega_{c},L)$ (11) $\displaystyle\times$
$\displaystyle q^{\rm p}_{f}(x_{1}+\Delta x_{1},Q^{2})\bar{q}^{\rm
A}_{f}(x_{2},Q^{2})f_{q}(z)$ $\displaystyle+$ $\displaystyle P_{g}(\Delta
E,\omega_{c},L)g^{\rm p}(x_{1}+\Delta x_{1},Q^{2})[q^{\rm
A}_{f}(x_{2},Q^{2})+\bar{q}^{\rm A}_{f}(x_{2},Q^{2})]f_{g}(z)$
$\displaystyle+$ $\displaystyle(x_{1}\leftrightarrow x_{2})\\},$

where
$\rho_{A}(\sqrt{b^{2}+y^{2}})=(\rho_{0}/A)\Theta(R_{A}-\sqrt{b^{2}+y^{2}})$
with $\rho_{0}$ the nuclear density.

##  III Results and discussion

In the present analysis, the experimental data are taken from E906[46] and
E866[47] Collaborations at Fermilab. E866 Collaboration reported the
observation of the ratios of the Drell-Yan cross section per nucleon for an
800 GeV proton beam incident on Be, Fe and W targets. The Drell-Yan events
were recorded in the range $4.0<M<8.4$ GeV, $0.01<x_{2}<0.12$,
$0.21<x_{1}<0.95$ and $0.13<x_{\rm F}<0.93$. E906 Collaboration performed the
observation of the ratios of the Drell-Yan cross section per nucleon for an
120 GeV proton beam incident on C, Fe and W targets. Muon pairs were recorded
in the range in the range $4.5<M<5.5$ GeV and $0.1<x_{2}<0.3$. To be
emphasized, E866 and E906 data cover the momentum fraction of the target
parton from 0.01 to 0.12, and from 0.1 to 0.3, respectively. The fact makes
E866 and E906 data show possibly the measurable kinematic variables dependence
of the quark transport coefficient in the cold nuclear medium.

In order to study the property of the quark transport coefficient, we
calculate the Drell-Yan cross section ratio of two different nuclear targets
bombarded by proton,

$R_{A_{1}/A_{2}}(x_{\rm F})=\int\frac{d^{2}\sigma^{{\rm p}A_{1}}}{dMdx_{\rm
F}}dM\Bigg{/}\int\frac{d^{2}\sigma^{{\rm p}A_{2}}}{dMdx_{\rm F}}dM,$ (12)

in perturbative QCD next-to-leading order approximation. The comparison is
performed with selected experimental data. The integral range in above
equation is given by means of the relative experimental kinematic region.

In order to research the nuclear effects from parton distribution functions on
the Drell-Yan differential cross section ratio, $\chi^{2}$ is calculated with
the Drell-Yan differential cross section rations $R_{A_{1}/A_{2}}$ as

$\chi^{2}=\sum_{j}\frac{(R^{\rm data}_{A_{1}/A_{2},j}-R^{\rm
theo}_{A_{1}/A_{2},j})^{2}}{(R^{\rm err}_{A_{1}/A_{2},j})^{2}},$ (13)

where the experimental error is given by $R^{\rm err}_{A_{1}/A_{2},j}$, and
$R^{\rm data}_{A_{1}/A_{2},j}(R^{\rm theo}_{A_{1}/A_{2},j})$ indicates the
experimental data (theoretical) value for the Drell-Yan differential cross
section ratio $R_{A_{1}/A_{2}}$ as a function of Feynman variable. We employ
the nNNPDF2.0 nuclear parton distribution functions from the NNPDF
collaboration[39] in our calculations without the consideration of quark
energy loss effect. The calculated values of $\chi^{2}$ per number of degrees
of freedom are $0.432$, $3.174$, $6.626$ and $7.819$ for the Drell-Yan
differential cross section ratio Fe/Be, W/Be, Fe/C and W/C, respectively. Our
obtained values of $\chi^{2}/{\rm ndf}$ are 1.803 and 7.222 for the E866 and
E906 experimental data, respectively. In total, the value of $\chi^{2}/{\rm
ndf}$ is given by 4.126 for the E906 and E866 experimental data(see TABLE I).
The theoretical results in the global fit are compared with the measured
Drell-Yan differential cross-section ratios from E906 and E866 collaboration
in Fig.1(black solid curves). Obviously, the calculated results with only
considering the nuclear effects of parton distribution do not agree with the
E866 and E906 experimental data. In addition, the main contribution of
$\chi^{2}/{\rm ndf}$ is from the E906 experimental data in the global fit. It
is found that the value of $\chi^{2}/{\rm ndf}$ by the cross-section ratio
Fe/Be is remarkably lower than those from W/Be, Fe/C and W/C. It is shown that
the theoretical prediction is in good agreement with the experimental data on
the cross section ratio Fe/Be. We think that the theoretical result
overestimates the nuclear effects on parton distribution functions in the
Drell-Yan differential cross section ratio Fe/Be. Therefore, it is necessary
to refine the nuclear parton distribution in the future. The accurate nuclear
parton distribution can highlight the potential of the fixed-target Drell-Yan
measurements to probe the parton energy loss mechanism in a robust manner.

Table 1: The values of $\hat{q}_{0}$ and $\chi^{2}/{\rm ndf}$ extracted from fitting the selected experimental data on the Drell-Yan differential cross section ratio from the E906 and E866 experiments by considering the nuclear effects on parton distribution functions(NEPDF) and adding energy loss(EL) effect without and with the nuclear geometry effect. | E866 | E906
---|---|---
| Fe/Be | W/Be | Fe/C | W/C
| $\chi^{2}/{\rm ndf}=0.432$ | $\chi^{2}/{\rm ndf}=3.174$ | $\chi^{2}/{\rm ndf}=6.626$ | $\chi^{2}/{\rm ndf}=7.819$
NEPDF | $\chi^{2}/{\rm ndf}=1.803$ | $\chi^{2}/{\rm ndf}=7.222$
| $\chi^{2}/{\rm ndf}=4.126$
| $\hat{q}_{0}=0$ | $\hat{q}_{0}=1.005\pm 0.234$ | $\hat{q}_{0}=0.718\pm 0.061$ | $\hat{q}_{0}=0.331\pm 0.025$
EL | $\chi^{2}/{\rm ndf}=0.432$ | $\chi^{2}/{\rm ndf}=0.552$ | $\chi^{2}/{\rm ndf}=1.567$ | $\chi^{2}/{\rm ndf}=0.689$
$(L=3/4R_{A})$ | $\hat{q}_{0}=1.028\pm 0.219$ | $\hat{q}_{0}=0.346\pm 0.022$
| $\chi^{2}/{\rm ndf}=0.674$ | $\chi^{2}/{\rm ndf}=2.201$
| $\hat{q}_{0}=0.343\pm 0.022\quad\quad\chi^{2}/{\rm ndf}=1.562$
EL($L=\sqrt{R_{A}^{2}-b^{2}}+y$) | $\hat{q}_{0}=0.277\pm 0.001\quad\quad\chi^{2}/{\rm ndf}=1.334$
Figure 1: Ratios of the differential cross section per nucleon for Drell-Yan
reaction versus Feynman variable $x_{\rm F}$. The black solid curves are the
predicted cross-section ratios in the global fitting E906 and E866
experimental data without energy loss effect. The blue dashed curves are the
predicted results from the fitting respectively E906 and E866 measurements
without the target nuclear geometry effect. As for the global fitting E906 and
E866 measurements, the red dotted(green dot-dashed) curves correspond to the
theoretical predictions without(with) target nuclear geometry effect. The
experimental data are taken from E906 and E866 Collaboration.

On the basis of the nuclear effects on parton distribution functions, the
energy loss effect of the incoming parton is added in the following. The
analytic expressions[44] for the quark and gluon quenching weight can be given
by using the BDMPS gluon radiation spectrum for an incoming parton in QCD
media[52]. The simple analytic expressions can easily be used to compute the
Drell-Yan differential cross section. By means of the CERN subroutine
MINUIT[53], the undetermined parameters in the quark transport coefficient
$\hat{q}$ can be obtained by minimizing $\chi^{2}$.

We first make the quark transport coefficient $\hat{q}$ as a constant
$\hat{q}_{0}$ without its kinematic variable dependence. If we neglect the
target nuclear geometry effect, the incoming hard parton will travel an
average path length $L=(3/4)R_{A}$ in the target nuclear medium. Consequently,
only one parameter $\hat{q}_{0}$ needs to be pinned down. The values of
$\hat{q}_{0}$ and $\chi^{2}/{\rm ndf}$ extracted from fitting the selected
experimental data on the Drell-Yan differential cross section ratio are listed
in TABLE I. The calculated values of $\hat{q}_{0}$($\chi^{2}/{\rm ndf}$) are
$0.0$ GeV2/fm $(0.432)$, $1.005\pm 0.234$ GeV2/fm $(0.552)$, $0.718\pm 0.061$
GeV2/fm $(1.567)$ and $0.331\pm 0.025$ GeV2/fm $(0.689)$ for the Drell-Yan
differential cross section ratio Fe/Be, W/Be, Fe/C and W/C, respectively. The
obtained values of $\hat{q}_{0}$($\chi^{2}/{\rm ndf}$) are $1.028\pm 0.219$
GeV2/fm $(0.674)$ and $0.346\pm 0.022$ GeV2/fm $(2.201)$ for the E866 and E906
experimental data, respectively. In view of the measured different $x_{2}$
range in the E906 and E866 experiment, we can came obviously to the conclusion
that the quark transport coefficient $\hat{q}$ depends on the momentum
fraction of the target parton. However, it is a great pity that the global
fitting the E906 and E866 experimental data gives $\hat{q}_{0}=0.343\pm 0.022$
GeV2/fm with $\chi^{2}/{\rm ndf}=1.562$. Therefore, it is worth intensively
looking into the kinematic variable dependence of the quark transport
coefficient. Just to confirm this conclusion, we need more accurate
experimental data on the Drell-Yan differential cross section ratio. We
strongly desire that Fermilab Experiment 906 collaboration can refine their
experimental data, and report their precise measurement of the ratios of the
Drell-Yan cross section per nucleon in the future published paper.

To demonstrate intuitively the energy loss effect of an incoming parton on the
Drell-Yan cross section ratio, the calculated results combining nNNPDF2.0
nuclear parton distribution functions are compared with the selected
experimental data in Fig.1. The blue dashed curves are the predicted cross-
section ratios for the case of fitting respectively E906 and E866 measurements
without the target nuclear geometry effect. As for the global fitting E906 and
E866 measurements, the red dotted curves are the theoretical predictions
without target nuclear geometry effect. In particular is worth mentioning that
the Drell-Yan cross section ratio of Fe and C nuclear targets bombarded by
proton. As shown in TABLE I and Fig.1, the agreement is bad between the
experimental data and the theoretical results from fitting the E906
measurement and only the cross section ratio Fe/C. The fact reminds us that
the precise measurement of the ratios of the Drell-Yan cross section per
nucleon and accurate nuclear parton distribution functions are indispensable
for knowing deeply the energy loss mechanism in the cold nuclear medium.

Furthermore, we explore the energy loss with the target nuclear geometry
effect on the Drell-Yan cross section ratio. In this case, the path length
$L=\sqrt{R_{\rm A}^{2}-b^{2}}+y$ for the incident parton traversing the
nucleus. Our calculated result indicates that $\hat{q}_{0}=0.277\pm 0.001$
GeV2/fm with $\chi^{2}/{\rm ndf}=1.334$ from the global fitting the E906 and
E866 experimental data. The comparison of our theorized expectations (green
dot-dashed curves) is presented with the experimental measurements in Fig.1.
From TABLE I, it is found that the target nuclear geometry effect could reduce
the quark transport coefficient by as much as $19.24\%$. Therefore, we should
consider the target nuclear geometry effect over the course of studying the
nuclear Drell-Yan reaction.

Now, let us investigate the kinematic variable dependence of the quark
transport coefficient. In our preceding article[22], we discovered the
relation between the Bjorken variable $x_{\rm G}$ in the quark transport
coefficient and the momentum fraction of the target parton in Drell-Yan
process. In other words, the Bjorken variable $x_{\rm G}$ can be replaced with
the momentum fraction $x_{2}$. Consequently, we can deduce that the virtuality
$Q^{2}_{\rm G}$ should also be expressed with the measurable scale $Q^{2}$,
which is the square of the lepton pair mass $M$. By means of fitting the
experimental data on the Drell-Yan cross section ratio, the quark transport
coefficient has been determined as a function of the Bjorken variable $x_{2}$
and scale $Q^{2}$.

As regarding the gluon density $x_{\rm G}G(x_{\rm G},Q^{2}_{\rm G})$ from the
quark transport coefficient $\hat{q}$, $x_{\rm G}\ll 1$, and $Q^{2}_{\rm G}<1$
GeV2[22]. Phenomenological work[54] provided a model based on the concept of
saturation for small $Q^{2}_{\rm G}$ and small $x_{\rm G}$. A good description
of data on the proton structure function was presented by means of the proton
saturation scale. According to the saturation model, the gluon density

$x_{\rm G}G(x_{\rm G},Q^{2}_{\rm G})\sim x_{\rm G}^{-\lambda}.$ (14)

With using the Bjorken variable $x_{2}$ instead of $x_{\rm G}$,

$x_{\rm G}G(x_{\rm G},Q^{2}_{\rm G})\sim x_{2}^{-\lambda}.$ (15)

Hence, the Bjorken variable $x_{2}$ dependence of the quark transport
coefficient can be written as

$\hat{q}=\hat{q_{0}}x_{2}^{\alpha}.$ (16)

Furthermore, adding the intermediate and large $x_{2}$ correction with the
evolution of gluon distribution with $Q^{2}$, the quark transport coefficient
can be supposed to be

$\hat{q}(x_{2},Q^{2})=\hat{q_{0}}\alpha_{\rm
s}(Q^{2})x_{2}^{\alpha}(1-x_{2})^{\beta}\ln^{\gamma}(Q^{2}/Q^{2}_{0}),$ (17)

where $Q^{2}_{0}=1$ GeV2 in order to make the argument in the logarithm
dimensionless. Four undetermined parameters are $\hat{q_{0}}$, $\alpha$,
$\beta$ and $\gamma$.

In fact, the gluon density $x_{\rm G}G(x_{\rm G},Q^{2}_{\rm G})$ in the quark
transport coefficient should be that of the bound nucleon in the target
nucleus. The gluon density $x_{\rm G}G(x_{\rm G},Q^{2}_{\rm G})$ depends on
the atomic mass number in the fixed-target experiment. The saturation scale of
a nucleus[55,56] is enhanced relative to the nucleon one by a factor
$A^{1/3}$. With the atomic mass number $A$ and $x_{2}$ dependence of the
saturation scale in the nucleus, the gluon density

$x_{\rm G}G(x_{\rm G},Q^{2}_{\rm G})\sim A^{1/3}x_{\rm G}^{-\lambda}\sim
A^{1/3}x_{2}^{-\lambda}.$ (18)

Therefore, we can give the atomic mass, Bjorken variable and scale dependence
of the quark transport coefficient,

$\hat{q}(A,x_{2},Q^{2})=\hat{q_{0}}A^{1/3}\alpha_{\rm
s}(Q^{2})x_{2}^{\alpha}(1-x_{2})^{\beta}\ln^{\gamma}(Q^{2}/Q^{2}_{0}).$ (19)

Thus, there are still four parameters to fit: $\hat{q_{0}}$, $\alpha$, $\beta$
and $\gamma$.

We perform the global analysis of the quark transport coefficient by comparing
with the experimental data from the E906 and E866 collaboration. The parameter
values are pinned down in the quark transport coefficient.

Table 2: The parameter values of $\hat{q}$ and $\chi^{2}/{\rm ndf}$ extracted by the global analysis of E866 and E906 experimental data. $\hat{q}$(GeV2/fm) | $\hat{q_{0}}$ | $\alpha$ | $\beta$ | $\gamma$ | $\chi^{2}/{\rm ndf}$
---|---|---|---|---|---
$\hat{q}(x_{2},Q^{2})$ (without geometry effect) | $0.424\pm 0.022$ | $-0.083\pm 0.026$ | $3.879\pm 0.383$ | $1.426\pm 0.045$ | $1.561$
$\hat{q}(x_{2},Q^{2})$ (with geometry effect) | $0.393\pm 0.018$ | $-0.106\pm 0.006$ | $4.441\pm 0.052$ | $1.452\pm 0.021$ | $1.218$
$\hat{q}(A,x_{2},Q^{2})$ (without geometry effect) | $0.076\pm 0.005$ | $-0.242\pm 0.028$ | $4.399\pm 0.452$ | $1.174\pm 0.051$ | $1.524$
$\hat{q}(A,x_{2},Q^{2})$ (with geometry effect) | $0.061\pm 0.004$ | $-0.138\pm 0.006$ | $4.557\pm 0.073$ | $1.448\pm 0.033$ | $1.352$
Figure 2: The calculated Drell-Yan cross section ratios $R_{\rm A/B}$ versus
Feynman variable $x_{\rm F}$. The black solid(red dashed) curves are the
predicted cross-section ratios from the quark transport coefficient
$\hat{q}(x_{2},Q^{2})$ without(with) target nuclear geometry effect. The blue
dotted(green dot-dashed) curves correspond to the theoretical predictions for
the quark transport coefficient $\hat{q}(A,x_{2},Q^{2})$ without(with) target
nuclear geometry effect. The experimental data are taken from E906 and E866
Collaboration.

As for the quark transport coefficient without atomic mass dependence and
target nuclear geometry effect, the extracted parameter values are
$\hat{q}_{0}=0.424\pm 0.022$ GeV2/fm with the relative uncertainty
$\delta\hat{q}_{0}/\hat{q}_{0}\simeq 5.2\%$, $\alpha=-0.083\pm 0.026$ with
$\delta\alpha/\alpha\simeq 31.3\%$, $\beta=3.879\pm 0.383$ with
$\delta\beta/\beta\simeq 9.9\%$, and $\gamma=1.426\pm 0.045$ with
$\delta\gamma/\gamma\simeq 3.2\%$ and $\chi^{2}/{\rm ndf}=1.561$. In regard to
the quark transport coefficient with no atomic mass dependence and target
nuclear geometry effect, the obtained parameter values are
$\hat{q}_{0}=0.393\pm 0.018$ GeV2/fm with $\delta\hat{q}_{0}/\hat{q}_{0}\simeq
4.5\%$, $\alpha=-0.106\pm 0.006$ with $\delta\alpha/\alpha\simeq 5.6\%$,
$\beta=4.441\pm 0.052$ with $\delta\beta/\beta\simeq 1.1\%$, and
$\gamma=1.452\pm 0.021$ with $\delta\gamma/\gamma\simeq 1.4\%$ and
$\chi^{2}/{\rm ndf}=1.218$. As regards the quark transport coefficient with
atomic mass dependence and no target nuclear geometry effect, the theoretical
parameter values are $\hat{q}_{0}=0.076\pm 0.005$ GeV2/fm with
$\delta\hat{q}_{0}/\hat{q}_{0}\simeq 6.6\%$, $\alpha=-0.242\pm 0.028$ with
$\delta\alpha/\alpha\simeq 11.6\%$, $\beta=4.399\pm 0.452$ with
$\delta\beta/\beta\simeq 10.2\%$, and $\gamma=1.174\pm 0.051$ with
$\delta\gamma/\gamma\simeq 4.3\%$ and $\chi^{2}/{\rm ndf}=1.524$. In the case
of the quark transport coefficient with atomic mass dependence and target
nuclear geometry effect, the determined parameter values are
$\hat{q}_{0}=0.061\pm 0.004$ GeV2/fm with $\delta\hat{q}_{0}/\hat{q}_{0}\simeq
6.5\%$, $\alpha=-0.138\pm 0.006$ with $\delta\alpha/\alpha\simeq 4.3\%$,
$\beta=4.557\pm 0.073$ with $\delta\beta/\beta\simeq 1.6\%$, and
$\gamma=1.448\pm 0.033$ with $\delta\gamma/\gamma\simeq 2.2\%$ and
$\chi^{2}/{\rm ndf}=1.352$. The comparison of our theorized results is
presented with the experimental measurements in Fig.2. In Table II, we
summarize the extracted parameter values of the quark transport coefficient.
From Table II, it is shown that the atomic mass dependence reduces the
constant factor $\hat{q}_{0}$ in the quark transport coefficient by
approximately $82.1\%$ without the target nuclear geometry effect. After
incorporating the target nuclear geometry effect, the atomic mass dependence
reduces the constant factor $\hat{q}_{0}$ in the quark transport coefficient
by as much as $84.5\%$. Therefore, the atomic mass dependence has a
significant impact on the constant factor $\hat{q}_{0}$ in the quark transport
coefficient in cold nuclear matter.

Two other research groups have explored the transport coefficient in cold
nuclear matter. François Arleo et al.[21, 57] studied the transport
coefficient parameterization as a function of Bjorken variable without the
virtuality $Q^{2}$ evolution. The coefficient $\hat{q}_{0}$ is the only
parameter of their model. The value of $\hat{q}_{0}$ is given by
$0.07-0.09$GeV2/fm. Peng Ru et al.[58] performed the global analysis of the
jet transport coefficient in the framework of the generalized QCD
factorization formalism. The jet transport coefficient has the Bjorken-$x$ and
scale $Q^{2}$ dependent parametrization. The coefficient $\hat{q}_{0}$ is
$0.0195^{+0.0085}_{-0.0065}$GeV2/fm. Therefore, it is hoped that the
theoretical progress should be made in the future. With the precision
experimental data on the semi-inclusive deep-inelastic scattering and nuclear
Drell-Yan reaction, a quantitative understanding of the parton energy loss
mechanism will be thus achieved in cold nuclear matter.

##  IV Concluding remarks

We study the Drell-Yan process in proton-nucleus collisions, supplementing the
perturbative QCD factorized formalism with radiative parton energy loss. By
means of the nuclear parton distributions determined without the fixed-target
Drell-Yan process experimental data and the analytic expression of quenching
weight based on BDMPS formalism, a next-to-leading order analyses on the
differential cross section ratios from the nuclear Drell-Yan process are
performed, and compared with the experimental data from Fermilab E906 and E866
Collaborations. It is found that the calculated results with only considering
the nuclear effects of parton distribution do not agree with the E866 and E906
experimental data. The incoming quark energy loss effect is indispensable in
nuclear Drell-Yan process. With the quark transport coefficient as a constant
and no target nuclear geometry effect, the global fitting the E906 and E866
experimental data gives $\hat{q}_{0}=0.343\pm 0.022$ GeV2/fm. With the target
nuclear geometry effect, $\hat{q}_{0}=0.277\pm 0.001$ GeV2/fm. The suppression
due to the target nuclear geometry effect is approximately $19.24\%$ for the
quark transport coefficient. It is shown that we should consider the nuclear
geometry effect over the course of studying the Drell-Yan reaction on nuclear
targets. On the basis of Bjorken variable and scale dependence of the quark
transport coefficient, the atomic mass dependence is combined. The quark
transport coefficient is determined as a function of the atomic mass, Bjorken
variable $x_{2}$ and scale $Q^{2}$ by the global fit of the experimental data.
The constant factor $\hat{q}_{0}$ of the quark transport coefficient are
respectively $0.393\pm 0.018$ GeV2/fm and $0.061\pm 0.004$ GeV2/fm without and
with the atomic mass dependence. The suppression due to atomic mass dependence
is approximately $84.5\%$ for the constant factor $\hat{q}_{0}$ in the quark
transport coefficient. It is found that the atomic mass dependence has a
remarkable effect on the constant factor $\hat{q}_{0}$ in the quark transport
coefficient in cold nuclear matter.

It is well known that there are the quark energy loss effect and the nuclear
effects on the parton distributions in the nuclear Drell-Yan process. The
accurate nuclear parton distribution functions are crucial for studying deeply
the energy loss mechanism in the cold nuclear medium. The proton incident
nuclei Drell-Yan data can result in an overestimate for nuclear modification
of the sea quark distribution function if leaving the quark energy loss effect
out. Therefore, we suggest that the new global analysis of nuclear parton
distribution functions should not employ the available experimental data on
the fixed-target hadron-nucleus Drell-Yan process.

The precise measurement of the ratios of the proton-nucleus Drell-Yan cross
section per nucleon is essential for clarifying the energy loss mechanism in
the cold nuclear medium. Our results strongly suggest that Fermilab Experiment
906 collaboration can refine their experimental data, and report their precise
measurement of the ratios of the Drell-Yan cross section per nucleon in the
future published paper.

The investigation into the energy loss mechanism in cold nuclear matter helps
to understand the similar process appearing in relativistic heavy ion
collisions. The hadron-induced Drell-Yan reaction on nuclei is an excellent
process to explore the incoming parton energy loss in cold nuclear matter. We
expect that our determined parametrization of the quark transport coefficient
provides a useful information on the identification of the transport property
of the quark-gluon plasma. It is desirable to operate precise measurements in
the nuclear Drell-Yan process from J-PARC experiment[59] and LHCb-SMOG
experiment[60]. These new experimental data should shed light on the parton
propagation mechanism in nuclear matter.

Acknowledgments This work is supported in part by the National Natural Science
Foundation of China(NSFC) under Grants No.11975090 and No.11575052.

## References

  * (1) J. D. Bjorken, Phys. Rev. D 27, 140 (1983).
  * (2) D. d’Enterria, Jet quenching: Datasheet from Landolt-Börnstein - Group I Elementary Particles, Nuclei and Atoms, Volume 23: ”Relativistic Heavy Ion Physics” in SpringerMaterials, Springer (2010). arXiv:0902.2011.
  * (3) J. Casalderrey-Solana, C. A. Salgado, Acta Phys. Polon. B 38, 3731 (2007). arXiv:0712.3443.
  * (4) U. A. Wiedemann, Jet quenching in heavy ion collisions: Datasheet from Landolt-Börnstein - Group I Elementary Particles, Nuclei and Atoms, Volume 23: ”Relativistic Heavy Ion Physics” in SpringerMaterials, Springer (2010). arXiv:0908.2306.
  * (5) A. Majumder, M. Van Leeuwen, Prog. Part. Nucl. Phys. 66, 41 (2011). arXiv:1002.2206.
  * (6) S. Cao, X. Wang, Rept. Prog. Phys. 84, 024301 (2021). arXiv: 2002.04028.
  * (7) S. Cao et al., Phys. Rev. C 104, 024905 (2021). arXiv: 2102.11337.
  * (8) D. Everett et al., Phys. Rev. C 103, 054904 (2021). arXiv: 2011.01430.
  * (9) A. Kumar et al., Phys. Rev. C 101, 034908 (2020). arXiv:1909.03178
  * (10) S. Drell, T.M. Yan, Phys. Rev. Lett. 25, 316 (1970).
  * (11) L. H. Song, C. G. Duan, Phys. Rev. C 81, 035207 (2010). arXiv:1109.3836.
  * (12) L. H. Song, N. Liu, C. G. Duan, Chin. Phys. C 37, 104102 (2013). arXiv:1310.5285.
  * (13) L. H. Song, N. Liu, C. G. Duan, Chin. Phys. C 37, 084102 (2013). arXiv:1310.5692.
  * (14) N. Liu, et al., Phys. Lett. B 749, 88 (2015). arXiv:1511.00767.
  * (15) T. X. Bai, C. G. Duan, Eur. Phys. J. Plus 136, 1181 (2021). arXiv: 2011.14350.
  * (16) L. H. Song, C.G. Duan, N. Liu, Phys. Lett. B 708, 68 (2012). arXiv:1206.3815.
  * (17) C. G. Duan et al., Eur. Phys. J. C 50, 585 (2007). arXiv:hep-ph/0609057.
  * (18) C. G. Duan et al., Eur. Phys. J. C 29, 557 (2003). arXiv:hep-ph/0405113.
  * (19) C. G. Duan et al., Eur. Phys. J. C 39, 179 (2005). arXiv:hep-ph/0601188.
  * (20) C. G. Duan et al., Phys. Rev. C 79, 048201 (2009). arXiv:0811.0675.
  * (21) F. Arleo et al., JHEP 01, 129 (2019). arXiv:1810.05120.
  * (22) T. X. Bai, C. G. Duan, Eur. Phys. J. Plus 136, 649 (2021). arXiv: 2012.03167.
  * (23) A. Accardi, et al., Riv. Nuovo. Cim., 32, 439 (2010). arXiv:0907.3534.
  * (24) J. J. Aubert et al., Phys. Lett. B 123, 275 (1983).
  * (25) J. J. Ethier, E. R. Nocera, Ann. Rev. Nucl. Part. Sci. 70, 43 (2020). arXiv: 2001.07722.
  * (26) K. J. Eskola, V. J. Kolhinen, P. V. Ruuskanen, Nucl. Phys.B 535, 351 (1998). arXiv: hep-ph/9802350.
  * (27) K. J. Eskola et al., JHEP 05, 002 (2007). arXiv: hep-ph/0703104.
  * (28) K. J. Eskola, H. Paukkunen, C. A. Salgado, JHEP 04, 065 (2009). arXiv: 0902.4154.
  * (29) K. J. Eskola et al., Eur. Phys. J. C 77, 163 (2017). arXiv: 1612.05741.
  * (30) M. Hirai, S. Kumano, M. Miyama, Phys. Rev. D 64, 034003 (2001). arXiv: hep-ph/0103208.
  * (31) M. Hirai, S. Kumano, T. H. Nagai, Phys. Rev. C 70, 044905 (2004). arXiv: hep-ph/0404093.
  * (32) M. Hirai, S. Kumano, T. H. Nagai, Phys. Rev. C 76, 065207 (2007). arXiv: 0709.3038.
  * (33) D. de Florian, R. Sassot, Phys. Rev. D 69, 074028 (2004). arXiv: hep-ph/0311227.
  * (34) D. de Florian et al., Phys. Rev. D 85, 074028 (2012). arXiv: 1112.6324.
  * (35) I. Schienbein et al., Phys. Rev. D 80, 094004 (2009). arXiv: 0907.2357.
  * (36) K. Kovarik et al., Phys. Rev. D 93, 085037 (2016). arXiv: 1509.00792.
  * (37) M. Walt, I. Helenius, W. Vogelsang, Phys. Rev. D 100, 096015 (2019). arXiv: 1908.03355.
  * (38) R. A. Khalek, J. J. Ethier, J. Rojo, Eur. Phys. J. C 79, 471 (2019). arXiv: 1904.00018.
  * (39) R. A. Khalek et al., JHEP 09, 183 (2020). arXiv: 2006.14629.
  * (40) R. Baier et al., Nucl. Phys. B 484, 265 (1997). arXiv: hep-ph/9608322.
  * (41) R. Baier et al., Annu. Rev. Nucl. Part. Sci. 50, 37 (2000). arXiv: hep-ph/0002198.
  * (42) P. Bordalo et al., Phys. Lett. B 193, 373 (1987).
  * (43) D. M. Alde et al., Phys. Rev. Lett. 66, 2285 (1991).
  * (44) F. Arleo, JHEP 11, 044 (2002). arXiv: hep-ph/0210104.
  * (45) A. Airapetian, et al., HERMES Collaboration, Eur. Phys. J. A 47, 113 (2011). arXiv: 1107.3496.
  * (46) P. J. Lin, Ph.D. Thesis, Colorado University (2017).
  * (47) M. A. Vasiliev et al., Phys. Rev. Lett. 83, 2304 (1999). arXiv:hep-ex/9906010.
  * (48) G. Altarelli, R. K. Ellis, G. Martinelli, Nucl. Phys. B 143, 521 (1978).
  * (49) G. Altarelli, R. K. Ellis, G. Martinelli, Nucl. Phys. B 157, 461 (1979).
  * (50) G. Altarelli, R. K. Ellis, G. Martinelli, Nucl. Phys. B 146, 544(erratum) (1978).
  * (51) G. T. Garvey, J. C. Peng, Phys. Rev. Lett. 90, 092302 (2003). arXiv: hep-ph/0208145.
  * (52) R. Baier et al., Nucl. Phys. B 531, 403 (1998). arXiv: hep-ph/9804212.
  * (53) F. James, CERN Program Library Long Writeup D506.
  * (54) K. Golec-Biernat, M. Wusthoff, Phys. Rev. D 59, 014017 (1998). arXiv: hep-ph/9807513.
  * (55) H. Kowalski, T. Lappi, R. Venugopalan, Phys. Rev. Lett. 100, 022303 (2008). arXiv: 0705.3047.
  * (56) J. L. Albacete, C. Marquetb, Prog. Part. Nucl. Phys. 76, 1 (2014). arXiv: 1401.4866.
  * (57) F. Arleo, C. Naïm, JHEP 07, 220 (2020). arXiv:2004.07188.
  * (58) P. Ru et al., Phys. Rev. D 103, L031901 (2021). arXiv:1907.11808.
  * (59) S. Kumano, AIP Conf. Proc. 1056, 444 (2008). arXiv: 0807.4207.
  * (60) A. Bursche, et al., Technical Report LHCb-PUB-2018-015, CERN-LHCb-PUB-2018-015(2018).


* Kevin Murphy: Advisor on project, including its direction and the paper narrative.

* Mark Collier: Responsible for most experimental results involving heteroscedastic model e.g. Het upstream, Het$\to$Het, None$\to$ Het, BE$\to$BE$+$Het. Primary author of section <Ref>. Implemented CIFAR-10H and ImageNet ReaL-H label uncertainty tasks. Primary author of <Ref>.

Ran sensitivity analysis (together with Mike) to the number of JFT pretraining epochs and volume of pretraining data (JFT4B).

* Mike Dusenberry: Engineering lead. Designed, built, and owned infrastructure for Plex, including reproducible training scripts, utilities, experiment scaling components, and open-sourcing efforts. Trained and tuned None ViT baselines (e.g., L) to match existing results. Scaled up experiments to JFT-4B. Ablated JFT dataset size and number of epochs with Mark Collier. Ran experiments and wrote <Ref> for generalization to covariate distribution shifts. Advised and worked on infra and experiments for the active learning section.

Co-wrote a compute workflow tutorial for collaborators. Created a demo notebook for ViT-Plex. Open-sourced code and model checkpoints.

* Neil Band: Co-led design of experiments and narrative for integration of RETINA experiments, including selective prediction and distribution generalization. Led RETINA implementation and experiments. Co-wrote section on selective prediction and compute workflow tutorial for collaborators. Led open-sourcing of RETINA distribution shift tasks, baseline model checkpoints, evaluation, and plotting utilities.

* Nithum Thain: Contributed to experiments analyzing Deep Ensembles and their tradeoff with compute.

* Rodolphe Jenatton: Developed the code to evaluate deep ensembles in <https://github.com/google/uncertainty-baselines> and ran associated experiments (together with Nithum Thain). Developed the code to evaluate models and ensembles in <https://github.com/google-research/robustness_metrics>. Developed the code to wrap the -based <https://github.com/google-research/vmoe> in <https://github.com/google/uncertainty-baselines>, enabling the evaluation of sparse MoE models, namely E$^3$, V-MoE and ensembles thereof—ran corresponding experiments. Contributed to the shaping of the narrative of the ensembling section (together with Zelda Mariet). Helped with the computation of the FLOPs. Improved finetuning numbers by suggesting use of longer finetuning schedule.

* Tim G. J. Rudner: Co-led design of experiments and narrative for integration of RETINA experiments, including selective prediction and distribution generalization. Contributed to all RETINA experiments.
Led writing of section on selective prediction and contributed to writing of section on calibration AUROC.

* Yarin Gal: Advisor on project, including its direction and the paper narrative.

* Zachary Nado: Designed and contributed to core infrastructure that was the major platform to iterate on our models, tasks, and other results.

* Zelda Mariet: Ran BatchEnsemble experiments, ran ensemble diversity analysis, drew up vision for the ensembling analysis <Ref>, proposed and ran the upstream vs downstream comparison (<Ref>). Wrote the code for experimental analysis, hyperparameter tuning, visualization and summarization. Code refactoring for the overall project.

* Zi Wang: Led the
active learning effort including initial exploration as an engagement with internal teams as a close collaboration between Google Reliable Deep Learning and the Oxford OATML group (with Mike, Andreas, Joost, and Dustin). Contributed to refactoring BE and None to facilitate better integration with AL. Contributed to active learning code (debugging, adding new features etc). Extensively experimented with AL on RDL models, conducted analyses and obtained/wrote final results. Added discussions on the relations between prior learning and pretraining.

* Zoubin Ghahramani: Advisor on project, including its direction and the paper narrative.

§ DETAILS BEHIND KEY FIGURES

For the task radar plots of <Ref>, we list references used for task-specific state-of-the-art in <Ref>.

Task     Task-specific SOTA

MNLI     T5 11B [Raffel et al., 2020]

MNLI Mismatched (Subpop Shift)     T5 11B [Raffel et al., 2020]

MNLI Hans All (OOD)     RoBERTa base [Tu et al., 2020]

WikipediaTalk     Deep Ensembles [Kivlichan et al., 2021]

OOD Places 365     [Hendrycks et al., 2019]

AL Accuracy CIFAR10     [Yi et al., 2022]

AL Accuracy CIFAR100     [Citovsky et al., 2021]

AL Accuracy ImageNet     [Emam et al., 2021]

Accuracy RETINA     [Band et al., 2021]

Selective Prediction RETINA     [Band et al., 2021]

References for the task-specific state-of-the-art used in <Ref>.

For the model scaling plot of <Ref>, we aggregate all task metrics under a single scalar between 0 and 100. In order to do this, we normalize all metrics to be between 0 and 100; we then compute an unweighted average. Most metrics are already bounded between 0 and 100: for example, accuracy, expected calibration error (we do $100-\text{ECE}$ so higher is better), Calibration AUROC, and OOD AUROC. The one exception are scoring rules such as log-loss and Brier score. Because the output distributions are discrete, log-loss has a lower bound of 0 and an upper bound given by the highest entropy distribution (uniform). Therefore we rescale scoring rule values based on their lower and upper bounds so that they're now between 0 and 100 and so that higher is better.

§ ADDITIONAL DETAILS ON TASKS & DATASETS

Task         Metric

Calibration     $\downarrow$     Expected Calibration Error (ECE)

$\uparrow$     Calibration AUROC

Selective Prediction     $\uparrow$     Referral Rate Curve

$\uparrow$     Oracle Collaborative Accuracy

Open Set Recognition     $\uparrow$     AUROC

Label Uncertainty     $\downarrow$     KL Divergence

In-Dist. Generalization     $\uparrow$     Accuracy

$\downarrow$     Negative Log-Likelihood (NLL)

$\downarrow$     Brier Score

$\uparrow$     (Binary Classification) AUROC

$\uparrow$     (Binary Classification) AUPRC

Covariate Shift     $\uparrow$     Accuracy

$\downarrow$     Negative Log-Likelihood

$\downarrow$     Brier Score

$\uparrow$     (Binary Classification) AUROC

$\uparrow$     (Binary Classification) AUPRC

Subpopulation Shift     $\uparrow$     Accuracy

$\downarrow$     Negative Log-Likelihood (NLL)

$\downarrow$     Brier Score

$\uparrow$     (Binary Classification) AUROC

$\uparrow$     (Binary Classification) AUPRC

Few-Shot Learning     $\uparrow$     Accuracy

$\downarrow$     Negative Log-Likelihood (NLL)

Few-Shot Uncertainty     $\downarrow$      Expected Calibration Error

$\uparrow$     Calibration AUROC

$\uparrow$     (Zero-Shot and Few-Shot) AUROC

Active Learning     $\uparrow$     Accuracy vs # Labels Curve

Metrics used for each task. $\uparrow$ / $\downarrow$ mean higher / lower is better.

We summarize the list of metrics in <Ref>.

For out-of-distribution evaluation in vision, we use the following datasets.

* Covariate shift.

* CIFAR-10: CIFAR-10-C [Hendrycks and Dietterich, 2019].

* CIFAR-100: CIFAR-100-C [Hendrycks and Dietterich, 2019].

* ImageNet1K: ImageNet-A [Hendrycks et al., 2021], ImageNet-C [Hendrycks and Dietterich, 2019], ImageNetV2 [Recht et al., 2019], ImageNet-Vid-Robust, YTBB Robust [Shankar et al., 2021], ObjectNet [Barbu et al., 2019], and ImageNet-R [Hendrycks et al., 2021].

* RETINA: RETINA's Country Shift dataset [Band et al., 2021].
We train models on images of retinas obtained from patients in the United States (EyePACS) and evaluate trained models on images of retinas obtained from patients in India using different collection equipment (APTOS).

* Semantic (class) shift.

* CIFAR-10: CIFAR-100, SVHN.

* CIFAR-100: CIFAR-10, SVHN.

* ImageNet1K: Places365.

* RETINA: RETINA’s Severity Shift dataset [Band et al., 2021].
We train models on images of retinas exhibiting no worse than mild diabetic retinopathy, and consider a shifted evaluation dataset with images of moderate diabetic retinopathy or worse.
The evaluation data contains features not contained in the training images, such as vitreous hemorrhages.
The motivation for this shift is that images of retinas with more severe retinopathy are relatively scarce and that it is likely for a model to be trained only on more widely-available images of retinas exhibiting mild cases of diabetic retinopathy.

* Label uncertainty.
We use CIFAR-10H [Peterson et al., 2019] which captures human uncertainty over labels for CIFAR-10 dataset.
We also construct a larger-scale variant, which we call ImageNet ReaL-H. Individual human ratings were recollected for the original ImageNet test set, available as raw data from ImageNet ReaL [Beyer et al., 2020], and we use them to newly construct soft label targets.

* Subpopulation shift.
We use Semantically Partitioned CIFAR-10/100 [Yuan et al., 2022] for vision subpopulation shift. CIFAR-10/100 test data is partitioned into semantically similar subpopulations, where each subpopulation has its own data-generating distribution sampled from a meta subpopulation distribution. We aim to improve predictive performance on tail subpopulations.

For covariate and semantic shift in language, we use:

* the MNLI-mismatched [Williams et al., 2017] data as the OOD set for NLI, which contains sentence pairs from 5 genres that are distinct from those in MNLI training data.

* the CivilComments corpus [Borkan et al., 2019] as the OOD set for toxic comment prediction, which consists of one million public comments appearing on approximately 50 English-language news sites across the world.

We also analyze language subpopulation shift where long-tail groups from the in-distribution set are a desired area for generalization.

* HANS [McCoy et al., 2019] eval datasets for NLI, which contains template-generated examples attacking the surface-level heuristics that the neural models are found to rely on when predicting entailment relationships.

* CivilCommentsIdentity [Borkan et al., 2019] for toxic comments, which is a subset of CivilComments that has explicit mention of social identities (e.g., muslim, LGBTQ, etc) that the model are often found to generate mispredictions.

* NaLUE-tail dataset for CLU, which is a subset of NaLUE corresponding to utterances from 28 low-frequency intents categories.

§ DETAILS OF PLEX INGREDIENTS

BatchEnsemble (BE) BatchEnsembles [Wen et al., 2020] approximate deep ensembles [Lakshminarayanan et al., 2017], but reduce their computational and memory costs by sharing weights across the ensemble members. The weight matrix $\mW_i$ of any given ensemble member $i$ is written as the Hadamard product of a shared weight matrix $\mW_0$ and a local rank-1 matrix $r_i  s_i^\top$:

\begin{align}
\mW_i = \mW_0 \circ  r_i s_i^\top.
\end{align}

The vectors r and s are commonly referred to as fast weights.

Unless otherwise stated, Plex applies BE to all layers in the last 2 residual blocks of the network. This idea follows work for mixture of experts [Riquelme et al., 2021].

Spectral-normalized Neural Gaussian Process (SNGP) Unlike ensemble approaches, SNGP proposed by Liu et al., 2020 focuses on improving the uncertainty quality of a neural network given a fixed representation (a.k.a. deterministic uncertainty quantification setting [Van Amersfoort et al., 2020].
When applied to a DNN without pretraining, SNGP enhances the DNN uncertainty property by applying spectral normalization to the hidden weights, and replaces the output layer from a dense layer to a random-feature Gaussian process (GP) layer.  That is, given hidden representations $h(\vx)$, the GP layer enables scalable computation of a GP posterior by applying a random feature approximation $\phi$ to the predictive function and then a Laplace approximation to the predictive variance:

\begin{align*}
g(\vx) &\sim N(\logit(\vx), \var(\vx)) \\
\logit(\vx)	&= \phi(x)^\top\evbeta,
\quad	\textrm{where} \\
\phi(\vx) &= \cos(\mW h(\vx) + \vb) \\
\var(x)	&= \phi(\vx)^\top(I + \Phi^T\Phi)^{-1}\phi(\vx),
\end{align*}

where $(\mW, \vb)$ are frozen random weights of the random feature embedding $\phi(\vx) = \cos(\mW h(\vx) + \vb)$, and $\Phi^\top\Phi= \sum_i\phi(\vx_i)\phi(\vx_i)^\top$ is the covariance of the random feature embedding estimated using the training data.

Liu et al., 2020, Liu et al., 2022 show that this combined technique improves the model's awareness of the semantic distance between the test and train examples on the data manifold, leading to improved performance in calibration and out-of-domain detection. When applied to a large pretrained DNN, we find it sufficient to only use the last-layer Gaussian process (i.e., omit the spectral normalization regularization), as the pretrained embedding has already provided a semantic-distance-aware representation of the data.

Heteroscedastic last layer (Het)
Heteroscedastic last layers are designed to model input-dependent label noise/label uncertainty (a.k.a. aleatoric uncertainty [Kendall and Gal, 2017]) that is present in the data. We use the Heteroscedastic (Het) last layer introduced by  Collier et al., 2020, Collier et al., 2021 who place a multivariate Gaussian distribution over the logits in a standard DNN classifier. A low-rank approximation to the $K \times K$ covariance matrix ($K$ = number of classes/outputs) is made when $K$ is large and [Collier et al., 2021] further develop a parameter efficient version of the method with parameterization inspired by BE to enable scaling to tens of thousands of classes.

Among the above methods, None, Het, GP, and BE are both memory and compute efficient, since they only require a single forward pass from a single model to compute the output distribution. We also experiment with deep ensembles (<Ref>), which is the most expensive in both memory and compute, since they require forward passes from multiple trained models, as well as Monte Carlo Dropout [Gal and Ghahramani, 2016].

§ ADDITIONAL RESULTS FOR OPEN SET RECOGNITION

Comparing OOD detection methods across all the tasks and model types.

While we use MSP as the basic uncertainty score in the main text, we also would like to discuss other alternative uncertainty scores, and how is their performance improved by the Plex models. We study the following 4 more uncertainty scores,

* Mahalanobis distance (Maha) [Lee et al., 2018] which measures the distance between the test input and the fitted training distribution in the embedding space. The training distribution is fitted using a class conditional Gaussian. The uncertainty score is the distance.

* Relative Mahalanobis distance [Ren et al., 2021] is a modified version of Mahalanobis distance which corrects for the background confounding effect using another Gaussian distribution fitted using entire training data ignoring class labels.

* Entropy of the softmax probability. High entropy suggests high uncertainty. So the uncertainty score is entropy.

* Maximum over the logits (MaxLogit) [Hendrycks et al., 2019] which uses the maximum of the un-normalized logits as the confidence score. Then the uncertainty score is then the negative of the MaxLogit.

We study the performance of five different OOD detection methods, MSP, Maha, RMaha, Entropy and MaxLogit, across the various tasks and model types. As shown in <Ref>, Mahalanobis distance outperforms all other scores, and Relative Mahalanobis distance performs the second best.

Despite the good performance, one drawback of Mahalanobis distance based method is that its computational time is linear to the number of classes. For the ImageNet2012 vs Places365 task, the in-distribution data ImageNet2012 has 1,000 classes where by definition of Mahalanobis distance method, we need to fit 1K Gaussians for each of the classes and compute the distance between the test input to each of the fitted Gaussian. It becomes very time consuming and not scalable. Therefore in that task, we study the lightweight OOD scores, including MSP, Entropy, and MaxLogit. Interestingly, we noticed that MaxLogit is the best among the three methods for single models (None, GP, None$\to$GP) except for Het and None$\to$Het, while Entropy is the best for ensemble models (BE, DE based models).

Overall we suggest using Mahalanobis distance based methods to achieve the best performance, but in case there is a computation budget, MaxLogit is a good choice for single models and Entropy is a good choice for ensemble models.

5cC-100 vs C-10                3cImageNet vs Places

MSP       Entropy     MaxLogits     Maha               RMaha     MSP        Entropy             MaxLogits
None                 0.929     0.944       0.953         0.982     0.965     0.828      0.855               0.880

None I21K            0.924     0.934       0.940         0.964     0.943     0.838      0.861               0.887

None$\to$GP          0.927     0.940       0.948         0.975     0.956     0.824      0.851               0.877

None$\to$Het         0.948     0.962       0.948         0.985     0.970     0.827      0.854      0.827

None$\to$DE L     0.940     0.953       0.940         0.982     0.965     0.831      0.860      0.831

None$\to$BE          0.924     0.939       0.924         0.980     0.962     0.822      0.850      0.822

GP                  0.927     0.936       0.945         0.971     0.945     0.828      0.854               0.873

Het                 0.948     0.964       0.948         0.981     0.979     0.828      0.856      0.828

DE L             0.946     0.961       0.946         0.987     0.976     0.834      0.863      0.834

BE L             0.940     0.954       0.940         0.985     0.971     0.827      0.855      0.827

BE L (I21K)      0.934     0.943       0.934         0.975     0.959     0.842      0.864      0.842

BE$\to$BE$+$Het       0.954     0.969       0.954         0.984     0.980     0.831      0.857      0.831

BE B             0.906     0.927       0.906         0.966     0.959     0.801      0.831      0.801

BE S             0.864     0.889       0.864         0.923     0.916     0.773      0.803      0.773

The AUROC based on the five OOD measures on the most challenging near-OOD tasks CIFAR-100 vs CIFAR-10 and ImageNet2012 vs Places365.

§ ADDITIONAL RESULTS FOR SELECTIVE PREDICTION

We performed model selection on pretrained models finetuned with different hyperparameters using the area under the in-distribution selective-prediction accuracy curve.
We evaluated the predictive performance of the finetuned models on in-distribution and distributionally shifted data.
In particular, we computed models’ predictive accuracy, negative log-likelihood, expected calibration error, and the area under the selective prediction curve obtained by computing the area under the ROC curve for selective prediction referral rates from 0% to 99% using different evaluation metrics (accuracy and, where applicable, AUROC and AUPRC).

Plex L
Plex L
None L
None L

(Single Model)

Selective prediction performance, using AUROC as the evaluation metric, on the RETINA Country Shift task (for in and out-of-distribution evaluation).
For in-distribution, the SoTA (Ensemble) outperforms Plex L pretrained on both ImageNet-21K or JFT.
For out-of-distribution, Plex L pretrained on either ImageNet-21K or JFT outperforms state-of-the-art.

Country Shift: In-Distribution

Severity Shift: In-Distribution

Selective Prediction on RETINA (In-Distribution).

The in-distribution Country Shift task is concerned with diagnosing diabetic retinopathy.

The in-distribution Severity Shift task is concerned with diagnosing diabetic retinopathy

amongst images containing at worst moderate cases of diabetic retinopathy.
Accuracy is used as the evaluation metric and predictive entropy is used as the per-example uncertainty estimate.

§ EXTENSIONS TO SPARSE MIXTURES OF EXPERTS

In this section, we discuss extensions of some approaches of this paper in the light of recent advances in sparse mixtures of experts models (sparse MoEs).

§.§ Background

Sparse MoEs constitute a family of models that rely on conditional computation [Bengio et al., 2013, Bengio et al., 2015] to combine multiple submodels—referred to as experts—in an input-dependent fashion.
Sparse MoEs have successfully been applied both in NLP [Shazeer et al., 2017, Lepikhin et al., 2021, Fedus et al., 2022] and more recently in computer vision [Riquelme et al., 2021, Lou et al., 2021]. The goal of conditional computation is to grow the number of parameters of the model while maintaining its training and inference costs constant, by only activating a particular subset of experts given an input. This is in contrast with classical neural networks that use all their parameters for each input.

In the context of this paper, we focus on Allingham et al., 2021 that studied the robustness and uncertainty estimates of several extensions of ViT endowed with conditional computation, namely:

* V-MoE: The authors of Riquelme et al., 2021 extended ViT to sparse MoEs by placing experts at the level of the MLPs of the transformer architecture. Since ViT operates on image patches, the conditional computation of V-MoEs also operates on image patches. More specifically, the image patches are sparsely routed to $K$ out of $E$ available experts (in practice, $E=32$). The experts are selected by computing some gating weights in combination with a top-$K$ strategy, allowing for an end-to-end training procedure.
When only a single expert is selected ($K=1$), the training and inference costs of V-MoEs almost match those of a standard ViT model [Riquelme et al., 2021], while leading to improved predictive performance at both pretraining and finetuning time.

In the next section, we will use the acronym MoE to refer to V-MoE.

* Deep ensembles of V-MoEs: Allingham et al., 2021 explored the combination of the static ensembling of deep ensembles and the conditional computation of V-MoEs. They found that both effects are complementary. In particular, over tasks where either V-MoEs or deep ensembles are known to perform well (e.g., respectively, few-shot classification and OOD detection), the combined approach inherits from the best of both worlds, while matching the cost of standard deep ensembles.

In what follows, we will use the acronym $[\textsc{MoE}]_4$  to refer to a deep ensemble formed by four MoEs where both the upstream and downstream models were varied. In the situation where only the downstream models were varied for a single fixed pretrained model, we use the notation $\textsc{MoE}\rightarrow[\textsc{MoE}]_4$. We will use the same convention for the model without any changes, None. While the rest of the paper uses ensembles of size 3, we consider in this section ensembles of size 4 to be faithful to the setting of  Allingham et al., 2021.

* E$^3$: The authors of Allingham et al., 2021 further designed an efficient ensemble approach, referred to as E$^3$, to overcome the computational burden of the naive ensembling of V-MoEs described above. In a nutshell, E$^3$ jointly learns an ensemble of smaller sparse MoEs where all layers not equipped with experts (e.g., attention layers) are shared across the ensemble members. Interestingly, E$^3$ features some of the extensions of Plex due to some of its conceptual similarities with BE. Allingham et al., 2021 observed that E$^3$ tends to lie either on, or close to, the Pareto frontiers of several metrics—e.g., NLL, Brier score and few-shot classification error—versus the computational cost of the models, as measured by FLOPs.

§.§ Results

We present the results in Table <ref>. They extend the evaluation from Allingham et al., 2021 to the additional metrics of this paper related to the computer vision tasks. We summarize the results with the prediction, uncertainty and adaptation scores.

In agreement with the conclusions reported in Allingham et al., 2021, we can see that $[\textsc{MoE}]_4$ outperforms[A closer inspection at the results shows that the prediction score of $[\textsc{MoE}]_4$ is slightly worse than that of $[\textsc{None}]_4$ because of isolated, slightly worse performance on CIFAR-10.] the standard deep ensemble $[\textsc{None}]_4$ while having the same computational cost. This confirms the complementary of static ensembling with the adaptivity of sparse MoEs within the broader evaluation of this paper. More generally, combining ensembling and sparse MoEs seems to be a promising direction to improve the reliability of models.

As observed in Riquelme et al., 2021, MoE has strong performance in fewshot learning tasks, which is highlighted here by the adaptation scores of MoE and $[\textsc{MoE}]_4$.
Moreover, Allingham et al., 2021 further observed that MoE did not perform well in terms of calibration, as illustrated by its lower uncertainty score compared with None.
The efficient ensemble E$^3$ has performance comparable to deep ensembles formed from a single pretrained model (i.e., $\textsc{None}\rightarrow[\textsc{None}]_4$ and $\textsc{MoE}\rightarrow[\textsc{MoE}]_4$), while being substantially cheaper. This conclusion echoes the take-home messages from Allingham et al., 2021.

Score     Score prediction     Score uncertainty     Score adaptation

None                                                    85.99                         91.13                          92.71                         84.65

MoE                                                   86.76                         90.52                          91.97                         85.75

E$^3$                                           $-$                         91.44                          92.77                           $-$

$\textsc{None}\rightarrow[\textsc{None}]_4$                $-$                         91.41                          92.83                           $-$

$\textsc{MoE}\rightarrow[\textsc{MoE}]_4$                $-$                         91.07                          92.82                           $-$

$[\textsc{None}]_4$                                     88.54                         91.99                          93.44                         87.59

$[\textsc{MoE}]_4$                                     88.66                         91.53                          93.51                         87.77

Results of sparse MoE variants developed in Allingham et al., 2021. E$^3$, $\textsc{None}\rightarrow[\textsc{None}]_4$ and $\textsc{MoE}\rightarrow[\textsc{MoE}]_4$ do not have adaptation scores since those approaches are applied only at finetuning time.

§ ANALYSIS OF HETEROSCEDASTIC LAST LAYER

We add a heteroscedastic output layer to the base deterministic model and the BE base model. We assess whether enabling the network to model input-dependent (heteroscedastic) label noise results in better performance on datasets known to have noisy labels.

§.§ Heteroscedastic improves pretraining performance on JFT.

Performance on JFT for deterministic and heteroscedastic models.

Performance on ImageNet21K for deterministic and heteroscedastic models.

In <Ref> and <Ref> the heteroscedastic model, when applied on top of the base deterministic model, provides performance gains on the JFT dataset while performance is neutral on ImageNet-21K.

Performance on ImageNet1K for deterministic, heteroscedastic (upstream and downstream), deterministic upstream$\to$heteroscedastic downstream, BatchEnsemble L (upstream and downstream) and BatchEnsemble L upstream$\to$BatchEnsemble L + heteroscedastic downstream models.

Performance on CIFAR-10 for deterministic, heteroscedastic (upstream and downstream), deterministic upstream$\to$heteroscedastic downstream, BatchEnsemble L (upstream and downstream) and BatchEnsemble L upstream$\to$BatchEnsemble L + heteroscedastic downstream models.

Performance on CIFAR-100 for deterministic, heteroscedastic (upstream and downstream), deterministic upstream$\to$heteroscedastic downstream, BatchEnsemble L (upstream and downstream) and BatchEnsemble L upstream$\to$BatchEnsemble L + heteroscedastic downstream models.

§.§ Impact of heteroscedastic last layer downstream

In <Ref>, <Ref> and <Ref> we look at the core in-distribution performance metrics for downstream vision datasets. Het$\to$Het refers to a model trained with a heteroscedastic output layer upstream on JFT and finetuned on the target dataset with a heteroscedastic head. None$\to$Het is similar but where the cheaper and simpler baseline deterministic model without a heteroscedastic head is used upstream. BE$\to$BE + Het refers to pretraining using a Batch Ensemble model upstream on JFT and finetuning with a BE model with a heteroscedastic head downstream.

On all downstream datasets None$\to$Het performs similarly or better than Het$\to$Het.
The outperformance of a deterministically pre-trained model finetuned with a heteroscedastic last layer, indicates that the gains from a heteroscedastic model downstream can be realized without heteroscedastic upstream pretraining. This is a surprising result given that the heteroscedastic model improves JFT performance upstream. However this result suggests that when the representations are transferred from this upstream model the Deterministic model is sufficient. Given that heteroscedastic model is more expensive than the deterministic model and requires tuning the temperature and the rank for the low-rank approximation hyperparameters this is an encouraging result.

Plex: heteroscedastic last layer helps downstream.
None$\to$Het outperforms None on all metrics for ImageNet and CIFAR-100. Performance on CIFAR-10 is saturated and similar. This demonstrates that applying the heteroscedastic head on the downstream model improves downstream performance. Further, see <Ref> which shows that on a wide variety of reliability metrics beyond in-distribution accuracy, NLL and ECE None$\to$Het outperforms None. We will also see later that the heteroscedastic method models label uncertainty better than baseline models.
Heteroscedastic can be combined with BE upstream pretraining for excellent results. The results in the above table and the more comprehensive summary results in <Ref> show that the heteroscedastic head can be combined with a BE model to give the best performance single model on vision tasks (this is Plex). BE$\to$BE$+$Het (Plex) combines the epistemic uncertainty model capability of Batch Ensembles with the label uncertainty modeling of the heteroscedastic method. Plex approaches Deep Ensembles in terms of overall reliability, despite being substantially cheaper. Therefore we can recommend this model as our best single model for use by practitioners.

§ SUMMARIZATION OF LANGUAGE RESULTS

We first compare the performance across types of uncertainty methods, fixing the architecture size to T5-base.
We compare performances in prediction, uncertainty calibration, and human-model collaboration, across all datasets (MNLI, NaLUE and Toxic Comments) and all splits (In-domain, OOD, and tail-population).
<Ref> reports the full results, and <Ref> summarizes the rankings of uncertainty methods under each type of population shift (in-domain v.s. OOD v.s. tail-population).
Among all methods, DE$+$GP, Plex (i.e., BE$+$GP), BE, and MC Dropout tend to have the strongest performance. In particular, DE$+$GP almost always dominates the other methods on MNLI and NaLUE, and remains competitive in the case of label imbalance (i.e., Toxic Comments). However, DE$+$GP is an expensive method that costs x10 more in memory and compute and therefore is not competitive in scale (a more thorough analysis is in <Ref>). On the other hand, among the more efficient, single-model methods, BE and Plex perform well on MNLI and NaLUE (notably, outperform the most expensive DE), while MCD stands out in the Toxic Comments. The above observations suggest that, when the training example has a simple distribution, quantifying output-layer uncertainty alone is sufficient to attain strong performance. However, when there are pathologies in the data distribution (e.g., extreme label imbalance), quantifying the uncertainty within the model's intermediate representations (e.g., via some form of perturbation like BE) becomes important.

We then investigate how a model's uncertainty performance is impacted by the architecture size.
For model size scaling, we evaluate Plex, None, and MC Dropout, the three best-performing and efficient methods in the previous study. We evaluate the performance of each method under three progressively larger architectures: T5 S, T5 B, and T5 L, and observe how the behavior changes across the method and with respect to the architecture size.
<Ref> reports the full results, and <Ref> summarizes the rankings of uncertainty methods organized by the sizes of the architecture.
As shown, comparing across architecture sizes, we see a larger architecture almost always leads to stronger performance in collaborative performance. This trend remains largely consistent even when  out-of-distribution.

None B      Het B      GP B      BE B      Plex B      MCD B      DE B      DE-GP B

Task     Split     Score

9*MNLI       3*In-domain       calibration      0.381     0.384     0.372     black!500.401     0.388     black!90 0.416     0.383     black!100.4

generalization      0.938     black!500.949     0.944     black!500.949     0.948     0.946     0.938     black!90 0.95

select. pred.      0.961     0.971     0.968     black!500.973     black!500.973     black!500.973     0.961     black!90 0.975

3*OOD       calibration      0.391     0.401     0.393     black!500.413     0.394     black!100.41     0.388     black!90 0.416

generalization      0.937     black!100.948     0.941     black!500.949     black!100.948     0.946     0.938     black!90 0.95

select. pred.      0.959     0.971     0.966     black!500.973     black!500.973     0.972     0.96     black!90 0.975

3*Subpopulation       calibration      black!100.451     0.434     black!500.454     0.443     0.401     black!90 0.474     0.443     0.418

generalization      0.749     0.766     0.762     black!500.791     black!100.788     0.739     0.764     black!90 0.798

select. pred.      0.811     0.824     0.842     black!100.87     black!500.871     0.831     0.826     black!90 0.885

8*NaLUE       3*In-domain       calibration      black!500.498     0.484     black!90 0.512     0.464     0.486     0.471     0.487     black!100.494

generalization      0.939     0.932     black!100.94     0.935     black!100.94     0.938     black!500.942     black!90 0.944

select. pred.      0.936     0.935     0.932     black!90 0.938     black!90 0.938     0.932     0.937     black!90 0.938

OOS, Near       detection      0.706     0.673     0.719     black!500.768     0.716     black!100.766     0.721     black!90 0.771

OOS, Standard       detection      0.964     0.957     0.992     black!90 0.998     0.991     black!100.994     0.973     black!90 0.998
3*Subpopulation       calibration      black!100.518     0.511     black!90 0.553     0.514     black!500.519     0.466     0.513     0.514

generalization      0.866     0.846     0.869     0.87     black!500.873     0.858     black!100.871     black!90 0.882

select. pred.      black!90 0.862     0.82     0.828     0.845     black!100.856     0.829     black!500.861     0.851

Comments       3*In-domain       calibration      0.459     black!100.462     0.46     0.461     black!90 0.471     0.442     0.459     black!500.465

generalization      0.888     0.89     black!500.899     0.889     black!100.895     black!90 0.904     0.885     0.892

select. pred.      0.936     0.938     black!500.94     0.938     black!500.94     black!90 0.941     0.936     0.939

3*OOD       calibration      0.425     black!100.427     black!500.438     0.423     black!90 0.447     0.413     0.426     0.421

generalization      black!500.82     0.817     black!100.818     0.81     0.816     black!90 0.831     0.817     black!100.818

select. pred.      black!500.86     0.857     0.855     0.85     0.855     black!90 0.862     black!500.86     0.852

3*Subpopulation       calibration      0.415     0.405     black!500.421     0.412     black!90 0.428     0.405     black!100.416     0.4

generalization      black!500.806     0.803     black!100.804     0.795     0.801     black!90 0.814     0.801     0.803

select. pred.      black!500.831     0.828     0.828     0.821     0.826     black!90 0.835     black!100.83     0.823

Comparison of method performance between uncertainty methods. Black: Best. Dark Grey: Second. Light Grey: Third.

None S      MCD S      Plex S      None B      MCD B      Plex B      None L      MCD L      Plex L

Task     Split     Score

9*MNLI       3*In-domain       calibration      0.399     black!500.406     0.364     0.381     black!90 0.416     0.388     0.39     black!100.404     0.394

generalization      0.924     0.927     0.913     0.938     0.946     0.948     black!100.963     black!500.964     black!90 0.965

select. pred.      0.953     0.959     0.942     0.961     0.973     0.973     black!100.982     black!90 0.985     black!90 0.985

3*OOD       calibration      0.398     0.396     0.367     0.391     black!100.41     0.394     black!90 0.418     black!500.411     0.406

generalization      0.924     0.93     0.911     0.937     0.946     0.948     black!100.963     black!500.965     black!90 0.967

select. pred.      0.953     0.96     0.94     0.959     0.972     0.973     black!100.983     black!500.986     black!90 0.987

3*Subpopulation       calibration      black!500.555     black!90 0.56     black!100.514     0.451     0.474     0.401     0.417     0.45     0.447

generalization      0.648     0.619     0.59     0.749     0.739     0.788     black!90 0.817     black!500.807     black!100.803

select. pred.      0.747     0.717     0.677     0.811     0.831     0.871     black!100.875     black!90 0.896     black!500.876

8*NaLUE       3*In-domain       calibration      black!90 0.507     0.48     black!500.498     black!500.498     0.471     0.486     0.486     0.453     0.496

generalization      black!500.942     0.932     0.937     0.939     0.938     black!100.94     0.931     0.928     black!90 0.944

select. pred.      black!500.937     0.926     0.929     black!100.936     0.932     black!90 0.938     0.93     0.922     0.935

OOS, Near       detection      0.71     black!100.756     0.689     0.706     black!500.766     0.716     0.692     0.733     black!90 0.781

OOS, Standard       detection      0.968     black!100.992     black!90 0.999     0.964     black!500.994     0.991     0.956     0.991     0.991
3*Subpopulation       calibration      black!90 0.528     0.462     0.499     black!100.518     0.466     black!500.519     black!100.518     0.466     0.492

generalization      black!90 0.878     0.854     0.851     0.866     0.858     black!500.873     0.843     0.83     black!100.871

select. pred.      black!90 0.864     0.836     0.84     black!500.862     0.829     black!100.856     0.835     0.801     0.835

Comments       3*In-domain       calibration      0.455     0.445     black!90 0.478     black!100.459     0.442     black!500.471     0.448     0.451     0.436

generalization      0.879     black!100.898     0.863     0.888     black!500.904     0.895     0.886     black!90 0.906     0.89

select. pred.      0.932     0.938     0.919     0.936     black!100.941     0.94     0.936     black!90 0.944     black!500.942

3*OOD       calibration      0.423     0.412     0.425     0.425     0.413     black!500.447     black!100.432     0.417     black!90 0.459

generalization      0.81     black!100.823     0.807     0.82     black!500.831     0.816     black!100.823     black!90 0.837     0.816

select. pred.      0.85     0.851     0.838     0.86     0.862     0.855     black!500.865     black!90 0.869     black!100.863

3*Subpopulation       calibration      0.409     0.404     0.412     0.415     0.405     black!500.428     black!100.426     0.403     black!90 0.46

generalization      0.795     0.806     0.786     0.806     black!500.814     0.801     black!100.809     black!90 0.822     0.805

select. pred.      0.82     0.824     0.803     0.831     0.835     0.826     black!100.837     black!90 0.842     black!500.838

Comparison of method performance between architecture sizes. Black: Best. Dark Grey: Second. Light Grey: Third.

Generalization, IND.

Generalization., OOD.

Generalization, SUB.

Generalization, ALL.

Calibration, IND.

Calibration, OOD.

Calibration, SUB.

Calibration, ALL.

Selective Pred., IND.

Selective Pred., OOD.

Selective Pred., SUB.

Selective Pred., ALL.

T5-Plex model's ranking comparison between different uncertainty methods and across different evaluation datasets. IND: in-domain. OOD: out-of-domain. SUB: subpopulation shift. ALL: aggregated performance across all datasets.

Generalization, IND.

Generalization., OOD.

Generalization, SUB.

Generalization, ALL.

Calibration, IND.

Calibration, OOD.

Calibration, SUB.

Calibration, ALL.

Selective Pred., IND.

Selective Pred., OOD.

Selective Pred., SUB.

Selective Pred., ALL.

T5-Plex model's ranking comparison between architecture sizes and across different evaluation datasets. IND: in-domain. OOD: out-of-domain. SUB: subpopulation shift. ALL: aggregated performance across all datasets.