# Magnetic dipole moments of $\Xi_{bc}$ and $\Omega_{bc}$ baryons in light-
cone QCD

Ulaş Özdem<EMAIL_ADDRESS>Health Services Vocational School of Higher
Education, Istanbul Aydin University, Sefakoy-Kucukcekmece, 34295 Istanbul,
Türkiye

###### Abstract

The magnetic dipole moments of the spin-$\frac{1}{2}$ bottom-charm baryons are
extracted utilizing the QCD light-cone sum rule with the help of the photon
distribution amplitudes. While deriving the magnetic dipole moments of the
bottom charm baryons, two different possible interpolating currents that can
be coupled to these states are taken into account. The magnetic dipole moments
of the bottom-charm baryons include significant knowledge of their inner
structure and geometric shape. The magnetic dipole moment results of the study
are compared with estimations acquired in various other models and approaches.

Electromagnetic form factors, bottom-charm baryons, magnetic dipole moment,
QCD light-cone sum rules

##  I Motivation

Heavy quark baryons play an important role in hadron physics and the
investigation of their features may deepen our comprehension of QCD. In the
last few decades, measurements of heavy baryons have accelerated with
improvements in experimental facilities. Numerous states have been discovered,
but many states need confirmation; hence the heavy baryon sector is
theoretically attractive. One of the doubly-charmed baryons ($\Xi_{cc}^{+}$),
which contains two c and one d quark, was first announced by the SELEX
Collaboration with the mass $M_{\Xi^{+}_{cc}}=3519\pm 1~{}$MeV [1], although,
the result of SELEX could neither be confirmed nor excluded by other
experimental groups such as FOCUS [2], BABAR [3], and Belle [4]. In 2017, a
doubly-charmed baryon $\Xi_{cc}^{++}$ with the mass
$M_{\Xi_{cc}^{++}}=3621.40\pm 0.72\pm 0.27\pm 0.14~{}$MeV was discovered by
the LHCb Collaboration [5]. The observation of the $\Xi_{cc}^{++}$ state
opened the door to the experimental detection of doubly-heavy baryons, and
scientists expected to discover more doubly-heavy baryons experimentally. The
LHCb collaboration has also carried out searches for the doubly heavy baryons,
$\Xi^{+}_{bc}$, $\Xi^{0}_{bc}$ and $\Omega_{bc}$, yet these baryons are yet to
be observed [6, 7, 8]. Further measurements will be possible with larger data
samples, and additional decay modes are expected at the upgraded LHCb
experiments. The study of doubly-heavy baryons contributes to an in-depth
comprehension of the chiral dynamics, heavy quark symmetry, fundamental theory
of the strong interaction, and models inspired by QCD. Therefore, numerous
theoretical studies inspired by these discoveries also have been carried out
to define the characteristic features like the masses, decay widths, strong
coupling constants, and lifetimes of doubly-heavy baryons by investigating the
strong, electromagnetic, and weak decays of these baryons within different
theoretical models and approaches [9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38,
39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57,
58, 59, 60, 61, 62, 51, 52, 53, 55, 57, 63, 64, 65, 66, 67, 68, 69, 70, 71,
72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 48, 85, 43, 49, 86, 87,
88, 89, 25, 90, 91, 92, 93, 94, 95, 96, 97, 95]. It is worth noting that the
different approaches used to derive the dynamic and static properties of
double-heavy baryons lead to quite different estimations, which can be used to
distinguish between approaches.

Investigating the electromagnetic features of hadrons ensure us with helpful
knowledge about their inner organizations. One can extract information about
their shapes, sizes, and decay widths and compare them with the experimental
results. Baryons including two heavy quarks are particularly stimulating to
study since examining the electromagnetic features of two heavy quarks bound
to a light quark helps us figure out the internal interaction dynamics of
baryons containing heavy quarks. Moreover, the results can help us understand
the key features of QCD such as confinement and flavor effects. In the present
study, the magnetic dipole moments of spin-$\frac{1}{2}$ bottom-charm baryons
($B_{bc}$ for short) are obtained within the light-cone QCD sum rule (LCSR).
This method is powerful for studying the dynamic and static properties of
hadrons. In the LCSR method [98, 99, 100], a relevant two-point correlation
function within the external background electromagnetic field is evaluated in
two different ways. Firstly, the so-called hadronic representation is obtained
in terms of hadronic parameters such as form factors, magnetic dipole moments,
etc. In the second way, the so-called QCD representation is evaluated in terms
of quark-gluon degrees of freedom. The correlation functions acquired in these
two different ways are then correlated using the assumption of quark-hadron
duality. As a final step, Borel transform and continuum subtraction are
applied to eliminate the contributions of possible higher states and
continuum. In this way, the LCSR for the desired physical quantities, in our
case magnetic dipole moments, are acquired.

This work is organized as follows. In Sec. II, the calculation method used in
this work is briefly introduced. We present our results, compare them to other
works and give a discussion in Sec. III. Finally, a summary is given in Sec.
IV.

##  II Formalism

As we mentioned above, the starting point to evaluate any physical quantity in
the LCSR method is to write a convenient correlation function. Here for the
magnetic dipole moments, the correlation function is written as

$\Pi(p,q)=i\int d^{4}xe^{ip\cdot x}\langle
0|\mathcal{T}\\{J(x)\bar{J}(0)\\}|0\rangle_{\gamma},$ (1)

where q is the momentum of the photon, $\gamma$ is the external
electromagnetic field and $J(x)$ is the interpolating current having quantum
numbers $J^{P}=\frac{1}{2}^{+}$. The interpolating current $J(x)$ is one of
the key components necessary to extract the magnetic dipole moments of the
bottom-charm baryons employing the LCSR method. There are different ways to
construct the corresponding interpolating currents using the diquark-quark
configurations of different spin parities. Taking into account all the
properties of the bottom-charm baryons, the following interpolating currents
are constructed in Ref. [101], that we employ to evaluate the magnetic dipole
moments of these bottom-charm baryons:

$\displaystyle J_{1}(x)$
$\displaystyle=\varepsilon^{abc}\big{[}b_{a}^{T}(x)C\gamma_{\mu}c_{b}(x)\big{]}\gamma_{5}\gamma^{\mu}q_{c}(x),$
$\displaystyle J_{2}(x)$
$\displaystyle=\varepsilon^{abc}\big{[}b_{a}^{T}(x)C\gamma_{5}c_{b}(x)\big{]}q_{c}(x),$
(2)

where q is u, d or s-quark, a, b, and c are color indices; and C is the charge
conjugation matrix.

To evaluate the physical representation of the correlation function, we insert
a complete set of hadronic states having the same quantum numbers as that of
bottom-charm baryons interpolating currents and we get:

$\displaystyle\Pi^{Had}(p,q)$ $\displaystyle=$ $\displaystyle\frac{\langle
0\mid
J(x)\mid{B_{bc}}(p)\rangle}{[p^{2}-m_{{B_{bc}}}^{2}]}\langle{B_{bc}}(p)\mid{B_{bc}}(p+q)\rangle_{\gamma}\frac{\langle{B_{bc}}(p+q)\mid\bar{J}(0)\mid
0\rangle}{[(p+q)^{2}-m_{{B_{bc}}}^{2}]}+...,$ (3)

where dots stand for the contribution of the continuum and higher states. As
can be seen from in Eq. (3), we need explicit forms of matrix elements such as
$\langle 0\mid J_{B_{bc}}(0)\mid{B_{bc}}(p,s)\rangle$ and
$\langle{B_{bc}}(p)\mid{B_{bc}}(p+q)\rangle_{\gamma}$ and they are written as
follows

$\displaystyle\langle 0\mid J_{B_{bc}}(0)\mid{B_{bc}}(p,s)\rangle$
$\displaystyle=$ $\displaystyle\lambda_{{B_{bc}}}\nu(p,s),$ (4)
$\displaystyle\langle{B_{bc}}(p)\mid{B_{bc}}(p+q)\rangle_{\gamma}$
$\displaystyle=$
$\displaystyle\varepsilon^{\mu}\bar{\nu}(p)\Bigg{[}\big{(}F_{1}(Q^{2})+F_{2}(Q^{2})\big{)}\gamma_{\mu}+F_{2}(Q^{2})\,\frac{(2p+q)_{\mu}}{2m_{B_{bc}}}\Bigg{]}\nu(p),$
(5)

where $\nu(p)$ is the Dirac spinor, $\lambda_{{B_{bc}}}$ is the residue of the
corresponding baryon; and $F_{1,2}(Q^{2})$ are Lorentz invariant form factors
whose values at $Q^{2}=0$ are needed in determination of the magnetic dipole
moments. Summation over spins of $B_{bc}$ baryon is carried out as:

$\sum_{s}\nu(p,s)\bar{\nu}(u,s)=p\\!\\!\\!/+m_{B_{bc}},$ (6)

Using Eqs. (3)-(6) the hadronic representation of the correlation function
takes the form

$\displaystyle\Pi^{Had}(p,q)=$
$\displaystyle\frac{\lambda^{2}_{B_{bc}}}{[(p+q)^{2}-m^{2}_{B_{bc}}][p^{2}-m^{2}_{B_{bc}}]}\bigg{[}\Big{(}F_{1}(Q^{2})+F_{2}(Q^{2})\Big{)}\Big{(}2\,p\\!\\!\\!/\varepsilon\\!\\!\\!/p\\!\\!\\!/+p\\!\\!\\!/\varepsilon\\!\\!\\!/q\\!\\!\\!/+m_{B_{bc}}\,p\\!\\!\\!/\varepsilon\\!\\!\\!/$
$\displaystyle+2\,m_{B_{bc}}\,\varepsilon\\!\\!\\!/p\\!\\!\\!/+m_{B_{bc}}\,\varepsilon\\!\\!\\!/q\\!\\!\\!/+m_{B_{bc}}^{2}\varepsilon\\!\\!\\!/\Big{)}+\mbox{
other structures~{}proportional with the}F_{2}(Q^{2})\bigg{]}.$ (7)

At the static limit, namely $Q^{2}=0$, the magnetic dipole moment is
determined concerning the form factors $F_{1}(Q^{2}=0)$ and $F_{2}(Q^{2}=0)$
as follows

$\mu_{B_{bc}}=F_{1}(Q^{2}=0)+F_{2}(Q^{2}=0).$ (8)

As one can be seen from Eq. (II), there are different Lorentz structures.
Among them, we choose $\varepsilon\\!\\!\\!/q\\!\\!\\!/$ which includes the
magnetic dipole moment form factor $F_{1}(Q^{2}=0)+F_{2}(Q^{2}=0)$. As a
result, the hadronic representation of the correlation function, the magnetic
dipole moment for spin-$\frac{1}{2}$ $B_{bc}$ baryons can be written as:

$\displaystyle\Pi^{Had}(p,q)$ $\displaystyle=$
$\displaystyle\mu_{B_{bc}}\frac{\lambda^{2}_{B_{bc}}\,m_{B_{bc}}}{[(p+q)^{2}-m^{2}_{B_{bc}}][p^{2}-m^{2}_{B_{bc}}]}.$
(9)

The second representation of the correlation function, the QCD side, is
obtained by explicit use of the interpolating currents to the correlation
functions. Then the corresponding quark fields are contracted via Wick’s
theorem and the desired results are obtained. As a result of the above
procedures, the QCD representation of the correlation functions is obtained as
follows:

$\displaystyle\Pi_{1}^{QCD}(p,q)$
$\displaystyle=i\,\varepsilon^{abc}\varepsilon^{a^{\prime}b^{\prime}c^{\prime}}\int
d^{4}\,xe^{ip\cdot x}\langle 0\mid
Tr[\gamma_{\mu}S_{c}^{bb^{\prime}}(x)\gamma_{\nu}\tilde{S}_{b}^{aa^{\prime}}(x)]\big{(}\gamma_{5}\gamma^{\mu}S_{q}^{cc^{\prime}}(x)\gamma^{\nu}\gamma_{5}\big{)}\mid
0\rangle_{\gamma},$ (10) $\displaystyle\Pi_{2}^{QCD}(p,q)$
$\displaystyle=i\,\varepsilon^{abc}\varepsilon^{a^{\prime}b^{\prime}c^{\prime}}\int
d^{4}\,xe^{ip\cdot x}\langle 0\mid
Tr[\gamma_{5}S_{c}^{bb^{\prime}}(x)\gamma_{5}\tilde{S}_{b}^{aa^{\prime}}(x)]S_{q}^{cc^{\prime}}(x)\mid
0\rangle_{\gamma},$ (11)

where $\tilde{S}_{Q(q)}^{ij}(x)=CS_{Q(q)}^{{ij}^{T}}(x)C$ and, $S_{q}^{ij}(x)$
and $S_{Q}^{ij}(x)$ are the light and heavy quark propagators, respectively.
The light and heavy quark propagators are written as [102, 103],

$\displaystyle S_{q}(x)$
$\displaystyle=\frac{1}{2\pi^{2}x^{2}}\Big{(}i\frac{{x\\!\\!\\!/}}{x^{2}}-\frac{m_{q}}{2}\Big{)}-\frac{\bar{q}q}{12}\Big{(}1-i\frac{m_{q}x\\!\\!\\!/}{4}\Big{)}-\frac{\bar{q}\sigma.Gq}{192}x^{2}\Big{(}1-i\frac{m_{q}x\\!\\!\\!/}{6}\Big{)}-\frac{ig_{s}}{32\pi^{2}x^{2}}~{}G^{\mu\nu}(x)\bigg{[}\hbox
to0.0pt{/\hss}{x}\sigma_{\mu\nu}+\sigma_{\mu\nu}\hbox
to0.0pt{/\hss}{x}\bigg{]},$ (12) $\displaystyle S_{Q}(x)$ $\displaystyle=$
$\displaystyle\frac{m_{Q}^{2}}{4\pi^{2}}\bigg{[}\frac{K_{1}(m_{Q}\sqrt{-x^{2}})}{\sqrt{-x^{2}}}+i\frac{{x\\!\\!\\!/}~{}K_{2}(m_{Q}\sqrt{-x^{2}})}{(\sqrt{-x^{2}})^{2}}\bigg{]}-\frac{g_{s}m_{Q}}{16\pi^{2}}\int_{0}^{1}dv~{}G^{\mu\nu}(vx)\bigg{[}\big{(}\sigma_{\mu\nu}{x\\!\\!\\!/}+{x\\!\\!\\!/}\sigma_{\mu\nu}\big{)}\frac{K_{1}(m_{Q}\sqrt{-x^{2}})}{\sqrt{-x^{2}}}$
(13) $\displaystyle+2\sigma^{\mu\nu}K_{0}(m_{Q}\sqrt{-x^{2}})\bigg{]},$

where $G^{\mu\nu}$ is the gluon field strength tensor, $K_{0,1,2}$ are Bessel
functions of the second kind; and $m_{q}$ and $m_{Q}$ are the masses of the
light and heavy quarks, respectively.

The correlation functions contain two parts, that is, photon interacting with
light and heavy quarks perturbatively, and photon interacting with light quark
nonperturbatively. When the photon interacts with light and heavy quarks
perturbatively, one of the propagators in Eq. (11) is substituted by

$\displaystyle S^{free}(x)\rightarrow\int d^{4}y\,S^{free}(x-y)\,\hbox
to0.0pt{/\hss}{\\!A}(y)\,S^{free}(y)\,,$ (14)

where $S^{free}$ is the first term of the light or heavy quark propagators and
the remaining two quark propagators are taken as full quark propagators. When
the manipulations mentioned above are performed, perturbative contributions
are calculated.

In the case of the photon interacting with light quark nonperturbatively in
Eqs. (10) and (11) is substituted by

$\displaystyle
S_{\alpha\beta}^{ab}\rightarrow-\frac{1}{4}(\bar{q}^{a}\Gamma_{i}q^{b})(\Gamma_{i})_{\alpha\beta},$
(15)

and the remaining two propagators are substituted with the full quark
propagators. Since a photon interacts with quark fields nonperturbatively
there shows up the matrix elements of nonlocal operators
$\langle\gamma(q)\left|\bar{q}(x)\Gamma_{i}q(0)\right|0\rangle$ and
$\langle\gamma(q)\left|\bar{q}(x)\Gamma_{i}G_{\alpha\beta}q(0)\right|0\rangle$
between the photon state and vacuum. To evaluate the nonperturbative effects,
we require these matrix elements and those are determined in connection with
the photon distribution amplitude (DAs) with definite twists in Ref. [104].
The QCD representation of the correlation function can be evaluated in
connection with QCD parameters by replacing photon DAs and expressions for
heavy and light quarks propagators into Eq. (11).

The sum rules are acquired by equating the expression of the correlation
function in connection with QCD parameters to its expression in connection
with the hadron features, employing their spectral representation. To suppress
the subtraction terms in the spectral representation of the correlation
function, the double Borel transformation in connection with the variables
$p^{2}$ and $(p+q)^{2}$ is applied. After the transformation, contributions
from the higher states and continuum are also exponentially suppressed.
Finally, we pick the $\varepsilon\\!\\!\\!/q\\!\\!\\!/$ structure for the
magnetic dipole moments and obtain

$\displaystyle\mu_{1,B_{bc}}$ $\displaystyle=$
$\displaystyle\frac{e^{\frac{m^{2}_{1,B_{bc}}}{M^{2}}}}{\lambda^{2}_{1,B_{bc}}\,m_{1,B_{bc}}}\,\Pi_{1}^{QCD}(M^{2},s_{0}),$
(16) $\displaystyle\mu_{2,B_{bc}}$ $\displaystyle=$
$\displaystyle\frac{e^{\frac{m^{2}_{2,B_{bc}}}{M^{2}}}}{\lambda^{2}_{2,B_{bc}}\,m_{2,B_{bc}}}\,\Pi_{2}^{QCD}(M^{2},s_{0}).$
(17)

The analytical expressions of the functions $\Pi_{1}^{QCD}(M^{2},s_{0})$ and
$\Pi_{2}^{QCD}(M^{2},s_{0})$ are very similar forms, so, for the sake of
brevity, we only present the explicit expressions of the function
$\Pi_{1}^{QCD}(M^{2},s_{0})$.

$\displaystyle\Pi_{1}^{QCD}(M^{2},s_{0})$
$\displaystyle=\frac{e_{q}P_{1}P_{2}}{1327104\pi}\Big{[}-6\Big{(}I[0,2,2,0]-2I[0,2,3,0]+I[0,2,4,0]+2I[1,1,2,0]-4I[1,1,3,0]+2I[1,1,4,0]\Big{)}$
$\displaystyle\times
I_{2}[h_{\gamma}]+3\Big{(}5I[0,2,2,0]-10I[0,2,3,0]+5I[0,2,4,0]+8I[1,1,2,0]-16I[1,1,3,0]+8I[1,1,4,0]\Big{)}$
$\displaystyle\times\mathbb{A}[u_{0}]+2\chi\Big{(}4I[0,3,2,0]-8I[0,3,3,0]+4I[0,3,4,0]+3I[1,2,2,0]-6I[1,2,3,0]+3I[1,2,4,0]\Big{)}$
$\displaystyle\times\varphi_{\gamma}[u_{0}]\Big{]}$
$\displaystyle-\frac{P_{2}}{983040\pi}\Big{[}5e_{q}\Big{(}16m_{c}^{2}\big{(}I[0,3,1,0]-I[0,3,2,0]\big{)}I_{1}[\mathcal{S}]+3\big{(}3I[0,4,2,0]-6I[0,4,3,0]+3I[0,4,4,0]$
$\displaystyle+4I[1,3,2,0]-8I[1,3,3,0]+4I[1,3,4,0]\big{)}I_{1}[\mathcal{S}]+9\big{(}-I[0,4,3,0]+3I[0,4,4,0]-3I[0,4,5,0]$
$\displaystyle+I[0,4,6,0]-4I[1,3,3,0]+12I[1,3,4,0]-12I[1,3,5,0]+4I[1,3,6,0]\big{)}I_{2}[h_{\gamma}]+9\big{(}5I[0,4,3,0]$
$\displaystyle-15I[0,4,4,0]+15I[0,4,5,0]-5I[0,4,6,0]+16I[1,3,3,0]-48I[1,3,4,0]+48I[1,3,5,0]$
$\displaystyle-16I[1,3,6,0]+6I[2,2,3,0]-18I[2,2,4,0]+18I[2,2,5,0]-6I[2,2,6,0]\big{)}\mathbb{A}[u_{0}]+9\chi\big{(}4I[0,5,3,0]$
$\displaystyle-12I[0,5,4,0]+12I[0,5,5,0]-4I[0,5,6,0]+5I[1,4,3,0]-15I[1,4,4,0]+15I[1,4,5,0]$
$\displaystyle-5I[1,4,6,0]\big{)}\varphi_{\gamma}[u_{0}]\Big{)}+e_{b}\Big{(}32m_{b}m_{c}\big{(}I[0,3,1,0]-2I[0,3,2,0]+I[0,3,3,0]\big{)}+18I[0,4,2,0]$
$\displaystyle-54I[0,4,3,0]+54I[0,4,4,0]-18[0,4,5,0]+3m_{0}^{2}\Big{(}16m_{b}m_{c}\big{(}I[0,2,1,0]-2I[0,2,2,0]+I[0,2,3,0]$
$\displaystyle+I[1,1,1,0]-2I[1,1,2,0]+I[1,1,3,0]\big{)}+12I[0,3,2,0]-36I[0,3,3,0]+36I[0,3,4,0]-12I[0,3,5,0]$
$\displaystyle+45I[1,2,2,0]-135I[1,2,3,0]+135I[1,2,4,0]-45I[1,2,5,0]+6I[2,1,2,0]-18I[2,1,3,0]$
$\displaystyle+18I[2,1,4,0]-6I[2,1,5,0]\Big{)}\Big{)}+5e_{c}\Big{(}32m_{b}m_{c}\big{(}I[0,3,2,0]-I[0,3,3,0]\big{)}+18I[0,4,3,0]$
$\displaystyle-36I[0,4,4,0]+18I[0,4,5,0]+3m_{0}^{2}\Big{(}16m_{b}m_{c}\big{(}I[0,2,2,0]-I[0,2,3,0]+I[1,1,2,0]-I[1,1,3,0]\big{)}$
$\displaystyle+12I[0,3,3,0]-24I[0,3,4,0]+12I[0,3,5,0]+45I[1,2,3,0]-90I[1,2,4,0]+45I[1,2,5,0]$
$\displaystyle+6I[2,1,3,0]-12I[2,1,4,0]+6I[2,1,5,0]\Big{)}\Big{)}\Big{]}$
$\displaystyle+\frac{m_{q}m_{c}m_{b}}{1024\pi^{3}}\Big{[}e_{c}\big{(}I[0,4,2,0]-I[0,4,2,1]-I[0,4,3,0]\big{)}+e_{b}\big{(}I[0,4,1,0]-2I[0,4,1,1]+I[0,4,1,2]$
$\displaystyle-2I[0,4,2,0]+2I[0,4,2,1]+I[0,4,3,0]\big{)}\big{]}$
$\displaystyle+\frac{3m_{q}m_{0}^{2}}{327680\pi^{3}}\Big{[}e_{b}\Big{(}20I[0,4,2,0]-60I[0,4,2,1]+60I[0,4,2,2]-20I[0,4,2,3]-60I[0,4,3,0]$
$\displaystyle+120I[0,4,3,1]-60I[0,4,3,2]+60I[0,4,4,0]-60I[0,4,4,1]-20I[0,4,5,0]-I[0,5,2,0]$
$\displaystyle+3I[0,5,3,0]-3I[0,5,4,0]+I[0,5,5,0]-80\big{(}I[1,3,2,0]-3I[1,3,2,1]+3I[1,3,2,2]-I[1,3,2,3]$
$\displaystyle-3(I[1,3,3,0]-2I[1,3,3,1]+I[1,3,3,2]-I[1,3,4,0]+I[1,3,4,1])-I[1,3,5,0]\big{)}\Big{)}$
$\displaystyle+e_{c}\Big{(}20I[0,4,3,0]-40I[0,4,3,1]+20I[0,4,3,2]-40I[0,4,4,0]+40I[0,4,4,1]+20I[0,4,5,0]$
$\displaystyle-I[0,5,3,0]+2I[0,5,4,0]-I[0,5,5,0]-80\big{(}I[1,3,3,0]-2I[1,3,3,1]+I[1,3,3,2]-2I[1,3,4,0]$
$\displaystyle+2I[1,3,4,1]+I[1,3,5,0]\big{)}\big{)}\Big{]},$ (18)

where

$\displaystyle{M^{2}}=\frac{M_{1}^{2}M_{2}^{2}}{M_{1}^{2}+M_{2}^{2}},~{}~{}~{}u_{0}=\frac{M_{1}^{2}}{M_{1}^{2}+M_{2}^{2}},$

with $M_{1}^{2}$ and $M_{2}^{2}$ being the Borel parameters in the initial and
final states, respectively. We set $M_{1}^{2}=M_{2}^{2}=2M^{2}$ as the initial
and final states are the same. We will fix $M^{2}$ and continuum threshold
$s_{0}$ based on the standard prescription of the LCSR method in the next
section. Here $P_{1}=\langle g_{s}^{2}G^{2}\rangle$ is gluon condensate,
$\chi$ stands for the magnetic susceptibility of the quark condensate, and
$P_{2}=\langle\bar{q}q\rangle$, $e_{q}$ and $m_{q}$ are the quark condensate,
electric charge and mass of the u, d or s-quark, respectively.

The functions $I[n,m,l,k]$ and $I_{i}[\mathcal{F}]$ are defined as:

$\displaystyle I[n,m,l,k]$
$\displaystyle=\int_{4m_{c}^{2}}^{s_{0}}ds\int_{0}^{1}dt\int_{0}^{1}dw~{}e^{-s/M^{2}}~{}s^{n}\,(s-4\,m_{c}^{2})^{m}\,t^{l}\,w^{k},$
$\displaystyle I_{1}[\mathcal{F}]$ $\displaystyle=\int
D_{\alpha_{i}}\int_{0}^{1}dv~{}\mathcal{F}(\alpha_{\bar{q}},\alpha_{q},\alpha_{g})\delta(\alpha_{q}+\bar{v}\alpha_{g}-u_{0}),$
$\displaystyle I_{2}[\mathcal{F}]$
$\displaystyle=\int_{0}^{1}du~{}\mathcal{F}(u),$

where $\mathcal{F}$ denotes the corresponding photon DAs.

##  III Numerical analysis and conclusion

In the present section, numerical calculations for the spin-$\frac{1}{2}$
bottom-charm baryons have been presented. The numerical values of the
parameters used in this section of the analysis are as follows:
$m_{u}=m_{d}=0$, $m_{s}=93.4^{+8.6}_{-3.4}$ MeV, $m_{c}=1.27\pm 0.02$ GeV,
$m_{b}=4.18^{+0.03}_{-0.02}$ GeV, [105], $\langle\bar{q}q\rangle=(-0.24\pm
0.01)^{3}$ GeV3 [106], $m_{0}^{2}=0.8\pm 0.12$ GeV2, $\langle
g_{s}^{2}G^{2}\rangle=0.88~{}GeV^{4}$ [107] and $\chi=-2.85\pm 0.5$ GeV2
[108]. The masses and residues of the bottom-charm baryons are borrowed from
Ref. [101]. Another set of main input parameters are the photon wavefunctions
of different twists, entering the DAs. These wavefunctions are borrowed from
Ref. [104]. It is worth noting that the photon DAs used in this study include
contributions from only light quarks. The contribution of DAs terms with heavy
quark has not yet been calculated, therefore these contributions have been
neglected in calculations.

As one can see from Eqs. (16) and (17), the magnetic dipole moments of the
bottom-charm baryons, in addition to the above-mentioned input parameters,
contain also two helping parameters: the continuum threshold $s_{0}$ and the
Borel mass parameter $M^{2}$. To carry out numerical computations, it is also
necessary to choose the working intervals for the parameters $s_{0}$ and
$M^{2}$. We apply the OPE convergence and pole dominance conditions to
determine the working intervals of $s_{0}$ and $M^{2}$. From this perspective,
we determine the value of these helping parameters within the interval $56.0$
GeV${}^{2}\leq s_{0}\leq 58.0$ GeV2 and $7.0$ GeV2 $\leq$ M2 $\leq$ $9.0$ GeV2
for $\Xi_{bc}$ baryons; and $57.0$ GeV${}^{2}\leq s_{0}\leq 59.0$ GeV2 and
$7.2$ GeV2 $\leq$ M2 $\leq$ $9.2$ GeV2 for $\Omega_{bc}$ baryon. By using all
the inputs as well as the working intervals of the helping parameters, we plot
the variation of the magnetic dipole moments with respect to the helping
parameters for the considered structure in Fig. 1. This figure indicates the
mild dependence of the magnetic dipole moments on the variations of the
helping parameters in their working intervals.

The obtained magnetic dipole moment results are given in Table I, and in the
same table predictions of other theoretical models for magnetic dipole moments
of spin-1/2 bottom-charm baryons are also presented. When the results obtained
using two different interpolating currents are examined, the different
diquark-quark structure does not change the magnitude of the magnetic dipole
moments much but causes the results obtained to be of opposite sign. The
absolute values of these results are consistent with each other within errors.
It should be noted that in our calculations, the light quark comes through the
two heavy quarks and gives the dominant contribution. Our final remark on the
magnetic dipole moment results will be about the $SU(3)_{f}$ symmetry
violation. Though the $SU(3)_{f}$ breaking effects have been considered via a
nonzero s-quark mass and strange quark condensate, we observe that $SU(3)_{f}$
symmetry violation in the magnetic dipole moments is quite small for both
interpolating currents.

We present the comparison of the results obtained using other models such as
MIT bag model [70, 67], nonrelativistic quark model (NQM) [96, 25],
hypercentral constituent quark model (HCQM) [56, 40, 76], effective quark mass
(EQM) and shielded quark charge scheme (SQCS) [109], relativistic harmonic
confinement model (RHM) [57], and heavy baryon chiral perturbation theory
(HB$\chi$PT) [66] with our estimations in the Table I. As one can see from
this Table, different theoretical models have yielded quite different
estimations for the magnetic dipole moments of bottom-charm baryons, which can
be used to distinguish between models. Obviously, more theoretical and
experimental research is needed to understand the current situation. But,
direct measurement of magnetic dipole moments of bottom-charm baryons seems
unlikely soon. Therefore, any indirect projections of the magnetic dipole
moments of the bottom-charm baryons would be quite helpful.

Table 1:  Results of the magnetic dipole moments for the bottom-charm baryons. Models |  $\mu_{\Xi_{bc}^{+}}[\mu_{N}]$ |  $\mu_{\Xi_{bc}^{0}}[\mu_{N}]$ |  $\mu_{\Omega_{bc}^{0}}[\mu_{N}]$
---|---|---|---
NQM [96] |  -0.198 |  0.058 |  0.009
NQM [25] |  -0.475 |  0.518 |  0.368
RHM [57] |  -0.52 |  0.63 |  0.49
EQM [109] |  -0.387 |  0.499 |  0.399
SCQS [109] |  -0.369 |  0.48 |  0.407
Bag model [70] |  -0.35 |  0.446 |  0.378
Bag model [67] |  -0.25 |  0.13 |  0.08
HCQM [40, 76] |  -0.204 |  0.354 |  0.439
HCQM [56] |  -0.400 |  0.477 |  0.397
HB$\chi$PT [66] |  -0.54 |  0.56 |  0.49
This work ($J_{1}$) |  $-0.50^{+0.14}_{-0.12}$ |  $0.39^{+0.06}_{-0.05}$ |  $0.38^{+0.05}_{-0.04}$
This work ($J_{2}$) |  $0.57^{+0.13}_{-0.12}$ |  $-0.29^{+0.07}_{-0.06}$ |  $-0.26^{+0.06}_{-0.05}$

##  IV Summary

The observation of the $\Xi^{++}_{cc}$ baryon by the LHCb Collaboration has
aroused great interest in doubly-heavy baryon systems. The measurement of the
features of doubly-heavy baryons ensures insight into both their inner
structure and production mechanism. In this work, we have studied the magnetic
dipole moments of the bottom-charm baryons within the framework of QCD light-
cone sum rules. While calculating the magnetic dipole moments of the bottom
charm baryons, two different possible interpolating currents that can be
coupled to these states have been considered. When the results obtained using
two different interpolating currents are examined, the different diquark-quark
structure does not change the magnitude of the magnetic dipole moments much
but causes the results obtained to be of opposite sign. Comparison of the
acquired results on the magnetic dipole moments of the $\Omega_{bc}^{0}$
baryon with those of $\Xi_{bc}^{0}$ baryon presents a small $SU(3)_{f}$
symmetry violation. The magnetic dipole moment results of the study are
compared with estimations acquired in various other models and approaches. The
magnetic dipole moments of the bottom-charm baryons contain crucial data on
their inner structure and geometric shape. Moreover, understanding the
electromagnetic properties of doubly-heavy baryons is the keys to confinement
and heavy flavor effects. Future experimental efforts on the properties of
doubly heavy baryons may figure out inconsistencies among different model
estimations. We believe that our results will be helpful in future
experimental and theoretical attempts regarding doubly heavy baryons.

(a)  (b)

(c)  (d)

(e)  (f)

Figure 1:  The magnetic dipole moments of the bottom-charm baryons with
variations of Borel parameter $M^{2}$ at the different values of the continuum
threshold parameter $s_{0}$: (a), (c) and (e) for the $J_{1}$ current, (b),
(d) and (f) for the $J_{2}$ current.

## References

  * [1] M. Mattson, et al., First Observation of the Doubly Charmed Baryon $\Xi^{+}_{cc}$, Phys. Rev. Lett. 89 (2002) 112001.  arXiv:hep-ex/0208014, doi:10.1103/PhysRevLett.89.112001.
  * [2] S. P. Ratti, New results on c-baryons and a search for cc-baryons in FOCUS, Nucl. Phys. B Proc. Suppl. 115 (2003) 33–36.  doi:10.1016/S0920-5632(02)01948-5.
  * [3] B. Aubert, et al., Search for doubly charmed baryons Xi(cc)+ and Xi(cc)++ in BABAR, Phys. Rev. D 74 (2006) 011103.  arXiv:hep-ex/0605075, doi:10.1103/PhysRevD.74.011103.
  * [4] R. Chistov, et al., Observation of new states decaying into Lambda(c)+ K- pi+ and Lambda(c)+ K0(S) pi-, Phys. Rev. Lett. 97 (2006) 162001.  arXiv:hep-ex/0606051, doi:10.1103/PhysRevLett.97.162001.
  * [5] R. Aaij, et al., Observation of the doubly charmed baryon $\Xi_{cc}^{++}$, Phys. Rev. Lett. 119 (11) (2017) 112001.  arXiv:1707.01621, doi:10.1103/PhysRevLett.119.112001.
  * [6] R. Aaij, et al., Search for the doubly heavy ${\Xi}_{bc}^{0}$ baryon via decays to $DpK$, JHEP 11 (2020) 095.  arXiv:2009.02481, doi:10.1007/JHEP11(2020)095.
  * [7] R. Aaij, et al., Search for the doubly heavy baryons $\Omega^{0}_{bc}$ and $\Xi^{0}_{bc}$ decaying to $\Lambda^{+}_{c}\pi^{-}$ and $\Xi^{+}_{c}\pi^{-}$, Chin. Phys. C 45 (9) (2021) 093002.  arXiv:2104.04759, doi:10.1088/1674-1137/ac0c70.
  * [8] Search for the doubly heavy baryon $\it{\Xi}_{bc}^{+}$ decaying to $J/\it{\psi}\it{\Xi}_{c}^{+}$ (4 2022).  arXiv:2204.09541.
  * [9] E. Bagan, M. Chabab, S. Narison, Baryons with two heavy quarks from QCD spectral sum rules, Phys. Lett. B 306 (1993) 350–356.  doi:10.1016/0370-2693(93)90090-5.
  * [10] R. Roncaglia, D. B. Lichtenberg, E. Predazzi, Predicting the masses of baryons containing one or two heavy quarks, Phys. Rev. D 52 (1995) 1722–1725.  arXiv:hep-ph/9502251, doi:10.1103/PhysRevD.52.1722.
  * [11] D. Ebert, R. N. Faustov, V. O. Galkin, A. P. Martynenko, V. A. Saleev, Heavy baryons in the relativistic quark model, Z. Phys. C 76 (1997) 111–115.  arXiv:hep-ph/9607314, doi:10.1007/s002880050534.
  * [12] S.-P. Tong, Y.-B. Ding, X.-H. Guo, H.-Y. Jin, X.-Q. Li, P.-N. Shen, R. Zhang, Spectra of baryons containing two heavy quarks in potential model, Phys. Rev. D 62 (2000) 054024.  arXiv:hep-ph/9910259, doi:10.1103/PhysRevD.62.054024.
  * [13] C. Itoh, T. Minamikawa, K. Miura, T. Watanabe, Doubly charmed baryon masses and quark wave functions in baryons, Phys. Rev. D 61 (2000) 057502.  doi:10.1103/PhysRevD.61.057502.
  * [14] S. S. Gershtein, V. V. Kiselev, A. K. Likhoded, A. I. Onishchenko, Spectroscopy of doubly heavy baryons, Phys. Rev. D 62 (2000) 054021.  doi:10.1103/PhysRevD.62.054021.
  * [15] V. V. Kiselev, A. K. Likhoded, Baryons with two heavy quarks, Phys. Usp. 45 (2002) 455–506.  arXiv:hep-ph/0103169, doi:10.1070/PU2002v045n05ABEH000958.
  * [16] V. V. Kiselev, A. K. Likhoded, O. N. Pakhomova, V. A. Saleev, Mass spectra of doubly heavy Omega $QQ^{\prime}$ baryons, Phys. Rev. D 66 (2002) 034030.  arXiv:hep-ph/0206140, doi:10.1103/PhysRevD.66.034030.
  * [17] I. M. Narodetskii, M. A. Trusov, The Heavy baryons in the nonperturbative string approach, Phys. Atom. Nucl. 65 (2002) 917–924.  arXiv:hep-ph/0104019, doi:10.1134/1.1481486.
  * [18] R. Lewis, N. Mathur, R. M. Woloshyn, Charmed baryons in lattice QCD, Phys. Rev. D 64 (2001) 094509.  arXiv:hep-ph/0107037, doi:10.1103/PhysRevD.64.094509.
  * [19] D. Ebert, R. N. Faustov, V. O. Galkin, A. P. Martynenko, Mass spectra of doubly heavy baryons in the relativistic quark model, Phys. Rev. D 66 (2002) 014008\.  arXiv:hep-ph/0201217, doi:10.1103/PhysRevD.66.014008.
  * [20] N. Mathur, R. Lewis, R. M. Woloshyn, Charmed and bottom baryons from lattice NRQCD, Phys. Rev. D 66 (2002) 014502.  arXiv:hep-ph/0203253, doi:10.1103/PhysRevD.66.014502.
  * [21] J. M. Flynn, F. Mescia, A. S. B. Tariq, Spectroscopy of doubly charmed baryons in lattice QCD, JHEP 07 (2003) 066.  arXiv:hep-lat/0307025, doi:10.1088/1126-6708/2003/07/066.
  * [22] J. Vijande, H. Garcilazo, A. Valcarce, F. Fernandez, Spectroscopy of doubly charmed baryons, Phys. Rev. D 70 (2004) 054022.  arXiv:hep-ph/0408274, doi:10.1103/PhysRevD.70.054022.
  * [23] T.-W. Chiu, T.-H. Hsieh, Baryon masses in lattice QCD with exact chiral symmetry, Nucl. Phys. A 755 (2005) 471–474.  arXiv:hep-lat/0501021, doi:10.1016/j.nuclphysa.2005.03.090.
  * [24] S. Migura, D. Merten, B. Metsch, H.-R. Petry, Charmed baryons in a relativistic quark model, Eur. Phys. J. A 28 (2006) 41.  arXiv:hep-ph/0602153, doi:10.1140/epja/i2006-10017-9.
  * [25] C. Albertus, E. Hernandez, J. Nieves, J. M. Verde-Velasco, Static properties and semileptonic decays of doubly heavy baryons in a nonrelativistic quark model, Eur. Phys. J. A 32 (2007) 183–199, [Erratum: Eur.Phys.J.A 36, 119 (2008)].  arXiv:hep-ph/0610030, doi:10.1140/epja/i2007-10364-y.
  * [26] A. P. Martynenko, Ground-state triply and doubly heavy baryons in a relativistic three-quark model, Phys. Lett. B 663 (2008) 317–321.  arXiv:0708.2033, doi:10.1016/j.physletb.2008.04.030.
  * [27] L. Tang, X.-H. Yuan, C.-F. Qiao, X.-Q. Li, Study of Doubly Heavy Baryon Spectrum via QCD Sum Rules, Commun. Theor. Phys. 57 (2012) 435–444.  arXiv:1104.4934, doi:10.1088/0253-6102/57/3/15.
  * [28] X. Liu, H.-X. Chen, Y.-R. Liu, A. Hosaka, S.-L. Zhu, Bottom baryons, Phys. Rev. D 77 (2008) 014031.  arXiv:0710.0123, doi:10.1103/PhysRevD.77.014031.
  * [29] W. Roberts, M. Pervin, Heavy baryons in a quark model, Int. J. Mod. Phys. A 23 (2008) 2817–2860.  arXiv:0711.2492, doi:10.1142/S0217751X08041219.
  * [30] A. Valcarce, H. Garcilazo, J. Vijande, Towards an understanding of heavy baryon spectroscopy, Eur. Phys. J. A 37 (2008) 217–225.  arXiv:0807.2973, doi:10.1140/epja/i2008-10616-4.
  * [31] L. Liu, H.-W. Lin, K. Orginos, A. Walker-Loud, Singly and Doubly Charmed J=1/2 Baryon Spectrum from Lattice QCD, Phys. Rev. D 81 (2010) 094505.  arXiv:0909.3294, doi:10.1103/PhysRevD.81.094505.
  * [32] C. Alexandrou, J. Carbonell, D. Christaras, V. Drach, M. Gravina, M. Papinutto, Strange and charm baryon masses with two flavors of dynamical twisted mass fermions, Phys. Rev. D 86 (2012) 114501.  arXiv:1205.6856, doi:10.1103/PhysRevD.86.114501.
  * [33] T. M. Aliev, K. Azizi, M. Savci, Doubly Heavy Spin–1/2 Baryon Spectrum in QCD, Nucl. Phys. A 895 (2012) 59–70.  arXiv:1205.2873, doi:10.1016/j.nuclphysa.2012.09.009.
  * [34] T. M. Aliev, K. Azizi, M. Savci, The masses and residues of doubly heavy spin-3/2 baryons, J. Phys. G 40 (2013) 065003.  arXiv:1208.1976, doi:10.1088/0954-3899/40/6/065003.
  * [35] Y. Namekawa, et al., Charmed baryons at the physical point in 2+1 flavor lattice QCD, Phys. Rev. D 87 (9) (2013) 094512.  arXiv:1301.4743, doi:10.1103/PhysRevD.87.094512.
  * [36] M. Karliner, J. L. Rosner, Baryons with two heavy quarks: Masses, production, decays, and detection, Phys. Rev. D 90 (9) (2014) 094007.  arXiv:1408.5877, doi:10.1103/PhysRevD.90.094007.
  * [37] Z.-F. Sun, Z.-W. Liu, X. Liu, S.-L. Zhu, Masses and axial currents of the doubly charmed baryons, Phys. Rev. D 91 (9) (2015) 094030.  arXiv:1411.2117, doi:10.1103/PhysRevD.91.094030.
  * [38] H.-X. Chen, W. Chen, Q. Mao, A. Hosaka, X. Liu, S.-L. Zhu, P-wave charmed baryons from QCD sum rules, Phys. Rev. D 91 (5) (2015) 054034.  arXiv:1502.01103, doi:10.1103/PhysRevD.91.054034.
  * [39] Z.-F. Sun, M. J. Vicente Vacas, Masses of doubly charmed baryons in the extended on-mass-shell renormalization scheme, Phys. Rev. D 93 (9) (2016) 094002\.  arXiv:1602.04714, doi:10.1103/PhysRevD.93.094002.
  * [40] Z. Shah, K. Thakkar, A. K. Rai, Excited State Mass spectra of doubly heavy baryons $\Omega_{cc}$, $\Omega_{bb}$ and $\Omega_{bc}$, Eur. Phys. J. C 76 (10) (2016) 530.  arXiv:1609.03030, doi:10.1140/epjc/s10052-016-4379-z.
  * [41] V. V. Kiselev, A. V. Berezhnoy, A. K. Likhoded, Quark–Diquark Structure and Masses of Doubly Charmed Baryons, Phys. Atom. Nucl. 81 (3) (2018) 369–372.  arXiv:1706.09181, doi:10.1134/S1063778818030134.
  * [42] H.-X. Chen, Q. Mao, W. Chen, X. Liu, S.-L. Zhu, Establishing low-lying doubly charmed baryons, Phys. Rev. D 96 (3) (2017) 031501, [Erratum: Phys.Rev.D 96, 119902 (2017)].  arXiv:1707.01779, doi:10.1103/PhysRevD.96.031501.
  * [43] J. Hu, T. Mehen, Chiral Lagrangian with heavy quark-diquark symmetry, Phys. Rev. D 73 (2006) 054003.  arXiv:hep-ph/0511321, doi:10.1103/PhysRevD.73.054003.
  * [44] L. Meng, N. Li, S.-L. Zhu, Deuteron-like states composed of two doubly charmed baryons, Phys. Rev. D 95 (11) (2017) 114019.  arXiv:1704.01009, doi:10.1103/PhysRevD.95.114019.
  * [45] S. Narison, R. Albuquerque, Mass-splittings of doubly heavy baryons in QCD, Phys. Lett. B 694 (2011) 217–225.  arXiv:1006.2091, doi:10.1016/j.physletb.2010.09.051.
  * [46] J.-R. Zhang, M.-Q. Huang, Doubly heavy baryons in QCD sum rules, Phys. Rev. D 78 (2008) 094007.  arXiv:0810.5396, doi:10.1103/PhysRevD.78.094007.
  * [47] Z.-H. Guo, Prediction of exotic doubly charmed baryons within chiral effective field theory, Phys. Rev. D 96 (7) (2017) 074004.  arXiv:1708.04145, doi:10.1103/PhysRevD.96.074004.
  * [48] Q.-F. Lü, K.-L. Wang, L.-Y. Xiao, X.-H. Zhong, Mass spectra and radiative transitions of doubly heavy baryons in a relativized quark model, Phys. Rev. D 96 (11) (2017) 114006.  arXiv:1708.04468, doi:10.1103/PhysRevD.96.114006.
  * [49] L.-Y. Xiao, K.-L. Wang, Q.-f. Lu, X.-H. Zhong, S.-L. Zhu, Strong and radiative decays of the doubly charmed baryons, Phys. Rev. D 96 (9) (2017) 094005.  arXiv:1708.04384, doi:10.1103/PhysRevD.96.094005.
  * [50] X.-Z. Weng, X.-L. Chen, W.-Z. Deng, Masses of doubly heavy-quark baryons in an extended chromomagnetic model, Phys. Rev. D 97 (5) (2018) 054008.  arXiv:1801.08644, doi:10.1103/PhysRevD.97.054008.
  * [51] K. U. Can, G. Erkol, B. Isildak, M. Oka, T. T. Takahashi, Electromagnetic properties of doubly charmed baryons in Lattice QCD, Phys. Lett. B 726 (2013) 703–709.  arXiv:1306.0731, doi:10.1016/j.physletb.2013.09.024.
  * [52] T. Branz, A. Faessler, T. Gutsche, M. A. Ivanov, J. G. Korner, V. E. Lyubovitskij, B. Oexl, Radiative decays of double heavy baryons in a relativistic constituent three–quark model including hyperfine mixing, Phys. Rev. D 81 (2010) 114036.  arXiv:1005.1850, doi:10.1103/PhysRevD.81.114036.
  * [53] S. K. Bose, L. P. Singh, Magnetic Moments of Charmed and $B$ Flavored Hadrons in MIT Bag Model, Phys. Rev. D 22 (1980) 773.  doi:10.1103/PhysRevD.22.773.
  * [54] B. Patel, A. K. Rai, P. C. Vinodkumar, Masses and Magnetic Moments of Charmed Baryons Using Hyper Central Model, in: 12th International Conference on Hadron Spectroscopy, 2008.  arXiv:0803.0221.
  * [55] B. Silvestre-Brac, Spectrum and static properties of heavy baryons, Few Body Syst. 20 (1996) 1–25.  doi:10.1007/s006010050028.
  * [56] B. Patel, A. K. Rai, P. C. Vinodkumar, Masses and magnetic moments of heavy flavour baryons in hyper central model, J. Phys. G 35 (2008) 065001.  arXiv:0710.3828, doi:10.1088/0954-3899/35/6/065001.
  * [57] A. N. Gadaria, N. R. Soni, J. N. Pandya, Masses and magnetic moment of doubly heavy baryons, DAE Symp. Nucl. Phys. 61 (2016) 698–699.
  * [58] H. Liu, Y.-H. Zou, Y.-R. Liu, S.-Z. Jiang, Chiral Lagrangians for spin-$\frac{1}{2}$ and spin-$\frac{3}{2}$ doubly charmed baryons (4 2023).  arXiv:2304.04575.
  * [59] H.-Z. Tong, H.-S. Li, Chiral corrections to the masses of the doubly heavy baryons, Commun. Theor. Phys. 74 (8) (2022) 085201.  arXiv:2110.01380, doi:10.1088/1572-9494/ac78d7.
  * [60] Z.-Y. Li, G.-L. Yu, Z.-G. Wang, J.-Z. Gu, H.-T. Shen, Mass spectra of bottom-charm baryons (11 2022).  arXiv:2211.15111.
  * [61] H.-T. Li, X.-C. Zheng, J. Yan, X.-G. Wu, G. Chen, Hadronic production of $\Xi$bc with the intrinsic heavy-quark content at a fixed-target experiment at the LHC, Phys. Rev. D 106 (11) (2022) 114030.  arXiv:2211.06085, doi:10.1103/PhysRevD.106.114030.
  * [62] Z.-Y. Li, G.-L. Yu, Z.-G. Wang, J.-Z. Gu, H.-T. Shen, Mass spectra of double-bottom baryons (10 2022).  arXiv:2210.13085.
  * [63] B. Julia-Diaz, D. O. Riska, Baryon magnetic moments in relativistic quark models, Nucl. Phys. A 739 (2004) 69–88.  arXiv:hep-ph/0401096, doi:10.1016/j.nuclphysa.2004.03.078.
  * [64] A. Faessler, T. Gutsche, M. A. Ivanov, J. G. Korner, V. E. Lyubovitskij, D. Nicmorus, K. Pumsa-ard, Magnetic moments of heavy baryons in the relativistic three-quark model, Phys. Rev. D 73 (2006) 094013.  arXiv:hep-ph/0602193, doi:10.1103/PhysRevD.73.094013.
  * [65] K. U. Can, G. Erkol, B. Isildak, M. Oka, T. T. Takahashi, Electromagnetic structure of charmed baryons in Lattice QCD, JHEP 05 (2014) 125.  arXiv:1310.5915, doi:10.1007/JHEP05(2014)125.
  * [66] H.-S. Li, L. Meng, Z.-W. Liu, S.-L. Zhu, Magnetic moments of the doubly charmed and bottom baryons, Phys. Rev. D 96 (7) (2017) 076011.  arXiv:1707.02765, doi:10.1103/PhysRevD.96.076011.
  * [67] A. Bernotas, V. Simonis, Magnetic moments of heavy baryons in the bag model reexamined (9 2012).  arXiv:1209.2900, doi:10.3952/physics.v53i2.2668.
  * [68] D. B. Lichtenberg, Magnetic Moments of Charmed Baryons in the Quark Model, Phys. Rev. D 15 (1977) 345.  doi:10.1103/PhysRevD.15.345.
  * [69] Y.-s. Oh, D.-P. Min, M. Rho, N. N. Scoccola, Massive quark baryons as skyrmions: Magnetic moments, Nucl. Phys. A 534 (1991) 493–512.  doi:10.1016/0375-9474(91)90458-I.
  * [70] V. Simonis, Improved predictions for magnetic moments and M1 decay widths of heavy hadrons (3 2018).  arXiv:1803.01809.
  * [71] M.-Z. Liu, Y. Xiao, L.-S. Geng, Magnetic moments of the spin-1/2 doubly charmed baryons in covariant baryon chiral perturbation theory, Phys. Rev. D 98 (1) (2018) 014040.  arXiv:1807.00912, doi:10.1103/PhysRevD.98.014040.
  * [72] A. N. Hiller Blin, Z.-F. Sun, M. J. Vicente Vacas, Electromagnetic form factors of spin 1/2 doubly charmed baryons, Phys. Rev. D 98 (5) (2018) 054025\.  arXiv:1807.01059, doi:10.1103/PhysRevD.98.054025.
  * [73] L. Meng, H.-S. Li, Z.-W. Liu, S.-L. Zhu, Magnetic moments of the spin-${3\over 2}$ doubly heavy baryons, Eur. Phys. J. C 77 (12) (2017) 869.  arXiv:1710.08283, doi:10.1140/epjc/s10052-017-5447-8.
  * [74] R. Dhir, R. C. Verma, Magnetic Moments of (J**P = 3/2+) Heavy Baryons Using Effective Mass Scheme, Eur. Phys. J. A 42 (2009) 243–249.  arXiv:0904.2124, doi:10.1140/epja/i2009-10872-8.
  * [75] B. Mohan, T. M. S., A. Hazra, R. Dhir, Screening of the quark charge and mixing effects on transition moments and M1 decay widths of baryons, Phys. Rev. D 106 (11) (2022) 113007.  arXiv:2211.16418, doi:10.1103/PhysRevD.106.113007.
  * [76] Z. Shah, A. K. Rai, Excited state mass spectra of doubly heavy $\Xi$ baryons, Eur. Phys. J. C 77 (2) (2017) 129.  arXiv:1702.02726, doi:10.1140/epjc/s10052-017-4688-x.
  * [77] U. Özdem, Magnetic moments of doubly heavy baryons in light-cone QCD, J. Phys. G 46 (3) (2019) 035003.  arXiv:1804.10921, doi:10.1088/1361-6471/aafffc.
  * [78] U. Özdem, Magnetic dipole moments of the spin-$\frac{3}{2}$ doubly heavy baryons, Eur. Phys. J. A 56 (2) (2020) 34.  arXiv:1906.08353, doi:10.1140/epja/s10050-020-00049-4.
  * [79] H. Bahtiyar, Electromagnetic Structure of Spin-$\frac{1}{2}$ Doubly Charmed Baryons in Lattice QCD (9 2022).  arXiv:2209.05361.
  * [80] H. Mutuk, The status of $\Xi_{\mathrm{{cc}}}^{++}$ baryon: investigating quark–diquark model, Eur. Phys. J. Plus 137 (1) (2022) 10.  arXiv:2112.06205, doi:10.1140/epjp/s13360-021-02256-4.
  * [81] R.-X. Shi, L.-S. Geng, Magnetic moments of the spin-$\frac{3}{2}$ doubly charmed baryons in covariant baryon chiral perturbation theory, Phys. Rev. D 103 (11) (2021) 114004.  arXiv:2103.07260, doi:10.1103/PhysRevD.103.114004.
  * [82] A. Hazra, S. Rakshit, R. Dhir, Radiative M1 transitions of heavy baryons: Effective quark mass scheme, Phys. Rev. D 104 (5) (2021) 053002.  arXiv:2108.01840, doi:10.1103/PhysRevD.104.053002.
  * [83] H.-S. Li, L. Meng, Z.-W. Liu, S.-L. Zhu, Radiative decays of the doubly charmed baryons in chiral perturbation theory, Phys. Lett. B 777 (2018) 169–176.  arXiv:1708.03620, doi:10.1016/j.physletb.2017.12.031.
  * [84] F.-S. Yu, H.-Y. Jiang, R.-H. Li, C.-D. Lü, W. Wang, Z.-X. Zhao, Discovery Potentials of Doubly Charmed Baryons, Chin. Phys. C 42 (5) (2018) 051001.  arXiv:1703.09086, doi:10.1088/1674-1137/42/5/051001.
  * [85] E.-L. Cui, H.-X. Chen, W. Chen, X. Liu, S.-L. Zhu, Suggested search for doubly charmed baryons of $J^{P}=3/2^{+}$ via their electromagnetic transitions, Phys. Rev. D 97 (3) (2018) 034018.  arXiv:1712.03615, doi:10.1103/PhysRevD.97.034018.
  * [86] A. R. Olamaei, K. Azizi, S. Rostami, Strong coupling constants of the doubly heavy $\Xi_{QQ}$ Baryons with $\pi$ Meson, Eur. Phys. J. C 80 (7) (2020) 613.  arXiv:2003.12723, doi:10.1140/epjc/s10052-020-8194-1.
  * [87] S. Rostami, K. Azizi, A. R. Olamaei, Strong Coupling Constants of the Doubly Heavy Spin-1/2 Baryons with Light Pseudoscalar Mesons, Chin. Phys. C 45 (2) (2021) 023120.  arXiv:2008.12715, doi:10.1088/1674-1137/abd084.
  * [88] H.-Z. He, W. Liang, Q.-F. Lü, Strong decays of the low-lying doubly bottom baryons, Phys. Rev. D 105 (1) (2022) 014010.  arXiv:2106.11045, doi:10.1103/PhysRevD.105.014010.
  * [89] K. Azizi, A. R. Olamaei, S. Rostami, Strong interaction of doubly heavy spin-3/2 baryons with light vector mesons, Eur. Phys. J. C 80 (12) (2020) 1196\.  arXiv:2011.02919, doi:10.1140/epjc/s10052-020-08770-6.
  * [90] R.-H. Li, C.-D. Lü, W. Wang, F.-S. Yu, Z.-T. Zou, Doubly-heavy baryon weak decays: $\Xi_{bc}^{0}\to pK^{-}$ and $\Xi_{cc}^{+}\to\Sigma_{c}^{++}(2520)K^{-}$, Phys. Lett. B 767 (2017) 232–235.  arXiv:1701.03284, doi:10.1016/j.physletb.2017.02.003.
  * [91] W. Wang, F.-S. Yu, Z.-X. Zhao, Weak decays of doubly heavy baryons: the $1/2\rightarrow 1/2$ case, Eur. Phys. J. C 77 (11) (2017) 781.  arXiv:1707.02834, doi:10.1140/epjc/s10052-017-5360-1.
  * [92] W. Wang, Z.-P. Xing, J. Xu, Weak Decays of Doubly Heavy Baryons: SU(3) Analysis, Eur. Phys. J. C 77 (11) (2017) 800.  arXiv:1707.06570, doi:10.1140/epjc/s10052-017-5363-y.
  * [93] Y.-J. Shi, W. Wang, Y. Xing, J. Xu, Weak Decays of Doubly Heavy Baryons: Multi-body Decay Channels, Eur. Phys. J. C 78 (1) (2018) 56.  arXiv:1712.03830, doi:10.1140/epjc/s10052-018-5532-7.
  * [94] C.-Q. Geng, C.-W. Liu, A. Zhou, X. Yu, Semileptonic decays of doubly charmed baryons in the bag model, Phys. Rev. D 107 (5) (2023) 053008.  arXiv:2211.04372, doi:10.1103/PhysRevD.107.053008.
  * [95] G.-H. Yang, E.-P. Liang, Q. Qin, K.-K. Shao, Inclusive weak-annihilation decays and lifetimes of beauty-charmed baryons, Phys. Rev. D 106 (9) (2022) 093013\.  arXiv:2208.06834, doi:10.1103/PhysRevD.106.093013.
  * [96] B. Silvestre-Brac, Spectrum and static properties of heavy baryons, Few Body Syst. 20 (1996) 1–25.  doi:10.1007/s006010050028.
  * [97] Q. Qin, Y.-J. Shi, W. Wang, G.-H. Yang, F.-S. Yu, R. Zhu, Inclusive approach to hunt for the beauty-charmed baryons $\Xi_{bc}$, Phys. Rev. D 105 (3) (2022) L031902.  arXiv:2108.06716, doi:10.1103/PhysRevD.105.L031902.
  * [98] V. L. Chernyak, I. R. Zhitnitsky, B meson exclusive decays into baryons, Nucl. Phys. B 345 (1990) 137–172.  doi:10.1016/0550-3213(90)90612-H.
  * [99] V. M. Braun, I. E. Filyanov, QCD Sum Rules in Exclusive Kinematics and Pion Wave Function, Z. Phys. C 44 (1989) 157.  doi:10.1007/BF01548594.
  * [100] I. I. Balitsky, V. M. Braun, A. V. Kolesnichenko, Radiative Decay Sigma+ —$>$ p gamma in Quantum Chromodynamics, Nucl. Phys. B 312 (1989) 509–550.  doi:10.1016/0550-3213(89)90570-1.
  * [101] Z.-G. Wang, Q. Xin, Analysis of the bottom-charm baryon states with QCD sum rules, Int. J. Mod. Phys. A 37 (11n12) (2022) 2250074.  arXiv:2202.03828, doi:10.1142/S0217751X22500749.
  * [102] K.-C. Yang, W. Y. P. Hwang, E. M. Henley, L. S. Kisslinger, QCD sum rules and neutron proton mass difference, Phys. Rev. D 47 (1993) 3001–3012.  doi:10.1103/PhysRevD.47.3001.
  * [103] V. M. Belyaev, B. Y. Blok, CHARMED BARYONS IN QUANTUM CHROMODYNAMICS, Z. Phys. C 30 (1986) 151.  doi:10.1007/BF01560689.
  * [104] P. Ball, V. M. Braun, N. Kivel, Photon distribution amplitudes in QCD, Nucl. Phys. B 649 (2003) 263–296.  arXiv:hep-ph/0207307, doi:10.1016/S0550-3213(02)01017-9.
  * [105] R. L. Workman, et al., Review of Particle Physics, PTEP 2022 (2022) 083C01.  doi:10.1093/ptep/ptac097.
  * [106] B. L. Ioffe, QCD at low energies, Prog. Part. Nucl. Phys. 56 (2006) 232–277.  arXiv:hep-ph/0502148, doi:10.1016/j.ppnp.2005.05.001.
  * [107] R. D. Matheus, S. Narison, M. Nielsen, J. M. Richard, Can the X(3872) be a 1++ four-quark state?, Phys. Rev. D 75 (2007) 014005.  arXiv:hep-ph/0608297, doi:10.1103/PhysRevD.75.014005.
  * [108] J. Rohrwild, Determination of the magnetic susceptibility of the quark condensate using radiative heavy meson decays, JHEP 09 (2007) 073.  arXiv:0708.1405, doi:10.1088/1126-6708/2007/09/073.
  * [109] R. Dhir, C. S. Kim, R. C. Verma, Magnetic Moments of Bottom Baryons: Effective mass and Screened Charge, Phys. Rev. D 88 (2013) 094002.  arXiv:1309.4057, doi:10.1103/PhysRevD.88.094002.


acronymindexonlyfirsttrue

# Learning Risk-Aware Quadrupedal Locomotion using Distributional
Reinforcement Learning

Lukas Schneider
ETH Zurich
<EMAIL_ADDRESS>
&Jonas Frey
ETH Zurich
<EMAIL_ADDRESS>
Takahiro Miki
ETH Zurich
<EMAIL_ADDRESS>
&Marco Hutter
ETH Zurich
<EMAIL_ADDRESS>
Lukas Schneider1, Jonas Frey1,2, Takahiro Miki1, and Marco Hutter1 1 Robotic
Systems Lab, ETH Zurich, Zurich, Switzerland2 Max Planck Institute for
Intelligent Systems, Tübingen, Germany.This work was supported by the Swiss
National Science Foundation (SNSF) through project 188596, the National Centre
of Competence in Research Robotics (NCCR Robotics), the European Union’s
Horizon 2020 research and innovation program under grant agreement No
101016970, No 101070405, and No 852044, and an ETH Zurich Research Grant.
Jonas Frey is supported by the Max Planck ETH Center for Learning Systems.

###### Abstract

Deployment in hazardous environments requires robots to understand the risks
associated with their actions and movements to prevent accidents. Despite its
importance, these risks are not explicitly modeled by currently deployed
locomotion controllers for legged robots. In this work, we propose a risk
sensitive locomotion training method employing distributional reinforcement
learning to consider safety explicitly. Instead of relying on a value
expectation, we estimate the complete value distribution to account for
uncertainty in the robot’s interaction with the environment. The value
distribution is consumed by a risk metric to extract risk sensitive value
estimates. These are integrated into Proximal Policy Optimization (PPO) to
derive our method, Distributional Proximal Policy Optimization (DPPO). The
risk preference, ranging from risk-averse to risk-seeking, can be controlled
by a single parameter, which enables to adjust the robot’s behavior
dynamically. Importantly, our approach removes the need for additional reward
function tuning to achieve risk sensitivity. We show emergent risk sensitive
locomotion behavior in simulation and on the quadrupedal robot ANYmal.

##  I Introduction

Figure 1: Our robot learns to adapt its locomotion behavior in risky
situations. When commanded to walk up a large step, the risk-averse policy
($\blacksquare$) refuses while the risk-seeking policy ($\blacksquare$)
complies. The risk parameter, controlling the value distribution distortion,
can be adapted online during deployment. Right: Respective risk-metric
distorted value distribution per robot.

Legged robots can traverse rugged terrain inaccessible to wheeled or tracked
systems. They can overcome sloped and uneven terrain, stones, stairs, and even
large gaps by carefully choosing their foot placement. This versatility makes
them ideal for operation in hazardous environments, such as cave systems [9,
49], forests [15, 14], or the surfaces of other planets [12, 22]. In these
situations, the safe operation of the robot is paramount, as failure could
result in hardware damage or mission failure.

Recent developments in model-free deep Reinforcement Learning (RL) [40, 39]
have enabled legged robots to traverse difficult terrain reliably [24, 30].
While these systems are robust against perception failures, a notable
limitation remains: they do not account for risk explicitly. Encoding
behavior, such as avoiding dangerous obstacles or reducing locomotion speed on
rough ground, requires adaptation of the reward formulation. Such adaptations
are undesirable as learning a reliable locomotion already requires expensive
and time-consuming reward function tuning.

In this work, we overcome this limitation by adapting the RL algorithm to
allow risk sensitive behavior to arise without changing the reward
formulation. We adopt the distributional perspective on RL [4], learning the
entire distribution of returns instead of just its scalar expectation. This
value distribution models the intrinsic uncertainty of the agent’s interaction
with its environment. While estimating the full distribution was initially
proposed for more accurate value estimates, it also captures useful
information, such as the probability of catastrophic and high-return events.
We use the value distribution in the actor-critic framework [2] by
incorporating risk sensitive value estimates into Generalized Advantage
Estimation (GAE) [38]. To extract these scalar value estimates, we employ a
risk metric [27] that allows us to emphasize a specific part of the value
distribution, for example, unlikely but catastrophic events. The learned
locomotion policy accounts for risks explicitly. Since the metric encodes a
fixed risk preference based on a metric parameter, and no single risk
preference is appropriate under all circumstances, we condition the policy on
this parameter [47]. Conditioning the policy allows the adjustment of the
robot’s behavior to either seek or avoid risks on demand by an operator or a
high-level planner. For example, it enables the safe teleoperation of the
robot by an amateur in risk-averse mode while recovering the full range of
capabilities in risk-neutral / seeking mode. Figure 1 visualizes the behavior
of different risk sensitivities of a single policy.

We thoroughly analyze the risk sensitive behavior arising from our method and
study the effects of using different risk metrics and parameters in
simulation. We further ablate algorithmic augmentations and deploy the trained
policy to the quadrupedal ANYmal robot [20], demonstrating risk sensitive
behavior based on the risk parameter when climbing steps of varying heights.

Our contribution is threefold. Firstly, we integrate explicit risk modeling
into the locomotion controller, a capability that currently deployed
controllers do not possess. Secondly, we study the emergent risk sensitive
locomotion behavior in simulation and on hardware. Lastly, on the algorithmic
side, we explore introducing risk sensitivity into the RL formulation through
risk sensitive advantage estimates extracted from the value distribution.

##  II Related Work

Figure 2: Architecture overview. The critic learns to predict a value
distribution, used in combination with a risk metric to update the policy. The
policy is conditioned on the risk parameter. The risk parameter is part of the
command, set by the operator.

Legged Locomotion Legged locomotion is a thoroughly studied research area in
robotics. Early successes have been achieved by model-based methods [20, 7].
However, due to their limited ability to generalize to more complex and
unpredictable environments, learned methods have recently gained interest.
Several deep RL algorithms have solved simulated legged locomotion tasks [40,
39, 25, 31]. Beyond that, RL policies learned in simulation have been
successfully transferred to real-world scenarios, either through high
simulation fidelity and data augmentation [36, 1, 30, 52, 46, 24, 21] or test-
time adaptation [42].

Distributional Reinforcement Learning Distributional Reinforcement Learning
(Dist. RL) algorithms learn a value distribution, as defined by the recursive
distributional Bellman Equation [4], which replaces the role of the value
function in classical RL. Value-based Dist. RL algorithms [4, 10, 11, 53] use
this value distribution for value-iteration, commonly maximizing its
expectation when comparing different actions. These methods differ primarily
in how they represent the value distribution: as a categorical distribution
[4], by its cumulative distribution at fixed quantiles [10], or at arbitrary
quantiles [11, 53]. Generally, algorithms with more accurate representations
achieve better results. To extend Dist. RL to continuous actions, the value
distribution is used as the critic in an actor-critic architecture [2, 23, 26,
32, 41]. Several works explore Dist. RL theoretically [5, 44, 35, 34], use the
additional information contained in the value distribution to improve
exploration [28], or for risk sensitivity [11, 26, 47, 6, 43, 48]. Dist. RL
has been used for algorithm discovery [13] and as an AI racing agent in the
Gran Turismo 7 game [51]. Further, Dist. RL control policies have been
successfully employed in the real world: for robotic grasping in a lab
environment [8], robot soccer [18], and stratospheric balloon navigation [3].

Safe Reinforcement Learning Safe RL is a thoroughly studied research area
[17]. The work of [47] is most closely related to ours, learning a risk-averse
policy with online adaptation of risk-aversion. They model the value function
as a Gaussian distribution and integrate it into an actor-critic architecture.
When updating the policy, they maximize the Conditional Value at Risk (CVaR)
metric and condition the policy on the confidence level. [37] extend this work
to adjust risk sensitivity automatically. Further, Distributional Soft Actor
Critic (DSAC) is a state-of-the-art risk sensitive RL algorithm that
integrates a value distribution into the maximum entropy objective of Soft
Actor Critic (SAC) [19]. Several other works explore risk metrics in Dist. RL
with fixed risk parameters [11, 6, 43, 48, 8].

##  III Method

Our goal is to learn a locomotion policy $\pi_{\phi}(a|s;\beta)$ that maps the
robot’s state $s$ to desired joint positions $a$, is conditioned on a risk
parameter $\beta$, and has learned weights $\phi$. The state consists of
proprioceptive measurements (base velocity, projected gravity, joint positions
and velocities), the previous actions, samples from a local height map around
the robot, and the input commands (desired linear and angular base
velocities). The risk parameter is a scalar value and depends on the metric
used during training. We use the robot dynamics model from [36], including
learned actuator dynamics. An overview of the architecture can be found in
Figure 2.

Distributional Proximal Policy Optimization (DPPO) We base our RL algorithm on
the Proximal Policy Optimization (PPO) algorithm [39] as it shows superior
performance for learning legged locomotion in our setup, compared to Deep
Deterministic Policy Gradients (DDPG) [25], Twin-Delayed DDPG (TD3) [16], SAC
[19], and DSAC [26] (Experiment A: Section IV-B).

Critic Representation As the distributional critic, we use the Quantile
Regression Deep Q-Network (QR-DQN) [10]. It parameterizes the value
distribution as a uniform distribution supported on
$\left\\{\theta_{1}(s),...,\theta_{N}(s)\right\\}$, which can be written as

$Z_{\theta}(s)=\frac{1}{N}\sum_{i=1}^{N}\delta_{\theta_{i}(s)},$ (1)

where $\delta_{z}$ is a Dirac at $z\in\mathbb{R}$. The support positions
$\theta_{i}(s)$ are state-dependent and predicted by the critic’s neural
network.

Critic Updates To compute reward-to-go value targets, we rely on the sample-
replacement strategy $\text{SR}(\lambda)$ introduced by [32]. This technique
extends the temporal-difference $\text{TD}(\lambda)$ method [45] for computing
multi-step value targets to Dist. RL. As the name suggests, it computes value
targets $\operatorname{\mathcal{T}}Z_{\theta}(s)$ as a mixture of sample
distributions. It therefore starts with samples of the distribution at the
final time-step $Z_{\theta}(s_{T})$. Then, going backwards in time
($t=T-1,...,0$), this sample distribution is discounted, shifted by the
obtained reward $r_{t}$, and a fraction $1-\lambda$ of samples is replaced
with samples of the distribution $Z_{\theta}(s_{t})$. $1$-step and $N$-step
target distributions are recovered by setting $\lambda=0$ and $\lambda=1$,
respectively. The critic is updated by minimizing the energy distance

$\mathcal{L}=2\operatorname{\mathbb{E}}_{i,j}\left[\theta_{i}-\operatorname{\mathcal{T}}\theta_{j}\right]-\operatorname{\mathbb{E}}_{i,j}\left[\operatorname{\mathcal{T}}\theta_{i}-\operatorname{\mathcal{T}}\theta_{j}\right]-\operatorname{\mathbb{E}}_{i,j}\left[\theta_{i}-\theta_{j}\right]$
(2)

between the $\text{SR}(\lambda)$ target distribution and samples from the
predicted critic distribution $Z_{\theta}(s)$.

Figure 3: Application of a risk metric. The risk sensitivity selects how the
value distribution is distorted. The module outputs the mean of the distorted
distribution.

Rewards We adapt the rewards from [36], which primarily reward command
tracking and penalize colliding / expending energy. We introduce an alive
reward and modify the collision penalties to be impact force-dependent.
Further, the joint motion and torque penalties are increased based on a
curriculum.

Risk Quantification To quantify the risk associated with different states, we
apply a risk metric to the value distribution $Z_{\theta}(s)$, as depicted in
Figure 3. We thereby follow [27] who argue for using distortion risk metrics,
a class of metrics with properties sensible for robotics applications.
Specifically, we use the Wang [50] metric, which encodes a subjective measure
of risk. We further use Conditional Value at Risk (CVaR) [33], which measures
the expected worst-case return (beyond a confidence level). For the
distribution produced by the QR-DQN, we compute these metrics as the
expectation under a distortion $g(\tau)$ of the distribution supports [26]. We
denote the quantile fractions of the distribution as $(\tau_{i})_{i=0,...,N}$
with $\tau_{i}=\frac{i}{N}$. The distorted expectation is computed as

$V_{\beta}(s)=\int_{0}^{1}g_{\beta}^{\prime}(\tau)Z_{\theta}^{\tau}(s)d\tau=\sum_{k=1}^{N}(g_{\beta}({\tau}_{k})-g_{\beta}(\tau_{k-1}))\theta_{k}(s)$
(3)

where we denote the $\tau$-quantile of the value distribution as
$Z_{\theta}^{\tau}(s)$. We distort the quantile fractions for CVaR as

$g_{\beta}^{\text{CVaR}}(\tau)=\min\left(\frac{\tau}{\beta},1.0\right)$ (4)

where $\beta$ is a scalar risk parameter. We choose $\beta=1$ for risk-
neutrality and $0<\beta<1$ for risk aversion. The CVaR metric, in effect,
computes the expectation of the tail of the value distribution, where $\beta$
decides the cutoff. For the Wang metric, we compute the distorted quantile
fractions as

$g_{\beta}^{\text{Wang}}(\tau)=\Phi(\Phi^{-1}(\tau)+\beta)$ (5)

where $\Phi$ is the standard normal distribution and $\beta$ is, again, the
scalar risk parameter. We choose $\beta=0$ for risk-neutrality, $\beta>0$ for
risk aversion, and $\beta<0$ for risk affinity.

For the CVaR metric, we train the policy on risk parameters in the range
$\beta\in[0,1]$ while for the Wang metric, we use $\beta\in[-1.5,1.5]$. We
find that the risk sensitive behavior generalizes well beyond the parameter
bounds used during training. That is, using $\beta$ outside the training range
results in stable but more risk-averse or affine behavior.

Policy Updates We update the policy by maximizing the usual PPO clip-objective

$\begin{split}&\mathcal{L}=\min\left(\frac{\pi_{\phi}(s|a;r)}{\pi_{\phi_{\text{old}}}(s|a;r)}A^{\pi_{\phi_{\text{old}}}}(s,a;r),g\left(\epsilon,A^{\pi_{\phi_{\text{old}}}}(s,a;r)\right)\right)\\\
&\text{where}\;\;\;g(\epsilon,A)=\begin{cases}(1+\epsilon)A,&\text{if $A\geq
0$};\\\ (1-\epsilon)A,&\text{if $A<0$}.\end{cases}\end{split}$ (6)

When computing the advantages $A^{\pi}(s_{t},a_{t};\beta)$, we replace the
value estimates with our risk sensitive value estimates. To estimate the
advantage, we use a truncated version of Generalized Advantage Estimation
(GAE) [38] motivated by [31]. It approximates the advantage for each sample
$(s_{t},a_{t},r_{t},s_{t+1})$ by rolling out the policy $\pi(a|s;\beta)$ for
$T$ timesteps without looking beyond the $T$-th timestep. We compute truncated
GAE estimates as

$\begin{split}&A^{\pi}(s_{t},a_{t};\beta)=\sum_{l=0}^{T-t-1}(\lambda\gamma)^{l}\delta_{t+l}^{r}\\\
&\text{where}\;\;\;\delta_{t}^{r}=r_{t}+\gamma
V_{\beta}(s_{t+1})-V_{\beta}(s_{t})\end{split}$ (7)

and $\lambda$ is the GAE bias / variance trade-off hyperparameter.

##  IV Experiments

We evaluate our method through a series of experiments. In Sections IV-A to
IV-C, we conduct simulation experiments to study risk sensitivity and compare
our method with other approaches. In Section IV-D, we conduct hardware
experiments on the ANYmal robot [20].

Figure 4: Robot operated remotely along an obstacle course. Depending on the
chosen path, different risk sensitivities are preferable. For the gentle
incline along path (a), a risk-seeking policy ($\blacksquare$) can be chosen
for increased walking speed. Using a risk-averse policy ($\blacksquare$) when
descending the stairs along route (b) ensures the robot’s safety. A risk-
averse policy ($\blacksquare$) won’t climb the dangerous obstacle along route
(c) and thus would have to walk around it, along route (d). Meanwhile, setting
the policy to risk-seeking ($\blacksquare$) allows the robot to surmount the
obstacle (c). To step down into the deep pit along route (e), one must set the
sensitivity to risk-seeking ($\blacksquare$). The risk-averse policy
($\blacksquare$) will refuse to step into the pit (f) as it may lead to a
crash.

Training Setup To train policies, we used the legged_gym library [36], based
on the IsaacGym simulator. We used the same observation and action space,
except for the additional risk sensitivity. The risk sensitivity is uniformly
sampled with the command during training. The training environment comprises
seven different terrains with ten difficulty levels per terrain. Robots spawn
at the center of each terrain with a random offset.

Simulation Experiment Setup The quantitative simulation experiments (Sections
IV-B and IV-C) were conducted in a deterministic environment to enable a fair
comparison. In both experiments, we spawned 72 robots at the center of each
environment tile with initial z-rotations uniformly distributed across
$360^{\circ}$ and gave $1\,\text{ms}^{-1}$ forward commands.

###  IV-A Simulation Experiment: Obstacle Course

To demonstrate different risk sensitivities in a teleoperation scenario, we
navigated a robot through the obstacle course depicted in Figure 4. The
shortest path from start to goal in this environment is to descend steep
stairs (b) [$25$cm steps], cross the barrier (c) [$55$cm height], walk up the
ramp, and descend into the pit (e) [$85$cm depth]. To follow this route
successfully, the operator could make use of the different risk sensitivities.

Discussion Based on risk sensitivity, the robot exhibited different locomotion
behaviors. When using a risk-seeking policy, the robot generally walked faster
and attempted to overcome obstacles despite the risk of falling. When
commanding the robot over the barrier along the path (c), the risk-seeking
policy always attempted to follow the command. In most cases, it managed to
surmount the barrier but, in a few cases, got stuck on the barrier or fell
down the opposite side. When sending the risk-seeking policy along path (b),
it sometimes managed to descend the stairs but fell in most cases. We obtained
similar results when descending into the pit along the path (e), where the
robot complied each time but often crashed. The risk-averse policy, on the
other hand, demonstrated a slow but safe walking gate. It thus safely
descended the stairs along path (b) on each attempt. However, the risk-averse
policy refused to climb the barrier along the path (c) and descend into the
pit (f). This experiment showed that different risk sensitivities change the
behavior of the robot. This allows either an operator or a navigation system
to make use of the risk-sensitivity to increase safety during the mission.

###  IV-B Simulation Experiment: Algorithm Performance

Figure 5: Average return in the evaluation environment. Shaded regions
indicate $95\%$ confidence intervals across seeds and evaluation spawns.
Hyperparameters for DPPO were not tuned.

We compare the attained return of our method with PPO [39], commonly used in
state-of-the-art legged locomotion approaches [1, 30], and DSAC [26], a recent
distributional actor-critic algorithm that can learn risk sensitive policies.
DSAC has previously been tested on legged locomotion [37] and, with some
adaptations, been employed in complex control scenarios [51]. For each method,
we trained $10$ policies. In this experiment, our method was trained with
“risk-neutral” expectation instead of a risk metric. Due to computation
constraints, we employed Multi-Layer Perceptrons (MLPs) for both actor and
critic across all three methods. Figure 5 shows the learning curve for each
method.

Discussion We found that both DPPO (return of $27.62$ after $20$k iterations)
and PPO ($27.86$) significantly outperformed DSAC ($21.1$). Qualitatively,
DSAC, which also uses a Dist. RL formulation, failed to learn a proper
locomotion policy. Despite reusing the hyperparameters from [36], extensively
tuned for PPO, our method performs on-par with PPO. We thus establish our
method as competitive in terms of attained return. We highlight that the core
contribution of our work is not outperforming PPO in terms of attained return,
but to allow for risk sensitive behavior.

###  IV-C Simulation Experiment: Risk Sensitive Performance

Figure 6: Average undiscounted return, fraction of episodes in which the robot terminates before episode end, and $L2$ distance between commanded and actual linear velocity in the evaluation environment. Shaded regions indicate $95\%$ confidence intervals across evaluation spawns. Only a single seed was trained for each method due to computation constraints. Name | Loss Function | Value Target
---|---|---
DPPO-Wang | Energy |  $N$-step distribution [$\text{SR}(\lambda)$, $\lambda=1$]
DPPO-Wang-QH | Quantile Huber |  $N$-step distribution
DPPO-Wang-SRL | Energy |  $\text{SR}(\lambda)$, $\lambda=0.95$
DPPO-Wang-1Step | Energy |  1-step distribution [$\text{SR}(\lambda)$, $\lambda=0$]
TABLE I: Overview of ablated components on DPPO-Wang.

We further compare the attained return between different ablations of our
method (listed in Table I) as well as non-distributional reward-shaping
baselines, introduced below. Return is computed using the original reward
formulation. We show the undiscounted return in Figure 6. We also include the
fraction of early terminations and the linear velocity tracking error. These
two metrics underline our qualitative assessment on changing behavior with
risk sensitivity from Section IV-A with quantitative results.

PPO Baseline To contrast our distributional approach, we implement PPO
baselines that modulate “risk”-preference through the reward formulation. In
these baselines, risk-preference is introduced before accounting for
environment dynamics, setting them apart from our Dist. RL approach. Our first
approach, which we call PPO1, adapts individual terms of the reward
formulation. Specifically, we scale the linear velocity tracking reward by
$\beta\in[0,2]$. This scaling aims at trading-off command tracking and other
rewards, most notably the alive reward. The policy should thus learn,
depending on the risk parameter, to mediate differently between command
compliance and keeping the robot safe.

However, reward tuning is a complex process. It is often unclear how a reward
term affects the learned policy. Further, individual terms cannot be tuned in
isolation as they influence one another: e.g. high torque penalties may
prevent the algorithm from discovering the linear velocity tracking reward.
The approach of the PPO1 baseline further increases this design complexity
(especially when scaling multiple reward terms). Thus, in a second approach we
call PPO2, we rescale the full reward formulation in a principled manner. We
treat individual reward terms as Dirac impulses defining a probability
distribution (mirroring Equation 1) and compute the distorted mean by applying
the Wang metric. As risk parameter, we choose $\beta\in[-0.25,0.25]$ as we are
unable to learn a policy for a larger range.

The policy $\pi(a|s;\beta)$ learned by either baseline is conditioned on the
scalar parameter $\beta$, which emulates the risk parameter from our method.
In both methods, the critic learns to predict the expected return used for GAE
computation as in standard PPO. Aside from the critic, we use the same
training setup, rewards, and hyperparameters as for DPPO. We expect the PPO1
baseline to learn risk-averse behavior for $\beta\in[0,1)$ and risk-seeking
behavior for $\beta\in(1,2]$. It reverts to the initial, “risk-neutral”,
reward formulation for $\beta=1$. We expect the PPO2 baseline to exhibit
behavior appropriate for the Wang metric, as outlined in Section III.

Discussion For a policy to qualify as risk sensitive, we expect the Early
Terminations and the Linear Velocity Error to change appropriately across
different risk settings. All methods showed such risk sensitive behavior that
matches our intuition: risk-seeking policies had improved command tracking but
also more premature terminations; risk-averse policies sacrificed command
tracking to keep the robot safe.

The return generally increased with risk-averseness. This trend could be due
to high-reward actions which, despite often failing, would still be pursued by
a risk-seeking policy. In our environment, however, such risk-seeking behavior
was not rewarded. Our method significantly outperformed the reward-shaping
baselines regarding return. Especially the PPO2 baseline had severely lower
returns. These mirrored its high early termination rate and linear velocity
tracking error. For the PPO1 baseline, we experienced outlier behavior for the
most risk-averse setting: since the velocity tracking reward was set to zero
during training, the robot learned to stop walking. The undesired behavior of
PPO1 and low return of PPO2 demonstrate the difficulties when incorporating
risk sensitivity through the standard RL framework.

Comparing our method to its ablations, none attained similar return. DPPO-CVaR
most significantly underperformed in our experiments. This underperformance
may be due to the CVaR metric disregarding an entire portion of the value
distribution, appearing to result in less stable training. Thus, our design
decisions are experimentally justified.

###  IV-D Hardware Experiment

Figure 7: ANYmal showed different step-up behaviors depending on risk
sensitivity and step height.

Our learned policy seamlessly transferred to the real world with appropriate
domain randomization [36]. We conducted experiments on the ANYmal robot [20]
with onboard perception [29] to replicate the risk sensitive behavior observed
in simulation. We commanded the robot to walk up steps of height $27$cm and
$41$cm with a speed of $1.2\,\text{ms}^{-1}$ and counted refusals, failures,
and successes to surmount the obstacle. A high step has some risk of the robot
falling and, thus, a risk-averse policy might avoid it while a risk-seeking
policy should neglect the risk. The experiments are depicted in Figure 7. We
collected a total of $45$ tries, $8$ for each policy on the $41$cm step height
and $7$ on the $27$cm step height. The results are listed in Table II.

|  $27cm$ Step |  $41cm$ Step
---|---|---
Risk Sensitivity |  Refusal |  Failure |  Success |  Refusal |  Failure |  Success
Averse ($\beta=-1.5$) |  $25\%$ |  $50\%$ |  $25\%$ |  $85.7\%$ |  $14.3\%$ |  $0\%$
Neutral ($\beta=0.0$) |  $12.5\%$ |  $75\%$ |  $12.5\%$ |  $0\%$ |  $100\%$ |  $0\%$
Seeking ($\beta=+1.5$) |  $0\%$ |  $37.5\%$ |  $62.5\%$ |  $0\%$ |  $100\%$ |  $0\%$
TABLE II: Hardware experiment results. The robot received a
$1.2\,\text{ms}^{-1}$ command to walk up the step. We counted the number of
refusals, failures (active attempts to surmount the obstacle without ultimate
success), and successes. The given percentages indicate how often each
behavior was observed during the experiments.

Discussion The robot’s actions aligned well with the risk sensitivity of its
assigned policies. When commanded to be risk-averse, the robot hesitated
before climbing the step while it tried to follow with a risk-seeking command.
On the higher step height ($41$cm), the risk-averse policy mostly refused to
walk the step. We often observed the policy standing before the obstacle,
unresponsive to its forward command. The risk-seeking and neutral policies,
while failing to surmount the obstacle aswell, exhibited more aggressive
behavior. The difference between the risk-neutral and risk-seeking policies
became emphasized for the smaller step size ($27$cm). In this experiment, the
risk-seeking policy complied every time, often surmounting the obstacle. The
risk-neutral policy mostly attempted to overcome the obstacle but failed. A
slight inconsistency arose between the risk-neutral and risk-averse policies,
where the latter succeeded in walking the step more often. Overall, the risk-
neutral policy attempted to walk the step more often than the risk-averse
policy. These observations mirror our findings in simulation and confirm that
risk sensitivity successfully transfers to hardware.

##  V Conclusion

Our work presents a distributional reinforcement learning approach for
learning risk sensitive locomotion policies. These policies allow the robot to
adapt its behavior to environment risks. A risk preference is encoded through
a single parameter that can be modulated during deployment. To incorporate
risk sensitivity into the RL formulation, we introduce risk sensitive
advantage estimates. The emergent risk sensitive behavior, demonstrated in
simulation and on hardware, opens up a new application domain for Dist. RL in
real-world applications. Our work directly benefits teleoperation and
navigation, given that a user / planner can now change the risk affinity of
the robot during deployment, increasing safety while retaining full locomotion
capability. Further research is needed to establish procedures on evaluating
risk sensitivity and study risk sensitive Dist. RL from a theoretical
standpoint: how the choice of risk metric impacts the resulting policy and how
well the estimated value distribution matches the true return distribution are
some suggestions. Additionally, integrating DPPO into a navigation system that
utilizes risk sensitivity is promising for future work.

## References

  * [1] Ananye Agarwal, Ashish Kumar, Jitendra Malik and Deepak Pathak  “Legged Locomotion in Challenging Terrains using Egocentric Vision”  In _Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand_ 205, Proceedings of Machine Learning Research  PMLR, 2022, pp. 403–415  URL: https://proceedings.mlr.press/v205/agarwal23a.html
  * [2] Gabriel Barth-Maron et al.  “Distributed Distributional Deterministic Policy Gradients”  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_ OpenReview.net, 2018  URL: https://openreview.net/forum?id=SyZipzbCb
  * [3] Marc Bellemare et al.  “Autonomous navigation of stratospheric balloons using reinforcement learning”  In _Nature_ 588, 2020, pp. 77–82  DOI: 10.1038/s41586-020-2939-8
  * [4] Marc G. Bellemare, Will Dabney and Rémi Munos  “A Distributional Perspective on Reinforcement Learning”  In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_ 70, Proceedings of Machine Learning Research  PMLR, 2017, pp. 449–458  URL: http://proceedings.mlr.press/v70/bellemare17a.html
  * [5] Marc G. Bellemare et al.  “The Cramer Distance as a Solution to Biased Wasserstein Gradients.”  In _CoRR_ abs/1705.10743, 2017  URL: http://dblp.uni-trier.de/db/journals/corr/corr1705.html#BellemareDDMLHM17
  * [6] Julian Bernhard, Stefan Pollok and Alois Knoll  “Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning”  In _2019 IEEE Intelligent Vehicles Symposium (IV)_ IEEE, 2019  DOI: 10.1109/ivs.2019.8813791
  * [7] Gerardo Bledt et al.  “MIT Cheetah 3: Design and Control of a Robust, Dynamic Quadruped Robot”  In _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2018, pp. 2245–2252  DOI: 10.1109/IROS.2018.8593885
  * [8] Cristian Bodnar et al.  “Quantile QT-Opt for Risk-Aware Vision-Based Robotic Grasping”  In _Robotics: Science and Systems XVI_ Robotics: ScienceSystems Foundation, 2020  DOI: 10.15607/rss.2020.xvi.075
  * [9] Amanda Bouman et al.  “Autonomous Spot: Long-Range Autonomous Exploration of Extreme Environments with Legged Locomotion”  In _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2020, pp. 2518–2525
  * [10] Will Dabney, Mark Rowland, Marc G. Bellemare and Rémi Munos  “Distributional Reinforcement Learning With Quantile Regression”  In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_ AAAI Press, 2018, pp. 2892–2901  URL: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17184
  * [11] Will Dabney, Georg Ostrovski, David Silver and Rémi Munos  “Implicit Quantile Networks for Distributional Reinforcement Learning”  In _Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018_ 80, Proceedings of Machine Learning Research  PMLR, 2018, pp. 1104–1113  URL: http://proceedings.mlr.press/v80/dabney18a.html
  * [12] Alexander Dettmann et al.  “Towards a generic navigation and locomotion control system for legged space exploration”  16th Symposium on Advanced Space Technologies in RoboticsAutomation (ASTRA 2022), 2022
  * [13] Alhussein Fawzi et al.  “Discovering faster matrix multiplication algorithms with reinforcement learning”  In _Nature_ 610, 2022, pp. 47–53  DOI: 10.1038/s41586-022-05172-4
  * [14] Jonas Frey et al.  “Fast Traversability Estimation for Wild Visual Navigation”  In _Proceedings of Robotics: Science and Systems_ , 2023
  * [15] Jonas Frey, David Hoeller, Shehryar Khattak and Marco Hutter  “Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments”  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2022  DOI: 10.48550/ARXIV.2203.15854
  * [16] Scott Fujimoto, Herke Hoof and David Meger  “Addressing Function Approximation Error in Actor-Critic Methods.”  In _ICML_ 80, Proceedings of Machine Learning Research  PMLR, 2018, pp. 1582–1591  URL: http://dblp.uni-trier.de/db/conf/icml/icml2018.html#FujimotoHM18
  * [17] Javier García, Fern and Fernández  “A Comprehensive Survey on Safe Reinforcement Learning”  In _Journal of Machine Learning Research_ 16.42, 2015, pp. 1437–1480  URL: http://jmlr.org/papers/v16/garcia15a.html
  * [18] Tuomas Haarnoja et al.  “Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning”, 2023  arXiv:2304.13653 [cs.RO]
  * [19] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel and Sergey Levine  “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor”  In _Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018_ 80, Proceedings of Machine Learning Research  PMLR, 2018, pp. 1856–1865  URL: http://proceedings.mlr.press/v80/haarnoja18b.html
  * [20] Marco Hutter et al.  “ANYmal - a highly mobile and dynamic quadrupedal robot”  In _2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2016, pp. 38–44  DOI: 10.1109/IROS.2016.7758092
  * [21] Jemin Hwangbo et al.  “Learning agile and dynamic motor skills for legged robots”  In _Science Robotics_ 4.26 American Association for the Advancement of Science, 2019, pp. eaau5872
  * [22] Hendrik Kolvenbach, Manuel Breitenstein, Christian Gehring and Marco Hutter  “Scalability Analysis of Legged Robots for Space Exploration”  68th International Astronautical Congress (IAC), 2017  DOI: 10.3929/ethz-b-000183684
  * [23] Arsenii Kuznetsov, Pavel Shvechikov, Alexander Grishin and Dmitry P. Vetrov  “Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics”  In _Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event_ 119, Proceedings of Machine Learning Research  PMLR, 2020, pp. 5556–5566  URL: http://proceedings.mlr.press/v119/kuznetsov20a.html
  * [24] Joonho Lee et al.  “Learning quadrupedal locomotion over challenging terrain”  In _Science Robotics_ 5.47 American Association for the Advancement of Science (AAAS), 2020  DOI: 10.1126/scirobotics.abc5986
  * [25] Timothy P. Lillicrap et al.  “Continuous control with deep reinforcement learning”  In _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , 2016  URL: http://arxiv.org/abs/1509.02971
  * [26] Xiaoteng Ma et al.  “Distributional Soft Actor Critic for Risk Sensitive Learning”  In _CoRR_ abs/2004.14547, 2020  arXiv: https://arxiv.org/abs/2004.14547
  * [27] Anirudha Majumdar and Marco Pavone  “How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics”  In _Robotics Research, The 18th International Symposium, ISRR 2017, Puerto Varas, Chile, December 11-14, 2017_ 10, Springer Proceedings in Advanced Robotics  Springer, 2017, pp. 75–84  DOI: 10.1007/978-3-030-28619-4“˙10
  * [28] Borislav Mavrin et al.  “Distributional Reinforcement Learning for Efficient Exploration.”  In _CoRR_ abs/1905.06125, 2019  URL: http://dblp.uni-trier.de/db/journals/corr/corr1905.html#abs-1905-06125
  * [29] Takahiro Miki et al.  “Elevation Mapping for Locomotion and Navigation using GPU”  In _IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022, Kyoto, Japan, October 23-27, 2022_ IEEE, 2022, pp. 2273–2280  DOI: 10.1109/IROS47612.2022.9981507
  * [30] Takahiro Miki et al.  “Learning robust perceptive locomotion for quadrupedal robots in the wild”  In _Science Robotics_ 7.62, 2022, pp. eabk2822  DOI: 10.1126/scirobotics.abk2822
  * [31] Volodymyr Mnih et al.  “Asynchronous Methods for Deep Reinforcement Learning”  In _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016_ 48, JMLR Workshop and Conference Proceedings  JMLR.org, 2016, pp. 1928–1937  URL: http://proceedings.mlr.press/v48/mniha16.html
  * [32] Daniel Wontae Nam, Younghoon Kim and Chan Y. Park  “GMAC: A Distributional Perspective on Actor-Critic Framework”  In _Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event_ 139, Proceedings of Machine Learning Research  PMLR, 2021, pp. 7927–7936  URL: http://proceedings.mlr.press/v139/nam21a.html
  * [33] R. Rockafellar and Stanislav Uryasev  “Optimization of conditional value-at risk”  In _Journal of Risk_ 3, 2000, pp. 21–41
  * [34] Mark Rowland et al.  “An Analysis of Quantile Temporal-Difference Learning”  arXiv, 2023  DOI: 10.48550/ARXIV.2301.04462
  * [35] Mark Rowland et al.  “Statistics and Samples in Distributional Reinforcement Learning”  In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_ 97, Proceedings of Machine Learning Research  PMLR, 2019, pp. 5528–5536  URL: http://proceedings.mlr.press/v97/rowland19a.html
  * [36] Nikita Rudin, David Hoeller, Philipp Reist and Marco Hutter  “Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning”  In _5th Annual Conference on Robot Learning_ , 2021  URL: https://openreview.net/forum?id=wK2fDDJ5VcF
  * [37] Frederik Schubert, Theresa Eimer, Bodo Rosenhahn and Marius Lindauer  “Automatic Risk Adaptation in Distributional Reinforcement Learning”, 2021  arXiv:2106.06317 [cs.LG]
  * [38] John Schulman et al.  “High-Dimensional Continuous Control Using Generalized Advantage Estimation”  In _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_ , 2016  URL: http://arxiv.org/abs/1506.02438
  * [39] John Schulman et al.  “Proximal Policy Optimization Algorithms.”  In _CoRR_ abs/1707.06347, 2017  URL: http://dblp.uni-trier.de/db/journals/corr/corr1707.html#SchulmanWDRK17
  * [40] John Schulman et al.  “Trust region policy optimization”  In _International conference on machine learning_ , 2015, pp. 1889–1897  PMLR
  * [41] Rahul Singh, Keuntaek Lee and Yongxin Chen  “Sample-based Distributional Policy Gradient”  In _Learning for Dynamics and Control Conference, L4DC 2022, 23-24 June 2022, Stanford University, Stanford, CA, USA_ 168, Proceedings of Machine Learning Research  PMLR, 2022, pp. 676–688  URL: https://proceedings.mlr.press/v168/singh22a.html
  * [42] Laura Smith et al.  “Legged Robots that Keep on Learning: Fine-Tuning Locomotion Policies in the Real World”  In _2022 International Conference on Robotics and Automation (ICRA)_ , 2021, pp. 1593–1599
  * [43] Silvestr Stanko and Karel Macek  “Risk-averse Distributional Reinforcement Learning: A CVaR Optimization Approach”  In _International Joint Conference on Computational Intelligence_ , 2019
  * [44] Ke Sun et al.  “Interpreting Distributional Reinforcement Learning: A Regularization Perspective”  arXiv, 2021  DOI: 10.48550/ARXIV.2110.03155
  * [45] Richard S. Sutton and Andrew G. Barto  “Reinforcement Learning: An Introduction”  The MIT Press, 2018  URL: http://incompleteideas.net/book/the-book-2nd.html
  * [46] Jie Tan et al.  “Sim-to-Real: Learning Agile Locomotion For Quadruped Robots”  In _Robotics: Science and Systems XIV, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, June 26-30, 2018_ , 2018  DOI: 10.15607/RSS.2018.XIV.010
  * [47] Yichuan Charlie Tang, Jian Zhang and Ruslan Salakhutdinov  “Worst Cases Policy Gradients”  In _3rd Annual Conference on Robot Learning, CoRL 2019, Osaka, Japan, October 30 - November 1, 2019, Proceedings_ 100, Proceedings of Machine Learning Research  PMLR, 2019, pp. 1078–1093  URL: http://proceedings.mlr.press/v100/tang20a.html
  * [48] Thibaut Théate and Damien Ernst  “Risk-Sensitive Policy with Distributional Reinforcement Learning”, 2022  arXiv:2212.14743 [cs.LG]
  * [49] Marco Tranzatto et al.  “Team CERBERUS Wins the DARPA Subterranean Challenge: Technical Overview and Lessons Learned”, 2022  arXiv:2207.04914 [cs.RO]
  * [50] Shaun S. Wang  “A Class of Distortion Operators for Pricing Financial and Insurance Risks”  In _The Journal of Risk and Insurance_ 67.1 [American RiskInsurance Association, Wiley], 2000, pp. 15–36  URL: http://www.jstor.org/stable/253675
  * [51] Peter Wurman et al.  “Outracing champion Gran Turismo drivers with deep reinforcement learning”  In _Nature_ 602, 2022, pp. 223–228  DOI: 10.1038/s41586-021-04357-7
  * [52] Zhaoming Xie et al.  “Dynamics Randomization Revisited: A Case Study for Quadrupedal Locomotion”  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , 2020, pp. 4955–4961
  * [53] Derek Yang et al.  “Fully Parameterized Quantile Function for Distributional Reinforcement Learning”  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , 2019, pp. 6190–6199  URL: https://proceedings.neurips.cc/paper/2019/hash/f471223d1a1614b58a7dc45c9d01df19-Abstract.html