# Smoke and mirrors: Neutron star internal heating constraints on mirror
matter

David McKeen<EMAIL_ADDRESS>TRIUMF, 4004 Wesbrook Mall, Vancouver, BC V6T
2A3, Canada  Maxim Pospelov<EMAIL_ADDRESS>School of Physics and Astronomy,
University of Minnesota, Minneapolis, MN 55455, USA  William I. Fine
Theoretical Physics Institute, School of Physics and Astronomy, University of
Minnesota, Minneapolis, MN 55455, USA  Nirmal Raj<EMAIL_ADDRESS>TRIUMF, 4004
Wesbrook Mall, Vancouver, BC V6T 2A3, Canada

###### Abstract

Mirror sectors have been proposed to address the problems of dark matter,
baryogenesis, and the neutron lifetime anomaly. In this work we study a new,
powerful probe of mirror neutrons: neutron star temperatures. When neutrons in
the neutron star core convert to mirror neutrons during collisions, the
vacancies left behind in the nucleon Fermi seas are refilled by more energetic
nucleons, releasing immense amounts of heat in the process. We derive a new
constraint on the allowed strength of neutron–mirror-neutron mixing from
observations of the coldest (sub-40,000 Kelvin) neutron star, PSR
2144$-$3933\. Our limits compete with laboratory searches for neutron–mirror-
neutron transitions but apply to a range of mass splittings between the
neutron and mirror neutron that is 19 orders of magnitude larger. This heating
mechanism, also pertinent to other neutron disappearance channels such as
exotic neutron decay, provides a compelling physics target for upcoming
ultraviolet, optical and infrared telescopes to study thermal emissions of
cold neutron stars.

Introduction.— Mirror world scenarios are one of the earliest ideas of “dark
sectors,” i.e. a set of particles very weakly coupled to the Standard Model
(SM) and neutral under SM gauge forces. They were originally motivated to
restore parity symmetry by introducing states duplicate to the Standard Model
(SM) charged under a new gauge group Kobzarev  _et al._ (1966);
*mirrorintro:Foot:1991bp; *mirrorintro:Foot:1991py, and more recently
entertained in “Twin Higgs” solutions to the gauge hierarchy problem Chacko
_et al._ (2006). Mirror states may provide a plethora of new effects: they
could constitute cosmological dark matter Berezhiani  _et al._ (1996); Hodges
(1993); *dm:Berezhiani:2000gw; *dm:Berezhiani:2005vv;
*twinhiggs:Garcia:2015loa; *twinhiggs:Garcia:2015toa, address the cosmological
asymmetry of baryons Bento and Berezhiani (2001); *baryog:Berezhiani:2008zza,
relax the GZK limit of cosmic rays Berezhiani and Bento (2006a, b), and
collect in hidden stellar structures Blinnikov (1996);
*mirrorstars:Mohapatra:1999ih; *mirrorstars:Foot:1999hm;
*mirrorstars:Foot:2001ne; *mirrorstars:Curtin:2019lhm;
*mirrorstars:Curtin:2019ngc; *mirrorstars:Hippert:2021fch. For reviews, see,
e.g. Ref. Okun (2007); *review:Foot:2014mia; *review:Berezhiani:2018zvs.

A $\mathbb{Z}_{2}$ exchange symmetry connects the mirror sector to the SM. If
this symmetry is respected by the vacuum expectation values of the scalars in
the two sectors, the mirror states are expected to be exactly mass-degenerate
with SM states. This can be problematic for precision cosmology if the light
degrees of freedom in the mirror sector carry a comparable amount of energy to
ordinary photons and neutrinos. This can be avoided if the $\mathbb{Z}_{2}$ is
broken and the two sectors are asymmetrically reheated, $T_{\rm SM}>T_{\rm
mirror}$ Berezhiani  _et al._ (1996). In such a case, nothing forbids a tiny
mixing between the two sectors, leading to the possibility of transition
between neutral SM states and those in the mirror sector. In this paper, we
will address one of the most consequential of such mixings: that between the
neutron, $n$ and the mirror neutron, $n^{\prime}$, which can lead to $n\to
n^{\prime}$ transitions.

The $n$-$n^{\prime}$ system can be simply studied using a two-state
Hamiltonian (with $c=1$),

$H=\begin{pmatrix}m_{n}+\Delta E&\epsilon_{nn^{\prime}}\\\
\epsilon_{nn^{\prime}}&m_{n^{\prime}}\end{pmatrix},$ (1)

where $m_{n}\simeq m_{n^{\prime}}$ are the $n$ and $n^{\prime}$ masses,
$\epsilon_{nn^{\prime}}$ is the mixing amplitude, and $\Delta E$ is the
(medium-dependent) energy splitting between the two states. $n\to n^{\prime}$
transitions occur with a rate determined by $\epsilon_{nn^{\prime}}$ and
$\Delta E$ and have been searched for using ultracold neutrons (UCN) Ban  _et
al._ (2007); *PSIlike:Serebrov:2009zz; *PSI:Altarev:2009tg;
*Berezhiani:globalexpt:2017jkn; Abel  _et al._ (2021) and cold neutron beams
Berezhiani and Bento (2006a). Assuming $n$-$n^{\prime}$ mass degeneracy and a
vanishing mirror magnetic field, $B^{\prime}=0$, the splitting $\Delta E$ can
be neglected in such experiments and the transition probability after a time
$t$ can be written $P_{n\to
n^{\prime}}=(\epsilon_{nn^{\prime}}t)^{2}\equiv(t/\tau_{nn^{\prime}})^{2}$
(setting $\hbar=1$). The strongest constraint on this transition is
$\tau_{nn^{\prime}}>352~{}\rm s$ or $\epsilon_{nn^{\prime}}<1.87\times
10^{-18}~{}\rm eV$ at 95% C. L. Abel  _et al._ (2021). Non-vanishing values of
$m_{n}-m_{n^{\prime}}$ or $\Delta E$ quench $n\to n^{\prime}$ transitions and
the limits from UCN experiments practically disappear for
$\left|m_{n}-m_{n^{\prime}}\right|\gtrsim 10^{-12}~{}\rm eV$.

While $n\to n^{\prime}$ conversions are kinematically forbidden in stable
nuclei due to nuclear binding energy [unless $m_{n}-m_{n^{\prime}}\gtrsim{\cal
O}(\rm MeV)$] they can proceed in a neutron star (NS). In Ref. Goldman  _et
al._ (2019) a limit of $\epsilon_{nn^{\prime}}\lesssim 10^{-13}$ eV was
obtained from the change in the spin period of a NS induced by the production
of mirror neutrons in the NS core.

In this Letter we present a new and powerful probe of mirror neutrons, namely
NS temperatures. When $n\to n^{\prime}$ oscillations occur in inter-nucleon
encounters in NS cores, tremendous heat is released: both the converted
neutron and nucleon on which it scattered leave behind holes in the Fermi sea,
which are refilled by higher-energy nucleons. This is analogous to the Auger
effect in which energy is released as electrons cascade from higher energy
atomic levels to unfilled lower energy ones. We find that luminosity
measurements of the coldest pulsar observed constrain

$\epsilon_{nn^{\prime}}\lesssim 10^{-17}~{}{\rm eV},~{}{\rm
or}~{}\tau_{nn^{\prime}}\gtrsim 10{\rm~{}s},$ (2)

and that future measurements with ultraviolet, optical and infrared telescopes
could further improve sensitivity, perhaps down to
$\epsilon_{nn^{\prime}}\simeq 10^{-20}$ eV ($\tau_{nn^{\prime}}\simeq 10^{5}$
s). In the NS system $\Delta E\simeq 10-100~{}\rm MeV$ due to nuclear
interactions, thus our limits apply to
$\left|m_{n}-m_{n^{\prime}}\right|\lesssim{\cal O}(10~{}\rm MeV)$, a much
wider range of $m_{n^{\prime}}$ than that of terrestrial UCN searches. We
recently used this novel NS heating mechanism to set extensive limits on
exotic neutron decay modes ($n\to$ BSM states) McKeen  _et al._ (2020). While
in that study we had used a simplified picture of the NS to obtain order of
magnitude estimates, in the present work we treat in more careful detail (i)
the dynamics and constituents of the NS core and their implications for
neutron disappearance, and (ii) astronomical measurements. We also note that
the mechanism of generating energy inside NSs via $n\to n^{\prime}$
conversions was first pointed out in a pioneering work Goldman  _et al._
(2019) with application to pulsar binary observables. Here we update this
calculation and apply it to what turns out to be a more sensitive observable:
the NS luminosity.

n-n′ conversion in NS.—We start from the effective $n$-$n^{\prime}$
Hamiltonian in Eq. (1). In the NS medium, nuclear interactions induce values
of $\Delta E$ that range from $\sim 10$ to $100~{}\rm MeV$ Glendenning (2000);
McKeen  _et al._ (2018), and we can therefore ignore the ${\cal O}(\rm keV)$
splitting arising from the NS’s magnetic field. We also set
$m_{n}=m_{n^{\prime}}$, noting that this does not materially change our
results so long as $\left|m_{n}-m_{n^{\prime}}\right|\lesssim{\cal O}(10~{}\rm
MeV)$.

NS cores contain large numbers of degenerate neutrons with a roughly 10%
admixture of protons in beta equilibrium. Therefore, mirror neutrons can be
created in the scattering of neutrons on neutrons and protons, $nn\to
n^{\prime}n$ and $np\to n^{\prime}p$. Due to the extreme degeneracy of
nucleons in the NS core, Pauli blocking must be taken into account when
computing the rates for these processes, as done in e.g. Ref. Reddy  _et al._
(1998); *BertoniNelsonReddy. In the part of parameter space we will consider,
the rate of mirror neutron production is small compared to the inverse
lifetime of neutron stars. This means that the $n^{\prime}$ number density is
not appreciable compared to that of nucleons and we do not have to consider
their Pauli blocking. The rate of $n^{\prime}$ production is therefore written

$\Gamma_{n^{\prime}}=\sum_{N=n,p}f_{N}n_{N}\langle\sigma_{n^{\prime}N}v\rangle_{p_{N}>p_{F_{N}}}~{},$
(3)

where $n_{N}$ is the nucleon number density and $f_{n}=1/2$, $f_{p}=1$ takes
into account the scattering of identical states or not. The cross section to
produce a mirror neutron through scattering on a nucleon, $\sigma_{nN\to
n^{\prime}N}$, is averaged over the initial particle momenta, taking their
relative velocity $v$ into account, as well as the Pauli blocking of the final
state nucleon as indicated by the subscript $p_{N}>p_{F_{N}}$. To do this, we
take the initial nucleon momentum distributions to be those of non-interacting
fermions at zero temperature,

$f_{N}(p)=\Theta\left(p_{F_{N}}-p\right),$ (4)

where $p_{F_{N}}$ is the Fermi momentum of the nucleon $N$. The nucleon number
density is related to $p_{F_{N}}$ through $n_{N}=p_{F_{N}}^{3}/(3\pi^{2})$.

We relate the $n^{\prime}$ production cross sections to the elastic neutron
scattering cross sections:

$\sigma_{n^{\prime}N}\simeq g_{N}\left(\frac{\epsilon_{nn^{\prime}}}{\Delta
E}\right)^{2}\sigma_{nN\to nN}~{},$ (5)

with $g_{n}=2$, $g_{p}=1$ a multiplicity factor (note that
$f_{n}g_{n}=f_{p}g_{p}=1$) and $\epsilon_{nn^{\prime}}/\Delta E$ an effective
in-medium $n$-$n^{\prime}$ mixing angle.

We obtain neutron-nucleon cross sections using experimentally determined phase
shifts NNO . We conservatively use only the $s$-wave phase shifts, noting that
higher order partial waves are subdominant. The cross sections can be written
in terms of the spin singlet and triplet phase shifts, $\delta_{S}$ and
$\delta_{T}$ respectively, as

$\displaystyle\sigma_{nn\to nn}$
$\displaystyle\simeq\frac{1}{4}\times\frac{16\pi}{m_{N}^{2}v^{2}}\sin^{2}\delta_{S},$
(6) $\displaystyle\sigma_{np\to np}$
$\displaystyle\simeq\frac{1}{4}\times\frac{16\pi}{m_{N}^{2}v^{2}}\left(\sin^{2}\delta_{S}+3\sin^{2}\delta_{T}\right)~{},$
(7)

where the factors of $1/4$ come from spin-averaging. In the energy region of
interest $\sigma_{np}\gtrsim 4\sigma_{nn}$.

Finally, we perform the final state phase space integration taking into
account the nucleon momentum distribution in Eq. (4) and only include final
state nucleon momenta above the Fermi momentum as indicated in the subscript
of Eq. (3). With all of these taken into account, for total nucleon density
$n_{\rm nuc}$ (with $n_{p}\simeq 0.1n_{n}$) we find $n^{\prime}$ production
rates of

$\Gamma_{n^{\prime}}=\frac{1}{1.2\times 10^{11}~{}\rm
yr}\left(\frac{\epsilon_{nn^{\prime}}}{10^{-17}~{}{\rm
eV}}\right)^{2}\left(\frac{n_{\rm nuc}}{0.3~{}{\rm fm}^{-3}}\right)~{}.$ (8)

The observable in which we are interested is the NS temperature, related to
the energy liberated in the $n\to n^{\prime}$ conversions described above. In
any given conversion, there are three sources of energy transmitted to the
rest of the star: (i) the energy from the filling of the hole left by the
converted neutron, $\widetilde{\mu}_{n}-E_{n}^{i}$, (ii) that from filling the
hole left by the spectator nucleon, $\widetilde{\mu}_{N}-E_{N}^{i}$, and (iii)
the excess kinetic energy of the final state nucleon kicked up above the Fermi
sea, $E_{N}^{f}-\widetilde{\mu}_{N}$. Here $\widetilde{\mu}_{N}$ refers to the
kinetic energy of a nucleon at the top of the Fermi sea, which is not
$p_{F_{N}}^{2}/2m_{N}$ when the potential energy of strong interactions is
accounted for, and the $i$ and $f$ superscripts refer to initial or final
state particles. Putting all of this together, the energy released in any
given conversion is therefore
$\widetilde{\mu}_{n}-p_{n^{\prime}}^{2}/2m_{n^{\prime}}$ and the total energy
injection rate is:

$\dot{E}_{n^{\prime}}=\sum_{N=n,p}f_{N}n_{N}\left\langle\left(\widetilde{\mu}_{n}-\frac{p_{n^{\prime}}^{2}}{2m_{n^{\prime}}}\right)\sigma_{n^{\prime}N}v\right\rangle_{p_{N}>p_{F_{N}}}~{}.$
(9)

We implicitly take scattering at these energies to be elastic, a conservative
assumption as it releases less energy than inelastic scattering.

Neutron star heating.— The total energy released per unit time by $n\to
n^{\prime}$ conversions in the NS is

$L_{n\to n^{\prime}}=\int d^{3}r\,n_{n}({\bf r})\dot{E}_{n^{\prime}}({\bf
r})~{}.$ (10)

To compute this quantity, we assume spherical symmetry and use high-density
nuclear equations of state (EoS) from the literature to solve the
Tolman–Oppenheimer–Volkoff (TOV) equations that govern the hydrodynamic
stability of the star Tolman (1939); *TOV:Oppenheimer:1939ne. From this, we
can use the resulting $n_{N}=p_{F_{N}}^{3}/3\pi^{2}$, $\widetilde{\mu}_{N}$,
and $\Delta E$ (which we approximate as the average energy per nucleon as a
proxy for the neutron self-energy Glendenning (2000); McKeen  _et al._ (2018))
to compute $\dot{E}_{n^{\prime}}$ as a function of radial co-ordinate $r$. We
can then ask whether the heating rate computed in Eq. (10) for a given NS is
compatible with its temperature, $T_{\rm NS}$. Old, cold NSs of age $\gtrsim
10^{5}$ yr cool mainly via photon emission from their surfaces and have
approximately uniform temperatures Yakovlev and Pethick (2004). Taking the
luminosity to be that of a blackbody, $L_{\gamma}=4\pi R_{\rm
NS}^{2}\sigma_{\rm SB}T_{\rm NS}^{4}$, where $\sigma_{\rm SB}$ is the Stefan-
Boltzmann constant, we can set an upper bound on the transition amplitude
$\epsilon_{nn^{\prime}}$ by requiring that $L_{n\to n^{\prime}}\leq
L_{\gamma}$.

Solving the TOV equations with the ‘BSk24’ EoS from Ref. Pearson  _et al._
(2018) for a central density that produces a NS with mass $M_{\rm
NS}=1.5M_{\odot}$ and radius $R_{\rm NS}=12.6~{}\rm km$, characteristic of a
typical NS, and computing the luminosity in Eq. (10), we arrive at the
following upper bound on the transition amplitude applicable for an old, cold
NS with surface temperature $T_{\rm NS}$,

$\epsilon_{nn^{\prime}}<1.6\times 10^{-17}~{}{\rm eV}\bigg{(}\frac{T_{\rm
NS}}{3\times 10^{4}~{}{\rm K}}\bigg{)}^{2}~{}.$ (11)

We see that measurements or limits on temperatures of such NSs at the
$10^{4}-10^{5}~{}\rm K$ level provide limits that are competitive with
searches for transitions of UCN to mirror neutrons. Our NS configuration
corresponds to volume-averaged values of $\langle\Delta E\rangle$ = 29 MeV
(with $\Delta E\rangle$ = 79 MeV at the NS center and 2 MeV near the outer
edge of the core), $\langle\widetilde{\mu}_{n}\rangle=68$ MeV,
$\langle\widetilde{\mu}_{p}\rangle=13$ MeV, $\langle n_{n}\rangle=0.22$ fm-3,
$\langle n_{p}\rangle=0.02$ fm-3. Remarkably, the proton-catalyzed
$n^{\prime}$ production contribution to $L_{n\to n^{\prime}}$ dominates over
that from neutron-catalyzed production by a factor of $\sim 4$ for two
reasons: (i) $n_{p}<n_{n}$ implies less Pauli blocking and (ii)
$\sigma_{np}>\sigma_{nn}$ as mentioned above.

The tightest upper bound on $\epsilon_{nn^{\prime}}$ comes from the coldest NS
that has been observed, PSR J2144$-$3933 Guillot  _et al._ (2019). As in Ref.
Guillot  _et al._ (2019), we can determine an upper bound on this NS’s
temperature by modelling its spectral flux density as

$f_{\nu}=\bigg{(}\frac{R_{\infty}}{d_{\star}}\bigg{)}^{2}\frac{2\pi
hf^{3}_{\nu}}{c^{2}}\frac{10^{-1.24c_{\rm
ex}}}{\exp(h\nu/k_{B}T_{\infty})-1}~{},$ (12)

where $T_{\infty}=T_{\rm NS}/(1+z)$, $R_{\infty}=R_{\rm NS}(1+z)$, with
$(1+z)~{}=~{}(1-2GM_{\rm NS}/R_{\rm NS}c^{2})^{-1/2}$. (We have restored $c$
and $h=2\pi\hbar$ here.) We then require that the average spectral flux
density in the frequency range $1.51\times 10^{15}\,{\rm Hz}<\nu<2.21\times
10^{15}\,{\rm Hz}$ not exceed $5.9~{}\rm nJy$, for $157\,{\rm
pc}<d_{\star}<192\,{\rm pc}$ and $0<c_{\rm ex}<0.06$. This translates into a
range of upper bounds $T_{\rm NS}<29600~{}\rm K$ to $T_{\rm NS}<34100~{}\rm
K$. The left panel of Fig. 1 shows the resulting upper limit on
$\epsilon_{nn^{\prime}}$ from this range using Eq. (11).

Figure 1: Left. Limits from neutron star temperatures on neutron-mirror mixing
$\epsilon_{nn^{\prime}}=1/\tau_{nn^{\prime}}$, assuming the ‘BSk24’ Pearson
_et al._ (2018) high-density equation of state giving an NS mass and radius of
1.5 $M_{\odot}$ and 12.6 km. The strongest upper bound comes from the coldest
NS observed, PSR J2144$-$3933, and we indicate the range in upper bounds on
$\epsilon_{nn^{\prime}}$ coming from the spread in upper bounds on its
temperature. The largest values of $\epsilon_{nn^{\prime}}$ are probed by the
youngest NS confirmed, Cas A. Other pulsars consistent with a temperature of
$10^{5.1}$ K also provide bounds in the intermediate region. Future
ultraviolet, optical and infrared telescopes could measure NS temperatures
down to 1000 K: the green region depicts the resulting sensitivity to
$\epsilon_{nn^{\prime}}$ for a given $T_{\rm NS}$. Also shown is the bound
obtained by Goldman et al. in Ref. Goldman  _et al._ (2019) from NS rotation
periods. Right. Limits from terrestrial experiments looking for $nn^{\prime}$
oscillations as a function of an unknown local mirror magnetic field strength
$B^{\prime}$, as adopted from Ref. Abel  _et al._ (2021); also shown is the
limit for $B^{\prime}=0$. The arXiv numbers of the various experimental
publications are indicated. These limits apply only in the extremely mass-
degenerate limit and weaken considerably for $m_{n}^{\prime}-m_{n}>10^{-18}$
MeV, whereas our NS heating limits are valid so long as
$m_{n}^{\prime}-m_{n}<\Delta E\simeq 10$ MeV. For comparison, the region
shaded red is excluded by our NS heating limits. See text for more details.

The limit on $\epsilon_{nn^{\prime}}$ can change slightly if we consider
different NS masses. In particular, larger NS masses lead to higher densities
and Fermi momenta, reducing the $n\to n^{\prime}$ rate; using the same EoS as
above, ‘BSk24’, with $M_{\rm NS}=2.27M_{\odot}$ and $R_{\rm NS}=11~{}\rm km$
gives a 2.5 times weaker bound on $\epsilon_{nn^{\prime}}$. However, keeping
the NS mass fixed to our nominal value, $M_{\rm NS}=1.5M_{\odot}$, and varying
the EoS changes the bound by less than 10% as we confirmed by comparing with
results using ‘BSk22’, ‘BSk25’, and ‘BSk26’ Pearson  _et al._ (2018).

There is a ceiling to the limit we have derived above—a value above which our
analysis no longer simply applies—occurring roughly at the value of
$\epsilon_{nn^{\prime}}$ that results in a volume average of the $n^{\prime}$
production rate in Eq. (8) that is equal to the inverse age of the NS. To
extend our analysis to larger $\epsilon_{nn^{\prime}}$ would require
considering a non-negligible $n^{\prime}$ number density in the NS. The non-
trivial phase space distribution of the mirror neutrons complicates the
problem and we defer an analysis to future work McKeen  _et al._ . The
lifetime of PSR J2144$-$3933 is estimated to be $3\times 10^{8}~{}\rm yr$
Guillot  _et al._ (2019), cutting off our exclusion at
$\epsilon_{nn^{\prime}}=6.2\times 10^{-12}~{}\rm eV$.

In addition, the left panel of Fig. 1 shows similarly derived limits from the
thermal luminosities of the NSs PSR B0950$+$08 Pavlov  _et al._ (2017), PSR
J0108$-$1431 Mignani  _et al._ (2008), PSR J0437$-$4715 Durant  _et al._
(2012), and PSR J2124$-$3358 Rangelov  _et al._ (2017), all of which are
consistent with $T_{\rm NS}=10^{5.1}~{}\rm K$. The youngest of these is PSR
B0950$+$08 with a spin-down age of $10^{7.2}~{}\rm yr$ Pavlov  _et al._
(2017), setting the ceiling in this temperature range.

To probe larger values of $\epsilon_{nn^{\prime}}$ requires younger NSs. In
contrast to older NSs, those younger than about $10^{5}$ yr cool through
emission of neutrinos from the high density core, which scales differently
with the core temperature than blackbody emission from the surface considered
above. The youngest observed NS is Cas A (CXOU J232327.8$+$584842) with an age
of $340~{}\rm yr$. Its surface temperature is $2\times 10^{6}~{}\rm K$,
corresponding to an internal temperature $T_{\rm int}=3\times 10^{8}~{}\rm K$
assuming an Fe envelope Heinke and Ho (2010). Assuming a minimal cooling
scenario through the modified Urca process Page  _et al._ (2004); Yakovlev and
Pethick (2004), its neutrino luminosity is $L_{\nu}\simeq 10^{32}\,{\rm
erg/s}\,(T_{\rm int}/10^{8}~{}{\rm K})^{8}$. Requiring $L_{n\to
n^{\prime}}<L_{\nu}$ results in the upper limit
$\epsilon_{nn^{\prime}}<3\times 10^{-13}~{}\rm eV$, also shown in the left
panel of Fig. 1. Given Cas A’s age, this applies for
$\epsilon_{nn^{\prime}}\lesssim 2\times 10^{-9}~{}\rm eV$. If we assume that
the dust blob in the location of SN 1987A is powered by the thermal luminosity
of a 34 yr-old NS Page  _et al._ (2020), we could access
$\epsilon_{nn^{\prime}}\lesssim 6\times 10^{-9}~{}\rm eV$. The upper limit on
$\epsilon_{nn^{\prime}}$ from this potential NS, whose internal temperature is
constrained to be $10^{9}~{}\rm K$ Page  _et al._ (2020), is
$\epsilon_{nn^{\prime}}\lesssim 6\times 10^{-11}$ eV assuming modified Urca
neutrino cooling.

The green region in the left panel of Fig. 1 shows the future reach obtainable
by measuring the thermal luminosity of PSR J2144$-$3933 (which should have
reached a temperature of 1000 K at $t\simeq 2\times 10^{7}$ yr and must now be
much colder than the present upper bound on its temperature Page  _et al._
(2004); Yakovlev and Pethick (2004)) or other, yet undiscovered, cold NSs, by
current and imminent ultraviolet, optical and infrared survey telescopes, e.g.
LUVOIR Team (2019), Rubin/LSST Peterson (2016), Dark Energy Survey Diehl and
et al. (2019), and Roman/WFIRST Rom (2012). It has also been proposed that NSs
down to 1000 K temperatures could be observed in upcoming infrared telescopes
over realistic integration times Baryakhtar  _et al._ (2017) (leading to
studies of NSs heated to infrared temperatures via capture of ambient dark
matter Baryakhtar  _et al._ (2017); Raj  _et al._ (2018); *NSvIR:SelfIntDM;
*NSvIR:Bell2018:Inelastic; *NSvIR:GaraniGenoliniHambye;
*NSvIR:Queiroz:Spectroscopy; *NSvIR:Bell2019:Leptophilic;
*NSvIR:Hamaguchi:Rotochemical; *NSvIR:GaraniHeeck:Muophilic; *NSvIR:Pasta;
*NSvIR:Riverside:LeptophilicShort; *NSvIR:Marfatia:DarkBaryon;
*NSvIR:Riverside:LeptophilicLong; *NSvIR:Bell:Improved;
*NSvIR:DasguptaGuptaRay:LightMed; *NSvIR:GaraniGuptaRaj:Thermalizn;
*NSvIR:Bell:ImprovedLepton; *NSvIR:Queiroz:BosonDM and exotic neutron decay
McKeen  _et al._ (2020)). The ceiling here corresponds to
$\Gamma^{-1}_{n^{\prime}}=10^{7}$ yr, describing the age of an NS beyond which
we expect it to have $\lesssim\mathcal{O}(10^{3})$ K temperatures Page  _et
al._ (2004); Yakovlev and Pethick (2004).

In addition, we show the upper bound on the transition amplitude that comes
from the analysis of Ref. Goldman  _et al._ (2019) from the change in the spin
period of NSs along with our limits in the left panel of Fig. 1.

For comparison, in the right panel of Fig. 1 we show the limit on
$\epsilon_{nn^{\prime}}=\tau_{nn^{\prime}}^{-1}$ as a function of the local
mirror magnetic field $B^{\prime}$, from terrestrial $n$-$n^{\prime}$
oscillation searches Ban  _et al._ (2007); *PSIlike:Serebrov:2009zz;
*PSI:Altarev:2009tg; *Berezhiani:globalexpt:2017jkn; Abel  _et al._ (2021).
The dot-dashed grey line depicts the strongest limit for $B^{\prime}=0$,
$\tau_{nn^{\prime}}>352$ s Abel  _et al._ (2021). While these limits are
comparable to those that we have derived from NS heating, they only apply to
the situation of extreme degeneracy,
$|m_{n^{\prime}}-m_{n}|<\mu_{n}B^{\prime}\simeq 10^{-18}~{}$MeV, with the
oscillation probability quenched for larger splittings. On the other hand our
NS limits apply for mass splittings up to 19 orders of magnitude greater,
$|m_{n^{\prime}}-m_{n}|\lesssim\mathcal{O}(10)$ MeV, as determined by the
$n$-$n^{\prime}$ splitting in the NS medium.

Discussion.— We have shown in this study that, via liberating heat from the
energy stored in the Fermi sea of neutron stars, neutron oscillations to
mirror neutrons could be probed in NS thermal emissions in wide regions of
parameter space inaccessible to terrestrial searches. A strong hint of $n\to
n^{\prime}$ conversions would be a universal minimum temperature detected over
a statistical ensemble of NSs using current and imminent telescopes. Whether
such a signal can be definitely attributed to $n\to n^{\prime}$ oscillations
would depend much on the success of efforts to pinpoint NS masses and radii,
and the high-density equation of state of nuclear matter such as undertaken by
the NICER experiment Riley  _et al._ (2019); *NICER2:Raaijmakers:2019qny;
*NICER3:Miller:2019cac. If the EoS can be precisely constrained, uncertainties
on our prediction for the NS luminosity in Eq. (10), containing several EoS-
dependent quantities, could be minimized. (We note that if “dark baryons” such
as mirror neutrons are present in NSs in appreciable quantities, the EoS may
be non-trivially impacted PaoloNS:2008db; *PaoloNS:2010ji; McKeen  _et al._
(2018); *SheltonNS; *MottaNS; *EllisPattavinaNS.)

Another scenario probed by NS heating is neutron decay to any exotic final
state, as we had studied for dark baryons in Ref. McKeen  _et al._ (2020) and
will explore in more detail in forthcoming work McKeen  _et al._ . Here NS
heating is the sole probe of dark baryons for tiny mixings with neutrons. The
need for astronomical observations of low-temperature neutron stars is usually
motivated by the need to constrain cooling models and the EoS Potekhin  _et
al._ (2020); Yakovlev and Pethick (2004); Page  _et al._ (2004), and to check
for reheating mechanisms, e.g. rotochemical heating Yanagi  _et al._ (2020) or
possible accretion of interstellar material. Our studies of $nn^{\prime}$
oscillation and $n$ decay, probing the properties of neutrons directly in NSs,
provide further compelling motivations for such a campaign. We also note that
precise measurements of cooling curves of young NSs would also be useful, as
these would help set a sharper upper bound on the neutron mixing than
presented here. Precision measurements and fits to cooling models would also
make our bounds stronger than the ones obtained via our loose criterion,
$L_{n\to n^{\prime}}\leq L_{\rm NS}$.

We remark that NS heating will not be as competitive as terrestrial searches
in probing the $|\Delta B|=2$ process of neutron oscillations to anti-neutrons
($\bar{n}$). This process would release at least $2m_{n}\simeq$ 2 GeV per
$n\to\bar{n}$ conversion in the NS core via the $\bar{n}$ annihilating with
surrounding neutrons. Keeping all other quantities fixed, the NS luminosity in
Eq. (10) is now 10–100 times higher, implying that the limits on the the off-
diagonal mass $\epsilon_{n\bar{n}}$ are 3–10 times stronger than in Fig. 1.
The current limit on this parameter from dinucleon decay in Super-K oxygen
nuclei Abe  _et al._ (2015) is $\epsilon_{n\bar{n}}<2.4\times 10^{-24}$ eV
Phillips  _et al._ (2016); thus to probe new parameter space NS temperatures
$\ll 100$ K must be constrained, an unrealistic possibility with upcoming
telescopes.

Finally, as our bounds only impact $\epsilon_{nn^{\prime}}\lesssim 2\times
10^{-9}~{}\rm eV$, set by the age of Cas A, our analysis does not exclude
$n$-$n^{\prime}$ oscillation explanations to the neutron lifetime anomaly.
This requires $\epsilon_{nn^{\prime}}\sim\mathcal{O}(10^{-7})$ eV to suppress
proton production in the beam measurements of the neutron lifetime Berezhiani
(2019). To probe this region requires extending our analysis to consider a
nontrivial $n^{\prime}$ density which we reserve for future work McKeen  _et
al._ .

## Acknowledgments

We thank Joe Bramante, Rebecca Leane, and Shirley Li for helpful conversations
on future telescopes. The work of D. M. and N. R. is supported by the Natural
Sciences and Engineering Research Council of Canada. Triumf receives federal
funding via a contribution agreement with the National Research Council
Canada. M.P. is supported in part by U.S. Department of Energy Grant No.
desc0011842.

## References

  * Kobzarev  _et al._ (1966) I. Y. Kobzarev, L. B. Okun, and I. Y. Pomeranchuk, Sov. J. Nucl. Phys. 3, 837 (1966).
  * Foot  _et al._ (1991) R. Foot, H. Lew, and R. R. Volkas, Phys. Lett. B 272, 67 (1991).
  * Foot  _et al._ (1992) R. Foot, H. Lew, and R. R. Volkas, Mod. Phys. Lett. A 7, 2567 (1992).
  * Chacko  _et al._ (2006) Z. Chacko, H.-S. Goh, and R. Harnik, Phys. Rev. Lett. 96, 231802 (2006), arXiv:hep-ph/0506256 .
  * Berezhiani  _et al._ (1996) Z. G. Berezhiani, A. D. Dolgov, and R. N. Mohapatra, Phys. Lett. B 375, 26 (1996), arXiv:hep-ph/9511221 .
  * Hodges (1993) H. M. Hodges, Phys. Rev. D 47, 456 (1993).
  * Berezhiani  _et al._ (2001) Z. Berezhiani, D. Comelli, and F. L. Villante, Phys. Lett. B 503, 362 (2001), arXiv:hep-ph/0008105 .
  * Berezhiani  _et al._ (2006) Z. Berezhiani, S. Cassisi, P. Ciarcelluti, and A. Pietrinferni, Astropart. Phys. 24, 495 (2006), arXiv:astro-ph/0507153 .
  * Garcia Garcia  _et al._ (2015a) I. Garcia Garcia, R. Lasenby, and J. March-Russell, Phys. Rev. D 92, 055034 (2015a), arXiv:1505.07109 [hep-ph] .
  * Garcia Garcia  _et al._ (2015b) I. Garcia Garcia, R. Lasenby, and J. March-Russell, Phys. Rev. Lett. 115, 121801 (2015b), arXiv:1505.07410 [hep-ph] .
  * Bento and Berezhiani (2001) L. Bento and Z. Berezhiani, Phys. Rev. Lett. 87, 231304 (2001).
  * Berezhiani (2008) Z. Berezhiani, Eur. Phys. J. ST 163, 271 (2008).
  * Berezhiani and Bento (2006a) Z. Berezhiani and L. Bento, Phys. Rev. Lett. 96, 081801 (2006a), arXiv:hep-ph/0507031 .
  * Berezhiani and Bento (2006b) Z. Berezhiani and L. Bento, Phys. Lett. B 635, 253 (2006b), arXiv:hep-ph/0602227 .
  * Blinnikov (1996) S. I. Blinnikov, in  _Conference on Baryonic Matter in the Universe and Its Spectroscopic Studies_ (1996) arXiv:astro-ph/9801015 .
  * Mohapatra and Teplitz (1999) R. N. Mohapatra and V. L. Teplitz, Phys. Lett. B 462, 302 (1999), arXiv:astro-ph/9902085 .
  * Foot (1999) R. Foot, Phys. Lett. B 452, 83 (1999), arXiv:astro-ph/9902065 .
  * Foot and Silagadze (2001) R. Foot and Z. K. Silagadze, Acta Phys. Polon. B 32, 2271 (2001), arXiv:astro-ph/0104251 .
  * Curtin and Setford (2020a) D. Curtin and J. Setford, Phys. Lett. B 804, 135391 (2020a), arXiv:1909.04071 [hep-ph] .
  * Curtin and Setford (2020b) D. Curtin and J. Setford, JHEP 03, 041 (2020b), arXiv:1909.04072 [hep-ph] .
  * Hippert  _et al._ (2021) M. Hippert, J. Setford, H. Tan, D. Curtin, J. Noronha-Hostler, and N. Yunes, (2021), arXiv:2103.01965 [astro-ph.HE] .
  * Okun (2007) L. B. Okun, Phys. Usp. 50, 380 (2007), arXiv:hep-ph/0606202 .
  * Foot (2014) R. Foot, Int. J. Mod. Phys. A 29, 1430013 (2014), arXiv:1401.3965 [astro-ph.CO] .
  * Berezhiani (2018) Z. Berezhiani, Int. J. Mod. Phys. A 33, 1844034 (2018).
  * Ban  _et al._ (2007) G. Ban _et al._ , Phys. Rev. Lett. 99, 161603 (2007), arXiv:0705.2336 [nucl-ex] .
  * Serebrov  _et al._ (2009) A. P. Serebrov _et al._ , Nucl. Instrum. Meth. A 611, 137 (2009), arXiv:0809.4902 [nucl-ex] .
  * Altarev  _et al._ (2009) I. Altarev _et al._ , Phys. Rev. D 80, 032003 (2009), arXiv:0905.4208 [nucl-ex] .
  * Berezhiani  _et al._ (2018) Z. Berezhiani, R. Biondi, P. Geltenbort, I. A. Krasnoshchekova, V. E. Varlamov, A. V. Vassiljev, and O. M. Zherebtsov, Eur. Phys. J. C 78, 717 (2018), arXiv:1712.05761 [hep-ex] .
  * Abel  _et al._ (2021) C. Abel _et al._ (nEDM), Phys. Lett. B 812, 135993 (2021), arXiv:2009.11046 [hep-ph] .
  * Goldman  _et al._ (2019) I. Goldman, R. N. Mohapatra, and S. Nussinov, Phys. Rev. D 100, 123021 (2019), arXiv:1901.07077 [hep-ph] .
  * McKeen  _et al._ (2020) D. McKeen, M. Pospelov, and N. Raj, (2020), arXiv:2012.09865 [hep-ph] .
  * Reddy  _et al._ (1998) S. Reddy, M. Prakash, and J. M. Lattimer, Phys. Rev. D 58, 013009 (1998), arXiv:astro-ph/9710115 .
  * Bertoni  _et al._ (2013) B. Bertoni, A. E. Nelson, and S. Reddy, Phys. Rev. D 88, 123505 (2013), arXiv:1309.1721 [hep-ph] .
  * (34) https://nn-online.org/.
  * Tolman (1939) R. C. Tolman, Phys. Rev. 55, 364 (1939).
  * Oppenheimer and Volkoff (1939) J. R. Oppenheimer and G. M. Volkoff, Phys. Rev. 55, 374 (1939).
  * Glendenning (2000) N. Glendenning,  _Compact Stars: Nuclear Physics, Particle Physics, and General Relativity_, Astronomy and Astrophysics Library (Springer New York, 2000).
  * McKeen  _et al._ (2018) D. McKeen, A. E. Nelson, S. Reddy, and D. Zhou, Phys. Rev. Lett. 121, 061802 (2018), arXiv:1802.08244 [hep-ph] .
  * Yakovlev and Pethick (2004) D. G. Yakovlev and C. J. Pethick, Ann. Rev. Astron. Astrophys. 42, 169 (2004), arXiv:astro-ph/0402143 [astro-ph] .
  * Pearson  _et al._ (2018) J. M. Pearson, N. Chamel, A. Y. Potekhin, A. F. Fantina, C. Ducoin, A. K. Dutta, and S. Goriely, Mon. Not. Roy. Astron. Soc. 481, 2994 (2018), [Erratum: Mon.Not.Roy.Astron.Soc. 486, 768 (2019)], arXiv:1903.04981 [astro-ph.HE] .
  * Guillot  _et al._ (2019) S. Guillot, G. Pavlov, C. Reyes, A. Reisenegger, L. Rodriguez, B. Rangelov, and O. Kargaltsev, Astrophys. J. 874, 175 (2019), arXiv:1901.07998 [astro-ph.HE] .
  * (42) D. McKeen, M. Pospelov, and N. Raj, In preparation .
  * Pavlov  _et al._ (2017) G. G. Pavlov, B. Rangelov, O. Kargaltsev, A. Reisenegger, S. Guillot, and C. Reyes, Astrophys. J. 850, 79 (2017), arXiv:1710.06448 [astro-ph.HE] .
  * Mignani  _et al._ (2008) R. P. Mignani, G. G. Pavlov, and O. Kargaltsev, Astron. Astrophys. 488, 1027 (2008), arXiv:0805.2586 [astro-ph] .
  * Durant  _et al._ (2012) M. Durant, O. Kargaltsev, G. G. Pavlov, P. M. Kowalski, B. Posselt, M. H. van Kerkwijk, and D. L. Kaplan, Astrophys. J. 746, 6 (2012), arXiv:1111.2346 [astro-ph.HE] .
  * Rangelov  _et al._ (2017) B. Rangelov, G. G. Pavlov, O. Kargaltsev, A. Reisenegger, S. Guillot, C. Reyes, and M. H. van Kerkwijk, Astrophys. J. 835, 264 (2017), arXiv:1701.00002 [astro-ph.HE] .
  * Heinke and Ho (2010) C. O. Heinke and W. C. G. Ho, Astrophys. J. Lett. 719, L167 (2010), arXiv:1007.4719 [astro-ph.HE] .
  * Page  _et al._ (2004) D. Page, J. M. Lattimer, M. Prakash, and A. W. Steiner, Astrophys. J. Suppl. 155, 623 (2004), arXiv:astro-ph/0403657 [astro-ph] .
  * Page  _et al._ (2020) D. Page, M. V. Beznogov, I. Garibay, J. M. Lattimer, M. Prakash, and H.-T. Janka, Astrophys. J. 898, 125 (2020), arXiv:2004.06078 [astro-ph.HE] .
  * Team (2019) T. L. Team, “The luvoir mission concept study final report,” (2019), arXiv:1912.06219 [astro-ph.IM] .
  * Peterson (2016) J. R. Peterson,  (2016), 10.2172/1272167.
  * Diehl and et al. (2019) H. T. Diehl and et al.,  (2019), 10.2172/1596042.
  * Rom (2012) “Wide-field infrared survey telescope (wfirst) final report,” (2012), arXiv:1208.4012 [astro-ph.IM] .
  * Baryakhtar  _et al._ (2017) M. Baryakhtar, J. Bramante, S. W. Li, T. Linden, and N. Raj, Phys. Rev. Lett. 119, 131801 (2017), arXiv:1704.01577 [hep-ph] .
  * Raj  _et al._ (2018) N. Raj, P. Tanedo, and H.-B. Yu, Phys. Rev. D 97, 043006 (2018), arXiv:1707.09442 [hep-ph] .
  * Chen and Lin (2018) C.-S. Chen and Y.-H. Lin, JHEP 08, 069 (2018), arXiv:1804.03409 [hep-ph] .
  * Bell  _et al._ (2018) N. F. Bell, G. Busoni, and S. Robles, JCAP 1809, 018 (2018), arXiv:1807.02840 [hep-ph] .
  * Garani  _et al._ (2019) R. Garani, Y. Genolini, and T. Hambye, JCAP 1905, 035 (2019), arXiv:1812.08773 [hep-ph] .
  * Camargo  _et al._ (2019) D. A. Camargo, F. S. Queiroz, and R. Sturani, JCAP 1909, 051 (2019), arXiv:1901.05474 [hep-ph] .
  * Bell  _et al._ (2019) N. F. Bell, G. Busoni, and S. Robles, JCAP 1906, 054 (2019), arXiv:1904.09803 [hep-ph] .
  * Hamaguchi  _et al._ (2019) K. Hamaguchi, N. Nagata, and K. Yanagi, Phys. Lett. B795, 484 (2019), arXiv:1905.02991 [hep-ph] .
  * Garani and Heeck (2019) R. Garani and J. Heeck, Phys. Rev. D100, 035039 (2019), arXiv:1906.10145 [hep-ph] .
  * Acevedo  _et al._ (2020) J. F. Acevedo, J. Bramante, R. K. Leane, and N. Raj, JCAP 03, 038 (2020), arXiv:1911.06334 [hep-ph] .
  * Joglekar  _et al._ (2019) A. Joglekar, N. Raj, P. Tanedo, and H.-B. Yu, (2019), arXiv:1911.13293 [hep-ph] .
  * Keung  _et al._ (2020) W.-Y. Keung, D. Marfatia, and P.-Y. Tseng, JHEP 07, 181 (2020), arXiv:2001.09140 [hep-ph] .
  * Joglekar  _et al._ (2020) A. Joglekar, N. Raj, P. Tanedo, and H.-B. Yu, (2020), arXiv:2004.09539 [hep-ph] .
  * Bell  _et al._ (2020) N. F. Bell, G. Busoni, S. Robles, and M. Virgato, (2020), arXiv:2004.14888 [hep-ph] .
  * Dasgupta  _et al._ (2020) B. Dasgupta, A. Gupta, and A. Ray, JCAP 10, 023 (2020), arXiv:2006.10773 [hep-ph] .
  * Garani  _et al._ (2021) R. Garani, A. Gupta, and N. Raj, Phys. Rev. D 103, 043019 (2021), arXiv:2009.10728 [hep-ph] .
  * Bell  _et al._ (2021) N. F. Bell, G. Busoni, S. Robles, and M. Virgato, JCAP 03, 086 (2021), arXiv:2010.13257 [hep-ph] .
  * Maity and Queiroz (2021) T. N. Maity and F. S. Queiroz, “Detecting bosonic dark matter with neutron stars,” (2021), arXiv:2104.02700 [hep-ph] .
  * Riley  _et al._ (2019) T. E. Riley _et al._ , Astrophys. J. Lett. 887, L21 (2019), arXiv:1912.05702 [astro-ph.HE] .
  * Raaijmakers  _et al._ (2019) G. Raaijmakers _et al._ , Astrophys. J. Lett. 887, L22 (2019), arXiv:1912.05703 [astro-ph.HE] .
  * Miller  _et al._ (2019) M. C. Miller _et al._ , Astrophys. J. Lett. 887, L24 (2019), arXiv:1912.05705 [astro-ph.HE] .
  * Potekhin  _et al._ (2020) A. Y. Potekhin, D. A. Zyuzin, D. G. Yakovlev, M. V. Beznogov, and Y. A. Shibanov, Mon. Not. Roy. Astron. Soc. 496, 5052 (2020), arXiv:2006.15004 [astro-ph.HE] .
  * Yanagi  _et al._ (2020) K. Yanagi, N. Nagata, and K. Hamaguchi, Mon. Not. Roy. Astron. Soc. 492, 5508 (2020), arXiv:1904.04667 [astro-ph.HE] .
  * Abe  _et al._ (2015) K. Abe _et al._ (Super-Kamiokande), Phys. Rev. D 91, 072006 (2015), arXiv:1109.4227 [hep-ex] .
  * Phillips  _et al._ (2016) D. G. Phillips, II _et al._ , Phys. Rept. 612, 1 (2016), arXiv:1410.1100 [hep-ex] .
  * Berezhiani (2019) Z. Berezhiani, Eur. Phys. J. C 79, 484 (2019), arXiv:1807.07906 [hep-ph] .


# ToolAlpaca: Generalized Tool Learning for Language Models
with 3000 Simulated Cases

Qiaoyu Tang1,3, Ziliang Deng1,3, Hongyu Lin1, Xianpei Han1,211footnotemark: 1,
Qiao Liang1,3, Boxi Cao1,3, Le Sun1,2 Corresponding Authors

###### Abstract

Enabling large language models to utilize real-world tools effectively is
crucial for achieving embodied intelligence. Existing approaches to tool
learning have either primarily relied on extremely large language models, such
as GPT-4, to attain generalized tool-use abilities in a zero-shot manner, or
utilized supervised learning to train limited scopes of tools on compact
models. However, it remains uncertain whether smaller language models can
achieve generalized tool-use abilities without tool-specific training. To
address this question, this paper introduces ToolAlpaca, a novel framework
designed to automatically generate a diverse tool-use corpus and learn
generalized tool-use abilities on compact language models with minimal human
intervention. Specifically, ToolAlpaca first automatically creates a highly
diversified tool-use corpus by building a multi-agent simulation environment.
The corpus contains 3938 tool-use instances from more than 400 real-world tool
APIs spanning 50 distinct categories. Subsequently, the constructed corpus is
employed to fine-tune compact language models, resulting in two models, namely
ToolAlpaca-7B and ToolAlpaca-13B, respectively. Finally, we evaluate the
ability of these models to utilize previously unseen tools without specific
training. Experimental results demonstrate that ToolAlpaca achieves effective
generalized tool-use capabilities comparable to those of extremely large
language models like GPT-3.5, demonstrating that learning generalized tool-use
ability is feasible for compact language models. 111Our code and data are
available at https://github.com/tangqiaoyu/ToolAlpaca.

##  1 Introduction

Embodied intelligence, the ability to meaningfully interact with the
environment, stands as a core attribute of advanced cognitive systems and a
crucial advancement in artificial intelligence. The ability to create and use
tools has expanded human beings’ physical capabilities to interact with
environments and augmented cognitive functions. Such evolutionary milestone
has not only broadened our range of physical actions, but also brought about
transformative changes in our problem-solving abilities and innovative
thinking. The pursuit of incorporating tool-use capabilities into artificial
intelligence holds great significance in advancing the development of general
intelligent systems.

Recent advancements in enhancing large language models (LLMs) such as GPT-4
(OpenAI 2023) with tool-use abilities have made significant progress in this
area. These models have shown their ability to effectively employ external
tools through integrated plugins, thereby expanding their versatility and
enhancing the precision and quality of their outputs. Unfortunately, due to a
lack of understanding of how existing large language models acquire the
general tool-use capability, currently compact language models still do not
possess such general ability. Consequently, substantial research efforts are
dedicated to fine-tuning smaller language models to acquire the capacity for
tool usage (Komeili, Shuster, and Weston 2022; Parisi, Zhao, and Fiedel 2022;
Schick et al. 2023) on a limited range of tools, which lacks the ability to
generalize to unseen tools. This discrepancy between the generalized tool-use
abilities of larger models and the more constrained capabilities of compact
models presents an intriguing question: Can these compact language models
learn to generalize their tool-use abilities, thus enabling interaction with a
broader spectrum of tools?

In this paper, we explore whether it is feasible for compact language models
to learn generalized tool-use abilities. Intuitively, previous studies have
demonstrated the possibility of equipping compact language models with
generalized instruction-following abilities by fine-tuning them on diversified
instruction datasets (Taori et al. 2023; Zhou et al. 2023). Hence, a promising
strategy for equipping language models with generalized tool-use abilities
would involve fine-tuning them on a corpus containing highly-diversified tool-
use instances. Unfortunately, such a diversified corpus is currently
unavailable. This absence can be attributed to several crucial factors. First,
the absence of a set of available tool APIs that can accommodate various tool
usage scenarios for language models presents a considerable challenge in
assembling a diverse collection of tools. Second, real-world tool-use
instances often entail complex, intricate, and multi-turn interactions between
the language model, users, and tools. This complexity significantly heightens
the difficulty and manual effort involved in creating instances encompassing a
wide array of tools on a large scale. Consequently, these factors
significantly restrict the efforts to construct a diversified tool-use corpus
for language model training efficiently.

Figure 1: A high-level overview of ToolAlpaca, consisting of three components:
(1)Toolset construction, where structured documentation for each tool is
generated based on the brief introductions provided by public-apis. (2) Tool-
use instance generation via multi-agent simulation. (3) ToolAlpaca model
training, which involves fine-tuning language models on generated tool-use
corpus to get ToolAlpaca.

To this end, we propose a framework named ToolAlpaca, which is designed to
automatically create a diverse and well-structured toolset for LLMs and
generate multi-turn complex tool-use instances for generalized tool learning.
The overall structure of ToolAlpaca is shown in Figure 1. Specifically, to
ensure the diversity and comprehensiveness of the toolset, ToolAlpaca
leverages LLM’s text generation capability to construct a comprehensive
toolset. ToolAlpaca gathers a substantial amount of brief introductions of
potentially valuable tools from the internet. It’s important to note that
there is no requirement for these tools’ APIs to be functional or for them to
possess structured documentation directly usable by LLMs. Building on this
foundation, ToolAlpaca employs the generative capacity of LLMs by taking the
brief introduction of relevant tools as input and prompts the model to produce
detailed, structured documentation for each tool. By employing this
methodology, ToolAlpaca has collected more than 400 tool descriptions spanning
50 categories. Each tool is uniformly represented using a standardized
documentation format. Subsequently, in order to acquire tool-use instances
involving the aforementioned tools, we have designed a simulation environment
aimed at emulating the multi-step interactions among language models, users,
and tools. Specifically, we utilize LLMs to simulate the interactions between
the model, users, and the APIs of the tools by leveraging LLMs to serve as
different kinds of agents. In this way, our simulation environment can
generate a substantial volume of tool-use instances without any manual
intervention. Consequently, we have crafted an inclusive tool-use dataset that
comprises 3938 instances, effectively showcasing the practical application of
over 400 distinct tools.

To verify whether our corpus can empower compact language models with the
generalized tool-use ability, we conduct experiments to train ToolAlpaca model
on Vicuna (Chiang et al. 2023), a representative compact language model, and
subsequently evaluate its performance on various unseen tools. Through machine
evaluation with GPT-4, we find that ToolAlpaca can effectively equip numerous
unseen tools, ranging from real-world APIs to multi-modal tools, and it
exhibits competitive performance with GPT-3.5. Furthermore, we investigate the
effect of diversity. It is observed that even with the same number of
instances, the model trained on more varied toolsets will achieve better
performance. This underscores that diversity is a pivotal factor for
ToolAlpaca to generalize tool learning with 3000 simulated cases.

In summary, the main contributions of this paper are:

  * •

To the best of our knowledge, this paper is the first work that verifies the
feasibility of equipping compact language models with generalized tool-use
capacities as extremely large language models.

  * •

This paper presents ToolAlpaca, a simple framework for the automated
generation of tool-use corpus and the enhancement of the compact language
model’s generalized tool-use ability.

  * •

We create a diverse tool-use corpus containing 3.9k tool-use instances from
more than 400 tools across 50 distinct categories. It serves as a solid
foundation for compact language models to acquire generalized tool-use
ability.

Figure 2: An instance of a tool documentation, composed of five essential
parts: name, introduction, description, function documentation, OpenAPI
specification.

##  2 Related Work

#### Tool Use

The utilization of external tools in LLMs has emerged as a rapidly growing
research area (Mialon et al. 2023; Qin et al. 2023a). Current approaches can
be divided into two distinct categories. The first category leverages the
capabilities of LLMs, prompting them to interact with various tools, ranging
from highly specialized ones such as code interpreters (Gao et al. 2022; Chen
et al. 2022), search engines (Yao et al. 2022), retrieval models (Khattab et
al. 2023) and AI models (Shen et al. 2023; Lu et al. 2023), to more versatile
toolsets (Qin et al. 2023a; Li et al. 2023; Song et al. 2023). Large language
models have already demonstrated robust generalization capabilities in tool
usage and enable to equip numerous unseen tools via prompting. In contrast,
the second category concentrates on enhancing the tool-specific usage
capabilities of compact language models through fine-tuning with datasets
specifically designed for the specialized tools (Parisi, Zhao, and Fiedel
2022; Schick et al. 2023; Xu et al. 2023). Concurrent with our work, GPT4Tools
(Yang et al. 2023) fine-tuning compact models to incorporate multi-modal
tools, which concentrates on a set of quite similar multi-modal tools. ToolLLM
(Qin et al. 2023b) facilitates language models to master massive APIs.
However, their data collection strategy requires the prior accumulation of
massive authentic APIs, which requires manual efforts to obtain and verify.
Despite their effectiveness, the domain of generalized tool-use abilities in
compact language models remains largely unexplored upon the accomplishment of
this paper. This study aims to bridge this research gap by automatically
constructing a diverse dataset on tool utilization that encompasses various
tool-use scenarios.

#### LLMs for Data Generation

Many research studies have employed LLMs for data generation, focusing on
various tasks such as question answering (Wang et al. 2021; Agrawal et al.
2022; Chen, Chen, and tau Yih 2023), semantic similarity predictions (Schick
and Schütze 2021), and instruction tuning (Honovich et al. 2022; Wang et al.
2023). Furthermore, in the context of tool use, several works (Schick et al.
2023; Patil et al. 2023; Yang et al. 2023) have already employed model-
synthesized data to enhance specific tool-use capabilities. However, the
generation of generalized tool-use data poses more significant challenges, as
it involves extensive and diverse tools and more intricate multi-turn
interactions.

##  3 Diversified Tool-use Corpus Generation via Multi-agent Simulation

In this section, we introduce ToolAlpaca, a multi-agent simulation framework
designed to generate a diversified tool-use corpus with minimal human
intervention. As shown in Figure 1, our framework consists of two stages:

  1. 1.

Toolset Construction. This step aims to construct a collection of tools and
represent them using a standardized format as {name, introduction,
description, function documentation, OpenAPI specification}. Specifically, we
initiate the process by sourcing tool names and introductions from the
internet and then utilize LLMs to enrich them with structured documentation
that thoroughly delineates the functionality and usage of each tool. In this
way, we can construct a diverse and structured toolset that closely resembles
real-world scenarios.

  2. 2.

Tool-use Instance Generation. Given the toolset, this phase’s objective is to
generate tool-use instances within a simulation environment automatically.
This environment is engineered through the orchestration of three distinct
virtual agents, each embodied by a large language model: the user, the tool
executor, and the assistant. Through the multi-turn interplay among these
agents, we can generate tool-use instances that reflect real-world tool
utilization scenarios. Each tool-use instance consists of three key elements:
{the user’s instructions, the actions and their corresponding tool outputs,
final response}.

###  3.1 Diverse Toolset Construction

This section describes how to construct a diverse toolset and represent them
in a uniform format. The process initiates with the accumulation of an
extensive API collection from the internet, reflecting real-world tool usage
scenarios. Given the rudimentary descriptions and lack of uniform
representation in these APIs, we further leverage the generative capabilities
of LLM to create comprehensive documentation for each tool. This documentation
assists language models in understanding the functionality and usage of each
tool. Subsequently, we adhere to OpenAPI standards to generate a uniform
specification for each API, enabling automated computer invocation and
facilitating subsequent tool execution simulation. In this way, each tool can
be represented as a quintuple {name, introduction, description, function
documentation, OpenAPI specification}. Figure 2 provides an example, where the
name, description, and introduction offer basic information and the purpose of
the public holiday tool, the function documentation provides the
functionality, inputs and outputs of various functions (getHolidays,
searchHolidays, getHolidayDetails) contained within the tool, and the OpenAPI
Specification provides a more comprehensive and structured document. The
detailed construction steps are elaborated as follows.

#### Tool Collection.

Various tools are commonly utilized by human beings, typically manifested in
the form of web-based APIs. To facilitate the utilization and discovery of
these APIs, a plethora of repositories exist on the Internet, aggregating a
vast collection of practical and commonly used APIs. Consequently, this step
leverages the representative API repository, public-apis
222https://github.com/public-apis/public-apis, as our target toolset. This
repository encompasses over 1400 APIs spanning more than 50 distinct
categories. From this, we collect the name and introduction of each tool.

#### Documentation Generation.

To enhance the language model’s comprehension of tools’ functionalities and
usage, this step employs LLMs to automatically generate documentation for each
tool, including its description and function documentation. Specifically,
given the tool’s name and introduction, we leverage the powerful generative
capabilities of LLMs, such as ChatGPT, and construct corresponding prompts to
generate the description and function documentation for each tool. As
illustrated in Figure 2, for the tool description, we expand the introduction
to provide a general overview of the tool’s purpose and functionality,
enabling the language model to understand the appropriate task scenarios for
using the tool. For the function documentation, we prompt LLMs to generate
more specific functions within the scope described in the description,
including their respective input and output parameters. It ensures that the
LLM comprehends how to use the tool’s different functionalities. By employing
this approach, we lay a solid foundation for subsequent user instruction
generation and the creation of the simulated tool-use environment.

#### OpenAPI Specification Generation.

Finally, we prompt LLM to generate a more formalized and structured
representation for each tool in the form of OpenAPI Specification. OpenAPI
Specification (OAS) defines a standard and language-agnostic interface for
describing APIs, including information about endpoints, expected input/output
types, and possible error responses. OAS provides consistency and readability
for both humans and machines, making it an ideal choice for our tool-use
corpus. This comprehensive documentation serves as the foundation for
simulating tool execution in the subsequent stages of our framework.

In this way, we construct a diverse, uniformly represented toolset, which
provides a solid foundation for the multi-agent simulation environment
building and further tool-use corpus generation.

###  3.2 Automatic Tool-use Instances Generation

Given the toolset, this section describes how to automatically construct a
tool-use corpus, so that language models can be trained to acquire generalized
tool-use ability. Specifically, as depicted in Figure 3, each tool-use
instance can be represent as a triple {Instruction, Actions, Response}:

Figure 3: An illustration of the tool-use instance generation process within
the simulation environment. The user agent initiates the sequence by providing
an instruction. The assistant agent then interprets the instruction and
engages in a multi-turn interaction with the user and the tool executor until
a suitable response is generated.

  * •

Instruction: A user query that requires tool assistance for resolution. ”… so
can you tell me the list of holidays in Japan next year?” serves as an
instruction in our example.

  * •

Actions: The process of resolving an instruction may involve executing
multiple actions in a specific order. Following React (Yao et al. 2022), each
action is represented by a tuple that includes the thought, the function name,
the input parameters, and the corresponding tool response. For example, as
shown in Figure 3, the tuple (”I need to get the list of holidays in Japan for
2024.”, ”getHolidays”, {”country”: ”Japan”, ”year”: 2024}, ”Status Code: 200
Response:…”) represents an action.

  * •

Response: This refers to the model’s conclusive response after the multi-turn
interaction, integrating the tool responses to provide a comprehensive
solution to the original instruction. For instance, the response in our
example is: ”The list of holidays in Japan for 2024 is …”.

However, constructing a diverse and authentic tool-use dataset is a
challenging task. Firstly, the wide variety within our toolset makes it
impracticable to manually draft instructions for each tool. Given the vast
array of tools, spanning from recreational to professional domains, and the
fact that the construction of instructions relies on understanding the
functionality and potential use cases of the tools, the burden of manual
annotation becomes overwhelming. Secondly, tool usage in real-world scenarios
often involves a multi-round iterative process of trial and error, making the
automated construction of tool-use instances that reflect real situations
highly challenging.

To this end, we design a simulation environment to generate a tool-use corpus,
encompassing three virtual agents: the user, the assistant, and the tool
executor. Tool-use instances are generated through the interplay among these
agents. Specifically, each agent is simulated by a large language model with a
specific prompt. The distinct roles of each agent are detailed as follows:

  * •

User Agent is designed to mimic the tool user, with its functionalities
encompassing: (1) drafting task instructions for the current tool based on its
function documentation; (2) responding to the assistant’s queries based on the
current interaction context, providing essential information that might be
missing from the initial instruction. For each functionality, we construct
corresponding prompt templates to guide LLMs to generate appropriate outputs.
Moreover, to ensure diversity in task instructions, we have employed various
prompts to generate instructions of different formats, including commands,
questions, and others. Leveraging the large model’s proficiency across
virtually all domains, this approach enables the generation of high-quality
and diversified instructions based on tool documentation. This effectively
addresses the previously mentioned issues with manual annotation.

  * •

Assistant Agent is designed to simulate an assistant with tool utilization
capabilities. It receives instructions from the user agent and determines the
subsequent actions. This involves choosing the appropriate tools and
functions, generating commands for the tool executor, and summarizing the
interaction to generate the final response. As shown in Figure 3, following
ReAct (Yao et al. 2022), we employ a (thought, action, observation) format
template to guide LLM in accomplishing these tasks.

  * •

Tool Executor Agent is constructed to emulate the execution of tools,
receiving requests from the assistant agent and generating responses based on
the tool’s predefined functionalities. Specifically, after conducting format
and parameter checks on the assistant’s requests, these requests are converted
into network request formats. Then the tool executor prompts LLM with the
tool’s OpenAPI specification and the assistant’s requests to generate
simulated execution results. Leveraging LLMs’ robust simulation and generation
capabilities, we mitigate the intricacies involved in constructing actual API
calls. This method has been empirically validated as both accurate and
effective, as evidenced in the following section.

Given the above agents, tool-use cases are generated through multiple rounds
of interaction between them. Initially, the user agent generates instructions
based on the tool information. Subsequently, the assistant agent selects an
appropriate action and its corresponding input and awaits simulation execution
and response from the tool executor. This iterative procedure of action
selection and tool response collection continues until the assistant agent
deems it has gathered sufficient information to respond to the user’s
instructions. Through this multi-agent interaction, we can simulate realistic
tool-use scenarios and generate comprehensive and diversified tool-use
instances.

##  4 ToolAlpaca Corpus

###  4.1 Construction Details

Leveraging the aforementioned multi-agent simulation framework, we have
constructed the ToolAlpaca corpus. Specifically, the process begins with
randomly selecting 500 APIs from the public-apis repository. Subsequently, we
utilize ChatGPT to generate more comprehensive documentation, resulting in a
varied and well-structured toolset. Within our simulation environment, we use
ChatGPT as the user agent to generate ten instructions for each tool, and the
tool executor to simulate tool execution. We appoint GPT-3.5 as the assistant
agent due to its superior performance in structured output
generation.333Preliminary experiments demonstrated an occasional inability of
ChatGPT to strictly adhere to the prescribed output formats.

To guarantee the quality of the data, we implement a simple yet effective
filtering process on the generated corpus. We systematically exclude tools
that might incorporate non-textual content within their inputs or outputs. In
terms of instances, we discard those that exceed five interaction steps, lack
relevant function calls, or exhibit parsing errors in their output.

Finally, we automatically construct an extensive and diversified tool-use
corpus. As shown in Table 1, it encompasses 426 distinctive tools from 50
categories, totaling 3938 instances. In the following sections, we will
analyze the diversity and quality of our corpus.

statistics |
---|---
# of Tool Categories | 50
# of Tools | 426
# of Instance | 3, 938
# of single function call | 2, 512
# of multiple function calls | 1, 426
avg. functions per tool | 4.85
avg. steps | 1.66
avg. instruction length | 23.42
avg. output length | 36.19

Table 1: Statistics of ToolAlpaca corpus.

###  4.2 Diversity

As previously underscored, diversity is pivotal for large models to acquire
generalized capabilities and adapt to a multitude of unseen scenarios (Wang et
al. 2023). ToolAlpaca corpus demonstrates diversity in two aspects:

Figure 4: Length distribution of generated instructions and final outputs.

  * •

Toolset. As outlined in Table 1, our toolset demonstrates diversity in
multiple aspects: (1) The toolset encompasses 50 categories of tools, ranging
from common categories, such as jobs and news, to specialized categories like
blockchain and finance. (2) Each tool provides an average of five functions,
highlighting the diversity and comprehensiveness of its capabilities. (3) The
range of function inputs varies from simple to complex scenarios, including
arrays and objects, further enhancing the richness and complexity of our
toolset.

  * •

Instances. The instances within the ToolAlpaca corpus demonstrate diversity in
terms of instruction, function calls, and error handling. Specifically, we
employ a variety of prompts during instruction generation to stimulate the
language model in producing diverse instructions. The wide-ranging
distribution of instruction length, as illustrated in Figure 4, partly
substantiates this point. Additionally, our dataset contains about 1.5k
instances that require multiple function invocations for resolution, further
underscoring the comprehensiveness of our dataset. Furthermore, our data
adequately reflects the potential errors that may be encountered in authentic
tool usage scenarios, encompassing instances that involve various types of
errors, such as invalid actions, parsing errors, and incorrect parameters.

###  4.3 Quality

To evaluate the quality of ToolAlpaca corpus, we randomly sample 100 instances
and engage a human annotator for assessment. The evaluation tests the
solvability of the instructions generated by the user agent, the precision of
the output from the tool executor agent, and the accuracy of the assistant
agent’s actions and responses. As illustrated in Table 2, we observe that the
metrics for assessing the capabilities of the three agents all exceed 80%.
This substantiates that each agent is proficient in their respective roles,
demonstrating the reliability of data constructed based on simulation and
affirming the decent quality of our dataset.

Quality |  Yes%
---|---
solvability of instructions | 88%
effectiveness of Tool agent’s response | 92%
accuracy of action sequences and final output | 80%

Table 2: Data quality review for ToolAlpaca corpus.

##  5 Experiment

In this section, we investigate whether a set of simulated data can empower
compact language models to acquire generalized tool-use capabilities. To
verify this, we conduct zero-shot experiments on various tools that have not
appeared in the training set, ranging from simulated tools, real-world tools,
to out-of-dataset multi-modal tools. Furthermore, we investigate how the
diversity of the toolset impacts the generalized tool-use ability of language
models.

###  5.1 Experimental Settings

#### Training

We fine-tune Vicuna models (Vicuna-7B and Vicuna-13B) on ToolAlpaca corpus.
The fine-tuning process consists of three epochs, with a batch size of 128 and
a learning rate of 2e-5.

Model | Simulated Tools | Real-world APIs
---|---|---
Procedure | Response | Overall | Human | Procedure | Response | Overall
GPT-3.5 | 77.0 | 85.0 | 75.0 | 79.0 | 75.4 | 80.7 | 72.8
Vicuna-7B | 19.0 | 21.0 | 17.0 | 16.0 | 7.9 | 11.4 | 7.9
ToolAlpaca-7B | 63.0 | 69.0 | 60.0 | 73.0 | 63.2 | 57.9 | 55.3
Vicuna-13B | 17.0 | 31.0 | 16.0 | 25.0 | 13.2 | 16.7 | 12.3
ToolAlpaca-13B | 70.0 | 73.0 | 70.0 | 75.0 | 66.7 | 67.5 | 61.4

Table 3: Evaluation results on unseen simulated tools and real-world APIs. We
can observe that after training on our corpus, ToolAlpaca’s performance
significantly surpasses that of the Vicuna model, reaching comparable
performance with GPT-3.5.

#### Evaluation

To measure the generalized tool-use ability of the language model, we create
an evaluation dataset through our data generation framework and manually
annotate the data. This evaluation dataset consists of two subsets: (1) a
simulated subset that includes 10 simulated tools, which were not part of the
training toolset; (2) a real-world subset comprising 11 real-world APIs from
various domains, designed to assess the divergence between our simulated data
and real-world data.

To evaluate the models, we utilize GPT-4 for machine evaluation across all
experiments, with an additional manual evaluation conducted specifically for
the simulated subset. We prompt GPT-4 with the tool documentation and the
standard answer from the human annotator and expect it to evaluate the
performance in the following aspects:

  * •

Procedure: This metric evaluates the model’s proficiency in accurately
selecting suitable actions, utilizing correct parameters, and avoiding
redundant actions.

  * •

Response: This criterion measures whether the final response can satisfy the
user’s instruction.

  * •

Overall: This metric evaluates the whole process, requiring the correctness of
procedure and response.

###  5.2 Results

#### Effectiveness of ToolAlpaca corpus.

Table 3 presents the main results from the simulated set, evidencing that
fine-tuning on ToolAlpaca corpus can foster generalized tool learning for
compact models. Without fine-tuning on our corpus, Vicuna models demonstrate
constrained tool-use capabilities, with the human accept rate of 16 and 25,
respectively. These statistics emphasize the existing compact models’
insufficiency in achieving the generalized tool-use capacity like larger
models. Nevertheless, our ToolAlpaca models can attain 73 (+57) and 75 (+50)
accept rates, respectively. ToolAlpaca-13B even achieves comparable
performance to GPT-3.5. This evidences the feasibility of instilling
generalized tool-use capabilities into compact language models by only
training on 3000 instances generated by our framework. Furthermore, the
consistency between the human accept rate and the overall accuracy proves that
machine evaluation can serve as a suitable evaluation method for this task.

#### Generalization on real-world tools.

The effectiveness of our corpus is further validated through testing on real-
world APIs, demonstrating that simulation serves as an exceptionally efficient
data collection method. Table 3 exhibits the performance of ToolAlpaca on the
real-world test set, where it achieves an overall accuracy of 55.3 and 61.4,
respectively, significantly surpassing the performance of Vicuna models. This
suggests that training on simulated data can indeed adapt to real-world tool
usage scenarios. We attribute this to the current LLMs’ robust simulation
capabilities, which provide compelling evidence for future simulation-based
data construction.

Model | $SR_{t}$ | $SR_{act}$ | $SR_{args}$ | $SR$
---|---|---|---|---
GPT-3.5 | 99.5 | 99.5 | 91.5 | 91.5
Vicuna-13B | 84.4 | 43.7 | 46.7 | 26.2
GPT4Tools | 98.2 | 97.0 | 92.2 | 90.6
ToolAlpaca-13B* | - | 95.5 | 85.3 | 83.7

Table 4: Evaluation results on unseen tools from GPT4Tools test set. Metrics:
successful rate of thought, action, arguments, and the entire instance. We can
observe that ToolAlpaca, with 3.9k cases, reaches the same performance level
as GPT4Tools, which has been trained on 71k instances generated by the same
process with the test set. *: As our training set does not include data not
involving tool use, we exclude 50 out of 652 test cases that do not involve
tool usage.

Moreover, to evaluate ToolAlpaca’s generalization on out-of-dataset scenarios,
we conduct experiments on GPT4Tools(Yang et al. 2023) test set, which
encompasses 8 multi-modal tools. As shown in Table 4, ToolAlpaca, trained on
merely 3.9k cases, demonstrate 83.7 success rate on out-of-dataset evaluation,
which is close to GPT4Tools, trained on 71k instances constructed with the
same process. This observation indicates that the language model can invoke
out-of-dataset tools after training on ToolAlpaca corpus. We speculate that
the performance may be attributed to the diversity of instances and toolset,
and we delve into it in the subsequent experiment.

#### Impact of diversity.

The diversity of the dataset is crucial for the generalization of tool
learning. To investigate this, we maintain the number of instances and
construct datasets on 10, 40, 100, and 400 tools, respectively. Subsequently,
we fine-tune Vicuna-13B on these datasets with the same experimental settings
and utilize GPT-4 to evaluate the validation set. As shown in Figure 5, as the
diversity of the toolset increases, the performance on the validation set
gradually improves. Specifically, training with a dataset of 10 different
tools resulted in a mere 51 overall accuarcy. In contrast, when the variety of
tools increases to 400 and keeps the number of instances, the performance
escalates to 70. This finding highlights the significant role of toolset
diversity in generalizing tool learning. This provides valuable insight for
the construction of datasets for generalized ability learning.

Figure 5: Performance variation with the increasing toolset diversity.

##  6 Conclusion

In this paper, we introduce ToolAlpaca, an automated framework designed to
improve the generalized tool-use capability of language models. Specifically,
we first create a comprehensive corpus spanning a broad range of tools with
various usage instances. Subsequently, this corpus serves as the basis for
fine-tuning compact language models, leading to the generation of the
ToolAlpaca models. Experimental results indicate that ToolAlpaca performs
comparably to GPT-3.5 in generalized tool-use scenarios. This finding not only
substantiates the potential of our data generation framework but also
highlights the feasibility of mastering generalized tool use in compact-size
models.

## References

  * Agrawal et al. (2022) Agrawal, P.; Alberti, C.; Huot, F.; Maynez, J.; Ma, J.; Ruder, S.; Ganchev, K.; Das, D.; and Lapata, M. 2022.  QAmeleon: Multilingual QA with Only 5 Examples.  arXiv:2211.08264.
  * Chen, Chen, and tau Yih (2023) Chen, M.; Chen, X.; and tau Yih, W. 2023.  Efficient Open Domain Multi-Hop Question Answering with Few-Shot Data Synthesis.  arXiv:2305.13691.
  * Chen et al. (2022) Chen, W.; Ma, X.; Wang, X.; and Cohen, W. W. 2022.  Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks.  arXiv:2211.12588.
  * Chiang et al. (2023) Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; Stoica, I.; and Xing, E. P. 2023.  Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality.
  * Gao et al. (2022) Gao, L.; Madaan, A.; Zhou, S.; Alon, U.; Liu, P.; Yang, Y.; Callan, J.; and Neubig, G. 2022.  PAL: Program-aided Language Models.  _arXiv e-prints_ , arXiv:2211.10435.
  * Honovich et al. (2022) Honovich, O.; Scialom, T.; Levy, O.; and Schick, T. 2022.  Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor.  arXiv:2212.09689.
  * Khattab et al. (2023) Khattab, O.; Santhanam, K.; Li, X. L.; Hall, D.; Liang, P.; Potts, C.; and Zaharia, M. 2023.  Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP.  arXiv:2212.14024.
  * Komeili, Shuster, and Weston (2022) Komeili, M.; Shuster, K.; and Weston, J. 2022.  Internet-Augmented Dialogue Generation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 8460–8478. Dublin, Ireland: Association for Computational Linguistics.
  * Li et al. (2023) Li, M.; Song, F.; Yu, B.; Yu, H.; Li, Z.; Huang, F.; and Li, Y. 2023.  API-Bank: A Benchmark for Tool-Augmented LLMs.  arXiv:2304.08244.
  * Lu et al. (2023) Lu, P.; Peng, B.; Cheng, H.; Galley, M.; Chang, K.-W.; Wu, Y. N.; Zhu, S.-C.; and Gao, J. 2023.  Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models.  arXiv:2304.09842.
  * Mialon et al. (2023) Mialon, G.; Dessì, R.; Lomeli, M.; Nalmpantis, C.; Pasunuru, R.; Raileanu, R.; Rozière, B.; Schick, T.; Dwivedi-Yu, J.; Celikyilmaz, A.; Grave, E.; LeCun, Y.; and Scialom, T. 2023.  Augmented Language Models: a Survey.  arXiv:2302.07842.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.  arXiv:2303.08774.
  * Parisi, Zhao, and Fiedel (2022) Parisi, A.; Zhao, Y.; and Fiedel, N. 2022.  TALM: Tool Augmented Language Models.  _CoRR_ , abs/2205.12255.
  * Patil et al. (2023) Patil, S. G.; Zhang, T.; Wang, X.; and Gonzalez, J. E. 2023.  Gorilla: Large Language Model Connected with Massive APIs.  arXiv:2305.15334.
  * Qin et al. (2023a) Qin, Y.; Hu, S.; Lin, Y.; Chen, W.; Ding, N.; Cui, G.; Zeng, Z.; Huang, Y.; Xiao, C.; Han, C.; Fung, Y. R.; Su, Y.; Wang, H.; Qian, C.; Tian, R.; Zhu, K.; Liang, S.; Shen, X.; Xu, B.; Zhang, Z.; Ye, Y.; Li, B.; Tang, Z.; Yi, J.; Zhu, Y.; Dai, Z.; Yan, L.; Cong, X.; Lu, Y.; Zhao, W.; Huang, Y.; Yan, J.; Han, X.; Sun, X.; Li, D.; Phang, J.; Yang, C.; Wu, T.; Ji, H.; Liu, Z.; and Sun, M. 2023a.  Tool Learning with Foundation Models.  arXiv:2304.08354.
  * Qin et al. (2023b) Qin, Y.; Liang, S.; Ye, Y.; Zhu, K.; Yan, L.; Lu, Y.; Lin, Y.; Cong, X.; Tang, X.; Qian, B.; Zhao, S.; Tian, R.; Xie, R.; Zhou, J.; Gerstein, M.; Li, D.; Liu, Z.; and Sun, M. 2023b.  ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs.  arXiv:2307.16789.
  * Schick et al. (2023) Schick, T.; Dwivedi-Yu, J.; Dessì, R.; Raileanu, R.; Lomeli, M.; Zettlemoyer, L.; Cancedda, N.; and Scialom, T. 2023.  Toolformer: Language Models Can Teach Themselves to Use Tools.  _CoRR_ , abs/2302.04761.
  * Schick and Schütze (2021) Schick, T.; and Schütze, H. 2021.  Generating Datasets with Pretrained Language Models.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , 6943–6951. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics.
  * Shen et al. (2023) Shen, Y.; Song, K.; Tan, X.; Li, D.; Lu, W.; and Zhuang, Y. 2023.  HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face.  arXiv:2303.17580.
  * Song et al. (2023) Song, Y.; Xiong, W.; Zhu, D.; Li, C.; Wang, K.; Tian, Y.; and Li, S. 2023.  RestGPT: Connecting Large Language Models with Real-World Applications via RESTful APIs.  arXiv:2306.06624.
  * Taori et al. (2023) Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023.  Stanford Alpaca: An Instruction-following LLaMA model.  https://github.com/tatsu-lab/stanford˙alpaca.
  * Wang et al. (2021) Wang, S.; Liu, Y.; Xu, Y.; Zhu, C.; and Zeng, M. 2021.  Want To Reduce Labeling Cost? GPT-3 Can Help.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , 4195–4205. Punta Cana, Dominican Republic: Association for Computational Linguistics.
  * Wang et al. (2023) Wang, Y.; Kordi, Y.; Mishra, S.; Liu, A.; Smith, N. A.; Khashabi, D.; and Hajishirzi, H. 2023.  Self-Instruct: Aligning Language Models with Self-Generated Instructions.  arXiv:2212.10560.
  * Xu et al. (2023) Xu, Q.; Hong, F.; Li, B.; Hu, C.; Chen, Z.; and Zhang, J. 2023.  On the Tool Manipulation Capability of Open-source Large Language Models.  arXiv:2305.16504.
  * Yang et al. (2023) Yang, R.; Song, L.; Li, Y.; Zhao, S.; Ge, Y.; Li, X.; and Shan, Y. 2023.  GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction.  arXiv:2305.18752.
  * Yao et al. (2022) Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.; and Cao, Y. 2022.  ReAct: Synergizing Reasoning and Acting in Language Models.  _arXiv preprint arXiv:2210.03629_.
  * Zhou et al. (2023) Zhou, C.; Liu, P.; Xu, P.; Iyer, S.; Sun, J.; Mao, Y.; Ma, X.; Efrat, A.; Yu, P.; Yu, L.; Zhang, S.; Ghosh, G.; Lewis, M.; Zettlemoyer, L.; and Levy, O. 2023.  LIMA: Less Is More for Alignment.  arXiv:2305.11206.

##  Appendix A Implementation Details

In this section, we show the details of prompt templates in ToolAlpaca. Figure
6, Figure 7, and Figure 8 delineate the prompts employed during toolset
construction. Figure 9 and Figure 10 illustrate the corresponding prompts for
user agent’s two responsibility, generating user instructions and providing
missing information. The prompts designed for to the assistant agent and the
tool executor agent are detailed in Figure 11 and Figure 12. ”${…}” within the
prompts are placeholders, will be replaced by real variables during the
generation process.

##  Appendix B Experiment Details

###  B.1 Hyperparameters

The fine-tuning configuration for ToolAlpaca is recorded in Table 5.

Hyperparameters | Value
---|---
optimizer | AdamW
learning rate | 2e-5
weight decay | 0.0
warmup ratio | 0.03
lr scheduler type | cosine
num train epochs | 3
batch size | 128
max length | 2048

Table 5: The fine-tuning configuration for ToolAlpaca.

###  B.2 Evaluation Dataset Details

To evaluate the generalized tool-use ability, we construct the evaluation
dataset via our framework, which consists two subsets: a simulated subset with
10 simulated tools and 100 instances, a real-world subset with 11 real-world
APIs and 114 instances. The toolset used in the evaluation datset is detailed
in Table 6.

Name | Category | Introduction
---|---|---
Simulated Tools |  |
Axolotl | Animals | Collection of axolotl pictures and facts
AniAPI | Anime | Anime discovery, streaming & syncing with trackers
AbuseIPDB | Anti-Malware | IP/domain/URL reputation
Améthyste | Art & Design | Generate images for Discord users
Auth0 | Authentication & Authorization | Easy to implement, adaptable authentication and authorization platform
Abstract Public Holidays | Calendar | Data on national, regional, and religious holidays via API
1Forge | Currency Exchange | Forex currency market data
A Bíblia Digital | Books | Do not worry about managing the multiple versions of the Bible
Apache Superset | Business | API to manage your BI dashboards and data sources on Superset
Lob.com | Data Validation | US Address Verification
Real-world APIs |  |
Nager.Date | Calendar | Public holidays for more than 90 countries
airportsapi | Transportation | Get name and website-URL for airports by ICAO code
AviationAPI | Transportation | FAA Aeronautical Charts and Publications, Airport Information, and Airport Weather
chucknorris.io | Entertainment | JSON API for hand curated Chuck Norris jokes
Random Useless Facts | Entertainment | Get useless, but true facts
apilayer weatherstack | Weather | Real-Time & Historical World Weather Data API
Free Dictionary | Dictionaries | Definitions, phonetics, pronounciations, parts of speech, examples, synonyms
WolframAlpha | Machine Learning | Provides specific answers to questions using data and algorithms
Fruityvice | Food & Drink | Data about all kinds of fruit
Cataas | Animals | Cat as a service (cats pictures and gifs)
CurrencyBeacon | Currency Exchange | Real-time and historical currency rates JSON API

Table 6: Tools used in our evaluation dataset.

###  B.3 Evaluation Prompt

Following the evaluation method used by Vicuna (Chiang et al. 2023), we use
GPT-4 as our evaluator. The evaluation prompt is shown in Figure 13.

###  B.4 Case Study

Through training on a set of diverse simulated tool-use instances, ToolAlpaca
can equip various tools, even real-world APIs, some selected cases are shown
in Figure 14, Figure 15 and Figure 16.

Figure 6: Description generation prompt. Figure 7: Function documentation
generation prompt. Figure 8: Openapi specification generation prompt. Figure
9: User agent prompt 1 for instruction generation. Figure 10: User agent
prompt 2 for providing missing information. Figure 11: Assistant agent prompt.
Figure 12: Tool executor agent prompt. Figure 13: Evaluation prompt for GPT-4.
Figure 14: Cases for real-world API: Nager.Date. Figure 15: Cases for real-
world API: Currency Beacon. Figure 16: Cases for real-world API: apilayer
weatherstack.