# Electrical detection of parallel parametric amplification and attenuation in
$\mathrm{Y}_{3}\mathrm{Fe}_{5}\mathrm{O}_{12}$/$\mathrm{Pt}$ bilayer disk

Geil Emdi1 Tomosato Hioki 1,2<EMAIL_ADDRESS>Koujiro
Hoshi1,3 Eiji Saitoh1,2,3,4 1Department of Applied Physics,The University of
Tokyo,Tokyo 113-8656,Japan  2Advanced Institute for Materials Research, Tohoku
University, Sendai 980-857,Japan  3Institute for AI and Beyond, The University
of Tokyo,Tokyo 113-8656, Japan  4Advanced Science Research Center, Japan
Atomic Energy Agency, Tokai, Ibaraki 319-1195, Japan

###### Abstract

We report a systematic quantitative evaluation of parametric amplification
gain of magnetization dynamics in ytirrium iron garnet
($\mathrm{Y}_{3}\mathrm{Fe}_{5}\mathrm{O}_{12}$) thin disk via a.c. spin
pumping and inverse spin Hall effect. We demonstrate its signature phase-
dependence where amplification and attenuation occur every $\frac{\pi}{2}$
phase shift of the input signal. The results also show the pump-power
dependence of the gain that is explained well by our theoretical model.
Finally, the optimal conditions for the amplification is investigated by
measuring the magnetic field dependence, where we find the highest gain of
11.4 dB.

††preprint: APS/123-QED

Degenerate parametric amplification refers to an amplification of input waves
by using an excitation whose frequency is twice the resonance frequency. This
amplification is phase sensitive due to the non-linearity of a medium. Owing
to this phase-sensitive nature, the parametric process can selectively amplify
a signal based on its phase and frequency. This selective amplification is
widely used in applications such as wavelength conversion of light [1, 2, 3]
and microwave amplification at cryogenic temperatures [4, 5, 6, 7].

Spin waves, a precessional motion of magnetization in a magnetic material that
propagates as waves, can undergo parametric amplification due to the strong
nonlinearity from dipolar and exchange interactions [8, 9, 10, 11, 12, 13,
14]. In magnonics, which aims to realize unconventional computing systems
using spin waves, the phase dependence of parametric amplification is
extremely important for encoding information into the phase of magnons, a
quanta of spin waves. Many paths have been proposed to utilize parametric
amplification process in spin wave circuits [9, 10, 11]. By exploiting the
fact that in a parametric oscillation process, the phase of the output signal
wave can be discretized into two, which can be used for probabilistic bit
operation to realize optimization solvers [11, 15, 16, 17].

In this study, by measuring the parametric amplification process of spin waves
using a spin current measurement method, we systematically evaluated its
amplification rate, thereby clarifying the systematic behavior of the spin
wave amplification gain. In the experiments, by investigating gain dependence
on the phase of the spin wave signal and the pump amplitude, we demonstrate
that the spin wave amplification gain can exceed of more than an order of
magnitude under optimal conditions, and we show that this behavior can be
theoretically modelled by considering loss of spin waves in the magnet and
electromagnetic waves in a pick up circuit.

We use an ytirrium iron garnet
($\mathrm{Y}_{3}\mathrm{Fe}_{5}\mathrm{O}_{12}$; YIG) micro-sized thin disk
that is covered with platinum (Pt) thin layer as shown in Fig. 1(a). The
diameter of the YIG disk is 500 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$, and
the thickness is 1.4 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$. We grow the YIG
layer by Liquid Phase Epitaxy (LPE) process on top of gadolinium gallium
garnet (GGG) substrate. Then, the 10-nm-thick Pt film is sputtered on top of
the YIG layer. The Pt film is used for measuring the magnetization precession
in the YIG disk as a.c. voltage via the a.c. spin pumping and ISHE [18, 19,
20, 21, 22, 23, 24, 25, 26, 27, 28, 29]. The disk shape of YIG/Pt bilayer is
patterned by photolithography and Ar-ion milling process. Moreover, two gold
electrodes are sputtered at the edge of the Pt film. The fabricated sample is
placed on top of a pump coplanar waveguide (CPW) that is short-ended, with the
width of 100 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$. One of the gold
electrodes is grounded, while the other is connected to an input CPW [see Fig.
1(a)]. We measure the reflection ($|S_{11}|$) spectrum of the sample from the
two CPW ports and we choose the frequency of the weak input field to be 1f =
2.15 GHz to satisfy the ferromagnetic resonance (FMR) condition at static
magnetic field $\mu_{0}H=27.2\ $\mathrm{m}\mathrm{T}$$ applied in an in-plane
direction [see Fig. 1(b)].

Firstly, we generate the RF input field, $h_{1f}$, perpendicular to the static
field, through the input CPW and the gold electrode which drives the initial
magnetization precession in the sample at frequency 1f. Next, the pump field,
$h_{2f}$, that is parallel to static field, is generated from the short-ended
CPW [see Fig. 1(c)]. In the magnetic thin disk, the trajectory of the
magnetization precession is distorted into an ellipse owing to the dynamical
demagnetization field, which leads to the temporal change in the longitudinal
component of magnetization with doubled frequency of ferromagnetic resonance,
$M_{z,2f}$ [Fig. 1(d)]. Consequently, the pump field couples with the
longitudinal component, leading to the parametric amplification of the
precession dynamics [11, 9, 30]. The wavelength of the amplified spin waves is
determined by dispersion relation [Fig. 1(e)]. The resulting precession can be
detected via a.c. spin pumping and inverse spin hall effect (ISHE) as the
voltage read-out from the sample measured by signal analyzer (SA)[Fig. 1(c)].
In this experiment, we set the frequency of the pump field to be 2f = 4.3 GHz
and the input power to be $P_{1f}=10\
$\mathrm{\SIUnitSymbolMicro}\mathrm{W}$$.

Figure 1:  (a) Sample configuration of YIG/Pt bilayer. (b) $|S_{11}|$ spectrum
taken at $\mu_{0}H=27.2\ $\mathrm{m}\mathrm{T}$$ from different ports: Input
CPW (blue) and Pump CPW (pink). (c) The microwave circuit diagram. (d)
Schematic of elliptical trajectory of magnetisation precession in magnetic
thin disk. $M_{\mathrm{in}}$ is the initial and $M_{\mathrm{out}}$ is the
final magnetization. (e) Spin waves dispersion relation. A photon converts
into two magnon with half of the photon frequency,
$f_{\mathrm{p}}=\frac{\omega_{\mathrm{p}}}{2\pi}$.

To estimate the gain, we distinguish the ISHE voltage representing the
magnetization precession from the voltage that is simply reflected off the
sample by subtracting the voltage from different field ($\mu_{0}H=23\
$\mathrm{m}\mathrm{T}$$). We evaluate the gain as:

$\mathrm{Gain}=\frac{V_{2f}^{\mathrm{FMR}}-V_{2f}^{\mathrm{Non-
FMR}}}{V_{1f}^{\mathrm{FMR}}-V_{1f}^{\mathrm{Non-FMR}}},$ (1)

where $V_{2f}^{\mathrm{FMR}}$ is the voltage read-out when both $h_{2f}$ and
$h_{1f}$ are applied, while $V_{1f}^{\mathrm{FMR}}$ is the read-out when only
$h_{1f}$ is applied (pump field, $h_{2f}$ is switched off) at $\mu_{0}H=27.2\
$\mathrm{m}\mathrm{T}$$. Lastly, $V_{2f}^{\mathrm{Non-FMR}}$ and
$V_{1f}^{\mathrm{Non-FMR}}$ are the read-out voltages at $\mu_{0}H=23\
$\mathrm{m}\mathrm{T}$$.

We have performed the systematic measurement of the gain as a function of pump
power and input phase, as shown in the heat map in Fig. 2(a). The measurement
is taken below the pump threshold ($P_{2f}<P_{\mathrm{th}}$). The pump
threshold is defined to be the critical pump power at which amplification
takes place even in the absence of the input field to drive the initial
magnetisation dynamics.

In Fig. 2(b), we demonstrate the periodic dependence of the gain on the input
phase, $\varphi_{\mathrm{s}}$. The period of the phase-dependence is $\pi$
with maximum gain (amplification) and minimum gain (attenuation) occurring
every $\frac{\pi}{2}$ phase-shift of $h_{1f}$.

Figure 2:  (a) Heat map of the gain (dB) with respect to pump power and input
phase. (b) Gain dependence on input phases at three different pump power. (c)
The dependence of maximum gain (amplification) and minimum gain (attenuation)
on pump power. The black dashed-curves represent the 0.68 confidence bound of
the fit curves. The grey dotted-line shows the gain when the pump field is
switched off.

To explain this phase-dependence of the gain, we consider that in a parametric
amplification, a weak input signal wave and a strong pump wave are introduced
into this non-linear system. The signal wave is then amplified and an idler
wave, whose frequency is equal to the signal’s frequency, is generated.
Therefore, the resulting output wave is the superposition of the signal and
idler waves. The two waves interfere with each other constructively or
destructively, leading to amplification or attenuation of the signal which
depends on the input signal phase.

To maximize the energy transferred from the pump into the signal and idler
waves [10, 31, 32] and to meet the phase-matching condition (see Supplemental
Material [33]), the phases of the signal ($\varphi_{\mathrm{s}}$), idler
$(\varphi_{\mathrm{i}})$, and pump $(\varphi_{\mathrm{p}})$ waves must satisfy
the relation :

$\varphi_{\mathrm{s}}+\ \varphi_{\mathrm{i}}=\ \varphi_{\mathrm{p}}+\ \
\pi/2.$ (2)

From this relation, the phase difference between signal and idler,
$\Delta\varphi_{\mathrm{si}}=\varphi_{\mathrm{s}}-\varphi_{\mathrm{i}}$, in
accordance to equation (1), follows:

$\Delta\varphi_{\mathrm{si}}=2\Delta\varphi_{\mathrm{sp}}-\frac{\pi}{2},$ (3)

where the phase difference between the signal and pump,
$\Delta\varphi_{\mathrm{sp}}=\varphi_{\mathrm{s}}-\frac{\varphi_{\mathrm{p}}}{2}$.

Following equation (3), when $\Delta\varphi_{\mathrm{sp}}=\frac{\pi}{4}$
($\frac{3\pi}{4}$), the signal-idler phase difference becomes
$\Delta\varphi_{\mathrm{si}}=0$ ($\pi$) and they interfere constructively
(destructively) (see Supplemental Material [33]). As a result, for a fixed
pump’s phase, we can expect an attenuation and amplification to occur every
$\frac{\pi}{2}$ shift of the input signal’s phase.

The dependence of the maximum gain and minimum gain on the amplitude of pump
power is shown in Fig. 2(c). From the data, there are non-linear rise
(decrease) of the maximum (minimum) gain as a function of pump power.

In order to discuss the dependence of the gain on the pump power and the input
phase, we consider the microscopic picture of parallel parametric
amplification process. Using the input-output formalism [34, 7], this process
can be described by a system which interacts with the heat bath and the signal
port, where the input and output signals are considered. The Hamiltonian
describing this process is written as [7]:

${\hat{H}}=\hat{H}_{\mathrm{system}}+{\hat{H}}_{\mathrm{port}}+{\hat{H}}_{\mathrm{bath}}.$
(4)

The system Hamiltonian is expressed as [34]

$\hat{H}_{\mathrm{system}}={\hat{H}}_{0}+{\hat{H}}_{\mathrm{int}}+{\hat{H}}_{\mathrm{pump}},$
(5)

where ${\hat{H}}_{0}$ is the Hamiltonian describing harmonic oscillators
system, described as:

${\hat{H}}_{0}=\
\hbar\omega{\hat{c}}^{\dagger}{\hat{c}}+\hbar\omega_{\mathrm{p}}{\hat{a}}^{\dagger}{\hat{a}},$
(6)

where $\hat{c}$ ($\hat{c}^{\dagger}$) is the annihilation (creation) operator
of the magnon with frequency $\omega=2\pi f$ (after diagonalization of the
quadratic terms in the dipolar interaction Hamiltonian [8]). ${\hat{a}}$
($\hat{a}^{\dagger}$) is the annihilation (creation) operator of pump photon
of frequency $\omega_{\mathrm{p}}$. In the degenerate parametric amplification
case, we consider $\omega_{\mathrm{p}}=2\omega$.

The interaction Hamiltonian, ${\hat{H}}_{\mathrm{int}}$ describes the
parametric process that splits a pump photon into two magnons; it is written
as:

${\hat{H}}_{\mathrm{int}}=\frac{i\hbar}{2}\rho_{k}\left({\hat{c}}^{\dagger}{\hat{c}}^{\dagger}{\hat{a}}-{\hat{c}}{\hat{c}}{\hat{a}}^{\dagger}\right),$
(7)

where the coupling parameter between the pump photon and magnon,
$\rho_{k}=\frac{\omega_{M}}{4}\sqrt{\frac{\omega_{M}}{\omega}}\sin^{2}{\theta_{k}e^{-i2\phi_{k}}}$;
we use $\omega_{M}=\gamma\mu_{0}M_{\mathrm{s}}$ with $M_{s}$ as the saturation
magnetization, $\gamma$ as the gyromagnetic ratio, $\theta_{k}$ and $\phi_{k}$
as the polar and azimuthal angle of the wavevector, $\mathbf{k}$ [8].

${\hat{H}}_{\mathrm{pump}}$ represents the driving pump fields in generating
the pump photons. It is written as

${\hat{H}}_{\mathrm{pump}}=i\hbar\left(\varepsilon_{\mathrm{p}}e^{-i\left(\omega_{\mathrm{p}}t+\varphi_{\mathrm{p}}\right)}{\hat{a}}^{\dagger}-{\varepsilon_{\mathrm{p}}e}^{i\left(\omega_{\mathrm{p}}t+\varphi_{\mathrm{p}}\right)}{\hat{a}}\right),$
(8)

where $\varphi_{\mathrm{p}}$ is the pump’s phase, $\varepsilon_{\mathrm{p}}$
is the amplitude of the classical pump field,
$\varepsilon_{\mathrm{p}}=\gamma|h_{2f}|$ .

The Hamiltonian describing the interaction between the magnon in the system
and the input and output fields are expressed as

${\hat{H}}_{\mathrm{port}}=i\hbar\sqrt{\frac{\gamma_{\mathrm{0}}}{2\pi}}\left[\left({\hat{b}}_{\mathrm{in}}{\hat{c}}^{\dagger}-{{\hat{b}}_{\mathrm{in}}^{\dagger}{\hat{c}}}\right)+\left({\hat{b}}_{\mathrm{out}}{\hat{c}}^{\dagger}-{{\hat{b}}_{\mathrm{out}}^{\dagger}{\hat{c}}}\right)\right],$
(9)

where ${\hat{b}}_{\mathrm{in}}$(${\hat{b}}_{\mathrm{in}}^{\dagger}$) and
${\hat{b}}_{\mathrm{out}}$(${\hat{b}}_{\mathrm{out}}^{\dagger}$) are the
annihilation (creation) operators of the input and output field respectively.
$\gamma_{0}$ is the damping constant between the system and the signal port.

By input-output theory, the relation between the output and input operators to
the magnon operator in the system are expressed as [7, 35, 34, 36]

${\hat{b}}_{\mathrm{out}}={\hat{b}}_{\mathrm{in}}-\sqrt{2\pi\gamma_{0}}{\hat{c}}.$
(10)

Lastly the damping of photon and magnons to the phonons bath are assumed to be
written as

${\hat{H}}_{\mathrm{bath}}={\hat{c}}{\hat{\Gamma}}_{1}^{\dagger}+{\hat{c}}^{\dagger}{\hat{\Gamma}}_{1}+{\hat{a}}{\hat{\Gamma}}_{2}^{\dagger}+{\hat{a}}^{\dagger}{\hat{\Gamma}}_{2},$
(11)

where ${\hat{\Gamma}}_{i}$$\left(\hat{\Gamma}_{i}^{\dagger}\right)$ with
$i=1,2$ are the annihilation (creation) operators of the harmonic oscillator
in the thermal bath of phonons.

We consider the coherent state of magnon mode $\left|c\right\rangle$ with
eigenvalue $c$, such that
${\hat{c}}\left|c\right\rangle=c\left|c\right\rangle$. Similarly, we have
$\left|a\right\rangle$ with eigenvalue $a$ and
${\hat{a}}\left|a\right\rangle=a\left|a\right\rangle$ for the photon mode.
Lastly, for the input and output operators, we have
${\hat{b}}_{j}\left|b_{j}\right\rangle=b_{j}\left|b_{j}\right\rangle$
($j=\mathrm{in},\mathrm{out}$), where
$b_{\mathrm{in}}=\left|b_{\mathrm{in}}\right|e^{-i\varphi_{\mathrm{s}}}$ and
$\varphi_{\mathrm{s}}$ is the phase of the input signal.

In the rotating frame of frequency $\omega=\frac{\omega_{\mathrm{p}}}{2}$ , we
derive the quantum Langevin equations by taking the trace over the reservoir
to be

$\displaystyle\begin{split}\frac{dc}{dt}=\sqrt{\frac{\gamma_{\mathrm{0}}}{2\pi}}\left(b_{\mathrm{in}}+b_{\mathrm{out}}\right)+\rho_{k}c^{\ast}a-\gamma_{1}c+\sqrt{\rho_{k}a}\eta_{1},\end{split}$
(12a)
$\displaystyle\begin{split}\frac{dc^{\ast}}{dt}=\sqrt{\frac{\gamma_{\mathrm{0}}}{2\pi}}\left(b_{\mathrm{in}}^{*}+b_{\mathrm{out}}^{*}\right)+\rho_{k}ca^{\ast}-\gamma_{1}c^{\ast}+\sqrt{\rho_{k}a^{\ast}}\eta_{1}^{\dagger},\end{split}$
(12b)
$\displaystyle\begin{split}\frac{da}{dt}=\varepsilon_{\mathrm{p}}e^{-i\varphi_{\mathrm{p}}}-\frac{\rho_{k}}{2}c^{2}-\gamma_{2}a,\end{split}$
(12c)

where $\gamma_{1}$ and $\gamma_{2}$ are the damping rate constants of the
magnon and photon modes decay to phonon bath. $\eta_{1}\left(t\right)$ is the
delta-correlated random force with zero mean (see Supplemental Material [33]).

By using the steady state solutions of equation (12) and considering
substitutions with equation (10), we can get the expression of output field.
The gain is then defined as the square of ratio of output field when pump
microwave ($h_{2f}$) is applied
$\left|b_{\mathrm{out}}(\varepsilon_{\mathrm{p}})\right|$ to its amplitude
when the pump field is switched off
$\left|b_{\mathrm{out}}(\varepsilon_{\mathrm{p}}=0)\right|$. The gain is
expressed as

$\begin{split}\mathrm{Gain}&=\left|\frac{b_{\mathrm{out}}(\varepsilon_{\mathrm{p}})}{b_{\mathrm{out}}(\varepsilon_{\mathrm{p}}=0)}\right|^{2}\\\
&=\beta\Big{[}\mathrm{X}_{\mathrm{g}}\cos^{2}{(\Delta\varphi_{\mathrm{sp}})}+\mathrm{Y}_{\mathrm{g}}\sin^{2}{(\Delta\varphi_{\mathrm{sp}})}\Big{]},\end{split}$
(13)

where
$\mathrm{X}_{\mathrm{g}}=\left(\frac{\gamma_{0}-\gamma_{1}+\tilde{\rho}\varepsilon_{\mathrm{p}}}{\gamma_{0}+\gamma_{1}-\tilde{\rho}\varepsilon_{\mathrm{p}}}\right)^{2}$,$\mathrm{Y}_{\mathrm{g}}=\left(\frac{\gamma_{0}-\gamma_{1}-\tilde{\rho}\varepsilon_{\mathrm{p}}}{\gamma_{0}+\gamma_{1}+\tilde{\rho}\varepsilon_{\mathrm{p}}}\right)^{2}$,
and
$\beta=\left(\frac{\gamma_{0}+\gamma_{1}}{\gamma_{0}-\gamma_{1}}\right)^{2}$
(see Supplemental Material). The phase difference between pump and signal is
$\Delta\varphi_{\mathrm{sp}}=\varphi_{\mathrm{s}}-\frac{\varphi_{\mathrm{p}}}{2}$;
we use approximation, $|c|<<|a|$, to substitute
$a\approx\frac{\varepsilon_{\mathrm{p}}}{\gamma_{2}}e^{-i\varphi_{\mathrm{p}}}$;
and $\tilde{\rho}=\frac{\rho_{k}}{\gamma_{2}}$ is the effective coupling
constant between the magnon and the pump photon. The threshold amplitude of
pump field is
$\varepsilon_{\mathrm{th}}=\frac{\gamma_{0}+\gamma_{1}}{\tilde{\rho}}$.

Below threshold , $\varepsilon_{\mathrm{p}}<\varepsilon_{\mathrm{th}}$, there
is a clear dependence of the gain on input phase $\varphi_{s}$ and pump field
amplitude, $\varepsilon_{\mathrm{p}}$. Maximum gain (amplification) occurs
when $\Delta\varphi_{\mathrm{sp}}=0$, where
$\mathrm{Gain}=\beta\mathrm{X}_{\mathrm{g}}=\beta\left(\frac{\gamma_{0}-\gamma_{1}+\tilde{\rho}\varepsilon_{\mathrm{p}}}{\gamma_{0}+\gamma_{1}-\tilde{\rho}\varepsilon_{\mathrm{p}}}\right)^{2}>1$
; and minimum gain (attenuation) occurs when
$\Delta\varphi_{\mathrm{sp}}=\pi/2$, where
$\mathrm{Gain}=\beta\mathrm{Y}_{\mathrm{g}}=\beta\left(\frac{\gamma_{0}-\gamma_{1}-\tilde{\rho}\varepsilon_{\mathrm{p}}}{\gamma_{0}+\gamma_{1}+\tilde{\rho}\varepsilon_{\mathrm{p}}}\right)^{2}<1$.
For a fixed pump phase, $\varphi_{\mathrm{p}}$, these two conditions are
alternatingly satisfied every time there is a phase-shift of $\frac{\pi}{2}$
of the input field.

Figure 3:  (a) Heat map of the gain with respect to pump power and input phase
at high power range from 0.2 to 0.7 W. (b) The dependence of maximum gain
(amplification) and minimum gain (attenuation) on pump power. (c) Heat map of
the gain with respect to static magnetic field and input phase. Measurement is
taken at pump power $P_{2f}=0.398\ $\mathrm{W}$$. (d) The dependence of
maximum gain (amplification) and minimum gain (attenuation) on static magnetic
field. (e) The ISHE voltages at varying fields, taken at $1f=2.15\
$\mathrm{G}\mathrm{H}\mathrm{z}$$, with pump field (green) and without pump
field (red). (f) Numerically calculated spatial patterns of standing spin
waves at $\mu_{0}H=27.2\ $mT for $f=2.13\ $\mathrm{G}\mathrm{H}\mathrm{z}$$
and $f=2.15\ $\mathrm{G}\mathrm{H}\mathrm{z}$$.

Using equation (13), we can fit the data presented in Figs. 2(b) and (c). We
note that there is a fixed offset of the phase 10.5 rad, owing to the
different pump and input channel in the experimental circuit. In Fig. 2(c), we
consider pump power, $P_{2f}\propto{\varepsilon_{\mathrm{p}}}^{2}$ and
threshold power, $P_{\mathrm{th}}\propto{\varepsilon_{\mathrm{th}}}^{2}$. From
the fitting parameter, we can derive the threshold power at resonance
condition $\mu_{0}H=27.2\ $mT to be $P_{\mathrm{th}}=\left(0.8\pm 0.1\right)\
$\mathrm{W}$$, the damping constant of magnon mode to the heat bath,
$\gamma_{1}=(1.5\pm 0.1)\ $\mathrm{M}\mathrm{H}\mathrm{z}$$, and the damping
rate between the system and signal port to be $\gamma_{0}=(55.6\pm 0.4)\
$\mathrm{M}\mathrm{H}\mathrm{z}$$.

On one hand, the damping rate $\gamma_{1}$ is in the same order of magnitude
as the ferromagnetic resonance (FMR) linewidth $\alpha f=2.5\
$\mathrm{M}\mathrm{H}\mathrm{z}$$, where we have evaluated the effective
Gilbert damping parameter, $\alpha=1.15\times 10^{-3}$, from the field
dependence of the linewidth of FMR (see Supplemental Material [33]); thus,
verifying that $\gamma_{1}$ describes the rate of energy dissipation of the
system to the lattice. On another hand, the damping rate $\gamma_{0}$ is
consistent with the value of the linewidth of $|S_{11}|$ spectrum taken from
the input CPW port with $\Delta f_{\mathrm{Input}}=53.4\
$\mathrm{M}\mathrm{H}\mathrm{z}$$ [see Fig. 1(b)]; and thus, affirming that
$\gamma_{0}$ describes the energy loss between the system- YIG/Pt bilayer
disk, to the signal port and the input CPW.

Next, a systematic measurement has been performed at higher pump power range
as shown in Fig. 3(a) and (b). The pump power is varied from 0.2 W to 0.7 W.
In Fig. 3(a), we present the heat map of gain as a function of input phase and
pump power. The alternating attenuation and amplification across the
horizontal axis correspond to the periodic dependence of gain on the input
phase.

Figure 3(b) shows the maximum and minimum gain as a function of pump power. As
we increase the pump power, the maximum gain rises and starts to saturate at
around 0.4 W then it decreases at higher power. We found this saturation power
to be lower than the predicted threshold power $P_{\mathrm{th}}=(0.8\pm
0.1)$\mathrm{W}$$. The saturation of gain can be attributed to the Kerr non-
linearity which limit the parametrically amplified magnon number[16, 37, 32,
31]. The magnon Kerr effect induces an appreciable shift of the magnon
frequency[38], leading to a frequency mismatch between the magnon and the
driving fields. Hence, increasing the pump power cannot amplify the magnon
number any further. The subsequent decrease of the gain at higher pump power
may be due to other non-linear effects that come from scatterings between
different magnon modes, such as the second order Suhl instability which opens
up an additional decay channel for the amplified magnons[39, 40, 31].

To search for the best condition in obtaining the largest gain, we measure the
magnetic field dependence of the gain as shown in Figs. 3(c) and (d). The
measurement is taken at pump power $P_{2f}=0.398\ $\mathrm{W}$$. We observe
two peaks structure at $\mu_{0}H=27.2\ $\mathrm{m}\mathrm{T}$$ with maximum
gain of $10.7\ $\mathrm{d}\mathrm{B}$$ and at $\mu_{0}H=27.6\
$\mathrm{m}\mathrm{T}$$ with the gain of $11.4\ $\mathrm{d}\mathrm{B}$$ .

We first present the ISHE voltage data in Fig. 3(e), defined as
$V_{i}^{\mathrm{FMR}}-V_{i}^{\mathrm{Non-FMR}}$ where $i=1f,2f$, to explain
the origin of the two peaks in the field dependence. When $h_{2f}$ is switched
off (red markers), we observe a single peak at $\mu_{0}H=27.2\
$\mathrm{m}\mathrm{T}$$ , corresponding to a uniform precession excited by
ferromagnetic resonance (FMR) [ see also Fig. 1(b)]. However, upon applying
$h_{2f}$ (green markers), we observe an additional peak at $\mu_{0}H=27.6\
$\mathrm{m}\mathrm{T}$$, which indicate an amplification of a different magnon
mode.

One possible mode at $\mu_{0}H=27.6\ $\mathrm{m}\mathrm{T}$$ is the in-plane
standing wave mode with $|\mathbf{k}|>0$ [41, 42]. In Fig. 1(b), we show the
$|S_{11}|$ spectrum taken from the pump CPW (pink curve) at $\mu_{0}H=27.2\
$\mathrm{m}\mathrm{T}$$. The peak occurs at the frequency, $f=2.13\
$\mathrm{G}\mathrm{H}\mathrm{z}$$, less than that of the experimental
condition at $f=2.15\ $\mathrm{G}\mathrm{H}\mathrm{z}$$ (grey dashed line).
This is expected for a standing spin wave mode with
$\mathbf{k}\parallel\mathbf{M}$ as frequency decreases with increasing
wavevector. In consequence, at a fixed frequency of $f=2.15\
$\mathrm{G}\mathrm{H}\mathrm{z}$$, the excitation of $|\mathbf{k}|>0$ standing
wave mode will appear at a higher field corresponding to the second peak
structure.

Figure 3(f) shows the numerically calculated spatial amplitude profiles of the
two modes under the applied static field of $\mu_{0}H=27.2\
$\mathrm{m}\mathrm{T}$$. These results present the different standing waves
modes: $|\mathbf{k}|>0$ at $f=2.13\ $\mathrm{G}\mathrm{H}\mathrm{z}$$ and FMR
mode at $f=2.15\ $\mathrm{G}\mathrm{H}\mathrm{z}$$, which are consistent to
our interpretation of the experimental data. The standing wave pattern at
$f=2.13\ $\mathrm{G}\mathrm{H}\mathrm{z}$$ is slightly inclined due to
presence of an anisotropy field that is directed at an angle to the external
static field (see Supplemental Material [33]).

From the experimental data and numerical calculations, we conclude that the
two modes, FMR and $|\mathbf{k}|>0$ standing wave, are the optimal conditions
of parametric amplification potentially due to the coupling efficiency between
magnons and phonons, which is determined by coupling parameter and mode
overlap. The coupling parameter between photon and magnon,
$\rho_{k}=\frac{\omega_{M}}{4}\sqrt{\frac{\omega_{M}}{\omega}}\sin^{2}{\theta_{k}e^{-i2\phi_{k}}}$,
is maximized at $\theta_{k}=\frac{\pi}{2}$ and at minimum frequency, $\omega$.
As shown in the dispersion curve in Fig. 1(b), the FMR mode satisfies this
condition. The coupling efficiency between the magnons in the YIG/Pt disk and
pump photons in CPW is also affected by the mode overlaps between the magnons
and photons [43, 44, 45]. As the CPW’s width ($100\
$\mathrm{\SIUnitSymbolMicro}\mathrm{m}$$) is narrower than the disk’s diameter
($500\ $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$$), the maximum modes overlap
occurs locally on their area of contact [44]. As we see in Fig. 3(f), the two
modes, FMR and $|\mathbf{k}|>0$ standing waves would cover the area of contact
between the pumping CPW and the magnetic disk; thus, satisfying the second
factor for the optimal amplification.

In summary, we systematically investigate parallel parametric amplification
and attenuation in YIG/Pt bilayer disk by means of ac spin pumping and inverse
spin-Hall effect. The experimental data of the gain dependence on input phase
and pump power well below the pump threshold are in good agreement with the
theoretical analysis. Furthermore, we have measured the gain dependence on the
pump power at higher power range and its dependence on the static magnetic
field.

###### Acknowledgements.

We thank T. Makiuchi and H. Shimizu for fruitful discussions. This work was
partially supported by JST CREST (JPMJCR20C1 and JPMJCR20T2), JSPS KAKENHI
(JP19H05600, JP22K14584, and JP22H05114), Advanced Technology Institute
Research Grants 2022, Institute for AI and Beyond of the University of Tokyo,
and IBM–UTokyo lab.

## References

  * [1] R. Loudon, The Quantum Theory of Light (Oxford, Oxford University Press, 2000)
  * [2] D. J. Parker, M. Savytskyi, W. Vine, A. Laucht, T. Duty, A. Morello, A. L. Grimsmo, and J. J. Pla, Phys. Rev. Appl. 17, 034064 (2022).
  * [3] C. M. Caves and D. D. Crouch, J. Opt Soc Am B. 4, 1535 (1987).
  * [4] J. Aumentado, IEEE Microw. Mag. 21 no. 8, 45 (2020).
  * [5] J. Aumentado, Prospects for parametric amplifiers in large-scale superconducting quantum computing, in 2022 IEEE/MTT-S International Microwave Symposium - IMS 2022 (2022) pp. 76–79.
  * [6] T. Yamamoto, K. Inomata, M. Watanabe, et al., Appl. Phys. Lett. 93, 042510 (2008).
  * [7] T. Yamamoto, K. Koshino, and Y. Nakamura, ParametricAmplifier and Oscillator Based on Josephson Junction Circuitry, in Lecture Notes in Physics, Berlin Springer Verlag, Vol. 911, edited by Y. Yamamoto and K. Semba (2016) p. 495.
  * [8] S. M. Rezende, Fundamentals of Magnonic (Cham, Springer Nature Switzerland AG, 2020).
  * [9] T. Brächer, F. Heussner, P. Pirro, et al., Sci. Rep. 6, 38235 (2016).
  * [10] T. Brächer, P. Pirro, and B. Hillebrands, Phys. Rep. 699, 1 (2017).
  * [11] T. Makiuchi, T. Hioki, Y. Shimazu, et al., Appl. Phys. Lett. 118, 022402 (2021).
  * [12] A.A. Serga, S. O. Demokritov, B. Hillebrands, S. Min, and A.N. Slavin, J. Appl. Phys. 93, 8585 (2003).
  * [13] G.A. Melkov, A.A. Serga, A.N. Slavin, et al., J. Exp. Theor. Phys. 89, 1189 (1999).
  * [14] G.A. Melkov, A.A. Serga, V. S. Tiberkevich, Y.V. Kobljanskij, A.N. Slavin, Phys. Rev. E 63, 066607 (2001).
  * [15] T. Hioki, H. Shimizu, T. Makiuchi, and E. Saitoh, Phys. Rev. B 104, L100419 (2021).
  * [16] M. Elyasi, E. Saitoh, and G. E. W. Bauer, Phys. Rev. B 105, 054403 (2022).
  * [17] T. Hioki and E. Saitoh, J. Appl. Phys. 132, 203901 (2022).
  * [18] D. Wei, M. Obstbaum, M. Ribow, C. H. Back, and G. Woltersdorf, Nat. Commun. 5, 3768 (2014).
  * [19] C. Hahn, G. de Loubens, M. Viret, O. Klein, V. V. Naletov, and J. Ben Youssef, Phys. Rev. Lett. 111, 217204 (2013).
  * [20] H.J. Jiao and G. E. W. Bauer, Phys. Rev. Lett. 110, 217602 (2013).
  * [21] M. Weiler, J. M. Shaw, H. T. Nembach, and T. J. Silva, Phys. Rev. Lett. 113, 157204 (2014).
  * [22] E. Saitoh, M. Ueda, H. Miyajima, and G. Tatara, Appl. Phys. Lett. 88, 182509 (2006).
  * [23] A. Azevedo, L. H. Vilela Le˜ao, R. L. Rodriguez-Suarez, A. B. Oliveira, and S. M. Rezende, J. Appl. Phys. 97, 10C715 (2005).
  * [24] T. Kimura, Y. Otani, T. Sato, S. Takahashi, and S. Maekawa, Phys. Rev. Lett. 98, 156601 (2007).
  * [25] S. O. Valenzuela and M. Tinkham, Nature 442, 176 (2006).
  * [26] M. V. Costache, M. Sladkov, S. M. Watts, C. H. van der Wal, and B. J. van Wees, Phys. Rev. Lett. 97, 216603 (2006).
  * [27] S. Mizukami, Y. Ando, and T. Miyazaki, Phys. Rev. B 66, 104413 (2002).
  * [28] Y. Kajiwara, K. Harii, S. Takahashi, J. Ohe, K. Uchida, M. Mizuguchi, H. Umezawa, H. Kawai, K. Ando, K. Takanashi, S. Maekawa, and E. Saitoh, Nature 464, 262 (2010).
  * [29] Y. Tserkovnyak, A. Brataas, and G. E. W. Bauer, Phys. Rev. Lett. 88, 117601 (2002).
  * [30] K. Hoshi, T. Hioki, and E. Saitoh, Appl. Phys. Lett. 121, 212404 (2022).
  * [31] V. L’vov, Wave Turbulence Under Parametric Excitation (Heidelberg: Springer-Verlag Berlin- Heidelberg, 1994).
  * [32] V. E. Zakharov, V. S. L’vov, and S. S. Starobinets, PHYS-USP+ 17, 896 (1975).
  * [33] See Supplementary Material at [URL] for detailed derivation of the gain equation, the evaluation of Gilbert damping, and system parameters for numerical calculation.
  * [34] D. Walls and G. Milburn, Quantum Optics (Heidelberg: Springer-Verlag Berlin- Heidelberg, 2008).
  * [35] C. W. Gardiner and M. J. Collett, Phys. Rev. A 31, 3761 (1985).
  * [36] K. Jacobs, Input–output theory, in Quantum Measurement Theory and its Applications (Cambridge University Press, 2014) p. 458–474.
  * [37] H. Yuan, Y. Cao, A. Kamra, R. A. Duine, and P. Yan, Phys. Rep. 965, 1 (2022).
  * [38] Y. P. Wang, G. Q. Zhang, D. Zhang, X. Q. Luo, W. Xiong, S. P. Wang, T. F. Li, C.M. Hu, and J. Q. You, Phys. Rev. B 94, 224410 (2016).
  * [39] S. Rezende and F. de Aguiar, Proc. IEEE 78, 893 (1990).
  * [40] H. Suhl, J Phys Chem Solids 1, 209 (1957).
  * [41] F. Guo, L. M. Belova, and R. D. McMichael, Phys. Rev. B 89, 104422 (2014).
  * [42] T. Hioki, T. Araki, K. Umemura, K. Hoshi, and E. Saitoh, Appl. Phys. Lett. 121, 132402 (2022).
  * [43] H. Huebl, C. W. Zollitsch, J. Lotze, F. Hocke, M. Greifenstein, A. Marx, R. Gross, and S. T. B. Goennenwein, Phys. Rev. Lett. 111, 127003 (2013).
  * [44] X. Zhang, C. L. Zou, L. Jiang, and H. X. Tang, Phys. Rev. Lett. 113, 156401 (2014).
  * [45] X. Zhang, C. L. Zou, N. Zhu, F. Marquardt, L. Jiang, and H. X. Tang, Nat. Commun. 6, 8914 (2015).


# GraphReason: Enhancing Reasoning Capabilities of Large Language Models
through A Graph-Based Verification Approach

Lang Cao
University of Illinois Urbana-Champaign
Department of Computer Science
<EMAIL_ADDRESS>

###### Abstract

Large Language Models (LLMs) have showcased impressive reasoning capabilities,
particularly when guided by specifically designed prompts in complex reasoning
tasks such as math word problems. These models typically solve tasks using a
chain-of-thought approach, which not only bolsters their reasoning abilities
but also provides valuable insights into their problem-solving process.
However, there is still significant room for enhancing the reasoning abilities
of LLMs. Some studies suggest that the integration of an LLM output verifier
can boost reasoning accuracy without necessitating additional model training.
In this paper, we follow these studies and introduce a novel graph-based
method to further augment the reasoning capabilities of LLMs. We posit that
multiple solutions to a reasoning task, generated by an LLM, can be
represented as a reasoning graph due to the logical connections between
intermediate steps from different reasoning paths. Therefore, we propose the
Reasoning Graph Verifier (GraphReason) to analyze and verify the solutions
generated by LLMs. By evaluating these graphs, models can yield more accurate
and reliable results.Our experimental results show that our graph-based
verification method not only significantly enhances the reasoning abilities of
LLMs but also outperforms existing verifier methods in terms of improving
these models’ reasoning performance.

GraphReason: Enhancing Reasoning Capabilities of Large Language Models through
A Graph-Based Verification Approach

Lang Cao University of Illinois Urbana-Champaign Department of Computer
Science<EMAIL_ADDRESS>

##  1 Introduction

Figure 1: An example of chain-of-thought reasoning in a math word problem,
using data from the GSM8K dataset. Large language models learn from exemplars
that provide step-by-step solutions, subsequently generating their reasoning
path for the current question.

Large Language Models (LLMs) have demonstrated exceptional capabilities in a
variety of human tasks Zhao et al. (2023). Among the many abilities LLMs
possess, their reasoning capacity is of paramount importance Kojima et al.
(2023); Huang and Chang (2023). This has been substantiated by recent
progresses Wei et al. (2022); Zhou et al. (2023); Lampinen et al. (2022a).
Equipped with the ability to reason, especially in a multi-step manner, LLMs
can decompose complex problems into simpler tasks, thereby facilitating their
resolution. In everyday life, many complex tasks typically require multi-step
solutions. A prime example of a reasoning task is arithmetic reasoning, also
known as solving math word problems Zhang et al. (2019). These math word
problems represent simplified versions of complex real-life situations.

The reasoning ability is inherent in Large Language Models (LLMs), but it
necessitates specific methods for manifestation. To activate the robust
reasoning capability of LLMs, the use of specially designed prompts should be
considered. Numerous methods have been proposed to tap into this potential,
among which chain-of-thought reasoning Wei et al. (2022) and in-context
learning Lampinen et al. (2022b) are two notable approaches. Chain-of-thought
reasoning can elucidate the reasoning paths during the process. In-context
learning furnishes LLMs with exemplary cases, thereby enabling them to learn
from and simulate these examples for improved results. In the arithmetic
reasoning scenario, GPT-4 can achieve an accuracy of 92% on the GSM8K dataset
using 5-shot chain-of-thought prompts Cobbe et al. (2021a). This represents a
level of difficulty that a bright middle school student should be capable of
handling. As depicted in Figure 1, this illustrates a multi-step arithmetic
reasoning process in LLMs.

In addition to further training of LLMs and prompt design, some methods have
been proposed to enhance the reasoning capabilities of LLMs from the
perspective of output verification. The primary idea is to have LLMs generate
reasoning paths multiple times, and then design a verifier to evaluate these
paths and deliver the final results. Wang et al. (2023) introduces the concept
of self-consistency, based on the intuition that a complex reasoning problem
usually allows for multiple thought processes, all leading to a unique correct
answer. Li et al. (2023) also proposes All Roads Lead to Rome, which
introduces a step-aware verifier to analyze reasoning paths not just through
the entire path, but at every step. However, both methods treat each reasoning
path as an independent entity and do not consider the potential interrelation
and interaction between different reasoning paths. Once reasoning paths are
disassembled into steps, intermediate steps from one path may bear reasoning
relations to other reasoning paths. These methods do not perceive all LLM
outputs for a given input as a collective entity, thereby failing to analyze
the internal relations of all candidate paths in depth.

Inspired by these observations, we propose Reasoning Graph Verifier
(GraphReason) in this paper. We posit that reasoning paths of one question can
form reasoning graphs, where similar intermediate reasoning steps can be
merged into the same node. With a graph structure, we can more effectively
model and capture the reasoning logic between intermediate steps from
different reasoning paths. Specifically, we first construct a reasoning graph
based on all outputs from LLMs, and then train a verifier to learn the
relationship between the graph structure and the final answer. During the
prediction stage, we process the data in the same way as in the training
stage, and use the verifier to evaluate each reasoning graph. We then select
the reasoning graph with the highest score, using its answer as the final
answer. To the best of our knowledge, we are the first to approach reasoning
logic of LLMs from a graph perspective. We conduct extensive experiments to
demonstrate the improvements over the original LLMs, and show that our method
outperforms other verifiers.

In summary, our contributions are as follows:

  * •

We propose a graph-based verification method, GraphReason, aimed at
significantly enhancing the reasoning capabilities of large language models
without the need for additional training of LLMs.

  * •

We establish an arithmetic reasoning benchmark using three Math Word Problem
datasets to illustrate the fundamental reasoning performance of large language
models, and to provide a fair comparison of the performance of various
existing verifiers.

  * •

Our experimental results indicate that the method proposed in this paper
outperforms other enhancement methods. We also provide an extensive analysis
of the limitations and future potential of GraphReason.

##  2 Related Works

Reasoning of Fine-tuning Models has been extensively studied. It focuses on
addressing reasoning tasks using a general sequence-to-sequence approach,
enhanced by reasoning-aware pre-training or fine-tuning of language models.
Cobbe et al. (2021a) proposed training a verifier to rank solutions sampled
from fine-tuned language models. Yoran et al. (2022); Wang et al. (2022)
suggested equipping language models with reasoning abilities by generating
training examples with human-designed templates. Pi et al. (2022) proposed
injecting reasoning capabilities into language models by continually pre-
training on program execution data.

Several studies have focused on imbuing PLM with reasoning ability for
specific tasks, such as arithmetic reasoning Cobbe et al. (2021a); Miao et al.
(2020); Patel et al. (2021), commonsense reasoning Talmor et al. (2019), and
inductive reasoning Sinha et al. (2019). For instance, various strategies have
been proposed to improve language models’ performance on arithmetic reasoning
tasks, often referred to as math word problems. Xie and Sun (2019) proposed a
tree-structured decoder to generate an equation tree, while Zhang et al.
(2020) applied graph convolutional networks to extract relationships of
quantities in math problems. Li et al. (2022) used contrastive learning to
better learn patterns in math word problems. However, Valmeekam et al. (2023);
Rae et al. (2022) suggested that reasoning, particularly multi-step reasoning,
is often a weakness in language models and other NLP models.

Reasoning of Large Language Models has garnered significant attention and
demonstrated immense potential. Recent advancements in LLMs suggest that the
ability for multi-step reasoning is already embedded within these large-scale
models Kojima et al. (2023); Huang and Chang (2023), such as PaLM Chowdhery et
al. (2022), GPT-4 OpenAI (2023). Therefore, providing an adequate prompt is
sufficient to utilize this reasoning ability. For example, the prompting
method proposed by Kojima et al. (2023); Wei et al. (2022), which is based on
a chain-of-thought, could aid LLMs in generating text with arithmetic
reasoning and common factual knowledge. Following Wei et al. (2022),
experiments on current language models demonstrated that chain-of-thought
prompting could enhance the accuracy of solving math problems from 18% to 57%.
Lampinen et al. (2022b) included explanations in the in-context examples and
tested the influence of explanations by evaluating the score between explain-
then-predict and predict-then-explain. Moreover, Zhou et al. (2023) suggested
a two-stage prompting strategy, least-to-most prompting, which breaks down a
complex problem into a series of subproblems and solves them step-by-step. Li
et al. (2023) proposed sampling multiple times from diverse prompts to enhance
the variety of responses.

Figure 2: The framework of GraphReason. In the training stage, GraphReason
processes generated solutions from LLMs to construct reasoning graphs, and
then trains a verifier to judge them according to graph classification. In the
prediction stage, GraphReason evaluates candidate solutions to assign a score,
and selects the solution with the highest score as the final answer.

In addition to designing prompts, adopting additional strategies like verifier
has contributed to enhancing the performance of reasoning abilities of large
language models. For instance, Wang et al. (2023) proposes self-consistency,
which involves sampling different reasoning paths from the language model, and
then returning the most consistent final answer via majority voting. Li et al.
(2023) used a step-aware voting verifier to enhance the reasoning ability of
LLMs from two perspectives. These methods strive to augment the reasoning
abilities or yield superior reasoning results without necessitating additional
training of LLMs. Our work continues this research direction, with a specific
focus on developing a novel graph-based verifier to boost the reasoning
capabilities of LLMs.

##  3 Methodology

###  3.1 GraphReason Framework

######  Problem 1 (Reasoning to Solve Problems)

Given a set of $n$ math word problems
$\mathbf{Q}=\\{Q_{1},Q_{2},...,Q_{n}\\}$, where each $Q_{i}$ is represented by
the text description of a single math word problem, the goal of reasoning to
solve math word problems is to generate the answers
$\mathbf{A}=\\{A_{1},A_{2},...,A_{n}\\}$ for these problems. Here, each
$A_{i}$ represents the generated text of the corresponding answer. During the
process of large language models generating answers, a set of $n$ reasoning
paths for solutions $\mathbf{S}=\\{S_{1},S_{2},...,S_{n}\\}$ is also produced.
Each solution $S_{i}$ is represented as
$S_{i}=\\{Q,Step_{1},Step_{2},...,Step_{l},A\\}$, where each $Step_{i}$
denotes the intermediate steps in the step-by-step solutions.

We propose GraphReason to verify the solutions generated by LLMs in order to
improve the final answer accuracy. This method is a graph-based verification
technique that analyzes reasoning paths from generated solutions from a graph
perspective. The final answer is obtained without modifying the original LLMs,
functioning much like a plugin. As illustrated in Figure 2, there are two
steps in the training stage: Graph Construction and Graph Classification. In
the Graph Construction step, we obtain the generated solution from LLMs with
the specific designed prompt and group them according to their final answers.
We split reasoning paths by steps and then merge intermediate steps with
identical expression to form reasoning graphs. In the Graph Classification
step, we classify these reasoning graphs with the additional feature of the
sum of scores from the base verifier to train the integrated verifier model.
In the prediction stage, the candidate solutions are first generated by LLMs.
We process them in the same manner as in the training stage, then we use
trained verifier to evaluate the scores of each candidate solution. The best
solution, denoted by the highest score, is selected as the final predicted
answer. We will now provide a detailed introduction to the entire process.

###  3.2 Prompt Design

To improve the output of Language Models (LLMs) in providing solutions, it is
essential to design effective prompts. We incorporate chain-of-thought and in-
context learning to enable LLMs to generate step-by-step answers for math word
problems. The language models generate output $\mathbf{y}$ based on the input
$\mathbf{x}$ using the following equation:

$p(\mathbf{y}|C,\mathbf{x})=\prod_{t=1}^{|\mathbf{y}|}p_{LM}(\mathbf{y}_{t}|C,\mathbf{x},\mathbf{y}<t),$
(1)

where, $C$ represents the input provided to the LLMs prior to the current math
word problem’s question. $C$ is a concatenation of $k$ exemplars, denoted as:

$C=[(Q_{1},S_{1},A_{1});(Q_{2},S_{2},A_{2}),...;(Q_{k},S_{k},A_{k})],$ (2)

where, $Q_{i}$ represents the question, $S_{i}$ represents the intermediate
steps of the solution, and $A_{i}$ represents the answer. We set $k$ to five
in this study, resulting in a prompt that consists of five question-answer
pairs sampled from the training split of a math word problem dataset.
Therefore, the prompt can be denoted as:

$\mathbf{Prompt}=[C;Q],$ (3)

where $Q$ represents the question of the current math word problem.

Using a greedy decoding approach to sample one output from LLMs may not be
robust. It can lead to instability and occasional errors. To address this,
Wang et al. (2023) propose the concept of self-consistency. This approach
involves sampling different reasoning paths from the language model and then
selecting the most consistent final answer through majority voting. Instead of
using greedy decoding to sample only once and verify, they utilize sampling
decoding to sample $N_{1}$ times. We also follow the idea presented by Li et
al. (2023) in their work named All Roads Lead to Rome. This approach involves
generating $N_{2}$ diverse prompts for LLMs to produce multiple outputs. By
employing multiple sampling decodes on diverse prompts, we can obtain
generated solutions from different sources. Specifically, we obtain
$N=N_{1}\times N_{2}$ diverse reasoning paths for each question. In our main
experiments, we set $N_{1}=10$ and $N_{2}=3$. These solutions will be further
processed and verified using our designed verifier.

Figure 3: The graph constructor in GraphReason. We detail the process of
transforming ‘Generated Solutions with the Same Answer A1’ to ‘Graph 1’.

###  3.3 Reasoning Graph Construction

After generating multiple solutions for a question, it becomes necessary to
construct reasoning graphs based on the reasoning paths taken by these
solutions.

As shown in Figure 3, we begin by grouping all the generated solutions for a
particular question according to their final answer. Since these solutions
originate from the same question, their reasoning paths will share the same
starting point. Similarly, solutions with the same final answer will have the
same endpoint, as their reasoning paths converge. Therefore, a group of
generated solutions with the same final answer can form a reasoning graph with
a uniform start node (question node) and end node (answer node). We define
this division process as follows:

$\displaystyle\mathbf{S}=\\{S_{A_{1}},S_{A_{2}},...,S_{A_{n}}\\},$ (4)

where $\mathbf{S}$ represents the set of generated solutions for a question,
and $S_{A_{i}}=\\{S_{1},S_{2},...,S_{m}\\}$ is the subset of generated
solutions that all have the same final answer $A_{i}$.

Figure 4: The process of reasoning graph construction. The primary operation
here is the merging of identical intermediate steps in reasoning paths into a
single graph node.

For each subset of generated solutions $S_{A_{i}}$, we construct a reasoning
graph. This construction is motivated by the understanding that each step in
the reasoning path of a generated solution does not exist in isolation from
the other solutions. The steps from one solution’s reasoning path can impact
the steps from another solution, enhancing the overall reasoning process. We
utilize the graph structure to model and capture these relationships between
steps from different solutions. As the different reasoning paths can benefit
each other, we construct a reasoning graph to link these paths together. As
shown in Figure 4, the primary operation here is the merging of identical
intermediate nodes in reasoning paths into a single graph node. We first
compare the reasoning steps from any two solution reasoning paths. If they
have the same intermediate steps of arithmetic expression, we merge them into
the same node, and if they differ, we do not. For reasoning math word problems
here, we define reasoning steps as the current arithmetic expression without
other language text in the current reasoning step for clarity. It can help us
simplify construction of reasoning graphs in the reasoning task. The detailed
algorithm for constructing a reasoning graph is shown in Algorithm 1.

Algorithm 1 Reasoning graph construction algorithm

Input: generated solutions $S_{A_{i}}$ which have the same final answers
Output: a reasoning graph $G_{A_{i}}$

1: $node\\_num\leftarrow 0$

2: $node2id\leftarrow dict()$

3: $edges\leftarrow list()$

4: for each $reason\\_path$ in $S_{A_{i}}$ do

5: for each $step$ in $reason\\_path$ do

6: if $step$ not in $node2id.keys()$ then

7: $node2id[step]\leftarrow node\\_num$

8: $node\\_num\leftarrow node\\_num+1$

9: end if

10: end for

11: end for

12: for each $reason\\_path$ in $S_{A_{i}}$ do

13: for each $step$ in $reason\\_path$ do

14: $start\\_node\leftarrow node2id[last\\_step]$

15: $end\\_node\leftarrow node2id[step]$

16: if $(start\\_node,end\\_node)$ not in $edges$ then

17: $edges.add((start\\_node,end\\_node))$

18: end if

19: $last\\_step\leftarrow step$

20: end for

21: end for

22: $G_{A_{i}}\leftarrow graph(node2id,edges)$

The generated solutions, divided by their final answers
$\\{S_{A_{1}},S_{A_{2}},...,S_{A_{n}}\\}$, can be transformed into $n$
reasoning graphs of generated solutions
$\\{G_{A_{1}},G_{A_{2}},...,G_{A_{n}}\\}$.

Regarding the node features in the graph, we select the score from the Base
Verifier and the node degree. We believe the score from the Base Verifier
encapsulates the semantic information of solutions, and the node degree
contains information about the graph structure. The Base Verifier is trained
independently from the whole framework. It is designed to judge whether a
single reasoning path of one solution is correct, which is a binary text
classification task. After training, it can be used to verify any single
solution and assign a $score\in(0,1)$ to evaluate the likelihood of the
solution being correct, where $score=0.99$ suggests a $99\%$ probability of
the solution being correct. We use the score from the Base Verifier to better
incorporate solution semantic information because, according to our
experiments, it is challenging to model semantic information while modeling
reasoning logic information. The score of a step is the same as its solution
score. Therefore, for one step node $V$, it has many scores
$\\{score^{a},score^{b},...,score^{c}\\}$ from different solutions. The
feature of one node $V_{i}$ in the graph is then concatenated by the selected
feature, which can be represented as:

$\displaystyle\mathbf{V}=[score_{i}^{mean},score_{i}^{max},score_{i}^{min},$
(5) $\displaystyle score_{i}^{num},in\\_degree_{i}],$

where $\mathbf{V}\in\mathbb{R}^{5}$, $score_{i}^{mean}$ is the mean of all
scores of one step $V_{i}$, $score_{i}^{max}$ is the maximum score,
$score_{i}^{min}$ is the minimum score, $score_{i}^{num}$ is the number of
scores, and $in\\_degree_{i}$ is the in-degree of the step node $V_{i}$.

In this way, we can obtain multiple reasoning graphs to represent all
generated solutions from LLMs for a single math word problem question.

###  3.4 Verifier Design

Our designed verifier GraphReason,is used to evaluate the answer of a
generated solutions group, which is also represented as a reasoning graph.
This verifier has two inputs: the graph and the sum of solution scores. We
employ the Graph Isomorphism Network (GIN) Xu et al. (2019) to perform node
feature propagation, thereby encoding the information from the reasoning
graphs we obtained. The node feature is propagated and aggregated as follows:

$h_{v}^{(k)}=MLP^{(k)}\big{(}(1+\varepsilon^{(k)})\cdot
h_{v}^{(k-1)}+\sum\nolimits_{u\in N(v)}{h_{u}^{(k-1)}}\big{)},$ (6)

where $h_{v}^{(k)}$ represents the state of node $v$ after the $k^{th}$
update. $MLP^{(k)}$ refers to a multi-layer perceptron in the $k^{th}$ layer.
$N(v)$ denotes all the neighbors of node $v$ and $\varepsilon$ is a learnable
parameter. Then, we perform a sum readout to obtain the representation of the
reasoning graph:

$h_{G}=\sum_{v\in G}h_{v}^{(k)},$ (7)

where $h_{G}\in\mathbb{R}^{5}$. We set $k$ to 3, signifying the application of
three layers of GIN. Concurrently, the sum of the scores of solutions with the
same final answer, $A_{i}$, denoted as $score_{A_{i}}$, is represented as
follows:

$score_{A}=\sum_{i\in S_{A}}score_{i}.$ (8)

Then a reasoning graph can then be represented as:

$\mathbf{G}=[h_{G},score_{A}],$ (9)

where $\mathbf{G}\in\mathbb{R}^{6}$.

The target label of the graph $\mathbf{y}\in\\{0,1\\}$ indicates whether the
final answer matches the correct final answer. We compute the loss and train
the verifier model by:

$\mathcal{L}=\sum_{i=1}^{n}\mathcal{L}_{BCE}(label_{i},f(\mathbf{G}_{i})),$
(10)

where $i$ represents the number of solution subset among all $n$ subsets after
grouping solutions. The corresponding reasoning graph for this subset is
denoted by $\mathbf{G}_{i}$, and $f()$ is a linear classifier.

###  3.5 Answer Verification

During the prediction stage, all generated solutions are processed in the same
way as in the training stage. The trained verifier is then used to evaluate
the scores of each reasoning graph, each of which represents a group of
solutions that yield the same final answer. The final answer associated with
the highest score is selected as our final predicted answer:

$\hat{\mathbf{y}}=\mathbf{Answer}[\underset{i}{\arg\max}score_{i}],$ (11)

where $score_{i}$ denotes the score of the reasoning graph $\mathbf{G}_{i}$,
as determined by our verifier. $\mathbf{Answer}$ represents the list of all
candidate final answers. By predicting the number of the optimal reasoning
graph, we can determine the final predicted result of the current reasoning
task.

##  4 Experiments

| GSM8K | SVAMP | ASDiv-a | StrategyQA
---|---|---|---|---
Fine-tuning SOTA | $57^{a}$ | $57.4^{b}$ | $75.3^{c}$ | $73.9^{d}$
9–12 year olds | 60 | - | - | -
gpt-3.5-turbo: |  |  |  |
Greedy Decode | 72.7 | 78.7 | 93.0 | 65.0
Self-Consistency (Voting) | 82.3 | 82.9 | 95.6 | 66.0
Verifier | 66.9 | 73.1 | 92.8 | 69.3
Voting Verifier | 85.4 | 84.8 | 96.9 | 70.7
DIVERSE (Step-aware Voting Verifier) | 85.0 | 85.1 | 96.8 | 66.9
Reasoning Graph Verifier (Ours) | 85.7 | 85.4 | 97.0 | 71.2
Table 1: The comparison experiment results of GraphReason, other verifiers,
and other baselines. We primarily compare GraphReason with other verifiers
which are all based on the same generated solutions from gpt-3.5-turbo

In this section, we conducted extensive experiments to demonstrate the
performance of GraphReason, along with a more in-depth analysis. Universally,
we reproduced all types of verifiers to report their results based on the same
generated solutions. Our experiments are conducted in two settings: Arithmetic
Reasoning and Commonsense Reasoning. We ensured a fair comparison by setting
the same random seed, using the same hardware environment, and applying
similar hyperparameters. We used accuracy as the metric to evaluate the
ability of solving math word problems, which determines whether the final
answer is correct or not.

###  4.1 Training Details

For LLMs sampling, we use gpt-3.5-turbo as our base LLMs and set the
temperature $t$ to 1. All verifiers use the same LLMs’ output. Regarding
verifier training, we fine-tune on bert-base-uncased Devlin et al. (2019). We
employ the AdamW optimizer Loshchilov and Hutter (2019) to optimize the model
parameters during training. We apply differential learning rates, setting the
learning rate of the final linear classifier to 4e-2, while the other graph
neural network layers are set to 4e-3. The activation layer between them is
ReLU Agarap (2019). The batch size in each training step is set to 2. The
batch size is small because the verifier needs to verify multiple reasoning
graphs for a single question.

To ensure a fair comparison between the Voting Verifier, Simple Verifier, and
GraphReason, we use the same trained base verifier for all three approaches.

The details of the datasets and baselines are provided in Appendix A and
Appendix B, respectively.

###  4.2 Main Results

| GSM8K | $\bigtriangledown$ | SVAMP | $\bigtriangledown$ | ASDiv-a | $\bigtriangledown$
---|---|---|---|---|---|---
Reasoning Graph Verifier (Ours) | 85.7 | - | 85.4 | - | 97.0 | -
w/o solution semantic from base verifier | 81.2 | -4.5 | 83.1 | -2.3 | 94.3 | -2.7
w/o solution scores sum | 82.8 | -2.9 | 83.2 | -2.2 | 95.6 | -1.4
w/o reasoning graphs | 85.4 | -0.3 | 84.8 | -0.6 | 96.9 | -0.1
Table 2: The ablation experiment results of GraphReason. Missing each
component leads to a decline in the final result.

We present the main results in Table 1. As can be seen from the table,
GraphReason significantly enhances the original gpt-3.5-turbo’s reasoning
abilities across all three datasets, for instance, improving accuracy by 13.0%
(72.7% $\rightarrow$ 85.7%) on GSM8K. It is also evident that our method
surpasses other verifier methods with the same output from LLMs and achieves
the state-of-the-art on all three datasets.

Additionally, the Step-aware Voting Verifier improves upon the Voting Verifier
by recognizing that not all steps in an incorrect reasoning path are equally
erroneous, and some steps may still be useful for reasoning. We believe this
hypothesis is overly simplistic and cannot describe complex logical
relationships among steps. According to Table 1, it leads to some metric
decline, and the same finding also observed in the original paper.
Furthermore, it does not perform well in the StrategyQA task, because there
are no gold reasoning paths for the training of this commonsense reasoning
task. In this task, the reasoning paths are generated and pseudo, indicating a
requirement for gold labels at each step of the reasoning process. However,
our paper consistently improves upon the Voting Verifier by considering
complex relationship between different reasoning paths through reasoning
graphs. We enhance the previous method, which did not consider relations in
steps between different solutions, by 0.3% (85.4% $\rightarrow$ 85.7%), 0.3%
(85.1% $\rightarrow$ 85.4%), 0.1% (96.9% $\rightarrow$ 97.0%), and 0.5% (70.7%
$\rightarrow$ 71.2%) across the four datasets.

Moreover, GraphReason yields only a slight improvement in performance on
ASDiv-a, and the results are nearly identical. One reason for this is that the
math word problems from ASDiv-a are simpler compared to those in the other two
datasets, based on our observations. In most cases, these problems do not
require complex reasoning from a graph perspective to generate a satisfactory
answer. It demonstrates that our method is particularly well-suited for such
situations. We believe that GraphReason can offer more substantial
improvements in the more complex scenario.

###  4.3 Ablation Study

We conducted an ablation study to evaluate the impact of each component on the
overall performance of our method. Table 2 presents the results of this study,
highlighting how these modules contribute to the improvement of the base model
in distinct ways. It can be observed that the omission of any component leads
to a decline in the final result. The solution semantics from the base
verifier appear to be most crucial to GraphReason. The current method still
relies on semantic information, which is reasonable since reasoning steps from
different solutions require semantic information for better reasoning. We also
notice that reasoning graphs bring a slight improvement to the entire method,
thereby proving effectiveness of graph structure. The improvement is not
substantial because we do not model the graph structure and semantic
information simultaneously, and create a training gap here. Another essential
factor is the complexity of graph classification, compounded by the presence
of noise and limitations in our training data.

###  4.4 GraphReason with Different LLMs

| gpt-3.5-turbo | gpt-4 | PaLM-2
---|---|---|---
Greedy Decode | 72.7 | 87.0 | 53.0
Voting | 82.3 | 94.0 | 71.0
Simple Verifier | 66.9 | 89.0 | 36.0
Voting Verifier | 85.4 | 97.0 | 77.0
DIVERSE | 85.0 | 97.0 | 75.0
Ours | 85.7 | 94.0 | 78.0

Table 3: The experimental results of GraphReason with different LLMs.

To evaluate the compatibility of GraphReason and its effectiveness across
various models, we additionally include gpt-4 OpenAI (2023) and PaLM-2 Google
(2023) in our experiments. Given our limited computing resources, we utilize
the same training data previously sampled from gpt-3.5-turbo. For testing in
the GSM8K task, we select samples from 100 pieces of data from gpt-4 and
PaLM-2 respectively. We conduct the sampling 10 times using three types of
five exemplars, maintaining the same settings as in our previous experiments.
Our method aims to enhance the original reasoning capabilities. Therefore, we
do not include small-sized LMs, which typically exhibit weaker reasoning
abilities.

From Table 3, it is evident that our method enhances the original reasoning
performance of both GPT-4 and PaLM-2. However, there is a performance decline
in gpt-4 when compared with the best baselines. The performance of GraphReason
is comparable to that of the voting method. We hypothesize that this is
because the reasoning patterns of GPT-4 differ from those of GPT-3.5-Turbo,
and our verifier is trained specifically on GPT-3.5-Turbo samples in this
setting.

##  5 Conclusion

In this paper, we propose GraphReason, a novel and general method to enhance
the reasoning abilities of large language models. Our method is the first to
approach reasoning logic of large language models from a graph perspective and
verifies candidate reasoning paths accordingly. We demonstrate the superiority
of GraphReason through extensive experiments.

## Limitations

There are several limitations in the current research that contribute to
performance that is not as good as expected:

  * •

Computing Resources. Despite the impressive performance it achieves, our
framework requires large language models like GPT3.5. Inference with these
models is more time-consuming and costly than fine-tuning models like
BERTDevlin et al. (2019). Some experiments, such as hyperparameter analysis,
have already been conducted in related previous work and are not replicated
here. Furthermore, due to limited computing resources, we have not conducted
experiments with additional LLMs. We have chosen solely to use the
representative LLM, GPT3.5, to compare the performance of the verifiers.

  * •

Labeled CoT data. GraphReason is a complex verifier method that builds on
graph classification, which requires more labeled data with well-annotated
chain-of-thought reasoning paths for training. In the training of GraphReason,
we use reasoning paths from LLMs’ output which may introduce significant
noise. If the training data included labeled reasoning graphs, the performance
would improve significantly.

  * •

Other Reasoning Tasks. There are many types of reasoning tasks beyond math
word problems, such as Commonsense Reasoning Talmor et al. (2019), Inductive
Reasoning Sinha et al. (2019), etc. Given that graph construction is a complex
process, we have focused mainly on solving math word problems (Arithmetic
Reasoning). This focus allows for a more convenient implementation of the
merging of intermediate steps. In other cases, identifying similar steps can
be challenging. On the other hand, a math word problem typically presents a
greater variety of potential solutions.

Nevertheless, we believe that future studies, conducted by us or others, can
overcome these limitations and further improve upon our approach.

## References

  * Agarap (2019) Abien Fred Agarap. 2019.  Deep learning using rectified linear units (relu).  _Preprint_ , arXiv:1803.08375.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022.  Palm: Scaling language modeling with pathways.  _Preprint_ , arXiv:2204.02311.
  * Cobbe et al. (2021a) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021a.  Training verifiers to solve math word problems.  _arXiv preprint arXiv:2110.14168_.
  * Cobbe et al. (2021b) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021b.  Training verifiers to solve math word problems.  _Preprint_ , arXiv:2110.14168.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _Preprint_ , arXiv:1810.04805.
  * Geva et al. (2021) Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021.  Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies.  _Transactions of the Association for Computational Linguistics (TACL)_.
  * Google (2023) Google. 2023.  Palm 2 technical report.  _Preprint_ , arXiv:2305.10403.
  * Huang and Chang (2023) Jie Huang and Kevin Chen-Chuan Chang. 2023.  Towards reasoning in large language models: A survey.  _Preprint_ , arXiv:2212.10403.
  * Kojima et al. (2023) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2023.  Large language models are zero-shot reasoners.  _Preprint_ , arXiv:2205.11916.
  * Lampinen et al. (2022a) Andrew Lampinen, Ishita Dasgupta, Stephanie Chan, Kory Mathewson, Mh Tessler, Antonia Creswell, James McClelland, Jane Wang, and Felix Hill. 2022a.  Can language models learn from explanations in context? In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 537–563, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Lampinen et al. (2022b) Andrew K Lampinen, Ishita Dasgupta, Stephanie CY Chan, Kory Matthewson, Michael Henry Tessler, Antonia Creswell, James L McClelland, Jane X Wang, and Felix Hill. 2022b.  Can language models learn from explanations in context?  _arXiv preprint arXiv:2204.02329_.
  * Li et al. (2023) Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2023.  Making language models better reasoners with step-aware verifier.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 5315–5333, Toronto, Canada. Association for Computational Linguistics.
  * Li et al. (2022) Zhongli Li, Wenxuan Zhang, Chao Yan, Qingyu Zhou, Chao Li, Hongzhi Liu, and Yunbo Cao. 2022.  Seeking patterns, not just memorizing procedures: Contrastive learning for solving math word problems.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2486–2496, Dublin, Ireland. Association for Computational Linguistics.
  * Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019.  Decoupled weight decay regularization.  _Preprint_ , arXiv:1711.05101.
  * Miao et al. (2020) Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020.  A diverse corpus for evaluating and developing english math word problem solvers.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 975–984.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report.  _Preprint_ , arXiv:2303.08774.
  * Patel et al. (2021) Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021.  Are NLP models really able to solve simple math word problems? In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2080–2094, Online. Association for Computational Linguistics.
  * Pi et al. (2022) Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Qiang Fu, Yan Gao, Jian-Guang Lou, and Weizhu Chen. 2022.  Reasoning like program executors.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 761–779, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Rae et al. (2022) Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens, Xiang Lorraine Li, Adhiguna Kuncoro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Mensch, Jean-Baptiste Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d’Autume, Yujia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Aurelia Guy, Chris Jones, James Bradbury, Matthew Johnson, Blake Hechtman, Laura Weidinger, Iason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. 2022.  Scaling language models: Methods, analysis & insights from training gopher.  _Preprint_ , arXiv:2112.11446.
  * Sinha et al. (2019) Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, and William L. Hamilton. 2019.  CLUTRR: A diagnostic benchmark for inductive reasoning from text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4506–4515, Hong Kong, China. Association for Computational Linguistics.
  * Talmor et al. (2019) Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019.  CommonsenseQA: A question answering challenge targeting commonsense knowledge.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4149–4158, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Valmeekam et al. (2023) Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2023.  Large language models still can’t plan (a benchmark for llms on planning and reasoning about change).  _Preprint_ , arXiv:2206.10498.
  * Wang et al. (2022) Siyuan Wang, Wanjun Zhong, Duyu Tang, Zhongyu Wei, Zhihao Fan, Daxin Jiang, Ming Zhou, and Nan Duan. 2022.  Logic-driven context extension and data augmentation for logical reasoning of text.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 1619–1629, Dublin, Ireland. Association for Computational Linguistics.
  * Wang et al. (2023) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023.  Self-consistency improves chain of thought reasoning in language models.  _Preprint_ , arXiv:2203.11171.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903_.
  * Xie and Sun (2019) Zhipeng Xie and Shichao Sun. 2019.  A goal-driven tree-structured neural model for math word problems.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_ , pages 5299–5305. International Joint Conferences on Artificial Intelligence Organization.
  * Xu et al. (2019) Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019.  How powerful are graph neural networks? In _International Conference on Learning Representations_.
  * Yoran et al. (2022) Ori Yoran, Alon Talmor, and Jonathan Berant. 2022.  Turning tables: Generating examples from semi-structured tables for endowing language models with reasoning skills.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6016–6031, Dublin, Ireland. Association for Computational Linguistics.
  * Zhang et al. (2019) Dongxiang Zhang, Lei Wang, Luming Zhang, Bing Tian Dai, and Heng Tao Shen. 2019.  The gap of semantic parsing: A survey on automatic math word problem solvers.  _Preprint_ , arXiv:1808.07290.
  * Zhang et al. (2020) Jipeng Zhang, Lei Wang, Roy Ka-Wei Lee, Yi Bin, Yan Wang, Jie Shao, and Ee-Peng Lim. 2020.  Graph-to-tree learning for solving math word problems.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3928–3937, Online. Association for Computational Linguistics.
  * Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023.  A survey of large language models.  _Preprint_ , arXiv:2303.18223.
  * Zhou et al. (2023) Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023.  Least-to-most prompting enables complex reasoning in large language models.  _Preprint_ , arXiv:2205.10625.

##  Appendix A Datasets

We compared GraphReason with other methods on three different math word
problem datasets: GSM8K Cobbe et al. (2021a), SVAMP Patel et al. (2021), and
ASDiv-a Miao et al. (2020) and one commonsense reasoning dataset: StrategyQA
Geva et al. (2021). We selected the subset ASDiv-a (arithmetic) from the
original dataset ASDiv, which only involves arithmetic operations.

These three arithmetic reasoning datasets are more challenging than other math
word problem datasets, making them more suitable for testing the reasoning
capability of LLMs with a verifier. As the GSM8K dataset is the only one
providing step-by-step solutions as chain-of-thought exemplars, we chose
exemplars from the GSM8K training dataset and tested them on all three
datasets. Additionally, the training data for the verifier also used the GSM8K
training data. In this setting, we could also demonstrate the transfer
learning and generalization ability of our method. The size of the training
split from GSM8k is 1000. The test data sizes for GSM8K, SVAMP, and ASDiv-a
are 1319, 1000, and 1218, respectively.

In the StrategyQA commonsense reasoning task, we set the number of exemplars
to 8 and select pseudo-exemplars from Li et al. (2023). Additionally, we
conduct five sampling iterations for each context of LLMs. From the entire
dataset, we select a subset of 1,000 instances, allocating 700 for training
and 300 for testing.

##  Appendix B Baselines

In our evaluation, we consider the following baselines:

  * •

Greedy Decode is a simple method that uses a greedy decoding strategy to
sample once.

  * •

Self-Consistency (Voting) Wang et al. (2023) samples multiple times and
selects the final answers based on majority voting.

  * •

Simple Verifier Cobbe et al. (2021b), which is also known as the Sampling and
Re-ranking strategy, uses a verifier to assign scores to sampled solutions and
selects the final answer with the highest score.

  * •

Voting Verifier Li et al. (2023) combines the Voting and Verifier approaches.
It assigns total scores to answers from scores of all candidate solutions and
selects the final answer with the highest score.

  * •

DIVERSE (Step-aware Voting Verifier) Li et al. (2023), which is the state-of-
the-art method, considers the reasoning steps throughout the entire reasoning
path. It recognizes that not all steps in an incorrect reasoning path are
equally wrong and that some steps may still be useful for reasoning.

We primarily compare GraphReason with other verifiers using the same generated
solutions from gpt-3.5-turbo. Additionally, we include some previous Fine-
tuning state-of-the-art methods to reflect the strong reasoning ability of
LLMs. The previous Fine-tuning SOTA methods are denoted as follows: a: Cobbe
et al. (2021b), b: Pi et al. (2022), c: Miao et al. (2020), d: Chowdhery et
al. (2022).


# Is autoencoder truly applicable for 3D CT Super-Resolution?

###### Abstract

Featured by a bottleneck structure, autoencoder (AE) and its variants have
been largely applied in various medical image analysis tasks, such as
segmentation, reconstruction and de-noising. Despite of their promising
performances in aforementioned tasks, in this paper, we claim that AE models
are not applicable to single image super-resolution (SISR) for 3D CT data. Our
hypothesis is that the bottleneck architecture that resizes feature maps in AE
models degrades the details of input images, thus can sabotage the performance
of super-resolution. Although U-Net proposed skip connections that merge
information from different levels, we claim that the degrading impact of
feature resizing operations could hardly be removed by skip connections. By
conducting large-scale ablation experiments and comparing the performance
between models with and without the bottleneck design on a public CT lung
dataset , we have discovered that AE models, including U-Net, have failed to
achieve a compatible SISR result ($p<0.05$ by Student’s t-test) compared to
the baseline model. Our work is the first comparative study investigating the
suitability of AE architecture for 3D CT SISR tasks and brings a rationale for
researchers to re-think the choice of model architectures especially for 3D CT
SISR tasks. The full implementation and trained models can be found at:
https://github.com/Roldbach/Autoencoder-3D-CT-SISR

Index Terms—  Autoencoder, super-resolution, CT

Fig. 1: Architectures of different models used in this study. (a) Plain CNN;
(b) AE models; (c) U-Net.

##  1 Introduction

High resolution (HR) volumetric data generated by Computed Tomography (CT) can
capture small structures and provide detailed textural information about human
anatomy and pathology, thus facilitate the diagnostic procedure . However, the
acquisition of HRCT data requires exposure to high-dose radiation, which can
bring potential health risks to patients. More importantly, HRCT data are
often downsampled by increasing the slice interval to reduce the intrinsically
high storage requirement. Unfortunately, the downsampled data are less likely
to be re-used in subsequent image analysis that requires high-quality input.

To address these dilemmas, single image super-resolution (SISR) has attracted
increasing attention as it requires only one low resolution (LR) instance to
reconstruct the HR counterpart without affecting the raw data acquisition.
Compared with 2D SISR, 3D SISR is considerably more challenging. First, the
size of 3D volumetric data easily leads to memory bottlenecks and prolonged
training time. Moreover, 3D data contain vastly more contextual and structural
details that impose additional difficulties for 3D SISR model training.
Finally, the use of 3D convolutional layers inevitably necessitates a much
higher number of parameters than in 2D, so the size of the model must be very
carefully considered.

Among 3D super resolution models, a popular memory efficient solution is the
utilization of the autoencoder (AE) architecture [1], where feature maps could
be substantially downsampled to a great extent in the middle of the model. By
downsampling feature maps, the number of parameters and the time required for
optimization are largely reduced. Typically, common downsampling layers
include pooling [2], stride convolution [3] and interpolation [4]. Besides of
its applications on SR tasks, the AE structure has also been successfully
applied in other tasks such as segmentation [5] and detection [6].

In this paper, however, we show that models utilizing AE, including U-Net,
have substantial limitations for 3D CT SISR. Specifically, we compare various
AE models with the baseline model, and provide statistically significant
evidence to show that AE structures cause an unrecoverable loss of information
during the data processing and potentially increases the training difficulty.
We also demonstrated that skip connections and the feature map concatenations
in U-Nets may mitigate the negative effect caused by the feature map down
sampling, but they can not fully compensate for the information loss.

##  2 Methods

In this section, we describe the implementation of each of the models used in
the experiments. We first build the baseline model, “Plain CNN”, upon the
simplest backbone to avoid any possible benefits brought by the architecture
itself. All AE models are then constructed by introducing resizing layers into
Plain CNN to ensure the feature resizing operation is the only control
variable in the comparison experiments. Moreover, we adapt the U-Net,
primarily to investigate the effect of skip connections on the performance of
AE models.

###  2.1 Plain CNN

Fig. 1(a) illustrates the structure of Plain CNN, which consists of 12 basic
building blocks connected in series. Each block consists of a standard 3D
convolutional layer with 64 filters of size $3\times 3\times 3$ and a Leaky
Rectified Linear Unit (Leaky ReLU) with slope of 0.1 as the nonlinear
activation function. To avoid resizing effects, we keep the same dimension for
all feature maps by setting the stride to 1 for every convolutional layer and
adding zero padding before convolution. We abandon the use of Batch
Normalisation (BN) layer within the model. According to [7], BN not only
occupies too much memory but also discards valuable feature range flexibility
in SR. Finally, we apply the global residual learning as suggested by [8] to
ease training and prevent the gradient vanishing problem.

###  2.2 Autoencoder

Given the baseline model, we insert a downsampling layer after every 2
building blocks and its corresponding upsampling layer symmetrically, turning
the model into the AE architecture shown in Fig. 1(b). Each of these layers
resizes the feature map in all dimensions by a factor of 2 and the channel
number is adjusted to compensate for this effect. We consider 2 options for
the downsampling layer: 1) max-pooling with a filter of size $2\times 2\times
2$, stride of 2 and dilation of 1; and 2) 3D convolutional layer with a filter
of size $3\times 3\times 3$, setting stride to 2 and padding to 1. To prevent
any checkerboard artifacts generated by the transpose convolution [9] during
the upsampling operation, we use trilinear interpolation to resize feature
maps followed by the standard convolution.

###  2.3 U-Net

Our implementation of the U-Net model is based on [10], which is shown in Fig.
1(c). We replace all 2D convolutional layers and pooling layers with their
corresponding 3D versions without changing their configurations. For the
reasons mentioned above, BN layers are not used and all transpose convolution
operations are substituted with “trilinear interpolation + convolution” in the
decoder pathway. To ensure a relatively fair comparison with AE models, we
simplify the U-Net to have a similar model size and depth by: 1) setting the
channel number in the convolutional layer at the first level to 32 instead of
64; 2) reducing the level of hierarchical feature maps from 5 to 4.

Scale | Methods | PSNR | SSIM | RMSE
---|---|---|---|---
$\times 2$ | AE-Maxpool | 35.14 (2.50)* | 0.9649 (0.0062)* | 4.64 (1.20)*
AE-Conv | 35.38 (2.53)* | 0.9641 (0.0072)* | 4.52 (1.21)*
U-Net | 39.40 (1.23)* | 0.9795 (0.0040)* | 2.76 (0.40)*
Plain CNN |  43.52 (0.95) * |  0.9839 (0.0049) * |  1.71 (0.19) *
$\times 4$ | AE-Maxpool | 26.89 (2.51)* | 0.8711 (0.0213)* | 11.99 (3.12)*
AE-Conv | 25.49 (1.29)* | 0.8741 (0.0207)* | 13.71 (2.1)*
U-Net | 29.31 (1.73)* | 0.9131 (0.0112)* | 8.92 (1.94)*
Plain CNN |  34.51 (0.65) * |  0.9345 (0.0130) * |  4.81 (0.35) *
$\times 8$ | AE-Maxpool | 24.20 (1.86)* | 0.7890 (0.0320)* | 16.08 (3.22)*
AE-Conv | 23.34 (1.59)* | 0.7838 (0.0306)* | 17.63 (2.88)*
U-Net | 30.24 (1.20)* | 0.8663 (0.0220) * | 7.92 (1.19)*
Plain CNN |  31.03 (1.25) * | 0.8644 (0.0226) * |  7.23 (0.96) *
|  |  |  |
$\times 2$ | AE-Maxpool | 33.66 (2.16)* | 0.9539 (0.0094)* | 5.44 (1.14)*
AE-Conv | 32.25 (3.23)* | 0.9468 (0.0108)* | 6.63 (2.13)*
U-Net | 37.52 (1.43)* | 0.9770 (0.0035)* | 3.44 (0.63)*
Plain CNN |  43.16 (1.02) * |  0.9837 (0.0048) * |  1.79 (0.21) *
$\times 4$ | AE-Maxpool | 23.52 (1.06)* | 0.8434 (0.0260)* | 17.12 (1.96)*
AE-Conv | 25.70 (2.54)* | 0.8420 (0.0290)* | 13.77 (3.57)*
U-Net | 33.45 (0.97)* | 0.9287 (0.0152)* | 5.45 (0.61)*
Plain CNN |  36.54 (0.69) * |  0.9422 (0.0134) * |  3.81 (0.30) *
$\times 8$ | AE-Maxpool | 22.36 (1.48)* | 0.7562 (0.0302)* | 19.70 (2.98)*
AE-Conv | 20.86 (1.42)* | 0.7400 (0.0368)* | 23.38 (3.55)*
U-net | 27.21 (2.14)* | 0.8538 (0.0198)* | 11.50 (3.36)*
Plain CNN |  31.35 (1.34) * |  0.8757 (0.0197) * |  6.98 (0.99) *

Table 1: Quantitative comparisons of different models. Best results are shown in Bold. * indicates statistically significant evidence to support the difference with Plain CNN. Top: Mean (STD) using trilinear interpolation in LR generation; Bottom: Mean (STD) using same insertion in LR generation. Methods | #Parameter (M) | Inference Time (s)
---|---|---
AE-Maxpool | 6.88 | 18.46
AE-Conv | 7.44 | 18.62
U-Net | 5.30 | 10.04
Plain CNN | 1.11 | 33.87
Table 2: Computational comparisons of different models.

##  3 Experiments

###  3.1 Dataset

We use the AAPM-Mayo Clinic Low-Dose CT Grand Challenge Dataset
111https://www.aapm.org/grandchallenge/lowdosect/ provided by Mayo Clinic for
model training and testing. From the CT scans collected from 140 patients, we
use 48 chest CT scans and further split these into 37 training, 5 validation
and 6 test volumes. The data for each patient consists of normal-dose CT
(NDCT) scans and the corresponding synthetic low-dose CT (LDCT) scans with
additional Poisson noise. Only NDCT are used as ground truth HR data in the
experiments. All volumes within the dataset contain an uneven number of slices
of size $512\times 512$ with 1.5mm thickness. To enable a feasible training
time and solve the memory limitation, we pre-downsampled each slice from
$512\times 512$ to $256\times 256$ using bilinear interpolation.

###  3.2 LR Data Generation

Our LR data are degraded from ground truth HR data by the following steps: 1)
Truncate the leading and trailing slices evenly so that the dimension of each
volume can perfectly fit the non-overlapping patch extraction algorithm; 2)
Downsample the volume in the axial direction by removing slices at a constant
interval; 3) Clip all HU values into the range [-1024, 1476] and normalize
them to [0,1]; and 4) Upsample the volume to its original dimension either by
trilinear interpolation or by inserting the same slice at the previous
position.

###  3.3 Evaluation Metrics

The performance of each model is evaluated by using: Peak Signal-To-Noise-
Ratio (PSNR); Structural Similarity Index (SSIM); and the Root Mean Square
Error (RMSE). Both PSNR and RMSE focus on the pixel-level error between the
reconstructed volume and the ground truth label while SSIM is more suitable
for reflecting the structural correspondences.

Fig. 2: Visual comparisons of different models using same insertion in LR
generation under $\times 2$ scaling factor.

###  3.4 Statistical Test

We use Shapiro-Wilk test to check the normality of the difference in the image
quality between each AE model and Plain CNN and paired Student’s t-test to
determine whether there is statistically significant evidence to support this
difference. We use Wilcoxon signed-rank test instead when the normality of the
sample cannot be satisfied. We set the significance level to 0.05 in all
statistical tests.

##  4 Results and Discussion

In Table 1, we compare the quantitative performance of all AE models with
Plain CNN under $\times 2$, $\times 4$, and $\times 8$ scaling factors. Next,
we present the total number of parameters and the average inference time used
by one volume for every model in Table 2. Finally, We show visual comparisons
of different models in Fig. 2.

The results in Table 1 show that AE is unsuitable for 3D CT SISR. It can be
seen that there is an obvious performance drop for all AE models compared with
Plain CNN in almost all cases. At the same time, there also exists
statistically significant evidence to support this performance drop in almost
every comparison experiment. We contend that the main reason for the
performance drop is the resizing operation within AE since this is the only
architectural difference between AE models and the baseline model.
Surprisingly, those results also reveals the fact that skip connections, which
are designed to benefit U-Net by increasing the high-resolution feature re-
usability, cannot fully compensate for the aforementioned performance gap.
From Table 2, AE models have a lower computational cost than Plain CNN
reflected by the reduced average inference time, but at the cost of a larger
model size.

We show comparisons between AE models and Plain CNN visually in Fig. 2. These
results again argue that AE, including U-Net, is not suitable basis for 3D
SISR tasks. It is clearly shown that AE models generate noticeable artifacts
around edges and significant deviations in regions with abundant textures.
This shows the loss of diagnostically important information due to resizing.
In contrast, Plain CNN can recover comparatively more natural textures and
smoothing edges and produce results that are almost indistinguishable from the
ground truth HR data.

##  5 Conclusion

In this paper, we have shown that AE models, including U-Net, are unsuitable
for 3D CT SISR primarily due to the information loss in feature resizing
operations. We have presented a carefully designed set of experiments,
adjusting model architectures for a fair comparison. We have evaluated the
models on a publicly available CT lung dataset and have concluded that
although AE models can achieve faster inference, they do so at the cost of
inferior performance compared to the baseline CNN.

In future work we hope to explore whether the use of other loss functions such
as Mean Absolute Error (MAE) and perceptual loss could compensate for the
aforementioned information loss. We also plan to investigate the performance
of AE-based generative adversarial networks (GAN).

##  6 Compliance with ethical standards

This research study was conducted retrospectively using human subject data
made available in open access by 2016 Low-dose CT AAPM Grand Challenge.
Ethical approval was not required as confirmed by the license attached with
the open access data.

##  7 acknowledgments

This study was supported in part by the ERC IMI (101005122), the H2020
(952172), the MRC (MC/PC/21013), the Royal Society (IEC/NSFC/211235), the
Imperial College Undergraduate Research Opportunities Programme (UROP), the
NVIDIA Academic Hardware Grant Program, the SABER project supported by
Boehringer Ingelheim Ltd, NIHR Imperial Biomedical Research Centre (RDA01),
and the UKRI Future Leaders Fellowship (MR/V023799/1).

## References

  * [1] Y. Zhou, Y. Zhang, and X. Xie,  “Image super-resolution based on dense convolutional auto-encoder blocks,”  Neurocomputing, vol. 423, January 2021.
  * [2] S. Nasrin, M.Z. Alom, and R. Burada,  “Medical image denoising with recurrent residual u-net (r2u-net) base auto-encoder,”  in 2019 IEEE National Aerospace and Electronics Conference (NAECON). IEEE, 2019, pp. 345–350.
  * [3] J. Gurrola-Ramos, O. Dalmau, and T.E. Alarcón,  “A residual dense u-net neural network for image denoising,”  IEEE Access, vol. 9, February 2021.
  * [4] A. Kudo, Y. Kitamura, and Y. Li,  “Virtual thin slice: 3d conditional gan-based super-resolution for ct slice interval,”  in Machine Learning for Medical Image Reconstruction. Springer International Publishing, 2019, pp. 91–100.
  * [5] S. Chen, G. Bortsova, and A. García-Uceda Juárez,  “Multi-task attention-based semi-supervised learning for medical image segmentation,”  in Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Springer International Publishing, 2019, pp. 457–465.
  * [6] H. Uzunova, S. Schultz, and H. Handels,  “Unsupervised pathology detection in medical images using conditional variational autoencoders,”  International Journal of Computer Assisted Radiology and Surgery, vol. 14, March 2019.
  * [7] B. Lim, S. Son, and H. Kim,  “Enhanced deep residual networks for single image super-resolution,”  in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2017, pp. 1132–1140.
  * [8] K. He, X. Zhang, and S. Ren,  “Deep residual learning for image recognition,”  in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016, pp. 770–778.
  * [9] A. Odena, V. Dumoulin, and C. Olah,  “Deconvolution and checkerboard artifacts,”  Distill, 2016.
  * [10] O. Ronneberger, P. Fischer, and T. Brox,  “U-net: Convolutional networks for biomedical image segmentation,”  in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Springer International Publishing, 2015, pp. 234–241.