# Efficiency of Nonthermal Particle Acceleration in Magnetic Reconnection

Masahiro Hoshino  Department of Earth and Planetary Science, The University of
Tokyo, Tokyo 113-0033, Japan<EMAIL_ADDRESS>
###### Abstract

The nonthermal particle acceleration during magnetic reconnection remains a
fundamental topic in several astrophysical phenomena, such as solar flares,
pulsar wind, magnetars, etc, for more than half a century, and one of the
unresolved questions is its efficiency. Recently, nonthermal particle
acceleration mechanisms during reconnection have been extensively studied by
particle-in-cell simulations, yet it is an intriguing enigma as to how the
magnetic field energy is divided into thermally heated plasmas and nonthermal
particles. Here we study both non-relativistic and relativistic magnetic
reconnections using large-scale particle-in-cell simulation for a pair plasma,
and indicate that the production of the nonthermal particle becomes efficient
with increasing the plasma temperature. In the relativistic hot plasma case,
we determine that the heated plasmas by reconnection can be approximated by a
kappa distribution function with the kappa index of approximately $3$ or less
(equivalent to $2$ or less for the power-law index), and the nonthermal energy
density of reconnection is approximately over $95\%$ of the total internal
energy in the downstream exhaust.

magnetic reconnection — plasmas — particle acceleration

##  I Introduction

Recently, the rapid magnetic energy release by magnetic reconnection, as well
as the supersonic bulk flow energy release by collisionless shock waves, is
regarded as an important particle acceleration process, and reconnection has
garnered significant attention in various plasma environments such as the
solar flares, Earth’s magnetosphere, pulsar magnetospheres, and magnetars
[e.g., 1, 2, 3, 4, 5]. In these hot and rarefied magneto-active plasmas, the
magnetic energy stored in a plasma sheet with an anti-parallel magnetic field
component can be efficiently converted into not only the Alfvénic bulk flow
energy, but also the energies of plasma heating and nonthermal particle
acceleration by magnetic reconnection. While the reconnection paradigm has
been established as a powerful magnetic energy dissipation process in the
plasma universe [e.g., 6, 7, 8, 9, 10], it is recently being recognized that
the reconnection plays a much more important role than before in the
astrophysical phenomena, because it can efficiently generate the nonthermal
particles, and the nonthermal energy spectra observed in various astrophysical
objects often indicate a hard power-law spectrum, which cannot be easily
explained by the conventional acceleration processes such as the diffusive
Fermi shock acceleration [e.g. 11, 12].

From the satellite observations of the solar flares and substorms in the
Earth’s magnetotail, it is recognized that not only the plasma heating, but
also the nonthermal particle acceleration is an ubiquitous process of
reconnection, and it is suggested that the reconnection process with a
relativistic hot gas temperature such as gamma-ray flares in the Crab nebula
can generate nonthermal particles as well. In fact, by employing particle-in-
cell (PIC) simulation studies, it is discussed that a hard non-thermal energy
spectrum $N(\varepsilon)\propto\varepsilon^{-s}$ with its power-law index
$s\leq 2$ can be formed for a relativistic reconnection using 2-dimensional
PIC simulations [e.g. 13, 14]. It is revealed in three-dimensional PIC
simulations [e.g. 15, 16, 18, 17] that such a hard power-spectrum can be
formed for a large magnetization parameter $\sigma=B^{2}/(4\pi\rho c^{2})$,
whereas the power-law slope becomes softer as the magnetization parameter
$\sigma$ decreases, where $B$ and $\rho$ are the magnetic field and mass
density, respectively.

While the magnetic reconnection is understood as an efficient particle
accelerator in plasma universe, one of the remaining challenges is to
quantitatively understand the energy conversion rate into nonthermal
particles. The efficiency of nonthermal particle production against the total
internal plasma energy during the magnetic energy dissipation is not
necessarily understood in a consistent way. Ball et al. [18] discussed the
nonthermal particle efficiency in trans-relativistic reconnection using PIC
simulations, and determined that the efficiency increases with increasing
magnetization parameter $\sigma$. However, it is interesting to know the
efficiency of the nonthermal production for a wider range of $\sigma$ values,
from non-relativistic cold plasmas to relativistic hot plasma reconnections.
It is also important to carefully discuss the spectral feature from the low-
energy to high-energy nonthermal range, and to understand the energy partition
between the thermal and nonthermal high energies by analyzing the reconnection
energy spectrum. The energy partition between the thermal plasma and
nonthermal particles is still an important open question.

To understand the energy partition between the thermal and nonthermal
energies, we investigate magnetic reconnection using particle-in-cell
simulations by focusing on the time and spatial evolutions of the energy
spectrum. The energy spectrum obtained by the reconnection simulation, in
general, comprises the thermal and non-thermal populations, and a model
fitting of the energy spectrum by the combination of Maxwellian and kappa
distribution functions is made as well. [e.g. 19]. We quantify the thermal
plasma and nonthermal populations as the function of the initial plasma
temperature, both in non-relativistic and relativistic regimes, and prove that
the production of the nonthermal particle becomes efficient with an increase
in plasma temperature. In the relativistic hot plasma case, we determine that
the nonthermal power-law index $s$ can be close to $2$, and the nonthermal
energy density of reconnection reaches over approximately $95\%$ of the total
heated plasma in the exhaust.

##  II Overview of Simulation Study

We study the plasma heating and particle acceleration of magnetic reconnection
using two-dimensional PIC simulation [20, 21, 22] with a periodic boundary in
the $x$-direction for $x=-215\lambda$ and $x=215\lambda$, and the conducting
walls for the upper and lower boundaries at $y=\pm 215\lambda$, where
$\lambda$ is the thickness of the initial plasma sheet. The total system size
for our fiducial runs is $L_{x}\times L_{y}=430\lambda\times 430\lambda$ and
the computational grid size is $5376\times 5376$. The total number of
particles is $8.4\times 10^{10}$, to calculate the high-$\beta$ plasma sheet
as accurately as possible.

For simplicity, we adopt the Harris solution [23] for the pair plasma with the
same mass $m$ and temperature $T$ uniform in space, and focus on the energy
partition of an idealized magnetic reconnection in a collisionless plasma
system. The magnetic field ${\bf B}=B_{x}(y){\bf e_{x}}$ and plasma density
$N(y)$ are given by

$B_{x}(y)=B_{0}\tanh(y/\lambda),$ (1)

and

$N(y)=N_{0}\cosh^{-2}(y/\lambda)+N_{\rm b},$ (2)

respectively, where $N_{\rm b}$ is the background uniform plasma density
adopted to demonstrate continuous plasma injection from the outside plasma
sheet to the reconnection exhaust. The background plasma density $N_{\rm b}$
is set to be $5\%$ of the maximum Harris plasma density $N_{0}$. Note that the
relationships of the pressure balance $B_{0}^{2}/8\pi=2N_{0}T$ and force
balance $2T/\lambda=|e|u_{d}B_{0}/c$ are satisfied, where $u_{d}$ is the drift
velocity , and the initial electric field ${\bf E}$ is zero.

We study eight different cases of plasma temperature from a cold non-
relativistic to hot relativistic temperature of

$T/mc^{2}=10^{n/2-2}\quad{\rm with}\quad n=0,1,2,..,7.$ (3)

The above temperatures correspond to the magnetization parameter
$\sigma=B_{0}^{2}/(8\pi N_{0}mc^{2})=2\times 10^{n/2-2}$, if we use the number
density for the value of the plasma sheet $N_{0}$, owing to the pressure
balance between the gas pressure inside the plasma sheet and magnetic pressure
outside the plasma sheet. Note that the Alfvén speed $v_{A}$ is given by
$v_{A}=c\sqrt{\sigma/(1+\sigma)}$ in a cold plasma limit, where the density is
used for the value of the central plasma sheet and the magnetic field is the
value of the outside plasma sheets. We keep the ratio of the gyro-radius
$r_{g}$ and thickness of the plasma sheet $\lambda$ for all simulation runs,
and set $r_{g}/\lambda=0.45$, where
$r_{g}=mc^{2}\sqrt{\gamma_{th}^{2}-1}/(eB_{0})$, because a thin plasma sheet
whose thickness is almost equal to the gyro-radius has been formed before the
explosive growth of magnetic reconnection in the pair plasma [22] and has been
observed before the onset of substorms in the Earth’s magnetotail [e.g. 24,
25]. For simplicity, we have assumed $(\gamma_{th}-1)mc^{2}/T=1$, and then the
drift speed $u_{d}$ can be calculated by
$u_{d}/c=2(r_{g}/\lambda)/\sqrt{1+2mc^{2}/T}=0.9/\sqrt{1+2\times 10^{2-n/2}}$
.

We add a small initial perturbation for the vector potential $\delta A_{z}$ to
the Harris equilibrium, which is given by

$\delta
A_{z}(x,y)\propto\exp(-(\frac{x}{\lambda_{x}})^{2}-(\frac{y}{\lambda_{y}})^{2})\cosh(\frac{y}{\lambda_{y}}),$

where $\lambda_{x}=2\lambda$, $\lambda_{y}=\lambda/2$, and the initial
amplitude of the reconnected magnetic field is set to be
$\max(B_{y}/B_{0})=10^{-2}$ in the neutral sheet.

Figure 1: Time evolution of the collisionless magnetic reconnection for a
Harris current sheet obtained by PIC simulation for $T/mc^{2}=10^{-2}$. The
top panel ($t/\tau_{A}=119$) shows the early stage of reconnection, and the
X-type neutral point is formed at the center of the simulation box. The middle
panel ($t/\tau_{A}=176$) is the nonlinear stage where several plasmoids are
coalescing and forming the larger plasmoids, and the bottom panel
($t/\tau_{A}=308$) is the almost final stage when two large plasmoids are just
coalescing into one large plasmoid in the system. The thick white lines
indicate the magnetic field lines that pass through the X-type neutral line,
i.e., the separatrix between the downstream exhaust and the upstream plasma
sheet. The color bars in the right-hand side indicate that the logarithmic
plasma density normalized the initial plasma sheet density. Figure 2: Energy
spectra for the downstream exhaust region for $T/mc^{2}=10^{-2}$ (left),
$T/mc^{2}=1/\sqrt{10}$ (middle) and $T/mc^{2}=10$ (right), respectively. The
color lines indicate the time evolution of spectra, whose time stages are
indicated in the right-hand side bar. The dark blue line is the initial state,
and the red one is the final stage when a large plasmoid is formed in the
simulation box. The thick solid lines are the model fitting for the composed
function of the Maxwellian and kappa distributions $N_{{\rm
M}+\kappa}(\gamma)$. The black dashed lines in the lower energy regime
represent the Maxwellian part of the fitting result $N_{\rm M}(\gamma)$, while
the other dashed lines in the higher energy regime are the kappa distribution
parts $N_{\kappa}(\gamma)$. The bottom panels indicate the errors of the model
fitting, which show the difference between the fitting model function and
simulation result. The dashed lines in the bottom plot are depicted as
reference of an acceptable level of error.

Illustrated in Figure 1 are three-time stages of the reconnection structure
obtained by the particle-in-cell simulation for a non-relativistic
reconnection with the temperature of $T/mc^{2}=0.01$. The plasma density and
magnetic field lines are indicated by the color contour and white lines,
respectively. The thick white lines indicate the separatrix of the magnetic
reconnection for the X-type neutral point formed around the center
$x/\lambda\sim 0$, i.e., the most recently reconnected magnetic field lines.
The downstream of the separatrix is occupied by the reconnection heated
plasmas. These lines of the separatrix can be obtained by the contour lines of
the vector potential $A_{z}$ that has the minimum value in the neutral line of
$y=0$. A part of the simulation system is illustrated as well.

The top panel is the early time stage at $t/\tau_{A}=119$ when the
reconnection just started at the center, where $\tau_{A}=\lambda/v_{A}$ is the
Alfvén transit time, and several small scale plasmoids have been formed along
$y=0$. The middle panel indicates the time stage when several plasmoids grow
rapidly and coalesce with each other. As we imposed the initial small
perturbation $\delta A_{z}$ at $x/\lambda=0$, the most prominent reconnection
occurs at the center, but in addition to that, we can observe several other
small size plasmoids embedded in the downstream domain of the main
reconnection. The bottom panel is the approximate final stage of the active
reconnection in our periodic system after the two large plasmoids have merged
into one large magnetic island.

##  III Model Fitting of Energy Spectrum

As our objective is to understand the energy partition between the thermal and
nonthermal populations during reconnection, we study the energy spectrum in
the downstream of the separatrix lines in details. Figure 2 illustrates the
time evolutions of three energy spectra for $T/mc^{2}=10^{-2}$ (left),
$1/\sqrt{10}$ (middle), and $10$ (right), integrated over the reconnection
region sandwiched by the separatrix, i.e., the thick white lines defined in
Figure 1. Note that if the width of the separatrix is smaller than the
thickness of the plasma sheet $\lambda$, the integration is conducted for the
plasma sheet with the size $\lambda$. The horizontal axis is the particle
energy $(\gamma-1)mc^{2}$, and the vertical axis is the number density
$N(\gamma)d\gamma$, where $\gamma=1/\sqrt{1-(v/c)^{2}}$. The color lines
indicate the time evolution of the energy spectra, whose time stages are
indicated in the right-hand color bars. The blueish and reddish colors
correspond to the earlier and later time stages, respectively. As we assumed a
uniform plasma temperature for both the $\cosh$-type Harris density population
and background populations in Equation (2), the dark blue color spectra in
Figure 2 illustrate approximately the initial plasma state with the initial
Maxwellian distribution functions. For the relativistic case in Figure 2c,
however, one-dimensional energy spectrum is modified by the effect of a high-
speed drift velocity with $u_{d}/c\sim 0.82$.

As time goes on, we can clearly observe those Maxwellian plasmas are gradually
heated owing to the reconnection heating process, and the high-energy
components are further accelerated and form the nonthermal population. During
the time evolution, by looking at the energy interval of the time evolution of
the energy spectra, it is determined that the energy intervals are wider in
the early acceleration stage compared with those in the later stage,
suggesting that the rapid energy gain happens in the early stage.

As the spectra with the red curve is approximately stable and at the final
stage, we discuss the spectra behavior for the red curve by making a model
function fitting in detail. It is evident that the energy spectra cannot be
simply approximated by a Maxwell distribution function given by,

$N_{M}(\gamma)\propto\gamma\sqrt{\gamma^{2}-1}\exp(-\frac{\gamma-1}{T_{M}/mc^{2}}),$
(4)

where, for simplicity, a three-dimensional Maxwell distribution function has
been projected into a one-dimensional distribution in the simulation frame as
the function of the particle energy $\gamma$.

To make a better model fitting of the observed distribution, we first tested a
kappa distribution function given by,

$N_{\kappa}(\gamma)\propto\gamma\sqrt{\gamma^{2}-1}\left(1+\frac{\gamma-1}{\kappa
T_{\kappa}/mc^{2}}\right)^{-(1+\kappa)}f_{cut}(\gamma).$ (5)

The kappa distribution comprises a thermal Maxwellian at low energies and a
nonthermal population approximated by a power-law function at high energies
[e.g. 19]. It is known that the observed solar wind distribution function can
be well modeled by a kappa ($\kappa$) distribution function [e.g 19, 26], and
the kappa modeling is widely applied for the space plasma investigation [e.g
27, 28]. In our analysis, we added a high energy cutoff function $f_{cut}$ to
represent a possible high energy cutoff in the simulation data [29, 30], which
is given by,

$f_{cut}(\gamma)=\left\\{\begin{array}[]{@{\,}ll}1&\mbox{for
$\gamma\leq\gamma_{cut}$ }\\\
\exp\left(-(\gamma-\gamma_{cut})/\gamma_{cut}\right)&\mbox{for
$\gamma>\gamma_{cut}$ }\end{array}\right.$

The high energy cutoff may come from the finite time evolution of reconnection
under the limited system size [30]. Note that the high energies of
$N_{\kappa}$ can be approximated by a power-law function of
$N_{\kappa}\propto\gamma^{-\kappa+1}$ with the power-law index $s=\kappa-1$
for $\gamma\gg T_{\kappa}/mc^{2}$.

Although the model function was not necessarily a good approximation, we
determined that this kappa model function is better than the Maxwellian
fitting. As the spectra for the red lines in Figure 2 can be observed, the
contributions of the cold plasma freshly transported from the outside plasma
sheet is not necessarily negligible. Note that the energetic particles
generated in the X-type region/magnetic diffusion region can escape along the
magnetic field line, and simultaneously, the pre-heated plasma is transported
from the outside plasma sheet by $E\times B$ drift motion in association with
the magnetic field lines. Therefore, the separatrix region contains both the
cold and hot plasmas [e.g. 31].

Having the above plasma transport, we finally determined that the best-fitted
model function $N_{M+\kappa}$ is a combination of the Maxwellian and kappa
distribution functions, i.e.,

$N_{M+\kappa}(\gamma)=N_{M}(\gamma)+N_{\kappa}(\gamma).$ (6)

In the top three panels in Figure 2, the sold lines are the model fitting
curves for $N_{M+\kappa}$ at the final time stage, while the two dashed lines
are the fitting curve of $N_{M}$ at lower energies and $N_{\kappa}$ for higher
energies, respectively. It is determined that three simulation spectra in the
downstream exhaust for $T/mc^{2}=10^{-2}$ (left), $1/\sqrt{10}$ (middle), and
$10$ (right) can be approximated by the composed model function of Maxwellian
and kappa distributions $N_{{\rm M}+\kappa}(\gamma)$. We have also confirmed
that five other simulation cases of $T/mc^{2}=10^{n/2-2}$ with $n=1,2,4,5$ and
$7$ express the similar good model fitting.

The bottom three panels indicate the error of the model fitting given by,

$\displaystyle{\rm Error}(\gamma)=|N_{data}(\gamma)-N_{M+\kappa}(\gamma)|/$
$\displaystyle{\rm min}(N_{data}(\gamma),N_{M+\kappa}(\gamma)),$ (7)

where ${\rm min()}$ means a function to return the minimum element from
$N_{data}$ and $N_{M+\kappa}$. If
$\frac{1}{2}N_{data}<N_{M+\kappa}<2N_{data}$, then Error becomes less than
$1$, as depicted by the dashed line in the bottom panel of Figure 2. We can
observe that the errors in the wide energy ranges are less than $10^{-1}$, and
the model fitting of $N_{M+\kappa}$ can well describe the energy spectra
obtained in the simulation runs. Note that a small bump seen around
$\gamma\sim 8$ for $T/mc^{2}=10^{-2}$ is the contribution from the small-scale
plasmoids behind the main large-scale plasmoid, but the total energy
contribution is not necessarily large. Note that the bumpy structure in the
energy spectrum is not a permanent structure and disappears after the small
scale plasmoid is absorbed into the large plasmoid. For the model fitting for
$T/mc^{2}=10$ for the right-hand panel in Figure 2, we can observe a large
discrepancy between the simulation data and model fitting at high energies. By
choosing another cutoff function given by
$f_{cut}(\gamma)=\exp(-((\gamma-\gamma_{cut})/\gamma_{cut})^{2})$, we obtain
the better fitting result (not indicated here).

Figure 3: The model fitting results for eight different initial plasma
temperatures of $T/mc^{2}=10^{-2}\sim 10^{3/2}$. (Left) The temperatures of
the Maxwellian part $T_{M}$ (red dots) and kappa distribution part
$T_{\kappa}$ (blue dots) as the function of the initial plasma temperature.
The kappa index $\kappa$ (middle) and cutoff energy $\gamma_{\rm cut}$ (right)
as the function of the initial plasma temperature.

We quantitatively discuss the energy partition between the thermal and
nonthermal plasmas. Figure 3 illustrates the model fitting results of (a) the
temperatures $T_{M}$ (red dots) and $T_{\kappa}$ (blue dots), (b) the $\kappa$
index with a power-law tail, and (c) the cutoff energy $\gamma_{cut}$ as the
function of the initial plasma temperature $T/mc^{2}$, respectively. We
determine that the temperature of $T_{M}$ remains approximately the same as
the initial background temperature $T$, suggesting that the exhaust region
sandwiched by the separatrix, as indicated by the thick white lines in Figure
1, still contains a part of non-heated plasmas, and that those temperatures
may represent the flux tubes just transported into the reconnection exhaust,
and are not heated strongly [31]. Note that during the transport from the
outside plasma sheet into the plasma sheet, the plasmas are slightly cooled
down owing to the adiabatic expansion of the magnetic flux tube, because the
slow mode expansion waves are emitted from the X-type reconnection region.
However, the temperatures of the kappa distribution function $T_{\kappa}$ are
higher than that of the Maxwellian function $T_{M}$. The ratio of
$T_{\kappa}/T_{M}$ is large for the non-relativistic reconnection cases for
$T/mc^{2}<1$, while those for the relativistic reconnection regime of
$T/mc^{2}>1$ become less than 2. $\kappa$ index, which is equivalent to the
power-law index $s=\kappa-1$ is illustrated in Figure 3b. For the non-
relativistic case of $T/mc^{2}=10^{-2}$, $\kappa$ indicates a large value,
suggesting that the spectrum is close to a Maxwell distribution and the
efficiency of the nonthermal particle production is very low, whereas for the
relativistic regime of $T/mc^{2}>1$, the $\kappa$ values are approximately $3$
and the nonthermal tails are well developed. Note that $\kappa=3$ can be
rephrased as the power-law index of $2$. Figure 3c is the cutoff energy
$\gamma_{cut}$. Except for the case of $T/mc^{2}=10^{-2}$, we determine that
the ratio of $\gamma_{cut}/(T_{\kappa}/mc^{2})$ is approximately $50\sim
10^{2}$, suggesting that the nonthermal tails are well extended above the
thermal population of the kappa distribution function.

It would be interesting to mention the relationship between the gyro-radius of
the accelerated particle and the size of the magnetic island. The initial
gyro-radius was set to be 0.45 $\lambda$ for all runs, and we can find in
Figure 2 that the highest energies after reconnection attained to several
$10^{2}$ \- $10^{3}$ times larger than the initial thermal energies.
Therefore, we can estimate that the gyro-radius of the highest energy
particles is about $100-200\lambda$, which is almost same as the size of the
magnetic island.

It would be better to make a short comment on the model fitting error to
estimate the fitting parameters. As can be observed in Figure 2, the spectral
profile in the final stages of the simulation have approximately the same
spectral structure in the logarithmic scale, but the fitting parameters may
have approximately 10% variations depending on the time stages.

##  IV Efficiency of Nonthermal Density and Energy Production

Figure 4: The model fitting results for the same data set as Figure 3. (Left)
The percentage in the number density of the kappa distribution part $\xi_{\rm
den}$ (red) and percentage in the energy density $\xi_{\rm ene}$ (blue).
(Right) The percentage in the number density of the nonthermal population of
the kappa distribution against the total distribution $\varepsilon_{\rm den}$
(red) and percentage in the energy density of the nonthermal population
$\varepsilon_{\rm ene}$ (blue).

Based on the above model fitting, we can study the efficiency of the
nonthermal particle acceleration in reconnection. In the previous section, we
discovered that the energy spectrum in the exhaust region can be approximated
by the combination of Maxwellian and kappa distribution functions. To quantify
the efficiency of the production of the nonthermal particles, we calculate the
percentage of the nonthermal population from the fitted model function.

As illustrated in Figure 4, the left-hand panel a is the percentage of the
number density and the energy density for the population of the $\kappa$
distribution against the entire population of the energy spectrum as the
function of the initial plasma temperature $T/mc^{2}$. The solid red and
dashed blue lines are the number and energy densities defined by,

$\displaystyle\xi_{\rm
den}=\int_{1}^{\infty}N_{\kappa}(\gamma)d\gamma/\int_{1}^{\infty}N_{{\rm
M}+\kappa}(\gamma)d\gamma,$

and

$\displaystyle\xi_{\rm
ene}=\int_{1}^{\infty}(\gamma-1)N_{\kappa}(\gamma)d\gamma/\int_{1}^{\infty}(\gamma-1)N_{{\rm
M}+\kappa}(\gamma)d\gamma,$

respectively. We discussed that the energy spectrum can be approximated by the
combination of Maxwellian and kappa distribution functions in Figure 2, and
the above denominator indicates the total contribution to the energy spectrum
coming from both the Maxwellian and $\kappa$ portions. The percentages of the
number and energy densities of the population of the kappa distribution
function weakly depend on the plasma temperature $T/mc^{2}$, and we determine
that the kappa distributions occupy more than $95\%$ and $70\%$ the total
energy and total number densities, respectively. The exhaust region contains
the freshly injected plasma from the outside plasma sheet, but the
contribution of the cold Maxwellian population is less than $5\%$ in the total
energy density.

Figure 4b illustrates the ratios of the nonthermal to total number densities
(solid red line), and the nonthermal energy to total energy densities (dashed
blue line) as the function of the initial plasma temperature $T/mc^{2}$,
respectively. We estimate the percentage of the nonthermal number density by
the following definition of

$\displaystyle\varepsilon_{\rm
den}=\int_{1}^{\infty}(N_{\kappa}(\gamma)-N_{\kappa}^{\rm
M}(\gamma))d\gamma/\int_{1}^{\infty}N_{{\rm M}+\kappa}(\gamma)d\gamma,$

where $N_{\kappa}^{\rm M}(\gamma)$ represents the portion of Maxwellian
distribution function out of the $\kappa$ distribution function, i.e., the
$\kappa$ value is replaced by $\kappa=\infty$ by keeping other fitted
parameters. Therefore, the numerator of $N_{\kappa}(\gamma)-N_{\kappa}^{\rm
M}$ corresponds to only the nonthermal power-law component. Note that a kappa
distribution function approaches a Maxwellian distribution as
$\kappa\to\infty$, as follows,

$\displaystyle\lim_{\kappa\to\infty}\left(1+\frac{\gamma-1}{\kappa
T_{\kappa}}\right)^{-(\kappa+1)}\simeq\exp\left(-\frac{\gamma-1}{T_{\kappa}}\right).$
(8)

Similarly, the nonthermal energy density can be calculated by,

$\displaystyle\varepsilon_{\rm
ene}=\int_{1}^{\infty}(\gamma-1)(N_{\kappa}(\gamma)-N_{\kappa}^{\rm
M}(\gamma))d\gamma/$ $\displaystyle\int_{1}^{\infty}(\gamma-1)N_{{\rm
M}+\kappa}(\gamma)d\gamma.$

From Figure 4b, it is determined that the nonthermal number and energy
densities increase with an increase in the plasma temperature in the non-
relativistic regime of $T/mc^{2}<1$, while the number and energy densities are
approximately constant in the relativistic regime of $T/mc^{2}>1$. We
determine that the relativistic reconnection can efficiently generate a lot of
nonthermal particles compared with the non-relativistic reconnection. The
percentage of the nonthermal number density saturates at $60\%$, while that of
the energy density is approximately $95\%$ of the total internal energy in the
downstream exhaust. This result suggests that the energy density of the
nonthermal population dominates for the relativistic reconnection with a
relativistic temperature, and the thermal energy density population is
negligible. The similar nonthermal efficiency has been discussed for trans-
relativistic reconnection by Ball et al. [18].

##  V Evolution of Nonthermal Plasma in Association with Magnetic Flux Tube
Transport

So far, we discussed the nonthermal energy spectra integrated all particles in
the exhaust plasma sheet, which is sandwiched by the last reconnected magnetic
field line; however, it would be interesting to discuss where and how those
nonthermal particles are accelerated in the plasma sheet. It is known that the
exhaust plasma sheet has several distinct structures responsible for particle
acceleration: (i) the magnetic diffusion region with weak magnetic fields,
where the fresh particles transported from the outside plasma sheet are picked
up and accelerated by the reconnection electric field, (ii) the plasma sheet
boundary layer between the outside and exhaust plasma sheets where the plasma
gas pressure suddenly increases, (iii) the plasmoids where the high-
temperature and high-density plasmas are confined by the closed magnetic flux
rope, and (iv) the magnetic field pile-up region where the Alfvénic jet from
the diffusion region collides with the pre-existing plasma sheet/plasmoid etc.
In association with those distinct regions, their roles on plasma heating and
acceleration processes have been discussed by paying attention to the
individual particle motion [e.g 34, 35, 36, 31, 37, 38, 39].

Figure 5: The colour contour of the plasma density structure of magnetic
reconnection for $T/mc^{2}=10^{-2}$. The logarithmic density contour is
indicated in the right-hand side bar. The white lines indicate the magnetic
field lines, i.e., the contour lines of the vector potential $A_{z}(x,y)$. The
interval of the contour line is chosen in such a way that the downstream
exhaust region is divided by ten magnetic field lines. The thick white lines
are depicted with the flux tube number for every five lines, which corresponds
to the spatial evolution of the energy spectrum indicated by the right-hand
color box in Figure 6. Figure 6: Energy spectra for $T/mc^{2}=10^{-2}$ (left),
$T/mc^{2}=1/\sqrt{10}$ (middle) and $T/mc^{2}=10$ (right), respectively. The
same data set as Figure 2. The color lines correspond to the energy spectra
for the 19 divided regions illustrated in Figure 5. The dark blue color
spectrum is taken for the outermost region, while the red one is the innermost
region. The right-hand side color bar indicates the region number. The solid
black lines are the energy spectra for the downstream exhaust regions, which
are the same spectra as illustrated in Figure 2. The black dashed lines are
the downstream exhaust spectra in which the contributions in the inner most
flux tube are subtracted.

The relative importance of the efficiency of nonthermal particle production in
those different regions, however, has not necessarily been discussed in a
systematic manner. To understand the relative importance, we focus on the
spatial evolution of the nonthermal energy spectrum in association with the
magnetic flux transportation. We divide the exhaust plasma sheet with 10
magnetic flux tubes by the equal interval of the vector potential $\Delta
A_{z}=(A_{z}^{\rm max}-A_{z}^{\rm min})/10$, where $A_{z}^{\rm max}$ and
$A_{z}^{\rm min}$ correspond to the vector potential for the last reconnected
magnetic field line at the X-type neutral point, and that for the minimum
bottom of the O-type point in the plasmoid, respectively. The reconnection
structure divided by the 19 magnetic flux tubes for $T/mc^{2}=10^{-2}$ is
illustrated in Figure 5, and we mark labels on the magnetic field lines from 1
– 19 from inside to outside. The last reconnected magnetic field line
corresponds to the label number of 10. The underlying assumption of this
analysis is that the thermal and energetic particles are transported together
with the magnetic flux, because we have confirmed that the frozen-in condition
is approximately satisfied [32].

Figure 6 illustrates the energy spectra for those 19 divided regions labeled
by the rainbow colors indicated by the right-hand side color bar. To focus on
the nonthermal tail, the energy range in the horizontal axis is chosen to be
three orders of magnitudes. The red line is the energy spectrum for the flux
tube number 1, which corresponds to the innermost region almost occupied by
the yellow color in Figure 5. The green line of the label number 10
corresponds to the flux tube just attached to the X-type neutral point, and
the dark blue is that of outside the plasma sheet. Evidently, the blueish
lines are basically the thermal Maxwellian spectrum with the initial plasma
temperature almost maintained, thus suggesting that no plasma heating and
acceleration occurs during the plasma transport from the outside plasma sheet
toward the separatrix region. (Note that this statement is not necessarily
correct if the evolution of the velocity distribution function is discussed in
detail, because the adiabatic plasma process can be observed in association
with the change of flux tube, which is initiated by the slow mode expansion
waves generated around the magnetic diffusion region [32]).

However, when the magnetic flux tube is attached to the separatrix, the rapid
increase in the nonthermal particle population can be observed, regardless of
the initial plasma temperature. The flux numbers of 10 and 11 with the
greenish colors corresponds to this region, and the flux number 11 has been
highlighted by the thick green color. This behavior suggests that the initial
significant particle acceleration can happen during the reconnection of two
different magnetic flux tube. The flux tube number 11 is just situated outside
the separatrix, but the magnetic diffusion region at the X-type neutral point
has a finite width in $y$-direction, and the flux tube is contaminated by
those accelerated particles around the diffusion region.

The main energy gain in the flux tube numbers 10 and 11 comes from the
meandering particles in the diffusion region [35, 37] where the meandering
particles are accelerated by the inductive electric field $E_{z}$. Note that
the resonant acceleration in the diffusion region can be enhanced in a
relativistic plasma due to the relativistic inertia effect [13]. In addition
to the acceleration in the diffusion region, the beam plasma heating can occur
in the plasma sheet boundary, because a part of the accelerated particles
around the diffusion region may have a small pitch-angle against the local
magnetic field, and those particles can escape outward along the magnetic
field line. The interaction of the field-aligned beam component and cold
plasma transported by $E\times B$ motion from the outside plasma sheet may
lead to the plasma heating together with the generation of Alfvénic waves
[33].

After rapid energization in the magnetic diffusion region and along the
separatrix magnetic field lines, those flux tubes are convected downward, and
further gradual energization can be observed, as indicated by the yellowish
and reddish lines. However, we determine that those energy gains are not
necessarily strong, compared with those in the last reconnecting magnetic flux
tube. Several bumpy structures in the spectra may arise from the dynamical
evolution of plasmoids.

The thick black solid curves are the energy spectra for the downstream exhaust
region integrated from the flux tube number 1 – 10, which is the same as the
final stage illustrated in Figure 2. By looking at the spatial evolution of
the energy spectra from No.10 (flux tube of the separatrix) to No.2 (inner
flux tube), we more or less observe the gradual heating and particle
acceleration. However, we determine that the spectra for the innermost flux
tube No. 1 has a slightly different behavior, because of the pre-existed and
pre-heated plasma before the onset of reconnection. To distinguish the plasma
heating between the pre-existed plasma and freshly injected particles, we
indicate that the energy spectra integrated the region from the flux tube of
No.2 – No.10 as the black dashed line, i.e., we excluded the innermost flux
tube. As can be observed, the difference appears only around the middle energy
range between the thermal Maxwellian part and nonthermal power-law part, and
the kappa index becomes slightly harder as well. Regarding the case of the
relativistic reconnection, we obtain the kappa index of $\kappa<2$, and the
result is roughly consistent with the previous studies [13, 14, 15, 16]. The
efficiency of nonthermal density discussed in the previous section, however,
does not change so much.

Figure 7:  Time evolution of energy spectra for fixed magnetic flux tubes for
$T/mc^{2}=10^{-2}$ (left), $T/mc^{2}=1/\sqrt{10}$ (middle) and $T/mc^{2}=10$
(right), respectively. The same data set as Figure 2. The color lines indicate
the time evolution of spectra, whose time stages are indicated in the right-
hand side bar. The dark blue line is the initial state when the magnetic flux
tube is situated in the upstream. On the other hand, the red one is the final
stage when the flux tube is transported into the downstream exhaust. The thick
solid lines are the model fitting for the composed function of the Maxwellian
and kappa distributions $N_{{\rm M}+\kappa}(\gamma)$. The black dashed lines
in the lower energy regime represent the Maxwellian part of the fitting result
$N_{\rm M}(\gamma)$, while the other dashed lines in the higher energy regime
are the kappa distribution parts $N_{\kappa}(\gamma)$. The bottom panels
indicate the errors of the model fitting in the same format as Figure 2.

Although we discussed the spatial evolution of the energy spectra in Figure 6,
it is also interesting to mention the time evolution of the energy spectrum
for a fixed magnetic flux tube, which is transported from the outside plasma
sheet into the downstream exhaust region during reconnection. Figure 7
illustrates the time evolution of the energy spectra for the fix magnetic flux
tubes sandwiched between $\left|y\right|/\lambda=9$ and $12$ at the initial
state $t=0$. As could be easily expected, the behavior of the energization is
basically same as Figure 6. The dark blue color spectra are taken when the
flux tubes are situated in the upstream, and those temperatures almost remain
the initial Maxwellian. As time goes on, we observed the rapid energization
and formation of the nonthermal tail when two flux tubes just reconnect at the
X-type neutral point at $t/\tau_{A}\sim 120$. The timing of the attachment of
two flux tubes normalized by the Alfvén transit time $\tau_{A}$ was almost
same for three cases.

After the attachment of the two magnetic flux tubes, those reconnected
magnetic flux tubes are further transported into the downstream, and the high
energy particles are gradually produced. The red lines are the almost final
stages of the magnetic reconnection. The thick solid lines are the model
fitting for the composed function of the Maxwellian and kappa distributions
$N_{{\rm M}+\kappa}(\gamma)$. The black dashed lines in the lower energy
regime represent the Maxwellian part of the fitting result $N_{\rm
M}(\gamma)$, while the other dashed lines in the higher energy regime are the
kappa distribution parts $N_{\kappa}(\gamma)$. The fitted parameters of
($T_{M}/mc^{2}$, $T_{\kappa}/mc^{2}$, $\kappa$, $\gamma_{cut}$) are ($0.013$,
$0.078$, $6.1$, $6.2\times 10^{2}$), ($0.79$, $0.44$, $3.0$, $2.2\times
10^{1}$), and ($15.$, $9.0$, $2.6$, $4.7\times 10^{2}$) for
$T/mc^{2}=10^{-2}$, $1/\sqrt{10}$, and $10$, respectively. These fitted
parameters are basically same as those obtained in Figures 2 and 3. The
contributions of the Maxwellian part, however, are smaller than the kappa
distribution part compared with Figure 2, because the magnetic flux tubes are
convected into the deep downstream and the upstream cold plasma are heated.
The bottom panels indicate the errors of the model fitting in the same format
as Figure 2. We find that the model fittings are very good for a wide energy
ranges.

##  VI Discussions and Summary

During the last two decades, it has been discussed that magnetic reconnection
can produce not only the hot thermal plasma, but also the high-energy supra-
thermal particles. Specifically, relativistic reconnection can efficiently
produce nonthermal particles, whose energy spectrum is approximated by a hard
power-law function. However, how the efficiency changes between the non-
relativistic and relativistic reconnection has remained an open question.
Meanwhile, the detailed acceleration process focusing on the individual
particle motion has been well discussed by several people [e.g. 31, 35, 36,
37, 38, 39].

The interaction between the reconnection electric field $E_{z}$ and meandering
motion under the anti-parallel magnetic field $B_{x}$ in the neutral sheet is
known to play an important role as the primary acceleration process, and the
accelerated particles in and around the X-type neutral line can be ejected
outward by the Lorentz force because of the reconnecting magnetic field [34,
37]. Zenitani and Hoshino [13] investigated this process to the relativistic
reconnection, and argued that the interaction efficiency between the
reconnection electric field and meandering particle can be enhanced in the
relativistic reconnection, because of the relativistic inertia effect during
the Speiser motion [34].

In addition to the particle acceleration in and around the magnetic diffusion
region, several other possible mechanisms have been proposed. The collision
between the Alfvénic jet ejected from the magnetic diffusion region and
magnetic field pile-up region ahead of the plasmoid can provide the betatron-
like acceleration owing to the gradient-B drift motion [e.g 35]. Drake et al.
[36] proposed that the particles trapped by the plasmoid can be energized by a
Fermi process, which is the curvature drift motion in association of the time
evolution of the plasmoid. Drake et al. [36], Dahlin et al. [38] investigated
the above mechanisms responsible for energy gain using the guiding center
formalism. Oka et al. [39] argued that the particles can be accelerated during
coalescence of two plasmoids with the similar process that occurs in the
magnetic diffusion region, but with a stronger inductive electric field in
association with the dynamic coalescence.

We did not analyze such detail acceleration mechanisms in this study, rather
focused on the evolution of energy spectrum during the plasma transport in
association with the magnetic flux tube. We not only observed the primary
particle acceleration around the magnetic diffusion region, but also the
additional acceleration during the plasma transport in the exhaust. These
heating and acceleration are basically consistent with the previous studies.
In addition, we determined that the energy spectra obtained by PIC simulations
can be well approximated by a kappa distribution function for the high energy
population. By examining the model fitting results, we determined that the
efficiency of the nonthermal particle acceleration increases with increasing
plasma temperature, and the nonthermal energy density reaches over $95\%$ of
the total internal energy in the relativistic temperature of the exhaust,
whereas the nonthermal number density remains approximately $40\%\sim 60\%$
for the total number density in the exhaust.

Our simulation study in a two-dimensional system with the anti-parallel
magnetic field topology is only a first step toward understanding the energy
partition between thermal and nonthermal plasmas. We investigated the
efficiency of the nonthermal particle production for a pair plasma, but it is
important to understand the acceleration efficiency for an ion-electron
plasma. Recently, several people discussed the energy partition between ion
and electron, and it is revealed that ions are preferentially heated, compared
to electrons during magnetic reconnection by PIC simulations [40, 41, 42]. It
has been discussed that the ratio of the ion heating to the electron
$T_{i}/T_{e}$ can be approximated by $(m_{i}/m_{e})^{1/4}$ for the magnetic
reconnection without guide field, where $m_{i}$ and $m_{e}$ are the ion and
electron mass, respectively [32], whereas the electrons receive larger
energies than the ions for guide field reconnection [43]. Although ions seem
to receive much thermal energies compared with electrons, suprathermal
electrons seem to be efficiently generated, while the proton nonthermal
spectrum seems not to be necessarily well developed in particle simulations.
Significant attention has been paid to electron nonthermal particle
acceleration, but the study of ion nonthermal production is limited. We are
probably required to study a longer time evolution of reconnection for a large
system size.

It is important to check the dependence of the energy spectrum on the system
size of our PIC simulation. So far, we fixed the system size with (a)
$430\lambda\times 430\lambda$, but we studied other cases with (b)
$215\lambda\times 215\lambda$, (c) $860\lambda\times 215\lambda$, and (d)
$1720\lambda\times 430\lambda$ for $T/mc^{2}=0.1$. The fitting results for
$T_{M}$, $T_{\kappa}$, $\kappa$, $\varepsilon_{\rm den}$ and $\varepsilon_{\rm
ene}$ does not strongly depend on the system size, and we found that those
differences of $T_{M}$, $T_{\kappa}$, $\kappa$ on the system size were less
than $15\%$. However, the cutoff energy seems to weakly depend on the system
size, except for case (b). Regarding case (b), $\gamma_{\rm cut}$ was
approximately 6.7, but other cases of (a),(c) and (d) were $\gamma_{\rm
cut}\sim 9-12$.

It would be important to mention the effect of the plasma temperature in the
outside plasma sheet, which is denoted by $N_{b}$ in Equation (2). So far, we
assumed that the background temperature is the same as that for the Harris
plasma component $N_{0}$, but we can choose any temperature to keep an
equilibrium state. In astrophysical settings, it is likely that the plasma
temperature outside the plasma sheet is colder than the Harris plasma
temperature. In fact, for the case of the Earth’s magnetotail, the temperature
in the lobe is known to be colder than that in the plasma sheet [e.g. 24, 25].
We have studied the cold background plasma temperature $T_{b}/mc^{2}=10^{-4}$
by keeping other plasma parameters the same. We determine that the efficiency
of the nonthermal energy density is almost the same as the case of the uniform
background plasma temperature discussed in this study. We will discuss this
behavior in a separate paper.

Our final comment is on the effect of the background density $N_{b}$. It is
well known that the reconnection rate increases with an increase in the Alfvén
speed in the inflow region. Furthermore, the reconnection electric field
$E_{z}$ becomes larger with a decrease in the background density, and the
magnetic energy dissipation rate being converted into the thermal and
nonthermal plasmas would be enhanced as well. Although the nonthermal particle
production would be expected to increase with a decrease in the background
plasma density, the energy partition between the thermal and nonthermal plasma
production should be carefully studied. In our preliminary study for the low
background density with $N_{b}/N_{0}=1/80$, we obtained the nonthermal
$\kappa$ values of $2.2$, $2.6$, $2.7$, and $4.3$ for $T/mc^{2}=10$, $1$,
$10^{-1}$ and $10^{-2}$, respectively. We find that these $\kappa$ values are
about $30\%$ smaller than those of $N_{b}/N_{0}=1/20$ shown in Figure 3b. It
would be also interesting to study the effects of background temperature and
density for the generalized magnetization parameter $\sigma=B^{2}/(4\pi w)$
and Alfvén speed $v_{A}=c/\sqrt{1+4\pi(e+p)/B^{2}}$, where $w$, $e$ and $p$
are the enthalpy density, the total energy density and the gas pressure,
respectively. We will discuss these effects in a separate paper together with
the effect of the guide field.

###### Acknowledgements.

This study was supported by JSPS KAKENHI, Grant Number 20K20908.

## DATA AVAILABILITY

The data support the findings of this study are available from the
corresponding author upon reasonable request.

##  Appendix A Nonthermal contribution in $\kappa$ distribution function

We shortly discuss the basic behavior of $\kappa$ distribution function by
focusing on the thermal and non-thermal populations. As the $\kappa$
distribution function are given by two parameters of the temperature $T$ and
$\kappa$ index, the nonthermal contribution can be given by two parameters. By
assuming that the thermal population of the $\kappa$ distribution function is
given by keeping the same temperature but by setting $\kappa=\infty$, we can
easily estimate the percentages of the nonthermal number and nonthermal energy
densities. The top and bottom panels in Figure 8 illustrate the nonthermal
number density $\varepsilon_{\rm den}$ and the nonthermal energy density
$\varepsilon_{\rm ene}$, respectively. The horizontal axes are the $\kappa$
index and plasma temperature normalized by the rest mass energy $T/mc^{2}$. We
can observe that the nonthermal contributions become small for a large
$\kappa$ value. For the case of a small $\kappa$ and relativistic hot plasma,
we determine that the nonthermal contributions become significant. Regarding
$\kappa=3$, the nonthermal number density contribution $\varepsilon_{\rm den}$
becomes $\varepsilon_{\rm den}\rightarrow 7/9$ for
$T/mc^{2}\rightarrow\infty$, whereas the nonthermal energy density
contribution is $\varepsilon_{\rm ene}\rightarrow 1$ for
$T/mc^{2}\rightarrow\infty$. For $\kappa=4$ and $T/mc^{2}\rightarrow\infty$,
$\varepsilon_{\rm den}\rightarrow 5/8$ and $\varepsilon_{\rm ene}\rightarrow
29/32$.

Figure 8: The percentage of the nonthermal number density $\varepsilon_{\rm
den}$ (top) and nonthermal energy density $\varepsilon_{\rm ene}$ (bottom) as
the functions of the temperature $T/mc^{2}$ and kappa index $\kappa$.

It is also interesting to note the effect of the cutoff energy function
introduced in the simulation analysis. In our simulation results, the cutoff
energies were much larger than the heated plasma temperature, i.e.,
$\gamma_{cut}-1\sim 10^{2}\times T/mc^{2}$. By assuming that
$\gamma_{cut}-1=10^{2}\times T/mc^{2}$, we determined that the difference
between the nonthermal contribution with and without the cut-off energy is
less than several $\%$ for $\kappa>3$.

## References

  * Birn & Priest [2007] Birn, J. & Priest, E. R. 2007, Reconnection of magnetic fields : magnetohydrodynamics and collisionless theory and observations / edited by J. Birn and E. R. Priest. Cambridge : Cambridge University Press, 2007. ISBN: 9780521854207 (hbk.)
  * Zweibel & Yamada [2009] Zweibel, E. G. & Yamada, M. 2009, Ann. Rev. Astron. Astrophys., 47, 291. doi:10.1146/annurev-astro-082708-101726
  * Hoshino & Lyubarsky [2012] Hoshino, M., & Lyubarsky, Y. 2012, Space Science Reviews, 173, 521
  * Uzdensky [2016] Uzdensky, D. A. 2016, Magnetic Reconnection: Concepts and Applications, 473
  * Blandford et al. [2017] Blandford, R., Yuan, Y., Hoshino, M., & Sironi, L., 2017, Space Science Reviews, 207, 291
  * Coppi et al. [1966] Coppi, B., Laval, G., & Pellat, R. 1966, Phys. Rev. Lett. , 16, 1207
  * Hoh [1966] Hoh, F. C. 1966, Physics of Fluids, 9, 277
  * Schindler [1966] Schindler, K. 1966, Phenomena in Ionized Gases, Volume II, VII International Conference, 736
  * Biskamp & Schindler [1971] Biskamp, D. & Schindler, K. 1971, Plasma Physics, 13, 1013. doi:10.1088/0032-1028/13/11/003
  * Galeev & Zelenyǐ [1976] Galeev, A. A. & Zelenyǐ, L. M. 1976, Soviet Journal of Experimental and Theoretical Physics, 43, 1113
  * Bell [1978] Bell, A. R. 1978, Monthly Notices of the Royal Astronomical Society, 182, 147
  * Blandford and Ostriker [1978] Blandford, R. D., & Ostriker, J. P. 1978, The Astrophysical Journal, 221, L29
  * Zenitani and Hoshino [2001] Zenitani, S., & Hoshino, M. 2001, The Astrophysical Journal, 562, L63
  * Jaroschek et al. [2004] Jaroschek, C. H., Lesch, H., & Treumann, R. A. 2004, The Astrophysical Journal, 605, L9
  * Guo et al. [2014] Guo, F., Li, H., Daughton, W., & Liu, Y.-H. 2014, Physical Review Letters, 113, 155005
  * Sironi et al. [2014] Sironi, L., & Spitkovsky, A. 2014, The Astrophysical Journal, 783, L21
  * Guo et al. [2020] Guo, F., Liu, Y.-H., Li, X., Li, H., Daughton, W., and Kilian, P. 2020, Physics of Plasmas, 27, 080501. doi:10.1063/5.0012094
  * Ball et al. [2018] Ball D., Sironi L., Özel F., 2018, ApJ, 862, 80. doi:10.3847/1538-4357/aac820
  * Vasyliunas [1968] Vasyliunas, V. M. 1968, J. Geophys. Res., 73, 2839. doi:10.1029/JA073i009p02839
  * Hoshino [1987] Hoshino, M. 1987, J. Geophys. Res., 92, 7368. doi:10.1029/JA092iA07p07368
  * Hoshino [2020] Hoshino, M. 2020, Astrophys. J. , 900, 66. doi:10.3847/1538-4357/aba59d
  * Hoshino [2021] Hoshino, M. 2021, Physics of Plasmas, 28, 062106. doi:10.1063/5.0050389
  * Harris [1962] Harris, E. G. 1962, Il Nuovo Cimento, 23, 115
  * Asano et al. [2003] Asano, Y., Mukai, T., Hoshino, M., Saito, Y., Hayakawa, H., & Nagai, T. 2003, Journal of Geophysical Research (Space Physics), 108, 1189. doi:10.1029/2002JA009785
  * Sharma et al. [2008] Sharma, A. S., Nakamura, R., Runov, A., Grigorenko, E. E., Hasegawa, H., Hoshino, M., Louarn, P., Owen, C. J., Petrukovich, A., Sauvaud, J. -A., et al. 2008, Annales Geophysicae, 26, 955. doi:10.5194/angeo-26-955-2008
  * Feldman et al. [1975] Feldman, W. C., Asbridge, J. R., Bame, S. J., Montgomery, M. D., & Gary, S. P. 1975, J. Geophys. Res., 80, 4181. doi:10.1029/JA080i031p04181
  * Livadiotis & McComas [2013] Livadiotis, G. & McComas, D. J. 2013, Space Sci. Rev., 175, 183. doi:10.1007/s11214-013-9982-9
  * Lazar et al. [2017] Lazar, M., Pierrard, V., Shaaban, S. M., Fichtner, H., & Poedts, S. 2017, A & A, 602, A44. doi:10.1051/0004-6361/201630194
  * Werner et al. [2016] Werner, G.R., Uzdensky, D. A., Cerutti, B., Nalewajko, K., & Begelman, M. C. 2016, Astrophys. J. Lett., 816, L8. doi:10.3847/2041-8205/816/1/L8
  * Petropoulou & Sironi [2018] Petropoulou, M. & Sironi, L. 2018, Mon. R. Astron. Soc., 481, 5687. doi:10.1093/mnras/sty2702
  * Hoshino et al. [1998] Hoshino, M., Mukai, T., Yamamoto, T., & Kokubun, S. 1998, J. Geophys. Res., 103, 4509. doi:10.1029/97JA01785
  * Hoshino [2018] Hoshino, M. 2018, Astrophys. J. Lett., 868, L18. doi:10.3847/2041-8213/aaef3a
  * Hesse et al. [2018] Hesse, M., Norgren, C., Tenfjord, P., Burch, J. L., Liu, Y.-H., Chen, L.-J., Bessho, N., Wang, S., Nakamura, R., Eastwood, J. P., et al. 2018, Physics of Plasmas, 25, 122902. doi:10.1063/1.5054100
  * Speiser [1965] Speiser, T. W. 1965, J. Geophys. Res., 70, 4219. doi:10.1029/JZ070i017p04219
  * Hoshino et al. [2001] Hoshino, M., Mukai, T., Terasawa, T., & Shinohara, I. 2001, J. Geophys. Res., 106, 25979. doi:10.1029/2001JA900052
  * Drake et al. [2006] Drake, J. F., Swisdak, M., Schoeffler, K. M., Rogers, B. N., & Kobayashi, S. 2006, Geophys. Res. Lett., 33, L13105. doi:10.1029/2006GL025957
  * Pritchett [2001] Pritchett, P. L. 2001, J. Geophys. Res., 106, 3783. doi:10.1029/1999JA001006
  * Dahlin et al. [2014] Dahlin, J. T., Drake, J. F., & Swisdak, M. 2014, Physics of Plasmas, 21, 092304. doi:10.1063/1.4894484
  * Oka et al. [2010] Oka, M., Phan, T.-D., Krucker, S., Fujimoto, M., & Shinohara, I. 2010, Astrophys. J. , 714, 915. doi:10.1088/0004-637X/714/1/915
  * Shay et al. [2014] Shay, M. A., Haggerty, C. C., Phan, T. D., Drake, J. F., Cassak, P. A., Wu, P., Oieroset, M., Swisdak, M., & Malakit, K. 2014, Physics of Plasmas, 21, 122902. doi:10.1063/1.4904203
  * Haggerty et al. [2015] Haggerty, C. C., Shay, M. A., Drake, J. F., Phan, T. D., & McHugh, C. T. 2015, Geophys. Res. Lett., 42, 9657. doi:10.1002/2015GL065961
  * Rowan et al. [2017] Rowan, M. E., Sironi, L., & Narayan, R. 2017, Astrophys. J. , 850, 29. doi:10.3847/1538-4357/aa9380
  * Rowan et al. [2019] Rowan, M. E., Sironi, L., & Narayan, R. 2019, Astrophys. J. , 873, 2. doi:10.3847/1538-4357/ab03d7


11institutetext: Istanbul Technical University, Istanbul, Turkey
22institutetext: Mehmet Akif Ersoy Thoracic and Cardiovascular Surgery
Training and Research Hospital, Istanbul, Turkey 33institutetext: Acibadem
University, Istanbul, Turkey 44institutetext: King’s College London
∗Corresponding author: 44email<EMAIL_ADDRESS>
# Shifted Windows Transformers for Medical Image Quality Assessment

Caner Özer ** 1 1 Arda Güler 22 Aysel Türkvatan Cansever 22 Deniz Alis 33
Ercan Karaarslan 33 İlkay Öksüz 1144

###### Abstract

To maintain a standard in a medical imaging study, images should have
necessary image quality for potential diagnostic use. Although CNN-based
approaches are used to assess the image quality, their performance can still
be improved in terms of accuracy. In this work, we approach this problem by
using Swin Transformer, which improves the poor-quality image classification
performance that causes the degradation in medical image quality. We test our
approach on Foreign Object Classification problem on Chest X-Rays (Object-CXR)
and Left Ventricular Outflow Tract Classification problem on Cardiac MRI with
a four-chamber view (LVOT). While we obtain a classification accuracy of 87.1%
and 95.48% on the Object-CXR and LVOT datasets, our experimental results
suggest that the use of Swin Transformer improves the Object-CXR
classification performance while obtaining a comparable performance for the
LVOT dataset. To the best of our knowledge, our study is the first vision
transformer application for medical image quality assessment.

###### Keywords:

Visual Transformers Medical Image Quality Assessment Chest X-Rays Cardiac MRI.

##  1 Introduction

It is the responsibility of technicians to ensure that the acquired medical
scan should be of good quality so that the physician, who asked for the
medical scan, will be able to diagnose accurately. However, this condition may
not be satisfied due to the medical image quality problems such as patient
breathing or arrhythmia in Cardiac Magnetic Resonance Imaging (MRI), ringing
artifact presence in Computed Tomography (CT), and low dose or foreign object
appearance on the Chest X-Rays (CXR). These image quality issues would also
constitute a critical problem for automated processing of medical scans within
a clinical decision support system consisting of big data.

Maintaining a high standard in image quality is imperative also for a machine
learning-guided medical image analysis downstream tasks. More specifically,
existence of poor quality images may lead a machine learning methodology to
learn spurious set of features which are not clinically useful. Hence,
identifying, eliminating or correcting the samples with poor image quality is
necessary for downstream tasks such as segmentation and disease diagnosis. The
reasons for an image that should be eliminated from the pipeline include but
are not limited to artefacts in MR Imaging [11], foreign object existence
[15], or undesirable region appearance [12] within the scans. For this
purpose, several datasets on a variety of modalities have been developed to
overcome such problems. Ma et. al. [11] has developed a brain MRI dataset in
which the patient data has been labelled according to the level of motion
artefact. The authors have also developed a 4-layer convolutional neural
network model to classify the samples according to their motion degradation
level. JFHealthcare [8] proposed a MIDL 2020 challenge which comes up with a
dataset that aims to localize the foreign objects within chest X-Rays.

Swin-Transformers are becoming more common in medical image analysis,
especially in the field of diagnosis. For instance, Zhang et al. [18] uses a
cascaded mechanism for chest CT COVID-19 diagnosis which uses a U-Net to
extract lungs as a foreground and background segmentation mechanism, and Swin-
Transformer uses these segmented lungs to find out whether the patient has
COVID-19 or not. Chen et al. [2] combines a statistical significance-based
slice sampling scheme and Swin-Transformers to propose a framework that can
detect COVID-19 from chest CT scans by using a single slice. In addition to
diagnosis, Swin-Transformers are becoming more common to be used for other
problems, such as brain tumor segmentation [1] [5], and cardiac CT
reconstruction [13]. However, despite being promising for these problems in
medical imaging, Swin-Transformers’ have yet to be studied for medical image
quality assessment tasks.

In this work, we propose a SwinTransformer-based [10] approach for medical
image quality analysis where our contributions are as follows:

  * •

We train a variety of SwinTransformer models in order to classify X-Ray images
with foreign objects and Cardiac MRI images with a 4-chamber view with Left
Ventricular Outflow Tract (LVOT).

  * •

We compare our approach with deep neural network (DNN) based approaches, such
as ResNet [6] and EfficientNet [14].

  * •

To our best knowledge, this is the first paper to perform medical image
quality assessment through SwinTransformers.

##  2 Methodology

Figure 1: Swin Transformer pipeline for medical image quality analysis.

###  2.1 Pre-processing

In order to train our model, we apply a number of preprocessing procedures
prior to feeding the input images to the network. In this regard, instead of
manually defining the data augmentations and their corresponding
hyperparameters, we use RandAugment [3] which uses a previously found
automated data augmentation strategy to train a DNN. After that, we resize
these images to a pre-determined resolution of $H\times W$. Finally, we
normalize the images by using ImageNet statistics that they become ready-to-
use in a pre-trained model.

###  2.2 Swin Transformer

Following the pre-processing procedure, we used the Swin Transformer model
that is being demonstrated in Fig. 1. Given an input image, Swin Transformer
first divides the image into patches using a Patch Partition layer that
generates $48$-dimensional representations for each of the $4\times 4$ patches
on the image. After that, as shown in Stage 1, the Linear Embedding layer
transforms $48$-d representations into $C$ dimensions.

The second component of Stage 1 is the Swin Transformer Blocks which has been
inherited from the general Visual Transformer (ViT) [4] architecture. Despite
having the same high-level block structure as the one in ViT, multiheaded
self-attention (MSA) blocks are replaced with an alternate ordering of window-
based MSA (W-MSA) and shifted window-based MSA (SW-MSA), sequentially. In
W-MSA, the feature map is divided into windows and MSA computes the output on
each of them. However, this brings a cost of missing global connectivity,
since this was not an issue for MSA in ViT that operates globally. For this
reason, cyclic shifting operation is used to overcome this problem with SW-MSA
that, the feature map is circularly shifted on the height and width axes by
the half-length of the patch size. Then, W-MSA is executed on the resulting
feature map, and a reverse circular shift operation is used to obtain a
reverted feature map without any shifts.

In all of Stages 2, 3, and 4, we have a Patch Merging layer and a varying
number of Swin Transformer blocks. In the Patch Merging layer, each of the
pixels located on the $2\times 2$ patches on the feature map are concatenated
across the channel axis, i.e., $\frac{H}{2}\times\frac{W}{2}\times C$ to
$\frac{H}{4}\times\frac{W}{4}\times 4C$. Then, after applying the layer
normalization layer, a linear layer is used to reduce the dimensionality to
half, i.e., to $\frac{H}{4}\times\frac{W}{4}\times 2C$. The output is then
transferred to the Swin Transformer blocks for further processing. Lastly,
after using layer normalization followed by an average pooling layer to
calculate the features of the Swin Transformer model, the output is used for
classification in a single Linear layer. This model classifies if there is
anything present in the input image that can affect the image quality
adversely. The hyperparameter settings can be found in Table 1. Stochastic
depth refers to the probability of disregarding the transformation that is
applied on a sample within a batch during training, while being processed
within a Swin Transformer block, embedding dims $C$ is the dimensionality of
the patch-wise embeddings, depths and MSA heads correspond to the number of
repeated Swin Transformer blocks and head count of the MSA layer for each
stage, respectively.

Table 1: Hyperparameter settings for different Swin Transformer models. Model Name |  | Stochastic Depth
---
| Embedding Dims $(C)$
---
Depths | MSA Heads
Swin-Tiny | 0.2 | 96 | 2, 2, 6, 2 | 3, 6, 12, 24
Swin-Small | 0.3 | 96 | 2, 2, 18, 2 | 3, 6, 12, 24
Swin-Base | 0.2 | 128 | 2, 2, 18, 2 | 4, 8, 16, 32

###  2.3 Training the Model

We adopt the official code of Swin
Transformers111https://github.com/microsoft/Swin-Transformer in which we
performed some modifications to run it on PyTorch 1.10.2. We also used
TorchVision 0.11.3 to inherit the pre-trained models for EfficientNet and
ResNet pre-trained architectures. We apply the learning rate warm-up scheme
and learning rate scheduling and leverage color jittering, stochastic depth
(only for Swin Transformer) [7], Random Erasing [19] MixUp [17], CutMix [16]
and RandAugment [3] data augmentation methods for promoting model
generalization. For the foreign object classification model, we set the batch
size to 16, gradient accumulation steps to 8 and window size to 8. Instead for
the LVOT classification model, we set the batch size to 64, gradient
accumulation steps to 2 and window size to 7, due to lower input image
resolution comparing chest X-Rays. We employ an AdamW [9] optimizer with
weight-decay of $10^{-8}$ to fine-tune both of these models by using an
ImageNet pretrained model, as we set the initial learning rate to $0.00006$
and use a linear learning-rate scheduler. The fine-tuning procedure takes $60$
epochs with $5$ epochs of a warm-up period. We also trained the models from
scratch for $300$ epochs that, the learning rate is set to $0.0005$ and the
objective function is Cross Entropy. You can access to our repository by using
the following link: https://github.com/canerozer/qct.

##  3 Experimental Results

###  3.1 Datasets

We demonstrate our results on the Object-CXR [8], a challenge dataset where
the aim is to classify the Chest X-Ray scans with foreign objects and to
localize the foreign objects, and the LVOT classification dataset, which is
constructed by selecting a subset of the Cardiac MR (CMR) scans with 4-chamber
view that were acquired in the Mehmet Akif Ersoy Thoracic and Cardiovascular
Surgery Training and Research Hospital between the years 2016 and 2019.
Object-CXR dataset contains 8,000 samples for training and 1,000 samples for
each of the validation and testing splits. There is an equal number of
positive and negative samples for each split which does and does not contain
foreign objects. We only use the class label, but not the localization ground
truth for this dataset during training. We set the fixed input size to
$H=W=1024$ since the images can reach a spatial resolution of $4048\times
4932$. On the other hand, LVOT classification dataset contains 272 positive
and 278 negative samples that are used for training, 34 positive and 35
negative samples for validation, and 35 positive and 35 negative samples for
testing. These images have relatively way less spatial resolution than the
ones in Object-CXR, hence, we set $H=W=224$. In addition, since our pipeline
is capable of processing 2D images, we slice the 3D CMR patient scans into 2D
images, which is also helpful in terms of increasing the number of samples for
each split by a factor of 25.

###  3.2 Left Ventricular Outflow Tract Classification Results

First, we start by comparing the performances of different Swin Transformer
models that are named as Swin-Tiny, Swin-Small, and Swin-Base in Table 2. We
additionally measured the differences between using a pretrained model and
training from scratch. We notice that, the accuracy is the highest for the
Swin-Tiny model with an accuracy of $95.59\%$. Nevertheless, Swin-Small’s AUC
(Area Under Curve) score is performing the best among these three models with
a score of $0.980$. For this reason, we decided to include both of the Swin-
Small and Swin-Tiny models to compare them with the other baselines.
Meanwhile, switching from pre-trained model to training from scratch
significantly decreases the accuracy of at least around $12.35\%$ and AUC
score of $0.059$. We relate this to the data-hungry regime of Swin
Transformers, as they need abundant data samples to capture the inductive bias
during training. Hence, it becomes necessary to use the pretrained models or
to increase the number of samples to the order of tens of thousands.

Table 2: Comparison among Swin Transformer models with different capacity for the LVOT classification task.  Model | Pretrained? | Acc | AUC |  | # of params
---
GFLOPs |  | Throughput (img/sec)
---
Swin-Tiny | Yes |  95.59 | 0.969 | 27,520,892 | 4.5 | 515
No | 72.93 | 0.757
Swin-Small | Yes | 95.48 |  0.980 | 48,838,796 | 8.7 | 313
No | 83.13 | 0.921
Swin-Base | Yes | 93.86 | 0.976 | 86,745,274 | 15.4 | 222
No | 67.01 | 0.738
Table 3: Our results on the LVOT dataset indicate that the performance is comparable to the other DNN-based methods. Model | Acc | AUC |  | # of params
---
GFLOPs | Throughput
EfficientNet-B0 | 93.74 | 0.979 | 4,010,110 | 0.4 | 2274
EfficientNet-B1 | 93.68 | 0.971 | 6,515,746 | 0.6 | 1565
EfficientNet-B2 | 94.61 | 0.971 | 7,703,812 | 0.7 | 1480
ResNet-34 | 95.19 | 0.974 | 21,285,698 | 3.7 | 1963
ResNet-50 | 95.77 | 0.979 | 23,512,130 | 4.1 | 1005
ResNet-101 | 95.59 | 0.969 | 42,504,258 | 7.8 | 587
ResNet-152 |  96.00 |  0.988 | 58,147,906 | 11.6 | 409
Swin-Tiny | 95.59 | 0.969 | 27,520,892 | 4.5 | 515
Swin-Small | 95.48 | 0.980 | 48,838,796 | 8.7 | 313

Then, we compare our approach with other deep learning baselines as ResNet [6]
and EfficientNet [14] as shown in Table 3. We see that, despite the
differences in the number of parameters and GFLOPs, the performance varies
insignificantly ($p>0.05$) with outstanding accuracy and AUC metrics. We
relate this to the triviality of the LVOT classification task, where the goal
is to check if fifth chamber exists on the Cardiac MRI scan - generally
located at the centre. As a result, it becomes possible to obtain testing
accuracy of over $95\%$ and AUC scores exceeding $0.969$ for all ResNet
architectures and Swin-Small without overfitting. Still, these models are
ready to be deployed for a real-world scenario for alerting LVOT regions on
4-chamber Cardiac MRIs almost instantaneously.

###  3.3 Foreign Object Classification Results

Table 4: Results for the Object-CXR dataset. Model | Acc | AUC |  | # of params
---
GFLOPs | Throughput
EfficientNet-B0 | 82.1 | 0.873 | 4,010,110 | 8.3 | 119
EfficientNet-B1 | 83.0 | 0.882 | 6,515,746 | 12.3 | 83
EfficientNet-B2 | 83.2 | 0.890 | 7,703,812 | 14.2 | 78
ResNet-34 | 82.9 | 0.895 | 21,285,698 | 91.8 | 94
ResNet-50 | 82.8 | 0.897 | 23,512,130 | 102.7 | 42
ResNet-101 | 84.1 | 0.906 | 42,504,258 | 195.8 | 26
ResNet-152 | 85.1 | 0.909 | 58,147,906 | 241.5 | 22
Swin-Base |  87.1 |  0.922 | 86,766,330 | 324.6 | 11

The results show Swin Transformer’s strength on a higher dimensional input,
namely, on the Chest X-Ray data for foreign object classification. The model
complexity is highly correlated with the model performance that, the best
performance is obtained by the Swin-Base model with $87.1\%$ testing accuracy
and $0.922$ testing AUC score. The closest performance to the best model is
the ResNet-152 model, which has $1.5\times$ less number of parameters and
$25\%$ less floating point operations, at a cost of $2\%$ less testing
accuracy and $0.013$ less AUC score. Despite Swin-Base is working slightly
slower than real-time due to its huge input size, its significant performance
gains ($p<0.005$) makes it a promising approach.

###  3.4 Qualitative Analysis

Prior to utilizing our model in a clinical study, we further examine our model
under extreme conditions. Firstly, we start with the Object-CXR classification
task as some cases are shown in Fig. 2. Regardless of the orientation and
contrast factors, our model can correctly classify the X-Ray images which does
and does not contain foreign objects as demonstrated in Fig. 2a and 2b,
respectively. What is interesting especially for the rightmost image of Fig.
2b is that, even though the foreign objects may not be visible to human eye
without altering the contrast, our model can still successfully classify the
image as it contains foreign objects. Foreign objects are marked with a blue
bounding box in Fig. 2b.

Figure 2: Some true challenging samples due to their medical image quality
factors, from Object-CXR dataset.

We also perform a qualitative analysis on the classification results as some
of the edge cases are shown in Fig. 3 for the LVOT classification task. In
Fig. 3a, where all 3 images are labelled as good quality images without LVOT
appearance, the model can correctly classify even when the images are subject
to motion artefacts and local noise upto some level. Also, as shown in Fig.
3b, our model can correctly classify the images containing LVOT regions under
low contrast. We show the regions with LVOT with a blue bounding box in Fig.
3b.

Figure 3: Some true challenging samples due to their medical image quality
factors, from LVOT dataset.

##  4 Discussion & Conclusion

In this work, we propose to use Swin Transformers for medical image quality
analysis, where we validated our approach on two datasets working on two
different modalities. We outperform a variety of ResNet and EfficientNet
baselines on the Object-CXR dataset by obtaining a testing accuracy of
$87.1\%$ while obtaining a comparable performance for the LVOT classification
objective. Comparing Swin Transformer to the other baselines and performing a
qualitative analysis demonstrate its potential for utilizing it in a clinical
setting.

Although the problem setting has been restricted to foreign object
classification in Chest X-Rays and LVOT classification for Cardiac MRI’s, it
is possible to generalize the approach for other medical image quality
problems such as motion artefact grade classification. In addition, the
interpretability aspect of the Swin Transformers is a critical avenue of
research for the safe transition of an automated medical image analysis
pipeline.

## Acknowledgments

This paper has been produced benefiting from the 2232 International Fellowship
for Outstanding Researchers Program of TUBITAK (Project No: 118C353). However,
the entire responsibility of the publication/paper belongs to the owner of the
paper. The financial support received from TUBITAK does not mean that the
content of the publication is approved in a scientific sense by TUBITAK.

## References

  * [1] Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., Wang, M.: Swin-unet: Unet-like pure transformer for medical image segmentation (2021)
  * [2] Chen, G.L., Hsu, C.C., Wu, M.H.: Adaptive distribution learning with statistical hypothesis testing for covid-19 ct scan classification. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). pp. 471–479 (2021). https://doi.org/10.1109/ICCVW54120.2021.00057
  * [3] Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.: Randaugment: Practical automated data augmentation with a reduced search space. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems. vol. 33, pp. 18613–18624. Curran Associates, Inc. (2020), https://proceedings.neurips.cc/paper/2020/file/d85b63ef0ccb114d0a3bb7b7d808028f-Paper.pdf
  * [4] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net (2021), https://openreview.net/forum?id=YicbFdNTTy
  * [5] Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H., Xu, D.: Swin UNETR: swin transformers for semantic segmentation of brain tumors in MRI images. CoRR abs/2201.01266 (2022), https://arxiv.org/abs/2201.01266
  * [6] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. pp. 770–778. IEEE Computer Society (2016). https://doi.org/10.1109/CVPR.2016.90, https://doi.org/10.1109/CVPR.2016.90
  * [7] Huang, G., Sun, Y., Liu, Z., Sedra, D., Weinberger, K.Q.: Deep networks with stochastic depth. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV. Lecture Notes in Computer Science, vol. 9908, pp. 646–661. Springer (2016). https://doi.org/10.1007/978-3-319-46493-0_39, https://doi.org/10.1007/978-3-319-46493-0_39
  * [8] JFHealthcare: Automatic detection of foreign objects on chest x-rays. https://academictorrents.com/details/fdc91f11d7010f7259a05403fc9d00079a09f5d5 (2020), accessed on 11/27/2020
  * [9] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015), http://arxiv.org/abs/1412.6980
  * [10] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 10012–10022 (October 2021)
  * [11] Ma, J.J., Nakarmi, U., Kin, C.Y.S., Sandino, C.M., Cheng, J.Y., Syed, A.B., Wei, P., Pauly, J.M., Vasanawala, S.S.: Diagnostic image quality assessment and classification in medical imaging: Opportunities and challenges. In: 17th IEEE International Symposium on Biomedical Imaging, ISBI 2020, Iowa City, IA, USA, April 3-7, 2020. pp. 337–340. IEEE (2020). https://doi.org/10.1109/ISBI45749.2020.9098735, https://doi.org/10.1109/ISBI45749.2020.9098735
  * [12] Oksuz, I., Ruijsink, B., Puyol-Antón, E., Sinclair, M., Rueckert, D., Schnabel, J.A., King, A.P.: Automatic left ventricular outflow tract classification for accurate cardiac mr planning. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). pp. 462–465 (2018). https://doi.org/10.1109/ISBI.2018.8363616
  * [13] Pan, J., Wu, W., Gao, Z., Zhang, H.: Mist-net: Multi-domain integrative swin transformer network for sparse-view CT reconstruction. CoRR abs/2111.14831 (2021), https://arxiv.org/abs/2111.14831
  * [14] Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA. Proceedings of Machine Learning Research, vol. 97, pp. 6105–6114. PMLR (2019), http://proceedings.mlr.press/v97/tan19a.html
  * [15] Xue, Z., Candemir, S., Antani, S., Long, L.R., Jaeger, S., Demner-Fushman, D., Thoma, G.R.: Foreign object detection in chest x-rays. In: 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 956–961 (2015). https://doi.org/10.1109/BIBM.2015.7359812
  * [16] Yun, S., Han, D., Chun, S., Oh, S.J., Yoo, Y., Choe, J.: Cutmix: Regularization strategy to train strong classifiers with localizable features. In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. pp. 6022–6031. IEEE (2019). https://doi.org/10.1109/ICCV.2019.00612, https://doi.org/10.1109/ICCV.2019.00612
  * [17] Zhang, H., Cissé, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net (2018), https://openreview.net/forum?id=r1Ddp1-Rb
  * [18] Zhang, L., Wen, Y.: A transformer-based framework for automatic covid19 diagnosis in chest cts. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). pp. 513–518 (2021). https://doi.org/10.1109/ICCVW54120.2021.00063
  * [19] Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random erasing data augmentation. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. pp. 13001–13008. AAAI Press (2020), https://aaai.org/ojs/index.php/AAAI/article/view/7000