# A systematic formulation of chiral anomalous magnetohydrodynamics

Michael Landry and Hong Liu  Center for Theoretical Physics, MIT, Cambridge,
MA 02139, USA

###### Abstract

We present a new way of deriving effective theories of dynamical
electromagnetic fields in general media. It can be used to give a systematic
formulation of magnetohydrodynamics (MHD) with strong magnetic fields,
including systems with chiral matter and Adler-Bell-Jackiw (ABJ) anomaly. We
work in the regime in which velocity and temperature fluctuations can be
neglected. The resulting chiral anomalous MHD incorporates and generalizes the
chiral magnetic effect, the chiral separation effect, the chiral electric
separation effect, as well as recently derived strong-field MHD, all in a
single coherent framework. At linearized level, the theory predicts that the
chiral magnetic wave survives strong dynamical magnetic fields, and predicts
the wave velocity. We also introduce a simple, but solvable nonlinear model to
explore the fate of the chiral instability.

††preprint: MIT-CTP/5509

###### Contents

  1. I Introduction
  2. II EFT of dynamical electromagnetism in general media
  3. III Chiral anomalous MHD
  4. IV Chiral plasma instability and a nonlinear model
  5. V Chiral magnetic wave in the strong field regime
  6. VI Discussions
  7. A Choices of $\Theta$ for dynamical KMS symmetry
  8. B Comparison with the results of Vardhan  _et al._ (2022)

##  I Introduction

Consider a system in which electromagnetic fields interact strongly with
charged matter at a finite temperature. The magnetohydrodynamic approximation
applies to the regime in which, in addition to conserved quantities, the only
other “slow” variables are magnetic fields111In fact, from a modern
perspective Gaiotto  _et al._ (2015); Grozdanov  _et al._ (2017), magnetic
fields can also be viewed as conserved charges associated with a one-form
symmetry.; MHD is a universal effective theory for these slow variables. MHD
is a powerful framework with applications to a wide range of disciplines
including nuclear physics, condensed matter physics, astrophysics, and
cosmology. In systems with chiral matter, the chiral symmetry can be broken by
the Adler-Bell-Jackiw (ABJ) anomaly, in which case the chiral charge density
is not conserved, and strictly speaking drops out of the hydrodynamic
description. However, if the anomaly coefficient is sufficiently small such
that the time and length scales responsible for the non-conservation of the
chiral symmetry is much larger than typical microscopic relaxation scales, we
can treat the chiral charge density as an approximately conserved quantity and
keep it in the formulation of MHD, resulting in a chiral anomalous MHD.

There is a large literature pertaining to macroscopic effects of quantum
anomalies on transports; with effects from t’ Hooft anomalies in which the
external fields are not dynamical, these effects are fairly well understood
(see Landsteiner (2016); Kharzeev  _et al._ (2016) for reviews). For ABJ
anomalies with dynamical electromagnetic fields, the story is more intricate,
see e.g. Golkar and Son (2015); Hou  _et al._ (2012); Jensen  _et al._ (2013);
Gorbar  _et al._ (2013); Figueroa and Shaposhnikov (2018); Figueroa  _et al._
(2019); Das  _et al._ (2022) for discussions. There have also been many
discussions of chiral anomalous MHD in the weak magnetic field regime Boyarsky
_et al._ (2015); Pavlović  _et al._ (2017); Giovannini (2016); Yamamoto
(2016); Rogachevskii  _et al._ (2017); Hattori  _et al._ (2019, 2022). In this
paper we present a systematic formulation of chiral anomalous MHD in the
strong field regime. We first present a formulation of EFTs for
electromagnetic fields in general media, and then specify to MHD. The
incorporation of the ABJ anomaly can be readily done in this approach. We will
outline the main idea and results, leaving technical details to a longer
exposition Landry and Liu (pear).

Note added: While this paper was nearing completion, we learned of Das  _et
al._ (pear), which approaches chiral magnetohydrodynamics using one-form
symmetry.

##  II EFT of dynamical electromagnetism in general media

Consider the electromagnetic vector field $a_{\mu}$ in a general medium at a
finite temperature $T_{0}=1/{\beta}_{0}$. We will work with the normal phase,
i.e. the $U(1)$ gauge symmetry is not spontaneously broken. For the moment,
let us assume that other than the $U(1)$ gauge symmetry, the system does not
have any other internal continuous global symmetry. We are interested in the
behavior of the electromagnetic field at distance and time scales much larger
than the typical relaxation scale, which we will denote as $\ell$.222For
notational simplicity, we use $\ell$ to also denote the relaxation time
assuming there is a typical velocity to convert one to the other. For this
purpose, we can imagine integrating out all matter fields to obtain an
effective action for $a_{\mu}$. We will ignore the motion of the medium and
temperature fluctuations, although they can be readily taken into account
using the method discussed below. Such a generalization will be presented
elsewhere. To formulate a real-time action we should use the Schwinger-Keldysh
(or closed time path) formalism Keldysh (1964) in which we have two copies of
$a_{\mu}$, written in terms of the so-called $r-a$ variables $a_{r\mu}$ and
$a_{a\mu}$ (below we will suppress $r$ in $a_{r\mu}$), with $a_{\mu}$ the
physical field and $a_{a\mu}$ the corresponding noise field.

The effective action $I_{\rm EFT}$ for the dynamical electromagnetic field can
be obtained using the framework developed in Crossley  _et al._ (2017);
Glorioso  _et al._ (2017); Glorioso and Liu (2018) as follows. We first
imagine the $U(1)$ symmetry is global, and $a_{\mu},a_{a\mu}$ are external
sources for two copies of the conserved current. The hydrodynamical variables
for the current conservation are two scalars $\phi,\phi_{a}$. The hydrodynamic
action $I_{\rm hydro}$ depends on $a_{\mu},a_{a\mu},\phi,\phi_{a}$ only
through the combinations $A_{\mu}=a_{\mu}+\partial_{\mu}\phi$ and $A_{\mu
a}=a_{a\mu}+\partial_{\mu}\phi_{a}$, i.e. $I_{\rm hydro}=I_{\rm
hydro}[A_{\mu},A_{\mu a}]$, and satisfies the following conditions:

  1. 1.

It enjoys translational invariance in $x^{\mu}=(t,x^{i})$ and rotationally
invariance in $x^{i}$ directions.

  2. 2.

There are various unitarity constraints (suppressing $\mu$ indices):

$\displaystyle I^{*}_{\rm EFT}[A,A_{a}]=-I_{\rm EFT}[A,-A_{a}],$ (1)
$\displaystyle{\rm Im}\,I_{\rm EFT}\geq 0,$ (2) $\displaystyle I_{\rm
EFT}[A,A_{a}=0]=0\ .$ (3)

  3. 3.

It is invariant under the time-independent local transformation

$A_{i}\to A_{i}^{\prime}=A_{i}-\partial_{i}\lambda(x^{i})\ ,$ (4)

which means that the action can depend on $A_{i}$ only through
$\partial_{t}A_{i}$ or $\vec{B}={\nabla}\times\vec{A}$.333$\vec{A}$ denotes
the spatial vector.

  4. 4.

It is invariant under dynamical KMS transformations, i.e. $I_{\rm
EFT}[A,A_{a}]=I_{\rm EFT}[\tilde{A},\tilde{A}_{a}]$ where dynamical KMS
transformations are $Z_{2}$ transformations of the form

$\tilde{A}_{\mu}(x)={\Theta}A_{\mu}(x),\quad\tilde{A}_{\mu
a}(x)={\Theta}A_{a\mu}(x)+i{\beta}_{0}{\Theta}\partial_{t}A_{\mu}(x)\ .$ (5)

Here ${\Theta}$ is a discrete symmetry of the underlying microscopic system
involving time reversal. The dynamical KMS symmetry as well as unitarity
constraints (1)–(3) ensure that the theory has an entropy current with non-
negative divergence Glorioso and Liu (2016).

  5. 5.

Impose whatever other discrete symmetries (formed from
${{\mathcal{C}}},{{\mathcal{P}}}$) preserved by the underlying microscopic
system.444If a system has some discrete internal symmetries, they should of
course also be imposed.

Now promote $a_{\mu},a_{a\mu}$ to be dynamical fields (i.e. gauging the $U(1)$
symmetry) by integrating over them in the path integrals. We can then absorb
$\phi,\phi_{a}$ into the redefinitions of them to obtain

$\int Da_{\mu}Da_{a\mu}D\phi D\phi_{a}\,e^{iI_{\rm
hydro}[A_{\mu},A_{a\mu}]}=C\int DA_{\mu}DA_{a\mu}e^{iI_{\rm
EFT}[A_{\mu},A_{a\mu}]}$ (6)

where $\phi,\phi_{a}$ integrals drop out, $C$ is an infinite constant, and

$I_{\rm EFT}[A_{\mu},A_{a\mu}]=I_{\rm hydro}[A_{\mu},A_{a\mu}]\ .$ (7)

$A_{\mu},A_{a\mu}$ are now dynamical fields which give the same field strength
as $a_{\mu},a_{a\mu}$. They are invariant under “gauge” transformations of
$a_{\mu},a_{a\mu}$ as any such transformations can be absorbed by shifts of
$\phi,\phi_{a}$, in analogue with the standard Higgs mechanism. But because of
(4), $A_{i}$ still transforms under a local symmetry. While (4), referred to
as the diagonal shift in Crossley  _et al._ (2017), has nothing to do with the
original gauge symmetry, it is as if there is some residual gauge symmetry
left. With a slight abuse of language, we will say that $a_{\mu}$ undergoes a
“partial Higgs mechanism” to become $A_{\mu}$.555In a superconducting phase,
where the $U(1)$ gauge symmetry is spontaneously broken, we drop the
requirement to be invariant under (4).

To formulate an action we also need to choose a derivative expansion scheme.
There are three possible regimes. The first is the standard dielectric regime
where $A_{\mu}$ is assigned weight $0$ and $\partial_{t},\partial_{i}$ weight
$1$, which applies to the regime of “weak” interactions of electromagnectic
field with insulators. The second is the MHD regime where $A_{i}$ is assigned
weight $-1$, $A_{0}$ weight $0$, $\partial_{i}$ weight $1$, $\partial_{t}$
weight $2$. Thus $B_{i}$ has weight zero and $E_{i}$ has weight $1$. This
weight assignment is motivated by the fact that in the MHD regime the dominant
behavior is magnetic diffusion (i.e. $\partial_{t}\sim\partial_{i}^{2}$) and
we would like to have arbitrary dependence on magnetic field $B_{i}$, which
then must have weight $0$. The weight assignments for $A_{a\mu}$ can be
deduced from those of $A_{\mu}$ by using (5). The last possible scheme is for
$A_{\mu}$ to have weight $-1$, and $\partial_{\mu}$ weight $1$, which leads to
both $E_{i}$ and $B_{i}$ having weight zero.666See Kovtun (2016) for a recent
discussion of thermodynamics of such systems. In this paper we focus on the
MHD regime, leaving the others to Landry and Liu (pear).

As an example, suppose the only preserved discrete symmetry of a system is
${\mathcal{T}}$, we then need to take ${\Theta}={\mathcal{T}}$. In the MHD
regime, up to weight 2 the Lagrangian density has the form

$\displaystyle{{\mathcal{L}}}_{{\mathcal{T}}}=b_{0}B_{i}A_{ai}+nA_{a0}-H_{i}B_{ai}+i(r^{-1})_{ij}A_{ai}(T_{0}A_{aj}+i\partial_{t}A_{j})$
(8)

where $n$ and $H$ can be obtained from a single function $F(A_{0},B^{2})$ as

$n=-{\partial F\over\partial A_{0}},\quad H_{i}={\partial F\over\partial
B_{i}}\equiv{B_{i}\over\mu},\quad B^{2}=\vec{B}^{2},\quad
h_{i}\equiv{B_{i}\over B}$ (9)

and $r^{-1}$ is the inverse of matrix $r_{ij}$ whose matrix elements can be
parameterized using rotational symmetry as

$r_{ij}=r_{\parallel}h_{i}h_{j}+r_{\perp}\Delta_{ij}+r_{\times}\epsilon_{ijk}h_{k},\quad\Delta_{ij}=\delta_{ij}-h_{i}h_{j}\
.$ (10)

All coefficients in $r_{ij}$ are functions of $A_{0}$ and $B^{2}$, while
$b_{0}$ is a constant. We may interpret $F$ as the equilibrium free energy
density, and $A_{0}$ as the (dynamical) chemical potential. Then $n$ can be
interpreted as the local charge density, and $H_{i}$ is the medium magnetic
field. From rotational symmetry $H_{i}$ must be proportional to $B_{i}$, and
$\mu(A_{0},B^{2})$, can be interpreted as the magnetic permeability. Equation
(2) implies that

$r_{\parallel},r_{\perp}\geq 0,$ (11)

while $r_{\times}$ can have either sign.

The equation of motion for $A_{a0}$ implies that $n=0$, i.e. the local charge
density vanishes, which in turn implies $A_{0}=f(B^{2})$, where $f$ is some
function. Then all other coefficients $\mu,r_{\parallel},r_{\perp},r_{\times}$
can be viewed as functions of $B^{2}$. With $b_{0}$ being a constant
independent of any dynamical variables777When we include dynamical
temperature, then $b_{0}$ will be replaced by ${b_{0}\over{\beta}}$ with
$b_{0}$ a constant. The term is still topological, as other than a universal
$1/{\beta}$, there is no dependence on physical condition of the system., the
$b_{0}$ term should be viewed as a topological term. Thermodynamic stability
also requires $\mu\geq 0$.

Note that nonzero $A_{0}=f(B^{2})$ and $r_{\times}$ are consequences of charge
conjugation violation, while a nonzero $b_{0}$ is a consequence parity
violation.888At the level of the action, we can have $r_{\times}\neq 0$ appear
even when ${{\mathcal{C}}}$ symmetry is respected; the requirement is merely
that it be an odd function of $A_{0}$. But then $A_{0}=0$ on shell if
${{\mathcal{C}}}$ symmetry is respected, so we ultimately find that
$r_{\times}=0$ on the equations of motion. If we consider instead a system
where the only preserved discrete symmetry is
${\Theta}={{\mathcal{C}}}{{\mathcal{P}}}{\mathcal{T}}$, then we find that all
of them vanish.

The equations of motion for $A_{ai}$ give

$\displaystyle
E_{i}=\partial_{i}A_{0}-\partial_{0}A_{i}=r_{ij}(\epsilon_{jkl}\partial_{k}H_{l}-b_{0}B_{j})+\partial_{i}f,$
(12)

which is the constitutive relation of the electric field in terms of the
magnetic field. In the weak magnetic field limit, we can expand
$r_{\parallel}(B^{2}=0)=r_{\perp}(B^{2}=0)=r$. Equation (12) can be written to
linear order in $B$ as

$\vec{E}={r\over\mu}{\nabla}\times\vec{B}+b_{0}r\vec{B},\quad\vec{B}\to 0$
(13)

which then identifies $r={1\over\sigma}$ with $\sigma$ the electrical
conductivity. We stress that for general $\vec{B}$,
$r_{\parallel},r_{\perp},r_{\times}$ are new transport coefficients and are
not related to conductivity.

MHD has recently been reformulated using the one-form symmetry Grozdanov  _et
al._ (2017) (see Hernandez and Kovtun (2017); Grozdanov and Poovuttikul
(2019); Glorioso and Son (2018); Armas and Jain (2020); Gralla and Iqbal
(2019); Benenowski and Poovuttikul (2019); Landry (2021); Armas and Camilloni
(2022); Das  _et al._ (2022); Vardhan  _et al._ (2022) for various
developments). In particular, the constitutive relation of the electric field
that violates charge and parity conjugation in the strong magnetic field
regime has recently been written down in Vardhan  _et al._ (2022),
generalizing the previous formulation of Goldreich and Reisenegger (1992). It
can be checked that (12) is exactly equivalent to the expression in Vardhan
_et al._ (2022) except that the $b_{0}$ term was missing there. See Appendix
B. It can be shown that the two formulations are related by a duality
transformation, and the reason that the $b_{0}$ term is absent in Vardhan  _et
al._ (2022) is due to the fact that the term becomes non-local in the one-form
symmetry formulation. See Landry and Liu (pear) for details.

The same approach can be applied to gravity to derive effective field theories
of dynamical gravity strongly interacting with matter fields, and will be
pursued elsewhere.

##  III Chiral anomalous MHD

We now generalize the above discussion to including chiral matter, in which
case there is a chiral current $J_{\mu}^{5}$ satisfying the equation

$\partial_{\mu}J_{5}^{\mu}=\frac{c}{4}\epsilon^{\mu\nu\alpha\beta}F_{\mu\nu}F_{\alpha\beta}\
.$ (14)

where $F=dA$. As mentioned at the beginning, we will assume the anomaly
coefficient $c$ is sufficiently small that the non-conservation due to (14)
happens at much longer scales than the microscopic relaxation scale.

Denote ${{\mathcal{A}}}_{\mu},{{\mathcal{A}}}_{a\mu}$ and
$\varphi,\varphi_{a}$ respectively as external sources and hydrodynamic
degrees of freedom for $J^{\mu}_{5}$. We again suppressed subscript $r$ in
${{\mathcal{A}}}_{\mu}$ and $\varphi$. The EFT action should have the
structure Glorioso  _et al._ (2019)

$I_{\text{EFT}}=I_{\text{anom}}+I_{\text{inv}},$ (15)

where $I_{\text{inv}}=I_{\text{inv}}[A_{\mu},A_{a\mu};C_{\mu},C_{a\mu}]$,
depending on $\varphi,\varphi_{a}$ only through the combinations

$C_{\mu}={{\mathcal{A}}}_{\mu}+\partial_{\mu}\varphi,\quad C_{\mu
a}={{\mathcal{A}}}_{\mu a}+\partial_{\mu}\varphi_{a},$ (16)

and $I_{\text{anom}}$ has the form Glorioso  _et al._ (2019)

$\displaystyle I_{\text{anom}}$ $\displaystyle=$ $\displaystyle
c\int(\varphi_{a}F\wedge F+2\varphi F\wedge F_{a})\ .$ (17)

$I_{\text{anom}}$ is designed to capture the anomaly (14), and indeed it can
be readily checked that equations of motion for $\varphi,\varphi_{a}$ give two
copies of (14), one for each leg of the S-K contour. In addition to the
conditions mentioned earlier, $I_{\rm EFT}$ should now also be invariant under
the shift Glorioso  _et al._ (2019); Glorioso and Liu (2018)

$\varphi\to\varphi+\Lambda(\vec{x}),$ (18)

and the dynamical KMS transformation should now also include the
transformation of $C_{\mu},C_{a\mu}$

$\tilde{C}_{\mu}(x)={\Theta}C_{\mu}(x),\quad\tilde{C}_{a\mu}(x)={\Theta}C_{a\mu}(x)+i{\beta}_{0}{\Theta}\partial_{t}C_{\mu}(x)\
.$ (19)

Under ${{\mathcal{C}}},{{\mathcal{P}}},{\mathcal{T}}$ transformations,
$C_{\mu},C_{a\mu}$ transform the same as $J_{5}^{\mu}$, i.e.

${{\mathcal{C}}}:C_{\mu}\to
C_{\mu},\quad{{\mathcal{P}}}:C_{\mu}\to(-C_{0},C_{i}),\quad{\mathcal{T}}:C_{\mu}\to(C_{0},-C_{i})\
.$ (20)

The weight assignments are that
$C_{0}={{\mathcal{A}}}_{0}+\partial_{t}\varphi\equiv\mu_{5}$ has weight $0$,
while $\vec{{\mathcal{A}}},C_{a0},\vec{C}_{a}$ respectively have weight
$-1,2,1$. Here $\mu_{5}$ is the local chemical potential for the chiral charge
density. Note that using the one-form symmetry formulation to describe the
anomaly equation (14) is challenging as the right hand side is then expressed
in terms of the two-form current which itself has to be determined
dynamically.

In the MHD regime the Lagrangian density up to weight $2$ has the form (to
linear order in external sources ${{\mathcal{A}}}_{a\mu}$) Landry and Liu
(pear)

$\begin{split}{\pazocal
L}=nA_{a0}+n_{5}\partial_{t}\varphi_{a}-H_{i}B_{ai}+2c\partial_{t}\vec{A}\cdot\vec{B}\varphi_{a}-(2c\mu_{5}-b_{00})B_{i}A_{ai}\\\
+i(r^{-1})_{ij}A_{ai}(T_{0}A_{aj}+i\partial_{t}A_{j})+i\kappa_{ij}\partial_{i}\varphi_{a}(T_{0}\partial_{j}\varphi_{a}+i\partial_{j}\mu_{5})\\\
+i\lambda_{ij}((T_{0}\partial_{i}\varphi_{a}+i\partial_{i}\mu_{5})A_{aj}+\partial_{i}\varphi_{a}(T_{0}A_{aj}+i\partial_{t}A_{j}))\\\
+\gamma_{ij}(\partial_{i}\varphi_{a}\partial_{t}A_{j}-\partial_{i}\mu_{5}A_{aj})+{{\mathcal{A}}}_{a0}J^{0}_{5}+{{\mathcal{A}}}_{ai}J^{i}_{5}\end{split}$
(21)

where $r_{ij}$ has the same decomposition as (10), and

$\displaystyle n=-{\partial F\over\partial A_{0}},\quad n_{5}=-\frac{\partial
F}{\partial\mu_{5}},\quad H_{i}={\partial F\over\partial
B_{i}}\equiv{B_{i}\over\mu},$ (22)
$\displaystyle\kappa_{ij}=\kappa_{\parallel}h_{i}h_{j}+\kappa_{\perp}\Delta_{ij}+\kappa_{\times}\epsilon_{ijk}h_{k},\quad\lambda_{ij}=\lambda_{\parallel}h_{i}h_{j}+\lambda_{\perp}\Delta_{ij}+\lambda_{\times}\epsilon_{ijk}h_{k},$
(23)
$\displaystyle\gamma_{ij}=\gamma_{\parallel}h_{i}h_{j}+\gamma_{\perp}\Delta_{ij}+\gamma_{\times}\epsilon_{ijk}h_{k},\quad\lambda_{ij}^{\pm}\equiv\lambda_{ij}\pm\gamma_{ij}$
(24) $\displaystyle J^{0}_{5}=n_{5},\quad
J^{i}_{5}=(b_{50}-2cA_{0})B_{i}-\kappa_{ij}\partial_{j}\mu_{5}-\lambda^{-}_{ij}\partial_{t}A_{j}+\epsilon_{ijk}\partial_{j}(mB_{k}).$
(25)

All coefficients should be understood as functions of $A_{0},\mu_{5},B^{2}$
except that $b_{00},b_{50}$ are constants. Depending on the choice of
${\Theta}$, some coefficients may vanish or have constraints. For example, for
${\Theta}={\mathcal{T}}$, we have
$\lambda_{\times}=\gamma_{\parallel,\perp}=0$. For
$\Theta={{\mathcal{C}}}{{\mathcal{P}}}{\mathcal{T}}$, we have that
$b_{00}=b_{50}=0$ and the coefficients
$F,r_{\parallel,\perp},\kappa_{\parallel,\perp},\lambda_{\parallel,\perp,\times},m$
are even under $A_{0}\to-A_{0},\mu_{5}\to-\mu_{5}$, while
$r_{\times},\kappa_{\times},\gamma_{\parallel,\perp,\times}$ are odd under
$A_{0}\to-A_{0},\mu_{5}\to-\mu_{5}$. In Appendix A we give the full set of
constraints for all choices of ${\Theta}$.

The equation of motion for $A_{a0}$ again gives $n=0$, which can be used to
solve for $A_{0}$ in terms of $\mu_{5},B^{2}$. Then all scalar coefficients,
except for $b_{00}$ and $b_{50}$, which are constants, should be understood as
functions of $\mu_{5}$ and $B^{2}$. Again terms involving $b_{00}$ and
$b_{50}$ should be topological.

After solving for $A_{0}$, we find that $n_{5}$ and $H_{i}$ satisfy the
integrability condition

$\partial_{\mu_{5}}H_{i}=-\partial_{B_{i}}n_{5}\ .$ (26)

Equation (25) gives the constitutive relations for $J^{\mu}_{5}$. From (25),
we see that $n_{5}$ is the chiral charge density and $\kappa_{ij}$ is the
chiral conductivity tensor. The term in $J_{5}^{i}$ proportional to $c$ gives
the Chiral Separation Effect (CSE) Son and Zhitnitsky (2004); Metlitski and
Zhitnitsky (2005) (see also Jensen  _et al._ (2013); Gorbar  _et al._ (2013)
for discussion with dynamical fields) in the strong field regime. Note that
$A_{0}$ there is in general a nontrivial function of $\mu_{5}$ and $B^{2}$,
which can be found from the equilibrium free energy density
$F(A_{0},\mu_{5},B^{2})$. The $b_{50}$ term is new, as mentioned earlier; it
vanishes in a ${{\mathcal{C}}}{{\mathcal{P}}}{\mathcal{T}}$ invariant system.
The $\lambda^{-}_{ij}$ term describes the relation between the chiral current
and the dynamical part of the electric field, generalizing the Chiral Electric
Separation Effect (CESE) Huang and Liao (2013); Jiang  _et al._ (2015).
$\lambda_{ij}^{-}$ may be called the CESE conductivity tensor Kharzeev and Yee
(2011). The last term in (25) is proportional to the curl of $m\vec{B}$. It
does not contribute to the conservation equation of $J_{5}^{\mu}$, but does
affect the current itself. It is analogous to the Ampère term for the electric
current, but here with the chiral current. $1/m$ may be thus interpreted as
the chiral magnetic permeability, and the term may be referred to as the
chiral Ampère effect.

From (2) we find that various transport coefficients satisfy the inequalities

$r_{\parallel}>0,\quad
r_{\perp}>0\quad\kappa_{\parallel}\geq\lambda_{\parallel}^{2}r_{\parallel}>0,\quad\kappa_{\perp}>0,\quad\kappa_{\perp}r_{\perp}\geq(\lambda_{\perp}^{2}+\lambda_{\times}^{2})(r_{\perp}^{2}+r_{\times}^{2}),$
(27)

which hold for all choices of ${\Theta}$. For our discussion below it is also
convenient to introduce

$\chi_{5}\equiv{\partial n_{5}\over\partial\mu_{5}},\quad{\partial
n_{5}\over\partial
B_{i}}=-\partial_{\mu_{5}}H_{i}={B_{i}\over\mu^{2}}{\partial\mu\over\partial\mu_{5}}\equiv\eta_{5}B_{i}$
(28)

where $\chi_{5}$ can be identified as the chiral susceptibility, and
$\eta_{5}$ chiral magnetic susceptibility. Thermodynamic stability of the
system also implies that

$\chi_{5}\geq 0,\quad\mu\geq 0,\quad\mu-B\partial_{B}\mu\geq 0,$ (29)

while $\eta_{5}$ can have either sign.

By design, the equation of motion obtained from varying $\varphi_{a}$ gives
(14), while as in last section, the equations of motion obtained from varying
$A_{ai}$ give the constitutive relations of the electric field. More
explicitly, they have the form

$\displaystyle\chi_{5}\partial_{t}\hat{\mu}_{5}+\eta_{5}B_{i}\partial_{t}B_{i}-\partial_{i}(\kappa_{ij}\partial_{j}\hat{\mu}_{5})-\partial_{i}(\lambda^{-}_{ij}\partial_{t}A_{j})=2cB_{i}\partial_{t}A_{i},$
(30)
$\displaystyle\partial_{t}A_{i}=-r_{ij}(\epsilon_{jkl}\partial_{k}H_{l}+2c\hat{\mu}_{5}B_{j}+\lambda^{+}_{kj}\partial_{k}\hat{\mu}_{5}),\quad\hat{\mu}_{5}\equiv\mu_{5}-{b_{00}\over
2c},$ (31)

The term proportional to $c$ in (31) is the dynamical field generalization of
the chiral magnetic effect Vilenkin (1980); Kharzeev  _et al._ (2008);
Fukushima  _et al._ (2008) (see e.g. Kharzeev (2014); Kharzeev and Yee (2011);
Landsteiner (2016) for reviews).

After some further manipulations we can write the above equations as

$\displaystyle\chi_{5}\partial_{t}\hat{\mu}_{5}+(2cB)^{2}r_{\parallel}\hat{\mu}_{5}+2cr_{\parallel}B_{i}\epsilon_{ijk}\partial_{j}H_{k}+\eta_{5}B_{i}\partial_{t}B_{i}+2cB_{i}(\partial_{i}(\hat{\mu}_{5}\Lambda^{-}_{\parallel})+\Lambda^{+}_{\parallel}\partial_{i}\hat{\mu}_{5})={{\mathcal{G}}},$
(32)
$\displaystyle\partial_{t}B_{i}=-2c\epsilon_{ijk}\partial_{j}(r_{\parallel}\hat{\mu}_{5}B_{k})+{{\mathcal{H}}}_{i},\qquad\Lambda^{\pm}_{\parallel}=r_{\parallel}(\lambda_{\parallel}\pm{{\gamma}}_{\parallel})\
.$ (33)

where ${{\mathcal{G}}}$ and ${{\mathcal{H}}}_{i}$ denote the second derivative
terms whose forms are

$\displaystyle{{\mathcal{H}}}_{i}=-\epsilon_{ijk}\partial_{j}[r_{kl}(\vec{\nabla}\times\vec{H})_{l}+(r\lambda^{+T})_{kl}\partial_{l}\hat{\mu}^{5}),$
(34)
$\displaystyle{{\mathcal{G}}}=\partial_{i}(\kappa_{ij}\partial_{j}\hat{\mu}_{5})-\partial_{i}[(\lambda^{-}r)_{ij}(\vec{\nabla}\times\vec{H})_{j}+(\lambda^{-}r\lambda^{+T})_{ij}\partial_{j}\hat{\mu}^{5}],$
(35)

with $\lambda^{+T}$ denoting the transpose of $\lambda^{+}$. We see that all
the leading spatial derivative terms in the above equation come from the
anomaly. Note that there can be additional second derivative terms that are
proportional to $c$, which are neglected. Since we assume $c$ is small, such
terms are smaller than the anomaly-independent second derivative terms (which
give rise to diffusion of chiral charge density and $B_{i}$) included here.

Now consider homogeneous equilibrium solutions999In next section we will find
more inhomogenous equilibrium solutions. to (32)–(33). We see there are two
branches

$\displaystyle\hat{\mu}_{5}={\rm const},\quad B_{i}=0,$ (36)
$\displaystyle\hat{\mu}_{5}=0,\quad B_{i}={\rm const},$ (37)

which are connected at $\hat{\mu}_{5}=B_{i}=0$. The consequence of having
$b_{00}\neq 0$ is that in (37), there is a nonzero equilibrium value
$\mu_{5}^{\rm eq}={b_{00}\over 2c}$.

Note that $\hat{\mu}_{5}\neq 0,B_{i}\neq 0$ is not an equilibrium
configuration due to effects of the anomaly. In fact it is well known that in
the presence of a nonzero $B_{i}$, a nonzero $\hat{\mu}_{5}$ will decay (see
e.g. Das  _et al._ (2022) for recent discussions). Now suppose we initially
have a small nonzero $\hat{\mu}_{5}^{0}$; we then have

$\hat{\mu}_{5}=\hat{\mu}_{5}^{0}e^{-{{\Gamma}}t},\quad{{\Gamma}}={(2cB)^{2}r_{\parallel}\over\chi_{5}}\biggr{|}_{\hat{\mu}^{5}=0}\
,$ (38)

where both $\chi_{5}$ and $r_{\parallel}$ are functions of $B$.101010As
stressed earlier, $r_{\parallel}$ is not the same as the inverse of
conductivity in the presence of a nonzero $B$. For a finite nonzero
$\hat{\mu}^{5}_{0}$, its precise evolution depends on the explicit
$\hat{\mu}_{5}$-dependence of $\chi_{5}$ and $r_{\parallel}$, and need not be
exponential.

##  IV Chiral plasma instability and a nonlinear model

We now consider small perturbations around the equilibrium configuration (36).
We denote the perturbation of $\hat{\mu}_{5}$ as $\nu_{5}$ and the background
value simply as $\hat{\mu}_{5}$. We can expand all quantities in small $B$; at
leading order

$r_{ij}=r{{\delta}}_{ij},\;\kappa_{ij}=\kappa{{\delta}}_{ij},\;\lambda_{ij}=\lambda{{\delta}}_{ij},\;{{\gamma}}_{ij}={{\gamma}}{{\delta}}_{ij},$
(39)

where $r,\kappa,\lambda,{{\gamma}},\mu,\eta_{5}$ are all functions of
$\hat{\mu}_{5}$.

At linearized level equations (32)–(33) become decoupled equations

$\displaystyle\chi_{5}\partial_{t}\nu_{5}-\tilde{\kappa}{\nabla}^{2}\nu_{5}=0,\quad\tilde{\kappa}=\kappa-r(\lambda^{2}-{{\gamma}}^{2})>0,$
(40)
$\displaystyle\partial_{t}B_{i}=-2cr\hat{\mu}_{5}\epsilon_{ijk}\partial_{j}B_{k}+D_{B}{\nabla}^{2}B_{i},\quad
D_{B}={r\over\mu},$ (41)

where (40) describes diffusion of chiral charge with diffusion constant
$D_{5}={\tilde{\kappa}\over\chi_{5}}$. Equation (41) can be readily solved by
performing a Fourier transform in spatial directions. For sufficiently small
$k$, there is an unstable mode growing exponentially with time, which has been
termed the chiral plasma instability Akamatsu and Yamamoto (2013) (see also
Kaplan  _et al._ (2017); Hattori  _et al._ (2019)). Taking momentum
${\vec{k}}=ke_{z}$ to be along $z$ direction, it can be written as

$B_{x}={{\mathcal{B}}}(t)\cos kz,\quad B_{y}={{\mathcal{B}}}(t)\sin
kz,\quad{{\mathcal{B}}}(t)=B_{0}e^{2cr\hat{\mu}_{5}kt-D_{B}k^{2}t}\ .$ (42)

The exponential in ${{\mathcal{B}}}(t)$ becomes decaying for
$k>2c\hat{\mu}_{5}\mu$ and the exponential growth is maximal for
$k=c\hat{\mu}_{5}\mu$.111111One may worry that the derivative expansion breaks
down when we equate linear and quadratic $k$ terms in the exponential of
${{\mathcal{B}}}(t)$. This is ok as ${{\gamma}}\propto c$ is a small
parameter. Suppose the coefficients of $k^{2}$ and $k^{3}$ terms are
controlled by the same scale $\ell$. The cubic and higher terms in $k$ are
suppressed provided that $c\hat{\mu}_{5}\mu\ell\ll 1$.

To understand the fate of the instability, we need to consider the full
nonlinear equations. Here we consider a simple model where all transport
coefficients are isotropic, as in (39), and all the parameters
$r,\kappa,\lambda,{{\gamma}},\mu,\eta_{5}$ are independent of $\mu_{5}$ and
$B$. In this case equations (32)–(33) become

$\displaystyle\chi_{5}\partial_{t}\hat{\mu}_{5}+(2cB)^{2}r\hat{\mu}_{5}+\eta_{5}B_{i}\partial_{t}B_{i}-\tilde{\kappa}{\nabla}^{2}\hat{\mu}_{5}+2cD_{B}\epsilon_{ikl}B_{i}\partial_{k}B_{l}+4cr\lambda
B_{i}\partial_{i}\hat{\mu}_{5}=0,$ (43)
$\displaystyle\partial_{t}B_{i}=-2cr\epsilon_{ijk}\partial_{j}(\hat{\mu}_{5}B_{k})+D_{B}{\nabla}^{2}B_{i}\
.$ (44)

Now consider a helical magnetic field ansatz of the form

$\vec{B}={{\mathcal{B}}}(t)(\cos kz,\sin
kz,0),\quad\hat{\mu}_{5}=\hat{\mu}_{5}(t),\quad{\nabla}\times B=-k\vec{B};$
(45)

i.e. $\hat{\mu}_{5}$ has no spatial dependence. Equations (43)–(44) then
reduce to

$\displaystyle\chi_{5}\partial_{t}\hat{\mu}_{5}=-\left(2c+\eta_{5}k\right)(2cr\hat{\mu}_{5}-D_{B}k){{\mathcal{B}}}^{2},$
(46)
$\displaystyle\partial_{t}{{\mathcal{B}}}=k(2cr\hat{\mu}_{5}-D_{B}k){{\mathcal{B}}}\
.$ (47)

We immediately see that the above equation has a static state

$\hat{\mu}_{5}={D_{B}k\over 2cr}={k\over 2c\mu}\equiv\hat{\mu}_{5}^{\rm eq}$
(48)

for any value of ${{\mathcal{B}}}$.

Now let us consider what happens if we have some initial condition

$\hat{\mu}_{5}(t=0)=\hat{\mu}_{5}^{0},\quad{{\mathcal{B}}}(t=0)=B_{0}\ .$ (49)

Note that for ${k\over 2c\mu}>\hat{\mu}_{5}^{0}$, ${{\mathcal{B}}}(t)$ decays
with time to zero and linearized analysis is enough. We will thus restrict to
those $k$ such that ${k\over 2c\mu}<\hat{\mu}_{5}^{0}$. The qualitative
behavior of the evolution can be deduced from (46)–(47) without solving them
explicitly. Suppose $2c+\eta_{5}k$ is positive, then the right hand sides of
(46) and (47) are respectively initially negative and positive. So
$\hat{\mu}_{5}$ will decrease with time and ${{\mathcal{B}}}(t)$ will increase
with time, until $\hat{\mu}_{5}$ reaches the equilibrium value (48) at which
point the right hand sides of both equations become zero and the system
reaches a steady state. The final state will be
$\hat{\mu}_{5}=\hat{\mu}_{5}^{\rm eq}$, and ${{\mathcal{B}}}(t)$ is some
finite constant. If $2c+\eta_{5}k$ is negative 121212$\eta_{5}$ can in
principle have either sign., then for both equations the right hand sides are
positive, and $\hat{\mu}_{5}$ and ${{\mathcal{B}}}$ grow with time
indefinitely. The system then appears to have a nonlinear instability. Of
course, this may resulting from the fact that our model is too crude. Non-
isotropic effects resulting from magnetic fields and dependence of various
transport coefficients on $\mu_{5},B$ will also be important at some point.
Furthermore, when the magnetic field is large enough, temperature fluctuations
and velocity effects could also be important.

Equations (46)–(47) can actually be solved exactly to give

$\displaystyle\hat{\mu}_{5}(t)=\tilde{\mu}_{5}^{0}{a_{k}B_{0}^{2}+bk\over
bk+a_{k}B_{0}^{2}e^{(a_{k}B_{0}^{2}+bk)t}}+\hat{\mu}_{5}^{\rm eq},$ (50)
$\displaystyle{{\mathcal{B}}}(t)=B_{0}{a_{k}B_{0}^{2}+bk\over
a_{k}B_{0}^{2}+bke^{-(a_{k}B_{0}^{2}+bk)t}},$ (51) $\displaystyle
a_{k}\equiv{2cr\over\chi_{5}}\left(2c+\eta_{5}k\right),\quad
b=4cr\tilde{\mu}_{5}^{0},\quad\tilde{\mu}_{5}^{0}\equiv\hat{\mu}_{5}^{0}-\hat{\mu}_{5}^{\rm
eq}\ .$ (52)

For $a_{k}>0$, the final value of ${{\mathcal{B}}}$ is given by

${{\mathcal{B}}}(\infty)=B_{0}+{bk\over
a_{k}B_{0}}=B_{0}+{2\chi_{5}k\tilde{\mu}_{5}^{0}\over\left(2c+\eta_{5}k\right)B_{0}}\
.$ (53)

It is unintuitive that the smaller the initial value $B_{0}$, the larger
${{\mathcal{B}}}(\infty)$, i.e. $B_{0}\to 0$ limit is not smooth.131313This
behavior has to do with the ${{\mathcal{B}}}^{2}$-dependence on the right hand
side of (46). We again expect this feature should not be there when
anisotropic effects and dependence on $\mu_{5},B$ are included.

For ${\Theta}={{\mathcal{C}}}{{\mathcal{P}}}{\mathcal{T}}$, we have
$b_{00}=0$, meaning that $\hat{\mu}^{5}=\mu^{5}$ and $n_{5}$—and by extension
$\eta_{5}$—is an odd function of $\mu_{5}$. So treating $\eta_{5}$ as a
constant is inconsistent. Nevertheless the qualitative discussion below (49)
still applies.

The conclusion we reached here for $2c+\eta_{5}k>0$ is qualitatively similar
to an earlier discussion in Kaplan  _et al._ (2017), although their nonlinear
equations are different. The quantitative conclusions are thus different. For
example, Kaplan  _et al._ (2017) predicts a final value for the magnitude of
the helical mode that is independent of initial conditions.

##  V Chiral magnetic wave in the strong field regime

Now let us consider small perturbations around the branch (37). We take the
equilibrium magnetic field to be $\vec{B}=B_{0}e_{z}$, and denote its
perturbation as $\vec{b}$, i.e. $\vec{B}=B_{0}e_{z}+\vec{b}$. To linear order
in $\hat{\mu}_{5}$ and $\vec{b}$, equations for (32) and (33) become

$\displaystyle\chi_{5}\partial_{t}\hat{\mu}_{5}+(2cB_{0})^{2}r_{\parallel}\hat{\mu}_{5}+4cB_{0}r_{\parallel}\lambda_{\parallel}\partial_{z}\hat{\mu}_{5}+2cr_{\parallel}H_{0}\epsilon_{ab}\partial_{a}b_{b}-{{\mathcal{D}}}_{ij{\alpha}}\partial_{i}\partial_{j}\hat{b}_{\alpha}=0,$
(54)
$\displaystyle\partial_{t}b_{i}=-2cB_{0}r_{\parallel}{{\delta}}_{i}^{a}\epsilon_{ab}\partial_{b}\hat{\mu}_{5}-{{\mathcal{E}}}_{ijk{\alpha}}\partial_{j}\partial_{k}\hat{b}_{\alpha},\quad
H_{0}=-{\partial F\over\partial
B_{0}},\quad\hat{b}_{\alpha}=(\hat{\mu}_{5},b_{i}),\quad a,b=x,y\ .$ (55)

The explicit expressions of ${{\mathcal{D}}}_{ij{\alpha}}$ and
${{\mathcal{E}}}_{ijk{\alpha}}$ are a bit long, which we will not need below.
They are independent of anomaly coefficient $c$ and describe coupled
diffusions of chiral charge density and $b_{i}$. Going to Fourier space for
both $t$ and ${\vec{x}}$ we find the equation of $\hat{\mu}_{5}$ gives the
dispersion relation

$\omega=-i{{\Gamma}}+v_{z}k_{z}+O(k^{2}_{z,\perp}),\quad{{\Gamma}}\equiv{(2cB_{0})^{2}r_{\parallel}\over\chi_{5}},\quad
v_{z}\equiv{4cB_{0}r_{\parallel}\lambda_{\parallel}\over\chi_{5}}.$ (56)

Denote the initial value of $\hat{\mu}_{5}$ (i.e. at $t=0$) by
$f_{0}({\vec{x}})$; we then find

$\displaystyle\hat{\mu}_{5}({\vec{x}},t)=f_{0}(\vec{x}_{\perp},z-v_{z}t)e^{-{{\Gamma}}t}\
.$ (57)

We thus find, in addition to an overall exponential decay, along the
$z$-direction there is a propagating wave with velocity $v_{z}$. As
$r_{\parallel}$ is non-negative, the sign of $v_{z}$ is determined from that
of $\lambda_{\parallel}$. There is a propagating wave only along the positive
$z$ or negative $z$ direction depending on the sign of $v_{z}$; i.e. the
propagation is chiral. This is precisely the dynamical version of the chiral
magnetic wave discussed in Kharzeev and Yee (2011) with non-dynamical external
fields.141414An earlier discussion of chiral MHD Hattori  _et al._ (2019)
appears not to find it. The last equation of (56) gives its velocity.
Including $O(k^{2})$ terms, there are also diffusion in $x,y$ directions, and
$k$-dependent attenuation of the chiral magnetic wave.

Now consider (55). At leading order in spatial derivatives, the contribution
to $\partial_{t}b_{z}$ comes from $O(k^{2})$ terms, and the dominant behavior
should be diffusion. For $b_{a}$ it is determined by the spatial derivative of
$\hat{\mu}_{5}$; we thus find that

$\partial_{t}b_{a}=-2cB_{0}r_{\parallel}\epsilon_{ab}\partial_{b}f_{0}(\vec{x}_{\perp},z-v_{z}t)e^{-{{\Gamma}}t},$
(58)

which gives

$b_{a}(t,{\vec{x}})=b_{a}^{0}({\vec{x}})-2cB_{0}r_{\parallel}\epsilon_{ab}\partial_{b}\left(g_{0}(\vec{x}_{\perp},z-v_{z}t)e^{-{{\Gamma}}t}-g_{0}({\vec{x}})\right),$
(59)

where $g_{0}({\vec{x}})$ is a function which can be obtained from $f_{0}$ by
an integral transform. Thus $b_{a}(t,{\vec{x}})$ also exhibit a decaying
chiral magnetic wave. When the second derivative terms are included, there are
again diffusions under which the initial perturbation $b_{a}^{0}({\vec{x}})$
will evolve to zero.

##  VI Discussions

We have given a general formulation of EFTs for electromagnetic fields in
general media. The specific form of the action depends on the choice of a
derivative expansion scheme. Here we focused on the regime for MHD, and
restrict to the case where flow velocities and temperature variations can be
neglected. We then considered the situation with chiral charged matter and ABJ
anomaly. The resulting theory gives dynamical field generalization of the
chiral magnetic effect, the chiral separation effect, and the chiral electric
separation effect. Linearized around the equilibrium configuration of a
constant magnetic field reveals the presence of the chiral magnetic wave and
predicts its wave velocity. We also present a simple full nonlinear model for
the chiral plasma instability.

Understanding possible field-dependence of various transport parameters in
realistic applications (e.g. heavy ion collisions, Dirac and Weyl semimetals,
electroweak plasmas and neutron stars) is clearly of crucial importance. It
would be instructive to find toy models in which such field-dependences can be
explicitly worked out, and study the physical implications of such dependence
on the evolution of the MHD equations. Holographic systems should provide a
good such laboratory.

###### Acknowledgements.

We would like to thank Paolo Glorioso and Nabil Iqbal for their
correspondence. This work is supported by the Office of High Energy Physics of
U.S. Department of Energy under grant Contract Number DE-SC0012567 and the
ALFA foundation.

##  Appendix A Choices of $\Theta$ for dynamical KMS symmetry

Consider the most general Lagrangian for chiral anomalous plasma up to weight-
two in the derivative expansion such that under a dynamical KMS transformation
(irrespective of choice of $\Theta$) all terms that do not involve $a$-type
fields are total derivatives,151515For simplicity, we work to linear order in
${{\mathcal{B}}}_{ai}$ and include the minimal factors of
${{\mathcal{B}}}_{i}$ necessary to ensure dynamical KMS symmetry is satisfied.
Higher-order terms in ${{\mathcal{B}}}_{i}$ may be important for correlation
functions, but they alter neither the equations of motion nor the constitutive
relation for $J^{\mu}_{5}$ and thus fall outside the scope of this work.

$\begin{split}{{\mathcal{L}}}=\hat{{\mathcal{L}}}_{\text{anom}}+b_{00}B_{i}A_{ai}+b_{50}({{\mathcal{B}}}_{i}A_{ai}+B_{i}C_{ai})+nA_{a0}+n^{5}C_{a0}-H^{i}B_{ai}\\\
+i(r^{-1})_{ij}A_{ai}(T_{0}A_{aj}+i\partial_{t}A_{j})+i\kappa_{ij}\partial_{i}\varphi_{a}(T_{0}\partial_{j}\varphi_{a}+i\partial_{j}\mu_{5})\\\
+i\lambda_{ij}((T_{0}\partial_{i}\varphi_{a}+i\partial_{i}\mu_{5})A_{aj}+\partial_{i}\varphi_{a}(T_{0}A_{aj}+i\partial_{t}A_{j}))\\\
+\gamma_{ij}(\partial_{i}\varphi_{a}\partial_{t}A_{j}-\partial_{i}\mu_{5}A_{aj})+m(B_{i}{{\mathcal{B}}}_{ai}+B_{ai}{{\mathcal{B}}}_{i})+m_{a}B_{i}{{\mathcal{B}}}_{i}\
,\end{split}$ (60)

for which $b_{00},b_{05}$ are constant, (22) is satisfied and

$\displaystyle m_{a}={\partial m\over\partial A_{0}}A_{a0}+{\partial
m\over\partial C_{0}}C_{a0}+{\partial m\over\partial B_{i}}B_{ai},$ (61)
$\displaystyle\hat{{\mathcal{L}}}_{\text{anom}}=-2c(-\partial_{t}\vec{A}\cdot\vec{B}\varphi_{a}+(\dot{\varphi}\vec{B}-\partial_{t}\vec{A}\times\vec{{\mathcal{A}}})\cdot\vec{A}_{a}+A_{0}(\vec{{\mathcal{A}}}\cdot\vec{B}_{a}+\vec{B}\cdot\vec{{\mathcal{A}}}_{a})+\vec{B}\cdot\vec{{\mathcal{A}}}A_{a0})\
.$ (62)

We have defined $\hat{{\mathcal{L}}}_{\text{anom}}$ to be the minimal set of
terms containing the anomaly that is invariant under all symmetries; it has no
free parameters and is thus independent of our choice of ${\Theta}$.

The form of this leading-order Lagrangian is necessary, but not sufficient to
ensure that dynamical KMS symmetry is imposed. We will now investigate how the
various choices of ${\Theta}$ affect the action; for each choice, we must
impose certain additional constraints as follows.

  1. 1.

${\Theta}={\mathcal{T}}$: The only constraints are
$\lambda_{\times}=\gamma_{\parallel,\perp}=0$.

  2. 2.

${\Theta}={{\mathcal{P}}}{\mathcal{T}}$: $b_{00}=0$ and the coefficients

$F,\quad
r_{\parallel,\perp,\times},\quad\kappa_{\parallel,\perp,\times},\quad\lambda_{\times}\quad\gamma_{\parallel,\perp},\quad
m$ (63)

are all even functions of $\mu^{5}$ while $\lambda_{\parallel,\perp}$ and
$\gamma_{\times}$ are all odd functions.

  3. 3.

${\Theta}={{\mathcal{C}}}{\mathcal{T}}$: $b_{50}=0$ and that the coefficients

$F,\quad
r_{\parallel,\perp},\quad\kappa_{\parallel,\perp},\quad\gamma_{\parallel,\perp,\times}$
(64)

are all even functions of $A_{0}$ while

$r_{\times},\quad\kappa_{\times},\quad\lambda_{\parallel,\perp,\times},\quad
m$ (65)

are all odd functions. Further, notice that the equation of of motion for
$A_{0}$ is $n=-\partial_{A_{0}}F=0$. As $F$ is an even function of $A_{0}$
this equation is solved by $A_{0}=0$, meaning that on the the equations of
motion, (65) all vanish.

  4. 4.

${\Theta}={{\mathcal{C}}}{{\mathcal{P}}}{\mathcal{T}}$: Both topological
coefficients vanish, $b_{00}=b_{50}=0$. And the coefficients

$F,\quad
r_{\parallel,\perp},\quad\kappa_{\parallel,\perp},\quad\lambda_{\parallel,\perp,\times},\quad
m$ (66)

are even under $A_{0}\to-A_{0},\mu^{5}\to-\mu^{5}$, while

$r_{\times},\quad\kappa_{\times},\quad\gamma_{\parallel,\perp,\times}$ (67)

are all odd. The equation of motion for $A_{a0}$ is $n=-\partial_{A_{0}}F=0$,
which can be solved for $A_{0}$ in terms of $B$ and $\mu^{5}$. Because $F$ is
even in $A_{0},\mu^{5}$, the expression for $A_{0}$ must be odd in $\mu^{5}$.
As a result, on shell, (66) are understood to be even functions of $\mu^{5}$
while (67) are odd.

Interestingly, unlike the case of ordinary plasmas, the various choices of
${\Theta}$ do not correspond to more symmetric versions of the
${\Theta}={\mathcal{T}}$ case.

##  Appendix B Comparison with the results of Vardhan  _et al._ (2022)

Here we compare (12) with the corresponding expressions in Vardhan  _et al._
(2022). From (12) we have

$\displaystyle E_{i}$ $\displaystyle=$
$\displaystyle{\nabla}A_{0}-\partial_{0}A_{i}=\partial_{i}f+(r_{\parallel}h_{i}h_{j}+r_{\perp}{{\Delta}}_{ij}+r_{\times}\epsilon_{ijk}h_{k})(({\nabla}\times\vec{H})_{j}-b_{0}B_{j})$
(68) $\displaystyle=$
$\displaystyle\partial_{i}f-r_{\parallel}b_{0}B_{i}+(r_{\parallel}h_{i}h_{j}+r_{\perp}{{\Delta}}_{ij}+r_{\times}\epsilon_{ijk}h_{k})({\nabla}\times\vec{H})_{j}$
(69)

where $H_{i}={B_{i}\over\mu}$. Note that

${\nabla}\times\vec{H}={\nabla}\times\left({\vec{B}\over\mu}\right)={\vec{j}_{B}\over\mu},\quad{\vec{j}}_{B}\equiv{\vec{j}}-\vec{\nabla}\ln\mu\times{\vec{B}},\quad\vec{j}\equiv{\nabla}\times\vec{B}\
.$ (70)

We then find that

$\displaystyle{\vec{E}}=\vec{\nabla}f-b_{0}r_{\parallel}{\vec{B}}+c_{\eta}\,{\vec{j}}_{B}+c_{a}\,({\vec{B}}\cdot{\vec{j}})\,{\vec{B}}+c_{H}\,{\vec{j}}_{B}\times{\vec{B}},$
(71) $\displaystyle c_{\eta}={r_{\perp}\over\mu},\quad
c_{a}=\frac{r_{\parallel}-r_{\perp}}{\mu B^{2}}\quad
c_{H}=\frac{r_{\times}}{\mu B}\ .$ (72)

Equation (71) can also be written as

$\displaystyle{\vec{E}}$
$\displaystyle=\vec{\nabla}f-b_{0}r_{\parallel}{\vec{B}}+\eta\,{\vec{j}}+c_{a}\,({\vec{j}}\times{\vec{B}})\times{\vec{B}}+c_{H}\,{\vec{j}}\times{\vec{B}}$
(73) $\displaystyle-c_{H}(\vec{\nabla}\ln
a\times{\vec{B}})\times{\vec{B}}-c_{\eta}\vec{\nabla}\ln
a\times{\vec{B}},\qquad\eta={r_{\parallel}\over\mu}\ .$ (74)

Equations (71) and (74) are exactly the same as equations (6) and (9) of
Vardhan  _et al._ (2022) (except that $b_{0}$-term is missing there) with the
identifications with parameters there

$\mu(B^{2})=a(G^{2}),\quad r_{\perp}=2\beta_{0}d,\quad
r_{\times}=-{pB\over\mu},\quad
r_{\parallel}=2\beta_{0}d+\frac{4\beta_{0}\tilde{d}B^{2}}{\mu^{2}}\ .$ (75)

For background $\vec{B}=B_{0}e_{z}$, the transport coefficients computed in
Vardhan  _et al._ (2022) can be expressed in our notation by

$\chi_{\parallel}={\mu^{2}\over\mu-B\partial_{B}\mu},\quad\chi_{\perp}=\mu,\quad\sigma^{\perp}_{1}=r_{\perp},\quad\sigma_{2}^{\perp}={r_{\times}},\quad\sigma^{\parallel}=r_{\parallel}\
.$ (76)

The fact that the partial Higgs approach and the 1-form symmetry method yield
the same equations of motion is no accident; there is a duality transformation
that can convert one action into the other Landry and Liu (pear). This also
explains the missing $b_{0}$ term there, as this term is non-local in the one-
form symmetry formulation.

## References

  * Vardhan  _et al._ (2022) S. Vardhan, S. Grozdanov, S. Leutheusser, and H. Liu, (2022), arXiv:2207.01636 [astro-ph.HE] .
  * Gaiotto  _et al._ (2015) D. Gaiotto, A. Kapustin, N. Seiberg, and B. Willett, JHEP 02, 172 (2015), arXiv:1412.5148 [hep-th] .
  * Grozdanov  _et al._ (2017) S. Grozdanov, D. M. Hofman, and N. Iqbal, Phys. Rev. D95, 096003 (2017), arXiv:1610.07392 [hep-th] .
  * Landsteiner (2016) K. Landsteiner, Acta Phys. Polon. B 47, 2617 (2016), arXiv:1610.04413 [hep-th] .
  * Kharzeev  _et al._ (2016) D. E. Kharzeev, J. Liao, S. A. Voloshin, and G. Wang, Prog. Part. Nucl. Phys. 88, 1 (2016), arXiv:1511.04050 [hep-ph] .
  * Golkar and Son (2015) S. Golkar and D. T. Son, JHEP 02, 169 (2015), arXiv:1207.5806 [hep-th] .
  * Hou  _et al._ (2012) D.-F. Hou, H. Liu, and H.-c. Ren, Phys. Rev. D 86, 121703 (2012), arXiv:1210.0969 [hep-th] .
  * Jensen  _et al._ (2013) K. Jensen, P. Kovtun, and A. Ritz, JHEP 10, 186 (2013), arXiv:1307.3234 [hep-th] .
  * Gorbar  _et al._ (2013) E. V. Gorbar, V. A. Miransky, I. A. Shovkovy, and X. Wang, Phys. Rev. D 88, 025025 (2013), arXiv:1304.4606 [hep-ph] .
  * Figueroa and Shaposhnikov (2018) D. G. Figueroa and M. Shaposhnikov, JHEP 04, 026 (2018), [Erratum: JHEP 07, 217 (2020)], arXiv:1707.09967 [hep-ph] .
  * Figueroa  _et al._ (2019) D. G. Figueroa, A. Florio, and M. Shaposhnikov, JHEP 10, 142 (2019), arXiv:1904.11892 [hep-th] .
  * Das  _et al._ (2022) A. Das, R. Gregory, and N. Iqbal, (2022), arXiv:2205.03619 [hep-th] .
  * Boyarsky  _et al._ (2015) A. Boyarsky, J. Frohlich, and O. Ruchayskiy, Phys. Rev. D 92, 043004 (2015), arXiv:1504.04854 [hep-ph] .
  * Pavlović  _et al._ (2017) P. Pavlović, N. Leite, and G. Sigl, Phys. Rev. D 96, 023504 (2017), arXiv:1612.07382 [astro-ph.CO] .
  * Giovannini (2016) M. Giovannini, Phys. Rev. D 94, 081301 (2016), arXiv:1606.08205 [hep-th] .
  * Yamamoto (2016) N. Yamamoto, Phys. Rev. D 93, 125016 (2016), arXiv:1603.08864 [hep-th] .
  * Rogachevskii  _et al._ (2017) I. Rogachevskii, O. Ruchayskiy, A. Boyarsky, J. Fröhlich, N. Kleeorin, A. Brandenburg, and J. Schober, Astrophys. J. 846, 153 (2017), arXiv:1705.00378 [physics.plasm-ph] .
  * Hattori  _et al._ (2019) K. Hattori, Y. Hirono, H.-U. Yee, and Y. Yin, Phys. Rev. D 100, 065023 (2019), arXiv:1711.08450 [hep-th] .
  * Hattori  _et al._ (2022) K. Hattori, M. Hongo, and X.-G. Huang, Symmetry 14, 1851 (2022), arXiv:2207.12794 [hep-th] .
  * Landry and Liu (pear) M. J. Landry and H. Liu, (to appear).
  * Das  _et al._ (pear) A. Das, N. Iqbal, and N. Poovuttikul, (to appear).
  * Keldysh (1964) L. Keldysh, Zh.Eksp.Teor.Fiz. 47, 1515 (1964).
  * Crossley  _et al._ (2017) M. Crossley, P. Glorioso, and H. Liu, JHEP 09, 095 (2017), arXiv:1511.03646 [hep-th] .
  * Glorioso  _et al._ (2017) P. Glorioso, M. Crossley, and H. Liu, JHEP 09, 096 (2017), arXiv:1701.07817 [hep-th] .
  * Glorioso and Liu (2018) P. Glorioso and H. Liu, (2018), arXiv:1805.09331 [hep-th] .
  * Glorioso and Liu (2016) P. Glorioso and H. Liu, (2016), arXiv:1612.07705 [hep-th] .
  * Kovtun (2016) P. Kovtun, JHEP 07, 028 (2016), arXiv:1606.01226 [hep-th] .
  * Hernandez and Kovtun (2017) J. Hernandez and P. Kovtun, JHEP 05, 001 (2017), arXiv:1703.08757 [hep-th] .
  * Grozdanov and Poovuttikul (2019) S. Grozdanov and N. Poovuttikul, JHEP 04, 141 (2019), arXiv:1707.04182 [hep-th] .
  * Glorioso and Son (2018) P. Glorioso and D. T. Son, (2018), arXiv:1811.04879 [hep-th] .
  * Armas and Jain (2020) J. Armas and A. Jain, JHEP 01, 041 (2020), arXiv:1811.04913 [hep-th] .
  * Gralla and Iqbal (2019) S. E. Gralla and N. Iqbal, Phys. Rev. D 99, 105004 (2019), arXiv:1811.07438 [hep-th] .
  * Benenowski and Poovuttikul (2019) B. Benenowski and N. Poovuttikul, (2019), arXiv:1911.05554 [hep-th] .
  * Landry (2021) M. J. Landry, (2021), arXiv:2101.02210 [hep-th] .
  * Armas and Camilloni (2022) J. Armas and F. Camilloni, (2022), arXiv:2201.06847 [hep-th] .
  * Goldreich and Reisenegger (1992) P. Goldreich and A. Reisenegger, Astrophys. J. 395, 250 (1992).
  * Glorioso  _et al._ (2019) P. Glorioso, H. Liu, and S. Rajagopal, JHEP 01, 043 (2019), arXiv:1710.03768 [hep-th] .
  * Son and Zhitnitsky (2004) D. T. Son and A. R. Zhitnitsky, Phys. Rev. D 70, 074018 (2004), arXiv:hep-ph/0405216 .
  * Metlitski and Zhitnitsky (2005) M. A. Metlitski and A. R. Zhitnitsky, Phys. Rev. D 72, 045011 (2005).
  * Huang and Liao (2013) X.-G. Huang and J. Liao, Phys. Rev. Lett. 110, 232302 (2013).
  * Jiang  _et al._ (2015) Y. Jiang, X.-G. Huang, and J. Liao, Phys. Rev. D 91, 045001 (2015).
  * Kharzeev and Yee (2011) D. E. Kharzeev and H.-U. Yee, Phys. Rev. D 83, 085007 (2011), arXiv:1012.6026 [hep-th] .
  * Vilenkin (1980) A. Vilenkin, Phys. Rev. D 22, 3080 (1980).
  * Kharzeev  _et al._ (2008) D. E. Kharzeev, L. D. McLerran, and H. J. Warringa, Nucl. Phys. A 803, 227 (2008), arXiv:0711.0950 [hep-ph] .
  * Fukushima  _et al._ (2008) K. Fukushima, D. E. Kharzeev, and H. J. Warringa, Phys. Rev. D 78, 074033 (2008), arXiv:0808.3382 [hep-ph] .
  * Kharzeev (2014) D. E. Kharzeev, Prog. Part. Nucl. Phys. 75, 133 (2014), arXiv:1312.3348 [hep-ph] .
  * Akamatsu and Yamamoto (2013) Y. Akamatsu and N. Yamamoto, Phys. Rev. Lett. 111, 052002 (2013), arXiv:1302.2125 [nucl-th] .
  * Kaplan  _et al._ (2017) D. B. Kaplan, S. Reddy, and S. Sen, Phys. Rev. D 96, 016008 (2017), arXiv:1612.00032 [hep-ph] .


# Object Detection in Thermal Images Using Deep Learning for Unmanned Aerial
Vehicles

Minh Dang Tu1, Kieu Trang Le1, Manh Duong Phung2 1Minh Dang Tu, Kieu Trang Le
are with Vietnam National University, Hanoi, Vietnam.2Manh Duong Phung is with
Fulbright University Vietnam, Ho Chi Minh City, Vietnam.
<EMAIL_ADDRESS>
###### Abstract

This work presents a neural network model capable of recognizing small and
tiny objects in thermal images collected by unmanned aerial vehicles. Our
model consists of three parts, the backbone, the neck, and the prediction
head. The backbone is developed based on the structure of YOLOv5 combined with
the use of a transformer encoder at the end. The neck includes a BI-FPN block
combined with the use of a sliding window and a transformer to increase the
information fed into the prediction head. The prediction head carries out the
detection by evaluating feature maps with the Sigmoid function. The use of
transformers with attention and sliding windows increases recognition accuracy
while keeping the model at a reasonable number of parameters and computation
requirements for embedded systems. Experiments conducted on public dataset
VEDAI and our collected datasets show that our model has a higher accuracy
than state-of-the-art methods such as ResNet, Faster RCNN, ComNet, ViT,
YOLOv5, SMPNet, and DPNetV3. Experiments on the embedded computer Jetson AGX
show that our model achieves a real-time computation speed with a stability
rate of over 90%.

###### Index Terms:

Deep learning, thermal image, unmanned aerial vehicle

##  I Introduction

Unmanned aerial vehicles (UAVs) are being used for specialized tasks such as
search and rescue, surveillance, and military operations. To carry out these
tasks, UAVs are equipped with image sensors and onboard processing units to
collect data and then analyze it. The analysis often requires object
recognition and classification from which relevant operations can be
conducted. For UAV object recognition, color images are commonly used as input
data. However, in many environmental conditions such as nighttime or low
light, the use of color images is ineffective as objects and the environment
blend together. Using thermal images is then considered a more relevant
solution with advantages such as the ability to identify small and tiny
objects as long as their heat emission is different from the ambient
temperature, and the possibility to search in challenging conditions such as
fog and low light. The main challenge, however, is how to deal with low
resolution, uneven thermal background, and high noise in thermal images. A
number of studies have been proposed to address these problems with details as
follows.

###  I-A Small object detection

Methods for detecting small objects have been developed for years with state-
of-the-art methods such as Viola Jones Detectors [1], Histogram of Oriented
Gradients (HOG) detector [2], Deformable Part-based Model (DPM) [3], etc.
However, the development of machine learning techniques, especially deep
learning, has made it a trend due to the superior results and advantages over
traditional techniques. In deep learning, object detection methods are divided
into two main methods, one-stage and two-stage detection. The one-stage method
classifies objects and generates bounding boxes without using pre-generated
region proposals. It thus has a fast execution time with representatives such
as Retical net, SSD[4], Yolo[5], CornerNet, FCOS[6], FSAF[7]. The two-stage
method first generates region proposals and then classifies objects in each
region proposal. It is therefore more accurate but slower with notable
representatives such as mask R-CNN[8], Faster R-CNN[9]. Continuous
advancements in deep learning have led to the popularity of CNN models for
object detection, leading to the development of new models such as VGG,
Googlenet, DenseNet, Resnet, Efficientnet[10], ViT[11], which have higher
quality feature maps and can detect objects more accurately.

The progress of object detection models also comes from changing the
computation method in individual layers with large computational depth [12],
model ensemble strategies [13], and region-based convolutional neural network
[13]… The improvements also include changing recognition layers from global to
local feature maps [13], using new detection kernels [12], and combining
feature maps [14]. The accuracy of the models thus gradually increases.
However, there are two challenges when using these models: (1) their
deployment on an embedded device is relatively complex, and (2) the
performance of the models is not yet truly stable.

###  I-B Combining models in object detection

Deep learning is a flexible and scalable approach that can adapt to the
increasing amount of data used for training. One drawback of this method is
that it depends on the data used, and when encountering entirely new data, the
weights of the network will change accordingly leading to an increase in the
variance. One approach to mitigate this is to train multiple models and then
combine their predictions.

There are three commonly used methods to combine predicted bounding boxes
including the non-maximum suppression (NMS) [15], Soft-NMS [16], and weighted
boxes fusion (WBF) [17]. In NMS, if the bounding box overlaps and exceeds a
certain Intersection over Union (IoU) threshold, they are considered to belong
to one object. NMS then only retains the box with the highest confidence and
removes the rest. Soft-NMS improves the NMS by using a function to reduce the
confidence of adjacent bounding boxes based on the IoU threshold. WBF operates
differently from NMS by merging boxes instead of removing them to produce the
final result. For color images provided by UAVs, WBF yields good results.
However, with thermal images, this method may mistakenly detect a background
area as an object when its thermal dissipation is similar to an object.
Therefore in our approach, Soft-NMS is used to combine predicted bounding
boxes.

###  I-C Thermal image processing

Different from color images with three channels, the information of a thermal
image is only equivalent to a grayscale image that represents the response of
the capturing area to infrared frequencies. Most approaches therefore use the
two-stage method to extract sparse features of harsh environments [18]; learn
features that maximize the information between visible and infrared
frequencies [19]; or detect low-resolution objects from UAVs [20]. The use of
two-stage methods however limits the deployment of those models to systems
that have limited computational capability.

In this paper, we present a CNN model that solves two issues in recognizing
objects from thermal images taken by UAVs. The first issue is the limited
information of thermal images when representing objects with varying
temperatures such as boats or parked vehicles. The second involves objects
that are close together, when taken by a UAV, will likely be recognized as a
single object. To solve these problems, our approach focuses on extracting
feature maps through a multi-dimensional pyramid and using an attention
mechanism to enrich data and expand information regions, as shown in Fig.1. We
use attention layers instead of increasing the depth of CNN to reduce
computation cost. In addition, we apply the Bi-FPN method instead of other FPN
to obtain a feedforward computation process that targets high-value regions.
In the final layer, we use Soft-NMS [16] to combine predicted bounding boxes
of the training models.

Our contributions in this work include:

  * •

Analyze the characteristics of small object detection problems and study the
methods used for object recognition from both color images and thermal images
taken by UAVs to come up with a new network model. The model can identify
small objects by using sliding windows to remove regions of low correlation
with their surrounding areas and output high-potential regions for
identification. The model utilizes ViT and YOLOv5, the network architectures
that perform well with color images, and augments them with attention blocks
to better identify objects.

  * •

Modify the backbone architecture to its equivalence but with fewer parameters
to better use on UAVs. Furthermore, we also carry out the process of
compressing the model’s weights to ONNX and TensorRT to increase the execution
speed of the model. We also use self-training to reinforce the classification
ability of objects and eliminate incorrect recognition features during
training.

  * •

Our model performs well on thermal images subjected to high noise. The model
is designed as a module that can be deployed on embedded systems such as UAVs.
The model achieves high accuracy in both public and our own datasets.

##  II The proposed model

Figure 1: The architecture of our model

In this section, we present the proposed method for object detection using
thermal images from UAVs. Our model is built on the architecture of YOLOv5
with the following 3 structures: backbone, neck, and head, as shown in Fig.1.
We use attention layers in both the layers after the backbone and transformers
to expand the features of the input matrix to overcome limitations related to
the low information of thermal images, especially with small objects, people,
and vehicles in the image. The use of attention layers has the role of
expanding the computational area and making features of the object clear for
identification. Due to the low information content in thermal images, we
perform calculations on surrounding pixels to detect the bounding box of the
object.

###  II-A Number of prediction heads

For the YOLOv5 model [5], after passing through the backbone and neck layers,
three feature matrices are produced to input into the prediction heads.
However, these three prediction heads are only sufficient for cases where the
objects in the image occupy a large portion of the image. In our
implementation environment, which involves images captured from drones as in
the VEDAI [21] dataset, objects in the images occupy a small portion of the
image. To identify these objects, we use 4 prediction heads instead of 3. The
additional prediction head, extracted from the low-level feature layer, helps
to reduce variance in identifying the bounding box and increase the ability to
detect new objects that might have been omitted in higher feature layers.
After adding one prediction head, the number of calculation parameters
increases, but the object detection performance is significantly improved.

###  II-B Backbone optimization

Due to the requirement to use on UAVs, a recognition model that can extract
feature maps fast enough is needed. We thus employ the feature extraction
methods through the use of GhostConv to reduce the number of parameters that
need to be calculated. To address the problem of low thermal information, we
use computation layers of Bottleneck net to extract additional information.
For an input $X\in R^{c\times h\times w}$, where $c$ is the number of input
dimensions, $h$ and $w$ are the size of the input matrix, the value of feature
layer $n$ is calculated as:

$\displaystyle Y=X*f+b,$ (1)

where $Y\in R^{h^{\prime}\times w^{\prime}\times n}$ is the result of feature
layer $n$, $b$ is the bias term, and $f\in R^{c\times k\times k\times n}$ is
the filter of the convolutional block. The GhostNet method produces the result
through the processes of standard convolution and series of linear operations
on each intrinsic feature. In standard convolution, $Y\in R^{h^{\prime}\times
w^{\prime}\times m}$ with $m<n$ is obtained from input $X\in R^{c\times
h\times w}$ and convolutional block matrices $f^{{}^{\prime}}\in R^{c\times
k\times k\times m}$ according to formula $Y^{{}^{\prime}}=X*f^{{}^{\prime}}$.
The series of linear operations convert tensor $Y^{{}^{\prime}}$ to matrix
$Y^{{}^{\prime\prime}}$ by the transformation matrix $g(x)$ according to

$\displaystyle y{{}^{\prime\prime}}_{ij}=g_{ij}\cdot{y{{}^{\prime}}_{ij}}\quad
i\in{1,\dots,m};j\in{1,\dots,s}$ (2)

Each value of $y{{}^{\prime}}$ gives a value of $y{{}^{\prime\prime}}$, where
$m$ and $s$ are the size of $y{{}^{\prime}}$ and $y{{}^{\prime\prime}}$.
Finally, $y{{}^{\prime}}$ and $y{{}^{\prime\prime}}$ are combined to generate
the output. With the use of $g_{ij}$, the computational cost is lower with
fewer parameters than a conventional CNN network.

###  II-C Transformer encoder block

Inspired by the transformer method applied in Vision Transformer (ViT), we
apply the transformer block at the end of the backbone block to replace the
Bottleneck net convolution layer in the original version of YOLOv5. The input
of this layer is the output of the forward convolution layer and the
information distributed through the ASPP block [22]. The input of the
transformer block is represented as $I_{IR}\in R_{IR}^{c\times h\times w}$
given from feature map $F_{IR}\in R_{IR}^{c\times h\times w}$. $I_{IR}$ is fed
into the attention process with 3 parameters $Q$, $K$, $V$ computed as:

$\displaystyle Q=I_{IR}\cdot W_{Q}$ (3) $\displaystyle K=I_{IR}\cdot W_{K}$
(4) $\displaystyle V=I_{IR}\cdot W_{V},$ (5)

where $W_{Q}$, $W_{K}$, $W_{V}$ are matrices trained in the model. The output,
$Z$, of the attention block is then calculated as:

$\displaystyle Z=Attention(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{d_{k}}})\cdot
V,$ (6)

where $d_{k}$ is the parameter maintaining the convergence of the $Softmax$
result. From $Z$, the activation layers GELU is used to compute the output.
Using GELU instead of ReLU allows for performing regression on the data
instead of discarding the data with negative parameters, thereby reducing the
variance.

Figure 2: Result in the public dataset VEDAI

(a) (b)

(c) (d)

Figure 3: Result with our dataset in different scenarios: (a) and (b) normal
scenarios, (c) City with dense vehicles and (d) Night time when the visual
images are totally black

###  II-D Convolution using sliding windows and attention

In our model, we combine the use of sliding windows with the attention
mechanism instead of using the usual Transformer method to achieve fast object
detection. This module is also simple enough to integrate into the CNN system
without causing the computation system to exceed the processing amount in the
YOLOv5 architecture. This block has two processes including splitting the
input matrix by a sliding window and performing Attention on the obtained
windows.

The input of this module is the output feature maps of the Bi-FPN. With the
feature map having size $(h\times w\times c)$ through the sliding window
mechanism, it is reduced to size $(n\times mh\times mw\times c)$,along with
creating a mask matrix of size size $(n\times mh\cdot mw\times mh\cdot
mw\times c)$. With the values $h$, $w$, $c$ being the width, height, and depth
of the input feature matrix, respectively; $mh$, $mw$ are the length and width
of the sliding window respectively. Self-attention is then computed on the
sliding windows and their surroundings to extract features on those windows.
Here, we use GELU layers to ensure accuracy when working with large data. The
output is computed as:

$\displaystyle\hat{z}^{l}=WA(LN(z^{l-1})+z^{l-1}$ (7) $\displaystyle
z^{l}=MLP(LN(\hat{z}^{l}))+\hat{z}^{l},$ (8)

where $z^{l}$ and $\hat{z}^{l}$ are respectively the outputs of the windows
attention (WA) and multilayer perception (MLP) modules in block $l$; WA
represents a multi-head self-attention block using sliding windows; LN denotes
a linear block with linear operations on $z$. The result of the block is the
feature map augmented with feature information extracted by the attention
layers. This module thus helps to identify the areas of interest that the
model should focus on to detect objects.

###  II-E Training and execution process

For large models, the process of detecting objects can take a lot of time.
During training, we adjust the input size to minimize the loss of information
with the input image size being 0.67 and 0.85 of the original image size. We
also rotate and flip images to diversify the training data. In the experiment,
we decided to use the input size of 0.67 of the original image size, and
retain the grayscale or color scale of the thermal image. We use techniques
such as layer stacking, and expanding the calculation area to increase the
recognition speed. We also combine with models having a low number of
parameters through the use of ConvGhost and Bottleneck Ghost to reduce the
time to compute feature maps.

##  III Results

To evaluate the proposed model, we have conducted experiments on a public
dataset named VEDAI and the datasets collected from our UAV. We have also
carried out comparisons with other state-of-the-art models on two platforms
including a high-performance server and an embedded computer.

Figure 4: Drone Mavic 2 Enterprise Advanced used to collect data

###  III-A Datasets

Our data is collected by the drone named Mavic 2 Enterprise Advanced equipped
with cameras as shown in Fig. 4. The thermal camera has a resolution of
$640\times 512$ px, speed of 30 fps, 16$\times$ zoom, and accuracy of $\pm 2$
Celcius degrees. The drone is also equipped with a color camera having a
resolution of 48 MP. This camera, however, only uses for reference purposes.
The video data collected has a duration of 11 hours and is recorded at
different times and locations. It thus provides us with millions of thermal
images for training and testing. Besides, we also use the public dataset named
VEDAI to utilize the thermal images collected from the drone in various
conditions, especially in low light. Images in the datasets are pre-processed
and labeled to ensure their quality when used for the models. In comparisons,
all the models use the same image size of $800\times 800$.

###  III-B Evaluation metrics and platforms

In all comparisons, we use the mean average precision (mAP) metric to evaluate
the accuracy of the models at two IoU thresholds, 0.5 and 0.95. The platform
used for training the models is the servers with GPU A100 3.2G 40G. The
optimization function used is Adam with a learning rate of $3.2\times
10^{-5}$. We use 10 initial epochs for the purpose of stabilizing the
training, and 150 actual training epochs with a batch size of 16. This process
is performed on both our own datasets and the public dataset.

###  III-C Evaluation results

####  III-C1 Results

We compare our model with the popular models used for object detection
including Faster R-CNN, ResNet, YOLO net, Vision Transformer (ViT), E-net,
ComNet[23], SMPNet [24], DPNetV3 [25]. The results are presented in Tables I
and II. It can be seen that our model is more accurate than other methods at
both values 0.5 and 0.95. Compared to methods that use equivalent amounts of
parameters, our model shows better results on both datasets. Specifically, our
model performs better by 4.1% to 7.01% on the VEDAI dataset for mAP50 and by
4.42% to 9.83% on our dataset for mAP95. Besides, our model has relatively low
number of parameters thanks to the replacement of multiple CNN blocks with
computationally efficient layers such as attention layers and feedforward
linear layers. As a result, our model achieves higher performance compared to
other models and has the ability to be deployed on embedded systems.

Model | Image size | Parameter | mAp50 | mAp95
---|---|---|---|---
ResNet | 800*800 | 20M | 25,82 | 14,25
Faster RCNN | 800*800 | 46M | 44,75 | 33,74
ComNet | 800*800 | 51M | 53,48 | 30,44
ViT | 800*800 | 80M | 90,33 | 54,94
Yolov5 | 800*800 | 80M | 67.59 | 35.62
SMPNet | 800*800 | 65M | 77.31 | 41.25
DPNetV3 | 800*800 | 127M | 81,70 | 57,28
Our model | 800*800 | 45M | 94.57 | 60.23
TABLE I: Comparison results on our datasets Model | Image size | Parameter | mAp50 | mAp95
---|---|---|---|---
ResNet | 800*800 | 20M | 21,48 | 11,62
Faster RCNN | 800*800 | 46M | 40.60 | 24,23
ComNet | 800*800 | 51M | 45,67 | 26,86
ViT | 800*800 | 80M | 85,62 | 50,87
Yolov5 | 800*800 | 80M | 60.24 | 29.98
SMPNet | 800*800 | 65M | 67.53 | 35.59
DPNetV3 | 800*800 | 127M | 82,78 | 56,84
Our model | 800*800 | 45M | 89.76 | 57.42
TABLE II: Comparison results on public dataset VEDAI

Figure 2 shows some results on the VEDAI data, where the first row represents
the thermal image, the second row represents the corresponding color image for
reference. It is recognizable that our model performs well on common objects
like boats, planes, and cars. These images also reflect the challenge with
object detection from UAV data when the objects have a very small size.
Figures 3 display some results on the images collected by our drone under
normal and challenging conditions such as urban environments or low light.
Here, due to UAV configuration, the thermal data is represented in red color
scale instead of greyscale. We also added the third row to provide a close
view of our detection results. It can be seen that our model can recognize
objects, even those with small sizes and low thermal contrast. Figure 3d shows
the clear advantage of using thermal images over color images where the
objects can be detected at night.

In another experiment to evaluate the capability of deploying models on
embedded devices such as UAVs, we compressed the model through methods that
maintain the quality of the model. Specifically, we used transfer learning
methods to reduce the model size while preserving its performance. The device
used for the experiment was an embedded computer named Jetson AGX 30W. We set
up all models to run on the Nvidia device’s GPU thread. Table III shows the
experimental results for different model including the full model, half model,
ONNX, TensorRT, and transfer model. It can be seen that with the use of the
transfer model, our model can be deployed on the embedded device with real-
time speed and high stability. Therefore, the model can be applied to
practical systems such as UAVs.

Model | Image size | Speed | Stability
---|---|---|---
Full Model | 800*800 | 0.02 s | 1
Half Model | 800*800 | 0.005s | 0.45
ONNX | 800*800 | 0.01s | 0.87
TensorRT | 800*800 | 3ms | 0.84
Transfer model | 800*800 | 4ms | 0.91
TABLE III: Performance of different models on the Jetson AGX embedded computer

##  IV Conclusion

In this paper, we have proposed a new neural network model for recognizing
objects on thermal images collected by UAVs. Our model uses an improved
backbone structure based on YOLOv5 combined with matching classifiers in the
neck and prediction head to increase recognition efficiency. Especially, the
use of attention blocks before the prediction head greatly increases the
ability to identify small objects. Experimental results on both the public and
our datasets show that our model performs better than the state-of-the-art
methods. It has a low number of parameters and can achieve real-time
computation on embedded systems. The model is therefore deployable on real
systems such as drones for practical applications.

## References

  * [1] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I–I, 2001.
  * [2] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol. 1, pp. 886–893, Ieee, 2005.
  * [3] P. Felzenszwalb, D. McAllester, and D. Ramanan, “A discriminatively trained, multiscale, deformable part model,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, 2008.
  * [4] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, pp. 21–37, Springer, 2016.
  * [5] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” 2022\.
  * [6] Z. Tian, C. Shen, H. Chen, and T. He, “Fcos: Fully convolutional one-stage object detection,” in Proceedings of the IEEE/CVF international conference on computer vision, pp. 9627–9636, 2019.
  * [7] C. Zhu, Y. He, and M. Savvides, “Feature selective anchor-free module for single-shot object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 840–849, 2019.
  * [8] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 2961–2969, 2017.
  * [9] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems, vol. 28, 2015.
  * [10] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International conference on machine learning, pp. 6105–6114, PMLR, 2019.
  * [11] K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu, Z. Yang, Y. Zhang, and D. Tao, “A survey on vision transformer,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 87–110, 2023.
  * [12] M. Hong, S. Li, Y. Yang, F. Zhu, Q. Zhao, and L. Lu, “Sspnet: Scale selection pyramid network for tiny person detection from uav images,” IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1–5, 2021.
  * [13] X. Yu, Z. Han, Y. Gong, N. Jan, J. Zhao, Q. Ye, J. Chen, Y. Feng, B. Zhang, X. Wang, et al., “The 1st tiny object detection challenge: Methods and results,” in Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pp. 315–323, Springer, 2020.
  * [14] B. Stojanović, O. Marques, A. Nešković, and S. Puzović, “Fingerprint roi segmentation based on deep learning,” in 2016 24th Telecommunications Forum (TELFOR), pp. 1–4, 2016.
  * [15] J. Hosang, R. Benenson, and B. Schiele, “Learning non-maximum suppression,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4507–4515, 2017.
  * [16] N. Bodla, B. Singh, R. Chellappa, and L. S. Davis, “Soft-nms–improving object detection with one line of code,” in Proceedings of the IEEE international conference on computer vision, pp. 5561–5569, 2017.
  * [17] R. Solovyev, W. Wang, and T. Gabruseva, “Weighted boxes fusion: Ensembling boxes from different object detection models,” Image and Vision Computing, vol. 107, p. 104117, 2021.
  * [18] M. Krišto, M. Ivasic-Kos, and M. Pobar, “Thermal object detection in difficult weather conditions using yolo,” IEEE access, vol. 8, pp. 125459–125476, 2020.
  * [19] F. Munir, S. Azam, and M. Jeon, “Sstn: Self-supervised domain adaptation thermal object detection for autonomous driving,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 206–213, IEEE, 2021.
  * [20] F. S. Leira, H. H. Helgesen, T. A. Johansen, and T. I. Fossen, “Object detection, recognition, and tracking from uavs using a thermal camera,” Journal of Field Robotics, vol. 38, no. 2, pp. 242–267, 2021.
  * [21] S. Razakarivony and F. Jurie, “Vehicle detection in aerial imagery: A small target detection benchmark,” Journal of Visual Communication and Image Representation, vol. 34, pp. 187–203, 2016.
  * [22] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 834–848, 2017.
  * [23] M. Li, X. Zhao, J. Li, and L. Nan, “Comnet: Combinational neural network for object detection in uav-borne thermal images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 8, pp. 6662–6673, 2021.
  * [24] Q. Wang, Z. Tao, Q. Gao, and L. Jiao, “Multi-view subspace clustering via structured multi-pathway network,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–7, 2022.
  * [25] V. Team, “Visdrone 2020 leaderboard. website, 2020.”