The boundary disorder correlation for the Ising model on a cylinder

Rafael L. Greenblatt

Dipartimento di Matematica, Università degli Studi di Roma “Tor Vergata”,
Rome, Italy

<EMAIL_ADDRESS>
## Abstract

I give an expression for the correlation function of disorder insertions on
the edges of the critical Ising model on a cylinder as a function of the
aspect ratio (rescaled in the case of anisotropic couplings). This is obtained
from an expression for the finite size scaling term in the free energy on a
cylinder in periodic and antiperiodic boundary conditions in terms of Jacobi
theta functions.

Copyright attribution to authors.
This work is a submission to SciPost Physics Core.
License information to appear upon publication.
Publication information to appear upon publication. Received Date
Accepted Date
Published Date

###### Contents

  1. 1 Introduction
  2. 2 The partition function in the McCoy-Wu solution
  3. 3 Asymptotic analysis
  4. 4 Conclusions
  5. A Acknowledgements

##  1 Introduction

I study the disorder correlation

$\left\langle\mu_{\textup{top}}\mu_{\textup{bottom}}\right\rangle_{\pm}:=\frac{Z_{\mp}}{Z_{\pm}}$
(1)

where $Z_{+}$ ($Z_{-}$) is the partition function of the Ising model on a
discrete cylinder with open and (anti-)periodic boundary conditions, that is

$Z_{\pm}=\sum_{\sigma\in\left\\{-1,+1\right\\}^{4\mathcal{M}\mathcal{N}}}\begin{aligned}
&\exp\left(\sum_{j=1}^{2\mathcal{M}}\sum_{k=-\mathcal{N}+1}^{\mathcal{N}-1}\right.\beta
E_{1}\sigma_{j,k}\sigma_{j,k+1}+\sum_{j=1}^{2\mathcal{M}-1}\sum_{k=-\mathcal{N}+1}^{\mathcal{N}}\beta
E_{2}\sigma_{j,k}\sigma_{j+1,k}\\\
&\quad\quad\quad\left.\pm\sum_{j=1}^{2\mathcal{M}}\beta
E_{1}\sigma_{j,\mathcal{N}}\sigma_{j,-\mathcal{N}+1}\right).\end{aligned}$ (2)

This quantity appears when expressing the correlation function in periodic
boundary conditions for an odd number of spins on each boundary as a Pfaffian
in the exact solution [1], via the Kadanoff-Ceva Fermion two-point function

$\left\langle\psi_{x,{\textup{top}}}\psi_{y,{\textup{bottom}}}\right\rangle_{-}=\left\langle\sigma_{x}\mu_{\textup{top}}\sigma_{y}\mu_{\textup{bottom}}\right\rangle_{-}=\left\langle\sigma_{x}\sigma_{y}\right\rangle_{+}\left\langle\mu_{\textup{top}}\mu_{\textup{bottom}}\right\rangle_{-}$
(3)

which can be characterized via the solution of a linear boundary-value problem
[2] and evaluated via a modified Fourier transform, giving a particularly
explicit form in the continuum limit [3]. Higher spin correlations can be
calculated from

$\left\langle\psi_{x_{1},{\textup{top}}}\dots\psi_{x_{m},{\textup{top}}}\psi_{y_{1},{\textup{bottom}}}\dots\psi_{y_{n},{\textup{bottom}}}\right\rangle_{-}=\left\langle\sigma_{x_{1}}\dots\sigma_{x_{m}}\sigma_{y_{1}}\dots\sigma_{y_{n}}\right\rangle_{+}\left\langle\mu_{\textup{top}}\mu_{\textup{bottom}}\right\rangle_{-}$
(4)

($m,n$ odd) which can again be related to a linear boundary value problem, the
solution of which is the Pfaffian of a matrix whose elements are values of the
two-point function in Equation 3 or

$\left\langle\psi_{x,{\textup{top}}}\psi_{x^{\prime},{\textup{top}}}\right\rangle_{-}=\left\langle\sigma_{x}\sigma_{x^{\prime}}\right\rangle_{-},\quad\left\langle\psi_{y,{\textup{bottom}}}\psi_{y^{\prime},{\textup{bottom}}}\right\rangle_{-}=\left\langle\sigma_{y}\sigma_{y^{\prime}}\right\rangle_{-}$
(5)

The result is an expression for the multi-point boundary spin correlation
function (with periodic boundary conditions) as a Pfaffian in terms of the
two-point boundary spin correlation with either periodic or antiperiodic
boundary conditions and the disorder correlation of Equation 1 [1]. Note that
this is slightly different from the situation for with an even number of spins
on each boundary, where (as for simply connected domains) the boundary spin
correlation function is immediately equal to a Kadanoff-Ceva correlation
function and so is given by a Pfaffian in terms of the two point spin function
in the same boundary conditions, without any additional factor.

At the critical temperature in the limit of $\mathcal{M},\mathcal{N}\to\infty$
with $\mathcal{M}/\mathcal{N}^{2}\to 0$, I will show that

$\left\langle\mu_{\textup{top}}\mu_{\textup{bottom}}\right\rangle_{\pm}\sim\left(\frac{\theta_{2}(0,e^{-2\pi\zeta})}{2\theta_{3}(0,e^{-2\pi\zeta})}\right)^{\pm
1/2}=\left(\frac{\theta_{2}(0|2i\zeta)}{2\theta_{3}(0|2i\zeta)}\right)^{\pm
1/2}$ (6)

where $\zeta=\xi\mathcal{M}/\mathcal{N}$ is a rescaled aspect ratio with a
factor $\xi$ depending only on the ratio of coupling constants $E_{1}/E_{2}$
(cf. [4]), with $\xi=1$ in the isotropic case $E_{1}=E_{2}$; and $\theta_{j}$
are Jacobi theta functions [5, Chapter 20]. This follows from an asymptotic
expansion of the logarithm of the partition functions at the critical
temperature to constant order, i.e.

$\log
Z_{\pm}=p\mathcal{M}\mathcal{N}+s\mathcal{N}+z_{\pm}(\zeta)+\mathcal{O}(1/\mathcal{N})+\mathcal{O}(\mathcal{M}/\mathcal{N}^{2}).$
(7)

Note that in this case the constant-order term (also called the finite size
scaling term) is the first term which differs between the periodic and
antiperiodic cases, and so it entirely describes the leading behaviour of the
disorder correlation in Equation 1. The presence of such a term for the Ising
model was first noted by [6] for the torus, and it was subsequently noted that
the behaviour of such terms (including at least some of their dependence on
boundary conditions) could be explained by the identification of the scaling
limit of the Ising model with a specific conformal field theory [7, 8, 9],
stimulating a number of further calculations in other geometries and boundary
conditions, e.g. [10, 11, 12, 13]. For the periodic cylindrical case this
expansion was first discussed in [14] although only part of the result was
presented; a more detailed treatment (again in the periodic case) was
presented [15]. A similar expansion has been carried out in [16], giving a
less explicit expression which however is also valid away from the critical
temperature (as in a massive scaling limit), in both periodic and antiperiodic
boundary conditions. Similar expansions have also been obtained for the dimer
model (e.g. [17, 18, 19]), including some cases giving formulae for similar
ratios of partition functions [20, 21, 22].

##  2 The partition function in the McCoy-Wu solution

The starting point is the exact solution of the Ising model on a cylinder due
to McCoy and Wu [23, p. 113-20], which expresses the partition function in
terms of products of $2\times 2$ matrices similar to the transfer matrix of
the one-dimensional Ising model, also used in the study of disordered models
where these become random matrices [24, 25, 26], and can be generalized (not
without complications) to a rectangle with open boundary conditions [27]. Note
that McCoy and Wu also allowed an additional term in the Hamiltonian coupling
to the sites on one of the boundaries, but we take this term to be zero
(setting $\mathfrak{H}=0$). Since the solution is based on the high-
temperature contour expansion, the main parameters are

$z_{j}:=\tanh\beta E_{j},\quad j=1,2;$ (8)

in what follows I will mostly consider the critical temperature characterized
by

$z_{2}=\frac{1-z_{1}}{1+z_{1}}$ (9)

(which reduces to $z_{1}=\sqrt{2}-1$ in the isotropic case
$E_{1}=E_{2}\implies z_{1}=z_{2}$), and use this to simplify the expressions
in [23] by eliminating $z_{2}$.

With these specializations, the partition function is given by

$Z_{\pm}=\frac{1}{2}\left(2\cosh\beta J\right)^{MN}\left(\cosh\beta
J\right)^{N(M-1)}\sqrt{|A_{\pm}|}$ (10)

where $A$ is an antisymmetric $4MN\times 4MN$ matrix made up of $4\times 4$
blocks

$\displaystyle A_{\pm;x,y;x,y}=\begin{bmatrix}0&1&-1&-1\\\ -1&0&1&-1\\\
1&-1&0&1\\\ 1&1&-1&0\end{bmatrix},\;1\leq x\leq M,\;1\leq y\leq N$ (11)
$\displaystyle A_{\pm;x,y;x,y+1}=-A^{T}_{x,y+1;x,y}=\begin{bmatrix}0&t&0&0\\\
0&0&0&0\\\ 0&0&0&0\\\ 0&0&0&0\end{bmatrix},\;1\leq x\leq M,\;1\leq y<N$ (12)
$\displaystyle A_{\pm;x,y;x+1,y}=-A^{T}_{x+1,y;x,y}=\begin{bmatrix}0&0&0&0\\\
0&0&0&0\\\ 0&0&0&t\\\ 0&0&0&0\end{bmatrix},\;1\leq x<M,\;1\leq y\leq N$ (13)
$\displaystyle A_{\pm;x,N;x,1}=-A^{T}_{x,1;x,N}=\begin{bmatrix}0&\mp t&0&0\\\
0&0&0&0\\\ 0&0&0&0\\\ 0&0&0&0\end{bmatrix},\;1\leq x\leq M$ (14)

and all other entries zero; note that only the last case depends on the
parity. Carrying out a Fourier transform as in [23] (the antiperiodic case can
be conducted as in the discrete torus) block-diagonalizes the matrix $A$,
giving

$\det A_{\pm}=\prod_{\theta\in Q_{\pm}(N)}\det B_{\pm}(\theta)$ (15)

where the sum is over

$Q_{+}(\mathcal{N})=\left\\{\frac{\pi(2n-1)}{2\mathcal{N}}:n=1,\dots,2\mathcal{N}\right\\}\textup{
or }Q_{-}(\mathcal{N})=\left\\{\frac{\pi
n}{\mathcal{N}}:n=0,\dots,2\mathcal{N}-1\right\\},$ (16)

according to the boundary condition. The matrix $B_{\pm}(\theta)$ is the same
sparse matrix appearing in [23, Chapter 6, Equation (3.6)] from which

$Z_{\pm}^{2}=(2\cosh\beta E_{1})^{8\mathcal{M}\mathcal{N}}(\cosh\beta
E_{2})^{4\mathcal{N}(2\mathcal{M}-1)}\prod_{\theta\in
Q_{\pm}(\mathcal{N})}\left\\{\left|1+z_{1}e^{i\theta}\right|^{4\mathcal{M}}\lambda^{2\mathcal{M}}\left[v^{2}+v^{\prime
2}\alpha^{-4\mathcal{M}}\right]\right\\}$ (17)

(cf. [23, Equation (3.26)]; as noted above $\mathfrak{H}=0$, so $z=0$, giving
a slight simplification), where

$\lambda=\frac{z_{2}(1-z_{1}^{2})}{\left|1+z_{1}e^{i\theta}\right|^{2}}\alpha,$
(18)

and specializing to the critical case

$\begin{split}\alpha&=\frac{1}{2(1-z_{1})^{2}}\begin{aligned}
&\left\\{2\frac{(1+z_{1}^{2})^{2}}{(1+z_{1})^{2}}-\frac{4z_{1}^{2}}{(1+z_{1})^{2}}(e^{i\theta}+e^{-i\theta})\right.\\\
&\left.\quad+\frac{4z_{1}}{(1+z_{1})^{2}}\left[(1-z_{1}^{2}e^{i\theta})(1-z_{1}^{2}e^{-i\theta})(1-e^{i\theta})(1-e^{-i\theta})\right]^{1/2}\right\\}\end{aligned}\\\
&=\frac{1}{(1-z_{1}^{2})^{2}}\left\\{(1+z_{1}^{2})^{2}-4z_{1}^{2}\cos\theta+2z_{1}\left[2(1-\cos\theta)(1+z_{1}^{4}-2z_{1}^{2}\cos\theta)\right]^{1/2}\right\\}\\\
&=1+\frac{2z_{1}}{1-z_{1}^{2}}|\theta|+\left(\frac{z_{1}}{1-z_{1}^{2}}\right)^{2}\theta^{2}+\mathcal{O}(\theta^{3})=\exp\left(\frac{2z_{1}}{1-z_{1}^{2}}|\theta|\right)+\mathcal{O}(\theta^{3})\\\
&=:\exp\left(\xi|\theta|\right)+\mathcal{O}(\theta^{3}),\quad\theta\to
0.\end{split}$ (19)

using the criticality condition to simplify, in particular
$\alpha_{1}=z_{1}^{2},\alpha_{2}=1$; and $v,v^{\prime}\geq 0$ are such that

$v^{2}+v^{\prime
2}=1,\quad\frac{v}{v^{\prime}}=\frac{i(z_{2}^{2}-\lambda)}{\mathfrak{a}z_{2}}$
(20)

with

$\mathfrak{a}=\frac{2iz_{1}\sin\theta}{|1+z_{1}e^{i\theta}|^{2}},$ (21)

so that

$\frac{v^{\prime}}{v}=\frac{2z_{1}\sin\theta}{z_{2}|1+z_{1}e^{i\theta}|^{2}-(1-z_{1}^{2})\alpha},\quad
v^{2}=\left(1+\frac{v^{\prime 2}}{v^{2}}\right)^{-1}.$ (22)

##  3 Asymptotic analysis

In identifying the limit I separate factors in Equation 17 as

$Z_{\pm}=Y(\mathcal{M},\mathcal{N})\prod_{\theta\in
Q_{\pm}(\mathcal{N})}W_{\mathcal{M}}(\theta)X_{\mathcal{M}}(\theta),$ (23)

with

$\displaystyle Y(\mathcal{M},\mathcal{N})$ $\displaystyle=(2\cosh\beta
E_{1})^{4\mathcal{M}\mathcal{N}}(\cosh\beta
E_{2})^{2\mathcal{N}(2\mathcal{M}-1)}z_{2}^{\mathcal{M}\mathcal{N}}(1-z_{1}^{2})^{\mathcal{M}\mathcal{N}},$
(24) $\displaystyle W_{\mathcal{M}}(\theta)$
$\displaystyle=\alpha^{\mathcal{M}}|v|,$ (25) $\displaystyle
X_{\mathcal{M}}(\theta)$ $\displaystyle=\left[1+\frac{v^{2}}{v^{\prime
2}}\alpha^{-4\mathcal{M}}\right]^{1/2}.$ (26)

The first term is entirely straightforward. For the other two parts, the
behavior of various quantities near $\theta=0$ is particularly important. It
is easy enough to see from Equation 19 that $\alpha$ depends smoothly on
$\theta\in(0,2\pi)$ but its derivative is discontinuous at zero, with

$\frac{d}{d\theta}\log\alpha(\theta)=\pm\xi+\mathcal{O}(\theta^{2}),\quad\theta\to
0^{\pm}$ (27)

With this in mind, we use the Euler-MacLaurin formula [5, Section 2.10(i)] to
note that

$\begin{split}\sum_{\theta\in
Q_{+}(\mathcal{N})}f(\theta)=&\sum_{n=1}^{2\mathcal{N}}f\left(\frac{\pi(2n-1)}{2\mathcal{N}}\right)\\\
=&\frac{\mathcal{N}}{\pi}\int_{\pi/2\mathcal{N}}^{(2-1/2\mathcal{N})\pi}f(\theta)\,\text{\rm
d}\theta+\tfrac{1}{2}f(\pi/2\mathcal{N})+\tfrac{1}{2}f((2-1/2\mathcal{N})\pi)\\\
&+\frac{1}{12}\frac{\pi}{\mathcal{N}}\left[f^{\prime}((2-1/2\mathcal{N})\pi)-f^{\prime}(\pi/2\mathcal{N})\right]+\mathcal{O}\left(\frac{1}{\mathcal{N}^{2}}\right)\\\
=&\frac{\mathcal{N}}{\pi}\int_{0}^{2\pi}f(\theta)\,\text{\rm
d}\theta+\frac{1}{24}\frac{\pi}{\mathcal{N}}\left[f^{\prime}(0)-f^{\prime}(2\pi)\right]+\mathcal{O}\left(\frac{1}{\mathcal{N}^{2}}\right),\quad\mathcal{N}\to\infty\end{split}$
(28)

for any $f$ which is three times differentiable on $(0,2\pi)$ with bounded
third derivative, using

$\begin{split}\frac{\mathcal{N}}{\pi}\int_{0}^{\pi/2\mathcal{N}}f(\theta)\,\text{\rm
d}\theta&=\frac{1}{2}f(\pi/2\mathcal{N})-\int_{0}^{\pi/2\mathcal{N}}\int_{x}^{\pi/2\mathcal{N}}f^{\prime}(y)\,\text{\rm
d}y\,\text{\rm d}x\\\
&=\frac{1}{2}f(\pi/2\mathcal{N})-\frac{1}{8}\frac{\pi}{\mathcal{N}}f^{\prime}(0)+\mathcal{O}\left(\frac{1}{\mathcal{N}^{2}}\right),\quad\mathcal{N}\to\infty\end{split}$
(29)

to obtain the last expression; similarly, with $f(0)=f(2\pi)$

$\begin{split}\sum_{\theta\in
Q_{-}(\mathcal{N})}f(\theta)=&\sum_{n=0}^{2\mathcal{N}}f\left(\frac{n\pi}{\mathcal{N}}\right)-\tfrac{1}{2}f(0)-\tfrac{1}{2}f(2\pi)\\\
=&\frac{\mathcal{N}}{\pi}\int_{0}^{2\pi}f(\theta)\,\text{\rm
d}\theta-\frac{1}{12}\frac{\pi}{\mathcal{N}}\left[f^{\prime}(0)-f^{\prime}(2\pi)\right]+\mathcal{O}\left(\frac{1}{\mathcal{N}^{2}}\right),\quad\mathcal{N}\to\infty\end{split}$
(30)

Applying these expressions,

$\log Y(\mathcal{M},\mathcal{N})+\sum_{\theta\in Q_{\pm}(\mathcal{N})}\log
W_{\mathcal{M}}(\theta)=p\mathcal{M}\mathcal{N}+s\mathcal{N}+w_{\pm}\frac{\mathcal{M}}{\mathcal{N}}+\mathcal{O}\left(\frac{1}{\mathcal{N}}+\frac{\mathcal{M}}{\mathcal{N}^{2}}\right),$
(31)

where $p$ and $s$ are coefficients independent of the boundary condition, and

$w_{+}=\frac{\pi}{12}\xi,\quad w_{-}=-\frac{\pi}{6}\xi;$ (32)

note that this is the only term in Equation 31 which depends on the boundary
condition, and there is no term proportional to $\mathcal{M}$.

For the remaining factor, first note that

$\begin{split}&\sum_{\theta\in
Q_{\pm}(\mathcal{N})\cap[\mathcal{M}^{-1/3},2\pi-\mathcal{M}^{-1/3}]}\log
X_{\mathcal{M}}(\theta)=\mathcal{O}\left(\sum_{\theta\in
Q_{\pm}(\mathcal{N})\cap[\mathcal{M}^{-1/3},2\pi-\mathcal{M}^{-1/3}]}\alpha(\theta)^{-4\mathcal{M}}\right)\\\
&\quad=\mathcal{O}\left(\sum_{n={\left\lfloor\mathcal{N}\mathcal{M}^{-1/3}\right\rfloor}}^{\infty}e^{-c(\mathcal{M}/\mathcal{N})n}\right)=\mathcal{O}\left(\frac{\mathcal{N}}{\mathcal{M}}e^{-c\mathcal{M}^{2/3}}\right),\end{split}$
(33)

for some $c>0$, which is negligible; the remaining part can be handled with
$\theta\to 0$ asymptotic expressions: noting

$\begin{split}\left|\log
X^{2}_{\mathcal{M}}(\theta)-\log(1+e^{-4\xi\mathcal{M}\theta})\right|&\leq\left|\left(\frac{v^{\prime}}{v}\right)^{2}\alpha^{-4\mathcal{M}}-e^{-4\xi\mathcal{M}|\theta|}\right|\\\
&\leq\left|\left(\frac{v^{\prime}}{v}\right)^{2}-1\right|\alpha^{-4\mathcal{M}}+\left|\alpha^{-4\mathcal{M}}-e^{-4\xi\mathcal{M}|\theta|}\right|.\end{split}$
(34)

Using the asymptotic expansion in Equation 19 in Equation 22 gives

$\frac{v^{\prime}}{v}=\frac{2z_{1}\theta+\mathcal{O}(\theta^{2})}{\alpha-1+\mathcal{O}(\theta^{2})}=-\frac{\theta}{|\theta|}+\mathcal{O}(\theta)\implies\left(\frac{v^{\prime}}{v}\right)^{2}=1+\mathcal{O}(\theta),\quad\theta\to
0,$ (35)

and again using the asymptotics from Equation 19 this gives

$\left|\left(\frac{v^{\prime}}{v}\right)^{2}-1\right|\alpha^{-4\mathcal{M}}\leq
C\theta e^{-\mathcal{M}\theta},\quad|\theta|\leq\mathcal{M}^{-1/3};$ (36)

also

$\left|\alpha^{-4\mathcal{M}}-e^{-4\xi\mathcal{M}\theta}\right|\leq
4\mathcal{M}\left|\alpha-e^{\xi\theta}\right|\left[\min(\alpha,e^{\xi\theta})\right]^{-4\mathcal{M}-1}\leq
C\mathcal{M}\theta^{2}e^{-c\mathcal{M}\theta}$ (37)

and putting all this together (note that $X_{\mathcal{M}}$ is even in $\theta$
with $X_{\mathcal{M}}(0)=1$)

$\displaystyle\begin{split}\sum_{\theta\in Q_{+}(\mathcal{N})}\log
X_{\mathcal{M}}(\theta)&-\log\prod_{n=1}^{\infty}\left[1+\exp\left(-2\pi\xi\frac{\mathcal{M}}{\mathcal{N}}(2n-1)\right)\right]\\\
&=\mathcal{O}\left(\mathcal{N}\int_{0}^{\infty}(k+\mathcal{M}k^{2})e^{-k\mathcal{M}}\,\text{\rm
d}k\right)=\mathcal{O}\left(\frac{\mathcal{N}}{\mathcal{M}^{2}}\right)\end{split}$
(38) $\displaystyle\begin{split}\sum_{\theta\in Q_{-}(\mathcal{N})}\log
X_{\mathcal{M}}(\theta)&-\log\prod_{n=1}^{\infty}\left[1+\exp\left(-4\pi\xi\frac{\mathcal{M}}{\mathcal{N}}n\right)\right]\\\
&=\mathcal{O}\left(\mathcal{N}\int_{0}^{\infty}(k+\mathcal{M}k^{2})e^{-k\mathcal{M}}\,\text{\rm
d}k\right)=\mathcal{O}\left(\frac{\mathcal{N}}{\mathcal{M}^{2}}\right)\end{split}$
(39)

As noted in [6], the products appearing here can be expressed in terms of
Jacobi theta functions [5, Chapter 20, especially Equations (20.5.3–4)] as

$\displaystyle\prod_{n=1}^{\infty}\left[1+\exp\left(-2\pi\zeta(2n-1)\right)\right]^{2}$
$\displaystyle=\frac{\theta_{3}(e^{-2\pi\zeta})}{\theta_{0}(e^{-2\pi\zeta})},$
(40) $\displaystyle\prod_{n=1}^{\infty}\left[1+\exp\left(-4\pi\zeta
n\right)\right]^{2}$
$\displaystyle=e^{\frac{\pi}{2}\xi\frac{M}{N}}\frac{\theta_{2}(e^{-2\pi\zeta})}{2\theta_{0}(e^{-2\pi\zeta})}$
(41)

where $\zeta=\xi\mathcal{M}/\mathcal{N}$ and $\theta_{j}(q)$ is an
abbreviation for $\theta_{j}(0,q)$, and

$\theta_{0}(q):=\prod_{n=1}^{\infty}(1-q^{2n})=q^{-1/12}\left[\tfrac{1}{2}\theta_{2}(q)\theta_{3}(q)\theta_{4}(q)\right]^{1/3},$
(42)

see [5, Equations (20.4.6), (23.15.9), (23.17.8)].

Collecting all this,

$\sum_{\theta\in Q_{\pm}(\mathcal{N})}\log
X_{\mathcal{M}}(\theta)=x_{\pm}\left(\zeta\right)-w_{\pm}\frac{\mathcal{M}}{\mathcal{N}}+\mathcal{O}\left(\frac{\mathcal{N}}{\mathcal{M}^{2}}\right)$
(43)

where $w_{\pm}$ is as in Equation 32 and

$x_{+}(\xi)=\frac{1}{6}\log\left(\frac{2\theta_{3}^{2}(e^{-2\pi\xi})}{\theta_{2}(e^{-2\pi\xi})\theta_{4}(e^{-2\pi\xi})}\right),\quad
x_{-}(\xi):=\frac{1}{6}\log\left(\frac{\theta_{2}^{2}(e^{-2\pi\xi})}{4\theta_{3}(e^{-2\pi\xi})\theta_{4}(e^{-2\pi\xi})}\right).$
(44)

Equation 6 then follows by combining this with the other expressions obtained
above.

##  4 Conclusions

As a check on the calculation we can compare the asymptotics of this finite
size scaling terms with the predictions of conformal field theory, as has
already been done for the corresponding terms in a number of other boundary
conditions [14, 10] and for related expressions for cylindrical models [4,
16]. To do so, we need to examine the leading-order behavior of $x_{\pm}$ as
$\xi\to\infty$ (i.e. $\mathcal{M}>>\mathcal{N}$) and $\xi\to 0$ (i.e.
$\mathcal{N}>>\mathcal{M}$). For the former limit, where $q:=e^{-2\pi\xi}\to
0$, we use the series representations [5, Equations (20.2.2)–(20.2.4)] of the
theta functions to write

$\displaystyle\theta_{2}(q)\sim 2q^{1/4}$ (45) $\displaystyle\theta_{3}(q)\sim
1$ (46) $\displaystyle\theta_{4}(q)\sim 1,$ (47)

all for $q\to 0$, so that

$z_{+}(\xi)\sim\frac{1}{6}\log q^{-1/4}=\frac{\pi}{12}\xi,\quad
z_{-}(\xi)\sim\frac{1}{6}\log q^{1/2}=-\frac{\pi}{6}\xi,$ (48)

corresponding to the prediction of [8, especially Table 1] for an infinite
tube with periodic or antiperiodic boundary conditions and the central charge
$c=1/2$ of the Ising model.

For the limit $\xi\to 0$, we use Jacobi’s imaginary transformation ([5,
Equations (20.7.31-33)]; note the use of the parameter $\tau$ with
$q=e^{i\pi\tau}$, so $\tau=2i\zeta$ and $\tau^{\prime}=1/\tau=i/2\zeta$),
which for the quantities at hand takes the form

$\displaystyle\theta_{2}(e^{-\pi x})=x^{-1/2}\theta_{4}(e^{-\pi/x})$ (49)
$\displaystyle\theta_{3}(e^{-\pi x})=x^{-1/2}\theta_{3}(e^{-\pi/x})$ (50)
$\displaystyle\theta_{4}(e^{-\pi x})=x^{-1/2}\theta_{2}(e^{-\pi/x}),$ (51)

to rewrite Equation 44 as

$\displaystyle z_{+}(\xi)$
$\displaystyle=\frac{1}{6}\log\left(\frac{2\theta_{3}^{2}\left(e^{-\pi/2\xi}\right)}{\theta_{2}\left(e^{-\pi/2\xi}\right)\theta_{4}\left(e^{-\pi/2\xi}\right)}\right)\sim\frac{\pi}{48}\xi^{-1}$
(52) $\displaystyle z_{-}(\xi)$
$\displaystyle=\frac{1}{6}\log\left(\frac{\theta_{4}^{2}\left(e^{-\pi/2\xi}\right)}{4\theta_{3}\left(e^{-\pi/2\xi}\right)\theta_{4}\left(e^{-\pi/2\xi}\right)}\right)\sim\frac{\pi}{48}\xi^{-1}$
(53)

so that in this limit both versions give the same form predicted for an
infinite strip with open boundary conditions.

##  Appendix A Acknowledgements

This paper arose out of discussions with Alessandro Giuliani. The work was
supported by the MIUR Excellence Department Project MatMod@TOV awarded to the
Department of Mathematics, University of Rome Tor Vergata, CUP
E83C23000330006.

## References

  * [1] G. R. Cava, A. Giuliani and R. L. Greenblatt,  _The scaling limit of boundary spin correlations in non-integrable Ising models_ ,  In preparation.
  * [2] D. Chelkak, D. Cimasoni and A. Kassel,  _Revisiting the combinatorics of the 2D Ising model_ ,  Annales de l’Institut Henri Poincaré D 4, 309 (2017),  10.4171/AIHPD/42.
  * [3] G. Antinucci, A. Giuliani and R. L. Greenblatt,  _Non-integrable Ising models in cylindrical geometry: Grassmann representation and infinite volume limit_ ,  Annales Henri Poincaré (2021),  10.1007/s00023-021-01107-3.
  * [4] H. Hobrecht and A. Hucht,  _Critical Casimir force scaling functions of the two-dimensional Ising model at finite aspect ratios_ ,  Journal of Statistical Mechanics: Theory and Experiment 2017(2), 024002 (2017),  10.1088/1742-5468/aa5280.
  * [5] _NIST digital library of mathematical functions_ ,  https://dlmf.nist.gov/, Release 1.2.1 of 2024-06-15,  F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain, eds.
  * [6] A. E. Ferdinand and M. E. Fisher,  _Bounded and inhomogeneous Ising models. I. Specific-heat anomaly of a finite lattice_ ,  Physical Review 185(2), 832 (1969),  10.1103/PhysRev.185.832.
  * [7] J. L. Cardy,  _Conformal invariance and universality in finite-size scaling_ ,  Journal of Physics A: Mathematical and General 17(7), L385 (1984),  10.1088/0305-4470/17/7/003.
  * [8] J. L. Cardy,  _Effect of boundary conditions on the operator content of two-dimensional conformally invariant theories_ ,  Nuclear Physics B 275(2), 200 (1986),  10.1016/0550-3213(86)90596-1.
  * [9] I. Affleck,  _Universal term in the free energy at a critical point and the conformal anomaly_ ,  Physical Review Letters 56(7), 746 (1986),  10.1103/PhysRevLett.56.746.
  * [10] N. S. Izmailian, K. Oganesyan and C.-K. Hu,  _Exact finite-size corrections for the square-lattice Ising model with Brascamp-Kunz boundary conditions_ ,  Physical Review E 65(5), 056132 (2002),  10.1103/PhysRevE.65.056132.
  * [11] A. Hucht,  _The square lattice Ising model on the rectangle II: finite-size scaling limit_ ,  Journal of Physics A: Mathematical and Theoretical 50(26), 265205 (2017),  10.1088/1751-8121/aa6b7a.
  * [12] N. Izmailian,  _Universal amplitude ratios for scaling corrections on Ising strips with fixed boundary conditions_ ,  Nuclear Physics B 854(1), 184 (2012),  10.1016/j.nuclphysb.2011.08.019.
  * [13] D. Cimasoni,  _The dimer and Ising models on Klein bottles_ ,  Annales de l’Institut Henri Poincaré D 11, 503 (2024),  10.4171/AIHPD/166.
  * [14] W. T. Lu and F. Wu,  _Ising model on nonorientable surfaces: Exact solution for the Möbius strip and the Klein bottle_ ,  Physical Review E 63(2), 026107 (2001),  10.1103/PhysRevE.63.026107.
  * [15] R. L. Greenblatt,  _The ising model on a cylinder: universal finite size corrections and diagonalized action_ (2014), 1409.1852.
  * [16] H. Hobrecht and F. Hucht,  _Anisotropic scaling of the two-dimensional Ising model II: surfaces and boundary fields_ ,  SciPost Physics 8(3), 032 (2020),  10.21468/SciPostPhys.8.3.032.
  * [17] A. E. Ferdinand,  _Statistical mechanics of dimers on a quadratic lattice_ ,  Journal of Mathematical Physics 8(12), 2332 (1967),  10.1063/1.1705162.
  * [18] N. S. Izmailian, K. Oganesyan and C.-K. Hu,  _Exact finite-size corrections of the free energy for the square lattice dimer model under different boundary conditions_ ,  Physical Review E 67(6), 066114 (2003),  10.1103/PhysRevE.67.066114.
  * [19] P. Bleher, B. Elwood and D. Petrović,  _Dimer model: Full asymptotic expansion of the partition function_ ,  Journal of Mathematical Physics 59(9), 091407 (2018),  10.1063/1.5035271.
  * [20] J. Dubédat and R. Gheissari,  _Asymptotics of height change on toroidal Temperleyan dimer models_ ,  Journal of Statistical Physics 159(1), 75 (2015),  10.1007/s10955-014-1181-x.
  * [21] R. L. Greenblatt,  _Discrete and zeta-regularized determinants of the Laplacian on polygonal domains with Dirichlet boundary conditions_ ,  Journal of Mathematical Physics 64(4) (2023),  10.1063/5.0062138.
  * [22] K. Izyurov and M. Khristoforov,  _Asymptotics of the determinant of discrete Laplacians on triangulated and quadrangulated surfaces_ ,  Communications in Mathematical Physics 394(2), 531 (2022),  10.1007/s00220-022-04437-3.
  * [23] B. McCoy and T. Wu,  _The two-dimensional Ising model_ ,  Harvard University Press (1973).
  * [24] R. Shankar and G. Murthy,  _Nearest-neighbor frustrated random-bond model in $d=2$: Some exact results_,  Physical Review B 36(1), 536 (1987),  10.1103/PhysRevB.36.536.
  * [25] F. Comets, G. Giacomin and R. L. Greenblatt,  _Continuum limit of random matrix products in statistical mechanics of disordered systems_ ,  Communications in Mathematical Physics 369, 171 (2019),  10.1007/s00220-019-03466-9.
  * [26] G. Giacomin and R. L. Greenblatt,  _Lyapunov exponent for products of random Ising transfer matrices: the balanced disorder case_ ,  ALEA 19, 701 (2022),  10.30757/ALEA.v19-27.
  * [27] A. Hucht,  _The square lattice Ising model on the rectangle I: finite systems_ ,  Journal of Physics A: Mathematical and Theoretical 50(6), 065201 (2017),  10.1088/1751-8121/aa5535.


# Optomagnetism with plasmonic skyrmion

Vage Karakhanyan  Optics Department, FEMTO-ST Institute UMR 6174, University
of Franche-Comté – CNRS
Besançon, France  Thierry Grosjean  Optics Department, FEMTO-ST Institute UMR
6174, University of Franche-Comté – CNRS
Besançon, France  Corresponding author email<EMAIL_ADDRESS>
—————————— Abstract ——————————
Research at the frontier between optics and magnetism is revealing a wealth of
innovative phenomena and avenues of exploration. Optical waves are
demonstrating the capacity to induce ultrafast magnetism, while optical
analogs of magnetic states, such as magnetic skyrmions, offer the prospect of
novel spin-optical states. In this paper, we strengthen the synergy between
light and magnetism by exploring the ability of plasmonic Neel skyrmions to
create a stationary magnetic field within a thin gold film. We show that, when
generated using a focused radially-polarized vortex beam, a plasmonic Neel
skyrmion emerges as an optimum for inducing optomagnetism in a thin gold film.
Optical skyrmions offer new degrees of freedom for enhancing and controlling
optomagnetism in plasmonic nanostructures, with direct application in all-
optical magnetization switching, magnetic recording, and the excitation of
spin waves.

—————————————————————————

Magnetic skyrmions are topologically-protected magnetic states showing space-
variant spin distributions [1]. Since their first observation in 2009 [2, 3,
4] these spin textures have attracted much interest from both fundamental
aspects and potential applications in novel spintronic devices.

Recently, the concept of magnetic skyrmion has been extended to optics,
leading to novel optical-field or optical-spin textures on the subwavelength
scale [5, 6]. As their magnetic counterpart, optical skyrmions of different
structures can be densely packed in square or hexagonal lattices [5, 7].
Optical skyrmions are a promising building block of the emerging spin-based
optics, including optical nano-imaging, quantum information processing,
metrology and data storage.

In this paper, we study the ability of a Neel-type optical skyrmion to
generate a stationary magnetic field via the inverse Faraday effect (IFE) [8,
9, 10]. The IFE has been extensively explored due to its potential to generate
ultrafast magnetic data storage [11, 12, 13] and non-contact excitation of
spin-waves [14, 15, 16, 17, 18]. Moreover, the IFE can be enhanced and
tailored via the excitation of surface plasmons in noble metals [19, 20, 21,
22, 23, 24, 25, 26, 27, 28] and in hybrid structures combining noble metals
and magnetic materials [29, 30, 31, 32]. Since the IFE is induced by both the
SAM and OAM of light, spin-orbit interaction plays a significant role in the
optomagnetism [33]. The spin texture of an optical Neel skyrmion, which is
also strongly influenced by spin-orbit interaction [6, 5, 34], presents new
avenues for generating and controlling optomagnetism. We show here that
plasmonic Neel Skyrmion holds promise to maximize optomagnetic effects in a
thin gold film. The optical counterpart of magnetic skyrmions offers
intriguing solutions to increase and control optomagnetic effects is
nanostructures.

A plasmonic Neel skyrmion is a metal/air surface mode whose electric field or
spin angular momentum (SAM) density is both topologically invariant and
radially distributed along the surface. Following approaches introduced by Du
et al. [6] and Tsesses et al. [5], we generate optical Neel skyrmions by
tightly focusing either a radially-polarized vortex beam (RPVB) of topological
charge $l=\pm$1 or a circularly polarized beam (CPB) onto a thin gold film
lying on a glass substrate (Fig. 1(a)). RPVB belong to the family of the
vector vortex beams, characterized by an inhomogeneous vector polarization
state and a helical phase [35, 36, 37]. Focused circularly polarized fields
are employed as a standard for inducing IFE [8, 9, 10, 11, 12, 13, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28].

The waist of these incoming paraxial beams, characterized by a Gaussian
profile, is projected onto the exit pupil plane of a microscope objective
where it is spatially filtered with an narrow annular slit within an opaque
screen. The annular pupil restricts the incidence angles of incoming light
waves to a narrow range around a mean value defined by the slit diameter. This
angular range is set either at 1∘ or 0.1∘, regardless of the average incidence
angle. The $1/e$ width of the beam waist coincides with the pupil diameter of
the microscope objective, whose numerical aperture (NA) of 1.3 enables light
focusing in the substrate beyond the critical angle.

Using the theory established by Richards and Wolf [38, 39], the electric
optical field at focus reads:

$\begin{split}&\textbf{E}(r,\xi,z)=-\frac{ik_{1}f\exp[-ik_{1}f]}{2\pi}\frac{1}{\sqrt{n_{1}}}\times\\\
&\int^{\theta_{0}+\frac{\Delta\theta}{2}}_{\theta_{0}-\frac{\Delta\theta}{2}}G(\theta)\int^{2\pi}_{0}\textbf{e}(\theta,\psi,z)\exp\left[i\alpha
r\cos(\psi-\xi)\right]d\theta d\psi,\end{split}$ (1)

where $G(\theta)=\cos^{\frac{1}{2}}(\theta)\sin(\theta)F(\theta)$ $(r,\xi,z)$
are cylindrical coordinates, $f$ is the focal length of the microscope
objective, $\theta$ and $\psi$ are directional angles and $\alpha$ is a
function of $\theta$. $\Delta\theta$ is the focusing angular range.

For the RPVB of the first order, the vector $\textbf{e}(\theta,\psi,z)$ takes
the form:

$\displaystyle\mathbf{e}(\theta,\psi,z)$
$\displaystyle=\mathbf{e_{TM}}(\theta,\psi,z)$ (2)
$\displaystyle\propto\exp[\pm
i\psi]\begin{pmatrix}-t^{r}_{TM}(z)\cos\theta\cos\psi\\\
-t_{TM}^{r}(z)\cos\theta\sin\psi\\\ t_{TM}^{z}(z)\sin\theta\end{pmatrix}$

$t_{TM}^{r}$ and $t_{TM}^{z}$ are coefficients under the form
$C_{TM}^{+}\exp[iw_{2}z]+C_{TM}^{-}\exp[-iw_{2}z]$ where $w_{2}$ is the
component of the wave vector normal to the surfaces and $i=\sqrt{-1}$.
Coefficients $C_{TM}^{+}$ and $C_{TM}^{-}$ are obtained by applying boundary
conditions of the optical fields at the metal surfaces (see for instance in
Ref. [40]).

For a CPB, the vector $\textbf{e}(\theta,\psi,z)$ is written as:

$\mathbf{e}(\theta,\psi,z)\propto-\frac{\sqrt{2}}{2}\left[\mathbf{e_{TM}}(\theta,\psi,z)+i\mathbf{e_{TE}}(\theta,\psi,z)\right],$
(3)

where,

$\mathbf{e_{TE}}(\theta,\psi,z)=t_{TE}(z)\exp[\pm
i\psi]\begin{pmatrix}-\sin\psi\\\ \cos\psi\\\ 0\end{pmatrix}.$ (4)

$t_{TE}$ is a coefficient under the form
$C_{TE}^{+}\exp[iw_{2}z]+C_{TE}^{-}\exp[-iw_{2}z]$. Coefficients $C_{TE}^{+}$
and $C_{TE}^{-}$ are obtained by applying boundary conditions of the optical
fields at the metal surfaces. The phase term $exp[\pm i\psi]$ in Eq. 4 reveals
spin-orbit interaction at focus [41].

The apodization function at the pupil plane of the microscope objective is
approximated by the function:

$F(\theta)=\frac{2}{w_{0}}\sqrt{\frac{Z_{0}P_{0}}{\pi}}\exp\left[\frac{-f^{2}\sin^{2}\theta}{w_{0}^{2}}\right],$
(5)

with $w_{0}=f\sin\theta_{M}$.

$P_{0}$ and $w_{0}$ are the power and $1/e$ width of the incoming paraxial
beam and $Z_{0}$ is the vacuum impedance. The optical magnetic field is
calculated by replacing $\textbf{e}(r,\xi,z)$ by
$\textbf{h}(r,\xi,z)=\textbf{k}\times\textbf{e}(r,\xi,z)/\omega\mu_{0}$ in Eq.
1. $\omega$ and $\mu_{0}$ are the angular frequency and permeability of
vacuum, respectively.

At focus, the SAM density of the fields transmitted through the gold film is
defined as:

$\mathbf{s}=\left[\operatorname{Im}(\varepsilon_{0}\mathbf{E}\times\mathbf{E^{*}})+\operatorname{Im}(\mu_{0}\mathbf{H}\times\mathbf{H^{*}})\right]/2\omega,$
(6)

where $\varepsilon_{0}$ is the permittivity of vacuum [42, 43]. The SAM
density can also be expressed as:

$\mathbf{s}=\frac{w}{\omega}\boldsymbol{\sigma},$ (7)

where
$w=1/2\left[\varepsilon_{0}|\mathbf{E}|^{2}+\mu_{0}|\mathbf{H}|^{2}\right]$ is
the energy density and $\boldsymbol{\sigma}$ is the local polarization
helicity vector. The three components of $\boldsymbol{\sigma}$ are comprised
between -1 and 1, the values 0 and $\pm 1$ correspond to the linear and
circular polarization states, respectively.

To confirm the analogy to magnetic skyrmions, we calculate the skyrmion number
($n$) associated with the SAM distribution:

$n=\dfrac{1}{4\pi}\int_{A}\mathbf{S_{n}}\cdot\left(\dfrac{\partial\mathbf{S_{n}}}{\partial
x}\times\dfrac{\partial\mathbf{S_{n}}}{\partial y}\right)dxdy,$ (8)

where $S_{n}$ is the unit vector in the direction of the local SAM, $A$ is the
area of a unit cell of the skyrmion. When $n=1$, the photonic spin structure
is the analogue to the magnetization texture of either a Neel or a Bloch
skyrmion in magnetic materials [44, 45]. An analysis of the vector spin
distribution allows to identify the exact nature of the optical skyrmion.

On the basis of a hydrodynamic model of the free electrons in a metal, we have
recently proposed a new framework to describe the IFE in plasmonic
nanostructures [27, 28, 33]. In the present study, the opto-induced drift
current densities within the metal film are represented as follows:

$\begin{split}J^{b}_{\xi}\approx\frac{2\mu_{0}|\gamma_{\omega}|^{2}}{n_{0}e}\langle\mathbf{\Pi}\rangle_{\xi},\end{split}$
(9)

in the metal bulk, and:

$J^{s}_{\xi}\approx\pm\frac{|\gamma_{\omega}|^{2}}{n_{0}e\omega}\operatorname{Im}\left[E_{\omega}^{z}(0^{-})E_{\omega}^{\xi\ast}(0^{-})\right],$
(10)

at the two metal surfaces [27, 33]. $\langle\mathbf{\Pi}\rangle_{\xi}$ is the
azimuthal component of the time averaged Poynting vector
$\langle\mathbf{\Pi}\rangle=0.5\operatorname{Re}\left(\mathbf{E}\times\mathbf{H^{*}}\right)$.
$E_{\omega}^{z}(0^{-})$ and $E_{\omega}^{\xi\ast}(0^{-})$ are the longitudinal
and azimuthal components of the optical electric field at the two metal
surfaces. The $+$ and $-$ sign in Eq. 10 corresponds to the current densities
at the lower and upper interfaces of the metal film, respectively. The
optomagnetic field is deduced from Eqs. 9 and 10 by using the Biot and Savart
law.

Figure 1: (a) Schematic diagram of the focusing system designed to generate a
Neel-type plasmonic skyrmion. (b) Blue line: reflectance of the thin gold film
as a function of the incidence angle of a $TM$-polarized plane wave in the
glass substrate ($\lambda$=800 nm). The vertical dashed line identifies the
critical angle of total internal reflection. Red and green lines: skyrmion
number of the field transmitted through the thin gold film versus the mean
incidence angle of the focused field, for the RPVB and the CPB, respectively.
The skyrmion number is calculated at a distance of 10 nm beyond the top
surface. Focusing occurs across an angular range $\Delta\theta=0.1^{\circ}$.
(c-e) SAM structure of the transmitted optical fields at plasmon resonance,
for an incoming RPVB: vector components of the polarization helicity
$\boldsymbol{\sigma}$ (Eq. 7) at an incidence angle of 43.25∘, with
$\Delta\theta=1^{\circ}$.

The calculations were conducted at a wavelength of $\lambda=800$ nm. We
identified a surface plasmon excitation at an incidence angle $\theta$ of
43.25∘ (following the Kretschmann configuration; see Fig. 1(b)). Figure 1(b-e)
shows an analysis of the vector spin structure generated at a distance of 10
nm beyond the gold film’s upper interface by focusing RPVB and CPB. With a
skyrmion number of 1 beyond the critical angle of 41.8∘ (Fig. 1(b)), the
photonic spin structures generated at plasmon resonance by both beams are an
optical analogue to a magnetic skyrmion. The radially-distributed spin texture
observed in Figs. 1(c-e) for the RPVB identifies a plasmonic Neel skyrmion. A
similar spin texture is obtained for circular polarization. Outside the
plasmon resonance, below the critical angle, the SAM texture transitions to an
azimuthally polarized state, akin to the magnetization texture observed in
Bloch skyrmions. The skyrmion number is however insufficient to categorize the
optical field as an optical analogue to the Bloch skyrmion.

Figure 2: (a) Blue curve: Reflectance of the metal film versus the incidence
angle of a $TM$-polarized plane wave in the glass substrate ($\lambda$=800
nm). Red and green curves: total energy of the optomagnetic field upon
illumination with a focused RPVB (red line) or a CPB (green line),
respectively. (b-e): Amplitude of the optomagnetic field in a longitudinal
cross-section, for (b,d) a focused RPVB of the first order, and (c,e) a
focused CPB. The stationary magnetic fields are calculated (b,c) on resonance
and (d,e) off-resonance, with $\Delta\theta=1^{\circ}$ falling within the
shaded regions depicted in (a).

Fig. 2 shows the optomagnetic response of the thin gold film upon illumination
with focused RPVB and CPB, with $\Delta\theta=1^{\circ}$. Upon scanning the
average incidence angle, the total optomagnetic energy (defined in the inset
of Fig. 2(a)) peaks right at the plasmon angle for both incident polarization
states, coinciding with the minimum optical reflectance (Fig. 2(a)). At
surface plasmon excitation, the optomagnetic energy is enhanced by
approximately 13-fold and 128-fold for RPVB and CPB, respectively. This
confirms the role of surface plasmons in amplifying optomagnetism in metals.
However, despite their common skyrmion nature at plasmon resonance, the
transmitted fields obtained with the RPVB and CPB result in distinct levels of
optomagnetism: the optomagnetic energy is 11.5 times larger with the plasmonic
Neel-type skyrmion produced by the RPVB.

Fig. 2(b) and (c) show the optomagnetic field generated when a plasmonic Neel
skyrmion is applied to the thin gold film. Using the focused RPVB (Fig. 2(b)),
the maximum optomagnetic field is localized at the upper surface of the gold
film, at the surface plasmon resonance. Off-resonance (Fig. 2(d)), the
optomagnetic field is confined at the lower interface with its maximum reduced
by a factor of 3. As a comparison, the optomagnetic fields generated on and
off plasmon resonance with the focused CPB are attenuated by 4 and 30 times,
respectively (see Figs. 2(c) and 2(e), respectively). The optomagnetic fields
obtained at plasmon resonance for both polarizations show similar spatial
distributions (Figs. 2(b) and (c)). A similar morphology of optomagnetic field
remains off-resonance for the RPVB (cf. Figs. 2(b) and (d)), but is lost with
the CPB (cf. Figs. 2(c) and (e)).

Figure 3: (a-d) Normalized current densities at (a,b) the upper and (c,d)
lower surfaces of the thin gold film, with (a,c) a focused RPVB and (b,d) a
focused CPB. (e,f) Normalized volume current densities in a (xz) cross-section
perpendicular to the surfaces, with focused RPVB and CPB, respectively.

According to Eqs. 1 and 2, a focused RPVB generates an axis-symmetrical ”$TM$
optical state” within the metal film, which is constituted of $TM$-polarized
Fourier components. In contrast, when a focused CPB is applied, a combination
of axis-symmetrical $TE$ and $TM$ optical states arises within the metal film
(see Eqs. 1, 3 and 4). The ”$TE$ optical state”, constituted of $TE$-polarized
Fourier components, is primarily confined within the skin depth at the lower
interface.

In the case of the RPVB, the entirety of the incident field intensity
contributes to the generation of the $TM$ optical state within the metal film,
which includes surface plasmons (cf. Eq 2). By comparing Eqs. 2 and 3, we see
that only half of the incident field intensity is involved for circular
polarization. As the surface and volume current densities stem from a second-
order nonlinear optical effect (see Eqs. 9 and 10) [27], the plasmonic Neel
skyrmion produced with the RPVB exhibits twice the efficiency in generating
opto-induced drift currents compared to the one obtained with the CPB, as
shown in Fig. 3. Since the $TE$ optical state is confined within the lower
skin depth, the plasmon-induced drift current densities exhibit similar
spatial distributions in the upper half of the gold film, as shown in Figs.
3(a,b) and 3(e,f). However, circular polarization results in a maximum
amplitude of the drift current densities that is halved.

With the focused RPVB, the plasmonic Neel skyrmion generates surface and
volume drift current densities of the same handedness, forming a uniform
current loop (see Figs. 3(a,c,e)). In contrast, the presence of a $TE$ optical
state with the CPB induces a pair of current loops with opposite handedness in
the lower half of the metal film, which is less efficient in generating an
optomagnetic field (Figs. 3(d,f)). Therefore, the existence of $TE$ optical
states in the plasmonic film both attenuates amplitude of the plasmon field
and hinders the optomagnetic effect in the lower half of the metal film. As a
result, in our study, the optomagnetic field is reduced by four with the
focused CPB compared to the focused RPVB (which lacks $TE$ optical state),
although an optical Neel skyrmion is transmitted through the metal film in
both cases. Therefore, by creating a pure $TM$ optical state in a thin gold
film, the plasmonic Neel skyrmion obtained with the RPVB emerges as an optimum
for generating plasmon-induced magnetism.

To conclude, an optical counterpart of magnetic Neel skyrmion is shown to
generate a static magnetic field in a thin gold film, via plasmon-enhanced
IFE. We evidence that circular polarization, the standard for producing IFE,
is not the optimal choice for inducing plasmon-enhanced optomagnetism in a
thin gold film. The optimum is achieved with a plasmonic Neel skyrmion
obtained from a vector vortex beam. Vector vortex beams, which produce optical
skyrmions by exploiting spin-orbit interaction, open new avenues for
generating and controlling plasmon-induced magnetism in metal films, and by
extension, in resonant plasmonic nanoantennas.

## Funding

EIPHI Graduate School (contract ANR-17-EURE-0002); Region ”Bourgogne Franche-
Comte”; French Agency of Research (contracts ANR-18-CE42-0016 and
ANR-23-CE42-0021); French Renatech network; Equipex+ Nanofutur (21-ESRE-0012)

## Disclosures

The authors declare no conflicts of interest.

## References

  * [1] Albert Fert, Vincent Cros and Joao Sampaio  “Skyrmions on the track”  In _Nature nanotechnology_ 8.3 Nature Publishing Group UK London, 2013, pp. 152–156
  * [2] Sebastian Muhlbauer et al.  “Skyrmion lattice in a chiral magnet”  In _Science_ 323.5916 American Association for the Advancement of Science, 2009, pp. 915–919
  * [3] A Neubauer et al.  “Topological Hall effect in the A phase of MnSi”  In _Physical review letters_ 102.18 APS, 2009, pp. 186602
  * [4] C Pappas et al.  “Chiral paramagnetic skyrmion-like phase in MnSi”  In _Physical review letters_ 102.19 APS, 2009, pp. 197202
  * [5] S Tsesses et al.  “Optical skyrmion lattice in evanescent electromagnetic fields”  In _Science_ 361.6406 American Association for the Advancement of Science, 2018, pp. 993–996
  * [6] Luping Du, Aiping Yang, Anatoly V Zayats and Xiaocong Yuan  “Deep-subwavelength features of photonic skyrmions in a confined electromagnetic field with orbital angular momentum”  In _Nat. Phys._ 15.7 Nature Publishing Group, 2019, pp. 650–654
  * [7] Xinrui Lei et al.  “Photonic spin lattices: symmetry constraints for skyrmion and meron topologies”  In _Phys. Rev. Lett._ 127.23 APS, 2021, pp. 237403
  * [8] PS Pershan, JP Van der Ziel and LD Malmstrom  “Theoretical discussion of the inverse Faraday effect, Raman scattering, and related phenomena”  In _Physical review_ 143.2 APS, 1966, pp. 574
  * [9] Daria Popova, Andreas Bringer and Stefan Blügel  “Theory of the inverse Faraday effect in view of ultrafast magnetization experiments”  In _Phys. Rev. B_ 84.21 APS, 2011, pp. 214421
  * [10] Riccardo Hertel  “Theory of the inverse Faraday effect in metals”  In _J. Magn. Magn. Mater._ 303.1 Elsevier, 2006, pp. L1–L4
  * [11] E Beaurepaire, J-C Merle, A Daunois and J-Y Bigot  “Ultrafast spin dynamics in ferromagnetic nickel”  In _Phys. Rev. Lett._ 76.22 APS, 1996, pp. 4250
  * [12] CD Stanciu et al.  “All-optical magnetic recording with circularly polarized light”  In _Phys. Rev. Lett._ 99.4 APS, 2007, pp. 047601
  * [13] Andrei Kirilyuk, Alexey V Kimel and Theo Rasing  “Ultrafast optical manipulation of magnetic order”  In _Rev. Mod. Phys._ 82.3 APS, 2010, pp. 2731
  * [14] AV Kimel et al.  “Ultrafast non-thermal control of magnetization by instantaneous photomagnetic pulses”  In _Nature_ 435.7042 Nature Publishing Group, 2005, pp. 655–657
  * [15] AM Kalashnikova et al.  “Impulsive excitation of coherent magnons and phonons by subpicosecond laser pulses in the weak ferromagnet FeBO 3”  In _Phys. Rev. B_ 78.10 APS, 2008, pp. 104301
  * [16] Takuya Satoh et al.  “Directional control of spin-wave emission by spatially shaped light”  In _Nat. Photonics_ 6.10 Nature Publishing Group, 2012, pp. 662–666
  * [17] IV Savochkin et al.  “Generation of spin waves by a train of fs-laser pulses: a novel approach for tuning magnon wavelength”  In _Sci. Rep._ 7.1 Nature Publishing Group, 2017, pp. 1–10
  * [18] Keita Matsumoto et al.  “Observation of evanescent spin waves in the magnetic dipole regime”  In _Phys. Rev. B_ 101.18 APS, 2020, pp. 184407
  * [19] Igor I Smolyaninov et al.  “Plasmon-induced magnetization of metallic nanostructures”  In _Phys. Rev. B_ 71.3 APS, 2005, pp. 035425
  * [20] Yu Gu and Konstantin G Kornev  “Plasmon enhanced direct and inverse Faraday effects in non-magnetic nanocomposites”  In _JOSA B_ 27.11 Optical Society of America, 2010, pp. 2165–2173
  * [21] Oscar Hsu-Cheng Cheng, Dong Hee Son and Matthew Sheldon  “Light-induced magnetism in plasmonic gold nanoparticles”  In _Nat. Photon._ 14.6 Nature Publishing Group, 2020, pp. 365–368
  * [22] KL Koshelev, V Yu Kachorovskii and M Titov  “Resonant inverse Faraday effect in nanorings”  In _Phys. Rev. B_ 92.23 APS, 2015, pp. 235426
  * [23] SM Hamidi, M Razavinia and MM Tehranchi  “Enhanced optically induced magnetization due to inverse Faraday effect in plasmonic nanostructures”  In _Opt. Commun._ 338 Elsevier, 2015, pp. 240–245
  * [24] Athavan Nadarajah and Matthew T Sheldon  “Optoelectronic phenomena in gold metal nanostructures due to the inverse Faraday effect”  In _Opt. Express_ 25.11 Optical Society of America, 2017, pp. 12753–12764
  * [25] Jérôme Hurst, Peter M Oppeneer, Giovanni Manfredi and Paul-Antoine Hervieux  “Magnetic moment generation in small gold nanoparticles via the plasmonic inverse Faraday effect”  In _Phys. Rev. B_ 98.13 APS, 2018, pp. 134439
  * [26] Ritwik Mondal et al.  “Relativistic interaction Hamiltonian coupling the angular momentum of light and the electron spin”  In _Phys. Rev. B_ 92.10 APS, 2015, pp. 100402
  * [27] Vage Karakhanyan, Yannick Lefier, Clément Eustache and Thierry Grosjean  “Optomagnets in nonmagnetic plasmonic nanostructures”  In _Opt. Lett._ 46.3 OSA, 2021, pp. 613–616  DOI: 10.1364/OL.411108
  * [28] Vage Karakhanyan, Clément Eustache, Yannick Lefier and Thierry Grosjean  “Plasmon-induced 0.13 T optomagnetic field in a gold coaxial nanoaperture”  In _OSA Continuum_ 4.5 OSA, 2021, pp. 1598–1608  URL: http://www.osapublishing.org/osac/abstract.cfm?URI=osac-4-5-1598
  * [29] Alan Hwader Chu et al.  “Hybrid magneto photonic material structure for plasmon assisted magnetic switching”  In _Opt. Mat. Express_ 10.12 Optical Society of America, 2020, pp. 3107–3118
  * [30] DO Ignatyeva et al.  “Plasmonic layer-selective all-optical switching of magnetization with nanometer resolution”  In _Nat. Commun._ 10.1 Nature Publishing Group, 2019, pp. 1–7
  * [31] Song-Jin Im et al.  “All-optical magnetization switching by counterpropagataion or two-frequency pulses using the plasmon-induced inverse Faraday effect in magnetoplasmonic structures”  In _Phys. Rev. B_ 99.4 APS, 2019, pp. 041401
  * [32] Feng Cheng et al.  “All-Optical Manipulation of Magnetization in Ferromagnetic Thin Films Enhanced by Plasmonic Resonances”  In _Nano Lett._ 20.9 ACS Publications, 2020, pp. 6437–6443
  * [33] Vage Karakhanyan, Clément Eustache, Yannick Lefier and Thierry Grosjean  “Inverse Faraday effect from the orbital angular momentum of light”  In _Physical Review B_ 105.4 APS, 2022, pp. 045406
  * [34] Luping Du, Aiping Yang, Anatoly V Zayats and Xiaocong Yuan  “Deep-subwavelength features of photonic skyrmions in a confined electromagnetic field with orbital angular momentum”  In _Nature Physics_ 15.7 Nature Publishing Group UK London, 2019, pp. 650–654
  * [35] Xiang Hao, Cuifang Kuang, Tingting Wang and Xu Liu  “Phase encoding for sharper focus of the azimuthally polarized beam”  In _Optics letters_ 35.23 Optical Society of America, 2010, pp. 3928–3930
  * [36] Zhe Zhao, Jian Wang, Shuhui Li and Alan E Willner  “Metamaterials-based broadband generation of orbital angular momentum carrying vector beams”  In _Optics letters_ 38.6 Optical Society of America, 2013, pp. 932–934
  * [37] Cheng-Wei Qiu et al.  “Engineering light-matter interaction for emerging optical manipulation applications”  In _Nanophotonics_ 3.3 De Gruyter, 2014, pp. 181–201
  * [38] Bernard Richards and Emil Wolf  “Electromagnetic diffraction in optical systems, II. Structure of the image field in an aplanatic system”  In _Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences_ 253.1274 The Royal Society London, 1959, pp. 358–379
  * [39] Lukas Novotny and Bert Hecht  “Principles of nano-optics”  Cambridge university press, 2012
  * [40] Max Born and Emil Wolf  “Principles of optics: electromagnetic theory of propagation, interference and diffraction of light”  Elsevier, 2013
  * [41] Konstantin Y Bliokh, Daria Smirnova and Franco Nori  “Quantum spin Hall effect of light”  In _Science_ 348.6242 American Association for the Advancement of Science, 2015, pp. 1448–1451
  * [42] Michael V Berry  “Optical currents”  In _J.Opt. A_ 11.9 IOP Publishing, 2009, pp. 094001
  * [43] Konstantin Y Bliokh, Aleksandr Y Bekshaev and Franco Nori  “Extraordinary momentum and spin in evanescent waves”  In _Nat. Commun._ 5.1 Nature Publishing Group UK London, 2014, pp. 3300
  * [44] Sandip Bera and Sudhansu S Mandal  “Theory of the skyrmion, meron, antiskyrmion, and antimeron in chiral magnets”  In _Physical Review Research_ 1.3 APS, 2019, pp. 033109
  * [45] XZ Yu et al.  “Transformation between meron and skyrmion topological spin textures in a chiral magnet”  In _Nature_ 564.7734 Nature Publishing Group UK London, 2018, pp. 95–98


11institutetext: Multimedia Research Centre, University of Alberta, 116 St 85
Ave, Edmonton, AB, Canada
11email<EMAIL_ADDRESS>
http://crome.cs.ualberta.ca/mrc/

# Feature CAM: Interpretability in Image Classification with Improved Visual
Attention

Frincy Clement 11 0000-0003-0382-653X Ji Yang 22 0000-0001-9302-4738 Irene
Cheng 11 0000-0001-9699-4895

###### Abstract

Deep Neural Networks have often been called ‘the black box’ because of the
complex, deep architecture and non-transparency presented by the inner layers.
There is a lack of trust to use Artificial Intelligence in critical and high-
precision fields such as security, finance, health, and manufacturing
industries. A lot of focused work has been done to provide interpretable
models, intending to deliver meaningful insights into the thoughts and
behavior of neural networks.

In our research, we compare the state-of-the-art methods in the Activation-
based methods (ABM) for interpreting predictions of CNN models, specifically
in the application of Image Classification. We then extend the same for eight
CNN-based architectures to compare the differences in visualization and thus
interpretability. We introduced a novel technique Feature CAM, which falls in
the perturbation-activation combination, to create fine-grained, class-
discriminative visualizations. The resulting saliency maps from our
experiments proved to be 3-4 times better human interpretable than the state-
of-the-art in ABM. At the same time it preserves machine interpretability,
which is the average confidence scores in classification.

###### Keywords:

Interpretable AI Deep Learning Visual Attention Image Classification Black Box
Convolutional Neural Network.

##  1 Introduction

Applications that make use of Artificial Intelligence (AI) have seen an upward
trend in the past few years. Due to the advances in this research, various
neural network models and architectures were introduced and a number of them
have demonstrated accurate and robust performances. While it is a fact that
the models can perform complex tasks quite easily, humans are unable to
explain how it arrives at a decision. Hence, there is still hesitation to use
AI in high-precision applications such as security, health, and manufacturing
industries. Because of this, Deep Neural Network (DNN) is often referred to as
“the black box”.

Figure 1: Top 5 Visualizations using Feature CAM in comparison to that of
Grad-CAM++ on NasNet Mobile (Top) and InceptionV3 (Bottom). Our methods show
visual features of the input image better than Grad-CAM on both classifiers.

Interpretable models aim to resolve this issue by providing users with an
intuitive method to understand the reason behind the model’s prediction. When
it comes to the application of image classification, interpretable models
create a visualization of features or a map of object localization. It helps
to understand where the network is looking in the image, to arrive at that
classification. Various approaches have been introduced to tackle this issue
which try to interpret the network via image-like representation. But these
visualizations are very generic and differ only in localization and does not
provide any class-representative features, when visualized for the same
classifier.

Our research, which is based on the Activation-Based Methods (ABM) introduces
fine-grained details, which are representative of the class the object belongs
to, as seen on “Fig. 1”. This makes the visualizations more class-
discriminative, which means the resulting visualizations using our approach
can better interpret the classification results because of the additional
class-representative visual features that were introduced.

We compared the saliency maps generated by the state-of-the-art techniques in
ABM such as Grad-CAM, Grad-CAM ++ and Smooth Grad-CAM ++ and our proposed
technique Feature CAM, on eight different CNN-based classifiers, in addition
to VGG16 on which all the previous work has been compared. A qualitative
analysis which measured the human interpretability as well as a quantitative
analysis which measured the machine interpretability were performed for the
evaluation. The results prove that our saliency maps were 3-4 times better
interpretable than the existing techniques, while preserving the machine
interpretability.

The remainder of this paper is organized as follows. Section 2 introduces the
related works for interpretable models done in the field of image
classification. Section 3 details the ABM techniques that were used as
baseline for our approach. Our Proposed method is explained in detail in
Section 4. Section 5 discusses evaluation methodology and results. Finally,
Section 6 gives the conclusion and future work.

##  2 Related Work

The research in the field of interpretability in image classification can be
broadly divided into three categories, based on the type of operation they
perform to make the model interpretable.

###  2.1 Backpropagation Based Methods (BBM)

This family of methods uses the backpropagation of error signals as a way of
measuring the importance of each pixel in an image. It creates an importance
map that is overlaid on the original image to understand where the network is
looking, as a way of interpreting their decision.

Simonyan et al. [2], proposed GRAD, first of its kind, which introduced two
techniques for visualization, based on the gradient of class scores with
respect to the input image. They generated a synthetic image which maximized
the class score and computed a class saliency map to represent pixel-wise
importance for each input.

Zeiler et al. [7] added a Deconvnet layer to each convolutional layer to
visualize the features, which were more discriminative and invariant as they
go to the higher layers, and called it the Deconvolution. Springenberg et al.
[6], introduced Guided Backpropagation, a variant of deconvolution approach
which helped to visualize features, by adding an additional guidance signal to
the backpropagation.

Excitation Backpropagation [14] introduced the concept of contrastive
attention to make the top-down attention maps more discriminative and to
identify the task-relevant neurons in the network. Other methods such as Layer
wise Relevance Propagation (LRP) [8], DeepLift [11] also employ a top-down
propagation for visual representation.

The approach by Sundararajan et al. [10] makes few calls to the gradient
operator, to attribute the predictions of a deep neural network to its input
features. Smoothgrad [9] introduced a new concept of adding noise - to remove
noise and to sharpen the gradient-based sensitivity maps by applying a
smoothing effect.

While BBMs are computationally fast, the importance maps produced by them are
of low quality and they are not able to provide clear object localization
compared to PBM and ABM discussed in the next two sections.

###  2.2 Perturbation Based Methods (PBM)

In this approach, some form of perturbation is performed on the input image to
monitor the change in the model’s prediction, in order to understand the
regions importance for each prediction.

Zeiler et al [7] devised a method, which occluded the input image with a gray
square, from left to right and top to bottom, while observing the changes in
class probabilities to understand the importance of a feature.

Following Zeiler’s work, there were a few approaches developed for
interpretation using PBM, such as discrepancy maps by Zhou et al. [15], LIME
by Ribeiro et al. [16], Regional multi-scale approach by Seo et al. [17] and
RISE by Petsiuk et al. [18] However, a major drawback of PBM is that they are
often time-consuming and fail to generate fine-grained explanations.

###  2.3 Activation Based Methods (ABM)

Activation-based methods create visualization called saliency maps, by using a
linear combination of activations from convolutional layers of an image
classifier. This approach is comparatively newer and is the focus of our
research. The latest ABM are capable of producing saliency maps with
exceptional localization. When combined with other methods like Guided-
Backpropagation which is a BBM, the ABM provides fine-grained class-
discriminative visual explanations on top of localization, an example for this
is Guided Grad-CAM. In our research, we have taken a similar approach of
combining ABM with perturbed version of original input to create fine-grained
saliency maps with good localization capability.

##  3 ABM as Baseline For Our Approach

First of its kind, Class Activation Mapping (CAM) technique was introduced in
2016 by Zhou et al.[1] Class activation maps indicate the class discriminative
image regions used by CNN models to identify a class of the object. Saliency
maps are generated by performing global average pooling on the penultimate
layer feature maps (convolutional layer just before final fully-connected
layer) and by calculating a weighted sum of these values. This sum is used as
an input to the final dense layer, which outputs a visual heatmap localizing
the object.

For a given image, fk(x,y) represents activation of unit k in the last
convolutional layer at spatial location (x,y), Mc(x,y) is the class activation
map for each spatial element and wck is the weight corresponding to class c
for unit k. The class activation map Mc is given by:

$M_{c}(x,y)=\sum_{k}w^{c}_{k}f_{k}(x,y)$ (1)

Thus the given equation depicts the importance of activation at spatial
location (x,y), which finally leads to the object being classified as class c.
However, the major drawback of the method is that they were not generalized to
be applied to all CNN-based classifiers.

Selvaraju et al. [3], introduced Grad-CAM which added GAP (Global Average
Pooling) on the gradient of class scores w.r.t feature maps as weights in the
linear combination of activations, rather than weights in the network, which
resulted in a better discriminative object localization and visualization. As
per Eqn.(2) this new weight  ck is obtained by applying GAP on gradient of
class scores Yc w.r.t feature maps Ak of convolutional layer.

$\alpha^{c}_{k}=\frac{1}{Z}\operatorname*{\sum_{\textit{i}}\sum_{\textit{j}}}^{GAP}\frac{\partial
Y^{c}}{\partial A^{k}_{ij}}$ (2)

The class discriminative localization heatmap Lc for Grad-CAM is generated by
applying ReLU on the weighted combination activation units Ak.

$L^{c}_{Grad-CAM}=ReLU\left(\sum_{k}\alpha^{c}_{k}A^{k}\right)$ (3)

Even though, Grad-CAM produced better results and good accuracy rate compared
to CAM, it failed to classify images with multiple occurrences of the same
class.

To tackle the above limitation, Grad-CAM++ [4] introduced by Chattopadhay et
al., used a pixel-wise weighted combination of positive partial derivatives of
the feature maps from the penultimate layer. This technique required only
single backward pass for computing the linear combination of activations and
hence, was computationally efficient compared to the prior methods. wck is the
new weight used in Grad-CAM ++ which is obtained by taking weighted average of
pixel-wise gradients.

$w^{c}_{k}=\operatorname*{\sum_{\textit{i}}\sum_{\textit{j}}}\alpha^{kc}_{ij}.relu\left(\frac{\partial
Y^{c}}{\partial A^{k}_{ij}}\right)$ (4)

kcij is weighted coefficients of gradients at pixel (i,j) for class c at unit
k.

$\alpha^{kc}_{i,j}=\frac{\frac{\partial^{2}Y^{c}}{(\partial
A^{k}_{i,j})^{2}}}{2\frac{\partial^{2}Y^{c}}{(\partial
A^{k}_{i,j})^{2}}+\sum_{a}\sum_{b}A^{k}_{a,b}\frac{\partial^{3}Y^{c}}{(\partial
A^{k}_{i,j})^{3}}}$ (5)

To improve the quality of visualization by Grad-CAM++, Smooth Grad-CAM ++ was
introduced by Omeiza et al. [5] It addressed the need for getting enhanced
visual sharpness in the activation maps. Smooth Grad-CAM++, combined Smooth
Grad (which is a BBM) with Grad-CAM++, to get a smoothening effect which
resulted in improved visualization and interpretability as per the author. In
this method heatmaps were sharpened by taking random instances adjacent to the
input x and taking the mean of all saliency maps where n is the number of
sample images, N(0, 2) represents Gaussian noise with standard deviation . Mc
is new saliency map after applying smoothening effect.

$M_{c}(x)=\frac{1}{n}\sum_{1}^{n}M_{c}(x+N(0,\sigma^{2}))$ (6)

The resulting image covers a larger area while preserving the localization.
But, our quantitative study on the saliency maps generated by the three
techniques on VGG16 indicates that the Grad-CAM++ gives the best confidence
scores and is considered to be the state-of-the-art in ABM. Our motivation was
to improve the interpretability obtained through the ABM by making it more
fine-grained and class-discriminative along with its localization
capabilities, thereby increasing confidence of the application users in
Artificial Intelligence. In our research, we investigated several approaches
by enhancing state-of-the-art ABM methods, leading to a better visualization
aspect of the interpretable models for image classification.

##  4 Proposed Method: Feature CAM

While the various methods provide good interpretability, there is usually a
trade-off between localized vs fine-grained explanation. Hence, we propose to
devise a method that combines both localization and fine-grained details. The
motivation for our research was to improve the human interpretability of the
saliency maps generated by the Grad-CAMs while preserving the machine
interpretability.

Human interpretability is measured by the improvement in human interpretation
provided by the saliency maps, evaluated through a qualitative study. Machine
interpretability is measured by comparing the average confidence scores of
classification provided by the original pixels represented by Feature CAM
saliency maps, in comparison to the original image as well as the saliency
maps generated by Grad-CAMs.

We started our research by extending the existing ABM techniques to be
generalized on multiple CNN-based architectures. These techniques provide
either fine-grained or otherwise localized interpretation, but not both. We
performed a series of different experiments to create novel technique which
can improve the interpretability of saliency maps generated by CNN-based
classifiers. Our proposed technique falls in the category of a combination of
perturbation and activation-based methods, as one of the input is a feature
descriptor, in other words, a perturbed version of the input image.

For our proposed technique Feature CAM, we investigated optimal feature
descriptors of the original input image, to be combined with the saliency maps
of Grad-CAMs. We selected three edge-based feature descriptors using
Holistically Nested Edge Detection (HED) [21]. It is a deep learning model
used for detecting edges of an image. The feature descriptor is then used to
create two feature enhanced input images at specific blending ratios 2:1 and
1:1, which become second and third input. The inputs used in all of the
experiments are as follows: (i) Feature only, (ii) Feature enhanced original
image version 1 and (iii) Feature enhanced original image version 2 as shown
below.

Figure 2: Different versions of perturbed inputs used for performing
experiments

We combine the perturbed versions of input image with the saliency maps by
Grad-CAMs using three experiments, such that it adds fine-grained details of
the class in the saliency maps without loosing the localization. An initial
evaluation was done by comparing the results using 8 sample inputs on nine
classifiers to qualitatively understand human interpretability. The same
experiments were performed on selected classifiers (two lighter models VGG16
and NasNetMobile, two heavier models InceptionV3 and ResNet) to quantitatively
understand machine interpretability. This resulted in selecting top 5 out of
the nine combinations (three inputs and three experiments) across all nine
classifiers.

Figure 3: Stages of creation of an explanation map. L to R: Original Image,
Saliency Map, Mask and Explanation Map across classifiers VGG16 and Xception
on Grad-CAM++

For performing this evaluation, we created explanation maps for the saliency
maps generated by Feature CAM and Grad-CAMs as seen on “Fig. 3”. First, binary
thresholding was applied on the saliency maps to extract its mask, where 80
percentile of intensities were passed to the foreground. The pixels of the
original image on the corresponding spatial location are then copied on the
mask to get the explanation map. These explanation maps by Feature CAM are
then passed to the classifiers and compared with the results given by passing
original image as well as explanation maps of Grad CAMs.

Figure 4: A processing pipeline explaining the intuition behind Feature CAM.
The top half gives an overview of the Grad-CAMs starting from extracting
penultimate activations before Global Average Pooling and performing linear
combinations to create saliency maps. The bottom half shows Feature CAM, its
experiments and the top 5 visualizations generated using a lighter classifier
(VGG16) and larger classifier (Inception V3)

We used three metrics for initial evaluation. They are: (i) Percentage of
number of right classifications given by Feature CAM compared to that of
original image; (ii) Percentage of number of times there is an increase in
confidence score by Feature CAM from that of the original image and (iii)
Percentage number of times there is an increase in confidence score by Feature
CAM, compared to the existing Grad-CAM techniques. Following the initial
evaluation, we validate the findings by evaluating them on a larger dataset,
for which the results are presented in the next section. A comprehensive
visualization of the implementation pipeline of Feature CAM can be seen in
Fig. 4.

Below, we present the experiments and the results of initial evaluation.

Experiment 1: Pixel-wise addition of perturbed images with Grad-CAMs : Pixel-
wise addition was performed between the perturbed images and the saliency maps
of three Grad-CAMs. The three input versions were weighted by a threshold k
between 0 and 0.5 such that the resulting visualizations still preserve the
heatmap but add recognizable features inside it. For our experiment, we have
kept the value of k constant at 0.25.

$k.F_{hed}(x_{i})+M_{c}(x_{i})$ (7)

Fhed denotes the three input versions using HED and Mc denotes the saliency
map produced by Grad-CAMs.

This experiment yielded the best results on all three metrics for input 2. It
was followed by input 3 and input 1, with varied performance on all metrics.
So, all of the 3 inputs combinations using experiment 1 was selected in the
Top 5 combinations.

Experiment 2: Pixel-wise multiplication of perturbed images with CAMs : In
this experiment, the perturbed images were multiplied with the saliency maps
of three Grad-CAMs.

$F_{hed}(x_{i})\circ M_{c}(x_{i})$ (8)

This experiment did not yield as expected in any of the three metrics. Hence
no result from this experiment was selected.

Experiment 3: Pixel-wise multiplication of complemented perturbed images with
CAMs : In this experiment, the complement of perturbed images were multiplied
with the saliency maps of three Grad-CAMs, which resulted in a completely
different set of results.

$F_{hed}^{\prime}(x_{i})\circ M_{c}(x_{i})$ (9)

The experiment yielded better results for input 3 on all the metrics and for
input 1 for two metrics. Hence, two of the input combinations were selected
from this experiment into the top 5 combinations.

Figure 5: Top five visualizations using Feature CAM on larger classifier -
Inception ResNetV2 (left) and lighter classifier - MobileNet(right) in
comparison to Existing Grad-CAM techniques

##  5 Empirical Evaluation of Results

The top 5 saliency maps selected from the initial analysis are then used for
performing qualitative analysis on a group of 25 individuals and quantitative
analysis is performed on a larger data set. We chose a subset of images from
30 ImageNet classes to be evaluated on two smaller classifiers (VGG16 and
NasNet Mobile) and a larger classifier (InceptionV3).

###  5.1 Qualitative Analysis

Since the quality of visualization is subjective, there is a need for defining
human interpretability in terms of some metrics which would help us compare
the results. We performed qualitative analysis for Feature-CAM by conducting a
survey in a controlled environment with 25 participants, who are aged between
25 and 30 years and are graduate students in Computing Science. The survey
aimed at studying two metrics: (i) Human Faith % and (ii) Interpretability
Index.

(i) Human Faith %: The problem that we were addressing was to improve the
human faith in AI by providing better human interpretable visualizations.
Hence, understanding the increase in human faith was imperative. To measure
human faith % we provided the users with one saliency map at a time and asked
them to evaluate if they can interpret the original image from a small pool of
images by just using the saliency map. Using this, we measure human faith as
the percentage of the number of users, who could interpret the class from a
saliency map, when a set of original images were given as the reference.

As given in Table 1, human faith % in Feature CAM is twice better than Grad-
CAMs for top 3 results, followed by the remaining two as well.

(ii) Interpretability Index: To compare the interpretability of Feature CAM
visualizations with existing Grad-CAMs, we used Interpretability Index. It is
measured by the average rating given by the users on a scale of 0-6 in terms
of how best they think the saliency map interprets the original image. As
given in the Table 1, interpretability index for Feature-CAM is 3-4 times
better than existing Grad-CAMs. This means that Feature-CAM visualizations are
3-4 times better human interpretable than existing Grad-CAMs.

Table 1: Results of Qualitative Analysis for Feature CAM | Grad-CAMs | Ours 1 | Ours 2 | Ours 3 | Ours 4 | Ours 5
---|---|---|---|---|---|---
Human Faith % | 48% | 96% | 92% | 92% | 80% | 76%
Interpretability Index | 1.02 | 4.05 | 3.63 | 3.47 | 3.24 | 3.30

###  5.2 Quantitative Analysis

To understand the machine interpretability, we need to quantitatively measure
the confidence scores of explanation maps generated by Feature CAMs on a
larger ImageNet dataset. The explanation maps by Feature CAM and existing
techniques were passed through the same classifier that generated them for
prediction and the confidence scores were calculated.

The input dataset we have used for this analysis is a subset of ImageNet with
30 classes. This analysis was performed for one stronger classifier
(InceptionV3) and two lighter classifiers (VGG16 and NasNet Mobile). We
measure the performance of explanation maps using two metrics: Right
Classification % and Confidence Scores

(i) Right Classification (%): This metric compares the number of times the
explanation maps by Feature CAM are rightly classified compared to the
original classification by Grad-CAMs. As per the results given on Table 2, for
InceptionV3, three versions of Feature CAM (1,2 & 5) performs similar to the
Grad-CAMs. For a smaller classifier like VGG16, the % of right classification
has improved really well compared to all the Grad-CAM techniques, especially
for Grad-CAM. Similarly, for NasNet Mobile, Feature CAM has improved the % of
right classification by a large margin, especially for Grad-CAM ++ and Smooth
Grad-CAM++.

Table 2: Comparison of % of Right Classification: Grad-CAMs Vs Feature CAM (Ours) on ImageNet 30 classes Classifier | Grad-CAM Versions | Grad-CAM | Ours 1 | Ours 2 | Ours 3 | Ours 4 | Ours 5
---|---|---|---|---|---|---|---
Inception | Grad-CAM | 93% | 90% | 92% | 92% | 85% | 93%
V3 | Grad-CAM ++ | 95% | 95% | 95% | 88% | 80% | 95%
| SMGrad-CAM ++ | 97% | 96% | 94% | 87% | 77% | 95%
VGG16 | Grad-CAM | 52% | 78% | 81% | 51% | 50% | 66%
| Grad-CAM ++ | 80% | 88% | 86% | 84% | 81% | 84%
| SMGrad-CAM ++ | 80% | 88% | 86% | 84% | 81% | 84%
NasNet | Grad-CAM | 85% | 92% | 95% | 88% | 87% | 90%
Mobile | Grad-CAM ++ | 75% | 88% | 93% | 79% | 79% | 83%
| SMGrad-CAM ++ | 73% | 90% | 93% | 79% | 79% | 89%

Inference: Feature CAM improves the percentage of right classification for
lighter/ smaller classifiers by a large margin compared to Grad-CAM
techniques. For larger/stronger classifiers, Feature CAM does not improve the
values compared to Grad-CAM techniques, but attempts to preserve it even after
adding the fine-grained details of the class.

(ii) Confidence Score (%): To understand the machine interpretability, the
average confidence scores of explanations by visualizations of Grad-CAMs and
novel Feature CAM were compared with that of the original image. As per the
results given in Table 3, for larger classifiers like InceptionV3 as well as
lighter classifiers like VGG16 and NasNet Mobile, the Feature CAM preserves
the confidence scores and hence the machine interpretability, when compared to
Grad-CAMs as well as the original image.

Table 3: Comparison of Avg. Confidence Scores: Original Vs Grad-CAMs(G-CAMs) Vs Feature CAM (Ours) on ImageNet 30 classes Classifier | G-CAM Versions | Original | G-CAMs | Ours 1 | Ours 2 | Ours 3 | Ours 4 | Ours 5
---|---|---|---|---|---|---|---|---
Inception | Grad-CAM | 97% | 97% | 98% | 98% | 97% | 97% | 97%
V3 | Grad-CAM ++ |  | 96% | 97% | 97% | 96% | 95% | 96%
| SMGrad-CAM ++ |  | 96% | 96% | 98% | 96% | 97% | 97%
VGG16 | Grad-CAM | 86% | 82% | 83% | 86% | 84% | 84% | 84%
| Grad-CAM ++ |  | 88% | 88% | 89% | 88% | 85% | 86%
NasNet | Grad-CAM | 87% | 78% | 85% | 84% | 78% | 80% | 82%
Mobile | Grad-CAM ++ |  | 73% | 84% | 85% | 78% | 77% | 83%
| SMGrad-CAM ++ |  | 75% | 83% | 84% | 77% | 77% | 81%

Inference: Feature CAM preserves the confidence scores and thus machine
interpretability for both lighter/smaller classifiers as well as for larger
classifiers.

In this section, we performed various analysis to compare the saliency maps by
Feature CAM with the latest Grad-CAM techniques[3, 4, 5]. We demonstrated that
our technique Feature CAM were able to produce fine-grained as well as class-
discriminative visualizations, which are 3-4 times better human interpretable
than the state-of-the-art techniques. Moreover, the quantitative analysis
proved that Feature CAM preserves and at times outperforms the machine
interpretability compared to Grad-CAMs and the original image.

##  6 Conclusion and Future Work

Our research was focused on increasing the human faith in AI by improving the
interpretability of Deep Neural Networks. Our research has the following
contribution:

We created a novel technique called Feature CAM which improved the localized
saliency maps generated by Grad-CAMs to make it fine-grained and much more
class-discriminative. The resulting top 5 versions of saliency maps generated
from our technique are 3-4 times better than the state-of-the-art methods
based on human interpretation. This was measured through a comprehensive
qualitative study with 25 participants. The original pixels represented by the
generated saliency maps also preserves the machine interpretability for
classification, in comparison to the original image and the saliency maps by
ABM techniques such as Grad-CAMs. Machine interpretability was evaluated by
comparing the average confidence scores of a subset of images from 30 ImageNet
classes for a smaller classifier (VGG16) and a larger classifier
(InceptionV3). Our techniques presented similar confidence scores and
occasionally outperformed the scores from original images as well.

A limitation of Feature-CAM is that it depends on Grad-CAMs for the
localization. In the future, we would like to improve our work by creating a
new baseline for localization, which performs well for larger as well as
smaller classifiers, so the localization of Feature CAM will not have to
depend upon Grad-CAMs.

Acknowledgments. The authors would like to thank Dr. Lihang Ying and the
Multimedia Research Centre (MRC), University of Alberta for their support and
motivation. We would also like to acknowledge some of initial contributions in
literature review given by Kirtan Shah and Dhara Pancholi.

## References

  * [1] Zhou, Bolei, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. “Learning Deep Features for Discriminative Localization.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016). doi:10.1109/cvpr.2016.319.
  * [2] Simonyan, Karen, Andrea Vedaldi, and Andrew Zisserman. ”Deep inside convolutional networks: Visualising image classification models and saliency maps.” arXiv preprint arXiv:1312.6034 (2013).
  * [3] Selvaraju, Ramprasaath R., Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization.” 2017 IEEE International Conference on Computer Vision (ICCV) (October 2017). doi:10.1109/iccv.2017.74.
  * [4] Chattopadhay, Aditya, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. “Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks.” 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (March 2018). doi:10.1109/wacv.2018.00097.
  * [5] Omeiza, Daniel, Skyler Speakman, Celia Cintas, and Komminist Weldermariam. ”Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models.” arXiv preprint arXiv:1908.01224 (2019).
  * [6] Springenberg, Jost Tobias, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. ”Striving for simplicity: The all convolutional net.” arXiv preprint arXiv:1412.6806 (2014).
  * [7] Zeiler, Matthew D., and Rob Fergus. “Visualizing and Understanding Convolutional Networks.” Lecture Notes in Computer Science (2014): 818–833. doi:10.1007/978-3-319-10590-1_53.
  * [8] Bach, Sebastian, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. “On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation.” Edited by Oscar Deniz Suarez. PLOS ONE 10, no. 7 (July 10, 2015): e0130140. doi:10.1371/journal.pone.0130140.
  * [9] Smilkov, Daniel, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. ”Smoothgrad: removing noise by adding noise.” arXiv preprint arXiv:1706.03825 (2017).
  * [10] Sundararajan, Mukund, Ankur Taly, and Qiqi Yan. ”Axiomatic attribution for deep networks.” Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
  * [11] Shrikumar, Avanti, Peyton Greenside, and Anshul Kundaje. ”Learning important features through propagating activation differences.” Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
  * [12] Du, Mengnan, Ninghao Liu, Qingquan Song, and Xia Hu. “Towards Explanation of DNN-Based Prediction with Guided Feature Inversion.” Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (July 19, 2018). doi:10.1145/3219819.3220099.
  * [13] Wagner, Jorg, Jan Mathias Kohler, Tobias Gindele, Leon Hetzel, Jakob Thaddaus Wiedemer, and Sven Behnke. “Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks.” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2019). doi:10.1109/cvpr.2019.00931.
  * [14] Zhang, Jianming, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. “Top-Down Neural Attention by Excitation Backprop.” Lecture Notes in Computer Science (2016): 543–559. doi:10.1007/978-3-319-46493-0_33.
  * [15] Zhou, Bolei, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. ”Object detectors emerge in deep scene cnns.” arXiv preprint arXiv:1412.6856 (2014).
  * [16] Ribeiro, Marco, Sameer Singh, and Carlos Guestrin. “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier.” Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations (2016). doi:10.18653/v1/n16-3020.
  * [17] Seo, Dasom, Kanghan Oh, and Il-Seok Oh. “Regional Multi-Scale Approach for Visually Pleasing Explanations of Deep Neural Networks.” IEEE Access 8 (2020): 8572–8582. doi:10.1109/access.2019.2963055.
  * [18] Petsiuk, Vitali, Abir Das, and Kate Saenko. ”Rise: Randomized input sampling for explanation of black-box models.” arXiv preprint arXiv:1806.07421 (2018).
  * [19] Chollet, Francois. “Xception: Deep Learning with Depthwise Separable Convolutions.” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (July 2017). doi:10.1109/cvpr.2017.195.
  * [20] Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. “Rethinking the Inception Architecture for Computer Vision.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016). doi:10.1109/cvpr.2016.308.
  * [21] Xie, Saining, and Zhuowen Tu. “Holistically-Nested Edge Detection.” 2015 IEEE International Conference on Computer Vision (ICCV) (December 2015). doi:10.1109/iccv.2015.164.