# Axiomatization of Gradient Smoothing in Neural Networks

Linjiang Zhou
School of Cyber Science and Engineering
Wuhan University
Hubei, China
<EMAIL_ADDRESS>
& Xiaochuan Shi
School of Cyber Science and Engineering
Wuhan University
Hubei, China
<EMAIL_ADDRESS>
& Chao Ma
School of Cyber Science and Engineering
Wuhan University
Hubei, China
<EMAIL_ADDRESS>
&Zepeng Wang
School of Cyber Science and Engineering
Wuhan University
Hubei, China
<EMAIL_ADDRESS>

###### Abstract

Gradients play a pivotal role in neural networks explanation. The inherent
high dimensionality and structural complexity of neural networks result in the
original gradients containing a significant amount of noise. While several
approaches were proposed to reduce noise with smoothing, there is little
discussion of the rationale behind smoothing gradients in neural networks. In
this work, we proposed a gradient smooth theoretical framework for neural
networks based on the function mollification and Monte Carlo integration. The
framework intrinsically axiomatized gradient smoothing and reveals the
rationale of existing methods. Furthermore, we provided an approach to design
new smooth methods derived from the framework. By experimental measurement of
several newly designed smooth methods, we demonstrated the research potential
of our framework.

_Keywords_ Explainable Artificial Intelligence $\cdot$ Neural Networks $\cdot$
Gradient Smoothing

##  1 Introduction

Explanation for Artificial Intelligence (AI) is an inevitable part of AI
applications with human interaction. For instance, explanation techniques are
crucial in fields like medical image analysis Alvarez Melis and Jaakkola
(2018), financial data analysis Zhou et al. (2023), and autonomous driving
Abid et al. (2022), where AI is applied to these data-sensitive and decision-
sensitive fields. Additionally, given the prevalence of personal data
protection laws in most countries and regions of the world, fully black-box AI
models often face intense legal scrutiny Doshi-Velez and Kim (2017).

After the development in recent years, some explanation methods try to explain
the decisions of neural networks by the visualization of the decision basis
and depiction of feature importance Nauta et al. (2023). These local
explanation methods aim to provide an explanation of neural networks on an
individual sample. Moreover, the interpretation process or algorithm of these
methods often utilizes the gradient of the neural network. For instance, Grad-
CAM Selvaraju et al. (2017), Grad-CAM++ Chattopadhay et al. (2018) and Score-
CAM Wang et al. (2020a) produces class activation map using gradients, and
Integrate-Gradient Sundararajan et al. (2017) calculates integral gradients
via sampling from gradients.

(a) Input image

(b) The noisy original gradient

Figure 1: An example of noisy original gradient in image classification model
VGG16.

However, as noted in Smilkov et al. (2017) and Bykov et al. (2022), neural
networks often have a significant amount of noise in their raw gradients due
to their complex structure and high input dimensions. This noise might
seriously impair interpretation. fig. 1 shows an example of an original but
noisy gradient. Therefore, Smilkov et al. (2017) proposed a simple and
effective method for smoothing the gradient called SmoothGrad. This method
reduces noise by adding Gaussian noise to the samples multiple times and
calculating the average of the gradients of these noise-added samples. Bykov
et al. (2022) further expected to eliminate more noise by adding perturbations
to the model parameters, called NoiseGrad.

As mentioned in Ancona et al. (2017); Nie et al. (2018); Bykov et al. (2022);
Nauta et al. (2023), the rationale by which the SmoothGrad and NoiseGrad
methods reduce noise is currently unknown. Specifically, it is unclear how
SmoothGrad effectively eliminates the majority of noise in the neural network
gradient in a simple manner. That is, these explanation methods are inherently
lacking in explainability.

So the relationship between the smoothed gradient and the raw gradient needs
to be further studied. It is important to note that the principle behind the
smoothed noise is unknown, therefore it cannot be guaranteed that the smoothed
gradient is a fidelity representation of the original gradient. For instance,
The research of Adebayo et al. (2018); Kindermans et al. (2019); Yeh et al.
(2019) has shown that many explanation methods do not accurately describe the
real decisions of the model, but adopt some (in)fidelity approaches to improve
the visualization performance.

Also, the rationale behind the phenomenon of gradient smoothing experiments
also needs to be discussed. After adding Gaussian noise a sufficient number of
times, it is unclear whether the smoothed gradient will converge in theory.
The hyperparameter selection in gradient smoothing is based on empirical
heuristics and lacks theoretical guidance.

Therefore, we aim to address these concerns by developing a comprehensive
axiomatization framework of gradient smoothing, thus establishing a complete
attribution chain of neural networks \- interpretable methods \- mathematical
theorems. By utilizing the function mollification theorem and Monte Carlo
integration, we propose the Monte Carlo Gradient Mollification Formulation to
present a theoretical explanation for gradient smoothing. The formulation
uncovers the mathematical principles of gradient smoothing, which allows us to
design a large variety of new gradient smoothing methods. Additionally, we
present a straightforward and effective approach with mathematical principles
and practical computational limitations. We utilized this approach to design
five gradient smoothing kernel functions and examined the performance of
smooth methods produced by these kernel functions on several metrics.

Our main contributions include:

  * •

We presented a comprehensive axiomatization framework of gradient smoothing,
and revealed the basic mechanisms of the gradient smoothing methods.

  * •

We proposed an approach for designing new gradient smoothing methods, and
designed four new kernel functions for producing new gradient smoothing
methods.

  * •

The performance of these new methods has been experimentally explored using a
range of evaluation metrics, demonstrating the research potential of this
theoretical framework.

##  2 Preliminary

In general, consider a neural network as a function
$f(x;\theta):\mathbb{R}^{D}\to\mathbb{R}^{C}$, with the trainable parameters
$\theta$. In the example of a classification function, the neural network will
output a score for each class $c$, where $c\in\\{0,1,\cdots,C\\}$. When
further considering only the output of the neural network in a single class
$c$, the neural network can be simplified to a function
$f^{c}(x;\theta):\mathbb{R}^{D}\to\mathbb{R}$, which maps the input
$\mathbb{R}^{D}$ to the 1-dimension $\mathbb{R}$ space. For simplicity, we
will use $f(x)$ to represent the neural network and only consider the output
of the neural network on a single class.

The gradient of $f(x)$ could be presented as,

$g(x)=\frac{\partial f(x)}{\partial x}$ (1)

The rationale for interpreting neural networks using original gradients, or
using gradients multiplied by input values, is derived from Taylor’s theorem.
The first-order Taylor expansion on $f(x)$ is given by,

$f(x)=f(x_{0})+[\,g(x_{0})]\,^{\top}(x-x_{0})+R(x;x_{0},\theta)$ (2)

If the residual term $R(x;x_{0},\theta)$ is assumed to be a locally stable
constant value, the gradient $g(x)$ could denote the contribution of each
feature in $x$ to final outputs. Considering the local surroundings of a
sample, neural networks tend not to present an ideal linearity but rather have
a very rough and nonlinear decision boundary. [Need a Figure to explain] This
complexity of neural networks and the input features usually leads to the
assumption not being valid. Thus, this is a possible explanation for the large
amount of noise in the original gradient.

Recently, methods for reducing gradient noise can be roughly categorized into
the following two groups:

Adding noise to reduce noise. SmoothGrad proposed by Smilkov et al. (2017)
introduces stochasticity to smooth the noisy gradients. SmoothGrad averages
the gradients of random samples in the neighborhood of the input $x_{0}$. This
could be described in,

$sg(x)=\frac{1}{N}\sum_{i}^{N}g(x+\varepsilon)$ (3)

where the sample times is $N$, and $\varepsilon$ is distributed as a Gaussian
$\mathcal{N}(0,\sigma^{2})$ with standard deviation $\sigma$. Similarly to
SmoothGrad, NoiseGrad and FusionGrad presented in Bykov et al. (2022)
additionally add perturbations to model parameters. The NoiseGrad could be
defined as follows,

$ng(x)=\frac{1}{M}\sum_{i}^{M}g(x;\theta\cdot\eta)$ (4)

where the sample times is $M$, and $\eta$ follows distribution
$\mathcal{N}(1,\sigma^{2})$. And the FusionGrad is a mixup of NoiseGrad and
SmoothGrad, which could be described in the same way,

$fg(x)=\frac{1}{M}\frac{1}{N}\sum_{j}^{M}\sum_{i}^{N}g(x+\varepsilon;\theta\cdot\eta)$
(5)

These simple methods are experimentally verified to be efficient and robust
Dombrowski et al. (2019).

Improving backpropagation to reduce noise. Deconvolution Zeiler and Fergus
(2014) and Guided Backpropagation Springenberg et al. (2015) directly modifies
the gradient computation algorithm of the ReLU function. Some other methods
such as Feature Inversion Du et al. (2018), Layerwise Relevance Propagation
Bach et al. (2015), DeepLift Shrikumar et al. (2017), Occlusion Ancona et al.
(2017), Deep Taylor Montavon et al. (2017), etc. employ some additional
features to approximate or improve the gradient for precise visualization.

In the rest of this paper, we will focus on constructing a theoretical
framework to explain the rationale for adding noise to reduce noise.

##  3 Methodology

The Monte Carlo gradient smoothing formulation will be introduced in this
section. In order to facilitate the understanding of the derivation of the
formula, we present it in a bottom-up manner.

###  3.1 Convolution and Dirac Function

Convolution is an important functional operation. Referring to Bak et al.
(2010), the convolution operation of function $f$ and $g$ could be defined as
follows:

######  Definition 3.1.

Let $f$ and $g$ be functions (generalized function). Symbol $*$ is defined as
the convolution operator. The $f*g$ is defined by

$(f*g)(x)=\int_{-\infty}^{\infty}f(t)g(x-t)\,dt$ (6)

Obviously, we can derive some useful lemmas:

######  Lemma 3.2.

If $f*g$ exists,

$f*g=g*f$ (7)

######  Lemma 3.3.

If $f*g$ exists and $f^{\prime}$ and $g^{\prime}$ exist,

$(f*g)^{\prime}=f^{\prime}*g=f*g^{\prime}$ (8)

######  Lemma 3.4.

If $g$ is continuous, then $f*g$ is continuous.

The Dirac function is a generalized function and describes the limit of a set
called the Dirac function sequence. In general, the 1-dimensional dirac
function in $\mathbb{R}$ is defined as follows:

######  Definition 3.5.

The general definition of the Dirac function is,

$\delta(x)=\begin{cases}+\infty&\text{if }x=0\\\ 0&\text{if }x\neq
0\end{cases}$ (9)

and it is constrained by,

$\int_{-\infty}^{\infty}\delta(x)=1$ (10)

The Dirac function has some fundamental and important properties.

######  Lemma 3.6.

If $f*\delta$ exists,

$f*\delta=f$ (11)

Using lemma 3.6 and lemma 3.3, we could easily get,

######  Lemma 3.7.

If $f*\delta$ exists and $f^{\prime}$ exist,

$(f*\delta)^{\prime}=f^{\prime}*\delta=\delta*f^{\prime}=f^{\prime}$ (12)

lemma 3.6 and lemma 3.7 also apply to function $f$ and Dirac functions
$\delta$ in n-dimensions. All the proofs are in appendix A.

###  3.2 Function Mollification

Function mollification refers to the use of a mollifier to mollify the target
function so that the mollified function becomes continuous or smooth.

######  Definition 3.8.

A function $\varphi(x):\mathbb{R}^{n}\to\mathbb{R}$ is a mollifier if it
satisfies the following properties.

  1. 1.

$\varphi(x)$ is of compact support,

  2. 2.

$\int_{\mathbb{R}^{n}}\varphi(x)\,dx=1$,

  3. 3.

$\lim_{\epsilon\to 0}\varphi_{\epsilon}(x):=\lim_{\epsilon\to
0}\epsilon^{-n}\varphi(\frac{x}{\epsilon})=\delta(x)$, where $\delta(x)$ is
the Dirac function.

The 3rd property in definition 3.8 holds for almost all general functions
consisting of primitive functions that satisfy the first two properties111The
details could be found in Stein and Weiss (1971) with Theorem 1.18..

######  Definition 3.9.

The convolution with a mollifier is called mollification,

$f_{\epsilon}=f*\varphi_{\epsilon}$ (13)

By lemma 3.4, if the mollifier $\varphi_{\epsilon}$ is continuous,
$f_{\epsilon}$ is continuous. From lemma 3.4, if $\varphi_{\epsilon}$ is
continuous or locally continuous, $f_{\varepsilon}$ is also continuous or
locally continuous. In other words, a mollifier $\varphi_{\epsilon}$ is a tool
that smooths out the rough parts of a function $f$, similar to sandpaper with
a granularity size of $1/\epsilon$.

Noted that, in definition 3.8, $\varphi_{\epsilon}(x)$ defined a special
function sequence, and the limit of the sequence is the Dirac function.
Similarly, the definition of the Dirac sequence is presented as,

######  Definition 3.10.

if $\psi$ is absolute integrable function defined in $\mathbb{R}^{n}$ with
$\int_{\mathbb{R}^{n}}\psi(x)\,dx=1$, the Dirac sequence is defined as,

$\psi_{\epsilon}(x)=\epsilon^{-n}\psi(\frac{x}{\epsilon})$ (14)

In fact, mollification is a special case of convolution, and mollifier is a
special case of Dirac sequences. By definition 3.10, $\varphi(x)$ defined a
mollifier sequence $\\{\varphi_{\epsilon}(x)\\}$. So, $\varphi(x)$ is called
kernel function because it determines the characteristic of the sequence
$\\{\varphi_{\epsilon}(x)\\}$.

###  3.3 Monte Carlo Gradient Mollification Formulation

Given the noisy gradients and decision boundaries in most deep neural
networks, we aim to smooth the neural networks with mollification. As
introduced in section 2, We simplify the neural networks into a function
$f(x)$. With a proper mollifier $\varphi_{\epsilon}(x)$ given, we could use
this mollifier to smooth the neural networks with mollification. Thus, by
definition 3.9, we present the mollification of neural networks as,

$\displaystyle f_{\epsilon}(x)$ $\displaystyle=(f*\varphi_{\epsilon})(x)$ (15)
$\displaystyle=\int_{\mathbb{R}^{n}}f(x-t)\varphi_{\epsilon}(t)\,dt$
$\displaystyle=\int_{\mathbb{R}^{n}}f(t)\varphi_{\epsilon}(x-t)\,dt$

Accordingly, by lemma 3.3, we could get the expression for the gradient of the
smoothed neural network with mollification as,

$g_{\epsilon}(x)=(g*\varphi_{\epsilon})(x)=\int_{\mathbb{R}^{n}}g(x-t)\varphi_{\epsilon}(t)\,dt$
(16)

Considering the computation complexity of the gradient, $f(t)$ and $g(t)$ are
not easy to compute. So we chose to express $g_{\epsilon}(x)$ in the form of
eq. 16 instead of $\int_{\mathbb{R}^{n}}g(t)\varphi_{\epsilon}(x-t)\,dt$. And
importantly, it is almost impossible to solve the analytic expression for
$f_{\epsilon}$ in eq. 15 and $g_{\epsilon}(x)$ in eq. 16. Therefore,
$g_{\epsilon}(x)$ could only be solved numerically.

Note that $g_{\epsilon}(x)$ is essentially an integral and $x$ is a very high
dimensional vector like $224\times 224\times 3$ in some image classification
tasks. Therefore, Monte Carlo integration is almost the only efficient method
for solving this problem. Thus, we give the Monte Carlo Gradient Mollification
Formulation as follows,

######  Theorem 3.11.

Given an instance $x_{0}$ and a $n$-dimension random variable $T$ obeying a
distribution $P$ with probability density function $p(t)$, randomly sample $T$
for $N$ times to obtain a series of samples $t_{i}$. the Monte Carlo
approximation of $g_{\epsilon}(x_{0})$ is presented as,

$mg_{\epsilon}(x_{0})=\frac{1}{N}\sum_{i=1}^{N}\frac{g(x_{0}-t_{i})\varphi_{\epsilon}(t_{i})}{p(t_{i})}$
(17)

In theorem 3.11, $mg_{\epsilon}(x_{0})$ is actually a statistic of the random
variable $T$. We prove in the appendix B that this statistic is unbiased and
consistent, which means that as
$\lim_{N\to\infty}mg_{\epsilon}(x_{0})=g_{\epsilon}(x_{0})$.

###  3.4 Axiomatization of Gradient Smoothing

The existing gradient smoothing methods, SmooothGrad, NoiseGrad, and
FusionGrad, are introduced in section 2. Next, We aim to generalize these
methods to theorem 3.11. That is, These methods are a special case of theorem
3.11, and they could all be derived from theorem 3.11.

Derivation of SmoothGrad. In theorem 3.11, if $\varphi$ is the Probability
Density Function (PDF) of a normal distribution, i.e. a Gaussian function,
$\varphi$ satisfies all conditions in definition 3.8. So the Gaussian function
is a reasonable mollifier. Next, set $\varphi_{\epsilon}=p$, that is, the PDF
of $P$ is also $\varphi_{\epsilon}$. So SmoothGrad could be described as,

######  Remark 3.12.

Given:

$\displaystyle\varphi_{\epsilon}(t)=\frac{1}{\sqrt{(2\pi)^{n}}\epsilon^{n}}\exp{-\frac{xx^{\top}}{2\epsilon^{2n}}}$
(18) $\displaystyle p=\varphi_{\epsilon}$

In theorem 3.11, $T\sim p(t)$, i.e. $T\sim\mathcal{N}(0,\epsilon^{2})$,

$mg_{\epsilon}(x_{0})=\frac{1}{N}\sum_{i=1}^{N}g(x_{0}-t_{i}).$ (19)

Thus, $mg_{\epsilon}(x_{0})=sg(x_{0})$ in eq. 3. SmoothGrad $sg$ is a special
case of $mg_{\epsilon}$ with condition eq. 18.

Derivation of NoiseGrad. With the same given condition of SmoothGrad, consider
the parameter $\theta$ in neural networks as a smoothed target.

######  Remark 3.13.

Given:

$\displaystyle\varphi_{\epsilon}(t)=\frac{1}{\sqrt{(2\pi)^{n}}\epsilon^{n}\theta}\exp{-\frac{(x-\theta)(x-\theta)^{\top}}{2\epsilon^{2n}}}$
(20) $\displaystyle p=\varphi_{\epsilon}$

There we choose $T=\theta-\theta X,X\sim\mathcal{N}(0,\epsilon^{2})$, because
this makes $\theta-T=\theta X$ corresponding to $\theta\cdot\eta$ in eq. 4.
So, in theorem 3.11, $T\sim p(t)$, i.e.
$T\sim\mathcal{N}(\theta,\theta^{2}\epsilon^{2})$,

$mg_{\epsilon}(x_{0})=\frac{1}{N}\sum_{i=1}^{N}g(x_{0};\theta+t_{i}).$ (21)

Thus, $mg_{\epsilon}(x_{0})=ng(x_{0})$ in eq. 3. NoiseGrad $ng$ is a special
case of $mg_{\epsilon}$ with condition eq. 20.

Derivation of FusionGrad. FusionGrad is a combination of SmoothGrad and
NoiseGrad. It smooths the gradients with both input $x_{0}$ and parameters
$\theta$. So it is easy to derive FusionGrad from theorem 3.11.

######  Remark 3.14.

Given:

$\displaystyle\varphi_{\epsilon}^{(x)}(t)=\frac{1}{\sqrt{(2\pi)^{n}}\epsilon^{n}\theta}\exp{-\frac{(x-\theta)(x-\theta)^{\top}}{2\epsilon^{2n}}}$
(22) $\displaystyle p^{(x)}=\varphi_{\epsilon}^{(x)}$
$\displaystyle\varphi_{\epsilon}^{(\theta)}(t)=\frac{1}{\sqrt{(2\pi)^{n}}\epsilon^{n}\theta}\exp{-\frac{(x-\theta)(x-\theta)^{\top}}{2\epsilon^{2n}}}$
$\displaystyle p^{(\theta)}=\varphi_{\epsilon}^{(\theta)}$

There mark $\varphi_{\epsilon}^{(x)}$ and $\varphi_{\epsilon}^{(\theta)}$ to
distinguish the mollifiers for smoothing input $x_{0}$ and $\theta$. As
described in remark 3.12 and remark 3.13, we could get, in theorem 3.11,
$T^{(x)}\sim p^{(x)}(t^{(x)})$ and $T^{(\theta)}\sim
p^{(\theta)}(t^{(\theta)})$,

$mg_{\epsilon}(x_{0})=\frac{1}{N^{(\theta)}}\frac{1}{N^{(x)}}\sum_{i^{(\theta)}=1}^{N^{(\theta)}}\sum_{i^{(x)}=1}^{N^{(x)}}g(x_{0}+t^{(x)}_{i^{(x)}};\theta+t^{(\theta)}_{i^{(\theta)}}).$
(23)

Thus, $mg_{\epsilon}(x_{0})=fg(x_{0})$ in eq. 5. FusionGrad $fg$ is a special
case of $mg_{\epsilon}$ with condition eq. 22.

##  4 Implications

In this section, we will further discuss the implication of theorem 3.11. And
since our study has uncovered a potentially infinite number of possible
gradient smoothing methods, we expect to provide some useful guidelines for
discovering potentially efficient gradient smoothing methods.

###  4.1 Explanation for Gradient Smoothing

The theorem 3.11 reveals that the rational of adding noise to reduce noise is
a Monte Carlo approximation of mollification for neural networks.

1\. As mentioned in lemma 3.4, using continuous or locally continuous mollifer
could make the smoothed function become continuous or locally continuous as
well.

As shown in fig. 2, the mollification could smooth out the rugged shape of the
function, and the degree of smoothing is determined by $\epsilon$. The larger
the value of $\epsilon$, the smoother the mollified function will be, and the
smaller the value of $\epsilon$, the more the mollified function will retain
its original undulations. As $\epsilon$ tends to 0, lemma 3.7 shows the
smoothed function will be the same as the original function. Thus, it answered
the RQ1 and so explained why gradient smoothing methods work.

Figure 2: An example of mollification. A simple piecewise function
$f(x):\mathbb{R}\to\mathbb{R}$ has a noncontinuous and noisy bound. And it was
mollified by Gaussian kernel with different $\epsilon$. The details of the
example can be found in appendix C

.

2\. The smoothed gradient converges as the number of samples increases, the
limit of convergence is determined by the mollifier, and the speed of
convergence is determined by the sampling distribution.

In theorem 3.11, $mg_{\epsilon}$ will eventually converge to $g_{\epsilon}$ as
the number of samples increases. The limit of convergence is only related to
mollifier $\varphi_{\epsilon}(x)$. That is, after sampling enough times, the
explanation performance of the smoothed gradient is determined by
$\varphi_{\epsilon}(x)$. Thus, it answered the RQ2 and means that we could get
different smoothing methods by designing different $\varphi_{\epsilon}(x)$.
And there are almost infinitely $\varphi_{\epsilon}(x)$ applicable in
definition 3.8. Therefore, theorem 3.11 reveals the existence of more possible
research potential for gradient smoothing.

3\. The difference between the three methods, SmoothGrad, NoiseGrad, and
FusionGrad, is that the input dimensions for smoothing are different, but they
all employ a Gaussian kernel as the mollification kernel function.

In detail, SmoothGrad smooths function $f(x;\theta)$ with
$f(x;\theta)*\varphi_{\epsilon}$, NoiseGrad smooths function $f(\theta;x)$
with $f(\theta;x)*\varphi_{\epsilon}$, and FusionGrad smooths function
$f(x,\theta)$. Moreover, FusionGrad smooths it in two times, that is
$f(x,\theta)*\varphi_{\epsilon}^{(x)}*\varphi_{\epsilon}^{(\theta)}$. For
simplicity, we will use SG, NG, and FG to denote these three smooth modes
respectively.

###  4.2 Approximation to Dirac Function

A large number of kernel functions satisfy definition 3.8, but not all of them
are capable of efficiently smoothing neural networks. Therefore, we explore
some mathematical and computational constraints and provide a simple approach
to construct a suitable kernel function. We apply this approach to provide
four kernel functions in addition to the Gaussian kernel function.

Based on some simple mathematical intuition Febrianto et al. (2021); Rohner
(2019), the kernel function generally selected needs to satisfy the following
conditions,

$\left\\{\begin{array}[]{ll}\varphi(x)>0&,\text{for }\forall
x\in\mathbb{R},\\\ \varphi(x)=\varphi(-x)&,\text{for }\forall
x\in\mathbb{R}.\end{array}\right.$ (24)

To address the above limitations, we design a simple but effective method to
quickly obtain the appropriate kernel function. Note that the indefinite
integral of the Dirac function $\int_{-\infty}^{x}\delta(t)\,dt$ is a unit
step function. That is, we can utilize an approximation of the Unit Step
Function (USF) and then obtain an approximation of the Dirac function by
solving for its derivatives. In the following, we construct five kernel
functions including the Gaussian kernel using several common approximations of
USF.

Gaussian kernel222Normal distribution could perform fast sampling with other
methods. So there we do not present the inverse function $\Phi^{-1}(x)$.:

$\begin{array}[]{ll}&\Phi(x)=\frac{\text{Erf}(x/\sqrt{2})+1}{2}\\\
&\varphi(x)=\frac{1}{\sqrt{2\pi}}e^{-\frac{x^{2}}{2}}\end{array}$ (25)

Poisson kernel333This function is called Poisson kernel because it is one case
of the Poisson formula for the solution of the Dirichlet problem in the upper
half-plane:

$\begin{array}[]{lll}&\Phi(x)=\frac{\arctan\left(x\right)}{\pi}+\frac{1}{2}\\\
&\Phi^{-1}(x)=\tan{(\frac{\pi}{2}(2x-1))},0<x<1\\\
&\varphi(x)=\frac{1}{\pi\left(x^{2}+1\right)}\end{array}$ (26)

Hyperbolic kernel:

$\begin{array}[]{lll}&\Phi(x)=\frac{\tanh(x)}{2}+\frac{1}{2}\\\
&\Phi^{-1}(x)=\text{arctanh}(2x-1),0<x<1\\\
&\varphi(x)=\frac{1}{2\cosh^{2}(x)}\end{array}$ (27)

Sigmoid kernel:

$\begin{array}[]{ll}&\Phi(x)=\frac{e^{x}}{e^{x}+1}\\\
&\Phi^{-1}(x)=-\ln(\frac{1}{x}-1),0<x<1\\\
&\varphi(x)=\frac{e^{x}}{\left(e^{x}+1\right)^{2}}\end{array}$ (28)

Rectangular (Rect) kernel:

$\begin{array}[]{ll}&\Phi(x)=\begin{cases}0&,x<-1\\\
\frac{1}{2}x+\frac{1}{2}&,-1\leq x\leq 1\\\ 1&,x>1\end{cases}\\\
&\Phi^{-1}(x)=2x-1,0<x<1\\\ &\varphi(x)=\begin{cases}\frac{1}{2}&,|x|\leq 1\\\
0&,\text{otherwise}\end{cases}\end{array}$ (29)

Figure 3: Visualization of Gaussian, Poisson, Hyperbolic, Sigmoid, Rect kernel
with $\epsilon=1$

These kernel functions define the corresponding Dirac sequences. As shown in
fig. 3, they have different shapes, which leads them to have different
smoothing properties. In $n$-dimensions, assuming that the dimensions are
independently and identically distributed,
$\varphi(x)=\prod_{i}^{n}\varphi(x_{i})$. And using variable substitution, it
is also easy to obtain $\Phi_{\epsilon}^{-1}(x)=\Phi^{-1}(x)\times\epsilon$.

###  4.3 Hyperparameter Selection

In our proposed smooth framework, there are two hyperparameters $N$ and
$\epsilon$. From the perspective of Monte Carlo integration, the value of $N$
should be large enough for $mg$ to converge as much as possible. Combining the
empirical values in Smilkov et al. (2017); Bykov et al. (2022), as shown in
fig. 4, in practice SG has converged to a reasonable result at $N=50$.
Therefore, to balance performance and computation, we suggest that $N$ is
taken to be around 50, and for FG then $N^{(\theta)}=M^{(x)}=50$.

With the constraints mentioned in section 4.2, $\varphi_{\epsilon}$ is set the
same as $p$. Therefore, $\varphi_{\epsilon}$ can be considered as a
probability density function. This allows us to understand the selection of
$\epsilon$ in a new sight, rather than on the basis of empirical values.

Figure 4: Visual explanation for random image labeled as dog using SmoothGrad
with Gaussian kernel and Sigmoid kernel. Visualization performance improves
with varying numbers of samples, maintaining a convincing quality when
reaching $N=50$. Figure 5: Visual explanation random image labeled as cat
using SmoothGrad with Gaussian kernel and Hyperbolic kernel. Visualization
performance ranges with different $\alpha$, and when $\alpha=0.9$, the
gradient map could present a decent performance, while when $\alpha=0.95$ it
instead loses some details of the object edge.

Similar to confidence intervals and confidence levels, we expect that within a
given interval $[-r,r]$, an appropriate $\epsilon$ will be selected such that
the samples $t_{i}\sim\varphi_{\epsilon}(t)$ lies within that interval
satisfies a given confidence level $\alpha$. Thus, $r$,$\epsilon$ and $\alpha$
satisfy the following equation,

$\frac{\alpha}{2}=\int_{0}^{r}\varphi_{\epsilon}(x)\,dx$ (30)

The value of $\epsilon$ could be calculated by substituting the given $r$ and
$\alpha$ in eq. 30. The analytical solutions of $\epsilon$ for these five
kernel functions are shown in appendix D. For SG, $r$ is suggested to be set
to $r=max(x)-\bar{x}$, i.e. the maximum value of the dataset minus the mean
value. For NG and FG, when adding noise to parameters of high layers in larger
models, the slight disturbance could cause numerical calculation errors. Thus,
referring to Bykov et al. (2022), the $\sigma$ in eq. 4 is set to $0.2$. Based
on the $3\sigma$ rule of normal distribution, it is recommended to set
$r=0.01$. So this makes it less likely to sample values that are outside the
range of the dataset. According to statistical conventions, the value of
$\alpha$ is recommended to be set to $0.9$ or $0.95$. In fig. 5, we show the
visualization of SG for different $\alpha$ values at $t=2$. Details of the
hyperparameter settings and comparison with manual settings are shown in
appendix E.

Table 1: Performance comparison among 15 smoothing methods for three smoothing
modes and five kernel function combinations. Original represents the original
gradient. $\uparrow$ and $\downarrow$ represent whether these indicators are
better higher or lower, respectively. The best-performing kernel function in
smooth mode is marked in bold.

Kernel | Consistency($\downarrow$) |  |  | Invariance($\uparrow$) |  |  | Localization($\uparrow$) |  |  | Sparseness($\uparrow$) |  |
---|---|---|---|---|---|---|---|---|---|---|---|---
| SG | NG | FG | SG | NG | FG | SG | NG | FG | SG | NG | FG
Gaussian | 0.03234 | 0.02817 | 0.03070 | 0.5921 | 0.5031 | 0.5380 | 0.7040 | 0.6857 | 0.7085 | 0.5200 | 0.5659 | 0.3276
Poisson | 0.03031 | 0.02759 | 0.02881 | 0.5947 | 0.5056 | 0.2805 | 0.6747 | 0.6923 | 0.6325 | 0.5075 | 0.5324 | 0.2596
Hyperbolic | 0.03214 | 0.02821 | 0.03082 | 0.5966 | 0.5030 | 0.5386 | 0.6917 | 0.6867 | 0.7050 | 0.5195 | 0.5639 | 0.3268
Sigmoid | 0.03238 | 0.02826 | 0.03065 | 0.5921 | 0.5031 | 0.5382 | 0.6917 | 0.6940 | 0.6970 | 0.5202 | 0.5645 | 0.3243
Rect | 0.03211 | 0.02814 | 0.03069 | 0.5972 | 0.5034 | 0.5392 | 0.6930 | 0.6950 | 0.6965 | 0.5177 | 0.5624 | 0.3310
Original | 0.02811 |  |  | 0.5016 |  |  | 0.6405 |  |  | 0.5587 |  |

##  5 Experiment

In this section, we test the performance of these five kernel functions in
different explainability metrics.

###  5.1 Experimental Settings

Metrics The employment of solitary metrics to gauge gradient-based methods is
a contentious issue Nauta et al. (2023), and numerous current studies Hedström
et al. (2023); Yang et al. (2022); Liu et al. (2022); Han et al. (2022); Jiang
et al. (2021) propose a plethora of evaluation metrics that even incorporate
human evaluation. Nevertheless, we have chosen four properties to assess
performance, as they align with the two fundamental objectives of model
interpretation: reflecting model decisions Adebayo et al. (2020); Sixt et al.
(2020); Crabbe et al. (2020); Fernando et al. (2019); Samek et al. (2016) and
facilitating human understandingBarkan et al. (2023); Wu et al. (2022);
Ibrahim and Shafiq (2022); Arras et al. (2022); Lu et al. (2021).

Consistency \- Consistency, first introduced in Adebayo et al. (2018), is
primarily concerned with whether the XAI method is compatible with the model’s
learning capability.

Invariance \- Invariance, proposed in Kindermans et al. (2019), checks the XAI
method to keep the output invariant in the case of constant data offsets in
datasets with the same model architecture.

Localization \- Localization, employed in Zhou et al. (2016); Selvaraju et al.
(2017); Zhang et al. (2018); Wang et al. (2020a); Barkan et al. (2023),
assesses the capability of the XAI method to accurately represent human-
recognisable features in the input image.

Sparseness \- Sparseness, applied in Chalasani et al. (2020), measures whether
the feature maps output by the XAI method are distinguishable and
identifiable.

Datasets and Models To apply these metrics for quantitative evaluation,
referring to the set up in Adebayo et al. (2018) and Kindermans et al. (2019),
we chose MNIST LeCun and Cortes (2005) and CIFAR10 Krizhevsky et al. (2009)
for experiments on Consistency and Invariance, and ILSVRC15 Russakovsky et al.
(2015) with target detection labels for experiments on Localization and
Sparseness.

Correspondingly, we constructed MLP and CNN models for the Consistency and
Invariance experiments. To measure the performance of the kernel function on
models of different parameter sizes, we selected pre-trained VGG16 Simonyan
and Zisserman (2014), MobileNetHoward et al. (2017), and InceptionSzegedy et
al. (2015) for the Localization and Sparseness experiments.

###  5.2 Results

Based on the hyperparameter settings outlined in section 4.3, we conducted
experiments to test the performance of the five kernel functions. The aim of
our study is not to identify a kernel function that completely outperforms
existing methods but rather to explore the research potential of theorem 3.11
and the possibility of designing new methods. This could help researchers to
design better smoothing methods for different models and datasets based on
theorem 3.11.

The experimental results are summarised in table 1, showing the performance of
15 different smoothing methods for three smoothing modes and five combinations
of kernel functions for different metrics. It could be observed that for
Consistency and Invariance, the kernel functions Poisson and Rect perform
better, while for Localization and Sparseness, the kernel functions Gaussian
and Rect perform better. In general, we believe that Gaussian (which
corresponds to existing methods like SmoothGrad), Poisson and Rect are
convincing options for gradient smoothing. More qualitative and quantitative
experimental results can be found in appendix F.

##  6 Conclusion

In this paper, we present a theoretical framework to explain the gradient
smoothing approach. The existing gradient smoothing methods are axiomatized
into a Monte Carlo smoothing formula, which reveals the mathematical nature of
gradient smoothing and explains the rationale that enable these methods to
remove gradient noise. Additionally, the theorem holds the potential for
designing new smoothing methods. Therefore, we also suggest methods for
designing smoothing approaches and propose several novel smoothing methods.
Through experiments, we comprehensively measure the performance of these
smoothing methods on different metrics and demonstrate the significant
research potential of the theorem.

Limitations and Future work

Kernel function selection. The selection of the kernel function should be
related to the model parameters and dataset, and how to select the appropriate
kernel function should be further explored.

Acceleration of Smooth Methods. We only explored the selection of mollifer and
ignored the sampling distribution. However, computing the gradient for NG and
FG is time-consuming. Therefore, selecting a suitable sampling distribution to
accelerate the calculation is a potential research topic.

Combination with other gradient-based methods. SmoothGrad has been used to
enhance explanation performance in Smooth Grad-CAM++Omeiza et al. (2019), SS-
CAMWang et al. (2020b), Smooth IGGoh et al. (2021), etc. So the integration of
this theoretical framework with other explanation approaches could be further
explored.

## References

  * Alvarez Melis and Jaakkola [2018] David Alvarez Melis and Tommi Jaakkola.  Towards robust interpretability with self-explaining neural networks.  _Advances in neural information processing systems_ , 31, 2018.
  * Zhou et al. [2023] Linjiang Zhou, Xiaochuan Shi, Yaxiong Bao, Lihua Gao, and Chao Ma.  Explainable artificial intelligence for digital finance and consumption upgrading.  _Finance Research Letters_ , 58:104489, 2023.
  * Abid et al. [2022] Abubakar Abid, Mert Yuksekgonul, and James Zou.  Meaningfully debugging model mistakes using conceptual counterfactual explanations.  In _International Conference on Machine Learning_ , pages 66–88. PMLR, 2022.
  * Doshi-Velez and Kim [2017] Finale Doshi-Velez and Been Kim.  Towards a rigorous science of interpretable machine learning.  _arXiv preprint arXiv:1702.08608_ , 2017.
  * Nauta et al. [2023] Meike Nauta, Jan Trienes, Shreyasi Pathak, Elisa Nguyen, Michelle Peters, Yasmin Schmitt, Jörg Schlötterer, Maurice van Keulen, and Christin Seifert.  From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai.  _ACM Computing Surveys_ , 55(13s):1–42, 2023.
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 618–626, 2017.
  * Chattopadhay et al. [2018] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian.  Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.  In _2018 IEEE winter conference on applications of computer vision (WACV)_ , pages 839–847. IEEE, 2018.
  * Wang et al. [2020a] Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu.  Score-cam: Score-weighted visual explanations for convolutional neural networks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops_ , pages 24–25, 2020a.
  * Sundararajan et al. [2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic attribution for deep networks.  In _International conference on machine learning_ , pages 3319–3328. PMLR, 2017.
  * Smilkov et al. [2017] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg.  Smoothgrad: removing noise by adding noise.  _arXiv preprint arXiv:1706.03825_ , 2017.
  * Bykov et al. [2022] Kirill Bykov, Anna Hedström, Shinichi Nakajima, and Marina M-C Höhne.  Noisegrad—enhancing explanations by introducing stochasticity to model weights.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pages 6132–6140, 2022.
  * Ancona et al. [2017] Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross.  Towards better understanding of gradient-based attribution methods for deep neural networks.  _arXiv preprint arXiv:1711.06104_ , 2017.
  * Nie et al. [2018] Weili Nie, Yang Zhang, and Ankit Patel.  A theoretical explanation for perplexing behaviors of backpropagation-based visualizations.  In _International conference on machine learning_ , pages 3809–3818. PMLR, 2018.
  * Adebayo et al. [2018] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim.  Sanity checks for saliency maps.  _Advances in neural information processing systems_ , 31, 2018.
  * Kindermans et al. [2019] Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Schütt, Sven Dähne, Dumitru Erhan, and Been Kim.  The (un) reliability of saliency methods.  _Explainable AI: Interpreting, explaining and visualizing deep learning_ , pages 267–280, 2019.
  * Yeh et al. [2019] Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Suggala, David I Inouye, and Pradeep K Ravikumar.  On the (in) fidelity and sensitivity of explanations.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Dombrowski et al. [2019] Ann-Kathrin Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus-Robert Müller, and Pan Kessel.  Explanations can be manipulated and geometry is to blame.  _Advances in neural information processing systems_ , 32, 2019.
  * Zeiler and Fergus [2014] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13_ , pages 818–833. Springer, 2014.
  * Springenberg et al. [2015] J Springenberg, Alexey Dosovitskiy, Thomas Brox, and M Riedmiller.  Striving for simplicity: The all convolutional net.  In _ICLR (workshop track)_ , 2015.
  * Du et al. [2018] Mengnan Du, Ninghao Liu, Qingquan Song, and Xia Hu.  Towards explanation of dnn-based prediction with guided feature inversion.  In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 1358–1367, 2018.
  * Bach et al. [2015] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  _PloS one_ , 10(7):e0130140, 2015.
  * Shrikumar et al. [2017] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje.  Learning important features through propagating activation differences.  In _International conference on machine learning_ , pages 3145–3153. PMLR, 2017.
  * Montavon et al. [2017] Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, and Klaus-Robert Müller.  Explaining nonlinear classification decisions with deep taylor decomposition.  _Pattern recognition_ , 65:211–222, 2017.
  * Bak et al. [2010] Joseph Bak, Donald J Newman, and Donald J Newman.  _Complex analysis_ , volume 8.  Springer, 2010.
  * Stein and Weiss [1971] Elias M Stein and Guido Weiss.  _Introduction to Fourier analysis on euclidean spaces (PMS-32), volume 32_.  Princeton Mathematical Series. Princeton University Press, Princeton, NJ, November 1971.
  * Febrianto et al. [2021] Eky Febrianto, Michael Ortiz, and Fehmi Cirak.  Mollified finite element approximants of arbitrary order and smoothness.  _Computer Methods in Applied Mechanics and Engineering_ , 373:113513, 2021.
  * Rohner [2019] Timo Rohner.  Test functions, mollifiers and convolution.  https://timorohner.com/files/mollifiers.pdf, 2019.  Last accessed on 2024-01-27.
  * Hedström et al. [2023] Anna Hedström, Leander Weber, Daniel Krakowczyk, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina M-C Höhne.  Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations and beyond.  _Journal of Machine Learning Research_ , 24(34):1–11, 2023.
  * Yang et al. [2022] Scott Cheng-Hsin Yang, Nils Erik Tomas Folke, and Patrick Shafto.  A psychological theory of explainability.  In _International Conference on Machine Learning_ , pages 25007–25021. PMLR, 2022.
  * Liu et al. [2022] Yibing Liu, Haoliang Li, Yangyang Guo, Chenqi Kong, Jing Li, and Shiqi Wang.  Rethinking attention-model explainability through faithfulness violation test.  In _International Conference on Machine Learning_ , pages 13807–13824. PMLR, 2022.
  * Han et al. [2022] Tessa Han, Suraj Srinivas, and Himabindu Lakkaraju.  Which explanation should i choose? a function approximation perspective to characterizing post hoc explanations.  _Advances in Neural Information Processing Systems_ , 35:5256–5268, 2022.
  * Jiang et al. [2021] Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei.  Layercam: Exploring hierarchical class activation maps for localization.  _IEEE Transactions on Image Processing_ , 30:5875–5888, 2021.
  * Adebayo et al. [2020] Julius Adebayo, Michael Muelly, Ilaria Liccardi, and Been Kim.  Debugging tests for model explanations.  _Advances in Neural Information Processing Systems_ , 33:700–712, 2020.
  * Sixt et al. [2020] Leon Sixt, Maximilian Granz, and Tim Landgraf.  When explanations lie: Why many modified bp attributions fail.  In _International Conference on Machine Learning_ , pages 9046–9057. PMLR, 2020.
  * Crabbe et al. [2020] Jonathan Crabbe, Yao Zhang, William Zame, and Mihaela van der Schaar.  Learning outside the black-box: The pursuit of interpretable models.  _Advances in neural information processing systems_ , 33:17838–17849, 2020.
  * Fernando et al. [2019] Zeon Trevor Fernando, Jaspreet Singh, and Avishek Anand.  A study on the interpretability of neural retrieval models using deepshap.  In _Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval_ , pages 1005–1008, 2019.
  * Samek et al. [2016] Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller.  Evaluating the visualization of what a deep neural network has learned.  _IEEE transactions on neural networks and learning systems_ , 28(11):2660–2673, 2016.
  * Barkan et al. [2023] Oren Barkan, Yuval Asher, Amit Eshel, Noam Koenigstein, et al.  Visual explanations via iterated integrated attributions.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 2073–2084, 2023.
  * Wu et al. [2022] Yuxi Wu, Changhuai Chen, Jun Che, and Shiliang Pu.  Fam: Visual explanations for the feature representations from deep convolutional networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10307–10316, 2022.
  * Ibrahim and Shafiq [2022] Rami Ibrahim and M Omair Shafiq.  Augmented score-cam: High resolution visual interpretations for deep neural networks.  _Knowledge-Based Systems_ , 252:109287, 2022.
  * Arras et al. [2022] Leila Arras, Ahmed Osman, and Wojciech Samek.  Clevr-xai: A benchmark dataset for the ground truth evaluation of neural network explanations.  _Information Fusion_ , 81:14–40, 2022.
  * Lu et al. [2021] Yang Young Lu, Wenbo Guo, Xinyu Xing, and William Stafford Noble.  Dance: Enhancing saliency maps using decoys.  In _International Conference on Machine Learning_ , pages 7124–7133. PMLR, 2021.
  * Zhou et al. [2016] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2921–2929, 2016.
  * Zhang et al. [2018] Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff.  Top-down neural attention by excitation backprop.  _International Journal of Computer Vision_ , 126(10):1084–1102, 2018.
  * Chalasani et al. [2020] Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha.  Concise explanations of neural networks using adversarial training.  In _International Conference on Machine Learning_ , pages 1383–1391. PMLR, 2020.
  * LeCun and Cortes [2005] Yann LeCun and Corinna Cortes.  The mnist database of handwritten digits, 2005.  URL https://api.semanticscholar.org/CorpusID:60282629.  Last accessed on 2024-01-27.
  * Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * Russakovsky et al. [2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  _International Journal of Computer Vision (IJCV)_ , 115(3):211–252, 2015.  doi:10.1007/s11263-015-0816-y.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ , 2014.
  * Howard et al. [2017] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  _arXiv preprint arXiv:1704.04861_ , 2017.
  * Szegedy et al. [2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1–9, 2015.
  * Omeiza et al. [2019] Daniel Omeiza, Skyler Speakman, Celia Cintas, and Komminist Weldermariam.  Smooth grad-cam++: An enhanced inference level visualization technique for deep convolutional neural network models.  _arXiv preprint arXiv:1908.01224_ , 2019.
  * Wang et al. [2020b] Haofan Wang, Rakshit Naidu, Joy Michael, and Soumya Snigdha Kundu.  Ss-cam: Smoothed score-cam for sharper visual feature localization.  _arXiv preprint arXiv:2006.14255_ , 2020b.
  * Goh et al. [2021] Gary SW Goh, Sebastian Lapuschkin, Leander Weber, Wojciech Samek, and Alexander Binder.  Understanding integrated gradients with smoothtaylor for deep neural network attribution.  In _2020 25th International Conference on Pattern Recognition (ICPR)_ , pages 4949–4956. IEEE, 2021.
  * Rudin [1987] Walter Rudin.  _Real and complex analysis, Third Edition_.  McGraw-Hill, 1987.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * LeCun et al. [1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.

##  Appendix A Proofs in section 3.1

The mathematical concepts mentioned in section 3.1 are in the field of
functional analysis Rudin [1987]. Thus for the convenience of readers to
understand, we have provided all the proofs below.

Proof of lemma 3.2.

###### Proof.

In definition 3.1,

$(f*g)(x)=\int_{-\infty}^{\infty}f(t)g(x-t)\,dt$ (31)

and using substitution, let $t=x-t^{\prime}$. So we could get,

$(f*g)(x)=\int_{\infty}^{-\infty}-f(x-t^{\prime})g(t^{\prime})dt^{\prime}$
(32)

Next, exchange the upper and lower limits of integrals to obtain,

$(f*g)(x)=\int_{-\infty}^{\infty}f(x-t^{\prime})g(t^{\prime})dt^{\prime}$ (33)

In the form, we finally get $g*f$. Thus, lemma 3.2 is proved. ∎

Proof of lemma 3.3.

###### Proof.

We compute the following

$\displaystyle(f*g)^{\prime}(x)$ $\displaystyle=\lim_{\Delta x\to
0}\frac{(f*g)(x+\Delta x)-(f*g)(x)}{\Delta x}$ (34)
$\displaystyle=\lim_{\Delta x\to 0}\frac{\int_{-\infty}^{\infty}f(y)g(x+\Delta
x-y)\,dy-\int_{-\infty}^{\infty}f(y)g(x-y)\,dy}{\Delta x}$
$\displaystyle=\lim_{\Delta x\to
0}\int_{-\infty}^{\infty}f(y)\left(\frac{g(x+\Delta x-y)-g(x-y)}{\Delta
x}\right)dy$ $\displaystyle=\int_{-\infty}^{\infty}f(y)\left(\lim_{\Delta x\to
0}\frac{g(x+\Delta x-y)-g(x-y)}{\Delta x}\right)dy$
$\displaystyle=\int_{-\infty}^{\infty}f(y)g^{\prime}(x-y)dy$
$\displaystyle=(f*g^{\prime})(x)$

Next, using lemma 3.2, we could get
$(f*g)^{\prime}=f*g^{\prime}=f^{\prime}*g$. ∎

Proof of lemma 3.4

###### Proof.

If $g$ is continuous, and then for any convergent sequence
$\\{x_{m}\\}\subset\mathbb{R}^{n},x_{m}\to x\in\mathbb{R}^{n}$,

$\lim_{m\to\infty}g(x_{m})=g(x)$ (35)

Then, we could get

$\displaystyle\lim_{m\to\infty}(f*g)(x_{m})$
$\displaystyle=\lim_{m\to\infty}\int_{\mathbb{R}^{n}}f(y)g(x_{m}-y)\,dy$ (36)
$\displaystyle=\int_{\mathbb{R}^{n}}f(y)\lim_{m\to\infty}g(x_{m}-y)\,dy$
$\displaystyle=\int_{\mathbb{R}^{n}}f(y)g(x-y)\,dy$ $\displaystyle=(f*g)(x)$

According to the definition of continuous, $f*g$ is continuous. ∎

Proof of lemma 3.6.

###### Proof.

Using the definition 3.5 and definition 3.1 we could get

$\displaystyle\int_{-\infty}^{\infty}\delta(y)f(x-y)\,dy$
$\displaystyle=\lim_{\epsilon\to
0}\int_{-\epsilon}^{\epsilon}\delta(y)f(x-y)\,dy$ (37)
$\displaystyle=f(x)\lim_{\epsilon\to 0}\int_{-\epsilon}^{\epsilon}\delta(y)dy$
$\displaystyle=f(x)$

Thus, $\delta*f=f*\delta=f$ ∎

Proof of lemma 3.7.

###### Proof.

lemma 3.7 could be easily proved using lemma 3.3 and lemma 3.6. ∎

##  Appendix B Proofs of the Unbiasedness and Consistency

Proof of the unbiasedness of statistic $mg_{\varepsilon}(x_{0})$. Unbiasedness
means that the sample expectation of a statistic is the true value of the
overall parameter.

###### Proof.

According to the definition of mathematical expectations,

$\displaystyle\mathbb{E}\left[mg_{\varepsilon}(x_{0})\right]$
$\displaystyle=\mathbb{E}\left[\frac{1}{N}\sum_{i=1}^{N}\frac{g(x_{0}-t_{i})\varphi_{\epsilon}(t_{i})}{p(t_{i})}\right]$
(38)
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\mathbb{E}\left[\frac{g(x_{0}-t_{i})\varphi_{\epsilon}(t_{i})}{p(t_{i})}\right]$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\int_{\mathbb{R}^{n}}\frac{g(x_{0}-t)\varphi_{\epsilon}(t)}{p(t)}p(t)\,dt$
$\displaystyle=\int_{\mathbb{R}^{n}}g(x_{0}-t)\varphi_{\epsilon}(t)\,dt$

By eq. 16,

$\mathbb{E}\left[mg_{\varepsilon}(x_{0})\right]=\int_{\mathbb{R}^{n}}g(x_{0}-t)\varphi_{\epsilon}(t)\,dt=g_{\epsilon}(x_{0})$
(39)

∎

Proof of the consistency of statistic $mg_{\varepsilon}(x_{0})$. Consistency
means that as the number of samples increases, the estimates converge more and
more to the true value. Specifically, the variance of the statistic converges
to 0 as the sample size increases.

###### Proof.

According to the definition of variance,

$\displaystyle\sigma^{2}\left[mg_{\epsilon}(x_{0})\right]$
$\displaystyle=\sigma^{2}\left[\frac{1}{N}\sum_{i=1}^{N}\frac{g(x_{0}-t_{i})\varphi_{\epsilon}(t_{i})}{p(t_{i})}\right]$
(40)
$\displaystyle=\frac{1}{N^{2}}\sum_{i=1}^{N}\sigma^{2}\left[\frac{g(x_{0}-t_{i})\varphi_{\epsilon}(t_{i})}{p(t_{i})}\right]$

Note that $G=\frac{g(x_{0}-T)\varphi_{\epsilon}(T)}{p(T)}$,

$\sigma^{2}\left[mg_{\epsilon}(x_{0})\right]=\frac{1}{N^{2}}\sum_{i=1}^{N}\sigma^{2}[G_{i}]=\frac{1}{N^{2}}(N\sigma^{2}[G])=\frac{\sigma^{2}[G]}{N}$
(41)

That is,

$\sigma\left[mg_{\epsilon}(x_{0})\right]=\frac{\sigma[G]}{\sqrt{N}}$ (42)

Clearly, the variance of statistic $mg_{\epsilon}(x_{0})$ decreases as $N$
increases, i.e., $\lim_{N\to 0}\sigma\left[mg_{\epsilon}(x_{0})\right]=0$. ∎

##  Appendix C Details of the Mollification Example

We construct a piecewise function $f(x):\mathbb{R}\to\mathbb{R}$ to simulate a
rough and unsmooth function. The expression of $f(x)$ is,

$f(x)=\left\\{\begin{array}[]{llllll}0&,\,x\leq 0\\\ 2x&,\,0<x\leq 1\\\
4-2x&,\,1\leq x<\frac{3}{2}\\\ -1+2x&,\,\frac{3}{2}\leq x<3\\\ 8-x&,\,3\leq
x<4\\\ 4&,\,4\leq x\end{array}\right.$ (43)

The selected mollifier $\varphi_{\epsilon}(x)$ is,

$\varphi_{\epsilon}(x)=\frac{1}{\sqrt{2\pi}\epsilon}e^{-\frac{x^{2}}{2\epsilon^{2}}}$
(44)

The mollification $f*\varphi_{\epsilon}$ could be calculated as,

$\displaystyle f*\varphi_{\epsilon}(x)=$
$\displaystyle\frac{1}{2}(x-4)\text{erf}\left(\frac{x-4}{\sqrt{2}\epsilon}\right)-\frac{3}{2}(x-3)\text{erf}\left(\frac{x-3}{\sqrt{2}\epsilon}\right)+2\text{erf}\left(\frac{x-1}{\sqrt{2}\epsilon}\right)$
(45)
$\displaystyle+x\left(-2\text{erf}\left(\frac{x-1}{\sqrt{2}\epsilon}\right)+\text{erf}\left(\frac{x}{\sqrt{2}\epsilon}\right)+2\text{erf}\left(\frac{2x-3}{2\sqrt{2}\epsilon}\right)\right)+\frac{5}{2}\text{erf}\left(\frac{3-2x}{2\sqrt{2}\epsilon}\right)$
$\displaystyle+\frac{\epsilon\left(2e^{-\frac{x^{2}}{2\epsilon^{2}}}+4e^{-\frac{(3-2x)^{2}}{8\epsilon^{2}}}+e^{-\frac{(x-4)^{2}}{2\epsilon^{2}}}-3e^{-\frac{(x-3)^{2}}{2\epsilon^{2}}}-4e^{-\frac{(x-1)^{2}}{2\epsilon^{2}}}\right)}{\sqrt{2\pi}}+2\sqrt{\frac{1}{\epsilon^{2}}}\epsilon$

##  Appendix D Details of Kernel Functions

The five kernel function is derived from USF approximation. fig. 6 shows
images of these five USF approximation functions.

Figure 6: Visualization of the five USF approximation functions.

The USF approximation allows for quick computation of the analytical solution
of $\epsilon$ in eq. 30 when generating the kernel function. In contrast, the
standard mollifier, as shown in eq. 46, usually adopted in the field of
partial differential equations is difficult to get a simple format for Monte
Carlo sampling.

$\varphi_{\epsilon}(x)=\begin{cases}\frac{1}{C}e^{(|x|^{2}-\epsilon)^{-1}}&,\text{if
}|x|<\epsilon,\\\ 0&,\text{if }|x|>\epsilon.\end{cases}$ (46)

The expressions for $\epsilon$ corresponding to these five kernel functions
are given below,

Gaussian kernel:

$\epsilon=\frac{r}{\sqrt{2}\text{Erfinv}(\alpha)}$ (47)

There, Erfinv is the inverse error function.

Poisson kernel:

$\epsilon=\frac{r}{\tan(\frac{\pi}{2}\alpha)}$ (48)

Hyperbolic kernel:

$\epsilon=\frac{r}{\arctan(\alpha)}$ (49)

Sigmoid kernel:

$\epsilon=\frac{r}{\ln(\frac{1+\alpha}{1-\alpha})}$ (50)

Rect kernel:

$\epsilon=\frac{2r}{\alpha}$ (51)

##  Appendix E Experiment for Hyperparameter Selection

In section 4.3, we provided an explanation for the hyperparameter selections
in previous works. By eq. 30, a reasonable $\epsilon$ could be calculated with
confidence level $\alpha$ and interval $[-r,r]$. fig. 7 shows that in the same
$\alpha$ and $r$ settings, the shape of these five kernel functions will tend
to align, but still retain some differences, which causes the functions to
have varying smoothing performance. In Smilkov et al. [2017], $\epsilon$ is
recommended to set by,

$\epsilon=\beta\times(x_{min}-x_{max})$ (52)

Following the experimental setup in section 5.1, we tested the performance of
the SG method using the Gaussian kernel function in different $\beta$. Our
method is equivalent to $\beta$ value between 0.2 and 0.3. The experimental
performance is shown in table 2. The evaluation of Localization and Sparseness
was only performed on VGG16. The performance difference among these
hyperparameter settings is relatively slight. Our purpose is not to find an
optimal hyperparameter but to show that our theoretical explanation of the
settings of the hyperparameter $\epsilon$ in the gradient smoothing method is
applicable.

Figure 7: Visualization of five kernel functions with the same $\alpha$ and $r$ settings. Table 2: Performance comparison among different $\epsilon$ settings. $\epsilon$ value | Consistency($\downarrow$) | Invariance($\uparrow$) | Localization($\uparrow$) | Sparseness($\uparrow$)
---|---|---|---|---
$\beta=0.1$ | 0.03084 | 0.5217 | 0.699 | 0.5438
$\beta=0.2$ | 0.03157 | 0.5584 | 0.694 | 0.5320
$\beta=0.3$ | 0.03217 | 0.5922 | 0.698 | 0.5151
ours($\beta=0.2\sim 0.3$) | 0.03234 | 0.5922 | 0.699 | 0.5243
original | 0.02811 | 0.5016 | 0.709 | 0.5393

##  Appendix F More Experimental Details

###  F.1 Details of Metrics

Consistency

In Adebayo et al. [2018], the explanation methods are required to be
consistent with the learning ability of the model. Similar to the statistical
randomization test, the output of the explanation method should differentiate
between models that are randomized and those that are trained normally. We
refer to the work of Adebayo et al. [2018] and employ Spearman rank
correlation to compute the difference between the outputs. Similarly, we use
two regularisation methods to regularise the outputs, thus facilitating the
computation of Spearman rank correlation. In table 1, we show the average of
the two methods.

Invariance

Proposed in Kindermans et al. [2019], the explanation method was considered to
maintain invariance to feature shifts that do not contribute. The authors
applied a constant offset to the dataset to simulate invalid feature shifts.
However, the authors only utilize visual analysis to determine whether the
output of the explanation method remains invariant. Therefore, we further
refer to the work of Hedström et al. [2023] and use rank correlation to
quantify invariance.

Localization

Localization ability is a common evaluation metric for feature attribution-
based explanation methods. Especially for a visual model-based explanation,
this metric is optimized as almost the only goal by many explanation methods
Wu et al. [2022], Jiang et al. [2021], Wang et al. [2020a]. However, it is
important to note that many studies that measure localization ability based on
object localization tasks tend to employ additional heuristics for generating
bounding boxes for saliency maps. Therefore, based on the goals of our
research, we do not select this approach, and instead, we employ the Point
Game Hedström et al. [2023], Zhang et al. [2018] approach for evaluating
localization ability. Specifically, the top-k (in our experiment, $k=5$)
points with the maximal values on the output saliency map are selected, and we
evaluate whether these points are in the bounding box of the ground truth or
not. The accuracy in Point Game then quantitatively represents the
localization ability.

Sparseness

In Chalasani et al. [2020], sparseness was introduced to evaluate the
visualization capability of explanation methods. The idea of sparseness is
that the explanation output should mainly focus on the important features,
thus the saliency of the output should be as sparse as possible in order to
make it easier for humans to visualize important features. Referring to
Chalasani et al. [2020], we adopted the Gini index to quantify the sparsity of
the output.

###  F.2 Details of Experimental Environment

VGG16, MobileNet, and Inception are constructed by pre-trained models released
in Torchvision444https://pytorch.org/vision/stable/index.html. The MLP
architecture consists of two linear layers with 200 and 10 units respectively.
The MLP was trained on MNIST by SGD optimizer with 20 epochs and the learning
rate was set to 0.01. The CNN architecture contains a few layers as follows:
[Conv(6), Maxpool(2), Conv(16), Maxpool(2), Linear(120), Linear(84),
Linear(10)]. The CNN was trained on CIFAR10 by SGD optimizer with 20 epochs
and the learning rate was set to 0.01.

Due to the time-consuming computation of FG and NG (on average, it took 3-4
min to compute the gradient of one sample on our hardware), we randomly
sampled 1,000 samples instead of all validation or test set samples for the
comparison experiments. Our experiments were conducted on a machine with
4$\times$NVIDIA RTX 4090 and 2$\times$Intel Xeon Gold 6128. All experimental
codes and saved model parameters with detailed results can be found in the
Supplementary Material.

###  F.3 Additional Experimental Results

For the comparison experiments on CNN, there are some differences in
Consistency and Invariance metrics. Therefore, we present the detailed data
separately in table 3. It can be noticed that the values of Invariance metrics
are significantly lower than the values in table 1, and we speculate that this
may be due to the low accuracy (0.5922 in our experiment) of the CNN model in
the CIFAR10 dataset.

Table 3: Performance comparison among 15 smoothing methods of CNN in CIFAR10. Kernel | Consistency($\downarrow$) |  |  | Invariance($\uparrow$) |  |
---|---|---|---|---|---|---
| SG | NG | FG | SG | NG | FG
Gaussian | 0.01721 | 0.01691 | 0.01574 | 0.02951 | 0.02720 | 0.02509
Poisson | 0.02042 | 0.01679 | 0.01838 | 0.07720 | 0.02077 | 0.02111
Hyperbolic | 0.01833 | 0.01722 | 0.01558 | 0.03096 | 0.02760 | 0.02604
Sigmoid | 0.01745 | 0.01647 | 0.01743 | 0.02959 | 0.02745 | 0.02484
Rect | 0.01660 | 0.01677 | 0.01635 | 0.03022 | 0.02739 | 0.02600
Original | 0.01567 |  |  | 0.02622 |  |

To visualize the difference in performance between these five kernel
functions, we show an example of gradient smoothing for all five kernel
functions on the MNIST dataset in fig. 8. It can be observed that the
smoothing performance of the Poisson kernel function differs significantly
from that of the other kernel functions. As shown in fig. 7, this difference
is due to the distinct shape of the Poisson kernel function.

Figure 8: Visual explanation for a random image from MNIST labeled as 0 with
five kernel functions and three smoothing modes.

Localization is probably one of the most interesting metrics because it is
directly related to the visualization of the gradient map. Therefore, we
present the Localization metrics for the different gradient smoothing methods
on the three models separately in Table table 4.

It is worth noting that the FG mode requires perturbing both model parameters
and inputs. Therefore, in our experiments, we found that for models with
larger parameters and deeper layers, the FG mode is extremely vulnerable to
lead to numerical errors when perturbing the higher layers. Therefore, for
models with fewer parameters such as VGG16 and MobileNet, the FG method was
able to output valid gradient values, while for the Inception model, we
dropped the anomalous data due to the presence of too many invalid gradients.
Thus, in table 1 and table 4, these outliers are excluded from our statistics.

In fact, in Bykov et al. [2022], where FusionGrad is proposed, FusionGrad was
only tested on ResNet18 He et al. [2016], VGG, and LeNet LeCun et al. [1998].
The number of parameters in each of these models is relatively small.
Therefore, we do not recommend using this method on models with a relatively
large number of parameters.

Table 4: Localization performance comparison among 15 smoothing methods Kernel | VGG16 |  |  | MobileNet |  |  | Inception |  |
---|---|---|---|---|---|---|---|---|---
| SG | NG | FG | SG | NG | FG | SG | NG | FG
Gaussian | 0.699 | 0.691 | 0.723 | 0.664 | 0.600 | 0.694 | 0.749 | 0.766 | -
Poisson | 0.712 | 0.693 | 0.707 | 0.552 | 0.624 | 0.558 | 0.760 | 0.760 | -
Hyperbolic | 0.701 | 0.695 | 0.722 | 0.636 | 0.612 | 0.688 | 0.738 | 0.753 | -
Sigmoid | 0.698 | 0.696 | 0.734 | 0.632 | 0.618 | 0.660 | 0.745 | 0.768 | -
Rect | 0.692 | 0.694 | 0.721 | 0.636 | 0.616 | 0.672 | 0.751 | 0.775 | -
Original | 0.709 |  |  | 0.572 |  |  | 0.761 |  |