# Improving IoT-over-Satellite Connectivity
using Frame Repetition Technique

School of Engineering, RMIT University, Melbourne, Australia 3000  Bisma
Manzoor, , Akram Al-Hourani, ,
and Bassel Al Homssi, .
This research is supported by the Australian Government Research Training
Program (RTP) Scholarship. The authors are with the School of Engineering,
RMIT University, Melbourne, VIC 3000, Australia

###### Abstract

Through Non-Terrestrial Networks (NTN), a global coverage connecting areas
with minimal or no terrestrial services is envisaged using satellite and air-
borne platforms. However, one of the challenges to delivering NTN
communications is the increased levels of path-loss, relative to typical
terrestrial scenarios, due to the vast communication distance towards
satellite platforms. Typically, cellular IoT technologies adopt frame
repetitions where many redundant data transmissions are included to enhance
its success rate. This can be also extended to satellite links to counter the
effect of long distance communications on the link budget. In this paper, we
put forth an analytic framework that captures the repetition behavior based on
the probability of line-of-sight (LoS), as it heavily influences the
propagation conditions. We analyze the coverage performance in terms of the
frame success rate, which allows us to tune the repetitions behavior to suit a
given satellite admittance region. Furthermore, we obtain an optimal global
average success rate by considering the availability of satellites. The
presented framework can facilitate coverage enhancements for IoT-over-
Satellite networks.

_Keywords - IoT-over-satellite, frame repetitions, Non Terrestrial Networks,
stochastic geometry, satellite networks_

##  I Introduction

The current era is witnessing a massive surge in IoT adoptions arising from a
wide range of applications such as, smart farming, logistics, environmental
monitoring, and smart cities. The number of cellular IoT subscriptions are
expected to reach a staggering number of approximately 5 billion by 2026 [1].
To support the increasingly ubiquitous IoT applications, it is required to
have a network offering an extended worldwide coverage [2]. So far, the
traditional terrestrial network has been limited to providing robust
connectivity to urban, suburban and some rural areas. However, due to the lack
of economical, and sometimes technical, feasibility such terrestrial networks
fell short in providing such an envisioned global coverage. As such, Third
Generation Partnership Project (3GPP) included Non-Terrestrial Networks (NTN)
[3] as part of 5G road-map towards next generation hybrid networks [4, 5]. The
NTN comprises space and aerial platforms, i.e., satellites and UAVs, that act
either as relaying nodes or as stand-alone base-stations, i.e., gNB, and hence
paving the road for global coverage services. However, the application of IoT
over NTN comes with many challenges. The NTN channel has much higher path-loss
when compared to terrestrial, where most of the propagation path is under free
space condition and only interacts with a clutter layer, terrestrial
scatterers, just before reaching the ground user. This is especially
applicable for lower frequencies suitable for some IoT applications in the L
and S bands. This is also quite different than terrestrial channels where all
the propagation happens within the clutter layer. As such, LoS probability
plays a major role in determining the radio channel conditions. In turn, the
LoS probability is highly dependent on the elevation angle [6], which is the
angle between the satellite and the horizon as seen by the ground user. The
LoS dependency in an NTN channel impacts the signal power and coverage
probability.
One of the attributes adopted by cellular IoT access technologies,
specifically by Narrow-band Internet-of-Things (NB-IoT), is to improve the
coverage by means of frame repetition. In such mechanisms, devices are
configured to repeat a message multiple times to improve the detection
probability at the receiver. As a result, devices that operate under poor
coverage conditions tend to transmit more frame repetitions than those under
better conditions. Multiple works evaluate the performance of repetition
mechanism on terrestrial network [7, 8]. Moreover, the evaluation of IoT over
satellite access technologies has been proposed in several good works such as
[9, 10, 11], where work in [9] analyzes the performance of NB-IoT and
addresses the challenges related to delay, while the work in [10] presents the
link budget analysis for IoT devices. However, many challenges remain
unresolved such as the impact of interference induced by large numbers of
repetition and the performance of frame repetition in satellite radio
channels.
In this paper, we leverage the concept of frame repetition, which is already
deployed in several terrestrial IoT access technologies, and extend its
application in IoT-over-satellite under realistic satellite-to-ground radio
propagation conditions. We further evaluate the coverage probability of IoT
devices in the admittance area of a satellite, while implementing repetition
scheme. The contribution of this paper is the use of tools from stochastic
geometry for analytically formulating a framework that models frame
repetitions in terms of probability of line-of-sight. The paper further
demonstrates the ability to tune the repetition factor and the admittance
region such that an optimal repetition profile maximizes the global average
success rate while considering the satellite availability.

##  II System Model

###  II-A Geometric Model

Consider a satellite in a circular orbit at an altitude $h$ from the mean sea
level. Ground IoT devices are assumed to be distributed by an isotropic and
homogeneous Poisson Point Process (PPP) on the Earth’s surface. An important
parameter that impacts the performance is the angle formed between the
satellite point, the Earth center, and the observer, this angle is termed as
zenith angle, denoted $\varphi$.The elevation angle $\theta$, which is the
angle between satellite and horizon seen by ground user, can be obtained from
the zenith angle using geometric reasoning as follows [4]

$\displaystyle\theta=\mathrm{acot}\left(\frac{\sin~{}\varphi}{\cos~{}\varphi-\alpha}\right),~{}\text{where}~{}\alpha=\frac{R}{h+R}.$
(1)

The domain of the elevation angle is $\theta\in[0,\frac{\pi}{2}]$, where
$\theta=0$ represents the case when the satellite is at the horizon and
$\theta=\pi/2$ is when the satellite is at the device zenith. Accordingly,
$\varphi\in[\varphi_{\mathrm{h}},0]$ $\rightarrow$
$\theta\in[0,\frac{\pi}{2}]$, where $\varphi_{\mathrm{h}}$ is the maximum
zenith angle formed at the horizon. The scenario is depicted in Fig. 1. It can
be shown that the maximal horizon zenith angle is given by [12],
$\varphi_{\mathrm{h}}=\text{acos}~{}\alpha$. As such, the serving satellite
can be geometrically seen from any point in a certain spherical cap region.
However, note that IoT devices that have very low elevation angle are deemed
to have an extremely poor radio link performance as a result of deep path-
loss. Consequently, the probability of successful transmission in such cases
is low. As such, we define a minimum elevation angle, denoted as
$\theta_{\mathrm{min}}$, which limits the admittance region of IoT devices.
There are multiple ways to limit the admittance region practically; for
example by controlling the beamwidth of the satellite or by programmaticaly
preventing IoT devices from transmitting when the satellite is below the
threshold elevation angle, i.e., $\theta\leq\theta_{\mathrm{min}}$. Due to
this impact, the zenith angle becomes lower than $\varphi_{\mathrm{h}}$ and
thus limited by the general threshold $\varphi_{\mathrm{max}}$, i.e.
$\varphi\in[0,\varphi_{\mathrm{max}}]$ $\rightarrow$
$\theta\in[\frac{\pi}{2},\theta_{\mathrm{min}}]$. The relation between these
two angles is given by (1) as follows

$\displaystyle\theta_{\mathrm{min}}=\mathrm{acot}\left(\frac{\sin~{}\varphi_{\mathrm{max}}}{\cos~{}\varphi_{\mathrm{max}}-\alpha}\right),~{}\text{where}~{}\alpha=\frac{R}{h+R}.$
(2)

Due to the near ground objects and foliage, i.e., also called clutter,
geometric LoS is sometimes blocked by these obstacles. The probability of
line-of-sight $p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}$ is an
important parameter that heavily influences the path-loss and the probability
of successful transmission. Therefore, we adopt the LoS model developed in
[12]. However, we re-write the
$p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}$ in terms of $\varphi$ as
follows

$\displaystyle p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}$
$\displaystyle=\exp\left(-\beta\cot\theta\right)=\exp\left(-\beta\frac{\sin~{}\varphi}{\cos~{}\varphi-\alpha}\right),$
(3)

where $\beta$ is a controllable modeling parameter related to the underlaying
propagation environment. Note that the probability of non-line-sight is given
as
$p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{NLoS}$}}}}=1-p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}$.

Figure 1: Illustration of a satellite admittance region, zenith angles and the
IoT device at a given elevation angle.

###  II-B Repetition Model

Since the purpose of repetitions to improve the coverage [13], the devices
with lower line-of-sight probability are expected to rely on this feature more
often in order to achieve successful transmission. This intuitively indicates
that devices with larger zenith angle, $\varphi$, will need more repetitions.
Hence, the developed model in [14] which primarily depends on the distance
towards the terrestrial base station is adopted but with changing this
dependency to the zenith angle. Assuming an initial duty cycle of
$D_{\mathrm{o}}$111$D_{\mathrm{o}}$ is ratio of the time-on-air of message
$T_{\mathrm{m}}$ to the maximum duration till next update $T_{\mathrm{max}}$,
the effective duty cycle, when, repetition is applied, is modeled as

$D\left(\varphi\right)=D_{\mathrm{o}}+(1-D_{\mathrm{o}})g(\varphi),$ (4)

where $g(\varphi)$ is a custom shaping function that satisfies the criterion
$g\left(\varphi\right)\in\left[0,1\right]$,
$D\in\left[D_{\mathrm{o}},1\right]$, since the duty cycle cannot exceed unity.
A suitable function that satisfies the above requirement is a modified version
of the probability of the non-line-of-sight as follows,

$D\left(\varphi\right)=D_{\mathrm{o}}+(1-D_{\mathrm{o}})\left[1-\exp\left(-a\beta\frac{\sin~{}\varphi}{\cos~{}\varphi-\alpha}\right)\right],$
(5)

where $\varphi\in\left(0,\varphi_{\mathrm{max}}\right)$ and $a$
$\in\left[0,1\right]$ is a controllable tuning factor. Accordingly, the number
of transmissions, i.e., repetitions is give by

$N\left(\varphi\right)=\left\lceil{\frac{D\left(\varphi\right)}{D_{\mathrm{o}}}}\right\rceil.$
(6)

The impact of tuning factor $a$ on the duty cycle and repetitions is depicted
in Fig. 2 and Fig. 3, respectively. Smaller values of $a$ correspond to lower
overall repetitions and vice-versa. Furthermore, the figures also illustrate
the effect of the admittance angle $\theta_{\mathrm{min}}$. It is to be noted
that both $a$ and $\theta_{\mathrm{min}}$ can be tuned in order to optimize
the overall success probability. Due to increased transmissions as a result of
repetition, the effective device density in the admittance region becomes
$\lambda(\varphi)=\lambda_{o}D\left(\varphi\right)$, where
$\lambda_{\mathrm{o}}$ is the density of ground devices. That is, in case of
no repetition, $D(\varphi)\to D_{\mathrm{o}}$ and
$\lambda(\varphi)\to\lambda_{\mathrm{o}}$. A thorough explanation of
repetition model is given in [14].

Figure 2: Effective duty cycle $D(\varphi)$ in (5) vs $\theta$ for various
combinations of tuning factor $a$ and $\theta_{\mathrm{min}}$. Figure 3:
Repetitions $N(\varphi)$ in (6) vs $\theta$ for various combinations of tuning
factor $a$ and $\theta_{\mathrm{min}}$.

###  II-C Zenith angle distribution

For a given satellite admittance region defined by the angle domain,
$\theta\in\left[\theta_{\mathrm{min}},\frac{\pi}{2}\right)$, the users will
view the serving satellite within the zenith angle domain,
$\varphi\in\left(0,\varphi_{\mathrm{max}}\right)$, where
$\varphi_{\mathrm{max}}\leq\varphi_{\mathrm{h}}$, as shown in Fig. 1. As
introduced earlier, the elevation angle or the equivalent zenith angle of the
satellite heavily impacts the repetition rate. Thus, the analysis of
satellite’s zenith angle distribution is essential to obtain the probability
of successful transmission. Assuming PPP, the ground devices or IoT devices
are randomly placed within the admittance region. Accordingly, the probability
of finding devices within a given angle $\varphi_{\mathrm{o}}$ is formulated
as follows

$\displaystyle\scriptstyle
F_{\varphi}(\varphi_{\mathrm{o}})=\mathbb{P}\left(\varphi\leq\varphi_{\mathrm{o}}\right)$
$\displaystyle=\frac{\text{avg. no. of active devices in
$\varphi_{\mathrm{o}}$}}{\text{avg. no. of active devices in
$\varphi_{\mathrm{max}}$}}~{},$ (7)

which represents the cumulative distribution function (CDF) of the satellite
zenith angle. The average number of points can be found by invoking Campbell
theorem [15] for a PPP. Accordingly, the average number of points in a region
bounded by an angle $\left[0,\varphi\right]$ is given by

$K(\varphi)=\int_{0}^{\varphi}2\pi
R^{2}\lambda(\varphi)\sin(\varphi)~{}\mathrm{d}\varphi,$ (8)

because the spherical strip area is given by, $2\pi
R^{2}\sin(\varphi)\mathrm{d}\varphi$. Substituting in (8) and replacing
$\lambda(\varphi)=\lambda_{\mathrm{o}}D(\varphi)$ we obtain

$F_{\varphi}(\varphi_{\mathrm{o}})=\frac{K(\varphi_{\mathrm{o}})}{K(\varphi_{\mathrm{max}})}=\frac{\int_{\mathrm{0}}^{\varphi_{o}}\sin(\varphi_{\mathrm{o}})D(\varphi_{\mathrm{o}})~{}\text{d}\varphi}{\int_{0}^{\varphi_{\text{max}}}\sin(\varphi)D(\varphi)~{}\text{d}\varphi}.$
(9)

The probability density function (PDF) is thus obtained by differentiating (9)
as follows

$\displaystyle
f_{\varphi}(\varphi_{\mathrm{o}})=\frac{\sin(\varphi_{\mathrm{o}})D(\varphi_{\mathrm{o}})}{\int_{0}^{\varphi_{\text{max}}}\sin(\varphi)D(\varphi)~{}\text{d}\varphi}.$
(10)

An illustration of the CDF at $\theta_{\mathrm{min}}=10^{\mathrm{o}}$ is shown
in Fig. 4 along with Monte-carlo simulations to verify the analytical results.
Note that the distribution changes for different tuning factor $a$.

Figure 4: The CDF of the satellite zenith angle $\varphi$ in (9) within the
admittance region. Showing the results of various tuning factors $a=[0,1\times
10^{-5},1]$, at admittance $\theta_{\mathrm{min}}=10^{o}$.

##  III Link Performance

Majority of IoT technologies rely on simple random access techniques in order
to reduce the complexity of the IoT devices. Examples can be found in LoRaWAN
and in the initial random access stage of NB-IoT. Accordingly, collision of
frames is bound to happen due to the lack of coordination.

###  III-A Channel Model

For a given device with transmitting power $p_{\mathrm{t}}$, at an angle
$\varphi_{\mathrm{o}}$, the received signal power in the uplink can be modeled
as $S=p_{\mathrm{t}}l(\varphi_{\mathrm{o}})\zeta(\varphi_{\mathrm{o}})$[16],
where $l(\varphi_{\mathrm{o}})$ is the reciprocal of the free space path-loss
(FSPL) $\left(\frac{4\pi df}{c}\right)^{2}$, while
$\zeta(\varphi_{\mathrm{o}})$ is the excess gain caused by ground clutter, and
its distribution follows simple Gaussian mixture, this path-gain heavily
depends on the probability of LoS
$p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}$ as given in eq. (7) in
[17].

###  III-B Interference

For a given device located at $x_{\mathrm{o}}$, the simultaneous transmissions
imposed by other devices within the admittance region cause interference with
a total aggregated power, denoted $I$, obtained as follows

$\displaystyle
I=\sum_{x_{\mathrm{i}}\in\Phi\text{\textbackslash}x_{\mathrm{o}}}\kappa
p_{\mathrm{t}}\zeta(\varphi_{i})l(\varphi_{i}),$ (11)

where $\Phi$ is the set of all devices in the admittance region, $\kappa$ is a
system parameter that captures the impact of interference mitigation caused by
resource coordination in the access system. Note that $\kappa=1$ indicates the
worst case scenario when no coordination is implemented. Accordingly, the
spatial average of the interference is obtained by invoking Campbell’s theorem
of sums and is given as

$\displaystyle\mathbb{E}[I]=2\pi
R^{2}\lambda_{\mathrm{o}}\int_{o}^{\varphi_{\mathrm{max}}}\bar{\zeta}(\varphi)l(\varphi)\sin(\varphi)D(\varphi)\mathrm{d}\varphi$
(12)

where $\bar{\zeta}$ is the average excess path-gain expressed as [6]

$\displaystyle\bar{\zeta}=p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}\exp\left(-\mu_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}+\frac{\sigma^{2}_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}}{2}\right)+p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{NLoS}$}}}}\exp\left(-\mu_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{NLoS}$}}}}+\frac{\sigma^{2}_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{NLoS}$}}}}}{2}\right),$
(13)

where $\mu$ and $\sigma$ are the mean and standard deviation parameters
associated with respective LoS and non-LoS components. To validate the above
analytic results, we depict in Fig. 5 a comparison between the analytic
results of (12) and Monte Carlo simulations at various values of $a$ and
$\theta_{\mathrm{min}}$. Note the increase in the aggregated interference
power when deploying the repetition scheme. Nevertheless, it is shown in the
next subsection that the probability of success can be further increased with
careful tuning of $a$ and $\theta$.

Figure 5: The average interference power given in (12) versus the tuning
factor $a$ with different admittance angles $\theta_{\mathrm{min}}$.

###  III-C Probability of success

To measure the probability of success under influence of frame repetition, we
utilize a vital metric of communication link which is the ratio of received
signal power to the interference plus noise power (SINR). The probability of a
frame having an SINR greater than a given design threshold $\gamma$, is
equivalent to the frame success probability. For a given IoT device viewing
the satellite at a zenith angle $\varphi_{\mathrm{o}}$, the success
probability of a single transmission is formulated as follows

$\displaystyle p_{|\varphi_{\mathrm{o}}}(1)$
$\displaystyle=\mathbb{P}\left(SINR>\gamma\right)=\mathbb{P}\left(\frac{S}{\bar{I}+W}>\gamma\right)$
$\displaystyle=\mathbb{P}\left({\zeta}>\frac{\gamma(\bar{{I}}+{W})}{\mathrm{p}_{\mathrm{t}}l(\varphi)}\right)=1-F_{\mathrm{\zeta}}\left(\frac{\gamma(\bar{{I}}+{W})}{\mathrm{p}_{\mathrm{t}}l(\varphi)}\right),$
(14)

where ${W}$ is the average thermal noise power, $\bar{I}$ is given in (12) and
$F_{\mathrm{\zeta}}$ is given as [4]

$\displaystyle\scriptstyle
F_{\mathrm{\zeta}}=p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}F_{\mathrm{X}}\left(x,-\rho\mu_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}},\frac{\sigma_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}^{2}\rho^{2}}{2}\right)+p_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{NLoS}$}}}}F_{\mathrm{X}}\left(x,-\rho\mu_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{NLoS}$}}}},\frac{\sigma_{\vbox{\hbox{\scalebox{0.5}{$\mathrm{LoS}$}}}}^{2}\rho^{2}}{2}\right),$
(15)

where $F_{\mathrm{X}}$ is the CDF of the log-normal distribution as
${F_{\mathrm{X}}(x)=\frac{1}{2}+\frac{1}{2}\text{erf}\left(\frac{\ln
x-\mu}{\sigma\sqrt{2}}\right)}$ and $\rho=\frac{\ln 10}{10}$. For a device
attempting its transmission $N$ number of times, the probability of at least
one of the frames to be successfully received is formulated as

$\displaystyle
p_{|\varphi_{\mathrm{o}}}(N)=1-[1-p_{|\varphi_{\mathrm{o}}}(1)]^{N}.$ (16)

We illustrate the success probability in Fig. 6 with respect to a range of
elevation angles for three given examples of the admittance region. We note
that for smaller admittance region, i.e., larger values of
$\theta_{\mathrm{min}}$, the probability of success increases. For small
$\theta_{\mathrm{min}}$ the increase in admittance region causes devices to
incur more repetition, hence, increasing the interference level, which
overshadows the performance enhancement as shown for
$\theta_{\mathrm{min}}=5^{\circ}$. Subsequently, to find the average success
probability for a given admittance region bounded by $\varphi_{\mathrm{max}}$,
we decondition (16) over the distribution of $\varphi$ as follows

$\displaystyle
p(N)=\int_{0}^{\varphi_{\mathrm{max}}}p_{|\varphi_{\mathrm{o}}}(N)f_{\varphi}(\varphi_{\mathrm{o}})~{}\mathrm{d}\varphi.$
(17)

It is illustrated in Fig. 7 where the success probability of the repetition
scenario clearly outperforms the no repetition scenario. Note that this
performance is for a single satellite spot, i.e. single admittance region. The
following section extends results to a constellation of satellites.

###  III-D Satellite Constellation Performance

Assuming most of the satellites are positioned in a non-GEO (Geosynchronous
Equatorial Orbit), which are in a relative motion with respect to the ground
IoT devices. As such, when reducing the size of the admittance region, the
probability of having the coverage spot of the device is reduced. Accordingly
there is a trade-off between satellite availability and its success rate
within the spot. To capture this trade-off, we extend the performance by
including the satellite availability, which is the ratio of satellite
admittance area to the total surface area of Earth’s sphere. For a small
number of satellites, denoted $k$, and assuming non-overlapping spots, then
the probability that a device is within the coverage is given as

$\displaystyle p_{\mathrm{spot}}$ $\displaystyle=\frac{k~{}2\pi
R^{2}\left(1-\cos\varphi_{\mathrm{max}}\right)}{4\pi
R^{2}}=\frac{k\left(1-\cos\varphi_{\mathrm{max}}\right)}{2},$ (18)

where $0\leq p_{\mathrm{spot}}\leq 1$. Accordingly, the probability of a
successful frame delivery in a constellation of $k$ satellite with non-
overlapping spots is given by

$\displaystyle p_{\mathrm{s}}=p_{\mathrm{spot}}\times p(N).$ (19)

We depict in Fig. 8 the global average success probability evaluated for one
tuning factor and various minimum elevation angles. We can clearly note the
trade-off induced by adjusting the admittance region, i.e. by controlling
$\theta_{\mathrm{min}}$. Moreover, as the shape of the curve is impacted by
the tuning factor $a$, an optimal point for a given admittance angle exists.
In the following section we shed light on the joint optimization of these two
parameters.

##  IV Discussion on Results

In order to understate the joint impact of the admittance region size and the
tuning factor on the global coverage probability, we depict in Fig. 9 the
global success probability while varying factor $a$ and admittance angle
$\theta_{\mathrm{min}}$. The solid line represents the optimal combination
values for both parameters. Other factors that impact this optimal solution
are related to the initial density of the ground devices
$\lambda_{\mathrm{o}}$ and the propagation conditions manifested in channel
parameters since these parameters influence (19). As such, network designer
can optimize the admittance region and tuning factor based on the operating
parameters. To validate our analytical results we utilize Monte Carlo
simulations that consider non-overlapping circular orbit satellites. Other
simulation parameters include a satellite altitude of 550 km, noise average
power of -138 dBm [6], center frequency at 2000 MHz, device $p_{\mathrm{t}}$
at 23 dBm as utilized in many low power IoT devices, and a bit rate of 8 Kbps
resulting in $D_{\mathrm{o}}$ of $1\times 10^{-6}$.

Figure 6: Conditional success probability in (16) at $a=2\times 10^{-4}$,
$\gamma=-10$, and $\theta_{min}=\left[5^{\circ},10^{\circ},15^{\circ}\right]$
for repetition scenario, and at $\theta_{min}=\left[10^{\circ}\right]$ for no
repetition scenario. Figure 7: The average success probability in (17) within
a single satellite admittance region, at $a=5\times 10^{-5}$ and $\gamma=-10$
Figure 8: Global average success probability in (19) evaluated at $a=5\times
10^{-5}$ and $\gamma=-10$, $k=10$. Figure 9: Global average success
probability in (19) calculated for various $\theta_{\mathrm{min}}$, $a$ and at
$k=10$.

##  V Conclusion

This paper presented an analytic model that captures the performance of IoT-
over-satellite in the uplink for non-GEO orbits using NTN radio channel. The
model employs frame repetitions to enhance the success rate of transmissions.
A trade-off between the satellite coverage region, admittance region, and
satellite availability is formulated in order to evaluate the performance of a
constellation of satellites. Accordingly, we found an optimal combination of
parameters that maximizes the frame success rate. These results can facilitate
network design and optimization for IoT-over-satellite links.

## References

  * [1] Ericsson. IoT connections outlook. [Online]. Available: {https://www.ericsson.com/en/mobility-report/dataforecasts/iot-connections-outlook}
  * [2] B. Manzoor, A. Al-Hourani, K. G. Chavez, S. Kandeepan, and M. Ding, “2D Versus 3D Geometric Modelling for Massive Access Networks in 5G-IoT Applications,” in _2018 12th Intl. Conf. on Signal Processing and Comm. Systems (ICSPCS)_ , Dec 2018, pp. 1–6.
  * [3] 3GPP-Third Generation Partnership Project, “Study on New Radio (NR) to support non-terrestrial networks,” 3GPP, Technical Report (TR) TR 38.811, Sep. 2020.
  * [4] B. Al Homssi and A. Al-Hourani, “Modeling uplink coverage performance in hybrid satellite-terrestrial networks,” _IEEE Comm. Letters_ , 2021.
  * [5] Z. Lin, M. Lin, B. Champagne, W.-P. Zhu, and N. Al-Dhahir, “Secure and Energy Efficient Transmission for RSMA-Based Cognitive Satellite-Terrestrial Networks,” _IEEE Wireless Comm. Letters_ , vol. 10, no. 2, pp. 251–255, Sep. 2021.
  * [6] A. Al-Hourani, “An Analytic Approach for Modeling the Coverage Performance of Dense Satellite Networks,” _IEEE Wireless Communications Letters_ , vol. 10, no. 4, pp. 897–901, Jan. 2021.
  * [7] R. Harwahyu, R.-G. Cheng, W.-J. Tsai, J.-K. Hwang, and G. Bianchi, “Repetitions Versus Retransmissions: Tradeoff in Configuring NB-IoT Random Access Channels,” _IEEE Internet of Things Journal_ , vol. 6, no. 2, pp. 3796–3805, Jan. 2019.
  * [8] S. Narayanan, D. Tsolkas, N. Passas, and L. Merakos, “NB-IoT: A Candidate Technology for Massive IoT in the 5G Era,” in _2018 IEEE 23rd International Workshop on CAMAD_ , Sep. 2018, pp. 1–6.
  * [9] R. Barbau, V. Deslandes, G. Jakllari, J. Tronc, J.-F. Chouteau, and A.-L. Beylot, “NB-IoT over GEO Satellite: Performance Analysis,” in _2020 10th Adv. Sat. Multimedia Systems Conf. and the 16th Sig. Processing for Space Comms. Workshop (ASMS/SPSC)_ , Oct. 2020, pp. 1–8.
  * [10] M. Conti, A. Guidotti, C. Amatetti, and A. Vanelli-Coralli, “NB-IoT over Non-Terrestrial Networks: Link Budget Analysis,” in _GLOBECOM 2020 - 2020 IEEE Global Comm. Conf._ , Dec 2020, pp. 1–6.
  * [11] Z. Lin, M. Lin, T. de Cola, J.-B. Wang, W.-P. Zhu, and J. Cheng, “Supporting IoT With Rate-Splitting Multiple Access in Satellite and Aerial-Integrated Networks,” _IEEE Internet of Things Journal_ , vol. 8, no. 14, pp. 11 123–11 134, Jan 2021.
  * [12] A. Al-Hourani, S. Kandeepan, and A. Jamalipour, “Modeling air-to-ground path loss for low altitude platforms in urban environments,” in _2014 IEEE Global Comm. Conf._ , Feb. 2014, pp. 2898–2904.
  * [13] B. Manzoor, A. Al-Hourani, B. Al Homssi, K. Magowe, S. Kandeepan, and K. G. Chavez, “Evaluating Coverage Performance of NB-IoT in the ISM-band,” in _2020 27th Intl. Conf. Telecom._ , Oct. 2020, pp. 1–5.
  * [14] B. Manzoor, B. Al Homssi, A. Al-Hourani, and S. Kandeepan, “Optimal Repetition Rate for Maximal Coverage,” _IEEE Wireless Communications Letters_ , vol. 10, no. 4, pp. 800–804, Dec. 2021.
  * [15] M. Haenggi, _Stochastic Geometry for Wireless Networks_. Cambridge University Press, Nov 2012.
  * [16] A. Al-Hourani and I. Guvenc, “On Modeling Satellite-to-Ground Path-Loss in Urban Environments,” _IEEE Comm. Letters_ , vol. 25, no. 3, pp. 696–700, Nov. 2021.
  * [17] A. Al-Hourani, “Optimal satellite constellation altitude for maximal coverage,” _IEEE Wireless Comm. Letters_ , vol. 10, no. 7, pp. 1444–1448, March 2021.


# Resilience of Bayesian Layer-Wise Explanations under Adversarial Attacks

Ginevra Carbone  Dept. of Mathematics and Geosciences
University of Trieste
Trieste, Italy  Guido Sanguinetti  SISSA
Trieste, Italy  School of Informatics
University of Edinburgh
Edinburgh, United Kingdom  Luca Bortolussi  Dept. of Mathematics and
Geosciences
University of Trieste
Trieste, Italy  Modeling and Simulation Group
Saarland University
Saarland, Germany

###### Abstract

We consider the problem of the stability of saliency-based explanations of
Neural Network predictions under adversarial attacks in a classification task.
We empirically show that, for deterministic Neural Networks, saliency
interpretations are remarkably brittle even when the attacks fail, i.e. for
attacks that do not change the classification label. By leveraging recent
results, we provide a theoretical explanation of this result in terms of the
geometry of adversarial attacks. Based on these theoretical considerations, we
suggest and demonstrate empirically that saliency explanations provided by
Bayesian Neural Networks are considerably more stable under adversarial
perturbations. Our results not only confirm that Bayesian Neural Networks are
more robust to adversarial attacks, but also demonstrate that Bayesian methods
have the potential to provide more stable and interpretable assessments of
Neural Network predictions.

##  1 Introduction

Deep Neural Networks (DNNs) are the core engine of the modern AI revolution.
Their universal approximation capabilities, coupled with advances in hardware
and training algorithm, have resulted in remarkably strong predictive
performance on a variety of applications, from computer vision to natural
language to bioinformatics.

While the story of DNNs is indubitably one of success, it is tempered with a
number of potentially very serious drawbacks which are somehow the natural
flip side of dealing with extremely flexible and complex models. The first
such drawback is the black box nature of DNNs: their expressivity and training
on large data sets empirically results in very strong predictive power, but in
general it does not provide any intuition about the possible explanations
underlying the decisions. The opaqueness of DNNs decisions is a major hurdle
towards their usage in scientific applications, where frequently explanations
for decisions are even more important than accuracy. It is even more
problematic in many societal applications, where several countries mandate by
law the explainability of any algorithmic decision in a variety of contexts.

A second major drawback of DNN predictions is their vulnerability to
adversarial attacks: empirically, it has been observed in many applications
that well chosen infinitesimal changes in inputs can produce catastrophic
changes in output [Goodfellow et al., 2015], leading to paradoxical
classifications and a clear problem in any application to safety critical
systems. Such brittleness appears to be intimately related to the geometry of
the data itself [Carbone et al., 2020], and is therefore likely to be an
intrinsic feature of standard DNN predictions.

In this paper, we argue theoretically and empirically111Code is available at
https://github.com/ginevracoal/BayesianRelevance that these two problems are
interlinked, and that therefore solutions that ameliorate resilience against
adversarial attacks will also lead to more stable and reliable
interpretations. We work within the framework of (pixel-wise) saliency
explanations, which attempt to interpret post-hoc DNN decisions by
apportioning a relevance score to each input feature for each data point.
Specifically, we use the popular Layer-wise Relevance Propagation (LRP) [Bach
et al., 2015], a method to assess the contribution of each pixel to the final
classification score which backpropagates the prediction in the neural network
until it reaches the input, using a set of suitable propagation rules. LRP
saliency interpretations are well known to be unstable under perturbations of
the inputs [Ghorbani et al., 2019, Kindermans et al., 2019, Alvarez-Melis and
Jaakkola, 2018, Zhang et al., 2020]; recently, Bykov et al. [2020] suggested
that a Bayesian treatment might ameliorate these stability problems.

Here, we consider the stability of saliency interpretations under targeted
adversarial attacks that aim to change the classification under perturbations
of the input. We introduce a novel notion of LRP robustness under adversarial
attacks. To our surprise, we found that the LRP robustness of deterministic
DNN predictions was remarkably low even when the adversarial attack failed to
change the overall classification of the data point, i.e., LRP interpretations
appear to be less robust than actual classifications. We analyse theoretically
the geometry of LRP robustness in a suitably defined large-data limit [Carbone
et al., 2020, Rotskoff and Vanden-Eijnden, 2018, Du et al., 2019, Mei et al.,
2018], and argue that the observed lack of robustness might be imputable to
large gradients of the prediction function in directions orthogonal to the
data manifold. Leveraging recent results from Carbone et al. [2020], we argue
that a Bayesian training of the DNNs should lead to an improved LRP
robustness, as well as stronger robustness to adversarial attacks. We
empirically validate our hypothesis on the popular MNIST and Fashion MNIST
benchmarks.

The main contributions of this paper are:

  1. 1.

the definition of a novel metric of robustness of interpretations of DNN
results (Sec. 3.1);

  2. 2.

a theoretical analysis of the effects of adversarial attacks on prediction
interpretations, and of the improvements offered by a Bayesian treatment (Sec.
3.2);

  3. 3.

an empirical study showing that indeed Bayesian training and prediction leads
to more robust interpretations of classifications (Sec. 4).

##  2 Background

###  2.1 Layer-Wise Relevance Propagation

Let $f:\mathbb{R}^{d}\rightarrow[0,1]$ be a single-class image classifier,
where $f(x,w)$ is the probability that an image $x\in\mathbb{R}^{d}$ belongs
to the class of interest and $w$ is the set of learnable weights. Without loss
of generality, this concept can be extended to multi-class classifiers or to
non probabilistic outputs. The idea of pixel-wise decomposition of a given
image $x$ is to understand how its pixels contribute to the prediction
$f(x,w)$. In particular, LRP associates with each pixel $p$ a relevance score
$R(x_{p},w)$, which is positive when the pixel contributes positively to the
classification, negative when it contributes negatively to the classification
and zero when it has no impact on the classification. All the relevance scores
for a given image $x$ can be stored in a heatmap
$R(x,w)=\\{R(x_{p},w)\\}_{p}$, whose values quantitatively explain not only
whether pixels contribute to the decision, but also by which extent. By
leveraging suitable propagation rules, one can ensure that relevance heatmap
catch saliency features. For this purpose, the heatmap should be
_conservative_ , i.e. the sum of the assigned relevances should correspond to
the total relevance detected by the model

$f(x,w)=\sum_{p}R(x_{p},w).$

Several propagation rules satisfy the conservative property, each of them
leading to different relevance measures. A practical example of propagation
rule is the _Epsilon rule_ ($\epsilon$-LRP) [Montavon et al., 2019]. Let
$a_{j}$ be a neuron activation computed at the $j$-th layer during the forward
pass. Let $w_{jk}$ be the weight connecting $a_{j}$ to a subsequent neuron
$a_{k}$, $w_{0k}$ be the bias weight and $a_{0}=1$. At each step of
backpropagation LRP computes the relevance score $R_{j}$ of the $j$-th
activation $a_{j}$, by backpropagating the relevance scores of all neurons in
the subsequent layer. The resulting $\epsilon$-LRP score for a chosen
$\epsilon>0$ amounts to

$\displaystyle
R_{j}=\sum_{k}\frac{a_{j}w_{jk}}{\epsilon+\sum_{0,j}a_{j}w_{jk}}R_{k}.$ (1)

Bach et al. [2015] also presented LRP using a functional approach, i.e.
independently of the network’s topology. Then, Montavon et al. [2017] used
_deep Taylor decomposition_ to interpret rule-based approaches under the
functional setting. Their method builds on the standard first-order Taylor
expansion of a non-linear classifier at a chosen _root point_ $\tilde{x}$,
such that $f(\tilde{x})=0$,

$\displaystyle\begin{split}f(x)&=f(\tilde{x})+\nabla_{x}f(\tilde{x})\cdot(x-\tilde{x})+\gamma\\\
&=\sum_{p}\frac{\partial f}{\partial
x_{p}}\Big{|}_{x=\tilde{x}}\cdot(x_{p}-\tilde{x}_{p})+\gamma,\end{split}$ (2)

where $\gamma$ denotes higher-order terms. The root point $\tilde{x}$
represents a neutral image which is similar to $x$, but does not influence
classification, i.e. whose relevance is everywhere null. The nearest root
point to the original image $x$ can be obtained by solving an iterative
minimization problem [Montavon et al., 2017].

###  2.2 Adversarial Attacks

Figure 1:  LRP heatmaps of an image $x$ (second column) and an FGSM
adversarial perturbation $\tilde{x}$ (third column) which fails on a
deterministic network and a Bayesian network. The two models have the same
architecture (Tab. 2) and are both trained on the MNIST dataset. Explanations
are computed on the pre-softmax layer, i.e. layer idx $=5$ from Tab. 2.
Bayesian LRP is computed using $100$ posterior samples. For each heatmap we
only show the $100$ most relevant pixels. The LRP robustness amounts to $0.58$
in the deterministic case and to $0.93$ in the Bayesian case.

Adversarial attacks are small perturbations of input data that lead to large
changes in output (in the classification case, changes in predicted label)
Goodfellow et al. [2015]. Broadly speaking, most attack strategies utilise
information on the DNN loss function to detect an optimal perturbation
direction, either through explicit knowledge of the loss function (white box
attacks) or via querying the DNN (black box attacks, generally weaker than
their white box counterpart).

White box attacks generally utilize gradient information on the loss function
to determine the attack direction. One of the best known gradient-based
attacks is the Fast Gradient Sign Method (FGSM) [Goodfellow et al., 2015], a
one-step untargeted attack (i.e., an attack strategy which is independent of
the class of the attacked point). For a given network $f(\cdot,w)$ it crafts a
perturbation in the direction of the greatest loss w.r.t. the input

$\displaystyle\tilde{x}=x+\delta\;\text{sgn}\nabla_{x}L(x,w),$

where $L$ is the training loss and $\delta$ is the perturbation magnitude
(strength of the attack). An iterative, improved version of FGSM is Projected
Gradient Descent (PGD) Kurakin et al. [2016]. PGD starts from a random
perturbation in an $\epsilon\text{-L}_{\infty}$ ball around the input sample.
At each step, it performs FSGM with a smaller step size, $\alpha<\epsilon$,
and projects the adversary back in the $\epsilon\text{-L}_{\infty}$ ball

$x_{t+1}=\text{Proj}\\{x_{t}+\alpha\cdot\text{sgn}\nabla_{x}L(x_{t},w)\\}.$

The size of the resulting perturbation is smaller than $\epsilon$.

The evaluation of adversarial attacks can be either qualitative or
quantitative. In the qualitative case, one simply observes whether an attack
strategy with a certain strength is successful in switching the classification
label of the given data point; in general, the evaluation then reports the
fraction of successful attacks on the whole data set. A quantitative metric to
evaluate network performances against adversarial attacks is provided by the
notion of _softmax robustness_ [Carbone et al., 2020], which computes the
softmax difference between original and adversarial predictions

$\displaystyle 1-||f(x)-f(\tilde{x})||_{\infty}.$ (3)

and is a number between zero (maximal fragility) and one (complete robustness)
for every data point.

###  2.3 Bayesian Neural Networks

Bayesian models capture the inherent uncertainty intrinsic in any model by
replacing individual models with ensembles weighted by probability laws. In
the DNN setup, a Bayesian Neural Network (BNN) consists of an ensemble of DNNs
whose weights are drawn from the posterior measure $p(w|D)$, where $D$ denotes
the available training data. To compute the posterior measure, one needs to
first define a prior distribution $p(w)$ on the network’s weights $w$; by
Bayes’ theorem, the posterior is then obtained by combining the prior and the
likelihood $p(w|D)\propto p(D|w)p(w)$, where the likelihood term $p(D|w)$
quantifies the fit of the network with weights $w$ to the available training
data. Exact computation of the posterior distribution $p(w|D)$ is in general
infeasible, thus one needs to resort to approximate inference methods for
training Bayesian NNs, namely Hamiltonian Monte Carlo (HMC) [Neal et al.,
2011] and Variational Inference (VI) [Wainwright and Jordan, 2008].

Once the posterior distribution has been computed, BNNs produce predictions
through the posterior predictive distribution

$\displaystyle\begin{split}p(f(x)|D)&=\int dwp(f(x)|w)p(w|D)\\\
&\simeq\sum_{w_{j}\sim p(w|D)}p(f(x)|w_{j})\end{split}$ (4)

where $f(x)$ is the output value at a new point $x$, and the equality
represents the celebrated Bayesian model averaging procedure.

In the BNN setting, adversarial attacks are crafted against the posterior
predictive distribution $p(f(x)|D)$. For example, FGSM attack on an ensemble
of $N$ networks $f(\cdot,w_{i})$ with weights drawn from $p(w|D)$ becomes

$\displaystyle\tilde{x}$
$\displaystyle=x+\delta\;\text{sgn}\Big{(}\mathbb{E}_{p(w|D)}\big{[}\nabla_{x}L(x,w)\big{]}\Big{)}$
$\displaystyle\simeq
x+\delta\;\text{sgn}\sum_{i=1}^{N}\nabla_{x}L(x,w_{i})\qquad w_{i}\sim
p(w|D).$

In a similar way, it is possible to introduce a concept of Bayesian
explanations for BNN predictions. As is clear from the definition(s) of LRP in
Section 2.1, the relevance score assigned to an input feature depends on the
neural network weights. Therefore, in the Bayesian setting, the relevance
becomes a random variable; we define as a Bayesian explanation the expected
relevance of a feature under the posterior distribution of the weights.

Recent research shows that Bayesian neural networks are adversarially robust
to gradient-based attacks in the over-parametrised and infinite data limit
[Carbone et al., 2020, Rotskoff and Vanden-Eijnden, 2018]. It is therefore of
interest to investigate whether such robustness also extends to the learned
explanations. To do so, we compare the explanations of deterministic NNs to
that of Bayesian NNs against such attacks. Fig. 1 shows an example of failed
FGSM attack for a deterministic network and a Bayesian network with the same
architecture. Despite the fact that the attack did not manage to change the
overall classification, we can see immediately a large difference between the
deterministic LRP explanation of the original image, $R(x)$, and of the
adversarial image, $R(\tilde{x})$ (top row). On the other hand, in the
Bayesian case (bottom row), the saliency maps before and after the attack are
essentially identical. In the next sections, we will provide both a
theoretical explanation of this phenomenon, and systematically substantiate
empirically the robustness of Bayesian explanations to adversarial attacks.

##  3 Methodology

###  3.1 LRP robustness

Figure 2: $\text{Top}_{200}$ pixels in an image $x$ from Fashion MNIST dataset
and an FGSM adversarial perturbation $\tilde{x}$.

We define the _$k$ -LRP robustness_ of relevance heatmaps to adversarial
attacks and use this measure to assess how adversarial perturbations affect
the explanations.

######  Definition 3.1.

Let $x$ be an image with relevance heatmap $R(x,w)$ and $\tilde{x}$ an
adversarial perturbation with relevance heatmap $R(\tilde{x},w)$. Let
$\text{Top}_{k}(R)$ denote the set of $k$ most relevant pixel indexes in a
heatmap $R$, where $P\in\mathbb{N}$ is the total number of pixels in $x$ and
$k\leq P$. The _$k$ -LRP robustness_ of $x$ w.r.t. the attack $\tilde{x}$ is

$\displaystyle
k\text{-LRP}_{\rho}(x,\tilde{x},w):=\frac{|\text{Top}_{k}(R(x,w))\cap\text{Top}_{k}(R(\tilde{x},w))|}{k}.$
(5)

In other words, $k\text{-LRP}_{\rho}(x,\tilde{x},w)$ is the fraction of common
$k$ most relevant pixels for $x$ and $\tilde{x}$. Fig. 2 gives an intuition of
this computation. Notice that the LRP robustness of a point depends only
implicitly on the strength of the attack through the attacked point
$\tilde{x}$.

#### Inner layers explanations

Figure 3:  LRP heatmaps of an image $x$ and an FGSM adversarial perturbation
$\tilde{x}$. Explanations are computed w.r.t. the learnable layers of a
deterministic network trained on the MNIST dataset. For each layer, we only
show the $100$ most relevant pixels for both the original image and its
adversarial counterpart, i.e. the ones selected for the computation of
$100\text{-LRP}_{\rho}(x,\tilde{x},w,l)$ from Eq. (5), where $l\in[0,3,7]$.

We analyse the behaviour of LRP representations in the internal layers of the
network, thus we also extend the computation of LRP heatmaps to any feature
representation of the input $x$ at a learnable layer $l\in\mathbb{N}$. We
denote it by $R(x,w,l)$, where $l\leq L$ and $L$ is the maximum number of
layers available in the architecture. The corresponding LRP robustness will be
denoted by $k\text{-LRP}_{\rho}(x,\tilde{x},w,l)$. In such case, the
robustness does not refer anymore to explanations in the classification phase
(pre-softmax layer), but rather to the explanations in the learning phases,
hence it gives an idea of the most relevant pixels determining an internal
representation. Fig. 3 shows an example of internal LRP heatmaps on a
deterministic NN with learnable layers indexed by $l\in[0,3,7]$. For
illustrative purposes, heatmaps appearing on the same row are normalized in
$[-1,1]$ before selecting the $\text{Top}_{k}$ pixels, since numeric scales
are significantly different across the different internal representations.

#### Bayesian LRP robustness

The notion of LRP robustness can be naturally generalised to the Bayesian
setting using the concept of Bayesian model averaging introduced in Section
2.3. Hence, the LRP heatmap of a BNN is computed as the average of all the
deterministic heatmaps from the ensemble:

$\displaystyle\mathbb{E}_{p(w|D)}\big{[}k\text{-LRP}_{\rho}(x,\tilde{x},w,l)\big{]}.$

In this regard, we emphasise that Bayesian interpretations are affected by the
chosen number of posterior samples drawn from the learned distribution.

###  3.2 Geometric meaning of adversarial interpretations

To better conceptualise the impact of a Bayesian treatment on LRP robustness,
it is convenient to consider the thermodynamic limit of infinite data and
infinite expressivity of the network, as formalised in Du et al. [2019], Mei
et al. [2018], Rotskoff and Vanden-Eijnden [2018]. We refer the reader to
Carbone et al. [2020] for a thorough mathematical discussion of the
assumptions corresponding to this thermodynamic limit. For the purposes of our
discussion, the main ingredients are the data manifold $\mathcal{M}_{D}$, a
piecewise smooth submanifold of the input space where the data lie, and the
true input/output function $g(x)$, which is assumed to be smooth and hence
representable through an infinitely wide DNN. Practically, this limit might be
well approximated on large data sets where the networks achieve high accuracy.

In this limit, it was proved in Du et al. [2019], Mei et al. [2018], Rotskoff
and Vanden-Eijnden [2018] that the DNN $f(x,w)$ trained via SGD will converge
to the true underlying function $g(x)$ over the whole data manifold
$\mathcal{M}_{D}$. Because the data manifold is assumed to be piecewise
smooth, it is possible to define a tangent space to the data manifold almost
everywhere, and therefore to define two operators $\nabla_{x}^{\perp}$ and
$\nabla_{x}^{\parallel}$ which define the gradient along the normal and
tangent directions to the data manifold $\mathcal{M}_{D}$ at a point $x$ of a
function defined over the whole input space.

LRP and gradient-based adversarial strategies both share a reliance on
gradient information. In the adversarial attacks cases, one evaluates the
gradient of the loss function which, by the chain rule is given by

$\nabla_{x}L(f,g)=\frac{\delta L(f,g)}{\delta f}\frac{\partial f}{\partial x}$
(6)

In the thermodynamic limit, the DNN function $f(x,w)$ coincides with the true
function everywhere on the data manifold, and therefore the tangent gradient
of the loss function is identically zero. The normal gradient of the loss,
however, is unconstrained by the data, and, particularly in a high dimensional
setting, might achieve very high values along certain directions, creating
therefore weaknesses that may be exploited by an adversarial attacker. The
main result of Carbone et al. [2020] was to show that the orthogonal component
of the loss gradient has expectation zero under the posterior weight
distribution, therefore showing that BNNs are robust against adversarial
attacks.

In the LRP setup, we instead consider gradients of the prediction function, as
opposed to the loss, nevertheless the insight remains valid. The tangent
components of the gradient of the prediction function $f(x,w)$ will coincide
with the gradients of the true function $g(x)$, and therefore represent
directions of true sensitivity of the decision function which are correctly
recognised as relevant. However, such directions might be confounded or
dwarfed by normal gradient components, which create directions of apparent
relevance which, by construction, are targeted by gradient-based adversarial
attacks.

Because the normal gradient of the loss function (6) is proportional to the
normal gradient of the prediction function, the zero-averaging property under
the posterior will be inherited by the gradient of the prediction function, so
that

$E_{p(w|D)}[\nabla_{x}^{\perp}f(x,w)]=0.$

Therefore, BNNs in the thermodynamic limit will only retain relevant
directions along the data manifold, which correspond to genuine directions of
high relevance.

##  4 Experimental Results

In this section we corroborate the insights described in Section 3 with an
experimental evaluation, comparing empirically the LRP robustness using the
popular MNIST [LeCun and Cortes, 2010] and Fashion MNIST [Xiao et al., 2017]
benchmark data sets. Both data sets are composed of 60.000 images belonging to
ten classes: in the MNIST case, these are hand-written digits, while the
Fashion MNIST data set consists of stylized Zalando images of clothing items.
While MNIST is considered a relatively trivial data set, with accuracies over
99% being regularly reported, Fashion MNIST is considerably more complex, and
the best architectures report accuracies around 95%. We do not experiment on
more complex data sets such as CIFAR-10 [Krizhevsky et al., ] or ImageNet
[Deng et al., 2009], because of the very high computational costs of running
Bayesian inference on very deep networks trained on very large data sets 222We
do not experiment with scalable Monte Carlo dropout methods Gal and Ghahramani
[2016] here since the properties of the posterior approximation they provide
are not well understood..

On both data sets, we train deterministic DNNs and BNNs using both HMC and VI:
this allows us to contrast the effect of a locally Gaussian approximation to
the posterior against the asymptotically exact (but computationally more
expensive) approximation provided by HMC. Because we require high accuracy in
order to approximate the asymptotic conditions described in Section 3.2,
different architectures were used on the two data sets and between VI and HMC.
In all cases, however, the BNN is compared with a DNN with the same
architecture, to ensure fairness of the comparisons. Full details of the
architectures used are reported in Sec. 6.1 of the Appendix.

Adversarial attacks in our tests are Fast Gradient Sign Method (FGSM)
[Goodfellow et al., 2015] and Projected Gradient Descent (PGD) [Kurakin et
al., 2016], with a maximum perturbation size of $0.25$. We rely on _TorchLRP_
library 333https://github.com/fhvilshoj/TorchLRP for the computation of LRP
explanations and set $\epsilon=0.1$ in the Epsilon rule.

###  4.1 Bayesian interpretations are robust against attacks

Figure 4:  LRP robustness distributions computed on $500$ test points from
MNIST dataset. Bayesian networks are trained with VI and tested on an
increasing number of samples ($10,50,100$). Layer indexes refer to the
learnable layers in the architecture (Tab. 1 in the Appendix), which is shared
across models. Figure 5:  LRP robustness distributions computed on $500$ test
points from Fashion MNIST dataset. Bayesian networks are trained with VI and
tested on an increasing number of samples ($10,50,100$). Layer indexes refer
to the learnable layers in the architecture (Tab. 1 in the Appendix), which is
shared across models. Figure 6: LRP robustness distributions computed on $500$
test points from MNIST dataset. Bayesian networks are trained with HMC and
tested on an increasing number of samples ($10,50,100$). Layer indexes refer
to the learnable layers in the architecture (Tab. 2 in the Appendix), which is
shared across models. Figure 7: LRP robustness distributions computed on $500$
test points from Fashion MNIST dataset. Bayesian networks are trained with HMC
and tested on an increasing number of samples ($10,50,100$). Layer indexes
refer to the learnable layers in the architecture (Tab. 2 in the Appendix),
which is shared across models.

Our first significant result is that Bayesian explanations are considerably
more robust under attacks than deterministic architectures. For both data sets
and both VI and HMC training, LRP robustness scores are significantly higher
than their deterministic counterparts. These results are shown for the MNIST
data set in Fig. 6 (HMC training) and 4 (VI training), and in Fig. 7 (HMC) and
5 (VI) for Fashion MNIST. The last row of the figures represents the standard
LRP (computed from the pre-softmax layer), while the top row is the initial
feature representation (after the first non-linear layer), and the middle row
represents the LRP of an internal layer. We tested Bayesian representations
using an increasing number of posterior samples, i.e. $10,50,100$ and two
different choices of $k$, $k=10$ and $k=30$ (columns). We attacked $500$
randomly selected test images, whose choice is balanced w.r.t. the available
classes, and plot the distribution of the resulting LRP scores.

The first notable observation is that in both data sets and with both
inference method the Bayesian (pre-softmax) LRP has robustness values very
close to one. Interestingly, the behaviour of the LRP in the inner layers
appears to be rather different between VI and HMC: while HMC seems stable
across all layers, networks trained with VI appear to have a broadening of the
LRP distribution in the intermediate layers, suggesting that compensatory
changes happen between the early and late layers. Importantly, we see that
deterministic DNNs have very low LRP robustness, particularly in the $k=10$
case: this confirms empirically the conjecture of Section 3.2 that components
of the gradient that are normal to the data manifold (and are therefore the
ones likely to be changed in an attack) are often major contributors to the
relevance in DNN. On the contrary, the Bayesian averaging process greatly
reduces the expected relevance of such direction; importantly, we see that
this reduction is larger the greater the number of samples, as should be
expected.

Figure 8: LRP vs Softmax Robustness of deterministic and Bayesian NNs trained
on Fashion MNIST dataset. LRP Robustness is computed on the $100$ most
relevant pixels, i.e. using $\text{Top}_{100}$. Bayesian Networks are trained
with VI and tested on an increasing number of samples ($10,50,100$). Layer
indexes refer to the learnable layers in the architecture (Tab. 1 in the
Appendix), which is shared across all models.  Figure 9:  LRP vs Softmax
Robustness of deterministic and Bayesian NNs trained on MNIST dataset. LRP
Robustness is computed on the $100$ most relevant pixels, i.e. using
$\text{Top}_{100}$. Bayesian Networks are trained with HMC and tested on an
increasing number of samples ($10,50,100$). Layer indexes refer to the
learnable layers in the architecture (Tab. 2 in the Appendix), which is shared
across all models.

###  4.2 Bayesian LRP robustness is correlated with softmax robustness

A simple explanation for the improved LRP robustness of BNNs lies in the fact
that BNNs are provably immune to gradient-based attacks [Carbone et al.,
2020]. Therefore, one might argue that the stability of the LRP is a trivial
byproduct of the stability of the classifications.

To explore this question more in depth, we consider the relationship between
the LRP robustness of a test point (stability of the explanation) and its
softmax robustness (resilience of the classification against an attack). Fig.
8 and 9 present scatterplots of these two quantities for the models from the
previous experiments. An immediate observation is that deterministic
explanations are weak against adversarial perturbations even when their
softmax robustness (3) is close to $1$. Therefore, even in the cases where the
classification is unchanged, deterministic saliency heatmaps are fragile. In
fact, there are no significant changes in LRP robustness between data points
that are vulnerable to attacks and data points that are robust to attacks.
Bayesian models, instead, show a strong positive correlation between LRP and
softmax robustness, especially as the number of posterior samples increases.
While it is immediately evident that Bayesian predictions are robust to
adversarial attacks (since most data points have softmax robustness greater
than 0.5), it is also clear from this correlation that attacks which are more
successful (i.e. lower softmax robustness) also alter more substantially the
interpretation of the classification, and are therefore likely to represent
genuine directions of change of the true underlying decision function along
the data manifold.

##  5 Conclusions

Deep neural networks are fundamental to modern AI, yet many of the
mathematical structures underlying their success are still poorly understood.
Unfortunately, an unavoidable consequence of this situation is that we also
lack principled tools to address the weaknesses of deep learning systems.

In this paper, we harness the geometric perspective to adversarial attacks
introduced in Carbone et al. [2020] to study the resilience of Layer-wise
Relevance Propagation heatmaps to adversarial attacks. The geometric analysis
suggests a fundamental link between the fragility of DNNs against adversarial
attacks and the difficulties in understanding their predictions: because of
the unconstrained nature of classifiers defined on high dimensional input
spaces but trained on low dimensional data, gradients of both the loss
function and the prediction function tend to be dominated by directions which
are orthogonal to the data manifold. These directions both give rise to
adversarial attacks, and provide spurious explanations which are orthogonal to
the natural parametrisation of the data manifold. As shown in Carbone et al.
[2020], a Bayesian treatment should, in the limit of infinite data, remedy the
situation by averaging out irrelevant gradient directions in expectation. Our
empirical findings suggest that the predictions of the infinite data limit
seem to hold reasonably well on real data sets: not only BNN interpretations
are considerably more robust than deterministic DNN, but we also observe a
correlation between softmax (adversarial) robustness and LRP robustness which
suggests that indeed Bayesian interpretations capture relevant
parametrisations of the data manifold.

While promising, these results are to some extent preliminary: more thorough
testing against different adversarial strategies and different perturbations
is certainly warranted. Nevertheless, we believe that the insights provided by
a geometric interpretations will be helpful towards a better understanding of
both the strengths and the weaknesses of deep learning.

## References

  * Alvarez-Melis and Jaakkola [2018] David Alvarez-Melis and Tommi S Jaakkola.  Towards robust interpretability with self-explaining neural networks.  _arXiv preprint arXiv:1806.07538_ , 2018.
  * Bach et al. [2015] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  _PloS one_ , 10(7):e0130140, 2015.
  * Bykov et al. [2020] Kirill Bykov, Marina M-C Höhne, Klaus-Robert Müller, Shinichi Nakajima, and Marius Kloft.  How much can i trust you?–quantifying uncertainties in explaining neural networks.  _arXiv preprint arXiv:2006.09000_ , 2020.
  * Carbone et al. [2020] Ginevra Carbone, Matthew Wicker, Luca Laurenti, Andrea Patane', Luca Bortolussi, and Guido Sanguinetti.  Robustness of bayesian neural networks to gradient-based attacks.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, _Advances in Neural Information Processing Systems_ , volume 33, pages 15602–15613. Curran Associates, Inc., 2020.  URL https://proceedings.neurips.cc/paper/2020/file/b3f61131b6eceeb2b14835fa648a48ff-Paper.pdf.
  * Deng et al. [2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  ImageNet: A Large-Scale Hierarchical Image Database.  In _CVPR09_ , 2009.
  * Du et al. [2019] Simon Du, Jason Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai.  Gradient descent finds global minima of deep neural networks.  In _International Conference on Machine Learning_ , pages 1675–1685. PMLR, 2019.
  * Gal and Ghahramani [2016] Yarin Gal and Zoubin Ghahramani.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _international conference on machine learning_ , pages 1050–1059. PMLR, 2016.
  * Ghorbani et al. [2019] Amirata Ghorbani, Abubakar Abid, and James Zou.  Interpretation of neural networks is fragile.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 3681–3688, 2019.
  * Goodfellow et al. [2015] Ian Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  _International Conference on Learning Representations_ , 2015.  URL http://arxiv.org/abs/1412.6572.
  * Kindermans et al. [2019] Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T Schütt, Sven Dähne, Dumitru Erhan, and Been Kim.  The (un) reliability of saliency methods.  In _Explainable AI: Interpreting, Explaining and Visualizing Deep Learning_ , pages 267–280. Springer, 2019.
  * [11] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  Cifar-10 (canadian institute for advanced research).  URL http://www.cs.toronto.edu/~kriz/cifar.html.
  * Kurakin et al. [2016] Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio.  Adversarial examples in the physical world.  _CoRR_ , abs/1607.02533, 2016.  URL http://arxiv.org/abs/1607.02533.
  * LeCun and Cortes [2010] Yann LeCun and Corinna Cortes.  MNIST handwritten digit database.  2010\.  URL http://yann.lecun.com/exdb/mnist/.
  * Mei et al. [2018] Song Mei, Andrea Montanari, and Phan-Minh Nguyen.  A mean field view of the landscape of two-layer neural networks.  _Proceedings of the National Academy of Sciences_ , 115(33):E7665–E7671, 2018.
  * Montavon et al. [2017] Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, and Klaus-Robert Müller.  Explaining nonlinear classification decisions with deep taylor decomposition.  _Pattern Recognition_ , 65:211–222, 2017.
  * Montavon et al. [2019] Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller.  Layer-wise relevance propagation: an overview.  _Explainable AI: interpreting, explaining and visualizing deep learning_ , pages 193–209, 2019.
  * Neal et al. [2011] Radford M Neal et al.  Mcmc using hamiltonian dynamics.  _Handbook of markov chain monte carlo_ , 2(11):2, 2011.
  * Rotskoff and Vanden-Eijnden [2018] Grant M Rotskoff and Eric Vanden-Eijnden.  Neural networks as interacting particle systems: Asymptotic convexity of the loss landscape and universal scaling of the approximation error.  _stat_ , 1050:22, 2018.
  * Wainwright and Jordan [2008] Martin J Wainwright and Michael Irwin Jordan.  _Graphical models, exponential families, and variational inference_.  Now Publishers Inc, 2008.
  * Xiao et al. [2017] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  _arXiv preprint arXiv:1708.07747_ , 2017.
  * Zhang et al. [2020] Xinyang Zhang, Ningfei Wang, Hua Shen, Shouling Ji, Xiapu Luo, and Ting Wang.  Interpretable deep learning under fire.  In _29th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 20)_, 2020.

##  6 Appendix

###  6.1 Learnable layers

Learnable layers in conv. architecture
---
Idx | Layer | Parameters
$0$ | 2D Convolution |  |  in_channels = $784$
---
out_channels = $16$
kernel_size = $5$
$3$ | 2D Convolution |  |  in_channels = $16$
---
out_channels = hidden size
kernel_size = $5$
$7$ | Fully connected |  out_features = $10$
Table 1: Learnable layers and corresponding indexes in the convolutional
architecture. Hidden size equals $512$ for models trained on MNIST and $1024$
for models trained on Fashion MNIST.  Learnable layers in f.c. architecture
---
Idx | Layer | Parameters
$1$ | Fully connected |  |  in_features = $784$
---
out_features = hidden size
$3$ | Fully connected |  |  in_features = hidden size
---
out_features = hidden size
$5$ | Fully connected |  |  in_features = hidden size
---
out_features = $10$
Table 2: Learnable layers and corresponding indexes in the fully connected
architecture. Hidden size equals $512$ for models trained on MNIST and $1024$
for models trained on Fashion MNIST.