# Training on Foveated Images Improves Robustness to Adversarial Attacks

Muhammad A. Shah & Bhiksha Raj
Language Technologies Institute
Carnegie Mellon University
Pittsburgh, PA 15213
{mshah1<EMAIL_ADDRESS>

###### Abstract

Deep neural networks (DNNs) have been shown to be vulnerable to adversarial
attacks – subtle, perceptually indistinguishable perturbations of inputs that
change the response of the model. In the context of vision, we hypothesize
that an important contributor to the robustness of human visual perception is
constant exposure to low-fidelity visual stimuli in our peripheral vision. To
investigate this hypothesis, we develop R-Blur, an image transform that
simulates the loss in fidelity of peripheral vision by blurring the image and
reducing its color saturation based on the distance from a given fixation
point. We show that compared to DNNs trained on the original images, DNNs
trained on images transformed by R-Blur are substantially more robust to
adversarial attacks, as well as other, non-adversarial, corruptions, achieving
up to 25% higher accuracy on perturbed data.

##  1 Introduction

Deep Neural Networks (DNNs) are exceptionally adept at many computer vision
tasks and have emerged as one of the best models of the biological neurons
involved in visual object recognition (Yamins et al., 2014; Cadieu et al.,
2014). However, their lack of robustness to subtle image perturbations that
humans are largely invariant (Szegedy et al., 2014; Geirhos et al., 2018;
Dodge and Karam, 2017) to has raised questions about their reliability in
real-world scenarios. Of these perturbations, perhaps the most alarming are
adversarial attacks, which are specially crafted distortions that can change
the response of DNNs when added to their inputs (Szegedy et al., 2014; Ilyas
et al., 2019) but are either imperceptible to humans or perceptually
irrelevant enough to be ignored by them.

While several defenses have been proposed over the years to defend DNNs
against adversarial attacks, only a few of them have sought inspiration from
biological perception, which, perhaps axiomatically, is one of the most robust
perceptual systems in existence. Instead, most methods seek to teach DNNs to
be robust to adversarial attacks by exposing them to adversarially perturbed
images (Madry et al., 2018; Wong et al., 2019; Zhang et al., 2019) or random
noise (Cohen et al., 2019; Fischer et al., 2020; Carlini et al., 2022) during
training. While this approach is highly effective in making DNNs robust to the
types of perturbations used during training, the robustness often does not
generalize to other types of perturbations (Joos et al., 2022; Sharma and
Chen, 2017; Schott et al., 2018). In contrast, biologically-inspired defenses
seek to make DNNs robust by integrating into them biological mechanisms that
would bring their behavior more in line with human/animal vision (Paiton et
al., 2020; Bai et al., 2021; Dapello et al., 2020; Jonnalagadda et al., 2022;
Luo et al., 2015; Gant et al., 2021; Vuyyuru et al., 2020). As these defenses
do not require DNNs to be trained on any particular type of perturbation, they
yield models that, like humans, are robust to a variety of perturbations
(Dapello et al., 2020) in addition to adversarial attacks. For this reason,
and in light of the evidence indicating a positive correlation between
biological alignment and adversarial robustness (Dapello et al., 2020), we
believe biologically inspired defenses are more promising in the long run.

Following this line of inquiry, we investigate the contribution of low-
fidelity visual sensing that occurs in peripheral vision to the robustness of
human/animal vision. Unlike DNNs, which sense visual stimuli at maximum
fidelity at every point in their visual field, humans sense most of their
visual field in low fidelity, i.e without fine-grained contrast (Stewart et
al., 2020) and color information (Hansen et al., 2009). In adults with fully
developed vision, only a small region (less than 1% by area) of the visual
field around the point of fixation (Kolb, 2005) can be sensed with high
fidelity. In the remainder of the visual field (the periphery), the fidelity
of the sensed stimuli decreases exponentially with distance from the fixation
point (Dragoi and Tsuchitani, 2020). This phenomenon is called “foveation”.
Despite this limitation, humans can accurately categorize objects that appear
in the visual periphery into high-level classes (Ramezani et al., 2019).
Meanwhile, the presence of a small amount of noise or blurring can decimate
the accuracy of an otherwise accurate DNN. Therefore, we hypothesize that the
experience of viewing the world at multiple levels of fidelity, perhaps even
at the same instant, causes human vision to be invariant to low-level
features, such as textures, and high-frequency patterns, that can be exploited
by adversarial attacks.

In this paper, we propose R-Blur (short for Retina Blur), which simulates
foveation by blurring the image and reducing its color saturation adaptively
based on the distance from a given fixation point. This causes regions further
away from the fixation point to appear more blurry and less vividly colored
than those closer to it. Although adaptive blurring methods have been proposed
as computational approximations of foveation (Deza and Konkle, 2021; Pramod et
al., 2018; Wang and Cottrell, 2017), their impact on robustness has not been
evaluated to the best of our knowledge. Furthermore, color sensitivity is
known to decrease in the periphery of the visual field (Hansen et al., 2009;
Johnson, 1986), yet most of the existing techniques do not account for this
phenomenon.

Similar to how the retina preprocesses the visual stimuli before it reaches
the visual cortex, we use R-Blur to preprocess the input before it reaches the
DNN. To measure the impact of R-Blur, we evaluate the object recognition
capability of ResNets (He et al., 2016) trained with and without R-Blur on
three image datasets: CIFAR-10 (Krizhevsky et al., ), Ecoset (Mehrer et al.,
2021) and Imagenet (Russakovsky et al., 2015), under different levels of
adversarial attacks and common image corruptions (Hendrycks and Dietterich,
2019). We find that R-Blur models retain most of the high classification
accuracy of the base ResNet while being more robust. Compared to the base
ResNet, R-Blur models achieve 12-25 percentage points (pp) higher accuracy on
perturbed images. Furthermore, the robustness achieved by R-Blur is
certifiable using the approach from (Cohen et al., 2019). We also compare
R-Blur with two biologically inspired preprocessing defenses, namely VOneBlock
(Dapello et al., 2020), a fixed parameter module that simulates the primate
V1, and a non-uniform sampling-based foveation technique (Vuyyuru et al.,
2020), which we refer to as R-Warp. We observe that R-Blur induces a higher
level of robustness, achieving accuracy up to 33 pp higher than R-Warp and up
to 15 pp higher than VOneBlock against adversarial attacks. Compared to
adversarial training (AT) (Madry et al., 2018; Wong et al., 2019) – the state-
of-the-art non-biological defense, R-Blur achieves up to 7 pp higher accuracy
on average against non-adversarial corruptions of various types and strengths
thus indicating that the robustness of R-Blur generalizes better to non-
adversarial perturbations than AT. Finally, an ablation study showed that both
adaptive blurring and desaturation contribute to the improved robustness of
R-Blur.

##  2 Retinal Blur: An Approximation for Peripheral Vision

Figure 1:  R-Blur adds Gaussian noise to image (a) with the fixation point
(red dot) to obtain (b). It then creates a colored and a grayscaled copy of
the image and applies adaptive Gaussian blurring to them to obtain the low-
fidelity images (c) and (d), where the numbers indicate the standard deviation
of the Gaussian kernel applied in the region bounded by the boxes. The blurred
color and gray images are combined in a pixel-wise weighted combination to
obtain the final image (e), where the weights of the colored and gray pixels
are a function of their respective estimated acuity values (see 2.2).

To simulate the loss in contrast and color sensitivity of human perception
with increasing eccentricity, we propose R-Blur, an adaptive Gaussian
blurring, and color desaturation technique. The operations performed by
R-Blur, given an image and fixation point, are shown in Figure 1. First,
R-Blur adds Gaussian noise to the image to simulate stochastic firing rates of
biological photoreceptors. It then creates color and greyscale copies of the
image and estimates the acuity of color and grayscale vision at each pixel
location, using distributions that approximate the relationship between
distance from the fixation point (eccentricity) and visual acuity levels in
humans. R-Blur then applies adaptive Gaussian blurring to both image copies
such that the standard deviation of the Gaussian kernel at each pixel in the
color and the grayscale image is a function of the estimated color and
grayscale acuity at that pixel. Finally, R-Blur combines the two blurred
images in a pixel-wise weighted combination in which the weights of the
colored and gray pixels are a function of their respective estimated acuity
values. Below we describe some of the more involved operations in detail.

###  2.1 Eccentricity Computation

The distance of a pixel location from the fixation point, i.e. its
eccentricity, determines the standard deviation of the Gaussian kernel applied
to it and the combination weight of the color and gray images at this
location. While eccentricity is typically measured radially, in this paper we
use a different distance metric that produces un-rotated square level sets.
This property allows us to efficiently extract regions having the same
eccentricity by simply slicing the image tensor. Concretely, we compute the
eccentricity of the pixel at location $(x_{p},y_{p})$ as

$e_{x_{p},y_{p}}=\frac{\max(|x_{p}-x_{f}|,|y_{p}-y_{f}|)}{W_{V}},$ (1)

where $(x_{f},y_{f})$ and $W_{V}$ represent the fixation point and the width
of the visual field, i.e. the rectangular region over which R-Blur operates
and defines the maximum image size that is expected by R-Blur. We normalize by
$W_{V}$ to make the $e_{x_{p},y_{p}}$ invariant to the size of the visual
field.

###  2.2 Visual Acuity Estimation

We compute the visual acuity at each pixel location based on its eccentricity.
The biological retina contains two types of photoreceptors. The first type,
called cones, are color sensitive and give rise to high-fidelity visual
perception at the fovea, while the second type, called rods, are sensitive to
only illumination but not color and give rise to low-fidelity vision in the
periphery. The acuity of color and grayscale vision, arising from the cones
and rods, is estimated at each pixel location, $(x,y)$, using two sampling
distributions $D_{R}(e_{x,y})$ and $D_{C}(e_{x,y})$, respectively. In this
work, we use:

$\displaystyle\mathcal{D}(e;\sigma,\alpha)$
$\displaystyle=\max\left[\lambda(e;0,\sigma),\gamma(e;0,\alpha\sigma)\right]$
(2) $\displaystyle D_{C}(e;\sigma_{C},\alpha)$
$\displaystyle=\mathcal{D}(e;\sigma_{C},\alpha)$ (3) $\displaystyle
D_{R}(e;\sigma_{R},\alpha,p_{max})$
$\displaystyle=p_{max}(1-\mathcal{D}(e;\sigma_{R},\alpha)),$ (4)

where $\lambda(.;\mu,\sigma)$ and $\gamma(.;\mu,\sigma)$ are the PDFs of the
Laplace and Cauchy distribution with location and scale parameters $\mu$ and
$\sigma$, and $\alpha$ is a parameter used to control the width of the
distribution. We set $\sigma_{C}=0.12,\sigma_{R}=0.09,\alpha=2.5$ and
$p_{max}=0.12$. We choose the above equations and their parameters to
approximate the curves of photopic and scotopic visual acuity from (Dragoi and
Tsuchitani, 2020). The resulting acuity estimates are shown in Figure 2(b).
Unfortunately, the measured photopic and scotopic acuity curves from (Dragoi
and Tsuchitani, 2020) cannot be reproduced here due to copyright reasons,
however, they can be viewed at
https://nba.uth.tmc.edu/neuroscience/m/s2/chapter14.html (see Figure 14.3).

###  2.3 Quantizing the Visual Acuity Estimate

In the form stated above, we would need to create and apply as many Gaussian
kernels as the distance between the fixation point and the farthest vertex of
the visual field. This number can be quite large as the size of the image
increases and will drastically increase the per-image computation time. To
mitigate this issue we quantize the estimated acuity values. As a result, the
locations to which the same kernel is applied no longer constitute a single
pixel perimeter but become a much wider region (see Figure 1 (c) and (d)),
which allows us to apply the Gaussian kernel in these regions very efficiently
using optimized implementations of the convolution operator.

To create a quantized eccentricity-acuity mapping, we do the following. We
first list all the color and gray acuity values possible in the visual field
by assuming a fixation point at $(0,0)$, computing eccentricity values
$e_{0,y}$ for $y\in[0,W_{V}]$ and the corresponding values of
$\mathcal{D}_{R}=\\{D_{R}(e_{0,y})|y\in[0,W_{V}]\\}$ and
$\mathcal{D}_{C}=\\{D_{C}(e_{0,y})|y\in[0,W_{V}]\\}$. We then compute and
store the histograms, $H_{R}$ and $H_{C}$, from $\mathcal{D}_{R}$ and
$\mathcal{D}_{C}$, respectively. To further reduce the number of kernels we
need to apply and increase the size of the region each of them is applied to,
we merge the bins containing less than $\tau$ elements in each histogram with
the adjacent bin to their left. After that, given an image to process, we will
compute the color and gray visual acuity for each pixel, determine in which
bin it falls in $H_{R}$ and $H_{C}$, and assign it the average value of that
bin.

(a) unquantized

(b) quantized

Figure 2: Estimated visual acuity of sharp and colorful, photopic, and gray
and blurry, scotopic, vision.

Figure 3: Illustration of increasing the viewing distance (left to right) for
a given image is that more of the image is brought into focus. In this paper,
mostly $vd=3$ is used in inference.

###  2.4 Changing the Viewing Distance

Increasing the viewing distance can be beneficial as it allows the viewer to
gather a more global view of the visual scene and facilitates object
recognition. To increase the viewing distance we drop the $k$ lowest acuity
bins and shift the pixels assigned to them $k$ bins ahead such that the pixels
that were in bins 1 through $k-1$ are now assigned to bin 1. Figure 3 shows
the change in the viewing distance as the value of $k$ increases from 0 to 5.
Formally, given the quantized $D_{C}(e_{x,y})$ and $D_{R}(e_{x,y})$, let
$D=[d_{1},...,d_{n}]$ represent the value assigned to each bin and $P_{i}$ be
the pixel locations assigned to the $i^{th}$ bin, with $P_{1}$ and $P_{n}$
corresponding to points with the lowest and highest eccentricity,
respectively. To increase the viewing distance, we merge bins 1 through $k$
such that $D^{\prime}=[d_{1},...,d_{n-k}]$ and the corresponding pixels are
$P^{\prime}_{1}=[P_{1},...,P_{k}]$ and $P_{i>1}=P_{k+1}$.

###  2.5 Blurring and Color Desaturation

We map the estimated visual acuity at each pixel location, $(x_{p},y_{p})$, to
the standard deviation of the Gaussian kernel that will be applied at that
location as $\sigma_{(x_{p},y_{p})}=\beta W_{V}(1-D(e_{x,y}))$, where $\beta$
is constant to control the standard deviation and is set to $\beta=0.05$ in
this paper, and $D=D_{C}$ for pixels in the colored image and $D=D_{R}$ for
pixels in the grayscaled image. We then apply Gaussian kernels of the
corresponding standard deviation to each pixel in the colored and grayscale
image to obtain an adaptively blurred copy of each, which we combine in a
pixel-wise weighted combination to obtain the final image. The weight of each
colored and gray pixel is given by the normalized color and gray acuity,
respectively, at that pixel. Formally, the pixel at $(x_{p},y_{p})$ in the
final image has value

$v^{f}_{(x_{p},y_{p})}=\frac{v^{c}_{(x_{p},y_{p})}D_{C}(e_{x,y};\sigma_{C},\alpha)+v^{g}_{(x_{p},y_{p})}D_{R}(e_{x,y};\sigma_{C},\alpha)}{D_{C}(e_{x,y};\sigma_{C},\alpha)+D_{R}(e_{x,y};\sigma_{C},\alpha)},$

$v^{c}_{(x_{p},y_{p})}$ and $v^{g}_{(x_{p},y_{p})}$ are the pixel value at
$(x_{p},y_{p})$ in the blurred color and gray images respectively.

##  3 Evaluation

In this section, we determine the accuracy and robustness of R-Blur by
evaluating it on clean data and data that has been perturbed by either
adversarial attacks or common – non-adversarial – corruptions. We compare the
performance of R-Blur with an unmodified ResNet, two existing biologically-
inspired defenses, R-Warp (Vuyyuru et al., 2020) and VOneBlock (Dapello et
al., 2020), and two non-biological adversarial defenses: Adversarial Training
(AT) (Madry et al., 2018) and Randomized Smoothing (RS) (Cohen et al., 2019).
We show R-Blur to be significantly more robust to adversarial attacks and
common corruptions than the unmodified ResNet and prior biologically inspired
methods. Moreover, the certified accuracy of R-Blur is close to that of RS,
and even better in certain cases. While AT is more robust than R-Blur against
adversarial attacks, R-Blur is more robust than AT against high-intensity
common corruptions, thus indicating that the robustness of R-Blur generalizes
better to different types of perturbation than AT. We also analyze the
contribution of the various components of R-Blur in improving robustness.

###  3.1 Experimental Setup

Datasets: We use natural image datasets, namely CIFAR-10 (Krizhevsky et al.,
), Imagenet ILSVRC 2012 (Russakovsky et al., 2015), Ecoset (Mehrer et al.,
2021) and a 10-class subset of Ecoset (Ecoset-10). Ecoset contains around 1.4M
images, mostly obtained from ImageNet (the database (Deng et al., 2009) not
the ILSVRC dataset), that are organized into 565 basic object classes. The
classes in Ecoset correspond to commonly used nouns that refer to concrete
objects. To create Ecoset-10, we order the classes in the Ecoset dataset by
the number of images that each class is assigned to and then select the top 10
classes respectively. The training/validation/test splits of Ecoset-10 and
Ecoset are 48K/859/1K, and 1.4M/28K/28K respectively. For most experiments
with Ecoset and Imagenet, we use 1130, and 2000 test images, with an equal
number of images per class. During training, we use random horizontal flipping
and padding + random cropping, as well as AutoAugment (Cubuk et al., 2018) for
CIFAR-10 and RandAugment for Ecoset and Imagenet. All Ecoset and Imagenet
images were resized and cropped to $224\times 224$.

Model Architectures: For CIFAR-10 we use a Wide-Resnet (Zagoruyko and
Komodakis, 2016) model with 22 convolutional layers and a widening factor of
4, and for Ecoset and Imagenet we use XResNet-18 from fastai (Howard and
Gugger, 2020) with a widening factor of 2. Results for additional
architectures are presented in Appendix C.

Baselines and Existing Methods: We compare the performance of R-Blur to two
baselines: (1) an unmodified ResNet trained on clean data (ResNet), and (2) a
ResNet which applies five affine transformations 111We apply rotation,
translation, and shearing, with their parameters sampled from
$[-8.6^{\circ},8.6^{\circ}]$, $[-49,49]$ and $[-8.6^{\circ},8.6^{\circ}]$
respectively. The ranges are chosen to match the ranges used in RandAugment.
The random seed is fixed during evaluation to prevent interference with
adversarial attack generation. to the input image and averages the logits
(RandAffine). We also compare R-Blur with two biologically inspired defenses:
VOneBlock pre-processing proposed in (Dapello et al., 2020), which simulates
the receptive fields and activations of the primate V1 222As in (Dapello et
al., 2020), we remove the first conv, batch norm, ReLU, and MaxPool from the
ResNet with VOneBlock. , and R-Warp preprocessing proposed in (Vuyyuru et al.,
2020), which simulates foveation by resampling input images such that the
sampling density of pixels is maximal at the point of fixation and decays
progressively in regions further away from it. Finally, we compare R-Blur with
two non-biological adversarial defenses: fast adversarial training (Wong et
al., 2019) with $\|\delta\|_{\infty}=0.008$ (AT), and Randomized Smoothing
(RS) (Cohen et al., 2019).

(a) R-Blur

(b) R-Warp

Figure 4: Illustration of fixation selection. The initial fixation point is
set to top-left (0,0) and the image at $t_{0}$ is processed with R-Blur
/R-Warp to get the image at $t_{1}$. DeepGaze-III is used to generate a
fixation heatmap from this image. The next fixation point is sampled from the
heat map, and R-Blur /R-Warp is applied to get the image at $t_{2}$. The
region in the heatmap around the chosen fixation point is masked with an
inverted Gaussian kernel to prevent spatial clustering of fixation points.
This process is repeated to get a sequence of fixation points.

Fixation Selection for R-Blur and R-Warp: While training models with R-Blur
and R-Warp, we split each batch into sub-batches of 32 images, and for each
sub-batch, we randomly sample a single fixation point that we use to apply
R-Blur or R-Warp to all the images in that sub-batch. While training the
R-Blur model, we also set the viewing distance uniformly at random using the
procedure described in 2.4. During inference, we determine a sequence of five
fixation points (a scanpath) using DeepGaze-III (Kümmerer et al., 2022). Given
an image, DeepGaze-III passes it through a pretrained CNN backbone
(DenseNet-201 in (Kümmerer et al., 2022)) and extracts the activations from
several intermediate layers of the CNN. It then applies a sequence of
pointwise convolution and normalization layers to the activations to obtain a
heatmap indicating where a human is likely to fixate. We found that it was
more efficient to not use the scanpath prediction module in DeepGaze-III, and
instead obtain scanpaths by keeping track of the past fixation points, and
masking the predicted heatmap at these locations prior to sampling the next
fixation point from it. This process is illustrated in Figure 4. We trained
two instances of DeepGaze-III using the ResNets we trained with R-Blur and
R-Warp as the CNN backbone. We use the corresponding DeepGaze-III models to
predict the scanpaths for R-Blur and R-Warp models.

###  3.2 Results

R-Blur Improves Robustness to White-Box Attacks: We evaluate robustness by
measuring the accuracy of models under Auto-PGD (APGD)(Croce and Hein, 2020)
attack, which is a state-of-the-art white-box adversarial attack. We run APGD
for 25 steps on each image. We find that increasing the number of steps beyond
25 only minimally reduces accuracy (see Appendix A). We take a number of
measures to ensure that we avoid the pitfalls of gradient obfuscation (Athalye
et al., 2018a; Carlini et al., 2019) so that our results reflect the true
robustness of R-Blur. These steps and detailed settings used for adversarial
attacks are mentioned in Appendix A.

To determine if R-Blur improves robustness, we compare R-Blur with the
unmodified ResNet and RandAffine under the APGD attack. We observe that R-Blur
is significantly more robust than the unmodified ResNet and RandAffine models,
consistently achieving higher accuracy than the two on all datasets and
against all perturbation types and sizes, while largely retaining accuracy on
clean data (Figure 5). While RandAffine does induce some level of robustness,
it significantly underperforms R-Blur. On smaller datasets, R-Blur suffers
relatively little loss in accuracy at small to moderate levels
($\|\delta\|_{\infty}\leq 0.004$, $\|\delta\|_{2}\leq 1$) of adversarial
perturbations, while the accuracy of baseline methods quickly deteriorates to
chance or worse. On larger datasets – Ecoset and Imagenet, even the smallest
amount of adversarial perturbation ($\|\delta\|_{\infty}=0.002$,
$\|\delta\|_{2}=0.5$) is enough to drive the accuracy of the baselines to
$\sim$10%, while R-Blur still is able to achieve 35-44% accuracy. As the
perturbation is increased to $\|\delta\|_{\infty}=0.004$ and
$\|\delta\|_{2}=1.0$, the accuracy of the baselines goes to 0%, while R-Blur
achieves 18-22%. We do observe that the accuracy of R-Blur on clean data from
Ecoset and Imagenet is noticeably lower than that of the baseline methods.

We also compare R-Blur to two existing biologically motivated adversarial
defenses: VOneBlock and R-Warp, and find that R-Blur achieves higher accuracy
than both of them at all perturbation sizes and types. From Figure 6 we see
that R-Blur achieves up to 33pp higher accuracy than R-Warp, and up to 15 pp
higher accuracy than VOneBlock on adversarially perturbed data.

(a) CIFAR-10

(b) Ecoset-10

(c) Ecoset

(d) Imagenet

Figure 5:  Comparison of accuracy on various datasets (a-d) under adversarial
attacks of several $\ell_{2}$ (top) and $\ell_{\infty}$ (bottom) norms between
R-Blur (green) and two baseline methods: RandAffine (orange) and ResNet
(blue). The dashed lines indicate accuracy on clean images. R-Blur models
consistently achieve higher accuracy than baseline methods on all datasets,
and adversarial perturbation sizes.

(a) Ecoset

(b) Imagenet

Figure 6:  The difference in accuracy under adversarial attacks of several
$\ell_{2}$ and $\ell_{\infty}$ norms between R-Blur and two biologically
inspired defenses: R-Warp (blue) and VOneBlock (orange). R-Blur consistently
achieves higher accuracy on all adversarial perturbation sizes than R-Warp and
VOneBlock.

R-Blur is Certifiably Robust: To verify that the gains in robustness observed
above are indeed reliable, we use the certification method (Certify) from
(Cohen et al., 2019) to provide formal robustness guarantees for R-Blur. This
entails obtaining predictions for an input under a very large number
($10^{5}$) of noise samples drawn from $\mathcal{N}(0,\sigma_{c})$, and using
a hypothesis test to determine the certified radius around the input in which
the model’s prediction is stable with high probability ($\geq 99.9\%$). Given
a dataset, we can compute the certified accuracy at a radius $r$ as the
proportion of data points for which the certified radius is $\geq r$ and the
model’s prediction is correct. We compute the certified accuracy of R-Blur on
200 images sampled from Imagenet and Ecoset, and compare it with a model
trained on data perturbed with Gaussian noise, and is known to achieve high
certified accuracy (Cohen et al., 2019). We call this model G-Noise.

We expose both R-Blur and G-Noise to Gaussian noise of scale
$\sigma_{t}=0.125$ during training and compute their certified accuracy at
radii $r\in\\{0.5,1.0\\}$. According to (Cohen et al., 2019), if the scale of
the noise used in Certify is $\sigma_{c}$, then the maximum radius for which
certified accuracy can be computed (with $10^{5}$ noise samples) is
$r=4\sigma_{c}$. Therefore, when computing certified accuracy at $r\leq 0.5$
Certify adds noise of the same scale as was used during training
($\sigma_{c}=0.125=\sigma_{t}$), thus we call this the matched setting.
However, to compute certified accuracy at $r\leq 1.0$ Certify adds noise of a
larger scale than was used during training ($\sigma_{c}=0.25>\sigma_{t}$), and
thus in order to achieve high certified accuracy at $r\leq 1.0$ the model must
be able to generalize to a change in noise distribution. We call this the
unmatched setting.

Figure 7(a) and 7(b) show the certified accuracy of R-Blur and the G-Noise on
Ecoset and Imagenet at several $\ell_{2}$ norm radii under matched and
unmatched settings. In both settings, we see that R-Blur achieves a high
certified accuracy on both Ecoset and Imagenet, with the certified accuracy at
$r\approx 0.5$ and $r\approx 1.0$ being close to the ones observed in Figure
5, indicating that our earlier results are a faithful representation of
R-Blur’s robustness. Furthermore, we see that even if R-Blur was trained
without any noise, it can still achieve more than 50% of the certified
accuracy achieved by R-Blur trained with noise. This indicates that adaptive
blurring and desaturation do in fact endow the model with a significant level
of robustness. Finally, we note that while G-Noise has (slightly) higher
certified accuracy than R-Blur in the matched setting, R-Blur achieves
significantly higher certified accuracy in the unmatched setting, outstripping
G-Noise by more than 10 pp at $r\approx 1.0$ on Imagenet. This shows that the
robustness of R-Blur generalizes beyond the training conditions, while G-Noise
overfits to them. This makes R-Blur particularly suited for settings in which
the exact adversarial attack budget is not known, and the model must be able
to generalize.

(a) Matched Setting $(\sigma_{t}=\sigma_{c}=0.125$)

(b) Unmatched Setting $(\sigma_{t}<\sigma_{c}=0.25$)

Figure 7: The certified accuracy at various $\ell_{2}$-norm radii of R-Blur
and G-Noise models. R-Blur-CFI uses 1 fixation at the center of the image, and
R-Blur-5FI, averages logits from 5 fixation (corners + center). $\sigma_{t}$
denotes the scale of noise added during training and is 0.125 unless
specified, whereas $\sigma_{c}$ is the scale of the noise used to compute the
certified accuracy. G-Noise outperforms R-Blur in the matched scenario, while
R-Blur is superior in the unmatched scenario indicating that the robustness of
R-Blur is more generalizable.

R-Blur Improves Accuracy on Common (Non-Adversarial) Corruptions: Adversarial
perturbations constitute only a small subset of perturbations that human
vision is invariant to, therefore we evaluate the R-Blur on some other common
types of image corruptions that humans are largely invariant to but DNNs are
not. At this point, we also include the adversarially trained ResNet (AT) in
the evaluation to measure how well the robustness endowed by adversarial
training generalizes to other, non-adversarial, corruptions, and compare it
with biologically motivated methods. We sampled 2 images/class from Imagenet
and 5 images/class from Ecoset. Then we applied the 17 of the 19 333We exclude
Gaussian blur and Gaussian noise since they are similar to the transformations
done by R-Blur. common (non-adversarial) corruptions proposed in (Hendrycks
and Dietterich, 2019) at 5 different severity levels to generate 85 corrupted
versions of each image leading to corrupted versions of Imagenet and Ecoset
containing 170K and 240K images, respectively.

Table 1 shows the accuracy achieved by the models on Ecoset and Imagenet under
adversarial and non-adversarial perturbations. As expected, AT achieved the
highest accuracy on whitebox attacks by virtue of being trained on data
perturbed with attacks very similar to APGD. After AT, the most adversarially
robust model is R-Blur, followed by VOneBlock. We see that R-Blur is the most
accurate model on non-adversarial common corruptions, followed by VOneBlock,
which supports our hypothesis that the robustness of biologically motivated
methods in general, and R-Blur in particular, is highly generalizable. In
contrast, the accuracy of AT on common corruptions is almost the same as that
of the unmodified ResNet, indicating that the robustness of AT does not
generalize well.

|  Overall | Perturbation |  |  Overall | Perturbation |
---|---|---|---|---|---|---
Method |  Mean |  $\text{Mean}_{\text{L}}$ |  $\text{Mean}_{\text{H}}$ |  CC |  Wb |  Clean |  Mean |  $\text{Mean}_{\text{L}}$ |  $\text{Mean}_{\text{H}}$ |  CC |  Wb | Clean
Ecoset | Imagenet
ResNet |  37.1 |  25.5 |  12.7 |  39.4 |  0.8 |  71.2 |  34.7 |  21.7 |  9.7 |  33.6 |  0.1 | 70.3
RandAffine |  35.7 |  27.6 |  11.1 |  35.8 |  3.6 |  67.6 |  33.7 |  23.0 |  8.6 |  30.8 |  2.0 | 68.3
AT |  49.0 |  50.9 |  35.5 |  38.5 |  47.5 |  61.1 |  46.3 |  48.0 |  30.6 |  34.2 |  43.5 | 61.3
R-Warp |  38.5 |  29.8 |  13.2 |  40.0 |  4.5 |  71.1 |  34.1 |  19.5 |  16.2 |  32.5 |  2.2 | 67.7
VOneBlock |  42.9 |  43.7 |  16.1 |  40.7 |  16.1 |  72.0 |  38.8 |  37.5 |  12.3 |  35.8 |  11.9 | 68.7
R-Blur |  44.2 |  48.7 |  24.2 |  45.6 |  23.8 |  63.3 |  38.9 |  41.0 |  18.2 |  39.0 |  17.2 | 60.5
best, second best
Table 1: Accuracy of the evaluated models on clean, and perturbed data from
Imagenet. “WB” refers to the accuracy under APGD attacks, while “CC” refers to
the accuracy under common non-adversarial corruption (Hendrycks and
Dietterich, 2019). $\text{Mean}_{\text{L}}$ and $\text{Mean}_{\text{H}}$ refer
to the average accuracy on low intensity ($\|\delta\|_{\infty}=0.002$,
$\|\delta\|_{2}=0.5$, severity $\leq 3$) and high intensity
($\|\delta\|_{\infty}\in\\{0.004,0.008\\}$, $\|\delta\|_{2}\in\\{1.5,2.\\}$
perturbations, severity $>3$). R-Blur significantly improves the robustness of
ResNet, and outperforms prior biologically motivated defenses, while
approaching the performance of AT.

###  3.3 Ablation Study

Having established that R-Blur indeed improves the robustness of image
recognition model, we examine, by way of ablation, how much each component of
R-Blur contributes towards this improvement as shown in Figure 8. The most
significant contributor to robustness is the addition of noise during
training, followed by adaptive blurring. Importantly, applying non-adaptive
blurring by convolving the image with a single Gaussian kernel having
$\sigma=10.5$ ($\sigma=10.9$ is the maximum used for adaptive blurring)
improves robustness very slightly while degrading clean accuracy, thus,
indicating the significance of simulating foveation via adaptive blurring. The
next most significant factor is evaluating at multiple fixation points which
improved robustness significantly compared to a single fixation point in the
center of the image, which suggests that, multiple fixations and saccades are
important when the image is hard to recognize and presents a promising
direction for future work. Furthermore, not adaptively desaturating the colors
reduces the robustness slightly. To summarize, all the biologically-motivated
components of R-Blur contribute towards improving the adversarial robustness
of object recognition DNNs from close to 0% to 45% ($\ell_{\infty}=0.008$ for
Ecoset-10)

Figure 8: Accuracy on clean and APGD ($\|\delta\|_{\infty}=0.008$) perturbed
Ecoset-10 images when individual components of R-Blur are removed (left), and
when non-adaptive transforms are used (right). Removing the biologically-
motivated components of R-Blur harms the robustness

##  4 Related Work

Non-biological defenses: Perhaps the most successful class of adversarial
defenses are adversarial training algorithms (Madry et al., 2018; Zhang et
al., 2019; Rebuffi et al., 2021; Bai et al., 2021; Wong et al., 2019), which
train models on adversarially perturbed data generated by backpropagating
gradients from the loss to the input during each training step. Another
popular class of defenses is certified defenses (Cohen et al., 2019; Fischer
et al., 2020; Kumar and Goldstein, 2021; Li et al., 2019) which are
accompanied by provable guarantees of the form: with probability $1-\delta$,
the model’s output will not change if a given image is perturbed at most
$\epsilon$. Perhaps, most closely related to our work are preprocessing
defenses (Guo et al., 2017; Raff et al., 2019) that apply a large number of
transforms to the input during inference. Usually, these defenses rely on non-
differentiable transformations, and a high degree of randomization in the
number, sequence, and parameters of the transforms they apply to each image.
Therefore, these defenses tend to obfuscate gradients (Athalye et al., 2018a),
and have been shown to be compromised by attacks with a higher step budget. We
would like to point out that R-Blur does not have these aforementioned
pitfalls – the transforms that R-Blur applies (Gaussian blur and desaturation)
are fully differentiable and totally deterministic. In general, it is our
opinion that by not being cognizant of the biological basis of robust vision,
current approaches are excluding a large set of potentially effective
approaches for defending against adversarial attacks.

Biologically inspired defenses: These defenses involve integrating
computational analogues of biological processes that are absent from common
DNNs, such as predictive/sparse coding (Paiton et al., 2020; Bai et al.,
2021), biologically constrained visual filters, nonlinearities, and
stochasticity (Dapello et al., 2020), foveation (Jonnalagadda et al., 2022;
Luo et al., 2015; Gant et al., 2021), non-uniform retinal sampling and
cortical fixations (Vuyyuru et al., 2020), into DNNs. The resulting models are
made more robust to adversarially perturbed data.

Retina Models for Robustness: Some prior works study the impact of simulating
foveation in CNNs on their adversarial robustness, but the body of literature
in this space is rather sparse. One of the earliest works (Luo et al., 2015)
implements a foveation-based defense by cropping the input image using various
schemes. (Vuyyuru et al., 2020) proposed a defense that simulates the non-
uniform sampling of the visual stimuli that occurs in the retina. They do this
by resampling input images so that the sampling density of pixels is maximal
at the point of fixation and decays progressively in regions further away from
it. More recently, (Gant et al., 2021) proposed a neural network that modifies
the textures in the image such that the modified image appears identical to
human observers when viewed in peripheral vision. They report improvement in
adversarial robustness when DNNs are trained on images to which this transform
has been applied. While we do not implement and evaluate their approach in
this paper, we note that they report 40% accuracy against perturbation of
$\ell_{\infty}$ norm 0.005 by a 5-step PGD attack, while we achieve an
accuracy close to 50% at $\ell_{\infty}$ norm 0.008 with a 25-step APGD
attack. Although the accuracy of these biologically inspired approaches is
higher than that of an unmodified model under adversarial attack, there is
still a large gap in robustness between the proposed approaches and non-
biological defenses, indicating that there is room for improvement.

##  5 Limitations

Adding R-Blur reduces accuracy on clean data, however, it is possible to
significantly improve the accuracy of R-Blur by developing better methods for
selecting the fixation point. Further experimental results presented in
Appendix B show that if the optimal fixation point was chosen by an oracle the
clean accuracy of R-Blur can be improved to within 2% of the accuracy of the
unmodified ResNet.

##  6 Conclusion

Since the existence of adversarial attacks presents a divergence between DNNs
and humans, we ask if some aspect of human vision is fundamental to its
robustness that is not modeled by DNNs. To this end, we propose R-Blur, a
foveation technique that blurs the input image and reduces its color
saturation adaptively based on the distance from a given fixation point. We
evaluate R-Blur and other baseline models against APGD attacks on two datasets
containing real-world images. R-Blur outperforms other biologically inspired
defenses. Furthermore, R-Blur also significantly improves robustness to
common, non-adversarial corruptions and achieves accuracy greater than that of
adversarial training. The robustness achieved by R-Blur is certifiable using
the approach from (Cohen et al., 2019) and the certified accuracy achieved by
R-Blur is at par or better than that achieved by randomized smoothing (Cohen
et al., 2019). Our work provides further evidence that biologically inspired
techniques can improve the accuracy and robustness of AI models.

## References

  * Yamins et al. [2014] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo.  Performance-optimized hierarchical models predict neural responses in higher visual cortex.  _Proceedings of the national academy of sciences_ , 111(23):8619–8624, 2014.
  * Cadieu et al. [2014] Charles F Cadieu, Ha Hong, Daniel LK Yamins, Nicolas Pinto, Diego Ardila, Ethan A Solomon, Najib J Majaj, and James J DiCarlo.  Deep neural networks rival the representation of primate it cortex for core visual object recognition.  _PLoS computational biology_ , 10(12):e1003963, 2014.
  * Szegedy et al. [2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  In _ICLR_ , 2014.  URL http://arxiv.org/abs/1312.6199.
  * Geirhos et al. [2018] Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann.  Generalisation in humans and deep neural networks.  _Advances in neural information processing systems_ , 31, 2018.
  * Dodge and Karam [2017] Samuel Dodge and Lina Karam.  A study and comparison of human and deep learning recognition performance under visual distortions.  In _2017 26th international conference on computer communication and networks (ICCCN)_ , pages 1–7. IEEE, 2017.
  * Ilyas et al. [2019] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry.  Adversarial examples are not bugs, they are features.  _Advances in neural information processing systems_ , 32, 2019.
  * Madry et al. [2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  In _International Conference on Learning Representations_ , 2018.
  * Wong et al. [2019] Eric Wong, Leslie Rice, and J Zico Kolter.  Fast is better than free: Revisiting adversarial training.  In _International Conference on Learning Representations_ , 2019.
  * Zhang et al. [2019] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan.  Theoretically principled trade-off between robustness and accuracy.  In _International conference on machine learning_ , pages 7472–7482. PMLR, 2019.
  * Cohen et al. [2019] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter.  Certified adversarial robustness via randomized smoothing.  In _International Conference on Machine Learning_ , pages 1310–1320. PMLR, 2019.
  * Fischer et al. [2020] Marc Fischer, Maximilian Baader, and Martin Vechev.  Certified defense to image transformations via randomized smoothing.  _Advances in Neural Information Processing Systems_ , 33:8404–8417, 2020.
  * Carlini et al. [2022] Nicholas Carlini, Florian Tramer, J Zico Kolter, et al.  (certified!!) adversarial robustness for free!  _arXiv preprint arXiv:2206.10550_ , 2022.
  * Joos et al. [2022] Sander Joos, Tim Van hamme, Davy Preuveneers, and Wouter Joosen.  Adversarial robustness is not enough: Practical limitations for securing facial authentication.  In _Proceedings of the 2022 ACM on International Workshop on Security and Privacy Analytics_ , pages 2–12, 2022.
  * Sharma and Chen [2017] Yash Sharma and Pin-Yu Chen.  Attacking the madry defense model with $l\\_1$-based adversarial examples.  _arXiv preprint arXiv:1710.10733_ , 2017.
  * Schott et al. [2018] Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel.  Towards the first adversarially robust neural network model on mnist.  _arXiv preprint arXiv:1805.09190_ , 2018.
  * Paiton et al. [2020] Dylan M Paiton, Charles G Frye, Sheng Y Lundquist, Joel D Bowen, Ryan Zarcone, and Bruno A Olshausen.  Selectivity and robustness of sparse coding networks.  _Journal of vision_ , 20(12):10–10, 2020.
  * Bai et al. [2021] Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang.  Recent advances in adversarial training for adversarial robustness.  _arXiv preprint arXiv:2102.01356_ , 2021.
  * Dapello et al. [2020] Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, and James J DiCarlo.  Simulating a primary visual cortex at the front of cnns improves robustness to image perturbations.  _Advances in Neural Information Processing Systems_ , 33:13073–13087, 2020.
  * Jonnalagadda et al. [2022] Aditya Jonnalagadda, William Yang Wang, B.S. Manjunath, and Miguel Eckstein.  Foveater: Foveated transformer for image classification, 2022.  URL https://openreview.net/forum?id=mqIeP6qPvta.
  * Luo et al. [2015] Yan Luo, Xavier Boix, Gemma Roig, Tomaso Poggio, and Qi Zhao.  Foveation-based mechanisms alleviate adversarial examples.  _arXiv preprint arXiv:1511.06292_ , 2015.
  * Gant et al. [2021] Jonathan M Gant, Andrzej Banburski, and Arturo Deza.  Evaluating the adversarial robustness of a foveated texture transform module in a cnn.  In _SVRHM 2021 Workshop@ NeurIPS_ , 2021.
  * Vuyyuru et al. [2020] Manish Reddy Vuyyuru, Andrzej Banburski, Nishka Pant, and Tomaso Poggio.  Biologically inspired mechanisms for adversarial robustness.  _Advances in Neural Information Processing Systems_ , 33:2135–2146, 2020.
  * Stewart et al. [2020] Emma E. M. Stewart, Matteo Valsecchi, and Alexander C. Schütz.  A review of interactions between peripheral and foveal vision.  _Journal of Vision_ , 20(12):2–2, 11 2020.  ISSN 1534-7362.  doi: 10.1167/jov.20.12.2.  URL https://doi.org/10.1167/jov.20.12.2.
  * Hansen et al. [2009] Thorsten Hansen, Lars Pracejus, and Karl R Gegenfurtner.  Color perception in the intermediate periphery of the visual field.  _Journal of vision_ , 9(4):26–26, 2009.
  * Kolb [2005] Helga Kolb.  Facts and figures concerning the human retina - ncbi bookshelf, May 2005\.  URL https://www.ncbi.nlm.nih.gov/books/NBK11556/.
  * Dragoi and Tsuchitani [2020] Valentin Dragoi and Chieyeko Tsuchitani.  Visual processing: Eye and retina (section 2, chapter 14) neuroscience online: An electronic textbook for the neurosciences: Department of neurobiology and anatomy - the university of texas medical school at houston, 2020.  URL https://nba.uth.tmc.edu/neuroscience/m/s2/chapter14.html.
  * Ramezani et al. [2019] Farzad Ramezani, Saeed Reza Kheradpisheh, Simon J Thorpe, and Masoud Ghodrati.  Object categorization in visual periphery is modulated by delayed foveal noise.  _Journal of Vision_ , 19(9):1–1, 2019.
  * Deza and Konkle [2021] Arturo Deza and Talia Konkle.  Emergent properties of foveated perceptual systems, 2021.  URL https://openreview.net/forum?id=2_Z6MECjPEa.
  * Pramod et al. [2018] RT Pramod, Harish Katti, and SP Arun.  Human peripheral blur is optimal for object recognition.  _arXiv preprint arXiv:1807.08476_ , 2018.
  * Wang and Cottrell [2017] Panqu Wang and Garrison W Cottrell.  Central and peripheral vision for scene recognition: A neurocomputational modeling exploration.  _Journal of vision_ , 17(4):9–9, 2017.
  * Johnson [1986] MARY A Johnson.  Color vision in the peripheral retina.  _American journal of optometry and physiological optics_ , 63(2):97–103, 1986.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * [33] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  Cifar-10 (canadian institute for advanced research).  URL http://www.cs.toronto.edu/~kriz/cifar.html.
  * Mehrer et al. [2021] Johannes Mehrer, Courtney J Spoerer, Emer C Jones, Nikolaus Kriegeskorte, and Tim C Kietzmann.  An ecologically motivated image dataset for deep learning yields better models of human vision.  _Proceedings of the National Academy of Sciences_ , 118(8):e2011417118, 2021.
  * Russakovsky et al. [2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al.  Imagenet large scale visual recognition challenge.  _International journal of computer vision_ , 115:211–252, 2015.
  * Hendrycks and Dietterich [2019] Dan Hendrycks and Thomas Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Cubuk et al. [2018] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le.  Autoaugment: Learning augmentation policies from data.  _arXiv preprint arXiv:1805.09501_ , 2018.
  * Zagoruyko and Komodakis [2016] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks.  _British Machine Vision Conference_ , 2016.
  * Howard and Gugger [2020] Jeremy Howard and Sylvain Gugger.  Fastai: A layered api for deep learning.  _Information_ , 11(2):108, 2020.
  * Kümmerer et al. [2022] Matthias Kümmerer, Matthias Bethge, and Thomas SA Wallis.  Deepgaze iii: Modeling free-viewing human scanpaths with deep learning.  _Journal of Vision_ , 22(5):7–7, 2022.
  * Croce and Hein [2020] Francesco Croce and Matthias Hein.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In _International conference on machine learning_ , pages 2206–2216. PMLR, 2020.
  * Athalye et al. [2018a] Anish Athalye, Nicholas Carlini, and David Wagner.  Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples.  In _International conference on machine learning_ , pages 274–283. PMLR, 2018a.
  * Carlini et al. [2019] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin.  On evaluating adversarial robustness.  _arXiv preprint arXiv:1902.06705_ , 2019.
  * Rebuffi et al. [2021] Sylvestre-Alvise Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann.  Data augmentation can improve robustness.  _Advances in Neural Information Processing Systems_ , 34:29935–29948, 2021.
  * Kumar and Goldstein [2021] Aounon Kumar and Tom Goldstein.  Center smoothing: Certified robustness for networks with structured outputs.  _Advances in Neural Information Processing Systems_ , 34:5560–5575, 2021.
  * Li et al. [2019] Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin.  Certified adversarial robustness with additive noise.  _Advances in neural information processing systems_ , 32, 2019.
  * Guo et al. [2017] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten.  Countering adversarial images using input transformations.  _arXiv preprint arXiv:1711.00117_ , 2017.
  * Raff et al. [2019] Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean.  Barrage of random transforms for adversarially robust defense.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6528–6537, 2019.
  * Athalye et al. [2018b] Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok.  Synthesizing robust adversarial examples.  In _International conference on machine learning_ , pages 284–293. PMLR, 2018b.
  * Tolstikhin et al. [2021] Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al.  Mlp-mixer: An all-mlp architecture for vision.  _Advances in Neural Information Processing Systems_ , 34:24261–24272, 2021.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.

##  Appendix A Preventing Gradient Obfuscation

We take a number of measures to ensure that our results correspond to the true
robustness of our method, and we avoid the pitfalls of gradient obfuscation
[Athalye et al., 2018a, Carlini et al., 2019]. Firstly, we remove inference
time stochasticity from all the models we test. We do this by sampling the
Gaussian noise used in R-Blur and VOneBlock once and applying the same noise
to all test images. Similarly, we sample the affine transform parameters for
RandAffine once and use them for all test images. We also compute the fixation
point sequences for R-Blur and R-Warp on unattacked images and do not update
them during or after running APGD. Secondly, we ran APGD for 1 to 100
iterations and observed that as the number of iterations increases the success
rate of the attack increases (Figure 9(a)). The success rate plateaus at 50
iterations. Since the attack success rate with 25 steps is only 0.1% lower
than the success rate with 50 steps, we run APGD with 25 steps in most of our
experiments. Thirdly, we applied expectation over transformation [Athalye et
al., 2018b] by computing 10 gradient samples at each APGD iteration and
averaging them to obtain the final update. We found this did not change the
attack success rate so we take only 1 gradient sample in most of our
experiments (Figure 9(b)). Finally, we also used a straight-though-estimator
to pass gradients through R-Blur in case it may be obfuscating them and found
that doing so reduces the attack success rate, thus indicating that gradients
that pass through R-Blur retain valuable information that can be used by the
adversarial attack (Figure 9(b)).

(a)

(b)

Figure 9: Accuracy of a R-Blur model trained on Imagenet under APGD attack
with different settings. (a) shows the accuracy when APGD attack is applied
with different numbers of update steps. (b) shows the accuracy when 10 step of
expectation-over-transformation (EOT-10) [Athalye et al., 2018b] is used and
R-Blur is converted into a straight-through-estimator (STE) in the backward
pass. The dashed line in (b) shows the accuracy of a 25-step APGD attack
without EOT and normal gradient computation for R-Blur. Together these results
strongly indicate that R-Blur does not obfuscate gradients and legitimately
improves the adversarial robustness of the model.

##  Appendix B Fixation Point Selection

In this study, we did not attempt to develop an optimal fixation point
selection algorithm, and instead, we operate under the assumption that points
at which humans tend to fixate are sufficiently informative to perform
accurate object classification. Therefore, we used DeepGaze-III [Kümmerer et
al., 2022], which is a neural network model trained to model the human gaze.
DeepGaze-III uses a deep CNN backbone to extract features from the image, and
based on these features another DNN predicts a heatmap that indicates, for
each spatial coordinate, the probability that a human will fixate on it.
However, it is possible that this algorithm is sub-optimal, and with further
study, a better one could be developed. Though developing such an algorithm is
out of the scope of this paper, we conduct a preliminary study to determine if
it is possible to select better fixation points than the ones predicted by
DeepGaze-III.

To this end, we run the following experiment to pick an optimal fixation point
for each image during inference. For each testing image, we select 49 fixation
points, spaced uniformly in a grid. Using the models we trained in earlier
(see section 3) we obtain predictions for each image and each of the 49
fixation points. If there was at least one fixation point at which the model
was able to correctly classify the image, we consider it to be correctly
classified for the purpose of computing accuracy. We repeat this experiment
for Ecoset-10, Ecoset, and Imagenet, using clean and adversarially perturbed
data. We obtain the adversarially perturbed images for each of the 49 fixation
points by fixing the fixation point at one location running the APGD attack
with $\ell_{\infty}$-norm bounded to 0.004. Figure 11 illustrates this
experiment with some example images.

The results are presented in Figure 10. We see that when the optimal fixation
point is chosen accuracy on both clean and adversarially perturbed data
improves, with the improvement in clean accuracy being the most marked. The
clean accuracy on Ecoset-10, Ecoset, and Imagenet improved by 5%, 11%, and 10%
respectively, which makes the clean accuracy of the R-Blur model on par or
better than the clean accuracy achieved by the unmodified ResNet. Furthermore,
when the optimal fixation point, is chosen R-Blur obtains higher clean
accuracy than AT on all the datasets.

(a) Optimal Fixation

(b) DeepGaze-III Five Fixations

(c) Adversarial Training

Figure 10: The accuracy obtained on clean and adversarial data when (a) the
optimal fixation point was selected, (b) when the five fixation approach from
Section 3 was used, and (c) an adversarially trained model was used. Figure
11: This figure indicates the locations of the optimal fixation points for
some sample images. Each square in the grid corresponds to one of 49 fixation
locations and represents the highest resolution region of the image if the
model fixates at the center of the square. Squares that are shaded green
indicate that the model’s prediction at the corresponding fixation point was
correct, while squares shaded red indicate that the model’s prediction at the
corresponding fixation point was incorrect. We see that there are certain
images in which there are only a few optimal fixation points and they may not
be in the center or in the corners of the image.

These results are meant to lay the groundwork for future work toward
developing methods for determining the optimal fixation point based on the
input image. However, they also illustrate that models trained with R-Blur
learn features that are not only more adversarially robust features than
ResNet but also allow the model to make highly accurate predictions on clean
data.

##  Appendix C Evaluations With Different Architectures

To demonstrate that the benefits of R-Blur are not limited to CNNs, we trained
MLP-Mixer [Tolstikhin et al., 2021] and ViT [Dosovitskiy et al., 2020] models
with R-Blur preprocessing and evaluated their robustness. We use the
configuration of MLP-Mixer referred to as S16 in [Tolstikhin et al., 2021].
Our ViT has a similar configuration, with 8 layers each having a hidden size
of 512, an intermediate size of 2048, and 8 self-attention heads. We train
both models with a batch size of 128 for 60 epochs on Ecoset-10 using the Adam
optimizer. The learning rate of the optimizer is linearly increased to 0.001
over 12 epochs and is decayed linearly to almost zero over the remaining
epochs. The results are shown in Figure 12.

We observe that R-Blur significantly improves the robustness of MLP-Mixer
models, and achieves greater accuracy than R-Warp at higher levels of
perturbations. These results show that the robustness endowed to ResNets by
R-Blur was not dependent on the model architecture, and they further
strengthen our claim that loss in fidelity due to foveation contributes to the
robustness of human and computer vision.

(a) ResNet-18

(b) MLP-Mixer

(c) ViT

Figure 12: The accuracy obtained on Ecoset-10 against adversarial
perturbations of various $\ell_{\infty}$ norms when R-Blur is used with
ResNet, MLP-Mixer and ViT backbones.

##  Appendix D Breakdown of Accuracy Against Common Corruption by Corruption
Type

In Figure 14 we break down the performance of the models on common corruptions
by higher-level corruption categories. The individual members of each category
are listed in Table 2. We see that in most of the categories, R-Blur achieves
the highest median accuracy against the most severe corruptions. We also note
that R-Blur exhibits a remarkable degree of robustness to noise, which is
substantially greater than all the other models we evaluated. It is pertinent
to note here that Gaussian noise was just 1 of the 4 types of noise included
in the noise category, and thus the performance of R-Blur can not be
attributed to overfitting on Gaussian noise during training. Furthermore,
robustness to one type of random noise does not typically generalize to other
types of random noise [Geirhos et al., 2018]. Therefore, the fact that R-Blur
exhibits improved robustness to multiple types of noise indicates that it is
not just training on Gaussian noise, but rather the synergy of all the
components of R-Blur that is likely the source of its superior robustness.

Noise | Blur | Weather | Digital
---|---|---|---
gaussian noise | defocus blur | snow | contrast
shot noise | glass blur | frost | elastic transform
impulse noise | motion blur | fog | pixelate
speckle noise | zoom blur | brightness | jpeg compression
| gaussian blur | spatter | saturate
Table 2: Categories of corruptions used to evaluate robustness to common
corruptions. This categorization follows the one from [Hendrycks and
Dietterich, 2019]

(a) Imagenet

(b) Ecoset

(c) blur

(d) noise

(e) weather

(f) digital

(g) CIFAR-10

Figure 14: The accuracy achieved by R-Blur and baselines on various classes of
common corruptions, proposed in [Hendrycks and Dietterich, 2019]. The boxplot
shows the distribution of accuracy values on 4-5 different corruptions in each
class applied at different severity levels (x-axis) with 1 referring to least
severe and 5 being the most severe corruption. R-Blur generally achieves the
highest median accuracy on the highest severity levels.

##  Appendix E Sensitivity Analysis of Hyperparameters in R-Blur

To measure the influence of the various Hyperparameters of R-Blur we conduct a
sensitivity analysis. First, we vary the scale of the Gaussian noise added to
the image, the viewing distance during inference, and the value of $\beta$
from Section 2.5, which is the scaling factor that maps eccentricity (see
equation 1 to standard deviation, and measure the impact on accuracy on clean
as well as adversarially perturbed data. The results of this analysis are
presented in Figure 15. We see that, as expected, increasing the scale of the
noise improves accuracy on adversarially perturbed data, however, this
improvement does not significantly degrade clean accuracy. It appears that the
adaptive blurring is mitigating the deleterious impact of Gaussian noise on
clean accuracy. On the other hand, increasing $\beta$ beyond 0.01 surprisingly
does not have a significant impact on accuracy and robustness. We also
measured the accuracy on clean and perturbed data after varying the viewing
distance (see 2.4) and the number of fixation points over which the logits are
aggregated. These results are plotted in Figure 16, and they show that
accuracy on clean and perturbed data is maximized when the width of the in-
focus region is 48 (this corresponds to $vd=3$) and aggregating over more
fixation points improves accuracy on clean and perturbed data.

Figure 15: The impact of the hyperparameters of R-Blur on the accuracy and
robustness of models trained on Ecoset-10. (left) the standard deviation of
Gaussian noise, and (right) $\beta$ from Section 2.5.

Figure 16: The impact of the size of the in-focus region by varying the
viewing distance (left) and the number of fixation points over which the
logits are aggregated (right) on accuracy. The plots are computed from a
R-Blur model trained on Imagenet, and the perturbed data is obtained by
conducting a 25-step APGD attack with $\|\delta\|_{\infty}=0.004$. We see that
accuracy on clean and perturbed data is maximized when the width of the in-
focus region is 48 (this corresponds to $vd=3$) and aggregating over more
fixation points improves accuracy on clean and perturbed data.

##  Appendix F Training Configuration

Table 3 presents the configurations used to train the models used in our
evaluation. For all the models the SGD optimizer was used with Nesterov
momentum=0.9.

Dataset | Method | Batch Size | nEpochs | LR | LR-Schedule | Weight Decay | nGPUs
---|---|---|---|---|---|---|---
CIFAR-10 | ResNet | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-5 | 1
| AT | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-5 | 1
| R-Warp | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-5 | 1
| R-Blur | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-5 | 1
| G-Noise | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-5 | 1
Ecoset-10 | ResNet | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-4 | 1
| AT | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-4 | 1
| R-Warp | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-4 | 1
| R-Blur | 128 | 0.1 | 60 | L-Warmup-Decay(0.1) | 5e-4 | 1
| VOneBlock | 128 | 0.1 | 60 | L-Warmup-Decay(0.1) | 5e-4 | 1
| G-Noise | 128 | 0.4 | 60 | L-Warmup-Decay(0.2) | 5e-4 | 1
Ecoset | ResNet | 256 | 0.2 | 25 | L-Warmup-Decay(0.2) | 5e-4 | 2
| AT | 256 | 0.2 | 25 | L-Warmup-Decay(0.2) | 5e-4 | 4
| R-Warp | 256 | 0.1 | 25 | L-Warmup-Decay(0.2) | 5e-4 | 4
| R-Blur | 256 | 0.1 | 25 | C-Warmup-2xDecay(0.1) | 5e-4 | 4
| VOneBlock | 256 | 0.1 | 25 | C-Warmup-2xDecay(0.1) | 5e-4 | 4
| G-Noise | 256 | 0.1 | 25 | C-Warmup-2xDecay(0.1) | 5e-4 | 4
Imagenet | ResNet | 256 | 0.2 | 25 | L-Warmup-Decay(0.2) | 5e-4 | 2
| AT | 256 | 0.2 | 25 | L-Warmup-Decay(0.2) | 5e-4 | 4
| R-Warp | 256 | 0.1 | 25 | L-Warmup-Decay(0.2) | 5e-4 | 4
| R-Blur | 256 | 0.1 | 25 | C-Warmup-2xDecay(0.1) | 5e-4 | 4
| VOneBlock | 256 | 0.1 | 25 | C-Warmup-2xDecay(0.1) | 5e-4 | 4
| G-Noise | 256 | 0.1 | 25 | C-Warmup-2xDecay(0.1) | 5e-4 | 4
Table 3: The configurations used to train the models used in our evaluation.
L-Warmup-Decay($f$) represents a schedule that linearly warms up and decays
the learning rate and $f$ represents the fraction of iterations devoted to
warmup. C-Warmup-2xDecay(0.1) is similar except that the warmup and decay
follow a cosine function, and there are two decay phases. Both the schedulers
are implemented using torch.optim.lr_scheduler.OneCycleLR from Pytorch.

##  Appendix G Implementation Details

We used Pytorch v1.11 and Python 3.9.12 to for our implementation. We used the
implementation of Auto-PGD from the Torchattacks library
(https://github.com/Harry24k/adversarial-attacks-pytorch). For R-Warp we used
the code from the official repo https://github.com/mvuyyuru/adversary.git.
Likewise, for VOneBlock we used the code from
https://github.com/dicarlolab/vonenet, and for DeepGaze-III models we used the
code from https://github.com/matthias-k/DeepGaze. The training code for
DeepGaze-III with R-Blur and R-Warp backbones is based on
https://github.com/matthias-k/DeepGaze/blob/main/train_deepgaze3.ipynb, and
can be found in
adversarialML/biologically_inspired_models/src/fixation_prediction/train_deepgaze.py.
Our clones of these repositories are included in the supplementary material.
For multi-gpu training, we used Pytorch Lightning v1.7.6. We used 16-bit mixed
precision training to train most of our models. The code for R-Blur can be
found in adversarialML/biologically_inspired_models/src/retina_preproc.py
which is part of the supplemental material.

##  Appendix H Hardware Details

We trained our models on compute clusters with Nvidia GeForce 2080 Ti and V100
GPUs. Most of the Imagenet and Ecoset models were trained and evaluated on the
V100s, while the CIFAR-10 and Ecoset-10 models were trained and evaluated on
the 2080 Ti’s.


# Static and spherically symmetric vacuum spacetimes with non-expanding
principal null directions in $f(R)$ gravity

Alberto Guilabert  email<EMAIL_ADDRESS>Fundacion Humanismo y
Ciencia, Guzmán el Bueno, 66, 28015 Madrid, Spain  Departamento de Física
Aplicada, Universidad de Alicante, Campus de San Vicente del Raspeig, E-03690
Alicante, Spain  Pelayo V. Calzada  email<EMAIL_ADDRESS>Departamento
de Física Aplicada, Universidad de Alicante, Campus de San Vicente del
Raspeig, E-03690 Alicante, Spain  Pedro Bargueño  email<EMAIL_ADDRESS>Departamento de Física Aplicada, Universidad de Alicante, Campus de San
Vicente del Raspeig, E-03690 Alicante, Spain  Salvador Miret-Artés  email:
<EMAIL_ADDRESS>Instituto de Física Fundamental, Consejo Superior de
Investigaciones Científicas, Serrano 123, 28006, Madrid, Spain

###### Abstract

In this work we characterize all the static and spherically symmetric vacuum
solutions in $f(R)$ gravity when the principal null directions of the Weyl
tensor are non-expanding. In contrast to General Relativity, we show that the
Nariai spacetime is not the only solution of this type when general $f(R)$
theories are considered. In particular, we find four different solutions for
the non-constant Ricci scalar case, all of them corresponding to the same
theory, given by $f(R)=r_{0}^{-1}\left\lvert R-3/r_{0}^{2}\right\rvert^{1/2}$,
where $r_{0}$ is a non-null constant. Finally, we briefly present some
geometric properties of these solutions.

††preprint: APS/123-QED

##  I Introduction

The Nariai spacetime, presented back in the 1950’s by Nariai [1, 2], can be
described in suitable coordinates by the line element

$ds^{2}=\left(1-\dfrac{r^{2}}{r_{0}^{2}}\right)dt^{2}-\left(1-\dfrac{r^{2}}{r_{0}^{2}}\right)^{-1}dr^{2}-r_{0}^{2}d\Omega^{2},$
(1)

with $r\in(-r_{0},r_{0})$, where $r_{0}$ is a non-null constant, and
$d\Omega^{2}$ is the line element of the 2-sphere. This spacetime is a well
known static and spherically symmetric solution for General Relativity (GR)
with positive cosmological constant $\lambda=1/r_{0}^{2}$.

This spacetime is usually characterized as a special limit of the
Schwarzschild–de Sitter solution when the event and cosmological horizons
coincide, see e.g. [3]. Nevertheless, this limit is not defined in a
meaningful sense, as it has been pointed out in [4]. In fact, the resulting
spacetimes depend on the coordinate choice when taking limits in the metric
tensor [5].

Therefore, we may turn to Petrov classification and the Newman-Penrose (NP)
formalism as elegant tools to fully characterize this solution, given their
usefulness in the study of algebraically special spacetimes.

Considering the Weyl decomposition of the Riemann tensor, Petrov developed a
classification of spacetimes [6] examining the algebraic structure of the
curvature tensor. This classification can be carried out by studying the
eigenbivectors of the Weyl tensor, which are associated to four null vectors
that determine the so called principal null directions of the Weyl tensor.
When there is at least one degeneracy between the four principal null
directions the spacetime is said to be algebraically special. Moreover, the
Petrov classification was also studied later in terms of spinors, see e.g. [7,
8], and recently extended to higher dimensions [9].

Every spherically symmetric spacetime is algebraically special [10], which
naturally induces the use of the NP formalism. In particular, static and
spherically symmetric spacetimes are type D (or O). That is, there are two
degenerated principal null directions (or the spacetime is conformally flat,
which corresponds to type O). As it is usual in the NP formalism, we construct
a null tetrad with two null vectors $l^{\mu}$ and $n^{\mu}$, which we take to
be aligned with the two principal null directions [11]. The tetrad is
completed by combining a pair of real orthogonal spacelike unit vectors to
define a complex null vector $m^{\mu}$ and its complex conjugate
$\bar{m}^{\mu}$. Such a null tetrad allows us to take advantage of the
symmetries of the Weyl tensor in algebracially special solutions.

In the spherically symmetric case, the congruences associated to the principal
null directions are indeed geodesic, non-rotating and shear-free. In terms of
the optical scalars in the NP formalism that is $\kappa=\omega=\sigma=0$. In
particular, for static and spherically symmetric spacetimes, $\rho$ will
determine the congruence expansion of $l$ and $n$ [12]. In this context, we
restrict our analysis to spacetimes whose principal null directions are non-
expanding. That is a subset of Kundt class spacetimes, which are defined as
the ones admitting a non-expanding, non-rotating and shear-free congruence of
null geodesics, see e.g. [13]. With the previous assumptions, $\rho=0$ implies
that the spacetime can be decomposed as the direct product of two 2-spaces.

The Nariai spacetime, introduced in Eq. (1), is the paradigmatic prototype for
a solution with the previous properties in GR. In fact, it is the only static
and spherically symmetric solution with non-expanding principal null
directions in GR with cosmological constant [14]. Moreover, its uniqueness has
also been recently proved in larger dimensions [15].

At this point, the following question naturally arises: is the Nariai solution
the only static and spherically symmetric vacuum spacetime with non-expanding
principal null directions beyond GR?

In this context, Extended Theories of Gravity (ETGs) have been developed to
approach some known problems associated to GR, such as the existence of black
hole and cosmological singularities [16] or the lack of a satisfactory
description for the accelerated expansion of the universe. Among different
ETGs (see e.g. the standard reference from Capozziello and Faraoni [17] and
references therein), two paradigmatic examples are scalar-tensor [18] or
$f(R)$ theories [19].

In particular, $f(R)$ theories can be useful as toy models which allow us to
study gravity modifications in a simplified way. Even more, it is known that
the only possibility to obtain a potentially stable local modification of GR
is to make the Lagrangian an arbitrary function of the Ricci scalar [20].

Through the last years, many exact solutions of $f(R)$ gravity have been
discovered [17, 19, 21]. For static and spherically symmetric vacuum
spacetimes, the case with expanding principal null directions have been
studied by Multamäki and Vilja [22] and by Sebastiani and Zerbini [23]. The
non-vacuum case has also been studied describing the equilibrium configuration
of a star [24, 25].

Regarding the vacuum case, it has been shown that constant Ricci scalar
solutions are indeed Einstein spaces, which is the case of the Nariai
spacetime. In this case, an existence condition for compatible $f(R)$ theories
has been given [26, 27, 28].

Following this motivation, our aim is to fully characterize all the static and
spherically symmetric vacuum solutions with non-expanding principal null
directions for $f(R)$ theories.

This manuscript is organized as follows: In Sec. II we give a brief
introduction to the $f(R)$ gravity formalism and the corresponding field
equations. In Sec. III we construct the line element for a static and
spherically symmetric spacetime when the non-expanding condition on the
principal null directions is considered. Then, we present the field equations
in the new basis constructed with the principal null directions and we solve
for the metric, giving the compatible $f(R)$ theories. In particular, we
briefly comment some relevant geometric properties about the solutions.
Finally, in Sec. IV we point out the main results.

Through all the manuscript we use the signature (1,3) for the metric and the
Penrose and Rindler’s sign convention for the Riemann curvature tensor [29].

##  II $f(R)$ gravity

For the vacuum case, the general action in $f(R)$ formalism is [17]

$S[g_{\mu\nu}]=\int d^{4}x\sqrt{-g}f(R),$ (2)

where we have set $16\pi G=1$, with $G$ denoting the gravitational constant.

Applying the variational principle $\delta S=0$, the field equations for
$f(R)$ gravity are

$-R_{\mu\nu}=\Delta t_{\mu\nu},$ (3)

where we have defined the effective stress-energy tensor as

$\Delta t_{\mu\nu}\equiv F(R)^{-1}\Big{(}-\frac{1}{2}f(R)g_{\mu\nu}\\\
+\big{[}\nabla_{\mu}\nabla_{\nu}-g_{\mu\nu}\square\big{]}F(R)\Big{)},$ (4)

with $F(R)\coloneqq df(R)/dR$. Then, the associated trace equation is

$R=F(R)^{-1}\left(2f(R)+3\square F(R)\right).$ (5)

In the following section we rewrite these field equations employing the NP
formalism for our particular case of study.

##  III static and spherically symmetric vacuum solutions with non-expanding
principal null directions

We start with a general static and spherically symmetric spacetime $(M,g)$,
whose line element is given by

$ds^{2}=p(r)dt^{2}-s(r)dr^{2}-q(r)d\Omega^{2},$ (6)

where $p(r),s(r)$ and $q(r)$ are positive functions.

In this case, the principal null directions are

$\displaystyle l_{\mu}$ $\displaystyle=$
$\displaystyle\dfrac{1}{\sqrt{2}}\left(p(r)^{1/2},s(r)^{1/2},0,0\right),$
$\displaystyle n_{\mu}$ $\displaystyle=$
$\displaystyle\dfrac{1}{\sqrt{2}}\left(p(r)^{1/2},-s(r)^{1/2},0,0\right),$ (7)

and we complete the null tetrad with

$m_{\mu}=\dfrac{q(r)}{\sqrt{2}}\left(0,0,1,-i\sin\theta\right),$ (8)

where $l_{\mu}n^{\mu}=-m_{\mu}\bar{m}^{\mu}=1$.

Imposing the non-expanding condition on $l$, i.e. $\rho=0$, it is
straightforward to verify that $q(r)=r_{0}^{2}$, where $r_{0}$ can be taken as
a positive constant without loss of generality. Note that this choice of
$q(r)$ assures the non-expanding property of $n$.

This case cannot be transformed by a redefinition of coordinates into the
expanding case with $q(r)=r^{2}$, which is usually considered in spherical
symmetry, as the aforementioned expansion, $\rho$, is independent of the
coordinate choice. Note that this distinction was discussed in detail in [16,
13, 30] and more recently in [14]. In addition, this differentiation was
explicitly studied by Kinnersley when solving for all Type D vacuum metrics
within GR [31]. Unfortunately, the ansatz with $q(r)=r^{2}$ is still generally
assumed as the most general static and spherically symmetric spacetime,
despite the fact that these two cases are not related.

After considering the non-expanding condition, the resulting spacetime $(M,g)$
can be decomposed as a product manifold $M=N\times S^{2}(r_{0})$ with
$g=g_{N}\oplus g_{S^{2}(r_{0})}$, where $g_{N}$ is the metric of a Lorentzian
2-surface and $g_{S^{2}(r_{0})}$ is the metric tensor of the Riemannian
2-sphere of radius $r_{0}$.

Therefore, the change of coordinates

$\tilde{r}(r)=\int_{\alpha_{0}}^{r}du\left(s(u)p(u)\right)^{1/2}$ (9)

transforms the metric given in Eq. (6) into

$ds^{2}=p(r)dt^{2}-p(r)^{-1}dr^{2}-r_{0}^{2}d\Omega^{2},$ (10)

after renaming $\tilde{r}$ as $r$ 111The function $\tilde{r}(r)$ is strictly
increasing so there exists its inverse function $r(\tilde{r})$..

As commented in Sec. I, we present the field equations given in Eq. (3) in the
null tetrad basis defined in Eqs. (III) and (8). Note that in these new
coordinates this basis is expressed as

$\displaystyle l_{\mu}$ $\displaystyle=$
$\displaystyle\dfrac{1}{\sqrt{2}}\left(p(r)^{1/2},p(r)^{-1/2},0,0\right),$
$\displaystyle n_{\mu}$ $\displaystyle=$
$\displaystyle\dfrac{1}{\sqrt{2}}\left(p(r)^{1/2},-p(r)^{-1/2},0,0\right),$
(11) $\displaystyle m_{\mu}$ $\displaystyle=$
$\displaystyle\dfrac{r_{0}}{\sqrt{2}}\left(0,0,1,-i\sin\theta\right).$

In order to state the field equations, we introduce the following notation.

On one hand, the so called Ricci scalars, $\Phi_{ab}$ with
$a,b\in\\{0,1,2\\}$, are defined as the contractions of the Ricci tensor with
the null tetrad vectors (see e.g. [11, 12]). These are the components of the
Ricci tensor in the new basis introduced in Eq. (III). For the metric given by
Eq. (10) the only non-vanishing scalar is

$\Phi_{11}=-\dfrac{1}{2}R_{\mu\nu}l^{\mu}n^{\nu}+3\Lambda,$ (12)

where $\Lambda=R/24$, with $R$ being the Ricci scalar curvature.

On the other hand, we define the physical contractions of the effective
stress-energy tensor in an analogous way as the Ricci scalars. Using the field
equations given in Eq. (3), the only non-vanishing scalars are

$\displaystyle\Phi^{ph}_{00}$ $\displaystyle=$
$\displaystyle\dfrac{1}{2}\Delta t_{\mu\nu}l^{\mu}l^{\mu},$ (13)
$\displaystyle\Phi^{ph}_{11}$ $\displaystyle=$
$\displaystyle\dfrac{1}{2}\Delta t_{\mu\nu}l^{\mu}n^{\nu}+3\Lambda^{ph},$ (14)
$\displaystyle\Phi^{ph}_{22}$ $\displaystyle=$
$\displaystyle\dfrac{1}{2}\Delta t_{\mu\nu}n^{\mu}n^{\nu},$ (15)

with $\Lambda^{ph}=R/24$, where $R$ is now obtained in terms of $f(R)$ by
using Eq. (5).

At this point, the only field equations which are not identically zero in the
new basis are given by $\Phi^{ph}_{00}=\Phi^{ph}_{22}=0$,
$\Phi^{ph}_{11}=\Phi_{11}$ and $\Lambda^{ph}=\Lambda$. For convenience we will
take the following linear combinations:

  * •

$\Phi^{ph}_{00}=\Phi_{00}$

$F^{\prime\prime}(r)=0,$ (16)

  * •

$\Phi^{ph}_{11}=\Phi_{11}$

$-\dfrac{F^{\prime}(r)p^{\prime}(r)+p(r)F^{\prime\prime}(r)}{F(r)}=\dfrac{2}{r_{0}^{2}}+p^{\prime\prime}(r),$
(17)

  * •

$\Phi^{ph}_{11}-\Lambda^{ph}=\Phi_{11}-\Lambda$

$-\dfrac{f(r)}{2F(r)}=\dfrac{1}{r_{0}^{2}}+p^{\prime\prime}(r),$ (18)

where $F(r)\equiv F(R(r))$ and $f(r)\equiv f(R(r))$.

Observe that Eq. (16) implies

$F(r)=a(1+br),$ (19)

where $a$ and $b$ are constants having dimensions of one over length.

In addition, from Eq. (19) it is natural to consider two different cases
depending whether $b=0$ or $b\neq 0$. Interestingly, as we show along this
section, $b=0$ corresponds to constant Ricci scalar and $b\neq 0$ to non-
constant Ricci scalar. This will allow us to fully characterize the solutions
in terms of the Ricci scalar.

###  III.1 Non-constant Ricci scalar solutions

Assuming $b\neq 0$ in Eq. (17), the solution is given by

$p(r)=c_{1}-\dfrac{r}{br_{0}^{2}}-\dfrac{r^{2}}{2r_{0}^{2}}+\dfrac{\gamma}{b^{2}r_{0}^{2}}\log\left|1+br\right|.$
(20)

where $\gamma=1+c_{2}br_{0}^{2}$, being $c_{1}$ and $c_{2}$ two arbitrary
constants. Indeed, their relevance on the properties of the resulting
spacetimes is shown below.

Observe that in the limit case $b\to 0$ on Eq. (20) the Nariai solution is
recovered as

$\lim_{b\to 0}p(r)=c_{1}-c_{2}r-\dfrac{r^{2}}{r_{0}^{2}},$ (21)

which always can be transformed into Eq. (1) by a suitable change of
coordinates.

Moreover, the Ricci scalar is given by

$R(r)=\dfrac{1}{r_{0}^{2}}\left(3+\dfrac{\gamma}{\left(1+br\right)^{2}}\right),$
(22)

and it shows a curvature singularity at $r=-1/b$.

Now we focus on the study of the compatible $f(R)$ theories. Eq. (18) can be
written in terms of the Ricci scalar as

$R-\dfrac{3}{r_{0}^{2}}=\dfrac{f(R)}{2f^{\prime}(R)},$ (23)

and from this equation we deduce

$f(R)=\dfrac{\alpha}{r_{0}}\left\lvert
R-\dfrac{3}{r_{0}^{2}}\right\rvert^{1/2},$ (24)

where $\alpha$ is a dimensionless non-null constant. We take $\alpha=1$
without loss of generality 222A constant multiple of the action does not
modify the field equations when the variational principle is applied.. Also
note that GR is not recovered in the limit case when $b\to 0$ shown in Eq.
(21) since the theory, determined by Eq. (24), is independent of $b$.

It remains to verify the consistency between Eqs. (19), (22) and (24). That
is,

$\frac{df}{dR}(R(r))=F(r),$ (25)

which yields

$F(r)=\dfrac{\gamma}{2\left|\gamma\right|^{3/2}}\left|1+br\right|,$ (26)

from where $a=\text{sign}(1+br)\gamma\left|\gamma\right|^{-3/2}/2$. Note that
$a$ is constant in each connected subinterval of the metric domain (as
$r=-1/b$ is not included). Moreover, $a\in\mathbb{R}$ imposes $\gamma\neq 0$.
Therefore, by Eq. (22), the Ricci scalar is non-constant.

The solution given in Eq. (20) has, if any, the following Killing horizons
generated by $\partial_{t}$, which are determined by

$r_{h}=-\dfrac{1}{b}\left(1\pm\sqrt{-\gamma
W_{k}\bigl{(}\xi(c_{1},\gamma)\bigr{)}}\right),$ (27)

where we have defined

$\xi(c_{1},\gamma)=-\dfrac{1}{\gamma}e^{\frac{-1-2c_{1}b^{2}r_{0}^{2}}{\gamma}}$
(28)

and $W_{k}$ is $k$-th branch of the Lambert’s function. As $r$ must be a real
value, the only considered branches are $k\in\\{-1,0\\}$.

At this point, we differentiate into four different cases for
$\xi(c_{1},\gamma)$. Each case represents the number of real values of the
Lambert $W$ function for the two considered branches, as Fig. 1 shows. This
determines the number of Killing horizons, according to Eq. (27).

Figure 1: Lambert’s $W$ function for the two branches $k\in\\{0,1\\}$. The
principal and $k=-1$ branches are represented in solid and dashed lines,
respectively. Case I is defined when no real value of the Lambert function
exists. In case II the real value of the two branches coincides. Case III
determines two different real values for the Lambert $W$ function. Finally,
case IV is defined when the principal branch is the only real value. Note
that, in accordance with Eq. (28), $\xi(c_{1},\gamma)=0$ is excluded.

  * •

Case I is determined by $\xi(c_{1},\gamma)<-1/e$. Under this condition, there
are no Killing horizons generated by $\partial_{t}$ and the metric function is
$p(r)<0$ for all $r\in\mathbb{R}\setminus\\{-1/b\\}$. Then, no static patch is
found.

  * •

Case II is given by the relation $\xi(c_{1},\gamma)=-1/e$. In this situation,
there are two different Killing horizons (these are degenerated, in the sense
that the two branches coincide) and the metric function is $p(r)\leq 0$ for
all $r\in\mathbb{R}\setminus\\{-1/b\\}$. Thus, there is no static patch in
this case.

  * •

Case III is defined for $-1/e<\xi(c_{1},\gamma)<0$. There exist four different
Killing horizons, which lead to two disconnected static patches.

A representative example of the metric function $p(r)$ is shown in Fig. 2 for
cases I, II and III. Observe that in these cases $\gamma>0$, so the Ricci
scalar is greater than $3/r_{0}^{2}$. Moreover, the curvature singularity lies
outside the static region.

Note that by varying $c_{1}$ it is possible to switch between cases
$\texttt{I},\texttt{II}$ and III, as it is shown in Fig. 3. Moreover, as the
number of Killing horizons generated by $\partial_{t}$ must be invariant under
local isometries, the constant $c_{1}$ cannot be removed under changes of
coordinates.

  * •

Case IV is determined by $\xi(c_{1},\gamma)>0$. There are only two different
Killing horizons which lead to the existence of one static patch. Moreover,
they are independent of the possible values of $c_{1}$. For illustration, a
representation of $p(r)$ is shown in Fig. 4.

Observe that, by Eq. (22), the Ricci scalar changes its sign at

$r=-\dfrac{1}{b}\left(1\pm\sqrt{-\gamma/3}\right),$ (29)

and is upper bounded by $3/r_{0}^{2}$. Also notice that the constant $c_{2}$
cannot be removed either by changes of coordinates because by varying $c_{2}$
it is possible to switch from case IV to the other cases (see Fig. 3) which
are not isometric as the different range of the Ricci scalar reveals.

In addition, observe that the curvature singularity at $r=-1/b$ is in this
case located inside the static region.

Figure 2: Metric function $p(r)$ against coordinate $r$ obtained for
$b=r_{0}=1$, $\gamma=3$ and different values of $c_{1}$ in the three possible
regions with positive $\gamma$. With solid line we show $c_{1}$ in region I,
with dashed line in region II and with dot-dashed line in region III. The
filled region represents the static patch. Figure 3: Different values of
$c_{1}$ and $\gamma$ define different cases as shown in Fig. 1. For
$\gamma>0$, we can go through cases I, II and III by varying $c_{1}$. Case IV
corresponds to $\gamma<0$ regardless of the value of constant $c_{1}$. Note
that it is possible to switch between case IV and the other cases by varying
$\gamma$ (or equivalently, $c_{2}$). The case $\gamma=0$, represented in
dashed line, is excluded. Figure 4: Metric function $p(r)$ against coordinate
$r$ in region IV obtained for $b=r_{0}=1$, $c_{1}=2$ and $\gamma=-1$. The
filled region represents the static patch.

Finally, the constant Ricci scalar case will be briefly examined for
completeness.

###  III.2 Constant Ricci scalar solutions

Assuming $b=0$, the solution of Eq. (17) is

$p(r)=c_{1}+c_{2}r-\dfrac{r^{2}}{r_{0}^{2}},$ (30)

where $c_{1}$ and $c_{2}$ are arbitrary constants that can be removed by a
suitable coordinate changes. Thus, we get

$p(r)=1-\dfrac{r^{2}}{r_{0}^{2}},$ (31)

which is the Nariai solution introduced in Eq. (1). Indeed, the Ricci scalar
is constant with $R(r)=R_{0}\equiv 4/r_{0}^{2}$.

From Eq. (18) it can be seen that the set of compatible $f(R)$ theories are
those functions fulfilling the one point differential equation

$\frac{df}{dR}\left(\frac{4}{r_{0}^{2}}\right)=\frac{r_{0}^{2}}{2}f\left(\frac{4}{r_{0}^{2}}\right).$
(32)

This can be rewritten as

$R_{0}\dfrac{df}{dR}(R_{0})=2f(R_{0}),$ (33)

which is actually the trace equation, introduced in Eq. (5), for constant
Ricci scalar. Note that Eq. (33) was also presented in [26, 27, 28].

At this point, we can conclude that the only constant Ricci scalar solution to
any theory satisfying Eq. (33) is the Nariai spacetime. Note that, in general,
these theories are not necessarily GR although the corresponding field
equations can be interpreted, for constant $R=R_{0}$, in terms of GR with a
cosmological constant given by
$\lambda=\frac{R_{0}}{2}-\frac{f(R_{0})}{2F(R_{0})}$ [17]. In our particular
case, Eq. (33) implies $\lambda=R_{0}/4=1/r_{0}^{2}$, which corresponds to the
cosmological constant for the Nariai solution.

##  IV Final Remarks

This manuscript aims to extend the study carried out by Multamäki and Vilja
[22] for static and spherically symmetric vacuum spacetimes in $f(R)$ gravity
for the case of non-expanding principal null directions, dividing our analysis
in two different cases depending on whether the Ricci scalar is constant or
not.

For the non-constant Ricci scalar case we have shown that there exists only
one $f(R)$ theory which has the form

$f(R)=\dfrac{1}{r_{0}}\left\lvert R-\dfrac{3}{r_{0}^{2}}\right\rvert^{1/2}.$
(34)

This theory admits four different spherically symmetric vacuum solutions
(cases I–IV) with non-expanding principal null directions, which exhibit a
curvature singularity at $r=-1/b$. Interestingly, only two of them (cases III
and IV) present static patches.

In case I, no Killing horizons generated by $\partial_{t}$ are found and there
is no static patch. Similarly, no static region is defined in case II despite
there exist two Killing horizons. In case III, there are four different
Killing horizons which define two static patches. Finally, in case IV there
are two Killing horizons resulting in a static region.

In cases I–III, the curvature singularity is found to be outside the static
region. On the contrary, in case IV, the curvature singularity is located
inside the static patch.

We note that our case III with $c_{1}=\gamma=1$ was recently obtained in the
context of scale-dependent gravity [34]. Given that there is a scale-dependent
representation for any $f(R)$ theory [35], this would enable the
interpretation of all the solutions obtained in this manuscript in terms of
variable gravitational couplings. We left this for future work.

For completeness, we have considered the constant Ricci scalar case, where the
Nariai spacetime is shown to be the only static and spherically symmetric
vacuum spacetimes with non-expanding principal null directions in $f(R)$
gravity.

Therefore, we can finally assure that the Nariai solution is not the only
static and spherically symmetric vacuum spacetime with non-expanding principal
null directions in $f(R)$ gravity.

## Acknowledgments

We acknowledge financial support from the Generalitat Valenciana through
PROMETEO PROJECT CIPROM/2022/13. A. G. acknowledges Fundación Humanismo y
Ciencia for financial support. P. V. C. acknowledges support from Generalitat
Valenciana grant CIACIF/2021/268. A.G. acknowledges M. Carmen and Jant for
continuous support.

## References

  * Nariai [1950] H. Nariai, On some static solutions of Einstein’s gravitational field equations in a spherically symmetric case, Sci. Rep. Tohoku Univ. Eighth Ser. 34, 160 (1950).
  * Nariai [1951] H. Nariai, On a new cosmological solution of Einstein’s field equations of gravitation, Sci. Rep. Tohoku Univ. Eighth Ser. 35, 46 (1951).
  * Ginsparg and Perry [1983] P. Ginsparg and M. J. Perry, Semiclassical perdurance of de sitter space, Nuclear Physics B 222, 245 (1983).
  * Stotyn [2015] S. Stotyn, A tale of two horizons, Canadian Journal of Physics 93, 995 (2015).
  * Geroch [1969] R. Geroch, Limits of spacetimes, Communications in Mathematical Physics 13, 180 (1969).
  * Petrov [1954] A. Z. Petrov, The classification of spaces defining gravitational fields, Uchenye Zapiski Kazanskogo Gosudarstvennogo Universiteta im. V. I. Ulyanovicha-Lenina [Scientific Proceedings of Kazan State University 114, 55 (1954).
  * Witten [1959] L. Witten, Invariants of General Relativity and the Classification of Spaces, Physical Review 113, 357 (1959).
  * Penrose [1960] R. Penrose, A spinor approach to general relativity, Annals of Physics 10, 171 (1960).
  * Coley  _et al._ [2004] A. Coley, R. Milson, V. Pravda, and A. Pravdová, Classification of the Weyl tensor in higher dimensions, Classical and Quantum Gravity 21, L35 (2004), arXiv:gr-qc/0401008 [gr-qc] .
  * Wald [1984] R. M. Wald,  _General Relativity_ (Chicago Univ. Pr., 1984).
  * Newman and Penrose [1962] E. Newman and R. Penrose, An Approach to Gravitational Radiation by a Method of Spin Coefficients, Journal of Mathematical Physics 3, 566 (1962).
  * Bargueño and Contreras [2023] P. Bargueño and E. Contreras,  _The Geroch-Held-Penrose Calculus: Fundamentals and Applications_ (Springer Nature Switzerland, 2023).
  * Stephani  _et al._ [2003] H. Stephani, D. Kramer, M. MacCallum, C. Hoenselaers, and E. Herlt,  _Exact solutions of Einstein’s field equations_, Cambridge Monographs on Mathematical Physics (Cambridge Univ. Press, Cambridge, 2003).
  * Schleich and Witt [2010] K. Schleich and D. M. Witt, A simple proof of Birkhoff’s theorem for cosmological constant, Journal of Mathematical Physics 51, 112502 (2010), arXiv:0908.4110 [gr-qc] .
  * Katona and Lucietti [2024] D. Katona and J. Lucietti, Uniqueness of the extremal Schwarzschild de Sitter spacetime, Lett. Math. Phys. 114, 18 (2024), arXiv:2309.04238 [gr-qc] .
  * Hawking and Ellis [1973] S. W. Hawking and G. F. R. Ellis,  _The large-scale structure of space-time._ , Cambridge Monographs on Mathematical Physics (Cambridge University Press, 1973).
  * Faraoni and Capozziello [2011] V. Faraoni and S. Capozziello,  _Beyond Einstein Gravity: A Survey of Gravitational Theories for Cosmology and Astrophysics_ (Springer, Dordrecht, 2011).
  * Capozziello and de Laurentis [2011] S. Capozziello and M. de Laurentis, Extended Theories of Gravity, Physics Reports 509, 167 (2011), arXiv:1108.6266 [gr-qc] .
  * Sotiriou and Faraoni [2010] T. P. Sotiriou and V. Faraoni, f(R) theories of gravity, Reviews of Modern Physics 82, 451 (2010), arXiv:0805.1726 [gr-qc] .
  * Woodard [2007] R. Woodard, Avoiding dark energy with 1/r modifications of gravity, in  _The Invisible Universe: Dark Matter and Dark Energy_ (Springer Berlin Heidelberg, Berlin, Heidelberg, 2007) pp. 403–433.
  * Gao and Shen [2016] C. Gao and Y.-G. Shen, Exact solutions in F( R) theory of gravity, General Relativity and Gravitation 48, 131 (2016), arXiv:1602.08164 [gr-qc] .
  * Multamäki and Vilja [2006] T. Multamäki and I. Vilja, Spherically symmetric solutions of modified field equations in f(R) theories of gravity, Phys. Rev. D 74, 064022 (2006), arXiv:astro-ph/0606373 [astro-ph] .
  * Sebastiani and Zerbini [2011] L. Sebastiani and S. Zerbini, Static spherically symmetric solutions in F( R) gravity, European Physical Journal C 71, 1591 (2011), arXiv:1012.5230 [gr-qc] .
  * Multamaki and Vilja [2007] T. Multamaki and I. Vilja, Static spherically symmetric perfect fluid solutions in f(R) theories of gravity, Phys. Rev. D 76, 064021 (2007), arXiv:astro-ph/0612775 .
  * Shojai and Shojai [2012] A. Shojai and F. Shojai, Some static spherically symmetric interior solutions of f ( R) gravity, General Relativity and Gravitation 44, 211 (2012), arXiv:1109.2190 [gr-qc] .
  * Barrow and Ottewill [1983] J. D. Barrow and A. C. Ottewill, The stability of general relativistic cosmological theory, Journal of Physics A Mathematical General 16, 2757 (1983).
  * Cognola  _et al._ [2005] G. Cognola, E. Elizalde, S. Nojiri, S. D. Odintsov, and S. Zerbini, One-loop f(R) gravity in de Sitter universe, Journal of Cosmology and Astroparticle Physics 2005 (2), 010, arXiv:hep-th/0501096 [hep-th] .
  * Sebastiani  _et al._ [2013] L. Sebastiani, D. Momeni, R. Myrzakulov, and S. D. Odintsov, Instabilities and (anti)-evaporation of Schwarzschild-de Sitter black holes in modified gravity, Physical Review D 88, 104022 (2013), arXiv:1305.4231 [gr-qc] .
  * Penrose and Rindler [1984] R. Penrose and W. Rindler,  _Spinors and space-time._, Cambridge Monographs on Mathematical Physics (Cambridge Univ. Press, Cambridge, UK, 1984).
  * Nariai [1999] H. Nariai, Editor’s Note: On Some Static Solutions of Einstein’s Gravitational Field Equations in a Spherically Symmetric Case, General Relativity and Gravitation 31, 945 (1999).
  * Kinnersley [1969] W. Kinnersley, Type D Vacuum Metrics, Journal of Mathematical Physics 10, 1195 (1969).
  * Note [1] The function $\tilde{r}(r)$ is strictly increasing so there exists its inverse function $r(\tilde{r})$.
  * Note [2] A constant multiple of the action does not modify the field equations when the variational principle is applied.
  * Rincón and Bargueño [2023] A. Rincón and P. Bargueño, Nariai-like black holes in light of scale-dependent gravity, Eur. Phys. J. C 83, 836 (2023).
  * Calzada  _et al._ [2023] P. V. Calzada, A. Rincón, and P. Bargueño, Comparing f(R) and scale-dependent gravities, Eur. Phys. J. C 83, 1101 (2023).