[1]Yi Xue

[1]Department of Biomedical Engineering, University of California, Davis, 451
Health Sciences Dr., Davis, 95616, CA, United States

# Fluorescence Diffraction Tomography using Explicit Neural Fields

Renzhi He<EMAIL_ADDRESS>Yucheng Li<EMAIL_ADDRESS>Junjie Chen
<EMAIL_ADDRESS><EMAIL_ADDRESS>*

###### Abstract

Solving the 3D refractive index (RI) from fluorescence images provides both
fluorescence and phase information about biological samples. However,
accurately retrieving the phase of partially coherent light to reconstruct the
unknown RI of label-free phase objects over a large volume, at high
resolution, and in reflection mode remains challenging. To tackle this
challenge, we developed fluorescence diffraction tomography (FDT) with
explicit neural fields that can reconstruct 3D RI from defocused fluorescence
speckle images. The successful reconstruction of 3D RI using FDT relies on
four key components: coarse-to-fine modeling, self-calibration, a differential
multi-slice rendering model, and partial coherent masks. Specifically, the
explicit representation efficiently integrates with the coarse-to-fine
modeling to achieve high-speed, high-resolution reconstruction. Moreover, we
advance the multi-slice equation to differential multi-slice rendering model,
which enables the self-calibration method for the extrinsic and intrinsic
parameters of the system. The self-calibration facilitates high accuracy
forward image prediction and RI reconstruction. Partial coherent masks are
digital masks to resolve the discrepancies between the coherent light model
and the partial coherent light data accurately and efficiently. FDT
successfully reconstructed the RI of 3D cultured label-free 3D MuSCs tube in a
530 $\times$ 530 $\times$ 300 $\mu m^{3}$ volume at 1024$\times$1024 pixels
across 24 $z$-layers from fluorescence images, demonstrating high fidelity 3D
RI reconstruction of bulky and heterogeneous biological samples in vitro.

###### keywords:

Optical diffraction tomography, neural fields, 3D reconstruction, multi-model
imaging

##  1 Introduction

Fluorescence and phase microscopy are two powerful techniques in the field of
biological imaging, providing essential insights into the molecular
specificity and quantitative biophysical properties of biological samples.
Fluorescence microscopy captures molecular specified structural and functional
information through exogenous fluorescence labeling or intrinsic
autofluorescence. Phase microscopy quantitatively evaluate the biophysical
properties of biological samples by measuring their refractive index (RI).
Combining these distinct imaging modalities to image the same sample allows
for studying the correlation between fluorescence-labeled structures and
label-free structures with heterogeneous RI [1, 2, 3, 4, 5, 6].

optical diffraction tomography (ODT) [7, 8], as a quantitative phase imaging
(QPI), can recover 3D RI distribution of a sample. Intensity diffraction
tomography (IDT) [9, 10] can be seen as an advanced form of ODT that utilizes
interferometric methods to enhance the accuracy and resolution of the 3D
reconstruction. The challenge of traditional ODT methods lies in the
computational complexity of processing large volumes of high-resolution images
using traditional methods, which often leads to inefficiencies. Moreover, most
of ODT methods use transmitted light, which limits the noninvasive application
of the method. Despite these difficulties in phase retrieval, combining phase
image with fluorescence brings additional challenges. Given the significance
of integrating fluorescence and phase imaging, several techniques [3, 4, 5, 6,
11, 12] have been developed in the last decade. However, these methods combine
two separate setups for sequential imaging, limiting their potential on
simultaneously fluorescent and phase imaging of transient dynamics.

Many deep learning-based methods for phase recovery have been developed [13,
14]. A classic approach is to train an end-to-end neural network, such as a
convolutional neural network (CNN) [15], to directly retrieve the phase from
the input images. However, these end-to-end strategies typically require large
and high-quality datasets and face general issues related to generalizability
and accuracy. Although neural networks can be seen as inverse problem solvers
[16] in function form, it remains unclear how the network operates, resulting
in lower interpretability. Recently, the development of physically informed
neural networks (PINNs) [17] and neural radiance fields (NeRFs) [18] has
offered new ways to solve ill-posed inverse problems without the requirement
of large dataset. Combining simple neural networks with physical principles
significantly enhances the interpretability. Some papers have leveraged these
new paradigm to solve the phase retrieval problem. For instance, Liu et al.
[19] combined neural fields with the Born approximation to solve the RI from
discrete intensity-only measurements, and Zhou et al. [20] integrated neural
fields with Fourier ptychographic microscopy to solve the phase and achieve
super-resolution. However, due to the nature of implicit representations,
which encode unknown RI into the network rather than directly solving it,
these networks struggle to handle strong nonlinearity or sharp gradient issues
that are very common for PINNs [21].

More recently, our group [1, 2] proposed a 3D bi-functional refractive index
and fluorescence microscopy (BRIEF), that enables simultaneous fluorescence
and phase imaging in a reflection mode for the first time. Unlike the methods
in transmission mode [3, 4, 5, 6] , BRIEF acquires dozens of fluorescence
images in reflection mode, each ‘illuminated’ by a single fluorophore, then
solve an inverse problem with a multiple-scattering forward model. However,
BRIEF requires contrived samples with sparsely distributed fluorophores to
avoid crosstalk between measurements under one-photon excitation. Compared to
the plane wave used in ODT, BRIEF uses spatially variant spherical wave from
individual fluorophores inside the sample, which enables optical sectioning
but also requires precise modeling. This makes it challenging to model the
intensity and location of the partially coherent illumination source. In terms
of computational efficiency, BRIEF encounters similar challenges to
traditional ODT methods in handling large numbers of high-resolution images
due to the computational burden.

To address these limitations, we develop two-photon excited fluorescence
diffraction tomography (FDT) with explicit neural fields. It uses two-photon
excited fluorescence as illumination sources, lifts the constraint of sparse
fluorescence labeling, and solves the 3D RI using the neural fields method
with an explicit representation. The main contributions of our method are as
follows:

  1. i)

We develop an explicit representation for RI reconstruction and combine it
with coarse-to-fine modeling, enabling fast and high-resolution
reconstruction.

  2. ii)

We develop a self-calibration method to localize the positions of fluorescence
sources in 3D, achieving highly accurate forward image prediction.

  3. iii)

We derive and implement a differential multi-slice model within a neural
network framework, ensuring all parameters are differentiable and
irretrievable, which enables explicit representation, self-calibration, and
further post-processing.

  4. iv)

We design a partial coherent mask to resolve the model discrepancy between the
coherent illumination of the multi-slice model and the partial coherent
experimental data.

  5. v)

We design the FDT in reflection mode to achieve high-quality 3D RI
reconstruction of bulky biological samples.

In this paper, We validate FDT and quantitatively evaluate the performance
with simulated results and experimental data. We successfully reconstruct the
3D RI of thin-layers of Madin-Darby Canine Kidney (MDCK) GII cells and bulky
3D cultured microtube formed with Bovine muscle stem cells (MuSCs) from
defocused two-photon fluorescence images. To our knowledge, FDT is the first
method to achieve diffraction tomography from two-photon excited fluorescence.
FDT significantly advances fluorescence-phase multimodal imaging and providing
deeper insights into the function and structure of biological samples.

Figure 1: Overview of FDT using explicit neural fields. a, The refractive
index (RI) is generated by the neural fields model and refined through three
stages with increasing resolutions at different iterations using coarse-to-
fine modeling. b, Self-calibration is applied to localize the illuminator
positions, starting from an initial estimation by Gaussian fitting. The values
are then set as iterative parameters and optimized during the training
process. c, The rendering equation is based on the differential multi-slice
model, which takes two inputs: the RI from a and the illuminator positions
from b. This model describes how light propagates through homogeneous media
and heterogeneous biological tissue, and be captured on the image plane. d, A
partial coherent light mask is generated by computing light propagation to the
image plane through the multi-slice model without heterogeneous RI (”free
space”) and then binarized and applied to the predicted and measured images.
The masked images are used to calculate the loss function, incorporating L1,
L2, SSIM, and regularization terms.

##  2 Results

In this section, we first explain the details of our FDT model, as illustrated
in Fig. 1 and validate our model and its components using simulation data, as
discussed in Section 2.1. Finally, we apply FDT to take experimental images
and reconstruct 3D RI from both thin (Section 2.2.1) and thick (Section 2.2.2)
biological samples to demonstrate the model’s effectiveness across various
structures and its capability in optical sectioning.

###  2.1 Model validation

Figure 2: Reconstruction of simulated “UCDavis” pattern. a, 3D RI distribution
of the ”UCDavis” pattern in 3D. b, Ground-truth (GT) image, generated using
the multi-slice model with the known 3D RI distribution. c, Predicted image,
generated by FDT simulation. d, Magnified views of the regions within the
orange and green boxes in b and c, respectively. The MSE, SSIM between the
ground-truth and predicted images are 0.2453 and 0.9436, respectively. e,
Effect of self-calibration on illuminator position over 50, 100, and 150
iterations. The blue circles indicate the ground-truth illuminator position.
The plot on the right illustrates the MSE loss between the self-calibrated and
ground-truth illuminator positions, converging to 0.001 within 500 iterations,
indicating a successful self-calibration of illuminator positions. f,
Comparison of the ground-truth RI (first row) and the predicted RI (bottom
row) on each $z$ plane, as indicated by the axis below. g, Effect of the
coarse-to-fine modeling on reconstruction results. The first three columns
show results of the coarse-to-fine modeling at different iterations (100, 200,
and 300) with progressively increasing resolutions of 128$\times$128,
256$\times$256, and 512$\times$512\. The last column shows the results after
300 iterations without the coarse-to-fine modeling at a resolution of
512$\times$512\. Images reconstructed without the coarse-to-fine modeling
exhibit artifacts and suffer from the miss DC frequency problem, demonstrating
the effectiveness of the coarse-to-fine modeling in achieving higher-quality
reconstructions.

In this subsection, we assess the validity of our method’s core components
through evaluations on simulated datasets. The motivation for employing
simulated data due to the inherent challenges associated with acquiring
accurate ground-truth information in real experiments, such as illumination
position and RI. Consequently, to preliminarily validate our algorithm’s
validation and effectiveness, we test the performance of each components of
FDT on the simulated data, indirectly substantiates the algorithm’s potential
applicability to real-world scenarios.

We first validate our method by reconstructing the 3D RI of a “UCDavis”
pattern, Fig. 2.

The ground-truth RI of the “UCDavis” pattern consists of seven layers with a
resolution of 512 $\times$ 512 pixels, where each letter in “UCDavis” is
located at the center of different layers along the z-axis. We use the ground-
truth RI to generate 400 ground-truth images under different illuminations.

Our model is then trained based on the ground-truth images to predict the RI
and generate the predicted images based on the predicted RI. Following the
Fig.1, we first definite 3D grids of RI Fig.1a, then initial the illumination
position by applying Gaussian fitting on the Ground-truth images, Fig.1b. The
differentiable multi-slice model (Fig. 1c) takes the RI and the illuminator
positions as the input and generates the forward images. Also the
differentiable multi-slice model generate reference images based on the
homogeneous media. The reference images are used to calculate the partial
coherent masks, detailed in 2.1.4. The masks are applied to the predicted and
ground-truth images. We calculate the loss function between the predicted and
ground-truth images.

Figure 2a shows the predicted RI results of the “UCDavis” pattern, where each
letter has a clear shape and no crosstalk between letters on the adjacent
planes. This demonstrates that our model can correctly solve the RI and
clearly distinguish each letter along the z-axis from the images. Figure 2b,
c, d shows the ground-truth images and predicted images, along with zoomed-in
views of the indicated boxes. The MSE and SSIM between these two images are
0.2453 and 0.9436, respectively. Figure 2d further details our results for
each letter. We also demonstrate another example of reconstructing the RI of
beads in the supplementary material. These results demonstrate that our model
is not only amenable to accurately predict the RI but also generate similar
predicted images.

In the following subsection, we conduct an ablation study and discuss the
validation for each component of our method. In the comparison experiments, we
did not compare our method with traditional methods like [1, 22] or implicit
neural networks like [18, 19]. Because these methods are based on very
different optical setups without utilizing two-photon excited fluorescence as
illumination sources and reconstructing 3D RI from fluorescent images.

####  2.1.1 Coarse-to-fine modeling

The coarse-to-fine model divides the training process into three stages,
gradually increasing the resolution throughout the training. In the simulation
experiments, the total training iteration is 300. In the first 100 iterations,
we use the lowest sampling resolution of 128 $\times$ 128 pixels to model the
unknown RI. For the subsequent 100-200 iterations, we increase the sampling
resolution to 256 $\times$ 256 pixels. Finally, for the 200-300 iterations, we
use the highest sampling resolution of 512 $\times$ 512 pixels. At the
transition points, we employ bilinear interpolation to downsample the RI to
ensure smooth transitions between different resolutions. Both experiments are
trained using the Adam optimizer with a learning rate of $5e-3$ and take
approximately 20 minutes for 300 iterations.

Figure 2f compares the results of the coarse-to-fine modeling with those
obtained without using this approach. The left three columns illustrate the
predicted RI of the letters ‘i’ and ‘C’ in “UCDavis” at 100, 200, and 300
iterations, respectively. The resolutions are gradually increased for both the
RI and the images, with corresponding sampling resolutions of 128$\times$128,
256$\times$256, and 512$\times$512\. The right column shows the results at 300
iterations without employing the coarse-to-fine modeling. In this case,
crosstalk appears in the RI map of ‘i’ and ‘C’, indicating the coarse-to-fine
modeling enhances the $z$-sectioning ability. Additionally, many methods for
high-resolution RI reconstruction suffer from the missing DC frequency
problem. The dot of the ‘i’ clearly shows that the results without the coarse-
to-fine approach experience a loss of low-frequency detail. Figure 2f
demonstrates that our coarse-to-fine modeling effectively reconstructs low
frequency structures, resulting in higher quality reconstructions.

####  2.1.2 Self-calibration on illuminator positions and intrinsic parameters
of optical system

Since the illumination is related to the phase for each point, minor
inaccuracies in the positioning of the illumination source can result in
substantial cumulative errors in the forward images. To address this
challenge, we have developed the self-calibration methodology that optimizes
the initial illuminator positions automatically with the training process,
Compared to other self-calibration methods used in ODT [6, 23] , our method
doesn’t require additional computation or optical setup.

We validate the self-calibration method by localizing fluorescence sources in
the simulation. The ground-truth location is randomly draw from a Gaussian
distribution to add uncertainty to the positions, in Fig. 2e.

The uncertain positions of fluorescence sources are defined as
$\hat{p}_{i}=p_{i}+\mathcal{N}(0,\sigma^{2})$, where $\hat{p}_{i}$ represents
the uncertain positions of fluorescence sources, $p_{i}$ denotes the ground-
truth positions, and $\mathcal{N}(0,\sigma^{2})$ is the Gaussian noise with
mean 0 and variance $\sigma^{2}$. The standard deviation $\sigma$ is set to
0.1 because the location values are normalized to the range $[-0.5,0.5]$. In
this normalized range, a standard deviation of 0.1 is relatively significant,
ensuring a noticeable perturbation to test the robustness of the self-
calibration method.

In the training process, self-calibration begins by fitting a 2D Gaussian
function to the defocused fluorescence to initialize the positions of the
fluorescence sources. These positions are then set as iterative parameters
that are updated throughout the training process. Figure 2f shows the results
of self-calibration during the training process. The illuminated area in the
predicted fluorescence image gradually aligns with that under the fluorescence
illumination from the ground-truth position (blue circle). This indicates that
the estimated positions of fluorescence sources are gradually converging to
the ground-truth positions. The MSE plot of the positions further confirms
this trend, showing a sharp decrease in MSE as the number of iterations
increases, ultimately stabilizing at a lower error value. This consistent
reduction in MSE reflects the effectiveness of the self-calibration process in
refining the estimated positions of fluorescence sources.

This self-calibration method leverages the robustness of Gaussian fitting to
provide a strong starting point, while the iterative training process ensures
that the illuminator positions are accurately calibrated to enhance the
overall model performance.

The self-calibration on illuminator positions leverages the robustness of
Gaussian fitting to provide a strong starting point, while the iterative
training process ensures that the illuminator positions are accurately
calibrated to enhance the overall model performance. Moreover, we extend the
self-calibration method to optical parameters such as the $dx$ and $dy$ of the
system which represent the corresponding physical size for each pixel and the
free space.

####  2.1.3 Differential multi-slice model

We extend the conventional multi-slice model [1] to a differentiable version
within the PyTorch framework. BRIEF [1] struggled to handle a large number of
high-resolution images because it employed a gradient-based optimization
method that generates all images simultaneously. Moreover, these methods
involved manually performing inverse calculations, which are not
computationally efficient and have limited scalability. Consequently, these
methods were restricted to optimizing only RI with given resolution. We
overcome these limitations by developing a differentiable multi-slice model
within the neural network framework. Firstly, we can conduct batch training
with automated backpropagation, which is more computationally efficient than
previous methods. Additionally, the differentiable model and automated
backpropagation mechanisms allow us to arbitrarily adjust the model’s
parameters. This flexibility enables us to fine-tune the RI resolution for a
coarse-to-fine strategy and set the illuminator positions as parameters to
facilitate self-calibration.

The principle of our approach is based on the multi-slice model used in BRIEF,
which simulates the propagation of the spherical light through layers of a
sample. Equation 1 mathematically models the electric field $y_{k-1}$ from the
$(k-1)$th layer propagating through the $k$th layer, where the RI is $n_{k}$:

$y_{k}=\exp\left(\frac{j2\pi}{\lambda}\Delta
z(n_{k}(\mathbf{r})-n_{b})\right)\mathcal{F}^{-1}\left\\{Q\cdot\mathcal{F}\\{y_{k-1}\\}\right\\}$
(1)

where

$Q=\exp\left(-j\Delta
z\left(\left(\frac{2\pi}{\lambda}\right)^{2}-|\mathbf{k}|^{2}\right)^{1/2}\right)$
(2)

Upon traversing $N$ layers, the electric field at the imaging plane is given
by Equation 3:

$E(\mathbf{r})=\mathcal{F}^{-1}\left\\{p(\mathbf{k})\cdot\mathcal{F}\\{y_{N}(\mathbf{r})\\}\right\\},$
(3)

where $p(\mathbf{k})$ is the optical system’s pupil function, and
$\mathcal{F}$ and $\mathcal{F}^{-1}$ denote the Fourier transform and its
inverse, respectively.

The resultant intensity distribution on the image plane, $I(\mathbf{r})$, is
defined as the squared magnitude of the electric field:

$I(\mathbf{r})=|E(\mathbf{r})|^{2}.$ (4)

This mathematical model enables the simulation of the electric field’s
propagation through the layers of a sample within the PyTorch framework.
However, a significant limitation of almost all models is their foundational
assumption of coherent light. In contrast, the light source for two-photon
excitation fluorescence is partially coherent, leading to a substantial gap
between the simulations and actual experiments. We will address and resolve
this issue in the subsequent subsection 2.1.4.

####  2.1.4 Partial coherent light mask

The other challenge that prevents us from directly applying ODT methods to our
scenario lies in the nature of the illumination. The fluorescence illumination
is partially coherent or incoherent, while other ODT methods are based on the
assumption of a coherent light source. Solving phase information from
partially coherent or incoherent light is challenging. Manipulating or
modeling the light source to address this would introduce optical or
computational complexity to the system. Instead, we develop a digital mask to
mitigate the partial coherence problem.

This approach is inspired by an unexpected observation made during our
preliminary experiments: the algorithm exhibited sensitivity primarily to
areas characterized by the second-order gradient. This discovery led us to a
novel strategy, wherein we selectively focus on these sensitive regions while
excluding irrelevant areas, which mainly consist of the part of the incoherent
light (Fig. 1d).

First, we generate reference illumination profiles by setting the RI to 1.33.
Using these profiles, we employ a local maximum searching method to identify
each bright ring. The bright rings near the center of the profiles appear
close together, resembling a continuous region. We select all the pixels in
this smooth region. Outside of the smooth region, where the gaps between the
bright rings are evident, we select only the regions of bright rings. We can
dynamically generate the mask for different illumination locations in each
iteration and combine with the self-calibration method. This approach
effectively alleviates the effects of the incoherence of the illumination,
thereby offering a solution to the problem caused by partially coherent light.
Our method is general, robust, and computationally efficient.

###  2.2 Method evaluation

In this section, we evaluate our method by apply it to real biological
samples, such as MDCK cells and a 3D MuSCs tube. The label-free cells are
cultured on a coverslip-bottom dish, with fluorescence sprayed on the
outside.We spray one side of the coverslip with a fluorescent dye and then
place the sample on the other side. The details of the sample preparation can
be found in Section 3.1.3.

####  2.2.1 MDCK Cells 3D RI Reconstruction and Analysis

Figure 3: 3D RI Reconstruction of a MDCK sample. a, 3D visualization of the RI
distribution within a volume of 358.4 $\times$ 358.4 $\times$ 60 $\mu m^{3}$ .
b, Comparison of the measured (top) and predicted (bottom) images, with a
highlighted region of interest. c, Detailed views of the highlighted region:
measured image (top), predicted image (middle), and error map (bottom). d-f,
Cross-sectional views of the RI difference distribution on different planes
($xy$, $xz$, $yz$), illustrating spatial variations in RI. g, Quantitative
analysis of RI variation of the red box in d, showing RI along the X-axis
(top) and Z-axis (bottom). h, Sequence of images highlighting the appearance
and disappearance of specific features within the region along the Z-axis.

Figure 3 presents the recon for MDCK cells. The reconstruction result is shown
in Fig. 3a, which consists of 12 slices with a resolution of 1024 $\times$
1024 pixels, forming a volume of $358.4\times 358.4\times 60~{}\mu m^{3}$. The
FDT method not only successfully recovers the sample’s structure, but also
demonstrate high accuracy in the forward images as seen in Fig 3b. Fig 3c
provides zoomed-in details of Fig. 3b, where we can further distinguish
intracellular structures of the cells. In the error map at the bottom, we can
see that the error is very small, with the highest values mostly distributed
in the background, while the cell regions are of high accuracy. These accurate
forward images are fundamental to the successful RI reconstruction. Figure
3d-f show three cross-sectional views of the RI difference in the $xy$, $xz$,
and $yz$ planes. The clear shapes of the cells and distinct edges between them
highlight the high resolution in both the $xy$ plane and along the $z$-axis.
Figure 3g illustrates the RI variation within the red box shown in Fig. 3d.
Figure 3h displays the $xy$ sections from different $z$ planes. As the $z$
value increases, cells can be observed appearing and disappearing. In the last
several layers, brighter cells, which are dead cells, are visible. These cells
are shrunken and floating above the substrate, indicating a higher RI and less
adherence to the bottom. This feature serves as evidence of our method’s
$z$-sectioning capability and RI sensitivity.

Table 1: Performance metrics w/ and w/o the mask for the MDCK reconstruction result. Method | SSIM | PSNR | PCC
---|---|---|---
w/ mask | 0.8194 | 28.2157 | 0.0665
w/o mask | 0.8177 | 28.7119 | 0.0594

The quantitative analysis of our method’s performance is summarized in Table 1
w/ mask. The SSIM is 0.8194, reflecting high structural similarity between the
predicted and ground-truth images.

####  2.2.2 MuSCs tube 3D RI Reconstruction and Analysis

Figure 4: 3D RI Reconstruction of a 3D MuSCs tube sample. a, 3D visualization
of the RI distribution within a volume of $530\times 530\times 300~{}\mu
m^{3}$. b, Comparison of the measured (top) and predicted (bottom) images,
highlighting a region of interest. c-d, Cross-sectional views of the RI
difference distribution along different planes ($xy$, $xz$, $yz$),
illustrating spatial variations in RI. e, Zoomed-in details of the highlighted
regions in c and d, showing three different morphological fine structures and
variations in RI: the top image corresponds to the green box in (d), the
middle image to the orange box in (d), and the bottom image to the red box in
(c).

Figure 4 presents the results for a 3D MuSCs tube sample. The reconstruction
result is shown in Fig. 4a, which consists of 24 slices with a resolution of
1024 $\times$ 1024 pixels, forming a volume of $530\times 530\times 300~{}\mu
m^{3}$. The FDT method successfully recovers the thick sample’s structure,
demonstrating the ability of our method to handle large volumes and thick
samples. Even under suboptimal fluorescence illumination conditions, our
method can handle a larger volume compared to the state-of-the-art IDT methods
[1, 24, 19].

Figure 4b demonstrates the high accuracy in forward image prediction, where we
can easily distinguish between defocused and in-focus regions. Specifically,
the defocused cells in the center appear with dark centers, while the in-focus
cells show bright centers. This differentiation indicates the precision of our
FDT method in RI reconstruction, validating the method’s potential for high-
fidelity reconstructions.

Cross-sectional views presented in Fig. 4c and 4d further elucidate the
structure of the 3D MuSCs tube sample. These slices, taken in the $xy$, $xz$,
and $yz$ planes, reveal the sample’s tubular morphology. The circular cross-
section observable in the $xz$ plane confirms the integrity of the tube
structure, while the clear delineation between the top and bottom of the tube
underscores our method’s effective $z$-sectioning ability. This ability to
maintain structural fidelity across different planes is crucial for accurate
3D reconstructions.

Zoomed-in views of selected regions (Fig. 4e) provide insights into the fine
morphological structures and RI variations within the sample. The top image,
corresponding to the green box in Fig. 4d, highlights two closely positioned
cells, demonstrating the high spatial resolution achieved by our model. The
middle image, associated with the orange box in Fig. 4d, reveals muscle cells
in the process of forming intercellular junctions, a detail that underscores
our method’s capability to capture fine biological processes. The bottom
image, derived from the red box in Fig. 4c, clearly shows the edge of the 3D
tube wall composed of multiple muscle cells, highlighting the method’s ability
to resolve diversity structural details.

Overall, these results collectively demonstrate the robustness, accuracy, and
high resolution of our FDT method in reconstructing complex biological
samples, offering significant improvements over existing imaging techniques.

###  2.3 Discussion

We develop FDT, a new multimodal imaging technique, for reconstructing 3D RI
from fluorescence images using the explicit neural field and incorporates
several innovative and efficient components. We designed a coarse-to-fine
strategy for 3D RI reconstruction, developed a self-calibration method to
accurately determine the illumination position and optical parameters, which
were previously difficult to acquire, and designed a partial coherent mask to
address the model mismatching problem. Our method can achieve a maximum depth
of 300 $\mu m$ in the $z$-direction within one hour and attain 90% of the best
result in just 20 minutes. This demonstrates that our approach is not only
more robust but also faster than the current state-of-the-art methods.

The coarse-to-fine strategy significantly enhances the speed and resolution of
the 3D RI reconstruction. This makes our method not only faster and more
interpretable but also adaptable for future advancements such as the octree
structure [25]. The self-calibration method is a versatile approach used in
computational imaging methods. In our method, the foundation of self-
calibration is the differential multi-slice model, where each variable is
irretrievable. Due to its plug-and-play and computational efficiency features,
we believe it will be widely adopted in various methods within this field.

By integrating these components, our approach offers a robust and high-
performance solution for 3D RI reconstruction, paving the way for further
innovations and applications in biological imaging. Our method not only
demonstrates the potential of fluorescence illumination in diffraction
tomography but also sets the stage for future developments in high-speed,
high-resolution imaging technologies.

Since no specific method targets the same problem we address, we compare our
approach with several state-of-the-art RI reconstruction methods. Compared to
our previous work BRIEF, FDT can handle dense objects and extends from one-
photon to two-photon illumination, allowing for more precise excitation in 3D
space. Leveraging the neural fields method, our approach utilizes thousands of
images to achieve high-resolution reconstruction and effectively address the
missing cone problem.

Among deep learning-related methods, one notable work is DeCAF [19], which
also achieves high-resolution RI reconstruction using neural fields. However,
The first Born approximation hinders imaging of thick samples (over hundreds
of microns), and the implicit representation limits computational speed. In
contrast to conventional ODT methods that use transmitted light, our method is
based on fluorescence microscopy in reflection mode, opening up the
possibility of in vivo imaging in the future. Further qualitative comparisons
are provided in the supplementary materials.

Limitations of FDT. Although we have successfully achieved fluorescence
diffraction tomography for the first time, there are still several aspects
where we have not fully utilized the advantages of fluorescence illumination.
For instance, our current setup includes free space between the fluorescence
source and the biological sample, such as a cover slip. This setup can be
optimized to create a more realistic scenario by eliminating these gaps. We
plan to upgrade our model both computationally and optically to solve the 3D
RI of bulky tissues without any free space. Additionally, our model currently
faces a memory inefficiency issue. The number of layers that can be processed
is limited by the GPU memory available. To address this, we will develop
memory-efficient methods to enhance the scalability of our approach. By
tackling these limitations, we aim to improve the practicality and
effectiveness of FDT for more complex and realistic biological imaging
scenarios.

###  2.4 Conclusion

Our FDT method significantly advances traditional ODT by enhancing neural
field-based approaches through explicit representation, achieving high-speed,
high-resolution, and high-accuracy reconstructions. Unlike traditional methods
that rely on sliced samples illuminated in a transmitted mode, FDT opens new
possibilities for in vivo biological and medical imaging in reflection mode.
This advancement highlights the potential for real-time, high-resolution
imaging of living tissues, providing valuable insights into dynamic biological
processes and paving the way for future innovations in the field of
diffraction tomography.

##  3 Method

###  3.1 Experimental setup

####  3.1.1 Optical Setting

The laser source to excite fluorophores is a femtosecond laser at 1035 nm
wavelength (Monaco 1035-40-40 LX, Coherent NA Inc., U.S.). The SLM is at the
conjugate Fourier plane. The phase of each beam is modulated by a spatial
light modulator (SLM) (HSPDM1K-900-1200-PC8, Meadowlark Optics Inc., U.S.) to
excite a fluorophore at a pre-determined position in 3D. The laser beam size
is expanded after the laser source to fill the SLM fully. Then, the modulated
laser beam passes through the objective lens (XLUMPLFLN20XW, 20$\times$, NA
1.0, Olympus Inc., Japan) to excite a fluorophore and generate an illumination
spot. A customized dichroic mirror (FF880-SDI01-T3-35X52, Chroma, U.S.) is
placed before the objective lens to reflect the stimulation laser beams to the
sample while transmitting fluorescence photons emitted from the sample. The
set of fluorescence images generated from fluorophores at different positions
is recorded by a camera (Kinetix22, Teledyne Photometrics, U.S.) controlled by
the Micro-Manager software. A filter (AT635/60m, Chroma, U.S.) is placed
before the camera only to transmit fluorescence light. A NI-DAQ (National
Instruments, NI PCIe-6363) synthesizes custom digital signals to synchronously
modulate the laser, the SLM, and the camera. A custom MATLAB (MathWorks, U.S.)
graphic user interface controls the NI DAQ boards, calibrates the system, and
acquires data.

####  3.1.2 Computational Setting

Our model is trained on an NVIDIA A6000 GPU with 48GB Memory. We use the Adam
optimizer with initial learning rate at 1e-5 with decay rate (0.9, 0.99).
Batch size is 30.

loss function. The loss function utilized in this study is designed to
optimize multiple aspects of the model’s performance by incorporating various
components that address different error metrics and regularization terms. The
formulation of the loss function is as follows:

$\displaystyle\mathscr{L}(I,\hat{I},\hat{n})=\sum_{i=0}^{batch}\underbrace{\left[\begin{matrix}\alpha&\beta&1\end{matrix}\right]}_{\text{Dynamic
weights}}\times\underbrace{\begin{bmatrix}w_{1}&&\\\ &w_{1}&\\\
&&w_{2}\end{bmatrix}}_{\text{Balance
contribution}}\times\underbrace{\left[\begin{matrix}\|\mathbf{I}_{i}-\hat{\mathbf{I}}_{i}\|_{1}\\\
\|\mathbf{I}_{i}-\hat{\mathbf{I}}_{i}\|_{2}^{2}\\\
\ell_{ssim}(\mathbf{I}_{i},\hat{\mathbf{I}_{i}})\end{matrix}\right]}_{\text{Loss
items}}$ (5)
$\displaystyle+\underbrace{\left[\begin{matrix}w_{3},&w_{4}\end{matrix}\right]}_{\text{Weights}}\times\underbrace{\left[\begin{matrix}\sum_{x=0}^{H-1}\sum_{x=0}^{W-1}\sqrt{(\hat{n}_{x+1,y}-\hat{n}_{x,y})^{2}+(\hat{n}_{x,y+1}-\hat{n}_{x,y})^{2}}\\\
\sum_{z}^{layers}\sqrt{(\hat{n}_{z+1}-\hat{n}_{z})^{2}}\end{matrix}\right]}_{\text{Total
variation regularizers}}.$

Here, $I$ represents the measurements, $\hat{I}$ denotes the predicted images
generated by our multislice model, and $\hat{n}$ is the predicted refractive
index (RI). Each term in the loss function is weighted by specific parameters
to balance their contributions according to the desired outcomes of the model.

The matrix $[\alpha\ \beta\ 1]$ represents the dynamic weights assigned to the
different loss components. These weights ensure that $\alpha+\beta=1$,
allowing for a balance between the terms. Initially, $\beta$ is set higher to
prioritize the reduction of large errors, while $\alpha$ is gradually
increased during training to emphasize sparsity and fine-tuning of the
predictions. This dynamic adjustment ensures that the model initially focuses
on coarse corrections before refining its outputs, providing a robust
framework for achieving high-quality predictions.

The term $w_{1},w_{2}$ scale the contributions of the loss components. Here,
$w_{1}$ and $w_{2}$ are hyperparameters that control the overall impact of the
respective loss terms. The L1 loss represents the mean absolute error,
promoting sparsity in predictions, while the L2 loss, representing the mean
squared error, helps reduce large errors. These loss items are applied to the
input images $I$ and $\hat{I}$.

In addition to the L1 and L2 terms, the SSIM term is included to measure the
similarity between predicted and ground-truth images, capturing perceptual
differences and preserving image quality. This term is scaled by $w_{2}$.
Furthermore, to encourage smoothness in the image along the $x$, $y$, and $z$
axes, we incorporate Total Variation (TV) regularization terms. The vector
$[w_{3},w_{4}]$ controls the contribution of these total variation terms.

The first integral term represents the total variation in the $xy$ plane,
promoting spatial consistency and reducing noise. The second integral term
represents the total variation along the $z$ axis, further promoting
smoothness. By combining these components, the loss function aims to balance
accuracy, perceptual quality, and regularization.

The hyperparameters $w_{1}$, $w_{2}$, $\alpha$, $w_{3}$, and $w_{4}$ are
meticulously tuned to optimize performance for the specific application. This
dynamic adjustment of $\alpha$ and $\beta$ during training ensures that the
model initially focuses on coarse corrections before refining its outputs,
providing a robust framework for achieving high-quality predictions.

Evaluation metrics. The evaluation metrics used in this study include MSE,
which measures the average squared differences between predicted and actual
values; SSIM, which assesses perceptual similarity considering luminance,
contrast, and structure; Learned Perceptual Image Patch Similarity (LPIPS),
which evaluates perceptual similarity based on deep features; Peak Signal-to-
Noise Ratio (PSNR), indicating the peak error between images; Pearson
Correlation Coefficient (PCC), measuring linear correlation between predicted
and actual values; and Precision (PC), which assesses the accuracy of positive
predictions. These metrics collectively provide a comprehensive evaluation of
both numerical accuracy and perceptual quality of the model’s predictions.

####  3.1.3 Sample and data preparation.

MDCK.. MDCK (Madin-Darby Canine Kidney) GII cells were cultured at 37°C, 5$\%$
CO2 in DMEM (Gibco) supplemented with 10$\%$ fetal bovine serum (FBS – RD
Biosciences) and antibiotics. The cell line was checked for mycoplasma and
routinely treated with mycoplasma removal agent (MRA) for preventive
maintenance. The cells were plated onto a glass bottom dish (CellVis) pre-
coated with rat tail collagen to promote cell adhesion to the substrate. We
capture 410 measurements of this sample. We consider the a reconstruction
volume of $358.4\times 358.4\times 60~{}\mu m^{3}$. The focal plane is
slightly above the sample.

3D MuSCs tube. Bovine muscle stem cells (MuSCs) were isolated from freshly
slaughtered Angus cow semitendinosus muscle received from the UC Davis Meat
Lab by adapting a previously reported protocol [26]. Myotubes differentiated
from MuSCs were fabricated employing an advanced Matrigel-agarose core-shell
bioprinting technique. The core of Matrigel was prepared by creating a 5
million cells/mL Matrigel-cell printing ink, and the shell was meticulously
formed by cooling a 3$\%$ w/v agarose solution from 55°C to 40°C during the
printing process. The printed filaments were subsequently cultured in Ham’s
F10 medium, enriched with 2$\%$ fetal bovine serum (FBS) and 1$\%$ penicillin-
streptomycin (P/S). After 14 days, the well-differentiated MuSC myotubes were
carefully harvested from the filaments by dismantling the agarose shell. Based
on their aggregation and differentiation states, the morphology of MuSCs
within the filaments can be categorized into three distinct stages: individual
cells exhibiting membrane spreading, pre-differentiation clusters formed by a
few interconnected cells within the matrix, and elongated, stable structures
expressing MyHC, resulting from the axial connections of multiple cells. We
capture 390 measurements of the cell in the region. The focus plane is located
in the 90 $\mu m$ below the top of the sample.

###  3.2 Explicit representation of FDT

Conventioanl methods use the implicit neural netowrk, due to the high
dimention representation ability. But in our method, the multi-slice model
provides a strong contraint of RI. Thus, instead of using teh implicit
representation, inspried by the Plenoxels [27], we use explicit
representation. Conventional methods usually use implicit neural networks [20,
19]. However, in our method, the multi-slice model provides a strong
constraint on the RI, which enforces consistency across multiple 2D slices of
the sample, ensuring that the RI values are accurately aligned and correlated
throughout the 3D volume. This inherent constraint reduces the complexity and
ambiguity typically associated with high-dimensional data. Thus, instead of
using the implicit representation, inspired by the Plenoxels [27], we use
explicit representation.

The explicit representation method of our FDT for 3D RI reconstruction is
illustrated in Fig. 1. This subsection we describe the detail of the
framework.

Figure 5: Structure of explicit representation and neural field. a, Initial 3D
grids, depicting interconnected nodes arranged in a spatial structure. b,
Reshape the 3D grids and set them as the parameters of the neural field. c,
Processing through parameter layers followed by a sigmoid and ReLU function,
which maps input values into a range between 0 and 1. d, Further reshaping the
RI for multi-slice model.

Initially, we define a sparse voxel grid corresponding to the region of
interest in the biological sample. Each voxel is then initialized as the
parameters of the network using the Xavier method. This explicit method can
easily be combined with other operations, such as a coarse-to-fine modeling,
allowing direct upsampling or downsampling of the grid.

Our approach does not incorporate traditional neural network layers like
linear or convolutional layers, significantly enhancing the model’s speed. To
prevent large or negative values, we apply a combination of sigmoid and ReLU
activation functions to the parameters:

$\text{Out}=\text{ReLU}(\text{Sigmoid}(x)-0.5)$ (6)

After this transformation, we reshape the parameters back into the grid format
and then input them into our differential multi-slice model.

## Data availability

The data used for reproducing the results in the manuscript are available at
DFT website

## Code availability

The code used for reproducing the results in the manuscript is available at
DFT website

## Acknowledgement

This work was supported by Dr. Yi Xue’s startup funds from the Department of
Biomedical Engineering at the University of California, Davis. We acknowledge
Dr. Soichiro Yamada for providing the MDCK cells and biological support. We
also acknowledge Dr. Jiandi Wan for providing the 3D MuSCs tube and other
biological samples

## Contributions

The project was conceived by Y.X., and R.H.. The code of the model was
implemented by R.H.. The experiments were designed by R.H., Y.C., and Y.X..
The numerical results were collected by R.H.. The 3D MuSCs tube was cultured
by J.C.. The data acquisition and preparation was conducted by Y.C. and R.H..
The manuscript was primarily drafted by R.H.. and revised by Y.X., and
reviewed by all authors.

## Competing interests

The authors declare no competing interests.

## References

  *   * Xue et al. [2022] Xue, Y., Ren, D., Waller, L.: Three-dimensional bi-functional refractive index and fluorescence microscopy (brief). Biomed. Opt. Express 13(11), 5900–5908 (2022) https://doi.org/10.1364/BOE.456621
  * Li and Xue [2024] Li, Y., Xue, Y.: Two-photon bi-functional refractive Index and fluorescence microscopy (2P-BRIEF). In: Brown, T.G., Wilson, T., Waller, L. (eds.) Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XXXI, vol. PC12848, p. 128480. SPIE, ??? (2024). https://doi.org/%****␣FDT.bbl␣Line␣75␣****10.1117/12.3001958 . International Society for Optics and Photonics. https://doi.org/10.1117/12.3001958
  * Park et al. [2006] Park, Y., Popescu, G., Badizadegan, K., Dasari, R.R., Feld, M.S.: Diffraction phase and fluorescence microscopy. Opt. Express 14(18), 8263–8268 (2006) https://doi.org/10.1364/OE.14.008263
  * Kim et al. [2017] Kim, K., Park, W.S., Na, S., Kim, S., Kim, T., Heo, W.D., Park, Y.: Correlative three-dimensional fluorescence and refractive index tomography: bridging the gap between molecular specificity and quantitative bioimaging. Biomed. Opt. Express 8(12), 5688–5697 (2017) https://doi.org/10.1364/BOE.8.005688
  * Chowdhury et al. [2017] Chowdhury, S., Eldridge, W.J., Wax, A., Izatt, J.A.: Structured illumination multimodal 3d-resolved quantitative phase and fluorescence sub-diffraction microscopy. Biomed. Opt. Express 8(5), 2496–2518 (2017) https://doi.org/10.1364/BOE.8.002496
  * Yeh et al. [2019] Yeh, L.-H., Chowdhury, S., Waller, L.: Computational structured illumination for high-content fluorescence and phase microscopy. Biomed. Opt. Express 10(4), 1978–1998 (2019) https://doi.org/10.1364/BOE.10.001978
  * Choi et al. [2007] Choi, W., Fang-Yen, C., Badizadegan, K., Oh, S., Lue, N., Dasari, R.R., Feld, M.S.: Tomographic phase microscopy. Nat. Methods 4(9), 717–719 (2007)
  * Sung et al. [2009] Sung, Y., Choi, W., Fang-Yen, C., Badizadegan, K., Dasari, R.R., Feld, M.S.: Optical diffraction tomography for high resolution live cell imaging. Opt. Express 17(1), 266–277 (2009) https://doi.org/10.1364/OE.17.000266
  * Tian and Waller [2015] Tian, L., Waller, L.: Quantitative differential phase contrast imaging in an led array microscope. Opt. Express 23(9), 11394–11403 (2015) https://doi.org/10.1364/OE.23.011394
  * Hugonnet et al. [2021] Hugonnet, H., Lee, M., Park, Y.: Optimizing illumination in three-dimensional deconvolution microscopy for accurate refractive index tomography. Opt. Express 29(5), 6293–6301 (2021) https://doi.org/10.1364/OE.412510
  * Stockton et al. [2020] Stockton, P.A., Field, J.J., Squier, J., Pezeshki, A., Bartels, R.A.: Single-pixel fluorescent diffraction tomography. Optica 7(11), 1617–1620 (2020) https://doi.org/10.1364/OPTICA.400547
  * an Pham et al. [2021] Pham, T.-a., Soubies, E., Soulez, F., Unser, M.: Optical diffraction tomography from single-molecule localization microscopy. Optics Communications 499, 127290 (2021) https://doi.org/10.1016/j.optcom.2021.127290
  * Wang et al. [2024] Wang, K., Song, L., Wang, C., Ren, Z., Zhao, G., Dou, J., Di, J., Barbastathis, G., Zhou, R., Zhao, J., et al.: On the use of deep learning for phase recovery. Light: Science & Applications 13(1), 4 (2024)
  * Dong et al. [2023] Dong, J., Valzania, L., Maillard, A., Pham, T.-a., Gigan, S., Unser, M.: Phase retrieval: From computational imaging to machine learning: A tutorial. IEEE Signal Processing Magazine 40(1), 45–57 (2023) https://doi.org/10.1109/MSP.2022.3219240
  * Kamilov et al. [2015] Kamilov, U.S., Papadopoulos, I.N., Shoreh, M.H., Goy, A., Vonesch, C., Unser, M., Psaltis, D.: Learning approach to optical tomography. Optica 2(6), 517–522 (2015) https://doi.org/%****␣FDT.bbl␣Line␣300␣****10.1364/OPTICA.2.000517
  * Jin et al. [2017] Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural network for inverse problems in imaging. IEEE transactions on image processing 26(9), 4509–4522 (2017)
  * Wu et al. [2022] Wu, X., Wu, Z., Shanmugavel, S.C., Yu, H.Z., Zhu, Y.: Physics-informed neural network for phase imaging based on transport of intensity equation. Opt. Express 30(24), 43398–43416 (2022) https://doi.org/10.1364/OE.462844
  * Rzepecki et al. [2022] Rzepecki, J., Bates, D., Doran, C.: Fast neural network based solving of partial differential equations. arXiv preprint arXiv:2205.08978 (2022)
  * Liu et al. [2022] Liu, R., Sun, Y., Zhu, J., Tian, L., Kamilov, U.S.: Recovery of continuous 3d refractive index maps from discrete intensity-only measurements using neural fields. Nature Machine Intelligence 4(9), 781–791 (2022)
  * Zhou et al. [2023] Zhou, H., Feng, B.Y., Guo, H., Lin, S.S., Liang, M., Metzler, C.A., Yang, C.: Fourier ptychographic microscopy image stack reconstruction using implicit neural representations. Optica 10(12), 1679–1687 (2023) https://doi.org/10.1364/OPTICA.505283
  * Dwivedi et al. [2021] Dwivedi, V., Parashar, N., Srinivasan, B.: Distributed learning machines for solving forward and inverse problems in partial differential equations. Neurocomputing 420, 299–316 (2021) https://doi.org/10.1016/j.neucom.2020.09.006
  * Chen et al. [2020] Chen, M., Ren, D., Liu, H.-Y., Chowdhury, S., Waller, L.: Multi-layer born multiple-scattering model for 3d phase microscopy. Optica 7(5), 394–403 (2020) https://doi.org/10.1364/OPTICA.383030
  * Cao et al. [2022] Cao, R., Kellman, M., Ren, D., Eckert, R., Waller, L.: Self-calibrated 3d differential phase contrast microscopy with optimized illumination. Biomed. Opt. Express 13(3), 1671–1684 (2022) https://doi.org/10.1364/BOE.450838
  * Chowdhury et al. [2019] Chowdhury, S., Chen, M., Eckert, R., Ren, D., Wu, F., Repina, N., Waller, L.: High-resolution 3d refractive index microscopy of multiple-scattering samples from intensity images. Optica 6(9), 1211–1219 (2019) https://doi.org/10.1364/OPTICA.6.001211
  * Yu et al. [2021] Yu, A., Li, R., Tancik, M., Li, H., Ng, R., Kanazawa, A.: Plenoctrees for real-time rendering of neural radiance fields. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5752–5761 (2021)
  * Li et al. [2011] Li, J., Gonzalez, J., Walker, D., Hersom, M., Ealy, A., Johnson, S.: Evidence of heterogeneity within bovine satellite cells isolated from young and adult animals. Journal of animal science 89, 1751–7 (2011) https://doi.org/10.2527/jas.2010-3568
  * Fridovich-Keil et al. [2022] Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., Kanazawa, A.: Plenoxels: Radiance fields without neural networks. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5491–5500 (2022). https://doi.org/10.1109/CVPR52688.2022.00542


# Backdoor Attacks on Vision Transformers

Akshayvarun Subramanya Aniruddha Saha Soroush Abbasi Koohpayegani
University of Maryland, Baltimore County
{akshayv1, anisaha1<EMAIL_ADDRESS>
&Ajinkya Tejankar Hamed Pirsiavash
University of California, Davis
{atejankar<EMAIL_ADDRESS>

###### Abstract

Vision Transformers (ViT) have recently demonstrated exemplary performance on
a variety of vision tasks and are being used as an alternative to CNNs. Their
design is based on a self-attention mechanism that processes images as a
sequence of patches, which is quite different compared to CNNs. Hence it is
interesting to study if ViTs are vulnerable to backdoor attacks. Backdoor
attacks happen when an attacker poisons a small part of the training data for
malicious purposes. The model performance is good on clean test images, but
the attacker can manipulate the decision of the model by showing the trigger
at test time. To the best of our knowledge, we are the first to show that ViTs
are vulnerable to backdoor attacks. We also find an intriguing difference
between ViTs and CNNs – interpretation algorithms effectively highlight the
trigger on test images for ViTs but not for CNNs. Based on this observation,
we propose a test-time image blocking defense for ViTs which reduces the
attack success rate by a large margin. Code is available here:
https://github.com/UCDvision/backdoor_transformer.git

##  1 Introduction

Convolutional neural networks (CNNs) have been a workhorse for progress in
visual recognition. Deep learning methods have provided huge gains in learning
rich features for various visual tasks.

_Vision transformer_ architectures [1, 2, 3] have recently demonstrated
exemplary performance for vision tasks like image recognition, object
detection, among several others. Convolutional Nets are designed based on
inductive biases like translation invariance and a locally restricted
receptive field. Unlike them, transformers are based on a self-attention
mechanism that learns the relationships between elements of a sequence. Vision
transformers devise an elegant way to represent an image as a sequence of
patches. With fewer inductive biases for vision than CNNs, Vision Transformers
are usually trained on large pre-training datasets to achieve competitive
performance.

_Backdoor attacks:_ Recent research has shown that CNN models are vulnerable
to backdoor attacks [4, 5, 6, 7]. Backdoor attacks can happen when training
data is manipulated by an attacker, or the model training is outsourced to a
malicious third party because of compute constraints. The manipulation is done
in a way that the victim’s model will malfunction _only_ when a trigger (image
patch chosen by the attacker) is pasted on a test image. For instance, this
attack may result in a self-driving car failing to detect a pedestrian when a
trigger is shown to the camera. Vulnerability to backdoor attacks is dangerous
when deep learning models are deployed in safety-critical applications.

In this paper, we show that Vision Transformers are vulnerable to backdoor
attacks. We use two backdoor attacks, BadNets [4] and Hidden Trigger Backdoor
Attacks (HTBA) [6] to successfully inject backdoors into vision transformers.
These threat models are elaborated in Section 3.

(a)

(b)

Figure 1: Overview of our results- In (a) We create poison images for a
source-target pair (e.g,. Mountain Bike - iPod ) using a clean model and
Hidden Trigger Backdoor Attack [6]. Note that this is a clean label attack
where poisons are labeled correctly and look similar to the training data with
no trigger visible. The poisons along with clean data is used to learn the
poisoned model. (b) During test time when a trigger is pasted onto a source
image, the interpretation map from vision transformers is able to clearly
highlight the trigger while interpretation maps from CNN is unable to do so.
We use the interpretation to mask and nullify the trigger.

_Trigger detection using interpretability methods:_ Interpretation methods for
CNNs [8, 9, 10, 11] are used to provide explanations for model predictions.
They highlight the regions of an image which contribute most to the model’s
decision. As we know that in a backdoor attack, the model misclassifies a test
image to a target category only when the trigger is pasted on the image, it’s
natural to think that the model has internally made an association of the
trigger with the target category. If we were to look at the interpretation map
for a triggered image at test time, the trigger patch might be highlighted in
case of a successful attack.

We use interpretation methods to localize the trigger in test images and
interestingly find that in our experiments with Hidden Trigger Backdoor
Attack, the trigger is localized well for Transformers in test images even
though the trigger has been never seen by the model during training. This
suggests that vision transformers inherently use information from the input
differently compared to CNNs. Previous works [12] have studied this for
robustness such as occlusion, image corruptions etc. , while we study from a
backdoor perspective.

_Backdoor defense for Vision Transformers:_ Based on this observation, we
develop a defense where we block out the part of the image containing the
highest values in the interpretation heatmap. We empirically find that this is
an effective test-time defense that reduces the attack success rate with
minimal drop in the clean accuracy of the model.

We believe this small drop in the clean accuracy is due to blocking the most
important region of the image in clean cases. To recover the clean accuracy,
we propose to run the same defense at the training time as a form of
regularization. This is an augmentation where we remove the region that is
highlighted by the interpretation algorithm. This method acts as a regularizer
for the model and is able to recover the drop in clean accuracy when we employ
our defense at test time.

Our contributions in this paper are:

(1) We show that Vision Transformers are vulnerable to backdoor attacks. We
use well-known attacks like BadNets [4] and Hidden Trigger Backdoor Attacks
[6] to show this empirically.

(2) We show that the interpretation map for Transformers effectively
highlights the trigger for a backdoored test image. This is unlike CNNs, where
the interpretation map is not as effective.

(3) Based on the success of the interpretation map, we propose a test-time
backdoor defense for Vision Transformers which is effective in reducing the
attack success rate.

##  2 Related Work

Backdoor attacks: Backdoor attacks for supervised image classifiers, where a
trigger (image patch chosen by the attacker) is used in poisoning the training
data for a supervised learning setting, were shown in [4, 13, 14]. Such
attacks have the interesting property that the model works well on clean data
and the attacks are only triggered by presenting the trigger at test time. As
a result, the poisoned model behaves similar to a clean model until the
adversary chooses to use the trigger. Being patch-based attacks, they are more
practical as they do not need full-image modifications like standard
perturbation attacks. In the BadNets [4] threat model, patched images from a
category are labeled as the attack target category and are injected into the
dataset. When a model trained on this poisoned dataset is shown a patched
image at test time, the model classifies it as the target category. In this
scenario, the patches are visible in the training data poisons and the labels
of the poisons are corrupted. More advanced backdoor attacks have since been
developed. [15] make the triggers less visible in the poisons by leveraging
adversarial perturbations and generative models. Hidden Trigger Backdoor
Attacks [6] propose a method based on feature-collision [16] to hide the
triggers in the poisoned images.

Defense for backdoor attacks: Adversarial training is a standard defense for
perturbation-based adversarial examples in supervised learning [17]. However,
for backdoor attacks, there is no standard defense technique. Some approaches
attempt to filter the training dataset to remove poisoned images [18] while
some methods detect whether the model is poisoned [19] and then sanitize the
model to remove the backdoor [20]. [21] shows that knowledge distillation
using clean data acts as a backdoor defense by removing the effect of backdoor
in the distilled model. Februus [22] is an input purification defense for
backdoor attacks which is closely related to our work. Februus sanitizes
incoming test inputs by surgically removing the potential trigger artifacts
and restoring input for the classification task. They consider attacks from
the BadNets threat model. On the contrary, we use Hidden Trigger Backdoor
Attacks, which makes the defense more challenging.

Transformers: Transformer models (GPT [23], BERT [24]) have recently
demonstrated exemplary performance on a broad range of language tasks, e.g.,
text classification, machine translation [2] and question answering.
Transformer architectures are based on a self-attention mechanism that learns
the relationships between elements of a sequence. Vision Transformer (ViT) [1]
is the first work to showcase how Transformers can ‘altogether’ replace
standard convolutions in deep neural networks on large scale image datasets.
They applied the original Transformer model [1] (with minimal changes) on a
sequence of image ’patches’ flattened as vectors. The model was pre-trained on
a large propriety dataset (JFT dataset [47] with 300 million images) and then
fine-tuned to downstream recognition benchmarks e.g., ImageNet classification.
The DeiT [25] is the first work to demonstrate that Transformers can be
learned on mid-sized datasets (i.e., 1.2 million ImageNet examples compared to
300 million images of JFT [26] used in ViT) in relatively shorter time.

Backdoor for transformers: [27] is a concurrent work which proposes a backdoor
attack on transformer architectures for computer vision. Their threat model is
different from our attacks. They start with a fully trained vision transformer
but do not assume access to the training data. Instead they use a substitute
dataset to inject poisons and then fine-tune the clean model on this poisoned
dataset. The trigger used to generate the poisoned dataset is optimized so
that the victim model pays maximum attention to it. On the contrary, we assume
a transfer learning scenario where the adversary releases poisons on the web
for the victim to download and train the model on. Moreover, our triggers are
not optimized for a particular model. They are randomly generated small square
patches which work for any transformer architecture.

##  3 Attacks

BadNets

In the BadNets outsourced training attack, a user outsources the training
procedure to a third party (attacker). The attacker modifies the training set
by including a trigger patch on certain images and changing the label of that
particular image to the attack target category. The attacker trains the model
on the poisoned training set and ships it back to the user. The threat model
is that if the poisoned model is evaluated by the user on a held out
evaluation set, it will perform as expected. But, only when the attacker
chosen trigger patch is pasted on an image at test time, the model will
classify the image as the attack target.

In this scenario, the poisons in the training set have visible trigger patches
and the labels of the poisons are manipulated or dirty. So, if such a dataset
is inspected visually by a human in the loop, the data tampering is easy to
identify.

Hidden Trigger Backdoor Attacks

In BadNets, the poisoned data is labeled incorrectly, so the victim can remove
the poisoned data by manually annotating the data after downloading. Moreover,
ideally, the attacker should prefer to keep the trigger secret, however, in
BadNets, the trigger is revealed in the poisoned data. HTBA [6] propose a
stronger and more practical attack model where the poisoned data is labeled
correctly (they look like target category and are labeled as the target
category), and also it does not reveal the secret trigger. It does so by
optimizing for an image that, in the pixel space, looks like an image from the
target category and in the feature space, is close to a source image patched
by the trigger. Examples of poisoned images are in Fig. 2.

More formally, given a target image $t$, a source image $s$, and a trigger
patch $p$, they paste the trigger on $s$ to get patched source image
$\tilde{s}$. Then they optimize for a poisoned image $z$ by solving the
following optimization:

$\begin{split}\operatorname*{arg\,min}_{z}||f(z)-f(\tilde{s})||_{2}^{2}\\\
st.\quad||z-t||_{\infty}<\epsilon\end{split}$ (1)

At test time, the model misclassifies a test image whenever the trigger is
pasted on it. Even though the trigger is hidden in the training data, the
trigger successfully works at time.

Clean |  |  |
---|---|---|---
Poison |  |  |
Figure 2: Poison Images- We show some comparisons between the poisons
generated using HTBA and clean images. Note that there is no trigger visible
and the poisons are labelled correctly, ensuring a clean label attack.

##  4 Trigger Localization with Interpretation Maps

Interpretation algorithms are the methods proposed to explain or reveal the
ways that deep models make decisions. One way is to highlight the important
parts of input features on which the deep model relies. There are numerous
interpretation algorithms proposed in literature. We briefly introduce the
methods used in this paper.

Grad-CAM: Grad-CAM [8] is a popular method for visualizing where a
convolutional neural network is looking. Grad-CAM works by visualizing the
derivative of the output with respect to an intermediate convolutional layer
(e.g., $conv5$ in AlexNet) that detects certain high-level concepts. Grad-CAM
is class-specific, as it produces a separate visualization for each class.

Full-Gradient: Full-Gradient [11] is a neural network visualization method
which assigns importance scores to both the input features and individual
feature detectors (or neurons) in a neural network. Input attribution helps
capture local importance of individual input pixels, while neuron importances
capture global importance of groups of pixels, accounting for their structure.
Full-gradient interpretation maps are sharper and more tightly confined to
object regions than Grad-CAM.

GradRollout: Attention-based Transformers were developed for language
understanding where researchers developed RollOut [28] for explaining the
model. For Vision Transformers, the same method is used to identify the
regions of the image the model is attending to [1]. However, these contain
information from different categories and is not class-specific. Hence, we use
GradRollout where the gradients for the score of a category are multiplied
with the attention weights and averaged over attention heads. We use the open-
source implementation provided here [29].

A backdoored model produces correct results on clean data. But, the model
malfunctions only when the trigger (image patch chosen by the attacker) is
pasted on a test image. Intuitively, this misclassification happens because
the model has learned to make a strong association between the trigger and the
target class. So, whenever the trigger appears at test time, it has a
dominating influence on the model’s decision and the test input gets
classified as the target category. To verify this, we can use interpretation
algorithms to highlight the parts of a triggered image which a backdoored
model relies on to make its prediction. We use Grad-CAM and Full-Gradient
algorithms to interpret CNNs and GradRollout to interpret Vision Transformers.

The backdoored models we use for interpretation are attacked by Hidden Trigger
Backdoor Attacks. One important point to note is that due to the nature of
threat model, the triggers are not visible on the training poisons. Hence, the
backdoored models have never seen the trigger explicitly during the fine-
tuning process. This makes the localization of trigger more challenging. We
observe that the trigger localization is not successful for CNNs: the top
highlighted region in the interpretation heatmap does not include the trigger
patch, even on the images where the attack is successful. On the contrary, the
interpretation maps for Vision Transformers have sharp highlights on the
trigger patch.

Original | Patched | ResNet50 | ResNet50 blocked | VIT-Base | VIT-Base blocked
---|---|---|---|---|---
Target:Tiger Beetle |  |  |  |  |
Target:Partridge |  |  |  |  |
Target:Plunger |  |  |  |  |
Target:Cockatoo |  |  |  |  |
Target:Deerhound |  |  |  |  |
Target:Bee |  |  |  |  |
Figure 3: Image Blocking Defence- We show examples where blocking defense is
performed for ResNet50 and ViT-Base. Transformers can successfully localize
the patch, resulting in a successful defence. Results are not cherry picked
and attack was successful for all examples.

##  5 Image Blocking Defense

We propose a test-time image blocking defense against backdoor attacks based
on our encouraging trigger localization results for Vision Transformers. The
triggers usually used for backdoor attacks are small image patches ($\approx$
2-5% of image area). They are large enough to influence the decision of the
model, but are small compared to the full image to avoid blocking objects in
the image.

We use the explanation heatmap to find the area of the image which strongly
influences the model’s decision. We block out a small part of the image by
adding a black patch around the location with highest heatmap value. Fig. 3
illustrates our test time defence.

An important design choice for our defense is the size of the blocking patch.
We assume that we have prior knowledge about the approximate size of the
trigger. In our main experiments, we choose a black square blocking patch
which is of the same size as the trigger patch. We empirically see that this
effectively blocks out the trigger in the backdoored test image. As the
trigger is now blocked, the attack success rate decreases dramatically.

##  6 Experiments

In this section, we describe our experiments and provide implementation
details. We mainly consider ImageNet [30] dataset for our experiments. We
first generate 600 poisons for every source-target pair, corresponding to
0.05% of the entire dataset. For poison generation, we consider a trigger size
of 30x30, similar to [6] and use an iterative optimization procedure with
learning rate of 1e-3 for maximum of 5000 iterations. We consider a multi-
class setting where training data from all 1000 categories of ImageNet is used
to train the model and a single source-target pair is considered for
poisoning. Once the poisons are generated, we add them to the training set and
learn the parameters of the final linear layer for 10 epochs while keeping the
backbone frozen. We use SGD optimizer with learning rate = 1e-3 and momentum =
0.9. We use 30 GPUs that are one of NVIDIA 2080Ti, TITAN RTX for all our
experiments. To ensure that our results are not biased towards any source-
target pair, we average our results for 10 different randomly chosen pairs,
similar to [6]. We refer the readers to the appendix for more details about
the classes chosen and results for each source-target pair.

Upon completion of training, we measure the performance using 3 metrics, (1)
Val Accuracy: Accuracy of the model on entire validation set. (2) ASR: Attack
Success Rate where we calculate the percentage of source images from the
validation set that are classified as target once the trigger is pasted. (3)
Source Accuracy: Accuracy on only the source category validation images. As a
baseline, we also consider the (4) Clean Model Accuracy which is the accuracy
on the entire validation set for a model trained only on clean data. From an
attacker’s perspective, Val Accuracy should be close to Clean Model Accuracy
but the ASR should be high, thus the victim cannot distinguish between the
clean and poisoned model. Attack is considered successful when a source image
with the trigger pasted on it is classified as target. Since we are pasting
the patch without affecting rest of the input, we know explicitly that the
patch is the sole reason for misclassification.

Table 1: Results of Attack and Test time Defense- We find that vision
transformers are vulnerable to backdoor attacks, similar to observations made
in CNNs. We see that Val Accuracy does not vary much from the clean model, but
the ASR is significantly high. All clean models have 0 (%) ASR. We also
observe that the test-time defense works significantly better in the case of
transformers, due to the effective localization of the trigger.

|  | Attack | Defense
---|---|---|---
Model | Clean Model | Val | Source | ASR | Val | Source | ASR
| Accuracy (%) | Accuracy (%) | Accuracy (%) | (%) | Accuracy (%) | Accuracy (%) | (%)
VGG16[31] | 71.58 | 71.59 | 71.4 | 55.00 | 58.95 | 58.80 | 49.00
ResNet18 [32] | 66.68 | 66.67 | 67.20 | 41.80 | 55.37 | 56.20 | 42.60
ResNet50 [32] | 73.94 | 73.94 | 74.00 | 34.80 | 63.53 | 60.60 | 37.20
ViT-Base[1] | 79.09 | 79.04 | 77.40 | 61.4 | 76.94 | 73.20 | 16.40
PatchConv [2] | 80.00 | 80.26 | 80.80 | 38.40 | 76.00 | 76.40 | 14.40
CaiT[3] | 85.30 | 85.30 | 87.40 | 83.9 | 74.2 | 72.00 | 31.00

Table 2: Blocking during training -  We perform blocking during training and
see that this improves both clean performance of the model and the ASR.

| Before Defense | After Defense
---|---|---
Model | Source Accuracy (%) | ASR (%) | Source Accuracy (%) | ASR (%)
ViT-Base | 77.40 | 61.4 | 73.2 | 16.40
ViT-Base (Attn Blocking) | 79.8 | 59.0 | 76.4 | 12.6

Table 3: Vision Transformers are vulnerable to Backdoor attacks- We show that vision transformers are vulnerable to data poisoning and backdoor attacks. We use ViT-Base architecture [1] and commonly used attacks such as [4, 6] to illustrate this phenomenon. Model | Val | ASR
---|---|---
| Accuracy (%) | (%)
Clean | 79.09 | 0
BadNets [4] | 78.79 | 69.6
HTBA [6] | 79.04 | 61.4

We see from Table 3 that Vision Transformers are indeed vulnerable to Backdoor
attacks. We consider BadNets[4] and HTBA [6] attacks to illustrate the
vulnerability. To the best of our knowledge, ours is the first work that
demonstrates that backdoors can be inserted into Vision Transformers by using
data poisoning and keeping the trigger hidden. Note that the adversary assumes
no access to the training procedures.

As described in previous section, we use GradRollout to highlight the regions
of the image which are responsible for prediction. Interestingly, when we
consider the interpretation for the predicted category of triggered images in
Vision Transformers, we observe that the trigger is highlighted clearly. We
show qualitative examples in Figure 3, highlighting the difference between
results for vision transformers and CNNs. Based on this phenomenon, we
highlight the region of the image responsible for predicted category and block
it to negate the effect of the trigger. We observe that such a defense can
decrease ASR significantly for vision transformer architecture. Our hypothesis
is that since backdoor attacks work on using triggers with relatively small
area, such an algorithm can be effective in localizing and negating the
backdoor trigger. For the case of clean images where there is no trigger
present, blocking out a small region of an object should have relatively
lesser influence on the prediction. We provide empirical evidence for this in
Table 1

Blocking during training: To overcome the reduction in accuracy on clean
images, we consider a procedure where we perform blocking during training to
ensure that the model becomes robust to such mechanisms for clean images.
Similar to our test-time approach, we consider an attention based blocking
which calculates the attention map for the predicted category and masks a
small portion of the region. Note that during the training procedure of HTBA
[6], the trigger is not present. We refer to this procedure as Attn
Blocking.We show our results in Table 2. Note that, although there is still a
gap in source accuracy before and after defense, the training procedure
reduces the gap and boosts performance favourably.

##  7 Discussion

In this section, we analyze some aspects of the method and try to understand
why this phenomenon occurs in vision transformer architectures.

Localization efficiency We measure the detection performance of the attention
based method using the IoU (Intersection over Union) metric. We calculate the
IoU between the trigger and predicted block mask for different architectures.
We observe from Table 5 that vision transformers clearly have a higher IoU
compared to CNNs, leading to lower attack success rates. This experiment shows
that Vision Transformers find it easier to localize the trigger for attacked
images. The interpretation map is always calculated for the predicted category
and results are averaged across 10 source-target pairs.

Original | Patched | Blocked
---|---|---
Target:Deerhound |  |
Coffeepot | Deerhound | Coffeepot
Target:Tiger Beetle |  |
Electric Locomotive | Tiger Beetle | Electric Locomotive
Target:iPod |  |
Mountain Bike | iPod | Mountain Bike

Figure 4: Recovering source category- Qualitative examples showcasing the
effectiveness of our defense. In the first column, we see the original image
predicted correctly, second column shows that once the trigger is pasted, the
prediction changes to the target category. In the third column, we observe
that original source prediction was recovered once the trigger is blocked.
Table 4: Effect on Source Accuracy- We observe that our defense is able to
improve the source accuracy significantly for vision transformers. We
calculate the percentage of attacked images that were classified as source
category, before and after defense. Qualitative examples can be found in
Figure 4.

| Before | After
---|---|---
| Defense | Defense
Model | Source Accuracy (%) | Source Accuracy (%)
| (Attacked Images) | (Attacked Images)
ViT-Base | 21.4 | 66
PatchConv | 44.8 | 67
CaiT | 5.8 | 56.8

Source label recovery: We observe that due to the successful nature of the
defense, once the trigger is blocked the original prediction of the source
image is recovered as shown in Table 4 and Figure 4. Different from the metric
Source Accuracy on non-patched images, we calculate the Source Accuracy for
attacked images as the percentage of patched images that are classified as
source, before and after defense. Due to the simple and low-cost nature of
approach, we believe that our test time defense can be combined with existing
defenses for backdoor attacks[33, 34].

Variation in blocking area: One assumption that we make in our method is that
the defender has some knowledge of the size of the trigger that is encountered
at test time. We believe this is a reasonable assumption since trigger sizes
are usually small. We conduct an experiment to study the dependency of the
blocking area used in our defense to the Attack Success Rate for patched
images and Source Accuracy on clean images. As seen in Figure 5, we vary the
size of blocking area from 10x10 to 70x70 and find that variation in Source
Accuracy is small. As expected, we get the lowest ASR when the block size
equals our trigger size (30x30), but more importantly we find that defense
does better than the baseline (no defense) for almost all values of the
blocking parameter. This suggests that the our method can be used even when
the defender has no knowledge about the size of the trigger, sacrificing to a
small extent the performance on clean data.

Figure 5:  Dependency on blocking area- We find that for different block
sizes, the source accuracy does not vary much. As expected we obtain the
lowest ASR when the block size equals trigger size (30x30). We also see that
defended ASR is consistently lower compared to the baseline of no defense,
showing that our defense is useful even with limited knowledge of trigger
size.

Table 5: IoU between predicted region and trigger- IoU betweeen the trigger
and predicted blocking mask is higher for vision transformers than CNNs.

Model | IoU $\in$[0,1]
---|---
VGG16 | 0.19
ResNet18 | 0.07
ResNet50 | 0.039
ViT-Base | 0.47
PatchConv | 0.27
CaiT | 0.66

Table 6: CNNs with other explanation methods- We try 3 different explanation
method for ResNet18 architecture and find that none of them can localize the
patch correctly. Hence we do not see much difference in ASR.

| Before Defense | After Defense
---|---|---
Method | ASR (%) | ASR (%)
GradCAM [8] | 41.80 | 42.60
Score-CAM [9] | 41.80 | 42.18
FullGrad [11] | 41.80 | 43.20

Using different interpretation algorithms for CNNs: We also try different
explanation algorithms for CNN architectures to ensure that our results are
not biased towards a particular explanation method. Our defense results for 3
explanation methods [8, 11, 9] on ResNet18 architecture is shown in Table 6.
Note that the ‘Before Defense’ results would be the same for all 3 rows. We
find that non of the 3 explanation methods can help with localizing the patch.
This show that CNNs cannot localize the patch due to the architecture, rather
than the explanation algorithms.

##  8 Conclusion

We show that existing threat models like BadNets and Hidden Trigger Backdoor
Attacks are effective against Vision Transformers, thus showcasing the
vulnerability of transformers to data poisoning. Interestingly, we find that
in transformers, interpretation algorithms like GradRollout effectively
highlights the trigger patch for a backdoor test-input. On the contrary, the
trigger localization is not effective for CNNs. Based on this observation, we
propose a test-time image blocking defense for backdoors. We empirically show
that this defense effectively reduces the attack success rate for
transformers. We hope that our results will encourage the community to study
how Vision Transformers compare to CNNs in terms of adversarial robustness.

Limitations: In our threat model, the defender makes an assumption about the
size of trigger patch encountered during test time. We also observe that the
test-time image blocking causes a drop in the accuracy of clean test images.
Additionally, by doing test time image blocking defence, our inference time
increase by factor of $2$ since we need to forward twice per image.

Societal Impact: An adversary can use this form of attack to backdoor Vision
Transformer models. Our results can be exploited by an adversary for unethical
applications and backdoor transformer architectures. But we also propose a
defense algorithm which greatly improves robustness to such adversaries. By
highlighting such results, the research community can learn to build more
robust architectures. On the other hand, it can also enable adversaries to
construct better fooling algorithms.

Acknowledgment: This material is based upon work partially supported by the
United States Air Force under Contract No. FA8750-19-C-0098, funding from SAP
SE, NSF grants 1845216 and 1920079, and also financial assistance award number
60NANB18D279 from U.S. Department of Commerce, National Institute of Standards
and Technology. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the authors and do not necessarily
reflect the views of the United States Air Force, DARPA, or other funding
agencies.

## References

  * [1] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2020.
  * [2] Hugo Touvron, Matthieu Cord, Alaaeldin El-Nouby, Piotr Bojanowski, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, and Herv’e J’egou.  Augmenting convolutional networks with attention-based aggregation.  arXiv preprint arXiv:2112.13692, 2021.
  * [3] Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Hervé Jégou.  Going deeper with image transformers.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 32–42, October 2021.
  * [4] Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg.  Badnets: Identifying vulnerabilities in the machine learning model supply chain.  arXiv preprint arXiv:1708.06733, 2017.
  * [5] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song.  Targeted backdoor attacks on deep learning systems using data poisoning.  arXiv preprint arXiv:1712.05526, 2017.
  * [6] Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash.  Hidden trigger backdoor attacks.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11957–11965, 2020.
  * [7] Aniruddha Saha, Ajinkya Tejankar, Soroush Abbasi Koohpayegani, and Hamed Pirsiavash.  Backdoor attacks on self-supervised learning.  2022\.
  * [8] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  * [9] Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu.  Score-cam: Score-weighted visual explanations for convolutional neural networks.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 24–25, 2020.
  * [10] Harish Guruprasad Ramaswamy et al.  Ablation-cam: Visual explanations for deep convolutional network via gradient-free localization.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 983–991, 2020.
  * [11] Suraj Srinivas and François Fleuret.  Full-gradient representation for neural network visualization.  Advances in neural information processing systems, 32, 2019.
  * [12] Muhammad Muzammal Naseer, Kanchana Ranasinghe, Salman H Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang.  Intriguing properties of vision transformers.  Advances in Neural Information Processing Systems, 34, 2021.
  * [13] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang.  Trojaning attack on neural networks.  2017\.
  * [14] Yuntao Liu, Yang Xie, and Ankur Srivastava.  Neural trojans.  In 2017 IEEE International Conference on Computer Design (ICCD), pages 45–48. IEEE, 2017.
  * [15] Alexander Turner, Dimitris Tsipras, and Aleksander Madry.  Clean-label backdoor attacks.  2018\.
  * [16] Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein.  Poison frogs! targeted clean-label poisoning attacks on neural networks.  In Advances in Neural Information Processing Systems, pages 6103–6113, 2018.
  * [17] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [18] Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal.  Strip: A defence against trojan attacks on deep neural networks.  In Proceedings of the 35th Annual Computer Security Applications Conference, pages 113–125, 2019.
  * [19] Soheil Kolouri, Aniruddha Saha, Hamed Pirsiavash, and Heiko Hoffmann.  Universal litmus patterns: Revealing backdoor attacks in cnns.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 301–310, 2020.
  * [20] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao.  Neural cleanse: Identifying and mitigating backdoor attacks in neural networks.  In 2019 IEEE Symposium on Security and Privacy (SP), pages 707–723. IEEE, 2019.
  * [21] Kota Yoshida and Takeshi Fujino.  Disabling backdoor and identifying poison data by using knowledge distillation in backdoor attacks on deep neural networks.  In Proceedings of the 13th ACM Workshop on Artificial Intelligence and Security, pages 117–127, 2020.
  * [22] Bao Gia Doan, Ehsan Abbasnejad, and Damith C Ranasinghe.  Februus: Input purification defense against trojan attacks on deep neural network systems.  In Annual Computer Security Applications Conference, pages 897–912, 2020.
  * [23] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever.  Improving language understanding by generative pre-training.
  * [24] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [25] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou.  Training data-efficient image transformers & distillation through attention.  In International Conference on Machine Learning, pages 10347–10357. PMLR, 2021.
  * [26] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta.  Revisiting unreasonable effectiveness of data in deep learning era.  In Proceedings of the IEEE international conference on computer vision, pages 843–852, 2017.
  * [27] Peizhuo Lv, Hualong Ma, Jiachen Zhou, Ruigang Liang, Kai Chen, Shengzhi Zhang, and Yunfei Yang.  Dbia: Data-free backdoor injection attack against transformer networks.  arXiv preprint arXiv:2111.11870, 2021.
  * [28] Samira Abnar and Willem Zuidema.  Quantifying attention flow in transformers.  arXiv preprint arXiv:2005.00928, 2020.
  * [29] Jacob Gildenblat.  https://github.com/jacobgil/vit-explain.git.
  * [30] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  International Journal of Computer Vision (IJCV), 2015.
  * [31] K. Simonyan and A. Zisserman.  Very deep convolutional networks for large-scale image recognition.  In International Conference on Learning Representations, 2015.
  * [32] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
  * [33] Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma.  Anti-backdoor learning: Training clean models on poisoned data.  Advances in Neural Information Processing Systems, 34, 2021.
  * [34] Dongxian Wu and Yisen Wang.  Adversarial neuron pruning purifies backdoored deep models.  Advances in Neural Information Processing Systems, 34, 2021.

##  Appendix A Appendix

We report the results for each pair of categories in Tables A1-A4. Please
refer to the caption for details. Also, Figure A1 shows some qualitative
visualization.

Original | Patched | ResNet50 | ResNet50 blocked | VIT-Base | VIT-Base blocked
---|---|---|---|---|---
Target:Bee |  |  |  |  |
Lighter | Bee |  | Bee |  | Lighter
Target:Partridge |  |  |  |  |
Unicycle | Partridge |  | Partridge |  | Unicycle
Target:Deerhound |  |  |  |  |
Coffeepot | Deerhound |  | Deerhound |  | Coffeepot
Target:Toyshop |  |  |  |  |
Hotdog | Toyshop |  | Toyshop |  | Hotdog
Target:Tiger Beetle |  |  |  |  |
Electric Locomotive | Tiger Beetle |  | Tiger Beetle |  | Electric Locomotive
Target:Plunger |  |  |  |  |
Theater Curtain | Plunger |  | Plunger |  | Theater Curtain
Target:iPod |  |  |  |  |
Mountain Bike | iPod |  | iPod |  | Mountain Bike
Target:Goblet |  |  |  |  |
Wing | Goblet |  | Goblet |  | Wing

Figure A1: Image Blocking Defence- Qualitative results similar to Figure 3 of
the main submission. Interestingly, we observe for the last case that the
trigger does involve some heatmap for ResNet50, but since it is shared with
the main object in the image and not the maximum region of the attention, the
blocking mechanism does not work as expected. Attack was successful for all
examples.  Table A1: Results of Attack and Test time Defense- To save space in
the main article, we reported the results averaged over 10 random pairs of
categories. In this table, we report the results for all pairs with ViT-Base
architecture (similar to Table 1). The pairs of categories are the same random
pairs used in HTBA [6]. Note that each pair of categories (each row)
corresponds to a different attack task, so depending on the similarity of
source and target categories, that attack may be easy or difficult. Hence, we
do not expect a low standard deviation of ASR across these tasks. A similar
large standard deviation was also reported in HTBA.

|  | Attack | Defense
---|---|---|---
Source | Target | Val | Source | ASR | Val | Source | ASR
|  | Accuracy (%) | Accuracy (%) | (%) | Accuracy (%) | Accuracy (%) | (%)
Slot | Australian Terrier | 79.02 | 92.00 | 56.00 | 76.94 | 92.00 | 6.00
Lighter | Bee | 79.06 | 66.00 | 58.00 | 76.95 | 70.00 | 28.00
Theater Curtain | Plunger | 78.96 | 90.00 | 82.00 | 76.95 | 78.00 | 20.00
Unicycle | Partridge | 79.04 | 92.00 | 70.00 | 76.99 | 70.00 | 14.00
Mountain Bike | Ipod | 79.04 | 78.00 | 68.00 | 76.86 | 66.00 | 30.00
Coffeepot | Deerhound | 79.04 | 64.00 | 52.00 | 76.93 | 66.00 | 16.00
Can Opener | Cuckatoo | 79.00 | 72.00 | 32.00 | 76.90 | 70.00 | 12.00
Hotdog | Toyshop | 79.02 | 90.00 | 60.00 | 76.90 | 80.00 | 22.00
Electric Locomotive | Tiger Beetle | 79.04 | 88.00 | 84.00 | 76.99 | 92.00 | 6.00
Wing | Goblet | 79.18 | 42.00 | 52.00 | 76.98 | 48.00 | 10.00
Average | 79.04 | 77.4 | 61.4 | 76.94 | 73.2 | 16.4
Standard Deviation | 0.05 | 16.5 | 15.40 | 0.04 | 13.10 | 8.47

Table A2: Results of Attack and Test time Defense- Similar to Table A1 for
ResNet50 architecture.

|  | Attack | Defense
---|---|---|---
Source | Target | Val | Source | ASR | Val | Source | ASR
|  | Accuracy (%) | Accuracy (%) | (%) | Accuracy (%) | Accuracy (%) | (%)
Slot | Australian Terrier | 74.06 | 92.00 | 6.00 | 63.83 | 92.00 | 10.00
Lighter | Bee | 73.97 | 64.00 | 52.00 | 63.46 | 48.00 | 42.00
Theater Curtain | Plunger | 73.92 | 76.00 | 52.00 | 63.5 | 70.00 | 42.00
Unicycle | Partridge | 73.96 | 72.00 | 30.00 | 63.44 | 60.00 | 34.00
Mountain Bike | Ipod | 73.89 | 74.00 | 42.00 | 63.49 | 38.00 | 62.00
Coffeepot | Deerhound | 73.95 | 58.00 | 20.00 | 63.45 | 60.00 | 26.00
Can Opener | Cuckatoo | 73.88 | 70.00 | 18.00 | 63.59 | 60.00 | 22.00
Hotdog | Toyshop | 73.84 | 78.00 | 60.00 | 63.41 | 36.00 | 60.00
Electric Locomotive | Tiger Beetle | 74.00 | 92.00 | 28.00 | 63.66 | 88.00 | 30.00
Wing | Goblet | 73.90 | 64.00 | 40.00 | 63.55 | 54.00 | 44.00
Average | 73.94 | 74.00 | 34.8 | 63.538 | 60.6 | 37.2
Standard Deviation | 0.06 | 11.27 | 17.33 | 0.12 | 18.69 | 16.28

Table A3: Results of Attack and Test time Defense- Similar to Table A1 for
ResNet18 architecture.

|  | Attack | Defense
---|---|---|---
Source | Target | Val | Source | ASR | Val | Source | ASR
|  | Accuracy (%) | Accuracy (%) | (%) | Accuracy (%) | Accuracy (%) | (%)
Slot | Australian Terrier | 66.74 | 92.00 | 22.00 | 55.32 | 84.00 | 18.00
Lighter | Bee | 66.84 | 52.00 | 34.00 | 55.44 | 56.00 | 30.00
Theater Curtain | Plunger | 66.58 | 78.00 | 32.00 | 55.00 | 68.00 | 42.00
Unicycle | Partridge | 66.53 | 70.00 | 46.00 | 55.43 | 46.00 | 42.00
Mountain Bike | Ipod | 66.66 | 68.00 | 62.00 | 55.47 | 28.00 | 62.00
Coffeepot | Deerhound | 66.57 | 52.00 | 36.00 | 55.57 | 54.00 | 34.00
Can Opener | Cuckatoo | 66.75 | 58.00 | 42.00 | 55.64 | 48.00 | 42.00
Hotdog | Toyshop | 66.67 | 70.00 | 42.00 | 55.16 | 48.00 | 64.00
Electric Locomotive | Tiger Beetle | 66.81 | 82.00 | 48.00 | 55.43 | 80.00 | 46.00
Wing | Goblet | 66.59 | 50.00 | 54.00 | 55.32 | 50.00 | 46.00
Average | 66.67 | 67.2 | 41.80 | 55.37 | 56.2 | 42.60
Standard Deviation | 0.11 | 14.18 | 11.53 | 0.18 | 16.85 | 13.73

Table A4: Results of Attack and Test time Defense- Similar to Table A1 for
PatchConv architecture.

|  | Attack | Defense
---|---|---|---
Source | Target | Val | Source | ASR | Val | Source | ASR
|  | Accuracy (%) | Accuracy (%) | (%) | Accuracy (%) | Accuracy (%) | (%)
Slot | Australian Terrier | 80.19 | 94.00 | 58.00 | 75.96 | 96.00 | 2.00
Lighter | Bee | 80.67 | 84.00 | 64.00 | 76.31 | 70.00 | 24.00
Theater Curtain | Plunger | 80.23 | 84.00 | 42.00 | 75.97 | 78.00 | 18.00
Unicycle | Partridge | 80.25 | 88.00 | 32.00 | 75.97 | 76.00 | 16.00
Mountain Bike | Ipod | 80.28 | 86.00 | 28.00 | 75.93 | 74.00 | 18.00
Coffeepot | Deerhound | 80.19 | 68.00 | 34.00 | 76.11 | 66.00 | 8.00
Can Opener | Cuckatoo | 80.19 | 82.00 | 6.00 | 76.04 | 80.00 | 2.00
Hotdog | Toyshop | 80.22 | 92.00 | 18.00 | 75.92 | 90.00 | 36.00
Electric Locomotive | Tiger Beetle | 80.16 | 88.00 | 80.00 | 75.95 | 88.00 | 4.00
Wing | Goblet | 80.18 | 42.00 | 22.00 | 75.93 | 46.00 | 16.00
Average | 80.26 | 80.8 | 38.4 | 76.00 | 76.40 | 14.40
Standard Deviation | 0.15 | 15.35 | 22.82 | 0.12 | 14.13 | 10.78