# Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep
Robust Classifiers

Jonas Ngnawé
IID-Université Laval and Mila
<EMAIL_ADDRESS>
&Sabyasachi Sahoo
IID-Université Laval and Mila
<EMAIL_ADDRESS>
Yann Pequignot
IID-Université Laval
<EMAIL_ADDRESS>
&Frédéric Precioso
Université Côte d’Azur, CNRS, INRIA, I3S, Maasai
<EMAIL_ADDRESS>
&Christian Gagné
IID-Université Laval, Mila and Canada CIFAR AI Chair
<EMAIL_ADDRESS>

###### Abstract

Despite extensive research on adversarial training strategies to improve
robustness, the decisions of even the most robust deep learning models can
still be quite sensitive to imperceptible perturbations, creating serious
risks when deploying them for high-stakes real-world applications. While
detecting such cases may be critical, evaluating a model’s vulnerability at a
per-instance level using adversarial attacks is computationally too intensive
and unsuitable for real-time deployment scenarios. The input space margin is
the exact score to detect non-robust samples and is intractable for deep
neural networks. This paper introduces the concept of margin consistency – a
property that links the input space margins and the logit margins in robust
models – for efficient detection of vulnerable samples. First, we establish
that margin consistency is a necessary and sufficient condition to use a
model’s logit margin as a score for identifying non-robust samples. Next,
through comprehensive empirical analysis of various robustly trained models on
CIFAR10 and CIFAR100 datasets, we show that they indicate strong margin
consistency with a strong correlation between their input space margins and
the logit margins. Then, we show that we can effectively use the logit margin
to confidently detect brittle decisions with such models and accurately
estimate robust accuracy on an arbitrarily large test set by estimating the
input margins only on a small subset. Finally, we address cases where the
model is not sufficiently margin-consistent by learning a pseudo-margin from
the feature representation. Our findings highlight the potential of leveraging
deep representations to efficiently assess adversarial vulnerability in
deployment scenarios.

##  1 Introduction

Deep neural networks are known to be vulnerable to adversarial perturbations,
visually insignificant changes in the input resulting in the so-called
adversarial examples that alter the model’s prediction (Biggio et al., 2013;
Goodfellow et al., 2015). They constitute actual threats in real-world
scenarios (Evtimov et al., 2017; Gnanasambandam et al., 2021), jeopardizing
their deployment in sensitive and safety-critical systems such as autonomous
driving, aeronautics, and health care. Research in the field has been intense
and produced various adversarial training strategies to defend against the
vulnerability to adversarial perturbations with bounded $\ell_{p}$ norm (e.g.,
$p=2$, $p=\infty$) through augmentation, regularization, and detection (Xu et
al., 2017; Madry et al., 2018; Zhang et al., 2019; Carmon et al., 2019; Wang
et al., 2020; Wu et al., 2020; Rice et al., 2020), to cite a few. The
empirical robustness (adversarial accuracy) of these adversarially trained
models is still far behind their high performance in terms of accuracy. It is
typically estimated by assessing the vulnerability of samples of a given test
set using adversarial attacks (Carlini & Wagner, 2016; Madry et al., 2018) or
an ensemble of attacks such as the standard _AutoAttack_ (Croce & Hein,
2020b). The objective of that evaluation is to determine if, for a given
normal sample, an adversarial instance exists within a given $\epsilon$-ball
around it. Yet, this robustness evaluation over a specific test set gives a
global property of the model but not a local property specific to a single
instance (Seshia et al., 2018; Dreossi et al., 2019). Beyond that particular
test set, obtaining this information for each new sample would typically
involve rerunning adversarial attacks or performing a formal robustness
verification, which in certain contexts may be computationally prohibitive in
terms of resources and time. Indeed, in high-stakes deployment scenarios,
knowing the vulnerability of single instances in real-time (i.e., their
susceptibility to adversarial attacks) would be valuable, for example, to
reduce risk, prioritize resources, or monitor operations. Current research
lacks efficient and scalable ways to determine the vulnerability of a sample
in a deployment context.

The input space margin (i.e., the distance of the sample to the model’s
decision boundary in the input space), or input margin in short, can be used
as a score to determine whether the sample is non-robust and, as such, likely
to be vulnerable to adversarial attacks. Computing the exact input margin is
intractable for deep neural networks (Katz et al., 2017; Elsayed et al., 2018;
Jordan & Dimakis, 2020). These input margins may not be meaningful for fragile
models with zero adversarial accuracies as all samples are vulnerable (close
to the decision boundary). However, for robustly trained models, where only
certain instances are vulnerable, the input margin is very useful in
identifying the critical samples. Previous research studies have explored
input margins of deep neural networks during training, focusing on their
temporal evolution (Mickisch et al., 2020; Xu et al., 2023), and their
exploitation in improving adversarial robustness through instance-reweighting
with approximations (Zhang et al., 2020; Liu et al., 2021) and margin
maximization (Elsayed et al., 2018; Ding et al., 2020; Xu et al., 2023).
However, to the best of our knowledge, no previous research studies the
relationship between the input space margin and the logit margin of robustly
trained deep classifiers in the context of vulnerability detection.

In this paper, we investigate how the deep representation of robust models can
provide information about the vulnerability of any single sample to
adversarial attacks. We specifically address whether the logit margin as an
approximation of the distance to the decision boundary in the feature space of
the deep neural network (penultimate layer) can reliably serve as a proxy of
the input margin for vulnerability detection. When this holds, we will refer
to the model as being _margin-consistent_. The margin consistency property
implies that the model can directly identify instances where its robustness
may be compromised simply from a simple forward pass using the logit margin.
Fig. 1 illustrates this idea of margin consistency.

Figure 1: Illustration of the input space margin, margin in the feature space
and margin consistency. The margin-consistent model preserves the relative
position of samples to the decision boundary in the input space to the feature
space.

The following contributions are presented in the paper:

  * •

We introduce the notion of _margin consistency_ 111Code available at:
https://github.com/ngnawejonas/margin-consistency, a property to characterize
robust models that allow using their logit margin as a proxy estimation for
the input space margin in the context of non-robust sample detection. We prove
that margin consistency is a necessary and sufficient condition to reliably
use the logit margin for detecting non-robust samples.

  * •

Through an extensive empirical investigation of pre-trained models on CIFAR10
and CIFAR100 with various adversarial training strategies, mainly taken from
RobustBench (Croce et al., 2021), we provide evidence that almost all the
investigated models display strong margin consistency, i.e., there is a strong
correlation between the input margin and the logit margin.

  * •

We confirm experimentally that models with strong margin consistency perform
well in detecting samples vulnerable to adversarial attacks based on their
logit margin. In contrast, models with weaker margin consistency exhibit
poorer performance. Leveraging margin consistency, we can also estimate the
robust accuracy on an arbitrarily large test set by estimating the input
margins only on a small subset.

  * •

For models where margin consistency does not hold, exhibiting a weak
correlation between the input margin and the logit margin, we simulate margin
consistency by learning to map the model’s feature representation to a pseudo-
margin with a better correlation through a simple learning scheme.

(a) CIFAR10

(b) CIFAR100

Figure 2: Margin consistency of various models: there is a strong correlation
between input space margin and logit margin for most $\ell_{\infty}$ robust
models tested, the exceptions being DI0 and XU8 on CIFAR10. See Table 1 for
the references on the models. The plots are given with standard error for the
y-axis values in each interval.

##  2 Methodology

###  2.1 Notation and Preliminaries

Notation We consider $f_{\theta}:\mathbb{R}^{n}\to\mathbb{R}^{C}$ a deep
neural network classifier with weights $\mathbf{\theta}$ trained on a dataset
of samples drawn iid from a distribution $\mathcal{D}$ on a product space
$\mathcal{X}\times\mathcal{Y}$. Each sample $\mathbf{x}$ in the input space
$\mathcal{X}\subset\mathbb{R}^{n}$ has a unique corresponding label
$y\in\mathcal{Y}=\\{1,2,\ldots,C\\}$. The prediction of $\mathbf{x}$ is given
by
$\hat{y}(\mathbf{x})=\operatorname*{arg\,max}\nolimits_{j\in\mathcal{Y}}f_{\theta}^{j}(\mathbf{x})$,
where $f_{\theta}^{j}(\mathbf{x})$ is the $j$-th component of
$f_{\theta}(\mathbf{x})$. We consider that a deep neural classifier is
composed of a feature extractor $h_{\psi}:\mathcal{X}\to\mathbb{R}^{m}$ and a
linear head with $C$ linear classifiers $\\{\mathbf{w_{j}},b_{j}\\}$ such that
$f_{\theta}^{j}(\mathbf{x})=\mathbf{w_{j}}^{\top}h_{\psi}(\mathbf{x})+b_{j}$.
The predictive distribution $p_{\theta}(y|\mathbf{x})$ is obtained by taking
the softmax of the output $f_{\theta}(\mathbf{x})$. A perturbed sample
$\mathbf{x}^{\prime}$ can be obtained by adding a perturbation $\delta$ to
$\mathbf{x}$ within an $\epsilon$-ball $B_{p}(\mathbf{x},\epsilon)$, an
$\ell_{p}$-norm ball of radius $\epsilon>0$ centered at $\mathbf{x}$;
$B_{p}(\mathbf{x},\epsilon):=\\{\mathbf{x}^{\prime}:\|\mathbf{x}^{\prime}-\mathbf{x}\|_{p}=\|\delta\|_{p}<\epsilon\\}$.
The distance $\|\mathbf{x}^{\prime}-\mathbf{x}\|_{p}=\|\mathbf{\delta}\|_{p}$
represents the perturbation size defined as
$\left(\sum\nolimits_{i=1}^{n}|\delta_{i}|^{p}\right)^{\frac{1}{p}}$. In this
paper, we will focus on $\ell_{\infty}$ norm
$\left(\|\mathbf{x}\|_{\infty}=\max\nolimits_{i=1,\ldots,n}|x_{i}|\right)$,
which is the most commonly used norm in the literature.

Local robustness Different notions of local robustness exist in the literature
(Gourdeau et al., 2021; Zhong et al., 2021; Han et al., 2023). In this paper,
we equate local robustness to $\bm{\ell_{p}}$-robustness, a standard notion
corresponding to the invariance of the decision within the $\ell_{p}$
$\epsilon$-ball around the sample (Bastani et al., 2016; Fawzi et al., 2018)
and formalized in terms of $\epsilon$-robustness.

######  Definition 1.

A model $f$ is $\bm{\epsilon}$-robust at point $\mathbf{x}$ if for any
$\mathbf{x}^{\prime}\in B_{p}(\mathbf{x},\epsilon)$ ($\mathbf{x}^{\prime}$ in
the $\epsilon$-ball around $\mathbf{x}$), we have
$\hat{y}(\mathbf{x}^{\prime})=\hat{y}(\mathbf{x})$.

For a given robustness threshold $\epsilon$, a data instance is said to be
non-robust for the model if this model is not $\epsilon$-robust on it. This
means it is possible to construct an adversarial sample from that instance in
its vicinity (i.e., within an $\epsilon$-ball distance from the original
instance). A vulnerable sample to adversarial attacks is necessarily non-
robust. This notion of local robustness can be quantified in the worst-case
or, on average, inside the $\epsilon$-ball. We focus here on the worst-case
measurement given by the input margin, also referred to as the _minimum
distortion_ or the _robust radius_ (Szegedy et al., 2014; Carlini & Wagner,
2016; Weng, 2019)

The input space margin is the distance to the decision boundary of $f$ in the
input space. It is the norm of a minimal perturbation required to change the
model’s decision at a test point $\mathbf{x}$:

$d_{in}(\mathbf{x})=\inf\\{\|\mathbf{\delta}\|_{p}:\mathbf{\delta}\in\mathbb{R}^{n}\text{
s.t.
}\hat{y}(\mathbf{x})\neq\hat{y}(\mathbf{x}+\mathbf{\delta)}\\}=\sup\\{\epsilon:f\text{
is }\epsilon\text{-robust at }\mathbf{x}\\}.$ (1)

An instance $\mathbf{x}$ is non-robust for a robustness threshold $\epsilon$
if $d_{in}(\mathbf{x})\leq\epsilon$. Evaluating Eq. 1 for deep networks is
known to be intractable in the general case. An upper bound approximation can
be obtained using a point $\mathbf{x}_{0}^{\prime}$, the closest adversarial
counterpart of $\mathbf{x}$ in $\ell_{p}$ norm by
$\hat{d}_{in}(\mathbf{x})=\|\mathbf{x}-\mathbf{x}_{0}^{\prime}\|_{p}$ (see
Fig. 1).

The logit margin is the difference between the two largest logits. For a
sample $\mathbf{x}$ classified as
$i=\hat{y}(\mathbf{x})=\operatorname*{arg\,max}\nolimits_{j\in\mathcal{Y}}f_{\theta}^{j}(\mathbf{x})$
the logit margin is defined as
$\left(f_{\theta}^{i}(\mathbf{x})-\max\limits_{j,j\neq
i}f_{\theta}^{j}(\mathbf{x})\right)>0$. It is an approximation of the distance
to the decision boundary of $f_{\theta}$ in the feature space. The decision
boundary in the feature space around $\mathbf{z}=h_{\psi}(\mathbf{x})$, the
feature representation of $\mathbf{x}$, is composed of $(C-1)$ linear decision
boundaries (hyperplanes)
$\text{DB}_{ij}=\\{\mathbf{z}^{\prime}\in\mathbb{R}^{m}:\mathbf{w}_{i}^{\top}\mathbf{z}^{\prime}+b_{i}=\mathbf{w}_{j}^{\top}\mathbf{z}^{\prime}+b_{j}\\}$
($j\neq i$). The margin in the feature space is, therefore, the distance to
the closest hyperplane, i.e. $\min\limits_{j,j\neq
i}d(\mathbf{z},\text{DB}_{ij})$, where the distance
$d(\mathbf{z},\text{DB}_{ij})$ from $\mathbf{z}$ to a hyperplane
$\text{DB}_{ij}$ has a closed-form expression:

$d(\mathbf{z},\text{DB}_{ij})=\inf\\{\|\mathbf{\eta}\|_{p}:\mathbf{\eta}\in\mathbb{R}^{m}\,\text{
s.t.
}\,\mathbf{z}+\mathbf{\eta}\in\text{DB}_{ij}\\}=\frac{f_{\theta}^{i}(\mathbf{x})-f_{\theta}^{j}(\mathbf{x})}{\|\mathbf{w}_{i}-\mathbf{w}_{j}\|_{q}},$
(2)

where $\|\cdot\|_{q}$ is the dual norm of $p$, $q=\frac{p}{p-1}$ for $p>1$
(Moosavi-Dezfooli et al., 2016; Elsayed et al., 2018).

When the classifiers $\mathbf{w_{j}}$ are equidistant
($\|\mathbf{w}_{i}-\mathbf{w}_{j}\|_{q}=\omega>0,\,\forall i,j$), the margin
becomes:

$\min\limits_{j,j\neq
i}\frac{f_{\theta}^{i}(\mathbf{x})-f_{\theta}^{j}(\mathbf{x})}{\omega}=\frac{1}{\omega}\min\limits_{j,j\neq
i}\bigg{(}f_{\theta}^{i}(\mathbf{x})-f_{\theta}^{j}(\mathbf{x})\bigg{)}=\frac{1}{\omega}\bigg{(}\underbrace{f_{\theta}^{i}(\mathbf{x})-\max\limits_{j,j\neq
i}f_{\theta}^{j}(\mathbf{x})}_{\text{logit margin}}\bigg{)}.$ (3)

Under the equidistance assumption, the logit margin is proportional (equal up
to a scaling factor) to the margin in the feature space. We will denote the
logit margin of $\mathbf{x}$ by $d_{out}(\mathbf{x})$:

$d_{out}(\mathbf{x})=f_{\theta}^{i}(\mathbf{x})-\max\limits_{j,j\neq
i}f_{\theta}^{j}(\mathbf{x})$ (4)

###  2.2 Margin Consistency

######  Definition 2.

A model is margin-consistent if there is a monotonic relationship between the
input space margin and the logit margin, i.e., $d_{in}(\mathbf{x}_{1})\leq
d_{in}(\mathbf{x}_{2})\Leftrightarrow d_{out}(\mathbf{x}_{1})\leq
d_{out}(\mathbf{x}_{2}),\,\forall\mathbf{x}_{1},\mathbf{x}_{2}\in\mathcal{X}$.

A margin-consistent model preserves the relative position of samples to the
decision boundary from the input space to the feature space. A sample further
from (closer to) the decision boundary in the input space remains further from
(closer to) the decision boundary in the feature space with respect to other
samples, as illustrated in Fig. 1.

We can evaluate margin consistency by computing the Kendall rank correlation
($\bm{\tau}\in[-1,1]$) between the output scores and the input margins over a
test set. The Kendall rank correlation tests the existence and strength of a
monotonic relationship between two variables. It makes no assumption on the
distribution of the variables and is robust to outliers (Chattamvelli, 2024).
Perfect margin consistency corresponds to an absolute value of $1$, and $0$
means the absence of margin consistency.

###  2.3 Non-robust Samples Detection

Non-robust detection can be defined as a scored-based binary classification
task where non-robust samples constitute the positive class, and the input
margin $d_{in}$ induces a perfect discriminative function $g$ for that:

$g(\mathbf{x})=\mathbbm{1}_{[d_{in}(\mathbf{x})\leq\epsilon]}(\mathbf{x})=\begin{cases}1&\text{
if }\,\,\mathbf{x}\,\,\text{ is non-robust}\\\ 0&\text{ if
}\,\,\mathbf{x}\,\,\text{ is robust}\\\ \end{cases}.$

If a model is margin-consistent, its logit margin can also be a discriminative
score to detect non-robust samples. The following theorem establishes that
this is a necessary and sufficient condition. Therefore, the degree to which a
model is margin-consistent should determine the discriminative power of the
logit margin.

######  Theorem 1.

If a model is margin-consistent, then for any robustness threshold $\epsilon$,
there exists a threshold $\lambda$ for the logit margin $d_{out}$ that
separates perfectly non-robust samples and robust samples. Conversely, if for
any robustness threshold $\epsilon$, $d_{out}$ admits a threshold $\lambda$
that separates perfectly non-robust samples from robust samples, then the
model is margin-consistent.

_Proof sketch_. Fig. 3 presents intuition behind the proof of Theorem 1. For
the first part of the theorem (see Fig. 3(a)), if there is a monotonic
relationship between $d_{in}$ and $d_{out}$ (margin consistency), any point
$\mathbf{x}$ with $d_{in}$ less than the threshold $\epsilon$ (non-robust)
will also have $d_{out}$ less than $\lambda=d_{out}(\mathbf{x_{0}})$ (with
$d_{in}(\mathbf{x_{0}})=\epsilon$). For the second part (see Fig. 3(b)), if
there are two points $\mathbf{x}_{1}$ and $\mathbf{x}_{2}$ with non-concordant
$d_{in}$ and $d_{out}$ (no margin consistency), then for a threshold
$\epsilon_{0}$ between $d_{in}(x_{1})$ and $d_{out}(x_{2})$, they will both
have different classes but no threshold of $d_{out}$ (horizontal line) can
classify them both correctly. The complete proof of Theorem 1 is deferred to
Appendix A.

(a) Margin consistency implies $d_{out}$ can perfectly separate non robust
samples in $A_{\epsilon}$ from robust samples.

(b) Without margin consistency, $d_{out}$ cannot be a good discriminator for
robust and non-robust samples.

Figure 3: Illustration of Theorem 1’s proof.

Common metrics for detection include (Hendrycks & Gimpel, 2017; Corbière et
al., 2019; Zhu et al., 2023): the Area Under the Receiver Operating Curve
(AUROC), which ensures the ability of a model to distinguish between the
positive and negative classes across all possible thresholds; the Area Under
the Precision-Recall Curve (AUPR), which evaluates the trade-off between
precision and recall and is less sensitive to imbalance between positive and
negative classes; and the False Positive Rate (FPR) at a 95% True Positive
Rate (TPR) (FPR@95), that is crucial in systems where missing positive cases
can have serious consequences, such as minimizing the number of vulnerable
samples missed. The AUROC and AUPR of a perfect classifier is 1, while 0.5 for
a random classifier.

###  2.4 Sample Efficient Robustness Evaluation

Margin consistency enables empirical robustness evaluation over an arbitrarily
large test set by only estimating the input margins of a small subset of test
samples. For a robustness evaluation at threshold $\epsilon$ (e.g.,
$\epsilon=8/255$ in $\ell_{\infty}$ norm on CIFAR10 and CIFAR100), we randomly
sample a small subset of the large test set and determine the threshold
$\lambda$ for the logit margin that corresponds to $\epsilon$. The threshold
$\lambda$ is then used to detect vulnerable samples. With the true labels of
these test sets, we can determine the proportion of correct non-vulnerable
samples, which is the standard robust accuracy as described in Algorithm 1.

Algorithm 1 Sample Efficient Robustness Estimation

1: Input: Test Dataset $(X,Y)\in(\mathcal{X}\times\mathcal{Y})^{N}$,
Robustness threshold $\epsilon>0$, Subset size $n\ll N$.

2: Output: Robust Accuracy Estimation $\mathcal{A}_{r}$

3: \- Select uniformly at random a subset $X_{n}$ of $n$ samples from $X$.

4: \- Compute the estimations of the input margins on $X_{n}$,
$D_{n}=\\{\hat{d}_{in}(\mathbf{x}):\mathbf{x}\in X_{s}\\}$

5: \- Create ground truth labels for vulnerability at threshold $\epsilon$
i.e. $\mathbbm{1}_{[\hat{d}_{in}(\mathbf{x})\leq\epsilon]}(\mathbf{x})$, for
$\mathbf{x}\in X_{s}$.

6: \- Determine the threshold $\lambda$ of $d_{out}$ that gives best
approximation of robust accuracy on $X_{s}$.

7: \- $\mathcal{A}_{r}=|\\{\mathbf{x}\in X:\,d_{out}(\mathbf{x})>\lambda\text{
and }\hat{y}(\mathbf{x})=y\\}|/N$

A naive way to set the threshold $\lambda$ at line 6 of Algorithm 1 would be
to set it to the detection threshold at $\alpha=95\%$ TPR or $\alpha=90\%$
TPR, but the logit margin threshold could vary from one model to ano;
therefore a better way is to select it by tuning over values $\alpha>=0.80$
that gi ves the best approximation of the robust accuracy in terms of the
absolute error on the small subset $X_{s}$. The same logic applies if we want
to estimate the vulnerability of a large dataset without the labels.

##  3 Evaluation

###  3.1 Experimental Setup

Datasets and models We investigate various pre-trained models on CIFAR10 and
CIFAR100 datasets. The majority of models were loaded from the _RobustBench_
model zoo222https://github.com/RobustBench/robustbench (Croce et al., 2021),
with a few more models that are ResNet-18 (He et al., 2016) models we trained
on CIFAR10 with Standard Adversarial Training (Madry et al., 2018), TRADES
(Zhang et al., 2019), Logit Pairing (ALP and CLP, Kannan et al. (2018)), and
MART (Wang et al., 2020), using the experimental setup of Wang et al. (2020).

Input margin estimation This is done using FAB attack (Croce & Hein, 2020a),
which is an attack that minimally perturbs the initial instance. Xu et al.
(2023) used it in their adversarial training strategy as a reliable way to
compute the closest boundary point given enough iterations. We perform the
untargeted FAB attack without restricting the distortion to find the boundary
for all the samples in the test set instead of constraining the perturbation
inside a given $\epsilon$-ball when evaluating robustness. As a sanity check
for the measured distances, we compare the ratio of correct samples
$\mathbf{x}$ with estimated input margins greater than $\epsilon=8/255$ and
the robust accuracy in $\ell_{\infty}$ norm measured with _AutoAttack_ (Croce
& Hein, 2020b) at $\epsilon=8/255$. Both quantities estimate the same thing,
with a mean and a maximum absolute difference over the models respectively of
$1.3$ and $6.1$ for CIFAR10, $0.48$ and $0.75$ on CIFAR100, which are
reasonable (cf. Fig. 8 in appendix B.1 for the comparison for all the models).

The estimation of the input margins over the $10,000$ test samples allows us
to create for a given threshold $\epsilon$ a pool of vulnerable samples that
can be successfully attacked at threshold $\epsilon$ and non-vulnerable
samples that were not able to be attacked. Training and distance estimations
were run on an NVIDIA Titan Xp GPU (1x).

###  3.2 Results and Analysis

Correlation analysis The results presented in Fig. 2 show that the logit
margin has a strong correlation (up to 0.86) with the input margin, which
means that they have a level of margin consistency for those models. The plots
are given with standard error for the y-axis values in each interval.

(a) CIFAR10

(b) CIFAR100

Figure 4: Distribution of the correlation between input margins and logit
margins in $\ell_{\infty}$ with robust accuracy. The strength of the
correlation, which indicates the level of margin consistency, does not depend
on the robust accuracy. References on models are given in Table 1.

However, we also observe that two models (i.e., DI0 (Ding et al., 2020) and
XU80 (Xu et al., 2023) WideResNets on CIFAR10) have a weaker correlation. We
show in Sec. 3.3 that we can learn to map the feature representation of these
models to a pseudo-margin that reflects the distance to the decision boundary
in the input space.

Vulnerable samples detection We present the results for the robustness
threshold $\epsilon=8/255$ in Table 1. As expected with the strong
correlations, the performance over the non-robust detection task is excellent.
We can note that the metrics are lower for the two models with low
correlations with particularly very high FPR@95. The performance remains quite
good with different values of $\epsilon$ (cf. appendix B.2).

| Model ID |  Kendall $\bm{\tau}$ ($\uparrow$) |  AUROC ($\uparrow$) |  AUPR ($\uparrow$) |  FPR@95 ($\downarrow$) | Acc | Rob. Acc | Architecture
---|---|---|---|---|---|---|---|---
CIFAR10 |  DI0 (Wu et al., 2020) | 0.28 | 67.49 | 70.91 | 82.56 | 84.36 | 41.44 | WideResNet-28-4
XU80 (Xu et al., 2023) | 0.43 | 83.30 | 80.50 | 83.42 | 93.69 | 63.89 | WideResNet-28-10
MR0 (Wang et al., 2020) | 0.68 | 92.95 | 94.92 | 29.76 | 79.69 | 39.12 | ResNet-18
DM0 (Debenedetti et al., 2023) | 0.71 | 94.31 | 93.20 | 32.76 | 91.30 | 57.27 | XCiT-M12
AL0 (Kannan et al., 2018) | 0.72 | 94.67 | 95.98 | 24.93 | 80.38 | 40.21 | ResNet-18
CU80 (Cui et al., 2023) | 0.73 | 96.87 | 94.42 | 17.90 | 92.16 | 67.73 | WideResNet-28-10
WA80 (Wang et al., 2023) | 0.74 | 96.82 | 94.33 | 17.60 | 92.44 | 67.31 | WideResNet-28-10
SE10 (Sehwag et al., 2021) | 0.74 | 96.03 | 94.66 | 19.13 | 84.59 | 55.54 | ResNet-18
EN0 (Engstrom et al., 2019) | 0.74 | 95.16 | 95.07 | 24.10 | 87.03 | 49.25 | ResNet-50
TR0 (Zhang et al., 2019) | 0.74 | 94.63 | 96.13 | 30.93 | 80.72 | 42.23 | ResNet-18
DS0 (Debenedetti et al., 2023) | 0.75 | 95.80 | 95.08 | 24.65 | 90.06 | 56.14 | XCiT-S12
MD0 (Madry et al., 2018) | 0.75 | 95.36 | 97.00 | 23.23 | 81.85 | 36.91 | ResNet-18
ZH0 (Zhang et al., 2019) | 0.75 | 95.86 | 95.65 | 24.91 | 84.92 | 53.08 | WideResNet-34-10
HE0 (Hendrycks et al., 2019) | 0.76 | 96.35 | 95.68 | 20.01 | 87.11 | 54.92 | WideResNet-28-10
CL0 (Kannan et al., 2018) | 0.77 | 95.93 | 96.98 | 20.01 | 81.12 | 40.08 | ResNet-18
RE80 (Rebuffi et al., 2021) | 0.77 | 97.33 | 95.70 | 13.87 | 87.33 | 60.73 | WideResNet-28-10
PA20 (Pang et al., 2022) | 0.78 | 97.65 | 96.39 | 14.40 | 88.61 | 61.04 | WideResNet-28-10
AD20 (Addepalli et al., 2022) | 0.81 | 97.67 | 97.46 | 13.42 | 85.71 | 52.48 | ResNet-18
AD10 (Addepalli et al., 2021) | 0.82 | 97.86 | 97.68 | 13.26 | 80.24 | 51.06 | ResNet-18
CIFAR100 |  HE1 (Hendrycks et al., 2019) | 0.74 | 94.43 | 97.39 | 30.40 | 59.23 | 28.42 | WideResNet-28-10
WU1 (Wu et al., 2020) | 0.78 | 95.81 | 98.00 | 23.34 | 60.38 | 28.86 | WideResNet-34-10
RE81 (Rebuffi et al., 2021) | 0.80 | 96.87 | 98.30 | 18.06 | 62.41 | 32.06 | WideResNet-28-10
DS1 (Debenedetti et al., 2023) | 0.81 | 96.78 | 98.30 | 19.18 | 67.34 | 32.19 | XCiT-S12
CU41 (Cui et al., 2023) | 0.82 | 97.07 | 98.48 | 17.21 | 64.08 | 31.65 | WideResNet-34-10
CU81 (Cui et al., 2023) | 0.83 | 97.41 | 98.24 | 15.62 | 73.85 | 39.18 | WideResNet-28-10
RI1 (Rice et al., 2020) | 0.83 | 96.61 | 99.05 | 18.14 | 53.83 | 18.95 | PreActResNet-18
PA21 (Pang et al., 2022) | 0.83 | 97.66 | 98.82 | 13.83 | 63.66 | 31.08 | WideResNet-28-10
WA81 (Wang et al., 2023) | 0.83 | 97.51 | 98.28 | 14.96 | 72.58 | 38.83 | WideResNet-28-10
AD21 (Addepalli et al., 2022) | 0.84 | 97.46 | 98.92 | 16.00 | 65.45 | 27.67 | ResNet-18
AD1 (Addepalli et al., 2021) | 0.84 | 97.65 | 98.99 | 13.88 | 62.02 | 27.14 | PreActResNet-18
RE11 (Rebuffi et al., 2021) | 0.85 | 97.97 | 99.05 | 13.21 | 56.87 | 28.50 | PreActResNet-18
RA11 (Rade & Moosavi-Dezfooli, 2021) | 0.85 | 98.01 | 99.08 | 12.36 | 61.50 | 28.88 | PreActResNet-18

Table 1: Correlations and vulnerable points detection performance at
$\epsilon=8/255$ on different adversarially trained models.

Sample Efficient Robustness Estimation We recover the robust accuracy of the
investigated models evaluated with over $10,000$ using only a small subset.
Fig. 5 shows the absolute error of the estimation for 500 samples. The
estimations are still descent with 100 samples (cf. Fig. 10 in appendix B.3).

(a) CIFAR10

(b) CIFAR100

Figure 5: Estimations of the robust accuracy reported by Robustbench using
logit margins with only 500 samples are quite accurate both on CIFAR10 and
CIFAR100 for strongly margin-consistent models. The numbers indicate the
absolute difference between the two values, averaged over ten subsets. See
Table 1 for the specific references on the model ID.

Margin Consistency and Lipschitz Smoothness A neural network $f$ is said to be
$L$-Lipschitz if $\|f(\mathbf{x_{1}})-f(\mathbf{x_{2}}\|\leq
L\|\mathbf{x_{1}}-\mathbf{x_{2}}\|$, $\forall\mathbf{x_{1}},\mathbf{x_{2}}$.
Lipschitz smoothness is important for adversarial robustness because a small
Lipschitz constant $L$ guarantees the network’s output cannot change more than
a factor $L$ of the change in the input. There are strategies to directly
constraint the Lipschitz constant to achieve $1$-Lipschitz networks (Cisse et
al., 2017; Li et al., 2019; Serrurier et al., 2021; Araujo et al., 2023).
Empirical adversarial training strategies aim to achieve Lipschitz’s
smoothness indirectly. Note, however, that Lipschitz continuity does not imply
margin consistency, for example, two points $\mathbf{x_{1}}$ and
$\mathbf{x_{2}}$ with $0<d_{in}(\mathbf{x_{1}})<d_{in}(\mathbf{x_{2}})$. While
the $L$-Lipschitz condition implies that $d_{out}(\mathbf{x_{i}})\leq
Ld_{in}(\mathbf{x_{i}})$ for $i=1,2$, it is clearly possible to have
$d_{out}(\mathbf{x_{2}})<d_{out}(\mathbf{x_{1}})$, thus violating the margin
consistency condition.

Fig. 4(a) and 4(b) show that the strength of the correlation, i.e. the level
of margin consistency, does not depend on the robust accuracy.

Insight into when margin consistency may hold? We hypothesize that margin
consistency can occur when the feature extractor $h_{\psi}$ behaves locally as
an isometry (preserving distances, up to a scaling factor $\kappa$), i.e.,
$\|h_{\psi}(\mathbf{x})-h_{\psi}(\mathbf{x}^{\prime})\|_{p}=\kappa\|\mathbf{x}-\mathbf{x}^{\prime}\|_{p}$.
We can experimentally see that there is a high correlation between the input
margin ($\|\mathbf{x}-\mathbf{x}^{\prime}\|$) and the distance between the
feature representations of $\mathbf{x}$ and $\mathbf{x}^{\prime}$ (Fig. 6).
Given an input sample $\mathbf{x}$, by definition
$d_{out}(\mathbf{x})=\|\mathbf{z}-\mathbf{z}^{\prime}\|_{p}$ where
$\mathbf{z}=h_{\psi}(\mathbf{x})$ and $\mathbf{z}^{\prime}$ an orthogonal
projection of $\mathbf{z}$ on the boundary hyperplane. The points
$\mathbf{z}$, $\mathbf{z^{\prime}}$ and $h_{\psi}(\mathbf{x}^{\prime})$ will
form a right triangle so the side $\|\mathbf{z}-\mathbf{z}^{\prime}\|_{p}$
will directly correlate with side
$\|h_{\psi}(\mathbf{x})-h_{\psi}(\mathbf{x}^{\prime})\|_{p}$.

(a) CIFAR10.

(b) CIFAR100.

Figure 6: The correlations between the input margin
($\|\mathbf{x}-\mathbf{x}^{\prime}\|$) and the distance between the feature
representations of samples and their adversarial counterparts
($\|h_{\psi}(\mathbf{x})-h_{\psi}(\mathbf{x}^{\prime})\|$) suggests a local
behaviour of the feature extractor as an isometry (up to a scaling factor).
See Table 1 for the specific references on the model ID. The plots are given
with standard error for the y-axis values in each interval.

###  3.3 Learning a Pseudo-Margin

For the two models that are weakly margin-consistent, we are proposing to
directly learn a mapping that maps the feature representation of a sample to a
pseudo-margin that reflects the relative position of the samples to the
decision in the input space. We use a learning scheme similar to the one of
Corbière et al. (2019), with a small ad hoc neural network for learning the
confidence of the instances (cf. Fig. 12 in appendix B.4). Given some samples
with estimations of their input margins, the objective is to learn to map
their feature representation to a pseudo-margin that correlates with the input
margins. This learning task can be seen as a learning-to-rank problem. We use
a simple learning-to-rank algorithm for that purpose, which is a pointwise
regression approach (He et al., 2008) relying on the mean squared error as a
surrogate loss.

For the experiment, we used a similar architecture and training protocol as
(Corbière et al., 2019) with a fully connected network with five dense layers
of 512 neurons, with ReLU activations for the hidden layers and a sigmoid
activation at the output layer. We learn using 5000 examples sampled randomly
from the training set, with $20\%$ (1000 examples) held as a validation. Fig.
7 and Table 2 show the improved correlation on the learned score compared to
the logit margin for both models. The plots are given with standard error for
the y-axis values in each interval. The network has learned to recover the
relative positions of the samples from the feature representation.

Figure 7: Correlation improvement of the learned pseudo-margin over the logit
margin for DI0 (Ding et al., 2020) and XU80 (Xu et al., 2023).

Model ID | Margin |  Kendall $\bm{\tau}$ ($\uparrow$) |  AUROC ($\uparrow$) |  AUPR ($\uparrow$) |  FPR@95 ($\downarrow$) | Acc. | Rob. Acc
---|---|---|---|---|---|---|---
DI0 (Ding et al., 2020) | Logit margin | 0.28 | 67.49 | 70.91 | 82.56 | 84.36 | 41.44
Learned pseudo-margin | 0.57 | 88.49 | 89.04 | 51.13
XU80 (Xu et al., 2023) | Logit margin | 0.43 | 83.30 | 80.50 | 83.42 | 93.69 | 63.89
Learned pseudo-margin | 0.62 | 93.66 | 90.22 | 33.00

Table 2: Comparison of the correlation and detection performance between the
actual logit margin and the pseudo-margin learned. The models are initially
weakly margin-consistent, but the pseudo-margin learned from feature
representations simulates the margin consistency with higher correlation and
better discriminative power.

##  4 Related Work

Detection tasks in machine learning are found to be of three main types:

  * •

Adversarial Detection The goal of adversarial detection (Xu et al., 2017;
Carlini & Wagner, 2017) is to discriminate adversarial samples from clean and
noisy samples. An adversarial example is a malicious example found by
adversarially attacking a sample; it has a different class while being close
to the original sample. A vulnerable (non-robust) sample is a normal sample
that admits an adversarial example close to it. The two detection tasks are
very distinct. Adversarial detection is a defence mechanism like adversarial
training; Tramer (2022) has established that both tasks are equivalent
problems with the same difficulty.

  * •

Out-of-Distribution (OOD) detection In OOD detection (Hendrycks & Gimpel,
2017; Peng et al., 2024), the objective is to detect instances that have a
different label from the labels on which the model was trained on. For
example, for a model trained on the CIFAR10 dataset, samples from the SVHN
dataset are OOD samples for such a model.

  * •

Misclassification Detection (MisD) It consists in detecting if the
classifier’s prediction is incorrect. This is also referred to as Failure
Detection (Corbière et al., 2019) or Trustworthiness Detection (Jiang et al.,
2018; Luo et al., 2021). MisD is often used for selective classification
(classification with a reject option) (Geifman & El-Yaniv, 2017) to abstain
from predicting samples on which the model is likely to be wrong. A score for
non-robust detection cannot tell if the sample is incorrect, as a vulnerable
sample could be from any side of the decision boundary. Recent work by Mouton
et al. (2024) shows that input margins can predict the generalization gap only
in specific constrained directions that explain the variance of the training
data but not in general.

Formal robustness verification aims at certifying whether a given sample is
$\epsilon$-robust or if it is not an adversarial counter-example can be
provided (Brix et al., 2023b). Some complete exact methods based on solving
Satisfiability Modulo Theory problems (Katz et al., 2017; Carlini et al.,
2017; Huang et al., 2017) or Mixed-Integer Linear Programming (Cheng et al.,
2017; Lomuscio & Maganti, 2017; Fischetti & Jo, 2017) provide formal
certification given enough time. However, in practice, they are tractable only
up to 100,000 activations (Tjeng et al., 2019). Incomplete but effective
methods based on linear and convex relaxation methods and Branch-and-Bound
methods (Zhang et al., 2018; Salman et al., 2019; Xu et al., 2020, 2021; Zhang
et al., 2022; Shi et al., 2023) are faster but conservative, without
guaranteed certifications even if given enough time. Scaling them to larger
architectures such as WideResNets and large Transformers is still challenging
even with GPU accelartion(Brix et al., 2023a; König et al., 2024). Weng et al.
(2018) converts the problem of finding the robust radius (input margin) as a
local Lipschitz constant estimation problem. Computing the Lipschitz constant
of Deep Nets is NP-hard (Virmaux & Scaman, 2018) and Jordan & Dimakis (2020)
proved that there is no efficient algorithm to compute the local Lipschitz
constant. The estimation provided by Weng et al. (2018) requires random
sampling and remains computationally expensive to obtain a good approximation.
Vulnerability detection with margin-consistent models does not provide
certificates but an empirical estimation of the robustness of a sample as
evaluated by adversarial attacks. At scale, it can help filter the samples to
undergo formal verification and a more thorough adversarial attack for
resource prioritization.

##  5 Limitations and Perspectives

Vulnerability detection scope The scope of this work is $\ell_{p}$ robustness
measured by the input space margin; the minimum distortion that changes the
model’s decision while this does not give a full view of the $\ell_{p}$
robustness. Samples may be at the same distance to the decision boundary and
have unequal unsafe neighbourhoods given an average estimation over the
$\epsilon$-neighbourhood considered. The average estimation of local
robustness for a given $\epsilon$-neighborhood remains an open problem, so
whether it is possible to extract other notions of robustness from the feature
representation efficiently could be a potential avenue for further
exploration.

Attack-based verification The margin consistency property does not rely on
attacks; however, its verification and the learning of a pseudo-margin with an
attack-based estimation may not be possible if the model cannot be attacked on
sufficient samples. The implicit assumption is that we can always successfully
provide the closest point to the decision with a sufficient budget. This is a
reasonable assumption since the studied models are not perfectly robust, and
the empirical evidence so far with adaptive attacks is that no defence is
foolproof, which justifies the need to detect the non-robust samples. In some
cases, we might need to combine with an attack such as _CW-attack_ (Carlini &
Wagner, 2016) to find the closest adversarial sample.

Influence of terminal phase of training The work of Papyan et al. (2020) shows
that when deep neural network classifiers are trained beyond zero training
error and beyond zero cross-entropy loss (aka terminal phase of training),
they fall into a state known as _neural collapse_. Neural collapse is a state
where the within-class variability of the feature representations collapses to
their class means, the class means, and the classifiers become self-dual and
converge to a specific geometric structure, an equiangular tight frame (ETF)
simplex, and the network classifier converges to nearest train class center.
This implies that we may lose the margin consistency property. While neural
collapse predicts that all representations collapse on their class mean, in
practice, perfect collapse is not quite achieved, and it is precisely the
divergence of a representation from its class mean (or equivalently its
classifier’s class mean) that encodes the information we seek about the
distance to the decision boundary in the input space. Exploring the impact of
the neural collapse on margin consistency as models tend toward a collapsed
state could provide valuable insights into generalization and adversarial
robustness.

##  6 Conclusion

This work addresses the question of efficiently estimating local robustness in
the $\ell_{p}$ sense at a per-instance level in robust deep neural classifiers
in deployment scenarios. We introduce margin consistency as a necessary and
sufficient condition to use the logit margin of a deep classifier as a
reliable proxy estimation of the input margin for detecting non-robust
samples. Our investigation of various robustly trained models shows that they
have strong margin consistency, which leads to a high performance of the logit
margins in detecting vulnerable samples to adversarial attacks and estimating
robust accuracy on arbitrarily large test sets using only a small subset. We
also find that margin consistency does not always hold, with some models
having a weak correlation between the input margin and the logit margin. In
such cases, we show that it is possible to learn to map the feature
representation to a better-correlated pseudo-margin that simulates the margin
consistency and performs better on vulnerability detection. Finally, we
present some limitations of this work, mainly the scope of robustness, the
attack-based verification and the impact of neural collapse in terminal phases
of training. Beyond its highly practical importance, we see this as a
motivation to extend the analysis of robust models and the properties of their
feature representations in the context of vulnerability detection.

## Acknowledgements

This work is supported by the DEEL Project CRDPJ 537462-18 funded by the
Natural Sciences and Engineering Research Council of Canada (NSERC) and the
Consortium for Research and Innovation in Aerospace in Québec (CRIAQ),
together with its industrial partners Thales Canada inc, Bell Textron Canada
Limited, CAE inc and Bombardier inc.333https://deel.quebec

## References

  * Addepalli et al. (2021) Addepalli, S., Jain, S., Sriramanan, G., Khare, S., and Radhakrishnan, V. B.  Towards achieving adversarial robustness beyond perceptual limits.  In _ICML 2021 Workshop on Adversarial Machine Learning_ , 2021.  URL https://openreview.net/forum?id=SHB_znlW5G7.
  * Addepalli et al. (2022) Addepalli, S., Jain, S., Sriramanan, G., and Venkatesh Babu, R.  Scaling adversarial training to large perturbation bounds.  In _European Conference on Computer Vision_ , pp. 301–316. Springer, 2022.
  * Araujo et al. (2023) Araujo, A., Havens, A. J., Delattre, B., Allauzen, A., and Hu, B.  A unified algebraic perspective on lipschitz neural networks.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=k71IGLC8cfc.
  * Bastani et al. (2016) Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D., Nori, A., and Criminisi, A.  Measuring neural net robustness with constraints.  _Advances in neural information processing systems_ , 29, 2016.
  * Biggio et al. (2013) Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., and Roli, F.  Evasion attacks against machine learning at test time.  In _Joint European conference on machine learning and knowledge discovery in databases_ , pp. 387–402. Springer, 2013.
  * Brix et al. (2023a) Brix, C., Bak, S., Liu, C., and Johnson, T. T.  The fourth international verification of neural networks competition (vnn-comp 2023): Summary and results.  _arXiv preprint arXiv:2312.16760_ , 2023a.
  * Brix et al. (2023b) Brix, C., Müller, M. N., Bak, S., Johnson, T. T., and Liu, C.  First three years of the international verification of neural networks competition (vnn-comp).  _International Journal on Software Tools for Technology Transfer_ , 25(3):329–339, 2023b.
  * Carlini & Wagner (2017) Carlini, N. and Wagner, D.  Adversarial examples are not easily detected: Bypassing ten detection methods.  In _Proceedings of the 10th ACM workshop on artificial intelligence and security_ , pp. 3–14, 2017.
  * Carlini & Wagner (2016) Carlini, N. and Wagner, D. A.  Towards evaluating the robustness of neural networks.  _2017 IEEE Symposium on Security and Privacy (SP)_ , pp. 39–57, 2016.
  * Carlini et al. (2017) Carlini, N., Katz, G., Barrett, C., and Dill, D. L.  Provably minimally-distorted adversarial examples.  _arXiv preprint arXiv:1709.10207_ , 2017.
  * Carmon et al. (2019) Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J. C., and Liang, P. S.  Unlabeled data improves adversarial robustness.  _Advances in neural information processing systems_ , 32, 2019.
  * Chattamvelli (2024) Chattamvelli, R.  _Correlation in Engineering and the Applied Sciences: Applications in R_.  Springer Nature, 2024.
  * Cheng et al. (2017) Cheng, C.-H., Nührenberg, G., and Ruess, H.  Maximum resilience of artificial neural networks.  In _Automated Technology for Verification and Analysis: 15th International Symposium, ATVA 2017, Pune, India, October 3–6, 2017, Proceedings 15_ , pp. 251–268. Springer, 2017.
  * Cisse et al. (2017) Cisse, M., Bojanowski, P., Grave, E., Dauphin, Y., and Usunier, N.  Parseval networks: Improving robustness to adversarial examples.  In _International conference on machine learning_ , pp. 854–863. PMLR, 2017.
  * Corbière et al. (2019) Corbière, C., Thome, N., Bar-Hen, A., Cord, M., and Pérez, P.  Addressing failure prediction by learning model confidence.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Croce & Hein (2020a) Croce, F. and Hein, M.  Minimally distorted adversarial examples with a fast adaptive boundary attack.  In _International Conference on Machine Learning_ , pp. 2196–2205. PMLR, 2020a.
  * Croce & Hein (2020b) Croce, F. and Hein, M.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In _International conference on machine learning_ , pp. 2206–2216. PMLR, 2020b.
  * Croce et al. (2021) Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., and Hein, M.  Robustbench: a standardized adversarial robustness benchmark.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2021.  URL https://openreview.net/forum?id=SSKZPJCt7B.
  * Cui et al. (2023) Cui, J., Tian, Z., Zhong, Z., Qi, X., Yu, B., and Zhang, H.  Decoupled kullback-leibler divergence loss.  _arXiv preprint arXiv:2305.13948_ , 2023.
  * Debenedetti et al. (2023) Debenedetti, E., Sehwag, V., and Mittal, P.  A light recipe to train robust vision transformers.  In _2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)_ , pp. 225–253. IEEE, 2023.
  * Ding et al. (2020) Ding, G. W., Sharma, Y., Lui, K. Y. C., and Huang, R.  Mma training: Direct input space margin maximization through adversarial training.  In _International Conference on Learning Representations_ , 2020.  URL https://openreview.net/forum?id=HkeryxBtPB.
  * Dreossi et al. (2019) Dreossi, T., Ghosh, S., Sangiovanni-Vincentelli, A., and Seshia, S. A.  A formalization of robustness for deep neural networks.  _arXiv preprint arXiv:1903.10033_ , 2019.
  * Elsayed et al. (2018) Elsayed, G., Krishnan, D., Mobahi, H., Regan, K., and Bengio, S.  Large margin deep networks for classification.  _Advances in neural information processing systems_ , 31, 2018.
  * Engstrom et al. (2019) Engstrom, L., Ilyas, A., Salman, H., Santurkar, S., and Tsipras, D.  Robustness (python library), 2019.  URL https://github.com/MadryLab/robustness.
  * Evtimov et al. (2017) Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., and Song, D.  Robust physical-world attacks on machine learning models.  _arXiv preprint arXiv:1707.08945_ , 2(3):4, 2017.
  * Fawzi et al. (2018) Fawzi, A., Fawzi, H., and Fawzi, O.  Adversarial vulnerability for any classifier.  _Advances in neural information processing systems_ , 31, 2018.
  * Fischetti & Jo (2017) Fischetti, M. and Jo, J.  Deep neural networks as 0-1 mixed integer linear programs: A feasibility study.  _arXiv preprint arXiv:1712.06174_ , 2017.
  * Geifman & El-Yaniv (2017) Geifman, Y. and El-Yaniv, R.  Selective classification for deep neural networks.  _Advances in neural information processing systems_ , 30, 2017.
  * Gnanasambandam et al. (2021) Gnanasambandam, A., Sherman, A. M., and Chan, S. H.  Optical adversarial attack.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 92–101, 2021.
  * Goodfellow et al. (2015) Goodfellow, I. J., Shlens, J., and Szegedy, C.  Explaining and harnessing adversarial examples.  In Bengio, Y. and LeCun, Y. (eds.), _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , 2015.  URL http://arxiv.org/abs/1412.6572.
  * Gourdeau et al. (2021) Gourdeau, P., Kanade, V., Kwiatkowska, M., and Worrell, J.  On the hardness of robust classification.  _The Journal of Machine Learning Research_ , 22(1):12521–12549, 2021.
  * Han et al. (2023) Han, T., Srinivas, S., and Lakkaraju, H.  Efficient estimation of local robustness of machine learning models.  In _ICML 3rd Workshop on Interpretable Machine Learning in Healthcare (IMLH)_ , 2023.  URL https://openreview.net/forum?id=ZGSfAElJmp.
  * He et al. (2008) He, C., Wang, C., Zhong, Y.-X., and Li, R.-F.  A survey on learning to rank.  In _2008 International Conference on Machine Learning and Cybernetics_ , volume 3, pp. 1734–1739. Ieee, 2008.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * Hendrycks & Gimpel (2017) Hendrycks, D. and Gimpel, K.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  In _International Conference on Learning Representations_ , 2017.  URL https://openreview.net/forum?id=Hkg4TI9xl.
  * Hendrycks et al. (2019) Hendrycks, D., Lee, K., and Mazeika, M.  Using pre-training can improve model robustness and uncertainty.  In _International conference on machine learning_ , pp. 2712–2721. PMLR, 2019.
  * Huang et al. (2017) Huang, X., Kwiatkowska, M., Wang, S., and Wu, M.  Safety verification of deep neural networks.  In _International conference on computer aided verification_ , pp. 3–29. Springer, 2017.
  * Jiang et al. (2018) Jiang, H., Kim, B., Guan, M., and Gupta, M.  To trust or not to trust a classifier.  _Advances in neural information processing systems_ , 31, 2018.
  * Jordan & Dimakis (2020) Jordan, M. and Dimakis, A. G.  Exactly computing the local lipschitz constant of relu networks.  _Advances in Neural Information Processing Systems_ , 33:7344–7353, 2020.
  * Kannan et al. (2018) Kannan, H., Kurakin, A., and Goodfellow, I.  Adversarial logit pairing.  _arXiv preprint arXiv:1803.06373_ , 2018.
  * Katz et al. (2017) Katz, G., Barrett, C., Dill, D. L., Julian, K., and Kochenderfer, M. J.  Reluplex: An efficient smt solver for verifying deep neural networks.  In _Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30_ , pp. 97–117. Springer, 2017.
  * König et al. (2024) König, M., Bosman, A. W., Hoos, H. H., and van Rijn, J. N.  Critically assessing the state of the art in neural network verification.  _Journal of Machine Learning Research_ , 25(12):1–53, 2024.
  * Li et al. (2019) Li, Q., Haque, S., Anil, C., Lucas, J., Grosse, R. B., and Jacobsen, J.-H.  Preventing gradient attenuation in lipschitz constrained convolutional networks.  _Advances in neural information processing systems_ , 32, 2019.
  * Liu et al. (2021) Liu, F., Han, B., Liu, T., Gong, C., Niu, G., Zhou, M., Sugiyama, M., et al.  Probabilistic margins for instance reweighting in adversarial training.  _Advances in Neural Information Processing Systems_ , 34:23258–23269, 2021.
  * Lomuscio & Maganti (2017) Lomuscio, A. and Maganti, L.  An approach to reachability analysis for feed-forward relu neural networks.  _arXiv preprint arXiv:1706.07351_ , 2017.
  * Luo et al. (2021) Luo, Y., Wong, Y., Kankanhalli, M. S., and Zhao, Q.  Learning to predict trustworthiness with steep slope loss.  _Advances in Neural Information Processing Systems_ , 34:21533–21544, 2021.
  * Madry et al. (2018) Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A.  Towards deep learning models resistant to adversarial attacks.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=rJzIBfZAb.
  * Mickisch et al. (2020) Mickisch, D., Assion, F., Greßner, F., Günther, W., and Motta, M.  Understanding the decision boundary of deep neural networks: An empirical study.  _arXiv preprint arXiv:2002.01810_ , 2020.
  * Moosavi-Dezfooli et al. (2016) Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P.  Deepfool: a simple and accurate method to fool deep neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2574–2582, 2016.
  * Mouton et al. (2024) Mouton, C., Theunissen, M. W., and Davel, M. H.  Input margins can predict generalization too.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 38(13):14379–14387, Mar. 2024.  doi: 10.1609/aaai.v38i13.29351.  URL https://ojs.aaai.org/index.php/AAAI/article/view/29351.
  * Pang et al. (2022) Pang, T., Lin, M., Yang, X., Zhu, J., and Yan, S.  Robustness and accuracy could be reconcilable by (proper) definition.  In _International Conference on Machine Learning_ , pp. 17258–17277. PMLR, 2022.
  * Papyan et al. (2020) Papyan, V., Han, X., and Donoho, D. L.  Prevalence of neural collapse during the terminal phase of deep learning training.  _Proceedings of the National Academy of Sciences_ , 117(40):24652–24663, 2020.
  * Peng et al. (2024) Peng, B., Luo, Y., Zhang, Y., Li, Y., and Fang, Z.  Conjnorm: Tractable density estimation for out-of-distribution detection.  In _Proceedings of the International Conference on Learning Representations_ , 2024.
  * Rade & Moosavi-Dezfooli (2021) Rade, R. and Moosavi-Dezfooli, S.-M.  Helper-based adversarial training: Reducing excessive margin to achieve a better accuracy vs. robustness trade-off.  In _ICML 2021 Workshop on Adversarial Machine Learning_ , 2021.  URL https://openreview.net/forum?id=BuD2LmNaU3a.
  * Rebuffi et al. (2021) Rebuffi, S.-A., Gowal, S., Calian, D. A., Stimberg, F., Wiles, O., and Mann, T.  Fixing data augmentation to improve adversarial robustness.  _arXiv preprint arXiv:2103.01946_ , 2021.
  * Rice et al. (2020) Rice, L., Wong, E., and Kolter, Z.  Overfitting in adversarially robust deep learning.  In _International Conference on Machine Learning_ , pp. 8093–8104. PMLR, 2020.
  * Salman et al. (2019) Salman, H., Yang, G., Zhang, H., Hsieh, C.-J., and Zhang, P.  A convex relaxation barrier to tight robustness verification of neural networks.  _Advances in Neural Information Processing Systems_ , 32:9835–9846, 2019.
  * Sehwag et al. (2021) Sehwag, V., Mahloujifar, S., Handina, T., Dai, S., Xiang, C., Chiang, M., and Mittal, P.  Robust learning meets generative models: Can proxy distributions improve adversarial robustness?  _arXiv preprint arXiv:2104.09425_ , 2021.
  * Serrurier et al. (2021) Serrurier, M., Mamalet, F., González-Sanz, A., Boissin, T., Loubes, J.-M., and Del Barrio, E.  Achieving robustness in classification using optimal transport with hinge regularization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 505–514, 2021.
  * Seshia et al. (2018) Seshia, S. A., Desai, A., Dreossi, T., Fremont, D. J., Ghosh, S., Kim, E., Shivakumar, S., Vazquez-Chanlatte, M., and Yue, X.  Formal specification for deep neural networks.  In _International Symposium on Automated Technology for Verification and Analysis_ , pp. 20–34. Springer, 2018.
  * Shi et al. (2023) Shi, Z., Jin, Q., Kolter, J. Z., Jana, S., Hsieh, C.-J., and Zhang, H.  Formal verification for neural networks with general nonlinearities via branch-and-bound.  _2nd Workshop on Formal Verification of Machine Learning (WFVML 2023)_ , 2023.
  * Szegedy et al. (2014) Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R.  Intriguing properties of neural networks.  In Bengio, Y. and LeCun, Y. (eds.), _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings_ , 2014.  URL http://arxiv.org/abs/1312.6199.
  * Tjeng et al. (2019) Tjeng, V., Xiao, K. Y., and Tedrake, R.  Evaluating robustness of neural networks with mixed integer programming.  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=HyGIdiRqtm.
  * Tramer (2022) Tramer, F.  Detecting adversarial examples is (nearly) as hard as classifying them.  In _International Conference on Machine Learning_ , pp. 21692–21702. PMLR, 2022.
  * Virmaux & Scaman (2018) Virmaux, A. and Scaman, K.  Lipschitz regularity of deep neural networks: analysis and efficient estimation.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Wang et al. (2020) Wang, Y., Zou, D., Yi, J., Bailey, J., Ma, X., and Gu, Q.  Improving adversarial robustness requires revisiting misclassified examples.  In _International Conference on Learning Representations_ , 2020.  URL https://openreview.net/forum?id=rklOg6EFwS.
  * Wang et al. (2023) Wang, Z., Pang, T., Du, C., Lin, M., Liu, W., and Yan, S.  Better diffusion models further improve adversarial training.  In _International Conference on Machine Learning (ICML)_ , 2023.
  * Weng (2019) Weng, T.-W.  Proven: Verifying robustness of neural networks with a probabilistic approach - powerpoint presentation.  https://icml.cc/media/Slides/icml/2019/grandball(11-11-00)-11-12-15-4739-proven_verifyi.pdf, 2019.  (Accessed on 05/23/2023).
  * Weng et al. (2018) Weng, T.-W., Zhang, H., Chen, P.-Y., Yi, J., Su, D., Gao, Y., Hsieh, C.-J., and Daniel, L.  Evaluating the robustness of neural networks: An extreme value theory approach.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=BkUHlMZ0b.
  * Wu et al. (2020) Wu, D., Xia, S.-T., and Wang, Y.  Adversarial weight perturbation helps robust generalization.  _Advances in Neural Information Processing Systems_ , 33:2958–2969, 2020.
  * Xu et al. (2020) Xu, K., Shi, Z., Zhang, H., Wang, Y., Chang, K.-W., Huang, M., Kailkhura, B., Lin, X., and Hsieh, C.-J.  Automatic perturbation analysis for scalable certified robustness and beyond.  _Advances in Neural Information Processing Systems_ , 33, 2020.
  * Xu et al. (2021) Xu, K., Zhang, H., Wang, S., Wang, Y., Jana, S., Lin, X., and Hsieh, C.-J.  Fast and Complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers.  In _International Conference on Learning Representations_ , 2021.  URL https://openreview.net/forum?id=nVZtXBI6LNn.
  * Xu et al. (2017) Xu, W., Evans, D., and Qi, Y.  Feature squeezing: Detecting adversarial examples in deep neural networks.  _arXiv preprint arXiv:1704.01155_ , 2017.
  * Xu et al. (2023) Xu, Y., Sun, Y., Goldblum, M., Goldstein, T., and Huang, F.  Exploring and exploiting decision boundary dynamics for adversarial robustness.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=aRTKuscKByJ.
  * Zhang et al. (2018) Zhang, H., Weng, T.-W., Chen, P.-Y., Hsieh, C.-J., and Daniel, L.  Efficient neural network robustness certification with general activation functions.  _Advances in Neural Information Processing Systems_ , 31:4939–4948, 2018.  URL https://arxiv.org/pdf/1811.00866.pdf.
  * Zhang et al. (2019) Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., and Jordan, M.  Theoretically principled trade-off between robustness and accuracy.  In _International conference on machine learning_ , pp. 7472–7482. PMLR, 2019.
  * Zhang et al. (2022) Zhang, H., Wang, S., Xu, K., Wang, Y., Jana, S., Hsieh, C.-J., and Kolter, Z.  A branch and bound framework for stronger adversarial attacks of ReLU networks.  In _Proceedings of the 39th International Conference on Machine Learning_ , volume 162, pp. 26591–26604, 2022.
  * Zhang et al. (2020) Zhang, J., Zhu, J., Niu, G., Han, B., Sugiyama, M., and Kankanhalli, M.  Geometry-aware instance-reweighted adversarial training.  _arXiv preprint arXiv:2010.01736_ , 2020.
  * Zhong et al. (2021) Zhong, Z., Tian, Y., and Ray, B.  Understanding local robustness of deep neural networks under natural variations.  In _International Conference on Fundamental Approaches to Software Engineering_ , pp. 313–337. Springer, Cham, 2021.
  * Zhu et al. (2023) Zhu, F., Cheng, Z., Zhang, X.-Y., and Liu, C.-L.  Openmix: Exploring outlier samples for misclassification detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 12074–12083, 2023.

##  Appendix A Proof of Theorem 1

######  Theorem 1.

If a model is margin-consistent, then for any robustness threshold $\epsilon$,
there exists a threshold $\lambda$ for the logit margin $d_{out}$ that
perfectly separates non-robust samples and robust samples. Conversely, if for
any robustness threshold $\epsilon$, $d_{out}$ admits a threshold $\lambda$
that perfectly separates non-robust samples from robust samples, then the
model is margin-consistent.

###### Proof.

Formally, for a finite sample $S$ and non-negative values $\epsilon\geq 0$,
$\lambda\geq 0$, we define:

$A^{S}_{\epsilon}:=\\{x\in
S:d_{in}(\mathbf{x})\leq\epsilon\\}\quad\text{and}\quad
B^{S}_{\lambda}:=\\{x\in S:d_{out}(\mathbf{x})\leq\lambda\\}.$

We say that $d_{out}$ perfectly separates non-robust samples from robust
samples if for any finite sample $S\subseteq\mathcal{X}$ and every
$\epsilon\geq 0$ there exists $\lambda\geq 0$ such that
$A^{S}_{\epsilon}=B^{S}_{\lambda}$.

Necessity: Let’s assume that the model is not margin-consistent, i.e., there
exist two samples $\mathbf{x}_{1}$ and $\mathbf{x}_{2}$ such that
$d_{out}(\mathbf{x}_{1})\leq d_{out}(\mathbf{x}_{2})$ and
$d_{in}(\mathbf{x}_{1})>d_{in}(\mathbf{x}_{2})$. By taking
$S=\\{\mathbf{x}_{1},\mathbf{x}_{2}\\}$ and $\epsilon=d_{in}(\mathbf{x}_{2})$
we have that $A^{S}_{\epsilon}=\\{\mathbf{x}_{2}\\}$. However for any
$\lambda\geq 0$, if $\mathbf{x}_{2}\in B^{S}_{\lambda}$, then
$d_{out}(\mathbf{x}_{1})\leq d_{out}(\mathbf{x}_{2})\leq\lambda$ and so
$\mathbf{x}_{1}\in B^{S}_{\lambda}$. Therefore $d_{out}$ does not perfectly
separates non-robust samples from robust samples.

Sufficiency: Let’s assume that the model is margin-consistent. Let $S$ be a
finite sample and consider a threshold $\epsilon$. Let $\mathbf{x}_{0}$ be the
element of the finite set $A^{S}_{\epsilon}$ with maximum
$d_{in}(\mathbf{x}_{0})$ and $d_{out}(\mathbf{x}_{0})$. Since the model is
margin-consistent, then for $\mathbf{x}\in S$:

$x\in A^{S}_{\epsilon}\Leftrightarrow
d_{in}(\mathbf{x})\leq\epsilon\Leftrightarrow\underbrace{d_{in}(\mathbf{x})\leq
d_{in}(\mathbf{x}_{0})\Leftrightarrow d_{out}(\mathbf{x})\leq
d_{out}(\mathbf{x}_{0})}_{\text{margin consistency}}\Leftrightarrow
d_{out}(\mathbf{x})\leq\lambda\Leftrightarrow x\in B^{S}_{\lambda}.$

This means we have $A^{S}_{\epsilon}=B^{S}_{\lambda_{0}},$, which shows that
$d_{out}$ perfectly separates non-robust samples from robust samples. ∎

##  Appendix B Supporting Materials

###  B.1 Input Margins Estimation Sanity Check

One way to verify the reliability of our estimated input margins is to compare
the robust accuracy measured by _AutoAttack_ at $\epsilon=8/255$ and the
proportion of correctly classified test samples with estimated input margins
greater than $\epsilon$; both quantities should be approximately equal which
happens to be the case –See Fig. 8.

(a) CIFAR10

(b) CIFAR100

Figure 8: Robust Accuracy reported in Robustbench vs Robust Accuracy estimated
with the input margins estimations. They are both close, indicating a good
estimation of the input margins.

###  B.2 Detection Performance at Different Values of $\epsilon$

We present in Fig. 9 the performance of the detection for various values of
the robustness threshold. We can see that the strong margin consistency allows
the logit margin to be a good proxy for detection at various thresholds. Note
that below $\epsilon=2/255$ and beyond $\epsilon=16/255$, the ratio of
vulnerable points to non-vulnerable points becomes too imbalanced, with little
to no positive instances beyond $\epsilon=32/255$.

(a) Model AD10 (Addepalli et al., 2021) on CIFAR10

(b) RA11 (Rade & Moosavi-Dezfooli, 2021) on CIFAR100

Figure 9: Variation of AUROC score for different values of the threshold
$\epsilon$.

###  B.3 Sample Efficient Robust Accuracy Estimation

We plot the variation of the absolute error with subset size for the
approximation of the _AutoAttack_ Robust Accuracy by the estimation of
Algorithm 1. Results are presented in Fig. 10 and Fig. 11 for CIFAR10 and
CIFAR100, respectively. From 100 samples, the approximation is already good
for some models.

Figure 10: CIFAR10: Variation of the absolute error, with respect to subset
size. The results are averaged over $10$ variants for each subset size. The
Kendall $\tau$ approximation is also given above the boxplot. The correlation
estimation does not vary a lot with sample size. In general, the variance of
the absolute error and the value decreases with the increase in subset size.
In most cases, the average is already below $2$ from 300 samples. The two
models with low margin consistency (DI0 and XU8) also have a bad
approximation. Figure 11: CIFAR100: Variation of the absolute error with
respect to subset size. The results are averaged over $10$ variants for each
subset size. The Kendall $\tau$ approximation is also given above the boxplot.
The correlation estimation does not vary a lot with sample size. In general,
the variance of the absolute error and the value decreases with the increase
in subset size. In most cases, the average is already below $2$ from 300
samples.

###  B.4 Pseudo-margin Learning Setup

The architecture and learning setup for the pseudo-margin is inspired from
Corbière et al. (2019). A multilayer perceptron (Fig. 12) learns a pseudo-
margin from the feature representations of the samples by minimizing the mean-
squared error loss between the output pseudo-margin and an estimation of the
input margin.

Figure 12: Learning Setup for pseudo-margin inspired from Corbière et al.
(2019).

###  B.5 Verification of Equidistance Assumption of the Linear Classifiers

Eq. 3 in Sec. 2.1 shows that we can approximate the margin in the feature
space by the logit margin if the classifiers $\mathbf{w_{j}}$ are equidistant,
i.e. $\|w_{i}-wj\|=C,\,\forall i,j\in\\{1,..,C\\}$. For each model, we
computed the $\frac{C(C-1)}{2}$ possible values of the distances between pairs
of classifiers ($45$ for CIFAR10 and $4950$ for CIFAR100). We confirm this
hypothesis for our investigated models in Fig. 13 by plotting the boxplot of
the distribution of values. For each model, the values vary only in a small
range.

(a) CIFAR10

(b) CIFAR100

Figure 13: Equidistance of classifiers: the boxplot reports the distances’
minimum value, lower quartile (Q1), median, upper quartile (Q3), and maximum
value. The interquartile range is $Q3-Q1$ is small enough for most of the
models.


# Learning to Estimate Kernel Scale and Orientation of Defocus Blur with
Asymmetric Coded Aperture

###### Abstract

Consistent in-focus input imagery is an essential precondition for machine
vision systems to perceive the dynamic environment. A defocus blur severely
degrades the performance of vision systems. To tackle this problem, we propose
a deep-learning-based framework estimating the kernel scale and orientation of
the defocus blur to adjust lens focus rapidly. Our pipeline utilizes 3D
ConvNet for a variable number of input hypotheses to select the optimal slice
from the input stack. We use random shuffle and Gumbel-softmax to improve
network performance. We also propose to generate synthetic defocused images
with various asymmetric coded apertures to facilitate training. Experiments
are conducted to demonstrate the effectiveness of our framework.

Index Terms—  Blur kernel estimation, focus adjustment, asymmetric coded
aperture

##  1 Introduction

Defocus blur is common in the digital image capture process but unwelcome in
the machine vision system, as it degrades features and leads to a system
accuracy drop. The imaging techniques and optical performance have improved
rapidly during the past decade, but the defocus problem remains to be solved.
Consumer cameras handle this problem by introducing autofocus functions. In
contrast, cameras like surveillance cameras lack autofocus function or have a
fixed focus distance. While seeing remarkable results on curated web image
datasets like ImageNet [1], a real-world application often fails to achieve a
comparable accuracy using footage frequently contains out-of-focused imagery.
In this work, we took a step further toward real-world application and
proposed an end-to-end framework to estimate the defocus blur scale and
orientation for auto lens focus adjustment.

Conventional methods to solve out-of-focus problems can be categorized into
two approaches: introducing a focus tracking mechanism [2] or performing a
deblurring algorithm [3][4] to recover the all-in-focus image. A focus
tracking mechanism enables the vision system to capture a focused image
directly, in which the optimal location of the focal plane should be
determined in advance in order to adjust the lens focus. The focal plane
location can be determined either by estimating the blur kernel [3] or object
distance [5] (estimation method) or through repetitive tuning [2] (trial and
error method), i.e., dialing the focal plane back and forth. We notice that a
lens adjustment could perform accurately without iteration yielding lower
delay using the estimation method. In comparison, the trial and error method
takes a few frames to converge, making the estimation method an ideal option.

In computational photography literature, coded aperture techniques are used
for both defocus deblurring [6][7] and depth estimation [3][8]. Though the
deblurring functionality seems to solve the defocus problem straightforwardly,
an incorrect point spread function(PSF) used in the deconvolution can cause
more artifacts against our original intention. Meanwhile, whether deblurred
images can deliver the same precision on machine vision tasks as in-focus
images do stay unknown. The other practice of coded apertures, depth
estimation, can well serve the blur scale estimation in our framework, though
very few efforts have been put to distinguish which side of the object rests
with respect to the focal plane, which causes the focal plane ambiguity in
real-world applications.

Inspired by previous works [7][8], we tackle the defocus blur problem by
estimating blur scale and orientation to guide the lens focus mechanism. With
a predicted defocus blur scale and orientation, the optical system can respond
with rapid lens adjustment, yielding lower delay and better focus accuracy. To
achieve this, we propose a novel end-to-end framework with a 3D ConvNet to
predict the defocus blur scale and orientation at the same time. We use
asymmetric coded apertures to better solve the focal plane ambiguity problem
neglected in previous research. Our proposed pipeline computes a stack of
deblurred images with varied PSF hypotheses as input and predicts each pixel’s
correct blur label (i.e., blur size and orientation). With a 3D ConvNet
design, our method is applicable to different numbers of blur scale and
orientation proposals. A full discussion of such design is conducted in
Section 3. The contribution of our work is three-fold: First, we propose a
novel deep-learning-based pipeline to construct a deblurred defocused image
volume and estimate the scale and orientation from it. Second, we present a
defocused image generation method to address the data-hungry problem. Third,
we conduct extensive experiments to examine the robustness of this design
among different conditions. ††footnotetext: We acknowledge funding from the
Shenzhen Science and Technology Program(Grant No.KQTD20190929172829742).

##  2 Related Work

Substantial research has been laid out on blur parameter estimations. In this
section, we discuss the methodology foundation, as well as its limitations.

A defocus blur is caused by object distances away from the camera focal plane.
In order to establish a one-to-one correspondence between the object distance
and the defocus blur, we must solve the focal plane ambiguity, i.e.,
distinguish which side of the object rests with respect to the focal plane.
This ambiguity is non-trivial since the commonly used circular aperture
employed in cameras will produce identical blur for items resting before and
behind the focal plane. In [8], Sellen et al. detailed how asymmetric coded
apertures can solve this ambiguity by creating different blur for items at
each side of the focal plane. Following this scheme, we adopted an asymmetric
coded aperture in our system design, though our pipeline is not rigged with
one particular aperture.

Estimating the object distance or blur scale is a sub-topic of the well-
studied coded aperture dilemma. Numerous coded apertures
[3][7][6][8][9][10][11][12] have been proposed for different intentions. As
[8] points out, scenarios that object located at both sides of the focal plane
have not been discussed in previous studies. Moreover, these methods are
paired with a particular aperture design, which means variant on PSF caused by
lens distortion will affect its performance. With a predicted PSF, defocus
deblurring can be performed in either frequency domain [13][4] or spatial
domain [3][14]. Although the frequency domain process is more computation
friendly, methods performed in the spatial domain like the algorithm in [3]
produce better perceptual results.

In this work, we designed our pipeline to avoid these limitations. Our
framework generates a deblurred image stack from the defocused image captured
using one of above mentioned asymmetric coded apertures. As no extra
modification is needed when altering coded apertures or deblurring algorithms
in the proposed method, we decoupled the performance dependency from them. We
showed that our system could achieve similar performance on different
asymmetric coded apertures with two separate deblurring algorithms in Section
4.

The topic Depth from Defocus (DFD) [15] is similar to blur scale estimation as
well as recent research on defocus map estimation [16][17] or defocus blur
detection [18][19], which also involves the consideration of blur scale
estimation. As these topics themselves are broad studies and neglect to
address the focal plane ambiguity, we did not compare them in this discussion.

##  3 Methodology

###  3.1 Problem Formulation

The input of our pipeline is a spatially-varying defocused image. Spatially-
varying indicates the blur at each pixel may differ from the neighbor in terms
of scale and orientation. It has been shown in [20] that performing a
deblurring algorithm with the wrong kernel yields low-quality images while the
correct one produces high-quality results. Using this insight, we formulate
blur kernel estimation to separate a correct deblurred result from the wrong
ones to teach the neural network how to choose the right kernel scale and
orientation. Thus, in our framework, we first perform a deblurring algorithm
with PSF hypotheses (varying in scale and orientation) on a defocused image.
These results are then concatenated and fed into the network, which is trained
to identify the optimal deblurred result index for each pixel. The PSF kernel
associated with the optimal deblurred image quality reports the correct scale
and orientation.

Our training target can be formulated as an optimization problem:

$\displaystyle\min_{\theta}{\cal L}_{c}($ $\displaystyle
pred_{\theta},RS(gt))+\lambda_{s}{\cal L}_{s}(GS(pred_{\theta})),$ (1)
$\displaystyle pred_{\theta}=f_{\theta}(RS(d_{1},d_{2},\dots,d_{n},dI)),$

where a weighted sum of cross-entropy loss ${\cal L}_{c}$ and smoothness loss
${\cal L}_{s}$ is minimized. The network (denoted as $f_{\theta}(\cdot)$, with
model parameters $\theta$) takes the deblurred results
$d_{1},d_{2},\dots,d_{n}$ and the original defocused image $dI$ as input. We
perform random shuffle (denoted as $RS(\cdot)$) on the input and the ground-
truth label $gt$ to avoid overfitting and impose a smoothness loss on the
network output $pred_{\theta}$ for better performance. In computing the
smoothness loss, we apply the Gumbel-Softmax [21] technique (denoted as
$GS(\cdot)$) in order to convert the 3D output $pred_{\theta}$ into 2D. The
smoothness loss is then weighted by $\lambda_{s}$. We detailed the motivation
to perform random shuffle and the Gumbel-softmax methods in Section 3.2.

###  3.2 Blur Kernel Estimation

Input and Output With a defocused image as input, we first perform a
deblurring algorithm with several PSF hypotheses to form the deblurred image
stack. We adopt the deblurring approach proposed in [3], where the image is
deblurred in the spatial domain using the Conjugate Gradient method, as well
as the method in the frequency domain using Wiener Deconvolution. There is no
extra modification that needs to be made in our framework when changing the
deblurring algorithm. The deblurred image stack is a concatenation along depth
channel of $d_{1},d_{2},$ $\dots,d_{n}$ and $dI$ with dimension [height,
width, depth, channel]. We further append the stack with $23-n$ black padding
(pixel values with all 0) along the depth dimension, making the depth always
24 for consistent behavior with variable deblurring hypotheses $n$. The
defocused image $dI$ is also included in the input, as some pixels presented
in $dI$ may already be of optimal quality, and there will be no better options
in the deblurred image stack.

The network output is a logit volume with a size of [height, width, depth].
Vectors along the depth dimension reveal the likelihood that slices in the
input are sharp (high-quality) or not. The final blur scale and orientation
for each pixel can be determined by looking up the blur kernel parameter
associated with the highest logit value.

Architecture We design a fully convolutional network followed the architecture
in [22], which utilizes the 3D ConvNet to handle the input with a variable
length of depth. Our network is trained to award the image with optimal
quality in the input stack while punishing lower-quality ones. A 3D ConvNet
design suits this purpose perfectly, as sliding the convolution kernel along
the depth dimension imitates the behavior of examining the image one by one.
In our scheme, each 3D convolution layer is followed by one layer-
normalization layer [23]. The architecture is detailed in Table. 1.

Layer | s | d | n | depth | in | out | input
---|---|---|---|---|---|---|---
conv1_1 | 1 | 1 | 8 | 24/24 | 1 | 1 | stack
conv1_2 | 2 | 1 | 16 | 24/12 | 1 | 2 | conv1_1
conv2_1 | 1 | 1 | 16 | 12/12 | 2 | 2 | conv1_2
conv2_2 | 2 | 1 | 32 | 12/6 | 2 | 4 | conv2_1
conv3_1 | 1 | 1 | 32 | 6/6 | 4 | 4 | conv2_2
conv3_2 | 1 | 1 | 32 | 6/6 | 4 | 4 | conv3_1
conv3_3 | 2 | 1 | 64 | 6/3 | 4 | 8 | conv3_2
conv4_1 | 1 | 2 | 64 | 3/3 | 8 | 8 | conv3_3
conv4_2 | 1 | 2 | 64 | 3/3 | 8 | 8 | conv4_1
conv4_3 | 1 | 2 | 64 | 3/3 | 8 | 8 | conv4_2
nnup_5 |  |  |  | 3/6 | 8 | 4 | conv3_3+conv4_3
conv5_1 | 1 | 1 | 32 | 6/6 | 4 | 4 | nnup_5
conv5_2 | 1 | 1 | 32 | 6/6 | 4 | 4 | conv5_1
conv5_3 | 1 | 1 | 32 | 6/6 | 4 | 4 | conv5_2
nnup_6 |  |  |  | 6/12 | 4 | 2 | conv2_2+conv5_3
conv6_1 | 1 | 1 | 16 | 12/12 | 2 | 2 | nnup_6
conv6_2 | 1 | 1 | 16 | 12/12 | 2 | 2 | conv6_1
nnup_7 |  |  |  | 12/24 | 2 | 1 | conv1_2+conv6_2
conv7_1 | 1 | 1 | 8 | 24/24 | 1 | 1 | nnup_7
conv7_2 | 1 | 1 | 8 | 24/24 | 1 | 1 | conv7_1
conv7_3 | 1 | 1 | 1 | 24/24 | 1 | 1 | conv7_2
|  |  |  |  |  |  |

Table 1: Our proposed network architecture. All convolution kernels have a
size of 3. Column s is the stride, d the kernel dilation, n the number of
filters, depth the number of layer input/output depth, in and out the
accumulated stride of layer input/output, input the layer input with + meaning
concatenation along the channel dimension. Layers starting with “nnup” perform
2x nearest neighbor upsampling.

Data Augmentation We expect the network to identify the blur kernel by
examining input images’ quality. However, preliminary experiments show
overfitting trends when changing the input stack order, which means the
network exploits the ordering pattern but not the image quality we intended.
We perform a random shuffle on the input stack and the label map along the
depth dimension during training in improved schemes, and the overfitting issue
is resolved.

Loss Our network training is formulated as a classification problem. We index
each proposed blur kernel employed in the deblurring step and compute the
predicted softmax from the output logit volume. The cross-entropy ${\cal
L}_{c}$ between the label and the prediction can be computed and serve as our
primary loss.

We expect the estimated blur map to be smooth as the blur scale and
orientation share the same continuous character as scene depth does in most
areas. To compute smoothness loss on our output, we need to convert the 3D
logit volume into a 2D indexing map. As the $\arg\max$ operation to flat a 3D
volume into 2D is not feasible here with its non-differentiable property, we
apply Gumbel-Softmax [21] to fulfill this need. It takes a logit vector as
input, and outputs the index of the element associated with the largest logit
value. Then, we compute ${\cal L}_{s}$ through taking the absolute average of
the first-order gradient of the 2D indexing map. The final loss ${\cal L}$ is
a weighted sum of ${\cal L}_{c}$ and ${\cal L}_{s}$: ${\cal L}={\cal
L}_{c}+\lambda_{s}{\cal L}_{s}$, where we weight ${\cal L}_{s}$ by
$\lambda_{s}$.

##  4 Experiments

###  4.1 Data Generation

Our training procedure needs a large portion of spatially-varying defocused
images, each with a per-pixel ground-truth blur scale and orientation to
supervise the network training. With very few previous works sharing coded
aperture images, none of them involves scene objects resting on both sides of
the focal plane. As we intend to address the focal plane ambiguity problem, we
need to generate a new dataset to meet the need. Here we present an approach
to acquire defocused images from CG generated all-in-focus images.

All-in-Focus Image and Defocus Generation We generate all-in-focus images each
with its depth map using UnrealCV [24]. Defocused images are further generated
following the thin-lens model (Fig. 1) from all-in-focus images, similar to
one described in [17]. With setting the focal plane at distance $S_{1}$, the
circle of confusion (COC) diameter $c(x)$ for image pixel with depth $x$ can
be determined according to Eq. 2.

$\centering
c(x)=\alpha\frac{x-S_{1}}{x},\,\,with\,\,\alpha=\frac{f_{1}}{S_{1}}D\@add@centering$
(2)

Fig. 1: The thin-lens Model. $S_{1}$ is the object-space focal distance,
$f_{1}$ the image-space focal distance, $F$ the focal length, $D$ the diameter
of aperture. $c(x)$ is the COC diameter for object with distance $x$.

The sign and magnitude of $c(x)$ indicate the orientation and the scale of the
blur kernel, respectively. With an all-in-focus image ${I}_{k}$ and
coresponding depth map ${d}_{k}$, for each pixel ${p}_{i}\in{I}_{k}$ and its
depth value ${x}_{{p}_{i}}$, we can compute the blur parameter
$c({x}_{{p}_{i}})$ according to Eq. 2. We round
$\\{c({x}_{{p}_{i}})|\forall{p}_{i}\in{I}_{k}\\}$ to obtain a set of
$\\{{c}_{{I}_{k}}\\}$ where
$[c({x}_{{p}_{i}})]\in\\{{c}_{{I}_{k}}\\},\forall{p}_{i}\in{I}_{k}$. Here
$[\cdot]$ denotes the rounding operation. With ${c}_{i}\in\\{{c}_{{I}_{k}}\\}$
, we resize, flip, and normalize the coded aperture to form a PSF
${F}_{{c}_{i}}$ according to ${c}_{i}$. Convolution is then performed on each
pixel ${p}_{i}$ in the all-in-focus image ${I}_{k}$ with kernel
${F}_{[c({x}_{{p}_{i}})]}$ to form the spatially-varying defocused image
$\hat{{I}}_{k}$. In addition, we save
$[c({x_{{p}_{i}}})],\forall{p}_{i}\in{I}_{k}$ as the per-pixel ground-truth
label for the training process. To avoid producing unrealistic blurry images,
we restrict the maximum blur size during defocus generation.

We capture 100 all-in-focus images from scenes provided in UnrealCV, and
generate defocused images for each one with 12 different focal distances
($S_{1}$ in Fig. 1). Thus 1200 defocused images are acquired totally with each
coded aperture we used. The maximum blur scale is restricted to be 7, 9, and
10 pixels to form the depth-13, depth-17, depth-19 dataset, respectively. We
later use these datasets vary in coded apertures and maximum blur scales to
verify our framework’s generalization performance.

(a) Coded aperture from [8]

(b) Coded aperture [7]

Fig. 2: Asymmetric coded apertures used in experiments

Fig. 3: Qualitative results of two scene picked from our dataset. The middle
column is the groud-truth label while the right column is predicted by our
framework

In our experiments, we split the entire dataset for each coded aperture and
maximum blur scale to 936 defocused images for training, 108 for validation,
and 156 for evaluation. The split is implemented in such a manner that there
are no overlapping scenes across the training/validation/evaluation subsets,
which means all 12 defocused images generated from one particular scene are
forced to belong to one of these three subsets.

###  4.2 Implementiation Details

We implement our proposed network architecture in Table. 1 using TensorFlow
API [25]. In the training procedure, the network is optimized by the SGD
optimizer with the random shuffle applied on the input stack and label. The
learning rate, batch size, and smoothness weight $\lambda_{s}$ are 0.01, 1,
and 0.1, respectively. The temperature argument for the Gumbel-softmax
function is 0.5. We train the network on a single GPU (GeForce RTX 2080 Ti),
and the network takes around five days to converge.

###  4.3 Evaluation

We formulate our blur kernel estimation as a classification problem. Logit
volume from the network output indicates whether pixels in the deblurred stack
are classified as optimal-quality or not. The predicted blur scale and
orientation can be determined by indexing the highest value along the depth
channel of the logit volume. We report the prediction N-1 accuracy and the N-3
accuracy in the following manner: N-1 accuracy indicates the percentage of
pixels holds the same predicted blur scale and orientation as the ground-truth
label does. The N-3 accuracy is computed similar to N-1 but also counting the
nearby prediction (i.e., the pixel is predicted within the range of its
ground-truth label $\pm 1$) as a correct prediction. For example, pixel with
label +3 predicted as +2 (or +4) is not counted as a correct prediction in N-1
but in N-3. The intention behind reporting the N-3 accuracy is to compensate
for the inaccuracy introduced during rounding COC diameter $c(x)$. The N-3
accuracy also registers how effective this framework will perform in a real-
world application. Since even the prediction is off by 1 pixel, adjusting
focus with it will still result in a much more in-focus image.

| depth-13 | depth-17 | depth-19
---|---|---|---
N-1 Accuracy | 63.08% | 63.47% | 63.81%
N-3 Accuracy | 82.90% | 82.93% | 85.59%
Table 2: Evaluation on input with different valid hypotheses. The model is trained on depth-17, and is evaluated on depth-13, 17 and 19. | SoS | SoZ | ZoZ (fine-tuned)
---|---|---|---
N-1 Accuracy | 63.47% | 54.16% | 59.17%
N-3 Accuracy | 82.93% | 79.00% | 81.47%
Table 3: Evaluation results on dataset generated with different asymmetric
coded apertures. SoS means network is trained and evaluated on coded aperture
proposed by Sellent et al. [8] . SoZ means trained on Sellent’s but evaluated
on Zhou’s coded aperture[7]. ZoZ (fine-tuned) shows the results where the
model is pre-trained on Sellent’s then fine-tuned and evaluated on Zhou’s
coded aperture.

Evaluation on different number of blur hypotheses As shown in Table. 2, we
demonstrate our framework’s performance to handle the input with a variable
number of blur hypotheses. Our model is trained on dataset depth-17 and
reports an N-3 accuracy of 82.9%, 82.93%, and 85.59% on evaluation subsets of
datasets depth-13, 17, and 19, respectively. The similar high percentage
results among three datasets show our framework is able to generalize over
input with a variable number of blur hypotheses and estimate the blur scale
and orientation of defocused image with high accuracy. To challenge our
network’s generalization capability even more, we alter the coded aperture
used in the dataset generation and deblurring step. In particular, we train
our model on the dataset generated using Sellent’s coded aperture and evaluate
on two separate datasets generated using Sellent’s and Zhou’s coded aperture
(Fig. 2 (a) and (b)). In addition, Sellent’s coded aperture is designed for
depth estimation, while Zhou’s is intended for defocus deblurring. The
evaluation accuracy is presented in Table. 3. Although we notice the results
of SoZ are lower than SoS, dropping from 82.93% to 79% in the N-3 accuracy, we
observe improved accuracy in ZoZ after the network fine-tuned on the dataset
corresponding to Zhou’s coded aperture. The performance gap between SoS and
ZoZ indicates the coded aperture chosen in the framework would affect the
overall performance. Nevertheless, we do not recognize a strong dependency of
robust system performance on a particular coded aperture, thus proving our
network’s generation capability.

Evaluation on different coded apertures We also assess the network performance
with two deblurring algorithms generating the input stack from defocused
images. These two algorithms are the spatial-domain method proposed in [3] and
the Wiener Deconvolution, which is a frequency-domain method. While the model
is trained with [3] generating the deblurred image stack, experiments show
that swapping to the Wiener Deconvolution algorithm does not influence the
performance: The N-1 accuracy of evaluating on spatial-domain vs. frequency-
domain deblurring algorithm is 63.47% vs. 62.48%, and N-3 accuracy of 82.90%
vs. 82.64%, showing that our framework is insensitive to the deblurring
algorithm type.

Qualitative evaluation Some qualitative results are exhibited in Fig. 3. We
plot the blur map similar to a depth map that pixels with similar colors
indicate a similar blur parameter. Our network can generally discriminate
areas with different blur kernels and classify them accurately. Some pixels
are misclassified to a kernel closer to the true one. This is because
deblurring with a kernel similar to ground-truth yields a sub-optimal image
quality but can be hardly distinguished by the network. The network can not
always discriminate between an optimal result (deblurred by the true kernel)
and a sub-optimal result. Nevertheless, we achieve 82.93% N-3 accuracy. This
indicates that our network can identify the optimal results and assign a high
probability to them.

After further analyzing the incorrect prediction, which significantly differs
from its neighbor region’s prediction and the ground-truth label, such as the
bottom right part of the first row (dark area) in Fig. 3, we find that a large
piece of them comes from the texture-less region in the all-in-focus image. We
hold this is an ill-posed challenge as defocus texture-less images with
various blur scale and orientation show a tiny variation in between. Also, in
real-world applications, a texture-less region can hardly be a region of
interest.

##  5 Conclusion

In this paper, we present a deep-learning-based framework to estimate blur
scale and orientation from a defocused image to assist rapid focus adjustment.
Benefiting from the 3D ConvNet architecture, our pipeline is able to process a
variable depth of deblurred candidates across complex scenarios. We employ a
random shuffle technique to avoid overfitting and adopted the Gumbel-softmax
method with smoothness constraint to improve the performance. Experiments show
that our approach achieves reliable accuracy per-pixel blur scale and
orientation results. When examined on different coded aperture datasets, our
framework generalizes well to achieve above 80% N-3 accuracy after fine-
tuning. Furthermore, our system performance does not depend on a particular
deblurring algorithm or coded aperture.

## References

  * [1] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei,  “Imagenet: A large-scale hierarchical image database,”  in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255.
  * [2] Jie He, Rongzhen Zhou, and Zhiliang Hong,  “Modified fast climbing search auto-focus algorithm with adaptive step size searching technique for digital camera,”  IEEE transactions on Consumer Electronics, vol. 49, no. 2, pp. 257–262, 2003.
  * [3] Anat Levin, Rob Fergus, Frédo Durand, and William T Freeman,  “Image and depth from a conventional camera with a coded aperture,”  ACM transactions on graphics (TOG), vol. 26, no. 3, pp. 70–es, 2007\.
  * [4] Leon B Lucy,  “An iterative technique for the rectification of observed distributions,”  The astronomical journal, vol. 79, pp. 745, 1974.
  * [5] Andreas N Dorsel, Kenneth L Staton, Cassandra Dey, and George P Tsai,  “Apparatus and method for autofocus,” Nov. 26 2002,  US Patent 6,486,457.
  * [6] Changyin Zhou, Stephen Lin, and Shree Nayar,  “Coded aperture pairs for depth from defocus,”  in 2009 IEEE 12th international conference on computer vision. IEEE, 2009, pp. 325–332.
  * [7] Changyin Zhou and Shree Nayar,  “What are good apertures for defocus deblurring?,”  in 2009 IEEE international conference on computational photography (ICCP). IEEE, 2009, pp. 1–8.
  * [8] Anita Sellent and Paolo Favaro,  “Which side of the focal plane are you on?,”  in 2014 IEEE international conference on computational photography (ICCP). IEEE, 2014, pp. 1–8.
  * [9] Max Grosse, Gordon Wetzstein, Anselm Grundhöfer, and Oliver Bimber,  “Coded aperture projection,”  ACM Transactions on Graphics (TOG), vol. 29, no. 3, pp. 1–12, 2010\.
  * [10] Hajime Nagahara, Changyin Zhou, Takuya Watanabe, Hiroshi Ishiguro, and Shree K Nayar,  “Programmable aperture camera using lcos,”  in European Conference on Computer Vision. Springer, 2010, pp. 337–350.
  * [11] Jingyu Lin, Xing Lin, Xiangyang Ji, and Qionghai Dai,  “Separable coded aperture for depth from a single image,”  IEEE Signal Processing Letters, vol. 21, no. 12, pp. 1471–1475, 2014.
  * [12] Yosuke Bando, Bing-Yu Chen, and Tomoyuki Nishita,  “Extracting depth and matte using a color-filtered aperture,”  in ACM SIGGRAPH Asia 2008 papers, pp. 1–9. 2008.
  * [13] William Hadley Richardson,  “Bayesian-based iterative method of image restoration,”  JoSA, vol. 62, no. 1, pp. 55–59, 1972.
  * [14] Anat Levin, Yair Weiss, Fredo Durand, and William T Freeman,  “Understanding and evaluating blind deconvolution algorithms,”  in 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009, pp. 1964–1971.
  * [15] Subhasis Chaudhuri and Ambasamudram N Rajagopalan,  Depth from defocus: a real aperture imaging approach,  Springer Science & Business Media, 2012.
  * [16] Shaojie Zhuo and Terence Sim,  “Defocus map estimation from a single image,”  Pattern Recognition, vol. 44, no. 9, pp. 1852–1858, 2011.
  * [17] Junyong Lee, Sungkil Lee, Sunghyun Cho, and Seungyong Lee,  “Deep defocus map estimation using domain adaptation,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 12222–12230.
  * [18] Jianping Shi, Li Xu, and Jiaya Jia,  “Just noticeable defocus blur detection and estimation,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 657–665.
  * [19] Ming Ma, Wei Lu, and Wenjing Lyu,  “Defocus blur detection via edge pixel dct feature of local patches,”  Signal Processing, p. 107670, 2020.
  * [20] Mina Masoudifar and Hamid Reza Pourreza,  “Image and depth from a single defocused image using coded aperture photography,”  arXiv preprint arXiv:1603.04046, 2016.
  * [21] Eric Jang, Shixiang Gu, and Ben Poole,  “Categorical reparameterization with gumbel-softmax,”  arXiv preprint arXiv:1611.01144, 2016.
  * [22] Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar,  “Local light field fusion: Practical view synthesis with prescriptive sampling guidelines,”  ACM Transactions on Graphics (TOG), 2019.
  * [23] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton,  “Layer normalization,”  arXiv preprint arXiv:1607.06450, 2016.
  * [24] Weichao Qiu, Fangwei Zhong, Yi Zhang, Siyuan Qiao, Zihao Xiao, Tae Soo Kim, and Yizhou Wang,  “Unrealcv: Virtual worlds for computer vision,”  in Proceedings of the 25th ACM international conference on Multimedia, 2017, pp. 1221–1224.
  * [25] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al.,  “Tensorflow: a system for large-scale machine learning.,”  in OSDI, 2016, vol. 16, pp. 265–283.