# Smoothed Embeddings for Certified Few-Shot Learning

Mikhail Pautov
Skolkovo Institute of Science and Technology
&Olesya Kuznetsova
Skolkovo Institute of Science and Technology
&Nurislam Tursynbek
University of North Carolina at Chapel Hill
&Aleksandr Petiushko
Moscow State University, Nuro, Inc.
&Ivan Oseledets
Skolkovo Institute of Science and Technology, AIRI
The code for this paper is available at our repository.

###### Abstract

Randomized smoothing is considered to be the state-of-the-art provable defense
against adversarial perturbations. However, it heavily exploits the fact that
classifiers map input objects to class probabilities and do not focus on the
ones that learn a metric space in which classification is performed by
computing distances to embeddings of classes prototypes. In this work, we
extend randomized smoothing to few-shot learning models that map inputs to
normalized embeddings. We provide analysis of Lipschitz continuity of such
models and derive robustness certificate against $\ell_{2}$-bounded
perturbations that may be useful in few-shot learning scenarios. Our
theoretical results are confirmed by experiments on different datasets.

##  1 Introduction

Few-shot learning is a setting in which a classification model is evaluated on
the classes not seen during the training phase. Nowadays quite a few few-shot
learning approaches based on neural networks are known. Unfortunately, neural
network based classifiers are intimately vulnerable to adversarial
perturbations [41, 10] – accurately crafted small modifications of the input
that may significantly alter the model’s prediction. This vulnerability poses
a restriction on the deployment of such approaches in safety-critical
scenarios, so the research interest in the field of attacks on neural networks
has been great in recent years.

Several works studied this phenomena in different practical applications –
image classification [2, 34, 33, 40], object detection [17, 26, 48, 50], face
recognition [18, 5, 54], semantic segmentation [7, 12, 50]. These studies show
that it is easy to force a model to behave in the desired way by applying
imperceptible change to its input. As a result, defenses, both empirical [4,
55, 14] and provable [51, 21, 3, 47, 52, 15, 46, 35], were proposed recently.
Although empirical ones can be (and often are) broken by more powerful
attacks, the provable ones are of a big interest since they make it possible
to provide _guarantees_ of the correctness of the work of a model under
certain assumptions, and, thus, possibly broaden the scope of tasks which may
be trusted to the neural networks.

Randomized smoothing [21, 3, 25] is the state-of-the-art approach used for
constructing classification models provably robust against small-norm additive
adversarial perturbations. This approach is scalable to large datasets and can
be applied to any classifier since it does not use any assumptions about
model’s architecture. Generally, the idea is following. Suppose, we are given
a base neural network classier $f:\mathbb{R}^{D}\to\left[0,1\right]^{K}$ that
maps an input image $x$ to a fixed number of $K$ class probabilities. Its
smoothed version with the standard Gaussian distribution is:

$g(x)=\underset{\varepsilon\sim\mathcal{N}(0,\Sigma)}{\mathbb{E}}f(x+\varepsilon).$
(1)

As shown in [3], the new (smoothed) classifier is provably robust at $x$ to
$\ell_{2}$-bounded perturbations if the base classifier $f$ is confident
enough at $x$. However, the proof of certification heavily exploits the fact
that classifiers are restricted to map an input to a fixed number of class
probabilities. Thus, directly applying randomized smoothing to classifiers in
metric space, such as in few-shot learning, is a challenging task.

There are several works that aim at improving the robustness for few-shot
classification [20, 9, 53, 28]. However, the focus in such works is either on
the improvement of empirical robustness or probabilistic guarantees of
certified robustness; none of them provide theoretical guarantees on the
worst-case model behavior.

$n$Image $x$Noisy imagesModel
$f(\cdot)$$\frac{1}{n}\sum\frac{f(x+\varepsilon)}{\|f(x+\varepsilon)\|_{2}}$Smoothed
embeddingsAdversarial embedding risk $\gamma$Lipschitz constant
$L$$\|\delta\|_{2}<\frac{\gamma}{L}?$YesNoAbstain fromclassificationCertify

Figure 1: Illustration of certification pipeline for a single image $x$. Given
a model $f(\cdot)$ and $n$ realisations of zero-mean Gaussian noise
$\varepsilon_{1},\dots,\varepsilon_{n}\sim\mathcal{N}(0,\sigma^{2}I)$, an
estimate $\hat{g}(x)=\frac{1}{n}\sum_{i=1}^{n}f(x+\varepsilon_{i})$ of
$g(x)=\mathbb{E}_{\varepsilon\sim\mathcal{N}(0,\sigma^{2}I)}f(x+\varepsilon)$
is computed. Note that $g(x)$ is $L$-Lipschitz with
$L=\sqrt{{2}/{\pi\sigma^{2}}},$ according to the Theorem 1. The number of
samples $n$ is increased until adversarial embedding risk $\gamma$ from the
Theorem 2 is computed with Algorithms 1-2 and certified radius
$r=\frac{\gamma}{L}$ is determined. The model $g(\cdot)$ is treated as
certified at $x$ for all additive perturbations $\delta:\|\delta\|_{2}<r$.

In this work, we fill this gap, by generalizing and theoretically justifying
the idea of randomized smoothing to few-shot learning. In this scenario,
provable certification needs to be obtained not in the space of output class
probabilities, but in the space of descriptive embeddings. This work is the
first, to our knowledge, where the theoretical robustness guarantees for few-
shot scenario is provided.

Our contributions are summarized as follows:

  * •

We provide the first theoretical robustness guarantee for few-shot learning
classification.

  * •

Analysis of Lipschitz continuity of such models and providing the robustness
certificates against $\ell_{2}-$bounded perturbations for few-shot learning
scenarios.

  * •

We propose to estimate confidence intervals not for distances between the
approximation of smoothed embedding and class prototype but for the dot
product of vectors which has expectation equal to the distance between actual
smoothed embedding and class prototype.

##  2 Problem statement

###  2.1 Notation

We consider a few-shot classification problem where we are given a set of
labeled objects $(x_{1},y_{1}),\dots,(x_{m},y_{m})$ where
$x_{i}\in\mathbb{R}^{D}$ and $y_{i}\in\\{1,\dots,K\\}$ are corresponding
labels. We follow the notation from [39] and denote $S_{k}$ as the set of
objects of class $k.$

###  2.2 Few-shot learning classification

Suppose we have a function $f:\mathbb{R}^{D}\to\mathbb{R}^{d}$ that maps input
objects to the space of normalized embeddings. Then, $d-$dimensional
prototypes of classes are computed as follows (expression is given for the
prototype of class $k$):

$c_{k}=\frac{1}{|S_{k}|}\sum_{x\in S_{k}}f(x).$ (2)

In order to classify a sample, one should compute the distances between its
embedding and class prototypes – a sample is assigned to the class with the
closest prototype. Namely, given a distance function
$\rho:\mathbb{R}^{d}\times\mathbb{R}^{d}\to[0,+\infty)$, the class $c$ of the
sample $x$ is computed as below:

$c=\underset{k\in\\{1,\dots,K\\}}{\arg\min}\rho\left(f(x),c_{k}\right).$ (3)

Given an embedding function $f$, our goal is to construct a classifier $g$
provably robust to additive perturbations $\Delta$ of a small norm. In other
words, we want to find a norm threshold $t$ such that equality

$\underset{k\in\\{1,\dots,K\\}}{\arg\min}\rho\left(g(x),c_{k}\right)=\underset{k\in\\{1,\dots,K\\}}{\arg\min}\rho\left(g(x+\delta),c_{k}\right),$
(4)

will be satisfied for all $\delta:\|\delta\|_{2}\leq t.$

In this paper, the solution of a problem of constructing a classifier that
satisfies the condition in (4) is approached by extending the analysis of the
robustness of smoothed classifiers described in (1) to the case of vector
functions. The choice of the distance metric in (4) is motivated by an
analysis of Lipschitz-continuity given in the next section.

##  3 Randomized smoothing

###  3.1 Background

Randomized smoothing [21, 3] is described as a technique of convolving a base
classifier $f$ with an isotropic Gaussian noise such that the new classifier
$g(x)$ returns the most probable prediction of $f$ of a random variable
$\xi\sim\mathcal{N}(x,\sigma^{2}I)$, where the choice of Gaussian distribution
is motivated by the restriction on $g$ to be robust against additive
perturbations of bounded norm. In this case, given a classifier
$f:\mathbb{R}^{D}\to[0,1]$ and smoothing distribution
$\mathcal{N}(0,\sigma^{2}I)$, the classifier $g$ looks as follows:

$g(x)=\frac{1}{(2\pi\sigma^{2})^{\frac{n}{2}}}\int_{\mathbb{R}^{D}}f(x+\varepsilon)\exp\left(-\frac{\|\varepsilon\|^{2}_{2}}{2\sigma^{2}}\right)d\varepsilon.$
(5)

One can show by Stein’s Lemma that given the fact that the function $f$ in (5)
is bounded (namely, $\forall x\in D(f),|f(x)|\leq 1$), then the function $g$
is $L-$Lipschitz:

$\forall x,x^{\prime}\in D(g),\ \|g(x)-g(x^{\prime})\|_{2}\leq
L\|x-x^{\prime}\|_{2},$ (6)

with $L=\sqrt{\frac{2}{\pi\sigma^{2}}}$, what immediately produce theoretical
robustness guarantee on $g.$

Although this approach is simple and effective, it has a serious drawback: in
practice, it is impossible to compute the expectation in (5) exactly and,
thus, impossible to compute the prediction of the smoothed function $g$ at any
point. Instead the integral is computed with the use of Monte-Carlo
approximation with $n$ samples to obtain the prediction with an arbitrary
level of confidence. Notably, to achieve an appropriate accuracy of the Monte-
Carlo approximation, the number of samples $n$ should be large enough that may
dramatically affect inference time.

In this work, we generalize the analysis of Lipschitz-continuity to the case
of vector functions $g:\mathbb{R}^{D}\to\mathbb{R}^{d}$ and provide robustness
guarantees for classification performed in the space of embeddings. The
certification pipeline is illustrated in the Figure 1.

###  3.2 Randomized smoothing for vector functions

##### Lipschitz-continuity of vector function.

In the work of [38], the robustness guarantee from [3] is proved by estimating
the Lipschitz constant of a smoothed classifier. Unfortunately, a
straightforward generalization of this approach to the case of vector
functions leads to the estimation of the expectation of the norm of a
multivariate Gaussian which is known to depend on the number of dimensions of
the space. Instead, we show that a simple adjustment to this technique may be
done such that the estimate of the Lipschitz constant is the same as for the
function in (5). Our results are formulated in the theorems below proofs of
which are moved to the Appendix in order not to distract the reader.

######  Theorem 1.

(Lipschitz-continuity of smoothed vector function) Suppose that
$f:\mathbb{R}^{D}\to\mathbb{R}^{d}$ is a deterministic function and
$g(x)={\mathbb{E}_{\varepsilon\sim\mathcal{N}(0,\sigma^{2}I)}}f(x+\varepsilon)$
is continuously differentiable for all $x$. If for all $x$, $\|f(x)\|_{2}=1$,
then $g(x)$ is $L-$Lipschitz in $l_{2}-$norm with
$L=\sqrt{\frac{2}{\pi\sigma^{2}}}.$

######  Remark 1.

We perform the analysis of Lipschitz-continuity in the Theorem 1 in
$l_{2}-$norm, so the distance metric in (4) is $l_{2}-$distance. We do not
consider other norms in this paper.

##### Robust classification in the space of embeddings.

To provide certification for a classification in the space of embeddings, one
should estimate the maximum deviation of the classified embedding that does
not change the closest class prototype. In the Theorem 2, we show how this
deviation is connected with the mutual arrangement of embedding and class
prototype.

$\mathbf{c}_{1}$$\mathbf{c}_{2}$$\mathbf{g(x)}$$\mathbf{\Delta}$

Figure 2: Illustration of the Theorem 2, one-shot case. The direction of
adversarial risk in the space of embeddings is always parallel to the vector
$c_{1}-c_{2}$. This is also true for the case of
$\|c_{1}\|_{2}\neq\|c_{2}\|_{2}$.

######  Theorem 2.

(Adversarial embedding risk) Given an input image $x\in\mathbb{R}^{D}$ and the
embedding $g:\mathbb{R}^{D}\to\mathbb{R}^{d}$ the closest point on to decision
boundary in the embedding space (see Figure 2) is located at a distance
(defined as adversarial embedding risk):

$\gamma=\|\Delta\|_{2}=\frac{\|c_{2}-g(x)\|^{2}_{2}-\|c_{1}-g(x)\|^{2}_{2}}{2\|c_{2}-c_{1}\|^{2}_{2}},$
(7)

where $c_{1}\in\mathbb{R}^{d}$ and $c_{2}\in\mathbb{R}^{d}$ are the two
closest prototypes. The value of $\gamma$ is the distance between classifying
embedding and the decision boundary between classes represented by $c_{1}$ and
$c_{2}.$ Note that this is the minimum $l_{2}-$distortion in the embedding
space required to change the prediction of $g.$

Two theorems combined result in an $l_{2}-$robustness guarantee for few-shot
classification as:

######  Theorem 3.

(Robustness guarantee) $L_{2}$-robustness guarantee $r$ for an input image $x$
in the $n-$dimensional input metric space under classification by a classifier
$g$ from the Theorem 1 is $r=\frac{\gamma}{L},$ where $L$ is the Lipschitz
constant from the Theorem 1 and $\gamma$ is the adversarial risk from the
Theorem 2. The value of $r$ is the certified radius of $g$ at $x$, or, in
other words, minimum $l_{2}-$distortion in the input space required to change
the prediction of $g.$ The proof of this fact straightforwardly follows from
the definition (6) and results from Theorems 1-2.

##  4 Certification protocol

In this section, we describe the numerical implementation of our approach and
estimate its failure probability.

###  4.1 Estimation of prediction of smoothed classifier

As mentioned previously, the procedure of few-shot classification is performed
by assigning an object $x$ to the closest class prototype. Unfortunately,
given the smoothed function $g$ in the form from the Theorem 1 and class
prototype $c_{k}$ from (2), it is impossible to compute the value
$\rho(g(x),c_{k})$ explicitly as well as to determine the closest prototype,
since it is in general unknown how does $g(x)$ look like. We propose both to
estimate the closest prototype for classification and to estimate the distance
to the closest decision boundary $\gamma$ from Theorem 2 as the largest class-
preserving perturbation in the embedding space by computing two-sided
confidence intervals for random variables

$\displaystyle\xi_{1}=\|\hat{g}(x)-c_{1}\|_{2}^{2},\ \dots,\
\xi_{K}=\|\hat{g}(x)-c_{K}\|_{2}^{2},$ (8)

where $\hat{g}(x)=\frac{1}{n}\sum_{i=1}^{n}f(x+\varepsilon_{i})$ is the
estimation of $g(x)$ computed as empirical mean by $n$ samples of noise, and
one-sided confidence interval for $\gamma$ from the Theorem 2, respectively.
Pseudo-code for both procedures is presented in Algorithms 1-2.

Function CLOSEST($f,\sigma,x,n,\\{c_{i}\\}_{i=1}^{K},T,\alpha$)

$fs\leftarrow\emptyset$

$i\leftarrow 0$

$n_{0}=n$

while $n\leq T$ do

$\varepsilon_{1},\dots,\varepsilon_{n}\sim\mathcal{N}(0,\sigma^{2}I)$

$\hat{g}(x)\leftarrow\frac{1}{n}\sum_{i=1}^{n}f(x+\varepsilon_{i})$

$fs\leftarrow fs\cup\\{\hat{g}(x)\\}$

for all $i\in[1,\dots,K]$ do

$C=c_{i}$

$dsToC\leftarrow\rho(fs,C)$

$(l_{C},u_{C})\leftarrow$ TwoSidedConfInt($dsToC,\alpha)$

$l_{i}\leftarrow l_{C},u_{i}\leftarrow u_{C}$

end for

$A\leftarrow\arg\min\ \\{l_{i}\\}_{i=1}^{K}$

if $u_{A}<\min(\\{l_{i}\\}_{i=1,i\neq A}^{K})$ then

Return A, fs

else

$n\leftarrow n+n_{0}$ {Increase number of samples used for computing an
approximation $\hat{g}(x)$ until the number of observations is large enough to
determine two leftmost intervals or until $n=T$}

end if

end while

EndFunction

Algorithm 1 Closest prototype computation algorithm.
GIVEN: base classifier $f$, noise $\sigma$, object $x$, number of samples $n$
for $\hat{g}(x)$, class prototypes $\\{c_{i}\\}_{i=1}^{K}$, maximum number of
samples $T$ and confidence level $\alpha$,
RETURNS: index $A$ of the closest prototype.

The Algorithm 1 describes an inference procedure for the smoothed classifier
from the Theorem 1; the Algorithm 2 uses Algorithm 1 and, given input
parameters, estimates an adversarial risk from the Theorem 2 – it determines
two closest to smoothed embedding $g(x)$ prototypes $A$ and $B$ and produces
the lower confidence bound for the distance between $g(x)$ and the decision
boundary between $A$ and $B.$ Combined with analysis from the Theorem 1, it
provides the certified radius for a sample – the smallest value of
$l_{2}-$norm of perturbation in the input space required to change the
prediction of the smoothed classifier. In the next subsection, we discuss in
detail the procedure of computing confidence intervals in Algorithms 1-2.

###  4.2 Applicability of algorithms

The computations of smoothed function and distances to class prototypes and
decision boundary in Algorithms 1-2 are based on the estimations of
corresponding random variables, thus, it is necessary to analyze applicability
of the algorithms. In this section, we propose a way to compute confidence
intervals for squares of the distances between the estimates of embeddings and
class prototypes.

Computation of confidence intervals for the squares of distances. Recall that
one way to estimate the value of a parameter of a random variable is to
compute a confidence interval for the corresponding statistic. In this work,
we construct intervals by applying well-known Hoeffding inequality [13] in the
form

$\mathbb{P}\left(|\overline{X}-\mathbb{E}(\overline{X})|\geq t\right)\leq
2\exp\left(-\frac{2t^{2}n^{2}}{\sum_{i=1}^{n}(b_{i}-a_{i})^{2}}\right),$ (9)

where $\overline{X}$ and $\mathbb{E}(\overline{X})$ are sample mean and
population mean of random variable $X$, respectively, $n$ is the number of
samples and numbers $a_{i},b_{i}$ are such that
$\mathbb{P}(X_{i}\in(a_{i},b_{i}))=1$.

Function EMBEDDING-RISK($f,\sigma,x,n,\\{c_{i}\\}_{i=1}^{K},T,\alpha$)

$A,\ fs_{A}\leftarrow$ CLOSEST($f,\sigma,x,n,\\{c_{i}\\}_{i=1}^{K},T,\alpha$)

$B,\ fs_{B}\leftarrow$ CLOSEST($f,\sigma,x,n,\\{c_{i}\\}_{i=1,i\neq
A}^{K},T,\alpha$)

$fs\leftarrow fs_{A}\cup fs_{B}$

$\gamma s\leftarrow\emptyset$

for all $f\in fs$ do

$\gamma=\frac{\|c_{B}-g\|^{2}-\|c_{A}-g\|^{2}}{2\|c_{B}-c_{A}\|^{2}}$

$\gamma s\leftarrow\gamma s\cup\\{\gamma\\}$

end for

$\Gamma\leftarrow\texttt{LowerConfBound}(\gamma s,\alpha)$

Return $\Gamma$

EndFunction

Algorithm 2 Adversarial embedding risk computation algorithm.
GIVEN: base classifier $f$, noise $\sigma$, object $x$, number of samples $n$
for $\hat{g}(x)$, class prototypes $\\{c_{i}\\}_{i=1}^{K}$, maximum number of
samples $T$ for $\hat{g}(x)$ and confidence level $\alpha$
RETURNS: lower bound $\Gamma$ for the adversarial risk $\gamma$.

However, a confidence interval for the distance $\|\hat{g}(x)-c_{k}\|_{2}$
with a certain confidence covers an _expectation of distance_
$\mathbb{E}(\|\hat{g}(x)-c_{k}\|_{2})$, not the _distance for expectation_
$\|\mathbb{E}(\hat{g}(x)-c_{k})\|_{2}=\|g(x)-c_{k}\|_{2}.$

To solve this problem, we propose to compute confidence intervals for the dot
product of vectors. Namely, given a quantity
$\xi_{x,k}=\langle{g}(x)-c_{k},{g}(x)-c_{k}\rangle,$ we sample its unbiased
estimate with at maximum $2n$ samples of noise (here we have to mention that
the number of samples $n$ from Algorithm 1 actually doubles since we need a
pair of estimates $\hat{g}(x)$ of smoothed embeddings):

$\displaystyle\hat{\xi}_{x,k}=\langle\frac{1}{n}\sum_{i=1}^{n}f(x+\varepsilon_{i})-c_{k},\frac{1}{n}\sum_{j=n+1}^{2n}f(x+\varepsilon_{j})-c_{k}\rangle$
(10)

and compute confidence interval $(l_{x,k},u_{x,k})$ such that given

$\frac{\alpha}{3}=2\exp\left(-\frac{2t^{2}n^{2}}{\sum_{i=1}^{n}(b_{i}-a_{i})^{2}}\right),$
(11)

the population mean $\mathbb{E}(\hat{\xi}_{x,k})$ is most probably located
within it, or, equivalently,
$\mathbb{P}\left(l_{x,k}\leq\mathbb{E}(\hat{\xi}_{x,k})\leq u_{x,k}\right)\geq
1-\alpha.$ We have to mention that there are three confidence intervals for
three terms (one with quadratic number of samples and two with the linear
numbers of samples) in the expression (10), that is why there is fraction
$\frac{1}{3}$ in the equation (11).

Also note that the population mean $\mathbb{E}(\hat{\xi}_{x,k})$ is exactly
$\|g(x)-c_{k}\|^{2}_{2}$, since

$\displaystyle\mathbb{E}(\hat{\xi}_{x,k})=\mathbb{E}\langle\frac{1}{n}\sum_{i=1}^{n}f(x+\varepsilon_{i})-c_{k},\frac{1}{n}\sum_{j=n+1}^{2n}f(x+\varepsilon_{j})-c_{k}\rangle=\|g(x)-c_{k}\|_{2}^{2}$
(12)

since $f(x+\varepsilon_{i})$ and $f(x+\varepsilon_{j})$ are independent random
variables for $i\neq j$. Finally, note that the confidence interval
$(l_{x,k},u_{x,k})$ for the quantity $\|{g}(x)-c_{k}\|^{2}_{2}$ implies
confidence interval

$(\sqrt{l_{x,k}},\sqrt{u_{x,k}})$ (13)

for the quantity $\|{g}(x)-c_{k}\|_{2}.$ Thus, the procedures TwoSidedConfInt
and LowerConfBound from algorithms return an interval from (13) and its left
bound for the random variable representing corresponding distance,
respectively.

##  5 Experiments

###  5.1 Datasets

For the experimental evaluation of our approach we use several well-known
datasets for few-shot learning classification. Cub-200-2011 [45] is a dataset
with $11,788$ images of $200$ bird species, where $5864$ images of $100$
species are in the train subset and $5924$ images of other $100$ species are
in the test subset. It is notable that a lot of species presented in dataset
have degree of visual similarity, making classification of ones a challenging
task even for humans. miniImageNet [44] is a substet of images from ILSVRC
2015 [37] dataset with $64$ images categories in train subset, $16$ categories
in validation subset and $20$ categories in test subset with $600$ images of
size $84\times 84$ in each category. CIFAR FS [1] is a subset of CIFAR 100
[19] dataset which was generated in the same way as miniImageNet and contains
$37800$ images of $64$ categories in the train set and $11400$ images of $20$
categories in the test set. Experimental setup for all the datasets is
presented in the next section.

(a) Cub-200-2011

(b) CIFAR-FS

(c) miniImageNet

Figure 3: Dependency of certified accuracy on attack radius $\varepsilon$ for
different $\sigma$, 1-shot case, $n=1000$.

(a) Cub-200-2011

(b) CIFAR-FS

(c) miniImageNet

Figure 4: Dependency of certified accuracy on attack radius $\varepsilon$ for
different $\sigma$, 5-shot case, $n=1000$.

###  5.2 Experimental settings and computation cost

Following [3], we compute approximate certified test set accuracy to estimate
the performance of the smoothed model prediction with the Algorithm 1 and
embedding risk computation with the Algorithm 2. The baseline model we used
for experiments is a prototypical network introduced in [39] with ConvNet-4
backbone. Compared to the original architecture, an additional fully-connected
layer was added in the tail of the network to map embeddings to
512-dimensional vector space. The model was trained to solve 1-shot and 5-shot
classification tasks on each dataset, with 5-way classification on each
iteration.

Parameters of expeiments. For data augmentation, we applied Gaussian noise
with zero mean, unit variance and probability $0.3$ of augmentation. Each
dataset was certified on a subsample of 500 images with default parameters for
the Algorithm 1: number of samples $n=1000$, confidence level $\alpha=0.001$
and variance $\sigma=1.0$, unless stated otherwise. For our settings, it may
be shown from simple geometry that values $(a_{i},b_{i})$ from (9) are such
that $b_{i}-a_{i}\leq 4$ so we use $b_{i}-a_{i}=4.$ The maximum number of
samples $T$ in the Algorithm 1 is set to be $T=5\times 10^{5}.$

Computation cost. In the table below, we report the computation time of the
certification procedure per image on Tesla V100 GPU for Cub-200-2011 dataset.
Standard deviation in seconds appears to be significant because the number of
main loop iterations required to separate the two leftmost confidence
intervals varies from image to image in the test set.

Table 1: Computation time per image of implementation of the Algorithm 2, Cub-200-2011. n | $10^{3}$ | $10^{4}$ | $10^{5}$
---|---|---|---
t, sec | 0.044 $\pm$ 0.030 | 0.509 $\pm$ 0.403 | 4.744 $\pm$ 2.730

###  5.3 Results of experiments

In this section, we report results of our experiments. In our evaluation
protocol, we compute approximate certified test set accuracy, $CA$. Given a
sample $x$, a smoothed classifier $g(\cdot)$ from the Algorithm 1 with an
assigned classification rule
$h(x)=\arg\min_{i\in\\{1,\dots,K\\}}\|g(x)-c_{k}\|_{2}$, threshold value
$\varepsilon$ for $l_{2}-$norm of additive perturbation and the robustness
guarantee $r=r(x)$ from the Algorithm 3, we compute $CA$ on test set $S$ as
follows:

$\displaystyle CA(S,\varepsilon)=\frac{|(x,y)\in S:r(x)>\varepsilon\ \&\
h(x)=y|}{|S|}.$ (14)

In other words, we treat the model $g(\cdot)$ as certified at point $x$ under
perturbation of norm $\varepsilon$ if $x$ is correctly classified by
$g(\cdot)$ (what means that the procedure of classification described in the
Algorithm 1 does not abstain from classification of $x$) and $g(\cdot)$ has
the value of certified radius $r(x)>\varepsilon.$

Visualization of results. The figures 3-4 represent dependencies of certified
accuracy on the value of norm of additive perturbation for different learning
settings (1-shot and 5-shot learning). The value of attack radius corresponds
to the threshold $\varepsilon$ from (14). For Cub-200-2011 dataset we provide
a dependency of certified accuracy for different sample size $n$ for the
Algorithm 1 (in the Figure 5).

##  6 Limitations

In this section, we provide failure probability of Algorithms 1-2, discuss
abstains from classification in the Algorithm 1 and speculate on the
application of our method in other few-shot scenarios.

###  6.1 Estimation of errors of algorithms

Note that the value of $\alpha$ from (11) is the probability of the value of
$\rho_{x,k}=\|g(x)-c_{k}\|_{2}$ not to belong to the corresponding interval of
the form from (13). Given a sample $x$, the procedure in the Algorithm 1
returns two closest prototypes to the $g(x).$ To determine two leftmost
confidence intervals, all the distances $\rho_{x,k}$ have to be located within
corresponding intervals, thus, according to the independence of computing
these two intervals, the error probability for the Algorithm 1 is
$q_{1}=K\alpha$, where $K$ is the number of classes. Similarly, the procedure
in the Algorithm 2 outputs the lower bound for the adversarial risk with
coverage at least $1-\alpha$ and depend on the output of the Algorithm 1
inside, and, thus, has error probability
$q_{2}=1-(1-\alpha)(1-K\alpha)=\alpha+K\alpha-K\alpha^{2}$ that corresponds to
returning an overestimated lower bound for the adversarial risk from the
Theorem 2.

###  6.2 Abstains from classification and extension to other few-shot
approaches

It is crucial to note that the procedure in the Algorithm 1 may require a lot
of samples to distinguish two leftmost confidence intervals and sometimes does
not finish before reaching threshold value $T$ for sample size. As a result,
there may be input objects at which the smoothed classifier can be neither
evaluated nor certified. In this subsection, we report the fraction of objects
in which the Algorithm 1 abstains from determining the closest class prototype
(see Tables 2-3).  Figure 5: Dependency of certified accuracy on attack radius
$\varepsilon$ for different number of samples $n$ of the Algorithm 1, CIFAR-FS
dataset, 5-shot case. Is is notable that relatively small number of samples
may be used to achieve satisfactory level of certified accuracy. Table 2:
Percentage of non-certified objects in test subset, 1-shot case.
$\alpha=10^{-2}$ $\alpha=10^{-3}$ $\alpha=10^{-4}$ Cub-200-2011 1.6% 1.6% 1.6%
CIFAR-FS 2.2% 2.2% 2.4% miniImageNet 1.9% 2.2% 2.4% Table 3: Percentage of
non-certified objects in test subset, 5-shot case. $\alpha=10^{-2}$
$\alpha=10^{-3}$ $\alpha=10^{-4}$ Cub-200-2011 1.2% 1.2% 1.4% CIFAR-FS 3.0%
3.4% 3.8% miniImageNet 2.9% 2.9% 3.0%

Since our method is based on randomized smoothing, among the approaches
presented in [43] it is applicable for matching networks and to some extent to
MAML networks. In case of matching networks, new sample is labeled as weighted
cosine distance to support samples, so it is easy to transfer guarantees for
$l_{2}-$distance to the ones for cosine distance in case of normalized
embeddings. For MAML, the smoothing may be applied to the embedding function
as well, but theoretical derivations of certificates are required.

##  7 Related work

Breaking neural networks with adversarial attacks and empirical defending from
them have a long history of cat-and-mouse game. Namely, for a particular
proposed defense against existing adversarial perturbations, a new more
aggressive attack is found. This motivated researchers to find defenses that
are mathematically provable and certifiably robust to different kinds of input
manipulations. Several works proposed exactly verified neural networks based
on Satisfiability Modulo Theories solvers [16, 6], or mixed-integer linear
programming [30, 8]. These methods are found to be computationally
inefficient, although they guarantee to find adversarial examples, in the case
if they exist. Another line of works use more relaxed certification [47, 11,
36]. Although these methods aim to guarantee that an adversary does not exist
in a certain region around a given input, they suffer from scalability to big
networks and large datasets. The only scalable to large datasets provable
defense against adversarial perturbations is _randomized smoothing_.
Initially, it was found as an empirical defense to mitigate adversarial
effects in neural networks [29, 49]. Later several works showed its
mathematical proof of certified robustness [21, 25, 3, 38]. Lecuyer et al [21]
first provided proof of certificates against adversarial examples using
differential privacy. Later, Cohen et al [3] provided the tightest bound using
Neyman-Pearson lemma. Interestingly, alternative proof using Lipschitz
continuity was found [38]. The scalability and simplicity of randomized
smoothing attracted significant attention, and it was extended beyond
$l_{2}-$perturbations [22, 42, 27, 24, 23, 20, 32, 51].

##  8 Conclusion and future work

In this work, we extended randomized smoothing as a defense against additive
norm-bounded adversarial attacks to the case of classification in the
embedding space that is used in few-shot learning scenarios. We performed an
analysis of Lipschitz continuity of smoothed normalized embeddings and derived
a robustness certificate against $l_{2}-$attacks. Our theoretical findings are
supported experimentally on several datasets. There are several directions for
future work: our approach can possibly be extended to other types of attacks,
such semantic transformations; also, it is important to reduce the
computational complexity of the certification procedure.

## References

  * [1] Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi.  Meta-learning with differentiable closed-form solvers.  arXiv preprint arXiv:1805.08136, 2018.
  * [2] Nicholas Carlini and David Wagner.  Towards evaluating the robustness of neural networks.  In 2017 ieee symposium on security and privacy (sp), pages 39–57. IEEE, 2017.
  * [3] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter.  Certified adversarial robustness via randomized smoothing.  In International Conference on Machine Learning, pages 1310–1320. PMLR, 2019.
  * [4] Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, and Anima Anandkumar.  Stochastic activation pruning for robust adversarial defense, 2018.
  * [5] Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, and Jun Zhu.  Efficient decision-based black-box adversarial attacks on face recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7714–7722, 2019.
  * [6] Ruediger Ehlers.  Formal verification of piece-wise linear feed-forward neural networks.  In International Symposium on Automated Technology for Verification and Analysis, pages 269–286. Springer, 2017.
  * [7] Volker Fischer, Mummadi Chaithanya Kumar, Jan Hendrik Metzen, and Thomas Brox.  Adversarial examples for semantic image segmentation.  arXiv preprint arXiv:1703.01101, 2017.
  * [8] Matteo Fischetti and Jason Jo.  Deep neural networks as 0-1 mixed integer linear programs: A feasibility study.  arXiv preprint arXiv:1712.06174, 2017.
  * [9] Micah Goldblum, Liam Fowl, and Tom Goldstein.  Adversarially robust few-shot learning: A meta-learning approach.  Advances in Neural Information Processing Systems, 33, 2020.
  * [10] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [11] Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli.  On the effectiveness of interval bound propagation for training verifiably robust models.  arXiv preprint arXiv:1810.12715, 2018.
  * [12] Jan Hendrik Metzen, Mummadi Chaithanya Kumar, Thomas Brox, and Volker Fischer.  Universal adversarial perturbations against semantic image segmentation.  In Proceedings of the IEEE international conference on computer vision, pages 2755–2764, 2017.
  * [13] Wassily Hoeffding.  Probability inequalities for sums of bounded random variables.  In The collected works of Wassily Hoeffding, pages 409–426. Springer, 1994.
  * [14] Yunseok Jang, Tianchen Zhao, Seunghoon Hong, and Honglak Lee.  Adversarial defense via learning to generate diverse attacks.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2740–2749, 2019.
  * [15] Jinyuan Jia, Xiaoyu Cao, Binghui Wang, and Neil Zhenqiang Gong.  Certified robustness for top-k predictions against adversarial perturbations via randomized smoothing.  arXiv preprint arXiv:1912.09899, 2019.
  * [16] Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer.  Reluplex: An efficient smt solver for verifying deep neural networks.  In International Conference on Computer Aided Verification, pages 97–117. Springer, 2017.
  * [17] Edgar Kaziakhmedov, Klim Kireev, Grigorii Melnikov, Mikhail Pautov, and Aleksandr Petiushko.  Real-world attack on mtcnn face detection system.  In 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), pages 0422–0427. IEEE, 2019.
  * [18] Stepan Komkov and Aleksandr Petiushko.  Advhat: Real-world adversarial attack on arcface face id system.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 819–826. IEEE, 2021.
  * [19] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [20] Aounon Kumar and Tom Goldstein.  Center smoothing: Certified robustness for networks with structured outputs.  Advances in Neural Information Processing Systems, 34, 2021.
  * [21] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana.  Certified robustness to adversarial examples with differential privacy.  In 2019 IEEE Symposium on Security and Privacy (SP), pages 656–672. IEEE, 2019.
  * [22] Guang-He Lee, Yang Yuan, Shiyu Chang, and Tommi S Jaakkola.  Tight certificates of adversarial robustness for randomly smoothed classifiers.  arXiv preprint arXiv:1906.04948, 2019.
  * [23] Alexander Levine and Soheil Feizi.  Robustness certificates for sparse adversarial attacks by randomized ablation.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4585–4593, 2020.
  * [24] Alexander Levine and Soheil Feizi.  Wasserstein smoothing: Certified robustness against wasserstein adversarial attacks.  In International Conference on Artificial Intelligence and Statistics, pages 3938–3947. PMLR, 2020.
  * [25] Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin.  Certified adversarial robustness with additive noise.  arXiv preprint arXiv:1809.03113, 2018.
  * [26] Debang Li, Junge Zhang, and Kaiqi Huang.  Universal adversarial perturbations against object detection.  Pattern Recognition, 110:107584, 2021.
  * [27] Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, and Bo Li.  Tss: Transformation-specific smoothing for robustness certification, 2021\.
  * [28] Fan Liu, Shuyu Zhao, Xuelong Dai, and Bin Xiao.  Long-term cross adversarial training: A robust meta-learning method for few-shot classification tasks.  arXiv preprint arXiv:2106.12900, 2021.
  * [29] Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh.  Towards robust neural networks via random self-ensemble.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 369–385, 2018.
  * [30] Alessio Lomuscio and Lalit Maganti.  An approach to reachability analysis for feed-forward relu neural networks.  arXiv preprint arXiv:1706.07351, 2017.
  * [31] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  arXiv preprint arXiv:1706.06083, 2017.
  * [32] Jeet Mohapatra, Ching-Yun Ko, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel.  Higher-order certification for randomized smoothing.  arXiv preprint arXiv:2010.06651, 2020.
  * [33] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard.  Universal adversarial perturbations.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1765–1773, 2017.
  * [34] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard.  Deepfool: a simple and accurate method to fool deep neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2574–2582, 2016.
  * [35] Mikhail Pautov, Nurislam Tursynbek, Marina Munkhoeva, Nikita Muravev, Aleksandr Petiushko, and Ivan Oseledets.  CC-Cert: A probabilistic approach to certify general robustness of neural networks.  arXiv preprint arXiv:2109.10696, 2021.
  * [36] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang.  Certified defenses against adversarial examples.  arXiv preprint arXiv:1801.09344, 2018.
  * [37] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al.  Imagenet large scale visual recognition challenge.  International journal of computer vision, 115(3):211–252, 2015\.
  * [38] Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, and Sebastien Bubeck.  Provably robust deep learning via adversarially trained smoothed classifiers.  arXiv preprint arXiv:1906.04584, 2019.
  * [39] Jake Snell, Kevin Swersky, and Richard S Zemel.  Prototypical networks for few-shot learning.  arXiv preprint arXiv:1703.05175, 2017.
  * [40] Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai.  One pixel attack for fooling deep neural networks.  IEEE Transactions on Evolutionary Computation, 23(5):828–841, 2019\.
  * [41] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  arXiv preprint arXiv:1312.6199, 2013.
  * [42] Jiaye Teng, Guang-He Lee, and Yang Yuan.  $\ell\\_1$ adversarial robustness certificates: a randomized smoothing approach.  2019\.
  * [43] Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, et al.  Meta-dataset: A dataset of datasets for learning to learn from few examples.  arXiv preprint arXiv:1903.03096, 2019.
  * [44] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al.  Matching networks for one shot learning.  Advances in neural information processing systems, 29:3630–3638, 2016.
  * [45] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie.  The caltech-ucsd birds-200-2011 dataset.  2011\.
  * [46] Lily Weng, Pin-Yu Chen, Lam Nguyen, Mark Squillante, Akhilan Boopathy, Ivan Oseledets, and Luca Daniel.  Proven: Verifying robustness of neural networks with a probabilistic approach.  In International Conference on Machine Learning, pages 6727–6736. PMLR, 2019.
  * [47] Eric Wong and Zico Kolter.  Provable defenses against adversarial examples via the convex outer adversarial polytope.  In International Conference on Machine Learning, pages 5286–5295. PMLR, 2018.
  * [48] Zuxuan Wu, Ser-Nam Lim, Larry S Davis, and Tom Goldstein.  Making an invisibility cloak: Real world adversarial attacks on object detectors.  In European Conference on Computer Vision, pages 1–17. Springer, 2020.
  * [49] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille.  Mitigating adversarial effects through randomization.  arXiv preprint arXiv:1711.01991, 2017.
  * [50] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille.  Adversarial examples for semantic segmentation and object detection.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1369–1378, 2017.
  * [51] Greg Yang, Tony Duan, J Edward Hu, Hadi Salman, Ilya Razenshteyn, and Jerry Li.  Randomized smoothing of all shapes and sizes.  In International Conference on Machine Learning, pages 10693–10705. PMLR, 2020.
  * [52] Dinghuai Zhang, Mao Ye, Chengyue Gong, Zhanxing Zhu, and Qiang Liu.  Black-box certification with randomized smoothing: A functional optimization based framework.  arXiv preprint arXiv:2002.09169, 2020.
  * [53] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan.  Theoretically principled trade-off between robustness and accuracy.  In International Conference on Machine Learning, pages 7472–7482. PMLR, 2019.
  * [54] Yaoyao Zhong and Weihong Deng.  Towards transferable adversarial attack against deep face recognition.  IEEE Transactions on Information Forensics and Security, 16:1452–1466, 2020.
  * [55] Jianli Zhou, Chao Liang, and Jun Chen.  Manifold projection for adversarial defense on face recognition.  In European Conference on Computer Vision, pages 288–305. Springer, 2020.

##  Appendix A Proofs

In this section, we provide proofs of the main results stated in our work.

######  Theorem 1.

Suppose that $f:\mathbb{R}^{D}\to\mathbb{R}^{d}$ is a deterministic function
with corresponding continuously differentiable smoothed function
$g(x)={\mathbb{E}_{\varepsilon\sim\mathcal{N}(0,\sigma^{2}I)}}f(x+\varepsilon)$.
Then if for all $x$, $\|f(x)\|_{2}=1$, then $g(x)$ is $L-$Lipschitz in
$l_{2}-$norm with $L=\sqrt{\frac{2}{\pi\sigma^{2}}}.$

###### Proof.

It is known that everywhere differentiable function $g(x)$ with Jacobian
matrix ${J}_{{g}}(x):({J}_{{g}}(x))_{ij}=\frac{\partial g_{i}}{\partial
x_{j}}$ is $L-$Lipschitz in $l_{2}-$norm with
$L=\underset{x\in\mathbb{R}^{D}}{\sup}\|J_{g}(x)\|_{2},$ where
$\|J_{g}(x)\|_{2}=\underset{v:\|v\|_{2}=1}{\sup}\|J_{g}(x)\cdot v\|_{2}$ is
the spectral norm of $J_{g}(x).$

Taking into account the fact that

$\displaystyle g(x)$
$\displaystyle=\underset{\varepsilon\sim\mathcal{N}(0,\sigma^{2}I)}{\mathbb{E}}f(x+\varepsilon)=\frac{1}{(2\pi\sigma^{2})^{n/2}}\int_{\mathbb{R}^{D}}f(x+\varepsilon)\exp\left(-\frac{\|\varepsilon\|^{2}_{2}}{2\sigma^{2}}\right)d\varepsilon,$
(15)

we may derive its Jacobian matrix:

$\displaystyle J_{g}$ $\displaystyle=\nabla
g(x)=\nabla\underset{\varepsilon\sim\mathcal{N}(0,\sigma^{2}I}{\mathbb{E}}f(x+\varepsilon)=\nabla\left(\frac{1}{(2\pi\sigma^{2})^{n/2}}\int_{\mathbb{R}^{D}}f(x+\varepsilon)\exp\left(-\frac{\|\varepsilon\|^{2}_{2}}{2\sigma^{2}}\right)d\varepsilon\right)=$
(16)
$\displaystyle=\nabla\left(\frac{1}{(2\pi\sigma^{2})^{n/2}}\int_{\mathbb{R}^{D}}f(y)\exp\left(-\frac{\|y-x\|^{2}_{2}}{2\sigma^{2}}\right)dy\right)=$
(17)
$\displaystyle=\frac{1}{M}\int_{\mathbb{R}^{D}}f(y)(x-y)^{\top}\exp\left(-\frac{\|y-x\|^{2}_{2}}{2\sigma^{2}}\right)dy,$
(18)

where $M=(2\pi\sigma^{2})^{n/2}\sigma^{2}.$ In order to estimate the spectral
norm of $J_{g}$, one can estimate the norm of dot product with normalized
vector $v$:

$\displaystyle\|J_{g}\cdot
v\|_{2}=\left\|\frac{1}{M}\int_{\mathbb{R}^{D}}f(y)(x-y)^{\top}\cdot
v\exp\left(-\frac{\|y-x\|^{2}_{2}}{2\sigma^{2}}\right)dy\right\|_{2}.$ (19)

Here, we apply a trick: it is possible to rotate vectors in dot product
$(x-y)^{\top}\cdot v$ in such a way that one of the resulting vectors will
have one nonzero component after rotation (without loss of generality, assume
that this is the first component, $e_{1}$). Namely, given a rotation matrix
$Q=Q(v)$ that is unitary ($QQ^{\top}=I$), the expression from Eq. 19 becomes

$\displaystyle\|J_{g}\cdot
v\|_{2}=\left\|\frac{1}{M}\int_{\mathbb{R}^{D}}f(y)(x-y)^{\top}QQ^{\top}\cdot
v\exp\left(-\frac{\|y-x\|^{2}_{2}}{2\sigma^{2}}\right)dy\right\|_{2}.$ (20)

Now, since the rotation does not affect the $l_{2}-$norm,
$\|Q^{\top}v\|_{2}=\|v\|_{2}=1$ and thus $Q^{\top}v=e$ and $|e_{1}|=1$. More
than that, under the change of the variables $z^{\top}=(x-y)^{\top}Q,$ the
following holds:

  * •

$\|z^{\top}\|_{2}=\|(x-y)^{\top}Q\|_{2}=\|(x-y)^{\top}\|_{2}$ since rotation
is $l_{2}-$norm preserving operation;

  * •

$y=x-Qz$;

  * •

$dz=-Qdy$ for the diffentials, leading to $dy=-Q^{\top}dz.$

Thus, expression from Eq. 20 becomes

$\displaystyle\|J_{g}\cdot
v\|_{2}=\left\|\frac{1}{M}\int_{\mathbb{R}^{D}}f(x-Qz)z^{\top}e\exp\left(-\frac{\|z\|^{2}_{2}}{2\sigma^{2}}\right)(-Q^{\top})dz\right\|_{2}.$
(21)

Now, we bound the norm from Eq. 21 using Cauchy–Schwarz inequality:

$\displaystyle\|J_{g}\cdot
v\|_{2}\leq\frac{1}{M}\int_{\mathbb{R}^{D}}\left\|f(x-Qz)z^{\top}e\exp\left(-\frac{\|z\|^{2}_{2}}{2\sigma^{2}}\right)(-Q^{\top})\right\|_{2}dz\leq$
(22)
$\displaystyle\leq\frac{1}{(2\pi\sigma^{2})^{n/2}\sigma^{2}}\int_{\mathbb{R}^{D}}|z_{1}|\exp\left(-\frac{\|z\|^{2}_{2}}{2\sigma^{2}}\right)dz=\frac{1}{\sigma^{2}}\underset{z\sim\mathcal{N}(0,\sigma^{2}I)}{\mathbb{E}}|z_{1}|.$
(23)

since $\|f(x)\|_{2}=1\ \forall x$ and $\|Q\|_{2}=1$, and
$\|z^{T}e\|_{2}=|z_{1}|$. Here $z_{1}$ is the first component of $z^{\top}e$.

The expectation from Eq. 22 is known to be equal to $\sigma\
\sqrt{\frac{2}{\pi}},$ and, thus

$\displaystyle\|J_{g}\cdot v\|_{2}\leq\sqrt{\frac{2}{\pi\sigma^{2}}}\ \forall
v:\|v\|_{2}=1.$ (24)

Taking a supremum over all unit vectors $v$, we immediately get
$L\leq\sqrt{\frac{2}{\pi\sigma^{2}}}$ what finalizes the proof.

∎

######  Theorem 2.

(Adversarial embedding risk) Given an input image $x\in\mathbb{R}^{D}$ and the
embedding $g:\mathbb{R}^{D}\to\mathbb{R}^{d}$ the closest point on to decision
boundary in the embedding space (see Figure 2) is located at a distance
(defined as adversarial embedding risk):

$\gamma=\|\Delta\|_{2}=\frac{\|c_{2}-g(x)\|^{2}_{2}-\|c_{1}-g(x)\|^{2}_{2}}{2\|c_{2}-c_{1}\|^{2}_{2}},$
(25)

where $c_{1}\in\mathbb{R}^{d}$ and $c_{2}\in\mathbb{R}^{d}$ are the two
closest prototypes. The value of $\gamma$ is the distance between classifying
embedding and the decision boundary between classes represented by $c_{1}$ and
$c_{2}.$ Note that this is the minimum $l_{2}-$distortion in the embedding
space required to change the prediction of $g.$

###### Proof.

For the convenience, redraw Figure 2 in the Figure 6 with labeled points.

$\mathbf{c}_{1}$$\mathbf{c}_{2}$$\mathbf{g(x)}$$\mathbf{\Delta}$$O$$A$$B$$C$$D$

Figure 6: The direction of adversarial risk in the space of embeddings is
always parallel to the vector $c_{1}-c_{2}$. This is also true for the case of
$\|c_{1}\|_{2}\neq\|c_{2}\|_{2}$.

In the Figure 6, $O$ is the origin, $\overrightarrow{OA}=c_{1}$,
$\overrightarrow{OB}=c_{2}$, $\overrightarrow{OC}=g(x)$,
$\overrightarrow{CD}=\Delta$, $\overrightarrow{AB}=c_{2}-c_{1}$.

We need to solve the following problem:

$\displaystyle\min\|\overrightarrow{CD}\|_{2}$ $\displaystyle=\|\Delta\|_{2}$
(26) $\displaystyle\text{s.t. }\|\overrightarrow{BD}\|_{2}$
$\displaystyle\leq\|\overrightarrow{AD}\|_{2}$ (27)

It is obvious that to satisfy minimality requirement we should consider
$\|\overrightarrow{BD}\|_{2}=\|\overrightarrow{AD}\|_{2}$. Thus we have
$\overrightarrow{AB}$ is perpendicular to the ray $\overrightarrow{OD}$.
Therefore, to minimize $\overrightarrow{CD}=\Delta$, we need to find distance
from $C$ to the ray $\overrightarrow{OD}$.

The closest distance is perpendicular to $\overrightarrow{OD}$.

$\left.\begin{array}[]{r@{\mskip\thickmuskip}l}\overrightarrow{AB}\perp\overrightarrow{OD}\mskip
5.0mu plus 5.0mu\\\ \overrightarrow{CD}\perp\overrightarrow{OD}\mskip 5.0mu
plus
5.0mu\end{array}\right\\}\implies\overrightarrow{CD}\parallel\overrightarrow{AB}$
(28)

$\implies\Delta=\overrightarrow{CD}=\gamma\frac{\overrightarrow{AB}}{\|\overrightarrow{AB}\|_{2}}=\gamma\frac{c_{2}-c_{1}}{\|c_{2}-c_{1}\|_{2}}$
(29)

$\implies\overrightarrow{OD}=\overrightarrow{OC}+\overrightarrow{CD}=g(x)+\gamma\frac{c_{2}-c_{1}}{\|c_{2}-c_{1}\|_{2}}$
(30)

$\implies\left\\{\begin{array}[]{r@{\mskip\thickmuskip}l}\overrightarrow{AD}=\overrightarrow{OD}-\overrightarrow{OA}=g(x)+\gamma\frac{c_{2}-c_{1}}{\|c_{2}-c_{1}\|_{2}}-c_{1}\mskip
5.0mu plus 5.0mu\\\
\overrightarrow{BD}=\overrightarrow{OD}-\overrightarrow{OB}=g(x)+\gamma\frac{c_{2}-c_{1}}{\|c_{2}-c_{1}\|_{2}}-c_{2}\mskip
5.0mu plus 5.0mu\end{array}\right.$ (31)

Solving equation $\|\overrightarrow{BD}\|_{2}=\|\overrightarrow{AD}\|_{2}$
implies

$\left\|g(x)+\Delta-c_{1}\right\|^{2}_{2}=\left\|g(x)+\Delta-
c_{2}\right\|_{2}^{2}.$ (32)
$\|g(x)-c_{1}\|_{2}^{2}+\|\Delta\|_{2}^{2}+2(g(x)-c_{1})^{T}\Delta=\|g(x)-c_{2}\|_{2}^{2}+\|\Delta\|_{2}^{2}+2(g(x)-c_{2})^{T}\Delta$
(33)
$\|g(x)-c_{1}\|_{2}^{2}+2(g(x)-c_{1})^{T}\Delta=\|g(x)-c_{2}\|_{2}^{2}+2(g(x)-c_{2})^{T}\Delta$
(34)
$2(g(x)-c_{1})^{T}\Delta-2(g(x)-c_{2})^{T}\Delta=\|g(x)-c_{2}\|_{2}^{2}-\|g(x)-c_{1}\|_{2}^{2}$
(35) $2(c_{2}-c_{1})^{T}\Delta=\|g(x)-c_{2}\|_{2}^{2}-\|g(x)-c_{1}\|_{2}^{2}$
(36)

Using the fact that $\Delta=\gamma\frac{c_{2}-c_{1}}{\|c_{2}-c_{1}\|_{2}}$ we
find that

$\gamma=\|\Delta\|_{2}=\frac{\|c_{2}-g(x)\|^{2}_{2}-\|c_{1}-g(x)\|^{2}_{2}}{2\|c_{2}-c_{1}\|^{2}_{2}}.$
(37)

∎

##  Appendix B Additional experiments

In this section, we provide results of additional experiments conducted to
ease the assessment of our method.

###  B.1 The distribution of required sample size

In the figure 7 we depict the histograms showing the distribution of the
smallest sample size required to reach confidence level on each dataset. All
the experiments were conducted with the following parameters: the variance of
additive noise $\sigma=1.0,$ maximum number of samples $n=100000$. Note that
for all the experiments and for all values $\alpha$ of interest, the required
number of samples is less than $10000$ for most of the input images.

(a) Cub-200-2011

(b) CIFAR-FS

(c) miniImageNet

Figure 7: Histogram of the required sample sizes, $n\leq 100000$,
$\sigma=1.0.$

###  B.2 Empirical robustness and effect of data augmentation

In this section, we assess our approach against random attacks, against
adversarial attack and evaluate the effect of the data augmentation during the
baseline model training.

For each dataset, we have sampled a random subset $S$ of $500$ test images and
used it as a test set.

The random attacks were conducted on the plain baseline model $f.$ In this
experiment, the input $x$ of the model were perturbed by a random noise
$\delta$ of the fixed magnitude. In this case, empirical robust accuracy is
computed as follows:

$CA(S,\varepsilon)=\frac{|(x,y)\in S:h(f(x))=h(f(x+\delta))=y|}{|S|},\
\|\delta\|_{2}=\varepsilon.$ (38)

Here and below, $h(\cdot)$ corresponds to the classification rule from Section
5.3.

For the adversarial attack, the smoothed model $g$ was attacked at input point
$x$ with the FGSM attack [10]. Namely, given an approximation of the smoothed
model in the form $\hat{g}(x)=\frac{1}{n}\sum f(x+\varepsilon),$ the additive
perturbation is computed as follows:

$\delta(x)=sign\left(\frac{1}{n}\sum\nabla_{x}f(x+\varepsilon)\right),$ (39)

so the additive perturbation corresponds to the projection of the mean
gradient. In this case, the empirical robust accuracy looks as follows:

$CA(S,\varepsilon)=\frac{|(x,y)\in S:h(g(x))=h(g(x+\delta))=y|}{|S|},\
\|\delta\|_{2}=\varepsilon.$ (40)

Also we have evaluated the effect of the data augmentation during the training
of the baseline model. For all datasets for both $1-$shot and $5-$shot
settings, we have trained new baseline models without data augmentation and
evaluated associated smoothed models.

In the figures 8-9, we report results of the experiments. For all the smoothed
models, we fixed the confidence level $\alpha=10^{-4}$ and number of samples
$n=1000$. The variance $\sigma$ of additive noise for Cub-200-2011 and CIFAR-
FS datasets is set $\sigma=1.0,$ for miniImageNet dataset $\sigma=0.5.$

In the figures, _SM (no aug)_ corresponds to the certified accuracy of the
smoothed model with the baseline model trained without augmentation, _SM
(aug)_ corresponds to the certified accuracy of the smoothed model with the
baseline model trained with augmentation, _PM (random attack)_ corresponds to
empirical robust accuracy of plain baseline model under random attack and _SM
(adv attack)_ corresponds to the empirical robust accuracy of the smoothed
model under adversarial attack.

(a) Cub-200-2011

(b) CIFAR-FS

(c) miniImageNet

Figure 8: Certified accuracy of smoothed models, empirical robust accuracy of
plain model under random attack and empirical robust accuracy of smoothed
model under adversarial attack, $1-$shot case.

(a) Cub-200-2011

(b) CIFAR-FS

(c) miniImageNet

Figure 9: Certified accuracy of smoothed models, empirical robust accuracy of
plain model under random attack and empirical robust accuracy of smoothed
model under adversarial attack, $5-$shot case.

From the pictures, we made several observations. Firstly, the augmentation of
the baseline model with an additive noise increases empirical robustness of
the smoothed model, so it is natural to augment the data to have both better
robustness and larger accuracy when there is no attack. Secondly, even the
baseline models could not be attacked with the random additive perturbation.
Although the fact that the probability of randomly generated additive noise to
be adversarial is low, it is interesting to show that it holds for the
prototypical networks too. Finally, we observe that even adversarial attack is
not effective against smoothed models.

Note that our theoretical guarantees are provided for the worst-case behavior
of the smoothed models, in practice smoothed encoders may have very strong
empirical robustness.

####  B.2.1 Adversarial attack on the smoothed model: FGSM vs PGD

It is important to mention that one-step FGSM attack is not the best tool to
assess model’s robustness. In this section, we report results of additional
experiments where we compare FGSM and its multi-step version, PGD [31]
(projected gradient descent).

In all experiments, we run PGD attack for $s=20$ iterations. Attack norm on
each iteration is $s$ times smaller than the one in FGSM experiment; it is
done to compare methods of attacks on corresponding magnitudes. For all the
smoothed models, we fixed the confidence level $\alpha=10^{-4}$ and number of
samples $n=1000.$ The variance $\sigma$ of additive noise for Cub-200-2011 and
CIFAR-FS datasets is set $\sigma=1.0$, for miniImageNet dataset $\sigma=0.5.$

All the models were evaluated in $1-$shot setting. Results are reported in the
figure 10.

(a) Cub-200-2011

(b) CIFAR-FS

(c) miniImageNet

Figure 10: Certified accuracy of smoothed models, empirical robust accuracy of
smoothed models under FGSM and PGD attacks, $1-$shot case.

It is notable that multi-step attack is more effective against smoothed model,
especially on larger norms of perturbations. Still, the model performs well
even against PGD attack.


SF2A 2023

# Insight from laboratory measurements on dust in debris discs

J. Milli  Univ. Grenoble Alpes, CNRS, IPAG, F-38000 Grenoble, France  O. Poch1
J. B. Renard  LPC2E, Université d’Orléans, CNRS, Orléans, France  J. C.
Augereau1 P. Beck1 E. Choquet  Aix Marseille Université, CNRS, CNES, LAM,
Marseille, France  J. M. Geffrin  Aix Marseille Univ, CNRS, Centrale
Marseille, Institut Fresnel, Marseille, France  E. Hadamcik  LATMOS, Sorbonne
Université, CNRS, CNES, Paris, France  J. Lasue  IRAP, Université de Toulouse,
CNES, CNRS, UPS, Toulouse, Franc  F. Menard1 A. Peronne1 C. Baruteau6 R.
Tazaki1 V. Tobon Valencia1

###### Abstract

Extreme adaptive optics instruments have revealed exquisite details on debris
discs, allowing to extract the optical properties of the dust particles such
as the phase function, the degree of polarisation and the spectral
reflectance. These are three powerful diagnostic tools to understand the
physical properties of the dust : the size, shape and composition of the dust
particles. This can inform us on the population of parent bodies, also called
planetesimals, which generate those particles through collisions. It is
however very rare to be able to combine all those three observables for the
same system, as this requires different high-contrast imaging techniques to
suppress the starlight and reveal the faint scattered light emission from the
dust. Due to its brightness, the ring detected around the A-type star HR 4796
is a notable exception, with both unpolarised and polarised images covering
near-infrared wavelengths. Here, we show how measurements of dust particles in
the laboratory can reproduce the observed near-infrared photo-polarimetric
properties of the HR 4796 disc. Experimental characterisation of dust allows
to bypass the current limitations of dust models to reproduce simultaneously
the phase function, the degree of polarisation and the spectral reflectance.

###### keywords:

HR 4796, light scattering, debris disc, cosmic dust

##  1 Introduction

Debris discs represent an evolutionary stage of a planetary system after a few
million years when the primordial gas-rich protoplanetary disc has dissipated,
and leaves behind planetesimals that are thought to be leftovers from the
planetary formation (see Wyatt 2008, for a review). Their mutual collisions
produce an optically thin dust disc that is detectable through the infrared
excess emission above the stellar photosphere. The initial reservoir of
planetesimals depletes with time, hence also the amount of dust. The detection
rate of debris discs is indeed $75\%$ in the young 18 Myr $\beta$ Pictoris
moving group (Pawellek et al. 2021), but goes down to $\sim 20\%$ around Gyr-
old stars (e.g. Sibthorpe et al. 2018). With an age of $10\pm 3$ Myrs, the
A-type star HR 4796 located at 70.8 pc (Gaia Collaboration 2020) hosts a
particularly well studied debris disc with a large infrared excess of 0.5%
(Moór et al. 2006). Most of the dust is located in a narrow ring at $\sim 80$
au from the central star and a width of about $\sim 10$ au (Kennedy et al.
2018). Because of its relatively high surface brightness compared to other
debris discs, this system has been extensively studied in many wavelengths, as
visible in Fig. 2.

Despite this extensive spectral coverage, the composition, shape and size of
the particles remain a mystery since the high-contrast imagers VLT/SPHERE and
Gemini/GPI have revealed the scattering phase function (hereafter SPF) and
degree of polarisation (hereafter DoLP) of the particules (Perrin et al. 2015;
Milli et al. 2017). We will focus here on the near-infrared scattering
properties of the disc to show how laboratory measurements of dust analogues
can be used to shed light on the nature of the particles in this system.

Figure 1: The HR 4796 disc at various wavelengths. The semi-major axis of the
disc is $\sim 1$”. North is up, East to the left.

##  2 The scattered light near-infrared properties of the dust

Thanks to the polarimetric capabilities of the high-contrast instruments
SPHERE (Beuzit et al. 2018) and GPI (Macintosh et al. 2014), the surface
brightness of the ring can be measured at almost all azimuth angles, even
along the minor axis. Each position along the ring corresponds to a unique
angle between the illumination direction and the line of sight, called the
scattering angle. The ring is inclined by $\sim 76.5^{\circ}$ from pole-on,
meaning that the minor axes correspond to scattering angles of $13.5^{\circ}$
and $166.5^{\circ}$ respectively for the bright North-West forward-scattering
side, and the faint South-East backward scattering side. One can therefore
extract the SPF and DoLP of the dust particles in this range of scattering
angles, as shown in Fig 2.

Figure 2: Left: Total intensity scattering phase function of the disc, as
extracted from GPI and SPHERE using a parametric modelling (double-component
Heyney Greenstein model) in the J ($1.2$\text{\,}\mathrm{\SIUnitSymbolMicro
m}$$), H ($1.6$\text{\,}\mathrm{\SIUnitSymbolMicro m}$$) and K
($2.2$\text{\,}\mathrm{\SIUnitSymbolMicro m}$$) band (Chen et al. 2020).
Right: Degree of linear polarisation as extracted from GPI in the K band
(Arriaga et al. 2020)

Modelling of the dust particles assuming compact spheres (Mie theory) made of
basic cometary-like materials used traditionally for debris disc, e.g.
silicates, amorphous carbon, water ice and porosity (see for instance Augereau
et al. 1999), could not reproduce the total intensity scattering phase
function of the dust particles (Milli et al. 2017). If one relaxes the
assumption on the composition of the particles, Chen et al. (2020) showed that
the total intensity SPF can be fit using the Distribution of Hollow Spheres
(DHS, Min et al. 2005) but required a relatively high volume fraction of
metallic iron of $37^{+15}_{9}\%$, a highly absorbing mineral, in addition to
amorphous carbon and silicates. However, when adding the DoLP as a constraint,
no model using the Mie nor the DHS theory could simultaneously reproduce the
SPF and the DoLP (Arriaga et al. 2020). This points out that one of our
initial assumption is wrong, either particles in the HR 4796 disc cannot be
modelled by compact spheres, as suggested already in Milli et al. (2014), or a
simple differential power-law size distribution is too simplistic. We decided
to use laboratory experiments rather than numerical experiments to find good
analogues to the HR 4796 dust. The choice of the sample was driven by the fact
that there are few micron-size particles with a peak in DoLP at scattering
angles as small as $40^{\circ}$, while larger mm-size pebbles can do so (e.g.
Muñoz et al. 2021) but are incompatible with the spectral energy distribution
of the disc that requires a minimum particle size of $\sim
2$\text{\,}\mathrm{\SIUnitSymbolMicro m}$$. We found an interesting sample in
the PROGRA2 database***https://www.icare.univ-
lille.fr/progra2-en/?noredirect=en_US, pyrrhotite, a non-stoichiometric form
of iron sulphide Fe1-xS ($0<x\leq 0.125$).

##  3 Laboratory measurements of dust particles with PROGRA2

###  3.1 The PROGRA2 instrument and pyrrhotite sample

The PROGRA2 instrument is dedicated to the study of light scattered by solid
particles deposited on a surface or lifted in clouds (Worms et al. 2000;
Hadamcik et al. 2002). We used here a sample of pyrrhotite (Fig. 3) that was
lifted in clouds in microgravity conditions during parabolic flights that took
place between 2018 and 2021. The sample is contained in a vial where it
receives the light from a laser. The light scattered by the particles is split
using a polarised beam splitter in two channels where it is measured on two
detectors.

Figure 3: Left: Scanning Electron Microscope view of the sample . Right:
Granulometry of the sample as measured using the camera onboard PROGRA2 in the
R band. The resolution of the camera is limited to
$40$\text{\,}\mathrm{\SIUnitSymbolMicro m}$$.

###  3.2 Measurements and conclusions

PROGRA2 allows measurements of the total intensity SPF at optical wavelengths,
and the DoLP at both optical and NIR wavelengths. The comparison to the HR
4796 particles are shown in Fig. 4. We notice that the peak in DoLP around a
scattering angle of $40^{\circ}$ is well reproduced by the sample. There are
very few samples with particles with sizes in the
1-200$\text{\,}\mathrm{\SIUnitSymbolMicro m}$ range that can reproduce a peak
at so small scattering angles in the large PROGRA2 database: only silicon
carbides (CSi) could also produce this behaviour, with an overall match not as
good as pyrrhotite. The maximum DoLP as measured in the optical and near-
infrared (NIR) is in the range 35-45% for the
1-200$\text{\,}\mathrm{\SIUnitSymbolMicro m}$ pyrrhotite, slightly smaller
than the HR 4796 particles at $2.2$\text{\,}\mathrm{\SIUnitSymbolMicro m}$$,
but the maximum DoLP can increase with wavelength for large
(35-150$\text{\,}\mathrm{\SIUnitSymbolMicro m}$) levitating grains (Renard et
al. 2014) and can also largely dependent on the minimum particle size or size
distribution (Milli et al. in prep.). Regarding the total intensity SPF (Fig.
4 right), the backward scattering behaviour seen on HR 4796 in the NIR is well
reproduced by the pyrrhotite sample measured in the optical. Unfortunately,
PROGRA2 does not have the capability to measure the SPF in the NIR for a
rigorous comparison at the same wavelength.

Figure 4: Left: DoLP of the measured pyrrhotite sample compared to the DoLP of
HR4796 in the K band (Arriaga et al. 2020). Right: Optical SPF of the
pyrrhotite sample compared to the NIR SPF of HR4796 (Chen et al. 2020)

Overall, the pyrrhotite sample is an interesting candidate material to
reproduce simultaneously the total intensity SPF and the DoLP of the particles
in HR 4796. In this study, no attempt to adjust the sample to match exactly
the HR 4796 properties has been made, in particular the particle sizes, but we
present here for the first time, archival PROGRA2 measurements done on a
sample of pyrrhotite with sizes dominated by particles smaller than
$100$\text{\,}\mathrm{\SIUnitSymbolMicro m}$$. The presence of a highly
absorbing material such as iron sulphide confirms the finding of Chen et al.
(2020) using the total intensity SPF alone and bring guidance in the attempt
to reconcile the SPF and DoLP with more irregular particles than the compact
spheres tested in Arriaga et al. (2020). The presence of iron sulphides would
not be surprising in circumstellar dust. Stratospheric interstellar dust
particles, Antarctic Micro-Meteorites, comet Wild 2 samples, all contain
sulphides in the form of troilite FeS, among other (Dobrica et al. 2009). Iron
is also present in the comet 67P/Churyumov-Gerasimenko, with a mass fraction
of 7.5% (Bardyn et al. 2017). It is probably mostly present in the form of
iron sulphides and Fe-Ni alloys, which are opaque minerals responsible for the
dark reflectance from optical to NIR wavelengths of cometary and primitive
asteroids surfaces (Quirico et al. 2016), they could therefore also be present
on the surface of the HR 4796 particles, explaining at the same time the red
reflectance spectrum of the disc (Milli et al. 2017).

###### Acknowledgements.

We dedicate this paper to the memory of Anny-Chantal Levasseur-Regourd (ACLR),
who encouraged and inspired our efforts to compare remote observations of
exoplanetary dust to laboratory measurements. This work was carried out as
part of the EPOPEE project (Etude des POussières Planétaires Et
Exoplanétaires) project. This research group is funded by the Programme
National de Planétologie (PNP) of CNRS-INSU in France, from which we
acknowledge financial support. ACLR was one of the most enthusiastic members,
always willing to share her wide expertise with the younger generation, and
she has vastly contributed to this scientific collaboration between the disc
and solar system community. She sadly passed away only a few weeks after
attending our June 2022 EPOPEE working group meeting in Grenoble.

## References

  * Arriaga et al. (2020) Arriaga, P., Fitzgerald, M. P., Duchêne, G., et al. 2020, AJ, 160, 79
  * Augereau et al. (1999) Augereau, J. C., Lagrange, A. M., Mouillet, D., Papaloizou, J. C. B., & Grorod, P. A. 1999, A&A, 348, 557
  * Bardyn et al. (2017) Bardyn, A., Baklouti, D., Cottin, H., et al. 2017, MNRAS, 469, S712
  * Beuzit et al. (2018) Beuzit, J.-L., Mouillet, D., Sauvage, J.-F., Fusco, T., & Dohlen, K. 2018, in Adaptive Optics Systems VI, Proc. SPIE
  * Chen et al. (2020) Chen, C., Mazoyer, J., Poteet, C. A., et al. 2020, ApJ, 898, 55
  * Dobrica et al. (2009) Dobrica, E., Engrand, C., Duprat, J., et al. 2009, Meteoritics and Planetary Science, 44, 1643
  * Gaia Collaboration (2020) Gaia Collaboration. 2020, VizieR Online Data Catalog, I/350
  * Hadamcik et al. (2002) Hadamcik, E., Renard, J. B., Worms, J. C., Levasseur-Regourd, A. C., & Masson, M. 2002, Icarus, 155, 497
  * Kennedy et al. (2018) Kennedy, G. M., Marino, S., Matrà, L., et al. 2018, MNRAS, 475, 4924
  * Macintosh et al. (2014) Macintosh, B., Graham, J. R., Ingraham, P., et al. 2014, Proceedings of the National Academy of Science, 111, 12661
  * Milli et al. (2014) Milli, J., Lagrange, A.-M., Mawet, D., et al. 2014, A&A, 566, A91
  * Milli et al. (2017) Milli, J., Vigan, A., Mouillet, D., et al. 2017, A&A, 599, A108
  * Min et al. (2005) Min, M., Hovenier, J. W., & de Koter, A. 2005, A&A, 432, 909
  * Moór et al. (2006) Moór, A., Ábrahám, P., Derekas, A., et al. 2006, ApJ, 644, 525
  * Muñoz et al. (2021) Muñoz, O., Frattin, E., Jardiel, T., et al. 2021, ApJS, 256, 17
  * Pawellek et al. (2021) Pawellek, N., Wyatt, M., Matrà, L., Kennedy, G., & Yelverton6, B. 2021, MNRAS, 502, 5390
  * Perrin et al. (2015) Perrin, M. D., Duchene, G., Millar-Blanchaer, M., et al. 2015, ApJ, 799, 182
  * Quirico et al. (2016) Quirico, E., Moroz, L. V., Schmitt, B., et al. 2016, Icarus, 272, 32
  * Renard et al. (2014) Renard, J. B., Hadamcik, E., Couté, B., Jeannot, M., & Levasseur-Regourd, A. C. 2014, J. Quant. Spec. Radiat. Transf., 146, 424
  * Sibthorpe et al. (2018) Sibthorpe, B., Kennedy, G. M., Wyatt, M. C., et al. 2018, MNRAS, 475, 3046
  * Worms et al. (2000) Worms, J. C., Renard, J. B., Hadamcik, E., Brun-Huret, N., & Levasseur-Regourd, A. C. 2000, Planet. Space Sci., 48, 493
  * Wyatt (2008) Wyatt, M. C. 2008, ARA&A, 46, 339