# Value-aware Approximate Attention

Ankit Gupta
Tel Aviv University
<EMAIL_ADDRESS>
&Jonathan Berant
Tel Aviv University,
Allen Institute for AI
<EMAIL_ADDRESS>

###### Abstract

Following the success of dot-product attention in Transformers, numerous
approximations have been recently proposed to address its quadratic complexity
with respect to the input length. However, all approximations thus far have
ignored the contribution of the _value vectors_ to the quality of
approximation. In this work, we argue that research efforts should be directed
towards approximating the true output of the attention sub-layer, which
includes the value vectors. We propose a value-aware objective, and show
theoretically and empirically that an optimal approximation of a value-aware
objective substantially outperforms an optimal approximation that ignores
values, in the context of language modeling. Moreover, we show that the choice
of kernel function for computing attention similarity can substantially affect
the quality of sparse approximations, where kernel functions that are less
skewed are more affected by the value vectors.

##  1 Introduction

The Transformer architecture Vaswani et al. (2017) has been widely successful
in a wide range of natural language processing tasks, including machine
translation Edunov et al. (2018), language modeling Roy et al. (2020),
question-answering Karpukhin et al. (2020), and many more. Transformers pre-
trained on large amounts of text with a language modeling (LM) objective, have
become the standard in NLP, exhibiting surprising amounts of linguistic and
world knowledge Peters et al. (2018); Devlin et al. (2019); Petroni et al.
(2019); Hewitt and Manning (2019); Roberts et al. (2020).

The contextualizing component of the Transformer is the attention layer where
all positions in an input sequence of length $L$ aggregate information from
the entire sequence in parallel. At its core, given $L$ query, key and value
vectors, the dot-product attention function outputs111Usually, the term is
$\mathrm{softmax}(QK^{\top}/\sqrt{d})V$ but $\sqrt{d}$ can be dropped via
scaling of queries and keys. $\mathrm{softmax}(QK^{\top})V$ where the
$\mathrm{softmax}$ function is applied row-wise on the matrix
$QK^{\top}\in\mathbb{R}^{L\times L}$, consisting of similarity scores of the
query-key pairs. Unfortunately, computing $\Omega(L\cdot L)$ similarity scores
is prohibitive for long sequences.

To alleviate this, past work proposed to compute an approximation of
$\mathrm{softmax}(QK^{\top})$. One major line of research focused on sparse
attention variants, where only a few similarity scores are computed per
position, and the rest are ignored. Methods differ by which query-key pairs
are selected Child et al. (2019); Ye et al. (2019); Qiu et al. (2019); Roy et
al. (2020); Kitaev et al. (2020); Beltagy et al. (2020); Gupta and Berant
(2020). A second line of research explored dense variants Katharopoulos et al.
(2020); Wang et al. (2020); Tay et al. (2020a) (cf. Tay et al. (2020b) for a
survey). E.g., instead of computing the attention scores exactly for only a
few query-key pairs, Choromanski et al. (2020) compute an approximation of
scores for all pairs.

In this work, we point to a lacuna in current research on efficient
Transformers. While recent work focused on approximating the attention scores
$\mathrm{softmax}(QK^{\top})$, the true target of approximation should be the
output of the attention sub-layer, namely $H=\mathrm{softmax}(QK^{\top})V$,
which also includes the value vectors, $V$. We show that ignoring value
vectors leads to unwarranted consequences both theoretically and empirically.

To demonstrate the importance of value-aware approximation, we analyze
_optimal sparse attention_ , that is, the case where, in hindsight, the model
computes dot product similarity only with the most similar key vectors, while
still ignoring the value vectors. We show that in the popular masked language
modeling (MLM) setup, optimal sparse attention dramatically _under-performs_
compared to an optimal approximation of the true output of the attention sub-
layer, $H$, leading to an error increase of $8$-$20$ points. Next, by
theoretically focusing on the case where queries compute similarity to the
_single_ most similar key vector, we show that approximating
$\mathrm{softmax}(QK^{\top})$ is equivalent to approximating $H$ when the
value vectors $V$ satisfy strong orthogonality and norm constraints.
Conversely, when they do not, ignoring $V$ can lead unbounded approximation
error.

Second, we discuss the kernel-based view of attention, where efficiency is
gained by replacing the exponential kernel (corresponding to
$\mathrm{softmax}$) with other kernel functions Katharopoulos et al. (2020).
We theoretically show that while in the exponential kernel case (corresponding
to $\mathrm{softmax}$), the effect of the norm of the value vectors is
potentially small, switching to other kernels can dramatically increase the
importance of the value vectors. We empirically test this by comparing optimal
sparse attention given different kernel functions, and see that indeed
approximation quality decreases when replacing the exponential kernel,

To conclude, we theoretically and empirically show that approximating the
attention score matrix alone is insufficient, and propose that the research
community should instead approximate the true output of the sub-attention
layer, which importantly includes value vectors. Our code and trained models
are available at https://github.com/ag1988/value_aware_attn.

##  2 Background

We review the kernel-based view of attention Tsai et al. (2019), which will be
instructive in §3.

#### Generalized Attention

Let $\kappa(x,y)=\langle\Phi(x),\Phi(y)\rangle\geq 0$ be a kernel function
with feature map $\Phi:\mathbb{R}^{d}\mapsto\mathcal{H}$ for some implicit
reproducing kernel Hilbert space (RKHS) $\mathcal{H}$. Given a query vector
$q$, keys $k_{1},\ldots,k_{L}$, values $v_{1},\ldots,v_{L}$, all in
$\mathbb{R}^{d}$:

$\mathrm{att}_{\kappa}(q,k_{1},\ldots,v_{1},\ldots)=\frac{\sum_{i=1}^{L}\kappa(q,k_{i})v_{i}}{\sum_{i=1}^{L}\kappa(q,k_{i})},$
(1)

where the normalization induces a probability distribution
$\boldsymbol{\alpha}$ over the value vectors with
$\alpha_{i}=\kappa(q,k_{i})/\sum_{i}\kappa(q,k_{i})$. The most popular use
case is the exponential kernel $\kappa(x,y)=\exp(\langle x,y\rangle)$,
referred to as dot-product attention in Transformers. Some other examples
include the degree-$2$ polynomial kernel $\kappa(x,y)=\langle x,y\rangle^{2}$
and the recently proposed elu kernel $\langle\Phi(x),\Phi(y)\rangle$ with
$\Phi(\cdot)=1+\mathrm{ELU}(\cdot)$ Katharopoulos et al. (2020).

Given $L\gg d$ queries, the attention function (Eq. 1) requires computing
$L\cdot L$ similarity scores for the query-key pairs, which is prohibitive for
long sequences. Sparse attention variants relax this requirement and compute
only a few similarity scores, ignoring the rest:

$\mathrm{att}_{\kappa,S}=\frac{\sum_{i\in S}\kappa(q,k_{i})v_{i}}{\sum_{i\in
S}\kappa(q,k_{i})},$ (2)

for some $S\subseteq\\{1,\ldots,L\\},|S|\ll L$. Methods differ in how $S$ is
determined given the queries and keys, and include use of locality bias
Beltagy et al. (2020), global memory Gupta and Berant (2020), and LSH hashing
Kitaev et al. (2020), among others. Conversely, instead of exactly computing
the attention scores only on a few query-key pairs, dense variants compute an
approximation of the true kernel values for all pairs. Such methods output
$\sum_{i}\beta_{i}\cdot v_{i}$ for some approximation $\boldsymbol{\beta}$ of
the the true attention distribution $\boldsymbol{\alpha}$ Choromanski et al.
(2020); Peng et al. (2021).

##  3 Optimal Sparse Attention

Prior methods for approximating attention, ignored the contribution of the
values vectors $V$. As the true output of the attention sub-layer also depends
on $V$, a natural question is whether it is possible to design better
approximation methods by incorporating $V$, and if so, how much improvement is
even possible?

To answer this, we focus on sparse attention, and analyze the difference
between an oracle sparse approximation that considers the value vectors, and
an oracle approximation that does not. That is, we look at the difference
between the two approximations from the perspective of _expressivity_ ,
ignoring any memory and computational constraints. We denote an optimal value-
aware approximation that uses $r$ key vectors per query by _optimal-v-aware-r_
, and an optimal approximation that ignores value vectors by _optimal-v-
oblivious-r_. We define _optimal-v-oblivious-r_ as the output of Eq. 2 in
which $S$ is selected to be the $r$ indices with the highest attention scores
$\alpha_{i}$’s. This is a natural baseline since this is what current sparse
methods are trying to emulate. We now explicitly derive and analyze the value-
aware objective.

#### Value-aware objective

Let $o=\sum_{i=1}^{L}\alpha_{i}v_{i}$ be a convex combination of
$v_{1},\ldots,v_{L}\in\mathbb{R}^{d}$, corresponding to the true output of the
attention sub-layer. Let $C_{r}=\\{\sum_{i=1}^{L}\beta_{i}v_{i}:\forall i\
\beta_{i}\geq 0,\sum_{i}\beta_{i}=1,|\\{\beta_{i}:\beta_{i}>0\\}|\leq r\\}$
denote the set of points in the polytope of $v_{i}$’s that can be expressed as
a convex combination of at most $r$ value vectors $v_{i}$. The goal of value-
aware approximation is to solve for the point in the constrained region
$C_{r}$ closest to the true output $o$, i.e. $\mathrm{argmin}_{\tilde{o}\in
C_{r}}||o-\tilde{o}||^{2}$. As mentioned, this solution is termed optimal-v-
aware-r.

We consider two extreme cases of $r$: $r=1$ and $r\geq d+1$. For $r\geq d+1$,
the Carathéodory Theorem Bárány and Karasev (2012) states that
$o=\sum_{i}\alpha_{i}v_{i}$ can be expressed as a convex combination of at
most $d+1$ $v_{i}$’s. Hence, if $r\geq d+1$ then $o\in C_{r}$ and the optimal
approximation error is $0$. In most popular architectures, such as BERT Devlin
et al. (2019), $d=64\ll L$. This means that from the point of expressivity,
_optimal-v-aware-65_ can obtain a perfect approximation. Conversely, we will
show in §4 that the performance of _optimal-v-oblivious-65_ is substantially
lower.

At the other extreme, when $r=1$ (a single value vector), the above objective
is equivalent to $\mathrm{argmin}_{i\in(1,\dots,L)}||o-v_{i}||^{2}$ and can be
simplified as

$\begin{split}&\ \mathrm{argmin}_{i}||o||^{2}+||v_{i}||^{2}-2\langle
v_{i},o\rangle\\\ =&\ \mathrm{argmin}_{i}||v_{i}||^{2}-2\langle
v_{i},\sum_{j}\alpha_{j}v_{j}\rangle\\\ =&\
\mathrm{argmin}_{i}||v_{i}||^{2}(0.5-\alpha_{i})-\sum_{j\neq
i}\alpha_{j}\langle v_{i},v_{j}\rangle.\end{split}$ (3)

This equation induces a ranking over value vectors that _depends_ on the value
vectors themselves, in contrast to a value-oblivious ranking induced solely by
attention weights $\boldsymbol{\alpha}$.

If $v_{1},\ldots,v_{L}$ are orthogonal, the above equation further simplifies
to $\mathrm{argmin}_{i}||v_{i}||^{2}(0.5-\alpha_{i})-\sum_{j\neq
i}\alpha_{j}\cdot 0=\mathrm{argmin}_{i}||v_{i}||^{2}(0.5-\alpha_{i})$. In this
case, if some $\alpha_{i}\geq 0.5$ or if $v_{1},\ldots,v_{L}$ have equal
norms, this would further simplify to $\mathrm{argmax}_{i}\alpha_{i}$, and
would therefore be independent of the value-vectors $v_{i}$’s, implying that a
value-oblivious approximation would work well.

But such assumptions on $v_{1},\ldots,v_{L}$ do not hold in general and thus
an approximation that only depends on $\alpha_{i}$’s can be sub-optimal. E.g.,
let $v_{1},v_{2},v_{3}$ be orthogonal vectors $(1,0,0)$, $(0,2,0)$, $(0,0,3)$
respectively and let $\alpha_{1},\alpha_{2},\alpha_{3}$ be $0.25,0.35,0.4$.
Then $v_{3}$ with the highest attention weight $\alpha_{3}$ has a squared
distance of $3.79$ from the true output $\sum_{i}\alpha_{i}v_{i}$ whereas
$v_{1}$ with the least attention weight $\alpha_{1}$ has only $2.49$. In this
case, _optimal-v-aware-1_ induces exactly the opposite ranking of value
vectors compared to _optimal-v-oblivious-1_. Moreover, if we increase the
value $3$ in $v_{3}$ to infinity, the approximation error will also infinitely
grow. This example and, in general, Eq. 3 also show that the optimal ranking
can be significantly different from the one induced by $\alpha_{i}||v_{i}||$
proposed recently by Kobayashi et al. (2020) for obtaining better
interpretability of attention models.

#### Effect of kernel function

Recently, Linear Transformer Katharopoulos et al. (2020) proposed to replace
the existing exponential kernel with more efficient kernels. We now show that
replacing the exponential kernel with a polynomial kernel can lead to a drop
in quality for current sparse approximation methods.

Intuitively, because the kernel function affects the skewness of
$\boldsymbol{\alpha}$, it also affects the difference between the ranking
induced by the optimal-value-aware approximation and the optimal-value-
oblivious one. For simplicity, consider the case of orthogonal value vectors
in which Eq. 3 simplifies to
$\mathrm{argmin}_{i}||v_{i}||^{2}(0.5-\alpha_{i})$. From Eq. 1, we have
$\alpha_{i}=\kappa(q,k_{i})/\sum_{j}\kappa(q,k_{j})$ which is $\langle
q,k_{i}\rangle^{C}/\sum_{j}\langle q,k_{j}\rangle^{C}$ for the degree-$C$
polynomial kernel. For $C=0$, we have $\alpha_{i}=1/L$, which gives
$\mathrm{argmin}_{i}||v_{i}||^{2}$. In this case, the value vectors become
crucial when $\boldsymbol{\alpha}$ is uniform. On the other hand, assuming
distinct inner products, for $C\gg 0$ we will obtain $\max_{i}\alpha_{i}\geq
0.5$, thereby reducing us to $\mathrm{argmax}_{i}\alpha_{i}$, where value
vectors do not affect the approximation. The complexity of the Transformer
grows exponentially with the degree $C$ and thus in practice a low $C$ must be
used (e.g., degree-$2$ polynomial). In such case, $\boldsymbol{\alpha}$ is
likely to be less skewed compared to the exponential kernel and more likely to
induce a sub-optimal ranking.

In the next section, we empirically verify the above observations and show a
significant performance gap between value-oblivious approximations and value-
aware ones.

##  4 Experiments

We empirically verify our observations in the context of training causal and
masked language models, which are known to strongly correlate with performance
on downstream applications Radford et al. (2019); Devlin et al. (2019).

#### Masked LM task

We form examples by sampling sequences and replacing sub-words with <mask>
following the procedure in Devlin et al. (2019). The model is trained to
maximize the log probability of the masked out tokens and we evaluate the
_error_ of the model as the percentage of masked tokens predicted incorrectly.
As approximate attention becomes increasingly relevant for long sequences, we
train RoBERTa-$4096$ on sequences of length $4096$ (Fig. 3). Training was
warm-started using RoBERTa-base Liu et al. (2019). Full details on the
experimental setup are in §A.1. After training the model for $\sim 2.5$M
steps, the error of the model (that is, proportion of incorrect predictions)
on the evaluation set was $24.2$ (compared to $26.6$ for an analogous training
on $512$-long sequences), ensuring that tokens in RoBERTa-$4096$ indeed attend
over longer distances and result in higher quality representations. We then
replace the attention function of the trained model with various approximation
schemes and evaluate the resulting model on the evaluation set.

We first compare _optimal-v-oblivious-r_ to _optimal-v-aware-r_. We know that
the approximation error of value-aware approximation is $0$ for $r>64$. For
$r=1$, we exhaustively go through all possible values and choose the one that
minimizes the value-aware objective. As seen in Fig. 1 and Table 1, there is
substantial gap between the two approximations. For instance, _optimal-v-
oblivious-65_ gives an MLM error of $43.5$ whereas the error of _optimal-v-
aware-65_ is $24.2$, since it can perfectly approximate full attention.
Moreover, we compare _optimal-v-oblivious-r_ to existing approximations: (a)
_sliding-window-r_ , where a position attends to $r/2$ positions to its left
and right), (b) LSH attention Kitaev et al. (2020) and (c) ORF attention
Choromanski et al. (2020). Fig. 1 shows that _sliding-window-r_ trails behind
_optimal-v-oblivious-r_. LSH attention, which tries to emulate _optimal-v-
oblivious-r_ , either requires a large number of hash rounds or a large chunk
size. Similarly, the dense approximation, ORF, provides an unbiased
approximation of the exponential kernel but suffers from high variance in
practice.

exact |  | OVO
---
$1$
| OVO
---
$65$
| OVA
---
$1$
| OVA
---
$65$
| ORF
---
$256$ features
| LSH
---
$r=64$
$4$-rounds
| LSH
---
$r=512$
$16$-rounds
24.2 | 96.6 | 43.5 | 88.6 | 24.2 | 89.79 | 90.39 | 26.11
Table 1: MLM error of RoBERTa-$4096$ on the evaluation set using approximate attention described in §4. OVO: _optimal-v-oblivious_ , OVA: _optimal-v-aware_. In LSH, each query attends to a total of $r$ keys per hash round. | exact | OVO-1 | OVO-65 | OVA-1 | OVA-65
---|---|---|---|---|---
exponential | 30.5 | 1031.1 | 33.5 | 280.3 | 30.5
polynomial (deg 2) | 34.2 | 6700.2 | 310.2 | 1005.4 | 34.2
elu | 35.3 | 1770.6 | 62.7 | 837.4 | 35.3
Table 2: Evaluation perplexity of models using approximate attention. OVO:
_optimal-v-oblivious_ , OVA: _optimal-v-aware_. Figure 1: Evaluation of MLM
error of RoBERTa-$4096$ after replacing vanilla attention with approximation
schemes. Dashed line denotes error using vanilla attention. Figure 2:
Evaluation loss (base e) of _optimal-v-oblivious-r_ oracle on the causal LM
task for distinct kernel functions.

#### Causal LM task

To investigate the effect of the kernel function on the quality of value-
oblivious methods, we train a $6$-layer Transformer LM over 512 tokens on
WikiText-103 Merity et al. (2017) (details in §A.2). We train $3$ models with
identical hyperparameters using the exponential, degree-$2$ polynomial, and
elu kernels respectively and evaluate the trained models with value-aware and
value-oblivious approximations. Again, _optimal-v-aware-r_ substantially
outperforms _optimal-v-oblivious-r_ (Table 2), pointing to the potential of
working on approximating the value-aware objective. More importantly,
comparing the approximation quality across different kernel functions (Fig.
2), we see that the gap between the three kernels is small when using full
attention (512 keys) vectors. However, convergence is much slower for the elu
kernel, and especially the degree-$2$ polynomial, demonstrating that the
approximation based on the top-$r$ key vectors is sub-optimal when switching
to a less skewed kernel, which is more affected by the value vectors.

##  5 Conclusions

In this work, we provide theoretical and empirical evidence against current
practice of focusing on approximating the attention matrix in Transformers,
while ignoring the value vectors. We propose a value-aware objective and argue
that the efforts to develop more efficient Transformers should consider this
objective function as a research target.

## Acknowledgments

This research was partially supported by The Yandex Initiative for Machine
Learning, and the European Research Council (ERC) under the European Union
Horizons 2020 research and innovation programme (grant ERC DELPHI 802800).

## References

  * Bárány and Karasev (2012) I. Bárány and R. Karasev. 2012.  Notes about the carathéodory number.  _Discrete & Computational Geometry_, 48:783–792.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020.  Longformer: The long-document transformer.  _arXiv preprint arXiv:2004.05150_.
  * Child et al. (2019) Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. 2019.  Generating long sequences with sparse transformers.  _arXiv preprint arXiv:1904.10509_.
  * Choromanski et al. (2020) Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, A. Gane, Tamás Sarlós, P. Hawkins, J. Davis, Afroz Mohiuddin, L. Kaiser, D. Belanger, Lucy J. Colwell, and Adrian Weller. 2020.  Rethinking attention with performers.  _ArXiv_ , abs/2009.14794.
  * Cohan et al. (2018) Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, W. Chang, and Nazli Goharian. 2018.  A discourse-aware attention model for abstractive summarization of long documents.  In _NAACL-HLT_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _North American Association for Computational Linguistics (NAACL)_ , pages 4171–4186, Minneapolis, Minnesota.
  * Edunov et al. (2018) Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018.  Understanding back-translation at scale.  In _Empirical Methods in Natural Language Processing (EMNLP)_.
  * Gupta and Berant (2020) Ankit Gupta and Jonathan Berant. 2020.  Gmat: Global memory augmentation for transformers.  _arXiv preprint arXiv:2006.03274_.
  * Hewitt and Manning (2019) John Hewitt and Christopher D Manning. 2019.  A structural probe for finding syntax in word representations.  In _North American Association for Computational Linguistics (NAACL)_ , pages 4129–4138.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  _arXiv preprint arXiv:2004.04906_.
  * Katharopoulos et al. (2020) Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. 2020.  Transformers are rnns: Fast autoregressive transformers with linear attention.  In _International Conference on Machine Learning_.
  * Kitaev et al. (2020) Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020.  Reformer: The efficient transformer.  In _International Conference on Learning Representations_.
  * Kobayashi et al. (2020) Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020.  Attention is not only a weight: Analyzing transformers with vector norms.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7057–7075, Online. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Merity et al. (2017) Stephen Merity, Caiming Xiong, James Bradbury, and R. Socher. 2017.  Pointer sentinel mixture models.  In _International Conference on Learning Representations_.
  * Peng et al. (2021) Hao Peng, Nikolaos Pappas, Dani Yogatama, Roy Schwartz, Noah Smith, and Lingpeng Kong. 2021.  Random feature attention.  In _International Conference on Learning Representations_.
  * Peters et al. (2018) M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018.  Deep contextualized word representations.  In _North American Association for Computational Linguistics (NAACL)_.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander Miller, and Sebastian Riedel. 2019.  Language models as knowledge bases? In _Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)_.
  * Qiu et al. (2019) Jiezhong Qiu, Hao Ma, Omer Levy, Scott Wen-tau Yih, Sinong Wang, and Jie Tang. 2019\.  Blockwise self-attention for long document understanding.  _arXiv preprint arXiv:1911.02972_.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Rae et al. (2020) Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier, and Timothy P. Lillicrap. 2020.  Compressive transformers for long-range sequence modelling.  In _International Conference on Learning Representations_.
  * Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. 2020.  How much knowledge can you pack into the parameters of a language model?  _ArXiv_ , abs/2002.08910.
  * Roy et al. (2020) Aurko Roy, Mohammad Saffar, Ashish Vaswani, and David Grangier. 2020.  Efficient content-based sparse attention with routing transformers.  _arXiv preprint arXiv:2003.05997_.
  * Saxe et al. (2013) Andrew M Saxe, James L McClelland, and Surya Ganguli. 2013.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks.  _arXiv preprint arXiv:1312.6120_.
  * Tay et al. (2020a) Yi Tay, Dara Bahri, Liu Yang, Donald Metzler, and Da-Cheng Juan. 2020a.  Sparse sinkhorn attention.  In _International Conference on Machine Learning_.
  * Tay et al. (2020b) Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2020b.  Efficient transformers: A survey.
  * Tsai et al. (2019) Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019.  Transformer dissection: An unified understanding for transformer’s attention via the lens of kernel.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems (NIPS)_ , pages 5998–6008.
  * Wang et al. (2020) Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. 2020.  Linformer: Self-attention with linear complexity.  _ArXiv_ , abs/2006.04768.
  * Ye et al. (2019) Zihao Ye, Qipeng Guo, Quan Gan, Xipeng Qiu, and Zheng Zhang. 2019.  Bp-transformer: Modelling long-range context via binary partitioning.  _arXiv preprint arXiv:1911.04070_.
  * Zhu et al. (2015) Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015.  Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.  In _The IEEE International Conference on Computer Vision (ICCV)_.

##  Appendix A Supplemental Material

###  A.1 Masked LM task

The instances for the MLM task (§4) were formed separately using the corpora
listed in Table 3. For each dataset, after appending </s> token at the end of
each document, the documents were arranged in a random order and concatenated
into a single long text which was then tokenized into a list of sub-words.
Depending upon the final input sequence length $L$ of the experiment
($512$/$4096$) this list was chunked into full length $L-2$ sequences which
were then masked randomly following Devlin et al. (2019) and enclosed within
<s> and </s> tokens. To handle sequences longer than $512$ tokens, the
positional embeddings were used following Gupta and Berant (2020). The
learning curves of RoBERTa-$512$ and RoBERTa-$4096$ are in Fig. 3.

Figure 3: Evaluation error on the MLM task using vanilla attention (computing the full attention matrix). corpus | all | training | evaluation
---|---|---|---
Wikipedia ($10$/$2017$) |  $2.67$B |  $1.53$B |  $1.02$M
BookCorpus Zhu et al. (2015) |  $1.06$B |  $1.02$B |  $1.02$M
ArXiv Cohan et al. (2018) |  $1.78$B |  $1.53$B |  $1.02$M
PubMed Cohan et al. (2018) |  $0.47$B |  $510$M |  $1.02$M
PG19 Rae et al. (2020) |  $3.06$B |  $510$M |  $1.02$M
Table 3: Number of tokens in the datasets used for MLM training.

#### Hyperparameters

For convenience, we denote the training hyperparameters using the following
abbreviations, INS: number of training instances, BSZ: number of instances in
a batch, ISZ: instance size, SQL: final input sequence length after
rearranging BSZ instances each of length ISZ, LR: learning rate, WRM: linear
LR warm-up proportion, EP: number of epochs, STP: number of optimizer steps,
GAC: gradient accumulation steps, POSq: whether (y/n) $q$ part is included in
positional embeddings. The hyperparameters are listed in Table 4.

model | init | BSZ | ISZ | SQL | LR | WRM | EP | STP | POSq
---|---|---|---|---|---|---|---|---|---
RoBERTa-$512$ | RoBERTa | $8$ | $512$ | $512$ |  $5$e-$6$ | $0.1$ | $2$ |  $2.476$M | n
RoBERTa-$4096$ | RoBERTa | 8 | $512$ | $4096$ |  $5$e-$6$ | $0.1$ | $2$ |  $2.476$M | y
Table 4: Training hyperparameters. Common parameters: INS=$10$M, dropout-
rate=$0.0$, optimizer=Bert-Adam, $\beta_{1}$=$0.9$, $\beta_{2}$=$0.98$,
weight-decay=$0.01$, max-grad-norm=$5.0$, seed=$42$, GAC=$1$.

#### Details of LSH attention

Given $L$ queries and $L$ keys in $\mathbb{R}^{d}$, in each hash round, we
sample a new matrix $R\in\mathbb{R}^{\frac{C}{2}\times d}$ of standard
gaussians and hash the queries and keys as
$H_{R}(x)=\mathrm{argmax}([-Rx;Rx])\in\\{1,\ldots,C\\}$. We rearrange the
queries (and similarly keys) according to their hash value, breaking ties
using the original position, and then chunk them into $L/B$ chunks of $B$
vectors each. Denoting these chunks as $Q_{1},\ldots,Q_{L/B}$ and
$K_{1},\ldots,K_{L/B}$, for each query in $Q_{i}$ we compute its similarity
scores with respect to all keys in $K_{i-1},K_{i}$. I.e. in each hash round a
query attends to $r=2B$ keys. For each query, these similarity scores are
accumulated over different hash rounds, and at the end normalized by their sum
to get normalized attention scores over the keys. As recommended in the
original paper Kitaev et al. (2020), we use $C=2L/B=4L/r$ which in practice
can be sub-optimal as rearrangement destroys the original locality structure.

#### Details of ORF attention

Given $L$ queries and $L$ keys in $\mathbb{R}^{d}$ we divide each vector by
$d^{\frac{1}{4}}$ to account for the temperature term in dot-product
attention. For a given number $F$ of features, we sample a random orthogonal
matrix $R\in\mathbb{R}^{F\times d}$ as described in Saxe et al. (2013) and
provided as a tensor initialization option in PyTorch. We then map each vector
to the feature space as
$\Phi(x)=\frac{1}{\sqrt{F}}\exp\left(Rx-\frac{||x||^{2}}{2}\right)\in\mathbb{R}^{F}$
where ($-$) and $\exp$ operations are applied element-wise. Similarity score
of a query-key pair $(q,k)$ is computed as $\langle\Phi(q),\Phi(k)\rangle$ and
and is normalized by the sum of the similarity scores of $q$ with all the
keys. Computing this directly leads to numerical instability so we instead
compute
$\Phi(q)=\frac{1}{\sqrt{F}}\exp\left(Rq-\frac{||q||^{2}}{2}-\max(Rq)\right)$
for queries and
$\Phi(k)=\frac{1}{\sqrt{F}}\exp\left(Rk-\frac{||k||^{2}}{2}-\max(RK)\right)$
where $K$ is the matrix of all keys and $\max$ is over all elements of input.

The main idea behind ORF attention is that, for a vector $w$ of standard
gaussians, $\langle w,x\rangle\sim\mathcal{N}(0,||x||^{2})$ and from the
properties of log-normal distributions, $\mathbb{E}_{w}[\exp(\langle
w,x\rangle)]=\exp(\frac{||x||^{2}}{2})$. So, $\mathbb{E}_{w}[\exp(\langle
w,q\rangle)\cdot\exp(\langle w,k\rangle)]=\mathbb{E}_{w}[\exp(\langle
w,q+k\rangle)]=\exp(\frac{||q+k||^{2}}{2})=\exp(\langle
q,k\rangle+\frac{||q||^{2}}{2}+\frac{||k||^{2}}{2})$. Appropriately scaling
both sides gives, $\mathbb{E}_{w}[\exp(\langle
w,q\rangle-\frac{||q||^{2}}{2})\cdot\exp(\langle
w,k\rangle-\frac{||k||^{2}}{2})]=\exp(\langle q,k\rangle)$, which is exactly
the term for the exponential kernel.

###  A.2 Causal LM task

For this task, we used the language modeling framework provided by
Faiseq222https://github.com/pytorch/fairseq.

#### Model and training details

number of decoder layers: $6$, hidden size: $512$, head size: $64$, number of
model parameters: $156$M, dataset: WikiText-$103$, training examples:
$1801350$, input sequence length: $512$, $\beta_{1}$=$0.9$,
$\beta_{2}$=$0.98$, weight-decay: $0.01$, gradient clip-norm: none, learning
rate: $0.0005$, learning rate schedule: inverse square root, number of warmup
updates: $4000$, batch size: $128$, epochs: $20$, number of steps: $31520$,
minimum context-window during evaluation on test-set: $400$.


# Privacy-Preserving Face Recognition Using Random Frequency Components

Yuxi Mi1 Yuge Huang211footnotemark: 1 Jiazhen Ji2 Minyi Zhao1
Jiaxiang Wu2 Xingkun Xu2 Shouhong Ding2 Shuigeng Zhou1
1 Fudan University 2 Tencent Youtu Lab
{yxmi20, zhaomy20<EMAIL_ADDRESS>
{yugehuang, royji, willjxwu, xingkunxu<EMAIL_ADDRESS>Equal
contributions.Corresponding author.

###### Abstract

The ubiquitous use of face recognition has sparked increasing privacy
concerns, as unauthorized access to sensitive face images could compromise the
information of individuals. This paper presents an in-depth study of the
privacy protection of face images’ visual information and against recovery.
Drawing on the perceptual disparity between humans and models, we propose to
conceal visual information by pruning human-perceivable low-frequency
components. For impeding recovery, we first elucidate the seeming paradox
between reducing model-exploitable information and retaining high recognition
accuracy. Based on recent theoretical insights and our observation on model
attention, we propose a solution to the dilemma, by advocating for the
training and inference of recognition models on randomly selected frequency
components. We distill our findings into a novel privacy-preserving face
recognition method, PartialFace. Extensive experiments demonstrate that
PartialFace effectively balances privacy protection goals and recognition
accuracy. Code is available at: https://github.com/Tencent/TFace.

##  1 Introduction

Face recognition (FR) is a landmark biometric technique that enables a person
to be identified or verified by face. It has seen remarkable methodological
breakthroughs and rising adoptions in recent years. Currently, considerable
applications of face recognition are carried out online to bypass local
resource constraints and to attain high accuracy [19]: Face images are
collected by local devices such as cell phones or webcams, then outsourced to
a service provider, that uses large convolutional neural networks (CNN) to
extract the faces’ identity-representative templates and matches them with
records in its database.

During the process, the original face images are often considered sensitive
data under regulatory demands that are unwise to share without privacy
protection. This fosters the studies of privacy-preserving face recognition
(PPFR), where cryptographic [33, 8, 14, 17, 7, 21, 47, 44] and perturbation-
based [48, 3, 46, 12, 29, 18, 28, 30] measures are taken to prevent face
images from unauthorized access of, _e.g_., wiretapping third parties. Face
images are converted to protective representations that their visual
information is both concealed and cannot be easily recovered [2].

This paper advocates a novel PPFR scheme, to learn face images from random
combinations of their partial frequency components. Our proposed PartialFace
can protect face images’ visual information and prevent recovery while
maintaining the high distinguishability of their identities.

Figure 1: Paradigm comparison among other frequency-based methodologies and
PartialFace. (a) Pruning low-frequency channels can conceal visual information
but stop no recovery. (b) A vanilla method that uses fixed channel subsets to
impede recovery suffers downgraded accuracy. (c) PartialFace addresses the
dilemma by training and inferring from random channels.

We start with the disparity in how humans and models perceive images. Recent
image classification studies suggest that models’ predictions are determined
mainly by the images’ high-frequency components [39, 45], which carry
negligible visual information and are barely picked up by humans. We extend
the theory to face recognition from a privacy perspective, to train and infer
the model on pruned frequency inputs: As depicted in Fig. 1(a), we decouple
the face images’ frequency compositions via discrete cosine transform (DCT),
which breakdown every spatial domain into a certain number of (typically 64)
constituent bands, _i.e_., frequency channels. We prune the human-perceivable
low-frequency channels and exploit the remaining. We find the processed face
images become almost visually indiscernible, and the model still works
accurately.

Pruning notably conceals visual information. However, to what degree can it
impede recovery? We define recovery as the general attempts to reveal the
faces’ visual appearances from shared protected features using trained attack
models. Notice we may prune very few (say, about 10) channels, if according to
human perception. At the same time, the remaining high-frequency channels
being shared, hence exposed, are quite numerous and carry a wealth of model-
perceivable features. While the information abundance can benefit a
recognition model, it is also exploitable by a model carrying out attacks, as
both may share similar perceptions. Therefore, as we later experimentally
show, the attacker can recover visual features from high-frequency channels
with ease in the absence of additional safeguards, rendering privacy
protection useless.

An intuitive tactic to reinstate protection is to reduce the attacker’s
exploitable features by training on a small portion of fixed channels, as
shown in Fig. 1(b). However, we find the reduction also severely impairs the
accuracy of trained recognition models. Evidence on the models’ attention, as
later shown, attributes their utility downgrade to being incapable of learning
a complete set of facial features, as vital channels describing some local
features may be pruned.

Training on subsets of channels hence seems contradictory to the privacy-
accuracy equilibrium. Fortunately, we can offer a reconciliation getting
inspired by a recent time-series study [51]. It proves under mild conditions,
models trained on random frequency components can preserve more entirety’s
information than on fixed ones, plausibly by alternately learning from
complementary features. We hence propose a novel address to the equilibrium
based on its theoretical findings and our observation on model attention: For
any incoming face image, we arbitrarily pick a small subset of its high-
frequency channels. Therefore, our recognition model is let trained and
inferred from image-wise random chosen channels, illustrated in Fig. 1(c).

We further show that randomness can be adjusted to a moderate level, by
choosing channels from pre-specified combinations and perturbations called
ranks, to keep privacy protection while reconciling technical constraints to
ease training. At first glance, our randomized approach may seem counter-
intuitive as it is common wisdom that models require consistent forms of
inputs to learn stably. However, since DCT produces spatially correlated
frequency channels that preserve the face’s structural information, as later
illustrated in Fig. 3, it turns out the model generalizes quite naturally.
Experimental analyses shows our PartialFace well balances privacy and
accuracy.

The contributions of our paper are three-fold:

  1. 1.

We present an in-depth study of the privacy protection of face images,
regarding the privacy goals of concealing visual information and impeding
recovery.

  2. 2.

We propose two methodological advances to fulfill the privacy goals, pruning
low-frequency components and using randomly selected channels, based on the
observation of model perception and learning behavior.

  3. 3.

We distill our findings into a novel PPFR method, PartialFace. We demonstrate
by extensive experiments that our proposed method effectively safeguards
privacy and maintains satisfactory recognition accuracy.

Figure 2: Pipeline of PartialFace. DCT turns face images into the frequency
domain, where low-frequency channels are first pruned to remove the human
perception. A small subset of channels is selected and permuted at random
according to pre-specified combinations $\\{\mathbf{S},\mathbf{P}\\}$, or
ranks. The model is trained and inferred from random subsets to address the
equilibrium between accuracy and privacy.

##  2 Related work

###  2.1 Face recognition

The current method of choice for face recognition is CNN-based embedding. The
service provider trains a CNN with a softmax-based loss to map face images
into one-dimensional embedding features which achieve large inter-identity and
small intra-identity discrepancies. While the state-of-the-art (SOTA) FR
methods [6, 38, 20] achieve impressive task utility in real-world
applications, their attention to privacy protection could be deficient.

###  2.2 Privacy-preserving face recognition

The past decade witnessed significant advances in privacy-preserving face
recognition [41, 24, 25]. We roughly categorize the related arts into two
branches:

Cryptographic methods perform recognition on encrypted images, or by executing
dedicated security protocols. Many pioneering works fall under the category of
homomorphic encryption (HE) [33, 8, 14] or secure multiparty computation (MPC)
[21, 47, 44], to securely carry out necessary computations such as model’s
feature extraction. Some methods also employ various crypto-primitives
including one-time-pad [7], matrix encryption [17], and functional encryption
[1]. The major pain points of these methods are generally high latency and
expensive computational costs.

Perturbation-based methods transform face images into protected
representations that are difficult for unauthorized parties to discern or
recover. Many methods leverage differential privacy (DP) [48, 3, 18, 22], in
which face images are obfuscated by a noise mechanism. Some use autoencoders
[28] or adversarial generative networks (GAN) [29, 18] to recreate visually
distinct faces that maintain a constant identity. Others compress the original
images into compact representations by mapping discriminative components into
subspaces [16, 4, 27], or anonymize the images by clustering their features
[12]. These methods face a common bottleneck of task utility: As the
techniques they employ essentially distort the original images, their
protection often impairs recognition accuracy.

###  2.3 Learning in the frequency domain

Converting spatial or temporal data into the frequency domain provides a
powerful way to extract interesting signals. For instance, the Fourier-based
discrete cosine transform [5] is used by the JPEG standard [37] to enable
image compression. Researches in deep learning [9, 45] suggest models trained
on images’ frequency components can perform as well as trained on the original
images. Advance [39] further reveals humans and models perceive low- and high-
frequency components differently. In the realm of PPFR, three recent methods
[42, 26, 15] are closely related to ours as we all conceal visual information
by exploiting the split in human and model perceptions. However, these methods
bear inadequacy in defending recovery, according to our previous discussion of
channel redundancy and later testified in experiments.

##  3 Methodology

This section discusses the motivation and technical details behind
PartialFace. PartialFace is named after its key protection mechanism, where
the model only exploits face images’ partial frequency components to reduce
information exposure. Figure 2 describes its framework.

###  3.1 Overview

Recall our privacy goals are to conceal the face images’ visual information
and to prevent recovery attacks on them. They respectively target the
adversarial capability of humans and models. We naturally concretize them into
two technical aims, on how to eliminate human perception, and, on how to
reduce exploitable information for attack models.

To eliminate human perception, we leverage the finding that models’ utility
can be maintained almost in full on the images’ high-frequency channels [39].
Contrarily, humans mostly perceive the images’ low-frequency channels, as only
they carry signals of conspicuous amplitude for human eyes to discern. Whereby
spatial-frequency transforms, these channels can be easily located and pruned
from raw images. We concretely opt for DCT as our transform as it facilitates
the calculation of energy, to serve as the quantification for human
perceivable information. Experimentally, we can prune a very small portion of
channels to eliminate 95% of total energy, hence satisfying our aim.

Reducing the attacker’s exploitable information requires further pruning of
high-frequency channels, as previously discussed. However, it is equally
crucial to maintain the recognition model’s accuracy, which presents a seeming
dilemma, as the training features utilizable by recognition and attack models
are tightly interwoven in these channels. We offer a viable way as our major
contribution: We notice different channel subsets each carrying certain local
facial features. Hence, we feed the model with a random subset from each face
image, with the hope to let the model learn the entire face’s impression from
different images’ complementary local feature partitions. This approach is
proved feasible by recent theoretical studies [51] and demonstrated by our
experiments. Therefore, information exposure is minimized as only each image’s
chosen subset is exposed, and the model still performs surprisingly well.

Despite being well justified by theory, we find sampling channels under
complete randomness could under-perform in actual training due to two
technical limitations: the inadequacy in training samples and the biased
sampling within mini-batches. We propose two targeted fixes to reconcile the
constraints, by augmenting samples and seeking a moderate level of randomness.
We find the modified approach satisfactorily addresses the privacy-accuracy
equilibrium.

###  3.2 Conceal visual information

Figure 3: The DCT process. The produced (d) frequency channels keep the
spatial structure to (a) the original image, though only low-frequency
channels are discernible bu humans. We use (c) 2D grids to describe the
frequency spectrum. Each cell in the grid represents one frequency channel of
$H$$\times$$W$.

We first set up some basic notions: $\langle X,y\rangle$ denotes a data sample
of a face image and its corresponding label. $\mathbf{x}$ denotes the
frequency composition of $X$ and $x_{i}$ denotes its individual frequency
channels. $f(\cdot;\theta)$ denotes the recognition model parameterized by
$\theta$. $l(\cdot,\cdot)$ denotes a generic loss function (_e.g_., ArcFace).
$\mathcal{T}(\cdot)$ denotes the discrete cosine transform and
$\mathcal{T}^{-1}(\cdot)$ denotes its inverse transform.

To conceal visual information, recall humans and models mainly perceive low-
and high-frequency components, respectively. Therefore, we need to find a
frequency decomposition of $X$=$\\{\mathbf{x}_{l},\mathbf{x}_{h}\\}$, where
$\mathbf{x}_{l},\mathbf{x}_{h}$ are the respective low- and high-frequency
channels, then prune $\mathbf{x}_{l}$.

We presume $X$ is with the shape of $(H,W)$. We employ DCT to transform $X$’s
spatial channels into a frequency spectrum. While an RGB image typically has 3
spatial channels, we pick one for simplicity. We perform an 8-fold up-sampling
ahead to turn $X$ into $(8H,8W)$. As DCT later divides $H$ and $W$ by 8, this
makes sure the resulting frequency channels can be fed into the model as
usual. Figure 3 illustrates the process of DCT, where
$\mathbf{x}$=$\mathcal{T}(X)$. Concretely, $X$ is divided into $(8,8)$-pixel
blocks. DCT turns each block into a 1D array of 64 frequency coefficients and
reorganizes all coefficients from the same frequency across blocks into an
$(H,W)$ frequency channel (there are 64 of them), that is spatially correlated
to the original $X$. As a result, $X$ is turned into $\mathbf{x}$ of
$(64,H,W)$.

We then decouple $\mathbf{x}$ into $\\{\mathbf{x}_{l},\mathbf{x}_{h}\\}$.
Notice that human-perceivable low-frequency channels should meanwhile be those
with higher amplitude, as humans preferentially identify signals with
conspicuous value changes. We measure a channel’s amplitude by its channel
energy $e(\cdot)$, which is the mean of amplitudes of all its elements:

$e(x)=\frac{1}{HW}\sum_{i=0}^{H-1}{\sum_{j=0}^{W-1}{|x^{i,j}|}}.$ (1)

We choose $\sigma$=10 highest-energy channels as $\mathbf{x}_{l}$ to be pruned
and the rest as $\mathbf{x}_{h}$ to be available to models. We experimentally
find $\sum_{x\in\mathbf{x}_{l}}{e(x)}\geq 0.95\sum_{x\in\mathbf{x}}{e(x)}$,
and a model trained with
$\operatorname*{arg\,min}_{\theta}{l(f(\mathbf{x}_{h},\theta),y)}$ obtain
close accuracy to one trained with
$\operatorname*{arg\,min}_{\theta}{l(f(\mathbf{x},\theta),y)}$. For privacy,
example of a channel-pruned image in Fig. 4(a) show that most visual
information is concealed. While its recovery is still carried out quite
successfully, we are to address the issue in the following.

###  3.3 Impede easy recovery

Figure 4: The visual appearance and recovery of an example image of (a) after
pruning, (b) a fixed subset, and (c) a random subset. This shows protection is
enhanced by removing human perception, reducing the number of channels, and
randomness, respectively.

To elucidate the seeming dilemma between reducing model-available channels and
retaining high recognition accuracy, we first introduce an intuitive
protection, referred to as the vanilla method. It trains models on a fixed
small subset of channels straightforwardly: Concretely for every $X$, the
model pick $s$<$d$ channels $\mathbf{x}_{s}$=$(x_{a_{1}},\dots,x_{a_{s}})$
from its high-frequency $\mathbf{x}_{h}$=$(x_{1},\dots,x_{d})$, where
$a_{1}$<$a_{2}$<$\cdots$<$a_{s}$ are fixed indices. We benchmark the trained
model on IJB-B/C with $s$=9,18,36 and $d$=54, and verify its privacy.

The model indeed effectively prevents recovery, as in Fig. 4(b) its recovered
image is very blurred. However, the benchmark in Fig. 5(a-b) suggests its
accuracy dropped by at most 9%. To find out why is the model’s performance
impaired, we inspect its attention via Grad-CAM [34]. Results are exemplified
in Fig. 5(c-d), which we note show similar patterns among different face
images. We derive two observations: (1) Unlike high-utility models that
typically have attention to the full face, this model only gains attention to
certain local features, which suggests some vital face-describing information
is missing in $\mathbf{x}_{s}$; (2) By training vanilla models on different
$\mathbf{x}_{s}$, we find it can acquire distinct information from different
channels, suggesting its attention is correlated to the specific choices of
$\mathbf{x}_{s}$.

The first observation suggests the cause of the accuracy downgrade. Meanwhile,
the second pursues us to consider a viable bypass: We can gather complementary
local features from different images and initiate mixed training, to let the
model learn a holistic impression of the entirety while keeping the individual
information exposure minimized. Our idea is corroborated by [51], which proves
in time-series, training models on random frequency components can preserve
more information compared to training on fixed ones.

We concrete our theory into a randomized strategy, by advocating training and
inferring the recognition model on image-wise randomly chosen channels.
Formally, we construct matrix $S\in\\{0,1\\}^{d\times s}$, with $s_{ij}$=1 if
$i$=$a_{j}$ and $s_{ij}$=0 if otherwise. We also construct permutation matrix
$P\in\\{0,1\\}^{s\times s}$. For each $X$, we draw $(a_{1},\dots,a_{s})$
(therefore $S$) and $P$ uniformly at random, and calculate

Figure 5: The privacy and accuracy dilemma. (a-b) The vanilla model trained on
fixed channels under-performs on IJB-B/C (orange), compared to the unprotected
baseline (blue) and PartialFace (green). (c-d) The model gains local attention
on foreheads and cheeks, suggesting missing face-describing information. The
attention varies by the specific choice of $\mathbf{x}_{s}$.

$\mathbf{x}_{s}=\mathbf{x}_{h}\cdot S\cdot{P}.$ (2)

Here, $S$ randomly pick $s$ channels out of $\mathbf{x}_{h}$, then their order
is permuted by $P$. We introduce $P$ to further impede recovery, as later
discussed in Sec. 3.5. The model is trained in the same way as standard FR
using $\langle\mathbf{x}_{s},y\rangle$. However, it does not require to know
the sample-wise specific $\\{S,P\\}$, which is favorable for privacy. The
approach sounds a bit counter-intuitive since receiving random inputs seems to
mess with the model’s understanding. However, recall that the DCT frequency
channels are spatially correlated to $X$ and each other. The randomness only
pertains to the frequency spectrum while the face’s structural information is
preserved and unaltered. The model, therefore, can associate different
$\mathbf{x}_{s}$ to the same $X$ quite naturally.

###  3.4 Reconcile technical constraints

Equation 2 distills our core idea to sampling $\mathbf{x}_{s}$ at complete
random. However, for practical training, there need to further reconcile two
technical constraints to achieve satisfying model performance: First to note
that $\langle\mathbf{x}_{s},y\rangle$ are more varied in feature
representations than the unprotected $\langle X,y\rangle$, owing to the
randomized frequency. It hence could plausibly take the model with more
training samples and a longer training time to approach a well-generalized
convergence. Second, ideally, channels at random should be sampled with equal
probability to ensure balanced learning of different local features. However,
when the training data is partitioned into small mini-batches, sampling is
often less unbiased batch-wise so the occurrence of different channel
combinations may vary greatly, which we find could undermine the training
stability.

Targeting the constraints, we slightly adjust our established approach in two
ways. We first augment the training dataset by picking multiple
$\mathbf{x}_{s}$ each time from a face image $X$: From $X$ we sample
$\\{\mathbf{x}_{s}^{1},\dots,\mathbf{x}_{s}^{r}\\}$, where $r$ determines the
degree of augmentation. Each $\mathbf{x}_{s}^{i}$ is independently drawn and
appended to the training dataset as individual training samples. The dataset
is then fully shuffled so that $\mathbf{x}_{s}^{i}$ corresponding to the same
face image cannot be easily associated.

We also control the randomness to a moderate level by specifying the possible
combinations of $S,P$ in advance. Concretely, we opt for choosing one $S,P$
from $\mathbf{S}$=$\\{S_{1},\dots,S_{m}\\}$ and
$\mathbf{P}$=$\\{P_{1},\dots,P_{n}\\}$ respectively, where
$\mathbf{S},\mathbf{P}$ are determined by the service provider. We
specifically require $\\{S_{1},\dots,S_{n}\\}$ to be a non-overlapping
partition of channels from $\mathbf{x}_{h}$ (_i.e_., divide $\mathbf{x}_{h}$
into equal-length subsets) to maximize the use of $\mathbf{x}_{h}$ and reduce
model bias. Therefore, each $\mathbf{x}_{s}^{i}$ is picked from one of
$m$$\times$$n$ fixed combinations of channels, called ranks. This allows us to
facilitate the training and overcome sampling biases. The service provider
shares $\\{\mathbf{S},\mathbf{P}\\}$ with all local devices, so the latter can
generate their query $\mathbf{x}_{s}$ accordingly. During inference, the model
is expected to provide consistent results of the same query $X$ regardless of
the choice of rank, since it learned about a mapping from local features to
the face’s entirety. We later testify to it in Sec. 4.3. Meanwhile, the model
is still unaware of the sample-wise specific choice of $\\{S,P\\}$ as the
recognition relies on nothing else but $\mathbf{x}_{s}$ alone. Hence privacy
is maintained.

To conclude, we present PartialFace that train and infer the recognition model
with $\operatorname*{arg\,min}_{\theta}{l(f(\mathbf{x}_{s}^{i},\theta),y)}$,
where $X$=$\\{\mathbf{x}_{l},\mathbf{x}_{h}\\}$ and
$\mathbf{x}_{s}^{i}$=$\mathbf{x}_{h}\cdot S\cdot{P}$ in random, parameterized
by $\\{\mathbf{S},\mathbf{P}\\}$ and $(\sigma,s,r,m,n)$. Later analyses in
Secs. 4.2 and 4.3 show PartialFace overcomes the drawback of the vanilla
method, plus outperforms most prior arts, to achieve satisfactory recognition
accuracy.

###  3.5 Enhance privacy with randomness

The benefit of randomness is multi-fold. We have discussed it for now on
helping address the accuracy and privacy balance and enhance recognition
performance. In retrospect, we briefly elaborate on how randomness further
safeguards privacy to a large extent.

Recall we remove visual information by pruning $\mathbf{x}_{l}$ and impede
recovery by choosing a subset of $\mathbf{x}_{s}$. After that, randomness
further obstructs recovery: To impose recovery, the attacker exploits not only
the channels’ information but also their relative orders and positions in the
frequency spectrum. We introduce $P$ to distort the order of channels to this
end. As the recognition model does not require sample-wise $\\{S,P\\}$, they
won’t be exposed to the attacker. Figure 4(c) shows the improvement against
recovery if the attacker doesn’t know the specific choice of subsets.

##  4 Experiments

###  4.1 Experimental settings

We compare PartialFace with the unprotected baseline and prior PPFR methods on
three criteria: recognition performance, privacy protection of visual
information, and that against recovery attack. We further study the
computation and cost of PartialFace. We mainly employ an IR-50 [11] trained on
the MS1Mv2 [10] dataset as our FR model, while also using a smaller
combination of IR-18 and the BUPT [40] dataset on some resource-consuming
experiments. We set $(\sigma,s,r,m,n)$=$(10,9,18,6,6)$ and use fixed
$\\{\mathbf{S},\mathbf{P}\\}$, if not else specified. Benchmarks are carried
out on 5 widely used, regular-size datasets, LFW [13], CFP-FP [35], AgeDB
[31], CPLFW [49] and CALFW [50], and 2 large-scale datasets, IJB-B [43] and
IJB-C [23].

###  4.2 Benchmarks on recognition accuracy

Compared methods. We compare PartialFace with 2 unprotected baselines, 4
perturbation-based PPFR methods, and 3 methods base on the frequency domain
that share close relation to us. Results are summarized in Tab. 1. Here, (1)
ArcFace [6] denotes the unprotected SOTA trained directly on RGB images; (2)
ArcFace-FD [45] is the ArcFace trained on the image’s all frequency channels;
(3) PEEP [3] is a differential-privacy-based method with a privacy budget
$\epsilon$=5; (4) Cloak [27] perturbs and compresses its input feature space.
Its accuracy-privacy trade-off parameter is set to 100; (5) InstaHide [14]
mixes up $k$=2 images and performs a distributed encryption; (6) CPGAN [36]
generates compressed protected representation by a joint effort of GAN and
differential privacy; (7) PPFR-FD111The results of PPFR-FD on IJB-B is
unattainable due its non-disclosure of source code. The rest is quoted from
its paper [42]. Please note its experimental condition may have slight
inconsistency with ours. [42] adopts a channel-wise shuffle-and-mix strategy
in the frequency domain; (8) DCTDP [15] perturbs the frequency components by a
noise disturbance mask with learnable privacy budget, where we set
$\epsilon$=1; (9) DuetFace [26] is a two-party framework that employs channel
splitting and attention transfer.

Method | PPFR | LFW | CFP-FP | AgeDB | CPLFW | CALFW | IJB-B | IJB-C
---|---|---|---|---|---|---|---|---
ArcFace [6] | No | 99.77 | 98.30 | 97.88 | 92.77 | 96.05 | 94.13 | 95.60
ArcFace-FD [45] | No | 99.78 | 98.04 | 98.10 | 92.48 | 96.03 | 94.08 | 95.64
PEEP [3] | Yes | 98.41 | 74.47 | 87.47 | 79.58 | 90.06 | 5.82 | 6.02
Cloak [27] | Yes | 98.91 | 87.97 | 92.60 | 83.43 | 92.18 | 33.58 | 33.82
InstaHide [14] | Yes | 96.53 | 83.20 | 79.58 | 81.03 | 86.24 | 61.88 | 69.02
CPGAN [36] | Yes | 98.87 | 94.61 | 96.98 | 90.43 | 94.79 | 92.67 | 94.31
PPFR-FD [42] | Yes | 99.68 | 95.04 | 97.37 | 90.78 | 95.72 | / | 94.10
DCTDP [15] | Yes | 99.77 | 96.97 | 97.72 | 91.37 | 96.05 | 93.29 | 94.43
DuetFace [26] | Yes | 99.82 | 97.79 | 97.93 | 92.35 | 96.10 | 93.66 | 95.30
PartialFace (ours) | Yes | 99.80 | 97.63 | 97.79 | 92.03 | 96.07 | 93.64 | 94.93
Table 1: Benchmarks on recognition accuracy. PartialFace is compared with the
unprotected baselines and PPFR SOTAs.

Performance. Results are reported on LFW, CFP-FP, AgeDB, CPLFW, and CALFW by
accuracy, and on IJB-B and IJB-C by TPR@FPR(1e-4). Table 1 shows PartialFace
achieves close performance to the unprotected baseline, with a small accuracy
gap of $\leq$ 0.8%. In comparison, perturbation-based methods all generalize
unsatisfactorily on large-scale benchmarks. PartialFace outperforms all PPFR
prior arts but DuetFace. While DuetFace achieves slightly higher accuracy, its
protection is inferior to ours as it only covers the inference phase and can
be easily nullified by recovery attacks, later see Secs. 4.4 and 4.5.

###  4.3 Comparison with the vanilla method

We elaborate that randomized PartialFace outperforms the vanilla method that
fixes channels in not only recognition accuracy but also robustness. As
PartialFace employs data augmentation, for a fair comparison regarding the
volume of training data, we consider two experimental settings: the standard
PartialFace and one without augmentation ($r$=1). We also set $n$=1 to rule
out permutation: $P$ is introduced for privacy and is out of our interest
here. There are therefore $m$$\times$$n$=6 combinations of channels, or ranks.
As channels from different ranks carry distinct information that may affect
the performance, we train one vanilla model on each rank and average their
test accuracy, with the range marked in parentheses. Though PartialFace is
able to infer from arbitrary rank, we evaluate its robustness by inferring
from each fixed rank separately. The model is robust if it performs on
different ranks consistently (small range). Results are reported on IJB-B/C by
TPR@FPR(1e-4) in Tab. 2.

Method | IJB-B (range) | IJB-C (range)
---|---|---
ArcFace | 86.83 | 90.35
Vanilla | 78.24 (-14.49/7.36) | 80.73 (-16.37/7.45)
PF w/o aug. | 81.43 (-4.19/2.88) | 82.81 (-4.29/2.79)
PartialFace | 85.77 (-1.95/1.25) | 88.48 (-1.71/1.18)
Table 2: Comparison with the vanilla method by accuracy and robustness.
Experiments are conducted on IR-18 + BUPT. “PF w/o aug.” indicates PartialFace
without augmentation.

Performance. PartialFace achieves an average accuracy gain of 7.53% and 7.75%
on IJB-B/C, respectively, compared to the vanilla. The performance is close to
the unprotected baseline, showing that the privacy-accuracy trade-off of
PartialFace is highly efficient. Even PartialFace without augmentation
outperforms the vanilla for about 3%. We further note: (1) The range shows
PartialFace is more robust than the vanilla under different $S_{i}$. (2)
PartialFace outperforms the vanilla for all individual $S_{i}$. This suggests
that randomness empowers PartialFace with the knowledge of the entirety
instead of that of certain informative ranks.

Figure 6: Visualization of model attention via Grad-CAM. (a) PartialFace (1st
row) compared with vanilla models (2nd row) on each of 6 different ranks. (b)
Integrated attention of all ranks.

Visualization. We visualize the rank-wise attention of PartialFace and the
vanilla via Grad-CAM [34], see Fig. 6(a). The attention of each vanilla model
is restrained to local facial features, which indicates the inadequate
learning of the entirety features. PartialFace generate accurate attention on
the entire face regardless the rank. We integrated the attention across all
ranks in Fig. 6(b). All vanilla models’ attention combined gains attention on
the entire face, which testifies to our “learn-entirety-from-local” theory.
Also to note that PartialFace has similar attention across all ranks. This
allows it to recognize $X$ using arbitrary rank, as they produce aligned
outcomes.

###  4.4 Protection of visual information

We investigate PartialFace’s protection of privacy. We reiterate that the
first privacy goal is to conceal the visual information of face images. We
compare ParitalFace with 3 PPFR SOTAs using the frequency domain: PPFR-FD
[42], DCTDP [15], and DuetFace [26]. These prior arts are closely related to
ours since we all leverage the perceptual difference between humans and models
as means of protection. However, they differ in the processing of frequency
components: Both PPFR-FD and DCTDP remove the component with the highest
energy (the DC component). To obfuscate the remaining components, PPFR-FD
employs mix-and-shuffle and DCTDP applies a noise mechanism. DuetFace is the
most related method in the pruning of low-frequency components. We note in
special that they retain 36, 63, and 54 high-frequency channels, respectively,
while PartialFace retains 9. We now demonstrate the methodological differences
and varied number of channels result in contrasting privacy protection
capacities.

Figure 7: Example face images of (a) unprotected ArcFace, (b-d) compared
SOTAs, and (e) PartialFace, during both training and inference phase.
PartialFace best conceals visual information.

Figure 7 exemplifies face images processed (therefore are supposed to be
protective) by each compared method. As all methods carry out protection in
the frequency domain, we convert images back by padding any removed frequency
components with zero and applying an inverse transform. PartialFace, _e.g_.,
has $X^{\prime}$=$\mathcal{T}^{-1}(\\{\mathbf{0},\mathbf{x}_{h}\\})$. We
visualize the training and inference phases separately, as the compared
methods’ processing on them may vary: During inference, we argue PPFR-FD and
DuetFace (Fig. 7(b)(d)) provide inadequate protection, as one can still
discern the face quite clearly in their processed images. DCTDP effectively
conceals visual information after obfuscating the channels with its proposed
noise mask (Fig. 7(c)). However, its protection during training is impaired,
as it requires access to abundant non-obfuscated components to learn the mask.
DuetFace manifests a similar weakness as it relies on original images to
rectify the model’s attention (Fig. 7(d)).

PartialFace first discards most visual information by pruning
$\mathbf{x}_{l}$. The rest is further partitioned when sampling
$\mathbf{x}_{s}$ from $\mathbf{x}_{h}$. By energy, each $\mathbf{x}_{s}$
carries less than 1% of visual information compared to the original $X$. As a
result, Fig. 7(e) shows PartialFace provides better protection on visual
information than related SOTAs in both phases.

###  4.5 Protection against recovery

We here provide an in-depth analysis of how PartialFace impedes recovery. Our
primary aiming is to show how the removal of model-exploitable information and
the randomness of channels contribute to the defense. Assume the recovery
attacker possesses a collection of face images $\\{X\\}$ and is aware of the
protection mechanism. It can locally generate protected representations
$X^{\prime}$ ($\mathbf{x}_{h}$ in our case) from $X$, then train a malicious
model $g(\cdot)$ with
$\operatorname*{arg\,min}_{\delta}l^{\prime}(g(X^{\prime},\delta),X)$, with
the aiming to inversely fit $X$ from $X^{\prime}$. Having intercepted a
recognition query, the attacker leverages $g(\cdot)$ to recover the concealed
visual information. Upon the knowledge it possesses, a black-box attacker is
one unknowing of necessary parameters ($\\{\mathbf{S},\mathbf{P}\\}$ in our
case), whilst a white-box attacker possesses such knowledge. We also study the
capability of PPFR-FD, DCTDP, and DuetFace against the white-box attacker. We
further investigate an enhanced white-box attacker, who imposes threats
dedicated to the randomness of PartialFace.

Figure 8: Examples of recovered face images of PartialFace under attackers
with varied capabilities: (a) a black-box, (b) a white-box, and (c) an
enhanced white-box attacker. PartialFace effectively prevents recovery as all
images are blurry and hardly identifiable.

Black-box attacker. It is concretized as a malicious third party, uninvolved
in the recognition yet eager to wiretap the transmission. For each attacker,
We employ a full-scale U-Net [32] as the recovery model and train it on BUPT.
Unknowing of $\\{\mathbf{S},\mathbf{P}\\}$, it must produce $X^{\prime}$ based
on its own conjectured $\\{\mathbf{S}^{\prime},\mathbf{P}^{\prime}\\}$, which
is believably inconsistent with that applied to the recognition model. Thus
produced $X^{\prime}$ are invalid samples and training $g(\cdot)$ on them will
nullify the recovery, as shown in Fig. 8(a).

White-box attacker. The candidate combinations $\\{\mathbf{S},\mathbf{P}\\}$
is known by any party participates in the recognition, _e.g_., an honest-but-
curious server. Such an attacker can generate $X^{\prime}$ correctly. However,
receiving a query $\mathbf{x}_{s}$, the attacker is unknowing of its channels’
position and order, since the specific $\\{S,P\\}$ doesn’t come with
$\mathbf{x}_{s}$. The missing information is vital to recovery. Consequently,
the attack is obstructed by $\mathbf{x}_{s}$’s randomness in Fig. 8(b): The
recovered image is blurry and hardly identifiable.

Figure 9: Recovered images from (a-c) compared SOTAs and (d) PartialFace.
PartialFace impedes recovery better than the rest.

We plus compare with PPFR-FD, DCTDP and DuetFace under the white-box settings.
Among them, DuetFace shows almost no resistance to recovery (Fig. 9(c)). PPFR-
FD and DCTDP provide inadequate protection, as images recovered from them
(Fig. 9(a-b)) are blurred yet still clearly identifiable. The protection is
impaired as their processed images retain most high-frequency components, and
the excessive perceivable information is learned by the recovery model. In
comparison, PartialFace (Fig. 9(d)) offers significantly outperformed
protection.

Enhanced white-box attacker. A resource-unbounded attacker may brute-forcibly
break the randomness of PartialFace leveraging a more sophisticated attack: it
trains a series of attack models, one for each candidate $\\{S,P\\}$.
Receiving $\mathbf{x}_{s}$, the attacker feeds it into every model to try
every combination of $\\{S,P\\}$, until one produces the best recovery. The
vague face images in Fig. 8(c) imply the attempted recovery is unsuccessful,
even if the attacker finds the correct $\\{S,P\\}$. This is attributed to the
reduction of model-exploitable channels.

Method | SSIM (↓) | PSNR (↓) | Accuracy (↓)
---|---|---|---
PPFR-FD | 0.713 | 15.66 | 83.73
DCTDP | 0.687 | 15.42 | 79.60
DuetFace | 0.866 | 19.88 | 96.52
PartialFace | 0.591 | 13.70 | 65.35
Table 3: Quantitative analyses of the recovery quality. Lower SSIM, PSNR and
accuracy suggest better protection. Here, we remind the accuracy of
verification is lower bounded by 50.00.

Quantitative comparison. To quantitatively assess the recovery quality, we
measure the average structural similarity index (SSIM) and peak signal-to-
noise ratio (PSNR) of the recovered images. Additionally, to study if the
attacker can exploit its outcome for recognition proposes, we feed the images
into a pre-trained model and measure verification accuracy. Lower SSIM, PSNR,
and recognition accuracy suggest deficient recovery, thus indicating a higher
level of protection. The results in Tab. 3 suggest PartialFace outperforms its
competitors in all three evaluated metrics.

###  4.6 Protection against recovery from templates

Figure 10: Recovered images from extracted identity templates, that are blurry
and can hardly be inferred by humans or models.

By the principle of FR, all $\mathbf{x}_{s}$ of face image $X$ are later
extracted to very similar identity templates, so the recognition on them
produces aligned results. Here, one would arise concern that recovery can also
be carried out on the extracted templates, which theoretically contain the
faces’ full identity information, to realize better inversion of images. We
demonstrate that such a proposed attack is also ineffective to PartialFace.

Although there could be dedicated attacks on templates, in practice, high-
quality recovery from them is difficult as templates are far more compact
representations than images and channel subsets. In Fig. 10, the recovered
images are highly blurred, and can hardly be inferred by humans or models,
hence won’t impose an effective threat.

###  4.7 Ablation study

PartialFace is parameterized by $(\sigma,s,r,m,n)$. Among them, we note
$\sigma$ is chosen according to channel energy, and $s$ is determined by
$(\sigma,m)$. Table 4 analyzes the choice of the remaining parameters. Results
show augmentation ($r$) enhances the model’s performance, at the cost of
taking longer time to train. $m$ affects the performance mainly by its
influence on $s$, as larger $m$ leading to fewer channels in each
$\mathbf{x}_{s}$. We introduce $P$ solely for privacy purposes. Results on $n$
indicate a trade-off between accuracy and privacy. Generally, PartialFace is
robust to the choice of parameters.

Settings | LFW CFP-FP AgeDB CPLFW CALFW
---|---
ArcFace |  99.38 |  92.31 |  94.65 |  89.41 |  94.78
$r$ | 36 |  99.51 |  91.27 |  94.44 |  88.93 |  94.81
6 |  98.69 |  86.79 |  90.65 |  84.50 |  92.67
$m$ | 3 |  99.42 |  91.68 |  93.95 |  88.70 |  94.69
9 |  99.32 |  90.74 |  93.56 |  87.30 |  94.36
$n$ | 1 |  99.37 |  91.55 |  94.25 |  88.73 |  94.70
12 |  99.35 |  89.21 |  93.17 |  87.19 |  94.26
Default |  99.38 |  91.20 |  93.72 |  88.11 |  94.42
Table 4: Ablation study on combinations of hyperparameters. Experiments are
carried out on IR-18 + BUPT. When changing one of them, we keep the rest to
default values, _i.e_., $(r,m,n)$=$(18,6,6)$.

###  4.8 Complexity and compatibility

Settings | LFW | CFP-FP | AgeDB | CPLFW
---|---|---|---|---
CosFace | 99.53 | 92.89 | 95.15 | 89.52
PartialFace | 99.35 | 89.54 | 93.97 | 87.90
Table 5: Compatibility of PartialFace. FR models are trained using CosFace on
IR-18 + BUPT. Combining PartialFace with CosFace also demonstrates high
utility, compared to its baseline.

Though PartialFace is proposed by studying the model’s behavior, privacy
protection is solely realized by processing the face images. The decoupling
with model architecture and training tactics benefits resources and
compatibility.

PartialFace is resource-efficient. Compared to the unprotected baseline,
PartialFace doesn’t increment model size as they share identical model
architectures. Training does take more ($r$) time and storage due to
augmentation, while we argue it is acceptable for the service provider. The
crucial inference time remains the same as the baseline, as DCT and sampling
$\mathbf{x}_{s}$ only increase negligible computation costs.

PartialFace is well compatible. The decoupling of preprocessing and training
also allows PartialFace to serve as a convenient plug-in: PartialFace can be
integrated with SOTA FR methods to enjoy enhanced privacy protection.
Specifically, we demonstrate the recognition accuracy on CosFace [38], a major
competitor of ArcFace, in Tab. 5. Combining PartialFace with CosFace also
results in high utility, as compared to its baseline.

##  5 Conclusion

This paper presents an in-depth study of the privacy protection of face
images. Based on the observations on model perception and training behavior,
we present two methodological advances, pruning low-frequency components and
using randomly selected channels, to address the privacy goal of concealing
visual information and impeding recovery. We distill our findings into a novel
privacy-preserving face recognition method, PartialFace. Extensive experiments
demonstrate that PartialFace effectively balances privacy protection goals and
recognition accuracy.

## References

  * [1] Michel Abdalla, Florian Bourse, Angelo De Caro, and David Pointcheval.  Simple functional encryption schemes for inner products.  In Jonathan Katz, editor, Public-Key Cryptography - PKC 2015 - 18th IACR International Conference on Practice and Theory in Public-Key Cryptography, Gaithersburg, MD, USA, March 30 - April 1, 2015, Proceedings, volume 9020 of Lecture Notes in Computer Science, pages 733–751. Springer, 2015.
  * [2] Insaf Adjabi, Abdeldjalil Ouahabi, Amir Benzaoui, and Abdelmalik Taleb-Ahmed.  Past, present, and future of face recognition: A review.  Electronics, 9(8), 2020.
  * [3] Mahawaga Arachchige Pathum Chamikara, Peter Bertók, Ibrahim Khalil, Dongxi Liu, and Seyit Camtepe.  Privacy preserving face recognition utilizing differential privacy.  Comput. Secur., 97:101951, 2020.
  * [4] Thee Chanyaswad, J. Morris Chang, Prateek Mittal, and Sun-Yuan Kung.  Discriminant-component eigenfaces for privacy-preserving face recognition.  In Francesco A. N. Palmieri, Aurelio Uncini, Kostas I. Diamantaras, and Jan Larsen, editors, 26th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2016, Vietri sul Mare, Salerno, Italy, September 13-16, 2016, pages 1–6. IEEE, 2016.
  * [5] Wen-Hsiung Chen, C. Harrison Smith, and S. C. Fralick.  A fast computational algorithm for the discrete cosine transform.  IEEE Trans. Commun., 25(9):1004–1009, 1977.
  * [6] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.  Arcface: Additive angular margin loss for deep face recognition.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 4690–4699. Computer Vision Foundation / IEEE, 2019.
  * [7] Ovgu Ozturk Ergun.  Privacy preserving face recognition in encrypted domain.  In 2014 IEEE Asia Pacific Conference on Circuits and Systems, APCCAS 2014, Ishigaki, Japan, November 17-20, 2014, pages 643–646. IEEE, 2014.
  * [8] Zekeriya Erkin, Martin Franz, Jorge Guajardo, Stefan Katzenbeisser, Inald Lagendijk, and Tomas Toft.  Privacy-preserving face recognition.  In Ian Goldberg and Mikhail J. Atallah, editors, Privacy Enhancing Technologies, 9th International Symposium, PETS 2009, Seattle, WA, USA, August 5-7, 2009. Proceedings, volume 5672 of Lecture Notes in Computer Science, pages 235–253. Springer, 2009.
  * [9] Lionel Gueguen, Alex Sergeev, Rosanne Liu, and Jason Yosinski.  Faster neural networks straight from JPEG.  In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings. OpenReview.net, 2018.
  * [10] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao.  Ms-celeb-1m: A dataset and benchmark for large-scale face recognition.  In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling, editors, Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III, volume 9907 of Lecture Notes in Computer Science, pages 87–102. Springer, 2016.
  * [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770–778. IEEE Computer Society, 2016.
  * [12] Katsuhiro Honda, Masahiro Omori, Seiki Ubukata, and Akira Notsu.  A study on fuzzy clustering-based k-anonymization for privacy preserving crowd movement analysis with face recognition.  In Mario Köppen, Bing Xue, Hideyuki Takagi, Ajith Abraham, Azah Kamilah Muda, and Kun Ma, editors, 7th International Conference of Soft Computing and Pattern Recognition, SoCPaR 2015, Fukuoka, Japan, November 13-15, 2015, pages 37–41. IEEE, 2015.
  * [13] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller.  Labeled faces in the wild: A database for studying face recognition in unconstrained environments.  Technical Report 07-49, University of Massachusetts, Amherst, October 2007\.
  * [14] Yangsibo Huang, Zhao Song, Kai Li, and Sanjeev Arora.  Instahide: Instance-hiding schemes for private distributed learning.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 4507–4518. PMLR, 2020.
  * [15] Jiazhen Ji, Huan Wang, Yuge Huang, Jiaxiang Wu, Xingkun Xu, Shouhong Ding, Shengchuan Zhang, Liujuan Cao, and Rongrong Ji.  Privacy-preserving face recognition with learnable privacy budgets in frequency domain.  In Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XII, volume 13672 of Lecture Notes in Computer Science, pages 475–491. Springer, 2022.
  * [16] Tom A. M. Kevenaar, Geert Jan Schrijen, Michiel van der Veen, Anton H. M. Akkermans, and Fei Zuo.  Face recognition with renewable and privacy preserving binary templates.  In Proceedings of the Fourth IEEE Workshop on Automatic Identification Advanced Technologies (AutoID 2005), 16-18 October 2005, Buffalo, NY, USA, pages 21–26. IEEE Computer Society, 2005.
  * [17] Xiaoyu Kou, Ziling Zhang, Yuelei Zhang, and Linlin Li.  Efficient and privacy-preserving distributed face recognition scheme via facenet.  In ACM TURC 2021: ACM Turing Award Celebration Conference \- Hefei, China, 30 July 2021 - 1 August 2021, pages 110–115. ACM, 2021.
  * [18] Yuancheng Li, Yimeng Wang, and Daoxing Li.  Privacy-preserving lightweight face recognition.  Neurocomputing, 363:212–222, 2019.
  * [19] Fan Liu, Delong Chen, Fei Wang, Zewen Li, and Feng Xu.  Deep learning based single sample face recognition: a survey.  Artif. Intell. Rev., 56(3):2723–2748, 2023.
  * [20] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song.  Sphereface: Deep hypersphere embedding for face recognition.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 6738–6746. IEEE Computer Society, 2017.
  * [21] Zhuo Ma, Yang Liu, Ximeng Liu, Jianfeng Ma, and Kui Ren.  Lightweight privacy-preserving ensemble classification for face recognition.  IEEE Internet Things J., 6(3):5778–5790, 2019.
  * [22] Yunlong Mao, Jinghao Feng, Fengyuan Xu, and Sheng Zhong.  A privacy-preserving deep learning approach for face recognition with edge computing.  In Irfan Ahmad and Swaminathan Sundararaman, editors, USENIX Workshop on Hot Topics in Edge Computing, HotEdge 2018, Boston, MA, July 10, 2018. USENIX Association, 2018.
  * [23] Brianna Maze, Jocelyn C. Adams, James A. Duncan, Nathan D. Kalka, Tim Miller, Charles Otto, Anil K. Jain, W. Tyler Niggel, Janet Anderson, Jordan Cheney, and Patrick Grother.  IARPA janus benchmark - C: face dataset and protocol.  In 2018 International Conference on Biometrics, ICB 2018, Gold Coast, Australia, February 20-23, 2018, pages 158–165. IEEE, 2018.
  * [24] Blaz Meden, Peter Rot, Philipp Terhörst, Naser Damer, Arjan Kuijper, Walter J. Scheirer, Arun Ross, Peter Peer, and Vitomir Struc.  Privacy-enhancing face biometrics: A comprehensive survey.  IEEE Trans. Inf. Forensics Secur., 16:4147–4183, 2021.
  * [25] Pietro Melzi, Christian Rathgeb, Rubén Tolosana, Rubén Vera-Rodríguez, and Christoph Busch.  An overview of privacy-enhancing technologies in biometric recognition.  CoRR, abs/2206.10465, 2022.
  * [26] Yuxi Mi, Yuge Huang, Jiazhen Ji, Hongquan Liu, Xingkun Xu, Shouhong Ding, and Shuigeng Zhou.  Duetface: Collaborative privacy-preserving face recognition via channel splitting in the frequency domain.  In João Magalhães, Alberto Del Bimbo, Shin’ichi Satoh, Nicu Sebe, Xavier Alameda-Pineda, Qin Jin, Vincent Oria, and Laura Toni, editors, MM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022, pages 6755–6764. ACM, 2022.
  * [27] Fatemehsadat Mireshghallah, Mohammadkazem Taram, Ali Jalali, Ahmed Taha Elthakeb, Dean M. Tullsen, and Hadi Esmaeilzadeh.  Not all features are equal: Discovering essential features for preserving prediction privacy.  In Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia, editors, WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pages 669–680. ACM / IW3C2, 2021\.
  * [28] Vahid Mirjalili, Sebastian Raschka, Anoop M. Namboodiri, and Arun Ross.  Semi-adversarial networks: Convolutional autoencoders for imparting privacy to face images.  In 2018 International Conference on Biometrics, ICB 2018, Gold Coast, Australia, February 20-23, 2018, pages 82–89. IEEE, 2018.
  * [29] Vahid Mirjalili, Sebastian Raschka, and Arun Ross.  Gender privacy: An ensemble of semi adversarial networks for confounding arbitrary gender classifiers.  In 9th IEEE International Conference on Biometrics Theory, Applications and Systems, BTAS 2018, Redondo Beach, CA, USA, October 22-25, 2018, pages 1–10. IEEE, 2018.
  * [30] Vahid Mirjalili and Arun Ross.  Soft biometric privacy: Retaining biometric utility of face images while perturbing gender.  In 2017 IEEE International Joint Conference on Biometrics, IJCB 2017, Denver, CO, USA, October 1-4, 2017, pages 564–573. IEEE, 2017\.
  * [31] Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou.  Agedb: The first manually collected, in-the-wild age database.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA, July 21-26, 2017, pages 1997–2005. IEEE Computer Society, 2017.
  * [32] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In Nassir Navab, Joachim Hornegger, William M. Wells III, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 - 18th International Conference Munich, Germany, October 5 - 9, 2015, Proceedings, Part III, volume 9351 of Lecture Notes in Computer Science, pages 234–241. Springer, 2015.
  * [33] Ahmad-Reza Sadeghi, Thomas Schneider, and Immo Wehrenberg.  Efficient privacy-preserving face recognition.  In Dong Hoon Lee and Seokhie Hong, editors, Information, Security and Cryptology - ICISC 2009, 12th International Conference, Seoul, Korea, December 2-4, 2009, Revised Selected Papers, volume 5984 of Lecture Notes in Computer Science, pages 229–244. Springer, 2009.
  * [34] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  Int. J. Comput. Vis., 128(2):336–359, 2020.
  * [35] Soumyadip Sengupta, Jun-Cheng Chen, Carlos Domingo Castillo, Vishal M. Patel, Rama Chellappa, and David W. Jacobs.  Frontal to profile face verification in the wild.  In 2016 IEEE Winter Conference on Applications of Computer Vision, WACV 2016, Lake Placid, NY, USA, March 7-10, 2016, pages 1–9. IEEE Computer Society, 2016.
  * [36] Bo-Wei Tseng and Pei-Yuan Wu.  Compressive privacy generative adversarial network.  IEEE Trans. Inf. Forensics Secur., 15:2499–2513, 2020.
  * [37] Gregory K. Wallace.  The JPEG still picture compression standard.  Commun. ACM, 34(4):30–44, 1991.
  * [38] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu.  Cosface: Large margin cosine loss for deep face recognition.  In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 5265–5274. Computer Vision Foundation / IEEE Computer Society, 2018.
  * [39] Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P. Xing.  High-frequency component helps explain the generalization of convolutional neural networks.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 8681–8691. Computer Vision Foundation / IEEE, 2020.
  * [40] Mei Wang and Weihong Deng.  Mitigate bias in face recognition using skewness-aware reinforcement learning.  arXiv, 2019.
  * [41] Mei Wang and Weihong Deng.  Deep face recognition: A survey.  Neurocomputing, 429:215–244, 2021.
  * [42] Yinggui Wang, Jian Liu, Man Luo, Le Yang, and Li Wang.  Privacy-preserving face recognition in the frequency domain.  In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022, pages 2558–2566. AAAI Press, 2022.
  * [43] Cameron Whitelam, Emma Taborsky, Austin Blanton, Brianna Maze, Jocelyn C. Adams, Tim Miller, Nathan D. Kalka, Anil K. Jain, James A. Duncan, Kristen Allen, Jordan Cheney, and Patrick Grother.  IARPA janus benchmark-b face dataset.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017, Honolulu, HI, USA, July 21-26, 2017, pages 592–600. IEEE Computer Society, 2017.
  * [44] Can Xiang, Chunming Tang, Yunlu Cai, and Qiuxia Xu.  Privacy-preserving face recognition with outsourced computation.  Soft Comput., 20(9):3735–3744, 2016.
  * [45] Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren.  Learning in the frequency domain.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 1737–1746. Computer Vision Foundation / IEEE, 2020.
  * [46] Kai Xu, Zhikang Zhang, and Fengbo Ren.  LAPRAN: A scalable laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part X, volume 11214 of Lecture Notes in Computer Science, pages 491–507. Springer, 2018.
  * [47] Xiaopeng Yang, Hui Zhu, Rongxing Lu, Ximeng Liu, and Hui Li.  Efficient and privacy-preserving online face recognition over encrypted outsourced data.  In IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), iThings/GreenCom/CPSCom/SmartData 2018, Halifax, NS, Canada, July 30 - August 3, 2018, pages 366–373. IEEE, 2018.
  * [48] Chen Zhang, Xiongwei Hu, Yu Xie, Maoguo Gong, and Bin Yu.  A privacy-preserving multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition.  Frontiers Neurorobotics, 13:112, 2019.
  * [49] T. Zheng and W. Deng.  Cross-pose lfw: A database for studying cross-pose face recognition in unconstrained environments.  Technical Report 18-01, Beijing University of Posts and Telecommunications, February 2018.
  * [50] Tianyue Zheng, Weihong Deng, and Jiani Hu.  Cross-age LFW: A database for studying cross-age face recognition in unconstrained environments.  arXiv, 2017.
  * [51] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin.  Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting.  In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 27268–27286. PMLR, 2022.