∎

22institutetext: Center for Nonlinear Science & Engineering, School of
Electrical & Electronics Engineering, SASTRA Deemed University, Thanjavur,
Tamil Nadu 613 401, India 22email<EMAIL_ADDRESS>33institutetext:
Department of Physics, School of Advanced Sciences, Vellore Institute of
Technology, Vellore– 632014, India44institutetext: 44email:
<EMAIL_ADDRESS>55institutetext: Center for Nonlinear Science &
Engineering, School of Electrical & Electronics Engineering, SASTRA Deemed
University, Thanjavur, Tamil Nadu 613 401, India66institutetext: 66email:
<EMAIL_ADDRESS>
∗Corresponding Author.

# Characterizing nonbilocal correlation: A geometric perspective

R. Muthuganesan∗ S. Balakrishnan  V. K. Chandrasekar

(Received: date / Accepted: date)

###### Abstract

Exploiting the notion of measurement-induced nonlocality [Phys. Rev. Lett.
106, 120401 (2011)], we introduce a new measure to quantify the nonbilocal
correlation. We establish a simple relation between the nonlocal and
nonbilocal measures for the arbitrary pure input states. Considering the mixed
states as inputs, we derive two upper bounds of affinity-based nonbilocal
measure. Finally, we have studied the nonbilocality of a different
combinations of input states.

###### Keywords:

Entanglement Quantum correlation Nonlocality Nonbilocality Projective
measurements Nonlocality

##  1 Introduction

Nonlocality, the most fundamental and intriguing feature of a composite
quantum system, is a direct consequence of the superposition principle, which
creates a distinction between the behavior of the quantum and classical
systems Nielsen2010 . Nonlocality is referred as ”spooky-action-at-a-distance”
by Einstein Einstein  and Schrodinger scho . Understanding this perplexing
phenomenon in a simplest composite system, namely the bipartite system, is a
fundamental issue and of practical importance in developing quantum
technologies. In the realm of Bell’s nonlocality Bell , the presence of
nonlocal character or entanglement is witnessed by the violation of Bell
inequality. In the earliest quantum information theory era, it is believed
that the entanglement is the complete manifestation of nonlocality of a
quantum system.

Since the inception of Werner’s work Werner  and quantum discord Ollivier2001
, it has been debated whether the entanglement can manifest the nonlocal
aspects of a quantum system or not. Buscemi showed that entanglement is
considered as the complete manifestation of nonlocality. In other words, ”all
entangled quantum states are nonlocal” Buscemi . On the other hand, Werner
constructed a mixed state family, which admits the local hidden variable model
even the state is entangled Werner . Further, it is shown that the presence of
noise or mixedness is responsible for the destroying nonlocal correlation
between local constituents of the composite system, and hence some of the
mixed entangled states behave locally Almeida . Ollivier and Zurek introduced
a measure called, quantum discord to quantify the quantum correlations beyond
entanglement. It can capture the correlation between the marginal states which
cannot grasp by the entanglement Ollivier2001 . Recently, a new variant of
nonlocality, called measurement-induced nonlocality (MIN) is introduced
Luo2011 . MIN is based on the fact that local disturbance due to locally
invariant von Neumann projective measurements on the marginal state can
influence globally. MIN is quite different from the entanglement and violation
of the Bell inequalities, more importantly it goes beyond entanglement.

In quantum entanglement swapping experiment, the independence between the
multi-sources induces the nonlocal behavior of probability distributions and
is called nonbilocal correlations Branciard2010 ; Branciard2012 . This kind of
correlation is demonstrated and captured using nonlinear inequalities and one
important class of these inequalities is the so-called binary-input-and-output
bilocality inequality which is known as the bilocality inequality
Branciard2010 ; Branciard2012 . In recent times, the considerable progress has
been made in this context Fritz2012 ; Fritz2016 ; Wood2015 ; Henson ;
Chaves2015 ; Tavakoli2014 ; Tavakoli2016a ; Tavakoli2016b ; Chaves2016 ;
Rosset . Gisin et al. have shown that pair of entangled states can violate the
bilocality inequality, implying that tensorizing states may possess nonlocal
correlation Gisin2017 .

When two bipartite states with vanishing correlation are combined, it is
always interesting to check that the tensorizing state possess any nonlocal
advantage or not. It is shown recently that the combining two quantum systems
exhibit better quantum advantages than the individual system. This is known as
superactivation of nonlocality, symbolized as $0+0>0$ and it cannot occur in
the classical world. The superactivation of nonlocality provides an answer for
”can the state $\rho\otimes\rho$ be nonlocal if the $\rho$ is local”.
Recently, Palazuelos explained the superactivation of quantum nonlocality in
the sense of violating certain Bell inequalities with an entangled bound state
Palazuelos . The same study is carried out in the context of tensor networks
Cavalcanti2011 ; Cavalcanti2012 . Further, the superactivation was also
considered for arbitrary entangled states by allowing local preprocessing on
the tensor product of different quantum states $(\rho\otimes\rho^{\prime})$
Masanes  and symbolized as $1+0>1$. With our best knowledge, the
quantification of this nonbilocal correlation is limited in the literature.

To quantify the nonbilocal correlation, we extend the notion of bipartite
measurement-induced nonlocality (MIN) to a bilocal quantum system. This paper
proposes a new version of the non-bilocal correlation measure. The relation
between the nonlocal and non-bilocal correlation measure is established, and
it is shown that non-bilocality is always greater than the nonlocal
correlation. Further, the upper bound of the bilocal correlation measure is
obtained for the arbitrary mixed input states. To validate the properties of
nonbilocal measure, we study the proposed quantity for a few examples.

This paper is organized as follows. In Sec. 2, we review the notion of
measurement-induced nonlocality and provide the definition of affinity based
MIN. In Sec. 3, we introduce a nonbilocal measure based on the affinity
induced metric and we derive the analytical formula of the measure when the
input states are pure. Considering the input states are arbitrary mixed
states, the upper bounds of bilocal correlation are presented in Sec. 4. In
Sec. 5, the proposed measure studied for a few examples. Finally, in Sec. 6 we
present the conclusion.

##  2 Measurement-induced nonlocality

To capture the bipartite quantum correlation beyond entanglement, Luo and Fu
introduced a new measure of quantum correlation called measurement-induced
nonlocality (MIN) using locally invariant projective measurement. It is
originally defined as maximal square of Hilbert-Schmidt norm of difference of
pre- and post- measurement states and is defined as Luo2011

$N(\rho)=\leavevmode\nobreak\
^{\text{max}}_{\Pi^{a}}\|\rho-\Pi^{a}(\rho)\|^{2},$ (1)

where the maximization is taken over the von Neumann projective measurements
on subsystem $a$,
$\Pi^{a}(\rho)=\sum_{k}(\Pi^{a}_{k}\otimes\mathds{1}^{b})\rho(\Pi^{a}_{k}\otimes\mathds{1}^{b})$,
and $\Pi^{a}=\\{\Pi^{a}_{k}\\}=\\{|k\rangle\langle k|\\}$ being the projective
measurements on the subsystem $a$, which do not change the marginal state
$\rho^{a}$ locally i.e., $\Pi^{a}(\rho^{a})=\rho^{a}$. Here
$\|\mathcal{O}\|=\sqrt{\text{Tr}\mathcal{O}^{\dagger}\mathcal{O}}$ is the
Hilbert-Schmidt norm of operator $\mathcal{O}$. The dual of this quantity is
geometric discord (GD) of the given state $\rho$ and it is defined as Dakic ;
Luo2010pra

$D(\rho)=\leavevmode\nobreak\
^{\text{min}}_{\Pi^{a}}\|\rho-\Pi^{a}(\rho)\|^{2}.$ (2)

In general, if $\rho^{a}$ is nondegenerate, then the optimization is not
required and the above measures are equal. This is more general than the
conventionally mentioned quantum nonlocality related to Bell’s version of
nonlocality. Apart from the quantification of bipartite quantum correlation,
this quantity provides a novel classification scheme for bipartite states, and
is a useful resource quite different from entanglement. In particular, MIN is
a more secured resource for cryptographic communication. Due to its geometric
nature, MIN is easy to compute and realizable. Nevertheless, both the MIN and
geometric discord are not useful quantifiers of quantum correlation due to the
local ancilla problem, which is pointed out by Piani Piani2012 . A natural way
to circumvent this issue is to modify the definition of MIN using any
contractive distance measure. One such distance measure between the states
$\rho$ and $\sigma$ is defined as

$\displaystyle d_{\mathcal{A}}(\rho,\sigma)=1-\mathcal{A}(\rho,\sigma),$ (3)

where
$\mathcal{A}(\rho,\sigma)=\text{Tr}\left(\sqrt{\rho}\sqrt{\sigma}\right)$ is
the affinity between the states. Analogous to fidelity Jozsa1994 , the
affinity is also a measure of closeness between the states $\rho$ and $\sigma$
Luo2004 ; Bhattacharyya  and shares all the properties of fidelity Jozsa1994 .
Also, affinity is useful in the quantification of nonclassical correlations
Muthu2019 ; Muthu2020  and quantum coherence Muthu2021 . It is worth
mentioning that the affinity between the states is realizable using the
quantum circuit Ekert . Due to its realization, affinity based measures may
have a good impact in the research of quantum information.

The affinity-based MIN is defined as Muthu2020

$\displaystyle N^{\text{MIN}}_{\mathcal{A}}(\rho)=\leavevmode\nobreak\
^{\text{{max}}}_{\Pi^{a}}\leavevmode\nobreak\
d_{\mathcal{A}}(\rho,\Pi^{a}(\rho))=1-\leavevmode\nobreak\
^{\text{{min}}}_{\Pi^{a}}\leavevmode\nobreak\
\text{Tr}\left(\sqrt{\rho}\Pi^{a}(\sqrt{\rho})\right).$ (4)

Here also the optimization is taken over von Neumann projective measurements.
It is worth reiterating that the $N_{\mathcal{A}}(\rho)$ fixes the local
ancilla problem using the multiplicative property of affinity. Hence,
$N^{\text{MIN}}_{\mathcal{A}}(\rho)$ is a faithful quantifier of quantum
correlation or quantumness of the system. Further, we have shown that affinity
based MIN is closely related to local quantum uncertainty Girolami2013LQU  and
interferometric power of a quantum state Girolami2014 .

##  3 Nonbilocality Measure

In this section, we introduce the notion and measure of nonbilocality using
affinity. We consider two input states $\rho^{ab}$ (shared between $a$ and
$b$) and $\rho^{cd}$ (shared between $c$ and $d$) in the separable composite
finite-dimensional Hilbert space
$\mathcal{H}=\mathcal{H}_{a}\otimes\mathcal{H}_{b}\otimes\mathcal{H}_{c}\otimes\mathcal{H}_{d}$.
Then nonbilocal measure is defined as

$\displaystyle N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=\leavevmode\nobreak\
^{\text{\leavevmode\nobreak\ max}}_{\Pi^{bc}}\leavevmode\nobreak\
d_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd},\Pi^{bc}(\rho_{ab}\otimes\rho_{cd}))$
$\displaystyle=1-\leavevmode\nobreak\
^{\text{min}}_{\Pi^{bc}}\mathcal{A}(\rho_{ab}\otimes\rho_{cd},\Pi^{bc}(\rho_{ab}\otimes\rho_{cd})),$
(5)

where maximization/minimization is taken over the von Neumann projective
measurement $\Pi^{bc}=\\{\Pi^{bc}_{k}\\}$, which leaves
$\rho^{bc}=\text{Tr}_{ad}(\rho_{ab}\otimes\rho_{cd})$ invariant locally,
$d_{\mathcal{A}}(\cdot,\cdot)$ is affinity-induced metric and the post-
measurement state is
$\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}})=\sum_{k,l}(\mathds{1}^{a}\otimes\Pi^{bc}_{kl}\otimes\mathds{1}^{d})\sqrt{\rho_{ab}\otimes\rho_{cd}}(\mathds{1}^{a}\otimes\Pi^{bc}_{kl}\otimes\mathds{1}^{d})$.
Here $\rho^{b}=\sum_{i}\lambda_{i}|i_{b}\rangle\langle i_{b}|$ and
$\rho^{c}=\sum_{j}\lambda_{j}|j_{c}\rangle\langle j_{c}|$ are the marginal
states of $\rho_{bc}$, if any one of the states is nondegenerate, then the
measurement takes the form $\Pi^{bc}=\\{\Pi^{b}\otimes\Pi^{c}\\}$.

It is worth mentioning that above defined quantity satisfies all the necessary
axioms of a quantum correlation measure. Like MIN, the nonbilocal measure is
also a useful resource for cryptographic communication. Next, we demonstrate
some interesting properties of nonbilocality measure given by Eq. (5).

  1. (i)

$N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})$ is nonnegative,

  2. (ii)

$N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=0$ for any product states
$\rho_{ab}=\rho^{a}\otimes\rho^{b}$ and $\rho_{cd}=\rho^{c}\otimes\rho^{d}$.
Further, the nonbilocal measure also vanishes for classical-quantum state
$\rho_{ab}=\sum_{i}\rho^{a}_{i}\otimes p_{i}|i_{b}\rangle\langle i_{b}|$ and
$\rho_{cd}=\sum_{j}q_{j}|j_{c}\rangle\langle j_{c}|\otimes\rho^{d}_{j}$.

  3. (iii)

$N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})$ is locally unitary invariant in
the sense that

$\displaystyle N_{\mathcal{A}}((U_{ab}\otimes
U_{cd})\rho_{ab}\otimes\rho_{cd}(U_{ab}\otimes
U_{cd})^{\dagger})=N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd}),$ (6)

where $U_{ab}=U_{a}\otimes U_{b}$ and $U_{cd}=U_{c}\otimes U_{d}$ are the
local unitary operators.

  4. (iv)

$N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})$ is positive, at least any one of
the input states is entangled.

  5. (v)

Although $N_{\mathcal{A}}(\rho_{ab})=N_{\mathcal{A}}(\rho_{cd})=0$,
nevertheless $N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})>0$.

Since the properties (i) - (v) are inherited directly from the properties of
affinity-based MIN, we omit the proving process. Next, we establish a simple
relation between the nonbilocal measure and MIN (nonlocal).

Theorem 1: The nonbilocal measure and affinity-based MIN are connected as

$\displaystyle N_{\mathcal{A}}(\rho_{ba}\otimes\rho_{ab})\geq
N_{\mathcal{A}}^{\text{MIN}}(\rho).$ (7)

Proof: To prove this, first we recall the definition of affinity-based
nonbilocal measure,

$\displaystyle N_{\mathcal{A}}(\rho_{ba}\otimes\rho_{ab})=$
$\leavevmode\nobreak\ {}^{\text{\leavevmode\nobreak\
max}}_{\Pi^{aa}}\leavevmode\nobreak\
d_{\mathcal{A}}(\rho_{ba}\otimes\rho_{ab},\Pi^{aa}(\rho_{ba}\otimes\rho_{ab})),$
$\displaystyle=$ $\displaystyle 1-\leavevmode\nobreak\
^{\text{min}}_{\Pi^{aa}}\mathcal{A}(\rho_{ba}\otimes\rho_{ab},\Pi^{aa}(\rho_{ba}\otimes\rho_{ab})),$
$\displaystyle\geq$ $\leavevmode\nobreak\ {}^{\text{\leavevmode\nobreak\
max}}_{\Pi^{a}}\leavevmode\nobreak\
d_{\mathcal{A}}(\rho_{ba}\otimes\rho_{ab},(\Pi^{a}\otimes\Pi^{a})(\rho_{ba}\otimes\rho_{ab})),$
$\displaystyle=$ $\displaystyle 1-\leavevmode\nobreak\
^{\text{\leavevmode\nobreak\
min}}_{\Pi^{a}}\text{Tr}(\sqrt{\rho_{ab}}\otimes\sqrt{\rho_{ab}})(\Pi^{a}(\sqrt{\rho_{ab}})\otimes\Pi^{a}(\sqrt{\rho_{ab}})),$
$\displaystyle=$ $\displaystyle 1-\leavevmode\nobreak\
^{\text{\leavevmode\nobreak\
min}}_{\Pi^{a}}\text{Tr}(\sqrt{\rho_{ab}}\Pi^{a}(\sqrt{\rho_{ab}}))^{2},$
$\displaystyle\geq$ $\displaystyle 1-\leavevmode\nobreak\
^{\text{\leavevmode\nobreak\
min}}_{\Pi^{a}}\text{Tr}(\sqrt{\rho_{ab}}\Pi^{a}(\sqrt{\rho_{ab}})),$
$\displaystyle=$ $\displaystyle N_{\mathcal{A}}^{MIN}(\rho),$

where the first inequality follows from the fact that $\Pi^{a}\otimes\Pi^{a}$
is not necessarily optimal and the second inequality is due to the square of
the affinity which is always equal to or less than the affinity. Hence the
theorem is proved. The above theorem provides a closer connection between the
nonbilocal and nonlocal measures and implying that the nonbilocal measure is
always greater than MIN. Next, we compute the nonbilocal measure for pure
input states.

Theorem 2: Let $|\Psi_{ab}\rangle=\sum_{i}\sqrt{s_{i}}|i_{a}i_{b}\rangle$ and
$|\Psi_{ab}\rangle=\sum_{j}\sqrt{r_{j}}|j_{c}j_{d}\rangle$ are the pure input
states, then

$\displaystyle
N_{\mathcal{A}}(|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle)=1-\sum_{i,j}s_{i}^{4}r_{j}^{4},$
(8)

where $s_{i}$ and $r_{j}$ are Schmidt coefficients of $|\Psi_{ab}\rangle$ and
$|\Psi_{cd}\rangle$ respectively.

The proof of the theorem is given in the appendix.

##  4 Nonbilocal correlation for mixed states

To compute nonbilocality of any arbitrary mixed input states, first, we recall
some basic notation in the operator Hilbert space. Let
$\mathcal{L}(\mathcal{H}_{\alpha})$ be the Hilbert space of linear operators
on $\mathcal{H}_{\alpha}(\alpha=a,b,c,d)$ with the inner product $\langle
X|Y\rangle=\text{Tr}X^{\dagger}Y$. An arbitrary $m\times n$ dimensional
bipartite state can be written as

$\displaystyle\sqrt{\rho_{ab}}=\sum_{i,j}\lambda^{ab}_{ij}X_{i}\otimes Y_{j},$

where $\\{X_{i}:i=0,1,\cdots,m^{2}-1\\}$ and
$\\{Y_{j}:j=0,1,\cdots,n^{2}-1\\}$ are the orthonormal operator bases of the
subsystem $a$ and $b$ respectively and satisfies the relation
$\text{Tr}X_{k}X_{l}=\delta_{kl}$, and
$\lambda^{ab}_{ij}=\text{Tr}(\sqrt{\rho_{ab}}\leavevmode\nobreak\ X_{i}\otimes
Y_{j})$ are real elements of matrix $\Lambda_{ab}$. Similarly, one can define
the orthonormal operator bases as
$\\{P_{k}:k=0,1,\cdots,u^{2}-1\\}\leavevmode\nobreak\
\text{and\leavevmode\nobreak\ }\\{Q_{l}:l=0,1,\cdots,v^{2}-1\\}$ for another
input state $\rho_{cd}$ with $u$ and $v$ are the dimensions of the marginal
systems $c$ and $d$ respectively. Then the state $\rho_{cd}$ is defined as

$\displaystyle\sqrt{\rho_{cd}}=\sum_{k,l}\lambda^{cd}_{kl}P_{k}\otimes Q_{l},$

where $\lambda^{cd}_{kl}=\text{Tr}(\sqrt{\rho_{cd}}\leavevmode\nobreak\
P_{k}\otimes Q_{l})$ are the matrix elements of matrix $\Lambda_{cd}$. Then
the bilocal state is written as

$\displaystyle\sqrt{\rho_{ab}\otimes\rho_{cd}}=\sum_{i,j}\sum_{k,l}\lambda^{ab}_{ij}\lambda^{cd}_{kl}X_{i}\otimes
Y_{j}\otimes P_{k}\otimes Q_{l}.$ (9)

Theorem 3: For any arbitrary bilocal input states represented in Eq. (9), the
upper bound of nonbilocal measure is

$\displaystyle N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})\leq
1-\sum_{s=1}^{nu}\mu_{s},$ (10)

where $\mu_{s}$ are the eigenvalues of the matrix
$\Lambda_{ab,cd}\Lambda^{t}_{ab,cd}$ arranged in increasing order and $t$
denotes the transpose of a matrix.

To prove the theorem, first, we compute the post-measured state. Here, the
measurement operators are
$\Pi^{bc}=\\{\mathds{1}^{a}\otimes\Pi^{bc}_{h}\otimes\mathds{1}^{d}\\}$ and we
have

$\displaystyle\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}})=$
$\displaystyle\sum_{h}\sum_{ijkl}\lambda^{ab}_{ij}\lambda^{cd}_{kl}X_{i}\otimes\Pi^{bc}_{h}(Y_{j}\otimes
P_{k})\Pi^{bc}_{h}\otimes Q_{l}$ $\displaystyle=$
$\displaystyle\sum_{h}\sum_{ijj^{\prime}kk^{\prime}l}\lambda^{ab}_{ij}\lambda^{cd}_{kl}\gamma_{hjk}\gamma_{hj^{\prime}k^{\prime}}X_{i}\otimes
Y_{j^{\prime}}\otimes P_{k^{\prime}}\otimes Q_{l},$ (11)

where $\gamma_{hjk}=\text{Tr}\Pi^{bc}_{h}(Y_{j}\otimes P_{k})$ are the
elements of matrix $\Gamma$. After the straight forward calculation the
affinity between pre- and post-measurement states is

$\displaystyle\mathcal{A}(\sqrt{\rho_{ab}\otimes\rho_{cd}},\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}}))=$
$\displaystyle\sum_{h}\sum_{ijj^{\prime}kk^{\prime}l}\leavevmode\nobreak\
\lambda^{ab}_{ij}\lambda^{cd}_{kl}\gamma_{hjk}\gamma_{hj^{\prime}k^{\prime}}\lambda^{ab}_{ij^{\prime}}\lambda^{cd}_{k^{\prime}l}$
$\displaystyle=$
$\displaystyle\Gamma\Lambda_{ab,cd}\Lambda^{t}_{ab,cd}\Gamma^{t},$ (12)

where $\Gamma$ is $nu\times n^{2}u^{2}$ dimensional matrix. Then,

$\displaystyle
N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=1-\leavevmode\nobreak\
^{\text{min}}_{\leavevmode\nobreak\ \Gamma}\leavevmode\nobreak\
\Gamma\Lambda_{ab,cd}\Lambda^{t}_{ab,cd}\Gamma^{t}\leq
1-\sum_{s=1}^{nk}\mu_{s},$ (13)

where $\mu_{s}$ are the eigenvalues of the matrix
$\Lambda_{ab,cd}\Lambda^{t}_{ab,cd}$ listed in increasing order. Hence the
theorem is proved.

Theorem 4: If the marginal state $\rho^{b}$ is nondegenerate, the nonbilocal
measure $N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})$ due to the measurement
$\Pi^{bc}$ has the upper bound as

$\displaystyle N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})\leq
1-{\mathcal{A}}(\rho_{ab},\Pi^{b}(\rho_{ab})\times(\sum_{\tau=1}^{u}\mu_{\tau}),$
(14)

where $\mu_{\tau}$ are the eigenvalues of matrix
$\Lambda_{cd}\Lambda^{t}_{cd}$ arranged in an increasing order and
$\mathcal{A}(\rho_{ab},\Pi^{b}(\rho_{ab}))$ is the affinity between the state
$\rho_{ab}$ and post-measurement state $\Pi^{b}(\sqrt{\rho_{ab}})$.

To show this, we recall that if the marginal state is nondegenerate, then the
optimization is not required, Here the state $\rho^{b}$ is nondegenerate and
the optimization is taken over $\Pi^{c}$ alone. The measurement operator is
defined as
$\Pi^{b}\otimes\Pi^{c}=\\{\Pi^{b}_{j}\otimes\Pi^{c}_{k}\\}=\\{|j_{b}\rangle\langle
j_{b}|\otimes\Pi^{c}_{k}\\}$. The nonbilocality measure is

$\displaystyle N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=$ $\displaystyle
1-\leavevmode\nobreak\ ^{\text{min}}_{\Pi^{bc}}\leavevmode\nobreak\
{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd},\Pi^{bc}(\rho_{ab}\otimes\rho_{cd}))$
$\displaystyle=$ $\displaystyle 1-\leavevmode\nobreak\
^{\text{min}}_{\Pi^{bc}}\leavevmode\nobreak\
\text{Tr}\sqrt{\rho_{ab}\otimes\rho_{cd}}\cdot\Pi^{bc}\sqrt{(\rho_{ab}\otimes\rho_{cd})}$
$\displaystyle=$ $\displaystyle 1-\leavevmode\nobreak\
^{\text{min}}_{\Pi^{c}}\leavevmode\nobreak\
\text{Tr}\sqrt{\rho_{ab}\otimes\rho_{cd}}\cdot(\Pi^{b}\otimes\Pi^{c})(\sqrt{\rho_{ab}\otimes\rho_{cd}})$
$\displaystyle=$ $\displaystyle
1-\text{Tr}\sqrt{\rho_{ab}}\Pi^{b}(\sqrt{\rho_{ab}})\cdot\leavevmode\nobreak\
^{\text{min}}_{\Pi^{c}}\leavevmode\nobreak\ \text{Tr}\leavevmode\nobreak\
\sqrt{\rho_{cd}}\Pi^{c}(\sqrt{\rho_{cd}}),$ (15)

where the quantity $\text{Tr}\sqrt{\rho_{ab}}\Pi^{b}(\sqrt{\rho_{ab}})$ is the
affinity between the state $\rho_{ab}$ and post-measurement state
$\Pi^{b}(\sqrt{\rho_{ab}})$. Following the optimization procedure given in
Muthu2020 , we write the second quantity as

$\leavevmode\nobreak\ {}^{\text{min}}_{\leavevmode\nobreak\
\Pi^{c}}\leavevmode\nobreak\ \text{Tr}\leavevmode\nobreak\
\sqrt{\rho_{cd}}\Pi^{c}(\sqrt{\rho_{cd}})=\leavevmode\nobreak\
^{\text{min}}_{\leavevmode\nobreak\ C}\leavevmode\nobreak\
\text{Tr}\leavevmode\nobreak\ C\Lambda_{cd}\Lambda_{cd}^{t}C^{t}.$ (16)

Then, we have

$\displaystyle N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=$ $\displaystyle
1-\mathcal{A}(\rho_{ab},\Pi^{b}(\sqrt{\rho_{ab}}))\leavevmode\nobreak\
^{\text{min}}_{\leavevmode\nobreak\ C}\leavevmode\nobreak\
\text{Tr}\leavevmode\nobreak\ C\Lambda_{cd}\Lambda_{cd}^{t}C^{t}$
$\displaystyle\leq$ $\displaystyle
1-\mathcal{A}(\rho_{ab},\Pi^{b}(\rho_{ab}))\times\sum_{\tau=1}^{u}\mu_{\tau},$
(17)

where $\mu_{\tau}$ are the eigenvalues of matrix
$\Lambda_{cd}\Lambda^{t}_{cd}$ arranged in an increasing order.

Theorem 5: If the marginal states $\rho^{b}$ and $\rho^{c}$ are nondegenerate
and the dimension of the $\rho^{c}$ is $u=2$, then the closed formula of
nonbilocal measure $N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})$ is expressed
as

$\displaystyle
N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=1-\mathcal{A}(\rho_{ab},\Pi^{b}(\rho_{ab}))\times\|\bm{\lambda}_{cd}\|+\lambda_{min},$
(18)

where
$\bm{\lambda}_{cd}=(\lambda^{cd}_{00},\lambda^{cd}_{01},\cdots,\lambda^{cd}_{0(v^{2}-1)})$
is a $v^{2}$ dimensional row vector, and
$\Lambda=((\lambda^{cd})_{kl})_{k=1,2,3;\leavevmode\nobreak\ l=0,1,\cdots
v^{2}-1}$ is a $3\times v^{2}$ dimensional matrix and $\lambda_{min}$ is the
least eigenvalues of $\Lambda\Lambda^{t}$

Using the completeness relation $\sum_{k}\Pi_{k}^{c}=\mathds{1}^{c}$, we show
that $c_{0k}=-c_{1k}(k=1,2,3)$, then $\sum_{k}c_{0k}^{2}=1$. Then the vector
$\mathbf{c}=\sqrt{2}(c_{0k}\leavevmode\nobreak\ c_{0k}\leavevmode\nobreak\
c_{0k})$ with $|\mathbf{c}|=1$. Now,

$\displaystyle C=\frac{1}{\sqrt{2}}\begin{pmatrix}1&\mathbf{c}\\\
1&-\mathbf{c}\end{pmatrix},$ (19)

and

$\displaystyle\Lambda_{cd}=\begin{pmatrix}\bm{\lambda}_{cd}\\\
\Lambda\end{pmatrix},$ (20)

where
$\bm{\lambda}_{cd}=(\lambda^{cd}_{00},\lambda^{cd}_{01},\cdots,\lambda^{cd}_{0(v^{2}-1)})$
is a $v^{2}$ dimensional row vector, and
$\Lambda=((\lambda^{cd})_{kl})_{k=1,2,3;\leavevmode\nobreak\
l=0,1,\cdots,v^{2}-1}$ is a $3\times v^{2}$ dimensional matrix. We have

${}^{\text{min}}_{\leavevmode\nobreak\ C}\leavevmode\nobreak\
\text{Tr}C\Lambda_{cd}\Lambda_{cd}^{t}C^{t}=\|\bm{\lambda}_{cd}\|+\lambda_{\text{min}},$
(21)

where $\lambda_{min}$ is the least eigenvalue of matrix
$\mathbf{c}RR^{t}\mathbf{c}^{t}$. Then we have computed the closed formula of
nonbilocal measure

$\displaystyle N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})=\leavevmode\nobreak\
1-\mathcal{A}(\rho_{ab},\Pi^{b}(\sqrt{\rho_{ab}}))\times(\|\bm{\lambda}_{cd}\|+\lambda_{\text{min}})$
(22)

to complete the proof.

##  5 Illustrations

In this section, we compute the affinity-based measurement-induced
nonbilocality for some input states.

Example 1: Let $|\Psi_{ab}\rangle=|00\rangle$ and
$|\Psi_{cd}\rangle=(|00\rangle+|11\rangle)/\sqrt{2}$ are the two input states.
According to Theorem. 2, the nonbilocal measure is

$\displaystyle
N_{\mathcal{A}}(|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle)=1-\sum_{i,j}s_{i}^{4}r_{j}^{4}.$
(23)

The Schmidt coefficients for $|\Psi_{ab}\rangle$ are 0 and 1. Similarly,
$|\Psi_{ab}\rangle$ has the Schmidt coefficients $1/\sqrt{2}$ and
$1/\sqrt{2}$. Then,
$N_{\mathcal{A}}(|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle)=0.5$. The above
example validate the property (iv) of the
$N_{\mathcal{A}}(\rho_{ab}\otimes\rho_{cd})$.

Example 2: The input state is
$|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle=(|00\rangle+|11\rangle)/\sqrt{2}\otimes(|00\rangle+|11\rangle)/\sqrt{2}$.
Then,

$\displaystyle
N_{\mathcal{A}}(|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle)=1-4\times\frac{1}{4}\times\frac{1}{4}=\frac{3}{4}.$
(24)

Example 3: Next, we consider the combination of maximally entangled state as

$\displaystyle\rho_{ab}=\frac{1}{3}(|\Psi^{+}\rangle\langle\Psi^{+}|+|\Psi^{-}\rangle\langle\Psi^{-}|+|\Phi^{+}\rangle\langle\Phi^{+}|),$

where
$|\Psi^{\pm}\rangle=(|00\rangle\pm|11\rangle)/\sqrt{2}\leavevmode\nobreak\
\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\ \leavevmode\nobreak\
|\Phi^{\pm}\rangle=(|01\rangle\pm|10\rangle)/\sqrt{2}$. In straight forward,
the square root of $\rho_{ab}$ is

$\displaystyle\sqrt{\rho_{ab}}=\frac{1}{\sqrt{3}}(|\Psi^{+}\rangle\langle\Psi^{+}|+|\Psi^{-}\rangle\langle\Psi^{-}|+|\Phi^{+}\rangle\langle\Phi^{+}|).$

The Bloch form of $\sqrt{\rho_{ab}}$ can be written as

$\displaystyle\sqrt{\rho_{ab}}=\frac{1}{4}\left[\sqrt{3}(\mathds{1}\otimes\mathds{1})+\frac{1}{\sqrt{3}}(\sigma_{x}\otimes\sigma_{x})+\frac{1}{\sqrt{3}}(\sigma_{y}\otimes\sigma_{y})+\frac{1}{\sqrt{3}}(\sigma_{z}\otimes\sigma_{z})\right],$

where $\sigma_{i}$ are Pauli’s spin matrices. Here, the eigenprojective
measurements are
$\Pi^{bc}=\\{|\Psi^{+}\rangle\langle\Psi^{+}|,|\Psi^{-}\rangle\langle\Psi^{-}|,\\\
|\Phi^{+}\rangle\langle\Phi^{+}|,|\Phi^{-}\rangle\langle\Phi^{-}|\\}$. Then,
the nonbilocal measure is computed as

$\displaystyle N(\rho_{ba}\otimes\rho_{ab})=1-\leavevmode\nobreak\
^{\text{min}}_{\leavevmode\nobreak\ \Gamma}\leavevmode\nobreak\
\Gamma\Lambda_{ab,cd}\Lambda^{t}_{ab,cd}\Gamma^{t}\geq
1-\frac{7}{12}=\frac{5}{12}.$ (25)

The affinity-based MIN is
$N_{\mathcal{A}}^{\text{MIN}}(\rho)=1/6<N(\rho_{ba}\otimes\rho_{ab})$,
resulting the consequence of Theorem. 1.

Example 4: In this case, the input states are

$\displaystyle\rho_{ab}=\rho_{cd}=\frac{1}{2}|0\rangle\langle
0|\otimes|0\rangle\langle 0|+\frac{1}{2}|1\rangle\langle
1|\otimes|1\rangle\langle 1|$ (26)

shared between $a$ and $b$. The MIN of the above state is zero. We obtain

$\displaystyle\Lambda_{ab}=\Lambda_{cd}=\begin{pmatrix}\frac{1}{\sqrt{2}}&0&0&0\\\
0&0&0&0\\\ 0&0&0&0\\\ 0&0&0&\frac{1}{\sqrt{2}}\end{pmatrix}.$ (27)

Here we choose the optimal von Neumann measurements as
$\\{\Pi^{bc}\\}=\\{H^{\otimes 2}|ij\rangle\langle ij|H^{\otimes 2}\\}$ with
$i,j=0,1$ and $H$ is

$\displaystyle H=\frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\\ 1&-1\end{pmatrix},$
(28)

the popular single-qubit Hadamard gate. After the straight forward
calculation, we have

$\displaystyle N(\rho_{ba}\otimes\rho_{ab})=1-\leavevmode\nobreak\
^{\text{min}}_{\leavevmode\nobreak\ \Gamma}\leavevmode\nobreak\
\Gamma\Lambda_{ab,cd}\Lambda^{t}_{ab,cd}\Gamma^{t}=\frac{3}{4}.$ (29)

##  6 Conclusion

Nonlocality is often related to the entanglement or violation of Bell’s
inequality. In this article, we have employed the nonlocality in different
notion, namely measurement-induced nonlocality (MIN). Extending the definition
of affinity-based MIN, we have introduced a new variant of quantifier to
quantify the nonlocal correlation contained in tensorizing a local state with
itself called nonbilocal correlation and also demonstrated the computational
properties of the proposed measure. A closer connection between the affinity-
based measurement-induced nonlcality and measurement-induced nonbilcality is
also derived. An analytical formula of nonbilocal measure is derived when the
input states are pure. Moreover, two upper bounds of nonbilocal measure are
also obtained for mixed input state. For illustration, we have studied the
nonbilocality for different input states.

Like MIN, the nonbilocality measure may also useful resource for remote state
preparation, quantum dense coding and cryptographic communication and hence,
the proposed nonbilocal measure provides more insight into quantum information
theory.

## Appendix

Let $|\Psi_{ab}\rangle=\sum_{i}s_{i}|i_{a}i_{b}\rangle$ and
$|\Psi_{cd}\rangle=\sum_{j}r_{j}|j_{c}j_{d}\rangle$ are the two pure input
states with $s_{i}$ and $r_{j}$ are the respective Schmidt coefficients of
input states.

Noting that

$\displaystyle\rho_{ab}\otimes\rho_{cd}$
$\displaystyle=|\Psi_{ab}\rangle\langle\Psi_{ab}|\otimes|\Psi_{cd}\rangle\langle\Psi_{cd}|$
(30)
$\displaystyle=\sum_{ii^{{}^{\prime}}jj^{{}^{\prime}}}s_{i}s_{i^{{}^{\prime}}}r_{j}r_{j^{{}^{\prime}}}|i_{a}\rangle\langle
i^{{}^{\prime}}_{a}|\otimes|i_{b}\rangle\langle
i^{{}^{\prime}}_{b}|\otimes|j_{c}\rangle\langle
j^{{}^{\prime}}_{c}|\otimes|j_{d}\rangle\langle j^{{}^{\prime}}_{d}|.$

One can compute the marginal state

$\displaystyle\rho^{bc}$
$\displaystyle=\text{Tr}_{ad}(|\Psi_{ab}\rangle\langle\Psi_{ab}|\otimes|\Psi_{cd}\rangle\langle\Psi_{cd}|)=\sum_{ij}s_{i}^{2}r_{j}^{2}|i_{b}j_{c}\rangle\langle
i_{b}j_{c}|.$ (31)

For pure state $\sqrt{\rho}=\rho$. From the above equation, the post-
measurement state $\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}})$ can be
rewritten as

$\displaystyle\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}})=\Pi^{bc}(\rho_{ab}\otimes\rho_{cd})$
$\displaystyle=$
$\displaystyle\sum_{hk}(\mathds{1}^{a}\otimes\Pi_{hk}^{bc}\otimes\mathds{1}^{d})(|\Psi_{ab}\rangle\langle\Psi_{ab}|\otimes|\Psi_{cd}\rangle\langle\Psi_{cd}|)(\mathds{1}^{a}\otimes\Pi_{hk}^{BC}\otimes\mathds{1}^{d})$
$\displaystyle=$
$\displaystyle\sum_{hk}(\mathds{1}^{a}\otimes\Pi_{hk}^{bc}\otimes\mathds{1}^{d})(\sum_{ii^{{}^{\prime}}jj^{{}^{\prime}}}s_{i}s_{i^{{}^{\prime}}}r_{j}r_{j^{{}^{\prime}}}|i_{a}\rangle\langle
i^{{}^{\prime}}_{a}|\otimes|i_{b}\rangle\langle
i^{{}^{\prime}}_{b}|\otimes|j_{c}\rangle\langle
j^{{}^{\prime}}_{c}|\otimes|j_{d}\rangle\langle
j^{{}^{\prime}}_{d}|)(\mathds{1}^{a}\otimes\Pi_{hk}^{bc}\otimes\mathds{1}^{d})$
$\displaystyle=$
$\displaystyle\sum_{hk}\sum_{ii^{{}^{\prime}}jj^{{}^{\prime}}}s_{i}s_{i^{{}^{\prime}}}r_{j}r_{j^{{}^{\prime}}}|i_{a}\rangle\langle
i^{{}^{\prime}}_{a}|\otimes\Pi^{bc}_{hk}|i_{b}j_{c}\rangle\langle
i^{{}^{\prime}}_{b}j^{{}^{\prime}}_{c}|\Pi^{bc}_{hk}\otimes|j_{d}\rangle\langle
j^{{}^{\prime}}_{d}|$ $\displaystyle=$
$\displaystyle\sum_{hk}\sum_{ii^{{}^{\prime}}jj^{{}^{\prime}}}s_{i}s_{i^{{}^{\prime}}}r_{j}r_{j^{{}^{\prime}}}|i_{a}\rangle\langle
i^{{}^{\prime}}_{a}|\otimes U|h_{b}k_{c}\rangle\langle
h_{b}k_{c}|U^{{\dagger}}|i_{b}j_{c}\rangle\langle
i^{{}^{\prime}}_{b}j^{{}^{\prime}}_{c}|U|h_{b}k_{c}\rangle\langle
h_{b}k_{c}|U^{{\dagger}}\otimes|j_{d}\rangle\langle j^{{}^{\prime}}_{d}|.$

Here the von Neumann projective measurement is expressed as

$\displaystyle\Pi^{bc}=\\{\Pi^{bc}_{hk}\equiv U|h_{b}k_{c}\rangle\langle
h_{b}k_{c}|U^{{\dagger}}\\}$ (32)

Consequently,

$\displaystyle\sqrt{\rho_{ab}\otimes\rho_{cd}}\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}})$
$\displaystyle=$
$\displaystyle(\sum_{ii^{{}^{\prime}}jj^{{}^{\prime}}}s_{i}s_{i^{{}^{\prime}}}r_{j}r_{j^{{}^{\prime}}}|i_{a}\rangle\langle
i^{{}^{\prime}}_{a}|\otimes|i_{b}\rangle\langle
i^{{}^{\prime}}_{b}|\otimes|j_{c}\rangle\langle
j^{{}^{\prime}}_{c}|\otimes|j_{d}\rangle\langle
j^{{}^{\prime}}_{d}|)(\sum_{hk}\sum_{uu^{{}^{\prime}}vv^{{}^{\prime}}}s_{u}s_{u^{{}^{\prime}}}r_{v}r_{v^{{}^{\prime}}}$
$\displaystyle\langle h_{b}k_{c}|U^{{\dagger}}|u_{b}v_{c}\rangle\langle
u^{{}^{\prime}}_{b}v^{{}^{\prime}}_{c}|U|h_{b}k_{c}\rangle|u_{a}\rangle\langle
u^{{}^{\prime}}_{a}|\otimes U|h_{b}k_{c}\rangle\langle
h_{b}f_{c}|U^{{\dagger}}\otimes|v_{d}\rangle\langle v^{{}^{\prime}}_{d}|)$
$\displaystyle=$
$\displaystyle\sum_{ii^{{}^{\prime}}jj^{{}^{\prime}}}\sum_{hk}\sum_{uu^{{}^{\prime}}vv^{{}^{\prime}}}s_{i}s_{i^{{}^{\prime}}}r_{j}r_{j^{{}^{\prime}}}s_{u}s_{u^{{}^{\prime}}}r_{v}r_{v^{{}^{\prime}}}\langle
e_{B}f_{C}|U^{{\dagger}}|u_{B}v_{C}\rangle\langle
u^{{}^{\prime}}_{b}v^{{}^{\prime}}_{c}|U|h_{b}k_{f}\rangle|i_{a}\rangle$
$\displaystyle\langle i^{{}^{\prime}}_{a}|u_{a}\rangle\langle
u^{{}^{\prime}}_{a}|\otimes|i_{b}j_{c}\rangle\langle
i^{{}^{\prime}}_{b}j^{{}^{\prime}}_{c}|U|h_{b}k_{c}\rangle\langle
h_{b}k_{c}|U^{{\dagger}}\otimes|j_{d}\rangle\langle
j^{{}^{\prime}}_{d}|v_{d}\rangle\langle v^{{}^{\prime}}_{d}|.$

Then the affinity between the pre- and post-measurement state is computed as

$\displaystyle\mathcal{A}(\rho_{ab}\otimes\rho_{cd},\Pi^{bc}(\rho_{ab}\otimes\rho_{cd}))=$
$\displaystyle\text{Tr}\sqrt{\rho_{ab}\otimes\rho_{cd}}\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}})$
$\displaystyle=$
$\displaystyle\sum_{iujvhk}s_{i}^{2}s_{u}^{2}r_{j}^{2}r_{v}^{2}\langle
h_{b}k_{c}|U^{{\dagger}}|u_{b}v_{c}\rangle\langle
i_{b}j_{c}|U|h_{b}k_{c}\rangle\langle u_{b}v_{c}|U|h_{b}k_{c}\rangle\langle
h_{b}k_{c}|U^{{\dagger}}|i_{b}j_{c}\rangle$ $\displaystyle=$
$\displaystyle\sum_{hk}(\langle
h_{b}k_{c}|U^{{\dagger}}\rho_{bc}U|h_{b}k_{c}\rangle)^{2}.$

The nonbilocal measure for pure state is

$\displaystyle N_{\mathcal{A}}(|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle)=$
$\displaystyle\max_{\Pi^{bc}}d_{\mathcal{A}}(\sqrt{\rho_{ab}\otimes\rho_{cd}},\Pi^{bc}(\sqrt{\rho_{ab}\otimes\rho_{cd}}))$
$\displaystyle=$ $\displaystyle
1-\min_{\Pi^{bc}}\mathcal{A}(\rho_{ab}\otimes\rho_{cd},\Pi^{bc}(\rho_{ab}\otimes\rho_{cd}))$
$\displaystyle=$ $\displaystyle 1-\min_{\Pi^{bc}}\sum_{hk}(\langle
h_{b}k_{c}|U^{{\dagger}}\rho^{bc}U|h_{b}k_{c}\rangle)^{2},$

where the optimization is over all von Neumann measurements given in Eq. (32),
leaving the marginal state $\rho^{bc}$ invariant. That is,

$\rho^{bc}=\sum_{hk}\langle
h_{b}k_{c}|U^{{\dagger}}\rho^{bc}U|h_{b}k_{c}\rangle
U|h_{b}k_{c}\rangle\langle h_{b}k_{c}|U^{{\dagger}}$

is a spectral decomposition of $\rho^{bc}$ since $\\{U|h_{b}k_{c}\rangle\\}$
is an orthonormal base. Comparing the above equation with Eq. (31), we
obtained

$\displaystyle
N_{\mathcal{A}}(|\Psi_{ab}\rangle\otimes|\Psi_{cd}\rangle)=1-\sum_{i,j}s_{i}^{4}r_{j}^{4}.$
(33)

Hence the theorem is proved.

## Acknowledgment

This work was financially supported by the Council of Scientific and
Industrial Research (CSIR), Government of India, under Grant No.
03(1444)/18/EMR-II.

## References

  * (1) M. Nielsen and I. Chuang.: Quantum Computation and Quantum Information, Cam-bridge University Press, Cambridge (2010).
  * (2) A. Einstein, B. Podolsky and N. Rosen, Phys. Rev. 47, 777 (1935).
  * (3) E. Schrodinger, Proc. Cambridge Philos. Soc. 31, 555 (1935).
  * (4) J. S. Bell, Physics 1, 195 (1964).
  * (5) R. F. Werner, Phys. Rev. A 40, 4277 (1989).
  * (6) H. Ollivier and W.H. Zurek, Phys. Rev. Lett. 88, 017901 (2001).
  * (7) F.Buscemi, Phys. Rev. Lett. 108, 200401 (2012).
  * (8) M. L. Almeida, S. Pironio, J. Barrett, G. Toth and A. Acin, Phys. Rev. Lett. 99, 040403 (2007).
  * (9) S. Luo and S. Fu, Phys. Rev. Lett. 106, 120401 (2011).
  * (10) C. Branciard, N. Gisin and S. Pironio, Phys. Rev. Lett. 104, 170401 (2010).
  * (11) C. Branciard, D. Rosset, N. Gisin and S. Pironio, Phys. Rev. A 85, 032119 (2012).
  * (12) T. Fritz, New J. Phys. 14, 103001 (2012).
  * (13) T. Fritz, Commum. Math. Phys. 341, 391-434 (2016).
  * (14) C. J. Wood and R. W. Spekkens, New J. Phys. 17, 033002 (2015).
  * (15) J. Henson, R. Lal and M. F. Pusey, New J. Phys. 16, 113043 (2014).
  * (16) R. Chaves, J. B. Brask and N. Brunner, Phys. Rev. Lett. 115, 110501 (2015).
  * (17) A. Tavakoli, P. Skrzypczyk, D. Cavalcanti and A. Acín, Phys. Rev. A 90, 062109 (2014).
  * (18) A. Tavakoli, J. Phys. A: Math. Theor. 49, 145304 (2016).
  * (19) A. Tavakoli, Phys. Rev. A 93, 030101 (2016).
  * (20) R. Chaves, Phys. Rev. Lett. 116, 010402 (2016).
  * (21) D. Rosset, C. Branciard, T. J. Barnea, G. Putz, N. Brunner, and N. Gisin, Phys. Rev. Lett. 116, 010403 (2016).
  * (22) N. Gisin, Q. X. Mei, A. Tavakoli, M. O. Renou and N. Brunner, Phys. Rev. A 96, 020304 (2017).
  * (23) C. Palazuelos, Phys. Rev. Lett. 109, 190401 (2012).
  * (24) D. Cavalcanti, M. L. Almeida, V. Scarani and A. Acin, Nature Commun. 2, 184 (2011).
  * (25) D. Cavalcanti, R. Rabelo, and V. Scarani, Phys. Rev. Lett. 108, 040402 (2012).
  * (26) Ll. Masanes, Y.-C. Liang, and A. C. Doherty,Phys. Rev.Lett. 100, 090403 (2008).
  * (27) B. Dakic, V. Vedral and C.Brukner, Phys. Rev. Lett. 105, 190502 (2010).
  * (28) S. Luo and S. Fu, Phys. Rev. A 82, 034302 (2010).
  * (29) M. Piani, Phys. Rev. A 86, 034101 (2012).
  * (30) R. Jozsa, J. Mod. Opt. 41, 2315 (1994).
  * (31) S. Luo and Q. Zhang, Phys. Rev. A 69, 032106 (2004).
  * (32) A. Bhattacharyya, Bulletin of the Calcutta Mathematical Society 35, 99 (1943).
  * (33) A. K. Ekert, C.M. Alves, D. K. L. Oi, M. Horodecki, P. Horodecki and L. C. Kwek, 88, 217901 (2002)
  * (34) R. Muthuganesan and V. K. Chandrasekar, Quantum Inf Process 18, 223 (2019).
  * (35) R. Muthuganesan and V.K. Chandrasekar, Commun. Theor. Phys. 72 075103 (2020).
  * (36) R. Muthuganesan, V.K. Chandrasekar and R. Sankaranarayanan, Phys. Lett. A 394, 127205 (2021).
  * (37) D. Girolami, T. Tufarelli and G. Adesso, Phys. Rev. Lett. 110, 240402 (2013).
  * (38) D. Girolami, A.M. Souza, V. Giovannetti, T. Tufarelli, J.G. Filgueiras, Phys. Rev. Lett. 112 (2014) 210401.


# Identifying Spurious Correlations using Counterfactual Alignment

Joseph Paul Cohen*
Stanford University
<EMAIL_ADDRESS>Louis Blankemeier
Stanford University  Akshay Chaudhari
Stanford University

###### Abstract

Models driven by spurious correlations often yield poor generalization
performance. We propose the counterfactual alignment method to detect and
explore spurious correlations of black box classifiers. Counterfactual images
generated with respect to one classifier can be input into other classifiers
to see if they also induce changes in the outputs of these classifiers. The
relationship between these responses can be quantified and used to identify
specific instances where a spurious correlation exists as well as compute
aggregate statistics over a dataset. Our work demonstrates the ability to
detect spurious correlations in face attribute classifiers. This is validated
by observing intuitive trends in a face attribute classifier as well as
fabricating spurious correlations and detecting their presence, both visually
and quantitatively. Further, utilizing the CF alignment method, we demonstrate
that we can rectify spurious correlations identified in classifiers.

00footnotetext: 1Work done prior to joining Amazon

##  1 Introduction

Challenges related to neural network generalization and fairness often arise
due to covariate shift [19] and shortcut learning [10]. Shortcut learning can
lead to models making decisions based on factors not aligned with expectations
of the model creator. These powerful models (vision models in this work)
leverage a wide array of features and relationships, which may inadvertently
incorporate undesired, spurious correlations. These spurious relationships may
stem from sample bias (e.g., predicting cows when cows are observed on grass
but not on a beach) or may be inherent to the class definition (e.g.,
predicting cows when an animal has four legs) [6].

Our primary goal is to ensure that models make predictions without relying on
spurious correlates, a concept coined as “right for the right reasons” [25].
This means that the features employed for prediction align with expectations
of the model creator, rather than features which are spuriously correlated
with the target label. Models driven by such spurious correlations often yield
poor generalization performance [6, 10]. Consequently, it is crucial to
develop methods for uncovering and correcting these unwanted correlations.

Our objective is to understand black-box classifiers, such as pre-trained
models, as this is a common use case encountered by practitioners. In this
analysis, we utilize counterfactual (CF) images, synthetic images simulating a
change in the class label of an image [35]. These synthetic images have
features modified such that the prediction of the classifier changes. We can
then view the synthetic images to understand the reasons that a prediction was
made.

Specifically, we are interested in CF images that are directly generated using
the gradients of a classifier [8, 12, 3]. Generating CF images with respect to
a classifier is rooted in similar logic to that of crafting adversarial
examples. However, the key distinction lies in the constraint that CF images
remain within the data manifold of plausible images. The latent space of an
autoencoder provides such data manifold. This approach enables us to study the
specific features used by a given classifier on a particular input. By keeping
the classifier and autoencoder independent, different classifiers can be
analysed using a singular fixed autoencoder as a reference point.

The task of interpreting these CF images presents new challenges of scale.
Spurious correlations may only exist in a handful of samples where rare
features occur together. Locating these samples requires investigating the
features used for each prediction, which can be automated using the approaches
we present in this study.

The approach we take is to generate CF images for the positive predictions of
a classifier over an entire dataset and study the resulting CF images using a
collection of different classifiers, which we refer to as downstream
classifiers, to observe what outputs are impacted. If the CF images generated
for one classifier also change the predictions of other classifiers, we can
conclude there is shared feature usage. This shared feature usage can prompt
us to investigate unexpected relationships. The relative change metric (Eq. 3)
can be used to identify spurious correlations which may occur for a specific
example or to identify trends over an entire dataset. We study face attribute
classification, recognizing its advantage in allowing visual verification by
readers.

Figure 1: Overview of the alignment methodology. An image is encoded,
reconstructed, and then processed by a classifier. The counterfactual is
generated by subtracting the gradient of the classifier output w.r.t. the
latent representation. The resulting representation is reconstructed back into
an image. The reconstructed images are processed with multiple classifiers and
the classifier outputs can be plotted side by side to study their alignment.
The base model value can be used as the x-axis to more easily compare it to
the predictions of another classifier. The output changes can be quantified
and compared using relative change.

Overall, given the problem setting of spurious correlations, the contributions
of this work include:

  * •

We propose CF alignment to reason about the feature relationships between
classifiers. This approach allows for both aggregate quantification and
targeted querying of models to identify instances where incorrect reasoning
leads to a prediction.

  * •

Our work demonstrates the ability to detect spurious correlations in face
attribute classifiers. This is validated by observing intuitive trends in a
face attribute classifier as well as inducing spurious correlations and then
detecting their presence, both visually and quantitatively.

  * •

Utilizing the CF alignment method, we demonstrate that we can rectify spurious
correlations identified in classifiers. Experimental results show a bias
reduction in 10 out of 12 classifiers.

##  2 Related Work

Counterfactual (CF) generation can be done in a variety of ways [35].
Classifier specific approaches generally perturb a latent space representation
guided by a classifier. Some methods use the gradient of a classifier to guide
movement in the latent space by computing the gradient directly [8] or
defining a loss that is optimized [12, 3]. Another classifier based approach
is to train a model that predicts, using training examples, where to walk in
the latent space to change the classifier’s prediction [28]. This has the
benefit of working with classifiers that are not differentiable. Another
approach is conditional generation which doesn’t use a classifier and instead
generates images based on a conditioning variable. This conditioning can be a
label [18, 5, 27, 28, 11, 4, 31] or text [7] or by manually adjusting the
latent representation [29]. Conditional generation learns a representation
from the data and does not capture the exact features utilized by a
classifier, as the gradient based methods do.

Several methods exist for studying neural network based classifiers to
understand the features they use. Early approaches include gradient based
attribute maps (aka saliency heatmaps) that could be overlaid on the image
[30]. This was extended to capture more caveats of neural network
reasoning[33, 32] but these methods still only generate heatmap explanations.
Other work focuses on leveraging occlusions to identify discrete image regions
[23]. Other work inspects individual neural network neurons to identify where
class information was propagating using linear probing [1] and manual
identification [21, 20]. Additional methods focus on concepts represented by
layer activations and identifying images that share these representations
[14]. Another approach [2] generates synthetic images using human annotators
such that the images have known feature similarity (e.g. a face with all
features held constant except for skin color). These images are then used to
identify spurious correlations between attributes.

##  3 Alignment Methodology

This work proposes the CF alignment approach as described in Fig. 1. For this
approach, we generate counterfactuals using Latent Shift [8]. This approach
requires an encoder/decoder model $D(E(x))$ where $E$ is the encoder and $D$
is the decoder, and a classifier $f$ which predicts a target $y$ as follows:
$y=f(x)$. The autoencoder and the classifier are trained independently without
specific requirements except for being differentiable and operating on the
same data domain. To compute a counterfactual, an input image $x$ is encoded
using $E(x)$, to produce a latent representation $z$. Perturbations of the
latent space are computed for a base classifier $f_{b}$ in Eq. 1 which is then
used to produce $\lambda$-shifted samples as shown in Eq. 2. $\lambda$ is
determined using an iterative search where the value is decreased in steps
until the classifier’s prediction is reduced by 0.6 or begins to increase.
$z_{\lambda}=z-\lambda\frac{\partial f_{b}(D(z))}{\partial z}$ (1)
$x^{\prime}_{\lambda}=D(z_{\lambda})$ (2)

These counterfactual samples can then be input into downstream classifiers
that predict various attributes (e.g. $f_{1}$, $f_{2}$) and observe how their
predictions change. Relative change (Eq. 3) allows us to quantify the
relationship between a base and downstream classifier, similar to correlation,
but accounts for the magnitude of the change. In our experiments, we found
correlation would produce false positives when there was only a slight change
in the prediction of $f_{1}$ compared to the base classifier $f_{b}$.

$\frac{f_{1}(D(z_{\lambda}))-f_{1}(D(z_{0}))}{f_{b}(D(z_{\lambda}))-f_{b}(D(z_{0}))}$
(3)

##  4 Experiments

The experiments in this work are performed on the CelebA dataset [17] that
contains over 200k celebrity images with 40 facial attribute labels per image.
The resolution of the images is 1024$\times$1024.

The pre-trained face classifiers used in this work are sourced from [34]. They
were trained on the CelebA dataset [17] to predict 40 different facial
attributes on images of dimensions 224$\times$224\. We chose this model
because it has good performance, is publicly available, and was implemented in
PyTorch.

We leverage the VQ-GAN autoencoder from [9] trained on the FacesHQ dataset,
which combines the CelebA dataset [17] and the Flickr-Faces-HQ (FFHQ) dataset
[13]. The resolution of this model is 256x256.

The Captum library [15] is utilized for baseline attribution methods. PyTorch
[22] is used for efficient tensor computation and automatic differentiation.
The source code and model weights for all experiments will be released
publicly online111https://github.com/ieee8023/latentshift.

Figure 2: Relationships between face attribute classifiers as measured by CF-
induced relative changes (left), classifier predictions (middle) and training
data labels (right). In (a), base classifiers are along the rows and
downstream classifiers are along the columns. Comparing (a) to (b) and (c),
shows that many relationships reflected in the CF outputs are preserved from
correlations in the training data. The non-symmetric relationship between
_male_ and _wearing_lipstick_ , highlighted in green, reflects expected
interactions between the two face attribute classifiers. The relationship
between _male_ and _big_nose_ , highlighted in red, is strong in both the
classifier predictions and ground truth labels but low in CF alignment,
indicating that although correlated, these features aren’t exploited by the
classifier.

(a) CF alignment for _male_ showing alignment with classifiers related to hair
and inverse alignment with classifiers related to makeup. The CF image
visually appears to add makeup and remove hair.

(b) CF alignment for _attractive_ showing alignment with classifiers related
to makeup, face shape, and age and an inverse alignment with nose size. The CF
image visually appears to remove makeup and add bags under the eyes.

(c) CF alignment for _heavy_makeup_ showing alignment with classifiers related
to makeup and face shape and inverse alignment with classifiers for nose size
and bags under the eyes. The CF image visually shows makeup being removed.

Figure 3: Examples of alignment for specific samples showing the visual
counterfactual changes as well as other classifiers with aligned and inverse
aligned relationships.

###  4.1 Aggregate statistics over a dataset

Viewing aggregate CF alignment statistics over a dataset can be useful when
investigating a model for bias or spurious correlations. The average relative
change between pairs of classifiers for 400 samples per class is shown in Fig.
2a. Figures 2b and 2c show correlations between classifier predictions and the
ground truth training labels. Only an interesting subset of classifiers are
shown in this figure for clarity, with the full CF alignment matrix in
Appendix Fig. 9.

We draw the readers attention to the example relationship between _male_ and
_big_nose_ in red, which is high in both classifier prediction and ground
truth labels but very low using CF alignment. This indicates that while the
attributes (labels) are correlated, the _male_ classifier does not largely use
_big_nose_ as a feature to predict _male_ \- an expected insight that is
verified. A similar insight is found for the class of _mouth_slightly_open_ ,
which has a relatively strong relationship with _smiling_.

We also observe many non-symmetric relationships such as _male_ and
_wearing_lipstick_ in green, as well as _young_ and _attractive_. These are
also intuitive relationships that we would expect the classifiers to learn.
The _male_ classifier likely uses lipstick as a feature to reduce the
likelihood that the image contains a _male_ subject (high inverse correlation
in Fig. 2c), while the _wearing_lipstick_ classifier would be more focused on
the color of the lips. Overall, these results help qualitatively depict that
counterfactual reasoning can provide model introspection in a manner that is
consistent with human reasoning.

###  4.2 Studying specific examples

Using the aggregate analysis from §4.1 as a guide, classifiers can be selected
to investigate undesired relationships. We will first select a sample which
contains an inverse relationship between a base classifier for _male_ and
downstream classifier for _heavy_makeup_ in Fig. 3(a). Visually, we can see
makeup and lipstick in the CF image and the predictions increase as expected
in the classifiers for _heavy_makeup_ , _wearing_lipstick_ , and _no_beard_.
Similarly, there is a decrease in the classifier for _goatee_. These appear
reasonable and provide insight about the features used for prediction.
Exploring the more subjective base classifier label _attractive_ in Fig. 3(b)
we observe _big_nose_ and _bags_under_eyes_ increase, which can depict human
preferences and biases in determining an _attractive_ face.

Figure 4: Example of saliency maps failing to provide meaningful localization
when the concepts overlap with each other. Here, saliency map methods localize
around the nose which doesn’t provide the ability to distinguish between a big
or pointy nose prediction.

###  4.3 Comparison with saliency maps

Saliency maps for feature attribution could potentially be used to perform a
similar analysis by looking at relationships between their generated heatmaps.
Here, we present an example that demonstrates their limitation. The
attribution methods input gradients, integrated gradients [33], and guided
backprop [32] are used to explain the classifier predictions of _pointy_nose_
and _big_nose_. In Fig. 4 the salient areas for _pointy_nose_ and _big_nose_
are the same and cannot be disambiguated. Comparing these heatmaps would yield
a high similarity but we observe in Fig. 2 that these two classifiers have an
inverse relationship. Using CF alignment, we can gain a deeper understanding
of what features the model is using and better reason about why the decision
was made.

###  4.4 Validation by inducing spurious correlations

In order to further verify the CF alignment approach, we construct a
classifier with a known spurious correlation and then demonstrate that this
bias is observable in the CF alignment plot. A spurious correlation can be
induced in the classifier by composing classifiers:

$f_{biased}(x)=f_{smiling}(x)+0.3f_{arched\\_eyebrows}(x)$ (4)

The CF alignment plot for the base _smiling_ classifier predictions in Fig.
5(a) show that arched_eyebrows does not change and we can confirm this in the
CF image. The resulting biased classifier can be observed using the
_arched_eyebrows_ features in Fig. 5(b) both visually as well as in the CF
alignment plot. The CF alignment plot shows that the prediction of
_arched_eyebrows_ now changes and is aligned with _smiling_.

(a) CF for _smiling_ showing eyebrows are unchanged. The horizontal line
indicates the prediction of _arched_eyebrows_ is not influenced by the
features used for _smiling_ in this image.

(b) CF for the modified _smiling_ classifier with an arched eyebrow spurious
correlation showing that eyebrows are now changed.

Figure 5: Example of detecting a spurious correlation in a biased classifier.
The classifier is biased with arched eyebrows and this is observed in the
alignment plot as well as in the counterfactual image. In order to decrease
the prediction of the (now biased) classifier, the CF image can remove the
arched eyebrows as well.

###  4.5 Rectifying bias by composing classifiers

This section demonstrates the use of CF alignment to fix model bias in
classifiers. We can reuse the relative change between model predictions as a
loss function that can be minimized.

Correcting model bias and spurious correlations is an active area of research.
Approaches include averaging model weights [36] or methods of invariant
optimization [26, 16, 24, 37]. Composing classifiers can change their response
to specific samples that contain relevant features as shown in §4.4. By using
this additive composition approach we avoid the variance of model training
which makes it easy and reproducible to perform experiments.

First, we demonstrate a single example of correcting bias. In Fig. 6, the CF
image and CF alignment plot for the _heavy_makeup_ base classifier show
unexpected reductions in lip size. Composing the model with the _big_lips_
classifier using a negative coefficient causes the classifier to lower any
feature change focused on the lips as this would increase the prediction of
the composed classifier (where the goal of the CF is to decrease it). We can
also visually observe other features that change, such as skin color, bushy
eyebrows, and the amount that the mouth is open.

(a) As makeup is removed from the face, the lip size also shrinks.

(b) The lip size remains the same as the makeup is removed.

Figure 6: An example of a _big_lips_ spurious correlation being corrected for
the _heavy_makeup_ classifier.

Next, this unbiasing is scaled up to a larger number of samples and
classifiers. A collection of classifiers that contain varying spurious
relationships to a _big_nose_ base classifier (shown in Fig. 7a) are modified
to remove their bias until their relative change is as close to 0 as possible.
This experiment is performed with a train, validation, and hold out test set
in order to demonstrate the generalization of this unbiasing process to unseen
data.

The classifiers are modified using a single coefficient ($\beta$) such that
the resulting classifier is

$f^{\prime}_{target}(x)=f_{target}(x)+\beta f_{big\\_nose}(x)$ (5)

The optimization to compute $\beta$ is performed using a pseudo gradient
descent where the gradient is approximated by the mean relative change between
the target classifier and the _big_nose_ classifier. By subtracting the
relative change from $\beta$, scaled by a learning rate, the relative change
($\psi$) with that classifier will be reduced. All together our training
objective (including momentum) is

$\beta_{n}=0.001\psi(f_{target},f_{big\\_nose})+0.1\beta_{n-1}$ (6)

We find that using small minibatches of 10 samples works well because the
computation time for each CF can take over 1 second on a GPU. Additionally,
training with samples which induce a small change in the base classifier
during the CF generation process can be challenging. As the classifiers are
modified, this base change is reduced, which causes the relative change to
become more erratic and prevents the optimization from converging. To prevent
this, we use samples with a base change $>0.6$.

The resulting training metrics in Fig. 7 show model biases being summarized
and minimized similar to a differentiable training loss. We observe a bias
generalization gap between the train, valid, and test sets, indicating some
degree of overfitting. Early stopping (red vertical line) is used on the
validation set to determine the optimal parameters. We use momentum during
training to average over the noise. The resulting reduction in bias is shown
in Fig 7b. All biases are considerably reduced except for _attractive_ and
_big_nose_ , which could be the result of overlapping features used by both
classifiers that cannot be changed.

Figure 7: Training curves showing mean relative change during the optimization
of a classifier with respect to the _big_nose_ classifier during each
iteration. The red vertical line is the early stopping point based on the
validation set. This demonstrates how bias can be quantified and corrected
using CF alignment. Figure 8: CF alignment matrices before and after
optimization to remove the bias for _big_nose_ on a hold out test set. The
figure shows a dramatic reduction in relative change for most classifiers and
_big_nose_ with limited residual impact on other relationships.

##  5 Limitations

There are limitations and challenges to CF generation [35] where bias can also
exist in the CF generation method. Furthermore, although we are optimistic
that our method will generalize to additional domains, we focus our current
analysis on face attribute classification. Although we do not demonstrate the
generalizability of our method to additional domains, using face attribute
classification is a deliberate choice enabling qualitative evaluation, in
addition to our quantitative evaluation. Additionally, since CF generation
relies on a separate autoencoder, improving the representational capability of
the autoencoder may improve the fidelity of the CF generation.

##  6 Conclusion

In this work we propose the counterfactual (CF) alignment approach to reason
about the feature relationships in classifiers in aggregate statistics, to
locate specific examples with spurious correlations, and propose strategies to
mitigate the effect of spurious correlations. These claims are supported with
an analysis of face attribute classifiers that identify many expected and
unexpected spurious correlations such as _attractive_ with _heavy_makeup_ and
_heavy_makeup_ with _big_nose_. The validity of the CF alignment method is
confirmed by inducing and quantifying spurious correlations via additive
composition. We then explore rectifying classifiers based on observed spurious
correlations. Overall, the proposed approach may serve as an end-to-end or
human-in-the-loop system to automatically detect, quantify, and correct
spurious correlations for image classification tasks that lead to biased
classifier outputs.

##  7 Acknowledgments

We would like to thank Stanford University and the Stanford Research Computing
Center for providing computational resources and support that contributed to
these research results.

## References

  * Alain and Bengio [2016] Guillaume Alain and Yoshua Bengio.  Understanding intermediate layers using linear classifier probes.  _International Conference on Learning Representations_ , 2016.
  * Balakrishnan et al. [2021] Guha Balakrishnan, Yuanjun Xiong, Wei Xia, and Pietro Perona.  Towards Causal Benchmarking of Bias in Face Analysis Algorithms.  In _Computer Vision and Pattern Recognition_ , 2021.
  * Balasubramanian et al. [2020] Rachana Balasubramanian, Samuel Sharpe, Brian Barr, Jason Wittenbach, and C. Bayan Bruss.  Latent-CF: A Simple Baseline for Reverse Counterfactual Explanations.  In _Neural Information Processing Systems (NeurIPS) Fair AI in Finance Workshop_ , 2020.
  * Barredo-Arrieta and Del Ser [2020] Alejandro Barredo-Arrieta and Javier Del Ser.  Plausible Counterfactuals: Auditing Deep Learning Classifiers with Realistic Adversarial Examples.  In _International Joint Conference on Neural Networks_ , 2020.
  * Baumgartner et al. [2018] Christian F. Baumgartner, Lisa M. Koch, Kerem Can Tezcan, Jia Xi Ang, and Ender Konukoglu.  Visual Feature Attribution Using Wasserstein GANs.  In _Computer Vision and Pattern Recognition_ , 2018.
  * Beery et al. [2018] Sara Beery, Grant Van Horn, and Pietro Perona.  Recognition in Terra Incognita.  In _European Conference on Computer Vision_ , 2018.
  * Chambon et al. [2022] Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P Langlotz, and Akshay Chaudhari.  RoentGen: Vision-Language Foundation Model for Chest X-ray Generation, 2022.
  * Cohen et al. [2021] Joseph Paul Cohen, Rupert Brooks, Sovann En, Evan Zucker, Anuj Pareek, Matthew P. Lungren, and Akshay Chaudhari.  Gifsplanation via Latent Shift: A Simple Autoencoder Approach to Counterfactual Generation for Chest X-rays.  _Medical Imaging with Deep Learning_ , 2021.
  * Esser et al. [2021] Patrick Esser, Robin Rombach, and Björn Ommer.  Taming transformers for high-resolution image synthesis.  In _Computer Vision and Pattern Recognition_ , 2021.
  * Geirhos et al. [2020] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann.  Shortcut learning in deep neural networks.  _Nature Machine Intelligence_ , 2020.
  * Hasenstab et al. [2023] Kyle A. Hasenstab, Justin Huynh, Samira Masoudi, Guilherme M. Cunha, Michael Pazzani, and Albert Hsiao.  Feature Interpretation Using Generative Adversarial Networks (FIGAN): A Framework for Visualizing a CNN’s Learned Features.  _IEEE Access_ , 2023.
  * Joshi et al. [2018] Shalmali Joshi, Oluwasanmi Koyejo, Been Kim, and Joydeep Ghosh.  xGEMs: Generating Examplars to Explain Black-Box Models, 2018.
  * Karras et al. [2021] Tero Karras, Samuli Laine, and Timo Aila.  A Style-Based Generator Architecture for Generative Adversarial Networks.  In _Transactions on Pattern Analysis and Machine Intelligence_ , 2021\.
  * Kim et al. [2018] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres.  Interpretability beyond feature attribution: Quantitative Testing with Concept Activation Vectors (TCAV).  In _International Conference on Machine Learning_ , 2018.
  * Kokhlikyan et al. [2020] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson.  Captum: A unified and generic model interpretability library for PyTorch, 2020.
  * Krueger et al. [2021] David Krueger, Ethan Caballero, Joern Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville.  Out-of-Distribution Generalization via Risk Extrapolation.  In _International Conference of Machine Learning_ , 2021.
  * Liu et al. [2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In _International Conference on Computer Vision_ , 2015.
  * Mirza and Osindero [2014] Mehdi Mirza and Simon Osindero.  Conditional Generative Adversarial Nets, 2014.
  * Moreno-Torres et al. [2012] Jose G. Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodríguez, Nitesh V. Chawla, and Francisco Herrera.  A unifying view on dataset shift in classification.  _Pattern Recognition_ , 2012.
  * Olah et al. [2017] Chris Olah, Alexander Mordvintsev, and Ludwig Schubert.  Feature Visualization.  _Distill_ , 2017.
  * Olah et al. [2020] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter.  Zoom In: An Introduction to Circuits.  _Distill_ , 2020.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  PyTorch: An Imperative Style, High-Performance Deep Learning Library.  _Neural Information Processing Systems_ , 2019.
  * Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier.  2016\.
  * Rieger et al. [2020] Laura Rieger, Chandan Singh, W. James Murdoch, and Bin Yu.  Interpretations are useful: Penalizing explanations to align neural networks with prior knowledge.  In _International Conference on Machine Learning_ , 2020.
  * Ross et al. [2017] Andrew Ross, Michael C Hughes, and Finale Doshi-Velez.  Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations.  In _International Joint Conference on Artificial Intelligence_ , 2017\.
  * Sagawa et al. [2020] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang.  Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization.  _International Conference on Learning Representations_ , 2020.
  * Samangouei et al. [2018] Pouya Samangouei, Ardavan Saeedi, Liam Nakagawa, and Nathan Silberman.  ExplainGAN: Model Explanation via Decision Boundary Crossing Transformations.  In _European Conference on Computer Vision_ , 2018.
  * Schutte et al. [2020] Kathryn Schutte, Olivier Moindrot, Paul Hérent, Jean-Baptiste Schiratti, and Simon Jégou.  Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images.  In _Medical Imaging meets NeurIPS_ , 2020.
  * Seah et al. [2019] Jarrel C.Y. Seah, Jennifer S.N. Tang, Andy Kitchen, Frank Gaillard, and Andrew F. Dixon.  Chest radiographs in congestive heart failure: Visualizing neural network learning.  _Radiology_ , 2019.
  * Simonyan et al. [2014] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.  In _International Conference on Learning Representations (ICLR)_ , 2014.
  * Singla et al. [2023] Sumedha Singla, Motahhare Eslami, Brian Pollack, Stephen Wallace, and Kayhan Batmanghelich.  Explaining the black-box smoothly—A counterfactual approach.  _Medical Image Analysis_ , 2023.
  * Springenberg et al. [2015] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller.  Striving for Simplicity: The All Convolutional Net.  In _International Conference on Learning Representations Workshop_ , 2015.
  * Sundararajan et al. [2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic Attribution for Deep Networks.  In _International Conference on Machine Learning_ , 2017.
  * Vandenhende et al. [2020] Simon Vandenhende, Stamatios Georgoulis, Bert De Brabandere, and Luc Van Gool.  Branched Multi-Task Networks: Deciding What Layers To Share.  _British Machine Vision Conference_ , 2020.
  * Verma et al. [2020] Sahil Verma, Varich Boonsanong, Minh Hoang, Keegan E. Hines, John P. Dickerson, and Chirag Shah.  Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review.  In _Neural Information Processing Systems (NeurIPS) Retrospectives Workshop_ , 2020.
  * Wortsman et al. [2022] Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt.  Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time.  In _International Conference on Machine Learning_ , 2022.
  * Zeng et al. [2023] Huimin Zeng, Zhenrui Yue, Ziyi Kou, Yang Zhang, Lanyu Shang, and Dong Wang.  Fairness-aware training of face attribute classifiers via adversarial robustness.  _Knowledge-Based Systems_ , 2023.

Supplementary Material

##  8 Appendix

Figure 9: The complete matrix of relative changes for all classifiers.
Ordering is determined by clustering to group similar classifiers.

(a) _pointy_nose_

(b) _smiling_

(c) _attractive_

Figure 10: Visualization of the training from §4.5 reducing the bias of
_big_nose_. The model being evaluated is
$f^{\prime}_{target}(x)=f_{target}(x)+\beta f_{big\\_nose}(x)$ where $\beta$
is specified by the coefficient column. The metric here is relative change
computed between the $f^{\prime}_{target}(x)$, where $target$ is specified in
the caption, and the classifier specified by the column header. The top row is
from the training set performed on batches (which explains the variance) and
the second row is the results on the entire validation set. The row of each
table is for a specific iteration. The iterations proceed downward. The
training of _attractive_ shown here does not result in a debiased model, the
relative change diverges from 0 indicating a failed training.