# Remarks about weighted energy integrals over Minkowski spectral functions
from Euclidean lattice data

Thomas DeGrand  Department of Physics, University of Colorado, Boulder, CO
80309 USA<EMAIL_ADDRESS>
###### Abstract

I make some simple observations about the calculation of weighted averages
over energy of Minkowski space spectral densities from weighted averages over
time of Euclidean space correlation functions, measured in latice simulations.
The correlator of two vector currents is used as an example, where it appears
that a determination of a weighted average of the spectral function near the
rho pole at the five per cent level is possible from lattice simulations.

Finding connections between theoretical calculations done in Euclidean space
and results of experiments done in Minkowski space is a longstanding problem
in many areas of physics and involves many approaches. Lattice studies of QCD
and other related systems are no exception. This short note describes a simple
technique for extracting weighted averages over energy of Minkowski space
spectral densities from Euclidean space lattice correlation functions.
Examples are motivated by calculations of the hadronic vacuum polarization
contribution to the muon anomalous magnetic moment (with Ref. Aoyama:2020ynm
as my primary reference) though there are obvious applications to many similar
processes Hansen:2017mnd .

When the Euclidean correlation $\Pi(Q)$ is related to the Minkowski space
spectral function $\rho(\omega)$ measured at energy $\omega$ by a once-
subtracted dispersion relation, the connection between a Euclidean space
correlation function defined at Euclidean time $t$, $G_{E}(t)$, and the
spectral function is

$G_{E}(t)=\frac{1}{2\pi}\int_{0}^{\infty}d\omega[\omega^{2}\rho(\omega)]\exp(-\omega
t).$ (1)

Inverting Eq. 1 to predict $\rho(\omega)$ from $G_{E}(t)$ is a difficult
problem. However, it seems easy to compare a weighted average of $G_{E}(t)$ to
a weighted average of $\rho(\omega)$,

$\hat{\rho}(Q_{0})\equiv\int_{0}^{\infty}R_{E}(Q_{0},t)G(t)dt=\int_{0}^{\infty}d\omega\rho(\omega)T(Q_{0},\omega)$
(2)

with the connection

$T(\omega)=\frac{\omega^{2}}{2\pi}\int_{0}^{\infty}e^{-\omega
t}R_{E}(Q_{0},t)dt.$ (3)

$Q_{0}$ is shorthand for possible tunable parameter(s) in the weighting
function. The most prominent present-day example of such a connection is the
calculation of the hadronic vacuum polarization contribution for the anomalous
magnetic moment of the muon $a_{\mu}^{HVP}$. The Euclidean weighting function
$R_{E}(Q_{0},t)$ for $a_{\mu}$ is specified by a QED calculation. But of
course, one could imagine doing the weighting with any function
$R_{E}(Q_{0},t)$.

Each choice of $R_{E}(Q_{0},t)$ amounts to its own (indirect) comparison of
theory ($G_{E}(t)$) with experiment ($\rho(\omega)$, processed into
$\hat{\rho}(Q_{0})$). Families of related $R_{E}(Q_{0},t)$’s can be combined
into more extensive views of the spectral density. I don’t want to speculate
on whether this could do a better job of probing the spectral function than
the standard technique of fitting $G_{E}(t)$ to a functional form with a set
of parameters (masses and coupling constants) and then continuing the fit
function from Minkowski to Euclidean space. I just want to raise the
possibility that analysis methods for $a_{\mu}^{HVP}$ might have wider
applications.

Some choices of $R_{E}$ are going to be more interesting than others, and a
desirable goal would be to find an $R_{E}((Q_{0},t)$ whose $T(\omega)$ is
peaked around some energy range. To jump to the conclusion, the dominant
feature of an $R_{E}(Q_{0},t)$ which does that is a restriction to a range of
$t$ values $t_{min}<t<t_{max}$; the overall shape of $R_{E}(Q_{0},t)$ does not
seem to be important for the examples I display. And given what is published
about the precision of $a_{\mu}^{HVP}$ lattice results, it seems possible to
make a lattice determination of a weighted average of $\rho(\omega)$ with
enough accuracy to be phenomenologically interesting. (I have in mind the few
per cent tension in the $\pi\pi$ channel in the 0.6-0.9 GeV range described in
Ref. Aoyama:2020ynm , between the KLOE experiment KLOE-2:2017fda  and other
groups.)

I think I am saying obvious things, but I haven’t found a discussion of this
approach in the literature.

The idea described here is just a trivial variation on the “coordinate space
representation” for $a_{\mu}^{HVP}$: there is an implicit assumption that
$R_{E}(t)$ is a smooth function of $t$, and replacing an integral over
continuous $t$ by a sum over a set of discrete lattice points is no different
than replacing any continuous integral by a grid sum. There is also a large
literature proposing solutions to the “inverse problem:” given a
$G_{E}(t_{i})$ defined at a set of discrete $t_{i}$ values, various approaches
have different criteria for defining and constructing a weighted $\hat{\rho}$.
Often, no smoothness assumptions go into the choice and in fact the
$R_{E}(t_{i})$’s found in the literature are far from smooth. Recent
references (a very incomplete set for this vast field) are Ref. Hansen:2017mnd
, which uses the Backus - Gilbert method Backus ; NR , related work by Ref.
Hansen:2019idp , and Chebychev techniques by Refs. Gambino:2020crt ;
Fukaya:2020wpp ; Bailas:2020qmv .

I will continue the note focussing on $a_{\mu}^{HVP}$. There is a small
literature associated with modifications to its $R_{E}$. Probably the most
prominent one is the “intermediate window method” of Ref. RBC:2018dos . It is
a time - sliced version of the $a_{\mu}^{HVP}$ weighting:

$R_{E}(Q_{0},t)=R_{E}^{a_{\mu}}(Q_{0},t)[\Theta(t,t_{min},\Delta)-\Theta(t,t_{max}\Delta)]$
(4)

where $\Theta(t,t_{0},\Delta)$ is a smoothed step function. Another approach
to weighting, called “finite energy sum rules,” starts by writing a dispersion
relation for a reweighted $\Pi_{E}(Q)$. For a discussion, see Refs.
Davier:1998dz ; Maltman:1998uzw .

To set conventions, we are interested in the correlator of two vector currents

$\Pi(q)_{\mu\nu}=\int d^{4}xe^{iqx}\left\langle
0|J_{\mu}(x)J_{\nu}(0)|0\right\rangle.$ (5)

We remove the indices with a transverse projection,

$\Pi_{\mu\nu}=[q_{\mu}q_{\nu}=g_{\mu\nu}q^{2}]\Pi(q^{2})$ (6)

and then the spectral function $\rho(\omega)$ is proportional to the
discontinuity of $\Pi$ across the real energy axis (setting
$q_{\mu}=(\omega,\vec{0})$). It is also proportional to the R-ratio,
$R(\omega)=\sigma(e^{+}e^{-}\rightarrow
hadrons)/\sigma(e^{+}e^{-}\rightarrow\mu^{+}\mu^{-})$. The standard lattice
vector - vector correlator contracts $\rho_{\mu\nu}$ against polarization
vectors $\epsilon^{i}_{\mu}\epsilon^{j}_{\nu}$ where typically
$\epsilon^{i}_{\mu}=(0,\vec{\epsilon}^{i})$ is a unit vector. This means that
in Eq. 1 $\rho(\omega)=R(\omega)/(6\pi)$ and

$G_{E}(t)=\sum_{i}\int d^{3}x\left\langle
J_{i}(\vec{x},t)J_{i}(0,0)\right\rangle$ (7)

where $J_{i}(x,t)=e_{q}\bar{\psi}(x,t)\gamma_{i}\psi(x,t)$ for a quark of
charge $e_{q}$ (in units of the electric charge).

The two relevant pictures are shown in Fig. 1: the familiar plot of the
R-ratio in panel (a) and the expected $G_{E}(t)$ in panel (b), using Eq. 1 to
do the inversion. “Experiment” in these pictures are the phenomenological
model for $\rho(\omega)$ from Ref. Bernecker:2011gh  (in black) and a
compilation of $R(\omega)$ from a table in the Review of Particle Properties
ParticleDataGroup:2020ssz  (in red). Of course, the question to try to answer
is: Given a calculation $G_{E}(t)$, what can one say about $\rho(\omega)$?

Figure 1:  (a) $R(\omega)$ the R-ratio from the phenomenological model for
$\rho(\omega)$ from Ref. Bernecker:2011gh  (in black) and from the table in
the Review of Particle Properties ParticleDataGroup:2020ssz  (in red). (b)
$G_{E}(t)$ from the phenomenological model for $\rho(\omega)$ from Ref.
Bernecker:2011gh  (in black) and from the table in the Review of Particle
Properties ParticleDataGroup:2020ssz  (in red). The inversion uses Eq. 1. The
dotted line is the contribution from a stable rho meson with $f_{V}=0.25$.

This question is partially answered by the one theoretical line in panel (b)
of Fig. 1: The straight line is the contribution of a stable rho meson at 770
MeV with a decay constant $f_{V}=0.25$:

$G_{E}^{V}(t)=\frac{(\left\langle q\right\rangle
m_{V}^{2}f_{V})^{2}}{2m_{V}}\exp(-m_{V}t).$ (8)

The quantity $\left\langle q\right\rangle$ is the expectation value of the
quarks’ charges in the meson: $[2/3-(-1/3)]/\sqrt{2}=1/\sqrt{2}$ for the rho,
$1/18$ for the omega, $1/9$ for the phi, and so on. $G_{E}(t)$ is flatter than
$G_{E}^{V}(t)$ at very large $t$ due to the contribution of two-pion states
with an invariant mass smaller than the rho mass, and it is steeper than
$G_{E}^{V}(t)$ at small $t$ due to the phi meson and to the flat high energy
part of $R(\omega)$. Nowhere does $G_{E}^{V}(t)$ saturate $G_{E}(t)$.

I can rephrase the question to try to answer as: Given a lattice calculation
of $G_{E}(t)$, what can one say about $\rho(\omega)$? Then there are more
constraints. The large $\omega$ region, where $\omega>1/a$ and $a$ is the
lattice spacing, is contaminated by lattice artifacts, and is inaccessible to
a lattice calculation. Unfortunately, so is the small $\omega$ or large $t$
region. There are two reasons for this. First, the lattice signal becomes
noisy. This is a usual issue in lattice simulations Lepage:1989hd ;
Parisi:1983ae ; Endres:2011jm . The data in Ref. FermilabLattice:2019ugu
provide an example – see their Fig. 2. The collaboration has data at lattice
spacings between 0.15 and 0.06 fm. Their data is only usable out to distances
$t\sim 2.5$ fm. This precludes, at least for the present, studies of
$\rho(\omega)$ near threshold. This situation is well known and documented in
the $a_{\mu}^{HVP}$ literature Aoyama:2020ynm .

A second reason that the small $\omega$ region is difficult is that it is
dominated by two pion states. Lattice simulations are done in a finite box
(say, of size $L$) and particle momenta are quantized,
$\vec{p}_{n}=2\pi\vec{n}/L$ for integer valued $\vec{n}$, so that there is no
two pion continuum in a lattice simulation, just a set of exponentially
falling contributions $\propto\exp(-2\sqrt{p_{n}^{2}+m_{\pi}^{2}}t)$.
Presumably, these contributions interpolate into the continuum result when the
volume is taken sufficiently large, Finite volume $\rho(\omega)$’s are sums of
delta functions, but the smearing washes out this behavior.

Parenthetically, the vector correlator presents a somewhat special case
compared to most lattice studies, where the lightest state in (continuum)
$\rho(\omega)$ is an isolated pole. Then, simply going to large $t$ gives a
$G_{E}(t)$ which is dominated by properties of the pole. Standard lattice
techniques (fits to exponentials) are more efficient at producing high quality
results than the proposal of weighting $G_{E}(t)$ given here.

So we are pushed back to the region of $\omega$ near the rho mass. The
physical rho meson is broad. Is it possible to say anything about
$\rho(\omega)$ for $\omega$ near $m_{\rho}$? This seems to be a serious issue
for $a_{\mu}^{HVP}$ determinations.

Dividing up the contributions to $G_{E}(t)$ from different energy intervals
shows the way to go. See Fig. 2, which shows the fractional contributions to
$G_{E}(t)$ from different $\omega$ regions. Here $\rho(\omega)$ is taken from
the phenomenological model of Ref. Bernecker:2011gh . What is noticable is
that there is a fairly wide region at intermediate $t$ where the region around
the rho mass contributes heavily. Of course, this can be seen by eye in Fig.
1. (This is basically just the phenomenon of vector meson dominance.)

Figure 2:  Fractional contributions to $G_{E}(t)$ where $\rho(\omega)$ is
taken from the phenomenological model of Ref. Bernecker:2011gh . The curves
label (a) $2m_{\pi}<\omega<4m_{\pi}$; (b) $4m_{\pi}<\omega<6m_{\pi}$; (c)
$6m_{\pi}<\omega<8m_{\pi}$; (d) $8m_{\pi}<\omega<10m_{\pi}$.

As an application of this remark, suppose that the experimental $\rho(\omega)$
is not precisely known over some energy range, that two experiments differ by
a fraction $\delta\rho(\omega)/\rho(\omega)$. Assuming that the difference is
confined to some small region of $\omega$, there will be a change in
$G_{E}(t)$ (constructing it from Eq. 1 with each experimental $\rho(\omega)$)
of $\delta G_{E}(t)/G_{E}(t)\sim f\delta\rho/\rho$ where $f$ is the fractional
contribution of the $\omega$ region of $\rho(\omega)$ to $G_{E}(t)$.

A simple example comes from modifying the model for $\rho(\omega)$ from Ref.
Bernecker:2011gh  over a range $\omega_{min}<\omega<\omega_{max}$, by
multiplication by a weighting factor

$w(\omega)=1+a\sin\pi\left(\frac{\omega-\omega_{min}}{\omega_{max}-\omega_{min}}\right).$
(9)

The fractional change in $G_{E}(t)$ is shown in Fig. 3 for the choice
$\omega_{min}=0.6$ GeV, $\omega_{max}=0.9$ GeV, $a=0.05$.

Notice the qualifier “assuming that the difference is confined to some small
region of $\omega$.” $G_{E}(t)$ at any $t$ value is built of contributions
from all $\omega$, and a measurement of $G_{E}(t)$ at any $t$ or for any range
of $t$ values does not make an absolute prediction about $\rho(\omega)$ at any
particular $\omega$ value. However, lattice results could still be useful to
distinguish between the different experimental $\rho(\omega)$’s.

Fig. 3 shows that a five per cent variation in $\rho(\omega)$ translates into
a 2.7 per cent variation in $G_{E}(t)$ over a fairly wide range of $t$. This
is really the end of the story I can tell – I cannot write about uncertainties
in either $G_{E}(t)$ or $\rho(\omega)$. Lattice data for $G_{E}(t)$ is
typically highly correlated and it is almost impossible to estimate
correlation uncertainties in a lattice data set without access to it.
Similarly, the experimental data sets which give $\rho(\omega)$ are highly
correlated. But, 2.7 per cent seems to be an easy target, given that
contemporary lattice measurements of $a_{\mu}^{HVP}$ are well under a per
cent. It seems likely that lattice calculations could determine $\rho(\omega)$
over the range 0.6-0.9 GeV at the five per cent level.

Figure 3:  The fractional change in $G_{E}(t)$ from a five per cent variation
in the phenomenological model for $\rho(\omega)$ of Ref. Bernecker:2011gh
over the range 0.6-0.9 GeV, as described in the text.

To do this in practice, we need a weighting function. There seem to be many
possible choices. But Figs. 2 and 3 indicate that all that is significant for
$R_{E}(t)$ is that it has a cutoff at small and large $t$, that it is nonzero
for $t_{min}<t<t_{max}$. Two choices of $R_{E}(t)$ illustrate that claim.

First consider a family of power laws, $R_{E}(t)=(t/t_{0})^{n}/n!$ for a range
$t_{min}<t<t_{max}$. $t_{0}$ and the $n!$ factor are just rescalings, useful
for plots across $n$ or for comparing weighted lattice data at different
lattice spacings. (This is a hard cutoff; Fig. 3 indicates that a soft cutoff
would perform similarly.) Fig. 4 shows the contribution of
$4m_{\pi}<\omega<6m_{\pi}$ to the integral of Eq. 2. Each panel is for a
particular $n$ value and shows a set of curves: each curve is the fraction of
the integral from $t_{min}$ to $t_{max}$ from this $\omega$ range, varying
$t_{min}$ at fixed $t_{max}$. A range of $t$ in the range 1-2 fm gives an
integral where the contribution of $4m_{\pi}<\omega<6m_{\pi}$ region of
$\rho(\omega)$ approaches 70 per cent, essentially independent of $n$. The
curves extending out to $t_{max}=5$ fm show the obvious result that the
contribution of the rho region to the integral becomes very small when taking
$t_{min}>2$ fm.

Figure 4:  The contribution of $4m_{\pi}<\omega<6m_{\pi}$ to the integral of
Eq. 2 for a power law $R_{E}(t)=(t/t_{0})^{n}/n!$ with $t_{0}=0.15$ fm, for a
range $t_{min}<t<t_{max}$ plotted versus $t_{min}$ for $t_{max}=1.2$, 1.44,
1.8, 2.4 and 5 fm. (a) $n=0$; (b) $n=2$; (c) $n=4$; (d) $n=6$.

Fig. 5 shows the fractional change in the integral $\hat{\rho}$ from the model
weighting factor of Eq. 9. This is the analog of Fig. 3 and the result is the
same – the sensitivity to variation in $\rho(\omega)$ depends most on the
range of $t$ spanned by $R_{E}(t)$.

Figure 5:  Fractional change in the integral $\hat{\rho}$ of Eq. 2 under a
five per cent variation in $\rho(\omega)$ for $4m_{\pi}<\omega<6m_{\pi}$
parameterized as in Eq. 9, for a power law $R_{E}(t)=(t/t_{0})^{n}/n!$ with
$t_{0}=0.15$ fm, for a range $t_{min}<t<t_{max}$ plotted versus $t_{min}$ for
$t_{max}=1.2$, 1.44, 1.8, 2.4 and 5 fm. (a) $n=0$; (b) $n=2$; (c) $n=4$; (d)
$n=6$.

Fig. 6 shows similar results for the smearing kernel used for $a_{\mu}^{HVP}$
in its intermediate window guise. The figures are nearly identical to the ones
for power law weighting. The conclusion seems to be that a lattice calculation
of $G_{E}(t)$ with an accuracy of 2-3 per cent (in the continuum limit, of
course) over the range of 1-2 fm can distinguish a five per cent variation in
$\rho(\omega)$ in the rho region.

Figure 6:  The analog of Figs. 4 and 5 for the smearing kernel for
$a_{\mu}^{HVP}$. Panel (a) is the contribution of $4m_{\pi}<\omega<6m_{p}i$ to
the integral of Eq. 2. Panel (b) is the fractional change in the integral
$\hat{\rho}$ of Eq. 2 under a five per cent variation in $\rho(\omega)$ for
$4m_{\pi}<\omega<6m_{\pi}$ parameterized as in Eq. 9. Again, the curves are
$t_{min}<t<t_{max}$ plotted versus $t_{min}$ for $t_{max}=1.2$, 1.44, 1.8, 2.4
and 5 fm.

At this point I should stop and hope for an analysis by one of the lattice
groups using its own data sets. The idea I have presented is trivial, but it
also seems simple to implement. I think that $\rho(\omega)$ (and related
quantities) are interesting in and of themselves, and that trying to extract
features of $\rho(\omega)$ which have nothing to do with $a_{\mu}^{HVP}$ from
$G_{E}(t)$ could be a useful project. And, of course, identical weighting
techniques can connect other inclusive processes with Euclidean correlators.

###### Acknowledgements.

Questions at the end of a seminar by Finn Stokes at the MIT Virtual Lattice
Colloquium Series motivated this note. I would also like to thank Maarten
Golterman for correspondence and Chris Polly for encouragement. This material
is based upon work supported by the U.S. Department of Energy, Office of
Science, Office of High Energy Physics under Award Number DE-SC-0010005.

## References

  * (1) T. Aoyama, N. Asmussen, M. Benayoun, J. Bijnens, T. Blum, M. Bruno, I. Caprini, C. M. Carloni Calame, M. Cè and G. Colangelo, et al. Phys. Rept. 887, 1-166 (2020) doi:10.1016/j.physrep.2020.07.006 [arXiv:2006.04822 [hep-ph]].
  * (2) See the set of potential targets in M. T. Hansen, H. B. Meyer and D. Robaina, Phys. Rev. D 96, no.9, 094513 (2017) doi:10.1103/PhysRevD.96.094513 [arXiv:1704.08993 [hep-lat]].
  * (3) A. Anastasi et al. [KLOE-2], JHEP 03, 173 (2018) doi:10.1007/JHEP03(2018)173 [arXiv:1711.03085 [hep-ex]].
  * (4) G. Backus and F. Gilbert, Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 266, 123 (1970), ISSN 0080-4614.
  * (5) W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes: The Art of Scientific Computing (2007), Cambridge, UK: University Press.
  * (6) M. Hansen, A. Lupo and N. Tantalo, Phys. Rev. D 99, no.9, 094508 (2019) doi:10.1103/PhysRevD.99.094508 [arXiv:1903.06476 [hep-lat]].
  * (7) P. Gambino and S. Hashimoto, Phys. Rev. Lett. 125, no.3, 032001 (2020) doi:10.1103/PhysRevLett.125.032001 [arXiv:2005.13730 [hep-lat]].
  * (8) H. Fukaya, S. Hashimoto, T. Kaneko and H. Ohki, Phys. Rev. D 102, no.11, 114516 (2020) doi:10.1103/PhysRevD.102.114516 [arXiv:2010.01253 [hep-lat]].
  * (9) G. Bailas, S. Hashimoto and T. Ishikawa, PTEP 2020, no.4, 043B07 (2020) doi:10.1093/ptep/ptaa044 [arXiv:2001.11779 [hep-lat]].
  * (10) T. Blum et al. [RBC and UKQCD], Phys. Rev. Lett. 121, no.2, 022003 (2018) doi:10.1103/PhysRevLett.121.022003 [arXiv:1801.07224 [hep-lat]].
  * (11) M. Davier, L. Girlanda, A. Hocker and J. Stern, Phys. Rev. D 58, 096014 (1998) doi:10.1103/PhysRevD.58.096014 [arXiv:hep-ph/9802447 [hep-ph]].
  * (12) K. Maltman, Phys. Lett. B 440, 367 (1998) doi:10.1016/S0370-2693(98)01093-4 [arXiv:hep-ph/9901239 [hep-ph]].
  * (13) D. Bernecker and H. B. Meyer, Eur. Phys. J. A 47, 148 (2011) doi:10.1140/epja/i2011-11148-6 [arXiv:1107.4388 [hep-lat]].
  * (14) P. A. Zyla et al. [Particle Data Group], PTEP 2020, no.8, 083C01 (2020) doi:10.1093/ptep/ptaa104 See http://pdg.lbl.gov/current/xsect/ for the tabulation. An early description of the data set can be found in V. V. Ezhela, S. B. Lugovsky and O. V. Zenin, [arXiv:hep-ph/0312114 [hep-ph]].
  * (15) G. P. Lepage, “The Analysis Of Algorithms For Lattice Field Theory,” invited lectures at the 1989 TASI summer school, Boulder CO, June 4-30, 1989. CLNS-89-971.
  * (16) G. Parisi, Phys. Rept. 103, 203-211 (1984) doi:10.1016/0370-1573(84)90081-4
  * (17) M. G. Endres, D. B. Kaplan, J. W. Lee and A. N. Nicholson, Phys. Rev. Lett. 107, 201601 (2011) doi:10.1103/PhysRevLett.107.201601 [arXiv:1106.0073 [hep-lat]].
  * (18) C. T. H. Davies et al. [Fermilab Lattice, LATTICE-HPQCD and MILC], Phys. Rev. D 101, no.3, 034512 (2020) doi:10.1103/PhysRevD.101.034512 [arXiv:1902.04223 [hep-lat]].


# Visualizing and Understanding Contrastive Learning ††thanks: This research
received funding from the FWO (Grants G014718N, G0A4720N and 1SB5721N) and
from imec through the AAA project Trustworthy AI Methods (TAIM). F. Sammani,
B. Joukovsky and N. Deligiannis are with the Department of Electronics and
Informatics, Vrije Universiteit Brussel, Pleinlaan 2, B-1050 Brussels, Belgium
and also with imec, Kapeldreef 75, B-3001 Leuven, Belgium.

Fawaz Sammani, Boris Joukovsky, and Nikos Deligiannis

###### Abstract

Contrastive learning has revolutionized the field of computer vision, learning
rich representations from unlabeled data, which generalize well to diverse
vision tasks. Consequently, it has become increasingly important to explain
these approaches and understand their inner workings mechanisms. Given that
contrastive models are trained with interdependent and interacting inputs and
aim to learn invariance through data augmentation, the existing methods for
explaining single-image systems (e.g., image classification models) are
inadequate as they fail to account for these factors and typically assume
independent inputs. Additionally, there is a lack of evaluation metrics
designed to assess pairs of explanations, and no analytical studies have been
conducted to investigate the effectiveness of different techniques used to
explaining contrastive learning. In this work, we design visual explanation
methods that contribute towards understanding similarity learning tasks from
pairs of images. We further adapt existing metrics, used to evaluate visual
explanations of image classification systems, to suit pairs of explanations
and evaluate our proposed methods with these metrics. Finally, we present a
thorough analysis of visual explainability methods for contrastive learning,
establish their correlation with downstream tasks and demonstrate the
potential of our approaches to investigate their merits and drawbacks.

###### Index Terms:

Explaining Contrastive Learning, Explaining Similarity Networks

##  I Introduction

Contrastive learning is a technique used to train machine learning models to
recognize similarities between two inputs. One input is designated as a
positive sample and the other as a negative sample [1]. The aim of contrastive
learning is to maximize the similarity between positive inputs while
minimizing it between negative ones. Recently, contrastive learning has
revolutionized the field of model pretraining [2], [3],[4],[5], which involves
learning representations from large amounts of unlabeled data. Pretrained
models have demonstrated notable benefits over supervised learning by
effectively adapting to specific downstream tasks through fine-tuning, while
also learning more general, non-biased features. In recent years, contrastive
learning has become increasingly reliant on data augmentations to generate two
distinct views of the same image, which are used as positive samples. This
characteristic makes contrastive learning a fully unsupervised task, as it
does not require positive or negative mining [6],[7].

Figure 1: Contrastive models compare pairs of inputs and determine whether
they are similar or not. Our work involves the development of methods to
explain how these models make decisions and how they learn to recognize
similarities.

Explainable Artificial Intelligence (XAI) involves building techniques to
understand the underlying mechanisms behind the decision-making process of a
machine learning model. It is crucial for building trust and accountability in
AI systems as it enables users to understand and improve their models [8] and
identify any data biases present [9], [10]. This ensures fairness and
equability [11]. Local post-hoc attribution methods attribute what features
are responsible for a specific prediction, and have gained significant
popularity in recent years. They are roughly divided into three categories,
mostly aimed towards explaining image classification models: perturbation-
based [12], [13], activation-based [14, 15, 16] and gradient-based methods
[17, 18, 19].

While XAI has made significant progress in explaining image classification
models, its application to explaining contrastive learning models remains
scarce. BiLRP [20] explains similarity models involving a pair of images using
Layer-wise Relevance Propagation (LRP) [16], designing new rules which re-
distribute the similarity score through the layers back to the input. However,
it requires knowledge about which LRP-rule to be used at which layer, and new
rules have to be derived for different models including LSTMs [21], Batch
Normalization [22, 23], Multi-Head Attention [24, 25], and Transformers [26].
Another work, Similarity Decomposition [27] provides an explanation which
reflects the similarity of features, by computing the dot product between each
spatial location of one image and the average-pooled vector of the other
image. However, this method is restricted to models which contrast the
average-pooled features directly. This limitation hinders the explanation map
to provide any information about the invariance encoded in the model. As the
recent self-supervised contrastive models [28, 2, 29, 30] learn similarity
through invariance, this component cannot be overlooked. The projection head
connecting the average-pooled features and the final contrasted features is
responsible for encoding this invariance [31, 29, 28]. Our proposed methods
address this gap by considering features in both the similarity and invariance
space. Contrastive learning can also be used to learn a new model that is
tuned to be explainable for a given explanation method [32].

Given the increasing reliance on contrastive learning models, its importance
cannot be overlooked. In this regard, we discuss several reasons for why it is
crucial to explain contrastive models. First, this enables us to determine
whether the contrastive learning task is effective and whether the model is
able to learn meaningful concepts between the two images. Second, this allows
us to gain a better understanding of the process of data augmentation (e.g.,
which augment better helps the model). Third, many models are based on
learning from two inputs without being finetuned. Image Steganography [33],
face recognition with Siamese networks [1], anomaly detection in videos [34],
[35], CrossVIT [36] and MultiView Transformer [37] are popular examples. Our
techniques are directly applicable to these models as well. Fourth, since the
pretrained model is used to initialize weights in the fine-tuning process, any
data bias introduced in the pretrained model will also transfer to the fine-
tuned model. Lastly, pretrained contrastive models before finetuning are
usually used as feature extractors (e.g., CLIP [3]) for different computer
vision applications [38], [39]. All these reasons motivate us to visualize and
understand contrastive models.

Existing XAI methods designed for explaining models that learn from a single
image (e.g., image classification models) are not well-suited to explain
contrastive models. Unlike single-image systems, contrastive models processes
information differently as they learn from shared, dependent, and entangled
inputs that interact with each other. As a result, simply employing existing
XAI techniques, which typically assume independent inputs, would lead to
inaccurate explanations. Moreover, recent contrastive models [2, 3, 4, 5] aim
towards learning invariance from data augmentations. That is, they should be
invariant to input transformations. Consequently, invariance should also be
considered as a crucial aspect in the design of XAI methods for contrastive
learning. Our work thus aims at developing techniques to explain the
underlying mechanisms of contrastive learning, including understanding how
they learn similarity and the rationale behind the decisions they make.
Likewise, most evaluation metrics in the literature of explainability [40, 41,
42, 43] are designed for single image systems. In the case of contrastive
learning, the model takes a pair of images as input and outputs a similarity
score. As a result, two explanation maps are obtained rather than a single
one. The need for metrics to evaluate pairs of explanations is currently not
fulfilled and therefore, such metrics are necessary. Hence, we additionally
adapt existing metrics used to evaluate explanations of single-image models to
a simultaneous and conditional setting, which accommodates the evaluation of
pairs of explanations. Finally, there exists no analytical study on XAI
techniques for contrastive learning. As a result, it is unclear which specific
method or family of methods is optimal and most effective for specific
purposes and use-cases. Our study is designed to be all-encompassing. It
involves an examination of contrastive models and an overview of the
effectiveness of diverse methods and their connection to downstream tasks.

In summary, the contributions of this work are as follows:

  * •

We propose post-hoc visual explanation techniques for explaining contrastive
vision models that learn from a pair of images.

  * •

We extend existing evaluation metrics designed for supervised learning tasks
(especially, image classification) to suit pair of explanations and evaluate
the proposed methods with these metrics.

  * •

We offer a comprehensive study and analysis of XAI techniques for contrastive
learning. This includes a thorough exploration of contrastive models, a
summary of the performance of various methods, their relationship to
downstream tasks as well as some explorations and insights. This equips users
with a comprehensive guide to understanding contrastive models.

The remainder of the paper is as follows: Section II reviews background
information and Section III presents our explainability techniques. Section IV
elaborates on the designed metrics to quantitatively evaluate our
explanations. In Section V, we perform both quantitative and qualitative
analyses, present a summary of the results, and explore applications for our
methods. Additionally, we examine the relationship between explainability and
downstream task performance. Finally, we offer further insights and
explorations in Section VI.

##  II Background on related XAI techniques

In this Section, we discuss earlier works in the literature of XAI for image
classification systems, which we will re-visit in our work.

###  II-A Saliency-based Methods

Saliency maps [17] are methods for visualizing the importance of individual
pixels of an image $I$ to any given neuron $c$. A saliency map is defined as:
$I\circ\frac{\partial c}{\partial I}$, where $\circ$ denotes the Hadamard
product. A high gradient for a pixel in $I$ signifies that alterations of that
pixel would increase the value of $c$. $c$ is usually defined as the neuron
representing an output class. Saliency maps have been found to be noisy due to
shattering gradients and saturation, resulting in the development of new
techniques. Smooth-Grad [18] removes noise from saliency maps by adding random
Gaussian noise to the input image for several steps, producing local samples
around the sample of interest and eliminating local variations in non-smoothly
varying partial derivatives. Integrated Gradients [19] addresses the issue of
gradient saturation which arrises when models are trained until saturation.
This is achieved by desaturating the input image. Guided Backpropagation [44]
removes noise from saliency maps by backpropagating only the positive ReLU
gradients. As a result, pixels that negatively correlate with the output score
are not shown in the saliency map.

###  II-B Occlusion Analysis

Occlusion analysis [12] is a fundamental and straightforward approach used to
understand the significance of different regions in an image $I$ on a neuron
$c$. The method involves the use of a black or gray mask to obscure specific
regions of the image, and the resultant drop in $c$ relative to the baseline
is directly measured and recorded. The reduction in score provides evidence
that the obscured region is vital for prediction. The mask is applied using a
sliding window approach to cover all areas of the image. This perturbation-
based technique does not require access to model structure, weights, or
gradients, having an advantage over other methods [16, 45, 17, 15].

###  II-C Class Activation Maps (CAM)

CAM [14], later generalized to Grad-CAM [15], is a visualization technique
used to examine the activations of a specific convolutional layer in a neural
network. Specifically, Grad-CAM considers an activation map
$A\in\mathbb{R}^{K\times w\times h}$ of depth $K$ and spatial dimensions
$w\times h$, and defines the importance map as
$\sum_{k}[\alpha_{k}]_{+}A^{k}$. The notation $[\alpha_{k}]_{+}$ means only
the positive values of $\alpha_{k}$ are retained. The coefficients
$\alpha_{k}=\frac{1}{K}\sum_{k=1}^{K}\frac{\partial c}{\partial A^{k}}$
correspond to the spatially-averaged gradients of the model’s output class $c$
with respect to (w.r.t.) $A^{k}$. Several variants of CAM have been proposed,
primarily focused on developing new methods for weighting the activation maps,
including Grad-CAM++ [41], Score-CAM [43], and Relevance-CAM [46]. However,
the underlying principle remains the same. More recent extensions of CAM such
as HighResCAM [47] and LayerCAM [48] proposed to directly weight the
activation map $A^{k}$ point-wise with the non-averaged gradients, showing
that this approach leads to better coverage of the salient objects in the
importance map for earlier layers.

##  III Proposed Methods

This section describes our proposed techniques for explaining contrastive
vision models. We note that the methods are architecture-independent, that is,
they are applicable to both CNNs and Transformers. Our code is publicly
available111https://github.com/annonymoussubmission/explain-cl. Moreover, we
provide a web-demo222https://huggingface.co/spaces/AnnonSubmission/xai-cl to
facilitate users in effectively visualizing and understanding different
contrastive learning models.

###  III-A Saliency-based Methods

####  III-A1 Baseline

We start by outlining a straightforward extension of Input $\times$ Grad [17]
and Smooth-Grad [18] to two-image models, which can serve as our baselines. A
simple approach to extend these methods to two-image models would be to apply
them separately to each image. In the case of Input $\times$ Grad, the
produced explanations maps are: $I_{1}\circ\frac{\partial s}{\partial
A_{1}},I_{2}\circ\frac{\partial s}{\partial A_{2}}\in\mathbb{R}^{w\times h}$.
However, given that contrastive models utilize different and random data
augmentations to generate two views of the same image, this simple extension
ignores how the change in augmentation of each pixel impacts the output score.
We now describe our proposed method, coined Average Transforms.

####  III-A2 Averaged Transforms

Since numerous contrastive models are trained to be invariant to input
transformations, it is essential to incorporate the invariance component into
the the explainability algorithm. However, before doing so, we aim to validate
the contrastive models’ learning objective qualitatively. In the appendix, we
present an experiment that demonstrates how invariance progresses for both
contrastive models and image classification models. The experiment shows that
as the layer depth increases, the image saturation and colors de-emphasize and
decrease. This indicates that the latent representations learned by
contrastive models are more invariant to input transformations than those of
image classification models.

The objective of Averaged Transforms is to examine the impact of data
augmentations and invariance on the similarity result. During the training
process of contrastive models, data augmentation is applied to input samples
to create diverse positive and negative pairs. A crucial aspect of data
augmentation is the random level of strength applied to each augment, which
affects the magnitude of the transformation. As an example, the widely used
"color jitter" augment applies random brightness, contrast, saturation, and
hue levels to distort image color. In the case of "Gaussian blur", random
values within a range of standard deviations (strengths) are selected and used
to create Gaussian kernels which blur the image. Consequently, all different
strengths must be considered in order to fully understand the effect of a
specific data augmentation technique on the model, since different strength
values can potentially lead to different interpretations of the prediction.

The purpose of this experiment is to determine the effects that a particular
data transformation $T$ has on two images. We start by decomposing a
transformation $T$ into $[t^{1},t^{2},\dots,t^{Z}]$, where $t^{z}$ represent a
specific strength of the transformation $T$ and $z$ gradually varies from the
lowest to the highest strength possible that does not fully distort the image
and cause a distribution shift. Let $I_{1}$ and $I_{2}$ be two input images,
and let $t^{z}(I_{2})$ denote the $z$th transformation of $I_{2}$. We then
compute the similarity score between $I_{1}$ and $t^{z}(I_{2})$ for all values
of $z$ from $1$ to $Z$:
$S_{z}=\operatorname{sim}\left(I_{1},t^{z}\left(I_{2}\right)\right),\quad\forall
z=1,\ldots,Z$, where $\mathrm{sim}$ is the cosine similarity function. We can
then compute the gradient of the similarity score with respect to each of the
input images $I_{1}$ and $I_{2}$:
$\nabla_{I_{1}}S_{z},\quad\nabla_{I_{2}}S_{z},\quad\forall z=1,\ldots,Z$.
Finally, we average the gradients across different strengths for each image to
obtain a final saliency map:
$\mathcal{S}_{I_{1}}=I_{1}\circ\frac{1}{Z}\sum_{z=1}^{Z}\nabla_{I_{1}}S_{z},\quad\mathcal{S}_{I_{2}}=I_{2}\circ\frac{1}{Z}\sum_{z=1}^{Z}\nabla_{I_{2}}S_{z}$.
$\mathcal{S}_{I_{1}}$ and $\mathcal{S}_{I_{2}}$ represent the saliency maps
for $I_{1}$ and $I_{2}$, respectively.

Practically, we can implement the aforementioned process by applying $t^{Z}$
to $I_{2}$ directly, such that $I_{Z}=t^{Z}(I_{2})$, and then gradually
transform $I_{1}$ to $I_{2}$ by applying $o^{z}=\rho I_{1}+(1-\rho)I_{Z}$,
where $\rho$ is the step size that gradually decreases from 1 to 0 by a factor
of 0.1. We may (optionally) choose to remove noise from the saliency map by
using noise removal techniques such as Smooth-Grad [18] or Guided
Backpropagation [44]. In Section V, we experiment and ablate all possible
options of our method (w/ and w/o Smooth-Grad, w/ and w/o Guided
Backpropagation). Finally, we may also blur the pixel-wise saliency map output
with a Gaussian Kernel for visually appealing results (not performed when
evaluating the methods).

###  III-B Perturbation-based Methods

####  III-B1 Baseline

We start by describing a straightforward extension to two-image models coined
Conditional Occlusion: given the input images $I_{1}$ and $I_{2}$ and their
similarity score $s$, we successively zero-out parts of $I_{1}$ according to a
square sliding-window, while $I_{2}$ remains unchanged. Let
$I_{1}^{\operatorname{p}}$ denote the perturbed sample. For each window
location, we record the output of the constrastive model fed with
$(I_{1}^{\operatorname{p}},I_{2})$: a larger drop means that the occluded
region is more important for deeming these two images as similar, conditioned
on $I_{2}$ being fully observed. We plot an importance map for $I_{1}$ by
subtracting the scores of the perturbed images from the baseline $s$. The
process can be carried out for the second image by swapping $I_{1}$ and
$I_{2}$. For all quantitative and qualitative results, we use a perturbation
mask size of 64 $\times$ 64 and a stride of 8. We now describe the problems of
the baseline method and introduce our proposed method, coined Pairwise
Occlusion.

Figure 2: An example of computing conditional occlusion of the second image
conditioned on the first image. The contextual information surrounding the
dropped region (indicated by a black box) in the second image is sufficient to
produce a high similarity prediction with the fully-observed first image.
Figure 3: A 2D plot representing the interaction of activation vectors for
augmented image pairs. The $x$-axis represents feature values from the first
image’s activation vector and the $y$-axis represents those of the second
image. The color bar indicates the magnitude of their interaction (their
element-wise multiplication).

####  III-B2 Pairwise Occlusion

A problem with Conditional Occlusion is that the similarity score may not drop
when perturbing a region because the context around the dropped region being
present in both images may be sufficient for a high similarity prediction.
This problem is illustrated in Fig. 2. Ideally, we would like a region to be
considered significant only if its prediction cannot be inferred from its
context. We propose an approach which considers simultaneous perturbations of
the two images while mitigating two issues that may arise with this approach:
(1) testing all the combinations of regions between the two images would
result in a quadratic increase of required forward passes, and (2) less-
significant regions might be given too much importance. We alleviate the first
risk by randomly choosing different occlusions sizes, locations and aspect
ratios for each image, which asymptotically approximates all possible
combinations of perturbations. The second risk may occur when a drop in
similarity is caused by removing either of the perturbed regions. As an
example, consider two pictures of a dog on neutral backgrounds, the first is
perturbed with an occlusion of the dog’s face, and the second contains an
occlusion in the background. In this case, the similarity drop is mainly
caused by the occlusion of the dog’s face, and therefore must be given more
importance. To alleviate this issue, we perform the following: after obtaining
the similarity scores of all perturbed pairs, we perform a softmax operation
on the scores to obtain a distribution of importances among all different
simultaneous perturbations. Then, for each pair, we assign the softmax score
to the perturbed region of the image whose high-level features have the lowest
$\ell_{2}$ norm, indicating that this region likely contained highly activated
features. For all quantitative and qualitative results, we sampled $100$
different perturbation masks with different occlusion sizes, locations and
aspect ratios. The scale of the mask ranges from $10\%-30\%$ of the image.

###  III-C Activation Visualization Methods

####  III-C1 Baseline

We start by describing a straightforward extension of Grad-CAM [15] to two-
image models, which serves as our baseline. We consider two activation maps
$A_{1},A_{2}\in\mathbb{R}^{K\times w\times h}$ obtained respectively by
feeding the first and second input images to the contrastive model. Also, let
$s$ denote the predicted similarity score. Next, we compute the gradient maps
$\frac{\partial s}{\partial A_{1}},\frac{\partial s}{\partial
A_{2}}\in\mathbb{R}^{K\times w\times h}$. By this approach, the importance
maps $E_{1}$ and $E_{2}$ are obtained separately by weighing the activation
maps with their rectified gradients $W^{k}_{1,2}=\frac{\partial s}{\partial
A^{k}_{1,2}}$. Eq. (1) thus shows the expressions of $E_{1}$ and $E_{2}$, and
$\odot$ denotes the Hadamard product. We refer to this model as Baseline-Grad-
CAM in the further experiments of Section V.

$E_{1,2}=\textstyle\sum_{k}[W_{1,2}^{k}]_{+}\odot A_{1,2}^{k}.$ (1)

Notably, Baseline-Grad-CAM does not incorporate any joint term in the
formulation, which is crucial for reflecting the similarity score derived from
two input images. Baseline-Grad-CAM disentangles this joint process and
explains each image independently of the other. As it treats each image
separately, it misses out on capturing the interdependence between images
learned by the contrastive model.

####  III-C2 Interaction-CAM

We now describe our proposed method. Since the similarity score is computed by
considering two input images, the interaction of their corresponding
activations and gradients should be reflected by the produced explainability
maps. Unlike existing methods that rely solely on the activations and
gradients of individual images, our approach considers the joint activations
and gradients of both images to produce explainability maps that highlight the
common features and how they influence each other. Specifically, Interaction-
CAM further augments the baseline method by introducing a joint activation
term $J$ and a gradient interaction term $G$ to capture common activations and
gradients between $A_{1}$ and $A_{2}$.

The joint activation $J=[j^{1},\dots,j^{K}]$ is defined by the element-wise
product of two $K$-dimensional vectors $J_{1}\odot J_{2}$, where $j_{1,2}^{k}$
results from a spatial reduction operation of the corresponding feature map
$A_{1,2}^{k}$. Therefore, each element of $J$ indicates whether both images
simultaneously activate similar features. We provide a visual example in Fig.
3. When a feature value from the activation vector is high for the first image
but low for the second image (as shown by the data points in the bottom-right
quadrant of both plots), the resulting interaction value is notably low
(indicated by the purple hue). Conversely, the inverse scenario holds true, as
shown by the data points in the top-left quadrant of the plot. This allows us
to visualize important features which are jointly active in both images, and
suppress features that are active in one image but low in the other. Later in
Section V, we experiment and ablate three different reduction operations:
average pooling, max pooling and self-attention with the query set as the
average-pool of $A^{k}_{1,2}$, and the keys and values as $A^{k}_{1,2}$.

We define the global gradient interaction matrix $G\in\mathbb{R}^{K\times K}$
(a cross-correlation matrix of the gradients) by computing inner products of
gradients across the spatial dimension, that is,
$G^{kl}=\operatorname{vec}(\frac{\partial s}{\partial
A_{1}^{k}})^{T}\operatorname{vec}(\frac{\partial s}{\partial A_{2}^{l}})$. The
inner product thus encodes the similarity of the channel gradients a given
spatial locations of the first and second image. It provides a measure to
which extent the two gradient vectors are in the same direction. Then, for
each feature map and for each input image, we retrieve a unique coefficient
$g_{1,2}^{k}$ via a reduction operator on $G$ against the other image:
$g_{1}^{k}=\max_{l^{\prime}}(G^{kl^{\prime}})$ and
$g_{2}^{k}=\max_{k^{\prime}}(G^{k^{\prime}l})$. Note that the maximum value
defines a high similarity between the gradients of all pixels from both images
at a specific dimension. In simple terms, the gradient interaction term
encodes the interaction dynamics of gradients, specifically how the channel
gradients of the two images affected each other.

The two final importance maps are obtained by aggregating all feature maps,
weighted by the associated joint activation and gradient interaction terms, as
shown in Eq. (2):

$E_{1,2}=\textstyle\sum_{k}[j^{k}\cdot g_{1,2}^{k}]_{+}\cdot A_{1,2}^{k}.$ (2)

TABLE I: Evaluation scores on perturbation, activation and saliency-based
methods for SimCLRv2 (1$\times$) and SimCLRv2 (2$\times$). SI: Simultaneous
Insertion, SD: Simultaneous Deletion, CI: Conditional Insertion, CD:
Conditional Deletion, SAD: Simultaneous Average Drop, CAD: Conditional Average
Drop. AR stands for the type we use for Activation Reduction and GI indicates
whether or not we use Gradient Interaction. We ablate all possible
combinations.

| SimCLRv2 (1$\times$) | SimCLRv2 (2$\times$)
---|---|---
| AR | GI | SI $\uparrow$ | SD $\downarrow$ | CI $\uparrow$ | CD $\downarrow$ | SAD $\downarrow$ | CAD $\downarrow$ | SI $\uparrow$ | SD $\downarrow$ | CI $\uparrow$ | CD $\downarrow$ | SAD $\downarrow$ | CAD $\downarrow$
Conditional Occlusion |  |  | 0.781 | 0.748 | 0.611 | 0.217 | 0.155 | 0.295 | 0.801 | 0.716 | 0.647 | 0.212 | 0.161 | 0.285
Pairwise Occlusion |  |  | 0.757 | 0.632 | 0.602 | 0.278 | 0.371 | 0.612 | 0.773 | 0.640 | 0.643 | 0.274 | 0.347 | 0.564
Deep Similarity [27] |  |  | 0.794 | 0.770 | 0.631 | 0.229 | 0.137 | 0.297 | 0.806 | 0.742 | 0.663 | 0.213 | 0.166 | 0.302
Baseline-Grad-CAM |  |  | 0.781 | 0.764 | 0.623 | 0.236 | 0.135 | 0.287 | 0.798 | 0.737 | 0.661 | 0.211 | 0.166 | 0.298
Int-CAM | Mean | ✗ | 0.778 | 0.739 | 0.613 | 0.237 | 0.157 | 0.341 | 0.794 | 0.721 | 0.651 | 0.212 | 0.200 | 0.367
Int-CAM | Mean | $\checkmark$ | 0.789 | 0.765 | 0.616 | 0.228 | 0.151 | 0.332 | 0.798 | 0.743 | 0.650 | 0.210 | 0.187 | 0.343
Int-CAM | Max | ✗ | 0.786 | 0.761 | 0.625 | 0.233 | 0.157 | 0.411 | 0.804 | 0.730 | 0.661 | 0.211 | 0.188 | 0.435
Int-CAM | Max | $\checkmark$ | 0.795 | 0.778 | 0.627 | 0.228 | 0.149 | 0.414 | 0.805 | 0.749 | 0.658 | 0.212 | 0.174 | 0.421
Int-CAM | Attn | ✗ | 0.785 | 0.773 | 0.617 | 0.239 | 0.149 | 0.567 | 0.802 | 0.739 | 0.652 | 0.220 | 0.166 | 0.602
Int-CAM | Attn | $\checkmark$ | 0.793 | 0.782 | 0.620 | 0.236 | 0.127 | 0.579 | 0.803 | 0.751 | 0.651 | 0.223 | 0.151 | 0.598
Input x Gradient |  |  | 0.895 | 0.895 | 0.558 | 0.102 | 0.048 | 0.931 | 0.888 | 0.839 | 0.582 | 0.102 | 0.091 | 0.937
Input x Gradient (G) |  |  | 0.899 | 0.906 | 0.678 | 0.075 | 0.062 | 0.813 | 0.904 | 0.867 | 0.726 | 0.069 | 0.082 | 0.798
Smooth-Grad |  |  | 0.894 | 0.913 | 0.606 | 0.072 | 0.073 | 0.853 | 0.884 | 0.854 | 0.639 | 0.069 | 0.119 | 0.832
Smooth-Grad (G) |  |  | 0.882 | 0.897 | 0.658 | 0.101 | 0.078 | 0.748 | 0.883 | 0.847 | 0.693 | 0.112 | 0.106 | 0.735
Avg. Transforms |  |  | 0.898 | 0.904 | 0.557 | 0.100 | 0.043 | 0.929 | 0.892 | 0.849 | 0.581 | 0.100 | 0.080 | 0.935
Avg. Transforms (G) |  |  | 0.901 | 0.915 | 0.680 | 0.075 | 0.050 | 0.807 | 0.907 | 0.877 | 0.728 | 0.069 | 0.069 | 0.795
Avg. Transforms (N) |  |  | 0.855 | 0.843 | 0.569 | 0.089 | 0.115 | 0.911 | 0.843 | 0.779 | 0.604 | 0.092 | 0.154 | 0.912

TABLE II: Evaluation scores on perturbation, activation and saliency-based
methods for Barlow Twins

| AR | GI | SI $\uparrow$ | SD $\downarrow$ | CI $\uparrow$ | CD $\downarrow$ | SAD $\downarrow$ | CAD $\downarrow$
---|---|---|---|---|---|---|---|---
Conditional Occlusion |  |  | 0.655 | 0.421 | 0.540 | 0.180 | 0.517 | 0.549
Pairwise Occlusion |  |  | 0.652 | 0.497 | 0.537 | 0.188 | 0.549 | 0.682
Deep Similarity [27] |  |  | 0.706 | 0.484 | 0.578 | 0.177 | 0.362 | 0.415
Baseline-Grad-CAM |  |  | 0.688 | 0.489 | 0.571 | 0.180 | 0.386 | 0.433
Int-CAM | Mean | ✗ | 0.680 | 0.473 | 0.567 | 0.169 | 0.416 | 0.447
Int-CAM | Mean | $\checkmark$ | 0.693 | 0.490 | 0.571 | 0.169 | 0.381 | 0.427
Int-CAM | Max | ✗ | 0.693 | 0.479 | 0.574 | 0.170 | 0.437 | 0.531
Int-CAM | Max | $\checkmark$ | 0.701 | 0.490 | 0.577 | 0.171 | 0.398 | 0.502
Int-CAM | Attn | ✗ | 0.693 | 0.494 | 0.567 | 0.172 | 0.458 | 0.694
Int-CAM | Attn | $\checkmark$ | 0.699 | 0.498 | 0.571 | 0.174 | 0.451 | 0.679
Input x Gradient |  |  | 0.718 | 0.577 | 0.495 | 0.119 | 0.472 | 0.981
Input x Gradient (G) |  |  | 0.825 | 0.599 | 0.703 | 0.069 | 0.471 | 0.938
Smooth-Grad |  |  | 0.749 | 0.572 | 0.540 | 0.089 | 0.460 | 0.972
Smooth-Grad (G) |  |  | 0.832 | 0.606 | 0.713 | 0.077 | 0.456 | 0.928
Avg. Transforms |  |  | 0.724 | 0.585 | 0.494 | 0.117 | 0.437 | 0.982
Avg. Transforms (G) |  |  | 0.829 | 0.617 | 0.705 | 0.070 | 0.453 | 0.937
Avg. Transforms (N) |  |  | 0.714 | 0.548 | 0.504 | 0.111 | 0.505 | 0.976

TABLE III: Evaluation scores on perturbation, activation and saliency-based
methods for SimSiam

| AR | GI | SI $\uparrow$ | SD $\downarrow$ | CI $\uparrow$ | CD $\downarrow$ | SAD $\downarrow$ | CAD $\downarrow$
---|---|---|---|---|---|---|---|---
Conditional Occlusion |  |  | 0.809 | 0.549 | 0.640 | 0.279 | 0.286 | 0.380
Pairwise Occlusion |  |  | 0.723 | 0.459 | 0.600 | 0.357 | 0.798 | 0.593
Deep Similarity [27] |  |  | 0.827 | 0.606 | 0.663 | 0.274 | 0.221 | 0.312
Baseline-Grad-CAM |  |  | 0.816 | 0.585 | 0.659 | 0.279 | 0.243 | 0.337
Int-CAM | Mean | ✗ | 0.815 | 0.565 | 0.654 | 0.268 | 0.280 | 0.382
Int-CAM | Mean | $\checkmark$ | 0.828 | 0.606 | 0.655 | 0.263 | 0.236 | 0.351
Int-CAM | Max | ✗ | 0.822 | 0.575 | 0.662 | 0.269 | 0.294 | 0.433
Int-CAM | Max | $\checkmark$ | 0.827 | 0.613 | 0.662 | 0.266 | 0.252 | 0.411
Int-CAM | Attn | ✗ | 0.816 | 0.566 | 0.654 | 0.267 | 0.280 | 0.385
Int-CAM | Attn | $\checkmark$ | 0.828 | 0.606 | 0.656 | 0.263 | 0.237 | 0.355
Input x Gradient |  |  | 0.885 | 0.780 | 0.595 | 0.192 | 0.262 | 0.966
Input x Gradient (G) |  |  | 0.925 | 0.820 | 0.809 | 0.124 | 0.242 | 0.876
Smooth-Grad |  |  | 0.897 | 0.787 | 0.639 | 0.154 | 0.247 | 0.949
Smooth-Grad (G) |  |  | 0.924 | 0.819 | 0.812 | 0.132 | 0.232 | 0.867
Avg. Transforms |  |  | 0.888 | 0.786 | 0.595 | 0.189 | 0.234 | 0.965
Avg. Transforms (G) |  |  | 0.926 | 0.830 | 0.810 | 0.126 | 0.227 | 0.874
Avg. Transforms (N) |  |  | 0.875 | 0.747 | 0.602 | 0.178 | 0.300 | 0.963

TABLE IV: Evaluation scores on perturbation, activation and saliency-based
methods for MoCov3 ViT-Base

| AR | GI | SI $\uparrow$ | SD $\downarrow$ | CI $\uparrow$ | CD $\downarrow$ | SAD $\downarrow$ | CAD $\downarrow$
---|---|---|---|---|---|---|---|---
Conditional Occlusion |  |  | 0.809 | 0.622 | 0.714 | 0.280 | 0.483 | 0.533
Pairwise Occlusion |  |  | 0.807 | 0.685 | 0.706 | 0.435 | 0.378 | 0.439
Deep Similarity [27] |  |  | 0.782 | 0.718 | 0.627 | 0.414 | 0.260 | 0.270
Baseline-Grad-CAM |  |  | 0.805 | 0.667 | 0.690 | 0.376 | 0.393 | 0.603
Int-CAM | Mean | ✗ | 0.795 | 0.653 | 0.670 | 0.389 | 0.476 | 0.800
Int-CAM | Mean | $\checkmark$ | 0.787 | 0.716 | 0.599 | 0.478 | 0.314 | 0.410
Int-CAM | Max | ✗ | 0.799 | 0.666 | 0.683 | 0.382 | 0.422 | 0.670
Int-CAM | Max | $\checkmark$ | 0.787 | 0.714 | 0.606 | 0.467 | 0.296 | 0.350
Int-CAM | Attn | ✗ | 0.794 | 0.656 | 0.670 | 0.388 | 0.467 | 0.780
Int-CAM | Attn | $\checkmark$ | 0.787 | 0.715 | 0.606 | 0.468 | 0.314 | 0.422
Input x Gradient |  |  | 0.902 | 0.760 | 0.746 | 0.363 | 0.280 | 0.904
Input x Gradient (G) |  |  | 0.901 | 0.749 | 0.752 | 0.341 | 0.331 | 0.889
Smooth-Grad |  |  | 0.909 | 0.750 | 0.762 | 0.309 | 0.284 | 0.833
Smooth-Grad (G) |  |  | 0.911 | 0.749 | 0.769 | 0.295 | 0.301 | 0.817
Avg. Transforms |  |  | 0.904 | 0.763 | 0.747 | 0.359 | 0.259 | 0.903
Avg. Transforms (G) |  |  | 0.902 | 0.755 | 0.752 | 0.339 | 0.307 | 0.887
Avg. Transforms (N) |  |  | 0.904 | 0.733 | 0.748 | 0.348 | 0.345 | 0.886

TABLE V: Maximum Sensitivity evaluation on saliency-based methods. Lower is
better.

| SimCLR(1X) | SimCLR(2X) | SimSiam | MoCov3
---|---|---|---|---
Input x Gradient | 1.386 | 1.799 | 2.306 | 3.121
Input x Gradient (G) | 1.402 | 2.178 | 3.168 | 3.231
Smooth-Grad | 1.208 | 1.257 | 2.534 | 3.317
Smooth-Grad (G) | 1.652 | 1.780 | 1.950 | 2.700
Avg. Transforms | 1.581 | 1.910 | 1.876 | 2.772
Avg. Transforms (G) | 1.448 | 2.301 | 1.711 | 2.270
Avg. Transforms (N) | 0.923 | 0.976 | 1.907 | 2.186

##  IV Evaluation Metrics

In this section, we first re-visit metrics commonly used to evaluate single
explanations of image classification systems, and then show how we build upon
these metrics and adapt them to suit the evaluation of pairs of explanations.

###  IV-A Metrics for Evaluating Explanations of Image Classification Systems

We consider the following metrics: insertion and deletion [40], average drop
[41], maximum sensitivity [42] and sanity checks [49]. In the insertion and
deletion metric, we start from a baseline (a highly blurred image for
insertion and the full image for deletion) and gradually add (or remove) image
pixels starting from the pixels deemed most important by the explanation
algorithm. The output score curve is then plotted as a function of the pixels
being added or removed and the Area Under the Curve (AUC) is computed. A sharp
increase (or decrease) in this curve generally reflects that the explanation
is faithful and complete. The Average Drop [41] was developed to measure how
the model’s confidence drops when providing only the explanation map regions
as input, with lower scores being better. The Maximum sensitivity [42]
measures to which extent the explanation changes when the input is slightly
perturbed with noise. Sanity Check [49] randomizes each layer of the model,
starting from the last layer up to the first layer. At each step of
randomization, the explanation map is visualized to ensure that the
explainability methods do not act as edge detectors, and that they rely on the
model parameters for a prediction. Lastly, we refrain from evaluating the
explanations of contrastive models with localization since this is only valid
for evaluating explanations of models trained for image classification. As a
result, the model must be able to localize the object adequately in order to
classify the object present in the image. However, this is not the case for
contrastive learning paradigms where the objective is to learn similarity. In
fact, it is very rare to see the explanation of a similarity model covering
the full object from both images. Instead, we expect to see associations
between the two images that convey meaningful and relevant concepts about the
object.

###  IV-B Adaptation to Pairs of Explanations

Unlike image classification systems which yield a single explanation, the
output of our explanation algorithms produce pairs of explanations. We
therefore expand the existing metrics discussed in Section IV-A to suit the
evaluation of pairs of explanations. We perform this under two settings: a
simultaneous and a conditional setting, each of which is described below.

####  IV-B1 Simultaneous Evaluation

In this setting, we simultaneously evaluate both explanation maps. In the case
of insertion and deletion, we add or remove $L$ pixels from both the first and
second image together at each step of the iteration process, starting from the
most important pixels in the explanation map. We then measure the increase or
drop in the similarity score between the two altered images with $L$ added or
removed pixels each (in our experiments we set $L$ to 224). Applying this
setting on the insertion and deletion metrics yields simultaneous insertion
(SI) and simultaneous deletion (SD), respectively. Similarly, applying it on
the average drop metric yields simultaneous average drop (SAD). However, this
approach assumes a correspondence between the two explanation maps, meaning
that a point in the first image must correspond to a point in the second
image. In that case, the two explanation maps are dependant on each other and
treated as a whole. This is made evident by observing a gradual, less sharp
increase or decrease in the output score curve as pixels are added or removed.
This mainly indicates that the similarity can still be computed from the
context around the two images. This motivates us to propose another variant of
evaluation, namely Conditional Evaluation.

####  IV-B2 Conditional Evaluation

In this setting, we evaluate one explanation map conditioned on the other
image being fully observed. In the case of insertion and deletion, we add or
remove $L$ pixels from the first image while holding the other image constant.
At each step, we measure the increase or drop in the similarity score between
the altered image with $L$ added or removed pixels, and the original unaltered
second image. The operation is done for each of the two images and the results
are averaged. Applying this setting on the insertion and deletion metrics
yields conditional insertion (CI) and conditional deletion (CD), respectively.
Similarly, applying it on the average drop metric yields conditional average
drop (CAD). The main drawback of this setting is that it doubles the
evaluation time for each image in the evaluation set.

In what follows, we assess all our techniques by subjecting them to
simultaneous and conditional evaluations across all six metrics.

Figure 4: Qualitative examples of Averaged Transforms compared with Smooth-
Grad. On the left, we show the augmentation dissection process of gradually
varying the transformation strength of the first image until it reaches the
second image. Figure 5: Qualitative examples of Averaged Transforms compared
with Smooth-Grad, for grayscale and color jitter transformations. Figure 6:
Qualitative examples of Averaged Transforms compared to InputXGrad and Smooth-
Grad, for a 90 degree rotation transformation. On top, we show the
augmentation dissection process. Figure 7: 6 qualitative examples of
Perturbation-based methods. Our methods are not limited to explaining
similarity on augmented pairs. We can also apply them to explain similarity of
different images from the same class. The bottom example in the first column
and the top and bottom examples in the second column present qualitative
results on different images of the same class. The other qualitative examples
(the top example in the first column and the two examples in the last column)
are for augments of an image. Red signifies more important regions while blue
signifies less important regions. Figure 8: Qualitative examples of the
proposed Interaction-CAM compared to Baseline-Grad-CAM.

Figure 9: Examples of meaningful concepts learned by the model and detected by
Interaction-CAM, where the same concepts related to a particular object are
learned. The positions of these meaningful concepts are aligned between the
two images (e.g., the wolf’s eye, the flamingo wing

).

Figure 10: Examples of meaningful concepts learned by the model and detected
by Interaction-CAM, where different semantic concepts associated to the same
object are learned (e.g., different parts of the dog’s and cat’s faces).
Figure 11: Interaction-CAM applied on two different images from the same
class. Figure 12: A visualization of perturbation and activation-based methods
applied on the same image pairs. DeepSim is the technique from [27]

##  V Experiments

In this section, we conduct a qualitative and a quantitative analysis (using
the six proposed metrics) of the proposed explanation methods.

###  V-A Experimental Setup

We randomly choose a subset of $2000$ images from the ImageNet ILSVRC-2012
[50] validation set. We experiment with five different models: SimCLRv2
1$\times$, SimCLRv2 2$\times$ [2], Barlow Twins [30], SimSiam [29] and MoCov3
[31]. All methods are evaluated using the cosine similarity measure. SimCLRv2
is a contrastive model which utilizes data augmentation to obtain two views of
the same image and learns a similarity task; namely, augments of the same
image should similar, while augments from different images within the same
batch of samples should be dissimilar. It consists of a vision backbone (e.g.,
CNN [51] or Transformer [24]) followed by a projection head that learns
invariance. SimCLRv2 2$\times$ doubles the width of the CNN, and has shown to
perform much better than SimCLRv2 1$\times$. SimSiam [29] is a contrastive
model that eliminates the need of negative pairs through methodologies such as
the stop-gradient mechanism, a predictor network, and momentum-based encoding.
MoCov3 is another contrastive model that is similar to SimCLRv2, but uses a
Vision Transformer as the backbone and uses separate models to encode each
view of the image: the query encoder and the key encoder. The query encoder
has an extra prediction head, and the key encoder is updated by a moving
average of the query encoder. Barlow Twins is a self-supervised model which is
inspired by Principal Component Analysis. Barlow Twins estimates a covariance
matrix which is trained to be as close to the identity matrix as possible. As
in SimCLR, data augmentation is used to obtain two different views of the same
image. It is worth noting that the pretext task of Barlow Twins is not
contrastive; however, it still requires two inputs in order to learn. As a
result, we can use our methods to analyze the similarity aspect of this model
as well.

###  V-B Qualitative Analysis

We initially present visual examples of our methods. In Figs. 4 and 5, we show
how one of the baseline methods (Smooth-Grad) highlights incorrect
corresponding regions as a result of considering only one level of
augmentation, which can be misleading. Our proposed Averaged Transforms method
mitigates this issue by averaging over all augmentation levels, thereby
diminishing irrelevant parts. We present further qualitative results for
Averaged Transforms using a 90-degree rotation transformation, as shown in
Fig. 6. The top row shows the dissected rotation steps. We compare the results
with InputXGrad and SmoothGrad in the second and third column, respectively.
It is worth highlighting that while the rotation transformation is not part of
the contrastive learning methods studied, its assessment is still relevant.
This is because the contrastive model is expected to be invariant across
various transformations, even across those that it has not been trained on,
such as rotations. In Fig. 7, we observe how in many cases, the baseline
Conditional Occlusion deems the background as highly important, due to the
problem discussed in Section III-B. Our proposed Pairwise Occlusion method
alleviates this problem. Particularly, regions identified as unimportant by
Conditional Occlusion are sometimes deemed significant in Pairwise Occlusion.
Next, we show visual examples of each technique. Figure 8 shows the output of
Interaction-CAM compared to Baseline-Grad-CAM. In spite of the high similarity
score between the two images, the baseline conveys irrelevant concepts.
Interaction-CAM concentrates all the energy in similar concepts which jointly
contribute to the high similarity output. Moreover, we observe in multiple
cases that contrastive models can learn to attend to semantically meaningful
concepts, despite being trained in a self-supervised class-free setting. These
concepts are often visually salient due to their distinctive textures and
geometric patterns. Figure 9 illustrates this point and shows that the
important visual cues are associated to the main subject of the scene, and the
locations of the meaningful concepts are correctly mapped between the two
images, including the region around the wolf’s eye or the back of the
flamingo. Figure 10 also shows how different concepts are highlighted when
those are associated to the same semantic object (different parts of the dog’s
and cat’s faces). The ability to learn distinctive visual cues is also
illustrated in Fig. 11 where two different images from the same class are fed
to the similarity model, showing a strong attribution to distinctive patterns.
In Fig. 12, we show qualitative results from our perturbation- and activation-
based approaches, with all methods applied on the same pair of images. This
shows how different approaches interpret the same image. Activation-based
methods are also compared with DeepSim [27]. As evident from the Figure, most
methods are in consensus regarding what they identify as important.

###  V-C Quantitative Analysis

We will now present the evaluation scores for SimCLRv2 1$\times$, SimCLRv2
2$\times$, Barlow Twins, SimSiam and MoCov3 across all metrics proposed in
Section IV-B. Subsequently, we will provide a summary of our findings through
an additional analysis to enhance the comprehensibility and clarity of the
results. Evaluation scores from the SimCLRv2 (1$\times$, 2$\times$)
contrastive models [2] are presented in Table I. Barlow Twins model results
are displayed in Table II. SimSiam and MoCov3 evaluation scores are shown in
Tables III and IV, respectively. SimCLRv2 (1$\times$), SimCLRv2 (2$\times$),
Barlow Twins and SimSiam use a ResNet-50 [52] backbone, whereas MoCov3 uses a
Vision Transformer backbone (ViT-Base). Since Interaction-CAM belongs to the
activation visualization category of explainability, we also compare
Interaction-CAM with the Deep Similarity technique [27] on all metrics. Upon
examining the results presented in the tables, it is apparent that the
baseline conditional occlusion method outperforms our proposed pairwise
occlusion approach on nearly all metrics. This can be attributed to the fact
that the pairwise occlusion method tends to generate sparse regions with equal
pixel values due to the assignment of the softmax score to all pixels within
the erased area. The regions—specifically, the perturbed rectangles—end up
with pixels sharing identical values (the softmax score). Yet, our evaluation
mechanism assesses based on individual pixels rather than broader regions. As
a result, when a whole region is assigned one uniform value, it causes a
plateau in the evaluation curve. Therefore, the evaluation curve does not
exhibit sharp drops or increases when pixels are added or removed, as is
observed in the case of conditional occlusion. Consequently, this leads to a
reduction in the AUC. For saliency-based methods, we observe that Averaged
Transforms achieves superior performance compared to baseline methods across
most metrics. Particularly, Averaged Transforms variants have obtained the
highest scores in 5 out of 7 metrics for SimCLRv2 ($1\times$), and 6 out of 7
metrics for SimCLRv2 ($2\times$). We discuss which variant of Averaged
Transforms is generally better in Section V-D. For activation-based methods,
we observe that the proposed Interaction-CAM variants outperform the baseline
methods in 4 out of 6 metrics for SimCLRv2 ($1\times$) and SimCLRv2
($2\times$). We have additionally demonstrated the impact of the gradient
interaction term (GI) on all method variants, revealing a notable enhancement
in most performance metrics upon its inclusion. Conversely, Interaction-CAM
only outperforms the baseline method on the SD and CD metrics for the Barlow
Twins model. This difference in performance could be attributed to the fact
that Barlow Twins is not trained with contrastive learning. Thus, assessing
the model’s explanations using the similarity measure may not yield the most
optimal results. In Section V-D, we also investigate which variant of
Interaction-CAM is generally better. In Table V, we report evaluation scores
of Averaged Transforms for the maximum sensitivity metric. As shown, our
proposed Averaged Transform achieves the lowest sensitivity when compared to
all other baseline measures across all contrastive models.

Figure 13: Sanity check for Interaction-CAM (mean), Interaction-CAM (max),
Interaction-CAM (attention), Average Transforms (guided backpropagation),
Average Transforms (without guided backpropagation), Conditional Occlusion and
Pairwise Occlusion, ordered from top to bottom.

In Fig. 13, we perform a sanity check on Interaction-CAM (mean), Interaction-
CAM (max), Interaction-CAM (attention), Average Transforms (w/ guided
backpropagation), Average Transforms (w/o guided backpropagation), Conditional
Occlusion and Pairwise Occlusion, ordered from top to bottom. We start with
the original explainability map of the two images (second column) and
randomize the weights of the model starting from the top layer, progressively,
all the way to the bottom layer. At each randomization step, we run the
explainability method and obtain the explanation map. To avoid visual clutter,
we only show the results after every three layers. We can see that the
explanation completely becomes meaningless as randomization is applied. After
a certain layer (e.g., layer 18 in Fig. 13), the explanation map gets
completely zeroed out. Thus, our proposed techniques are sensitive to model
parameters and have passed the sanity check.

###  V-D Summary of the Results and Applications

Figure 14: Summary of scores of five self-supervised models for the three
groups of metrics (a) SI, CI, (b) SD, CD, and (c) SAD, CAD Figure 15:
Correlation with downstream tasks on the (SI, CI) and (SAD, CAD) group of
metrics shown along with the linear correlation coefficient in green, for
perturbation, activation and saliency-based methods. In all plots except the
last, a positive correlation is observed between the evaluation scores and
downstream task performance.

In Section V, we have presented evaluation scores of our explainability
techniques for the five models. In this section, our aim is to provide a
concise and informative summary of the results, thereby facilitating better
interpretation. We perform the following analysis to summarize our results. We
cluster the evaluation metrics into 3 groups (SI, CI), (SD, CD) and (SAD,
CAD), and compute the average score for the first group, and
$(1-\text{average})$ for the last two groups, such that a higher score
indicates better performance. We thus obtain a single value for each group. It
is worth noting that various evaluation metrics are designed to measure
different aspects of explanation faithfulness. Deciding which set of metrics
to prioritize would depend on what one seeks to assess from the explanation.
The first set of metrics, namely (SI, CI), primarily assess the faithfulness
of the explanation. Conversely, the last two sets of metrics, (SD, CD) and
(SAD, CAD), are primarily focused on capturing the confidence level associated
with the explanation. As a result, we refrain from calculating a global
average of all six metric. Figure 14 depicts a heatmap of scores for each of
the three groups of metrics for five different models. From Fig. 14, we draw
the following observations by comparing all methods (row-wise) for each model:
For the first group of metrics (SI, CI), Averaged Transforms (G) scores the
best on average. For the second and third group of metrics represented by (SD,
CD) and (SAD, CAD), respectively, Conditional Occlusion scores best on
average.

Numerous factors play a role in determining the appropriate question that each
method answers, i.e., which method to utilize in specific situations. However,
we emphasize two key factors that one should consider before making a
decision: 1) The user’s intent from the explanation, and 2) the computational
speed requirements for the explanation. To serve the user with accurate
information, it is imperative to determine precisely what the user seeks to
comprehend from the contrastive model. For example, Averaged Transforms is a
method that can be used to understand how data augmentations and invariance
affect the similarity output. On the other hand, perturbation-based method and
activation-visualization methods should be used to study the primary causes of
the predicted similarity. Another important factor to consider in practical
applications is the speed requirements. Table VI presents the execution time
of both the baseline and proposed methods, which is averaged over five
separate runs. All experiments were performed on a single NVIDIA RTX 3090 GPU.
Our findings show that activation visualization-based methods have the fastest
execution time. Therefore, such methods should be favoured if speed is a
critical consideration. In general, all methods allow to understand
contrastive models, but each differs in its objective and purpose it defines.

TABLE VI: Execution time (in seconds) of the baseline and proposed methods Method | Time (s)
---|---
Conditional Occlusion | 0.980
Pairwise Occlusion | 0.257
Saliency | 0.296
Smooth-Grad | 2.258
Avg. Transforms | 0.411
Baseline-Grad-CAM | 0.215
Int-CAM (w/o GI) | 0.079
Int-CAM (w/ GI) | 0.050

###  V-E Relationship to Downstream Tasks

Although the general focus of this work is to explain contrastive models, we
also endeavor to explore the correlation and determine the relationship
between the explainability aspect of contrastive self-supervised models and
their performance on downstream tasks after finetuning. We use the common
image classification task on ImageNet as the downstream task in our
experiments. Figure 15 displays the correlation plots of perturbation,
activation, and saliency-based methods, along with their corresponding
correlation coefficients. The x-axis denotes the explanation score acquired
using the same procedure outlined in Section V-D, for both the (SI, CI) and
(SAD, CAD) groups of metrics. The $y$-axis denotes the ImageNet linear probe
evaluation accuracy. In almost all plots, a positive correlation is observed
between the evaluation scores and downstream task performance. Note that for
all methods other than perturbation-based ones, we have excluded SimSiam
because it was identified as an outlier, which interfered with a correct
calculation of the correlation coefficient. Among all family of methods, it
can be observed that perturbation-based are mostly correlated with downstream
task performance for both metric groups, with a correlation coefficient of
0.75 and 0.65 for (SI, CI) and (SAD, CAD), respectively. We recognize that
analyzing only five models may not provide an accurate result, and examining a
larger number of models would yield a more precise outcome.

##  VI Exploration and Insights

Figure 16: By using Interaction-CAM, we find examples of models which focus on
meaningless concepts, despite the same image content being present in both
images. Figure 17: By utilizing Interaction-CAM, we find that a model predicts
an image of a "tiger cat" and a "tiger" are similar because of the "stripes"
texture, despite the fact that the two images are different. The similarity
scores for each of the image pairs (from left to right) are 0.70 and 0.71.
Figure 18: Pairs of images with a low similarity score (<0.5), despite the two
images containing the same content. By utilizing Interaction-CAM, we identify
that the combination of the blur and blackening harms the performance as the
model is not able to learn any meaningful concepts. TABLE VII: Explanation
scores for Averaged Transforms using several data augmentation techniques.

Transformation | SI $\uparrow$ | SD $\downarrow$ | SAD $\downarrow$ | SI $\uparrow$ | SD $\downarrow$ | SAD $\downarrow$
---|---|---|---|---|---|---
| Non-Guided | Guided
| SimCLRv2 (1X)
Color Jitter | 0.945 | 0.907 | 0.052 | 0.961 | 0.922 | 0.051
Blur | 0.495 | 0.849 | 0.001 | 0.473 | 0.892 | 0.006
Grayscale | 0.961 | 0.935 | 0.055 | 0.975 | 0.960 | 0.047
Solarization | 0.824 | 0.907 | 0.014 | 0.822 | 0.925 | 0.023
Rotation@30 | 0.746 | 0.759 | 0.007 | 0.730 | 0.773 | 0.012
Rotation@60 | 0.670 | 0.744 | 0.002 | 0.643 | 0.761 | 0.004
Rotation@90 | 0.785 | 0.898 | 0.005 | 0.746 | 0.904 | 0.017
Rotation@180 | 0.869 | 0.925 | 0.011 | 0.812 | 0.926 | 0.027
Horizontal Flipping | 0.955 | 0.948 | 0.028 | 0.960 | 0.955 | 0.042
| SimCLRv2 (2X)
Color Jitter | 0.940 | 0.865 | 0.086 | 0.960 | 0.883 | 0.077
Blurring | 0.510 | 0.774 | 0.009 | 0.470 | 0.827 | 0.021
Grayscale | 0.948 | 0.887 | 0.111 | 0.973 | 0.928 | 0.056
Solarization | 0.807 | 0.862 | 0.044 | 0.814 | 0.880 | 0.050
Rotation@30 | 0.697 | 0.698 | 0.027 | 0.693 | 0.724 | 0.048
Rotation@60 | 0.605 | 0.679 | 0.011 | 0.560 | 0.715 | 0.032
Rotation@90 | 0.731 | 0.827 | 0.016 | 0.668 | 0.841 | 0.040
Rotation@180 | 0.830 | 0.861 | 0.027 | 0.743 | 0.869 | 0.054
Horizontal Flipping | 0.944 | 0.908 | 0.044 | 0.948 | 0.913 | 0.084

Figure 19: Contrastive models slowly learn to discard (learn to be invariant
to) transformations (e.g. color) throughout the layers

This section presents a series of observations and findings on how the
contrastive model SimCLRv2 [2] attributes to different visual features to
infer similarity scores for pairs of images. The following findings are based
on a thorough assessment of explanation maps, of which we display some
representative examples for illustration. We acknowledge that these analyses
are only qualitative and may omit other key points that could be discovered by
reviewing a greater number of examples.

####  VI-1 Meaningless concepts and biasness towards texture

By examining some samples with our methods, we find that visual cues do not
always correspond to the class of interest, which is to be expected from
contrastive models. In Fig. 16, the example shows more focus on the road than
the tricycle, despite the same content being present in both images. In Fig.
17, tigers and cats from different image classes are easily confused due to
their similar textures, which seems to suggest that the concept of striped fur
is more easily learned than the animal itself. We also sampled 50 image pairs
containing different objects but with a common texture (Fig. 17). We observe
that in 64% of the cases the methods explain similarity based on texture.

####  VI-2 Identifying harmful data augments

We analyze a set of examples which lead to a low similarity score, with the
aim of understanding the underlying cause of the low score through the use of
our proposed methods. Through the analysis, we identify that the blur and
blackening augmentation harms the performance of the contrastive model and is
more likely to result in bad similarity estimation. An image blackening effect
is caused by a characteristic of the color jitter transform. Color jitter is
characterized by alternating brightness, contrast, saturation, and hue values
(in this case, alternating brightness is referred to as blackening). Fig. 18
illustrates this fact and the associated heatmaps show small overlap with any
of the main subjects of the scene. That is, they do not learn any meaningful
concepts. This suggests that the brightness aspect in color jitter transforms
should be carefully considered. Therefore, our methods can act as tools to
detect data augmentations that can be beneficial or harmful for a model. To
assess the percentage of time this phenomenon happens, we sampled 50 image
pairs and apply the blur and blackening effect to them. We find that 94% of
the explanations highlight meaningless concepts (Fig. 18).

####  VI-3 Dissecting Data Augmentations

Contrastive models employ a sequential approach for data augmentation,
combining color jittering with grayscale and Gaussian blurring to create
diverse data samples. Barlow Twins [30], on the other hand, utilizes
solarization in addition to these augmentations. In this section, we aim to
investigate the individual effect of each augmentation method on the model’s
performance, in addition to other augmentations that are not commonly used in
contrastive models, such as rotations. We present an analysis in Table VII. By
observing the top-scoring augments for each metric, we find that a 60-degree
rotation for SimCLRv2 (1$\times$) has the highest impact on the model’s
invariance performance as measured by the explainability metrics. For SimCLRv2
(2$\times$), Gaussian blurring, grayscale transformations and 60-degree
rotations are all equally impactful on the model’s invariance performance.
Gaussian blurring, by eliminating high-frequency details from images, and
grayscale transformations, by stripping away color information, both generate
more challenging examples that are meaningful for learning invariance. The
results further indicate that contrastive models are also invariant to
rotations, despite not being trained with this transformation.

##  VII Conclusion

We have proposed a set of techniques aimed at visualizing and understanding
contrastive vision learning models, which operate on pairs of inputs. These
techniques encompass saliency, activation, and perturbation-based approaches.
For each technique, we established a baseline, which is a straightforward
extension of the techniques used to explain single-image models, such as image
classification models. We then highlighted the deficiencies of the baselines
and proposed new explanation methods that can overcome these shortcomings.
Furthermore, we have extended existing evaluation metrics to suit contrastive
models. We then evaluated our techniques using these metrics and conducted an
analysis, including a summary of the results, the relevance to downstream
tasks, and various explorations and findings.

[Exploring Invariance: A Visual Investigation] In order to validate the
invariance of contrastive models towards transformations like color and
saturation, we conducted an experiment using feature inversion, as described
in [53]. We initialized an input image with random noise and optimized it to
reconstruct features from the 3rd, 4th, and 5th layers of a ResNet-50
backbone, which was trained using contrastive learning. We then compared the
resulting synthesized images with those obtained from the same model trained
with a different objective, namely image classification. The results are shown
in Fig. 19. For the contrastive model, the image saturation diminishes with
increasing layer depth, indicating that SimCLRv2 learns latent representations
that are more invariant to colors resulting from the data augmentation-based
training. This also aligns with the observation found in [54] in that self-
supervised models fail to retain colour information compared to supervised
models. For this experiment, we use the mean squared error (MSE) as a loss
function to minimize the loss of features from the synthesized image and the
given feature vector at a particular layer. We initialize the synthesized
image from the normal distribution with a standard deviation of 0.1 and use
stochastic gradient descent (SGD) with a momentum of $0.9$ to optimize the
image for $200$ iterations with a learning rate of $1e4$, which is reduced by
a factor of $0.1$ every $50$ iterations. We use two different natural image
priors which are derived from general properties of natural images, and which
help in guiding the optimization process towards more realistic
reconstructions: the total variation regularizer with a weight of $1e-8$ and
the $\alpha$-norm regularizer with $\alpha$ set to $6$ and a weight of $1e-7$.

## References

  * [1] G. Koch, “Siamese neural networks for one-shot image recognition,” Ph.D. dissertation, University of Toronto, 2015.
  * [2] T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. E. Hinton, “Big self-supervised models are strong semi-supervised learners,” _Advances in neural information processing systems_ , vol. 33, pp. 22 243–22 255, 2020\.
  * [3] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark _et al._ , “Learning transferable visual models from natural language supervision,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 8748–8763.
  * [4] K. He, H. Fan, Y. Wu, S. Xie, and R. B. Girshick, “Momentum contrast for unsupervised visual representation learning,” _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 9726–9735, 2020.
  * [5] M. Caron, H. Touvron, I. Misra, H. J’egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 9630–9640, 2021.
  * [6] T.-Y. Lin, P. Goyal, R. B. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 42, pp. 318–327, 2017.
  * [7] A. Shrivastava, A. K. Gupta, and R. B. Girshick, “Training region-based object detectors with online hard example mining,” _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 761–769, 2016.
  * [8] S. A. Bargal, A. Zunino, V. Petsiuk, J. Zhang, K. Saenko, V. Murino, and S. Sclaroff, “Guided zoom: Zooming into network evidence to refine fine-grained model decisions,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 43, pp. 4196–4202, 2021.
  * [9] S. Bach, A. Binder, G. Montavon, K.-R. Müller, and W. Samek, “Analyzing classifiers: Fisher vectors and deep neural networks,” _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 2912–2920, 2016.
  * [10] S. Lapuschkin, S. Wäldchen, A. Binder, G. Montavon, W. Samek, and K.-R. Müller, “Unmasking clever hans predictors and assessing what machines really learn,” _Nature Communications_ , vol. 10, 2019.
  * [11] B. Goodman and S. Flaxman, “European union regulations on algorithmic decision-making and a "right to explanation",” _AI Mag._ , vol. 38, pp. 50–57, 2017.
  * [12] M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in _ECCV_ , 2014.
  * [13] M. T. Ribeiro, S. Singh, and C. Guestrin, “"why should i trust you?": Explaining the predictions of any classifier,” _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 2016\.
  * [14] B. Zhou, A. Khosla, À. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 2921–2929, 2016.
  * [15] R. R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” _International Journal of Computer Vision_ , vol. 128, pp. 336–359, 2019.
  * [16] S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,” _PLOS ONE_ , vol. 10, no. 7, pp. 1–46, 07 2015. [Online]. Available: https://doi.org/10.1371/journal.pone.0130140
  * [17] K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” _CoRR_ , vol. abs/1312.6034, 2014.
  * [18] D. Smilkov, N. Thorat, B. Kim, F. B. Viégas, and M. Wattenberg, “Smoothgrad: removing noise by adding noise,” _ArXiv_ , vol. abs/1706.03825, 2017.
  * [19] M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” _ArXiv_ , vol. abs/1703.01365, 2017.
  * [20] O. Eberle, J. Büttner, F. Kräutli, K.-R. Müller, M. Valleriani, and G. Montavon, “Building and interpreting deep similarity models,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, pp. 1149–1161, 2022.
  * [21] L. Arras, G. Montavon, K.-R. Müller, and W. Samek, “Explaining recurrent neural network predictions in sentiment analysis,” in _WASSA@EMNLP_ , 2017\.
  * [22] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in _International conference on machine learning_. PMLR, 2015, pp. 448–456.
  * [23] M. Guillemot, C. Heusèle, R. Korichi, S. Schnebert, and L. Chen, “Breaking batch normalization for better explainability of deep neural networks through layer-wise relevance propagation,” _ArXiv_ , vol. abs/2002.11018, 2020.
  * [24] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [25] E. Voita, D. Talbot, F. Moiseev, R. Sennrich, and I. Titov, “Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned,” in _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 2019, pp. 5797–5808.
  * [26] H. Chefer, S. Gur, and L. Wolf, “Transformer interpretability beyond attention visualization,” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 782–791, 2021.
  * [27] A. Stylianou, R. Souvenir, and R. Pless, “Visualizing deep similarity networks,” _2019 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , pp. 2029–2037, 2019.
  * [28] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in _International conference on machine learning_. PMLR, 2020, pp. 1597–1607.
  * [29] X. Chen and K. He, “Exploring simple siamese representation learning,” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 15 745–15 753, 2021.
  * [30] J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny, “Barlow twins: Self-supervised learning via redundancy reduction,” in _ICML_ , 2021.
  * [31] X. Chen, S. Xie, and K. He, “An empirical study of training self-supervised vision transformers,” _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 9620–9629, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:233024948
  * [32] V. Pillai, S. A. Koohpayegani, A. Ouligian, D. Fong, and H. Pirsiavash, “Consistent explanations by contrastive learning,” _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 10 203–10 212, 2022.
  * [33] S. Baluja, “Hiding images in plain sight: Deep steganography,” in _Neural Information Processing Systems_ , 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:29764034
  * [34] W. Sultani, C. Chen, and M. Shah, “Real-world anomaly detection in surveillance videos,” _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 6479–6488, 2018.
  * [35] O. Köpüklü, J. ming Zheng, H. Xu, and G. Rigoll, “Driver anomaly detection: A dataset and contrastive learning approach,” _2021 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , pp. 91–100, 2021\.
  * [36] C.-F. Chen, Q. Fan, and R. Panda, “Crossvit: Cross-attention multi-scale vision transformer for image classification,” _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 347–356, 2021.
  * [37] S. Yan, X. Xiong, A. Arnab, Z. Lu, M. Zhang, C. Sun, and C. Schmid, “Multiview transformers for video recognition,” _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 3323–3333, 2022.
  * [38] S. Shen, L. H. Li, H. Tan, M. Bansal, A. Rohrbach, K.-W. Chang, Z. Yao, and K. Keutzer, “How much can clip benefit vision-and-language tasks?” _ArXiv_ , vol. abs/2107.06383, 2022.
  * [39] F. Sammani, T. Mukherjee, and N. Deligiannis, “Nlx-gpt: A model for natural language explanations in vision and vision-language tasks,” _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 8312–8322, 2022.
  * [40] V. Petsiuk, A. Das, and K. Saenko, “Rise: Randomized input sampling for explanation of black-box models,” in _BMVC_ , 2018.
  * [41] A. Chattopadhyay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, “Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks,” _2018 IEEE Winter Conference on Applications of Computer Vision (WACV)_ , pp. 839–847, 2018.
  * [42] C.-K. Yeh, C.-Y. Hsieh, A. S. Suggala, D. I. Inouye, and P. Ravikumar, “How sensitive are sensitivity-based explanations?” _ArXiv_ , vol. abs/1901.09392, 2019.
  * [43] H. Wang, Z. Wang, M. Du, F. Yang, Z. Zhang, S. Ding, P. Mardziel, and X. Hu, “Score-cam: Score-weighted visual explanations for convolutional neural networks,” _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , pp. 111–119, 2020.
  * [44] J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. A. Riedmiller, “Striving for simplicity: The all convolutional net,” _CoRR_ , vol. abs/1412.6806, 2015\.
  * [45] H. J. Kwon, H. I. Koo, J. W. Soh, and N. I. Cho, “Inverse-based approach to explaining and visualizing convolutional neural networks,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 33, pp. 7318–7329, 2021.
  * [46] J. R. Lee, S. Kim, I. Y. Park, T. Eo, and D. Hwang, “Relevance-cam: Your model already knows where to look,” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 14 939–14 948, 2021.
  * [47] R. L. Draelos and L. Carin, “Use hirescam instead of grad-cam for faithful explanations of convolutional neural networks,” _ArXiv_ , vol. abs/2011.08891, 2020. [Online]. Available: https://arxiv.org/abs/2011.08891
  * [48] P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, “Layercam: Exploring hierarchical class activation maps for localization,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 5875–5888, 2021.
  * [49] J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [50] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and L. Fei-Fei, “Imagenet large scale visual recognition challenge,” _International Journal of Computer Vision_ , vol. 115, pp. 211–252, 2015.
  * [51] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proc. IEEE_ , vol. 86, pp. 2278–2324, 1998.
  * [52] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 770–778, 2016.
  * [53] A. Mahendran and A. Vedaldi, “Understanding deep image representations by inverting them,” _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 5188–5196, 2015.
  * [54] L. Ericsson, H. Gouk, and T. M. Hospedales, “How well do self-supervised models transfer?” _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 5410–5419, 2021.