# DXAI: Explaining Classification by Image Decomposition

Elnatan Kadar Guy Gilboa

###### Abstract

We propose a new way to explain and to visualize neural network classification
through a decomposition-based explainable AI (DXAI). Instead of providing an
explanation heatmap, our method yields a decomposition of the image into
class-agnostic and class-distinct parts, with respect to the data and chosen
classifier. Following a fundamental signal processing paradigm of analysis and
synthesis, the original image is the sum of the decomposed parts. We thus
obtain a radically different way of explaining classification. The class-
agnostic part ideally is composed of all image features which do not posses
class information, where the class-distinct part is its complementary. This
new visualization can be more helpful and informative in certain scenarios,
especially when the attributes are dense, global and additive in nature, for
instance, when colors or textures are essential for class distinction. Code is
available at https://github.com/dxai2024/dxai.

##  1 Introduction

Understanding classification reasoning of neural networks is of paramount
importance. It can lead to better understanding of the classificaction
process, help in debugging and validation and also serve as an additional
informative output for the user at inference. Thus, the research on the topic
of explainable artificial intelligence (XAI) is extensive [11, 18, 2, 13].

Figure 1: What makes a cat a Cat? Explained through decomposition. Examples of
class-distinct, $\psi_{Distinct}$, and class-agnostic, $\psi_{Agnostic}$,
parts of Animals dataset [8], where the input
$x=\psi_{Agnostic}+\psi_{Distinct}$. The distinct part gives high resolution,
dense, color and texture information explaining the classification decision.
This is in contrast to standard XAI (bottom two rows).

The most common way to show the explanation of the network in image
classification is by producing an _explanation heatmap_. This map is a per-
pixel indication of the relevance of that pixel to the final classification
decision of the network (see Fig. 2). In this visualization, the larger the
value - the more relevant this pixel is to the classification. These maps can
be at some lower resolution, where visualization is done by upsampling, or by
using superpixels. In addition, some methods give also negative values,
indicating pixels which reduce the confidence for the final classification.
The heatmaps themsleves do not resemble actual images and to understand the
role of the pixels in a heatmap common practices are either to show the input
image and the heatmap side by side, to overlay the heatmap on the image, or
(less commonly) to show an image where the brightness of the pixels are
weighted by the (normalized) heatmap, Fig. 2.

This visual explanation method is mostly adequate when the explanation is
spatially _sparse_. That is, there are just a few small regions in the image
which contribute mostly to the classification. However, there are many
classification problems in which the explanations are _dense_ in the image
domain. This can happen in several scenarios, for instance:

  1. 1.

The object to be classified spans a large portion of the image domain and
contains many diverse features, all contributing to the final classification.

  2. 2.

A main feature contributing to the classification is color change, which
appears throughout the image.

  3. 3.

The class distinction is based on some global disturbance or statistical
change, which spans the entire image domain.

In such scenarios, a heatmap is much less informative. Depending on the
algorithm, it is either too focused on just a small portion of highly dominant
features or it shows uniform large areas spanning most of the image. See some
examples in Fig. 3. In both cases, we lack a clear constructive explanation
for the network’s decision.

Figure 2: Three visualizations using a heatmap. Generated by Grad-CAM [29],
highlighting cars in an aerial image (left).  Figure 3: Our method vs.
heatmaps, illustrating three scenarios mentioned in Section 1: 1) Many details
spread across large portions of the image are helpful for accurate
classification (top row), heatmaps show only partial relevant information. 2)
Distinguishing between types of peppers that differ mainly by color (2nd row),
heatmaps struggle to provide color data. 3) Detecting additive statistical
disturbance (a class of clean images and a class of images with noise). Here
as well, heatmaps face difficulties explaining the reason for classification.

To alleviate the above problems, we propose a different way of computing and
visualizing explanations of the classification. It does not use heatmaps and
is based on decomposition. Image decomposition is a long standing paradigm in
classical image processing. It is based on the notion that images (and signals
in general) can often be well decomposed into meaningful additive parts. Given
a 1D signal $x$ of length $N$ and some orthonormal basis of $N$ elements
$\\{\phi_{i}\\}$, $i=1,..\,N$, one can express $x$ as

$x=\sum\limits_{i=1}^{N}\alpha_{i}\phi_{i},$

with $\alpha_{i}=\langle x,\phi_{i}\rangle$, where $\langle\cdot,\cdot\rangle$
denotes inner-product. Some well known examples which fall into this category
are Fourier and wavelet basis [23, 9]. In overcomplete representations [10, 1]
a common practice is to decompose a signal into only a few $K\ll N$ elements
(sparse representation) of an overcomplete dictionary $\\{\psi_{i}\\}$,
$i=1,..D\gg N$, allowing some small error $e$,

$x=\sum\limits_{i=1}^{K}\alpha_{i}\psi_{i}+e.$

There are many other studies attempting to decompose an image into meaningful
additive parts. For example, image decomposition into structure and texture
$x=\psi_{Structure}+\psi_{Texture}(+e)$ (with possible noise or small error)
[4, 3, 12].

In this work we propose to express XAI by the following image decomposition:

$x=\psi_{Agnostic}+\psi_{Distinct},$ (1)

where $\psi_{Agnostic}$ is the _class agnostic_ part of the image, which
ideally does not entail information about the class, and $\psi_{Distinct}$ is
the _class distinct_ part which holds the discriminative information, allowing
the classifier to obtain distinction from other classes. We use style transfer
based on generative AI to accomplish this. We show this way of explaining
classification brings new computational and visualization tools, which, for
some cases, are much more natural and informative. Our main contributions are
as follows:

  1. 1.

We present a detailed computational framework to estimate Eq. (1), for a given
classifier, training set and classification task. The decomposition is of high
resolution, allowing to portray well fine and delicate details.

  2. 2.

We show, for the first time, class agnostic images, based on decomposition.
This provides new information and insights on the classification problem.

  3. 3.

The method is fast since results are produced at inference time of generative
models (no gradients are computed).

  4. 4.

We provide extensive examples and experimental data showing the advantages
(and limitations) of the method, compared to heatmaps.

##  2 Related Work

In this work, we focus on visually explaining image classification.
Specifically, we concentrate on algorithms that, given an image classification
result, highlight the areas contributing to the classification [29, 27, 22, 5,
30, 32, 21, 20]. As previously discussed, high values indicate significance,
while values near 0 are of negligible contribution. A leading approach is
backpropagation-based methods [29, 5, 30]. It involves tracing the
classifier’s solution backward through the model’s layers to measure the
contribution of each layer to the subsequent one. These methods encompass
gradient-based and attribution-based techniques. Perturbation-based methods
[22, 27, 32] evaluate the impact on the output of changes in the input.
Changes leading to strong output variations are deemed important.

Attention-based methods [20, 14, 35] identify relationships within the input
to discern important image characteristics. These methods often require
specific classifier architectures. Other methods estimate the uncertainty in
the solution and use this to find explanations with high confidence [24, 37].
Generative models are also used to explain differences between classes. Many
of the methods recently try to provide counterfactual explanations [17, 26].
However, it appears to be difficult to use these explanations in order to
produce a map of clearly highlighted differences. Current XAI methods
typically suffer from at least one of the following drawbacks:

  1. 1.

Resolution. Low resolution that hinders fine detail distinction, usually
caused by calculating importance in spatially coarse internal layers.

  2. 2.

Compute. Extended runtime or high memory consumption due to gradient or
attribution calculations across multiple layers or using numerous perturbation
iterations.

  3. 3.

Architecture. Certain solutions impose architectural constraints, such as
specific activation layers or even a dedicated architecture designed solely
for XAI.

  4. 4.

Single channel. Typically, these algorithms produce grayscale images
representing the importance of each pixel, lacking color distinction (multi-
channel data in general). This can lead to inadequate explanations, as seen,
for example, in our peppers dataset experiment (Fig. 11).

Our proposed approach comprehensively addresses these challenges, as detailed
below.

##  3 Method

Figure 4: Class-distinct component $\psi_{Distinct}$ by our DXAI algorithm and
by heatmap manipulation, in which the input image is weighted by the
normalized heatmap (see details in Sec. 4). We obtain high resolution, dense
multi-channel explanations.

We aim at partitioning the image to a component which is neutral for the
classifier (agnostic) and to a component which explains the classification
(distinct). We provide some definitions and formulate the DXAI problem. This
is a very hard nonconvex minimization problem. In the second part we present
an architecture and a training procedure to obtain an approximate solution.

###  3.1 The DXAI problem

We first introduce the basic setting and notations. We assume a multiclass
supervised learning setting. Let $X$ be the space of input images and $x\in X$
be an input image belonging to one of $c$ classes. Let $y\in Y$ be a class
label $y\in\\{0,c-1\\}$. The training set consists of $M$ pairs
$(x_{i},y_{i})\in X\times Y$, $i=1,..\,,M$. We denote by $x^{y}$ an image $x$
belonging to class $y$. We are given an image classifier $C$, which provides a
class probability vector $p$. We would like to extract the relevant features,
and only them, which contribute to the classification decision by the network.
An additive contribution, as in Eq. (1), is assumed. We want to have a
reference image, which is neutral in terms of classification (for a given
classifier), using the following definition:

######  Definition 1 (Class Agnostic)

For an image $\psi\in X$, given $c$ classes of images and a classifier $C$
providing a vector $p(\psi)\in\mathbb{R}^{c}$ of class probabilities, $\psi$
is class agnostic if the probability is uniform, that is
$p_{i}(\psi)=\frac{1}{c}$, $\forall i=1,..\,c$, where $p_{i}$ is the $i$’s
entry of $p$.

We denote by $X_{A}\subset X$ the space of all $\psi\in X$ which are class
agnostic, as defined above. Given an image $x\in X$ with class probability
vector $p(x)$, and a classifier $C$, we would like to solve the following
minimization problem:

$\min_{\psi\in X_{A}}d(x,\psi),$ (2)

where $d(\cdot,\cdot)$ is some distance measure. For instance, our
implementation uses a combination of $L^{1}$ and $L^{2}$ norms for this
distance. We refer to Eq. (2) as the DXAI (decomposition-based XAI) problem.
We can now define the class distinct and class agnostic parts of an image $x$.

######  Definition 2 (Class Distinct / Agnostic parts of $x$)

Let $x\in X$ be an image with class probability vector $p(x)$ provided by the
classifier $C$. Let $\psi^{*}$ be a minimizer of (2). Then
$\psi_{Agnostic}=\psi^{*}$ is a class agnostic part of image $x$ and
$\psi_{Distinct}=x-\psi^{*}$ is a class distinct part.

In words, for a given image $x$, we find the closest image (with respect to
$d$) which is neutral, in terms of class (agnostic part). The difference
between $x$ and this neutral image explains the probability vector $p(x)$ and
the reason, according to the classifier, why it deviates from neutrality. We
do not assume a unique solution. The pair
$\\{\psi_{Agnostic},\psi_{Distinct}\\}$ provide detailed and dense class
explanation. We note that pure agnostic images are hard to obtain. A more
relaxed requirement is to have $|p_{i}(\psi)-\frac{1}{c}|\leq\varepsilon$,
$0<\varepsilon\ll\frac{1}{c}$ in Def. 1. In any case, problem (2) is very hard
to solve. We give below a suggested approximation. Alternative methods may be
further investigated.

###  3.2 Approximate solution for the DXAI problem

Given an image $x$ classified to class $y$ by the classifier $C$, we want to
approximate a pair $\\{\psi_{Agnostic},\psi_{Distinct}\\}$. We use generative
models for the decomposition. We leverage style transfer as the primary tool
for discerning inter-class differences and generating class-explanations. Our
implementation uses style-transfer GANs [8]. Following [6], which shows better
descriptive capacity of branched GANs, we use several generators for the
agnostic part, which usually contains most of the image data. We assume to
have $n$ style transfer generators $G_{i}(x^{y},s_{y})$, which take as inputs
an image $x^{y}$ and a style vector $s_{y}$. The style vector is generated by
a dedicated mapping network, as detailed in [8]. This network takes the class
as input and produces the corresponding style representation. Given an image
$x^{y}$ of class $y$ we approximate it by the following decomposition into $n$
components (branches):

$\hat{x}^{y}=\psi_{1}^{y}+\sum\limits_{i=2}^{n}\psi_{i}^{y},$ (3)

where $\hat{x}^{y}\approx x^{y}$ and $\psi_{i}^{y}=G_{i}(x^{y},s_{y})$. We
assign

$\psi_{Distinct}=\psi_{1}^{y};\quad\psi_{Agnostic}=\sum\limits_{i=2}^{n}\psi_{i}^{y}.$
(4)

Our aim is to transform an image from class $y$ into an image representative
of target class $\tilde{y}$. Successful style transfer requires the
identification and modification of distinct class-specific characteristics.
Style transfer is accomplished by using the style vector associated with class
$\tilde{y}$ in the generators. That it,
$\psi_{i}^{\tilde{y}}=G_{i}(x^{y},s_{\tilde{y}})$.

In addition, our architecture incorporates a multi-head discriminator ($D$), a
versatile component that serves dual roles within our framework. This
discriminator takes an image as input, producing a vector whose length
corresponds to the number of classes. Each element in this vector is a
“grade”, reflecting the authenticity of the input image concerning the class
it represents. Beyond its role as a discriminator, $D$ also serves as a
classifier, effectively classifying the input image by selecting the class
with the highest “grade” using argmax of its output vector.

Lastly, we have a pre-trained classifier ($C$), for which DXAI is computed. In
this context, the generators aim to deceive the classifier by aligning their
outputs with the intended class representation. Here, our multi-head
discriminator’s primary function is to evaluate the extent to which the
generators successfully mislead the pre-trained classifier while ensuring the
overall quality and realism of the generated images.

###  3.3 Training

We list below the essential training procedure and losses. See Fig. 5 for an
overall diagram of the general architecture, consisting of several style-
transfer generators, our proposed $\alpha$-blending procedure, style and
quality discriminator and classifier. More details, including ablation
studies, appear in the supplementary material.

Figure 5: Training process diagram. The class distinct part (existence/absence
of white rectangle) is in the first branch (top, in red), whereas the class
agnostic components, which belong to both classes, are generated by subsequent
branches. The $\alpha$-blended generation is a major mechanism controlling the
training.

New $\alpha$-blended generation. In order for the first channel to contain
class distinct information, we propose a novel $\alpha$-blending mechanism.
For each batch, a random vector $\alpha$ of length $n-1$ is drawn, where each
element is uniformly distributed in the range $[0,1]$. Two images are then
generated during training as follows:

$\displaystyle\hat{x}^{y}=\psi_{1}^{y}+\sum\limits_{i=2}^{n}\alpha_{i-1}\psi_{i}^{y}+\sum\limits_{i=2}^{n}(1-\alpha_{i-1})\psi_{i}^{\tilde{y}},$
(5)
$\displaystyle\hat{x}^{\tilde{y}}=\psi_{1}^{\tilde{y}}+\sum\limits_{i=2}^{n}\alpha_{i-1}\psi_{i}^{\tilde{y}}+\sum\limits_{i=2}^{n}(1-\alpha_{i-1})\psi_{i}^{y},$

where $y$ is the class of the input image and $\tilde{y}$ represents a random
alternative class, $\tilde{y}\neq y$. The proposed method encourages the
generators to generate identical images for both components in the sum
$\psi_{i}^{y}\approx\psi_{i}^{\tilde{y}}$, $i=2,..\,,n$, and thus to isolate
the distinction between the classes to $\psi_{1}$. In the ideal case, where
the components in the sum are identical and the distinction is only in
$\psi_{1}$ we converge to Eq. 3. The proposed $\alpha$-blending method allows
a stable and effective training. We note that other alternatives, such as
attempting to use norm-based losses, e.g.
$\|\psi_{i}^{y}-\psi_{i}^{\tilde{y}}\|$, often yield degenerate solutions,
with $\psi_{i}^{y}\approx 0$.

Classification loss. Since a pre-trained classifier is integrated into our
system, there is no need to further train it on authentic images. Instead, we
leverage its classification and attempt to explain it. We enable the
generators to produce images that correspond to the classifier’s predictions
through the following loss function:

$L_{class-
fake}=CrossEntropy\Bigl{(}C\Bigl{(}G(x^{y},s_{y_{trg}})\Bigl{)},y_{trg}\Bigl{)}.$
(6)

In our GAN-based model, the discriminator is used also as a classifier, in
addition to its classical role. This is in order to distinguish between real
and fake images in each class. We use a Kullback-Leibler divergence loss
between the classification output of the discriminator and that of the pre-
trained classifier $C$. This promotes having a high value in the discriminator
output only for images which appear real and fit the correct class.

Reconstruction loss.

Our generated image $\hat{x}^{y}$ is only an approximation of $x$, see Eqs.
(3), (5). To obtain a good approximation, $\hat{x}^{y}\approx x$, we use a
fidelity measure, based on $L^{1}$ and $L^{2}$ norms,

$d(u,v)=\|u-v\|_{L^{1}}+\|u-v\|_{L^{2}},$ (7)

to penalize small and large changes. We would also like the style transferred
class to be similar to the input image. Thus, the reconstruction loss is with
respect to the generated images of both classes,

$L_{rec}=d(x,\hat{x}^{y})+d(x,\hat{x}^{\tilde{y}}),$ (8)

where $\hat{x}^{y}$, $\hat{x}^{\tilde{y}}$ are given in (5).

DXAI and zero distinction. Let us see how the training process and losses
above approximate the DXAI problem. The reconstruction loss promotes
$x\approx\hat{x}^{y}\approx\hat{x}^{\tilde{y}}$ . The classification loss
promotes that $\hat{x}^{\tilde{y}}$ belongs to class $\tilde{y}\neq y$. Thus,
the shared components $\psi_{i}^{y}\approx\psi_{i}^{\tilde{y}}$, $i=2,..\,n$,
should belong to the class-agnostic part. The effects of branch
specialization, as shown in [6], encourage each $\psi_{i}$ to contain
different image characteristics. Since class $\tilde{y}$ is random, following
(4), we get that $\psi_{Agnostic}$ is not committed to any specific class.

Figure 6: Effect of resetting the distinction branch. This reset yields
enhanced and cleaner class explanation (cars), in contrast to less prominent
and possibly negative explanation otherwise.

In addition, we chose to set $\psi_{1}^{\tilde{y}}=0$ in Eq. 5. This is in
line with the DXAI formulation, Eq. 2. We would like to choose the agnostic
part which is closest to the input image. We explain below additional benefits
of this setting. Since our algorithm is of additive nature, it can offer two
types of explanations. The more intuitive approach is to highlight unique
class features positively, effectively adding distinctiveness to an image with
neutral attributes. This ensures the appearance of the differences in the
distinction map. Alternatively, it is possible also to _subtract_ distinct
features. Negative explanations are less preferred, since they are less
intuitive for class explanation. For instance, when our classifier predicts an
image contains cars, we prefer to receive an image of cars in
$\psi_{Distinct}$, rather than a subtraction when it predicts the absence of
cars. Setting the distinction of the alternative class to zero diminishes
negative explanations. Moreover, this promotes the full network to produce as
realistic images as possible by the alternative generators, reducing spurious
features and undesired details in the class-distinct component, see Fig. 6.

Orthogonality loss. We also incorporate a loss function to promote low
correlation between branches in the feature space. This loss is defined as:

$L_{orth}=\sum_{l=1}^{L}\sum_{i\neq
j}(\textrm{corr}(F^{l}_{i},F^{l}_{j})-I)^{2},$ (9)

where corr is a correlation matrix, $F^{l}_{i}$ are the features of the $i$’th
generator at layer $l$ and $I$ is the identity matrix. Although this loss is
not mandatory, we noticed it reduces overall correlation of $\psi$’s and
emphasizes unique characteristics in each component.

The total loss is a weighted sum of the mentioned losses, in addition to the
standard adversarial loss of GAN architecture [8]. Inference is performed by
providing an image $x$ and a class $y$ (the inference diagram and additional
training and inference details are in the supplementary).

Figure 7: Linear reduction in class distinction. We examine the classification results of the following generated images $x_{\beta}=x-\beta\cdot\psi_{Distinct}$. The bar plots show the average probabilities (softmax output) for each class, from left: Cat, Dog, Wild. An image example of $x_{\beta}$ is shown for each class. We observe that as $\beta$ increases, class distinction decreases, where for $\beta=1$ we have $x_{\beta}=\psi_{Agnostic}$. Ideally we should obtain equal probabilities for all classes, we observe our algorithm produces some overshoots toward alternative classes.  | Datasets |  |  |  |
---|---|---|---|---|---
Method | Peppers | AFHQ | CelebA | BraTS | DOTA
Ours | 0.542 | 0.754 | 0.651 | 0.649 | 0.608
LRP [5] | 0.971 | 0.999 | 0.999 | 0.997 | 0.963
IG [30] | 0.881 | 0.908 | 0.946 | 0.934 | 0.828
G-SH [19] | 0.87 | 0.907 | 0.947 | 0.935 | 0.831
I-Inf. [21] | 0.865 | 0.995 | 0.996 | 0.928 | 0.844
GGC [29] | 1.0 | 1.0 | 1.0 | 0.999 | 0.981
GC [29] | 0.966 | 1.0 | 0.999 | 0.996 | 0.901
Random | 0.914 | 0.993 | 0.891 | 0.891 | 0.881
Table 1: AUC of the curve: classification accuracy of $x_{\beta}$ as a function of $\beta$, see Eq. 10, with $\beta$ ranging from 0 to 1. _Lower AUC is better_. See full explanation in Sec. 4. Figure 8: BraTS [25]: Tumors in MRI images, class-distinct and class-agnostic parts. The dataset included two classes, with and without tumors.  Ours | LRP | IG | G-SH | II | GGC | GC
---|---|---|---|---|---|---
0.125 | 0.156 | 0.171 | 0.172 | 0.177 | 0.179 | 0.192
Table 2: AUC comparison of the Faithfulness index on the BraTS dataset among various XAI methods. Ours is compared to LRP, Integrated gradients, Gradient SHAP, Internal influence, Guided Grad-Cam and Grad-Cam. AUC values were calculated based on the deletion of pixels, with up to 20% removed. When our CD part may be interpreted as importance (by pixel amplitude) one can validate our accurate XAI mapping also by standard evaluation methods.  | DOTA (cars) |  | AFHQ |  | Celeb A |
---|---|---|---|---|---|---
AUC according to: | ResNet18 | Simple | ResNet18 | Simple | ResNet18 | Simple
Trained by ResNet18 | 0.793 | 0.903 | 0.737 | 0.759 | 0.616 | 0.707
Trained by Simple-Classifier | 0.914 | 0.729 | 0.865 | 0.623 | 0.69 | 0.609
Table 3: Matching explanation to classifier. At each row DXAI training is
performed using a different classifier. AUC (similiar to Tab. 1) is shown,
computed using the two classifiers, on three different data sets. AUC is
smaller (better) when the classifier in training and the one used to compute
the AUC match. This shows the relevance of the DXAI explanations to the
classifier. Figure 9: Cars (DOTA [34]): Class-distinct and Class-agnostic
parts. The dataset included two classes, images with and without cars.  Figure
10: CelebA [15]: Class-distinct and Class-agnostic parts. The dataset included
male and female classes.  Figure 11: Peppers [31]: Class-distinct and Class-
agnostic parts. The dataset included four classes of different colored
peppers.

##  4 Experiments

We show here various experiments examining the validity of our algorithm and
the applicability of using the proposed method to infer useful classification
explanations for various diverse data sets. Unless noted otherwise, we use
ResNet18 [16] as our image classifier. The classification problem is solved
sufficiently well in all cases with about 95% or higher test accuracy.

In Fig. 7 we show a linear progression from the original image to the agnostic
part by generating the images,

$x_{\beta}=x-\beta\cdot\psi_{Distinct},$ (10)

where $\beta\in\\{0,0.25,0.5,0.75,1\\}$. For $\beta=0$ we have the original
image and for $\beta=1$ the agnostic part. The average probability vector
$p(x_{\beta})$ is depicted, averaged for each class. A single image for each
class is shown, illustrating this progression. We see class distinction
diminishes. Note we do not obtain precise agnostic images for $\beta=1$ but
only an approximation.

In Table 1 we show the results of a quantitative experiment comparing our
method to other possible XAI decompositions. In this experiment we follow Eq.
10 for $\beta$ ranging from $0$ to $1$ in increments of $0.1$. As $\beta$
grows accuracy should drop. We check the area under the curve (AUC) of
accuracy vs. $\beta$, where lower AUC is better. More details appear in the
supplementary. Since, as far as we know, we are the first to propose DXAI, we
obtain a decomposition based on established XAI algorithms, implemented using
Captum library [19]. One can produce a decomposition from a heatmap $H\geq 0$
by a normalization, having a weight for each pixel,
$w=\frac{H}{\max(H)}\in[0,1],$ and defining for an image $x$,
$\psi_{Distinct}:=w\cdot x$, $\psi_{Agnostic}:=(1-w)\cdot x$. We get a
deomposition in the form of Eq. 1, such that
$x=\psi_{Distinct}+\psi_{Agnostic}$, see examples in Fig. 4. Table 1
demonstrates that for all data sets our decomposition outperforms all other
methods by a considerable margin. This indicates our proposed method has
evidently different qualitative properties, such that trivial manipulations of
the heatmap cannot generate high quality class-distinct components.

Our algorithm does not provide importance ranking of each pixels, with respect
to its contribution for the classification. This is one of its limitations.
Thus we cannot use the standard way to evaluate XAI accuracy by removing
pixels gradually based on importance, as done e.g. in [28]. In some problems,
however, a simple ranking can be inferred by our algorithm. In a binary
classification problem, when one class is predominantly decided based on the
existence of certain lighter pixels, we can use the amplitude of pixels in
$\psi_{Distinct}$ as a reasonable importance ranking. In this case standard
XAI evaluation can be made. The BraTS dataset [25] is such a case. It contains
MRI scans of the human brain used for brain tumor segmentation research. The
dataset includes scans from patients with brain tumors, along with expert
annotations for tumor regions. In many cases, bright regions indicate evidence
of tumors. We divided the dataset into two classes: images containing tumors
and images that do not. Some example results are shown in Fig. 8. Table 2
shows the standard AUC evaluation on this set (AUC of accuracy vs. number of
pixels removed, ordered by importance). We see our algorithm behaves favorably
both qualitatively and quantitatively.

An XAI algorithm should naturally depend on the specific classifier $C$ at
hand. Different classifiers may yield different class distinct and class
agnostic parts. We check this is indeed the case in Table 3. We compare our
results when $C$ is ResNet18 and when $C$ is a simpler classifier with fewer
layers, referred to as “Simple” (yielding less accurate results, details in
the supplementary). We compare AUC as done for Tab. 1. Here however, the
accuracy graph is computed twice - using each of the above classifiers. We
show AUC drops more sharply when $C$ used for obtaining $\psi_{Distinct}$ in
Eq. 10 matches the classifier for computing the AUC. The experiment
demonstrates that our map effectively captures the characteristics of the
target classifier.

In Figs. 9, 10 and 11 we show additional examples for various datasets. In
general, our method is especially effective in cases where the differences
between the classes have an additive nature. This occurs in situations where
the differences involve textures (head- and facial-hair in Fig. 10), colors
(Fig. 11), and the presence or absence of details, such as cars (Fig. 9), red
lips or heavy eyebrows (Fig. 10).

##  5 Discussion and Conclusion

We propose an alternative way to analyze and to visualize the reasons for
classification by neural networks. It is based on decomposing the image into a
part which does not contribute to the classification and to one which holds
the class-related cues. This approach may not be ideal for all applications
and has several _limitations and drawbacks_ , compared to standard XAI
methods: 1\. The method requires training, for a specific training set and
classifier (training is slow, inference is fast); 2\. There is no natural
ranking of the significance of pixels in the image. The amplitude of pixels in
the class-distinct part can serve as a good approximation; 3\. Our
implementation uses GANs. We note that image realism is of secondary
significance for XAI. Our concept does not rely on a GAN architecture and
improvements may be achieved by diffusion-type generative models, such as [17,
36, 33, 7].

Major advantages afforded by this method are of obtaining detailed, dense,
high-resolution, multi-channel information for class distinction. We propose,
for the first time, a decomposition problem for XAI purposes, allowing to
clearly visualize also image components which are not relevant for the
classification task (agnostic part). We examine and compare our method in the
context of several classification tasks and data sets, showing the
applicability and additional insights afforded by this new approach.

## References

  * Aharon et al. [2006] Michal Aharon, Michael Elad, and Alfred Bruckstein.  K-svd: An algorithm for designing overcomplete dictionaries for sparse representation.  _IEEE Transactions on signal processing_ , 54(11):4311–4322, 2006.
  * Alicioglu and Sun [2022] Gulsum Alicioglu and Bo Sun.  A survey of visual analytics for explainable artificial intelligence methods.  _Computers & Graphics_, 102:502–520, 2022.
  * Aujol and Chambolle [2005] Jean-François Aujol and Antonin Chambolle.  Dual norms and image decomposition models.  _International journal of computer vision_ , 63:85–104, 2005.
  * Aujol et al. [2006] Jean-François Aujol, Guy Gilboa, Tony Chan, and Stanley Osher.  Structure-texture image decomposition—modeling, algorithms, and parameter selection.  _International journal of computer vision_ , 67:111–136, 2006.
  * Binder et al. [2016] Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek.  Layer-wise relevance propagation for neural networks with local renormalization layers.  In _Artificial Neural Networks and Machine Learning–ICANN 2016: 25th International Conference on Artificial Neural Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part II 25_ , pages 63–71. Springer, 2016.
  * Brokman and Gilboa [2022] Jonathan Brokman and Guy Gilboa.  Analysis of branch specialization and its application in image decomposition.  _arXiv preprint arXiv:2206.05810_ , 2022.
  * Chen et al. [2023] Jingwen Chen, Yingwei Pan, Ting Yao, and Tao Mei.  Controlstyle: Text-driven stylized image generation using diffusion priors.  In _Proceedings of the 31st ACM International Conference on Multimedia_ , pages 7540–7548, 2023.
  * Choi et al. [2020] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha.  Stargan v2: Diverse image synthesis for multiple domains.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020.
  * Daubechies [1992] Ingrid Daubechies.  _Ten lectures on wavelets_.  SIAM, 1992.
  * Donoho et al. [2005] David L Donoho, Michael Elad, and Vladimir N Temlyakov.  Stable recovery of sparse overcomplete representations in the presence of noise.  _IEEE Transactions on information theory_ , 52(1):6–18, 2005.
  * Dwivedi et al. [2023] Rudresh Dwivedi, Devam Dave, Het Naik, Smiti Singhal, Rana Omer, Pankesh Patel, Bin Qian, Zhenyu Wen, Tejal Shah, Graham Morgan, et al.  Explainable ai (xai): Core ideas, techniques, and solutions.  _ACM Computing Surveys_ , 55(9):1–33, 2023.
  * Fadili et al. [2009] M Jalal Fadili, Jean-Luc Starck, Jérôme Bobin, and Yassir Moudden.  Image decomposition and separation using sparse representations: An overview.  _Proceedings of the IEEE_ , 98(6):983–994, 2009.
  * Ghassemi et al. [2021] Marzyeh Ghassemi, Luke Oakden-Rayner, and Andrew L Beam.  The false hope of current approaches to explainable artificial intelligence in health care.  _The Lancet Digital Health_ , 3(11):e745–e750, 2021.
  * Gkartzonika et al. [2022] Ioanna Gkartzonika, Nikolaos Gkalelis, and Vasileios Mezaris.  Learning visual explanations for dcnn-based image classifiers using an attention mechanism.  In _European Conference on Computer Vision_ , pages 396–411. Springer, 2022.
  * Guo et al. [2016] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao.  Ms-celeb-1m: A dataset and benchmark for large-scale face recognition.  In _Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14_ , pages 87–102. Springer, 2016.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Jeanneret et al. [2022] Guillaume Jeanneret, Loïc Simon, and Frédéric Jurie.  Diffusion models for counterfactual explanations.  In _Proceedings of the Asian Conference on Computer Vision_ , pages 858–876, 2022.
  * Kamakshi and Krishnan [2023] Vidhya Kamakshi and Narayanan C Krishnan.  Explainable image classification: The journey so far and the road ahead.  _AI_ , 4(3):620–651, 2023.
  * Kokhlikyan et al. [2020] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson.  Captum: A unified and generic model interpretability library for pytorch, 2020.
  * Komorowski et al. [2023] Piotr Komorowski, Hubert Baniecki, and Przemyslaw Biecek.  Towards evaluating explanations of vision transformers for medical imaging.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 3725–3731, 2023.
  * Leino et al. [2018] Klas Leino, Shayak Sen, Anupam Datta, Matt Fredrikson, and Linyi Li.  Influence-directed explanations for deep convolutional networks.  In _2018 IEEE international test conference (ITC)_ , pages 1–8. IEEE, 2018.
  * Lundberg and Lee [2017] Scott M Lundberg and Su-In Lee.  A unified approach to interpreting model predictions.  _Advances in neural information processing systems_ , 30, 2017.
  * Mallat [1989] Stephane G Mallat.  A theory for multiresolution signal decomposition: the wavelet representation.  _IEEE transactions on pattern analysis and machine intelligence_ , 11(7):674–693, 1989.
  * Marx et al. [2023] Charles Marx, Youngsuk Park, Hilaf Hasson, Yuyang Wang, Stefano Ermon, and Luke Huan.  But are you sure? an uncertainty-aware perspective on explainable ai.  In _International Conference on Artificial Intelligence and Statistics_ , pages 7375–7391. PMLR, 2023.
  * Menze et al. [2014] Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al.  The multimodal brain tumor image segmentation benchmark (brats).  _IEEE transactions on medical imaging_ , 34(10):1993–2024, 2014.
  * Mertes et al. [2022] Silvan Mertes, Tobias Huber, Katharina Weitz, Alexander Heimerl, and Elisabeth André.  Ganterfactual—counterfactual explanations for medical non-experts using generative adversarial learning.  _Frontiers in artificial intelligence_ , 5:825565, 2022.
  * Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ” why should i trust you?” explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1135–1144, 2016.
  * Samek et al. [2016] Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller.  Evaluating the visualization of what a deep neural network has learned.  _IEEE transactions on neural networks and learning systems_ , 28(11):2660–2673, 2016.
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE international conference on computer vision_ , pages 618–626, 2017.
  * Sundararajan et al. [2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic attribution for deep networks.  In _International conference on machine learning_ , pages 3319–3328. PMLR, 2017.
  * Thi Phuong Chung and Van Tai [2019] Dang Thi Phuong Chung and Dinh Van Tai.  A fruits recognition system based on a modern deep learning technique.  In _Journal of physics: conference series_ , page 012050. IOP Publishing, 2019.
  * Vermeire et al. [2022] Tom Vermeire, Dieter Brughmans, Sofie Goethals, Raphael Mazzine Barbossa de Oliveira, and David Martens.  Explainable image classification with evidence counterfactual.  _Pattern Analysis and Applications_ , 25(2):315–335, 2022.
  * Wang et al. [2023] Zhizhong Wang, Lei Zhao, and Wei Xing.  Stylediffusion: Controllable disentangled style transfer via diffusion models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 7677–7689, 2023.
  * Xia et al. [2018] Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang.  Dota: A large-scale dataset for object detection in aerial images.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3974–3983, 2018.
  * Xue et al. [2022] Mengqi Xue, Qihan Huang, Haofei Zhang, Lechao Cheng, Jie Song, Minghui Wu, and Mingli Song.  Protopformer: Concentrating on prototypical parts in vision transformers for interpretable image recognition.  _arXiv preprint arXiv:2208.10431_ , 2022.
  * Yang et al. [2023] Serin Yang, Hyunmin Hwang, and Jong Chul Ye.  Zero-shot contrastive loss for text-guided diffusion image style transfer.  _arXiv preprint arXiv:2303.08622_ , 2023.
  * Zhang et al. [2022] Xiaoge Zhang, Felix TS Chan, and Sankaran Mahadevan.  Explainable machine learning in image classification models: An uncertainty quantification perspective.  _Knowledge-Based Systems_ , 243:108418, 2022.

##  Appendix A Detailed description of the method

###  A.1 Training

In the main paper, we presented the training method and primary loss
functions. However, for brevity, we omitted details on functions that are
neither innovative nor crucial for understanding the method’s general concept.
Additionally, these functions serve similar purposes to those already
discussed.

One such function, integral to the GAN architecture implementation but not
explicitly mentioned, is the adversarial loss as defined in [8]

$L_{adv}=\mathop{\mathbb{E}}_{x,y}\left[logD_{y}(x^{y})\right]+\mathop{\mathbb{E}}_{x,\tilde{y}}\left[log(1-D_{\tilde{y}}(\hat{x}^{\tilde{y}}))\right],$
(11)

where $D_{y}$ represents the $y$’th element of a vector of length $c$, which
is an output of the discriminator. Its role is to ensure that the generators
produce images resembling real ones, for a given class.

Another function, briefly mentioned, directs our discriminator not only to
classify images but also to align with the classifications of our trained
classifier. This encourages the discriminator to assign a high score only to
the correct class according to the classifier

$L_{KLD}=KLD\Bigl{(}D(x^{y}),C(x^{y})\Bigl{)},$ (12)

where $KLD$ is the Kullback-Leibler distance.

In addition to the reconstruction loss explained in the main part, one can use
additional constraints on the reconstruction to enhance results. Specifically,
we observed challenges in reproducing areas with significant differences
between classes. To address this, we incorporated an additional constraint for
reconstruction between pixels with a high amplitude in the distinction branch.
High amplitude signifies differences between classes due to the additive
nature of the model. We remind that we use the following distance measure

$d(u,v)=\|u-v\|_{L^{1}}+\|u-v\|_{L^{2}},$

with a reconstruction loss of the form

$L_{rec}=d(x,\hat{x}^{y})+d(x,\hat{x}^{\tilde{y}}).$

The proposed loss function is

$L_{dis-rec}=d(x\odot\mathbb{I},\hat{x}^{y}\odot\mathbb{I}),$ (13)

where

$\mathbb{I}=\begin{cases}1&\text{if $|\psi_{1}^{y}|>mean(|\psi_{1}^{y}|)$}\\\
0&\text{else},\end{cases}$

and $\odot$ denotes element-wise product.

The full objective loss is:

$\displaystyle
L_{Total}=\lambda_{adv}L_{adv}+\lambda_{KLD}L_{KLD}+\lambda_{cf}L_{class-
fake}$ (14) $\displaystyle+\lambda_{rec}L_{rec}+\lambda_{dr}L_{dis-
rec}+\lambda_{o}L_{orth}.$

In our experiments we used the following weights:

$\displaystyle\lambda_{adv}=2,\lambda_{KLD}=1,\lambda_{cf}=2,$ (15)
$\displaystyle\lambda_{rec}=4,\lambda_{dr}=4,\lambda_{o}=1.$

A note on how weights can be adjusted. The rule of thumb is that for
applications where particularly good reconstruction is required, the weights
for reconstruction should be increased. Conversely, when style transition is
challenging and there are more hidden characteristics, the weights for
classification and adversarial loss should be increased.

###  A.2 Inference Stage

The inference stage is described in Fig. 12. The input $x^{y}$ is an image $x$
of class $y$. The class predicted by the classifier is $\hat{y}$. We would
like to explain the classification result by the trained DXAI model. The class
distinct part is given by the first generator. The rest of the summed branches
provide the agnostic part.

Figure 12: Inference stage: the input image and the predicted class are used to guide the generators. The output is comprised of the class distinction map in the first branch and the agnostic part by a sum of the rest of the branches. Datasets | Train set | Test set
---|---|---
AFHQ [8] | 14630 | 1500
CelebA [15] | 28000 | 2000
DOTA(cars) [34] | 172465 | 20696
BraTS [25] | 15000 | 1232
Peppers [31] | 2478 | 826
Tomatoes [31] | 5103 | 1707
Apples [31] | 6404 | 2134
Table 4: Number of images in each dataset. Datasets | ResNet18 | Simple | Discriminator
---|---|---|---
AFHQ | 0.975 | 0.906 | 0.994
CelebA | 0.978 | 0.96 | 0.988
DOTA(cars) | 0.863 | 0.749 | 0.858
BraTS | 0.912 | 0.877 | 0.982
Peppers | 1 | 0.944 | 1
Tomatoes | 1 | 0.985 | 1
Apples | 0.983 | 0.909 | 1
Table 5: Test accuracy of each classifier. Simple-Classifier Architecture | Dimension
---|---
Input image | 256x256x3
Conv2d(3, 16, 3x3) | 256x256x16
MaxPool2d(3x3) | 85x85x16
ReLU |
Conv2d(16, 32, 3x3) | 85x85x32
MaxPool2d(3x3) | 28x28x32
Linear(in=28*28*32, out=128) | 128
ReLU |
Linear(in=128, out=c) | $c$
Table 6: Simple-Classifier Architecture from input to output. $c$ is the number of classes and input image can be also a gray scale image. Disriminator Architecture | Dimension
---|---
Input image | 256x256x3
Conv1x1 | 256x256x64
ResBlk | 128x128x128
ResBlk | 64x64x256
ResBlk | 32x32x512
ResBlk | 16x16x512
ResBlk | 8x8x512
ResBlk | 4x4x512
Leaky-ReLU |
Conv4x4 | 1x1x512
Leaky-ReLU |
Conv1x1 | $c$
Table 7: Discriminator Architecture as given in [8] from input to output. $c$
is the number of classes and unlike the original architecture the input image
can be also a gray scale image. Figure 13: Noise distinction. Celeb-A where
one class contains clean images and the other class contains noisy images,
contaminated by additive white Gaussian noise (AWGN). In this case, standard
XAI algorithms yield partial and inaccurate explanation of the class
distinction. Our algorithm provides a correct explanation distincting the two
classes by isolating well the noise.

###  A.3 Datasets

In the main paper several datasets were used. We provide some more technical
details on those sets. In Tab. 4 the number of images of each set is given.
Some datasets were adjusted or filtered to suit the classification task. For
instance, in the case of DOTA [34] , which contains high-resolution aerial
images with objects marked in bounding boxes, we divided the images into
patches of size 256x256 pixels and categorized them into two classes. One
class comprised images featuring at least 70% of a single car (in many cases
there are multiple cars), while the other class included images with no cars
at all. The classifier was trained to distinguish between these two classes of
patches. As expected, the explanation (distinct) image is comprised of
isolated cars.

To identify fruits, we employed the Fruits 360 dataset [31]. From there, we
selected specific classes of peppers, tomatoes, and apples, which mainly
differ in color and texture, and created a dataset for each.

For BraTS [25], we only considered a subset of the images—specifically, those
in which the tumor occupied an area of at least 20 pixels were included in the
class designated to contain tumors. Images that did not contain a tumor at all
but featured a full cross-section of the brain were included in the second
class. Subsequently, we randomly split the images into train and test sets.

No modifications were made to the remaining datasets.

###  A.4 Classifiers

In our experiments three different classifiers were employed for the
classification block $C$: a ResNet18-type classifier, a simple classifier
described in Tab. 6, and our discriminator $D$, which possesses classification
capabilities, as the output is a vector of length $c$, indicating class
probabilities (details of the architecture are in Tab. 7). The test accuracy
of each classifier is given in Tab. 5. We note that the discriminator exhibits
the best classification performance. Thus, we selected it in the main paper to
show the explanation images by the various DXAI and XAI algorithms. Note that
we found a mistake in the explanation text of the paper, which will be
corrected in the final version. We conducted experiments also on ResNet18 and
on the Simple classifier, showing consistently similar trends. We provide
later some examples of ResNet18 results on the same datasets.

The intentionally designed simple classifier, with lower classification
ability, provides distinct classification results from a well-known classifier
like ResNet18. This intentional dissimilarity allows for meaningful
comparisons of their results, as detailed in the main paper.

(a)

(b)

Figure 14: AFHQ ablation - need of more than 2 branches. The run of the
experiment was stopped after 150K iterations, after the tendency was
understood.

(a)

(b)

Figure 15: CelebA ablation - need of more than 2 branches. The run of the
experiment was stopped after 150K iterations, after the tendency was
understood.

(a)

(b)

Figure 16: Ablation of $L_{dis-rec}$, it can be seen that without this loss
function the PSNR is reduced

##  Appendix B Ablation

In the ablation study, we investigated two key factors. Firstly, we examined
the impact of the number of branches. While decomposing the classified image
into several images, we primarily focus on solving only two images—distinct
and agnostic. One might question why not use only two branches for the
solution, since two branches can be trained to achieve a similar solution.

We show that when branches are used for the agnostic part results are better
both in terms of reconstruction and of the generators’ ability to produce
images that explain the classifier. For instance, as demonstrated in Figs. 14
and 15, PSNR decreases when using only two branches. Additionally, the loss
$L_{class-fake}$, representing the generators’ ability to produce meaningful
images of a specific class according to the given classifier, is higher for
the entire training with two branches. In other words, the classifier
interprets the images less accurately as the desired class, making the images
less reliable. The additional generators provide better generation capacity
and can be trained in an easier and more stable manner.

In addition, we evaluated the impact of the loss $L_{dis-rec}$ described in
Eq. 13 on the reconstruction quality. As explained earlier, we employed it
because we observed that reconstruction, especially in areas with differences
between the classes, was challenging. We conducted experiments both with and
without it. We show that it indeed contributes to the quality of the
reconstruction in terms of PSNR, as illustrated in Fig. 16.

It should be noted that, due to time constraints, we did not run the ablation
experiments until the convergence of an optimal solution. Instead, we stopped
the training after observing a discernible trend in the change of solution
quality.

##  Appendix C More results

In the main paper we presented qualitative results of our algorithm for
different datasets with selected comparisons to other methods. Here we provide
more detail. We show for each image the full comparison to 6 established XAI
methods. We also provide more examples in each dataset, so the reader can
judge better the quality and stability of our algorithm. We show the results
for the case $C=D$ where the classifier is the discriminator (where
classification accuracy is best) and for $C=ResNet18$, cases which were not
shown in the main paper. In Fig. 17 we show comparison between Female and Male
classes. There are many subtle cues for such distinction. One can observe that
our method provides dense informative explanations which highlight well the
distinctions. In Fig. 18 the peppers are shown, which differ mainly by color,
as our proposed method shows clearly. Heatmap visualization cannot show well
global color explanations. In Fig. 19 we show that the cars are well isolated
in high resolution by our method. This case is less dense and can be handled
well also by some other methods (such as LRP, Guided grad-cam or Gradient
SHAP). Our method appears to visualize the explanation very clearly. In Fig.
20 and Fig. 21 we show additional datasets of tomatoes and apples classes,
which did not appear in the main paper. Here again color changes are a main
feature of distinction, which is shown clearly by our method. In Fig. 22
additional examples of the BraTS dataset are provided, showing our ability to
isolate the lesions well. In Fig. 23 we observe the dense distinction images
of animal classes, where different fur textures are significant features.
These are very hard to visualize by heatmaps. The above images were using the
discriminator as the classifier. In Fig. 24, Fig. 25, Fig. 26 and Fig. 27 we
show results for ResNet18 as the classifier. We can expect that since both
classifiers are relatively advanced, with high precision, they consider
similar features. We can observe that our method produce qualitatively quite
similar results and is stable. For the peppers case, Fig. 27, we observe a
stronger difference, where the agnostic part is more of a mix of orange and
red colors. In any case, the color distinction is clear and is much better
visualized than in standard XAI methods. In Figs 28, 29, 30, 31 the
classification explanations for the Simple classifier are shown. One can
observe that the distinction map is less accurate, more blurry, containing
some spurious background features. This can be expected for a lower quality
classifier.

Figure 17: CelebA (Female and Male classes, $C=D$). The class distinct part
shows for Female distinction by red lips, hair texture, eye-lashes and
eyebrows. Fro Male – facial hair, heavy eyebrows, strong chin and visibility
of ears. Other XAI algorithms capture only parts of the full explanation for
such a complex classification task, which involves many subtle cues. Figure
18: Peppers dataset ($C=D$). Class distinction is mainly by color, which is
well visualized by our explanation method. Figure 19: DOTA (cars) dataset
($C=D$). Cars are isolated well in the distinction part. Figure 20: Tomatoes
dataset ($C=D$). Distinction is mostly based on color (and some global
texture). Figure 21: Apples dataset ($C=D$). Distinction by color is well
captured by our methods. Figure 22: BraTS dataset ($C=D$). Highly localized
regions of the lesions are visualized well in high resolution by our method.
Figure 23: AFHQ animals dataset ($C=D$). Fur textures, as well as local
features of eyes and nose give a full and rich explanation for class
distinction. Some spurious background features appear, indicating background
cues may contribute, in some cases, for the classifier’s decision. Figure 24:
AFHQ animals dataset ($C=ResNet18$). Here ResNet18 is used as a classifier.
Since both the discriminator and ResNet18 are highly accurate we can expect
mostly similar distinct features. Whereas some XAI algorithm provide similar
heatmaps, others are less stable. This depends also on meta-parameters, such
as layer selection. Our algorithm shows only minor decomposition differences
and appears stable. Figure 25: CelebA dataset ($C=ResNet18$). Figure 26: DOTA
(cars) dataset ($C=ResNet18$). Figure 27: Peppers dataset ($C=ResNet18$). Here
we get different agnostic and distinction parts, compared to the case $C=D$.
However, color is still the main distinctive feature and is well visualized,
compared to heatmap methods. Figure 28: DOTA (cars) dataset ($C=Simple$). It
can be observed that the distinction map is different and less clear compared
to the maps generated by the discriminator and ResNet18. We hypothesize that
this is due to the classifier’s lower accuracy percentages, leading to a less
precise differentiation between important features and those that are not.
Figure 29: BraTS dataset ($C=Simple$). As in Fig. 28, here too, you can
observe that the distinction maps are less clean, and we assume that this is
due to similar reasons. Figure 30: AFHQ dataset ($C=Simple$). Figure 31:
CelebA dataset ($C=Simple$).