# Signal Is Harder To Learn Than Bias:
Debiasing with Focal Loss

Moritz Vandenhirtz, Laura Manduchi, Ričards Marcinkevičs and Julia E. Vogt
Department of Computer Science, ETH Zurich
{moritz.vandenhirtz, laura.manduchi, ricardsm<EMAIL_ADDRESS>
###### Abstract

Spurious correlations are everywhere. While humans often do not perceive them,
neural networks are notorious for learning unwanted associations, also known
as biases, instead of the underlying decision rule. As a result, practitioners
are often unaware of the biased decision-making of their classifiers. Such a
biased model based on spurious correlations might not generalize to unobserved
data, leading to unintended, adverse consequences. We propose Signal is Harder
(SiH), a variational-autoencoder-based method that simultaneously trains a
biased and unbiased classifier using a novel, disentangling reweighting scheme
inspired by the focal loss. Using the unbiased classifier, SiH matches or
improves upon the performance of state-of-the-art debiasing methods. To
improve the interpretability of our technique, we propose a perturbation
scheme in the latent space for visualizing the bias that helps practitioners
become aware of the sources of spurious correlations.

##  1 Introduction

The generalization capability of deep neural networks (DNN) highly depends on
the quality of the training data. If spurious correlations are present, the
model might ignore the intrinsic signal attributes while still performing
reasonably well in classification tasks. However, such a biased model will not
be robust and will not generalize outside the training distribution. To
increase the trustworthiness of machine learning algorithms and prevent
unwanted consequences, it is crucial to avoid deploying biased models (Geirhos
et al., 2020). Thus, there has been an increased interest in the community to
mitigate this problem. While many methods assume and utilize an observed
variable that captures the source of bias for each data point, recently, some
effort has been made to alleviate this prohibitive assumption (Nam et al.,
2020).

For example, consider a dataset comprising of images of vehicles. A DNN might
implicitly use the bias attribute “sky” as a shortcut for classifying planes
because most images of airplanes are shot while they are in the air.
Throughout the paper, we will call samples bias-aligned when their bias
attributes are strongly correlated with the label. Here, leveraging the bias
as a decision rule leads to the correct predicted label, e.g. airplane in the
sky. Conversely, bias-conflicting data points are the samples for which the
biased decision rule leads to the wrong prediction, e.g. aircraft in the
hangar.

Recent efforts by Nam et al. (2020) aim to eliminate the need for an observed
variable that captures the source of bias for each data point. They assume
that malignant bias attributes are easier to learn than the underlying signal.
Based on this easy-to-learn assumption, they train a biased classifier that
focuses on the easy, bias-aligned samples. Simultaneously, they train an
unbiased classifier by upweighting the remaining hard, bias-conflicting
samples. We propose an alternative reweighting for the unbiased classifier
based on the focal loss (Lin et al., 2017) that does not require the
previously utilized subtle, distorting stability measures. We motivate the
usage of this loss function through the easy-to-learn assumption, which infers
that signal is harder to learn than bias.

In addition, we extend the literature by integrating a variational autoencoder
(VAE) (Kingma & Welling, 2014) into the model. At inference time, this allows
us to make use of latent perturbations to remove the biasing attributes from
the embeddings, which we then feed to the decoder to visualize debiased
images. Comparing these images with the original reconstructions can help
practitioners uncover unknown biases.

#### Contribution

We propose a novel reweighting scheme, coined Signal is Harder (SiH), for
training an unbiased classifier.111Our code is publicly available at
https://github.com/mvandenhi/Signal-is-Harder Due to the lack of labels for
the unknown bias, SiH exploits the assumption that signal is harder to learn
than bias and utilizes a reweighting based on the well-established focal loss
(Lin et al., 2017). We show that this direct mechanism improves the debiasing
capabilities compared to the existing, more complex reweighting scheme by Nam
et al. (2020). Additionally, by training a VAE simultaneously with the
classifiers, the unknown bias can be visualized in the reconstructions. For
this, the proposed algorithm perturbs the latent bias embeddings at inference
time to remove the bias without creating artifacts in the reconstructions. We
improve upon previous methods as our minimal perturbation does not change
other aspects of the reconstruction, unambiguously unveiling the unknown
spurious attribute.

##  2 Related work

#### Separating samples by difficulty

Recent works separate bias-conflicting from bias-aligned samples to train an
unbiased classifier (Nam et al., 2020; Lee et al., 2021; Kim et al., 2021).
This separation can be achieved by differentiating data points through the
difficulty of predicting their label. In a standard classification setting,
Zhang & Sabuncu (2018) propose the Generalized Cross Entropy (GCE) loss to
reduce the weight on samples whose labels are hard to predict:

$GCE(\hat{y},y)=\frac{1-\hat{y}^{q}}{q},$ (1)

where $\hat{y}$ is the predicted probability of the correct label $y$
according to the classifier, and $q\in(0,1]$ is a hyperparameter to control
the strength of emphasis. The GCE is best understood by inspecting its
derivative $\frac{\partial
GCE(\hat{y},y)}{\partial\bm{\theta}}=\hat{y}^{q}\frac{\partial
CE(\hat{y},y)}{\partial\bm{\theta}}$, where $\bm{\theta}$ are the learnable
neural network parameters. This loss upweighs samples that the classifier
already predicts well, ignoring samples for which the current decision rule
does not work. Contrary to the GCE loss, the Focal Loss (FL) by Lin et al.
(2017) puts more focus on hard, misclassified examples:

$FL(\hat{y},y)=(1-\hat{y})^{q}CE(\hat{y},y)$ (2)

With this reweighting scheme, the samples whose labels are hard to predict are
upweighted such that the classifier does not ignore the samples for which
finding a decision rule is a hard problem.

#### Debiasing without supervision

Previous works focused on predictions with respect to known sensitive
attributes (Sagawa et al., 2020; Edwards & Storkey, 2016; Kim et al., 2019),
which are often difficult to retrieve. For this reason Nam et al. (2020)
propose LfF, a new approach to debias a classifier, which does not require
bias attributes. They assume that bias is only malignant if it is easier to
learn than the true signal attribute and leverage the GCE loss to focus on the
easy, bias-aligned samples to train a biased classifier. Simultaneously, they
train an unbiased classifier, designed to learn the true, underlying signal.
For this they upweigh the bias-conflicting samples, i.e. the data points for
which the bias can not be utilized to predict the label, by the relative
difficulty score (RDS)

$RDS(\hat{y}_{s},\hat{y}_{b},y)=\frac{CE(\hat{y}_{b},y)}{CE(\hat{y}_{s},y)+CE(\hat{y}_{b},y)},$
(3)

where $\hat{y}_{s}$ and $\hat{y}_{b}$ are the predicted probabilities of the
correct label $y$ according to the unbiased and biased classifier,
respectively. However, before inserting the CE terms into the above formula,
they apply an empirically motivated exponential moving average and a class-
wise normalization by the maximum CE to each term. In the following section,
we will propose an enhanced upweighting mechanism for the unbiased classifier
that does not require weight-distorting stability measures.

Lee et al. (2021) extend the method of Nam et al. (2020) by additionally
swapping the learned latent bias embeddings of different inputs to decouple
the bias from the label. At inference time, they train a decoder to visualize
the embeddings with and without swapped bias, such that the unknown bias can
be discovered by analyzing the differences between the two reconstructions. To
avoid misleading artifacts in the visualization, we will propose a more
conservative perturbation, which relies on a VAE trained simultaneously with
the classifiers. Further discussion can be found in Appendix A.

##  3 Method

We propose a debiasing algorithm, coined Signal is Harder (SiH), consisting of
a VAE-based architecture and a new weighting mechanism for training the
unbiased classifier. The VAE uses two encoders to map the input into signal
and bias embeddings, which are concatenated and passed through the decoder to
reconstruct the original input. Additionally, we train an unbiased and biased
classifier on signal and bias embeddings, respectively. The biased classifier
is trained by upweighting bias-aligned samples through the GCE loss. In
contrast, the unbiased classifier is trained by upweighting bias-conflicting
samples through our novel focal-loss-based weighting scheme, which we will
introduce in the next paragraph. The generative nature of the model allows us
to produce bias visualizations that help discover the unknown source of bias.
We depict the proposed model structure in Figure 1.

Figure 1: Graphical overview of our model’s structure for a bias-conflicting
image. The input x is passed through the signal and bias encoders $E_{s}$ and
$E_{b}$ to obtain the latent signal and bias embeddings $\textbf{z}_{s}$ and
$\textbf{z}_{b}$, which in this example should be the digit two and the color
blue, respectively. These representations are then passed through their
respective classifier $C_{s}$ and $C_{b}$ to predict the label. Lastly,
$\textbf{z}_{s}$ and $\textbf{z}_{b}$ are concatenated and passed through the
decoder $D$ to reconstruct the image.

#### Reweighting by focal loss

We hereby introduce a new reweighting scheme, aiming to utilize the easy-to-
learn assumption not only for the biased but also for the unbiased classifier.
Similar to the GCE that upweighs bias-aligned samples for the biased
classifier, we want to utilize a mirrored loss function that upweighs the
remaining bias-conflicting samples for the unbiased classifier.

The aforementioned reasons motivate the inclusion of the focal loss for
training the unbiased classifier. We use this loss to identify samples for
which the biased classifier struggles to predict the correct class and
emphasize these presumably bias-conflicting samples by upweighting them when
training the unbiased classifier. As the information learned by the biased
classifier should leverage the unbiased classifier, but not the other way
around, we detach the weighting factor from the computational graph during
backpropagation and obtain the following update for the unbiased classifier:

$\frac{\partial\mathcal{L}_{s}(\hat{y}_{s},\hat{y}_{b},y)}{\partial\bm{\theta}_{s}}=(1-\hat{y}_{b})^{q}\frac{\partial
CE(\hat{y}_{s},y)}{\partial\bm{\theta}_{s}},$ (4)

where $q\in(0,1]$ is a hyperparameter controlling the strength of emphasis.
With this loss, we exploit that bias-conflicting samples are hard to learn for
a biased classifier. By focusing on these data points, the unbiased classifier
is forced to learn the signal, as here, leveraging the bias does not lead to
the correct prediction. Most importantly, the straightforward integration of
the focal loss for training the unbiased classifier removes the need for
weight-distorting stability measures.

#### Latent adversarial perturbation

To make practitioners aware of the unknown spurious correlations in a dataset,
we propose a visualization approach by perturbing the bias at inference time.
We perturb the bias embeddings, such that the bias contained within is
removed, and reconstruct the debiased image. We argue that such a perturbation
should be as small as possible, such that no artifacts are created in the
process since a practitioner needs to consider every change to the input as a
potential bias. To achieve this, we adapt and utilize the adversarial
perturbations from Deepfool (Moosavi-Dezfooli et al., 2016). This algorithm is
designed to find a minimal perturbation to the input that fools the classifier
into predicting the wrong class. Thus, we perturb the bias representations
such that the biased classifier can no longer predict the correct class;
effectively, the perturbation removes the bias from the bias embeddings.

Having trained a VAE, we can use the decoder at inference time to visualize
the perturbed bias embeddings together with the unchanged signal
representation. By comparing the original reconstruction with the debiased
visualization, it is possible to identify the spurious correlations in the
image. In contrast to DisEnt, we train the decoder simultaneously with the
classifiers to encode all image-relevant information in the latent
representations, thus, supporting the unbiased classifier in finding the
signal attributes and improving reconstruction quality.

##  4 Experiments

To compare the performance of our method, SiH, with previous works, we
evaluate it on Colored MNIST (Kim et al., 2019) and Corrupted CIFAR-10
(Hendrycks & Dietterich, 2019) with a varying percentage of bias-conflicting
images during the training. For a detailed description of the datasets, we
refer to Appendix C. We determine three baselines to which we compare the
proposed approach. The first baseline we implement is a Vanilla model
consisting of one encoder and classifier, which measures the standard
performance without any debiasing scheme. The second model we compare the
proposed approach to, is LfF from Nam et al. (2020). Lastly, we compare SiH to
DisEnt by Lee et al. (2021), a recently proposed state-of-the-art debiasing
algorithm, which also visualizes the bias.

###  4.1 Quantitative evaluation

#### Comparison on test sets

In Table 1, we show the performance of all models on the unbiased test set of
Colored MNIST and Corrupted CIFAR-10. The estimates differ from the values
presented in the baseline papers (Nam et al., 2020; Lee et al., 2021) because
we also vary random seeds over dataset generation instead of only over the
weight initialization.

Table 1: Unbiased test set accuracy + standard deviation in %. The method with the significantly highest accuracy is denoted in bold. Otherwise, insignificantly different methods are underlined. Dataset | Ratio | Vanilla | LfF | DisEnt | SiH
---|---|---|---|---|---
Colored MNIST | 20% |  94.92 $\pm$ 0.24 | 70.18 $\pm$ 4.19 | 90.94 $\pm$ 1.46 | 85.24 $\pm$ 1.60
10% |  91.24 $\pm$ 0.26 | 81.99 $\pm$ 5.01 | 89.12 $\pm$ 1.44 | 85.35 $\pm$ 1.23
5% |  85.48 $\pm$ 0.50 | 81.18 $\pm$ 2.94 |  85.54 $\pm$ 2.49 |  86.14 $\pm$ 1.78
2% | 73.28 $\pm$ 0.56 | 76.97 $\pm$ 2.49 |  82.38 $\pm$ 1.68 |  83.80 $\pm$ 1.28
1% | 59.41 $\pm$ 0.39 | 68.91 $\pm$ 5.01 | 76.33 $\pm$ 3.41 |  80.03 $\pm$ 2.04
0.5% | 43.70 $\pm$ 0.83 | 60.42 $\pm$ 2.72 | 63.98 $\pm$ 4.78 |  71.63 $\pm$ 2.49
Corrupted CIFAR-10 | 20% |  67.57 $\pm$ 0.41 | 64.50 $\pm$ 2.17 | 60.99 $\pm$ 5.84 |  66.75 $\pm$ 1.34
10% | 57.11 $\pm$ 0.76 |  59.29 $\pm$ 3.16 | 53.47 $\pm$ 4.43 |  61.26 $\pm$ 2.06
5% | 46.89 $\pm$ 0.78 |  55.77 $\pm$ 2.33 | 46.40 $\pm$ 5.81 |  55.63 $\pm$ 1.54
2% | 34.90 $\pm$ 0.81 |  47.26 $\pm$ 1.56 | 36.98 $\pm$ 4.43 |  43.66 $\pm$ 1.81
1% | 28.22 $\pm$ 0.73 |  39.39 $\pm$ 2.16 | 31.22 $\pm$ 2.69 | 35.17 $\pm$ 1.19
0.5% | 22.26 $\pm$ 1.03 |  30.04 $\pm$ 1.67 |  31.97 $\pm$ 3.34 | 27.30 $\pm$ 2.04

We observe that Vanilla outperforms the debiasing algorithms for the 10% and
20% cases of Colored MNIST. Thus, the easy-to-learn assumption is likely not
fulfilled for these training sets. The debiasing methods show their benefit
only for a lower amount of bias-conflicting samples. Here, SiH outperforms or
at least matches all baselines, while DisEnt is the runner-up. Especially for
the 0.5% setting, there is a considerable gap in performance between our and
other methods.

For Corrupted CIFAR-10, the best-performing models are LfF and SiH. We observe
that for higher percentages of bias-conflicting samples, SiH is better than
LfF, while for lower proportions, the opposite is the case. DisEnt seems to be
generally worse than the other debiasing methods. Finally, Vanilla performs
worse than the debiasing methods except for the 20% case. Thus, the debiasing
methods present an improvement over a standard empirical risk minimizer.

#### Focal loss vs. RDS weighting

Figure 2: A random collection of bias visualizations for Colored MNIST. The
randomly selected images are varied over random seeds and the percentage of
bias-conflicting images in the training set.

Table 2: Unbiased accuracy + standard deviation in % for Colored MNIST and
Corrupted CIFAR-10.

Dataset | Ratio | SiHRDS | SiH
---|---|---|---
Colored MNIST | 20% | 80.15 $\pm$ 5.27 |  85.24 $\pm$ 1.60
10% |  86.13 $\pm$ 3.07 |  85.35 $\pm$ 1.23
5% |  84.10 $\pm$ 3.03 |  86.14 $\pm$ 1.78
2% | 79.38 $\pm$ 2.37 |  83.80 $\pm$ 1.28
1% | 74.22 $\pm$ 3.21 |  80.03 $\pm$ 2.04
0.5% | 64.17 $\pm$ 5.74 |  71.63 $\pm$ 2.49
Corrupted CIFAR-10 | 20% |  64.09 $\pm$ 5.64 |  66.75 $\pm$ 1.34
10% | 56.89 $\pm$ 5.09 |  61.26 $\pm$ 2.06
5% | 51.25 $\pm$ 4.10 |  55.63 $\pm$ 1.54
2% | 38.22 $\pm$ 3.77 |  43.66 $\pm$ 1.81
1% | 31.64 $\pm$ 2.50 |  35.17 $\pm$ 1.19
0.5% | 24.59 $\pm$ 1.84 |  27.30 $\pm$ 2.04

In Table 4.1, we show the performance on the unbiased test set of Colored
MNIST and Corrupted CIFAR-10 using two different ways of weighting the samples
for training the unbiased classifier. While SiH stands for our proposed
method, in SiHRDS, we utilize the RDS proposed by Nam et al. (2020) when
reweighting the data points. The results suggest that on average our proposed
reweighting mechanism significantly increases performance while the inclusion
of the VAE leads to an accuracy-interpretability tradeoff. Additionally, the
focal loss reduces the variability in accuracy across multiple runs.

Overall, the quantitative results show that the proposed reweighting scheme
improves performance. For settings where the easy-to-learn assumption is
likely to be fulfilled, SiH shows promising results compared to baselines. The
ablation study shows that integrating our reweighting for the unbiased
classifier is critical in improving its accuracy.

###  4.2 Qualitative evaluation

Figure 2 displays the bias visualization from DisEnt and SiH for a few
randomly selected images. Additionally, in Figure 7 of Appendix D, we show the
random bias visualizations for Corrupted CIFAR-10. We will not analyze the
latter images further, as here, signal and bias are not disentangled well
enough for visualizing the bias for either method.

For Colored MNIST, the swapping of DisEnt perturbs the bias representations so
strongly that this also leads to an unwanted change in the digit. This change
is likely due to the bias and signal representations not being perfectly
disentangled. Thus, the leftover signal in the bias dimensions gets swapped
too. On the other hand, SiH does not perturb the digit while regularly
perturbing the color. However, due to the weaker magnitude of change, our
approach sometimes does not visibly change the image.

SiH is more conservative when generating perturbations, which is advantageous
for visualizing bias in realistic cases where learned signal and bias
embeddings are not perfectly disentangled. Although our changes are more
subtle, we believe that, for a practitioner, our perturbation method should be
preferred, as it does not induce artifacts, which otherwise have to be
considered as a possible bias.

##  5 Conclusion and future work

In the presence of bias, a classifier often leverages these spurious
correlations rather than the underlying signal. The application of such an
algorithm can have adverse consequences in critical situations. This work
advances the research in building unbiased deep learning models by
investigating a novel reweighting scheme. We propose SiH, which trains a bias
classifier to be as biased as possible and simultaneously trains an unbiased
classifier by upweighting samples for which the biased decision rule fails to
predict the correct labels. We show that the proposed weighting factor based
on the focal loss can match or outperform existing works. Additionally, by
training a generative model, users are able to visualize and identify the bias
at inference time. For this, the proposed approach leverages latent
adversarial perturbations that do not introduce undesirable artifacts.

#### Future work

Although SiH has demonstrated its effectiveness on simple datasets, its
efficacy on more challenging datasets and other modalities requires further
investigation. For this, it is crucial to use more expressive generative
models such as generative adversarial networks (Goodfellow et al., 2020) or
diffusion models (Sohl-Dickstein et al., 2015; Song & Ermon, 2019; Ho et al.,
2020). Moreover, while SiH consists of established individual components,
their combination is not rigorously derived. In fact, the entire field would
profit from greater mathematical rigor, beginning with the establishment of a
theoretical definition of what constitutes the “ease of learning”.

## References

  * Bahng et al. (2020) Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh.  Learning de-biased representations with biased representations.  In _International Conference on Machine Learning_ , pp. 528–539. PMLR, 2020.
  * Cadène et al. (2019) Rémi Cadène, Corentin Dancette, Hédi Ben-Younes, Matthieu Cord, and Devi Parikh.  Rubi: Reducing unimodal biases for visual question answering.  In _Advances in Neural Information Processing Systems_ , 2019.  URL https://proceedings.neurips.cc/paper/2019/hash/51d92be1c60d1db1d2e5e7a07da55b26-Abstract.html.
  * Clark et al. (2019) Christopher Clark, Mark Yatskar, and Luke Zettlemoyer.  Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing_ , pp. 4067–4080, 2019.
  * Darlow et al. (2020) Luke Darlow, Stanisław Jastrzebski, and Amos Storkey.  Latent adversarial debiasing: Mitigating collider bias in deep neural networks.  _arXiv preprint arXiv:2011.11486_ , 2020.
  * Edwards & Storkey (2016) Harrison Edwards and Amos Storkey.  Censoring representations with an adversary.  In _International Conference on Learning Representations_ , 2016.
  * Geirhos et al. (2020) Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann.  Shortcut learning in deep neural networks.  _Nature Machine Intelligence_ , 2(11):665–673, 2020.
  * Goodfellow et al. (2020) Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio.  Generative adversarial networks.  _Commun. ACM_ , 63(11):139–144, 2020.  doi: 10.1145/3422622.  URL https://doi.org/10.1145/3422622.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Conference on Computer Vision and Pattern Recognition_ , pp. 770–778. IEEE Computer Society, 2016.  doi: 10.1109/CVPR.2016.90.  URL https://doi.org/10.1109/CVPR.2016.90.
  * Hendrycks & Dietterich (2019) Dan Hendrycks and Thomas G. Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  In _International Conference on Learning Representations_. OpenReview.net, 2019.  URL https://openreview.net/forum?id=HJz6tiCqYm.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_ , 2020.  URL https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html.
  * Kim et al. (2019) Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim.  Learning not to learn: Training deep neural networks with biased data.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 9012–9020, 2019.
  * Kim et al. (2021) Eungyeup Kim, Jihyeon Lee, and Jaegul Choo.  Biaswap: Removing dataset bias with bias-tailored swapping augmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 14992–15001, 2021.
  * Kingma & Ba (2015) Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations_ , 2015.  URL http://arxiv.org/abs/1412.6980.
  * Kingma & Welling (2014) Diederik P. Kingma and Max Welling.  Auto-encoding variational bayes.  In _International Conference on Learning Representations_ , 2014.  URL http://arxiv.org/abs/1312.6114.
  * Krizhevsky & Hinton (2009) Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images.  Technical report, University of Toronto, Toronto, Ontario, 2009.
  * Lecun et al. (1998) Yann Lecun, Leon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.  doi: 10.1109/5.726791.
  * Lee et al. (2021) Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo.  Learning debiased representation via disentangled feature augmentation.  In _Advances in Neural Information Processing Systems_ , 2021.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In _IEEE International Conference on Computer Vision_ , pp. 2999–3007. IEEE Computer Society, 2017.  doi: 10.1109/ICCV.2017.324.  URL https://doi.org/10.1109/ICCV.2017.324.
  * Liu et al. (2021) Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn.  Just train twice: Improving group robustness without training group information.  In _International Conference on Machine Learning_ , pp. 6781–6792. PMLR, 2021.
  * Moosavi-Dezfooli et al. (2016) Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard.  Deepfool: a simple and accurate method to fool deep neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2574–2582, 2016.
  * Nam et al. (2020) Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin.  Learning from failure: De-biasing classifier from biased classifier.  In _Advances in Neural Information Processing Systems_ , 2020.
  * Sagawa et al. (2020) Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang.  Distributionally robust neural networks.  In _International Conference on Learning Representations_. OpenReview.net, 2020.  URL https://openreview.net/forum?id=ryxGuJrFvS.
  * Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In Francis R. Bach and David M. Blei (eds.), _Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015_ , volume 37 of _JMLR Workshop and Conference Proceedings_ , pp. 2256–2265. JMLR.org, 2015.  URL http://proceedings.mlr.press/v37/sohl-dickstein15.html.
  * Song & Ermon (2019) Yang Song and Stefano Ermon.  Generative modeling by estimating gradients of the data distribution.  In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pp. 11895–11907, 2019.  URL https://proceedings.neurips.cc/paper/2019/hash/3001ef257407d5a371a96dcd947c7d93-Abstract.html.
  * van den Oord et al. (2017) Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu.  Neural discrete representation learning.  In _Advances in Neural Information Processing Systems_ , pp. 6306–6315, 2017.  URL https://proceedings.neurips.cc/paper/2017/hash/7a98af17e63a0ac09ce2e96d03992fbc-Abstract.html.
  * Wang et al. (2019) Haohan Wang, Zexue He, Zachary C Lipton, and Eric P Xing.  Learning robust representations by projecting superficial statistics out.  In _International Conference on Learning Representations_ , 2019.
  * Zhang & Sabuncu (2018) Zhilu Zhang and Mert R. Sabuncu.  Generalized cross entropy loss for training deep neural networks with noisy labels.  In _Advances in Neural Information Processing Systems_ , pp. 8792–8802, 2018.  URL https://proceedings.neurips.cc/paper/2018/hash/f2925f97bc13ad2852a7a551802feea0-Abstract.html.

##  Appendix A Further related work

#### Debiasing without explicit supervision

To train an unbiased classifier, several existing approaches (Bahng et al.,
2020; Clark et al., 2019; Wang et al., 2019; Cadène et al., 2019) leverage
some form of implicit knowledge about the bias present in the dataset. They
use this insight to design and train a model architecture susceptible to the
specific bias attribute. They then train an unbiased model to learn decision
rules different from the biased model. For example, Bahng et al. (2020)
capture texture bias in image classification by training a convolutional
neural network with a small receptive field. Simultaneously, they train an
unbiased model by forcing it to learn representations that are independent of
the biased ones.

#### Debiasing without implicit supervision

Liu et al. (2021) slightly adapt the idea of Nam et al. (2020). Instead of
training both classifiers simultaneously, they divide the training into two
stages. First, the biased classifier is trained. Second, they train the
unbiased classifier and upweigh all samples that were misclassified by the
biased classifier. Hence, they also try to focus on training on the bias-
conflicting samples.

DisEnt by Lee et al. (2021) builds upon Nam et al. (2020) by utilizing their
training scheme to create a disentangled representation useful for feature
augmentation. As motivation, they show that the diversity of training samples
is an important factor in training. Instead of training the unbiased
classifier only on the sparse bias-conflicting samples, they try to synthesize
additional samples for which using the bias as decision rule does not work.
Their algorithm works as follows: First, Lee et al. (2021) train the base
structure from Nam et al. (2020) using GCE and RDS to create disentangled
representations where signal and bias dimensions can be separated. After a
predetermined number of updates, they start to swap the bias dimensions of
different samples to synthesize representations with the same signal but
different bias. Hence, making the bias unusable as decision rule for the label
because it originates from a different sample. They then train their
classifiers on those representations as well as on the original samples.

A caveat of Nam et al. (2020) is the absence of a mechanism for visualizing
the unknown bias. For this, Lee et al. (2021) propose to train a decoder ex-
post to reconstruct the images given their latent representations. By
repeating their swapping process and reconstructing the images after the swap,
they expect to visualize the bias present in the dataset as they can compare
reconstructions with the same signal but swapped bias dimensions. This will
work if the learned embeddings are perfectly disentangled but might introduce
artifact if they are not.

Further research for debiasing and visualizing the bias has been done by
Darlow et al. (2020), which leverage a vector quantized variational
autoencoder (van den Oord et al., 2017) and add a simple biased classifier on
top of the latent representations. After training, they perturb the latent
dimensions such that the biased classifier is as unsure as possible about the
label. This perturbed representation is then passed through the decoder to
generate images without relevant bias. Consequently, a second, unbiased
classifier is trained on these images, which should not contain bias that can
be used for predicting the label.

##  Appendix B Implementation details

In line with previous works (Nam et al., 2020; Lee et al., 2021; Kim et al.,
2021), we utilize an MLP for Colored CMNIST for all methods. Each encoder
consists of three linear layers with a bottleneck of size 100 for signal and
bias, respectively. The decoder is again consisting of three linear layers. As
activation function, we use the Rectified Linear Unit (ReLU). For the
classifiers, we solely use one linear layer for both datasets. This is because
for us, the difficulty of a dataset is determined by the complexity of the
connection between latent variables and the realization x thereof. If we knew
the latent variables, inferring the label would be simple. Thus, a single
linear layer suffices.

We adapt the encoder-decoder structure for Corrupted CIFAR-10 from an MLP to a
CNN, where we use a ResNet18 (He et al., 2016) with bottleneck dimension of
512 for the encoders of all methods and a ResNet18-like decoder.

We do not perform any preprocessing for Colored MNIST. For the preprocessing
of Corrupted CIFAR-10, we take random crops consisting of at least $50\%$ of
the original image and resize them to the original $32\times 32$ size.
Additionally, we allow horizontal flips of the images and standardize the
pixel values over the entire dataset. To calculate the reconstruction loss, we
transform the standardized pixel values back into $[0,1]$ so that its size is
comparable among all datasets.

To have visualizations that capture the original image well, for SiH, we
upweigh the reconstruction loss by the factor 100. Additionally, we rescale
the reconstruction and KL term by dividing through $3L$, where $L$ is the
number of pixels, to be invariant to image resolution and number of channels
while retaining their relative loss magnitude.

For updating the model weights, we use the Adam optimizer (Kingma & Ba, 2015)
with a learning rate of 0.001 and batch size of 256 for both datasets. The
hyperparameter $q$ is chosen to be $0.7$ by following the GCE coiners Zhang &
Sabuncu (2018).

We perform early stopping and reduce the learning rate when plateauing by
computing this loss function on a held-out 10% of the training set. For
Colored MNIST, we use an early stop patience of 2 versus 20 for Corrupted
CIFAR-10. The patience for the learning rate reduction is one-half of the
early stop patience and reduces the learning rate by a factor of 10.

We do not perform hyperparameter tuning on specific settings, as there must
not be an unbiased or bias labelled validation set in the setting of unknown
bias. We would like to encourage future work to do the same.

For creating visualization of the bias, we adapt Deepfool (Moosavi-Dezfooli et
al., 2016) for our purposes. Originally, the algorithm was developed for
perturbing pixels in an input image, while we use it for perturbing latent
dimensions. Thus, while pixel values need to be clamped in $[0,1]$, we do not
require this. For the distance measure of the perturbation, we use the
$\ell_{2}-$norm.

For the bias visualization, we use the trained model of SiH as backbone. This
is vital in ensuring that discrepancies in the visualizations of DisEnt and
SiH can be attributed solely to the differing visualization techniques. We
perturb images for which the biased and unbiased classifiers predict the
correct class. With this, we aim to find bias-aligned images, which we can
then perturb into neutral images. We randomly sample a different target class
and apply our perturbation. For DisEnt, we pick a second image, for which the
biased classifier predicts the same sampled target class and swap the bias
embeddings.

##  Appendix C Dataset visualizations

To compare the performance of our method SiH with previous works, we evaluate
it on two datasets. These datasets consist of a training set for which we
define varying percentages of bias-conflicting images to analyse the
performance. We assess the debiasing potential of all methods on an unbiased
test set where signal and bias are independently and uniformly distributed.
For SiH, we do not perform hyperparameter tuning on each framework because an
unbiased validation set does not exist in the setting of unknown bias.

The first dataset is Colored MNIST by Kim et al. (2019), which consists of the
popular handwritten digit database MNIST (Lecun et al., 1998) synthetically
infused with a color bias. To each digit, we randomly assign distinct mean
colors, which serve as bias attributes. Hence, the signal $\textbf{z}_{s}$ is
the digit while the easy-to-learn bias $\textbf{z}_{b}$ manifests itself as
the color. In Figure 3, we display bias-aligned images, for which leveraging
the color as decision rule would lead to the correct label. A minority of
samples in the training set consists of bias-conflicting samples, showed in
Figure 4, for which the biased decision rule leads to the wrong prediction.
Learning to recognize the digit instead of the color is the only valid
decision rule with which bias-aligned as well as bias-conflicting samples can
be correctly classified. The second dataset we apply SiH to is the Corrupted
CIFAR-10 dataset (Hendrycks & Dietterich, 2019). It is based on the standard
CIFAR-10 dataset (Krizhevsky & Hinton, 2009), injected with synthetically
generated corruptions such as fog, brightness, or saturation for each class.
These synthetic biasing perturbations are designed to be as realistic as
possible. A collection of randomly selected bias-aligned and bias-conflicting
images for both datasets can be found in Figure 5 and Figure 6, respectively.

Figure 3: Bias-aligned images of the Colored MNIST dataset. The columns show
different digits $\textbf{z}_{s}$ with their respective colors
$\textbf{z}_{b}$ that predominantly manifest in combination. Figure 4: Bias-
conflicting images of the Colored MNIST dataset. The columns show different
digits $\textbf{z}_{s}$ with colors $\textbf{z}_{b}$ that are usually not
observed together. Figure 5: Bias-aligned images of the Corrupted CIFAR-10
dataset. The columns show the different classes $\textbf{z}_{s}$ with their
respective corruptions $\textbf{z}_{b}$ that predominantly manifest in
combination. For example, the class birds often has foggy images, while ships
are frequently pixelated.

Figure 6: Bias-conflicting images of the Corrupted CIFAR-10 dataset. The
columns show the different classes $\textbf{z}_{s}$ with corruptions
$\textbf{z}_{b}$ that are usually not observed together.

##  Appendix D Bias visualizations for Corrupted CIFAR-10

Figure 7: A random collection of bias visualizations for Corrupted CIFAR-10.
The randomly selected images are varied over random seeds and the percentage
of bias-conflicting images in the training set.


11institutetext: Faculty of Informatics, Technical University of Munich,
Munich, Germany 22institutetext: Siemens AG, Munich, Germany
33institutetext: Indian Institute of Technology Delhi, New Delhi, India
44institutetext: Clinic for Diagnostic and Interventional Radiology,
University Hospital Bonn, Germany
55institutetext: Helmholtz AI, Helmholz Zentrum Munich, Munich, Germany
Corresponding author: Agnieszka Tomczak<EMAIL_ADDRESS>
# What can we learn about a generated image corrupting its latent
representation?

Agnieszka Tomczak 1122 Aarushi Gupta 33 Slobodan Ilic 1122 Nassir Navab 1144
Shadi Albarqouni 445511 0000-0003-2157-2211

###### Abstract

Generative adversarial networks (GANs) offer an effective solution to the
image-to-image translation problem, thereby allowing for new possibilities in
medical imaging. They can translate images from one imaging modality to
another at a low cost. For unpaired datasets, they rely mostly on cycle loss.
Despite its effectiveness in learning the underlying data distribution, it can
lead to a discrepancy between input and output data. The purpose of this work
is to investigate the hypothesis that we can predict image quality based on
its latent representation in the GANs bottleneck. We achieve this by
corrupting the latent representation with noise and generating multiple
outputs. The degree of differences between them is interpreted as the strength
of the representation: the more robust the latent representation, the fewer
changes in the output image the corruption causes. Our results demonstrate
that our proposed method has the ability to i) predict uncertain parts of
synthesized images, and ii) identify samples that may not be reliable for
downstream tasks, e.g., liver segmentation task.

###### Keywords:

GANs Image Synthesis Uncertainty Image quality.

##  1 Introduction

Generative Adversarial Networks (GANs) [8] are state-of-the-art methods for
image-to-image translation problems. It has been found that GANs are a
promising technique for generating images of one modality based on another.
Creating such images in a clinical setting could be highly effective, but only
if the images retain anatomical details and serve the downstream task. As
Cohen et al. [5] described, GANs based on cycle consistency can "hallucinate"
features (for example tumors) causing potentially wrong diagnoses. A
clinically useful example of modality translation would be generating Computer
Tomography (CT) images from Magnetic Resonance Imaging (MRI) scans, and vice
versa. This problem has already been investigated multiple times [3, 23, 6, 4,
7, 20] as an image-to-image translation task with multiple approaches: some of
them focusing on shape consistency to preserve anatomical features [21, 23, 6,
7, 11], others proposing a multimodal approach to deal with the scalability
issues [19, 12, 16]. Nevertheless, the main challenge remains: how to
determine when a generative adversarial network can be trusted? The issue has
a considerable impact on the medical field, where generated images with
fabricated features have no clinical value. Recently, Upadhyay et al. [18]
tackled this problem by predicting not only the output images but also the
corresponding aleatoric uncertainty and then using it to guide the GAN to
improve the final output. Their method requires changes in the optimization
process (additional loss terms) and network architecture (additional output).
They showed that for paired datasets it results in improved image quality, but
did not clearly address the point that in medical imaging the visual quality
of images does not always transfer to the performance on a downstream task.
Our goal was to examine this problem from a different perspective and test the
hypothesis: the more robust the image representation, the better the quality
of the generated output and the end result. To this end, we present a noise
injection technique that allows generating multiple outputs, thus quantifying
the differences between these outputs and providing a confidence score that
can be used to determine the uncertain parts of the generated image, the
quality of the generated sample, and to some extent their impact on a
downstream task.

##  2 Methodology

We design a method to test the assumption that the stronger the latent
representation the better the quality of a generated image. In order to check
the validity of this statement, we corrupt the latent representation of an
image with noise drawn from normal distribution and see how it influences the
generated output image. In other terms, given an image $x\in X$, domain $y\in
Y$ and a Generative Adversarial Network $G$, we assume a hidden representation
$h=E(x)$ with dimensions $n,m,l$, where $E$ stands for the encoding part of
$G$, and $D$ for the decoding part. We denote the generated image as
$\hat{x}$. Next, we construct $k$ corrupted representation latent codes
$\hat{h}$, adding to $h$ noise vector $\eta$:

$\begin{gathered}\eta_{1,..,k}\thicksim\mathcal{N}(0,\alpha\sigma_{h_{1,...,l}}^{2})\end{gathered}$
(1)

where $\sigma_{h_{1,...,l}}^{2}$ is channel-wise standard deviation of input
representation $h$. We can control the noise level with factor $\alpha$.
Before we add the noise vector, we eliminate the background noise with
operation $bin$ by masking it with zeros for all the channels where the output
pixels are equal to zero, so do not contain any information.

$\begin{gathered}bin(h)=h[\hat{x}>0]\\\
\hat{h}_{1,...,k}=h_{1}+bin(h_{1})\eta_{1},...,h_{k}+bin(h_{k})\eta_{k}\end{gathered}$
(2)

Having now multiple representations for a single input image we can pass them
to decoder $D$ and generate multiple outputs:

$\begin{gathered}\hat{x}_{1,...,k}=D(\hat{h}_{1}),...,D(\hat{h}_{k})\end{gathered}$
(3)

We use the multiple outputs to quantify the uncertainty connected with the
representation of a given image. We calculate two scores: the variance (the
average of the squared deviations from the mean) $\gamma$ of our $k$ generated
images

$\begin{gathered}\gamma=Var(\hat{x_{1}},..,\hat{x_{k}})\end{gathered}$ (4)

and the Mutual Information (MI) between the multiple outputs and our primary
output $\hat{x}$ produced without noise injection.

$\begin{gathered}MI(X;Y)=\sum_{y\in Y}\sum_{x\in
X}p(x,y)\log\Big{(}\frac{p(x,y)}{p(x)p(y)}\Big{)}\\\
\delta=\frac{1}{k}\sum_{0}^{k}MI\big{(}\hat{x},(\hat{x_{i}})\big{)}\end{gathered}$
(5)

We interpret the $\gamma$ and $\delta$ as the measures of the representation
quality. The variance $\gamma$ can be considered as an uncertainty score - the
higher the variance of generated outputs with the corrupt representations, the
more uncertain the encoder is about produced representation. On the other
hand, the MI $\delta$ score can be interpreted as a confidence score,
quantifying how much information is preserved between the original output
$\hat{x}$ and the outputs produced from corrupted representations
$\hat{x_{1}},...,\hat{x_{k}}$. We calculate the MI based on a joint (2D)
histogram, with number of bins equal to $\lfloor\sqrt{n/5}\rfloor$, where n is
a number of pixels per image as proposed by [2].

##  3 Experiments and Results

We conducted a number of experiments using state-of-the-art architectures with
the goal of demonstrating the effectiveness of our proposed method and
confirming our hypothesis that the stronger the latent representation, the
better and more reliable the image quality. Our proposed method was evaluated
on two publicly available datasets, namely CHAOS [14] and LiTS [1] datasets.

###  3.1 Network Architectures and implementation details

#### TarGAN

As our main baseline we use TarGAN [3] network which uses a shape-consistency
loss to preserve the shape of the liver. We trained the model for 100 epochs.
We kept all the parameters unchanged with respect to the official
implementation provided by the authors of TarGAN. We use PyTorch 1.10 to
implement all the models and experiments. During inference, we constructed
$k=10$ corrupted representation with noise level $\alpha=3$ and used them for
evaluation of our method.

#### UP-GAN

We adapted the UP-GAN network from [18] to run on unpaired dataset as shown in
[17]. UP-GAN uses an uncertainty guided loss along the standard cycle loss
during training. The uncertainty loss defined for UP-GAN was used in every
component of cycleGAN - identity loss and cycle loss for training of both
generators. We kept the learning rate at $10^{-4}$ for T1 to T2 transfer and
at $10^{-3}$ for CT to T1 and T2 transfer. We tuned the hyperparameters in the
following manner: 0.5 for each of the discriminator losses, while the
generators had a factor 1 with their cycle losses, 0.01 with the uncertainty
cycle loss and factors of 0.1 and 0.0015 with identity losses. We trained all
the three models for 100 epochs.

#### Datasets.

We use data of each modality (CT, T1 and T2) from 20 different patients
provided by the publicly available CHAOS19 dataset [14]. We randomly selected
50% of the dataset to be the training set and used the rest for testing. We
followed [3] in setting liver as the target area, as the CT scans only have
liver annotations. Besides, we used LiTS [1] dataset to evaluate our method on
the pathological samples. The dataset contains CT scans of patients with liver
tumors and corresponding segmentation masks. All images were resized to the
size of $256\times 256$.

#### Evaluation metrics.

We use FID [10] score to evaluate the quality of generated images and DICE
score to evaluate segmentation results. We calculate the FID score using
features extracted with Model Genesis [24].

###  3.2 Can we use the noise injections to identify uncertain parts of a
synthesized image?

First, we conduct a sanity check experiment by blacking out a random $50\times
50$ pixel patch from the input images (perturbed input) and measuring the
proposed uncertainty and confidence scores on the corresponding synthesized
images. Table 1 report the mean, median and variance of both uncertainty score
($\gamma$ in eq.4) and confidence score ($\delta$ in eq.5) on both the
original and perturbed images. One could observe that perturbed input has
large variance and low confidence compared to the original input. This has
been nicely visualized in Figure 1 where the confidence scores of the
perturbed corrupted images are much lower than the corresponding ones for the
original images. This demonstrates the effectiveness of our proposed method in
detecting uncertain synthesized images, e.g., perturbed images. The results
suggest the possibility of finding a potential confidence threshold to
eliminate uncertain synthesized images. Surprisingly, the model was able to
nicely synthesize perturbed images hallucinating and replacing the masked
regions with reasonable healthy tissues. Nevertheless, our heatmaps were able
to capture such uncertainty as shown in Figure 2.

Figure 1: Histograms of confidence values for original images and images
corrupted by blacking out 50x50 pixel square. We can see significant drop of
confidence for the corrupted inputs in all of the target modalities.

To validate our proposed method in more realistic clinical setting, we run the
inference on the LiTS dataset, which consists of CT scans with tumors present.
While one would expect to see high uncertainty for input images with tumors,
since the model was trained on healthy data, we only observed this for the
translation from CT to T1 and only for small tumors (_cf._ first two columns
in Figure 3). This was not the case for bigger tumors (third column) and for
CT to T2 translation (last two columns). It seems that the network was
confidently preserving big tumors with T1 target modality and all of the
pathologies with T2 target modality, while small lesions in translation from
CT to T1 - the ones that were not generated and probably filtered out as
artifacts - caused spike in the uncertainty value.

Table 1: The variance of multiple outputs with corrupted representation
increases as we introduce a black patch on the input. Variance ($\gamma$)
---
Source $\rightarrow$ Target |  Original input |  Perturbed input
CT $\rightarrow$ T2 |  0.0176 (0.0176)$\pm$0.0018 |  0.0197 (0.0195)$\pm$0.0024
T2 $\rightarrow$ CT |  0.0155 (0.0155)$\pm$0.0006 |  0.0199 (0.0195)$\pm$0.0031
T1 $\rightarrow$ T2 |  0.0022 (0.0022)$\pm$0.0003 |  0.0031 (0.0028)$\pm$0.0010
T2 $\rightarrow$ T1 |  0.0045 (0.0044)$\pm$0.0007 |  0.0104 (0.0115)$\pm$0.0048
T1 $\rightarrow$ CT |  0.0179 (0.0183)$\pm$0.0015 |  0.0185 (0.0185)$\pm$0.0027
CT $\rightarrow$ T1 |  0.0567 (0.0573)$\pm$0.0034 |  0.0676 (0.0670)$\pm$0.0072
Confidence ($\delta$)
Source $\rightarrow$ Target |  Original input |  Perturbed input
CT $\rightarrow$ T2 |  2.2608 (2.2737)$\pm$0.0515 |  1.9398 (1.9529)$\pm$0.0939
T2 $\rightarrow$ CT |  2.7330 (2.7410)$\pm$0.0348 |  2.2738 (2.1684)$\pm$0.3819
T1 $\rightarrow$ T2 |  2.0281 (2.0019)$\pm$0.0700 |  1.8230 (1.8364)$\pm$0.1736
T2 $\rightarrow$ T1 |  2.2648 (2.2573)$\pm$0.0360 |  1.9289 (1.8636)$\pm$0.2300
T1 $\rightarrow$ CT |  2.7736 (2.7971)$\pm$0.0592 |  2.1120 (2.0829)$\pm$0.3806
CT $\rightarrow$ T1 |  2.1806 (2.1818)$\pm$0.0517 |  1.7222 (1.6939)$\pm$0.1122
Figure 2: Samples generated using input with randomly blacked out $50\times
50$ pixel patch. We can see on uncertainty heat maps the lack of confidence
when generating this part of the images. It is not reflected in the output
image.

###  3.3 Can we use the noise injections to improve the quality of a
synthesized image?

Our next experiment involved injecting noise into the training process. We
investigated whether it would result in better image quality and a more robust
representation. According to Table 2, injecting a small amount of noise
($\alpha=0.5$) into half of the synthesized samples, during the training
process, improved the final image quality. Nevertheless, it did not seem to
translate to the end task: segmentation accuracy did not improve. We found out
that introducing excess noise ($\alpha>0.5$) or corrupting the majority of
samples during training can cause confusion in the model, leading to
deteriorated performance.

Figure 3: CT slides from LiTS dataset containing tumor pathologies. From top to bottom, we show the input CT slice, corresponding tumor segmentation map, generated T1 or T2 images, the uncertainty heat map and the overlay of segmentation mask and uncertainty map.  Table 2: Slightly corrupting the latent representation in GANs bottleneck leads to improved image quality. The lower the FID score, the better. Image Quality [FID] |
---|---
Source $\rightarrow$ Target | TarGAN |  | TarGAN
---
($\alpha=0.5$)
| TarGAN
---
($\alpha=0.8$)
| TarGAN
---
($\alpha=1.0$)
T1 $\rightarrow$ CT |  0.0649 |  0.0612 |  0.0616 |  0.0441
T2 $\rightarrow$ CT |  0.1475 |  0.1496 |  0.1704 |  0.1787
CT $\rightarrow$ T1 |  0.0651 |  0.0508 |  0.0584 |  0.0563
T2 $\rightarrow$ T1 |  0.1200 |  0.1135 |  0.1278 |  0.1450
CT $\rightarrow$ T2 |  0.0469 |  0.0471 |  0.0601 |  0.0647
T1 $\rightarrow$ T2 |  0.0604 |  0.0545 |  0.0583 |  0.0668

###  3.4 Can we correlate our confidence score with the quality of downstream
task, e.g., segmentation, on the synthesized image?

To address this question, we train three U-Net [15] networks to perform liver
segmentation on three imaging modalities, namely CT, T1 and T2, and then run
the inference on both the same imaging modality and the transferred
synthesized ones and report the results in Table 3. On the diagonal we present
the scores for the original modality which range from 0.95 for CT to 0.82 for
T1, which are slightly different from the ones reported in [3] due to the fact
of using a standard 2D U-Net and no enrichment technique [9]. Nevertheless,
the segmentation results are acceptable for the CT to T1, CT to T2 and T1 to
CT transferred images. However, the performance deteriorates for images where
T2 scans are the source modality. This is reflected in the correlation scores
as well (_cf._ Table 3 and Table 4). There is a correlation around and higher
than 0.5 for translations where the segmentor worked as well. This suggests
that our method can be used most efficiently in cases where the images are
generated well enough for the downstream task network to also perform well. If
the generated images are of so low quality, that the segmentor fails
completely (DICE < 0.5) the confidence value does not correlate with the DICE
score.

Table 3: Segmentation results on original input images (diagonal) and images
transferred with TarGAN.  Segmentation quality [DICE]
---
from\to |  CT |  T1 |  T2
CT |  0.9506 (0.9711)$\pm$0.1003 |  0.6806 (0.7315)$\pm$0.2225 |  0.7302 (0.7529)$\pm$0.1692
T1 |  0.6900 (0.8546)$\pm$0.3735 |  0.8276 (0.9578)$\pm$0.3134 |  0.5272 (0.6671)$\pm$0.4059
T2 |  0.4085 (0.5321)$\pm$0.3652 |  0.5088 (0.5302)$\pm$0.3662 |  0.8349 (0.9537)$\pm$0.2777
Table 4: The FID scores and the absolute value of a correlation between the confidence score and DICE coefficient for our method and UP-GAN [18]. | Noise injections | UP-GAN [18]
---|---|---
Source $\rightarrow$ Target |  FID |  Correlation |  FID |  Correlation
CT $\rightarrow$ T1 |  0.0651 |  0.5423 |  0.2022 |  0.0029
T1 $\rightarrow$ CT |  0.0649 |  0.5441 |  0.1619 |  0.2188
CT $\rightarrow$ T2 |  0.0469 |  0.4946 |  0.1557 |  0.2540
T2 $\rightarrow$ CT |  0.1475 |  0.2536 |  0.6540 |  0.0010
T1 $\rightarrow$ T2 |  0.0604 |  0.0546 |  0.1396 |  0.1021
T2 $\rightarrow$ T1 |  0.1200 |  0.3105 |  0.1656 |  0.0827

###  3.5 How does the noise injection method compare to other uncertainty
estimation techniques?

We compare our method to the existing way of estimating aleatoric uncertainty,
described in UP-GAN [18]. The quality of generated images is measured with FID
scores and the correlation between the DICE coefficient and the mean of the
estimated aleatoric uncertainty values as defined in [17]. It is not
surprising that the FID scores are slightly lower than those of a TarGAN
considering the absence of a shape-optimizing loss term. Furthermore, the
aleatoric uncertainty does not correlate well with the DICE score, indicating
that even though the aleatoric uncertainty might be useful in improving image
quality as demonstrated in the paper, it does not translate directly into the
downstream task of segmentation and cannot be used to indicate unsuitable
samples. Among the differences between our method and [18], we emphasize that
ours only affects the inference stage and can be used with basically any
architecture, while the UP-GAN involves significant differences in the
architecture (extra outputs of the network) and the optimization process
(extra loss terms requiring parameter tuning).

##  4 Conclusion

In this work, we investigated the hypothesis that a robust latent
representation results in a higher quality of a generated image and higher
performance on a downstream task. We showed that there are indicators that the
quality of latent representation corresponds to the final quality of a
generated image. If the downstream task network performs well, it is possible
to correlate it with the latent representation’s quality. Additionally, we
discovered that small noise injections during the training phase lead to more
robust representation and slightly higher image quality. However, this does
not necessarily lead to better segmentation results. We compared the noise
injections to the aleatoric uncertainty estimation method as proposed by [17].
Although our approach has a smaller impact on the image quality itself, it is
more indicative of performance on downstream tasks. Our method is easier to
incorporate as it does not require changes in the model’s architecture or the
optimization process. The future work includes extending this method using
adversarial attack techniques such as [13, 22], investigating how it
influences different end tasks, for example classification, and validating the
method in more real-life scenarios.

## References

  * [1] Bilic, P., Christ, P., Vorontsov, E., Chlebus, G., Chen, H., Dou, Q., Fu, C.W., Han, X., Heng, P.A., Hesser, J., Kadoury, S., Konopczyński, T., Le, M., li, F., Li, X., Lipkovà, J., Lowengrub, J., Meine, H., Moltz, J., Wu, J.: The liver tumor segmentation benchmark (lits) (01 2019)
  * [2] Cellucci, C.J., Albano, A.M., Rapp, P.E.: Statistical validation of mutual information calculations: Comparison of alternative numerical algorithms. Phys. Rev. E 71, 066208 (Jun 2005). https://doi.org/10.1103/PhysRevE.71.066208, https://link.aps.org/doi/10.1103/PhysRevE.71.066208
  * [3] Chen, J., Wei, J., Li, R.: Targan: Target-aware generative adversarial networks for multi-modality medical image translation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer (2021)
  * [4] Chen, S., Qin, A., Zhou, D., Yan, D.: Technical note: U-net-generated synthetic ct images for magnetic resonance imaging-only prostate intensity-modulated radiation therapy treatment planning. Medical Physics 45, 5659–5665 (2018)
  * [5] Cohen, J.P., Luck, M., Honari, S.: Distribution matching losses can hallucinate features in medical image translation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing (2018)
  * [6] Emami, H., Dong, M., Nejad-Davarani, S., Glide-Hurst, C.: Sa-gan: Structure-aware generative adversarial network for shape-preserving synthetic ct generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2021)
  * [7] Ge, Y., Wei, D., Xue, Z., Wang, Q., Zhou, X., Zhan, Y., Liao, S.: Unpaired mr to ct synthesis with explicit structural constrained adversarial learning. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). pp. 1096–1099. IEEE (2019)
  * [8] Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. In: Advances in neural information processing systems (NIPS) (2014)
  * [9] Gupta, L., Klinkhammer, B., Boor, P., Merhof, D., Gadermayr, M.: GAN-Based Image Enrichment in Digital Pathology Boosts Segmentation Accuracy, pp. 631–639 (10 2019). https://doi.org/10.1007/978-3-030-32239-7_70
  * [10] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. p. 6629–6640. NIPS’17, Curran Associates Inc., Red Hook, NY, USA (2017)
  * [11] Horvath, I., Paetzold, J., Schoppe, O., Al-Maskari, R., Ezhov, I., Shit, S., Li, H., Ertürk, A., Menze, B.: Metgan: Generative tumour inpainting and modality synthesis in light sheet microscopy. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 227–237 (2022)
  * [12] Huang, P., Li, D., Jiao, Z., Wei, D., Li, G., Wang, Q., Zhang, H.: CoCa-GAN: Common-Feature-Learning-Based Context-Aware Generative Adversarial Network for Glioma Grading, pp. 155–163 (10 2019). https://doi.org/10.1007/978-3-030-32248-9_18
  * [13] Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., Madry, A.: Adversarial examples are not bugs, they are features. In: ArXiv preprint arXiv:1905.02175 (2019)
  * [14] Kavur, A.E., Gezer, N.S., Barış, M., Aslan, S., Conze, P.H., Groza, V., Pham, D.D., Chatterjee, S., Ernst, P., Özkan, S., Baydar, B., Lachinov, D., Han, S., Pauli, J., Isensee, F., Perkonigg, M., Sathish, R., Rajan, R., Sheet, D., Dovletov, G., Speck, O., Nürnberger, A., Maier-Hein, K.H., Bozdağı Akar, G., Ünal, G., Dicle, O., Selver, M.A.: CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis 69, 101950 (Apr 2021). https://doi.org/https://doi.org/10.1016/j.media.2020.101950, http://www.sciencedirect.com/science/article/pii/S1361841520303145
  * [15] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. pp. 234–241. Springer International Publishing, Cham (2015)
  * [16] Shen, L., Zhu, W., Wang, X., Xing, L., Pauly, J.M., Turkbey, B., Harmon, S.A., Sanford, T.H., Mehralivand, S., Choyke, P.L., Wood, B.J., Xu, D.: Multi-domain image completion for random missing input data. IEEE Trans. Med. Imaging 40(4), 1113–1122 (2021). https://doi.org/10.1109/TMI.2020.3046444, https://doi.org/10.1109/TMI.2020.3046444
  * [17] Upadhyay, U., Chen, Y., Akata, Z.: Robustness via uncertainty-aware cycle consistency (2021)
  * [18] Upadhyay, U., Chen, Y., Hebb, T., Gatidis, S., Akata, Z.: Uncertainty guided progressive gans for medical image translation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer (2021)
  * [19] Xin, B., Hu, Y., Zheng, Y., Liao, H.: Multi-modality generative adversarial networks with tumor consistency loss for brain mr image synthesis. In: The IEEE International Symposium on Biomedical Imaging (ISBI) (2020)
  * [20] Yang, J., Dvornek, N.C., Zhang, F., Chapiro, J., Lin, M., Duncan, J.S.: Unsupervised domain adaptation via disentangled representations: Application to cross-modality liver segmentation. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.T., Khan, A. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. pp. 255–263. Springer International Publishing, Cham (2019)
  * [21] Yu, B., Zhou, L., Wang, L., Shi, Y., Fripp, J., Bourgeat, P.: Ea-gans: Edge-aware generative adversarial networks for cross-modality mr image synthesis. IEEE Transactions on Medical Imaging 38(7), 1750–1762 (2019). https://doi.org/10.1109/TMI.2019.2895894
  * [22] Zhang, J., Chao, H., Kalra, M.K., Wang, G., Yan, P.: Overlooked trustworthiness of explainability in medical ai. medRxiv (2021). https://doi.org/10.1101/2021.12.23.21268289, https://www.medrxiv.org/content/early/2021/12/24/2021.12.23.21268289
  * [23] Zhang, Z., Yang, L., Zheng, Y.: Translating and segmenting multimodal medical volumes with cycle- and shape-consistency generative adversarial network. pp. 9242–9251 (06 2018). https://doi.org/10.1109/CVPR.2018.00963
  * [24] Zhou, Z., Sodha, V., Rahman Siddiquee, M.M., Feng, R., Tajbakhsh, N., Gotway, M.B., Liang, J.: Models genesis: Generic autodidactic models for 3d medical image analysis. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. pp. 384–393. Springer International Publishing (2019)