11institutetext: Inception Institute of AI, UAE

# Unsupervised Domain Adaptation Using Feature Disentanglement And GCNs For
Medical Image Classification

Dwarikanath Mahapatra

###### Abstract

The success of deep learning has set new benchmarks for many medical image
analysis tasks. However, deep models often fail to generalize in the presence
of distribution shifts between training (source) data and test (target) data.
One method commonly employed to counter distribution shifts is domain
adaptation: using samples from the target domain to learn to account for
shifted distributions. In this work we propose an unsupervised domain
adaptation approach that uses graph neural networks and, disentangled semantic
and domain invariant structural features, allowing for better performance
across distribution shifts. We propose an extension to swapped autoencoders to
obtain more discriminative features. We test the proposed method for
classification on two challenging medical image datasets with distribution
shifts - multi center chest Xray images and histopathology images. Experiments
show our method achieves state-of-the-art results compared to other domain
adaptation methods.

###### Keywords:

Unsupervised domain adaptation, Graph convolution networks, Camelyon17,
CheXpert, NIH Xray.

##  1 Introduction

The success of convolutional neural networks (CNNs) has set new benchmarks for
many medical image classification tasks such as diabetic retinopathy grading
[27], digital pathology image classification [50, 149, 36, 19, 175, 143, 44,
151, 37, 12, 104] and chest X-ray images [33, 177, 91, 92, 108, 111, 60, 59,
62, 119, 58, 135, 133, 132, 144, 53], as well as segmentation tasks [47, 154,
24, 9, 7, 77, 67, 182, 141, 89, 64, 48, 63, 107]. Clinical adoption of
algorithms has many challenges due to the domain shift problem: where the
target dataset has different characteristics than the source dataset on which
the model was trained. These differences are often a result of different image
capturing protocols, parameters, devices, scanner manufacturers, etc. Since
annotating samples from hospitals and domains is challenging due to scarcity
of experts and the resource intensive nature of labeling, it is essential to
design models from the beginning that perform consistently on images acquired
from multiple domains.

Most approaches to the domain shift problem can be categorized based on the
amount of data from the target domain. Fully-supervised domain adaptation (DA)
and semi-supervised DA assume availability of a large amount and a small
amount of fully-labeled instances from the target domain, respectively, along
with data from a source domain [157, 17, 146, 78, 155, 171, 112, 79, 38, 173,
82, 10, 80, 41, 13, 180, 103, 86, 106]. Unsupervised domain adaptation (UDA)
techniques [16, 153, 166, 167, 74, 168, 85, 160, 159, 172, 170, 65, 116, 169,
115, 181, 88, 87] use only unlabeled data from the target domain and labeled
source domain data. UDA methods aim to learn a domain-invariant representation
by enforcing some constraint (e.g. Maximum Mean Discrepancy [46]) that brings
the latent space distributions, $z$, of distinct domains closer; ideally
leading to more comparable performance in classification/segmentation.

Convolutional neural network (CNN) based state-of-the-art (SOTA) UDA methods
often obtain impressive results but typically only enforce alignment of global
domain statistics [179, 93, 113, 140, 45, 110, 164, 165, 109, 121, 163, 61,
90, 139, 137, 138, 136, 55] resulting in loss of semantic class label
information. Semantic transfer methods [51, 152, 57, 176, 56, 54, 134, 128,
130, 131, 129, 125, 126, 120, 117, 127, 142, 114] address this by propagating
class label information into deep adversarial adaptation networks.
Unfortunately, it is difficult to model and integrate semantic label transfer
into existing deep networks. To deal with the above limitations [52, 147, 148,
150, 20, 18, 124, 145, 72, 42, 71, 40, 34, 35, 11, 83, 70, 81, 162, 8, 68, 69,
102, 76, 43, 25, 122, 75, 66, 73, 6, 94, 95] propose a Graph Convolutional
Adversarial Network (GCAN) for unsupervised domain adaptation. Graph based
methods better exploit global relationship between different nodes (or
samples) than CNNs and learn both global and local information beneficial for
DA.

One way to address domain shift is with feature disentanglement [49] by
separating latent feature representations into domain invariant (often called
structure or content) and domain variant (texture or style) components, thus
minimizing the impact of domain variation. In this work we combine feature
disentanglement with graph convolutional networks (GCN) for unsupervised
domain adaptation and apply it to two different standard medical imaging
datasets for classification and compare it to SOTA methods.

Related Work: Prior works in UDA focused on medical image classification [2],
object localisation, and lesion segmentation [29, 39], and histopathology
stain normalization [15]. Heimann et al. [29] used GANs to increase the size
of training data and demonstrated improved localisation in X-ray fluoroscopy
images. Likewise, Kamnitsas et al. [39] used GANs for improved lesion
segmentation in magnetic resonance imaging (MRI). Ahn et al. [2] use a
hierarchical unsupervised feature extractor to reduce reliance on annotated
training data. Chang et al. [15] propose a novel stain mix-up for
histopathology stain normalization and subsequent UDA for classification.
Graph networks for UDA [52, 178, 99, 5, 98, 4, 100, 101, 97, 96, 118, 105, 23,
3, 123, 32, 161] have been used in medical imaging applications [1] such as
brain surface segmentation [26] and brain image classification [30, 31].
However, none of them combine feature disentanglement with GCNs for UDA.

Our Contributions: Although previous works used feature disentanglement and
graph based domain adaptation separately, they did not combine them for
medical image classification. In this work: 1) we propose a feature
disentanglement module trained to align structural components coupled with a
Graph Convolutional Adversarial Network (GCAN) for unsupervised domain
adaptation in medical images. 2) We perform feature disentanglement using
swapped autoencoders to obtain texture and structural features, which are used
for graph construction and defining generator losses. Since, in medical
imaging applications, images from the same modality (showing the same organ)
have similar texture, we introduce a novel cosine similarity loss components
into the loss function that enforces the preservation of the structure
component in the latent representation; 3) We demonstrate our method’s
effectiveness on multiple medical image datasets.

##  2 Method

Given a source data set,
$\mathcal{D}_{S}=\\{(x^{S}_{i},y^{S}_{i})\\}_{i=1}^{n_{s}}$, consisting of
$n_{s}$ labeled samples, and a target data set,
$\mathcal{D}_{T}=\\{(x^{T}_{i})\\}_{i=1}^{n_{t}}$ consisting of $n_{t}$
unlabeled target samples, unsupervised domain adaptation aims to learn a
classifier that can reliably classify unseen target samples. Here,
$x^{S}_{i}\sim p_{S}$ is a source data point sampled from source distribution
$p_{S}$, $y^{S}_{i}\in\mathcal{Y}_{S}$ is the label, and $x^{T}_{i}\sim p_{T}$
is a target data point sampled from target distribution $p_{T}$. As per the
covariate shift assumption, we assume that $p_{S}(y|x)=p_{T}(y|x)~{}\forall
x$. Thus the only thing that changes between source and target is the
distribution of the input samples, $x$.

The overall block diagram of our system is depicted in Figure 1. Our proposed
approach consists of a feature disentanglement module (Figure 1-b) which
separates semantic textural and structural features (often called style and
content respectively). The output of the feature disentanglement module is
constructed into a graph and then fed into an adversarial generator based
graph neural network (Figure 1-a) that generates features which are domain
invariant. The graph neural network learns more global relationships between
samples, which in turn leads to more discriminative and domain invariant
feature learning.

|
---|---
(a) | (b)
Figure 1: (a) Workflow of our proposed method. Given images from source and
target domains we disentangle features into texture and structure features,
and generate similarity scores to construct an adjacency matrix. Generated
features by the GCN are domain invariant and along with the latent features
are used to train a domain classifier. (b) Architecture of feature
disentanglement network with swapped structure features

###  2.1 Feature Disentanglement Network:

Figure 1(b) shows the architecture of our feature disentanglement network
(FDN). The FDN consists of two encoders ($E_{S}(\cdot),E_{T}(\cdot)$) and two
decoder (generator) networks ($G_{S}(\cdot),G_{T}(\cdot)$), for the source and
target domains respectively. Similar to a classic autoencoder, each encoder,
$E_{\bullet}(\cdot)$, produces a latent code $z_{i}$ for image
$x_{i}^{\bullet}\sim p_{\bullet}$. Each decoder, $G_{\bullet}(\cdot)$,
reconstructs the original image from $z_{i}$. Furthermore, we divide the
latent code, $z_{i}$, into two components: a texture component, $z^{tex}_{i}$,
and a structural component, $z^{str}_{i}$. Structural information is the local
information that describes an image’s underlying structure and visible organs
or parts, and is expected to be invariant across similar images from different
domains. The disentanglement network is trained using the following loss
function:

$\mathcal{L}_{Disent}=\mathcal{L}_{Rec}+\lambda_{1}\mathcal{L}_{swap}+\lambda_{2}\mathcal{L}_{str}$
(1)

$\mathcal{L}_{Rec}$, is the commonly used image reconstruction loss and is
defined as:

$\mathcal{L}_{Rec}=\mathbb{E}_{x_{i}^{S}\sim
p_{S}}\left[\left\|x_{i}^{S}-G_{S}(E_{S}(x_{i}^{S}))\right\|\right]+\mathbb{E}_{x_{j}^{T}\sim
p_{T}}\left[\left\|x_{j}^{T}-G_{T}(E_{T}(x_{j}^{T}))\right\|\right]$ (2)

As shown in Figure 1(b), we also combine the structure of source domain images
($z^{str}_{i}$) with the texture of target images ($z^{tex}_{j}$) and send it
though $G_{T}$. the reconstructed images, $\hat{x}_{ij}$, should contain the
structure of the original source domain image but, should look like an image
from the target domain. To enforce this property, we use the adversarial loss
term $\mathcal{L}_{swap}$ defined as:

$\mathcal{L}_{swap}=\mathbb{E}_{x_{j}\sim
p_{T}}[\log(1-D_{FD}(x_{j}))]+\mathbb{E}_{x_{i}\sim p^{S},x_{j}\sim
p^{T}}\left[\log(D_{FD}(G_{T}([z_{i}^{str},z_{j}^{tex}])))\right].$ (3)

Here, $D$ is a discriminator trained with target domain images, thus the
generator should learn to produce images that appear to be from the target
domain.

Furthermore, the newly generated image, $\hat{x}_{ij}$, should retain the
structure of the original source domain image $x_{i}^{S}$ from where the
original structure was obtained. In [156] the authors use co-occurrent patch
statistics from $\hat{x}_{ij}$ and $x_{i}^{S}$ such that any patch from
$x_{i}^{S}$ cannot be distinguished from a group of patches from
$\hat{x}_{ij}$. This constraint is feasible in natural images since different
images have far more varied style and content, but is challenging to implement
in medical images since images from the same modality (showing the same organ)
appear similar. Thus limited unique features are learnt under such a
constraint. To overcome this limitation we introduce a novel component into
the loss function. The output, $\hat{x}_{ij}$, when passed through encoder
$E_{S}$ produces the disentangled features $z^{str}_{ij},z^{tex}_{ij}$. Since
$\hat{x}_{ij}$ is generated from the combination of $z^{str}_{i}$ and
$z^{tex}_{j}$, $z^{str}_{ij}$ should be similar to $z^{str}_{i}$ (i.e. the
structure should not change). This is enforced using the cosine similarity
loss

$\mathcal{L}_{str}=1-\langle z^{str}_{i},z^{str}_{ij}\rangle$ (4)

where $\langle\cdot\rangle$ denotes cosine similarity.

###  2.2 Graph Convolutional Adversarial Network

To learn more global relationships between samples, the features from the
feature disentanglement module is used to construct an instance graph and
which is processed with a Graph Convolutional Network (GCN). The GCN learns
layer-wise propagation operations that can be applied directly on graphs.
Given an undirected graph with $m$ nodes, the set of edges between nodes, and
an adjacency matrix $\textbf{A}\in R^{m\times m}$, a linear transformation of
graph convolution is defined as the multiplication of a graph signal
$\textbf{X}\in R^{k\times m}$ with a filter $\textbf{W}\in R^{k\times c}$ :

$\textbf{Z}=\hat{\textbf{D}}^{-\frac{1}{2}}\hat{\textbf{A}}\hat{\textbf{D}}^{-\frac{1}{2}}\textbf{X}^{T}\textbf{W},$
(5)

where $\hat{\textbf{A}}=\textbf{A}+\textbf{I}$, I being the identity matrix,
and $\hat{\textbf{D}_{ii}}=\sum_{j}\hat{\textbf{A}}_{ij}$. The output is a
$c\times m$ matrix Z. The GCN can be constructed by stacking multiple graph
convolutional layers followed by a non-linear operation (such as ReLU).

In our implementation, each node in the instance graph represents a latent
feature of a sample (either source or target domain) $[z^{tex},z^{str}]$. The
adjacency matrix is constructed using the cosine similarity score as
$\hat{\textbf{A}}=\textbf{X}_{sc}\textbf{X}_{sc}^{T}$. Note that the cosine
similarity score quantifies the semantic similarity between two samples . Here
$\textbf{X}_{sc}\in R^{w\times h}$ is the matrix of cosine similarity scores,
$w$ is the batch size, and $h=2$ is the dimension of the similarity scores of
each sample obtained from corresponding $[z_{tex},z_{str}]$.

The flow of our proposed approach is shown in Figure 1(a). Our network is
trained by minimizing the following objective function:

$\mathcal{L}(\mathcal{X}_{S},\mathcal{Y}_{S},\mathcal{X}_{T})=\mathcal{L}_{C}(\mathcal{X}_{S},\mathcal{Y}_{S})+\lambda_{adv}\mathcal{L}_{Adv}(\mathcal{X}_{S},\mathcal{X}_{T})$
(6)

The classification loss $\mathcal{L}_{C}$ is a cross entropy loss
$\mathcal{L}_{C}(\mathcal{X}_{S},\mathcal{Y}_{S})=-\sum_{c=1}^{n_{s}}y_{c}\log(p_{c})$,
$y_{c}$ is the indicator variable and $p_{c}$ is the class probability.

The second loss component is an adversarial loss function defined in Eq. 7. A
domain classifier $D$ identifies whether features from the graph neural
network $G_{GNN}$ are from the source or target domain. Conversely, $G_{GNN}$,
is being trained to produce samples that can fool $D$. This two-player minimax
game will reach an equilibrium state when features from $G_{GNN}$ are domain-
invariant.

$\mathcal{L}_{Adv}(\mathcal{X}_{S},\mathcal{X}_{T})=\mathbb{E}_{x\in
D_{S}}[\log(1-D(G_{GNN}(x)))]+\mathbb{E}_{x\in D_{T}}[\log(D(G_{GNN}(x)))]$
(7)

Training And Implementation: Given X and A the GCN features are obtained
according to Eq.5. Source and target domain graphs are individually
constructed and fed into the parameters-shared GCNs to learn representations.
The dimension of $z_{tex}$ is $256$, while $z_{str}$ is $64\times 64$. VAE
Network: The encoder consists of $3$ convolution blocks followed by max
pooling. The decoder is symmetrically designed to the encoder. $3\times 3$
convolution filters are used and $64,32,32$ filters are used in each conv
layer. The input to the VAE is $256\times 256$. For the domain classifier we
use DenseNet-121 network with pre-trained weights from ImageNet and fine-tuned
using self supervised learning. As a pre-text task we use the classifier to
predict the intensity values of masked regions.

##  3 Experiments And Results

###  3.1 Results For CAMELYON17 Dataset

Dataset Description: We use the CAMELYON17 dataset [14] to evaluate the
performance of the proposed method on tumor/normal classification. In this
dataset, a total of 500 $H\&E$ stained WSIs are collected from five medical
centers (denoted as by $C1_{17},C2_{17},C3_{17},C4_{17}$, $C5_{17}$
respectively). $50$ of these WSIs include lesion-level annotations. All
positive and negative WSIs are randomly split into training/validation/test
sets and provided by the organizers in a $50/30/20\%$ split for the individual
medical centers to obtain the following split: $C1_{17}$:37/22/15, $C2_{17}$:
34/20/14, $C3_{17}$: 43/24/18, $C4_{17}$: 35/20/15, $C5_{17}$: 36/20/15.
$256\times 256$ image patches are extracted from the annotated tumors for
positive patches and from tissue regions of WSIs without tumors for negative
patches. We use $\lambda_{1}=0.9,\lambda_{2}=1.2,\lambda_{adv}=1.3$.

Implementation Details Since the images have been taken from different medical
centers their appearance varies despite sharing the same disease labels. This
is due to slightly different protocols of $H\&E$ staining. Stain normalization
has been a widely explored topic which aims to standardize the appearance of
images across all centers, which is equivalent to domain adaptation. Recent
approaches to stain normalization/domain adaptation favour use of GANs and
other deep learning methods. We compare our approach to recent approaches and
also with [15] which explicitly performs UDA using MixUp.

To evaluate our method’s performance: 1) We use $C1_{17}$ as the source
dataset and train a ResNet-50 classifier [28] (ResNetC1). Each remaining
dataset from the other centers are, separately, taken as the target dataset,
the corresponding domain adapted images are generated, and classified using
ResNetC1. As a baseline, we perform the experiment without domain adaptation
denoted as $No-UDA$ where ResNetC1 is used to classify images from other
centers. We report results for a network trained in a fully-supervised manner
on the training set from the same domain ($FSL-SameDomain$) to give an upper-
bound expectation for a UDA model trained on other domain’s data, where a
ResNet-50 is trained on the training images and used to classify test images,
all from the same hospital. This approach will give the best results for a
given classifier (in our case ResNet-50). All the above experiments are
repeated using each of $C2_{17},C3_{17},C4_{17},C5_{17}$ as the source
dataset. Table 1 summarizes our results. PyTorch was used in all
implementations.

The results in Table 1 show that UDA methods are better than conventional
stain normalization approaches as evidenced by the superior performance of our
proposed method and [15]. Our method performs the best amongst all the
methods, and approaches $FSL-SameDomain$, the theoretical maximum performance
a domain adaptation method can achieve. This shows that our proposed GCN based
approach performs better than other UDA methods. The ablation studies also
show that our proposed individual loss term $\mathcal{L}_{Str}$ has a
significant contribution to the overall performance of our method and
excluding it significantly degrades the performance.

Comparison and Proposed Method Proposed Ablation Study Results No UDA MMD
CycleGAN [174] [21] [84] [15] Method FSL-Same $\mathcal{L}_{Str}$
$\mathcal{L}_{Swap}$ $Center~{}1$ 0.8068 0.8742 0.9010 0.9123 0.9487 0.9668
0.979 0.988 0.991 0.979 0.953 $Center~{}2$ 0.7203 0.6926 0.7173 0.7347 0.8115
0.8537 0.948 0.963 0.972 0.951 0.941 $Center~{}3$ 0.7027 0.8711 0.8914 0.9063
0.8727 0.9385 0.946 0.958 0.965 0.948 0.932 $Center~{}4$ 0.8289 0.8578 0.8811
0.8949 0.9235 0.9548 0.965 0.979 0.986 0.967 0.949 $Center~{}5$ 0.8203 0.7854
0.8102 0.8223 0.9351 0.9462 0.942 0.949 0.957 0.934 0.921 $Average$ 0.7758
0.8162 0.8402 0.8541 0.8983 0.9320 0.956 0.967 0.974 0.956 0.939 $p$ 0.0001
0.0001 0.002 0.003 0.013 0.024 0.017 - 0.07 0.01 0.001

Table 1: Classification results in terms of AUC measures for different domain
adaptation methods on the CAMELYON17 dataset. Note: $FSL-SD$ is a fully-
supervised model trained on target domain data.

###  3.2 Results on Chest Xray Dataset

We use the following chest Xray datasets: NIH Chest Xray Dataset: The NIH
ChestXray14 dataset [177] has $112,120$ expert-annotated frontal-view X-rays
from $30,805$ unique patients and has $14$ disease labels. Original images
were resized to $256\times 256$, and
$\lambda_{1}=0.9,\lambda_{2}=1.2,\lambda_{adv}=1.2$. CheXpert Dataset: This
datset [33] has $224,316$ chest radiographs of $65,240$ patients labeled for
the presence of $14$ common chest conditions. The validation ground-truth is
obtained using majority voting from annotations of $3$ board-certified
radiologists. Original images were resized to $256\times 256$, and
$\lambda_{1}=0.95,\lambda_{2}=1.1,\lambda_{adv}=1.3$. These two datasets have
the same set of disease labels.

We divide both datasets into train/validation/test splits on the patient level
at $70/10/20$ ratio, such that images from one patient are in only one of the
splits. Then we train a DenseNet-121 [158] classifier on one dataset (say
NIH’s train split). Here the NIH dataset serves as the source data and
CheXpert is the target dataset. We then apply the trained model on the
training split of the NIH dataset and tested on the test split of the same
domain the results are denoted as $FSL-Same$. When we apply this model to the
test split of the CheXpert data without domain adaptation the results are
reported under No-$UDA$.

Tables 2,3 show classification results for different DA techniques where,
respectively, the NIH and CheXpert dataset were the source domain and the
performance metrics are for, respectively, CheXpert and NIH dataset’s _test
split_. We observe that UDA methods perform worse than $FSL-Same$. This is
expected since it is very challenging to perfectly account for domain shift.
However all UDA methods perform better than fully supervised methods trained
on one domain and applied on another without domain adaptation.

The DANN architecture [22] outperforms MMD and cycleGANs, and is on par with
graph convolutional methods GCAN [52] and GCN2 [31]. However our method
outperforms all compared methods which can be attributed to the combination of
GCNs, which learn more useful global relationships between different samples,
and feature disentanglement which in turn leads to more discriminative feature
learning.

Atel. Card. Eff. Infil. Mass Nodule Pneu. Pneumot. Consol. Edema Emphy. Fibr.
PT. Hernia No UDA 0.697 0.814 0.761 0.652 0.739 0.694 0.703 0.781 0.704 0.792
0.815 0.719 0.728 0.811 MMD 0.741 0.851 0.801 0.699 0.785 0.738 0.748 0.807
0.724 0.816 0.831 0.745 0.754 0.846 CycleGANs 0.765 0.874 0.824 0.736 0.817
0.758 0.769 0.832 0.742 0.838 0.865 0.762 0.773 0.864 DANN 0.792 0.902 0.851
0.761 0.849 0.791 0.802 0.869 0.783 0.862 0.894 0.797 0.804 0.892 FSL$-Same$
0.849 0.954 0.903 0.814 0.907 0.825 0.844 0.928 0.835 0.928 0.951 0.847 0.842
0.941 GCAN 0.798 0.908 0.862 0.757 0.858 0.803 0.800 0.867 0.776 0.865 0.908
0.811 0.799 0.904 GCN2 0.809 0.919 0.870 0.765 0.871 0.807 0.810 0.882 0.792
0.883 0.921 0.817 0.812 0.914 Proposed 0.825 0.931 0.884 0.788 0.890 0.818
0.828 0.903 0.818 0.910 0.934 0.828 0.830 0.923 Ablation Study Results
$\mathcal{L}_{Str}$ 0.811 0.914 0.870 0.771 0.872 0.808 0.819 0.884 0.803
0.887 0.916 0.809 0.812 0.907 $\mathcal{L}_{Swap}$ 0.787 0.885 0.845 0.741
0.843 0.780 0.791 0.856 0.776 0.859 0.887 0.785 0.786 0.883

Table 2: Classification results on the CheXpert dataset’s test split using NIH
data as the source domain. Note: $FSL-SD$ is a fully-supervised model trained
on target domain data.

Atel. Card. Eff. Infil. Mass Nodule Pneu. Pneumot. Consol. Edema Emphy. Fibr.
PT Hernia No UDA 0.718 0.823 0.744 0.730 0.739 0.694 0.683 0.771 0.712 0.783
0.803 0.711 0.710 0.785 MMD 0.734 0.846 0.762 0.741 0.785 0.738 0.709 0.793
0.731 0.801 0.821 0.726 0.721 0.816 CycleGANs 0.751 0.861 0.785 0.761 0.817
0.758 0.726 0.814 0.746 0.818 0.837 0.741 0.737 0.836 DANN 0.773 0.882 0.819
0.785 0.837 0.791 0.759 0.838 0.770 0.836 0.863 0.766 0.762 0.861 FSL$-Same$
0.814 0.929 0.863 0.821 0.869 0.825 0.798 0.863 0.805 0.872 0.904 0.802 0.798
0.892 GCAN 0.78 0.895 0.811 0.777 0.828 0.782 0.751 0.832 0.765 0.828 0.857
0.761 0.756 0.851 GCN2 0.786 0.906 0.833 0.789 0.831 0.802 0.763 0.835 0.774
0.837 0.868 0.768 0.763 0.860 Proposed 0.798 0.931 0.884 0.799 0.843 0.818
0.773 0.847 0.784 0.849 0.879 0.779 0.774 0.868 Ablation Study Results
$\mathcal{L}_{Str}$ 0.788 0.919 0.875 0.786 0.832 0.804 0.768 0.834 0.778
0.835 0.863 0.766 0.765 0.853 $\mathcal{L}_{Swap}$ 0.770 0.903 0.860 0.771
0.811 0.788 0.748 0.814 0.761 0.816 0.845 0.746 0.749 0.836

Table 3: Classification results on the NIH Xray dataset’s test split using
CheXpert data as the source domain. Note: $FSL-SD$ is a fully-supervised model
trained on target domain data.

##  4 Conclusion

In this paper we propose an UDA method using graph adversarial convolutional
networks and feature disentanglement using a novel loss term. Our novel loss
term extends the swapped autoencoder architecture thus learning more
discriminative features. Our graph convolutional network based unsupervised
domain adaptation method outperforms conventional CNN methods as graphs better
learn the interaction between samples by focusing on more global interactions
while CNNs focus on the local neighborhood. This enables GCN to perform better
UDA as demonstrated by results on multiple datasets. While feature
disentanglement also contributes to improved performance, there is scope for
improvement. In future work we wish to explore stronger disentanglement
techniques, and aim to extend this approach for segmentation-based tasks.

## References

  * [1] Ahmedt-Aristizabal, D., Armin, M.A., Denman, S., Fookes, C., Petersson, L.: Graph-based deep learning for medical diagnosis and analysis: Past, present and future. Sensors 21(14), 47–58 (Jul 2021)
  * [2] Ahn, E., Kumar, A., Fulham, M., Feng, D., Kim, J.: Unsupervised domain adaptation to classify medical images using zero-bias convolutional auto-encoders and context-based feature augmentation. IEEE TMI 39(7), 2385–2394 (2020)
  * [3] Antony, B., Sedai, S., Mahapatra, D., Garnavi, R.: Real-time passive monitoring and assessment of pediatric eye health. In: US Patent App. 16/178,757 (2020)
  * [4] Bastide, P., Kiral-Kornek, I., Mahapatra, D., Saha, S., Vishwanath, A., Cavallar, S.V.: Machine learned optimizing of health activity for participants during meeting times. In: US Patent App. 15/426,634 (2018)
  * [5] Bastide, P., Kiral-Kornek, I., Mahapatra, D., Saha, S., Vishwanath, A., Cavallar, S.V.: Visual health maintenance and improvement. In: US Patent 9,993,385 (2018)
  * [6] Bastide, P., Kiral-Kornek, I., Mahapatra, D., Saha, S., Vishwanath, A., Cavallar, S.V.: Crowdsourcing health improvements routes. In: US Patent App. 15/611,519 (2019)
  * [7] Bozorgtabar, B., Mahapatra, D., von Teng, H., Pollinger, A., Ebner, L., Thiran, J.P., Reyes, M.: Informative sample generation using class aware generative adversarial networks for classification of chest xrays. Computer Vision and Image Understanding 184, 57–65 (2019)
  * [8] Bozorgtabar, B., Mahapatra, D., von Teng, H., Pollinger, A., Ebner, L., Thiran, J.P., Reyes, M.: Informative sample generation using class aware generative adversarial networks for classification of chest xrays. In: arXiv preprint arXiv:1904.10781 (2019)
  * [9] Bozorgtabar, B., Mahapatra, D., Thiran, J.P.: Exprada: Adversarial domain adaptation for facial expression analysis. In Press Pattern Recognition 100, 15–28 (2020)
  * [10] Bozorgtabar, B., Mahapatra, D., Thiran, J.P., Shao, L.: SALAD: Self-supervised aggregation learning for anomaly detection on x-rays. In: In Proc. MICCAI. pp. 468–478 (2020)
  * [11] Bozorgtabar, B., Mahapatra, D., Vray, G., Thiran, J.P.: Anomaly detection on x-rays using self-supervised aggregation learning. In: arXiv preprint arXiv:2010.09856 (2020)
  * [12] Bozorgtabar, B., Mahapatra, D., Zlobec, I., Rau, T., Thiran, J.: Computational pathology. Frontiers in Medicine 7 (2020)
  * [13] Bozorgtabar, B., Rad, M.S., Mahapatra, D., Thiran, J.P.: Syndemo: Synergistic deep feature alignment for joint learning of depth and ego-motion. In: In Proc. IEEE ICCV (2019)
  * [14] Bándi, P., , et al.: From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge. IEEE Trans. Med. Imag. 38(2), 550–560 (2019)
  * [15] Chang, J.R., Wu, M.S., et al.: Stain mix-up: Unsupervised domain generalization for histopathology images. In: MICCAI 2021. pp. 117–126 (2021)
  * [16] Chen, C., Dou, Q., Chen, H., et al.: Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Trans. Med. Imag. 39(7), 2494–2505 (2020)
  * [17] Das, S.D., Dutta, S., Shah, N.A., Mahapatra, D., Ge, Z.: Anomaly detection in retinal images using multi-scale deep feature sparse coding. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). pp. 1–5 (2022). https://doi.org/10.1109/ISBI52829.2022.9761713
  * [18] Das, S.D., Dutta, S., Shah, N.A., Mahapatra, D., Ge, Z.: Anomaly detection in retinal images using multi-scale deep feature sparse coding. In: arXiv preprint arXiv:2201.11506 (2022)
  * [19] Devika, K., Mahapatra, D., Subramanian, R., Oruganti, V.R.M.: Outlier-based autism detection using longitudinal structural mri. IEEE Access 10, 27794–27808 (2022). https://doi.org/10.1109/ACCESS.2022.3157613
  * [20] Devika, K., Mahapatra, D., Subramanian, R., Oruganti, V.R.M.: Outlier-based autism detection using longitudinal structural mri. In: arXiv preprint arXiv:2202.09988 (2022)
  * [21] Gadermayr, M., Appel, V., et al.: Which way round?: A study on the performance of stain-translation for segmenting arbitrarily dyed histological images. In: MICCAI (I). pp. 165–173 (2018)
  * [22] Ganin, Y., Ustinova, E., et al.: Domain-adversarial training of neural networks (2016)
  * [23] Garnavi, R., Mahapatra, D., Roy, P., Tennakoon, R.: System and method to teach and evaluate image grading performance using prior learned expert knowledge base. In: US Patent App. 10,657,838 (2020)
  * [24] Ge, Z., Mahapatra, D., Chang, X., Chen, Z., Chi, L., Lu, H.: Improving multi-label chest x-ray disease diagnosis by exploiting disease and health labels dependencies. In press Multimedia Tools and Application pp. 1–14 (2019)
  * [25] Ge, Z., Mahapatra, D., Sedai, S., Garnavi, R., Chakravorty, R.: Chest x-rays classification: A multi-label and fine-grained problem. In: arXiv preprint arXiv:1807.07247 (2018)
  * [26] Gopinath, K., Desrosiers, C., Lombaert, H.: Graph domain adaptation for alignment-invariant brain surface segmentation (2020)
  * [27] Gulshan, V., Peng, L., , et. al.: Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA 316(22), 2402–2410 (12 2016)
  * [28] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: In Proc. CVPR (2016)
  * [29] Heimann, T., Mountney, P., John, M., Ionasec, R.: Learning without labeling: Domain adaptation for ultrasound transducer localization. In: MICCAI 2013. pp. 49–56 (2013)
  * [30] Hong, Y., Chen, G., Yap, P.T., Shen, D.: Multifold acceleration of diffusion mri via deep learning reconstruction from slice-undersampled data. In: IPMI. pp. 530–541 (2019)
  * [31] Hong, Y., Kim, J., Chen, G., Lin, W., Yap, P.T., Shen, D.: Longitudinal prediction of infant diffusion mri data via graph convolutional adversarial networks. IEEE Transactions on Medical Imaging 38(12), 2717–2725 (2019)
  * [32] Hoog, J.D., Mahapatra, D., Garnavi, R., Jalali, F.: Personalized monitoring of injury rehabilitation through mobile device imaging. In: US Patent App. 16/589,046 (2021)
  * [33] Irvin, J., Rajpurkar, P., et. al.: CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: arXiv preprint arXiv:1901.07031 (2017)
  * [34] Ju, L., Wang, X., Wang, L., Liu, T., Zhao, X., Drummond, T., Mahapatra, D., Ge, Z.: Relational subsets knowledge distillation for long-tailed retinal diseases recognition. In: arXiv preprint arXiv:2104.11057 (2021)
  * [35] Ju, L., Wang, X., Wang, L., Mahapatra, D., Zhao, X., Harandi, M., Drummond, T., Liu, T., Ge, Z.: Improving medical image classification with label noise using dual-uncertainty estimation. In: arXiv preprint arXiv:2103.00528 (2020)
  * [36] Ju, L., Wang, X., Wang, L., Mahapatra, D., Zhao, X., Zhou, Q., Liu, T., Ge, Z.: Improving medical images classification with label noise using dual-uncertainty estimation. IEEE Transactions on Medical Imaging pp. 1–1 (2022). https://doi.org/10.1109/TMI.2022.3141425
  * [37] Ju, L., Wang, X., Zhao, X., Lu, H., Mahapatra, D., Bonnington, P., Ge, Z.: Synergic adversarial label learning for grading retinal diseases via knowledge distillation and multi-task learning. IEEE JBHI 100, 1–14 (2020)
  * [38] Ju, L., Wang, X., Zhao, X., Lu, H., Mahapatra, D., Ge, Z.: Relational subsets knowledge distillation for long-tailed retinal diseases recognition. In: In MICCAI 2021. pp. 1–11 (2021)
  * [39] Kamnitsas, K., Baumgartner, C., et al.: Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In: IPMI. pp. 597–609 (2017)
  * [40] Kuanar, S., Athitsos, V., Mahapatra, D., Rajan, A.: Multi-scale deep learning architecture for nucleus detection in renal cell carcinoma microscopy image. In: arXiv preprint arXiv:2104.13557 (2021)
  * [41] Kuanar, S., Athitsos, V., Mahapatra, D., Rao, K., Akhtar, Z., Dasgupta, D.: Low dose abdominal ct image reconstruction: An unsupervised learning based approach. In: In Proc. IEEE ICIP. pp. 1351–1355 (2019)
  * [42] Kuanar, S., Mahapatra, D., Athitsos, V., Rao, K.: Gated fusion network for sao filter and inter frame prediction in versatile video coding. In: arXiv preprint arXiv:2105.12229 (2021)
  * [43] Kuanar, S., Rao, K., Mahapatra, D., Bilas, M.: Night time haze and glow removal using deep dilated convolutional network. In: arXiv preprint arXiv:1902.00855 (2019)
  * [44] Kuanar, S., Mahapatra, D., Bilas, M., Rao, K.: Multi-path dilated convolution network for haze and glow removal in night time images. The Visual Computer 38(3), 1121–1134 (2022)
  * [45] Kuang, H., Guthier, B., Saini, M., Mahapatra, D., Saddik, A.E.: A real-time smart assistant for video surveillance through handheld devices. In: In Proc: ACM Intl. Conf. Multimedia. pp. 917–920 (2014)
  * [46] Kumagai, A., Iwata, T.: Unsupervised domain adaptation by matching distributions based on the maximum mean discrepancy via unilateral transformations 33, 4106–4113 (Jul 2019)
  * [47] Li, F., Li, W., Qin, S., Wang, L.: Mdfa-net: Multiscale dual-path feature aggregation network for cardiac segmentation on multi-sequence cardiac mr. Knowledge-Based Systems 215, 106776 (2021)
  * [48] Li, Z., Mahapatra, D., J.Tielbeek, Stoker, J., van Vliet, L., Vos, F.: Image registration based on autocorrelation of local structure. IEEE Trans. Med. Imaging 35(1), 63–75 (2016)
  * [49] Liu, A.H., Liu, Y.C., Yeh, Y.Y., Wang, Y.C.F.: A unified feature disentangler for multi-domain image translation and manipulation. Advances in neural information processing systems 31 (2018)
  * [50] Liu, Y., Gadepalli, K., et al.: Detecting cancer metastases on gigapixel pathology images. In: arXiv preprint arXiv:1703.02442 (2017)
  * [51] Luo, Z., Zou, Y., et al.: Label efficient learning of transferable representations across domains and tasks. In: Proceedings of NeuRIPS. p. 164–176 (2017)
  * [52] Ma, X., Zhang, T., Xu, C.: Gcan: Graph convolutional adversarial network for unsupervised domain adaptation. In: IEEE CVPR. pp. 8258–8268 (2019)
  * [53] Mahapatra, D.: Elastic registration of cardiac perfusion images using saliency information. Sequence and Genome Analysis – Methods and Applications pp. 351–364 (2011)
  * [54] Mahapatra, D.: Neonatal brain mri skull stripping using graph cuts and shape priors. In: In Proc: MICCAI workshop on Image Analysis of Human Brain Development (IAHBD) (2011)
  * [55] Mahapatra, D.: Cardiac lv and rv segmentation using mutual context information. In: Proc. MICCAI-MLMI. pp. 201–209 (2012)
  * [56] Mahapatra, D.: Groupwise registration of dynamic cardiac perfusion images using temporal information and segmentation information. In: In Proc: SPIE Medical Imaging (2012)
  * [57] Mahapatra, D.: Landmark detection in cardiac mri using learned local image statistics. In: Proc. MICCAI-Statistical Atlases and Computational Models of the Heart. Imaging and Modelling Challenges (STACOM). pp. 115–124 (2012)
  * [58] Mahapatra, D.: Skull stripping of neonatal brain mri: Using prior shape information with graphcuts. J. Digit. Imaging 25(6), 802–814 (2012)
  * [59] Mahapatra, D.: Cardiac image segmentation from cine cardiac mri using graph cuts and shape priors. J. Digit. Imaging 26(4), 721–730 (2013)
  * [60] Mahapatra, D.: Cardiac mri segmentation using mutual context information from left and right ventricle. J. Digit. Imaging 26(5), 898–908 (2013)
  * [61] Mahapatra, D.: Graph cut based automatic prostate segmentation using learned semantic information. In: Proc. IEEE ISBI. pp. 1304–1307 (2013)
  * [62] Mahapatra, D.: Joint segmentation and groupwise registration of cardiac perfusion images using temporal information. J. Digit. Imaging 26(2), 173–182 (2013)
  * [63] Mahapatra, D.: An automated approach to cardiac rv segmentation from mri using learned semantic information and graph cuts. J. Digit. Imaging. 27(6), 794–804 (2014)
  * [64] Mahapatra, D.: Combining multiple expert annotations using semi-supervised learning and graph cuts for medical image segmentation. Computer Vision and Image Understanding 151(1), 114–123 (2016)
  * [65] Mahapatra, D.: Retinal image quality classification using neurobiological models of the human visual system. In: In Proc. MICCAI-OMIA. pp. 1–8 (2016)
  * [66] Mahapatra, D.: Consensus based medical image segmentation using semi-supervised learning and graph cuts. In: arXiv preprint arXiv:1612.02166 (2017)
  * [67] Mahapatra, D.: Semi-supervised learning and graph cuts for consensus based medical image segmentation. Pattern Recognition 63(1), 700–709 (2017)
  * [68] Mahapatra, D.: Amd severity prediction and explainability using image registration and deep embedded clustering. In: arXiv preprint arXiv:1907.03075 (2019)
  * [69] Mahapatra, D.: Generative adversarial networks and domain adaptation for training data independent image registration. In: arXiv preprint arXiv:1910.08593 (2019)
  * [70] Mahapatra, D.: Registration of histopathogy images using structural information from fine grained feature maps. In: arXiv preprint arXiv:2007.02078 (2020)
  * [71] Mahapatra, D.: Interpretability-driven sample selection using self supervised learning for disease classification and segmentation. In: arXiv preprint arXiv:2104.06087 (2021)
  * [72] Mahapatra, D.: Learning of inter-label geometric relationships using self-supervised learning: Application to gleason grade segmentation. In: arXiv preprint arXiv:2110.00404 (2021)
  * [73] Mahapatra, D., Agarwal, K., Khosrowabadi, R., Prasad, D.: Recent advances in statistical data and signal analysis: Application to real world diagnostics from medical and biological signals. In: Computational and mathematical methods in medicine (2016)
  * [74] Mahapatra, D., Antony, B., Sedai, S., Garnavi, R.: Deformable medical image registration using generative adversarial networks. In: In Proc. IEEE ISBI. pp. 1449–1453 (2018)
  * [75] Mahapatra, D., Bozorgtabar, B.: Retinal vasculature segmentation using local saliency maps and generative adversarial networks for image super resolution. In: arXiv preprint arXiv:1710.04783 (2017)
  * [76] Mahapatra, D., Bozorgtabar, B.: Progressive generative adversarial networks for medical image super resolution. In: arXiv preprint arXiv:1902.02144 (2019)
  * [77] Mahapatra, D., Bozorgtabar, B., Garnavi, R.: Image super-resolution using progressive generative adversarial networks for medical image analysis. Computerized Medical Imaging and Graphics 71, 30–39 (2019)
  * [78] Mahapatra, D., Bozorgtabar, B., Ge, Z.: Medical image classification using generalized zero shot learning. In: In IEEE CVAMD 2021. pp. 3344–3353 (2021)
  * [79] Mahapatra, D., Bozorgtabar, B., Kuanar, S., Ge, Z.: Self-supervised multimodal generalized zero shot learning for gleason grading. In: In MICCAI-DART 2021. pp. 1–11 (2021)
  * [80] Mahapatra, D., Bozorgtabar, B., Shao, L.: Pathological retinal region segmentation from oct images using geometric relation based augmentation. In: In Proc. IEEE CVPR. pp. 9611–9620 (2020)
  * [81] Mahapatra, D., Bozorgtabar, B., Thiran, J.P., Shao, L.: Pathological retinal region segmentation from oct images using geometric relation based augmentation. In: arXiv preprint arXiv:2003.14119 (2020)
  * [82] Mahapatra, D., Bozorgtabar, B., Thiran, J.P., Shao, L.: Structure preserving stain normalization of histopathology images using self supervised semantic guidance. In: In Proc. MICCAI. pp. 309–319 (2020)
  * [83] Mahapatra, D., Bozorgtabar, B., Thiran, J.P., Shao, L.: Structure preserving stain normalization of histopathology images using self supervised semantic guidance. In: arXiv preprint arXiv:2008.02101 (2020)
  * [84] Mahapatra, D., Bozorgtabar, B., Thiran, J.P., Shao, L.: Structure preserving stain normalization of histopathology images using self supervised semantic guidance. In: In Proc. MICCAI. pp. 309–319 (2020)
  * [85] Mahapatra, D., Bozorgtabar, S., Hewavitahranage, S., Garnavi, R.: Image super resolution using generative adversarial networks and local saliencymaps for retinal image analysis,. In: In Proc. MICCAI. pp. 382–390 (2017)
  * [86] Mahapatra, D., Bozorgtabar, S., Thiran, J.P., Reyes, M.: Efficient active learning for image classification and segmentation using a sample selection and conditional generative adversarial network. In: In Proc. MICCAI (2). pp. 580–588 (2018)
  * [87] Mahapatra, D., Buhmann, J.: Obtaining consensus annotations for retinal image segmentation using random forest and graph cuts. In: In Proc. OMIA. pp. 41–48 (2015)
  * [88] Mahapatra, D., Buhmann, J.: Visual saliency based active learning for prostate mri segmentation. In: In Proc. MLMI. pp. 9–16 (2015)
  * [89] Mahapatra, D., Buhmann, J.: Visual saliency-based active learning for prostate magnetic resonance imaging segmentation. SPIE Journal of Medical Imaging 3(1), 014003 (2016)
  * [90] Mahapatra, D., Buhmann, J.: Automatic cardiac rv segmentation using semantic information with graph cuts. In: Proc. IEEE ISBI. pp. 1094–1097 (2013)
  * [91] Mahapatra, D., Buhmann, J.: Analyzing training information from random forests for improved image segmentation. IEEE Trans. Imag. Proc. 23(4), 1504–1512 (2014)
  * [92] Mahapatra, D., Buhmann, J.: Prostate mri segmentation using learned semantic knowledge and graph cuts. IEEE Trans. Biomed. Engg. 61(3), 756–764 (2014)
  * [93] Mahapatra, D., Buhmann, J.: A field of experts model for optic cup and disc segmentation from retinal fundus images. In: In Proc. IEEE ISBI. pp. 218–221 (2015)
  * [94] Mahapatra, D., Garnavi, R., Roy, P., Tennakoon, R.: System and method to teach and evaluate image grading performance using prior learned expert knowledge base. In: US Patent App. 15/459,457 (2018)
  * [95] Mahapatra, D., Garnavi, R., Roy, P., Tennakoon, R.: System and method to teach and evaluate image grading performance using prior learned expert knowledge base. In: US Patent App. 15/814,590 (2018)
  * [96] Mahapatra, D., Garnavi, R., Sedai, S., Roy, P.: Joint segmentation and characteristics estimation in medical images. In: US Patent App. 15/234,426 (2017)
  * [97] Mahapatra, D., Garnavi, R., Sedai, S., Roy, P.: Retinal image quality assessment, error identification and automatic quality correction. In: US Patent 9,779,492 (2017)
  * [98] Mahapatra, D., Garnavi, R., Sedai, S., Tennakoon, R.: Classification of severity of pathological condition using hybrid image representation. In: US Patent App. 15/426,634 (2018)
  * [99] Mahapatra, D., Garnavi, R., Sedai, S., Tennakoon, R.: Generating an enriched knowledge base from annotated images. In: US Patent App. 15/429,735 (2018)
  * [100] Mahapatra, D., Garnavi, R., Sedai, S., Tennakoon, R., Chakravorty, R.: Early prediction of age related macular degeneration by image reconstruction. In: US Patent App. 15/854,984 (2018)
  * [101] Mahapatra, D., Garnavi, R., Sedai, S., Tennakoon, R., Chakravorty, R.: Early prediction of age related macular degeneration by image reconstruction. In: US Patent 9,943,225 (2018)
  * [102] Mahapatra, D., Ge, Z.: Combining transfer learning and segmentation information with gans for training data independent image registration. In: arXiv preprint arXiv:1903.10139 (2019)
  * [103] Mahapatra, D., Ge, Z.: Training data independent image registration with gans using transfer learning and segmentation information. In: In Proc. IEEE ISBI. pp. 709–713 (2019)
  * [104] Mahapatra, D., Ge, Z.: Training data independent image registration using generative adversarial networks and domain adaptation. Pattern Recognition 100, 1–14 (2020)
  * [105] Mahapatra, D., Ge, Z., Sedai, S.: Joint registration and segmentation of images using deep learning. In: US Patent App. 16/001,566 (2019)
  * [106] Mahapatra, D., Ge, Z., Sedai, S., Chakravorty., R.: Joint registration and segmentation of xray images using generative adversarial networks. In: In Proc. MICCAI-MLMI. pp. 73–80 (2018)
  * [107] Mahapatra, D., Gilani, S., Saini., M.: Coherency based spatio-temporal saliency detection for video object segmentation. IEEE Journal of Selected Topics in Signal Processing. 8(3), 454–462 (2014)
  * [108] Mahapatra, D., J.Tielbeek, Makanyanga, J., Stoker, J., Taylor, S., Vos, F., Buhmann, J.: Automatic detection and segmentation of crohn’s disease tissues from abdominal mri. IEEE Trans. Med. Imaging 32(12), 1232–1248 (2013)
  * [109] Mahapatra, D., J.Tielbeek, Makanyanga, J., Stoker, J., Taylor, S., Vos, F., Buhmann, J.: Active learning based segmentation of crohn’s disease using principles of visual saliency. In: Proc. IEEE ISBI. pp. 226–229 (2014)
  * [110] Mahapatra, D., J.Tielbeek, Makanyanga, J., Stoker, J., Taylor, S., Vos, F., Buhmann, J.: Combining multiple expert annotations using semi-supervised learning and graph cuts for crohn’s disease segmentation. In: In Proc: MICCAI-ABD (2014)
  * [111] Mahapatra, D., J.Tielbeek, Vos, F., Buhmann, J.: A supervised learning approach for crohn’s disease detection using higher order image statistics and a novel shape asymmetry measure. J. Digit. Imaging 26(5), 920–931 (2013)
  * [112] Mahapatra, D., Kuanar, S., Bozorgtabar, B., Ge, Z.: Self-supervised learning of inter-label geometric relationships for gleason grade segmentation. In: In MICCAI-DART 2021. pp. 57–67 (2021)
  * [113] Mahapatra, D., Li, Z., Vos, F., Buhmann, J.: Joint segmentation and groupwise registration of cardiac dce mri using sparse data representations. In: In Proc. IEEE ISBI. pp. 1312–1315 (2015)
  * [114] Mahapatra, D., Routray, A., Mishra, C.: An active snake model for classification of extreme emotions. In: IEEE International Conference on Industrial Technology (ICIT). pp. 2195–2199 (2006)
  * [115] Mahapatra, D., Roy, P., Sedai, S., Garnavi, R.: A cnn based neurobiology inspired approach for retinal image quality assessment. In: In Proc. EMBC. pp. 1304–1307 (2016)
  * [116] Mahapatra, D., Roy, P., Sedai, S., Garnavi, R.: Retinal image quality classification using saliency maps and cnns. In: In Proc. MICCAI-MLMI. pp. 172–179 (2016)
  * [117] Mahapatra, D., Roy, S., Sun, Y.: Retrieval of mr kidney images by incorporating shape information in histogram of low level features. In: In 13th International Conference on Biomedical Engineering. pp. 661–664 (2009)
  * [118] Mahapatra, D., Saha, S., Vishwanath, A., Bastide, P.: Generating hyperspectral image database by machine learning and mapping of color images to hyperspectral domain. In: US Patent App. 15/949,528 (2019)
  * [119] Mahapatra, D., Saini, M.: A particle filter framework for object tracking using visual-saliency information. Intelligent Multimedia Surveillance pp. 133–147 (2013)
  * [120] Mahapatra, D., Saini, M., Sun, Y.: Illumination invariant tracking in office environments using neurobiology-saliency based particle filter. In: IEEE ICME. pp. 953–956 (2008)
  * [121] Mahapatra, D., Sch$\ddot{u}$ffler, P., Tielbeek, J., Vos, F., Buhmann, J.: Semi-supervised and active learning for automatic segmentation of crohn’s disease. In: Proc. MICCAI, Part 2. pp. 214–221 (2013)
  * [122] Mahapatra, D., Sedai, S., Garnavi, R.: Elastic registration of medical images with gans. In: arXiv preprint arXiv:1805.02369 (2018)
  * [123] Mahapatra, D., Sedai, S., Halupka, K.: Uncertainty region based image enhancement. In: US Patent App. 10,832,074 (2020)
  * [124] Mahapatra, D., Singh, A.: Ct image synthesis using weakly supervised segmentation and geometric inter-label relations for covid image analysis. In: arXiv preprint arXiv:2106.10230 (2021)
  * [125] Mahapatra, D., Sun, Y.: Nonrigid registration of dynamic renal MR images using a saliency based MRF model. In: Proc. MICCAI. pp. 771–779 (2008)
  * [126] Mahapatra, D., Sun, Y.: Registration of dynamic renal mr images using neurobiological model of saliency. In: Proc. ISBI. pp. 1119–1122 (2008)
  * [127] Mahapatra, D., Sun, Y.: Using saliency features for graphcut segmentation of perfusion kidney images. In: In 13th International Conference on Biomedical Engineering (2008)
  * [128] Mahapatra, D., Sun, Y.: Joint registration and segmentation of dynamic cardiac perfusion images using mrfs. In: Proc. MICCAI. pp. 493–501 (2010)
  * [129] Mahapatra, D., Sun, Y.: Mrf based joint registration and segmentation of dynamic renal mr images. In: Second International Conference on Digital Image Processing. vol. 7546, pp. 285–290 (2010)
  * [130] Mahapatra, D., Sun., Y.: An mrf framework for joint registration and segmentation of natural and perfusion images. In: Proc. IEEE ICIP. pp. 1709–1712 (2010)
  * [131] Mahapatra, D., Sun, Y.: Retrieval of perfusion images using cosegmentation and shape context information. In: Proc. APSIPA Annual Summit and Conference (ASC). vol. 35 (2010)
  * [132] Mahapatra, D., Sun, Y.: Rigid registration of renal perfusion images using a neurobiology based visual saliency model. EURASIP Journal on Image and Video Processing. pp. 1–16 (2010)
  * [133] Mahapatra, D., Sun, Y.: Mrf based intensity invariant elastic registration of cardiac perfusion images using saliency information. IEEE Trans. Biomed. Engg. 58(4), 991–1000 (2011)
  * [134] Mahapatra, D., Sun, Y.: Orientation histograms as shape priors for left ventricle segmentation using graph cuts. In: In Proc: MICCAI. pp. 420–427 (2011)
  * [135] Mahapatra, D., Sun, Y.: Integrating segmentation information for improved mrf-based elastic image registration. IEEE Trans. Imag. Proc. 21(1), 170–183 (2012)
  * [136] Mahapatra, D., Tielbeek, J., Buhmann, J., Vos, F.: A supervised learning based approach to detect crohn’s disease in abdominal mr volumes. In: Proc. MICCAI workshop Computational and Clinical Applications in Abdominal Imaging(MICCAI-ABD). pp. 97–106 (2012)
  * [137] Mahapatra, D., Tielbeek, J., Vos, F., ., J.B.: Crohn’s disease tissue segmentation from abdominal mri using semantic information and graph cuts. In: Proc. IEEE ISBI. pp. 358–361 (2013)
  * [138] Mahapatra, D., Tielbeek, J., Vos, F., Buhmann, J.: Localizing and segmenting crohn’s disease affected regions in abdominal mri using novel context features. In: Proc. SPIE Medical Imaging (2013)
  * [139] Mahapatra, D., Tielbeek, J., Vos, F., Buhmann, J.: Weakly supervised semantic segmentation of crohn’s disease tissues from abdominal mri. In: Proc. IEEE ISBI. pp. 832–835 (2013)
  * [140] Mahapatra, D., Vos, F., Buhmann, J.: Crohn’s disease segmentation from mri using learned image priors. In: In Proc. IEEE ISBI. pp. 625–628 (2015)
  * [141] Mahapatra, D., Vos, F., Buhmann, J.: Active learning based segmentation of crohns disease from abdominal mri. Computer Methods and Programs in Biomedicine 128(1), 75–85 (2016)
  * [142] Mahapatra, D., Winkler, S., Yen, S.: Motion saliency outweighs other low-level features while watching videos. In: SPIE HVEI. pp. 1–10 (2008)
  * [143] Mahapatra, D.: Registration and segmentation methodology for perfusion mr images: Application to cardiac and renal images. - pp. – (2011)
  * [144] Mahapatra, D.: Registration and segmentation methodology for perfusion mr images: Application to cardiac and renal images. - pp. – (2011)
  * [145] Mahapatra, D.: Multimodal generalized zero shot learning for gleason grading using self-supervised learning. In: arXiv preprint arXiv:2111.07646 (2021)
  * [146] Mahapatra, D., Ge, Z.: MR image super resolution by combining feature disentanglement CNNs and vision transformers. In: Medical Imaging with Deep Learning (2022)
  * [147] Mahapatra, D., Ge, Z.: Mr image super resolution by combining feature disentanglement cnns and vision transformers. In: - (2022)
  * [148] Mahapatra, D., Ge, Z.: Mr image super resolution by combining feature disentanglement cnns and vision transformers. In: - (2022)
  * [149] Mahapatra, D., Ge, Z., Reyes, M.: Self-supervised generalized zero shot learning for medical image classification using novel interpretable saliency maps. IEEE Transactions on Medical Imaging pp. 1–1 (2022). https://doi.org/10.1109/TMI.2022.3163232
  * [150] Mahapatra, D., Korevaar, S., Tennakoon, R.: Gcn based unsupervised domain adaptation with feature disentanglement for medical image classification. In: \- (2022)
  * [151] Mahapatra, D., Poellinger, A., Shao, L., Reyes, M.: Interpretability-driven sample selection using self supervised learning for disease classification and segmentation. IEEE TMI pp. 1–15 (2021)
  * [152] Motiian, S., Jones, Q., et al.: Few-shot adversarial domain adaptation. In: Proc. NeurIPS. p. 6673–6683 (2017)
  * [153] Ouyang, C., Kamnitsas, K., et al.: Data efficient unsupervised domain adaptation for cross-modality image segmentation. In: MICCAI. p. 669–677 (2019)
  * [154] Painchaud, N., Skandarani, Y., et al.: Cardiac segmentation with strong anatomical guarantees. IEEE Trans. Med. Imag. 39(11), 3703–3713 (2020)
  * [155] Pandey, A., Paliwal, B., Dhall, A., Subramanian, R., Mahapatra, D.: This explains that: Congruent image–report generation for explainable medical image analysis with cyclic generative adversarial networks. In: In MICCAI-iMIMIC 2021. pp. 1–11 (2021)
  * [156] Park, T., Zhu, J.Y., Wang, O., Lu, J., Shechtman, E., Efros, A.A., Zhang, R.: Swapping autoencoder for deep image manipulation. In: Advances in Neural Information Processing Systems (2020)
  * [157] Puybareau, É., Zhao, Z., et al.: Left atrial segmentation in a few seconds using fully convolutional network and transfer learning. In: STACOM Atrial Segmentation and LV Quantification Challenges. pp. 339–347 (2019)
  * [158] Rajpurkar, P., Irvin, J., et al.: Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. In: arXiv preprint arXiv:1711.05225, (2017)
  * [159] Roy, P., Chakravorty, R., Sedai, S., Mahapatra, D., Garnavi, R.: Automatic eye type detection in retinal fundus image using fusion of transfer learning and anatomical features. In: In Proc. DICTA. pp. 1–7 (2016)
  * [160] Roy, P., Tennakoon, R., Cao, K., Sedai, S., Mahapatra, D., Maetschke, S., Garnavi, R.: A novel hybrid approach for severity assessment of diabetic retinopathy in colour fundus images,. In: In Proc. IEEE ISBI. pp. 1078–1082 (2017)
  * [161] Roy, P., Mahapatra, D., Garnavi, R., Tennakoon, R.: System and method to teach and evaluate image grading performance using prior learned expert knowledge base. In: US Patent App. 10,984,674 (2021)
  * [162] Saini, M., Guthier, B., Kuang, H., Mahapatra, D., Saddik, A.: szoom: A framework for automatic zoom into high resolution surveillance videos. In: arXiv preprint arXiv:1909.10164 (2019)
  * [163] Sch$\ddot{u}$ffler, P., Mahapatra, D., Tielbeek, J., Vos, F., Makanyanga, J., Pends, D., Nio, C., Stoker, J., Taylor, S., Buhmann, J.: A model development pipeline for crohns disease severity assessment from magnetic resonance images. In: In Proc: MICCAI-ABD (2013)
  * [164] Sch$\ddot{u}$ffler, P., Mahapatra, D., Tielbeek, J., Vos, F., Makanyanga, J., Pends, D., Nio, C., Stoker, J., Taylor, S., Buhmann, J.: Semi automatic crohns disease severity assessment on mr imaging. In: In Proc: MICCAI-ABD (2014)
  * [165] Schüffler, P.J., Mahapatra, D., Vos, F.M., Buhmann, J.M.: Computer aided crohn’s disease severity assessment in mri. In: VIGOR++ Workshop 2014-Showcase of Research Outcomes and Future Outlook. pp. – (2014)
  * [166] Sedai, S., Mahapatra, D., Antony, B., Garnavi, R.: Joint segmentation and uncertainty visualization of retinal layers in optical coherence tomography images using bayesian deep learning. In: In Proc. MICCAI-OMIA. pp. 219–227 (2018)
  * [167] Sedai, S., Mahapatra, D., Ge, Z., Chakravorty, R., Garnavi, R.: Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in x-ray images. In: In Proc. MICCAI-MLMI. pp. 267–275 (2018)
  * [168] Sedai, S., Mahapatra, D., Hewavitharanage, S., Maetschke, S., Garnavi, R.: Semi-supervised segmentation of optic cup in retinal fundus images using variational autoencoder,. In: In Proc. MICCAI. pp. 75–82 (2017)
  * [169] Sedai, S., Roy, P., Mahapatra, D., Garnavi, R.: Segmentation of optic disc and optic cup in retinal fundus images using shape regression. In: In Proc. EMBC. pp. 3260–3264 (2016)
  * [170] Sedai, S., Roy, P., Mahapatra, D., Garnavi, R.: Segmentation of optic disc and optic cup in retinal images using coupled shape regression. In: In Proc. MICCAI-OMIA. pp. 1–8 (2016)
  * [171] Srivastava, S., Yaqub, M., Nandakumar, K., Ge, Z., Mahapatra, D.: Continual domain incremental learning for chest x-ray classification in low-resource clinical settings. In: In MICCAI-FAIR 2021. pp. 1–11 (2021)
  * [172] Tennakoon, R., Mahapatra, D., Roy, P., Sedai, S., Garnavi, R.: Image quality classification for dr screening using convolutional neural networks. In: In Proc. MICCAI-OMIA. pp. 113–120 (2016)
  * [173] Tong, J., Mahapatra, D., Bonnington, P., Drummond, T., Ge, Z.: Registration of histopathology images using self supervised fine grained feature maps. In: In Proc. MICCAI-DART Workshop. pp. 41–51 (2020)
  * [174] Vahadane, A., Peng, T., et al.: Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans. Med. Imag. 35(8), 1962–1971 (2016)
  * [175] Verma, R., Kumar, N., Patil, A., Kurian, N.C., Rane, S., Graham, S., Vu, Q.D., Zwager, M., Raza, S.E.A., Rajpoot, N., Wu, X., Chen, H., Huang, Y., Wang, L., Jung, H., Brown, G.T., Liu, Y., Liu, S., Jahromi, S.A.F., Khani, A.A., Montahaei, E., Baghshah, M.S., Behroozi, H., Semkin, P., Rassadin, A., Dutande, P., Lodaya, R., Baid, U., Baheti, B., Talbar, S., Mahbod, A., Ecker, R., Ellinger, I., Luo, Z., Dong, B., Xu, Z., Yao, Y., Lv, S., Feng, M., Xu, K., Zunair, H., Hamza, A.B., Smiley, S., Yin, T.K., Fang, Q.R., Srivastava, S., Mahapatra, D., Trnavska, L., Zhang, H., Narayanan, P.L., Law, J., Yuan, Y., Tejomay, A., Mitkari, A., Koka, D., Ramachandra, V., Kini, L., Sethi, A.: Monusac2020: A multi-organ nuclei segmentation and classification challenge. IEEE Transactions on Medical Imaging 40(12), 3413–3423 (2021). https://doi.org/10.1109/TMI.2021.3085712
  * [176] Vos, F.M., Tielbeek, J., Naziroglu, R., Li, Z., Sch$\ddot{u}$ffler, P., Mahapatra, D., Wiebel, A., Lavini, C., Buhmann, J., Hege, H., Stoker, J., van Vliet, L.: Computational modeling for assessment of IBD: to be or not to be? In: Proc. IEEE EMBC. pp. 3974–3977 (2012)
  * [177] Wang, X., Peng, Y., et al.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: In Proc. CVPR (2017)
  * [178] Wu, M., Pan, S., Zhou, C., Chang, X., Zhu, X.: Unsupervised domain adaptive graph convolutional networks. In: Proceedings of The Web Conference 2020. pp. 1457–1467 (2020)
  * [179] Xie, S., Zheng, Z., Chen, L., Chen, C.: Learning semantic representations for unsupervised domain adaptation. In: Proc. ICML. vol. 80, pp. 5423–5432 (2018)
  * [180] Xing, Y., Ge, Z., Zeng, R., Mahapatra, D., Seah, J., Law, M., Drummond, T.: Adversarial pulmonary pathology translation for pairwise chest x-ray data augmentation. In: In Proc. MICCAI. pp. 757–765 (2019)
  * [181] Zilly, J., Buhmann, J., Mahapatra, D.: Boosting convolutional filters with entropy sampling for optic cup and disc image segmentation from fundus images. In: In Proc. MLMI. pp. 136–143 (2015)
  * [182] Zilly, J., Buhmann, J., Mahapatra, D.: Glaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation. In Press Computerized Medical Imaging and Graphics 55(1), 28–41 (2017)