11institutetext: Department of Electrical and Computer Engineering, Vanderbilt
University22institutetext: Department of Computer Science, Vanderbilt
University
22email<EMAIL_ADDRESS>
# MAP: Domain Generalization via Meta-Learning on Anatomy-Consistent Pseudo-
Modalities

Dewei Hu 11 Hao Li 11 Han Liu 22 Xing Yao 22 Jiacheng Wang 22 Ipek Oguz 1122

###### Abstract

Deep models suffer from limited generalization capability to unseen domains,
which has severely hindered their clinical applicability. Specifically for the
retinal vessel segmentation task, although the model is supposed to learn the
anatomy of the target, it can be distracted by confounding factors like
intensity and contrast. We propose Meta learning on Anatomy-consistent Pseudo-
modalities (MAP), a method that improves model generalizability by learning
structural features. We first leverage a feature extraction network to
generate three distinct pseudo-modalities that share the vessel structure of
the original image. Next, we use the episodic learning paradigm by selecting
one of the pseudo-modalities as the meta-train dataset, and perform meta-
testing on a continuous augmented image space generated through Dirichlet
mixup of the remaining pseudo-modalities. Further, we introduce two loss
functions that facilitate the model’s focus on shape information by clustering
the latent vectors obtained from images featuring identical vasculature. We
evaluate our model on seven public datasets of various retinal imaging
modalities and we conclude that MAP has substantially better generalizability.
Our code is publically available at https://github.com/DeweiHu/MAP.

###### Keywords:

domain generalization vessel segmentation meta-learning Dirichlet mixup

##  1 Introduction

In the absence of a single standardized imaging paradigm, medical images
obtained from different devices may exhibit considerable domain variation.
Fig. 1 demonstrates three types of domain shift among images delineating the
retinal vessels. The presence of such distribution mismatch can significantly
degrade the performance of deep learning models on unseen datasets, thus
impeding their widespread clinical deployment. To address the domain
generalization (DG) problem [26], a straightforward idea is to focus on the
domain-invariant patterns for the specific downstream task. For retinal vessel
segmentation, the morphology of vessels can be deemed such a domain-invariant
pattern. Hence, our hypothesis is that emphasizing the structural
characteristics of the vasculature can enhance the model’s DG performance.
Following a similar idea, Hu et al. [9] proposed to explicitly delineate the
vessel shape by a Hessian-based vector field. However, the dependency on the
image gradient makes this approach vulnerable to low-quality data with poor
contrast and/or high noise. In contrast, we instead propose an implicit way of
exploiting the morphological features by adopting the meta-learning paradigm
on anatomy-consistent pseudo-modalities (MAP).

First, we leverage a structural feature extraction network (Fig. 2(a))
generate three pseudo-modalities, similar to [9]. The network is defined by
setting the bottleneck of the U-Net [18] backbone to have the same width and
height with the input image. Given its capability to extract interpretable
visualization, this architecture is often implemented in representation
disentanglement [16] and unsupervised segmentation [8]. Supervised by the
binary vessel map, the latent image preserves the vasculature structure while
the style exhibits some randomness, as illustrated in Fig. 2(b). Therefore, we
refer to these latent images as anatomy-consistent pseudo-modalities.

|  |  |  |
---|---|---|---|---
(a)healthy fundus | (b)diseased fundus | (c)OCT-A site-1 | (d)OCT-A site-2 | (e) FA
Figure 1: Domain shift examples. Type I: pathological phenotypes (a vs. b).
Type II: cross-site shifts (c vs. d). Type III: cross-modality shifts (a-b vs.
c-d vs. e).

Meta-learning has recently emerged as a popular technique for addressing the
DG problem [4, 10]. Following the idea of episodic training presented in MAML
[6], researchers split their training data into two subsets, meta-train and
meta-test, to mimic the scenario of encountering out-of-distribution (OOD)
data during training. Liu et al. [13] proposed to conduct meta-learning in a
continuous frequency space created by mixing up [23, 11] the amplitude
spectrum. They keep the phase spectrum unchanged to preserve the anatomy in
the generated images. In contrast, given our pseudo-modalities with identical
underlying vasculature, we are able to create a continuous image space via
Dirichlet mixup [19] without affecting the vasculature. We regard images in
each pseudo-modality as a corner of a tetrahedron, as depicted in Fig. 2(c).
The red facet of the tetrahedron is a continuous space created by the convex
combination of images from the three pseudo-modalities. We use images in one
pseudo-modality (blue node) for meta-train and the mixup space (red facet) for
meta-test. An important property of the mixup space is that all the samples
share the same vessel structure while the image style may differ drastically.
Hence, employing proper constraints on the relationship between features can
implicitly encourage the model to learn the shape of vessels. Inspired by [4],
we leverage a similarity loss to express the feature consistency between the
meta-train and meta-test stages. Additionally, we propose a normalized cross-
correlation (NCC) loss to differentiate latent features extracted from images
with different anatomy. In the context of contrastive learning, these loss
functions cluster positive pairs and separate negative pairs.

In our study, we use seven public datasets including color fundus, OCT
angiography (OCT-A) and fluorescein angiography (FA) images. We train MAP on
fundus data and test on all modalities. We show that MAP exhibits outstanding
generalization ability in most conditions. Our main contributions are:

  * ❖

We generate a continuous space of anatomy-consistent pseudo-modalities with
Dirichlet mixup.

  * ❖

We present an episodic learning scheme employed on synthesized images.

  * ❖

We propose a normalized cross-correlation loss function to cluster the feature
vectors with regard to the vessel structure.

  * ❖

We conduct extensive experiments on seven public datasets in various
modalities which show the superior DG performance of MAP.

Figure 2: The key components of MAP, clockwise. (a) $f(\cdot)$ is the
synthesis network. $\mathbf{x}_{i}$ is the $i^{th}$ color fundus input and
$\mathbf{y}_{i}$ is its ground truth vessel map. $k$ indexes three different
models that generate diverse pseudo-modalities. (b) An example image in four
pseudo-modalities: $\mathcal{D}^{0}$ is the histogram equalization of
intensity-reversed green channel of input $\mathbf{x}$ and $\mathcal{D}^{k}$,
$k=1,2,3$ are generated by $f_{e}^{k}$. (c) The four pseudo-modalities of an
input $\mathbf{x}_{i}$ form the corners of a tetrahedron. The colored facet is
a continuous image space created by Dirichlet mixup. $\mathbf{s}_{i}^{(m)}$
denotes the $m^{th}$ sample from the image space. Anatomy $i$ represents the
underlying shape of vasculature in $\mathbf{x}_{i}$, which is consistent for
all samples $\mathbf{s}_{i}^{(m)}$. (d) The meta-learning scheme. $g(\cdot)$
is the segmentation network, $M$ is the number of samples drawn, $\mathbf{z}$
is the latent feature vector.

##  2 Methods

###  2.1 Problem Definition

Given a source domain
$\mathcal{S}=\\{(\mathbf{x}_{i},\mathbf{y}_{i})|i\in\\{1,\cdots,N\\}\\}$ that
includes $N$ pairs of raw images $\mathbf{x}_{i}$ and ground truth labels
$\mathbf{y}_{i}$, our goal is to train a segmentation network $g(\cdot)$ that
can robustly work on the target domain
$\mathcal{T}=\\{T^{p}|p\in\\{1,\cdots,P\\}\\}$ with $P$ unseen datasets. In
practice, we include only fundus images in $\mathcal{S}$ since there are many
public annotated fundus datasets. For $\mathcal{T}$, data from three different
modalities (fundus, OCT-A and FA) are included. We test the model
generalization on datasets with three distinct types of domain shift: (I) data
with pathological phenotypes, (II) cross-site shifts, (III) cross-modality
shifts.

###  2.2 Pseudo-modality Synthesis

The features in the latent space of a U-Net [18] backbone is usually a low-
dimensional representation of the input images. In some applications (e.g.,
representation disentanglement), it is desirable for the latent features to
show visually intuitive structural characteristics. In such scenarios, the
bottleneck of the feature extraction network is set to have the same width and
height as the input image. We adopt the approach presented in [9] to
synthesize pseudo-modalities by exploiting this idea (Fig. 2(a)). Both the
encoder $f_{e}$ and the decoder $f_{d}$ are residual U-Nets. The input
$\mathbf{x}_{i}\in\mathbb{R}^{3\times H\times W}$ is a color image while
$\mathbf{y}_{i}$ is the binary vessel map. The model is trained by optimizing
a segmentation loss which is the sum of cross-entropy and the Dice loss [14],
i.e., $\mathcal{L}_{seg}=\mathcal{L}_{CE}+\mathcal{L}_{Dice}$. Without direct
supervision, the latent image $\mathbf{x}_{i}^{k}$ can have a different
appearance when the model is re-trained. Such randomness is purely introduced
by the stochastic gradient descent (SGD) in the optimization process.
$k=1,2,3$ indexes three different models and their corresponding synthesized
image. For a fair comparison, we use the pre-trained models provided in [9] to
generate the three pseudo-modalities ($\mathcal{D}^{1}$, $\mathcal{D}^{2}$,
and $\mathcal{D}^{3}$) illustrated in Fig. 2(b).

An essential property of the generated images is that despite significant
intensity variations, they consistently maintain the shared anatomical
structure of the vasculature. Therefore, the $\mathcal{D}^{k}$ are termed
anatomy-consistent pseudo-modalities. To convert the input color fundus image
$\mathbf{x}_{i}$ to grayscale, we conduct histogram equalization (CLAHE) [17]
on the intensity-reversed green channel and denote it as $\mathbf{x}_{i}^{0}$.
The pseudo-modality of these pre-processed images is $\mathcal{D}^{0}$.

###  2.3 Meta-learning on Anatomy Consistent Image Space

Developed from the few-shot learning paradigm, meta-learning seeks to enhance
a model’s generalizability to unseen data when presented with limited training
sets. This is achieved by an episodic training paradigm that consists of two
stages: meta-train and meta-test. The source domain $\mathcal{S}$ is split
into two subsets $\mathcal{S}_{train}$ and $\mathcal{S}_{test}$ to mimic
encountering OOD data during training.

Mixup is a common strategy for data augmentation as it generates new samples
via linear interpolation in either image [11] or feature space [21]. Zhang et
al. [25] showed Mixup improves model generalization and robustness. In [13],
Liu et al. conduct meta-learning on generated images that are synthesized by
mixing the amplitude spectrum in frequency domain. They preserve larger
structures such as the optic disc by keeping the phase spectrum un-mixed.
Given our anatomy-consistent pseudo-modalities, we are able to directly work
on the images rather than the frequency domain. We select $\mathcal{D}^{1}$ as
the meta-train data, and we mixup the remaining three pseudo-modalities
($\mathcal{D}^{0}$,$\mathcal{D}^{2}$, and $\mathcal{D}^{3}$) to form a
continuous space (red facet in Fig. 2(c)) from which we draw meta-test
samples.

In order to mixup three examples, we set a coefficient vector
$\boldsymbol{\lambda}$ follow the Dirichlet distribution, i.e.,
$\boldsymbol{\lambda}\sim\text{Dirichlet}(\boldsymbol{\alpha})$ where
$\boldsymbol{\lambda},\boldsymbol{\alpha}\in\mathbb{R}^{3}$. The probability
density function (PDF) is defined as follows:

$P(\boldsymbol{\lambda})=\frac{\Gamma(\alpha_{0})}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})}\prod_{i=1}^{3}\lambda_{i}^{\alpha_{i}-1}\mathds{1}(\boldsymbol{\lambda}\in
H),$ (1)

with $H=\\{\boldsymbol{\lambda}\in\mathbb{R}^{3}:\lambda_{i}\geq
0,\sum_{i=1}^{3}\lambda_{i}=1\\}$ and $\Gamma(\alpha_{i})=(\alpha_{i}-1)!$.
Examples of PDFs with different hyperparameters $\boldsymbol{\alpha}$ are
shown in the top row of Fig. 3.

The mixup image $\mathbf{s}_{i}$ is created by sampling the coefficient vector
$\boldsymbol{\lambda}$ from $P(\boldsymbol{\lambda})$, i.e.,
$\mathbf{s}_{i}=\lambda_{1}\mathbf{x}_{i}^{0}+\lambda_{2}\mathbf{x}_{i}^{2}+\lambda_{2}\mathbf{x}_{i}^{3}$.
It is evident from the bottom row of Fig. 1 that the samples drawn from
different distributions drastically vary in terms of contrast and vessel
intensity. Thus, the Dirichlet mixup can augment the training data with
varying styles of images without altering the vessel structure. To thoroughly
exploit the continuous image space, we set $\boldsymbol{\alpha}=[1,1,1]$ such
that $P(\boldsymbol{\lambda})$ is a uniform distribution and all samples are
considered equally.

| $\boldsymbol{\alpha}=[5,5,5]$ | $\boldsymbol{\alpha}=[4,2,2]$ | $\boldsymbol{\alpha}=[1.5,5,5]$ | $\boldsymbol{\alpha}=[1.5,5,1.5]$ |
---|---|---|---|---|---
$P(\boldsymbol{\lambda})$ |  |  |  |  |
samples |  |  |  |  |
Figure 3: Examples of Dirichlet distribution and corresponding sample images.

###  2.4 Structural Correlation Constraints

Next, we design constraints to facilitate the model’s concentration on the
vessel morphology. We tackle this by delineating the correlation between
latent features, as illustrated in Fig. 2(d). For two input images
$\mathbf{x}_{i}$ and $\mathbf{x}_{j}$ ($i\neq j$), the features
$\mathbf{z}_{i}$ and $\mathbf{z}_{j}$ are desired to be far apart, as their
anatomies differ. In contrast, the $M$ mixup samples $\mathbf{s}_{i}^{(m)}$
for $m\in\\{1,\cdots,M\\}$ are all anatomy-consistent, thus the corresponding
features $\mathbf{z}_{i}^{(m)}$ should form subject-specific clusters, as
shown in Fig. 4(left). Based on this intuition, we propose two loss functions.

Similarity loss $\mathcal{L}_{sim}$. As mentioned in Sec. 2.3, we set
$\mathcal{S}_{train}=\mathcal{D}^{1}$. The feature vector extracted during
meta-training can be regarded as an anchor in the latent space; we denote it
as $\mathbf{z}^{a}_{i}$. Then the latent features $\mathbf{z}_{i}^{(m)}$ from
samples $\mathbf{s}_{i}^{(m)}$, $m\in\\{1,\cdots,M\\}$, should be close to the
anchor $\mathbf{z}^{a}_{i}$. Here, we simply use the L1 norm as the similarity
loss
$\mathcal{L}_{sim}=\sum_{i=1}^{N}\sum_{m=1}^{M}\|\mathbf{z}_{i}^{(m)}-\mathbf{z}^{a}_{i}\|_{1}$,
where $N$ is the number of input images. $\mathcal{L}_{sim}$ is used to reduce
the distance between sample features and the anchor within the clusters, as
shown in Fig. 4(left).

|
---|---
Figure 4: Left: Feature clusters. Each dot represents a feature vector.
Samples representing different anatomies are shown in different colors. The
highlighted dots are the latent anchor features extracted from
$\mathbf{x}_{i}^{1}$, $\mathbf{x}_{j}^{1}$ and $\mathbf{x}_{k}^{1}$ during
meta-training. Right: NCC matrix. Each entry of the matrix is the cross-
correlation between two feature vectors.

Normalized cross-correlation loss $\mathcal{L}_{ncc}$. In the context of
contrastive learning, the Barlow Twins objective function [22] was proposed to
minimize the redundant information contained in the embedding vectors. This is
realized by computing an empirical cross-correlation matrix of two vectors and
bringing it closer to identity such that unmatched entries are not correlated.
We extend this idea to a stack of vectors, as illustrated in Fig. 4(right).
Feature vectors are color coded in the same way as the left panel of the
figure. The normalized cross-correlations (NCC) between each pair of features
form a symmetric matrix $\mathcal{C}$. As an example, the NCC of
$\mathbf{z}_{i}^{(3)}$ and $\mathbf{z}_{j}^{(2)}$:

$\mathcal{C}_{3,5}=\mathcal{C}_{5,3}=\frac{\mathbf{z}_{i}^{(3)}\cdot\mathbf{z}_{j}^{(2)}}{\sqrt{\mathbf{z}_{i}^{(3)}\cdot\mathbf{z}_{i}^{(3)}}\sqrt{\mathbf{z}_{j}^{(2)}\cdot\mathbf{z}_{j}^{(2)}}}$
(2)

In the ideal ground truth $\mathcal{C}^{*}$, the entries in the black region
are 1, indicating similar features. Conversely, the white region entries are
0, representing dissimilarity. Then the NCC loss can be defined by
$\mathcal{L}_{ncc}=\|\mathcal{C}^{*}-\mathcal{C}\|_{F}^{2}$.

The total loss for the meta-test stage is
$\mathcal{L}_{test}=\omega_{1}\mathcal{L}_{seg}+\omega_{2}\mathcal{L}_{sim}+\omega_{3}\mathcal{L}_{ncc}$.
Empirically, we set $\omega_{1}=\omega_{2}=100$, $\omega_{3}=1$.

###  2.5 Experimental settings

Datasets. We use 7 public datasets listed in Table 1. The source domain
$\mathcal{S}$ includes three color fundus datasets: DRIVE, STARE and healthy
samples in ARIA. By testing on the target domain $\mathcal{T}$, we evaluate
the model’s ability to generalize across pathological, cross-site, and cross-
modality shift conditions.

Implementation Details. The segmentation network $g(\cdot)$ is a 6-layer
residual U-Net. If the number of channels $n$ for a layer is denoted as
$C_{n}$, then the architecture is: $C_{8}-C_{32}-C_{32}-C_{64}-C_{64}-C_{16}$.
The synthesis model $f(\cdot)$ only functions on color fundus images in
$\mathcal{S}$ during training. At test-time, fundus images are converted to
grayscale by applying CLAHE on intensity-reversed green channel, while OCT-A
and FA images are passed to the segmentation network $g(\cdot)$ directly.
$g(\cdot)$ is trained and tested on an NVIDIA RTX 2080TI 11GB GPU. We set the
batch size to 10 and train for 30 epochs. We utilize the Adam optimizer with
the initial learning rate $\eta_{train}=1\times 10^{-3}$ for meta-training and
$\eta_{test}=5\times 10^{-3}$ meta-testing, both decayed by 0.5 for every 3
epochs.

dataset |  modality |  resolution |  number |  domain
---|---|---|---|---
DRIVE [20] |  fundus |  $565\times 584$ |  20 |  $\mathcal{S}$
STARE [7] |  fundus |  $700\times 605$ |  20 |  $\mathcal{S}$
ARIA[5] healthy |  fundus |  $768\times 576$ |  $61$ |  $\mathcal{S}$
AMD |  fundus |  $768\times 576$ |  $59$ |  $\mathcal{T}$
diabetic |  fundus |  $768\times 576$ |  $23$ |  $\mathcal{T}$
PRIME-FP20 [3] |  fundus |  $4000\times 4000$ |  $15$ |  $\mathcal{T}$
ROSE [15] |  OCT-A |  $304\times 304$ |  30 |  $\mathcal{T}$
OCTA-500(6M) [12] |  OCT-A |  $400\times 400$ |  300 |  $\mathcal{T}$
RECOVERY-FA19 [2] |  FA |  $3900\times 3072$ |  8 |  $\mathcal{T}$
Table 1: Datasets. Rows indicating the source domains have a white background while the target domains are shaded according to domain shift type. From top to bottom, (I) pathology: light gray, (II) cross-site: medium gray, (III) cross-modality: dark gray. Episodic |  $\displaystyle\mathcal{L}_{sim}$ |  $\displaystyle\mathcal{L}_{ncc}$ |  Type I |  Type II |  Type III |  Average
---|---|---|---|---|---|---
- |  - |  - |  $\displaystyle 62.93$ |  $\displaystyle 60.04$ |  $\displaystyle 63.94$ |  $\displaystyle 62.95$
- |  - |  ✓ |  $\displaystyle 64.73$ |  $\displaystyle 62.48$ |  $\displaystyle 68.06$ |  $\displaystyle 66.02$
✓ |  - |  - |  $\displaystyle\bf{67.50}$ |  $\displaystyle 63.40$ |  $\displaystyle 64.25$ |  $\displaystyle 65.19$
✓ |  ✓ |  - |  $\displaystyle 64.75$ |  $\displaystyle 66.24$ |  $\displaystyle 68.30$ |  $\displaystyle 66.77$
✓ |  - |  ✓ |  $\displaystyle 66.10$ |  $\displaystyle\underline{66.99}$ |  $\displaystyle\underline{69.71}$ |  $\displaystyle\underline{68.05}$
✓ |  ✓ |  ✓ |  $\displaystyle\underline{67.39}$ |  $\displaystyle\bf{66.99}$ |  $\displaystyle\bf{71.60}$ |  $\displaystyle\bf{69.43}$
Table 2: The ablation study on the main components of MAP on data with three
types of distribution shift. Boldface: best result, underline: second-best
result.

##  3 Results

Method | ARIA |  |  |  |
---|---|---|---|---|---
amd |  diabetic |  PRIME-FP20 |  OCTA 500 |  ROSE |  RECOVERY
baseline |  $\displaystyle 63.82$ |  $\displaystyle 65.19$ |  $\displaystyle 47.31$ |  $\displaystyle 73.16$ |  $\displaystyle 67.41$ |  $\displaystyle 51.25$
Regular[1] |  $\displaystyle 64.89$ |  $\displaystyle 66.97$ |  $\displaystyle 55.76$ |  $\displaystyle 73.54$ |  $\displaystyle 68.36$ |  $\displaystyle 55.20$
BigAug[24] |  $\displaystyle\underline{65.55}$ |  $\displaystyle 67.27$ |  $\displaystyle 59.97$ |  $\displaystyle 76.88$ |  $\displaystyle 69.32$ |  $\displaystyle\bf{63.20}$
MASF[4] |  $\displaystyle 65.33$ |  $\displaystyle\underline{67.75}$ |  $\underline{\scalebox{0.9}{\mbox{$\displaystyle 65.96$}}}$ |  $\displaystyle 77.65$ |  $\displaystyle 67.25$ |  $\displaystyle 50.74$
VFT[9] |  $\displaystyle 61.81$ |  $\displaystyle 64.05$ |  $\displaystyle 54.64$ |  $\displaystyle\underline{77.91}$ |  $\displaystyle\underline{72.81}$ |  $\displaystyle 48.28$
MAP |  $\scalebox{0.9}{\mbox{$\displaystyle\bf{66.69}$}}^{\sim}$ |  $\scalebox{0.9}{\mbox{$\displaystyle\bf{68.08}$}}^{\sim}$ |  $\scalebox{0.9}{\mbox{$\displaystyle\bf{68.21}$}}^{\dagger}$ |  $\scalebox{0.9}{\mbox{$\displaystyle\bf{78.71}$}}^{\dagger}$ |  $\scalebox{0.9}{\mbox{$\displaystyle\bf{74.25}$}}^{\dagger}$ |  $\scalebox{0.9}{\mbox{$\displaystyle\underline{61.85}$}}^{\dagger}$
oracle |  $\displaystyle 73.34$ |  $\displaystyle 70.65$ |  $\displaystyle 77.80$ |  $\displaystyle 86.57$ |  $\displaystyle 76.03$ |  $\displaystyle 74.54$
Table 3: The Dice values (%) for testing on target domains. Boldface: best
result, underline: second best result. ${}^{\sim}:\text{p-value}\geq 0.05$,
${}^{\dagger}:\text{p-value}\ll 0.05$ in paired t-test compared to the
baseline. The background is encoded the same way as Table 1.

Ablation Study. In Table 2, we investigate the contribution of the three major
components of the proposed method: the episodic training paradigm, the
similarity loss $\mathcal{L}_{sim}$ and the normalized cross-correlation loss
$\mathcal{L}_{ncc}$. Note that $\mathcal{L}_{sim}$ requires the access to the
latent anchor and thus is only applicable when using meta-training strategy.
Without $\mathcal{L}_{sim}$ and $\mathcal{L}_{ncc}$, the model is trained with
only the segmentation loss $\mathcal{L}_{seg}$. Our results show that the
introduction of the episodic training provides noticeable improvement in all
types of distribution shift. Both loss functions also contribute positively in
general, and the proposed method ranks the best in types II and III, and
second best in type I.

Comparison to Competing Methods. There are three major classes of approaches
to solve the DG problem: data augmentation, domain alignment, and meta-
learning. We compare against a representative algorithm from each: BigAug
[24], domain regularization network [1], and MASF [4], respectively. We also
compare to VFT [9] as it also focuses on leveraging shape information and
pseudo-modalities. Moreover, we train a residual U-Net on $\mathcal{S}$ as a
baseline model, and a residual U-Net on each target domain
$T^{p}\in\mathcal{T}$ as an oracle model, to provide an indication of the
lower and upper bounds of generalization performance.

Table 3 compares the Dice coefficients (%) of the competing methods. MAP ranks
the best in almost all target domains (except RECOVERY, where it ranks
second), which proves that the proposed MAP algorithm effectively enhances the
robustness of the model under all three domain shift conditions. For some of
the datasets such as ROSE and the diabetic subset of ARIA, the MAP’s
performance approaches the oracle. Compared to the VFT which explicitly models
the tubular vessel shape, the implicit constraints provide a better guidance
for the deep model to learn the structural features.

##  4 Conclusion

We present MAP, a method that approaches the DG problem by implicitly
encouraging the model to learn about the vessel structure, which is considered
to be a domain-agnostic feature. This is achieved by providing the model with
synthesized images that have consistent vasculature but with significant
variations in style. Then by setting constraints with regard to the
correlation between latent features, the model is able to focus more on the
target vessel structure. Our model’s generalization capability is assessed on
test data with different sources of domain shift, including data with
pathological phenotypes, cross-site shifts, and cross-modality shifts. The
results indicate that the proposed method can greatly improve the robustness
of the deep learning models across all three domain shift configurations.

Acknowledgements. This work is supported by the NIH grant R01EY033969 and the
Vanderbilt University Discovery Grant Program.

## References

  * [1] Aslani, S., Murino, V., Dayan, M., Tam, R., Sona, D., Hamarneh, G.: Scanner invariant multiple sclerosis lesion segmentation from mri. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). pp. 781–785. IEEE (2020)
  * [2] Ding, L., Bawany, M.H., Kuriyan, A.E., Ramchandran, R.S., Wykoff, C.C., Sharma, G.: A novel deep learning pipeline for retinal vessel detection in fluorescein angiography. IEEE Transactions on Image Processing 29, 6561–6573 (2020)
  * [3] Ding, L., Kuriyan, A.E., Ramchandran, R.S., Wykoff, C.C., Sharma, G.: Weakly-supervised vessel detection in ultra-widefield fundus photography via iterative multi-modal registration and learning. IEEE Transactions on Medical Imaging 40(10), 2748–2758 (2020)
  * [4] Dou, Q., Coelho de Castro, D., Kamnitsas, K., Glocker, B.: Domain generalization via model-agnostic learning of semantic features. Advances in Neural Information Processing Systems 32 (2019)
  * [5] Farnell, D., Hatfield, F., Knox, P., Reakes, M., Spencer, S., Parry, D., Harding, S.P.: Enhancement of blood vessels in digital fundus photographs via the application of multiscale line operators. Journal of the Franklin institute 345(7), 748–765 (2008)
  * [6] Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning. pp. 1126–1135. PMLR (2017)
  * [7] Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE TMI 19(3), 203–210 (2000)
  * [8] Hu, D., Cui, C., Li, H., Larson, K.E., Tao, Y.K., Oguz, I.: Life: a generalizable autodidactic pipeline for 3d oct-a vessel segmentation. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24. pp. 514–524. Springer (2021)
  * [9] Hu, D., Li, H., Liu, H., Oguz, I.: Domain generalization for retinal vessel segmentation with vector field transformer. In: International Conference on Medical Imaging with Deep Learning. pp. 552–564. PMLR (2022)
  * [10] Khandelwal, P., Yushkevich, P.: Domain generalizer: a few-shot meta learning framework for domain generalization in medical imaging. In: Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning: Second MICCAI Workshop, DART 2020, and First MICCAI Workshop, DCL 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4–8, 2020, Proceedings 2\. pp. 73–84. Springer (2020)
  * [11] Kim, J.H., Choo, W., Song, H.O.: Puzzle mix: Exploiting saliency and local statistics for optimal mixup. In: International Conference on Machine Learning. pp. 5275–5285. PMLR (2020)
  * [12] Li, M., Chen, Y., Ji, Z., Xie, K., Yuan, S., Chen, Q., Li, S.: Image projection network: 3D to 2D image segmentation in OCTA images. IEEE TMI (2020)
  * [13] Liu, Q., Chen, C., Qin, J., Dou, Q., Heng, P.A.: Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1013–1023 (2021)
  * [14] Ma, J., Chen, J., Ng, M., Huang, R., Li, Y., Li, C., Yang, X., Martel, A.L.: Loss odyssey in medical image segmentation. Medical Image Analysis 71, 102035 (2021)
  * [15] Ma, Y., Hao, H., Xie, J., Fu, H., Zhang, J., Yang, J., Wang, Z., Liu, J., Zheng, Y., Zhao, Y.: ROSE: a retinal OCT-angiography vessel segmentation dataset and new model. IEEE TMI 40(3), 928–939 (2020)
  * [16] Ouyang, J., Adeli, E., Pohl, K.M., Zhao, Q., Zaharchuk, G.: Representation disentanglement for multi-modal brain mri analysis. In: Information Processing in Medical Imaging: 27th International Conference, IPMI 2021, Virtual Event, June 28–June 30, 2021, Proceedings 27. pp. 321–333. Springer (2021)
  * [17] Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. Journal of VLSI signal processing systems for signal, image and video technology 38, 35–44 (2004)
  * [18] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. pp. 234–241. Springer (2015)
  * [19] Shu, Y., Cao, Z., Wang, C., Wang, J., Long, M.: Open domain generalization with domain-augmented meta-learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9624–9633 (2021)
  * [20] Staal, J., Abràmoff, M.D., Niemeijer, M., Viergever, M.A., Van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE TMI (2004)
  * [21] Verma, V., Lamb, A., Beckham, C., Najafi, A., Mitliagkas, I., Lopez-Paz, D., Bengio, Y.: Manifold mixup: Better representations by interpolating hidden states. In: International conference on machine learning. pp. 6438–6447. PMLR (2019)
  * [22] Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: Self-supervised learning via redundancy reduction. In: International Conference on Machine Learning. pp. 12310–12320. PMLR (2021)
  * [23] Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
  * [24] Zhang, L., Wang, X., Yang, D., Sanford, T., Harmon, S., Turkbey, B., Wood, B.J., Roth, H., Myronenko, A., Xu, D., et al.: Generalizing deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE transactions on medical imaging 39(7), 2531–2540 (2020)
  * [25] Zhang, L., Deng, Z., Kawaguchi, K., Ghorbani, A., Zou, J.: How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.04819 (2020)
  * [26] Zhou, K., Liu, Z., Qiao, Y., Xiang, T., Loy, C.C.: Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022)