# MammoDG: Generalisable Deep Learning Breaks the Limits of Cross-Domain
Multi-Center Breast Cancer Screening

Yijun Yanga,b∗, Shujun Wanga,e, Lihao Liua, Sarah Hickmanc,d, Fiona J
Gilbertd,
Carola-Bibiane Schönlieba, Angelica I. Aviles-Riveroa

aDAMTP, University of Cambridge, Cambridge, UK
bROAS, The Hong Kong University of Science and Technology (Guangzhou),
Guangzhou, China
cDepartment of Radiology, Barts Health NHS Trust, The Royal London Hospital,
UK
dDepartment of Radiology, Biomedical Research Centre, University of Cambridge,
Cambridge, UK
eThe Department of Biomedical Engineering, The Hong Kong Polytechnic
University, Hong Kong, China

∗Corresponding author: Yijun YANG<EMAIL_ADDRESS>
Work done while interning at the University of Cambridge.

###### Abstract

Breast cancer is a major cause of cancer death among women, emphasising the
importance of early detection for improved treatment outcomes and quality of
life. Mammography, the primary diagnostic imaging test, poses challenges due
to the high variability and patterns in mammograms. Double reading of
mammograms is recommended in many screening programs to improve diagnostic
accuracy but increases radiologists’ workload. Researchers explore Machine
Learning models to support expert decision-making. Stand-alone models have
shown comparable or superior performance to radiologists, but some studies
note decreased sensitivity with multiple datasets, indicating the need for
high generalisation and robustness models. This work devises MammoDG, a novel
deep-learning framework for generalisable and reliable analysis of cross-
domain multi-center mammography data. MammoDG leverages multi-view mammograms
and a novel contrastive mechanism to enhance generalisation capabilities.
Extensive validation demonstrates MammoDG’s superiority, highlighting the
critical importance of domain generalisation for trustworthy mammography
analysis in imaging protocol variations.

##  1 Introduction

Breast cancer is the second leading cause of cancer death in women
worldwide111https://www.cancer.org/cancer/types/breast-cancer/about/how-
common-is-breast-cancer.html. Early cancer detection is relevant for treatment
and improvement of life quality and outcomes. Mammography is the primary
imaging test for diagnosis yet its interpretation is a major challenge (Marmot
et al., 2013; Pharoah et al., 2013). The number of false-positive and false-
negative findings is due to the high variability and patterns in the
mammograms. Therefore, it is often necessary to advocate a double reading of
mammograms, which increases radiologists’ workload, cost, and time (Royal
College of Radiologists, 2019).

Some prior research has been devoted to developing Machine Learning (ML)
models to support expert decision-making and achieve comparable to or superior
performance to radiologists with stand-alone tools (Rodriguez-Ruiz et al.,
2019; McKinney et al., 2020). However, in other studies, the sensitivity
performance is observed to decrease or without change when facing large
cohorts from different sites and dataset characteristics (Schaffter et al.,
2020). The reason is that the large cohort contains the out-of-distribution
(OOD) data collected from different vendor machines and protocols in different
sites leading to a distribution shift of the imaging data.

The body of literature on ML for mammography cancer diagnosis can be broadly
divided into three main categories: 1) single view-based models (W. Zhu et
al., 2017; Wu et al., 2019; Yala et al., 2019); 2) multiple view-based models
(Geras et al., 2017; Khan et al., 2019; Zhao et al., 2020; Wei et al., 2022);
and 3) patch-based techniques (Mercan et al., 2017; Agarwal et al., 2019; Wu
et al., 2019). Moreover, these models can use either a single ML model or
ensembles. However, they do not include any mechanism or are designed to
address the above problem of distribution shift from large cohorts of
mammography data. Whilst the ML community has studied this topic with domain
generalization for other real-world applications (Zhou et al., 2022), the
works on domain generalisation for analysing mammograms are scarce. In recent
work, Z. Li et al. (2021) uses contrastive learning principles to further
augment the generalization capability of the deep learning model considering 4
seen vendors and one unseen vendor. However, that approach still is limited in
terms of extracting more richer and statistical information.

In this work, we address the challenging question of – how to design deep
learning models that can be generalisable, robust, and reliable to multi-
center OOD data. With this purpose in mind, we introduce a novel deep learning
framework based on domain generalisation to mitigate the distribution shift
problem, on mammography screening tasks, namely MammoDG. Our new framework
considers multi-view mammograms. The key of our framework is how we harmonise
richer statistical information from multiple views and enforce fine-grained
detection via a proposed contrastive mechanism. Our contributions are
summarised as follows.

  1. (i)

We propose a novel domain generalisation framework, MammoDG (Figure 1), for
breast-level mammography diagnosis (classification). We highlight an
interpretable multi-view strategy with a Cross-Channel Cross-View Enhancement
module (Figure 2(a)). This module seeks to effectively harmonise the
statistical information from CC and MLO views in the middle feature phase
(Figure 2(b)).

  2. (ii)

We introduce a novel Multi-instance Contrastive Learning mechanism (MICL) to
enhance generalisation and fine-grained detection capabilities of our model.
Our mechanism enforces local and global knowledge to address the out-of-
distribution samples drawn from different vendors and hospitals large-scale
acquisitions (as shown in Figure 2(c)).

  3. (iii)

We extensively validate our new framework using benchmarking and in-home
datasets from different vendor machines and sites, three of which are seen and
two of which are unseen. We demonstrate that our model leads to better
performance than existing deep learning models by a large margin on both seen
and unseen datasets.

  4. (iv)

We have shown that domain generalisation is critical to ensure trustworthiness
and reliable deep learning models for mammography analysis, where limited data
and substantial variations across imaging protocols and vendors machines.

##  2 Methods

In this section, we describe in detail our proposed MammoDG framework for
addressing the out-of-distribution problem in breast cancer screening. Figure
1 depicts our domain generalisation framework for breast-level mammography
classification. We consider a training set of multiple source domains
$\mathcal{S}=\\{S_{1},...,S_{K}\\}$, where each domain $S_{k}$ contains
$N_{k}$ weakly labelled samples
${(a_{i}^{k},b_{i}^{k},y_{i}^{k})}_{i=1}^{N_{k}}$ representing the CC and MLO
views, and breast-level labels respectively. Our framework learns a domain-
agnostic model, $f_{\theta}:X\rightarrow Y$, using $K$ distributed source
domains so that it can generalise to a completely unseen domain $\mathcal{T}$
without performance degradation.

The CC and MLO views are first fed into two-stream view-specific learning
networks to obtain their multi-level feature representations. A Cross-Channel
Cross-View Enhancement (CVE) module is then proposed to learn the data
statistical knowledge. We also introduce a Transformer as global encoder for
better final feature fusion. The view-specific and shared decoder subnetworks
are then adopted to provide image-level and breast-level predictions. To
extract domain-invariant features from data from different vendors, we propose
Multi-Instance Contrastive Learning (MICL), which uses the principles of
Multi-Instance Learning and Contrastive Learning for boosting performance by
detecting abnormal critical instances (patches) across domains.

Figure 1:  Overview of our MammoDG framework. A batch of CC and MLO pair
views, from different domains, are fed into two-stream view-specific learning
networks. Our CVE modules learn their statistical knowledge from the same
pair, at the first three levels, while global encoder further integrates two-
stream feature maps $\hat{\mathcal{F}_{cc}},\hat{\mathcal{F}_{mlo}}$ at the
last level. The share decoder, consisting of two fully connected layers and a
sigmoid layer, generates breast-level predictions. To give a strong
supervision by discovering patch information across domains, MICL plays view-
specific learning and generates image-level predictions. Figure 2: (a) CVE
module. First, task-relevant features are distilled from the input feature
$\mathcal{F}$ to achieve Cross-Channel Enhancement for each view. Secondly, in
Cross-View Enhancement, the geometric-attended vector $\textbf{v}_{mlo}$
computed from the channel-enhanced feature $\tilde{\mathcal{F}}^{+}_{mlo}$ is
multiplied by the self-attention map of the CC view to integrate the
complementary information from the MLO view into the feature of CC view. (b)
The visualisation of our geometric-attended vector v helps understand the
principle of Cross-View Enhancement. The attended value of abnormal tissues in
the $k$-th column in CC view is summarised in $v_{k}$ to provide valuable
geometric information for the corresponding column in MLO view. (c) Multi-
Instance Contrastive Learning strategy. $\hat{\mathcal{F}}$ is a mini-batch of
enhanced feature maps by CVE module obtained from ResNet18 while $\mathcal{F}$
is the same mini-batch of original feature maps from ResNet18.

###  2.1 Cross-Channel Cross-View Enhancement

Previous work in Multi-view Mammography Classification either adopted a
single-stream network to separately process different views (Y. Shen et al.,
2021; Z. Li et al., 2021), or directly concatenate the outputs of the multi-
stream network in the late fusion level (Geras et al., 2017; Wu et al., 2019;
Khan et al., 2019). However, existing works do not consider the statistical
information shared by two views, of the same breast, at the middle feature
level. To this end, we introduce a CVE module to enhance the feature
representation of one view by exploiting complementary knowledge from the
other view. The CVE includes two parts, _i.e._ , cross-channel and cross-view
feature enhancement, as illustrated in Figure 2(a). First, we leverage
Instance Normalisation (IN) to perform style normalisation by normalising
feature statistics from different distributions (domains). While IN has the
power to better the generalisation ability of networks, it inevitably results
in weaker discrimination capability. To recover task-relevant discriminative
feature from the IN removed information, we conduct cross-channel enhancement.
Specifically, we distill the task-relevant feature from the residual feature
$\mathcal{R}$ of the original feature $\mathcal{F}$ and the normalised feature
$\tilde{\mathcal{F}}$, which reads:
$\mathcal{R}=\mathcal{F}-\tilde{\mathcal{F}}.$ We highlight task-relevant part
$\mathcal{R}^{+}$ from $\mathcal{R}$ through a learned channel-wise attention
vector $\textbf{t}=[t_{1},t_{2},...,t_{C}]\in\mathbb{R}^{C}$:

$\displaystyle\mathcal{R}^{+}(:,k)=t_{k}\mathcal{R}(:,k),$ (1) t
$\displaystyle=\sigma(\theta_{2}\delta(\theta_{1}\text{GAP}(\mathcal{R}))),$

where the attention module is implemented by a spatial global average pooling
layer (GAP), followed by two $1\times 1$ convolutional layers (that are
parameterised by $\theta_{1}\in\mathbb{R}^{c\times(c/r)}$ and
$\theta_{2}\in\mathbb{R}^{(c/r)\times c}$), $\sigma(\cdot)$ and
$\delta(\cdot)$ represent sigmoid activation function and ReLU function,
respectively. To reduce the number of parameters, a dimension reduction ratio
$r$ is empirically set to 16. After that, we obtain the channel-enhanced
feature by adding the distilled task-relevant feature $\mathcal{R}^{+}$ to the
normalised feature $\tilde{\mathcal{F}}$ as:

$\displaystyle\tilde{\mathcal{F}}^{+}$
$\displaystyle=\tilde{\mathcal{F}}+\mathcal{R}^{+}.$ (2)

Once we have obtained the channel-enhanced feature representations from
different views, one critical task is to effectively integrate them.
Intuitively, as the CC and MLO views capture the same breast from above and
side, abnormal tissues in the same breast can be observed in both views. To
exploit the correlations between the two views, we propose using the
geometric-attended vector. Specifically, we calculate their feature-level
attention maps by a $3\times 3$ convolutional layer ($\theta_{3}$) with a
sigmoid function as

$\displaystyle w_{cc}=\sigma(\theta_{3}(\tilde{\mathcal{F}}^{+}_{cc})),\
w_{mlo}=\sigma(\theta_{3}(\tilde{\mathcal{F}}^{+}_{mlo})),$ (3)

We then aggregate the complementary information into a learned column-wise
geometric-attended vector
$\textbf{v}=[v_{1},v_{2},...,v_{W}]\in\mathbb{R}^{W}$ to enhance the other
view. We regard the maximum weight of each column, $w$, as the summarised
value in our geometric-attended vector v. For example, as Figure 2(b) shows,
the abnormal tissue in the $k$-th column of CC view should exist in the
corresponding column of the MLO view, and thus the geometric information is
summarised in $v_{k}$ by assigning the bigger attended value. After obtaining
the geometric-attended vector, we multiply it by the attention map of the
other view to differentiate the pixels in the same column. This process reads:

$\displaystyle\hat{w}_{cc}=w_{cc}\cdot\textbf{v}_{mlo},\hat{w}_{mlo}=w_{mlo}\cdot\textbf{v}_{cc}.$
(4)

Finally, we achieve cross-view enhancement by adding the attended feature to
the input feature as

$\displaystyle\hat{\mathcal{F}}_{cc}=\tilde{\mathcal{F}}^{+}_{cc}+\hat{w}_{cc}\cdot\tilde{\mathcal{F}}^{+}_{cc},\
\hat{\mathcal{F}}_{mlo}=\tilde{\mathcal{F}}^{+}_{mlo}+\hat{w}_{mlo}\cdot\tilde{\mathcal{F}}^{+}_{mlo}.$
(5)

The cross-channel cross-view enhanced feature representation
$\hat{\mathcal{F}}$ is propagated to the next layer of each stream network to
capture and integrate multi-level information.

###  2.2 Multi-Instance Contrastive Learning

Regions of interest (ROI) in mammography images, such as masses, asymmetries,
and microcalcifications, are often small and sparsely distributed over the
breast, and may present as subtle changes in the breast tissue pattern. The
Multiple Instance Learning (MIL) technique is a great solution to improve
fine-grained detection when ROI annotations are not available (W. Zhu et al.,
2017). However, due to the absence of global guidance, the instance classifier
is much more likely to be confused by local knowledge in patches from images
of different distributions. It is hard to fully leverage MIL when samples come
from different domains. On the other hand, Z. Li et al. (2021) recently
proposed employing self-supervised Contrastive Learning to attain the goal of
generalization robustness in mammography detection tasks. However, they depend
on CycleGAN (J.-Y. Zhu et al., 2017) to generate multi-style multi-view
images, which may have poor quality due to unexpected changes in tiny tissues
of patches.

To address these limitations, we propose Multi-Instance Contrastive Learning
(MICL) scheme by integrating MIL and Contrastive Learning to more effectively
enhance both the generalization and fine-grained detection capability of the
model. As Figure 1 shows, we treat our MICL module as view-specific decoder
subnetworks to preserve the special knowledge of each view while the shared
information can be learned in the shared decoder. The detailed procedure of
MICL is illustrated in Figure 2(c). Specifically, we adopt a dual-stream MIL
aggregator (B. Li et al., 2021) to jointly learn a patch (instance) and an
image (bag) classifier. Before feeding the cross-channel cross-view enhanced
feature map $\hat{\mathcal{F}}$ to MICL, we divide it into $n\times n$ tiles
along the spatial dimension to generate the bag of $n^{2}$ instances. Let
$B=\\{p_{1},...,p_{n^{2}}\\}$ denotes a bag of instances of one view. The MIL
aggregator first decides the critical instance $p_{m}$ in a bag by using the
instance classifier $f_{m}(\cdot)$ on each instance embedding $p_{i}$ and max-
pooling the scores. This process is given by:

$\displaystyle x=p_{m}$ $\displaystyle=\mathop{argmax}\limits_{p_{i}\in
B}f_{m}(p_{i}),$ (6) $\displaystyle S_{m}(B)$
$\displaystyle=\mathop{max}\limits_{p_{i}\in B}f_{m}(p_{i}).$

Secondly, the MIL aggregator measures the distance between each instance and
the critical instance $p_{m}$, and then produces a bag embedding by summing
the instance embeddings using the distances as weights. More specifically,
each instance embedding $p_{i}$ (including critical instance $p_{m}$) is
transformed into two vectors, query $q_{i}$ and information $v_{i}$, by linear
layers. The distance $d_{i}$ denotes the similarity between queries of the
instance embedding $p_{i}$ and critical instance embedding $p_{m}$, which is
calculated by inner product and softmax. The bag score is further given by the
bag classifier $f_{b}(\cdot)$:

$\displaystyle S_{b}$ $\displaystyle(B)=f_{b}(\sum_{i}^{n^{2}}d_{i}v_{i}).$
(7)

The final score $S(B)$ is the average of the scores of the dual streams:

$\displaystyle S(B)$ $\displaystyle=\frac{1}{2}(S_{m}(B)+S_{b}(B)).$ (8)

As the critical instance represents its bag and plays a significant role in
both streams, it is necessary to guide the network to select the correct
instance in a bag. To this end, we integrate weakly-supervised contrastive
learning into multiple instance learning. First of all, we separate the
critical instances of bags in a mini-batch into the malignant set
$P=\\{x^{+}_{i}\mid y_{i}=1\\}$ and the benign set $Q=\\{x^{-}_{j}\mid
y_{j}=0\\}$ according to breast-level labels. Then for each malignant critical
instance $x^{+}_{i}$ as an anchor, we adopt its out-of-distribution view
$\bar{x}^{+}_{i}$ as the positive sample while all benign critical instances
are negative samples. Instead of standard data augmentation that cannot
perturb the distribution of images and may destroy details in breast tissues,
we apply a feature-level augmentation protocol comprised of Mixstyle (Zhou et
al., 2021) and random noise on the whole feature maps to obtain out-of-
distribution instance embeddings. Mixstyle is inserted between layers in the
CNN architecture to perturbing the distribution information of images from
source domains inspired by Adaptive Instance Normalization. More specifically,
given an input batch of feature maps F and the shuffled batch
$\textbf{F}^{{}^{\prime}}$, Mixstyle computes their feature statistics, _i.e._
, the mean $\gamma(\textbf{F}),\gamma(\textbf{F}^{{}^{\prime}})$ and variance
$\beta(\textbf{F}),\beta(\textbf{F}^{{}^{\prime}})$. Then, we mix their
feature statistics by linear interpolation:

$\displaystyle\gamma_{mix}=m\gamma(\textbf{F})+(1-m)\gamma(\textbf{F}^{{}^{\prime}}),\
\beta_{mix}=m\beta(\textbf{F})+(1-m)\beta(\textbf{F}^{{}^{\prime}}),$ (9)

where $m$ is randomly sampled from the uniform distribution, $m\sim U(0,1.0)$.
Finally, the mixture of feature statistics is applied to the distribution-
normalized F:

$\displaystyle\textbf{F}_{mix}=\beta_{mix}\cdot\frac{\textbf{F}-\gamma(\textbf{F})}{\beta(\textbf{F})}+\gamma_{mix}.$
(10)

Note that we randomly shuffle the order in the batch dimension of F to obtain
$\textbf{F}^{{}^{\prime}}$. Mixstyle only perturbs the distribution
information of images, promising that the correlations among patches from one
image remain invariant. Based on $\textbf{F}_{mix}$, we additionally inject
slight feature noise to alleviate over-fitting.

Similar to InfoNCE contrastive loss (Oord et al., 2018), we applied our
modified contrastive loss on the sampled features to give a stronger and more
stable supervision:

$\displaystyle\mathcal{L}_{cl}=-\frac{1}{\left|P\right|}\sum_{x^{+}_{i}\in
P}\log{\frac{e^{h(x^{+}_{i})\cdot
h(\bar{x}^{+}_{i})/\tau}}{e^{h(x^{+}_{i})\cdot
h(\bar{x}^{+}_{i})/\tau}+\sum_{x^{-}_{j}\in Q}e^{h(x^{+}_{i})\cdot
h(x^{-}_{j})/\tau}}},$ (11)

where $\left|P\right|$ is the cardinality of $P$, $\tau$ is a scalar
temperature hyper-parameter, and $h(\cdot)$ denote global average pooling and
the normalization operation to convert instance embeddings into normalized
feature vectors. Finally, the view-specific objective function of our MICL can
be formulated as:

$\displaystyle\mathcal{L}_{cc}=\mathcal{L}_{bce}(S_{cc}(B^{k}_{i}),y^{k}_{i})+\lambda\mathcal{L}_{cl},\
\mathcal{L}_{mlo}=\mathcal{L}_{bce}(S_{mlo}(B^{k}_{i}),y^{k}_{i})+\lambda\mathcal{L}_{cl},$
(12)

where $L_{bce}(\cdot)$ is binary cross entropy for supervised learning, and
$\lambda$ is a balancing hyper-parameter.

Our MICL scheme has several inherent advantages compared with the original MIL
and self-supervised Contrastive Learning: (1) Hard negative mining: The
selection of negative samples is crucial for learning contrastive features
effectively (Kalantidis et al., 2020). Instead of including all instances in a
bag into contrastive learning, we only consider the critical instance that has
the highest score. This naturally provides our MICL with the ability to mine
hard negative samples since the critical instance is most likely to be the
false positive in a negative bag. (2) Mini-batch training: To improve the
generalisation robustness, we ensure that the mini-batch is composed of all
source domains evenly during training. Our MICL can effectively suppress the
confusion caused by patches from different domains not only because negative
samples come from diverse distributions but also because Mixstyle enforces
positive samples to contain the distribution information of negative samples,
making the model more focus on task-related information.

###  2.3 Global Encoder

After MICL enforces view-specific learning, we aggregate the feature maps
$\hat{\mathcal{F}}$ from CC and MLO branches using Transformer as the final
global encoder to incorporate the global context for two views due to their
complementary nature, as shown in Figure 1. Specifically, we introduce
Transformer (Vaswani et al., 2017) to apply a multi-head self-attention
mechanism, and operate on grid structured feature maps to discover the spatial
dependencies between patches. Let the grid-structured feature map of a single
view be a 3D tensor with dimensions $H\times W\times C$. For CC and MLO views,
their features are stacked together to form a sequence with dimension
$(2\times H\times W)\times C$. We add a learnable positional embedding, which
is a trainable parameter with dimension $(2\times H\times W)\times C$, so
allow the network to infer spatial dependencies between different tokens
during training. The input sequence and positional embedding are combined
using element-wise summation to form a tensor of dimension $(2\times H\times
W)\times C$ as the input of the transformer. The output is then reshaped into
two feature maps of dimension $H\times W\times C$ and fed back into each
branch using an element-wise summation with the existing feature maps.

To save computational cost, we downsample higher resolution feature maps using
average pooling to a fixed resolution of $H=W=16$ before passing them as
inputs to the transformer and upsample the output to the original resolution
using bilinear interpolation before element-wise summation with the existing
feature maps. After Transformer, the feature map is converted into a
512-dimensional feature vector by global average pooling. The feature vectors
from both views are further combined via element-wise summation. This final
512-dimensional feature vector g constitutes a compact representation that
encodes the global context of two views. This is then fed to the shared
decoder subnetwork which consists of two fully connected layers($\theta_{4}$)
to obtain the breast-level prediction. The objective function of the shared
decoder subnetwork is formulated as:

$\displaystyle\mathcal{L}_{sh}=\mathcal{L}_{bce}(\sigma(\theta_{4}(\textbf{g}^{k}_{i})),y^{k}_{i}).$
(13)

Finally, we formulate a unified and end-to-end trainable framework. The
overall loss function can be formulated as follows:

$\displaystyle\mathcal{L}_{total}=\mathcal{L}_{sh}+L_{cc}+\mathcal{L}_{mlo}.$
(14)

###  2.4 Implementation Details

Our proposed framework was trained on an NVIDIA A100 GPU and implemented on
the Pytorch platform. The backbone of our framework was first pre-trained on
BI-RADS labels following (Y. Shen et al., 2021) and then finetuned on our seen
domains. Our framework was empirically trained for 50 epochs in an end-to-end
manner and the Adam optimizer was applied. The initial learning rate was set
to $2\times 10^{-5}$ and decayed by 10% every 5 epochs. During training, we
first resized and randomly crop the mammography images to $512\times 512$, and
then applied the image augmentation protocol including random horizontal
flipping (p=0.5), random rotation ($-15^{\circ}$ to $15^{\circ}$), random
translation (up to 10% of image size), scaling by a random factor between 0.8
and 1.6, random shearing ($-25^{\circ}$ to $25^{\circ}$), and pixel-wise
Gaussian noise ($\mu=0,\sigma=0.005$). A batch of 12 cases evenly composed of
three seen domains (_i.e._ , CBIS, CMMD, TOMMY1) was fed into the network each
time.

##  3 Results

In this section, we present a comprehensive account of all the experiments we
conducted to validate our proposed MammoDG framework.

###  3.1 Data Description

In this study, we use four datasets to evaluate the model generalisation
performance, including three public datasets: CBIS, CMMD, and INBreast, and
one private dataset TOMMY. Due to the large size of the TOMMY dataset, we
split it into two non-overlapping parts TOMMY1 and TOMMY2 based on the patient
level. To assess the generalisation ability of the models, all the datasets
utilized in this study are split into the Seen domain and the Unseen domain.
Seen domain means that the datasets contain both training and testing samples,
_i.e._ , their training samples are seen to the model, while Unseen domain
means that the whole dataset is utilised for only testing. Experimentally, we
regard the CBIS, CMMD, and TOMMY1 as the Seen domain, TOMMY2 and INBreast as
the Unseen domain for the final performance evaluation. For model selection
strategy, we chose the checkpoint with the best performance on Unseen domain
as the final model. Data splits for Seen domain were created at the breast
level, meaning that exams from a given breast were all in the same split.

#### CBIS-DDSM dataset

CBIS-DDSM (Lee et al., 2017) is a public database of scanned film mammography
studies containing cases categorized as normal, benign, and malignant with
verified pathology information. It is a collection of mammograms from
Massachusetts General Hospital, Wake Forest University School of Medicine,
Sacred Heart Hospital, and Washington University of St Louis School of
Medicine. Mammography image data from CBIS-DDSM is an updated version of the
DDSM providing easily accessible data. We followed the official splits but
discarded the cases that did not have both CC and MLO views, resulting in 572
benign, 475 malignant for training and 153 benign, 102 malignant for testing.
We did not use any data from DDSM for testing given that it is a scanned film
dataset.

#### CMMD dataset

CMMD (Cui et al., 2021) is a large public mammography database collected from
patients from China, categorized as benign and malignant with verified
pathology information. Mammography image data were acquired on a GE Senographe
DS mammography system. We split the breast-level cases with complete views
into 80%/20% training/model selection splits, resulting in 423 benign, 1,021
malignant studies for training and 115 benign, 246 malignant studies for
testing.

#### INBreast dataset

INBreast (Moreira et al., 2012) is a small public mammography database with a
relatively balanced benign and malignant case. We split data from patient-
level into breast-level and excluded cases with incomplete views, resulting in
125 benign and 46 malignant out of 171 studies.

#### TOMMY dataset

TOMMY (Gilbert et al., 2015) is a rich and well-labeled dataset with over
7,000 patients (over 1,000 malignant) collected through six NHS Breast
Screening Program (NHSBSP) centers throughout the UK and read by expert
radiologists. To keep the number of breast-level cases consistent with other
datasets, we just sampled a part of TOMMY for experiments. TOMMY1 as Seen
domain has 1,560 benign cases and 364 malignant cases for training, 406 benign
and 76 malignant cases for testing. TOMMY2 with 2,108 benign and 394 malignant
cases was all treated as Unseen domain. The TOMMY1 and TOMMY2 datasets were
obtained from Hologic vendor machines.

Table 1: Quanlitative results of our method compared to many state-of-the-art
methods on our setting. All the models are trained on the training sets of
seen domains and evaluated on the test set of seen domains and the whole set
of unseen domains. The best performance is highlighted in red colour while the
second best results are in blue colour.

Datasets | Metric | BIRADS | DMV-CNN | MVFF | GMIC |  MSVCL (ResNet) |  MSVCL (FCOS) | Baseline | Ours
---|---|---|---|---|---|---|---|---|---
Seen Domain | CBIS | AUC | 0.6660 | 0.6654 | 0.7344 | 0.7666 | 0.6874 | 0.7045 | 0.7544 | 0.7798
TPR | 0.6078 | 0.6078 | 0.6536 | 0.6993 | 0.6209 | 0.6013 | 0.6732 | 0.6932
TNR | 0.6176 | 0.6176 | 0.6569 | 0.7034 | 0.6275 | 0.6176 | 0.6765 | 0.6863
ACC | 0.6118 | 0.6118 | 0.6549 | 0.7020 | 0.6235 | 0.6078 | 0.6745 | 0.6884
CMMD | AUC | 0.6661 | 0.6818 | 0.7686 | 0.8157 | 0.7878 | 0.8070 | 0.8018 | 0.8181
TPR | 0.6087 | 0.6435 | 0.6957 | 0.7304 | 0.7130 | 0.7217 | 0.7291 | 0.7391
TNR | 0.6098 | 0.6382 | 0.6870 | 0.7398 | 0.7195 | 0.7236 | 0.7217 | 0.7439
ACC | 0.6094 | 0.6399 | 0.6898 | 0.7368 | 0.7175 | 0.7230 | 0.7241 | 0.7424
TOMMY1 | AUC | 0.6624 | 0.6977 | 0.7178 | 0.7146 | 0.7039 | 0.7535 | 0.6665 | 0.7235
TPR | 0.5936 | 0.6108 | 0.6576 | 0.6404 | 0.6601 | 0.7069 | 0.6010 | 0.6724
TNR | 0.6053 | 0.6184 | 0.6579 | 0.6579 | 0.6579 | 0.7105 | 0.6184 | 0.6711
ACC | 0.5954 | 0.6120 | 0.6577 | 0.6432 | 0.6598 | 0.7075 | 0.6037 | 0.6722
Average | AUC | 0.6648 | 0.6816 | 0.7403 | 0.7656 | 0.7264 | 0.7550 | 0.7409 | 0.7738
TPR | 0.6034 | 0.6207 | 0.6690 | 0.6900 | 0.6647 | 0.6766 | 0.6678 | 0.7016
TNR | 0.6109 | 0.6247 | 0.6673 | 0.7004 | 0.6683 | 0.6839 | 0.6722 | 0.7004
ACC | 0.6055 | 0.6212 | 0.6675 | 0.6940 | 0.6669 | 0.6794 | 0.6674 | 0.7010
Overall | AUC | 0.8005 | 0.8062 | 0.8264 | 0.8445 | 0.8258 | 0.8394 | 0.8225 | 0.8491
TPR | 0.7300 | 0.7285 | 0.7374 | 0.7567 | 0.7270 | 0.7329 | 0.7270 | 0.7596
TNR | 0.7311 | 0.7288 | 0.7382 | 0.7618 | 0.7288 | 0.7358 | 0.7288 | 0.7618
ACC | 0.7304 | 0.7286 | 0.7377 | 0.7587 | 0.7277 | 0.7341 | 0.7277 | 0.7605
Unseen Domain | TOMMY2 | AUC | 0.6298 | 0.6466 | 0.6760 | 0.6798 | 0.6714 | 0.6919 | 0.6994 | 0.7288
TPR | 0.5954 | 0.6029 | 0.6248 | 0.6314 | 0.6157 | 0.6271 | 0.6461 | 0.6769
TNR | 0.5939 | 0.6041 | 0.6269 | 0.6345 | 0.6168 | 0.6294 | 0.6447 | 0.6777
ACC | 0.5951 | 0.6031 | 0.6251 | 0.6319 | 0.6159 | 0.6275 | 0.6459 | 0.6771
INBreast | AUC | 0.4692 | 0.5195 | 0.6522 | 0.6791 | 0.7097 | 0.7649 | 0.6623 | 0.7889
TPR | 0.4080 | 0.5200 | 0.5520 | 0.7120 | 0.6080 | 0.7040 | 0.5760 | 0.7520
TNR | 0.4348 | 0.5217 | 0.5870 | 0.6304 | 0.6304 | 0.6957 | 0.5870 | 0.6957
ACC | 0.4152 | 0.5205 | 0.5614 | 0.6901 | 0.6140 | 0.6998 | 0.5789 | 0.7368
Average | AUC | 0.5495 | 0.5831 | 0.6641 | 0.6795 | 0.6906 | 0.7284 | 0.6809 | 0.7589
TPR | 0.5017 | 0.5615 | 0.5884 | 0.6717 | 0.6119 | 0.6656 | 0.6111 | 0.7145
TNR | 0.5144 | 0.5629 | 0.6070 | 0.6325 | 0.6236 | 0.6626 | 0.6159 | 0.6867
ACC | 0.5052 | 0.5618 | 0.5933 | 0.661 | 0.6150 | 0.6637 | 0.6124 | 0.7070
Overall | AUC | 0.6343 | 0.6494 | 0.6784 | 0.6792 | 0.6750 | 0.6955 | 0.6979 | 0.7341
TPR | 0.5979 | 0.6082 | 0.6229 | 0.6341 | 0.6238 | 0.6301 | 0.6413 | 0.6767
TNR | 0.6000 | 0.6091 | 0.6250 | 0.6341 | 0.6250 | 0.6295 | 0.6432 | 0.6773
ACC | 0.5982 | 0.6083 | 0.6233 | 0.6341 | 0.6240 | 0.6300 | 0.6416 | 0.6768
Average | AUC | 0.6187 | 0.6422 | 0.7098 | 0.7312 | 0.7120 | 0.7444 | 0.7169 | 0.7678
TPR | 0.5627 | 0.5970 | 0.6367 | 0.6827 | 0.6435 | 0.6722 | 0.6451 | 0.7067
TNR | 0.5723 | 0.6000 | 0.6431 | 0.6732 | 0.6504 | 0.6754 | 0.6497 | 0.6949
ACC | 0.5654 | 0.5975 | 0.6378 | 0.6808 | 0.6461 | 0.6731 | 0.6454 | 0.7034
Overall | AUC | 0.7386 | 0.7476 | 0.7646 | 0.7702 | 0.7634 | 0.7806 | 0.7654 | 0.8013
TPR | 0.6735 | 0.6735 | 0.6859 | 0.7049 | 0.6935 | 0.6959 | 0.6945 | 0.7258
TNR | 0.6736 | 0.6736 | 0.6863 | 0.7049 | 0.6956 | 0.6968 | 0.6956 | 0.7269
ACC | 0.6736 | 0.6736 | 0.686 | 0.7049 | 0.6940 | 0.6961 | 0.6948 | 0.7261

#### Vendor-Specific Mammography Scanner Information.

In our study, we utilised four distinct mammography datasets collected from
different scanners to examine the impact of scanner variability on mammography
analysis. The CBIS-DDSM dataset was digitalized with four different scanners:
DBA scanner at MGH, HOWTEK scanner at MGH, LUMISYS scanner at Wake Forest
University, and HOWTEK scanner at ISMD. Additional information about this
dataset can be found here
222http://www.eng.usf.edu/cvprg/mammography/database.html. The CMMD dataset
was acquired on a GE Senographe DS mammography system. The InBreast dataset
was captured with MammoNovation Siemens FFDM equipment at the Breast Centre in
CHSJ, Porto (Moreira et al., 2012). Lastly, the TOMMY dataset was collected by
a commercially available (Hologic) digital mammography system (Gilbert et al.,
2011). By analysing and processing these diverse datasets, we aim to
investigate the generalisation ability of the proposed MammoDG framework under
the influence of scanner characteristics on mammography and explore potential
implications for clinical applications.

###  3.2 Performance Evaluation

To quantitatively evaluate the performance of our method, we adopt four
popular classification metrics for all experiments, _i.e._ , the area under
receiver operator characteristic curve (AUC), true positive recall (TPR), true
negative recall (TNR) and accuracy (ACC). All models are trained on the
training sets of seen domains and evaluated on the test set of each domain,
respectively. It is worth noting that, to obtain average performance, we
simply average the metric values of all the target domains, _i.e._ , different
thresholds are adopted across domains. For overall performance, we aggregate
the test set of all target domains and then evaluate the model on the mixed
test set, _i.e._ , the same threshold is adopted for the classification of all
domains.

Figure 3: The visualisation of heatmap after cross-channel cross-view
enhancement. Three malignant cases, one from each dataset, are tested. While
the red and blue regions denote abnormal and normal tissues with high
confidence. It is important to note that the yellow regions represent abnormal
tissues with low confidence, and therefore should be verified through
additional scrutiny.

###  3.3 Comparison with state-of-the-art methods

We compare our network against state-of-the-art mammography classification
methods, including BIRADS (Geras et al., 2017), DMV-CNN (Wu et al., 2019),
MVFF (Khan et al., 2019) and GMIC (Y. Shen et al., 2021). To further
demonstrate the generalisation ability of our model, we also reimplement a
generalizable mammography detection framework MSVCL (Z. Li et al., 2021) in
two ways. MSVCL(ResNet), MSVCL(FCOS) both utilize ResNet as the backbone but
the latter additionally incorporates feature pyramid network to leverage
multi-level features as FCOS (Tian et al., 2019) does. BIRADS, DMV-CNN and
MVFF are designed in the multi-view fashion while GMIC and MSVCL are the
single-view frameworks. For a fair comparison, we obtain breast-level
predictions by averaging their image-level predictions for those single-view
frameworks. As displayed in Table 1, our framework produces superior
performance on both Seen domain and Unseen domain. For Seen domain, our
MammoDG surpasses the second-best method GMIC by 0.0082 in AUC and 0.0070 in
ACC of the average performance, 0.0045 in AUC and 0.0018 in ACC of the overall
performance, respectively. For Unseen domain, our method improves the average
and overall performance by a considerable margin of 0.0305 in AUC and 0.0433
in ACC, 0.0386 in AUC and 0.0468 in ACC, respectively, than the generalisable
method MSVCL(FCOS). The consistent improvements on all datasets result in
superb advances in all domains in view of all four metrics of the average and
overall performance.

###  3.4 Ablation Studies

Table 2: Quantitatively ablation studies on our domain generalisation setting.
The public datasets CBIS and CMMD are treated as Seen domain while INBreast is
treated as Unseen domain. The module “CVE” means Cross-Channel Cross-View
Enhancement while the module “MS” denotes Mixstyle. “GE” denotes Global
Encoder while “MICL” represents Multi-Instance Contrastive Learning. The top
values are bold.

Method | Metrics | Baseline | +CVE | +MS | +GE | +MICL
---|---|---|---|---|---|---
Seen | CBIS | AUC | 0.6928 | 0.7066 | 0.7590 | 0.6960 | 0.7602
TPR | 0.6144 | 0.6275 | 0.6863 | 0.6340 | 0.6928
TNR | 0.6275 | 0.6471 | 0.6961 | 0.6569 | 0.6863
ACC | 0.6196 | 0.6353 | 0.6902 | 0.6431 | 0.6902
CMMD | AUC | 0.7881 | 0.7926 | 0.7840 | 0.8567 | 0.8309
TPR | 0.7043 | 0.7217 | 0.7043 | 0.7652 | 0.7391
TNR | 0.7033 | 0.7317 | 0.7114 | 0.7805 | 0.7440
ACC | 0.7036 | 0.7285 | 0.7092 | 0.7756 | 0.7424
Average | AUC | 0.7405 | 0.7496 | 0.7715 | 0.7764 | 0.7956
TPR | 0.6594 | 0.6746 | 0.6953 | 0.6996 | 0.7160
TNR | 0.6654 | 0.6894 | 0.7038 | 0.7187 | 0.7152
ACC | 0.6616 | 0.6819 | 0.6997 | 0.7094 | 0.7163
Overall | AUC | 0.7777 | 0.7869 | 0.7983 | 0.8037 | 0.8201
TPR | 0.7052 | 0.7239 | 0.7089 | 0.7164 | 0.7463
TNR | 0.7069 | 0.7213 | 0.7126 | 0.7184 | 0.7500
ACC | 0.7062 | 0.7224 | 0.7110 | 0.7175 | 0.7484
Unseen | INBreast | AUC | 0.6048 | 0.7780 | 0.8193 | 0.8064 | 0.8289
TPR | 0.5600 | 0.6960 | 0.7280 | 0.7200 | 0.7920
TNR | 0.5870 | 0.7391 | 0.7391 | 0.7391 | 0.8043
ACC | 0.5673 | 0.7076 | 0.7310 | 0.7251 | 0.7953
Average | AUC | 0.6952 | 0.7591 | 0.7874 | 0.7864 | 0.8067
TPR | 0.6262 | 0.6817 | 0.7062 | 0.7064 | 0.7413
TNR | 0.6393 | 0.7060 | 0.7155 | 0.7255 | 0.7449
ACC | 0.6302 | 0.6905 | 0.7101 | 0.7146 | 0.7426
Overall | AUC | 0.7781 | 0.7993 | 0.8207 | 0.8213 | 0.8364
TPR | 0.6997 | 0.7201 | 0.7354 | 0.7455 | 0.7659
TNR | 0.7081 | 0.7234 | 0.7386 | 0.7487 | 0.7691
ACC | 0.7039 | 0.7217 | 0.7370 | 0.7471 | 0.7675

In this part, we conduct extensive ablation studies on the publicly available
datasets, _i.e._ , CBIS, CMMD as Seen domain, and INBreast as Unseen domain.

#### The Effectiveness of Each Module.

As shown in Table 2, we validate the effectiveness of each module in our
framework. The Baseline model consists of two Resnet18 branches where CC and
MLO views are encoded respectively and then fused by concatenating two feature
embeddings for the final breast-level prediction. The Cross-Channel Cross-View
Enhancement module (+CVE) dramatically advances the overall performance over
the Baseline model by 0.0212 in AUC and 0.0178 in ACC. The Mixstyle
augmentation strategy (+MS) is further incorporated at each stage to mitigate
the domain shift problem and achieve significant improvement, particularly on
unseen domains. While the Global Encoder (+GE) explores the shared
representation of two views, the Multi-Instance Contrastive Learning strategy
(+MICL) conducts view-specific learning and endows the full MammoDG model with
the overall performance of 0.8364, 0.7659, 0.7691, 0.7675 in AUC, TPR, TNR,
ACC.

#### Details of CVE Module.

We quantitatively explore the efficacy of each component in our CVE module in
Table 3(a) and conduct the experiments based on the full MammoDG. The model
without the entire CVE module achieves an overall performance of 0.8014,
0.7141 in AUC, ACC, respectively. Cross-Channel Enhancement (CE) brings 0.0300
AUC gains and 0.0280 ACC gains in overall performance while Cross-View
Enhancement (VE) further improves by 0.0050 and 0.0254, respectively. To
qualitatively verify the effectiveness of our CVE module, we visualize the
heatmap of three samples after manipulating cross-channel cross-view
enhancement. Figure 3 clearly demonstrates that our method successfully
detects malignant tissues after these enhancements.

#### Discussion on view-specific learning.

In Table 3(c), we conduct experiments on different strategies of view-specific
learning. We first replace our MICL with a vanilla classifier head to give
supervision on image-level classification, which degrades the overall
performance by 0.0128 and 0.0227 in AUC and ACC. Additionally, we replace our
MICL with a MIL aggregator B. Li et al. (2021), leading to a significant drop
of 0.0117 and 0.0166 in overall AUC and ACC.

#### Discussion on the balancing hyper-parameters.

We discuss the best choice of the balancing ratio of breast-level prediction
and image-level predictions in Table 3(b). The equal weight (1:1:1) for CC,
MLO and breast-level predictions achieves the best performance. We also
discuss the balancing hyper-parameter $\lambda$ to weight supervision loss and
contrastive loss in MICL in Table 3(d). $\lambda$ should be set as 0.5 to
obtain the best overall AUC and ACC.

#### Details of the MICL strategy.

In Table 3(e), we explore the effects of the number of tiles (instances) in
one bag on our MICL strategy. The experiment results show that when one image
is divided into $4\times 4$ tiles, the best overall performance is achieved.

| w/o CVE | CE |  CVE (ours)
---|---|---|---
Seen | 0.8070 | 0.8035 | 0.8201
0.7276 | 0.7164 | 0.7463
0.7328 | 0.7241 | 0.7500
0.7305 | 0.7208 | 0.7484
Unseen | 0.7012 | 0.8355 | 0.8289
0.6080 | 0.7360 | 0.7920
0.6304 | 0.7826 | 0.8043
0.6140 | 0.7485 | 0.7953
Overall | 0.8014 | 0.8314 | 0.8364
0.7125 | 0.7430 | 0.7659
0.7157 | 0.7411 | 0.7691
0.7141 | 0.7421 | 0.7675
(a) Each component in CVE module. CE brings 0.0300 AUC gains and 0.0280 ACC
gains in overall performance while VE further improves by 0.0050 and 0.0254,
respectively.

| w/o ens | 1:1:2 |  1:1:1 (ours)
---|---|---|---
Seen | 0.8230 | 0.8245 | 0.8201
0.7388 | 0.7425 | 0.7463
0.7471 | 0.7443 | 0.7500
0.7435 | 0.7435 | 0.7484
Unseen | 0.8219 | 0.8221 | 0.8289
0.7520 | 0.7520 | 0.7920
0.8043 | 0.8043 | 0.8043
0.7661 | 0.7661 | 0.7953
Overall | 0.8306 | 0.8317 | 0.8364
0.7532 | 0.7608 | 0.7659
0.7563 | 0.7614 | 0.7691
0.7548 | 0.7612 | 0.7675
(b) The balancing ratio of breast-level and image-level predictions.  The
equal weight for CC, MLO and breast-level predictions promise the best
performance.

| vanilla | MIL |  MICL (ours)
---|---|---|---
Seen | 0.8040 | 0.8098 | 0.8201
0.7276 | 0.7351 | 0.7463
0.7328 | 0.7385 | 0.7500
0.7305 | 0.7370 | 0.7484
Unseen | 0.8217 | 0.8266 | 0.8289
0.7440 | 0.7600 | 0.7920
0.7609 | 0.7826 | 0.8043
0.7485 | 0.7661 | 0.7953
Overall | 0.8236 | 0.8247 | 0.8364
0.7431 | 0.7481 | 0.7659
0.7463 | 0.7538 | 0.7691
0.7448 | 0.7509 | 0.7675
(c) Vanilla classifier vs. MICL as view-specific decoders. MICL improves the
AUC and ACC in overall performance by 0.0128 and 0.0227.

$\lambda$ | 0.2 |  0.5 (ours) | 1.0
---|---|---|---
Seen | 0.8137 | 0.8201 | 0.8146
0.7201 | 0.7463 | 0.7276
0.7270 | 0.7500 | 0.7299
0.7240 | 0.7484 | 0.7289
Unseen | 0.8369 | 0.8289 | 0.7821
0.7760 | 0.7920 | 0.5840
0.7826 | 0.8043 | 0.7174
0.7778 | 0.7953 | 0.6199
Overall | 0.8345 | 0.8364 | 0.8313
0.7710 | 0.7659 | 0.7481
0.7640 | 0.7691 | 0.7538
0.7675 | 0.7675 | 0.7509
(d) The balancing hyper-parameter $\lambda$.  The best value to weight cross
entropy loss and contrastive loss in MICL is 0.5 with best AUC and ACC.

$n$ | 3 |  4 (ours) | 5 | 6
---|---|---|---|---
Seen | 0.8178 | 0.8201 | 0.8254 | 0.8093
0.7463 | 0.7463 | 0.7500 | 0.7276
0.7443 | 0.7500 | 0.7500 | 0.7328
0.7453 | 0.7484 | 0.7500 | 0.7305
Unseen | 0.8241 | 0.8289 | 0.8231 | 0.8270
0.7520 | 0.7920 | 0.8000 | 0.7600
0.7826 | 0.8043 | 0.7826 | 0.7826
0.7673 | 0.7953 | 0.7953 | 0.7661
Overall | 0.8310 | 0.8364 | 0.8325 | 0.8348
0.7543 | 0.7659 | 0.7583 | 0.7659
0.7614 | 0.7691 | 0.7614 | 0.7691
0.7579 | 0.7675 | 0.7598 | 0.7675
(e) The number of tiles $n$ in MICL.  The overall performance achieves the
best when $n=4$.

Table 3: Quantitatively ablation studies on details of each parts in our
MammoDG. All models are trained on the train set of CBIS and CMMD and test on
INBreast and the test set of CBIS and CMMD. (Four metrics from up to bottom
are AUC, TPR, TNR, ACC.)

##  4 Discussion

#### MammoDG outperforms traditional supervised methods on mammography
diagnosis.

In our comparison with traditional supervised methods (”Seen” Category in
Table 1) for mammography diagnosis, MammoDG demonstrated superior performance
across all metrics. This is largely attributed to the effective use of multi-
view mammograms framework and the innovative contrastive mechanism that
enhances generalisation capabilities. Traditional models often struggle with
the high variability and complex patterns found in mammograms, while MammoDG
was designed to robustly manage this inherent complexity. In terms of AUC,
TPR, TNR, and ACC, our method consistently outperformed traditional supervised
methods, highlighting the benefit of leveraging advanced domain generalisation
mechanisms for this task.

#### MammoDG consistently surpasses the generalisable mammography diagnosis
methods on unseen domains.

Another distinguishing feature of MammoDG is its ability to maintain superior
performance even when tested on unseen domains. This was a limitation observed
in previous studies with other generalisable mammography diagnosis methods.
According to the ”Unseen” part of Table 1 & 2, MammoDG’s robustness to out-of-
distribution data, collected from various vending machines and protocols,
allows it to handle the data distribution shift in large cohorts effectively.
This shows the feasibility of the deployment of MammoDG in real-world
scenarios across various centres and hospitals.

#### MammoDG saves the cost of annotation in target domains.

MammoDG’s ability to achieve high performance with limited annotations is
crucial to the medical image analysis community. Given the difficulty and
expense of acquiring reliable annotations, a model that can still excel with
such limitations is invaluable. As compared to traditional supervised models
that require extensive and costly annotations for training, MammoDG
substantially cuts the cost of annotation in target domains, which makes it an
efficient and cost-effective solution for large-scale mammography analysis
across multiple centers.

#### MammoDG provides reliable evidence for clinical decisions.

The results from this study have significant practical implications for the
healthcare industry, specifically for radiologists and healthcare providers
engaged in breast cancer detection. The machine learning model developed in
this research demonstrated robust performance across various datasets, with
promising implications for real-world application. In the domain of breast
cancer diagnosis, MammoDG is an especially powerful tool as it considers both
CC and MLO views, providing a comprehensive analysis that leverages cross-view
complementary information. As depicted in Figure 3, MammoDG consistently
generates reliable attention regions, providing evidence that matches well
with radiologists’ diagnoses. The intersection over union between our model’s
attention regions and the areas highlighted by radiologists consistently
exceeded a threshold, indicating MammoDG’s capability to provide trustworthy
and actionable insights for clinical decisions.

In the future, we aim to conduct reader studies to measure the extent to which
accuracy improves when radiologists use our system and to evaluate their level
of trust in it. Given the potential benefits of AI assistance, particularly
for less-experienced readers, further investigation will be valuable in
comparing the benefits of this system for both sub-specialists and community
radiologists who might be called on to do this work only occasionally.

#### MammoDG’s Limitations

Despite its strengths, this study also has several limitations. First,
although the model was evaluated on several diverse datasets, these are
primarily from China and the UK. Additional validation on datasets from other
regions and ethnicities would be valuable in assessing the global
applicability of our model. Second, the results of this study are contingent
on the accuracy of the ground truth labels, which are based on human
interpretation and thus subject to inter-observer variability. Lastly, while
the model demonstrated strong performance in distinguishing between benign and
malignant cases, there remains a need to further investigate its efficacy in
detecting early stage cancers, as this is crucial for improving patient
outcomes. Future work should aim to address these limitations, refine the
model’s capabilities, and assess its performance in a real-world clinical
setting.

##  5 Conclusion

Our work exhibits our ability to develop a pioneering deep-learning framework
for generalisable, robust, and reliable analysis of cross-domain multi-center
mammography data. Our framework MammoDG outperforms traditional models when
trained on limited data. We are able to provide a generalisable network that
performs comparably to radiologists on breast cancer analysis without
requiring specific training when transferring to new clinical sites. Extensive
experiments further validate the critical importance of domain generalisation
for trustworthy mammography analysis in the presence of imaging protocol
variations.

##  6 Check List Information

## Data availability

This study involved four datasets. Three of them are published data and the
remaining one is the private dataset. The CBIS dataset is Breast Cancer Image
Dataset from Kaggle (https://www.kaggle.com/datasets/awsaf49/cbis-ddsm-breast-
cancer-image-dataset), and CMMD is The Chinese Mammography Database from
https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=70230508,
and InBreast dataset from Kaggle
(https://www.kaggle.com/datasets/martholi/inbreast). The TOMMY dataset
(Gilbert et al., 2015) is not currently permitted for public release by their
respective Institutional Review Boards.

## Code availability

The code for this project, including all libraries used and their versions, is
available online at https://github.com/needupdate.

## Acknowledgements

LL gratefully acknowledges the financial support from a GSK scholarship and a
Girton College Graduate Research Fellowship at the University of Cambridge.
FJG acknowledges support by the NIHR Cambridge Biomedical Research Centre and
an early detection programme grant from Cancer Research UK. AIAR acknowledges
support from CMIH and CCIMI, University of Cambridge, ESPRC Digital Core
Capability Award. CBS acknowledges the Philip Leverhulme Prize, the EPSRC
fellowship EP/V029428/1, EPSRC grants EP/T003553/1, EP/N014588/1, Wellcome
Trust 215733/Z/19/Z and 221633/Z/20/Z, Horizon 2020 No. 777826 NoMADS and the
CCIMI.

## References

  *   * Agarwal et al. (2019) Agarwal, R., Diaz, O., Lladó, X., Yap, M. H., and Martí, R.  (2019).  Automatic mass detection in mammograms using deep convolutional neural networks.  _Journal of Medical Imaging_ , _6_(3), 031409.
  * Cui et al. (2021) Cui, C., Li, L., Cai, H., Fan, Z., Zhang, L., Dan, T., … Wang, J.  (2021).  The chinese mammography database (cmmd): An online mammography database with biopsy confirmed types for machine diagnosis of breast.  _The Cancer Imaging Archive_ , _1_.
  * Geras et al. (2017) Geras, K. J., Wolfson, S., Shen, Y., Wu, N., Kim, S., Kim, E., … Cho, K.  (2017).  High-resolution breast cancer screening with multi-view deep convolutional neural networks.  _arXiv preprint arXiv:1703.07047_.
  * Gilbert et al. (2011) Gilbert, F., Gillan, M., Michell, M., Young, K., Dobson, H., Cooke, J., … Duffy, S.  (2011).  Tommy trial (a comparison of tomosynthesis with digital mammography in the uk nhs breast screening programme) setting up a multicentre imaging trial.  _Breast Cancer Research_ , _13_ , 1–13.
  * Gilbert et al. (2015) Gilbert, F., Tucker, L., Gillan, M. G., Willsher, P., Cooke, J., Duncan, K. A., … others  (2015).  The tommy trial: a comparison of tomosynthesis with digital mammography in the uk nhs breast screening programme–a multicentre retrospective reading study comparing the diagnostic performance of digital breast tomosynthesis and digital mammography with digital mammography alone.
  * Kalantidis et al. (2020) Kalantidis, Y., Sariyildiz, M. B., Pion, N., Weinzaepfel, P., and Larlus, D.  (2020).  Hard negative mixing for contrastive learning.  _Advances in Neural Information Processing Systems_ , _33_ , 21798–21809.
  * Khan et al. (2019) Khan, H. N., Shahid, A. R., Raza, B., Dar, A. H., and Alquhayz, H.  (2019).  Multi-view feature fusion based four views model for mammogram classification using convolutional neural network.  _IEEE Access_ , _7_ , 165724–165733.
  * Kim et al. (2020) Kim, H.-E., Kim, H. H., Han, B.-K., Kim, K. H., Han, K., Nam, H., … Kim, E.-K.  (2020).  Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study.  _The Lancet Digital Health_ , _2_(3), e138–e148.
  * Lee et al. (2017) Lee, R. S., Gimenez, F., Hoogi, A., Miyake, K. K., Gorovoy, M., and Rubin, D. L.  (2017).  A curated mammography data set for use in computer-aided detection and diagnosis research.  _Scientific data_ , _4_(1), 1–9.
  * B. Li et al. (2021) Li, B., Li, Y., and Eliceiri, K. W.  (2021).  Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning.  In _Proceedings of the ieee/cvf conference on computer vision and pattern recognition_ (pp. 14318–14328).
  * Z. Li et al. (2021) Li, Z., Cui, Z., Wang, S., Qi, Y., Ouyang, X., Chen, Q., … Cheng, J.-Z.  (2021).  Domain generalization for mammography detection via multi-style and multi-view contrastive learning.  In _Medical image computing and computer assisted intervention–miccai 2021: 24th international conference, strasbourg, france, september 27–october 1, 2021, proceedings, part vii 24_ (pp. 98–108).
  * Lotter et al. (2021) Lotter, W., Diab, A. R., Haslam, B., Kim, J. G., Grisot, G., Wu, E., … others  (2021).  Robust breast cancer detection in mammography and digital breast tomosynthesis using an annotation-efficient deep learning approach.  _Nature Medicine_ , _27_(2), 244–249.
  * Marmot et al. (2013) Marmot, M. G., Altman, D., Cameron, D., Dewar, J., Thompson, S., and Wilcox, M.  (2013).  The benefits and harms of breast cancer screening: an independent review.  _British journal of cancer_ , _108_(11), 2205–2240.
  * McKinney et al. (2020) McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., … others  (2020).  International evaluation of an ai system for breast cancer screening.  _Nature_ , _577_(7788), 89–94.
  * Mercan et al. (2017) Mercan, C., Aksoy, S., Mercan, E., Shapiro, L. G., Weaver, D. L., and Elmore, J. G.  (2017).  Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images.  _IEEE transactions on medical imaging_ , _37_(1), 316–325.
  * Moreira et al. (2012) Moreira, I. C., Amaral, I., Domingues, I., Cardoso, A., Cardoso, M. J., and Cardoso, J. S.  (2012).  Inbreast: toward a full-field digital mammographic database.  _Academic radiology_ , _19_(2), 236–248.
  * Oord et al. (2018) Oord, A. v. d., Li, Y., and Vinyals, O.  (2018).  Representation learning with contrastive predictive coding.  _arXiv preprint arXiv:1807.03748_.
  * Pharoah et al. (2013) Pharoah, P. D., Sewell, B., Fitzsimmons, D., Bennett, H. S., and Pashayan, N.  (2013).  Cost effectiveness of the nhs breast screening programme: life table model.  _Bmj_ , _346_.
  * Rodriguez-Ruiz et al. (2019) Rodriguez-Ruiz, A., Lång, K., Gubern-Merida, A., Broeders, M., Gennaro, G., Clauser, P., … others  (2019).  Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists.  _JNCI: Journal of the National Cancer Institute_ , _111_(9), 916–922.
  * Royal College of Radiologists (2019) Royal College of Radiologists.  (2019).  _Clinical radiology uk workforce census 2019 report._ The Royal College of Radiologists London.
  * Salim et al. (2020) Salim, M., Wåhlin, E., Dembrower, K., Azavedo, E., Foukakis, T., Liu, Y., … Strand, F.  (2020).  External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms.  _JAMA oncology_ , _6_(10), 1581–1588.
  * Schaffter et al. (2020) Schaffter, T., Buist, D. S., Lee, C. I., Nikulin, Y., Ribli, D., Guan, Y., … others  (2020).  Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms.  _JAMA network open_ , _3_(3), e200265–e200265.
  * L. Shen et al. (2019) Shen, L., Margolies, L. R., Rothstein, J. H., Fluder, E., McBride, R., and Sieh, W.  (2019).  Deep learning to improve breast cancer detection on screening mammography.  _Scientific reports_ , _9_(1), 1–12.
  * Y. Shen et al. (2021) Shen, Y., Wu, N., Phang, J., Park, J., Liu, K., Tyagi, S., … others  (2021).  An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization.  _Medical image analysis_ , _68_ , 101908.
  * Shu et al. (2020) Shu, X., Zhang, L., Wang, Z., Lv, Q., and Yi, Z.  (2020).  Deep neural networks with region-based pooling structures for mammographic image classification.  _IEEE transactions on medical imaging_ , _39_(6), 2246–2255.
  * Tian et al. (2019) Tian, Z., Shen, C., Chen, H., and He, T.  (2019).  Fcos: Fully convolutional one-stage object detection.  In _Proceedings of the ieee/cvf international conference on computer vision_ (pp. 9627–9636).
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … Polosukhin, I.  (2017).  Attention is all you need.  _Advances in neural information processing systems_ , _30_.
  * Wei et al. (2022) Wei, T., Aviles-Rivero, A. I., Wang, S., Huang, Y., Gilbert, F. J., Schönlieb, C.-B., and Chen, C. W.  (2022).  Beyond fine-tuning: Classifying high resolution mammograms using function-preserving transformations.  _Medical Image Analysis_ , _82_ , 102618.
  * Wu et al. (2019) Wu, N., Phang, J., Park, J., Shen, Y., Huang, Z., Zorin, M., … others  (2019).  Deep neural networks improve radiologists’ performance in breast cancer screening.  _IEEE transactions on medical imaging_ , _39_(4), 1184–1194.
  * Yala et al. (2019) Yala, A., Lehman, C., Schuster, T., Portnoi, T., and Barzilay, R.  (2019).  A deep learning mammography-based model for improved breast cancer risk prediction.  _Radiology_ , _292_(1), 60–66.
  * Zhao et al. (2020) Zhao, X., Yu, L., and Wang, X.  (2020).  Cross-view attention network for breast cancer screening from multi-view mammograms.  In _Icassp 2020-2020 ieee international conference on acoustics, speech and signal processing (icassp)_ (pp. 1050–1054).
  * Zhou et al. (2022) Zhou, K., Liu, Z., Qiao, Y., Xiang, T., and Loy, C. C.  (2022).  Domain generalization: A survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_.
  * Zhou et al. (2021) Zhou, K., Yang, Y., Qiao, Y., and Xiang, T.  (2021).  Domain generalization with mixstyle.  _arXiv preprint arXiv:2104.02008_.
  * J.-Y. Zhu et al. (2017) Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A.  (2017).  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In _Proceedings of the ieee international conference on computer vision_ (pp. 2223–2232).
  * W. Zhu et al. (2017) Zhu, W., Lou, Q., Vang, Y. S., and Xie, X.  (2017).  Deep multi-instance networks with sparse label assignment for whole mammogram classification.  In _International conference on medical image computing and computer-assisted intervention_ (pp. 603–611).

## Supplementary Material

This supplementary document provides additional information and visual results
to complement the main paper, aiming to elaborate on the practical aspects and
offer further insights into our approach and experimental findings.

##  7 AUROC Curves, Data Distribution & Further Statistics

To strengthen the validity of our findings, we present the receiver operating
characteristic (ROC) curves comparing our technique with existing methods. The
results, depicted in Fig. 4, showcase a collection of ROC curves across the
test set of the Seen Domains and all data of the Unseen Domains. These curves
serve as a powerful visual representation that illustrates the balance between
clinical sensitivity and specificity. Upon careful examination of Fig. 4, it
becomes evident that our model outperforms the existing networks in terms of
its discriminative capacity for cancer diagnostics. MammoDG, in particular,
exhibits curves that are positioned closer to the top-left corner, indicating
superior performance.

To strengthen the validation of our technique, we conducted a comprehensive
statistical analysis displayed in Fig. 6. Firstly, we performed a Friedman
test for multiple comparisons, which allowed us to assess the overall
performance of different methods. Subsequently, we employed a pair-wise
comparison using the Wilcoxon test to further support our findings.

The results of these tests revealed that our technique exhibited significantly
elevated performance, particularly in the case of unseen domains. This
emphasises the robustness and effectiveness of our approach, showcasing its
ability to generalise well beyond the training data and adapt to previously
unseen domains.

Figure 4: Receiver Operating Characteristic (ROC) curves comparing our
technique with existing methods on the test set of the Seen Domains and all
data of the Unseen Domains.

To provide additional evidence for the effectiveness of our MammoDG model, we
highlight the significant differences in intensity between datasets obtained
from various sites and vendor machines. Upon closer examination in Fig. 5, it
becomes apparent that there is a considerable domain shift between these
datasets. This poses a challenge for the learning process and necessitates the
use of domain generalisation techniques capable of effectively handling these
variations.

Figure 5: Intensity Distribution in Datasets from Various Sites and Vendor
Machines. Figure 6:  Statistical comparison between our technique and existing
methods for Unseen Domains.

##  8 Comparative Analysis: Rationale for Selected Technique

In this section, we present a comprehensive table that includes a detailed
overview of existing techniques along with their respective implementation
platforms. The selected techniques for comparison are marked with a tick (✓)
in the table, indicating the availability of their source code. It is
important to note that many existing techniques do not provide open-source
code, making it impossible to directly compare our approach against them.

The selection of techniques for comparison was based on the availability of
their implementation details and the accessibility of their source code. It is
worth mentioning that several techniques in the literature did not report
sufficient information regarding their implementation, limiting our ability to
include them in the comparative analysis.

We acknowledge the significance of open-source code availability for fostering
reproducibility and facilitating fair comparisons. In line with this, we
affirm that the source code for our proposed approach will be made available
upon acceptance, ensuring transparency and enabling researchers to validate
our results and conduct further investigations.

Table 4: A survey on deep learning-based mammography analysis literature.
NYUBCS: NYU breast cancer screening dataset. Compared means whether this
method is compared in this paper.

Method | Datasets |  | ROI
---
Annotation
Compared | Platform
| Weakly supervised localization for breast
---
cancer screening (GMIC) (Y. Shen et al., 2021)
| NYUBCS
---
DDSM
- | ✓ | PyTorch
| Domain Generalization via Multi-style and Multi-view
---
Contrastive Learning (MSVCL) (Z. Li et al., 2021)
| 5 datasets
---
- | ✓ | PyTorch
| Multi-view Hypercomplex Neural
---
Networks (Z. Li et al., 2021)
| DDSM
---
INbreast
- | - | PyTorch
| Region-based pooling structure
---
(Shu et al., 2020)
| DDSM
---
INbreast
- | - | PyTorch
| External evaluation of 3 commercial
---
AI algorithms (Salim et al., 2020)
CSAW | - | - | -
| Robust detection with annotation-efficient
---
deep learning (Lotter et al., 2021)
| DDSM/OMI-DB
---
private dataset
✓ | - | Keras
| Different level analysis for lesion,
---
breast, and case (McKinney et al., 2020)
UK/US datasets | ✓ | - | Tensorflow
| Two stages convolutional neural
---
network (Kim et al., 2020)
| South Korea/
---
UK/US
✓ | - | PyTorch
| Deep Learning to Improve Breast
---
Cancer Detection (L. Shen et al., 2019)
DDSM | ✓ | ✓ | Keras
| Deep Neural Networks Improve Radiologists’
---
Performance (DMV-CNN) (Wu et al., 2019)
NYU-V1.0 | ✓ | ✓ | Tensorflow
| Multi-View Feature Fusion Based
---
Four Views Model (MVFF) (Khan et al., 2019)
| DDSM
---
mini-MIAS
- | ✓ | Tensorflow
| Multi-view network (BIRADS)
---
(Geras et al., 2017)
| DDSM/INbreast
---
others
- | ✓ | PyTorch
| Deep multi-instance networks with
---
sparse label assignment (W. Zhu et al., 2017)
INbreast | - | - | Keras