1IBM Research, Beijing, China; 2IBM Research, Yorktown Heights, NY, USA

# Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive
Modeling with Electronic Health Records

Yiqin Yu1 Pin-Yu Chen2 Yuan Zhou1 Jing Mei1

Abstract

With the successful adoption of machine learning on electronic health records
(EHRs), numerous computational models have been deployed to address a variety
of clinical problems. However, due to the heterogeneity of EHRs, models
trained on different patient groups suffer from poor generalizability. How to
mitigate domain shifts between the source patient group where the model is
built upon and the target one where the model will be deployed becomes a
critical issue. In this paper, we propose a data augmentation method to
facilitate domain adaptation, which leverages knowledge from the source
patient group when training model on the target one. Specifically,
adversarially generated samples are used during domain adaptation to fill the
generalization gap between the two patient groups. The proposed method is
evaluated by a case study on different predictive modeling tasks on MIMIC-III
EHR dataset. Results confirm the effectiveness of our method and the
generality on different tasks.

## 1 Introduction

Benefiting from the accumulation of large scale electronic health records
(EHRs), numerous computational models have been developed to facilitate
clinical decision support processes with improved quality of health care and
decreased costs. Data mining and machine learning technologies are widely used
to build computational models to address a variety of clinical problems such
as predictive modeling for risk prediction, disease progression modeling, and
patient phenotyping [1, 2, 3]. Although advanced machine learning methods,
especially deep learning models, have gained notable performance improvements
on the patient group where they are trained on, nevertheless, they still
suffer from poor generalizability when deployed in new environments (e.g.,
downgraded AUROC or accuracy on the new dataset). This is mainly due to the
heterogeneity of EHRs among different patient groups[4].

There are several aspects that will lead to the heterogeneity. First, the
distributions of variables in EHR data which are used as inputs of the model
as well as the modeling targets often differ among different patient groups
[5]. For example, mid-life weight was more closely associated with mid-life
blood pressure than weight at age 20 years [6]. In patients hospitalized with
myocardial infarction, women had higher mortality than men within the same age
group [7]. Second, the heterogeneity can also be subject to data biases [8]
such as the occurrences of missing data caused by the irregular ordering of
lab tests and patients whose data is not collected at regular intervals. In
other words, due to the heterogeneity of EHRs, the curated data of a specific
patient group used to train the model is not fully representative of what the
model will encounter when it is deployed.

Domain adaptation [9, 10] (DA) is one of the key methodologies of transfer
learning to leverage knowledge from a different but related domain as
additional information when training models on the target domain, where the
input and label spaces remain unchanged while the data distributions change
among two domains. In the context of predictive modeling on EHRs, a domain
usually corresponds to a patient group. Consequently, DA can enable modeling
and improve machine learning performance on a target patient group by
mitigating the domain shift between this patient group and another different
yet related patient group. DA has been shown in plenty of applications that it
can improve the generalizability of models, especially when there lacks labels
(data sample annotations) in the target domain [10, 11].

While there are plenty of works applying domain adaptation methodology on
EHRs, the main focus is on medical images [12, 13, 14, 15], and only a few
works study structured longitudinal EHR data. In recent years, several
researchers use recurrent neural network based models to learn domain
invariant temporal relationships between domains on multivariate time series
EHR data [16, 17, 5]. Mainly, the domain adversarial neural network (DANN)
architecture [18] is incorporated to deal with the domain discrepancy via
adversarial training between the label classifier and the domain
discriminator. Even though these DANN oriented models achieve relatively high
performance on unsupervised tasks, the intrinsic mechanism remains unclear.
For example, the feature extractor in DANN is supposed to learn the domain
invariant representation, but the selection of network architecture for the
feature extractor is arbitrary and the learnt representation is hard to
explain.

In this paper, we propose a data augmentation method with adversarially
generated samples to facilitate domain adaptation. Adversarial samples are
formed by applying small but intentionally worst-case (automated and guided)
perturbations to samples from the training dataset [19]. By involving
adversarial samples into the training process (so called adversarial
training), the trained model can achieve improved adversarial robustness of
the model (capacity of defending against adversarial attacks) [19, 20, 21]. In
spite of focusing on the adversarial robustness, several works study the
effect of adversarial samples on model generalizability during domain
adaptation. One work [10] proposes a method to augment the source domain
dataset by iteratively appending adversarial samples during the training
process and result in better generalizability to other unseen domains. Another
work [22] generates adversarial samples with DANN to fill in the gap between
the source and the target domain, and adversarially trains the deep
classifiers to make consistent predictions on adversarial samples. While these
works provide strong evidence that adversarial samples can be a promising data
augmentation mechanism for deep neural network based domain adaptation on
image data, there still lacks comprehensive studies and performance analysis
on simple but more explainable algorithms such as logistic regression and more
heterogeneous data such as EHRs.

Motivated by plentiful and practical domain adaptation requirements with EHRs,
in this paper we introduce the proposed adversarial sample enhanced domain
adaptation method with a case study on predictive modeling tasks with EHRs. We
implement the method based on logistic regression (LR), which is the most
dominant algorithm that has been used extensively in health and biomedical
problems. Comparing to deep neural networks, LR is simple, easy to interpret
and with less training time and computational cost yet acceptable performance.
Furthermore, the proposed domain adaptation framework evaluated on LR in this
paper can be easily applied to other empirical risk minimization based deep
neural networks. For the case study, we utilize existing logistic regression
benchmarks [23] based on MIMIC-III EHR dataset [24] with three types of
predictive modeling tasks: in-hospital mortality prediction, length-of-stay
prediction and phenotype classification. A series of domain adaptation
experiments are performed to compare among different patient groups with
different variants of our method. Experimental results show that our method
improves the generalizability of the trained models, and the method has good
generality cross different predictive modeling tasks.

## 2 Dataset and Task Description

We perform our case study on the MIMIC-III EHR dataset [24]. MIMIC-III is the
largest publicly available database of de-identified EHR from ICU patients. It
contains data that covers 38,597 distinct adult patients and 49,785 hospital
admissions. Based on its large volume of data size and variety of data types,
a recent work [23] constructed four predictive modeling benchmarks. Among
them, the following tasks are selected to evaluate our domain adaptation
methods:

  * •

In-hospital mortality prediction. Predicting in-hospital mortality based on
the first 48 hours of an ICU stay. As a binary classification task, it is
mainly evaluated by area under the receiver operating characteristic (AUROC).

  * •

Length-of-stay prediction. Predicting remaining time spent in ICU at each hour
of stay. It is formed as a multi-class classification problem with 10 classes
(one for ICU stays shorter than a day, seven day-long buckets for each day of
the first week, one for stays of over one week but less than two, and one for
stays of over two weeks). Cohen’s linear weighted kappa score [25] is used for
evaluation.

  * •

Phenotype classification. Classifying which of the 25 care conditions are
present in a given patient ICU stay. These conditions include both acute
conditions such as acute cerebrovascular disease, acute myocardial infarction,
and chronic conditions such as chronic kidney disease, diabetes mellitus
without complication and essential hypertension. As a multi-label
classification problem, it takes macro-averaged AUROC as the main metric.

All tasks share common feature space consisting of 17 clinical variables such
as capillary refill rate, diastolic blood pressure and glucose. For each
variable, six different sample statistic features (minimum, maximum, mean,
standard deviation, skew and number of measurements) are computed on seven
different sub-sequences of a given time series (the full time series, the
first 10% of time, first 25% of time, first 50% of time, last 50% of time,
last 25% of time and last 10% of time). Totally, there are $17\times 7\times
6=714$ features per time series. To test the domain shifts among different
patient groups, we follow a most recent domain shifts evaluation work [26] on
MIMIC-III and divide the whole patient group into the following sub-groups
according to the heterogeneity of EHR data: 1) Age. Two sub-groups are
created: Older, which comprises of patients who are 60 years old and above,
and Younger, which included patients who are younger than 60; 2) Gender. There
are also two sub-groups: Female and Male.

## 3 Method

We first define the notations used in our method. A domain
$\mathcal{D}=(\mathcal{X},\mathcal{Y})$ denotes a patient group which is
represented by a number of patients whose observed features are in the input
space $\mathcal{X}$ and the target outcomes in the output space $\mathcal{Y}$.
$f(\theta)$ denotes a predictive model trained with logistic regression on
$\mathcal{D}$ with parameter $\theta$. We define the source domain as
$D_{S}=(X_{S},Y_{S})$, where $X_{S}\in\mathbbm{R}^{m\times d}$ and
$Y_{S}\in\mathbbm{R}^{m\times k}$, and the target domain as
$D_{T}=(X_{T},Y_{T})$, where $X_{T}\in\mathbbm{R}^{n\times d}$ and
$Y_{T}\in\mathbbm{R}^{n\times k}$. Here $m$ and $n$ is the number of samples
in $D_{S}$ and $D_{T}$, respectively, and $d$ is the number of features with
$k$ being the number of target outcomes. A sample
$(x,y)\in(\mathcal{X},\mathcal{Y})$ is an individual record processed by the
model. Note that the output space $\mathcal{Y}$ for different modeling tasks
will vary. In this paper, we deal with three types of tasks:

  * •

Binary classification. There is only one target outcome in the output space
$\mathcal{Y}$ with value of true or false. So here we set $k=1$ and
$\mathcal{Y}=\\{0,1\\}$. For example, in in-hospital mortality prediction, the
target outcome is whether (true or false) the death occurs during
hospitalization.

  * •

Multi-class classification. There is only one target outcome in the output
space, but there can be more than two classes. Each class in the target
outcome is exclusive, which means that each training instance can only be
labelled to one class. So here $k=1$ and $\mathcal{Y}=\\{0,\cdots,C-1\\}$
where $C$ is the number of classes. The task of length-of-stay prediction is a
multi-class classification when dividing the actual length of stay for each
patient into 10 classes. Note that in practice, the multi-class classification
problem can be decomposed into the binary classification problem, each of
which has the output space $\mathcal{Y}=\\{0,1\\}$.

  * •

Multi-label classification. Here more than one target outcome exist in the
task with $k>1$ (e.g., phenotype classification). For each outcome, it could
be either a binary classification or a multi-class classification. For each
patient, more than one outcome can be set to true.

### 3.1 Logistic Regression

We first consider modeling tasks on the same domain, say the domain $D=(X,Y)$.
As in the aforementioned discussion, both multi-class and multi-label
classification can be decomposed into the binary classification problem. So
without loss of generality we use $k=1$ throughout the method part. All the
formulas can be easily generalized to multi-class and multi-label
classification problems. Given a sample $(x,y)$ from $D$, the conditional
distribution of $y$ given $x$ in logistic regression is modeled as

$p(y|x;\theta)=\left[g\left(\theta^{T}x\right)\right]^{y}\cdot\left[1-g\left(\theta^{T}x\right)\right]^{1-y}$
(1)

where $\theta$ is the parameter of the model $f(\cdot)$, and $g(\cdot)$ is the
logistic function $g\left(\theta^{T}x\right)=\frac{1}{1+e^{-\theta^{T}x}}$.
The loss function can be written as

$l_{D}(\theta)=-\log\prod_{i=1}^{n}p\left(y_{i}|x_{i};\theta\right)=-\sum_{i=1}^{n}\left[y_{i}\log
g\left(\theta^{T}x_{i}\right)+\left(1-y_{i}\right)\log\left(1-g\left(\theta^{T}x_{i}\right)\right)\right]$
(2)

During the training process, the parameters are estimated by minimizing the
loss function:

$\theta=\arg\min_{\theta}l_{D}(\theta)$ (3)

### 3.2 Domain Adaptation with Logistic Regression

Given the source domain $D_{S}=(X_{S},Y_{S})$, the predictive model
$f_{S}(\theta_{S})$ trained on $D_{S}$ and the target domain
$D_{T}=(X_{T},Y_{T})$, the domain adaptation framework aims to train the
target model $f_{T}(\theta_{T})$ by leveraging knowledge from $D_{S}$.
According to different parts of $D_{S}$ where $f_{T}$ can leverage knowledge
from, there are mainly two scenarios. In the first scenario, the source domain
data $(X_{S},Y_{S})$ is known and is used to compare domain discrepancy
between $(X_{S},Y_{S})$ and $(X_{T},Y_{T})$. The second one is to leverage
knowledge from $f_{S}$, in case that $(X_{S},Y_{S})$ can not be accessed when
training $f_{T}$ on $D_{T}$. In this paper, we will focus on the second
situation as it occurs frequently in our considered scenarios (e.g., due to
privacy of EHR, only the source model is accessible, but not the raw data).

Following the previous work [27], we involve the parameter $\theta_{S}$ of the
logistic regression model $f_{S}$ trained on $D_{S}$ into the training process
of $f_{T}$. Suppose $\theta_{S}$ is obtained beforehand via the following
training process:

$\theta_{S}=\arg\min_{\theta_{S}}l_{D_{S}}(\theta_{S}).$ (4)

Then, $\theta_{T}$ is optimized on $(X_{T},Y_{T})$ with the following loss
function:

$\theta_{T}=\arg\min_{\theta_{T}}l_{D_{T}}(\theta_{T})+\lambda\|\theta_{T}-\theta_{S}\|^{2},$
(5)

where the second part is a modified $L_{2}$ regularizer to penalize the
discrepancy between parameters of the source model $\theta_{S}$ and parameters
of the target model $\theta_{T}$. $\lambda$ is used to trade off the
proportion of knowledge that can be leveraged from the treatment of the model
discrepancy.

### 3.3 Adversarial Sample Enhanced Domain Adaptation

The idea of using adversarial samples as a method of data augmentation during
domain adaptation is motivated by the following findings: 1) minimizing the
model training error with adversarial samples can potentially bound the error
on the target domain [28], and 2) training with adversarial samples can aid in
finding domain-invariance and generalizable data features. [29, 21, 30].

As in the preceding equation 5, domain adaptation evaluates the discrepancy
between two domains by comparing model parameters $\theta_{S}$ and
$\theta_{T}$. However, due to the heterogeneity of different domains, the
discrepancy always exists. Moreover, with the increasing of domain
discrepancy, the training process will fail to adapt. In order to deal with
this issue, we generate adversarial samples during domain adaptation and use
them as part of the training set to make the adaptation more smoothly. In
other words, the adversarial samples play two roles. First, it makes the
trained model more robust and have better generalization ability; second, it
plays as the intermediate distribution between the source and the target
domain, so that the knowledge transfer will be more smooth.

More concretely, we follow the iterative fast gradient sign method [19, 31] to
generate adversarial samples by applying perturbations in the direction of the
gradient of the loss function defined in Eq. 4 and Eq. 5, with the goal of
increasing the training loss:

$x_{0}^{adv}=x,\quad
x_{Z}^{adv}=\operatorname{Clip}_{x,\epsilon}\left\\{x_{Z-1}^{adv}+\frac{\epsilon}{Z}\operatorname{sign}\left(\nabla_{x}l\left(\theta,x_{Z-1}^{adv},y\right)\right)\right\\},$
(6)

where $Z$ is the number of iteration,
$\operatorname{Clip}_{x,\epsilon}\\{\cdot\\}$ is a function to perform
feature-wise clipping of the generated adversarial sample to make it being in
$L_{\infty}$ $\epsilon$-neighbourhood of the source sample $x$, and
$\operatorname{sign}(\cdot)$ refers to being positive (+1) or negative (-1) of
the gradient. Adversarial samples are generated for both the source domain and
the target domain with Eq. 6 and $\theta_{S}$ and $\theta_{T}$ from Eq. 4 and
Eq. 5, respectively. Firstly, we train the LR model with both normal samples
$x_{S}$ and adversarial generated samples $x_{S}^{adv}$ on the source domain
to get $\theta_{S}$ with Eq. 7. Then, $\theta_{S}$ is used in the domain
adaptation process with both normal samples $x_{T}$ and adversarial generated
samples $x_{T}^{adv}$ on the target domain to obtain $\theta_{T}$ with Eq. 8.
The modified loss functions for the source domain model and domain adaptation
on the target domain are as follows:

$\theta_{S}=\arg\min_{\theta_{S}}l_{D_{S}}(\theta_{S},x_{S},y_{S})+\alpha
l_{D_{S}}(\theta_{S},x_{S}^{adv},y_{S})$ (7)
$\theta_{T}=\arg\min_{\theta_{T}}l_{D_{T}}(\theta_{T},x_{T},y_{T})+\alpha
l_{D_{T}}(\theta_{T},x_{T}^{adv},y_{T})+\lambda\|\theta_{T}-\theta_{S}\|^{2}$
(8)

where $\alpha\in(0,1]$ is a weighting coefficient to adjust contributions from
the adversarial samples.

## 4 Results

We evaluate the proposed domain adaptation method on three predictive modeling
tasks with four domain adaptation tasks on MIMIC-III. The sizes of samples on
different patient groups for each modeling task are shown in Table 1. Note
that in the length-of-stay prediction tasks each sample is the data record
collected hourly. For domain adaptation, there are four tasks with different
patient groups as the source domain and the target domain: Older$\to$Younger,
Younger$\to$Older, Female$\to$Male, and Male$\to$Female. For each domain, the
data is further divided into the training set with 85% samples and the test
set with remain 15% samples. For example, for domain Female in in-hospital
mortality prediction, the total sample number is 9510, and the sizes of the
training set and the test set are 8071 and 1439, respectively.

All models are trained based on logistic regression (Eq. 3) with
hyperparameters: learning rate$=0.01$, batch size$=100$ and number of
epochs$=100$. In order to intervene into the empirical risk minimization
process to perform adversarial training and domain adaptation with LR, we
rewrite the algorithm with PyTorch [32] instead of using scikit-learn [33] as
in the benchmark [23]. To make the series of experiments simple, we also
remove the $L_{2}$ regularizer which leads to a slight downgrade of
performance in our experiment.

Table 1: Size of prediction instances for different modeling tasks. Modeling Tasks | Gender | Age | Total
---|---|---|---
| Female | Male | Younger | Older |
In-hospital mortality prediction | 9510 | 11629 | 7387 | 13752 | 21139
Length-of-stay prediction | 1520754 | 1930592 | 1279008 | 2172338 | 3451346
Phenotype classification | 18477 | 23425 | 15929 | 25973 | 41902
Figure 1: Domain shifts evaluated on different target domains.

### 4.1 Domain Shift Evaluation

In order to better understand how the domain adaptation framework works, we
first perform domain shift evaluation for all three modeling tasks with
results shown in Figure 1. Each modeling task is evaluated with different
measurements where AUROC is used for in-hospital mortality prediction, Kappa
score is used for length-of-stay prediction, and Macro AUROC is used for
phenotype classification. Taking the second bar “male $\to$ female” in in-
hospital mortality prediction as an example, we first train the model with
regular logistic regression (Eq. 3) on the domain Male (trained on its
training set and tested on its test set), and then test it with the test set
of domain Female to get the AUROC value as 0.799. For each modeling task,
domain shift evaluation for the same target domain is grouped together. For
instance, for the target domain Female, there are three models being compared:
the model trained on the same domain Female, the model trained on the
corresponding domain Male, and the one trained on the full domain All with
data from Female and Male mixed together.

We can see that the model trained on All and tested on Female gets the best
performance comparing to the model trained on sub-groups. It is consistent
with common consensus that the size of training data has dominated impact on
the performance. The most interesting finding is that the model trained on
Male is even slightly better than the one trained on Female, which implies the
greater possibility to transfer knowledge from Male to Female. Contrarily,
with Male as the target domain, the model trained on its own training set
performs best among all models, which indicates Male might have strong
representability of the feature space for in-hospital mortality prediction. As
a result, we can suspect that knowledge from Female might have trivial effect
on Male during domain adaptation.

Figure 2: Adversarial training versus normal training for in-hospital
mortality prediction (evaluated on AUROC). Figure 3: Comparison between
$\theta$ from sparsity regularized normal training and $\theta$ from
adversarial training for in-hospital mortality prediction on Female$\to$Female
(cosine similarity=0.81).

### 4.2 Adversarial Sample as Data Augmentation

Before using adversarial samples for domain adaptation, we also evaluate its
effect on models trained on the source domain (Adversarial Training with Eq. 6
and Eq. 7) and tested on the target domain, and compare with the normal
training methods (Normal Training with Eq. 3). The results for in-hospital
mortality prediction are shown in Figure 2 with weight comparison on
Female$\to$Female in Figure 3. The adversarial training is performed with
hyperparameters $\epsilon=0.1$, $T=20$ and $L_{\infty}$-norm as the
perturbation type. Considering that the effect of adversarial samples on each
patient group will differ, we test the adversarial training with different
values of $\alpha$ among $(0,1]$ and report the best performance in Figure 2.

We can see that on almost all the tasks, the adversarial training models for
in-hospital mortality prediction outperform the normal trained models except
the case of All $\to$ Younger. Note that each adversarial training model
achieves its best performance on different $\alpha$. For example, the AUROC
for Male $\to$ Male achieves the largest value 0.8534 with $\alpha=0.1$ and
monotonically decreases with larger $\alpha$. Unlike Male $\to$ Male, the
AUROC for Male $\to$ Female get its highest value 0.8283 when $\alpha=0.3$,
indicating that the adversarial samples of Male play more important role on
domain shift mitigation between Male and Female. In Figure 2, we can also see
that the performance gain via adversarial training are greater in Figure 2 (a)
than in Figure 2 (b). Recall the domain shifts evaluated for Male in Figure 1,
it might be due to the strong representability of the feature space of Male
for in-hospital mortality prediction. Another observation is that the
adversarial training performs better on sub-groups than on the full groups
(The only downgrading of adversarial training performance happens on All $\to$
Younger). It is reasonable because on sub-groups, the performance of the model
is more likely to be limited by the training size, while on the full group,
training data is relatively sufficient, so the effect of adversarial samples
as a data augmentation method will be diminished.

### 4.3 Adversarial Sample Enhanced Domain Adaptation

We implement our adversarial sample enhanced domain adaptation framework on
all three predictive modeling tasks. More concretely, we first do adversarial
training on the source domain, and then perform adversarial training with
domain adaptation on the target domain. Finally the performance is evaluated
on the test set of the corresponding target domain. In Figure 4, we compare
the performance of each domain adaptation task (e.g., Male$\to$Female) on two
baseline settings and three domain adaptation variants:

  * •

NT_source. Normal training on the source domain and testing on the target
domain.

  * •

NT_target. Normal training and testing on the target domain (the source domain
is not used).

  * •

DA with NT_source, NT_target. First training on the source domain and then
training with domain adaptation on the target domain without adversarial
samples (Eq. 5).

  * •

DA with AT_source, NT_target. Training procedure the same with the previous
one, but use adversarial training on the source domain (Eq. 7).

  * •

DA with AT_source, AT_target. Training procedure remaining with adversarial
samples injected during domain adaptation on the target domain (Eq. 8).

We can see that in almost all the circumstances, the three domain adaptation
methods outperform the two baselines. It means that domain adaptation is
capable to leverage knowledge from both the source domain and the target
domain. Comparing “DA with AT_source, NT_target” with “DA with NT_source,
NT_target”, the adversarial training on the source domain works for domain
adaptation in most tasks (11 out of 12 tasks). Note that the adversarial
training on the source domain is not guided to optimize the domain adaptation
task on the target domain. It plays only the role as data augmentation for the
source domain. Between the later two variants of domain adaptation methods, we
can also observe that the adversarial training on the target domain is able to
improve the performance (9/12 tasks with adversarial training on the target
domain have increased performance when compared to “DA with AT source, NT
target”).

Figure 4: Domain adaptation results.

## 5 Discussion

Heterogeneity of EHRs. The main reason of poor generalizability of predictive
models trained on EHR data is the heterogeneity of EHRs. In this paper, we
evaluate domain shifts mainly on the heterogeneity between patient groups with
different age and gender groups. These two metrics are also frequently used in
contemporary predictive modeling tasks to do risk stratification [34, 35].
There are also other kinds of heterogeneity such as differences among label
distributions and measurements discrepancies [26]. We consider domain
adaptation analysis on these scenarios as our future works.

Effect of adversarial samples. Adversarial samples are originally proposed as
an attack methods to test the adversarial robustness of a model, and then
injected into the training data during the training process to defend the
corresponding attacks [19, 31]. While the adversarial training ensures the
adversarial robustness of the model, recent works [36, 37, 29] argue that
there might be a trade-off between the adversarial robustness and the
generalizability, reflected by higher robust accuracy and lower standard
accuracy comparing with normal trained models. As these works are mainly
evaluated on large volume image datasets, the effect of adversarial samples on
more complex data types and feature distributions has not been well studied.
For example, one work [38] shows that adversarial training can improve the
generalizability on text classification tasks. Our results also show that
adversarial samples can help the generalizability especially for modeling
tasks on sub-groups, implying its capacity as data augmentation on data with
complex data types and feature distributions. Especially for logistic
regression, it can be proved that jointly train the LR model with adversarial
samples is equivalent to a sparsity regularized version of LR [39] (One can
see in Figure 3 that the $\theta$ of sparsity regularized LR and the one from
adversarial trained LR are very similar with cosine similarity being 0.81).
This finding also opens up a perspective for our further research.

Domain adaptation types. In this paper, we consider domain adaptation by
transferring knowledge from the model trained on the source domain given the
source domain data can not be accessed when training model on the target
domain. There are a lot of other domain adaptation scenarios given whether
certain elements in a domain
$\mathcal{D}=\\{(\mathcal{X},\mathcal{Y}),f(\theta)\\}$ are absent or not. For
example, unsupervised domain adaptation deal with unlabeled target domain data
(where $Y_{T}$ is absent) by learning a domain invariant joint representation
on feature distributions of both the source domain and the target domain [16,
17, 5]. There are also strong interests in domain adaptation when the size of
target domain data is very small, leading to subdivided research topics such
as few-shot domain adaptation . Limited to the length of the paper, we do not
include the evaluation of our method on these domain adaptation tasks, but as
our method achieves better performance on domains on sub-groups with smaller
training size, it is expected to perform well under more restricted situations
as well.

Effect of domain adaptation. Domain adaptation is proposed to leverage
knowledge from other different but related domain(s) during model training on
the target domain. One critical issue is to balance knowledge from the source
domain and knowledge from the target training data. In this paper, we use grid
search to optimize the hyperparameter $\lambda$ in Eq. 8. There are also some
works aiming to automate the tuning of the hyperparameter, which will be
brought into our future works. In this paper, we empirically prove that
adversarial samples as a method of data augmentation can improve the
generalizability for domain adaptation. It is worth to further analyse the
relationship between this effect and other key attributes of the training task
such as the data size as well as the heterogeneity of data.

## 6 Conclusion

In this paper, we propose a domain adaptation framework and evaluate its
effectiveness via a case study on different predictive modeling tasks with EHR
data. Furthermore, adversarially generated samples are used as a data
augmentation method in the domain adaptation procedure to improve its
performance. Results show that our method improves the generalizability of
predictive models on three predictive modeling tasks with the MIMIC-III EHR
dataset.

## References

  * 1. Pranjul Yadav, Michael Steinbach, Vipin Kumar, and Gyorgy Simon.  Mining electronic health records (EHR): A survey.  ACM Computing Surveys, 2018, 50(6):1–40.
  * 2. Benjamin Shickel, Patrick James Tighe, Azra Bihorac, and Parisa Rashidi.  Deep ehr: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis.  IEEE journal of biomedical and health informatics, 2017, 22(5):1589–1604.
  * 3. Kenney Ng, Amol Ghoting, Steven R Steinhubl, Walter F Stewart, Bradley Malin, and Jimeng Sun.  Paramo: A parallel predictive modeling platform for healthcare analytic research using electronic health records.  Journal of biomedical informatics, 2014, 48:160–170.
  * 4. George Hripcsak and David J Albers.  Next-generation phenotyping of electronic health records.  Journal of the American Medical Informatics Association, 2013, 20(1):117–121.
  * 5. Yuan Zhang, Xi Yang, Julie Ivy, and Min Chi.  Time-aware adversarial networks for adapting disease progression modeling.  In IEEE International Conference on Healthcare Informatics, 2019\.
  * 6. Sundström J, Lind L, Lampa E, Angerås O, Bachus E, Bergström G, Carlberg B, Engström G, Engvall JE, Eriksson M, Gigante B, Hagström E, Hjelmgren O, Jansson JH, Jernberg T, Mannila M, Nyström FH, Oldgren J, Persson M, Sandström A, Swahn E, Söderberg S, Torén K, Östgren CJ, and Rosengren A.  Weight gain and blood pressure.  J Hypertens, 2020, (3):387–394.
  * 7. John G Canto, William J Rogers, Robert J Goldberg, Eric D Peterson, Nanette K Wenger, Viola Vaccarino, Catarina I Kiefe, Paul D Frederick, George Sopko, Zhi-Jie Zheng, et al.  Association of age and sex with myocardial infarction symptom presentation and in-hospital mortality.  Jama, 2012, 307(8):813–822.
  * 8. Milena A Gianfrancesco, Suzanne Tamang, Jinoos Yazdany, and Gabriela Schmajuk.  Potential biases in machine learning algorithms using electronic health record data.  JAMA internal medicine, 2018, 178(11):1544–1547.
  * 9. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan.  A theory of learning from different domains.  Machine learning, 2010, 79(1-2):151–175.
  * 10. Wouter M Kouw.  An introduction to domain adaptation and transfer learning.  arXiv preprint arXiv:1812.11806, 2018.
  * 11. Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He.  A comprehensive survey on transfer learning.  arXiv preprint arXiv:1911.02685, 2019.
  * 12. M Hamed Mozaffari and Won-Sook Lee.  Domain adaptation for ultrasound tongue contour extraction using transfer learning: A deep learning approach.  The Journal of the Acoustical Society of America, 2019, 146(5).
  * 13. Yanyang Gu, Zongyuan Ge, C Paul Bonnington, and Jun Zhou.  Progressive transfer learning and adversarial domain adaptation for cross-domain skin disease classification.  IEEE journal of biomedical and health informatics, 2019.
  * 14. Yue Zhang, Shun Miao, Tommaso Mansi, and Rui Liao.  Unsupervised x-ray image segmentation with task driven generative adversarial networks.  Medical Image Analysis, 2020, 61:101664.
  * 15. Yaxin Shen, Bin Sheng, Ruogu Fang, Huating Li, Ling Dai, Skylar Stolte, Jing Qin, Weiping Jia, and Dinggang Shen.  Domain-invariant interpretable fundus image quality assessment.  Medical Image Analysis, 2020, 61:101654.
  * 16. Sanjay Purushotham, Wilka Carvalho, Tanachat Nilanon, and Yan Liu.  Variational recurrent adversarial deep domain adaptation.  2017\.
  * 17. Tiago Alves, Alberto Laender, Adriano Veloso, and Nivio Ziviani.  Dynamic prediction of icu mortality risk using domain adaptation.  In IEEE International Conference on Big Data, 2018.
  * 18. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  The Journal of Machine Learning Research, 2016, 17(1):2096–2030.
  * 19. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  In International Conference on Learning Representations, 2015.
  * 20. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  In International Conference on Learning Representations, 2018.
  * 21. Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang.  MMA training: Direct input space margin maximization through adversarial training.  In International Conference on Learning Representations, 2020.
  * 22. Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan.  Transferable adversarial training: A general approach to adapting deep classifiers.  In International Conference on Machine Learning, 2019.
  * 23. Hrayr Harutyunyan, Hrant Khachatrian, David C Kale, Greg Ver Steeg, and Aram Galstyan.  Multitask learning and benchmarking with clinical time series data.  Scientific data, 2019, 6(1).
  * 24. Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark.  Mimic-iii, a freely accessible critical care database.  Scientific data, 2016, 3:160035.
  * 25. Jacob Cohen.  A coefficient of agreement for nominal scales.  Educational and psychological measurement, 1960, 20(1):37–46.
  * 26. Deepta Rajan Jayaraman J. Thiagarajan and Prasanna Sattigeri.  Understanding behavior of clinical models under domain shifts.  In Workshop on the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2019.
  * 27. Ciprian Chelba and Alex Acero.  Adaptation of maximum entropy capitalizer: Little data can help a lot.  Computer Speech & Language, 2006, 20(4):382–399.
  * 28. Jaeho Lee and Maxim Raginsky.  Minimax statistical learning with wasserstein distances.  In Advances in Neural Information Processing Systems, 2018.
  * 29. Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael I. Jordan.  Theoretically principled trade-off between robustness and accuracy.  In International Conference on Machine Learning, 2019.
  * 30. Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry.  Adversarial examples are not bugs, they are features.  In Advances in Neural Information Processing Systems, 2019.
  * 31. Alexey Kurakin, Ian Goodfellow, and Samy Bengio.  Adversarial examples in the physical world.  In International Conference on Learning Representations, 2017.
  * 32. Pytorch.  https://pytorch.org/.
  * 33. scikit-learn.  https://scikit-learn.org/stable/.
  * 34. Tina J Hieken, Amy E Glasgow, Elizabeth Ann L Enninga, Lisa A Kottschade, Roxana S Dronca, Svetomir N Markovic, Matthew S Block, and Elizabeth B Habermann.  Sex-based differences in melanoma survival in a contemporary patient cohort.  Journal of Women’s Health, 2020.
  * 35. Minh-Phuong Huynh-Le, Tor Åge Myklebust, Christine H Feng, Roshan Karunamuni, Tom Børge Johannesen, Anders M Dale, Ole A Andreassen, and Tyler M Seibert.  Age dependence of modern clinical risk groups for localized prostate cancer—a population-based study.  Cancer, 2020.
  * 36. Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin-Yu Chen, and Yupeng Gao.  Is robustness the cost of accuracy?–a comprehensive study on the robustness of 18 deep image classification models.  In Proceedings of the European Conference on Computer Vision, 2018\.
  * 37. Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry.  Robustness may be at odds with accuracy.  In International Conference on Learning Representations, 2019.
  * 38. Qi Lei, Lingfei Wu, Pin-Yu Chen, Alexandros G Dimakis, Inderjit S Dhillon, and Michael Witbrock.  Discrete adversarial attacks and submodular optimization with applications to text classification.  In International Conference on Systems and Machine Learning, 2019\.
  * 39. Adversarial robustness - theory and practice.  https://adversarial-ml-tutorial.org.