# The Health Gym:
Synthetic Health-Related Datasets for the Development of Reinforcement
Learning Algorithms

Nicholas I-Hsien Kuo  Centre for Big Data Research in Health, University of
New South Wales, Sydney, Australia  Corresponding author: Nicholas I-Hsien Kuo
<EMAIL_ADDRESS> Mark N. Polizzotto  Australian National University,
Canberra, Australia  Simon Finfer  The George Institute for Global Health,
Sydney, Australia  University of New South Wales, Sydney, Australia  Imperial
College London, London, United Kingdom
Federico Garcia  Hospital Universitario San Cecilio, Granada, Spain  Anders
Sönnerborg  Karolinska Institutet, Stockholm, Sweden  Maurizio Zazzi
Università degli Studi di Siena, Siena, Italy  Michael Böhm
Universitätsklinikum Köln, Köln, Germany
Louisa Jorm  Centre for Big Data Research in Health, University of New South
Wales, Sydney, Australia  Sebastiano Barbieri  Centre for Big Data Research in
Health, University of New South Wales, Sydney, Australia

###### Abstract

In recent years, the machine learning research community has benefited
tremendously from the availability of openly accessible benchmark datasets.
Clinical data are usually not openly available due to their highly
confidential nature. This has hampered the development of reproducible and
generalisable machine learning applications in health care. Here we introduce
the Health Gym -– a growing collection of highly realistic synthetic medical
datasets that can be freely accessed to prototype, evaluate, and compare
machine learning algorithms, with a specific focus on reinforcement learning.
The three synthetic datasets described in this paper present patient cohorts
with acute hypotension and sepsis in the intensive care unit, and people with
human immunodeficiency virus (HIV) receiving antiretroviral therapy in
ambulatory care. The datasets were created using a novel generative
adversarial network (GAN). The distributions of variables, and correlations
between variables and trends over time in the synthetic datasets mirror those
in the real datasets. Furthermore, the risk of sensitive information
disclosure associated with the public distribution of the synthetic datasets
is estimated to be very low.

## Background & Summary

Reinforcement learning[1] (RL) is an area of artificial intelligence (AI)
which learns a behavioural policy – a mapping from states to actions – which
maximises a cumulative reward in an evolving environment. Recent studies that
combine RL with neural networks have achieved super-human performances in
tasks from video games[2] to complex board games[3]. The success of RL was
greatly facilitated by the availability of standard benchmark problems: tasks
with publicly available datasets which allowed the community to develop, test,
and compare RL algorithms (e.g., OpenAI Gym[4], DeepMind Lab[5], and D4RL[6]).
Health-related data is, however, not as easily accessible due to privacy
concerns around the disclosure of private information. To address this
challenge, this paper introduces the Health Gym project – a collection of
highly realistic synthetic medical datasets that can be freely accessed to
facilitate the development of machine learning (ML) algorithms, with a
specific focus on RL.

### Reinforcement Learning for Health Care: Promises and Challenges

Clinicians treating individuals with chronic disorders (e.g., human
immunodeficiency virus (HIV) infection) or with potentially life-threatening
conditions (e.g., sepsis) often prescribe a series of treatments to maximise
the chances of favourable outcomes. This generally requires modifying the
duration, dosage, or type of treatment over time; and is challenging due to
patient heterogeneity in responses to treatments, potential relapses, and
side-effects. Clinicians often rely, at least in part, on clinical judgment to
assign sequences of treatment, because the clinical evidence base is
incomplete and available evidence may not represent the diversity of real-life
clinical states. There is thus vast potential for RL algorithms to optimise
personalised treatment regimens, as shown by early research on antiretroviral
therapy in HIV[7], radiotherapy planning in lung cancer[8], and management of
sepsis[9]. Nonetheless, some authors have highlighted the lack of
reproducibility and potential for patient harm inherent in these methods[10].
In particular, recommendations made by RL algorithms may not be safe if the
training data omit variables that influence clinical decision making, or if
the effective sample size is small[11].

One of the main difficulties in developing robust RL algorithms for healthcare
is the highly confidential nature of clinical data. Researchers are often
required to establish formal collaborations and execute extensive data use
agreements before sharing data. One approach to overcome these barriers is to
generate synthetic data that closely resembles the original dataset but does
not allow re-identification of individual patients and can therefore be freely
distributed. Synthetic data generation has previously been applied to computed
tomography images[12] and electronic health records[13]; and early studies
found that both linear[14] and non-linear[15] models could generate continuous
and categorical variables. More recently, deep learning techniques such as
Generative Adversarial Networks[16] (GANs) have also been used to generate
realistic medical time series[17].

### The Health Gym Project

The Health Gym project is a growing collection of synthetic but realistic
datasets for developing RL algorithms. Here we introduce the first three
datasets related to the management of acute hypotension[18], sepsis[9], and
HIV[19]. All datasets were generated using GANs and the MIMIC-III[20] and
EuResist[21] databases. MIMIC-III comprises health-related data for patients
who stayed in intensive care units (ICUs) of the Beth Israel Deaconess Medical
Centre (Boston, USA) between 2001 and 2012. Within MIMIC-III, we identified
two cohorts of patients: $3,910$ patients with acute hypotension and $2,164$
patients with sepsis. Similarly, the EuResist Integrated Database was used to
extract longitudinal information related to $8,916$ people with HIV. For
severe hypotension and sepsis, we extracted the related timeseries of vital
signs, laboratory test results, medications (e.g., administered intravenous
fluids and vasopressors), and demographics. For people with HIV, we included
demographics and time series of antiretroviral medications, cluster of
differentiation 4+ T-lymphocytes (CD4) count and viral load measurements.

Both MIMIC-III and EuResist contain only de-identified data (i.e., personal
identifiers have been removed and other treatments such as date shifting
applied to minimise disclosure risk); however, there is a small remaining risk
that personal information may be disclosed if an “attacker” or “adversary” (a
person or process seeking to learn sensitive information about an individual)
is able to link our published data back to personal identifiers. To minimise
this risk, we evaluated the synthetic data using current best practices[22] in
terms of both membership disclosure (i.e., that no record in the synthetic
data can be mapped directly to a record in the real data) and attribute
disclosure (i.e., that even if part of the data are known to an attacker, the
remaining attributes cannot be recovered exactly). In Appendix A, we provide a
broader impact statement to discuss the implementations and applications of
our work.

## Methods

In this work, we applied GANs to longitudinal data extracted from the MIMIC-
III[20] and EuResist[21] databases to generate three synthetic datasets. The
inclusion and exclusion criteria used to define the patient cohorts were
adapted from previous studies: Gottesman et al.[18] for defining the patient
cohort with acute hypotension, Komorowski et al.[9] to define the sepsis
cohort, and Parbhoo et al.[19] to define the HIV cohort. Our synthetic
datasets thus include variables that can be used to define the observations,
actions, and rewards associated with RL problems for the management of these
clinical conditions.

In order to describe our data generation procedure, we start by describing the
real datasets and then provide details on our neural network design for
generating the synthetic datasets. Our synthetic datasets include all
variables in their real counterparts, in the identical formats, and are
described in the Data Records section.

### The Real Datasets

The set of variables contained in each dataset is reported below. We refer
interested readers to previous studies for the descriptive statistics (i.e.,
quantiles and mean values) of the real datasetsiiiIt is out of the scope of
this paper to address the descriptive statistics of the real datasets.
Instead, we suggest interested readers to refer to Feng et al.[23] for such
details on the MIMIC-III dataset; and likewise, refer to Oette et al.[24] for
those in EuResist..

#### Acute Hypotension

The real dataset for the management of acute hypotension was originally
proposed in the work of Gottesman et al.[18]. It was derived from MIMIC-III
and contains the following clinical variables, measured over a 48-hour time
period in 3,910 patients:

  * •

mean arterial pressure (MAP), diastolic and systolic blood pressures (BPs);

  * •

laboratory results of Alanine and Aspartate Aminotransferase (ALT and AST),
lactate, and serum creatinine;

  * •

mechanical ventilation parameters such as partial pressure of oxygen (PaO2)
and fraction of inspired oxygen (FiO2);

  * •

Glasgow Coma Scale (GCS) score;

  * •

administered fluid boluses and vasopressors; and

  * •

urine outputs.

Further details are reported in Table 1. Data were aggregated for every hour
in the time series; there are hence 48 data points per variable for each
patient. Data missingness in clinical time series is usually highly
informative, indicating e.g., the need for specific laboratory tests. Hence
the real dataset includes variables with suffix (M) to indicate whether a
variable was measured at a specific point in time.

In their work, Gottesman et al. used this dataset to develop an RL agent which
suggested the optimal amounts of fluid boluses and vasopressors for the
management of acute hypotension. Notably, they binned both fluid boluses and
vasopressors into multiple categories for the RL agent to make decisions in a
discrete action space. Appendix B.1 contains technical details for deriving
this real dataset.

#### Sepsis

The real sepsis dataset constructed by Komorowski et al.[9] was also derived
from MIMIC-III. It is more complex than the real hypotension dataset and
comprises 44 variables, including vital signs, laboratory results, mechanical
ventilation information, and various patient measurements. The complete list
of variables is reported in Tables 2 and 3.

The real sepsis dataset contains time series data for 2,164 patients. However,
the duration of hospital stay varies for each patient. The shortest record is
8 hours long and the longest record lasts 80 hours. Furthermore, the data are
reported in 4-hour windows; hence, the shortest patient record contains 2 data
points, whereas the longest contains 20 data points. Appendix B.2 describes
the technical details for deriving the real sepsis dataset.

In their paper, Komorowski et al. employed an RL agent to prescribe different
doses of intravenous fluids and vasopressors based on a patient’s clinical
variables. Their RL agent was trained by assigning rewards depending on
whether the patients transitioned to a more favourable health state following
the actions taken.

We purposely left out some variables from the work of Komorowski et al. in our
real sepsis dataset. Namely, we did not include the four items of FiO2 ratio
(P/F ratio), shock index, sequential organ failure assessment score (SOFA
score), and systemic inflammatory response syndrome criteria (SIRS criteria).
These items were excluded because they can easily be derived from the other
variables that are included. In Appendix B.2.2, we provide further information
on deriving these auxiliary variables.

#### HIV

Our real HIV dataset is based on the study of Parbhoo et al.[19]. In their
paper, Parbhoo et al. extracted a cohort of people with HIV from the
EuResist[21] database; and proposed a mixture-of-experts approach for the
therapy selection for people with HIV. They first used kernel-based methods to
identify clusters of similar people, and then they employed an RL agent to
optimise the treatment strategy.

Although our real HIV dataset was based on their work, we made additional
changes to the real HIV dataset in order to reflect a recent guideline
published by the World Health Organisation (WHO)[25] on the standardisation of
antiretroviral therapy for HIV. We included 8,916 people from the EuResist
database who started therapy after 2015 and were treated with the top 50
medication combinations spanning 21 medications. Appendix B.3.1 provides a
discussion on the WHO guideline.

The variables in our real HIV dataset are reported in Table 4. They include
demographics, viral load (VL), CD4 counts, and regimen information. VL
reflects how much HIV virus is in a person’s body; and this variable allows
medical experts to surmise the state of infection, select appropriate
medications, and infer the effectiveness of past treatments. CD4 counts
measure how many T-cells (a type of white blood cell) are in the body; and can
be used to infer the well-being of the immune system of a person. People with
very low CD4 counts are at risk of negative health outcomes. Following the
aforementioned WHO guideline, we deconstructed each person’s medication
regimen into a collection of categorical variables representing the most
commonly used base medication combinations, as well as auxiliary medications
from different medication classesiiiiiiThe medication classes listed in Table
4 are the integrase inhibitors (INIs), non-nucleotide reverse transcriptase
inhibitors (NNRTIs), protease inhibitors (PIs), and pharmacokinetic enhancers
(pk-En). The medication classes that are not explicitly mentioned included the
nucleoside reverse transcriptase inhibitors (NRTIs) and nucleotide reverse
transcriptase inhibitors (NtRTIs). NRTIs and NtRTIs are not listed separately
in the table because they already formed the backbone of the most commonly
found medication combinations. See further discussion in Appendix B.3.3. .

Similar to the real sepsis dataset, the length of therapy in the real HIV
dataset varies across people. Thus, we truncated the records and modified
their lengths to the closest multiples of 10-month periods. Hence, the real
HIV dataset consists of people with 10, 20, 30, etc …month-long data. The
shortest patient record is 10 months long whereas the longest patient record
is 100 months long. Since each entry recorded 1 month of patient data, the
shortest record is of length 10, and the longest record is of length 100.
Similar to the hypotension dataset (Table 1), the real HIV dataset is very
sparse and thus we included binary variables with suffix (M) to indicate
whether a variable was measured at a specific time. Appendix B.3.3 reports
further details on the derivation process of the real HIV dataset.

### The Heath Gym GAN

The overarching pipeline of the Generative Adversarial Network[16] (GAN) is
shown in Figure 1(a). The setup iteratively and concurrently fine-tunes two
networks – the generator and the discriminatoriiiiiiiiiIn this paper, we
adopted the terminology given in the original GAN paper[16]. The original
paper referred to the network that classified the real and fake data as the
discriminator. However in the WGAN paper[26], it was referred as the critic;
but it was again referred as the discriminator in the WGAN-GP paper[27]. – to
create highly realistic synthetic data.

#### The GAN Setup

The process of training a GAN model can be thought as a two-player-game with
two complementary training dynamics. At first, the generator produces
synthetic data samples. Then, these synthetic data are compared with samples
of the real clinical data by the discriminator. The job of the discriminator
is to distinguish between real and synthetic data. A mathematical description
of the training procedure for the GAN is reported below. Training is concluded
when the discriminator can no longer tell the real and synthetic data apart.
That is, a generator is considered to be able to create highly realistic
synthetic data when the discriminator is guessing randomly. When the training
ends, we use the generator to create our Health Gym synthetic datasets.

For our descriptions below, we denote the generator as $G$ and the
discriminator as $D$. Furthermore, we use $X_{\text{real}}$ and
$X_{\text{syn}}$ to represent the real and synthetic datasets; and likewise,
we designate $x_{\text{real}}$ and $x_{\text{syn}}$ as real and synthetic data
batches respectively.

#### The Models

As shown in Figure 1(a), the generator $G$ creates the synthetic data based on
pseudo-random inputs $z$. The elements of $z$ are sampled from a multivariate
Gaussian distribution, and they can be considered latent variables that
describe intrinsic aspects of the clinical dataset. The task of network $G$ is
to transform a time series of latent descriptions into a set of synthetic but
realistic time series of clinical variables $G:z\rightarrow x_{\text{syn}}$.

The intermediate steps of the generator transformation are illustrated in
Figure 1(b). Since the input is a set of latent time series, we employ a
bidirectional Long Short-Term Memory [28, 29] (biLSTM) recurrent neural
network (RNN) module to interpolate the relations among the latent features
along the time dimension. The RNN is then followed by three fully connected
dense layers[30] – high dimensional non-linear transformations responsible for
feature extraction and synthetic data construction.

In order to evaluate the realisticness of the synthetic data $x_{\text{syn}}$,
we forward the synthetic data along with a batch of real data
$x_{\text{real}}$ to the discriminator $D$. As shown in Figure 1(c), the
discriminator network is also a mixture of recurrent and feedforward modules.
To facilitate training, the generator and the discriminator were designed to
have a similar number of parameters. Since the input to $D$ (both the
synthetic data and the real data) contains binary and categorical variables,
we use soft embeddings [31, 32] to represent them as numeric vectors in a
machine-readable format. The discriminator $D$ employs two fully connected
dense layers to interconnect all features among the variables of the data.
Then, it employs a biLSTM RNN to interpolate the extrated features along the
time dimension; before using a third fully connected dense layer to combine
all features to output a realisticness score. Appendix C.1 reports the
technical details on the network’s dimensionality and the variable embedding.

#### Training the GAN Model

We adopted the training objective of Wasserstein GAN with Gradient Penalty
[26, 27] (WGAN-GP) to train our GAN model. The networks were updated using

$\displaystyle\text{the discriminator loss:}\hskip
28.45274ptL_{D}=\underbrace{\mathbb{E}\left[D\big{(}G(z)\big{)}\right]-\mathbb{E}\left[D(x_{\text{real}})\right]}_{\text{Wasserstein
value
function}}+\underbrace{\lambda_{\text{GP}}\mathbb{E}\left[\left(\left\lVert\nabla_{x_{\text{syn}}}D(x_{\text{syn}})\right\rVert_{2}-1\right)^{2}\right]}_{\text{Gradient
penalty loss}}\hskip 14.22636pt\text{and}$ (1) $\displaystyle\text{the
generator loss:}\hskip 44.10176ptL_{G}=-\mathbb{E}\left[D(G(z))\right].$ (2)

The discriminator network was trained by minimising $L_{D}$; and likewise the
generator network was trained by minimising $L_{G}$.

The first two terms of Equation (1) form the Wassertein value function [33,
34] which was constructed through the Kantorovich-Rubinstein duality theorem
[35]. This required the theoretical guarantees on the smoothness of network
$D$; in practical terms, this was enforced by the gradient penalty loss term
to satisfy the Lipschitz continuity with the gradient normality of $1$.
Furthermore, the constant $\lambda_{\text{GP}}$ served as a regularisation
term that controlled the strength of the gradient penalty loss.

An intuitive interpretation of Equations (1) and (2) can be obtained by noting
that for both losses, the component $D\big{(}G(z)\big{)}$ is identical to
$D(x_{\text{syn}})$. Component $D\big{(}G(z)\big{)}$ can hence be
conceptualised as a score of the realisticness of the synthetic data. Thus,
the generated data is considered more realistic if Equation (2) is minimised.
In the discriminator loss of Equation (1), a two-player-game takes place to
make it possible to iteratively fine-tune both subnetworks. The Wasserstein
value function leverages the discriminator network $D$ as a critic function to
compare the realisticness of the synthetic data $D\big{(}G(z)\big{)}$ against
the ground truth $D(x_{\text{real}})$. While the generator $G$ is trained to
fool the discriminator $D$ by maximising the realisticness
$\mathbb{E}\left[D\big{(}G(z)\big{)}\right]$ivivivMaximising
$\mathbb{E}\left[D\big{(}G(z)\big{)}\right]$ is equivalent to minimising the
unrealisticness $-\mathbb{E}\left[D\big{(}G(z)\big{)}\right]$, of which is
Equation (2)., the discriminator $D$ is fine-tuned to maximise the difference
in the realisticness between the real data and the synthetic data
$\mathbb{E}\left[D(x_{\text{real}})\right]-\mathbb{E}\left[D\big{(}G(z)\big{)}\right]$vvvMaximising
$\mathbb{E}\left[D(x_{\text{real}})\right]-\mathbb{E}\left[D\big{(}G(z)\big{)}\right]$
is equivalent to minimising
$\mathbb{E}\left[D\big{(}G(z)\big{)}\right]-\mathbb{E}\left[D(x_{\text{real}})\right]$,
of which is the Wassertein value function in Equation (1). Maximising the
difference in the realisticness between the real and synthetic data means that
the discriminator $D$ considers the real data to be more realistic than the
synthetic data.. This allowed the discriminator to become better at
differentiating between real and synthetic data, and in turn yielded a higher
loss in Equation (2) to further fine-tune the generator $G$.

Prior studies in GANs are mostly focused on generating static images for
computer vision tasks. However, our aim for the Health Gym is to generate
contiguous time series data. That is, we are concerned with both the realistic
distributions of individual variables and the correlation among variables over
time. To ensure that correlations among variables are captured correctly by
the GAN model, we found it useful to make a slight modification to the
generator loss function of Equation (2). We augmented the vanilla generator
loss function as

$\displaystyle
L_{G}=-\mathbb{E}\left[D(G(z))\right]+\underbrace{\lambda_{\text{corr}}\sum_{i=1}^{n}\sum_{j=1}^{i-1}\left\lVert
r^{(i,j)}_{\text{syn}}-r^{(i,j)}_{\text{real}}\right\lVert_{L_{1}}}_{\text{Alignment
loss}}$ (3)

where the additional term is denoted as the alignment loss. We first calculate
the Pearson’s r correlation [36] $r^{(i,j)}$ for every unique pair of
variables $X^{(i)}$ and $X^{(j)}$; then the alignment loss is calculated as
the $L_{1}$ loss between the differences in correlations between the synthetic
data $r_{\text{syn}}$ and their real counterparts $r_{\text{real}}$.
Furthermore, $\lambda_{\text{corr}}$ is a positive constant which serves as a
weight to control the strength of the alignment loss. Appendix C.2 reports
more details on the training procedure and on the selection of hyper-
parameters.

## Data Records

All of our synthetic datasets are stored as comma separated value (CSV) files
and are accessible through the Health Gym website (see healthgym.ai). The
synthetic hypotension and sepsis datasets are currently hosted on
PhysioNet[37] – a research resource for complex physiologic signals which also
hosts the official MIMIC-III[20] database.

All synthetic datasets follow the formats of their real counterparts which we
described in The Real Datasets in Methods. This section describes specific
properties of the synthetic datasets. Quality assurance tests are reported
later in the Technical Validation section.

### The Synthetic Hypotension Dataset

The synthetic hypotension dataset is $21.7$ MB and follows the format of the
real hypotension dataset of Gottesman et al.[18] containing $3,910$ synthetic
patients. Like its real counterpart, there are $48$ data points per patient
representing time series of $48$ hours. There are hence $187,680$
(=$3,910\times 48$) records (rows) in total.

The synthetic hypotension data comprises 22 variables (columns). The first 20
variables are listed in Table 1 – there are 9 numeric variables, 4
categorical, and 7 binary variables. The 21${}^{\text{st}}$ variable contains
the synthetic patient IDs and the 22${}^{\text{nd}}$ variable indicates the
hour in the time series. The units and descriptive statistics of the clinical
variables are shown in Table 1. The descriptive statistics column shows the
first, second, and third quartiles (i.e., the 25${}^{\text{th}}$ percentile,
median, and 75${}^{\text{th}}$ percentile) for the numeric variables; and the
share, in percentage, of each unique class for the binary and categorical
variables.

The information presented in this table corresponds to the distributions of
the synthetic variables in Figure 3. Several numeric variables (e.g., urine,
serum creatinine) are right-skewed, whereas binary and categorical variables
are heavily class imbalanced. This will likely require variable transformation
for downstream machine learning applications. Interested readers may consider
our proposed pre-processing scheme in Appendix C.1.2.

### The Synthetic Sepsis Dataset

The synthetic sepsis dataset is $16$ MB and follows the format of the real
sepsis dataset of Komorowski et al.[9] containing $2,164$ synthetic patients.
The synthetic dataset is designed with 20 data points per patient representing
times series of 80 hours of data reported in 4-hour windows ($80=20\times 4$).
There are hence $43,280$ ($=2,164\times 20$) records in total.

The synthetic sepsis dataset contains 46 variables – the first 44 variables
are listed in Tables 2 and 3. Similar to the synthetic hypotension dataset,
the 45${}^{\text{th}}$ variable contains the synthetic patient IDs and the
46${}^{\text{th}}$ variable indicates the time steps in the time series. Table
2 presents the 35 numeric variables along with their units and descriptive
statistics (i.e., the first, second, and third quartiles). Table 3 lists the 3
binary variables and 6 categorical variables; together with the share, in
percentage, of each unique class. Unlike the synthetic hypotension dataset,
the sepsis dataset contains two quasi-identifiers[38], age and gender, that
may be used to disclose personal information. A disclosure risk assessment is
reported in the Technical Validation section.

The distributions of the variables in the dataset are shown in Figures 6 and
ix. We also observe several numeric variables in the sepsis dataset are right-
skewed and will likely need to be transformed before being used for downstream
machine learning applications. Interested readers may consider our proposed
pre-processing scheme in Appendix C.1.2.

There are two types of categorical variables in the sepsis dataset. GCS, for
example, is a categorical variable by design – it is a clinical point-based
system to measure a person’s level of consciousness. The 5 variables of SpO2,
Temp, PTT, PT, and INR were instead originally stored as numeric variables in
the MIMIC-III database[20]. These 5 variables were converted into categorical
variables because their original distributions were extremely skewed and it
was difficult to apply appropriate power-transformations. We decided to
categorise these 5 numeric variables into deciles; the 10 classes of each
variableviviviThe 10 categories correspond to the deciles of the original
numeric distributions of the variables in the real sepsis dataset. For
instance, category C1 corresponds to values that lie within the
0${}^{\text{th}}$ to the 0.1${}^{\text{st}}$ quantile; and that category C5
corresponds to values that lie within the 0.4${}^{\text{th}}$ to the
0.5${}^{\text{th}}$ quantile. are reported in Table 3.

### The Synthetic HIV Dataset

The synthetic HIV dataset is $44.7$ MB and is similar to the real HIV dataset
employed by Parbhoo et al.[19]. It contains $8,916$ synthetic patients
associated with time series of 60 months. The HIV data are reported in 1-month
intervals; and hence there are 60 data points per patient and $534,960$
($=8,916\times 60$) records in total.

The synthetic HIV dataset contains 15 variables – the first 13 variables are
listed in Table 4 together with descriptive statistics, and the two remaining
variables contain the synthetic patient IDs and the month in the time. There
are 3 numeric, 5 binary, and 5 categorical variables. As for the other
synthetic datasets, the descriptive statistics include the first, second, and
third quartiles for the numeric variables; and the share, in percentage, of
each unique class for the binary and categorical variables. This dataset also
contains the two quasi identifiers, gender and ethnicity, and a disclosure
risk assessment is reported in the Technical Validation section.

The distributions of the variables in the dataset are shown in Figure 10. The
numeric variables are all right-skewed and require appropriate transformation
before the dataset can be used for further analysis. Interested readers may
consider our proposed pre-processing scheme in Appendix C.1.2. Furthermore,
the variables of complementary INI, complementary NNRTI, and extra PI, all
have the option of Not Applied. This was because the medications in these
categories can be substituted by medications from the other classes. A general
discussion on medications for ART can be found in Appendix B.3.3.

## Technical Validation

This section includes a Realisticness Validation Procedure and a Disclosure
Risk Assessment. The first part demonstrates the quality of the generated
synthetic datasets; and the second part discusses the potential risk of an
adversary learning sensitive information about a real person from the
synthetic records.

Based on previous work on the validation of synthetic medical data [22], the
Realisticness Validation Procedure serves to confirm that our synthetic
datasets fulfil the fidelity of individual data points and the fidelity of the
population. That is, we first ensure that the distributions of individual
variables are sufficiently similar between the real and the synthetic
datasets. We then check that all correlations between variables and trends
over time in the real datasets are mirrored in the synthetic datasets.

In the Disclosure Risk Assessment, we show that while our synthetic datasets
are realistic, it remains very unlikely for an adversary to learn any
sensitive information about a real person using our synthetic datasets. Based
on risk metrics from the disclosure control literature[38], we will show that
our synthetic datasets have a low identity disclosure risk and a low attribute
disclosure risk. Identity disclosure refers to the scenario where an adversary
is able to match a synthetic record to a real person; and attribute disclosure
occurs when an adversary is able to learn new information about a real person,
despite the datasets being anonymised.

### Realisticness Validation Procedure

Our validation procedure goes beyond prior work[39, 40, 41, 42] that leveraged
GANs to create synthetic data and evaluated the generated data only
qualitatively. We summarised the elements of our three-stage validation
procedure in Figure 2. The first two stages analyse the static properties of
the synthetic data and assess whether the distributions and statistical
moments (mean, variance) of the real and synthetic variables are sufficiently
similar. Since our generated data are time series, the third stage conducts an
additional set of visual comparisons to test the properties of the synthetic
variables over time.

#### Stage One: Qualitative Analysis

In the first stage, we superimposed the probability density function of a
synthetic numeric variable $X_{\text{syn}}$ on top of the probability density
function of its corresponding real variable $X_{\text{real}}$. These plots
were generated using kernel density estimations (KDE) [43]. Binary and
categorical variables were compared by means of side-by-side histogram plots.

#### Stage Two: Statistical Tests

The statistical tests in stage two include the two-sample Kolmogorov-Smirnov
test[44] (KS test), the two independent Student’s t-test[45] (t-test), the
Snedecor’s F-test[46] (F-test), and the three sigma rule test[47]. The KS test
compares the overall similarity between the distributions of real and
synthetic variables. The t-test determines whether there are significant
differences between the mean values of the real and synthetic variables; and
the F-test compares their variances. Furthermore, the three sigma rule test
uses the standard deviations of the real data to check whether the majority of
the synthetic data was comprised within a probable range of the real variable
values. Definitions and implementations of each test are reported in
Appendices D.1 \- D.4.

We organised the statistical tests in a hierarchical manner. Each synthetic
variable (both numeric and categorical) was first assessed using the KS test.
The KS test is the most difficult test; and when it was passed, we concluded
that a synthetic variable faithfully represents its real counterpart. If a
synthetic numeric variable failed the KS test, we applied the t-test, the
F-test, and the three sigma rule test. If a synthetic categorical variable
failed the KS test, we assessed it further using the analysis of variance
(ANOVA) F-test and the three sigma rule test. The categorical ANOVA F-test
checks the similarity in variances but over different classes. An overview of
this procedure can be found in Algorithm 1 of Appendix D.5.

#### Stage Three: Correlations

Our third stage of validation considers correlations between variables and
between trends over time, computed using Kendall’s rank correlation
coefficients [48]. A brief description is provided below, technical details
and a discussion on alternative correlation measures can be found in Appendix
E.

First, we calculated the static correlation coefficients for each pair of
variables in the synthetic dataset $X_{\text{syn}}$ and the real dataset
$X_{\textit{real}}$ (see Appendix E.2). Next, the correlation coefficients for
the two datasets were displayed side-by-side for visual comparison. Ideally,
the synthetic dataset should mirror both the directions (positive or negative)
and magnitudes of correlations between variables in the real dataset.

Though informative, the correlation between variables does not provide any
information about whether trends over time are captured by the synthetic
dataset. Hence, we linearly decomposed each variable as a trend with
cycle[49]. The trend indicates the general upward or downward slope of
variable over time, and the cycle refers to local periodic patterns. Then, we
computed and compared the average correlation in trends and average
correlation in cycles (see Appendix E.3).

#### Validation Outcomes

Acute Hypotension
The plots for the first stage of the validation procedure for the hypotension
dataset are shown in Figure 3. There were no major visual misalignments
between the distributions of the real and synthetic datasets, and we proceeded
to stage two for statistical confirmations.

The results of stage two are shown in the hierarchically structured Table 5.
The initial KS test was passed by $17$ out of $20$ synthetic variables. The
$3$ remaining variables ALT, AST, and PaO2 passed both the t-test and the
F-test. This means that these synthetic variables do not perfectly capture the
real variable distributions; however, their means and variances are still
representative of their real counterparts. These observations are supported by
the subplots in Figure 3: despite some differences between the real and
synthetic data, the overall behaviours are appropriately captured.
Furthermore, all of these $3$ variables pass the three sigma rule test. Hence
we conclude that all synthetic variables capture the features of the real
variable distributions. Appendix D.6.1 contains the complete statistical
results.

After confirming the realisticness of the individual synthetic variables, we
assessed the relations between variables and their longitudinal properties in
the third validation stage. We illustrate the static correlations in Figure 4
and the dynamic correlations in Figure 5. There is no major misalignment
between the static correlations of the real and synthetic datasets. However,
the synthetic dataset slightly increases the magnitudes of some correlations.
For instance, there is a stronger positive correlation between lactic acid and
AST in the synthetic dataset than in the real counterpart. Likewise, there is
a stronger negative correlation between the synthetic variables of serum
creatinine and urine than in the real pair of variables. Nonetheless, the
generated data is still highly reliable. In Figure 5, the dynamic correlations
(both in trends and in cycles) of the decomposed synthetic time series
strongly resemble their real counterparts. This indicates that the
characteristics of the generated time series variables are realistic. All
three stages of our validation confirmed that the synthetic hypotension
dataset adequately characterises the properties of the real dataset.

Sepsis
For the synthetic sepsis dataset, we observe in Figures 6 and ix that all
synthetic variable distributions were very similar to their real counterparts.
In stage two, we found that $43$ out of $44$ variables passed the KS test and
therefore almost all synthetic variables mirrored the distributions of their
real counterparts. The only variable that failed the KS test was Max Vaso.
Since the variable also failed the following F-test, this was because of the
differences in the variance (see Table 6). As shown in Figure ix, Max Vaso is
highly skewed. As discussed in the Data Records section, we could have
transformed Max Vaso into a categorical variable but decided to keep it as
numeric because the closely related variables Input Total, Input 4H, Output
Total, and Output 4H are all numeric. These 5 variables collectively describe
the input/output measurements of the patients and should therefore share one
common data type. Nonetheless, Max Vaso did pass the three sigma rule test.
This indicated that while there was a difference in variance for the synthetic
Max Vaso variable, the generated data were within the plausible range of the
real data. The complete results of all statistical tests are reported in
Appendix D.6.2.

The correlations computed in stage three of the validation procedure are
visualised in Figures 8 and 9 for a subset of the 20 variables that were
associated with the strongest correlations. Both the static and the dynamic
correlations were very similar between the real and synthetic dataset. Figures
13 – 15 in Appendix F show the full correlation matrices for all variable
pairs.

HIV
Qualitative comparisons between the distributions of the real and synthetic
HIV datasets are shown in Figure 10, indicating high similarity. As presented
in Table 7, $12$ out of $13$ variables passed the KS test, suggesting that the
distributions of most synthetic variables matched their real counterparts. The
only variable that failed the KS test is VL. VL also failed the F-test,
similarly to Max Vaso in the synthetic sepsis dataset. However, VL still
passed the three sigma rule test and therefore we can conclude that all
variables in the synthetic dataset are highly realistic. Appendix D.6.3
contains the complete statistical results.

For stage three, we present the correlations in Figures 11 and 12. Both the
static and dynamic correlations reflect that the synthetic dataset captures
the relations among the variables in the real dataset.

### Disclosure Risk Assessment

We performed two tests to evaluate the likelihood of an attacker learning
sensitive information about an individual from the generated synthetic
datasets.

#### Euclidean Distances

The first test was to ensure that no records in the real datasets were simply
copied by the GAN to the synthetic datasets. We computed the Euclidean
distances ($L_{2}$ norms) between records in the real dataset
$X_{\text{real}}$ and records in the synthetic dataset $X_{\text{syn}}$. We
verified that all distances were greater than zero, i.e., that no records in
the synthetic datasets perfectly matched any records in the real datasets.

#### Disclosure Risks

The second test concerned the disclosure risks associated with the public
distribution of the synthetic datasets. Despite being anonymised, the data may
contain sets of variables (e.g., age and gender) which, in combination, may be
used by an adversary to uniquely identify a person (e.g., via linking the data
with voter registration lists[50]). Variables which in combination constitute
personally identifying information are known as quasi-identifiers. Individuals
with the same combination of quasi-identifiers (e.g., all 21-year-old males)
form an equivalent class.

El Emam et al.[38] introduced two types of disclosure risks based on the
concepts of quasi-identifiers and equivalent classes. Depending on the
direction of attack[51], an adversary may attempt to learn new information
about a person either by finding out whether an individual in the population
(or database) is also included in the real or synthetic dataset (population-
to-sample attack) or by linking an individual in the real or synthetic dataset
back to the original database (sample-to-population attack).

Whereas El Emam et al.. assume that the real dataset is sampled randomly from
the database, in this study the real datasets were constructed using publicly
accessible inclusion and exclusion criteria. Therefore, we assumed that the
adversary had access not only to the database (e.g., MIMIC-III or EuResist)
but also to the real dataset. One of the likely reasons to conduct a
population-to-sample attack is to determine whether an individual has a
specific condition or illness that led to their inclusion in the dataset. When
the inclusion criteria are known, population-to-sample attacks become less
relevant than sample-to-population attacks, which may be used to learn
additional sensitive information about an individual in the synthetic dataset.

The risk of a successful synthetic-to-real attack (i.e., the chance of
matching a random individual in the synthetic dataset to an individual in the
real dataset) can be computed as

$\displaystyle\frac{1}{S}\sum_{s=1}^{S}\left(\frac{1}{F_{s}}\times
I_{s}\right)$ (4)

where $S$ is the number of records in the synthetic dataset, $F_{s}$ is the
size of the equivalent class in the real dataset that shares the same
combination of quasi-identifiers as a specific record $s$ in the synthetic
sample, and $I_{s}$ is a binary indicator variable equal to one if at least
one real record matches the synthetic records $s$. Interested readers can find
more descriptions on El Emam et al.’s metric in Appendix G.1.

To assess the risk of information disclosure, we adopted the acceptable risk
threshold valueviiviiviiIn their work, El Emam et al.[38] used $5\%$ instead.
See page 9 under the section of Risk Assessment Parameters in the referenced
work. of $9\%$ proposed by the European Medicines Agency[52] and Health
Canada[53] for the public release of clinical trial data. Some alternative
disclosure risk metrics are discussed in Appendix G.2.

#### Risk Assessment Outcomes

Acute Hypotension
As shown in Table 1, all variables in the synthetic hypotension dataset are
associated with the patient’s bio-physiological states and do not contain any
quasi-identifiers or sensitive information. For this reason, for this dataset
we tested the Euclidean distances but not the disclosure risk.

No records in the synthetic dataset completely matched any records in the real
hypotension dataset. The smallest distance between any synthetic record and
any real record was $49.06$ ($>0$). Therefore no record was leaked into the
synthetic dataset.

Sepsis
Through the Euclidean distance test, we found that no record in the synthetic
sepsis dataset was identical to any record in the real sepsis dataset. The
smallest distance between real and synthetic records was $328.78$ ($>0$),
which was considerably larger than the smallest distance for the hypotension
data ($49.06$). This is likely due to the larger number of variables in the
sepsis dataset (44 vs 20 in the hypotension dataset, compare Table 1 with
Tables 2 and 3). Furthermore, many sepsis variables are highly skewed (see
Figure ix) hence exaggerating any value differences. Importantly, for both the
synthetic hypotension dataset and the synthetic sepsis dataset, the minimal
Euclidean distance is greater than zero.

The sepsis variables include the quasi-identifiers age and gender. Therefore,
age (rounded down to the closest year) and gender were combined to create
different equivalence classes e.g., all 21-year-old males and all 22-year-old
makes were in separate equivalence classes. The risk of a successful
synthetic-to-real attack was estimated to be $0.80\%$. This risk is much lower
than the suggested threshold of $9\%$[52, 53], indicating that there is
minimal risk of sensitive information disclosure associated with the release
of the synthetic sepsis dataset.

HIV
The minimal Euclidean distance between any pair of real and synthetic HIV
records was $0.11$ ($>0$); and hence no data leaked from the real dataset into
the synthetic dataset. This value is relatively low for two reasons: 1) there
are very few variables in the HIV dataset; 2) most variables are either binary
or categorical. The reasons that inflate the Euclidean distance for sepsis are
thus the same reasons that deflate the Euclidean distance for HIV.

The HIV variables include the quasi-identifiers gender and ethnicity. These
two variables were combined to create different equivalence classes (e.g.,
male Asian and female Caucasian). The risk of a successful synthetic-to-real
attack was estimated to be $0.041\%$. This risk is again much lower than the
typical $9\%$ threshold, indicating that also the synthetic HIV dataset can be
released with minimal risk of sensitive information disclosure.

## Code availability

The software codes related to the Health Gym project is publicly available at
https://github.com/Nic5472K/ScientificData2021_HealthGym.

## Figures & Tables

Variable Name | Data Type | Unit | Descriptive Statistics
---|---|---|---
Mean Arterial Pressure (MAP) | numeric | mmHg | Median: $65.34$ (Q1: $59.30$, Q3: $71.19$)
Diastolic Blood Pressure (BP) | numeric | mmHg | Median: $54.33$ (Q1: $48.37$, Q3: $60.26$)
Systolic BP | numeric | mmHg | Median: $113.21$ (Q1: $104.23$, Q3: $121.60$)
Urine | numeric | mL | Median: $106.21$ (Q1: $68.92$, Q3: $164.23$)
Alanine Aminotransferase (ALT) | numeric | IU/L | Median: $32.55$ (Q1: $24.59$, Q3: $46.09$)
Aspartate Aminotransferase (AST) | numeric | IU/L | Median: $46.82$ (Q1: $35.81$, Q3: $67.75$)
Partial Pressure of Oxygen (PaO2) | numeric | mmHg | Median: $103.02$ (Q1: $91.34$, Q3: $114.66$)
Lactate | numeric | mmol/L | Median: $1.50$ (Q1: $1.29$, Q3: $1.80$)
Serum Creatinine | numeric | mg/dL | Median: $1.11$ (Q1: $0.83$, Q3: $1.62$)
Fluid Boluses | categorical | mL | 4 Classes
|  |  |  $[0,250):97.32\%$; $[250,500):0.28\%$
|  |  |  $[500,1000):1.46\%$; $\geq 1000:0.94\%$
Vasopressors | categorical | mcg/kg/min | 4 Classes
|  |  |  $0:84.14\%$; $(0,8.4):8.34\%$
|  |  |  $[8.4,20.28):3.68\%$; $\geq 20.28:3.83\%$
Fraction of Inspired Oxygen (FiO2) | categorical | fraction | 10 Classes
|  |  |  $\leq 0.2:0.00\%$; $0.2:0.54\%$
|  |  |  $0.3:2.84\%$; $0.4:10.85\%$
|  |  |  $0.5:63.30\%$; $0.6:8.58\%$
|  |  |  $0.7:1.32\%$; $0.8:0.20\%$
|  |  |  $0.9:2.63\%$; $1.0:9.75\%$
Glasgow Coma Scale Score (GCS) | categorical | point | 13 Classes
|  |  |  $3:6.61\%$ $4:2.16\%$ $5:0.00\%$ $6:3.00\%$
|  |  |  $7:4.77\%$ $8:0.00\%$ $9:2.22\%$ $10:4.32\%$
|  |  |  $11:2.46\%$ $12:3.56\%$ $13:1.00\%$
|  |  |  $14:9.80\%$ $15:60.09\%$
Urine Data Measured (Urine (M)) | binary | \- - | False: $63.07\%$ True: $36.93\%$
ALT or AST Data Measured (ALT/AST (M)) | binary | \- - | False: $98.50\%$ True: $1.50\%$
FiO2 (M) | binary | \- - | False: $92.49\%$ True: $7.51\%$
GCS (M) | binary | \- - | False: $81.49\%$ True: $18.51\%$
PaO2 (M) | binary | \- - | False: $97.56\%$ True: $2.44\%$
Lactic Acid (M) | binary | \- - | False: $96.98\%$ True: $3.02\%$
Serum Creatinine (M) | binary | \- - | False: $95.26\%$ True: $4.74\%$
Table 1: Variables in the Acute Hypotension Dataset
This table presents the variables shared by the real and synthetic datasets
for the management of acute hypotension. The data types are colour-coded with
cyan for numeric, magenta for categorical, and brown for binary. Those
variables with suffix (M) indicate whether a data point has been measured
(which is usually highly informative in medical time series). The descriptive
statistics in this table are only for the synthetic
dataset${}^{\text{\ref{FN:StatisticsForRealDatasets}}}$. For the numeric
variables, we list the median as well as the first and third quantiles (Q1 and
Q3). As for the categorical and binary variables, we report the share of each
unique class in the synthetic dataset. The information in this table should be
compared with the illustrations of Figure 3.

Variable Name | Data Type | Unit | Descriptive Statistics
---|---|---|---
|  |  | Median | Q1 | Q3
Age | numeric | year | $65.40$ | $58.29$ | $72.95$
Heart Rate (HR) | numeric | bpm | $89.09$ | $78.46$ | $99.82$
Systolic BP | numeric | mmHg | $123.67$ | $114.43$ | $133.03$
Mean BP | numeric | mmHg | $81.02$ | $75.18$ | $86.91$
Diastolic BP | numeric | mmHg | $58.90$ | $50.40$ | $66.95$
Respiratory Rate (RR) | numeric | bpm | $21.46$ | $18.69$ | $24.28$
Potassium (K+) | numeric | meq/L | $4.12$ | $3.78$ | $4.45$
Sodium (Na+) | numeric | meq/L | $140.01$ | $136.59$ | $143.57$
Chloride (Cl-) | numeric | meq/L | $105.23$ | $102.08$ | $108.03$
Calcium (Ca++) | numeric | mg/dL | $8.02$ | $7.37$ | $8.66$
Ionised Ca++ | numeric | mg/dL | $1.11$ | $1.04$ | $1.18$
Carbon Dioxide (CO2) | numeric | meq/L | $25.27$ | $23.44$ | $27.29$
Albumin | numeric | g/dL | $3.01$ | $2.68$ | $3.32$
Hemoglobin (Hb) | numeric | g/dL | $10.20$ | $9.17$ | $11.23$
Potential of Hydrogen (pH) | numeric | \- - | $7.39$ | $7.34$ | $7.44$
Arterial Base Excess (BE) | numeric | meq/L | $0.16$ | $-2.04$ | $2.48$
Bicarbonate (HCO3) | numeric | meq/L | $24.38$ | $22.63$ | $26.13$
FiO2 | numeric | fraction | $0.45$ | $0.38$ | $0.55$
Glucose | numeric | mg/dL | $134.11$ | $108.21$ | $167.06$
Blood Urea Nitrogen (BUN) | numeric | mg/dL | $25.38$ | $19.89$ | $31.92$
Creatinine | numeric | mg/dL | $1.13$ | $0.90$ | $1.44$
Magnesium (Mg++) | numeric | mg/dL | $2.04$ | $1.83$ | $2.29$
Serum Glutamic Oxaloacetic Transaminase (SGOT) | numeric | u/L | $50.78$ | $31.53$ | $88.97$
Serum Glutamic Pyruvic Transaminase (SGPT) | numeric | u/L | $39.99$ | $26.20$ | $65.66$
Total Bilirubin (Total Bili) | numeric | mg/dL | $1.19$ | $0.66$ | $2.32$
White Blood Cell Count (WBC) | numeric | E9/L | $10.60$ | $7.99$ | $13.92$
Platelets Count (Platelets) | numeric | E9/L | $184.44$ | $141.97$ | $239.41$
PaO2 | numeric | mmHg | $109.07$ | $84.22$ | $139.63$
Partial Pressure of CO2 (PaCO2) | numeric | mmHg | $39.32$ | $34.92$ | $44.97$
Lactate | numeric | mmol/L | $1.82$ | $1.41$ | $2.40$
Total Volume of Intravenous Fluids (Input Total) | numeric | mL | $4867.46$ | $1887.84$ | $11155.76$
Intravenous Fluids of Each 4-Hour Period (Input 4H) | numeric | mL | $58.66$ | $13.83$ | $229.01$
Maximum Dose of Vasopressors in 4H (Max Vaso) | numeric | mcg/kg/min | $0.0002$ | $0.0$ | $0.0017$
Total Volume of Urine Output (Output Total) | numeric | mL | $2505.54$ | $585.47$ | $6733.69$
Urine Output in 4H (Output 4H) | numeric | mL | $159.33$ | $44.74$ | $361.69$
Table 2: Numeric Variables in the Sepsis Dataset
The format of this table follows that of Table 1; and with more results in
Table 3. Only the first three columns are shared by both the real and
synthetic sepsis datasets. The remaining columns show the descriptive
statistics that are specific for the synthetic dataset. The content in this
table should be compared with the illustrations in Figures 6 and ix.

Variable Name | Data Type | Unit | Descriptive Statistics
---|---|---|---
Gender | binary | \- - | Male: $73.41\%$ True: $26.59\%$
Readmission of Patient (Readmission) | binary | \- - | False: $60.20\%$ True: $39.80\%$
Mechanical Ventilation (Mech) | binary | \- - | False: $56.89\%$ True: $43.11\%$
GCS | categorical | point | 13 Classes
|  |  |  $3:8.71\%$ $4:0.38\%$ $5:0.50\%$ $6:6.30\%$
|  |  |  $7:0.74\%$ $8:2.27\%$ $9:1.52\%$ $10:9.31\%$
|  |  |  $11:9.12\%$ $12:6.31\%$ $13:2.53\%$
|  |  |  $14:15.45\%$ $15:36.85\%$
Pulse Oximetry Saturation (SpO2) | categorical | $\%$ | 10 Classes (C)
|  |  | C1: $[0.00,93.83):13.38\%$; C2: $[93.83,95.14):8.12\%$;
|  |  | C3: $[95.14,96.00):4.48\%$; C4: $[96.00,96.70):10.64\%$;
|  |  | C5: $[96.70,97.33):12.61\%$; C6: $[97.33,98.00):11.36\%$;
|  |  | C7: $[98.00,98.60):11.52\%$; C8: $[98.60,99.22):11.84\%$;
|  |  | C9: $[99.22,99.86):8.39\%$; C10: $[99.86,100.0]:7.66\%$;
Temperature (Temp) | categorical | Celsius | 10 Classes (C)
|  |  | C1: $[15.11,35.95):7.83\%$; C2: $[35.95,36.28):6.55\%$;
|  |  | C3: $[36.28,36.50):12.87\%$; C4: $[36.50,36.69):16.56\%$;
|  |  | C5: $[36.69,36.88):4.21\%$; C6: $[36.88,37.06):8.21\%$;
|  |  | C7: $[37.06,37.28):7.10\%$; C8: $[37.28,37.56):9.37\%$;
|  |  | C9: $[37.56,37.93):10.96\%$; C10: $[37.93,40.52]:16.33\%$;
Partial Thromboplastin Time (PTT) | categorical | s | 10 Classes (C)
|  |  | C1: $[17.80,24.53):7.69\%$; C2: $[24.53,26.63):6.71\%$;
|  |  | C3: $[26.63,28.20):10.02\%$; C4: $[28.20,29.60):12.44\%$;
|  |  | C5: $[29.60,31.45):5.46\%$; C6: $[31.45,34.00):9.27\%$;
|  |  | C7: $[34.00,37.10):9.99\%$; C8: $[37.10,42.80):11.47\%$;
|  |  | C9: $[42.80,57.90):12.38\%$; C10: $[57.90,150.00]:14.58\%$;
Prothrombin Time (PT) | categorical | s | 10 Classes (C)
|  |  | C1: $[9.90,12.20):7.89\%$; C2: $[12.20,12.90):8.2\%$;
|  |  | C3: $[12.90,13.30):11.02\%$; C4: $[13.30,13.80):9.84\%$;
|  |  | C5: $[13.80,14.30):9.45\%$; C6: $[14.30,14.90):6.59\%$;
|  |  | C7: $[14.90,15.90):10.37\%$; C8: $[15.90,17.51):10.51\%$;
|  |  | C9: $[17.51,22.00):13.27\%$; C10: $[22.00,146.70]:12.85\%$;
International Normalised Ratio (INR) | categorical | \- - | 10 Classes (C)
|  |  | C1: $[0.00,1.00):0.19\%$; C2: $[1.00,1.10):8.88\%$;
|  |  | C3: $[1.10,1.20):23.35\%$; C4: $[2.21,17.60]:0.09\%$
|  |  | C5: $[1.20,1.30):15.64\%$; C6: $[1.30,1.31):10.22\%$;
|  |  | C7: $[1.31,1.50):7.53\%$; C8: $[1.50,1.70):9.71\%$;
|  |  | C9: $[1.70,2.21):10.67\%$; C10: $[2.21,17.60]:13.70\%$;
Table 3: Non-Numeric Variables in the Sepsis Dataset
The format of this table follows that of Table 1; and it is a continuation of
Table 2.

Variable Name | Data Type | Unit | Descriptive Statistics
---|---|---|---
Viral Load (VL) | numeric | copies/mL | Median: $54.77$ (Q1: $16.51$, Q3: $209.03$)
Absolute Count for CD4 (CD4) | numeric | cells/$\mu$L | Median: $465.81$ (Q1: $279.26$, Q3: $840.34$)
Relative Count for CD4 (Rel CD4) | numeric | cells/$\mu$L | Median: $25.57$ (Q1: $18.20$, Q3: $35.72$)
Gender | binary | \- - | Male: $93.42\%$ Female: $6.58\%$
Ethnicity | categorical | \- - | 4 Classes
|  |  | Asian: $0.47\%$; African: $2.55\%$
|  |  | Caucasian: $26.81\%$; Other: $70.17\%$
Base Drug Combination | categorical | \- - | 6 Classes
(Base Drug Combo) |  |  | FTC + TDF: $73.66\%$; 3TC + ABC $14.08\%$
|  |  | FTC + TAF: $0.98\%$
|  |  | DRV + FTC + TDF: $5.50\%$
|  |  | FTC + RTVB + TDF: $2.30\%$
|  |  | Other: $3.47\%$
Complementary INI | categorical | \- - | 4 Classes
(Comp. INI) |  |  | DTG: $11.96\%$; RAL: $0.49\%$
|  |  | EVG: $4.69\%$; Not Applied: $82.86\%$
Complementary NNRTI | categorical | \- - | 4 Classes
(Comp. NNRTI) |  |  | NVP: $0.19\%$; EFV: $9.27\%$
|  |  | RPV: $43.76\%$; Not Applied: $46.78\%$
Extra PI | categorical | \- - | 6 Classes
|  |  | DRV: $0.69\%$ RTVB: $4.02\%$
|  |  | LPV: $1.08\%$ RTV: $2.02\%$
|  |  | ATV: $4.26\%$ Not Applied: $87.92\%$
Extra pk Enhancer (Extra pk-En) | binary | \- - | False: $96.70\%$ True: $3.30\%$
VL Measured (VL (M)) | binary | \- - | False: $79.35\%$ True: $20.65\%$
CD4 (M) | binary | \- - | False: $83.39\%$ True: $16.61\%$
Drug Recorded (Drug (M)) | binary | \- - | False: $15.56\%$ True: $84.44\%$
Table 4: Variables in the HIV Dataset
This table presents the variables shared by the real and synthetic datasets
for antiretroviral therapy in HIV. The format of this table follows that of
Table 1. Only the first three columns are shared by both the real and
synthetic HIV datasets. The last column shows the descriptive statistics that
are specific to the synthetic dataset. The content in this table should be
compared with the illustrations in Figure 10.

(a) An Overview

(b) The Generator Network

(c) The Discriminator Network Figure 1: The GAN Pipeline for the Health Gym
Project
The overview of the GAN pipeline is shown in (a). It conjointly trains a
generator network which synthesises data, and a discriminator network which
aims to classify the data as either real or fake. In (b), we show that the
generator consists of one biLSTM layer followed by three fully conneted dense
layers. Whereas in (c), the discriminator first embeds non-numeric data, then
it passes all input to two fully connected layers, a biLSTM layer, then
another fully connected layer.

Figure 2: A Summary of the Realisticness Validation Procedure
The validation includes three stages. First, we perform a qualitative analysis
which compares the distributions of real and synthetic variables. Next, we
perform a series of statistical tests to assess whether the generated data
captured the real data distribution. As a final step, we validate whether the
synthetic data captured the correlations between variables over time.

Figure 3: Distribution Plots for Acute Hypotension
This figure presents visual comparisons between the distributions of variables
in the real and synthetic datasets for the management of acute hypotension.
The distributions of real variables are plotted in orange and their synthetic
counterparts in blue.

Passed the KS Test | MAP, Diastolic BP, Systolic BP, Serum Creatinine, Fluid Boluses, Vasopressors,
---|---
| FiO2, GCS, Urine, Lactic Acid, Urine (M), ALT/AST (M), FiO2 (M), GCS (M),
| PaO2 (M), Lactic Acid (M), Serum Creatinine (M)
Failed the KS Test | Variable Name | t-Test Status | F-Test Status
| ALT | ✓ | ✓
| AST | ✓ | ✓
| PaO2 | ✓ | ✓
The Three Sigma Rule Test | passed | ALT, AST, PaO2
| failed | \- -
Table 5: The Stage Two Validation Results for Acute Hypotension
This table summarises the results of the statistical tests. The tests were
conducted in the order of the KS-test, then the t-test and F-test, and finally
the three sigma rule test. Only those variables that failed the KS-test
underwent the additional tests. $17$ of the $20$ variables in the synthetic
hypotension dataset passed the KS-test and did not have different
distributions from their real counterparts. The remaining $3$ variables passed
all additional tests. Therefore, all variables of the synthetic hypotension
dataset were realistic. This table should be compared with Figure 3 and Table
1.

Figure 4: The Static Correlations for Acute Hypotension
This is a side-by-side comparison of the static correlations in the synthetic
dataset and the real dataset. It illustrates the correlation between all pairs
of variables, across all patients and timepoints. Positive correlations are
coloured in red and negative correlations are in blue. The magnitudes of the
correlations are indicated by colour saturation.

(a) The Average Correlations in Trends

(b) The Average Correlations in Cycles Figure 5: The Dynamic Correlations for
Acute Hypotension
This is a side-by-side comparison of the dynamic correlations in the synthetic
dataset and the real dataset. Unlike the static correlations of Figure 4, all
variables are treated as time series and are linearly decomposed into trends
and cycles. They illustrate the average correlation between all pairs of
variables for each individual patient. Refer to Figure 4 for details on the
colour scheme.

Figure 6: Distribution Plots for Sepsis
This figure presents visual comparisons between the distributions of variables
in the real and synthetic datasets for the management of sepsis. It is
continued in Figure ix.

Figure 7: Distribution Plots for Sepsis (continued)
This figure serves as a continuation to Figure 6. All variables are strictly
positive but may appear t include negative values as an artefact of using
kernel density estimation for plotting the distributionsixixixAlso see the
related discussion on Anon. Stack Exchange. (2014).
Retrieved from https://stats.stackexchange.com/questions/109549/negative-
density-for-non-negative-variables.

Passed the KS Test | Age, HR, Systolic BP, Mean BP, Diastolic BP, RR, K, Na, Cl-, Ca,
---|---
| Ionised Ca, CO2, Albumin, HB, pH, BE, HCO3, FiO2, Glucose, BUN, Creatinine,
| Mg, SGOT, SGPT, Total Bili, WBC, Platelets, PaO2,PaCO2, Lactate,
| Input Total, Input 4H, Output Total, Output 4H, Gender, Readmission, Mech,
GCS,
| SpO2, Temp, PTT, PT, INR
Failed the KS Test | Variable Name | t-Test Status | F-Test Status
| Max Vaso | ✓ | $\times$
The Three Sigma Rule Test | passed | Max Vaso
| failed | \- -
Table 6: The Stage Two Validation Results for Sepsis
This table presents the statistical results for the synthetic sepsis dataset.
It follows the format of Table 5; and should be compared with Figures 6 and ix
and Tables 2 and 3.

Figure 8: The Top 20 Static Correlations for Sepsis
This figure presents the static correlations between a subset of the variables
in the sepsis dataset. It follows the format of Figure 4; the full correlation
plots for all variables are shown in Figure 13.

(a) The Top 20 Average Correlations in Trends

(b) The Top 20 Average Correlations in Cycles Figure 9: The Top 20 Dynamic
Correlations for Sepsis
This figure presents the dynamic correlations between a subset of the
variables in the sepsis dataset. It follows the format of Figure 5; the full
correlation plots for all variables are shown in Figures 14 and 15.

Figure 10: Distribution Plots for HIV
This figure presents visual comparisons between the distributions of variables
in the real and synthetic datasets for the optimisation of antiretroviral
therapy for HIV.

Passed the KS Test | CD4, Rel CD4, Gender, Ethnic, Base Drug Combo, Comp. INI, Comp. NNRTI, Extra PI
---|---
| Extra pk-En, VL (M), CD4 (M), Drug (M)
Failed the KS Test | Variable Name | t-Test Status | F-Test Status
| VL | ✓ | $\times$
The Three Sigma Rule Test | passed | VL
| failed | \- -
Table 7: The Stage Two Validation Results for HIV
This table presents the statistical results for the synthetic HIV dataset. It
follows the format of Table 5; and should be compared with Figure 10 and Table
4.

Figure 11: The Static Correlations for HIV
This figure presents the static correlations between the variables in the HIV
dataset. It follows the format of Figure 4.

(a) The Average Correlations in Trends

(b) The Average Correlations in Cycles Figure 12: The Dynamic Correlations for
HIV
This figure presents the dynamic correlations between the variables in the HIV
dataset. It follows the format of Figure 5.

## References

  * [1] Sutton, R. S. & Barto, A. G.  _Reinforcement Learning: An Introduction_ (MIT press, 2018).
  * [2] Mnih, V. _et al._ Playing Atari with Deep Reinforcement Learning.  In _arXiv preprint arXiv:1312.5602_ (2013).
  * [3] Silver, D. _et al._ Mastering the Game of Go with Deep Neural Networks and Tree search.  In _Nature_ (2016).
  * [4] Brockman, G. _et al._ OpenAI Gym.  In _arXiv preprint arXiv:1606.01540_ (2016).
  * [5] Beattie, C. _et al._ DeepMind Lab.  In _arXiv preprint arXiv:1612.03801_ (2016).
  * [6] Fu, J., Kumar, A., Nachum, O., Tucker, G. & Levine, S.  D4RL: Datasets for Deep Data-Driven Reinforcement Learning.  In _arXiv preprint arXiv:2004.07219_ (2020).
  * [7] Yu, C., Dong, Y., Liu, J. & Ren, G.  Incorporating Causal Factors into Reinforcement Learning for Dynamic Treatment Regimes in HIV.  In _BMC Medical Informatics and Decision Making_ (2019).
  * [8] Tseng, H.-H. _et al._ Deep Reinforcement Learning for Automated Radiation Adaptation in Lung Cancer.  In _Medical physics_ (2017).
  * [9] Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C. & Faisal, A. A.  The Artificial Intelligence Clinician Learns Optimal Treatment Strategies for Sepsis in Intensive Care.  In _Nature Medicine_ (2018).
  * [10] Challen, R. _et al._ Artificial Intelligence, Bias and, Clinical Safety.  In _BMJ Quality & Safety_ (2019).
  * [11] Gottesman, O. _et al._ Guidelines for Reinforcement Learning in Healthcare.  In _Nature Medicine_ (2019).
  * [12] Kim, J. _et al._ Implementation of a Novel Algorithm for Generating Synthetic CT images from Magnetic Resonance Imaging Data Sets for Prostate Cancer Radiation Therapy.  In _the International Journal of Radiation Oncology* Biology* Physics_ (2015).
  * [13] Walonoski, J. _et al._ Synthea: An Approach, Method, and Software Mechanism for Generating Synthetic Patients and the Synthetic Electronic Health Care Record.  In _the Journal of the American Medical Informatics Association_ (2018).
  * [14] Fienberg, S. E. & Steele, R. J.  Disclosure Limitation using Perturbation and Related Methods for Categorical Data.  In _the Journal of Official Statistics_ (1998).
  * [15] Caiola, G. & Reiter, J. P.  Random Forests for Generating Partially Synthetic, Categorical Data.  In _the Transactions on Data Privacy_ (2010).
  * [16] Goodfellow, I. _et al._ Generative Adversarial Nets.  In _the Advances in Neural Information Processing Systems_ (2014).
  * [17] Esteban, C., Hyland, S. L. & Rätsch, G.  Real-Valued (Medical) Time Series Generation with Recurrent Conditional GANs.  In _arXiv preprint arXiv:1706.02633_ (2017).
  * [18] Gottesman, O. _et al._ Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions.  In _the International Conference on Machine Learning_ (2020).
  * [19] Parbhoo, S., Bogojeska, J., Zazzi, M., Roth, V. & Doshi-Velez, F.  Combining Kernel and Model Based Learning for HIV Therapy Selection.  In _the American Medical Informatics Association Summits on Translational Science Proceedings_ (2017).
  * [20] Johnson, A. E. _et al._ MIMIC-III, A Freely Accessible Critical Care Database.  In _Scientific Data_ (2016).
  * [21] Zazzi, M. _et al._ Predicting Response to Antiretroviral Treatment by Machine Learning: The EuResist Project.  In _Intervirology_ (2012).
  * [22] Goncalves, A. _et al._ Generation and Evaluation of Synthetic Patient Data.  In _BMC Medical Research Methodology_ (2020).
  * [23] Feng, M. _et al._ Transthoracic Echocardiography and Mortality in Sepsis: Analysis of the MIMIC-III Database.  In _Intensive Care Medicine_ (2018).
  * [24] Oette, M. _et al._ Efficacy of Antiretroviral Therapy Switch in HIV-infected Patients: A 10-year Analysis of the EuResist Cohort.  In _Intervirology_ (2012).
  * [25] World Health Organisation.  Consolidated Guidelines on the Use of Antiretroviral Drugs for Treating and Preventing HIV Infection: Recommendations for a Public Health Approach (2016).
  * [26] Arjovsky, M., Chintala, S. & Bottou, L.  Wasserstein Generative Adversarial Networks.  In _the International Conference on Machine Learning_ (2017).
  * [27] Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V. & Courville, A. C.  Improved Training of Wasserstein GANs.  In _the Advances in Neural Information Processing Systems_ (2017).
  * [28] Hochreiter, S. & Schmidhuber, J.  Long Short-Term Memory.  In _Neural Computation_ (1997).
  * [29] Graves, A., Fernández, S. & Schmidhuber, J.  Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition.  In _the International Conference on Artificial Neural Networks_ (2005).
  * [30] Rumelhart, D. E., Hinton, G. E. & Williams, R. J.  Learning Representations by Back-Propagating Errors.  In _Nature_ (1986).
  * [31] Landauer, T. K., Foltz, P. W. & Laham, D.  An Introduction to Latent Semantic Analysis.  In _Discourse Processes_ (1998).
  * [32] Mottini, A., Lheritier, A. & Acuna-Agost, R.  Airline Passenger Name Record Generation using Generative Adversarial Networks.  In _arXiv preprint arXiv:1807.06657_ (2018).
  * [33] Mallows, C. L.  A Note on Asymptotic Joint Normality.  In _the Annals of Mathematical Statistics_ (1972).
  * [34] Levina, E. & Bickel, P.  The Earth Mover’s Distance is the Mallows Distance: Some Insights from Statistics.  In _the IEEE International Conference on Computer Vision_ (2001).
  * [35] Villani, C.  Optimal Transport: Old and New.  In _Grundlehren der mathematischen Wissenschaften_ (2008).
  * [36] Mukaka, M. M.  A Guide to Appropriate Use of Correlation Coefficient in Medical Research.  In _the Malawi Medical Journal_ (2012).
  * [37] Kuo, N. I. _et al._ Synthetic Acute Hypotension and Sepsis Datasets Based on MIMIC-III and Published as Part of the Health Gym Project.  In _arXiv preprint arXiv:2112.03914_ (2021).
  * [38] El Emam, K., Mosquera, L. & Bass, J.  Evaluating Identity Disclosure Risk in Fully Synthetic Health Data: Model Development and Validation.  In _the Journal of Medical Internet Research_ (2020).
  * [39] Mirza, M. & Osindero, S.  Conditional Generative Adversarial Nets.  In _arXiv preprint arXiv:1411.1784_ (2014).
  * [40] Reed, S. _et al._ Generative Adversarial Text to Image Synthesis.  In _the International Conference on Machine Learning_ (2016).
  * [41] Choi, E. _et al._ Generating Multi-Label Discrete Patient Records using Generative Adversarial Networks.  In _the Machine Learning for Healthcare Conference_ (2017).
  * [42] Zhang, Y. _et al._ Adversarial Feature Matching for Text Generation.  In _the International Conference on Machine Learning_ (2017).
  * [43] Davis, R. A., Lii, K.-S. & Politis, D. N.  Remarks on Some Nonparametric Estimates of a Density Function.  In _the Selected Works of Murray Rosenblatt_ (2011).
  * [44] Hodges, J. L.  The Significance Probability of the Smirnov Two-Sample Test.  In _the Arkiv för Matematik_ (1958).
  * [45] Yuen, K. K.  The Two-Sample Trimmed t for Unequal Population Variances.  In _Biometrika_ (1974).
  * [46] Snedecor, G. W. & Cochran, W. G.  Statistical Methods.  In _the Iowa State University Press_ (1989).
  * [47] Pukelsheim, F.  The Three Sigma Rule.  In _the American Statistician_ (1994).
  * [48] Kendall, M. G.  The Treatment of Ties in Ranking Problems.  In _Biometrika_ (1945).
  * [49] Hyndman, R. J. & Athanasopoulos, G.  _Forecasting: Principles and Practice_ (OTexts, 2018).
  * [50] Benitez, K. & Malin, B.  Evaluating Re-Identification Risks with Respect to the HIPAA Privacy Rule.  In _the Journal of the American Medical Informatics Association_ (2010).
  * [51] Elliot, M. & Dale, A.  Scenarios of Attack: The Data Intruder’s Perspective on Statistical Disclosure Risk.  In _Netherlands Official Statistics_ (1999).
  * [52] European Medicines Agency.  European Medicines Agency Policy on Publication of Clinical Data for Medicinal Products for Human Use (2014).
  * [53] Health Canada.  Guidance document on Public Release of Clinical Information (2014).
  * [54] Li, J., Cairns, B. J., Li, J. & Zhu, T.  Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applications.  In _arXiv_ (2021).
  * [55] Kuo, N. I. _et al._ An Input Residual Connection for Simplifying Gated Recurrent Neural Networks.  In _the International Joint Conference on Neural Networks_ (2020).
  * [56] Dhariwal, P. & Nichol, A.  Diffusion Models Beat GANs on Image Synthesis.  In _the Advances in Neural Information Processing Systems_ (2021).
  * [57] Yang, M. _et al._ CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models.  In _the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2021).
  * [58] KAWASAKI, Z., SHIBATA, K. & TAJIMA, M.  A Guide to the SQL Standard: A User’s Guide to the Standard Database Language SQL A Guide to the SQL Standard: A User’s Guide to the Standard Database Language SQL, 1997.  In _the IEICE transactions on information and systems_ (2003).
  * [59] Teasdale, G. & Jennett, B.  Assessment of Coma and Impaired Consciousness: A Practical Scale.  In _the Lancet_ (1974).
  * [60] Singer, M. _et al._ The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).  In _JAMA_ (2016).
  * [61] MATLAB.  _version 7.10.0 (R2010a)_ (The MathWorks Inc., Natick, Massachusetts, 2010).
  * [62] Group, I. S. S.  Initiation of Antiretroviral Therapy in Early Asymptomatic HIV Infection.  In _the New England Journal of Medicine_ (2015).
  * [63] Daniluk, M., Rocktäschel, T., Welbl, J. & Riedel, S.  Frustratingly Short Attention Spans in Neural Language Modelling.  In _the International Conference on Learning Representations_ (2019).
  * [64] Box, G. E. & Cox, D. R.  An Analysis of Transformations.  In _the Journal of the Royal Statistical Society: Series B (Methodological)_ (1964).
  * [65] Van Rossum, G.  Python Tutorial.  In _Centrum voor Wiskunde en Informatica_ (1995).
  * [66] Virtanen, P. _et al._ SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.  In _Nature Methods_ (2020).
  * [67] Pedregosa, F. _et al._ Scikit-learn: Machine Learning in Python.  In _the Journal of Machine Learning Research_ (2011).
  * [68] Kingma, D. P. & Ba, J.  Adam: A Method for Stochastic Optimization.  In _the International Conference on Learning Representations_ (2015).
  * [69] Bengio, Y., Louradour, J., Collobert, R. & Weston, J.  Curriculum learning.  In _the International Conference on Machine Learning_ (2009).
  * [70] Press, O., Bar, A., Bogin, B., Berant, J. & Wolf, L.  Language Generation with Recurrent Generative Adversarial Networks without Pre-training.  In _arXiv preprint arXiv:1706.01399_ (2017).
  * [71] Kolmogorov, A.  Sulla Determinazione Empirica di una Lgge di Distribuzione.  In _Inst. Ital. Attuari, Giorn._ (1933).
  * [72] Smirnov, N.  Table for Estimating the Goodness of Fit of Empirical Distributions.  In _the Annals of Mathematical Statistics_ (1948).
  * [73] “Student” Gosset, W. S.  The Probable Error of a Mean.  In _Biometrika_ (1908).
  * [74] Johnson, N. L., Kotz, S. & Balakrishnan, N.  _Continuous Univariate Distributions_ (John Wiley & Sons, 1995).
  * [75] McKinney, W. _et al._ Data Structures for Statistical Computing in Python.  In _the Proceedings of Python in Science Conference_ (2010).
  * [76] Kowalski, C. J.  On the Effects of Non-Normality on the Distribution of the Sample Product-Moment Correlation Coefficient.  In _the Journal of the Royal Statistical Society: Series C (Applied Statistics)_ (1972).
  * [77] Bracewell, R. N. & Bracewell, R. N.  _The Fourier Transform and Its Applications_ (McGraw-Hill New York, 1986).
  * [78] Mann, H. B. & Whitney, D. R.  On a Test of Whether One of Two Random Variables is Stochastically Larger than the Other.  In _the Annals of Mathematical Statistics_ (1947).
  * [79] Woo, M.-J., Reiter, J. P., Oganian, A. & Karr, A. F.  Global Measures of Data Utility for Microdata Masked for Disclosure Limitation.  In _the Journal of Privacy and Confidentiality_ (2009).
  * [80] Kullback, S. & Leibler, R. A.  On Information and Sufficiency.  In _the Annals of Mathematical Statistics_ (1951).
  * [81] El Emam, K. & Malin, B.  Concepts and Methods for De-Identifying Clinical Trial Data.  In _the Committee on Strategies for Responsible Sharing of Clinical Trial Data_ (2014).
  * [82] De Maesschalck, R., Jouan-Rimbaud, D. & Massart, D. L.  The Mahalanobis Distance.  In _Chemometrics and Intelligent Laboratory Systems_ (2000).
  * [83] Samarati, P.  Protecting Respondents Identities in Microdata Release.  In _the IEEE Transactions on Knowledge and Data Engineering_ (IEEE, 2001).
  * [84] Machanavajjhala, A., Kifer, D., Gehrke, J. & Venkitasubramaniam, M.  l-Diversity: Privacy Beyond k-Anonymity.  In _the ACM Transactions on Knowledge Discovery from Data_ (2007).
  * [85] Li, N., Li, T. & Venkatasubramanian, S.  t-Closeness: Privacy Beyond k-Anonymity and l-Diversity.  In _the International Conference on Data Engineering_ (2007).

## Acknowledgements

We would like to thank Sarah Garland for her contribution to accessing
EuResist data and synthetic data validation. This study benefited from data
provided by EuResist Network EIDB; and this project has been funded by a
Wellcome Trust Open Research Fund (reference number 219691/Z/19/Z).

## Author Contributions Statement

N.K. and S.B. designed, implemented and validated the deep learning models
used to generate the synthetic datasets. L.J. contributed to the design of the
study and provided expertise regarding the risk of sensitive information
disclosure. M.P. provided clinical expertise on antiretroviral therapy for
HIV. S.F. provided clinical expertise on sepsis. F.G., A.S., M.Z., and M.B.
contributed patient data as part of the EuResist Integrated Database.
Furthermore, N.K. wrote the manuscript and S.B. designed the study. All
authors contributed to the interpretation of findings and manuscript
revisions.

## Competing Interests

The authors declare no competing interests.

## The Appendices for the Health Gym Paper

In this study, we presented a novel approach for creating synthetic healthcare
datasets (related to acute hypotension[11], sepsis[9], and HIV[19, 25]) using
the machine learning algorithm of GAN[16]. We carried out several statistical
tests to verify the realisticness of the synthetic datasets and employed
disclosure control techniques[38] to ensure that the public release of these
datasets is associated with very low risk of sensitive information disclosure.
This appendix provides further technical details and discusses some of the
concepts which were presented only briefly in the main body of the paper.

Table of Contents
A: Future Work and Broader Impact
B: Instructions on Deriving the Real Datasets
B.1: Acute Hypotension Dataset
B.2: Sepsis Dataset
B.3: HIV Dataset
C: The Training Details of GANs
C.1: The Dimensionality in the GAN Setup
C.2: Remarks on Training the GAN Model
D: The Statistical Tests of Stage 2
D.1: The Two Sample KS Test
D.2: The Two Independent Sample Student’s t-Test
D.3: The Snedecor’s F-Test
D.4: The Three Sigma Rule Test
D.5: Iteratively Executing the Statistical Tests
D.6: The Statistical Outcomes
E: The Correlations of Stage 3
E.1: Kendall’s Rank Correlation
E.2: Correlation Between Variables
E.3: Average Correlation in Trends and in Cycles
E.4: Remarks on Kendall’s Rank Correlation and Alternative Validation Metrics
F: Full Correlation Plots for Sepsis
G: Assessment of Disclosure Risk
G.1: El Emam et al.’s Disclosure Risk Metrics
G.2: Alternative Disclosure Risk Metrics

##  Appendix A Future Work and Broader Impact

Clinical data is highly confidential and therefore difficult to share between
researchers, which has hampered the development of robust machine learning
algorithms in health care. In the Health Gym project we have shown that
generative models could be used to create highly-realistic, longitudinal,
datasets with mixed numerical and categorical variables. The risk of sensitive
information disclosure associated with the release of these datasets was
estimated to be very low. We aim to add further health-related datasets to the
Health Gym in the near future and hope that this project will foster the
development of reproducible and generalisable machine learning, and in
particular reinforcement learning, applications in health care.

We are aware of only one other GAN-based model which attempted to create both
numeric and non-numeric variables at the same time. Li et al.[54] proposed a
twin-encoder approach to separately embed numeric and non-numeric data, which
required adding a matching loss for training the generative model. In
contrast, the GAN model proposed in this study requires only one encoder and
one decoder, since binary and categorical variables are mapped to continuous
vectors through the use of soft-embeddings. Nonetheless, the recurrent
components of the Health Gym GAN could potentially benefit from existing work
on network simplification[55]. Diffusion models are an alternative approach
for generating synthetic data, and they have recently achieved results
comparable to state-of-the-art GANs[56]. In future work, we plan to explore
the use of diffusion models for improving the sample diversity and robustness
of the training process of our generative models. Furthermore, the generated
data could be made more realistic, and the generative process more
explainable, by incorporating causal layers[57].

The authors of this manuscript would like to emphasise that the generated
synthetic datasets should not be regarded as replacements for the real
datasets. The synthetic datasets are realistic, but it is currently unknown
whether training a machine learning model with synthetic data leads to the
same performance using the real data. We intend to conduct more investigations
in this direction as part of future work.

##  Appendix B Instructions on Deriving the Real Datasets

This appendix aims to clarify the details for preparing The Real Datasets
described in the Methods section. Interested readers should also refer to the
official website of MIMIC-III[20] at
https://physionet.org/content/mimiciii/1.4/; and likewise consult the official
website of EuResist[21] at https://www.euresist.org/.

Our readers can click here to resume to the The Real Datasets in the Methods
section.

###  B.1 Acute Hypotension Dataset

We mainly followed the instructions provided in Appendix D on page 15 of
Gottesman et al.[18]. Although the authors published their codes in an open
repository (see https://github.com/dtak/interpretable_ope_public), they did
not release the SQL[58] code used to query the raw data from the MIMIC-III
database. We derived a larger real hypotension dataset which consisted of a
cohort of $3,910$ patients; there were $1,733$ distinct ICU admissions in
Gottesman et al.’s original cohort of patients.

####  B.1.1 Inclusion/Exclusion Criteria

Gottesman et al. provided the selection criteria for their patient cohort in
Appendix D.1 on page 15 of their paper. In brief, the authors included adult
patients ($\geq 18$ years old) in the MICU MIMIC-III sub-dataset with those
with at least 24 hours of data collected from the MetaVision (MV) clinical
information system. Then, they aggregated 48 hours of clinical variables from
those patients with seven or more mean arterial pressure (MAP) values of $65$
mmHg or less, which indicated probable acute hypotension.

####  B.1.2 Data Preparation

Almost all of the variables in Table 1, except the Glasgow coma scale (GCS)
score, are originally recorded as continuous numeric variables in the MIMIC-
III database. GCS is a point-based system to reflect a patient’s state of
consciousness[59]. Fluid boluses, vasopressors, and fraction of inspired
oxygen (FiO2) were treated as categorical variables, for reasons described
below.

As documented in Appendix D.2 on page 15 of Gottesman et al., most variables
were queried as-is from the MIMIC-III database. However, total vasopressor
measurements were derived from the administered fluids following the code
provided by Komorowski et al.[9] (refer to Vasopressors from MetaVision in the
script AIClinician_Data_extract_MIMIC3_140219.ipynb in
https://gitlab.doc.ic.ac.uk/AIClinician/AIClinician).

After following the previous steps, we converted some variables from numeric
to categorical as according to Appendix D.4 on page 16 of Gottesman et al.. In
this section, Gottesman et al. explained that they categorised fluid boluses
and vasopressors for training their RL algorithm in a discretised action
space. Fluid boluses were binned into one of the four categories of none, low,
medium, and high reflecting the numeric ranges of
$0,[250,500),[500,1000),\text{and}\geq 1000$, in mL units. Whereas for
vasopressor, they binned the variable according to its first and third
quantiles (i.e., Q1 and Q3) and categorised vasopressor into the four classes
of $0,(0,8.1),[8.1,21.58),\text{and}\geq 21.58$ in the unit of total mcg of
drug given each hour per kg body weight. However, our cohort of patients was
slightly different to theirs and hence we binned vasopressors according to our
re-calculated Q1 and Q3 values. Our vasopressor variable was thus categorised
according to the ranges of $0,(0,8.4),[8.4,20.28),\text{and}\geq 20.28$,
respectively.

Another variable which we made categorical was FiO2. Since FiO2 was the
fraction of inspired oxygen, we binned the variable into $10$ classes
according to the ranges of $[0,0.1),[0.1,0.2),\ldots,[0.8,0.9),\text{and
}[0.9,1.0]$.

####  B.1.3 Missing Values

As commented in Appendix D.5 on page 16, Gottesman et al. noted that the
MIMIC-III database contains missing values. However, these missing values do
not occur at random and can be highly informative. For instance, updated
laboratory test are often ordered when a patient is transferred from one ICU
to another. To deal with the missing values, we followed Gottesman et al. and
we constructed indicator variables (the binary (M) variables of Table 1) that
denoted whether or not a value was measured. Missing values were then replaced
with the last available recorded data.

###  B.2 Sepsis Dataset

We mainly followed the steps in Komorowski et al.[9]’s repository
https://gitlab.doc.ic.ac.uk/AIClinician/AIClinician for preparing the real
sepsis dataset. They queried the MIMIC-III database with their SQL file
AIClinician_Data_extract_MIMIC3_140219.ipynb and processed the queired data
with their codes in
AIClinician_sepsis3_def_160219.m.

####  B.2.1 Inclusion/Exclusion Criteria

Komorowski et al.. did not explicitly state the inclusion/exclusion criteria
for the sepsis cohort in their paper. However, the processing code indicates
that they only included adult patients. Then for all adults, they sought for
any suspicious infections based on the history of antibiotic administrations;
and they included all clinical variables from at least $44$ hours before the
suspected infection and up to $28$ hours after the infection. The use of such
time period for defining a sepsis event is supported by a 2016 sepsis
definition paper[60] which states that one should “consider[s] a period as
great as 48 hours before and up to 24 hours after onset of infection…”;
Komorowski et al.. likely relaxed this condition with a -4/+4 window.

####  B.2.2 Data Preparation

The data preparation procedure for the real sepsis dataset was simple, thanks
to the code provided by Komorowski et al..in their repository. The file
containing the SQL queries should be executed first, followed by the
processing code. Further details are provided below.

We executed the entire file containing the SQL queries. The sepsis dataset
contained a much larger subset of data from MIMIC-III than the hypotension
dataset. Whereas the hypotension dataset only included patients from the MICU
branch, the sepsis dataset included patients from the NICU, SICU, CSRU, CCU,
MICU, and TSICU branches. Furthermore, while the hypotension dataset only
considered those patients who had their information recorded by the MetaVision
(MV) system, the sepsis dataset also included those patients whose data were
collected by the CareVue (CV) system.

After executing the file containing the SQL queries, we executed lines 1 to
791 of the processing code. This was the part which we used to prepare all
variables listed in Tables 2 and 3. We did not execute the remaining lines of
that code. This was because those lines were for deriving the secondary
auxiliary items of P/F ratio, shock index, SOFA score, and SIRS criteria. For
instance, the P/F ratio is computed by dividing PaO2 by FiO2, whereas the
shock index is based on HR and systolic BP. All variables required to compute
these secondary items are included in the dataset (see Tables 2 and 3).

####  B.2.3 Missing Values

Whereas Gottesman et al. chose to fill the missing values in the hypotension
dataset by last observation carried forward, Komorowski et al. chose to impute
the missing values. In line 739 of the procesing code, they mainly used
Matlab’s[61] built-in knnimpute function and their own code in fixgaps.m. They
do not appear to have used the code contained in fastknnsearch.m.

###  B.3 HIV Dataset

We aimed to create a real HIV dataset from the EuResist[21] database with a
similar purpose to the one that was presented in the work of Parbhoo et
al.[19]. However, unlike the work of Gottesman et al. (for hypotension) and
Komorowski et al. (for sepsis), Parbhoo et al. did not make their codes
publicly available and thus we mainly consulted with HIV medical experts to
create our own version of the real HIV dataset.

####  B.3.1 WHO Guidelines

As previously mentioned in the Methods section, we based our real HIV dataset
on the work of Parbhoo et al. but we also incorporated a published WHO
guideline[25] which called for the standardisation of antiretroviral therapy
for HIV. Chapter 4 “Clinical Guidelines: Antiretroviral Therapy”, starting
from page 71 in the referenced work, also accessible at
https://www.who.int/hiv/pub/arv/chapter4.pdf.

The referenced WHO guideline contains an extensive discussion on the strategic
timing of antiretroviral treatment. Prior to the WHO guideline, there was no
standardised approach for initiating antiretroviral therapy in HIV.
Antiretroviral therapy (ART) medications could be initialised when a person’s
CD4 count dropped either below 500 cells/$\mu$L (i.e., moderate) or below 350
cells/$\mu$L (i.e., critical). However, some studies have observed that early
ART initiation increases the overall survival rate of people with HIV.
Nonetheless, close monitoring in the first 3 months following ART initiation
is required because people can develop hypersensitivity towards these
medications, with potentially severe consequences.

More evidence on the benefits of the early adoption of ART medications can be
found in a randomised controlled trial by the INSIGHT START Study Group[62].
In this longitudinal study over 4 years, the intervention group received
immediate-initiation of ART, whereas the control group deferred-initiation.
The median time until initiation in the control group was 3 years. The
immediate-initiation group had either equivalent or better survival rates
across a variety of AIDS-related and non-AIDS-related events (see Table 2 on
page 7 in the referenced paper).

####  B.3.2 Inclusion/Exclusion Criteria

Parbhoo et al. extracted their cohort of patients from the entire EuResist
database. They included people who were treated with the 312 most common
medication combinations (including 20 medications). They also included patient
demographics; however, their code is not publicly available and it is unclear
which individual medications and demographic information were included in
their dataset.

In order to create a similar dataset while incorporating information from the
2015 WHO guidelines[25], we restricted our cohort of patients to people
initiated ART after 2015. Among the included population, we further selected
those who were treated with the 50 most common medication combinations
(including 21 medications). This choice is discussed further in the Data
Preparation subsection below.

####  B.3.3 Data Preparation

Class Imbalance
There were a total of 11,252 people in the EuResist database who initiated ART
after 2015. Among these people, there were 20 who identified as neither female
nor male (i.e., less than $0.02\%$ of the total population). During our
experiments, we found that our GAN model, consistently over more than 50
random initialisations, was unable to create datasets containing patient
information related to this minority group. We therefore decided to exclude
this group of people from our cohort; however, we recognise this as a
limitation of our model that will need to be addressed by future algorithmic
improvements.

Ethnicities in the EuResist database were also considerably imbalanced. They
were originally categorised as Asian, African, Caucasian, Hispanic, Other, and
Unknown. Since there were a very low number of 179 Hispanic people, we decided
to combine Hispanic, Other, and Unknown into Other.

Persistent Low Standardisation in HIV ART
In Sections B.3.1 and B.3.2, we mentioned how our inclusion criteria were
based on the publication of the WHO guidelines in 2015. Indeed, we observed
several changes in medication regimens over time. Before 2015 and especially
in the early 2000s, it was not uncommon to see a regimen composed of more than
10 medications. However, after 2015, almost all regimens contained less than 6
medications; with most regimens consisting of only 3 or 4 medications.

Despite the continuous changes, there is still a lack of standardisation in
the practice of HIV ART therapy. For instance, there were a total of 1,179
different medication combinations prescribed post-2015; and this number
greatly exceeded the 14 suggested first line treatment combinations for adults
in the WHO guideline (see the first three rows of Table 4.1 on page 97 of the
referenced work[25]). Upon closer analysis, we found that most medication
combinations were only prescribed once, likely to suit just one patient’s
need. Moreover, we also found that the 10 most common medication combinations,
which included a small subset of only 14 different medications, made up
34.1$\%$ of all prescription records. Thus, our aim was to extract a real HIV
dataset which preserved the diversity in medication combinations while
limiting an over-representation of single-use medication combinations.

Ultimately, we decided to include only the 50 most common medication
combinations. For the 11,252 people in EuResist who initiated ART post-2015,
there were 22,622 records (i.e., rows in the database) in total. Including
only the 50 most common medication combinations resulted in 17,660 records
(78.1$\%$ of the total). Furthermore, the 50 most common medication
combinations included 21 individual medications and made up 51.2$\%$ of all
medication combinations prescribed post-2015.

Handling Long Gaps
Within our selected cohort, the shortest records were ten months long while
the longest lasted for more than three years. However, a large proportion of
people had long gaps in their records. This was most apparent in the EuResist
regimen information table – someone who had just finished one therapy
treatment would usually not be immediately prescribed with another treatment.
We found that over a third of the people had one gap of over six months in
their regimen treatment regimen; and that more than half of the remaining
people had multiple (i.e., two or three) treatment gaps in their records.

The use of data imputation techniques (such as splines) to fill these long
gaps was unlikely to yield meaningful results. Instead, we decided to split
the original records into several shorter sub-records describing continuous
period of ART. Furthermore, to facilitate training of our GAN model, we
truncated the sub-records’ lengths to the closest multiple of ten (e.g., 32
months were truncated at 30 months, and 51 months were truncated at 50
months). As shown in Figure 1, both our generator and discriminator included
RNNs; and it has been shown previously[63] that RNNs are good for processing
mid-length data but not necessarily well-suited for very long data.

Missing Values
Any other missing data values were filled in using last observation carried
forwards,as discussed for the hypotension dataset in Section B.1.3.

Decomposing Patient Regimens
After managing the gaps and missing values in the data, we investigated
different ways to represent the ART treatment in a meaningful manner. As
mentioned in the Persistent Low Standardisation in HIV ART subsection, our
cohort were treated with 50 medication combinations including 21 individual
medications. The medications belonged to five different
classes${}^{\text{\ref{FN:DClasses}}}$: nucleoside reverse transcriptase
inhibitors (NRTIs), nucleotide reverse transcriptase inhibitors (NtRTIs), non-
nucleotide reverse transcriptase inhibitors (NNRTIs), integrase inhibitor
(INI), and protease inhibitors (PIs). In addition to the medications belonging
to the previous classes, it was common among people in the EuResist database
to be prescribed pharmacokinetic enhancers (pk-En) to boost the effectiveness
of the other medications.

ART medications tend to be associated with strong positive inter-class and
strong negative intra-class correlations. For instance, if the INI medication
of dolutegravir (DTG) was selected for a person’s therapy, it was highly
unlikely that another INI such as raltegravir (RAL) was required for the same
therapy regimen. Contrarily, it is unnecessary to select an INI whenever an
NNRTI was already present in the regimen (compare Table 4.1 on page 97 in the
WHO guideline[25]).

We found that if we tried to represent each medication as a binary (or
categorical) variable, the combination of strong positive and negative
correlations would lead our GAN model into creating under-diversified
synthetic datasets. For instance, our generator sub-network could have heavily
favoured the coexistence of emtricitabine (FTC), an NRTI, with DTG, an INI;
and thus became unable to learn any association between any other pair of
NRTIs and INIs.

To address this problem, we decided to represent medication regimens using two
categorical variables. The first variable includes base drug combination,
while the second variable includes other auxiliary and secondary medications.
For instance, a person could be treated with FTC + TDF \+ DTG + pk-En. Since
all NRTI and NtRTI medications were included in the base medication
combination variable, it was unnecessary to create separate categorical
variables representing these classes.

##  Appendix C The Training Details of GANs

This appendix aims to provide more technical details on the training of The
Health Gym GAN introduced in the Methods section. The codes for this paper are
also made publicly available; see the Data Records section for more details.

Summary of the GAN dimensionality: Input dimension $128$, hidden dimension
$128$, output dimension is dependent on the dataset. The embedded dimension of
numeric data is $1$; and it is $2$ for binary data and $4$ for categorical
data.

Our readers can click here to resume to the The Health Gym GAN in the Methods
section.

###  C.1 The Dimensionality in the GAN Setup

As mentioned previously, the GAN setting included a generator $G$ and a
discriminator $D$. The generator was trained to map vectors of multivariate
Gaussian inputs to the synthetic data $G:z\rightarrow x_{\text{syn}}$; whereas
the discriminator took a batch of either real data $x_{\text{real}}$ or
synthetic data $x_{\text{syn}}$, and scored the realisticness of the data
$D:x\rightarrow\mathbb{R}$. The important dimensionalities in this setup were
thus associated with the variables of $z$, $x_{\text{syn}}$ and
$x_{\text{real}}$, within the networks of $G$ and $D$, and the outputs of the
two networks.

####  C.1.1 The Input and Hidden Dimensions of the Generator

The dimensionality of the multivariate latent vectors $z$ was
$\mathbb{R}^{\mathscr{I}}$ per synthetic patient per instance. That is, in
order to generate a synthetic patient record over $T$ units of time, we would
need a matrix of $z^{(1:T)}\in\mathbb{R}^{\mathscr{I}\times T}$; and we
referred to $\mathscr{I}$ as the input dimension. Both $\mathscr{I}$ and $T$
were hyper-parameters and hence were selected prior to the training phase of
the GAN model. For this work, we chose $\mathscr{I}=128$ for all datasets.
However, the values for $T$ varied across different datasets. This was because
whereas the real hypotension data had 48 time points for all patients; the
real sepsis data had non-uniform length for each patient – the shortest record
was 2 time points while the longest was 20 time points. Likewise, the shortest
real HIV data had 10 time points while the longest was of length 100. To
generate synthetic datasets that closely represented their real counterparts,
we selected $T=48$ for hypotension, $T=20$ for sepsis, and $T=60$ for HIV.

Each instance of $z^{(1:T)}$ was forwarded to the generator network $G$ one at
a time (recall that the first module in the generator is biLSTM). All modules
in network $G$ had a dimensionality of $\mathscr{H}$. The mapping learnt by
network $G$ was hence
$\mathbb{R}^{\mathscr{I}}\rightarrow\mathbb{R}^{\mathscr{H}}$. We referred to
$\mathscr{H}$ as the hidden dimension; it was a hyper-parameter and we
selected it to be $\mathscr{H}=128$ across all datasets.

####  C.1.2 The Output Dimension of the Generator and the Contents of
$X_{\text{syn}}$ and $X_{\text{real}}$

The output of the generator was also the input of the discriminator. Hence the
output dimension of the generator $\mathscr{O}$ not only described the amount
of features in the synthetic data $X_{\text{syn}}$, but it was also identical
to the input dimension of the discriminator network. Since the synthetic data
mimicked the real data $X_{\text{real}}$, the dimensionality of $\mathscr{O}$
varied across different datasets because each real dataset comprised a
different number of clinical variables. As reported in the Methods section,
there were $20$ variables for acute hypotension, $44$ for sepsis, and $13$ for
HIV. However, the output dimensions were not $20$, $44$, and $13$.

The reason for this was the presence of various data types in our datasets.
There were continuous numeric variables (e.g., systolic BP), binary variables
(e.g., gender), and also categorical variables (e.g., ethnicity). Whereas the
numeric variables could be forwarded directly as float values to the
discriminator, binary and categorical variables required additional
transformations to present them in a machine-readable format to the
discriminator.

Pre-processing Variables in $X_{\text{real}}$
For hypotension, we first transformed each real numeric variable according to
their optimal Box-Cox [64] transformation using the stats.boxcox function in
the Python [65] package of Scipy [66]. Afterwards, we re-centred and re-scaled
the variables within the range of $[0,1]$ with the preprocessing.MinMaxScaler
scaler of the Python [65] package of Scikit-learn [67].

For sepsis, we first analysed if the real numeric variables had extremely long
tails. As discussed in the Data Records section, we decided to subdivide
numeric variables with extremely long tails into deciles and process them as
categorical variables. The remaining real numeric variables were log-
transformed if necessary, before re-centring and re-scaling within the range
of $[0,1]$ with preprocessing.MinMaxScaler.

Numeric variable processing was slightly easier for the HIV dataset. This was
because the only numeric variables were VL and CD4. The former required an
optimal Box-Cox transformation followed with re-centring and re-scaling to
$[0,1]$; whereas the latter required log-transformation followed by re-
centring and re-scaling to $[0,1]$.

While processing the numeric variables of the real datasets $X_{\text{real}}$,
it was important to record the transformation procedure for every individual
variable. Only by doing so, we would be able to back-transform (i.e., revert
the re-centring and re-scaling procedure) the synthetic data $X_{\text{syn}}$
generated by network $G$. The dimensionality of real continuous numeric
variables (without an extremely long tail) was preserved after pre-processing.
Binary and categorical variables were converted into one-hot-encoded vectors
with total length equal to two (for binary variables) and equal to the number
of unique classes (for categorical variables).

Example: Pre-processing the Real Acute Hypotension Dataset
For the real hypotension dataset, there were $20$ variables as listed in Table
1. The dataset comprised $9$ numeric variables, $4$ categorical variables, and
$7$ binary variables. After transforming the $9$ numeric variables, there were
still $9$ features. The transformation of the $7$ binary variables resulted in
$14$ features; and there were $4+4+10+13=31$ unique classes for the
categorical variables of fluid boluses, vasopressors, FiO2, and GCS. Hence,
the output dimension of the generator network $G$ was also
$\mathscr{O}=9+14+31=54$. Note that this was the feature dimension for both
$X_{\text{real}}$ and $X_{\text{syn}}$.

Case Study: An Interpretation of the Output of the Generator for Hypotension
As mentioned earlier in this appendix, the generator network for hypotension
served as a mapping for $G:z\rightarrow x_{\text{syn}}$ from dimension
$\mathbb{R}^{128}\rightarrow\mathbb{R}^{54}$. Dimensions $1-9$ corresponded to
the clinical variables in the strict order of MAP, diastolic BP, systolic BP,
urine, ALT, AST, PaO2, lactate, and serum creatinine, respectively.
Furthermore, since all of the real numeric variables were readjusted to lie
within the range of $[0,1]$ (see Pre-processing Variables in
$X_{\text{real}}$), we passed the first $9$ dimensions of the generator output
to the sigmoid activation function to match the extreme values allowed for the
numeric variables.

Dimensions $10-40$ corresponded to the categorical variables. It was $10-13$
for fluid boluses, $14-17$ for vasopressors, $18-27$ for FiO2, and $28-40$ for
GCS. To represent these dimensions of the synthetic data as categorical
variables, we passed the corresponding dimensions to a softmax activation
function. For instance, after passing dimension $10-13$ through softmax,
dimension $10$ represented class I of fluid boluses; and likewise, $11$ for
class II, $12$ for class III, and $13$ for class IV for the fluid boluses
categories, respectively. A similar procedure was repeated for all other
categorical variables, and likewise for all the binary variables (dimensions
41-54).

####  C.1.3 On Soft Embedding

Before discussing the dimensionalities of the discriminator network $D$, we
need to address the soft-embedding function in Figure 1(c). The input to
network $D$ (i.e., the grey box on the left of 1(c)) was either
$x_{\text{syn}}$ or $x_{\text{real}}$. As depicted in the figure, all
continuous numeric variables were treated as-is from the data, but the binary
and categorical variables were subject to a soft embedding transformation. We
created a trainable embedding matrix $W_{\text{emb}}$ for every binary or
categorical variable. The weights were of size
$\mathbb{R}^{(\text{あ},\text{か})}$ where あ denotse the size of the unique
classes of the binary or categorical variables and か denotes the pre-specified
size of the projection (2 for binary variables and 4 for categorical
variables). The soft embedding helped us to represent the binary and
categorical variable features as vectors of high dimensional floating values
$x^{\text{T}}W_{\text{emb}}$ for the discriminator.

Case Study: Soft Embedding for the Hypotension Data
Let us denote $X$ as the hypotension data prior to soft embedding and refer to
$U$ as the data after soft embedding. The first $9$ dimensions of the
hypotension data $X^{(1:9)}$ represented continuous numeric variables and were
therefore not subject to soft embedding. Then for each binary or categorical
variable, we defined a unique matrix $W_{emb}$. If the variable was binary, we
would set か$=2$; and categorical variables had か$=4$ instead. These were
hyper-parameters that were specified before training the GANs model; and these
settings were applied to all of the synthetic datasets.

Of the 20 variables in the hypotension data $X\in\mathbb{R}^{54}$, the one-
hot-encoded binary variables for urine (M) was stored in $X^{(41:42)}$. Thus,
the dimension of the soft embedding matrix for urine (M) was ${2\times 2}$ and
the soft embedding procedure was hence $U^{\text{urine
(M)}}=X^{(41:42)^{\text{T}}}W_{\text{emb}}^{\text{urine (M)}}$ resulted in a
continuous feature vector of size $2$.

As an additional example, the information related to GCS was stored in
$X^{(28:40)}$. The dimension of the soft embedding matrix for GCS was $13$
(the number of unique classes of GCS) by $4$ (the projection size for
categorical variables). The soft embedding of GCS was hence
$U^{\text{GCS}}=X^{(28:40)^{\text{T}}}W_{\text{emb}}^{\text{GCS}}$ resulting
in a continuous feature vector of size $4$.

After the binary and categorical variables were transformed through soft
embedding, they were concatenated with the numeric variables such that
$U=[X^{(1:9)}\bigoplus U^{\text{Fluid Boluses}}\bigoplus
U^{\text{Vasopressors}}\bigoplus\ldots\bigoplus U^{\text{Serum Creatinine
(M)}}]$. Hence, the total dimension of the post-embedding vector $U$ was
$9+4\times 4+7\times 2=39$; that is, $9$ dimensions for numeric with feature
of size $4$ for the $4$ categorical variables and feature vectors of size $2$
for the $7$ binary variables in the hypotension data. In conclusion, the input
dimension of the discriminator network $D$ was $39$ for the hypotension
dataset.

####  C.1.4 On the Dimensionality of the Discriminator

The discriminator $D$ had an input dimension that varied across task (see
Section C.1.3), but it had a fixed hidden dimension of size $128$, and a fixed
output dimension of $1$. The output dimension of the discriminator was not a
hyper-parameter; instead, it was fixed as $1$ due to the design of the GAN
architecture[16]. This was because the output of the discriminator correspond
to a realisticness score of the data $X$ where $D(X)\in\mathbb{R}$.

###  C.2 Remarks on Training the GAN Model

The scheme for training the WGAN-GP was mostly identical to the setting in the
original paper [27]. The Health Gym GAN received the training data in batches
of size $32$ and was updated using the Adam optimiser [68] with learning rate
$1\times 10^{-3}$. The model was regularised with $\lambda_{\text{GP}}=10$ for
the gradient penalty loss (see Equation (1)) and $\lambda_{\text{corr}}=10$
for the alignment loss (see Equation (3)). To stabilise the learning curve of
the model, we adopted a curriculum learning [69, 70] strategy where we first
trained the GAN to create short sequences and then progressively generated
longer synthetic time series. Furthermore, the generator network was updated
once for every $5$ updates of the discriminator network (see Algorithm 1 on
page 4 of the referenced work [27]).

The Health Gym GAN was capable of generating relatively good data after the
first $100$ epochs of training. However, we found that the first $100$ epochs
of training could be highly unstable, causing the quality of the synthetic
data to vary greatly (mostly from seed to seed). The quality of the generated
data stabilised when we prolonged the training to $500$ epochs. This was when
we terminated the training of the GAN model and used the generator network to
synthesise our datasets.

The Final Transformation After Synthetic Data Generation
As mentioned in Appendix C.1.2, all of the synthetic data $X_{\text{syn}}$
were passed through either a sigmoid activation function (for the numeric
variables) or a softmax (for the binary or categorical variables). To ensure
that each synthetic variable could be meaningfully interpreted, it was
necessary to execute a back-transformation.

For continuous numeric variables this involved reverting the centring and
scaling steps; and also applying the inverse of any prior power
transformations. For the binary and categorical variables we only had to seek
the largest probability in the softmax activated values. For instance, since
the hypotension GCS features were stored in $X_{\text{syn}}^{(28:40)}$, we
would have to find which value among those dimensions was the largest. Say
that it was dimension $29$, then it meant that the second class of the GCS
features was the most probable – and hence it was a GCS of score $4$ (remember
that GCS scores starts from 3, so that $3+1=4$).

##  Appendix D The Statistical Tests of Stage 2

This appendix contains the definitions of the statistical test used in stage 2
of our realisticness validation procedure and their outcomes.

Our readers can click here to resume to Stage Two: Statistical Tests in the
Realisticness Validation Procedure section.

###  D.1 The Two Sample KS Test

The null hypothesis of the two sample KS test assumes that there are

no significant differences between the distributions

of the synthetic data and the real data for a specific variable $X$. The KS
test accepts both numeric and categorical data types. Given a sample of data
$B_{S}(\mathbf{x})$ from the synthetic variable and a sample of data
$B_{R}(\mathbf{x})$ from the real variable, the test computes the statistic

$\displaystyle
K_{B_{S},B_{R}}=\text{sup}_{\mathbf{x}}\left|C_{B_{S}}(\mathbf{x})-C_{B_{R}}(\mathbf{x})\right|$
(5)

based on the supremum of two empirical cumulative distribution functions
(ECDFs) $C$; with $C_{B_{S}}$ and $C_{B_{R}}$ denoting the ECDFs of
$B_{S}(\mathbf{x})$ and $B_{R}(\mathbf{x})$ respectively. The ECDFs are
computed as

$\displaystyle
C_{B}=\frac{1}{n_{B}}\sum^{n_{B}}_{j=1}\mathbf{1}_{(-\infty,\mathbf{x}]}(x_{j})$
(6)

for $x_{j}\in B(\mathbf{x})$ of size $n_{B}$. We denote
$\mathbf{1}_{(-\infty,\mathbf{x}]}(x_{j})$ as the indicator function: $1$ if
$x_{j}\leq\mathbf{x}$ and $0$ otherwise. The null hypothesis is rejected if
the statistic $K_{B_{S},B_{R}}$ exceeds a critical value with a statistical
significance level $\alpha_{KS}$ following the Kolmogorov distribution[71,
72]; we use $\alpha_{KS}=0.05$. In our work, we implemented the two sample KS
test with the stats.ks_2samp function of the Python [65] package of Scipy
[66].

###  D.2 The Two Independent Sample Student’s t-Test

The null hypothesis of the two independent sample Student’s t-test assumes
that there are

no significant differences between the means

of the synthetic data and the real data for a specific variable $X$. The
t-test only accepts numeric data types. Following the notations given in
Appendix D.1, the test statistic is

$\displaystyle
t_{B_{S},B_{R}}=\frac{\overline{B_{S}(\mathbf{x})}-\overline{B_{R}(\mathbf{x})}}{S}\sqrt{\frac{n_{B_{S}}n_{B_{R}}}{n_{B_{S}}+n_{B_{R}}}}$
(7)

where $\overline{B_{S}(\mathbf{x})}$ and $\overline{B_{R}(\mathbf{x})}$ refers
to the respective sample means of $B_{S}(\mathbf{x})$ and $B_{R}(\mathbf{x})$;
and the pooled standard deviation $S$ is

$\displaystyle S=\sqrt{\frac{S_{B_{S}}^{2}+S_{B_{R}}^{2}}{2}}$ (8)

where $S_{B_{S}}^{2}$ and $S_{B_{R}}^{2}$ denote the respective unbiased
estimators for the variances of $B_{S}(\mathbf{x})$ and $B_{R}(\mathbf{x})$.
The null hypothesis is rejected if the statistic $t_{B_{S},B_{R}}$ exceeds a
critical value of the Student’s t-distribution [73] with a pre-determined
threshold level $\alpha_{t}$; we use $\alpha_{t}=0.05$. In our work, we
implemented the two independent sample t-test with the stats.ttest_ind
function of the Python [65] package of Scipy [66].

###  D.3 The Snedecor’s F-Test

The null hypothesis of the Snedecor’s F-test assumes that there are

no significant differences between the variances

of the synthetic data and the real data for a specific variable X. The F-test
accepts both numeric and categorical data types. Following the notations given
in Appendices D.1 and D.2, the test statistic for the numeric case is

$\displaystyle F_{B_{S},B_{R}}=\frac{S_{B_{S}}^{2}}{S_{B_{R}}^{2}}$ (9)

and it measures the deviation of the two population variances. The more this
value deviates from 1, the stronger the evidence for unequal variances. The
null hypothesis is rejected if the statistics $F_{B_{S},B_{R}}$ is greater
than a pre-determined critical value from the F distribution [74] with
significance level $\alpha_{F}$; we use $\alpha_{F}=0.05$. We implemented the
F-test with the stats.f.cdf function of the Python [65] package of Scipy [66].
For the categorical case, the F-statistic measured the between-group variance
against the within-group variance instead; and it was implemented with the
stats.f_oneway function of Scipy.

###  D.4 The Three Sigma Rule Test

The three sigma rule test, also known as the 68-95-99.7 rule test, is an
empirical rule to determine whether a value (or a set of values) lies within
an interval estimate of a normal distribution. A variable $X$ sampled from a
normal distribution with mean $\mu$ and standard deviation $S$ has the
following approximated probabilities of lying within one, two, or three
standard deviations from the mean:

$\displaystyle\mathbb{P}(\mu-S\leq X\leq\mu+S)$ $\displaystyle\approx 68\%,$
(10) $\displaystyle\mathbb{P}(\mu-2S\leq X\leq\mu+2S)$ $\displaystyle\approx
95\%\text{, and}$ (11) $\displaystyle\mathbb{P}(\mu-3S\leq X\leq\mu+3S)$
$\displaystyle\approx 99.7\%.$ (12)

In this study, we used the three sigma rule test to access whether a synthetic
variable covered a similar range as the real counterpart, if it failed the KS
test. This test was added because both the t-test and the F-test could only
reveal properties of the synthetic data distribution (and explain potential
shortcomings) but they could not be used to determine whether the synthetic
data was still acceptable for downstream machine learning applications. We
chose to use the two standard deviation confidence interval (CI with $\pm
2S$); see also the following Appendix D.5.

###  D.5 Iteratively Executing the Statistical Tests

Algorithm 1 The Psuedo-Code for Executing Stage 2

1:$\eta_{KS}$ $=$ $\eta_{t}$ $=$ $\eta_{F}$ $=$ $\eta_{3S}$ $=0$
$\triangleright$ Initiate the counters as 0.

2:.

3:for
$\xi=1,{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\Xi}$
do$\triangleright$ Run the statistical tests for $\Xi$ iterations.

4: $\\{\\}_{3S}$ $=\emptyset$ $\triangleright$ Instantiate an empty set for
the three sigma rule test.

5: .

6: $B^{*}_{S}(\textbf{x})\sim X_{\text{syn}}\text{ and
}B^{*}_{R}(\textbf{x})\sim X_{\text{real}}$ $\triangleright$ Sample raw data
with size $n_{B}$.

7:
$B_{S}(\textbf{x})=\texttt{transform}\left(B^{*}_{S}(\textbf{x})\right)\text{
and }B_{R}(\textbf{x})=\texttt{transform}\left(B^{*}_{R}(\textbf{x})\right)$
$\triangleright$ Optional minmax transformation.

8: .

9: if $X_{\text{real}}\text{ is numeric }$ then

10: $P_{KS}=\texttt{KS test}(B_{S}(\textbf{x}),B_{R}(\textbf{x}))$
$\triangleright$ Acquire p-values for the statistical hypothesis test.

11: $P_{t}=\texttt{t-test}(B_{S}(\textbf{x}),B_{R}(\textbf{x}))$

12: $P_{F}=\texttt{F-test}(B_{S}(\textbf{x}),B_{R}(\textbf{x}))$

13: .

14: $\epsilon_{\text{lower}},\epsilon_{\text{upper}}=\mu(B_{R}(\textbf{x}))\pm
2S(B_{R}(\textbf{x}))$$\triangleright$ Compute real value CI with 2 standard
deviations.

15: $\\{\\}_{3S}$ $=$ $\\{\\}_{3S}$
$\cup_{i=1}^{n_{B}}B_{S}(\textbf{x}_{i})\text{ if
}B_{S}(\textbf{x}_{i})\in[\epsilon_{\text{lower}},\epsilon_{\text{upper}}]$$\triangleright$
Append the list with synthetic values within the CI.

16: .

17: $\eta_{KS}$ $\text{ }+=1\text{ if }P_{KS}>$ $\alpha_{KS}$ $\triangleright$
Increase the counter if the null hypothesis is kept.

18: $\eta_{t}$ $\text{ }+=1\text{ if }P_{t}>$ $\alpha_{t}$$\triangleright$
(Note the “$>$” sign.)

19: $\eta_{F}$ $\text{ }+=1\text{ if }P_{F}>$ $\alpha_{F}$

20: .

21: $\eta_{3S}$ $\text{ }+=1\text{ if }|$ $\\{\\}_{3S}$ $|>\tau_{3S}$
$\triangleright$ Increase the counter if the majority of synthetic data is in
the CI.

22: end if

23: .

24: if $X_{\text{real}}\text{ is categorical }$ then

25: $P_{KS}=\texttt{KS test}(B_{S}(\textbf{x}),B_{R}(\textbf{x}))$

26: $P_{F}=\texttt{F-test}(B_{S}(\textbf{x}),B_{R}(\textbf{x}))$

27: .

28: $\eta_{KS}$ $\text{ }+=1\text{ if }P_{KS}>$ $\alpha_{KS}$

29: $\eta_{F}$ $\text{ }+=1\text{ if }P_{F}>$ $\alpha_{F}$

30: end if

31:end for

We summarised the execution procedure for the statistical tests in the pseudo-
code overleaf. The purpose of the algorithm was to compare a synthetic
variable $X_{\text{syn}}$ against its real counterpart $X_{\text{real}}$.
Since the synthetic datasets of this paper were primarily designed for machine
learning algorithms including RL, we tested the generated data using repeated
mini-batch sampling, a scenario that resembles the iterative training
phase[30] of a neural network. The iterative process also helped us estimate
the Type II error of the statistical tests. A detailed description of the
pseudo-code is provided below.

In order to mimic the iterative training phase of a nueral network, we ran the
statistical tests for a pre-specified maximum number of iterations $\Xi=100$
(Line 3) with a sampling size of $n_{B}=32$ (Line 6). Prior to the tests, we
defined $\eta_{KS}$, $\eta_{t}$, $\eta_{F}$, $\eta_{3S}$ as the counters for
non-rejected KS tests, t-tests, F-tests, and three sigma rule tests,
respectively (Line 1). Numeric variables were rescaled as in the pre-
processing step for machine learning model building (Line 7). We increased the
counters by 1 whenever we kept a null hypotheses (Lines 17 - 19), i.e., when
the p-values were greater than the statistical significance level of
$\alpha=0.05$. When the iterations ended, we used the counters as indicators
of the realisticness of the synthetic variables. If the final counter values
were $70\%$ or higher than the amount of total iterations ($\eta$ $>0.7$
$\Xi$), we concluded that the synthetic variable had accurately captured the
distributional features of the real variable. Of note, only the KS test and
F-test were suitable for categorical variables. The three sigma rule test is
not a statistical hypothesis test, and is discussed separately below.

The $70\%$ threshold was chosen arbitrarily. However, it was because that it
was a relatively strict threshold for preventing Type I error, since we only
regarded synthetic variables to be realistic when $70\%$ of the trials were
associated with less than $5\%$ risk (the $5\%$ significance level) of
concluding that a difference existed when there was no true difference. For
completeness, we reported the rates of passing each test for all variables in
the tables of Appendix D.6.

To conduct the three sigma rule test, we instantiated an empty set
$\\{\\}_{3S}$ at the beginning of each iteration (Line 4). Then, for this
study, we chose the $2S$ interval of the real variable as the range for
reliable values (Line 14). After defining the lower and upper bounds, we
appended $\\{\\}_{3S}$ with synthetic variables that lied within the defined
range (Line 15). We set a threshold $\tau_{3S}=0.7$ $n_{B}$; that is, we
considered a mini-batch of synthetic variable passing the three sigma rule
test if more than $70\%$ of the synthetic data of that batch fell within the
two standard deviation range (Line 21). Finally, if more than $70\%$ of all
random mini-batches passed the aforementioned threshold ($\eta_{3S}$$>0.7$
$\Xi$), we considered the synthetic variable to be sufficiently accurate even
if it had failed the KS test.

###  D.6 The Statistical Outcomes

The content below supplements the Validation Outcomes section of the Technical
Validation.

####  D.6.1 Acute Hypotension

Variable Name | . KS-Test . | . t-Test . | . F-Test . | Three Sigma Rule Test
---|---|---|---|---
MAP | $96/100$ | $95/100$ | $86/100$ | $100/100$
Diastolic BP | $92/100$ | $94/100$ | $93/100$ | $100/100$
Systolic BP | $98/100$ | $98/100$ | $91/100$ | $97/100$
Urine | $95/100$ | $92/100$ | $96/100$ | $100/100$
ALT | $62/100$ | $94/100$ | $90/100$ | $94/100$
AST | $59/100$ | $95/100$ | $91/100$ | $97/100$
PaO2 | $40/100$ | $94/100$ | $88/100$ | $97/100$
Lactic Acid | $74/100$ | $96/100$ | $83/100$ | $98/100$
Serum Creatinine | $90/100$ | $94/100$ | $94/100$ | $100/100$
Fluid Boluses | $100/100$ | \- - | $84/100$ | \- -
Vasopressors | $100/100$ | \- - | $95/100$ | \- -
FiO2 | $100/100$ | \- - | $93/100$ | \- -
GCS | $97/100$ | \- - | $95/100$ | \- -
Urine (M) | $97/100$ | \- - | $91/100$ | \- -
ALT/AST (M) | $100/100$ | \- - | $75/100$ | \- -
FiO2 (M) | $100/100$ | \- - | $85/100$ | \- -
GCS (M) | $100/100$ | \- - | $95/100$ | \- -
PaO2 (M) | $100/100$ | \- - | $86/100$ | \- -
Lactic Acid (M) | $100/100$ | \- - | $88/100$ | \- -
Serum Creatinine (M) | $100/100$ | \- - | $92/100$ | \- -
Table 8: The Stage Two Statistics for Acute Hypotension

An overview of the statistic tests for the synthetic hypotension dataset is
presented in Table 8. As described in Appendix D.5, the statistical tests were
repeated for $\Xi=100$ iterations and were considered “passed” if the null
hypothesis was not rejected $70\%$ or more of the times ($\eta$ $>0.7$ $\Xi$).
The denominators of the scores in the table refer to the maximum number of
iterations, and the numerators correspond to the number of times that the null
hypothesis was not rejected. For instance, MAP with a score of $96/100$ meant
that $96$ out of $100$ times we kept the null hypothesis of the KS-test at a
significance level of $0.05$. This passes the $70\%$ mark and is highlighted
in blue. Failed tests are highlighted in red.

All variables in the synthetic hypotension datasets are realistic. Although 3
variables (ALT, AST, and PaO2) fail the KS test, their means and variances are
captured appropriately; and all variables are within plausible ranges of the
real variables.

####  D.6.2 Sepsis

Of all the $44$ variables, only Max Vaso behaves undesirably. However, as we
previously mentioned in the Technical Validation section, this was because Max
Vaso was highly skewed. As described in the Data Records section, Max Vaso was
not transformed into a categorical variable because its most related variables
(i.e., Input Total, Input 4H, Output Total, and Output 4H) were numeric.

Nonetheless, the synthetic Max Vaso variable performed well regarding the
three sigma rule test, indicating that the synthetic variable was within a
plausible range of the real variable values. To conclude, the synthetic sepsis
dataset is highly realistic.

Variable Name | . KS Test . | . t-Test . | . F-Test . | Three Sigma Rule Test
---|---|---|---|---
Age | $96/100$ | $96/100$ | $98/100$ | $100/100$
HR | $98/100$ | $95/100$ | $95/100$ | $100/100$
Systolic BP | $96/100$ | $93/100$ | $96/100$ | $100/100$
Mean BP | $94/100$ | $95/100$ | $92/100$ | $100/100$
Diastolic BP | $95/100$ | $94/100$ | $92/100$ | $100/100$
RR | $97/100$ | $95/100$ | $95/100$ | $98/100$
K | $94/100$ | $97/100$ | $91/100$ | $100/100$
Na | $97/100$ | $95/100$ | $91/100$ | $100/100$