-

Corresponding author: Olaf Wolkenhauer (olaf.wolkenhauer@uni-rostock.de),
ORCID: 0000-0001-6105-2937

# ConvGeN: Convex space learning improves deep-generative oversampling for
tabular imbalanced classification on smaller datasets

KRISTIAN SCHULTZ1, SAPTARSHI BEJ1, WALDEMAR HAHN3, MARKUS WOLFIEN3, PRASHANT
SRIVASTAVA1, OLAF WOLKENHAUER1,2  Institute of Computer Science, University of
Rostock, Germany  Leibniz-Institute for Food Systems Biology, Technical
University of Munich, Freising, 85354, Germany  Institute for Medical
Informatics and Biometry, Faculty of Medicine Carl Gustav Carus, Technische
Universität Dresden, Germany

###### Abstract

Data is commonly stored in tabular format. Several fields of research are
prone to small imbalanced tabular data. Supervised Machine Learning on such
data is often difficult due to class imbalance. Synthetic data generation,
i.e., oversampling, is a common remedy used to improve classifier performance.
State-of-the-art linear interpolation approaches, such as LoRAS and ProWRAS
can be used to generate synthetic samples from the convex space of the
minority class to improve classifier performance in such cases. Deep
generative networks are common deep learning approaches for synthetic sample
generation, widely used for synthetic image generation. However, their scope
on synthetic tabular data generation in the context of imbalanced
classification is not adequately explored. In this article, we show that
existing deep generative models perform poorly compared to linear
interpolation based approaches for imbalanced classification problems on
smaller tabular datasets. To overcome this, we propose a deep generative
model, ConvGeN that combines the idea of convex space learning with deep
generative models. ConvGeN learns the coefficients for the convex combinations
of the minority class samples, such that the synthetic data is distinct enough
from the majority class. Our benchmarking experiments demonstrate that our
proposed model ConvGeN improves imbalanced classification on such small
datasets, as compared to existing deep generative models, while being at-par
with the existing linear interpolation approaches. Moreover, we discuss how
our model can be used for synthetic tabular data generation in general, even
outside the scope of data imbalance and thus, improves the overall
applicability of convex space learning.

###### Index Terms:

Imbalanced data, Convex space learning, LoRAS, GAN, Tabular data

=-15pt

##  I Introduction

Tabular datasets are a popular and convenient format of storing data. Most
data is stored in relational databases and are therefore tabular by nature.
Yet, another major challenge associated to such fields is that often it is not
possible to obtain balanced data for classification and decision-making
problems. In fact, in many important practical problems, class imbalance
intrinsically occurs. For example, in case of fraud detection problems it is
natural that only a small fraction of the population would commit a fraud. It
is well known that class imbalance makes it difficult for Machine Learning
(ML) models to adapt well to the data and learn latent patterns because there
are certain classes with only very few instances. Therefore, classification
models tend to be biased towards the majority class [5]. Classes with fewer
number of instances are called minority classes, while the larger class is
called majority classes. The region in the data manifold where these two
classes intersect is often called the borderline region, and is often of
particular interest for ML applications. In this manuscript, we limit our
investigation to binary classification problems and therefore address one
majority and one minority class [2].

One major approach of improving imbalanced classification is synthetic sample
generation for the minority class to balance the classes in a dataset.
Historically, the SMOTE algorithm proposed in 2002, has been the first
algorithm to generate synthetic samples for the minority class [5]. The
algorithm is based on a linear interpolation-based approach, where a convex
linear combination of two minority class samples that are closely located, is
considered a synthetic sample. A convex combination is a linear combination of
vectors, such that the sum of the coefficients of the linear combination is
one [2]. However, with time, researchers pointed out several limitations of
the SMOTE algorithm, such as ‘over-generalization of the minority class’ or
that the samples generated in a minority data neighborhood has high variance
[3, 2, 4]. This can be interpreted as high variance of the synthetic samples
in data neighborhoods, especially in the borderline region, that leads to
undesirable interference of the synthetic samples with the majority class, and
therefore, create confusion among the classifiers[2]. A recent article
documents 85 SMOTE-based oversampling algorithms and performs a detailed
benchmarking study on them, demonstrating that the problem of class imbalance
is still open to research and improvement [7].

Recently, the approach of convex space learning has been proposed to model the
convex space of the minority class in an improved manner to overcome the
identified limitations problems of the SMOTE algorithms. The Localized Random
Affine Shadowsampling (LoRAS) algorithm does this by generating synthetic
samples using convex linear combinations of multiple shadowsamples, which are
Gaussian noise added to the original minority samples in a minority data
neighborhood [3]. This approach reduces the variance of the synthetic samples
and prevents them from interfering with the majority class [3]. Another
improved version of the LoRAS algorithm called ProWRAS makes the modeling of
the convex space even more precise. The specialty of this algorithm is that,
it offers four oversampling schemes depending on the choice of two of its
parameters, enabling it to generate customized synthetic samples for several
classifiers [2]. Oversampling algorithms before ProWRAS were known to perform
well only for specific classifiers. However, the ProWRAS algorithm, unlike its
predecessors, can be compatible with a wide range of classifiers [2]. We call
this philosophy of rigorous modeling of the convex data space to sample
synthetic data from it, and thereby enhance ML model performance for a
specific task (in this case classification), as convex space learning or
convex space modeling.

Generative Adversarial Network (GAN) is a very popular neural network-based
model for synthetic data generation. Using two competing neural networks,
generator and discriminator, GANs can create realistic synthetic data from
random noise [6]. While training the model, a random noise vector is passed
into the generator, which creates synthetic samples from the noise, while the
discriminator learns to differentiate between real samples against the
synthetic samples generated by the generator. Over several iterations of
competitive training, the generator learns to create more and more realistic
synthetic data [6]. A huge amount of research has been done on GANs with the
goal of ‘realistic data generation’. Diverse models based on the GAN approach
have been developed since the introduction of GANs in 2014 [9, 17, 16].

Although mostly such models are focused on synthetic image generation, since
2017, multiple deep-generative models have been developed focusing on
synthetic data generation for tabular datasets. Note that, realistic data
generation’, while easy to perceive for image based data, is harder to realize
or judge for tabular datasets, as there is no direct visual aid for such data.
MedGAN [1], the first of such architectures, can handle either binary or count
data. Another model TableGAN [14], based on deep convolutional GAN (DCGAN),
also consisting of an additional classifier neural network can generate
numerical and categorical values. TGAN or tabular GAN[20] handles multi-modal
data in continuous variables through Gaussian Mixture Models (GMM) and can
create categorical values with the help of Gumbel-Softmax as activation
function, using a Long-Short-Term-Memory (LSTM) generator. Another model from
2019, CTGAN introduces conditional vectors for categorical values [21]. CTAB-
GAN proposed in 2021, is a combination of TableGAN and CTGAN [23].

However, GAN-based synthetic data generation has several natural limitations
when it comes to smaller tabular datasets that are meant for imbalanced
classification:

  * •

Generative adversarial models are not specifically aimed at generating
synthetic data for classification tasks. If realistic data generation means
the ability of a network to generate exact copies of the existing data, then
intuitively, that does not add much additional information to classifiers.
Clearly, for imbalanced classification, the variation in the synthetic data is
a key factor and generating exact copies of the original data reduces it
drastically.

  * •

GANs are more difficult to train on small amounts of data [10].

In this article, we will investigate through our experiments the effectiveness
of oversampling using GAN-based models to improve classification on smaller
tabular imbalanced data. Note that, we considered an empirical assumption that
small imbalanced data means at most $150$ minority data points being used for
synthetic data generation, while training a data rebalancing model and
datasets containing at most $3500$ data points in total, with an imbalance
ratio of at least $15:1$ (the ratio of the number of majority points to the
number of minority points). We identified $14$ benchmarking datasets in the
public domain satisfying our criterion of being ‘small’ that are subsequently
used for our benchmarking studies.

Moreover, we here propose a novel deep-generative model called Convex-space
Generative Network (ConvGeN) that combines the philosophy of linear
interpolation based approaches with that of deep-generative networks to
improve synthetic sample generation in the context of imbalanced
classification for small datasets (as per our empirical assumption). The model
consists of a generator-discriminator architecture. However, instead of
competing among themselves as seen in regular GANs, in ConvGeN, they cooperate
with each other. The generator takes randomly shuffled minority neighborhood
batches as input, instead of taking noise as input in traditional models. The
generator then creates convex linear combinations of the input neighborhood as
synthetic samples. The discriminator learns to differentiate the synthetic
samples from a shuffled batch of majority class samples (that can be chosen
such that the majority batches are proximal to the respective minority
batches) chosen as inputs at each training step. In this way, during the
training process, the generator learns the coefficients of the convex linear
combination that are more suitable for synthetic sample generation, in the
sense that the synthetic samples can be identified distinctly by the
discriminator. An additional bonus is that during the training, the
discriminator is automatically trained as a classifier customized to the
synthetic samples.

Interestingly, we have chosen a ‘Repeater’ model as a baseline of our
experiments. This model creates exact copies of minority class samples to
balance the two classes. We noticed that for smaller tabular datasets, state-
of-the-art generative models specialized for tabular datasets like CTGAN and
CTAB-GAN, in most cases have poor performance than the baseline. Our
experiments show that ConvGeN performs at-par with the state-of-the-art
interpolation based approach ProWRAS, while being superior to the traditional
GAN, including CTGAN and CTAB-GAN, and in terms of popular metrics, such as
F1-score and $\kappa$-score for imbalanced classification problems on tabular
datasets of small size.

The novelty of ConvGeN is that it generates synthetic samples from the convex
space, but unlike existing liner interpolation based models, such as SMOTE,
LoRAS or ProWRAS, the coefficients of convex combinations are not chosen
randomly or decided as a part of the algorithm, but are learned by the model
itself during the training. The model thus provides a non-linearly linear’
method for synthetic data generation, in the sense that the synthetic samples
are convex linear combinations of the original samples, but the coefficients
of the convex linear combinations are learned in a non-linear way.

Figure 1: Figure showing the architecture of the ConvGeN model. The generator
receives shuffled minority neighborhood batches as input and generates a
convex combination of the minority neighborhood samples as output. The
discriminator classifies these generated samples against a random majority
batch of the same size as the minority neighborhood. The discriminator can be
retrained as the DoC classifier after the training of the ConvGeN model.

##  II Related research

We have used six algorithms in total for synthetic data generation in our
experiments. These algorithms also majorly consist of related research for
ConvGeN. Thus, we hereby discuss these related algorithms. A more detailed
rationale of why these algorithms are closely related to ConvGeN is given in
Section IV-A.

###  II-A Repeater

This model was used as a baseline for our experiments. The repeater model
simply generates multiple copies of minority class samples to balance both the
classes. Exact copies of the data points in the minority class are generated
sequentially until the classes are balanced. Conventionally, the scenario with
no oversampling is used as a benchmark [3, 2]. However, since it is well-known
that balanced datasets produce better classification performances, this is the
simplest model one could build to balance the datasets.

###  II-B ProWRAS

ProWRAS[2] (GIT link: https://github.com/COSPOV/ProWRAS.git) is a multi-
schematic oversampling algorithm using convex linear combinations of minority
shadowsamples (Gaussian noise added to original minority samples), for
synthetic sample generation. SMOTE and its extension algorithms are known to
perform with different efficiency for different classifiers. For example,
although the LoRAS algorithm works very well for kNN, it produces relatively
poor performance with RF. The ProWRAS algorithm overcomes this by using a more
rigorous modeling of the convex space of the minority class [3, 2].

In brief, the minority set is divided into partitions depending on the
proximity of the minority samples to the majority class. To each partition, a
weight, according to its distance to the majority set is assigned. The weights
are used to determinate the number of points used from each partition for
synthetic sample generation. The closer the partition to the majority set, the
more synthetic samples are generated from that partition. Moreover, the
weights are also used to decide the number of shadowsamples used for the
convex combination to generate a synthetic sample. For a minority partition
closer to the majority class, more shadowsamples are used for the convex
combination, because it reduces the variance of the synthetic samples,
preventing them from interfering with the majority class[2].

The model offers four oversampling schemes depending on the choice of two
parameters _max_conv_ and _neb_conv_. _max_conv_ selects the number of
shadowsamples used to create one synthetic data point. The oversampling
schemes are designed such that, with the proper choice of oversampling scheme,
the algorithm can be compatible with multiple classifiers [2].

###  II-C GAN

Formally, a Generative model $G$ takes a random input $z$ and outputs a
$x_{G}=G(z)$. Ideally, for a trained GAN, $x_{G}$ should follow the
distribution of the original samples [6]. The Discriminator network receives
as input $x$ that can be an original sample or a synthetic sample and returns
the probability of $D(x)$ of $x$ to be an original sample [6]. These two
models $G$ and $D$ compete in a two-player min-max game with value function:

$\min_{G}\max_{D}V(D,G)=\mathbb{E}_{D}+\mathbb{E}_{G}$ (1)

where,

$\mathbb{E}_{D}=\mathbb{E}_{x\sim P_{data}(X)}[\log(D(x))]$ (2)

and,

$\mathbb{E}_{G}=\mathbb{E}_{z\sim P_{z}(L)}[\log(1-D(G(z)))]$ (3)

$x\in X$ are data points sampled from the data distribution $P_{data}$ and $z$
are random noises sampled from some distribution $P_{z}$;
$\mathbb{E}_{P}[\cdot]$ represents expectation of a random variable over a
distribution $P$ [6]. The training procedure consists of alternating between
$k$ optimizing steps for $D$ and one optimizing step for $G$ by applying a
proper optimizer. Therefore, during training, $D$ is optimized to correctly
classify training data and samples generated from $G$, assigning $1$ and $0$
respectively. On the other hand the generator is optimized to confuse the
discriminator [6]. Thus, the generator learns through backpropagation from
each decision of the discriminator, which leads the generator to produce more
and more realistic samples until the discriminator can not distinguish between
real and synthetic data points anymore. In our implementation as a pre-
processing step, we scale the data by a factor of $\alpha^{-1}$, where

$\alpha=\max\left(1.0,1.1\cdot\max_{x\in data}\|x\|_{\infty}\right)$

with $\|x\|_{\infty}=\max\left\\{|x_{i}|:i\in\\{1,2,3,\ldots,n\\}\right\\}$ is
the $L^{\infty}$ norm. The final layer uses the _softsign_ function to project
to the output in range
$\mathcal{B}=\\{(x_{1},\ldots,x_{n})\in\mathbb{R}^{n}:-1\leq x_{i}\leq 1\\}$
where $n$ is the number of features in a dataset. The generated synthetic
points are scaled back by multiplying with $\alpha$, after training. This step
was essential due to the range of the _softsign_ function, $[-1,1]$, as shown
in Table I. Both the generator and the discriminator are Multi-Layered
Perceptrons (MLP). The intermediate layers of our GAN model use _ReLU_ as
activation function. The used noise size is $\nu=16n$. The layer for the
generator have the size $2\nu,4n$ and $2n$. The discriminator uses the layer
sizes $40n$, $20n$ and $10n$ with _ReLU_ as output function. The last layer
uses the _sigmoid_ activation function.

TABLE I: Different activation functions used for the GAN model.

Name | Function | Range
---|---|---
ReLU | $f(x)=\max\\{0,x\\}$ | $0\leq f(x)$
softsign | $f(x)={x/(|x|+1)}$ | $-1<f(x)<1$
sigmoid | $f(x)={1/(1+\exp(-x}))$ | $0<f(x)<1$

###  II-D CTGAN

CTGAN (GIT link: https://github.com/sdv-dev/CTGAN) is a synthetic data
generation library for tabular datasets based on Conditional Generative
Adversarial Networks. To model Non-Gaussian and multimodal continuous
distributions Xu et al. [21] use three steps. First, they use a variational
Gaussian mixture model (VGM) to estimate the number of modes for a given
continuous column. Then, for each value in the selected column the probability
of belonging to a specific mode is computed. In the last step, a value is
sampled from that probability density and normalized for the sampled mode
(mode-specific normalization). Each continuous value is therefore represented
with a one hot encoded mode and a continuous mode-specific normalized scalar.
Discrete data is represented using one-hot encoding [21].

To address challenges with imbalanced discrete data the algorithm uses a
conditional generator and training by sampling. Therefore, a conditional (one-
hot-encoded) vector was introduced. This vector represents the selection of
one specific value for just one discrete column. Thus, the size of the vector
is the sum of the cardinality of each discrete column. During training a
column with discrete values is uniformly selected. Then, a specific value is
sampled according to the PMF across all possible values (the probability mass
of each value is the logarithm of its frequency in that column) [21]. For a
selected value in the column, the conditional vector is set accordingly. The
generator is fed with this conditional vector and a noise vector consisting of
random values. The loss function of the generator has been altered to
additionally penalize the generator in cases where it does not produce the
selected column with the selected value. To prevent mode collapse the idea of
‘packing’ is introduced where the discriminator works on multiple samples
simultaneously [22].

To train CTGAN, a list of discrete columns needs to be provided. Other columns
are treated as continuous data. For our experiments, we automatized the
decision, so that columns with integer data only are considered discrete, and
columns with at least one fractional value are considered continuous.

###  II-E CTAB-GAN

Zhao et al. [23] introduced CTAB-GAN (GIT link: https://github.com/Team-
TUD/CTAB-GAN) for synthetic tabular data generation, which is also based on
Conditional Generative Adversarial Networks. In addition to a Generator and
Discriminator, CTAB-GAN uses an auxiliary Classifier. This Classifier
(multilayer perceptron) is trained to improve semantic integrity of the
synthetic data. Both Generator and Discriminator are implemented using CNNs.
The feature vector is represented by a square matrix of the size
$\lceil\sqrt{n}\rceil$ where $n$ is the number of features in the vector.
Missing values are padded by $0$. To improve the Generator, two additional
losses has been added: information loss (to keep the shape of the data) and
classification loss (to prevent the creation of invalid data, for example,
female with prostate cancer). Zhao et al. use a _conditional generator_ and
_training-by-sampling_ , which is inspired by the paper of Xu et al. [21]. The
development of CTAB-GAN was motivated by the limitation of prior state-of-the-
art:

  * •

_Mixed data type variables_ : Tabular data can contain values with special
meaning or limitations. For example a continuous variable could have
$0$-values representing _not available_.

  * •

_Skewed multi-mode continuous variables_ : The histogram of the data can have
various peaks in different heights.

  * •

_Long tail distribution_ : Most of the values are near one point and rare
values are far away from that point.

A Mixed-type Encoder is proposed to deal with mixed data. A column is
considered mixed if it contains both categorical and continuous values or
continuous values with missing values [23]. Mixed variables are treated as
value pairs, consisting of a categorical part (used also for missing values)
and a continuous part. The individual parts are handled similarly in CTGAN
[23]. However, these two parts build just one vector for the variable.

The concept of training by sampling introduced in CTGAN paper has been
extended here to include continuous variables. During training each column has
the same probability of being selected. For continuous variables a specific
mode is then sampled from the probability distribution (logarithm of the
frequency) of the modes [23].

To handle long tail distributions Zhao et al. pre-process variables with such
distribution with a logarithm transformation [23].

##  III The ConvGeN algorithm

An overview of the philosophy behind ConvGeN: ConvGeN generates synthetic
samples in the convex space of the minority class, much like SMOTE and its
extensions or like ProWRAS and LoRAS[5, 3, 2]. However, a key contrasting
feature of ConvGeN against such conventional linear interpolation models is
that the coefficients of convex combinations are not chosen randomly, but
learned by the model itself during the training, using two co-operating neural
networks. LoRAS and ProWRAS, although they developed the idea of a more
rigorous modeling of the convex space to improve imbalanced classification,
this modeling of the convex space has to be done in a dataset-specific fashion
by rigorous parameter tuning [2]. This is the reason why ProWRAS works best
when the oversampling algorithm is customized for each dataset and classifier
[2]. Deep co-operative learning enables ConvGeN to model the convex space
automatically over learning iterations.

ConvGeN has a generator-discriminator architecture similar to GANs. However,
there are key differences:

  * •

To begin with, the generator and the discriminator of ConvGeN, unlike
conventional GANs, do not compete against each other, but cooperate among
themselves. The optimization of the loss functions for the generator and the
discriminator are not in contradiction to each other. ConvGeN is thus not an
adversarial network.

  * •

The generator for ConvGeN takes shuffled batches of minority data points as
input and generates convex linear combinations of input data points as
synthetic samples. For conventional GANs, the generator takes random noise as
input and generates realistic synthetic samples that are not necessarily
convex combinations of the original samples.

  * •

The discriminator of ConvGeN learns to classify the synthetic points created
by the generator against a randomly selected and shuffled batch from the
majority class. The generator learns the coefficients of convex combinations
for synthetic data generation for every minority class data neighborhood in
such a way, that it also leads to an improved classification for the
discriminator. Note that, for this model, the discriminator can itself be used
as a minority-majority classifier. For conventional GANs this is not the
purpose of the discriminator. There, the discriminator differentiates whether
the given sample is real or synthetic.

To ensure that ConvGeN can learn the minority data manifold in every small
minority data neighborhood, it is trained using neighborhood data batches,
that is, each batch consists of minority data from a single minority
neighborhood. This is necessary to avoid the limitation of traditional GANs
that learn some portions of the data space and tend to generate synthetic
samples relevant only to the learned portions [18]. ConvGeN will be able to
learn the relevant convex space for all minority data neighborhoods and
therefore generate synthetic data at will from the convex space of any
minority neighborhood of choice.

Mathematical formalization of the ConvGeN model: For a given data point $x\in
X$, where $X$ is the minority class, we can denote the $k$-neighborhood of
$x\in X$ as $N(x)\subseteq X$. We assume that $N(x)$ (in short, we will
sometimes call this $N$) has been randomly shuffled. The Generator $G$ will
take as input $N(x)$ and map it to the convex space of $N(x)$, which we can
call $\mathbb{C}_{N(x)}$. Random shuffling of $N(x)$ is crucial, as it can
increase the number of training instances, if there is a less number of
samples. For example, with $k=10$, one can generate $10!=1024$ combinations
that can be treated as individual training samples. Also, ideally, when we
talk about learning the coefficients of convex combinations, customized to
each minority neighborhood, it means that these coefficients are learned for
any shuffle of the neighborhood since we assume no sense of order among data
points in a minority data neighborhood.

$G:N(x)\subseteq X\rightarrow C_{N}(x)\subseteq\mathbb{C}_{N(x)},$

where $C_{N}(x)$ (in short, we will call this $C_{N}$ sometimes) is a convex
combination of the samples in $N(x)$. Note that here we impose a constraint on
$G$ generating synthetic samples from the convex space of the original data
neighborhood. This ensures that the distribution of generated synthetic
samples, if considered as random variables from the corresponding minority
neighborhood, have identical mean as the distribution of the original samples.
This is evident as given identically distributed random variables
$X_{1},\dots,X_{n}$, with mean $\mathbb{E}[X_{i}]=\mu$ their convex
combination $Y=\sum_{i=1}^{n}\alpha_{i}X_{i}$ has the mean [3]

$\mathbb{E}[Y]=\mathbb{E}\bigg{[}\sum_{i=1}^{n}\alpha_{i}X_{i}\bigg{]}=\mu\sum_{i=1}^{n}\alpha_{i}=\mu=\mathbb{E}[X_{i}].$

Therefore, the objective of the ConvGeN model is to learn the optimal
coefficients for each data neighborhood, so that the synthetic samples are
distinct enough from the majority class, while also being similar to the
original samples. In simple words, the discriminator is designed to ensure
that the variance of the synthetic samples optimally learned, in the sense
that the synthetic samples do not spread too much outside the neighborhood.
The discriminator $D$, receives as input the synthetic samples generated from
a particular minority neighborhood $N(x)$ by $G$ and learns to classify them
against a random shuffled batch of majority samples $M$, so that
$|M|=|C_{N}|$.

$D:C_{N}\oplus M\subset X^{c}\rightarrow[0,1],$

where $\oplus$ denotes a concatenation and $X^{c}$, is the majority class,
which is the complement of $X$. $D$ thus ensures that the synthetic samples
generated as convex combinations of samples in $N(x)$ are distinct from the
majority data space. $D$ can pass on its knowledge to $G$ during the
interactive training process, such that $G$ learns to choose appropriate
convex coefficients for synthetic sampling generation within $N(x)$. Learning
the proper convex coefficients by $G$ can be intuitively interpreted as if the
model makes sure, the distribution of generated samples, in addition to having
an identical mean as the distribution of the original samples, also has an
optimal variance that assists the classification task.

The discriminator thus receives a sample, which can either be a majority
sample or a synthetic sample from the convex minority space. We used binary
cross-entropy loss to train the discriminator, which is defined as:

$L_{D}=-\sum_{c\in C_{N},y\in M}\log(D(c))+\log(1-D(y))$

There is no separate training for the generator, as we assume that the convex
combinations of minority data points generated by $G$ are already similar to
the original samples (distributions share identical means). The weights of $G$
are updated when the whole ConvGeN model is trained. The ConvGeN model uses
the mean-squared error as a loss function:

$L_{\text{ConvGeN}}=\text{MSE}(Y_{N\oplus M},\hat{Y}_{N\oplus M})$

where $Y_{N\oplus M}$ are the labels of the concatenated minority neighborhood
batch (shuffled) $N$ and the random majority batch $M$ (shuffled), while
$\hat{Y}_{N\oplus M}$ are the corresponding predicted labels. We provide the
pseudocode for ConvGeN in Algorithms 1 and 2.

Detailed architecture, parameters, and working mechanism of ConvGeN: An
adversarial learning process can be viewed as a min-max game, where the
discriminator seeks to maximize the quantity $-L(D)$ whereas the generator
seeks to achieve the reverse. For a conventional GAN, the discriminator tries
to minimize its own loss ($L_{D}$), whereas the generator tries otherwise. The
competition between the generator and the discriminator leads to synthetic
samples with distribution similar to the original data. However, in the case
of ConvGeN, $D$ and $G$ do not compete, but cooperate. $G$ does not try to
minimize $-L_{D}$ in ConvGeN, as it happens in case of GANs. $D$ receives the
original labels of the samples. The aim of ConvGeN is to learn the subset $S$
of the convex space of the minority class $\mathbb{C}_{X}$, such that sampling
from that subset can help the discriminator to minimize $L_{D}$, meaning that
the classification task is facilitated by the synthetic data generation
process. The underlying assumption is of course, that the generated synthetic
samples from $S$ guarantees a reasonable classification model. Note that,
given the performance of ProWRAS and LoRAS, this is a reasonable assumption to
make [3, 2]. Figure 1 depicts the architecture of ConvGeN.

We kept the architecture of ConvGeN relatively simple refraining from making
the neural networks too deep. In the architecture of $G$ in ConvGeN, the input
layer has $f$ nodes, $f$ being the number of features. At a time, it receives
a shuffled batch of neb original minority samples. The specialty is that each
input batch is a neighborhood $N$ (neb number of the closest data points) of
some minority class data point. Thus, the dimension of the input is neb$\times
f$. After the input layer a 1-$D$ convolution layer is used to reduce the
dimension of the input to $k^{\prime}\times f$, where
$k^{\prime}<\texttt{neb}$. Then the output of the convolution layer is
flattened and reshaped to obtain a matrix $K_{N}$ of dimension neb$\times$gen,
such that each column of $K_{N}$ sums to 1 and the entries of $K_{N}$ are non
negative (this is achieved using ReLU activation). Then a residual connection
with the input $N$ yields the matrix $C_{N}=K_{N}^{T}\times N$ ($K_{N}^{T}$
being the transpose of $K_{N}$), whose every row is a convex combination of
the minority neighborhood data points in $N$.

The architecture of $D$ in ConvGeN is straightforward. The input layer has $f$
nodes, $f$ being the number of features. At each training step, it receives a
batch of gen=neb synthetic samples $C_{N}$ from $G$ concatenated with a batch
of gen=neb majority samples $M$ as input. We used three dense layers of size
$250$, $125$, and $75$ respectively, while the output layer consists of two
nodes for two classes. The true labels of the inputs are also revealed to $D$,
such that, synthetic samples from $G$ are labeled as minority data. The
architecture of ConvGeN is shown in Figure 1.

Computational implementation: In terms of computational implementation, the
algorithm has four parameters: neb: size of the shuffled minority data
neighborhood that will be used for modeling the convex space of the minority
class, gen: number of generated synthetic points in form of convex
combinations of the minority neighborhood batch by generator at a time,
disc_train_count: number of extra training steps for the discriminator before
updating the generator once, and maj_proximal: an indicator for sampling the
majority batches either from the entire majority class or a borderline subset
of the majority class, that is, a subset of the majority class, which is
relatively closer to the minority class.

We have considered gen$=$neb for our model. The parameter maj_proximal, if set
to be true, the algorithm first detects a subset of the majority class that is
closest to the minority class, which we call the proximal majority set. This
is done by calculating the neb nearest neighbors of each minority sample in
the majority class (for the training data) and then taking the union of those
majority class samples. The proximal majority set is the union of the majority
class nearest neighbors of all minority samples/data points. We tested four
different settings of the ConvGeN model, shown in Table II. The motivation
behind this, is the practical usability of the model to handle larger tabular
datasets, which we have discussed later in more detail (see Section V:
ConvGeN(min,maj) model produces the best classification scores among different
ConvGeN parameter settings). However, for our comparative benchmarking study
on small tabular imbalanced datasets, we have consistently used the ConvGeN
(min,maj) setting.

Note that, in conventional deep learning, we use the term epoch to denote one
training step where a deep leaning model is trained on the entire training
data once. However, for ConvGeN, we use minority neighborhoods as input for
the generator. Therefore, in case of ConvGeN, we use the term neighborhood
epoch for a training step. During a training step $G$ receives all minority
neighborhoods as one input batch (each time concatenated with a randomly
shuffled majority batch of equal size), that is for $x\in X$, $N(x)$ is passed
through $G$ once. That is why in Algorithm 1, we see that the outer for loop
runs in the range $(1,$neb_epochs$)$. The entire training procedure is coded
in pseudocode for Algorithm 3. We used Python (V 3.8.10) for implementations
of the algorithms and experiments.

TABLE II: Different ConvGeN models that we have tested in our study, according
to parameter settings. These parameter settings for ConvGeN can be retrieved
as standard parameter settings and can be used as a starting point to optimize
the model for future works.

model name | parameter settings
---|---
ConvGeN(5,maj) |  neb$=5$, maj_proximal$=$False
ConvGeN(min,maj) |  neb$=|X|$, maj_proximal$=$False
ConvGeN(5,prox) |  neb$=5$, maj_proximal$=$True
ConvGeN(min,prox) |  neb$=|X|$, maj_proximal$=$True

Inputs :
data Labeled data with minority and majority class.

Parameters : disc_train_count | $\geq 0$ |  Extra training steps for the discriminator before updating generator once. (used value: $5$)
---|---|---
neb | $\geq 2$ | Size of the minority neighborhood used to model convex minority data space.
gen | $\geq\texttt{neb}$ |  Number of generated synthetic points by generator at a time. (used value: $\texttt{gen}=\texttt{neb}$)

Function train(data)

begin

$\texttt{labels}\leftarrow\big{(}\underbrace{(1,0),...,(1,0)}_{gen},\underbrace{(0,1),...,(0,1)}_{gen}\big{)}$

For $1\ldots\texttt{neb\\_epochs}$ do

For $1\ldots$ disc_train_count do

For $x\in$ minority class do

$\texttt{train\\_discriminator}(\texttt{x},\texttt{labels},\texttt{data})$

endfor

endfor

For $x\in$ minority class do

(concat_sample, min_batch, maj_batch)
$\leftarrow\texttt{train\\_discriminator}(\texttt{x},\texttt{labels},\texttt{data})$

train generator by using
$\texttt{discriminator}\big{(}(\texttt{generator}(\texttt{min\\_batch})~{}\oplus~{}\texttt{maj\\_batch})\big{)}$

with concat_sample $\mapsto$ labels

endfor

endfor

end

Algorithm 1 ConvGeN train algorithm

Inputs :
x Minority class data point. labels List of $2\texttt{gen}$ labels for the
training step. (1,0): item is in minority class. (0,1): item is in majority
class. data Labeled data with minority and majority class.

Parameters : neb | $\geq 2$ | Size of the neighborhood used to create one synthetic point.
---|---|---
gen | $\geq\texttt{neb}$ |  Number of generated synthetic points per step. (used value: $gen=neb$)
maj_proximal |  |  True: Train with gen points from nearest neighborhood for x in majority class.
|  |  False Train with gen random points in the majority class.

Function train_discriminator(x, labels, data)

begin

min_batch $\leftarrow$ Set of neb nearest points to x in the minority class in
randomized order.

conv_samples $\leftarrow$ generate gen points using the neighborhood of x.

If maj_proximal then

maj_batch $\leftarrow$ list of gen points in the nearest neighborhood of x in
the majority class in random order.

else

maj_batch $\leftarrow$ list of gen random points in the majority class.

endif

concat_sample $\leftarrow(\texttt{conv\\_samples}\oplus\texttt{maj\\_batch})$

train discriminator with concat_sample $\mapsto$ labels

return (concat_sample, min_batch, maj_batch)

end

Algorithm 2 ConvGeN train algorithm (discriminator)

##  IV Experimental protocols

###  IV-A General protocols for benchmarking

As far as data science related experiments are concerned, we have closely
followed the published work of Bej et al. [2]. In brief, for each dataset we
have used $5\times 5$ stratified cross-validation. This means we have used
five random shuffles of data and performed a $5$-fold cross validation on all
shuffles. The average values of the evaluation metrics or performance
measures, F1-score, and $\kappa$-score over all the shuffles and folds are
reported. Moreover, we ensured that the test data for any given fold in each
shuffle is not used for oversampling meaning that, we only used the training
data for this purpose.

Following the work of Bej et al. [2], we evaluated the classification model
performances using F1-score and Cohen’s $\kappa$-score. F1-score is the
harmonic mean of precision and recall and is a well-suited measure for the
overall classification performance for the minority class. Given a
classification problem, the $\kappa$ measure is formally defined as:

$\kappa=\frac{P_{o}-P_{e}}{1-P_{e}},$

where $P_{o}$ is the measure of observed agreement among the chosen classifier
and the ground truths of the classification problem. $P_{e}$ is the measure of
agreement by chance, among a chosen classifier and the ground truths of the
classification problem. The $\kappa$-score gives us a quantification of how
good the classification is, considering both the majority and the minority
class.

Inputs :
data List of data points with minority and majority class.

Parameters : n_folds | $\geq 1$ |  Number of parts the data is divided to. (used value: $5$)
---|---|---
n_shuffles | $\geq 1$ |  Number of benchmark steps to do. (used value: $5$)
classifiers |  | Set of classifiers to evaluate the algorithm with.

Function benchmark(gan, data)

begin

Shuffle data.

For $1\ldots\texttt{n\\_shuffles}$ do

Shuffle data.

$\texttt{fold}_{1},...,\texttt{fold}_{n\\_folds}\leftarrow$ split data equally
to n_folds parts. The proportion of each class in each part has to be the same
proportion of each class in the entire dataset. The last part might contain
less data-points.

For $k=1\ldots n\\_folds$ do

$\texttt{data\\_test}\leftarrow\texttt{fold}_{k}$

$\texttt{data\\_train}\leftarrow\texttt{fold}_{1}\cup\ldots\cup\texttt{fold}_{k-1}\cup\texttt{fold}_{k+1}\cup\ldots\cup\texttt{fold}_{\texttt{n\\_folds}}$

reset ConvGeN

train ConvGeN with data_train

$\texttt{n\\_synthetic}\leftarrow$ (size of majority class in data_train) $-$
(size of minority class in data_train)

$\texttt{data\\_synthetic}\leftarrow$ generate n_synthetic points with ConvGeN

$\texttt{data\\_train\\_full}\leftarrow\texttt{data\\_train}\cup\texttt{data\\_synthetic}$

For $\texttt{c}\in\texttt{classifiers}$ do

train c with data_train_full

predict data_test with c

score the prediction with $f_{1}$-score and $\kappa$-score

endfor

endfor

endfor

end

Algorithm 3 benchmark algorithm

###  IV-B Datasets and Algorithms used

TABLE III: Table showing statistics of datasets used for the benchmarking
study.

Dataset | Features | Total data size | Minority class size | Imbalance Ratio
---|---|---|---|---
abalone 17 vs 7 8 9 10 | 8 | 2338 | 58 | 39.31
abalone (9-18) | 8 | 731 | 42 | 16.40
car good | 6 | 1728 | 69 | 24.04
car vgood | 6 | 1728 | 65 | 25.58
flare-F | 11 | 1066 | 43 | 23.79
hypothyroid | 25 | 3163 | 151 | 19.95
kddcup guess passwd vs satan | 38 | 1642 | 53 | 29.98
kr-vs-k 3 vs 11 | 6 | 2935 | 81 | 35.23
kr-vs-k zero-one vs draw | 6 | 2901 | 105 | 26.63
shuttle 2 vs 5 | 9 | 3316 | 49 | 66.67
winequality red 4 | 11 | 1599 | 53 | 29.17
yeast4 | 10 | 1484 | 51 | 28.10
yeast5 | 10 | 1484 | 44 | 32.73
yeast6 | 10 | 1484 | 35 | 41.40

Our benchmarking datasets are a subset of the $104$ publicly available
imbalanced datasets used for the benchmarking studies in Kovács et al. [7]. In
Table III, we show the statistics for the relevant datasets.

Out of $104$ publicly available imbalanced datasets, satisfying all the
following three criteria, to ensure that our choice of datasets for the
studies are impartial, following the work of Bej et al. [2]. The criteria are:

  * •

Datasets with an imbalance ratio of at least $15:1$ were chosen. This is to
ensure that the performance of the compared oversampling algorithms are
tested, particularly on datasets with high imbalance following Bej et al. [2].

  * •

We choose datasets with a minority class samples in the range $35$ to $151$.
For datasets with very less minority class samples, classifier performances
are often affected by high stochastically and the results are often
statistically unreliable [2]. While too many samples provide the generative
networks too many training samples for synthetic sample generation, which is a
case outside the scope of this study, we are investigating only tabular
datasets of small size.

  * •

Datasets with at most $3500$ samples were used as we are investigating small
tabular data.

Recall that, we chose six oversampling algorithms for our benchmarking study,
which are discussed in more detail in II. Below we discuss the rationale
behind the choice of the oversampling algorithms for our benchmarking study.

  * •

The repeater model was chosen as the baseline for our benchmarking
experiments. In common articles on oversampling datasets without oversampling
are conventionally chosen as the baseline, which obviously perform worse than
the proposed models. The rationale behind our choice of this baseline is that,
even without the availability of any benchmarking algorithm, a modeler can
balance the classes by simply repeatedly sampling existing minority samples
with replacement, the effect of which has been unexplored.

  * •

The traditional GAN model is the pioneer of deep generative modeling, for
which it is important to compare ConvGeN against it.

  * •

Moreover, we used two variants CTGAN (2019) and CTAB-GAN (2021) of the GAN
model that are not only recent, but also extensively cited.

  * •

We used the ProWRAS algorithm for benchmarking, since it is a state-of-the-art
model for convex space learning, a philosophy transferred from ProWRAS into
ConvGeN.

ProWRAS has been used with default settings. The CTGAN model was trained for
$300$ (default value) epochs and columns with all integer values were marked
as discrete columns while for the rest of the model parameters the default
settings were used. The CTAB-GAN model was also trained for $300$ (default
value) epochs, while for the rest of the model parameters the default settings
were used. GAN, CTGAN, and CTAB-GAN were trained only on the minority class.
The GAN model that we used was also trained for $300$ epochs maintaining
consistency with CTGAN and CTAB-GAN. Finally, the ConvGeN model was trained
for $10$ neighborhood epochs for four different parameter settings as shown in
Table II. However, for comparative studies we only show the ConvGeN(min,maj)
model.

We used four commonly used traditional machine learning classifiers, following
the work of Bej et al. [2]. The Logistic Regression (LR) classifier was chosen
because it is a relatively fast model. The k-Nearest Neighbors (kNN) is also
observed to perform well for imbalanced datasets in the benchmarking studies
of Bej et al. [3, 2]. Moreover, we used Gradient Boosting (GB) and Random
Forest (RF) as ensemble models, as they are seen to perform well with the
ProWRAS algorithm [2]. Although ensemble models can perform well for
classification on small tabular imbalanced data, especially with a large
number of features, there is evidence that ensemble models are not robust [15,
19]. For instance, for GB, this occurs due to the working principle of
Boosting models where erroneous decisions on data points, which are difficult
to classify, are corrected through multiple iterations by making the model
learn these data points multiple times, hence over emphasizing on these data
points. One could therefore argue that, simpler models like kNN, albeit under-
performing, can be safer to use for decision-making on unseen data.
Considering this, we investigated multiple classifiers in our studies (LR,
kNN, RF and GB). We used default parameters as recommended in scikit-learn (V
0.21.2) documentation.

Moreover, recall that the discriminator of a trained ConvGeN model can itself
act as a classifier on the data generated by ConvGeN. This is why, for
ConvGeN-generated synthetic samples, we use the discriminator of the trained
ConvGeN model (DoC), also as a classifier. After the training of ConvGeN is
finished, we can use the generator to generate synthetic samples from each
neighborhood of the convex space. These synthetic samples are used to produce
a balanced dataset, which is used to retrain the discriminator for some epochs
(we used $10$ epochs). The purpose of this is that, the discriminator, while
training of ConvGeN, only sees random batches of the majority class of the
training data. This retraining ensures that the retrained discriminator
includes all majority class samples in its training.

TABLE IV: Table showing F1-score/$\kappa$\- score for compared oversampling strategies for LR classifier on 14 benchmarked imbalanced datasets. Models with the best performances are marked in bold for each dataset. ConvGeN yields the best average $\kappa$\- score and F1-Score.  dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | Repeater | ProWRAS | GAN | CTGAN | CTAB GAN | ConvGeN(min,maj)
---|---|---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.275 / 0.244 |  0.337 / 0.311 | 0.304 / 0.276 | 0.221 / 0.191 | 0.200 / 0.164 |  0.337 / 0.310
abalone9 18 | 0.458 / 0.409 | 0.570 / 0.538 | 0.462 / 0.418 | 0.341 / 0.288 | 0.389 / 0.332 |  0.575 / 0.544
car good | 0.106 / 0.035 | 0.100 / 0.029 |  0.162 / 0.101 | 0.106 / 0.035 | 0.109 / 0.038 | 0.109 / 0.038
car vgood | 0.362 / 0.320 | 0.392 / 0.354 | 0.377 / 0.337 | 0.331 / 0.287 | 0.354 / 0.311 |  0.422 / 0.387
flare F | 0.263 / 0.210 | 0.278 / 0.228 | 0.272 / 0.223 | 0.287 / 0.236 | 0.293 / 0.244 |  0.323 / 0.279
hypothyroid | 0.358 / 0.305 |  0.399 / 0.352 |  0.393 / 0.370 | 0.311 / 0.252 | 0.317 / 0.260 | 0.378 / 0.328
kddcup guess passwd vs satan | 0.997 / 0.996 | 0.996 / 0.996 | 0.981 / 0.980 |  1.000 / 1.000 |  1.000 / 1.000 | 0.995 / 0.995
kr vs k three vs eleven | 0.942 / 0.941 |  0.965 / 0.964 | 0.964 / 0.964 | 0.785 / 0.777 | 0.848 / 0.843 |  0.965 / 0.965
kr vs k zero one vs draw | 0.650 / 0.632 | 0.745 / 0.733 | 0.725 / 0.712 | 0.575 / 0.552 | 0.583 / 0.561 |  0.836 / 0.830
shuttle 2 vs 5 | 0.998 / 0.998 | 0.996 / 0.996 |  1.000 / 1.000 | 0.984 / 0.984 | 0.967 / 0.967 | 0.996 / 0.996
winequality red 4 | 0.113 / 0.056 |  0.143 / 0.090 | 0.077 / 0.044 | 0.127 / 0.080 | 0.109 / 0.051 | 0.135 / 0.080
yeast4 | 0.236 / 0.188 | 0.243 / 0.197 |  0.307 / 0.269 | 0.216 / 0.168 | 0.227 / 0.178 | 0.277 / 0.235
yeast5 | 0.574 / 0.555 | 0.591 / 0.574 | 0.595 / 0.577 | 0.522 / 0.500 | 0.511 / 0.488 |  0.620 / 0.604
yeast6 | 0.242 / 0.210 | 0.294 / 0.265 |  0.375 / 0.351 | 0.317 / 0.290 | 0.242 / 0.210 | 0.353 / 0.328
Average | 0.470 / 0.436 | 0.504 / 0.473 | 0.500 / 0.473 | 0.437 / 0.403 | 0.439 / 0.403 |  0.523 / 0.494
TABLE V: Table showing F1-score/$\kappa$\- score for compared oversampling strategies for RF classifier on 14 benchmarked imbalanced datasets. ConvGeN yields the second-best average $\kappa$\- score and F1-Score. Models with the best performances are marked in bold for each dataset. The Repeater model only outperforms the ConvGeN model strictly for 5 out of 14 datasets for both F1-score and $\kappa$\- score. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | Repeater | ProWRAS | GAN | CTGAN | CTAB GAN | ConvGeN(min,maj)
---|---|---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.190 / 0.180 | 0.169 / 0.161 | 0.256 / 0.242 | 0.239 / 0.220 |  0.329 / 0.310 | 0.280 / 0.268
abalone9 18 | 0.322 / 0.297 | 0.367 / 0.340 | 0.319 / 0.286 |  0.395 / 0.359 | 0.376 / 0.333 | 0.348 / 0.326
car good |  0.899 / 0.895 | 0.581 / 0.572 | 0.772 / 0.765 | 0.533 / 0.504 | 0.804 / 0.798 | 0.659 / 0.650
car vgood |  0.985 / 0.984 | 0.914 / 0.911 | 0.949 / 0.947 | 0.812 / 0.803 | 0.959 / 0.957 | 0.929 / 0.927
flare F | 0.222 / 0.178 | 0.095 / 0.074 | 0.127 / 0.107 |  0.294 / 0.245 | 0.121 / 0.102 | 0.120 / 0.099
hypothyroid |  0.800 / 0.791 | 0.725 / 0.714 | 0.777 / 0.767 | 0.774 / 0.760 | 0.777 / 0.767 | 0.798 / 0.788
kddcup guess passwd vs satan |  1.000 / 1.000 |  1.000 / 1.000 | 0.994 / 0.994 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000
kr vs k three vs eleven |  1.000 / 1.000 | 0.991 / 0.991 | 0.992 / 0.992 | 0.950 / 0.948 | 0.995 / 0.995 | 0.992 / 0.992
kr vs k zero one vs draw | 0.946 / 0.944 | 0.939 / 0.936 |  0.947 / 0.945 | 0.843 / 0.836 | 0.937 / 0.935 | 0.946 / 0.944
shuttle 2 vs 5 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000
winequality red 4 | 0.014 / 0.009 | 0.005 / -0.002 | 0.024 / 0.018 | 0.153 / 0.125 |  0.173 / 0.133 | 0.086 / 0.055
yeast4 | 0.320 / 0.306 |  0.329 / 0.313 | 0.284 / 0.272 | 0.284 / 0.252 | 0.311 / 0.300 | 0.321 / 0.304
yeast5 | 0.720 / 0.712 | 0.706 / 0.698 |  0.758 / 0.751 | 0.647 / 0.634 | 0.740 / 0.733 | 0.738 / 0.730
yeast6 | 0.518 / 0.508 | 0.570 / 0.562 | 0.493 / 0.485 | 0.453 / 0.437 | 0.530 / 0.522 |  0.575 / 0.567
Average | 0.638 / 0.629 | 0.599 / 0.591 | 0.621 / 0.612 | 0.598 / 0.580 |  0.647 / 0.635 | 0.628 / 0.618
TABLE VI: Table showing F1-score/$\kappa$\- score for compared oversampling strategies for GB classifier on 14 benchmarked imbalanced datasets. Models with the best performances are marked in bold for each dataset. ConvGeN yields the best average $\kappa$\- score and the second-best F1-Score. The Repeater model only outperforms the ConvGeN model strictly for 4 out of 14 datasets for both F1-score and $\kappa$\- score. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | Repeater | ProWRAS | GAN | CTGAN | CTAB GAN | ConvGeN(min,maj)
---|---|---|---|---|---|---
abalone 17 vs 7 8 9 10 |  0.333 / 0.312 | 0.323 / 0.304 | 0.202 / 0.194 | 0.234 / 0.219 | 0.204 / 0.198 | 0.291 / 0.278
abalone9 18 | 0.383 / 0.346 |  0.399 / 0.362 | 0.344 / 0.319 |  0.396 / 0.362 | 0.343 / 0.320 | 0.318 / 0.292
car good | 0.839 / 0.831 | 0.849 / 0.843 | 0.849 / 0.843 | 0.533 / 0.504 | 0.840 / 0.834 |  0.864 / 0.858
car vgood | 0.938 / 0.935 | 0.971 / 0.970 | 0.978 / 0.977 | 0.812 / 0.803 | 0.980 / 0.979 |  0.982 / 0.982
flare F | 0.285 / 0.237 | 0.132 / 0.111 | 0.231 / 0.214 |  0.315 / 0.268 | 0.225 / 0.208 | 0.181 / 0.164
hypothyroid | 0.781 / 0.768 | 0.795 / 0.785 | 0.808 / 0.799 | 0.774 / 0.760 |  0.810 / 0.801 | 0.800 / 0.789
kddcup guess passwd vs satan |  1.000 / 1.000 |  1.000 / 1.000 | 0.986 / 0.986 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000
kr vs k three vs eleven |  0.995 / 0.995 |  0.995 / 0.995 |  0.995 / 0.995 | 0.950 / 0.949 |  0.995 / 0.995 |  0.995 / 0.995
kr vs k zero one vs draw | 0.944 / 0.942 | 0.959 / 0.958 | 0.959 / 0.958 | 0.825 / 0.818 | 0.955 / 0.953 |  0.972 / 0.971
shuttle 2 vs 5 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 | 0.989 / 0.989 |  1.000 / 1.000
winequality red 4 | 0.127 / 0.094 | 0.099 / 0.073 | 0.066 / 0.054 |  0.130 / 0.104 | 0.128 / 0.086 | 0.107 / 0.066
yeast4 |  0.377 / 0.349 |  0.371 / 0.349 | 0.275 / 0.260 | 0.246 / 0.218 | 0.308 / 0.293 | 0.351 / 0.329
yeast5 | 0.714 / 0.705 | 0.690 / 0.680 |  0.746 / 0.739 | 0.617 / 0.603 | 0.716 / 0.707 | 0.724 / 0.716
yeast6 | 0.473 / 0.458 |  0.569 / 0.559 | 0.494 / 0.484 | 0.432 / 0.417 | 0.526 / 0.517 | 0.564 / 0.554
Average |  0.656 / 0.641 | 0.654 / 0.642 | 0.638 / 0.630 | 0.590 / 0.573 | 0.644 / 0.634 |  0.654 / 0.642
TABLE VII: Table showing F1-score/$\kappa$\- score for compared oversampling strategies for kNN classifier on 14 benchmarked imbalanced datasets. Models with the best performances are marked in bold for each dataset. Convex space modeling approaches ProWRAS and ConvGeN yield a better average performance compared to existing deep-generative models. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | Repeater | ProWRAS | GAN | CTGAN | CTAB GAN | ConvGeN(min,maj)
---|---|---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.315 / 0.291 |  0.336 / 0.313 | 0.272 / 0.243 | 0.180 / 0.173 | 0.280 / 0.268 | 0.315 / 0.296
abalone9 18 | 0.342 / 0.286 |  0.386 / 0.338 | 0.258 / 0.197 | 0.276 / 0.256 | 0.338 / 0.313 | 0.136 / 0.120
car good | 0.357 / 0.312 |  0.720 / 0.707 | 0.427 / 0.394 | 0.229 / 0.171 | 0.433 / 0.395 | 0.509 / 0.479
car vgood | 0.413 / 0.375 | 0.805 / 0.797 | 0.653 / 0.634 | 0.322 / 0.277 | 0.595 / 0.572 |  0.879 / 0.874
flare F | 0.245 / 0.191 | 0.293 / 0.249 | 0.270 / 0.228 | 0.267 / 0.214 | 0.293 / 0.246 |  0.306 / 0.265
hypothyroid | 0.568 / 0.540 | 0.615 / 0.592 | 0.573 / 0.547 |  0.618 / 0.595 | 0.565 / 0.537 | 0.606 / 0.582
kddcup guess passwd vs satan |  0.991 / 0.991 |  0.991 / 0.991 |  0.991 / 0.991 |  0.991 / 0.991 |  0.991 / 0.991 |  0.991 / 0.991
kr vs k three vs eleven | 0.901 / 0.898 | 0.932 / 0.929 | 0.903 / 0.900 | 0.804 / 0.797 | 0.807 / 0.801 |  0.961 / 0.960
kr vs k zero one vs draw | 0.840 / 0.833 | 0.885 / 0.880 | 0.828 / 0.821 | 0.767 / 0.756 | 0.786 / 0.776 |  0.897 / 0.892
shuttle 2 vs 5 | 0.951 / 0.951 |  0.956 / 0.955 | 0.936 / 0.935 | 0.951 / 0.951 | 0.949 / 0.948 | 0.932 / 0.931
winequality red 4 | 0.069 / 0.016 | 0.072 / 0.020 |  0.085 / 0.028 | 0.047 / 0.001 | 0.057 / 0.003 | 0.058 / -0.001
yeast4 | 0.329 / 0.293 |  0.363 / 0.332 | 0.332 / 0.294 | 0.304 / 0.267 | 0.313 / 0.273 | 0.348 / 0.313
yeast5 | 0.670 / 0.657 |  0.691 / 0.679 | 0.598 / 0.581 | 0.542 / 0.522 | 0.507 / 0.484 | 0.646 / 0.631
yeast6 | 0.409 / 0.387 |  0.463 / 0.445 | 0.363 / 0.339 | 0.383 / 0.360 | 0.304 / 0.276 | 0.426 / 0.406
Average | 0.529 / 0.502 |  0.608 / 0.588 | 0.535 / 0.509 | 0.477 / 0.452 | 0.516 / 0.492 | 0.572 / 0.553

##  V Results

Figure 2: Average by oversampling algorithms for $f_{1}~{}$score (on the left)
and $\kappa~{}$score (on the right) for different classifiers over 14
benchmarking datasets. Ensemble models yield better average classification
performance.

Figure 3: Average by classifier for $f_{1}~{}$score (on the left) and
$\kappa~{}$score (on the right) for different oversampling models over 14
benchmarking datasets. Note that the ConvGeN(min,maj) model performs either
the best or the second best for all classifiers. For every classifier, it
performs better than the existing deep-generative models.

Ensemble models perform better on smaller tabular datasets. (See Figure 2) Our
benchmarking studies indicate that ensemble models GB and RF perform better in
terms of both the F1-score and $\kappa$-score, for smaller tabular datasets.
Such models rely on an ensemble of weak classifiers for decision-making.
However, we would like to point out here that all our benchmarking datasets
contain a relatively low number of features (see Table III). Such a scenario
is ideal for ensemble models. For high-dimensional datasets, such models tend
to be less robust. In such a scenario, simpler models are more reliable [19].
We also noticed that the kNN is consistently better than the LR. Moreover, for
the ProRWAS oversampling algorithm, the kNN performs even better than the RF
model contrary to Bej et al., as we did not customize ProWRAS for RF, that is
to choose the proper oversampling scheme for every dataset [2]. The
performance of the DoC classifier is highly comparable to the RF model.
However, we notice that, it performs better than the kNN and LR classifier for
all oversampling algorithms.

The baseline Repeater model is compatible with ensemble models. (See Figure 3)
An interesting phenomenon that we inferred from our results is that the
baseline repeater model performs the second best for RF in terms of both
F1-Score and $\kappa$-Score. For GB, it performs best in terms of F1-Score and
third best in terms of $\kappa$-Score. This shows that for ensemble models on
datasets with fewer features, there might be not much need to use complex
oversampling algorithms. Rather, balancing the classes by simply bootstrapping
on the minority class can be good enough. However, for simpler models, such as
kNN and LR, which might be more effective in specific scenarios, we observe
that this approach does not improve the classification performance. Synthetic
sample generation using convex space learning is clearly the most effective
strategy of data rebalancing in such cases.

ConvGeN performs better than existing state-of-the-art deep-generative models
for tabular datasets in the context of imbalanced classification. (See Figure
3) It is evident from our experiments that ConvGeN(min,maj) outperforms the
conventional GAN model, as well as the CTGAN models for smaller tabular
imbalanced datasets for every classifier. CTAB-GAN performs better than
ConvGeN only for the RF classifier. Interestingly, for smaller tables, we
observed that conventional GAN works better than CTGAN for all classifiers.
GAN also performs better than CTAB-GAN for all non-ensemble classifiers (LR
and kNN). On the other hand, among the two GAN variants, CTAB-GAN performs
better than CTGAN consistently for all classifiers.

Convex space learning approaches are better for simpler models like kNN and LR
models. (See Figure 3) Convex space learning approaches are better for simpler
models like kNN and LR models. For the kNN algorithm however, the ProWRAS
approach clearly outperforms ConvGeN(min,maj), while ConvGeN(min,maj) still
performs better than the other GAN based approaches. For the LR algorithm, the
ConvGeN(min,maj) approach clearly outperforms ProWRAS, while ProWRAS still
performs better than the other GAN based approaches. Since both ProWRAS and
ConvGeN(min,maj) rely on the philosophy of convex space learning, we can
conclude that this approach is better suited for simpler classifiers, such as
kNN and LR, compared to existing deep-generative approaches.

Convex space learning is more suitable for oversampling on smaller tabular
imbalanced datasets: (See table VIII) We have ranked the oversampling models
according to their performance the classifiers GB, RF, kNN, LR based on F1 and
$\kappa$ scores. We observe that ConvGeN produces the best average rank for
both the performance measures, while ProWRAS produces the second-best average
rank. Interestingly these are the only two models that produce an average rank
better than the baseline Repeater model. Thus, we can conclude that convex
space learning is a more suitable synthetic sample generation strategy
compared to deep adversarial learning, when the synthetic samples are to be
used for class rebalancing, especially for smaller datasets.

TABLE VIII: Oversampling models are ranked according to their performance the
classifiers GB, RF, kNN, LR based on F1 and $\kappa$ scores. ConvGen(min,maj)
produces the best average rank over all classifiers and ProWRAS produces the
second best performance.

| Repeater | ProWRAS | GAN | CTGAN | CTAB-GAN | ConvGen(min,maj)
---|---|---|---|---|---|---
GB (F1-score) | 1 | 2 | 5 | 6 | 4 | 3
RF (F1-score) | 2 | 5 | 4 | 6 | 1 | 3
kNN (F1-score) | 4 | 1 | 3 | 6 | 5 | 2
LR (F1-score) | 4 | 2 | 3 | 6 | 5 | 1
Average rank (F1-score) | 2.75 | 2.5 | 3.75 | 6 | 3.75 | 2.25
GB ($\kappa$-score) | 3 | 2 | 5 | 6 | 4 | 1
RF ($\kappa$-score) | 2 | 5 | 4 | 6 | 1 | 3
kNN ($\kappa$-score) | 4 | 1 | 3 | 6 | 5 | 2
LR ($\kappa$-score) | 4 | 2 | 3 | 6 | 5 | 1
Average rank ($\kappa$-score) | 3.25 | 2.5 | 3.75 | 6 | 3.75 | 1.75

TABLE IX: The p-values for a Wilcoxon’s signed rank test comparing ConvGeN,
with each of the benchmarked oversampling models, and performance metric,
pairwise. Case studies for which ConvGeN improves significantly over a
compared model (p-value$<0.05$) are marked in bold font.

| Repeater | ProWRAS | GAN | CTGAN | CTAB-GAN
---|---|---|---|---|---
GB (F1-score) | 0.85 | 0.62 | 0.23 | 0.05 | 0.38
GB ($\kappa$-score) | 1 | 0.85 | 0.30 | 0.04 | 0.43
RF (F1-score) | 0.92 | 0.02 | 0.27 | 0.20 | 0.23
RF ($\kappa$-score) | 1 | 0.01 | 0.28 | 0.11 | 0.34
kNN (F1-score) | 0.13 | 0.13 | 0.03 | 0.03 | 0.03
kNN ($\kappa$-score) | 0.12 | 0.13 | 0.03 | 0.03 | 0.03
LR (F1-score) | 0.00 | 0.03 | 0.15 | 0.00 | 0.00
LR ($\kappa$-score) | 0.00 | 0.05 | 0.24 | 0.00 | 0.00

TABLE X: Table showing F1-score/$\kappa$-score comparisons among different parameter settings for ConvGeN. The ConvGeN(min,maj) model is seen to perform the best on average for all the tested classifiers.  dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | ConvGeN(5,maj) | ConvGeN(min,maj) | ConvGeN(5,prox) | ConvGeN(min,prox)
---|---|---|---|---
LR-Average | 0.489 / 0.457 |  0.523 / 0.494 | 0.488 / 0.457 | 0.515 / 0.486
RF-Average | 0.600 / 0.591 |  0.628 / 0.618 | 0.604 / 0.594 | 0.622 / 0.611
GB-Average | 0.627 / 0.615 |  0.654 / 0.642 | 0.631 / 0.619 | 0.648 / 0.637
kNN-Average | 0.557 / 0.532 |  0.572 / 0.553 | 0.557 / 0.532 | 0.569 / 0.550
DoC-Average | 0.611 / 0.590 |  0.625 / 0.609 | 0.614 / 0.594 | 0.614 / 0.596

Following the work of Bej et al., we also tested the performance of the
ConvGeN(min,maj) model against other oversampling models using Wilcoxon’s
signed rank test[2]. We used the scipy.stats library for this test. A
limitation of the library is that exact p-value calculation does not work if
there are ties. Since there are ties in our results, the test produces
approximate measures. The results of the test are presented in Table IX. We
observe that ConvGeN(min,maj) significantly improves model performance for non
ensemble models, such as kNN and LR, compared to existing deep-generative
models, especially CTGAN and CTAB-GAN, confirming that convex space modeling
is indeed more suitable for tabular imbalanced datasets of small size compared
to deep adversarial learning.

ConvGeN(min,maj) model produces the best classification scores among different
ConvGeN parameter settings. We tested four different parameter settings for
the ConvGeN model. We have provided in Table X a comparison of the ConvGeN
model with these four parameter settings. For all the classifiers that we have
tested, including the DoC classifier, we observe that ConvGeN(min,maj)
performs the best. These parameter settings for ConvGeN mean that we consider
the entire minority class as the minority neighborhood and use no constraint
on the majority class. However, it is important to note that all datasets
under investigation for benchmarking have a small minority class. That is why
the parameter neb can be an important addition to the model for datasets
dealing with larger data with possibly a larger minority class. Interestingly,
we observe that among the models ConvGeN(5,maj) and ConvGeN(5,prox), the
latter works better. From this, we conclude that restricting sampling majority
batches from a proximal majority subset can be useful when we are modeling
smaller minority neighborhoods, that is when we are using a smaller value for
the parameter neb. This indicates that both these parameters could be useful
in improving the applicability of the model.

##  VI Discussion

Figure 4: Figure showing synthetic sample generation for different models on
three datasets with diverse data distributions. We observe that approaches
based on the philosophy of convex space modeling, especially ProWRAS and
ConvGeN(5,maj) generate synthetic data that are visually closer to the
minority data space.

First, we point out some limitations of the ConvGeN model. The model does not
include a mechanism for categorical data generation in particular. This might
limit its applicability for synthetic data generation of tabular data, in
general. Additionally, the model has not been tested for multi-class
classification. Nevertheless, the model can certainly be adapted for multi-
classification, simply by choosing the discriminator to be multi-class and
training the generator for all minority classes. However, well-established
multi-class imbalanced datasets for benchmarking were not found in the
literature from related works. Moreover, we also did not experiment
extensively with the complexity and architecture of the model, such as
increasing the depth of the neural networks, use of activation functions,
regularization, loss functions, and optimizers. This was due to the fact that
the benchmarking datasets were mostly low dimensional and therefore, using
simple architectures would already provide us with a good enough idea about
the effect of convex space learning on tabular imbalanced classification on
small datasets.

In this article, we introduced the ConvGeN model for synthetic data generation
particularly in the context of imbalanced classification. However, the scope
of synthetic sample generation can also be set on a larger scale. In
particular, in clinical data science, synthetic data can be used to likewise
build and optimize machine learning models and protect the privacy of patients
[11]. Note that the ConvGeN model can be easily adapted to generate synthetic
data from entire datasets. In such a case, we can train the generator using
neighborhood batches from entire datasets, and the discriminator can be
trained to differentiate a data neighborhood from a randomly sampled shuffled
batch from the complement of the neighborhood, which is passed to the
generator during one training iteration. The distribution of the real data can
be considered to decide on the distribution of the synthetic data. However,
the current version of ConvGeN does not have a particular strategy for
generating categorical data in particular, which would be crucial to broaden
its applicability for future applications. Unfortunately, the benchmarking
datasets commonly available for imbalanced classification mostly lack diverse
categorical data types, for which the applicability of ConvGeN could not be
investigated. A more general version of the ConvGeN algorithm can find
applications in several research domains, such as clinical and finance. Such a
model could help circumvent real world problems like prevention of data
sharing due to privacy issues and class imbalance in clinical trials [11, 12]
and as byproducts can generate necessary benchmarking datasets for categorical
tabular imbalanced classification. However, some crucial observations arise
from this research, which are discussed below.

Deep-generative models, since their inception, have received a lot of
attention from data scientists. A vast amount of research has been done on
GANs in the domain of synthetic image generation. The capability of GANs to
create realistic images have been proven time and again. However, for image
datasets, we have a perceptional advantage, in the sense, that we can visually
judge how realistic the synthetic image is, just by looking at it. In
particular, for a medical image, a professional radiologist can easily verify
whether a GAN-generated image looks realistic enough. This method of
validation, popularly used in clinical sciences is called a Visual Turing test
[13]. Unfortunately for tabular datasets, we lose the advantage of clear
visual perception and there is no known analog for a Visual Turing test for
tabular data. That is why for such datasets, it is difficult to judge how
realistic the generated data is. The best we can do is to construct a two or
three dimensional projection, using semi-supervised algorithms.

In Figure 4, we show Principal component analysis (PCA) plots of real and
synthetic data generated by the oversampling models: ProWRAS, GAN, CTAB-GAN,
CTGAN, and ConvGeN(5,maj) for three datasets from our benchmarking study:
‘abalone9 18’, ‘hypothyroid’, and ‘winequality red 4’. Note that the ’abalone9
18’ dataset has three clusters, each of which have some minority samples, the
‘hypothyroid’ dataset has two clusters where the minority samples are present
in only one cluster and there are no particularly visible clusters in the
‘winequality red 4’ dataset. We can observe that the data generated by ProWRAS
and ConvGeN(5,maj) are well-embedded in the minority data space and hence can
be interpreted visually as realistic compared to the rest.

Our experiments also raise the question, whether synthetic sample generation
should be task-specific, that is, whether different approaches/algorithms of
synthetic data generation are more effective/applicable for diverse machine-
learning tasks, say unsupervised data stratification or classification. A
common perception might be that the more realistic the synthetic samples, the
better the classification. However, it is clear from Figure 4, that this is
not necessarily the case. For example, for the hypothyroid dataset, the RF
model along with the generation of synthetic samples using CTAB-GAN is the
best among all models. But from Figure 4, visual perception at least indicates
that the synthetic data generated by CTAB-GAN for this dataset has the highest
variance or spread among all the models. Our results can be considered as a
strong evidence that convex space modeling does improve classifier
performances compared to existing deep-generative models, such as GAN, CTGAN,
and CTAB-GAN for smaller imbalanced tabular datasets. Moreover, the synthetic
sample distribution is visually more consistent with the original data
distribution.

##  VII Conclusion

Our conducted research revealed significant findings in the context of
imbalanced classification for which ensemble models have been so far known to
perform better. Linear interpolation-based models that generate synthetic
samples from the convex space of the minority class outperform nonlinear deep-
generative models for small-size tabular imbalanced datasets. We also conclude
that visually realistic synthetic data generation (based on PCA plots) does
not necessarily guarantee good classification performance for such datasets.
Thereby, we point out the scope of research for task-specific synthetic data
generation. ConvGeN improves imbalanced classification as compared to existing
deep-generative models, such as GAN, CTGAN, and CTAB-GAN for tabular
imbalanced datasets of small size. In conclusion, we assert that convex space
learning has a broad applicative potential, even outside the context of
imbalanced classification, since the synthetic sample distribution is visually
more consistent with the original data distribution, in addition to improving
imbalanced classification.

## Author Contribution

SB proposed the basic architecture of ConvGeN. KS experimented with the idea
extensively to improve it further and developed the final ConvGeN
architecture. KS worked on the computational implementation and benchmarking
studies. KS and SB wrote the first version of the manuscript and are the joint
first authors of this work. WH, MW, PS, and OW critically discussed and
revised the approach and content of the manuscript. All the authors have read
and agreed to the contents of the manuscript.

## Conflict of Interest

The authors have no conflict of interest.

## Availability of code and results

To support transparency, re-usability, and reproducibility, we provide an
implementation of the algorithm for binary classification problems using
Python (V 3.8.10) and several Jupyter Notebooks from our benchmarking study in
a ConvGeN GitHub repository. Other important python library versions we used
are keras (V 2.8.0), torch (V 1.11.0), tensorflow (V 2.8.0), scikit-learn (V
1.0.2), numpy (V 1.22.3) and ctgan (V 0.5.1).

## Acknowledgment

We thank the German Network for Bioinformatics Infrastructure (de.NBI) and
Establishment of Systems Medicine Consortium in Germany e:Med for their
support, as well as the German Federal Ministry for Education and Research
(BMBF) program (FKZ 01ZX1709C) and the EU Social Fund (ESF/14-BM-A55-0027/18)
for funding.

## References

  * [1] Baowaly Mrinal Kanti, Lin Chia-Ching, Liu Chai-Lin, Chen Kuan-Ta. Synthesizing electronic health records using improved generative adversarial networks. _J Am Med Informatics Assoc_ , 2019;26:228–241. https://doi.org/10.1093/JAMIA/OCY142
  * [2] Bej Saptarshi, Schultz Kristian, Srivastava Prashant, Wolfien Markus, Wolkenhauer Olaf. A Multi-Schematic Classifier-Independent Oversampling Approach for Imbalanced Datasets. _IEEE Access_ , 2021;9:123358-123374. https://doi.org/10.1109/ACCESS.2021.3108450
  * [3] Bej Saptarshi, Davtyan Narek, Wolfien Markus, Nassar Miriam, Wolkenhauer Olaf LoRAS: an oversampling approach for imbalanced datasets. _Machine Learning_ , 2021;110:279–301. https://doi.org/10.1007/s10994-020-05913-4
  * [4] Blagus Rok, Lusa Lara. SMOTE for high-dimensional class-imbalanced data. _BMC Bioinformatics_ , 2013;14:106. https://doi.org/10.1186/1471-2105-14-106
  * [5] Chawla N, Bowyer K, Hall L, Kegelmeyer W. SMOTE: Synthetic Minority Over-sampling Technique. _J. Artif. Intell. Res. (JAIR)_ , 2002;16:321-357. https://doi.org/10.1613/jair.953
  * [6] Goodfellow Ian, Pouget-Abadie Jean, Mirza Mehdi, Xu Bing, Warde-Farley David, Ozair Sherjil, Courville Aaron, Bengio Yoshua. Generative Adversarial Networks _Communications of the ACM_ , November 2020;63(11):139-144. https://doi.org/10.1145/3422622
  * [7] György Kovács. An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets. _Applied Soft Computing_ , 2019;83:105662. ISSN 1568-4946. https://doi.org/10.1016/j.asoc.2019.105662
  * [8] Hekler Eric B, Klasnja Predrag, Chevance Guillaume, Golaszewski Nataline M, Lewis Dana, Sim Ida. Why we need a small data paradigm. _BMC Med 17_ , 2019:133. https://doi.org/10.1186/s12916-019-1366-x
  * [9] Karras Tero, Aila Timo, Laine Samuli, Lehtinen Jaakko. Progressive Growing of GANs for Improved Quality, Stability, and Variation. _arXiv_ , 2018. https://doi.org/10.48550/arXiv.1710.10196
  * [10] Karras Tero, Aittala Miika, Hellsten Janne, Laine Samuli, Lehtinen Jaakko, Aila Timo. Training Generative Adversarial Networks with Limited Data. _arXiv_ , 2020. https://arxiv.org/abs/2006.06676
  * [11] Liaw Siaw-Teng, Liyanage Harshana, Kuziemsky Craig, Terry Amanda L, Schreiber Richard, Jonnagaddala Jitendra, de Lusignan Simon. Ethical Use of Electronic Health Record Data and Artificial Intelligence: Recommendations of the Primary Care Informatics Working Group of the International Medical Informatics Association. _Yearbook of Medical Informatics_ , 2020;29(01):051-057; https://doi.org/10.1055/S-0040-1701980
  * [12] Abay Nazmiye Ceren, Zhou Yan, Kantarcioglu Murat, Thuraisingham Bhavani, Sweeney Latanya. Privacy Preserving Synthetic Data Release Using Deep Learning. _Machine Learning and Knowledge Discovery in Databases_ , 2019:510-526. Edited by Berlingerio Michele, Bonchi Francesco, Gärtner Thomas, Hurley Neil, Ifrim Georgiana. Publisher: Springer International Publishing.
  * [13] Park Ho Young, Bae Hyun-Jin, Hong Gil-Sun, Kim Minjee, Yun JiHye, Park Sungwon, Chung Won Jung, Kim NamKug Realistic High-Resolution Body Computed Tomography Image Synthesis by Using Progressive Growing Generative Adversarial Network: Visual Turing Test. _JMIR Med Inform_ , 2021;9(3):e23328 https://doi.org/10.2196/23328
  * [14] Park N, Mohammadi M, Gorde K, Jajodia S, Park H, Kim Y. Data Synthesis based on Generative Adversarial Networks. _Proceedings of the VLDB Endowment_ ; 2018;11(10):1071–1083. https://doi.org/10.14778/3231751.3231757
  * [15] Park Yubin and Ho Joyce. Tackling Overfitting in Boosting for Noisy Healthcare Data. _IEEE Transactions on Knowledge and Data Engineering_ , 2022;33(7): 2995-3006. https://doi.org/10.1109/TKDE.2019.2959988
  * [16] Richardson Elad, Alaluf Yuval, Or Patashnik, Nitzan Yotam, Azar Yaniv, Shapiro Stav, Cohen-Or Daniel. Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation. _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2021: 2287-2296. https://doi.org/10.1109/CVPR46437.2021.00232
  * [17] Sandfort Veit, Yan Ke, Pickhardt Perry J, Summers Ronald M. Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. _Scientific Reports_ , 2019;9;16884. https://doi.org/10.1038/s41598-019-52737-x
  * [18] Wei Li, Li Fan, Zhenyu Wang, Ma Chao, Cui Xiaohui. Tackling mode collapse in multi-generator GANs with orthogonal vectors. _Pattern Recognition_ , 2021;110:107646. ISSN 0031-3203. https://doi.org/10.1016/j.patcog.2020.107646
  * [19] Xu Baoxun, Huang Joshua Zhexue, Williams Graham, Wang Qiang, Ye Yunming. Classifying Very High-Dimensional Data with Random Forests Built from Small Subspaces. _International Journal of Data Warehousing and Mining (IJDWM)_ , 2012;8(2):44-63. http://doi.org/10.4018/jdwm.2012040103
  * [20] Xu Lei, Veeramachaneni Kalyan. Synthesizing Tabular Data using Generative Adversarial Networks. _arXiv_ , 2018. https://arxiv.org/abs/1811.11264
  * [21] Xu Lei, Skoularidou Maria, Cuesta-Infante Alfredo, Veeramachaneni Kalyan, Modeling Tabular data using Conditional GAN, _Advances in Neural Information Processing Systems_ , 2019;32. doi=10.48550/ARXIV.1907.00503 https://arxiv.org/abs/1907.00503
  * [22] Lin Zinan, Khetan Ashish, Fanti Giuila, Oh Sewoong. PacGAN: The Power of Two Samples in Generative Adversarial Networks. _IEEE Journal on Selected Areas in Information Theory_ , 2020;1(1):324-335. https://doi.org/10.1109/JSAIT.2020.2983071
  * [23] Zhao Zilong, Kunar Aditya, Birke Robert, Chen Lydia Y. CTAB-GAN: Effective Table Data Synthesizing _Proceedings of The 13th Asian Conference on Machine Learning_ , 2021:97-112. https://proceedings.mlr.press/v157/zhao21a/zhao21a.pdf

|  Kristian Schultz received his first degrees in mathematics and computer
science in 2014. After that, he worked for two years in the field of discrete
mathematics on Sperner families in the Deptartment of Mathematics at the
University in Rostock, Germany. In between, he extended his software
developing skills in industry. Since 2020 he returned to the University of
Rostock, Department of Systems Biology and Bioinformatics, where he focuses on
the correctness, efficiency, and implementation of algorithms.
---|---
|  Saptarshi Bej received his Master’s degree in Mathematics from Indian
Institute of Science Education and Research, Kolkata in 2014. In 2021, he
completed his PhD in Computer Science from the University of Rostock, Germany.
He is currently a research associate at the Institute of Computer Science,
University of Rostock, Germany and a guest scientist in the Leibniz-Institute
for Food Systems Biology at the Technical University of Munich, Germany. His
primary research focus is on the development of machine learning algorithms
and their applications in the life sciences. His special interests include
machine learning on small and imbalanced datasets and literature mining.
---|---
|  Waldemar Hahn received his Master’s degree in Computer Science in 2019.
Since 2020 he works as a research assistant at the Center for Medical
Informatics within the Institute for Medical Informatics and Biometry (Faculty
of Medicine Carl Gustav Carus, Technische Universität Dresden). His current
research is related to the generation and evaluation of tabular data in
healthcare.
---|---
|  Markus Wolfien received his Master’s degree in Medical Biotechnology and
completed his PhD on the topic ‘Customized workflow development and omics data
integration concepts in systems medicine’ from the University of Rostock. He
is currently leading the “Data Science” research group at the Center for
Medical Informatics within the Institute for Medical Informatics and Biometry
(Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden). He
focuses on the application of AI-based approaches and develops flexible
workflows for the joint analysis of low and high-throughput data, such as
clinical blood measurements, protein expression, RNA sequencing data (bulk,
single cell, and spatial), and environmental information. To achieve this, he
combines state-of-the-art tools including R and Python, as well as further
downstream analysis approaches, such as network analyses and machine learning
in general (classical ML and Deep Learning).
---|---
|  Prashant Srivastava received his Master’s degree in Physics from Indian
Institute of Science Education and Research, Kolkata. After that he did a
second Master’s degree in Data Science and Analytics from Department of
Computer Science, Royal Holloway, University of London. He is currently a
research assistant in the Department of Systems Biology and Bioinformatics,
Institute of Computer Science at the University of Rostock, Germany. His
current research interests are algorithm development and their applications in
literature mining.
---|---
|  Olaf Wolkenhauer received his first degrees in systems and control
engineering and his PhD for research in possibility theory with applications
to data analysis. He spent over ten years at the University of Manchester
Institute of Science and Technology in the UK. In 2005, he became a fellow of
the Stellenbosch Institute for Advanced Study and holds professorships at Case
Western Reserve University, USA and Chhattisgarh Swami Vivekanand Technical
University, India. In 2003, he was appointed as professor for systems biology
and bioinformatics at the University of Rostock, Germany. In 2015, he was
elected a member of the Foundations in Medicine and Biology review panel of
the German Research Foundation (DFG). His research combines data-driven
modelling with model-driven experimentation, using a wide range of approaches,
including machine learning, and systems theory.
---|---

##  VIII Supplementary data

TABLE XI: Table showing F1-score/$\kappa$-score for different parameter settings for ConvGeN, using the LR classifier. The ConvGeN(min,maj) model is seen to perform the best for all the tested classifiers. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | ConvGeN(5,maj) | ConvGeN(min,maj) | ConvGeN(5,prox) | ConvGeN(min,prox)
---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.292 / 0.262 | 0.337 / 0.310 | 0.292 / 0.261 |  0.344 / 0.318
abalone9 18 | 0.538 / 0.500 |  0.575 / 0.544 | 0.541 / 0.503 | 0.574 / 0.543
car good | 0.108 / 0.038 | 0.109 / 0.038 | 0.107 / 0.036 | 0.110 / 0.039
car vgood | 0.373 / 0.332 |  0.422 / 0.387 | 0.377 / 0.337 | 0.400 / 0.362
flare F | 0.273 / 0.223 | 0.323 / 0.279 | 0.276 / 0.226 | 0.317 / 0.274
hypothyroid | 0.363 / 0.312 | 0.378 / 0.328 | 0.360 / 0.308 |  0.429 / 0.386
kddcup guess passwd vs satan | 0.989 / 0.989 | 0.995 / 0.995 | 0.994 / 0.994 |  0.998 / 0.998
kr vs k three vs eleven | 0.952 / 0.951 |  0.965 / 0.965 | 0.948 / 0.947 | 0.963 / 0.962
kr vs k zero one vs draw | 0.700 / 0.686 |  0.836 / 0.830 | 0.710 / 0.696 | 0.761 / 0.750
shuttle 2 vs 5 | 0.994 / 0.994 | 0.996 / 0.996 | 0.989 / 0.989 | 0.992 / 0.992
winequality red 4 | 0.130 / 0.075 | 0.135 / 0.080 | 0.126 / 0.071 | 0.131 / 0.076
yeast4 | 0.254 / 0.209 | 0.277 / 0.235 | 0.249 / 0.204 | 0.276 / 0.234
yeast5 | 0.582 / 0.564 |  0.620 / 0.604 | 0.577 / 0.559 | 0.601 / 0.584
yeast6 | 0.295 / 0.266 | 0.353 / 0.328 | 0.290 / 0.261 | 0.316 / 0.289
Average | 0.489 / 0.457 |  0.523 / 0.494 | 0.488 / 0.457 | 0.515 / 0.486
TABLE XII: Table showing F1-score/$\kappa$-score for different parameter settings for ConvGeN, using the RF classifier. The ConvGeN(min,maj) model is seen to perform the best for all the tested classifiers. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | ConvGeN(5,maj) | ConvGeN(min,maj) | ConvGeN(5,prox) | ConvGeN(min,prox)
---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.234 / 0.222 | 0.280 / 0.268 | 0.253 / 0.240 |  0.283 / 0.270
abalone9 18 | 0.324 / 0.291 | 0.348 / 0.326 | 0.335 / 0.302 | 0.359 / 0.337
car good | 0.634 / 0.624 | 0.659 / 0.650 | 0.655 / 0.646 | 0.647 / 0.638
car vgood | 0.910 / 0.907 | 0.929 / 0.927 | 0.904 / 0.901 | 0.921 / 0.918
flare F | 0.062 / 0.043 | 0.120 / 0.099 | 0.064 / 0.044 | 0.133 / 0.113
hypothyroid | 0.782 / 0.772 |  0.798 / 0.788 | 0.791 / 0.781 | 0.796 / 0.786
kddcup guess passwd vs satan |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000
kr vs k three vs eleven | 0.993 / 0.993 | 0.992 / 0.992 | 0.993 / 0.993 | 0.992 / 0.992
kr vs k zero one vs draw | 0.941 / 0.939 | 0.946 / 0.944 | 0.937 / 0.935 |  0.949 / 0.947
shuttle 2 vs 5 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000 |  1.000 / 1.000
winequality red 4 | 0.079 / 0.062 | 0.086 / 0.055 | 0.067 / 0.050 | 0.070 / 0.041
yeast4 | 0.251 / 0.240 | 0.321 / 0.304 | 0.238 / 0.228 | 0.314 / 0.298
yeast5 | 0.691 / 0.684 |  0.738 / 0.730 | 0.705 / 0.698 | 0.727 / 0.718
yeast6 | 0.498 / 0.490 |  0.575 / 0.567 | 0.510 / 0.503 | 0.510 / 0.501
Average | 0.600 / 0.591 | 0.628 / 0.618 | 0.604 / 0.594 | 0.622 / 0.611
TABLE XIII: Table showing F1-score/$\kappa$-score for different parameter settings for ConvGeN, using the GB classifier. The ConvGeN(min,maj) model is seen to perform the best for all the tested classifiers. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | ConvGeN(5,maj) | ConvGeN(min,maj) | ConvGeN(5,prox) | ConvGeN(min,prox)
---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.291 / 0.270 | 0.291 / 0.278 | 0.310 / 0.289 | 0.311 / 0.297
abalone9 18 | 0.367 / 0.329 | 0.318 / 0.292 | 0.343 / 0.304 | 0.318 / 0.292
car good | 0.714 / 0.705 | 0.864 / 0.858 | 0.733 / 0.723 |  0.869 / 0.864
car vgood | 0.933 / 0.931 |  0.982 / 0.982 | 0.951 / 0.949 |  0.982 / 0.982
flare F | 0.139 / 0.125 | 0.181 / 0.164 | 0.162 / 0.147 | 0.156 / 0.138
hypothyroid | 0.789 / 0.777 | 0.800 / 0.789 | 0.789 / 0.777 |  0.815 / 0.806
kddcup guess passwd vs satan | 0.995 / 0.995 |  1.000 / 1.000 | 0.996 / 0.996 |  1.000 / 1.000
kr vs k three vs eleven |  0.995 / 0.995 |  0.995 / 0.995 | 0.992 / 0.992 |  0.995 / 0.995
kr vs k zero one vs draw | 0.959 / 0.958 |  0.972 / 0.971 | 0.960 / 0.958 | 0.961 / 0.960
shuttle 2 vs 5 |  1.000 / 1.000 |  1.000 / 1.000 | 0.994 / 0.994 |  1.000 / 1.000
winequality red 4 |  0.133 / 0.095 | 0.107 / 0.066 |  0.133 / 0.095 | 0.102 / 0.063
yeast4 | 0.257 / 0.242 | 0.351 / 0.329 | 0.261 / 0.245 | 0.333 / 0.311
yeast5 | 0.713 / 0.705 | 0.724 / 0.716 | 0.731 / 0.723 | 0.714 / 0.705
yeast6 | 0.490 / 0.481 |  0.564 / 0.554 | 0.481 / 0.472 | 0.520 / 0.510
Average | 0.627 / 0.615 |  0.654 / 0.642 | 0.631 / 0.619 | 0.648 / 0.637
TABLE XIV: Table showing F1-score/$\kappa$-score for different parameter settings for ConvGeN, using the kNN classifier. The ConvGeN(min,maj) model is seen to perform the best for all the tested classifiers. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | ConvGeN(5,maj) | ConvGeN(min,maj) | ConvGeN(5,prox) | ConvGeN(min,prox)
---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.290 / 0.261 | 0.315 / 0.296 | 0.273 / 0.243 | 0.311 / 0.291
abalone9 18 | 0.361 / 0.305 | 0.136 / 0.120 | 0.347 / 0.290 | 0.165 / 0.149
car good | 0.558 / 0.531 | 0.509 / 0.479 | 0.534 / 0.505 | 0.585 / 0.561
car vgood | 0.659 / 0.641 |  0.879 / 0.874 | 0.700 / 0.684 | 0.794 / 0.784
flare F | 0.269 / 0.220 |  0.306 / 0.265 | 0.277 / 0.228 | 0.302 / 0.262
hypothyroid | 0.576 / 0.548 | 0.606 / 0.582 | 0.578 / 0.551 |  0.646 / 0.628
kddcup guess passwd vs satan | 0.982 / 0.982 |  0.991 / 0.991 | 0.985 / 0.985 |  0.991 / 0.991
kr vs k three vs eleven | 0.895 / 0.891 |  0.961 / 0.960 | 0.894 / 0.890 | 0.925 / 0.923
kr vs k zero one vs draw | 0.864 / 0.858 |  0.897 / 0.892 | 0.865 / 0.859 | 0.853 / 0.847
shuttle 2 vs 5 | 0.954 / 0.954 | 0.932 / 0.931 |  0.960 / 0.960 | 0.943 / 0.942
winequality red 4 |  0.074 / 0.016 | 0.058 / -0.001 |  0.074 / 0.016 | 0.059 / 0.000
yeast4 | 0.334 / 0.296 | 0.348 / 0.313 | 0.338 / 0.301 | 0.345 / 0.309
yeast5 | 0.650 / 0.636 | 0.646 / 0.631 | 0.640 / 0.624 | 0.659 / 0.645
yeast6 | 0.337 / 0.311 | 0.426 / 0.406 | 0.334 / 0.308 | 0.390 / 0.367
Average | 0.557 / 0.532 | 0.572 / 0.553 | 0.557 / 0.532 | 0.569 / 0.550
TABLE XV: Table showing F1-score/$\kappa$-score for different parameter settings for ConvGeN, using the DoC classifier. The ConvGeN(min,maj) model is seen to perform the best for all the tested classifiers. dataset ($f_{1}~{}$score$~{}/~{}\kappa~{}$score) | ConvGeN(5,maj) | ConvGeN(min,maj) | ConvGeN(5,prox) | ConvGeN(min,prox)
---|---|---|---|---
abalone 17 vs 7 8 9 10 | 0.306 / 0.278 |  0.393 / 0.372 | 0.317 / 0.289 | 0.366 / 0.345
abalone9 18 | 0.477 / 0.433 | 0.538 / 0.512 | 0.512 / 0.473 |  0.549 / 0.525
car good | 0.741 / 0.729 | 0.659 / 0.646 |  0.759 / 0.748 | 0.577 / 0.558
car vgood | 0.772 / 0.763 | 0.721 / 0.712 |  0.776 / 0.766 | 0.753 / 0.743
flare F |  0.321 / 0.279 | 0.223 / 0.193 | 0.302 / 0.262 | 0.267 / 0.234
hypothyroid | 0.627 / 0.604 |  0.703 / 0.688 | 0.625 / 0.602 | 0.690 / 0.674
kddcup guess passwd vs satan | 0.991 / 0.991 |  1.000 / 1.000 | 0.996 / 0.996 |  1.000 / 1.000
kr vs k three vs eleven | 0.993 / 0.992 | 0.995 / 0.995 | 0.994 / 0.994 |  0.996 / 0.996
kr vs k zero one vs draw | 0.957 / 0.955 | 0.951 / 0.949 |  0.965 / 0.964 | 0.941 / 0.939
shuttle 2 vs 5 |  1.000 / 1.000 | 0.996 / 0.996 | 0.994 / 0.994 | 0.998 / 0.998
winequality red 4 | 0.118 / 0.065 |  0.123 / 0.073 | 0.115 / 0.063 | 0.105 / 0.052
yeast4 | 0.312 / 0.274 | 0.308 / 0.271 | 0.311 / 0.273 |  0.334 / 0.299
yeast5 | 0.578 / 0.560 |  0.652 / 0.639 | 0.571 / 0.553 | 0.637 / 0.623
yeast6 | 0.366 / 0.342 |  0.491 / 0.475 | 0.358 / 0.334 | 0.382 / 0.359
Average | 0.611 / 0.590 |  0.625 / 0.609 | 0.614 / 0.594 | 0.614 / 0.596