# GANs for learning from very high class conditional noisy labels

Sandhya Tripathi<EMAIL_ADDRESS>N. Hemachandra  IEOR, IIT Bombay

###### Abstract

We use Generative Adversarial Networks (GANs) to design a class conditional
label noise (CCN) robust scheme for binary classification. It first generates
a set of correctly labelled data points from noisy labelled data and 0.1% or
1% clean labels such that the generated and true (clean) labelled data
distributions are close; generated labelled data is used to learn a good
classifier. The mode collapse problem while generating correct feature-label
pairs and the problem of skewed feature-label dimension ratio ($\sim$ 784:1)
are avoided by using Wasserstein GAN and a simple data representation change.
Another WGAN with information theoretic flavour on top of the new
representation is also proposed. The major advantage of both schemes is their
significant improvement over the existing ones in presence of very high CCN
rates, _without_ either estimating or cross validating over the noise rates.
We proved that KL divergence between clean and noisy distribution increases
w.r.t. noise rates in symmetric label noise model; can be extended to high CCN
rates. This implies that our schemes perform well due to the adversarial
nature of GANs. Further, use of generative approach (learning clean joint
distribution) while handling noise enables our schemes to perform better than
discriminative approaches like GLC, LDMI and GCE; even when the classes are
highly imbalanced. Using Friedman F test and Nemenyi posthoc test, we showed
that on high dimensional binary class synthetic, MNIST and Fashion MNIST
datasets, our schemes outperform the existing methods and demonstrate
consistent performance across noise rates.

###### keywords:

Binary classification , Wasserstein GANs , Representation learning , Class
conditional label noise , Imbalanced data , Multi Layer Perceptron

##  1 Introduction

Label noise in supervised data for classification task has become ubiquitous
due to the deluge of data availability, but from unreliable sources, in many
cases. It renders the learning algorithms with very bad performance. In
particular, for currently popular Deep Neural Networks (DNNs), [1] have shown
that even though DNNs have high capacity to fit random labels, they lead to
large generalization error. Noise type, defined in terms of label flipping
probability or noise rates, can be broadly categorized as follows: (1)
Uniform/Symmetric/Random label noise (SLN): label flipping probability is same
across the dataset, (2) Class conditional noise (CCN): label flipping
probability depends on the class of data point, (3) Instance dependent noise
(IDN): label flipping probability depends on the data point and hence
different for all data points. Four major approaches for learning in the
presence of label noise [2] are as follows: (1) Noise cleaning: correct labels
are restored (2) Eliminating noisy points: after identifying the noisy points
they are eliminated (3) Designing schemes for dealing with label noise: goal
is to minimize the effect of label noise (4) Noise tolerant algorithms:
designing algorithms that are unaffected by the label noise.

Recently, use of a small set of clean labels by algorithms for learning from
label noise is gaining popularity [3, 4]. Also, ideas like importance re-
weighting [5], human assisted approaches for noise matrix estimation [6],
dimensionality of deep representations [7], have been used for dealing with
label noise. However, most of the solutions estimate the noise rates or
demonstrate the results on noise rates which reduce to SLN in binary class
setup. Generative Adversarial Networks (GANs) [8] have taken everything by
storm by being able to provide a solution in some way to almost every learning
problem. GANs exploit the adversarial relation between two networks: generator
and discriminator, to synthesize new samples by mimicking a target
distribution.

In this paper, using a small set of clean labels along with noisy labels, we
propose a GAN based class conditional label noise robust binary classification
framework. We adapt GANs in our setup by feeding small set of clean labelled
data points to the discriminator and noisy labelled data points to the
generator and asking the generator to output _correctly_ labelled data points.
Unlike the noise cleaning approach where only labels are corrected, our GANs
based correctly labelled data generation scheme generates correct new feature-
label pairs. Also, unlike existing schemes, our scheme completely avoids any
estimation of or cross-validation over the noise rates. It decreases one
source of estimation error. Also, due to the adversarial nature of GANs, we
observe that it performs better than the existing schemes in the presence of
very high noise.

For imbalanced datasets, class conditional label noise can swap majority and
minority class. However, due to the generative nature of our schemes, they are
able to maintain original imbalance and hence doesn’t require any special
modification for learning. Our main contributions are as follows:

  * 1.

We propose novel Wasserstein GAN based schemes for learning from CCN corrupted
binary classification data when a small set of clean labels can be procured.

    * (a)

They do not require the knowledge (by estimation or cross-validation) of noise
rates and hence are robust to the datasets where an opponent/adversary is
strategically corrupting the labels.

    * (b)

They use representation and information theoretic ideas.

    * (c)

In presence of very high noise (close to noise rate value of $0.5$), they show
significant advantage over the existing label noise learning methods that use
clean labels and those which do not require any knowledge about the noise
rates.

  * 2.

Generative nature of the proposed schemes enable them to learn from class
imbalanced and CCN corrupted data, without any modifications in the schemes.

  * 3.

In addition to WGAN, we attempted to adapt PacGAN [9], WGAN-GP [10] and a
variant inspired by VEEGAN [11] for generating correctly labelled data but
they did not work (see B.4); we adapt WGAN for our problem (which is known to
address possible mode collapse).

  * 4.

By using Friedman F test111The samples used by the Friedman F test are the
accuracy or AM values for different noise rates for a fixed dataset; For a
fixed dataset, null hypothesis is that the average performance (accuracy or AM
values) is same for all schemes. and Nemenyi posthoc tests222If, for a given
dataset, the null hypothesis for Friedman test is rejected, then pairwise
tests are performed to check if there is a statistically significant
difference between the accuracy or AM values of the schemes in the considered
pair., we demonstrated that WGAN based schemes lead to statistically
significant improvement of accuracy or AM values over the loss function based
discriminative approaches, viz., GLC, GCE, LDMI.

A schematic of our schemes can be seen in Figure 1.

Figure 1: A schematic of our WGAN based schemes WGANXtraY(k) and
WGANXtraYEntr(k). Stage 1 uses clean (from $\mathcal{D}$ distribution) and
noisy (from $\tilde{\mathcal{D}}$ distribution) labelled data to generate
correctly labelled (from $\mathcal{D}^{\prime}$) data. Stage 2 uses correctly
labelled generated data to train classification model $M$.

###  1.1 Related work

In the last decade, label noise problem has gained a lot of attention from
researchers due to its prevalence in various real life situations. [12, 13,
14, 15, 5, 16] focus on effect of label noise problems on non-deep
classification schemes and provide solutions which either consider a label
noise robust loss function or modifies the loss function to make it robust.
For deep learning schemes, [17] propose two algorithms called forward and
backward loss correction to learn from label noise corrupted data, [18]
identify label noise robust loss functions to be used by neural networks and
[19] provide consistency results when the noise is instance dependent. [20]
propose a co-teaching approach using two communicating neural networks. [21]
propose a relation between noisy data test accuracy and noise matrix. The
above schemes which modify the loss function either require the knowledge of
noise rates or estimate them. [22] propose a theoretically sound loss function
called bi-tempered loss function that is a non-convex generalization of
logistic loss and requires tuning two temperature parameters. However, it is
shown to be robust only for SLN case. [23] propose a loss function, viz.,
generalized cross entropy loss (GCE) that is shown to be asymptotically robust
to SLN uniformly and CCN under some conditions. Also, [24] propose a loss
function, viz., $L_{DMI}$ that is invariant to the type and level of noise. We
compared our schemes with the above two methods and observed that at high CCN
rates our schemes have higher accuracies when the final classifier is learnt
using a multi-layer perceptron. We recently came across the work of [25] and
[26] and they are observed to be performing better than GCE. However, the
negative learning based approach [25] is observed to be non-converging at high
noise rates and the robustness of symmetric cross entropy [26] is approximate
and holds uniformly only for SLN but not CCN.

Of late, a reasonable assumption of access to a small set of clean labels
during training the model from noisy labelled data is being used [27, 28, 29,
4, 3]. The clean label set is used to estimate the noise rates in all the
above schemes. [3] show that their scheme is better than others mentioned
before. Hence, we compare our scheme to Gold Loss Correction (GLC) proposed by
[3] and demonstrate that without even estimating the noise rates, our schemes
lead to significant improvement over GLC at very high noise rates.

[30] and [31] study label noise problem for conditional GAN (cGAN) and
Auxiliary classifier GAN (ACGAN). Former work proposes two schemes, viz.,
RCGAN, which assumes knowing true noise matrix and RCGAN-U which estimates the
noise matrix to make cGAN robust to label noise. [31] also propose two
schemes, viz., rACGAN and rcGAN where the discriminator is modified by
including a noise transition model. We would like to emphasize that the above
mentioned problem is different as label noise is affecting the sample
generation quality and related aspects. However, in our problem label noise is
affecting the classification performance. To summarize, our scheme differs
from the aforementioned works in the context of (1) the task at hand, i.e.,
sample generation task vs classification task (2) noise rate, i.e., estimation
vs non-estimation (3) the type of noise, i.e., uniform flipping and asymmetric
multi-class noise (reduces to SLN in case of binary class) vs. CCN (4) our
schemes’ implicit capability of handling class imbalance in addition to CCN.

Organization We introduce the label noise problem and GAN framework in Section
2.1 and 2.2. The main result on WGAN based schemes is provided in Section 2.3
along with the theoretical results and some insights about which GAN
architecture didn’t work and why WGAN based schemes work in imbalanced data
setup too. We demonstrate the performance of our schemes and compare them to
existing schemes in Section 3. We conclude with discussion in Section 4.

##  2 GANs for generating clean labelled data

In this section, we first briefly describe label noise problem and basics of
GANs. Then, we show which variants of GANs can be adapted for generating
correctly (clean) labelled data as per true data distribution. We propose two
schemes based on representational changes and information theoretic changes in
objective function which leads to better results in final classification
model. Finally, we explain why our schemes work well without any modification,
in the presence of label noise and class imbalance.

###  2.1 Label noise problem

Let $\mathcal{D}$ be the joint distribution over $\mathbf{X}\times Y$ with
$\mathbf{X}\in\mathcal{X}\subseteq\mathbb{R}^{n}$ and
$Y\in\mathcal{Y}=\\{-1,1\\}.$ Let the in-class probability on $\mathcal{D}$ be
denoted by $\eta(\mathbf{x}):=P(Y=1|\mathbf{x})$. Let $\tilde{\mathcal{D}}$
denote the distribution on $\mathbf{X}\times\tilde{Y}$ obtained by inducing
noise to $\mathcal{D}$ with $\tilde{Y}\in\tilde{\mathcal{Y}}=\\{-1,1\\}$. The
corrupted sample is
$\tilde{S}=\\{(\mathbf{x}_{1},\tilde{y}_{1}),\ldots,(\mathbf{x}_{m},\tilde{y}_{m})\\}\sim\tilde{\mathcal{D}}^{m}.$
The label noise is class dependent and the noise rates are defined as
$\rho_{+}:=P(\tilde{Y}=-1|Y=1,\mathbf{X}=\mathbf{x})$,
$\rho_{-}:=P(\tilde{Y}=1|Y=-1,\mathbf{X}=\mathbf{x})$. This model is referred
to Class Conditional Noise (CCN) model. In such cases, the corrupted in-class
probability is
$\tilde{\eta}(\mathbf{x}):=P(\tilde{Y}=1|\mathbf{x})=(1-\rho_{+}-\rho_{-})\eta(\mathbf{x})+\rho_{-}$.
If $\rho_{+}=\rho_{-}=\rho$, then the model is known as Symmetric Label Noise
(SLN) model. Most existing solutions in literature for label noise problems
either cross-validate over the noise rates or estimate them. Since, estimating
noise rate is a density estimation problem (a hard problem), an alternative
and clean solution is the one which doesn’t require any knowledge (estimation
or cross-validation) about the noise rates.

###  2.2 Generative Adversarial Networks (GANs)

GANs [8] are defined by a two player min-max game between a discriminator and
a generator with the objective that after training the generator starts
generating samples from the true data distribution. The generator is a neural
network that uses samples from a simple prior distribution like Gaussian or
uniform to generate samples from true data distribution. The discriminator is
also a neural network which is fed with samples from true data distribution
(real) and samples generated by the generator (fake) and its job is to
distinguish real samples from fake ones and provide feedback to the generator.
Mathematically, the objective function for the two players can be described as
a min-max optimization problem given below:

$\min\limits_{G}\max\limits_{D}\mathbb{E}_{p_{\mathcal{D}}(\mathbf{x})}[f(D(\mathbf{x}))]+\mathbb{E}_{p(\mathbf{z})}[f(-D(G(\mathbf{z})))]$
(1)

where $G:\mathcal{Z}\mapsto\mathcal{X}$ is the generator network, which maps
from the latent space (prior) $\mathcal{Z}$ to the input space $\mathcal{X}$;
$D:\mathcal{X}\mapsto\mathbb{R}$ is the discriminator network, which maps from
the input space to the decision space whether the sample is real or fake; and
$f$ is concave function. $p(\mathbf{z})$ and $p_{\mathcal{D}}(\mathbf{x})$ are
the density functions of the random variables in $\mathcal{Z}$ and
$\mathcal{X}$ respectively. The traditional GAN formulation can be recovered
with $f(\mathbf{x})=-\log(1+\exp(-\mathbf{x}))$ and WGAN [32] with
$f(\mathbf{x})=\mathbf{x}$. The two major issues with GANs are of mode
collapse and instability of the iterates from the optimization method while
training. In our context, mode collapse is a serious issue because samples
containing only one mode (class) will lead to a biased and extreme classifier.
Instability would have adverse effects as in label noise models the
performance of a scheme is evaluated by averaging over various random trials.
Instability in GAN training would induce a lot of variations in final
performance across various trials of the classification model trained on
generated samples. In this paper, we use WGAN as it is known to be more stable
while training and less prone to the problem of mode collapse. We would like
to emphasize that conventionally GANs are generative models and are generally
not used for classification. Hence, for them the role of labels is minimal.

###  2.3 WGAN for generating correctly labelled data

Due to label noise problem, we have to deal with two data distributions: clean
$\mathcal{D}$ and label noise corrupted $\tilde{\mathcal{D}}$ with one of
marginals $\mathbf{X}$ common and the other marginal $Y$ and $\tilde{Y}$
different. Usually, we only have access to corrupted sample $\tilde{S}$.
However, these days a small set of clean labels is easily procurable [3, 4].
We also leverage this fact and use a small number ($m_{cl}$) of data points
with clean labels, say set
$S_{cl}=\\{(\mathbf{x}_{1},{y}_{1}),\ldots,(\mathbf{x}_{m_{cl}},{y}_{m_{cl}})\\}\sim{\mathcal{D}}^{m_{cl}}$.
Now, consider the framework where the samples (with clean labels) from
$\mathcal{D}$ are fed to the discriminator in place of the data points from
true data distribution and the samples (with noisy labels) from
$\tilde{\mathcal{D}}$ are fed to the generator in place of the sample points
$\mathbf{z}$. The goal of the generator is to generate samples (with clean
labels) from a distribution $\mathcal{D}^{\prime}$ which is as close as
possible to $\mathcal{D}$. Using this idea, we propose 2 schemes, WGANXtraY(k)
and WGANXtraYEntr(k) described in following subsections.

A natural thought would be that the job of generator has been simplified as
$\tilde{\mathcal{D}}$ and $\mathcal{D}$ are already very close as marginals of
$\mathbf{X}$ are same and transforming one to another shouldn’t be difficult.
However, the reality is quite counter intuitive due to the adversarial nature
of WGANs. We observed that farther the two distributions are ($\mathcal{D}$
and $\mathcal{Z}$ in WGANs or $\mathcal{D}$ and $\tilde{\mathcal{D}}$ in our
case), the better is the performance of WGANs for generating samples from
$\mathcal{D}$. In our case, we can quantify whether the distributions are far
or not by computing the KL divergence between $\mathcal{D}$ and
$\tilde{\mathcal{D}}$. With this context, we first show in Lemma 1 that when
$\tilde{\mathcal{D}}$ is SLN corrupted, the KL divergence between
$\mathcal{D}$ and $\tilde{\mathcal{D}}$ is an increasing function of noise
rate $\rho$. A proof is available in A.1.

###### Lemma 1

Let $\mathcal{D}$ and $\tilde{\mathcal{D}}$ be clean distribution and
symmetric label noise corrupted distribution with noise rate $\rho$
respectively. Then, the KL divergence between $\mathcal{D}$ and
$\tilde{\mathcal{D}}$ given as follows:

$\displaystyle
KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{SLN}=\mathbb{E}_{\mathbf{X}}\left[-\eta(\mathbf{x})\log\left(1+\rho\left(\frac{1-2\eta(\mathbf{x})}{\eta(\mathbf{x})}\right)\right)\right.$
$\displaystyle-\left.(1-\eta(\mathbf{x}))\log\left(1-\rho\left(\frac{1-2\eta(\mathbf{x})}{\eta(\mathbf{x})}\right)\right)\right],$
(2)

is an increasing function of $\rho\in(0,0.5)$.

If one uses the observation that higher the adversarial relation (in terms of
divergence between $\mathcal{D}$ and $\tilde{\mathcal{D}}$), better is the
performance of WGAN, then, Lemma 1 implies that performance of our WGAN based
scheme for generating correctly labelled data will improve as the noise rates
increase. Since, we do not have a measure to see the absolute improvement, we
consider the improvement relative to the existing method of GLC from [3]. And,
indeed, our claim is true as verified empirically for SLN corrupted 3 binary
versions of MNIST dataset and 1 binary version of Fashion-MNIST dataset where
for noise rate $\rho\in(0.45,0.5)$ our scheme outperforms GLC. Details are
available in Table 6 and 7 of B.2.

Next, similarly, for the case of CCN, we would like to argue that our WGAN
based schemes for generating correctly labelled data, work well at higher
noise rate due to increase in adversarial nature via monotonicity of
$KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{CCN}$. However, the monotonicity in
case of CCN is not straight forward because we would have to verify the
increasing nature of KL divergence between $\mathcal{D}$ and
$\tilde{\mathcal{D}}$, $KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{CCN}$ w.r.t. two
variables, viz., $\rho_{+},\rho_{-}$. Unfortunately, for the two ways, viz.,
projection based and total derivative approach, in which we considered
monotonicity lead to a negative result. We formalize this in Lemma 2 whose
proof is available in A.2.

###### Lemma 2

Let $\mathcal{D}$ and $\tilde{\mathcal{D}}$ be clean distribution and class
conditional label noise corrupted distribution with noise rates $\rho_{+}$ and
$\rho_{-}$. Then, the KL divergence between $\mathcal{D}$ and
$\tilde{\mathcal{D}}$ given as follows:

$\displaystyle
KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{CCN}=\mathbb{E}_{\mathbf{X}}\left[-\eta(\mathbf{x})\log\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)\right.$
$\displaystyle-\left.(1-\eta(\mathbf{x}))\log\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right)\right],$

is not an increasing function of $\rho_{+}$ and $\rho_{-}$.

Since, our claim is for high noise rates only, we can use the continuity
arguments from SLN case to show that in the considered regime of high noise
rates, the adversarial relation plays a role in good performance of our scheme
even when the noise is of CCN type. Based on our empirical experience (Table 6
and 7 of B.2), we observe that the following inequality holds in most of the
cases:

$h(S,\tilde{S})_{\rho,met}>h(S,\tilde{S})_{\rho,GLC},$

where $\rho$ is the symmetric label noise rate, $h(\cdot)$ is a performance
metric that can be taken as the test accuracy and
$met\in\\{WGANXtraY(k),WGANXtraYEntr(k))\\}$. Because of the strict
inequality, if we make an $\epsilon$ perturbation to $\rho\in(0.45,0.49)$ to
obtain $\rho_{+}\in(0.45,0.49)$ and $\rho_{-}\in(0.45,0.49)$, we get the
following relation,

$h(S,\tilde{S})_{\rho_{+},\rho_{-},met}>h(S,\tilde{S})_{\rho_{+},\rho_{-},GLC}.$

This implies that at high noise rates, a large value of
$KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{CCN}$ leads to better performance of
our schemes relative to GLC as verified empirically on 2 synthetic datasets
and 10 real datasets in Figure 2 and 3.

We would like to emphasize that our scheme is different from conventional WGAN
in two ways: 1) data representation approach, i.e., the labels are appended in
addition to the features and 2) the latent variable is not standard normal or
uniform, instead it follows corrupted and hence _unknown_ data distribution
$\tilde{\mathcal{D}}$. Also, unlike the existing work on learning with noisy
labels, use of a generative model in label noise problem allows us to
circumvent the problem of either estimating or tuning the noise rates. Even
though [7, 33, 16] do not need the noise rates, the context is restricted
either to symmetric label noise or non-high dimensional datasets. Next, we
formally describe our scheme.

####  2.3.1 WGANY

Consider a WGANY (Y is due to use of labelled data) with generator
$G_{dn}:\mathcal{X}\times\tilde{\mathcal{Y}}\mapsto\mathcal{X}\times\mathcal{Y}$
and discriminator (critic)
$D_{dn}:\mathcal{X}\times\mathcal{Y}\mapsto\mathbb{R}$. If the generator
models a distribution $\mathcal{D}^{\prime}$, then the goal is to minimize the
divergence between _clean_ distribution $\mathcal{D}$ and _correctly labelled
generated_ distribution $\mathcal{D}^{\prime}$. Hence, the WGANY objective
function (adapted from [10]) is given as follows:

$\min\limits_{G_{dn}}\max\limits_{D_{dn}}\mathop{\mathbb{E}}_{(\mathbf{x},y)\sim\mathcal{D}}[D_{dn}(\mathbf{x},y)]-\mathop{\mathbb{E}}_{(\mathbf{x}^{\prime},y^{\prime})\sim\mathcal{D}^{\prime}}[D_{dn}(\mathbf{x}^{\prime},y^{\prime})],$
(3)

where $D_{dn}$ belongs to the set of $1$-Lipschitz functions and
$\mathcal{D}^{\prime}$ is model distribution implicitly defined by
$(\mathbf{x}^{\prime},y^{\prime}):=(G^{f}_{dn}(\mathbf{x},\tilde{y}),G^{l}_{dn}(\mathbf{x},\tilde{y}))=G_{dn}(\mathbf{x},\tilde{y})$,
with $(\mathbf{x},\tilde{y})\sim\tilde{\mathcal{D}}$. Lipschitz constraint is
enforced by clipping the weights of the discriminator to lie within a compact
set $[-c,c]$ for some constant $c>0$. As the last dimension of the output,
$(\mathbf{x}^{\prime},y^{\prime})$, of the generator $G_{dn}$ is supposed to
be a label, we first convert the last dimension of the generator output to a
probability value using sigmoid function. Now, the later can be interpreted as
in-class probability $\eta(\mathbf{x}^{\prime})$ and thresholded at 0.5 to get
the label, i.e., $\eta(\mathbf{x}^{\prime})>0.5\Rightarrow y^{\prime}=1$ else
$y^{\prime}=-1$.

As the objective function of WGAN is a strict adversarial divergence and we
have just appended the label vector as an extra dimension, the convergence
result of [34] continues to hold for our scheme. However, there are other
aspects that are peculiar to the use of WGANY for generating correctly
labelled data. For a given corrupted data point $(\mathbf{x},\tilde{y})$ we
expect our generator to either correct the label and output $(\mathbf{x},y)$
or synthesize a new feature vector along with its correct label
$(\mathbf{x}^{\prime},y^{\prime})$. Since, the feature space is generally
continuous, we believe that second event happens with probability 1. Now, the
feature dimension $n$ being far greater than the label dimension of $1$
obstructs in learning the correct feature-label pair by the generator
$G_{dn}$. We demonstrate this in Figure 8 and 9 of B.3. In addition, binary
nature of $Y$ complicates this problem. To resolve this problem, we propose
following two solutions.

####  2.3.2 WGANXtraY(k)

WGANXtraY($k$) stands for WGAN with extra $k$ dimensions consisting of scaled
$Y$ values. Here, $k>2$ is taken to be an odd natural number to avoid ties. In
other words, a $(n+1)$ dimensional data point $(\mathbf{x},y)$ would become a
$(n+k)$ dimensional data point $(\mathbf{x},y,ly,\cdots,l(k-1)y)$ where $l>1$
can be chosen arbitrarily. We exploit the fact that for binary classification,
sign of label $y$ is sufficient and scale the labels so that the label
information is same, i.e., for a given $y\in\\{-1,1\\}$,
$sign(y)=sign(l(k-1)y),~{}k>2$. For example, if for a negatively labelled data
point $(\mathbf{x},-1)$, we take $k=3$ and $l=5$, then the $(n+3)$ dimensional
appended data point is $(\mathbf{x},-1,-5,-10)$. For new generated samples,
unlike WGANY where sigmoid function is applied to only last dimension, in
WGANXtraY(k), the sigmoid transformation is applied to last $k$ dimensions
individually. If $(k+1)/2$ dimensions out of last $k$ dimensions of generator
output have probability values greater than $0.5$, then the last dimension is
assigned the correct label as $y^{\prime}=1$ else $y^{\prime}=-1$.

####  2.3.3 WGANXtraYEntr(k)

We require that the label $y^{\prime}$ assigned by the generator to the
generated sample $(\mathbf{x}^{\prime},y^{\prime})$ should be correct and not
noisy. Inspired by the use of entropy while making class assignments as in
[35], we attempt to satisfy this requirement by adding entropy terms to the
generator and discriminator of WGANXtraY(k). We want the class distribution
$p(y^{\prime}|\mathbf{x}^{\prime})$ conditioned on generated data point
$\mathbf{x}^{\prime}$ to be highly peaked, i.e.,
$G_{dn}(\mathbf{x},\tilde{y})$ should be certain of the label assigned. This
can be achieved by minimizing the Shannon entropy
$H[p(y^{\prime}|\mathbf{x}^{\prime})]$ for the sample
$(\mathbf{x}^{\prime},y^{\prime})$, as we would want that for any draw from
$\mathbf{x}^{\prime}$, the conditional distribution should lead to same label
$y^{\prime}$ almost every time. The conditional entropy over the samples from
the generator $G_{dn}$ can be written as the expectation of
$H[p(y^{\prime}|\mathbf{x}^{\prime})]$ over the prior distribution
$\tilde{\mathcal{D}}$, i.e.,
$\mathop{\mathbb{E}}_{(\mathbf{x},\tilde{y})\sim\tilde{\mathcal{D}}}[H[p(G^{l}_{dn}(\mathbf{x},\tilde{y})|G^{f}_{dn}(\mathbf{x},\tilde{y}))]]$
and its estimate is given below:

$\displaystyle\mathcal{L}_{G_{dn},entr}=\frac{1}{m_{g}}\sum\limits_{i=1}^{m_{g}}H[p(G^{l}_{dn}(\mathbf{x}_{i},\tilde{y}_{i})|G^{f}_{dn}(\mathbf{x}_{i},\tilde{y}_{i}))]$
$\displaystyle=\frac{1}{m_{g}}\sum\limits_{i=1}^{m_{g}}\left(-\sum\limits_{u\in\\{-1,1\\}}p(y^{\prime}_{i}=u|\mathbf{x}^{\prime}_{i})\log
p(y^{\prime}_{i}=u|\mathbf{x}^{\prime}_{i})\right),$ (4)

where $m_{g}$ is the number of independently drawn noisy samples
$(\mathbf{x},\tilde{y})$ from corrupted training dataset $\tilde{S}$. The
conditional probability $p(y^{\prime}|\mathbf{x}^{\prime})$ can be obtained
from the last $k$th dimension of $G_{dn}(\mathbf{x},\tilde{y})$ before the
final label is determined by using majority voting from last $k$ dimensions.
Similarly, for the discriminator, we would want that $D_{dn}$ should decide
that the samples from $\mathcal{D}$ are _clean_ and the generated samples
$G_{dn}(\mathbf{x},\tilde{y})$ from model distribution $\mathcal{D}^{\prime}$
are _correctly labelled generated (gen_cor)_ data points. To achieve this, it
should have low entropy on the conditional distribution of _clean_ samples
from $\mathcal{D}$ and high entropy on the conditional distribution of
_gen_cor_ samples from $\mathcal{D}^{\prime}$. The former requirement can be
satisfied by minimizing estimate of
$E_{(\mathbf{x},y)\sim\mathcal{D}}[H[p(A|(\mathbf{x},y))]]$ given in Eq. (5)
and the later can be satisfied by maximizing the estimate of
$E_{(\mathbf{x},\tilde{y})\sim\tilde{D}}[H[p(A|G_{dn}(\mathbf{x},\tilde{y}))]]$
given in Eq. (6). Here, $A$ denotes the type of data point fed in
discriminator and takes value $a$ belonging to the set
$\\{clean,~{}gen\\_cor\\}$.

$\displaystyle\mathcal{L}_{D_{dn},entr}^{(1)}=\frac{1}{m_{d}}\sum\limits_{i=1}^{m_{d}}H[p(A|(\mathbf{x}_{i},y_{i}))]$
$\displaystyle=\frac{1}{m_{d}}\sum\limits_{i=1}^{m_{d}}\big{(}-\sum_{a}p(a|(\mathbf{x}_{i},y_{i}))\log
p(a|(\mathbf{x}_{i},y_{i}))\big{)}$ (5)
$\displaystyle\mathcal{L}_{D_{dn},entr}^{(2)}=\frac{1}{m_{d}}\sum\limits_{i=1}^{m_{d}}H[p(A|(G_{dn}(\mathbf{x}_{i},\tilde{y}_{i})))]$
$\displaystyle=\frac{1}{m_{d}}\sum\limits_{i=1}^{m_{d}}\big{(}-\sum_{a}p(a|(G_{dn}(\mathbf{x}_{i},\tilde{y}_{i})))\log
p(a|(G_{dn}(\mathbf{x}_{i},\tilde{y}_{i})))\big{)},$ (6)

where $m_{d}$ is the number of independently drawn samples from clean dataset
$S_{cl}$ (for Eq. (5)) and number of labelled generated data points obtained
from generator (for Eq. (6)). Thus, we want to minimize the following term:

$\displaystyle\mathcal{L}_{D_{dn},entr}=\mathcal{L}_{D_{dn},entr}^{(1)}-\mathcal{L}_{D_{dn},entr}^{(2)}.$
(7)

Note that adding entropy maximization terms have been observed to be
performing a regularization [36]. From this perspective, WGANXtraYEntr(5) can
be considered as an entropy regularized version of the WGANXtraY(5).

Algorithm WGAN*(k) presents the overall classification scheme to make
predictions on clean test data using a classifier learnt on CCN label
corrupted training data _without using any knowledge about noise rates
$\rho_{+},\rho_{-}$_. We consider 4 parts of the datasets, $S_{cl}$ and
$S_{te}$ are clean and $\tilde{S}_{GAN,tr}$ and $\tilde{S}_{G,tr,M}$ are
noisy. $S_{cl}$ is a small set (usually $0.1\%$ or $1\%$ of training dataset)
whose correct labels are procured for training $D_{dn}$ and $G_{dn}$ and we
refer to it as gold fraction as in [3]. $\tilde{S}_{GAN,tr}$ is noisy dataset
which is used to train $D_{dn}$ and $G_{dn}$. $\tilde{S}_{G,tr,M}$ is fed to
the trained generator $G_{dn}$ to obtain correctly labelled data which in turn
is used to train the final classification model $M$. $S_{te}$ is the test
dataset used to evaluate the classification model $M$. The two schemes,
WGANXtraY(k) and WGANXtraYEntr(k) differ in objective functions for $D_{dn}$
and $G_{dn}$ which is reflected in line number 5 and 10 of Algorithm WGAN*(k).
We would like to point out that due to use of feature-label pair as one data
point (entity) to be fed in discriminator, using the later as a classifier
network is not straightforward.

Algorithm WGAN*(k) Schemes WGANXtraY(k) and WGANXtraYEntr(k) for learning from
CCN corrupted data

1:Input: Training data: ${S}_{cl}$, $\tilde{S}_{GAN,tr}$,
$\tilde{S}_{G,tr,M}$; test data: $S_{te}$

2:Require: Learning rate $\alpha$, Clipping parameter $c$, Batch size $m_{b}$,
No. of critic iterations $n_{c}$, No. of iterations $n_{it}$, No. of appends
$k$, Scaling number $l$, Initial critic parameters $w_{0}$, Initial generator
parameters $\theta_{0}$.

3:Output: Predicted labels for test data $S_{te}$.

4:for $t=0,\cdots,n_{it}$ do

5: for $t^{\prime}=0,\cdots,n_{c}$ do

6: Sample clean data $\\{(\mathbf{x}_{i},y_{i})\\}_{i=1}^{m_{b}}$ from
$S_{cl}$; construct
$\mathbf{d}_{i}:=(\mathbf{x}_{i},y_{i},ly_{i},\cdots,l(k-1)y_{i}),\forall i$.

7: Sample noisy data $\\{(\mathbf{x}_{i},\tilde{y}_{i})\\}_{i=1}^{m_{b}}$ from
$\tilde{S}_{GAN,tr}$; construct
$\tilde{\mathbf{d}}_{i}:=(\mathbf{x}_{i},\tilde{y}_{i},l\tilde{y}_{i},\cdots,l(k-1)\tilde{y}_{i}),\forall
i$.

8: WGANXtraY(k):
$g_{w}\leftarrow\nabla_{w}[\frac{1}{m_{b}}\sum\limits_{i=1}^{m_{b}}D_{dn}(\mathbf{d}_{i})-\frac{1}{m_{b}}\sum\limits_{i=1}^{m_{b}}D_{dn}(G_{dn}(\tilde{\mathbf{d}}_{i}))]$

9:OR WGANXtraYEntr(k): $g_{w}\leftarrow
g_{w}-\nabla_{w}\mathcal{L}_{D_{dn},entr}$

10: $w\leftarrow w+\alpha\cdot$ RMSProp$(w,g_{w})$

11: $w\leftarrow$ clip$(w,-c,c)$

12: end for

13: Sample noisy data $\\{(\mathbf{x}_{i},\tilde{y}_{i})\\}_{i=1}^{m_{b}}$
from $\tilde{S}_{GAN,tr}$; construct
$\tilde{\mathbf{d}}_{i}:=(\mathbf{x}_{i},\tilde{y}_{i},l\tilde{y}_{i},\cdots,l(k-1)\tilde{y}_{i}),\forall
i$.

14: WGANXtraY(k):
$g_{\theta}\leftarrow-\nabla_{\theta}\frac{1}{m_{b}}\sum\limits_{i=1}^{m_{b}}D_{dn}(G_{dn}(\tilde{\mathbf{d}}_{i}))$

15:OR WGANXtraYEntr(k): $g_{\theta}\leftarrow
g_{\theta}+\nabla_{\theta}\mathcal{L}_{G_{dn},entr}$

16: $\theta\leftarrow\theta-\alpha\cdot$RMSProp$(\theta,g_{\theta})$

17:end for

18:Sample noisy data
$\\{(\mathbf{x}_{i},\tilde{y}_{i})\\}_{i=1}^{|\tilde{S}_{G,tr,M}|}$ from
$\tilde{S}_{G,tr,M}$; construct
$\tilde{\mathbf{d}}_{i}:=(\mathbf{x}_{i},\tilde{y}_{i},l\tilde{y}_{i},\cdots,l(k-1)\tilde{y}_{i}),\forall
i$.

19:Get correctly labelled generated samples
$(\mathbf{x}_{i}^{\prime},y^{\prime}_{i})=G_{dn}(\tilde{\mathbf{d}_{i}}),\forall
i=1,\ldots,|\tilde{S}_{G,tr,M}|$, where $y^{\prime}_{i}$ is obtained by
majority voting in the last $k$ dimensions of the generator output.

20:Train classification model $M$ on
$\\{(\mathbf{x}^{\prime}_{i},y^{\prime}_{i})\\}_{i=1}^{|\tilde{S}_{G,tr,M}|}$.

21:Obtain predictions for test data $S_{te}$ using $M$.

####  2.3.4 Good GANs for generating samples need not be good for generating
correctly labelled data

The GAN literature is growing continuously with new architectures, new
objective functions, new priors, new regularization techniques and a lot of
theoretical analysis too. We explored the recent PacGAN [9] which uses the
fact that there is a fundamental connection between packing samples and mode
collapse. Unfortunately, it did not help for generating correctly labelled
data as the labels $y$ got packed along with the feature vectors. This lead to
the loss of label information content in between the features and hence no
diversity in samples. As VEEGAN [11] is known for reducing mode collapse, we
used a similar idea by adding a reconstruction loss to the generator. This
reconstruction loss is the mean squared error between the feature vector of
corrupted sample $(\mathbf{x},\tilde{y})$ and feature vector of the generated
sample $(\mathbf{x}^{\prime},y^{\prime})$. The issue of trade-off between
generating new sample such that it is close to the sample fed to the generator
and making sure the label is correct, doesn’t let the generator learn the
original clean distribution $\mathcal{D}$. In addition, the gradient penalty
proposed by [10] for WGAN also did not led to much improvement for the task of
generating correctly labelled data. These observations are based on
experiments on binary version of MNIST [37] and Fashion MNIST [38] datasets.
Details in Table 10, 11, 12 and 13 of B.4.

####  2.3.5 WGANs based schemes for noisy imbalanced data

For imbalanced and symmetric label noise ($\rho<0.5$) corrupted data, majority
(minority) class continues to be in majority (minority) (Lemma 3 [16]).
However, this result doesn’t hold for CCN corrupted data, which makes learning
from such data a difficult problem. We observe that our WGAN based schemes for
generating correctly labelled data need no modification even when the data is
imbalanced (Figure 4). This is because our schemes attempt to learn the
distribution with original imbalance ratio even though the available corrupted
data has different imbalance ratio. This advantage makes our scheme suitable
for learning from imbalanced noisy data without any knowledge of imbalance
ratio or $\rho_{+},\rho_{-}$. Two algorithms to deal with the noisy imbalanced
dataset problem proposed in [39] require tuning over the asymmetric
classification cost $\alpha$ and cross validation over noise rates. Also,
radial basis oversampling proposed in [40] only demonstrates their results
when the majority class doesn’t flip to minority (or vice versa) and hence
doesn’t completely deal with the complexity of imbalanced label noise
learning. [16] provide solution for the label noise (but only SLN) and
imbalance problem without requiring the noise rate.

##  3 Experiments

In this section, we empirically verify the performance of our schemes for
label noise robust classification. Exact details about synthetic data
generation scheme, sampling imbalanced data, dataset sizes and neural network
architecture are provided in B.1.
Datasets We consider 2 synthetic datasets (100 and 300 dimensional), binary
class balanced versions for 6 pairs of MNIST [37] and 4 pairs of Fashion MNIST
[38] datasets to demonstrate effectiveness of our scheme at very high (near
0.5) noise rates. We also consider 6 binary imbalanced versions of MNIST
datasets using different imbalance ratios. In the binary digit pair $(a,b)$,
$a$ corresponds to positive class and $b$ corresponds to negative class except
for the case when either $a$ or $b$ is 0. In the later case, $0$ corresponds
to negative class and the other digit to positive class. To account for
randomness in the flips to simulate a given noise rate, we repeat each
experiment 5 times, with independent corruptions of the dataset for same noise
($\rho_{+},\rho_{-}$) setting. The binary train and test set are constructed
from respective multi-class train and test datasets. The train dataset is then
split into $0.1\%$ (10-12 and 4-6 total clean data points in balanced and
imbalanced case respectively) or $1\%$ (100-120 and 50-60 total clean data
points in balanced and imbalanced case respectively) gold fraction, $\sim
84\%$ to train WGAN based scheme and $\sim 15\%$ for training final
classification model M. In every trial, the last two partitions are induced
with class conditional noise.
Framework The generator $G_{dn}$ and $D_{dn}$ are multilayer perceptron (MLP)
with 4 and 3 layers respectively. We trained a 4 layered MLP and a 3 layered
Convolutional Neural Network (CNN) with dropout after second convolution layer
as the final classification model M. The parameters used in Algorithm WGAN*(k)
are as follows: $\alpha=10^{-3}$, $c=0.01$, $n_{c}=5$, $m_{b}=64$. The number
of iterations $n_{it}=500(1000)$ for WGANXtraY(k) (WGANXtraYEntr(k)), if not
mentioned otherwise. To determine the influence of $l$ on the performance of
our scheme, we tried l= 2, 5 and 8 and the accuracies were within 2% of each
other with no single winner. In addition to k=5, we tried k=10 but didn’t
observe any improvement. Hence, in our experiments, we have used $k=5$ and
$l=5$. We consider 15 pairs of noise rates $(\rho_{+},\rho_{-})$ out of which
11 cases have both noise rates more than 0.4 (including some with values $\geq
0.5$) which we treat as very high noise rates. We use accuracy of the final
classification model $M$ on test data as an evaluation metric for balanced
datasets. For imbalanced datasets, we use arithmetic mean (AM) of true
positive rate (TPR) and true negative rate (TNR) to evaluate the quality of
final predictions from $M$. For low to moderate noise rates, not satisfactory
performance of our scheme (B.5) further asserts the claim of Lemma 2 that
performance of our schemes (with MLP based model M) improve with increase in
noise rates.

We use model $M$ trained on only noisy data (SimpleNN (final MLP classifier),
Simple CNN), GLC (with same gold fraction), GCE [23] (with q = 0.7 in model M)
and LDMI [24]. The code provided by [3] without any modification in terms of
architecture and parameters is used for GLC results. For LDMI, the code
provided by the authors is used suitably modified for the model $M$
architecture. As none of the methods used for comparison claim of working in
imbalanced settings, it might not be a fair comparison. However, since we do
not make any changes to our schemes to account for imbalance, for the sake of
completeness, we compare it to GLC, GCE, LDMI and SimpleNN/SimpleCNN. The
results presented in Figure 2, 3 and 4 depict the curves of average accuracies
and AM when model M is MLP. Since, standard deviation values are very high for
almost all noise rates, we present them along with accuracies in detail in C.
In addition, Table 25 to 30 and 35 to 38 of C, have results for 11 more very
high noise rates, when $\rho_{+},\rho_{-}$ values are swapped. The results
presented in Figure 5, 6 and 7 depict the accuracies and AM values for the
real datasets when the model M is a CNN. Based on various binary MNIST
versions with the same MLP model M, we observed that dropout and early
stopping helps only in case of low uniform label noise (SLN) and not for CCN
model which we are interested in. These techniques have been shown to be
working well in those noise models which estimate noise rates and hence we do
not make a comparison to these schemes.
Observations when model M is MLP For the case of synthetic dataset SD100 and
SD300, plots in column 1 and 2 of Figure 3 show that our scheme WGANXtraY(5)
has strictly higher accuracies in comparison to existing methods. Also, for
SD300, WGANXtraY(5) is not at all affected by very high CCN rates as its
accuracies are $\sim 99\%$ always. For binary versions of MNIST balanced
datasets in Figure 2, both our schemes top, especially when both
$\rho_{+},\rho_{-}$ are more than 0.45. For binary balanced Fashion MNIST
datasets in Figure 3, even though GLC and LDMI show comparable average
accuracy for some noise rates the variation across noise trials for them is
very high as observed in Table 31 to 38 of C. Figure 4 shows that for
imbalanced MNIST 0-8 and 1-7 datsets, WGANXtraY(5) always outperforms
SimpleNN, GCE and GLC; LDMI is comparable sometime. For MNIST 4-9 imbalanced
dataset with imbalance ratio $imb\\_r=0.75$, the performance is comparable to
GLC. We would like to point out that the imbalance ratio at two extreme levels
is considered which when combined with CCN results in a difficult problem. We
carried out statistical tests for multiple algorithm comparisons using [41] to
test the significance of our results. First, we performed Friedman F test [42]
individually for every binary dataset to check if there are any significant
differences between the performance of various label noise robust schemes. If
the the null hypothesis about the equal performance across all learning scheme
is rejected we performed pairwise Nemenyi posthoc test. The p values from all
the tests are presented in Table 1. As can be seen the p value for
approximately half of the cases (50/108 in blue colour) is less than 0.1, we
conclude that our WGAN based schemes lead to a significant improvement over
the existing schemes. Also, for rest of the 58 pair of schemes across various
datasets, the performance is comparable. We also observe that there is no
significant difference between the performance of WGANXtraY(5) and
WGANXtraYEntr(5). Further, based on the p value (in green) we observe that GLC
and LDMI have significantly better performance than GCE but have comparable
performance among themselves.
Observations when model M is a CNN As can be seen in Figure 5, the overall
accuracy across all balanced binary CCN corrupted MNIST versions for all
schemes has increased. However, the clear improvement of our WGAN based
schemes over GLC, LDMI, GCE as observed in Figure 2 is not visible here.
Similar kind of phenomenon is observed for Fashion MNIST and imbalanced binary
MNIST versions in Figure 6 and 7. The reason for this phenomenon is as
follows: WGAN based schemes use MLP in WGAN architecture and CNN to learn the
final classifier whereas the other schemes use CNN throughout the learning
process and hence get an undue advantage over WGAN based schemes. This
suggests that if one is interested in using a CNN as model M, then one has to
identify a suitable CNN architecture compatible representation that can be
used in the WGAN based schemes.
Finally, unlike other schemes like LDMI (see Figure 2, 3 and 4) that show a
decreasing trend in accuracy as noise rate increases, our schemes show stable
accuracy and AM values across noise rates in most of the cases. Also, we would
like to point out that GCE is theoretically shown to be SLN noise robust and
has been observed to be working empirically for CCN too in the literature.
However, in the current experiments, due to high class conditional noise
rates, its performance deteriorated.

Figure 2: Average accuracy plots for various binary class MNIST datasets across 5 trials when the final classification model M is a MLP. This plot demonstrates that even with only $0.1\%$ clean data points, at high noise rates close to 0.5, our scheme significantly improves over the state of the methods like GLC, GCE and LDMI in most cases. Figure 3: Average accuracy plots for Synthetic datasets SD100 and SD300 and binary class Fashion-MNIST datasets across 5 trials when the final classification model M is a MLP. This plot demonstrates that even with only $0.1\%$ clean data points, at high noise rates close to 0.5, WGANXtraY(5) dominates WGANXtraYEntr(5) and other stat of the art on SD100 and SD300 (Column 1and 2 from left). For Fashion-MNIST 0-6 (Tshirt and Shirt) and Fashion-MNIST 7-9 (Sneakers and Boots), our schemes achieve higher accuracy values than GLC and LDMI in more than half noise rates cases. Fashion-MNIST 2-4 (Pullover and Coat) inherently has low accuracy and adding label noise further decreases it. For Fashion-MNIST 2-3 (Pullover and Dress) our schemes are either better or on par with GLC and LDMI. Figure 4: Average AM (Arithmetic Mean of TPR and TNR) value plots for various imbalanced binary class MNIST datasets across 5 trials when the final classification model M is a MLP. “imb_r” denotes the fraction of positive data points. MNIST 0-8 and 1-7 use $0.1\%$ clean data and MNIST 4-9 uses $1\%$ clean data. AM values from SimpleNN on clean imbalanced datasets (from left to right) are $96.3\%,99.15\%,98.89\%,98.30\%,94.86\%,96.36\%$. Adverse effects of class conditional noise with imbalanced data on SimpleNN (red line) and GCE (purple line) can be seen by huge drop from aforementioned AM values to $\sim 50\%$ AM value. Figure 5: Average accuracy plots for various binary class MNIST datasets across 5 trials when the final classification model M is a CNN. WGAN based schemes, GLC and LDMI are observed to have accuracies within 5% of each other in most of the cases. However, GCE is no better than using a simple CNN for high noise rates. Also, unlike other schemes WGAN based schemes have very low variation across noise rates. Figure 6: Average accuracy plots for binary class Fashion-MNIST datasets across 5 trials when the final classification model M is a CNN. For Fashion-MNIST 0-6 (Tshirt and Shirt) and Fashion-MNIST 7-9 (Sneakers and Boots), our schemes achieve higher accuracy values than LDMI in more than half noise rates cases and has comparable accuracy to GLC. Fashion-MNIST 2-4 (Pullover and Coat) inherently is a difficult dataset and adding label noise further deteriorates its performance. For Fashion-MNIST 2-3 (Pullover and Dress) our schemes are either better or on par with GLC and LDMI. Figure 7: Average AM (Arithmetic Mean of TPR and TNR) value plots for various imbalanced binary class MNIST datasets across 5 trials when the final classification model M is a CNN. “imb_r” denotes the fraction of positive data points. MNIST 0-8 and 1-7 use $0.1\%$ clean data and MNIST 4-9 uses $1\%$ clean data. Our schemes perform better than LDMI and GLC in more than 3 datasets and for more than half of the noise rates. Also, the AM values from our schemes demonstrate more stability across noise rates in comparison to LDMI, GCE and GLC. | MNIST Balanced | Synthetic data | Fashion MNIST | MNIST imbalanced
---|---|---|---|---
Dataset | 5-6 digit | 7-9 digit | 4-9 digit | 1-7 digit | 0-6 digit | 0-1 digit | SD100 | SD300 | 7-9 pair | 0-6 pair | 2-4 pair | 2-3 pair | 0-8 (0.1) | 0-8(0.8) | 1-7(0.2) | 1-7(0.7) | 4-9(0.3) | 4-9(0.75)
| Friedman Test
---
p values
0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000
Algorithm pairs | Nemenyi posthoc test (pairwise) p values
| WGANXtraY(5)
---
vs GLC
1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.182573 | 0.000000 | 0.000000 | 1.000000 | 0.790576 | 1.000000 | 1.000000 | 0.000783 | 0.078567 | 0.037922 | 0.214194 | 1.000000 | 1.000000
| WGANXtraYEntr(5)
---
vs GLC
1.000000 | 1.000000 | 1.000000 | 0.000001 | 1.000000 | 0.603580 | 0.003984 | 0.001602 | 1.000000 | 1.000000 | 1.000000 | 0.455527 | 0.000783 | 0.078567 | 0.037922 | 0.214194 | 1.000000 | 1.000000
| WGANXtraY(5)
---
vs GCE
0.000101 | 0.000172 | 0.000997 | 0.000025 | 0.000172 | 0.000000 | 0.603580 | 0.603580 | 0.006168 | 0.000014 | 0.065782 | 0.000004 | 0.000006 | 0.000004 | 0.000000 | 0.000006 | 0.000001 | 0.000059
| WGANXtraYEntr(5)
---
vs GCE
0.000006 | 0.000101 | 0.000008 | 0.000001 | 0.000006 | 0.000006 | 1.000000 | 1.000000 | 0.003984 | 0.000613 | 0.002542 | 0.131472 | 0.000006 | 0.000004 | 0.000000 | 0.000006 | 0.000001 | 0.000059
| WGANXtraY(5)
---
vs LDMI
1.000000 | 1.000000 | 1.000000 | 0.603580 | 1.000000 | 0.131472 | 0.001602 | 0.000613 | 1.000000 | 1.000000 | 0.603580 | 1.000000 | 0.900907 | 1.000000 | 0.691773 | 0.900907 | 0.037922 | 0.078567
| WGANXtraYEntr(5)
---
vs LDMI
0.339817 | 1.000000 | 1.000000 | 0.093555 | 1.000000 | 0.455527 | 1.000000 | 0.603580 | 1.000000 | 1.000000 | 0.603580 | 1.000000 | 0.900907 | 1.000000 | 0.691773 | 0.900907 | 0.037922 | 0.078567
| WGANXtraY(5) vs
---
WGANXtraYEntr(5)
1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.339817 | 0.603580 | 1.000000 | 1.000000 | 1.000000 | 0.182573 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000
| Simple NN vs
---
WGANXtraY(5)
0.011614 | 0.007640 | 0.093555 | 0.000372 | 0.011614 | 0.000289 | 0.000045 | 0.000045 | 0.093555 | 0.006168 | 0.790576 | 0.002542 | 0.000783 | 0.000613 | 0.002021 | 0.000289 | 0.000223 | 0.000289
| Simple NN vs
---
WGANXtraYEntr(5)
0.001266 | 0.004965 | 0.002542 | 0.000014 | 0.000783 | 0.002021 | 0.250551 | 0.131472 | 0.065782 | 0.093555 | 0.065782 | 1.000000 | 0.000783 | 0.000613 | 0.002021 | 0.000289 | 0.000223 | 0.000289
GLC vs GCE | 0.004965 | 0.001266 | 0.021275 | 0.021275 | 0.003187 | 0.037922 | 0.001602 | 0.001602 | 0.003984 | 0.045702 | 0.045702 | 0.000025 | 1.000000 | 0.292219 | 0.182573 | 0.131472 | 0.001266 | 0.000997
Simple NN vs GLC | 0.214194 | 0.037922 | 0.790576 | 0.131472 | 0.111070 | 1.000000 | 1.000000 | 1.000000 | 0.065782 | 1.000000 | 1.000000 | 0.009434 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.078567 | 0.003984
GLC vs LDMI | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.603580 | 1.000000 | 0.002542 | 1.000000 | 0.790576 | 1.000000 | 0.455527 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.455527
Simple NN vs GCE | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.131472 | 0.131472 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000
GCE vs LDMI | 0.078567 | 0.011614 | 0.014254 | 0.093555 | 0.004965 | 0.054913 | 1.000000 | 0.603580 | 1.000000 | 0.000613 | 0.000014 | 0.000372 | 0.021275 | 0.007640 | 0.006168 | 0.021275 | 0.214194 | 1.000000
Simple NN vs LDMI | 1.000000 | 0.214194 | 0.603580 | 0.455527 | 0.004965 | 1.000000 | 1.000000 | 1.000000 | 0.045702 | 0.093555 | 0.000997 | 0.065782 | 0.455527 | 0.214194 | 1.000000 | 0.250551 | 1.000000 | 1.000000
Table 1: The table depicts the p value for the Friedman F test (row 3) and
Nemenyi posthoc test on the MLP based classifiers’ performance. Since all
Friedman F test based p values are 0, there is a significant difference
between the performance of the presented schemes. The p value in  blue  colour
represents the cases where we reject the null hypothesis that the performance
of WGAN based schemes is same as the existing scheme (column 1) at 10% level
of significance. Hence, we conclude that WGAN based schemes have either
comparable performance or lead to a statistically significant improvement in
the accuracy or AM values.  Green colour shows the significant values for the
comparison among existing schemes.

##  4 Discussion

We propose WGAN based schemes for CCN robust binary classification using a
small set of clean labels (0.1% or 1% of the training data). We exploit the
adversarial nature of WGANs for this purpose. But, we need to overcome a
couple of challenges to adapt conventional WGANs to be CCN robust. First, we
feed clean labelled data to the discriminator and corrupted labelled data to
the generator. Next, we include a new representation to address skewed ratio
of feature to label dimension. The second scheme, in addition to above, also
uses entropy ideas. We theoretically show that high noise rates lead to higher
KL divergence between clean and SLN corrupted distribution and in turn, to
better performance of our schemes. These aspects along with the generative
nature of our scheme makes it improve over existing discriminative schemes
like GLC, GCE and LDMI at high noise rates (close to 0.5), using small neural
networks and without knowing estimating the noise rates. The generative nature
of our schemes impart them with the potential of being CCN robust without any
modification, even when the data has class imbalance. To make a well-aware
choice of a GAN for generating labelled data, we explored WGAN, PacGAN, VEEGAN
and WGAN-GP and based on their performances selected WGAN in our work. We
demonstrate the good performance of our schemes and comparison to GLC, GCE and
LDMI on high dimensional binary class synthetic, MNIST and Fashion MNIST
datasets. We perform Friedman F test and Nemenyi posthoc test to statistically
support our claim about the improvement of our schemes over the existing
schemes. Finally, in most of our experiments we observed that the performance
(accuracy or AM values) of WGAN based schemes is stable across noise rates.
This is due to the fact that the labelled data used for training final
classification model M is generated by the WGAN while enforcing the fact that
its distribution should be as close as possible to the clean distribution.

Our work is an initial attempt to use GANs for CCN robust classification
without knowing/estimating the noise rates. A natural extension is to use it
for multi-class datasets where the representation idea has to be accordingly
changed in addition to the notion of high noise rates which would be close to
1. As we are not explicitly modelling the noise, in addition to SLN and CCN,
our scheme can possibly handle instance-dependent noise. The use of WGANs with
MLP architecture and classification model M with CNN architecture demonstrates
‘just’ comparable performance to existing schemes unlike the significantly
better performance when classification model M is MLP. This opens the
direction to identify representations which helps us use convolution in WGAN
for generating feature-label pair. An interesting avenue is to treat the
considered framework as a semi-supervised learning problem. An idea worth
exploring based on [43] could be to devise a framework that only uses 1 or 2
clean labelled samples to generate more labelled data and use it for
classification. The question here would be identify a suitable label
representation that is compatible with the scheme proposed in [43]. In
addition, one has to also answer the question of integrating the additional
noisy labelled data in the above setup, ignoring which could lead to loss of
information. Finally, the thread about the good performance of our schemes
could be complete if one could theoretically show that higher divergence
between the true and latent distribution in GAN leads to better performance of
GAN w.r.t. an appropriate measure.

## References

  * [1] C. Zhang, S. Bengio, M. Hardt, B. Recht, O. Vinyals, Understanding deep learning requires rethinking generalization, in: International Conference on Learning Representations (ICLR), 2017.
  * [2] P. Sastry, N. Manwani, Robust learning of classifiers in the presence of label noise, in: Pattern Recognition and Big Data, World Scientific, 2017, pp. 167–197.
  * [3] D. Hendrycks, M. Mazeika, D. Wilson, K. Gimpel, Using trusted data to train deep networks on labels corrupted by severe noise, in: Advances in Neural Information Processing Systems, 2018, pp. 10456–10465.
  * [4] M. Ren, W. Zeng, B. Yang, R. Urtasun, Learning to reweight examples for robust deep learning, in: International Conference on Machine Learning, 2018, pp. 4331–4340.
  * [5] T. Liu, D. Tao, Classification with noisy labels by importance reweighting, IEEE Transactions on Pattern Analysis and Machine Intelligence 38 (3) (2016) 447–461.
  * [6] B. Han, J. Yao, G. Niu, M. Zhou, I. Tsang, Y. Zhang, M. Sugiyama, Masking: A new perspective of noisy supervision, in: Advances in Neural Information Processing Systems, 2018, pp. 5836–5846.
  * [7] X. Ma, Y. Wang, M. E. Houle, S. Zhou, S. M. Erfani, S.-T. Xia, S. Wijewickrema, J. Bailey, Dimensionality-driven learning with noisy labels, in: International Conference on Machine Learning, 2018, pp. 3361–3370.
  * [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Advances in Neural Information Processing Systems, 2014, pp. 2672–2680.
  * [9] Z. Lin, A. Khetan, G. Fanti, S. Oh, PacGAN: The power of two samples in generative adversarial networks, in: Advances in Neural Information Processing Systems, 2018, pp. 1498–1507.
  * [10] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, A. C. Courville, Improved training of Wasserstein GANs, in: Advances in Neural Information Processing Systems, 2017, pp. 5767–5777.
  * [11] A. Srivastava, L. Valkov, C. Russell, M. U. Gutmann, C. Sutton, VEEGAN: Reducing mode collapse in gans using implicit variational learning, in: Advances in Neural Information Processing Systems, 2017, pp. 3308–3318.
  * [12] N. Natarajan, I. S. Dhillon, P. K. Ravikumar, A. Tewari, Learning with noisy labels, in: Advances in Neural Information Processing Systems, 2013, pp. 1196–1204.
  * [13] A. Ghosh, N. Manwani, P. Sastry, Making risk minimization tolerant to label noise, Neurocomputing 160 (2015) 93–107.
  * [14] B. Van Rooyen, A. Menon, R. C. Williamson, Learning with symmetric label noise: The importance of being unhinged, in: Advances in Neural Information Processing Systems, 2015, pp. 10–18.
  * [15] G. Patrini, F. Nielsen, R. Nock, M. Carioni, Loss factorization, weakly supervised learning and label noise robustness, in: International Conference on Machine Learning, 2016, pp. 708–717.
  * [16] S. Tripathi, N. Hemachandra, Cost sensitive learning in the presence of symmetric label noise, in: PAKDD, Springer, 2019, pp. 15–28.
  * [17] G. Patrini, A. Rozza, A. K. Menon, R. Nock, L. Qu, Making deep neural networks robust to label noise: A loss correction approach, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 2233–2241.
  * [18] A. Ghosh, H. Kumar, P. Sastry, Robust loss functions under label noise for deep neural networks., in: AAAI, 2017, pp. 1919–1925.
  * [19] A. K. Menon, B. van Rooyen, N. Natarajan, Learning from binary labels with instance-dependent noise, Machine Learning 107 (8-10) (2018) 1561–1595.  doi:10.1007/s10994-018-5715-3.
URL https://doi.org/10.1007/s10994-018-5715-3

  * [20] B. Han, Q. Yao, X. Yu, G. Niu, M. Xu, W. Hu, I. Tsang, M. Sugiyama, Co-teaching: Robust training of deep neural networks with extremely noisy labels, in: Advances in Neural Information Processing Systems, 2018, pp. 8536–8546.
  * [21] P. Chen, B. B. Liao, G. Chen, S. Zhang, Understanding and utilizing deep neural networks trained with noisy labels, in: International Conference on Machine Learning, 2019, pp. 1062–1070.
  * [22] E. Amid, M. K. Warmuth, R. Anil, T. Koren, Robust bi-tempered logistic loss based on bregman divergences, in: Advances in Neural Information Processing Systems, 2019, pp. 14987–14996.
  * [23] Z. Zhang, M. Sabuncu, Generalized cross entropy loss for training deep neural networks with noisy labels, in: Advances in neural information processing systems, 2018, pp. 8778–8788.
  * [24] Y. Xu, P. Cao, Y. Kong, Y. Wang, L_dmi: A novel information-theoretic loss function for training deep nets robust to label noise, in: Advances in Neural Information Processing Systems, 2019, pp. 6222–6233.
  * [25] Y. Kim, J. Yim, J. Yun, J. Kim, Nlnl: Negative learning for noisy labels, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 101–110.
  * [26] Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, J. Bailey, Symmetric cross entropy for robust learning with noisy labels, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 322–330.
  * [27] Y. Li, J. Yang, Y. Song, L. Cao, J. Luo, L.-J. Li, Learning from noisy labels with distillation, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1910–1918.
  * [28] A. Vahdat, Toward robustness against label noise in training deep discriminative neural networks, in: Advances in Neural Information Processing Systems, 2017, pp. 5596–5605.
  * [29] A. Veit, N. Alldrin, G. Chechik, I. Krasin, A. Gupta, S. Belongie, Learning from noisy large-scale datasets with minimal supervision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 839–847.
  * [30] K. K. Thekumparampil, A. Khetan, Z. Lin, S. Oh, Robustness of conditional gans to noisy labels, in: Advances in Neural Information Processing Systems, 2018, pp. 10271–10282.
  * [31] T. Kaneko, Y. Ushiku, T. Harada, Label-noise robust generative adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 2467–2476.
  * [32] M. Arjovsky, S. Chintala, L. Bottou, Wasserstein generative adversarial networks, in: International Conference on Machine Learning, 2017, pp. 214–223.
  * [33] Z. Zhao, L. Chu, D. Tao, J. Pei, Classification with label noise: A Markov chain sampling framework, Data Mining and Knowledge Discovery (2018) 1–37.
  * [34] S. Liu, O. Bousquet, K. Chaudhuri, Approximation and convergence properties of generative adversarial learning, in: Advances in Neural Information Processing Systems, 2017, pp. 5545–5553.
  * [35] J. T. Springenberg, Unsupervised and semi-supervised learning with categorical generative adversarial networks, in: International Conference on Learning Representations (ICLR), 2016.
URL https://arxiv.org/abs/1511.06390

  * [36] Y. Grandvalet, Y. Bengio, Semi-supervised learning by entropy minimization, in: Advances in neural information processing systems, 2005, pp. 529–536.
  * [37] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, et al., Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324.
  * [38] H. Xiao, K. Rasul, R. Vollgraf, Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms, arXiv preprint arXiv:1708.07747.
  * [39] N. Natarajan, I. S. Dhillon, P. Ravikumar, A. Tewari, Cost-sensitive learning with noisy labels, Journal of Machine Learning Research 18 (155) (2018) 1–33.
URL http://jmlr.org/papers/v18/15-226.html

  * [40] M. Koziarski, B. Krawczyk, M. Woźniak, Radial-based oversampling for noisy imbalanced data classification, Neurocomputing 343 (2019) 19–33.
  * [41] I. Rodríguez-Fdez, A. Canosa, M. Mucientes, A. Bugarín, STAC: a web platform for the comparison of algorithms using statistical tests, in: Proceedings of the 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2015.
  * [42] J. Demšar, Statistical comparisons of classifiers over multiple data sets, Journal of Machine learning research 7 (Jan) (2006) 1–30.
  * [43] T. R. Shaham, T. Dekel, T. Michaeli, Singan: Learning a generative model from a single natural image, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 4570–4580.
  * [44] Jochen, Interchange of derivative with an expectation, https://math.stackexchange.com/questions/217702/when-can-we-interchange-the-derivative-with-an-expectation (2016).
  * [45] B. Efron, R. Tibshirani, Improvements on cross-validation: the 632+ bootstrap method, Journal of the American Statistical Association 92 (438) (1997) 548–560.

##  Appendix A Proofs

###  A.1 Proof of Lemma 1

##### Proof:

Let the joint density corresponding to clean ($\mathcal{D}$) and noisy
distribution ($\tilde{\mathcal{D}}$) be $p_{c}(\mathbf{x},y)$ and
$p_{n}(\mathbf{x},y)$ respectively. Then, we have

$\displaystyle
KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{SLN}=\int\limits_{\mathbf{x},y}p_{c}(\mathbf{x},y)\log\frac{p_{c}(\mathbf{x},y)}{p_{n}(\mathbf{x},y)}d\mathbf{x}dy$
$\displaystyle=\int\limits_{\mathbf{x},y}p_{c}(y|\mathbf{x})p_{c}(\mathbf{x})\log\frac{p_{c}(y|\mathbf{x})p_{c}(\mathbf{x})}{p_{n}(y|\mathbf{x})p_{n}(\mathbf{x})}d\mathbf{x}dy.$

Since, in $\tilde{\mathcal{D}}$ only label $Y$ is corrupted
$p_{n}(\mathbf{x})=p_{c}(\mathbf{x})$. Also, since $Y$ is binary label, we can
write the KL divergence as follows:

$\displaystyle
KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{SLN}=\int\limits_{\mathbf{x}}\left[\sum\limits_{y}p_{c}(y|\mathbf{x})\log\frac{p_{c}(y|\mathbf{x})}{p_{n}(y|\mathbf{x})}\right]p_{x}(\mathbf{x})d\mathbf{x}$
$\displaystyle=\mathbb{E}_{\mathbf{X}}[KL(p_{c}(Y|\mathbf{X})\|p_{n}(Y|\mathbf{X}))].$
(8)

The KL term inside expectation in Eq. (8) can be simplified using the fact
that $p_{c}(Y=1|\mathbf{X}=\mathbf{x})=\eta(\mathbf{x})$ and
$p_{n}(Y=1|\mathbf{X}=\mathbf{x})=\tilde{\eta}(\mathbf{x})$.

$\displaystyle
KL(p_{c}(Y|\mathbf{X}=\mathbf{x})\|p_{n}(Y|\mathbf{X}=\mathbf{x}))=\eta(\mathbf{x})\log\frac{\eta(\mathbf{x})}{\tilde{\eta}(\mathbf{x})}$
$\displaystyle+(1-\eta(\mathbf{x}))\log\frac{1-\eta(\mathbf{x})}{1-\tilde{\eta}(\mathbf{x})}.$
(9)

Also, since for SLN case,
$\tilde{\eta}(\mathbf{x})=(1-2\rho)\eta(\mathbf{x})+\rho$, we have,

$\displaystyle
KL(p_{c}(Y|\mathbf{X}=\mathbf{x})\|p_{n}(Y|\mathbf{X}=\mathbf{x}))$
$\displaystyle=-\eta(\mathbf{x})\log\left(1+\rho\left(\frac{1-2\eta(\mathbf{x})}{\eta(\mathbf{x})}\right)\right)$
$\displaystyle-(1-\eta(\mathbf{x}))\log\left(1-\rho\left(\frac{1-2\eta(\mathbf{x})}{\eta(\mathbf{x})}\right)\right).$
(10)

Taking derivative of Eq. (10) w.r.t. noise rate $\rho$, we get

$\frac{\rho(1-2\eta(\mathbf{x}))^{2}}{(\eta(\mathbf{x})+\rho(1-2\eta(\mathbf{x})))(1-\eta(\mathbf{x})-\rho(1-2\eta(\mathbf{x})))}.$
(11)

The derivative in Eq. (11) is always positive $\forall~{}\mathbf{x}$ implying
that the KL divergence between conditional clean and noisy distribution for a
given data point $\mathbf{x}$ is an increasing function of $\rho$. Next, to
show the monotonicity of $KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{SLN}$, using
Eq. (8), we have to verify the following:

$\frac{d}{d\rho}\mathbb{E}_{\mathbf{X}}[KL(p_{c}(Y|\mathbf{X})\|p_{n}(Y|\mathbf{X}))]=\mathbb{E}_{\mathbf{X}}\frac{d}{d\rho}KL(p_{c}(Y|\mathbf{X})\|p_{n}(Y|\mathbf{X})).$

The above condition holds if the derivative in Eq. (11) can be uniformly upper
bounded by an integrable function [44]. Since, $\rho<0.5$ lower bounds both
the terms in the denominator of Eq. (11), we obtain the following:

$\displaystyle\frac{\rho(1-2\eta(\mathbf{x}))^{2}}{(\eta(\mathbf{x})+\rho(1-2\eta(\mathbf{x})))(1-\eta(\mathbf{x})-\rho(1-2\eta(\mathbf{x})))}<\frac{\rho(1-2\eta(\mathbf{x}))^{2}}{\rho^{2}}.$
(12)

Now, since, $\eta(\mathbf{x})\in[0,1]$, $1$ is a trivial upper bound of
$(1-2\eta(\mathbf{x}))^{2}$, implying that the derivative in Eq. (11) can be
upper bounded by $1/\rho$. This implies that the required upper bound on
derivative from [44] is a constant and hence, integrable trivially. Since, the
conditions for interchanging the derivative and expectation are satisfied, the
monotonically increasing nature of
$KL(p_{c}(Y|\mathbf{X})\|p_{n}(Y|\mathbf{X}))$ implies that
$KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{SLN}$ is also a monotone increasing
function of noise rate $\rho$.

$\square$

###  A.2 Proof of Lemma 2

##### Proof:

Let the joint density corresponding to clean ($\mathcal{D}$) and noisy
distribution ($\tilde{\mathcal{D}}$) be $p_{c}(\mathbf{x},y)$ and
$p_{n}(\mathbf{x},y)$ respectively. Then, similar to Eq. (8) we have,

$\displaystyle
KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{CCN}=\mathbb{E}_{\mathbf{X}}[KL(p_{c}(Y|\mathbf{X})\|p_{n}(Y|\mathbf{X}))],$
$\displaystyle=\mathbb{E}_{\mathbf{X}}\left[\eta(\mathbf{x})\log\frac{\eta(\mathbf{x})}{\tilde{\eta}(\mathbf{x})}+(1-\eta(\mathbf{x}))\log\frac{1-\eta(\mathbf{x})}{1-\tilde{\eta}(\mathbf{x})}\right].$

Since, the in-class probability relation for CCN case is
$\tilde{\eta}(\mathbf{x})=(1-\rho_{+}-\rho_{-})\eta(\mathbf{x})+\rho_{-}$, we
have,

$\displaystyle
KL(p_{c}(Y|\mathbf{X}=\mathbf{x})\|p_{n}(Y|\mathbf{X}=\mathbf{x}))$
$\displaystyle=-\eta(\mathbf{x})\log\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)$
$\displaystyle-(1-\eta(\mathbf{x}))\log\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right).$
(13)

Taking the derivative of Eq. (13) w.r.t. $\rho_{+}$, we get

$\displaystyle\eta(\mathbf{x})\left[\frac{1}{\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)}-\frac{1}{\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right)}\right]$
$\displaystyle=\left[\frac{\eta(\mathbf{x})(\eta(\mathbf{x})\rho_{+}-(1-\eta(\mathbf{x}))\rho_{-})}{(1-\eta(\mathbf{x}))\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right)}\right].$
(14)

Next, taking derivative w.r.t $\rho_{-}$, we have,

$\displaystyle(1-\eta(\mathbf{x}))\left[\frac{-1}{\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)}+\frac{1}{\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right)}\right]$
$\displaystyle=\left[\frac{(1-\eta(\mathbf{x}))((1-\eta(\mathbf{x}))\rho_{-}-\eta(\mathbf{x})\rho_{+})}{\eta(\mathbf{x})\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right)}\right].$
(15)

Clearly, from Eq. (14) and (15) both the derivatives w.r.t. $\rho_{+}$ and
$\rho_{-}$ cannot be positive simultaneously. Hence, the projection approach
of showing monotonicity of a function of two variables fails. Next, for the
total derivative approach along the diagonal, we take the inner product of the
derivatives with the direction $h=(1,1)$, and obtain the following,

$\displaystyle\frac{(2\eta(\mathbf{x})-1)(\eta(\mathbf{x})\rho_{+}-(1-\eta(\mathbf{x}))\rho_{-})}{\eta(\mathbf{x})(1-\eta(\mathbf{x}))\left(1-\rho_{+}-\rho_{-}\left(1-\frac{1}{\eta(\mathbf{x})}\right)\right)\left(1-\rho_{-}+\rho_{+}\left(\frac{\eta(\mathbf{x})}{1-\eta(\mathbf{x})}\right)\right)}.$

This total derivative can be negative for some $\mathbf{x}$ and positive for
some other $\mathbf{x}$ and hence,
$KL[\mathcal{D}\|\tilde{\mathcal{D}}]_{CCN}$ is not an increasing function of
noise rates $\rho_{+}$ and $\rho_{-}$. $\square$

##  Appendix B Additional computational experiments

In this section, we provide empirical evidence for the various observations
made in the main paper. First, we empirically show the implications of Lemma
1. Next, we show how change in representation of the data lead to improvement
in the basic setup. Finally, we demonstrate our claim about some variants of
GAN which are not good for generating correctly labelled data, even though in
the conventional GAN setup they work well.

###  B.1 Dataset sizes and architecture details of different neural networks
used

In this section, we provide the synthetic data generation scheme, our approach
of constructing binary datasets and the architecture of the neural networks
used.

####  B.1.1 Synthetic dataset generation

For synthetic data generation, we use the method given by [45]. We first
generate $6000$ train and $750$ test binary class labels $Y\sim Bern(0.5)$.
Next, given feature dimension $n$, we generate two $n$ dimensional mean
vectors
$\bm{\mu}_{+}=(\mu_{+,1},\cdots,\mu_{+,n}),~{}\bm{\mu}_{-}=(\mu_{-,1},\cdots,\mu_{-,n})$
as follows: for every dimension $j=1,\cdots,n$, draw a sample from Unif(-2,2)
and assign it to $\mu_{+,j}$; next draw a sample from Bern(0.4), if it is 1,
$\mu_{-,j}=\mu_{+,j}$ else $\mu_{-,j}=-\mu_{+,j}$. Also, the covariance matrix
$\Sigma$ is such that all the diagonal elements (variances) are $8$ and the
non-diagonal elements (covariances) are $0$. Now, a n-dimensional feature
vector $\mathbf{X}$ for each label $Y$ is drawn from two different Gaussian
distributions: $\mathbf{X}|Y=1\sim N(\bm{\mu}_{+},\Sigma)$ $\&$
$\mathbf{X}|Y=-1\sim N(\bm{\mu}_{-},\Sigma)$. This process is repeated for
both $n=100$ and $n=300$ to obtain the synthetic datasets SD100 and SD300 used
in Section 3.

####  B.1.2 Real datasets

To obtain the binary versions, say class (a,b) of multi-class MNIST and
Fashion MNIST datasets, we combine class $a$ and class $b$ examples only to
obtain the final binary datasets. In case of imbalanced dataset with imbalance
ratio $imb\\_r=r$, we randomly sample $100*r$ percent of the data points from
class $a$ and $100*(1-r)$ percent of the data points from class $b$. In case
when one of the class is $0$, we use the following strategy: if $a(b)=0$, we
sample $100*r\%$ from class $b(a)$ and rest $100*(1-r)\%$ from class $a(b)$.
The exact sample sizes for various datasets used in Section 3 are given in
Table 2.

S. no. | Dataset name | $m_{tr}(p_{tr},n_{tr})$ | $m_{te}(p_{te},n_{te})$
---|---|---|---
1 | MNIST 5-6 | 10404 (4987, 5417) | 1850 (892, 958)
2 | MNIST 7-9 | 11169 ( 5715, 5454) | 2037 (1028, 1009)
3 | MNIST 4-9 | 10761 (5307, 5454) | 1991 (982, 1009)
4 | MNIST 1-7 | 11894 (6179, 5715) | 2163 (1135, 1028)
5 | MNIST 0-6 | 10861 (5417, 5444) | 1938 (958, 980)
6 | MNIST 0-1 | 11623 (6179, 5444) | 2115 (1135, 980)
7 |  | Fashion MNIST 7-9
---
(Sneakers and Boots)
12000 (6000, 6000) | 2000 (1000, 1000)
8 |  | Fashion MNIST 2-4
---
(Pullover and Coat)
12000 (6000, 6000) | 2000 (1000, 1000)
9 |  | Fashion MNIST 0-6
---
(Tshirt and Shirt)
12000 (6000, 6000) | 2000 (1000, 1000)
10 |  | Fashion MNIST 2-3
---
(Pullover and Dress)
12000 (6000, 6000) | 2000 (1000, 1000)
11 | MNIST 0-8 (imb_r = 0.1) | 5437 (538, 4899) | 979 (97,882)
12 | MNIST 0-8 (imb_r = 0.8) | 5399 (4311, 1088) | 974 (779, 195)
13 | MNIST 1-7 (imb_r = 0.2) | 5807 (1235, 4572) | 1049 (222, 822)
14 | MNIST 1-7 (imb_r = 0.7) | 6039 (4325, 1714) | 1102 (794, 308)
15 | MNIST 4-9 (imb_r = 0.3) | 5409 (1592, 3817) | 1000 (294, 706)
16 | MNIST 4-9 (imb_r = 0.75) | 5343 (3980, 1363) | 988 (736, 252)
Table 2: Dataset details for the binary versions of MNIST and Fashion MNIST
datasets. $m_{tr}$ and $m_{te}$ denote the number of train and test data
points. $p_{tr}(n_{tr})$ and $p_{te}(n_{te})$ denote number of positive
(negative) data points in the training and test datasets. Dataset number 1 to
10 are balanced and 11-16 are imbalanced with imbalance ratio $imb\\_r$.

####  B.1.3 Architecture details

We have used multilayer perceptrons for the generator $G_{dn}$, discriminator
$D_{dn}$.The final classification model $M$ can either be an MLP or a CNN. The
exact details are provided in Table 3, Table 4 and Table 5.

Generator | Discriminator
---|---
Layer | Neurons | Activation | Layer | Neurons | Activation
Input | 785+4 | - | Input | 785+4 | -
Fully connected | 64 | Relu | Fully connected | 128 | Relu
Fully connected | 128 | Relu | Fully connected | 128 | Relu
Fully connected | 128 | Relu | Output | 1 | Linear
Output | 785+4 | - |  |  |
Table 3: The table depicts the architecture of generator and discriminator used in all the computations. For the generator, the last layer has extra 4 dimensions appended once the label has been decided after majority voting. Classification model M (MLP) | GLC network (MLP)
---|---
Layer | Neurons | Activation | Layer | Neurons | Activation
Input | 784 | - | Input | 784 | -
Fully connected | 185 | Relu | Fully connected | 128 | Relu
Fully connected | 200 | Relu | Fully connected | 128 | Relu
Fully connected | 185 | Relu | Output | 2 | Softmax
Output | 2 | Softmax |  |  |
Table 4: The table provides the architecture of classification network $M$
which is an MLP. It is used by SimpleNN, GCE and LDMI to train on noisy data
and our schemes to train on the correctly labelled data points generated by
the generator. GLC network is used by the GLC [3] method for classification.
Classification model M (CNN)
---
Layer | Filter/kernel size | Activation
Input | 1*28*28 | -
Conv2D |  | In channel=1, out channel =16, kernel size = 5,
---
Maxpool (kernel size =2)
Relu
Conv2D |  | In channel=16, out channel =32, kernel size = 5,
---
Maxpool(kernel size =2), Dropout (p=0.5)
Relu
Output | 2 | Linear
Table 5: The table provides the architecture of classification network M which
is a CNN. It is used by Simple CNN, GLC, GCE and LDMI to train on noisy data
and our schemes to train on the correctly labelled data points generated by
the generator.

###  B.2 Experiments to show that increase in SLN rate $\rho$ leads to
improvement in performance of WGANXtraY

In this section, we present empirical evidence for the implications of Lemma 1
in Section 2.3 using the experimental results from 4 binary classification
image datasets,viz., MNIST 1-7, MNIST 0-6, MNIST 4-9, and Fashion-MNIST 2-3.
As can be seen in Table 6 and 7, relative performance of WGANXtraY(5) improves
with increase in noise rate $\rho$. The size of clean dataset used is $0.1\%$
of the total training dataset for both GLC and our scheme. Number of outer
loop iterations $n_{it}=1000.$ To account for the randomness while inducing
noise, we present averaged accuracy and standard deviation across 5 trials.

$\rho$ | MNIST 1-7 (0.1% clean) | MNIST 0-6 (0.1% clean)
---|---|---
| WGANXtraY(5) | GLC | WGANXtraY(5) | GLC
0.45 |  93.90 $\pm$ 1.88 | 94.59 $\pm$ 3.32 |  85.20$\pm$ 4.31 | 89.39 $\pm$ 6.81
0.46 | 95.92 $\pm$ 0.30 |  91.63 $\pm$ 6.57 | 91.86 $\pm$ 1.49 |  84.07$\pm$ 8.79
0.47 | 94.88$\pm$ 1.32 |  84.49$\pm$ 8.94 | 90.33 $\pm$ 1.67 |  83.85 $\pm$ 10.18
0.48 | 94.84$\pm$ 1.58 |  82.61 $\pm$ 16.51 |  87.88 $\pm$ 2.89 | 88.10 $\pm$ 4.56
0.49 | 93.58 $\pm$ 1.87 |  76.36 $\pm$ 11.02 | 89.90 $\pm$ 0.91 |  84.81 $\pm$ 5.05
Table 6: Average accuracy and standard deviation over 5 trials on MNIST 1-7 and MNIST 0-6 datasets when the classification model M is an MLP. Table values demonstrates the relative improvement of WGANXtraY(5) over GLC as the SLN rates are increased. This phenomenon is due to increase in adversarial relation (measured in terms of KL divergence between $\mathcal{D}$ and $\tilde{\mathcal{D}}$) as the noise rate increases. Bold-faced values denote highest accuracy values in a row. $\rho$ | MNIST 4-9 (0.1% clean) | Fas-MNIST 2-3 (0.1% clean)
---|---|---
| WGANXtraY(5) | GLC | WGANXtraY(5) | GLC
0.45 |  74.64 $\pm$ 5.32 | 76.66 $\pm$ 7.23 | 96.03 $\pm$ 0.51 |  95.55 $\pm$ 1.29
0.46 |  72.07 $\pm$ 4.64 | 79.38 $\pm$ 5.01 | 86.69 $\pm$ 17.31 |  84.5 $\pm$ 14.12
0.47 | 72.73 $\pm$ 2.76 |  63.21 $\pm$ 11.00 | 95.97 $\pm$ 0.90 |  88.56 $\pm$ 16.43
0.48 | 76.67 $\pm$ 2.25 |  64.74 $\pm$ 11.11 | 94.16 $\pm$ 2.64 |  90.61 $\pm$ 7.06
0.49 | 74.20 $\pm$ 2.42 |  58.29 $\pm$ 9.00 | 92.75 $\pm$ 2.15 |  85.25 $\pm$ 15.92
Table 7: Average accuracy and standard deviation over 5 trials on MNIST 4-9
and Fashion-MNIST 2-3 (Pullover and Dress) datasets when the classification
model M is an MLP. Table values demonstrates the relative improvement of
WGANXtraY(5) over GLC as the SLN rates are increased. This phenomenon is due
to increase in adversarial relation (measured in terms of KL divergence
between $\mathcal{D}$ and $\tilde{\mathcal{D}}$) as the noise rate increases.
Bold-faced values denote higher accuracy values.

###  B.3 Experiments to show bad performance of WGANY

In this section, we present the experimental results showing that both the
generated image quality and the performance of final classifier is not good
for WGANY (scheme without representation changes). We demonstrate this using 2
binary datasets, viz., MNIST 7-9 and Fashion-MNIST 7-9 (Sneakers and Boots).
As can be seen from Table 8 and 9, WGANY has low accuracies in comparison to
WGANXtraY(5) across all CCN rates. This situation doesn’t improve even when
the clean sample size is increased to $1\%$, as can be seen by comparing
column 3 and 4 of Table 8 and 9. In addition to the accuracy, we show that the
correct pairing of feature and label is bad for WGANY but improves by using
WGANXtraY(5) as shown in Figure 8 and Figure 9.

Figure 8: The feature-label pairs generated by WGANY and WGANXtraY(5) when the noise rates were $(0.1,0.2)$ and $(0.46,0.48)$. 7 is positive class and 9 is negative class. First two rows have more inconsistency in the labels assigned to the generated images than last row, demonstrating the improvement due to WGANXtraY(5). Percentage of clean data used is mentioned along with the images. Figure 9: The feature-label pairs generated by WGANY and WGANXtraY(5) when the noise rates were $(0.1,0.2)$ and $(0.46,0.48)$. Sneakers (7) is positive class and Boots (9) is negative class. First two rows have more inconsistency in the labels assigned to the generated images than last row, demonstrating the improvement due to WGANXtraY(5). Percentage of clean data used is mentioned along with the images. $\rho_{+},\rho_{-}$ | WGANY | WGANXtraY(5)
---|---|---
clean % | 0.1% | 1% | 0.1%
0.1,0.2 |  53.03 $\pm$ 11.67 |  60.03 $\pm$ 7.11 | 77.20 $\pm$ 1.36
0.2,0.49 |  54.22$\pm$ 5.52 |  44.82 $\pm$ 14.45 | 80.88 $\pm$ 1.85
0.42,0.35 |  53.49$\pm$ 3.98 |  43.93 $\pm$ 11.67 | 81.90 $\pm$ 2.23
0.3,0.3 |  58.46 $\pm$ 10.84 |  64.81$\pm$ 17.49 | 80.56 $\pm$ 5.04
0.4,0.49 |  54.00 $\pm$ 10.31 |  59.73$\pm$ 7.33 | 78.8$\pm$ 1.33
0.42,0.45 |  51.71 $\pm$ 3.81 |  56.14 $\pm$ 17.99 | 75.9$\pm$ 3.87
0.45,0.46 |  47.94 $\pm$ 4.13 |  63.05 $\pm$ 11.70 | 81.49$\pm$ 1.2
0.45,0.48 |  49.19 $\pm$ 1.19 |  58.85 $\pm$ 17.48 | 81.1$\pm$ 2.56
0.45,0.5 |  51.22 $\pm$ 3.12 |  51.12 $\pm$ 7.14 | 78.0$\pm$ 4.2
0.46,0.48 |  49.23 $\pm$ 12.08 |  50.08 $\pm$ 19.61 | 80.79 $\pm$ 2.37
0.46,0.5 |  54.11 $\pm$ 9.75 |  52.45 $\pm$ 10.29 | 80.3 $\pm$ 1.88
0.47,0.49 |  48.89 $\pm$ 2.02 |  51.84 $\pm$ 9.44 | 76.9$\pm$ 4.54
0.47,0.51 |  52.00 $\pm$ 2.52 |  53.42 $\pm$ 15.04 | 78.9$\pm$ 1.70
0.48,0.49 |  53.18 $\pm$ 6.84 |  50.20 $\pm$ 7.09 | 80.7 $\pm$ 2.07
0.5,0.5 |  51.69$\pm$ 3.16 |  51.59$\pm$ 7.07 | 79.2$\pm$ 1.51
Table 8: Averaged accuracy and standard deviation values for MNIST 7-9 dataset for 5 trials when the classification model M is an MLP. The table demonstrates that WGANY doesn’t perform well and WGANXtraY(5) show a significant improvement (last column) over WGANY even when clean data percent is only 0.1%. The number of iterations $n_{it}$ is 1000 in all 3 cases.  $\rho_{+},\rho_{-}$ | WGANY | WGANXtraY(5)
---|---|---
clean % | 0.1% | 1% | 0.1%
0.1,0.2 |  49.14 $\pm$ 12.72 |  61.48 $\pm$ 19.92 | 87.76$\pm$ 1.28
0.2,0.49 |  35.78 $\pm$ 17.54 |  63.27 $\pm$ 12.65 | 87.18 $\pm$ 1.80
0.42,0.35 |  42.36 $\pm$ 21.24 |  59.92$\pm$ 11.73 | 85.94 $\pm$ 2.08
0.3,0.3 |  70.82$\pm$ 17.49 |  45.67 $\pm$ 8.39 | 85.56 $\pm$ 4.88
0.4,0.49 |  57.57 $\pm$ 15.13 |  61.25 $\pm$ 15.53 | 86.35 $\pm$ 2.02
0.42,0.45 |  35.56 $\pm$ 24.41 |  59.06 $\pm$ 5.95 | 78.66 $\pm$ 14.43
0.45,0.46 |  40.23 $\pm$ 20.11 |  48.39 $\pm$ 30.03 | 87.16 $\pm$ 1.03
0.45,0.48 |  55.61 $\pm$ 7.01 |  48.39 $\pm$ 18.08 | 87.95 $\pm$ 1.41
0.45,0.5 |  46.44 $\pm$ 28.41 |  26.97 $\pm$ 20.02 | 87.38 $\pm$ 1.74
0.46,0.48 |  48.47 $\pm$ 7.18 |  45.74 $\pm$ 8.59 | 87.53 $\pm$ 0.69
0.46,0.5 |  49.07 $\pm$ 22.59 |  51.48 $\pm$ 29.91 | 82.55 $\pm$ 10.03
0.47,0.49 |  49.2 $\pm$ 1.59 |  64.57$\pm$ 13.01 | 85.85 $\pm$ 2.81
0.47,0.51 |  52.25 $\pm$ 22.08 |  45.01 $\pm$ 23.33 | 70.55 $\pm$ 35.29
0.48,0.49 |  45.91 $\pm$ 10.40 |  51.01 $\pm$ 11.79 | 86.07 $\pm$ 1.32
0.5,0.5 |  55.1 $\pm$ 10.2 |  26.9 $\pm$ 23.36 | 85.61 $\pm$ 1.71
Table 9: Averaged accuracy and standard deviation values for Fashion-MNIST 7-9
(Sneakers and Boots) dataset for 5 trials when the classification model M is
an MLP. The table demonstrates that WGANY doesn’t perform well and
WGANXtraY(5) show a significant improvement (last column) over WGANY even when
clean data percent is only 0.1%. The number of iterations $n_{it}$ is 1000 in
all 3 cases.

###  B.4 Experiments to show that good GANs for generating samples (feature
vector/images) need not be good for generating correctly labelled data

In this section, we present empirical evidence to show what choices of GAN
(good for generating features (images) only) did not work for us with reasons
given in Section 2.3.4. We verify this on 4 datasets, viz., MNIST 7-9, MNIST
1-7, Fashion-MNIST 7-9 and Fashion-MNIST 2-3 and the results are presented in
Table 10, 11, 12 and 13. We present the results for WGANXtraY(5) with Gradient
Penalty with parameter $\lambda=10$ (WGANXtraY-GP), PacGAN adapted to our
setup with packing number as 3 (PacGAN(3)), and a variant of WGANXtraY with
reconstruction loss (pixel-wise mean squared error between the images) added
to the generator objective (WGANXraY-R). The last reconstruction based variant
is enforcing the generator $G_{dn}$ that for a corrupted data point
$(\mathbf{x},\tilde{y})$, the new data point
$(\mathbf{x}^{\prime},y^{\prime})$ should have $\mathbf{x}^{\prime}$ close to
$\mathbf{x}$ along with correct label $y^{\prime}$. After trying different
weights for the reconstruction loss term in the generator, we use weight value
of 1 as it worked best for us. Also, since, for PacGAN(3) and WGANXtraY-GP
using $0.1\%$ gold fraction lead to issue of the minibatch size being more
than the size of clean labelled dataset, we use $1\%$ clean data for the
computations presented. We continue to take $k=5$ for all WGANXtraY based
schemes. We compare the accuracies on final classification model trained on
correctly labelled generated data from the schemes aforementioned and
WGANXtraY. The number of iterations $n_{it}=5000$ for PacGAN(3) and WGANXtraY-
GP and $n_{it}=1000$ for WGANXtraY and WGANXtraY-R. The batch size is taken to
be 64 for all schemes except for PacGAN(3) for which batch size is 32.

Clearly, PacGAN(3) adapted for the task of generating correctly labelled data
from noisy data doesn’t work as seen by the very low accuracy values of the
final classification model in all 4 datasets. Even though, WGANXtraY-R is
comparable in some cases to WGANXtraY, the variation as quantified by the
standard deviation across 5 trials is very high in almost all cases.
WGANXtraY-GP performs comparably to WGANXtraY in some cases and better than
WGANXtraY in other cases for Fashion MNIST 7-9 and 2-3 datasets. We attribute
this behaviour to the use of large number of iterations ($n_{it}=5000$).
Since, we wanted to use minimal number of clean labels and minimal iterations
we chose WGANXtraY over WGANXtraY-GP.

$\rho_{+},\rho_{-}$ | WGANXtraY | WGANXtraY-GP | WGANXtraY-R | PacGAN(3)
---|---|---|---|---
0.4,0.49 | 90.89 $\pm$ 1.85 |  81.07 $\pm$ 14.96 |  84.00 $\pm$ 2.84 |  49.30 $\pm$ 2.24
0.42,0.45 | 89.31 $\pm$ 1.36 |  80.93 $\pm$ 15.39 |  87.43 $\pm$ 2.85 |  50.37$\pm$ 1.27
0.45,0.46 | 89.13 $\pm$ 2.40 |  86.88 $\pm$ 4.22 |  85.70$\pm$ 2.40 |  48.72 $\pm$ 4.04
0.45,0.48 | 88.84 $\pm$ 3.09 |  79.87$\pm$ 15.19 |  85.43$\pm$ 4.53 |  53.96 $\pm$ 5.11
0.45,0.5 | 90.01 $\pm$ 1.99 |  88.64$\pm$ 2.86 |  85.42$\pm$ 4.62 |  52.02$\pm$ 5.17
0.46,0.48 | 89.11 $\pm$ 2.50 |  85.10 $\pm$ 4.55 |  83.69 $\pm$ 4.06 |  39.07 $\pm$ 21.75
0.46,0.5 | 89.35 $\pm$ 0.81 |  88.76 $\pm$ 2.01 |  86.02 $\pm$ 4.57 |  30.09$\pm$ 24.57
0.47,0.49 | 90.56 $\pm$ 0.90 |  88.05 $\pm$ 3.09 |  86.33 $\pm$ 2.46 |  50.21 $\pm$ 0.59
0.47,0.51 |  89.13 $\pm$ 1.23 |  84.75 $\pm$ 3.81 | 89.44 $\pm$ 0.87 |  39.15 $\pm$ 19.84
0.48,0.49 | 89.53 $\pm$ 0.61 |  86.90 $\pm$ 4.54 |  88.67 $\pm$ 0.57 |  41.65 $\pm$ 21.11
0.5,0.5 | 90.01 $\pm$ 1.19 |  85.90 $\pm$ 4.43 |  88.67 $\pm$ 0.57 |  47.41 $\pm$ 4.05
Table 10: Averaged accuracy and standard deviation values for MNIST 7-9 dataset across 5 trials when the classification model M is an MLP. The table implies that WGANXtraY is better than other schemes most of the time when clean data percent available is only 1%. This demonstrates that good GANs (columns 3,4 and 5) for generating samples need not be good for generating correctly labelled data points from noisy labelled data points.  $\rho_{+},\rho_{-}$ | WGANXtraY | WGANXtraY-GP | WGANXtraY-R | PacGAN(3)
---|---|---|---|---
0.4,0.49 | 96.06 $\pm$ 1.02 |  93.39$\pm$ 2.71 |  93.52$\pm$ 1.77 |  48.60 $\pm$ 3.55
0.42,0.45 |  95.85 $\pm$ 1.00 | 96.23$\pm$ 1.56 |  90.81 $\pm$ 7.39 |  49.50 $\pm$ 2.42
0.45,0.46 | 95.58 $\pm$ 2.72 |  91.68$\pm$ 5.82 |  92.21 $\pm$ 4.27 |  51.56 $\pm$ 6.41
0.45,0.48 | 95.09 $\pm$ 1.06 |  93.34 $\pm$ 4.14 |  91.00$\pm$ 5.71 |  51.82 $\pm$ 11.04
0.45,0.5 | 95.55 $\pm$ 1.21 |  86.13 $\pm$ 17.26 |  91.93 $\pm$ 1.82 |  54.96 $\pm$ 10.16
0.46,0.48 | 95.95 $\pm$ 1.25 |  94.40 $\pm$ 4.63 |  90.50 $\pm$ 8.44 |  47.46 $\pm$ 5.21
0.46,0.5 | 96.56 $\pm$ 0.45 |  82.92 $\pm$ 17.00 |  93.25$\pm$ 3.15 |  50.28 $\pm$ 3.59
0.47,0.49 | 95.10 $\pm$ 0.56 |  93.40 $\pm$ 3.70 |  94.23 $\pm$ 1.82 |  38.65 $\pm$ 20.34
0.47,0.51 |  95.58$\pm$ 0.35 | 95.64 $\pm$ 1.53 |  88.68 $\pm$ 9.69 |  49.07 $\pm$ 4.54
0.48,0.49 | 96.06$\pm$ 1.69 |  94.85 $\pm$ 3.44 |  91.86 $\pm$ 1.14 |  51.50 $\pm$ 1.98
0.5,0.5 | 96.06 $\pm$ 0.41 |  94.93 $\pm$ 3.35 |  95.04 $\pm$ 1.48 |  40.11 $\pm$ 20.19
Table 11: Averaged accuracy and standard deviation values for MNIST 1-7 dataset across 5 trials when the classification model M is an MLP. The table implies that WGANXtraY is better than other schemes most of the time when clean data percent available is only 1%. This demonstrates that good GANs (columns 3,4 and 5) for generating samples need not be good for generating correctly labelled data points from noisy labelled data points.  $\rho_{+},\rho_{-}$ | WGANXtraY | WGANXtraY-GP | WGANXtraY-R | PacGAN(3)
---|---|---|---|---
0.4,0.49 | 89.23$\pm$ 1.29 |  89.16 $\pm$ 1.10 |  87.52 $\pm$ 3.76 |  49.30 $\pm$ 2.24
0.42,0.45 | 89.67 $\pm$ 0.75 |  87.45 $\pm$ 3.25 |  89.16 $\pm$ 0.57 |  50.37$\pm$ 1.27
0.45,0.46 | 89.36 $\pm$ 0.49 |  86.01 $\pm$ 6.00 |  78.59 $\pm$ 11.83 |  48.72 $\pm$ 4.04
0.45,0.48 |  89.92 $\pm$ 0.29 | 90.17 $\pm$ 0.42 |  85.5 $\pm$ 4.04 |  53.96 $\pm$ 5.11
0.45,0.5 |  88.99 $\pm$ 1.43 | 89.39 $\pm$ 0.84 |  86.60$\pm$ 2.50 |  52.02$\pm$ 5.17
0.46,0.48 |  87.31 $\pm$ 2.60 | 87.92 $\pm$ 2.72 |  79.97 $\pm$ 15.19 |  39.07 $\pm$ 21.75
0.46,0.5 |  88.94 $\pm$ 0.63 |  88.81 $\pm$ 0.95 | 89.21 $\pm$ 0.74 |  30.09$\pm$ 24.57
0.47,0.49 |  87.76 $\pm$ 4.75 | 89.53 $\pm$ 1.53 |  88.19 $\pm$ 3.189 |  50.21 $\pm$ 0.59
0.47,0.51 |  89.22 $\pm$ 0.81 | 89.57 $\pm$ 0.82 |  78.80 $\pm$ 15.21 |  39.15 $\pm$ 19.84
0.48,0.49 |  86.43 $\pm$ 3.84 | 89.54 $\pm$ 0.75 |  78.03 $\pm$ 14.61 |  41.65 $\pm$ 21.11
0.5,0.5 |  86.99$\pm$ 3.69 | 90.39 $\pm$ 0.93 |  89.24 $\pm$ 0.48 |  47.41 $\pm$ 4.05
Table 12: Averaged accuracy and standard deviation values for Fashion MNIST 7-9 (Sneakers and boots) dataset across 5 trials when the classification model M is an MLP. The table demonstrates that WGANXtraY is comparable to WGANXtraY-GP (but with more iterations) most of the time when clean data percent available is only 1%. This implies that good GANs (columns 3,4 and 5) for generating samples need not be good for generating correctly labelled data points from noisy labelled data points.  $\rho_{+},\rho_{-}$ | WGANXtraY | WGANXtraY-GP | WGANXtraY-R | PacGAN(3)
---|---|---|---|---
0.4,0.49 |  93.22 $\pm$ 3.60 | 94.91 $\pm$ 2.94 |  90.5 $\pm$ 3.24 |  30.08 $\pm$ 24.56
0.42,0.45 |  85.01 $\pm$ 17.72 |  92.04 $\pm$ 4.61 | 92.41 $\pm$ 3.50 |  31.76 $\pm$ 32.26
0.45,0.46 |  92.53 $\pm$ 3.45 | 94.98 $\pm$ 1.21 |  90.58 $\pm$ 2.35 |  40.13 $\pm$ 20.06
0.45,0.48 |  87.49 $\pm$ 14.81 | 93.57 $\pm$ 3.93 |  93.03 $\pm$ 0.96 |  50.0 $\pm$ 0.0
0.45,0.5 | 95.49 $\pm$ 1.08 |  91.56 $\pm$ 4.02 |  91.91$\pm$ 5.63 |  34.37 $\pm$ 29.19
0.46,0.48 |  95.7 $\pm$ 1.23 | 96.36 $\pm$ 0.30 |  89.56 $\pm$ 5.75 |  40.0 $\pm$ 20.0
0.46,0.5 |  81.00$\pm$ 19.00 | 95.57 $\pm$ 0.81 |  87.55 $\pm$ 7.37 |  49.98 $\pm$ 0.04
0.47,0.49 |  95.1 $\pm$ 1.47 | 95.67 $\pm$ 0.79 |  94.49 $\pm$ 1.61 |  50.0 $\pm$ 0.0
0.47,0.51 |  91.67 $\pm$ 7.29 | 96.28 $\pm$ 0.64 |  91.52 $\pm$ 7.34 |  42.04 $\pm$ 12.20
0.48,0.49 |  93.62 $\pm$ 2.65 | 93.74 $\pm$ 3.17 |  93.08 $\pm$ 3.83 |  40.0 $\pm$ 20.0
0.5,0.5 |  94.20 $\pm$ 2.46 |  91.64 $\pm$ 9.72 | 94.22 $\pm$ 2.11 |  40.3 $\pm$ 20.15
Table 13: Averaged accuracy and standard deviation values for Fashion MNIST
2-3 (Pullover and Dress) dataset across 5 trials when the classification model
M is an MLP. The table demonstrates that WGANXtraY is comparable to WGANXtraY-
GP (but with more iterations) most of the time when clean data percent
available is only 1%. This implies that good GANs (columns 3,4 and 5) for
generating samples need not be good for generating correctly labelled data
points from noisy labelled data points.

###  B.5 Results for low and moderate CCN rates

In this section, we demonstrate the performance of WGANXtraY(5) in comparison
to GLC at low and moderate noise rates $\rho_{+},\rho_{-}$. We perform the
experiments when the gold fraction available is 0.1% and 1% for both schemes
on 6 binary balanced version of MNIST datasets. For both cases of gold
fraction, number of iterations $n_{it}=1000$. The results are presented in
Table 14, 15 and 16. For MNIST 0-1 dataset and MNIST 1-7 dataset, our scheme
has either comparable or within $5\%$ accuracies of GLC in all cases of noise
rates. For other datasets, not satisfactory performance could be due to small
KL divergence between $\mathcal{D}$ and $\tilde{\mathcal{D}}$ and inturn low
adversarial relation as shown in Lemma 1 and 2.

($\rho_{+},\rho_{-}$) | WGANXtraY | GLC | WGANXtraY | GLC
---|---|---|---|---
Clean % | 0.1 % | 0.1% | 1% | 1%
Dataset | MNIST 5-6
0.01, 0.02 |  88.49$\pm$ 1.93 | 89.61 $\pm$ 18.91 |  93.87$\pm$1.93 | 98.99 $\pm$ 0.24
0.3,0.3 |  82.69 $\pm$ 5.15 | 97.10 $\pm$ 1.22 |  94.87$\pm$0.35 | 97.58 $\pm$ 0.30
0.2,0.49 |  83.68 $\pm$ 7.95 | 97.29 $\pm$ 0.28 |  95.25$\pm$0.77 | 96.74 $\pm$ 1.06
0.42,0.35 |  81.65 $\pm$ 4.22 | 94.83$\pm$ 0.64 |  95.31$\pm$0.32 | 95.75 $\pm$ 2.32
0.1,0.45 |  85.25 $\pm$ 7.33 | 97.71$\pm$ 1.01 |  93.81$\pm$1.18 | 98.05$\pm$ 0.29
Dataset | MNIST 7-9
0.01, 0.02 |  81.07 $\pm$ 3.12 | 98.95$\pm$ 0.37 |  90.42$\pm$1.63 | 99.13 $\pm$ 0.23
0.3,0.3 |  79.27 $\pm$ 2.14 | 95.82 $\pm$ 0.78 |  88.25$\pm$2.50 | 96.63 $\pm$ 1.11
0.2,0.49 |  79.45 $\pm$ 3.43 | 95.85 $\pm$ 0.96 |  90.03$\pm$1.38 | 96.80$\pm$ 0.94
0.42,0.35 |  82.54 $\pm$ 1.20 | 95.42 $\pm$ 0.72 |  89.10$\pm$4.99 | 96.02 $\pm$ 0.84
0.1,0.45 |  80.13$\pm$ 1.95 | 97.89 $\pm$ 0.21 |  89.44$\pm$1.44 | 97.80 $\pm$ 0.25
Table 14: Averaged accuracy and standard deviation values for low and moderate noise corrupted MNIST 5-6 and MNIST 7-9 datasets across 5 trials when the classification model M is an MLP. ($\rho_{+},\rho_{-}$) | WGANXtraY | GLC | WGANXtraY | GLC
---|---|---|---|---
Clean % | 0.1 % | 0.1% | 1% | 1%
Dataset | MNIST 4-9
0.01, 0.02 |  73.05 $\pm$ 1.44 | 98.55 $\pm$ 0.18 |  90.72$\pm$1.59 | 98.65 $\pm$ 0.28
0.3,0.3 |  74.33 $\pm$ 0.80 | 95.59 $\pm$ 1.61 |  90.39$\pm$1.83 | 96.68 $\pm$ 0.36
0.2,0.49 |  70.23 $\pm$ 4.75 | 95.90 $\pm$ 0.46 |  81.17$\pm$15.35 | 94.87 $\pm$ 1.40
0.42,0.35 |  77.0 $\pm$ 3.19 | 93.30 $\pm$ 1.93 |  90.03$\pm$1.05 | 92.05 $\pm$ 3.31
0.1,0.45 |  76.45 $\pm$ 3.45 | 96.63 $\pm$ 0.28 |  91.47$\pm$1.44 | 97.18 $\pm$ 0.97
Dataset | MNIST 1-7
0.01, 0.02 |  95.34 $\pm$ 1.33 | 99.54$\pm$ 0.08 |  95.62$\pm$1.61 | 99.60 $\pm$ 0.14
0.3,0.3 |  95.58 $\pm$ 1.90 | 98.90 $\pm$ 0.14 |  95.88$\pm$2.44 | 98.70 $\pm$ 0.60
0.2,0.49 |  94.49$\pm$ 1.56 | 98.89$\pm$ 0.26 |  96.64$\pm$0.88 | 98.86 $\pm$ 0.40
0.42,0.35 |  96.03 $\pm$ 0.91 | 97.11 $\pm$ 0.97 |  95.61$\pm$1.13 | 97.88 $\pm$ 1.29
0.1,0.45 |  95.32$\pm$ 2.78 | 98.90 $\pm$ 0.20 |  96.68$\pm$0.93 | 98.89$\pm$ 0.67
Table 15: Averaged accuracy and standard deviation values for low and moderate noise corrupted MNIST 4-9 and MNIST 1-7 datasets across 5 trials when the classification model M is an MLP. ($\rho_{+},\rho_{-}$) | WGANXtraY | GLC | WGANXtraY | GLC
---|---|---|---|---
Clean % | 0.1 % | 0.1% | 1% | 1%
Dataset | MNIST 0-6
0.01, 0.02 |  88.73$\pm$ 3.77 | 99.45 $\pm$ 0.14 |  96.17 $\pm$ 0.56 | 99.46 $\pm$ 0.05
0.3,0.3 |  88.99$\pm$ 3.37 | 98.21$\pm$ 0.46 |  86.25 $\pm$ 17.8 | 98.26 $\pm$ 0.73
0.2,0.49 |  88.15 $\pm$ 2.43 | 98.53 $\pm$ 0.29 |  95.34 $\pm$ 0.76 | 98.15 $\pm$ 0.79
0.42,0.35 |  91.91 $\pm$ 2.58 | 95.95 $\pm$ 1.87 |  94.0 $\pm$ 2.66 | 97.10 $\pm$ 0.84
0.1,0.45 |  90.68$\pm$ 2.07 | 98.76 $\pm$ 0.65 |  95.24 $\pm$ 1.04 | 98.68 $\pm$ 0.28
Dataset | MNIST 0-1
0.01, 0.02 |  99.46 $\pm$ 0.28 | 99.93 $\pm$ 0.03 |  99.38 $\pm$ 0.55 | 99.88 $\pm$ 0.07
0.3,0.3 |  97.70 $\pm$ 2.10 | 99.49$\pm$ 0.67 |  99.13$\pm$ 1.17 | 99.75$\pm$ 0.16
0.2,0.49 | 99.23 $\pm$ 0.78 |  98.47 $\pm$ 2.78 |  99.56 $\pm$ 0.21 | 99.70 $\pm$ 0.21
0.42,0.35 | 99.42$\pm$ 0.34 |  95.20 $\pm$ 8.11 |  99.28 $\pm$ 0.52 | 99.62 $\pm$ 0.23
0.1,0.45 |  99.19 $\pm$ 0.55 | 99.71$\pm$ 0.19 |  98.99 $\pm$ 1.23 | 99.82 $\pm$ 0.12
Table 16: Averaged accuracy and standard deviation values for low and moderate
noise corrupted MNIST 0-6 and MNIST 0-1 datasets across 5 trials when the
classification model M is an MLP.

##  Appendix C Details for the plots presented in Section 3

In this section, we present a detailed comparison of our scheme and GLC on
both synthetic and real datasets. We provide the average accuracy and standard
deviation values obtained from 5 random trials. All results presented in this
section are obtained by using $0.1\%$ clean labelled data except for
imbalanced MNIST 4-9 datasets for which clean percentage used is $1\%$. Bold
faced values denotes the highest accuracy across the row.

Synthetic data results are reported in Table 17 and 18. WGANXtraY(5)
outperforms the others methods for both the datasets.

Next, we present the average accuracies along with the standard deviation
values for binary balanced class MNIST datasets in Table 19, 20, 21, 22, 23
and 24. In almost all cases for noise rates close to $0.5$, either
WGANXtraY(5) or WGANXtraYEntr(5) has higher accuracy value. We also present
the results of swapped noise rates in Table 25, 26, 27, 28, 29 and 30. These
results also demonstrate better performance of our schemes at high noise
rates.

In Table 31, 32, 33 and 34, we present the average accuracies and the standard
deviation values for binary balanced class Fashion MNIST datasets. The results
for the same datasets when the noise rates are swapped are provided in Table
35, 36, 37 and 38. One point to note here is that even though GLC has
comparable performance in some cases, the standard deviation is very high for
almost all cases.

Finally, in Table 39, 40, 41, 42, 43 and 44, we present the average AM and
standard deviation values for binary imbalanced MNIST datasets for which plots
are presented in Figure 4 of Section 3. Number of iterations are taken to be
$1000$. Except for the case of MNIST 4-9 (imb_r = 0.75), our schemes with low
variations outperform GLC, GCE and LDMI. For MNIST 4-9 (imb_r = 0.75) dataset,
out of the 3 cases where GLC has higher accuracy, for 2 cases our schemes are
within 2 percent of GLC.

$\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  73.81 $\pm$ 7.47 | 96.98 $\pm$ 0.35 |  82.69 $\pm$ 17.89 |  64.16 $\pm$ 14.86 |  83.91 $\pm$ 8.31 |  84.42 $\pm$ 12.61
0.42,0.45 |  71.89 $\pm$ 5.61 | 97.57 $\pm$ 0.70 |  73.65 $\pm$ 20.97 |  46.74$\pm$ 19.86 |  95.65 $\pm$ 3.69 |  89.92 $\pm$ 8.04
0.45,0.46 |  63.75$\pm$ 5.57 | 97.01 $\pm$ 0.98 |  63.46 $\pm$ 11.37 |  46.16 $\pm$ 12.92 |  87.76 $\pm$ 10.50 |  85.03 $\pm$ 10.41
0.45,0.48 |  64.53 $\pm$ 9.92 | 95.06 $\pm$ 4.96 |  76.37 $\pm$ 22.41 |  54.85 $\pm$ 8.23 |  77.38 $\pm$ 10.95 |  73.25 $\pm$ 12.71
0.45,0.5 |  60.05 $\pm$ 12.94 | 96.21 $\pm$ 3.0 |  75.78 $\pm$ 22.16 |  51.12 $\pm$ 13.51 |  71.03 $\pm$ 11.41 |  65.73 $\pm$ 8.06
0.46,0.48 |  64.08 $\pm$ 5.78 | 96.61 $\pm$ 1.49 |  87.78 $\pm$ 9.64 |  57.2 $\pm$ 9.58 |  83.86 $\pm$ 7.16 |  63.86 $\pm$ 6.44
0.46,0.5 |  51.83 $\pm$ 10.91 | 96.13 $\pm$ 0.83 |  90.16 $\pm$ 12.12 |  52.88 $\pm$ 9.27 |  68.16 $\pm$ 10.46 |  61.04 $\pm$ 19.39
0.47,0.49 |  60.0 $\pm$ 12.68 | 94.64 $\pm$ 5.28 |  75.67 $\pm$ 18.66 |  53.92 $\pm$ 5.84 |  80.0 $\pm$ 9.14 |  62.74 $\pm$ 9.72
0.47,0.51 |  53.62 $\pm$ 3.39 | 97.33 $\pm$ 0.71 |  90.74 $\pm$ 9.58 |  50.02 $\pm$ 5.80 |  58.96 $\pm$ 4.99 |  50.93$\pm$ 3.07
0.48,0.49 |  62.69 $\pm$ 7.48 | 97.54 $\pm$ 1.32 |  84.16 $\pm$ 15.88 |  50.05 $\pm$ 3.35 |  76.08 $\pm$ 10.80 |  61.27 $\pm$ 11.10
0.5,0.5 |  50.72 $\pm$ 10.71 | 95.65 $\pm$ 1.87 |  82.50$\pm$ 18.00 |  51.52 $\pm$ 4.47 |  53.68 $\pm$ 7.09 |  48.74 $\pm$ 3.35
Table 17: Averaged accuracy and standard deviation values for Synthetic dataset SD100 across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGANXtraY shows a significant improvement (third column) over GLC, GCE and LDMI even when clean data percent available is only 0.1%.  $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  73.11 $\pm$ 6.36 | 99.92 $\pm$ 0.10 |  84.4 $\pm$ 18.12 |  59.49 $\pm$ 11.56 |  85.17 $\pm$ 3.07 |  83.89 $\pm$ 13.20
0.42,0.45 |  72.05 $\pm$ 8.63 | 99.81 $\pm$ 0.19 |  78.66 $\pm$ 15.97 |  58.8 $\pm$ 10.52 |  96.10 $\pm$ 4.62 |  79.49 $\pm$ 11.73
0.45,0.46 |  60.4 $\pm$ 9.47 | 99.94 $\pm$ 0.10 |  64.26 $\pm$ 14.30 |  58.56 $\pm$ 10.22 |  87.86 $\pm$ 9.40 |  75.92 $\pm$ 17.99
0.45,0.48 |  66.69 $\pm$ 11.73 | 99.89 $\pm$ 0.05 |  85.81 $\pm$ 18.56 |  54.58 $\pm$ 9.94 |  90.13 $\pm$ 3.80 |  74.26 $\pm$ 18.60
0.45,0.5 |  56.26 $\pm$ 12.38 | 99.30 $\pm$ 0.98 |  84.45 $\pm$ 17.85 |  52.56 $\pm$ 2.32 |  70.58 $\pm$ 9.55 |  61.54 $\pm$ 9.79
0.46,0.48 |  68.58 $\pm$ 7.22 | 99.60 $\pm$ 0.50 |  92.85 $\pm$ 7.33 |  53.89 $\pm$ 9.28 |  84.98 $\pm$ 3.05 |  57.25 $\pm$ 5.40
0.46,0.5 |  55.62 $\pm$ 8.64 | 99.92 $\pm$ 0.10 |  78.10 $\pm$ 20.71 |  47.46 $\pm$ 3.55 |  78.98 $\pm$ 9.27 |  54.0 $\pm$ 4.14
0.47,0.49 |  55.09 $\pm$ 9.30 | 100.0 $\pm$ 0.0 |  86.53 $\pm$ 9.85 |  50.77 $\pm$ 3.34 |  75.70 $\pm$ 9.87 |  57.2 $\pm$ 5.39
0.47,0.51 |  53.01 $\pm$ 8.63 | 99.94 $\pm$ 0.10 |  68.29 $\pm$ 20.32 |  49.89 $\pm$ 11.21 |  60.32 $\pm$ 9.73 |  56.15 $\pm$ 3.48
0.48,0.49 |  63.70 $\pm$ 10.13 | 99.89 $\pm$ 0.15 |  85.46 $\pm$ 18.89 |  50.8 $\pm$ 4.46 |  73.35 $\pm$ 10.31 |  55.86 $\pm$ 4.75
0.5,0.5 |  50.45 $\pm$ 12.62 | 99.57 $\pm$ 0.72 |  94.05 $\pm$ 5.78 |  58.98 $\pm$ 8.71 |  56.4 $\pm$ 13.94 |  47.28 $\pm$ 4.46
Table 18: Averaged accuracy and standard deviation values for Synthetic dataset SD300 across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGANXtraY shows a significant improvement (third column) over GLC, GCE and LDMI even when clean data percent available is only 0.1%. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  49.28$\pm$ 2.14 |  78.8 $\pm$ 4.8 | 88.71 $\pm$ 3.23 |  84.35 $\pm$ 10.29 |  48.21 $\pm$ 0.0 |  83.47 $\pm$ 9.92
0.42,0.45 |  65.81 $\pm$ 16.3 |  86.88$\pm$ 5.76 |  87.21 $\pm$ 3.29 | 88.18 $\pm$ 4.94 |  48.56 $\pm$ 0.69 |  43.67 $\pm$ 34.29
0.45,0.46 |  85.98 $\pm$ 6.75 |  83.08 $\pm$ 3.69 |  80.46 $\pm$ 10.73 |  86.52 $\pm$ 14.70 |  67.72 $\pm$ 16.04 | 92.63 $\pm$ 1.81
0.45,0.48 |  74.30 $\pm$ 8.89 |  84.5 $\pm$ 9.63 | 89.47 $\pm$ 3.46 |  75.85 $\pm$ 16.8 |  51.31 $\pm$ 6.20 |  76.61 $\pm$ 10.46
0.45,0.5 |  48.22 $\pm$ 0.02 | 91.2 $\pm$ 1.81 |  79.62 $\pm$ 6.70 |  64.6 $\pm$ 24.7 |  48.21 $\pm$ 0.0 |  69.70 $\pm$ 14.31
0.46,0.48 |  63.36 $\pm$ 18.2 | 88.3 $\pm$ 6.06 |  79.81 $\pm$ 4.46 |  72.30 $\pm$ 14.99 |  56.46 $\pm$ 12.60 |  69.25 $\pm$ 13.69
0.46,0.5 |  53.51 $\pm$ 10.6 |  83.8 $\pm$ 7.10 | 84.98 $\pm$ 7.93 |  79.0 $\pm$ 12.48 |  48.21 $\pm$ 0.0 |  75.09 $\pm$ 10.60
0.47,0.49 |  48.3 $\pm$ 0.22 | 86.1 $\pm$ 6.24 |  86.09 $\pm$ 2.02 |  71.01 $\pm$ 14.67 |  48.21 $\pm$ 0.0 |  64.84 $\pm$ 8.56
0.47,0.51 |  48.21 $\pm$ 0.0 |  86.1 $\pm$ 2.53 | 87.97 $\pm$ 3.56 |  72.45 $\pm$ 11.54 |  48.21 $\pm$ 0.0 |  57.72 $\pm$ 9.38
0.48,0.49 |  53.69 $\pm$ 4.66 |  82.7 $\pm$ 8.47 | 83.28 $\pm$ 4.58 |  62.51 $\pm$ 9.93 |  50.05 $\pm$ 2.53 |  64.95 $\pm$ 10.07
0.5,0.5 |  50.19 $\pm$ 2.69 |  79.77 $\pm$ 2.73 | 80.50 $\pm$ 5.78 |  62.37$\pm$ 7.30 |  51.37 $\pm$ 1.68 |  58.62 $\pm$ 13.50
Table 19: Averaged accuracy and standard deviation values for MNIST 5-6 dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC, GCE and LDMI. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  50.46$\pm$ 0.0 |  78.8$\pm$ 1.33 |  78.76 $\pm$ 3.18 |  86.86 $\pm$ 7.70 |  50.46 $\pm$ 0.0 | 86.42 $\pm$ 5.62
0.42,0.45 |  67.99 $\pm$ 11.96 | 75.9$\pm$ 3.87 |  82.43 $\pm$ 1.29 |  86.04 $\pm$ 7.14 |  50.46 $\pm$ 0.0 | 92.75 $\pm$ 0.57
0.45,0.46 |  75.79 $\pm$ 11.96 |  81.49$\pm$ 1.2 |  80.10 $\pm$ 2.85 |  82.8 $\pm$ 7.5 |  51.98 $\pm$ 2.19 | 85.73 $\pm$ 3.42
0.45,0.48 |  50.6 $\pm$ 0.451 |  81.1$\pm$ 2.56 |  81.99 $\pm$ 1.58 | 84.2 $\pm$ 2.78 |  50.46 $\pm$ 0.0 |  78.03 $\pm$ 7.19
0.45,0.5 |  50.46 $\pm$ 0.0 |  78.0$\pm$ 4.2 | 80.81 $\pm$ 2.36 |  67.5 $\pm$ 17.68 |  50.46 $\pm$ 0.0 |  68.19 $\pm$ 7.0
0.46,0.48 |  58.91 $\pm$ 11.82 | 80.79 $\pm$ 2.37 |  80.32 $\pm$ 2.44 |  72.4 $\pm$ 8.9 |  50.46 $\pm$ 0.0 |  76.76 $\pm$ 15.33
0.46,0.5 |  50.46 $\pm$ 0.0 | 80.3 $\pm$ 1.88 |  80.01 $\pm$ 1.09 |  76.94 $\pm$ 2.79 |  50.46 $\pm$ 0.0 |  63.49 $\pm$ 10.09
0.47,0.49 |  56.85$\pm$ 11.02 |  76.9$\pm$ 4.54 | 77.83 $\pm$ 1.50 |  78.88$\pm$ 4.66 |  50.47 $\pm$ 0.01 |  71.08 $\pm$ 13.62
0.47,0.51 |  50.496 $\pm$ 0.058 |  78.9$\pm$ 1.70 | 78.96 $\pm$ 3.65 |  70.10 $\pm$ 7.63 |  50.46 $\pm$ 0.0 |  59.31$\pm$ 11.4
0.48,0.49 |  62.4 $\pm$ 7.42 |  80.7 $\pm$ 2.07 | 81.81$\pm$ 1.19 |  56.77 $\pm$ 14.93 |  50.44 $\pm$ 0.03 |  59.70 $\pm$ 9.3
0.5,0.5 |  41.9 $\pm$ 6.39 | 79.2$\pm$ 1.51 |  79.09 $\pm$ 4.17 |  61.5 $\pm$ 7.90 |  49.88 $\pm$ 0.78 |  57.00$\pm$ 9.56
Table 20: Averaged accuracy and standard deviation values for MNIST 7-9 dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have high or comparable accuracies and low variation among the trials than GLC, GCE and LDMI. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  49.32$\pm$ 0.0 |  71.47 $\pm$ 4.22 |  75.07 $\pm$ 4.43 |  83.92$\pm$ 4.60 |  49.32 $\pm$ 0.0 | 86.82$\pm$ 2.08
0.42,0.45 |  54.32 $\pm$ 8.98 |  72.2 $\pm$ 4.81 |  74.56 $\pm$ 1.74 | 88.10 $\pm$ 4.16 |  52.25 $\pm$ 5.86 |  81.98$\pm$ 6.48
0.45,0.46 |  53.67 $\pm$ 5.06 |  72.22 $\pm$ 3.4 |  76.30 $\pm$ 1.59 |  79.4 $\pm$ 11.41 |  49.55 $\pm$ 0.46 | 84.88 $\pm$ 2.73
0.45,0.48 |  49.35 $\pm$ 0.06 |  73.93 $\pm$ 4.26 |  77.62 $\pm$ 3.47 |  69.5 $\pm$ 10.31 |  49.32 $\pm$ 0.0 | 78.52$\pm$ 7.18
0.45,0.5 |  49.35 $\pm$ 0.06 |  74.15 $\pm$ 2.27 | 74.89 $\pm$ 1.55 |  60.0 $\pm$ 11.66 |  49.32 $\pm$ 0.0 |  65.18$\pm$ 4.24
0.46,0.48 |  55.1 $\pm$ 10.73 | 75.63 $\pm$ 1.42 |  72.74 $\pm$ 3.07 |  66.8 $\pm$ 16.03 |  50.72 $\pm$ 2.81 |  72.28$\pm$ 13.43
0.46,0.5 |  49.32$\pm$ 0.0 |  73.42 $\pm$ 1.89 | 76.60 $\pm$ 2.77 |  65.89 $\pm$ 11.42 |  49.32 $\pm$ 0.0 |  46.55$\pm$ 11.99
0.47,0.49 |  60.73 $\pm$ 13.2 |  73.52 $\pm$ 3.92 | 74.91 $\pm$ 3.20 |  60.1 $\pm$ 6.88 |  50.81 $\pm$ 2.99 |  60.43$\pm$ 15.92
0.47,0.51 |  49.7$\pm$ 0.94 |  75.8 $\pm$ 2.75 | 75.90 $\pm$ 2.32 |  73.0 $\pm$ 6.77 |  49.32 $\pm$ 0.0 |  56.07 $\pm$ 6.84
0.48,0.49 |  52.4$\pm$ 2.32 |  73.69 $\pm$ 2.69 | 78.32$\pm$ 2.08 |  57.06 $\pm$ 7.17 |  48.39 $\pm$ 1.84 |  60.34$\pm$ 3.87
0.5,0.5 |  52.4$\pm$ 4.26 |  70.46 $\pm$ 2.80 | 76.82$\pm$ 2.90 |  55.82 $\pm$ 7.49 |  49.37 $\pm$ 1.39 |  46.74$\pm$ 8.34
Table 21: Averaged accuracy and standard deviation values for MNIST 4-9 dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC, GCE and LDMI in most cases. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  53.08 $\pm$ 1.22 |  94.16 $\pm$ 2.0 | 95.72 $\pm$ 0.50 |  94.5 $\pm$ 2.22 |  52.47 $\pm$ 0.0 |  87.78 $\pm$ 5.18
0.42,0.45 |  85.04 $\pm$ 7.74 |  94.44 $\pm$ 1.94 | 95.36 $\pm$ 1.39 |  93.06 $\pm$ 5.66 |  53.25 $\pm$ 1.57 |  94.70 $\pm$ 3.14
0.45,0.46 |  87.54 $\pm$ 11.78 | 96.0 $\pm$ 1.20 |  94.66 $\pm$ 1.84 |  95.44 $\pm$ 2.6 |  64.95 $\pm$ 15.04 |  90.22 $\pm$ 8.04
0.45,0.48 |  74.49 $\pm$ 17.09 |  92.78 $\pm$ 3.91 | 95.92 $\pm$ 1.40 |  90.70 $\pm$ 9.84 |  52.47 $\pm$ 0.0 |  81.01 $\pm$ 12.38
0.45,0.5 |  52.9 $\pm$ 0.64 |  92.7 $\pm$ 3.9 | 94.08 $\pm$ 2.24 |  80.87 $\pm$ 14.9 |  52.47 $\pm$ 0.0 |  79.01 $\pm$ 15.39
0.46,0.48 |  60.36 $\pm$ 15.7 | 94.9 $\pm$ 0.798 |  89.52 $\pm$ 11.86 |  77.17 $\pm$ 28.45 |  52.47 $\pm$ 0.0 |  84.58 $\pm$ 10.62
0.46,0.5 |  52.475 $\pm$ 0.0 | 95.3 $\pm$ 2.41 |  94.87 $\pm$ 1.34 |  70.3 $\pm$ 12.04 |  52.47 $\pm$ 0.0 |  81.11 $\pm$ 10.67
0.47,0.49 |  65.89 $\pm$ 15.2 |  94.7 $\pm$ 1.15 | 94.80 $\pm$ 2.07 |  73.38 $\pm$ 19.57 |  50.49 $\pm$ 2.42 |  82.35 $\pm$ 16.16
0.47,0.51 |  52.47 $\pm$ 0.0 |  94.4 $\pm$ 1.77 | 94.83 $\pm$ 2.20 |  79.32 $\pm$ 13.71 |  52.47 $\pm$ 0.0 |  77.48 $\pm$ 20.32
0.48,0.49 |  56.9 $\pm$ 10.18 |  94.7 $\pm$ 1.19 | 95.49 $\pm$ 1.57 |  83.94 $\pm$ 12.04 |  57.13 $\pm$ 9.32 |  65.05 $\pm$ 13.86
0.5,0.5 |  44.45$\pm$ 11.4 |  95.0$\pm$ 0.93 | 95.22 $\pm$ 2.35 |  67.94 $\pm$ 6.33 |  51.78 $\pm$ 4.20 |  55.34 $\pm$ 15.52
Table 22: Averaged accuracy and standard deviation values for MNIST 1-7 dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC, GCE and LDMI. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  50.85 $\pm$ 1.53 |  90.78 $\pm$ 1.79 |  91.71 $\pm$ 2.64 |  93.44 $\pm$ 3.09 |  49.43 $\pm$ 0.0 | 94.47$\pm$ 3.11
0.42,0.45 |  80.69 $\pm$ 15.58 |  91.05 $\pm$ 1.83 |  91.49 $\pm$ 2.91 | 94.95 $\pm$ 1.73 |  55.32 $\pm$ 11.78 |  91.43 $\pm$ 5.74
0.45,0.46 |  83.43$\pm$ 8.76 |  88.85 $\pm$ 2.47 |  89.95 $\pm$ 2.45 |  91.5 $\pm$ 3.58 |  73.12 $\pm$ 13.73 | 93.25 $\pm$ 3.00
0.45,0.48 |  61.63 $\pm$ 10.55 |  87.89 $\pm$ 3.08 |  89.98 $\pm$ 2.90 |  85.46$\pm$ 18.07 |  55.89 $\pm$ 12.21 | 90.48 $\pm$ 6.91
0.45,0.5 |  49.46 $\pm$ 0.06 | 90.85 $\pm$ 1.57 |  87.17 $\pm$ 5.09 |  77.80 $\pm$ 17.29 |  49.43 $\pm$ 0.0 |  62.09 $\pm$ 14.84
0.46,0.48 |  60.44 $\pm$ 12.34 |  85.30 $\pm$ 4.19 | 92.49 $\pm$ 1.54 |  80.01 $\pm$ 15.33 |  49.66 $\pm$ 0.47 |  83.90 $\pm$ 9.54
0.46,0.5 |  50.90 $\pm$ 2.49 |  87.9 $\pm$ 2.89 | 92.04 $\pm$ 0.77 |  71.45 $\pm$ 9.14 |  49.43 $\pm$ 0.0 |  68.29$\pm$ 16.68
0.47,0.49 |  66.25$\pm$ 15.07 | 87.83 $\pm$ 2.12 |  85.26 $\pm$ 7.76 |  72.6 $\pm$ 18.53 |  49.43 $\pm$ 0.0 |  82.96 $\pm$ 14.51
0.47,0.51 |  49.43 $\pm$ 0.0 |  87.97 $\pm$ 1.08 | 90.80 $\pm$ 2.19 |  80.06 $\pm$ 10.2 |  49.43 $\pm$ 0.0 |  66.02$\pm$ 18.37
0.48,0.49 |  69.49 $\pm$ 12.48 | 91.49 $\pm$ 1.32 |  89.52 $\pm$ 3.34 |  76.06 $\pm$ 17.8 |  56.95 $\pm$ 6.02 |  59.87 $\pm$ 16.22
0.5,0.5 |  48.29 $\pm$ 1.98 |  86.4 $\pm$ 1.23 | 88.31 $\pm$ 3.51 |  81.1 $\pm$ 7.62 |  49.18 $\pm$ 1.70 |  50.88$\pm$ 4.37
Table 23: Averaged accuracy and standard deviation values for MNIST 0-6 dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC, GCE and LDMI. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  60.84 $\pm$ 9.05 | 99.38 $\pm$ 0.39 |  98.52 $\pm$ 1.82 |  98.70 $\pm$ 1.34 |  53.66 $\pm$ 0.0 |  89.98$\pm$ 11.62
0.42,0.45 |  95.4 $\pm$ 2.87 |  98.83 $\pm$ 0.85 |  97.80 $\pm$ 3.16 |  90.4 $\pm$ 16.93 |  58.43 $\pm$ 5.99 | 99.60$\pm$ 0.55
0.45,0.46 |  96.46 $\pm$ 1.60 | 99.24 $\pm$ 0.61 |  98.91 $\pm$ 0.74 |  94.4 $\pm$ 8.48 |  67.02 $\pm$ 15.52 |  92.42 $\pm$ 8.94
0.45,0.48 |  66.50 $\pm$ 14.5 |  98.8 $\pm$ 0.81 | 99.28 $\pm$ 0.29 |  91.13 $\pm$ 9.6 |  53.66 $\pm$ 0.0 |  92.42$\pm$ 7.58
0.45,0.5 |  54.45 $\pm$ 1.09 | 98.59 $\pm$ 1.14 |  98.22 $\pm$ 2.45 |  85.06 $\pm$ 15.83 |  53.63 $\pm$ 0.0 |  81.56$\pm$ 18.26
0.46,0.48 |  66.14 $\pm$ 13.01 |  98.94 $\pm$ 0.27 | 99.43 $\pm$ 0.32 |  95.79 $\pm$ 3.89 |  55.40 $\pm$ 3.0 |  96.28$\pm$ 6.35
0.46,0.5 |  59.9 $\pm$ 8.68 | 98.42 $\pm$ 1.7 |  97.95 $\pm$ 1.44 |  96.3$\pm$ 4.49 |  53.66 $\pm$ 0.0 |  94.51$\pm$ 8.98
0.47,0.49 |  82.2 $\pm$ 12.77 | 98.73 $\pm$ 1.35 |  99.58$\pm$ 0.15 |  93.76$\pm$ 5.43 |  53.66 $\pm$ 0.0 |  89.50$\pm$ 9.02
0.47,0.51 |  53.66 $\pm$ 7.10 | 98.43 $\pm$ 1.41 |  98.08$\pm$ 2.29 |  78.2 $\pm$ 15.93 |  53.66 $\pm$ 0.0 |  75.05$\pm$ 15.96
0.48,0.49 |  76.7 $\pm$ 20.01 | 99.29 $\pm$ 0.43 |  98.15 $\pm$ 2.07 |  89.82 $\pm$ 6.26 |  58.00 $\pm$ 10.56 |  95.00 $\pm$ 4.01
0.5,0.5 |  41.21$\pm$ 11.5 |  98.63 $\pm$ 1.26 | 99.61 $\pm$ 0.20 |  79.68 $\pm$ 13.28 |  51.14 $\pm$ 6.62 |  69.64 $\pm$ 24.33
Table 24: Averaged accuracy and standard deviation values for MNIST 0-1 dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC, GCE and LDMI in most cases. $\rho_{+},\rho_{-}$ | SimpleNN | WGANXtraY | WGANXtraYEntr | GLC
---|---|---|---|---
0.49,0.4 |  51.78 $\pm$ 0.0 |  83.27 $\pm$ 4.70 |  88.51 $\pm$ 2.35 | 92.30 $\pm$ 3.78
0.45,0.42 |  63.1$\pm$ 10.34 |  81.41$\pm$ 6.27 |  82.12 $\pm$ 3.46 | 91.22 $\pm$ 4.87
0.46,0.45 |  71.2 $\pm$ 12.37 |  82.88 $\pm$ 6.83 | 85.44$\pm$ 6.57 |  80.86 $\pm$ 13.14
0.48,0.45 |  57.3 $\pm$ 10.13 |  86.1$\pm$ 3.21 | 87.84$\pm$ 3.07 |  68.18$\pm$ 25.37
0.5,0.45 |  51.783 $\pm$ 0.0 |  81.0 $\pm$ 8.43 | 83.23 $\pm$ 6.71 |  64.42 $\pm$ 16.42
0.48,0.46 |  56.5 $\pm$ 4.28 | 84.29 $\pm$ 5.75 |  83.94 $\pm$ 5.96 |  83.3 $\pm$ 11.93
0.5,0.46 |  51.78$\pm$ 0.0 |  83.97 $\pm$ 5.68 | 87.89 $\pm$ 3.24 |  71.29 $\pm$ 9.05
0.49,0.47 |  51.83$\pm$ 0.08 | 85.16 $\pm$ 4.85 |  82.05 $\pm$ 9.76 |  66.85 $\pm$ 15.13
0.51,0.47 |  52.2 $\pm$ 1.01 |  85.2 $\pm$ 5.52 | 89.50$\pm$ 2.38 |  59.7 $\pm$ 6.93
0.49,0.48 |  57.93$\pm$ 7.64 |  86.69$\pm$ 2.22 | 87.63 $\pm$ 3.02 |  68.3 $\pm$ 13.33
Table 25: Averaged accuracy and standard deviation values for MNIST 5-6 dataset across 5 trials when the classification model M is an MLP. The noise rates have been swapped. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC. $\rho_{+},\rho_{-}$ | SimpleNN | WGANXtraY | WGANXtraYEntr | GLC
---|---|---|---|---
0.49,0.4 |  49.5 $\pm$ 0.0 |  76.5$\pm$ 2.28 |  78.48 $\pm$ 3.61 | 84.61 $\pm$ 6.27
0.45,0.42 |  67.9 $\pm$ 12.86 |  80.58 $\pm$ 2.42 |  78.81 $\pm$ 3.44 | 92.55 $\pm$ 1.99
0.46,0.45 |  73.08$\pm$ 9.97 |  82.16 $\pm$ 1.43 |  80.11 $\pm$ 4.14 | 89.24 $\pm$ 5.11
0.48,0.45 |  54.8 $\pm$ 10.72 | 79.86$\pm$ 2.52 |  79.43 $\pm$ 4.25 |  79.14 $\pm$ 8.44
0.5,0.45 |  49.5 $\pm$ 0.0 |  79.3 $\pm$ 4.44 | 80.84 $\pm$ 2.54 |  80.2 $\pm$ 7.94
0.48,0.46 |  54.3 $\pm$ 7.7 |  79.98 $\pm$ 3.12 |  78.03 $\pm$ 2.61 | 80.22 $\pm$ 8.54
0.5,0.46 |  49.53 $\pm$ 0.0 |  81.17 $\pm$ 1.84 | 81.37 $\pm$ 1.60 |  76.62 $\pm$ 9.06
0.49,0.47 |  56.7 $\pm$ 9.07 |  79.04 $\pm$ 1.23 | 80.85 $\pm$ 1.77 |  73.19 $\pm$ 9.23
0.51,0.47 |  49.53 $\pm$ 0.0 |  80.31 $\pm$ 0.87 | 82.68 $\pm$ 1.32 |  57.77 $\pm$ 10.59
0.49,0.48 |  52.53 $\pm$ 4.51 |  76.30 $\pm$ 8.98 | 79.45 $\pm$ 4.14 |  57.89 $\pm$ 6.18
Table 26: Averaged accuracy and standard deviation values for MNIST 7-9 dataset across 5 trials when the classification model M is an MLP. The noise rates have been swapped. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC. $\rho_{+},\rho_{-}$ | SimpleNN | WGANXtraY | WGANXtraYEntr | GLC
---|---|---|---|---
0.49,0.4 |  50.67 $\pm$ 0.0 |  73.32 $\pm$ 3.47 |  72.30 $\pm$ 4.85 | 86.58 $\pm$ 3.78
0.45,0.42 |  57.4 $\pm$ 3.57 |  73.2$\pm$ 3.840 |  73.95 $\pm$ 1.41 | 78.70 $\pm$ 14.49
0.46,0.45 |  59.47 $\pm$ 10.2 |  74.47 $\pm$ 3.11 |  75.43 $\pm$ 1.99 | 75.93 $\pm$ 5.38
0.48,0.45 |  53.63 $\pm$ 5.9 | 76.30 $\pm$ 1.84 |  72.06$\pm$ 6.07 |  74.86 $\pm$ 13.01
0.5,0.45 |  50.67$\pm$ 0.0 |  73.52 $\pm$ 2.12 | 73.81 $\pm$ 4.41 |  59.50$\pm$ 14.12
0.48,0.46 |  56.39 $\pm$ 7.3 | 75.72 $\pm$ 0.71 |  73.26 $\pm$ 4.69 |  74.07$\pm$ 3.04
0.5,0.46 |  50.67$\pm$ 0.0 | 74.87 $\pm$ 1.95 |  73.68 $\pm$ 2.47 |  63.25 $\pm$ 7.62
0.49,0.47 |  57.38 $\pm$ 7.19 |  75.34$\pm$ 2.76 | 77.54 $\pm$ 2.52 |  60.31 $\pm$ 16.9
0.51,0.47 |  50.67 $\pm$ 0.0 | 72.55 $\pm$ 3.60 |  70.84 $\pm$ 4.47 |  62.9 $\pm$ 9.09
0.49,0.48 |  50.7$\pm$ 0.802 |  74.08 $\pm$ 3.77 | 76.86 $\pm$ 3.18 |  64.82$\pm$ 7.35
Table 27: Averaged accuracy and standard deviation values for MNIST 4-9 dataset across 5 trials when the classification model M is an MLP. The noise rates have been swapped. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC. $\rho_{+},\rho_{-}$ | SimpleNN | WGANXtraY | WGANXtraYEntr | GLC
---|---|---|---|---
0.49,0.4 |  47.52 $\pm$ 0.0 |  94.11 $\pm$ 2.62 | 96.47 $\pm$ 0.93 |  95.03 $\pm$ 4.32
0.45,0.42 |  82.94 $\pm$ 16.53 |  96.18 $\pm$ 0.25 |  94.13 $\pm$ 2.15 | 97.37$\pm$ 0.75
0.46,0.45 |  91.09 $\pm$ 4.14 |  95.42 $\pm$ 1.21 |  95.70 $\pm$ 0.53 | 96.93$\pm$ 1.72
0.48,0.45 |  52.05$\pm$ 7.58 |  94.76 $\pm$ 1.81 | 95.74 $\pm$ 1.51 |  86.82 $\pm$ 8.57
0.5,0.45 |  47.52 $\pm$ 0.0 |  95.45 $\pm$ 0.86 | 95.92 $\pm$ 0.36 |  83.53 $\pm$ 16.28
0.48,0.46 |  84.89 $\pm$ 18.70 |  95.58$\pm$ 0.92 | 95.70 $\pm$ 1.25 |  81.68 $\pm$ 6.72
0.5,0.46 |  56.74 $\pm$ 18.39 |  93.58 $\pm$ 1.76 | 95.93 $\pm$ 0.56 |  69.8$\pm$ 18.35
0.49,0.47 |  59.15 $\pm$ 16.95 |  95.17 $\pm$ 0.44 | 95.77 $\pm$ 0.97 |  87.66$\pm$ 6.74
0.51,0.47 |  47.52 $\pm$ 0.0 | 96.19 $\pm$ 0.68 |  95.94 $\pm$ 0.65 |  72.5 $\pm$ 19.72
0.49,0.48 |  66.77 $\pm$ 15.79 |  96.05$\pm$ 0.3 | 96.51 $\pm$ 0.75 |  80.6 $\pm$ 11.17
Table 28: Averaged accuracy and standard deviation values for MNIST 1-7 dataset across 5 trials when the classification model M is an MLP. The noise rates have been swapped. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC. $\rho_{+},\rho_{-}$ | SimpleNN | WGANXtraY | WGANXtraYEntr | GLC
---|---|---|---|---
0.49,0.4 |  53.13 $\pm$ 3.47 |  85.41 $\pm$ 9.57 |  85.92 $\pm$ 0.71 | 86.4 $\pm$ 16.17
0.45,0.42 |  86.92 $\pm$ 8.73 |  89.83 $\pm$ 2.17 |  91.73 $\pm$ 1.26 | 91.87 $\pm$ 8.83
0.46,0.45 |  87.81 $\pm$ 5.72 |  89.64 $\pm$ 3.54 | 90.06 $\pm$ 3.26 |  78.97 $\pm$ 29.79
0.48,0.45 |  72.05 $\pm$ 15.0 | 90.20 $\pm$ 2.23 |  88.95 $\pm$ 5.18 |  89.02 $\pm$ 7.82
0.5,0.45 |  50.56 $\pm$ 0.0 |  87.34 $\pm$ 1.61 | 89.01 $\pm$ 1.26 |  87.31 $\pm$ 9.92
0.48,0.46 |  62.94 $\pm$ 14.05 |  89.09 $\pm$ 3.78 |  86.82 $\pm$ 6.97 | 92.98 $\pm$ 3.90
0.5,0.46 |  60.29 $\pm$ 13.79 | 87.63 $\pm$ 1.45 |  84.48$\pm$ 4.10 |  76.19$\pm$ 14.64
0.49,0.47 |  60.14 $\pm$ 10.33 |  85.00 $\pm$ 6.33 | 90.30 $\pm$ 2.18 |  62.04 $\pm$ 18.35
0.51,0.47 |  50.567 $\pm$ 0.0 | 87.00 $\pm$ 5.30 |  82.06 $\pm$ 15.81 |  65.6 $\pm$ 18.09
0.49,0.48 |  57.91 $\pm$ 13.11 | 91.26 $\pm$ 0.79 |  91.13 $\pm$ 1.82 |  67.8 $\pm$ 10.19
Table 29: Averaged accuracy and standard deviation values for MNIST 0-6 dataset across 5 trials when the classification model M is an MLP. The noise rates have been swapped. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC. $\rho_{+},\rho_{-}$ | SimpleNN | WGANXtraY | WGANXtraYEntr | GLC
---|---|---|---|---
0.49,0.4 |  53.43$\pm$ 14.15 | 99.47$\pm$ 0.41 |  98.49$\pm$ 1.77 |  96.58 $\pm$ 2.73
0.45,0.42 |  98.78$\pm$ 1.09 |  98.45 $\pm$ 1.17 | 99.63$\pm$ 0.22 |  96.52$\pm$ 5.47
0.46,0.45 |  98.95$\pm$ 0.77 | 99.3 $\pm$ 0.23 |  96.43 $\pm$ 6.44 |  95.15 $\pm$ 6.12
0.48,0.45 |  88.31 $\pm$ 20.9 | 99.28 $\pm$ 0.53 |  99.25 $\pm$ 0.39 |  96.02 $\pm$ 2.72
0.5,0.45 |  51.06 $\pm$ 7.90 |  97.96 $\pm$ 1.60 | 99.24$\pm$ 0.68 |  87.13 $\pm$ 14.9
0.48,0.46 |  67.34 $\pm$ 25.74 | 99.47 $\pm$ 0.19 |  99.41 $\pm$ 0.26 |  73.50 $\pm$ 14.18
0.5,0.46 |  50.07 $\pm$ 7.47 |  98.97$\pm$ 0.61 | 99.13 $\pm$ 0.92 |  95.9 $\pm$ 3.18
0.49,0.47 |  51.64 $\pm$ 4.29 |  99.15 $\pm$ 0.94 | 99.59$\pm$ 0.18 |  91.35$\pm$ 13.29
0.51,0.47 |  46.81 $\pm$ 0.96 |  99.2 $\pm$ 0.78 | 99.62$\pm$ 0.07 |  83.26 $\pm$ 16.59
0.49,0.48 |  65.36 $\pm$ 23.45 |  98.02 $\pm$ 2.39 | 99.16$\pm$ 1.21 |  80.5 $\pm$ 17.81
Table 30: Averaged accuracy and standard deviation values for MNIST 0-1 dataset across 5 trials when the classification model M is an MLP. The noise rates have been swapped. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  66.9 $\pm$ 8.86 |  86.35 $\pm$ 2.02 |  86.61 $\pm$ 0.93 |  89.41 $\pm$ 1.03 |  50.0 $\pm$ 0.0 | 89.47 $\pm$ 2.31
0.42,0.45 |  84.86 $\pm$ 3.15 |  78.66 $\pm$ 14.43 |  86.4 $\pm$ 1.78 | 90.16 $\pm$ 1.3 |  70.21 $\pm$ 14.74 |  85.52 $\pm$ 6.05
0.45,0.46 |  87.0 $\pm$ 3.26 |  87.16 $\pm$ 1.03 |  80.08 $\pm$ 15.08 |  86.87 $\pm$ 6.28 |  87.65 $\pm$ 2.80 | 89.61 $\pm$ 1.88
0.45,0.48 |  73.03 $\pm$ 18.83 |  87.95 $\pm$ 1.41 |  84.59 $\pm$ 1.64 | 89.05 $\pm$ 0.92 |  50.0 $\pm$ 0.0 |  85.75 $\pm$ 3.41
0.45,0.5 |  65.51 $\pm$ 15.09 | 87.38 $\pm$ 1.74 |  77.35 $\pm$ 14.58 |  85.0 $\pm$ 3.82 |  50.0 $\pm$ 0.0 |  83.13 $\pm$ 7.56
0.46,0.48 |  80.49 $\pm$ 4.82 | 87.53 $\pm$ 0.69 |  86.63 $\pm$ 0.61 |  84.28 $\pm$ 5.23 |  54.96 $\pm$ 9.87 |  84.8 $\pm$ 5.15
0.46,0.5 |  58.79 $\pm$ 10.39 |  82.55 $\pm$ 10.03 |  85.26 $\pm$ 4.37 |  70.01 $\pm$ 16.46 |  50.0 $\pm$ 0.0 | 87.49 $\pm$ 2.6
0.47,0.49 |  62.87$\pm$ 14.08 |  85.85 $\pm$ 2.81 | 86.89 $\pm$ 0.41 |  84.6 $\pm$ 4.85 |  58.31 $\pm$ 13.23 |  86.72 $\pm$ 2.87
0.47,0.51 |  50.17 $\pm$ 0.34 |  70.55 $\pm$ 35.29 | 88.25 $\pm$ 1.18 |  86.44 $\pm$ 1.15 |  50.0 $\pm$ 0.0 |  76.4 $\pm$ 19.08
0.48,0.49 |  60.14 $\pm$ 13.36 |  86.07 $\pm$ 1.32 | 88.03 $\pm$ 0.59 |  89.47 $\pm$ 0.40 |  51.42 $\pm$ 2.85 |  67.86 $\pm$ 12.61
0.5,0.5 |  38.52 $\pm$ 9.61 | 85.61 $\pm$ 1.71 |  79.67 $\pm$ 14.86 |  75.35 $\pm$ 16.16 |  47.71 $\pm$ 4.53 |  64.72 $\pm$ 14.46
Table 31: Averaged accuracy and standard deviation values for Fashion MNIST 7-9 (Sneakers and Boots) dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GCE. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---
0.4,0.49 |  50.4 $\pm$ 0.95 |  77.36 $\pm$ 1.11 |  76.75 $\pm$ 2.51 |  66.85 $\pm$ 10.82 |  50.0 $\pm$ 0.0 | 79.47$\pm$ 1.24
0.42,0.45 |  73.58 $\pm$ 3.83 |  78.07 $\pm$ 0.25 |  76.28 $\pm$ 1.65 | 79.23 $\pm$ 0.58 |  56.6 $\pm$ 9.99 |  78.91 $\pm$ 1.26
0.45,0.46 |  75.95 $\pm$ 2.36 |  72.63 $\pm$ 11.32 |  72.15 $\pm$ 11.09 |  71.71 $\pm$ 10.92 |  54.67 $\pm$ 7.89 | 78.05$\pm$ 1.39
0.45,0.48 |  54.52 $\pm$ 8.98 | 78.04 $\pm$ 0.44 |  71.93 $\pm$ 10.98 |  68.2 $\pm$ 13.37 |  50.0 $\pm$ 0.0 |  76.94 $\pm$ 5.25
0.45,0.5 |  50.0 $\pm$ 0.0 | 78.50 $\pm$ 0.34 |  76.09 $\pm$ 2.23 |  66.76 $\pm$ 11.03 |  50.0 $\pm$ 0.0 |  73.62 $\pm$ 5.22
0.46,0.48 |  60.14 $\pm$ 4.34 | 78.11 $\pm$ 0.58 |  69.58 $\pm$ 11.55 |  54.52 $\pm$ 8.96 |  50.04 $\pm$ 0.08 |  77.88 $\pm$ 2.08
0.46,0.5 |  53.25 $\pm$ 5.41 |  77.37 $\pm$ 1.86 |  76.4 $\pm$ 2.73 |  67.83 $\pm$ 13.00 |  50.0 $\pm$ 0.0 | 78.15 $\pm$ 1.38
0.47,0.49 |  53.78 $\pm$ 5.63 |  77.74 $\pm$ 1.06 | 77.93 $\pm$ 0.22 |  69.97 $\pm$ 9.37 |  50.0 $\pm$ 0.0 |  72.55 $\pm$ 10.18
0.47,0.51 |  50.0 $\pm$ 0.0 | 78.23 $\pm$ 0.42 |  77.91 $\pm$ 0.98 |  74.62 $\pm$ 7.72 |  50.0 $\pm$ 0.0 |  73.37 $\pm$ 4.20
0.48,0.49 |  52.77$\pm$ 4.13 |  76.46 $\pm$ 2.08 |  77.97 $\pm$ 0.23 | 78.59 $\pm$ 0.72 |  52.82 $\pm$ 4.59 |  72.08 $\pm$ 10.52
0.5,0.5 |  56.15 $\pm$ 6.079 |  77.14 $\pm$ 0.83 | 77.35 $\pm$ 0.66 |  71.10 $\pm$ 7.51 |  49.83 $\pm$ 0.20 |  55.16 $\pm$ 25.36
Table 32: Averaged accuracy and standard deviation values for Fashion MNIST 0-6 (Tshirt and Shirt) dataset across 5 trials when the classification model M is an MLP. The table demonstrates that for very high noise rates, i.e., close to 0.5, WGAN based schemes have higher accuracies and low variation among the trials than GLC, GCE and LDMI on most cases. $\rho_{+},\rho_{-}$ | Simple NN | WGANXtraY | WGANXtraYEntr | GLC | GCE | LDMI
---|---|---|---|---|---|---