# Oversampling Adversarial Network for Class-Imbalanced Fault Diagnosis

Masoumeh Zareapoor , Pourya Shamsolmoali , Jie Yang  School of Electronic,
Information and Electrical Engineering
Shanghai Jiao Tong University, Shanghai, China.

###### Abstract

The collected data from industrial machines are often imbalanced, which poses
a negative effect on learning algorithms. However, this problem becomes more
challenging for a mixed type of data or while there is overlapping between
classes. Class- imbalance problem requires a robust learning system which can
timely predict and classify the data. We propose a new adversarial network for
simultaneous classification and fault detection. In particular, we restore the
balance in the imbalanced dataset by generating faulty samples from the
proposed mixture of data distribution. We designed the discriminator of our
model to handle the generated faulty samples to prevent outlier and
overfitting. We empirically demonstrate that; (i) the discriminator trained
with a generator to generates samples from a mixture of normal and faulty data
distribution which can be considered as a fault detector; (ii), the quality of
the generated faulty samples outperforms the other synthetic resampling
techniques. Experimental results show that the proposed model performs well
when comparing to other fault diagnosis methods across several evaluation
metrics; in particular, coalescing of generative adversarial network (GAN) and
feature matching function is effective at recognizing faulty samples.

###### keywords:

Adversarial network, Class-imbalanced, Faulty sample, Fault diagnosis,
Classification.

††journal: MSSP

##  1 Introduction

Recently, industry applications changed to electronic machines. When the
manufacturing machines perform undesirable, failures will be occurred, which
may lead to disastrous malfunctions such as economic losses and environmental
pollution [1]. Therefore, timely diagnosis of the failures is very important
to prevent extreme costs for operations and maintenance. On the other side,
the amount of data generated in industrial fields is increasing day by day.
This massive amount of data requires a robust learning system that can predict
and classify the data efficiently. In the last few years, detecting machine
failure has been a hot research topic for engineering areas [2, 3]. In this
paper, we focus on the fault detection problem, in which, the objective is to
successfully identify the abnormal or faulty operating conditions. The
existing fault detection approaches have shown a fair amount of success and
mostly are based on data-driven techniques [4, 5, 6]. These approaches have
higher requirements on available samples for model learning; i.e. for
imbalanced class data, these approaches generalize poorly. The collected data
from industrial environments usually suffer from masses of labeled data, since
the data are naturally imbalanced, i.e. the genuine/normal data are more than
faulty data in a real industrial environment. General classifiers when dealing
with imbalanced data they emphasize the majority class samples. Data
imbalanced problem has its applications in many real-world problems such as
credit card fraud, biology and cancer detection, industrial fault diagnosis
and many other aspects, even improper classification in these cases may have
serious repercussions. Numerous methods and strategies have been developed to
address the imbalancedness problem, in which sample handling is one of them.
The sampling techniques are often easy to implement and usually, they fall
under oversampling (increases number of minority class samples) or
undersampling (delete some part of majority class samples) [7]. SMOTE [8] is
an advanced oversampling method which generates synthetic samples for the
minority instances and many of its variations have been proposed [9], and has
proven to be a practical and effective solution to the imbalanced problem. In
the last few years, deep learning techniques have emerged as a strong
framework that is applied to various applications including, dimensionality
reduction, motion capturing, speech and visual recognition and also more
recently used to address challenges in fault diagnosis of industrial machines
[10, 1, 11]. Deep learning models inherently have the capacity to deal with
complex learning problems that are difficult to solve by traditional machine
learning techniques. Generative adversarial network (GAN) introduced in [12],
is a deep learning model which can generate realistic data from a random
distribution, in a zero-sum game framework. Training of GAN tends to increase
the error rate of the discriminative network. Since the introduction of GAN,
it has been used on a number of tasks to boost classification accuracy by
generating synthetic samples [13, 14, 15, 16, 17].

In this paper, we are interested in a model for imbalanced fault diagnosis for
simultaneous classification and fault detection named Minority oversampling
Generative Adversarial Network (MoGAN). The proposed model is detailed in
Figure 1. We used the generative adversarial network to generate synthetic and
informative minority samples and successfully restore balance in our
imbalanced environment. The model consists of two interdependent networks,
such that, the proposed generative network is an efficient oversampling
technique to generate the synthetic minority samples by incorporating majority
class distribution for generating new minority samples, and the discriminative
network designed to correctly classify the input into several categories. The
discriminator of our model acts as a classifier and fault detector. In some
variation of GAN [18, 19], the discriminator is trained to classify the input
into multiple classes rather than two classes. In this work, we embrace this
idea for simultaneous classification and fault detection. During training, the
discriminator learns to identify the faulty samples from the normal samples,
thus, in fact, it became a fault detector (for given false positive rate).
Figure 2 shows the structure of our discriminator, it simultaneously acts as a
classifier and fault detector. The state of the arts in this regard suffers
from three main problems [2, 4, 20, 6]. First, these methods interfere with
the majority of data distribution. The second is the overfitting of the
training data. The third is diversity in data distribution since there are
different prototype data in imbalanced datasets.

Figure 1: The MoGAN architecture. The discriminator of MoGAN is designed for
simultaneous classification and fault detection. $e_{r}$ and $e_{f}$ indicates
as real and fake (generated) samples respectively. Our discriminator creates
several clusters in its embedding space to distinguish not only fake and real
image but also between normal and faulty samples.

Our proposed model conceived to address these main problems. We perform
comparative results with different fault diagnosis approaches, in particular,
most commonly oversampling techniques. Although the existing fault diagnosis
techniques show promising results, our model improves upon these techniques
across several evaluation metrics. In particular, we want to show that the
discriminator of our model can be a fault detector. We need to note here that,
in the entire paper, we indicated the faulty samples as minority class and the
normal samples as majority class. The rest of the paper is organized as
follows. Section 2 provides a brief background of the fault diagnosis
approaches in the industrial environment. Section 3 and 4 presents the
proposed fault diagnosis approach. Experiments are given in section 5 followed
by discussion and analysis in section 6, finally we conclude the model in
section 7.

Summary: We propose MoGAN a simulatanouse classifer and fault detector method;
the generative network of our model is an efficient oversampling technique to
generate the synthetic minority samples, and the discriminative network
designed to correctly classify the input into several categories; the
discriminator of our model acts as a classifier and fault detector. Given an
imbalanced data, the extracted feature vectors are fed to the discriminator D.
D predicts the faulty label from the normal data as well as distinguishes
between real and fake data instances; N, F1, F2 and F3 indicates the normal
and different type of faulty samples respectively. G generates minority
samples from the mixture of data distribution. The class information embedded
in the latent space in order to draw a target class. Note that, the proposed
MoGAN while generating a high-quality minority class sample; it also enables
the discriminator to create several clusters in its output space to better
distinguish the faulty samples from the normal samples

##  2 Related Works

In this section we discussed several approaches that are based on bearing
fault diagnosis. Handling imbalancedness in fault diagnosis is one of the many
ways in this field. The most recent reviews in fault diagnosis techniques can
be seen in [2, 3]. The authors deeply investigated relevant papers in machine
learning and deep learning techniques based on fault detection. The first work
that employed CNN to fault detection approaches is proposed by Singh et al.
[11]. Since then many papers have appeared [21, 10, 2, 22, 23, 7]. These works
mostly focused on a simple architecture of CNN, and the classification results
significantly outperform the conventional machine learning techniques. Since
our paper lies on the GAN based fault diagnosis, we mostly focused to discuss
these types of methods. GAN introduced by Goodfellow et al. [12] in 2014, it
has achieved outstanding effects in the field of image recognition. Since
then, various kinds of GAN-based algorithms have appeared. One of the earliest
works that used GAN for fault detection techniques is proposed by Lee et
al.[4].Their model termed as ADASYN, aims to improve the classification by
addressing the class imbalance problem in fault diagnosis techniques. ADASYN
method addresses the imbalanced problem by assigning a weight to the minority
samples using k-nearest neighbor. CatAAE [24] is another method uses GAN for
unsupervised fault diagnosis approaches. The model includes an autoencoder
into the adversarial training process and imposes a latent space distribution
for unsupervised learning. Then it uses an additional classifier to correctly
distinguish the fake distribution from the real distribution. Li et al.[25]
proposed a fault diagnosis method to learn the most valuable features and uses
these features in order to detect the abnormal behavior of the system. Feng
Jia et al.[21] proposed a deep neural network technique that consists of four
layers; input layer, local layer, feature layer, and output layer, to extract
shift-invariant features in order to identify mechanical health fault. The
authors achieved around 98% accuracy which is higher than other existing
methods. Chen et al.[22] proposed a new model for fault detection techniques
by integrating CNN and extreme learning machine (ELM). Their proposed model
consists of three steps as; pre-processing step by using continuous wavelet
transforms, feature extraction step by using CNN, classification step by using
ELM. More recently, a two-stage learning method contains sparse filtering and
a neural network is proposed for fault diagnosis techniques in order to learn
the discriminative features from raw signals [26]. More recently, Jiao et al.
[27] proposed RJAAN model for fault diagnosis when there are discrepany
distributions between the source domain and the target domain. The authors
employed the joint distribution alignment into their learning structure to
develop a comprehensive domain adaptation framework for fault diagnosis
models. In [28], Wang et al. proposed Multi-scale transfer learning for
bearing fault diagnosis model termed as ”MDIAN”. This model aligns the feature
distributions through multi-scale strategy. This authors demonstrated theor
model capacities through an extensive evaluations.

Although these methods have shown successful performance on induction motor
fault diagnosis, most of these approaches have a difficult training process
and has not been investigated fully advantage of deep learning models. Some
sampling-based methods are used for fault diagnosis in order to improve the
classification though rebalancing the dataset. Xie et al.[6] improved
classification performance by generating some potential synthetic minority
samples. EMICIL is proposed in [20] for class imbalanced learning based fault
diagnosis approaches. However, these methods can loosely be considered to be
similar to the proposed MoGAN, as they are not simulations techniques for
classification and fault detection.

##  3 GANs and cGANs

The original GAN consists of two networks, generative and discriminative which
are simultaneously trained. The generator trains to generate fake samples
which are very similar to real samples, and the discriminator trains to
distinguish the real samples from the fake samples. Given a set of sample $z$
from the real data distribution $D_{real}=(x_{i})_{i}^{n}$; let $G_{u}$
denotes the generators, where is often neural network in practice, and $u$ is
parameters of the generators; similarly, $D_{v}$ denotes the discriminators,
and $v$ is the parameters of the discriminator. GAN trains to obtain
$\theta^{(G)}$ to generate samples from the data distribution $D_{g}$, and the
discriminator learns to recognize whether the samples are generated $D_{g}$ or
real data $D_{real}$. The basic GAN [12] is the training parameters $u$, $v$
so as to optimize the following objective function:

$\begin{split}\min_{G_{u}}\max_{D_{v}}E_{x\sim D_{real}}\log
D(x;\theta_{d})\\\ +E_{z\sim D_{z}}\log(1-D(G(z;\theta_{g});\theta_{d}))\\\
\end{split}$ (1)

where, $D_{g}$ and $D_{z}$ are the empirical distributions of training
samples. For a random sample $x$, which either belong to $D_{real}$ or $D_{g}$
and also the parameter $\theta_{d}$, we have a binary class as
$D(x,\theta_{d})\in[0,1]$ which is a score based on the probability of x
(belong to real data or generated data). In Eq.LABEL:eq:e1, the discriminator
should give a high score for real samples, while minimizing it for generated
samples from $D_{g}$. In the other side, the generator works opposite of the
discriminator such that it learns to maximize the score for the fake samples;
i.e., it aims to minimize the $E_{z\sim
D_{z}}\log(1-D(G(z;\theta_{g});\theta_{d}))$ while maximizing the $E_{z\sim
P_{z}}\log(D(G(z;\theta_{g});\theta_{d}))$. The c-GAN [13] is an extension
model, which contains extra information from the data and termed as the
conditional generative adversarial network. The c-GAN framework can be
formulated as follows. For generator G; $G:Z\times Y\rightarrow X$ , and
similarly for the discriminator the adversarial algorithm in Eq.LABEL:eq:e1
can be reformulated as:

$\begin{split}\min_{G}\max_{D}V(D,G)=N_{D}+N_{G}\\\
f_{D}=E_{(}x,y~{}p_{real}(x,y))[\log D(x,y)]\\\
f_{G}=E_{z,p_{z},y\sim{p_{y}}}[\log(1-D(G(z,y),y))]\\\ \end{split}$ (2)

where $(x,y)\in X\times Y$ are sampled from the real data distribution
$p_{real}(x,y)$, and $z\in Z$ values are from the noise distribution
$p_{z}(z)$ and $y\in Y$ are sampled from conditional data vectors which in
this paper we define $y$ as a class of training and we want to generate
samples towards a specific target. In this paper, since we are dealing with
different class conditioning, we embrace the ”c-GAN” [13] idea and apply the
class information in the last space to draw desire samples towards a target
class. We discuss the model structure in details, in the next section.

Figure 2: The discriminator of our proposed MoGAN against CatAAE [24]. Other
methods like CatAEE in addition to discriminator; they have a classifier in
their architecture to classify the faulty samples from the normal one.

##  4 Fault Diagnoses with MoGAN

Inspired by the prior research, in this paper we present a new scheme for
imbalanced fault diagnosis approaches. The proposed model aims to restore the
balance by generating realistic minority samples from a mixture of data
distribution. Resampling techniques are widely used in fault diagnosis
techniques; however, they may generate unnecessary or improper samples which
lead to complicating the computational cost, and also, degraded the
classification results, for more details we refer the reader to [2, 3]. We are
interested in a model for simultaneous classification and fault detection,
which can be addressed by our proposed MoGAN. In this paper we show an
effective way to; generate the most informative minority samples; adopt a new
data distribution (a mixture of all available distribution) to avoid
generating improper samples; enable the model to generate samples with a high
variety; correctly classify the faulty and normal samples. In fact, we would
like to propose a fault diagnosis technique that not only correctly produces
samples for a target class, but also classifies the outputs as one of the
classes that are available during training (either normal or any type of
faulty samples). We design the discriminator in addition to its ability, to be
able to return the faulty samples to its corresponding class and became a
fraud detector. The architecture of the proposed model is given in Figure 1.
As we briefly explained the GAN in section 3, it contains two networks;
$G-network$ and $D-network$ that are trained simultaneously. However,
generating samples of imbalanced data from a latent vector $z$ is difficult,
and this requires an accurate estimation of probability distribution between
the class samples to prevent the overlapping problem. To overcome this
problem, we propose a mixture probability distribution for learning a joint
distribution of both the majority and minority class samples in the
adversarial training. We assume $G(x)$ is the probability of the generated
sample $x$, and $D(x)$ is the probability of classifying x either as fake
samples or one of the $c$ classes. However, in some formulation of GAN, the
discriminator considered as a multiclass classifier [29, 19] rather than $bi-
class$. It means that, instead of having two outputs as real or fake samples,
it has multiple outputs as $k+1$ and known as a multiclass discriminator for
simultaneous classification and fault detection. If real training set given by
$y_{k}\in R^{R}$ (where $k=0,1,2,...$) with the $c$ class, the generator G
aims to generate the fake samples according to the following [29]:

$\tilde{y}_{k}=G(\tilde{z}_{k},\tilde{a}_{k};u)=u\tilde{z}_{k}+\tilde{a}_{k}$
(3)

where, $\tilde{y}$ indicates fake samples; $\tilde{z}$ is a random variable;
$\tilde{a}$ is a distribution on different classes, and vector $u$ is the
parameters of the generator. Similarly, we define the discriminator as
$D(x;w)=f(x^{T}w)$, where $x$ is an input vector contains real and fake
samples; vector $w$ represents the discriminator parameters. Let $x$ be the
real sample from $p_{real}$ distribution, and $\hat{x}_{-}$, $\hat{x}_{+}$ are
the majority-minority classes respectively from the marginal distribution of
$p_{x}$. G is the generative model of our GAN, which maps a random vector $z$
to images that have the same support as $x$. We denote the distribution of
$g(z)$ by $p_{g}(x)$. We assume that our imbalanced dataset deals with two
sets class component and simply divide our true distribution into two
components as; Majority and Minority data distribution. One way to find the
probability distribution in an imbalanced dataset (including normal and faulty
samples) is to estimate a set of the majority density, then declare test
samples outside of the estimated level set as fault [29]. Unfortunately, these
methods are not accurate since they mostly related to the one-class density
(prone to majority class) and neglect estimating the minority density level,
due to severe lack of the samples. However, our idea is to learn these
components jointly rather than single one in order to approximate the data
distribution for the minority samples. We consider the density of the data
points of random class $c$ as $p_{c}(f)$ with a threshold
$\delta_{c}\in[0,1]^{(c+1)}$, and $f_{c}$ is the subset of data support, where
$f_{c}={f:p_{c}(k)>\delta_{c}}$ ; such that the $f_{c}$ disjoint with the
boundary margin if $\\{\delta_{c}\\}_{c=1}^{c}$.

We assume that there are two set class components as normal (majority)
$F_{Mj}$ and faulty (minority) $F_{Mi}$ class samples, $f_{j}\in F_{Mj}$ and
$f_{i}\in F_{Mi}$. If we have access to both densities (normal data and faulty
data) then according to [29, 15] the detection rate can be achieved with a
threshold ratio :

$\frac{F_{Mj}(x)}{F_{Mi}(x)}$ (4)

In practice, for an imbalanced dataset, it is difficult to have access to the
minority examples distribution, because there are not sufficient training
samples for such class. Our generative model generates samples $F_{Mi}$ from a
mixture distribution. Using mixture distribution for minority class samples
will be in the form of

$\pi f_{j}+(1-\pi)f_{c};where\pi\in[0,1]$ (5)

This observation implies that function $f$, can be as generator $G$, then the
discriminator $D$ will be optimized by the theories in the next section.

###  4.1 Theoretical analysis

Many data sampling techniques have had a fair amount of success in recent
years [2, 22, 3]. We found that, it is not easy to assist conventional GANs in
imbalanced data. This is due to the fact that, with fewer minority samples,
the random parameters of the generative model might generate similar samples
which may lead to an overfitting problem. To overcome this problem, proposed
`MoGAN` uses all the available data distribution to produces the realistic
samples for the minority component in the training set. To implement the
proposed model, let $D$ and $G$ denote the discriminator and generator, with
their corresponding distribution $P_{D},P_{G}$. The discriminator $D$ learns
an accurate decision boundary for all training set; (1) for any $(x,y)\in R$
there is $w_{y}^{T}f(x)>w_{c}^{T}f(x)$; (2) for any $x\in G$,there is
$\max_{c=1}^{c}w_{c}^{T}f(x)<0$. In this assumption, both conditions show the
classification correctness and true-fake correctness on the real data and the
generated data respectively. As we mentioned above, $P_{D}$ is trained over
$c+1$ classes, where $c$ classes are from true data and $(+1)$ is the fake
detected samples. In more details, the score at $(c+1)^{th}$ index
$D_{(c+1)}(.)$ indicates the probability of $p_{g}$ distribution, hence
represent a fake sample. Moreover, as shown in [30], if a generator produces
samples which meet $p_{g}(x)=p_{r}(data)$, the discriminator will be failed to
identify the sample classes. However, we believe that in the case of diverse
class data with an imbalancedness environment, a good generator has a form of
$p_{g}(x)\neq p_{r}(data)$, which is proved in [30]. In this assumption, the
generator is able to generate samples by using all the available data
distribution. As we mentioned above, our generative model contains mixture of
data distribution

$p_{g}(x)=\pi p_{mj}(x)+\pi(1-p_{r}(x))$ (6)

including majority class distribution $p_{mj}$ and minority class distribution
$(1-p_{r})$, suppose $Supp(p)=\cup_{c=1}^{c}Supp(p_{g})\cup Supp(p_{r})$.

$\delta_{j}^{T}f_{g}=\pi\delta_{j}^{T}f_{j}+\pi(1-\delta_{c}^{T}f_{c})\leq 0$
(7)

Moreover, according to the above conditions, the first statement is
$\delta_{j}^{T}f_{j}>0$ , therefore, in Eq.8,
$\delta_{j}^{T}f_{g}<\delta_{c}^{T}f_{c}$ for any $i\neq j$. Further, we
estimate the probability distance as, $T_{c}=T(\mu_{c},\Sigma c)$, where it
contains mean vector $\mu_{c}$ and covariance matrix $\Sigma c$. this equation
should stand for each class, and we compute $\mu_{c}$ and $\Sigma c$ for all
classes and then drawn a latent vector $z_{c}$ from distribution $T_{c}$.
Based on Eq. LABEL:eq:e2, the generator is trained by minimizing:

$\min_{\theta_{g}}E_{z\sim p_{z}}[\log(1-D(G(z;\theta_{g});\theta_{d}))]$ (8)

therefore, the optimal discriminator for false positive FP will be derived in
the form of

$\acute{D}=\frac{p_{r}(x)}{p_{r}(x)+p_{g}(x)}$ (9)

where $p_{r}(x)$ is a true/real data distribution. In this paper we use $"r"$
to denote the real data, $"Mj"$ and $"Mi"$ also denote the normal and fault
samples with their corresponding distribution as $p_{Mj},p_{Mi}$. If we have
access to $p_{Mj}$ and $p_{r}$, then the false positive rate will be in the
form of:

$\frac{p_{g}(x)}{p_{r}(x)}$ (10)

Now, we state and prove some important theory in what we follow.

Theory 1: For a fixed generator G, with a mixture data distribution, the
optimal discriminator $\acute{D}$ will be in the form of
$\acute{D}=\frac{p_{Mj}+(1-\pi)p_{r}}{P_{r}}$.

Proof. Given G and $V(D,G)$ can be formulated as:

$\begin{split}\frac{1-\acute{D}}{\acute{D}}=\frac{p_{g}(x)}{p_{r}(x)}\\\
=\frac{\pi p_{Mj}(x)+(1-\pi)p_{r}(x)}{p_{r}(x)}=\frac{\pi p_{Mj}(x)}{\pi
p_{r}(x)}+(1-\pi)\\\ \end{split}$ (11)

then according to Eq. LABEL:eq:e6, the optimal discriminator is also a false
positive rate for fault detection. It is worth to mention that, the proposed
discriminator is a neural network with an output of $d-dimensional$ component
(as, c-class, fake). It comprises two steps. First, discrimination between
real and fake is needed, such that, the discriminator encodes the input image
$x$, into $[d=\Sigma_{(i=1)}^{m}w_{i}\times\varphi(\mu_{i};\Sigma_{i})]$;
where $\varphi(.)$ is the Gaussian probability density function [19], $w_{i}$
iis the mixture weight either can be [0, 1], if the component produces the
highest similarity amidst the other components, indicate value 1, or value 0
otherwise. $\Sigma_{i}$ indicated as covariance matrix for Gaussian component
$i$. Calculating $d$, is the probability of being a real image. Furthermore,
in the second step, the model assigns the corresponding class as, normal
sample or faulty sample.

$\begin{split}V(D)=[E_{(Mj\sim p_{r})}\log(D(Mj))]\\\ +[E_{(Mi\sim
p_{r})}\log(D(Mi))]+[E_{z\sim p_{z})}\log(1-D(G(z)))]\\\
=\int_{x}[p_{Mj}\log(d(D(x)))+p_{Mi}\log(1-d(D(x)))]\\\ \end{split}$ (12)

Theory 2 : With $\acute{D}$, the optimal $G$ is attained when $Mj\sim P_{Mj}$
and $Mi\sim P_{Mi}$ have $zero$ mutual information; i.e., they should not
share similar information.

Figure 3: Confusion matrix for creating synthetic dataset;(a) using proposed
mixture data distribution and (b) using SVM classifier on generated data to
show the robustness of the balanced dataset.

Proof. This theory can be possible if the loss is $zero$. According to Eq 8,
to train such mixture data distribution that could generate minority samples
in low-density regions of the real data manifold, we need a loss function that
efficiently encourages generator to do that. We followed [30] that, the
authors proposed a loss function based on KLD (Kullback-Leibler divergence) as
follow:

$\begin{split}\min_{\theta_{g}}-\Gamma(p_{g}(x;\theta_{g})+E_{x\sim
p_{g}(x;\theta_{g})}\log_{P_{Mj}}(x)\\\ \rho[P_{Mj}(x)>\delta]+Loss(G)\\\
where;Loss(G)=\min_{\theta_{g}}\big{\|}E_{x\sim
p_{r}(x)}[f(G(z;\theta_{g}))]\big{\|}\end{split}$ (13)

where $\Gamma(.),\rho(.)$ are the marginal and joint entropy function.

Table 1: The details of CWRU dataset Data A | Data B | Data C
---|---|---
Normal |  IR |  OR |  BF |  Normal |  IR |  OR |  BF |  Normal |  IR |  OR |  BF
0.0000 |  0.007 |  0.014 |  0.021 |  0.0000 |  0.007 |  0.014 |  0.021 |  0.0000 |  0.007 |  0.014 |  0.021
800 |  800 |  800 |  800 |  800 |  800 |  800 |  800 |  800 |  800 |  800 |  800
Table 2: The details of IPF dataset labels |  Normal |  F1 |  F2 |  F3 |  F4 |  F5 |  F6
---|---|---|---|---|---|---|---
No. of samples |  65947 |  5140 |  3381 |  3412 |  68 |  127 |  28109

Figure 4: Classification Balanced accuracy (%) of minority class with
different number of hidden nodes on (a) S1 and (b) S2 datasets. We used NSVDD
[31] and DNCNN [7], Chen et al.[32], Jiang et al.[33] and RJAAN [27] as
baselines.

It comprises three terms; produce a generator with a distribution which not
intersects with a high density of real data (first & second term), meanwhile,
it is close to data manifold (third term). According to their results, the
generator that is trained with Feature matching loss (Eq. 8), is able to
generate multivariate samples which, falls on data manifold, and also samples
which are scattered outside of data manifold. It means that when a generator
is trained by feature matching loss is able to generate mixture samples from a
mixture of data distribution. Theory 2, by adjusting feature matching loss,
signifies that, $Mj\sim P_{Mj}$ and $Mi\sim P_{Mi}$ are completely
uncorrelated. Therefore, that is what we intend; generating faulty samples
(minority) from a mixture distribution which are different and recognizable
from the normal samples.
To summarize, we proposed a new adversarial network for imbalanced fault
diagnosis, to generate minority samples in a low-density area of the true data
by using feature matching loss that could enrich the generator distribution
and accordingly improves the detection rate of the discriminator. The main
differences between this model and other GAN variations are; the way to
generate the minority samples, stability, prevention of any changes in
majority class, and also, prove that the discriminator can be fault detector
in the case of fault diagnosis. In Section 5 we empirically demonstrate that
when our designed discriminator trained with a mixture generator based Feature
Matching loss, has an impressive ability to detect real fault samples.

###  4.2 Learning Proposed Fault Diagnosis Model

Based on the above observations, we proposed a fault diagnosis approach which
not only extracts the high level and discriminative features from raw data,
but it can rebalance the training data in order to improve the model
performance. Several authors applied GANs for imbalanced data classification
based fault diagnosis techniques [24, 10, 4]. Nonetheless, these approaches
are not directly applied to specific classes. Our proposed MoGAN learns all
the available data distribution and generates the minority class samples for
the training set. Then, the obtained results from the mixture generator fed to
the discriminator for the further process. For designing the generator and
discriminator networks we follow the convention as DCGAN [16]. However the
difference is in defining the extra condition in G-network in order to draw a
specific target, and also the objective function of the generative model,
which we use Feature Matching loss [15]. D-network also which is the false-
positive rate for the fault diagnosis; also, it is designed to identify the
wrong samples to the generator based on the corresponding classes.

###  4.3 Model training

First, we extract all the fault and normal features as rare and normal cases.
We use mixture data distribution [29] to generate the faulty samples and then
joint the generated samples to the original corresponding samples in for
balanced data. The generator and discriminator involve a $2Dtranspose$
convolutional network. Both networks are inspired by the pioneering work DCGAN
[16]. The generative model contains $5$ convolution transpose layers
(deconvolutional layer) to perform up-sampling on a $128-dimensional$ vector
from a proposed mixture distribution. We used instance normalization [34]
instead of using batch normalization between each of the deconvolution layers.
This allows the model to increase training stability. Parametric Rectified
linear units (PReLU) are used in all layers except the final layer. We
replaced the tanh nonlinearity function with Feature Matching loss to the
final output of the generative model. We implement our proposed model by using
Keras 2.1.2, the deep learning open-source library, and TensorFlow 1.3.0 GPU
as the backend deep learning engine. Python 3.6 is used for all the
implementations. All the implementations of the network are conducted on a
workstation equipped with an Intel i7-6850K CPU, a 64 GB Ram, and an NVIDIA
GTX Geforce 1080 Ti GPU and the operating system is Ubuntu 16.04.

##  5 Experiments and Protocol

We perform an extensive experimental on the proposed MoGAN and compare it to
several fault diagnosis approaches. We used different oversampling techniques
EWMOTE [35], EMICIL [20], WSMOTE [36] and Border-SMOTE [9] on classification
faulty samples. In addition, we compare our model to A2CNN [10], CatAAE [24],
DSN [37] and ADASYN [4]. To implement these methods we followed their
structure111https://github.com/openai/improved-gan, and for the false
detection rate between normal $\&$ faulty samples we defined the ratio
$\frac{1-D_{g}}{D_{g}}$. For a fair comparison, we trained the baselines and
our proposed model with the same system setting. Our experiments are carried
out on several relative datasets. We evaluated our proposed model over two
cases; the first case includes binary imbalanced class, while the second case
includes a multiclass imbalance problem. More details are given in the next
section. Recall that, in the training phase, we use our proposed GAN to
generate minority class samples from mixture distribution and in the testing
phase we show that our designed discriminator is able to correctly distinguish
the faulty samples from the normal ones.

Table 3: Comparative results on IMS dataset. Mi refers to Minority samples and Mj refers to majority samples. Best results are bold 333The results of [28], [38] and [32] are taken from their works. | S1 | S2
---|---|---
Methods |  Mi testing |  Mj testing |  Training set |  Testing set |  $G_{mean}$ |  Mi testing |  Mj testing |  Training set |  Testing set |  $G_{mean}$
SVMRBF |  0.5248 |  0.9707 |  0.7385 |  0.7631 |  0.9453 |  0.6381 |  0.9826 |  0.9139 |  0.9233 |  0.8663
DNCNN [7] |  0.8693 |  1.000 |  0.8991 |  0.9287 |  0.9183 |  0.7529 |  1.000 |  0.9364 |  0.918 |  0.8997
Jiang et al.[33] |  0.8817 |  1.000 |  0.923 |  0.9128 |  0.9321 |  0.8916 |  1.000 |  0.9752 |  0.9623 |  0.9141
Shen et al. [38] |  — |  — |  0.9813 |  0.9427 |  0.9607 |  — |  — |  0.9985 |  0.9341 |  0.946
Chen et al.[32] |  0.8336 |  0.9973 |  0.8966 |  0.9027 |  0.9399 |  0.8884 |  0.9994 |  0.9418 |  0.9532 |  0.8993
MDIAN [28] |  — |  — |  1.000 |  0.946 |  0.9347 |  — |  — |  0.9938 |  0.9826 |  0.9018
NSVDD [31] |  0.8505 |  1.000 |  0.9726 |  0.9348 |  0.9594 |  0.8737 |  0.9917 |  0.9905 |  0.9711 |  0.924
MoGAN |  0.9123 |  1.000 |  0.9853 |  0.9539 |  0.9789 |  0.9431 |  0.9857 |  0.9986 |  0.9838 |  0.9601

###  5.1 Dataset preparation and implementation setting

For developing an efficient model for the fault diagnosis, providing good data
is necessary. In this paper, the datasets are selected based on the given
information in [2].

  * 1.

CWRU dataset 444http://csegroups.case.edu/bearingdatacenter/home, Case Western
Reserve Lab is simulated under three working conditions: (load
$1=1hp/1772rpm$; load $2=2hp/1750rpm$; load $3=3hp/1730rpm$). In each working
condition, four status are collected, normal, inner fault (IF), outer fault
(OF), and ball fault. We randomly selected 2048 samples after removing
coefficients. We grouped these samples based on the mentioned conditions as A,
B, C respectively. Each condition contains ten classes (one normal + nine
faulty samples), with 800 samples in each. In fact, each dataset contains 8000
samples including normal and faulty. More details are given in Table 1.

  * 2.

Case 1 555http://www.industrial-bigdata.com, Wind turbine freezing failure
forecast (WTFF), held by MIIT of China, it contains $28-dimensional$ time
series. The data are binary labeled, as normal and faulty samples. The number
of normal samples is “25768”, while the faulty samples are around “1554”,
wherein the imbalancedness ratio can be about 1:17. For the testing set, we
extract all types of data (contains both the normal $\&$ fault) from the
90,000th to 149,999th timestamp, and the rest are used for training.

  * 3.

Case 2 666https://www.phmsociety.org/ Industrial plant failure detection
(IPF), is provided by PHM society in 2015. The data contains eight dimensional
time series and is a multiclass data with six type of fault. The highest
imbalancedness ratio may reach to 1:1000. Table 2, shows more details about
used datasets.

  * 4.

IMS bearing dataset 777https://ti.arc.nasa.gov/tech/, is a well-known data
which is provided by the Center for Intelligent Maintenance Systems (IMS),
University of Cincinnati, and generated from the “Prognostics Data Repository
of NASA”. Two different test-to-failure experiments are conducted as to
achieve following datasets: outer race fault ($S1$ data) and ball fault ($S2$
data). We follow [39] [31], and use their collected data in this experiments.
$S1$ contains 6000 samples in which we used 4300 as training and the rest as
testing. In the training set, there are 3800 normal samples, 500 faulty
samples. We chose the imbalance ratio around $10:1$. The same process also is
executed to generate the ball fault dataset $S2$.

It is worth to mention that, the implementations of the network are conducted
on a workstation equipped with an Intel i7-6850K CPU with a 64 GB Ram and an
NVIDIA GTX Geforce 1080 Ti GPU and the operating system is Ubuntu 16.04.

Table 4: Precision, Recall, FAM - Different Resampling Methods using Case 1. Methods |  Recall |  Precision |  FAM
---|---|---|---
Original GAN [12] |  0.4973 |  0.6873 |  0.7297
MixtureGAN (ours) |  0.8902 |  0.9427 |  0.9064
CaAE [40] |  0.8641 |  0.8873 |  0.8619
RJAAN [27] |  0.8814 |  0.9106 |  0.8653
B-SMOTE [9] |  0.8452 |  0.6470 |  0.6941
EMICIL [20] |  0.7723 |  0.9275 |  0.8799
EWMOTE [35] |  0.7691 |  0.9138 |  0.8827
WSMOTE [36] |  0.8502 |  0.9336 |  0.8993
Table 5: Precision, Recall, FAM - Different Resampling Methods using Case 2. Methods |  Recall |  Precision |  FAM
---|---|---|---
Original GAN [12] |  0.6733 |  0.7859 |  0.6983
MixtureGAN (ours) |  0.8531 |  0.8946 |  0.8730
CaAE [40] |  0.8016 |  0.8236 |  0.8253
RJAAN [27] |  0.8375 |  0.8514 |  0.8432
B-SMOTE [9] |  0.8531 |  0.8946 |  0.8730
EMICIL [20] |  0.8305 |  0.8426 |  0.8342
EWMOTE [35] |  0.8609 |  0.8527 |  0.8667
WSMOTE [36] |  0.8436 |  0.8709 |  0.8561

###  5.2 Assessment Metrics

For imbalance class problems, overall accuracy and mean square error are not
appropriate. Instead, we used balanced accuracy which is defined in [36] as:

$\displaystyle BAC$ $\displaystyle=\frac{1}{2}(sensitivity+specificity)$ (14)
$\displaystyle G_{mean}$ $\displaystyle=\sqrt{sensitivity\times specificity}$
$\displaystyle f_{measure}$
$\displaystyle=(1+\alpha^{2})\frac{sensitivity\times specificity}{\alpha\times
sensitivity+specificity}$

where $\alpha$ is generally taken to be 1) in order to provide a comprehensive
evaluation. In the confusion matrix, the left column represents minority class
samples of the original dataset and the right column is the majority one. The
proportion of these columns will be the class distribution in each dataset.
The metrics have been calculated as follows.

##  6 Results and Analysis

Motivated by GAN models, we present a novel architecture to generate minority
class samples from a mixture of data distribution in order to restore a
balance in imbalanced datasets. We also used a feature matching loss function
to handle such mixture data distribution. Figure 3, shows the confusion matrix
for GAN based Feature matching loss function on the WTFF dataset. From left to
right, these values represent true positive rate, false-positive rate, false-
negative rate, true negative rate. These values convey that, the normal
samples give more than 95% TPR, and the faulty samples around 89%. It means,
our proposed mixture data distribution can produce completely different faulty
samples which are recognizable from the normal sets.

In addition, we conducted several experiments in which the results are shown
in Figures 3 \- 9 . We also compared the proposed generative model with
different resampling methods and the results are summarized in Table 3 \- 5.

SVM performance on the simulated dataset: The support vector machine is a
statistical classifier introduced by [41]. In this paper, we show the
performance of applying SVM to the generated dataset. In our work, the linear
kernel is not appropriate to separate faulty samples from normal ones.
Therefore, we also use a radial basis function to properly separate the
labeled datasets.

Figure 5: Performance of training error and classification rate of MoGAN and
CatAAE [24] on the Case 1. Figure 6: The ROC curve comparisons between various
methods using Case 1 and Case 2. We used SMOTE [9], GAN [12], EMICIL [20],
CaAE [40], and RJAAN [27] as baselines.

Briefly, RBF kernel is calculated as
$E_{ij}=\exp(-\frac{\|\alpha\|^{2}}{2\rho^{2}})$, where, $\rho$ is a user-
defined parameters and $\|\alpha\|$ is the Eclidean distance between two
samples $x_{i}$ and $x_{j}$. We followed [42] to implement the hyperparameters
of RBF on the WTFF dataset. Figure 3 (right), shows the confusion matrix for
the labeled dataset (balanced dataset) using SVM based RBF kernel as a
classifier. Although the true positive accuracy rate is around70%, the TNR is
less than 55%. We can conclude that the normal samples with 66% TPR are well
recognized, but the faulty due to 62% TPR, they couldn’t well identify. SVM
despite using a balanced dataset gives a poor performance for TPR. Figure 3
(left) shows better classification performance compared to SVM.

Figure 7: The t-SNE feature visualization. (a, d) A2CNN [10], (b, e) RJAAN
[27] ; (c, f) MoGAN. Figure 8: The t-SNE feature visualization. (a) A2CNN
[10], (b) RJAAN [27] ; (c) MoGAN. Figure 9: Relationship between accuracy and
loss while applying cross-training on bearing 1$\leftrightarrow 2$ datasets.
(A) and (B) bearing $1\rightarrow 2$, (C) and (D) bearing $2\rightarrow 1$.
Figure 10: The convergence analysis of MoGAN. Generator and Discriminator loss
during training. (Left): Case 1 and (Right): Case 2

Experiments on IMS bearing data: In this section we show our models’
functionality using IMS dataset. We train our model over 70% of available data
to evaluate the prediction rate of our model. Table 3 summarizes the
prediction accuracy in each experiment. For fair comparisons, we used several
states of the arts methods, and compare our model with them. On order to keep
the results robust, we used the average of 50 trials as the final result. It
is worth to mentioning that for the highly imbalanced data, minority sample
prediction is more importance than the majority. As we can see in Table 3, the
minority testing accuracy of our model is around 4% and 6% higher than the
second best results in each datasets. $G_{mean}$ as defined in section 5.2 is
a promising evaluation metrics for the classification of imbalanced dataset.
The results convey that, G-mean value of MoGAN outperforms the other
baselines. In addition to these results we also used different imbalanced
ratio rate such as: $5:1\&3:1$, and compared our results with the same
baselines. Due to reducing the imbalancedness ratio in the training, we
achieve $0.983$ score in $G_{mean}$, and the results of NSVDD [31] and DNCNN
[7] are close to our results as: $0.973$ and $0.977$ respectively.

In Figure 4, we plot the balanced accuracy curves over S1 and S2 dataset based
on the number of hidden nodes. We compares our proposed model with Five
baselines that exploited the deep neural network in their architectures. The
results are achieved from the average 10 trials. As it observes, by growthing
the number of hidden nodes, our proposed model has a smooth and robust
performance. In both datasets, our model outperforms the other baselines in
terms of the ”minority testing accuracy”.

Diagnosis results of different methods: We conducted several experiments for
fault diagnosis problem. Our proposed model is trained and tested using two
different settings. First, we use Border-SMOTE [9] (B-SMOTE) and other
resampling techniques to traditionally resample the minority class instances
and simultaneously applied our discriminator to find the represented class for
each instance. Second, we use our proposed mixture data distribution
(including minority and majority class samples) to generate minority class
samples and then our proposed MoGAN is trained on the new dataset. As
suggested in [35], we used Recall, Precision, and FAM (average of AUC, MCC,
and F-measure) as evaluated metrics. The results are given in Table 4.

Although, using SMOTE displays better TPR for the normal samples, but it
performs very poor for the faulty samples due to the less number of samples.
From the results, it observes that there is a significant performance on both
the normal and faulty samples by using resampling techniques. SMOTE performs a
bit better than GAN, EWMOTE, and EMICIL on recall. It may due to generating
wrong faulty samples that fall insie the normal class region. Therefore, this
can improve the recall, accordingly reduce the precision and FAM, because some
of the faulty samples fall inside of the majority region. The second best
results can be found for WSMOTE.

Figure 11: The comparison results between various fault diagnosis techniques
based on Recall, Precision, F1, and AUC. The baselines are; Xie et al.[6],
Easy-SMT [5], SDAE [23], CatAAE [24]; left to right-CWRU dataset (Data A, B
and C). Figure 12: $BA$ (Balanced accuracy) values, $F_{measure}$ and
$G_{mean}$ values in different epochs on CWRU dataset using MoGAN; left to
right: Data A, B, and C. We note that, the iteation is up to ”$\times
10^{2}$”.

Based on all evaluation metrics, generated samples using our model are closer
to the original samples comparing to other resampling techniques. We also use
Case 2 \- which is the multiclass dataset - to compare the various
oversampling methods for imbalanced learning. The results summarized in Table
5. In contrast to Table 4, in Case-2 , SMOTE gives a poor performance on
recall, which means that it generated noisy and useless minority samples that
leads to misclassification problem, thus decrease the recall.

However, the recall of original GAN, WSMOTE, and EMICIL is much better than
SMOTE, but worse in precision and FAM scores. It is worth to mention that,
WSMOTE is better than EMICIL and EWSMOTE on precision, which means the
decision boundary, is prone to the majority class region in order to generate
the new samples. As it observes, the proposed model performs well on
imbalanced multiclass learning in all the assessment measures. We also plot
the ROC graph in Figure 6. We used Case 1 for binary classification and Case 2
for the multiclass classification. We used SMOTE [9], GAN [12], EMICIL [20],
CaAE [40], and RJAAN [27] as baselines. The results show, our proposed model
performs better than other sampling and none-sampling techniques for both the
binary and multiclass examples. However, in the binary classification, the
second-best result is for the Original GAN, and SMOTE. While in the multiclass
dataset, the second-best result gives to RJAAN which has a comparable
performance with MoGAN.

In addition, we compare our proposed MoGAN with CatAAE, since they have a
similar setting in sense of using GAN to generate minority samples and use
discriminator to distinguish the faulty samples from the normal samples. The
results are given in Figure 5. We achieved the results on the Case 1 dataset.
It observed that the proposed model has a less training error compares to
CatAAE, also it reaches the convergence after a few epochs (faster
convergence).

Feature visualization: In this expriment we demonstrate the effectiveness of
our model, and illustrate its versatility by tackling imbalancdeness problem,
using t-SNE [43] to visualize the feature representations. We used RJAAN [27]
and A2CNN [10] as baseline, and compared our proposed MoGAN with these models.
The expriment is evaluated on IMS and CWRU datasets. The results is presented
in Figure 7 and Figure 8. From the results can be observed that, MoGAN and
RJAAN can extract more effective features of each health status which further
help to improve the accuracy performance. However, our proposed MoGAN method
can better aggregate features of the same health condition for accurate
classification.The distribution of features extracted by the MoGAN is better
than other models. The second best results are obtained by RJAAN model.

In Figure 9 we show the bearing degradation assessment by using IMS (bearing 1
and 2) dataset on healthy indicators while using cross-training and testing.
We have shown the training-testing accuracy and loss while training on one
dataset and testing on the other one. Figure 9 demonstrates the relation
between accuracy and loss during the cross-training of the two bearings of IMS
datasets ($1\Leftrightarrow 2$). The results indicate that applying cross-
training helps to minimize the loss and also increases the adaptation
accuracy. In this evaluation, the best results are achieved from
$1\Leftrightarrow 2$, since, the number of the training samples in the bearing
$1$ dataset is more than bearing $2$ dataset and the testing samples in
bearing $2$, have low complexity.

Figure 10 represents the stability of the proposed model. The figure contains
two subplots, the discriminator loss (orange) and the generator loss (blue).
It can be observed, both losses are erratic before around iteration 50. As we
used pre-trained generator in our model, it prevents faster convergence of the
discriminator compare to the generator at the early epoch and maintaining the
stabilized learning process of GANs and reducing mode collapses.

Next, we show the proposed model effectiveness on the CWRU datasets. We
compared our model with different techniques based on fault diagnosis. We
evaluated the performance of our model with other baselines on several
assessment metrics, which are defined in section 5.2 like, BA, Recall,
Precision, AUC, G-mean, and F-measure. The results are shown in Figure 11 and
Figure 12. For both evaluations, we used CWRU with three data categories as A,
B, and C.

In dataset A, our model performs better than baselines in all evaluation
metrics, and the second-best goes for CatAAE and SDAE on precision. The
evaluation scores convey that, these methods correctly classified the faulty
and normal samples, but they failed to generate dense and real-look faulty
samples to improve the recall and AUC. For the dataset B and C, our model has
a compatible performance with SDAE. However, SDAE in both the datasets failed
to generate faulty samples that fall in the minority region. Easy-SMT has the
worst performance on all three datasets. The work by Xie at al.[6] has a
compatible performance with other baselines only on the dataset A, for the
other datasets, it doesn’t show any progress. For more details please see
Figure 11, 12.

Figure 13: BA: Balanced Accuracy (%) on different datasets. We used GAN [12],
SDAE [23], ADASYN [4], RJAAN [27], A2CNN [10] and WDCNN+AdaBN [44] as
baselines. Figure 14: Confusion matrix of A2CNN [10], RJAAN [24]; and MoGAN.

We also illustrate the performance of our proposed model base on mixture data
distribution over several synthetic sampling techniques to identify the faulty
samples, in Figure 13. We compared our model with several relative baselines
like; SDAE [23], ADASYN [4], RJAAN [27], A2CNN [10], WDCNN+AdaBN [44] and
original GAN [12]. We can find that for Case 1 and dataset C, the performances
of our model and RJAAN are very close. However, for Case 2, Dataset A and B,
our model gives better scores. As the results show, simple GAN performs poorly
in all datasets with average accuracy being around (53% - 62%). It proves that
simple GAN is not suitable for fault diagnosis techniques, in particular,
multiclass classification problems. For binary classification, GAN obtained
68% BA which is smoothly better than other scores. The second worse gives to
SDAE, which has poor performance for multiclass datasets, but for the binary
(Case 1) shows a good result. WDCNN with using AdaBN achieved 92% balanced
accuracy which is obviously greater than other baselines. This result conveys
that the samples learned by WDCNN+AdaBN are more invariant than the samples
learned by the other baselines. In addition, by comparing our model with A2CNN
we can find that in each dataset, the performance of our model is superior to
A2CNN. This means that generating minority samples from the mixture data
distribution and using feature matching function in order to handle the
mixture distributions, can significantly improve the fault diagnosis under
different datasets. It is also interesting that for binary datasets such as
case 1, the fault diagnosis balanced accuracy of the WDCNN+AdaBN, RJAAN, and
A2CNN is a bit better than others.

Finally, we show the classification result of each class in Case 2 datasets
including normal and six faulty samples (the dataset details are given in
Table 2) in Figure 14. $F1-F6$ indicates the number of faults and $N$ is the
normal sample. $F6$ is categorized as the low imbalanced ratio
($ratioratearound1:3$), $F1,F2$, and $F3$ are categorized as medium imbalance
ratio (around 1: 20) and $F4,F5$ is categorized as the severe imbalanced ratio
(around 1: 1000). From the results it can be easily inferred that, the
proposed MoGAN performs better for low and medium imbalanced ratio, but a
little get worse for severe imbalancedness (but still it is better than the
baselines). Moreover, our proposed model is more robust compares to the other
baselines on recognizing the faulty samples. In particular, it shows a good
performance with a high imbalanced ratio compares to the RJAAN and A2CNN.

##  7 Conclusion

The ability to recognize the faulty samples in an imbalanced dataset is very
important for many classification systems. The proposed MoGAN is an effective
end to end model for simultaneous classification and fault detection in
imbalanced fault diagnosis. To restore balance in the imbalanced dataset we
used mixture data distribution to generate minority class samples. In the case
of fault diagnosis with a mixture of data distribution, we proved that the
discriminator can be considered as a fault detector. We also trained our
generator which contains different data distribution with feature matching
loss function. In this way, the generator ables to generate samples in low-
density areas of data and helps to the discriminator for better observation of
detection. Three sets of data are used to evaluate our proposed model. The
comprehensive experiments show, the proposed solution is comparable to other
popular fault detection techniques, even in some cases outperforms them. We
also trained a SVM classifier on the generated samples for classification
task. The results report that SVM has poor performance comparing to our
proposed model. However, the obtained result from the generated data (66% -
51%) is better than the result of using the original dataset (48% - 27%). This
clearly reports that our proposed generator with mixture data distribution is
able to generate informative samples that are useful for the classification.

For the future, this work will be extended in order to reduce the
dimensionality to could improve fault diagnosis approaches under different
conditions. We would like to develop a new loss function to handle mixture
data distribution that enriches the generator performance and accordingly
improve the discriminator’s detection rate.

## Acknowledgment

This research is partly supported by NSFC, China (No: 61806125, 61977046),
Committee of Science and Technology, Shanghai, China (No. 19510711200).

## References

  * [1] M. Xia, T. Li, L. Xu, L. Liu, C. W. De Silva, Fault diagnosis for rotating machinery using multiple sensors and convolutional neural networks, IEEE/ASME Transactions on Mechatronics 23 (1) (2017) 101–110.
  * [2] S. Zhang, S. Zhang, B. Wang, T. G. Habetler, Machine learning and deep learning algorithms for bearing fault diagnostics-a comprehensive review, arXiv preprint arXiv:1901.08247 (2019).
  * [3] R. Liu, B. Yang, E. Zio, X. Chen, Artificial intelligence for fault diagnosis of rotating machinery: A review, Mechanical Systems and Signal Processing 108 (2018) 33–47.
  * [4] Y. O. Lee, J. Jo, J. Hwang, Application of deep neural network and generative adversarial network to industrial maintenance: A case study of induction motor fault detection, in: 2017 IEEE International Conference on Big Data (Big Data), IEEE, 2017, pp. 3248–3253.
  * [5] Z. Wu, W. Lin, Y. Ji, An integrated ensemble learning model for imbalanced fault diagnostics and prognostics, IEEE Access 6 (2018) 8394–8402.
  * [6] Y. Xie, T. Zhang, Imbalanced learning for fault diagnosis problem of rotating machinery based on generative adversarial networks, in: 2018 37th Chinese Control Conference (CCC), IEEE, 2018, pp. 6017–6022.
  * [7] F. Jia, Y. Lei, N. Lu, S. Xing, Deep normalized convolutional neural network for imbalanced fault classification of machinery and its understanding via visualization, Mechanical Systems and Signal Processing 110 (2018) 349–367.
  * [8] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: synthetic minority over-sampling technique, Journal of artificial intelligence research 16 (2002) 321–357.
  * [9] H. Han, W.-Y. Wang, B.-H. Mao, Borderline-smote: a new over-sampling method in imbalanced data sets learning, in: International conference on intelligent computing, Springer, 2005, pp. 878–887.
  * [10] B. Zhang, W. Li, J. Hao, X.-L. Li, M. Zhang, Adversarial adaptive 1-d convolutional neural networks for bearing fault diagnosis under varying working condition, arXiv preprint arXiv:1805.00778 (2018).
  * [11] O. Janssens, V. Slavkovikj, B. Vervisch, K. Stockman, M. Loccufier, S. Verstockt, R. Van de Walle, S. Van Hoecke, Convolutional neural network based fault detection for rotating machinery, Journal of Sound and Vibration 377 (2016) 331–345.
  * [12] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial nets, in: Advances in neural information processing systems, 2014, pp. 2672–2680.
  * [13] A. Odena, C. Olah, J. Shlens, Conditional image synthesis with auxiliary classifier gans, in: Proceedings of the 34th International Conference on Machine Learning-Volume 70, JMLR. org, 2017, pp. 2642–2651.
  * [14] P. Shamsolmoali, M. Zareapoor, R. Wang, D. K. Jain, J. Yang, G-ganisr: Gradual generative adversarial network for image super resolution, Neurocomputing 366 (2019) 140–153.
  * [15] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, Improved techniques for training gans, in: Advances in neural information processing systems, 2016, pp. 2234–2242.
  * [16] A. Radford, L. Metz, S. Chintala, Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint arXiv:1511.06434 (2015).
  * [17] P. Shamsolmoali, M. Zareapoor, L. Shen, A. H. Sadka, J. Yang, Imbalanced data learning by minority class augmentation using capsule adversarial networks, arXiv preprint arXiv:2004.02182 (2020).
  * [18] M. Zareapoor, H. Zhou, J. Yang, Perceptual image quality using dual generative adversarial network, Neural Computing and Applications (2019) 1–11.
  * [19] H. Eghbal-zadeh, W. Zellinger, G. Widmer, Mixture density generative adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 5820–5829.
  * [20] R. Razavi-Far, M. Farajzadeh-Zanjani, M. Saif, An integrated class-imbalanced learning scheme for diagnosing bearing defects in induction motors, IEEE Transactions on Industrial Informatics 13 (6) (2017) 2758–2769.
  * [21] F. Jia, Y. Lei, L. Guo, J. Lin, S. Xing, A neural network constructed by deep learning technique and its application to intelligent fault diagnosis of machines, Neurocomputing 272 (2018) 619–628.
  * [22] Z. Chen, K. Gryllias, W. Li, Mechanical fault diagnosis using convolutional neural networks and extreme learning machine, Mechanical Systems and Signal Processing 133 (2019) 106272.
  * [23] C. Lu, Z.-Y. Wang, W.-L. Qin, J. Ma, Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification, Signal Processing 130 (2017) 377–388.
  * [24] H. Liu, J. Zhou, Y. Xu, Y. Zheng, X. Peng, W. Jiang, Unsupervised fault diagnosis of rolling bearings using a deep neural network based on generative adversarial networks, Neurocomputing 315 (2018) 412–424.
  * [25] B. Li, P.-l. Zhang, H. Tian, S.-S. Mi, D.-S. Liu, G.-Q. Ren, A new feature extraction and selection scheme for hybrid fault diagnosis of gearbox, Expert Systems with Applications 38 (8) (2011) 10000–10009.
  * [26] Y. Lei, F. Jia, J. Lin, S. Xing, S. X. Ding, An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data, IEEE Transactions on Industrial Electronics 63 (5) (2016) 3137–3147.
  * [27] J. Jiao, M. Zhao, J. Lin, K. Liang, Residual joint adaptation adversarial network for intelligent transfer fault diagnosis, Mechanical Systems and Signal Processing 145 (2020) 106962.
  * [28] X. Wang, C. Shen, M. Xia, D. Wang, J. Zhu, Z. Zhu, Multi-scale deep intra-class transfer learning for bearing fault diagnosis, Reliability Engineering & System Safety (2020) 107050.
  * [29] M. Kliger, S. Fleishman, Novelty detection with gan, arXiv preprint arXiv:1802.10560 (2018).
  * [30] Z. Dai, Z. Yang, F. Yang, W. W. Cohen, R. R. Salakhutdinov, Good semi-supervised learning that requires a bad gan, in: Advances in neural information processing systems, 2017, pp. 6510–6520.
  * [31] C. Liu, K. Gryllias, A semi-supervised support vector data description-based fault detection method for rolling element bearings based on cyclic spectral analysis, Mechanical Systems and Signal Processing 140 (2020) 106682.
  * [32] Z. Chen, A. Mauricio, W. Li, K. Gryllias, A deep learning method for bearing fault diagnosis based on cyclic spectral coherence and convolutional neural networks, Mechanical Systems and Signal Processing 140 (2020) 106683.
  * [33] W. Jiang, C. Cheng, B. Zhou, G. Ma, Y. Yuan, A novel gan-based fault diagnosis approach for imbalanced industrial time series, arXiv preprint arXiv:1904.00575 (2019).
  * [34] D. Ulyanov, A. Vedaldi, V. Lempitsky, Instance normalization: The missing ingredient for fast stylization, arXiv preprint arXiv:1607.08022 (2016).
  * [35] W. Lin, Z. Wu, Y. Ji, An adaptive oversampling learning method for class-imbalanced fault diagnostics and prognostics, arXiv preprint arXiv:1811.07674 (2018).
  * [36] X. Zhang, Y. Zhuang, W. Wang, W. Pedrycz, Transfer boosting with synthetic instances for class imbalanced object recognition, IEEE transactions on cybernetics 48 (1) (2016) 357–370.
  * [37] C. Sun, M. Ma, Z. Zhao, X. Chen, Sparse deep stacking network for fault diagnosis of motor, IEEE Transactions on Industrial Informatics 14 (7) (2018) 3261–3270.
  * [38] C. Shen, D. Wang, F. Kong, W. T. Peter, Fault diagnosis of rotating machinery based on the statistical parameters of wavelet packet paving and a generic support vector regressive classifier, Measurement 46 (4) (2013) 1551–1564.
  * [39] W. Mao, L. He, Y. Yan, J. Wang, Online sequential prediction of bearings imbalanced fault diagnosis by extreme learning machine, Mechanical Systems and Signal Processing 83 (2017) 450–473.
  * [40] Z. Ren, Y. Zhu, K. Yan, K. Chen, W. Kang, Y. Yue, D. Gao, A novel model with the ability of few-shot learning and quick updating for intelligent fault diagnosis, Mechanical Systems and Signal Processing 138 (2020) 106608.
  * [41] V. N. Vapnik, An overview of statistical learning theory, IEEE transactions on neural networks 10 (5) (1999) 988–999.
  * [42] M. Zareapoor, P. Shamsolmoali, D. K. Jain, H. Wang, J. Yang, Kernelized support vector machine with deep learning: an efficient approach for extreme multiclass dataset, Pattern Recognition Letters 115 (2018) 4–13.
  * [43] L. v. d. Maaten, G. Hinton, Visualizing data using t-sne, Journal of machine learning research 9 (Nov) (2008) 2579–2605.
  * [44] W. Zhang, G. Peng, C. Li, Y. Chen, Z. Zhang, A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals, Sensors 17 (2) (2017) 425.