# Training Process of Unsupervised Learning Architecture
for Gravity Spy Dataset

Yusuke Sakai  Research Center for Space Science, Advanced Research
Laboratories, Tokyo City University, Setagaya-ku, Tokyo 158-0082, Japan
<EMAIL_ADDRESS>Yousuke Itoh  Graduate School of Science, Osaka
Metropolitan University, Sumiyoshi-ku, Osaka City, Osaka 558-8585, Japan
Nambu Yoichiro Institute of Theoretical and Experimental Physics (NITEP),
Osaka Metropolitan University, Sumiyoshi-ku, Osaka City, Osaka 558-8585, Japan
Piljong Jung  National Institute for Mathematical Sciences, Daejeon 34047,
Republic of Korea  Keiko Kokeyama  School of Physics and Astronomy, Cardiff
University, The Parade, Cardiff, CF24 3AA United Kingdom  Chihiro Kozakai
Gravitational Wave Science Project, Kamioka branch, National Astronomical
Observatory of Japan, Hida City, Gifu 506-1205, Japan  Katsuko T. Nakahira
Department of Information and Management Systems Engineering, Nagaoka
University of Technology, Nagaoka, Niigata 940-2188, Japan  Shoichi Oshino
Institute for Cosmic Ray Research, KAGRA Observatory, The University of Tokyo,
Hida City, Gifu 506-1205, Japan  Yutaka Shikano  Graduate School of Science
and Technology, Gunma University, Maebashi, Gunma 371-8510, Japan  Institute
for Quantum Studies, Chapman University, Orange, CA 92866, USA  JST PRESTO,
Kawaguchi, Saitama 332-0012, Japan  Hirotaka Takahashi  Research Center for
Space Science, Advanced Research Laboratories, Tokyo City University,
Setagaya-ku, Tokyo 158-0082, Japan  Institute for Cosmic Ray Research, The
University of Tokyo, Kashiwa City, Chiba 277-8582, Japan  Earthquake Research
Institute, The University of Tokyo, Bunkyo-ku, Tokyo 113-0032, Japan
<EMAIL_ADDRESS>Takashi Uchiyama  Institute for Cosmic Ray Research, KAGRA
Observatory, The University of Tokyo, Hida City, Gifu 506-1205, Japan  Gen
Ueshima  Department of Information and Management Systems Engineering, Nagaoka
University of Technology, Nagaoka, Niigata 940-2188, Japan  Tatsuki Washimi
Gravitational Wave Science Project, Kamioka branch, National Astronomical
Observatory of Japan, Hida City, Gifu 506-1205, Japan  Takahiro Yamamoto
Institute for Cosmic Ray Research, KAGRA Observatory, The University of Tokyo,
Hida City, Gifu 506-1205, Japan  Takaaki Yokozawa  Institute for Cosmic Ray
Research, KAGRA Observatory, The University of Tokyo, Hida City, Gifu
506-1205, Japan

###### Abstract

Transient noise appearing in the data from gravitational-wave detectors
frequently causes problems, such as instability of the detectors and
overlapping or mimicking gravitational-wave signals. Because transient noise
is considered to be associated with the environment and instrument, its
classification would help to understand its origin and improve the detector’s
performance. In a previous study, an architecture for classifying transient
noise using a time–frequency 2D image (spectrogram) is proposed, which uses
unsupervised deep learning combined with variational autoencoder and invariant
information clustering. The proposed unsupervised-learning architecture is
applied to the Gravity Spy dataset, which consists of Advanced Laser
Interferometer Gravitational-Wave Observatory (Advanced LIGO) transient noises
with their associated metadata to discuss the potential for online or offline
data analysis. In this study, focused on the Gravity Spy dataset, the training
process of unsupervised-learning architecture of the previous study is
examined and reported.

###### keywords:

deep learning, training process, hyperparameter tuning, classification,
transient noise

##  1 Introduction

Advanced Laser Interferometer Gravitational-Wave Observatory (Advanced LIGO)
detector [1] located at Livingston and Hanford, USA, made its first
observation of gravitational waves from the coalescence of a binary black hole
in September 2015 [2]. Following that, Advanced LIGO and Advanced Virgo [3] in
Pisa (Italy) have made three international joint observations and observed as
many as 90 events of gravitational waves emitted by the coalescence of the
compact binary [4, 5, 6, 7]. KAGRA [8, 9, 10, 11, 12, 13] in Japan will join
the next (fourth) observing run (O4) by Advanced LIGO and Advanced Virgo.

In the data analysis of gravitational waves, the technique of separating the
gravitational waves from the noise in the observed data is essential because
the signals of the gravitational waves are generally smaller than the detector
noise. Because the gravitational-wave detector is sensitive to environmental
and instrumental conditions, such as ground motions, air pressure, optics
suspensions, laser fluctuations, vacuum, and mirror, the non-stationary and
non-Gaussian noise called “transient noise” frequently appears in the
detector’s data [14]. The transient noise causes the detector to be unstable,
it can also hide and imitate gravitational-wave signals [15]. During O3b [7],
LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration
(LVK) reported that the transient noise rate with a signal-to-noise ratio
(SNR) of $>6.5$ was $1.17$ events per minute at LIGO Livingston.

Machine learning is increasingly being applied in the study of transient noise
[16, 17, 18]. Transient noise has various time-frequency characteristics that
are related to its causes. Classifying transient noise could provide a clue to
explore its origins and improve the performance of the detector [19]. Thus,
the Gravity Spy project [20, 21, 22, 23] attempted to classify the transient
noise. In the Gravity Spy project, the Omicron software [24] was used to
identify the signal of transient noise observed in the time-series data.
Following that, using the Omega Scan [25], a time-frequency spectrogram was
created around the identified transient noise as a 2D image. Based on a
portion of these created 2D images, 22 types of labels associated with the
characteristics or causes of transient noise were annotated for the analysis
using cloud resources in collaboration with LIGO detector characterization
experts and volunteer citizen scientists. Both the images and labels were
recorded. Finally, using the pre-classified images and labels, they classified
the transient noise in the remaining images using supervised learning.

Recently, there are some reports on the clustering and/or classification of
transient noise using unsupervised learning [26, 27, 28, 29, 30]. Sakai et al.
[26] discussed an architecture for the classification of transient noise using
unsupervised learning, which combines a variational autoencoder (VAE) [31]
[32] and invariant information clustering (IIC) [33]. The consistency between
the label annotated by the Gravity Spy project and the class provided by the
proposed unsupervised-learning architecture was confirmed using the Gravity
Spy dataset of LIGO O1, and the potential for the classification of transient
noise using unsupervised learning was discussed. Unsupervised learning is
expected to reduce annotation work for training data, increase classification
objectivity, and even classify a new class, such as the transient noise
because it does not require any pre-assigned labels for the training dataset.
Moreover, the training process of unsupervised learning is essential and some
detailed reports on the training process can be found in refs. [34, 35]. In
this study, focused on the Gravity Spy dataset, the training process of
unsupervised-learning architecture of ref. [26] is examined and reported.

The remainder of this study is organized as follows: In Section 2, we explain
the outline of VAE and IIC, which are used in the proposed architecture. In
Section 3, we review the architecture, which was proposed in our previous
research [26]. In Section 4, we report on how the training process of the
unsupervised learning architecture was conducted. We also give a summary of
the evaluation and obtained results discussed in [26]. Section 5 presents a
summary of the work.

##  2 Outline of Used Method for Proposed Architecture

The proposed architecture discussed in Section 3 to classify transient noise
is combined VAE [31] [32] and IIC [33], both of which are known as
unsupervised-learning methods.

###  2.1 Variational Autoencoder

We introduce VAE [31] [32] that forms the feature learning [36] [37] part of
our proposed architecture. VAE is a Bayesian inference method that uses a
predictive distribution for the parameters and was designed for unsupervised
learning. VAE has an architecture that compresses an image to the latent
variables before reconstructing the original image from the latent variables.
Let $\mathcal{X}\subset\mathbb{R}^{D}$ be the input space and
$\mathcal{Z}\subset\mathbb{R}^{J}$ be the latent space, where
$D,J\in\mathbb{Z}$ and $J<D$. Suppose a true (but unknown) probability
distribution is the parameterized model $P_{\bm{\theta}}$ of the variable
$\bm{x}\in\mathcal{X}$, and the latent variable $\bm{z}\in\mathcal{Z}$ is
represented instead of the parameters $\bm{\theta}$, which are not directly
observed. The marginal likelihood $P_{\bm{\theta}}(\bm{x})$, which we are
interested in, has the following relation (Bayes’ theorem),
$P_{\bm{\theta}}(\bm{x})=P_{\bm{\theta}}(\bm{x}|\bm{z})P_{\bm{\theta}}(\bm{z})/P_{\bm{\theta}}(\bm{z}|\bm{x})$.
Then, let the likelihood $P_{\bm{\theta}}(\bm{x}|\bm{z})$ be the generative
model, and a prior distribution $P_{\bm{\theta}}(\bm{z})$ be hypothesized to
be a Gaussian distribution. Unfortunately, $P_{\bm{\theta}}(\bm{x})$ is
intractable; therefore, we cannot evaluate it directly. However, considering
the posterior distribution as the inference model
$P_{\bm{\theta}}(\bm{z}|\bm{x})\sim Q_{\bm{\phi}}(\bm{z}|\bm{x})$ that is
parameterized by $\bm{\phi}$, we can evaluate $P_{\bm{\theta}}(\bm{x})$
indirectly. Then, the logarithm marginal likelihood can be expressed as

$\mathrm{ln}P_{\bm{\bm{\theta}}}(X)=D_{\mathrm{KL}}\left(Q_{\bm{\phi}}(\bm{z}|\bm{x})||P_{\bm{\theta}}(\bm{z}|\bm{x})\right)+L_{\bm{\theta},\bm{\phi}}(\bm{x}),$
(1)

where $D_{\mathrm{KL}}$ is the Kullback-Leibler divergence of two
distributions, and $L_{\bm{\theta},\bm{\phi}}(\bm{x})$ is known as the
variational lower bound, respectively. The first term on the right-hand side
of Equation (1) gradually approaches zero as the accuracy is increased.
Therefore, maximizing the log-likelihood can be replaced by the problem of
maximizing the variational lower bound as follows, $\mathop{\rm
arg~{}max}\limits_{\bm{\theta}}\mathrm{ln}P_{\bm{\bm{\theta}}}(\bm{x})=\mathop{\rm
arg~{}max}\limits_{\bm{\theta},\bm{\phi}}L_{\bm{\theta},\bm{\phi}}(\bm{x})$.
Then, the equation for the variational lower bound is expressed as

$\displaystyle L_{\bm{\theta},\bm{\phi}}(\bm{x})\sim-
D_{\mathrm{KL}}\left(Q_{\bm{\phi}}(\bm{z}|\bm{x})||P_{\bm{\theta}}({\bm{z}})\right)+\int{Q_{\bm{\phi}}(\bm{z}|\bm{x})\ln
P_{\bm{\theta}}(\bm{x}|\bm{z})}d\bm{z},$ (2)

where $\mathcal{X}=\\{\bm{x}=f_{\bm{\theta}}(\bm{z})|\bm{z}\in\mathcal{Z}\\}$
represents the input space and a map
$f_{\bm{\theta}}:\mathcal{Z}\to\mathcal{X}$ is known as a decoder. Similarly,
the latent space is also represented by
$\mathcal{Z}=\\{\bm{z}=g_{\bm{\phi}}(\bm{x})|\bm{x}\in\mathcal{X}\\}$ and a
map $g_{\bm{\phi}}:\mathcal{X}\to\mathcal{Z}$ is is known as an encoder.
Suppose the two multivariate Gaussian distributions in the latent space are
expressed as follows,
$Q_{\bm{\phi}}(\bm{z}|\bm{x})=N(\bm{z};\bm{\mu},\bm{\sigma}^{2}\bm{I})$ and
$P_{\bm{\theta}}(\bm{z})=N(\bm{z};\bm{0},\bm{I})$, where $\bm{\mu}$ is the
mean and $\bm{\sigma}$ is the variance, and $\bm{I}$ is the identity matrix.
Then, using the reparameterization trick [31], Equation (2) can be expressed
as

$\displaystyle
L_{\bm{\theta},\bm{\phi}}(\bm{x})=\frac{1}{2}\sum_{j=1}^{J}(1+\ln\sigma_{j}^{2}-\mu_{j}^{2}-\sigma_{j}^{2})+\frac{1}{L}\sum_{l=1}^{L}\mathrm{ln}P_{\bm{\bm{\theta}}}(\bm{x}|\bm{z}^{(l)}),$
(3)

where $\mu_{j}$ and $\sigma_{j}$ are the mean and variance of $J$-dimensional
Gaussian distribution, respectively, and $L$ is the number of samples per
datapoint. $L$ can be set to $1$ when the minibatch size is large enough [31].

###  2.2 Invariant Information Clustering

We briefly explain IIC [33] that forms the classification part of our proposed
architecture. Although classification using supervised learning generally
requires numerous labels for its training, IIC can classify without these
labels and achieve results comparable to supervised learning. The goal of IIC
is to learn what is in common between the paired data. These paired data, for
example, could be different images with the same characteristics. Let
$\bm{x},\bm{x}^{\prime}\in\mathcal{X}$ be paired data following a joint
probability distribution $P(\bm{x},\bm{x}^{\prime}$) and $C\in\mathbb{Z}$ be
the number of output classes. IIC learns a classifier (representation)
$\bm{\Phi}:\mathcal{X}\to\mathbb{R}^{C}$ that maximizes the mutual information
$I$ expressed by entropy $H$ as follows

$I(\bm{\Phi}(\bm{x}),\bm{\Phi}(\bm{x}^{\prime}))=H(\Phi(\bm{x}))-H(\Phi(\bm{x})|\Phi(\bm{x}^{\prime})).$
(4)

Moreover, the maximum value of Equation (4) such that $\bm{x}=\bm{x}^{\prime}$
can be analytically obtained as follows

$\max(I(\bm{\Phi}(\bm{x}))=\ln C.$ (5)

Calculating a criterion for data within the same class and data between
different classes is useful for evaluating the performance of the classifier.
Thus, let
$P_{ij}=\bm{\Phi}(\bm{x}^{(i)})\cdot\bm{\Phi}(\bm{x}^{\prime(j)})^{T}$ be the
conditional joint distribution and
$P_{i}=\sum_{j}\bm{\Phi}(\bm{x}^{(i)})\cdot\bm{\Phi}(\bm{x}^{\prime(j)})^{T}$
be the marginal distribution, where $\bm{x}^{(i)}\in\mathcal{X}$ are data
belonging to the $i$th class, and a notation $T$ means a transpose and
“$\cdot$” means an inner product. Then, the mutual information Equation (4)
can be expressed as

$I(\bm{\Phi}(\bm{x}),\bm{\Phi}(\bm{x}^{\prime}))=\sum_{i}^{C}\sum_{j}^{C}P_{ij}\ln\frac{P_{ij}}{P_{i}P_{j}}.$
(6)

The objective of training IIC is to construct the classifier $\Phi$ that
maximizes Equation (6).

##  3 Outline of Proposed Architecture

We review an unsupervised-learning architecture proposed in ref. [26], which
has deep convolutional neural networks for the classification of transient
noise. The proposed architecture consists of two processes: feature learning
using VAE and classification using IIC. We input four 2D images of transient
noise with different time durations to the proposed architecture, and its
shape is $(4,224,224)$ data having four square images $(224,224)$. More
details of the input 2D images are shown in Section 4.1. Regarding the
implementation of VAE, we used the architecture, as shown in Figure 1. Each
block shows the components of neural networks and their shape. The purpose of
VAE is to learn the latent variables, which are the compressed features from a
large dataset. We used four convolutional neural networks in our proposed
architecture because convolutional layers are known to be effective in image
processing. Additionally, we used a ReLU activation function to avoid the
vanishing gradient problem in deep neural networks and used a batch
normalization technique in all convolutional layers to stabilize training. For
the encoder part, the first convolution layer outputs a shape of
$(M,64,112,112)$ from the input 2D image having a shape of $(M,4,224,224)$,
where $M$ is a minibatch size. A max-pooling layer next to the convolutional
layer extracts the features and compresses them into a shape of
$(M,112,56,56)$. Following that, further features are extracted with three
convolutional layers and are averaged through an average-pooling layer. A
fully connected layer connects to all variables in neural networks. After
using reparameterization trick [31], the latent variables $\bm{z}$ are
obtained (in the case of Figure 1, its shape is $(512)$). For the decoder
part, the decoder constructs an image element whose shape is $(M,512,14,14)$
from the latent variables $\bm{z}$ through a fully connected layer and an
upsampling nearest layer. Because 2D nearest-neighbor upsampling simply
doubles the shape of the input, this layer is combined with a convolutional
layer to generate an image. The decoder has four upsampling and convolutional
layers. Following that, a $(M,4,224,224)$ image with the same shape as the
input was obtained.

IIC for the classification of transient noise uses VAE’s pre-trained encoder
as shown in Figure 2. Because the features of transient noise have already
been learned by the VAE’s pre-trained encoder, the IIC architecture was simply
configured with VAE’s pre-trained encoder and a fully connected layer.
Additionally, overclustering [33] was applied to IIC to improve the
performance of the architecture. Regarding the two inputs
$\bm{x},\bm{x}^{\prime}\in\mathcal{X}$ to IIC in training, we let $\bm{x}$ be
the center-cropped 2D image of transient noise and $\bm{x}^{\prime}$ be the
perturbed 2D image (Section 4.1) in the time direction, whose shapes of images
are the same as $(4,224,224)$, and
$g_{\mathrm{train}}:\mathcal{X}\to\mathcal{Z}$ is the mapping of pre-trained
encoder. Then, IIC trains to maximize the mutual information
$I(\bm{\Phi}(g_{\mathrm{train}}(\bm{x})),\bm{\Phi}(g_{\mathrm{train}}(\bm{x}^{\prime})))$
and overclustering
$I(\bm{\Phi}_{\mathrm{over}}(g_{\mathrm{train}}(\bm{x})),\bm{\Phi}_{\mathrm{over}}(g_{\mathrm{train}}(\bm{x}^{\prime})))$,
where $\bm{\Phi}\in\mathbb{R}^{C}$ and
$\bm{\Phi}_{\mathrm{over}}\in\mathbb{R}^{W}$. We also used several classifiers
(typically set as five) and backpropagated the ensemble average for this
mutual information to reduce the initial value dependence of the neural
networks. The details are shown in Section 4.3.

Following IIC training, although we could obtain a classification result from
one classifier, the softmax outputs of each classifier differ slightly due to
the initial value of the neural networks. Therefore, this result should be
averaged by multiple classifiers to obtain a uniform classification result.
Regarding unsupervised learning, the classification labels are given randomly
in training. For example, the first classifier classifies the data as class
label “0,” while the second classifier classifies the same data as class label
“1.” Therefore we could not calculate an ensemble average of multiple
classification results. As an alternative, we used an approach that constructs
one classification result from the features extracted by multiple
classification results using the spectral clustering [38]. Now, let
$D\in\mathbb{Z}$ be the total number of datasets and $K\in\mathbb{Z}$ be the
number of classifiers, $C\in\mathbb{Z}$ be the estimated number of classes,
and $\mathcal{M}_{D\times C}(\mathbb{R}$) be the classification matrix output
from one classifier. Based on the hypermatrix $\mathcal{H}_{D\times
CK}(\mathbb{R})$, which is composed of the concatenated results of multiple
classifiers, we calculated an affinity matrix $\mathcal{A}_{D\times
D}(\mathbb{R})$ using the Gaussian similarity function
$(A)_{i,j}=\exp{(-||h_{i}-h_{j}||^{2})}$, where $h_{i}$ is a row vector of the
hypermatrix $\mathcal{H}=(h_{1},\dots,h_{i},\dots,h_{D})^{T}$. After applying
spectral clustering to the affinity matrix, we obtained the following new
classification matrix: $\mathcal{M}_{D\times C}(\mathbb{R})$.

##  4 Training Process and Result

We report on the training process and results of our proposed architecture
step by step.

###  4.1 Outline of Dataset

The target dataset is the Gravity Spy dataset of LIGO O1 [20] [21] which
consists of 8535 transient noise images with 22 different labels (e.g.,
“Blip,” “Power_Line,” and “Koi_Fish”). All transient noise in the dataset has
been selected with the SNR $\geq 7.5$ by Omicron software [24]. Each transient
noise image is represented as a time-frequency spectrogram 2D image. One
transient noise is recorded in four time durations: 0.5, 1.0, 2.0, and 4.0 s.
More details of the labels, the distribution of the dataset, and 2D images of
the transient noise are explained in Figure 1 in ref. [26].

We also applied a preprocess that randomly shifts between $\pm$ 0-24 px from
the center of the image to the transient noise 2D image. The 2D images of the
transient noise in each time duration have the shape of 224 $\times$ 272 px.
The preprocessing randomly shifts the time direction of the image between 0
and 24 px and crops it at 224 $\times$ 224 px (square image). The purpose of
this preprocessing is to allow the proposed architecture to train transient
noise features without relying on small-time shifts. Reducing the size of
input images contributes to saving the training cost. More details of the
preprocessing are shown in Figure 7 in ref. [26].

We used these four original and perturbed 2D images as the input data, as
shown in Figures 1 and 2. More details of the dataset can be found in ref.
[26].

###  4.2 Training of VAE

The objective of VAE training is to maximize the variational lower bound
expressed by Equation (3). Regarding the log-likelihood expressed as the
second term of Equation (3), the original study [31] proposed two equations:
Bernoulli and Gaussian distributions. Moreover, we used the mean squared error
(MSE) as a reconstruction error. The three equations are expressed as

$\mathrm{ln}P_{\bm{\bm{\theta}}}(\bm{x}|\bm{z})=\begin{cases}\sum^{D}_{i}\left[x_{i}\ln
y_{i}+(1-x_{i})\ln(1-y_{i})\right]\ &\text{(Bernoulli distribution)}\\\
\sum^{D}_{i}(x_{i}-y_{i})^{2}&\text{(MSE)}\\\
1/2\ln(2\pi)+\sum^{D}_{i}[\ln|\sigma_{i}|+(x_{i}-\mu_{i})^{2}/(2\sigma^{2}_{i})]&\text{(Gaussian
distribution)}\end{cases},$ (7)

where $y_{i}$ is each pixel of the reconstructed image, $\mu_{i}$ and
$\sigma_{i}$ are the mean and variance with respect to each pixel of $x_{i}$,
respectively. Because the Gaussian distribution requires both $\mu_{i}$ and
$\sigma_{i}$, we doubled the output layer of the decoder in Figure 1.

Generally, deep learning has hyperparameters for optimal training. We
investigated the hyperparameters as shown in Table 1 (top) and conducted an
experiment using our proposed architecture as follows: the initial learning
rate (used Adam [39], which is one of the stochastic gradient descent methods)
is in the range of [$5\times 10^{-7},5\times 10^{-2}$] in increments of one
digit; the minibatch size is in the range of [$32,128$] in increments of
$\approx 32$; the training size is in the range of [60%, 90%] in increments of
10%; the dimensions of the latent variable $\bm{z}$ are $64,128,256,512,$ and
$1024$; VAE has the evaluation phase every five epochs to quantify the
performance of the model, and the size of the evaluation dataset is $(1-$
Training size), respectively. In this study, the case of representative
hyperparameters, as shown in Table 2 was used to explain the training process.

The training curves using the Bernoulli distribution in Table 2 are shown in
Figure 3 (top). Because “Case 1” and “Case 2” of evaluation curves (plotted as
dashed) are close to these training curves (plotted as solid), there is no
overfitting in the training process. Although the training curve of “Case 1”
(drawn in blue) progresses slowly, the training curve of “Case 2” (drawn in
pink) sets a higher learning rate than “Case 1”. It also progresses rapidly
and is stabilized near 100 epochs. Considering the Bernoulli case, an
optimizer with a small learning rate would be the cause of slow progress in
training. The reconstructed images from the decoders of “Case 1” at 300 epochs
and “Case 2” at 100 epochs are shown in Figure 3 (bottom). The “Case 1” image
may not have been reconstructed appropriately. However, the “Case 2” image
seems to extract the feature of the input image appropriately. Therefore,
“Case 2” is considered suitable in the case of the Bernoulli distribution.

The training curves using MSE in Table 2 are shown in Figure 4 (top). The
training curves converge in both cases, with the difference being that the
“Case 2” curve (drawn in green) is close to zero, and the “Case 1” curve
(drawn in red) is smaller than the “Case 2” curve. Therefore, we confirmed
that “Case 2” is more optimal for the hyperparameters than “Case 1” in the MSE
case. The reconstructed images of “Case 1” at 100 epochs and “Case 2” at 100
epochs are shown in Figure 4 (Bottom). The “Case 1” image would only be
roughly reconstructed because it is considered to be caused by insufficient
optimization of the hyperparameters. The “Case 2” reconstructed image seems to
be close to the input image, but this image is blurry. This is the reason why
the decoder of “Case 2” reconstructs images along with background features.
Therefore, the training curve might be close to zero.

The training curves using the Gaussian distribution are shown in Figure 5
(top). The “Case 1” training curve (drawn in yellow) has a difference between
the training and evaluation curves, and this training is considered
overfitting. The “Case 2” training curve (drawn in black) converges at 150
epochs and the evaluation curve is close to the training curve. “Case 2” is
stable. Therefore, “Case 2” is more appropriate than “Case 1” in the Gaussian
case. The reconstructed images “Case 1” at 50 epochs and “Case 2” at 150
epochs, which are $\mu_{i}$ of the decoder output in Equation (7), is shown in
Figure 5 (Bottom). Both images would be less clear than the input image. The
reconstructed image is averaged because the mean $\mu_{i}$ of the Gaussian
distribution was applied to the output.

We used two NVIDIA GeForce RTX 2080 Ti GPUs, Intel Xeon CPU E5-2637 v4 (eight
cores), and with 125 GB of main memory. The computational cost of each
training process is shown in Table 2. The computational cost increases as the
batch size and training size increase. Bernoulli and MSE cases have almost the
same computational cost because the neural network layers are the same.
However, the computational cost for the Gaussian case is higher than that for
the Bernoulli and MSE cases. The reason for the higher computational cost in
the Gaussian case is that the decoder optimizes the neural network through the
mean and variance of the output, which increases the neural network layers.

Table 1:  Hyperparameters on training of VAE (top) and IIC (bottom).  | Hyperparameter name | Range | Increment value
---|---|---|---
Training of VAE | Initial learning rate | $[5\times 10^{-7},5\times 10^{-2}]$ | $10^{-1}$
| Mini-batch size | $[32,128]$ | $\approx 32$
| Training size | $[60\%,90\%]$ | $10\%$
| Dimensions of $\bm{z}$ | $64,128,256,512,1024$ | -
Training of IIC | Initial learning rate | $[5\times 10^{-7},5\times 10^{-2}]$ | $10^{-1}$
| Mini-batch size | $[64,256]$ | $32$
| Number of classes | $[22,100]$ | $2$
| Number of over clustering | $[50,500]$ | $50$
| Classifier number | $3,5,10,20$ | -
Table 2:  Representative hyperparameters for the training process of the proposed architecture and its computational cost. All cases are adopted by Adam optimization [39].  |  | Initial learning rate | Batch size | Training size | Dimension of $\bm{z}$ | Computational cost
---|---|---|---|---|---|---
|  |  |  |  |  | (500 epochs)
Bernoullie | Case1 | $5\times 10^{-5}$ | 128 | 80% | 512 | 5.0 h
Case2 | $5\times 10^{-4}$ | 128 | 80% | 512 | 5.0 h
MSE | Case1 | $5\times 10^{-5}$ | 64 | 60% | 256 | 2.0 h
Case2 | $5\times 10^{-4}$ | 96 | 70% | 512 | 3.5 h
Gaussian | Case1 | $5\times 10^{-4}$ | 32 | 60% | 128 | 1.8 h
Case2 | $5\times 10^{-2}$ | 82 | 80% | 256 | 6.0 h

We also investigated the latent variables, which are outputs from the encoder.
The auxiliary analysis is frequently applied to a training model to understand
the learned features of the dataset. Latent variables are important for the
representation of the dataset. However, because the latent space is generally
the higher dimension, it is difficult to assess its utility. For this reason,
visualization methods are frequently applied by embedding from a high-
dimensional space to a lower-dimensional space. A typical approach is
principal component analysis (PCA), which is a linear dimension reduction
technique based on keeping the covariance of the data. Alternatively, the
t-distributed stochastic neighbor embedding (t-SNE) [40] is a nonlinear
technique that preserves the distance between the higher-dimensional and
lower-dimensional features. PCA leads to a better understanding of the data’s
global structure. Alternately, t-SNE helps in the understanding of the local
structure which is important for comprehending clustering visually. Therefore,
we applied the t-SNE method for the visualization of the latent space. Let
$p_{ij}\to\mathbb{R}$ be the joint distribution of the two latent variables
$\bm{z}_{i},\bm{z}_{j}\in\mathcal{Z}$ and $q_{ij}\to\mathbb{R}$ be the joint
probability of the lower -dimensional variables
$\bm{y}_{i},\bm{y}_{j}\in\mathcal{Y}$, where $\mathcal{Y}$ is the lower
-dimensional space. Suppose $p_{ij}$ follows the Gaussian distribution and
$q_{ij}$ follows t-distribution, respectively, then, the t-SNE objective $d$
is determined using the following equation

$d=\sum_{ij}p_{ij}\log\frac{p_{ij}}{q_{ij}}.$ (8)

Let $\mathcal{Y}\subset\mathbb{R}^{3}$ be the lower-dimensional space, and the
learning parameters of t-SNE be set as below: perplexity, which is related to
the number of nearest neighbors as $40$, the maximum number of iterations for
the optimization as $2000$, and random seed as $10$. We also used the Gravity
Spy labels to visualize how the Gravity Spy dataset is clustered in the latent
space. The t-SNE mapping was applied to each model: Bernoulli “Case 2” at 100
epochs, MSE “Case 2” at 100 epochs, and Gaussian “Case 2” at 150 epochs. The
t-SNE results of the Bernoulli, MSE, and Gaussian cases are shown in Figure 6
(top) , Figure 7 (top), and Figure 8 (top), respectively. Considering the
Bernoulli case, the data in each class appear to be clustered in 3D space, and
each cluster has a spatially expansive structure. However, considering the MSE
case, the data are also clustered, and the difference is that the structure of
each cluster seems to be smaller than that of the Bernoulli case. Similar
aspects to the MSE were found in the Gaussian case.

Moreover, the t-SNE results are quantitatively examined using the silhouette
coefficient. Let $\mathcal{C}_{\mathrm{in}}\subset\mathcal{Y}$ be the intra-
cluster dataset and $\mathcal{C}_{\mathrm{near}}\subset\mathcal{Y}$ be the
nearest-cluster dataset, where
$\mathcal{C}_{\mathrm{in}}\cap\mathcal{C}_{\mathrm{near}}=\emptyset$. The mean
intra-cluster distance of the $i$th data is defined as follows:
$a^{(i)}=1/(|\mathcal{C}_{\mathrm{in}}|-1)\sum_{\bm{y}^{(j)}\in\mathcal{C}_{\mathrm{in}}}||\bm{y}^{(i)}-\bm{y}^{(j)}||$,
and the mean nearest-cluster distance of the $i$th data is defined as follows:
$b^{(i)}=1/|\mathcal{C}_{\mathrm{near}}|\sum_{\bm{y}^{(j)}\in\mathcal{C}_{\mathrm{near}}}||\bm{y}^{(i)}-\bm{y}^{(j)}||$.
The silhouette coefficient $s^{(i)}$ is defined as

$s^{(i)}=\frac{b^{(i)}-a^{(i)}}{\max{(b^{(i)},a^{(i)})}}.$ (9)

Note that the silhouette coefficient is a mapping $\mathcal{Y}\to[-1,1]$ and
it approaches $1$ when the clustering is condensing and approaches $0$ when
the number of clusters is not appropriate. The negative value means that these
data overlapped with each cluster.

The silhouette coefficients for the Bernoulli, MSE, and Gaussian cases are
shown in Figure 6 (bottom), Figure 7 (bottom), and Figure 8 (bottom),
respectively. The average values (drawn in red) for all classes are $0.22$ in
the Bernoulli case, $0.19$ in the MSE case, and $0.19$ in the Gaussian case.
The average silhouette coefficient in the Bernoulli case is greater than that
in the other two cases. The MSE and Gaussian cases overlapped with other
classes because their clustering was more condensed than the Bernoulli case.
As a result, their silhouette coefficients decreased.

We summarize the results of VAE’s training process. The training optimization
was conducted in Equation (7) using three types of the reconstruction error.
The Bernoulli “Case 2” decoder at 100 epochs was used to reconstruct the
features of transient noise. We also confirmed the t-SNE results and
silhouette coefficients. As the results show, it was also suggested that the
Bernoulli “Case 2” encoder at 100 epochs was better for transient noise
classification. Therefore, we used the Bernoulli “Case 2” encoder at 100
epochs as a pre-trained encoder for IIC.

###  4.3 Training of IIC

Following VAE training, IIC uses a pre-trained encoder to classify the
transient noise 2D image. The objective of IIC training is to maximize the
mutual information expressed by Equation (6). The maximum mutual information
expressed by Equation (5) increases monotonically with the number of classes.
We divided Equation (6) by Equation (5), and obtained the normalized mutual
information $I(\bm{\Phi}(\bm{x}),\bm{\Phi}(\bm{x}^{\prime}))/\ln C$ for the
number of classes $C$. Our proposed architecture maximizes the normalized
mutual information in the training process. This normalized expression allows
for mutual information comparisons with the different numbers of classes.

The hyperparameters for IIC are also investigated as shown in Table 1 (bottom)
and experimented using our proposed architecture as follows: the initial
learning rate is in the range of [$5\times 10^{-7},5\times 10^{-2}$] in
increments of one digit (same as VAE training); the minibatch size is in the
range of [$64,256$] in increments of $32$; the number of classes is in the
range of [$22,100$] in increments of $2$; the number of over clustering is in
the range of [$50,500$] in increments of $50$; the classifier number is one of
$3,5,10,$ and $20$.

Regarding the initial learning rate, when other hyperparameters are fixed, the
normalized mutual information is not affected unless the value is far away,
for example, $5\times 10^{-7}$ and $5\times 10^{-2}$. Similar aspects to the
number of overclustering, the classifier number, and the minibatch size were
found in the training process. Therefore, in this study, we set the initial
learning rate to $5\times 10^{-4}$, the number of classifiers to five, the
number of over clustering to $250$, and the minibatch size to $128$. We
investigated the range of [$22,100$] to determine the number of classes that
affect the normalized mutual information. Because the previous studies [22]
[27] [28] presented new classes in addition to the 22 classes of the same
dataset, the representative training curves for $C=30,40,50,$ and $100$ are
shown in Figure 9. Note that because the training curve was oscillating and
covered by other training curves, we applied the exponential moving average to
these training curves, and the average span was set to $10$ epochs in Figure
9. All the cases converge and stabilize near 100 epochs, as shown in Figure 9.
The training curves of over clustering were found to be independent of the
number of classes $C$, and the normalized mutual information was high at
$C=30$ and $C=40$. Therefore, from Figure 9, the number of classes $C$ is
considered to be suitable between $30$ and $40$ in our proposed architecture.

###  4.4 Correspondence to Result of Supervised Learning (Gravity Spy labels)
and Summary of Obtained Result in Ref. [26]

After the IIC training discussed in Section 4.3, we investigated the
correspondence between the results of supervised learning (Gravity Spy labels)
and our unsupervised learning. Although the details can be found in ref. [26],
we briefly review the results. We used the accuracy in Equation (1) of ref.
[26] defined as follows,
$\sum_{i}^{C}\max(\bm{v}^{(i)})/|\sum_{j}^{C^{\prime}}{v}^{(i)}|$, where $C$
is the number of classes with unsupervised learning, $C^{\prime}=22$ is the
number of classes with supervised learning (Gravity Spy labels), and
$\bm{v}^{(i)}$ is the $i$th column in the confusion matrix (e.g., in Figure 4
of ref. [26]). By calculating the accuracy as a criterion, we determined the
number of classes in unsupervised learning. Furthermore, we used the spectral
clustering discussed in Section 3 to compress the multiple results of
classification into one result. We investigated the number of classes between
$30$ and $40$, which yields high values for the normalized mutual information
in Section 4.3. The results for the representative numbers $C=30,36,$ and $40$
are shown in Figure 10. The dashed line represents the average accuracy from
each classifier, which was set to five classifiers, and the solid line
represents the accuracy using the spectral clustering for the results of five
classifiers. In all cases, spectral clustering outperformed the average of the
five classifiers, with the highest accuracy of $90.9\%$ achieved with the
number of classes $C=36$ at $200$ epochs. Note that the Gravity Spy project
[20] using supervised learning achieved $97.1\%$ accuracy on the testing data
using the same dataset used here. In terms of accuracy, the proposed
architecture can classify with performance close to supervised learning, and
also has the advantage that unsupervised learning does not require the data
annotations. The proposed architecture shows the results of supervised
learning with a high level of consistency for classification. For example, the
data of “1080Lines” were classified into one class for both supervised and
unsupervised learning. Furthermore, our architecture indicates the existence
of the unrevealed classes from the Gravity Spy dataset. For example, the
“Blip” data that were classified as one class in supervised learning would be
separated into more detailed subclasses in unsupervised learning.

We confirmed the consistency between the label annotated by the Gravity Spy
project and the class provided by our proposed unsupervised-learning
architecture and provided the potential for the existence of the unrevealed
classes. More details on the result and discussion of unsupervised
classification are shown in ref. [26].

##  5 Summary

Transient noise appearing in the data from gravitational-wave detectors
frequently causes problems. Because transient noise is considered to be
associated with the environment and instrument, classifying it may help us
understand its origin and improve the detector’s performance. In our previous
study [26], an architecture of classifying the transient noise using a time-
frequency 2D image (spectrogram), which uses the unsupervised deep learning
combined with VAE and IIC was proposed. The training process of unsupervised
learning is essential. In this study, using the Gravity Spy dataset, we
reported on how the training process of unsupervised-learning architecture of
ref. [26] was conducted. We used three types of objective functions in the
training process for feature learning of transient noise using VAE. We
investigated the three types of training curves, reconstructed images from the
decoders, t-SNE mapping of the latent variables, and silhouette coefficient of
t-SNE mapping. The normalized objective function was used in the training
process using IIC for transient noise classification, and the results were
compared between the different classes to investigate the appropriate number
of classes. We also confirmed that the classification result compressed from
the multiple unsupervised classifications approaches the accuracy in
supervised learning.

A more detailed discussion of the results for the classification of transient
noise using our proposed unsupervised architecture can be found in ref. [26].

We applied the unsupervised classification to the Gravity Spy dataset of LIGO
O1 as a first step. In future work, we will apply our architecture to the
recent observation run (O2 and O3) dataset. Furthermore, we will use our
unsupervised classification in KAGRA to develop a transient noise system.
Additionally, we will extend our architecture to self-supervised learning [41]
to improve classification accuracy, in which the architecture generates pseudo
labels for a given dataset and re-trains it.

Acknowledgements

The authors grateful to the members of the Gravity Spy project for
enlightening discussions. This study was supported in part by the Inter-
University Research Program of the Institute for Cosmic Ray Research,
University of Tokyo, Japan. It was also supported in part by the Japan Society
for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research on
Innovative Areas, Grant No. 24103005 [JP17H06358, JP17H06361, and JP20H04731],
by JSPS Core-to-Core Program A, Advanced Research Networks, and by JSPS
KAKENHI [Grant No. 19H0190 (Y.I. and H.T.) and Nos. 19K14636 and 21H05599 (Y.
Shikano.)], by JST, PRESTO [Grant No. JPMJPR20M4 (Y. Shikano.) ].

Conflict of Interest

The authors declare no conflict of interest.

## References

  * [1] LIGO Scientific Collaboration,  _Classical and Quantum Gravity_ 2015, _32_ 074001\.
  * [2] LIGO Scientific Collaboration and Virgo Collaboration,  _Phys. Rev. Lett._ 2016, _116_ 131103\.
  * [3] Virgo Collaboration,  _Classical and Quantum Gravity_ 2015, _32_ 024001\.
  * [4] LIGO Scientific Collaboration and Virgo Collaboration,  _Phys. Rev. X_ 2019, _9_ 031040\.
  * [5] LIGO Scientific Collaboration and Virgo Collaboration,  _Phys. Rev. X_ 2021, _11_ 021053\.
  * [6] LIGO Scientific Collaboration and Virgo Collaboration,  _arXiv preprint arXiv:2108.01045_ 2021.
  * [7] LIGO Scientific Collaboration and Virgo Collaboration and KAGRA Collaboration,  _arXiv preprint arXiv:2111.03606_ 2021.
  * [8] KAGRA Collaboration,  _Nature Astronomy_ 2019, _3_ 35–40.
  * [9] KAGRA Collaboration,  _Progress of Theoretical and Experimental Physics_ 2021, _2021_ , 5 05A101.
  * [10] KAGRA Collaboration,  _Progress of Theoretical and Experimental Physics_ 2021, _2021_ , 5 05A103.
  * [11] KAGRA Collaboration,  _Progress of Theoretical and Experimental Physics_ 2021, _2021_ , 5 05A102.
  * [12] KAGRA Collaboration,  _arXiv preprint arXiv:2203.07011_ 2022.
  * [13] LIGO Scientific Collaboration and Virgo Collaboration and KAGRA Collaboration,  _Progress of Theoretical and Experimental Physics_ 2022, _2022_ , 6 063F01.
  * [14] P. Nguyen, _et al._ ,  _Classical and Quantum Gravity_ 2021, _38_ , 14 145001\.
  * [15] D. Davis, _et al._ ,  _Classical and Quantum Gravity_ 2021, _38_ , 13 135014\.
  * [16] R. Biswas, L. Blackburn, J. Cao, R. Essick, K. A. Hodge, E. Katsavounidis, K. Kim, Y.-M. Kim, E.-O. Le Bigot, C.-H. Lee, J. J. Oh, S. H. Oh, E. J. Son, Y. Tao, R. Vaulin, X. Wang,  _Phys. Rev. D_ 2013, _88_ 062003\.
  * [17] J. Powell, D. Trifirò, E. Cuoco, I. S. Heng, M. Cavaglià,  _Classical and Quantum Gravity_ 2015, _32_ , 21 215012\.
  * [18] N. Mukund, S. Abraham, S. Kandhasamy, S. Mitra, N. S. Philip,  _Phys. Rev. D_ 2017, _95_ 104059\.
  * [19] LIGO Scientific Collaboration,  _Classical and Quantum Gravity_ 2021, _38_ , 2 025016\.
  * [20] M. Zevin, S. Coughlin, S. Bahaadini, E. Besler, N. Rohani, S. Allen, M. Cabero, K. Crowston, A. K. Katsaggelos, S. L. Larson, T. K. Lee, C. Lintott, T. B. Littenberg, A. Lundgren, C. Østerlund, J. R. Smith, L. Trouille, V. Kalogera,  _Classical and Quantum Gravity_ 2017, _34_ , 6 064003\.
  * [21] S. Bahaadini, V. Noroozi, N. Rohani, S. Coughlin, M. Zevin, J. Smith, V. Kalogera, A. Katsaggelos,  _Information Sciences_ 2018, _444_ 172\.
  * [22] S. Soni, C. P. L. Berry, S. B. Coughlin, M. Harandi, C. B. Jackson, K. Crowston, C. Østerlund, O. Patane, A. K. Katsaggelos, L. Trouille, V.-G. Baranowski, W. F. Domainko, K. Kaminski, M. A. L. Rodriguez, U. Marciniak, P. Nauta, G. Niklasch, R. R. Rote, B. Téglás, C. Unsworth, C. Zhang,  _Classical and Quantum Gravity_ 2021, _38_ , 19 195016\.
  * [23] S. Bahaadini, N. Rohani, S. Coughlin, M. Zevin, V. Kalogera, A. K. Katsaggelos,  In _2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, IEEE, New Orleans, Louisiana, USA, 2017 2931–2935.
  * [24] F. Robinet, N. Arnaud, N. Leroy, A. Lundgren, D. Macleod, J. McIver,  _SoftwareX_ 2020, _12_ 100620\.
  * [25] S. Chatterji, L. Blackburn, G. Martin, E. Katsavounidis,  _Classical and Quantum Gravity_ 2004, _21_ S1809.
  * [26] Y. Sakai, Y. Itoh, P. Jung, K. Kokeyama, C. Kozakai, K. T. Nakahira, S. Oshino, Y. Shikano, H. Takahashi, T. Uchiyama, G. Ueshima, T. Washimi, T. Yamamoto, T. Yokozawa,  _Scientific reports_ 2022, _12_ , 1 1.
  * [27] D. George, H. Shen, E. A. Huerta,  _Physical Review D_ 2018, _97_ , 10 101501.
  * [28] S. Bahaadini, N. Rohani, A. K. Katsaggelos, V. Noroozi, S. Coughlin, M. Zevin,  In _2018 25th IEEE International Conference on Image Processing (ICIP)_. IEEE, Athens, Greece, 2018 748–752.
  * [29] S. Bini,  Unsupervised classification of short transient noise to improve gravitational wave detection,  https://etd.adm.unipi.it/t/etd-08302020-184201/ (accessed: 23 March 2022).
  * [30] Z. Ramezani, A. Pourdarvish,  _Physica A: Statistical Mechanics and its Applications_ 2021, _561_ 125273\.
  * [31] D. P. Kingma, M. Welling,  _arXiv preprint arXiv:1312.6114_ 2013.
  * [32] D. P. Kingma, M. Welling,  _Foundations and Trends in Machine Learning_ 2019, _12_ 307\.
  * [33] X. Ji, A. Vedaldi, J. Henriques,  In _Proceedings of the IEEE International Conference on Computer Vision_ , volume 2019-October. IEEE, Seoul, Korea,  ISSN 15505499, 2019 9865–9874.
  * [34] D. Choi, C. J. Shallue, Z. Nado, J. Lee, C. J. Maddison, G. E. Dahl,  _arXiv preprint arXiv:1910.05446_ 2019.
  * [35] A. C. Wilson, R. Roelofs, M. Stern, N. Srebro, B. Recht,  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , NIPS’17. Curran Associates Inc., Red Hook, NY, USA,  ISBN 9781510860964, 2017 4151–4161.
  * [36] G. Zhong, L.-N. Wang, X. Ling, J. Dong,  _The Journal of Finance and Data Science_ 2016, _2_ 265\.
  * [37] Y. Bengio, A. Courville, P. Vincent,  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 2013, _35_ 1798\.
  * [38] U. von Luxburg,  _Statistics and computing_ 2007, _17_ , 4 395.
  * [39] D. P. Kingma, J. Ba,  _arXiv preprint arXiv:1412.6980_ 2014.
  * [40] S. T. Roweis, L. K. Saul,  _Science_ 2000, _290_ , 5500 2323.
  * [41] D.-H. Lee,  In _Workshop on challenges in representation learning, ICML_ , volume 3. 2013 896\.

Figure 1:  Schematic view of the proposed VAE architecture for training the
latent variables from input images of transient noise. The encoder part is
from the input layer (at the left end) to the latent variables $\bm{z}$ (at
the center), and the decoder part is from $\bm{z}$ to the output layer (at the
right end). The blue block represents batch normalization, whereas the red
block represents the ReLU activation function. A solid line in
reparameterization-trick blocks simply means splitting the component in half,
and lines with an arrow mean the sampling from the mean $\bm{\mu}$ and the
variance $\bm{\sigma}$.  Figure 2:  Schematic view of the proposed IIC
architecture for image classification of transient noise. $C\in\mathbb{Z}$ is
the estimated number of classes of the transient noise and $W\in\mathbb{Z}$ is
the number of classes for overclustering, where $C<W$. IIC uses the VAE’s pre-
trained encoder and classifies transient noise from the latent variables
$\bm{z}$, and the softmax (also called normalized exponential function) layer
outputs the probabilities of classification of transient noise in the range of
$[0,1]$. The result of the mutual information, including the clustering of $C$
and the overclustering of $W$ are added (at the right end), and then IIC
trains to maximize this mutual information. The two input images to IIC are
the center-cropped image of transient noise and its perturbed image.  Figure
3:  VAE training curves using the Bernoulli distribution in Table 2 (Top). The
decoder’s output used the model at the specific epoch (Bottom). An input image
with the Gravity Spy label of “Koi_Fish” (bottom left). The output image by
“Case 1” at 300 epochs does not appropriately reconstruct the input image
(bottom center). The output image by “Case 2” at 100 epochs appropriately
reconstructs the input image (bottom right).  Figure 4:  Same representation
of Figure 3 but for MSE. The output image by “Case 1” at 100 epochs does not
appropriately reconstruct the input image (bottom center). The output image by
“Case 2” at 100 epochs appropriately reconstructs the input image (bottom
right). The training curve of “Case 1” is worse than “Case 2”. Regarding “Case
2”, the training curve is small, but it reconstructs the transient noise along
with the background. As a result, the reconstructed image seems to be blurred.
Therefore the MSE architecture is not suitable to reconstruct the transient
noise.  Figure 5:  Same representation of Figure 3 but for the Gaussian
distribution. An output image by “Case 1” at 50 epochs does not appropriately
reconstruct the input image (bottom center). An output image by “Case 2” at
150 epochs approximately reconstructs the input image (bottom right).

Figure 6:  Colors represents 22 types of Gravity Spy label. The t-SNE mapping
with Bernoulli “Case 2” at 100 epochs (Top). The silhouette coefficient of the
above t-SNE results (Bottom). A dashed line drawn in black represents the
average silhouette coefficient of each label, and the dashed red line
represents an average for all labels.

Figure 7:  Same representation of Figure 6 but for the MSE “Case 2” at 100
epochs.

Figure 8:  Same representation of Figure 6 but for the Gaussian “Case 2” at
150 epochs.  Figure 9:  Training curves of IIC. The solid lines represent the
average mutual information for each number of classes $C$. The dashed lines
represent the average mutual information with overclustering corresponding to
the number of classes $C$, and the classes of all overclustering are set to
$W=250$.  Figure 10:  Accuracy of the proposed architecture (unsupervised
learning) using the Gravity Spy label (supervised learning). The colored solid
lines show the results of the cases where the number of classes is $30$, $36$,
and $40$ (denoted by $C$ in the legend) with the spectral clustering of five
classifiers. The dashed line represents the average accuracy of five
classifiers.


# N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking

Taha Aksu$\dagger{\ddagger}$,, Zhengyuan Liu${\ddagger}$, Min-Yen
Kan$\dagger$, Nancy F. Chen${\ddagger}$
$\dagger$ National University of Singapore, Singapore
${\ddagger}$ Institute for Infocomm Research, A*STAR, Singapore
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>,
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

As the creation of task-oriented conversational data is costly, data
augmentation techniques have been proposed to create synthetic data to improve
model performance in new domains. Up to now, these learning-based techniques
(e.g. paraphrasing) still require a moderate amount of data, making
application to low-resource settings infeasible. To tackle this problem, we
introduce an augmentation framework that creates synthetic task-oriented
dialogues, operating with as few as 5 shots. Our framework utilizes belief
state annotations to define dialogue functions of each turn pair. It then
creates templates of pairs through de-lexicalization, where the dialogue
function codifies the allowable incoming and outgoing links of each template.
To generate new dialogues, our framework composes allowable adjacent templates
in a bottom-up manner. We evaluate our framework using TRADE as the base DST
model, observing significant improvements in the fine-tuning scenarios within
a low-resource setting. We conclude that this end-to-end dialogue augmentation
framework can be a practical tool for natural language understanding
performance in emerging task-oriented dialogue domains.

##  1 Introduction

Data augmentation is a method used for increasing the diversity and size of a
dataset by synthesizing new samples through applying transformations to the
original data. This method has proven to be beneficial in image processing
which includes 2D transformationsKrizhevsky et al. (2012); Cubuk et al.
(2019); Zagoruyko and Komodakis (2017). Text data, on the other hand, does not
have a continuous space that can be manipulated with such transformations and
unlike for an image, an atomic change in a sentence (e.g. a letter or a word)
can result in a noisy sampleSennrich et al. (2016); Fadaee et al. (2017);
Kobayashi (2018). Thus, it is more challenging to generate rational natural
language phrases through data augmentation.

Figure 1: Scenario with two dialogues from train booking domain. Dialogue
snippets, $S_{A}\&S_{B}$, have the same dialogue function and the new dialogue
created by replacing them and inserting proper slot values is still coherent
end to end.

Most augmentation methods in the NLP field are targeted towards written form
of text (e.g. passages, news articles, etc.), which operate with word or
sentence level mutations in the original text data creating new synthesized
sentences/paragraphsLiu et al. (2020). Although used commonly, these methods
do not fit well with conversational data, which is formed through information
exchange between two or more speakers. This study focuses on the augmentation
of a specific type of conversational data: Task-oriented dialogues.

A task-oriented dialogue is a form of information exchange where the user
tries to inform the agent of their preferences and the agent tries to extract
them. Dialogue state tracking (DST) is a fundamental task for task-oriented
dialogues, which aims to detect these preferences in a given dialogue. For
this task, each pair of utterances in a task-oriented dialogue is annotated
with slot-label & slot-value pairs (e.g. train-destination: Cambridge) and a
belief state. A belief state is a dictionary that shows the final values of
each slot label after the subject utterance.

There have been several attempts to augment conversational data in the
literature. Quan and Xiong (2019) up-sample the data through word or sentence
level modifications, following standard text augmentation techniques in NLP
such as synonym substitution, back-translation, or paraphrasing. Kurata et al.
(2016) perturb embeddings of single utterances and decode similarly functioned
synthetic utterances. Gao et al. (2020) create an end-to-end pipeline which
finds the utterances with similar dialogue functions and train a paraphrasing
model. None of the augmentation techniques introduced above fully exploit the
belief state annotations of task-oriented dialogues which embeds the
underlying linked structure of turn-pairs. These dialogue belief state
annotations guide our approach to an effective augmentation method.

The belief state, simply put, is like a pointer to slots that each turn-pair
discusses. Thus, we claim the dialogue state of a turn-pair combined with
dialogue states before and after it can be used to represent its dialogue
function. And further claim that the coherency of two dialogues should not get
harmed by replacing two of their turn pairs with each other, given that they
share the same dialogue function and necessary changes to slot values are made
to fit each dialogue, cf. Figure 1. Motivated from this hypothesis we then
delexicalize and store each turn-pair with their dialogue function, and
effectively reconstruct new dialogues from scratch.

We evaluate our framework with MultiWOZ, a multi-domain dialogue dataset
spanned over 7 different tasks Budzianowski et al. (2018). Each one of the
10.000 dialogues is annotated with turn belief states, system acts, and turn
slots. We conduct our experiments with TRADE, a SOTA architecture from
MultiWOZ leaderboard, and we use only 5 of the 7 domains following their paper
because the other two domains are limited in data amount. Experiments show
that our framework can significantly increase performance in almost all
domains. Specifically, we improve the slot-accuracy results significantly by
0.11, 0.7, 0.3, 0.2 of the train, restaurant, attraction, and hotel domains
respectively within a few-shot setting. We observe that the only domain which
does not benefit from our framework, the taxi domain, does not even favor more
original data and performs the same given 84 (%1 of full data) and 5 original
dialogues. This exceptional case naturally de-emphasizes the use case of any
augmentation method along with ours.

This work shows a different perspective for text augmentation in the task-
oriented dialogue domain which differently from past studies exploits the
interconnected conversational nature of a dialogue. We believe that it can be
a great tool if facilitated in limited data scenarios for task-oriented
dialogue tasks and ease training data-hungry models in emerging domains.

##  2 Related Work

###  2.1 Dialogue State Tracking

Dialogue state tracking (DST) is a core task of today’s conversational systems
where the challenge is to estimate the user’s preferences at every utterance
of the dialogue. Past models in the field used to cumulatively keep track of
utterances to obtain dialogue states Williams and Young (2007); Thomson and
Young (2010); Wang and Lemon (2013). Lei et al. (2018) introduced Sequicity,
to generate belief spans as an intermediate process and improve the end task.
Zhong et al. (2018) proposed to use a unique module for each slot, which
improves the tracking of unseen slot values. The majority of these systems
relied on an in-domain vocabulary and they are all evaluated on a single
domain. Ramadan et al. (2018) proposed to jointly train the domain and state
tracker using multiple bi-LSTM and allowed the learned parameters to be shared
across domains whereas Rastogi et al. (2017) used a multi-domain approach
using bi-GRU where the dialogue states are defined as distributions over a
candidate set (derived from dialogue history). The base model used in this
paper is proposed by Wu et al. (2019), in their architecture they use a copy
mechanism to overcome out of vocabulary (OOV) words, and they generate
probability distributions over three cases for each slot: none, don’t-care,
ptr.

###  2.2 Few-shot Dialogue State Tracking

Although there are many papers focused on low resource scenarios in the DST
field, the scale of data that these papers define as “low” varies. Despite
this variation, all studies are aimed towards comparable results between low
and rich resource settings. Existing studies show two promising ways for
addressing the low resource setting problem: (1) optimization functions aimed
to exploit smaller data amounts or (2) augmentation of the target data.

####  2.2.1 Few-shot Models and Techniques

Some approaches in the first class of solutions benefit from the recent
transformer trend. One such study finetunes the GPT-2 model and reports n-shot
slot-filling and intent recognition results on SNIPS dataset Madotto et al.
(2020). They achieve promising results compared to baselines with fewer shots,
however, they also report that the max-input length of GPT-2 is a limitation
over the number of max shots. TOD-BERT trains a language model on 9 task-
oriented dialogue datasets and reports results on four downstream tasks in
full and low resource settings Wu et al. (2020). Although powerful these
systems prove to be costly to deploy and use. There is another line of
research that tries to address the problem without using transformers, Span-
ConverRT, re-defining the slot-filling problem as turn-based span extraction
which they claim helps greatly in the few-shot setting Coope et al. (2020).
Huang et al. (2020) use the model agnostic meta-learning (MAML) algorithm for
adapting to new domains and show that it is able to outperform traditional
methods with fewer data. Coach on the other hand separates the slot-filling
task into two by first detecting the slot entities in a sentence followed by
the prediction of its entity type Liu et al. (2020). Their model is able to
achieve better adaptation than existing baselines.

####  2.2.2 Data Augmentation For Few-shot Setting

The second class of research, and this paper, is focused on enriching the data
in the target domain to improve the existing learning systems’ performance.
Quan and Xiong (2019) adopt four techniques for augmentation: synonym
substitution, stop-word deletion, translation and paraphrasing at sentence
level. Kurata et al. (2016) start by pretraining a dialogue encoder-decoder,
they then perturb the dialogue representations in their dataset and decode
them back obtaining synthetic dialogues. Another study by Jalalvand et al.
(2018) train a simple logistic regression model on the small target data
detecting most informative n-grams and then finding samples from an out-of-
domain corpus using these n-grams. One common thing that all these studies
share is that they do not exploit the structure of dialogues and can be
applied to any other text form. PARG on the other hand matches turns of a
task-oriented dialogue by their dialogue state, effectively creating pairs for
paraphrase generation Gao et al. (2020), they then jointly train the
paraphrase generator with the end task outperforming other dialogue
augmentation baselines. They, however, do not model the interaction of a turn-
pair with the next turn-pairs. Thus a paraphrased utterance may turn up noisy,
repeating a slot within the next turn. Moreover, the low-resource setting
defined by PARS is still required to be large enough to train a neural
paraphrase model from scratch, thus limiting its applicability to emerging
domains with a lack of data.

##  3 Method

Our method rises upon a simple hypothesis, which Figure 1 visualizes through
an example: We define the function of a pair of turns in a dialogue by its
slots and interactions with previous and next pairs of turns. For example, in
the figure, there are two turn-pairs $S_{a}$ from dialogue A and $S_{b}$ from
dialogue B. The belief states of turn-pairs that precede both $S_{a}$ and
$S_{b}$ are composed of the same set of slot labels. The same can be argued
for the belief states of turn-pairs following $S_{a}$ and $S_{b}$. Thus their
function in the dialogue is the same. We claim that one can replace these
pairs of turns after changing the values according to the parent dialogue
state and still assume an end-to-end coherent dialogue, like the one in the
figure. Our observations on the MultiWOZ dataset showed that this is largely
true for task-oriented dialogues as the belief state history represents the
ongoing topic, while the slot labels of the next turn give hints about the
system acts.

Based on this hypothesis, our framework runs in a fully automatic, end-to-end
three consecutive steps. First, we create turn-pair templates by
delexicalizing (replacing slot values with their respective slot label e.g.
Cambridge $\rightarrow$ train-destination) them, then store each template with
the previous, current, and next pair’s belief states. At this step, we also
create a dictionary of slot label-value pairs which will become handy in the
last step for filling in the templates. Next, we combine these pairs end-to-
end to create dialogue templates, constrained that two consecutive pairs’
dialogue functions do not break the coherency. We do this combination in a
breadth-first manner effectively finding a tree where each node is a turn-pair
template and every string of nodes from root to leaf is a dialogue template.
In the last step, we create final synthetic dialogues by filling the slot
labels in dialogue templates using the slot label-value dictionaries formed
previously.

###  3.1 Turn-pair Template Generation

Figure 2: Sample Turn-pair template (bottom) and the original dialogue (top)
it is extracted from. The subject template is composed of 5 elements:
Delexicalized agent and user utterances, belief state of current and past
turns in the original dialogue, and slot label & values of current and next
turns in the original dialogue. Figure 3: Dialogue templates in our framework
are generated through adding proper turn-pair templates end-to-end in a tree
structure. This tree eventually covers every probable dialogue template as a
path from root to a leaf node. Figure 4: The last step in our framework,
surface realization, first creates a dictionary of slot label & slot values
from original dialogues. It then populates the templates with every
permutation of possible values of each slot thus effectively creating final
synthetic dialogues.

Figure 2 depicts a sample turn-pair template that our framework generates.
Each turn-pair template in our framework consists of a pair of turns: a system
turn and a user turn. Our templates consist of pairs of turns simply because
consecutive turns (system-user) in the MultiWOZ dataset share the same
dialogue state annotation. Each turn-pair template consists of a delexicalized
pair of turns and its dialogue function formed as the combination of the
previous, current, and next turn belief states.

Our delexicalization, following prior work Hou et al. (2018), consists of
finding the slot values in the turn-pair and replacing them with “[slot-
name]”. MultiWOZ dataset does not provide the indexes for the values thus we
manually find each value by searching in the turn-pair. This brings up several
problems where two slots might have the same value or some categorical values
might not show up in the text (e.g. hotel-internet: { dontcare, yes, no}). We
filter out templates with the same values for different labels and we leave
the values for the categorical labels the same assuming they are not affected
by changes in other values. However, unlike non-categorical ones we are
limited from enriching the values of such slot types in our last step, surface
realization when we fill in our templates.

Each dialogue in the MultiWOZ dataset usually starts with a salutation and
ends with a farewell. To distinguish these pairs, we define two special cases:
(1) If a template’s turn-pair comes from the beginning of a dialogue then we
set its previous belief state to be “null”, (2) if it comes from the ending of
a dialogue then we set its next belief state to be “null”. We use these two
cases later in template generation to generate dialogues that are coherent
from start to end.

###  3.2 Dialogue Template Generation

During the generation, we form each dialogue template by combining a set of
turn-pair templates end-to-end. The combination is done with respect to the
previous belief state and next slot labels of each template. These are
respectively backward and forward limitations for adding a turn pair template
to an ongoing dialogue template. We form our dialogue templates using a tree
structure where each node corresponds to a turn pair template and a string of
nodes from root to leaf is a dialogue template. Our generation process is
shown in Figure 3. We start by defining a root node and setting its belief
state as null. For the initial addition, we ignore the next belief state
condition thus adding every template whose previous belief state is null. This
makes sure that each dialogue has a legitimate beginning such as a greeting.
At each level, we mark every newly added node as an active node. Then after
each level, we iterate through active nodes and expand each node with the set
of templates that they can be followed with. Two conditions need to be met to
add template A to the tail of template B: (1) A’s belief state slots should be
met by B’s next belief state slots and (2) B’s belief state slots should be
met by A’s past belief state slots. We continue adding templates until there
are no active nodes. Eventually, we end up with a tree structure where each
connected node represents a turn-pair. Each path from the root to a leaf node
in this tree is a unique dialogue template. We filter in the ones whose leaf
nodes have null as the next belief state. This ensures that the dialogue
template has a legitimate ending such as a farewell.

###  3.3 Surface Realization

The last step in our method is to fill in the de-lexicalized dialogue
templates. This step uses the slot value dictionaries for each slot label,
which is extracted back in the initial step. Using this dictionary, we fill
each dialogue with every possible combination of slot values thus effectively
sourcing synthetic augmented dialogues in an end-to-end manner, cf. Figure 4.
This final step brings us to the end of our method, returning a set of task-
oriented dialogues trainable on a learning system.

##  4 Experiments

| Hotel | Taxi | Rest. | Att. | Train
---|---|---|---|---|---
#dialogues | 4191 | 2056 | 4688 | 3513 | 4081
#turn / #dial. | 12.3 | 10.8 | 11.11 | 10.6 | 11.4
#value / #slot | 16.7 | 194.7 | 55.14 | 56.3 | 56.8
Slots | price, | leave, | price, | area, | dest.,
type, | dest., | food, | arrive, | arrive,
area, | depart., | area, | name, | depart.,
day, | arrive | day, |  | day,
name, |  | name, |  | people,
stay, |  | time, |  | leave
stars, |  | people |  |
internet, |  |  |  |
park, |  |  |  |
people |  |  |  |
Table 1: MultiWOZ dataset statistics categorized by domain.

Hotel Taxi Restaurant Attraction Train Joint Slot Joint Slot Joint Slot Joint
Slot Joint Slot Base Model trained on other 4 domains 0.12 0.64 0.60 0.73 0.12
0.54 0.18 0.54 0.22 0.49 BM fine tuned with %1 data (8̃0 dialogues) 0.21 0.76
0.61 0.75 0.21 0.77 0.43 0.74 0.61 0.91 5 Shot Augmentation on Target Domain
BM fine-tuned with 5 samples 0.12 0.65 0.59 0.75 0.12 0.58 0.25 0.59 0.25 0.66
BM fine-tuned with augmented samples 0.12 0.67* 0.58 0.75 0.13 0.62* 0.26 0.61
0.31* 0.77* 10 Shot Augmentation on Target Domain BM fine-tuned with 10
samples 0.14 0.68 0.6 0.76 0.13 0.63 0.30 0.63 0.37 0.81 BM fine-tuned with
augmented samples 0.15 0.69 0.6 0.76 0.16* 0.70* 0.32* 0.66* 0.39 0.83

Table 2: Evaluation results for 5 and 10 shot finetuning with TRADE model with
and without augmented data. First row shows the zero shot results, second row
the finetuning with %1 data ( 80 dialogues) for comparison with n-shot
results. Each number in the 5 and 10 shot sections is an average of 10 runs.
For each run the we randomly sample different dialogues from the dataset and
augment them using our framework. Bolded numbers in each section shows the
best performance within that section and the * indicates statistically
significant results with %95 confidence.

###  4.1 Dataset and Evaluation

We conduct experiments on MultiWOZ, a well-known dataset in the DST field.
When compared to its counterparts like WOZ Wen et al. (2017), DSTC2 Henderson
et al. (2014), and Restaurant-8k Coope et al. (2020) MultiWOZ is the richest
combining several domains with a variety of slot labels and values. MultiWOZ
is a multi-domain dialogue dataset that covers 10.000 dialogues between clerks
and tourists at an information center over 7 distinct domains such as
restaurant, taxi, hotel, etc.. Each dialogue is annotated with turn belief
states, system acts, and turn slots. The numbers for training, development,
and testing data samples are respectively 8438, 1000, 1000. Table 1 shows
further details over each domain in the MultiWOZ dataset. We evaluate the
model following the metrics used within the TRADE model Wu et al. (2019): Slot
Accuracy and Joint Accuracy. Slot accuracy stands for the proportion of
correctly predicted slot values whereas joint accuracy for the correctly
predicted turn dialogue states. For a turn dialogue state to be predicted
correctly all of the slot values it includes have to be predicted correctly.

###  4.2 Implementation Details

We use the TRADE Wu et al. (2019) model as a base model to finetune with our
augmented data, which is one of the several SOTA models in the MultiWOZ
leader-board. TRADE model is motivated in solving the unseen slot value
problem by establishing a copy-mechanism that generates dialogue states from
utterances while also exploiting the interrelations of domains through
knowledge transfer. Following Wu et al. (2019) we conduct our experiments on 5
of 7 domains (hotel, taxi, restaurant, attraction, and train).

Because we are interested in the transfer quality between domains using
augmented data, we follow the fine-tuning experiments done in the TRADE paper.
Specifically for each new domain, we train a base model on 4 other domains. We
then randomly sample a small set in the target domain and fine-tune the base
model with the sampled dialogues. And finally, we augment the small sample of
dialogues using our framework and fine-tune the base model on this
synthetically augmented dataset. We, then, compare the results of original and
augmented fine-tuning.

###  4.3 Results

Table 2 shows the results of our experiments. The results for the base model
are from a single run, whereas reported results for finetuned models are an
average of 10 runs. Specifically for each n-shot scenario, we sample 10
n-dialogue sets from the original dataset and create 10 separate augmented
sets using their corresponding original dialogues. We also report the
significance of results with %95 confidence along with averages in the table.
Our framework can sustain the performance of the model in every domain and for
4 out 5 of them, it significantly improves the results in either 5 or 10 shot
scenarios.

The only exception is the taxi domain where the additional augmented data does
not do any significant changes in both scenarios. We believe the reason behind
this is that the dialogues in the taxi domain follow a common pattern which
the model can adapt easily with fewer data. The fact that the performance of
the base model finetuned with %1 of data is already reached by finetuning the
same model within a 5-shot scenario (cf. Table 4.3) supports our claim. Like
any augmentation framework, our system cannot generate better quality
dialogues than the ones existent in the original distribution, thus it seems
unrealistic to get performance improvements through augmentation on the taxi
domain.

###  4.4 Effect of Augmentation Ratio

Figure 5: Effects of augmentation ratio for both 5 and 10 shot scenarios in
every domain. The dashed blue line shows the performance of finetuning with %1
of full data ( 80 dialogues) for comparison.

We have tried our augmentation scheme with several different augmentation
ratios in the 5 and 10 shot cases to inspect if the synthetic data amount
affects the results proportionally. Figure 5 shows the results in all 5
domains. Our framework increases the results steadily compared to base fine-
tuning, and the amount of synthetic data affects the results proportionally in
almost every case except the taxi domain as explained before (cf. Section
4.3). For the few cases where the performance drops with increasing synthetic
data, we verified through statistical testing that the differences are not
significant with %95 confidence.

Still, we think that these slight drops in performance might be happening due
to overfitting to certain template types. Because the number of templates that
can be generated from only 5/10 dialogues is limited, the exposure to the same
type of templates increases as the augmentation ratio goes up. Thus making the
system overfit to those. We believe this could be addressed through filtering
the synthetic dialogues by prioritizing those that the end system shows less
confidence in. We hope to investigate this further in future work.

###  4.5 Fine-grained Error Analysis

####  4.5.1 Slot-type Errors

Error type | Original n-shot | Synthetic n-shot
---|---|---
restaurant-food | 2041 | 1675
restaurant-pricerange | 1210 | 603
restaurant-name | 1133 | 1061
restaurant-area | 853 | 480
restaurant-book day | 743 | 335
restaurant-book people | 740 | 212
restaurant-book time | 1119 | 347
Table 3: The fine-grained errors of original and augmented fine-tuning in
restaurant domain classified by slot types.

Speaker |  Turn
---|---
User |  I am coming to town and looking for a good Spanish restaurant to try near the town center. Do you have any suggestions ?
Agent |  The Spanish restaurant near center is Wagamama. The address is 36 saint Andrews street .
User |  I’ll be needing a table for 4 at 5p.m. this next Monday , can you help ?
Agent |  Here is your reference number 6huyuygs is there anything else I can do for you?
User |  Yes I need to find a train to Cambridge from Kings Lynn.
Agent |  When would you like to travel ?
User |  Tuesday morning at 9 am
Agent |  train tr3447 leaves Kings Lynn at 09:11. Would you like to reserve a ticket ?
User |  Yes please. Make a booking for the same number of people .
Agent |  Okay, the booking was successful , the total fee is 58.8 gbp payable at the station . reference number is : fz6dqhqt. Anything else i can help you with?
User |  That is all i needed today.

Speaker |  Turn
---|---
User |  I am looking for a train to Cambridge that departs after 10:45.
Agent |  What day would you like to travel?
User |  I’ll be travelling on a Wednesday.
Agent |  Where would you be departing from?
User |  I will be leaving Norwich.
Agent |  On what day will you be leaving?
User |  I will be leaving on Wednesday.
Agent |  It is 18.88 pounds per ticket. How many tickets would you like to purchase?
User |  Just 2 tickets please and thank you.
Agent |  Your ticket has been reserved . Total fee is 18.88 gbp, payable at the station. Reference number zm5k0a4q. Can i help you with anything else today?
User |  Can you book Acorn guest house for 7 people starting on Wednesday for 4 nights, please ?
Agent |  Thank you so much for reserving my train ticket. I have the information for Acorn guest house you requested.
|  Would you like me to reserve it for you ?
User |  Yes. I need the reference number.
Agent |  The booking was successful. your reference number is dtkw11yu
User |  That is all I needed today, thank you for your help.

Table 4: Sample synthetic dialogues generated by our framework. (L) Dialogue
in restaurant domain generated by 2 different original dialogues. (R) Dialogue
in train domain generated by 3 different original dialogues.

Apart from performance in evaluation metrics we also analyze the error rates
of our framework in each specific slot type in the restaurant domain and
compare it to the model fine-tuned with original data. Table 3 shows the
results. Our framework consistently reduces error rates in every single slot
type. The drop in errors is least remarkable for the name and food slots, we
believe this is because the challenge in these slots is most largely unknown
vocabulary words. Although our framework enriches the dialogue templates with
several different slot values, it does so using the values from the original
set repeatedly. Thus it ends up less helpful for those slots suffering from
the unknown slot value problem and shows more significant improvements on
slots with arguably more isolated vocabulary (e.g. Book-day: 1, 2, 3 etc. or
price range: cheap, moderate, expensive).

To support the significance of results on fine-grained slot error types, we
use McNemar’s test ($\alpha=0.01$) upon creating the confusion matrix between
our framework and original fine-tuning. The results suggest that synthetic
data fine-tuning shows statistically significant improvements over the
original data fine-tuning, with $p<\alpha$.

####  4.5.2 Sample Synthetic Dialogues

In this section we showcase two synthetic dialogues generated with our
framework, cf. Table 4. These dialogues are generated by merging templates
from 2 and 3 different dialogues respectively. The first half of the dialogue
on the left includes slots from the restaurant domain, while the second half
has slots from the train domain. The dialogue on the right combines slots from
domains: train (from two different dialogues) and hotel (from another third
dialogue). Although both dialogues seem coherent in shape the sample on the
right has a redundancy where the system request the day information after the
user already stated it. We found out that this is because of a missing
annotation where the train-day slot in the belief state of the third turn is
missing. These kinds of annotations unavoidable but negligible as well because
this in some ways recaptures a misunderstanding by the agent specific to
dialogues and since both slots are filled with the same values the coherence
of the dialogue is not harmed in a great deal.

##  5 Conclusion

The studies on text augmentation vastly focus on written text passages
implementing sentence or word level transformations to augment original data
by generating synthetic samples. None of the studies published so far has
attempted to use the embedded information in dialogue states of a task-
oriented dialogue which gives important clues about the dialogue function of a
turn-pair through its relations with previous and next turn-pairs. In this
study, we develop a framework that exploits this idea by making templates
through delexicalazing every turn-pair in a dataset and storing them with
pointers to previous and next belief states in the dialogue. Using these
templates we effectively create augmented dialogue datasets from scratch.

Our experiments with the SOTA TRADE model shows increased performances on the
DST task for 4 out 5 domains in the MultiWOZ dataset within a low-resource
training scheme. The only domain not improved, taxi, does not favor even more
original data performing equally for 84 (%1 of full data) and 5 dialogues from
the original dataset. This exceptional case breaks the motivation of using an
augmentation framework. However, considering the significant improvements on
other 4 domains we claim that our data augmentation framework consistently
improves the performance over the DST task for task-oriented dialogues and it
can open doors for many task-oriented dialogue tasks in limited data
scenarios, easing the training of data-hungry models in emerging domains.

## References

  * Budzianowski et al. [2018] Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić.  MultiWOZ – A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling.  9 2018.  URL http://arxiv.org/abs/1810.00278.
  * Coope et al. [2020] Sam Coope, Tyler Farghly, Daniela Gerz, Ivan Vulić, and Matthew Henderson.  Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations.  2020\.  URL http://arxiv.org/abs/2005.08866.
  * Cubuk et al. [2019] Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le.  Randaugment: Practical data augmentation with no separate search.  _CoRR_ , abs/1909.13719, 2019.  URL http://arxiv.org/abs/1909.13719.
  * Fadaee et al. [2017] Marzieh Fadaee, Arianna Bisazza, and Christof Monz.  Data augmentation for low-resource neural machine translation.  _CoRR_ , abs/1705.00440, 2017.  URL http://arxiv.org/abs/1705.00440.
  * Gao et al. [2020] Silin Gao, Yichi Zhang, Zhijian Ou, and Zhou Yu.  Paraphrase Augmented Task-Oriented Dialog Generation.  pages 639–649, 2020.  URL http://arxiv.org/abs/2004.07462.
  * Henderson et al. [2014] Matthew Henderson, Blaise Thomson, and Jason Williams.  The Second Dialog State Tracking Challenge.  Technical report, 2014.
  * Hou et al. [2018] Yutai Hou, Yijia Liu, Wanxiang Che, and Ting Liu.  Sequence-to-Sequence Data Augmentation for Dialogue Language Understanding.  Technical report, 2018.  URL https://github.com/AtmaHou/Seq2SeqDataAugmentationForLU.
  * Huang et al. [2020] Yi Huang, Junlan Feng, Min Hu, Xiaoting Wu, Xiaoyu Du, and Shuo Ma.  Meta-Reinforced Multi-Domain State Generator for Dialogue Systems.  _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7109–7118, 2020.  URL https://www.aclweb.org/anthology/2020.acl-main.636.
  * Jalalvand et al. [2018] Shahab Jalalvand, Andrej Ljolje, and Srinivas Bangalore.  AUTOMATIC DATA EXPANSION FOR CUSTOMER-CARE SPOKEN LANGUAGE UNDERSTANDING A PREPRINT.  Technical report, 2018.
  * Kobayashi [2018] Sosuke Kobayashi.  Contextual augmentation: Data augmentation by words with paradigmatic relations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 452–457, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.  doi: 10.18653/v1/N18-2072.  URL https://www.aclweb.org/anthology/N18-2072.
  * Krizhevsky et al. [2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.  Imagenet classification with deep convolutional neural networks.  In _Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1_ , NIPS’12, page 1097–1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
  * Kurata et al. [2016] Gakuto Kurata, Bing Xiang, and Bowen Zhou.  Labeled Data Generation with Encoder-decoder LSTM for Semantic Slot Filling.  2016\.  doi: 10.21437/Interspeech.2016-727.  URL http://dx.doi.org/10.21437/Interspeech.2016-727.
  * Lei et al. [2018] Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, and Dawei Yin.  Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , volume 1, pages 1437–1447, Stroudsburg, PA, USA, 2018. Association for Computational Linguistics.  doi: 10.18653/v1/P18-1133.  URL http://aclweb.org/anthology/P18-1133.
  * Liu et al. [2020] P. Liu, X. Wang, C. Xiang, and W. Meng.  A survey of text data augmentation.  In _2020 International Conference on Computer Communication and Network Security (CCNS)_ , pages 191–195, 2020.  doi: 10.1109/CCNS50731.2020.00049.
  * Liu et al. [2020] Zihan Liu, Genta Indra Winata, Peng Xu, and Pascale Fung.  Coach: A coarse-to-fine approach for cross-domain slot filling, 2020.
  * Madotto et al. [2020] Andrea Madotto, Zihan Liu, Zhaojiang Lin, and Pascale Fung.  Language models as few-shot learner for task-oriented dialogue systems, 2020.
  * Quan and Xiong [2019] Jun Quan and Deyi Xiong.  Effective Data Augmentation Approaches to End-to-End Task-Oriented Dialogue.  Technical report, 2019.  URL https://github.com/vsuthichai/paraphraser.
  * Ramadan et al. [2018] Osman Ramadan, Paweł Budzianowski, and Milica Gašić.  Large-Scale Multi-Domain Belief Tracking with Knowledge Sharing.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , volume 2, pages 432–437, Stroudsburg, PA, USA, 2018. Association for Computational Linguistics.  doi: 10.18653/v1/P18-2069.  URL http://aclweb.org/anthology/P18-2069.
  * Rastogi et al. [2017] Abhinav Rastogi, Dilek Hakkani-Tur, and Larry Heck.  Scalable Multi-Domain Dialogue State Tracking.  _2017 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2017 - Proceedings_ , 2018-January:561–568, 12 2017.  URL http://arxiv.org/abs/1712.10224.
  * Sennrich et al. [2016] Rico Sennrich, Barry Haddow, and Alexandra Birch.  Improving neural machine translation models with monolingual data.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 86–96, Berlin, Germany, August 2016. Association for Computational Linguistics.  doi: 10.18653/v1/P16-1009.  URL https://www.aclweb.org/anthology/P16-1009.
  * Thomson and Young [2010] Blaise Thomson and Steve Young.  Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems.  _Computer Speech and Language_ , 24(4):562–588, 10 2010.  ISSN 08852308.  doi: 10.1016/j.csl.2009.07.003.
  * Wang and Lemon [2013] Zhuoran Wang and Oliver Lemon.  A Simple and Generic Belief Tracking Mechanism for the Dialog State Tracking Challenge: On the believability of observed information.  Technical report, 2013.  URL https://www.aclweb.org/anthology/W13-4067.
  * Wen et al. [2017] Tsung-Hsien Wen, David Vandyke, Nikola Mrkši´cmrkši´c, Milica Gaši´cgaši´c, Lina M Rojas-Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young.  A Network-based End-to-End Trainable Task-oriented Dialogue System.  Technical report, 2017.
  * Williams and Young [2007] Jason D. Williams and Steve Young.  Partially observable Markov decision processes for spoken dialog systems.  _Computer Speech and Language_ , 21(2):393–422, 4 2007.  ISSN 08852308.  doi: 10.1016/j.csl.2006.06.008.
  * Wu et al. [2019] Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung.  Transferable multi-domain state generator for task-oriented dialogue systems, 2019.
  * Wu et al. [2020] Chien-Sheng Wu, Steven Hoi, Richard Socher, and Caiming Xiong.  TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue.  Technical report, 2020.
  * Zagoruyko and Komodakis [2017] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks, 2017.
  * Zhong et al. [2018] Victor Zhong, Caiming Xiong, and Richard Socher.  Global-Locally Self-Attentive Encoder for Dialogue State Tracking.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1458–1467, Stroudsburg, PA, USA, 2018. Association for Computational Linguistics.  doi: 10.18653/v1/P18-1135.  URL http://aclweb.org/anthology/P18-1135.