# Interpretable Time-series Representation Learning
With Multi-Level Disentanglement

Yuening Li1, Zhengzhang Chen2, Daochen Zha1, Mengnan Du1, Denghui Zhang3,
Haifeng Chen2, Xia Hu1 1Department of Computer Science and Engineering, Texas
A&M University2NEC Labs America3Rutgers University<EMAIL_ADDRESS>
###### Abstract.

Time-series representation learning is a fundamental task for time-series
analysis. While significant progress has been made to achieve accurate
representations for downstream applications, the learned representations often
lack interpretability and do not expose semantic meanings. Different from
previous efforts on the entangled feature space, we aim to extract the
semantic-rich temporal correlations in the latent interpretable factorized
representation of the data. Motivated by the success of disentangled
representation learning in computer vision, we study the possibility of
learning semantic-rich time-series representations, which remains unexplored
due to three main challenges: 1) sequential data structure introduces complex
temporal correlations and makes the latent representations hard to interpret,
2) sequential models suffer from KL vanishing problem, and 3) interpretable
semantic concepts for time-series often rely on multiple factors instead of
individuals. To bridge the gap, we propose Disentangle Time-Series (DTS), a
novel disentanglement enhancement framework for sequential data. Specifically,
to generate hierarchical semantic concepts as the interpretable and
disentangled representation of time-series, DTS introduces multi-level
disentanglement strategies by covering both individual latent factors and
group semantic segments. We further propose an evidence lower bound (ELBO)
decomposition approach to balance the preference between correct inference and
fitting data distribution problems. We theoretically show how to alleviate the
KL vanishing problem: DTS introduces a mutual information maximization term,
while preserving a heavier penalty on the total correlation and the dimension-
wise KL to keep the disentanglement property. Experimental results on various
real-world datasets demonstrate that the representations learned by DTS
achieve superior performance in downstream applications, with high
interpretability of semantic concepts.

##  1\. Introduction

(a) Semantic factors for images, where a semantic factor controls the eye
glasses of a human facial image (shen2020interfacegan, ).

(b) Semantic factors of time-series, where a semantic factor controls the
sequential trend of a time-series.

Figure 1. Two traversal plot examples of disentanglement.

Time-series data are ubiquitous in various domains, such as self-driving
vehicles, financial transactions, and computer-aided medical diagnosis
(blackburn1960electrocardiogram, ; lai2020tods, ; zha2020meta, ). As a
fundamental task of time-series analysis, unsupervised time-series
representation learning aims to extract low-dimensional representations from
complex raw time-series without human supervision. These representations can
then be used to benefit many downstream tasks, such as classification
(liu2019single, ; huang2019graph, ), clustering (du2019towards, ; lai2020dual,
; zhou2020towards, ), and anomaly detection (li2019specae, ; li2020pyodds, ;
li2019deep, ; li2020autood, ). Recently, deep generative models have shown
great representation ability in modeling complex underlying distributions of
time-series data. The most representative examples include LSTM-VAE and its
different variants (semeniuta2017hybrid, ; fortuin2018som, ; fortuin2020gp, ).

While recent techniques have been proposed to learn effective representations
of time-series for downstream applications, the learned representations often
lack the interpretability to expose tangible semantic meanings (yang2019xdeep,
). In many cases, especially in high-stake domains, an interpretable
representation is critical for diagnosis or decision making. For example,
learning interpretable and semantic-rich representations is useful for
decomposing the ECG into cardiac cycles with recognizable phases as
independent factors. Furthermore, extracting and analyzing common sequential
patterns ($i.e.$, normal sinus rhythms) from massive ECG records may assist
clinicians to better understand the irregular symptoms (like sinus
tachycardia, sinus bradycardia, and sinus arrhythmia). In contrast, diagnostic
processes without transparency or accurate explanations may lead to suboptimal
or even risky treatments.

To enable semantically meaningful representations, researchers in computer
vision have been devoted to learning disentangled representations, which
decompose the representations into subspaces and encode them as separate
dimensions (guo2020interpretable, ). A disentangled representation can be
defined as one where single latent units are sensitive to changes in a single
latent factor while being relatively invariant to changes in other factors;
that is, different dimensions in the latent space are probabilistically
independent. Fig. 1 (a) shows an example, where a semantic factor controls the
eyeglasses of a human facial image. Learning factors of variations in the
images enables the emergence of semantic meanings in the underlying
distribution (shen2020interfacegan, ). Motivated by the success of
disentanglement in the image domain, in this work, we explore disentangled
representations for time-series data. Fig. 1 (b) shows an example of how the
learned semantic factor can control the shape of ECG time-series. Medically,
inverted, biphasic, or flattened T wave could also provide insights into the
abnormalities of the ventricular repolarization or secondary to abnormalities
in ventricular depolarisation. In addition, the QT interval as a group of
individual patterns from the beginning of the Q wave to the end of the T wave,
could represent the physiologic reactions for the ventricles of the heart to
depolarize and repolarize. Thus, there exists a vital need for the methods
that can enhance the interpretability of time-series representations from the
perspectives of both single factor disentanglement and group-level factor
disentanglement, to induce multi-level semantics.

However, time-series presents great challenges for learning disentangled
representations. Firstly, complex data structure with temporal correlations
makes latent codes hard to interpret. Time-series data usually contain
temporal correlations, which cannot be directly captured and interpreted by
traditional image-focused disentangle methods (shen2020interfacegan, ;
shao2020controlvae, ; chen2018isolating, ). While traditional sequential
models, like LSTM or LSTM-VAE, could be used to model the temporal
correlations, they neither provide interpretable predictions, as is often
criticized, nor have an disentangle mechanism. Secondly, sequential model may
suffer from KL vanishing problem. Recent work (wang2019riemannian, ) shows
that: since the LSTM generative model often has strong expressiveness, the
reconstruction term in the objective will dominate the KL divergence term. In
this case, the model would generate time-series without making effective use
of the latent codes. Besides, the latent variables $Z$ will become independent
of the observations $\mathbf{x}$ when the KL divergence term collapses to
zero. It’s referred to as the information preference problem
(chen2016variational, ). Thirdly, interpretable semantic concepts often rely
on multiple factors instead of individuals. A human-understandable sequential
pattern, called semantic components, is usually correlated with multiple
factors. It is hard to interpret time-series with a single latent factor and
its variations.

To address these challenges, we propose Disentangle Time-Series (DTS) for
learning semantically interpretable time-series representations. To the best
of our knowledge, DTS is the first attempt to incorporate disentanglement
strategies for time-series. It is one of the first to interpret time-series
representations with state-of-the-art deep models. In particular, we design a
multi-level time-series disentanglement strategy that accounts for both
individual latent factor and group-level semantic segments, to generate
hierarchical semantic concepts as the interpretable and disentangled
representations of time-series. For individual latent factor disentanglement,
DTS introduces a mutual information maximization term (MIM) to encourage high
mutual information between the latent codes and the original time-series. By
leveraging the MIM, DTS preserves the disentanglement property via evidence
lower bound (ELBO) decomposition, while alleviates the KL vanishing problem.
We further theoretically prove that the decomposition process can balance the
preference between correct inference and fitting data distribution. For group-
level semantic segment disentanglement, DTS learns decomposed semantic
segments that contain batches of independent latent variables via applying
gradient reversal layers on irrelevant tasks. The learned group segment
disentanglement can benefit many downstream tasks such as domain adaptation.
Extensive experiments on real-world datasets demonstrate that DTS could
provide more meaningful disentangled representations of time-series, and is
quantitatively effective for downstream tasks.

The contributions of this work are summarized as follows:

  * •

We introduce a novel and challenging problem ($i.e.$, disentangle time-series)
and propose DTS, to incorporate disentanglement strategies for time-series
representation learning task.

  * •

We devise multi-level time-series disentanglement strategies, covering both
individual latent factor and group-level semantic segments, to generate
hierarchical semantic concepts as the interpretable and disentangled
representation of time-series.

  * •

We propose an ELBO decomposition strategy to balance the preference between
correct inference and fitting data distribution problem. We theoretically show
how to alleviate the KL vanishing problem via introducing the mutual
information term. Meanwhile, we preserve a heavier penalty on the total
correlation and the dimension-wise KL to keep the disentangle property.

  * •

We conduct extensive experiments on the time-series representation and the
domain-adaptation tasks, and provide insights on interpreting the latent
representations with semantic meanings.

##  2\. Methodology

In this section, we propose DTS (see Fig. 2), a multi-level disentanglement
approach to enhance time-series representation learning. The key idea of DTS
is that the latent space should involve multiple independent factors as
semantic concepts rather than conjugated representation. We first introduce
the disentanglement problem (Section 2.1) and the challenges (Section 2.2).
Then, we introduce an individual disentanglement approach for learning
interpretable representations (Section 2.3). Finally, we introduce how to
learn disentangled group segments as semantic components (Section 2.4).

Figure 2. An overview of DTS. It consists of two components: (i) individual
factor disentangle: to learn semantic factors like $z_{0}$ (shown as pink) to
control the sequential pattern of the time-series, e.g., the time-series moves
down when adjusting $z_{0}$; and (ii) group segment disentangle: to learn more
complex semantic patterns $\mathbf{z}_{m},\mathbf{z}_{n}$ (illustrated as the
common and discriminative patterns). A, B denote two time-series.

###  2.1. Problem Statement

In this section, we describe the notations and formulate the disentanglement
problem. To facilitate understanding, we focus on univariate time-series in
the definitions; one can easily extend them to multivariate time-series.

Notations: We use lowercase alphabet $x\in\mathbb{R}$ to denote a scalar value
in a time-series, $z\in\mathbb{R}$ to denote a scalar latent variable,
lowercase boldface letter
$\mathbf{x}=[x_{1},x_{2},...,x_{T}]\in\mathbb{R}^{T}$ to denote a time-series
of length $T$, lowercase boldface letter
$\mathbf{z}=[z_{1},z_{2},...,z_{N}]\in\mathbb{R}^{N}$ to denote a segment of
latent variables with size $N$, uppercase alphabet $Z$ to denote a set of
latent variables, which either consists of segments, e.g.,
$Z=\\{\mathbf{z}_{1},\mathbf{z}_{2},...,\mathbf{z}_{N}\\}$, or consists of
scalars, e.g., $Z=\\{z_{1},z_{2},...,z_{N}\\}$, and $|Z|$ to denote the size
of all the latent variables, i.e., the dimension of the representations.

Disentanglement Problem: The independent latent factors and semantic segments
can be defined as follows.

###### Definition 1 (Independent Latent Factors and Segments).

Two latent variables $z_{i}$ and $z_{j}$ (or semantic segments
$\mathbf{z}_{i}$ and $\mathbf{z}_{j}$) are independent if the change of the
sequential patterns corresponding to variable $z_{i}$ (segment
$\mathbf{z}_{i}$) is relatively invariant to variable $z_{j}$ (segment
$\mathbf{z}_{j}$) and vice versa, denoted as $z_{i}\Perp z_{j}$
($\mathbf{z}_{i}\Perp\mathbf{z}_{j}$).

Based on Definition 1, we formally define the multi-level time-series
disentanglement problem as follows.

  * •

_Individual Latent Factor Disentanglement_ : We aim to learn a set of
decomposed latent variables $Z=\\{z_{1},z_{2},...,z_{N}\\}$, such that all the
pairs of latent variables in $Z$ are independent, i.e., $z_{i}\Perp
z_{j},\forall i,j$, where $i\in\\{1,2,...,N\\}$, $j\in\\{1,2,...,N\\}$, and
$i\neq j$.

  * •

_Group Segment Disentanglement_ : We aim to learn a set of decomposed semantic
segments $Z=\\{\mathbf{z}_{1},\mathbf{z}_{2},...,\mathbf{z}_{N}\\}$, such that
all the pairs of segments in $Z$ are independent, i.e.,
$\mathbf{z}_{i}\Perp\mathbf{z}_{j},\forall i,j$, where $i\in\\{1,2,...,N\\}$,
$j\in\\{1,2,...,N\\}$, and $i\neq j$.

###  2.2. $\beta$-VAE and KL Vanishing Problem

In this section, we focus on the latent codes disentanglement task using the
generative model family. Generative models have shown superior performance in
learning the complex distributions. LSTM-VAE (chung2015recurrent, ) is widely
used in time-series analytics. A latent variable generative model defines a
joint distribution between a feature space $Z\in\mathcal{Z},$ and the
observation space $\mathbf{x}_{1:T}\in\mathcal{X}$. Suppose we have a simple
prior distribution $p(Z)$ is placed over the latent variables. The
distribution of the observations conditioned on latent variables is modeled
with a deep network $p_{\theta}(\mathbf{x}_{1:T}\mid Z)$. If we have the true
underlying distribution as $p_{\mathcal{D}}(\mathbf{x})$ (empirically
approximated through the training set), then a natural training objective is
to maximize the marginal likelihood as:

(1) $\mathbb{E}_{p_{\mathcal{D}}(\mathbf{x})}\left[\log
p_{\theta}(\mathbf{x})\right]=\mathbb{E}_{p_{\mathcal{D}}(\mathbf{x}_{1:T})}\left[\log\mathbb{E}_{p(Z)}\left[p_{\theta}(\mathbf{x}_{1:T}\mid
Z)\right]\right].$

Directly optimizing the likelihood is generally intractable, since computing
$p_{\theta}(\mathbf{x})=\int_{z}p_{\theta}(\mathbf{x}\mid Z)p(Z)\mathrm{d}z$
requires integration. A widely-used solution (kingma2013auto, ) is to
approximate the posterior via an amortized inference distribution
$q_{\phi}(Z\mid\mathbf{x})$ and jointly optimize a lower bound to the log
likelihood as:

(2) $\displaystyle\mathcal{L}_{\mathrm{ELBO}}(\mathbf{x})$
$\displaystyle=-D_{\mathrm{KL}}\left(q_{\phi}(Z|\mathbf{x}_{1:T})\|p(Z)\right)+\mathbb{E}_{q_{\phi}(Z|\mathbf{x}_{1:T})}\left[\log
p_{\theta}(\mathbf{x}_{1:T}|Z)\right]$ $\displaystyle\leq\log
p_{\theta}(\mathbf{x}_{1:T}).$

A disentangled representation (burgess2018understanding, ) can be defined as
one where single latent units are sensitive to changes in single generative
factor, while being relatively invariant to changes in other factors.
$\beta$-VAE (burgess2018understanding, ) is a variant of the variational
autoencoder that attempts to learn a disentangled representation by optimizing
a heavily penalized objective with $\beta>1$. The penalization is capable of
obtaining models exhibiting disentangled effects in image datasets. With
increasing $\beta$, the latent codes could be more disentangled by allowing
distributions in the latent space to be further away from each other by less
satisfying the KL-term constraint to fit the marginally Gaussian distribution.
The semantically similar observations might be mapped closer, thus creating a
sort of clusters for interpretations.

(3) $\displaystyle\mathcal{L}_{\mathrm{\beta-ELBO}}(\mathbf{x})$
$\displaystyle=-\beta
D_{\mathrm{KL}}\left(q_{\phi}(Z|\mathbf{x}_{1:T})\|p(Z)\right)+\mathbb{E}_{q_{\phi}(Z|\mathbf{x}_{1:T})}\left[\log
p_{\theta}(\mathbf{x}_{1:T}|Z)\right]$ $\displaystyle\leq\log
p_{\theta}(\mathbf{x}_{1:T}).$

However, for sequential time-series data, prior work (wang2019riemannian, )
shows that: the LSTM generative model often has a strong expressiveness. The
reconstruction term in the objective will dominate the KL divergence term
during the training phase. In this case, the model will generate time-series
without effectively making use of latent codes. Besides, the latent variable
$Z$ becomes independent from observations $\mathbf{x}$, when the KL divergence
term collapses to zero. As a result, the latent variable $Z$ can not serve as
an effective representation for the input $\mathbf{x}$, and it can not be used
for downstream tasks such as classification and clustering. It can be further
referred to as the information preference problem (chen2016variational, ).
Thus, pushing Gaussian clouds away from each other in the latent space becomes
meaningless if latent distributions are unhooked with the observation space.

###  2.3. Individual Latent Factor Disentanglement

To alleviate the KL vanishing problem, as well as to preserve the disentangle
property, in this section, we introduce the mutual information maximization
term to the ELBO decomposition. The goal is to learn a better representation
$Z$ that could more effectively capture the semantic characteristics of the
input $\mathbf{x}$.

####  2.3.1. ELBO TC-Decomposition

To further understand the internal mechanism within the disentangle strategy,
we decompose the evidence lower bound (ELBO) to find evidence linking
factorial representations to disentanglement.

Mathematically, the KL term in Eq. 2 and 3 can be decomposed with a factorized
$p(Z)$ as:

(4) $\displaystyle
D_{\mathrm{KL}}(q({Z}\mid\mathbf{x}_{1:T})||p({Z}))=\underbrace{\mathrm{KL}(q(Z,\mathbf{x}_{1:T})||q(Z)p(\mathbf{x}_{1:T}))}_{\text{(i)
Index-Code MI }}$
$\displaystyle\qquad\qquad+\underbrace{\mathrm{KL}(q(Z)\|\prod_{j}q\left(z_{j}\right))}_{\text{(ii)
}\text{ Total Correlation
}}+\underbrace{\sum_{j}\mathrm{KL}\left(q\left(z_{j}\right)\|p\left(z_{j}\right)\right)}_{\text{(iii)
Dimension-wise KL }}$

where $z_{j}$ denotes the $j$th dimension of the latent variable. The proof
can be found in Appendix A.1.

The first term can be interpreted as the index-code mutual information (MI).
The index-code MI is the mutual information $I_{q_{\phi}}(Z;\mathbf{x})$
between the data variable and latent variable based on the empirical data
distribution $q_{\phi}(Z,\mathbf{x})$. From the information theory, the second
term is referred to as the total correlation (TC). TC acts as one of many
generalizations of mutual information to more than two random variables
(watanabe1960information, ). TC is also to measure the dependencies between
the variables. The penalty on TC forces the model to find statistically
independent factors in the data distribution. A heavier penalty on this term
induces a more disentangled representation, and that the existence of this
term is the reason $\beta$ -VAE has been successful. One recent study
empirically verifies (chen2018isolating, ; zhao2019infovae, ) that
$\mathrm{TC}$ is the most important term in this decomposition for learning
disentangled representations by only penalizing on this term. The last term is
referred to as the dimension-wise KL. This term mainly prevents individual
latent dimensions from deviating too far from their corresponding priors. It
plays a role as a complexity penalty on the aggregate posterior, which follows
from the minimum description length (hinton1994autoencoders, ) formulation of
the ELBO.

By decomposing the ELBO into separate components, we can have a new
perspective for the problem when the latent codes are becoming independent
from observations. Introducing a heavier penalty on the ELBO tends to neglect
the mutual information between $Z$ and $\mathbf{x}$. Thus, the mutual
information becomes vanishingly small. Intuitively, the reason is that the
learned $p_{\theta}(\mathbf{x}|Z)$ is the same for all $z\in\mathcal{Z}$,
implying that the $Z$ is not dependent on the observations $\mathbf{x}$. This
is undesirable because a major goal of unsupervised learning is to learn
meaningful latent features that should depend on the observations. If we
directly apply $\beta$-VAE models to time-series data for disentanglement,
$\beta$-VAE can not effectively trade-off weighting of the mutual information
and the reconstruction. Desired disentangled representations emerge when the
right balance is found between information preservation (reconstruction cost
as regularisation) and latent channel capacity restriction (when $\beta>1$).
Increasing the $\beta$ may intensify the mutual information vanishing problem:
when the model has a better quality of disentanglement within the learned
latent representations, it penalizes the mutual information simultaneously.
This, in turn, can lead to under-fitting or ignoring the latent variables.

####  2.3.2. ELBO DTS-Decomposition

We now introduce how to balance the preference between correct inference and
fitting data distribution, and provide a new ELBO decomposition solution that
sheds light on solving the information preference problem.

Previously, we discussed that optimizing the ELBO tends to push the
probability mass of $q_{\phi}(Z\mid\mathbf{x})$ too far from 0. This tendency
is a property of the ELBO objective and true for any ${x}$ and ${Z}$. However,
this is made worse by the fact that ${x}$ is often higher dimensional compared
to ${Z}$, so any error in fitting ${x}$ will be magnified compared to ${Z}$.
To encourage the model to use the latent codes, we add a mutual information
maximization term, which encourages a high mutual information between
$\mathbf{x}$ and $Z$. In other words, we can address the information
preference problem through balancing the preference between correct inference
and fitting data. Comparing with the ELBO in LSTM-VAE (in Eq. 2), we can
rewrite it as:

(5) $\displaystyle\mathcal{L}_{\mathrm{ELBO}}(\mathbf{x})$
$\displaystyle=-D_{\mathrm{KL}}\left(q_{\phi}(Z\mid\mathbf{x}_{1:T})\|p(Z)\right)+\alpha
I_{q_{\phi}}(\mathbf{x}_{1:T};Z)$
$\displaystyle+\mathbb{E}_{q_{\phi}(Z\mid\mathbf{x}_{1:T})}\left[\log
p_{\theta}(\mathbf{x}_{1:T}\mid Z)\right]$

where $I_{q_{\phi}}(\mathbf{x};Z)$ denotes the mutual information between
$\mathbf{x}$ and $Z$ under the distribution $q_{\phi}(\mathbf{x};Z)$. But the
objective can not be directly optimized. Thus, we rewrite it into another
equivalent form:

(6) $\displaystyle\mathcal{L}_{\mathrm{ELBO}}(\mathbf{x})$
$\displaystyle=-D_{\mathrm{KL}}\left(q_{\phi}(Z\mid\mathbf{x}_{1:T})\|p(Z)\right)+\alpha
D_{\mathrm{KL}}\left(q_{\phi}(Z)\|p(Z)\right)$
$\displaystyle+\mathbb{E}_{q_{\phi}(Z\mid\mathbf{x}_{1:T})}\left[\log
p_{\theta}(\mathbf{x}_{1:T}\mid Z)\right].$

We find that the mutual information maximization term (the second part of Eq.
6) plays the same role as the first term in the ELBO-TC decomposition (as
shown in Eq. 4). But the optimization directions are contrary. Thus,
increasing the disentanglement degree may intensify the KL vanishing problem,
and vice versa. To enforce the model to preserve the disentangle property
while alleviating the KL vanishing, here, we combine the mutual information
regularizer term with the ELBO-TC decomposition in Eq. 4 and merge the mutual
information maximization term, then the ELBO can be written as:

(7) $\displaystyle\mathcal{L}_{\mathrm{ELBO}}$
$\displaystyle(\mathbf{x})=-\beta
D_{\mathrm{KL}}(q(Z)\|\prod_{j}q\left(z_{j}\right))-\beta\sum_{j}D_{\mathrm{KL}}\left(q\left({z}_{j}\right)\|p\left({z}_{j}\right)\right)$
$\displaystyle+(\alpha-\beta)D_{\mathrm{KL}}\left(q_{\phi}(Z)\|p(Z)\right)+\mathbb{E}_{q_{\phi}(Z\mid\mathbf{x}_{1:T})}\left[\log
p_{\theta}(\mathbf{x}_{1:T}\mid Z)\right],$

where the mutual information regularizer merges with the first term in the
decomposition analysis. Mathematically, we alleviate the KL vanishing problem
by introducing the mutual information maximization term, while preserve a
heavier penalty (when $\beta>1$) on the total correlation and the dimension-
wise KL to keep the disentangle property.

###  2.4. Latent Group Segment Disentanglement

The individual factor disentanglement introduced in the last section can’t
guarantee interpretability. For time-series applications, it’s hard to
interpret the single latent factor, and human-interpretable semantic concepts
typically rely on multiple latent factors instead of a single factor. Thus, in
this section, we introduce how to disentangle latent codes of time-series into
group segments (Section 2.4.1). We further show that our proposed group
segment disentanglement could benefit the domain adaptation task (Section
2.4.2).

####  2.4.1. Group Segment Disentanglement

Different from individual variable analysis, in this section, we aim to learn
decomposed semantic segments that contain batches of latent variables. One
main advantage of the group segment disentanglement is to extract more than
one latent factors, which simultaneously contribute to one complex sequential
semantic concept.

Assume that the semantic segments are independent factors, i.e.,
$\mathbf{z}_{m}\Perp\mathbf{z}_{n}$ with every pairs of $m,n$ sampled from
$\mathbf{z}=\\{\mathbf{z}_{1}...\mathbf{z}_{k}\\}$, where $k$ is the number of
the segments. For ease of presentation, we take $m,n$ as two semantic segments
for illustration. Our method could be easily extended to multiple segments
scenarios. Given $\mathbf{x}$, the new time-series is generated from two
independent latent group segments, i.e., $\mathbf{z}_{m}$ encodes the
$m_{th}$-segment and $\mathbf{z}_{n}$ encodes the $n_{th}$-segment.

###### Theorem 2.

Assume that two group segments are independent, i.e.,
$\mathbf{z}_{m}\Perp\mathbf{z}_{n}$, and let
$\mathbf{z}=\left\\{\mathbf{z}_{m},\mathbf{z}_{n}\right\\}$, the empirical
error on the disentangled segments with a hypothesis $h$ is:

(8)
$\epsilon(h)=\mathrm{E}_{\mathbf{z}_{m}\sim\mathcal{Z}}\left[C_{m}\left(\mathbf{z}_{m}\right)-h\left(\mathbf{z}_{m}\right)\right]+\mathrm{E}_{\mathbf{z}_{n}\sim\mathcal{Z}}\left[C_{n}\left(\mathbf{z}_{n}\right)-h\left(\mathbf{z}_{n}\right)\right]$

where $\epsilon_{\mathrm{T}}^{\mathrm{y}}(h)$ denotes the empirical error of
DTS with respect to $h$.

Proof. Since $\mathbf{z}_{m}\Perp\mathbf{z}_{n}$, we can derive the empirical
error as follows:

(9) $\displaystyle\epsilon(h)$
$\displaystyle=\mathrm{E}_{(\mathbf{z}_{m},\mathbf{z}_{n})\sim\mathcal{Z}}\left[C\left(\mathbf{z}\right)-h\left(\mathbf{z}\right)\right]$
$\displaystyle=\mathrm{E}_{\mathbf{z}_{m}\sim\mathcal{Z}}\left[C_{m}\left(\mathbf{z}_{m}\right)-h\left(\mathbf{z}_{m}\right)\right]+\mathrm{E}_{\mathbf{z}_{n}\sim\mathcal{Z}}\left[C_{n}\left(\mathbf{z}_{n}\right)-h\left(\mathbf{z}_{n}\right)\right].$

Based on the independence property between $\mathbf{z}_{m}$ and
$\mathbf{z}_{n},$ the distribution of $\mathcal{Z}$ can be decomposed into two
parts so as to the error. Following the evidence lower bound of the marginal
likelihood in the Eq. 6, we get a similar form for group segments:

(10)
$\displaystyle\mathcal{L}_{\mathrm{ELBO-G}}(\mathbf{x})=-D_{\mathrm{KL}}\left(q_{\phi_{m}}(\mathbf{z}_{m}\mid\mathbf{x}_{1:T})\|p(\mathbf{z}_{n})\right)-D_{\mathrm{KL}}\left(q_{\phi_{n}}(\mathbf{z}_{n}\mid\mathbf{x}_{1:T})\|p(\mathbf{z}_{n})\right)$
$\displaystyle\qquad+\mathbb{E}_{q_{\phi_{m}}(\mathbf{z}_{m},\mathbf{z}_{n}\mid\mathbf{x}_{1:T})}\left[\log
p_{\theta}(\mathbf{x}_{1:T}\mid\mathbf{z}_{m},\mathbf{z}_{n})\right]+\alpha
D_{\mathrm{KL}}\left(q_{\phi}(Z)\|p(Z)\right).$

We assume that
$P\left(\boldsymbol{z}_{m}\right),P\left(\boldsymbol{z}_{n}\right)\sim\mathcal{N}(\mathbf{0},\mathbf{I})$,
and $\phi_{m}$ and $\phi_{n}$ are the parameters of the encoder. By applying a
reparameterization trick, we use sequential models (LSTMs or TCNs
(bai2018empirical, )) $H_{m}\left(G(\mathbf{x});\phi_{y}\right)$ and
$H_{n}\left(G(\mathbf{x});\phi_{d}\right)$ as the universal approximator of
$q$ to encode the data into $\mathbf{z}_{m}$ and $\mathbf{z}_{n}$,
respectively.

Meanwhile, we want to make the $\mathbf{z}_{m}$ task-$n$-invariant, and make
the $\mathbf{z}_{n}$ task-$m$ invariant. That is, we want to make the
$C_{m}\left(\mathbf{z}_{n};\theta_{m}\right)$ and
$C_{n}\left(\mathbf{z}_{m};\theta_{n}\right)$ less discriminative. To obtain
task-invariant representations, we seek the parameters $\theta_{m}$ of the
feature mapping that maximize the loss of the $q_{\theta_{m}}$ (by making the
distributions of different $\mathbf{z}_{m}$ as close as possible), while
simultaneously seeking the parameters $\theta_{n}$ that minimize the loss of
$\mathbf{z}_{m}$. We aim to solve:

(11)
$\mathbb{E}_{\text{m}}\left(\phi_{y},\theta_{m},\theta_{n}\right)=\mathbb{E}\left(C_{m}\left(\mathbf{z}_{m};\theta_{m}\right),y_{m}\right)-\lambda\mathbb{E}\left(C_{n}\left(\mathbf{z}_{m};\theta_{n}\right),y_{n}\right).$

Moreover, we seek to minimize the loss of $\mathbf{z}_{n}$. Similarly, we
have:

(12)
$\mathbb{E}_{\text{n}}\left(\phi_{y},\theta_{m},\theta_{n}\right)=\mathbb{E}\left(C_{n}\left(\mathbf{z}_{n};\theta_{n}\right),y_{n}\right)-\lambda\mathbb{E}\left(C_{m}\left(\mathbf{z}_{n};\theta_{m}\right),y_{m}\right).$

From Eq. 11 and 12, we are seeking the parameters
$\hat{\theta}_{f},\hat{\theta}_{y},\hat{\theta}_{d}$ as:

(13)
$\begin{array}[]{c}\left(\hat{\theta}_{f},\hat{\theta}_{y}\right)=\arg\min_{\theta_{f},\theta_{y}}E\left(\theta_{f},\theta_{y},\hat{\theta}_{d}\right)\\\
\hat{\theta}_{d}=\arg\max_{\theta_{d}}E\left(\hat{\theta}_{f},\hat{\theta}_{y},\theta_{d}\right),\end{array}$

where the parameter $\lambda$ controls the trade-off between the two
objectives that shape the features during training. The update process is very
similar to vanilla stochastic gradient descent (SGD) updates for feed-forward
deep models. One thing that needs to point out is the $-\lambda$ factor, which
tries to make disentangled features less discriminative for the irrelevant
task. Here, we use a gradient reversal layer (GRL) (ganin2015unsupervised, )
to exclude the discriminative information. During the forward propagation, GRL
acts as an identity transform. During the backpropagation, GRL takes the
gradient from the subsequent level, and multiplies the gradient by a negative
constant, then passes it to the preceding layer.

####  2.4.2. Domain Adaptation as Concrete Example

To further illustrate the benefits of the proposed group segments
disentanglement for time-series, we apply it to the domain adaptation problem
as a concrete application scenario.

In the unsupervised domain adaptation problem, we use the labeled samples
$D_{S}=\left\\{\boldsymbol{x}_{1:T}^{S},y_{i}^{S}\right\\}_{i=1}^{n_{S}}$ on
the source domain to classify the unlabeled samples $D_{T}=$
$\left\\{\boldsymbol{x}_{j}^{T}\right\\}_{j=1}^{n_{T}}$ on the target domain.
We aim to obtain two independent latent variables with disentanglement,
including a domain-dependent latent variable $\mathbf{z}_{d}$ and a class-
dependent latent variable $\mathbf{z}_{y}$. These two variables are expected
to encode the domain information and the class information, respectively.
Then, we can use the class-dependent latent variable for classification since
it is domain-invariant. Mathematically, deriving from Theorem 2, we have:

###### Theorem 3.

Assume that the class and the domain factors are independent, i.e.,
$\mathbf{z}_{y}\Perp\mathbf{z}_{d}.$ Let
$\mathbf{z}=\left\\{\mathbf{z}_{y},\mathbf{z}_{d}\right\\},$ and the error on
the disentangled source and target domain with a hypothesis $h$ is:
$\begin{aligned}
\@add@centering\centering\epsilon_{S}(h)&=\mathrm{E}_{\mathbf{z}_{y}\sim\mathcal{Z}_{S}}\left[C\left(\mathbf{z}_{y}\right)-h\left(\mathbf{z}_{y}\right)\right]+\mathrm{E}_{\mathbf{z}_{d}\sim\mathcal{Z}_{S}}\left[C\left(\mathbf{z}_{d}\right)-h\left(\mathbf{z}_{d}\right)\right]\\\
\epsilon_{T}(h)&=\mathrm{E}_{\mathbf{z}_{y}\sim\mathcal{Z}_{T}}\left[C\left(\mathbf{z}_{y}\right)-h\left(\mathbf{z}_{y}\right)\right]+\mathrm{E}_{\mathbf{z}_{d}\sim\mathcal{Z}_{T}}\left[C\left(\mathbf{z}_{d}\right)-h\left(\mathbf{z}_{d}\right)\right].\end{aligned}$

According to the Theorem 3, we can find that, the disentangled empirical
classification error rate with respect to $h$ in the source domain is lower
than before disentanglement
($\epsilon_{S}^{y}(h)=\epsilon_{S}(h)-\epsilon_{S}^{d}(h)$, where
$\epsilon_{S}^{d}(h)\geq 0$). Prior work (cai2019learning, ) theoretically
prove that the disentanglement of the representation space could be helpful
and necessary for obtaining a lower classification error rate. Furthermore, a
lower classification error rate on the source domain will tighten the error
bound at the target domain.

We measure the discrepancy distance between the source and target distribution
with respect to hypothesis $h$. Formally, we have:

(14) $\displaystyle
d_{\mathcal{H}\Delta\mathcal{H}}(\mathcal{S},\mathcal{T})=2\sup_{h_{1},h_{2}\in\mathcal{H}}|P_{\mathbf{f}\sim\mathcal{S}}\left[h_{1}(\mathbf{f})\neq
h_{2}(\mathbf{f})\right]-P_{\mathbf{f}\sim\mathcal{T}}\left[h_{1}(\mathbf{f})\neq
h_{2}(\mathbf{f})\right]|$

A probabilistic bound on the performance $\varepsilon_{\mathcal{T}}(h)$ of
classifier $h$ from $\mathcal{T}$ is evaluated on the target domain, given its
performance $\varepsilon_{\mathcal{S}}(h)$ on the source domain, where
$\mathcal{S}$ and $\mathcal{T}$ are source and target distributions,
respectively:

(15)
$\varepsilon_{\mathcal{T}}(h)\leq\varepsilon_{\mathcal{S}}(h)+\frac{1}{2}d_{\mathcal{H}\Delta\mathcal{H}}(\mathcal{S},\mathcal{T})+C$

where $C$ does not depend on a particular $h$. Prior work
(ganin2015unsupervised, ) proves that optimal discriminator gives the upper
bound for the $d_{\mathcal{H}\triangle\mathcal{H}}(\mathcal{S},\mathcal{T})$,
so that tighter bound $\varepsilon_{\mathcal{S}}(h)$ from the source domain
could lead to a better approximation of $\varepsilon_{\mathcal{T}}(h)$.

##  3\. Why does Proposed Method Work?

In this section, we introduce intuition as well as theoretical justifications
of DTS, and its relationships with existing methods.

###  3.1. Theoretical Possibility

One recent work essentially shows that without inductive biases on both models
and datasets, the unsupervised disentangle task is fundamentally impossible
(locatello2019challenging, ).

###### Theorem 1.

Suppose $p(\mathbf{z})$ is a d-dimensional distribution and $d>1$, let
$\mathbf{z}\sim P$ denote any distribution which admits a density
$p(\mathbf{z})=\prod_{i=1}^{d}p\left(\mathbf{z}_{i}\right).$ Then, there
exists an infinite family of bijective functions
$f:\operatorname{supp}(\mathbf{z})\rightarrow$
$\operatorname{supp}(\mathbf{z})$ such that $\frac{\partial
f_{i}(\boldsymbol{u})}{\partial u_{i}}\neq 0$ almost everywhere for all i and
$j$ (i.e., $\mathbf{z}$ and $f(\mathbf{z})$ are completely entangled) and
$P(\mathbf{z}\leq\boldsymbol{u})=P(f(\mathbf{z})\leq\boldsymbol{u})$ for all
$\boldsymbol{u}\in\operatorname{supp}(\mathbf{z})$ (i.e., they have the same
marginal distribution). (locatello2019challenging, )

Intuitively, after observing $\mathbf{x}$, we can construct infinitely many
generative models, which have the same marginal distribution of $\mathbf{x}$.
Any one of these models could be the true causal generative model for the
data, and the right model cannot be identified given only the distribution of
$\mathbf{x}$.

While Theorem 1 shows that unsupervised disentanglement learning is
fundamentally impossible for arbitrary generative models, it is still
theoretically possible for our proposed DTS. First, from the model structure
perspective, our proposed DTS exploits TCN and LSTM-like models, which impose
the inductive biases towards preserving contextual information over long
sequences in the design of the neural network architecture. Second, from the
data structure perspective, time-series as sequential data, contain trend and
seasonal patterns, with the serial correlation between subsequent
observations. Third, from the implicit supervision perspective, DTS could
incorporate additional weak supervision (like signals from the source domain)
to guide a better disentangled representation. Thus, we make explicit and
implicit inductive bias available during the training phase.

###  3.2. Relationship to Existing Models

Here, we discuss the relationship of DTS to existing models based on the
information bottleneck principle (tishby2015deep, ). The proposed DTS,
including individual latent factors and group segments, can be considered as a
product of variational decomposition of mutual information terms in the
information bottleneck (IB) framework.

####  3.2.1. Information Bottleneck for Unsupervised Models

The unsupervised IB can be treated as a compression process from $\mathbf{x}$
to $\mathbf{z}$ via the parametrized mapping
$q_{\phi}(\mathbf{z}\mid\mathbf{x})$. This process leads to a bottleneck
representation $\mathbf{z}$ yet preserving a certain level of information
$I_{x}$ in $\mathbf{z}$ about $\mathbf{x}$. Accordingly, this problem can be
formulated as:

(16) $\min_{\phi:I(\mathbf{Z};\mathbf{X})\geq
I_{x}}I_{\boldsymbol{\phi}}(\mathbf{X};\mathbf{Z}),$

and in the Lagrangian formulation as a minimization of:

(17)
$\mathcal{L}(\boldsymbol{\phi})=I_{\boldsymbol{\phi}}(\mathbf{X};\mathbf{Z})-\beta
I(\mathbf{Z};\mathbf{X}).$

####  3.2.2. Relation to LSTM-VAE and $\beta$-VAE

Encoders in LSTM-VAE maps a data point from the observation space into a
probabilistic output of Gaussian cloud with mean $\mu(\mathbf{x})$ and
‘ellipsoid’ orientation determined by the diagonal covariance matrix
$\operatorname{diag}(\sigma(\mathbf{x}))$. Comparing with LSTM-VAE,
$\beta$-VAE relaxes the stochastic ‘compression’ via mapping everything to a
Gaussian heap by applying the relaxation parameter that might give more
preference to the reconstruction loss. Our proposed DTS can be considered as
another form of compression by the minimization of
$I_{\boldsymbol{\phi}}(\mathbf{X};\mathbf{Z})$. DTS relaxes the condition to
map all conditional distributions to one Gaussian heap.

####  3.2.3. Relation to Domain Adaptation Methods

Existing domain adaptation methods focus on (i) utilizing maximum mean
discrepancy to measure the domain alignment (tzeng2014deep, ); and (ii)
extracting the domain-invariant representation as transferable common
knowledge on the feature space (cai2019learning, ; wilson2020multi, ).
Motivated by the success of disentanglement in the image domain, DTS aims to
extract the group segments in the latent disentangled semantic representation
of the data. DTS also introduces interpretability in the latent space via
weak-supervised signals as imposing special constraints on the latent codes.

####  3.2.4. Complexity Analysis

DTS can disentangle latent factors for sequential data, scale to complicated
datasets, and typically requires no more training time than vanilla VAEs. It
does not require an exponentially growing computational cost in the number of
factors.

##  4\. Experiments

We conduct experiments to answer research questions as follows:

  * •

Q1: Compared with non-disentangled methods, how quantitatively effective is
the proposed disentangled strategy?

  * •

Q2: Does individual latent factor disentanglement benefit the generation
process of the time-series in a more informative way?

  * •

Q3: Whether or not latent group segment disentanglement strategy could
separate semantic concepts?

###  4.1. Experiment Setup

In this section, we introduce benchmark datasets and baselines. For
hyperparameter settings and network configurations, please refer to Appendix
A.3 for the details.

####  4.1.1. Datasets

We evaluate DTS on five benchmark datasets for individual latent factor
disentanglement and group segment disentanglement with domain adaptation as a
concrete task. For the time-series representation task, it is label-free. For
the domain adaptation task, the training-test split is 80% and 20%
respectively, and the training data is further split into training-validation
with the same proportions. We segment the series into non-overlapping windows
of 128 time steps (more details in Appendix A.2).

  * •

Human Activity Recognition (HAR) (anguita2013public, ): contains sequential
accelerometer, gyroscope, and estimated body acceleration data from $30$
participants at 50 Hz.

  * •

Heterogeneity Human Activity Recognition (HHAR) (stisen2015smart, ): includes
accelerometer data from $31$ smartphones of different manufacturers and models
positioned in various orientations.

  * •

WISDM Activity Recognition (WISDM AR) (kwapisz2011activity, ): contains $33$
participants’ accelerometer data, which are sampled at 20 Hz.

  * •

uWave (liu2009uwave, ): is a large gesture library with over $4000$ samples
collected from eight users over an elongated period of time for a gesture
vocabulary with eight gesture patterns.

  * •

ECG Signal (blackburn1960electrocardiogram, ): contains heartbeats annotated
by at least two cardiologists. The annotations are mapped into $5$ groups in
accordance with the AAMI standard.

####  4.1.2. Baselines

We compare DTS with three state-of-the-art domain adaptation algorithms and
one time-series generative model:

  * •

Recurrent Domain Adversarial Neural Network(R-DANN) (ajakan2014domain, ):
employs an LSTM network, and promotes the emergence of features that are (i)
discriminative for the learning task on the source domain and (ii)
indiscriminate with respect to the shift between the domains.

  * •

Variational Recurrent Adversarial Deep Domain Adaptation (VRADA)
(purushotham2017variational, ): uses a variational RNN and trains
adversarially to capture temporal relationships that are domain-invariant.

  * •

Convolutional Deep Domain Adaptation (CoDATS) (wilson2020multi, ): leverages
domain-invariant domain adaptation methods to operate on time-series data, and
utilizes weak supervisions from labels.

  * •

LSTM-VAE (chung2015recurrent, ): is similar to an auto-encoder. It learns an
LSTM as the encoder that maps the sequential data to a latent representation
in a probabilistic manner, and decodes the latents back to data.

###  4.2. Performance Evaluation

To answer the research question Q1, we apply DTS to domain adaptation tasks to
validate the _effectiveness of the group segmentation_ (Section 2.4), and
compare DTS with the state-of-the-art algorithms. During the training phase,
only the training dataset is accessible: labeled data as the source domain and
unlabeled data as the target domain. During the evaluation, the test data
becomes available as the target domain for quantitative evaluations and
visualizations.

Problem | W/O | R-DANN | VRADA | CoDATS | DTS | Target
---|---|---|---|---|---|---
HAR 2 → 11 | 83.3 | 80.7 | 64.1 | 74.5 | 84.3 | 100.0
HAR 7 → 13 | 89.9 | 75.3 | 78.3 | 96.5 | 98.1 | 100.0
HAR 9 → 18 | 31.1 | 56.6 | 59.8 | 85.8 | 89.8 | 100.0
HAR 14 →19 | 62.0 | 71.3 | 64.4 | 98.6 | 100.0 | 100.0
HAR 18 →23 | 89.3 | 78.2 | 72.9 | 89.3 | 94.9 | 100.0
HAR 7 → 24 | 94.4 | 84.8 | 93.9 | 99.1 | 100.0 | 100.0
HAR 17 →25 | 57.3 | 66.3 | 52.0 | 97.6 | 100.0 | 100.0
HHAR 1 →3 | 77.8 | 85.1 | 81.3 | 90.8 | 93.7 | 99.2
HHAR 3 →5 | 68.8 | 85.4 | 82.3 | 94.3 | 95.9 | 99.0
HHAR 4 →5 | 60.4 | 70.4 | 71.6 | 94.2 | 94.9 | 99.0
HHAR 1 →6 | 72.1 | 81.7 | 74.9 | 90.8 | 92.1 | 98.8
HHAR 4 →6 | 48.0 | 64.6 | 62.7 | 85.3 | 92.3 | 98.8
HHAR 5 →6 | 65.1 | 54.4 | 60.0 | 91.7 | 92.5 | 98.8
HHAR 5 →8 | 95.3 | 82.5 | 87.5 | 95.8 | 97.9 | 99.3
WISDM 4 → 15 | 78.2 | 69.2 | 82.7 | 81.4 | 82.9 | 100.0
WISDM 2 → 25 | 81.1 | 57.8 | 72.2 | 90.6 | 95.8 | 100.0
WISDM 25 → 29 | 47.1 | 61.6 | 81.9 | 74.6 | 82.2 | 95.7
WISDM 7 → 30 | 62.5 | 41.7 | 61.9 | 73.2 | 89.2 | 100.0
WISDM 21 → 31 | 57.1 | 61.0 | 68.6 | 92.4 | 96.4 | 97.1
WISDM 2 → 32 | 60.1 | 49.0 | 66.7 | 68.6 | 70.7 | 100.0
WISDM 1 → 7 | 68.5 | 44.8 | 63.0 | 66.1 | 72.7 | 96.4
uWave 2 → 5 | 86.3 | 33.3 | 18.5 | 98.2 | 100.0 | 100.0
uWave 3 → 5 | 82.7 | 63.7 | 32.4 | 92.9 | 95.6 | 100.0
uWave 2 → 6 | 86.0 | 34.5 | 25.3 | 93.8 | 97.8 | 100.0
uWave 2 → 7 | 85.1 | 53.9 | 12.2 | 91.4 | 98.9 | 100.0
uWave 3 → 7 | 95.5 | 64.0 | 30.4 | 92.0 | 98.9 | 100.0
uWave 1 → 8 | 100.0 | 78.6 | 11.0 | 93.8 | 100.0 | 100.0
uWave 7 → 8 | 95.2 | 49.7 | 12.5 | 93.8 | 96.7 | 100.0
Table 1. Target classification accuracy (based on the class-dependent representation $\mathbf{z}_{y}$) for time-series domain adaptation (from source to target) on randomly-chosen problems for each dataset, adapting between different users. We include no adaptation as an approximate lower bound (W/O), and models trained directly on labeled target data as upper bound (Target) (more results are in Appendix A.4). Dataset | Acc/D-S | Acc/D-T | Acc/C-S | Acc/C-T | AUC/C-S | AUC/C-T
---|---|---|---|---|---|---
HAR | 100.0 | 100.0 | 100.0 | 97.2 | 100 | 99.2
HAR/r | 54.7 | 66.7 | 35.9 | 22.2 | 54.6 | 53.7
HHAR | 96.9 | 98.8 | 97.9 | 91.1 | 99.8 | 98.7
HHAR/r | 70.8 | 70.9 | 30.5 | 17.6 | 58.3 | 52.7
WISDM | 100 | 100 | 94.5 | 70.7 | 99.0 | 85.9
WISDM/r | 67.3 | 46.7 | 38.2 | 33.3 | 69.7 | 67.9
uWave | 100 | 100 | 99.1 | 94.4 | 99.0 | 85.9
uWave/r | 55.4 | 48.2 | 18.8 | 12.5 | 54.6 | 53.7
Table 2. Ablation studies: discriminability. The results are listed as X-Y (X:
D denotes discriminability on domains, C denotes discriminability on tasks; Y:
S and T denote features sampled from source and target domains, respectively).
Top rows: performance based on the disentangled domain-dependent
$\mathbf{z}_{d}$, or class-dependent $\mathbf{z}_{y}$ for time-series domain
adaptation. Bottom rows(/r): performance based on the domain-invariant
$\mathbf{z}_{y}$ and class-invariant $\mathbf{z}_{d}$ segments.

####  4.2.1. Quantitative Results

Table 1 compares the performance of DTS and the baselines on HAR, HHAR, WISDM
AR, and uWave datasets. We include no adaptation as an approximate lower
bound, and models trained directly on labeled target data as upper bound.
After the group disentanglement, two variables are used to encode the domain
information and the class information, respectively. We use the class-
dependent latent variable for classification since it is domain-invariant. We
observe that DTS outperforms the baselines with consistently $+3\%$ higher
accuracy over all datasets. These results ascertain the effectiveness of DTS
in boosting the performance of domain adaptation by obtaining domain-invariant
transferable components common knowledge. The enhanced results also validate
eliminating irrelevant information from group disentanglement could prevent
negative transferring.

Figure 3. Comparison of learned latent variables. Traversals depict an
interpretable property in generating time-series from eight-dimensions of the
latent codes $Z$ (shown as 8 subfigures in a group). Traversals are sampled
from two different time-series of ECG (separated by the black line), with
range [-4,4] (shown as 9 lines in one subfigure).

(a) Visualizing the generated time-series from LSTM-VAE.

(b) Visualizing the generated time-series from DTS.

Figure 4. Latent traversal plots from DTS on ECG. All figures of latent codes
traversal each block corresponds to the traversal of a single latent variable
while keeping others fixed to either their inferred. Each row represents a
different seed image used to infer the latent values with traversal over the
[-4, 4] range. Blue and green denote two time-series with different sequential
patterns. The first two rows denote the decline degree at the first turning
point, transition from rigid to a more mild manner; The last two rows denote
the rising trend at the second turning point, transition from inconspicuous to
obvious.

####  4.2.2. Ablation Studies

We study whether DTS can decompose the representations into domain-dependent
and class-dependent components with a series of ablation studies. We compare
the discriminability of the disentangled features, including domain-dependent
($\mathbf{z}_{d}$) and class-dependent ($\mathbf{z}_{y}$) components (see
Section 2.4.2), sampled from both source and target domains. The ablations
include (i) using domain-dependent features $\mathbf{z}_{d}$ and class-
dependent features $\mathbf{z}_{y}$, and (ii) domain-invariant and class-
invariant features (shown as /r), respectively. The comparison between DTS and
the ablations (/r) is shown in Table 2. We observe that DTS significantly
outperforms the ablations over all datasets. Disentangled task-dependent group
segments consistently help to improve the performance. Conversely, the class-
dependent features are invariant to the change of domains, and the domain-
dependent features are invariant to the change of classes. DTS could preserve
the discriminability of the disentangled features corresponding to the
specified task, and simultaneously make disentangled features less
discriminative for the irrelevant task. It indicates that these disentangled
group segments do not contain any useful semantic concepts for other
irrelevant tasks.

###  4.3. Individual Latent Factor Disentanglement

To answer the research question Q2, we provide latent traversals as
qualitative results to validate that DTS tends to consistently discover more
informative latent factors and provide more meaningful disentangled
representations of time-series (see Section 2.3).

####  4.3.1. Qualitative Comparison of Latent Codes

We train DTS on ECG data to evaluate disentanglement performance for
individual latent factors. We use the same traversal way to show the
disentanglement quality. Fig. 3 provides a qualitative comparison of the
disentanglement performance of DTS and LSTM-VAE. We edit the time-series by
altering the latent codes in the $\mathcal{Z}$ space. Here, the dimension of
the representation is set to be $12$. This setting helps reduce the impact of
differences in complexity by model frameworks. However, for a better
comparison, we only select eight dimensions that change more regularly. The
sequences visualized in panels are generated from ${Z}\sim
q\left({Z}\mid\boldsymbol{x}_{1:T}\right)$. Hence, the dynamics are imposed by
the encoder, but the identity is sampled from the prior.

Fig. 3 shows traversals in latent variables that depict an interpretable
property in generating time-series. Often it could generate more semantic
convincing time-series than LSTM-VAE. Comparing the visualization from panel
(a) and panel (b), DTS could generate a time-series in a more diverse way. It
can be seen that sampling from an entangled representation results in LSTM-VAE
(panel (a)) only reflects small differences according to the traversal
perturbations. One possible reason is that the LSTM-VAE is dominated by the
reconstruction term during the training phase. The slight changes only
correspond to the reconstruction distortion due to the latent codes and
observations are independent. Comparing with LSTM-VAE, some of the DTS latent
codes tend to learn a smooth continuous transformation over a wider range of
factor values as vibrations. A clear transition process can be observed from
the manipulation results with respect to the $\mathcal{Z}$ space. As the value
of individual latent factor increases, the semantic of the latent factor
changes across different sequential patterns. Other latent codes are robust
with the vibrations, as it does not play any role in the generation process.
Single latent units are sensitive to changes in single generative factor,
while being relatively invariant to changes in other factors. One possible
reason is that the representation of the time-series could be effectively
expressed with a few latent codes in the $\mathcal{Z}$. Individual latent
factor disentanglement process may help us to recognize the useful latent
codes, and discard the redundant parts. All these results demonstrate that DTS
is able to disentangle useful knowledge from sequential data, which is more
informative as interpretable factors in the latent space.

####  4.3.2. Traversal Plots to Discover Semantics

There is currently no general method for quantifying the degree of learned
disentanglement or optimize the hyperparameters (unless there are concept
ground-truth factors $v$ available, then the mutual information gap (MIG)
(chen2018isolating, ; shao2020controlvae, ) could be used to determine if
there exists a deterministic, invertible relationship between $z$ and $v$).
Therefore, there is no way to quantitatively compare the degree of
disentanglement achieved by different models or when optimizing the
hyperparameters of a single model. Fig. 4 plots the manipulation results of
the latent traversal results from DTS. Each block of the figure corresponds to
the traversal of a single latent variable while keeping others fixed. Each row
represents a different seed image used to infer the latent values with
traversal over the [-4, 4] range. The results show that our manipulation
approach performs well on all attributes in both positive and negative
directions. We observe that moving the latent codes can produce continuous
change, with the sequential patterns orthogonal to the others. According to
the editing process, the first and the third rows (sampled from two different
time-series of ECG) denote the decline degree at the first turning point,
transition from rigid to a more mild manner; the second and the last rows
(with the same sampling strategy) denote the rising trend at the second
turning point, transition from inconspicuous to obvious. It demonstrates that
DTS discovered latent factors in an unsupervised manner that encode sequential
trend and depict an interpretable property in the generation. These
observations provide strong evidence that DTS does not produce time-series
randomly, but learns some interpretable semantics in the latent space.

(a) T-SNE visualizations of the DTS activations on the distribution of domain-
dependent representation (left) and domain-invariant representations(right).
Blue points correspond to the source domain examples, while red ones
correspond to the target domain ones.

(b) T-SNE visualizations of DTS activations on the distribution of class-
dependent representation (left) and class-invariant representations (right).
Each color denotes one specific class.

Figure 5. The effect of (a) domain-dependent and -invariant (b) class-
dependent and -invariant disentanglement on the distribution of the extracted
features. In all cases, the adaptation in DTS makes the dependent/invariant
features from different sources more/less distinguishable, respectively.

###  4.4. Latent Group Segment Disentanglement

To answer the research question Q3, we visualize the disentangled segments
over the representation space (see Section 2.4.1). Fig. 5(a) shows the effect
of domain-dependent and domain-invariant disentanglement on the distribution
of the extracted features. We observe that, for all the datasets, the
adaptation in DTS makes the disentangled domain(class)-dependent features more
distinguishable, but the domain(class)-invariant features indistinguishable.
The results validate the DTS can learn decomposed segments that contain
independent semantic information. Furthermore, we can observe an apparent
clustering effect (the different colors denote different categories). A
widely-accepted assumption (ben2014domain, ) indicates that observation
distribution contains separated data clusters and data samples in the same
cluster share the same class label in domain adaptations. These results
validate the discriminative ability of the disentanglement, since DTS is
capable of yielding strong clustering in the target domain. And it almost
matches the prior perfectly, as the semantically similar observations are
mapped closer, and create clusters. This phenomenon gives us another insight
as the disentangled group segments could enhance the interpretability.

##  5\. Conclusion

In this paper, we investigated a novel and challenging problem of learning
disentangled time-series representations. DTS introduces a multi-level
disentanglement strategy, covering both individual latent factor and group
semantic segments, to generate hierarchical semantic concepts as the
interpretable and disentangled representation. It alleviates the KL vanishing
problem via introducing a mutual information maximization term, while
preserving a heavier penalty on the total correlation and the dimension-wise
KL to keep the disentangle property via balancing the preference between
correct inference and fitting data distribution. The experimental results
demonstrated the effectiveness of DTS in learning interpretable semantic
concepts with disentanglement.

## References

  * (1) Y. Shen, C. Yang, X. Tang, and B. Zhou, “Interfacegan: Interpreting the disentangled face representation learned by gans,” TPAMI, 2020.
  * (2) H. Blackburn and et al, “The electrocardiogram in population studies: a classification system,”
  * (3) K.-H. Lai, D. Zha, G. Wang, J. Xu, Y. Zhao, D. Kumar, Y. Chen, P. Zumkhawaka, M. Wan, D. Martinez, et al., “Tods: An automated time series outlier detection system,” arXiv preprint arXiv:2009.09822, 2020.
  * (4) D. Zha, K.-H. Lai, M. Wan, and X. Hu, “Meta-aad: Active anomaly detection with deep reinforcement learning,” arXiv preprint arXiv:2009.07415, 2020.
  * (5) N. Liu, Q. Tan, Y. Li, H. Yang, J. Zhou, and X. Hu, “Is a single vector enough? exploring node polysemy for network embedding,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
  * (6) X. Huang, Q. Song, Y. Li, and X. Hu, “Graph recurrent networks with attributed random walks,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
  * (7) M. Du, S. Pentyala, Y. Li, and X. Hu, “Towards generalizable forgery detection with locality-aware autoencoder,” arXiv preprint arXiv:1909.05999, 2019\.
  * (8) K.-H. Lai, D. Zha, Y. Li, and X. Hu, “Dual policy distillation,” arXiv preprint arXiv:2006.04061, 2020.
  * (9) K. Zhou, X. Huang, Y. Li, D. Zha, R. Chen, and X. Hu, “Towards deeper graph neural networks with differentiable group normalization,” arXiv preprint arXiv:2006.06972, 2020.
  * (10) Y. Li, X. Huang, J. Li, M. Du, and N. Zou, “Specae: Spectral autoencoder for anomaly detection in attributed networks,” in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019.
  * (11) Y. Li, D. Zha, P. Venugopal, N. Zou, and X. Hu, “Pyodds: An end-to-end outlier detection system with automated machine learning,” in Companion Proceedings of the Web Conference 2020, 2020.
  * (12) Y. Li, N. Liu, J. Li, M. Du, and X. Hu, “Deep structured cross-modal anomaly detection,” in 2019 International Joint Conference on Neural Networks (IJCNN), 2019.
  * (13) Y. Li, Z. Chen, D. Zha, K. Zhou, H. Jin, H. Chen, and X. Hu, “Autood: Automated outlier detection via curiosity-guided search and self-imitation learning,” arXiv preprint arXiv:2006.11321, 2020.
  * (14) S. Semeniuta and et al, “A hybrid convolutional variational autoencoder for text generation,” arXiv preprint arXiv:1702.02390, 2017.
  * (15) V. Fortuin and et al, “Som-vae: Interpretable discrete representation learning on time series,” arXiv preprint arXiv:1806.02199, 2018.
  * (16) V. Fortuin, D. Baranchuk, G. Rätsch, and S. Mandt, “Gp-vae: Deep probabilistic time series imputation,” in AISTAT, 2020.
  * (17) F. Yang, Z. Zhang, H. Wang, Y. Li, and X. Hu, “Xdeep: An interpretation tool for deep neural networks,” arXiv preprint arXiv:1911.01005, 2019.
  * (18) X. Guo, L. Zhao, Z. Qin, L. Wu, A. Shehu, and Y. Ye, “Interpretable deep graph generation with node-edge co-disentanglement,” in KDD, 2020.
  * (19) H. Shao, S. Yao, D. Sun, A. Zhang, S. Liu, D. Liu, J. Wang, and T. Abdelzaher, “Controlvae: Controllable variational autoencoder,” in ICML, 2020.
  * (20) R. T. Chen, X. Li, R. Grosse, and D. Duvenaud, “Isolating sources of disentanglement in variational autoencoders,” arXiv preprint arXiv:1802.04942, 2018.
  * (21) P. Z. Wang and W. Y. Wang, “Riemannian normalizing flow on variational wasserstein autoencoder for text modeling,” arXiv preprint arXiv:1904.02399, 2019.
  * (22) X. Chen, D. P. Kingma, T. Salimans, Y. Duan, P. Dhariwal, J. Schulman, I. Sutskever, and P. Abbeel, “Variational lossy autoencoder,” arXiv:1611.02731, 2016.
  * (23) J. Chung, K. Kastner, L. Dinh, K. Goel, A. Courville, and Y. Bengio, “A recurrent latent variable model for sequential data,” arXiv preprint arXiv:1506.02216, 2015.
  * (24) D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  * (25) C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, et al., “Understanding disentangling in beta-vae,” arXiv preprint arXiv:1804.03599, 2018.
  * (26) S. Watanabe, “Information theoretical analysis of multivariate correlation,” IBM Journal of research and development, 1960.
  * (27) S. Zhao, J. Song, and S. Ermon, “Infovae: Balancing learning and inference in variational autoencoders,” in AAAI, 2019.
  * (28) G. E. Hinton and R. S. Zemel, “Autoencoders, minimum description length, and helmholtz free energy,” NeurIPS, 1994.
  * (29) S. Bai et al., “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” arXiv preprint arXiv:1803.01271, 2018.
  * (30) Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in ICML, 2015.
  * (31) R. Cai, Z. Li, P. Wei, J. Qiao, K. Zhang, and Z. Hao, “Learning disentangled semantic representation for domain adaptation,” in IJCAI, 2019.
  * (32) F. Locatello and et al, “Challenging common assumptions in the unsupervised learning of disentangled representations,” in ICML, 2019.
  * (33) N. Tishby and N. Zaslavsky, “Deep learning and the information bottleneck principle,” in 2015 IEEE Information Theory Workshop (ITW), 2015.
  * (34) E. Tzeng and et al, “Deep domain confusion: Maximizing for domain invariance,” arXiv preprint arXiv:1412.3474, 2014.
  * (35) G. Wilson, J. R. Doppa, and D. J. Cook, “Multi-source deep domain adaptation with weak supervision for time-series sensor data,” in KDD, 2020.
  * (36) D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, “A public domain dataset for human activity recognition using smartphones.,” in Esann, 2013.
  * (37) A. Stisen and et al, “Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition,” in SenSys, 2015.
  * (38) J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity recognition using cell phone accelerometers,” ACM SigKDD Explorations Newsletter, 2011.
  * (39) J. Liu and et al, “uwave: Accelerometer-based personalized gesture recognition and its applications,” Pervasive and Mobile Computing, 2009.
  * (40) H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, and M. Marchand, “Domain-adversarial neural networks,” arXiv preprint arXiv:1412.4446, 2014\.
  * (41) S. Purushotham, W. Carvalho, T. Nilanon, and Y. Liu, “Variational recurrent adversarial deep domain adaptation.,” in ICLR, 2017.
  * (42) S. Ben-David and R. Urner, “Domain adaptation–can quantity compensate for quality?,” Annals of Mathematics and Artificial Intelligence, 2014.


# Security Code Review by LLMs: A Deep Dive into Responses

Jiaxin Yu, Peng Liang, Yujia Fu  School of Computer Science
Wuhan UniversityWuhanChina jiaxinyu, liangp<EMAIL_ADDRESS>,  Amjed
Tahir  School of Mathematical and Computational Sciences
Massey UniversityPalmerston NorthNew Zealand<EMAIL_ADDRESS>,  Mojtaba
Shahin  School of Computing Technologies
RMIT UniversityMelbourneAustralia<EMAIL_ADDRESS>,  Chong Wang
School of Computer Science
Wuhan UniversityWuhanChina<EMAIL_ADDRESS>and  Yangxiao Cai  School of
Computer Science
Wuhan UniversityWuhanChina<EMAIL_ADDRESS>
###### Abstract.

Security code review aims to combine automated tools and manual efforts to
detect security defects during development. The rapid development of Large
Language Models (LLMs) has shown promising potential in software development,
as well as opening up new possibilities in automated security code review. To
explore the challenges of applying LLMs in practical code review for security
defect detection, this study compared the detection performance of three
state-of-the-art LLMs (Gemini Pro, GPT-4, and GPT-3.5) under five prompts on
549 code files that contain security defects from real-world code reviews.
Through analyzing 82 responses generated by the best-performing LLM-prompt
combination based on 100 randomly selected code files, we extracted and
categorized quality problems present in these responses into 5 themes and 16
categories. Our results indicate that the responses produced by LLMs often
suffer from verbosity, vagueness, and incompleteness, highlighting the
necessity to enhance their conciseness, understandability, and compliance to
security defect detection. This work reveals the deficiencies of LLM-generated
responses in security code review and paves the way for future optimization of
LLMs towards this task.

Large Language Model, Security Code Review, Quality Problem

††ccs: Security and privacy Software security engineering

##  1\. Introduction

With the continuous release of various Large Language Models (LLMs),
demonstrating their potential in software development tasks like code
completion (Destefanis et al., 2023), program repair (Jiang et al., 2023;
Sohail et al., 2023) and test cases generation (Kang et al., 2023), there is a
deeper integration of LLMs into developers’ workflows to tackle more
challenging tasks (GitHub, 2024). Code review is a crucial stage to detect
security defects earlier in software development (Yu et al., 2023). The rapid
improvement of LLMs opens up avenues for automated security defect detection.
Compared to traditional techniques, LLMs have advantages in generating natural
language outputs and supporting human-friendly interactions. Their potential
for collaborating with humans in complex reasoning tasks, such as security
code review, is worth investigation.

Previous studies demonstrated that in terms of security defect detection, LLMs
like ChatGPT, showed limited capabilities in real-world datasets (Cheshkov et
al., 2023; Fu et al., 2023). Some works have explored enhancing the
effectiveness of LLMs by either enhancing prompts (Zhang et al., 2023) or
adding components to the models (Mathews et al., 2024; Silva et al., 2023).
However, most studies only focused on evaluating and improving the performance
of LLMs, without delving into the underlying problems. Specifically, the
existing quality problems in LLM-generated responses for identifying security
defects in real-world code reviews, which can serve as an indication for
future improvement, have not yet been explored.

With the goal of revealing the existing difficulties and optimization
directions of LLMs in security code review, we selected three representative
LLMs, i.e., Gemini Pro, GPT-4, and GPT-3.5 and compared their performance
under five prompts in security defect detection using an established code
review dataset (Yu et al., 2023). Then we extracted and analyzed the problems
present in the responses generated by the most effective LLM-prompt
combination, i.e., GPT-4 paired with the prompt $\textbf{P}_{cid}$. The most
prevalent problems in responses generated by GPT-4 based on the prompt
$\textbf{P}_{cid}$ is the inclusion of unnecessary information, with vague
statements and incomplete answers also frequently present. Therefore, apart
from improving the detection capabilities of LLMs, enhancing the conciseness
of LLM-generated responses is currently the foremost concern and it is also
significant to consider how to make these responses more understandable and
comprehensive when utilizing LLMs for automated security code review.

##  2\. Background & Related Work

###  2.1. Using LLMs in Security Code Review

Several studies have explored the use of LLMs in security code review. Zhang
et al. (Zhang et al., 2023) analyzed the performance of ChatGPT to detect
whether there are security defects in the code under different prompts,
demonstrating the effectiveness of using ChatGPT for prompt-enhanced
vulnerability detection. Fu et al. (Fu et al., 2023) compared GPT-4 with fine-
tuned smaller-scale language models in detecting, classifying, and repairing
security defects, and they found that GPT-4 performed worse across all tasks,
attributing this to a lack of pre-training on necessary software security
expertise such as specific security defect patterns. Unlike the aforementioned
studies, our study concentrated on the challenges of utilizing LLMs in real-
world security code review, rather than their performances.

###  2.2. Linguistic Analysis of LLM Responses

Most studies have chosen ChatGPT to conduct linguistic analysis of LLM
responses. Both Guo et al. (Guo et al., 2023) and Mitrović et al. (Mitrović et
al., 2023) compared ChatGPT-generated responses with human answers to
questions across different fields, pointing out the characteristics of the
former such as being more detailed, fabricating facts, and not possessing the
profundity. Kabir et al. (Kabir et al., 2023) compared ChatGPT responses and
Stack Overflow answers to software engineering questions in terms of quality
and correctness, indicating that conceptual errors are as pressing as factual
errors in ChatGPT responses. Taking inspiration from these studies, our work
focused on the task of detecting security defects identified by reviewers and
analyzed the quality problems present in LLM-generated responses.

##  3\. Methodology

###  3.1. Dataset

We leveraged the code review dataset constructed by Yu et al. (Yu et al.,
2023), and utilized LLMs to detect security defects identified by reviewers.
The dataset contains 614 review comments, each identifying a specific security
defect, from four projects: OpenStack’s Nova and Neutron, and Qt’s Base and
Creator. Considering the input token limits of LLMs, we only provided the code
of the single file where the review comments were raised, rather than the
entire patchset, in our prompts. The 614 security-related review comments were
reorganized as follows: First, we filtered out comments that were published at
the patchset level, as such data cannot be located in a specific file to
construct a prompt. Then, for each code file that these comments correspond
to, we consolidated all comments the file contained and corresponding security
defect records into a single data item. Last, we utilized Gerrit API to
retrieve detailed contents of these code files and the commit messages of
their corresponding patchsets, excluding cases where the API returned no data
due to errors in Gerrit. The final dataset consists of 549 code files, each
including the source code, commit message of the associated patchset, and
security defects identified by reviewers within this file.

###  3.2. Model Evaluation

####  3.2.1. Model Selection

We chose models from the GPT family, which are widely used LLMs (Wu et al.,
2023), and the Gemini family, which demonstrates performance comparable to or
even surpassing GPT models in several benchmarks (Google Deepmind, 2023).
Given that Gemini Ultra was not available at the time of doing our research,
we opted for Gemini Pro. Regarding the GPT family, through a preliminary
experiment, we selected the gpt-4 and text-davinci-002-render-sha models
provided by the ChatGPT platform. In the following text, these two models are
referred to as ‘GPT-4’ and ‘GPT-3.5’ respectively.

####  3.2.2. Prompt Design

Prompt engineering is a process of designing prompts to optimize model
performance on downstream tasks (Liu et al., 2023). To fully harness the
capabilities of LLMs in detecting security defects, with reference to the
three prompt design directions adopted by Zhang et al., i.e., basic prompting,
prompting enhanced by auxiliary information, and Chain-of-Thought (CoT)
prompting (Zhang et al., 2023), we formulated five prompts for comparison, as
presented in Table 1.

Table 1. Types of prompts. Prompt |  Template
---|---
$\textbf{P}_{b}$ |  Please review the code below to detect security defects. If any are found, please describe the security defect in detail and indicate the corresponding line number of code and solution. If none are found, state: ‘No security defects are detected in the code’. [CODE]
$\textbf{P}_{r}$ |  I want you to act as a code reviewer of [PROJECT] in [COMMUNITY]. Please review the code below to detect security defects. If any are found, please describe the security defect in detail and indicate the corresponding line number of code and solution. If none are found, state: ‘No security defects are detected in the code’. [CODE]
$\textbf{P}_{c}$ |  Please review the code below for security defects using the CWE (Common Weakness Enumeration) as a reference. If any are found, please describe the security defect in detail and indicate the corresponding line number of code and solution. If none are detected, state: ‘No security defects are detected in the code’. [CODE]
$\textbf{P}_{cid}$ |  $\textbf{P}_{cid}$: Please review the code below for security defects. You can consider defect types in terms of:  (1) CWE-284 (Improper Access Control) (2) CWE-435 (Improper Interaction Between Multiple Entities) (3) CWE-664 (Improper Control of a Resource Through its Lifetime) (4) CWE-682 (Incorrect Calculation) (5) CWE-691 (Insufficient Control Flow Management) (6) CWE-693 (Protection Mechanism Failure) (7) CWE-697 (Insufficient Comparison) (8) CWE-703 (Improper Check or Handling of Exceptional Conditions) (9) CWE-707 (Improper Neutralization) (10) CWE-710 (Improper Adherence to Coding Standards) If any are found, please describe the security defect in detail and indicate the corresponding line number of code and solution. If none are detected, states: ‘No security defects are detected in the code’.
$\textbf{P}_{cot-n}$ |  $\textbf{P}_{cot-1}$: Based on the given code from a commit, please generate supplementary code files according to the commit message.[Commit message][Code]
$\textbf{P}_{cot-2}$: Analyze the original code in the context of the
generated files for security defects. If any are found, please describe the
security defect in detail and indicate the corresponding line number of code
and solution. If none are detected, state: ‘No security defects are detected
in the code’.

Basic Prompting: We first proposed the basic prompt ($\textbf{P}_{b}$), in
which we directly instruct the model to review the provided source code to
detect security defects and clarify the expected output, i.e., defect
description, corresponding code lines, and solutions.

Prompting Enhanced by Auxiliary Information: Providing LLMs with auxiliary
information in reasoning tasks can effectively avoid wrong answers caused by
hallucination (Trivedi et al., 2022). Following the best practices released by
OpenAI (OpenAI, 2023), we instructed the model to play the role of a code
reviewer for the open-source project of provided code, so as to help it
understand project-specific components or functions, as presented in
$\textbf{P}_{r}$. Since Common Weakness Enumeration (CWE) (The MITRE
Corporation, 2023) has been frequently used in classifying security defects
(Paul et al., 2021; Bosu et al., 2016), we employed CWE as auxiliary
information in two ways: by directly instructing the model to reference CWE
for identifying security defects in $\textbf{P}_{c}$ and by listing the first
level CWEs in VIEW-1000 (Research Concepts) in $\textbf{P}_{cid}$. VIEW-1000
is a weaknesses classification framework designed for academic research,
widely adopted in prior work (Pan et al., 2023). It organizes CWEs into a
hierarchical structure based on parent-child relationships among CWE entries,
with CWEs at the first level considered to possess the most general concepts
and utilized in the prompt to encompass all security defect types as
inclusively as possible.

Chain-of-Thought (CoT) Prompting: According to Section 3.1, only a single code
file from the project is provided in the prompt. The absence of contextual
information of this file may hinder LLMs from fully comprehending its impact
on the entire project and impede the detection of security defects within this
file. Commit messages serve as crucial information for reviewers to grasp the
main content of the code change (Bosu et al., 2016; Tao et al., 2012), thus
regarded as a concise supplement to the missing code context. We leveraged the
strong capability of LLMs in code generation (Destefanis et al., 2023) and
adopted CoT, a prompt design method for facilitating reasoning tasks (Wei et
al., 2022), to utilize the commit message in prompt $\textbf{P}_{cot-n}$. We
first instructed the LLM to generate complementary code files according to the
commit message, then guided it to review the provided code for security
defects in the context of generated files.

####  3.2.3. Evaluation Metrics

Aiming to measure the performance of LLMs, we adopted the evaluation
methodology proposed by Mahajan et al. (Mahajan et al., 2020) and made some
adjustments to the classification of responses it formulated to match our
topic, as detailed in Table 2. The corresponding evaluation metrics are
I-Score ($\frac{I}{I+H+U+M}\times 100\%$), IH-Score
($\frac{I+H}{I+H+U+M}\times 100\%$), and M-Score ($\frac{M}{I+H+U+M}\times
100\%$). Higher I-Score and IH-Score indicate a stronger ability of the model
to detect security defects identified by reviewers, while a lower M-Score
signifies less misleading results produced during detection.

Table 2. Adjusted IHMU-category. Classification |  Definition
---|---
Instrumental (I) |  The response explicitly indicates the existence of a security defect identified by the reviewer and provides a fully accurate description.
Helpful
(H) |  The response raises concerns related to the security defects identified by the reviewer, but their descriptions may not be entirely accurate or specific enough.
Misleading (M) |  The response does not provide any helpful information. e.g. claiming no defects were found or failing to perform the detection task.
Uncertain
(U) |  The response does not mention defects identified by the reviewer but instead points out the existence of other security defects. Due to the knowledge limitation of the researchers, the actual existence of these security defects cannot be verified and the quality of the responses cannot be judged.

####  3.2.4. Pilot Data Labelling

We manually inspected the content of LLM-generated responses, their
corresponding source code, and security defects identified by reviewers to
classify these responses. To mitigate bias, a pilot data labelling was
conducted, in which we randomly selected 100 code files from our dataset,
constructed five prompts for each file based on the templates (see Table 1),
and fed these prompts into LLMs to collect responses. Using the four
categories described in Table 2 as labelling criteria, the first and third
authors labelled the collected responses separately. Discrepancies in
labelling results were discussed with the second author to reach a consensus.
The inter-rater reliability was calculated using the Cohen’s Kappa
coefficient, yielding a value of 0.89.

###  3.3. Data Extraction and Analysis

A formal experiment was conducted in the following steps to analyze quality
problems in the responses generated by the best-performing LLM-prompt
combination for security defect detection. In step 1, we constructed prompts
using five templates on the dataset of 549 code files and fed these prompts
into three LLMs to collect their responses. In step 2, the first author
manually labelled all the responses into the four categories (see Table 2) and
discussed any uncertain cases with the second author to settle on the
category. We calculated the performance scores based on the labelling results
and found that the combination of GPT-4 and prompt $\textbf{P}_{cid}$
outperforms the others (see Table 3). In step 3, under this best combination,
we randomly selected 100 code files and, due to the input token limit of
GPT-4, obtained 82 responses to conduct sentence-level data extraction.
Utilizing the open coding and constant comparative method (Glaser, 1965) and
employing MAXQDA as the coding tool, we extracted quality problems from the 82
responses. The specific coding, extraction, and analysis process is as
follows:

Based on the problem categories formulated by Kabir et al. (Kabir et al.,
2023) and referencing problems identified in previous studies (Borji, 2023;
Mitrović et al., 2023), the first and seventh authors firstly inspected a few
GPT-4 responses independently and recorded their observations. Then they
collaboratively reviewed the inspected responses and discussed to establish an
initial codebook, resulting in five themes: Correctness, Understandability,
Conciseness, Consistency, and Compliance, each with a series of problem types.
Following this, two rounds of pilot data extraction were conducted to improve
the agreement between the two authors. In each round, 10 of the 82 responses
were randomly selected and their problems were extracted by the two authors
based on the initial codebook. The extraction results were then compared and
discussed, with partial adjustments made to the codebook. Since a single
response could contain multiple problems, problem extraction is a multi-level,
multi-label classification task where the labels are not mutually exclusive.
Therefore, we calculated Krippendorff’s alpha (Krippendorff, 2004) with
Jaccard distance as the agreement statistic, which improved from 0.40 in the
first round to 0.80 in the second, indicating a suitably high level of
agreement. We ultimately identified 16 problem types under five themes, as
detailed in Section 4.2. The remaining data extraction was completed by the
first author and verified by two other coauthors. Throughout the entire
procedure from problem categories design to data extraction, we employed a
negotiated agreement approach (Campbell et al., 2013): any conflicts were
consulted and addressed by three authors, ensuring the reliability of data
analysis. The dataset of the formal experiment is provided at (Yu et al.,
2024).

##  4\. Results

###  4.1. Performance of LLMs

In the formal experiment comparing the performance of LLMs in different
prompts, since the token numbers of prompt templates and the token limits of
LLMs vary, the number of responses generated by different LLM-prompt
combinations varies accordingly. The response numbers and the performance
scores of all the LLM-prompt combinations in detecting security defects
identified by reviewers is presented in Table 3. Overall, all three LLMs show
very low performances, while GPT-4 performs better than GPT-3.5 and Gemini
Pro. Within GPT-4, $\textbf{P}_{cid}$ achieves the highest IH-Score (14.45%),
along with a relatively high I-Score (5.50%) and relatively low M-Score
(38.52%), which are highlighted with gray cells in Table 3. Consequently, we
considered GPT-4 with $\textbf{P}_{cid}$ as the best LLM-prompt combination,
and further conducted an analysis of the quality problems in the 82 responses
generated by this combination.

Table 3. Response count and performance scores for each LLM-Prompt combination on the full dataset. Model & Prompt | $\textbf{P}_{b}$ | $\textbf{P}_{r}$ | $\textbf{P}_{c}$ | $\textbf{P}_{cid}$ | $\textbf{P}_{cot-n}$
---|---|---|---|---|---
GPT-3.5 | Response Count | 278 | 278 | 278 | 276 | 273
I-Score | 0.36% | 2.52% | 1.08% | 0.72% | 4.40%
IH-Score | 1.08% | 5.04% | 2.16% | 1.09% | 9.16%
M-Score | 81.29% | 81.29% | 78.42% | 91.01% | 43.17%
GPT-4 | Response Count | 437 | 437 | 437 | 436 | 436
I-Score | 6.86% | 3.89% | 5.26% | 5.50% | 5.50%
H-Score | 10.98% | 9.61% | 13.27% | 14.45% | 11.70%
M-Score | 54.23% | 60.18% | 32.72% | 38.52% | 66.06%
Gemini Pro | Response Count | 403 | 398 | 398 | 403 | 396
I-Score | 0.00% | 0.00% | 0.50% | 1.99% | 0.25%
H-Score | 0.25% | 0.00% | 0.75% | 3.47% | 0.51%
M-Score | 95.29% | 99.25% | 91.71% | 48.64% | 96.97%

###  4.2. Problems in LLM-Generated Responses

A total of 16 categories of problems across 5 themes were identified. The
distribution of these problems is illustrated in Figure 1, where the same
color represents categories within the same theme, with darker shades
indicating the themes.

Figure 1. Proportion of each problem category present in all responses
generated by GPT-4.

####  4.2.1. Correctness

refers to the case in which responses contain content that contradicts
objective reality, domain-specific terminology, widely recognized concepts and
the like. Of the 82 GPT-4 responses, 16 (20%) exhibit correctness-related
problems; 9 (11%) demonstrate incorrect fact, in which the mentioned line
number or function name did not align with the actual code; 7 (9%) have the
incorrect concept problems, where the analysis deviates from the definitions
of CWE-IDs. As shown in the example of incorrect concept below, the problem of
inaccurate comparison logic described in the response does not fall under
Incorrect Calculation, thus indicating a misunderstanding of the concept of
CWE-682.

⬇

Human: [$\textbf{P}_{cid}$]

GPT-4: ...**CWE-682 (Incorrect Calculation):**

**Issue:** The method ‘_inventory_has_changed‘ uses a comparison logic that
might not be entirely accurate in all scenarios...

####  4.2.2. Understandability

includes problems like excessive or unreasonable content, ambiguous
expression, and incoherent or contradictory logic. Over half of the responses
(48, 59%) suffer from poor understandability. More precisely, 47 responses
(57%) exhibit vague statements, such as unclear localization and insufficient
descriptions. An example of unclear localization is shown below, in which
GPT-4 did not provide a specific location for the defect in the response.
Other less common problems include incoherent expression (2%), logical
contradiction (1%), and groundless speculation (7%).

⬇

Human: [$\textbf{P}_{cid}$]

GPT-4: ...

**CWE-703(Improper Check or Handling of Exceptional Conditions)**

**General Concern**: Throughout the code, there are instances where error
conditions might not be handled comprehensively...

####  4.2.3. Conciseness

reflects the problem of responses containing superfluous, irrelevant, or
redundant information beyond what is required for the specified task. It is a
widespread problem evident in almost all responses (74, 90%). 73 responses
(89%) include unnecessary information, such as explanations of nonexistent
defects and generic security suggestions. 32 responses (39%) include content
irrelevant to security defects like issues related to other quality attributes
(e.g., maintainability). There are also 11 abundant responses (13%) where
GPT-4 repetitively describes the same defect or reiterates no defect was
detected. Here is an example in which the response contains a description of
the nonexistent CWE-284, considered as unnecessary information.

⬇

Human: [$\textbf{P}_{cid}$]

GPT-4: ...**CWE-284 (Improper Access Control)**:

- There doesn’t seem to be any direct issue related to improper access control in the given code...

####  4.2.4. Consistency

is to gauge the alignment of GPT-4 responses with the reviewers’ assessments.
All 82 responses (100%) are more or less different from the reviewer’s
evaluation. In 22 cases (27%), the judgements of GPT-4 and reviewers on
whether there are security defects in code is inconsistent. In 71% of the
responses, GPT-4 identifies security defects that reviewers did not
(inconsistent defect), the existence of which could not be verified due to our
knowledge limitation. Additionally, for security defects detected by both
GPT-4 and reviewers, inconsistencies occur in location, analysis, and
solutions of the defects, accounting for 1%, 5%, and 1%.

####  4.2.5. Compliance

encompasses mistakes made by GPT-4 when generating responses in accordance
with the given task requirements. This primarily includes cases where the
output fails to include the undetected sentence required in the prompt, i.e.,
‘No security defects are detected in the code’, or lacks the required line
numbers and solutions of detected security defects. In a word, 57% of the
responses display compliance issues, the above two types of problems (non-
compliant output and incomplete answer) being present in 13% and 44% of all
the responses, respectively.

##  5\. Implications

The tendency of LLMs to produce generic text is a key factor that decreases
the quality of their responses. Over half of the responses generated by GPT-4
exhibit vague statements, such as generic advice, observations, and solutions.
Using generic text to answer domain-specific questions reduces the quality of
responses. Hence, we recommend the utilization of dedicated LLMs rather than
general-purpose LLMs for specific tasks in software development to obtain
answers that are more comprehensible and helpful.

GPT-4 fails to memorize details within prompts. We found that nearly half of
GPT-4’s responses were incomplete, lacking the content required in the prompt,
such as line number of code and solution. Additionally, there are also a few
responses containing line numbers of code or function names that mismatch the
source code provided in the prompts. These findings indicate that GPT-4’s
capability to memorize and reiterate detailed content from the input prompt is
not completely reliable, reflecting the need for improvement in attention
mechanisms and memory maintenance (Zhong et al., 2023).

Application barriers of LLMs vary across different tasks. In Kabir et al.’s
work (Kabir et al., 2023), which instructed an LLM to answer questions in
software development, over half of the LLM’s responses exhibit problems
related to correctness. However, in our study, which focused on detecting
security defects in code reviews, correctness problems were less prevalent. It
can be speculated that general LLMs encounters different challenges in
different application fields. Given that most previous studies on LLM
responses were conducted across a variety of domains, to better grasp the
optimization direction of applying LLMs in different tasks, analysis of LLM’s
responses to identify reasons for undetected security defects requires deeper
domain knowledge.

##  6\. Conclusions and Future Work

We investigated the challenges of utilizing LLMs for automated security code
review. We constructed prompts using five templates on 549 code files and fed
those prompts into three popular LLMs: GPT-3.5, GPT-4, and Gemini Pro. The
results revealed that those LLMs performed poorly in detecting security
defects. Problems related to conciseness, understandability, and compliance
frequently present in GPT-4’s responses to security defect detection,
indicating the key areas for further improvement. Potential future work
includes (1) analysis of LLM-generated responses to identify reasons for
undetected security defects, and (2) exploration of different prompts on the
presence and distribution of quality problems in responses, and (3)
examination of the existence of inconsistent defect in our study to discern
whether these are oversights of reviewers or false positives of LLMs.

## References

  * (1)
  * Borji (2023) Ali Borji. 2023.  A categorical archive of chatgpt failures.  _arXiv preprint arXiv:2302.03494_ (2023).
  * Bosu et al. (2016) Amiangshu Bosu, Jeffrey C Carver, Christian Bird, Jonathan Orbeck, and Christopher Chockley. 2016.  Process aspects and social dynamics of contemporary code review: Insights from open source development and industrial practice at Microsoft.  _IEEE Transactions on Software Engineering_ 43, 1 (2016), 56–75.
  * Campbell et al. (2013) John L Campbell, Charles Quincy, Jordan Osserman, and Ove K Pedersen. 2013.  Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement.  _Sociological Methods & Research_ 42, 3 (2013), 294–320.
  * Cheshkov et al. (2023) Anton Cheshkov, Pavel Zadorozhny, and Rodion Levichev. 2023.  Evaluation of ChatGPT Model for Vulnerability Detection.  _arXiv preprint arXiv:2304.07232_ (2023).
  * Destefanis et al. (2023) Giuseppe Destefanis, Silvia Bartolucci, and Marco Ortu. 2023.  A Preliminary Analysis on the Code Generation Capabilities of GPT-3.5 and Bard AI Models for Java Functions.  _arXiv preprint arXiv:2305.09402_ (2023).
  * Fu et al. (2023) Michael Fu, Chakkrit Tantithamthavorn, Van Nguyen, and Trung Le. 2023.  Chatgpt for vulnerability detection, classification, and repair: How far are we?  _arXiv preprint arXiv:2310.09810_ (2023).
  * GitHub (2024) GitHub 2024.  _A Developer’s Second Brain: Reducing Complexity Through Partnership with AI_.  GitHub.  https://github.blog/2024-01-17-a-developers-second-brain-reducing-complexity-through-partnership-with-ai/.
  * Glaser (1965) Barney G Glaser. 1965.  The constant comparative method of qualitative analysis.  _Social Problems_ 12, 4 (1965), 436–445.
  * Google Deepmind (2023) Google Deepmind 2023.  _Introducing Gemini: Our Largest and Most Capable AI Model_.  Google Deepmind.  https://blog.google/technology/ai/google-gemini-ai/.
  * Guo et al. (2023) Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023.  How close is chatgpt to human experts? comparison corpus, evaluation, and detection.  _arXiv preprint arXiv:2301.07597_ (2023).
  * Jiang et al. (2023) Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023.  Impact of Code Language Models on Automated Program Repair.  _arXiv preprint arXiv:2302.05020_ (2023).
  * Kabir et al. (2023) Samia Kabir, David N Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2023.  Who answers it better? an in-depth analysis of ChatGPT and stack overflow answers to software engineering questions.  _arXiv preprint arXiv:2308.02312_ (2023).
  * Kang et al. (2023) Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023.  Large language models are few-shot testers: Exploring llm-based general bug reproduction. In _Proceedings of the 45th International Conference on Software Engineering (ICSE)_. IEEE, 2312–2323.
  * Krippendorff (2004) Klaus Krippendorff. 2004.  Measuring the reliability of qualitative text analysis data.  _Quality and Quantity_ 38 (2004), 787–800.
  * Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _Comput. Surveys_ 55, 9 (2023), 1–35.
  * Mahajan et al. (2020) Sonal Mahajan, Negarsadat Abolhassani, and Mukul R Prasad. 2020.  Recommending stack overflow posts for fixing runtime exceptions using failure scenario matching. In _Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE)_. ACM, 1052–1064.
  * Mathews et al. (2024) Noble Saji Mathews, Yelizaveta Brus, Yousra Aafer, Mei Nagappan, and Shane McIntosh. 2024.  LLbezpeky: Leveraging Large Language Models for Vulnerability Detection.  _arXiv preprint arXiv:2401.01269_ (2024).
  * Mitrović et al. (2023) Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. 2023.  Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text.  _arXiv preprint arXiv:2301.13852_ (2023).
  * OpenAI (2023) OpenAI 2023.  _gpt-best-practices_.  OpenAI.  https://platform.openai.com/docs/guides/gpt-best-practices/six-strategies-for-getting-better-results/.
  * Pan et al. (2023) Shengyi Pan, Lingfeng Bao, Xin Xia, David Lo, and Shanping Li. 2023.  Fine-grained commit-level vulnerability type prediction by CWE tree structure. In _Proceedings of the 45th International Conference on Software Engineering (ICSE)_. IEEE, 957–969.
  * Paul et al. (2021) Rajshakhar Paul, Asif Kamal Turzo, and Amiangshu Bosu. 2021.  Why security defects go unnoticed during code reviews? a case-control study of the chromium os project. In _Proceedings of the 43rd International Conference on Software Engineering (ICSE)_. IEEE, 1373–1385.
  * Silva et al. (2023) André Silva, Sen Fang, and Martin Monperrus. 2023.  RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair.  _arXiv preprint arXiv:2312.15698_ (2023).
  * Sohail et al. (2023) Shahab Saquib Sohail, Faiza Farhat, Yassine Himeur, Mohammad Nadeem, Dag Øivind Madsen, Yashbir Singh, Shadi Atalla, and Wathiq Mansoor. 2023.  The future of gpt: A taxonomy of existing chatgpt research, current challenges, and possible future directions.  _SSRN preprint ssrn.4413921_ (2023).
  * Tao et al. (2012) Yida Tao, Yingnong Dang, Tao Xie, Dongmei Zhang, and Sunghun Kim. 2012.  How do software engineers understand code changes? An exploratory study in industry. In _Proceedings of the 20th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE)_. ACM, 1–11.
  * The MITRE Corporation (2023) The MITRE Corporation 2023.  _Common Weakness Enumeration_.  The MITRE Corporation.  https://cwe.mitre.org/.
  * Trivedi et al. (2022) Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022.  Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions.  _arXiv preprint arXiv:2212.10509_ (2022).
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022\.  Chain-of-thought prompting elicits reasoning in large language models. In _Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NeurIPS)_. PMLR, 24824–24837.
  * Wu et al. (2023) Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. 2023.  A brief overview of ChatGPT: The history, status quo and potential future development.  _IEEE/CAA Journal of Automatica Sinica_ 10, 5 (2023), 1122–1136.
  * Yu et al. (2023) Jiaxin Yu, Liming Fu, Peng Liang, Amjed Tahir, and Mojtaba Shahin. 2023.  Security Defect Detection via Code Review: A Study of the OpenStack and Qt Communities. In _Proceedings of the 17th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)_. IEEE, 1–12.
  * Yu et al. (2024) Jiaxin Yu, Peng Liang, Yujia Fu, Amjed Tahir, Mojtaba Shahin, Chong Wang, and Yangxiao Cai. 2024.  _Dataset of the Paper “Security Code Review by LLMs: A Deep Dive into Responses”_.
  * Zhang et al. (2023) Chenyuan Zhang, Hao Liu, Jiutian Zeng, Kejing Yang, Yuhong Li, and Hui Li. 2023.  Prompt-enhanced software vulnerability detection using chatgpt.  _arXiv preprint arXiv:2308.12697_ (2023).
  * Zhong et al. (2023) Zexuan Zhong, Tao Lei, and Danqi Chen. 2023.  Training language models with memory augmentation. In _Proceedings of the 27th Conference on Empirical Methods in Natural Language Processing (EMNLP)_. ACL, 5657–5673.