# Cycle-Consistent Adversarial Autoencoders
for Unsupervised Text Style Transfer

Yufang Huang1 Wentao Zhu2 Deyi Xiong3 Yiye Zhang4 Changjian Hu1 Feiyu Xu1
1Lenovo Research AI Lab, 2Kwai Inc., 3Tianjin University, 4Cornell University
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Unsupervised text style transfer is full of challenges due to the lack of
parallel data and difficulties in content preservation. In this paper, we
propose a novel neural approach to unsupervised text style transfer which we
refer to as Cycle-consistent Adversarial autoEncoders (CAE) trained from non-
parallel data. CAE consists of three essential components: (1) LSTM
autoencoders that encode a text in one style into its latent representation
and decode an encoded representation into its original text or a transferred
representation into a style-transferred text, (2) adversarial style transfer
networks that use an adversarially trained generator to transform a latent
representation in one style into a representation in another style, and (3) a
cycle-consistent constraint that enhances the capacity of the adversarial
style transfer networks in content preservation. The entire CAE with these
three components can be trained end-to-end. Extensive experiments and in-depth
analyses on two widely-used public datasets consistently validate the
effectiveness of proposed CAE in both style transfer and content preservation
against several strong baselines in terms of four automatic evaluation metrics
and human evaluation.

##  1 Introduction

00footnotetext:  This work is licensed under a Creative Commons Attribution
4.0 International License. License details:
http://creativecommons.org/licenses/by/4.0/.

Unsupervised text style transfer is to rewrite a text in one style into a text
in another style while the content of the text remains the same as much as
possible without using any parallel data. Style transfer can be utilized in
many tasks such as personalization in dialogue systems [2018, 2019], sentiment
and word decipherment [2017], offensive language translation [2018], and data
augmentation [2017, 2018, 2020], etc.

However, there are a variety of challenges to text style transfer in practice.
First, we do not have large-scale style-to-style parallel data to train a text
style transfer model in a supervised way. Second, even with non-parallel
corpora, the inherent discrete structure of text sequences aggravates the
difficulty of learning desirable continuous representations for style transfer
[2020, 2018, 2016, 2018]. Third, it is difficult to preserve the content of a
text when its style is transferred. To obtain good content preservation for
text style transfer, various disentanglement approaches [2017, 2017, 2018,
2019] are proposed to separate the content and style of a text in the latent
space. However, content-style disentanglement is not easily achievable as
content and style typically interact with each other in texts in subtle ways
[2019].

In order to solve the issues above, we propose a cycle-consistent adversarial
autoencoders (CAE) for unsupervised text style transfer. In CAE, we learn the
representation of a text where we embed both content and style in the same
space. Such space is constructed for each style from non-parallel data. We
then transfer the learned representation from one style space to another
space. To guarantee that the content is preserved during the style transfer
procedure, the transferred representation is transferred back to the original
space to minimize the distance between its original representation and the
reversely transferred representation.

Figure 1: The architecture of the proposed CAE network.

Figure 1 demonstrates the diagram of the proposed CAE. Without loss of
generality, we discuss CAE for text transfer between two styles. Multiple
styles can be factorized into two styles [2020a]. Multimodality Specifically,
CAE is composed of three essential components: LSTM autoencoders, adversarial
style transfer networks and a cycle-consistent constraint. The LSTM
autoencoder contains an encoder $enc$ to encode a sentence $\bm{x}^{s}_{i}$
from style $s$ into a hidden representation $\bm{z}^{s}_{i}$ in the
corresponding style space and a decoder $dec$ to generate sentences from
vectors $\bm{z}^{s}_{i}$ learned by the LSTM encoder (or $\tilde{\bm{z}}_{j}$
transferred from the other style space). The adversarial style transfer
networks learn a generator $\mathcal{T}$ to generate a representation
$\tilde{\bm{z}}^{s_{1}\rightarrow s_{2}}_{i}$ in style space $s_{2}$ from
$\bm{z}^{s_{1}}_{i}$ in style space $s_{1}$, or the other way around from
style space $s_{2}$ to $s_{1}$. It also uses a discriminator to ensure that
the transferred representations belong to the corresponding style space.

The top of Figure 2 displays the original sentences and style-transferred
sentences generated by the LSTM decoder from transferred representations
produced by the generator of the adversarial style transfer network. The
cycle-consistent constraint transfers back representations to their original
space and attempts to minimize their distances, as demonstrated in the bottom
of Figure 2.

In summary, our contributions are threefold as follows.

  * •

We propose a novel end-to-end framework with three components to learn text
style transfer without using parallel data.

  * •

To the best of our knowledge, our work is the first to use the cycle-
consistent constraint in the latent representational space for unsupervised
text style transfer.

  * •

The proposed CAE are validated on two widely-used datasets: Yelp restaurant
review sentiment transfer dataset and Yahoo QA topic transfer dataset.
Extensive experiments and analyses demonstrate that CAE obtains better
performance than several state-of-the-art baselines in both style transfer and
content preservation.

Figure 2: The visualization of style transfer and cycle-consistent constraint
in CAE. Upper: sentences in one style and style-transferred sentences in
another style. Bottom: cycle-consistent constraint enforcing that sentences
transferred into a different style can be translated back to its original
meaning in its original style.

##  2 Related work

A number of text style transfer approaches have been proposed in recent years
following the pioneering study of style transfer in images [2015]. These
approaches can be roughly categorized into two strands: methods that
disentangle representations of style and content and the others that do not.

In the first line of text style transfer, Hu et al. [2017] combine a
variational autoencoder (VAE) with style discriminators to enforce that styles
can be reliably inferred back from generated sentences. Shen et al. [2017]
uses discriminators to align hidden states of the transferred samples from one
style with the true samples in the other to obtain the shared latent content
distribution. Fu et al. [2018] use an adversarial network to separate content
representations from style representations. Prabhumoye et al. [2018] fix the
machine translation model and the encoder of the back-translation model to
obtain content representations, then generate texts with classifier-guided
style-specific generators. Li et al. [2018] extract content words by deleting
style indicator words, then combine the content words with retrieved style
words to construct the final output. Xu et al. [2018] use reinforcement
learning to jointly train a neutralization module which removes style words
based on a classifier and an emotionalization module. ARAE [2018] and DAAE
[2020b] train GAN-regularized latent representations to obtain style-
independent content representations, then decodes the content representations
conditioned on style. He et al. [2020] presents a new probabilistic graphical
model for unsupervised text style transfer.

In the second line of works that avoid disentangled representations of style
and content, Lample et al. [2019] use back-translation technique on denoising
autoencoder model with latent representation pooling to control the content
preservation. Their experiments and analyses show that the content-style
disentanglement is neither necessary nor always satisfied with practical
requirements, even with the domain adversarial training that explicitly aims
at learning disentangled representations. Style Transformer [2019] uses
Transformer as a basic module to train a style transfer system. DualRL [2019]
employs a dual reinforcement learning framework with two sequence-to-sequence
models in two directions, using style classifier and back-transfer
reconstruction probability as rewards.

We follow the second line and propose a novel method that makes no assumption
on the latent representation disentanglement. But differently, we perform
style transfer in the latent representational spaces of the source and target
style. And inspired by CycleGAN [2017, 2018] which uses a cycle loss on image
style transfer to enforce the back-translation of a transferred image to be
equivalent to the original image, we also impose a cycle-consistent constraint
on our style transfer network. However, training style transfer networks with
such a cycle constraint on discrete texts is quite different from those on
images and non-trivial. In order to enable cycle training on texts, we project
texts onto adversarially regularized latent space collectively learned by the
LSTM autoencoders and adversarial transfer networks. Different from latent
cross project with Euclidean distance for semi-supervised style transfer
[2019], we construct latent CycleGAN to generate high quality sentences for
unsupervised style transfer.

##  3 CAE: Cycle-consistent Adversarial Autoencoders

Suppose we have two non-parallel text datasets
$\mathbb{X}^{1}=\\{\bm{x}^{1}_{i}\\}_{i=1}^{n}$ and
$\mathbb{X}^{2}=\\{\bm{x}^{2}_{j}\\}_{j=1}^{m}$ with different styles $s_{1}$
and $s_{2}$. The CAE employs LSTM autoencoder models to encode discrete text
sequences $\bm{x}^{1}_{i},\bm{x}^{2}_{j}$ into representations
$\bm{z}^{1}_{i},\bm{z}^{2}_{j}$ , and to generate sentences
$\tilde{\bm{x}}^{1\rightarrow 2}_{i},\tilde{\bm{x}}^{2\rightarrow 1}_{j}$
based on latent representations $\tilde{\bm{z}}^{1\rightarrow
2}_{i},\tilde{\bm{z}}^{2\rightarrow 1}_{j}$ transferred by the adversarial
transfer network $\mathcal{T}^{1\rightarrow 2},\mathcal{T}^{2\rightarrow 1}$
from $\bm{z}^{1}_{i},\bm{z}^{2}_{j}$.

###  3.1 LSTM autoencoders

We use an LSTM [1997] autoencoder to learn the latent representation of a text
for each style. The encoder employs an LSTM recurrent neural network to map
the input sequence to a latent representation with a fixed size, and the
decoder utilizes the other LSTM network to generate an output sequence from a
hidden representation [2014]. Given the $i$-$th$ input text sequence
$\bm{x}^{1}_{i}=(x_{i,1}^{1},x_{i,2}^{1},\cdots,x_{i,L}^{1})$ in style
$s_{1}$, the LSTM autoencoder for style $s_{1}$ can be formulated as:

$p(\tilde{\bm{x}}_{i}^{1}|{\bm{x}}^{1}_{i};{enc}^{1},{dec}^{1})=\prod_{t=1}^{L}p(\tilde{x}^{1}_{i,t}|\bm{z}_{i}^{1},\tilde{\bm{x}}_{i,<t}^{1};{enc}^{1},{dec}^{1})$
(1)

where
${\tilde{\bm{x}}}^{1}_{i}=({\tilde{x}}_{i,1}^{1},\cdots,{\tilde{x}}_{i,L}^{1})$
is the reconstructed sequence with the same length $L$ as $\bm{x}^{1}_{i}$ ,
$\bm{z}^{1}_{i}={enc}^{1}(\bm{x}^{1}_{i})$ is the learned latent
representation from the encoder ${enc}^{1}$, $\tilde{\bm{x}}^{1}_{i,<t}$ are
the tokens generated before $\tilde{x}^{1}_{i,t}$ and we start the decoder by
a start-of-sentence symbol “$<$bos$>$” which is $\tilde{x}^{1}_{i,<1}$.
$p(\tilde{x}_{i,t}^{1}|\bm{z}_{i}^{1},\tilde{\bm{x}}^{1}_{i,<t};{enc}^{1},{dec}^{1})$
is the softmax output from decoder ${dec}^{1}$. For style $s_{2}$, similarly,
we construct the other LSTM autoencoder with encoder ${enc}^{2}$ and decoder
${dec}^{2}$ to learn latent representation $\bm{z}^{2}_{j}$.

The LSTM autoencoder tries to reconstruct the input sequence $\bm{x}^{k}_{i}$
with the output $\tilde{\bm{x}}^{k}_{i}$ from the networks
${enc}^{k},{dec}^{k}$, where $k=1,2$ for different styles. The training
objective function for the two LSTM autoencoders can be computed as:

$\\!\mathcal{L}_{R}({enc}^{1}\\!,\\!{dec}^{1}\\!,\\!{enc}^{2}\\!,\\!{dec}^{2})\\!=\\!-\\!\frac{1}{n}\\!\sum_{i=1}^{n}\\!\log
p(\tilde{\bm{x}}^{1}_{i}\\!=\\!\\!{\bm{x}}^{1}_{i}|{\bm{x}}^{1}_{i};\\!{enc}^{1}\\!,\\!{dec}^{1})\\!-\\!\frac{1}{m}\\!\\!\sum_{j=1}^{m}\\!\log
p(\tilde{\bm{x}}^{2}_{j}\\!=\\!{\bm{x}}^{2}_{j}|{\bm{x}}^{2}_{j};\\!{enc}^{2}\\!,\\!{dec}^{2})\\!\
$ (2)

The two LSTM autoencoders transform discrete sequences into latent continuous
representations, which facilitate the style transfer models to perform style
transfer and cycle training in the continuous space.

###  3.2 Adversarial style transfer networks

Once we obtain the representations of text sequences in different styles via
LSTM autoencoders, we learn two transformation functions
$\mathcal{T}^{1\rightarrow 2}$ and $\mathcal{T}^{2\rightarrow 1}$ to map a
representation in one style to the representation in the other style in the
learned latent spaces. The style transfer in this way is formulated as:

$\displaystyle\tilde{\bm{z}}^{1\rightarrow 2}=\mathcal{T}^{1\rightarrow
2}(\bm{z}^{1}),\quad\tilde{\bm{z}}^{2\rightarrow 1}=\mathcal{T}^{2\rightarrow
1}(\bm{z}^{2})$ (3)

where $\tilde{\bm{z}}^{1\rightarrow 2}$ is the generated latent representation
in style $s_{2}$ from its original representation $z^{1}$ in style $s_{1}$ by
the transformation $\mathcal{T}^{1\rightarrow 2}$, and
$\tilde{\bm{z}}^{2\rightarrow 1}$ is the generated latent representation in
style $s_{1}$ by $\mathcal{T}^{2\rightarrow 1}$.

We use generative adversarial networks [2014] to learn the two transformation
functions. Let’s consider the learning of the transformation
$\mathcal{T}^{1\rightarrow 2}$. We regard the function
$\mathcal{T}^{1\rightarrow 2}$ as the generator that generates a
representation in style $s_{2}$ from a representation in style $s_{1}$. We
then build a discriminator $\mathcal{D}^{2}$ to distinguish representations in
style $s_{2}$ from others. The generator tries to generate a representation
that is able to fool the discriminator. The adversarial learning of the
generator $\mathcal{T}^{1\rightarrow 2}$ and the discriminator
$\mathcal{D}^{2}$ is formulated as:

$\displaystyle\\!\min_{\mathcal{T}^{1\rightarrow
2}}\max_{\mathcal{D}^{2}}\mathcal{L}_{G}(\mathcal{T}^{1\rightarrow
2},\mathcal{D}^{2})$
$\displaystyle=\mathbb{E}_{\bm{z}^{2}\sim{p_{\bm{z}^{2}}}}\left[\log\mathcal{D}^{2}(\bm{z}^{2})\right]+\mathbb{E}_{\bm{z}^{1}\sim{p_{\bm{z}^{1}}}}\left[\log(1-\mathcal{D}^{2}(\mathcal{T}^{1\rightarrow
2}(\bm{z}^{1})))\right]$ (4) $\displaystyle=$
$\displaystyle\mathbb{E}_{\bm{x}^{2}\sim{p_{data}}}\\!\\!\left[\log\\!\mathcal{D}^{2}\\!({enc}^{2}(\\!\bm{x}^{2}))\right]\\!\\!+\\!\mathbb{E}_{\bm{x}^{1}\\!\sim{p_{data}}}\\!\left[\log(\\!1-\\!\mathcal{D}^{2}\\!(\mathcal{T}^{1\rightarrow
2}({enc}^{1}(\\!\bm{x}^{1}))))\right]$

Similarly, we can derive the generative adversarial loss
$\mathcal{L}_{G}(\mathcal{T}^{2\rightarrow 1},\mathcal{D}^{1})$ for style
transformation function $\mathcal{T}^{2\rightarrow 1}$ and discriminator
$\mathcal{D}^{1}$.

###  3.3 Cycle-consistent constraint

Theoretically, the adversarial style transfer networks described above are
capable of learning many different transformation functions that can generate
outputs in the distribution identical to the target style space [2017]. This
is because that the learning of the transformation functions lacks of
sufficient constraints and the two functions are learned in a relatively
separate way according to equations (4) and (5).

In order to learn desirable transformation functions, we use a cycle-
consistent constraint to tighten the learning of the two transformation
functions $\mathcal{T}^{1\rightarrow 2}$ and $\mathcal{T}^{2\rightarrow 1}$,
which is inspired by CycleGAN [2017]. The cycle-consistent constraint expects
that a transferred representation generated by a transformation function can
be translated back to its original representation by the other transformation
function.

Given a latent representation $\bm{z}^{1}$ in style $s_{1}$, the reconstructed
latent representation through the two style transformation functions
$\mathcal{T}^{1\rightarrow 2},\mathcal{T}^{2\rightarrow 1}$ can be obtained
as:

$\tilde{\bm{z}}^{(1\rightarrow 2)\rightarrow 1}=\mathcal{T}^{2\rightarrow
1}(\tilde{\bm{z}}^{1\rightarrow 2})=\mathcal{T}^{2\rightarrow
1}(\mathcal{T}^{1\rightarrow 2}(\bm{z}^{1}))$ (5)

Similarly, we can obtain the reconstructed latent representation
$\tilde{\bm{z}}^{(2\rightarrow 1)\rightarrow 2}$ for latent representation
$\bm{z}^{2}$ in style $s_{2}$.

To constrain the transformation functions $\mathcal{T}^{2\rightarrow 1}$ and
$\mathcal{T}^{1\rightarrow 2}$, the latent representational cycle-consistent
reconstruction loss can be formulated as:

$\displaystyle\mathcal{L}_{C}(\mathcal{T}^{2\rightarrow
1},\mathcal{T}^{1\rightarrow 2})=$ $\displaystyle\mathbb{E}_{\bm{z}^{1}\sim
p_{\bm{z}^{1}}}\left[\|\tilde{\bm{z}}^{(1\rightarrow 2)\rightarrow
1}-\bm{z}^{1}\|_{1}\right]+\mathbb{E}_{\bm{z}^{2}\sim
p_{\bm{z}^{2}}}\left[\|\tilde{\bm{z}}^{(2\rightarrow 1)\rightarrow
2}-\bm{z}^{2}\|_{1}\right]$ (6) $\displaystyle=$
$\displaystyle\mathbb{E}_{\bm{x}^{1}\sim
p_{data}}\left[\|\mathcal{T}^{2\rightarrow 1}(\mathcal{T}^{1\rightarrow
2}({enc}^{1}(\bm{x}^{1})))-{enc}^{1}(\bm{x}^{1})\|_{1}\right]$
$\displaystyle+\mathbb{E}_{\bm{x}^{2}\sim
p_{data}}\left[\|\mathcal{T}^{1\rightarrow 2}(\mathcal{T}^{2\rightarrow
1}({enc}^{2}(\bm{x}^{2})))-{enc}^{2}(\bm{x}^{2})\|_{1}\right]$

where $\|\cdot\|_{1}$ is $L_{1}$ norm.

This latent representational cycle-consistent reconstruction imposes the
constraint on the adversarial style transfer networks to palliate mode-
dropping in the latent style transfer, and to improve the content preservation
in the generated sentences.

###  3.4 Training and inference

As CAE has three components in its network architecture, the end-to-end
training objective of CAE is composed of three sub-objectives and is
formulated as:

$\mathcal{L}_{CAE}\\!=\\!\lambda_{1}\\!\mathcal{L}_{R}({enc}^{1}\\!,\\!{dec}^{1}\\!,\\!{enc}^{2}\\!,\\!{dec}^{2})\\!+\\!\lambda_{2}\\!\left(\mathcal{L}_{G}(\mathcal{T}^{2\rightarrow
1},\mathcal{D}^{1})\\!+\\!\mathcal{L}_{G}(\mathcal{T}^{1\rightarrow
2},\mathcal{D}^{2})\right)\\!+\\!\lambda_{3}\\!\mathcal{L}_{C}(\mathcal{T}^{2\rightarrow
1},\mathcal{T}^{1\rightarrow 2})$ (7)

where $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ control the relative
importance of the three sub-objectives. We aim to solve:

${enc}^{1},{dec}^{1},{enc}^{2},{dec}^{2},\mathcal{T}^{2\rightarrow
1},\mathcal{T}^{1\rightarrow
2},\mathcal{D}^{1},\mathcal{D}^{2}=\arg\min_{\\{{enc}^{1},{dec}^{1},{enc}^{2},{dec}^{2},\mathcal{T}^{2\rightarrow
1},\mathcal{T}^{1\rightarrow
2}\\}}\max_{\\{\mathcal{D}^{1},\mathcal{D}^{2}\\}}\mathcal{L}_{CAE}$ (8)

For the inference, let’s consider the transfer of a text $\bm{x}^{1}_{i}$ in
style $s_{1}$ into a text in style $s_{2}$. We first obtain latent
representation $\bm{z}_{i}^{1}={enc}^{1}(\bm{x}_{i}^{1})$ using encoder
${enc}^{1}$. We then perform style transfer and obtain the transferred latent
representation $\tilde{\bm{z}}^{1\rightarrow 2}_{i}$ in style $s_{2}$ based on
equation (3). Finally, we employ the decoder ${dec}^{2}$ to generate a
transferred sequence $\tilde{\bm{x}}^{1\rightarrow
2}_{i}\sim{dec}^{2}(\tilde{\bm{z}}_{i}^{1\rightarrow 2})$ in style $s_{2}$:

${\tilde{x}^{1\rightarrow 2}_{i,t}}=\mathop{\arg\max}_{\tilde{x}^{1\rightarrow
2}_{i,t}}p(\tilde{x}_{i,t}^{1\rightarrow 2}|\tilde{\bm{z}}^{1\rightarrow
2},\tilde{\bm{x}}^{1\rightarrow 2}_{i,<t};{dec}^{2})$ (9)

where $\tilde{\bm{x}}^{1\rightarrow 2}_{i}=({\tilde{x}^{1\rightarrow
2}}_{i,1},\cdots,{\tilde{x}^{1\rightarrow 2}_{i,L^{\prime}}})$ with length
$L^{\prime}$, $p(\cdot|\bm{z},\cdot;{dec}^{2})$ is the same as equation (1)
calculated by the softmax from ${dec}^{2}$ with previous tokens. The inference
of the entire sequence $\tilde{\bm{x}}_{i}^{1\rightarrow 2}$ in style $s_{2}$
from sequence $\bm{x}^{1}_{i}$ in style $s_{1}$ is formulated as:

$\displaystyle\tilde{\bm{x}}^{1\rightarrow 2}_{i}$
$\displaystyle\sim{dec}^{2}(\tilde{\bm{z}}_{i}^{1\rightarrow
2})={dec}^{2}(\mathcal{T}^{1\rightarrow
2}(\bm{z}_{i}^{1}))={dec}^{2}(\mathcal{T}^{1\rightarrow
2}({enc}^{1}(\bm{x}_{i}^{1})))$ (10)

Similarly, we can conduct style transfer from a sequence $\bm{x}^{2}_{j}$ in
style $s_{2}$ to generate a sequence $\tilde{\bm{x}}^{2\rightarrow 1}_{j}$ in
style $s_{1}$.

##  4 Experiments

To compare our work with previous approaches to text style transfer from non-
parallel data, we conducted experiments on two text transfer tasks: sentiment
transfer on the Yelp restaurant review corpus and topic transfer on the
“Yahoo! Answers Comprehensive Questions and Answers version 1.0” dataset. We
also carried out ablation experiments to study the impact of different
components of CAE on overall performance of style transfer.

###  4.1 Experimental setup

####  4.1.1 Datasets

For the Yelp dataset, we followed the same experimental setup and used the
same dataset as Cross-aligned auto-encoder [2017] and ARAE [2018] for
sentiment transfer on the Yelp restaurant reviews. The sentiment of a review
is labeled as positive if the rating is above three; otherwise, it is labeled
as negative. We used $70\%$ of the data for training, $10\%$ for validation
and the rest for testing.

For the Yahoo QA dataset, we chose two topics for style transfer:
“Entertainment $\&$ Music” and “Politics $\&$ Government”, and extracted
questions from these two topics to construct the final dataset. The partition
ratios of this dataset for training and testing are $80\%$ and $20\%$,
respectively. To reduce the vocabulary size, we pruned the vocabulary to keep
the most frequent words and replaced other words with “$<$unk$>$”. Table 1
shows the statistics of the two datasets.

Dataset | Yelp | Yahoo
---|---|---
Styles | Positive | Negative | Entertainment $\&$ Music | Politics $\&$ Government
$\\#$Sent. | 382K | 252K | 441K | 153K
$\\#$Vocab. | 10K | 116K
$\\#$Pruned Vocab. | 10K | 30K
Table 1: Statistics of Yelp review sentiment transfer dataset and Yahoo QA
topic transfer dataset.

####  4.1.2 Baselines

We compared CAE with the following baselines: (1) LSTM autoencoder (AE): using
only LSTM autoencoders in CAE for the two styles without the style transfer
networks and cycle-consistent constraint. (2) Cross-aligned autoencoder
(Cross-aligned AE) [2017]: aligning the hidden states of autoencoders
adversarially to learn a shared latent content distribution. (3) ARAE [2018]:
adversarially training GAN-regularized prior with a classifier to obtain
style-independent content representations, then conducting style transfer
through decoders conditioned on style. (4) Template-based method [2018]:
replacing the style words of source sentence with the other style words
retrieved from target sentences. (5) Cycled reinforcement learning approach
(Cycled RL) [2018]: using reinforcement learning to jointly train a
neutralization module which removes style words based on a classifier and an
emotionalization module.

####  4.1.3 Hyper-parameter settings

The used encoders ${enc}^{1},{enc}^{2}$ and decoders ${dec}^{1},{dec}^{2}$
were LSTM networks with one hidden layer of size $h_{n}=128$ on the Yelp
review dataset and of size $h_{n}=300$ on Yahoo QA dataset. The word embedding
size was the same as the number of hidden neurons $h_{n}$. The latent
variables $\bm{z}^{1},\bm{z}^{2}$, $\tilde{\bm{z}}^{2\rightarrow 1}$ and
$\tilde{\bm{z}}^{1\rightarrow 2}$ were $L_{2}$-normalized to $1$. The
transformation functions $\mathcal{T}^{1\rightarrow
2},\mathcal{T}^{2\rightarrow 1}$ were parameterized by two-layer fully-
connected neural networks ($h_{n}$-$h_{n}$-$h_{n}$ neurons). The
discriminators $\mathcal{D}^{1},\mathcal{D}^{2}$ were two-layer fully-
connected neural networks ($h_{n}$-$h_{n}$-$1$ neurons) with hyperbolic
tangent activation function in the first layer and sigmoid activation function
for the second layer. The $\lambda_{1},\lambda_{2},\lambda_{3}$ were set as
$0.1$, $1.0$, $1.0$ respectively based on the performance on the validation
set.

####  4.1.4 Evaluation metrics

We used four automatic metrics to quantitatively evaluate the proposed CAE:
Transfer, BLEU, PPL and RPPL, which have been widely used in previous
literature [2018]. Transfer is the style transfer success rate and implemented
as a classifier which is trained by the _fastText_ library [2017]. BLEU is
used to evaluate the content preservation between the source sequence and
transferred sequence [2002]. To evaluate the fluency of the transferred
sequence, we utilized the perplexity of the generated text denoted by PPL. We
also used the reverse perplexity (RPPL) to assess the representativeness of
generated texts with respect to the underlying data distribution and to detect
the mode collapse for generative models [2018]. RPPL scores were calculated by
training an RNN language model on generated samples to evaluate the perplexity
on real-world hold-out data [2018]. We used the code from Zhao et al. [2018]
(word embedding size of $300$ with dropout $0.2$, and one-layer LSTM of size
$300$ with dropout $0.2$) to build the language models and calculate PPL and
RPPL. These four evaluation metrics together form a comprehensive evaluation
and comparison between different approaches.

We also conducted human evaluation. We randomly chose 200 instances from each
style for the human evaluation. Four human annotators can proficiently
understand English texts and have sufficient background knowledge about this
evaluation task. The annotation is blind to them in random order. They grade
all sentences with scores from one to five for style transfer, content
preservation and fluency. Following Wu et al. [2019] and Li et al. [2018], we
regard a style transfer with scores larger than or equal to four on all three
measures (style transfer, content preservation and fluency) as a successful
transfer. We calculate the percentage of successful transfers and refer to
this percentage as Suc.

Dataset | Yelp | Yahoo
---|---|---
Models | Transfer$\uparrow$ | BLEU$\uparrow$ | PPL $\downarrow$ | RPPL$\downarrow$ | Transfer | BLEU | PPL | RPPL
AE | $59.3\%$ | 37.28 | 31.9 | 68.9 | $62.0\%$ | 32.2 | 62.7 | 57.3
Template | $80.2\%$ | 52.90 | 161.1 | 64.0 | 85.3% | 58.9 | 137.5 | 178.4
Cycled RL | $81.4\%$ | 22.00 | 81.9 | 85.6 | $75.0\%$ | 20.2 | 37.5 | 305.7
Cross-aligned AE | $77.1\%$ | 17.75 | 65.9 | 124.2 | $73.4\%$ | 11.5 | 36.5 | 79.3
ARAE | $81.8\%$ | 20.18 | 27.7 | 77.0 | $77.7\%$ | 21.1 | 34.4 | 53.7
CAE | 86.9% | 22.51 | 21.6 | 57.0 | $83.8\%$ | 23.0 | 25.4 | 52.6
|  |  |  |  |  |  |  |
Table 2:  Quantitative comparisons. Left: results on Yelp restaurant review
sentiment transfer dataset, right: result on Yahoo QA topic transfer dataset.

###  4.2 Results

####  4.2.1 Yelp restaurant reviews sentiment transfer

The results are shown in Table 2 (left), from which we clearly observe that
CAE obtains better performance than the five baseline approaches.
Specifically, CAE yields improvements of $5.1$, $6.1$ and $7.0$ points over
the best baselines for sentiment transfer in terms of transfer success rate
(Transfer), fluency (PPL) and mode reservation (RPPL), respectively.

The template-based method achieves the highest BLEU score since all content
words are guaranteed to be kept by templates with only style words replaced by
retrieved words. However, it obtains the worst perplexity, indicating that it
is very difficult for the template-based method to generate fluent sentences.
By contrast, our CAE achieves the lowest perplexity due to the strong LSTM
decoders. The cycle-consistent constraint enables CAE to yield the best RPPL
as it palliates mode dropping in style transfer. Additionally, the adversarial
style transfer network constrained by the cycle consistency loss facilitates
CAE to perform well on both style transfer and content preservation.

####  4.2.2 Yahoo questions topic transfer

We further evaluated CAE against five basedlines on the Yahoo QA topic
transfer task. Results in Table 2 (right) show that CAE obtains better
Transfer ($+6.1\%$), PPL ($-9$), BLEU and RPPL than the best baseline
approaches of Cycled RL, Cross-aligned AE and ARAE, demonstrating the
advantages of CAE on style transfer, fluency and content preservation.

The template-based method again achieves the highest BLEU score but it fails
to generate meaningful and fluent sentences (very high PPL and RPPL). The
reason for this is because simply replacing the style words with unreasonable
words may generate senmantically incorrect sentences. For example, the
template-based method transfers a source sentence of “is harrison ford married
?” into “is a state in the married ?”, which is meaningless. The template-
based method also achieves the highest transfer which is different with the
results on the Yelp dataset. The reason is that it is easier to differentiate
style words from content words in the Yahoo topic dataset than the Yelp
dataset which makes it more accurate for the template-based method to
substitute style words. It can be observed that the template-based method
achieves high BLEU scores at the cost of fluency and semantic correctness.
Taking all the four metrics into consideration, we believe that our approach
performs better than all the baselines.

####  4.2.3 Human evaluation

Table 3 shows the results of human evaluation. The CAE achieves the highest
style transfer, content preservation and fluency score on both datasets. It
also obtains the highest comprehensive successful rate of style transfer in
terms of Suc.

Dataset | Yelp | Yahoo
---|---|---
Models | Style $\uparrow$ | Content $\uparrow$ | Fluency $\uparrow$ | Suc $\uparrow$ | Style | Content | Fluency | Suc
Cross-aligned AE | 4.09 | 3.99 | 4.14 | 52$\%$ | 3.90 | 3.90 | 3.85 | 40$\%$
ARAE | 4.03 | 4.15 | 4.36 | 55$\%$ | 3.99 | 4.10 | 4.25 | 51$\%$
CAE | 4.19 | 4.23 | 4.45 | 65$\%$ | 4.16 | 4.20 | 4.34 | 63$\%$
|  |  |  |  |  |  |  |
Table 3: Human evaluation. Left: Yelp dataset. Right: Yahoo dataset. Models | Transfer $\uparrow$ | BLEU $\uparrow$ | PPL $\downarrow$ | RPPL $\downarrow$
---|---|---|---|---
CAE | $86.9\%$ | 22.51 | 21.6 | 57.0
w/o cycle-consistency | $88.2\%$ | 14.5 | 24.4 | 60.6
w/o discriminators | $99.2\%$ | 0.2 | 25.7 | 701.1
|  |  |  |
Table 4: Ablation study on Yelp dataset.

###  4.3 Ablation study

We further conducted ablation experiments on the Yelp dataset to study the
effect of the cycle-consistent constraint and discriminators in CAE. Table 4
shows the results. When we disable the cycle-consistent constraint, we can
train the model successfully. However, it leads to significant drop in BLEU
and higher PPL and RPPL with marginal improvement in Transfer compared to the
full CAE, which again confirms that the cycle-consistent constraint is helpful
for content mode preservation. When we disable the discriminators, the model
cannot preserve content, obtaining terribly low BLEU and high RPPL, indicating
complete mode collapse. Without the discriminators, the CAE has no constraint
and guidance to learn the transformation functions, which is prone to mode
collapse. Serious mode collapse results in poor content preservation. We
notice that most transferred sentences in the “negative” style contain words
“not” or “disappointed”, while most generated sentences in the “positive”
style have the word “good”. The discriminators are important for preventing
the model from collapse and hence further preserving the content.

Models | Positive $\rightarrow$ Negative | Negative $\rightarrow$ Positive
---|---|---
Source | it has a great atmosphere , with wonderful service . | i have n’t received any response to anything .
ARAE | it has no taste , with a complete jerk . | i have n’t received any problems to please
CAE | it has a horrible atmosphere , with awful service . | i have been pleased with a wonder time .
Source | the steak was really juicy with my side of salsa to balance the flavor. | my corn beef hash was mushy with uncooked veggies .
ARAE | the steak was really bland with the sauce and mashed potatoes . | my boyfriend brought shrimp and eggs with mushrooms
|  | were amazing .
CAE | the steak was really dry with my sauce on the salsa . | my beef hash was juicy and tender .
|  |

Models | Entertainment $\&$ Music $\rightarrow$ Politics $\&$ Government | Politics $\&$ Government $\rightarrow$ Entertainment $\&$ Music
---|---|---
Source | how do you publish a song ? | who do you think will be next president of the u . s ?
ARAE | how do you react a song ? | who do you think will be next to u ?
CAE | how do you handle a war ? | who do you think will be next american idol ?
Source | do you know a website that you can find people who want to | what is the gdp of the us currently ?
| join bands ? |
ARAE | do you think that you can find a person who is in prison ? | what is the largest of the us currently ?
CAE | do you know what a website that you can get to join the military ? | what is the name of the band currently ?
|  |

Table 5:  Style-transferred examples. Upper: from the Yelp dataset. Bottom:
from the Yahoo dataset.

###  4.4 Analyses

####  4.4.1 Style-transferred sentences

We display some examples to look into the differences between the CAE and
previous approach ARAE in Table 5 . In the first example in Table 5, we can
clearly see that CAE correctly detects the sentiment words “great” and
“wonderful” and successfully transfers the positive sentiment to the negative
sentiment by changing the two words into negative words “horrible” and
“awful”. It is worth noting that CAE preserves the substance of the source
sentence during the successful style (sentiment) transfer. In contrast, ARAE
fails to keep the background and the major content of original sentences when
it struggles to change the style of them not only in the first example but
also in other examples.

Examples from Yahoo dataset again demonstrate the advantages of CAE in both
style transfer and content preservation over ARAE. It can be obliviously found
that CAE is able to learn the patterns of the original questions and to change
the topic from “Entertainment $\&$ Music” to “Politics $\&$ Government” or
vice versa in the frame of the learned patterns.

####  4.4.2 Comparison with the nearest neighbour sequences from training data

We compared the transferred sequences with the nearest-neighbor sequences from
training data based on Jaccard distance (word-level intersection over union
for two sequences). The results are listed in Table 6. The transferred
sentences are very different from the retrieved nearest sentences in both
syntax and semantics. Additionally, they are also very fluent. This suggests
that CAE is capable of learning the style knowledge from training instances
and generalizing the learned knowledge to generate style-transferred sentences
from given source sentences.

Transferred sentences | Nearest neighbour in training data
---|---
it has a horrible atmosphere , with awful service . | horrible atmosphere , horrible service .
definitely a waste of time for sushi in las vegas ! | best sushi in las vegas !
the steak was really dry with my sauce on the salsa . | the philly was dry with no sauce .
|
Table 6:  Comparison between transferred sentences generated by CAE and
nearest-neighbour sentences.

##  5 Conclusion

We have presented a novel approach, CAE, to unsupervised text style transfer
from non-parallel text. We learn latent representations for sequences in
different styles with LSTM autoencoders. The learned representations are
transferred from their original style to another style via adversarial
transfer networks. The transfer networks are equipped with a cycle-consistent
constraint to guarantee content preservation during style transfer.
Experiments and analyses on the Yelp and Yahoo datasets sufficiently
demonstrate the powerful style transfer ability of CAE with good fluency and
content preservation against previous methods.

## References

  * [2016]  Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio.  2016\.  Generating sentences from a continuous space.  In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 10–21. Association for Computational Linguistics.
  * [2019]  Pierre Colombo, Wojciech Witon, Ashutosh Modi, James Kennedy, and Mubbasir Kapadia.  2019\.  Affect-driven dialog generation.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3734–3743, Minneapolis, Minnesota, June. Association for Computational Linguistics.
  * [2019]  Ning Dai, Jianze Liang, Xipeng Qiu, and Xuanjing Huang.  2019\.  Style transformer: Unpaired text style transfer without disentangled latent representation.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5997–6007, Florence, Italy, July. Association for Computational Linguistics.
  * [2018]  Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan.  2018\.  Style transfer in text: Exploration and evaluation.  In Thirty-Second AAAI Conference on Artificial Intelligence, pages 663–670.
  * [2015]  Leon A Gatys, Alexander S Ecker, and Matthias Bethge.  2015\.  A neural algorithm of artistic style.  arXiv preprint arXiv:1508.06576.
  * [2014]  Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  2014\.  Generative adversarial nets.  In Advances in neural information processing systems, pages 2672–2680.
  * [2020]  Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg-Kirkpatrick.  2020\.  A probabilistic formulation of unsupervised text style transfer.  In International Conference on Learning Representations.
  * [2018]  R Devon Hjelm, Athul Paul Jacob, Tong Che, Adam Trischler, Kyunghyun Cho, and Yoshua Bengio.  2018\.  Boundary-seeking generative adversarial networks.  In International Conference on Learning Representations.
  * [1997]  Sepp Hochreiter and Jürgen Schmidhuber.  1997\.  Long short-term memory.  Neural computation, 9(8):1735–1780.
  * [2017]  Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing.  2017\.  Toward controlled generation of text.  In International Conference on Machine Learning, pages 1587–1596.
  * [2020]  Yufang Huang, Joel C. Park, Kelly M. Axsom, Lakshminarayanan Subramanian, and Yiye Zhang.  2020\.  Dice: Deep significance clustering for outcome-driven stratification.  medRxiv.
  * [2017]  Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov.  2017\.  Bag of tricks for efficient text classification.  In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 427–431. Association for Computational Linguistics.
  * [2019]  Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Y-Lan Boureau, et al.  2019\.  Multiple-attribute text rewriting.  In International Conference on Learning Representations.
  * [2018]  Juncen Li, Robin Jia, He He, and Percy Liang.  2018\.  Delete, retrieve, generate: a simple approach to sentiment and style transfer.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865–1874. Association for Computational Linguistics.
  * [2019]  Fuli Luo, Peng Li, Jie Zhou, Pengcheng Yang, Baobao Chang, Zhifang Sui, and Xu Sun.  2019\.  A dual reinforcement learning framework for unsupervised text style transfer.  In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI 2019.
  * [2018]  A. Mikołajczyk and M. Grochowski.  2018\.  Data augmentation for improving deep learning in image classification problem.  In 2018 International Interdisciplinary PhD Workshop (IIPhDW), pages 117–122, May.
  * [2018]  Cicero Nogueira dos Santos, Igor Melnyk, and Inkit Padhi.  2018\.  Fighting offensive language on social media with unsupervised text style transfer.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 189–194. Association for Computational Linguistics.
  * [2018]  Shereen Oraby, Lena Reed, Shubhangi Tandon, Sharath T.S., Stephanie Lukin, and Marilyn Walker.  2018\.  Controlling personality-based stylistic variation with neural natural language generators.  In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, pages 180–190. Association for Computational Linguistics.
  * [2002]  Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.  2002\.  Bleu: a method for automatic evaluation of machine translation.  In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318. Association for Computational Linguistics, July.
  * [2017]  Luis Perez and Jason Wang.  2017\.  The effectiveness of data augmentation in image classification using deep learning.  arXiv preprint arXiv:1712.04621.
  * [2018]  Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black.  2018\.  Style transfer through back-translation.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 866–876. Association for Computational Linguistics.
  * [2019]  Mingyue Shang, Piji Li, Zhenxin Fu, Lidong Bing, Dongyan Zhao, Shuming Shi, and Rui Yan.  2019\.  Semi-supervised text style transfer: Cross projection in latent space.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4939–4948.
  * [2017]  Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola.  2017\.  Style transfer from non-parallel text by cross-alignment.  In Advances in Neural Information Processing Systems, pages 6830–6841.
  * [2020a]  Liyue Shen, Wentao Zhu, Xiaosong Wang, Lei Xing, John M Pauly, Baris Turkbey, Stephanie Anne Harmon, Thomas Hogue Sanford, Sherif Mehralivand, Peter Choyke, et al.  2020a.  Multi-domain image completion for random missing input data.  arXiv preprint arXiv:2007.05534.
  * [2020b]  Tianxiao Shen, Jonas Mueller, Regina Barzilay, and Tommi Jaakkola.  2020b.  Educating text autoencoders: Latent representation guidance via denoising.  ICML.
  * [2019]  Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Maheswaran.  2019\.  “transforming” delete, retrieve, generate approach for controlled text style transfer.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3269–3279, Hong Kong, China, November. Association for Computational Linguistics.
  * [2014]  Ilya Sutskever, Oriol Vinyals, and Quoc V Le.  2014\.  Sequence to sequence learning with neural networks.  In Advances in neural information processing systems, pages 3104–3112.
  * [2019]  Chen Wu, Xuancheng Ren, Fuli Luo, and Xu Sun.  2019\.  A hierarchical reinforced sequence operation method for unsupervised text style transfer.  In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 4873–4883.
  * [2018]  Jingjing Xu, Xu SUN, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li.  2018\.  Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 979–988, Melbourne, Australia, July. Association for Computational Linguistics.
  * [2018]  Junbo Zhao, Yoon Kim, Kelly Zhang, Alexander Rush, and Yann LeCun.  2018\.  Adversarially regularized autoencoders.  In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5902–5911, Stockholmsmässan, Stockholm Sweden, 10–15 Jul. PMLR.
  * [2017]  Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  2017\.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In Proceedings of the IEEE international conference on computer vision, pages 2242–2251.
  * [2018]  Wentao Zhu, Xiang Xiang, Trac D Tran, Gregory D Hager, and Xiaohui Xie.  2018\.  Adversarial deep structured nets for mass segmentation from mammograms.  In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 847–850. IEEE.
  * [2020]  Wentao Zhu, Andriy Myronenko, Ziyue Xu, Wenqi Li, Holger Roth, Yufang Huang, Fausto Milletari, and Daguang Xu.  2020\.  Neurreg: Neural registration and its application to image segmentation.  In The IEEE Winter Conference on Applications of Computer Vision, pages 3617–3626.