# Face Attribute Editing with
Disentangled Latent Vectors

Yusuf Dalva, Hamza Pehlivan, Oyku Irmak Hatipoglu, Cansu Moran, and Aysegul
Dundar  Y. Dalva, H. Pehlivan, I. Hatipoglu, C. Moran, and A. Dundar are with
the Department of Computer Science, Bilkent University, Ankara, Turkey.

###### Abstract

We propose an image-to-image translation framework for facial attribute
editing with disentangled interpretable latent directions. Facial attribute
editing task faces the challenges of targeted attribute editing with
controllable strength and disentanglement in the representations of attributes
to preserve the other attributes during edits. For this goal, inspired by the
latent space factorization works of fixed pretrained GANs, we design the
attribute editing by latent space factorization, and for each attribute, we
learn a linear direction that is orthogonal to the others. We train these
directions with orthogonality constraints and disentanglement losses. To
project images to semantically organized latent spaces, we set an encoder-
decoder architecture with attention-based skip connections. We extensively
compare with previous image translation algorithms and editing with pretrained
GAN works. Our extensive experiments show that our method significantly
improves over the state-of-the-arts. Project page:
https://yusufdalva.github.io/vecgan

###### Index Terms:

Image translation, generative adversarial networks, latent space manipulation,
face attribute editing.

##  1 Introduction

|  |  |  |  |  |
---|---|---|---|---|---|---
|  |  |  |  |  |
|  |  |  |  |  |
Input | Smile | Gender | Age | Hair Color | Eyeglasses | Bangs

Figure 1: VecGAN++ image translation results.

Facial attribute editing task has been a popular topic among the image
translation tasks, and significant improvements have been achieved with
generative adversarial networks (GANs) [4, 26, 36, 40, 20, 5]. Facial
attribute editing is one of the most challenging translation tasks because
only one attribute of a face is expected to be modified without affecting
other attributes, whereas humans are very good at detecting if a person’s
identity or any other attributes of the face change.

There are two main directions proposed for facial attribute editing, 1- end-
to-end trained image translation networks and 2- latent manipulation of
pretrained GAN networks. For the first one, different image translation
architectures are proposed with usually two networks, one for encoding style
and the other for image editing with modified styles are injected into [4,
20]. A style, an attribute, is usually encoded from another image or sampled
from a distribution. The attributes both in the encoding and editing phases
need to be disentangled. To achieve such disentanglement, works focus on style
encoding and progress from a shared style code, SDIT [33], to mixed style
codes, StarGANv2 [4], to hierarchical disentangled styles, HiSD [20]. Among
these works, HiSD independently learns styles of each attribute, bangs, hair
color, and eyeglasses, and introduces a local translator which uses attention
masks to avoid global manipulations. HiSD showcases successes on those three
local attribute editing tasks and is not tested for global attribute editing,
e.g., age, smile. Furthermore, one limitation of these works is the
uninterruptible style codes, as one cannot control the intensity of attributes
(e.g., blondness) in a straightforward manner.

The second class of methods builds on well-trained generative models,
specifically StyleGAN2 models [17], which organize their latent space as
disentangled representations with meaningful directions in a completely
unsupervised way. This approach has two steps; in the first step, an input
image is embedded into the generative model’s latent space via additional
training of an encoder or latent optimization. In the second step, the
embedded latent code is modified based on the discovered directions such that
when the edited code is decoded, an attribute is edited in the input image.
Embedding images in GAN’s space and exploring interpretable directions in
latent codes have emerged as important research endeavors on the fixed
pretrained GANs [27, 30, 9, 28, 35]. We refer to these models as StyleGAN
inversion-based methods throughout the manuscript. However, these models are
not trained end-to-end. Therefore, there is no guarantee that the encoded
image lies in the natural GAN space, which results in the codes with limited
editability. When encoders are forced to encode images into GAN’s natural
latent distribution, the results suffer from the lack of faithful
reconstruction. Additionally, since the model is not trained for this task,
there is no guarantee that interesting attributes would be disentangled (e.g.
eyeglasses versus age).

To overcome the challenges of facial attribute editing task, our previous
conference work proposed an image-to-image translation framework with
interpretable latent directions, VecGAN [5]. The attribute editing directions
of VecGAN were learned in the latent space and regularized to be orthogonal to
each other for style disentanglement. With this setup, VecGAN also provided a
knob for the controllable strength of the change, a scalar value. In this
work, we improve upon VecGAN [5] with a novel disentanglement loss and
architectural changes.

This manuscript extends its conference version [5] with the following
additions:

  * •

We introduce a disentanglement loss which stabilizes the training and improves
the results as explained in Section 3.5.

  * •

We augment our generator with attention-based skip connections. This way, the
network can pass only selected information to the decoder. The module is
explained in Section 3.3. We refer to our final model with updated
architecture and loss objective as VecGAN++. Example translation results of
VecGAN++ are given in Fig. 1.

  * •

We compare our method with several state-of-the-art image translation methods,
especially with popular pretrained GAN-based models. We provide results with
an extensive number of metrics for quality, attribute edit accuracy, identity,
and background preservation (Section 4.2). Our results show the effectiveness
of our framework with significant improvements over state-of-the-art as
provided in Section 4.3.

  * •

We provide a comprehensive analysis of our results in Section 5. We report
metrics as the strength of editing increases for our and competing methods. We
also analyze the projected style codes and show that they can classify the
targeted attributes of images, e.g. hair color, smile.

##  2 Related Work

Image to Image Translation. Image-to-image translation algorithms aim at
preserving a given content from the input image while changing targeted
attributes. They find a wide range of applications from translating semantic
maps into RGB images [32, 6, 21], RGB images to portrait drawings [38] and
very popularly to editing faces [4, 26, 36, 40, 20, 34, 7, 11, 2]. These
algorithms set an encoder-decoder architecture and train the models with
reconstruction and GAN losses [8]. When the translation is designed in a
unimodal setting, images are processed with an encoder and decoder to output
translated images from one domain to the other [32, 23]. The shortcoming of
such a setup is that a single input image may correspond to multiple possible
outputs, which makes the translation problem ambiguous. Because of that,
multi-modal image translation models are proposed in which style is encoded
separately from another image or sampled from a distribution [13, 4]. The
generator, decoder, receives style and content information. This information
is either concatenated channel-wise [41], or combined with a mask [20] or fed
separately to the decoder while content goes through the convolutional layers,
style goes through instance normalization blocks [13, 42]. These works use two
encoders, one is to extract style, and one is to encode content [20, 37].
However, in many cases, it is not clear what the content is and what the style
is. Usually, style is referred to the domain attributes one wants to change,
and the content is the rest of the attributes, which is an ambiguous problem
definition. In our work, we design the attribute as a learnable linear
direction in the latent space, and we do not employ separate style and content
encoders. Instead, we use a single encoder, resulting in a more intuitive
framework.

Editing with pretrained GANs (StyleGAN inversion-based models). Facial
attribute editing is also shown to be possible with pretrained GANs. State-of-
the-art GAN models organize their latent space with interpretable directions
[16, 17, 39] such that moving along the direction only changes one attribute
of the image. These directions are explored in supervised [27], and
unsupervised ways [30, 9, 28, 35], and many directions are found for face
editing, e.g. directions that change the smile, pose, age attributes are
found, to name a few. To edit a facial attribute of an input image, one needs
to project the image to a latent code in GANs’ latent space [1] such that the
generator reconstructs the input image from the latent code. There have been
various architectures [3, 31] and objectives proposed to project an image to
GAN’s embedding. However, they suffer from reconstruction-editability trade-
off [29]. That is if the image is faithfully reconstructed, it may not lie in
the true distribution of GANs latent space, and therefore, the directions do
not work as expected, which prevents editing the image. On the other hand, if
the projection is close to the true distribution, then the reconstruction is
poor. We also show this behavior in the Results section 4.3 when comparing our
method with state-of-the-art editing with pretrained GANs methods.

Even though these methods are not as successful as the image translation
methods, it is still quite remarkable when the generative network is only
taught to synthesize realistic images, it organizes the use of latent space
such that linear shifts on them change a specific attribute. Inspired by these
findings, our image-to-image translation framework is designed similarly such
that a linear shift in the encoded features is expected to change a single
attribute of an image. Unlike previous works, our framework is trained end-to-
end for translation task, allowing reference-guided attribute manipulation via
projection, and does not suffer from reconstruction-editability trade-off.

##  3 Method

In this section, we provide an overview of the generator architecture and the
training set-up. We follow the hierarchical labels defined by [20]. For a
single image, its attribute for tag $i\in\\{1,2,...,N\\}$ can be defined as
$j\in\\{1,2,...,M_{i}\\}$, where N is the number of tags and $M_{i}$ is the
number of attributes for tag $i$. For example, $i$ can be the tag of hair
color, and attribute $j$ can take the value of black, brown, or blonde.

Our framework has two main objectives. As the main task, we aim to be able to
perform the image-to-image translation task in a feature (tag) specific
manner. While performing this translation, as the second objective, we also
want to obtain an interpretable feature space that allows us to perform tag-
specific feature interpolation.

Figure 2: Our translator is built on the idea of interpretable latent
directions. We encode images with an Encoder to a latent representation from
which we change a selected tag ($i$), e.g. hair color with a learnable
direction $A_{i}$ and a scale $\alpha$. To calculate the scale, we subtract
the target style scale from the source style. This operation corresponds to
removing an attribute and adding an attribute. To remove the image’s
attribute, the source style is encoded and projected from the source image. To
add the target attribute, the target style scale is sampled from a
distribution mapped for the given attribute ($j$), e.g. black, blonde, or
encoded and projected from a reference image.

###  3.1 Generator Architecture

For the image-to-image translation task, we set an encoder-decoder based
architecture and latent space translation in the middle as given in Fig. 2. We
perform the translation in the encoded latent space, $e$, which is obtained by
$e=E(x)$ where $E$ refers to the encoder. The encoded features go through a
transformation $T$ which is discussed in the next section. The transformed
features are then decoded by $G$ to reconstruct the translated images. The
image generation pipeline following feature encoding is described in Eq. 1.

$\displaystyle e^{\prime}=T(e,\alpha,i)$ $\displaystyle
x^{\prime}=G(e^{\prime})$ (1)

Previous image-to-image translation networks [20, 37, 4] set a shallow
encoder-decoder architecture to translate an image while preserving the
content and a separate deep network for style encoding. In most cases, the
style encoder includes separate branches for each tag. The shallow
architecture used to translate images prevents the model from making drastic
changes in the images, which helps preserving the person’s identity. Our
framework is different as we do not employ a separate style encoder and
instead have a deep encoder-decoder architecture for translation. That is
because to be able to organize the latent space in an interpretable way, our
framework requires a full understanding of the image and, therefore, a larger
receptive field which results in a deeper network architecture. A deep
architecture with decreasing size of feature size, on the other hand, faces
the challenges of reconstructing all the fine details from the input image.

With the motivation of helping the network to preserve tag-independent
features such as the fine details from the background, we use attention-based
skip connections between our encoder and decoder as described in Section 3.3.

The architectural details of the encoder and decoder are as follows: For the
encoder, following a $1\times 1$ convolution, we use 8 successive blocks that
perform downsampling, which reduces feature map dimensions to 1x1. In our
decoder, we have an architecture symmetric to the encoder, which is composed
of 8 successive upsampling blocks. Except for the last downsampling block and
the first upsampling block, we use instance normalization denoted as (+IN).
The channels increase as {32, 64, 128, 256, 512, 512, 512, 1024, 2048} (for
output resolution $256\times 256$) in the encoder and decrease in a symmetric
way in the decoder. Each DownBlock and UpBlock has a residual block with
$3\times 3$ convolutional filters followed by a downsampling and upsampling
layer, respectively. For downsampling, we use average pooling; for upsampling,
we use nearest-neighbor. We use the LeakyReLU activation layer (with slope
0.2) and instance normalization layer in each convolutional module.

###  3.2 Translation Module

To achieve a style transformation, we perform the tag-based feature
manipulation in a linear fashion in the latent space. First, we set a feature
direction matrix $A$, which contains learnable feature directions for each
tag. In our formulation, $A_{i}$ denotes the learned feature direction for tag
$i$. The direction matrix $A$ is randomly initialized and learned during
training.

Our translation module is formulated in Eq. 2, which adds the desired shift on
top of the encoded features $e$ similar to [30].

$T(e,\alpha,i)=e+\alpha\times A_{i}$ (2)

We compute the shift by subtracting the target style from the source style as
given in Eq 3.

$\alpha=\alpha_{t}-\alpha_{s}$ (3)

Since the attributes are designed as linear steps in the learnable directions,
we find the style shift by subtracting the target attribute scale from the
source attribute scale. This way, the same target attribute $\alpha_{t}$ can
have the same impact on the translated images no matter what the attributes
were of the original images. For example, if our target scale corresponds to
brown hair, the source scale can be coming from an image with blonde or back
hair, but since we take a step for the difference of the scales, they can both
be translated to an image with the same shade of brown hair.

There are two alternative pathways to extract the target shifting scale for
feature (tag) $i$, $\alpha_{t}$. The first pathway, named the latent-guided
path, samples a $z\in\mathcal{U}[0,1)$ and applies a linear transformation
$\alpha_{t}=w_{i,j}\cdot z+b_{i,j}$, where $\alpha_{t}$ denotes sampled
shifting scale for tag $i$ and attribute $j$. We learn linear transformation
parameters $w_{i,j}$ and $b_{i,j}$ in training time. Here tag $i$ can be hair
color, and attribute $j$ can be blonde, brown, or back hair. We learn a
different transformation module for each attribute, denoted as $M_{i,j}(z)$.
Since we learn a single direction for every tag, e.g. hair color, this
transformation module can put the initially sampled $z$’s into the correct
scale in the linear line based on the target hair color attribute. As the
other alternative pathway, we encode the scalar value $\alpha_{t}$ in a
reference-guided manner. We extract $\alpha_{t}$ for tag $i$ from a provided
reference image by first encoding it into the latent space, $e_{r}$, and
projecting $e_{r}$ via by $A_{i}$ as given in Eq. 4.

$\alpha_{t}=P(e_{r},A_{i})=\dfrac{e_{r}\cdot A_{i}}{||A_{i}||}$ (4)

In the reference guidance set-up, we do not use the information of attribute
$j$, since it is encoded by the tag $i$ features of the image.

The source scale, $\alpha_{s}$, is obtained in the same way we obtain
$\alpha_{t}$ from the reference image. We perform the projection for the
corresponding tag we want to manipulate, $i$, by $P(e,A_{i})$. We formulate
our framework with the intuition that the scale controls the amount of
features to be added. Therefore, especially when the attribute is copied over
from a reference image, the amount of features that will be added will be
different based on the source image. For this reason, we find the amount of
shift by subtraction as given in Eq. 3. Our framework is intuitive and relies
on a single encoder-decoder architecture.

###  3.3 Attention-based Skip Connections

To separate encoded features into two branches based on feature relevancy for
edits, we include a skip network $S$ in our architecture. This network $S$
includes three consecutive convolutional layers that apply 1x1 convolutions
where input and output channels are set as 256. These convolutional layers are
followed by LeakyReLU activations (with a slope of 0.2), and a sigmoid
activation function follows the last one to mask features.

While encoding our input, we compute this attention mask using the encoded
features $e$ where feature dimensions are $256\times 64\times 64$. Using this
mask, we compute two feature tensors, $e^{\prime}$ and $s$, which correspond
to edit-relevant and edit-irrelevant features, respectively. We provide the
equation for our attention mechanism in equations 5 and 6.

$e^{\prime}=S(e)*e$ (5)

$s=(1-S(e))*e$ (6)

Following this feature filtering step, we encode the embedding representing
image features at the highest level using $e^{\prime}$. In our decoding step,
we combine edit-irrelevant features $s$ with decoded features with a summation
of $64x\times 64$ features. This enables our encoder to focus on face-relevant
features used in our edits and separate features encoding texture details. The
attention mechanism in the full pipeline is provided in Fig. 2.

###  3.4 Training pathways

We train our network using two different paths by modifying the translation
paths defined by [20]. For each iteration to optimize our model, we sample a
tag $i$ for shift direction, a source attribute $j$ as the current attribute,
and a target attribute $\hat{j}$.

Non-translation path. To ensure that the encoder-decoder structure preserves
the images’ details, we reconstruct the input image without applying any style
shifts. The resulting image is denoted as $x_{n}$ as given in Eq. 7.

$x_{n}=G(E(x))$ (7)

Cycle-translation path. We apply a cyclic translation to ensure we get a
reversible translation from a latent guided scale. In this path, we first
apply a style shift by sampling $z\in\mathcal{U}[0,1)$ and obtaining target
$\alpha_{t}$ with $M_{i,\hat{j}}(z)$ for target attribute $\hat{j}$. The
translation uses $\alpha$ that is obtained by subtracting $\alpha_{t}$ from
the source style. The decoder generates an image, $x_{t}$, as given in Eq. 8
where $e$ is encoded features from input image $x$, $e=E(x)$.

$\displaystyle x_{t}=G(T(e,M_{i,j}(z)-P(e,i),i))$ (8)

Then by using the original image, $x$, as a reference image, we aim to
reconstruct the original image by translating $x_{t}$. Overall, this path
attempts to reverse a latent-guided style shift with a reference-guided shift.
The second translation is given in Eq. 9 where $e_{t}=E(x_{t})$.

$\displaystyle x_{c}=G(T(e_{t},P(e,i)-P(e_{t},i),i))$ (9)

In our learning objectives, we use $x_{n}$ and $x_{c}$ for reconstruction and
$x_{t}$ and $x_{c}$ for adversarial losses, and $M_{i,j}(z)$ for the shift
reconstruction loss. Details about the learning objectives are given in the
next section.

###  3.5 Learning objectives

Given an input image $x_{i,j}\in\mathcal{X}_{i,j}$, where $i$ is the tag to
manipulate and $j$ is the current attribute of the image, we optimize our
model with the following objectives. In our equations, $x_{i,j}$ is shown as
$x$.

Adversarial Objective. We learn a discriminator employing an architecture with
decreasing resolution and increasing channel size. Like the generator, we
build our discriminator with channel sizes of {32, 64, 128, 256, 512, 512,
512, 1024, 2048}, reducing the feature map dimensions to 1x1. We concatenate
the extracted style $\alpha_{t}$ from the input image to this latent code and
apply a 1x1 convolution. This final convolution is specific to each tag-
attribute pair so that the model can use this information.

During training, our generator performs a style shift either in a latent-
guided or a reference-guided way, resulting in a translated image. In our
adversarial loss, we receive feedback from the two steps of the cycle-
translation path. As the first component of the adversarial loss, we feed a
real image $x$ with tag $i$ and attribute $j$ to the discriminator as the real
example. To give adversarial feedback to the latent-guided path, we use the
intermediate image generated in the cycle-translation path, $x_{t}$. Finally,
to provide adversarial feedback to the reference-guided path, we use the final
outcome of the cycle-translation path $x_{c}$. Only $x$ acts as a real image;
both $x_{t}$ and $x_{c}$ are translated images and are treated as fake images
with different attributes. The discriminator aims to classify whether an image
is real or fake, given its tag and attribute. The objective is given in Eq.
10.

$\begin{split}\mathcal{L}_{adv}=2log(D_{i,j}(x))+log(1-D_{i,\hat{j}}(x_{t}))\\\
+log(1-D_{i,j}(x_{c}))\end{split}$ (10)

Shift Reconstruction Objective. As the cycle-consistency loss performs
reference-guided generation followed by latent-guided generation, we utilize a
loss function to make these two methods consistent with each other [18, 13,
19, 20]. Specifically, we would like to obtain the same target scale,
$\alpha_{t}$, both from the mapping and the encoded reference image generated
by the mapped $\alpha_{t}$. The loss function is given in Eq. 11.

$\mathcal{L}_{shift}=||M_{i,j}(z)-P(e_{t},i)||_{1}$ (11)

Those parameters, $M_{i,j}(z)$ and $P(e_{t},i)$, are calculated for the cycle-
translation path as given in Eq. 8 and 9.

Image Reconstruction Objective. In all of our training paths, the purpose is
to be able to re-generate the original image again. To supervise this desired
behavior, we use $L_{1}$ loss for reconstruction loss. In our formulation,
$x_{n}$ and $x_{c}$ are outputs of the non-translation and cycle-translation
paths, respectively. Formulation of this objective is provided in Eq. 12.

$\begin{split}\mathcal{L}_{rec}=||x_{n}-x||_{1}+||x_{c}-x||_{1}\end{split}$
(12)

Orthogonality Objective. To encourage the orthogonality between directions, we
use soft orthogonality regularization based on the Frobenius norm, which is
given in Eq. 13. This orthogonality further encourages a disentanglement in
the learned style directions.

$\mathcal{L}_{ortho}={\|A^{T}A-I\|_{F}}$ (13)

Figure 3: Qualitative results of bangs attribute of our final model (VecGAN++
- Ours w/ Attn. Skip + Disen.), VecGAN and HiSD. Given reference images,
methods extract reference attributes and edit input images accordingly. All
methods achieve high-quality results. VecGAN++ achieves better edit quality
compared to VecGAN. It is important to note that, HiSD learns feature-based
local translators, which is a successful approach on local edits, e.g. bangs,
eyeglasses, and hair color but not smile, age, or gender. Our method achieves
comparable visual and better quantitative results than HiSD on this local task
and can also achieve global edits.

Disentanglement Objective We intend to change the scale for the desired
semantic in each translation. As a reflection of this criteria, we penalize
the changes in the attributes that are not subjected to any translation. For
translated tag $i$, input scales $\alpha$, and edited scales
$\alpha^{\prime}$, the disentanglement loss is formulated in equation 14. In
the formulation, $\alpha_{k}$ represents the semantic scale for tag $k$.
Scales are calculated based on the projection given in Eq. 4.

$\mathcal{L}_{dis}=\sum_{k\neq i}||\alpha_{k}-\alpha_{k}^{\prime}||$ (14)

We find this disentanglement to be complementary to our orthogonality
objective. When the model is trained with additional disentanglement loss, we
observe that orthogonality loss drops to a lower value.

Full Objective. Combining all of the loss components described, we reach the
overall objective for optimization as given in Eq. 15. Additionally, we add an
$L_{1}$ loss on the matrix $A$ parameters to encourage its sparsity.

$\begin{split}\underset{E,G,M,A}{\min}\underset{D}{\max}\lambda_{a}\mathcal{L}_{adv}+\lambda_{s}\mathcal{L}_{shift}+\lambda_{r}\mathcal{L}_{rec}\\\
+\lambda_{o}(\mathcal{L}_{ortho}+\mathcal{L}_{dis})+\lambda_{sp}\mathcal{L}_{sparse}\end{split}$
(15)

We set the following parameters; $\lambda_{a}=1$, $\lambda_{rec}=1.5$,
$\lambda_{s}=1$, $\lambda_{o}=1$ and $\lambda_{sp}=0.1$. We use a learning
rate of $10^{-4}$ and train our model for 600K iterations with a batch size of
8 on a single GPU.

Method | Lat. | Ref. | Avg.
---|---|---|---
SDIT [33] | 33.73 | 33.12 | 33.42
StarGANv2 [4] | 26.04 | 25.49 | 25.77
Elegant [36] | - | 22.96 | -
HiSD [20] | 21.37 | 21.49 | 21.43
VecGAN [5] | 20.17 | 20.72 | 20.45
Ours w/ Disen. | 20.23 | 20.57 | 20.40
Ours w/ Attn. Skip + Disen. | 20.15 | 20.08 | 20.12
TABLE I: Quantitative results for Setting I. Lat: Latent guided, Ref:
Reference guided.

##  4 Experiments

1\. Smile (+) |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---
2\. Smile (+) |  |  |  |  |  |  |  |
3\. Smile (+) |  |  |  |  |  |  |  |
4\. Smile (-) |  |  |  |  |  |  |  |
5\. Smile (-) |  |  |  |  |  |  |  |
6\. Bangs (+) |  |  |  |  |  |  |  |
7\. Bangs (+) |  |  |  |  |  |  |  |
8\. Age (+) |  |  |  |  |  |  |  |
9\. Age (+) |  |  |  |  |  |  |  |
| Input | VecGAN++ | VecGAN | e4e | HFGI | HyperStyle | StyleTransformer | StyleRes

Figure 4: Qualitative results of our and competing methods. StyleGAN
inversion-based methods do not faithfully reconstruct input images. They miss
many details from the background and foreground. StyleRes achieves better
reconstruction results compared to others but not VecGAN models. VecGAN++ and
VecGAN achieve high fidelity to the originals with only targeted attributes
manipulated naturally and realistically. We also observe that VecGAN++
significantly improves over VecGAN in many examples.

###  4.1 Dataset and Settings

We train our model on CelebA-HQ dataset [22], which contains 30,000 face
images. To extensively compare with state-of-the-arts, we follow two training-
evaluation protocols as follows:

Setting I. In our first setting, we follow the set-up from HiSD [20] to
compare our method with end-to-end based image translation algorithms.
Following HiSD, we use the first 3000 images of the CelebA-HQ dataset as the
test set and 27000 as the training set. These images include annotations for
different attributes from which we use hair color, the presence of glass, and
bangs attributes for translation tasks in this setting. The images are resized
to $128\times 128$. Following the evaluation protocol proposed by HiSD [20],
we compute FID scores on the bangs addition task. For each test image without
bangs, we translate them to images with bangs with latent and reference
guidance. In latent guidance, 5 images are generated for each test image by
randomly sampled scales from a uniform distribution. Then this generated set
of images is compared with images with attribute bangs in terms of their FIDs.
FIDs are calculated for these 5 sets and averaged. We randomly pick 5
reference images for reference guidance to extract the style scale. FIDs are
calculated for these 5 sets separately and averaged.

Setting II. We use our second setting to comprehensively compare our method
with StyleGAN2-based inversion and editing methods. For this setting, the
training/test split is obtained by re-indexing each image in CelebA-HQ back to
the original CelebA and following the standard split of CelebA. This results
in 27,176 training and 2,824 test images. Our models are trained for hair
color, the presence of glasses, bangs, age, smiling, and gender attributes.
Images are resized to $256\times 256$ resolution, which is the dimension
StyleGAN2-based inversion methods use. We evaluate our model with smile
addition and removal and bangs addition attributes. Since the task of smile
addition/removal requires a high-level understanding of the input face for
modifying multiple facial components simultaneously, it is considered one of
the most challenging attributes to edit. By benchmarking our model with such
an editing task, we demonstrate the effectiveness of our framework in terms of
image understanding.

TABLE II: Quantitative results for Setting II. (+) and (-) denote the scores for adding and removing an attribute. We compare the FID scores for bangs addition. We compare various metrics for smile addition and removal. Explanations of the metrics are given in Section 4.2. We achieve better scores than StyleGAN inversion-based methods. We also show FID improvements with the disentanglement loss and additional improvements, especially on the addition of bangs with attention-based skip connections. We set the attribute strength of StyleGAN inversion-based methods to 2 and 1 for smile and bangs attributes, respectively, to report the best FID scores. We provide more analysis on that in Fig. 5. | Bangs | Smile
---|---|---
Method | FID (+) | FID (+) | FID (-) | Acc (+) | Acc (-) | Id (+) | Id (-) | BG (+) | BG (-)
e4e [29] | 53.62 | 35.01 | 37.91 | 99.9 | 99.8 | 0.4536 | 0.4382 | 0.7709 | 0.7541
HyperStyle [3] | 41.37 | 25.25 | 24.64 | 96.8 | 98.0 | 0.6544 | 0.6771 | 0.7941 | 0.7713
HFGI [31] | 40.54 | 23.49 | 26.58 | 99.2 | 96.6 | 0.5522 | 0.5290 | 0.7320 | 0.7324
StyleTransformer [12] | 44.66 | 27.64 | 32.71 | 99.9 | 98.4 | 0.5311 | 0.5173 | 0.7767 | 0.7607
StyleRes [25] | 40.13 | 20.53 | 21.63 | 99.1 | 99.2 | 0.5647 | 0.5606 | 0.8547 | 0.8720
VecGAN [5] | 36.47 | 17.70 | 20.26 | 92.7 | 65.1 | 0.6120 | 0.7727 | 0.9151 | 0.9257
Ours w/Disen. | 32.40 | 17.14 | 19.40 | 92.5 | 89.2 | 0.6048 | 0.6517 | 0.9060 | 0.9136
Ours w/ Attn. Skip + Disen. | 26.92 | 17.37 | 19.78 | 95.2 | 76.2 | 0.5588 | 0.6665 | 0.8909 | 0.9017
|  |
---|---|---
|  |
Figure 5: Plots of FID, ID, and BG metrics as we change the intensity of the
attributes and the number of steps to take for explored directions. For each
intensity, we measure the attribute accuracy in the x-axis. The first row
plots present results for smile addition, and the second row presents them for
smile removal.

Smile (-) |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---
Smile (+) |  |  |  |  |  |  |  |
Smile (-) |  |  |  |  |  |  |  |
| Input | VecGAN++ | VecGAN | e4e | HFGI | HyperStyle | StyleTransformer | StyleRes

Figure 6: Generalization results of our and competing methods. StyleGAN
inversion-based methods do not faithfully reconstruct input images. The
reconstruction problem is more severe on these out-of-domain images than the
in-domain images presented in Fig. 4.

###  4.2 Metrics

We mostly build our evaluation on the FID metric as in previous works.
Additionally, for smile manipulation, we evaluate our results on other metrics
such as smile classification accuracy, Identity Preservation, and Background
Preservation, described as follows:

Frechet Inception Distance (FID): For the FID metric [10], we calculated the
distance between the feature vectors of original and generated images, which
are obtained using the Inception-V3 model.

Accuracy (Acc): We train an image classification network for smile attribute
classification on the training split of the CelebA-HQ dataset. We use an
ImageNet pretrained ResNet-50 model and fine-tune it for this task. The model
achieves $94\%$ accuracy on the validation set for the smile attribute. We use
this classifier to evaluate the accuracy of the generated images to test if
the attribute is correctly manipulated.

Identity Preservation (Id): To calculate the Id metric, we use the
CurricularFace model [14] to calculate the similarity between the original and
generated images. The CurricularFace model uses the ResNet-101 model as a
backbone for the feature extraction. We calculate the cosine similarity score
between the features of edited and original images.

Background Preservation (BG): For the BG metric, we first use the facial
attribute segmentation masks from the CelebAMask-HQ dataset to form background
masks. Using these masks, we calculate the mean structural similarity index
between the backgrounds of the original and edited images.

###  4.3 Results

We extensively compare our results with other end-to-end image translation
methods. In Setting I, as given in Table I, we compare with SDIT [33],
StarGANv2 [4], Elegant [36], and HiSD [20] models. Among these methods, HiSD
learns a hierarchical style disentanglement, whereas StarGANv2 learns a mixed
style code. Therefore StarGANv2, when translating images, also edits other
attributes and does not strictly preserve the identity. HiSD achieves
disentangled style edits. However, HiSD learns feature-based local
translators, an approach known to be successful on local edits, e.g. bangs,
and their model is trained for bangs, eyeglasses, and hair color attributes.
VecGAN achieves significantly better quantitative results than HiSD both in
latent-guided and reference-guided evaluations, even though they are compared
on a local edit task. We further achieve improvements with disentanglement
loss. We also observe with the disentanglement loss, the training becomes more
stable. With attention-based skip connections, both latent and reference-based
FID scores improve further.

Fig. 3 shows reference-guided results of our final model, VecGAN++, VecGAN,
and HiSD. As shown in Fig. 3, methods achieve attribute disentanglement, they
do not change any other attribute of the image than the bangs tag. VecGAN++
achieves better edit quality compared to VecGAN. It is important to note that,
HiSD learns feature-based local translators, which is a successful approach on
local edits, e.g. bangs, eyeglasses, and hair color but not smile, age, or
gender. Our method achieves comparable visual and better quantitative results
than HiSD on this local task and can also achieve global edits.

In our second set-up of evaluation, we compare our method with state-of-the-
art StyleGAN2 inversion-based methods, e4e [29], HyperStyle [3], HFGI [31],
StyleTransformer [12], and StyleRes [25] in Table II. We compare the methods
on a local (bangs) and a global attribute (smile) manipulation. For the smile
attribute, we use the direction explored by the InterfaceGAN method [27]. For
the bangs attribute, we use the direction discovered by the StyleCLIP method
[24]. The strength attribute is set to $2$ to report the best FID scores. We
provide more analysis on that in Fig. 5. We also show FID improvements with
the disentanglement loss and additional improvements, especially on the
addition of bangs with attention-based skip connections.

Our method and StyleGAN inversion-based methods provide a knob to control the
editing attribute intensity. We obtain plots provided in Fig. 5 by changing
the editing attribute intensity. As we increase the intensity, edits become
more detectable. We measure that with a smile classifier. Therefore, we plot
FID, Id (Identity), and BG (Background reconstruction) scores with respect to
the attribute intensity measured by the accuracy of the classifier.
Specifically, for VecGAN++ and VecGAN, we set $\alpha_{t}$ given in Eq. 3 to
$\\{0.0,0.33,0.5,0.66,1.0\\}$. For StyleGAN inversion-based methods, we set
the strength parameter to $\\{1,2,3\\}$. These models usually set the strength
to $3$ for successful smile edits. As shown in Fig. 5, VecGAN++ and VecGAN
achieve better FID scores compared to others consistently. With the highest
strength, where the accuracy of the classifier goes to $100\%$ for all models,
we observe that FID scores for StyleGAN inversion-based model scores
drastically get worse, whereas our results are robust. VecGAN++ achieves
better FID scores than VecGAN consistently. We find the Id score getting worse
as the edit strength increases. That results from changes in the person and
the limitations of the CurricularFace model. VecGAN++ achieves significantly
better BG scores than StyleGAN inversion-based models and slightly worse than
VecGAN. We observe that VecGAN++ achieves better edit quality, as reflected in
FID scores, than VecGAN. On the other hand, VecGAN does smaller edits,
resulting in better Id and BG scores.

We provide qualitative comparisons in Fig. 4. StyleGAN inversion-based methods
do not faithfully reconstruct input images. They miss many details from the
background and foreground. StyleRes achieves better reconstruction results
compared to others but still worse than ours as measured by BG metrics. Also
note that even though HyperStyle achieves high Id scores that are comparable
to ours, their reconstructions miss many image details, making the output
images look unrealistic. VecGAN++ and VecGAN achieve high fidelity to the
originals with only targeted attributes manipulated naturally and
realistically. We also observe that VecGAN++ visibly improves over VecGAN on
many examples, especially on the second and third examples for smile and on
samples with bangs edits. Additionally, VecGAN++ achieves successful age
edits, whereas StyleGAN inversion-based methods add eyeglasses very
frequently. This shows VecGAN++ and VecGAN provide with better disentanglement
between correlated attributes, e.g. age and eyeglasses. This is because our
models are trained end-to-end with labeled datasets for this task. We provide
more analysis on these comparisons in the next section.

(a) Histogram of $\alpha_{t}$ distribution of smile tags of VecGAN.

(b) Histogram of $\alpha_{t}$ distribution of smile tags of VecGAN++.

(c) Training images plotted based on their $\alpha_{t}$ values extracted for
smiling tag from VecGAN++. Zoom in for details.

Figure 7: Analysis of $\alpha_{t}$ for smile tag.

(a) Histogram of $\alpha_{t}$ distribution of hair color tag of VecGAN.

(b) Histogram of $\alpha_{t}$ distribution of hair color tag of VecGAN++.

(c) Training images plotted based on their $\alpha_{t}$ values extracted for
hair color tag from VecGAN++. Zoom in for details.

Figure 8: Analysis of $\alpha_{t}$ for hair color tag.

##  5 Analysis and Discussions

###  5.1 Comparing End-to-end Image Translation Networks versus StyleGAN
Inversion-based Methods

We propose an end-to-end trained image translation network in this work and
extensively compare our method with StyleGAN inversion-based methods. We note
the different advantages and disadvantages of both approaches.

We observe that end-to-end trained image translation networks, especially our
proposed framework, do not suffer from the reconstruction and editability
trade-off. This trade-off is pointed out for StyleGAN inversion-based methods
[29]. That is, when the inversion parameters are optimized to reconstruct the
input images faithfully, they do not lie in the natural StyleGAN distribution
space, and therefore the edit quality gets poor for those high-fidelity
inversions. That is the advantage of our method because it is trained end-to-
end, and we learn both reconstruction and editing together.

StyleGAN inversion-based methods enjoy many editing capabilities, whereas our
framework only achieves pre-defined edits for which it is trained. Those
methods that employ pre-trained StyleGANs rely on StyleGAN’s semantically rich
feature organizations. The editing directions are discovered after StyleGAN is
trained. Some methods discover directions in supervised and unsupervised ways.
Supervised methods, e.g. InterfaceGAN, require labeled datasets the same as
ours. On the other hand, with unsupervised methods and text-based editing
methods, directions are explored for those that do not have labeled datasets.
For example, with the GANSpace method [9], editing directions are found for
different expressions, and with the StyleCLIP method [24], editing directions
are found for different hairstyles (Mohawk hairstyle, Bob-cut hairstyle, Afro
hairstyle, e.g.). That is an advantage of StyleGAN inversion-based methods.

Lastly, our method achieves better generalization results as presented in Fig.
6. We apply our image translation methods and StyleGAN inversion-based methods
that are trained on high-quality face photographs on Metface images [15]
without any tuning. Metface dataset [15] includes face images extracted from
the collection of the Metropolitan Museum of Art and exhibits a domain gap
with CelebA [22] and FFHQ [16] datasets. StyleGAN inversion based methods do
not reconstruct the input images with high fidelity due to the domain gap and
output images similar to the style of CelebA and FFHQ. Our method has the
advantage of being robust to domain gaps whereas StyleGAN inversion-based
methods require fine-tuning the StyleGAN network and inversion encoders on new
domains.

###  5.2 Analysis of Projected Styles

We explore the behavior of encoded scales from reference images, $\alpha_{t}$.
These scales are supposed to provide information about the attribute of the
image (whether a person smiles or not) and its intensity (how big the smile
is). We plot the histograms of $\alpha_{t}$ values from validation images for
smile tags and use orange and blue colors depending on their ground-truth tags
from the validation list as shown in Fig. 7(a) and Fig. 7(b) for VecGAN and
VecGAN++, respectively. For the smiling tag, $\alpha_{t}$ values are mostly
disentangled with a small intersection. For VecGAN, we remove the outliers for
visualization purposes. There are some encoded scales far away from the
clusters. On the other hand, for VecGAN++ we do not have such a problem and do
not remove any data points. Other than that, we find VecGAN and VecGAN++
extracted scale attributes to be similar. Next, we visualize the samples for
the smiling tag for VecGAN++.

Fig. 7(c) shows a visualization of validation images plotted based on their
$\alpha_{t}$ values extracted for the smiling tag. We visualize a few samples
from each bin from the histogram above with the same frequency as the
histogram value. The visualization shows that $\alpha_{t}$ values encode the
intensity of the smile. The rightmost samples have large smiles, and the
leftmost samples look almost angry. On the other hand, the images in the
middle space are confusing ones. We also observe many wrong labeling in the
CelebA-HQ dataset by going through the middle space.

We repeat the same analysis for the hair color tag as provided in Fig. 8(a)
and Fig. 8(b) for VecGAN and VecGAN++ respectively, since hair color tag is a
challenging one as it is expected to have a continuous scale with no clear
separation between classes. We observe that VecGAN struggles to separate
attributes for hair color tag, whereas VecGAN++ does a better separation even
though it may not be perfect. We also visualize the validation images based on
their $\alpha_{t}$ values extracted for hair color tag in Fig. 8(c) from
VecGAN++. The images go from black hair to brown hair to blonde hair. We
observe that the shade of hair goes lighter, but we also note that the
extracted scales are not perfect, and there is room for improvement.

##  6 Conclusion

This paper introduces VecGAN++, an image-to-image translation framework with
interpretable latent directions. This framework includes a deep encoder and
decoder architecture with latent space manipulation in between. Latent space
manipulation is designed as vector arithmetic where for each attribute, a
linear direction is learned. This design is encouraged by the finding that
well-trained generative models organize their latent space as disentangled
representations with meaningful directions in a completely unsupervised way.
Therefore, we also extensively compare our method with StyleGAN inversion-
based methods and point out their advantages and disadvantages compared to our
method. Each change in the architecture and loss function is extensively
studied and compared with state-of-the-arts. Experiments show the
effectiveness of our framework.

## References

  * [1] R. Abdal, Y. Qin, and P. Wonka.  Image2stylegan: How to embed images into the stylegan latent space?  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4432–4441, 2019.
  * [2] R. Abdal, P. Zhu, N. J. Mitra, and P. Wonka.  Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows.  ACM Transactions on Graphics (ToG), 40(3):1–21, 2021.
  * [3] Y. Alaluf, O. Tov, R. Mokady, R. Gal, and A. Bermano.  Hyperstyle: Stylegan inversion with hypernetworks for real image editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18511–18521, 2022.
  * [4] Y. Choi, Y. Uh, J. Yoo, and J.-W. Ha.  Stargan v2: Diverse image synthesis for multiple domains.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
  * [5] Y. Dalva, S. F. Altindis, and A. Dundar.  Vecgan: Image-to-image translation with interpretable latent directions.  Proceedings of the European conference on computer vision (ECCV), 2022.
  * [6] A. Dundar, K. Sapra, G. Liu, A. Tao, and B. Catanzaro.  Panoptic-based image synthesis.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8070–8079, 2020.
  * [7] Y. Gao, F. Wei, J. Bao, S. Gu, D. Chen, F. Wen, and Z. Lian.  High-fidelity and arbitrary face editing.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16115–16124, 2021.
  * [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio.  Generative adversarial nets.  Advances in neural information processing systems, 27, 2014.
  * [9] E. Härkönen, A. Hertzmann, J. Lehtinen, and S. Paris.  Ganspace: Discovering interpretable gan controls.  Advances in Neural Information Processing Systems, 33, 2020.
  * [10] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  Advances in neural information processing systems, 30, 2017.
  * [11] X. Hou, X. Zhang, H. Liang, L. Shen, Z. Lai, and J. Wan.  Guidedstyle: Attribute knowledge guided style manipulation for semantic face editing.  Neural Networks, 145:209–220, 2022.
  * [12] X. Hu, Q. Huang, Z. Shi, S. Li, C. Gao, L. Sun, and Q. Li.  Style transformer for image inversion and editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11337–11346, June 2022.
  * [13] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz.  Multimodal unsupervised image-to-image translation.  Eur. Conf. Comput. Vis., 2018.
  * [14] Y. Huang, Y. Wang, Y. Tai, X. Liu, P. Shen, S. Li, and F. H. Jilin Li.  Curricularface: Adaptive curriculum learning loss for deep face recognition.  pages 1–8, 2020.
  * [15] T. Karras, M. Aittala, J. Hellsten, S. Laine, J. Lehtinen, and T. Aila.  Training generative adversarial networks with limited data.  Advances in Neural Information Processing Systems, 33:12104–12114, 2020.
  * [16] T. Karras, S. Laine, and T. Aila.  A style-based generator architecture for generative adversarial networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.
  * [17] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila.  Analyzing and improving the image quality of stylegan.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8110–8119, 2020.
  * [18] H.-Y. Lee, H.-Y. Tseng, J.-B. Huang, M. Singh, and M.-H. Yang.  Diverse image-to-image translation via disentangled representations.  In Proceedings of the European conference on computer vision (ECCV), pages 35–51, 2018.
  * [19] X. Li, J. Hu, S. Zhang, X. Hong, Q. Ye, C. Wu, and R. Ji.  Attribute guided unpaired image-to-image translation with semi-supervised learning.  arXiv preprint arXiv:1904.12428, 2019.
  * [20] X. Li, S. Zhang, J. Hu, L. Cao, X. Hong, X. Mao, F. Huang, Y. Wu, and R. Ji.  Image-to-image translation via hierarchical style disentanglement.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8639–8648, 2021.
  * [21] G. Liu, A. Dundar, K. J. Shih, T.-C. Wang, F. A. Reda, K. Sapra, Z. Yu, X. Yang, A. Tao, and B. Catanzaro.  Partial convolution for padding, inpainting, and image synthesis.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022\.
  * [22] Z. Liu, P. Luo, X. Wang, and X. Tang.  Deep learning face attributes in the wild.  In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  * [23] T. Park, M.-Y. Liu, T.-C. Wang, and J.-Y. Zhu.  Semantic image synthesis with spatially-adaptive normalization.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2337–2346, 2019.
  * [24] O. Patashnik, Z. Wu, E. Shechtman, D. Cohen-Or, and D. Lischinski.  Styleclip: Text-driven manipulation of stylegan imagery.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2085–2094, 2021.
  * [25] H. Pehlivan, Y. Dalva, and A. Dundar.  Styleres: Transforming the residuals for real image editing with stylegan.  arXiv preprint arXiv:2212.14359, 2022.
  * [26] W. Shen and R. Liu.  Learning residual images for face attribute manipulation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4030–4038, 2017.
  * [27] Y. Shen, J. Gu, X. Tang, and B. Zhou.  Interpreting the latent space of gans for semantic face editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9243–9252, 2020.
  * [28] Y. Shen and B. Zhou.  Closed-form factorization of latent semantics in gans.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1532–1540, 2021.
  * [29] O. Tov, Y. Alaluf, Y. Nitzan, O. Patashnik, and D. Cohen-Or.  Designing an encoder for stylegan image manipulation.  ACM Transactions on Graphics (TOG), 40(4):1–14, 2021.
  * [30] A. Voynov and A. Babenko.  Unsupervised discovery of interpretable directions in the gan latent space.  In International Conference on Machine Learning, pages 9786–9796. PMLR, 2020.
  * [31] T. Wang, Y. Zhang, Y. Fan, J. Wang, and Q. Chen.  High-fidelity gan inversion for image attribute editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11379–11388, 2022.
  * [32] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro.  High-resolution image synthesis and semantic manipulation with conditional gans.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8798–8807, 2018.
  * [33] Y. Wang, A. Gonzalez-Garcia, J. van de Weijer, and L. Herranz.  Sdit: Scalable and diverse cross-domain image translation.  In Proceedings of the 27th ACM International Conference on Multimedia, pages 1267–1276, 2019.
  * [34] P.-W. Wu, Y.-J. Lin, C.-H. Chang, E. Y. Chang, and S.-W. Liao.  Relgan: Multi-domain image-to-image translation via relative attributes.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 5914–5922, 2019.
  * [35] Z. Wu, D. Lischinski, and E. Shechtman.  Stylespace analysis: Disentangled controls for stylegan image generation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12863–12872, 2021.
  * [36] T. Xiao, J. Hong, and J. Ma.  Elegant: Exchanging latent encodings with gan for transferring multiple face attributes.  In Proceedings of the European conference on computer vision (ECCV), pages 168–184, 2018.
  * [37] G. Yang, N. Fei, M. Ding, G. Liu, Z. Lu, and T. Xiang.  L2m-gan: Learning to manipulate latent space semantics for facial attribute editing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2951–2960, 2021.
  * [38] R. Yi, Y.-J. Liu, Y.-K. Lai, and P. L. Rosin.  Unpaired portrait drawing generation via asymmetric cycle mapping.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8217–8225, 2020.
  * [39] N. Yu, G. Liu, A. Dundar, A. Tao, B. Catanzaro, L. S. Davis, and M. Fritz.  Dual contrastive loss and attention for gans.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6731–6742, 2021.
  * [40] G. Zhang, M. Kan, S. Shan, and X. Chen.  Generative adversarial network with spatial attention for face attribute editing.  In Proceedings of the European conference on computer vision (ECCV), pages 417–432, 2018.
  * [41] J.-Y. Zhu, R. Zhang, D. Pathak, T. Darrell, A. A. Efros, O. Wang, and E. Shechtman.  Multimodal image-to-image translation by enforcing bi-cycle consistency.  In Advances in neural information processing systems, pages 465–476, 2017.
  * [42] P. Zhu, R. Abdal, Y. Qin, and P. Wonka.  Sean: Image synthesis with semantic region-adaptive normalization.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5104–5113, 2020.