# The Euclidean Space is Evil: Hyperbolic Attribute Editing
for Few-shot Image Generation

Lingxiao Li1 Yi Zhang2 Shuhui Wang 3

1 Columbia University 2 University of Oxford
3 Institute of Computing Technology, Chinese Academy of Sciences
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Few-shot image generation is a challenging task since it aims to generate
diverse new images for an unseen category with only a few images. Existing
methods suffer from the trade-off between the quality and diversity of
generated images. To tackle this problem, we propose Hyperbolic Attribute
Editing (HAE), a simple yet effective method. Unlike other methods that work
in Euclidean space, HAE captures the hierarchy among images using data from
seen categories in hyperbolic space. Given a well-trained HAE, images of
unseen categories can be generated by moving the latent code of a given image
toward any meaningful directions in the Poincaré disk with a fixing radius.
Most importantly, the hyperbolic space allows us to control the semantic
diversity of the generated images by setting different radii in the disk.
Extensive experiments and visualizations demonstrate that HAE is capable of
not only generating images with promising quality and diversity using limited
data but achieving a highly controllable and interpretable editing process.

##  1 Introduction

Due to the persistent development of deep learning, the task of image
generation has received signiﬁcant research attention in recent years.
Specifically, the Generative Adversarial Networks (GANs) [17] and its variants
(e.g., StyleGANv2 [29]) have succeeded in generating high-fidelity and
realistic images, requiring a large number of high-quality data for model
training. However, considering the long-tail distribution and data imbalance
widely exists among different image categories [25], it is difficult for GANs
to be trained on categories with sufficient training images to generate new
realistic and diverse images for a category with only a few images. This task
is referred to as few-shot image generation [9, 21, 25, 23, 24, 22, 12]. A
variety of tasks can benefit from improvements in few-shot image generation,
for instance, low-data detection [14] and few-shot classification [48, 51].

Figure 1: Illustration of hierarchical attribute editing in hyperbolic space.
Hyperbolic space can naturally and compactly encode semantic hierarchical
structures within a large image data corpus. Changing the high-level, _i.e_.,
category-relevant attribute $\Delta\mathbf{w}^{r}$ changes the category of an
image. While changing low-level or category-irrelevant attribute
$\Delta\mathbf{w}^{ir}$ varies images within categories.

In general, existing GAN-based few-shot image generation mechanisms can be
classified into three categories. Transfer-based methods [9, 33] introduce
meta-learning or domain adaptation on GANs to generate new images by enforcing
knowledge transfer among categories. Fusion-based methods [2, 19, 25, 23]
perform feature fusion of multiple input images in a feature space and
generate images via decoding the fused features back to image space. However,
the output is still highly similar to the source images. Transformation-based
methods [1, 24, 22, 12] find intra-category transformations or inject random
perturbations to conditional unseen category samples to generate images
without tedious fine-tuning. By representing the images in the Euclidean
feature space, the above learning mechanisms tend to be complicated, and the
generated images are often collapsed due to limited diversity.

Similar to the ubiquity of hierarchies in language [37, 50, 11], the semantic
hierarchy is also common in images [30]. As Fig. 1 shows, many of the semantic
hierarchies constructed in the language domain can be instantiated with visual
images. From the visual perspective, an image can be regarded as a collection
of attributes of multiple levels. High-level attributes, a.k.a. category-
relevant attributes, define the category of an image, such as the shape and
color of an animal [12]. For instance, in the middle row of Fig. 1, changing
the high-level attributes of the given image of a Shih-Tzu dog, the category
can be changed to a Rhodesian Ridgeback Dog. While the low-level or fine-
grained attributes, including expressions, postures, _etc_., that vary within
the category as shown in the bottom of Fig. 1, are then called category-
irrelevant attributes. Therefore, an image can also be viewed as a descendant
of another image with the same category-relevant attributes by adding fine-
grained category-irrelevant attributes to its parent image. To edit the visual
attributions for high-quality image generation, it is crucial to capture the
attribute hierarchy within the large image data corpus and find a good space
to represent them. Ideally, we aim to find a hierarchical representation of
images in a latent space that allows us to change the category of an image by
moving the latent code in a category-relevant direction and perform few-shot
image generation by moving the code in a category-irrelevant direction.

Unfortunately, the Euclidean space and its corresponding distance metrics used
by existing GAN-based methods can not facilitate the hierarchical attribute
representation, thus the design of complicated attribute disentangling and
editing mechanisms seems to be crucial for the generation quality. Inspired by
the application of hyperbolic space in images [30] and videos [49], we found
that the metrics introduced in hyperbolic geometry can naturally and compactly
encode hierarchical structures. Unlike the general affine spaces, _e.g_., the
Euclidean space, hyperbolic spaces can be viewed as the continuous analog of a
tree since tree-like graphs can be embedded in ﬁnite-dimension with minimal
distortion [37]. This property of hyperbolic space provides continuous and up
to infinite semantic levels for attribute editing, allowing us to robustly
generate diverse images with only a few images from unseen categories with
simple operations.

Based on the above findings, we propose a simple but effective Hyperbolic
Attribute Editing (HAE) method for few-shot image generation. Our method is
based on the observation that hierarchical latent code manipulation can be
easily implemented in Hyperbolic space. The core of HAE is mapping the latent
vectors from the Euclidean space $\mathbb{R}^{n}$ to a hyperbolic space
$\mathbb{D}^{n}$. We minimize a supervised classification loss function to
ensure the images are hierarchically embedded in hyperbolic space. Once we
capture the attribute hierarchy among images, we can generate new images of
unseen categories by moving the latent code from one leaf to another with the
same parents by fixing the radius. Most importantly, the hyperbolic space
allows us to control the semantic diversity of generated images by setting
different radii in the Poincaré disk. Those operations can well facilitate
continually hierarchical attribute editing in hyperbolic space for flexible
few-shot image generation with both quality and diversity.

Our contributions can be summarized as follows:

  * •

We propose a simple yet effective method for few-shot image generation,_i.e_.,
hyperbolic attribute editing. In order to capture the hierarchy among images,
we use hyperbolic space as the latent space. To the best of our knowledge, HAE
is the first attempt to use hyperbolic latent spaces for few-shot image
generation.

  * •

We show that in our designed hyperbolic latent space, the semantic
hierarchical attribute relations among images can be reflected by their
distances to the center of the Poincaré disk.

  * •

Extensive experiments and visualization suggest that HAE achieves stable few-
shot image generation with high quality and diversity. HAE allows us to
generate images with good control of diversity by changing the semantic levels
of attributes we want to edit.

Figure 2: The overview of HAE. The Hyper layer is a hyperbolic feedforward
layer called Möbius linear layer which is used to project the latent code from
Euclidean space $\mathbb{R}^{n}$ to hyperbolic space $\mathbb{D}^{n}$ [6].
$\bar{z}_{\mathbb{D}i}$ can be viewed as the “parent” or average code of
$z_{\mathbb{D}i}$ and $z_{\mathbb{D}i}^{\prime}$. One can generate diverse
images without changing the category by moving the latent code from one child
to another of the same parent in the hyperbolic space.

##  2 Related Work

Few-shot image generation Recently, diverse methods have been proposed for
few-shot image generation. The transfer-based methods [9, 33] introduce multi-
step meta-learning or domain adaptation on GANs to generate new images by
designing alternative training methods. However, the models can hardly
generate realistic images. Fusion-based methods fuse the features by matching
the random vector with the conditional images [23] or formulating the problem
as a conditional generating task [19, 25]. Nevertheless, such simple fusion
operations limit the diversity of generated images. Furthermore,
transformation-based methods [1, 24, 22, 12] can generate images with only one
conditional image. The works focus on either capturing the cross-category or
intra-category transformations by injecting random perturbations [1]. These
works capture the transformations from the image differences that are not very
consistent. Ding et al. [12] propose the “editing-based” perspective, the
intra-category transformation can be modeled as category-irrelevant image
editing based on one sample instead of pairs of samples. Most recently, Ramesh
et al. [43] utilize powerful contrastive models,  _i.e_., CLIP [42], to
develop DALL$\cdot$E 2 that can generate realistic language-guided images in a
zero-shot fashion using diffusion model trained with a prohibitive size of
training data.

Hyperbolic Embedding The use of hyperbolic space in deep learning [37, 38, 50,
49, 30] is a pioneering work in recent years. It was first used in natural
language processing for hierarchical language representation [37, 38, 50]. The
Riemannian optimization algorithms are used to optimize models in hyperbolic
space [4, 3]. As hyperbolic space is successfully applied to represent
hierarchical data, Ganea et al. [15] derives hyperbolic versions of tools in
neural networks including multinomial logistic regression, feed-forward, and
recurrent neural networks. Following this, hyperbolic geometry is used in
image embedding [30] and video prediction [49]. Most recently, a few works
further apply hyperbolic geometry to auto-encoders aiming to encode graph data
into hyperbolic space [6, 40]. However, study on constructing hyperbolic
representation in image generation has not been found in the literature.

Latent Code Manipulation It has been shown that the latent spaces of GANs are
able to encode rich semantic information [16, 27, 45]. One of the popular
approaches is finding linear directions corresponding to changes in a given
binary labeled attributes, which might be difﬁcult to obtain for new datasets
and could require manual labeling effort [45, 16, 10]. Others [7, 52, 35, 26,
8] try to find semantic directions in an unsupervised manner. For instance,
PCA is applied in the latent space to create interpretable controls for
synthesizing images [26, 8]. Most recent works [47, 46] directly compute in
close form to find the meaningful semantic direction without training and
optimization. In comparison, our work HAE focuses on attributes in different
semantic levels in the latent space rather than trying hard to find
disentangled interpretable directions as in previous works.

##  3 Method

The overall framework of HAE is shown in Fig. 2, we first give a detailed
explanation of how to get the hierarchical representations in the hyperbolic
space, and then we introduce the whole framework of HAE and describe the loss
functions.

###  3.1 Hierarchical Representation

The major issue of our study is how to obtain the hierarchical representation
from real images to facilitate editing in different semantic levels, as
illustrated in Fig. 1. Therefore, hyperbolic space is introduced as the latent
space to achieve this goal.

Unlike Euclidean spaces with their zero curvature and spherical spaces with
their positive curvature, hyperbolic spaces with negative curvature have been
shown that it is more appropriate for learning hierarchical representation
[37, 38]. Informally, hyperbolic space can be viewed as a continuous analogy
of trees [37]. One important feature of hyperbolic space is that the length
grows exponentially with its radius while linearly in Euclidean space. This
unique property allows hyperbolic space to be naturally compatible with
hierarchical data [18] including text, images, videos, _etc_.

The $n$-dimensional hyperbolic space can be formally defined as a homogeneous,
simply connected $n$-dimensional Riemannian manifold, denoted as
$\mathbb{H}^{n}$ with constant negative sectional curvature111The curvature of
the hyperbolic space $c$ is set as $-1$ in this work.. We choose to work in
the Poincaré disk from the five isometric models of hyperbolic space [5] since
it is commonly used in gradient-based learning [37, 15, 38, 50, 49, 30]. The
Poincaré disk model $\left(\mathbb{D}^{n},g^{\mathbb{D}}\right)$ is defined by
the manifold $\mathbb{D}^{n}=\left\\{x\in\mathbb{R}^{n}:\|x\|<1\right\\}$
equipped with the following Riemannian metric:

$g_{x}^{\mathbb{D}}=\lambda_{x}^{2}g^{E},$ (1)

where $\lambda_{x}=\frac{2}{1-\|x\|^{2}}$, and $g^{E}$ is the Euclidean metric
tensor $g^{E}=\mathbf{I}^{n}$. The induced distance between two points
$\mathbf{x},\mathbf{y}\in\mathbb{D}^{n}$ can be defined by:

$d_{\mathbb{D}}(\mathbf{x},\mathbf{y})=\mathrm{arccosh}\left(1+2\frac{\|\mathbf{x}-\mathbf{y}\|^{2}}{\left(1-\|\mathbf{x}\|^{2}\right)\left(1-\|\mathbf{y}\|^{2}\right)}\right).$
(2)

Figure 3: Illustration of the property of hyperbolic space on the Poincaré
disk. Given two latent codes of Spaniel $z_{\mathbb{D}}^{c_{1}}$ and
${z_{\mathbb{D}}^{c_{1}}}^{\prime}$(red dots) on the edge of Poincaré disk,
the geodesic between these two points is the brown curve rather than a
straight line in Euclidean space. Therefore, their average latent code is
calculated as $\bar{z}_{\mathbb{D}}^{c_{1}}$(pink dot) which is closer to the
center $O$ (still a Spaniel, but less fine-grained). While the latent code of
a tiger $z_{\mathbb{D}}^{c_{2}}$(blue dot) locates far from the latent code of
a Spaniel. Thus, the hyperbolic average code of tiger and Spaniel
$\bar{z}_{\mathbb{D}}^{c_{1}2}$(purple dot) is closer to the center $O$ than
$\bar{z}_{\mathbb{D}}^{c_{1}}$ which is more abstract (a feline contains
features from both tiger and Spaniel).

In the hyperboloid model, the geodesic can be defined as the curve created by
intersecting the plane defined by two points and the origin with the
hyperboloid [32]. Thus, the mean of two latent codes in hyperbolic space
locates at the mid-point of the geodesic that is closer to the origin. This is
the key desired feature of hyperbolic space, _i.e_., the mean between two leaf
embeddings is not another leaf embedding, but the parent of those two
embeddings in the hierarchy [49]. This feature allows us to generate new
images by moving the latent code from one leaf to another with the same
parents. We can also change the semantic levels of attributes by determining
how abstract their parent is.

This unique property is visualized in Fig. 3 on a 2-D Poincaré disk. The image
embedding near the edge of the ball (with a large radius) represents a more
fine-grained image while the embedding near the center (which has a smaller
radius) represents an image with abstract features (an average face).

Although the hyperbolic space shares similar features with trees, it is
continuous. In other words, there is no fixed number of hierarchy levels.
Instead, there is a continuum from very fine-grained (closer to the border of
the Poincaré disk) to very abstract (closer to the center).

###  3.2 Network Architecture

Although we aim to embed and edit real images in hyperbolic space, the whole
network does not need to be implemented in a hyperbolic manner. Instead, we
can take advantage of the number of existing GAN inversion models and
optimization algorithms that have been fine-tuned for Euclidean space.

To achieve image editing, we need to invert the image back to the latent
space. In particular, we select pSp [44] as the backbone of HAE to encode
images to the $\mathcal{W^{+}}$-space of StyleGAN2 [29]:

$\mathbf{w}_{i}=\mathtt{pSp}\left(x_{i}\right),$ (3)

where $\mathbf{w}_{i}\in\mathbb{R}^{18\times 512}$ is the corresponding latent
vector of $x_{i}$ in the $\mathcal{W^{+}}$-space.

To manipulate latent code in hyperbolic space, we need to define a bijective
map from $\mathbb{R}^{n}$ to $\mathbb{D}_{c}^{n}$ to map Euclidean vectors to
the hyperbolic space222We choose Poincaré disk model in specific. and vice
versa. This can be done by mapping to Riemannian manifolds from their
corresponding tangent spaces. A manifold is a differentiable topological space
that locally resembles Euclidean space $\mathbb{R}^{n}$ [31, 32]. For
$x\in\mathbb{D}^{n}$, one can define the tangent space
$T_{x}\mathbb{D}^{n}_{c}$ of $\mathbb{D}^{n}_{c}$ at $x$ as the first order
linear approximation of $\mathbb{D}^{n}_{c}$ around $x$. Therefore, this
bijective map can be performed by exponential and logarithmic maps.
Specifically, the exponential map
$\exp_{\mathbf{x}}^{c}:T_{\mathbf{x}}\mathbb{D}_{c}^{n}\cong\mathbb{R}^{n}\to\mathbb{D}_{c}^{n}$,
maps from the tangent spaces into the manifold. While the logarithmic map
$\log_{\mathbf{x}}^{c}:\mathbb{D}_{c}^{n}\to
T_{\mathbf{x}}\mathbb{D}_{c}^{n}\cong\mathbb{R}^{n}$ is the reverse map of the
exponential map, check more details in Appendix A.

Figure 4: One-shot image generation from HAE on Animal Faces, Flowers and
VGGFaces.

In practice, we use exponential and logarithmic maps at center 0 for the
transformation between the Euclidean and hyperbolic representations. After we
get $\mathbf{w}_{i}$ in the $\mathcal{W^{+}}$-space, we first use a Multi-
layer Perceptron (MLP) encoder to reduce the dimension of latent vectors in
Euclidean space. Then we apply an exponential map to project the Euclidean
latent code $z_{\mathbb{R}i}$ to hyperbolic space. After that, we use the
hyperbolic feed-forward layer as [15] to obtain the final hierarchical
representation $z_{\mathbb{D}}$ as shown in Fig 2:

$z_{\mathbb{D}i}=f^{\otimes_{c}}(\exp_{\mathbf{0}}^{c}(\mathtt{MLP}_{E}(\mathbf{w}_{i}))),$
(4)

where $f^{\otimes_{c}}$ is the Möbius translation of feed-forward layer $f$ as
the map from $\mathbb{D}_{c}^{n}$ to $\mathbb{D}_{c}^{m}$, we call it Möbius
linear layer.

Finally, the hyperbolic representation $z_{\mathbb{D}}$ needs to be projected
back to the $\mathcal{W^{+}}$-space of StyleGAN2. In practice, this is
achieved by applying a logarithmic map followed by an MLP decoder:

$\mathbf{w}_{i}^{\prime}=\mathtt{MLP}_{D}(\log_{\mathbf{0}}^{c}(z_{\mathbb{D}i})),$
(5)

$\mathbf{w}_{i}^{\prime}$ then will be fed into a pre-trained StyleGAN2’s
generator $G$ to reconstruct the image $x_{i}^{\prime}$.

###  3.3 Loss Function

The loss function of HAE consists of two parts: the Hyperbolic loss ensures to
get the hierarchical representation in the hyperbolic space and the
reconstruction loss guarantees the quality of reconstruction images.

Hyperbolic Loss. To learn the semantic hierarchical representation of real
images in hyperbolic space, we minimize the distance between latent codes of
images with similar categories and attributes while pushing away the latent
codes from different categories. We choose the supervised approach to achieve
this. In order to perform multi-class classiﬁcation on the Poincaré disk
defined in Sec. 3.1, one needs to generalize multinomial logistic regression
(MLR), a.k.a. softmax regression, to the Poincaré disk defined in [15](Check
Appendix A for more details). After getting the softmax probability for each
class, one can simply use negative log-likelihood loss(NLLLoss) to calculate
the hyperbolic loss:

$\mathcal{L}_{\text{hyper}}=-\frac{1}{N}\sum_{n=1}^{N}\log(p_{n}),$ (6)

where $N$ is the batch size and $p_{n}$ is the probability predicted by the
model for the correct class.

As mentioned in Sec. 3.1, the distance between points grows exponentially with
their radius in the Poincaré disk. In order to minimize Eq. 6, the latent
codes of fine-grained images will be pushed to the edge of the ball to
maximize the distances between different categories while the embedding of
abstract images (images have common features from many categories) will be
located near the center of the ball. Since hyperbolic space is continuous and
differentiable, we are able to optimize Eq. 6 with stochastic gradient
descent, which learns the hierarchy of the images.

Reconstruction Loss. In order to guarantee the quality of the generated
images, we first use the $\mathcal{L}_{2}$ loss and LPIPS loss used in pSp
[44], given image $x_{i}$:

$\mathcal{L}_{2}(x_{i})=\|x_{i}-\mathtt{HAE}(x_{i})\|_{2}.$ (7)
$\mathcal{L}_{\text{LPIPS }}(x_{i})=\|F(x_{i})-F(\mathtt{HAE}(x_{i}))\|_{2},$
(8)

where $F(\cdot)$ denotes the perceptual feature extractor.

Since the pSp encoder and StyleGAN2 generator are pre-trained, we only train
the neural layers between the encoder and generator of $\mathtt{HAE}$. To
further guarantee the network to better project back to the
$\mathcal{W^{+}}$-space, the reconstructed $\mathbf{w}_{i}^{\prime}$ should be
the same as the original $\mathbf{w}_{i}$:

$\mathcal{L}_{\text{rec}}(w_{i})=\|\mathbf{w_{i}}-\mathbf{w}_{i}^{\prime}\|_{2},$
(9)

where $\mathbf{w}_{i}^{\prime}$ can be calculated by Eq. 4 and Eq. 5.

The overall loss function is:

$\mathcal{L}=\mathcal{L}_{2}(x_{i})+\lambda_{1}\mathcal{L}_{\text{LPIPS}}+\lambda_{2}\mathcal{L}_{\text{rec}}+\lambda_{3}\mathcal{L}_{\text{hyper}},$
(10)

where $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ are constants deﬁning the
loss weights. This curated set of loss functions ensures the model learns the
hierarchical representation and reconstructs images.

Figure 5: Images generated by HAE by adding the same perturbation on the
latent code of a given image with different hyperbolic radii on Animal Faces
and Flowers.

###  3.4 Image Generation

To study the generating quality of the trained model, a straightforward way is
to generate new images via interpolation between two designated images, or
random perturbation. In hyperbolic space, the shortest path with the induced
distance between two points is given by the geodesic defined in Eq. 2. The
geodesic equation between two embeddings $z_{\mathbb{D}\mathbf{i}}$ and
$z_{\mathbb{D}\mathbf{j}}$, denoted by $\gamma_{z_{\mathbb{D}i}\rightarrow
z_{\mathbb{D}j}}(t)$, is given by

$\gamma_{z_{\mathbb{D}i}\rightarrow
z_{\mathbb{D}j}}(t)=z_{\mathbb{D}i}\oplus_{c}t\otimes_{c}\left((-z_{\mathbb{D}i})\oplus_{c}z_{\mathbb{D}j}\right),\>t\in[0,1],$
(11)

where $\oplus_{c}$ denotes the Möbius addition (details in Appendix A) with
fixed sectional curvature of the space, denoted by $c$.

We adopt the following method to achieve generating via perturbation. For a
given image $x_{i}$, we first rescale its embedding $z_{\mathbb{D}i}$ to the
desired radius $r_{\mathbb{D}}$. Then we sample a random vector from seen
categories in $z_{\mathbb{D}j}$ with radius $r_{\mathbb{D}}$ fixed and take
the geodesic between them.

Figure 6: Interpolations in hyperbolic space along the edge of the Poincaré
disk (with $r_{\mathbb{D}}=6.2126$) on three datasets. Figure 7:
Interpolations by moving the latent codes from the edge to the center of the
Poincaré disk (from fine-grained to abstract) on three datasets.

##  4 Experiment

###  4.1 Implementation Details

In the training stage, we first train a StyleGAN2 [28] and pSp [44] with seen
categories. Given a trained pSp, the MLP encoder $\mathtt{MLP}_{E}$ is an
8-layer MLP with a Leaky-ReLU activation function. The dimension of the latent
code in hyperbolic space is chosen to be $512$. More details can be found in
Appendix B.

###  4.2 Datasets

We evaluate our method on Animal Faces [34], Flowers [39], and VGGFaces [41]
following the settings described in [12].

Animal Faces. We randomly select 119 categories as seen categories for
training and leave 30 as unseen categories for testing.

Flowers. The Flowers [39] dataset is split into 85 seen categories for
training and 17 unseen categories for testing.

VGGFaces. For VGGFaces [41], we randomly select 1802 categories for training
and 572 for testing.

Hyperbolic Radius |  Accuracy |  FID($\downarrow$) |  LPIPS($\uparrow$)
---|---|---|---
baseline |  67.34 |  - |  -
6.0 |  68.22 |  46.89 |  0.4520
5.5 |  68.56 |  48.68 |  0.4651
5.0 |  69.33 |  52.08 |  0.4823
4.5 |  68.22 |  60.87 |  0.5174
4.0 |  67.67 |  65.83 |  0.5386
3.5 |  67.33 |  68.44 |  0.6034
3.0 |  66.89 |  69.40 |  0.6316
Table 1: Ablation of same perturbation on different radii on Animal Faces. Method | Settings | Flowers | Animal Faces | VGG Faces*
---|---|---|---|---
|  FID($\downarrow$) |  LPIPS($\uparrow$) |  FID($\downarrow$) |  LPIPS($\uparrow$) |  FID($\downarrow$) |  LPIPS($\uparrow$)
DAWSON [33] |  $3$-shot |  188.96 |  0.0583 |  208.68 |  0.0642 |  137.82 |  0.0769
MatchingGAN [23] |  $3$-shot |  143.35 |  0.1627 |  148.52 |  0.1514 |  118.62 |  0.1695
F2GAN [25] |  $3$-shot |  120.48 |  0.2172 |  117.74 |  0.1831 |  109.16 |  0.2125
LoFGAN [19] |  $3$-shot |  79.33 |  0.3862 |  112.81 |  0.4964 |  20.31 |  0.2869
DeltaGAN [22] |  $1$-shot |  109.78 |  0.3912 |  89.81 |  0.4418 |  80.12 |  0.3146
FUNIT [34] |  $1$-shot |  105.65 |  0.4221 |  88.07 |  0.4362 |  - |  -
Disco-FUNIT [24] |  $1$-shot |  90.12 |  0.4436 |  71.44 |  0.4511 |  - |  -
AGE [12] |  $1$-shot |  45.96 |  0.4305 |  28.04 |  0.5575 |  34.86 |  0.3294
HAE (Ours) |  $1$-shot |  64.26 |  0.4739 |  28.93 |  0.5417 |  35.93 |  0.5919
Table 2: FID($\downarrow$) and LPIPS($\uparrow$) of images generated by
different methods for unseen categories on three datasets. Bold indicates the
best results and underline indicates the second best results. VGGFaces is
marked with * because different methods report different numbers of unseen
categories on this dataset(e.g. 552 in LoFGAN, 96 in DeltaGAN, 497 in L2GAN
and 572 in AGE). Note that: FUNIT [34] and Disco-FUNIT [24] do not provide
pre-trained models on VGG Faces [41] dataset. Figure 8: Comparison between
images generated by MatchingGAN, LoFGAN, AGE and HAE on Flowers, Animal Faces
and VGGFaces. Zoom in to see the details.

###  4.3 Analysis of Hierarchical Feature Editing

We analyze the emergent properties of the learned hierarchical
representations, and how the levels of attributes relate to their locations of
latent codes in hyperbolic space.

As we mentioned in Sec. 3.1, there is a continuum from fine-grained attributes
to abstract attributes, corresponding to the points from the peripheral to the
center of the ball. We define the hyperbolic radius $r_{\mathbb{D}}$333The
radius of the Poincaré disk in our experiment is about $6.2126$ as the
hyperbolic distance of the given latent code to the center of the Poincaré
disk. To study the influence of the radius of embeddings in hyperbolic space,
we run several experiments with different choices of $r_{\mathbb{D}}$.

Hyperbolic Perturbation and Interpolation. As mentioned in Sec. 3.4, we here
demonstrate the results of perturbation and interpolation. In addition to the
choice of perturbation, we can set the intensity of the perturbation by
controlling both the step distance and radius as shown in Fig. 5. The results
show that level of semantic attributes is highly related to $r_{\mathbb{D}}$.
With the radius becoming smaller, the attributes become more abstract. We
further visualize this property of hyperbolic space by moving the latent codes
of the given image from the edge of the Poincaré disk to the center. As Fig. 7
shows, the images change from very fine-grained to very abstract (the average
of all images). The results in Fig. 6 show that we can achieve smooth
interpolation in hyperbolic space without any distortion. More visualization
of interpolation and comparison with Euclidean latent space can be found in
Appendix E.

Ablation Study on Downstream Task. We conduct data augmentation via HAE for
image classification on Animal Faces [41]. We randomly select 30, 35 and 35
images for each category as train, val and test, respectively. Following [19],
a ResNet-18 backbone is initialized from the seen categories, then the model
is fine-tuned on the unseen categories referred to as the baseline. 60 images
are generated for each unseen category as data augmentation.

The result is presented in Tab. 1 (Check Appendix C for more results). The
diversity and quality of generated images are largely controlled by the
hyperbolic radii $r_{\mathbb{D}}$. As the radius becomes smaller, HAE
generates images of higher diversity but categories (referring to high-level
attributes) also gradually change to others. The data augmentation improves
the performance of the classifier when the hyperbolic radius $r_{\mathbb{D}}$
is larger than 4. $r_{\mathbb{D}}=5$ achieves the best performance on the
classification experiment mainly because it achieves the best trade-off
between the quality and diversity. However, the performance drops when the
radius is smaller than 4. This is because the semantic attributes change too
much and thus mislead the classifier.

Figure 9: UMAP visualization of hyperbolic 2-D embeddings of Animal Faces
dataset. We observe that similar categories are clustered and positioned near
the boundary, while ambiguous samples are located near the center. Zoom in to
see the details.

###  4.4 Few-shot Image Generation

As Fig. 5 shows, the image categories will be changed when $r_{\mathbb{D}}$ is
smaller than about $4$, and the category-irrelevant attributes of images will
be changed when $r_{\mathbb{D}}$ is larger than about 5. The embeddings of
Animal Faces are visualized in 2-D Poincaré disk using UMAP [36] shown in Fig.
9. As Fig. 6 shows, the posture and the angle of the images will be changed at
the early stage of interpolation without changing the category. Thus, the
images can be generated by moving the latent code of a given image to some
randomly selected semantic direction within the cluster of the category. In
practice, we select $r_{\mathbb{D}}=6.21$ and step size of perturbation as 8
to achieve few-shot image generation. As Fig. 4 shows the diverse images
generated by adding random perturbations from seen categories. We conduct
three experiments to show that HAE can achieve promising few-shot image
generation. Check Appendix for more examples of generated images.

Quantitative Comparison with State-of-the-art. We calculate the FID [20] and
LPIPS [53] to evaluate the fidelity and diversity of the generated images
following one-shot settings in [12]. The comparison results are shown in Tab.
2. our method achieves either the best or the second-best scores on most of
the FID and LPIPS metrics compared with state-of-the-art few-shot image
generation methods, which indicates that the improvement of our method from
the semantic aspect does not affect its performance on the traditional
evaluation metric.

Qualitative Evaluation. We qualitatively compare our method with MatchingGAN
[23], LoFGAN [19], DeltaGAN [22] and AGE [12]. As shown in Fig. 8, HAE can
generate images with diversity and fine-grained details. More importantly, the
newly generated images have more semantic diversity compared with others. For
instance, the shadow and skin color of the generated faces change with the
light condition, and this effect looks more natural.

Transferability. If we move latent codes at category-irrelevant levels, the
target perturbation is transferable across all categories. We edit the images
from three categories with the same editing direction, the output images are
shown in Fig. 10. It demonstrates that HAE achieves a highly controllable and
interpretable editing process.

Figure 10: Manipulate images from different categories with the same
perturbation (target 1&2).

###  4.5 Limitations and Future Work

Although HAE achieves reliable hierarchical attribute editing in hyperbolic
space for few-shot image generation, there are several limitations.

First, the boundary of category changing is hard to be qualified since the
hierarchical levels are continuous in the hyperbolic space. Besides, the
boundaries will vary slightly on different datasets. Users need to find a
“safe” boundary by trying different radii and step sizes of perturbation
before generating new images.

Second, the performance of HAE is limited by the pretrained styleGAN and the
inversion method. If the input image can not be well embedded, the editing
will also fail. This problem can be solved by changing more powerful
backbones, e.g., ViT [13], in future work.

Finally, we use supervised learning to get the hierarchical embedding in
hyperbolic space. However, the number of images in existing datasets with
labels for generation tasks is relatively small, which makes the embeddings in
the hyperbolic space not evenly distributed. The solution to this problem is
simple, use unsupervised learning with large-scale high-quality datasets.

##  5 Conclusion

In this work, we propose a simple yet effective method HAE to edit
hierarchical attributes in hyperbolic space. After learning the semantic
hierarchy from images, our model is able to edit continuous semantic
hierarchical features of images for flexible few-shot image generation in the
hyperbolic space. Experiments demonstrate that HAE is capable of achieving not
only stable few-shot image generation with high quality and diversity but a
controllable and interpretable editing process. Future work includes the
combination of HAE and large pretrained models and applications to more
downstream tasks.

## References

  * [1] Antreas Antoniou, Amos Storkey, and Harrison Edwards.  Data augmentation generative adversarial networks.  arXiv preprint arXiv:1711.04340, 2017.
  * [2] Sergey Bartunov and Dmitry P. Vetrov.  Few-shot generative modelling with generative matching networks.  In AISTATS, 2018.
  * [3] Silvère Bonnabel.  Stochastic gradient descent on riemannian manifolds.  IEEE Transactions on Automatic Control, 58(9):2217–2229, 2013.
  * [4] Gary Bécigneul and Octavian-Eugen Ganea.  Riemannian adaptive optimization methods.  In ICLR, 2019.
  * [5] James W Cannon, William J Floyd, Richard Kenyon, and Walter R Parry.  Hyperbolic geometry.  Flavors of geometry, 31:59–115, 1997.
  * [6] Ines Chami, Rex Ying, Christopher Ré, and Jure Leskovec.  Hyperbolic graph convolutional neural networks.  In NeurIPS, page 4868–4879, 2019.
  * [7] Anton Cherepkov, Andrey Voynov, and Artem Babenko.  Navigating the gan parameter space for semantic image editing.  In CVPR, pages 3670–3679, 2021.
  * [8] Jaewoong Choi, Junho Lee, Changyeon Yoon, Jung Ho Park, Geonho Hwang, and Myungjoo Kang.  Do not escape from the manifold: Discovering the local coordinates on the latent space of gans.  In ICLR, 2022.
  * [9] Louis Clouâtre and Marc Demers.  Figr: Few-shot image generation with reptile.  arXiv:1901.02199, 2019.
  * [10] Emily Denton, Ben Hutchinson, Margaret Mitchell, and Timnit Gebru.  Detecting bias with generative counterfactual face attribute augmentation.  CoRR, abs/1906.06439, 2019.
  * [11] Bhuwan Dhingra, Chris Shallue, Mohammad Norouzi, Andrew Dai, and George Dahl.  Embedding text in hyperbolic spaces.  arXiv preprint arXiv:1806.04313, 2018.
  * [12] Guanqi Ding, Xinzhe Han, Shuhui Wang, Shuzhe Wu, Xin Jin, Dandan Tu, and Qingming Huang.  Attribute group editing for reliable few-shot image generation.  In CVPR, pages 11184–11193, 2022.
  * [13] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In ICLR, 2021.
  * [14] Kun Fu, Tengfei Zhang, Yue Zhang, Menglong Yan, Zhonghan Chang, Zhengyuan Zhang, and Xian Sun.  Meta-ssd: Towards fast adaptation for few-shot object detection with meta-learning.  IEEE Access, 7:77597–77606, 2019.
  * [15] Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann.  Hyperbolic neural networks.  In NeurIPS, pages 5345–5355, 2018.
  * [16] Lore Goetschalckx, Alex Andonian, Aude Oliva, and Phillip Isola.  Ganalyze: Toward visual deﬁnitions of cognitive image properties.  In CVPR, page 5744–5753, 2019.
  * [17] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In NeurIPS, 2014.
  * [18] Michael Gromov.  Hyperbolic groups.  In Essays in group theory, 1987.
  * [19] Zheng Gu, Wenbin Li, Jing Huo, Lei Wang, and Yang Gao.  Lofgan: Fusing local representations for fewshot image generation.  In ICCV, 2021.
  * [20] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  In NeurIPS, 2017.
  * [21] Yan Hong, Li Niu, Jianfu Zhang, Jing Liang, and Liqing Zhang.  Deltagan: Towards diverse few-shot image generation with sample-specific delta.  In CVPR, 2020.
  * [22] Yan Hong, Li Niu, Jianfu Zhang, Jing Liang, and Liqing Zhang.  Deltagan: Towards diverse few-shot image generation with sample-specific delta.  In ECCV, 2022.
  * [23] Yan Hong, Li Niu, Jianfu Zhang, and Liqing Zhang.  Matchinggan: Matching-based few-shot image generation.  In 2020 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6, 2020.
  * [24] Yan Hong, Li Niu, Jianfu Zhang, and Liqing Zhang.  Few-shot image generation using discrete content representation.  In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, page 2796–2804, New York, NY, USA, 2022. Association for Computing Machinery.
  * [25] Yan Hong, Li Niu, Jianfu Zhang, Weijie Zhao, Chen Fu, and Liqing Zhang.  F2gan: Fusing-and-filling gan for few-shot image generation.  In Proceedings of the 28th ACM International Conference on Multimedia, MM ’20, page 2535–2543. Association for Computing Machinery, 2020\.
  * [26] Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris.  Ganspace: Discovering interpretable gan controls.  In NeurIPS, 2020.
  * [27] Ali Jahanian, Lucy Chai, and Phillip Isola.  On the” steerability” of generative adversarial networks.  In ICLR, 2020.
  * [28] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In CVPR, pages 4217–4228, 2019.
  * [29] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In CVPR, pages 8107–8116, 2020.
  * [30] Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lempitsky.  Hyperbolic image embeddings.  In CVPR, pages 6417–6427, 2020.
  * [31] John M Lee.  Riemannian manifolds: an introduction to curvature.  Springer Science & Business Media, 176, 2006.
  * [32] John M Lee.  Introduction to Smooth Manifolds.  Springer, 2013.
  * [33] Weixin Liang, Zixuan Liu, and Can Liu.  Dawson: A domain adaptive few shot generation framework.  arXiv preprint arXiv:2001.00576, 2020.
  * [34] Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz.  Few-shot unsueprvised image-to-image translation.  In ICCV, 2019.
  * [35] Yu-Ding Lu, Hsin-Ying Lee, Hung-Yu Tseng, and Ming-Hsuan Yang.  Unsupervised discovery of disentangled manifolds in gans.  arXiv preprint arXiv:2011.11842, 2020.
  * [36] Leland McInnes, John Healy, Nathaniel Saul, and Lukas Grossberger.  Umap: Uniform manifold approximation and projection.  The Journal of Open Source Software, 3(29):861, 2018.
  * [37] Maximillian Nickel and Douwe Kiela.  Generative visual manipulation on the natural image manifold.  In ECCV, 2017.
  * [38] Maximillian Nickel and Douwe Kiela.  Learning continuous hierarchies in the lorentz model of hyperbolic geometry.  In ICML, 2018.
  * [39] Maria-Elena Nilsback and Andrew Zisserman.  Automated flower classification over a large number of classes.  In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722–729, 2008.
  * [40] Jiwoong Park, Junho Cho, Hyung Jin Chang, and Jin Young Choi.  Unsupervised hyperbolic representation learning via message passing auto-encoders.  In CVPR, pages 5512–5522, 2021.
  * [41] Omkar M. Parkhi, Andrea Vedaldi, and Andrew Zisserman.  Deep face recognition.  In British Machine Vision Conference, 2015.
  * [42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision.  In ICML, 2021.
  * [43] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  arXiv preprint arXiv:2204.06125, 2022.
  * [44] Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or.  Encoding in style: a stylegan encoder for image-to-image translation.  In CVPR, pages 2287–2296, 2021.
  * [45] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou.  Interpreting the latent space of gans for semantic face editing.  In CVPR, pages 9240–9249, 2020.
  * [46] Yujun Shen and Bolei Zhou.  Closed-form factorization of latent semantics in gans.  In CVPR, pages 1532–1540, 2021.
  * [47] Nurit Spingarn-Eliezer, Ron Banner, and Tomer Michaeli.  Gan ”steerability” without optimization.  In ICLR, 2021.
  * [48] Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H.S. Torr, and Timothy M. Hospedales.  Learning to compare: Relation network for few-shot learning.  In CVPR, pages 1199–1208, 2018.
  * [49] Dídac Surís, Ruoshi Liu, and Carl Vondrick.  Learning the predictability of the future.  In CVPR, pages 12602–12612, 2021.
  * [50] Alexandru Tifrea, Gary Bécigneul, and OctavianEugen Ganea.  Poincaré glove: Hyperbolic word embeddings.  In ICLR, 2019.
  * [51] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra.  Matching networks for one shot learning.  In NeurIPS, 2016.
  * [52] Andrey Voynov and Artem Babenko.  Unsupervised discovery of interpretable directions in the gan latent space.  In ICML, pages 9786–9796, 2020.
  * [53] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In CVPR, pages 586–595, 2018.

##  Supplementary Material

## Overview

This appendix is organized as follows:

Appendix A provides the mathematical formulae used in hyperbolic neural
networks. Sec 3.2 & Sec 3.3

Appendix B gives more implementation details of HAE. Sec 4.1

Appendix C shows the results of the ablation study of downstream tasks for
Animal Faces [34], Flowers [39] and VGGFaces [41]. Sec 4.3

Appendix D compares the embeddings of images in hyperbolic space and Euclidean
space. Sec 4.4

Appendix E visualizes the interpolation on different radii in the Poincaré
disk, along the geodesic, and on $\mathcal{W^{+}}$-space. Sec 4.3

Appendix F shows the images generated with different radii in the Poincaré
disk. Sec 4.3

Appendix G compares the images generated by state-of-the-art few-shot image
generation method, _i.e_. AGE [12] and our methods HAE. Sec 4.4

Appendix H gives more examples generated by HAE. Sec 4.4

##  Appendix A Hyperbolic Neural Networks

For hyperbolic spaces, since the metric is different from Euclidean space, the
corresponding calculation operators also differ from Euclidean space. Recall
that in Eq. (11), we have two operations: Möbius addition and Möbius scalar
multiplication [30], given fixed curvature $c$.

For any given vectors $x,y\in\mathbb{H}^{n}$, the Möbius addition is defined
by:

$x\oplus_{c}y=\frac{\left(1-2c\langle
x,y\rangle-c\|y\|_{2}^{2}\right)x+\left(1+c\|x\|_{2}^{2}\right)y}{1-2c\langle
x,y\rangle+c^{2}\|x\|_{2}^{2}\|y\|_{2}^{2}},$ (12)

where $\|\cdot\|$ denotes the $2$-norm of the vector, and
$\langle\cdot,\cdot\rangle$ denotes the Euclidean inner product of the
vectors.

Similarly, the Möbius scalar multiplication of a scalar $r$ and a given vector
$x\in\mathbb{H}^{n}$ is defined by:

$r\otimes_{c}x=\tan_{c}\left(r\tan_{c}^{-1}\left(\|x\|_{2}\right)\right)\frac{x}{\|x\|_{2}}.$
(13)

We also would like to give explicit forms of the exponential map and the
logarithmic map which are used in our model to achieve the translation between
hyperbolic space and Euclidean space as mentioned in Sec 3.2.

The exponential map
$\exp_{x}^{c}:T_{x}\mathbb{D}_{c}^{n}\cong\mathbb{R}^{n}\to\mathbb{D}_{c}^{n}$,
that maps from the tangent spaces into the manifold, is given by

$\exp_{x}^{c}(v):=x\oplus_{c}\left(\tanh\left(\sqrt{c}\frac{\lambda_{x}^{c}\|v\|}{2}\right)\frac{v}{\sqrt{c}\|v\|}\right).$
(14)

The logarithmic map $\log_{x}^{c}(y):\mathbb{D}_{c}^{n}\to
T_{x}\mathbb{D}_{c}^{n}\cong\mathbb{R}^{n}$ is given by

$\log_{x}^{c}(y):=\frac{2}{\sqrt{c}\lambda_{x}^{c}}\operatorname{arctanh}\left(\sqrt{c}\left\|-x\oplus_{c}y\right\|\right)\frac{-x\oplus_{c}y}{\left\|-x\oplus_{c}y\right\|}.$
(15)

For the multinomial logistic regression (MLR) [15] mentioned in Sec 3.3, the
softmax probability can be computed as: Given $K$ classes and
$k\in\\{1,\ldots,K\\},p_{k}\in\mathbb{D}_{c}^{n},a_{k}\in
T_{p_{k}}\mathbb{D}_{c}^{n}\backslash\\{\mathbf{0}\\}$ :

$\displaystyle p(y=k\mid x)\propto\exp$
$\displaystyle\Biggl{(}\frac{\lambda_{p_{k}}^{c}\left\|a_{k}\right\|}{\sqrt{c}}\sinh^{-1}$
(16) $\displaystyle\biggl{(}\frac{2\sqrt{c}\left\langle-
p_{k}\oplus_{c}x,a_{k}\right\rangle}{\left(1-c\left\|-p_{k}\oplus_{c}x\right\|^{2}\right)\left\|a_{k}\right\|}\biggl{)}\Biggl{)},$
$\displaystyle\quad\forall x\in\mathbb{D}_{c}^{n}.$

where $\oplus_{c}$ denotes the Möbius addition defined in Eq. 12 with fixed
sectional curvature of the space, denoted by $c$.

##  Appendix B Implementation Details and Analysis

As mentioned in Sec 3.2, the output of $\mathtt{pSp}$:
$\mathbf{w}_{i}\in\mathbb{R}^{18\times 512}$. The MLP encoder
$\mathtt{MLP}_{E}$ used in HAE, is split into three parts:
$\mathrm{encoder}_{low}$, $\mathrm{encoder}_{mid}$, and
$\mathrm{encoder}_{high}$ for encoding lower layer attributes, middle layer
attributes, and higher layer attributes. The $\mathrm{encoder}_{low}$ is a
5-layer MLP with a Leaky-ReLU(slope=0.2) activation function. The first three
layers of $\mathbf{w}_{i}$ are then fed into $\mathrm{encoder}_{low}$. The
dimension of the output attribute is 128. Similar to $\mathrm{encoder}_{low}$,
the $\mathrm{encoder}_{mid}$ is also an 5-layer MLP. 3-7 layers of
$\mathbf{w}_{i}$ is then fed into $\mathrm{encoder}_{mid}$, the dimension of
the output attribute is also 128. Different from $\mathrm{encoder}_{low}$ and
$\mathrm{encoder}_{mid}$, $\mathrm{encoder}_{high}$ is an 8-layer MLP. And we
fed the last 12 layers attributes of $\mathbf{w}_{i}$ into it. The dimension
of the output attribute of $\mathrm{encoder}_{high}$ is 256. Therefore, the
final dimension of the Euclidean latent code $z_{\mathbb{R}i}$ is
128+128+256=512. While the MLP decoder $\mathtt{MLP}_{D}$ is the reversed
version of $\mathtt{MLP}_{E}$, taking
$z_{\mathbb{R}i}^{\prime}\in\mathbb{R}^{512}$ as input and output
$\mathbf{w}_{i}^{\prime}\in\mathbb{R}^{18\times 512}$.

During the training process, the constants defined in Eq. (10) are set as
$\lambda_{1}=1$, $\lambda_{3}=0.3$, and $\lambda_{2}$ changes dynamically
based on the value of $\mathcal{L}_{\text{rec}}$ which guarantees that
$0.6\geq\lambda_{2}$ and $\mathcal{L}_{\text{rec}}\geq 0.3$. Besides, we
employ Adam optimizer with a learning rate of $1\mathrm{e}{-4}$, and the batch
size is set to 8.

In addition, as a remark, we choose the largest radius as $6$ in most of our
experiments as in hyperbolic space since any vector asymptotically lying on
the surface unit $N$-sphere will have a hyperbolic length of approximately
$r=6.2126$, which can be directly calculated by Eq. (2).

Finally, we want to show that HAE can be easily trained. The size of trainable
parameters of HAE is around one hundred million which is small compared with
other models with billions of parameters. It can be trained well within one
day using a single NVIDIA TITAN RTX.

##  Appendix C Ablation Study of Downstream Task

Similar to the ablation study in Sec 4.3. We also conduct data augmentation
via HAE for image classification on Flowers [39] and VGGFaces [41]. Due to the
limited size of Flowers and VGGFaces datasets. We randomly select 10, 15 and
15 images for each category as train, val and test, respectively. Following
[19, 12], a ResNet-18 backbone is initialized from the seen categories, then
the model is fine-tuned on the unseen categories referred to as the baseline.
30 images are generated for each unseen category as data augmentation.

The result of Flowers is presented in Tab. 3. Similar to the result of Animal
Faces, the diversity and quality of generated images are largely controlled by
the hyperbolic radii $r_{\mathbb{D}}$. As the radius becomes smaller, HAE
generates images of higher diversity but categories also gradually change to
others. $r_{\mathbb{D}}=6$ achieves the best performance on the classification
experiment.

However, Tab. 4 shows that all accuracy drops when we do data augmentation on
VGGFaces dataset. We estimate that this is due to the low quality of inversion
that harms the performance of the classifier. Besides, since we only select 10
images for each category for training, with the limited size of VGGFaces, it
is easy to overfit. To evaluate our estimation, we further test the
performance of the classifier trained by the original images and inversion
images without any perturbation, denoted as inversion in Tab. 4. This result
proves that our estimation is correct. The accuracy of the classifier trained
by augmented images increases compared with the inversion, which shows that
the augmentation still works and improves the generalization performance of
our classifier. $r_{\mathbb{D}}=3$ achieves the best performance on the
classification experiment except the baseline.

It is also worth noticing that, the FID and LPIPS scores inTab. 3 and Tab. 4
are different from the scores we calculated in Sec 4.4. That is because we
only use a very small subset of the data in this ablation study which can not
represent the distribution of all images in the test dataset. Besides, the
improvement of a classifier trained on augmented data is trivial, we believe
this is due to the limitation of the encoding method,_i.e_. psp [44] and
generator,_i.e_. StyleGAN2 [29], which can not generate images with high
enough quality.

Hyperbolic Radius |  Accuracy |  FID($\downarrow$) |  LPIPS($\uparrow$)
---|---|---|---
baseline |  71.76 |  - |  -
6.0 |  79.21 |  94.35 |  0.4640
5.5 |  77.25 |  98.09 |  0.4871
5.0 |  75.29 |  97.81 |  0.5110
4.5 |  78.82 |  97.53 |  0.5330
4.0 |  75.29 |  97.58 |  0.5499
3.5 |  73.33 |  101.52 |  0.6152
3.0 |  72.55 |  105.05 |  0.6439
Table 3: Ablation of same perturbation on different radii on Flowers. Hyperbolic Radius |  Accuracy |  FID($\downarrow$) |  LPIPS($\uparrow$)
---|---|---|---
baseline |  77.99 |  - |  -
inversion |  69.53 |  25.46 |  0.2325
6.0 |  71.98 |  26.19 |  0.2702
5.5 |  72.53 |  26.46 |  0.2887
5.0 |  72.45 |  26.83 |  0.3080
4.5 |  72.32 |  26.92 |  0.3258
4.0 |  72.96 |  27.02 |  0.3405
3.5 |  74.05 |  26.35 |  0.4044
3.0 |  74.44 |  25.90 |  0.4411
Table 4: Ablation of same perturbation on different radii on VGGFaces.

##  Appendix D Comparison with Euclidean space

In this section, we mainly compare hyperbolic space with Euclidean space by
UMAP visualization [36], which is an extension of our analysis in Sec 4.4.
Following the UMAP visualization on hyperbolic space for the Animal Faces [41]
dataset, we first show the UMAP visualization of the embeddings of images in
$\mathcal{W}^{+}$-space, where each embedding is of $18\times 512$-dimension
and therefore we resize them for UMAP calculation. The results for Euclidean
UMAP of Animal Faces dataset are shown in Fig. 11. We observe that although
the transition across different categories is smooth, there are no obvious
clusters in the plot even for some significantly different species (_e.g_.,
polar bears and foxes).

Figure 11: UMAP visualization for Animal Faces dataset, while the embeddings
are in the $\mathcal{W^{+}}$-space of the same model.

We further carry on the experiments on the other two datasets. For Flowers
[39] dataset, we use all images in the test dataset to generate the embeddings
for both spaces, where there are 101 classes in total. The number of images in
each class varies due to the original setting of the dataset. The results of
the Flowers dataset are shown in Fig. 12, where clusters are more obvious in
hyperbolic space and the similarity between classes is also well-reflected.

For the VGGFaces dataset [41], the results are shown in Fig. 13. Similarly,
the clusters are better represented in hyperbolic settings. We observe that in
hyperbolic space, images with similar low-level attributes (_e.g_., wearing
black frame glasses, having mutton chops beard) are clustered. We need to pay
attention to the small clusters in both plots where images are represented
with tall rectangles (in the plot). These images do not share semantic
attributes but are clustered together, which can be the influence of heavy
watermarks on the images. This also encourages us to train the model with
high-quality datasets for better GAN inversions.

##  Appendix E Interpolation Visualization

In this section, we offer a more detailed comparison of different radii and
latent spaces. The results are shown in Fig. 14, where $r$ refers to the ratio
of the whole geodesic described in Eq. (11) starting from image A to image B,
_e.g_., when $r=0.5$, the interpolation is exactly the hyperbolic mean of
these two images. We observe that in $\mathcal{W}^{+}$-space, both high-level
attributes and low-level attributes changed together while in hyperbolic
space, we can achieve more detailed editing on low-level attributes while
keeping high-level attributes unchanged, while the radius can control the
degree of change more precisely. When the hyperbolic radius is large, the
category of the given image remains the same before reaching the middle point
of the curve. This property allows us to generate diverse images of the given
image without changing its category-relevant attributes. As the hyperbolic
radius becomes smaller, the higher-level attributes gradually change in the
early stage of the interpolation. The interpolation visualization on geodesic
shows that the image gradually changes from fine-grained to abstract to fine-
grained. These results also explain our method of adding details by rescaling
after taking the geodesic which will lead to a relatively abstract average.

##  Appendix F Images Generated with Different Radii

In this section, we give more examples of images generated by HAE with
different radii in the Poincaré disk. As Fig. 15 shows, the high-level
attributes, a.k.a. category-relevant attributes do not change when the radius
is large which allows us to generate diverse images without changing the
category. However, the images generated by HAE become more abstract and
semantically diverse when the hyperbolic radius $r_{\mathbb{D}}$ becomes
smaller. The images gradually lose fine-grained details and change higher-
level attributes as they move closer to the center of the Poincaré disk. For
the few-shot image generation task, large radii work well since we want to
change the category-irrelevant attributes of a given image. Nevertheless, our
method HAE is not only capable of few-shot image generation but has great
potential for other downstream tasks. For instance, HAE is able to generate a
bunch of images of felines given an image of a cat. This can be done by
rescaling the latent code to a relatively small radius in the hyperbolic
space. Then we can add random perturbation to get the average code of multiple
categories of felines. Finally, diverse images with fine-grained details of
felines can be generated by moving those average codes to their children with
larger radii.

##  Appendix G Comparison with State-of-the-art Few-shot Image Generation
Method

We also provide a comparison of images generated by the state-of-the-art
method, _i.e_. AGE [12] and our methods on three datasets. As Fig. 16 shows,
our method is able to generate images with more semantic diversity. For
instance, for dogs, HAE generates images with different light conditions and
angles compared with images generated by AGE. Furthermore, for the woman in
the third row from the bottom, HAE can change the image from a colored photo
to a monochrome photo without changing her identity. However, our method also
slightly changes some attributes compared with the original images, _e.g_.,
the hair color of dogs and the petal color of flowers. This is because the
color varies within the category in these datasets which can indicate color is
a category-irrelevant attribute for those categories. Therefore, our method
does not change the category-relevant attributes but has more semantic
diversity. Besides, images generated by HAE look more natural compared with
images generated by AGE. Most importantly, AGE requires datasets with labels
to learn the feature code book. However, the hierarchical representation in
HAE can be learned using unsupervised or self-supervised learning if we have
enough computing resources. Therefore, our work has great potential and can be
applied to many other downstream tasks in future work.

##  Appendix H Additional Examples Generated by HAE

We provide more samples generated by HAE in Fig. 17, Fig. 18, and Fig. 19. The
radius we choose is 6.2126 and the length of perturbation is 8.

Figure 12: UMAP Visualization for Flowers dataset. Left: Hyperbolic space.
Right: Euclidean space ($\mathcal{W}^{+}$-space). Figure 13: UMAP
Visualization for VGGFaces dataset. Left: Hyperbolic space. Right: Euclidean
space ($\mathcal{W}^{+}$-space). In each visualization, no images are from the
same category. Figure 14: Interpolation along the geodesic and different radii
in hyperbolic space and along the straight line in $\mathcal{W}^{+}$-space on
Animal Faces, Flowers, and VGGFaces. Figure 15: One-shot image generation by
HAE on different radii on Animal Faces. Figure 16: Comparison between images
generated by AGE and HAE on Animal Faces, Flowers, and VGGFaces. Figure 17:
One-shot image generation by HAE on Animal Faces. Figure 18: One-shot image
generation by HAE on Flowers. Figure 19: One-shot image generation by HAE on
VGGFaces.