# GAS-NeXt: Few-Shot Cross-Lingual Font Generator

Haoyang He∗
Electrical and Computer Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
<EMAIL_ADDRESS>
&Xin Jin∗
Electrical and Computer Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
<EMAIL_ADDRESS>
&Angela Chen∗
Electrical and Computer Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
<EMAIL_ADDRESS>

###### Abstract

Generating new fonts is a time-consuming and labor-intensive task, especially
in a language with a huge amount of characters like Chinese. Various deep
learning models have demonstrated the ability to efficiently generate new
fonts with a few reference characters of that style, but few models support
cross-lingual font generation. This paper presents GAS-NeXt, a novel few-shot
cross-lingual font generator based on AGIS-Net and Font Translator GAN, and
improve the performance metrics such as Fréchet Inception Distance (FID),
Structural Similarity Index Measure(SSIM), and Pixel-level Accuracy (pix-acc).
Our approaches include replacing the original encoder and decoder with the
idea of layer attention and context-aware attention from Font Translator GAN,
while utilizing the shape, texture, and local discriminators of AGIS-Net. In
our experiment on English-to-Chinese font translation, we observed better
results in fonts with distinct local features than conventional Chinese fonts
compared to results obtained from Font Translator GAN. We also validate our
method on multiple languages and datasets.

**footnotetext: Authors contributed equally to this work††footnotetext: Our
code can be found at https://github.com/cmu-11785-F22-55/GAS-NeXt

##  1 Introduction

Generating new fonts is a time-consuming and labor-intensive task, especially
in a language with a huge amount of characters like Chinese. Various deep
learning models have demonstrated the ability to efficiently generate new
fonts with a few reference characters of that style, but few models support
cross-lingual font generation. This paper aims to develop a few-shot cross-
lingual font generator based on AGIS-Net and Font Translator GAN and improve
the performance metrics mentioned in Section 3. Our approaches include
redesigning the encoder and adding a local discriminator. We will validate our
method on multiple languages and datasets mentioned in Section 4.

##  2 Related Work

###  2.1 Generative Adversarial Network

Generative Adversarial Networks is a framework for estimating generative
models via the adversarial net, first proposed by Goodfellow et al. in 2014
[7], featuring two neural networks (generator and discriminator) competing
against each other for a zero-sum game to achieve an optimized solution. Since
its introduction, there are several focuses that aim to improve the
performance of the generative models. Mirza et al. introduced conditional GAN
that allows the addition of conditions on both the generator and discriminator
in GANs [20]. Isola et al. proposed $Pix2Pix$, an image-to-image translation
network based on GANs [12]. Zhu et al. refined the image-to-image translation
problem by introducing a multimodal variant BicycleGAN and modeling a
distribution of possible outputs in conditional GAN setting [34]. To solve the
problem that paired data are hard to obtain, Lee et al. proposed a method for
using unpaired data with disentangled representations [14], and Zhu et al.
proposed Cycle-consistent GANs to use learn a mapping such that the
distribution of translated images is indistinguishable from the target image
distribution using an adversarial loss [33].

As the application of image generation is usually in situations where abundant
training data is not available, more recent approaches have been focused on
few-shot learning-based image generation tasks. Clouâtre and Demers proposed
FIGR that mediates the lack of training data with a GAN meta-trained by
Reptile [4]. Liu et al. proposed FUNIT, an unsupervised few-shot image-to-
image translation framework [18].

###  2.2 Font generation

Font generation is a well-studied field. Yang et al. introduced a Generative
Adversarial Network (GAN)-based approach in text style transfer [29]. Hayashi
et al. proposed a GAN-based method to generate new fonts from seen fonts while
maintaining consistency among the generated fonts and diversity from seen
fonts [8]. More recently, studies in font generation have been focused on non-
alphabetic languages such as Chinese, as the number of classes of characters
is huge compared to alphabetic languages, thus a font generator provides
feasibility in applications. Tian proposed a conditional adversarial network
for Chinese font generation with one-hot encoding for font styles, but its
application is limited to seen font styles [24]. Zhang et al. proposed a
method of separating style and content in font generation [32]. Gao et al.
proposed a GAN that encodes content and a few samples of the style and decodes
to the new style based on shape and texture and used a local discriminator in
addition to the shape and texture discriminator [30]. One of the latest
approaches is Font Translator GAN (Font Translator GAN) [15]. This method
features cross-lingual font generation capability using layer attention and
context-aware attention in its encoder and decoder. It incorporates the shape
discriminator and texture discriminator but does not have a discriminator
focusing on differentiating local features of the generated character.

One common approach to font generation is decomposing characters into smaller
components and studying their localized features. This approach often results
in better performance on characters with complex structures, such as Chinese
characters which can be decomposed into approximately 500 radicals or 40
strokes. DM-Font [3] introduced the component-wise extractor and stored
component features in dual memory. However, the model was only tested on
simple language (Korean and Thai) with fixed component size and failed on
language with complex glyphs such as Chinese characters. RD-GAN [11] and CC-
Font [22] both applied radical decomposition and succeeded in predicting
characters of unseen components, but the former failed in the cross-lingual
system. Calli-GAN [28] encoded the component sequence using a recurrent neural
network and does not require component labels, but it failed on cross-lingual
prediction or unseen styles and components. To generalize the model to a
cross-lingual system, MX-Font [23] applied component-conditioned methods and
interpreted the component assignment problem as a graph-matching problem. More
recently, CG-GAN [13] introduced the component-aware module with a simple
generator that succeeded in the cross-lingual generation, and it can be
further extended to scene text editing.

###  2.3 Style Transfer

Style transfer is the process of adapting the style of an image to another
source image to produce a new image with the adapted style while preserving
the content of the source image. Gatys et al. first introduced a convolutional
neural network-based method of image artistic style transfer [5]. Huang and
Belongie proposed a method that allows arbitrary style transfer in real-time
by adaptive instance normalization [10]. Li et al. added a pair of feature
transforms, whitening, and coloring embedded in the image reconstruction
network to improve the capability of generalizing to unseen styles and improve
generation quality [16]. Tuygen et al. introduced a method to generate
multiple images with a single image-style pair by utilizing deep correlational
multimodel style transfer [25]. The methods in image style transfer have also
been explored in the style transfer applications in other forms of signals.
Huang et al. proposed a method for video style transfer based on the feed-
forward network for image style transfer [9]. Verma and Smith adapted the work
by Gatys et al. for audio style transfer [26].

###  2.4 Similarity score

Many of the generative models for images tend to produce blurry images with
low bound approximation or restriction on the distribution [6]. A popular loss
like mean square error is not ideal for image quality assessment [27].
Structural similarity Index (SSIM) is a perceptual measure shown to be perfect
for different generative models and autoencoders. Using SSIM as the loss
metric can improve the perceptual quality of the generated images [27].

##  3 Baseline model

We adopt the architecture of AGIS-Net[30] to be our baseline model as shown in
Figure 1 and evaluate its performance using Fréchet Inception Distance (FID),
Structural Similarity Index Measure(SSIM), and Pixel-level Accuracy (pix-acc).

###  3.1 Problem Formulation

Our goal is to generate a stylized glyph image with the provided content and a
few sample images of the specified style. We need two sets of input: the
content reference image $x_{C}$ and the style reference input $X_{s}$. The
content reference image $x_{C}$ is a binary image in standard font with
minimal style information. Given that the stylized examples are usually
limited, we use $n$ to denote the size of the few-shot style reference set
$R_{s}$, and we randomly sample $m\leq n$ images from the reference set as
style reference input $X_{s}\subseteq R_{s}$.

###  3.2 Baseline Selection

Before AGIS-Net, previous approaches either focuses on shape synthesis or were
designed for texture transfer. Campbell and Kautz [2] built a font manifold
for Latin glyphs’ outlines. However, it can fail when the complexity of glyphs
greatly increases. Zeng et al. [31] attempted to utilize strokes from Chinese
glyphs and learn to produce corresponding strokes for others but in the same
style. However, it is not suitable for other scenarios like synthesizing Latin
glyphs. Therefore, the most relevant work for this task is MC-GAN [21] which
is the first architecture that solved learning styles of both texture and
shape. Moreover, it is capable to do glyph shape synthesis and texture
transfer as well. However, it is limited to 26 Latin capital letters as input,
and it performs badly in generating other writing systems like Chinese
characters. Thus, it’s definitely not the best choice as the baseline model
for our task. AGIS-Net [30] can effectively solve the above-mentioned
problems. Compared to MC-GAN [21], it has a wider application scenario. In
other words, it can not only handle Latin glyphs but also Chinese and any
other writing systems. Other competitors like BicycleGAN [33] which is a
leading general-purpose image-to-image translation model and TETGAN [29] which
is the state-of-the-art artistic font style transfer model, perform slightly
worse for cross-lingual tasks according to the performance result by Gao et
al. [30]. Overall, we believe that AGIS-Net is a competitive and robust model
for our task.

###  3.3 Model Description

AGIS-Net consists of a generator and three discriminators, as shown in Fig. 1.
The generator has two parallel working branches of encoders and decoders,
where the encoders are responsible for extracting the content and style
features independently, and the texture decoder generates the synthesized
image collaboratively with the shape decoder. The output synthesized image is
then fed into the shape and texture discriminators as negative samples during
training, and the real glyph images are cut patched and blurred to introduce
more fake samples for the local discriminator. The local discriminator helps
generate more realistic images with reduced noise and alleviates the problem
of unbalanced positive and negative glyph images in the few-shot setting.

Generator: Fig. 5 shows the architecture of the generator. A binary image with
minimal style information is inputted to the content encoder, and a set of
stylized images are randomly sampled from the few-shot reference set as the
input to the style encoder. The content and style features are extracted from
several down-sampling convolution layers and skipped connected to the input of
corresponding decoder layers as shown in 6. The shape decoder concatenates the
output from the previous layer and the extracted content and style features
from the corresponding encoder layer to generate the shape image $y_{gray}$.
The texture decoder further combines the texture information with the shape
information through concatenation, and the synthesized image $y$ is produced
by feeding the concatenated shape image $y_{gray}$ and texture information to
a convolution layer.

Objective Function: The objective function consists of four different loss
functions: the adversarial loss, the $L_{1}$ loss, the contextual loss, and
the local texture refinement loss

$L(G,D_{sha},D_{tex},D_{local})=L_{adv}+L_{1}+L_{CX}+L_{local},$

where $G$ is our generator, $D_{sha}$, $D_{tex}$, and $D_{local}$ represent
the shape, texture, and local discriminator respectively. We train our model
by optimizing

$\min_{G}\max_{D_{sha},D_{tex},D_{local}}L(G,D_{sha},D_{tex},D_{local}).$

  * •

Adversarial Loss: We train the generator $G$ and two discriminators $D_{sha}$
and $D_{tex}$ based on the following adversarial loss. $y_{gray}$ is the gray-
scale image generated from the shape decoder, $y$ is the synthesized image,
$t_{s}^{r}$ is the ground truth image, and $t_{gray}^{s}$ is the gray-scaled
ground truth image used to train the shape discriminator.
$\lambda_{adv\\_sha}$ and $\lambda_{adv\\_tex}$ are the weights set to 1.0
according to [30].

$\displaystyle L(D_{sha})$
$\displaystyle=\mathbb{E}_{t_{gray}^{r}}[\log(D_{sha}(t_{gray}^{r}))]+\mathbb{E}_{y_{gray}}[\log(1-D_{sha}(y_{gray}))]$
$\displaystyle L(D_{tex})$
$\displaystyle=\mathbb{E}_{t_{s}^{r}}[\log(D_{tex}(t_{s}^{r}))]+\mathbb{E}_{y}[\log(1-D_{tex}(y))]$
$\displaystyle L_{adv}$
$\displaystyle=\lambda_{adv\\_sha}\mathbb{E}_{y_{gray}}[\log(1-D_{sha}(y_{gray}))]+\lambda_{adv\\_tex}\mathbb{E}_{y}[\log(1-D_{tex}(y))].$

  * •

$L_{1}$ Loss: $L_{1}$ Loss helps stabilize the training. We simply combine the
$L_{1}$ loss of the gray-scale image and the texture image with weights
$\lambda_{L_{1gray}}=50.0$ and $\lambda_{L_{1tex}}=100.0$. These weights are
set to zero for the unseen data.

$L_{1}=\lambda_{L_{1gray}}\mathbb{E}_{y_{gray},\hat{y}_{gray}}\norm{y_{gray}-\hat{y}_{gray}}_{1}+\lambda_{L_{1tex}}\mathbb{E}_{y,\hat{y}}\norm{y-\hat{y}}_{1},$

where $\hat{y}$ and $\hat{y}_{gray}$ are ground truth images of $y$ and the
gray-scale version of the ground truth, and we will reuse them in the
following loss functions.

  * •

Contextual Loss: Contextual loss measures the similarity between feature map
collections of the images, and it focuses on high-level style features. The
similarity $CX$ between $x_{i}$, the i-th feature map of image $X$, and
$y_{j}$, the j-th feature map of image $Y$ is defined as the normalized
exponential of the shifted normalized cosine distances

$CX_{ij}=\frac{w_{ij}}{\sum_{k}w_{ik}},\hat{d}_{ij}=\frac{d_{ij}}{\min_{k}d_{ik}+\epsilon},w_{ij}=\exp{\frac{1-\hat{d_{ij}}}{h}},$
$CX(X,Y)=\frac{1}{N}\sum_{j}\max_{i}CX_{ij},$

where $d_{ij}$ is the cosine distance between $x_{i}$ and $y_{j}$, $h$ and
$\epsilon$ are hyper-parameters set to 0.5 and $1e-5$ according to [19].

We use pre-trained VGG19 to extract features. $\phi^{l}$ denotes the extracted
features from the $l$-th layer of VGG19, and $L$ is the total number of
layers. In our experiment, we use $\lambda_{CXgray}=15.0$ and
$\lambda_{CXtex}=25.0$ for the weights. The contextual loss is formulated as

$\displaystyle L_{CX}$
$\displaystyle=\lambda_{CX_{gray}}\Big{(}-\frac{1}{L}\sum_{l}\log(CX(\phi^{l}(y_{gray}),\phi^{l}(\hat{y}_{gray})))\Big{)}$
$\displaystyle+\lambda_{CX_{tex}}\Big{(}-\frac{1}{L}\sum_{l}\log(CX(\phi^{l}(y),\phi^{l}(\hat{y})))\Big{)}.$

  * •

Local Textual Refinement Loss: The local discriminator $D_{local}$ takes the
cut patched real images $p_{real}$ as positive samples, the blurred real
images $p_{blur}$, and our synthesized patches $p_{y}$ as negative samples. We
set the weights $\lambda_{local}=1.0$ based on [30].

$\displaystyle L(D_{local})$
$\displaystyle=\mathbb{E}_{p_{real}}[\log(D_{local}(p_{real}))])+\mathbb{E}_{p_{blur}}[\log(1-D_{local}(p_{blur}))]+$
$\displaystyle+\mathbb{E}_{p_{y}}[log(1-D_{local}(p_{y}))],$ $\displaystyle
L_{local}$
$\displaystyle=\lambda_{local}\mathbb{E}_{p_{y}}[\log(1-D_{local}(p_{y}))].$

Figure 1: AGIS-Net Architecture. The upper part is the generator consisting of
two parallel-working encoders and decoders, and the lower three are
discriminators.

###  3.4 Evaluation Metric

To evaluate the model performance, we adopt three commonly-used metrics in
image generation tasks: Fréchet Inception Distance (FID), Structural
Similarity Index (SSIM), and Pixel-level Accuracy (pix-acc).

Fréchet Inception Distance (FID)

FID compares the probability distributions of feature vectors. FID between the
fake image distribution $x$ and the ground truth image distribution $y$ is
computed as

$FID=\norm{\mu_{x}-\mu_{y}}^{2}+Tr(C_{x}+C_{y}-2\times\sqrt{C_{x}\times
C_{y}}),$

where $\mu_{x}$ and $C_{x}$ are the mean and covariance matrix of the feature
vectors distribution $x$. We use the module "pytorch-fid" to generate our FID
scores.

Structural Similarity Index (SSIM)

SSIM measures the structural similarity between the synthesized image $x$ and
the ground truth image $y$ by

$\displaystyle SSIM$
$\displaystyle=\frac{(2\mu_{x}\mu_{y}+c_{1})(2\sigma_{xy}+c_{2})}{(\mu_{x}^{2}+\mu_{y}^{2}+c_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2})},$

where $\mu_{x}$ and $\mu_{y}$ are the pixel sample mean of $x$ and $y$;
$\sigma_{x}$, $\sigma_{y}$, and $\sigma_{xy}$ are the variance of $x$, $y$,
and the covariance of $x$ and $y$; $c_{1}$ and $c_{2}$ are constants to avoid
instability when the denominator is small. In our evaluation, we adopt
"skimage.metrics" package to evaluate the SSIM score.

Pixel-level Accuracy (pix-acc)

$PA=\frac{\sum_{j=1}^{h}n_{jj}}{\sum_{j=1}^{k}t_{j}},$

where $n_{jj}$ denotes the number of correctly classified pixels in class $j$,
and $t_{j}$ is the number of pixels labelled as class $j$. We use the gray-
scale image for the evaluation, and thus the class is binary. Pixel-level
accuracy is then averaged to evaluate the overall quality of the synthesized
image.

##  4 Dataset

Gao et al. proposed a large-scale dataset using professional-designed fonts in
different shapes and texture styles[30]. We will use it to validate if our
method is effective and extendable at the first stage. CASIA also provides
standard datasets for handwritten Chinese character recognition[17]. The data
has been annotated at the character level, so it’s preferable to be tested on
for evaluating performance metrics. For English glyph images, we use the
dataset proposed by Azadi et al. [1]. Moreover, regarding plain context
inputs, we will take up the average font style as common as possible for
Chinese characters and Code New Roman as MC-GAN[21] for English letters. For
our extended model, we use the dataset proposed by Li et al. [15]. This
dataset is mainly for cross-lingual font style transfer. All the style inputs
are gray-scale Latin fonts, and all the context inputs are Chinese characters.
They are used to generate styled Chinese characters.

##  5 Baseline Implementation Completeness

Table 1:  Evaluation Metrics of AGIS-Net and our baseline model Evaluation
Metrics
---
Model |  FID |  SSIM |  pix-acc
AGIS-Net |  74.567 |  0.7389 |  0.6241
Baseline |  91.4947 |  0.6722 |  0.3917

From the $L_{1}$ losses computed during fine-tuning shown in Figure 10, we
observed that our baseline model got almost the same performance as stated in
the original paper[30]. We also compare the performance of the baseline model
with AGIS-Net (setting $m$=7, $n$=2) using the evaluation metrics used in
their paper which are FID, SSIM, and pixel-level accuracy. Higher values of
SSIM and pix-acc are better, whereas for FID, the lower the better. We didn’t
measure the inception score of our results since it can not directly reflect
the quality of our synthesized glyph images. As shown in Table 1, we achieved
roughly the same performance in terms of SSIM. Regarding FID, the results are
close, but for pix-acc we still need to improve our baseline model to match
the ideal performance. An example of a set of generated fonts is shown in
Figure 2.

Figure 2: Example of one set of generated font

##  6 Extended Methods

AGIS-Net considers font generation as an image-to-image translation task. The
difference is that it takes two conditions, which are style and content
images. However, the original paper only states that it is capable to generate
fonts using styles from the same language. Moreover, it heavily relies on the
fine-tuning process. To extend it, we leveraged the idea of layer attention
and context-aware attention as shown in Figure 12 and Figure 11 modules in the
generator of Font Translator GAN. The context-aware attention network captures
the local style feature, while the layer attention network captures the global
style feature. They work together to make our generator more flexible when
dealing with arbitrary font styles.

To be more detailed, in the context-aware attention network, we introduce a
self-attention module to make the new feature contain both the information in
their receptive fields and contextual information from other locations. We
followed the suggestions in the original paper to use 3 parallel context-aware
attentions.

After having local features and global features, we apply layer attention to
select which feature level the model should focus on by assigning scores to
them. After that, we take the mean and expand the encoded style features.

Figure 3: GAS-NeXt architecture

For discriminators, instead of using only context and style discriminators as
in Font Translator GAN, we keep using the local discriminator to retain a
better local structure performance. We also adapt the objective function and
adversarial loss of AGIS-Net. Overall, we show our architecture in Figure 3.

##  7 Results and Discussions

In our experiments with GAS-NeXt, we chose to use the dataset proposed by Li
et al. [15] to compare the cross-lingual font generation effects against the
state-of-the-art model Font Translator GAN. The metrics comparison is shown in
Table 2. The metrics of GAS-NeXt are very similar to that of Font Translator
GAN.

However, when examining the generation results, we can see that GAS-NeXt
achieves a similar level of visual results as Font Translator GAN for fonts
with similar local attributes as conventional fonts, while GAS-NeXt is able to
achieve a better visual result for fonts with distinct local attributes. As
shown in Figure 4, we can clearly observe a better feature captured by GAS-
NeXt in the circled regions. These regions have the commonality of being
sharp-edged in the target font but are more rounded in conventional Chinese
writings. This visual observation suggests that GAS-NeXt is able to capture
local attributes better than FTransGAN, which is a crucial factor in
generating a font with distinct styles.

This improvement in learning local distinct styles can be attributed to our
addition of the local discriminator inspired by AGIS-Net. By generating more
fake samples from a real sample’s local regions using patch cutting and
blurring, and using a third discriminator specifically learning the
distinctions in the local features, our model can capture these local
distinctions and generate results more similar to the ground truth.

Table 2:  Evaluation Metrics of Font Translator GAN and GAS-NeXt Content / Style | Accuracy | L1 | SSIM | FID
---|---|---|---|---
| FTransGAN
---
Unknown Content
| 97.28 (Content)
---
55.59 (Style)
| 0.12
---
0.12
| 0.50
---
0.49
| 50.44
---
321.60
| FTransGAN
---
Unknown Style
| 99.88
---
9.65
| 0.18
---
0.18
| 0.37
---
0.37
| 96.79
---
436.75
| GasNeXt
---
Unknown Content
| 96.52
---
55.01
| 0.1241
---
0.1241
| 0.4928
---
0.4928
| 52.012
---
324.33
| GasNeXt
---
Unknown Style
| 99.755
---
9.05
| 0.1865
---
0.1865
| 0.3459
---
0.3459
| 103.933
---
460.827
Figure 4: Visual results comparison between FTransGAN and GAS-NeXt

##  8 Conclusion and Future Work

In this paper, we proposed a novel few-shot cross-lingual font generator
network GAS-NeXt. Using the layer attention and context-aware attention
methods from Font Translator GAN, and the idea of a local discriminator from
AGIS-Net, we are able to achieve a similar metric with the state-of-the-art
model Font Translator GAN, while obtaining better visual results in fonts with
distinct local features. Although previous methods such as character
decomposition also attempt to address this problem, these methods are not able
to transfer the decomposition knowledge of characters from one language to
another. To our knowledge, there have not been literature discussions focusing
on cross-lingual font generation featuring local features. Our model lays a
groundwork for future studies in improving learning cross-lingual font
generation for fonts with distinct local features compared to conventional
writing.

In addition to further experimentation on the local discriminator, another
potential future work is more experimentation on colored font styles.
Currently, the colored font style generation with cross-lingual capability for
our model is non-ideal. We experimented on separating the three channels of
image inputs and experiment on each channel independently, then reconstructed
the three generated channels into a single image. However, this performed
poorly as the three channels generated results in different pixel locations,
causing incomplete overlaps.

## References

  * [1] Samaneh Azadi, Matthew Fisher, Vladimir Kim, Zhaowen Wang, Eli Shechtman, and Trevor Darrell.  Multi-content gan for few-shot font style transfer, 2017.
  * [2] Neill D. F. Campbell and Jan Kautz.  Learning a manifold of fonts.  ACM Trans. Graph., 33(4), jul 2014.
  * [3] Junbum Cha, Sanghyuk Chun, Gayoung Lee, Bado Lee, Seonghyeon Kim, and Hwalsuk Lee.  Few-shot compositional font generation with dual memory, 2020.
  * [4] Louis Clouâtre and Marc Demers.  Figr: Few-shot image generation with reptile, 2019.
  * [5] Leon A Gatys, Alexander S Ecker, and Matthias Bethge.  Image style transfer using convolutional neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2414–2423, 2016.
  * [6] Benyamin Ghojogh, Fakhri Karray, and Mark Crowley.  Theoretical insights into the use of structural similarity index in generative models and inferential autoencoders.  In Lecture Notes in Computer Science, pages 112–117. Springer International Publishing, 2020.
  * [7] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  * [8] Hideaki Hayashi, Kohtaro Abe, and Seiichi Uchida.  Glyphgan: Style-consistent font generation based on generative adversarial networks.  CoRR, abs/1905.12502, 2019.
  * [9] Haozhi Huang, Hao Wang, Wenhan Luo, Lin Ma, Wenhao Jiang, Xiaolong Zhu, Zhifeng Li, and Wei Liu.  Real-time neural style transfer for videos.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 783–791, 2017.
  * [10] Xun Huang and Serge Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In Proceedings of the IEEE international conference on computer vision, pages 1501–1510, 2017.
  * [11] Yaoxiong Huang, Mengchao He, Lianwen Jin, and Yongpan Wang.  Rd-gan: Few/zero-shot chinese character style transfer via radical decomposition and rendering.  In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision – ECCV 2020, pages 156–172, Cham, 2020. Springer International Publishing.
  * [12] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros.  Image-to-image translation with conditional adversarial networks, 2016\.
  * [13] Yuxin Kong, Canjie Luo, Weihong Ma, Qiyuan Zhu, Shenggao Zhu, Nicholas Yuan, and Lianwen Jin.  Look closer to supervise better: One-shot font generation via component-based discriminator, 2022.
  * [14] Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Kumar Singh, and Ming-Hsuan Yang.  Diverse image-to-image translation via disentangled representations.  In European Conference on Computer Vision, 2018.
  * [15] Chenhao Li, Yuta Taniguchi, Min Lu, and Shin’ichi Konomi.  Few-shot font style transfer between different languages.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 433–442, 2021.
  * [16] Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang.  Universal style transfer via feature transforms.  Advances in neural information processing systems, 30, 2017.
  * [17] Cheng-Lin Liu, Fei Yin, Da-Han Wang, and Qiu-Feng Wang.  Casia online and offline chinese handwriting databases.  In 2011 International Conference on Document Analysis and Recognition, pages 37–41, 2011.
  * [18] Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz.  Few-shot unsupervised image-to-image translation.  2019\.
  * [19] Roey Mechrez, Itamar Talmi, and Lihi Zelnik-Manor.  The contextual loss for image transformation with non-aligned data.  CoRR, abs/1803.02077, 2018.
  * [20] Mehdi Mirza and Simon Osindero.  Conditional generative adversarial nets, 2014.
  * [21] Hyojin Park, YoungJoon Yoo, and Nojun Kwak.  Mc-gan: Multi-conditional generative adversarial network for image synthesis, 2018.
  * [22] Jangkyoung Park, Ammar Ul Hassan, and Jaeyoung Choi.  Ccfont: Component-based chinese font generation model using generative adversarial networks (gans).  Applied Sciences, 12(16), 2022.
  * [23] Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, and Hyunjung Shim.  Multiple heads are better than one: Few-shot font generation with multiple localized experts, 2021.
  * [24] Yuchen Tian.  zi2zi: Master chinese calligraphy with conditional adversarial networks, 2017.
  * [25] Nguyen Quang Tuyen, Son Truong Nguyen, Tae Jong Choi, and Vinh Quang Dinh.  Deep correlation multimodal neural style transfer.  IEEE Access, 9:141329–141338, 2021.
  * [26] Prateek Verma and Julius O Smith.  Neural style transfer for audio spectograms.  arXiv preprint arXiv:1801.01589, 2018.
  * [27] Zhou Wang and Alan C. Bovik.  Mean squared error: Love it or leave it? a new look at signal fidelity measures.  IEEE Signal Processing Magazine, 26(1):98–117, 2009.
  * [28] Shan-Jean Wu, Chih-Yuan Yang, and Jane Yung-jen Hsu.  Calligan: Style and structure-aware chinese calligraphy character generator, 2020.
  * [29] Shuai Yang, Jiaying Liu, Wenjing Wang, and Zongming Guo.  Tet-gan: Text effects transfer via stylization and destylization.  In AAAI Conference on Artificial Intelligence, 2019.
  * [30] Gao Yue, Guo Yuan, Lian Zhouhui, Tang Yingmin, and Xiao Jianguo.  Artistic glyph image synthesis via one-stage few-shot learning.  ACM Trans. Graph., 38(6), 2019.
  * [31] Jinshan Zeng, Qi Chen, Yunxin Liu, Mingwen Wang, and Yuan Yao.  Strokegan: Reducing mode collapse in chinese font generation via stroke encoding.  2020\.
  * [32] Yexun Zhang, Ya Zhang, and Wenbin Cai.  Separating style and content for generalized style transfer.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8447–8455, 2018.
  * [33] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks, 2017.
  * [34] Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, and Eli Shechtman.  Toward multimodal image-to-image translation, 2017.

##  Appendix A AGIS-Net

Figure 5: AGIS-Net Generator Figure 6: AGIS-Net Generator Submodule

##  Appendix B Metrics for Baseline

Figure 7: MSE of test dataset Figure 8: pixel-level accuracy of test dataset
Figure 9: SSIM of test dataset

##  Appendix C L1 Loss for baseline model

Figure 10: L1 Loss in Fine-Tuning Stage

##  Appendix D Context-aware Attention

Figure 11: Context-aware attention

##  Appendix E Layer Attention

Figure 12: Layer attention